Abstract
Cohn and Umans proposed a framework for developing fast matrix multiplication algorithms based on the embedding computation in certain groups algebras [9]. In subsequent work with Kleinberg and Szegedy, they connected this to the search for combinatorial objects called strong uniquely solvable puzzles (strong USPs) [8]. We begin a systematic computer-aided search for these objects. We develop and implement algorithms based on reductions to 
and 
to verify that puzzles are strong USPs and to search for large strong USPs. We produce tight bounds on the maximum size of a strong USP for width 
, and construct puzzles of small width that are larger than previous work. Although our work only deals with puzzles of small-constant width and does not produce a new, faster matrix multiplication algorithm, we provide evidence that there exist families of strong USPs that imply matrix multiplication algorithms that are more efficient than those currently known.
Keywords: Matrix multiplication, Strong uniquely solvable puzzle, Arithmetic complexity, Integer programming, Satisfiability, Reduction, Application
Introduction
An optimal algorithm for matrix multiplication remains elusive despite substantial effort. We focus on the square variant of the matrix multiplication problem, i.e., given two n-by-n matrices A and B over a field 
, the goal is to compute the matrix product 
. The outstanding open question is: How many field operations are required to compute C? The long thought-optimal naïve algorithm based on the definition of matrix product is 
time. The groundbreaking work of Strassen showed that it can be done in time 
[24] using a divide-and-conquer approach. A long sequence of work concluding with Coppersmith and Winograd’s algorithm (CW) reduced the running time to 
[10, 21, 22, 25]. Recent computer-aided refinements of CW by others reduced the exponent to 
[13, 18, 26].
Approach. Cohn and Umans [9] introduced a framework for developing faster algorithms for matrix multiplication by reducing this to a search for groups with subsets that satisfy an algebraic property called the triple-product property that allows matrix multiplication to be embedded in the group algebra. Their approach takes inspiration from the 
algorithm for multiplying degree-n univariate polynomials by embedding in the group algebra of the fast Fourier transform, c.f., e.g., [11, Chapter 30]. Subsequent work [8] elaborated on this idea and developed the notion of combinatorial objects called strong uniquely solvable puzzles (strong USPs). These objects imply a group algebra embedding for matrix multiplication, and hence give a matrix multiplication algorithm as well.
A width-k puzzle P is a subset of 
, and the cardinality of P is the puzzle’s size. Each element of P is called a row of P, and each row consists of three subrows that are elements of 
, 
, 
respectively. Informally, a puzzle P is a uniquely solvable puzzle (USP) if there is no way to permute the subrows of P to form a distinct puzzle 
without cells with numbers overlapping. Figure 1 demonstrates a puzzle that is not a USP. A uniquely solvable puzzle is strong if a tighter condition for non-overlapping holds (see Definition 2). For a fixed width k, the larger the size of a strong USP, the faster matrix multiplication algorithm it gives [8]. In fact Cohn et al. show that there exist an infinite family of strong USPs that achieves 
.
Fig. 1.
The leftmost diagram is a width-4 size-5 puzzle P. The middle three diagrams are the three sets of subrows of P. The rightmost diagram is the puzzle 
resulting from reordering the subrows of P as indicated by the arrows and then recombining them. Since P can be rearranged as 
without overlap, P is not uniquely solvable.
We follow Cohn et al.’s program by: (i) developing verification algorithms to determine whether a puzzle is a strong USP, (ii) developing search algorithms to find large strong USPs, and (iii) implementing and running practical implementations of these algorithms. The most successful of the verification algorithms function by reducing the problem through 3D matching to 
and 
which are then solved with existing tools. The algorithms we develop are not efficient—they run in worst-case exponential time in the natural parameters. However, the goal is to find a sufficiently large strong USP that would provide a faster matrix multiplication algorithm, and the resulting algorithm’s running time is independent of the running time of our algorithms. The inefficiency of our algorithms limit the search space that we can feasibly examine.
Results. Our experimental results give new bounds on the size of the largest strong USP for small-width puzzles. For small-constant width, 
, we beat the largest sizes of [8, Proposition 3.8]. Our lower bounds on maximum size are witnessed by strong USPs we found via search. For 
we give tight upper bounds determined by exhaustively searching all puzzles up to isomorphism. Although our current experimental results do not beat [8] for unbounded k, they give evidence that there may exist families of strong USPs that give matrix multiplication algorithms that are more efficient than those currently known.
Related Work. There are a number of negative results known. Naïvely, the dimensions of the output matrix C implies that the problem requires at least 
time. Slightly better lower bounds are known in general and also for specialized models of computation, c.f., e.g., [16, 23]. There are also lower bounds known for a variety of algorithmic approaches to matrix multiplication. Ambainis et al. showed that the laser method cannot alone achieve an algorithm with 
[4]. A recent breakthrough on arithmetic progressions in cap sets [12] combined with a conditional result on the Erdös-Szemeredi sunflower conjecture [3] imply that Cohn et al.’s strong USP approach cannot achieve 
for some 
[7]. Subsequent work has generalized this barrier [1, 2] to a larger class of algorithmic techniques. Despite this, we are unaware of a concrete lower bound on 
implied by these negative results. There remains a substantial gap in our understanding between what has been achieved by the positive refinements of LeGall, Williams, and Stothers, and the impossibility of showing 
using the strong USP approach.
Organization. Section 2 begins with the formal definition of a strong USP. Sections 3 and 4, respectively, discuss our algorithms and heuristics for verifying that and searching for a puzzle that is a strong USP. Section 5 discusses our experimental results.
Preliminaries
For an integer k, we use 
to denote the set 
. For a set Q, 
denotes the symmetric group on the elements of Q, i.e., the group of permutations acting on Q. Cohn et al. introduced the idea of a puzzle [8].
Definition 1
(Puzzle). For 
, an (s, k)-puzzle is a subset 
with 
. We call s the size of P, and k the width of P.
We say that an (s, k)-puzzle has s rows and k columns. The columns of a puzzle are inherently ordered and indexed by [k]. The rows of a puzzle have no inherent ordering, however, it is often convenient to assume that they are ordered and indexed by the set of natural numbers [s].
Cohn et al. establish a particular combinatorial property of puzzles that allows one to derive group algebras that matrix multiplication can be efficiently embedded into. Such puzzles are called strong uniquely solvable puzzles.
Definition 2
(Strong USP). An (s, k)-puzzle P is strong uniquely solvable if for all 
: Either (i) 
, or (ii) there exists 
and 
such that exactly two of the following hold: 
, 
, 
.
Note that strong uniquely solvability is invariant to the (re)ordering of the rows or columns of a puzzle. We use this fact implicitly.
Cohn et al. show the following connection between the existence of strong USPs and upper bounds on the exponent of matrix multiplication 
.
Lemma 1
([8, Corollary 3.6]). Let 
, if there is a strong uniquely solvable (s, k)-puzzle, there is an algorithm for multiplying n-by-n matrices in time 
where
 |
This result motivates the search for large strong USPs that would result in faster algorithms for matrix multiplication. In the same article, the authors also demonstrate the existence of an infinite family of strong uniquely solvable puzzles, for width k divisible by three, that achieves a non-trivial bound on 
.
Lemma 2
([8, Proposition 3.8]). There is an infinite family of strong uniquely solvable puzzles that achieves 
.
Verifying Strong USPs
The core focus of this article is the problem of verifying strong USPs, i.e., given an (s, k)-puzzle P, output YES if P is a strong USP, and NO otherwise. In this section we discuss the design of algorithms to solve this computational problem as a function of the natural parameters s and k. Along the way we also discuss some aspects of our practical implementation that informed or constrained our designs. All the exact algorithms we develop in this section have exponential running time. However, asymptotic worst-case running time is not the metric we are truly interested in. Rather we are interested in the practical performance of our algorithms and their capability for locating new large strong USPs. The algorithm that we ultimately develop is a hybrid of a number of simpler algorithms and heuristics.
Brute Force
The obvious algorithm for verification comes directly from the definition of a strong USP. Informally, we consider all ways of permuting the ones and twos pieces relative to the zeroes pieces and check whether the non-overlapping condition of Definition 2 is met. A formal description of the algorithm is found in Algorithm 1.
The ones in Line 4 of Algorithm 1 denote the identity in 
, and 
is the Kronecker delta function which is one if 
and zero otherwise. Observe that Algorithm 1 does not refer to the 
of Definition 2. This is because the strong USP property is invariant to permutations of the rows and so 
can be thought of as an arbitrary phase. Hence, we fix 
to simplify the algorithm. Seeing that 
, we conclude that the algorithm runs in time 
where the last factor accounts for the operations on permutations of s elements. The dominant term in the running time is the contribution from iterating over pairs of permutations. Finally, notice that if P is a strong USP, then the algorithm runs in time 
, and that if P is not a strong USP the algorithm terminates early. The algorithm’s poor performance made it unusable in our implementation, however, its simplicity and direct connection to the definition made its implementation a valuable sanity check against later more elaborate algorithms (and it served as effective onboarding to the undergraduate students collaborating on this project).
Although Algorithm 1 performs poorly, examining the structure of a seemingly trivial optimization leads to substantially more effective algorithms. Consider the following function on triples of rows 
: 
We can replace the innermost loop in Lines 7 & 8 of Algorithm 1 with the statement 
. Observe that f neither depends on P, r, nor the permutations, and that Algorithm 1 no longer depends directly on k. To slightly speed up Algorithm 1 we can precompute and cache f before the algorithm starts and then look up values as the algorithm runs. We precompute f specialized to the rows in the puzzle P, and call it 
.
Strong USP Verification to 3D Matching
It turns out to be more useful to work with 
than with P. It is convenient to think of 
as a function 
that is the complement of the characteristic function of the relations of a tripartite hypergraph 
where the vertex set is the disjoint union of three copies of P and 
indicates the edges that are not present in 
.
Let 
be a tripartite 3-hypergraph. We say H has a 3D matching (
) iff there exists a subset 
with 
and for all distinct edges 
, 
and 
are vertex disjoint, i.e., 
. Determining whether a hypergraph has a 3D matching is a well-known 
-complete problem (c.f., e.g., [14]). We say that a 3D matching is non-trivial if it is not the set 
. Figure 2 demonstrates a 3-hypergraph with a non-trivial 3D matching.
Fig. 2.
An example hypergraph G with edges 
. The highlighted edges are a non-trivial 3D matching 
of G.
The existence of non-trivial 3D matchings in 
is directly tied to whether P is a strong USP.
Lemma 3
A puzzle P is a strong USP iff 
has no non-trivial 3D matching.
Proof
We first argue the reverse. Suppose that 
has a non-trivial 3D matching M. We show that P is not a strong USP by using M to construct 
that witness this. Let 
be the identity permutation. For each 
, define 
where 
. Note that q is well defined and unique because M is 3D matching and so has vertex disjoint edges. Similarly define 
where 
. Observe that by construction
 |
Since M is a matching of 
, 
. Because M is a non-trivial matching at least one edge in 
has either 
, 
, or 
. This implies, respectively, that as constructed 
, 
, or 
. In each case we have determined that 
, 
, and 
are not all identical. Thus we determined permutations such that for all 
, 
. This violates Condition (ii) of Definition 2, hence P is not a strong USP.
The forward direction is symmetric. Suppose that P is not a strong USP. We show that 
has a 3D matching. For P not to be a strong USP there must exist 
not all identical such that Condition (ii) of Definition 2 fails. Define 
and 
. Since Condition (ii) fails, we have that 
for all 
. This means that for all 
, 
and hence 
. Since 
is a permutation, 
. Observe that M is non-trivial because not all the permutations are identical and there must be some 
with e(r) having non-identical coordinates. Thus M is a non-trivial 3D matching. 
Note that although 3D matching is an 
-complete problem, Lemma 3 does not immediately imply that verification of strong USPs is 
-complete because 
is not an arbitrary hypergraph. As a consequence of Definition 2, verification is in 
. It remains open whether verification is 
-complete. Lemma 3 implies that to verify P is a strong USP it suffices to determine whether 
has a 3D matching. In the subsequent sections we examine algorithms for the later problem. We can, in retrospect, view Algorithm 1 as an algorithm for solving 3D matching.
The realization that verification of strong USPs is a specialization of 3D matching leads to a dynamic programming algorithm for verification that runs in linear-exponential time 
. Applying more advanced techniques like those of Björklund et al. can achieve a better asymptotic time of 
[6]. For brevity, we defer the details of our algorithm to the long version of this article.
3D Matching to Satisfiability
By Lemma 3, one can determine whether a puzzle P is a strong USP by constructing the graph 
and deciding whether it has a non-trivial 3D matching. Here we reduce our 3D matching problem to the satisfiability (SAT) problem on conjunctive normal form (CNF) formulas and then use a state-of-the-art SAT solver to resolve the reduced problem. To perform the reduction, we convert the graph 
into a CNF formula 
, a depth-2 formula that is the AND of ORs of Boolean literals. We construct 
so that 
is satisfiable iff 
has a non-trivial 3D matching.
Let 
be the 3D matching instance associated with the puzzle P. Our goal is to determine whether there is a non-trivial 3D matching 
. A naïve reduction would be to have variables 
indicating inclusion of each edge 
in the matching. This results in a formula 
with 
variables and size 
because including an edge 
excludes the 
edges 
with 
. To decrease the size of 
we instead use sets of variables to indicate which vertices in the second and third part of V are matched with each vertex in the first part. In particular we have Boolean variables 
and 
for all 
, and these variable map to assignments in the naïve scheme in the following way: 
.
We now write our CNF formula for 3D matching. First, we have clauses that prevents non-edges from being in the matching:
 |
1 |
Second, we add clauses require that every vertex in 
is matched with some edge:
 |
2 |
Third, we require that each vertex be matched with at most one edge and so have clauses that exclude matching edges that overlap on one or two coordinates.
 |
3 |
Fourth, we exclude the trivial 3D matching by requiring that at least one of the diagonal edges not be used: 
Finally, we AND these into the overall CNF formula: 
The size of the CNF formula 
is 
, has 
variables, and is a factor of 
smaller than the naïve approach. Thus we reduce 3D matching to satisfiability by converting the instance 
into the CNF formula 
.
To solve the reduced satisfiability instance we used the open-source solver MapleCOMPSPS from the 2016 International SAT Competition [5]. This solver is conflict driven and uses a learning rate branching heuristic to decide which variables are likely to lead to conflict and has had demonstrable success in practice [19]. We chose MapleCOMPSPS because it was state of the art at the time our project started. It is likely that more recently-developed solvers would achieve similar or better performance on our task.
3D Matching to Integer Programming
In parallel to the previous subsection, we use the connection between verification of strong USPs and 3D matching to reduce the former to integer programming, another well-known 
-complete problem (c.f., e.g., [17]). Again, let 
be the 3D matching instance associated with P. We construct an integer program 
over 
that is infeasible iff P is a strong USP. Here the reduction is simpler than the previous one because linear constraints naturally capture matching.
We use 
to denote a variable with values in 
to indicate whether the edge 
is present in the matching. To ensure that M is a subset of E we add the following edge constraints to 
: 
We also require that each vertex in each of the three parts of the graph is incident to exactly one edge in M. This is captured by the following vertex constraints in 
: 
Lastly, since we need that the 3D matching be non-trivial we add the constraint: 
To check whether P is a strong USP we determine whether 
is not feasible, i.e., that no assignment to the variables M satisfy all constraints. In practice this computation is done using the commercial, closed-source, mixed-integer programming solver Gurobi [15]. We note that reduction from 3D matching to IP is polynomial time and that there are 
variables in 
, and that the total size of the constraints is 
, similar to size of 
in the SAT reduction.
Heuristics
Although the exact algorithms presented in the previous sections make substantial improvements over the brute force approach, the resulting performance remains impractical. To resolve this, we also develop several fast verification heuristics that may produce the non-definitive answer MAYBE in place of YES or NO. Then, to verify a puzzle P we run this battery of fast heuristics and return early if any of the heuristics produce a definitive YES or NO. When all the heuristics result in MAYBE, we then run one of the slower exact algorithms that were previously discussed. The heuristics have different forms, but all rely on the structural properties of a strong USP. Here we discuss the two most effective heuristics, downward closure and greedy, and defer a deeper discussion of these and several less effective heuristics, including projection to 2D matching, to the full version of this article.
Downward Closed. The simplest heuristics we consider is based on the fact that strong USPs are downward closed.
Lemma 4
If P is a strong USP, then so is every subpuzzle 
.
Proof
Let P be a strong USP and 
. By Definition 2, for every 
not all identity, there exist 
and 
such that exactly two of the following hold: 
, 
, 
. Consider restricting the permutations to those that fix the elements of 
. For these permutations it must be the case that 
because otherwise 
and there is exactly one 
for which 
holds. Thus we can drop the elements of 
and conclude that for every tuple of permutations in 
the conditions of Definition 2 hold for 
, and hence that 
is a strong USP. 
This leads to a polynomial-time heuristic that can determine that a puzzle is not a strong USP. Informally, the algorithm takes an (s, k)-puzzle P and 
, and verifies that all subsets 
with size 
are strong USPs. If any subset 
is not a strong USP, the heuristic returns NO, otherwise it returns MAYBE. This algorithm runs in time 
where 
is the runtime for verifying an 
-puzzle. In practice we did not apply this heuristic for 
larger than 3, so the effective running time was 
, which is polynomial in s and k using the verification algorithms from the previous subsections that eliminate dependence on k for polynomial cost. This heuristic can be made even more practical by caching the results for puzzles of size 
, reducing the verification time per iteration to constant in exchange for 
time and 
space to precompute the values for all puzzles of size 
. From a practical point of view, running this heuristic is free for small constant 
, as even the reductions in the exact verification algorithms have a similar or higher running time.
Greedy. This heuristic attempts to greedily solve the 3D matching instance 
. The heuristic proceeds iteratively, determining the vertex of the first part of the 3D matching instance with the least edges and randomly selecting an edge of that vertex to put into the 3D matching. If the heuristic successfully constructs a 3D matching it returns NO indicating that the input puzzle P is not a strong USP. If the heuristic reaches a point were prior commitments have made the matching infeasible, the heuristic starts again from scratch. This process is repeated some number of times before it gives up and returns MAYBE. In our implementation we use 
attempts because it is similar to the running time of the reductions and it empirically reduced the number of instances requiring full verification in the domain of puzzles with 
while not increasing the running time by too much.
Hybrid Algorithm
Our final verification algorithm (Algorithm 2) is a hybrid of several exact algorithms and heuristics. The size thresholds for which algorithm and heuristic to apply were determined experimentally for small k and were focused on the values were our strong USP search algorithms were tractable 
(or nearly tractable 
). We decided to run both the reduction to SAT and IP in parallel because it was not clear which algorithm performed better. Since verification halts when either algorithm completes, the wasted effort is within a factor of two of what the better algorithm could have done alone. We also chose to do this because we experimentally observed that there were many instances that one of the algorithms struggled with that the other did not—this resulted in a hybrid algorithm that out performed the individual exact algorithms on average.
Searching for Strong USPs
In some ways the problem of constructing a large strong USP is similar to the problem of constructing a large set of linearly independent vectors. In both cases, the object to be constructed is a set, the order that elements are added does not matter, the underlying elements are sequences of numbers, and there is a notion of (in)dependence among sets of elements. There are well-known polynomial-time algorithms for determining whether a set of vectors are independent, e.g., Gaussian elimination, and we have a practical implementation for deciding whether a puzzle is a strong USP.
There is a straightforward greedy algorithm for constructing maximum-size sets of independent vectors: Start with an empty set S, and repeatedly add vectors to S that are linearly independent of S. After this process completes S is a largest set of linearly independent vectors. This problem admits such a greedy algorithm because the family of sets of linearly independent vectors form a matroid. The vector to be added each step can be computed efficiently by solving a linear system of equations for vectors in the null space of S.
Unfortunately this same approach does not work for generating maximum-size strong USPs. The set of strong USPs does not form a matroid, rather it is only an independence system, c.f., e.g., [20]. In particular, (i) the empty puzzle is a strong USP and (ii) the set of strong USP are downward closed by Lemma 4. The final property required to be a matroid, the augmentation property, requires that for every pair of strong USPs 
with 
there is a row of 
such that 
is also a strong USP. A simple counterexample with the strong USPs 
and 
concludes that neither 
nor 
are strong USPs, and hence the augmentation property fails. One consequence is that naïve greedy algorithms will likely be ineffective for finding maximum-size strong USPs. Furthermore, we do not currently have an efficient algorithm that can take a strong USP P and determine a row r such that 
is a strong USP aside from slight pruning of the 
possible next rows r.
That said, we have had some success applying general purpose search techniques together with our practical verification algorithm to construct maximum-size strong USPs for small k. In particular, we implemented variants of depth-first search (DFS) and breadth-first search (BFS). We defer the details of this to the full version of this article.
The actual running times of both of these algorithms are prohibitive even for 
, and the greater memory usage of BFS to store the entire search frontier is in the tens of terabytes even for 
. There are some silver linings, DFS can report intermediate results which are the maximally strong USPs that it has discovered so far. Both algorithms admit the possibility of eliminating puzzles from the search that are equivalent to puzzles that have already been searched, though it is easier to fit into the structure BFS as the puzzles are already being stored in a queue.
Experimental Results
Our experimental results come in three flavors for small-constant width k: (i) constructive lower bounds on the maximum size of width-k strong USPs witnessed by found puzzles, (ii) exhaustive upper bounds on the maximum size of width-k strong USPs, and (iii) experimental run times comparing the algorithms for verifying width-k strong USPs. BFS and DFS when able to run to completion search the entire space, up to puzzle isomorphism, and provide tight upper and lower bounds. When unable to run to completion they provide only results of form (i) and are not guaranteed to be tight.
New Bounds on the Size of Strong USPs
Figure 3 contains representative examples of maximum-size strong USPs we found for 
. Table 1 summarizes our main results in comparison with [8]. The lower bounds of [8] are from the constructions in their Propositions 3.1 and 3.8 that give families of strong USPs for k even or k divisible by three. The upper bounds of [8] follow from their Lemma 3.2 (and the fact that the capacity of strong USPs is bounded above by the capacity of USPs). The values of 
in this table are computed by plugging s and k into Lemma 1 and optimizing over m. For clarity we omit 
’s that would be larger than previous lines.
Fig. 3.
Representative maximum-size strong USPs found for width 
.
Table 1.
Comparison of bounds on maximum size of strong USPs with [8] for small k.
| [8] | This work | |||
|---|---|---|---|---|
| k | Maximum s |  |
Maximum s |  |
| 1 |  |
3.000 |  |
3.000 |
| 2 |  |
2.875 |  |
2.875 |
| 3 |  |
2.849 |  |
2.849 |
| 4 |  |
2.850 |  |
2.806 |
| 5 |  |
 |
2.777 | |
| 6 |  |
2.792 |  |
2.733 |
| 7 |  |
 |
2.722 | |
| 8 |  |
 |
2.719 | |
| 9 |  |
2.739 |  |
2.718 |
| 10 |  |
 |
2.706 | |
| 11 |  |
 |
2.678 | |
| 12 |  |
2.696 |  |
2.653 |
We derive tight bounds for all 
and constructively improve the known lower bounds for 
. The strong uniquely solvable (14, 6)-puzzles we found represent the greatest improvement in 
versus the construction of [8]. Further, our puzzle for 
is the result of taking the Cartesian product of two copies of a strong uniquely solvable (14, 6)-puzzle. Repeating this process with more copies of the puzzle gives a strong USP implying 
. Note that Proposition 3.8 of [8] gives an infinite family of strong USPs that achieves 
in the limit.
Based on the processing time we spent on 
, we conjecture that 
is tight for 
and that our lower bounds for 
are not. Our results suggests there is considerable room for improvement in the construction of strong USPs, and that it is likely that there exist large puzzles for 
that would beat [8]’s construction and perhaps come close to the Coppersmith-Winograd refinements. It seems that new insights into the search problem are required to proceed for 
.
Algorithm Performance
We implemented our algorithms in C++ (source code to be made available on github) and ran them on a 2010 MacPro running Ubuntu 16.04 with dual Xeon E5620 2.40 Ghz processors and 16 GB of RAM. Figure 4 contains log plots that describe the performance of our algorithms on sets of 10000 random puzzles at each point on a sweep through parameter space for width 
and size 
. We chose to test performance via random sampling because we do not have access to a large set of solved instances. This domain coincides with the frontier of our search space, and we tuned the parameters of the heuristics and algorithms in the hybrid algorithm to perform well in this domain. We did not deeply investigate performance characteristics outside of this domain.
Fig. 4.
Log plots of the average running times for verifying 10000 random puzzles of widths five to ten. Note that the legend in (a) applies to all six plots, and that the axes are named and labeled only on the edges of the page. Each plot describes the behavior of five algorithms brute force (BF), dynamic programming (DP), reduction to satisfiability (SAT), reduction to integer programming (IP), and the hybrid algorithm (HYB). The final dashed line indicates the percentage of strong USP found among the 10000 random puzzles.
The brute force and dynamic programming algorithms perform poorly except for very small size, 
, and their curves loosely match the 
time bounds we have. The plots for the two reduction-based algorithms (SAT and IP) behave similarly to each other. They are slower than brute force and dynamic programming for small values of s, and their behavior for large s is quite a bit faster. We speculate that the former is due to the cost of constructing the reduced instance and overhead of the third party tools. Further observe that the SAT reduction handily beats the IP reduction on large size for 
, but as k increases, the IP reduction becomes faster. We also note that across the six plots the IP reduction has effectively the same running time and is independent of k, this is likely because the size of the IP instance depends only on s. The hybrid algorithm generally performs best or close to best. Notice that it matches the dynamic programming algorithm closely for small values of s and then diverges when the reduction-based algorithms and heuristics are activated around 
. Observe that the hybrid algorithm is effectively constant time for large s. We expect this is because the density of strong USPs decreases rapidly with s, and that the randomly selected puzzles are likely far from satisfying Definition 2 and are quickly rejected by the downward closure heuristic.
Overall, our hybrid verification algorithm performs reasonably well in practice, despite reductions through 
-complete problems.
Conclusions
We initiated the first study of the verification of strong USPs and developed practical software for both verifying and searching for them. We give tight results on the maximum size of width-k strong USPs for 
. Although our results do not produce a new upper bound on the running time of matrix multiplication, they demonstrate there is promise in this approach. There are a number of open questions. Is strong USP verification 
-complete? What are tight bounds on maximum size strong USPs for 
and do these bound lead to asymptotically faster algorithms for matrix multiplication? The main bottleneck in our work is the size of the search space—new insights seem to be required to substantially reduce it. We have preliminary results that indicate that the size of the search space can be reduced by modding out by the symmetries of puzzles, though this has not yet led to new lower bounds.
Acknowledgments
The second and third authors thank Union College for Undergraduate Summer Research Fellowships funding their work. The authors thank the anonymous reviewers for their detailed and thoughtful suggestions for improving this work.
Contributor Information
Luca Pulina, Email: lpulina@uniss.it.
Martina Seidl, Email: martina.seidl@jku.at.
Matthew Anderson, Email: andersm2@union.edu.
Zongliang Ji, Email: jiz@union.edu.
Anthony Yang Xu, Email: xua@union.edu.
References
- 1.Alman, J., Williams, V.V.: Further limitations of the known approaches for matrix multiplication. In: Leibniz International Proceedings Information LIPIcs of the 9th Innovations in Theoretical Computer Science (ITCS), vol. 94, p. 15, Article no. 25. Schloss Dagstuhl. Leibniz-Zent. Inform., Wadern (2018)
- 2.Alman, J., Williams, V.V.: Limits on all known (and some unknown) approaches to matrix multiplication. In: 59th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pp. 580–591, October 2018. 10.1109/FOCS.2018.00061
- 3.Alon N, Shpilka A, Umans C. On sunflowers and matrix multiplication. Comput. Complex. 2013;22(2):219–243. doi: 10.1007/s00037-013-0060-1. [DOI] [Google Scholar]
- 4.Ambainis, A., Filmus, Y., Le Gall, F.: Fast matrix multiplication: limitations of the coppersmith-winograd method. In: 47th Annual ACM Symposium on Theory of Computing (STOC), pp. 585–593. ACM (2015)
- 5.Balyo, T., Heule, M.J., Jarvisalo, M.: SAT competition 2016: recent developments. In: 31st AAAI Conference on Artificial Intelligence (AAAI) (2017)
- 6.Björklund A, Husfeldt T, Kaski P, Koivisto M. Narrow sieves for parameterized paths and packings. J. Comput. Syst. Sci. 2017;87:119–139. doi: 10.1016/j.jcss.2017.03.003. [DOI] [Google Scholar]
- 7.Blasiak, J., Church, T., Cohn, H., Grochow, J.A., Umans, C.: Which groups are amenable to proving exponent two for matrix multiplication? arXiv preprint arXiv:1712.02302 (2017)
- 8.Cohn, H., Kleinberg, R., Szegedy, B., Umans, C.: Group-theoretic algorithms for matrix multiplication. In: 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pp. 379–388, October 2005. 10.1109/SFCS.2005.39
- 9.Cohn, H., Umans, C.: A group-theoretic approach to fast matrix multiplication. In: 44th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pp. 438–449, October 2003. 10.1109/SFCS.2003.1238217
- 10.Coppersmith D, Winograd S. Matrix multiplication via arithmetic progressions. J. Symbolic Comput. 1990;9(3):251–280. doi: 10.1016/S0747-7171(08)80013-2. [DOI] [Google Scholar]
- 11.Cormen TH, Leiserson CE, Rivest RL, Stein C. Introduction to Algorithms. 3. Cambridge: The MIT Press; 2009. [Google Scholar]
- 12.Croot E, Lev VF, Pach PP. Progression-free sets in are exponentially small. Ann. Math. 2017;185:331–337. doi: 10.4007/annals.2017.185.1.7. [DOI] [Google Scholar]
- 13.Davie AM, Stothers AJ. Improved bound for complexity of matrix multiplication. Proc. R. Soc. Edinb. Sect. A Math. 2013;143(2):351–369. doi: 10.1017/S0308210511001648. [DOI] [Google Scholar]
- 14.Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness (1979)
- 15.Gurobi Optimization LLC: Gurobi optimizer reference manual (2018). http://www.gurobi.com
- 16.Kaminski M. A lower bound on the complexity of polynomial multiplication over finite fields. SIAM J. Comput. 2005;34(4):960–992. doi: 10.1137/S0097539704442118. [DOI] [Google Scholar]
- 17.Korte B, Vygen J. Combinatorial Optimization. Heidelberg: Springer; 2012. [Google Scholar]
- 18.Le Gall, F.: Powers of tensors and fast matrix multiplication. In: 39th International Symposium on Symbolic and Algebraic Computation (ISSAC), pp. 296–303. ACM (2014)
- 19.Liang Jia Hui, Ganesh Vijay, Poupart Pascal, Czarnecki Krzysztof. Learning rate based branching heuristic for SAT solvers. In: Creignou Nadia, Le Berre Daniel., editors. Theory and Applications of Satisfiability Testing – SAT 2016; Cham: Springer; 2016. pp. 123–140. [Google Scholar]
- 20.Oxley JG. Matroid Theory. USA: Oxford University Press; 2006. [Google Scholar]
- 21.Pan, V.Y.: Strassen’s algorithm is not optimal trilinear technique of aggregating, uniting and canceling for constructing fast algorithms for matrix operations. In: 19th Annual Symposium on Foundations of Computer Science (FOCS), pp. 166–176. IEEE (1978)
- 22.Schönhage A. Partial and total matrix multiplication. SIAM J. Comput. 1981;10(3):434–455. doi: 10.1137/0210032. [DOI] [Google Scholar]
- 23.Shpilka A. Lower bounds for matrix product. SIAM J. Comput. 2003;32(5):1185–1200. doi: 10.1137/S0097539702405954. [DOI] [Google Scholar]
- 24.Strassen V. Gaussian elimination is not optimal. Numer. Math. 1969;13(4):354–356. doi: 10.1007/BF02165411. [DOI] [Google Scholar]
- 25.Strassen, V.: The asymptotic spectrum of tensors and the exponent of matrix multiplication. In: 27th Annual Symposium on Foundations of Computer Science (FOCS), pp. 49–54. IEEE (1986)
- 26.Williams, V.V.: Multiplying matrices faster than Coppersmith-Winograd. In: 44th Annual ACM Symposium on Theory of Computing (STOC), pp. 887–898. ACM (2012)