Powers of a (Square) Matrix

5.9 Powers of a (Square) Matrix

In this section, we discuss a special case of matrix³⁴: the square matrix³⁵. To be precise, we discuss a special operation of square matrix: the powers of a square matrix. Mathematically, we say that M⁰ = I and M^p =^p_i=1M . I is the Identity matrix³⁶ and p is the asked power of square matrix M . If we can do this operation in O(n³log p) – which is the main topic of this subsection, we can solve some more interesting problems in programming contests, e.g.:

• Compute a single Fibonacci number fib(p) in O(log p) time instead of O(p).

Imagine if p = 2³⁰. O(p) solution will get TLE whereas log₂(2³⁰) is just 30 steps.

Note: if we need all f ib(n) for all n∈ [1..n], use O(n) DP solution instead.

This is achievable by using the following equality:

[ 1 1 ]^p [ fib(p+1) fib(p) ] [ 1 0 ] = [ fib(p) fib(p-1) ]

For example, to compute f ib(11), we simply multiply the Fibonacci matrix 11 times, i.e. raise it to the power of 11.

[ 1 1 ]^11 [ 144 89 ] [ fib(12) fib(11) ] [ 1 0 ] = [ 89 55 ] = [ fib(11) fib(10) ]

• Compute number of paths of length L of a graph stored in an Adjacency Matrix in O(n³log L).

Adjacency Matrix is a square matrix. Example: see the small graph of size n = 4 stored in an Adjacency Matrix M below. The various paths from vertex 0 to vertex 1 with different lengths are shown in entry M [0][1] after M is raised to power L.

0->1 with length 1 -> possible: 0->1 (only 1 path) 0--1 0->1 with length 2 -> impossible

| 0->1 with length 3 -> possible: 0->1->2->1 (0->1->0->1) 2--3 0->1 with length 4 -> impossible

0->1 with length 5 -> possible: 0->1->2->3->2->1 (and 4 others) M = [ 0 1 0 0 ] M^2 = [ 1 0 1 0 ] M^3 = [ 0 2 0 1 ] M^5 = [ 0 5 0 3 ]

[ 1 0 1 0 ] [ 0 2 0 1 ] [ 2 0 3 0 ] [ 5 0 8 0 ]

[ 0 1 0 1 ] [ 1 0 2 0 ] [ 0 3 0 2 ] [ 0 8 0 5 ]

[ 0 0 1 0 ] [ 0 1 0 1 ] [ 1 0 2 0 ] [ 3 0 5 0 ]

5.9.1 The Idea of Efficient Exponentiation

For the sake of argument, let’s assume that built-in function like pow(A, p) or other related functions that can raise a number A to a certain integer power p does not exist. Now, if we do exponentiation

‘by definition’, we will have an inefficient O(p) solution, especially if p is large³⁷.

int normalExp(int base, int p) { // for simplicity, we just use int data type int ans = 1;

for (int i = 0; i < p; i++) ans *= base; // O(p)

return ans; }

34Matrix is a rectangular (2D) array of numbers. Matrix of size m × n has m rows and n columns. The elements of the matrix is usually denoted by the matrix name with two subscripts.

35A square matrix is a matrix with the same number of rows and columns, i.e. n × n.

36Identity matrix is a matrix with all cells filled with zeroes except that cells along the main diagonal are all ones.

37If you encounter input size of ‘gigantic’ value in programming problems, like 1B, the problem setter is usually looking for a logarithmic solution. Notice that log₂(1B) ≈ log₂(2³⁰) is still just 30!

5.9. POWERS OF A (SQUARE) MATRIX  Steven & Felixc

There is a better way that uses Divide & Conquer principle. We can express A^p as:

A⁰ = 1 and A¹ = A (two base cases).

A^p = A^p−1× A if p is odd.

A^p = (A^p/2)² if p is even. As this approach keep halving the value of p by two, it runs in O(log p).

For example, by definition: 2⁹ = 2× 2 × 2 × 2 × 2 × 2 × 2 × 2 × 2 ≈ O(p) multiplications.

But with Divide & Conquer: 2⁹= 2⁸× 2 = (2⁴)²× 2 = ((2²)²)²× 2 ≈ O(log p) multiplications.

A typical recursive implementation of this Divide & Conquer exponentiation is shown below:

int fastExp(int base, int p) { // O(log p)

if (p == 0) return 1;

else if (p == 1) return base;

else { int res = fastExp(base, p / 2); res *= res;

if (p % 2 == 1) res *= base;

return res; } } 5.9.2 Square Matrix Exponentiation

We can use the same efficient exponentiation technique shown above to perform square matrix ex-ponentiation in O(n³log p). Each matrix multiplication is O(n³). In case of 2×2 Fibonacci matrix, this is just 8 multiplications, thus we can compute f ib(p) in O(log p). The iterative implementation (for comparison with the recursive implementation shown earlier) is shown below:

#define MAX_N 105 // increase this if needed

struct Matrix { int mat[MAX_N][MAX_N]; }; // so that we can return a 2D array

Matrix matMul(Matrix a, Matrix b) { // O(n^3)

Matrix ans; int i, j, k;

for (i = 0; i < MAX_N; i++) for (j = 0; j < MAX_N; j++)

for (ans.mat[i][j] = k = 0; k < MAX_N; k++) // if necessary, ans.mat[i][j] += a.mat[i][k] * b.mat[k][j]; // do modulo arithmetic here return ans; }

Matrix matPow(Matrix base, int p) { // O(n^3 log p)

Matrix ans; int i, j;

for (i = 0; i < MAX_N; i++) for (j = 0; j < MAX_N; j++)

ans.mat[i][j] = (i == j); // prepare identity matrix

while (p) { // iterative version of Divide & Conquer exponentiation if (p & 1) ans = matMul(ans, base); // check if p is odd (last bit is on)

base = matMul(base, base); // square the base

p >>= 1; // divide p by 2

}

return ans; }

Example code: ch5 07 UVa10681.cpp; ch5 07 UVa10681.java Programming Exercises related to Powers of a (Square) Matrix:

1. UVa 10229 - Modular Fibonacci * (Fibonacci, matrix power, modulo arithmetic) 2. UVa 10681 - Teobaldo’s Trip * (power of AdjMatrix, as discussed in this section) 3. UVa 10870 - Recurrences * (form the required matrix first; power of matrix) 4. UVa 12045 - Fun with Strings (eq with 2 unknowns, Fibonacci, matrix power, modulo)

5.10. CHAPTER NOTES  Steven & Felixc

5.10 Chapter Notes

Compared to the first edition of this book, this chapter has grown almost twice the size. However, we are aware that there are still many more mathematics problems and algorithms that have not been discussed in this chapter. Here, we list some pointers to more advanced topics that you may be interested to explore further by reading number theory books, e.g. [31], investigating mathemat-ical topics in http://mathworld.wolfram.com/ or Wikipedia, and do many more programming exercises related to mathematics problems like the ones in http://projecteuler.net/ [8].

• There are many more combinatorics problems and formulas that are not yet discussed:

Burnside’s lemma, Cayley’s Formula, Derangement, Stirling Numbers, etc.

• For an even faster prime testing function than the one presented here, one can use the non deterministic Miller-Rabin’s algorithm – which can be made deterministic for contest environment with a known maximum input size N .

• In this chapter, we have seen a quite effective trial division method for finding prime factors of an integer and lots of its variant functions. For a faster integer factorization, one can use the Pollard’s rho algorithm that uses another cycle detection algorithm called Brent’s cycle finding method. However, if the integer to be factored is a large prime number, then this is still slow. This fact is the key idea of modern cryptography techniques.

• There are other theorems, hypothesis, and conjectures that cannot be discussed one by one, e.g. Carmichael’s function, Riemann’s hypothesis, Goldbach’s conjecture, Fer-mat’s Little Test, twin prime conjecture, Chinese Remainder Theorem, Sprague-Grundy Theorem, etc.

• To compute the solution of a system of linear equations, one can use techniques like Gaussian elimination.

As you can see, there are many topics about mathematics. This is not surprising since various mathematics problems have been investigated by people since many hundreds of years ago. Some of them are discussed in this chapter, many others are not, and yet only 1 or 2 will actually appear in a problem set. To do well in ICPC, it is a good idea to have at least one strong mathematician in your ICPC team in order to have those 1 or 2 mathematics problems solved. Mathematical prowess is also important for IOI contestants. Although the amount of topics to be mastered is smaller, many IOI tasks require some form of ‘mathematical insights’.

Note that (Computational) Geometry is also part of Mathematics, but since we have a special chapter for that, we reserve the discussions about geometry problems in Chapter 7.

There are ≈ 285 UVa (+ 10 others) programming exercises discussed in this chapter.

(Only 175 in the first edition, a 69% increase).

There are 29 pages in this chapter.

(Only 17 in the first edition, a 71% increase).

5.10. CHAPTER NOTES  Steven & Felixc

This page is intentionally left blank to keep the number of pages per chapter even.

Chapter 6