AMath 584: Applied Linear Algebra, Autumn 2017
Name:________________________
Applied Linear Algebra Final
Six pages. Total of 50 points. Show your work.
Closed book. No calculators. One page (8.5x11, one side only) of handwritten notes.
1. (10 points) Let A= 0 1 2 3 4 5 6 7 8 .
(a) (7 points) Factor Ain the formP LU, whereP is a permutation matrix, Lis unit lower triangular with all subdiagonal entries less than or equal to 1 in magnitude, and U is upper triangular.
First interchange rows 1 and 3:
P1 = 0 0 1 0 1 0 1 0 0 , A→ 6 7 8 3 4 5 0 1 2 .
Now replace row 2 by itself minus 1/2 times row 1:
L1 = 1 0 0 −1/2 1 0 0 0 1 , A→ 6 7 8 0 1/2 1 0 1 2 .
Now interchange rows 2 and 3:
P2 = 1 0 0 0 0 1 0 1 0 , A→ 6 7 8 0 1 2 0 1/2 1 .
Finally, replace row 3 by itself minus 1/2 times row 2:
L2 = 1 0 0 0 1 0 −1/2 0 1 , A→ 6 7 8 0 1 2 0 0 0 =U.
Thus we have written L2P2L1P1A = U, or, L2(P2L1P2−1)(P2P1)A = U,
or, A=P LU, where P = (P2P1)−1 = 0 1 0 0 0 1 1 0 0 , L= [L2(P2L1P −1 2 )] −1 = 1 0 0 0 1 0 1/2 1/2 1 . 1
(b) (3 points) Based on the calculation in (a), determine the rank of A. Since the rank of U is 2, the rank ofA is 2.
2. (5 points) Let A be a nonsingular m by m matrix. Suppose an algorithm for solving
Ax=b produces an approximate solution ˆx such that
kb−Axˆk
kAk kxˆk >> O(machine).
Show that if ˆx is the exact solution to a system of the form (A +E)ˆx = b, then
kEk/kAk must be much greater thanO(machine). Be sure to justify all steps.
If (A+E)ˆx=b, thenExˆ=b−Axˆ. Taking norms on each side,
kEk · kxˆk ≥ kExˆk=kb−Axˆk,
where the first inequality follows from the definition of an operator norm:
kEk = maxy6=0kEyk/kyk ⇒ kEk · kyk ≥ kEyk ∀y 6= 0. Now dividing by
kAk kxˆk, this becomes
kEk
kAk ≥
kb−Axˆk
3. (10 points) Let f be a smooth function and consider the following centered difference approximation tof0(x):
f0(x)≈ f(x+h)−f(x−h)
2h . (1)
It can be shown using Taylor’s theorem that the error in this approximation is:
h2
6|f
000
(ξ)|, ξ ∈(x−h, x+h). (2) (a) (2 points) Make a sketch of the truncation error defined in (2) as a function of h. [You do not have to show actual numbers in your plot, just the general shape of the curve, whether it is increasing or decreasing withh, etc.]
Truncation error (solid), and Rounding error (dashed)
h error
(b) (4 points) Suppose one uses formula (1) to approximatef0(x), wheref(x) = sin(x) and x=π/3 (so that the true value cos(π/3) is 0.5). Suppose the only rounding errors made are in rounding the valuesf(x+h) andf(x−h), so that the computed values aref(x+h)(1 +1) and f(x−h)(1 +2), where |1| and |2| are both less
than or equal to the machine precisionmachine. Assume that all other operations
are performed exactly. What is the maximum amount by which the computed difference quotient could differ from the exact difference quotient? On the same graph that you used in (a), sketch the rounding error as a function of h. [Again, just show the general shape of the curve, what happens as h → 0 and as h gets larger. Be sure to label which curve is which.]
We can write f(x+h)(1 +1)−f(x−h)(1 +2) 2h = f(x+h)−f(x−h) 2h + f(x+h)1−f(x−h)2 2h .
It follows that the rounding error is proportional to 1h, and it could be as large as about 2machine|f(x)|/(2h) =
√
3 machine/(2h).
(c) (4 points) Based on your results in (a) and (b), determine the approximate value of h for which you would expect to get the best accuracy and how accurate the computed approximation would be for that value ofh.
Comparing the rounding error, √3 machine/(2h), with the truncation
error, h2/12, we see that the two are equal when
h3 = 6√3 machine ⇒h= (6
√
3)1/31machine/3 .
For this value of h, the total error – truncation error plus rounding error – is about (6√3)2/32/3
4. (10 points) Let A= −1 3 0 0 3 5 −1 0 0 −1 12 −1 0 0 −1 16 .
(a) (5 points) Sketch the Gerschgorin row disks and indicate how many eigenvalues of A lie in each region.
-5 0 5 10 15 20 real -10 -8 -6 -4 -2 0 2 4 6 8 10 imaginary Gerschgorin Disks
There are two eigenvalues in the leftmost pair of disks, since they overlap. There is one eigenvalue in each of the other two disks.
(b) (5 points) Let D= diag(1,1,1, d),d >0. Then
DAD−1 = −1 3 0 0 3 5 −1 0 0 −1 12 −1/d 0 0 −d 16 ,
For what value ofd will the Gerschgorin row disks give the sharpest information about the eigenvalue near 16, and how close to 16 will they show it to be? Explain your answer.
We want d to be as small as possible, subject to the disk about 16 re-maining disjoint from the one about 12. These disks just touch when
12 + 1 + 1/d= 16−d→d= 3±
√
5 2 .
If we take djust greater than (3−√5)/2, then the row disks tell us that there is an eigenvalue within this distance of 16.
5. (4 points) Let Abe anmbym real symmetric matrix with eigenvaluesλ1 ≤. . .≤λm.
For any real m-vector xwith kxk2 = 1, define the Rayleigh quotient
r(x) =xTAx.
Show thatλ1 ≤r(x)≤λm.
Since A is a real symmetric matrix, it has a complete set of orthonormal eigenvectors, say, q1, . . . ,qm. Expand x as a linear combination of these:
x=Pm
j=1cjqj. Since the qj’s are orthonormal, cj =hx,qji and Pmj=1|cj|2 =
kxk2 = 1. Then, again using the orthonormality of the eigenvectors q
j: r(x) = m X j=1 cjqj T m X j=1 cjλjqj = m X j=1 |cj|2λj. It follows that λ1 m X j=1 |cj|2 ≤r(x)≤λm m X j=1 |cj|2;
that is, sincePm
j=1|cj|2 = 1, λ1 ≤r(x)≤λm.
6. (4 points) Let A be a real symmetric 3 by 3 matrix with eigenvalues λ1 = 1, λ2 = 2,
and λ3 = 3, and with corresponding orthonormal eigenvectors q1, q2, and q3. Let
s= 2 + 10−15.
(a) (1 point) What is the 2-norm condition number of A−sI?
The eigenvalues of A−sI are −1−10−15, −10−15, and 1−10−15, and sinceA−sI is real symmetric, the singular values are the absolute values of the eigenvalues. The 2-norm condition number is the ratio of largest to smallest singular value, which is (1 + 10−15)/10−15= 1015+ 1.
(b) (1 point) About how many (decimal) digits of acuracy would you expect in the computed solution to a linear system of the form (A−sI)x = b, if you solve it using a backward stable algorithm on a computer with unit roundoff machine
approximately equal to 10−16? Give a short reason for your answer.
I would expect to get about 1 digit of accuracy sinceκ(A)machine ≈10−1.
(c) (2 points) Suppose your computed solution xˆ is the exact solution to the linear system (A−sI)ˆx=b+ 10−16(q1+q2+q3). Express the errorˆx−xas a linear
combination of q1, q2, and q3. Which component is the largest?
We can write ˆ x−x = 10−16(A−sI)−1(q1+q2+q3) = − 10 −16 1 + 10−15q1−10 −1 q2+ 10−16 1−10−15q3 ≈ −10 −16 q1−10−1q2+ 10−16q3.
The largest component, by far, is in the direction of q2.
[This is why shifted inverse iteration and Rayleigh quotient iteration work in practice; the linear systems are not solved very accurately, but the main error is in the direction of the eigenvector that we are seeking!]
7. (7 points) Write down the result of one step of the unshifted QR algorithm for the matrix A= " 1 1 1 0 #
What would be the value of the Wilkinson shift for A?
The QR decomposition of A is A= " 1/√2 1/√2 1/√2 −1/√2 # " √ 2 1/√2 0 1/√2 # . Therefore A(1) =RQ= " √ 2 1/√2 0 1/√2 # " 1/√2 1/√2 1/√2 −1/√2 # = " 3/2 1/2 1/2 −1/2 # .
The Wilkinson shift would be the eigenvalue of A that is closest to 0. The eigenvalues ofAare (1±√5)/2, so the Wilkinson shift would be (1−√5)/2.
