Matrix Analysis. Computations. and

(1)

Matrix Analysis Computations and

OT173_BAI_FM_V2.indd 1

OT173_BAI_FM_V2.indd 1 5/10/2021 3:57:58 PM5/10/2021 3:57:58 PM

Downloaded 02/15/22 to 134.122.89.123 Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/page/terms

(2)

(3)

Society for Industrial and Applied Mathematics Philadelphia

Zhong-Zhi Bai

Academy of Mathematics and Systems Science Chinese Academy of Sciences

Beijing, China

Jian-Yu Pan

East China Normal University Shanghai, China

Matrix Analysis Computations and

(4)

All rights reserved. Printed in the United States of America. No part of this book may be reproduced, stored, or transmitted in any manner without the written permission of the publisher. For information, write to the Society for Industrial and Applied Mathematics, 3600 Market Street, 6th Floor, Philadelphia, PA 19104-2688 USA.

No warranties, express or implied, are made by the publisher, authors, and their employers that the programs contained in this volume are free of error. They should not be relied on as the sole basis to solve a problem whose incorrect solution could result in injury to person or property. If the programs are employed in such a manner, it is at the user’s own risk and the publisher, authors, and their employers disclaim all liability for such misuse.

Trademarked names may be used in this book without the inclusion of a trademark symbol. These names are used in an editorial context only; no infringement of trademark is intended.

MATLAB is a registered trademark of The MathWorks, Inc. For MATLAB product information, please contact The MathWorks, Inc., 3 Apple Hill Drive, Natick, MA 01760-2098 USA, 508-647-7000, Fax: 508-647-7001, [email protected], www.mathworks.com.

Publications Director Kivmars H. Bowling Executive Editor Elizabeth Greenspan Developmental Editor Mellisa Pascale Managing Editor Kelly Thomas Production Editor Ann Manning Allen

Copy Editor Claudine Dugan

Production Manager Donna Witzleben Production Coordinator Cally A. Shrader

Compositor Cheryl Hufnagle

Graphic Designer Doug Smock

Library of Congress Cataloging-in-Publication Data Names: Bai, Zhong-Zhi, author. | Pan, Jian-Yu, author.

Title: Matrix analysis and computations / Zhong-Zhi Bai, Jian-Yu Pan.

Description: Philadelphia : Society for Industrial and Applied Mathematics, [2021] | Includes bibliographical references and index. | Summary: “This book introduces the basics of matrix analysis and presents

representative methods and their corresponding theories in matrix computations”-- Provided by publisher.

Identifiers: LCCN 2021011456 (print) | LCCN 2021011457 (ebook) | ISBN 9781611976625 (paperback) | ISBN 9781611976632 (ebook) Subjects: LCSH: Matrices. | Numerical analysis.

Classification: LCC QA188 .B324 2021 (print) | LCC QA188 (ebook) | DDC 512.9/434--dc23

LC record available at https://lccn.loc.gov/2021011456 LC ebook record available at https://lccn.loc.gov/2021011457

is a registered trademark.

(5)

Preface

Analysis and computation, the most important and indispensable methods for processing matrices, are closely related but two significantly different areas. The former focuses more on theoretical analysis and belongs to the category of pure linear algebra, while the latter concentrates more on practical applications and is classified as numerical linear algebra, although both of them are located at the core bases of pure and computational mathematics. Moreover, matrix theory is the kernel and foundation of matrix computations, while matrix computations are the extensions and applications of matrix theory. Therefore, these two kinds of knowledge are absolute requirements for matrix analysts, computational scientists, and algorithmic practitioners.

This book provides graduate students, scientific researchers, and engineering technicians a judicious combination and organic whole of the knowledge on matrix analysis and matrix computations. It can be used as a standard textbook as well as a self-study tool and reference. One of the key features of the book is the worked-out exercise section at the end of each chapter.

The first part of the book introduces basics in matrix analysis that are necessary for matrix computations. In this part, we chiefly focus on exploring, probing, and extracting various geometric and algebraic structures and properties of matrices, which are useful for theoretical understanding and practical applications of such matrices. The second part of the book presents representative methods and the corresponding theories in matrix computations. In this part, we mainly concentrate on algorithmically designing and theoretically analyzing fast, accurate, and robust computational methods for various matrix problems, which are often fundamental parts of many areas associated with computational sciences and engineering applications.

This book consists of eight chapters. In Chapter 1, we introduce some basic knowledge in linear algebra, such as matrix norms, canonical forms, and Kronecker product. In Chapter 2, we introduce some useful decompositions of matrices, which play a very important role in matrix computations. Chapter 3 is about the numerical ranges, including the standard numerical range and its generalizations. Chapter 4 is about the nonnegative matrices and the M -matrices; some basic but important properties are discussed in detail. Chapter 5 is for the iteration methods and their convergence results for solving systems of linear equations, which are based on matrix split- tings. Chapter 6 is for the Krylov subspace iteration methods, including the popular ones such as CG, MINRES, GMRES, BiCGSTAB, etc. In Chapter 7, we discuss preconditioning, which is the key ingredient for successful use of the Krylov subspace iteration methods. Finally, in Chap- ter 8, we introduce some popular iteration methods and elementary preconditioning techniques for solving structured linear systems of the saddle-point type, which frequently arise in a wide variety of real-world applications.

This content is essentially based on the lectures, handouts, and manuscripts I delivered in many numerical linear algebra summer schools, short-term courses, and long-term semi- nars since 2000, for graduate students and young scholars, and at a number of Chinese in- stitutions and universities such as Academy of Mathematics and Systems Science of Chinese Academy of Sciences (Beijing: September 2008), Guizhou Normal University (Guiyang: July 2011), Lanzhou University (Lanzhou: May 2000 and July–August 2007), Shanghai University

vii

(8)

·viii· Preface

(Shanghai: December 2013–January 2014), South China Normal University (Guangzhou: July 2006), and Zhejiang University (Hangzhou: July–August 2007). I would like to take this opportunity to thank the main local organizers of these academic activities, including Professors Raymond H. Chan, Chuan-Qing Gu, Zheng-Da Huang, Wen Li, Lin-Zhang Lu, Ping Sun, and Yu-Jiang Wu. I also thank the second author of this book, Professor Jian-Yu Pan, for his efforts in re-typesetting and re-arranging these materials, which yielded part of the initial and workable draft of this book, for me; that is also exactly the only reason that he is my co-author of this book.

Lastly, I thank Professors Kuniyoshi Abe, Ljiljana Cvetkovic, Hua Dai, Walter Gander, Apos- tolos Hadjidimos, Ken Hayami, Yu-Mei Huang, Ilse C.F. Ipsen, Lev A. Krukier, Xin-Guo Liu, Galina V. Muratova, Maya G. Neytcheva, Min Tao, Zeng-Qi Wang, Yu-Jiang Wu, and many other friends and colleagues for providing helpful advice, suggestions, and comments. In partic- ular, I thank Drs. Fang Chen, Tian-Yi Li, Yao-Ning Liu, Kang-Ya Lu, Lu Wang, and Wen-Ting Wu for carefully reading the original version of this book and finding typos and errors in it; and the three reviewers of this book for strongly supporting its publication and expertly suggesting some modifications to its draft version. Last but not least, I would like to extend my thanks to the SIAM (Society for Industrial and Applied Mathematics) publisher, especially the Executive Editor, Elizabeth Greenspan, who made the publication of this book possible, and Mellisa Pas- cale, who patiently and competently dealt with all of my typesetting requests. In addition, I am indebted very much to Cheryl Hufnagle and Ann Manning Allen for their exquisite composition, meticulous proofreading, and elaborate correction.

I would also like to take the opportunity to thank my two lovely sons, Ge-Qi Bai and Ge-Yang Bai, for their understanding and support during a reasonably long period while I wrote this book!

Zhong-Zhi Bai

Zhong-Guan-Cun, Beijing, China June 28, 2020

(Revised and extended on June 30, 2021)

(9)

List of Symbols

O(n) A quantity of order n

brc The floor of r: The nearest integer not less than the real number r dre The ceiling of r: The nearest integer

not larger than the real number r

i Imaginary unit

R Domain of real numbers

C Domain of complex numbers

Rⁿ (or R^n×n) Linear space of all real n-vectors (or n × n-matrices) Cⁿ (or C^n×n) Linear space of all complex n-vectors (or n × n-matrices) x^| (or A^|) Transpose of vector x (or matrix A)

x^∗ (or A^∗) Conjugate transpose of vector x (or matrix A) length(x) Dimension of vector x

<(z) Real part of z ∈ C

=(z) Imaginary part of z ∈ C

hx, yi Inner product of vectors x and y x ⊥ y Vector x is orthogonal to vector y

z Conjugacy of z ∈ C

|z| Absolute value or modulus of z ∈ C

sign(z) Sign function of z ∈ C, which equals 0 if z = 0 and _|z|^z if z 6= 0

kxk_p (or kAkp) p-norm of vector x (or matrix A) kxk_A A-norm or energy norm of vector x a_ij, a_i,j The (i, j)-th entry of the matrix A = [a_ij] A(i, j), [A]ij, [A]i,j The (i, j)-th entry of the matrix A

I The identity matrix

In The identity matrix of dimension n

ei The i-th column vector of the identity matrix I

e The vector of all components equal to 1, i.e., e = [1, 1, . . . , 1]^| H (A), S (A) The Hermitian and the skew-Hermitian part of matrix A diag(·) diag(a11, a22, . . . , amm) is the diagonal matrix

with aiibeing the i-th diagonal entry

Diag(·) Diag(A11, A22, . . . , Amm) is the block-diagonal matrix with Aiibeing the i-th block-diagonal entry

kAkF Frobenius norm of matrix A

ρ(A) Spectral radius of matrix A

det(A) Determinant of matrix A

A⁻¹ Inverse of matrix A

ix

(10)

·x· List of Symbols

PA(λ) Characteristic polynomial of matrix A

tr(A) Trace of matrix A

rank(A) Rank of matrix A

κ(A), κp(A), κF(A) Condition number of matrix A in 2-norm, p-norm, F -norm

λ(A) Any eigenvalue of matrix A

λ_min(A), λmax(A) Largest and smallest eigenvalues of a Hermitian matrix A σ_min(A), σmax(A) Largest and smallest singular values of matrix A

diag(x) Diagonal matrix with the main diagonal elements equal to the elements of the vector x

Ker(A) Null space or kernel of matrix A Ran(A) Range space or range of matrix A A ◦ B Hadamard product of matrices A and B

A ⊕ B Direct sum of matrices A and B

A ⊗ B Kronecker (direct or tensor) product of matrices A and B F (A) Numerical range, or field of values, of matrix A

Fk(A), Fc(A), FC(A) k-numerical range, c-numerical range, C-numerical range of matrix A

r(A), rC(A) Numerical radius, C-numerical radius of matrix A σ(A), σC(A), ρC(A) Spectrum, C-spectrum, C-spectral radius of matrix A

kAkC C-spectral norm of matrix A

span(G) Vector space spanned by the columns of matrix G span{x₁, x₂, . . . , x_n} Vector space spanned by {x1, x₂, . . . , x_n},

where x1, x₂, . . . , x_nare vectors of the same size span{x}^⊥ The vertical space, or the orthogonal complement space,

of the space spanned by vector x

K_m(A, v) Krylov subspace with respect to the matrix A and the vector v, that is,

Km(A, v) = span{v, Av, . . . , A^m−1v}

dim(V ) Dimension of vector space V

Co(S) Convex hull of set S

C(c, r) The disc centered at c with radius r deg(p) The degree of the real polynomial p(t)

Pn The set of all real polynomials p(t) with deg(p) ≤ n E(c, d, a) The ellipse centered at c with the focal distance d

and the semi-major axis a

Matrix Analysis. Computations. and

Matrix Analysis Computations and

Zhong-Zhi Bai

Academy of Mathematics and Systems Science Chinese Academy of Sciences

Beijing, China

Jian-Yu Pan

East China Normal University Shanghai, China

Matrix Analysis Computations and

Contents

Preface

List of Symbols