Matrix Mathematics: Theory, Facts, and Formulas - Second Edition [Second ed.] 9781400833344

Table of contents :
Contents
Preface to the Second Edition
Preface to the First Edition
Special Symbols
Conventions, Notation, and Terminology
Chapter One. Preliminaries
Chapter Two. Basic Matrix Properties
Chapter Three Matrix Classes and Transformations
Chapter Four. Polynomial Matrices and Rational Transfer Functions
Chapter Five. Matrix Decompositions
Chapter Six. Generalized Inverses
Chapter Seven. Kronecker and Schur Algebra
Chapter Eight. Positive-Semidefinite Matrices
Chapter Nine. Norms
Chapter Ten. Functions of Matrices and Their Derivatives
Chapter Eleven. The Matrix Exponential and Stability Theory
Chapter Twelve. Linear Systems and Control Theory
Bibliography
Author Index
Index

Citation preview

Matrix Mathematics

Skew Reflectors

itar

y

itian

Re

fle

cto

rs

Herm

Un

w Ske

O l Ni

t po

en

t

Projectors Positiv

Positive Definite

e Semidefinite Hermitian

Norm

al

er ge H Ran

G

mit

ian

Inve roup

rtib

le

Matrix Mathematics

Theory, Facts, and Formulas

Dennis S. Bernstein

PRINCETON UNIVERSITY PRESS PRINCETON AND OXFORD

c Copyright 2009 by Princeton University Press Published by Princeton University Press, 41 William Street, Princeton, New Jersey 08540 In the United Kingdom: Princeton University Press, 6 Oxford Street, Woodstock, Oxfordshire, 0X20 1TW All Rights Reserved

Library of Congress Cataloging-in-Publication Data Bernstein, Dennis S., 1954– Matrix mathematics: theory, facts, and formulas / Dennis S. Bernstein. – 2nd ed. p. cm. Includes bibliographical references and index. ISBN 978-0-691-13287-7 (hardcover : alk. paper) ISBN 978-0-691-14039-1 (pbk. : alk. paper) 1. Matrices. 2. Linear systems. I. Title. QA188.B475 2008 512.9’434—dc22 2008036257 British Library Cataloging-in-Publication Data is available This book has been composed in Computer Modern and Helvetica. The publisher would like to acknowledge the author of this volume for providing the camera-ready copy from which this book was printed. Printed on acid-free paper. ∞ press.princeton.edu Printed in the United States of America 10

9

8

7

6

5

4

3

2

1

To the memory of my parents, Irma Shorrie (Hirshon) Bernstein and Milton Bernstein, whose love and guidance are everlasting

. . . vessels, unable to contain the great light flowing into them, shatter and break. . . . the remains of the broken vessels fall . . . into the lowest world, where they remain scattered and hidden — D. W. Menzi and Z. Padeh, The Tree of Life, Chayyim Vital’s Introduction to the Kabbalah of Isaac Luria, Jason Aaronson, Northvale, 1999

Thor . . . placed the horn to his lips . . . He drank with all his might and kept drinking as long as ever he was able; when he paused to look, he could see that the level had sunk a little, . . . for the other end lay out in the ocean itself. — P. A. Munch, Norse Mythology, AMS Press, New York, 1970

Contents

Preface to the Second Edition

xv

Preface to the First Edition

xvii

Special Symbols

xxi

Conventions, Notation, and Terminology

xxxiii

1. Preliminaries 1.1 Logic 1.2 Sets 1.3 Integers, Real Numbers, and Complex Numbers 1.4 Functions 1.5 Relations 1.6 Graphs 1.7 Facts on Logic, Sets, Functions, and Relations 1.8 Facts on Graphs 1.9 Facts on Binomial Identities and Sums 1.10 Facts on Convex Functions 1.11 Facts on Scalar Identities and Inequalities in One Variable 1.12 Facts on Scalar Identities and Inequalities in Two Variables 1.13 Facts on Scalar Identities and Inequalities in Three Variables 1.14 Facts on Scalar Identities and Inequalities in Four Variables 1.15 Facts on Scalar Identities and Inequalities in Six Variables 1.16 Facts on Scalar Identities and Inequalities in Eight Variables 1.17 Facts on Scalar Identities and Inequalities in n Variables 1.18 Facts on Scalar Identities and Inequalities in 2n Variables 1.19 Facts on Scalar Identities and Inequalities in 3n Variables 1.20 Facts on Scalar Identities and Inequalities in Complex Variables 1.21 Facts on Trigonometric and Hyperbolic Identities 1.22 Notes

1 1 2 3 4 6 9 11 15 16 23 25 33 42 50 52 52 52 66 74 74 81 84

2. Basic Matrix Properties

85

2.1 2.2 2.3 2.4

Matrix Algebra Transpose and Inner Product Convex Sets, Cones, and Subspaces Range and Null Space

85 92 97 101

x

CONTENTS

2.5 2.6 2.7 2.8 2.9 2.10 2.11 2.12 2.13 2.14 2.15 2.16 2.17 2.18 2.19 2.20 2.21 2.22

Rank and Defect Invertibility The Determinant Partitioned Matrices Facts on Polars, Cones, Dual Cones, Convex Hulls, and Subspaces Facts on Range, Null Space, Rank, and Defect Facts on the Range, Rank, Null Space, and Defect of Partitioned Matrices Facts on the Inner Product, Outer Product, Trace, and Matrix Powers Facts on the Determinant Facts on the Determinant of Partitioned Matrices Facts on Left and Right Inverses Facts on the Adjugate and Inverses Facts on the Inverse of Partitioned Matrices Facts on Commutators Facts on Complex Matrices Facts on Geometry Facts on Majorization Notes

3. Matrix Classes and Transformations 3.1 Matrix Classes 3.2 Matrices Related to Graphs 3.3 Lie Algebras and Groups 3.4 Matrix Transformations 3.5 Projectors, Idempotent Matrices, and Subspaces 3.6 Facts on Group-Invertible and Range-Hermitian Matrices 3.7 Facts on Normal, Hermitian, and Skew-Hermitian Matrices 3.8 Facts on Commutators 3.9 Facts on Linear Interpolation 3.10 Facts on the Cross Product 3.11 Facts on Unitary and Shifted-Unitary Matrices 3.12 Facts on Idempotent Matrices 3.13 Facts on Projectors 3.14 Facts on Reflectors 3.15 Facts on Involutory Matrices 3.16 Facts on Tripotent Matrices 3.17 Facts on Nilpotent Matrices 3.18 Facts on Hankel and Toeplitz Matrices 3.19 Facts on Tridiagonal Matrices 3.20 Facts on Hamiltonian and Symplectic Matrices 3.21 Facts on Matrices Related to Graphs 3.22 Facts on Triangular, Irreducible, Cauchy, Dissipative, Contractive, and Centrosymmetric Matrices 3.23 Facts on Groups 3.24 Facts on Quaternions 3.25 Notes

104 106 111 115 119 124 130 136 139 144 152 153 159 161 164 167 175 178 179 179 184 185 188 190 191 192 199 200 202 205 215 223 229 230 231 232 234 237 238 240 240 242 247 252

CONTENTS

4. Polynomial Matrices and Rational Transfer Functions 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10 4.11 4.12

Polynomials Polynomial Matrices The Smith Decomposition and Similarity Invariants Eigenvalues Eigenvectors The Minimal Polynomial Rational Transfer Functions and the Smith-McMillan Decomposition Facts on Polynomials and Rational Functions Facts on the Characteristic and Minimal Polynomials Facts on the Spectrum Facts on Graphs and Nonnegative Matrices Notes

5. Matrix Decompositions 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 5.11 5.12 5.13 5.14 5.15 5.16 5.17 5.18 5.19 5.20

Smith Form Multicompanion Form Hypercompanion Form and Jordan Form Schur Decomposition Eigenstructure Properties Singular Value Decomposition Pencils and the Kronecker Canonical Form Facts on the Inertia Facts on Matrix Transformations for One Matrix Facts on Matrix Transformations for Two or More Matrices Facts on Eigenvalues and Singular Values for One Matrix Facts on Eigenvalues and Singular Values for Two or More Matrices Facts on Matrix Pencils Facts on Matrix Eigenstructure Facts on Matrix Factorizations Facts on Companion, Vandermonde, Circulant, and Hadamard Matrices Facts on Simultaneous Transformations Facts on the Polar Decomposition Facts on Additive Decompositions Notes

6. Generalized Inverses 6.1 6.2 6.3 6.4 6.5 6.6 6.7

Moore-Penrose Generalized Inverse Drazin Generalized Inverse Facts on the Moore-Penrose Generalized Inverse for One Matrix Facts on the Moore-Penrose Generalized Inverse for Two or More Matrices Facts on the Moore-Penrose Generalized Inverse for Partitioned Matrices Facts on the Drazin and Group Generalized Inverses Notes

xi 253 253 256 258 261 267 269 271 276 282 288 297 307 309 309 309 314 318 321 328 330 334 338 345 350 362 369 369 377 385 391 393 394 396 397 397 401 404 411 422 431 438

xii

CONTENTS

7. Kronecker and Schur Algebra 7.1 7.2 7.3 7.4 7.5 7.6 7.7

Kronecker Product Kronecker Sum and Linear Matrix Equations Schur Product Facts on the Kronecker Product Facts on the Kronecker Sum Facts on the Schur Product Notes

8. Positive-Semidefinite Matrices 8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8 8.9 8.10 8.11 8.12 8.13 8.14 8.15 8.16 8.17 8.18 8.19 8.20 8.21 8.22 8.23

Positive-Semidefinite and Positive-Definite Orderings Submatrices Simultaneous Diagonalization Eigenvalue Inequalities Exponential, Square Root, and Logarithm of Hermitian Matrices Matrix Inequalities Facts on Range and Rank Facts on Structured Positive-Semidefinite Matrices Facts on Identities and Inequalities for One Matrix Facts on Identities and Inequalities for Two or More Matrices Facts on Identities and Inequalities for Partitioned Matrices Facts on the Trace Facts on the Determinant Facts on Convex Sets and Convex Functions Facts on Quadratic Forms Facts on the Gaussian Density Facts on Simultaneous Diagonalization Facts on Eigenvalues and Singular Values for One Matrix Facts on Eigenvalues and Singular Values for Two or More Matrices Facts on Alternative Partial Orderings Facts on Generalized Inverses Facts on the Kronecker and Schur Products Notes

9. Norms 9.1 9.2 9.3 9.4 9.5 9.6 9.7 9.8 9.9 9.10 9.11 9.12 9.13

Vector Norms Matrix Norms Compatible Norms Induced Norms Induced Lower Bound Singular Value Inequalities Facts on Vector Norms Facts on Matrix Norms for One Matrix Facts on Matrix Norms for Two or More Matrices Facts on Matrix Norms for Partitioned Matrices Facts on Matrix Norms and Eigenvalues for One Matrix Facts on Matrix Norms and Eigenvalues for Two or More Matrices Facts on Matrix Norms and Singular Values for One Matrix

439 439 443 444 445 450 454 458 459 459 461 465 467 473 474 486 488 495 501 514 523 533 543 550 556 558 559 564 574 577 584 595 597 597 601 604 607 613 615 618 627 636 649 653 656 659

CONTENTS

9.14 9.15 9.16

Facts on Matrix Norms and Singular Values for Two or More Matrices Facts on Linear Equations and Least Squares Notes

10. Functions of Matrices and Their Derivatives 10.1 10.2 10.3 10.4 10.5 10.6 10.7 10.8 10.9 10.10 10.11 10.12 10.13 10.14

Open Sets and Closed Sets Limits Continuity Derivatives Functions of a Matrix Matrix Square Root and Matrix Sign Functions Matrix Derivatives Facts on One Set Facts on Two or More Sets Facts on Matrix Functions Facts on Functions Facts on Derivatives Facts on Infinite Series Notes

11. The Matrix Exponential and Stability Theory 11.1 11.2 11.3 11.4 11.5 11.6 11.7 11.8 11.9 11.10 11.11 11.12 11.13 11.14 11.15 11.16 11.17 11.18 11.19 11.20 11.21 11.22 11.23 11.24

Definition of the Matrix Exponential Structure of the Matrix Exponential Explicit Expressions Matrix Logarithms Principal Logarithm Lie Groups Lyapunov Stability Theory Linear Stability Theory The Lyapunov Equation Discrete-Time Stability Theory Facts on Matrix Exponential Formulas Facts on the Matrix Sine and Cosine Facts on the Matrix Exponential for One Matrix Facts on the Matrix Exponential for Two or More Matrices Facts on the Matrix Exponential and Eigenvalues, Singular Values, and Norms for One Matrix Facts on the Matrix Exponential and Eigenvalues, Singular Values, and Norms for Two or More Matrices Facts on Stable Polynomials Facts on Stable Matrices Facts on Almost Nonnegative Matrices Facts on Discrete-Time-Stable Polynomials Facts on Discrete-Time-Stable Matrices Facts on Lie Groups Facts on Subspace Decomposition Notes

xiii

665 676 680 681 681 682 684 685 688 690 690 693 695 698 699 701 704 705 707 707 710 715 718 720 722 725 726 730 734 736 742 743 746 756 759 763 766 774 777 782 786 786 793

xiv

CONTENTS

12. Linear Systems and Control Theory 12.1 12.2 12.3 12.4 12.5 12.6 12.7 12.8 12.9 12.10 12.11 12.12 12.13 12.14 12.15 12.16 12.17 12.18 12.19 12.20 12.21 12.22 12.23 12.24

State Space and Transfer Function Models Laplace Transform Analysis The Unobservable Subspace and Observability Observable Asymptotic Stability Detectability The Controllable Subspace and Controllability Controllable Asymptotic Stability Stabilizability Realization Theory Zeros H2 System Norm Harmonic Steady-State Response System Interconnections Standard Control Problem Linear-Quadratic Control Solutions of the Riccati Equation The Stabilizing Solution of the Riccati Equation The Maximal Solution of the Riccati Equation Positive-Semidefinite and Positive-Definite Solutions of the Riccati Equation Facts on Stability, Observability, and Controllability Facts on the Lyapunov Equation and Inertia Facts on Realizations and the H2 System Norm Facts on the Riccati Equation Notes

795 795 798 800 805 807 808 816 820 822 830 838 841 842 845 847 850 855 859 862 863 866 872 875 879

Bibliography

881

Author Index

967

Index

979

Preface to the Second Edition

This second edition of Matrix Mathematics represents a major expansion of the original work. While the total number of pages is increased 57% from 752 to 1181, the increase is actually greater since this edition is typeset in a smaller font to facilitate a manageable physical size. The second edition expands on the first edition in several ways. For example, the new version includes material on graphs (developed within the framework of relations and partially ordered sets), as well as alternative partial orderings of matrices, such as rank subtractivity, star, and generalized L¨owner. This edition also includes additional material on the Kronecker canonical form and matrix pencils; matrix representations of finite groups; zeros of multi-input, multi-output transfer functions; equalities and inequalities for real and complex numbers; bounds on the roots of polynomials; convex functions; and vector and matrix norms. The additional material as well as works published subsequent to the first edition increased the number of cited works from 820 to 1540, an increase of 87%. To increase the utility of the bibliography, this edition uses the “back reference” feature of LATEX, which indicates where each reference is cited in the text. As in the first edition, the second edition includes an author index. The expansion of the first edition resulted in an increase in the size of the index from 108 pages to 161 pages. The first edition included 57 problems, while the current edition has 74. These problems represent extensions or generalizations of known results, sometimes motivated by gaps in the literature. In this edition, I have attempted to correct all errors that appeared in the first edition. As with the first edition, readers are encouraged to contact me about errors or omissions in the current edition, which I will periodically update on my home page.

Acknowledgments I am grateful to many individuals who kindly provided advice and material for this edition. Some readers alerted me to errors, while others suggested additional material. In other cases I sought out researchers to help me understand the precise nature of interesting results. At the risk of omitting those who were helpful, I am pleased to acknowledge the following: Mark Balas, Jason Bernstein, Sanjay Bhat, Gerald Bourgeois, Adam Brzezinski, Francesco Bullo, Vijay

xvi

PREFACE TO THE SECOND EDITION

Chellaboina, Naveena Crasta, Anthony D’Amato, Sever Dragomir, Bojana Drincic, Harry Dym, Matthew Fledderjohn, Haoyun Fu, Masatoshi Fujii, Takayumi Furuta, Steven Gillijns, Rishi Graham, Wassim Haddad, Nicholas Higham, Diederich Hinrichsen, Matthew Holzel, Qing Hui, Masatoshi Ito, Iman Izadi, Pierre Kabamba, Marthe Kassouf, Christopher King, Siddharth Kirtikar, Michael Margliot, Roy Mathias, Peter Mercer, Alex Olshevsky, Paul Otanez, Bela Palancz, Harish Palanthandalam-Madapusi, Fotios Paliogiannis, Isaiah Pantelis, Wei Ren, Ricardo Sanfelice, Mario Santillo, Amit Sanyal, Christoph Schmoeger, Demetrios Serakos, Wasin So, Robert Sullivan, Dogan Sumer, Yongge Tian, G¨ otz Trenkler, Panagiotis Tsiotras, Takeaki Yamazaki, Jin Yan, Masahiro Yanagida, Vera Zeidan, Chenwei Zhang, Fuzhen Zhang, and Qing-Chang Zhong. As with the first edition, I am especially indebted to my family, who endured four more years of my consistent absence to make this revision a reality. It is clear that any attempt to fully embrace the enormous body of mathematics known as matrix theory is a neverending task. After devoting more than two decades to this project of reassembling the scattered shards, I remain, like Thor, barely able to perceive a dent in the vast knowledge that resides in the hundreds of thousands of pages devoted to this fascinating and incredibly useful subject. Yet, it is my hope that this book will prove to be valuable to everyone who uses matrices, and will inspire interest in a mathematical construction whose secrets and mysteries have no bounds. Dennis S. Bernstein Ann Arbor, Michigan [email protected] March 2009

Preface to the First Edition

The idea for this book began with the realization that at the heart of the solution to many problems in science, mathematics, and engineering often lies a “matrix fact,” that is, an identity, inequality, or property of matrices that is crucial to the solution of the problem. Although there are numerous excellent books on linear algebra and matrix theory, no one book contains all or even most of the vast number of matrix facts that appear throughout the scientific, mathematical, and engineering literature. This book is an attempt to organize many of these facts into a reference source for users of matrix theory in diverse applications areas. Viewed as an extension of scalar mathematics, matrix mathematics provides the means to manipulate and analyze multidimensional quantities. Matrix mathematics thus provides powerful tools for a broad range of problems in science and engineering. For example, the matrix-based analysis of systems of ordinary differential equations accounts for interaction among all of the state variables. The discretization of partial differential equations by means of finite differences and finite elements yields linear algebraic or differential equations whose matrix structure reflects the nature of physical solutions [1269]. Multivariate probability theory and statistical analysis use matrix methods to represent probability distributions, to compute moments, and to perform linear regression for data analysis [517, 621, 671, 720, 972, 1212]. The study of linear differential equations [709, 710, 746] depends heavily on matrix analysis, while linear systems and control theory are matrix-intensive areas of engineering [3, 68, 146, 150, 319, 321, 356, 379, 381, 456, 515, 631, 764, 877, 890, 960, 1121, 1174, 1182, 1228, 1232, 1243, 1368, 1402, 1490, 1535]. In addition, matrices are widely used in rigid body dynamics [28, 745, 753, 811, 829, 874, 995, 1053, 1095, 1096, 1216, 1231, 1253, 1384], structural mechanics [888, 1015, 1127], computational fluid dynamics [313, 492, 1460], circuit theory [32], queuing and stochastic systems [659, 944, 1061], econometrics [413, 973, 1146], geodesy [1272], game theory [229, 924, 1264], computer graphics [65, 511], computer vision [966], optimization [259, 382, 978], signal processing [720, 1193, 1395], classical and quantum information theory [361, 720, 1069, 1113], communications systems [800, 801], statistics [594, 671, 973, 1146, 1208], statistical mechanics [18, 163, 164, 1406], demography [305, 828], combinatorics, networks, and graph theory [132, 169, 183, 227, 239, 270, 272, 275, 310, 311, 343, 282, 371, 415, 438, 494, 514, 571, 616, 654, 720, 868, 945, 956, 1172, 1421], optics [563, 677, 820], dimensional analysis [658, 1283], and number theory [865].

xviii

PREFACE TO THE FIRST EDITION

In all applications involving matrices, computational techniques are essential for obtaining numerical solutions. The development of efficient and reliable algorithms for matrix computations is therefore an important area of research that has been extensively developed [98, 312, 404, 583, 699, 701, 740, 774, 1255, 1256, 1258, 1260, 1347, 1403, 1461, 1465, 1467, 1513]. To facilitate the solution of matrix problems, entire computer packages have been developed using the language of matrices. However, this book is concerned with the analytical properties of matrices rather than their computational aspects. This book encompasses a broad range of fundamental questions in matrix theory, which, in many cases can be viewed as extensions of related questions in scalar mathematics. A few such questions follow. What are the basic properties of matrices? characterized, classified, and quantified?

How can matrices be

How can a matrix be decomposed into simpler matrices? A matrix decomposition may involve addition, multiplication, and partition. Decomposing a matrix into its fundamental components provides insight into its algebraic and geometric properties. For example, the polar decomposition states that every square matrix can be written as the product of a rotation and a dilation analogous to the polar representation of a complex number. Given a pair of matrices having certain properties, what can be inferred about the sum, product, and concatenation of these matrices? In particular, if a matrix has a given property, to what extent does that property change or remain unchanged if the matrix is perturbed by another matrix of a certain type by means of addition, multiplication, or concatenation? For example, if a matrix is nonsingular, how large can an additive perturbation to that matrix be without the sum becoming singular? How can properties of a matrix be determined by means of simple operations? For example, how can the location of the eigenvalues of a matrix be estimated directly in terms of the entries of the matrix? To what extent do matrices satisfy the formal properties of the real numbers? For example, while 0 ≤ a ≤ b implies that ar ≤ br for real numbers a, b and a positive integer r, when does 0 ≤ A ≤ B imply Ar ≤ B r for positive-semidefinite matrices A and B and with the positive-semidefinite ordering? Questions of these types have occupied matrix theorists for at least a century, with motivation from diverse applications. The existing scope and depth of knowledge are enormous. Taken together, this body of knowledge provides a powerful framework for developing and analyzing models for scientific and engineering applications.

PREFACE TO THE FIRST EDITION

xix

This book is intended to be useful to at least four groups of readers. Since linear algebra is a standard course in the mathematical sciences and engineering, graduate students in these fields can use this book to expand the scope of their linear algebra text. For instructors, many of the facts can be used as exercises to augment standard material in matrix courses. For researchers in the mathematical sciences, including statistics, physics, and engineering, this book can be used as a general reference on matrix theory. Finally, for users of matrices in the applied sciences, this book will provide access to a large body of results in matrix theory. By collecting these results in a single source, it is my hope that this book will prove to be convenient and useful for a broad range of applications. The material in this book is thus intended to complement the large number of classical and modern texts and reference works on linear algebra and matrix theory [11, 384, 516, 554, 555, 572, 600, 719, 812, 897, 964, 981, 988, 1033, 1072, 1078, 1125, 1172, 1225, 1269]. After a review of mathematical preliminaries in Chapter 1, fundamental properties of matrices are described in Chapter 2. Chapter 3 summarizes the major classes of matrices and various matrix transformations. In Chapter 4 we turn to polynomial and rational matrices whose basic properties are essential for understanding the structure of constant matrices. Chapter 5 is concerned with various decompositions of matrices including the Jordan, Schur, and singular value decompositions. Chapter 6 provides a brief treatment of generalized inverses, while Chapter 7 describes the Kronecker and Schur product operations. Chapter 8 is concerned with the properties of positive-semidefinite matrices. A detailed treatment of vector and matrix norms is given in Chapter 9, while formulas for matrix derivatives are given in Chapter 10. Next, Chapter 11 focuses on the matrix exponential and stability theory, which are central to the study of linear differential equations. In Chapter 12 we apply matrix theory to the analysis of linear systems, their state space realizations, and their transfer function representation. This chapter also includes a discussion of the matrix Riccati equation of control theory. Each chapter provides a core of results with, in many cases, complete proofs. Sections at the end of each chapter provide a collection of Facts organized to correspond to the order of topics in the chapter. These Facts include corollaries and special cases of results presented in the chapter, as well as related results that go beyond the results of the chapter. In some cases the Facts include open problems, illuminating remarks, and hints regarding proofs. The Facts are intended to provide the reader with a useful reference collection of matrix results as well as a gateway to the matrix theory literature.

Acknowledgments The writing of this book spanned more than a decade and a half, during which time numerous individuals contributed both directly and indirectly. I am grateful for the helpful comments of many people who contributed technical material and insightful suggestions, all of which greatly improved the presentation and content of the book. In addition, numerous individuals generously agreed to read sections or chapters of the book for clarity and accuracy. I wish to thank Jasim Ahmed, Suhail Akhtar, David Bayard, Sanjay Bhat, Tony Bloch, Peter Bullen, Steve Campbell, Agostino Capponi, Ramu Chandra, Jaganath Chandrasekhar, Nalin Chaturvedi, Vijay Chellaboina, Jie Chen, David Clements, Dan Davison,

xx

PREFACE TO THE FIRST EDITION

Dimitris Dimogianopoulos, Jiu Ding, D. Z. Djokovic, R. Scott Erwin, R. W. Farebrother, Danny Georgiev, Joseph Grcar, Wassim Haddad, Yoram Halevi, Jesse Hoagg, Roger Horn, David Hyland, Iman Izadi, Pierre Kabamba, Vikram Kapila, Fuad Kittaneh, Seth Lacy, Thomas Laffey, Cedric Langbort, Alan Laub, Alexander Leonessa, Kai-Yew Lum, Pertti Makila, Roy Mathias, N. Harris McClamroch, Boris Mordukhovich, Sergei Nersesov, JinHyoung Oh, Concetta Pilotto, Harish Palanthandalum-Madapusi, Michael Piovoso, Leiba Rodman, Phil Roe, Carsten Scherer, Wasin So, Andy Sparks, Edward Tate, Yongge Tian, Panagiotis Tsiotras, Feng Tyan, Ravi Venugopal, Jan Willems, Hong Wong, Vera Zeidan, Xingzhi Zhan, and Fuzhen Zhang for their assistance. Nevertheless, I take full responsibility for any remaining errors, and I encourage readers to alert me to any mistakes, corrections of which will be posted on the web. Solutions to the open problems are also welcome. Portions of the manuscript were typed by Jill Straehla and Linda Smith at Harris Corporation, and by Debbie Laird, Kathy Stolaruk, and Suzanne Smith at the University of Michigan. John Rogosich of Techsetters, Inc., provided invaluable assistance with LATEX issues, and Jennifer Slater carefully copyedited the entire manuscript. I also thank JinHyoung Oh and Joshua Kang for writing C code to refine the index. I especially thank Vickie Kearn of Princeton University Press for her wise guidance and constant encouragement. Vickie managed to address all of my concerns and anxieties, and helped me improve the manuscript in many ways. Finally, I extend my greatest appreciation for the (uncountably) infinite patience of my family, who endured the days, weeks, months, and years that this project consumed. The writing of this book began with toddlers and ended with a teenager and a twenty-year old. We can all be thankful it is finally finished. Dennis S. Bernstein Ann Arbor, Michigan [email protected] January 2005

Special Symbols

General Notation π

3.14159 . . .

e

2.71828 . . .



=

equals by definition

limε↓0 α

limit from the right

m

α(α−1)···(α−m+1) m!

m

n! m!(n−m)!

n a

largest integer less than or equal to a

δij

1 if i = j, 0 if i = j (Kronecker delta)

log

logarithm with base e

sign α

1 if α > 0, −1 if α < 0, 0 if α = 0

Chapter 1 {}

set (p. 2)

∈

is an element of (p. 2)

∈

is not an element of (p. 2)

∅

empty set (p. 2)

{ }ms

multiset (p. 2)

card

cardinality (p. 2)

∩

intersection (p. 2)

∪

union (p. 2)

Y\X X

∼

complement of X relative to Y (p. 2) complement of X (p. 3)

xxii

SPECIAL SYMBOLS

⊆

is a subset of (p. 3)

⊂

is a proper subset of (p. 3)

(x1, . . . , xn )

tuple or n-tuple (p. 3)

Z

integers (p. 3)

N

nonnegative integers (p. 3)

P

positive integers (p. 3)

R

real numbers (p. 3)

C j

complex numbers (p. 3) √ −1 (p. 3)

z

complex conjugate of z ∈ C (p. 4)

Re z

real part of z ∈ C (p. 4)

Im z

imaginary part of z ∈ C (p. 4)

|z|

absolute value of z ∈ C (p. 4)

OLHP

open left half plane in C (p. 4)

CLHP

closed left half plane in C (p. 4)

ORHP

open right half plane in C (p. 4)

CRHP

closed right half plane in C (p. 4)

jR

imaginary numbers (p. 4)

OUD

open unit disk in C (p. 4)

CUD

closed unit disk in C (p. 4)

CPP

closed punctured plane in C (p. 4)

OPP

open punctured plane in C (p. 4)

F

R or C (p. 4)

f: X → Y

f is a function with domain X and codomain Y (p. 4)

Graph(f )

{(x, f(x)): x ∈ X} (p. 4)

f •g

composition of functions f and g (p. 4)

f−1(S)

inverse image of S (p. 5)

rev(R)

reversal of the relation R (p. 7)

R∼

complement of the relation R (p. 7)

ref(R)

reflexive hull of the relation R (p. 7)

sym(R)

symmetric hull of the relation R (p. 7)

trans(R)

transitive hull of the relation R (p. 7)

xxiii

SPECIAL SYMBOLS

equiv(R) R

equivalence hull of the relation R (p. 7)

x=y

(x, y) is an element of the equivalence relation R (p. 7)

glb(S)

greatest lower bound of S (p. 8, Definition 1.5.9)

lub(S)

least upper bound of S (p. 8, Definition 1.5.9)

inf(S)

infimum of S (p. 9, Definition 1.5.9)

sup(S)

supremum of S (p. 9, Definition 1.5.9)

rev(G)

reversal of the graph G (p. 9)

∼

G

complement of the graph G (p. 9)

ref(G)

reflexive hull of the graph G (p. 9)

sym(G)

symmetric hull of the graph G (p. 9)

trans(G)

transitive hull of the graph G (p. 9)

equiv(G)

equivalence hull of the graph G (p. 9)

indeg(x)

indegree of the node x (p. 10)

outdeg(x)

outdegree of the node x (p. 10)

deg(x)

degree of the node x (p. 10)

Chapter 2 Rn

Rn×1 (real column vectors) (p. 85)

Cn

Cn×1 (complex column vectors) (p. 85)

Fn

Rn or Cn (p. 85)

x(i)

ith component of x ∈ Fn (p. 85)

x ≥≥ y

x(i) ≥ y(i) for all i (x − y is nonnegative) (p. 86)

x >> y

x(i) > y(i) for all i (x − y is positive) (p. 86)

R

n × m real matrices (p. 86)

n×m

Cn×m

n × m complex matrices (p. 86)

F

Rn×m or Cn×m (p. 86)

n×m

rowi(A)

ith row of A (p. 87)

coli(A)

ith column of A (p. 87)

A(i,j)

(i, j) entry of A (p. 87)

xxiv

SPECIAL SYMBOLS

i

A←b

matrix obtained from A ∈ Fn×m by replacing coli(A) with b ∈ Fn or rowi(A) with b ∈ F1×m (p. 87) 

dmax(A) = d1(A)

largest diagonal entry of A ∈ Fn×n having real diagonal entries (p. 87)

di(A)

ith largest diagonal entry of A ∈ Fn×n having real diagonal entries (p. 87) 

dmin(A) = dn(A)

smallest diagonal entry of A ∈ Fn×n having real diagonal entries (p. 87)

A(S1,S2 )

submatrix of A formed by retaining the rows of A listed in S1 and the columns of A listed in S2 (p. 88)

A(S)

A(S,S) (p. 88)

A ≥≥ B

A(i,j) ≥ B(i,j) for all i, j (A − B is nonnegative) (p. 88)

A >> B

A(i,j) > B(i,j) for all i, j (A − B is positive) (p. 88)

[A, B]

commutator AB − BA (p. 89)

adA(X)

adjoint operator [A, X] (p. 89)

x×y

cross product of vectors x, y ∈ R3 (p. 89)

K(x)

cross-product matrix for x ∈ R3 (p. 90)

0n×m , 0

n × m zero matrix (p. 90)

In , I

n × n identity matrix (p. 91)

Iˆn , Iˆ

n × n reverse permutation matrix  0 1 .. 1

.

(p. 91) 0

Pn

n × n cyclic permutation matrix (p. 91)

Nn , N

n × n standard nilpotent matrix (p. 92)

ei,n , ei

coli(In ) (p. 92)

Ei,j,n×m , Ei,j

ei,n eT j,m (p. 92)

1n×m

n × m ones matrix (p. 92)

T

A

transpose of A (p. 94)

tr A

trace of A (p. 94)

C

complex conjugate of C ∈ Cn×m (p. 95)

A∗

A conjugate transpose of A (p. 95)

Re A

real part of A ∈ Fn×m (p. 95)

T

xxv

SPECIAL SYMBOLS

Im A

imaginary part of A ∈ Fn×m (p. 95)

S

{Z : Z ∈ S} or {Z : Z ∈ S}ms (p. 95) ˆ

AT

ˆ TIˆ reverse transpose of A (p. 96) IA

A∗ˆ

ˆ ∗Iˆ reverse complex conjugate transpose of A IA (p. 96)

|x|

absolute value of x ∈ Fn (p. 96)

|A|

absolute value of A ∈ Fn×n (p. 96)

sign x

sign of x ∈ Rn (p. 97)

sign A

sign of A ∈ Rn×n (p. 97)

co S

convex hull of S (p. 98)

cone S

conical hull of S (p. 98)

coco S

convex conical hull of S (p. 98)

span S

span of S (p. 98)

aff S

affine hull of S (p. 98)

dim S

dimension of S (p. 98)

S⊥

orthogonal complement of S (p. 99)

polar S

polar of S (p. 99)

dcone S

dual cone of S (p. 99)

R(A)

range of A (p. 101)

N(A)

null space of A (p. 102)

rank A

rank of A (p. 104)

def A

defect of A (p. 104)

L

A

left inverse of A (p. 106)

AR

right inverse of A (p. 106)

−1

A−T

inverse of A (p. 110)  T −1 A (p. 111)

A−∗

(A∗ )−1 (p. 111)

det A

determinant of A (p. 112)

A[i;j]

submatrix A({i}∼ ,{j}∼ ) of A obtained by deleting rowi(A) and colj (A) (p. 114)

AA

adjugate of A (p. 114)

A

rs

A≤B

rank subtractivity partial ordering (p. 129, Fact 2.10.32)

xxvi ∗

A≤B

SPECIAL SYMBOLS

star partial ordering (p. 130, Fact 2.10.35)

Chapter 3 

a1

0 ..

diag(a1, . . . , an )

an

0

a1

revdiag(a1, . . . , an )

.



..

(p. 181) 0

an

diag(A1, . . . , Ak)

(p. 181)

.

0





⎡

⎢ block-diagonal matrix ⎣

A1

0 ..

.

0

Ai ∈ F J2n , J



ni ×mi

0 In −In 0

⎤ ⎥ ⎦, where

Ak

(p. 181)

(p. 183)

glF (n), plC (n), slF (n), u(n), su(n), so(n), sympF (2n), osympF (2n), affF (n), seF (n), transF (n)

Lie algebras (p. 185)

S1 ≈ S2

the groups S1 and S2 are isomorphic (p. 186)

GLF (n), PLF (n), SLF (n), U(n), O(n), U(n, m), O(n, m), SU(n), SO(n), P(n), A(n), D(n), C(n), SympF (2n), OSympF (2n), AffF (n), SEF (n), TransF (n)

groups (p. 187)

A⊥

complementary idempotent matrix or projector I − A corresponding to the idempotent matrix or projector A (p. 190)

ind A

index of A (p. 190)

H

quaternions (p. 247, Fact 3.24.1)

Sp(n)

symplectic group in H (p. 249, Fact 3.24.4)

Chapter 4 F[s]

polynomials with coefficients in F (p. 253)

deg p

degree of p ∈ F[s] (p. 253)

mroots(p)

multiset of roots of p ∈ F[s] (p. 254)

xxvii

SPECIAL SYMBOLS

roots(p)

set of roots of p ∈ F[s] (p. 254)

multp (λ)

multiplicity of λ as a root of p ∈ F[s] (p. 254)

Fn×m [s]

n × m matrices with entries in F[s] (n × m polynomial matrices with coefficients in F) (p. 256)

rank P

rank of P ∈ Fn×m [s] (p. 257)

Szeros(P )

set of Smith zeros of P ∈ Fn×m [s] (p. 259)

mSzeros(P )

multiset of Smith zeros of P ∈ Fn×m [s] (p. 259)

χA

characteristic polynomial of A (p. 262) 

λmax(A) = λ1(A)

largest eigenvalue of A ∈ Fn×n having real eigenvalues (p. 262)

λi (A)

ith largest eigenvalue of A ∈ Fn×n having real eigenvalues (p. 262) 

λmin(A) = λn(A)

smallest eigenvalue of A ∈ Fn×n having real eigenvalues (p. 262)

amultA(λ)

algebraic multiplicity of λ ∈ spec(A) (p. 262)

spec(A)

spectrum of A (p. 262)

mspec(A)

multispectrum of A (p. 262)

gmultA(λ)

geometric multiplicity of λ ∈ spec(A) (p. 267)

spabs(A)

spectral abscissa of A (p. 267)

sprad(A)

spectral radius of A (p. 267)

ν−(A), ν0 (A), ν+(A)

number of eigenvalues of A counting algebraic multiplicity having negative, zero, and positive real part, respectively (p. 267)

In A

inertia of A, that is, [ν−(A) ν0 (A) ν+(A)]T (p. 267)

sig A

signature of A, that is, ν+(A) − ν−(A) (p. 267)

μA

minimal polynomial of A (p. 269)

F(s)

rational functions with coefficients in F (SISO rational transfer functions) (p. 271)

Fprop(s)

proper rational functions with coefficients in F (SISO proper rational transfer functions) (p. 271)

reldeg g

relative degree of g ∈ Fprop(s) (p. 271)

Fn×m(s)

n × m matrices with entries in F(s) (MIMO rational transfer functions) (p. 271)

xxviii

SPECIAL SYMBOLS

Fn×m prop (s)

n × m matrices with entries in Fprop(s) (MIMO proper rational transfer functions) (p. 271)

reldeg G

relative degree of G ∈ Fn×m prop (s) (p. 271)

rank G

rank of G ∈ Fn×m(s) (p. 271)

poles(G)

set of poles of G ∈ Fn×m(s) (p. 271)

bzeros(G)

set of blocking zeros of G ∈ Fn×m(s) (p. 271)

McdegG

McMillan degree of G ∈ Fn×m(s) (p. 273)

tzeros(G)

set of transmission zeros of G ∈ Fn×m(s) (p. 273)

mpoles(G)

multiset of poles of G ∈ Fn×m(s) (p. 273)

mtzeros(G)

multiset of transmission zeros of G ∈ Fn×m(s) (p. 273)

mbzeros(G)

multiset of blocking zeros of G ∈ Fn×m(s) (p. 273)

B(p, q)

Bezout matrix of p, q ∈ F[s] (p. 277, Fact 4.8.6)

H(g)

Hankel matrix of g ∈ F(s) (p. 279, Fact 4.8.8)

Chapter 5 C(p)

companion matrix for monic polynomial p (p. 309)

Hl(q)

l × l or 2l × 2l hypercompanion matrix (p. 314)

Jl(q)

l × l or 2l × 2l real Jordan matrix (p. 315)

indA(λ)

index of λ with respect to A (p. 321)

σi (A)

ith largest singular value of A ∈ Fn×m (p. 328) 

σmax(A) = σ1(A) 

largest singular value of A ∈ Fn×m (p. 328)

σmin(A) = σn(A)

minimum singular value of a square matrix A ∈ Fn×n (p. 328)

PA,B

pencil of (A, B), where A, B ∈ Fn×n (p. 330)

spec(A, B)

generalized spectrum of (A, B), where A, B ∈ Fn×n (p. 330)

mspec(A, B)

generalized multispectrum of (A, B), where A, B ∈ Fn×n (p. 330)

χA,B

characteristic polynomial of (A, B), where A, B ∈ Fn×n (p. 332)

V (λ1, . . . , λn )

Vandermonde matrix (p. 387, Fact 5.16.1)

SPECIAL SYMBOLS

xxix

circ(a0 , . . . , an−1 )

circulant matrix of a0 , . . . , an−1 ∈ F (p. 388, Fact 5.16.7)

Chapter 6 A+

(Moore-Penrose) generalized inverse of A (p. 397)

D|A

Schur complement of D with respect to A (p. 401)

AD

Drazin generalized inverse of A (p. 401)

#

A

group generalized inverse of A (p. 403)

Chapter 7 vec A

vector formed by stacking columns of A (p. 439)

⊗

Kronecker product (p. 440)

Pn,m

Kronecker permutation matrix (p. 442)

⊕

Kronecker sum (p. 443)

A ◦B

Schur product of A and B (p. 444) α  Schur power of A, (A◦α )(i,j) = A(i,j) (p. 444)

A◦α

Chapter 8 Hn N

n

n × n Hermitian matrices (p. 459) n × n positive-semidefinite matrices (p. 459)

Pn

n × n positive-definite matrices (p. 459)

A≥B

A − B ∈ Nn (p. 459)

A>B

A − B ∈ Pn (p. 459)

A

(A∗A)1/2 (p. 474)

A#B

geometric mean of A and B (p. 508, Fact 8.10.43)

A#α B

generalized geometric mean of A and B (p. 510, Fact 8.10.45)

A :B

parallel sum of A and B (p. 581, Fact 8.21.18)

sh(A, B)

shorted operator (p. 582, Fact 8.21.19)

xxx

SPECIAL SYMBOLS

Chapter 9  xp

n 

H¨older norm

1/p |x(i) |p

(p. 598)

i=1

 Ap

H¨older norm

n,m 

1/p |A(i,j) |p

(p. 601)

i,j=1

AF

Frobenius norm

Aσp

Schatten norm

√ tr A∗A (p. 601)

rank A  i=1

1/p σip(A) (p. 602)

Aq,p

H¨older-induced norm (p. 608)

Acol

column norm A1,1 = maxi∈{1,...,m} coli(A)1 (p. 611)

Arow

row norm A∞,∞ = maxi∈{1,...,n} rowi(A)1 (p. 611)

(A)

induced lower bound of A (p. 613)

q,p (A)

H¨ older-induced lower bound of A (p. 614)

 · D

dual norm (p. 625, Fact 9.7.22)

Chapter 10 Bε (x)

open ball of radius ε centered at x (p. 681)

Sε (x)

sphere of radius ε centered at x (p. 681)

int S

interior of S (p. 681)

intS S

interior of S relative to S (p. 681)

cl S

closure of S (p. 681)

clS S

closure of S relative to S (p. 682)

bd S

boundary of S (p. 682)

bdS S

boundary of S relative to S (p. 682)

(xi )∞ i=1

sequence (x1, x2 , . . .) (p. 682)

vcone D

variational cone of D (p. 685)

D+f (x0 ; ξ)

one-sided directional derivative of f at x0 in the direction ξ (p. 685)

∂f(x0 ) ∂x(i)

partial derivative of f with respect to x(i) at x0 (p. 686)

xxxi

SPECIAL SYMBOLS

f (x)

derivative of f at x (p. 686)

df (x0 ) dx(i)

f (x0 ) (p. 686)

f (k)(x)

kth derivative of f at x (p. 688)

d+f (x0 ) dx(i)

right one-sided derivative (p. 688)

d−f (x0 ) dx(i)

left one-sided derivative (p. 688)

Sign(A)

matrix sign of A ∈ Cn×n (p. 690)

Chapter 11 eA or exp(A)

matrix exponential (p. 707)

L

Laplace transform (p. 710)

Ss(A)

asymptotically stable subspace of A (p. 729)

Su(A)

unstable subspace of A (p. 729)

Chapter 12 U(A, C)

unobservable subspace of (A, C) (p. 800) ⎡ ⎣

O(A, C)

C CA CA2

⎤

.. ⎦ (p. 801) .n−1

CA

C(A, B) K(A, B)  A G∼ C

B

controllable subspace of (A, B) (p. 809)   B AB A2B · · · An−1B (p. 809)



state space realization of G ∈ Fl×m prop [s] (p. 822)

D

Hi,j,k (G)

Markov block-Hankel matrix Oi (A, C)Kj (A, B) (p. 826)

H(G)

Markov block-Hankel matrix O(A, C)K(A, B) (p. 827)

 min

G ∼ H

A

B

C

D



state space realization of G ∈ Fl×m prop [s] (p. 828)   Σ Hamiltonian RA1 −A (p. 853) T

Conventions, Notation, and Terminology

The reader is encouraged to review this section in order to ensure correct interpretation of the statements in this book. When a word is defined, it is italicized. The definition of a word, phrase, or symbol should always be understood as an “if  and only if” statement, although for brevity “only if” is omitted. The symbol =  means equal by definition, where A = B means that the left-hand expression A is defined to be the right-hand expression B. A mathematical object defined by a constructive procedure is well defined if the constructive procedure produces a uniquely defined object. Analogous statements are written in parallel using the following style: If n is (even, odd), then n + 1 is (odd, even). The variables i, j, k, l, m, n always denote integers. Hence, k ≥ 0 denotes a nonnegative integer, k ≥ 1 denotes a positive integer, and the limit limk→∞ Ak is taken over positive integers. The imaginary unit

√ −1 is always denoted by dotless j.

The letter s always represents a complex scalar. The letter z may or may not represent a complex scalar. The inequalities c ≤ a ≤ d and c ≤ b ≤ d are written simultaneously as   a ≤ d. c≤ b The prefix “non” means “not” in the words nonconstant, nonempty, nonintegral, nonnegative, nonreal, nonsingular, nonsquare, nonunique, and nonzero. In some traditional usage, “non” may mean “not necessarily.” “Unique” means “exactly one.”

xxxiv

CONVENTIONS, NOTATION, AND TERMINOLOGY

“Increasing” and “decreasing” indicate strict change for a change in the argument. The word “strict” is superfluous, and thus is omitted. Nonincreasing means nowhere increasing, while nondecreasing means nowhere decreasing. A set can have a finite or infinite number of elements. A finite set has a finite number of elements. Multisets can have repeated elements. Hence, {x}ms and {x, x}ms are different. The listed elements α, β, γ of the conventional set {α, β, γ} need not be distinct. For example, {α, β, α} = {α, β}. In statements of the form “Let spec(A) = {λ1, . . . , λr },” the listed elements λ1, . . . , λr are assumed to be distinct. Square brackets are used alternately with parentheses. For example, f [g(x)] denotes f (g(x)). The order in which the elements of the set {x1, . . . , xn } and the elements of the multiset {x1, . . . , xn }ms are listed has no significance. The components of the ntuple (x1, . . . , xn ) are ordered. The notation (xi )∞ i=1 denotes the sequence (x1, x2 , . . .). A sequence can be viewed as a tuple with a countably infinite number of components, where the order of the components is relevant and the components need not be distinct. The composition of functions f and g is denoted by f • g. The traditional notation f ◦ g is reserved for the Schur product. S1 ⊂ S2 means that S1 is a proper subset of S2 , whereas S1 ⊆ S2 means that S1 is either a proper subset of S2 or is equal to S2 . Hence, S1 ⊂ S2 is equivalent to S1 ⊆ S2 and S1 = S2 , while S1 ⊆ S2 is equivalent to either S1 ⊂ S2 or S1 = S2 . The terminology “graph” corresponds to what is commonly called a “simple directed graph,” while the terminology “symmetric graph” corresponds to a “simple undirected graph.” The range of cos−1 is [0, π], the range of sin−1 is [−π/2, π/2], the range of tan−1 is (−π/2, π/2), and the range of cot−1 is (0, π). The angle between two vectors is an element of [0, π]. Therefore, by using cos−1, the inner product of two vectors can be used to compute the angle between two vectors. 





0! = 1, 0/0 = (sin 0)/0 = (1 − cos 0)/0 = (sinh 0)/0 = 1, and 1/∞ = 0. For all α ∈ C,

    0  α  = 1. For all k ∈ N, = 1. k 0

xxxv

CONVENTIONS, NOTATION, AND TERMINOLOGY 



For all square matrices A, A0 = I . In particular, 00n×n = In . With this convention, it is possible to write ∞  1 αi = 1−α i=0 for all −1 < α < 1. Of course, limx↓0 0x = 0, limx↓0 x0 = 1, and limx↓0 xx = 1. Neither ∞ nor −∞ is a real number. However, some operations are defined for these objects as extended real numbers, such as ∞ + ∞ = ∞, ∞∞ = ∞, and, for all nonzero real numbers α, α∞ = sign(α)∞. 0∞ and ∞ − ∞ are not defined. See [71, pp. 14, 15]. Let a and b be real numbers such that a < b. A finite interval is of the form (a, b), [a, b), (a, b], or [a, b], whereas an infinite interval is of the form (−∞, a), (−∞, a], (a, ∞), [a, ∞), or (−∞, ∞). An interval is either a finite interval or an infinite interval. An extended infinite interval includes either ∞ or −∞. For example, [−∞, a) and [−∞, a] include −∞, (a, ∞] and [a, ∞] include ∞, and [−∞, ∞] includes −∞ and ∞. The symbol F denotes either R or C consistently in each result. For example, in Theorem 5.6.3, the three appearances of “F” can be read as either all “C” or all “R.” The imaginary numbers are denoted by jR. Hence, 0 is both a real number and an imaginary number. The notation Re A and Im A represents the real and imaginary parts of A, respectively. Some books use Re A and Im A to denote the Hermitian and skew-Hermitian matrices 12 (A + A∗ ) and 12 (A − A∗ ). For the scalar ordering “≤,” if x ≤ y, then x < y if and only if x = y. For the entrywise vector and matrix orderings, x ≤ y and x =  y do not imply that x < y. Operations denoted by superscripts are applied before operations

represented by preceding operators. For example, tr (A + B)2 means tr (A + B)2 and cl S∼ means cl(S∼ ). This convention simplifies many formulas. A vector in Fn is a column vector, which is also a matrix with one column. In mathematics, “vector” generally refers to an abstract vector not resolved in coordinates. Sets have elements, vectors and sequences have components, and matrices have entries. This terminology has no mathematical consequence. The notation x(i) represents the ith component of the vector x.

xxxvi

CONVENTIONS, NOTATION, AND TERMINOLOGY

The notation A(i,j) represents the scalar (i, j) entry of A. Ai,j or Aij denotes a block or submatrix of A. All matrices have nonnegative integral dimensions. If a matrix has either zero rows or zero columns, then the matrix is empty. The entries of a submatrix Aˆ of a matrix A are the entries of A located in specified rows and columns. Aˆ is a block of A if Aˆ is a submatrix of A whose entries are entries of adjacent rows and columns of A. Every matrix is both a submatrix and block of itself. The determinant of a submatrix is a subdeterminant. Some books use “minor.” The determinant of a matrix is also a subdeterminant of the matrix. The dimension of the null space of a matrix is its defect. Some books use “nullity.” A block of a square matrix is diagonally located if the block is square and the diagonal entries of the block are also diagonal entries of the matrix; otherwise, the block is off-diagonally located. This terminology avoids confusion with a “diagonal block,” which is a block that is also a square, diagonal submatrix. A B ] ∈ F(n+m)×(k+l), it can be inferred that A ∈ Fn×k For the partitioned matrix [ C D and similarly for B, C, and D.

The Schur product of matrices A and B is denoted by A ◦ B. Matrix multiplication is given priority over Schur multiplication, that is, A ◦ BC means A ◦ (BC). The adjugate of A ∈ Fn×n is denoted by AA . The traditional notation is adj A, while the notation AA is used in [1259]. If A ∈ F is a scalar then AA = 1. In A particular, 0A 1×1 = 1. However, for all n ≥ 2, 0n×n = 0n×n . If F = R, then A becomes A, A∗ becomes AT, “Hermitian” becomes “symmetric,” “unitary” becomes “orthogonal,” “unitarily” becomes “orthogonally,” and “congruence” becomes “T-congruence.” A square complex matrix A is symmetric if AT = A and orthogonal if ATA = I. The diagonal entries of a matrix A ∈ Fn×n all of whose diagonal entries are real are ordered as dmax(A) = d1(A) ≥ d2(A) ≥ · · · ≥ dn(A) = dmin(A). Every n×n matrix has n eigenvalues. Hence, eigenvalues are counted in accordance with their algebraic multiplicity. The phrase “distinct eigenvalues” ignores algebraic multiplicity. The eigenvalues of a matrix A ∈ Fn×n all of whose eigenvalues are real are ordered as λmax(A) = λ1(A) ≥ λ2(A) ≥ · · · ≥ λn(A) = λmin(A).

CONVENTIONS, NOTATION, AND TERMINOLOGY

The inertia of a matrix is written as

xxxvii

⎡

⎤ ν−(A)  In A = ⎣ ν0 (A) ⎦. ν+(A)

Some books use the notation (ν(A), δ(A), π(A)). For A ∈ Fn×n, amultA(λ) is the number of copies of λ in the multispectrum of A, gmultA(λ) is the number of Jordan blocks of A associated with λ, and indA(λ) is the order of the largest Jordan block of A associated with λ. The index of A, denoted by ind A = indA(0), is the order of the largest Jordan block of A associated with the eigenvalue 0. The matrix A ∈ Fn×n is semisimple if the order of every Jordan block of A is 1, and cyclic if A has exactly one Jordan block associated with each of its eigenvalues. Defective means not semisimple, while derogatory means not cyclic. An n × m matrix has exactly min{n, m} singular values, exactly rank A of which are positive. 

The min{n, m} singular values of a matrix A ∈ Fn×m are ordered as σmax(A) =  σ1(A) ≥ σ2(A) ≥ · · · ≥ σmin{n,m}(A). If n = m, then σmin(A) = σn(A). The notation σmin(A) is defined only for square matrices. Positive-semidefinite and positive-definite matrices are Hermitian. A square matrix with entries in F is diagonalizable over F if and only if it can be transformed into a diagonal matrix whose entries are in F by means of a similarity transformation whose entries are in F. Therefore, a complex matrix is diagonalizable over C if and only if all of its eigenvalues are semisimple, whereas a real matrix is diagonalizable over 0 R1 if and only if all of its eigenvalues are semisimple and real. The real matrix −1 0 is diagonalizable 1 j over C, although it is not diagonalizable over R. The Hermitian matrix −j 2 is diagonalizable over C, and also has real eigenvalues. An idempotent matrix A ∈ Fn×n satisfies A2 = A, while a projector is a Hermitian, idempotent matrix. Some books use “projector” for idempotent and “orthogonal projector” for projector. A reflector is a Hermitian, involutory matrix. A projector is a normal matrix each of whose eigenvalues is 1 or 0, while a reflector is a normal matrix each of whose eigenvalues is 1 or −1. An elementary matrix is a nonsingular matrix formed by adding an outer-product matrix to the identity matrix. An elementary reflector is a reflector exactly one of whose eigenvalues is −1. An elementary projector is a projector exactly one of whose eigenvalues is 0. Elementary reflectors are elementary matrices. However, elementary projectors are not elementary matrices since elementary projectors are singular.

xxxviii

CONVENTIONS, NOTATION, AND TERMINOLOGY

A range-Hermitian matrix is a square matrix whose range is equal to the range of its complex conjugate transpose. These matrices are also called “EP” matrices. The polynomials 1 and s3 + 5s2 − 4 are monic. The zero polynomial is not monic. The rank of a polynomial matrix P is the maximum rank of P (s) over C. This quantity is also called the normal rank. We denote this quantity by rank P as distinct from rank P (s), which denotes the rank of the matrix P (s). The rank of a rational transfer function G is the maximum rank of G(s) over C excluding poles of the entries of G. This quantity is also called the normal rank. We denote this quantity by rank G as distinct from rank G(s), which denotes the rank of the matrix G(s). The symbol ⊕ denotes the Kronecker sum. Some books use ⊕ to denote the direct sum of matrices or subspaces. The notation |A| represents the matrix obtained by replacing every entry of A by its absolute value. The notation A represents the matrix (A∗A)1/2. Some books use |A| to denote this matrix. The H¨ older norms for vectors and matrices are denoted by  · p . The matrix norm induced by  · q on the domain and  · p on the codomain is denoted by  · p,q . The Schatten norms for matrices are denoted by  · σp , and the Frobenius norm is denoted by  · F . Hence,  · σ∞ =  · 2,2 = σmax(·),  · σ2 =  · F , and  · σ1 = tr ·.

CONVENTIONS, NOTATION, AND TERMINOLOGY

xxxix

Terminology Relating to Inequalities Let “≤” be a partial ordering, let X be a set, and consider the inequality f (x) ≤ g(x) for all x ∈ X.

(1)

Inequality (1) is sharp if there exists x0 ∈ X such that f (x0 ) = g(x0 ). The inequality f (x) ≤ f (y) for all x ≤ y

(2)

f (x) ≤ p(x) ≤ g(x) for all x ∈ X,

(3)

is a monotonicity result. The inequality

where p is not identically equal to either f or g on X, is an interpolation or refinement of (1). The inequality g(x) ≤ αf (x) for all x ∈ X,

(4)

where α > 1, is a reversal of (1). 

Defining h(x) = g(x) − f (x), it follows that (1) is equivalent to h(x) ≥ 0 for all x ∈ X.

(5)

Now, suppose that h has a global minimizer x0 ∈ X. Then, (5) implies that 0 ≤ h(x0 ) = min h(x) ≤ h(y) for all y ∈ X. x∈X

(6)

Consequently, inequalities are often expressed equivalently in terms of optimization problems, and vice versa. Many inequalities are based on a single function that is either monotonic or convex.

Matrix Mathematics

Chapter One

Preliminaries In this chapter we review some basic terminology and results concerning logic, sets, functions, and related concepts. This material is used throughout the book.

1.1 Logic Every statement is either true or false, but not both. Let A and B be statements. The negation of A is the statement (not A), the both of A and B is the statement (A and B), and the either of A and B is the statement (A or B). The statement (A or B) does not contradict (A and B), that is, the word “or” is inclusive. Exclusive “or” is indicated by the phrase “but not both.” The statements “A and B or C” and “A or B and C” are ambiguous. We therefore write “A and either B or C” and “either A or both B and C.” Let A and B be statements. The implication statement “if A is satisfied, then B is satisfied” or, equivalently, “A implies B” is written as A =⇒ B, while A ⇐⇒ B is equivalent to [(A =⇒ B) and (A ⇐= B)]. Of course, A ⇐= B means B =⇒ A. A tautology is a statement that is true regardless of whether the component statements are true or false. For example, the statement “(A and B) implies A” is a tautology. A contradiction is a statement that is false regardless of whether the component statements are true or false. For example, the statement “A implies (not )A” is a contradiction. Suppose that A ⇐⇒ B. Then, A is satisfied if and only if B is satisfied. The implication A =⇒ B (the “only if” part) is necessity, while B =⇒ A (the “if” part) is sufficiency. The converse statement of A =⇒ B is B =⇒ A. The statement A =⇒ B is equivalent to its contrapositive statement (not B) =⇒ (not A). A theorem is a significant statement, while a proposition is a theorem of less significance. The primary role of a lemma is to support the proof of a theorem or proposition. Furthermore, a corollary is a consequence of a theorem or proposition. Finally, a fact is either a theorem, proposition, lemma, or corollary. Theorems, propositions, lemmas, corollaries, and facts are provably true statements. Suppose that A =⇒ A =⇒ B =⇒ B . Then, A =⇒ B is a corollary of A =⇒ B.

2

CHAPTER 1

Let A, B, and C be statements, and assume that A =⇒ B. Then, A =⇒ B is a strengthening of the statement (A and C) =⇒ B. If, in addition, A =⇒ C, then the statement (A and C) =⇒ B has a redundant assumption.

1.2 Sets A set {x, y, . . .} is a collection of elements. A set may have a finite or infinite number of elements. A finite set has a finite number of elements. Let X be a set. Then, x∈X

(1.2.1)

means that x is an element of X. If w is not an element of X, then we write w ∈ X.

(1.2.2)

The statement “x ∈ X” is either true or false, but not both. The statement “X ∈ / X” is true by convention, and thus no set can be an element of itself. Therefore, there does not exist a set that contains every set. The set with no elements, denoted by ∅, is the empty set. If X = ∅, then X is nonempty. A set cannot have repeated elements. For example, {x, x} = {x}. However, a multiset is a collection of elements that allows for repetition. The multiset consisting of two copies of x is written as {x, x}ms . However, we do not assume that the listed elements x, y of the conventional set {x, y} are distinct. The number of distinct elements of the set S or not-necessarily-distinct elements of the multiset S is the cardinality of S, which is denoted by card(S). There are two basic types of mathematical statements for quantifiers. An existential statement is of the form there exists x ∈ X such that statement Z is satisfied,

(1.2.3)

while a universal statement has the structure for all x ∈ X, it follows that statement Z is satisfied, or, equivalently,

statement Z is satisfied for all x ∈ X.

(1.2.4) (1.2.5)

Let X and Y be sets. The intersection of X and Y is the set of common elements of X and Y given by  X∩Y= {x: x ∈ X and x ∈ Y} = {x ∈ X: x ∈ Y}

= {x ∈ Y: x ∈ X} = Y ∩ X,

(1.2.6) (1.2.7)

while the set of elements in either X or Y (the union of X and Y) is 

X ∪ Y = {x: x ∈ X or x ∈ Y} = Y ∪ X.

(1.2.8)

The complement of X relative to Y is 

Y\X = {x ∈ Y: x ∈ X}.

(1.2.9)

3

PRELIMINARIES

If Y is specified, then the complement of X is  X∼ = Y\X.

(1.2.10)

If x ∈ X implies that x ∈ Y, then X is contained in Y (X is a subset of Y), which is written as X ⊆ Y. (1.2.11) The statement X = Y is equivalent to the validity of both X ⊆ Y and Y ⊆ X. If X ⊆ Y and X = Y, then X is a proper subset of Y and we write X ⊂ Y. The sets X and Y are disjoint if X ∩ Y = ∅. A partition of X is a set of pairwise-disjoint and nonempty subsets of X whose union is equal to X. The operations “∩,” “∪,” and “\” and the relations “⊂” and “⊆” extend directly to multisets. For example, {x, x}ms ∪ {x}ms = {x, x, x}ms .

(1.2.12)

By ignoring repetitions, a multiset can be converted to a set, while a set can be viewed as a multiset with distinct elements. The Cartesian product X1 × · · · × Xn of sets X1, . . . , Xn is the set consisting of tuples of the form (x1, . . . , xn ), where xi ∈ Xi for all i ∈ {1, . . . , n}. A tuple with n components is an n-tuple. Note that the components of an n-tuple are ordered but need not be distinct. By replacing the logical operations “=⇒,” “and,” “or,” and “not” by “⊆,” “∪,” “∩,” and “∼ ,” respectively, statements about statements A and B can be transformed into statements about sets A and B, and vice versa. For example, the tautology A and (B or C) ⇐⇒ (A and B) or (A and C) is equivalent to

A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C).

1.3 Integers, Real Numbers, and Complex Numbers The symbols Z, N, and P denote the sets of integers, nonnegative integers, and positive integers, respectively. The symbols R and C denote the real and complex number fields, respectively, whose elements are scalars. Define √  j= −1. Let x ∈ C. Then, x = y + jz, where y, z ∈ R. Define the complex conjugate x of x by  x = y − jz (1.3.1) and the real part Re x of x and the imaginary part Im x of x by 

Re x = 12 (x + x) = y and

 Im x =

1 j2 (x

− x) = z.

(1.3.2) (1.3.3)

4

CHAPTER 1

Furthermore, the absolute value |x| of x is defined by   |x| = y 2 + z 2 .

(1.3.4)

The closed left half plane (CLHP), open left half plane (OLHP), closed right half plane (CRHP), and open right half plane (ORHP) are the subsets of C defined by  OLHP = {x ∈ C: Re x < 0}, (1.3.5)  CLHP = {x ∈ C: Re x ≤ 0}, 

ORHP = {x ∈ C: Re x > 0}, 

CRHP = {x ∈ C: Re x ≥ 0}.

(1.3.6) (1.3.7) (1.3.8)

The imaginary numbers are represented by jR. Note that 0 is both a real number and an imaginary number. Next, we define the open unit disk (OUD) and the closed unit disk (CUD) by  OUD = {x ∈ C: |x| < 1}

(1.3.9)

 {x ∈ C: |x| ≤ 1}. CUD =

(1.3.10)

and

The complements of the open unit disk and the closed unit disk are given, respectively, by the closed punctured plane (CPP) and the open punctured plane, which are defined by  CPP = {x ∈ C: |x| ≥ 1} (1.3.11) and 

OPP = {x ∈ C: |x| > 1}.

(1.3.12)

Since R is a proper subset of C, we state many results for C. In other cases, we treat R and C separately. To do this efficiently, we use the symbol F to consistently denote either R or C.

1.4 Functions Let X and Y be sets. Then, a function f that maps X into Y is a rule f : X → Y that assigns a unique element f(x) (the image of x) of Y to each element x of X. Equivalently, a function f : X → Y can be viewed as a subset F of X × Y such that, for all x ∈ X, it follows that there exists y ∈ Y such that (x, y) ∈ F and such that, if  (x, y1 ), (x, y2 ) ∈ F, then y1 = y2 . In this case, F = Graph(f ) = {(x, f(x)): x ∈ X}. The set X is the domain of f, while the set Y is the codomain of f. If f : X → X, then  f is a function on X. For X1 ⊆ X, it is convenient to define f(X1 ) = {f(x): x ∈ X1 }. The set f(X), which is denoted by R(f ), is the range of f. If, in addition, Z is a set and g : f(X) → Z, then g • f : X → Z (the composition of g and f ) is the function  (g • f )(x) = g[f(x)]. If x1, x2 ∈ X and f(x1 ) = f(x2 ) implies that x1 = x2 , then f

PRELIMINARIES

5

is one-to-one; if R(f ) = Y, then f is onto. The function IX : X → X defined by  IX (x) = x for all x ∈ X is the identity on X. Finally, x ∈ X is a fixed point of the function f : X → X if f (x) = x. The following result shows that function composition is associative. Proposition 1.4.1. Let X, Y, Z, and W be sets, and let f : X → Y, g : Y → Z, h : Z → W. Then, h • (g • f ) = (h • g) • f. (1.4.1) Hence, we write h • g • f for h • (g • f ) and (h • g) • f. ˆ be a partition of X. Furthermore, let f : X ˆ → X, Let X be a set, and let X ˆ where, for all S ∈ X, it follows that f (S) ∈ S. Then, f is a canonical mapping, and ˆ of X, it f (S) is a canonical form. That is, for all components S of the partition X follows that the function f assigns an element of S to the set S. Let f : X → Y. Then, f is left invertible if there exists a function g : Y → X (a left inverse of f ) such that g • f = IX , whereas f is right invertible if there exists a function h: Y → X (a right inverse of f ) such that f • h = IY. In addition, the function f : X → Y is invertible if there exists a function f −1 : Y → X (the inverse of f ) such that f −1 • f = IX and f • f −1 = IY. The inverse image f −1(S) of S ⊆ Y is defined by  f −1(S) = {x ∈ X: f(x) ∈ S}. (1.4.2) Note that the set f −1(S) can be defined whether or not f is invertible. In fact, f −1[f (X)] = X. Theorem 1.4.2. Let X and Y be sets, and let f : X → Y. Then, the following statements hold: i) f is left invertible if and only if f is one-to-one. ii) f is right invertible if and only if f is onto. Furthermore, the following statements are equivalent: iii) f is invertible. iv) f has a unique inverse. v) f is one-to-one and onto. vi) f is left invertible and right invertible. vii) f has a unique left inverse. viii) f has a unique right inverse. Proof. To prove i), suppose that f is left invertible with left inverse g : Y → X. Furthermore, suppose that x1, x2 ∈ X satisfy f(x1 ) = f(x2 ). Then, x1 = g[f(x1 )] = g[f(x2 )] = x2 , which shows that f is one-to-one. Conversely, suppose that f is one-to-one so that, for all y ∈ R(f ), there exists a unique x ∈ X such that f(x) = y.

6

CHAPTER 1 

Hence, define the function g : Y → X by g(y) = x for all y = f(x) ∈ R(f ) and by g(y) arbitrary for all y ∈ Y\R(f ). Consequently, g[f(x)] = x for all x ∈ X, which shows that g is a left inverse of f. To prove ii), suppose that f is right invertible with right inverse g : Y → X. Then, for all y ∈ Y, it follows that f [g(y)] = y, which shows that f is onto. Conversely, suppose that f is onto so that, for all y ∈ Y, there exists at least one x ∈ X such that f(x) = y. Selecting one such x arbitrarily, define g : Y → X by  g(y) = x. Consequently, f [g(y)] = y for all y ∈ Y, which shows that g is a right inverse of f. Definition 1.4.3. Let I ⊂ R be a finite or infinite interval, and let f : I → R. Then, f is convex if, for all α ∈ [0, 1] and for all x, y ∈ I, it follows that f [αx + (1 − α)y] ≤ αf (x) + (1 − α)f (y).

(1.4.3)

Furthermore, f is strictly convex if, for all α ∈ (0, 1) and for all distinct x, y ∈ I, it follows that f [αx + (1 − α)y] < αf (x) + (1 − α)f (y). A more general definition of convexity is given by Definition 8.6.14.

1.5 Relations Let X, X1, and X2 be sets. A relation R on X1 × X2 is a subset of X1 × X2 . A relation R on X is a relation on X × X. Likewise, a multirelation R on X1 × X2 is a multisubset of X1 × X2 , while a multirelation R on X is a multirelation on X × X. Let X be a set, and let R1 and R2 be relations on X. Then, R1 ∩ R2 , R1\R2 , and R1 ∪ R2 are relations on X. Furthermore, if R is a relation on X and X0 ⊆ X,  then we define R|X0 = R ∩ (X0 × X0 ), which is a relation on X0 . The following result shows that relations can be viewed as generalizations of functions. Proposition 1.5.1. Let X1 and X2 be sets, and let R be a relation on X1 × X2 . Then, there exists a function f : X1 → X2 such that R = Graph(f ) if and only if, for all x ∈ X1, there exists a unique y ∈ X2 such that (x, y) ∈ R. In this case, f (x) = y. Definition 1.5.2. Let R be a relation on the set X. Then, the following terminology is defined: i) R is reflexive if, for all x ∈ X, it follows that (x, x) ∈ R. ii) R is symmetric if, for all (x1, x2 ) ∈ R, it follows that (x2 , x1 ) ∈ R. iii) R is transitive if, for all (x1, x2 ) ∈ R and (x2 , x3 ) ∈ R, it follows that (x1, x3 ) ∈ R. iv) R is an equivalence relation if R is reflexive, symmetric, and transitive.

7

PRELIMINARIES

Proposition 1.5.3. Let R1 and R2 be relations on the set X. If R1 and R2 are (reflexive, symmetric) relations, then so are R1 ∩ R2 and R1 ∪ R2 . If R1 and R2 are (transitive, equivalence) relations, then so is R1 ∩ R2 . Definition 1.5.4. Let R be a relation on the set X. Then, the following terminology is defined:  i) The complement R∼ of R is the relation R∼ = (X × X)\R.

ii) The support supp(R) of R is the smallest subset X0 of X such that R is a relation on X0 . 

iii) The reversal rev(R) of R is the relation rev(R) = {(y, x) : (x, y) ∈ R}.  iv) The shortcut shortcut(R) of R is the relation shortcut(R) = {(x, y) ∈ X × X: x and y are distinct and there exist k ≥ 1 and x1, . . . , xk ∈ X such that (x, x1 ), (x1, x2 ), . . . , (xk , y) ∈ R}.

v) The reflexive hull ref(R) of R is the smallest reflexive relation on X that contains R. vi) The symmetric hull sym(R) of R is the smallest symmetric relation on X that contains R. vii) The transitive hull trans(R) of R is the smallest transitive relation on X that contains R. viii) The equivalence hull equiv(R) of R is the smallest equivalence relation on X that contains R. Proposition 1.5.5. Let R be a relation on the set X. Then, the following statements hold: i) ref(R) = R ∪ {(x, x) : x ∈ X}. ii) sym(R) = R ∪ rev(R). iii) trans(R) = R ∪ shortcut(R). iv) equiv(R) = R ∪ ref(R) ∪ sym(R) ∪ trans(R). v) equiv(R) = R ∪ ref(R) ∪ rev(R) ∪ shortcut(R). Furthermore, the following statements hold: vi) R is reflexive if and only if R = ref(R). vii) R is symmetric if and only if R = rev(R). viii) R is transitive if and only if R = trans(R). ix) R is an equivalence relation if and only if R = equiv(R). R

For an equivalence relation R on the set X, (x1, x2 ) ∈ R is denoted by x1 = x2 .  R If R is an equivalence relation and x ∈ X, then the subset Ex = {y ∈ X: y = x} of X is the equivalence class of x induced by R.

8

CHAPTER 1

Theorem 1.5.6. Let R be an equivalence relation on a set X. Then, the set {Ex : x ∈ X} of equivalence classes induced by R is a partition of X.  Proof. Since X = x∈X Ex , it suffices to show that if x, y ∈ X, then either Ex = Ey or Ex ∩ Ey = ∅. Hence, let x, y ∈ X, and suppose that Ex and Ey are not disjoint so that there exists z ∈ Ex ∩ Ey . Thus, (x, z) ∈ R and (z, y) ∈ R. Now, let w ∈ Ex . Then, (w, x) ∈ R, (x, z) ∈ R, and (z, y) ∈ R imply that (w, y) ∈ R. Hence, w ∈ Ey , which implies that Ex ⊆ Ey . By a similar argument, Ey ⊆ Ex . Consequently, Ex = Ey . The following result, which is the converse of Theorem 1.5.6, shows that a partition of a set X defines an equivalence relation on X. Theorem 1.5.7. Let X be a set, consider a partition of X, and define the relation R on X by (x, y) ∈ R if and only if x and y belong to the same partition subset of X. Then, R is an equivalence relation on X. Definition 1.5.8. Let R be a relation on the set X. Then, the following terminology is defined: i) R is antisymmetric if (x1, x2 ) ∈ R and (x2 , x1 ) ∈ R imply that x1 = x2 . ii) R is a partial ordering on X if R is reflexive, antisymmetric, and transitive. R

Let R be a partial ordering on X. Then, (x1, x2 ) ∈ R is denoted by x1 ≤ x2 . R

R

If x1 ≤ x2 and x2 ≤ x1, then, since R is antisymmetric, it follows that x1 = x2 . R R Furthermore, if x1 ≤ x2 and x2 ≤ x3 , then, since R is transitive, it follows that R x1 ≤ x3 . R

Definition 1.5.9. Let “≤” be a partial ordering on X. Then, the following terminology is defined: i) Let S ⊆ X. Then, y ∈ X is a lower bound for S if, for all x ∈ S, it follows R

that y ≤ x. ii) Let S ⊆ X. Then, y ∈ X is an upper bound for S if, for all x ∈ S, it follows R

that x ≤ y. iii) Let S ⊆ X. Then, y ∈ X is the least upper bound lub(S) for S if y is an upper bound for S and, for all upper bounds x ∈ X for S, it follows that R y ≤ x. In this case, we write y = lub(S). iv) Let S ⊆ X. Then, y ∈ X is the greatest lower bound for S if y is a lower R

bound for S and, for all lower bounds x ∈ X for S, it follows that x ≤ y. In this case, we write y = glb(S). R

v) ≤ is a lattice on X if, for all distinct x, y ∈ X, the set {x, y} has a least upper bound and a greatest lower bound. vi) R is a total ordering on X if, for all x, y ∈ X, it follows that either (x, y) ∈ R or (y, x) ∈ R.

9

PRELIMINARIES

For a subset S of the real numbers, it is traditional to write inf S and sup S for glb(S) and lub(S), respectively, where “inf” and “sup” denote infimum and supremum, respectively.

1.6 Graphs Let X be a finite, nonempty set, and let R be a relation on X. Then, the pair G = (X, R) is a graph. The elements of X are the nodes of G, while the elements of R are the arcs of G. If R is a multirelation on X, then G = (X, R) is a multigraph. The graph G = (X, R) can be visualized as a set of points in the plane representing the nodes in X connected by the arcs in R. Specifically, the arc (x, y) ∈ R from x to y can be visualized as a directed line segment or curve connecting node x to node y. The direction of an arc can be denoted by an arrow head. For example, consider a graph that represents a city with streets (arcs) connecting houses (nodes). Then, a symmetric relation is a street plan with no one-way streets, whereas an antisymmetric relation is a street plan with no two-way streets. Definition 1.6.1. Let G = (X, R) be a graph. Then, the following terminology is defined: i) The arc (x, x) ∈ R is a self-loop. ii) The reversal of (x, y) ∈ R is (y, x). iii) If x, y ∈ X and (x, y) ∈ R, then y is the head of (x, y) and x is the tail of (x, y). iv) If x, y ∈ X and (x, y) ∈ R, then x is a parent of y, and y is a child of x. v) If x, y ∈ X and either (x, y) ∈ R or (y, x) ∈ R, then x and y are adjacent. vi) If x ∈ X has no parent, then x is a root. vii) If x ∈ X has no child, then x is a leaf. Suppose that (x, x) ∈ R. Then, x is both the head and the tail of (x, x), and thus x is a parent and child of itself. Consequently, x is neither a root nor a leaf. Furthermore, x is adjacent to itself. Definition 1.6.2. Let G = (X, R) be a graph. Then, the following terminology is defined: 

i) The reversal of G is the graph rev(G) = (X, rev(R)).  ii) The complement of G is the graph G∼ = (X, R∼ ). 

iii) The reflexive hull of G is the graph ref(G) = (X, ref(R)).  iv) The symmetric hull of G is the graph sym(G) = (X, sym(R)). 

v) The transitive hull of G is the graph trans(G) = (X, trans(R)). 

vi) The equivalence hull of G is the graph equiv(G) = (X, equiv(R)). vii) G is reflexive if R is reflexive.

10

CHAPTER 1

viii) G is symmetric if R is symmetric. In this case, the arcs (x, y) and (y, x) in R are denoted by the subset {x, y} of X, called an edge. For the self-loop (x, x), the corresponding edge is the subset {x}. ix) G is transitive if R is transitive. x) G is an equivalence graph if R is an equivalence relation. xi) G is antisymmetric if R is antisymmetric. xii) G is partially ordered if R is a partial ordering on X. xiii) G is totally ordered if R is a total ordering on X. xiv) G is a tournament if G has no self-loops, is antisymmetric, and sym(R) = X × X. Note that a symmetric graph can include self-loops, whereas a reflexive graph has a self-loop at every node. Definition 1.6.3. Let G = (X, R) be a graph. Then, the following terminology is defined: i) The graph G = (X , R ) is a subgraph of G if X ⊆ X and R ⊆ R. ii) The subgraph G = (X , R ) of G is a spanning subgraph of G if supp(R) = supp(R ). iii) For x, y ∈ X, a walk in G from x to y is an n-tuple of arcs of the form (x, y) ∈ R for n = 1 and ( (x, x1 ), (x1, x2 ), . . . , (xn−1 , y) ) ∈ Rn for n ≥ 2. The length of the walk is n. The nodes x, x1, . . . , xn−1 , y are the nodes of the walk. Furthermore, if n ≥ 2, then the nodes x1, . . . , xn−1 are the intermediate nodes of the walk. iv) G is connected if, for all distinct x, y ∈ X, there exists a walk in G from x to y. v) For x, y ∈ X, a trail in G from x to y is a walk in G from x to y whose arcs are distinct and such that, if (w, z) is an arc of the walk, then (z, w) is not an arc of the walk. vi) For x, y ∈ X, a path in G from x to y is a trail in G from x to y whose intermediate nodes (if any) are distinct. vii) G is traceable if G has a path such that every node in X is a node of the path. Such a path is called a Hamiltonian path. viii) For x ∈ X, a cycle in G at x is a path in G from x to x whose length is greater than 1. ix) G is acyclic if G has no cycles. x) The period of G is the greatest common divisor of the lengths of the cycles in G. Furthermore, G is aperiodic if the period of G is 1. xi) G is Hamiltonian if G has a cycle such that every node in X is a node of the cycle. Such a cycle is called a Hamiltonian cycle.

11

PRELIMINARIES

xii) G is a tree if G has exactly one root x and, for all y ∈ X such that y = x, there exists a unique path from x to y. xiii) G is a forest if G is a union of trees. xiv) G is a chain if G is a tree and has exactly one leaf.  xv) The indegree of x ∈ X is indeg(x) = card{y ∈ X: y is a parent of x}. 

xvi) The outdegree of x ∈ X is outdeg(x) = card{y ∈ X: y is a child of x}. 

xvii) If G is symmetric, then the degree of x ∈ X is deg(x) = indeg(x) = outdeg(x). xviii) If X0 ⊆ X, then,

 G|X0 = (X0 , R|X0 ). 



xix) If G = (X , R ) is a graph, then G ∪ G = (X ∪ X , R ∪ R ) and G ∩ G = (X ∩ X , R ∩ R ). xx) Let X = X1 ∪ X2 , where X1 and X2 are nonempty and disjoint, and assume that X = supp(G). Then, (X1, X2 ) is a directed cut of G if, for all x1 ∈ X1 and x2 ∈ X2 , there does not exist a walk from x1 to x2 . Note that a graph that is acyclic cannot be aperiodic. Let G = (X, R) be a graph, and let w : X × X → [0, ∞), where w(x, y) > 0 if (x, y) ∈ R and w(x, y) = 0 if (x, y) ∈ / R. For each arc (x, y) ∈ R, w(x, y) is the weight associated with the arc (x, y), and the triple G = (X, R, w) is a weighted  graph. Every graph can be viewed as a weighted graph by defining w[(x, y)] = 1  for all (x, y) ∈ R and w[(x, y)] = 0 for all (x, y) ∈ / R. The graph G = (X , R , w ) is a weighted subgraph of G if X ⊆ X , R is a relation on X , R ⊆ R, and w is the restriction of w to R . Finally, if G is symmetric, then w is defined on edges {x, y} of G.

1.7 Facts on Logic, Sets, Functions, and Relations Fact 1.7.1. Let A and B be statements. Then, the following statements hold: i) not(A or B) ⇐⇒ [(not A) and (not B)]. ii) not(A and B) ⇐⇒ (not A) or (not B). iii) (A or B) ⇐⇒ [(not A) =⇒ B]. iv) (A =⇒ B) ⇐⇒ [(not A) or B]. v) [A and (not B)] ⇐⇒ [not(A =⇒ B)]. Remark: Each statement is a tautology. Remark: Statements i) and ii) are De Morgan’s laws. See [233, p. 24].

12

CHAPTER 1

Fact 1.7.2. The following statements are equivalent: i) A =⇒ (B or C). ii) [A and (not B)] =⇒ C. Remark: The statement that i) and ii) are equivalent is a tautology. Fact 1.7.3. The following statements are equivalent: i) A ⇐⇒ B. ii) [A or (not B)] and (not [A and (not B)]). Remark: The statement that i) and ii) are equivalent is a tautology. Fact 1.7.4. The following statements are equivalent: i) Not [for all x, there exists y such that statement Z is satisfied]. ii) There exists x such that, for all y, statement Z is not satisfied. Fact 1.7.5. Let A, B, and C be sets, and assume that each of these sets has a finite number of elements. Then, card(A ∪ B) = card(A) + card(B) − card(A ∩ B) and card(A ∪ B ∪ C) = card(A) + card(B) + card(C) − card(A ∩ B) − card(A ∩ C) − card(B ∩ C) + card(A ∩ B ∩ C). Remark: This result is the inclusion-exclusion principle. See [181, p. 82] or [1249, pp. 64–67]. Fact 1.7.6. Let A, B, C be subsets of a set X. Then, the following statements hold: i) A ∩ A = A ∪ A = A. ii) (A ∪ B)∼ = A∼ ∩ B∼ . iii) (A ∩ B)∼ = A∼ ∪ B∼ . iv) A = (A\B) ∪ (A ∩ B). v) [A\(A ∩ B)] ∪ B = A ∪ B. vi) (A ∪ B)\(A ∩ B) = (A ∩ B∼ ) ∪ (A∼ ∩ B). vii) A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C). viii) A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C). ix) (A\B)\C = A\(B ∪ C). x) (A ∩ B)\C = (A\C) ∩ (B\C). xi) (A ∩ B)\(C ∩ B) = (A\C) ∩ B.

13

PRELIMINARIES

xii) (A ∪ B)\C = (A\C) ∪ (B\C) = [A\(B ∪ C)] ∪ (B\C). xiii) (A ∪ B)\(C ∩ B) = (A\B) ∪ (B\C). xiv) (A ∪ B) ∩ (A ∪ B∼ ) = A. xv) (A ∪ B) ∩ (A∼ ∪ B) ∩ (A ∪ B∼ ) = A ∩ B. Fact 1.7.7. Define the relation R on R × R by 

R = {((x1, y1 ), (x2 , y2 )) ∈ (R × R) × (R × R) : x1 ≤ x2 and y1 ≤ y2 }. Then, R is a partial ordering. Fact 1.7.8. Define the relation L on R × R by  {((x1, y1 ), (x2 , y2 )) ∈ (R × R) × (R × R) : L=

x1 ≤ x2 and, if x1 = x2 , then y1 ≤ y2 }. Then, L is a total ordering on R × R. d

d

d

Remark: Denoting this total ordering by “≤,” note that (1, 4) ≤ (2, 3) and (1, 4) ≤ (1, 5). Remark: This ordering is the lexicographic ordering or dictionary ordering, where d

‘book’ ≤ ‘box’. Note that the ordering of words in a dictionary is reflexive, antisymmetric, and transitive, and that every pair of words can be ordered. Remark: See Fact 2.9.31. Fact 1.7.9. Let f : X → Y, and assume that f is invertible. Then, (f −1 )−1 = f. Fact 1.7.10. Let f : X → Y and g : Y → Z, and assume that f and g are invertible. Then, g • f is invertible and (g • f )−1 = f −1 • g −1. Fact 1.7.11. Let f : X → Y, and let A, B ⊆ X. Then, the following statements hold: i) If A ⊆ B, then f (A) ⊆ f (B). ii) f (A ∪ B) = f (A) ∪ f (B). iii) f (A ∩ B) ⊆ f (A) ∩ f (B). Fact 1.7.12. Let f : X → Y, and let A, B ⊆ Y. Then, the following statements hold: i) f [f −1 (A)] ⊆ A ⊆ f −1 [f (A)]. ii) f −1 (A ∪ B) = f −1 (A) ∪ f −1 (B). iii) f −1 (A ∩ B) = f −1 (A) ∩ f −1 (B). iv) f −1 (A\B) = f −1 (A)\f −1 (B).

14

CHAPTER 1

Fact 1.7.13. Let X and Y be finite sets, and let f : X → Y. Then, the following statements hold: i) If card(X) < card(Y), then f is not onto. ii) If card(Y) < card(X), then f is not one-to-one. iii) If f is one-to-one and onto, then card(X) = card(Y). Now, assume in addition that card(X) = card(Y). Then, the followings statements are equivalent: iv) f is one-to-one. v) f is onto. vi) card[f (X)] = card(X). Remark: See Fact 1.8.1. Fact 1.7.14. Let S1, . . . , Sm be finite sets, and let 

n=

m 

card(Si ).

i=1

Then,

n m

≤ max card(Si ). i=1,...,m

In addition, if m < n, then there exists i ∈ {1, . . . , m} such that card(Si ) ≥ 2. Remark: x is the smallest integer greater than or equal to x. Remark: This result is the pigeonhole principle. Fact 1.7.15. Let f : X → Y. Then, the following statements are equivalent: i) f is one-to-one. ii) For all A ⊆ X and B ⊆ X, it follows that f (A ∩ B) = f (A) ∩ f (B). iii) For all A ⊆ X, it follows that f −1 [f (A)] = A. iv) For all disjoint A ⊆ X and B ⊆ X, it follows that f (A) and f (B) are disjoint. v) For all A ⊆ X and B ⊆ X such that A ⊆ B, it follows that f (A\B) = f (A)\f (B). Proof: See [71, pp. 44, 45]. Fact 1.7.16. Let f : X → Y. Then, the following statements are equivalent: i) f is onto. ii) For all A ⊆ X, it follows that f [f −1 (A)] = A.

PRELIMINARIES

15

Fact 1.7.17. Let f : X → Y, and let g : Y → Z. Then, the following statements hold: i) If f and g are one-to-one, then f • g is one-to-one. ii) If f and g are onto, then f • g is onto. Remark: A matrix version of this result is given by Fact 2.10.3. Fact 1.7.18. Let f : X → Y, let g : Y → X, and assume that f and g are one-to-one. Then, there exists h: X → Y such that h is one-to-one and onto. Proof: See [1049, pp. 16, 17]. Remark: This result is the Schroeder-Bernstein theorem. Fact 1.7.19. Let X be a set, and let X denote the class of subsets of X. Then, “⊂” and “⊆” are transitive relations on X, and “⊆” is a partial ordering on X.

1.8 Facts on Graphs Fact 1.8.1. Let G = (X, R) be a graph. Then, the following statements hold: i) R is the graph of a function on X if and only if every node in X has exactly one child. Furthermore, the following statements are equivalent: ii) R is the graph of a one-to-one function on X. iii) R is the graph of an onto function on X. iv) R is the graph of a one-to-one and onto function on X. v) Every node in X has exactly one child and not more than one parent. vi) Every node in X has exactly one child and at least one parent. vii) Every node in X has exactly one child and exactly one parent. Remark: See Fact 1.7.13. Fact 1.8.2. Let G = (X, R) be a graph, and assume that R is the graph of a function f : X → X. Then, either f is the identity function or G has a cycle. Fact 1.8.3. Let G = (X, R) be a graph, and assume that G has a Hamiltonian cycle. Then, G has no roots and no leaves. Fact 1.8.4. Let G = (X, R) be a graph. Then, G has either a root or a cycle. Fact 1.8.5. Let G = (X, R) be a symmetric graph. Then, the following statements are equivalent: i) G is a forest. ii) G has no cycles.

16

CHAPTER 1

iii) No pair of nodes in X is connected by more than one path. Furthermore, the following statements are equivalent: iv) G is a tree. v) G is a connected forest. vi) G is connected and has no cycles. vii) G is connected and has card(X) − 1 edges. viii) G has no cycles and has card(X) − 1 edges. ix) Every pair of nodes in X is connected by exactly one path. Fact 1.8.6. Let G = (X, R) be a tournament. Then, G has a Hamiltonian path. Furthermore, every Hamiltonian path is a Hamiltonian cycle if and only if G is connected. Fact 1.8.7. Let G = (X, R) be a symmetric graph, where X ⊂ R2, assume  that n = card(X) ≥ 3, assume that G is connected, and assume that the edges in R can be represented by line segments that lie in the same plane and that are either disjoint or intersect at a node. Furthermore, let m denote the number of edges of G, and let f denote the number of disjoint regions in R2 whose boundaries are the edges of G. Then, n − m + f = 2. Furthermore,

f ≤ 2(n − 2),

and thus

m ≤ 3(n − 2).

Remark: The equality gives the Euler characteristic for a planar graph. A similar result holds for the surfaces of convex polyhedra in three dimensions, such as the tetrahedron, cube (hexahedron), octahedron, dodecahedron, and icosahedron. See [1152].

1.9 Facts on Binomial Identities and Sums Fact 1.9.1. The following statements hold: i) Let 0 ≤ k ≤ n. Then,

    n n = . k n−k

ii) Let 1 ≤ k ≤ n. Then, k iii) Let 2 ≤ k ≤ n. Then,

    n n−1 =n . k k−1

    n n−2 k(k − 1) = n(n − 1) . k k−2

17

PRELIMINARIES

iv) Let 0 ≤ k < n. Then,

    n n−1 (n − k) =n . k k

v) Let 1 ≤ k ≤ n. Then,

      n n−1 n−1 = + . k k k−1

vi) Let 0 ≤ m ≤ k ≤ n. Then,       n k n n−m = . k m m k−m vii) Let m, n ≥ 0. Then,

 m   n+i n

i=0

  n+m+1 = . m

viii) Let k ≥ 0 and n ≥ 1. Then,   (k + i)! k+n = k! . i! k+1

n−1  i=0

ix) Let 0 ≤ k ≤ n. Then,

n    i i=k

k

  n+1 = . k+1

x) Let n, m ≥ 0, and let 0 ≤ k ≤ min{n, m}. Then,    k    n m n+m = . i k−i k i=0 xi) Let n ≥ 0. Then,

   n    n n 2n = . i i−1 n+1 i=1

xii) Let 0 ≤ k ≤ n. Then, n−k 

n i

i=0



 n (2n)! . = (n − k)!(n + k)! k+i

xiii) Let 0 ≤ k ≤ n/2. Then, n−k 

i k

i=k

    n−i n+1 = . k 2k + 1

xiv) Let 1 ≤ k ≤ n/2. Then, k  i=0

 2

2i

n k−i

    n−k+i 2n = . 2i 2k

18

CHAPTER 1

xv) Let 1 ≤ k ≤ (n − 1)/2. Then,      k  n n−k+i 2n = . 22i+1 k−i 2i + 1 2k + 1 i=0 xvi) Let n ≥ 0. Then,

n  2  n

i

i=0

xvii) Let n ≥ 1. Then,

  n  2  n 2n − 1 i =n . i n−1 i=0

xviii) For all x, y ∈ C and n ≥ 0,

n    n n−i i x y. i i=0

(x + y)n = xix) Let n ≥ 0. Then,

n    n

i

i=0

xx) Let n ≥ 0. Then,

  2n = . n

n  i=0

= 2n.

  2n+1 − 1 1 n = . i+1 i n+1

xxi) Let n ≥ 0. Then,  n   2n + 1 i

i=0

xxii) Let n > 1. Then,

i

= 4n.

  2n = 4n−1 n. (n − i) i i=0 2

n/2 

 n = 2n−1. 2i

 i=0

(n−1)/2 

 n = 2n−1. 2i + 1

 i=0

xxv) Let n ≥ 0. Then,

 2n   2n i=0

n−1 

xxiii) Let n ≥ 0. Then,

xxiv) Let n ≥ 0. Then,

=

n/2



(−1)i

i=0

  n nπ . = 2n/2 cos 4 2i

xxvi) Let n ≥ 0. Then, 

(n−1)/2

 i=0

(−1)i

 n nπ . = 2n/2 sin 4 2i + 1

19

PRELIMINARIES

xxvii) Let n ≥ 1. Then,

n    n i = n2n−1. i i=1

xxviii) Let n ≥ 1. Then,

n    n i=0

2i

= 2n−1.

xxix) Let 0 ≤ k < n. Then,     n k n−1 = (−1) . (−1) i k i=0

k 

xxx) Let n ≥ 1. Then,

i

n 

(−1)i

i=0

xxxi) Let n ≥ 1. Then,

  n = 0. i

n n  2i 2n  1  . = i+1 n + 1 i=0 ni i=0

Proof: See [181, pp. 64–68, 78], [340], [598, pp. 1, 2], [686, pp. 2–10, 74], and [1207]. Statement xxxi) is given in [242, p. 55]. Remark: Statement x) is Vandermonde’s identity.     α  0  Remark: For all α ∈ C, = 1. For all k ∈ N, = 1. 0 k Fact 1.9.2. The following inequalities hold: i) Let n ≥ 2. Then,

4n < n+1

ii) Let n ≥ 7. Then,

 n n 3

iii) Let 1 ≤ k ≤ n. Then,  n k k iv) Let 0 ≤ k ≤ n. Then,

  2n < 4n . n

< n!
1, then x  1 < 3. 2 < 1+ x Proof: See [686, p. 137]. Fact 1.11.15. Let x be a nonnegative number, and let p and q be real numbers such that 0 < p ≤ q. Then, −x  p  q  1 x x x ≤ 1+ ≤ 1+ ≤ ex. e 1+ p p q

28

CHAPTER 1

Furthermore, if p < q, then equality holds if and only if x = 0. Finally, q  x lim 1 + = ex. q→∞ q Proof: See [280, pp. 7, 8]. Remark: For q → ∞, (1 + 1/q)q = e + O(1/q), whereas (1 + 1/q)q [1 + 1/(2q)] = e + O(1/q 2 ). See [853]. Fact 1.11.16. Let x be a positive number. Then, "  x x 1 2x + 1 e< 1+ e < x+1 x 2x + 2 and

" 1+

1 −1/[12x(x+1)] 2x + 2 1/[6(2x+1)2 ] e e < x 2x + 1 e x <  1 + x1 " 2 1 < 1 + e−1/[3(2x+1) ]. x

Proof: See [1190]. Fact 1.11.17. e is given by  lim

q→∞



and lim

q→∞

q+1 q−1

q/2 =e

 qq (q − 1)q−1 = e. − (q − 1)q−1 (q − 2)q−2

Proof: These expressions are given in [1187] and [853], respectively. Fact 1.11.18. Let x be a positive number. Then,  3/4 1/2  1 2 1+ < 1 + 3x + 1 x + 15 5/8  1 < 1+ 5 1 4x + 3 e x 1 + x1

< 

 < 1+ Proof: See [946].

1 x+

1/2 1 6

.

29

PRELIMINARIES

Fact 1.11.19. Let n be a positive integer. If n ≥ 3, then n! < 2n(n−1)/2. If n ≥ 6, then

 n n 3

< n!
> 0. ii) If y ∈ Rn is nonzero and y ≥≥ 0, then ATy = 0. Equivalently, exactly one of the following two statements is satisfied: iii) There exists a vector x ∈ Rm such that Ax >> 0. iv) There exists a nonzero vector y ∈ Rn such that y ≥≥ 0 and AT y = 0. Proof: See [161, p. 47] or [243, p. 23]. Remark: This result is Gordan’s theorem.  |A(i,j) | for Fact 4.11.17. Let A ∈ Cn×n, and define |A| ∈ Rn×n by |A|(i,j) = all i, j ∈ {1, . . . , n}. Then,

sprad(A) ≤ sprad(|A|). Proof: See [1023, p. 619]. Fact 4.11.18. Let A ∈ Rn×n, assume that A is nonnegative, and let α ∈ [0, 1]. Then,

sprad(A) ≤ sprad αA + (1 − α)AT . Proof: See [134].

306

CHAPTER 4

Fact 4.11.19. Let A, B ∈ Rn×n, where 0 ≤≤ A ≤≤ B. Then, sprad(A) ≤ sprad(B). In particular, B0 ∈ R

m×m

is a principal submatrix of B, then sprad(B0 ) ≤ sprad(B).

If, in addition, A = B and A + B is irreducible, then sprad(A) < sprad(B). Hence, if sprad(A) = sprad(B) and A + B is irreducible, then A = B. Proof: See [174, p. 27]. See also [459, pp. 500, 501]. Fact 4.11.20. Let A, B ∈ Rn×n, assume that B is diagonal, assume that A and A + B are nonnegative, and let α ∈ [0, 1]. Then, sprad[αA + (1 − α)B] ≤ α sprad(A) + (1 − α) sprad(A + B). Proof: See [1177, p. 9-5]. Fact 4.11.21. Let A ∈ Rn×n, assume that A >> 0, and let λ ∈ spec(A)\{sprad(A)}. Then, |λ| ≤

Amax − Amin sprad(A), Amax + Amin

where

7 8  Amax = max A(i,j) : i, j = 1, . . . , n

and

7 8  Amin = min A(i,j) : i, j = 1, . . . , n .

Remark: This result is Hopf ’s theorem. Remark: The equality case is discussed in [706]. Fact 4.11.22. Let A ∈ Rn×n, assume that A is nonnegative and irreducible, and let x, y ∈ Rn, where x > 0 and y > 0 satisfy Ax = sprad(A)x and ATy = sprad(A)y. Then, k l   1 lim 1l A = xyT. l→∞ sprad(A) k=1

If, in addition, A is primitive, then k  1 A = xyT. lim k→∞ sprad(A) Proof: See [459, p. 503] and [728, p. 516]. Fact 4.11.23. Let A ∈ Rn×n, assume that A is nonnegative, and let k and m be positive integers. Then,

m tr Ak ≤ nm−1 tr Akm. Proof: See [885].

POLYNOMIAL MATRICES AND RATIONAL TRANSFER FUNCTIONS

307

Remark: This result is the JLL inequality.

4.12 Notes Much of the development in this chapter is based on [1108]. Additional discussions of the Smith and Smith-McMillan forms are given in [809] and [1535]. The proofs of Lemma 4.4.8 and Leverrier’s algorithm Proposition 4.4.9 are based on [1157, pp. 432, 433], where it is called the Souriau-Frame algorithm. Alternative proofs of Leverrier’s algorithm are given in [147, 739]. The proof of Theorem 4.6.1 is based on [728]. Polynomial-based approaches to linear algebra are given in [283, 521], while polynomial matrices and rational transfer functions are studied in [573, 1402]. The term normal rank is often used to refer to what we call the rank of a rational transfer function.

Chapter Five

Matrix Decompositions

In this chapter we present several matrix decompositions, namely, the Smith, multicompanion, elementary multicompanion, hypercompanion, Jordan, Schur, and singular value decompositions.

5.1 Smith Form The first decomposition involves rectangular matrices subject to a biequivalence transformation. This result is the specialization of the Smith decomposition given by Theorem 4.3.2 to constant matrices.  Theorem 5.1.1. Let A ∈ Fn×m and r = rank A. Then, there exist nonsingular matrices S1 ∈ Fn×n and S2 ∈ Fm×m such that   Ir 0r×(m−r) S2 . (5.1.1) A = S1 0(n−r)×r 0(n−r)×(m−r)

Corollary 5.1.2. Let A, B ∈ Fn×m. Then, A and B are biequivalent if and only if A and B have the same Smith form. Proposition 5.1.3. Let A, B ∈ Fn×m. Then, the following statements hold: i) A and B are left equivalent if and only if N(A) = N(B). ii) A and B are right equivalent if and only if R(A) = R(B). iii) A and B are biequivalent if and only if rank A = rank B. Proof. The proof of necessity is immediate in i)–iii). Sufficiency in iii) follows from Corollary 5.1.2. For sufficiency in i) and ii), see [1157, pp. 179–181].

5.2 Multicompanion Form For the monic polynomial p(s) = sn + βn−1sn−1 + · · · + β1s + β0 ∈ F[s] of degree n ≥ 1, the companion matrix C(p) ∈ Fn×n associated with p is defined to

310 be

CHAPTER 5

⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢  ⎢ C(p) = ⎢ ⎢ ⎢ ⎢ ⎢ ⎣

··· .. . .. . ..

.

..

0 −β2

0

1

0

0

0

1

0 .. .

0 .. .

0 .. .

0

0

−β0

−β1

0

0

0

0

0 .

0 .. .

···

0

1

···

−βn−2

−βn−1

⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥. ⎥ ⎥ ⎥ ⎥ ⎥ ⎦

(5.2.1)

If n = 1, then p(s) = s + β0 and C(p) = −β0 . Furthermore, if n = 0 and p = 1,  then we define C(p) = 00×0 . Note that, if n ≥ 1, then tr C(p) = −βn−1 and n det C(p) = (−1) β0 = (−1)n p(0). It is easy to see that the characteristic polynomial of the companion matrix C(p) associated with p is p. For example, let n = 3 so that ⎡ ⎤ 0 1 0 0 1 ⎦, C(p) = ⎣ 0 (5.2.2) −β0 −β1 −β2 and thus

⎡

s sI − C(p) = ⎣ 0 β0

⎤ −1 0 s −1 ⎦. β1 s + β2

(5.2.3)

Adding s times the second column and s2 times the third column to the first column leaves the determinant of sI − C(p) unchanged and yields ⎡ ⎤ 0 −1 0 ⎣ 0 s −1 ⎦. (5.2.4) p(s) β1 s + β2 

Hence, χC(p) = p. If n = 0 and p = 1, then we define χC(p) = χ00×0 = 1. The following result shows that companion matrices have the same characteristic polynomial and the same minimal polynomial. Proposition 5.2.1. Let p ∈ F[s] be a monic polynomial having degree n. Then, there exist unimodular matrices S1, S2 ∈ Fn×n [s] such that   0(n−1)×1 In−1 sI − C(p) = S1(s) (5.2.5) S2(s). 01×(n−1) p(s) Furthermore, χC(p) = μC(p) = p.

(5.2.6)

 Proof. Since χC(p) = p, it follows that rank[sI − C(p)] = n. Next, since det [sI − C(p)][n;1] = (−1)n−1, it follows that Δn−1 = 1, where Δn−1 is the greatest common divisor (which is monic by definition) of all (n−1)×(n−1) subdeterminants of sI − C(p). Furthermore, since Δi−1 divides Δi for all i ∈ {2, . . . , n −1}, it follows that Δ1 = · · · = Δn−2 = 1. Consequently, p1 = · · · = pn−1 = 1. Since, by

311

MATRIX DECOMPOSITIONS

Proposition 4.6.2, χC(p) = μC(p) = p.

6n i=1

pi = pn and μC(p) = pn , it follows that χC(p) =

Next, we consider block-diagonal matrices all of whose diagonally located blocks are companion matrices. Lemma 5.2.2. Let p1, . . . , pn ∈ F[s] be monic polynomials such that pi divides n  pi+1 for all i ∈ {1, . . . , n − 1} and n = i=1 deg pi . Furthermore, define C = diag[C(p1 ), . . . , C(pn )] ∈ Fn×n. Then, there exist unimodular matrices S1, S2 ∈ Fn×n [s] such that ⎡ ⎤ p1(s) 0 ⎢ ⎥ .. (5.2.7) sI − C = S1(s)⎣ ⎦S2 (s). . 0

pn(s)

Proof. Letting ki = deg pi , Proposition 5.2.1 implies that the Smith form of sIki − C(pi ) is 00×0 if ki = 0 and diag(Iki −1, pi ) if ki ≥ 1. Note that p1 =  n · · · = pn0 = 1, where n0 = i=1 max{0, ki − 1}. By combining these Smith forms and rearranging diagonal entries, it follows that there exist unimodular matrices S1, S2 ∈ Fn×n [s] such that ⎡ ⎤ sIk1 − C(p1 ) ⎢ ⎥ .. sI − C = ⎣ ⎦ . ⎡ ⎢ = S1(s)⎣

p1(s) .. 0

.

sIkn − C(pn ) ⎤ 0 ⎥ ⎦S2(s). pn(s)

Since pi divides pi+1 for all i ∈ {1, . . . , n − 1}, it follows that this diagonal matrix is the Smith form of sI − C. The following result uses Lemma 5.2.2 to construct a canonical form, known as the multicompanion form, for square matrices under a similarity transformation. Theorem 5.2.3. Let A ∈ Fn×n, and let p1, . . . , pn ∈ F[s] denote the similarity invariants of A, where pi divides pi+1 for all i ∈ {1, . . . , n − 1}. Then, there exists a nonsingular matrix S ∈ Fn×n such that ⎡ ⎤ C(p1 ) 0 ⎢ ⎥ −1 .. A = S⎣ (5.2.8) ⎦S . . 0

C(pn )

 Proof. Lemma 5.2.2 implies that the n × n matrix sI − C, where C = diag[C(p1 ), . . . , C(pn )], has the Smith form diag(p1, . . . , pn ). Now, since sI − A has the same similarity invariants as C, it follows from Theorem 4.3.10 that A and C are similar.

312

CHAPTER 5

Corollary 5.2.4. Let A ∈ Fn×n. Then, μA = χA if and only if A is similar to C(χA ). Proof. Suppose that μA = χA . Then, it follows from Proposition 4.6.2 that pi = 1 for all i ∈ {1, . . . , n − 1} and pn = χA is the only nonconstant similarity invariant of A. Thus, C(pi ) = 00×0 for all i ∈ {1, . . . , n − 1}, and it follows from Theorem 5.2.3 that A is similar to C(χA ). The converse follows from (5.2.6), xi) of Proposition 4.4.5, and Proposition 4.6.3. Corollary 5.2.5. Let A ∈ Fn×n be a companion matrix. Then, A = C(χA ) and μA = χA . Note that, if A = In , then the similarity invariants of A are pi(s) = s − 1 for all i ∈ {1, . . . , n}. Thus, C(pi ) = 1 for all i ∈ {1, . . . , n}, as expected. Corollary 5.2.6. Let A, B ∈ Fn×n. Then, the following statements are equivalent: i) A and B are similar. ii) A and B have the same similarity invariants. iii) A and B have the same multicompanion form. The multicompanion form given by Theorem 5.2.3 provides a canonical form for A in terms of a block-diagonal matrix of companion matrices. As shown below, however, the multicompanion form is only one such decomposition. The goal of the remainder of this section is to obtain an additional canonical form by applying a similarity transformation to the multicompanion form. To begin, note that, if Ai is similar to Bi for all i ∈ {1, . . . , r}, then diag(A1, . . . , Ar ) is similar to diag(B1, . . . , Br ). Therefore, it follows from Corollary 5.2.6 that, if sI − Ai and sI − Bi have the same Smith form for all i ∈ {1, . . . , r}, then sI − diag(A1, . . . , Ar ) and sI − diag(B1, . . . , Br ) have the same Smith form. The following lemma is needed. Lemma 5.2.7. Let A = diag(A1, A2 ), where Ai ∈ Fni ×ni for i = 1, 2. Then, μA is the least common multiple of μA1 and μA2 . In particular, if μA1 and μA2 are coprime, then μA = μA1 μA2 . Proof. Since 0 = μA(A) = diag[μA(A1 ), μA(A2 )], it follows that μA(A1 ) = 0 and μA(A2 ) = 0. Therefore, Theorem 4.6.1 implies that μA1 and μA2 both divide μA . Consequently, the least common multiple q of μA1 and μA2 also divides μA . Since q(A1 ) = 0 and q(A2 ) = 0, it follows that q(A) = 0. Therefore, μA divides q. Hence, q = μA . If, in addition, μA1 and μA2 are coprime, then μA = μA1 μA2 . Proposition 5.2.8. Let p ∈ F[s] be a monic polynomial of positive degree n, and let p = p1 · · · pr , where p1, . . . , pr ∈ F[s] are monic and pairwise coprime polynomials. Then, the matrices C(p) and diag[C(p1 ), . . . , C(pr )] are similar.

MATRIX DECOMPOSITIONS

313

 Proof. Let pˆ2 = p2 · · · pr and Cˆ = diag[C(p1 ), C(ˆ p2 )]. Since p1 and pˆ2 are coprime, it follows from Lemma 5.2.7 that μCˆ = μC(p1 ) μC(pˆ2 ) . Furthermore, χCˆ = χC(p1 ) χC(pˆ2 ) = μCˆ . Hence, Corollary 5.2.4 implies that Cˆ is similar to C(χCˆ ). However, χCˆ = p1 · · · pr = p, so that Cˆ is similar to C(p). If r > 2, then the same argument can be used to decompose C(ˆ p2 ) to show that C(p) is similar to diag[C(p1 ), . . . , C(pr )].

Proposition 5.2.8 can be used to decompose every companion block of a multicompanion form into smaller companion matrices. This procedure can be carried out for every companion block whose characteristic polynomial has coprime factors. For example, suppose that A ∈ R10×10 has the similarity invariants pi(s) = 1 for all i ∈ {1, . . . , 7}, p8 (s) = (s + 1)2 , p9 (s) = (s + 1)2 (s + 2), and p10 (s) = (s + 1)2 (s + 2)(s2 + 3), so that, by Theorem 5.2.3, the multicompanion form of A is diag[C(p8 ), C(p9 ), C(p10 )], where C(p8 ) ∈ R2×2, C(p9 ) ∈ R3×3, and C(p10 ) ∈ R5×5 . According to Proposition 5.2.8, the companion matrices C(p9 ) and C(p10 ) can be further decomposed. For example, C(p9 ) is similar to diag[C(p9,1 ), C(p9,2 )], where p9,1 (s) = (s + 1)2 and p9,2 (s) = s + 2 are coprime. Furthermore, C(p10 ) is similar to four different diagonal matrices, three of which have two companion blocks while the fourth has three companion blocks. Since p8 (s) = (s + 1)2 does not have nonconstant coprime factors, however, it follows that the companion matrix C(p8 ) cannot be decomposed into smaller companion matrices. The largest number of companion blocks achievable by similarity transformation is obtained by factoring every similarity invariant into elementary divisors, which are powers of irreducible polynomials that are nonconstant, monic, and pairwise coprime. In the above example, this factorization is given by p9 (s) = p9,1(s)p9,2 (s), where p9,1(s) = (s + 1)2 and p9,2 (s) = s + 2, and by p10 = p10,1 p10,2 p10,3 , where p10,1(s) = (s + 1)2 , p10,2 (s) = s + 2, and p10,3 (s) = s2 + 3. The elementary divisors of A are thus (s + 1)2 , (s + 1)2 , s + 2, (s + 1)2 , s + 2, and s2 + 3, which yields blocks. Viewing A ∈ Cn×n we can further √ six companion √ factor p10,3 (s) = (s+j 3)(s−j 3), which yields a total of seven companion blocks. From Proposition 5.2.8 and Theorem 5.2.3 we obtain the elementary multicompanion form, which provides another canonical form for A. Theorem 5.2.9. Let A ∈ Fn×n, and let q1l1 , . . . , qhlh ∈ F[s] be the elementary divisors of A, where l1, . . . , lh are positive integers. Then, there exists a nonsingular matrix S ∈ Fn×n such that ⎤ ⎡   0 C q1l1 ⎥ ⎢ ⎥ −1 ⎢ .. (5.2.9) A = S⎢ ⎥S . . ⎣   ⎦ 0 C qhlh

314

CHAPTER 5

5.3 Hypercompanion Form and Jordan Form We now present an alternative form of the companion blocks of the elementary multicompanion form (5.2.9). To do this we define the hypercompanion matrix Hl(q) associated with the elementary divisor q l ∈ F[s], where l is a positive integer, as follows. For q(s) = s − λ ∈ C[s], define the l × l Toeplitz hypercompanion matrix ⎤ ⎡ λ 1 0 ⎥ ⎢ 0 λ 1 0 ⎥ ⎢ ⎥ ⎢ . . .. .. ⎥ ⎢  ⎥, (5.3.1) Hl(q) = λIl + Nl = ⎢ ⎥ ⎢ .. ⎥ ⎢ . 1 0 ⎥ ⎢ ⎣ 0 λ 1 ⎦ 0 λ whereas, for q(s) = s2 − β1s − β0 companion matrix ⎡ 0 ⎢ β0 ⎢ ⎢ ⎢ ⎢  ⎢ Hl(q) = ⎢ ⎢ ⎢ ⎢ ⎣

∈ R[s], define the 2l × 2l real, tridiagonal hyper1 β1 0

0

⎤ 1 0 β0

0 1 β1 .. .

1 .. . .. .

..

.

0 β0

⎥ ⎥ ⎥ ⎥ ⎥ ⎥. ⎥ ⎥ ⎥ ⎥ 1 ⎦ β1

(5.3.2)

The following result shows   that the hypercompanion matrix Hl(q) is similar to the companion matrix C q l associated with the elementary divisor q l of Hl(q). Lemma 5.3.1. Let l ∈ P, and let q(s) = s − λ ∈ C[s] or q(s) = s2 − β1s − β  0 ∈ R[s]. Then, q l is the only elementary divisor of Hl(q), and Hl(q) is similar to C q l . the order of Hl(q). Then, χHl(q) = q l and  Proof. Let k denote k−1 det [sI − Hl(q)][k;1] = (−1) . Hence, as in the proof of Proposition 5.2.1, it follows   that χHl(q) = μHl(q) . Corollary 5.2.4 now implies that Hl(q) is similar to C ql . Proposition 5.2.8 and Lemma 5.3.1 yield the following canonical form, which is known as the hypercompanion form. Theorem 5.3.2. Let A ∈ Fn×n, and let q1l1 , . . . , qhlh ∈ F[s] be the elementary divisors of A, where l1, . . . , lh are positive integers. Then, there exists a nonsingular matrix S ∈ Fn×n such that ⎡ ⎤ 0 Hl1(q1 ) ⎢ ⎥ −1 .. (5.3.3) A = S⎣ ⎦S . . 0

Hlh(qh )

315

MATRIX DECOMPOSITIONS

Next, consider Theorem 5.3.2 with F = C. In this case, every elementary divisor qili is of the form (s − λi )li , where λi ∈ C. Furthermore, S ∈ Cn×n, and the hypercompanion form (5.3.3) is a block-diagonal matrix whose diagonally located blocks are of the form (5.3.1). The hypercompanion form (5.3.3) with every diagonally located block of the form (5.3.1) is the Jordan form, as given by the following result. Theorem 5.3.3. Let A ∈ Cn×n, and let q1l1 , . . . , qhlh ∈ C[s] be the elementary divisors of A, where l1, . . . , lh are positive integers and each of the polynomials q1, . . . , qh ∈ C[s] has degree 1. Then, there exists a nonsingular matrix S ∈ Cn×n such that ⎡ ⎤ 0 Hl1(q1 ) ⎢ ⎥ −1 .. (5.3.4) A = S⎣ ⎦S . . 0

Hlh(qh )

Corollary 5.3.4. Let p ∈ F[s], let λ1, . . . , λr denote the distinct roots of p,   and, for i = 1, . . . , r, let li = mp (λi ) and pi(s) = s − λi . Then, C(p) is similar to diag[Hl1(p1 ), . . . , Hlr(pr )]. To illustrate the structure of the Jordan form, let li = 3 and qi(s) = s − λi , where λi ∈ C. Then, Hli(qi ) is the 3 × 3 matrix ⎡ ⎤ λi 1 0 Hli(qi ) = λi I3 + N3 = ⎣ 0 λi 1 ⎦ (5.3.5) 0 0 λi so that mspec[Hli(qi )] = {λi , λi , λi }ms . If Hli(qi ) is the only diagonally located block of the Jordan form associated with the eigenvalue λi , then the algebraic multiplicity of λi is equal to 3, while its geometric multiplicity is equal to 1. Now, consider Theorem 5.3.2 with F = R. In this case, every elementary divisor qili is either of the form (s − λi )li or of the form (s2 − β1i s − β0i )li, where β0i , β1i ∈ R. Furthermore, S ∈ Rn×n, and the hypercompanion form (5.3.3) is a block-diagonal matrix whose diagonally located blocks are real matrices of the form (5.3.1) or (5.3.2). In this case, (5.3.3) is the real hypercompanion form. Applying an additional real similarity transformation to each diagonally located block of the real hypercompanion form yields the real Jordan form. To do this, define the real Jordan matrix Jl(q) for the positive integer l as follows. For  q(s) = s − λ ∈ F[s] define Jl(q) = Hl(q), whereas, if q(s) = s2 − β1s − β0 ∈ F[s] is irreducible with a nonreal root λ = ν + jω, then define the 2l × 2l upper Hessenberg

316

CHAPTER 5

matrix

⎡

ν

⎢ ⎢ −ω ⎢ ⎢ ⎢ ⎢ ⎢ ⎢  ⎢ Jl(q) = ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣

⎤

ω

1

0

ν

0

1

..

ν

ω

1

−ω

ν

.

0 .. .

0 ..

.

..

.

..

..

.

1

..

.

0

0

.

ν −ω

⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥. ⎥ 0 ⎥ ⎥ ⎥ 1 ⎥ ⎥ ω ⎥ ⎦ ν

(5.3.6)

Theorem 5.3.5. Let A ∈ Rn×n, and let q1l1 , . . . , qhlh ∈ R[s], where l1, . . . , lh are positive integers, are the elementary divisors of A. Then, there exists a nonsingular matrix S ∈ Rn×n such that ⎡ ⎤ 0 Jl1(q1 ) ⎢ ⎥ −1 .. (5.3.7) A = S⎣ ⎦S . . 0

Jlh(qh )

Proof. For the irreducible quadratic q(s) = s2 − β1s− β0 ∈ R[s], we show that Jl(q) and Hl(q) are similar. Writing q(s) = (s − λ)(s − λ), it follows from Theorem 5.3.3 that Hl(q) ∈ R2l×2l is similar to diag(λIl + Nl , λIl + Nl ). Next, by using a permutation similarity transformation, it follows that Hl(q) is similar to ⎡ ⎤ λ 0 1 0 ⎢ ⎥ ⎢ 0 λ 0 1 0 ⎥ 0 ⎢ ⎥ ⎢ ⎥ 0 λ 0 1 0 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ 0 λ 0 1 ⎢ ⎥ ⎢ ⎥ . . . .. .. .. ⎢ ⎥, ⎢ ⎥ ⎢ ⎥ .. .. ⎢ ⎥ . . 1 0 ⎢ ⎥ ⎢ ⎥ . .. 0 1 ⎥ ⎢ ⎢ ⎥ ⎢ ⎥ 0 λ 0 ⎣ ⎦ 0 λ 

ˆ . . . , S) ˆ to the above Finally, applying the similarity transformation S = diag(S,

−j −j  1 j 1 −1 ˆ ˆ matrix, where S = 1 −1 and S = 2 j −1 , yields Jl(q). Example 5.3.6. Let A, B ∈ R4×4 ⎡ 0 ⎢ 0 A=⎢ ⎣ 0 −16

and C ∈ C4×4 be given by ⎤ 1 0 0 0 1 0 ⎥ ⎥, 0 0 1 ⎦ 0 −8 0

317

MATRIX DECOMPOSITIONS

⎡

0 ⎢ −4 B=⎢ ⎣ 0 0 ⎡

and

j2 1 ⎢ 0 2j C=⎢ ⎣ 0 0 0 0

1 0 0 1 0 0 0 −4

⎤ 0 0 ⎥ ⎥, 1 ⎦ 0

⎤ 0 0 0 0 ⎥ ⎥. −j2 1 ⎦ 0 −j2

Then, A is in companion form, B is in real hypercompanion form, and C is in Jordan form. Furthermore, A, B, and C are similar. Example 5.3.7. Let A, B ∈ R6×6 and C ∈ C6×6 be given by ⎤ ⎡ 0 1 0 0 0 0 ⎢ 0 0 1 0 0 0 ⎥ ⎥ ⎢ ⎢ 0 0 0 1 0 0 ⎥ ⎥, A=⎢ ⎢ 0 0 0 0 1 0 ⎥ ⎥ ⎢ ⎣ 0 0 0 0 0 1 ⎦ −27 54 −63 44 −21 6 ⎡ ⎢ ⎢ ⎢ B=⎢ ⎢ ⎢ ⎣ and

⎡ ⎢ ⎢ ⎢ C=⎢ ⎢ ⎢ ⎣

0 −3 0 0 0 0

√ 1+j 2 1√ 0 1+j 2 0 0 0 0 0 0 0 0

1 0 2 1 0 0 0 −3 0 0 0 0

0 0 0 0 1 0 2 1 0 0 0 −3

0 0 1√ 0 0√ 1+j 2 0 1−j 2 0 0 0 0

0 0 0 0 1 2

⎤ ⎥ ⎥ ⎥ ⎥, ⎥ ⎥ ⎦

0 0 0 0 0 0 1√ 0 1√ 1−j 2 0 1−j 2

⎤ ⎥ ⎥ ⎥ ⎥. ⎥ ⎥ ⎦

Then, A is in companion form, B is in real hypercompanion form, and C is in Jordan form. Furthermore, A, B, and C are similar. The next result shows that every matrix is similar to its transpose by means of a symmetric similarity transformation. This result, which improves Corollary 4.3.11, is due to Frobenius. Corollary 5.3.8. Let A ∈ Fn×n. Then, there exists a symmetric, nonsingular matrix S ∈ Fn×n such that A = SATS −1. Proof. It follows from Theorem 5.3.3 that there exists a nonsingular matrix ˆ Sˆ−1, where B = diag(B1, . . . , Br ) is the Jordan form Sˆ ∈ Cn×n such that A = SB ni ×ni of A, and Bi ∈ C for all i ∈ {1, . . . , r}. Now, define the symmetric nonsingu-

318

CHAPTER 5

    lar matrix S = SˆI˜SˆT, where I˜ = diag Iˆn1 , . . . , Iˆnr is symmetric and involutory. T ˜ ˜ Furthermore, note that Iˆni Bi Iˆni = BT i for all i ∈ {1, . . . , r} so that IBI = B , and T˜ ˜ thus IB I = B. Hence, it follows that SATS −1 = SSˆ−TBTSˆTS −1 = SˆI˜SˆTSˆ−TBTSˆTSˆ−TI˜Sˆ−1 ˜ TI˜Sˆ−1 = SB ˆ Sˆ−1 = A. = SˆIB If A is real, then a similar argument based on the real Jordan form shows that S can be chosen to be real. An extension of Corollary 5.3.8 to the case in which A is normal is given by Fact 5.9.11. F

n×n

Corollary 5.3.9. Let A ∈ Fn×n. Then, there exist symmetric matrices S1, S2 ∈ such that S2 is nonsingular and A = S1S2 .

Proof. From Corollary 5.3.8 it follows that there exists a symmetric, nonsin  gular matrix S ∈ Fn×n such that A = SATS −1. Now, let S1 = SAT and S2 = S −1. T T Note that S2 is symmetric and nonsingular. Furthermore, S1 = AS = SA = S1, which shows that S1 is symmetric. Note that Corollary 5.3.8 follows from Corollary 5.3.9. If A = S1S2 , where S1, S2 are symmetric and S2 is nonsingular, then A = S2−1S2 S1S2 = S2−1ATS2 .

5.4 Schur Decomposition The Schur decomposition uses a unitary similarity transformation to transform an arbitrary square matrix into an upper triangular matrix. Theorem 5.4.1. Let A ∈ Cn×n. Then, there exist a unitary matrix S ∈ Cn×n and an upper triangular matrix B ∈ Cn×n such that A = SBS ∗.

(5.4.1)

Proof. Let λ1 ∈ C be an eigenvalue of A with associated eigenvector x ∈ Cn

 chosen such that x∗x = 1. Furthermore, let S1 = x Sˆ1 ∈ Cn×n be unitary, where Sˆ1 ∈ Cn×(n−1) satisfies Sˆ1∗S1 = In−1 and x∗Sˆ1 = 01×(n−1) . Then, S1e1 = x, and col1(S1−1AS1 ) = S1−1Ax = λ1S1−1x = λ1e1. Consequently,

 A = S1

λ1

C1

0(n−1)×1

A1

 S1−1,

where C1 ∈ C1×(n−1) and A1 ∈ C(n−1)×(n−1). Next, let S20 ∈ C(n−1)×(n−1) be a unitary matrix such that   λ2 C2 −1 A1 = S20 S20 , 0(n−2)×1 A2

319

MATRIX DECOMPOSITIONS

where C2 ∈ C1×(n−2) and A2 ∈ C(n−2)×(n−2). Hence, ⎡ ⎤ λ1 C11 C12 λ2 C2 ⎦S2−1S1, A = S1S2⎣ 0 0 0 A2

1 0  where C1 = C11 C12 , C11 ∈ C, and S2 = 0 S20 is unitary. Proceeding in a  similar manner yields (5.4.1) with S = S1S2 · · · Sn−1 , where S1, . . . , Sn−1 ∈ Cn×n are unitary. Since A and B in (5.4.1) are similar and B is upper triangular, it follows from Fact 4.10.10 that A and B have the same eigenvalues with the same algebraic multiplicities. The real Schur decomposition uses a real orthogonal similarity transformation to transform a real matrix into an upper Hessenberg matrix with real 1 × 1 and 2 × 2 diagonally located blocks. Corollary 5.4.2. Let A ∈ Rn×n, and let mspec(A) = {λ1, . . . , λr }ms ∪ {ν1 + jω1, ν1−jω1, . . . , νl +jωl , νl −jωl }ms , where λ1, . . . , λr ∈ R and, for all i ∈ {1, . . . , l}, νi , ωi ∈ R and ωi = 0. Then, there exists an orthogonal matrix S ∈ Rn×n such that A = SBS T,

(5.4.2)

where B is upper block triangular and the diagonally located blocks B1, . . . , Br ∈ R  ˆ l ∈ R2×2 of B satisfy Bi = ˆi ) = ˆ1, . . . , B [λi ] for all i ∈ {1, . . . , r} and spec(B and B {νi + jωi , νi − jωi } for all i ∈ {1, . . . , l}. Proof. The proof is analogous to the proof of Theorem 5.3.5. See also [728, p. 82]. Corollary 5.4.3. Let A ∈ Rn×n, and assume that the spectrum of A is real. Then, there exist an orthogonal matrix S ∈ Rn×n and an upper triangular matrix B ∈ Rn×n such that (5.4.3) A = SBS T. The Schur decomposition reveals the structure of range-Hermitian matrices and thus, as a special case, normal matrices.  Corollary 5.4.4. Let A ∈ Fn×n, and define r = rank A. Then, A is range Hermitian if and only if there exist a unitary matrix S ∈ Fn×n and a nonsingular matrix B ∈ Fr×r such that   B 0 S ∗. A=S (5.4.4) 0 0

In addition, A is normal if and only if there exist a unitary matrix S ∈ Cn×n and a diagonal matrix B ∈ Cr×r such that (5.4.4) is satisfied. ˆ is ˆ ∗, where B Proof. Suppose that A is range Hermitian, and let A = SBS n×n upper triangular and S ∈ F is unitary. Assume that A is singular, and choose ˆ(j,j) = B ˆ(j+1,j+1) = · · · = B ˆ(n,n) = 0 and such that all other diagonal S such that B

320

CHAPTER 5

ˆ are nonzero. Thus, rown (B) ˆ = 0, which implies that en ∈ R(B) ˆ ⊥. entries of B ∗ ˆ = R(B ˆ ), and thus en ∈ R(B ˆ ∗ )⊥. Since A is range Hermitian, it follows that R(B) ˆ which implies that coln (B) ˆ = 0. Therefore, it follows from (2.4.13) that en ∈ N(B), ˆ(n−1,n−1) = 0, then coln−1(B) ˆ = 0. Repeating this argument shows If, in addition, B ˆ has the form [ B 0 ], where B ∈ Fr×r is nonsingular. that B 0 0 ˆ ∗, where B ˆ ∈ Cn×n is Now, suppose that A is normal, and let A = SBS n×n upper triangular and S ∈ C is unitary. Since A is normal, it follows that ˆB ˆ∗ = B ˆ ∗B. ˆ Since B ˆ is upper triangular, it folAA∗ = A∗A, which implies that B ˆ ∗B) ˆ(1,1) B ˆB ˆ ∗ )(1,1) = row1(B)[row ˆ ˆ ∗ ˆ (1,1) = B ˆ(1,1) , whereas (B lows that (B 1(B)] = n ˆ ˆ∗ ˆ ˆ ˆ∗ ˆ ˆ i=1 B(1,i) B(1,i) . Since (B B)(1,1) = (BB )(1,1) , it follows that B(1,i) = 0 for all ˆ is i ∈ {2, . . . , n}. Continuing in a similar fashion row by row, it follows that B diagonal. Corollary 5.4.5. Let A ∈ Fn×n, assume that A is Hermitian, and define r = rank A. Then, there exist a unitary matrix S ∈ Fn×n and a diagonal matrix B ∈ Rr×r such that (5.4.4) is satisfied. In addition, A is positive semidefinite if and only if the diagonal entries of B are positive, and A is positive definite if and only if A is positive semidefinite and r = n. 

Proof. Corollary 5.4.4 and x), xi) of Proposition 4.4.5 imply that there exist a unitary matrix S ∈ Fn×n and a diagonal matrix B ∈ Rr×r such that (5.4.4) is satisfied. If A is positive semidefinite, then x∗Ax ≥ 0 for all x ∈ Fn. Choosing ∗ x = Sei , it follows that B(i,i) = eT i S ASei ≥ 0 for all i ∈ {1, . . . , r}. If A is positive definite, then r = n and B(i,i) > 0 for all i = 1, . . . , n. Proposition 5.4.6. Let A ∈ Fn×n be Hermitian. Then, there exists a nonsingular matrix S ∈ Fn×n such that ⎡ ⎤ −Iν−(A) 0 0 ⎢ ⎥ ∗ 0 0ν0 (A)×ν0 (A) 0 A = S⎣ (5.4.5) ⎦S . 0 0 Iν+(A) Furthermore, rank A = ν+(A) + ν−(A)

(5.4.6)

def A = ν0 (A).

(5.4.7)

and Proof. Since A is Hermitian, it follows from Corollary 5.4.5 that there exist ˆ Sˆ∗. a unitary matrix Sˆ ∈ Fn×n and a diagonal matrix B ∈ Rn×n such that A = SB Choose S to order the diagonal entries of B such that B = diag(B1, 0, −B2 ),  ˆ = where the diagonal matrices B1, B2 are both positive definite. Now, define B  ˆ 1/2DB ˆ 1/2, where D = diag(Iν (A) , 0ν (A)×ν (A) , diag(B1, I, B2 ). Then, B = B − 0 0 1/2 ˆ 1/2 ˆ∗ ˆ ˆ −Iν+(A) ). Hence, A = SB DB S . The following result is Sylvester’s law of inertia.

321

MATRIX DECOMPOSITIONS

Corollary 5.4.7. Let A, B ∈ Fn×n be Hermitian. Then, A and B are congruent if and only if In A = In B. Proposition 4.5.4 shows that two or more eigenvectors associated with distinct eigenvalues of a normal matrix are mutually orthogonal. Thus, a normal matrix has at least as many mutually orthogonal eigenvectors as it has distinct eigenvalues. The next result, which is an immediate consequence of Corollary 5.4.4, shows that every n × n normal matrix has n mutually orthogonal eigenvectors. In fact, the converse is also true. Corollary 5.4.8. Let A ∈ Cn×n. Then, A is normal if and only if A has n mutually orthogonal eigenvectors. The following result concerns the real normal form. Corollary 5.4.9. Let A ∈ Rn×n be range symmetric. Then, there exist an  rank A, orthogonal matrix S ∈ Rn×n and a nonsingular matrix B ∈ Rr×r, where r =   such that B 0 S T. A=S (5.4.8) 0 0 In addition, assume that A is normal, and let mspec(A) = {λ1, . . . , λr }ms ∪ {ν1 + jω1, ν1−jω1, . . . , νl +jωl, νl −jωl }ms , where λ1, . . . , λr ∈ R and, for all i ∈ {1, . . . , l, } νi , ωi ∈ R and ωi = 0. Then, there exists an orthogonal matrix S ∈ Rn×n such that A = SBS T,

(5.4.9)

   ˆ1, . . . , B ˆ l ), Bi = ˆi = where B = diag(B1, . . . , Br , B [λi ] for all i ∈ {1, . . . , r}, and B νi ωi [ −ωi νi ] for all i ∈ {1, . . . , l}.

5.5 Eigenstructure Properties Definition 5.5.1. Let A ∈ Fn×n, and let λ ∈ C. Then, the index of λ with respect to A, denoted by indA(λ), is the smallest nonnegative integer k such that



(5.5.1) R (λI − A)k = R (λI − A)k+1 . That is, indA(λ) = ind(λI − A).

(5.5.2)

Note that λ ∈ spec(A) if and only if indA(λ) = 0. Hence, 0 ∈ spec(A) if and only if ind A = indA(0) = 0. Proposition 5.5.2. Let A ∈ Fn×n, and let λ ∈ C. Then, indA(λ) is the smallest nonnegative integer k such that



rank (λI − A)k = rank (λI − A)k+1 . (5.5.3) Furthermore, ind A is the smallest nonnegative integer k such that     rank Ak = rank Ak+1 .

(5.5.4)

322

CHAPTER 5



k+1 . ConseProof. Corollary 2.4.2 implies that R (λI − A)k ⊆ R (λI − A)

k k+1 if and only if quently, that R (λI − A) = R (λI − A) Lemma 2.3.4 implies rank (λI − A)k = rank (λI − A)k+1 . Proposition 5.5.3. Let A ∈ Fn×n, and let λ ∈ spec(A). Then, the following statements hold: i) The order of the largest Jordan block of A associated with λ is indA(λ). ii) The number of Jordan blocks of A associated with λ is gmultA(λ). iii) The number of linearly independent eigenvectors of A associated with λ is gmultA(λ). iv) indA(λ) ≤ amultA(λ). v) indA(λ) = amultA(λ) if and only if exactly one block is associated with λ. vi) gmultA(λ) ≤ amultA(λ). vii) gmultA(λ) = amultA(λ) if and only if every block associated with λ is of order equal to 1. viii) indA(λ) + gmultA(λ) ≤ amultA(λ) + 1. ix) indA(λ) + gmultA(λ) = amultA(λ) + 1 if and only if at most one block associated with λ is of order greater than 1. Definition 5.5.4. Let A ∈ Fn×n, and let λ ∈ spec(A). Then, the following terminology is defined: i) λ is simple if amultA(λ) = 1. ii) A is simple if every eigenvalue of A is simple. iii) λ is cyclic (or nonderogatory) if gmultA(λ) = 1. iv) A is cyclic (or nonderogatory) if every eigenvalue of A is cyclic. v) λ is derogatory if gmultA(λ) > 1. vi) A is derogatory if A has at least one derogatory eigenvalue. vii) λ is semisimple if gmultA(λ) = amultA(λ). viii) A is semisimple if every eigenvalue of A is semisimple. ix) λ is defective if gmultA(λ) < amultA(λ). x) A is defective if A has at least one defective eigenvalue. xi) A is diagonalizable over C if A is semisimple. xii) A ∈ Rn×n is diagonalizable over R if A is semisimple and every eigenvalue of A is real. Proposition 5.5.5. Let A ∈ Fn×n, and let λ ∈ spec(A). Then, λ is simple if and only if λ is cyclic and semisimple.

MATRIX DECOMPOSITIONS

Proposition 5.5.6. Let A ∈ Fn×n, and let λ ∈ spec(A). Then,   def (λI − A)indA(λ) = amultA(λ).

323

(5.5.5)

Theorem

5.3.3 yields the following result, which shows that the subspaces N (λI − A)k , where λ ∈ spec(A) and k = indA(λ), provide a decomposition of Fn. Proposition 5.5.7. Let A ∈ Fn×n, let spec(A) = {λ1, . . . , λr }, and, for all  i ∈ {1, . . . , r}, let ki = indA(λi ). Then, the following statements hold:



i) N (λi I − A)ki ∩ N (λj I − A)kj = {0} for all i, j ∈ {1, . . . , r} such that i = j.

r ki ii) = Fn. i=1 N (λi I − A) Proposition 5.5.8. Let A ∈ Fn×n, and let λ ∈ spec(A). Then, the following statements are equivalent: i) λ is semisimple.

ii) def(λI − A) = def (λI − A)2 .

iii) N(λI − A) = N (λI − A)2 . iv) indA(λ) = 1. Proof. To prove that i) implies ii), suppose that λ is semisimple so that gmultA(λ) = amultA(λ), and thus def(λI − A) = amultA(λ). Then, it follows from

 Proposition 5.5.6 that def (λI − A)k = amultA(λ), where k = indA(λ).

Therefore, (λI − A) 2 ≤ it follows from Corollary 2.5.7 that amultA(λ) = def(λI − A) ≤ def def (λI − A)k = amultA(λ), which implies that def(λI − A) = def (λI − A)2 . To prove that ii) implies iii), note that it follows from Corollary 2.5.7 that

N(λI − A) ⊆ N (λI − A)2 . Since, by ii), these subspaces have equal dimension, it follows from Lemma 2.3.4 that these subspaces are equal. To prove that iii) implies iv), note

that iii) implies 2ii),

and thus rank(λI−A) = 2 n − def(λI − A) = n − def (λI − A) = rank (λI − A) . Therefore, since R(λI −



A) ⊆ R (λI − A)2 it follows from Corollary 2.5.7 that R(λI − A) = R (λI − A)2 . Finally, since λ ∈ spec(A), it follows from Definition 5.5.1 that indA(λ) = 1. Finally, to prove that iv) implies i), note that iv) is equivalent to the fact that every Jordan block of A associated with λ has order 1, which is equivalent to the fact that the geometric multiplicity of λ is equal to the algebraic multiplicity of λ, that is, that λ is semisimple. Corollary 5.5.9. Let A ∈ Fn×n. Then, A is group invertible if and only if ind A ≤ 1.

324

CHAPTER 5

Proposition 5.5.10. Assume that A, B ∈ Fn×n are similar. Then, the following statements hold: i) mspec(A) = mspec(B). ii) For all λ ∈ spec(A), gmultA(λ) = gmultB (λ). Proposition 5.5.11. Let A ∈ Fn×n. Then, A is semisimple if and only if A is similar to a normal matrix. The following result is an extension of Corollary 5.3.9. Proposition 5.5.12. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A is semisimple, and spec(A) ⊂ R. ii) There exists a positive-definite matrix S ∈ Fn×n such that A = SA∗S −1. iii) There exist a Hermitian matrix S1 ∈ Fn×n and a positive-definite matrix S2 ∈ Fn×n such that A = S1S2 . Proof. To prove that i) implies ii), let Sˆ ∈ Fn×n be a nonsingular matrix ˆ Sˆ−1 , where B ∈ Rn×n is diagonal. Then, B = Sˆ−1ASˆ = Sˆ∗A∗Sˆ−∗. such that A = SB  ˆ ˆ∗ ˆ Sˆ∗A∗Sˆ−∗ )Sˆ−1 = (SˆSˆ∗ )A∗ (SˆSˆ∗ )−1 = SA∗S −1, where S = ˆ SS Hence, A = SB Sˆ−1 = S( ∗ −1 is positive definite. To show that ii) implies iii), note that A = SA S = S1S2 ,  ∗ where S1 = SA∗ and S2 = S −1. Since S1∗ = (SA∗ ) = AS ∗ = AS = SA∗ = S1, it follows that S1 is Hermitian. Furthermore, since S is positive definite, it follows that S −1, and hence S2 , is alsopositive definite. Finally, to prove that iii) implies  −1/2

i), note that A = S1S2 = S2

1/2

1/2

S2 S1S2

1/2

1/2

1/2

S2 . Since S2 S1S2

is Hermitian, it

1/2 1/2 S2 S1S2

is unitarily similar to a real diagonal follows from Corollary 5.4.5 that matrix. Consequently, A is semisimple and spec(A) ⊂ R. If a matrix is block triangular, then the following result shows that its eigenvalues and their algebraic multiplicity are determined by the diagonally located blocks. If, in addition, the matrix is block diagonal, then the geometric multiplicities of its eigenvalues are determined by the diagonally located blocks. Proposition 5.5.13. Let A ∈ Fn×n, assume that A is partitioned as A = ⎤ A11 ··· A1k ⎣ .. · ·. · .. ⎦, where, for all i, j ∈ {1, . . . , k}, Aij ∈ Fni ×nj , and let λ ∈ spec(A). . . . ⎡

Ak1 ··· Akk

Then, the following statements hold: i) If Aii is the only nonzero block in the ith column of blocks, then amultAii (λ) ≤ amultA(λ).

(5.5.6)

ii) If A is upper block triangular or lower block triangular, then amultA(λ) =

r  i=1

amultAii (λ)

(5.5.7)

325

MATRIX DECOMPOSITIONS

and mspec(A) =

k 9

mspec(Aii ).

(5.5.8)

i=1

iii) If Aii is the only nonzero block in the ith column of blocks, then gmultAii (λ) ≤ gmultA(λ).

(5.5.9)

iv) If A is upper block triangular, then gmultA11 (λ) ≤ gmultA(λ).

(5.5.10)

v) If A is lower block triangular, then gmultAkk (λ) ≤ gmultA(λ).

(5.5.11)

vi) If A is block diagonal, then gmultA(λ) =

r 

gmultAii (λ).

(5.5.12)

i=1  Proposition 5.5.14. Let A ∈ Fn×n, let spec(A) = {λ1, . . . , λr }, and let ki = indA(λi ) for all i ∈ {1, . . . , r}. Then,

μA(s) =

r !

(s − λi )ki

(5.5.13)

i=1

and deg μA =

r 

ki .

(5.5.14)

i=1

Furthermore, the following statements are equivalent: i) μA = χA . ii) A is cyclic. iii) For all λ ∈ spec(A), the Jordan form of A contains exactly one block associated with λ. iv) A is similar to C(χA ). Proof. Let A = SBS −1, where B = diag(B1, . . . , Bnh ) denotes the Jordan form of A given by (5.3.4). Let λi ∈ spec(A), and let Bj be a Jordan block associated with λi . Then, the order of Bj is less than or equal to ki . Consequently, (Bj − λi I)ki = 0. Next, let p(s) denote the right-hand side of (5.5.13). Thus,   r r ! ! ki ki p(A) = (A − λi I) = S (B − λi I) S −1 i=1

= S diag



i=1

r !

 r ! ki (B1 − λi I) , . . . , (Bnh − λi I) S −1 = 0.

i=1

ki

i=1

326

CHAPTER 5

Therefore, it follows from Theorem 4.6.1 that μA divides p. Furthermore, note that, if ki is replaced by kˆi < ki , then p(A) = 0. Hence, p is the minimal polynomial of A. The equivalence of i) and ii) is now immediate, while the equivalence of ii) and iii) follows from Theorem 5.3.5. The equivalence of i) and iv) is given by Corollary 5.2.4. Example 5.5.15. The standard nilpotent matrix Nn is in companion form, and thus is cyclic. In fact, Nn consists of a single Jordan block, and χNn (s) = μNn (s) = sn. 1 1 Example 5.5.16. The matrix −1 1 is normal but is neither symmetric nor 0 1 skew symmetric, while the matrix −1 0 is normal but is neither symmetric nor semisimple with real eigenvalues. 1 0 11 Example 5.5.17. The matrices −1 1 2 −1 and [ 0 2 ] are diagonalizable over R but not normal, while the matrix −2 1 is diagonalizable but is neither normal nor diagonalizable over R. Example 5.5.18. The product of the Hermitian matrices [ 12 21 ] and has no real eigenvalues.

2

1 1 −2

0 1 1 0 5.5.19. The matrices [ 10 02 ] and −2 3 are similar, whereas [ 0 1 ]

Example 0 1 and −1 2 have the same spectrum but are not similar. Proposition 5.5.20. Let A ∈ Fn×n. Then, the following statements hold: i) A is singular if and only if 0 ∈ spec(A). ii) A is group invertible if and only if either A is nonsingular or 0 ∈ spec(A) is semisimple. iii) A is Hermitian if and only if A is normal and spec(A) ⊂ R. iv) A is skew Hermitian if and only if A is normal and spec(A) ⊂ jR. v) A is positive semidefinite if and only if A is normal and spec(A) ⊂ [0, ∞). vi) A is positive definite if and only if A is normal and spec(A) ⊂ (0, ∞). vii) A is unitary if and only if A is normal and spec(A) ⊂ {λ ∈ C: |λ| = 1}. viii) A is shifted unitary if and only if A is normal and spec(A) ⊂ {λ ∈ C: |λ − 12 | = 12 }.

(5.5.15)

ix) A is involutory if and only if A is semisimple and spec(A) ⊆ {−1, 1}. x) A is skew involutory if and only if A is semisimple and spec(A) ⊆ {−j, j}. xi) A is idempotent if and only if A is semisimple and spec(A) ⊆ {0, 1}. xii) A is skew idempotent if and only if A is semisimple and spec(A) ⊆ {0, −1}. xiii) A is tripotent if and only if A is semisimple and spec(A) ⊆ {−1, 0, 1}. xiv) A is nilpotent if and only if spec(A) = {0}.

MATRIX DECOMPOSITIONS

327

xv) A is unipotent if and only if spec(A) = {1}. xvi) A is a projector if and only if A is normal and spec(A) ⊆ {0, 1}. xvii) A is a reflector if and only if A is normal and spec(A) ⊆ {−1, 1}. xviii) A is a skew reflector if and only if A is normal and spec(A) ⊆ {−j, j}. xix) A is an elementary projector if and only if A is normal and mspec(A) = {0, 1, . . . , 1}ms. xx) A is an elementary reflector if and only if A is normal and mspec(A) = {−1, 1, . . . , 1}ms . If, furthermore, A ∈ F2n×2n, then the following statements hold: xxi) If A is Hamiltonian, then mspec(A) = mspec(−A).   xxii) If A is symplectic, then mspec(A) = mspec A−1 . The following result is a consequence of Proposition 5.5.12 and Proposition 5.5.20. Corollary 5.5.21. Let A ∈ Fn×n, and assume that A is either involutory, idempotent, skew idempotent, tripotent, a projector, or a reflector. Then, the following statements hold: i) There exists a positive-definite matrix S ∈ Fn×n such that A = SA∗S −1. ii) There exist a Hermitian matrix S1 ∈ Fn×n and a positive-definite matrix S2 ∈ Fn×n such that A = S1S2 . Proposition 5.5.22. Let A, B ∈ Fn×n. Then, the following statements hold: i) Assume that A and B are normal. Then, A and B are unitarily similar if and only if mspec(A) = mspec(B). ii) Assume that A and B are projectors. Then, A and B are unitarily similar if and only if rank A = rank B. iii) Assume that A and B are (projectors, reflectors). Then, A and B are unitarily similar if and only if tr A = tr B. iv) Assume that A and B are semisimple. Then, A and B are similar if and only if mspec(A) = mspec(B). v) Assume that A and B are (involutory, skew involutory, idempotent). Then, A and B are similar if and only if tr A = tr B. vi) Assume that A and B are idempotent. Then, A and B are similar if and only if rank A = rank B. vii) Assume that A and B are tripotent. Then, A and B are similar if and only if rank A = rank B and tr A = tr B.

328

CHAPTER 5

5.6 Singular Value Decomposition The third matrix decomposition that we consider is the singular value decomposition. Unlike the Jordan and Schur decompositions, the singular value decomposition applies to matrices that are not necessarily square. Let A ∈ Fn×m, where A = 0, and consider the positive-semidefinite matrices AA∗ ∈ Fn×n and A∗A ∈ Fm×m. It follows from Proposition 4.4.10 that AA∗ and A∗A have the same nonzero eigenvalues with the same algebraic multiplicities. Since AA∗ and A∗A are positive semidefinite, it follows that they have the same positive eigenvalues with the same algebraic multiplicities. Furthermore, since AA∗ is Hermitian, it follows that the number of positive eigenvalues of AA∗ (or A∗A), counting algebraic multiplicity, is equal to the rank of AA∗ (or A∗A). Since rank A = rank AA∗ = rank A∗A, it thus  follows that AA∗ and A∗A both have r positive eigenvalues, where r = rank A. Definition 5.6.1. Let A ∈ Fn×m. Then, the singular values of A are the min{n, m} nonnegative numbers σ1(A), . . . , σmin{n,m} (A), where, for all i ∈ {1, . . . , min{n, m}}, 1/2  1/2 σi (A) = λi (AA∗ ) = λi (A∗A). (5.6.1) Hence, σ1(A) ≥ · · · ≥ σmin{n,m}(A) ≥ 0.

(5.6.2)



Let A ∈ Fn×m, and define r = rank A. If 1 ≤ r < min{n, m}, then σ1(A) ≥ · · · ≥ σr (A) > σr+1(A) = · · · = σmin{n,m}(A) = 0,

(5.6.3)

whereas, if r = min{m, n}, then σ1(A) ≥ · · · ≥ σr (A) = σmin{n,m}(A) > 0.

(5.6.4)

Consequently, rank A is the number of positive singular values of A. For convenience, define

and, if n = m,

 σ1(A) σmax (A) =

(5.6.5)

 σn(A). σmin (A) =

(5.6.6)

If n = m, then σmin (A) is not defined. By convention, we define σmax (0n×m ) = σmin (0n×n ) = 0,

(5.6.7)

and, for all i ∈ {1, . . . , min{n, m}}, σi (A) = σi (A∗ ) = σi (A) = σi (AT ).

(5.6.8)

Now, suppose that n = m. If A is Hermitian, then, for all i ∈ {1, . . . , n}, σi (A) = |λi (A)|,

(5.6.9)

while, if A is positive semidefinite, then, for all i ∈ {1, . . . , n}, σi (A) = λi (A).

(5.6.10)

MATRIX DECOMPOSITIONS

329

Proposition 5.6.2. Let A ∈ Fn×m. If n ≤ m, then the following statements are equivalent: i) rank A = n. ii) σn(A) > 0. If m ≤ n, then the following statements are equivalent: iii) rank A = m. iv) σm(A) > 0. If n = m, then the following statements are equivalent: v) A is nonsingular. vi) σmin (A) > 0. We now state the singular value decomposition.  Theorem 5.6.3. Let A ∈ Fn×m, assume that A is nonzero, let r = rank A, and  define B = diag[σ1(A), . . . , σr (A)]. Then, there exist unitary matrices S1 ∈ Fn×n and S2 ∈ Fm×m such that   B 0r×(m−r) S2 . (5.6.11) A = S1 0(n−r)×r 0(n−r)×(m−r)

Furthermore, each column of S1 is an eigenvector of AA∗ , while each column of S2∗ is an eigenvector of A∗A. Proof. For convenience, assume that r < min{n, m}, since otherwise the zero matrices become empty matrices. By Corollary 5.4.5 there exists a unitary matrix   2 U ∈ Fn×n such that B 0 U ∗. AA∗ = U 0 0

Partition U = U1 U2 , where U1 ∈ Fn×r and U2 ∈ Fn×(n−r). Since U ∗ U = In, it

 follows that U1∗U1 = Ir and U1∗U = Ir 0r×(n−r) . Now, define V1 = A∗ U1B −1 ∈ m×r F , and note that   2 0 B ∗ −1 ∗ ∗ −1 −1 ∗ U ∗ U1B −1 = Ir . V1 V1 = B U1AA U1B = B U1 U 0 0

Next, note that, since U2∗U = 0(n−r)×r In−r , it follows that  

B2 0 U ∗ = 0. U2∗AA∗ = 0 I 0 0 However, since R(A) = R(AA∗ ), it follows that U2∗A = 0. Finally, let V2 ∈

 Fm×(m−r) be such that V = V1 V2 ∈ Fm×m is unitary. Hence, we have   ∗   

B 0 V1 B 0 ∗ V = U1 U2 U = U1BV1∗ = U1BB −1 U1∗A 0 0 0 0 V2∗ = U1U1∗A = (U1U1∗ + U2 U2∗ )A = U U ∗A = A,

330

CHAPTER 5

which yields (5.6.11) with S1 = U and S2 = V ∗. An immediate corollary of the singular value decomposition is the polar decomposition. Corollary 5.6.4. Let A ∈ Fn×n. Then, there exists a positive-semidefinite matrix M ∈ Fn×n and a unitary matrix S ∈ Fn×n such that A = MS.

(5.6.12)

Proof. It follows from the singular value decomposition that there exist unitary matrices S1, S2 ∈ Fn×n and a diagonal positive-definite matrix B ∈ Fr×r,  rank A, such that A = S1[ B0 00 ]S2 . Hence, where r =   B 0 S1∗S1S2 = MS, A = S1 0 0 



where M = S1[ B0 00 ]S1∗ is positive semidefinite and S = S1S2 is unitary. 

Proposition 5.6.5. Let A ∈ Fn×m, let r = rank A, and define the Hermitian



T  matrix A = A0∗ A0 ∈ F(n+m)×(n+m). Then, In A = r 0 r , and the 2r nonzero eigenvalues of A are the r positive singular values of A and their negatives.   Proof. Since χA(s) = det s2I − A∗A , it follows that mspec(A)\{0, . . . , 0}ms = {σ1(A), −σ1(A), . . . , σr (A), −σr (A)}ms .



5.7 Pencils and the Kronecker Canonical Form Let A, B ∈ Fn×m, and define the polynomial matrix PA,B ∈ Fn×m [s], called a pencil, by  PA,B (s) = sB − A. The pencil PA,B is regular if rank PA,B = min{n, m} (see Definition 4.2.4). Otherwise, PA,B is singular.

PA,B

Let A, B ∈ Fn×m. Since PA,B ∈ Fn×m we define the generalized spectrum of by  spec(A, B) = Szeros(PA,B ) (5.7.1)

and the generalized multispectrum of PA,B by  mSzeros(PA,B ). mspec(A, B) =

(5.7.2)

Furthermore, the elements of spec(A, B) are the generalized eigenvalues of PA,B . The structure of a pencil is illuminated by the following result known as the Kronecker canonical form.

331

MATRIX DECOMPOSITIONS

Theorem 5.7.1. Let A, B ∈ Cn×m. Then, there exist nonsingular matrices S1 ∈ Cn×n and S2 ∈ Cm×m such that, for all s ∈ C, PA,B (s) = S1 diag(sIr1 − A1, sB2 − Ir2 , [sIk1−Nk1 − ek1 ], . . . , [sIkp−Nkp − ekp ], [sIl1−Nl1 − el1 ]T, . . . , [sIlq−Nlq − elq ]T, 0t×u )S2 ,

(5.7.3)

where A1 ∈ Cr1 ×r1 is in Jordan form, B2 ∈ Rr2 ×r2 is nilpotent and in Jordan form, k1, . . . , kp , l1, . . . , lq are positive integers, and [sIl −Nl −el ] ∈ Cl×(l+1). Furthermore, rank PA,B = r1 + r2 +

p 

ki +

i=1

q 

li .

(5.7.4)

i=1

Proof. See [68, Chapter 2], [555, Chapter XII], [809, pp. 395–398], [891], [897, pp. 128, 129], and [1261, Chapter VI]. In Theorem 5.7.1, note that n = r1 + r2 +

p 

ki +

i=1

q 

li + q + t

(5.7.5)

li + p + u.

(5.7.6)

i=1

and m = r1 + r2 +

p 

ki +

i=1

q  i=1

Proposition 5.7.2. Let A, B ∈ Cn×m, and consider the notation of Theorem 5.7.1. Then, PA,B is regular if and only if t = u = 0 and either p = 0 or q = 0. Let A, B ∈ Fn×m, and let λ ∈ C. Then, rank PA,B (λ) = rank(λI − A1 ) + r2 +

p 

ki +

i=1

q 

li .

(5.7.7)

i=1

Note that λ is a generalized eigenvalue of PA,B if and only if rank PA,B (λ) < rank PA,B . Consequently, λ is a generalized eigenvalue of PA,B if and only if λ is an eigenvalue of A1 , that is, spec(A, B) = spec(A1 ).

(5.7.8)

mspec(A, B) = mspec(A1 ).

(5.7.9)

Furthermore, The generalized algebraic multiplicity amultA,B (λ) of λ ∈ spec(A, B) is defined by 

amultA,B (λ) = amultA1(λ).

(5.7.10)

It can be seen that, for λ ∈ spec(A, B), 

gmultA1(λ) = rank PA,B − rank PA,B (λ). The generalized geometric multiplicity gmultA,B (λ) of λ ∈ spec(A, B) is defined by 

gmultA,B (λ) = gmultA1(λ).

(5.7.11)

332

CHAPTER 5

Now, assume that A, B ∈ Fn×n, that is, A and B are square, which, from (5.7.5) and (5.7.6), is equivalent to q +t = p+u. Then, the characteristic polynomial χA,B ∈ F[s] of (A, B) is defined by 

χA,B (s) = det PA,B (s) = det(sB − A). Proposition 5.7.3. Let A, B ∈ Fn×n. Then, the following statements hold: i) PA,B is singular if and only if χA,B = 0. ii) PA,B is singular if and only if deg χA,B = −∞. iii) PA,B is regular if and only if χA,B is not the zero polynomial. iv) PA,B is regular if and only if 0 ≤ deg χA,B ≤ n. v) If PA,B is regular, then multχA,B (0) = n − deg χB,A . vi) deg χA,B = n if and only if B is nonsingular. vii) If B is nonsingular, then χA,B = χB −1A , spec(A, B) = spec(B −1A), and mspec(A, B) = mspec(B −1A). viii) roots(χA,B ) = spec(A, B). ix) mroots(χA,B ) = mspec(A, B). x) If A or B is nonsingular, then PA,B is regular. xi) If all of the generalized eigenvalues of (A, B) are real, then PA,B is regular. xii) If PA,B is regular, then N(A) ∩ N(B) = {0}. xiii) If PA,B is regular, then there exist nonsingular matrices S1, S2 ∈ Cn×n such that, for all s ∈ C,      0 0 I A1 − S2 , (5.7.12) PA,B (s) = S1 s r 0 B2 0 In−r  deg χA,B , A1 ∈ Cr×r is in Jordan form, and B2 ∈ R(n−r)×(n−r) where r = is nilpotent and in Jordan form. Furthermore,

χA,B = χA1 ,

(5.7.13)

roots(χA,B ) = spec(A1 ), mroots(χA,B ) = mspec(A1 ).

(5.7.14) (5.7.15)

Proof. See [897, p. 128] and [1261, Chapter VI]. Statement xiii) is the Weierstrass canonical form for a square, regular pencil.

MATRIX DECOMPOSITIONS

333

Proposition 5.7.4. Let A, B ∈ Fn×n, assume that A is positive semidefinite, and assume that B is Hermitian. Then, the following statements hold: i) PA,B is regular. ii) There exists α ∈ F such that A + αB is nonsingular. iii) N(A) ∩ N(B) = {0}. A ]) = {0}. iv) N([ B

v) There exists nonzero α ∈ F such that N(A) ∩ N(B + αA) = {0}. vi) For all nonzero α ∈ F, N(A) ∩ N(B + αA) = {0}. vii) All generalized eigenvalues of (A, B) are real. If, in addition, B is positive semidefinite, then the following statement is equivalent to i)–vii): viii) There exists β > 0 such that βB < A. Proof. The results i) =⇒ ii) and ii) =⇒ iii) are immediate. Next, Fact 2.10.10 and Fact 2.11.3 imply that iii), iv), v), and vi) are equivalent. Next, to prove iii) =⇒ vii), let λ ∈ C be a generalized eigenvalue of (A, B). Since λ = 0 is real, suppose λ = 0. Since det(λB − A) = 0, let nonzero θ ∈ Cn satisfy (λB − A)θ = 0, and thus it follows that θ∗Aθ = λθ∗Bθ. Furthermore, note that θ∗Aθ and θ∗Bθ are real. Now, suppose θ ∈ N(A). Then, it follows from (λB − A)θ = 0 that θ ∈ N(B), which contradicts N(A) ∩ N(B) = {0}. Hence, θ ∈ N(A), and thus θ∗Aθ > 0 and, consequently, θ∗Bθ = 0. Hence, it follows that λ = θ∗Aθ/θ∗Bθ, and thus λ is real. Hence, all generalized eigenvalues of (A, B) are real. Next, to prove vii) =⇒ i), let λ ∈ C\R so that λ is not a generalized eigenvalue of (A, B). Consequently, χA,B (s) is not the zero polynomial, and thus (A, B) is regular. Next, to prove i)–vii) =⇒ viii), let θ ∈ Rn be nonzero, and note that N(A) ∩ N(B) = {0} implies that either Aθ = 0 or Bθ = 0. Hence, either θTAθ > 0 or θTBθ > 0. Thus, θT(A + B)θ > 0, which implies A + B > 0 and hence −B < A. Finally, to prove viii) =⇒ i)–vii), let β ∈ R be such that βB < A, so that βθ Bθ < θTAθ for all nonzero θ ∈ Rn. Next, suppose θˆ ∈ N(A) ∩ N(B) is nonzero. Hence, Aθˆ = 0 and B θˆ = 0. Consequently, θˆTB θˆ = 0 and θˆTAθˆ = 0, which ˆ Thus, N(A) ∩ N(B) = {0}. contradicts β θˆTB θˆ < θˆTAθ. T

334

CHAPTER 5

5.8 Facts on the Inertia Fact 5.8.1. Let A ∈ Fn×n, and assume that A is idempotent. Then, rank A = sig A = tr A and

⎤ 0 In A = ⎣ n − tr A ⎦. tr A ⎡

Fact 5.8.2. Let A ∈ Fn×n, and assume that A is involutory. Then, rank A = n, sig A = tr A, and

⎡

⎤ − tr A) ⎢ ⎥ 0 In A = ⎣ ⎦. 1 2 (n + tr A) 1 2 (n

Fact 5.8.3. Let A ∈ Fn×n, and assume that A is tripotent. Then, rank A = tr A2, sig A = tr A, and

  ⎤ tr A2 − tr A ⎥ ⎢ n − tr A2 In A = ⎣ ⎦.   1 2 2 tr A + tr A ⎡

1 2

Fact 5.8.4. Let A ∈ Fn×n, and assume that A is either skew Hermitian, skew involutory, or nilpotent. Then, sig A = ν−(A) = ν+(A) = 0 and

⎤ 0 In A = ⎣ n ⎦. 0 ⎡

Fact 5.8.5. Let A ∈ Fn×n, assume that A is group invertible, and assume that spec(A) ∩ jR ⊆ {0}. Then, rank A = ν−(A) + ν+(A) and def A = ν0 (A) = amultA(0). Fact 5.8.6. Let A ∈ Fn×n, and assume that A is Hermitian. Then, rank A = ν−(A) + ν+(A)

335

MATRIX DECOMPOSITIONS

⎡

and

⎤

ν−(A)

⎡

⎥ ⎢ ⎢ In A = ⎣ ν0 (A) ⎦ = ⎣

1 2 (rank A

n − rank A 1 2 (rank A

ν+(A)

− sig A)

⎤ ⎥ ⎦.

+ sig A)

Fact 5.8.7. Let A, B ∈ Fn×n, and assume that A and B are Hermitian. Then, In A = In B if and only if rank A = rank B and sig A = sig B. Fact 5.8.8. Let A ∈ Fn×n, assume that A is Hermitian, and let A0 be a principal submatrix of A. Then, ν−(A0 ) ≤ ν−(A) and

ν+(A0 ) ≤ ν+(A).

Proof: See [792]. Fact 5.8.9. Let A ∈ Fn×n, and assume that A is positive semidefinite. Then, rank A = sig A = ν+(A) ⎡

and

0

⎤

⎢ ⎥ In A = ⎣ def A ⎦. rank A Fact 5.8.10. Let A ∈ Fn×n, and assume that A is positive semidefinite. Then, ⎤ ⎡ 0 In A = ⎣ def A ⎦. rank A If, in addition, A is positive definite, then ⎡

⎤ 0 In A = ⎣ 0 ⎦. n

Fact 5.8.11. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A is an elementary projector. ii) A is a projector, and tr A = n − 1.  0  iii) A is a projector, and In A = 1 . n−1

Furthermore, the following statements are equivalent: iv) A is an elementary reflector. v) A is a reflector, and tr A = n − 2.  1  vi) A is a reflector, and In A = 0 . n−1

Proof: See Proposition 5.5.20.

336

CHAPTER 5

Fact 5.8.12. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A + A∗ is positive definite. ii) For all Hermitian matrices B ∈ Fn×n, In B = In AB. Proof: See [287]. Fact 5.8.13. Let A, B ∈ Fn×n, assume that AB and B are Hermitian, and assume that spec(A) ∩ [0, ∞) = ∅. Then, In(−AB) = In B. Proof: See [287]. Fact 5.8.14. Let A, B ∈ Fn×n, assume that A and B are Hermitian and nonsingular, and assume that spec(AB) ∩ [0, ∞) = ∅. Then, ν+(A) + ν+(B) = n. Proof: Use Fact 5.8.13. See [287]. Remark: Weaker versions of this result are given in [784, 1063]. Fact 5.8.15. Let A ∈ Fn×n, assume that A is Hermitian, and let S ∈ Fm×n. Then, ν−(SAS ∗ ) ≤ ν−(A) and

ν+(SAS ∗ ) ≤ ν+(A).

Furthermore, consider the following conditions: i) rank S = n. ii) rank SAS ∗ = rank A. iii) ν−(SAS ∗ ) = ν−(A) and ν+(SAS ∗ ) = ν+(A). Then, i) =⇒ ii) ⇐⇒ iii). Proof: See [459, pp. 430, 431] and [521, p. 194]. Fact 5.8.16. Let A ∈ Fn×n, assume that A is Hermitian, and let S ∈ Fm×n. Then, ν−(SAS ∗ ) + ν+(SAS ∗ ) = rank SAS ∗ ≤ min{rank A, rank S}, ν−(A) + rank S − n ≤ ν−(SAS ∗ ) ≤ ν−(A), ν+(A) + rank S − n ≤ ν+(SAS ∗ ) ≤ ν+(A). Proof: See [1087]. Fact 5.8.17. Let A, S ∈ Fn×n, assume that A is Hermitian, and assume that S is nonsingular. Then, there exist α1, . . . , αn ∈ [λmin (SS ∗ ), λmax (SS ∗ )] such that, for all i ∈ {1, . . . , n}, λi (SAS ∗ ) = αi λi (A).

337

MATRIX DECOMPOSITIONS

Proof: See [1473]. Remark: This result, which is due to Ostrowski, is a quantitative version of Sylvester’s law of inertia given by Corollary 5.4.7. Fact 5.8.18. Let A, S ∈ Fn×n, assume that A is Hermitian, and assume that S is nonsingular. Then, the following statements are equivalent: i) In(SAS ∗ ) = In A. ii) rank(SAS ∗ ) = rank A. iii) R(A) ∩ N(A) = {0}. Proof: See [112]. Fact 5.8.19. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, and assume that A is positive definite and C is negative definite. Then, ⎡ ⎤ ⎡ ⎤ n A B 0 In ⎣ B ∗ C 0 ⎦ = ⎣ m ⎦. 0 0 0l×l l Proof: This result follows from Fact 5.8.6. See [792]. Fact 5.8.20. Let A ∈ Rn×m. Then,     AA∗ 0 0 A = In In 0 −A∗A A∗ 0   0 AA+ = In 0 −A+A ⎤ ⎡ rank A = ⎣ n + m − 2 rank A ⎦. rank A Proof: See [459, pp. 432, 434]. Fact 5.8.21. Let A ∈ Cn×n, assume that A is Hermitian, and let B ∈ Cn×m. Then, ⎤ ⎡   rank B A B ≥≥ ⎣ n − rank B ⎦. In B∗ 0 rank B Furthermore, if R(A) ⊆ R(B), then  In

A B∗

B 0



⎤ rank B = ⎣ n + m − 2 rank B ⎦. rank B ⎡

Finally, if rank B = n, Then,  In

A B∗

B 0



⎤ n = ⎣ m − n ⎦. n ⎡

338

CHAPTER 5

Proof: See [459, pp. 433, 434] or [970]. Remark: Extensions are given in [970]. Remark: See Fact 8.15.28. Fact 5.8.22. Let A ∈ Fn×n. Then, there exist a nonsingular matrix S ∈ Fn×n and a skew-Hermitian matrix B ∈ Fn×n such that ⎛⎡ ⎤ ⎞ 0 0 Iν−(A+A∗ ) ⎜⎢ ⎥ ⎟ ∗ 0 0ν0 (A+A∗ )×ν0 (A+A∗ ) 0 A = S⎝ ⎣ ⎦ + B⎠S . 0

−Iν+(A+A∗ )

0

Proof: Write A = 12 (A+A∗ )+ 12 (A−A∗ ), and apply Proposition 5.4.6 to 12 (A+A∗ ).

5.9 Facts on Matrix Transformations for One Matrix Fact 5.9.1. Define S ∈ C3×3 by ⎡

0  1 ⎣ √ S= √0 2 2

⎤ 1 1 −j j ⎦. 0 0

Then, S is unitary, and K(e3 ) = S diag(0, j, −j)S −1. Remark: See Fact 5.9.2. Fact 5.9.2. Let x ∈ R3 , assume that either x(1) = 0 or x(2) = 0, and define ⎤ ⎡ a 1  ⎣ b ⎦= x, x 2 c  α=

 a2 c2 + b2 c2 + 2 − 3c2 + c4 , ⎡

and

a

⎢  ⎢ S= ⎣ b c

−b+jac α a+jbc α

−b−jac α a−jbc α

−j(1−c2 ) α

j(1−c2 ) α

⎤ ⎥ ⎥. ⎦

Then, α ≥ min{|a|, |b|} > 0, S is unitary, and K(x) = S diag(0, jx2 , −jx2 )S −1. Proof: See [855, p. 154]. Remark: If x(1) = x(2) = 0, then a = b = 0, c = 1, and α = 0. This case is considered by Fact 5.9.1. Problem: Find a decomposition of K(x) that holds for all x ∈ R3 .

339

MATRIX DECOMPOSITIONS

Fact 5.9.3. Let A ∈ Fn×n, and assume that spec(A) = {1}. Then, Ak is similar to A for all k ≥ 1. Fact 5.9.4. Let A ∈ Fn×n, and assume there exists a nonsingular matrix −1 S  ∈ Fn×n

 such that S AS is upper triangular. Then, for all r ∈ {1, . . . , n}, Ir R S 0 is an invariant subspace of A. Remark: Analogous results hold for lower triangular matrices and block-triangular matrices. Fact 5.9.5. Let A ∈ Fn×n. Then, there exist unique matrices B, C ∈ Fn×n such that the following properties are satisfied: i) B is diagonalizable over F. ii) C is nilpotent. iii) A = B + C. iv) BC = CB. Furthermore, mspec(A) = mspec(B). Proof: See [709, p. 112] or [746, p. 74]. Existence follows from the real Jordan form. The last statement follows from Fact 5.17.4. Remark: This result is the S-N decomposition or the Jordan-Chevalley decomposition. Fact 5.9.6. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A is similar to a skew-Hermitian matrix. ii) A is semisimple, and spec(A) ⊂ jR. Remark: See Fact 11.18.12. 

Fact 5.9.7. Let A ∈ Fn×n, and let r = rank A. Then, A is group invertible if and only if there exist a nonsingular matrix B ∈ Fr×r and a nonsingular matrix   S ∈ Fn×n such that B 0 S −1. A=S 0 0 

Fact 5.9.8. Let A ∈ Fn×n, and let r = rank A. Then, A is range Hermitian if and only if there exist a nonsingular matrix S ∈ Fn×n and a nonsingular matrix   B ∈ Fr×r such that B 0 S ∗. A=S 0 0 Remark: S need not be unitary for sufficiency. See Corollary 5.4.4.  ˆ SR, where Sˆ is unitary Proof: Use the QR decomposition Fact 5.15.9 to let S = and R is upper triangular. See [1309].

340

CHAPTER 5

Fact 5.9.9. Let A ∈ Fn×n. Then, there exists an involutory matrix S ∈ Fn×n such that AT = SAS T. Proof: See [430] and [591]. Remark: Note AT rather than A∗. Fact 5.9.10. Let A ∈ Fn×n. Then, there exists a nonsingular matrix S ∈ Fn×n such that A = SA∗S −1 if and only if there exist Hermitian matrices S1, S2 ∈ Fn×n such that A = S1S2 . Proof: See [1526, pp. 215, 216]. Remark: See Proposition 5.5.12. Remark: An analogous result in Hilbert space is given in [779] under the assumption that A is normal. Fact 5.9.11. Let A ∈ Fn×n, and assume that A is normal. Then, there exists a symmetric, nonsingular matrix S ∈ Fn×n such that AT = SAS −1 and such that S −1 = S. Proof: For F = C, let A = UBU ∗, where U is unitary and B is diagonal. Then,  AT = SAS = SAS −1, where S = U U −1. For F = R, use the real normal form and   T ˜ ˆ . . . , I). ˆ let S = UIU , where U is orthogonal and I˜ = diag(I, Remark: See Corollary 5.3.8. Fact 5.9.12. Let A ∈ Rn×n, and assume that A is normal. Then, there exists a reflector S ∈ Rn×n such that AT = SAS −1. Consequently, A and AT are orthogonally similar. Finally, if A is skew symmetric, then A and −A are orthogonally similar. Proof: Specialize Fact 5.9.11 to the case F = R. Fact 5.9.13. Let A ∈ Fn×n. Then, there exists a reverse-symmetric, nonsinˆ gular matrix S ∈ Fn×n such that AT = SAS −1. Proof: This result follows from Corollary 5.3.8. See [907]. Fact 5.9.14. Let A ∈ Fn×n. Then, there exist reverse-symmetric matrices S1, S2 ∈ Fn×n such that S2 is nonsingular and A = S1S2 . Proof: This result follows from Corollary 5.3.9. See [907]. Fact 5.9.15. Let A ∈ Rn×n, and assume that A is not of the form aI, where a ∈ R. Then, A is similar to a matrix with diagonal entries 0, . . . , 0, tr A. Proof: See [1125, p. 77].

MATRIX DECOMPOSITIONS

341

Remark: This result is due to Gibson. Fact 5.9.16. Let A ∈ Rn×n, and assume that A is not zero. Then, A is similar to a matrix whose diagonal entries are all nonzero. Proof: See [1125, p. 79]. Remark: This result is due to Marcus and Purves. Fact 5.9.17. Let A ∈ Rn×n, and assume that A is symmetric. Then, there exists an orthogonal matrix S ∈ Rn×n such that −1 ∈ / spec(S) and SAS T is diagonal. Proof: See [1125, p. 101]. Remark: This result is due to Hsu. Fact 5.9.18. Let A ∈ Rn×n, and assume that A is symmetric. Then, there exist a diagonal matrix B ∈ Rn×n and a skew-symmetric matrix C ∈ Rn×n such that A = [2(I + C)−1 − I]B[2(I + C)−1 − I]T. Proof: Use Fact 5.9.17. See [1125, p. 101]. Fact 5.9.19. Let A ∈ Fn×n. Then, there exists a unitary matrix S ∈ Fn×n such that S ∗AS has equal diagonal entries. Proof: See [501] or [1125, p. 78], or use Fact 5.9.20. Remark: The diagonal entries are equal to (tr A)/n. Remark: This result is due to Parker. See [549]. Fact 5.9.20. Let A ∈ Fn×n. Then, the following statements are equivalent: i) tr A = 0. ii) There exist matrices B, C ∈ Fn×n such that A = [B, C]. iii) A is unitarily similar to a matrix whose diagonal entries are zero. Proof: See [14, 549, 822, 838] or [641, p. 146]. Remark: This result is Shoda’s theorem. Remark: See Fact 5.9.21. Fact 5.9.21. Let R ∈ Fn×n, and assume that R is Hermitian. Then, the following statements are equivalent: i) tr R < 0. ii) R is unitarily similar to a matrix all of whose diagonal entries are negative. iii) There exists an asymptotically stable matrix A ∈ Fn×n such that R = A + A∗. Proof: See [124].

342

CHAPTER 5

Remark: See Fact 5.9.20. Fact 5.9.22. Let A ∈ Fn×n. Then, AA∗ and A∗A are unitarily similar. Fact 5.9.23. Let A ∈ Fn×n, and assume that A is idempotent. Then, A and A are unitarily similar. ∗

Proof: This result follows from Fact 5.9.29 and the fact that [ 10 a0 ] and [ a1 00 ] are unitarily similar. See [429]. Fact 5.9.24. Let A ∈ Fn×n, and assume that A is symmetric. Then, there exists a unitary matrix S ∈ Fn×n such that A = SBS T, where

 diag[σ1(A), . . . , σn(A)]. B=

Proof: See [728, p. 207]. Remark: A is symmetric, complex, and T-congruent to B. Fact 5.9.25. Let A ∈ Fn×n. Then, Proof: Use the unitary transformation Fact 5.9.26. Let n be a positive ⎧  −In/2 ⎪ ⎪ ⎪ S ⎪ ⎪ 0 ⎪ ⎪ ⎨ ⎡ Iˆn = −In/2 ⎪ ⎪ ⎪ ⎢ ⎪ ⎪ S 0 ⎣ ⎪ ⎪ ⎩ 0 where

A

0 0 −A



√1 I −I 2 I I

0 A ] are unitarily similar. and [ A 0

.

integer. Then,  0 S T, n even, −In/2 ⎤ 0 0 ⎥ 1 0 ⎦S T, n odd, 0 In/2

 ⎧  ˆn/2 − I I ⎪ n/2 1 ⎪ √ ⎪ , n even, ⎪ ⎪ ⎪ 2 Iˆn/2 In/2 ⎪ ⎨  ⎡ ⎤ S= In/2 0 −Iˆn/2 ⎪ ⎪ √ ⎪ ⎢ ⎥ ⎪ √1 ⎣ ⎪ 0 2 0 ⎦, n odd. ⎪ 2 ⎪ ⎩ Iˆn/2 0 In/2

Therefore, mspec(Iˆn ) =

⎧ ⎨{−1, 1, . . . , −1, 1}ms,

n even,

⎩{1, −1, 1, . . . , −1, 1} , n odd. ms

Remark: For even n, Fact 3.20.3 shows that Iˆn is Hamiltonian, and thus, by Fact 4.9.22, mspec(In ) = − mspec(In ). Remark: See [1444].

343

MATRIX DECOMPOSITIONS

Fact 5.9.27. Let n be a positive integer. Then,   0 jIn S ∗, J2n = S 0 −jIn 

where 

S= Hence,

√1 2

I

−I

jI

−jI

 .

mspec(J2n ) = {j, −j, . . . , j, −j}ms

and det J2n = 1. Proof: See Fact 2.19.3. Remark: Fact 3.20.3 shows that J2n is Hamiltonian, and thus, by Fact 4.9.22, mspec(J2n ) = − mspec(J2n ).  rank A. Fact 5.9.28. Let A ∈ Fn×n, assume that A is idempotent, and let r = r×(n−r) n×n Then, there exists a matrix B ∈ F and a unitary matrix S ∈ F such that   B Ir S ∗. A=S 0 0(n−r)×(n−r)

Proof: See [550, p. 46]. 

Fact 5.9.29. Let A ∈ Fn×n, assume that A is idempotent, and let r = rank A. Then, there exist a unitary matrix S ∈ Fn×n and positive numbers a1, . . . , ak such that      1 ak 1 a1 ,..., , Ir−k , 0(n−r−k)×(n−r−k) S ∗. A = Sdiag 0 0 0 0 Proof: See [429]. Remark: This result provides a canonical form for idempotent matrices under unitary similarity. See also [551]. Remark: See Fact 5.9.23. 

Fact 5.9.30. Let A ∈ Fn×m, assume that A is nonzero, let r = rank A, define B = diag[σ1(A), . . . , σr (A)], and let S1 ∈ Fn×n and S2 ∈ Fm×m be unitary matrices such that   B 0r×(m−r) A = S1 S2 . 0(n−r)×r 0(n−r)×(m−r) 

Then, there exist K ∈ Fr×r and L ∈ Fr×(m−r) such that KK ∗ + LL∗ = Ir

344

CHAPTER 5

and

 A = S1



BK

BL

0(n−r)×r

0(n−r)×(m−r)

S1∗.

Proof: See [119, 668]. Remark: See Fact 6.3.14 and Fact 6.6.16. Fact 5.9.31. Let A ∈ Fn×n, assume that A is unitary, and partition A as   A11 A12 , A= A21 A22 where A11 ∈ Fm×k, A12 ∈ Fm×q, A21 ∈ Fp×k, A22 ∈ Fp×q, and m + p = k + q = n. Then, there exist unitary matrices U, V ∈ Fn×n and nonnegative integers l, r such that ⎡ ⎤ Ir 0 0 0 0 0 ⎢ 0 Γ ⎥ 0 0 Σ 0 ⎢ ⎥ ⎢ ⎥ ⎢ 0 0 0 0 0 Im−r−l ⎥ ⎢ ⎥ A = U⎢ ⎥V, ⎢ 0 0 ⎥ 0 Iq−m+r 0 0 ⎢ ⎥ ⎢ ⎥ 0 0 −Γ 0 ⎣ 0 Σ ⎦ 0

0

Ik−r−l

0

0

0

where Γ, Σ ∈ Rl×l are diagonal and satisfy 0 < Γ(l,l) ≤ · · · ≤ Γ(1,1) < 1,

(5.9.1)

0 < Σ(1,1) ≤ · · · ≤ Σ(l,l) < 1,

(5.9.2)

and

Γ 2 + Σ 2 = Im .

Proof: See [550, p. 12] and [1261, p. 37]. Remark: This result is the CS decomposition. See [1086, 1088]. The entries Σ(i,i) and Γ(i,i) can be interpreted as sines and cosines, respectively, of the principal angles between a pair of subspaces S1 = R(X1 ) and S2 = R(Y1 ) such that [X1 X2 ] and [Y1 Y2 ] are unitary and A = [X1 X2 ]∗ [Y1 Y2 ]; see [550, pp. 25–29], [1261, pp. 40–43], and Fact 2.9.19. Principal angles can also be defined recursively; see [550, p. 25] and [551]. See also [821].  Fact 5.9.32. Let A ∈ Fn×n, and let r = rank A. Then, there exist S1 ∈ Fn×r, r×r n×r B ∈ R , and S2 ∈ F such that S1 is left inner, S2 is right inner, B is upper  6r triangular, I ◦ B = αI, where α = i=1 σi(A), and

A = S1BS2 . Proof: See [780]. Remark: Note that B is real. Remark: This result is the geometric mean decomposition.

345

MATRIX DECOMPOSITIONS

Fact 5.9.33. Let A ∈ Cn×n. Then, there exists a matrix B ∈ Rn×n such that AA and B 2 are similar. Proof: See [425].

5.10 Facts on Matrix Transformations for Two or More Matrices Fact 5.10.1. denote a root of q  0 H1(q) = β0

 2 Let q(s) = s − β1s − β0 ∈ R[s] be irreducible, and let λ = ν + jω so that β1 = 2ν and β0 = −(ν 2 + ω 2 ). Then,      1 0 ν ω 1 0 1 = SJ1(q)S −1. = −ν/ω 1/ω −ω ν ν ω β1

The matrix S = [ ν1 ω0 ] is not unique; an alternative choice is S =

ω transformation ν 2 2 . Similarly, 0 ν +ω ⎤ ⎡ ⎡ ⎤ ν ω 1 0 0 1 0 0 ⎢ ⎢ β 0 β1 1 0 1 ⎥ 0 ⎥ ⎥S −1 = SJ2 (q)S −1, ⎥ = S⎢ −ω ν H2 (q) = ⎢ ⎣ 0 ⎣ 0 0 ν ω ⎦ 0 0 1 ⎦ 0 0 −ω ν 0 0 β 0 β1 where

⎡

ω ⎢  ⎢ 0 S= ⎣ 0 0

ν ν 2 + ω2 0 0

⎤ ω ν ω ν 2 + ω2 + ν ⎥ ⎥. ⎦ −2ων 2ω 2 −2ω(ν 2 + ω 2 ) 0

 2 Fact 5.10.2. Let q(s) = s − 2νs + ν 2 + ω 2 ∈ R[s] with roots λ = ν + jω and λ = ν − jω. Then,        λ 0 1 1 1 −j ν ω 1 1 √ = √2 H1(q) = 2 j −j 1 j −ω ν 0 λ

and

where

⎡

ν ⎢ −ω H2 (q) = ⎢ ⎣ 0 0 ⎡

1 0 ⎢ 0  1 ⎢ j S = √2 ⎣ 0 1 0 j

ω ν 0 0

1 0 ν −ω

⎤ 1 0 −j 0 ⎥ ⎥, 0 1 ⎦ 0 −j

⎤ ⎡ λ 1 0 ⎢ 0 λ 1 ⎥ ⎥ = S⎢ ⎣ 0 0 ω ⎦ ν 0 0 ⎡

S −1

⎤ 0 0 0 0 ⎥ ⎥S −1, λ 1 ⎦ 0 λ

1 −j ⎢ 0 0 = √12⎢ ⎣ 1 j 0 0

⎤ 0 0 1 −j ⎥ ⎥. 0 0 ⎦ 1 j

Fact 5.10.3. Left equivalence, right equivalence, biequivalence, unitary left equivalence, unitary right equivalence, and unitary biequivalence are equivalence relations on Fn×m. Similarity, congruence, and unitary similarity are equivalence relations on Fn×n.

346

CHAPTER 5

Fact 5.10.4. Let A, B ∈ Fn×m. Then, A and B are in the same equivalence class of Fn×m induced by biequivalent transformations if and only if A and B are biequivalent to [ I0 00 ]. Now, let n = m. Then, A and B are in the same equivalence class of Fn×n induced by similarity transformations if and only if A and B have the same Jordan form. Fact 5.10.5. Let A, B ∈ Fn×n, and assume that A and B are similar. Then, A is semisimple if and only if B is. Fact 5.10.6. Let A ∈ Fn×n, and assume that A is normal. Then, A is unitarily similar to its Jordan form. Fact 5.10.7. Let A, B ∈ Fn×n, assume that A and B are normal, and assume that A and B are similar. Then, A and B are unitarily similar. Proof: Since A and B are similar, it follows that mspec(A) = mspec(B). Since A and B are normal, it follows that they are unitarily similar to the same diagonal matrix. See Fact 5.10.6. See [642, p. 104]. Remark: See [555, p. 8] for related results. 

Fact 5.10.8. Let A, B ∈ Fn×n, and let r = 2n2. Then, the following statements are equivalent: i) A and B are unitarily similar.

 ii) For all k1, . . . , kr , l1, . . . , lr ∈ N such that ri,j=1 (ki + lj ) ≤ r, it follows that tr Ak1Al1 ∗ · · · AkrAlr ∗ = tr B k1B l1 ∗ · · · B krB lr ∗. Proof: See [1103]. Remark: See [812, pp. 71, 72] and [224, 1221]. Remark: The number of distinct tuples of positive integers whose sum is a positive 2n2 2 integer k is 2k−1. The number of expressions in ii) is thus k=1 2k−1 = 4n − 1. Because of properties of the trace function, the number of distinct expressions is less than this number. Furthermore, in special cases, the number of expressions that need to be checked is significantly less than the number of distinct expressions. In the case n = 2, it suffices to check three equalities, specifically, tr A = tr B, tr A2 = tr B 2, and tr A∗A = tr B ∗B. In the case n = 3, it suffices to check 7 equalities. See [224, 1221]. Fact 5.10.9. Let A, B ∈ Fn×n, assume that A and B are idempotent, assume that sprad(A − B) < 1, and define

−1/2  S= (AB + A⊥B⊥ ) I − (A − B)2 . Then, the following statements hold: i) S is nonsingular. ii) If A = B, then S = I.

347

MATRIX DECOMPOSITIONS

−1/2 iii) S −1 = (BA + B⊥A⊥ ) I − (B − A)2 . iv) A and B are similar. In fact, A = SBS −1. v) If A and B are projectors, then S is unitary and A and B are unitarily similar. Proof: See [708, p. 412]. Remark: [I − (A − B)2 ]−1/2 is defined by ix) of Fact 10.13.1. Fact 5.10.10. Let A, B ∈ Fn×n, and assume that A and B are idempotent. Then, the following statements are equivalent: i) A and B are unitarily similar. ii) tr A = tr B and, for all i ∈ {1, . . . , n/2}, tr (AA∗ )i = tr (BB ∗ )i. iii) χAA∗ = χBB ∗. Proof: This result follows from Fact 5.9.29. See [429]. Fact 5.10.11. Let A, B ∈ Fn×n, and assume that either A or B is nonsingular. Then, AB and BA are similar. Proof: If A is nonsingular, then AB = A(BA)A−1, whereas, if B is nonsingular, then BA = B(AB)B −1. Fact 5.10.12. Let A, B ∈ Fn×n, and assume that A and B are projectors. Then, AB and BA are unitarily similar. Remark: This result is due to Dixmier. See [1141]. Fact 5.10.13. Let A ∈ Fn×n. Then, A is idempotent if and only if there exists an orthogonal matrix B ∈ Fn×n such that A and B are similar. Fact 5.10.14. Let A, B ∈ Fn×n, assume that A and B are idempotent, and assume that A + B − I is nonsingular. Then, A and B are similar. In particular, A = (A + B − I)−1B(A + B − I). Fact 5.10.15. Let A1, . . . , Ar ∈ Fn×n, and assume that Ai Aj = Aj Ai for all i, j ∈ {1, . . . , r}. Then, * r ! ni dim span Ai : 0 ≤ ni ≤ n − 1 for all i ∈ {1, . . . , r} ≤ 14 n2 + 1. i=1

Remark: This result gives a bound on the dimension of a commutative subalgebra. Remark: This result is due to Schur. See [884]. Fact 5.10.16. Let A, B ∈ Fn×n, and assume that AB = BA. Then, 8 7 dim span AiB j : 0 ≤ i ≤ n − 1, 0 ≤ j ≤ n − 1 ≤ n.

348

CHAPTER 5

Remark: This result gives a bound on the dimension of a commutative subalgebra generated by two matrices. Remark: This result is due to Gerstenhaber. See [154, 884]. Fact 5.10.17. Let A, B ∈ Fn×n, and assume that A and B are normal, nonsingular, and congruent. Then, In A = In B. Remark: This result is due to Ando. Fact 5.10.18. Let A, B ∈ Fn×m. Then, the following statements hold: i) The matrices A and B are unitarily left equivalent if and only if A∗A = B ∗B. ii) The matrices A and B are unitarily right equivalent if and only if AA∗ = BB ∗. iii) The matrices A and B are unitarily biequivalent if and only if A and B have the same singular values with the same multiplicity. Proof: See [734] and [1157, pp. 372, 373]. Remark: In [734] A and B need not be the same size. Remark: The singular value decomposition provides a canonical form under unitary biequivalence in analogy with the Smith form under biequivalence. Remark: Note that AA∗ = BB ∗ implies that R(A) = R(B), which implies right equivalence. This is an alternative proof of the immediate fact that unitary right equivalence implies right equivalence. Fact 5.10.19. Let A, B ∈ Fn×n. Then, the following statements hold: i) A∗A = B ∗B if and only if there exists a unitary matrix S ∈ Fn×n such that A = SB. ii) A∗A ≤ B ∗B if and only if there exists a matrix S ∈ Fn×n such that A = SB and S ∗S ≤ I. iii) A∗B + B ∗A = 0 if and only if there exists a unitary matrix S ∈ Fn×n such that (I − S)A = (I + S)B. iv) A∗B + B ∗A ≥ 0 if and only if there exists a matrix S ∈ Fn×n such that (I − S)A = (I + S)B and S ∗S ≤ I. Proof: See [728, p. 406] and [1144]. Remark: Statements iii) and iv) follow from i) and ii) by replacing A and B with A − B and A + B, respectively. Fact 5.10.20. Let A ∈ Fn×n, B ∈ Fm×m, and C ∈ Fn×m. Then, there exist matrices X, Y ∈ Fn×m satisfying AX + Y B + C = 0 

if and only if rank

A 0 0 −B



 = rank

A C 0 −B

 .

349

MATRIX DECOMPOSITIONS

Proof: See [1125, pp. 194, 195] and [1437]. Remark: AX + Y B + C = 0 is a generalization of Sylvester’s equation. See Fact 5.10.21. Remark: This result is due to Roth. Remark: An explicit expression for all solutions is given by Fact 6.5.7, which applies to the case in which A and B are not necessarily square and thus X and Y are not necessarily the same size. Fact 5.10.21. Let A ∈ Fn×n, B ∈ Fm×m, and C ∈ Fn×m. Then, there exists a matrix X ∈ Fn×m satisfying AX + XB + C = 0 if and only if the matrices



A 0 0 −B

are similar. In this case,    I A C = 0 0 −B



 ,

X I



A C 0 −B

A 0 0 −B





I 0

−X I

 .

Proof: See [1437]. For sufficiency, see [892, pp. 422–424] or [1125, pp. 194, 195]. Remark: AX +XB+C = 0 is Sylvester’s equation. See Proposition 7.2.4, Corollary 7.2.5, and Proposition 11.9.3. Remark: This result is due to Roth. See [221]. Fact 5.10.22. Let A, B ∈ Fn×n, and assume that A and B are idempotent. Then, the matrices     A+B 0 A+B A , 0 −A − B 0 −A − B are similar. In fact,    I A+B A = 0 0 −A − B

X I



A+B 0

0 −A − B



I 0

−X I

 ,

 1 where X = 4 (I + A − B).

Remark: This result is due to Tian. Remark: See Fact 5.10.21. Fact 5.10.23. Let A ∈ Fn×n, B ∈ Fm×m, and C ∈ Fn×m, and assume that A and B are nilpotent. Then, the matrices     A 0 A C , 0 B 0 B are similar if and only if rank



A 0

C B

 = rank A + rank B

350

CHAPTER 5

and AC + CB = 0. Proof: See [1326].

5.11 Facts on Eigenvalues and Singular Values for One Matrix Fact 5.11.1. Let A ∈ Fn×n, and assume that A is singular. If A is either simple or cyclic, then rank A = n − 1. Fact 5.11.2. Let A ∈ Rn×n, and assume that A ∈ SO(n). Then, amultA(−1) is even. Now, assume in addition that n = 3. Then, the following statements hold: i) amultA(1) is either 1 or 3. ii) tr A ≥ −1. iii) tr A = −1 if and only if mspec(A) = {1, −1, −1}ms. Fact 5.11.3. Let A ∈ Fn×n, let α ∈ F, and assume that A2 = αA. Then, spec(A) ⊆ {0, α}. Fact 5.11.4. Let A ∈ Fn×n, assume that A is Hermitian, and let α ∈ R. Then, A2 = αA if and only if spec(A) ⊆ {0, α}. Remark: See Fact 3.7.22. Fact 5.11.5. Let A ∈ Fn×n, and assume that A is Hermitian. Then, spabs(A) = λmax(A) and sprad(A) = σmax (A) = max{|λmin(A)|, λmax(A)}. If, in addition, A is positive semidefinite, then sprad(A) = σmax (A) = spabs(A) = λmax(A). Remark: See Fact 5.12.2. Fact 5.11.6. Let A ∈ Fn×n, and assume that A is skew Hermitian. Then, the eigenvalues of A are imaginary. Proof: Let λ ∈ spec(A). Since 0 ≤ AA∗ = −A2, it follows that −λ2 ≥ 0, and thus λ2 ≤ 0. Fact 5.11.7. Let A, B ∈ Fn×n, and assume that A and B are idempotent. Then, the following statements are equivalent: i) mspec(A) = mspec(B). ii) rank A = rank B. iii) tr A = tr B.

351

MATRIX DECOMPOSITIONS

Fact 5.11.8. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A is idempotent. ii) rank(I − A) ≤ tr(I − A), A is group invertible, and every eigenvalue of A is nonnegative. iii) A and I − A are group invertible, and every eigenvalue of A is nonnegative. Proof: See [666]. Fact 5.11.9. Let A ∈ Fn×n, and let mspec(A) = {λ1, . . . , λk , 0, . . . , 0}ms . Then,  k 2 k   2 |tr A| ≤ |λi | ≤ k |λi |2. i=1

i=1

Proof: Use Fact 1.17.3. Fact 5.11.10. Let A ∈ Fn×n, and assume that A has exactly k nonzero eigenvalues. Then, |tr A|2 ≤ k tr A∗A ≤ (rank A) tr A∗A. k|tr A2 | ≤ k tr (A2∗A2 )1/2 Furthermore, the upper left-hand inequality is an equality if and only if A is normal and all of the nonzero eigenvalues of A have the same absolute value. Moreover, the right-hand inequality is an equality if and only if A is group invertible. If, in addition, all of the eigenvalues of A are real, then (tr A)2 ≤ k tr A2 ≤ k tr A∗A ≤ (rank A) tr A∗A. Proof: The upper left-hand inequality in the first string is given in [1483]. The lower left-hand inequality in the first string is given by Fact 9.11.3. When all of the eigenvalues of A are real, the inequality (tr A)2 ≤ k tr A2 follows from Fact 5.11.9. Remark: The inequality |tr A|2 ≤ k|tr A2 | does not necessarily hold. Consider mspec(A) = {1, 1, j, −j}ms. Remark: See Fact 3.7.22, Fact 8.18.7, Fact 9.13.16, and Fact 9.13.17. Fact 5.11.11. Let A ∈ Rn×n, and let mspec(A) = {λ1, . . . , λn }ms . Then, n 

(Re λi )(Im λi ) = 0

i=1

and tr A2 =

n  i=1

(Re λi )2 −

n 

(Im λi )2.

i=1

Fact 5.11.12. Let n ≥ 2, let a1, . . . , an > 0, and define the symmetric matrix  A ∈ Rn×n by A(i,j) = ai + aj for all i, j ∈ {1, . . . , n}. Then, rank A ≤ 2

352

CHAPTER 5

and

mspec(A) = {λ, μ, 0, . . . , 0}ms ,

where 

λ=

n 

/ 0 n 0  ai + 1 n a2 , i

i=1



μ=

i=1

n 

/ 0 n 0  ai − 1 n a2 . i

i=1

i=1

Furthermore, the following statements hold: i) λ > 0. ii) μ ≤ 0. Moreover, the following statements are equivalent: iii) μ < 0. iv) At least two of the numbers a1, . . . , an > 0 are distinct. v) rank A = 2. In this case, λmin(A) = μ < 0 < tr A = 2

n 

ai < λmax(A) = λ.

i=1

T  a 1 · · · an Proof: A = a11×n + 1n×1 aT, where a = . Then, it follows from Fact 2.11.12 that rank A ≤ rank(a11×n ) + rank(1n×1 aT ) = 2. Furthermore, mspec(A) follows from Fact 5.11.13, while Fact 1.17.14 implies that μ ≤ 0. Remark: See Fact 8.8.7. Fact 5.11.13. Let x, y ∈ Rn. Then, ;     : mspec xyT + yxT = xTy + xTxyTy, xTy − xTxyTy, 0, . . . , 0 , ms

⎧  T T T xTy ≥ 0,  T  ⎨x y + x xy y, T . . sprad xy + yx = .  ⎩.xTy − xTxyTy .., xTy ≤ 0, and

   spabs xyT + yxT = xTy + xTxyTy.

If, in addition, x and y are nonzero, then v1, v2 ∈ Rn defined by 



1 − y y   are eigenvectors of xyT +yxT corresponding to xTy + xTxyTy and xTy − xTxyTy, respectively.

v1 =

1 x x

Proof: See [382, p. 539]. Example: The spectrum of



+

1 y y,

0n×n 1n×1 11×n 0



v2 =

1 x x

√ √ is {− n, 0, . . . , 0, n}ms .

Problem: Extend this result to C and xyT + zwT. See Fact 4.9.17.

353

MATRIX DECOMPOSITIONS

Fact 5.11.14. Let A ∈ Fn×n, and let mspec(A) = {λ1, . . . , λn }ms . Then,

7 8 mspec (I + A)2 = (1 + λ1 )2, . . . , (1 + λn )2 ms. If A is nonsingular, then

 7 8  −1 mspec A−1 = λ−1 1 , . . . , λn ms.

Finally, if I + A is nonsingular, then

7 8 mspec (I + A)−1 = (1 + λ1 )−1, . . . , (1 + λn )−1 ms and

7 8 mspec A(I + A)−1 = λ1(1 + λ1 )−1, . . . , λn(1 + λn )−1 ms.

Proof: Use Fact 5.11.15. Fact 5.11.15. Let p, q ∈ F[s], assume that p and q are coprime, and define g = p/q ∈ F(s). Furthermore, let A ∈ Fn×n, let mspec(A) = {λ1, . . . , λn }ms , assume  that roots(q) ∩ spec(A) = ∅, and define g(A) = p(A)[q(A)]−1. Then, 

mspec[g(A)] = {g(λ1 ), . . . , g(λn )}ms . Proof: Statement ii) of Fact 4.9.27 implies that q(A) is nonsingular. Fact 5.11.16. Let x ∈ Fn and y ∈ Fm. Then, √ σmax(xy ∗ ) = x∗xy ∗ y. If, in addition, m = n, then mspec(xy ∗ ) = {x∗y, 0, . . . , 0}ms , mspec(I + xy ∗ ) = {1 + x∗y, 1, . . . , 1}ms , sprad(xy ∗ ) = |x∗y|, spabs(xy ∗ ) = max{0, Re x∗ y}. Remark: See Fact 9.7.26. Fact 5.11.17. Let A ∈ Fn×n, and assume that rank A = 1. Then, σmax (A) = (tr AA∗ )1/2. Fact 5.11.18. Let x, y ∈ Fn, and assume that x∗y = 0. Then,   σmax (x∗y)−1xy ∗ ≥ 1. Fact 5.11.19. Let A ∈ Fn×m, and let α ∈ F. Then, for all i ∈ {1, . . . , min{n, m}}, σi (αA) = |α|σi (A). Fact 5.11.20. Let A ∈ Fn×m. Then, for all i ∈ {1, . . . , rank A}, it follows that σi (A) = σi (A∗ ).

354

CHAPTER 5

Fact 5.11.21. Let A ∈ Fn×n, and let λ ∈ spec(A). Then, the following inequalities hold: i) σmin (A) ≤ |λ| ≤ σmax (A).

ii) λmin 12 (A + A∗ ) ≤ Re λ ≤ λmax 12 (A + A∗ ) .     1 1 iii) λmin j2 (A − A∗ ) ≤ Im λ ≤ λmax j2 (A − A∗ ) . Remark: i) is Browne’s theorem, ii) is Bendixson’s theorem, and iii) is Hirsch’s theorem. See [319, p. 17] and [988, pp. 140–144]. Remark: See Fact 5.11.22, Fact 5.12.3, and Fact 9.11.8. Fact 5.11.22. Let A ∈ Fn×n, and let mspec(A) = {λ1, . . . , λn }ms . Then, for all k ∈ {1, . . . , n}, k k     2

1 σi2 [ j2 σn−i+1(A) − |λi |2 ≤ 2 (A − A∗ )] − | Im λi |2 i=1

and 2

i=1

k  

k   2

2 1 σn−i+1 σi (A) − |λi |2 . [ j2 (A − A∗ )] − | Im λi |2 ≤

i=1

i=1

Furthermore, n 

n   

1 σi2 [ j2 σi2 (A) − |λi |2 = 2 (A − A∗ )] − | Im λi |2 .

i=1

i=1

Finally, for all i ∈ {1, . . . , n}, σn(A) ≤ |Re λi | ≤ σ1(A) and

1 1 (A − A∗ )] ≤ | Im λi | ≤ σ1[ j2 (A − A∗ )]. σn[ j2

Proof: See [566]. Remark: See Fact 9.11.7. Fact 5.11.23. Let A ∈ Fn×n, let mspec(A) = {λ1, . . . , λn }ms , and let r denote the number of Jordan blocks in the Jordan decomposition of A. Then, for all k ∈ {1, . . . , r}, k k k    2 σn−i+1 (A) ≤ |λi |2 ≤ σi2 (A) i=1

and

k 

2 1 σn−i+1 [ j2 (A − A∗ )] ≤

i=1

Proof: See [566].

i=1 k  i=1

i=1

| Im λi |2 ≤

k  i=1

1 σi2 [ j2 (A − A∗ )].

355

MATRIX DECOMPOSITIONS

Fact 5.11.24. Let A ∈ Fn×n, and let mspec(A) = {λ1(A), . . . , λn (A)}ms , where λ1(A), . . . , λn (A) are ordered such that Re λ1(A) ≥ · · · ≥ Re λn (A). Then, for all k ∈ {1, . . . , n}, k 

Re λi (A) ≤

i=1

and

n 

k 



λi 12 (A + A∗ )

i=1

Re λi (A) = Re tr A = Re tr 12 (A + A∗ ) =

i=1

n 

λi

1

2 (A

+ A∗ ) .

i=1

In particular,



λmin 12 (A + A∗ ) ≤ Re λn (A) ≤ spabs(A) ≤ λmax 12 (A + A∗ ) .

Furthermore, the last right-hand inequality is an equality if and only if A is normal. Proof: See [201, p. 74]. Also, see xii) and xiv) of Fact 11.15.7. Remark: spabs(A) = Re λ1(A). Remark: This result is due to Fan. Fact 5.11.25. Let A ∈ Fn×n. Then, for all i ∈ {1, . . . , n},

−σi (A) ≤ λi 12 (A + A∗ ) ≤ σi (A). In particular, and



−σmin (A) ≤ λmin 12 (A + A∗ ) ≤ σmin (A) −σmax (A) ≤ λmax

1

2 (A

+ A∗ ) ≤ σmax (A).

Proof: See [708, p. 447], [730, p. 151], or [996, p. 240]. Remark: This result generalizes Re z ≤ |z| for z ∈ C. Remark: See Fact 5.11.27 and Fact 8.18.4. Fact 5.11.26. Let A ∈ Fn×n. Then, −σmax (A) ≤ −σmin (A)

≤ λmin 12 (A + A∗ ) ≤ spabs(A)   | spabs(A)| ≤ sprad(A) ≤ 1 ∗ 2 λmax(A + A ) ≤ σmax (A). Proof: Combine Fact 5.11.24 and Fact 5.11.25. Fact 5.11.27. Let A ∈ Fn×n, and let {μ1, . . . , μn }ms = { 12 |λ1 (A + A∗ )|, . . . ,

T 1 ∗ σ1(A) · · · σn (A) weakly 2 |λn (A+A )|}ms , where μ1 ≥ · · · ≥ μn ≥ 0. Then,

T . majorizes μ1 · · · μn

356

CHAPTER 5

Proof: See [996, p. 240]. Remark: See Fact 5.11.25. Fact 5.11.28. Let A ∈ Fn×n, and let mspec(A) = {λ1, . . . , λn }ms , where λ1, . . . , λn are ordered such that |λ1 | ≥ · · · ≥ |λn |. Then, for all k ∈ {1, . . . , n}, k !

|λi | ≤

i=1

k !

σi (A)

i=1

with equality for k = n, that is, |det A| =

n !

|λi | =

i=1

n !

σi (A).

i=1

Hence, for all k ∈ {1, . . . , n}, n !

σi (A) ≤

i=k

n !

|λi |.

i=k

Proof: See [201, p. 43], [708, p. 445], [730, p. 171], or [1521, p. 19]. Remark: This result is due to Weyl. Remark: See Fact 8.19.22 and Fact 9.13.18. Fact 5.11.29. Let A ∈ Fn×n, and let mspec(A) = {λ1, . . . , λn }ms , where λ1, . . . , λn are ordered such that |λ1 | ≥ · · · ≥ |λn |. Then, (n−1)/n

1/n σmin (A) ≤ σmax (A)σmin

1/n

(n−1)/n (A) ≤ |λn | ≤ |λ1 | ≤ σmin(A)σmax (A) ≤ σmax (A)

and n−1 n n−1 n (A) ≤ σmax (A)σmin (A) ≤ |det A| ≤ σmin (A)σmax (A) ≤ σmax (A). σmin

Proof: Use Fact 5.11.28. See [708, p. 445]. Remark: See Fact 8.13.1 and Fact 11.20.13. Fact 5.11.30. Let β0 , . . . , βn−1 ∈ F, ⎡ 0 1 0 ⎢ ⎢ 0 0 1 ⎢ ⎢ ⎢ 0 0 0 ⎢  ⎢ A=⎢ . .. .. ⎢ .. . . ⎢ ⎢ ⎢ 0 0 0 ⎣ −β0 −β1 −β2

define A ∈ Fn×n by ··· .. . .. .

0

0

0

0

0

..

.

..

0 .. .

···

0

1

···

−βn−2

−βn−1

.

⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥, ⎥ ⎥ ⎥ ⎥ ⎥ ⎦

357

MATRIX DECOMPOSITIONS 

and define α = 1 +

n−1

|βi |2. Then, "    σ1(A) = 12 α + α2 − 4|β0 |2 ,

i=0

σ2 (A) = · · · = σn−1(A) = 1, "    σn(A) = 12 α − α2 − 4|β0 |2 . In particular,

σ1(Nn ) = · · · = σn−1 (Nn ) = 1

and σmin (Nn ) = 0. Proof: See [699, p. 523] or [825, 841]. Remark: See Fact 6.3.27 and Fact 11.20.13. Fact 5.11.31. Let β ∈ C. Then,    1 2β = |β| + 1 + |β|2 σmax 0 1 

and

σmin

1 0

2β 1

 =

 1 + |β|2 − |β|.

Proof: See [923]. Remark: Inequalities for the singular values of block-triangular matrices are given in [923]. Fact 5.11.32. Let A ∈ Fn×m. Then,    I 2A 2 (A). = σmax (A) + 1 + σmax σmax 0 I Proof: See [699, p. 116]. Fact 5.11.33. For i = 1, . . . , l, let Ai ∈ Fni ×mi. Then, σmax [diag(A1, . . . , Al )] = max{σmax (A1 ), . . . , σmax (Al )}. 

Fact 5.11.34. Let A ∈ Fn×m, and let r = rank A. Then, for all i ∈ {1, . . . , r}, λi(AA∗ ) = λi(A∗A) = σi (AA∗ ) = σi (A∗A) = σi2 (A). In particular, and, if n = m, then

2 (A), σmax (AA∗ ) = σmax 2 (A). σmin (AA∗ ) = σmin

Furthermore, for all i ∈ {1, . . . , r}, σi (AA∗A) = σi3(A).

358

CHAPTER 5

Fact 5.11.35. Let A ∈ Fn×n. Then, σmax (A) ≤ 1 if and only if A∗A ≤ I. Fact 5.11.36. Let A ∈ Fn×n. Then, for all i ∈ {1, . . . , n}, n !   σj (A). σi AA = j=1 j=n+1−i

Proof: See Fact 4.10.9 and [1125, p. 149]. Fact 5.11.37. Let A ∈ Fn×n. Then, σ1(A) = σn(A) if and only if there exist λ ∈ F and a unitary matrix B ∈ Fn×n such that A = λB. Proof: See [1125, pp. 149, 165]. Fact 5.11.38. Let A ∈ Fn×n, and assume that A is idempotent. Then, the following statements hold: i) If σ is a singular value of A, then either σ = 0 or σ ≥ 1. ii) If A = 0, then σmax (A) ≥ 1. iii) σmax (A) = 1 if and only if A is a projector. iv) If 1 ≤ rank A ≤ n − 1, then σmax (A) = σmax (A⊥ ). v) If A = 0, then σmax (A) = σmax (A + A∗ − I) = σmax (A + A∗ ) − 1 and

2 (A) − 1]1/2. σmax (I − 2A) = σmax (A) + [σmax

Proof: See [551, 742, 765]. Statement iv) is given in [550, p. 61] and follows from Fact 5.11.39. Problem: Use Fact 5.9.28 to prove iv). Fact 5.11.39. Let A ∈ Fn×n, assume that A is idempotent, and assume that 1 ≤ rank A ≤ n − 1. Then, σmax (A) = σmax (A + A∗ − I) =

1 , sin θ

where θ ∈ (0, π/2] is defined by cos θ = max{|x∗ y| : (x, y) ∈ R(A) × N(A) and x∗ x = y ∗ y = 1}. Proof: See [551, 765]. Remark: θ is the minimal principal angle. See Fact 2.9.19 and Fact 5.12.17. Remark: Note that N(A) = R(A⊥ ). See Fact 3.12.3. Remark: This result is due to Ljance. Remark: This result yields statement iii) of Fact 5.11.38.

359

MATRIX DECOMPOSITIONS

Remark: See Fact 10.9.19. Fact 5.11.40. Let A ∈ Rn×n, where n ≥ 2, be the tridiagonal matrix ⎡ ⎤ b 1 c1 0 · · · 0 0 ⎢ a 0 0 ⎥ ⎢ 1 b 2 c2 · · · ⎥ ⎢ ⎥ .. ⎢ ⎥ . 0 0 b 0 a ⎢ ⎥ 2 3  ⎥, A=⎢ . . . .. .. .. ⎢ . ⎥ . . . . . . ⎥ . ⎢ . ⎢ ⎥ .. ⎢ ⎥ . bn−1 cn−1 ⎦ ⎣ 0 0 0 0

0

0

···

an−1

bn

and assume that, for all i ∈ {1, . . . , n − 1}, ai ci > 0 Then, A is simple, and every eigenvalue of A is real. Hence, rank A ≥ n − 1. 





Proof: SAS −1 is symmetric, where S = diag(d1, . . . , dn ), d1 = 1, and di+1 = (ci /ai )1/2di for all i ∈ {1, . . . , n − 1}. For a proof of the fact that A is simple, see [494, p. 198]. Remark: See Fact 5.11.41. Fact 5.11.41. Let A ∈ Rn×n, where n ≥ 2, be the tridiagonal matrix ⎡ ⎤ b 1 c1 0 · · · 0 0 ⎢ a 0 0 ⎥ ⎢ 1 b 2 c2 · · · ⎥ ⎢ ⎥ .. ⎢ ⎥ . 0 0 b 0 a ⎢ ⎥ 2 3  ⎢ A=⎢ . .. ⎥ .. . . .. .. ⎥, . . . . . ⎥ . ⎢ . ⎢ ⎥ .. ⎢ ⎥ . bn−1 cn−1 ⎦ 0 ⎣ 0 0 0

0

0

···

an−1

bn

and assume that, for all i ∈ {1, . . . , n − 1}, ai ci = 0. Then, A is reducible. Furthermore, let k+ and k− denote, respectively, the number of positive and negative numbers in the sequence 1, a1 c1, a1 a2 c1 c2 , . . . , a1 a2 · · · an−1 c1 c2 · · · cn−1 . Then, A has at least |k+ −k− | distinct real eigenvalues, of which at least max{0, n− 3 min{k+ , k− }} are simple. Proof: See [1410]. Remark: Note that k+ + k− = n and |k+ − k− | = n − 2 min{k+ , k− }. Remark: This result yields Fact 5.11.40 as a special case.

360

CHAPTER 5

Fact 5.11.42. Let A ∈ Rn×n be the tridiagonal matrix ⎡ 0 1 0 ⎢ ⎢ n −1 0 2 0 ⎢ ⎢ .. ⎢ . n−2 0 ⎢ 0 ⎢ ⎢  ⎢ .. .. .. .. A= . . . . ⎢ ⎢ .. .. ⎢ . . 0 n−2 0 ⎢ ⎢ ⎢ .. ⎢ . 2 0 n −1 0 ⎣ 0 1 0 Then, χA(s) =

n !

⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥. ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦

[s − (n + 1 − 2i)].

i=1

Hence,

* spec(A) =

{n − 1, −(n − 1), . . . , 1, −1},

n even,

{n − 1, −(n − 1), . . . , 2, −2, 0}, n odd.

Proof: See [1291]. Fact 5.11.43. Let A ∈ Rn×n, where n ≥ 1, be the tridiagonal, Toeplitz matrix ⎤ ⎡ b c 0 ··· 0 0 ⎥ ⎢ 0 0 ⎥ ⎢ a b c ··· ⎥ ⎢ .. ⎥ ⎢ . ⎥ ⎢ 0 0 0 a b  A=⎢ .. ⎥ .. .. ⎥, ⎢ .. .. . . . . . . ⎥ ⎢ . . ⎥ ⎢ .. ⎥ ⎢ . b c ⎦ ⎣ 0 0 0 0

0

0

···

a

b

and assume that ac > 0. Then,   √ iπ spec(A) = b + 2 ac cos : i ∈ {1, . . . , n} . n+1 Remark: See [699, p. 522]. Remark: See Fact 3.18.9.

361

MATRIX DECOMPOSITIONS

Fact 5.11.44. Let A ∈ Rn×n, where n ≥ 1, be the tridiagonal, Toeplitz matrix ⎤ ⎡ 0 1/2 0 · · · 0 0 ⎥ ⎢ 0 0 ⎥ ⎢ 1/2 0 1/2 · · · ⎥ ⎢ .. ⎥ ⎢ . ⎥ ⎢ 0 0 0 1/2 0  ⎢ A=⎢ . .. .. ⎥ .. .. .. ⎥. . . . . ⎢ . . . ⎥ ⎥ ⎢ .. ⎥ ⎢ . 0 1/2 ⎦ 0 0 ⎣ 0 0

0 

Then, spec(A) =

0

···

1/2

0

 iπ cos : i ∈ {1, . . . , n} . n+1

Furthermore, the associated eigenvectors v1, . . . , vn ⎡ iπ sin n+1 ⎢ 2iπ + ⎢ sin n+1 2 ⎢ vi = n+1 ⎢ .. ⎣ . niπ sin n+1

are given by ⎤ ⎥ ⎥ ⎥ ⎥ ⎦

and satisfy vi 2 = 1 for all i ∈ {1, . . . , n}, and are mutually orthogonal. Remark: See [846]. Fact 5.11.45. Let A ∈ Fn×n, and assume that A has real eigenvalues. Then, +

1 n−1 1 2 2 ≤ λ min(A) n tr A − n tr A − n (tr A) +

≤ n1 tr A − n21−n tr A2 − n1 (tr A)2 +

≤ n1 tr A + n21−n tr A2 − n1 (tr A)2 ≤ λmax(A) ≤

1 n tr A

+

+

n−1 n



tr A2 − n1 (tr A)2 .

Furthermore, for all i ∈ {1, . . . , n}, . + .

.λi (A) − 1 tr A. ≤ n−1 tr A2 − 1 (tr A)2 . n n n Finally, if n = 2, then + 1 1 1 2 2 tr A− n n tr A − n2 (tr A) = λmin(A) ≤ λmax(A) =

1 n tr A+

+

1 2 n tr A

−

1 2 n2 (tr A) .

Proof: See [1483, 1484]. Remark: These inequalities are related to Fact 1.17.12. 

Fact 5.11.46. Let A ∈ Fn×n, and let μ(A) = min{|λ| : λ ∈ spec(A)}. Then, + + 1 n−1 1 ∗ − 1 |tr A|2 ) ≤ μ(A) ≤ ∗ |tr A| − (tr AA n n n n tr AA

362

CHAPTER 5

+

and 1 n |tr A|

≤ sprad(A) ≤

1 n |tr A|

+

n−1 ∗ n (tr AA

− n1 |tr A|2 ).

Proof: See Theorem 3.1 of [1483]. Fact 5.11.47. Let A ∈ Fn×n, where n ≥ 2, be the bidiagonal matrix ⎡ ⎤ a1 b 1 0 · · · 0 0 ⎢ 0 a b2 · · · 0 0 ⎥ 2 ⎢ ⎥ ⎢ ⎥ .. ⎢ ⎥ . 0 0 0 0 a ⎢ ⎥ 3  ⎥, A=⎢ . . . .. .. .. ⎢ . ⎥ . . . . . . ⎥ . ⎢ . ⎢ ⎥ .. ⎢ ⎥ . an−1 bn−1 ⎦ 0 ⎣ 0 0 0

0

0

···

0

an

and assume that a1, . . . , an , b1, . . . , bn−1 are nonzero. Then, the following statements hold: i) The singular values of A are distinct. ii) If B ∈ Fn×n is bidiagonal and |B| = |A|, then A and B have the same singular values. iii) If B ∈ Fn×n is bidiagonal, |A| ≤ |B|, and |A| = |B|, then σmax (A) < σmax (B). iv) If B ∈ Fn×n is bidiagonal, |I ◦ A| ≤ |I ◦ B|, and |I ◦ A| = |I ◦ B|, then σmin (A) < σmin (B). v) If B ∈ Fn×n is bidiagonal, |Isup ◦ A| ≤ |Isup ◦ B|, and |Isup ◦ A| = |Isup ◦ B|, then σmin (B) < σmin (A). Proof: See [1006, p. 17-5]. Remark: Isup denotes the matrix all of whose entries on the superdiagonal are 1 and are 0 otherwise.

5.12 Facts on Eigenvalues and Singular Values for Two or More Matrices  Fact 5.12.1. Let A ∈ Fn×n and B ∈ Fn×m, let r = rank B, and define A B A = B ∗ 0 . Then, ν−(A) ≥ r, ν0 (A) ≥ 0, and ν+(A) ≥ r. If, in addition, n = m

T . and B is nonsingular, then In A = n 0 n 

Proof: See [736]. Remark: See Proposition 5.6.5. Fact 5.12.2. Let A, B ∈ Fn×n. Then, sprad(A + B) ≤ σmax (A + B) ≤ σmax (A) + σmax (B).

363

MATRIX DECOMPOSITIONS

If, in addition, A and B are Hermitian, then sprad(A + B) = σmax (A + B) ≤ σmax (A) + σmax (B) = sprad(A) + sprad(B) and λmin(A) + λmin(B) ≤ λmin(A + B) ≤ λmax (A + B) ≤ λmax (A) + λmax (B). Proof: Use Lemma 8.4.3 for the last string of inequalities. Remark: See Fact 5.11.5. Fact 5.12.3. Let A, B ∈ Fn×n, and let λ be an eigenvalue of A + B. Then, ∗ 1 2 λmin(A

+ A) + 12 λmin(B ∗ + B) ≤ Re λ ≤ 12 λmax (A∗ + A) + 12 λmax (B ∗ + B).

Proof: See [319, p. 18]. Remark: See Fact 5.11.21. Fact 5.12.4. Let A, B ∈ Fn×n be normal, and let mspec(A) = {λ1, . . . , λn }ms and mspec(B) = {μ1, . . . , μn }ms . Then, min Re

n 

λi μσ(i) ≤ Re tr AB ≤ max Re

i=1

n 

λi μσ(i) ,

i=1

where “max” and “min” are taken over all permutations σ of the eigenvalues of B. Now, assume in addition that A and B are Hermitian. Then, tr AB is real, and n  i=1

λi(A)λn−i+1(B) ≤ tr AB ≤

n 

λi(A)λi(B).

i=1

Furthermore, the last inequality is an equality if and only if there exists a unitary matrix S ∈ Fn×n such that A = S diag[λ1(A), . . . , λn(A)]S ∗ and B = S diag[λ1(B), . . . , λn(B)]S ∗. Proof: See [982]. For the second string of inequalities, use Fact 1.18.4. For the last statement, see [243, p. 10] or [916]. Remark: The upper bound for tr AB is due to Fan. Remark: See Fact 5.12.5, Fact 5.12.8, Proposition 8.4.13, Fact 8.12.29, and Fact 8.19.19. Fact 5.12.5. Let A, B ∈ Fn×n, and assume that B is Hermitian. Then, n  i=1

λi [ 12 (A + A∗ )]λn−i+1(B) ≤ Re tr AB ≤

n 

λi [ 12 (A + A∗ )]λi(B).

i=1

Proof: Apply the second string of inequalities in Fact 5.12.4. Remark: For A, B real, these inequalities are given in [861]. The complex case is given in [896]. Remark: See Proposition 8.4.13 for the case in which B is positive semidefinite.

364

CHAPTER 5 

Fact 5.12.6. Let A ∈ Fn×m and B ∈ Fm×n, and let r = min{rank A, rank B}. Then, r  |tr AB| ≤ σi (A)σi (B). i=1

Proof: See [996, pp. 514, 515] or [1125, p. 148].

0 Remark: Applying Fact 5.12.4 to A0∗ A 0 and B yields the weaker result |Re tr AB| ≤

r 

B∗ 0

and using Proposition 5.6.5

σi(A)σi(B).

i=1

Remark: See [243, p. 14]. Remark: This result is due to Mirsky. Remark: See Fact 5.12.7. Remark: A generalization of this result is given by Fact 9.14.3. Fact 5.12.7. Let A, B ∈ Fn×n, and assume that B is positive semidefinite. Then, |tr AB| ≤ σmax(A) tr B. Proof: Apply Fact 5.12.6. Remark: A generalization of this result is given by Fact 9.14.4.

1 2 (A

 Fact 5.12.8. Let A, B ∈ Rn×n, assume that B is symmetric, and define C = T + A ). Then,

λmin(C)tr B−λmin(B)[nλmin(C) − tr A] ≤ tr AB ≤ λmax(C)tr B − λmax(B)[nλmax(C) − tr A]. Proof: See [481]. Remark: See Fact 5.12.4, Proposition 8.4.13, and Fact 8.12.29. Extensions are given in [1098]. Fact 5.12.9. Let A, B, Q, S1, S2 ∈ Rn×n, assume that A and B are symmetric, and assume that Q, S1, and S2 are orthogonal. Furthermore, assume that S1TAS1 and S2TBS2 are diagonal with the diagonal entries arranged in non increasing order, and define the orthogonal matrices Q1, Q2 ∈ Rn×n by Q1 =  S1 revdiag(±1, . . . , ±1)S1T and Q2 = S2 diag(±1, . . . , ±1)S2T. Then, T T tr AQ1BQT 1 ≤ tr AQBQ ≤ tr AQ2BQ2 .

Proof: See [160, 916]. Remark: See Fact 5.12.8.

365

MATRIX DECOMPOSITIONS

Fact 5.12.10. Let A1, . . . , Ak , B1, . . . , Bk ∈ Fn×n, and assume that A1, . . . , Ak are unitary. Then, n  σi (B1 ) · · · σi (Bk ). |tr A1B1 · · · Ak Bk | ≤ i=1

Proof: See [996, p. 516]. Remark: This result is due to Fan. Remark: See Fact 5.12.9. Fact 5.12.11. Let A, B ∈ Rn×n, and assume that AB = BA. Then, sprad(AB) ≤ sprad(A) sprad(B) and

sprad(A + B) ≤ sprad(A) + sprad(B).

Proof: Use Fact 5.17.4. Remark: If AB =  BA, then both of these inequalities may be violated. Consider A = [ 00 10 ] and B = [ 01 00 ]. Fact 5.12.12. Let A, B ∈ Cn×n, assume that A and B are normal, and let mspec(A) = {λ1, . . . , λn }ms and mspec(B) = {μ1, . . . , μn }ms . Then, ⎧ ⎫ n n ⎨! ⎬ ! | det(A + B)| ≤ min max |λi + μj |, max |λi + μj | . i=1,...,n ⎩ j=1,...,n ⎭ i=1

j=1

Proof: See [1137]. Remark: Equality is discussed in [165]. Remark: See Fact 9.14.18. Fact 5.12.13. Let A ∈ Fn×m and B ∈ Fn×m. Then, m  ! ∗ ∗ σi (B) det(AA∗ ). det(ABB A ) ≤ i=1

Proof: See [459, p. 218]. Fact 5.12.14. Let A, B, C ∈ Fn×n, assume that spec(A) ∩ spec(B) = ∅, and assume that [A + B, C] = 0 and [AB, C] = 0. Then, [A, C] = [B, C] = 0. Proof: This result follows from Corollary 7.2.5. Remark: This result is due to Embry. See [221]. Fact 5.12.15. Let A, B ∈ Fn×n, and assume that A and B are projectors. Then, spec(AB) ⊂ [0, 1] and

spec(A − B) ⊂ [−1, 1].

366

CHAPTER 5

Proof: See [40], [550, p. 53], or [1125, p. 147]. Remark: The first result is due to Afriat. Fact 5.12.16. Let A, B ∈ Fn×n, and assume that A and B are projectors. Then, the following statements are equivalent: i) AB is a projector. ii) spec(A + B) ⊂ {0} ∪ [1, ∞). iii) spec(A − B) ⊂ {−1, 0, 1}. Proof: See [551, 612]. Remark: See Fact 3.13.20 and Fact 6.4.26. Fact 5.12.17. Let A, B ∈ Fn×n, assume that A and B are nonzero projectors, and define the minimal principal angle θ ∈ [0, π/2] by cos θ = max{|x∗ y| : (x, y) ∈ R(A) × R(B) and x∗ x = y ∗ y = 1}. Then, the following statements hold: i) σmax (AB) = σmax (BA) = cos θ. ii) σmax (A + B) = 1 + σmax (AB) = 1 + cos θ. iii) 1 ≤ σmax (AB) + σmax (A − B). iv) If σmax (A − B) < 1, then rank A = rank B. v) θ > 0 if and only if R(A) ∩ R(B) = {0}. Furthermore, the following statements are equivalent: vi) A − B is nonsingular. vii) R(A) and R(B) are complementary subspaces. viii) σmax (A + B − I) < 1. Now, assume in addition that A − B is nonsingular. Then, the following statements hold: ix) σmax (AB) < 1. x) σmax [(A − B)−1 ] = √

1 2 1−σmax (AB)

= 1/sin θ.

xi) σmin (A − B) = sin θ. 2 2 (A − B) + σmax (AB) = 1. xii) σmin

xiii) I − AB is nonsingular. xiv) If rank A = rank B, then σmax (A − B) = sin θ. Proof: Statement i) is given in [765]. Statement ii) is given in [551]. Statement iii) follows from the first inequality in Fact 8.19.11. For iv), see [459, p. 195] or [574, p. 389]. Statement v) is given in [574, p. 393]. Fact 3.13.24 shows that vi) and vii) are equivalent. Statement viii) is given in [278]; see also [550, p. 236]. Statement

367

MATRIX DECOMPOSITIONS

xiv) follows from [1261, pp. 92, 93]. Remark: Additional conditions for the nonsingularity of A − B are given in Fact 3.13.24. Remark: See Fact 2.9.19, Fact 5.11.39, and Fact 5.12.18. Fact 5.12.18. Let A ∈ Fn×n, and assume that A is idempotent. Furthermore, let P, Q ∈ Fn×n, where P is the projector onto R(A) and Q is the projector onto N(A). Then, the following statements hold: i) P − Q is nonsingular. ∗ . ii) (P − Q)−1 = A + A∗ − I = A − A⊥

iii) σmax (A) = √

1 2 1−σmax (P Q)

= σmax [(P − Q)−1 ] = σmax (A + A∗ − I).

iv) σmax (A) = 1/sin θ, where θ is the minimal principal angle θ ∈ [0, π/2] defined by cos θ = max{|x∗ y| : (x, y) ∈ R(P ) × R(Q) and x∗ x = y ∗ y = 1}. 2 2 (P − Q) = 1 − σmax (P Q). v) σmin

vi) σmax (P Q) = σmax (QP ) = σmax (P + Q − I) < 1. Proof: See [1142] and Fact 5.12.17. The nonsingularity of P − Q follows from Fact 3.13.24. Statement ii) is given by Fact 3.13.24 and Fact 6.3.24. The first equality in iii) is given in [278]. See also [551]. ∗ Remark: A⊥ is the idempotent matrix onto R(A)⊥ along N(A)⊥ . See Fact 3.12.3.

Remark: P = AA+ and Q = I − A+A. Fact 5.12.19. Let A, B ∈ Fn×n, and assume that A and B are idempotent. Then, A − B is idempotent if and only if A − B is group invertible and every eigenvalue of A − B is nonnegative. Proof: See [666]. Remark: This result is due to Makelainen and Styan. Remark: See Fact 3.12.29. Remark: Conditions for a matrix to be expressible as a difference of idempotents are given in [666].



Fact 5.12.20. Let A ∈ Rn×n, B ∈ Rn×m, and C ∈ Rm×m, define A =

∈ R(n+m)×(n+m), and assume that A is symmetric. Then,

A B BT C

λmin (A) + λmax (A) ≤ λmax (A) + λmax (C). Proof: See [227, p. 56].

368 R

r×r

CHAPTER 5

Fact 5.12.21. Let M ∈ Rr×r, assume that M is positive definite, let C, K ∈ , assume that C and K are positive semidefinite, and consider the equation

 Then, x(t) =



q(t) q(t) ˙



M q¨ + Cq˙ + Kq = 0. satisfies x(t) ˙ = Ax(t), where A is the 2r × 2r matrix   0 I  . A= −M −1K −M −1 C

Furthermore, the following statements hold: i) A, K, and M satisfy det A =

det K . det M

ii) A and K satisfy rank A = r + rank K. iii) A is nonsingular if and only if K is positive definite. In this case,   −K −1C −K−1M −1 . A = I 0 iv) Let λ ∈ C. Then, λ ∈ spec(A) if and only if det(λ2M + λC + K) = 0. v) If λ ∈ spec(A), Re λ = 0, and Im λ = 0, then λ is semisimple. vi) mspec(A) ⊂ CLHP. vii) If C = 0, then spec(A) ⊂ jR. viii) If C and K are positive definite, then spec(A) ⊂ OLHP.   1 1/2 √ K q(t)  2 ˆ satisfies x(t) ˙ = Ax(t), where ix) x ˆ(t) = √1 1/2 2

M

q(t) ˙

 

Aˆ =

0

K 1/2M −1/2

−M −1/2K 1/2

−M −1/2 CM −1/2

 .

If, in addition, C = 0, then Aˆ is skew symmetric.   1/2  M q(t) ˆ satisfies x(t) ˙ = Ax(t), where x) x ˆ(t) = 1/2 M q(t) ˙   0 I  . Aˆ = −M −1/2KM −1/2 −M −1/2 CM −1/2 If, in addition, C = 0, then Aˆ is Hamiltonian. Remark: M, C, and K are mass, damping, and stiffness matrices, respectively. See [190]. Remark: See Fact 5.14.34 and Fact 11.18.38. Problem: Prove v).

369

MATRIX DECOMPOSITIONS

that A and B are positive Fact 5.12.22. Let A, B ∈ Rn×n, and 0 assume

B semidefinite. Then, every eigenvalue λ of −A satisfies Re λ = 0. 0 Proof: Square this matrix. Problem: What happens if A and B have different sizes? In addition, let C ∈ Rn×n, and assume 0 that

C is (positive semidefinite, positive definite). Then, every eigenA value of −B −C satisfies (Re λ ≤ 0, Re λ < 0).

−C A A Problem: Consider also −C −B −C and −A −C .

5.13 Facts on Matrix Pencils Fact 5.13.1. Let A, B ∈ Fn×n, and assume that PA,B is a regular pencil.  Furthermore, let S ⊆ Fn, assume that S is a subspace, let k = dim S, let S ∈ Fn×k, and assume that R(S) = S. Then, the following statements are equivalent: i) dim(AS + BS) = dim S. ii) There exists a matrix M ∈ Fk×k such that AS = BSM. Proof: See [897, p. 144]. Remark: S is a deflating subspace of PA,B . This result generalizes Fact 2.9.25.

5.14 Facts on Matrix Eigenstructure Fact 5.14.1. Let A ∈ Fn×n, let λ ∈ spec(A), assume that λ is cyclic, let i ∈ {1, . . . , n} be such that rank (A − λI)({i}∼ ,{1,...,n}) = n − 1, and define x ∈ Cn by ⎤ ⎡ det(A − λI)[i;1] ⎥ ⎢ − det(A − λI)[i;2] ⎥ ⎢  ⎢ ⎥. x=⎢ .. ⎥ . ⎦ ⎣ (−1)n+1 det(A − λI)[i;n] Then, x is an eigenvector of A associated with λ. Proof: See [1372]. Fact 5.14.2. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A is group invertible.   ii) R(A) = R A2 . iii) ind A ≤ 1. iv) rank A = of A.

r i=1

amultA(λi ), where λ1, . . . , λr are the nonzero eigenvalues

370

CHAPTER 5 

Fact 5.14.3. Let n ≥ 2, x, y ∈ Fn, define A = xyT, and assume that rank A = 1, that is, A is nonzero. Then, the following statements are equivalent: i) A is semisimple. ii) yTx = 0. iii) tr A = 0. iv) A is group invertible. v) ind A = 1. vi) amultA(0) = n − 1. Furthermore, the following statements are equivalent: vii) A is defective. viii) yTx = 0. ix) tr A = 0. x) A is not group invertible. xi) ind A = 2. xii) A is nilpotent. xiii) amultA(0) = n. xiv) spec(A) = {0}. Remark: See Fact 2.10.19. Fact 5.14.4. Let A ∈ Fn×n, and assume that A is diagonalizable over F. Then, AT, A, A∗, and AA are diagonalizable. If, in addition, A is nonsingular, then A−1 is diagonalizable. Proof: See Fact 2.16.10 and Fact 3.7.10. Fact 5.14.5. Let A ∈ Fn×n, assume that A is diagonalizable over F with  eigenvalues λ1, . . . , λn , and let B = diag(λ1, . . . , λn ). If, x1, . . . , xn ∈ Fn are linearly independent eigenvectors of A associated with λ1, . . . , λn , respectively, then A =

 x1 · · · xn . Conversely, if S ∈ Fn×n is nonsingular and SBS −1, where S = A = SBS −1, then, for all i ∈ {1, . . . , n}, coli(S) is an associated eigenvector. Fact 5.14.6. Let A, S ∈ Fn×n, assume that S is nonsingular, let λ ∈ C, and assume that row1(S −1AS) = λeT 1 . Then, λ ∈ spec(A), and col1(S) is an associated eigenvector. n×n Fact 5.14.7. Let A . Then, A is cyclic if and only if there exists a ∈ F n vector b ∈ F such that b Ab · · · An−1b is nonsingular.

Proof: See Fact 12.20.13. Remark: (A, b) is controllable. See Corollary 12.6.3.

371

MATRIX DECOMPOSITIONS

Fact 5.14.8. Let A ∈ Fn×n, and define the positive integer m by  m=

max λ∈spec(A)

gmultA(λ).

Then, m is the smallest integer such that there exists B ∈ Fn×m such that

n−1 rank B AB · · · A B = n. Proof: See Fact 12.20.13. Remark: (A, B) is controllable. See Corollary 12.6.3. Fact 5.14.9. Let A ∈ Cn×n. Then, there exist v1, . . . , vn ∈ Cn such that the following statements hold: i) v1, . . . , vn ∈ Cn are linearly independent. ii) If λ ∈ spec(A) and A has a k × k Jordan block associated with λ, then there exist distinct integers i1 , . . . , ik such that Avi1 = λvi1 , Avi2 = λvi2 + vi1 , .. . Avik = λvik + vik−1 . iii) Let λ and vi1 , . . . , vik be given by ii). Then, span {vi1 , . . . , vik } = N[(λI − A)k ]. Remark: v1, . . . , vn are generalized eigenvectors of A. Remark: (vi1 , . . . , vik ) is a Jordan chain of A associated with λ. See [892, pp. 229– 231]. Remark: See Fact 11.13.7. Fact 5.14.10. Let A ∈ Rn×n. Then, A is cyclic and semisimple if and only if A is simple. Fact 5.14.11. Let A = revdiag(a1, . . . , an ) ∈ Rn×n. Then, A is semisimple if and only if, for all i ∈ {1, . . . , n}, ai and an+1−i are either both zero or both nonzero. Proof: See [641, p. 116], [827], or [1125, pp. 68, 86]. Fact 5.14.12. Let A ∈ Fn×n. Then, A has at least m real eigenvalues and m associated linearly independent eigenvectors if and only if there exists a positivesemidefinite matrix S ∈ Fn×n such that rank S = m and AS = SA∗. Proof: See [1125, pp. 68, 86]. Remark: The case m = n is given by Proposition 5.5.12. Remark: This result is due to Drazin and Haynsworth.

372

CHAPTER 5

Fact 5.14.13. Let A ∈ Fn×n, assume that A is normal, and let mspec(A) = {λ1, . . . , λn}ms . Then, there exist vectors x1, . . . , xn ∈ Cn such that x∗i xj = δij for all i, j ∈ {1, . . . , n} and n  A= λi xi x∗i . i=1

Furthermore, x1, . . . , xn are mutually orthogonal eigenvectors of A. Remark: See Corollary 5.4.8. Fact 5.14.14. Let A ∈ Fn×n, and let mspec(A) = {λ1, . . . , λn}ms , where |λ1 | ≥ · · · ≥ |λn |. Then, the following statements are equivalent: i) A is normal. ii) For all i ∈ {1, . . . , n}, |λi | = σi (A). n n 2 2 iii) i=1 |λi | = i=1 σi (A). iv) There exists p ∈ F[s] such that A = p(A∗ ). v) Every eigenvector of A is also an eigenvector of A∗. vi) AA∗ − A∗A is either positive semidefinite or negative semidefinite. vii) For all x ∈ Fn, x∗A∗Ax = x∗AA∗ x. viii) For all x, y ∈ Fn, x∗A∗Ay = x∗AA∗ y. In this case, sprad(A) = σmax (A). Proof: See [603] or [1125, p. 146]. Remark: See Fact 9.8.13 and Fact 9.11.2. Fact 5.14.15. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A is (simple, cyclic, derogatory, semisimple, defective, diagonalizable over F). ii) There exists α ∈ F such that A+ αI is (simple, cyclic, derogatory, semisimple, defective, diagonalizable over F). iii) For all α ∈ F, A + αI is (simple, cyclic, derogatory, semisimple, defective, diagonalizable over F). Fact 5.14.16. Let x, y ∈ Fn, assume that xTy = 1, and define the elementary  matrix A = I − xyT. Then, A is semisimple if and only if either xyT = 0 or T x y = 0. Remark: Use Fact 5.14.3 and Fact 5.14.15. Fact 5.14.17. Let A ∈ Fn×n, and assume that A is nilpotent. Then, A is nonzero if and only if A is defective. Fact 5.14.18. Let A ∈ Fn×n, and assume that A is either involutory or skew involutory. Then, A is semisimple.

MATRIX DECOMPOSITIONS

373

Fact 5.14.19. Let A ∈ Rn×n, and assume that A is involutory. Then, A is diagonalizable over R. Fact 5.14.20. Let A ∈ Fn×n, assume that A is semisimple, and assume that A = A2. Then, A is idempotent. 3

Fact 5.14.21. Let A ∈ Fn×n. Then, A is cyclic if and only if every matrix B ∈ Fn×n satisfying AB = BA is a polynomial in A. Proof: See [730, p. 275]. Remark: See Fact 2.18.9, Fact 5.14.22, Fact 5.14.23, and Fact 7.5.2. Fact 5.14.22. Let A ∈ Fn×n, assume that A is simple, let B ∈ Fn×n, and assume that AB = BA. Then, B is a polynomial in A whose degree is not greater than n − 1. Proof: See [1526, p. 59]. Remark: See Fact 5.14.21. Fact 5.14.23. Let A, B ∈ Fn×n. Then, B is a polynomial in A if and only if B commutes with every matrix that commutes with A. Proof: See [730, p. 276]. Remark: See Fact 4.8.13. Remark: See Fact 2.18.9, Fact 5.14.21, Fact 5.14.22, and Fact 7.5.2. Fact 5.14.24. Let A, B ∈ Cn×n, and assume that AB = BA. Furthermore, let x ∈ Cn be an eigenvector of A associated with the eigenvalue λ ∈ C, and assume that Bx = 0. Then, Bx is an eigenvector of A associated with the eigenvalue λ ∈ C. Proof: A(Bx) = BAx = B(λx) = λ(Bx). Fact 5.14.25. Let A ∈ Cn×n, and let x ∈ Cn be an eigenvector of A associated with the eigenvalue λ. If A is nonsingular, then x is an eigenvector of AA associated with the eigenvalue (det A)/λ. If rank A = n − 1, then x is an eigenvector of AA associated with the eigenvalue tr AA or 0. Finally, if rank A ≤ n − 2, then x is an eigenvector of AA associated with the eigenvalue 0. Proof: Use Fact 5.14.24 and the fact that AAA = AAA . See [362]. Remark: See Fact 2.16.8 or Fact 6.3.6. Fact 5.14.26. Let A, B ∈ Cn×n. Then, the following statements are equivalent: k l i) ∩n−1 k,l=1 N([A , B ]) = {0}. n−1 k l ∗ k l ii) k,l=1 [A , B ] [A , B ] is singular.

iii) A and B have a common eigenvector. Proof: See [561].

374

CHAPTER 5

Remark: This result is due to Shemesh. Remark: See Fact 5.17.1. Fact 5.14.27. Let A, B ∈ Cn×n, and assume that AB = BA. Then, there exists a nonzero vector x ∈ Cn that is an eigenvector of both A and B. Proof: See [728, p. 51]. Fact 5.14.28. Let A, B ∈ Fn×n. Then, the following statements hold: i) Assume that A and B are Hermitian. Then, AB is Hermitian if and only if AB = BA. ii) A is normal if and only if, for all C ∈ Fn×n, AC = CA implies that A∗C = CA∗. iii) Assume that B is Hermitian and AB = BA. Then, A∗B = BA∗. iv) Assume that A and B are normal and AB = BA. Then, AB is normal. v) Assume that A, B, and AB are normal. Then, BA is normal. vi) Assume that A and B are normal and either A or B has the property that distinct eigenvalues have unequal absolute values. Then, AB is normal if and only if AB = BA. Proof: See [366, 1462], [645, p. 157], and [1125, p. 102]. Fact 5.14.29. Let A, B, C ∈ Fn×n, and assume that A and B are normal and AC = CB. Then, A∗C = CB ∗. Proof: Consider [ A0 B0 ] and [ 00 C0 ] in ii) of Fact 5.14.28. See [642, p. 104] or [645, p. 321]. Remark: This result is the Putnam-Fuglede theorem. Fact 5.14.30. Let A, B ∈ Fn×n, and assume that A is dissipative and B is range Hermitian. Then, ind B = ind AB. Proof: See [193]. Fact 5.14.31. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m. Then,   A B ≤ ind A + ind C. max{ind A, ind C} ≤ ind 0 C 

If C is nonsingular, then ind

A B 0 C

 = ind A,

whereas, if A is nonsingular, then   A B = ind C. ind 0 C Proof: See [269, 1024].

375

MATRIX DECOMPOSITIONS

Remark: See Fact 6.6.14. Remark: The eigenstructure of a partitioned Hamiltonian matrix is considered in Fact 12.23.1. Fact 5.14.32. Let A, B ∈ Rn×n, and assume that A and B are skew symmetric. Then, there exists an orthogonal matrix S ∈ Rn×n such that   0(n−l)×(n−l) A12 ST A=S −AT A 22 12 

and B=S

B11

B12

T − B12

0l×l

 S T,



where l = n/2. Consequently,

  T   T ∪ mspec −A12 B12 , mspec(AB) = mspec −A12 B12

and thus every nonzero eigenvalue of AB has even algebraic multiplicity. Proof: See [32]. Fact 5.14.33. Let A, B ∈ Rn×n, and assume that A and B are skew symmetric. If n is even, then there exists a monic polynomial p of degree n/2 such that χAB (s) = p2 (s) and p(AB) = 0. If n is odd, then there exists a monic polynomial p(s) of degree (n − 1)/2 such that χAB (s) = sp2 (s) and ABp(AB) = 0. Consequently, if n is (even, odd), then χAB is (even, odd) and (every, every nonzero) eigenvalue of AB has even algebraic multiplicity and geometric multiplicity of at least 2. Proof: See [428, 592]. Fact 5.14.34. Let q(t) denote the displacement of a mass m > 0 connected to a spring k ≥ 0 and dashpot c ≥ 0 and subject to a force f(t). Then, q(t) satisfies m¨ q(t) + cq(t) ˙ + kq(t) = f(t) or

k 1 c q(t) ˙ + q(t) = f(t). m m m   Now, define the natural frequency ωn = k/m and, if k > 0, the damping ratio √  ζ= c/2 km to obtain q¨(t) +

˙ + ωn2 q(t) = q¨(t) + 2ζωn q(t)

1 f(t). m 



If k = 0, then set ωn = 0 and ζωn = c/2m. Next, define x1(t) = q(t) and x2 (t) = q(t) ˙ so that this equation can be written as        x˙ 1(t) 0 1 0 x1(t) f(t). = + −ωn2 −2ζωn 1/m x˙ 2 (t) x2 (t)

376

CHAPTER 5

  0 1  The eigenvalues of the companion matrix Ac = −ωn2 −2ζωn are given by ⎧ ⎪ 0 ≤ ζ ≤ 1, ⎨{−ζωn − jωd , −ζωn + jωd }ms , mspec(Ac ) = : ;   ⎪ ⎩ (−ζ − ζ 2 − 1)ωn , (−ζ + ζ 2 − 1)ωn , ζ > 1,   where ωd = ωn 1 − ζ 2 is the damped natural frequency. The matrix Ac has repeated eigenvalues in exactly two cases, namely, * ωn = 0, {0, 0}ms, mspec(Ac ) = {−ωn, −ωn }ms , ζ = 1. In both of these cases the matrix Ac is defective. In the case ωn = 0, the matrix Ac is also in Jordan form. In particular, in the case ζ = 1, it follows that Ac = SAJ S −1, −1 0 −ωn 1   where S = 0 −ωn . If Ac is not ωn −1 and AJ is the Jordan form matrix AJ = defective, that is, if ωn = 0 and ζ = 1, then the Jordan form AJ of Ac is given by  ⎧ ⎪ −ζω + jω 0 n d ⎪ ⎪ , 0 ≤ ζ < 1, ωn = 0, ⎪ ⎪ 0 −ζωn − jωd ⎪ ⎨  ⎤  AJ = ⎡   ⎪ 2 −1 ω ⎪ −ζ − ζ 0 n ⎪ ⎪  ⎦, ζ > 1, ωn = 0.  ⎪⎣  ⎪ ⎩ 0 −ζ + ζ 2 − 1 ωn In the case 0 ≤ ζ < 1 and ωn = 0, define the real normal form   ωd −ζωn  . An = −ωd −ζωn The matrices Ac , AJ , and An are related by the similarity transformations Ac = S1AJ S1−1 = S2 An S2−1, where



S1 =

1

−ζωn + jωd

−ζωn − jωd





1 S2 = ωd 

 S3 =



1



1 2ωd

1 −ζωn



1 −j 1 j

0 ωd

AJ = S3 An S3−1,

,

S1−1 S2−1

,



j = 2ωd  =



ωd ζωn

 1 S3−1 = ωd j

,

−ζωn − jωd

−1

ζωn − jωd

1

0 1

 ,



1 −j

,  .

In the case ζ > 1 and ωn = 0, the matrices Ac and AJ are related by Ac = S4 AJ S4−1, where

 

S4 =

1

1

−ζωn + jωd

−ζωn − jωd

 ,

S4−1

j = 2ωd



−ζωn − jωd

−1

ζωn − jωd

1

 .

377

MATRIX DECOMPOSITIONS

Finally, define the energy-coordinates matrix   0 ωn  . Ae = −ωn −2ζωn Then, Ae = S5 Ac S5−1, where 

S5 =

m



2

1 0 0 1/ωn

 .

Remark: See Fact 5.12.21.

5.15 Facts on Matrix Factorizations Fact 5.15.1. Let A ∈ Fn×n. Then, A is normal if and only if there exists a unitary matrix S ∈ Fn×n such that A∗ = AS. Proof: See [1125, pp. 102, 113]. C

n×n

Fact 5.15.2. Let A ∈ Cn×n. Then, there exists a nonsingular matrix S ∈ such that SAS −1 is symmetric.

Proof: See [728, p. 209]. Remark: The symmetric matrix is a complex symmetric Jordan form. Remark: See Corollary 5.3.8. Remark: The coefficient of the last matrix in [728, p. 209] should be j/2. Fact 5.15.3. Let A ∈ Cn×n, and assume that A2 is normal. Then, the following statements hold: i) There exists a unitary matrix S ∈ Cn×n such that SAS −1 is symmetric. ii) There exists a symmetric unitary matrix S ∈ Cn×n such that AT = SAS −1. Proof: See [1409]. Fact 5.15.4. Let A ∈ Fn×n, and assume that A is nonsingular. Then, A−1 and A∗ are similar if and only if there exists a nonsingular matrix B ∈ Fn×n such that A = B −1B ∗. Furthermore, A is unitary if and only if there exists a normal, nonsingular matrix B ∈ Fn×n such that A = B −1B ∗. Proof: See [407]. Sufficiency in the second statement follows from Fact 3.11.5. Fact 5.15.5. Let A ∈ Fn×m, and assume that rank A = m. Then, there exist a unique matrix B ∈ Fn×m and a matrix C ∈ Fm×m such that B ∗B = Im , C is upper triangular with positive diagonal entries, and A = BC. Proof: See [728, p. 15] or [1157, p. 206]. Remark: C ∈ UT+ (n). See Fact 3.23.12. Remark: This factorization is a consequence of Gram-Schmidt orthonormalization.

378

CHAPTER 5

Fact 5.15.6. Let A ∈ Fm×m and B ∈ Fn×n. Then, there exist matrices C ∈ Fm×n and D ∈ Fn×m such that A = CD and B = DC if and only if both of the following statements hold: i) The Jordan blocks associated with nonzero eigenvalues are identical in A and B. ii) Let n1 ≥ n2 ≥ · · · ≥ nr denote the orders of the Jordan blocks of A associated with 0 ∈ spec(A), and let m1 ≥ m2 ≥ · · · ≥ mr denote the orders of the Jordan blocks of B associated with 0 ∈ spec(B), where ni = 0 or mi = 0 as needed. Then, |ni − mi | ≤ 1 for all i ∈ {1, . . . , r}. Proof: See [793]. Remark: See Fact 5.15.7. Fact 5.15.7. Let A, B ∈ Fn×n, and assume that A and B are nonsingular. Then, A and B are similar if and only if there exist nonsingular matrices C, D ∈ Fn×n such that A = CD and B = DC. Proof: Sufficiency follows from Fact 5.10.11. Necessity is a special case of Fact 5.15.6. Fact 5.15.8. Let A, B ∈ Fn×n, and assume that A and B are nonsingular. Then, det A = det B if and only if there exist nonsingular matrices C, D, E ∈ Rn×n such that A = CDE and B = EDC. Remark: This result is due to Shoda and Taussky-Todd. See [262]. Fact 5.15.9. Let A ∈ Fn×n. Then, there exist matrices B, C ∈ Fn×n such that B is unitary, C is upper triangular, and A = BC. If, in addition, A is nonsingular, then there exist unique matrices B, C ∈ Fn×n such that B is unitary, C is upper triangular with positive diagonal entries, and A = BC. Proof: See [728, p. 112] or [1157, p. 362]. Remark: This result is the QR decomposition. The orthogonal matrix B is constructed as a product of elementary reflectors. 

Fact 5.15.10. Let A ∈ Fn×n, let r = rank A, and assume that the first r leading principal subdeterminants of A are nonzero. Then, there exist matrices B, C ∈ Fn×n such that B is lower triangular, C is upper triangular, and A = BC. Either B or C can be chosen to be nonsingular. Furthermore, both B and C are nonsingular if and only if A is nonsingular. Proof: See [728, p. 160]. Remark: This result is the LU decomposition. Remark: All LU factorizations of a singular matrix are characterized in [434].

379

MATRIX DECOMPOSITIONS

Fact 5.15.11. Let θ ∈ (−π, π). Then,     1 1 − tan(θ/2) cos θ − sin θ = sin θ 0 1 sin θ cos θ

0 1



1 − tan(θ/2) 0 1

 .

Remark: This result is a ULU factorization involving three shear factors. The matrix −I2 requires four shear factors. In general, all shear factors may be different. See [1271, 1343]. Fact 5.15.12. Let A ∈ Fn×n. Then, A is nonsingular if and only if A is the product of elementary matrices. Problem: How many factors are needed?  Fact 5.15.13. Let A ∈ Fn×n, assume that A is a projector, and let r = rank A. n ∗ Then, there exist nonzero vectors x1, . . . , xn−r ∈ F such that xi xj = 0 for all i = j and such that n−r !

A= I − (x∗i xi )−1xi x∗i .

i=1

Proof: A is unitarily similar to diag(1, . . . , 1, 0, . . . , 0), which can be written as the product of elementary projectors. Remark: Every projector is the product of mutually orthogonal elementary projectors. Fact 5.15.14. Let A ∈ Fn×n. Then, A is a reflector if and only if there exist a positive integer m ≤ n and nonzero vectors x1, . . . , xm ∈ Fn such that x∗i xj = 0 for all i = j and such that A=

m !

I − 2(x∗i xi )−1xi x∗i .

i=1

In this case, m is the algebraic multiplicity of −1 ∈ spec(A). Proof: A is unitarily similar to diag(±1, . . . , ±1), which can be written as the product of elementary reflectors. Remark: Every reflector is the product of mutually orthogonal elementary reflectors. Fact 5.15.15. Let A ∈ Rn×n. Then, A is orthogonal if and only if there exist a positive integer m and nonzero vectors x1, . . . , xm ∈ Rn such that det A = (−1)m and m !

−1 T I − 2(xT A= i xi ) xi xi . i=1

Remark: Every orthogonal matrix is the product of elementary reflectors. This factorization is a result of Cartan and Dieudonn´e. See [106, p. 24] and [1198, 1387]. The minimal number of factors is unsettled. See Fact 3.9.5 and Fact 3.14.4. The complex case is open.

380

CHAPTER 5

Fact 5.15.16. Let A ∈ Rn×n, where n ≥ 2. Then, A is orthogonal and det A = 1 if and only if there exist a positive integer m such that 1 ≤ m ≤ n(n−1)/2, θ1, . . . , θm ∈ R, and j1, . . . , jm , k1, . . . , km ∈ {1, . . . , n} such that A=

m !

P (θi , ji , ki ),

i=1

where  In + [(cos θ) − 1](Ej,j + Ek,k ) + (sin θ)(Ej,k − Ek,j ). P (θ, j, k) =

Proof: See [484]. Remark: P (θ, j, k) is a plane or Givens rotation. See Fact 3.9.5. Remark: Suppose that det A = −1, and let B ∈ Rn×n be an elementary reflector. Then, AB ∈ SO(n). Therefore, the factorization given above holds with an additional elementary reflector. Remark: See [912]. Problem: Generalize this result to Cn×n. Fact 5.15.17. Let A ∈ Fn×n. Then, A2∗A = A∗A2 if and only if there exist a projector B ∈ Fn×n and a Hermitian matrix C ∈ Fn×n such that A = BC. Proof: See [1141]. Fact 5.15.18. Let A ∈ Rn×n. Then, |det A| = 1 if and only if A is the product of n + 2 or fewer involutory matrices that have exactly one negative eigenvalue. In addition, the following statements hold: i) If n = 2, then 3 or fewer factors are needed. ii) If A = αI for all α ∈ R and det A = (−1)n, then n or fewer factors are needed. iii) If det A = (−1)n+1, then n + 1 or fewer factors are needed. Proof: See [306, 1139]. Remark: The minimal number of factors for a unitary matrix A is given in [427].   n and rk = rank Ak for all Fact 5.15.19. Let A ∈ Cn×n, and define r0 = k ∈ P. Then, there exists a matrix B ∈ Cn×n such that A = B 2 if and only if the sequence (rk − rk+1 )∞ k=0 does not contain two components that are the same odd integer and, if r0 − r1 is odd, then r0 + r2 ≥ 1 + 2r1. Now, assume in addition that A ∈ Rn×n. Then, there exists B ∈ Rn×n such that A = B 2 if and only if the above condition holds and, for every negative eigenvalue λ of A and for every positive integer k, the Jordan form of A has an even number of k × k blocks associated with λ.

Proof: See [730, p. 472]. Remark: See Fact 11.18.36.  Remark: For all l ≥ 2, A = Nl does not have a square root.

381

MATRIX DECOMPOSITIONS

Remark: Uniqueness is discussed in [791]. Square roots of A that are functions of A are defined in [696]. Remark: The principal square root is considered in Theorem 10.6.1. Remark: mth roots are considered in [337, 701, 1128, 1294]. Fact 5.15.20. Let A ∈ Cn×n, and assume that A is group invertible. Then, there exists B ∈ Cn×n such that A = B 2. Fact 5.15.21. Let A ∈ Fn×n, and assume that A is nonsingular and has no n×n n×n negative eigenvalues. Furthermore, define (Pk )∞ and (Qk )∞ k=0 ⊂ F k=0 ⊂ F by   P0 = A, Q0 = I, and, for all k ≥ 1,

  Pk + Qk−1 ,    1 −1 . = 2 Qk + Pk  1

Pk+1 = Qk+1 Then,

2

 lim Pk B=

k→∞

exists, satisfies B = A, and is the unique square root of A satisfying spec(B) ⊂ ORHP. Furthermore, lim Qk = A−1. 2

k→∞

Proof: See [406, 695]. Remark: All indicated inverses exist. Remark: This sequence is related to Newton’s iteration for the matrix sign function. See Fact 10.10.2. Remark: See Fact 8.9.33. Fact 5.15.22. Let A ∈ Fn×n, assume that A is positive semidefinite, and let r = rank A. Then, there exists B ∈ Fn×r such that A = BB ∗. 

Fact 5.15.23. Let A ∈ Fn×n, and let k ≥ 1. Then, there exists a unique matrix B ∈ Fn×n such that A = B(B ∗B)k. Proof: See [1118]. F

Fact 5.15.24. Let A ∈ Fn×n. Then, there exist symmetric matrices B, C ∈ , at least one of which is nonsingular, such that A = BC.

n×n

Proof: See [1125, p. 82]. Remark: Note that ⎡ β1 β2 ⎣ β2 1 1 0

⎤⎡ 0 1 0 ⎦⎣ 0 −β0 0

1 0 −β1

⎤ ⎡ 0 −β0 1 ⎦=⎣ 0 0 −β2

0 β2 1

⎤ 0 1 ⎦ 0

382

CHAPTER 5

and use Theorem 5.2.3. Remark: This result is due to Frobenius. The equality is a Bezout matrix factorization; see Fact 4.8.6. See [244, 245, 643]. Remark: B and C are symmetric for F = C. Fact 5.15.25. Let A ∈ Cn×n. Then, det A is real if and only if A is the product of four Hermitian matrices. Furthermore, four is the smallest number for which the previous statement is true. Proof: See [1494]. Fact 5.15.26. Let A ∈ Rn×n. Then, the following statements hold: i) A is the product of two positive-semidefinite matrices if and only if A is similar to a positive-semidefinite matrix. ii) If A is nilpotent, then A is the product of three positive-semidefinite matrices. iii) If A is singular, then A is the product of four positive-semidefinite matrices. iv) det A > 0 and A = αI for all α ≤ 0 if and only if A is the product of four positive-definite matrices. v) det A > 0 if and only if A is the product of five positive-definite matrices. Proof: [121, 643, 1493, 1494]. Remark: See [1494] for factorizations of complex matrices and operators. Example:  −1 0

0 −1



 =

2 0

0 1/2



5 7

7 10



13/2 −5

−5 4



8 5

5 13/4



25/8 −11/2

−11/2 10

 .

Fact 5.15.27. Let A ∈ Rn×n. Then, the following statements hold: i) A = BC, where B ∈ Rn×n is symmetric and C ∈ Rn×n is positive semidefinite, if and only if A2 is diagonalizable over R and spec(A) ⊂ [0, ∞). ii) A = BC, where B ∈ Rn×n is symmetric and C ∈ Rn×n is positive definite, if and only if A is diagonalizable over R. iii) A = BC, where B, C ∈ Rn×n are positive semidefinite, if and only if A = DE, where D ∈ Rn×n is positive semidefinite and E ∈ Rn×n is positive definite. iv) A = BC, where B ∈ Rn×n is positive semidefinite and C ∈ Rn×n is positive definite, if and only if A is diagonalizable over R and spec(A) ⊂ [0, ∞). v) A = BC, where B, C ∈ Rn×n are positive definite, if and only if A is diagonalizable over R and spec(A) ⊂ (0, ∞). Proof: See [724, 1488, 1493].

383

MATRIX DECOMPOSITIONS

Fact 5.15.28. Let A ∈ Fn×n. Then, A is either singular or the identity matrix if and only if A is the product of n or fewer idempotent matrices in Fn×n, each of whose rank is equal to rank A. Furthermore, rank(A − I) ≤ kdef A, where k ≥ 1, if and only if A is the product of k idempotent matrices. Proof: See [74, 129, 386, 473]. Example:



and

0 0 

1 0 2 0



 =

0 0



1 1/2 0 0 

=

1 1 0 0

 

0 0

1/2 1



 1 0 . 1 0

Fact 5.15.29. Let A ∈ Rn×n, assume that A is singular, and assume that A is not a 2 × 2 nilpotent matrix. Then, there exist nilpotent matrices B, C ∈ Rn×n such that A = BC and rank A = rank B = rank C. Proof: See [1246, 1492]. See also [1279]. Fact 5.15.30. Let A ∈ Fn×n, and assume that A is idempotent. Then, there exist B, C ∈ Fn×n such that B is positive definite, C is positive semidefinite, and A = BC. Proof: See [1356]. Fact 5.15.31. Let A ∈ Rn×n, and assume that A is nonsingular. Then, A is similar to A−1 if and only if A is the product of two involutory matrices. If, in addition, A is orthogonal, then A is the product of two reflectors. Proof: See [127, 424, 1486, 1487] or [1125, p. 108].

θ sin θ Problem: Construct these reflectors for A = −cos sin θ cos θ . Fact 5.15.32. Let A ∈ Rn×n. Then, |det A| = 1 if and only if A is the product of four or fewer involutory matrices. Proof: [128, 626, 1245]. Fact 5.15.33. Let A ∈ Rn×n, where n ≥ 2. Then, A is the product of two commutators. Proof: See [1494]. Fact 5.15.34. Let A ∈ Rn×n, and assume that det A = 1. Then, there exist nonsingular matrices B, C ∈ Rn×n such that A = BCB −1C −1. Proof: See [1222]. Remark: The product is a multiplicative commutator. Shoda.

This result is due to

Remark: For nonsingular matrices A, B, note that [A, B] = 0 if and only if ABA−1B = I.

384

CHAPTER 5

Remark: See Fact 5.15.35. Fact 5.15.35. Let A ∈ Rn×n, assume that A is orthogonal, and assume that det A = 1. Then, there exist reflectors B, C ∈ Rn×n such that A = BCB −1C −1. Proof: See [1299]. Remark: See Fact 5.15.34. Fact 5.15.36. Let A ∈ Fn×n, and assume that A is nonsingular. Then, there exist an involutory matrix B ∈ Fn×n and a symmetric matrix C ∈ Fn×n such that A = BC. Proof: See [591]. Fact 5.15.37. Let A ∈ Fn×n, and assume that n is even. Then, the following statements are equivalent: i) A is the product of two skew-symmetric matrices. ii) Every elementary divisor of A has even algebraic multiplicity. iii) There exists a matrix B ∈ Fn/2×n/2 such that A is similar to [ B0

0 B ].

Remark: In i) the factors are skew symmetric even when A is complex. Proof: See [592, 1494]. Fact 5.15.38. Let A ∈ Cn×n, and assume that n ≥ 4 and n is even. Then, A is the product of five skew-symmetric matrices in Cn×n. Proof: See [882, 883]. Fact 5.15.39. Let A ∈ Fn×n. Then, there exist a symmetric matrix B ∈ Fn×n and a skew-symmetric matrix C ∈ Fn×n such that A = BC if and only if A is similar to −A. Proof: See [1163]. 

Fact 5.15.40. Let A ∈ Fn×m, and let r = rank A. Then, there exist matrices B ∈ Fn×r and C ∈ Rr×m such that A = BC and rank B = rank C = r. Fact 5.15.41. Let A ∈ Fn×n. Then, A is diagonalizable over F with (nonnegative, positive) eigenvalues if and only if there exist (positive-semidefinite, positivedefinite) matrices B, C ∈ Fn×n such that A = BC. Proof: To prove sufficiency, use Theorem 8.3.6 and note that   A = S −1(SBS ∗ ) S −∗CS −1 S.

385

MATRIX DECOMPOSITIONS

5.16 Facts on Companion, Vandermonde, Circulant, and Hadamard Matrices Fact 5.16.1. Let p ∈ F[s], where p(s) = sn + βn−1sn−1 + · · · + β0 , and define Cb(p), Cr (p), Ct (p), Cl(p) ∈ Fn×n by ⎡ ⎤ 0 1 0 ··· 0 0 ⎢ ⎥ .. ⎢ 0 ⎥ . 0 0 0 1 ⎢ ⎥ ⎢ ⎥ .. ⎢ 0 ⎥ . 0 0 0 0 ⎢ ⎥  ⎢ ⎥, Cb(p) = ⎢ . ⎥ .. .. .. .. .. ⎢ .. ⎥ . . . . . ⎢ ⎥ ⎢ ⎥ ⎢ 0 ⎥ 0 0 ··· 0 1 ⎣ ⎦ −β0 −β1 −β2 · · · −βn−2 −βn−1 ⎡

0 1 0 .. .

⎢ ⎢ ⎢ ⎢  ⎢ Cr (p) = ⎢ ⎢ ⎢ ⎢ 0 ⎣ 0 ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢  ⎢ Ct (p) = ⎢ ⎢ ⎢ ⎢ ⎢ ⎣

0

0

··· ··· ··· .. . .. .

0

0

···

0 0 1 ..

.

.

0 0 0 .. .

⎤

−β0 −β1 −β2 .. .

0 −βn−2 1

−βn−1

⎥ ⎥ ⎥ ⎥ ⎥ ⎥, ⎥ ⎥ ⎥ ⎦

−βn−1

−βn−2

···

−β2

−β1

−β0

1 .. .

0

···

0 .. .

0 .. .

..

.

0 .. .

0

..

.

0

0

0

0

0

..

.

1

0

0

0

0

···

0

1

0

1 ··· .. . 0 .. . . . . 0 ··· 0 ··· 0 ···

0

0

0

0 .. .

0 .. .

0 .. .

0 0 0

1 0 0

0

⎡

..

−βn−1

⎢ ⎢ −βn−2 ⎢ ⎢ ..  ⎢ . Cl(p) = ⎢ ⎢ −β ⎢ 2 ⎢ ⎣ −β1 −β0 Then,

0 0 0 ..

.

Cr(p) = CbT(p), ˆ b(p)I, ˆ Ct(p) = IC

⎤

⎥ ⎥ ⎥ ⎥ ⎥ ⎥. 0 ⎥ ⎥ ⎥ 1 ⎦ 0

Cl(p) = CtT(p), ˆ r(p)I, ˆ Cl(p) = IC

⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥, ⎥ ⎥ ⎥ ⎥ ⎥ ⎦

386

CHAPTER 5

ˆ

Cl(p) = CbT(p),

ˆ

Ct(p) = CrT(p),

and χCb (p) = χCr(p) = χCt(p) = χCl(p) = p. Furthermore, and

Cr(p) = SCb(p)S −1 ˆ t(p)Sˆ−1, Cl(p) = SC

where S, Sˆ ∈ Fn×n are the Hankel matrices ⎡ β1 β2 · · · . ⎢ ⎢ β2 β3 . . ⎢ .  ⎢ . . S = ⎢ .. .. .. ⎢ . ⎢ .. 1 ⎣ β n−1

1 and

⎡ ⎢ ⎢ ⎢  ˆ Iˆ = ⎢ Sˆ = IS ⎢ ⎢ ⎢ ⎣

0 0

0

0 .. .

0 . ..

0

1

1 βn−1

··· ··· . .. . .. . .. ···

βn−1

1

⎤

0

⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ 0 ⎦

0

0

1 . ..

0 .. .

0

1

1 . ..

βn−1 .. .

β3

β2

β2

β1

⎤ ⎥ ⎥ ⎥ ⎥ ⎥. ⎥ ⎥ ⎦

Remark: (Cb(p), Cr(p), Ct(p), Cl(p)) are the (bottom, right, top, left) companion matrices. Note that Cb(p) = C(p). See [148, p. 282] and [809, p. 659]. Remark: S = B(p, 1), where B(p, 1) is a Bezout matrix. See Fact 4.8.6. Fact 5.16.2. Let p ∈ F[s], where p(s) = sn + βn−1sn−1 + · · · + β0 , assume that β0 = 0, and let ⎡ ⎤ 0 1 0 ··· 0 0 ⎢ ⎥ .. ⎢ 0 ⎥ . 0 0 0 1 ⎢ ⎥ ⎢ ⎥ .. ⎢ 0 ⎥ . 0 0 0 0 ⎢ ⎥  ⎥. Cb (p) = ⎢ ⎢ . ⎥ .. .. .. .. .. ⎢ .. ⎥ . . . . . ⎢ ⎥ ⎢ ⎥ ⎢ 0 ⎥ 0 0 ··· 0 1 ⎣ ⎦ −β0 −β1 −β2 · · · −βn−2 −βn−1

387

MATRIX DECOMPOSITIONS

Then,

⎡ ⎢ ⎢ ⎢ ⎢ −1 p) = ⎢ Cb (p) = Ct (ˆ ⎢ ⎢ ⎢ ⎣

−β1/β0

···

−βn−2/β0

−βn−1/β0

−1/β0

1 .. .

··· .. .

0 .. .

0 .. .

0 .. .

0

···

1

0

0

0

···

0

1

0

⎤ ⎥ ⎥ ⎥ ⎥ ⎥, ⎥ ⎥ ⎥ ⎦

where pˆ(s) = β0−1sn p(1/s). 

Remark: See Fact 4.9.9. Fact 5.16.3. Let λ1, . . . , λn ∈ F, and define the Vandermonde matrix V (λ1, . . . , λn ) ∈ Fn×n by ⎤ ⎡ 1 1 ··· 1 ⎢ λ1 λ2 ··· λn ⎥ ⎥ ⎢ ⎥ ⎢ λ2 2 2 λ · · · λ ⎥ ⎢ 1 2 n  ⎢ ⎥. V (λ1, . . . , λn ) = ⎢ 3 3 3 λ2 ··· λn ⎥ ⎥ ⎢ λ1 ⎥ ⎢ . . . . . . . ⎣ . · ·. · . . ⎦ λn−1 · · · λn−1 λn−1 n 1 2 Then, det V (λ1, . . . , λn ) =

!

(λi − λj ).

1≤i 4. Proof: See [925]. Fact 5.19.4. Let A ∈ Fn×n. Then, the following statements hold: i) A is positive semidefinite, tr A is an integer, and rank A ≤ tr A. ii) There exist projectors B1, . . . , Bl ∈ Fn×n, where l = tr A, such that A = l i=1 Bi . Proof: See [502, 1495].

395

MATRIX DECOMPOSITIONS

Remark: The minimal number of projectors needed in general is tr A. Remark: See Fact 5.19.7. Fact 5.19.5. Let A ∈ Fn×n, assume that A is Hermitian, 0 ≤ A ≤ I, and tr A is a rational number. Then, A is the average of a finite set of projectors in Fn×n. Proof: See [335]. Remark: The required number of projectors can be arbitrarily large. Fact 5.19.6. Let A ∈ Fn×n, assume that A is Hermitian, and assume that 0 ≤ A ≤ I. Then, A is a convex combination of log2 n + 2 projectors in Fn×n. Proof: See [335]. Fact 5.19.7. Let A ∈ Fn×n. Then, the following statements hold: i) tr A is an integer, and rank A ≤ tr A. ii)  There exist idempotent matrices B1, . . . , Bm ∈ Fn×n such that A = m i=1 Bi . iii) There exist a positive integer m and idempotent matrices B1, . . . , Bm ∈ Fn×n such that, for all i ∈ {1, . . . , m}, rank Bi = 1 and R(Bi ) ⊆ A, and such that A = m i=1 Bi . 

iv) There exist idempotent matrices B1, . . . , Bl ∈ Fn×n, where l = tr A, such l that A = i=1 Bi . Proof: See [667, 1247, 1495]. Remark: The minimal number of idempotent matrices is discussed in [1431]. Remark: See Fact 5.19.8. Fact 5.19.8. Let A ∈ Fn×n, and assume that 2(rank A−1) ≤ tr A ≤ 2n. Then, there exist idempotent matrices B, C, D, E ∈ Fn×n such that A = B + C + D + E. Proof: See [899]. Remark: See Fact 5.19.10. Fact 5.19.9. Let A ∈ Fn×n. If n = 2 or n = 3, then there exist b, c ∈ F and idempotent matrices B, C ∈ Fn×n such that A = bB + cC. Furthermore, if n ≥ 4, then there exist b, c, d ∈ F and idempotent matrices B, C, D ∈ Fn×n such that A = bB + cC + dD. Proof: See [1138]. Fact 5.19.10. Let A ∈ Cn×n, and assume that A is Hermitian. If n = 2 or n = 3, then there exist b, c ∈ C and projectors B, C ∈ Cn×n such that A = bB + cC. Furthermore, if 4 ≤ n ≤ 7, then there exist b, c, d ∈ F and projectors B, C, D ∈ Fn×n such that A = bB + cC + dD. If n ≥ 8, then there exist b, c, d, e ∈ C and projectors B, C, D, E ∈ Cn×n such that A = bB + cC + dD + eE.

396

CHAPTER 5

Proof: See [1056]. Remark: See Fact 5.19.8.

5.20 Notes The multicompanion form and the elementary multicompanion form are known as rational canonical forms [457, pp. 472–488], while the multicompanion form is traditionally called the Frobenius canonical form [150]. The derivation of the Jordan form by means of the elementary multicompanion form and the hypercompanion form follows [1108]. Corollary 5.3.8, Corollary 5.3.9, and Proposition 5.5.12 are given in [244, 245, 1288, 1289, 1292]. Corollary 5.3.9 is due to Frobenius. Canonical forms for congruence transformations are given in [909, 1306]. It is sometimes useful to define block-companion form matrices in which the scalars are replaced by matrix blocks [573, 574, 576]. The companion form provides only one of many connections between matrices and polynomials. Additional connections are given by the Leslie, Schwarz, and Routh forms [143]. Given a polynomial expressed in terms of an arbitrary polynomial basis, the corresponding matrix is in confederate form, which specializes to the comrade form when the basis polynomials are orthogonal. The comrade form specializes to the colleague form when Chebyshev polynomials are used. The companion, confederate, comrade, and colleague forms are called congenial matrices. See [143, 145, 148] and Fact 11.18.25 and Fact 11.18.27 for the Schwarz and Routh forms. The companion matrix is sometimes called a Frobenius matrix or the Frobenius canonical form, see [5]. Matrix pencils are discussed in [88, 167, 228, 866, 1373, 1385]. Computational algorithms for the Kronecker canonical form are given in [942, 1391]. Applications to linear system theory are discussed in [319, pp. 52–55] and [813]. Application of the polar decomposition to the elastic deformation of solids is discussed in [1099, pp. 140–142].

Chapter Six

Generalized Inverses

Generalized inverses provide a useful extension of the matrix inverse to singular matrices and to rectangular matrices that are neither left nor right invertible.

6.1 Moore-Penrose Generalized Inverse Let A ∈ Fn×m. If A is nonzero, then, by the singular value decomposition Theorem 5.6.3, there exist orthogonal matrices S1 ∈ Fn×n and S2 ∈ Fm×m such   that B 0r×(m−r) A = S1 S2 , (6.1.1) 0(n−r)×r 0(n−r)×(m−r) 



where B = diag[σ1(A), . . . , σr (A)], r = rank A, and σ1(A) ≥ σ2 (A) ≥ · · · ≥ σr (A) > 0 are the positive singular values of A. In (6.1.1), some of the bordering zero matrices may be empty. Then, the (Moore-Penrose) generalized inverse A+ of A is the m × n matrix   0r×(n−r) B −1 +  ∗ A = S2 S1∗ . (6.1.2) 0(m−r)×r 0(m−r)×(n−r)  0m×n , while, if m = n and det A = 0, then A+ = A−1. In If A = 0n×m, then A+ = general, it is helpful to remember that A+ and A∗ are the same size. It is easy to verify that A+ satisfies (6.1.3) AA+A = A,

A+AA+ = A+, + ∗

(6.1.4)

+

(6.1.5)

(A A) = A A.

(6.1.6)

(AA ) = AA , +

∗

+

Hence, for each A ∈ Fn×m there exists a matrix X ∈ Fm×n satisfying the four conditions AXA = A, (6.1.7) XAX = X, (6.1.8) (AX)∗ = AX, (XA)∗ = XA. We now show that X is uniquely defined by (6.1.7)–(6.1.10).

(6.1.9) (6.1.10)

398 F

m×n

CHAPTER 6

Theorem 6.1.1. Let A ∈ Fn×m. Then, X = A+ is the unique matrix X ∈ satisfying (6.1.7)–(6.1.10).

Proof. Suppose there exists a matrix X ∈ Fm×n satisfying (6.1.7)–(6.1.10). Then, X = XAX = X(AX)∗ = XX ∗A∗ = XX ∗(AA+A)∗ = XX ∗A∗A+∗A∗ = X(AX)∗ (AA+ )∗ = XAXAA+ = XAA+ = (XA)∗A+ = A∗X ∗A+ = (AA+A)∗X ∗A+ = A∗A+∗A∗X ∗A+ = (A+A)∗ (XA)∗A+ = A+AXAA+ = A+AA+ = A+.



Given A ∈ Fn×m, X ∈ Fm×n is a (1)-inverse of A if (6.1.7) holds, a (1,2)inverse of A if (6.1.7) and (6.1.8) hold, and so forth. Proposition 6.1.2. Let A ∈ Fn×m, and assume that A is right invertible. Then, X ∈ Fm×n is a right inverse of A if and only if X is a (1)-inverse of A. Furthermore, every right inverse (or, equivalently, every (1)-inverse) of A is also a (2,3)-inverse of A. Proof. Suppose that AX = In, that is, X ∈ Fm×n is a right inverse of A. Then, AXA = A, which implies that X is a (1)-inverse of A. Conversely, let X ˆ ∈ Fm×n denote a right be a (1)-inverse of A, that is, AXA = A. Then, letting X ˆ ˆ inverse of A, it follows that AX = AXAX = AX = In . Hence, X is a right inverse of A. Finally, if X is a right inverse of A, then it is also a (2,3)-inverse of A. Proposition 6.1.3. Let A ∈ Fn×m, and assume that A is left invertible. Then, X ∈ Fm×n is a left inverse of A if and only if X is a (1)-inverse of A. Furthermore, every left inverse (or, equivalently, every (1)-inverse) of A is also a (2,4)-inverse of A. It can now be seen that A+ is a particular (right, left) inverse when A is (right, left) invertible. Corollary 6.1.4. Let A ∈ Fn×m. If A is right invertible, then A+ is a right inverse of A. Furthermore, if A is left invertible, then A+ is a left inverse of A. The following result provides an explicit expression for A+ when A is either right invertible or left invertible. It is helpful to note that A is (right, left) invertible if and only if (AA∗, A∗A) is positive definite. Proposition 6.1.5. Let A ∈ Fn×m. If A is right invertible, then A+ = A∗ (AA∗ )−1

(6.1.11)

and A+ is a right inverse of A. If A is left invertible, then A+ = (A∗A)−1A∗ and A+ is a left inverse of A.

(6.1.12)

GENERALIZED INVERSES

Proof. It suffices to verify (6.1.7)–(6.1.10) with X = A+. Proposition 6.1.6. Let A ∈ Fn×m. Then, the following statements hold: i) A = 0 if and only if A+ = 0. ii) (A+ )+ = A. +

iii) A = A+ .  +  iv) A+T = AT = (A+ )T. 

v) A+∗ = (A∗ )+ = (A+ )∗. vi) R(A) = R(AA∗ ) = R(AA+ ) = R(A+∗ ) = N(I − AA+ ) = N(A∗ )⊥. vii) R(A∗ ) = R(A∗A) = R(A+A) = R(A+ ) = N(I − A+A) = N(A)⊥. viii) N(A) = N(A+A) = N(A∗A) = N(A+∗ ) = R(I − A+A) = R(A∗ )⊥. ix) N(A∗ ) = N(AA+ ) = N(AA∗ ) = N(A+ ) = R(I − AA+ ) = R(A)⊥. x) AA+ and A+A are positive semidefinite. xi) spec(AA+ ) ⊆ {0, 1} and spec(A+A) ⊆ {0, 1}. xii) AA+ is the projector onto R(A). xiii) A+A is the projector onto R(A∗ ). xiv) Im − A+A is the projector onto N(A). xv) In − AA+ is the projector onto N(A∗ ). xvi) x ∈ R(A) if and only if x = AA+x. xvii) rank A = rank A+ = rank AA+ = rank A+A = tr AA+ = tr A+A. xviii) rank(Im − A+A) = m − rank A. xix) rank(In − AA+ ) = n − rank A. xx) (A∗A)+ = A+A+∗. xxi) (AA∗ )+ = A+∗A+. xxii) AA+ = A(A∗A)+A∗. xxiii) A+A = A∗(AA∗ )+A. xxiv) A = AA∗A∗+ = A∗+A∗A. xxv) A∗ = A∗AA+ = A+AA∗. xxvi) A+ = A∗(AA∗ )+ = (A∗A)+A∗ = A∗(A∗AA∗ )+A∗. xxvii) A+∗ = (AA∗ )+A = A(A∗A)+. xxviii) A = A(A∗A)+A∗A = AA∗A(A∗A)+. xxix) A = AA∗(AA∗ )+A = (AA∗ )+AA∗A. xxx) If S1 ∈ Fn×n and S2 ∈ Fm×m are orthogonal, then (S1AS2 )+ = S2∗ A+S1∗.

399

400

CHAPTER 6

xxxi) A is (range Hermitian, normal, Hermitian, positive semidefinite, positive definite) if and only if A+ is. xxxii) If A is a projector, then A+ = A. xxxiii) A+ = A if and only if A is tripotent and A2 is Hermitian. xxxiv) If B ∈ Fn×l, then R(AA+B) is the projection of R(B) onto R(A). Proof. The last equality in xxvi) is given in [1539]. Theorem 2.6.4 shows that the equation Ax = b, where A ∈ Fn×m and b ∈ Fn, has a solution x ∈ Fm if and only if rank A = rank A b . In particular, Ax = b

has a unique solution x ∈ Fm if and only if rank A = rank A b = m, while Ax = b has infinitely many solutions if and only if rank A = rank A b < m. The following result characterizes these solutions in terms of the generalized inverse. Connections to least squares solutions are discussed in Fact 9.15.4. Proposition 6.1.7. Let A ∈ Fn×m and b ∈ Fn. Then, the following statements are equivalent: i) There exists a vector x ∈ Fm satisfying Ax = b.

ii) rank A = rank A b . iii) b ∈ R(A). iv) AA+ b = b. Now, assume in addition that i)–iv) are satisfied. Then, the following statements hold: v) x ∈ Fm satisfies Ax = b if and only if x = A+ b + (I − A+A)x.

(6.1.13)

vi) For all y ∈ Fm, x ∈ Fm given by x = A+ b + (I − A+A)y

(6.1.14)

satisfies Ax = b. vii) Let x ∈ Fm be given by (6.1.14), where y ∈ Fm. Then, y = 0 minimizes x∗x. viii) Assume that rank A = m. Then, there exists a unique vector x ∈ Fm satisfying Ax = b given by x = A+ b. If, in addition, AL ∈ Fm×n is a left inverse of A, then AL b = A+ b. ix) Assume that rank A = n, and let AR ∈ Fm×n be a right inverse of A. Then, x = AR b satisfies Ax = b. Proof. The equivalence of i)–iii) is immediate. To prove the equivalence of iv), note that, if there exists a vector x ∈ Fm satisfying Ax = b, then b = Ax = AA+Ax = AA+ b. Conversely, if b = AA+ b, then x = A+ b satisfies Ax = b.

401

GENERALIZED INVERSES

Now, suppose that i)–iv) hold. To prove v), let x ∈ Fm satisfy Ax = b so that A Ax = A+ b. Hence, x = x + A+ b −A+Ax = A+ b + (I −A+A)x. To prove vi), let y ∈ Fm, and let x ∈ Fm be given by (6.1.14). Then, Ax = AA+b = b. To prove vii), let y ∈ Fm, and let x ∈ Fn be given by (6.1.14). Then, x∗x = b∗A+∗A+b + y ∗(I − A+A)y. Therefore, x∗x is minimized by y = 0. See also Fact 9.15.4. +

To prove viii), suppose that rank A = m. Then, A is left invertible, and it follows from Corollary 6.1.4 that A+ is a left inverse of A. Hence, it follows from (6.1.13) that x = A+b is the unique solution of Ax = b. In addition, x = ALb. To prove ix), let x = AR b, and note that AAR b = b. Definition 6.1.8. Let A ∈ Fn×m, B ∈ Fn×l, C ∈ Fk×m, and D ∈ Fk×l, and  A B ] ∈ F(n+k)×(m+l). Then, the Schur complement A|A of A with [C define A = D respect to A is defined by  A|A = D − CA+B. (6.1.15) Likewise, the Schur complement D|A of D with respect to A is defined by 

D|A = A − BD+C.

(6.1.16)

6.2 Drazin Generalized Inverse We now introduce a different type of generalized inverse, which applies only to square matrices yet is more useful in certain applications. Let A ∈ Fn×n. Then, A has a decomposition   J1 0 A=S S −1, (6.2.1) 0 J2 where S ∈ Fn×n is nonsingular, J1 ∈ Fm×m is nonsingular, and J2 ∈ F(n−m)×(n−m) is nilpotent. Then, the Drazin generalized inverse AD of A is the matrix   −1 0 J1  S −1. (6.2.2) AD = S 0 0 Let A ∈ Fn×n. Then, it follows from Definition 5.5.1 that ind A = indA(0). Furthermore, A is nonsingular if and only if ind A = 0, whereas ind A = 1 if and only if A is singular and the zero eigenvalue of A is semisimple. In particular, ind 0n×n = 1. Note that ind A is the order of the largest Jordan block of A associated with the zero eigenvalue of A. It can be seen that AD satisfies ADAAD = AD, D

D

AA = A A, k+1 D

A

k

A =A,

(6.2.3) (6.2.4) (6.2.5)

where k = ind A. Hence, for all A ∈ Fn×n such that ind A = k there exists a matrix

402

CHAPTER 6

X ∈ Fn×n satisfying the three conditions XAX = X, AX = XA,

(6.2.6) (6.2.7)

Ak+1X = Ak.

(6.2.8)

We now show that X is uniquely defined by (6.2.6)–(6.2.8). 

Theorem 6.2.1. Let A ∈ Fn×n, and let k = ind A. Then, X = AD is the unique matrix X ∈ Fn×n satisfying (6.2.6)–(6.2.8). Proof. Let X ∈ Fn×n satisfy (6.2.6)–(6.2.8). If k = 0, then it follows from

(6.2.8) that X = A−1. Hence, let A = S J01 J02 S −1, where k = ind A ≥ 1, S ∈ Fn×n is nonsingular, J1 ∈ Fm×m and J2 ∈ F(n−m)×(n−m) is nilpotent.   ˆ isˆ nonsingular,   X X 1 12 ˆ = S −1XS = Now, let X be partitioned conformably with Aˆ = S −1AS = ˆ21 X ˆ2 X J1 0 ˆˆ ˆˆ ˆ ˆ ˆ ˆ 0 J2 . Since, by (6.2.7), AX = XA, it follows that J1X1 = X1J1, J1X12 = X12 J2 , k−1 k ˆ21J1, and J2 X ˆ2 J2 . Since J = 0, it follows that J1X ˆ21 = X ˆ2 = X ˆ12 J J2 X = 0, and 2 2 k−1 ˆ ˆ thus X12 J2 = 0. By repeating this argument, it follows that J1X12 J2 = 0, and ˆ12 = 0. Similarly, X ˆ21 = ˆ12 = 0, and thus X ˆ12 J2 = 0, which implies that J1X thus X   ˆ X 0 k+1 ˆ = 1 ˆ1 = J −1. ˆ1 = J k, and hence X 0, so that X . Now, (6.2.8) implies that J X 1

1

ˆ2 0 X

1

ˆ2 J2 X ˆ2 , which, together with J2 X ˆ2 J2 , yields ˆ2 = X ˆ2 = X Next, (6.2.6) implies that X k−1 2 2 k ˆ ˆ ˆ ˆ X2 J2 = X2 . Consequently, 0 = X2 J2 = X2 J2  , and  thus, by repeating this −1 ˆ1 0 D −1 J 0 X ˆ −1 = X. ˆ 1 S =S S −1 = SXS argument, X2 = 0. Hence, A = S 0

0

0 0 

Proposition 6.2.2. Let A ∈ Fn×n, and define k = ind A. Then, the following statements hold: D

i) A = AD .  D   ii) ADT = ATD = AT = (AD )T.   A∗D = (A∗ )D = (AD )∗. iii) AD∗ =  r   iv) If r ∈ P, then ADr = ArD = AD = (Ar )D.

v) R(Ak ) = R(AD ) = R(AAD ) = N(I − AAD ). vi) N(Ak ) = N(AD ) = N(AAD ) = R(I − AAD ). vii) rank Ak = rank AD = rank AAD = def(I − AAD ). viii) def Ak = def AD = def AAD = rank(I − AAD ). ix) AAD is the idempotent matrix onto R(AD ) along N(AD ). x) AD = 0 if and only if A is nilpotent. xi) AD is group invertible. xii) ind AD = 0 if and only if A is nonsingular. xiii) ind AD = 1 if and only if A is singular.

403

GENERALIZED INVERSES

xiv) (AD )D = (AD )# = A2AD. xv) (AD )D = A if and only if A is group invertible. xvi) If A is idempotent, then k = 1 and AD = A. xvii) A = AD if and only if A is tripotent. Let A ∈ Fn×n, and assume that ind A ≤ 1 so that, by Corollary 5.5.9, A is group invertible. In this case, the Drazin generalized inverse AD is denoted by A#, which is the group generalized inverse of A. Therefore, A# satisfies A#AA# = A#, #

#

AA = A A, #

AA A = A,

(6.2.9) (6.2.10) (6.2.11)

while A# is the unique matrix X ∈ Fn×n satisfying XAX = X,

(6.2.12)

AX = XA, AXA = A.

(6.2.13) (6.2.14)

Proposition 6.2.3. Let A ∈ Fn×n, and assume that A is group invertible. Then, the following statements hold: #

i) A = A# .  #   ii) A#T = AT# = AT = (A# )T.   A∗# = (A∗ )# = (A# )∗. iii) A#∗ =  r   iv) If r ∈ P, then A#r = Ar# = A# = (Ar )#.

v) R(A) = R(AA# ) = N(I − AA# ) = R(AA+ ) = N(I − AA+ ). vi) N(A) = N(AA# ) = R(I − AA# ) = N(A+A) = R(I − A+A). vii) rank A = rank A# = rank AA# = rank A#A. viii) def A = def A# = def AA# = def A#A. ix) AA# is the idempotent matrix onto R(A) along N(A). x) A# = 0 if and only if A = 0. xi) A# is group invertible. xii) (A# )# = A. xiii) If A is idempotent, then A# = A. xiv) A = A# if and only if A is tripotent. An alternative expression for the idempotent matrix onto R(A) along N(A) is given by Proposition 3.5.9.

404

CHAPTER 6

6.3 Facts on the Moore-Penrose Generalized Inverse for One Matrix Fact 6.3.1. Let A ∈ Fn×m , x ∈ Fm, b ∈ Fn, and y ∈ Fm, assume that A is right invertible, and assume that x = A+ b + (I − A+A)y, which satisfies Ax = b. Then, there exists a right inverse AR ∈ Fm×n of A such that  (I − A+A)y, x = AR b. Furthermore, if S ∈ Fm×n is such that z TSb = 0, where z = then one such right inverse is given by AR = A+ +

1 zz TS. z TSb

Fact 6.3.2. Let A ∈ Fn×m, and assume that rank A = 1. Then, A+ = (tr AA∗ )−1A∗. Consequently, if x ∈ Fn and y ∈ Fn are nonzero, then (xy ∗ )+ = (x∗xy ∗y)−1 yx∗ = In particular,

1+ n×m =

1 yx∗. x22 y22

1 nm 1m×n .

Fact 6.3.3. Let x ∈ Fn, and assume that x is nonzero. Then, the projector A ∈ Fn×n onto span {x} is given by A = (x∗x)−1xx∗. Fact 6.3.4. Let x, y ∈ Fn, assume that x, y are nonzero, and assume that x y = 0. Then, the projector A ∈ Fn×n onto span {x, y} is given by ∗

A = (x∗x)−1xx∗ + (y ∗y)−1yy ∗. Fact 6.3.5. Let x, y ∈ Fn, and assume that x, y are linearly independent. Then, the projector A ∈ Fn×n onto span {x, y} is given by A = (x∗xy ∗ y − |x∗y|2 )−1 (y ∗ yxx∗ − y ∗xyx∗ − x∗ yxy ∗ + x∗xyy ∗ ). 

Furthermore, define z = [I − (x∗x)−1 xx∗ ]y. Then, A = (x∗x)−1xx∗ + (z ∗z)−1zz ∗. Remark: For F = R, this result is given in [1237, p. 178]. Fact 6.3.6. Let A ∈ Fn×m, assume that rank A = n − 1, let x ∈ N(A) be nonzero, let y ∈ N(A∗ ) be nonzero, let α = 1 if spec(A) = {0} and the product of  the nonzero eigenvalues of A otherwise, and define k = amultA(0). Then, AA = In particular,

(−1)k+1α xy ∗. y ∗(Ak−1 )+ x

NnA = (−1)n+1E1,n .

405

GENERALIZED INVERSES

If, in addition, k = 1, then AA =

α xy ∗. y ∗x

Proof: See [973, p. 41] and Fact 3.17.4. Remark: This result provides an expression for ii) of Fact 2.16.8. Remark: If A is range Hermitian, then N(A) = N(A∗ ) and y ∗x = 0, and thus Fact 5.14.3 implies that AA is semisimple. Remark: See Fact 5.14.25. Fact 6.3.7. Let A ∈ Fn×m, and assume that rank A = n − 1. Then, A+ =

1 A∗ [AA∗ det[AA∗ +(AA∗ )A ]

+ (AA∗ )A ]A.

Proof: See [353]. Remark: Extensions to matrices of arbitrary rank are given in [353]. Fact 6.3.8. Let A ∈ Fn×m, B ∈ Fk×n, and C ∈ Fm×l, and assume that B is left inner and C is right inner. Then, (BAC)+ = C ∗A+B ∗. Proof: See [671, p. 506]. Fact 6.3.9. Let A ∈ Fn×n. Then,

rank [A, A+ ] = 2(rank A A∗ − rank A)   = rank A − A2A+   = rank A − A+A2 .

Furthermore, the following statements are equivalent: i) A is range Hermitian. ii) [A, A+ ] = 0.

iii) rank A A∗ = rank A. iv) A = A2A+. v) A = A+A2. Proof: See [1338]. Remark: See Fact 3.6.3, Fact 6.3.10, and Fact 6.4.13. Fact 6.3.10. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A is range Hermitian. ii) R(A) = R(A+ ). iii) A+A = AA+. iv) (I − A+A)⊥ = AA+.

406

CHAPTER 6

v) A = A2A+. vi) A = A+A2. vii) AA+ = A2(A+ )2. viii) (AA+ )2 = A2(A+ )2. ix) (A+A)2 = (A+ )2A2.  + x) ind A ≤ 1, and (A+ )2 = A2 . xi) ind A ≤ 1, and AA+A∗A = A∗A2A+. xii) A2A+ + A∗A+∗A = 2A. ∗  xiii) A2A+ + A2A+ = A + A∗. xiv) R(A − A+ ) = R(A − A3 ). xv) R(A + A+ ) = R(A + A3 ). Proof: See [331, 1313, 1328, 1363] and Fact 6.6.9. Remark: See Fact 3.6.3, Fact 6.3.9, and Fact 6.4.13. Fact 6.3.11. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A + A+ = 2AA+. ii) A + A+ = 2A+A. iii) A + A+ = AA+ + A+A. iv) A is range Hermitian, and A2 + AA+ = 2A. v) A is range Hermitian, and (I − A)2A = 0. Proof: See [1355, 1362]. Fact 6.3.12. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A+A∗ = A∗A+. ii) AA+A∗A = AA∗A+A. iii) AA∗A2 = A2A∗A. If these conditions hold, then A is star-dagger. If A is star-dagger, then A2(A+ )2 and (A+ )2A2 are positive semidefinite. Proof: See [668, 1313]. Remark: See Fact 6.3.15. Fact 6.3.13. Let A ∈ Fn×m, let B, C ∈ Fm×n, assume that B is a (1, 3) inverse of A, and assume that C is a (1, 4) inverse of A. Then, A+ = CAB. Proof: See [178, p. 48]. Remark: This result is due to Urquhart.

407

GENERALIZED INVERSES 

Fact 6.3.14. Let A ∈ Fn×m, assume that A is nonzero, let r = rank A, define B = diag[σ1(A), . . . , σr (A)], and let S ∈ Fn×n, K ∈ Fr×r, and L ∈ Fr×(m−r) be such that S is unitary, KK ∗ + LL∗ = Ir , 

and

 A=S

Then,

BK

BL

0(n−r)×r

0(n−r)×(m−r)

 +

A =S

K ∗B −1

0r×(n−r)

L∗B −1

0(m−r)×(n−r)

 S ∗.  S ∗.

Proof: See [119, 668]. Remark: See Fact 5.9.30 and Fact 6.6.16. Fact 6.3.15. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A is normal. ii) AA∗A+ = A+AA∗. iii) A is range Hermitian, and A+A∗ = A∗A+. iv) A(AA∗A)+ = (AA∗A)+A. v) AA+A∗A2A+ = AA∗. vi) A(A∗ + A+ ) = (A∗ + A+ )A. vii) A∗A(AA∗ )+A∗A = AA∗. viii) 2AA∗(AA∗ + A∗A)+AA∗ = AA∗. ix) There exists a matrix X ∈ Fn×n such that AA∗X = A∗A and A∗AX = AA∗. x) There exists a matrix X ∈ Fn×n such that AX = A∗ and A+∗X = A+. Proof: See [331]. Remark: See Fact 3.7.12, Fact 3.11.5, Fact 5.15.4, Fact 6.3.12, and Fact 6.6.11. Fact 6.3.16. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A is Hermitian. ii) AA+ = A∗A+. iii) A2A+ = A∗. iv) AA∗A+ = A. Proof: See [119]. Fact 6.3.17. Let A ∈ Fn×m, and assume that rank A = m. Then, (AA∗ )+ = A(A∗A)−2A∗.

408

CHAPTER 6

Remark: See Fact 6.4.10. Fact 6.3.18. Let A ∈ Fn×m. Then, A+ = lim A∗(AA∗ + αI)−1 = lim (A∗A + αI)−1A∗. α↓0

α↓0

Fact 6.3.19. Let A ∈ Fn×m, let χAA∗ (s) = sn + βn−1sn−1 + · · · + β1s + β0 , and let k denote the largest integer in {0, . . . , n − 1} such that βn−k = 0. Then,

−1 A∗ (AA∗ )k−1 + βn−1(AA∗ )k−2 + · · · + βn−k+1 I . A+ = −βn−k Proof: See [402]. Fact 6.3.20. Let A ∈ Fn×n, and assume that A is Hermitian. Then, In A = In A+ = In AD. If, in addition, A is nonsingular, then In A = In A−1. Fact 6.3.21. Let A ∈ Fn×n, and consider the following statements: i) A is idempotent. ii) rank A = tr A. iii) rank A ≤ tr A2A+A∗. Then, i) =⇒ ii) =⇒ iii). Furthermore, the following statements are equivalent: iv) A is idempotent. v) rank A = tr A = tr A2A+A∗. vi) There exist projectors B, C ∈ Fn×n such that A+ = BC. vii) A∗A+ = A+. viii) A+A∗ = A+. Proof: See [830] and [1215, p. 166]. Fact 6.3.22. Let A ∈ Fn×n, and assume that A is idempotent. Then, A∗A+A = A+A and

AA+A∗ = AA+.

Proof: Note that A∗A+A is a projector, and R(A∗A+A) = R(A∗ ) = R(A+A). Alternatively, use Fact 6.3.21. Fact 6.3.23. Let A ∈ Fn×n, and assume that A is idempotent. Then, A+A + (I − A)(I − A)+ = I and

AA+ + (I − A)+(I − A) = I.

409

GENERALIZED INVERSES

Proof: N(A) = R(I − A+A) = R(I − A) = R[(I − A)(I − A+ )]. Remark: The first equality states that the projector onto the null space of A is the same as the projector onto the range of I − A, while the second equality states that the projector onto the range of A is the same as the projector onto the null space of I − A. Remark: See Fact 3.13.24 and Fact 5.12.18. Fact 6.3.24. Let A ∈ Fn×n, and assume that A is idempotent. Then, A + A − I is nonsingular, and ∗

(A + A∗ − I)−1 = AA+ + A+A − I. Proof: Use Fact 6.3.22. Remark: See Fact 3.13.24, Fact 5.12.18, or [1023, p. 457] for a geometric interpretation of this equality. Fact 6.3.25. Let A ∈ Fn×n, and assume that A is idempotent. Then, 2A(A + A ) A is the projector onto R(A) ∩ R(A∗ ). ∗ + ∗

Proof: See [1352]. Fact 6.3.26. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A+ is idempotent. ii) AA∗A = A2. If A is range Hermitian, then the following statements are equivalent: iii) A+ is idempotent. iv) AA∗ = A∗A = A. The following statements are equivalent: v) A+ is a projector. vi) A is a projector. vii) A is idempotent, and A and A+ are similar. viii) A is idempotent, and A = A+. ix) A is idempotent, and AA+ = AA∗. x) A+ = A, and A2 = A∗. xi) A and A+ are idempotent. xii) A = AA+. Proof: See [1215, pp. 167, 168] and [1313, 1358, 1457]. Remark: See Fact 3.13.1.

410

CHAPTER 6 

Fact 6.3.27. Let A ∈ Fn×m, and let r = rank A. Then, the following statements are equivalent: i) AA∗ is a projector. ii) A∗A is a projector. iii) AA∗A = A. iv) A∗AA∗ = A∗. v) A+ = A∗. vi) σ1(A) = σr (A) = 1. In particular, Nn+ = NnT. Proof: See [178, pp. 219–220]. Remark: A is a partial isometry, which preserves lengths and distances with respect to the Euclidean norm on R(A∗ ). See [178, p. 219]. Remark: See Fact 5.11.30. 

Fact 6.3.28. Let A ∈ Fn×m, assume that A is nonzero, and let r = rank A. Then, for all i ∈ {1, . . . , r}, the singular values of A+ are given by −1 (A). σi (A+ ) = σr+1−i

In particular,

σr (A) = 1/σmax (A+ ).

If, in addition, A ∈ Fn×n and A is nonsingular, then   σmin (A) = 1/σmax A−1 . Fact 6.3.29. Let A ∈ Fn×m. Then, X = A+ is the unique matrix satisfying   A AA+ = rank A. rank A+A X Proof: See [496]. Remark: See Fact 2.17.10 and Fact 6.6.2. Fact 6.3.30. Let A ∈ Fn×n, and assume that A is centrohermitian. Then, A is centrohermitian. +

Proof: See [908]. Fact 6.3.31. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A2 = AA∗A. ii) A is the product of two projectors. iii) A = A(A+ )2A. Remark: This result is due to Crimmins. See [1141].

411

GENERALIZED INVERSES

Fact 6.3.32. Let A ∈ Fn×m. Then, A+ = 4(I + A+A)+A+(I + AA+ )+. Proof: Use Fact 6.4.41 with B = A. Fact 6.3.33. Let A ∈ Fn×n, and assume that A is unitary. Then, 1 k→∞ k

lim

k−1 

Ai = I − (A − I)(A − I)+.

i=0

Proof: Use Fact 11.21.13 and Fact 11.21.15, and note that (A − I)∗ = (A − I)+. See [641, p. 185]. Remark: I − (A − I)(A − I)+ is the projector onto {x: Ax = x} = N(A − I). Remark: This result is the ergodic theorem. Fact 6.3.34. Let A ∈ Fn×m, and define the sequence (Bi )∞ i=1 by 

Bi+1 = 2Bi − BiABi , 

2 where B0 = αA∗ and α ∈ (0, 2/σmax (A)). Then,

lim Bi = A+.

i→∞

Proof: See [148, p. 259] or [291, p. 250]. This result is due to Ben-Israel. Remark: This sequence is a Newton-Raphson algorithm. Remark: B0 satisfies sprad(I − B0 A) < 1. Remark: For the case in which A is square and nonsingular, see Fact 2.16.29. Problem: Does convergence hold for all B0 ∈ Fn×n satisfying sprad(I − B0 A) < 1? n×m , and assume that limi→∞ Ai Fact 6.3.35. Let A ∈ Fn×m, let (Ai )∞ i=1 ⊂ F + = A. Then, limi→∞ A+ = A if and only if there exists a positive integer k such i that, for all i > k, rank Ai = rank A.

Proof: See [291, pp. 218, 219] or [1208, pp. 199, 200].

6.4 Facts on the Moore-Penrose Generalized Inverse for Two or More Matrices Fact 6.4.1. Let A ∈ Fn×m and B ∈ Fm×n. Then, the following statements are equivalent: i) B = A+. ii) A∗AB = A∗ and B ∗BA = B ∗. iii) BAA∗ = A∗ and ABB ∗ = B ∗. Remark: See [671, pp. 503, 513].

412

CHAPTER 6

Fact 6.4.2. Let A ∈ Fn×m, and let x ∈ Fn and y ∈ Fm be nonzero. Furthermore, define 

d = A+x,





δ = d∗d, 

η = e∗e, 

λ = 1 + y ∗A+x, 

Then,



f = (I − AA+ )x,

e = A+∗ y,

φ = f ∗f,

ψ = g ∗ g,





μ = |λ|2 + δψ,

g = (I − A+A)y, 



ν = |λ|2 + ηφ.

rank(A + xy ∗ ) = rank A − 1

if and only if

x ∈ R(A),

y ∈ R(A∗ ),

λ = 0.

In this case, (A + xy ∗ )+ = A+ − δ −1dd∗A+ − η −1A+ee∗ + (δη)−1d∗A+ede∗. Furthermore,

rank(A + xy ∗ ) = rank A

if and only if one of the following conditions is satisfied: ⎧ ∗ ⎪ ⎨x ∈ R(A), y ∈ R(A ), λ = 0, x ∈ R(A), y ∈ / R(A∗ ), ⎪ ⎩ x∈ / R(A), y ∈ R(A∗ ). In this case, respectively, ⎧ ∗ + + −1 ∗ ⎪ ⎨(A + xy ) = A − λ de , ∗ + + (A + xy ) = A − μ−1 (ψdd∗A+ + δge∗ ) + μ−1 (λgd∗A+ − λde∗ ), ⎪ ⎩ (A + xy ∗ )+ = A+ − ν −1 (φA+ ee∗ + ηdf ∗ ) + ν −1 (λA+ef ∗ − λde∗ ). Finally,

rank(A + xy ∗ ) = rank A + 1

if and only if

x∈ / R(A),

y∈ / R(A∗ ).

In this case, (A + xy ∗ )+ = A+ − φ−1df ∗ − ψ −1ge∗ + λ(φψ)−1gf ∗. Proof: See [111]. To prove sufficiency in the first alternative of the third statement, let x ˆ ∈ Fm and yˆ ∈ Fn be such that x = Aˆ x and y = A∗ yˆ. Then, A + xy ∗ = ∗ A(I + x ˆy ). Since α = 0 it follows that −1 = y ∗A+x = yˆ∗AA+Aˆ x = yˆ∗Aˆ x = y ∗x ˆ. ∗ It now follows that I + x ˆy is an elementary matrix and thus, by Fact 3.7.19, is nonsingular. Remark: An equivalent version of the first statement is given in [338] and [740, p. 33]. A detailed treatment of the generalized inverse of an outer-product perturbation is given in [1430, pp. 152–157]. Remark: See Fact 2.10.25.

413

GENERALIZED INVERSES

Fact 6.4.3. Let A ∈ Fn×n, assume that A is Hermitian and nonsingular, and let x ∈ Fn and y ∈ Fn be nonzero. Then, A + xy ∗ is singular if and only if y ∗A−1x + 1 = 0. In this case, (A + xy ∗ )+ = (I − aa+ )A−1(I − bb+ ), 



where a = A−1x and b = A−1y. Proof: See [1208, pp. 197, 198]. Fact 6.4.4. Let A ∈ Fn×n, assume that A is Hermitian, let b ∈ Fn, and define S = I − A+A. Then, 

(A + bb∗ )+



⎧ I − (b∗(A+ )2b)−1A+bb∗A+ A+ I − (b∗(A+ )2b)−1A+bb∗A+ , 1+ b∗A+b = 0, ⎪ ⎪ ⎪ ⎪ ⎨ = A+ − (1 + b∗A+b)−1A+bb∗A+, 1+ b∗A+b = 0, ⎪ ⎪ ⎪ ⎪



⎩ I − (b∗Sb)−1Sbb∗ A+ I − (b∗Sb)−1bb∗S + (b∗Sb)−2Sbb∗S, b∗Sb = 0. Proof: See [1031]. F

Fact 6.4.5. Let A ∈ Fn×n, assume that A is positive semidefinite, let C ∈ , assume that C is positive definite, and let B ∈ Fn×m. Then,  −1 (A + BCB ∗ )+ = A+ − A+B C −1 + B ∗A+B B ∗A+

m×m

if and only if

AA+B = B.

Proof: See [1076]. Remark: AA+B = B is equivalent to R(B) ⊆ R(A). Remark: Extensions of the matrix inversion lemma are considered in [392, 500, 1031, 1154] and [671, pp. 426–428, 447, 448]. Fact 6.4.6. Let A ∈ Fn×m and B ∈ Fm×l. Then, AB = 0 if and only if B A = 0. + +

Proof: The result follows from ix) =⇒ i) of Fact 6.4.16. Fact 6.4.7. Let A ∈ Fn×m and B ∈ Fn×l. Then, A+B = 0 if and only if A B = 0. ∗

Proof: The result follows from Proposition 6.1.6. Fact 6.4.8. Let A ∈ Fn×m, let B ∈ Fm×p, and assume that rank B = m. Then, AB(AB)+ = AA+. Proof: See [1208, p. 215].

414

CHAPTER 6

Fact 6.4.9. Let A ∈ Fn×m, let B ∈ Fm×m, and assume that B is positive definite. Then, ABA∗ (ABA∗ )+A = A. Proof: See [1208, p. 215]. Fact 6.4.10. Let A ∈ Fn×m, assume that rank A = m, let B ∈ Fn×n, and assume that B is positive definite. Then, (ABA∗ )+ = A(A∗A)−1B −1 (A∗A)−1A∗. Proof: Use Fact 6.3.17. Fact 6.4.11. Let A ∈ Fn×m, let S ∈ Fm×m, assume that S is nonsingular,  and define B = AS. Then, BB + = AA+. Proof: See [1215, p. 144]. Fact 6.4.12. Let A ∈ Fn×r and B ∈ Fr×m, and assume that rank A = rank B = r. Then, (AB)+ = B +A+ = B ∗(BB ∗ )−1(A∗A)−1A∗. Remark: AB is a full-rank factorization. Remark: See Fact 6.4.13.  rank A, let B ∈ Fn×r and C ∈ Fr×n, and Fact 6.4.13. Let A ∈ Fn×n, let r = assume that A = BC and rank B = rank C = r. Then, the following statements are equivalent:

i) A is range Hermitian. ii) BB + = C +C. iii) N(B ∗ ) = N(C). iv) B = C +CB and C = CBB +. v) B + = B +C +C and C = CBB +. vi) B = C +CB and C + = BB +C +. vii) B + = B +C +C and C + = BB +C +. Proof: See [448]. Remark: See Fact 3.6.3, Fact 6.3.9, Fact 6.3.10, and Fact 6.4.12. Fact 6.4.14. Let A ∈ Fn×m and B ∈ Fm×l. Then,  + (AB)+ = (A+AB)+ ABB + . If, in addition, R(B) = R(A∗ ), then A+AB = B, ABB + = A, and (AB)+ = B +A+. Proof: See [1208, pp. 192] or [1333].

415

GENERALIZED INVERSES

Remark: This result is due to Cline and Greville. 

Fact 6.4.15. Let A ∈ Fn×m and B ∈ Fm×l, and define A1 = AB1 B1+ and B1 = A+AB. Then, AB = A1B1 

and

(AB)+ = B1+A+ 1.

Proof: See [1208, pp. 191, 192]. Fact 6.4.16. Let A ∈ Fn×m and B ∈ Fm×l. Then, the following statements are equivalent: i) (AB)+ = B +A+. ii) R(A∗AB) ⊆ R(B) and R(BB ∗A∗ ) ⊆ R(A∗ ). iii) (AB)(AB)+ = (AB)B +A+ and (AB)+(AB) = B +A+AB. iv) A∗AB = BB +A∗AB and ABB ∗ = ABB ∗A+A. v) AB(AB)+A = ABB + and A+AB = B(AB)+AB. vi) A∗ABB + and A+ABB ∗ are Hermitian. vii) (ABB + )+ = BB +A+ and (A+AB)+ = B +A+A. viii) B +(ABB + )+ = B +A+ and (A+AB)+A = B +A+. ix) A∗ABB ∗ = BB +A∗ABB ∗A+A. Proof: See [16, p. 53], [1208, pp. 190, 191], and [601, 1323]. Remark: The equivalence of i) and ii) is due to Greville. Remark: Conditions under which B +A+ is a (1)-inverse of AB are given in [1323]. Remark: See [1450]. Fact 6.4.17. Let A ∈ Fn×m and B ∈ Fm×l. Then, the following statements are equivalent: i) (AB)+ = B +A+ − B + [(I − BB + )(I − A+A)]+A+. ii) R(AA∗AB) = R(AB) and R[(ABB ∗B)∗ ] = R[(AB)∗ ]. Proof: See [1321]. Fact 6.4.18. Let A, B ∈ Fn×n, and assume that A and B are projectors. Then,

R([A, B]) = R (A − B)+ − (A − B) . Consequently, (A − B)+ = (A − B) if and only if AB = BA. Proof: See [1320].

416

CHAPTER 6

Fact 6.4.19. Let A, B ∈ Fn×n, and assume that A and B are projectors. Then, the following statements hold: i) (AB)+ = B(AB)+. ii) (AB)+ = (AB)+A. iii) (AB)+ = B(AB)+A. iv) (AB)+ = BA − B(B⊥A⊥ )+A. v) (AB)+, B(AB)+, (AB)+A, B(AB)+A, and BA − B(B⊥A⊥ )+A are idempotent. vi) AB = A(AB)+B. vii) (AB)2 = AB + AB(B⊥A⊥ )+AB. Proof: To prove i) note that R[(AB)+ ] = R[(AB)∗ ] = R(BA), and thus R[B(AB)+ ] = R[B(AB)∗ ] = R(BA). Hence, R[(AB)+ ] = R[B(AB)+ ]. It now follows from Fact 3.13.14 that (AB)+ = B(AB)+. Statement iv) follows from Fact 6.4.17. Statements v) and vi) follow from iii). Statement vii) follows from iv) and vi). Remark: The converse of the first result in v) is given by Fact 6.4.20. Remark: See Fact 6.3.26, Fact 6.4.14, and Fact 6.4.24. See [1321, 1457]. Fact 6.4.20. Let A ∈ Fn×n, and assume that A is idempotent. Then, there exist projectors B, C ∈ Fn×n such that A = (BC)+. Proof: See [330, 551]. Remark: The converse of this result is given by v) of Fact 6.4.19. Remark: This result is due to Penrose. Fact 6.4.21. Let A, B ∈ Fn×n, and assume that R(A) and R(B) are com  plementary subspaces. Furthermore, define P = AA+ and Q = BB +. Then, the + matrix (Q⊥P ) is the idempotent matrix onto R(B) along R(A). Proof: See [602]. Remark: See Fact 3.12.33, Fact 3.13.24, and Fact 6.4.22. Fact 6.4.22. Let A, B ∈ Fn×n, assume that A and B are projectors, and assume that R(A) and R(B) are complementary subspaces. Then, (A⊥B)+ is the idempotent matrix onto R(B) along R(A). Proof: See Fact 6.4.21, [607], or [765]. Remark: It follows from Fact 6.4.19 that (A⊥B)+ is idempotent. Remark: See Fact 3.12.33, Fact 3.13.24, and Fact 6.4.21. Fact 6.4.23. Let A, B ∈ Fn×n, assume that A and B are projectors, and assume that A − B is nonsingular. Then, I − BA is nonsingular, and (A⊥B)+ = (I − BA)−1B(I − BA).

417

GENERALIZED INVERSES

Proof: Combine Fact 3.13.24 and Fact 6.4.22. Fact 6.4.24. Let k ≥ 1, let A1, . . . , Ak ∈ Fn×n, assume that A1, . . . , Ak are projectors, and define B1, . . . , Bk−1 ∈ Fn×n by  Bi = (A1 · · · Ak−i+1 )+A1 · · · Ak−i ,

and

i = 1, . . . , k − 2,



Bk−1 = A2 · · · Ak (A1 · · · Ak )+.

Then, B1, . . . , Bk−1 are idempotent, and (A1 · · · Ak )+ = B1 · · · Bk−1 . Proof: See [1330]. Remark: When k = 2, the result that B1 is idempotent is given by vi) of Fact 6.4.19. Fact 6.4.25. Let A ∈ Fn×n and B ∈ Fm×n, and assume that A is idempotent. Then, A∗ (BA)+ = (BA)+. Proof: See [671, p. 514]. Fact 6.4.26. Let A, B ∈ Fn×n, and assume that A and B are projectors. Then, the following statements are equivalent: i) AB is a projector. ii) [(AB)+ ]2 = [(AB)2 ]+. Proof: See [1353]. Remark: See Fact 3.13.20 and Fact 5.12.16. Fact 6.4.27. Let A ∈ Fn×m. Then, B ∈ Fm×n satisfies BAB = B if and only if there exist projectors C ∈ Fn×n and D ∈ Fm×m such that B = (CAD)+. Proof: See [602]. Fact 6.4.28. Let A ∈ Fn×n. Then, A is idempotent if and only if there exist projectors B, C ∈ Fn×n such that A = (BC)+. Proof: Let A = I in Fact 6.4.27. Remark: See [608]. Fact 6.4.29. Let A, B ∈ Fn×n, and assume that A is range Hermitian. Then, AB = BA if and only if A+B = BA+. Proof: See [1312].

418

CHAPTER 6

Fact 6.4.30. Let A, B ∈ Fn×n, and assume that A and B are range Hermitian. Then, the following statements are equivalent: i) AB = BA. ii) A+B = BA+. iii) AB + = B +A. iv) A+B + = B +A+. Proof: See [1312]. Fact 6.4.31. Let A, B ∈ Fn×n, assume that A and B are range Hermitian, and assume that (AB)+ = A+B +. Then, AB is range Hermitian. Proof: See [665]. Remark: See Fact 8.21.21. Fact 6.4.32. Let A, B ∈ Fn×n, and assume that A and B are range Hermitian. Then, the following statements are equivalent: i) AB is range Hermitian. ii) AB(I − A+A) = 0 and (I − B +B)AB = 0. iii) N(A) ⊆ N(AB) and R(AB) ⊆ R(B). iv) N(AB) = N(A) + N(B) and R(AB) = R(A) ∩ R(B). Proof: See [665, 856]. Fact 6.4.33. Let A ∈ Fn×m and B ∈ Fm×l, and assume that rank B = m. Then, AB(AB)+ = AA+. Fact 6.4.34. Let A ∈ Fn×m, B ∈ Fm×n, and C ∈ Fm×n, and assume that BAA∗ = A∗ and A∗AC = A∗. Then, A+ = BAC. Proof: See [16, p. 36]. Remark: This result is due to Decell. Fact 6.4.35. Let A, B ∈ Fn×n, and assume that A + B is nonsingular. Then, the following statements are equivalent: i) rank A + rank B = n. ii) A(A + B)−1B = 0. iii) B(A + B)−1A = 0. iv) A(A + B)−1A = A. v) B(A + B)−1B = B. vi) A(A + B)−1B + B(A + B)−1A = 0.

419

GENERALIZED INVERSES

vii) A(A + B)−1A + B(A + B)−1B = A + B. viii) (A + B)−1 = [(I − BB + )A(I − B +B)]+ + [(I − AA+ )B(I − A+A)]+. Proof: See [1334]. Remark: See Fact 2.11.4 and Fact 8.21.23. Fact 6.4.36. Let A, B ∈ Fn×n, and assume that A and B are projectors. Then, the following statements hold: i) A(A − B)+B = B(A − B)+A = 0. ii) A − B = A(A − B)+A − B(B − A)+B. iii) (A − B)+ = (A − AB)+ + (AB − B)+. iv) (A − B)+ = (A − BA)+ + (BA − B)+. v) (A − B)+ = A − B + B(A − BA)+ − (B − BA)+A. vi) (A − B)+ = A − B + (A − AB)+B − A(B − AB)+. vii) (I − A − B)+ = (A⊥B⊥ )+ − (AB)+. viii) (I − A − B)+ = (B⊥A⊥ )+ − (BA)+. Furthermore, the following statements are equivalent: ix) AB = BA. x) (A − B)+ = A − B. xi) B(A − BA)+ = (B − BA)+A. xii) (A − B)3 = A − B. xiii) A − B is tripotent. Proof: See [330]. Remark: See Fact 3.12.22. Fact 6.4.37. Let A ∈ Fn×m, and let B ∈ Fn×p. Then, (AA∗ + BB ∗ )+ = (I − C +∗B ∗ )A+∗EA+ (I − BC + ) + (CC ∗ )+, where



C = (I − AA+ )B

and E = I − A∗B(I − C +C)[I + (I − C +C)B ∗(AA∗ )+B(I − C +C)]−1 (A+B)∗. 

Proof: See [1208, p. 196]. Fact 6.4.38. Let A, B ∈ Fn×m, and assume that A∗B = 0. Then, (A + B)+ = A+ + (I − A+B)(C + + D), where

 (I − AA+ )B C=

420

CHAPTER 6

and 

D = (I − C +C)[I + (I − C +C)B ∗(AA∗ )+B(I − C +C)]−1B ∗(AA∗ )+(I − BC + ). Proof: See [1208, p. 196]. Remark: See Fact 6.5.18, from which it follows that BA∗ = 0 implies that (I − A+B)(C + + D) = B + . Fact 6.4.39. Let A, B ∈ Fn×m, and assume that A∗B = 0 and BA∗ = 0. Then, (A + B)+ = A+ + B +. Proof: Use Fact 2.10.29 and Fact 6.4.40. [1208, p. 197].

See [347], [671, p.

513], or

Remark: This result is due to Penrose. Fact 6.4.40. Let A, B ∈ Fn×m, and assume that rank(A + B) = rank A + rank B. Then, (A + B)+ = (I − C +B)A+(I − BC + ) + C +,  where C = (I − AA+ )B(I − A+A).

Proof: See [347]. Fact 6.4.41. Let A, B ∈ Fn×m. Then, (A + B)+ = (I + A+B)+ (A+ + A+BA+ )(I + BA+ )+ if and only if AA+B = B = BA+A. Furthermore, if n = m and A is nonsingular, then  +  + (A + B)+ = I + A−1B A−1 + A−1BA−1 I + BA−1 . Proof: See [347]. Remark: If A and A+B are nonsingular, then the last statement yields (A+B)−1 = (A + B)−1(A + B)(A + B)−1 for which the assumption that A is nonsingular is superfluous. Fact 6.4.42. Let A, B ∈ Fn×m. Then, A+ − B + = B + (B − A)A+ + (I − B +B)(A∗ − B ∗ )A+∗A+ + B +B +∗ (A∗ − B ∗ )(I − AA+ ) = A+ (B − A)B + + (I − A+A)(A∗ − B ∗ )B +∗B + + A+A+∗ (A∗ − B ∗ )(I − BB+ ). Furthermore, if B is left invertible, then A+ − B + = B + (B − A)A+ + B +B +∗ (A∗ − B ∗ )(I − AA+ ), while, if B is right invertible, then A+ − B + = A+ (B − A)B + + (I − A+A)(A∗ − B ∗ )B +∗B +. Proof: See [291, p. 224].

421

GENERALIZED INVERSES

Fact 6.4.43. Let A ∈ Fn×m, B ∈ Fl×k, and C ∈ Fn×k. Then, there exists a matrix X ∈ Fm×l satisfying AXB = C if and only if AA+CB +B = C. Furthermore, X satisfies AXB = C if and only if there exists a matrix Y ∈ Fm×l such that X = A+CB + + Y − A+AYBB +. Finally, if Y = 0, then tr X ∗X is minimized. Proof: Use Proposition 6.1.7. See [973, p. 37] and, for Hermitian solutions, see [831]. Fact 6.4.44. Let A ∈ Fn×m, and assume that rank A = m. Then, AL ∈ Fm×n is a left inverse of A if and only if there exists a matrix B ∈ Fm×n such that AL = A+ + B(I − AA+ ). Proof: Use Fact 6.4.43 with A = C = In . Fact 6.4.45. Let A ∈ Fn×m, and assume that rank A = n. Then, AR ∈ Fm×n is a right inverse of A if and only if there exists a matrix B ∈ Fm×n such that AR = A+ + (I − A+A)B. Proof: Use Fact 6.4.43 with B = C = In . Fact 6.4.46. Let A, B ∈ Fn×n, and assume that A and B are projectors. Then, glb{A, B} = lim A(BA)k = 2A(A + B)+B. k→∞

Furthermore, 2A(A + B) B is the projector onto R(A) ∩ R(B). +

Proof: See [41] and [642, pp. 64, 65, 121, 122]. Remark: See Fact 6.4.47 and Fact 8.21.18. Fact 6.4.47. Let A ∈ Rn×m and B ∈ Rn×l. Then, R(A) ∩ R(B) = R[AA+(AA+ + BB + )+BB + ]. Remark: See Theorem 2.3.1 and Fact 8.21.18. Fact 6.4.48. Let A ∈ Rn×m and B ∈ Rn×l. Then, R(A) ⊆ R(B) if and only if BB A = A. +

Proof: See [16, p. 35]. Fact 6.4.49. Let A ∈ Rn×m and B ∈ Rn×l. Then, dim[R(A) ∩ R(B)] = rank AA+(AA+ + BB + )+BB +

= rank A + rank B − rank A B . Proof: Use Fact 2.11.1, Fact 2.11.12, and Fact 6.4.47. Remark: See Fact 2.11.8.

422

CHAPTER 6

Fact 6.4.50. Let A, B ∈ Fn×n, and assume that A and B are projectors. Then, lub{A, B} = (A + B)(A + B)+. Furthermore, lub{A, B} is the projector onto R(A) + R(B) = span[R(A) ∪ R(B)]. Proof: Use Fact 2.9.13 and Fact 8.7.5. Remark: See Fact 8.7.2. Fact 6.4.51. Let A, B ∈ Fn×n, and assume that A and B are projectors. Then, lub{A, B} = I − lim A⊥(B⊥A⊥ )k = I − 2A⊥(A⊥ + B⊥ )+B⊥. k→∞

Furthermore, I − 2A⊥(A⊥ + B⊥ )+B⊥ is the projector onto [R(A⊥ ) ∩ R(B⊥ )]⊥ = [N(A) ∩ N(B)]⊥ = [N(A)]⊥ + [N(B)]⊥ = R(A) + R(B) = span[R(A) ∪ R(B)]. Consequently,

I − 2A⊥(A⊥ + B⊥ )+B⊥ = (A + B)(A + B)+.

Proof: See [41] and [642, pp. 64, 65, 121, 122]. Remark: See Fact 6.4.47 and Fact 8.21.18. Fact 6.4.52. Let A, B ∈ Fn×m. Then, ∗

A≤B if and only if

A+A = A+B

and

AA+ = BA+.

Proof: See [669]. Remark: See Fact 2.10.35.

6.5 Facts on the Moore-Penrose Generalized Inverse for Partitioned Matrices Fact 6.5.1. Let A, B ∈ Fn×m. Then, (A + B)+ =

1 2



Im

Im

Proof: See [1310, 1314, 1334]. Remark: See Fact 2.17.5 and Fact 2.19.7.



A B

B A

+

In In

 .

423

GENERALIZED INVERSES

Fact 6.5.2. Let A1, . . . , Ak ∈ Fn×m. Then, ⎡ A1 A2 ⎢

⎢ Ak A1 (A1 + · · · + Ak )+ = k1 Im · · · Im ⎢ .. ⎢ .. ⎣ . . A2 A3

···

Ak

··· .. .

Ak−1 .. .

···

A1

⎤+ ⎡ ⎥ ⎥⎢ ⎥⎣ ⎥ ⎦

⎤ In .. ⎥ . ⎦. In

Proof: See [1314]. Remark: The partitioned matrix is block circulant. See Fact 2.17.6 and Fact 6.6.1. Fact 6.5.3. Let A, B ∈ Fn×m. Then, the following statements are equivalent:

 

  i) R AA∗A = R BB∗B . 

 

 ii) R AA+A = R BB+B . iii) A = B. Remark: This result is due to Tian. Fact 6.5.4. Let A ∈ Fn×m, B ∈ Fn×l, C ∈ Fk×m, and D ∈ Fk×l. Then,       A B − AA+B I 0 I A+B A B . = CA+ I 0 I C D C − CA+A D − CA+B Remark: See Fact 6.5.25. Remark: See [1322]. 

Fact 6.5.5. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, define A = and assume that B = AA+B. Then,



A B B∗ C

In A = In A + In(A|A). Remark: This result is the Haynsworth inertia additivity formula. See [1130]. Remark: If A is positive semidefinite, then B = AA+B. See Proposition 8.2.4. Fact 6.5.6. Let A ∈ Fn×m, B ∈ Fn×l, C ∈ Fk×m, and D ∈ Fk×l. Then, 

 rank A B = rank A + rank B − AA+B   = rank B + rank A − BB +A = rank A + rank B − dim[R(A) ∩ R(B)],  rank

A C



  = rank A + rank C − CA+A   = rank C + rank A − AC +C = rank A + rank C − dim[R(A∗ ) ∩ R(C ∗ )],

 rank

A C

B 0



    = rank B + rank C + rank In − BB + A Im − C +C ,

,

424

CHAPTER 6

and

 rank

A C

B D

 = rank A + rank X + rank Y     + rank Ik − Y Y + (D − CA+B) Il − X +X ,

  B − AA+B and Y = C − CA+A. Consequently, where X =   A B , rank A + rank(D − CA+B) ≤ rank C D

and, if AA+B = B and CA+A = C, then



rank A + rank(D − CA B) = rank +

Finally, if n = m and A is nonsingular, then   n + rank D − CA−1B = rank



A C

A C

B D

B D

 .

 .

Proof: See [298, 993], Fact 2.11.8, and Fact 2.11.11. Remark: With certain restrictions the generalized inverses can be replaced by (1)inverses. Remark: See Proposition 2.8.3 and Proposition 8.2.3. Fact 6.5.7. Let A ∈ Fn×m, B ∈ Fk×l, and C ∈ Fn×l. Then,   A C − rank A − rank B. rank(AX + Y B + C) = rank min 0 −B X∈Fm×l,Y ∈Fn×k Furthermore, X, Y is a minimizing solution if and only if there exist U ∈ Fm×k, U1 ∈ Fm×l, and U2 ∈ Fn×k, such that X = −A+C + UB + (Im − A+A)U1, Y = (AA+ − I)CB + − AU + U2 (Ik − BB + ). Finally, all such matrices X ∈ Fm×l and Y ∈ Fn×k satisfy AX + Y B + C = 0 

if and only if rank

A C 0 −B

 = rank A + rank B.

Proof: See [1317, 1335]. Remark: See Fact 5.10.20. Note that A and B are square in Fact 5.10.20.

Fact 6.5.8. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, and assume that

is a projector. Then,

A B B∗ C

rank(D − B ∗A+B) = rank C − rank B ∗A+B. Proof: See [1327].

425

GENERALIZED INVERSES

Remark: See [110]. Fact 6.5.9. Let A ∈ Fn×m and B ∈ Fn×l. Then, the following statements are equivalent:

i) rank A B = rank A + rank B. ii) R(A) ∩ R(B) = {0}. iii) rank(AA∗ + BB ∗ ) = rank A + rank B. iv) A∗(AA∗ + BB ∗ )+A is idempotent. v) A∗(AA∗ + BB ∗ )+A = A+A. vi) A∗(AA∗ + BB ∗ )+B = 0. Proof: See [973, pp. 56, 57]. Remark: See Fact 2.11.8. 

Fact 6.5.10. Let A ∈ Fn×m and B ∈ Fn×l, and define the projectors P = AA+  and Q = BB +. Then, the following statements are equivalent:

i) rank A B = rank A + rank B = n. ii) P − Q is nonsingular. In this case, (P − Q)−1 = (P − P Q)+ + (P Q − Q)+ = (P − QP )+ + (QP − Q)+ = P − Q + Q(P − QP )+ − (Q − QP )+P. Proof: See [330]. Fact 6.5.11. Let A ∈ Fn×m, B ∈ Fn×l, C ∈ Fl×n, D ∈ Fl×l, and assume that D is nonsingular. Then,   rank A = rank A − BD−1C + rank BD−1C if and only if there exist matrices X ∈ Fm×l and Y ∈ Fl×n such that B = AX, C = YA, and D = YAX. Proof: See [338]. Fact 6.5.12. Let A ∈ Fn×m, B ∈ Fn×l, C ∈ Fk×m, and D ∈ Fk×l. Then,   ∗ ∗ A AA A∗B + . rank A + rank(D − CA B) = rank CA∗ D Proof: See [1318].

426

CHAPTER 6

Fact 6.5.13. Let A11 ∈ Fn×m, A12 ∈ Fn×l, A21 ∈ Fk×m, and A22 ∈ Fk×l, A11 A12 B B   and define A = , where ∈ F(n+k)×(m+l) and B = AA+ = BT11 B12 A21 A22 22 12

B11 ∈ Fn×m, B12 ∈ Fn×l, B21 ∈ Fk×m, and B22 ∈ Fk×l. Then,



rank B12 = rank A11 A12 + rank A21 A22 − rank A. Proof: See [1340]. Remark: See Fact 3.12.20 and Fact 3.13.12. Fact 6.5.14. Let A, B ∈ Fn×n. Then,  

0 A = rank A + rank B I − A+A rank B I   A + rank B = rank I − BB +    = rank A + rank B + rank I − BB + I − A+A = n + rank AB. Hence, the following statements hold: i) rank AB = rank A + rank B − n if and only if (I − BB + )(I − A+A) = 0.

ii) rank AB = rank A if and only if B I − A+A is right invertible.   A iii) rank AB = rank B if and only if I−BB is left invertible. + Proof: See [993]. Remark: The generalized inverses can be replaced by arbitrary (1)-inverses. Fact 6.5.15. Let A ∈ Fn×m, B ∈ Fm×l, and C ∈ Fl×k. Then,   0 AB = rank B + rank ABC rank BC B = rank AB + rank BC + rank [(I − BC)(BC)+ ]B[(I − (AB)+ (AB)]. Furthermore, the following statements are equivalent: 0 i) rank [ BC

AB ] B

= rank AB + rank BC.

ii) rank ABC = rank AB + rank BC − rank B. iii) There exist matrices X ∈ Fk×l and Y ∈ Fm×n such that BCX + YAB = B. Proof: See [993, 1340] and Fact 5.10.20. Remark: This result is related to the Frobenius inequality. See Fact 2.11.14.

427

GENERALIZED INVERSES

Fact 6.5.16. Let x, y ∈ R3, and assume that x and y are linearly independent. Then,   x+(I3 − yφT )

+ x y , = φT 



where x+ = (xTx)−1xT , α = y T(I − xx+ )y, and φ = α−1 (I − xx+ )y. Now, let x, y, z ∈ R3, and assume that x and y are linearly independent. Then, ⎡

+ ⎤ (I2 − βwwT ) x y

+ ⎦, x y z =⎣

T x y + βw   [x y]+z and β = 1/(1 + wTw). where w =

Proof: See [1351]. Fact 6.5.17. Let A ∈ Fn×m and b ∈ Fn. Then,   A+(In − bφ∗ )

+ A b = φ∗ and

b

where  φ=

A

+

 =

φ∗ A+(In − bφ∗ )

 ,

⎧ ⎨(b − AA+b)+∗, b = AA+ b, ⎩

γ −1(AA∗ )+ b,

b = AA+ b.



and γ = 1 + b∗(AA∗ )+b. Proof: See [16, p. 44], [494, p. 270], or [1217, p. 148]. Remark: This result is due to Greville. Fact 6.5.18. Let A ∈ Fn×m and B ∈ Fn×l. Then,   A+ − A+B(C + + D)

+ A B = , C+ + D where



C = (I − AA+ )B

and D = (I − C +C)[I + (I − C +C)B ∗(AA∗ )+B(I − C +C)]−1B ∗(AA∗ )+(I − BC + ). 

428

CHAPTER 6

Furthermore,

 ⎧ ∗   A (AA∗ + BB ∗ )−1 ⎪ ⎪ ⎪ , rank = n, A B ⎪ ⎪ ⎪ B ∗(AA∗ + BB ∗ )−1 ⎪ ⎪ ⎪ ⎪  −1 ⎪  

+ ⎨ A∗A A∗B A∗ A B = , rank = m + l, A B ⎪ B ∗A B ∗B B∗ ⎪ ⎪ ⎪ ⎪  ⎪ ⎪ ⎪ ∗ ∗ −1 ⎪ ⎪ A (AA ) (I − BE) , rank A = n, ⎪ ⎩ E

where



−1  E = I + B ∗(AA∗ )−1B B ∗(AA∗ )−1.

Proof: See [346], [972, p. 14], or [1208, pp. 193–195].

Remark: If A B is square and nonsingular and A∗B = 0, then the second expression yields Fact 2.17.8. Remark: See Fact 6.4.38. Fact 6.5.19. Let A ∈ Fn×m and B ∈ Fn×l. Then,  +  

+ A A B = rank AA∗B − rank + B ∗  +  Hence, if A B = 0, then

+ A A B . = B+

BB ∗A .

Proof: See [1321]. Fact 6.5.20. Let A ∈ Fn×m and B ∈ Fn×l. Then, the following statements are equivalent:



+ 1 A B i) A B = 2 (AA+ + BB + ). ii) R(A) = R(B). Furthermore, the following statements are equivalent:  + 

+ A . = 12 iii) A B B+ iv) AA∗ = BB ∗. Proof: See [1332]. Fact 6.5.21. Let A ∈ Fn×m and B ∈ Fk×l. Then, +  +   A 0 0 A . = 0 B 0 B+ Fact 6.5.22. Let A ∈ Fn×m. Then, +   (In + AA∗ )−1 A In = 0m×n 0m×m A∗(In + AA∗ )−1

0n×m 0m×m

 .

429

GENERALIZED INVERSES

Proof: See [19, 1358]. Fact 6.5.23. Let A ∈ Fn×n, let B ∈ Fn×m, and assume that BB ∗ = I. Then, +    A B 0 B . = B∗ 0 B ∗ −B ∗AB Proof: See [459, p. 237]. Fact 6.5.24. Let A ∈ Fn×n, assume that A is positive semidefinite, and let B ∈ Fn×m. Then,  +  C + − C +BD+B ∗C +  C +BD+ A B , = B∗ 0 (C +BD+ )∗ DD+ − D+ where





C = A + BB ∗,

D = B ∗C +B.

Proof: See [973, p. 58]. Remark: Representations for the generalized inverse of a partitioned matrix are given in [178, Chapter 5] and [108, 115, 138, 176, 284, 291, 304, 609, 660, 662, 757, 930, 1021, 1022, 1024, 1025, 1026, 1073, 1147, 1165, 1310, 1342, 1308, 1452]. Problem: Show that the generalized inverses in this result and in Fact 6.5.23 are identical when A is positive semidefinite and BB ∗ = I. Fact 6.5.25. Let A ∈ Fn×n, x, y ∈ Fn, and a ∈ F, and assume that x ∈ R(A). Then,       A 0 A x I 0 I A+x . = yT a yT 1 0 1 yT − yTA a − yTA+x Remark: This factorization holds for the case in which A is singular and a = 0. See Fact 2.14.9, Fact 2.16.2, and Fact 6.5.4, and note that x = AA+x. Problem: Obtain a factorization for the case in which x ∈ / R(A) (and thus x is nonzero and A is singular) and a = 0. Fact 6.5.26. Let A ∈ Fn×m, assume that A is partitioned as ⎡ ⎤ A1 ⎢ ⎥ A = ⎣ ... ⎦, Ak and define

 B=



A+ 1

···

. A+ k

Then, the following statements hold: i) det AB = 0 if and only if rank A < n. ii) 0 < det AB ≤ 1 if and only if rank A = n. iii) If rank A = n, then det AB = 6k

det AA∗

i=1

det Ai A∗i

,

430

CHAPTER 6

and thus ∗

det AA ≤

k !

det Ai A∗i .

i=1

iv) det AB = 1 if and only if AB = I. v) AB is group invertible. vi) Every eigenvalue of AB is nonnegative. vii) rank A = rank B = rank AB = rank BA. k Now, assume in addition that rank A = i=1 rank Ai , and let β denote the product of the positive eigenvalues of AB. Then, the following statements hold: viii) 0 < β ≤ 1. ix) β = 1 if and only if B = A+. Proof: See [900, 1278]. Remark: Result iii) yields Hadamard’s inequality given by Fact 8.13.35 in the case in which A is square and each Ai has a single row. Fact 6.5.27. Let A ∈ Fn×m and B ∈ Fn×l. Then,   ∗ A A B ∗A = det(A∗A)det[B ∗(I − AA+ )B] det B ∗A B ∗B = det(B ∗B)det[A∗(I − BB + )A]. Remark: See Fact 2.14.25. n×n , B ∈ Fn×m, C ∈ Fm×n, and D ∈ Fm×m, assume Fact 6.5.28. Let A ∈ F A ] = rank A, and let A− ∈ Fn×n be that either rank A B = rank A or rank [ C a (1)-inverse of A. Then,   A B = (det A)det(D − CA−B). det C D

Proof: See [148, p. 266].  Fact 6.5.29. Let A =

Fl×(n+m), D ∈ Fl×l, and A lar. Then,

A11

A12 ∈ F(n+m)×(n+m), B A21 A22  B = [A C D ], and assume that A and

∈ F(n+m)×l, C ∈ A11 are nonsingu-

A|A = (A11|A)|(A11|A).

Proof: See [1125, pp. 18, 19]. Remark: This result is the Crabtree-Haynsworth quotient formula. See [736]. Remark: Extensions are given in [1531]. Problem: Extend this result to the case in which either A or A11 is singular.

431

GENERALIZED INVERSES

Fact 6.5.30. Let A, B ∈ Fn×m. Then, the following statements are equivalent: rs

i) A ≤ B. ii) AA+B = BA+A = BA+B = B.

A ] and BA+B = B. iii) rank A = rank A B = rank [ B Proof: See [1215, p. 45]. Remark: See Fact 8.21.7.

6.6 Facts on the Drazin and Group Generalized Inverses Fact 6.6.1. Let A1, . . . , Ak ∈ Fn×m. Then, ⎡ A1 ⎢

⎢ Ak (A1 + · · · + Ak )D = k1 In · · · In ⎢ ⎢ .. ⎣ . A2

A2

···

Ak

A1 .. .

··· .. .

Ak−1 .. .

A3

···

A1

⎤D ⎡ ⎥ ⎥⎢ ⎥⎣ ⎥ ⎦

⎤ Im .. ⎥ . ⎦. Im

Proof: See [1314]. Remark: See Fact 6.5.2. Fact 6.6.2. Let A ∈ Fn×n. Then, X = AD is the unique matrix satisfying   A AAD = rank A. rank ADA X Proof: See [1451, 1532]. Remark: See Fact 2.17.10 and Fact 6.3.29. Fact 6.6.3. Let A, B ∈ Fn×n, and assume that AB = 0. Then, (AB)D = A(BA)2D B. Remark: This result is Cline’s formula. Fact 6.6.4. Let A, B ∈ Fn×n, and assume that AB = BA. Then, (AB)D = B DAD, ADB = BAD, AB D = B DA. Fact 6.6.5. Let A, B ∈ Fn×n, and assume that AB = BA = 0. Then, (A + B)D = AD + B D. Proof: See [670]. Remark: This result is due to Drazin.

432

CHAPTER 6

Fact 6.6.6. Let A, B ∈ Fn×n, and assume that A and B are idempotent. Then, the following statements hold: i) If AB = 0, then and

(A + B)D = A + B − 2BA (A − B)D = A − B.

ii) If BA = 0, then and

(A + B)D = A + B − 2AB (A − B)D = A − B.

iii) If AB = A, then and

(A + B)D = 14 A + B − 34 BA (A − B)D = BA − B.

iv) If AB = B, then and

(A + B)D = A + 14 B − 34 BA (A − B)D = A − BA.

v) If BA = A, then and

(A + B)D = 14 A + B − 34 AB (A − B)D = AB − B.

vi) If BA = B, then and

(A + B)D = A + 14 B − 34 AB (A − B)D = A − AB.

vii) If AB = BA, then and

(A + B)D = A + B − 32 AB (A − B)D = A − B.

viii) If ABA = 0, then (A + B)D = A + B − 2AB − 2BA + 3BAB and

(A − B)D = A − B − BAB.

ix) If BAB = 0, then (A + B)D = A + B − 2AB − 2BA + 3ABA and

(A − B)D = A − B + ABA.

433

GENERALIZED INVERSES

x) If ABA = A, then (A + B)D = 18 (A + B)2 + 78 BA⊥B and

(A − B)D = −BA⊥B.

xi) If BAB = B, then (A + B)D = 18 (A + B)2 + 78 AB⊥A and

(A − B)D = AB⊥A.

xii) If ABA = B, then

(A + B)D = A − 12 B

and

(A − B)D = A − B.

xiii) If BAB = A, then

(A + B)D = − 12 A + B

and

(A − B)D = A − B.

xiv) If ABA = AB, then (A + B)D = A + B − 2BA − 34 AB + 54 BAB and

(A − B)D = A − B − AB + BAB.

xv) If ABA = BA, then (A + B)D = A + B − 2AB − 34 BA + 54 BAB and

(A − B)D = A − B − BA + BAB.

Now, assume in addition that A and B are projectors. Then, the following statements hold: xvi) If AB = A, then and xvii) If AB = B, then and

(A + B)D = − 12 A + B (A − B)D = A − B. (A + B)D = A − 12 B (A − B)D = A − B.

xviii) AB = BA if and only if (A − B)D = A − B. Proof: See [405].

434

CHAPTER 6

Fact 6.6.7. Let A ∈ Fn×n, and assume that ind A = rank A = 1. Then, −1  A# = tr A2 A. Consequently, if x, y ∈ Fn satisfy x∗y = 0, then (xy ∗ )# = (x∗y)−2xy ∗. In particular,

−2 1# n×n = n 1n×n . 

Fact 6.6.8. Let A ∈ Fn×n, and let k = ind A. Then,  + AD = Ak A2k+1 Ak. If, in particular, ind A ≤ 1, then

 + A# = A A3 A.

Proof: See [178, pp. 165, 174]. Fact 6.6.9. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A is range Hermitian. ii) A+ = AD. iii) ind A ≤ 1, and A+ = A#. iv) ind A ≤ 1, and A∗A#A + AA#A∗ = 2A∗. v) ind A ≤ 1, and A+A#A + AA#A+ = 2A+. Proof: See [331]. Remark: See Fact 6.3.10. Fact 6.6.10. Let A ∈ Fn×n, assume that A is group invertible, and let S, B ∈ F , where S is nonsingular, B is a Jordan canonical form of A, and A = SBS −1. Then, A# = SB #S −1 = SB +S −1. n×n

Proof: Since B is range Hermitian, it follows from Fact 6.6.9 that B # = B +. See [178, p. 158]. Fact 6.6.11. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A is normal. ii) ind A ≤ 1, and A#A∗ = A∗A#. Proof: See [331]. Remark: See Fact 3.7.12, Fact 3.11.5, Fact 5.15.4, and Fact 6.3.15.

435

GENERALIZED INVERSES

Fact 6.6.12. Let A ∈ Fn×n, and let k ≥ 1. Then, the following statements are equivalent: i) k ≥ ind A. ii) limα→0 αk(A + αI)−1 exists. iii) limα→0 (Ak+1 + αI)−1Ak exists. In this case,

AD = lim (Ak+1 + αI)−1Ak α→0

and lim αk(A + αI)−1

α→0

⎧ k−1 D k−1 ⎪ ⎨(−1) (I − AA )A , k = ind A > 0, = A−1, k = ind A = 0, ⎪ ⎩ 0, k > ind A.

Proof: See [1024]. 

Fact 6.6.13. Let A ∈ Fn×n, let r = rank A, let B ∈ Rn×r and C ∈ Rr×n, and assume that A = BC. Then, A is group invertible if and only if BA is nonsingular. In this case, A# = B(CB)−2 C. Proof: See [178, p. 157]. Remark: This result is due to Cline. Fact 6.6.14. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m. If A and C are D B singular, then ind [ A 0 C ] = 1 if and only if ind A = ind C = 1, and (I − AA )B(I − D CC ) = 0. Proof: See [1024]. Remark: See Fact 5.14.31. Fact 6.6.15. Let A ∈ Fn×n. Then, A is group invertible if and only if limα→0 (A + αI)−1A exists. In this case, lim (A + αI)−1A = AA#.

α→0

Proof: See [291, p. 138]. Fact 6.6.16. Let A ∈ Fn×n, assume that A is nonzero and group invertible,  let r = rank A, define B = diag[σ1(A), . . . , σr (A)], and let S ∈ Fn×n, K ∈ Fr×r, r×(n−r) and L ∈ F be such that S is unitary, 

KK ∗ + LL∗ = Ir , and

 A=S

BK

BL

0(n−r)×r

0(n−r)×(n−r)

 S ∗.

436

CHAPTER 6

Then,

 #

A =S

K −1B −1

K −1B −1K −1L

0(n−r)×r

0(n−r)×(n−r)

 S ∗.

Proof: See [119, 668]. Remark: See Fact 5.9.30 and Fact 6.3.14. Fact 6.6.17. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A is range Hermitian. ii) A is group invertible and AA+A+ = A#. iii) A is group invertible and AA#A+ = A#. iv) A is group invertible and A∗AA# = A∗. v) A is group invertible and A+AA# = A+. vi) A is group invertible and A#A+A = A+. vii) A is group invertible and AA# = A+A. viii) A is group invertible and A∗A+ = A∗A#. ix) A is group invertible and A+A∗ = A#A∗. x) A is group invertible and A+A+ = A+A#. xi) A is group invertible and A+A+ = A#A+. xii) A is group invertible and A+A+ = A#A#. xiii) A is group invertible and A+A# = A#A#. xiv) A is group invertible and A#A+ = A#A#. xv) A is group invertible and A+A# = A#A+. xvi) A is group invertible and AA+A∗ = A∗AA+. xvii) A is group invertible and AA+A# = A+A#A. xviii) A is group invertible and AA+A# = A#AA+. xix) A is group invertible and AA#A∗ = A∗AA#. xx) A is group invertible and AA#A+ = A+AA#. xxi) A is group invertible and AA#A+ = A#A+A. xxii) A is group invertible and A∗A+A = A+AA∗. xxiii) A is group invertible and A+AA# = A#A+A. xxiv) A is group invertible and A+A+A# = A+A#A+. xxv) A is group invertible and A+A+A# = A#A+A+. xxvi) A is group invertible and A+A#A+ = A#A+A+. xxvii) A is group invertible and A+A#A# = A#A+A#.

GENERALIZED INVERSES

437

xxviii) A is group invertible and A+A#A# = A#A#A+. xxix) A is group invertible and A#A#A+ = A#A+A#. Proof: See [119]. Fact 6.6.18. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A is normal. ii) A is group invertible and A∗A+ = A#A∗. iii) A is group invertible and A∗A# = A+A∗. iv) A is group invertible and A∗A# = A#A∗. v) A is group invertible and AA∗A# = A∗A#A. vi) A is group invertible and AA∗A# = A#AA∗. vii) A is group invertible and AA#A∗ = A#A∗A. viii) A is group invertible and A∗AA# = A#A∗A. ix) A is group invertible and A∗2A# = A∗A#A∗. x) A is group invertible and A∗A+A# = A#A∗A+. xi) A is group invertible and A∗A#A∗ = A#A2∗. xii) A is group invertible and A∗A#A+ = A+A∗A#. xiii) A is group invertible and A∗A#A# = A#A∗A#. xiv) A is group invertible and A+A∗A# = A#A+A∗. xv) A is group invertible and A+A#A∗ = A#A∗A+. xvi) A is group invertible and A#A∗A# = A#A#A∗. Proof: See [119]. Fact 6.6.19. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A is Hermitian. ii) A is group invertible and AA# = A∗A+. iii) A is group invertible and AA# = A∗A#. iv) A is group invertible and AA# = A+A∗. v) A is group invertible and A+A = A#A∗. vi) A is group invertible and A∗AA# = A. vii) A is group invertible and A2∗A# = A∗. viii) A is group invertible and A∗A+A+ = A#. ix) A is group invertible and A∗A+A# = A+. x) A is group invertible and A∗A+A# = A#.

438

CHAPTER 6

xi) A is group invertible and A∗A#A# = A#. xii) A is group invertible and A#A∗A# = A+. Proof: See [119]. Fact 6.6.20. Let A, B ∈ Fn×n, assume that A and B are group invertible, and consider the following conditions: i) ABA = B. ii) BAB = A. iii) A2 = B 2. Then, if two of the above conditions are satisfied, then the third condition is satisfied. Furthermore, if i)–iii) are satisfied, then the following statements hold: iv) A and B are group invertible. v) A# = A3 and B # = B 3. vi) A5 = A and B 5 = B. vii) A4 = B 4 = (AB)4. viii) If A and B are nonsingular, then A4 = B 4 = (AB)4 = I. Proof: See [482]. Fact 6.6.21. Let A ∈ Rn×n, where n ≥ 2, assume that A is positive, define B = sprad(A)I − A, let x, y ∈ Rn be positive, and assume that Ax = sprad(A)x and ATy = sprad(A)y. Then, the following statements hold: 

i) B +

1 xy T xTy

ii) B # = (B + iii) I − BB # =

is nonsingular. 1 xy T )−1 (I xTy

−

1 xy T ). xTy

1 xy T. xTy

iv) B # = limk→∞

 k−1

1 i i=0 [sprad(A)]i A

−

k xTy

 xy T .

Proof: See [1177, p. 9-4]. Remark: See Fact 4.11.4.

6.7 Notes A brief history of the generalized inverse is given in [177] and [178, p. 4]. The proof of the uniqueness of A+ is given in [973, p. 32]. Additional books on generalized inverses include [178, 249, 1145, 1430]. The terminology “range Hermitian” is used in [178]; the terminology “EP” is more common. Generalized inverses are widely used in least squares methods; see [241, 291, 901]. Applications to singular differential equations are considered in [290]. Applications to Markov chains are discussed in [758].

Chapter Seven

Kronecker and Schur Algebra

In this chapter we introduce Kronecker matrix algebra, which is useful for solving linear matrix equations.

7.1 Kronecker Product For A ∈ Fn×m define the vec operator as ⎡ ⎤ col1(A) ⎥  ⎢ .. nm vec A = ⎣ ⎦∈F , .

(7.1.1)

colm(A) which is the column vector of size nm × 1 obtained by stacking the columns of A. We recover A from vec A by writing A = vec−1(vec A).

(7.1.2)

Proposition 7.1.1. Let A ∈ Fn×m and B ∈ Fm×n. Then, T T   tr AB = vec AT vec B = vec BT vec A.

(7.1.3)

Proof. Note that tr AB = =

n  i=1 n 

rowi(A)coli(B)

  T coli AT coli(B)

i=1

=



 T ··· colT 1 A

T  = vec AT vec B.

⎡

⎤ col1(B)  T  ⎢ ⎥ .. colT ⎣ ⎦ n A . coln(B)

Next, we introduce the Kronecker product.



440

CHAPTER 7

Definition 7.1.2. Let A ∈ Fn×m and B ∈ Fl×k. Then, the Kronecker product A ⊗ B ∈ Fnl×mk of A and B is the partitioned matrix ⎡ ⎤ A(1,1) B A(1,2) B · · · A(1,m) B ⎥  ⎢ .. .. . .. A⊗B = (7.1.4) ⎣ ⎦. · .· · . . . A(n,1) B

A(n,2) B

···

A(n,m) B

Unlike matrix multiplication, the Kronecker product A ⊗ B does not entail a restriction on either the size of A or the size of B. The following results are immediate consequences of the definition of the Kronecker product. Proposition 7.1.3. Let α ∈ F, A ∈ Fn×m, and B ∈ Fl×k. Then, A ⊗ (αB) = (αA) ⊗ B = α(A ⊗ B),

(7.1.5)

A ⊗ B = A ⊗ B,

(7.1.6)

(A ⊗ B)T = AT ⊗ BT,

(7.1.7)

(A ⊗ B)∗ = A∗ ⊗ B ∗.

(7.1.8)

Proposition 7.1.4. Let A, B ∈ Fn×m and C ∈ Fl×k. Then,

and

(A + B) ⊗ C = A ⊗ C + B ⊗ C

(7.1.9)

C ⊗ (A + B) = C ⊗ A + C ⊗ B.

(7.1.10)

The next result shows that the Kronecker product is associative. Proposition 7.1.5. Let A ∈ Fn×m, B ∈ Fl×k, and C ∈ Fp×q. Then, A ⊗ (B ⊗ C) = (A ⊗ B) ⊗ C.

(7.1.11)

Hence, we write A ⊗ B ⊗ C for A ⊗ (B ⊗ C) and (A ⊗ B) ⊗ C. The next result illustrates a useful form of compatibility between matrix multiplication and the Kronecker product. Proposition 7.1.6. Let A ∈ Fn×m, B ∈ Fl×k, C ∈ Fm×q, and D ∈ Fk×p. Then, (A ⊗ B)(C ⊗ D) = AC ⊗ BD. (7.1.12)

441

KRONECKER AND SCHUR ALGEBRA

Proof. Note that the ij block of (A ⊗ B)(C ⊗ D) is given by ⎡ ⎤ C(1,j) D

⎢ ⎥ .. [(A ⊗ B)(C ⊗ D)]ij = A(i,1) B · · · A(i,m) B ⎣ ⎦ . C(m,j) D m  = A(i,k) C(k,j) BD = (AC)(i,j) BD k=1



= (AC ⊗ BD)ij . Next, we consider the inverse of a Kronecker product.

Proposition 7.1.7. Assume that A ∈ Fn×n and B ∈ Fm×m are nonsingular. Then, A ⊗ B is nonsingular, and (A ⊗ B)−1 = A−1 ⊗ B −1.

(7.1.13)

Proof. Note that   (A ⊗ B) A−1 ⊗ B −1 = AA−1 ⊗ BB −1 = In ⊗ Im = Inm .



Proposition 7.1.8. Let x ∈ Fn and y ∈ Fm. Then, xyT = x ⊗ yT = yT ⊗ x

(7.1.14)

vec xyT = y ⊗ x.

(7.1.15)

and

The following result concerns the vec of the product of three matrices. Proposition 7.1.9. Let A ∈ Fn×m, B ∈ Fm×l, and C ∈ Fl×k. Then,   (7.1.16) vec(ABC) = CT ⊗ A vec B. Proof. Using (7.1.12) and (7.1.15), it follows that vec ABC = vec

l 

Acoli(B)eT iC =

i=1

=

l 



l 

 T   vec Acoli(B) CTei

i=1

l

  CTei ⊗ [Acoli(B)] = CT ⊗ A ei ⊗ coli(B)

i=1

i=1 l 

  = CT ⊗ A

 T  vec coli(B)eT i = C ⊗ A vec B.



i=1

The following result concerns the eigenvalues and eigenvectors of the Kronecker product of two matrices.

442

CHAPTER 7

Proposition 7.1.10. Let A ∈ Fn×n and B ∈ Fm×m. Then, mspec(A ⊗ B) = {λμ: λ ∈ mspec(A), μ ∈ mspec(B)}ms .

(7.1.17)

If, in addition, x ∈ C is an eigenvector of A associated with λ ∈ spec(A) and y ∈ Cm is an eigenvector of B associated with μ ∈ spec(B), then x ⊗ y is an eigenvector of A ⊗ B associated with λμ. n

Proof. Using (7.1.12), we have (A ⊗ B)(x ⊗ y) = (Ax) ⊗ (By) = (λx) ⊗ (μy) = λμ(x ⊗ y).



Proposition 7.1.10 shows that mspec(A ⊗ B) = mspec(B ⊗ A). Consequently, it follows that det(A ⊗ B) = det(B ⊗ A) and tr(A ⊗ B) = tr(B ⊗ A). The following results are generalizations of these equalities. Proposition 7.1.11. Let A ∈ Fn×n and B ∈ Fm×m. Then, det(A ⊗ B) = det(B ⊗ A) = (det A)m (det B)n.

(7.1.18)

Proof. Let mspec(A) = {λ1, . . . , λn}ms and mspec(B) = {μ1, . . . , μm}ms . Then, Proposition 7.1.10 implies that ⎛ ⎞ ⎛ ⎞ n,m m m ! ! ! det(A ⊗ B) = λi μj = ⎝λm μj⎠ · · · ⎝λm μj⎠ 1 n i,j=1

j=1

j=1

= (λ1 · · · λn ) (μ1 · · · μm ) = (det A)m(det B)n. m

n



Proposition 7.1.12. Let A ∈ Fn×n and B ∈ Fm×m. Then, tr(A ⊗ B) = tr(B ⊗ A) = (tr A)(tr B).

(7.1.19)

Proof. Note that tr(A ⊗ B) = tr(A(1,1) B) + · · · + tr(A(n,n) B) = [A(1,1) + · · · + A(n,n) ]tr B = (tr A)(tr B).



Next, define the Kronecker permutation matrix Pn,m ∈ Fnm×nm by 

Pn,m =

n,m 

Ei,j,n×m ⊗ Ej,i,m×n .

(7.1.20)

i,j=1

Proposition 7.1.13. Let A ∈ Fn×m. Then, vec AT = Pn,m vec A.

(7.1.21)

443

KRONECKER AND SCHUR ALGEBRA

7.2 Kronecker Sum and Linear Matrix Equations Next, we define the Kronecker sum of two square matrices. Definition 7.2.1. Let A ∈ Fn×n and B ∈ Fm×m. Then, the Kronecker sum A ⊕ B ∈ Fnm×nm of A and B is 

A ⊕ B = A ⊗ Im + In ⊗ B.

(7.2.1)

Proposition 7.2.2. Let A ∈ Fn×n, B ∈ Fm×m, and C ∈ Fl×l. Then, A ⊕ (B ⊕ C) = (A ⊕ B) ⊕ C.

(7.2.2)

Hence, we write A ⊕ B ⊕ C for A ⊕ (B ⊕ C) and (A ⊕ B) ⊕ C. Proposition 7.1.10 shows that, if λ ∈ spec(A) and μ ∈ spec(B), then λμ ∈ spec(A ⊗ B). Next, we present an analogous result involving Kronecker sums. Proposition 7.2.3. Let A ∈ Fn×n and B ∈ Fm×m. Then, mspec(A ⊕ B) = {λ + μ: λ ∈ mspec(A), μ ∈ mspec(B)}ms .

(7.2.3)

Now, let x ∈ Cn be an eigenvector of A associated with λ ∈ spec(A), and let y ∈ Cm be an eigenvector of B associated with μ ∈ spec(B). Then, x ⊗ y is an eigenvector of A ⊕ B associated with λ + μ. Proof. Using (7.1.12), we have (A ⊕ B)(x ⊗ y) = (A ⊗ Im )(x ⊗ y) + (In ⊗ B)(x ⊗ y) = (Ax ⊗ y) + (x ⊗ By) = (λx ⊗ y) + (x ⊗ μy) = λ(x ⊗ y) + μ(x ⊗ y) = (λ + μ)(x ⊗ y).



The next result concerns the existence and uniqueness of solutions to Sylvester’s equation. See Fact 5.10.21 and Proposition 11.9.3. F

n×m

Proposition 7.2.4. Let A ∈ Fn×n, B ∈ Fm×m, and C ∈ Fn×m. Then, X ∈ satisfies AX + XB + C = 0 (7.2.4)

if and only if X satisfies

  BT ⊕ A vec X + vec C = 0.

(7.2.5)

Consequently, BT ⊕ A is nonsingular if and only if there exists a unique matrix X ∈ Fn×m satisfying (7.2.4). In this case, X is given by   −1 −1 T B ⊕ A vec C . (7.2.6) X = − vec

Furthermore, BT ⊕ A is singular and rank BT ⊕ A = rank BT ⊕ A vec C if and only if there exist infinitely many matrices X ∈ Fn×m (7.5.8). In this  satisfying  case, the set of solutions of (7.2.4) is given by X + N BT ⊕ A .

444

CHAPTER 7

Proof. Note that (7.2.4) is equivalent to

  0 = vec(AXI + IXB) + vec C = (I ⊗ A)vec X + B T ⊗ I vec X + vec C     = B T ⊗ I + I ⊗ A vec X + vec C = B T ⊕ A vec X + vec C,

which yields (7.2.5). The remaining results follow from Corollary 2.6.7. For the following corollary, note Fact 5.10.21. Corollary 7.2.5. Let A ∈ Fn×n, B ∈ Fm×m, and C ∈ Fn×m, and assume that spec(A) and spec(−B) are disjoint. Then, there X ∈ Fn×m

a unique A C matrix

A exists 0 satisfying (7.2.4). Furthermore, the matrices 0 −B and 0 −B are similar and satisfy       I −X A 0 I X A C . (7.2.7) = 0 I 0 −B 0 I 0 −B

7.3 Schur Product An alternative form of vector and matrix multiplication is given by the Schur product. If A ∈ Fn×m and B ∈ Fn×m, then A ◦ B ∈ Fn×m is defined by 

(A ◦ B)(i,j) = A(i,j) B(i,j) ,

(7.3.1)

that is, A ◦ B is formed by means of entry-by-entry multiplication. For matrices A, B, C ∈ Fn×m, the commutative, associative, and distributive equalities A ◦ B = B ◦ A,

(7.3.2)

A ◦ (B ◦ C) = (A ◦ B) ◦ C, A ◦ (B + C) = A ◦ B + A ◦ C

(7.3.3) (7.3.4)

hold. For a real scalar α ≥ 0 and A ∈ Fn×m, the Schur power A◦α is defined by α   (A◦α )(i,j) = A(i,j) . (7.3.5) Thus, A◦2 = A ◦ A. Note that A◦0 = 1n×m. Furthermore, α < 0 is allowed if A has no zero entries. In particular, A◦−1 is the matrix whose entries are the reciprocals of the entries of A. For all A ∈ Fn×m, A ◦ 1n×m = 1n×m ◦ A = A.

(7.3.6)

Finally, if A is square, then I ◦ A is the diagonal part of A. The following result shows that A ◦ B is a submatrix of A ⊗ B. Proposition 7.3.1. Let A, B ∈ Fn×m. Then, A ◦ B = (A ⊗ B)({1,n+2,2n+3,...,n2 },{1,m+2,2m+3,...,m2 }) .

(7.3.7)

If, in addition, n = m, then A ◦ B = (A ⊗ B)({1,n+2,2n+3,...,n2 }) , and thus A ◦ B is a principal submatrix of A ⊗ B. Proof. See [730, p. 304] or [987].

(7.3.8)

445

KRONECKER AND SCHUR ALGEBRA

7.4 Facts on the Kronecker Product Fact 7.4.1. Let x, y ∈ Fn. Then, x ⊗ y = (x ⊗ In )y = (In ⊗ y)x. Fact 7.4.2. Let x, y, w, z ∈ Fn. Then, xTwy Tz = (xT ⊗ y T )(w ⊗ z) = (x ⊗ y)T(w ⊗ z). Fact 7.4.3. Let A ∈ Fn×m and B ∈ F1×m. Then, A(B ⊗ Im ) = B ⊗ A. Fact 7.4.4. Let A ∈ Fn×n and B ∈ Fm×m, and assume that A and B are (diagonal, upper triangular, lower triangular). Then, so is A ⊗ B. Fact 7.4.5. Let A ∈ Fn×n, B ∈ Fm×m, and l ∈ P. Then, (A ⊗ B)l = Al ⊗ B l. Fact 7.4.6. Let A ∈ Fn×m. Then,

  vec A = (Im ⊗ A) vec Im = AT ⊗ In vec In .

Fact 7.4.7. Let A ∈ Fn×m and B ∈ Fm×l. Then, m      coli BT ⊗ coli(A). vec AB = (Il ⊗ A)vec B = BT ⊗ A vec Im = i=1

Fact 7.4.8. Let A ∈ Fn×m, B ∈ Fm×l, and C ∈ Fl×n. Then, tr ABC = (vec A)T(B ⊗ In )vec CT. Fact 7.4.9. Let A, B, C ∈ Fn×n, and assume that C is symmetric. Then, (vec C)T(A ⊗ B)vec C = (vec C)T(B ⊗ A)vec C. Fact 7.4.10. Let A ∈ Fn×m, B ∈ Fm×l, C ∈ Fl×k, and D ∈ Fk×n. Then,   tr ABCD = (vec A)T B ⊗ DT vec CT. Fact 7.4.11. Let A ∈ Fn×m, B ∈ Fm×l, and k ≥ 1. Then, (AB)⊗k = A⊗kB ⊗k, 

where A⊗k = A ⊗ A ⊗ · · · ⊗ A, with A appearing k times. Fact 7.4.12. Let A, C ∈ Fn×m and B, D ∈ Fl×k, assume that A is (left equivalent, right equivalent, biequivalent) to C, and assume that B is (left equivalent, right equivalent, biequivalent) to D. Then, A⊗B is (left equivalent, right equivalent, biequivalent) to C ⊗ D.

446

CHAPTER 7

Fact 7.4.13. Let A, B, C, D ∈ Fn×n, assume that A is (similar, congruent, unitarily similar) to C, and assume that B is (similar, congruent, unitarily similar) to D. Then, A ⊗ B is (similar, congruent, unitarily similar) to C ⊗ D. Fact 7.4.14. Let A ∈ Fn×n and B ∈ Fm×m, and let γ ∈ spec(A ⊗ B). Then,  gmultA(λ)gmultB (μ) ≤ gmultA⊗B (γ) ≤ amultA⊗B (γ)  = amultA(λ)amultB (μ), where both sums are taken over all λ ∈ spec(A) and μ ∈ spec(B) such that λμ = γ. Fact 7.4.15. Let A ∈ Fn×n. Then, sprad(A ⊗ A) = [sprad(A)]2. Fact 7.4.16. Let A ∈ Fn×n and B ∈ Fm×m, and let γ ∈ spec(A ⊗ B). Then, indA⊗B (γ) = 1 if and only if indA(λ) = 1 and indB (μ) = 1 for all λ ∈ spec(A) and μ ∈ spec(B) such that λμ = γ. Fact 7.4.17. Let A ∈ Fn×n and B ∈ Fn×n, and assume that A and B are (group invertible, range Hermitian, range symmetric, Hermitian, symmetric, normal, positive semidefinite, positive definite, unitary, orthogonal, projectors, reflectors, involutory, idempotent, tripotent, nilpotent, semisimple). Then, so is A ⊗ B. Remark: See Fact 7.4.33. Fact 7.4.18. Let A1, . . . , Al ∈ Fn×n, and assume that A1, . . . , Al are skew Hermitian. If l is (even, odd), then A1 ⊗ · · · ⊗ Al is (Hermitian, skew Hermitian). Fact 7.4.19. Let Ai,j ∈ Fni ×nj for all i ∈ {1, . . . , k} and j ∈ {1, . . . , l}. Then, ⎡ ⎡ ⎤ ⎤ A11 ⊗ B A22 ⊗ B · · · A1l ⊗ B A11 A22 · · · A1l ⎢ ⎢ ⎥ ⎥ . . ⎢ ⎢ A21 A22 · .· · A2l ⎥ · ·. · A2l ⊗ B ⎥ ⎢ ⎥ ⊗ B = ⎢ A21 ⊗ B A22 ⊗ B ⎥. ⎢ ⎢ . ⎥ .. . . .. . . .. ⎥ ⎣ ⎣ .. ⎦ ⎦ · · · · · · · ·. · · .· · . . . . . Ak1 ⊗ B Ak2 ⊗ B · · · Akl ⊗ B Ak1 Ak2 · · · Akl Fact 7.4.20. Let x ∈ Fk, and let Ai ∈ Fn×ni for all i ∈ {1, . . . , l}. Then,



x ⊗ A1 · · · Al = x ⊗ A1 · · · x ⊗ Al . Fact 7.4.21. Let x ∈ Fm, let A ∈ Fn×m, and let B ∈ Fm×l. Then, (A ⊗ x)B = (A ⊗ x)(B ⊗ 1) = (AB) ⊗ x. and B ∈ Fm×m. Then, the eigenvalues of Let A ∈ Fn×n  k,l Fact 7.4.22. k,l i j i j i,j=1,1 γij A ⊗ B are of the form i,j=1,1 γij λ μ , where λ ∈ spec(A) and μ ∈ spec(B), and an associated eigenvector is given by x⊗y, where x ∈ Fn is an eigenvector of A associated with λ ∈ spec(A) and y ∈ Fn is an eigenvector of B associated with μ ∈ spec(B).

KRONECKER AND SCHUR ALGEBRA

447

Proof: Let Ax = λx and By = μy. Then, γij (Ai ⊗ B j )(x ⊗ y) = γij λiμj (x ⊗ y). See [532], [892, p. 411], or [967, p. 83]. Remark: This result is due to Stephanos. Fact 7.4.23. Let A ∈ Fn×m and B ∈ Fl×k. Then, R(A ⊗ B) = R(A ⊗ Il×l ) ∩ R(In×n ⊗ B). Proof: See [1325]. Fact 7.4.24. Let A ∈ Fn×m and B ∈ Fl×k. Then, rank(A ⊗ B) = (rank A)(rank B) = rank(B ⊗ A). Consequently, A ⊗ B = 0 if and only if either A = 0 or B = 0. Proof: Use the singular value decomposition of A ⊗ B. Remark: See Fact 8.22.16. Fact 7.4.25. Let A ∈ Fn×m, B ∈ Fl×k, C ∈ Fn×p, D ∈ Fl×q. Then,

rank A ⊗ B C ⊗ D



* (rank A) rank B D + (rank D) rank A C − (rank A) rank D ≤



(rank B) rank A C + (rank C) rank B D − (rank B) rank C. Proof: See [1329]. Fact 7.4.26. Let A ∈ Fn×n and B ∈ Fm×m. Then, rank(I − A ⊗ B) ≤ nm − [n − rank(I − A)][m − rank(I − B)]. Proof: See [341]. Fact 7.4.27. Let A ∈ Fn×n and B ∈ Fm×m. Then, ind A ⊗ B = max{ind A, ind B}. Fact 7.4.28. Let A ∈ Fn×m and B ∈ Fm×n. Then, |n − m|min{n, m} ≤ amultA⊗B (0). Proof: See [730, p. 249]. Fact 7.4.29. Let A ∈ Fn×m and B ∈ Fl×k, and assume that nl = mk and n = m. Then, A ⊗ B and B ⊗ A are singular. Proof: See [730, p. 250]. Fact 7.4.30. The Kronecker permutation matrix Pn,m ∈ Rnm×nm has the following properties: i) Pn,m is a permutation matrix. T −1 = Pn,m = Pm,n . ii) Pn,m

448

CHAPTER 7

iii) Pn,m is orthogonal. iv) Pn,m Pm,n = Inm . v) Pn,n is orthogonal, symmetric, and involutory. vi) Pn,n is a reflector. n vii) Pn,m = i=1 ei,n ⊗ Im ⊗ eT i,n. viii) Pnp,m = (In ⊗ Pp,m )(Pn,m ⊗ Ip ) = (Ip ⊗ Pn,m )(Pp,m ⊗ In ). ix) sig Pn,n = tr Pn,n = n. x) The inertia of Pn,n is given by In Pn,n 2

xi) det Pn,n = (−1)(n

⎡

⎤ − n) ⎢ ⎥ 0 =⎣ ⎦. 1 2 (n + n) 2 1 2 2 (n

−n)/2

.

xii) P1,m = Im and Pn,1 = In . xiii) If x ∈ Fn and y ∈ Fm, then Pn,m (y ⊗ x) = x ⊗ y. xiv) If A ∈ Fn×m and b ∈ Fk, then Pk,n (A ⊗ b) = b ⊗ A and

Pn,k (b ⊗ A) = A ⊗ b.

xv) If A ∈ Fn×m and B ∈ Fl×k, then Pl,n (A ⊗ B)Pm,k = B ⊗ A, vec(A ⊗ B) = (Im ⊗ Pk,n ⊗ Il )[(vec A) ⊗ (vec B)], vec(AT ⊗ B) = (Pnk,m ⊗ Il )[(vec A) ⊗ (vec B)], vec(A ⊗ B T ) = (Im ⊗ Pl,nk )[(vec A) ⊗ (vec B)]. xvi) If A ∈ Fn×m, B ∈ Fl×k, and nl = mk, then tr(A ⊗ B) = [vec(Im ) ⊗ (Ik )]T [(vec A) ⊗ (vec B T )]. xvii) If A ∈ Fn×n and B ∈ Fl×l, then −1 = B ⊗ A. Pl,n (A ⊗ B)Pn,l = Pl,n (A ⊗ B)Pl,n

Hence, A ⊗ B and B ⊗ A are similar. xviii) If A ∈ Fn×m and B ∈ Fm×n, then tr AB = tr[Pm,n (A ⊗ B)]. xix) Pnp,m = Pn,pm Pp,nm = Pp,nm Pn,pm . xx) Pnp,m Ppm,n Pmn,p = I.

449

KRONECKER AND SCHUR ALGEBRA 



Now, let A ∈ Fn×m , let r = rank A, and define K = Pn,m (A∗ ⊗ A). Then, the following statements hold: xxi) K is Hermitian. xxii) rank K = r2. xxiii) tr K = tr A∗A. xxiv) K 2 = (AA∗ ) ⊗ (A∗A). xxv) mspec(K) = {σ12 (A), . . . , σr2 (A)} ∪ {±σi (A)σj (A): i < j, i, j = 1, . . . , r}. Proof: See [1208, pp. 308–311, 342, 343]. 2

Fact 7.4.31. Define Ψn ∈ Rn

×n2

by

 1

Ψn = 2 (In2 + Pn,n ), let x, y ∈ Fn, and let A, B ∈ Fn×n. Then, the following statements hold: i) Ψn is a projector. ii) Ψn = Ψn Pn,n = Pn,n Ψn . iii) Ψn (x ⊗ y) = 12 (x ⊗ y + y ⊗ x). iv) Ψn vec(A) = 12 vec(A + AT ). v) Ψn (A ⊗ B)Ψn = Ψn (B ⊗ A)Ψn . vi) Ψn (A ⊗ A)Ψn = Ψn (A ⊗ A) = (A ⊗ A)Ψn . vii) Ψn (A ⊗ B + B ⊗ A)Ψn = Ψn (A ⊗ B + B ⊗ A) = (A ⊗ B + B ⊗ A)Ψn = 2Ψn (B ⊗ A)Ψn . viii) (A ⊗ A)Ψn (AT ⊗ AT ) = Ψn (AAT ⊗ AAT ). Proof: See [1208, p. 312]. Fact 7.4.32. Let A ∈ Fn×m and B ∈ Fl×k. Then, (A ⊗ B)+ = A+ ⊗ B +. Fact 7.4.33. Let A ∈ Fn×n and B ∈ Fm×m. Then, (A ⊗ B)D = AD ⊗ B D. Now, assume in addition that A and B are group invertible. Then, A ⊗ B is group invertible, and (A ⊗ B)# = A# ⊗ B #. Remark: See Fact 7.4.17. Fact 7.4.34. For all i ∈ {1, . . . , p}, let Ai ∈ Fni ×ni . Then, mspec(A1 ⊗ · · · ⊗ Ap ) = {λ1 · · · λp : λi ∈ mspec(Ai ) for all i ∈ {1, . . . , p}}ms . If, in addition, for all i ∈ {1, . . . , p}, xi ∈ Cni is an eigenvector of Ai associated

450

CHAPTER 7

with λi ∈ spec(Ai ), then x1 ⊗ · · · ⊗ xp is an eigenvector of A1 ⊗ · · · ⊗ Ap associated with the eigenvalue λ1 · · · λp .

7.5 Facts on the Kronecker Sum Fact 7.5.1. Let A ∈ Fn×n. Then, (A ⊕ A)2 = A2 ⊕ A2 + 2A ⊗ A. Fact 7.5.2. Let A ∈ Fn×n. Then, n ≤ def(AT ⊕ −A) = dim {X ∈ Fn×n : AX = XA} and

rank(AT ⊕ −A) = dim {[A, X]: X ∈ Fn×n } ≤ n2 − n.

Proof: See Fact 2.18.9. Remark: rank(AT ⊕ −A) is the dimension of the commutant or centralizer of A. See Fact 2.18.9. Remark: See Fact 5.14.21 and Fact 5.14.23. Problem: Express rank(AT ⊕ −A) in terms of the eigenstructure of A. Fact 7.5.3. Let A ∈ Fn×n, assume that A is nilpotent, and assume that A ⊕ −A = 0. Then, A = 0. T

Proof: Note that AT ⊗ Ak = I ⊗ Ak+1, and use Fact 7.4.24. Fact 7.5.4. Let A ∈ Fn×n, and assume that, for all X ∈ Fn×n, AX = XA. Then, there exists α ∈ F such that A = αI. Proof: It follows from Proposition 7.2.3 that all of the eigenvalues of A are equal. Hence, there exists α ∈ F such that A = αI + B, where B is nilpotent. Now, Fact 7.5.3 implies that B = 0. Fact 7.5.5. Let A ∈ Fn×n and B ∈ Fm×m, and let γ ∈ spec(A ⊕ B). Then,  gmultA(λ)gmultB (μ) ≤ gmultA⊕B (γ) ≤ amultA⊕B (γ)  = amultA(λ)amultB (μ), where both sums are taken over all λ ∈ spec(A) and μ ∈ spec(B) such that λ + μ = γ. Fact 7.5.6. Let A ∈ Fn×n. Then, spabs(A ⊕ A) = 2 spabs(A). Fact 7.5.7. Let A ∈ Fn×n and B ∈ Fm×m, and let γ ∈ spec(A ⊕ B). Then, indA⊕B (γ) = 1 if and only if indA(λ) = 1 and indB (μ) = 1 for all λ ∈ spec(A) and μ ∈ spec(B) such that λ + μ = γ.

451

KRONECKER AND SCHUR ALGEBRA

Fact 7.5.8. Let A ∈ Fn×n and B ∈ Fm×m, and assume that A and B are (group invertible, range Hermitian, Hermitian, symmetric, skew Hermitian, skew symmetric, normal, positive semidefinite, positive definite, semidissipative, dissipative, nilpotent, semisimple). Then, so is A ⊕ B. Fact 7.5.9. Let A ∈ Fn×n and B ∈ Fm×m. Then, −1 = B ⊕ A. Pm,n (A ⊕ B)Pn,m = Pm,n (A ⊕ B)Pm,n

Hence, A ⊕ B and B ⊕ A are similar, and thus rank(A ⊕ B) = rank(B ⊕ A). Proof: Use xiii) of Fact 7.4.30. Fact 7.5.10. Let A ∈ Fn×n and B ∈ Fm×m. Then, n rank B + m rank A − 2(rank A)(rank B) ≤ rank(A ⊕ B)  nm − [n − rank(I + A)][m − rank(I − B)] ≤ nm − [n − rank(I − A)][m − rank(I + B)]. If, in addition, −A and B are idempotent, then rank(A ⊕ B) = n rank B + m rank A − 2(rank A)(rank B). Equivalently, rank(A ⊕ B) = (rank (−A)⊥ ) rank B + (rank B⊥ ) rank A. Proof: See [341]. Remark: Equality may not hold for the upper bounds when −A and B are idempotent. Fact 7.5.11. Let A ∈ Fn×n, let B ∈ Fm×m, assume that A is positive definite,  and define p(s) = det(I − sA), and let mroots(p) = {λ1, . . . , λn }ms . Then, det(A ⊕ B) = (det A)m

n !

det(λi B + I).

i=1

Proof: Specialize Fact 7.5.12. Fact 7.5.12. Let A, C ∈ Fn×n, let B, D ∈ Fm×m, assume that A is positive  definite, assume that C is positive semidefinite, define p(s) = det(C − sA), and let mroots(p) = {λ1, . . . , λn }ms . Then, det(A ⊗ B + C ⊗ D) = (det A)m

n !

det(λi D + B).

i=1

Proof: See [1027, pp. 40, 41]. Remark: The Kronecker product definition in [1027] follows the convention of [967], where “A ⊗ B” denotes B ⊗ A.

452

CHAPTER 7

Fact 7.5.13. Let A, D ∈ Fn×n, let C, B ∈ Fm×m, assume that rank C = 1, and assume that A is nonsingular. Then,   det(A ⊗ B + C ⊗ D) = (det A)m (det B)n−1det B + tr CA−1 D . Proof: See [1027, p. 41]. Fact 7.5.14. Let A ∈ Fn×n and B ∈ Fm×m. Then, spec(A)

and Aspec(−B)

0 C are disjoint if and only if, for all C ∈ Fn×m, the matrices A 0 −B and 0 −B are similar. Proof: Sufficiency follows from Fact 5.10.21, while necessity follows from Corollary 2.6.6 and Proposition 7.2.3. Fact 7.5.15. Let A ∈ Fn×n, B ∈ Fm×m, and C ∈ Fn×m, and assume that det(BT ⊕ A) = 0. Then, X ∈ Fn×m satisfies A2X + 2AXB + XB 2 + C = 0   −2 X = −vec−1 BT ⊕ A vec C .

if and only if

Fact 7.5.16. For all i ∈ {1, . . . , p}, let Ai ∈ Fni ×ni . Then, mspec(A1 ⊕ · · · ⊕ Ap ) = {λ1 + · · · + λp : λi ∈ mspec(Ai ) for all i ∈ {1, . . . , p}}ms . If, in addition, for all i ∈ {1, . . . , p}, xi ∈ Cni is an eigenvector of Ai associated with λi ∈ spec(Ai ), then x1 ⊕ · · · ⊕ xp is an eigenvector of A1 ⊕ · · · ⊕ Ap associated with λ1 + · · · + λp . Fact 7.5.17. Let A ∈ Fn×m, and let k ∈ P satisfy  1 ≤ k ≤ min{n, m}. Furthermore, define the kth compound A(k) to be the nk × m matrix whose k entries are k × k subdeterminants of A, ordered lexicographically. (Example: For n = k = 3, subsets of the rows and columns of A arechosen  in the order {1, 1, 1}, {1, 1, 2}, {1, 1, 3}, {1, 2, 1}, {1, 2, 2}, . . ..) Specifically, A(k) (i,j) is the k × k subdeterminant of A corresponding to the ith selection of k rows of A and the jth selection of k columns of A. Then, the following statements hold: i) A(1) = A. ii) (αA)(k) = αkA(k).  (k)  (k) T = A . iii) AT (k)

= A(k) . ∗  v) (A∗ )(k) = A(k) .

iv) A

vi) If B ∈ Fm×l and 1 ≤ k ≤ min{n, m, l}, then (AB)(k) = A(k)B (k). vii) If B ∈ Fm×n, then det AB = A(k)B (k). Now, assume in addition that m = n, let 1 ≤ k ≤ n, and let mspec(A) = {λ1, . . . , λn }ms . Then, the following statements hold:

453

KRONECKER AND SCHUR ALGEBRA

viii) If A is (diagonal, lower triangular, upper triangular, Hermitian, positive semidefinite, positive definite, unitary), then so is A(k). ix) Assume that A is skew Hermitian. If k is odd, then A(k) is skew Hermitian. If k is even, then A(k) is Hermitian. x) Assume that A is diagonal, upper triangular, or lower triangular, and let 1 ≤ i1 < · · · < ik ≤ n. Then, the (i1 + · · · + ik , i1 + · · · + ik ) entry of A(k) (k) is A(i1,i1 ) · · · A(ik ,ik ) . In particular, In = I(n) . k

(k)

xi) det A

n−1 = (det A)( k−1 ).

xii) A(n) = det A. 

xiii) SA(n−1)TS = AA, where S = diag(1, −1, 1, . . .). xiv) det A(n−1) = det AA = (det A)n−1. xv) tr A(n−1) = tr AA. −1  −1 (k)  = A . xvi) If A is nonsingular, then A(k)  (k)  xvii) mspec A = {λi1 · · · λik : 1 ≤ i1 < · · · < ik ≤ n}ms . In particular,   mspec A(2) = {λi λj : i, j = 1, . . . , n, i < j}ms .  xviii) tr A(k) = 1≤i1 0 for all i ∈ {1, . . . , r}, and i=1 αi ≥ 1. Then, r sprad(A1◦α1 ◦ · · · ◦ A◦α r )≤

r !

α

[sprad(Ai )] i.

i=1

In particular, let A ∈ R

, and assume that A ≥≥ 0. Then, for all α ≥ 1,

n×n

sprad(A◦α ) ≤ [sprad(A)] , α

whereas, for all α ≤ 1, Furthermore, and

[sprad(A)]α ≤ sprad(A◦α ).   sprad A◦1/2 ◦ AT◦1/2 ≤ sprad(A) 1/2

[sprad(A ◦ A)]

1/2

≤ sprad(A) = [sprad(A ⊗ A)]

.

If, in addition, B ∈ Rn×n is such that B ≥≥ 0, then sprad(A ◦ B) ≤ [sprad(A ◦ A) sprad(B ◦ B)]

1/2

≤ sprad(A) sprad(B),

457

KRONECKER AND SCHUR ALGEBRA

sprad(A ◦ B) ≤ sprad(A) sprad(B) + max [2A(i,i) B(i,i) − sprad(A)B(i,i) − sprad(B)A(i,i) ] i=1,...,n

≤ sprad(A) sprad(B),    sprad A◦1/2 ◦ B ◦1/2 ≤ sprad(A) sprad(B).

and

If, in addition, A >> 0 and B >> 0, then sprad(A ◦ B) < sprad(A) sprad(B). Proof: See [466, 480, 814]. The equality sprad(A) = [sprad(A ⊗ A)] Fact 7.4.15.

1/2

follows from

Remark: The inequality sprad(A ◦ A) ≤ sprad(A ⊗ A) follows from Fact 4.11.19 and Proposition 7.3.1. Remark: Some extensions are given in [750]. Fact 7.6.15. Let A, B ∈ Rn×n, and assume that A and B are nonsingular M-matrices. Then, the following statements hold: i) A ◦ B −1 is a nonsingular M-matrix. ii) If n = 2, then τ (A ◦ A−1 ) = 1. iii) If n ≥ 3, then

1 n

< τ (A ◦ A−1 ) ≤ 1.

iv) τ (A) mini=1,...,n (B −1 )(i,i) ≤ τ (A ◦ B −1 ). v) [τ (A)τ (B)]n ≤ |det(A ◦ B)|. vi) |(A ◦ B)−1 | ≤≤ A−1 ◦ B −1. Proof: See [730, pp. 359, 370, 375, 380]. Remark: The minimum eigenvalue τ (A) is defined in Fact 4.11.11. Remark: Some extensions are given in [750]. Fact 7.6.16. Let A, B ∈ Fn×m. Then, + sprad(A ◦ B) ≤ sprad(A ◦ A) sprad(B ◦ B). Consequently,

⎫ sprad(A ◦ A) ⎪ ⎬ sprad(A ◦ AT ) ≤ sprad(A ◦ A). ⎪ ⎭ sprad(A ◦ A∗ )

Proof: See [1224]. Remark: See Fact 9.14.33.

458

CHAPTER 7

Fact 7.6.17. Let A, B ∈ Rn×n, assume that A and B are nonnegative, and let α ∈ [0, 1]. Then, sprad(A◦α ◦ B ◦(1−α) ) ≤ spradα (A) sprad1−α (B). In particular,

sprad(A◦1/2 ◦ B ◦1/2 ) ≤



sprad(A) sprad(B).

Finally, sprad(A◦1/2 ◦ A◦1/2T ) ≤ sprad(A◦α ◦ A◦(1−α)T ) ≤ sprad(A). Proof: See [1224]. Remark: See Fact 9.14.36.

7.7 Notes A history of the Kronecker product is given in [683]. Kronecker matrix algebra is discussed in [263, 593, 685, 973, 1019, 1250, 1413]. Applications are discussed in [1148, 1149, 1396]. The fact that the Schur product is a principal submatrix of the Kronecker product is noted in [987]. A variation of Kronecker matrix algebra for symmetric matrices can be developed in terms of the half-vectorization operator “vech” and the associated elimination and duplication matrices [685, 972, 1377]. Generalizations of the Schur and Kronecker products, known as the blockKronecker, strong Kronecker, Khatri-Rao, and Tracy-Singh products, are discussed in [393, 733, 760, 864, 948, 950, 951, 953] and [1146, pp. 216, 217]. A related operation is the bialternate product, which is a variation of the compound operation discussed in Fact 7.5.17. See [532, 590], [804, pp. 313–320], and [967, pp. 84, 85]. The Schur product is also called the Hadamard product. The Kronecker product is associated with tensor analysis and multilinear algebra [431, 559, 599, 983, 984, 1019].

Chapter Eight

Positive-Semidefinite Matrices

In this chapter we focus on positive-semidefinite and positive-definite matrices. These matrices arise in a variety of applications, such as covariance analysis in signal processing and controllability analysis in linear system theory, and they have many special properties.

8.1 Positive-Semidefinite and Positive-Definite Orderings Let A ∈ Fn×n be a Hermitian matrix. As shown in Corollary 5.4.5, A is unitarily similar to a real diagonal matrix whose diagonal entries are the eigenvalues of A. We denote these eigenvalues by λ1, . . . , λn or, for clarity, by λ1(A), . . . , λn(A). As in Chapter 4, we employ the convention λ1 ≥ λ2 ≥ · · · ≥ λn ,

(8.1.1)

and, for convenience, we define  λ1, λmax(A) =

 λmin(A) = λn .

(8.1.2)

Then, A is positive semidefinite if and only if λmin(A) ≥ 0, while A is positive definite if and only if λmin(A) > 0. For convenience, let Hn, Nn, and Pn denote, respectively, the Hermitian, positive-semidefinite, and positive-definite matrices in Fn×n. Hence, Pn ⊂ Nn ⊂ Hn. If A ∈ Nn, then we write A ≥ 0, while, if A ∈ Pn, then we write A > 0. If A, B ∈ Hn, then A − B ∈ Nn is possible even if neither A nor B is positive semidefinite. In this case, we write A ≥ B or B ≤ A. Similarly, A − B ∈ Pn is denoted by A > B or B < A. This notation is consistent with the case n = 1, where H1 = R, N1 = [0, ∞), and P1 = (0, ∞). Since 0 ∈ Nn, it follows that Nn is a pointed cone. Furthermore, if A, −A ∈ N , then x∗Ax = 0 for all x ∈ Fn, which implies that A = 0. Hence, Nn is a one-sided cone. Finally, Nn and Pn are convex cones since, if A, B ∈ Nn, then αA + βB ∈ Nn for all α, β > 0, and likewise for Pn. The following result shows that the relation “≤” is a partial ordering on Hn. n

460

CHAPTER 8

Proposition 8.1.1. The relation “≤” is reflexive, antisymmetric, and transitive on Hn, that is, if A, B, C ∈ Hn, then the following statements hold: i) A ≤ A. ii) If A ≤ B and B ≤ A, then A = B. iii) If A ≤ B and B ≤ C, then A ≤ C. Proof. Since Nn is a pointed, one-sided, convex cone, it follows from Proposition 2.3.6 that the relation “≤” is reflexive, antisymmetric, and transitive. Additional properties of “≤” and “ 0, then αA > 0 for all α > 0, and αA < 0 for all α < 0. iii) αA + βB ∈ Hn for all α, β ∈ R. iv) If A ≥ 0 and B ≥ 0, then αA + βB ≥ 0 for all α, β ≥ 0. v) If A ≥ 0 and B > 0, then A + B > 0. vi) A2 ≥ 0. vii) A2 > 0 if and only if det A = 0. viii) If A ≤ B and B < C, then A < C. ix) If A < B and B ≤ C, then A < C. x) If A ≤ B and C ≤ D, then A + C ≤ B + D. xi) If A ≤ B and C < D, then A + C < B + D. Furthermore, let S ∈ Fm×n. Then, the following statements hold: xii) If A ≤ B, then SAS ∗ ≤ SBS ∗. xiii) If A < B and rank S = m, then SAS ∗ < SBS ∗. xiv) If SAS ∗ ≤ SBS ∗ and rank S = n, then A ≤ B. xv) If SAS ∗ < SBS ∗ and rank S = n, then m = n and A < B. xvi) If A ≤ B, then SAS ∗ < SBS ∗ if and only if rank S = m and R(S) ∩ N(B − A) = {0}. Proof. Results i)–xi) are immediate. To prove xiii), note that A < B implies that (B −A)1/2 is positive definite. Thus, rank S(A−B)1/2 = m, which implies that S(A−B)S ∗ is positive definite. To prove xiv), note that, since rank S = n, it follows that S has a left inverse S L ∈ Fn×m. Thus, xii) implies that A = S LSAS ∗S L∗ ≤ S LSBS ∗S L∗ = B. To prove xv), note that, since S(B − A)S ∗ is positive definite, it follows that rank S = m. Hence, m = n and S is nonsingular. Thus, xiii) implies that A = S −1SAS ∗S −∗ < S −1SBS ∗S −∗ = B. Statement xvi) is proved in [293].

461

POSITIVE-SEMIDEFINITE MATRICES

The following result is an immediate consequence of Corollary 5.4.7. Corollary 8.1.3. Let A, B ∈ Hn, and assume that A and B are congruent. Then, A is positive semidefinite if and only if B is positive semidefinite. Furthermore, A is positive definite if and only if B is positive definite.

8.2 Submatrices We first consider some equalities for a partitioned positive-semidefinite matrix. Lemma 8.2.1. Let A =



A11 A12 A∗12 A22



∈ Nn+m. Then,

+ A12 = A11A11 A12 ,

A12 =

(8.2.1)

A12 A22 A+ 22.

(8.2.2)

Proof. Since A ≥ 0, it follows from Corollary 5.4.5 that A = BB ∗, where 1  ∈ F(n+m)×r and r = rank A. Thus, A11 = B1B1∗, A12 = B1B2∗, and B = B B2 A22 = B2 B2∗. Since A11 is Hermitian, it follows from xxvii) of Proposition 6.1.6  ∗ ∗ + that A+ 11 is also Hermitian. Next, defining S = B1 − B1B1(B1B1 ) B1, it follows ∗ ∗ that SS = 0, and thus tr SS = 0. Hence, Lemma 2.2.3 implies that S = 0, and thus B1 = B1B1∗(B1B1∗ )+B1. Consequently, B1B2∗ = B1B1∗(B1B1∗ )+B1B2∗, that is, + A12 = A11 A11 A12 . The second result is analogous. Corollary 8.2.2. Let A =



A11 A12 A∗12 A22



∈ Nn+m. Then, the following statements

hold: i) R(A12 ) ⊆ R(A11 ). ii) R(A∗12 ) ⊆ R(A22 ).

iii) rank A11 A12 = rank A11 .

iv) rank A∗12 A22 = rank A22 . Proof. Results i) and ii) follow from (8.2.1) and (8.2.2), while iii) and iv) are consequences of i) and ii). 

Next, if (8.2.1) holds, then the partitioned Hermitian matrix A = can be factored as       I 0 A11 0 A11 A12 I A+ 11A12 , = 0 A11|A A∗12 A22 I 0 I A∗12 A+ 11 while, if (8.2.2) holds, then     A11 A12 A22|A I A12 A+ 22 = 0 A∗12 A22 0 I where

0 A22



A11|A = A22 − A∗12 A+ 11A12

I ∗ A+ 22 A12

0 I



A11 A12 ∗ A12 A22



(8.2.3)

 ,

(8.2.4) (8.2.5)

462

CHAPTER 8

and

∗ A22|A = A11 − A12 A+ 22 A12 .

(8.2.6)

Hence, it follows from Lemma 8.2.1 that, if A is positive semidefinite, then (8.2.3) and (8.2.4) are valid, and, furthermore, the Schur complements (see Definition 6.1.8) A11|A and A22|A are both positive semidefinite. Consequently, we have the following results.  Proposition 8.2.3. Let A =



A11 A12 A∗12 A22



∈ Nn+m. Then,

rank A = rank A11 + rank A11 |A

(8.2.7)

= rank A22 |A + rank A22

(8.2.8)

≤ rank A11 + rank A22 .

(8.2.9)

det A = (det A11 ) det(A11 |A)

(8.2.10)

det A = (det A22 ) det(A22 |A).

(8.2.11)

Furthermore,

and



Proposition 8.2.4. Let A = ments are equivalent:



A11 A12 A∗12 A22



∈ Hn+m. Then, the following state-

i) A ≥ 0. + + A12 , and A∗12 A11 A12 ≤ A22 . ii) A11 ≥ 0, A12 = A11 A11 + ∗ iii) A22 ≥ 0, A12 = A12 A22 A+ 22, and A12 A22 A12 ≤ A11 .

The following statements are also equivalent: iv) A > 0. v) A11 > 0 and A∗12 A−1 11A12 < A22 . ∗ vi) A22 > 0 and A12 A−1 22 A12 < A11 .

The following result follows from (2.8.16) and (2.8.17) or from (8.2.3) and (8.2.4).    A A12 ∗ Proposition 8.2.5. Let A = A11 ∈ Pn+m. Then, A 22 12 ⎡ ⎤ −1 −1 ∗ −1 −1 A−1 −A−1 11 + A11A12 (A11|A) A12 A11 11A12 (A11|A) ⎦ A−1 = ⎣ −1 ∗ −1 −1 −(A11|A) A12 A11 (A11|A) and

⎡ A−1 = ⎣

(A22|A)−1

−(A22|A)−1A12 A−1 22

∗ −1 −A−1 22 A12 (A22|A)

−1 −1 ∗ −1 A−1 22 A12 (A22|A) A12 A22 + A22

(8.2.12)

⎤ ⎦,

(8.2.13)

463

POSITIVE-SEMIDEFINITE MATRICES

where

∗ −1 A11|A = A22 − A12 A11A12

and Now, let A−1 =



B11 ∗ B12

(8.2.14)

∗ A22|A = A11 − A12 A−1 22 A12 .  B12 B22 . Then,

and

(8.2.15)

B11|A−1 = A−1 22

(8.2.16)

B22|A−1 = A−1 11 .

(8.2.17) 

Lemma 8.2.6. Let A ∈ Fn×n, b ∈ Fn, and a ∈ R, and define A = Then, the following statements are equivalent:

A b b∗ a

.

i) A is positive semidefinite. ii) A is positive semidefinite, b = AA+b, and b∗A+b ≤ a. iii) Either A is positive semidefinite, a = 0, and b = 0, or a > 0 and bb∗ ≤ aA. Furthermore, the following statements are equivalent: i) A is positive definite. ii) A is positive definite, and b∗A−1b < a. iii) a > 0 and bb∗ < aA. In this case,

  det A = (det A) a − b∗A−1b .

(8.2.18)

For the following result note that a matrix is a principal submatrix of itself, while the determinant of a matrix is also a principal subdeterminant of the matrix. Proposition 8.2.7. Let A ∈ Hn. Then, the following statements are equivalent: i) A is positive semidefinite. ii) Every principal submatrix of A is positive semidefinite. iii) Every principal subdeterminant of A is nonnegative. iv) For all i ∈ {1, . . . , n}, the sum of all i × i principal subdeterminants of A is nonnegative. v) β0 , . . . , βn−1 ≥ 0, where χA(s) = sn + βn−1sn−1 + · · · + β1s + β0 . Proof. To prove that i) =⇒ ii), let Aˆ ∈ Fm×m be the principal submatrix of A obtained from A by retaining rows and columns i1, . . . , im . Then, Aˆ = S TAS, where

 ei1 · · · eim ∈ Rn×m. Now, let x ˆ ∈ Fm. Since A is positive semidefinite, S= ∗ˆ ∗ T it follows that xˆ Aˆ ˆ ≥ 0, and thus Aˆ is positive semidefinite. x = xˆ S AS x

464

CHAPTER 8

Next, the statements ii) =⇒ iii) =⇒ iv) are immediate. To prove that iv) =⇒ i), note that it follows from Proposition 4.4.6 that χA(s) =

n 

βi s i =

n 

i=0

i=0

n  (−1)n−i γn−i si = (−1)n γn−i (−s)i ,

(8.2.19)

i=0

where, for all i ∈ {1, . . . , n}, γi is the sum of all i×i principal subdeterminants of A, and βn = γ0 = 1. By assumption, γi ≥ 0 for all i ∈ {1, . . . , n}. Now, suppose there n exists λ ∈ spec(A) such that λ < 0. Then, 0 = (−1)n χA(λ) = i=0 γn−i (−λ)i > 0, which is a contradiction. The equivalence of iv) and v) follows from Proposition 4.4.6. Proposition 8.2.8. Let A ∈ Hn. Then, the following statements are equivalent: i) A is positive definite. ii) Every principal submatrix of A is positive definite. iii) Every principal subdeterminant of A is positive. iv) Every leading principal submatrix of A is positive definite. v) Every leading principal subdeterminant of A is positive. Proof. To prove that i) =⇒ ii), let Aˆ ∈ Fm×m and S be as in the proof of Proposition 8.2.7, and let x ˆ be nonzero so that Sˆ x is nonzero. Since A is positive ˆx = xˆ∗S TASˆ definite, it follows that xˆ∗Aˆ x > 0, and hence Aˆ is positive definite. Next, the implications i) =⇒ ii) =⇒ iii) =⇒ v) and ii) =⇒ iv) =⇒ v) are immediate. To prove that v) =⇒ i), suppose that the leading principal submatrix Ai ∈ Fi×i has positive determinant for all i ∈ {1, . . . , n}. The result is true for n = 1. . Writing Ai+1 = For n ≥  2, we show that, if Ai is positive definite, then so is Ai+1   Ai bi ∗ −1 ∗ , it follows from Lemma 8.2.6 that det A = (det A ) a i+1 i i − biAi bi > 0, bi ai and hence ai − b∗iA−1 i bi = det Ai+1 /det Ai > 0. Lemma 8.2.6 now implies that Ai+1 is positive definite. Using this argument for all i ∈ {2, . . . , n} implies that A is positive definite.

0 shows that every principal subdeterminant of A, The example A = 00 −1 rather than just the leading principal subdeterminants of A, must be checked  1 1 1to  determine whether A is positive semidefinite. A less obvious example is A = 1 1 1 , 11 0 √ √ whose eigenvalues are 0, 1 + 3, and 1 − 3. In this case, the principal subdeterminant det A[1;1] = det [ 11 10 ] < 0. Note that condition iii) of Proposition 8.2.8 includes det A > 0 since the  3/2 −1 1 determinant of A is also a subdeterminant of A. The matrix A = −1 2 1 has 1

1 2

the property that every 1×1 and 2×2 subdeterminant is positive but is not positive definite. This example shows that the result iii) =⇒ ii) of Proposition 8.2.8 is false if the requirement that the determinant of A be positive is omitted.

POSITIVE-SEMIDEFINITE MATRICES

465

8.3 Simultaneous Diagonalization This section considers the simultaneous diagonalization of a pair of matrices A, B ∈ Hn. There are two types of simultaneous diagonalization. Cogredient diagonalization involves a nonsingular matrix S ∈ Fn×n such that SAS ∗ and SBS ∗ are both diagonal, whereas contragredient diagonalization involves finding a nonsingular matrix S ∈ Fn×n such that SAS ∗ and S −∗BS −1 are both diagonal. Both types of simultaneous transformation involve only congruence transformations. We begin by assuming that one of the matrices is positive definite, in which case the results are easy to prove. The first result involves cogredient diagonalization. Theorem 8.3.1. Let A, B ∈ Hn, and assume that A is positive definite. Then, there exists a nonsingular matrix S ∈ Fn×n such that SAS ∗ = I and SBS ∗ is diagonal. Proof. Setting S1 = A−1/2, it follows that S1AS1∗ = I. Now, since S1BS1∗ is Hermitian, it follows from Corollary 5.4.5 that there exists a unitary matrix S2 ∈ Fn×n such that SBS ∗ = S2 S1BS1∗S2∗ is diagonal, where S = S2 S1. Finally, SAS ∗ = S2 S1AS1∗S2∗ = S2IS2∗ = I. An analogous result holds for contragredient diagonalization. Theorem 8.3.2. Let A, B ∈ Hn, and assume that A is positive definite. Then, there exists a nonsingular matrix S ∈ Fn×n such that SAS ∗ = I and S −∗BS −1 is diagonal. Proof. Setting S1 = A−1/2, it follows that S1AS1∗ = I. Since S1−∗BS1−1 is n×n Hermitian, it follows that there exists such that  −∗a unitary  ∗ matrix S2 ∈ F −∗ −∗ −1 −1 −1 −∗ −1 S BS = S2 S1 BS1 S2 = S2 S1 BS1 S2 is diagonal, where S = S2 S1. Finally, SAS ∗ = S2 S1AS1∗S2∗ = S2 IS2∗ = I. Corollary 8.3.3. Let A, B ∈ Hn, and assume that A is positive definite. Then, AB is diagonalizable over F, all of the eigenvalues of AB are real, and In(AB) = In(B). Corollary 8.3.4. Let A, B ∈ Pn. Then, there exists a nonsingular matrix S ∈ Fn×n such that SAS ∗ and S −∗BS −1 are equal and diagonal. Proof. By Theorem 8.3.2 there exists a nonsingular matrix S1 ∈ Fn×n such 1/4  that S1AS1∗ = I and B1 = S1−∗BS1−1 is diagonal. Defining S = B1 S1 yields 1/2 SAS ∗ = S −∗BS −1 = B1 . The transformation S of Corollary 8.3.4 is a balancing transformation. Next, we weaken the requirement in Theorem 8.3.1 and Theorem 8.3.2 that A be positive definite by assuming only that A is positive semidefinite. In this case, however, we assume that B is also positive semidefinite.

466

CHAPTER 8

Theorem 8.3.5. Let A, B ∈ Nn. Then, there exists a nonsingular matrix S ∈ Fn×n such that SAS ∗ = [ I0 00 ] and SBS ∗ is diagonal. Proof. Let the nonsingular matrix S1 ∈ Fn×n be such that S1AS1∗ = [ I0 00 ], B B12 ∗ , which is positive semidefinite. Letting and similarly partition S1BS1∗ = B11 12 B22   +  S2 = I −B12 B22 , it follows from Lemma 8.2.1 that 0 I   + ∗ B12 0 B11 − B12 B22 ∗ ∗ . S2 S1BS1S2 = 0 B22 + ∗ B12 )U1∗ and Next, let U1 and U2 be unitary matrices such that U1(B11 − B12 B22

  U 0 U2 B22 U2∗ are diagonal. Then, defining S3 = 01 U2 and S = S3 S2 S1, it follows that SAS ∗ = [ I0 00 ] and SBS ∗ = S3 S2 S1BS1∗S2∗S3∗ is diagonal.

Theorem 8.3.6. Let A, B ∈ Nn. Then, there exists a nonsingular matrix S ∈ Fn×n such that SAS ∗ = [ I0 00 ] and S −∗BS −1 is diagonal. matrix such that S1AS1∗ = [ I0 00 ], and Proof. Let S1 ∈ Fn×n be a nonsingular  B11 B12 −∗ −1 ∗ similarly partition S1 BS1 = B12 B22 , which is positive semidefinite. Letting   +  S2 = I B11B12 , it follows that 0 I   0 B11 . S2−∗S1−∗BS1−1S2−1 = + ∗ 0 B22 − B12 B11B12 Now, let U1 and U2 be unitary matrices such that U1B11 U1∗ and U1 0   + ∗ U2 (B22 − B12 and S = B11 B12 )U2∗ are diagonal. Then, defining S3 = 0 U2 S3 S2 S1, it follows that SAS ∗ = [ I0 00 ] and S −∗BS −1 = S3−∗S2−∗S1−∗BS1−1S2−1S3−1 is diagonal. Corollary 8.3.7. Let A, B ∈ Nn. Then, AB is semisimple, and every eigenvalue of AB is nonnegative. If, in addition, A and B are positive definite, then every eigenvalue of AB is positive. Proof. It follows from Theorem 8.3.6 that there exists a nonsingular matrix S ∈ Rn×n such that A1 = SAS ∗ and B1 = S −∗BS −1 are diagonal with nonnegative diagonal entries. Hence, AB = S −1A1B1S is semisimple and has nonnegative eigenvalues. A more direct approach to showing that AB has nonnegative eigenvalues is  to use Corollary 4.4.11 and note that λi(AB) = λi B 1/2AB 1/2 ≥ 0. Corollary 8.3.8. Let A, B ∈ Nn, and assume that rank A = rank B = rank AB. Then, there exists a nonsingular matrix S ∈ Fn×n such that SAS ∗ = S −∗BS −1 and such that SAS ∗ is diagonal. Proof. By Theorem 8.3.6 there exists a nonsingular matrix S1 ∈ Fn×n such

 that S1AS1∗ = I0r 00 , where r = rank A, and such that B1 = S1−∗BS1−1 is diagonal.

−1 Ir 0 Hence, AB = S1 0 0 B1S1. Since rank A = rank B = rank AB = r, it follows

467

POSITIVE-SEMIDEFINITE MATRICES

 ˆ1 ∈ Fr×r is diagonal with positive diagonal entries. , where B    1/4    Hence, = Bˆ01 00 . Now, define S2 = Bˆ1 0 and S = S2 S1. Then, 0 I  1/2  SAS ∗ = S2 S1AS1∗ S2∗ = Bˆ1 0 = S2−∗S1−∗BS1−1S2−1 = S −∗BS −1. 

ˆ1 0 B 0 0 S1−∗BS1−1

that B1 =

0

0

8.4 Eigenvalue Inequalities Next, we turn our attention to inequalities for eigenvalues. We begin with a series of lemmas. Lemma 8.4.1. Let A ∈ Hn, and let β ∈ R. Then, the following statements hold: i) βI ≤ A if and only if β ≤ λmin(A). ii) βI < A if and only if β < λmin(A). iii) A ≤ βI if and only if λmax(A) ≤ β. iv) A < βI if and only if λmax(A) < β. Proof. To prove i), assume that βI ≤ A, and let S ∈ Fn×n be a unitary matrix such that B = SAS ∗ is diagonal. Then, βI ≤ B, which yields β ≤ λmin(B) = λmin(A). Conversely, let S ∈ Fn×n be a unitary matrix such that B = SAS ∗ is diagonal. Since the diagonal entries of B are the eigenvalues of A, it follows that λmin(A)I ≤ B, which implies that βI ≤ λmin(A)I ≤ S ∗BS = A. Results ii), iii), and iv) are proved in a similar manner. Corollary 8.4.2. Let A ∈ Hn. Then, λmin(A)I ≤ A ≤ λmax(A)I.

(8.4.1)

Proof. This result follows from i) and iii) of Lemma 8.4.1 with β = λmin(A) and β = λmax(A), respectively. The following result concerns the maximum and minimum values of the Rayleigh quotient. Lemma 8.4.3. Let A ∈ Hn. Then, λmin(A) =

x∈F

and λmax(A) =

x∗Ax \{0} x∗x

(8.4.2)

x∗Ax . \{0} x∗x

(8.4.3)

min n

max n

x∈F

Proof. It follows from (8.4.1) that λmin(A) ≤ x∗Ax/x∗x for all nonzero x ∈ Fn. Letting x ∈ Fn be an eigenvector of A associated with λmin(A), it follows that this lower bound is attained. This proves (8.4.2). An analogous argument yields (8.4.3).

468

CHAPTER 8

The following result is the Cauchy interlacing theorem. Lemma 8.4.4. Let A ∈ Hn, and let A0 be an (n − 1) × (n − 1) principal submatrix of A. Then, for all i ∈ {1, . . . , n − 1}, λi+1(A) ≤ λi(A0 ) ≤ λi(A).

(8.4.4)

Proof. Note that (8.4.4) is the chain of inequalities λn(A) ≤ λn−1(A0 ) ≤ λn−1(A) ≤ · · · ≤ λ2(A) ≤ λ1(A0 ) ≤ λ1(A). Suppose that this chain of inequalities does not hold. In particular, first suppose that the rightmost inequality that is not true is λj (A0 ) ≤ λj (A), so that λj (A) < λj (A0 ). Choose δ such that λj (A) < δ < λj (A0 ) and such that δ is not an eigenvalue of A0 . If j = 1, then A − δI is negative definite, while, if j ≥ 2, then λj (A) < δ < λj (A0 ) ≤ λj−1 (A0 ) ≤ λj−1(A), so that A − δI has j − 1 positive eigenvalues. Thus, ν+(A − δI) = j −1. Furthermore, since δ < λj (A0 ), it follows that ν+(A0 − δI) ≥ j. Now, assume for convenience that the rows and columns of A are ordered so   0 β that A0 is the (n −1) × (n −1) leading principal submatrix of A. Thus, A = A ∗ β γ , where β ∈ Fn−1 and γ ∈ F. Next, note the equality A − δI  = ∗

I

0

β (A0 − δI)−1

1



A0 − δI

0

0

γ − δ − β ∗(A0 − δI)−1β



I

(A0 − δI)−1β

0

1

 ,

where A0 − δI is nonsingular since δ is chosen to not be an eigenvalue of A0 . Since the right-hand side of this equality involves a congruence transformation, and since ν+(A0 − δI) ≥ j, it follows from Corollary 5.4.7 that ν+(A − δI) ≥ j. However, this inequality contradicts the fact that ν+(A − δI) = j − 1. Finally, suppose that the rightmost inequality in (8.4.4) that is not true is λj+1(A) ≤ λj (A0 ), so that λj (A0 ) < λj+1(A). Choose δ such that λj (A0 ) < δ < λj+1(A) and such that δ is not an eigenvalue of A0 . Then, it follows that ν+(A − δI) ≥ j + 1 and ν+(A0 − δI) = j − 1. Using the congruence transformation as in the previous case, it follows that ν+(A − δI) ≤ j, which contradicts the fact that ν+(A − δI) ≥ j + 1. The following result is the inclusion principle. Theorem 8.4.5. Let A ∈ Hn, and let A0 ∈ Hk be a k × k principal submatrix of A. Then, for all i ∈ {1, . . . , k}, λi+n−k (A) ≤ λi(A0 ) ≤ λi(A).

(8.4.5)

Proof. For k = n−1, the result is given by Lemma 8.4.4. Hence, let k = n−2, and let A1 denote an (n − 1) × (n − 1) principal submatrix of A such that the (n − 2) × (n − 2) principal submatrix A0 of A is also a principal submatrix of A1 . Therefore, Lemma 8.4.4 implies that λn(A) ≤ λn−1(A1 ) ≤ · · · ≤ λ2 (A1 ) ≤ λ2 (A) ≤ λ1(A1 ) ≤ λ1(A) and λn−1(A1 ) ≤ λn−2 (A0 ) ≤ · · · ≤ λ2 (A0 ) ≤ λ2 (A1 ) ≤ λ1(A0 ) ≤ λ1(A1 ). Combining these inequalities yields λi+2 (A) ≤ λi(A0 ) ≤ λi(A)

469

POSITIVE-SEMIDEFINITE MATRICES

for all i = 1, . . . , n − 2, while proceeding in a similar manner with k < n − 2 yields (8.4.5). Corollary 8.4.6. Let A ∈ Hn, and let A0 ∈ Hk be a k × k principal submatrix of A. Then,

and

λmin(A) ≤ λmin(A0 ) ≤ λmax(A0 ) ≤ λmax(A)

(8.4.6)

λmin(A0 ) ≤ λk(A).

(8.4.7)

The following result compares the maximum and minimum eigenvalues with the maximum and minimum diagonal entries. Corollary 8.4.7. Let A ∈ Hn. Then, λmin(A) ≤ dmin(A) ≤ dmax(A) ≤ λmax(A).

(8.4.8)

Lemma 8.4.8. Let A, B ∈ Hn, and assume that A ≤ B and mspec(A) = mspec(B). Then, A = B.  ˆ where Aˆ = Proof. Let α ≥ 0 be such that 0 < Aˆ ≤ B, A + αI and ˆ = mspec(B), ˆ and thus det Aˆ = det B. ˆ Next, ˆ = B + αI. Note that mspec(A) B ˆAˆ−1/2. Hence, it follows from i) of Lemma 8.4.1 that it follows that I ≤ Aˆ−1/2B ˆ ˆAˆ−1/2 ) ≥ 1. Furthermore, det(Aˆ−1/2B ˆAˆ−1/2 ) = det B/det λmin (Aˆ−1/2B Aˆ = 1, which −1/2 −1/2 ˆAˆ ˆAˆ−1/2 = I, implies that λi(Aˆ ) = 1 for all i ∈ {1, . . . , n}. Hence, Aˆ−1/2B B ˆ Hence, A = B. and thus Aˆ = B. 

The following result is the monotonicity theorem or Weyl’s inequality. Theorem 8.4.9. Let A, B ∈ Hn, and assume that A ≤ B. Then, for all i ∈ {1, . . . , n}, λi(A) ≤ λi(B). (8.4.9) If A = B, then there exists i ∈ {1, . . . , n} such that λi(A) < λi(B).

(8.4.10)

If A < B, then (8.4.10) holds for all i ∈ {1, . . . , n}. Proof. Since A ≤ B, it follows from Corollary 8.4.2 that λmin(A)I ≤ A ≤ B ≤ λmax(B)I. Hence, it follows from iii) and i) of Lemma 8.4.1 that λmin(A) ≤ λmin(B) and λmax(A) ≤ λmax(B). Next, let S ∈ Fn×n be a unitary matrix such that SAS ∗ = diag[λ1(A), . . . , λn(A)]. Furthermore, for 2 ≤ i ≤ n − 1, let A0 = diag[λ1(A), . . . , λi(A)], and let B0 denote the i × i leading principal submatrices of SAS ∗ and SBS ∗, respectively. Since A ≤ B, it follows that A0 ≤ B0 , which implies that λmin(A0 ) ≤ λmin(B0 ). It now follows from (8.4.7) that λi(A) = λmin(A0 ) ≤ λmin(B0 ) ≤ λi(SBS ∗ ) = λi(B), which proves (8.4.9). If A = B, then it follows from Lemma 8.4.8 that mspec(A) = mspec(B), and thus there exists i ∈ {1, . . . , n} such that (8.4.10) holds. If A < B, then λmin(A0 ) < λmin(B0 ), which implies (8.4.10) for all i ∈ {1, . . . , n}.

470

CHAPTER 8

Corollary 8.4.10. Let A, B ∈ Hn. Then, the following statements hold: i) If A ≤ B, then tr A ≤ tr B. ii) If A ≤ B and tr A = tr B, then A = B. iii) If A < B, then tr A < tr B. iv) If 0 ≤ A ≤ B, then 0 ≤ det A ≤ det B. v) If 0 ≤ A < B, then 0 ≤ det A < det B. vi) If 0 < A ≤ B and det A = det B, then A = B. Proof. Statements i), iii), iv), and v) follow from Theorem 8.4.9. To prove ii), note that, since A ≤ B and tr A = tr B, it follows from Theorem 8.4.9 that mspec(A) = mspec(B). Now, Lemma 8.4.8 implies that A = B. A similar argument yields vi). The following result, which is a generalization of Theorem 8.4.9, is due to Weyl. Theorem 8.4.11. Let A, B ∈ Hn. If i + j ≥ n + 1, then λi(A) + λj (B) ≤ λi+j−n (A + B).

(8.4.11)

λi+j−1(A + B) ≤ λi(A) + λj (B).

(8.4.12)

If i + j ≤ n + 1, then In particular, for all i ∈ {1, . . . , n}, λi(A) + λmin(B) ≤ λi(A + B) ≤ λi(A) + λmax(B),

(8.4.13)

λmin(A) + λmin(B) ≤ λmin(A + B) ≤ λmin(A) + λmax(B),

(8.4.14)

λmax(A) + λmin(B) ≤ λmax(A + B) ≤ λmax(A) + λmax(B).

(8.4.15)

Furthermore, if rank B ≤ r, then, for all i = 1, . . . , n − r, λi+r (A) ≤ λi(A + B),

(8.4.16)

λi+r (A + B) ≤ λi(A).

(8.4.17)

ν− (A + B) ≤ ν− (A) + ν− (B),

(8.4.18)

ν+ (A + B) ≤ ν+ (A) + ν+ (B).

(8.4.19)

Finally,

Proof. See [401], [728, p. 182], and [1208, pp. 112–115].

471

POSITIVE-SEMIDEFINITE MATRICES

Lemma 8.4.12. Let A, B, C ∈ Hn. If A ≤ B and C is positive semidefinite, then

tr AC ≤ tr BC.

(8.4.20)

If A < B and C is positive definite, then tr AC < tr BC.

(8.4.21)

Proof. Since C 1/2AC 1/2 ≤ C 1/2BC 1/2, it follows from i) of Corollary 8.4.10 that

tr AC = tr C 1/2AC 1/2 ≤ tr C 1/2BC 1/2 = tr BC.

Result (8.4.21) follows from ii) of Corollary 8.4.10 in a similar fashion. Proposition 8.4.13. Let A, B ∈ Fn×n, and assume that B is positive semidefinite. Then, 1 2 λmin(A

+ A∗ ) tr B ≤ Re tr AB ≤ 12 λmax(A + A∗ ) tr B.

(8.4.22)

If, in addition, A is Hermitian, then λmin(A) tr B ≤ tr AB ≤ λmax(A) tr B.

(8.4.23)

Proof. It follows from Corollary 8.4.2 that 12 λmin(A+A∗ )I ≤ 12 (A+A∗ ), while Lemma 8.4.12 implies that 12 λmin(A + A∗ ) tr B = tr 12 λmin(A + A∗ )IB ≤ tr 12 (A + A∗ )B = Re tr AB, which proves the left-hand inequality of (8.4.22). Similarly, the right-hand inequality holds. For results relating to Proposition 8.4.13, see Fact 5.12.4, Fact 5.12.5, Fact 5.12.8, and Fact 8.19.19. Proposition 8.4.14. Let A, B ∈ Pn, and assume that det B = 1. Then, (det A)1/n ≤

1 n tr AB.

(8.4.24) 1/n −1

Furthermore, equality holds if and only if B = (det A) A . Proof. Using the arithmetic-mean–geometric-mean inequality given by Fact 1.17.14, it follows that   n 1/n  1/n  !  1/n 1/2 1/2 1/2 1/2 (det A) = det B AB = λi B AB n    ≤ n1 λi B 1/2AB 1/2 =

i=1 1 n tr AB.

i=1

Equality holds if and only if there exists β > 0 such that B 1/2AB 1/2 = βI. In this case, β = (det A)1/n and B = (det A)1/nA−1. The following corollary of Proposition 8.4.14 is Minkowski’s determinant theorem.

472

CHAPTER 8

Corollary 8.4.15. Let A, B ∈ Nn, and let p ∈ [1, n]. Then, p  det A + det B ≤ (det A)1/p + (det B)1/p  n ≤ (det A)1/n + (det B)1/n ≤ det(A + B).

(8.4.25) (8.4.26) (8.4.27)

Furthermore, the following statements hold: i) If A = 0 or B = 0 or det(A + B) = 0, then (8.4.25)–(8.4.27) are equalities. ii) If there exists α ≥ 0 such that B = αA, then (8.4.27) is an equality. iii) If A + B is positive definite and (8.4.27) holds as an equality, then there exists α ≥ 0 such that either B = αA or A = αB. iv) If n ≥ 2, p > 1, A is positive definite, and (8.4.25) holds as an equality, then det B = 0. v) If n ≥ 2, p < n, A is positive definite, and (8.4.26) holds as an equality, then det B = 0. vi) If n ≥ 2, A is positive definite, and det A + det B = det(A + B), then B = 0. Proof. Inequalities (8.4.25) and (8.4.26) are consequences of the power-sum inequality Fact 1.17.35. Now, assume that A+B is positive definite, since otherwise (8.4.25)–(8.4.27) are equalities. To prove (8.4.27), Proposition 8.4.14 implies that   (det A)1/n + (det B)1/n ≤ n1 tr A[det(A + B)]1/n (A + B)−1   + n1 tr B[det(A + B)]1/n (A + B)−1 = [det(A + B)]1/n. Statements i) and ii) are immediate. To prove iii), suppose that A + B is positive definite and that (8.4.27) holds as an equality. Then, either A or B is positive definite. Hence, suppose that A is positive definite. Multiplying the equality (det A)1/n + (det B)1/n = [det(A + B)]1/n by (det A)−1/n yields 1/n   1/n  = det I + A−1/2BA−1/2 . 1 + det A−1/2BA−1/2 Letting λ1, . . . , λn denote the eigenvalues of A−1/2BA−1/2, it follows that 1 + (λ1 · · · λn )1/n = [(1 + λ1 ) · · · (1 + λn )]1/n. It now follows from Fact 1.17.34 that λ1 = · · · = λn . To prove iv), note that, since 1/p < 1, det A > 0, and equality holds in (8.4.25), it follows from Fact 1.17.35 that det B = 0. To prove v), note that, since 1/n < 1/p, det A > 0, and equality holds in (8.4.26), it follows from Fact 1.17.35 that det B = 0. To prove vi), note that (8.4.25) and (8.4.26) hold as equalities for all p ∈ [1, n].

473

POSITIVE-SEMIDEFINITE MATRICES

Therefore, det B = 0. Consequently, det A = det(A + B). Since 0 < A ≤ A + B, it follows from vi) of Corollary 8.4.10 that B = 0.

8.5 Exponential, Square Root, and Logarithm of Hermitian Matrices Let B ∈ Rn×n be diagonal, let D ⊆ R, let f : D → R, and assume that, for all i ∈ {1, . . . , n}, B(i,i) ∈ D. Then, we define 

f (B) = diag[f (B(1,1) ), . . . , f (B(n,n) )]. ∗

(8.5.1)

be Hermitian, where S ∈ F is unitary, Furthermore, let A = SBS ∈ F B ∈ Rn×n is diagonal, and assume that spec(A) ⊂ D. Then, we define f(A) ∈ Hn by  (8.5.2) f(A) = Sf(B)S ∗. n×n

n×n

Hence, with an obvious extension of notation, f : {X ∈ Hn : spec(X) ⊂ D} → Hn. If f : D → R is one-to-one, then its inverse f−1: {X ∈ Hn : spec(X) ⊂ f (D)} → Hn exists. It remains to be shown, however, that the definition of f (A) given by (8.5.2) is independent of the matrices S and B in the decomposition A = SBS ∗. The following lemma is needed. ˆ ∈ Rn×n denote the diagoLemma 8.5.1. Let S ∈ Fn×n be unitary, let D, D ˆ = diag(μ1In1 , . . . , μnr Inr ), where nal matrices D = diag(λ1In1 , . . . , λnr Inr ) and D ˆ = DS. ˆ λ1, . . . , λr , μ1, . . . , λr ∈ R, and assume that SD = DS. Then, SD

S12 . Then, it follows from SD = DS Proof. Let r = 2, and partition S = SS11 21 S22 that λ2 S12 = λ1S12 and λ 1S21 = λ S . Since λ =  λ2 , it follows that S12 = 0 and 1

2 21 ˆ = DS. ˆ A similar argument holds S21 = 0. Therefore, S = S011 S022 , and thus SD for r ≥ 3. Proposition 8.5.2. Let A = RBR∗ = SCS ∗ ∈ Fn×n be Hermitian, where R, S ∈ Fn×n are unitary and B, C ∈ Rn×n are diagonal. Furthermore, let D ⊆ R, let f : D → R, and assume that all diagonal entries of B are contained in D. Then, Rf (B)R∗ = Sf (C)S ∗. Proof. Let spec(A) = {λ1, . . . , λr }. Then, the columns of R and S can be ˜ S˜ ∈ Fn×n such that A = RD ˜ R ˜ ∗ = SD ˜ S˜∗ , rearranged to obtain unitary matrices R,   ˜∗ ˜ where D = diag(λ1In1 , . . . , λnr Inr ). Hence, UD = DU, where U = S R. It thus  ˆ = DU, ˆ where D ˆ = follows from Lemma 8.5.1 that UD f (D) = diag[f (λ1 )In1 , . . . , ˜D ˆR ˜ ∗ = S˜D ˆ S˜∗, while rearranging the columns of R ˜ and S˜ as f (λnr )Inr ]. Hence, R ∗ ∗ ˆ well as the diagonal entries of D yields Rf (B)R = Sf (C)S . R

n×n

Let A = SBS ∗ ∈ Fn×n be Hermitian, where S ∈ Fn×n is unitary and B ∈ is diagonal. Then, the matrix exponential is defined by eA = SeBS ∗ ∈ Hn , 



where, for all i ∈ {1, . . . , n}, (eB )(i,i) = eB(i,i) .

(8.5.3)

474

CHAPTER 8

Let A = SBS ∗ ∈ Fn×n be positive semidefinite, where S ∈ Fn×n is unitary and B ∈ Rn×n is diagonal with nonnegative entries. Then, for all r ≥ 0 (not necessarily an integer), Ar = SB rS ∗ is positive semidefinite, where, for all i ∈ {1, . . . , n},

r  (B r )(i,i) = B(i,i) . Note that A0 = I. In particular, the positive-semidefinite matrix A1/2 = SB 1/2S ∗ (8.5.4) is a square root of A since A1/2A1/2 = SB 1/2S ∗SB 1/2S ∗ = SBS ∗ = A.

(8.5.5)

The uniqueness of the positive-semidefinite square root of A given by (8.5.4) follows from Theorem 10.6.1; see also [730, p. 410] or [902]. Uniqueness can also be shown directly; see [459, pp. 265, 266] or [728, p. 405]. Hence, if C ∈ Fn×m, then C ∗C is positive semidefinite, and we define C  = (C ∗C)1/2. 

(8.5.6)

If A is positive definite, then Ar is positive definite for all r ∈ R, and, if r = 0, then (Ar )1/r= A. Now, assume that A ∈ Fn×n is positive definite. Then, the matrix logarithm is defined by  log A = S(log B)S ∗ ∈ Hn , (8.5.7) 

where, for all i ∈ {1, . . . , n}, (log B)(i,i) = log[B(i,i) ]. In chapters 10 and 11, the matrix exponential, square root, and logarithm are extended to matrices that are not necessarily Hermitian.

8.6 Matrix Inequalities Lemma 8.6.1. Let A, B ∈ Fn, assume that A and B are Hermitian, and assume that 0 ≤ A ≤ B. Then, R(A) ⊆ R(B). Proof. Let x ∈ N(B). Then, x∗Bx = 0, and thus x∗Ax = 0, which implies that Ax = 0. Hence, N(B) ⊆ N(A), and thus N(A)⊥ ⊆ N(B)⊥. Since A and B are Hermitian, it follows from Theorem 2.4.3 that R(A) = N(A)⊥ and R(B) = N(B)⊥. Hence, R(A) ⊆ R(B). The following result is the Douglas-Fillmore-Williams lemma [437, 503]. Theorem 8.6.2. Let A ∈ Fn×m and B ∈ Fn×l. Then, the following statements are equivalent: i) There exists a matrix C ∈ Fl×m such that A = BC. ii) There exists α > 0 such that AA∗ ≤ αBB ∗. iii) R(A) ⊆ R(B).

475

POSITIVE-SEMIDEFINITE MATRICES

Proof. First we prove that i) implies ii). Since A = BC, it follows that AA∗ = BCC ∗B ∗. Since CC ∗ ≤ λmax(CC ∗ )I, it follows that AA∗ ≤ αBB ∗, where  α = λmax(CC ∗ ). To prove that ii) implies iii), first note that Lemma 8.6.1 implies that R(AA∗ ) ⊆ R(αBB ∗ ) = R(BB ∗ ). Since, by Theorem 2.4.3, R(AA∗ ) = R(A) and R(BB ∗ ) = R(B), it follows that R(A) ⊆ R(B). Finally, to prove that iii) implies n×n 0 i), use Theorem 5.6.3 to write B = S1[ D and S2 ∈ Fl×l are 0 0 ]S2 , where S1 ∈ F  r×r unitary and D ∈ R is diagonal with positive diagonal entries, where r =

rank B. ∗ ∗ ∗ ∗ A1 D 0 Since R(S1A) ⊆ R(S1B) and S1B = [ 0 0 ]S2 , it follows that S1A = 0 , where A1 ∈ Fr×m. Consequently,        −1 A1 A1 D 0 0 ∗ D S2 S 2 = S1 = BC, A = S1 0 0 0 0 0 0 −1  where C = S2∗ D0 00 A01 ∈ Fl×m. n Proposition 8.6.3. Let (Ai )∞ i=1 ⊂ N satisfy 0 ≤ Ai ≤ Aj for all i ≤ j, and  n assume there exists B ∈ N satisfying Ai ≤ B for all i ≥ 1. Then, A = limi→∞ Ai exists and satisfies 0 ≤ A ≤ B.

Proof. Let k ∈ {1, . . . , n}, and let i < j. Since Ai ≤ Aj ≤ B, it follows that the sequence (Ar(k,k) )∞ r=1 is nondecreasing and bounded from above by B(k,k) . Hence,  A(k,k) = limr→∞ Ar(k,k) exists. Now, let l ∈ {1, . . . , n}, where l = k. Since Ai ≤ Aj , it follows that (ek + el )TAi(ek + el ) ≤ (ek + el )TAj (ek + el ), which implies that Ai(k,l) −Aj(k,l) ≤ 12 [Aj(k,k) −Ai(k,k) +Aj(l,l) −Ai(l,l) ]. Likewise, (ek −el )TAi(ek −el ) ≤ (ek −el )TAj (ek −el ) implies that Aj(k,l) −Ai(k,l) ≤ 12 [Aj(k,k) −Ai(k,k) +Aj(l,l) −Ai(l,l) ]. Hence, |Aj(k,l) − Ai(k,l) | ≤ 12 [Aj(k,k) − Ai(k,k) ] + 12 [Aj(l,l) − Ai(l,l) ]. Next, since ∞ (Ar(k,k) )∞ r=1 and (Ar(l,l) )r=1 are convergent sequences and thus Cauchy sequences, ∞ it follows that (Ar(k,l) )r=1 is a Cauchy sequence. Consequently, (Ar(k,l) )∞ r=1 is  ∞ convergent, and thus A(k,l) = limi→∞ Ai(k,l) exists. Therefore, (Ai )i=1 is conver gent, and thus A = limi→∞ Ai exists. Since Ai ≤ B for all i ≥ 1, it follows that A ≤ B. Proposition 8.6.4. Let A ∈ Fn×n, assume that A is positive definite, and let p > 0. Then, A−1 (A − I) ≤ log A ≤ p−1 (Ap − I)

(8.6.1)

log A = lim p−1 (Ap − I).

(8.6.2)

and p↓0

Proof. This result follows from Fact 1.11.26. Lemma 8.6.5. Let A ∈ Pn. If A ≤ I, then I ≤ A−1. Furthermore, if A < I, then I < A−1. Proof. Since A ≤ I, it follows from xii) of Proposition 8.1.2 that I = A−1/2AA−1/2 ≤ A−1/2IA−1/2 = A−1. Similarly, A < I implies that I = A−1/2AA−1/2 < A−1/2IA−1/2 = A−1.

476

CHAPTER 8

Proposition 8.6.6. Let A, B ∈ Hn, and assume that either A and B are positive definite or A and B are negative definite. If A ≤ B, then B −1 ≤ A−1. If, in addition, A < B, then B −1 < A−1. Proof. Suppose that A and B are positive definite. Since A ≤ B, it follows that B −1/2AB −1/2 ≤ I. Now, Lemma 8.6.5 implies that I ≤ B 1/2A−1B 1/2, which implies that B −1 ≤ A−1. If A and B are negative definite, then A ≤ B is equivalent to −B ≤ −A. The case A < B is proved in a similar manner. The following result is the Furuta inequality. Proposition 8.6.7. Let A, B ∈ Nn, and assume that 0 ≤ A ≤ B. Furthermore, let p, q, r ∈ R satisfy p ≥ 0, q ≥ 1, r ≥ 0, and p + 2r ≤ (1 + 2r)q. Then,

and

A(p+2r)/q ≤ (ArB pAr )1/q

(8.6.3)

(B rApB r )1/q ≤ B (p+2r)/q.

(8.6.4)

Proof. See [536] or [544, pp. 129, 130]. Corollary 8.6.8. Let A, B ∈ Nn, and assume that 0 ≤ A ≤ B. Then,  1/2 (8.6.5) A2 ≤ AB 2A 

and

BA2B

1/2

≤ B 2.

(8.6.6)

Proof. In Proposition 8.6.7 set r = 1, p = 2, and q = 2. Corollary 8.6.9. Let A, B, C ∈ Nn, and assume that 0 ≤ A ≤ C ≤ B. Then,  1/2  2 1/2 ≤ C 2 ≤ CB 2C . (8.6.7) CA C Proof. This result follows from Corollary 8.6.8. See also [1429]. The following result provides representations for Ar, where r ∈ (0, 1). Proposition 8.6.10. Let A ∈ Pn and r ∈ (0, 1). Then,  sin rπ rπ  I+ A = cos 2 π

∞

r

 xr+1 −1 r dx I − (A + xI) x 1 + x2

(8.6.8)

0

and Ar =

sin rπ π

∞

(A + xI)−1Axr−1 dx. 0

Proof. Let t ≥ 0. As shown in [197], [201, p. 143], ∞  π r xr+1 xr rπ  dx = t . − − cos 1 + x2 t+x sin rπ 2 0

(8.6.9)

477

POSITIVE-SEMIDEFINITE MATRICES

Solving for tr and replacing t by A yields (8.6.8). Likewise, replacing t by A in xxxii) of Fact 1.21.1 yields (8.6.9). The following result is the L¨ owner-Heinz inequality. Corollary 8.6.11. Let A, B ∈ Nn, assume that 0 ≤ A ≤ B, and let r ∈ [0, 1]. Then, Ar ≤ B r. If, in addition, A < B and r ∈ (0, 1], then Ar < B r. Proof. Let 0 < A ≤ B, and let r ∈ [0, 1]. In Proposition 8.6.7, replace p, q, r with r, 1, 0. The first result now follows from (8.6.3). Now, assume that A < B. Then, it follows from (8.6.8) of Proposition 8.6.10 as well as Proposition 8.6.6 that, for all r ∈ (0, 1], ∞

B r − Ar =

sin rπ (A + xI)−1 − (B + xI)−1 xr dx > 0. π 0

Hence, A < B . By continuity, it follows that Ar ≤ B r for all A, B ∈ Nn such that 0 ≤ A ≤ B and for all r ∈ [0, 1]. r

r

Alternatively, assume that A < B. Then, it follows from Proposition 8.6.6 that, for all x ≥ 0, (A + xI)−1A = I − x(A + xI)−1 < I − x(B + xI)−1 = (B + xI)−1B. It thus follows from (8.6.9) of Proposition 8.6.10 that ∞

sin rπ (B + xI)−1B − (A + xI)−1A xr−1 dx > 0. B −A = π r

r

0

Hence, Ar < B r. By continuity, it follows that Ar ≤ B r for all A, B ∈ Nn such that 0 ≤ A ≤ B and for all r ∈ [0, 1]. Alternative proofs are given in [544, p. 127] and [1521, p. 2]. For the case r = 1/2, let λ ∈ R be an eigenvalue of B 1/2 − A1/2, and let x ∈ Fn be an associated eigenvector. Then,      λx∗ B 1/2 + A1/2 x = x∗ B 1/2 + A1/2 B 1/2 − A1/2 x   = x∗ B − B 1/2A1/2 + A1/2B 1/2 − A x = x∗(B − A)x ≥ 0. 1/2 1/2 Since + A is positive semidefinite, it follows that either λ ≥ 0 or  B  x∗ B 1/2 + A1/2 x = 0. In the latter case, B 1/2x = A1/2x = 0, which implies that λ = 0. 

The L¨ owner-Heinz inequality does not extend to r > 1. In fact, A = [ 21 11 ] and  1 0 B = [ 0 0 ] satisfy A ≥ B ≥ 0, whereas, for all r > 1, Ar ≥ B r. For details, see [544, pp. 127, 128].

478

CHAPTER 8

Many of the results given so far involve functions that are nondecreasing or increasing on suitable sets of matrices. Definition 8.6.12. Let D ⊆ Hn, and let φ: D → Hm. Then, the following terminology is defined: i) φ is nondecreasing if, for all A, B ∈ D such that A ≤ B, it follows that φ(A) ≤ φ(B). ii) φ is increasing if φ is nondecreasing and, for all A, B ∈ D such that A < B, it follows that φ(A) < φ(B). iii) φ is strongly increasing if φ is nondecreasing and, for all A, B ∈ D such that A ≤ B and A = B, it follows that φ(A) < φ(B). iv) φ is (nonincreasing, decreasing, strongly decreasing) if −φ is (nondecreasing, increasing, strongly increasing). Proposition 8.6.13. The following functions are nondecreasing: 

i) φ: Hn → Hm defined by φ(A) = BAB ∗, where B ∈ Fm×n. 

ii) φ: Hn → R defined by φ(A) = tr AB, where B ∈ Nn.   A22|A, where A = iii) φ: Nn+m → Nn defined by φ(A) =



A11 A12 A∗12 A22

 .



iv) φ: Nn × Nm → Nnm defined by φ(A, B) = Ar1 ⊗ B r2 , where r1, r2 ∈ [0, 1] satisfy r1 + r2 ≤ 1.  v) φ: Nn × Nn → Nn defined by φ(A, B) = Ar1 ◦ B r2, where r1, r2 ∈ [0, 1] satisfy r1 + r2 ≤ 1.

The following functions are increasing:  λi(A), where i ∈ {1, . . . , n}. vi) φ: Hn → R defined by φ(A) = 

vii) φ: Nn → Nn defined by φ(A) = Ar, where r ∈ [0, 1]. 

viii) φ: Nn → Nn defined by φ(A) = A1/2.  ix) φ: Pn → −Pn defined by φ(A) = −A−r, where r ∈ [0, 1]. 

x) φ: Pn → −Pn defined by φ(A) = −A−1. 

xi) φ: Pn → −Pn defined by φ(A) = −A−1/2. 

xii) φ: −Pn → Pn defined by φ(A) = (−A)−r, where r ∈ [0, 1]. 

xiii) φ: −Pn → Pn defined by φ(A) = −A−1.  xiv) φ: −Pn → Pn defined by φ(A) = −A−1/2. 

xv) φ: Hn → Hm defined by φ(A) = BAB ∗, where B ∈ Fm×n and rank B = m.     A A12 xvi) φ: Pn+m → Pn defined by φ(A) = A22|A, where A = A∗11 . A 22 12

479

POSITIVE-SEMIDEFINITE MATRICES −1





xvii) φ: Pn+m → Pn defined by φ(A) = −(A22|A) , where A =



A11 A12 A∗12 A22

 .

 log A. xviii) φ: Pn → Hn defined by φ(A) =

The following functions are strongly increasing: 

xix) φ: Hn → [0, ∞) defined by φ(A) = tr BAB ∗, where B ∈ Fm×n and rank B = m. 

xx) φ: Hn → R defined by φ(A) = tr AB, where B ∈ Pn.  xxi) φ: Nn → [0, ∞) defined by φ(A) = tr Ar, where r > 0. 

xxii) φ: Nn → [0, ∞) defined by φ(A) = det A. Proof. For the proof of iii), see [922]. To prove xviii), let A, B ∈ Pn, and assume that A ≤ B. Then, for all r ∈ [0, 1], it follows from vii) that r−1 (Ar − I) ≤ r−1 (B r − I). Letting r ↓ 0 and using Proposition 8.6.4 yields log A ≤ log B, which proves that log is nondecreasing. See [544, p. 139]. To prove that log is increasing, assume that A < B, and let ε > 0 be such that A + εI < B. Then, it follows that log A < log(A + εI) ≤ log B. Finally, we consider convex functions defined with respect to matrix inequalities. The following definition generalizes Definition 1.4.3 in the case n = m = p = 1. Definition 8.6.14. Let D ⊆ Fn×m be a convex set, and let φ: D → Hp. Then, the following terminology is defined: i) φ is convex if, for all α ∈ [0, 1] and A1, A2 ∈ D, φ[αA1 + (1 − α)A2 ] ≤ αφ(A1 ) + (1 − α)φ(A2 ).

(8.6.10)

ii) φ is concave if −φ is convex. iii) φ is strictly convex if, for all α ∈ (0, 1) and distinct A1, A2 ∈ D, φ[αA1 + (1 − α)A2 ] < αφ(A1 ) + (1 − α)φ(A2 ).

(8.6.11)

iv) φ is strictly concave if −φ is strictly convex. Theorem 8.6.15. Let S ⊆ R, let φ : S1 → S2 , and assume that φ is continuous. Then, the following statements hold: i) Assume that S1 = S2 = (0, ∞) and φ: Pn → Pn is increasing. Then, ψ : Pn → Pn defined by ψ(x) = 1/φ(x) is convex. ii) Assume that S1 = S2 = [0, ∞). Then, φ: Nn → Nn is increasing if and only if φ: Nn → Nn is concave. iii) Assume that S1 = [0, ∞) and S2 = R. Then, φ: Nn → Hn is convex and φ(0) ≤ 0 if and only if ψ : Pn → Hn defined by ψ(x) = φ(x)/x is increasing. Proof. See [201, pp. 120–122].

480

CHAPTER 8

Lemma 8.6.16. Let D ⊆ Fn×m and S ⊆ Hp be convex sets, and let φ1 : D → S and φ2 : S → Hq. Then, the following statements hold: i) If φ1 is convex and φ2 is nondecreasing and convex, then φ2 • φ1: D → Hq is convex. ii) If φ1 is concave and φ2 is nonincreasing and convex, then φ2 • φ1: D → Hq is convex. iii) If S is symmetric, φ2 (−A) = −φ2 (A) for all A ∈ S, φ1 is concave, and φ2 is nonincreasing and concave, then φ2 • φ1: D → Hq is convex. iv) If S is symmetric, φ2 (−A) = −φ2 (A) for all A ∈ S, φ1 is convex, and φ2 is nondecreasing and concave, then φ2 • φ1: D → Hq is convex. Proof. To prove i) and ii), let α ∈ [0, 1] and A1, A2 ∈ D. In both cases it follows that φ2 (φ1[αA1 + (1 − α)A2 ]) ≤ φ2[αφ1(A1 ) + (1 − α)φ1(A2 )] ≤ αφ2[φ1(A1 )] + (1 − α)φ2[φ1(A2 )]. Statements iii) and iv) follow from i) and ii), respectively. Proposition 8.6.17. The following functions are convex: 

i) φ: Nn → Nn defined by φ(A) = Ar, where r ∈ [1, 2]. 

ii) φ: Nn → Nn defined by φ(A) = A2. 

iii) φ: Pn → Pn defined by φ(A) = A−r, where r ∈ [0, 1]. 

iv) φ: Pn → Pn defined by φ(A) = A−1.  v) φ: Pn → Pn defined by φ(A) = A−1/2. 

vi) φ: Nn → −Nn defined by φ(A) = −Ar, where r ∈ [0, 1]. 

vii) φ: Nn → −Nn defined by φ(A) = −A1/2.  viii) φ: Nn → Hm defined by φ(A) = γBAB ∗, where γ ∈ R and B ∈ Fm×n. 

ix) φ: Nn → Nm defined by φ(A) = BArB ∗, where B ∈ Fm×n and r ∈ [1, 2].  x) φ: Pn → Nm defined by φ(A) = BA−rB ∗, where B ∈ Fm×n and r ∈ [0, 1]. 

xi) φ: Nn → −Nm defined by φ(A) = −BArB ∗, where B ∈ Fm×n and r ∈ [0, 1]. 

xii) φ: Pn → −Pm defined by φ(A) = −(BA−rB ∗ )−p, where B ∈ Fm×n has rank m and r, p ∈ [0, 1]. 

xiii) φ: Fn×m → Nn defined by φ(A) = ABA∗, where B ∈ Nm. 

xiv) φ: Pn × Fm×n → Nm defined by φ(A, B) = BA−1B ∗. −1  −1  A + A−∗ . xv) φ: Pn → Nm defined by φ(A) =

481

POSITIVE-SEMIDEFINITE MATRICES 

xvi) φ: Nn × Nn → Nn defined by φ(A, B) = −A(A + B)+B.    A11 −A22|A, where A = xvii) φ: Nn+m → Nn defined by φ(A) = A∗12   −1  A xviii) φ: Pn+m → Pn defined by φ(A) = (A22|A) , where A = A∗11 12

A12 A22



.  A12 A22 .

 tr Ak, where k is a nonnegative even xix) φ: Hn → [0, ∞) defined by φ(A) = integer. 

xx) φ: Pn → (0, ∞) defined by φ(A) = tr A−r, where r > 0. −p

 xxi) φ: Pn → (−∞, 0) defined by φ(A) = −(tr A−r ) , where r, p ∈ [0, 1].  xxii) φ: Nn × Nn → (−∞, 0] defined by φ(A, B) = − tr (Ar + B r )1/r, where r ∈ [0, 1]. 1/2   xxiii) φ: Nn × Nn → [0, ∞) defined by φ(A, B) = tr A2 + B 2 .  − tr ArXB pX ∗, where X ∈ Fn×m, xxiv) φ: Nn × Nm → R defined by φ(A, B) = r, p ≥ 0, and r + p ≤ 1. 

xxv) φ: Nn → (−∞, 0) defined by φ(A) = − tr ArXApX ∗, where X ∈ Fn×n, r, p ≥ 0, and r + p ≤ 1. 

q

xxvi) φ: Pn ×Pm ×Fm×n → R defined by φ(A, B, X) = (tr A−pXB −rX ∗ ) , where r, p ≥ 0, r + p ≤ 1, and q ≥ (2 − r − p)−1. 

xxvii) φ: Pn × Fn×n → [0, ∞) defined by φ(A, X) = tr A−pXA−rX ∗, where r, p ≥ 0 and r + p ≤ 1.  xxviii) φ: Pn × Fn×n → [0, ∞) defined by φ(A) = tr A−pXA−rX ∗, where r, p ∈ n×n [0, 1] and X ∈ F . 

xxix) φ: Pn → R defined by φ(A) = − tr([Ar, X][A1−r, X]), where r ∈ (0, 1) and X ∈ Hn. 

xxx) φ: Pn → Hn defined by φ(A) = −log A. 

xxxi) φ: Pn → Hm defined by φ(A) = Alog A.  xxxii) φ: Nn \{0} → R defined by φ(A) = − log tr Ar, where r ∈ [0, 1]. 

xxxiii) φ: Pn → R defined by φ(A) = log tr A−1.  xxxiv) φ: Pn × Pn → (0, ∞) defined by φ(A, B) = tr[A(log A − log B)]. 

xxxv) φ: Pn × Pn → [0, ∞) defined by φ(A, B) = −e[1/(2n)]tr(log A+log B). 

xxxvi) φ: Nn → (−∞, 0] defined by φ(A) = −(det A)1/n.  xxxvii) φ: Pn → (0, ∞) defined by φ(A) = log det BA−1B ∗, where B ∈ Fm×n and rank B = m.  xxxviii) φ: Pn → R defined by φ(A) = −log det A. 

xxxix) φ: Pn → (0, ∞) defined by φ(A) = det A−1.

482

CHAPTER 8 

xl) φ: Pn → R defined by φ(A) = log(det Ak /det A), where k ∈ {1, . . . , n −1} and Ak is the leading k × k principal submatrix of A.  xli) φ: Pn → R defined by φ(A) = − det A/det A[n;n] .  −Ar1 ⊗ B r2 , where r1, r2 ∈ xlii) φ: Nn × Nm → −Nnm defined by φ(A, B) = [0, 1] satisfy r1 + r2 ≤ 1. 

xliii) φ: Pn × Nm → Nnm defined by φ(A, B) = A−r ⊗ B 1+r, where r ∈ [0, 1]. 

xliv) φ: Nn × Nn → −Nn defined by φ(A, B) = −Ar1 ◦B r2, where r1, r2 ∈ [0, 1] satisfy r1 + r2 ≤ 1.  k xlv) φ: Hn → R defined by φ(A) = i=1 λi(A), where k ∈ {1, . . . , n}.   n xlvi) φ: Hn → R defined by φ(A) = − i=k λi(A), where k ∈ {1, . . . , n}. Proof. Statements i) and iii) are proved in [45] and [201, p. 123]. Let α ∈ [0, 1] for the remainder of the proof. To prove ii) directly, let A1, A2 ∈ Hn. Since 1/2

1/2  (1 − α) − (1 − α)2 , α(1 − α) = α − α2 it follows that  1/2

1/2 2 0 ≤ α − α2 A1 − (1 − α) − (1 − α)2 A2 

 = α − α2 A21 + (1 − α) − (1 − α)2 A22 − α(1 − α)(A1A2 + A2 A1). Hence,

[αA1 + (1 − α)A2 ]2 ≤ αA21 + (1 − α)A22 ,

which shows that φ(A) = A2 is convex.  −1  −1  To prove iv) directly, let A1, A2 ∈ Pn. Then, AI1 AI and AI2 1 positive semidefinite, and thus  −1   −1  A1 I I A2 α + (1 − α) I A1 I A2   −1 I αA−1 1 + (1 − α)A2 = I αA1 + (1 − α)A2

I A2

 are

is positive semidefinite. It now follows from Proposition 8.2.4 that [αA1 + (1 − −1 −1 α)A2 ]−1 ≤ αA−1 is convex. 1 + (1 − α)A2 , which shows that φ(A) = A  A1/2 To prove v) directly, note that φ(A) = A−1/2 = φ2[φ1(A)], where φ1(A) =  −1 and φ2 (B) = B . It follows from vii) that φ1 is concave, while it follows from iv) that φ2 is convex. Furthermore, x) of Proposition 8.6.13 implies that φ2 is nonincreasing. It thus follows from ii) of Lemma 8.6.16 that φ(A) = A−1/2 is convex.

483

POSITIVE-SEMIDEFINITE MATRICES

To prove vi), let A ∈ Pn, and note that φ(A) = −Ar = φ2[φ1(A)], where   φ1(A) = A−r and φ2 (B) = −B −1. It follows from iii) that φ1 is convex, while it follows from iv) that φ2 is concave. Furthermore, x) of Proposition 8.6.13 implies that φ2 is nondecreasing. It thus follows from iv) of Lemma 8.6.16 that φ(A) = Ar is convex on Pn. Continuity implies that φ(A) = Ar is convex on Nn. To prove vii) directly, let A1, A2 ∈ Nn. Then, 2  1/2 1/2 , 0 ≤ α(1 − α) A1 − A2 which is equivalent to 2  1/2 1/2 αA1 + (1 − α)A2 ≤ αA1 + (1 − α)A2 . Using viii) of Proposition 8.6.13 yields 1/2

1/2

αA1 + (1 − α)A2

≤ [αA1 + (1 − α)A2 ]1/2.

Finally, multiplying by −1 shows that φ(A) = −A1/2 is convex. The proof of viii) is immediate. Statements ix), x), and xi) follow from i), iii), and vi), respectively. −p

To prove xii), note that φ(A) = −(BA−rB ∗ ) = φ2[φ1(A)], where φ1(A) = −BA B ∗ and φ2 (C) = C −p. Statement x) implies that φ1 is concave, while iii) implies that φ2 is convex. Furthermore, ix) of Proposition 8.6.13 implies that φ2 is −p nonincreasing. It thus follows from ii) of Lemma 8.6.16 that φ(A) = −(BA−rB ∗ ) is convex. −r

To prove xiii), let A1, A2 ∈ Fn×m, and let B ∈ Nm. Then, 0 ≤ α(1 − α)(A1 − A2 )B(A1 − A2 )∗ = αA1BA1∗ + (1 − α)A2 BA∗2 − [αA1 + (1 − α)A2 ]B[αA1 + (1 − α)A2 ]∗. Thus, [αA1 + (1 − α)A2 ]B[αA1 + (1 − α)A2 ]∗ ≤ αA1BA1∗ + (1 − α)A2 BA∗2 , which shows that φ(A) = ABA∗ is convex. To prove xiv), letA1, A2 ∈ Pn and B1, B2 ∈ Fm×n  . Then, it follows from ∗ ∗ B1A−1 B2 A−1 1 B1 B1 2 B2 B2 and are positive semidefinite, Proposition 8.2.4 that B1∗ A1 B2∗ A2 and thus     ∗ ∗ B1A−1 B2 A−1 B1 B2 1 B1 2 B2 α + (1 − α) B1∗ A1 B2∗ A2   −1 ∗ ∗ αB1 + (1 − α)B2 αB1A−1 1 B1 + (1 − α)B2 A2 B2 = αB1∗ + (1 − α)B2∗ αA1 + (1 − α)A2

484

CHAPTER 8

is positive semidefinite. It thus follows from Proposition 8.2.4 that [αB1 + (1 − α)B2 ][αA1 + (1 − α)A2 ]−1[αB1 + (1 − α)B2 ]∗ −1 ∗ ∗ ≤ αB1A−1 1 B1 + (1 − α)B2 A2 B2 ,

which shows that φ(A, B) = BA−1B ∗ is convex. Result xv) is given in [1003]. Result xvi) follows from Fact 8.21.18.   11 B12  A A12  n+m ∗ ∈ Pn+m and B = B To prove xvii), let A = A11 . Then, B12 B22 ∈ P 12 A22 it follows from xiv) with A1, B1, A2 , B2 replaced by A22 , A12 , B22 , B12 , respectively, that [αA12 + (1 − α)B12 ][αA22 + (1 − α)B22 ]−1[αA12 + (1 − α)B12 ]∗ −1 ∗ ∗ ≤ αA12 A−1 22 A12 + (1 − α)B12 B22 B12 .

Hence, −[αA22 +(1 − α)B22 ]|[αA + (1 − α)B] = [αA12 + (1 − α)B12 ][αA22 + (1 − α)B22 ]−1[αA12 + (1 − α)B12 ]∗ − [αA11 + (1 − α)B11 ]   −1 ∗ ∗ ≤ α A12 A−1 22 A12 − A11 + (1 − α)(B12 B22 B12 − B11 ) = α(−A22|A) + (1 − α)(−B22|B), 

which shows that φ(A) = −A22|A is convex. By continuity, the result holds for A ∈ Nn+m. To prove xviii), note that φ(A) = (A22|A)−1 = φ2[φ1(A)], where φ1(A) = A22|A and φ2 (B) = B −1. It follows from xv) that φ1 is concave, while it follows from iv) that φ2 is convex. Furthermore, x) of Proposition 8.6.13 implies that φ2 is  nonincreasing. It thus follows from Lemma 8.6.16 that φ(A) = (A22|A)−1 is convex. Result xix) is given in [243, p. 106]. Result xx) is given in by Theorem 9 of [931]. To prove xxi), note that φ(A) = −(tr A−r )−p = φ2[φ1(A)], where φ1(A) = tr A−r and φ2 (B) = −B −p. Statement iii) implies that φ1 is convex and that φ2 is concave. Furthermore, ix) of Proposition 8.6.13 implies that φ2 is nondecreasing. It thus follows from iv) of Lemma 8.6.16 that φ(A) = −(tr A−r )−p is convex. Results xxii) and xxiii) are proved in [294]. Results xxiv)–xxviii) are given by Corollary 1.1, Theorem 1, Corollary 2.1, Theorem 2, and Theorem 8, respectively, of [294]. A proof of xxiv) in the case p = 1 − r is given in [201, p. 273].

POSITIVE-SEMIDEFINITE MATRICES

485

Result xxix) is proved in [201, p. 274] and [294]. Result xxx) is given in [205, p. 113]. Result xxxi) is given in [201, p. 123], [205, p. 113], and [543]. To prove xxxii), note that φ(A) = − log tr Ar = φ2[φ1(A)], where φ1(A) = tr A and φ2 (x) = − log x. Statement vi) implies that φ1 is concave. Furthermore, φ2 is convex and nonincreasing. It thus follows from ii) of Lemma 8.6.16 that φ(A) = − log tr Ar is convex. r

Result xxxiii) is given in [1051]. Result xxxiv) is given in [201, p. 275]. Result xxxv) is given in [56]. To prove xxxvi), let A1, A2 ∈ Nn. From Corollary 8.4.15 it follows that (det A1 )1/n + (det A2 )1/n ≤ [det(A1 + A2 )]1/n. Replacing A1 and A2 by αA1 and (1 − α)A2 , respectively, and multiplying by −1 shows that φ(A) = −(det A)1/n is convex. Result xxxvii) is proved in [1051]. Result xxxviii) is a special case of result xxxvii). This result is due to Fan.

See [360] or [361, p. 679]. To prove xxxviii), note that φ(A) = −nlog (det A)1/n = φ2[φ1(A)], where φ1(A) = (det A)1/n and φ2 (x) = −nlog x. It follows from xix) that φ1 is concave. Since φ2 is nonincreasing and convex, it follows from ii) of Lemma 8.6.16 that φ(A) = − log det A is convex. To prove xxxix), note that φ(A) = det A−1 = φ2[φ1(A)], where φ1(A) = log det A−1 and φ2 (x) = ex. It follows from xx) that φ1 is convex. Since φ2 is nondecreasing and convex, it follows from i) of Lemma 8.6.16 that φ(A) = det A−1 is convex. Results xl) and xli) are given in [360] and [361, pp. 684, 685]. Next, xlii) is given in [201, p. 273], [205, p. 114], and [1521, p. 9]. Statement xliii) is given in [205, p. 114]. Statement xliv) is given in [1521, p. 9]. Finally, xlv) is given in [996, p. 478]. Statement xlvi) follows immediately from xlv). The following result is a corollary of xvii) of Proposition 8.6.17 for the case α = 1/2. Versions of this result appear in [298, 676, 922, 947] and [1125, p. 152].

486

CHAPTER 8

    A A12  B11 n+m ∗ ∗ ∈ F Corollary 8.6.18. Let A = A11 and B = A B12 22 12 and assume that A and B are positive semidefinite. Then,

B12 B22

A11|A + B11|B ≤ (A11 + B11 )|(A + B).



∈ Fn+m, (8.6.12)

The following corollary of xlv) and xlvi) of Proposition 8.6.17 gives a strong majorization condition for the eigenvalues of a pair of Hermitian matrices. Corollary 8.6.19. Let A, B ∈ Hn. Then, for all k ∈ {1, . . . , n}, k  i=1

n 

λi(A) +

λi(B) ≤

i=n−k+1

k 

λi(A + B) ≤

i=1

k 

[λi(A) + λi(B)]

(8.6.13)

i=1

with equality in both inequalities for k = n. Furthermore, for all k ∈ {1, . . . , n}, n 

[λi(A) + λi(B)] ≤

i=k

n 

λi(A + B)

(8.6.14)

i=k

with equality for k = 1. Proof. The lower bound in (8.6.13) is given in [1208, p. 116]. See also [201, p. 69], [328], [730, p. 201], or [996, p. 478]. Equality in Corollary 8.6.19 is discussed in [328].

8.7 Facts on Range and Rank Fact 8.7.1. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, there exists α > 0 such that A ≤ αB if and only if R(A) ⊆ R(B). In this case, rank A ≤ rank B. Proof: Use Theorem 8.6.2 and Corollary 8.6.11. Fact 8.7.2. Let A, B ∈ Fn×n. Then, R(A) + R(B) = R[(AA∗ + BB ∗ )1/2 ]. Proof: This result follows from Fact 2.11.1 and Theorem 2.4.3. Remark: See [42]. Fact 8.7.3. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, (A + B)(A + B)+ is the projector onto the subspace R(A) + R(B) = span[R(A) ∪ R(B)]. Proof: Use Fact 2.9.13 and Fact 8.7.5. Remark: See Fact 6.4.50.

POSITIVE-SEMIDEFINITE MATRICES

487

Fact 8.7.4. Let A ∈ Fn×n, and assume that A + A∗ ≥ 0. Then, the following statements hold: i) N(A) = N(A + A∗ ) ∩ N(A − A∗ ). ii) R(A) = R(A + A∗ ) + R(A − A∗ ).

iii) rank A = rank A + A∗ A − A∗ . Proof: Statements i) and ii) follow from Fact 8.7.5, while statement iii) follows from Fact 8.7.6. Fact 8.7.5. Let A, B ∈ Fn×n, assume that A is positive semidefinite, and assume that B is either positive semidefinite or skew Hermitian. Then, the following statements hold: i) N(A + B) = N(A) ∩ N(B). ii) R(A + B) = R(A) + R(B). Proof: Use [(N(A) ∩ N(B)]⊥ = R(A) + R(B). Fact 8.7.6. Let A, B ∈ Fn×n, assume that A is positive semidefinite, and assume that B is either positive semidefinite or skew Hermitian. Then,  

A rank(A + B) = rank A B = rank B If, in addition, B is positive semidefinite, then     A A+B A B = rank A + rank(A + B). = rank rank 0 A 0 A Proof: Using Fact 8.7.5, R



A B



  

A   =R A B = R A2 + BB ∗ B∗   = R A2 + R(BB ∗ ) = R(A) + R(B) = R(A + B).

Alternatively, for the case in which B is positive semidefinite, it follows from Fact 6.5.6 that



rank A B = rank A + B B = rank(A + B) + rank[B − (A + B)(A + B)+B]. Next, note that

 

rank[B − (A + B)(A + B)+B] = rank B 1/2 I − (A + B)(A + B)+ B 1/2  

≤ rank B 1/2 I − BB + B 1/2 = 0.

Fact 8.7.7. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then,     A A+B A B = rank A + rank(A + B). = rank rank 0 A 0 A Proof: Use Theorem 8.3.5 to simultaneously diagonalize A and B.

488

CHAPTER 8

Fact 8.7.8. Let A ∈ Fn×n, and let S ⊆ {1, . . . , n}. If A is either positive semidefinite or an irreducible, singular M-matrix, then the following statements hold: i) If α ⊂ {1, . . . , n}, then rank A ≤ rank A(α) + rank A(α∼ ) . ii) If α, β ⊆ {1, . . . , n}, then rank A(α∪β) ≤ rank A(α) + rank A(β) − rank A(α∩β) . iii) If 1 ≤ k ≤ n − 1, then   det A(α) ≤ (n − k) det A(α) . k {α: card(α)=k+1}

{α: card(α)=k}

If, in addition, A is either positive definite, a nonsingular M-matrix, or totally positive, then all three inclusions hold as equalities. Proof: See [963]. Remark: See Fact 8.13.37. Remark: Totally positive means that every subdeterminant of A is positive. See Fact 11.18.23.

8.8 Facts on Structured Positive-Semidefinite Matrices Fact 8.8.1. Let φ: R → C, and assume that, for all x1, . . . , xn ∈ R, the matrix  A ∈ Cn×n, where A(i,j) = φ(xi − xj ), is positive semidefinite. (The function φ is positive semidefinite.) Then, the following statements hold: i) For all x1, x2 ∈ R, it follows that |φ(x1 ) − φ(x2 )|2 ≤ 2φ(0)Re[φ(0) − φ(x1 − x2 )]. 

ii) The function ψ : R → C, where, for all x ∈ R, ψ(x) = φ(x), is positive semidefinite. 

iii) For all α ∈ R, the function ψ : R → C, where, for all x ∈ R, ψ(x) = φ(αx), is positive semidefinite. 

iv) The function ψ : R → C, where, for all x ∈ R, ψ(x) = |φ(x)|, is positive semidefinite. 

v) The function ψ : R → C, where, for all x ∈ R, ψ(x) = Re φ(x), is positive semidefinite. vi) If φ1 : R → C and φ2 : R → C are positive semidefinite, then φ3 : R → C,  where, for all x ∈ R, φ3 (x) = φ1 (x)φ2 (x), is positive semidefinite. vii) If φ1 : R → C and φ2 : R → C are positive semidefinite and α1, α2 are  positive numbers, then φ3 : R → C, where, for all x ∈ R, φ3 (x) = α1 φ1 (x)+ α2 φ2 (x), is positive semidefinite.

489

POSITIVE-SEMIDEFINITE MATRICES

viii) Let φ : R → C, and assume that φ is bounded and continuous. Further more, for all x, y ∈ R, define K: R × R → C by K(x, y) = φ(x − y). Then, φ is positive semidefinite if and only if, for every continuous integrable function f : R → C, it follows that R2

K(x, y)f (x)f (y) dx dy ≥ 0.

Proof: See [205, pp. 141–144]. Remark: The function K is a kernel function associated with a reproducing kernel space. See [560] for extensions to vector arguments. For applications, see [1205] and Fact 8.8.4. Fact 8.8.2. Let a1, . . . , an be positive numbers, and define A ∈ Rn×n by either of the following expressions: 

i) A(i,j) = min{ai , aj }. 

ii) A(i,j) =

1 . max{ai ,aj }

 ai

iii) A(i,j) =

aj ,

where a1 ≤ · · · ≤ an .

 iv) A(i,j) =

p ap i −aj ai −aj ,

where p ∈ [0, 1].

 v) A(i,j) =

p ap i +aj ai +aj ,

where p ∈ [−1, 1].

 log ai −log aj

vi) A(i,j) =

ai −aj

.

Then, A is positive semidefinite. If, in addition, α is a positive number, then A◦α is positive semidefinite. Proof: See [203], [205, pp. 153, 178, 189], and [432, p. 90]. Remark: The matrix A in iii) is the Schur product of the matrices defined in i) and ii). Fact 8.8.3. Let a1 < · · · < an be positive numbers, and define A ∈ Rn×n by  A(i,j) = min{ai , aj }. Then, A is positive definite, det A =

n !

(ai − ai−1 ),

i=1

and, for all x ∈ Rn, xTA−1x =

n  [x(i) − x(i−1) ]2 i=1



ai − ai−1

,



where a0 = 0 and x0 = 0. Remark: The matrix A is a covariance matrix arising in the theory of Brownian motion. See [691, p. 132] and [1489, p. 50].

490

CHAPTER 8

Fact 8.8.4. Let a1, . . . , an ∈ R, and define A ∈ Cn×n by either of the following expressions: i) A(i,j) =



1 1+j(ai −aj ) .

 ii) A(i,j) =

1 1−j(ai −aj ) .



1 1+(ai −aj )2 .



1 1+|ai −aj | .

iii) A(i,j) = iv) A(i,j) =

 j(ai −aj ) v) A(i,j) = e . 

vi) A(i,j) = cos(ai − aj ).  sin[(ai −aj )]

vii) A(i,j) =

ai −aj

.

viii) A(i,j) =



ai −aj sinh[(ai −aj )] .

 ix) A(i,j) =

sinh p(ai −aj ) sinh(ai −aj ) ,

 x) A(i,j) =

tanh[(ai −aj )] . ai −aj

where p ∈ (0, 1).



sinh[(ai −aj )] (ai −aj )[cosh(ai −aj )+p] ,



1 cosh(ai −aj )+p ,

xi) A(i,j) = xii) A(i,j) =  xiii) A(i,j) = 

cosh p(ai −aj ) cosh(ai −aj ) ,

where p ∈ (−1, 1].

where p ∈ (−1, 1].

where p ∈ [−1, 1].

xiv) A(i,j) = e−(ai −aj ) . 2

 −|ai −aj | e , where p ∈ [0, 2]. xv) A(i,j) = p

 xvi) A(i,j) =  xvii) A(i,j) =

1 1+|ai −aj | . 1+p(ai −aj )2 1+q(ai −aj )2 ,

where 0 ≤ p ≤ q.



xviii) A(i,j) = tr eB+j(ai −aj )C , where B, C ∈ Cn×n are Hermitian and commute. Then, A is positive semidefinite. Finally, if, α is a nonnegative number and A is defined by either ix), x), xi), xiii), xvi), or xvii), then A◦α is positive semidefinite. Proof: See [205, pp. 141–144, 153, 177, 188], [220], [432, p. 90], and [728, pp. 400, 401, 456, 457, 462, 463]. Remark: In each case, A is associated with a positive-semidefinite function. See Fact 8.8.1. Remark: xv) is related to the Bessis-Moussa-Villani conjecture. See Fact 8.12.31 and Fact 8.12.32. Problem: In each case, determine rank A and determine when A is positive definite.

491

POSITIVE-SEMIDEFINITE MATRICES

Fact 8.8.5. Define A ∈ Rn×n by either of the following expressions:    i+j i) A(i,j) = i .  (i + j)!. ii) A(i,j) = 

iii) A(i,j) = min{i, j}. 

iv) A(i,j) = gcd{i, j}. 

v) A(i,j) = ji . Then, A is positive semidefinite. If, in addition, α is a nonnegative number, then A◦α is positive semidefinite. Remark: Fact 8.22.2 guarantees the weaker result that A◦α is positive semidefinite for all α ∈ [0, n − 2]. Remark: i) is the Pascal matrix. See [5, 203, 460]. The fact that A is positive semidefinite follows from the equality   min{i,j}   i  j  i+j = . i k k k=0

Remark: The matrix defined in v), which is a special case of iii) of Fact 8.8.2, is the Lehmer matrix. Remark: The determinant of A defined in iv) can be expressed in terms of the Euler totient function. See [69, 257]. Fact 8.8.6. Let a1, . . . , an ≥ 0 and p ∈ R, assume that either a1, . . . , an are positive or p is positive, and, for all i, j ∈ {1, . . . , n}, define A ∈ Rn×n by 

A(i,j) = (ai aj )p. Then, A is positive semidefinite.

T   Proof: Let a = a1 · · · an and A = a◦pa◦pT. Fact 8.8.7. Let a1, . . . , an > 0, let α > 0, and, for all i, j ∈ {1, . . . , n}, define A ∈ Rn×n by 1  A(i,j) = . (ai + aj )α Then, A is positive semidefinite. Proof: See [203], [205, pp. 24, 25], or [1119]. Remark: See Fact 5.11.12. Remark: For α = 1, A is a Cauchy matrix. See Fact 3.22.9. Fact 8.8.8. Let a1 , . . . , an > 0, let r ∈ [−1, 1], and, for all i, j ∈ {1, . . . , n}, define A ∈ Rn×n by r r  a i + aj . A(i,j) = a i + aj Then, A is positive semidefinite.

492

CHAPTER 8

Proof: See [1521, p. 74]. Fact 8.8.9. Let a1 , . . . , an > 0, let q > 0, let p ∈ [−q, q], and, for all i, j ∈ {1, . . . , n}, define A ∈ Rn×n by A(i,j)

api + apj = q . ai + aqj 

Then, A is positive semidefinite. Proof: Let r = p/q and bi = aqi . Then, A(i,j) = (bri + brj )/(bi + bj ). Now, use Fact 8.8.8. See [1004] for the case q ≥ p ≥ 0. Remark: The case q = 1 and p = 0 yields a Cauchy matrix. In the case n = 2, A ≥ 0 yields Fact 1.12.33. Problem: When is A positive definite? Fact 8.8.10. Let a1, . . . , an > 0, let p ∈ (−2, 2], and define A ∈ Rn×n by  A(i,j) =

a2i

1 . + pai aj + a2j

Then, A is positive semidefinite. Proof: See [208]. Fact 8.8.11. Let a1, . . . , an > 0, let p ∈ (−1, ∞), and define A ∈ Rn×n by 

A(i,j) =

a3i

+

p(a2i aj

1 . + ai a2j ) + a3j

Then, A is positive semidefinite. Proof: See [208]. Fact 8.8.12. Let a1, . . . , an > 0, p ∈ [−1, 1], q ∈ (−2, 2], and, for all i, j ∈ {1, . . . , n}, define A ∈ Rn×n by 

A(i,j) =

api + apj . a2i + qai aj + a2j

Then, A is positive semidefinite. Proof: See [1518] or [1521, p. 76]. Fact 8.8.13. Let A ∈ Fn×n, and assume that A is Hermitian, A(i,i) > 0 for all i ∈ {1, . . . , n}, and, for all i, j ∈ {1, . . . , n}, + 1 |A(i,j) | < n−1 A(i,i) A(j,j) . Then, A is positive definite. Proof: Note that

 ∗ n  n−1   x(i) x Ax = x(j) ∗

i=1 j=i+1

1 n−1 A(i,i)

A(i,j)

A(i,j)

1 n−1 A(j,j)



x(i) x(j)

 .

493

POSITIVE-SEMIDEFINITE MATRICES

Remark: This result is due to Roup. Fact 8.8.14. Let A ∈ Rn×n, assume that A is positive semidefinite, assume that A(i,i) > 0 for all i ∈ {1, . . . , n}, and define B ∈ Rn×n by 

B(i,j) =

A(i,j) , μα (A(i,i) , A(j,j) )

where, for positive scalars α, x, y, 

μα (x, y) =

1

α 2 (x

1/α + yα) .

Then, B is positive semidefinite. If, in addition, A is positive definite, then B is positive definite. In particular, letting α ↓ 0, α = 1, and α → ∞, respectively, the matrices C,D, E ∈ Rn×n defined by A(i,j)   , C(i,j) = A(i,i) A(j,j)  D(i,j) =



E(i,j) =

2A(i,j) , A(i,i) + A(j,j)

A(i,j) max{A(i,i) , A(j,j) }

are positive semidefinite. Finally, if A is positive definite, then C, D, and E are positive definite. Proof: See [1180]. Remark: The assumption that all of the diagonal entries of A are positive can be weakened. See [1180]. Remark: See Fact 1.12.34. Problem: Extend this result to Hermitian matrices. Fact 8.8.15. Let α, β, γ ∈ [0, π], and define A ∈ R3×3 by ⎤ ⎡ 1 cos α cos γ 1 cos β ⎦. A = ⎣ cos α cos γ cos β 1 Then, A is positive semidefinite if and only if the following conditions are satisfied: i) α ≤ β + γ. ii) β ≤ α + γ. iii) γ ≤ α + β. iv) α + β + γ ≤ 2π. Furthermore, A is positive definite if and only if all of these inequalities are strict. Proof: See [153].

494

CHAPTER 8

Fact 8.8.16. Let λ1, . . . , λn ∈ C, assume that, for all i ∈ {1, . . . , n}, Re λi < 0, and, for all i, j ∈ {1, . . . , n}, define A ∈ Cn×n by 

A(i,j) =

−1 . λi + λj

Then, A is positive definite. Proof: Note that A = 2B ◦ (1n×n − C)◦−1, where B(i,j) = (λi +1)(λj +1) . (λi −1)(λj −1)

1 (λi −1)(λj −1)

and C(i,j) =

Then, note that B is positive semidefinite and that (1n×n − C)◦−1 =

1n×n + C + C ◦2 + C ◦3 + · · · . ) Remark: A is the solution of a Lyapunov equation. See Fact 12.21.18 and Fact 12.21.19. Remark: A is a Cauchy matrix. See Fact 3.18.4, Fact 3.22.9, and Fact 3.22.10. Remark: A Cauchy matrix is also a Gram matrix defined in terms of the inner product of the functions fi (t) = e−λi t. See [205, p. 3]. Fact 8.8.17. Let λ1, . . . , λn ∈ OUD, and let w1, . . . , wn ∈ C. Then, there exists a holomorphic function φ: OUD → OUD such that φ(λi ) = wi for all i ∈ {1, . . . , n} if and only if A ∈ Cn×n is positive semidefinite, where, for all i, j ∈ {1, . . . , n},  1 − wi wj A(i,j) = . 1 − λi λj Proof: See [1010]. Remark: A is a Pick matrix.

by

Fact 8.8.18. Let α0 , . . . , αn > 0, and define the tridiagonal matrix A ∈ Rn×n ⎤ ⎡ α0 + α1 −α1 0 0 ··· 0 ⎥ ⎢ −α1 α1 + α2 −α2 0 ··· 0 ⎥ ⎢ ⎥ ⎢  0 −α2 α2 + α3 −α3 · · · 0 A=⎢ ⎥. ⎥ ⎢ .. .. .. .. . .. ⎦ ⎣ · · · . . . . . . 0

0

0

0

···

αn−1 + αn

Then, A is positive definite. Proof:  For k = 2, . . . , n, the k × k leading principal subdeterminant of A is given k −1 by α0 α1 · · · αk . See [150, p. 115]. i=0 αi Remark: A is a stiffness matrix arising in structural analysis. Remark: See Fact 3.19.3.

495

POSITIVE-SEMIDEFINITE MATRICES

8.9 Facts on Identities and Inequalities for One Matrix Fact 8.9.1. Let n ≤ 3, let A ∈ Fn×n, and assume that A is positive semidefinite. Then, |A| is positive semidefinite. Proof: See [989]. Remark: |A| denotes the matrix whose entries are the absolute values of the entries of A. Remark: This result does not hold for n ≥ 4. Let ⎤ ⎡ √1 1 0 − √13 3 ⎥ ⎢ √1 √1 1 0 ⎥ ⎢ 3 3 ⎥. ⎢ A=⎢ ⎥ √1 √1 0 ⎣ 0 3 3 ⎦ √1 − √13 0 1 3 √ √ √ √ Then, mspec(A) √ = {1 −√ 6/3, 1 − 6/3, 1 + 6/3, 1 + 6/3}ms, whereas mspec(|A|) = {1, 1, 1 − 12/3, 1 + 12/3}ms . Fact 8.9.2. Let x ∈ Fn. Then, xx∗ ≤ x∗xI.  x∗xI − xx∗. Fact 8.9.3. Let x ∈ Fn, assume that x is nonzero, and define A = ∗ ∗ Then, A is positive semidefinite, mspec(A) = {x x, . . . , x x, 0}ms , and rank A = n − 1.

Fact 8.9.4. Let x, y ∈ Fn, assume that x and y are linearly independent, and  define A = (x∗x + y ∗ y)I − xx∗ − yy ∗. Then, A is positive definite. Now, let F = R. Then, mspec(A) = {xTx + y Ty, . . . , xTx + y Ty, + T T 1 1 T T 2 T 2 (x x + y y) + 2 4 (x x − y y) + (x y) , + T T 1 1 T T 2 T 2 2 (x x + y y) − 4 (x x − y y) + (x y) }ms .  x∗xI − xx∗ Proof: To show that A is positive definite, write A = B + C, where B =  and C = y ∗ yI − yy ∗. Then, using Fact 8.9.3 it follows that N(B) = span {x} and N(C) = span {y}. Now, it follows from Fact 8.7.5 that N(A) = N(B) ∩ N(C) = {0}. Therefore, A is nonsingular and thus positive definite. The expression for mspec(A) follows from Fact 4.9.17.

Fact 8.9.5. Let x1, . . . , xn ∈ R3, assume that span {x1, . . . , xn } = R3, and  n T define A = i=1 (xT i xi I − xi xi ). Then, A is positive definite. Furthermore, λ1(A) < λ2 (A) + λ3 (A) and d1(A) < d2 (A) + d3 (A).

496

CHAPTER 8

 Proof: Suppose that d1 (A) = A(1,1) . Then, d2 (A)+d3 (A)−d1 (A) = 2 ni=1 x2i(3) > n 0. Now, let S ∈ R3×3 be such that SAS T = i=1 (ˆ xT ˆi I − x ˆi x ˆT i x i ) is diagonal, where,  for i = 1, . . . , n, x ˆi = Sxi . Then, for i = 1, 2, 3, di (A) = λi (A). Remark: A is the inertia matrix for a rigid body consisting of n discrete particles. For a homogeneous continuum body B whose density is ρ, the inertia matrix is given by I =ρ (rTrI − rrT ) dxdy dz, 

where r =

x y z

B

.

Remark: The eigenvalues and diagonal entries of A represent the lengths of the sides of triangles. See Fact 1.13.17 and [1096, p. 220]. Fact 8.9.6. Let A ∈ F2×2 , assume that A is positive semidefinite and nonzero, and define B ∈ F2×2 by  −1/2  √ √  A + det AI . tr A + 2 det A B= Then, B = A1/2. Proof: See [644, pp. 84, 266, 267]. Fact 8.9.7. Let A ∈ Fn×n, and assume that A is Hermitian. Then, rank A = ν−(A) + ν+(A) and def A = ν0 (A). Fact 8.9.8. Let A ∈ Fn×n, assume that A is positive semidefinite, and assume there exists i ∈ {1, . . . , n} such that A(i,i) = 0. Then, rowi(A) = 0 and coli(A) = 0.

A(i,i)

Fact 8.9.9. Let A ∈ Fn×n, and assume that A is positive semidefinite. Then, ≥ 0 for all i ∈ {1, . . . , n}, and |A(i,j) |2 ≤ A(i,i) A(j,j) for all i, j ∈ {1, . . . , n}. Fact 8.9.10. Let A ∈ Fn×n. Then, A ≥ 0 if and only if A ≥ −A. Fact 8.9.11. Let A ∈ Fn×n, and assume that A is Hermitian. Then, A2 ≥ 0.

Fact 8.9.12. Let A ∈ Fn×n, and assume that A is skew Hermitian. Then, A ≤ 0. 2

Fact 8.9.13. Let A ∈ Fn×n, and let α > 0. Then, A2 + A2∗ ≤ αAA∗ + α1 A∗A. Equality holds if and only if αA = A∗. Fact 8.9.14. Let A ∈ Fn×n. Then, (A − A∗ )2 ≤ 0 ≤ (A + A∗ )2 ≤ 2(AA∗ + A∗A).

497

POSITIVE-SEMIDEFINITE MATRICES

Fact 8.9.15. Let A ∈ Fn×n, and let α > 0. Then, A + A∗ ≤ αI + α−1AA∗. Equality holds if and only if A = αI. Fact 8.9.16. Let A ∈ Fn×n, and assume that A is positive definite. Then, 2I ≤ A + A−1. Equality holds if and only if A = I. Furthermore, 2n ≤ tr A + tr A−1. Fact 8.9.17. Let A ∈ Fn×n, and assume that A is positive definite. Then, −1  11×n A−11n×1 1n×n ≤ A. Proof: Set B = 1n×n in Fact 8.22.14. See [1528]. A

n×n , and assume that A is positive definite. Then, Fact

8.9.18. Let A ∈ F is positive semidefinite.

I I A−1

Fact 8.9.19. Let A ∈ Fn×n, and assume that A is Hermitian. Then, A2 ≤ A if and only if 0 ≤ A ≤ I. Fact 8.9.20. Let A ∈ Fn×n, and assume that A is Hermitian. Then, αI + A ≥ 0 if and only if α ≥ −λmin(A). Furthermore, A2 + A + 14I ≥ 0. Fact 8.9.21. Let A ∈ Fn×m. Then, AA∗ ≤ In if and only if A∗A ≤ Im . Fact 8.9.22. Let A ∈ Fn×n, and assume that either AA∗ ≤ A∗A or A∗A ≤ AA∗. Then, A is normal. Proof: Use ii) of Corollary 8.4.10. Fact 8.9.23. Let A ∈ Fn×n, and assume that A is a projector. Then, 0 ≤ A ≤ I. Therefore,

0 ≤ I ◦ A ≤ I.

Fact 8.9.24. Let A ∈ Fn×n, assume that A is (semisimple, Hermitian), and assume that there exists a nonnegative integer k such that Ak = Ak+1. Then, A is (idempotent, a projector). Fact 8.9.25. Let A ∈ Fn×n, and assume that A is nonsingular. Then, @ −1 A = A∗ −1. A Fact 8.9.26. Let A ∈ Fn×m, and assume that A∗A is nonsingular. Then, A∗  = AA−1/2A∗.

498

CHAPTER 8

Fact 8.9.27. Let A ∈ Fn×n. Then, A is unitary if and only if there exists a nonsingular matrix B ∈ Fn×n such that A = B ∗ −1/2B. If, in addition, A is real, then det B = sign(det A). Proof: For necessity, set B = A. Remark: See Fact 3.11.31. Fact 8.9.28. Let A ∈ Fn×n. Then, A is normal if and only if A = A∗ . Remark: See Fact 3.7.12. Fact 8.9.29. Let A ∈ Fn×n. Then, −A − A∗  ≤ A + A∗ ≤ A + A∗ . Proof: See [911]. Fact 8.9.30. Let A ∈ Fn×n, assume that A is normal, and let α, β ∈ (0, ∞). Then, −αA − βA∗  ≤ αA + βA∗  ≤ αA + βA∗ . In particular,

−A − A∗  ≤ A + A∗  ≤ A + A∗ .

Proof: See [911, 1530]. Remark: See Fact 8.11.11. Fact 8.9.31. Let A ∈ Fn×n. The following statements hold: i) If A ∈ Fn×n is positive definite, then I + A is nonsingular and the matrices  I − B and I + B are positive definite, where B = (I + A)−1(I − A). ii) If I + A is nonsingular and the matrices I − B and I + B are positive  (I + A)−1(I − A), then A is positive definite. definite, where B = Proof: See [476]. Remark: For additional results on the Cayley transform, see Fact 3.11.21, Fact 3.11.22, Fact 3.11.23, Fact 3.20.12, and Fact 11.21.9. 1 (A − A∗ ) is positive definite. Fact 8.9.32. Let A ∈ Fn×n, and assume that j2 Then,

−1/2  1 ∗ 1/2 −1 ∗ 1 B= A A 2 (A + A∗ ) 2 (A + A )

is unitary. Proof: See [479]. 1 (A − A∗ ) is negative definite. A is strictly Remark: A is strictly dissipative if j2 dissipative if and only if −jA is dissipative. See [477, 478].

Remark: A−1A∗ is similar to a unitary matrix. See Fact 3.11.5. Remark: See Fact 8.13.11 and Fact 8.18.12.

499

POSITIVE-SEMIDEFINITE MATRICES

Fact 8.9.33. Let A ∈ Rn×n, assume that A is positive definite, assume that  A ≤ I, and define the sequence (Bk )∞ k=0 by B0 = 0 and    Bk+1 = Bk + 12 A − Bk2 . Then,

lim Bk = A1/2.

k→∞

Proof: See [174, p. 181]. Remark: See Fact 5.15.21. Fact 8.9.34. Let A ∈ Rn×n, assume that A is nonsingular, and define the  sequence (Bk )∞ k=0 by B0 = A and    Bk+1 = 12 Bk + Bk−T . −1/2  A. lim Bk = AAT

Then,

k→∞

Remark: The limit is a unitary matrix. See Fact 8.9.27. See [148, p. 224]. Fact 8.9.35. Let a, b ∈ R, and define the symmetric, Toeplitz matrix A ∈ Rn×n by  A= aIn + b1n×n . Then, A is positive definite if and only if a + nb > 0 and a > 0. Remark: See Fact 2.13.12 and Fact 4.10.16. Fact 8.9.36. Let x1, . . . , xn ∈ Rm , and define  1

x=

n 

n

xj ,

j=1

 1

S=

n 

n

(xj − x)(xj − x)T.

j=1

Then, for all i ∈ {1, . . . , n}, (xi − x)(xi − x)T ≤ (n − 1)S. Furthermore, equality holds if and only if all of the elements of {x1, . . . , xn }\{xi } are equal. Proof: See [776, 1070, 1364]. Remark: This result is an extension of the Laguerre-Samuelson inequality. See Fact 1.17.12. 

Fact 8.9.37. Let x1, . . . , xn ∈ Fn, and define A ∈ Fn×n by A(i,j) = x∗i xj for

 x1 · · · xn . Then, A = B ∗B. Consequently, all i, j ∈ {1, . . . , n}, and B = A is positive semidefinite and rank A = rank B. Conversely, let A ∈ Fn×n, and n assume that A is positive semidefinite. Then, there exist x1, . . . , xn ∈ F such that A = B ∗B, where B = x1 · · · xn . Proof: The converse is an immediate consequence of Corollary 5.4.5. Remark: A is the Gram matrix of x1, . . . , xn .

500

CHAPTER 8

Fact 8.9.38. Let A ∈ Fn×n, and assume that A is positive semidefinite. Then, there exists a matrix B ∈ Fn×n such that B is lower triangular, B has nonnegative diagonal entries, and A = BB ∗. If, in addition, A is positive definite, then B is unique and has positive diagonal entries. Remark: This result is the Cholesky decomposition. Fact 8.9.39. Let A ∈ Fn×m, and assume that rank A = m. Then, 0 ≤ A(A∗A)−1A∗ ≤ I. Fact 8.9.40. Let A ∈ Fn×m. Then, I − A∗A is positive definite if and only if I − AA∗ is positive definite. In this case, (I − A∗A)−1 = I + A∗(I − AA∗ )−1A. 

Fact 8.9.41. Let A ∈ Fn×m, let α be a positive number, and define Aα = (αI + A∗A)−1A∗. Then, the following statements are equivalent: i) AAα = Aα A. ii) AA∗ = A∗A. Furthermore, the following statements are equivalent: iii) Aα A∗ = A∗Aα . iv) AA∗A2 = A2A∗A. Proof: See [1331]. Remark: Aα is a regularized Tikhonov inverse. Fact 8.9.42. Let A ∈ Fn×n, and assume that A is positive definite. Then, A−1 ≤ 

1 (α + β)2 −1 α+β I− A≤ A , αβ αβ 4αβ



where α = λmax(A) and β = λmin(A). Proof: See [997]. Fact 8.9.43. Let A ∈ Fn×n, and assume that A is positive semidefinite. Then, the following statements hold: i) If α ∈ [0, 1], then

Aα ≤ αA + (1 − α)I.

ii) If α ∈ [0, 1] and A is positive definite, then [αA−1 + (1 − α)I]−1 ≤ Aα ≤ αA + (1 − α)I. iii) If α ≥ 1, then

αA + (1 − α)I ≤ Aα.

iv) If A is positive definite and either α ≥ 1 or α ≤ 0, then αA + (1 − α)I ≤ Aα ≤ [αA−1 + (1 − α)I]−1.

POSITIVE-SEMIDEFINITE MATRICES

501

Proof: See [544, pp. 122, 123]. Remark: This result is a special case of the Young inequality. See Fact 1.11.2 and Fact 8.10.46. Remark: See Fact 8.12.27 and Fact 8.12.28. Fact 8.9.44. Let A ∈ Fn×n, and assume that A is positive definite. Then, I − A−1 ≤ log A ≤ A − I. Furthermore, if A ≥ I, then log A is positive semidefinite, and, if A > I, then log A is positive definite. Proof: See Fact 1.11.22.

8.10 Facts on Identities and Inequalities for Two or More Matrices n ∞ n Fact 8.10.1. Let (Ai )∞ i=1 ⊂ H and (Bi )i=1 ⊂ H , assume that, for all i ∈ P,   Ai ≤ Bi , and assume that A = limi→∞ Ai and B = limi→∞ Bi exist. Then, A ≤ B.

Fact 8.10.2. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and assume that A ≤ B. Then, R(A) ⊆ R(B) and rank A ≤ rank B. Furthermore, R(A) = R(B) if and only if rank A = rank B. Fact 8.10.3. Let A, B ∈ Fn×n, and assume that A and B are Hermitian. Then, the following statements hold: i) λmin(A) ≤ λmin(B) if and only if λmin (A)I ≤ B. ii) λmax (A) ≤ λmax (B) if and only if A ≤ λmax (B)I. Fact 8.10.4. Let A, B ∈ Fn×n, assume that A and B are Hermitian, and consider the following conditions: i) A ≤ B. ii) For all i ∈ {1, . . . , n}, λi(A) ≤ λi(B). iii) There exists a unitary matrix S ∈ Fn×n such that A ≤ SBS ∗. Then, i) =⇒ ii) ⇐⇒ iii). Remark: i) =⇒ ii) is the monotonicity theorem given by Theorem 8.4.9. that Fact 8.10.5. Let A, B ∈ Fn×n, and assume   A and B are positive semidefinite. Then, 0 ≤ A < B if and only if sprad AB −1 < 1. Fact 8.10.6. Let A, B ∈ Fn×n, and assume that A and B are positive definite. Then,  −1 −1 A + B −1 = A(A + B)−1B.

502

CHAPTER 8

Fact 8.10.7. Let A, B ∈ Fn×n, and assume that A and B are positive definite. Then, (A + B)−1 ≤ 14 (A−1 + B −1 ). Equivalently,

A + B ≤ AB −1A + BA−1B.

In both inequalities, equality holds if and only if A = B. Proof: See [1526, p. 168]. Remark: See Fact 1.12.4. Fact 8.10.8. Let A, B ∈ Fn×n, and assume that A is positive definite, B is Hermitian, and A + B is nonsingular. Then, (A + B)−1 + (A + B)−1B(A + B)−1 ≤ A−1. If, in addition, B is nonsingular, then the inequality is strict. Proof: This inequality is equivalent to BA−1B ≥ 0. See [1077]. Fact 8.10.9. Let A, B ∈ Fn×n, assume that A and B are positive definite, and let α ∈ [0, 1]. Then, β[αA−1 + (1 − α)B −1 ] ≤ [αA + (1 − α)B]−1, where

 β=

min

μ∈mspec(A−1B)

4μ . (1 + μ)2

Proof: See [1042]. Remark: This result is a reverse form of an inequality based on convexity. Fact 8.10.10. Let A ∈ Fn×m and B ∈ Fm×m, and assume that B is positive semidefinite. Then, ABA∗ = 0 if and only if AB = 0. Fact 8.10.11. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, AB is positive semidefinite if and only if AB is normal. Fact 8.10.12. Let A, B ∈ Fn×n, assume that A and B are Hermitian, and assume that either i) A and B are positive semidefinite or ii) either A or B is positive definite. Then, AB is group invertible. Proof: Use Theorem 8.3.2 and Theorem 8.3.6. Fact 8.10.13. Let A, B ∈ Fn×n, assume that A and B are Hermitian, and assume that A and AB + BA are (positive semidefinite, positive definite). Then, B is (positive semidefinite, positive definite). Proof: See [205, p. 8], [903, p. 120], or [1464]. Alternatively, the result follows from Corollary 11.9.4.

POSITIVE-SEMIDEFINITE MATRICES

503

Fact 8.10.14. Let A, B, C ∈ Fn×n, assume that A, B, and C are positive semidefinite, and assume that A = B + C. Then, the following statements are equivalent: i) rank A = rank B + rank C. ii) There exists S ∈ Fm×n such that rank S = m, R(S) ∩ N(A) = {0}, and either B = AS ∗ (SAS ∗ )−1SA or C = AS ∗ (SAS ∗ )−1SA. Proof: See [293, 339]. Fact 8.10.15. Let A, B ∈ Fn×n, and assume that A and B are Hermitian and nonsingular. Then, the following statements hold: i) If every eigenvalue of AB is positive, then In A = In B. ii) In A − In B = In(A − B) + In(A−1 − B −1 ). iii) If In A = In B and A ≤ B, then B −1 ≤ A−1. Proof: See [53, 112, 1074]. Remark: The equality ii) is due to Styan. See [1074]. Remark: An extension to singular A and B is given by Fact 8.21.14. Fact 8.10.16. Let A, B ∈ Fn×n, assume that A and B are Hermitian, and assume that A ≤ B. Then, A(i,i) ≤ B(i,i) for all i ∈ {1, . . . , n}. Fact 8.10.17. Let A, B ∈ Fn×n, assume that A and B are Hermitian, and assume that A ≤ B. Then, sig A ≤ sig B. Proof: See [400, p. 148]. Fact 8.10.18. Let A, B ∈ Fn×n, assume that A and B are Hermitian, and assume that A ≤ B. Then, either A ≤ B or −A ≤ B. Proof: See [1529]. Fact 8.10.19. Let A, B ∈ Fn×n, and assume that A is positive semidefinite and B is positive definite. Then, A ≤ B if and only if AB −1A ≤ A. Fact 8.10.20. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and assume that A ≤ B. Then, there exists a matrix S ∈ Fn×n such that A = S ∗BS and S ∗S ≤ I. Proof: See [459, p. 269]. Fact 8.10.21. Let A, B, C, D ∈ Fn×n, assume that A, B, C, D are positive semidefinite, and assume that 0 < D ≤ C and BCB ≤ ADA. Then, B ≤ A. Proof: See [87, 308].

504

CHAPTER 8

Fact 8.10.22. Let A, B ∈ Fn×n, and assume that A and B are positive definite. Then, there exists a unitary matrix S ∈ Fn×n such that AB ≤ 12 S(A2 + B 2 )S ∗. Proof: See [93, 213]. Fact 8.10.23. Let A, B ∈ Fn×n, and assume that A and B are projectors. Then, ABA ≤ B if and only if AB = BA. Proof: See [1357]. Fact 8.10.24. Let A, B ∈ Fn×n, and assume that A is positive definite, 0 ≤ A ≤ I, and B is positive definite. Then, ABA ≤ 

(α + β)2 B. 4αβ



where α = λmin (B) and β = λmax (B). Proof: See [255]. Remark: This inequality is related to Fact 1.18.6. Fact 8.10.25. Let A, B ∈ Fn×n, and assume that A and B are projectors. Then, (A + B)1/2 ≤ A1/2 + B 1/2 if and only if AB = BA. Proof: See [1349, p. 30]. Fact 8.10.26. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and assume that 0 ≤ A ≤ B. Then,  1/2  1/2 A + 14A2 ≤ B + 14B 2 . Proof: See [1037]. and let Fact 8.10.27. Let A ∈ Fn×n, assume that A is positive   semidefinite, B ∈ Fl×n. Then, BAB ∗ is positive definite if and only if B A + A2 B ∗ is positive definite. Proof: Diagonalize A using a unitary transformation and note that BA1/2 and  1/2 B A + A2 have the same rank. Fact 8.10.28. Let A, B, C ∈ Fn×n, assume that A is positive definite, and assume that B and C are positive semidefinite. Then, 2tr B 1/2 C 1/2  ≤ tr(AB + A−1 C). Furthermore, there exists A such that equality holds if and only if rank B = rank C = rank B 1/2 C 1/2. Proof: See [37, 507]. Remark: A matrix A for which equality holds is given in [37].

505

POSITIVE-SEMIDEFINITE MATRICES

Remark: Applications to linear systems are given in [1476]. Fact 8.10.29. Let A, B ∈ Fn×n, assume that A and B are positive definite, let S ∈ Fn×n be such that SAS ∗ = diag(α1, . . . , αn ) and SBS ∗ = diag(β1, . . . , βn ), and define  Cl = S −1 diag(min{α1, β1 }, . . . , min{αn , βn })S −∗ and

 S −1 diag(max{α1, β1 }, . . . , max{αn , βn })S −∗. Cu =

Then, Cl and Cu are independent of the choice of S, and Cl ≤ A ≤ Cu , Cl ≤ B ≤ Cu . Proof: See [926]. Fact 8.10.30. Let A, B ∈ Hn×n. Then, glb{A, B} exists in Hn with respect to the ordering “≤” if and only if either A ≤ B or B ≤ A. Proof: See [806]. B = [ 00 01 ]. Then, C = 0 is a lower bound for {A, B}. Remark: Let A = [ 10 00 ] and √  √ √ −1 2 , which has eigenvalues −1 − 2 and −1 + 2, is also a Furthermore, D = √ 2 −1

lower bound for {A, B} but is not comparable with C.

Fact 8.10.31. Let A, B ∈ Hn×n, and assume that A and B are positive semidefinite. Then, the following statements hold: i) {A, B} does not necessarily have a least upper bound in Nn. ii) If A and B are positive definite, then {A, B} has a greatest lower bound in Nn if and only if A and B are comparable. iii) If A is a projector and 0 ≤ B ≤ I, then {A, B} has a greatest lower bound in Nn. iv) If A, B ∈ Nn are projectors, then the greatest lower bound of {A, B} in Nn is given by glb{A, B} = 2A(A + B)+ B, which is the projector onto R(A) ∩ R(B). v) glb{A, B} exists in Nn if and only if glb{A, glb{AA+, BB + }} and glb{B, glb{AA+, BB + }} are comparable. In this case, glb{A, B} = min{glb{A, glb{AA+, BB + }}, glb{B, glb{AA+, BB + }}}. vi) glb{A, B} exists if and only if sh(A, B) and sh(B, A) are comparable, where  sh(A, B) = limα→∞ (αB) : A. In this case, glb{A, B} = min{sh(A, B), sh(B, A)}. Proof: To prove i), let A = [ 10 00 ] and B = [ 00 01 ] , and suppose that Z is the least upper bound for A and A ≤ Z ≤ I and B ≤ Z ≤ I, and thus Z = I.   B. Hence,  4/3 2/3 Next, note that X = satisfies A ≤ X and B ≤ X. However, it is not true 2/3 4/3

506

CHAPTER 8

that Z ≤ X, which implies that {A, B} does not have a least upper bound. See [243, p. 11]. Statement ii) is given in [451, 564, 1047]. Statements iii) and v) are given in [1047]. Statement iv) is given in [41]. The expression for the projector onto R(A) ∩ R(B) is given in Fact 6.4.46. Statement vi) is given in [52]. Remark: The partially ordered cones Hn and Nn with the ordering “≤” are not lattices. Remark: sh(A, B) is the shorted operator, see Fact 8.21.19. However, the usage here is more general since B need not be a projector. See [52]. Remark: An alternative approach to showing that Nn is not a lattice is given in [926]. Remark: The cone Nn is a partially ordered set under the spectral order, see Fact 8.10.35. Fact 8.10.32. Let A1, . . . , Ak ∈ Fn×n, and assume that A1, . . . , Ak are positive definite. Then,  k −1 k   2 n Ai ≤ A−1 i . i=1

i=1

Remark: This result is an extension of Fact 1.17.38. Fact 8.10.33. Let A1, . . . , Ak ∈ Fn×n, assume that A1, . . . , Ak are positive semidefinite, and let p, q ∈ R satisfy 1 ≤ p ≤ q. Then,  k 1/p  k 1/q  p  q 1 Ai ≤ k1 Ai . k i=1

i=1

Proof: See [197]. Fact 8.10.34. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, let p be a real number, and assume that either p ∈ [1, 2] or A and B are positive definite and p ∈ [−1, 0] ∪ [1, 2]. Then, [ 12 (A + B)]p ≤ 12 (Ap + B p ). Proof: See [879]. Fact 8.10.35. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let p, q ∈ R satisfy p ≥ q ≥ 1. Then, 1 q



1/p q 1/q ≤ 12 (Ap + B p ) . 2 (A + B ) Furthermore,



μ(A, B) = lim

p→∞

exists and satisfies

A ≤ μ(A, B),

1

p 2 (A

1/p + Bp)

B ≤ μ(A, B).

Proof: See [175]. Remark: μ(A, B) is the least upper bound of A and B with respect to the spectral order. See [56, 817] and Fact 8.20.3.

507

POSITIVE-SEMIDEFINITE MATRICES

Remark: The result does not hold for p = 1 and q = 1/3. A counterexample is 3 8 10 A = [ 21 12 ] = [ 13 8 5 ], B = [ 0 0 ]. Fact 8.10.36. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, let p ∈ (1, ∞), and let α ∈ [0, 1]. Then, 1/p

α1−1/pA + (1 − α)1−1/pB ≤ (Ap + B p )

.

Proof: See [56]. Fact 8.10.37. Let A, B, C ∈ Fn×n. Then, A∗A + B ∗B = (B + CA)∗ (I + CC ∗ )−1(B + CA) + (A − C ∗B)(I + C ∗C)−1(A − C ∗B). Proof: See [736]. Remark: See Fact 8.13.30. Fact 8.10.38. Let A ∈ Fn×n, let α ∈ R, and assume that either A is nonsingular or α ≥ 1. Then, (A∗A)α = A∗(AA∗ )α−1A. Proof: Use the singular value decomposition. Remark: This result is given in [525, 540]. Fact 8.10.39. Let A, B ∈ Fn×n, let α ∈ R, assume that A and B are positive semidefinite, and assume that either A and B are positive definite or α ≥ 1. Then, (AB 2A)α = AB(BA2B)α−1BA. Proof: Use Fact 8.10.38. Fact 8.10.40. Let A, B, C ∈ Fn×n, assume that A is positive semidefinite, B is positive definite, and B = C ∗C, and let α ∈ [0, 1]. Then,  α C ∗ C −∗AC −1 C ≤ αA + (1 − α)B. If, in addition, α ∈ (0, 1), then equality holds if and only if A = B. Proof: See [1020]. Fact 8.10.41. Let A, B ∈ Fn×n, assume that A is positive semidefinite, and let p ∈ R. Furthermore, assume that either A and B are nonsingular or p ≥ 1. Then, (BAB ∗ )p = BA1/2 (A1/2B ∗BA1/2 )p−1A1/2B ∗. Proof: See [540] or [544, p. 129]. Fact 8.10.42. Let A, B ∈ Fn×n, assume that A and B are positive definite, and let p ∈ R. Then, (BAB)p = BA1/2 (A1/2B 2A1/2 )p−1A1/2B. Proof: See [538, 692].

508

CHAPTER 8

Fact 8.10.43. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Furthermore, if A is positive definite, then define  1/2  A#B = A1/2 A−1/2BA−1/2 A1/2, whereas, if A is singular, then define  lim(A + εI)#B. A#B =

ε↓0

Then, the following statements hold: i) A#B is positive semidefinite. ii) A#A = A. iii) A#B = B#A. iv) R(A#B) = R(A) ∩ R(B). v) If S ∈ Fm×n is right invertible, then (SAS ∗ )#(SBS ∗ ) ≤ S(A#B)S ∗. vi) If S ∈ Fn×n is nonsingular, then (SAS ∗ )#(SBS ∗ ) = S(A#B)S ∗. vii) If C, D ∈ Pn, A ≤ C, and B ≤ D, then A#B ≤ C#D. viii) If C, D ∈ Pn, then (A#C) + (B#D) ≤ (A + B)#(C + D). ix) If A ≤ B, then 4A#(B − A) = [A + A#(4B − 3A)]#[−A + A#(4B − 3A)]. x) If α ∈ [0, 1], then √

α(A#B) ±

A xi) A#B = max{X ∈ H: [ X

X B

1 2

√ 1 − α(A − B) ≤ 12 (A + B).

] is positive semidefinite}.

xii) Let X ∈ Fn×n, and assume that X is Hermitian and   A X ≥ 0. X B Then,  Furthermore,

A A#B A#B B



−A#B ≤ X ≤ A#B.   A −A#B and −A#B are positive semidefinite. B

xiii) If S ∈ Fn×n is unitary and A1/2SB 1/2 is positive semidefinite, then A#B = A1/2SB 1/2. Now, assume in addition that A is positive definite. Then, the following statements hold: xiv) (A#B)A−1(A#B) = B.  1/2 xv) For all α ∈ R, A#B = A1−α Aα−1BA−α Aα.  1/2 xvi) A#B = A A−1B = (BA−1 )1/2A.

509

POSITIVE-SEMIDEFINITE MATRICES



1/2 xvii) A#B = (A + B) (A + B)−1A(A + B)−1B . Now, assume in addition that A and B are positive definite. Then, the following statements hold: xviii) A#B is positive definite.  −1/2 −1/2 1/2 1/2 −1/2  A BA A B is unitary, and A#B = A1/2SB 1/2. xix) S =  xx) det A#B = (det A) det B. xxi) det (A#B)2 = det AB. xxii) (A#B)−1 = A−1 #B −1.    −1 −1 xxiii) Let A0 = A and B0 = B, and, for all k ∈ N, define Ak+1 = 2(A−1 k + Bk )  1 and Bk+1 = 2 (Ak + Bk ). Then, for all k ∈ N,

Ak ≤ Ak+1 ≤ A#B ≤ Bk+1 ≤ Bk and lim Ak = lim Bk = A#B. k→∞  A αA#B is positive definite. xxiv) For all α ∈ (−1, 1), αA#B B     A A#B A −A#B xxv) rank A#B = rank −A#B = n. B B 

k→∞

Furthermore, the following statements hold: √ √   det A and β = det B. Then, xxvi) Assume that n = 2, and let α = √ αβ A#B =  (α−1A + β −1B). det(α−1A + β −1B) 

xxvii) If 0 < A ≤ B, then φ: [0, ∞) → Pn defined by φ(p) = A−p #B p is nondecreasing. xxviii) If B is positive definite and A ≤ B, then A2 #B −2 ≤ A#B −1 ≤ I. xxix) If A and B are positive semidefinite and A ≤ B, then (BA2B)1/2 ≤ B 1/2 (B 1/2AB 1/2 )1/2B 1/2 ≤ B 2. Finally, let X ∈ Hn. Then, the following statements are equivalent: A xxx) [ X

X B

] is positive semidefinite.

−1

xxxi) XA X ≤ B. xxxii) XB −1X ≤ A. xxxiii) −A#B ≤ X ≤ A#B. Proof: See [47, 499, 597, 902, 1346]. For xiii), xix), and xxvi), see [205, pp. 108, 109, 111]. For xxvii), see [48]. Statement xxvii) implies xxviii), which, in turn, implies xxix).

510

CHAPTER 8

Remark: The square roots in xvi) indicate a semisimple matrix with positive diagonal entries. Remark: A#B is the geometric mean of A and B. A related mean is defined in [499]. Alternative means and their differences are considered in [22]. Geometric means for an arbitrary number of positive-definite matrices are discussed in [59, 832, 1039, 1111]. Remark: Inverse problems are considered in [43]. Remark: xxix) interpolates (8.6.6). Remark: Compare statements xiii) and xix) with Fact 8.11.6. Remark: See Fact 10.10.4 and Fact 12.23.4. Problem: For singular A and B, express A#B in terms of generalized inverses. Fact 8.10.44. Let A, B ∈ Fn×n, and assume that A and B are Hermitian. Then, the following statements are equivalent: i) A ≤ B. ii) For all t ≥ 0, I ≤ e−tA #etB. 

iii) φ: [0, ∞) → Pn defined by φ(t) = e−tA #etB is nondecreasing. Proof: See [48]. Fact 8.10.45. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let α ∈ [0, 1]. Furthermore, if A is positive definite, then define  α  A#α B = A1/2 A−1/2BA−1/2 A1/2, whereas, if A is singular, then define  lim(A + εI)#α B. A#α B =

ε↓0

Then, the following statements hold: i) A#α B = B#1−α A. ii) (A#α B)−1 = A−1 #α B −1. Fact 8.10.46. Let A, B ∈ Fn×n, assume that A and B are positive definite, and let α ∈ [0, 1]. Then,  1−α −1

−1 αA + (1 − α)B −1 ≤ A1/2 A−1/2BA−1/2 A1/2 ≤ αA + (1 − α)B, or, equivalently,

−1 −1 αA + (1 − α)B −1 ≤ A#1−α B ≤ αA + (1 − α)B, or, equivalently,

 α−1 A−1/2 ≤ αA−1 + (1 − α)B −1. [αA + (1 − α)B]−1 ≤ A−1/2 A−1/2BA−1/2

511

POSITIVE-SEMIDEFINITE MATRICES

Consequently,

  α−1 

≤ tr αA−1 + (1 − α)B −1 . tr [αA + (1 − α)B]−1 ≤ tr A−1 A−1/2BA−1/2

Remark: The left-hand inequality in the first string of inequalities is the Young inequality. See [544, p. 122] and Fact 1.12.21. Setting B = I yields Fact 8.9.43. The third string of inequalities interpolates the fact that φ(A) = A−1 is convex as shown by iv) of Proposition 8.6.17. Remark: Related inequalities are given by Fact 8.12.27 and Fact 8.12.28. See also Fact 8.21.18. Fact 8.10.47. Let A, B ∈ Fn×n, and assume that A and B are positive definite. Then, −1 −1  −1 2 2αβ A−1 + B −1 , + B −1 ≤ A#B ≤ 12 (A + B) ≤ (α+β) (α+β)2 (A + B) ≤ 2 A 2αβ where



α = min{λmin(A), λmin(B)} and

 max{λmax (A), λmax (B)}. β=

Proof: Use the second string of inequalities of Fact 8.10.46 with α = 1/2 along with results given in [1315] and [1526, p. 174]. ∞ n ∞ Fact 8.10.48. Let (xi )∞ i=1 ⊂ R , assume that i=1 xi exists, and let (Ai )i=1 n ⊂ N be such that Ai ≤ Ai+1 for all i ∈ P and limi→∞ tr Ai = ∞. Then, lim (tr Ak )−1

k→∞

k 

Aixi = 0.

i=1

If, in addition Ai is positive definite for all i ∈ P and {λmax(Ai )/λmin(Ai )}∞ i=1 is bounded, then k  −1 lim Ak Aixi = 0. k→∞

i=1

Proof: See [35]. Remark: These equalities are matrix versions of the Kronecker lemma. Remark: Extensions are given in [638]. Fact 8.10.49. Let A, B ∈ Fn×n, assume that A and B are positive definite, assume that A ≤ B, and let p ≥ 1. Then, p−1  λmax(A) p p A ≤ K(λmin(A), λmin(A), p)B ≤ B p, λmin (A) where

 p (p − 1)(ap − bp ) ap b − abp K(a, b, p) = . (p − 1)(a − b) p(ap b − abp ) 

Proof: See [253, 542] and [544, pp. 193, 194]. Remark: K(a, b, p) is the Fan constant.

512

CHAPTER 8

Fact 8.10.50. Let A, B ∈ Fn×n, assume that A is positive definite and B is positive semidefinite, and let p ≥ 1. Then, there exist unitary matrices U, V ∈ Fn×n such that p ∗ 1 K(λmin(A),λmin(A),p) U (BAB) U

≤ B pApB p ≤ K(λmin(A), λmin(A), p)V (BAB)p V ∗,

where K(a, b, p) is the Fan constant defined in Fact 8.10.49. Proof: See [253]. Remark: See Fact 8.12.22, Fact 8.19.27, and Fact 9.9.17. Fact 8.10.51. Let A, B ∈ Fn×n, assume that A is positive definite, B is positive semidefinite, and B ≤ A, and let p ≥ 1 and r ≥ 1. Then,   r 1/p Ar/2 A−1/2B pA−1/2 Ar/2 ≤ Ar. In particular,

C2/p B ≤ A2. A−1/2B pA1/2

Proof: See [55]. Fact 8.10.52. Let A, B ∈ Fn×n, and assume that A and B are positive definite. Then, the following statements are equivalent: i) B ≤ A. ii) For all r ∈ [0, ∞), p ∈ [1, ∞), and k ∈ N such that (k + 1)(r + 1) = p + r, 1 k+1  . B r+1 ≤ B r/2ApB r/2 iii) For all r ∈ [0, ∞), p ∈ [1, ∞), and k ∈ N such that (k + 1)(r + 1) = p + r, 1 k+1  ≤ Ar+1. Ar/2B pAr/2 Proof: See [940]. Remark: See Fact 8.20.1. Fact 8.10.53. Let A, B ∈ Fn×n, and assume that A is positive definite and B is positive semidefinite. Then, the following statements are equivalent: i) B ≤ A. ii) For all p, q, r, t ∈ R such that p ≥ 1, r ≥ 0, t ≥ 0, and q ∈ [1, 2], r+t+1   q  r+qt+qp Ar/2 At/2B pAt/2 Ar/2 ≤ Ar+t+1. iii) For all p, q, r, τ ∈ R such that p ≥ 1, r ≥ τ , q ≥ 1, and τ ∈ [0, 1], r−τ   q  r−qτ +qp Ar/2 A−τ/2B pA−τ/2 Ar/2 ≤ Ar−τ . iv) For all p, q, r, τ ∈ R be such that p ≥ 1, r ≥ τ , τ ∈ [0, 1], and q ≥ 1, r−τ +1   q  r−qτ +qp Ar/2 A−τ/2B pA−τ/2 Ar/2 ≤ Ar−τ +1.

POSITIVE-SEMIDEFINITE MATRICES

513

In particular, if B ≤ A, p ≥ 1, and r ≥ 1, then   r  r−1 r/2 −1/2 p −1/2 r/2 pr A A BA A ≤ Ar−1. Proof: Condition ii) is given in [525], iii) appears in [545], and iv) appears in [540]. See also [525, 526] and [544, p. 133]. Remark: Setting q = r and τ = 1 in iv) yields Fact 8.10.51. Remark: Condition iv), which is the generalized Furuta inequality, interpolates Proposition 8.6.7 and Fact 8.10.51. Fact 8.10.54. Let A, B ∈ Fn×n, and assume that A is positive definite, B is positive semidefinite, and B ≤ A. Furthermore, let t ∈ [0, 1], p ≥ 1, r ≥ t, and s ≥ 1, and define 1−t+r   (p−t)s+r  FA,B (r, s) = A−r/2 Ar/2 (A−t/2B pA−t/2 )sAr/2 A−r/2. Then, that is,

FA,B (r, s) ≤ FA,A (r, s), 1−t+r  (p−t)s+r  ≤ A1−t+r. Ar/2 (A−t/2B pA−t/2 )sAr/2

Furthermore, if r ≥ r and s ≥ s, then FA,B (r , s ) ≤ FA,B (r, s). Proof: See [540] and [544, p. 143]. Remark: This result extends iv) of Fact 8.10.53. Fact 8.10.55. Each of the following functions φ: (0, ∞) → (0, ∞) yields an increasing function φ: Pn → Pn : i) φ(x) =

xp+1/2 x2p +1 ,

where p ∈ [0, 1/2].

ii) φ(x) = x(1 + x) log(1 + 1/x). iii) φ(x) =

1 (1+x) log(1+1/x) .

iv) φ(x) =

x−1−log x (log x)2 .

v) φ(x) =

x(log x)2 x−1−log x .

vi) φ(x) =

x(x+2) log(x+2) . (x+1)2

vii) φ(x) =

x(x+1) (x+2) log(x+2) .

viii) φ(x) =

(x2 −1) log(1+x) . x2

ix) φ(x) = x) φ(x) =

x(x−1) (x+1) log(x+1) . (x−1)2 (x+1) log x .

514 xi) φ(x) = xii) φ(x) = xiii) φ(x) = xiv) φ(x) = xv) φ(x) =

CHAPTER 8 p−1 p



xp −1 xp−1 −1



, where p ∈ [−1, 2].

x−1 log x .

√

x.

2x x+1 . x−1 xp −1 ,

where p ∈ (0, 1].

Proof: See [548, 1111]. To obtain xii), xiii), and xiv), set p = 1, 1/2, −1, respectively, in xi). Fact 8.10.56. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite, A ≤ B, and AB = BA. Then, A2 ≤ B 2. Proof: See [113].

8.11 Facts on Identities and Inequalities for Partitioned Matrices Fact 8.11.1. Let A ∈ Fn×n, and assume that A is positive semidefinite. Then, the following statements hold: A −A A i) [ A are positive semidefinite. A A ] and −A A     αβ αA βA ii) If β γ ∈ F2×2 is positive semidefinite, then βA γA is positive semidefinite.     αβ αA βA iii) If A and β γ are positive definite, then βA γA is positive definite. Proof: Use Fact 7.4.17.

that BA∗ B Fact 8.11.2. Let A ∈ Fn×n, B ∈ Fn×m, C ∈ Fm×m C ∈   , assume αβ F(n+m)×(n+m) is positive semidefinite, and assume that β γ ∈ F2×2 is positive semidefinite. Then, the following statements hold:   α1n×n β1n×m is positive semidefinite. i) β1m×n γ1m×m   αA βB ii) βB ∗ γC is positive semidefinite.  

αA βB iii) If BA∗ B is is positive definite and α and γ are positive, then ∗ C βB γC positive definite. Proof: To prove i), use Proposition 8.2.4. Statements ii) and iii) follow from Fact 8.22.12. Fact 8.11.3. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite,   A11 A12 and assume that A and B are partitioned identically as A = A∗12 A22 and B =

515

POSITIVE-SEMIDEFINITE MATRICES



B11 B12 ∗ B12 B22

 . Then, A22 |A + B22 |B ≤ (A22 + B22 )|(A + B).

Now, assume in addition that A22 and B22 are positive definite. Then, equality −1 holds if and only if A12 A−1 22 = B12 B22 . Proof: See [498, 1084]. Remark: The first inequality, which follows from xvii) of Proposition 8.6.17, is an extension of Bergstrom’s inequality, which corresponds to the case in which A11 is a scalar. See Fact 8.15.19. Fact 8.11.4. Let A, B ∈ Fn×n, assume that A and B are positive semidef  A11 A12 inite, assume that A and B are partitioned identically as A = A∗12 and A22   11 B12 ∗ B= B B12 B22 , and assume that A11 and B11 are positive definite. Then, −1 ∗ (A12 + B12 )∗ (A11 + B11 )−1 (A12 + B12 ) ≤ A∗12 A−1 11 A12 + B12 B11 B12

and −1 ∗ ∗ −1 rank[A∗12 A−1 11 A12 + B12 B11 B12 − (A12 + B12 ) (A11 + B11 ) (A12 + B12 )] −1 B12 ). = rank(A12 − A11B11

Furthermore, det A det B det(A + B) = det[(A11 + B11 )|(A + B)]. + ≤ det A11 det B11 det(A11 + B11 ) Remark: The last inequality generalizes Fact 8.13.18.

 Fact 8.11.5. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, and define A =

. Then, the following statements hold:

A B B∗ C

i) If A is positive semidefinite, then 0 ≤ BC +B ∗ ≤ A. ii) If A is positive definite, then C is positive definite and 0 ≤ BC −1B ∗ < A. Now, assume in addition that n = m. Then, the following statements hold: iii) If A is positive semidefinite, then −A − C ≤ B + B ∗ ≤ A + C. iv) If A is positive definite, then −A − C < B + B ∗ < A + C. Proof: The first two statements follow from Proposition 8.2.4. To prove the last



  two statements, consider SAS T, where S = I I and S = I −I . Remark: See Fact 8.22.42.

516

CHAPTER 8 

Fact 8.11.6. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, and define A =

. Then, A is positive semidefinite if and only if A and C are positive semidefinite and there exists a semicontractive matrix S ∈ Fn×m such that



A B B∗ C

B = A1/2SC 1/2. Proof: See [738]. Remark: Compare this result with statements xiii) and xix) of Fact 8.10.43. Fact 8.11.7. Let A, B, C ∈ Fn×n, assume that semidefinite, and assume that AB = BA. Then,



A B B∗ C

∈ F2n×2n is positive

B ∗B ≤ A1/2CA1/2. Proof: See [1528]. Fact 8.11.8. Let A, B ∈ Fn×n, and assume that A and B are Hermitian. A B ] is positive semidefinite. Furthermore, Then, −A ≤ B ≤ A if and only if [ B A A B −A < B < A if and only if [ B A ] is positive definite. Proof: Note that   A I −I √1 2 I B I

B A



 √1 2

I −I

I I



 =

A−B 0

0 A+B

 .

Fact 8.11.9. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, assume that  is positive semidefinite, and let r = rank B. Then, for all k ∈ {1, . . . , r}, k ! i=1

σi (B) ≤

k !



A B B∗ C

max{λi (A), λi (C)}.

i=1

Proof: See[1528]. 

Fact 8.11.10. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, define A = and assume that A is positive definite. Then, tr A−1 + tr C −1 ≤ tr A−1. Furthermore, B is nonzero if and only if tr A−1 + tr C −1 < tr A−1. Proof: Use Proposition 8.2.5 or see [1020]. Fact 8.11.11. Let A ∈ Fn×m, and define   A∗  A  . A= A∗ A Then, A is positive semidefinite. If, in addition, n = m, then −A∗  − A ≤ A + A∗ ≤ A∗  + A. Proof: Use Fact 8.11.5. Remark: See Fact 8.9.30 and Fact 8.21.4.



A B B∗ C

,

517

POSITIVE-SEMIDEFINITE MATRICES

Fact 8.11.12. Let A ∈ Fn×n, assume that A is normal, and define   A A  . A= A∗ A Then, A is positive semidefinite. Proof: See [730, p. 213]. Fact 8.11.13. Let A ∈ Fn×n, and define   I A  . A= A∗ I Then, A is (positive semidefinite, positive definite) if and only if A is (semicontractive, contractive). Fact 8.11.14. Let A ∈ Fn×m and B ∈ Fn×l, and define   A∗A A∗B  . A= B ∗A B ∗B Then, A is positive semidefinite, and 0 ≤ A∗B(B ∗B)+B ∗A ≤ A∗A. If m = l, then

−A∗A − B ∗B ≤ A∗B + B ∗A ≤ A∗A + B ∗B.

If, in addition, m = l = 1 and B ∗B = 0, then |A∗B|2 ≤ A∗AB ∗B. Remark: This result is the Cauchy-Schwarz inequality. See Fact 8.13.23. Remark: See Fact 8.22.43. Fact 8.11.15. Let A, B ∈ Fn×m, and define   I + A∗A I − A∗B  A= I − B ∗A I + B ∗B 

and 

B=

I + A∗A

I + A∗B

I + B ∗A I + B ∗B

 .

Then, A and B are positive semidefinite, 0 ≤ (I − A∗B)(I + B ∗B)−1(I − B ∗A) ≤ I + A∗A, and

0 ≤ (I + A∗B)(I + B ∗B)−1(I + B ∗A) ≤ I + A∗A.

Remark: See Fact 8.13.26. Fact 8.11.16. Let A, B ∈ Fn×m. Then, I + AA∗ = (A + B)(I + B ∗B)−1 (A + B)∗ + (I − AB ∗ )(I + BB ∗ )−1 (I − BA∗ ).

518

CHAPTER 8

Therefore, (A + B)(I + B ∗B)−1 (A + B)∗ ≤ I + AA∗. Proof: Set C = A in Fact 2.16.23. See also [1526, p. 185]. Fact 8.11.17. Let A ∈ Fn×n and B ∈ Fn×m, assume that A is positive   semidefinite, and define A AB  . A= B ∗A B ∗AB 

Then, A=

A1/2 B ∗A1/2



A1/2

A1/2B ,

and thus A is positive semidefinite. Furthermore, 0 ≤ AB(B ∗AB)+B ∗A ≤ A. Now, assume in addition that n = m. Then, −A − B ∗AB ≤ AB + B ∗A ≤ A + B ∗AB. Fact 8.11.18. Let A ∈ Fn×n and B ∈ Fn×m, assume that A is positive definite, and define   A B  . A= B ∗ B ∗A−1B 

Then, A=

A1/2 B ∗A−1/2



A1/2

A−1/2B ,

and thus A is positive semidefinite. Furthermore,  + 0 ≤ B B ∗A−1B B ∗ ≤ A. Furthermore, if rank B = m, then

rank A − B(B ∗A−1B)−1B ∗ = n − m. Now, assume in addition that n = m. Then, −A − B ∗A−1B ≤ B + B ∗ ≤ A + B ∗A−1B. Proof: Use Fact 8.11.5. Remark: See Fact 8.22.44. Remark: The matrix I − A−1/2B(B ∗A−1B)+B ∗A−1/2 is a projector. Fact 8.11.19. Let A ∈ Fn×n and B ∈ Fn×m, assume that A is positive definite, and define   B ∗B B ∗AB  A= . B ∗B B ∗A−1B 

Then, A=

B ∗A1/2 ∗ −1/2

BA



A1/2B

A−1/2B ,

519

POSITIVE-SEMIDEFINITE MATRICES

and thus A is positive semidefinite. Furthermore, 0 ≤ B ∗B(B ∗A−1B)+B ∗B ≤ B ∗AB. Now, assume in addition that n = m. Then, −B ∗AB − B ∗A−1B ≤ 2B ∗B ≤ B ∗AB + B ∗A−1B. Proof: Use Fact 8.11.5. Remark: See Fact 8.13.24 and Fact 8.22.44. Fact 8.11.20. Let A, B ∈ Fn×m, let α, β ∈ (0, ∞), and define   −1 (A + B)∗ β I + αA∗A  . A= A+B α−1I + βBB ∗ Then,

 A=  =

β −1/2I

α1/2A∗

β 1/2B

α−1/2I

αA∗A

A∗

A

α−1I



 

β −1/2I

β 1/2B ∗

α1/2A

α−1/2I

β −1I

B∗

B

βBB ∗

+



 ,

and thus A is positive semidefinite. Furthermore, (A + B)∗(α−1I + βBB ∗ )−1(A + B) ≤ β −1I + αA∗A. Now, assume in addition that n = m. Then,   − β −1/2 + α−1/2 I − αA∗A − βBB ∗ ≤ A + B + (A + B)∗   ≤ β −1/2 + α−1/2 I + αA∗A + βBB ∗. Remark: See Fact 8.13.27 and Fact 8.22.45. Fact 8.11.21. Let A, B ∈ Fn×m, and assume that I − A∗A and thus I − AA∗ are nonsingular. Then, I − B ∗B − (I − B ∗A)(I − A∗A)−1(I − A∗B) = −(A − B)∗ (I − AA∗ )−1 (A − B). Now, assume in addition that I − A∗A is positive definite. Then, I − B ∗B ≤ (I − B ∗A)(I − A∗A)−1(I − A∗B). Now, assume in addition that I − B ∗B is positive definite. Then, I − A∗B is nonsingular. Next, define   (I − A∗A)−1 (I − B ∗A)−1  . A= (I − A∗B)−1 (I − B ∗B)−1 Then, A is positive semidefinite. Finally, −(I − A∗A)−1 − (I − B ∗B)−1 ≤ (I − B ∗A)−1 + (I − A∗B)−1 ≤ (I − A∗A)−1 + (I − B ∗B)−1.

520

CHAPTER 8

Proof: For the first equality, set D = −B ∗ and C = −A∗, and replace B with −B in Fact 2.16.22. See [49, 1087]. The last statement follows from Fact 8.11.5. Remark: The equality is Hua’s matrix equality. This result does not assume that either I − A∗A or I − B ∗B is positive semidefinite. The inequality and Fact 8.13.26 constitute Hua’s inequalities. See [1087, 1502]. Remark: Extensions to the case in which I − A∗A is singular are considered in [1087]. Remark: See Fact 8.9.40 and Fact 8.13.26. Fact 8.11.22. Let A ∈ Fn×n be semicontractive, and define B ∈ F2n×2n by   A (I − AA∗ )1/2  . B= (I − A∗A)1/2 −A∗ Then, B is unitary. Remark: See [521, p. 180]. Fact 8.11.23. Let A ∈ Fn×m, and define B ∈ F(n+m)×(n+m) by   −A∗ (I + AA∗ )−1/2 (I + A∗A)−1/2  . B= (I + AA∗ )−1/2A (I + AA∗ )−1/2  ˜ I, ˜ where I˜ = Then, B is unitary and satisfies A∗ = IA diag(Im , −In ). Furthermore, det B = 1.

Remark: See [655]. F

Fact 8.11.24. Let A ∈ Fn×m, assume that A is contractive, and define B ∈ by   (I − A∗A)−1/2 A∗ (I − AA∗ )−1/2  . B= (I − AA∗ )−1/2A (I − AA∗ )−1/2

(n+m)×(n+m)

 ˜ = I, ˜ where I˜ = Then, B is Hermitian and satisfies A∗IA diag(Im , −In ). Furthermore, det B = 1.

Remark: See [655]. Fact 8.11.25. Let X ∈ Fn×m, and define U ∈ F(n+m)×(n+m) by   −X ∗ (I + XX ∗ )−1/2 (I + X ∗X)−1/2  . U= (I + XX ∗ )−1/2X (I + XX ∗)−1/2 Furthermore, let A ∈ Fn×n, B ∈ Fn×m, C ∈ Fm×n, D ∈ Fm×m. Then, the following statements hold: 

i) Assume that D is nonsingular, and let X = D−1 C. Then, ⎡ ⎤   ∗ −1/2 ∗ ∗ −1/2 (A − BX)(I + X X) (B + AX )(I + XX ) A B ⎦U. =⎣ C D 0 D(I + XX ∗ )1/2

521

POSITIVE-SEMIDEFINITE MATRICES 

ii) Assume that A is nonsingular, and let X = CA−1. Then, ⎡ ⎤   (I + X ∗X)1/2A (I + X ∗X)−1/2 (B + X ∗D) A B ⎦. = U⎣ C D 0 (I + XX ∗ )−1/2 (D − XB) Remark: See Proposition 2.8.3 and Proposition 2.8.4. Proof: See [655]. Fact 8.11.26. Let X ∈ Fn×m, and define U ∈ F(n+m)×(n+m) by   X ∗ (I − XX ∗ )−1/2 (I − X ∗X)−1/2  . U= (I − XX ∗ )−1/2X (I − XX ∗ )−1/2 Furthermore, let A ∈ Fn×n, B ∈ Fn×m, C ∈ Fm×n, D ∈ Fm×m. Then, the following statements hold:  i) Assume that D is nonsingular, let X = D−1 C, and assume that X ∗X < I. Then, ⎡ ⎤   ∗ −1/2 ∗ ∗ −1/2 (A − BX)(I − X X) (B + AX )(I − XX ) A B ⎦U. =⎣ C D 0 D(I − XX ∗ )1/2 

ii) Assume that A is nonsingular, let X = CA−1, and assume that X ∗X < I. Then, ⎡ ⎤   (I − X ∗X)1/2A (I − X ∗X)−1/2 (B − X ∗D) A B ⎦. = U⎣ C D 0 (I − XX ∗ )−1/2 (D − XB) Proof: See [655]. Remark: See Proposition 2.8.3 and Proposition 2.8.4. Fact 8.11.27. Let A, B ∈ Fn×m and C, D ∈ Fm×m, assume that C and D are positive definite, and define   AC −1A∗ + BD−1B ∗ A + B  A= . (A + B)∗ C +D Then, A is positive semidefinite, and (A + B)(C + D)−1(A + B)∗ ≤ AC −1A∗ + BD−1B ∗. Now, assume in addition that n = m. Then, −AC −1A∗ − BD−1B ∗ − C − D ≤ A + B + (A + B)∗ ≤ AC −1A∗ + BD−1B ∗ + C + D. Proof: See [676, 933] or [1125, p. 151]. Remark: Replacing A, B, C, D by αB1, (1 − α)B2 , αA1, (1 − α)A2 yields xiv) of Proposition 8.6.17.

522

CHAPTER 8

Fact 8.11.28. Let A ∈ Rn×n, assume that A is positive definite, and let S ⊆ {1, . . . , n}. Then,  −1  −1  A(S) ≤ A (S) . Proof: See [728, p. 474]. Remark: Generalizations of this result are given in [336]. Fact 8.11.29. Let Aij ∈ Fni ×nj for ⎡ A11 ⎢ ..  A=⎢ ⎣ . A1k

all i, j ∈ {1, . . . , k}, define ⎤ · · · A1k . .. ⎥ · ·. · . ⎥ ⎦, · · · Akk

and assume that A is square and positive definite. Furthermore, define ⎡ ⎤ Aˆ11 · · · Aˆ1k ⎢ . . .. ⎥  .. · · · Aˆ = ⎢ . . ⎥ ⎣ ⎦, Aˆ1k · · · Aˆkk where Aˆij = 11×ni Aij 1nj ×1 is the sum of the entries of Aij for all i, j ∈ {1, . . . , k}. Then, Aˆ is positive definite. Proof: Aˆ = BABT, where the entries of B ∈ Rk×

k

i=1

ni

are 0’s and 1’s. See [44].

Fact 8.11.30. Let A, D ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, and assume that

∈ Fn×n is positive semidefinite, C is positive definite, and D is positive

B is positive definite. definite. Then, A+D B∗ C

A B B∗ C

Fact 8.11.31. Let A ∈ F(n+m+l)×(n+m+l), assume that A is positive semidefinite, and assume that A is of the form ⎡ ⎤ A11 A12 0 A = ⎣ A∗12 A22 A23 ⎦. 0 A∗32 A33 Then, there exist positive-semidefinite matrices B, C ∈ F(n+m+l)×(n+m+l) such that A = B + C and such that B and C have the form ⎡ ⎤ B11 B12 0 ∗ B22 0 ⎦ B = ⎣ B12 0 0 0 and

Proof: See [687].

⎡

0 C =⎣ 0 0

0 C22 ∗ C23

⎤ 0 C23 ⎦. C33

523

POSITIVE-SEMIDEFINITE MATRICES

8.12 Facts on the Trace Fact 8.12.1. Let A ∈ Fn×n, assume that A is positive definite, let p and q be real numbers, and assume that p ≤ q. Then, 1  1/q  p 1/p ≤ n1 tr Aq . n tr A Furthermore, lim p↓0

1

 p 1/p n tr A

= det A1/n.

Proof: Use Fact 1.17.30. Fact 8.12.2. Let A ∈ Fn×n, and assume that A is positive definite. Then, n2 ≤ (tr A) tr A−1. Finally, equality holds if and only if A = In . Remark: Bounds on tr A−1 are given in [103, 315, 1079, 1160]. Fact 8.12.3. Let A ∈ Fn×n, and assume that A is positive semidefinite. Then, the following statements hold: i) Let r ∈ [0, 1]. Then, for all k ∈ {1, . . . , n}, n 

λri (A) ≤

i=k

n 

dri (A).

i=k

In particular, tr Ar ≤

n 

Ar(i,i).

i=1

ii) Let r ≥ 1. Then, for all k ∈ {1, . . . , n}, k 

dri (A) ≤

i=1

In particular,

n 

k 

λri (A).

i=1

Ar(i,i) ≤ tr Ar.

i=1

iii) If either r = 0 or r = 1, then tr Ar =

n 

Ar(i,i).

i=1

iv) If r = 0 and r = 1, then r

tr A =

n 

Ar(i,i)

i=1

if and only if A is diagonal. Proof: Use Fact 2.21.7 and Fact 8.18.8. See [971] and [973, p. 217]. Remark: See Fact 8.18.8.

524

CHAPTER 8

Fact 8.12.4. Let A ∈ Fn×n, and let p, q ∈ [0, ∞). Then, tr (A∗pAp ) ≤ tr (A∗A)pq . q

Furthermore, equality holds if and only if tr A∗pAp = tr (A∗A)p. Proof: See [1239]. Fact 8.12.5. Let A ∈ Fn×n, p ∈ [2, ∞), and q ∈ [1, ∞). Then, A is normal if and only if q tr (A∗pAp ) = tr (A∗A)pq . Proof: See [1239]. Fact 8.12.6. Let A, B ∈ Fn×n, and assume that either A and B are Hermitian or A and B are skew Hermitian. Then, tr AB is real. Proof: tr AB = tr A∗B ∗ = tr (BA)∗ = tr BA = tr AB. Remark: See [1511] or [1526, p. 213]. Fact 8.12.7. Let A, B ∈ Fn×n, assume that A and B are Hermitian, and let k ∈ N. Then, tr (AB)k is real. Proof: See [57]. Fact 8.12.8. Let A, B ∈ Fn×n, and assume that A and B are Hermitian. Then,  tr AB ≤ |tr AB| ≤ (tr A2 ) tr B 2 ≤ 12 tr(A2 + B 2 ). The second inequality is an equality if and only if A and B are linearly dependent. The third inequality is an equality if and only if tr A2 = tr B 2. All four terms are equal if and only if A = B. Proof: Use the Cauchy-Schwarz inequality given by Corollary 9.3.9. Remark: See Fact 8.12.20. Fact 8.12.9. Let A, B ∈ Fn×n, assume that A and B are Hermitian, and assume that −A ≤ B ≤ A. Then, tr B 2 ≤ tr A2. Proof: 0 ≤ tr[(A − B)(A + B)] = tr A2 − tr B 2. See [1350]. Remark: For 0 ≤ B ≤ A, this result is a special case of xxi) of Proposition 8.6.13. Fact 8.12.10. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, AB = 0 if and only if tr AB = 0.

525

POSITIVE-SEMIDEFINITE MATRICES

Fact 8.12.11. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let p, q ≥ 1 satisfy 1/p + 1/q = 1. Then, tr AB ≤ tr AB ≤ (tr Ap )1/p(tr B q )1/q . Furthermore, equality holds for both inequalities if and only if Ap−1 and B are linearly dependent. Proof: See [971] and [973, pp. 219, 222]. Remark: This result is a matrix version of H¨older’s inequality. Remark: See Fact 8.12.12 and Fact 8.12.19. Fact 8.12.12. Let A1, . . . , Am ∈ Fn×n, assume that A1, . . . , Am are positive semidefinite, and let p1, . . . , pm ∈ [1, ∞) satisfy p11 + · · · + p11 = 1. Then, tr A1 · · · Am  ≤

m m !  1 pi (tr Api i )1/pi ≤ tr pi Ai . i=1

i=1

Furthermore, the following statements are equivalent: 6m i) tr A1 · · · Am  = i=1 (tr Api i )1/pi. m ii) tr A1 · · · Am  = tr i=1 p1i Api i. iii) Ap11 = · · · = Apmm. Proof: See [979]. Remark: The first inequality is a matrix version of H¨ older’s inequality. The first and third terms constitute a matrix version of Young’s inequality. See Fact 1.12.32 and Fact 1.17.31. Fact 8.12.13. Let A1, . . . , Am ∈ Fn×n, assume that A1, . . . , Amare positive m semidefinite, let α1, . . . , αm be nonnegative numbers, and assume that i=1 αi ≥ 1. Then, . . m m . ! . ! . αi . Ai . ≤ (tr Ai )αi. .tr . . i=1 i=1 m Furthermore, if holds if and only if A2 , . . . , Am are i=1 αi = 1, then equality m scalar multiples of A1, whereas, if i=1 αi > 1, then equality holds if and only if A2 , . . . , Am are scalar multiples of A1 and rank A1 = 1. Proof: See [325]. Remark: See Fact 8.12.11. Fact 8.12.14. Let A, B ∈ Fn×n. Then, |tr AB|2 ≤ (tr A∗A) tr BB ∗. Proof: See [1526, p. 25] or Corollary 9.3.9. Remark: See Fact 8.12.15.

526

CHAPTER 8

Fact 8.12.15. Let A ∈ Fn×m and B ∈ Fm×n, and let k ∈ N. Then, |tr (AB)2k | ≤ tr (A∗ABB ∗ )k ≤ tr (A∗A)k(BB ∗ )k ≤ [tr (A∗A)k ] tr (BB ∗ )k. In particular,

|tr (AB)2 | ≤ tr A∗ABB ∗ ≤ (tr A∗A) tr BB ∗.

Proof: See [1511] for the case n = m. If n = m, then A and B can be augmented with 0’s. Problem: Show that |tr AB|2 |tr (AB)2 |



≤ tr A∗ABB ∗ ≤ (tr A∗A) tr BB ∗.

See Fact 8.12.14. Fact 8.12.16. Let A, B ∈ Fn×n, assume that A and B are Hermitian, and let k ≥ 1. Then,  k  k tr A2B 2 ≤ tr A2B 2 *

and tr (AB)2k ≤ |tr (AB)2k | ≤

tr (A2B 2 )k tr (AB)2k 

≤ tr A2kB 2k.

Proof: Use Fact 8.12.15, and see [57, 1511]. Remark: It follows from Fact 8.12.7 that tr (AB)2k and tr (A2B 2 )k are real. Fact 8.12.17. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let k, m ∈ P, where m ≤ k. Then, m  k tr (AmB m ) ≤ tr AkB k . In particular,

tr (AB)k ≤ tr AkB k.

If, in addition, k is even, then

k/2  ≤ tr AkB k. tr (AB)k ≤ tr A2B 2

Proof: Use Fact 8.19.21 and Fact 8.19.28. Remark: It follows from Fact 8.12.7 that tr (AB)k is real. Remark: The result tr (AB)k ≤ tr AkB k is the Lieb-Thirring inequality. See [201, k/2  follows from Fact 8.12.22. See p. 279]. The inequality tr (AB)k ≤ tr A2B 2 [1501, 1511]. Fact 8.12.18. Let A, B ∈ Fn×n, assume that A is positive semidefinite, assume that B is Hermitian, and let α ∈ [0, 1]. Then, tr (AB)2 ≤ tr A2αBA2−2αB ≤ tr A2B 2 . Proof: See [535].

527

POSITIVE-SEMIDEFINITE MATRICES

Fact 8.12.19. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then,  1/2 tr AB ≤ tr AB 2A = tr AB ≤ 14 tr (A + B)2 and

tr (AB)2 ≤ tr A2B 2 ≤

1 16 tr (A

+ B)4.

Proof: See Fact 8.12.22 and Fact 9.9.18. Fact 8.12.20. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, tr AB = tr A1/2BA1/2  1/2 1/2  A1/2BA1/2 = tr A1/2BA1/2   1/2 2 ≤ tr A1/2BA1/2 ≤ (tr A)(tr B) ≤ 14 (tr A + tr B)2

≤ 12 (tr A)2 + (tr B)2 and tr AB ≤ ≤

√ √ tr A2 tr B 2 1 4

√ 2 √ tr A2 + tr B 2

  tr A2 + tr B 2

≤ 12 (tr A)2 + (tr B)2 . ≤

1 2

Remark: Use Fact 1.12.4. Remark: Note that

n 1/2   1/2 1/2 1/2 = λi (AB). tr A BA i=1

The second inequality follows from Proposition 9.3.6 with p = q = 2, r = 1, and A and B replaced by A1/2 and B 1/2. Remark: See Fact 2.12.16. Fact 8.12.21. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let p ≥ 1. Then, tr AB ≤ tr (Ap/2B pAp/2 )1/p. Proof: See [534].

528

CHAPTER 8

Fact 8.12.22. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let p ≥ 0 and r ≥ 1. Then,  pr p  tr A1/2BA1/2 ≤ tr Ar/2B rAr/2 . In particular,

2p   p tr A1/2BA1/2 ≤ tr AB 2A

and

tr AB ≤ tr (AB 2A)1/2 = tr AB.

Proof: Use Fact 8.19.21 and Fact 8.19.28. Remark: This result is the Araki-Lieb-Thirring inequality. See [72, 91] and [201, p. 258]. See Fact 8.10.50, Fact 8.19.27, and Fact 9.9.17. Problem: Referring to Fact 8.12.20, compare the upper bounds ⎧   2 1/2 1/2 1/2 ⎪ tr A BA ⎪ ⎪ ⎨ tr AB ≤ √tr A2 √tr B 2 ⎪ ⎪ ⎪ 1/2 ⎩  tr AB 2A . Fact 8.12.23. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let q ≥ 0 and t ∈ [0, 1]. Then, 2tq σmax (A) tr B tq ≤ tr(AtB tAt )q ≤ tr (ABA)tq .

Proof: See [91]. Remark: The right-hand inequality is equivalent to the Araki-Lieb-Thirring inequality, where t = 1/r and q = pr. See Fact 8.12.22. Fact 8.12.24. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let p ≥ r ≥ 0. Then,   p 1/p   r 1/r tr A1/2BA1/2 ≤ tr A1/2BA1/2 . In particular,

⎧  1/2 ⎪   tr AB 2A ⎨ 2 1/2 tr A1/2BA1/2 ≤ tr AB ≤   2 ⎪ ⎩ tr A1/2BA1/2 1/2 .

Proof: This result follows from the power-sum inequality Fact 1.17.35. See [377]. Fact 8.12.25. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, assume that A ≤ B, and let p, q ≥ 0. Then, tr ApB q ≤ tr B p+q. If, in addition, A and B are positive definite, then this inequality holds for all p, q ∈ R satisfying q ≥ −1 and p + q ≥ 0. Proof: See [250].

529

POSITIVE-SEMIDEFINITE MATRICES

Fact 8.12.26. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, assume that A ≤ B, let f : [0, ∞) → [0, ∞), and assume that f (0) = 0, f is continuous, and f is increasing. Then, tr f (A) ≤ tr f (B). Now, let p > 1 and q ≥ max{−1, −p/2}, and, if q < 0, assume that A is positive definite. Then, tr f (Aq/2B pAq/2 ) ≤ tr f (Ap+q ). Proof: See [541]. Fact 8.12.27. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let α ∈ [0, 1]. Then, tr AαB 1−α ≤ (tr A)α (tr B)1−α ≤ tr[αA + (1 − α)B]. Furthermore, the first inequality is an equality if and only if A and B are linearly dependent, while the second inequality is an equality if and only if A = B. Proof: Use Fact 8.12.11 or Fact 8.12.13 for the left-hand inequality and Fact 1.12.21 for the right-hand inequality. Fact 8.12.28. Let A, B ∈ Fn×n, assume that A and B are positive definite, and let α ∈ [0, 1]. Then, tr A−αB α−1  α 1−α

≤ tr A−1 tr B −1 ≤ tr αA−1 + (1 − α)B −1 −1 tr [αA + (1 − α)B] and

⎧  ⎫   ⎨ tr A−1 α tr B −1 1−α ⎬

−1 −1  tr [αA + (1 − α)B]−1 ≤    ≤ tr αA + (1 − α)B . ⎩tr A−1 A−1/2BA−1/2 α−1 ⎭

Remark: In the first string of inequalities, the upper left inequality and right-hand inequality are equivalent to Fact 8.12.27. The lower left inequality is given by xxxiii) of Proposition 8.6.17. The second string of inequalities combines the lower left inequality in the first string of inequalities with the third string of inequalities in Fact 8.10.46. Remark: These inequalities interpolate the convexity of φ(A) = tr A−1. See Fact 1.12.21. Fact 8.12.29. Let A, B ∈ Fn×n, and assume that B is positive semidefinite. Then, |tr AB| ≤ σmax (A)tr B. Proof: Use Proposition 8.4.13 and σmax (A + A∗ ) ≤ 2σmax (A). Remark: See Fact 5.12.4. Fact 8.12.30. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let p ≥ 1. Then, p  tr(Ap + B p ) ≤ tr (A + B)p ≤ (tr Ap )1/p + (tr B p )1/p .

530

CHAPTER 8

Furthermore, the second inequality is an equality if and only if A and B are linearly independent. Proof: See [250] and [971]. Remark: The first inequality is the McCarthy inequality. The second inequality is a special case of the triangle inequality for the norm  · σp and a matrix version of Minkowski’s inequality. Fact 8.12.31. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, let m be a positive integer, and define p ∈ F[s] by p(s) = tr (A + sB)m. Then, all of the coefficients of p are nonnegative. Remark: This result is the Bessis-Moussa-Villani trace conjecture. See [705, 934] and Fact 8.12.32. Fact 8.12.32. Let A, B ∈ Fn×n, assume that A is Hermitian and B is positive semidefinite, and define f (t) = eA+tB. Then, for all k ∈ N and t ≥ 0, (−1)k+1f (k) (t) ≥ 0. Remark: This result is a consequence of the Bessis-Moussa-Villani trace conjecture. See [705, 934] and Fact 8.12.31. Remark: See Fact 8.14.18. Fact 8.12.33. Let A, B ∈ Fn×n, assume that A and B are Hermitian, and let f : R → R. Then, the following statements hold: i) If f is convex, then there exist unitary matrices S1, S2 ∈ Fn×n such that f [ 12 (A + B)] ≤ 12 [S1( 12 [f (A) + f (B)])S1∗ + S2 ( 12 [f (A) + f (B)])S2∗ ]. ii) If f is convex and even, then there exist unitary matrices S1, S2 ∈ Fn×n such that f [ 12 (A + B)] ≤ 12 [S1 f (A)S1∗ + S2 f (B)S2∗ ]. iii) If f is convex and increasing, then there exists a unitary matrix S ∈ Fn×n such that f [ 12 (A + B)] ≤ S( 12 [f (A) + f (B)])S ∗. iv) There exist unitary matrices S1, S2 ∈ Fn×n such that A + B ≤ S1AS1∗ + S2 BS2∗. v) If f is convex, then tr f [ 12 (A + B)] ≤ tr 12 [f (A) + f (B)]. Proof: See [251, 252]. Remark: Result v), which is a consequence of i), is von Neumann’s trace inequality.

531

POSITIVE-SEMIDEFINITE MATRICES

Remark: See Fact 8.12.34. Fact 8.12.34. Let f : R → R, and assume that f is convex. Then, the following statements hold: i) If f (0) ≤ 0, A ∈ Fn×n is Hermitian, and S ∈ Fn×m is a contractive matrix, then tr f (S ∗AS) ≤ tr S ∗f (A)S. ii) If A1, . . . , Ak ∈ Fn×n are Hermitian and S1, . . . , Sk ∈ Fn×m satisfy k ∗ i=1 Si Si = I, then   k k   ∗ Si Ai Si ≤ tr Si∗f (Ai )Si . tr f i=1

iii) If A ∈ F

n×n

is Hermitian and S ∈ F

i=1 n×n

is a projector, then

tr Sf (SAS)S ≤ tr Sf (A)S. Proof: See [252] and [1066, p. 36]. Remark: Special cases are considered in [807]. Remark: The first result is due to Brown and Kosaki, the second result is due to Hansen and Pedersen, and the third result is due to Berezin. Remark: The second result generalizes statement v) of Fact 8.12.33. Fact 8.12.35. Let A, B ∈ Fn×n, assume that B is positive semidefinite, and assume that A∗A ≤ B. Then, |tr A| ≤ tr B 1/2. Proof: Corollary 8.6.11 with r = 2 implies that (A∗A)1/2 ≤ tr B 1/2 n. Letting mspec(A) = {λ , . . . , λ } , it follows from Fact 9.11.2 that |tr A| ≤ 1 n ms i=1 |λi | ≤ n ∗ 1/2 1/2 σ (A) = tr (A A) ≤ tr B . See [171]. i i=1 Fact 8.12.36. Let A, B ∈ Fn×n, assume that A is positive definite and B is positive semidefinite, let α ∈ [0, 1], and let β ≥ 0. Then,  α/(2−α) tr(−BA−1B + βB α ) ≤ β(1 − α2 ) tr αβ A . 2 If, in addition, either A and B commute or B is a multiple of a projector, then  α/(2−α) . −BA−1B + βB α ≤ β(1 − α2 ) αβ 2 A Proof: See [649, 650]. n×n , B, Q ∈ Fn×m, and C, R ∈ Fm×m, and assume Fact 8.12.37.   Let A, P ∈ F P Q A B (n+m)×(n+m) that B ∗ C , Q∗ R ∈ F are positive semidefinite. Then,

|tr BQ∗ |2 ≤ (tr AP )(tr CR). Proof: See [911, 1530].

532

CHAPTER 8

Fact 8.12.38. Let A, B ∈ Fn×n, and assume that A and B are projectors. Then, (tr AB)2 ≤ (rank AB)(tr ABAB). Furthermore, equality holds if and only if there exists α > 0 such that αAB is idempotent. Proof: See [116]. Fact 8.12.39. Let A, B ∈ Fn×m, let X ∈ Fn×n, and assume that X is positive definite. Then, |tr A∗B|2 ≤ (tr A∗XA)(tr B ∗X −1A).

∗ AB ∗ I and AA . See [911, 1530]. Proof: Use Fact 8.12.37 with X BA∗ BB ∗ I X−1 Fact 8.12.40. Let A, B, C ∈ Fn×n, and assume that A and B are Hermitian and C is positive semidefinite. Then, |tr ABC 2 − tr ACBC| ≤ 14 [λ1(A) − λn (A)][λ1(B) − λn (B)] tr C 2. Proof: See [254]. 



n×n , A12 ∈ Rn×m, and A22 ∈ Rm×m, define A = Fact  8.12.41. Let A11 ∈ R ∈ R(n+m)×(n+m), and assume that A is symmetric. Then, A is positive

A11 A12 T A12 A22

semidefinite if and only if, for all B ∈ Rn×m, 1/2  1/2 T 1/2 ≤ tr A BA B A . tr BAT 22 12 11 11 Proof: See [171].

Fact 8.12.42. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, and assume that

∈ F(n+m)×(n+m) is positive semidefinite. Then,  tr B ∗B ≤ (tr A2 )(tr C 2 ) ≤ (tr A)(tr C).

A B B∗ C

Proof: Use Fact 8.12.37 with P = A, Q = B, and R = C. Remark: The inequality consisting of the first and third terms is given in [1102]. Remark: See Fact 8.12.43 for the case n = m. Fact 8.12.43. Let A, B, C ∈ Fn×n, and assume that positive semidefinite. Then,



A B B∗ C

∈ F2n×2n is

|tr B|2 ≤ (tr A)(tr C) and

|tr B 2 | ≤ tr B ∗B ≤

 (tr A2 )(tr C 2 ) ≤ (tr A)(tr C).

Remark: The first result follows from Fact 8.12.44. In the second string, the first inequality is given by Fact 9.11.3,  while the second inequality is given by Fact 8.12.42. The inequality |tr B 2 | ≤ (tr A2 )(tr C 2 ) is given in [989].

533

POSITIVE-SEMIDEFINITE MATRICES

Fact 8.12.44. Let Aij ∈ Fn×n for all i, j ∈ {1, . . . , k}, define A ∈ Fkn×kn by ⎡ ⎤ A11 · · · A1k ⎢ . . .. ⎥  ⎢ . · ·. · A= . ⎥ ⎣ . ⎦, A∗1k · · · Akk and assume that A is positive semidefinite. Then, ⎡ ⎤ tr A11 · · · tr A1k ⎢ ⎥ .. . .. ⎣ ⎦≥0 · ·. · . . tr A∗1k

and

⎡

2 tr A11 .. .

⎢ ⎣

tr A∗1k A1k

···

tr Akk

··· . · ·. · ···

⎤ tr A∗1k A1k ⎥ .. ⎦ ≥ 0. . 2 tr Akk

Proof: See [394, 989, 1102]. Remark: See Fact 8.13.43.

8.13 Facts on the Determinant Fact 8.13.1. Let A ∈ Fn×n, assume that A is positive semidefinite, and let mspec(A) = {λ1, . . . , λn }ms . Then, (n−1)/n

λmin(A) ≤ λ1/n max(A)λmin

(A)

≤ λn ≤ λ1 1/n

≤ λmin(A)λ(n−1)/n (A) max ≤ λmax(A) and λnmin(A) ≤ λmax(A)λn−1 min(A) ≤ det A ≤ λmin(A)λn−1 max(A) ≤ λnmax(A). Proof: Use Fact 5.11.29. Fact 8.13.2. Let A ∈ Fn×n, and assume that A + A∗ is positive semidefinite. Then, det 12 (A + A∗ ) ≤ |det A|. Furthermore, if A + A∗ is positive definite, then equality holds if and only if A is

534

CHAPTER 8

Hermitian. Proof: The inequality follows from Fact 5.11.25 and Fact 5.11.28. Remark: This result is the Ostrowski-Taussky inequality. Remark: See Fact 8.13.2. Fact 8.13.3. Let A ∈ Fn×n, and assume that A + A∗ is positive semidefinite. Then, [det 12 (A + A∗ )]2/n + |det 12 (A − A∗ )|2/n ≤ |det A|2/n. Furthermore, if A + A∗ is positive definite, then equality holds if and only if every eigenvalue of (A + A∗ )−1(A − A∗ ) has the same absolute value. Finally, if n ≥ 2, then det 12 (A + A∗ ) ≤ det 12 (A + A∗ ) + |det 12 (A − A∗ )| ≤ |det A|. Proof: See [479, 783]. To prove the last result, use Fact 1.12.30. Remark: Setting A = 1 + j shows that the last result can fail for n = 1. Remark: −A is semidissipative. Remark: The last result interpolates Fact 8.13.2. Remark: Extensions to the case in which A + A∗ is positive definite are considered in [1300]. Fact 8.13.4. Let A, B ∈ Fn×n, and assume that A is positive semidefinite. Then, (det A)2/n + |det(A + B)|2/n ≤ |det(A + B)|2/n. Furthermore, if A is positive definite, then equality holds if and only if every eigenvalue of A−1B has the same absolute value. Finally, if n ≥ 2, then det A ≤ det A + |det B| ≤ |det(A + B)|. Remark: This result is a restatement of Fact 8.13.2 in terms of the Cartesian decomposition. Fact 8.13.5. Let A, B ∈ Fn×n, assume that A is positive semidefinite, assume that B is positive definite. Then, n !

[λ2i (A) + λ2i (B)]1/2 ≤ |det(A + jB)| ≤

i=1

n !

[λ2i (A) + λ2n−i+1 (B)]1/2.

i=1

Proof: See [162]. Fact 8.13.6. Let A, B ∈ Fn×n, and assume that A is positive semidefinite and B is skew Hermitian. Then, det A ≤ |det(A + B)|. Furthermore, if A and B are real, then det A ≤ det(A + B). Finally, if A is positive definite, then equality holds if and only if B = 0.

535

POSITIVE-SEMIDEFINITE MATRICES

Proof: See [671, p. 447] and [1125, pp. 146, 163]. Now, suppose that A and B are real. If A is positive definite, then A−1/2BA−1/2 is skew symmetric, and thus det(A + B) = (det A)det I + A−1/2BA−1/2 is positive. If A is positive semidefinite, then a continuity argument implies that det(A + B) is nonnegative. Remark: Extensions of this result are given in [223]. Fact 8.13.7. Let A, B ∈ Fn×n, and assume that A is positive definite and B is Hermitian. Then, n   1/2 ! 1 + σi2 A−1/2BA−1/2 det(A + jB) = (det A) . i=1

Proof: See [328]. Fact 8.13.8. Let A ∈ Fn×n, and assume that A is positive definite. Then, 1/2  , n + tr log A = n + log det A ≤ n(det A)1/n ≤ tr A ≤ ntr A2 with equality if and only if A = I. Remark: The inequality

(det A)1/n ≤

1 n tr A

is a consequence of the arithmetic-mean–geometric-mean inequality. Fact 8.13.9. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and assume that A ≤ B. Then, n det A + det B ≤ det(A + B). Proof: See [1125, pp. 154, 166]. Remark: Under weaker conditions, Corollary 8.4.15 implies that det A + det B ≤ det(A + B). Fact 8.13.10. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, √ det A + det B + (2n − 2) det AB ≤ det(A + B). If, in addition, B ≤ A, then

√ det A + (2n − 1) det B ≤ det A + det B + (2n − 2) det AB ≤ det(A + B).

Proof: See [1084] or [1215, p. 231]. Fact 8.13.11. Let A ∈ Rn×n, and assume that A+AT is positive semidefinite. Then,    1 T A ≤ 12 AA + AAT . 2 A+A Now, assume in addition that A + AT is positive definite. Then,     −1   det 12 A + AT 12 A + AT ≤ (det A) 12 A−1 + A−T .

536

CHAPTER 8

Furthermore,     −1   det 12 A + AT 12 A + AT < (det A) 12 A−1 + A−T   if and only if rank A − AT ≥ 4. Finally, if n ≥ 4 and A − AT is nonsingular, then       −1 (det A) 12 A−1 + A−T < det A − det 12 A − AT 12 A + AT . Proof: See [478, 782]. Remark: This result does not hold for complex matrices. Remark: See Fact 8.9.32 and Fact 8.18.12. Fact 8.13.12. Let A ∈ Fn×n, assume that A is Hermitian, and assume that, for all i = 1, . . . , n − 1, det A({1,...,i}) > 0. Then, sign[λmin (A)] = sign(det A). Furthermore, A is (positive semidefinite, positive definite) if and only if (det A ≥ 0, det A > 0). Finally, if det A = 0, then rank A = n − 1. Proof: Use Proposition 8.2.8 and Theorem 8.4.5. See [1208, p. 278]. Fact 8.13.13. Let A ∈ Rn×n, and assume that A is positive definite. Then, n 

[det A({1,...,i}) ]1/i ≤ (1 + n1 )n tr A < etr A.

i=1

Proof: See [31]. Fact 8.13.14. Let A ∈ Fn×n, assume that A is positive definite and Toeplitz,  and, for all i ∈ {1, . . . , n}, define Ai = A({1,...,i}) ∈ Fi×i. Then, (det A)1/n ≤ (det An−1 )1/(n−1) ≤ · · · ≤ (det A2 )1/2 ≤ det A1 . Furthermore, det A det An−1 det A3 det A2 ≤ ≤ ··· ≤ ≤ . det An−1 det An−2 det A2 det A1 Proof: See [360] or [361, p. 682]. Fact 8.13.15. Let A, B ∈ Fn×n, assume that B is Hermitian, and assume that A∗BA < A + A∗. Then, det A = 0. Fact 8.13.16. Let A, B ∈ Fn×n, assume that A and B are positive definite, and let α ∈ [0, 1]. Then, (det A)α(det B)1−α ≤ det[αA + (1 − α)B]. Furthermore, equality holds if and only if A = B. Proof: This inequality is a restatement of xxxviii) of Proposition 8.6.17. Remark: This result is due to Bergstrom.  Remark: α = 2 yields (det A) det B ≤ det[ 12 (A + B)].

537

POSITIVE-SEMIDEFINITE MATRICES

Fact 8.13.17. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, assume that either A ≤ B or B ≤ A, and let α ∈ [0, 1]. Then, det[αA + (1 − α)B] ≤ αdet A + (1 − α)det B. Proof: See [1440]. Fact 8.13.18. Let A, B ∈ Fn×n, and assume that A and B are positive definite. Then, det A det B det(A + B) .  + ≤ det A[1;1] det B[1;1] det A[1;1] + B[1;1] Proof: See [1125, p. 145]. Remark: This inequality is a special case of xli) of Proposition 8.6.17. Remark: See Fact 8.11.4. Fact 8.13.19. Let A1, . . . , Ak ∈ Fn×n, assume that A1, . . . , Ak are positive semidefinite, and let λ1, . . . , λk ∈ C. Then,   k   k   det λi Ai ≤ det |λi |Ai . i=1

i=1

Proof: See [1125, p. 144]. 

Fact 8.13.20. Let A, B, C ∈ Rn×n, let D = A + jB, and assume that CB + B C < D + D∗. Then, det A = 0. T T

Fact 8.13.21. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let m be a positive integer. Then, 1/m

n1/m(det AB)1/n ≤ (tr AmB m )

.

Proof: See [377]. Remark: Assuming det B = 1 and setting m = 1 yields Proposition 8.4.14. Fact 8.13.22. Let A, B, C ∈ Fn×n, define   A B  , A= B∗ C and assume that A is positive semidefinite. Then, |det(B + B ∗ )| ≤ det(A + C). If, in addition, A is positive definite, then |det(B + B ∗ )| < det(A + C). Remark: Use Fact 8.11.5. Fact 8.13.23. Let A, B ∈ Fn×m. Then, |det A∗B|2 ≤ (det A∗A)(det B ∗B). A∗A B ∗A . Proof: Use Fact 8.11.14 or apply Fact 8.13.43 to A ∗ B B ∗B

538

CHAPTER 8

Remark: This result is a determinantal version of the Cauchy-Schwarz inequality. Fact 8.13.24. Let A ∈ Fn×n, assume that A is positive definite, and let B ∈ Fm×n, where rank B = m. Then, (det BB ∗ )2 ≤ (det BAB ∗ )det BA−1B ∗. Proof: Use Fact 8.11.19. Fact 8.13.25. Let A, B ∈ Fn×n. Then, |det(A + B)|2 + |det(I − AB ∗ )|2 ≤ det(I + AA∗ )det(I + B ∗B) and

|det(A − B)|2 + |det(I + AB ∗ )|2 ≤ det(I + AA∗ )det(I + B ∗B).

Furthermore, the first inequality is an equality if and only if either n = 1, A+B = 0, or AB ∗ = I. Proof: This result follows from Fact 8.11.16. See [1526, p. 184]. Fact 8.13.26. Let A, B ∈ Fn×m, and assume that I − A∗A and I − B ∗B are positive semidefinite. Then, 0 ≤ det(I − A∗A)det(I − B ∗B) * |det(I − A∗B)|2 ≤ |det(I + A∗B)|2 ≤ det(I + A∗A)det(I + B ∗B). Now, assume in addition that n = m. Then, 0 ≤ det(I − A∗A)det(I − B ∗B) ≤ |det(I − A∗B)|2 − |det(A − B)|2 ≤ |det(I − A∗B)|2 ≤ |det(I − A∗B)|2 + |det(A + B)|2 ≤ det(I + A∗A)det(I + B ∗B) and 0 ≤ det(I − A∗A)det(I − B ∗B) ≤ |det(I + A∗B)|2 − |det(A + B)|2 ≤ |det(I + A∗B)|2 ≤ |det(I + A∗B)|2 + |det(A − B)|2 ≤ det(I + A∗A)det(I + B ∗B). Finally,



det[(I − A∗A)−1 ]

det[(I − A∗B)−1 ]

det[(I − B ∗A)−1 ] det[(I − B ∗B)−1 ]

 ≥ 0.

539

POSITIVE-SEMIDEFINITE MATRICES

Proof: The second inequality and Fact 8.11.21 are Hua’s inequalities. See [49]. The third inequality follows from Fact 8.11.15. The first interpolation in the case n = m is given in [1087]. Remark: Generalizations of the last result are given in [1502]. Remark: See Fact 8.11.21 and Fact 8.15.20. Fact 8.13.27. Let A, B ∈ Fn×n, and let α, β ∈ (0, ∞). Then, |det(A + B)|2 ≤ det(β −1I + αA∗A)det(α−1I + βBB ∗ ). Proof: Use Fact 8.11.20. See [1527]. Fact 8.13.28. Let A ∈ Fn×m, B ∈ Fn×l, C ∈ Fn×m, and D ∈ Fn×l. Then, |det(AC ∗ + BD∗ )|2 ≤ det(AA∗ + BB ∗ )det(CC ∗ + DD∗ ). 

A B ]. Proof: Use Fact 8.13.39 and AA∗ ≥ 0, where A = [ C D

Remark: See Fact 2.14.22. Fact 8.13.29. Let A ∈ Fn×m, B ∈ Fn×m, C ∈ Fk×m, and D ∈ Fk×m. Then, |det(A∗B + C ∗D)|2 ≤ det(A∗A + C ∗C) det(B ∗B + D∗D).  A B Proof: Use Fact 8.13.39 and A∗A ≥ 0, where A = [ C D ].

Remark: See Fact 2.14.18. Fact 8.13.30. Let A, B, C ∈ Fn×n. Then, |det(B + CA)|2 ≤ det(A∗A + B ∗B)det(I + CC ∗ ). Proof: See [736]. Remark: See Fact 8.10.37. F

n×n

Fact 8.13.31. Let A, B ∈ Fn×m. Then, there exist unitary matrices S1, S2 ∈ such that I + A + B ≤ S1(I + A)1/2S2(I + B)S2∗(I + A)1/2S1∗.

Therefore,

det(I + A + B) ≤ det(I + A)det(I + B).

Proof: See [49, 1301]. Remark: This result is due to Seiler and Simon. Fact 8.13.32. Let A, B ∈ Fn×n, assume that A+ A∗ > 0 and B + B ∗ ≥ 0, and let α > 0. Then, αI + AB is nonsingular and has no negative eigenvalues. Hence, det(αI + AB) > 0. Proof: See [628]. Remark: Equivalently, −A is dissipative and −B is semidissipative. Problem: Find a positive lower bound for det(αI + AB) in terms of α, A, and B.

540

CHAPTER 8

Fact 8.13.33. Let A ∈ Fn×n, assume that A is positive definite, and define  α=

1 n tr A

and  β=

n 

1 n(n−1)

|A(i,j) |.

i,j=1 i=j

Then,

|det A| ≤ (α − β)n−1[α + (n − 1)β].

Furthermore, if A = aIn + b1n×n , where a + nb > 0 and a > 0, then α = a + b, β = b, and equality holds. Proof: See [1060]. Remark: See Fact 2.13.12 and Fact 8.9.35. Fact 8.13.34. Let A ∈ Fn×n, assume that A is positive definite, and define 

β=

1 n(n−1)

n  i,j=1 i=j

|A(i,j) |  . A(i,i) A(j,j)

Then, |det A| ≤ (1 − β)n−1[1 + (n − 1)β]

n !

A(i,i) .

i=1

Proof: See [1060]. Remark: This inequality strengthens Hadamard’s inequality. See Fact 8.18.11. See also [422]. Fact 8.13.35. Let A ∈ Fn×n. Then, ⎛ ⎞1/2 n n n !  ! |det A| ≤ ⎝ |A(i,j) |2⎠ = rowi (A)2 . i=1

j=1

i=1

Furthermore, equality holds if and only if AA∗ is diagonal. Now, let α > 0 be such that, for all i, j ∈ {1, . . . , n}, |A(i,j) | ≤ α. Then, |det A| ≤ αn nn/2. If, in addition, at least one entry of A has absolute value less than α, then |det A| < αn nn/2. Remark: Replace A with AA∗ in Fact 8.18.11. Remark: This result is a direct consequence of Hadamard’s inequality. See Fact 8.18.11. Remark: See Fact 2.13.14 and Fact 6.5.26.

541

POSITIVE-SEMIDEFINITE MATRICES 

Fact 8.13.36. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, define A = F(n+m)×(n+m), and assume that A is positive definite. Then, n+m !   A(i,i). det A = (det A)det C − B ∗A−1B ≤ (det A)det C ≤



A B B∗ C

∈

i=1

Proof: The second inequality is obtained by successive application of the first inequality. Remark: det A ≤ (det A)det C is Fischer’s inequality.

 Fact 8.13.37. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, define A = BA∗ B C ∈  F(n+m)×(n+m), assume that A is positive definite, let k = min{m, n}, and, for  i = 1, . . . , n, let λi = λi (A). Then, n+m−k  k n+m ! ! !

2 1 λi ≤ (det A)det C ≤ λi 2 (λi + λn+m−i+1 ) . i=1

i=k+1

i=1

Proof: The left-hand inequality is given by Fact 8.13.36. The right-hand inequality is given in [1052]. Fact 8.13.38. Let A ∈ Fn×n, and let S ⊆ {1, . . . , n}. Then, the following statements hold: i) If α ⊂ {1, . . . , n}, then det A ≤ [det A(α) ] det A(α∼ ) . ii) If α, β ⊆ {1, . . . , n}, then det A(α∪β) ≤

[det A(α) ] det A(β) . det A(α∩β)

iii) If 1 ≤ k ≤ n − 1, then ⎛ ⎛ ⎞(n−1 k−1 ) ! ⎝ ≤⎝ det A(α) ⎠ {α: card(α)=k+1}

!

⎞(n−1 k ) det A(α) ⎠

.

{α: card(α)=k}

Proof: See [963]. Remark: The first result is Fischer’s inequality, see Fact 8.13.36. The second result is the Hadamard-Fischer inequality. The third result is Szasz’s inequality. See [361, p. 680], [728, p. 479], and [963]. Remark: See Fact 8.13.37. 

Fact 8.13.39. Let A, B, C ∈ Fn×n, define A = that A is positive semidefinite. Then,



A B B∗ C

∈ F2n×2n, and assume

0 ≤ (det A)det C − |det B|2 ≤ det A ≤ (det A)det C. Hence,

|det B|2 ≤ (det A)det C.

542

CHAPTER 8

Furthermore, A is positive definite if and only if |det B|2 < (det A)det C. Proof: Assuming that A is positive definite, it follows that 0 ≤ B ∗A−1B ≤ C, which implies that |det B|2/det A ≤ det C. Then, use continuity for the case in which A is singular. For an alternative proof, see [1125, p. 142]. For the case in which A is positive definite, note that 0 ≤ B ∗A−1B < C, and thus |det B|2/det A < det C. Remark: This result is due to Everitt. Remark: See Fact 8.13.43. Remark: When B is nonsquare, it is not necessarily true that |det(B ∗B)|2 < (det A)det C. See [1528]. A B  Fact 8.13.40. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, define A = B∗ C ∈ F(n+m)×(n+m) , and assume that A is positive semidefinite and A is positive definite. Then, 2  λmax(A) − λmin(A) ∗ −1 BA B ≤ C. λmax(A) + λmin(A) Proof: See [911, 1530].

 2n×2n , and assume Fact 8.13.41. Let A, B, C ∈ Fn×n, define A = BA∗ B C ∈ F that A is positive semidefinite. Then, 2n  λmax(A) − λmin(A) |det B|2 ≤ (det A)det C. λmax(A) + λmin(A) Hence,

2 λmax(A) − λmin(A) |det B| ≤ (det A)det C. λmax(A) + λmin(A)    det A det B ˆ= Now, define A ∈ F2×2. Then, ∗ det B det C 

2

2 ˆ − λmin(A) ˆ λmax(A) |det B| ≤ (det A)det C. ˆ + λmin(A) ˆ λmax(A) 

2

Proof: See [911, 1530]. Remark: The second and third bounds are not comparable. See [911, 1530]. A B  Fact 8.13.42. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, define A = B∗ C ∈ (n+m)×(n+m) F , assume that A is positive semidefinite, and assume that A and C are positive definite. Then, det(A|A)det(C|A) ≤ det A. Proof: See [736]. Remark: This result is the reverse Fischer inequality.

543

POSITIVE-SEMIDEFINITE MATRICES

Fact 8.13.43. Let Aij ∈ Fn×n for all i, j ∈ {1, . . . , k}, define ⎡ ⎤ A11 · · · A1k ⎢ . . .. ⎥  ⎢ . · ·. · A= . ⎥ ⎣ . ⎦, A∗1k · · · Akk assume that A is positive semidefinite, let ⎡ (k) A11 ⎢  . A˜k = ⎢ ⎣ .. ∗(k)

A1k

1 ≤ k ≤ n, and define ⎤ (k) · · · A1k . .. ⎥ ⎥ · ·. · . ⎦. ···

(k)

Akk

Then, A˜k is positive semidefinite. In particular, ⎡ ⎤ det A11 · · · det A1k ⎢ ⎥ .. . .. A˜n = ⎣ ⎦ · ·. · . . det A∗1k · · · det Akk is positive semidefinite. Furthermore, ˜ det A ≤ det A. Now, assume in addition that A is positive definite. Then, det A = det A˜ if and only if, for all distinct i, j ∈ {1, . . . , k}, Aij = 0. Proof: The first statement is given in [394]. The inequality as well as the final statement are given in [1298]. Remark: B (k) is the kth compound of B. See Fact 7.5.17. Remark: Note that every principal subdeterminant of A˜n is lower bounded by the determinant of a positive-semidefinite matrix. Hence, the inequality implies that A˜n is positive semidefinite. Remark: A weaker result is given in [396] and quoted in [986] in terms of elementary symmetric functions of the eigenvalues of each block. 1 0 1 0 Remark: The example A = 01 10 01 00 shows that A˜ can be positive definite while 00 01

A is singular. Remark: The matrix whose (i, j) entry is det Aij is a determinantal compression of A. See [395, 989, 1298]. Remark: See Fact 8.12.44.

8.14 Facts on Convex Sets and Convex Functions Fact 8.14.1. Let f : Rn → Rn, and assume that f is convex. Then, for all α ∈ R, the sets {x ∈ Rn : f (x) ≤ α} and {x ∈ Rn : f (x) < α} are convex. Proof: See [508, p. 108]. Remark: The converse is not true. Consider the function f (x) = x3.

544

CHAPTER 8

Fact 8.14.2. Let A ∈ Fn×n, assume that A is Hermitian, let α ≥ 0, and define  {x ∈ Fn : x∗Ax < α}. Then, the following statements hold: the set S = i) S is open. ii) S is a blunt cone if and only if α = 0. iii) S is nonempty if and only if either α > 0 or λmin (A) < 0. iv) S is convex if and only if A ≥ 0. v) S is convex and nonempty if and only if α > 0 and A ≥ 0. vi) The following statements are equivalent: a) S is bounded. b) S is convex and bounded. c) A > 0. vii) The following statements are equivalent: a) S is bounded and nonempty. b) S is convex, bounded, and nonempty. c) α > 0 and A > 0. Fact 8.14.3. Let A ∈ Fn×n, assume that A is Hermitian, let α ≥ 0, and define  the set S = {x ∈ Fn : x∗Ax ≤ α}. Then, the following statements hold: i) S is closed. ii) 0 ∈ S, and thus S is nonempty. iii) S is a pointed cone if and only if α = 0 or A ≤ 0. iv) S is convex if and only if A ≥ 0. v) The following statements are equivalent: a) S is bounded. b) S is convex and bounded. c) A > 0. Fact 8.14.4. Let A ∈ Fn×n, assume that A is Hermitian, let α ≥ 0, and define  {x ∈ Fn : x∗Ax = α}. Then, the following statements hold: the set S = i) S is closed. ii) S is nonempty if and only if either α = 0 or λmax (A) > 0. iii) The following statements are equivalent: a) S is a pointed cone. b) 0 ∈ S. c) α = 0. iv) S = {0} if and only if α = 0 and either A > 0 or A < 0.

POSITIVE-SEMIDEFINITE MATRICES

545

v) S is bounded if and only if either A > 0 or both α > 0 and A ≤ 0. vi) S is bounded and nonempty if and only if A > 0. vii) The following statements are equivalent: a) S is convex. b) S is convex and nonempty. c) α = 0 and either A > 0 or A < 0. viii) If α > 0, then the following statements are equivalent: a) S is nonempty. b) S is not convex. c) λmax (A) > 0. ix) The following statements are equivalent: a) S is convex and bounded. b) S is convex, bounded, and nonempty. c) α = 0 and A > 0. Fact 8.14.5. Let A ∈ Fn×n, assume that A is Hermitian, let α ≥ 0, and define  the set S = {x ∈ Fn : x∗Ax ≥ α}. Then, the following statements hold: i) S is closed. ii) S is a pointed cone if and only if α = 0. iii) S is nonempty if and only if either α = 0 or λmax (A) > 0. iv) S is bounded if and only if S ⊆ {0}. v) The following statements are equivalent: a) S is bounded and nonempty. b) S = {0}. c) α = 0 and A < 0. vi) S is convex if and only if either S is empty or S = Fn. vii) S is convex and bounded if and only if S is empty. viii) The following statements are equivalent: a) S is convex and nonempty. b) S = Fn. c) α = 0 and A ≥ 0. Fact 8.14.6. Let A ∈ Fn×n, assume that A is Hermitian, let α ≥ 0, and define  the set S = {x ∈ Fn : x∗Ax > α}. Then, the following statements hold: i) S is open. ii) S is a blunt cone if and only if α = 0.

546

CHAPTER 8

iii) S is nonempty if and only if λmax (A) > 0. iv) The following statements are equivalent: a) S is empty. b) λmax (A) ≤ 0. c) S is bounded. d) S is convex. Fact 8.14.7. Let A ∈ Cn×n, and define the numerical range of A by  Θ1(A) = {x∗Ax: x ∈ Cn and x∗x = 1}

and the set



Θ(A) = {x∗Ax: x ∈ Cn }.

Then, the following statements hold: i) Θ1(A) is a closed, bounded, convex subset of C. ii) Θ(A) = {0} ∪ cone Θ1(A). iii) Θ(A) is a pointed, closed, convex cone contained in C. iv) If A is Hermitian, then Θ1(A) is a closed, bounded interval contained in R. v) If A is Hermitian, then Θ(A) is either (−∞, 0], [0, ∞), or R. vi) Θ1(A) satisfies co spec(A) ⊆ Θ1(A) ⊆ co{ν1 + jμ1, ν1 + jμn , νn + jμ1, νn + jμn }, where



ν1 = λmax 

μ1 = λmax

1

2 (A



+ A∗ ) ,

1 j2 (A

 − A∗ ) ,



 νn = λmin 12 (A + A∗ ) ,    1 μn = λmin j2 (A − A∗ ) .

vii) If A is normal, then Θ1(A) = co spec(A). viii) If n ≤ 4 and Θ1(A) = co spec(A), then A is normal. ix) Θ1(A) = co spec(A) if and only if either A is normal or there exist matrices A1 ∈ Fn1 ×n1 and A2 ∈ Fn2 ×n 2 such that n1 + n2 = n, Θ1(A1 ) ⊆ Θ1(A2 ), and A is unitarily similar to A01 A02 . Proof: See [625] or [730, pp. 11, 52]. Remark: Θ1 (A) is called the field of values in [730, p. 5]. Remark: See Fact 4.10.25 and Fact 8.14.7. Remark: viii) is an example of the quartic barrier. See [359], Fact 8.15.33, and Fact 11.17.3.

547

POSITIVE-SEMIDEFINITE MATRICES

Fact 8.14.8. Let A ∈ Rn×n, and define the real numerical range of A by  Ψ1(A) = {xTAx: x ∈ Rn and xTx = 1}

and the set



Ψ(A) = {xTAx: x ∈ Rn }.

Then, the following statements hold: i) Ψ1(A) = Ψ1 [ 12 (A + AT )]. ii) Ψ1(A) = [λmin [ 12 (A + AT )], λmin [ 12 (A + AT )]]. iii) If A is symmetric, then Ψ1(A) = [λmin (A), λmax (A)]. iv) Ψ(A) = {0} ∪ cone Ψ1 (A). v) Ψ(A) is either (−∞, 0], [0, ∞), or R. vi) Ψ1(A) = Θ1(A) if and only if A is symmetric. Proof: See [730, p. 83]. Remark: Θ1(A) is defined in Fact 8.14.7. Fact 8.14.9. Let A, B ∈ Cn×n, assume that A and B are Hermitian, and   ∗  define x Ax  n ∗ : x ∈ C and x x = 1 ⊆ R2. Θ1(A, B) = x∗Bx Then, Θ1(A, B) is convex. Proof: See [1117]. Remark: This result is an immediate consequence of Fact 8.14.7. Fact 8.14.10. Let A, B ∈ Rn×n, assume that A and B are symmetric, and let α, β be real numbers. Then, the following statements are equivalent: i) There exists x ∈ Rn such that xTAx = α and xTBx = β. ii) There exists a positive-semidefinite matrix X ∈ Rn×n such that tr AX = α and tr BX = β. Proof: See [157, p. 84]. Fact 8.14.11. Let A, B ∈ Rn×n, assume that A and B are symmetric, and   T  define x Ax  n T : x ∈ R Ψ1(A, B) = and x x = 1 ⊆ R2 xTBx and 

Ψ(A, B) =



xTAx xTBx



 : x∈R

n

⊆ R2.

Then, Ψ(A, B) is a pointed, convex cone. If, in addition, n ≥ 3, then Ψ1(A, B) is convex. Proof: See [157, pp. 84, 89] or [416, 1117]. Remark: Ψ(A, B) = [cone Ψ1(A, B)] ∪ {[ 00 ]}.

548

CHAPTER 8

Remark: The set Ψ(A, B) is not necessarily closed. See [416, 1090, 1091]. Fact 8.14.12. Let A, B ∈ Rn×n, where n ≥ 2, assume that A and B are symmetric, let a, b ∈ Rn, let a0 , b0 ∈ R, assume that there exist real numbers α, β such that αA + βB > 0, and define  * xTAx + aTx + a0  n Ψ(A, a, a0 , B, b, b0) = : x∈R ⊆ R2. xTBx + bTx + b0 Then, Ψ(A, a, a0 , B, b, b0) is closed and convex. Proof: See [1117]. Fact 8.14.13. Let A, B, C ∈ Rn×n, where n ≥ 3, assume that A, B, and C are symmetric, and define ⎧⎡ T ⎫ ⎤ ⎨ x Ax ⎬  Φ1(A, B, C) = ⎣ xTBx ⎦ : x ∈ Rn and xTx = 1 ⊆ R3 ⎩ ⎭ xTCx and

⎧⎡ T ⎫ ⎤ ⎨ x Ax ⎬  ⎣ xTBx ⎦ : x ∈ Rn ⊆ R3. Φ(A, B, C) = ⎩ ⎭ xTCx

Then, Φ1(A, B, C) is convex and Φ(A, B, C) is a pointed, convex cone. Proof: See [264, 1114, 1117]. Fact 8.14.14. Let A, B, C ∈ Rn×n, where n ≥ 3, assume that A, B, and C are symmetric, and define ⎧⎡ T ⎫ ⎤ ⎨ x Ax ⎬  ⎣ xTBx ⎦ : x ∈ Rn ⊆ R3. Φ(A, B, C) = ⎩ ⎭ xTCx Then, the following statements are equivalent: i) There exist real numbers α, β, γ such that αA+βB +γC is positive definite. ii) Φ(A, B, C) is a pointed, one-sided, closed, convex cone, and, if x ∈ Rn satisfies xTAx = xTBx = xTCx = 0, then x = 0. Proof: See [1117]. Fact 8.14.15. Let A ∈ Fn×n, assume that A is Hermitian, let b ∈ Fn and c ∈ R, and define f : Fn → R by f(x) = x∗Ax + Re(b∗x) + c. 

Then, the following statements hold: i) f is convex if and only if A is positive semidefinite. ii) f is strictly convex if and only if A is positive definite. Now, assume in addition that A is positive semidefinite. Then, f has a minimizer if and only if b ∈ R(A). In this case, the following statements hold.

549

POSITIVE-SEMIDEFINITE MATRICES

iii) The vector x0 ∈ Fn is a minimizer of f if and only if x0 satisfies Ax0 = − 21 b. iv) x0 ∈ Fm minimizes f if and only if there exists a vector y ∈ Fm such that x0 = − 21A+b + (I − A+A)y. v) The minimum of f is given by f(x0 ) = c − x∗0 Ax0 = c − 14 b∗A+b. vi) If A is positive definite, then x0 = − 21A−1b is the unique minimizer of f, and the minimum of f is given by f(x0 ) = c − x∗0 Ax0 = c − 14 b∗A−1b. Proof: Use Proposition 6.1.7 and note that, for every x0 satisfying Ax0 = − 12 b, it follows that f (x0 ) = (x − x0 )∗A(x − x0 ) + c − x∗0 Ax0 = (x − x0 )∗A(x − x0 ) + c − 14 b∗A+b. Remark: This result is the quadratic minimization lemma. Remark: See Fact 9.15.4. Fact 8.14.16. Let A ∈ Fn×n, assume that A is positive definite, and define  → R by φ(B) = tr BAB ∗. Then, φ is strictly convex. φ: F m×n

Proof: tr[α(1 − α)(B1 − B2 )A(B1 − B2 )∗ ] > 0. Fact 8.14.17. Let p, q ∈ R, and define φ: Pn × Pn → (0, ∞) by  φ(A, B) = tr ApB q.

Then, the following statements hold: i) If p, q ∈ (0, 1) and p + q ≤ 1, then −φ is convex. ii) If either p, q ∈ [−1, 0) or p ∈ [−1, 0), q ∈ [1, 2], and p + q ≥ 1, or p ∈ [1, 2], q ∈ [−1, 0], and p + q ≥ 1, then φ is convex. iii) If p, q do not satisfy the hypotheses of either i) or ii), then neither φ nor −φ is convex. Proof: See [170]. Fact 8.14.18. Let B ∈ Fn×n, assume that B is Hermitian, let α1, . . . , αk ∈  k n (0, ∞), define r = i=1 αi , assume that r ≤ 1, let q ∈ R, and define φ: P × · · · × n P → [0, ∞) by q  k  φ(A1, . . . , Ak ) = − tr eB+ i=1 αi log Ai . If q ∈ (0, 1/r], then φ is convex. Furthermore, if q < 0, then −φ is convex. Proof: See [931, 958]. Remark: See [1014] and Fact 8.12.32.

550

CHAPTER 8

8.15 Facts on Quadratic Forms Fact 8.15.1. Let G = (X, R) be a symmetric graph, where X = {x1, . . . , xn }. Then, for all z ∈ Rn, it follows that  zTLz = 12 (z(i) − z(j) )2, where the sum is over the set {(i, j): (xi , xj ) ∈ R}. Proof: See [275, pp. 29, 30] or [1018]. Fact 8.15.2. Let A ∈ Fn×n, and assume that A is Hermitian. Then, N(A) ⊆ {x ∈ Fn : x∗Ax = 0}. Furthermore,

N(A) = {x ∈ Fn : x∗Ax = 0}

if and only if either A ≥ 0 or A ≤ 0. Fact 8.15.3. Let x, y ∈ Fn. Then, xx∗ ≤ yy ∗ if and only if there exists α ∈ F such that |α| ∈ [0, 1] and x = αy. Fact 8.15.4. Let x, y ∈ Fn. Then, xy ∗ + yx∗ ≥ 0 if and only if x and y are linearly dependent. Proof: Evaluate the product of the nonzero eigenvalues of xy ∗ + yx∗, and use the Cauchy-Schwarz inequality |x∗y|2 ≤ x∗xy ∗y. Fact 8.15.5. Let A ∈ Fn×n, assume that A is positive definite, let x ∈ Fn, and let a ∈ [0, ∞). Then, the following statements are equivalent: i) xx∗ ≤ aA. ii) x∗A−1x ≤ a.   A x ≥ 0. iii) x∗ a Proof: Use Fact 2.14.3 and Proposition 8.2.4. Note that, if a = 0, then x = 0. Fact 8.15.6. Let A, B ∈ Fn×n, assume that A and B are Hermitian, assume  (A + B)−1 (Aa + Bb). that A + B is nonsingular, let x, a, b ∈ Fn , and define c = Then, (x−a)∗A(x−a)+(x−b)∗B(x−b) = (x−c)∗ (A+B)(x−c) = (a−b)∗A(A+B)−1B(a−b). Proof: See [1215, p. 278]. Fact 8.15.7. Let A, B ∈ Rn×n, assume that A is symmetric and B is skew symmetric, and let x, y ∈ Rn . Then,  T    x x A B = (x + jy)∗ (A + jB)(x + jy). y y BT A Remark: See Fact 4.10.27.

551

POSITIVE-SEMIDEFINITE MATRICES

Fact 8.15.8. Let A ∈ Fn×n, assume that A is positive definite, and let x, y ∈ F . Then, 2Re x∗y ≤ x∗Ax + y ∗A−1y. n

Furthermore, if y = Ax, then equality holds. Therefore, x∗Ax = maxn [2Re x∗z − z ∗Az]. z∈F

 ∗  Proof: A1/2x − A−1/2y A1/2x − A−1/2y ≥ 0. Remark: This result is due to Bellman. See [911, 1530]. Fact 8.15.9. Let A ∈ Fn×n, assume that A is positive definite, and let x, y ∈ F . Then,   |x∗y|2 ≤ (x∗Ax) y ∗A−1y . n

Proof: Use Fact 8.11.14 with A replaced by A1/2x and B replaced by A−1/2 y. Fact 8.15.10. Let A ∈ Fn×n, assume that A is positive definite, and let x ∈ F . Then,   (α + β)2 ∗ 2 (x x) , (x∗x)2 ≤ (x∗Ax) x∗A−1x ≤ 4αβ n

  λmin(A) and β = λmax(A). where α =

Remark: The second inequality is the Kantorovich inequality. See Fact 1.17.37 and [24]. See also [952]. Fact 8.15.11. Let A ∈ Fn×n, assume that A is positive definite, and let x ∈ Fn. Then, (α − β)2 ∗ xx (x∗x)1/2 (x∗Ax)1/2 − x∗Ax ≤ 4(α + β) and

(x∗x)(x∗A2 x) − (x∗Ax)2 ≤ 14 (α − β)2 (x∗x)2, 



where α = λmin(A) and β = λmax(A). Proof: See [1106]. Remark: Extensions of these results are given in [770, 1106]. Fact 8.15.12. Let A ∈ Fn×n, assume that A is positive semidefinite, let / N(A). Then, r = rank A, let x ∈ Fn, and assume that x ∈ 

x∗Ax x∗x 1/2 2 − ≤ [λ1/2 max(A) − λr (A)] . x∗x x∗A+x If, in addition, A is positive definite, then, for all nonzero x ∈ Fn, x∗x x∗Ax 1/2 2 − ≤ [λ1/2 max(A) − λmin(A)] . x∗x x∗A−1x Proof: See [1041, 1106]. The left-hand inequality in the last string of inequalities is given by Fact 8.15.10. 0≤

552

CHAPTER 8

Fact 8.15.13. Let A ∈ Fn×n, assume that A is positive definite, let y ∈ Fn,  R by f(x) = |x∗y|2. Then, let α > 0, and define f : Fn → " α A−1 y x0 = y ∗A−1y minimizes f(x) subject to x∗Ax ≤ α. Furthermore, f(x0 ) = αy ∗A−1y. Proof: See [33]. Fact 8.15.14. Let A ∈ Fn×n, assume that A is positive semidefinite, and let x ∈ F . Then,  ∗ 2 2   x A x ≤ (x∗Ax) x∗A3x n

  (x∗Ax)2 ≤ (x∗x) x∗A2x .

and

Proof: Apply the Cauchy-Schwarz inequality given by Corollary 9.1.7. Fact 8.15.15. Let A ∈ Fn×n, assume that A is positive semidefinite, and let x ∈ F . If α ∈ [0, 1], then n

x∗Aαx ≤ (x∗x)1−α (x∗Ax) . α

Furthermore, if α > 1, then (x∗Ax)α ≤ (x∗x)α−1x∗Aαx. Remark: The first inequality is the H¨ older-McCarthy inequality, which is equivalent to the Young inequality. See Fact 8.9.43, Fact 8.10.43, [544, p. 125], and [546]. Matrix versions of the second inequality are given in [715]. Fact 8.15.16. Let A ∈ Fn×n, assume that A is positive semidefinite, let x ∈ Fn, and let α, β ∈ [1, ∞), where α ≤ β. Then, (x∗Aαx)1/α ≤ (x∗Aβx)1/β . Now, assume in addition that A is positive definite. Then, x∗ (log A)x ≤ log x∗Ax ≤

∗ α 1 α log x A x

≤

∗ β 1 β log x A x.

Proof: See [522]. Fact 8.15.17. Let A ∈ Fn×n, x, y ∈ Fn, and α ∈ (0, 1). Then, |x∗Ay| ≤ Aαx2 A∗ 1−αy2 . Consequently,

|x∗Ay| ≤ [x∗Ax]1/2 [y ∗A∗ y]1/2.

Proof: See [797]. Fact 8.15.18. Let A, B ∈ Fn×n, assume that A is positive semidefinite, assume that AB is Hermitian, and let x ∈ Fn. Then, |x∗ABx| ≤ sprad(B)x∗Ax. Proof: See [937].

553

POSITIVE-SEMIDEFINITE MATRICES

Remark: This result is the sharpening by Halmos of Reid’s inequality. Related results are given in [938]. Fact 8.15.19. Let A, B ∈ Fn×n, assume that A and B are positive definite, and let x ∈ Fn. Then,   x∗A−1xx∗B −1x ≤ 14 x∗A−1x + x∗B −1x . x∗ (A + B)−1x ≤ ∗ −1 ∗ −1 xA x+x B x In particular, 1 1 1 + ≤ . −1 −1 (A )(i,i) (B )(i,i) [(A + B)−1 ](i,i) Proof: See [973, p. 201]. The right-hand inequality follows from Fact 1.12.4. Remark: This result is Bergstrom’s inequality. Remark: This result is a special case of Fact 8.11.3, which is a special case of xvii) of Proposition 8.6.17. Fact 8.15.20. Let A, B ∈ Fn×m, assume that I −A∗A and I −B ∗B are positive semidefinite, and let x ∈ Cn. Then, x∗(I − A∗A)xx∗(I − B ∗B)x ≤ |x∗(I − A∗B)x|2. Remark: This result is due to Marcus. See [1087]. Remark: See Fact 8.13.26. Fact 8.15.21. Let A, B ∈ Rn, and assume that A is Hermitian and B is positive definite. Then,   x∗Ax λmax AB −1 = max{λ ∈ R: det(A − λB) = 0} = min . x∈Fn \{0} x∗Bx Proof: Use Lemma 8.4.3. Fact 8.15.22. Let A, B ∈ Fn×n, and assume that A is positive definite and B is positive semidefinite. Then, 4(x∗x)(x∗Bx) < (x∗Ax)2 for all nonzero x ∈ Fn if and only if there exists α > 0 such that αI + α−1B < A. In this case, 4B < A2, and hence 2B 1/2 < A. Proof: Sufficiency follows from αx∗x + α−1x∗Bx < x∗Ax. Necessity follows from Fact 8.15.23. The last result follows from (A − 2αI)2 ≥ 0 or 2B 1/2 ≤ αI + α−1B. Fact 8.15.23. Let A, B, C ∈ Fn×n, assume that A, B, C are positive semidefinite, and assume that 4(x∗Cx)(x∗Bx) < (x∗Ax)2 for all nonzero x ∈ Fn. Then, there exists α > 0 such that αC + α−1B < A. Proof: See [1110].

554

CHAPTER 8

Fact 8.15.24. Let A, B ∈ Fn×n, and assume that A is Hermitian and B is positive semidefinite. Then, x∗Ax < 0 for all x ∈ Fn such that Bx = 0 and x = 0 if and only if there exists α > 0 such that A < αB. Proof: To prove necessity, suppose that, for every α > 0, there exists a nonzero vector x such that x∗Ax ≥ αx∗Bx. Now, Bx = 0 implies that x∗Ax ≥ 0. Sufficiency is immediate. Fact 8.15.25. Let A, B ∈ Cn×n, and assume that A and B are Hermitian. Then, the following statements are equivalent: i) There exist α, β ∈ R such that αA + βB is positive definite. ii) {x ∈ Cn : x∗Ax = x∗Bx = 0} = {0}. Remark: This result is Finsler’s lemma. See [86, 167, 891, 1373, 1385]. Remark: See Fact 8.15.26, Fact 8.17.5, and Fact 8.17.6. Fact 8.15.26. Let A, B ∈ Rn×n, and assume that A and B are symmetric. Then, the following statements are equivalent: i) There exist α, β ∈ R such that αA + βB is positive definite. ii) Either xTAx > 0 for all nonzero x ∈ {y ∈ Fn : y TBy = 0} or xTAx < 0 for all nonzero x ∈ {y ∈ Fn : y TBy = 0}. Now, assume in addition that n ≥ 3. Then, the following statement is equivalent to i) and ii): iii) {x ∈ Rn : xTAx = xTBx = 0} = {0}. Remark: This result is related to Finsler’s lemma. See [86, 167, 1385]. Remark: See Fact 8.15.25, Fact 8.17.5, and Fact 8.17.6. Fact 8.15.27. Let A, B ∈ Cn×n, assume that A and B are Hermitian, and assume that x∗ (A + jB)x is nonzero for all nonzero x ∈ Cn. Then, there exists t ∈ [0, π) such that (sin t)A + (cos t)B is positive definite. Proof: See [363] or [1261, p. 282]. Fact 8.15.28. Let A ∈ Rn×n, assume that A is symmetric, and let B ∈ Rn×m. Then, the following statements are equivalent: i) xTAx > 0 for all nonzero x ∈ N(B T ).   A B = n. ii) ν+ BT 0 Furthermore, the following statements are equivalent: iii) xTAx ≥ 0 for all x ∈ N(B T ).   A B = rank B. iv) ν− BT 0 Proof: See [307, 970].

POSITIVE-SEMIDEFINITE MATRICES

555

Remark: See Fact 5.8.21 and Fact 8.15.29. n×m , Fact 8.15.29. Let A ∈ Rn×n , assume

that A is symmetric, let B ∈ R where m ≤ n, and assume that Im 0 B is nonsingular. Then, the following statements are equivalent:

i) xTAx > 0 for all nonzero x ∈ N(B T ). ii) For all i ∈ {m + 1, . . . , n}, the  sign of the i × i leading principal subdeter0 BT is (−1)m. minant of the matrix B A

Proof: See [97, p. 20], [961, p. 312], or [980]. Remark: See Fact 8.15.28. Fact 8.15.30. Let A ∈ Fn×n, assume that A is positive semidefinite and nonzero, let x, y ∈ Fn, and assume that x∗y = 0. Then, 2  λmax(A) − λmin (A) 2 ∗ (x∗Ax)(y ∗Ay). |x Ay| ≤ λmax(A) + λmin(A) Furthermore, there exist vectors x, y ∈ Fn satisfying x∗y = 0 for which equality holds. Proof: See [730, p. 443] or [911, 1530]. Remark: This result is the Wielandt inequality. A B  Fact 8.15.31. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, define A = B∗ C , and assume that A and C are positive semidefinite. Then, the following statements are equivalent: i) A is positive semidefinite. ii) |x∗By|2 ≤ (x∗Ax)(y ∗Cy) for all x ∈ Fn and y ∈ Fm. iii) 2|x∗By| ≤ x∗Ax + y ∗Cy for all x ∈ Fn and y ∈ Fm. If, in addition, A and C are positive definite, then the following statement is equivalent to i)–iii):   iv) sprad B ∗A−1BC −1 ≤ 1. Finally, if A is positive semidefinite and nonzero, then, for all x ∈ Fn and y ∈ Fm, 2  λmax(A) − λmin(A) ∗ 2 (x∗Ax)(y ∗Cy). |x By| ≤ λmax(A) + λmin(A) Proof: See [728, p. 473] and [911, 1530]. Fact 8.15.32. Let A ∈ Fn×n, assume that A is Hermitian, let x, y ∈ Fn, and assume that x∗x = y ∗y = 1 and x∗y = 0. Then, 2|x∗Ay| ≤ λmax(A) − λmin(A). Furthermore, there exist vectors x, y ∈ Fn satisfying x∗x = y ∗y = 1 and x∗y = 0 for which equality holds.

556

CHAPTER 8

Proof: See [911, 1530]. Remark: λmax(A) − λmin(A) is the spread of A. See Fact 9.9.30 and Fact 9.9.31. Fact 8.15.33. Let n ≤ 4, let A ∈ Rn×n, assume that A is symmetric, and assume that, for all nonnegative vectors x ∈ Rn, xTAx ≥ 0. Then, there exist B, C ∈ Rn×n such that B is positive semidefinite, C is symmetric and nonnegative, and A = B + C. Remark: This result does not hold for all n > 5. Hence, this result is an example of the quartic barrier. See [359], Fact 8.14.7, and Fact 11.17.3. Remark: A is copositive.

8.16 Facts on the Gaussian Density Fact 8.16.1. Let A ∈ Rn×n, and assume that A is positive definite. Then,

R

T π n/2 e−x Ax dx = √ . det A n

Remark: See Fact 11.13.16. Fact 8.16.2. Let A ∈ Rn×n, assume that A is positive definite, and define f : R → R by 1 T −1 e− 2 x A x √ . f (x) = (2π)n/2 det A n

Then, f (x) dx = 1, Rn

f (x)xxT dx = A, Rn

and − f (x) log f (x) dx = 12 log[(2πe)n det A]. Rn

Proof: See [360] or use Fact 8.16.5. Remark: f is the multivariate normal density. The last expression is the entropy. Fact 8.16.3. Let A, B ∈ Rn×n, assume that A and B are positive definite, and, for k = 0, 1, 2, 3, define  T k − 1 xTA−1x 1  √ Ik = x Bx e 2 dx. (2π)n/2 det A n R

557

POSITIVE-SEMIDEFINITE MATRICES

Then,

I0 = 1, I1 = tr AB, I2 = (tr AB)2 + 2 tr (AB)2,

I3 = (tr AB)3 + 6(tr AB) tr (AB)2 + 8 tr (AB)3.

Proof: See [1027, p. 80]. Remark: These equalities are Lancaster’s formulas. Fact 8.16.4. Let A, B, C ∈ Rn×n, assume that A is positive definite, assume that B and C are symmetric, and let μ ∈ Rn. Then, 1 √ det A

(2π)n/2

T −1

xTBxxTCxe− 2 (x−μ) A 1

Rn

(x−μ)

dx = tr(AB) tr(AC) + 2 tr(ACAB)

+ tr(AB)μTCμ + 4μTBACμ + μTBμ tr(CA) + μTBμμTCμ.

Proof: See [1208, p. 418, 419]. Remark: Setting μ = 0 and C = B yields I2 of Fact 8.16.3. Fact 8.16.5. Let A ∈ Rn×n, assume that A is positive definite, let B ∈ Rn×n, let a, b ∈ Rn, and let α, β ∈ R. Then,  T  T T x Bx + bTx + β e−(x Ax+a x+α) dx Rn



1 T −1  π n/2 = √ 2β + tr A−1B − bTA−1a + 12 aTA−1BA−1a e 4 a A a−α. 2 det A

Proof: See [671, p. 322]. Fact 8.16.6. Let A1, A2 ∈ Rn×n, assume that A1 and A2 are positive definite, and let μ1, μ2 ∈ Rn. Then, 1

e 2 (x−μ1 )

T −1 A1 (x−μ1 )

where

1

e 2 (x−μ2 )

T −1 A2 (x−μ2 )

1

= αe 2 (x−μ3 )

T −1 A3 (x−μ3 )

,

−1 −1 A3 = (A−1 1 + A2 ) , 

−1 μ3 = A3 (A−1 1 μ1 + A2 μ2 ), 

and



1

T −1

α = e 2 (μ1A1

−1 T −1 μ1 +μT 2A2 μ2 −μ3A3 μ3 )

.

Remark: A product of Gaussian densities is a weighted Gaussian density.

558

CHAPTER 8

8.17 Facts on Simultaneous Diagonalization Fact 8.17.1. Let A, B ∈ Fn×n, assume that A and B are Hermitian. Then, the following statements are equivalent: i) There exists a unitary matrix S ∈ Fn×n such that SAS ∗ and SBS ∗ are diagonal. ii) AB = BA. iii) AB and BA are Hermitian. If, in addition, A is nonsingular, then the following condition is equivalent to i)–iii): iv) A−1B is Hermitian. Proof: See [178, p. 208], [459, pp. 188–190], or [728, p. 229]. Remark: The equivalence of i) and ii) is given by Fact 5.17.7. Fact 8.17.2. Let A, B ∈ Fn×n, assume that A and B are Hermitian, and assume that A is nonsingular. Then, there exists a nonsingular matrix S ∈ Fn×n such that SAS ∗ and SBS ∗ are diagonal if and only if A−1B is diagonalizable over R. Proof: See [728, p. 229] or [1125, p. 95]. Fact 8.17.3. Let A, B ∈ Fn×n, assume that A and B are symmetric, and assume that A is nonsingular. Then, there exists a nonsingular matrix S ∈ Fn×n such that SAS T and SBS T are diagonal if and only if A−1B is diagonalizable. Proof: See [728, p. 229] and [1385]. Remark: A and B are complex symmetric. Fact 8.17.4. Let A, B ∈ Fn×n, and assume that A and B are Hermitian. Then, there exists a nonsingular matrix S ∈ Fn×n such that SAS ∗ and SBS ∗ are diagonal if and only if there exists a positive-definite matrix M ∈ Fn×n such that AMB = BMA. Proof: See [86]. Fact 8.17.5. Let A, B ∈ Fn×n, assume that A and B are Hermitian, and assume there exist α, β ∈ R such that αA + βB is positive definite. Then, there exists a nonsingular matrix S ∈ Fn×n such that SAS ∗ and SBS ∗ are diagonal. Proof: See [728, p. 465]. Remark: This result extends a result due to Weierstrass. See [1385]. Remark: Suppose that B is positive definite. Then, by necessity of Fact 8.17.2, it follows that A−1B is diagonalizable over R, which proves iii) =⇒ i) of Proposition 5.5.12. Remark: See Fact 8.17.6.

559

POSITIVE-SEMIDEFINITE MATRICES

Fact 8.17.6. Let A, B ∈ Fn×n, assume that A and B are Hermitian, assume that {x ∈ Fn : x∗Ax = x∗Bx = 0} = {0}, and, if F = R, assume that n ≥ 3. Then, there exists a nonsingular matrix S ∈ Fn×n such that SAS ∗ and SBS ∗ are diagonal. Proof: This result follows from Fact 5.17.9. See [975] or [1125, p. 96]. Remark: For F = R, this result is due to Pesonen and Milnor. See [1385]. Remark: See Fact 5.17.9, Fact 8.15.25, Fact 8.15.26, and Fact 8.17.5.

8.18 Facts on Eigenvalues and Singular Values for One Matrix Fact 8.18.1. Let A = ab cb ∈ F2×2, assume that A is Hermitian, and let mspec(A) = {λ1, λ2 }ms . Then, 2|b| ≤ λ1 − λ2 . Now, assume in addition that A is positive semidefinite. Then,    √ 2|b| ≤ λ1 − λ2 λ1 + λ2 . If c > 0, then

If a > 0 and c > 0, then

  |b| √ ≤ λ1 − λ2 . c λ − λ2 |b| √ ≤ 1 . ac λ1 + λ2

Finally, if A is positive definite, then λ1 − λ2 |b| ≤ √ a 2 λ1 λ2 and

λ2 − λ22 4|b| ≤ √1 . λ1 λ2

Proof: See [911, 1530]. Remark: These inequalities are useful for deriving inequalities involving quadratic forms. See Fact 8.15.30 and Fact 8.15.31. Fact 8.18.2. Let A ∈ Fn×m. Then, for all i ∈ {1, . . . , min{n, m}}, λi (A) = σi (A). Hence,



min{n,m}

tr A =

σi (A).

i=1

Fact 8.18.3. Let A ∈ Fn×n, and define   A∗ σmax (A)I  . A= A σmax (A)I

560

CHAPTER 8

Then, A is positive semidefinite. Furthermore, * A + A∗  ≤ 2σmax (A)I 2

∗ ≤ σmax (A) + 1 I. A + A  ≤ ∗ AA + I Proof: See [1528]. Fact 8.18.4. Let A ∈ Fn×n. Then, for all i ∈ {1, . . . , n},

−σi (A) ≤ λi 12 (A + A∗ ) ≤ σi (A). Hence,

|tr A| ≤ tr A.

Proof: See [1242]. Remark: See Fact 5.11.25. Fact 8.18.5. Let A ∈ Fn×n, and let mspec(A) = {λ1, . . . , λn }ms , where λ1, . . . , λn are ordered such that |λ1 | ≥ · · · ≥ |λn |. If p > 0, then, for all k ∈ {1, . . . , n}, k 

|λi |p ≤

i=1

k 

σip(A).

i=1

In particular, for all k ∈ {1, . . . , n}, k 

|λi | ≤

i=1

Hence, |tr A| ≤

n 

k 

σi (A).

i=1

|λi | ≤

i=1

n 

σi (A) = tr A.

i=1

Furthermore, for all k ∈ {1, . . . , n}, k 

|λi |2 ≤

i=1

k 

σi2(A).

i=1

Hence, Re tr A2 ≤ |tr A2 | ≤

n 

|λi |2 ≤

n n    @ A  σi A2 = tr A2 ≤ σi2(A) = tr A∗A.

i=1

Furthermore,

i=1 n 

i=1

|λi |2 = tr A∗A

i=1

if and only if A is normal. Finally, n 

λ2i = tr A∗A

i=1

if and only if A is Hermitian. Proof: This result follows from Fact 2.21.12 and Fact 5.11.28. See [201, p. 42], [730, p. 176], or [1521, p. 19]. See Fact 9.13.16 for the inequality tr A2  =

561

POSITIVE-SEMIDEFINITE MATRICES

 1/2 tr A2∗A2 ≤ tr A∗A. See Fact 3.7.13 and Fact 5.14.14. Remark: The first result is Weyl’s inequalities. The result Schur’s inequality. See Fact 9.11.3.

n i=1

|λi |2 ≤ tr A∗A is

Problem: Determine when equality holds for the remaining inequalities. Fact 8.18.6. Let A ∈ Fn×n, let mspec(A) = {λ1, . . . , λn }ms , where λ1, . . . , λn are ordered such that |λ1 | ≥ · · · ≥ |λn |, and let r > 0. Then, for all k ∈ {1, . . . , n}, k k ! ! (1 + r|λi |) ≤ [1 + σi (A)]. i=1

i=1

Proof: See [459, p. 222]. Fact 8.18.7. Let A ∈ Fn×n. Then, * |tr A2 | ≤

tr AA∗ 

tr A2  ≤ tr A2 = tr A∗A.

Proof: For the upper inequality, see [911, 1530]. For the lower inequalities, use Fact 8.18.4 and Fact 9.11.3. Remark: See Fact 5.11.10, Fact 9.13.16, and Fact 9.13.17. Fact 8.18.8. Let A ∈ Fn×n, and assume that A is Hermitian. Then, for all k ∈ {1, . . . , n}, k k   di(A) ≤ λi(A) i=1

i=1

with equality for k = n, that is, tr A =

n 

di(A) =

i=1

n 

λi(A).

i=1



T That is, λ1(A) · · · λn(A) strongly majorizes d1(A) · · · thus, for all k ∈ {1, . . . , n}, n 

λi(A) ≤

i=k

n 

dn(A)

T

, and

di(A).

i=k

In particular,

λmin (A) ≤ dmin (A) ≤ dmax (A) ≤ λmax (A).

T is an element of the convex hull of Furthermore, the vector d1(A) · · · dn(A)

T . the n! vectors obtaining by permuting the components of λ1(A) · · · λn(A) Proof: See [201, p. 35], [728, p. 193], [996, p. 218], or [1521, p. 18]. The last statement follows from Fact 3.9.6. Remark: This result is Schur’s theorem. Remark: See Fact 8.12.3.

562

CHAPTER 8

Fact 8.18.9. Let A ∈ Fn×n, assume that A is Hermitian, let k denote the number of positive diagonal entries of A, and let l denote the number of positive eigenvalues of A. Then, k l   d2i (A) ≤ λ2i (A). i=1

i=1

Proof: Write A = B + C, where B is positive semidefinite, C is negative semidefinite, and mspec(A) = mspec(B) ∪ mspec(C). Furthermore, without loss of generality, assume that A(1,1) , . . . , A(k,k) are the positive diagonal entries of A. Then, k  i=1

d2i (A) =

k 

A2(i,i) ≤

i=1

=

k 

(A(i,i) − C(i,i) )2

i=1

k 

2 B(i,i) ≤

i=1

n 

2 B(i,i) ≤ tr B 2 =

i=1

l 

λ2i (A).

i=1

Remark: This inequality can be written as tr (A + |A|)◦2 ≤ tr (A + A)2. Remark: This result is due to Y. Li. Fact 8.18.10. Let x, y ∈ Rn, where n ≥ 2. Then, the following statements are equivalent: i) y strongly majorizes by x. ii) x is an element of the convex hull of the vectors y1, . . . , yn! ∈ Rn, where each of these n! vectors is formed by permuting the components of y.

T iii) There exists a Hermitian matrix A ∈ Cn×n such that A(1,1) · · · A(n,n) = x and mspec(A) = {y(1) , . . . , y(n) }ms . Remark: This result is the Schur-Horn theorem. Schur’s theorem given by Fact 8.18.8 is iii) =⇒ i), while the result i) =⇒ iii) is due to [727]. The equivalence of ii) is given by Fact 3.9.6. The significance of this result is discussed in [157, 202, 266]. Remark: An equivalent version is given by Fact 3.11.14. Fact 8.18.11. Let A ∈ Fn×n, and assume that A is positive semidefinite. Then, for all k ∈ {1, . . . , n}, n !

λi(A) ≤

i=k

In particular, det A ≤

n !

di(A).

i=k n !

A(i,i) .

i=1

Now, assume in addition that A is positive definite. Then, equality holds if and only if A is diagonal. Proof: See [544, pp. 21–24], [728, pp. 200, 477], or [1521, p. 18].

563

POSITIVE-SEMIDEFINITE MATRICES

Remark: The case k = 1 is Hadamard’s inequality. Remark: See Fact 8.13.35 and Fact 9.11.1. Remark: A strengthened version is given by Fact 8.13.34. Remark: A geometric interpretation is discussed in [553]. 



Fact 8.18.12. Let A ∈ Fn×n, define H = 12 (A + A∗ ) and S = 12 (A − A∗ ), and assume that H is positive definite. Then, the following statements hold: i) A is nonsingular. ii)

1 −1 2 (A

+ A−∗ ) = (H + S ∗H −1S)−1.

iii) σmax (A−1 ) ≤ σmax (H −1 ). iv) σmax (A) ≤ σmax (H + S ∗H −1S). Proof: See [1003]. Remark: See Fact 8.9.32 and Fact 8.13.11. Fact 8.18.13. Let A ∈ Fn×n, and assume that A is Hermitian. {A(1,1) , . . . , A(n,n) }ms = mspec(A) if and only if A is diagonal.

Then,

Proof: Apply Fact 8.18.11 with A + βI > 0. I A is positive semidefinite if and Fact 8.18.14. Let A ∈ Fn×n . Then, A∗ I

I A only if σmax (A) ≤ 1. Furthermore, A∗ I is positive definite if and only if σmax (A) < 1. Proof: Note that 

I A∗

A I



 =

I A∗

0 I



I 0

0 I − A∗A



I 0

A I

 .

Fact 8.18.15. Let A ∈ Fn×n, and assume that A is Hermitian. Then, for all k ∈ {1, . . . , n}, k  λi = max{tr S ∗AS: S ∈ Fn×k and S ∗S = Ik } i=1

and

n 

λi = min{tr S ∗AS: S ∈ Fn×k and S ∗S = Ik }.

i=n+1−k

Proof: See [728, p. 191]. Remark: This result is the minimum principle. Fact 8.18.16. Let A ∈ Fn×n, assume that A is Hermitian, and let S ∈ Rk×n satisfy SS ∗ = Ik . Then, for all i ∈ {1, . . . , k}, λi+n−k(A) ≤ λi(SAS ∗ ) ≤ λi(A).

564

CHAPTER 8

Consequently,

k 

λi+n−k (A) ≤ tr SAS ∗ ≤

i=1

and

k !

k 

λi(A)

i=1

λi+n−k(A) ≤ det SAS ∗ ≤

i=1

k !

λi(A).

i=1

Proof: See [728, p. 190] or [1208, p. 111]. Remark: This result is the Poincar´e separation theorem.

8.19 Facts on Eigenvalues and Singular Values for Two or More Matrices Fact 8.19.1. Let A ∈ Fn×n, B Fn×m and C ∈ Fm×m, and assume that A A∈ B

, (n+m)×(n+m) and C are positive definite. Then, B ∗ C ∈ F is positive semidefinite if and only if σmax (A−1/2BC −1/2 ) ≤ 1.

(n+m)×(n+m) is positive definite if and only if Furthermore, BA∗ B C ∈F σmax (A−1/2BC −1/2 ) < 1. Proof: See [989]. Fact 8.19.2. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, assume that A and C are positive definite, and assume that

Then,

then





A B B∗ C

B

A B∗ C

2 σmax (B) ≤ σmin (A)σmin (C).

∈ F(n+m)×(n+m) is positive semidefinite. If, in addition, 2 σmax (B) < σmin (A)σmin (C),

∈ F(n+m)×(n+m) is positive definite.

Proof: Note that 2 (A−1/2BC −1/2 ) ≤ λmax (A−1/2BC −1B ∗A−1/2 ) σmax

≤ σmax (C −1 )λmax (A−1/2BB ∗A−1/2 ) ≤

1 λmax (B ∗A−1B) σmin (C)

≤

σmax (A−1 ) λmax (B ∗B) σmin (C)

=

1 σ 2 (B) σmin (A)σmin (C) max

≤ 1. The result now follows from Fact 8.19.1.

565

POSITIVE-SEMIDEFINITE MATRICES

Fact 8.19.3. Let A, B ∈ Fn, assume that A and B are Hermitian, and define

γ1 · · · γn , where the components of γ are the components of γ



= λ1(A) · · · λn(A) + λn(B) · · · λ1(B) arranged in decreasing order. Then, for all k ∈ {1, . . . , n}, k k   γi ≤ λi(A + B). 

i=1

i=1

Proof: This result follows from the Lidskii-Wielandt inequalities. See [201, p. 71] or [202, 388]. Remark: This result provides an alternative lower bound for (8.6.13). Fact 8.19.4. Let A, B ∈ Hn, let k ∈ {1, . . . , n}, and let 1 ≤ i1 ≤ · · · ≤ ik ≤ n. Then, k 

n 

λij (A) +

j=1

λj (B)] ≤

j=n−k+1

k 

λij (A + B) ≤

j=1

k 

[λij (A) + λj (B)].

j=1

Proof: See [1208, pp. 115, 116]. Fact 8.19.5. Let f : R → R be convex, define f : Hn → Hn by (8.5.2), let A, B ∈ Fn×n, and assume that A and B are Hermitian. Then, for all α ∈ [0, 1],

αλ1 [f(A)] + (1 − α)λ1 [f(B)] · · · αλn [f(A)] + (1 − α)λn [f(B)] weakly majorizes λ1 [f(αA + (1 − α)B)]

···

λn [f(αA + (1 − α)B)] .

If, in addition, f is either nonincreasing or nondecreasing, then, for all i ∈ {1, . . . , n}, λi [f(αA + (1 − α)B)] ≤ αλi [f(A)] + (1 − α)λi [f(B)]. Proof: See [94]. Remark: Convexity of f : R → R does not imply convexity of f : Hn → Hn. Fact 8.19.6. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. If r ∈ [0, 1], then

T λ1(Ar + B r ) · · · λn(Ar + B r ) weakly majorizes



λ1[(A + B)r ] · · ·

λn[(A + B)r ]

T

,

and, for all i ∈ {1, . . . , n}, 21−rλi[(A + B)r ] ≤ λi(Ar + B r ). If r ≥ 1, then weakly majorizes



λ1 [(A + B)r ]

···

λn[(A + B)r ]

λ1(Ar + B r )

···

λn(Ar + B r )

T

T

,

566

CHAPTER 8

and, for all i ∈ {1, . . . , n}, λi(Ar + B r ) ≤ 2r−1λi[(A + B)r ]. Proof: This result follows from Fact 8.19.5. See [60, 92, 94]. Fact 8.19.7. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, for all k ∈ {1, . . . , n}, k 

σi2 (A + jB) ≤

i=1 n  i=1 k 

k  [σi2(A) + σi2(B)], i=1

σi2 (A + jB) =

n  [σi2(A) + σi2(B)], i=1

2 [σi2 (A + jB) + σn−i (A + jB)] ≤

i=1 n 

k  [σi2(A) + σi2(B)], i=1

2 [σi2 (A + jB) + σn−i (A + jB)] =

i=1

n  [σi2(A) + σi2(B)], i=1

and k k   2 [σi2(A) + σn−i (B)] ≤ σi2 (A + jB), i=1

i=1

n n   2 [σi2(A) + σn−i (B)] = σi2 (A + jB). i=1

i=1

Proof: See [54, 328]. Remark: The first equality is given by Fact 9.9.40. Fact 8.19.8. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, the following statements hold: i) If p ∈ [0, 1], then √ ii) If p ≥ 2, then

p (A − B). σmax (Ap − B p ) ≤ σmax

σmax (Ap − B p ) ≤ p[max{σmax (A), σmax (B)}]p−1 σmax (A − B). iii) If a and b are positive numbers such that aI ≤ A ≤ bI and aI ≤ B ≤ bI, then σmax (Ap − B p ) ≤ b[bp−2 + (p − 1)ap−2 ]σmax (A − B). Proof: See [210, 840].

567

POSITIVE-SEMIDEFINITE MATRICES

Fact 8.19.9. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, for all i ∈ {1, . . . , n},   A 0 . σi(A − B) ≤ σi 0 B Proof: See [1286, 1519]. Fact 8.19.10. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, and assume that A ∈ F(n+m)×(n+m) defined by   A B  A= B∗ C is positive semidefinite. Then, for all i ∈ {1, . . . , min{n, m}}, 2σi(B) ≤ σi(A). Proof: See [219, 1286]. Fact 8.19.11. Let A, B ∈ Fn×n. Then, 8 7 2 2 (A), σmax (B) − σmax (AB) ≤ σmax (A∗A − BB ∗ ) max σmax and

7 2 8 7 2 8 2 2 (A), σmax (B) − min σmin (A), σmin (B) . σmax (A∗A − BB ∗ ) ≤ max σmax

Furthermore, 8 7 2 8 7 2 2 2 (A), σmax (B) + min σmin (A), σmin (B) ≤ σmax (A∗A + BB ∗ ) max σmax and

7 2 8 2 (A), σmax (B) + σmax (AB). σmax (A∗A + BB ∗ ) ≤ max σmax

Now, assume in addition that A and B are positive semidefinite. Then, max{λmax (A), λmax (B)} − σmax (A1/2B 1/2 ) ≤ σmax (A − B) and σmax (A − B) ≤ max{λmax (A), λmax (B)} − min{λmin(A), λmin(B)}. Furthermore, max{λmax (A), λmax (B)} + min{λmin(A), λmin(B)} ≤ λmax (A + B) and

λmax (A + B) ≤ max{λmax (A), λmax (B)} + σmax (A1/2B 1/2 ).

Proof: See [848, 1522]. Remark: See Fact 8.19.14 and Fact 9.13.8.

568

CHAPTER 8

Fact 8.19.12. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, max{σmax (A), σmax (B)}−σmax (A1/2B 1/2 ) ≤ σmax (A − B) ≤ max{σmax (A), σmax (B)} ≤ σmax (A + B) *  max{σmax (A), σmax (B)} + σmax A1/2B 1/2 ≤ σmax (A) + σmax (B) ≤ 2 max{σmax (A), σmax (B)}. Proof: See [842, 848], and use Fact 8.19.13. Remark: See Fact 8.19.14. Fact 8.19.13. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite, and let k ≥ 1. Then, for all i ∈ {1, . . . , n},  

2σi A1/2 (A + B)k−1 B 1/2 ≤ λi (A + B)k . Hence,

2σmax (A1/2B 1/2 ) ≤ λmax (A + B)

and

σmax (A1/2B 1/2 ) ≤ max{λmax (A), λmax (B)}.

Proof: See Fact 8.19.11 and Fact 9.9.18. Fact 8.19.14. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, max{λmax (A), λmax (B)} − σmax (A1/2B 1/2 ) ≤ σmax (A − B) and λmax (A + B)   +   2 2 ≤ 12 λmax (A) + λmax (B) + [λmax (A) − λmax (B)] + 4σmax A1/2B 1/2 * ≤

max{λmax (A), λmax (B)} + σmax (A1/2B 1/2 ) λmax (A) + λmax (B).

Furthermore, λmax (A + B) = λmax (A) + λmax (B) if and only if

1/2 σmax (A1/2B 1/2 ) = λ1/2 max (A)λmax (B).

Proof: See [842, 845, 848]. Remark: See Fact 8.19.11, Fact 8.19.12, Fact 9.14.15, and Fact 9.9.46. Problem: Is σmax (A − B) ≤ σmax (A + B)?

569

POSITIVE-SEMIDEFINITE MATRICES

Fact 8.19.15. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then,   1/2 σmax A1/2B 1/2 ≤ σmax (AB). Equivalently,

    2 λmax A1/2BA1/2 ≤ λ1/2 max AB A .

Furthermore, AB = 0 if and only if A1/2B 1/2 = 0. Proof: See [842] and [848]. Fact 8.19.16. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then,  1/2 tr AB ≤ tr AB 2A ≤ 14 tr (A + B)2, tr (AB)2 ≤ tr A2B 2 ≤ and

1 16 tr (A

+ B)4,



σmax (AB) ≤ 14 σmax (A + B)2 *1 1 1 2 2 2 2 2 σmax (A + B ) ≤ 2 σmax (A ) + 2 σmax (B ) ≤ 1 2 1 2 4 σmax (A + B) ≤ 4 [σmax (A) + σmax (B)] 2 2 ≤ 12 σmax (A) + 12 σmax (B).

 1/2 Proof: See Fact 9.9.18. The inequalities tr AB ≤ tr AB 2A and tr (AB)2 ≤ 2 2 tr A B follow from Fact 8.12.22. Fact 8.19.17. Let A, B ∈ Fn×n, assume that A is positive semidefinite, and assume that B is positive definite. Then, for all i, j, k ∈ {1, . . . , n} such that j + k ≤ i + 1, λi(AB) ≤ λj (A)λk (B) and

λn−j+1 (A)λn−k+1 (B) ≤ λn−i+1 (AB).

In particular, for all i ∈ {1, . . . , n}, λi (A)λn (B) ≤ λi (AB) ≤ λi (A)λ1 (B). Proof: See [1208, pp. 126, 127]. Fact 8.19.18. Let A, B ∈ Fn×n, and assume that A and B are positive definite. Then, for all i = 1, . . . , n, λ2i (AB) λ2i (AB) ≤ λi(A)λi(B) ≤ . λ1(A)λ1(B) λn(A)λn(B) Proof: See [1208, p. 137].

570

CHAPTER 8

Fact 8.19.19. Let A, B ∈ Fn×n, assume that A is positive semidefinite, and assume that B is Hermitian. Then, for all k ∈ {1, . . . , n}, k 

k 

λi(A)λn−i+1(B) ≤

i=1

and

k 

λi(AB)

i=1

λn−i+1(AB) ≤

i=1

k 

λi(A)λi(B).

i=1

In particular, n 

λi(A)λn−i+1(B) ≤ tr AB ≤

i=1

n 

λi(A)λi(B).

i=1

Proof: See [862] and [1208, p. 128]. Remark: See Fact 5.12.4, Fact 5.12.5, Fact 5.12.8, and Proposition 8.4.13. Remark: The upper and lower bounds for tr AB are related to Fact 1.18.4. See [204, p. 140]. Fact 8.19.20. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, let λ1(AB) ≥ · · · ≥ λn(AB) ≥ 0 denote the eigenvalues of AB, and let 1 ≤ i1 < · · · < ik ≤ n. Then, k 

λij (A)λn−j+1(B) ≤

j=1

k 

λij (AB) ≤

j=1

k 

λij (A)λj (B).

j=1

Furthermore, for all k = 1, . . . , n, k 

λij (A)λn−ij +1(B) ≤

j=1

k 

λj (AB).

j=1

In particular, for all k = 1, . . . , n, k 

λi(A)λn−i+1(B) ≤

i=1

k 

λi(AB) ≤

i=1

k 

λi(A)λi(B).

i=1

Proof: See [1208, p. 128] and [1422]. Remark: See Fact 8.19.23 and Fact 9.14.27. Fact 8.19.21. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. If p ≥ 1, then n n p    λpi(A)λpn−i+1(B) ≤ tr B 1/2AB 1/2 ≤ tr ApB p ≤ λpi(A)λpi(B). i=1

i=1

If 0 ≤ p ≤ 1, then n  i=1

n p   λpi(A)λpn−i+1(B) ≤ tr ApB p ≤ tr B 1/2AB 1/2 ≤ λpi(A)λpi(B). i=1

571

POSITIVE-SEMIDEFINITE MATRICES

Now, suppose that A and B are positive definite. If p ≤ −1, then n n p    λpi(A)λpn−i+1(B) ≤ tr B 1/2AB 1/2 ≤ tr ApB p ≤ λpi(A)λpi(B). i=1

i=1

If −1 ≤ p ≤ 0, then n 

λpi(A)λpn−i+1(B)

n p   1/2 1/2 ≤ tr A B ≤ tr B AB ≤ λpi(A)λpi(B). p

p

i=1

i=1

Proof: See [1423]. See also [285, 906, 935, 1426]. Remark: See Fact 8.12.22. See Fact 8.12.15 for the indefinite case. Fact 8.19.22. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, for all k ∈ {1, . . . , n}, k !

λi(AB) ≤

i=1

k !

σi (AB) ≤

i=1

k !

λi(A)λi(B)

i=1

with equality for k = n. Furthermore, for all k ∈ {1, . . . , n}, n !

n !

λi(A)λi(B) ≤

i=k

n !

σi (AB) ≤

i=k

λi(AB)

i=k

with equality for k = 1. Proof: Use Fact 5.11.28 and Fact 9.13.18. Fact 8.19.23. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, let λ1(AB) ≥ · · · ≥ λn(AB) ≥ 0 denote the eigenvalues of AB, and let 1 ≤ i1 < · · · < ik ≤ n. Then, k !

λij (AB) ≤

j=1

k !

λij (A)λj (B)

j=1

with equality for k = n. Furthermore, k !

λij (A)λn−ij +1(B) ≤

j=1

k !

λj (AB)

j=1

with equality for k = n. In particular, k !

λi(A)λn−i+1(B) ≤

i=1

k !

λi(AB) ≤

i=1

with equality for k = n. Proof: See [1208, p. 127] and [1422]. Remark: The first inequality is due to Lidskii. Remark: See Fact 8.19.20 and Fact 9.14.27.

k ! i=1

λi(A)λi(B)

572

CHAPTER 8

Fact 8.19.24. Let A, B ∈ Fn×n, assume that A and B are positive definite, and let λ ∈ spec(A). Then,   λ2min(A)λ2min(B) 2

2 2 n n λ2 (A) + λ2 (B) < λ < 2 λmax(A) + λmax(B) . min min Proof: See [748]. Fact 8.19.25. Let A, B ∈ Fn×n, assume that A and B are positive definite, and define  λmax(A)  λmax(B) , kB = , kA = λmin(A) λmin(B) and 

γ=

√ √ ( kA + 1)2 kB ( kA − 1)2 √ √ − . kA kA

Then, if γ < 0, then 1 2 λmax(A)λmax(B)γ

≤ λmin(AB + BA) ≤ λmax(AB + BA) ≤ 2λmax(A)λmax(B),

whereas, if γ > 0, then 1 2 λmin(A)λmin(B)γ

≤ λmin(AB + BA) ≤ λmax(AB + BA) ≤ 2λmax(A)λmax(B). 

Furthermore, if

kA kB < 1 +



kA +

 kB ,

then AB + BA is positive definite. Proof: See [1065]. Fact 8.19.26. Let A, B ∈ Fn×n, assume that A is positive definite, assume that B is positive semidefinite, and let α > 0 and β > 0 be such that αI ≤ A ≤ βI. Then, √ √ σmax (AB) ≤ 2α+β sprad(AB) ≤ 2α+β σ (AB). αβ αβ max In particular,

σmax (A) ≤

α+β √ 2 αβ

sprad(A) ≤

α+β √ σ (A). 2 αβ max

Proof: See [1344]. Remark: The left-hand inequality is tightest for α = λmin (A) and β = λmax (A). Remark: This result is due to Bourin. Fact 8.19.27. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, the following statements hold: i) If q ∈ [0, 1], then and ii) If q ∈ [0, 1], then

q (AB) σmax (AqB q ) ≤ σmax q (BAB). σmax (B qAqB q ) ≤ σmax

λmax(AqB q ) ≤ λqmax(AB).

573

POSITIVE-SEMIDEFINITE MATRICES

iii) If q ≥ 1, then

q (AB) ≤ σmax (AqB q ). σmax

iv) If q ≥ 1, then

λqmax(AB) ≤ λmax(AqB q ).

v) If p ≥ q > 0, then

1/q 1/p (AqB q ) ≤ σmax (ApB p ). σmax

vi) If p ≥ q > 0, then

q q 1/p p p λ1/q max(A B ) ≤ λmax(A B ).

Proof: See [201, pp. 255–258] and [537]. Remark: See Fact 8.10.50, Fact 8.12.22, Fact 9.9.16, and Fact 9.9.17. Remark: ii) is the Cordes inequality. Fact 8.19.28. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let p ≥ r ≥ 0. Then,  T 1/p p p 1/p p p λ1 (A B ) · · · λn (A B ) strongly log majorizes 

1/r

λ1 (ArB r ) In fact, for all q > 0,

···

1/r

T

λn (ArB r )

.

det(AqB q )1/q = (det A)det B.

Proof: See [201, p. 257] or [1521, p. 20] and Fact 2.21.12. Fact 8.19.29. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, and assume that   A B  ∈ F(n+m)×(n+m) A= B∗ C is positive semidefinite. Then, max{σmax (A), σmax (B)} ≤ σmax (A)    2 ≤ 12 σmax (A) + σmax (B) + [σmax (A) − σmax (B)]2 + 4σmax (C) ≤ σmax (A) + σmax (B) and max{σmax (A), σmax (B)} ≤ σmax (A) ≤ max{σmax (A), σmax (B)} + σmax (C). Proof: See [738]. Remark: See Fact 9.14.12.

574

CHAPTER 8

Fact 8.19.30. Let A, B ∈ Fn×n, and assume that A and B are positive definite. Then,

T λ1(log A + log B) · · · λn(log A + log B) strongly log majorizes λ1(log A1/2BA1/2 )

···

λn(log A1/2BA1/2 )

T

.

Consequently, log det AB = tr(log A + log B) = tr log A1/2BA1/2 = log det A1/2BA1/2. Proof: See [93]. Fact 8.19.31. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, the following statements hold:   2 1/2 i) σmax [log(I + A)log(I + B)] ≤ log 1 + σmax (AB) .   3 1/3 ii) σmax [log(I + B)log(I + A)log(I + B)] ≤ log 1 + σmax (BAB) .   2 iii) det[log(I + A)log(I + B)] ≤ det log I + AB1/2 . 

3 iv) det[log(I + B)log(I + A)log(I + B)] ≤ det log I + (BAB)1/3 . Proof: See [1382]. Remark: See Fact 11.16.6. Fact 8.19.32. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then,

σmax (AB) σmax (I + A)−1AB(I + B)−1 ≤  2 . 1/2 1 + σmax (AB) Proof: See [1382].

8.20 Facts on Alternative Partial Orderings Fact 8.20.1. Let A, B ∈ Fn×n, and assume that A and B are positive definite. Then, the following statements are equivalent: i) log B ≤ log A. ii) For all r ∈ (0, ∞), iii) For all r ∈ (0, ∞),

1/2  . B r ≤ B r/2ArB r/2 1/2  Ar/2B rAr/2 ≤ Ar.

iv) For all p, r ∈ (0, ∞) and k ∈ N such that (k + 1)r = p + r,

POSITIVE-SEMIDEFINITE MATRICES

575

1 k+1  B r ≤ B r/2ApB r/2 .

v) For all p, r ∈ (0, ∞) and k ∈ N such that (k + 1)r = p + r, 1 k+1  ≤ Ar. Ar/2B pAr/2 vi) For all p, r ∈ [0, ∞),

vii) For all p, r ∈ [0, ∞),

r r+p  B r ≤ B r/2ApB r/2 .

r r+p  ≤ Ar. Ar/2B pAr/2

viii) For all p, q, r, t ∈ R such that p ≥ 0, r ≥ 0, t ≥ 0, and q ∈ [1, 2],   q  r+t r/2 t/2 p t/2 r/2 r+qt+qp A A BA A ≤ Ar+t. Proof: See [525, 940, 1506] and [544, pp. 139, 200]. Remark: log B ≤ log A is the chaotic order. This order is weaker than the L¨ owner order since B ≤ A implies that log B ≤ log A, but not vice versa. Remark: Additional conditions are given in [940]. Fact 8.20.2. Let A, B ∈ Fn×n, assume that A is positive definite and B is positive semidefinite, and let α > 0. Then, the following statements are equivalent: i) B α ≤ Aα. ii) For all p, q, r, τ ∈ R such that p ≥ α, r ≥ τ , q ≥ 1, and τ ∈ [0, α], r−τ   q  r−qτ +qp Ar/2 A−τ/2B pA−τ/2 Ar/2 ≤ Ar−τ . Proof: See [525]. Fact 8.20.3. Let A, B ∈ Fn×n, and assume that A is positive definite and B is positive semidefinite. Then, the following statements are equivalent: i) For all k ∈ N, B k ≤ Ak. ii) For all α > 0, B α ≤ Aα. iii) For all p, r ∈ R such that p > r ≥ 0, 2p−r  p−r A−r/2B pA−r/2 ≤ A2p−r. iv) For all p, q, r, τ ∈ R such that p ≥ τ , r ≥ τ , q ≥ 1, and τ ≥ 0, r−τ   q  r−qτ +qp Ar/2 A−τ/2B pA−τ/2 Ar/2 ≤ Ar−τ . Proof: See [545].

576

CHAPTER 8

Remark: A and B are related by the spectral order. Fact 8.20.4. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, if two of the following statements hold, then the remaining statement also holds: rs

i) A ≤ B. rs

ii) A2 ≤ B 2. iii) AB = BA. Proof: See [113, 604, 605]. Remark: The rank subtractivity partial ordering is defined in Fact 2.10.32. Fact 8.20.5. Let A, B, C ∈ Fn×n, and assume that A, B, and C are positive semidefinite. Then, the following statements hold: i) If A2 = AB and B 2 = BA, then A = B. ii) If A2 = AB and B 2 = BC, then A2 = AC. Proof: Use Fact 2.10.33 and Fact 2.10.34. Fact 8.20.6. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite, and define ∗ A≤B if and only if

A2 = AB.

∗

Then, “≤” is a partial ordering on Nn×n. Proof: Use Fact 2.10.35 or Fact 8.20.5. ∗

Remark: The relation “≤” is the star partial ordering. Fact 8.20.7. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, ∗ A≤B if and only if

∗

B + ≤ A+. Proof: See [663]. Remark: The star partial ordering is defined in Fact 8.20.6. Fact 8.20.8. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, the following statements are equivalent: ∗

i) A ≤ B. rs

rs

ii) A ≤ B and A2 ≤ B 2. Remark: See [615].

577

POSITIVE-SEMIDEFINITE MATRICES

Remark: The star partial ordering is defined in Fact 8.20.6. Fact 8.20.9. Let A, B ∈ Fn×m, and define GL

A ≤ B if and only if all of the following conditions hold: i) A ≤ B. ii) R(A∗ ) ⊆ R(B ∗ ). iii) AB ∗ = AB. GL

Then, “ ≤ ” is a partial ordering on Fn×m. Furthermore, the following statements are equivalent: GL

iv) A ≤ B. GL

v) A∗ ≤ B ∗. vi) sprad(B +A) ≤ 1, R(A) ⊆ R(B), R(A∗ ) ⊆ R(B ∗ ), and AB ∗ = AB. rs

GL

Furthermore, if A ≤ B, then A ≤ B. Finally, if A, B ∈ Nn, then A ≤ B if and only GL

if A ≤ B. Proof: See [672]. GL

Remark: The relation “ ≤ ” is the generalized L¨ owner partial ordering. Remarkably, the L¨ owner, generalized L¨owner, and star partial orderings are linked through the polar decomposition. See [672].

8.21 Facts on Generalized Inverses Fact 8.21.1. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A + A∗ ≥ 0. ii) A+ + A+∗ ≥ 0. If, in addition, A is group invertible, then the following statement is equivalent to i) and ii): iii) A# + A#∗ ≥ 0. Proof: See [1361]. Fact 8.21.2. Let A ∈ Fn×n, and assume that A is positive semidefinite. Then, the following statements hold: i) A+ = AD = A# ≥ 0. ii) rank A = rank A+. +  1/2  iii) A+1/2 = A1/2 = (A+ ) .

578

CHAPTER 8 1/2

1/2

iv) A1/2 = A(A+ ) = (A+ ) A.  + v) AA+ = A1/2 A1/2 .   + vi) AA+A AA is positive semidefinite. + A vii) A+A + AA+ ≤ A + A+. viii) A+A ◦ AA+ ≤ A ◦ A+. Proof: See [1528] or Fact 8.11.5 and Fact 8.22.42 for vi)–viii). Fact 8.21.3. Let A ∈ Fn×n, and assume that A is positive semidefinite. Then, rank A ≤ (tr A) tr A+. Furthermore, equality holds if and only if rank A ≤ 1. Proof: See [117]. Fact 8.21.4. Let A ∈ Fn×m. Then, A∗  = AA+1/2A∗. Remark: See Fact 8.11.11. Fact 8.21.5. Let A ∈ Fn×m, and define S ∈ Fn×n by 

S = A + In − AA+. Then, S is positive definite, and SAA+S = AAA+A = AA∗. Proof: See [459, p. 432]. Remark: This result provides an explicit congruence transformation for AA+ and AA∗. Remark: See Fact 5.8.20. Fact 8.21.6. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, A = (A + B)(A + B)+A. Fact 8.21.7. Let A, B ∈ Fn×n, and assume that A and B are Hermitian. Then, the following statements are equivalent: rs

i) A ≤ B. ii) R(A) ⊆ R(B) and AB +A = A. Proof: See [604, 605]. Remark: See Fact 6.5.30.

POSITIVE-SEMIDEFINITE MATRICES

579

Fact 8.21.8. Let A, B ∈ Fn×n, assume that A and B are Hermitian, assume that ν− (A) = ν− (B), and consider the following statements: ∗

i) A ≤ B. rs

ii) A ≤ B. iii) A ≤ B. iv) R(A) ⊆ R(B) and AB +A ≤ A. Then, i) =⇒ ii ) =⇒ iii ) ⇐⇒ iv ). If, in addition, A and B are positive semidefinite, then the following statement is equivalent to iii) and iv): v) R(A) ⊆ R(B) and sprad(B +A) ≤ 1. Proof: i) =⇒ ii ) is given in [669]. See [113, 604, 615, 1254] and [1215, p. 229]. Remark: See Fact 8.21.7. Fact 8.21.9. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, the following statements are equivalent: i) A2 ≤ B 2. ii) R(A) ⊆ R(B) and σmax (B +A) ≤ 1. Proof: See [615]. Fact 8.21.10. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and assume that A ≤ B. Then, the following statements are equivalent: i) B + ≤ A+. ii) rank A = rank B. iii) R(A) = R(B). Furthermore, the following statements are equivalent: iv) A+ ≤ B +. v) A2 = AB. ∗

vi) A+ ≤ B +. Proof: See [663, 1028]. Fact 8.21.11. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, if two of the following statements hold, then the remaining statement also holds: i) A ≤ B. ii) B + ≤ A+. iii) rank A = rank B. Proof: See [114, 1028, 1456, 1491].

580

CHAPTER 8

Fact 8.21.12. Let A, B ∈ Fn×n, and assume that A and B are Hermitian. Then, if two of the following statements hold, then the remaining statement also holds: i) A ≤ B. ii) B + ≤ A+. iii) In A = In B. Proof: See [112]. Fact 8.21.13. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and assume that A ≤ B. Then, 0 ≤ AA+ ≤ BB +. If, in addition, rank A = rank B, then AA+ = BB +. Fact 8.21.14. Let A, B ∈ Fn×n, assume that A and B are Hermitian, and assume that R(A) = R(B). Then, In A − In B = In(A − B) + In(A+ − B + ). Proof: See [1074]. Remark: See Fact 8.10.15. Fact 8.21.15. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and assume that A ≤ B. Then,    +  0 ≤ AB +A ≤ A ≤ A + B I − AA+ B I − AA+ B ≤ B. Proof: See [663]. Fact 8.21.16. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then,

spec (A + B)+A ⊂ [0, 1]. Proof: Let C be positive definite and satisfy B ≤ C. Then, (A + C)−1/2 C(A + C)−1/2 ≤ I. The result now follows from Fact 8.21.17. Fact 8.21.17. Let A, B, C ∈ Fn×n, assume that A, B, C are positive semidefinite, and assume that B ≤ C. Then, for all i ∈ {1, . . . , n},

λi (A + B)+B ≤ λi (A + C)+C . Consequently,



tr (A + B)+B ≤ tr (A + C)+C .

Proof: See [1424]. Remark: See Fact 8.21.16.

581

POSITIVE-SEMIDEFINITE MATRICES

Fact 8.21.18. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and define  A:B = A(A + B)+B. Then, the following statements hold: i) A : B is positive semidefinite. ii) A : B = limε↓0 (A + εI) : (B + εI). iii) A : A = 12 A. iv) A : B = B : A = B − B(A + B)+B = A − A(A + B)+A. v) A : B ≤ A. vi) A : B ≤ B. vii) A : B = −



0

⎤ ⎤+ ⎡ 0 I I ⎦ ⎣ 0 ⎦. I 0

⎡

A 0 0 I ⎣ 0 B I I

viii) A : B = (A+ + B + )+ if and only if R(A) = R(B). ix) A(A + B)+B = ACB for every (1)-inverse C of A + B. x) tr(A : B) ≤ (tr B) : (tr A). xi) tr(A : B) = (tr B) : (tr A) if and only if there exists α ∈ [0, ∞) such that either A = αB or B = αA. xii) det(A : B) ≤ (det B) : (det A). xiii) R(A : B) = R(A) ∩ R(B). xiv) N(A :B) = N(A) + N(B). xv) rank(A : B) = rank A + rank B − rank(A + B). xvi) Let S ∈ Fp×n, and assume that S is right invertible. Then, S(A : B)S ∗ ≤ (SAS ∗ ) : (SBS ∗ ). xvii) Let S ∈ Fn×n, and assume that S is nonsingular. Then, S(A : B)S ∗ = (SAS ∗ ) : (SBS ∗ ). xviii) For all positive numbers α, β,  −1   −1  α A : β B ≤ αA + βB. xix) Let X ∈ Fn×n, and assume that X is Hermitian and   A+B A ≥ 0. A A−X Then, Furthermore,

X ≤ A : B. 

A+B

A

A

A − A:B

 ≥ 0.

582

CHAPTER 8 

xx) φ: Nn × Nn → −Nn defined by φ(A, B) = −A : B is convex. xxi) If A and B are projectors, then 2(A :B) is the projector onto R(A) ∩ R(B). xxii) If A + B is positive definite, then A : B = A(A + B)−1B. xxiii) A#B = [ 12 (A + B)]#[2(A : B)]. xxiv) If C, D ∈ Fn×n are positive semidefinite, then (A : B) : C = A : (B : C) and

A : C + B : D ≤ (A + B) : (C + D).

xxv) If C, D ∈ Fn×n are positive semidefinite, A ≤ C, and B ≤ D, then A : B ≤ C : D. xxvi) If A and B are positive definite, then −1  A : B = A−1 + B −1 ≤ 12 (A#B) ≤ 14 (A + B). xxvii) Let x, y ∈ Fn. Then, (x + y)∗ (A : B)(x + y) ≤ x∗Ax + y ∗By. xxviii) Let x, y ∈ Fn. Then, x∗ (A : B)x ≤ y ∗Ay + (x − y)∗B(x − y). xxix) Let x ∈ Fn. Then, x∗ (A : B)x = infn[y ∗Ay + (x − y)∗B(x − y)]. y∈F

xxx) Let x ∈ F . Then, n

x∗ (A : B)x ≤ (x∗Ax) : (x∗Bx). Proof: See [38, 39, 42, 597, 867, 1316], [1145, p. 189], and [1521, p. 9]. Remark: A : B is the parallel sum of A and B. Remark: See Fact 6.4.46 and Fact 6.4.47. Remark: A symmetric expression for the parallel sum of three or more positivesemidefinite matrices is given in [1316]. Fact 8.21.19. Let A, B ∈ Fn×n, assume that A is positive semidefinite, and assume that B is a projector. Then,  sh(A, B) = min{X ∈ Nn : 0 ≤ X ≤ A and R(X) ⊆ R(B)}

exists. Furthermore, sh(A, B) = A − AB⊥ (B⊥AB⊥ )+B⊥A. That is,

.  . A AB⊥ . . sh(A, B) = A . . B⊥A B⊥AB⊥

583

POSITIVE-SEMIDEFINITE MATRICES

Finally, sh(A, B) = lim (αB) : A. α→∞

Proof: Existence of the minimum is proved in [42]. The expression for sh(A, B) is given in [582]; a related expression involving the Schur complement is given in [38]. The last equality is shown in [42]. See also [52]. Remark: sh(A, B) is the shorted operator. Fact 8.21.20. Let B ∈ Rm×n, define 

S = {A ∈ Rn×n : A ≥ 0 and R(B TBA) ⊆ R(A)}, 

and define φ: S → −Nm by φ(A) = −(BA+B T )+. Then, S is a convex cone, and φ is convex. Proof: See [606]. Remark: This result generalizes xii) of Proposition 8.6.17 in the case r = p = 1. Fact 8.21.21. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. If (AB)+ = B +A+, then AB is range Hermitian. Furthermore, the following statements are equivalent: i) AB is range Hermitian. ii) (AB)# = B +A+. iii) (AB)+ = B +A+. Proof: See [1013]. Remark: See Fact 6.4.31. Fact 8.21.22. Let A ∈ Fn×n and C ∈ Fm×m, assume that A and C are  positive semidefinite, let B ∈ Fn×m, and define X = A+1/2BC +1/2. Then, the following statements are equivalent:

i) BA∗ B C is positive semidefinite. ii) AA+B = B and X ∗X ≤ Im . iii) BC +C = B and X ∗X ≤ Im . iv) B = A1/2XC 1/2 and X ∗X ≤ Im . v) There exists a matrix Y ∈ Fn×m such that B = A1/2YC 1/2 and Y ∗Y ≤ Im . Proof: See [1521, p. 15]. Fact 8.21.23. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, the following statements are equivalent: i) A(A + B)+B = 0. ii) B(A + B)+A = 0. iii) A(A + B)+A = A.

584

CHAPTER 8

iv) B(A + B)+B = B. v) A(A + B)+B + B(A + B)+A = 0. vi) A(A + B)+A + B(A + B)+B = A + B.

vii) rank A B = rank A + rank B. viii) R(A) ∩ R(B) = {0}. ix) (A + B)+ = [(I − BB + )A(I − B +B]+ + [(I − AA+ )B(I − A+A]+. Proof: See [1334]. Remark: See Fact 6.4.35.

8.22 Facts on the Kronecker and Schur Products Fact 8.22.1. Let A ∈ Fn×n, assume that A is positive semidefinite, and assume that every entry of A is nonzero. Then, A◦−1 is positive semidefinite if and only if rank A = 1. Proof: See [914]. Fact 8.22.2. Let A ∈ Fn×n, assume that A is positive semidefinite, assume that every entry of A is nonnegative, and let α ∈ [0, n − 2]. Then, A◦α is positive semidefinite. Proof: See [203, 504]. Remark: In many cases, A◦α is positive semidefinite for all α ≥ 0. See Fact 8.8.5. Fact 8.22.3. Let A ∈ Fn×n, assume that A is positive semidefinite, and let k ≥ 1. If r ∈ [0, 1], then  r ◦k (Ar ) ≤ A◦k . If r ∈ [1, 2], then

 ◦k r ◦k ≤ (Ar ) . A

If A is positive definite and r ∈ [0, 1], then  ◦k −r  −r ◦k ≤ A . A Proof: See [1521, p. 8]. Fact 8.22.4. Let A ∈ Fn×n, and assume that A is positive semidefinite. Then, (I ◦ A)2 ≤ 12 (I ◦ A2 + A ◦ A) ≤ I ◦ A2 and Hence,

A ◦ A ≤ I ◦ A2. n  i=1

A2(i,i) ≤

n  i=1

λ2i (A).

585

POSITIVE-SEMIDEFINITE MATRICES

Now, assume in addition that A is positive definite. Then, (A ◦ A)−1 ≤ A−1 ◦ A−1 and



A ◦ A−1

−1

2    ≤ I ≤ A1/2 ◦ A−1/2 ≤ 12 I + A ◦ A−1 ≤ A ◦ A−1.  A ◦ A−1 1n×1 = 1n×1



Furthermore,

  1 ∈ spec A ◦ A−1 .

and 



Next, let α = λmin(A) and β = λmax (A). Then, 1/2  2αβ 2αβ  2 αβ  A ◦ A−2 I + A2 ◦ A−2 ≤ A ◦ A−1. I≤ 2 ≤ 2 α2 + β 2 α + β2 α + β2  A ◦ A−1, and, for all k ≥ 1, define Define Φ(A) =    Φ Φ(k)(A) , Φ(k+1)(A) = 

where Φ(1)(A) = Φ(A). Then, for all k ≥ 1, Φ(k)(A) ≥ I and

lim Φ(k)(A) = I.

k→∞

Proof: See [493, 794, 1417, 1418], [728, p. 475], [1208, pp. 304, 305], and set B = A−1 in Fact 8.22.33. Remark: The convergence result also holds if A is an H-matrix [794]. A ◦ A−1 is the relative gain array. Remark: See Fact 8.22.40. Fact 8.22.5. Let A ∈ Fn×n, and assume that A is positive definite. Then, for all i ∈ {1, . . . , n}, 1 ≤ A(i,i) (A−1 )(i,i) . Furthermore, max

i=1,...,n

and max

i=1,...,n

n + +  A(i,i) (A−1 )(i,i) − 1 ≤ A(i,i) (A−1 )(i,i) − 1 i=1

n + +   A(i,i) (A−1 )(i,i) − 1 ≤ A(i,i) (A−1 )(i,i) − 1 . i=1

Proof: See [495, p. 66-6]. A B  n+m)×(n+m) , assume that A is positive Fact 8.22.6. Let A = B∗ C ∈ F −1 X Y definite, and partition A = Y ∗ Z conformably with A. Then,   A ◦ A−1 0 I≤ ≤ A ◦ A−1 0 Z ◦ Z −1

586

CHAPTER 8



and I≤

X ◦ X −1 0

0 C ◦ C −1



≤ A ◦ A−1.

Proof: See [136]. Fact 8.22.7. Let A ∈ Fn×n, let p, q ∈ R, assume that A is positive semidefinite, and assume that either p and q are nonnegative or A is positive definite. Then, A(p+q)/2 ◦ A(p+q)/2 ≤ Ap ◦ Aq. In particular,

I ≤ A ◦ A−1.

Proof: See [95]. Fact 8.22.8. Let A ∈ Fn×n, assume that A is positive semidefinite, and assume that In ◦ A = In . Then, det A ≤ λmin(A ◦ A). Proof: See [1442]. Fact 8.22.9. Let A ∈ Fn×n. Then, −A∗A ◦ I ≤ A∗ ◦ A ≤ A∗A ◦ I. Proof: Use Fact 8.22.43 with B = I. Fact 8.22.10. Let A ∈ Fn×n. Then,   ∗ AA◦ I 2 ≤ σmax (A)I. A ◦ A∗  ≤ A ◦ A∗  Proof: See [1528] and Fact 8.22.23.  Fact 8.22.11. Let A =

F



A11 A12 ∗ A12 A22



 ∈ F(n+m)×(n+m) and B =

B11

B12 B12 B22

∈

(n+m)×(n+m)

, and assume that A and B are positive semidefinite. Then,

(A11|A) ◦ (B11|B) ≤ (A11|A) ◦ B22 ≤ (A11 ◦ B11 )|(A ◦ B). Proof: See [922]. Fact 8.22.12. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, A ◦ B is positive semidefinite. If, in addition, B is positive definite and I ◦ A is positive definite, then A ◦ B is positive definite. Proof: By Fact 7.4.17, A ⊗ B is positive semidefinite, and the Schur product A ◦ B is a principal submatrix of the Kronecker product. If A is positive definite, use Fact 8.22.20 to obtain det(A ◦ B) > 0. See [1208, p. 300]. Remark: The first result is Schur’s theorem. The second result is Schott’s theorem. See [950] and Fact 8.22.20. Fact 8.22.13. Let A ∈ Fn×n, and assume that A is positive definite. Then, there exist positive-definite matrices B, C ∈ Fn×n such that A = B ◦ C.

POSITIVE-SEMIDEFINITE MATRICES

587

Remark: See [1125, pp. 154, 166]. Remark: This result is due to Djokovic. Fact 8.22.14. Let A, B ∈ Fn×n, and assume that A is positive definite and B is positive semidefinite. Then,  −1 11×n A−11n×1 B ≤ A ◦ B. Proof: See [497]. Remark: Setting B = 1n×n yields Fact 8.9.17. Fact 8.22.15. Let A, B ∈ Fn×n, and assume that A and B are positive defi −1 nite. Then, 11×n A−11n×1 11×n B −11n×1 1n×n ≤ A ◦ B. Proof: See [1528]. Fact 8.22.16. Let A ∈ Fn×n, assume that A is positive definite, let B ∈ Fn×n, and assume that B is positive semidefinite. Then, rank B ≤ rank(A ◦ B) ≤ rank(A ⊗ B) = (rank A)(rank B). Remark: See Fact 7.4.24, Fact 7.6.10, and Fact 8.22.14. Remark: The first inequality is due to Djokovic. See [1125, pp. 154, 166]. Fact 8.22.17. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. If p ≥ 1, then tr (A ◦ B)p ≤ tr Ap ◦ B p. If 0 ≤ p ≤ 1, then

tr Ap ◦ B p ≤ tr (A ◦ B)p.

Now, assume in addition that A and B are positive definite. If p ≤ 0, then tr (A ◦ B)p ≤ tr Ap ◦ B p. Proof: See [1426]. Fact 8.22.18. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, λmin(AB) ≤ λmin(A ◦ B). Hence,

λmin(AB)I ≤ λmin(A ◦ B)I ≤ A ◦ B.

Proof: See [787]. Remark: This result interpolates the penultimate inequality in Fact 8.22.21. Fact 8.22.19. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, for all i = 1, . . . , n, λmin (A)dmin (B) ≤ λi (A ◦ B) ≤ λmax (A)dmax (B). Proof: See [1208, pp. 303, 304].

588

CHAPTER 8

Fact 8.22.20. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then,  n  n ! ! det AB ≤ A(i,i) det B ≤ det(A ◦ B) ≤ A(i,i) B(i,i) . i=1

i=1

Equivalently, det AB ≤ [det(I ◦ A)] det B ≤ det(A ◦ B) ≤

n !

A(i,i) B(i,i) .

i=1

Furthermore,



2 det AB ≤

n !





A(i,i) det B +

i=1

n !

 B(i,i) det A ≤ det(A ◦ B) + (det A) det B.

i=1

Finally, the following statements hold: i) If I ◦ A and B are positive definite, then A ◦ B is positive definite. ii) If I ◦ A and B are positive definite and rank A = 1, then equality holds in the right-hand equality. iii) If A and B are positive definite, then equality holds in the right-hand equality if and only if B is diagonal. Proof: See [992, 1512] and [1215, p. 253]. Remark: In the first string, the first and third inequalities follow from Hadamard’s inequality Fact 8.18.11, while the second inequality is Oppenheim’s inequality. See Fact 8.22.12. Remark: The right-hand inequality in the third string of inequalities is valid when A and B are M-matrices. See [46, 326].  6n Problem: Compare the lower bounds det (A#B)2 and i=1 A(i,i) det B for det(A ◦ B). See Fact 8.22.21. Fact 8.22.21. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, let k ∈ {1, . . . , n}, and let r ∈ (0, 1]. Then, n !

λi(A)λi(B) ≤

i=k

n !

σi (AB) ≤

i=k

n !

λi(AB) ≤

i=k

n !

λ2i(A#B) ≤

i=k

n !

λi(A ◦ B)

i=k

and n !

λi(A)λi(B) ≤

i=k

n !

σi (AB) ≤

i=k

≤

n !

eλi (log A+log B) ≤

i=k

≤

n !

λi(AB) ≤

i=k n !

i=k

1/r

λi (Ar ◦ B r ) ≤

i=k

eλi [I◦(log A+log B)]

i=k n !

n !

n ! i=k

λi(A ◦ B).

1/r

λi (ArB r )

589

POSITIVE-SEMIDEFINITE MATRICES

Consequently, and

λmin(AB) ≤ λmin(A ◦ B) det AB = det (A#B)2 ≤ det(A ◦ B).

Proof: See [50, 493, 1416], [1208, p. 305], [1521, p. 21], Fact 8.10.43, and Fact 8.19.22. Fact 8.22.22. Let A, B ∈ Fn×n, assume that A and B are positive definite, let k ∈ {1, . . . , n}, and let r > 0. Then, n ! i=k

λ−r i (A ◦ B) ≤

n !

λ−r i (AB).

i=k

Proof: See [1415]. Fact 8.22.23. Let A, B ∈ Fn×n, let C, D ∈ Fm×m, assume that A, B, C, and D are Hermitian, A ≤ B, C ≤ D, and that either A and C are positive semidefinite, A and D are positive semidefinite, or B and D are positive semidefinite. Then, A ⊗ C ≤ B ⊗ D. If, in addition, n = m, then

A ◦ C ≤ B ◦ D.

Proof: See [45, 114]. Problem: Under which conditions are these inequalities strict? Fact 8.22.24. Let A, B, C, D ∈ Fn×n, assume that A, B, C, D are positive semidefinite, and assume that A ≤ B and C ≤ D. Then, 0 ≤ A⊗C ≤ B ⊗ D and

0 ≤ A ◦ C ≤ B ◦ D.

Proof: See Fact 8.22.23. Fact 8.22.25. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, A ≤ B if and only if A ⊗ A ≤ B ⊗ B. Proof: See [950]. Fact 8.22.26. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, assume that 0 ≤ A ≤ B, and let k ≥ 1. Then, A◦k ≤ B ◦k. Proof: 0 ≤ (B − A) ◦ (B + A) implies that A ◦ A ≤ B ◦ B, that is, A◦2 ≤ B ◦2. Fact 8.22.27. Let A1, . . . , Ak , B1, . . . , Bk ∈ Fn×n, and assume that A1, . . . , Ak , B1, . . . , Bk are positive semidefinite. Then, (A1 + B1 ) ⊗ · · · ⊗ (Ak + Bk ) ≤ A1 ⊗ · · · ⊗ Ak + B1 ⊗ · · · ⊗ Bk .

590

CHAPTER 8

Proof: See [1019, p. 143]. Fact 8.22.28. Let A1, A2 , B1, B2 ∈ Fn×n, assume that A1, A2 , B1, B2 are positive semidefinite, assume that 0 ≤ A1 ≤ B1 and 0 ≤ A2 ≤ B2 , and let α ∈ [0, 1]. Then, [αA1 + (1 − α)B1 ] ⊗ [αA2 + (1 − α)B2 ] ≤ α(A1 ⊗ A2 ) + (1 − α)(B1 ⊗ B2 ). Proof: See [1440]. Fact 8.22.29. Let A, B ∈ Fn×n, and assume that A and B are Hermitian. Then, for all i ∈ {1, . . . , n}, λn(A)λn(B) ≤ λi+n2 −n(A ⊗ B) ≤ λi(A ◦ B) ≤ λi(A ⊗ B) ≤ λ1(A)λ1(B). Proof: This result follows from Proposition 7.3.1 and Theorem 8.4.5. For A, B positive semidefinite, the result is given in [987]. Fact 8.22.30. Let A ∈ Fn×n and B ∈ Fm×m, assume that A and B are positive semidefinite, let r ∈ R, and assume that either A and B are positive definite or r is positive. Then, (A ⊗ B)r = Ar ⊗ B r. Proof: See [1044]. Fact 8.22.31. Let A ∈ Fn×m and B ∈ Fk×l. Then, A ⊗ B = A ⊗ B. Fact 8.22.32. Let A, B ∈ Fn×n , let C, D ∈ Fm×m, assume that A, B, C, D are positive semidefinite, let α and β be nonnegative numbers, and let r ∈ [0, 1]. Then, α(Ar ⊗ C 1−r ) + β(B r ⊗ D1−r ) ≤ (αA + βB)r ⊗ (αC + βD)1−r . Proof: See [918]. Fact 8.22.33. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. If r ∈ [0, 1], then Ar ◦ B r ≤ (A ◦ B)r. If r ∈ [1, 2], then

(A ◦ B)r ≤ Ar ◦ B r.

If A and B are positive definite and r ∈ [0, 1], then (A ◦ B)−r ≤ A−r ◦ B −r. Therefore, (A ◦ B)−1 ≤ A−1 ◦ B −1, (A ◦ B)2 ≤ A2 ◦ B 2, 1/2  , A ◦ B ≤ A2 ◦ B 2 A1/2 ◦ B 1/2 ≤ (A ◦ B)1/2.

591

POSITIVE-SEMIDEFINITE MATRICES

Furthermore, A2 ◦ B 2 − 14 (β − α)2I ≤ (A ◦ B)2 ≤ and



1 2

A2 ◦ B 2 + (AB)◦2 ≤ A2 ◦ B 2

1/2  α+β ≤ √ A ◦ B, A ◦ B ≤ A2 ◦ B 2 2 αβ 



where α = λmin(A ⊗ B) and β = λmax(A ⊗ B). Hence,  2  √ 2 A ◦ B − 14 β − α I ≤ A1/2 ◦ B 1/2   ◦2  1/2 1/2 1 ≤ 2 A◦B + A B ≤ A◦B 1/2  ≤ A2 ◦ B 2 α+β ≤ √ A ◦ B. 2 αβ Proof: See [45, 1043, 1417], [728, p. 475], and [1521, p. 8]. Fact 8.22.34. Let A, B ∈ Fn×n, and assume that A and B are Hermitian. Then, there exist unitary matrices S1, S2 ∈ Fn×n such that A ◦ B ≤ 12 [S1 (A ◦ B)S1∗ + S2 (A ◦ B)S2∗ ]. Proof: See [93]. Fact 8.22.35. Let A, B ∈ Fn×n, assume that A and B are positive definite, and let k, l be nonzero integers such that k ≤ l. Then,  k 1/k  l 1/l A ◦ Bk ≤ A ◦ Bl . In particular,

 −1 −1 A ◦ B −1 ≤ A◦B

and

(A ◦ B)−1 ≤ A−1 ◦ B −1.

Furthermore, for all k ≥ 1, A ◦ B ≤ (Ak ◦ B k )1/k and Finally,

A1/k ◦ B 1/k ≤ (A ◦ B)1/k. (A ◦ B)−1 ≤ A−1 ◦ B −1 ≤

(α + β)2 (A ◦ B)−1, 4αβ

  λmin(A ⊗ B) and β = λmax(A ⊗ B). where α =

Proof: See [1043].

592

CHAPTER 8

Fact 8.22.36. Let A, B ∈ Fn×n, and assume that A is positive definite, B is positive semidefinite, and I ◦ B is positive definite. Then, for all i ∈ {1, . . . , n}, [(A ◦ B)−1 ](i,i) ≤

(A−1 )(i,i) . B(i,i)

Furthermore, if rank B = 1, then equality holds. Proof: See [1512]. Fact 8.22.37. Let A, B ∈ Fn×n. Then, A is positive semidefinite if and only if, for every positive-semidefinite matrix B ∈ Fn×n, 11×n (A ◦ B)1n×1 ≥ 0. Proof: See [728, p. 459]. Remark: This result is Fejer’s theorem. Fact 8.22.38. Let A, B ∈ Fn×n, and assume that A and B are positive definite. Then, 11×n [(A − B) ◦ (A−1 − B −1 )]1n×1 ≤ 0. Furthermore, equality holds if and only if A = B. Proof: See [152, p. 8-8]. Fact 8.22.39. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, let p, q ∈ R, and assume that one of the following conditions is satisfied: i) p ≤ q ≤ −1, and A and B are positive definite. ii) p ≤ −1 < 1 ≤ q, and A and B are positive definite. iii) 1 ≤ p ≤ q. iv)

1 2

≤ p ≤ 1 ≤ q.

v) p ≤ −1 ≤ q ≤ − 21 , and A and B are positive definite. Then,

(Ap ◦ B p )

1/p

1/q

≤ (Aq ◦ B q )

.

Proof: See [1044]. Consider case iii). Since p/q ≤ 1, it follows from Fact 8.22.33 that Ap ◦ B p = (Aq )p/q ◦ (Aq )p/q ≤ (Aq ◦ B q )p/q . Then, use Corollary 8.6.11 with p replaced by 1/p. See [1521, p. 8]. Remark: See [95]. Fact 8.22.40. Let A, B ∈ Fn×n, and assume that A and B are positive definite. Then, 2I ≤ A ◦ B −1 + B ◦ A−1. Proof: See [1417, 1528]. Remark: Setting B = A yields an inequality given by Fact 8.22.4.

POSITIVE-SEMIDEFINITE MATRICES

593

Fact 8.22.41. Let A, B ∈ Fn×m, and define   A∗A ◦ B ∗B (A ◦ B)∗  . A= A◦B I Then, A is positive semidefinite. Furthermore, (A ◦ B)∗ (A ◦ B) ≤ 12 (A∗A ◦ B ∗B + A∗B ◦ B ∗A) ≤ A∗A ◦ B ∗B. Proof: See [732, 1417, 1528]. Remark: The inequality (A ◦ B)∗ (A ◦ B) ≤ A∗A ◦ B ∗B is Amemiya’s inequality. See [950]. Fact 8.22.42. Let A, B, C ∈ Fn×n, define   A B  , A= B∗ C and assume that A is positive semidefinite. Then, −A ◦ C ≤ B ◦ B ∗ ≤ A ◦ C and

|det(B ◦ B ∗ )| ≤ det(A ◦ C).

If, in addition, A is positive definite, then −A ◦ C < B ◦ B ∗ < A ◦ C and

|det(B ◦ B ∗ )| < det(A ◦ C).

Proof: See [1528]. Remark: See Fact 8.11.5. Fact 8.22.43. Let A, B ∈ Fn×m. Then, −A∗A ◦ B ∗B ≤ A∗B ◦ B ∗A ≤ A∗A ◦ B ∗B and

|det(A∗B ◦ B ∗A)| ≤ det(A∗A ◦ B ∗B). A∗A A∗B . Proof: Apply Fact 8.22.42 to B ∗ A B ∗B Remark: See Fact 8.11.14 and Fact 8.22.9. Fact 8.22.44. Let A, B ∈ Fn×n, and assume that A is positive definite. Then, −A ◦ B ∗A−1B ≤ B ◦ B ∗ ≤ A ◦ B ∗A−1B and

|det(B ◦ B ∗ )| ≤ det(A ◦ B ∗A−1B).

Proof: Use Fact 8.11.19 and Fact 8.22.42.

594

CHAPTER 8

Fact 8.22.45. Let A, B ∈ Fn×n, and let α, β ∈ (0, ∞).     − β −1/2I + αA∗A ◦ α−1/2I + βBB ∗ ≤ (A + B) ◦ (A + B)∗     ≤ β −1/2I + αA∗A ◦ α−1/2I + βBB ∗ . Remark: See Fact 8.11.20. Fact 8.22.46. Let A, B ∈ Fn×m, and define   A∗A ◦ I (A ◦ B)∗  . A= A◦B BB ∗ ◦ I Then, A is positive semidefinite. Now, assume in addition that n = m. Then, −A∗A ◦ I − BB ∗ ◦ I ≤ A ◦ B + (A ◦ B)∗ ≤ A∗A ◦ I + BB ∗ ◦ I and

−A∗A ◦ BB ∗ ◦ I ≤ A ◦ A∗ ◦ B ◦ B ∗ ≤ A∗A ◦ BB ∗ ◦ I.

Remark: See Fact 8.22.42. Fact 8.22.47. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then,   A ◦ B ≤ 12 A2 + B 2 ◦ I. Proof: Use Fact 8.22.46. Fact 8.22.48. Let A ∈ Fn×n, assume that A is positive semidefinite, and  define e◦A ∈ Fn×n by [e◦A ](i,j) = eA(i,j) . Then, e◦A is positive semidefinite. Proof: Note that e◦A = 1n×n + 12 A ◦ A + See [432, p. 10].

1 3! A ◦ A ◦ A

+ · · · , and use Fact 8.22.12.

Fact 8.22.49. Let A, B ∈ Fn×n, assume that A and B are positive definite, and let p, q ∈ (0, ∞) satisfy p ≤ q. Then, I ◦ (log A + log B) ≤ log (Ap ◦ B p )1/p ≤ log (Aq ◦ B q )1/q and

I ◦ (log A + log B) = lim log (Ap ◦ B p )

1/p

p↓0

.

Proof: See [1416]. Remark: log (Ap ◦ B p )1/p = 1p log(Ap ◦ B p ). Fact 8.22.50. Let A, B ∈ Fn×n, and assume that A and B are positive definite. Then, I ◦ (log A + log B) ≤ log(A ◦ B). Proof: Set p = 1 in Fact 8.22.49. See [45] and [1521, p. 8]. Remark: See Fact 11.14.21.

POSITIVE-SEMIDEFINITE MATRICES

595

Fact 8.22.51. Let A, B ∈ Fn×n, assume that A and B are positive definite, and let C, D ∈ Fm×n. Then,     (C ◦ D)(A ◦ B)−1(C ◦ D)∗ ≤ CA−1C ∗ ◦ DB −1D∗ . In particular, and

(A ◦ B)−1 ≤ A−1 ◦ B −1 (C ◦ D)(C ◦ D)∗ ≤ (CC ∗ ) ◦ (DD∗ ).

Proof: Form the Schur complement of the lower of the

right Bblock

Schur product A C∗ D∗ of the positive-semidefinite matrices C and −1 ∗ −1 ∗ . See [991, 1427], CA C D DB D [1521, p. 13], or [1526, p. 198]. Fact 8.22.52. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let p, q ∈ (1, ∞) satisfy 1/p + 1/q = 1. Then, (A ◦ B) + (C ◦ D) ≤ (Ap + C p )1/p ◦ (B q + Dq )1/q . Proof: Use xxiv) of Proposition 8.6.17 with r = 1/p. See [1521, p. 10]. Remark: Note the relationship between the conjugate parameters p, q and the barycentric coordinates α, 1 − α. See Fact 1.18.11. Fact 8.22.53. Let A, B, C, D ∈ Fn×n, assume that A, B, C, and D are positive definite. Then, (A#C) ◦ (B#D) ≤ (A ◦ B)#(C ◦ D). Furthermore,

(A#B) ◦ (A#B) ≤ (A ◦ B).

Proof: See [95].

8.23 Notes The ordering A ≤ B is traditionally called the L¨ owner ordering. Proposition 8.2.4 is given in [15] and [870] with extensions in [171]. The proof of Proposition 8.2.7 is based on [268, p. 120], as suggested in [1280]. The proof given in [554, p. 307] is incomplete. Theorem 8.3.5 is due to Newcomb [1062]. Proposition 8.4.13 is given in [717, 1048]. Special cases such as Fact 8.12.29 appear in numerous papers. The proofs of Lemma 8.4.4 and Theorem 8.4.5 are based on [1261]. Theorem 8.4.9 can also be obtained as a corollary of the Fischer minimax theorem given in [728, 996], which provides a geometric characterization of the eigenvalues of a symmetric matrix. Theorem 8.3.6 appears in [1145, p. 121]. Theorem 8.6.2 is given in [42]. Additional inequalities appear in [1032]. Functions that are nondecreasing on Pn are characterized by the theory of monotone matrix functions [201, 432]. See [1037] for a summary of the principal results.

596

CHAPTER 8

The literature on convex maps is extensive. Result xiv) of Proposition 8.6.17 is due to Lieb and Ruskai [933]. Result xxiv) is the Lieb concavity theorem. See [201, p. 271] or [931]. Result xxxiv) is due to Ando. Results xlv) and xlvi) are due to Fan. Some extensions to strict convexity are considered in [996]. See also [45, 1051]. Products of positive-definite matrices are studied in [121, 122, 123, 125, 1493]. Essays on the legacy of Issai Schur appear in [802]. Schur complements are discussed in [296, 298, 676, 922, 947, 1084]. Majorization and eigenvalue inequalities for sums and products of matrices are discussed in [202].

Chapter Nine

Norms

Norms are used to quantify vectors and matrices, and they play a basic role in convergence analysis. This chapter introduces vector and matrix norms and their properties.

9.1 Vector Norms For many applications it is useful to have a scalar measure of the magnitude of a vector x or a matrix A. Norms provide such measures. Definition 9.1.1. A norm  ·  on Fn is a function  · : Fn → [0, ∞) that satisfies the following conditions: i) x ≥ 0 for all x ∈ Fn. ii) x = 0 if and only if x = 0. iii) αx = |α|x for all α ∈ F and x ∈ Fn. iv) x + y ≤ x + y for all x, y ∈ Fn. Condition iv) is the triangle inequality. A norm  ·  on Fn is monotone if |x| ≤≤ |y| implies that x ≤ y for all x, y ∈ Fn, while  ·  is absolute if |x| = x for all x ∈ Fn. Proposition 9.1.2. Let  ·  be a norm on Fn. Then,  ·  is monotone if and only if  ·  is absolute. 

Proof. First, suppose that  ·  is monotone. Let x ∈ Fn, and define y = |x|. Then, |y| = |x|, and thus |y| ≤≤ |x| and |x| ≤≤ |y|. Hence, x ≤ y and y ≤ x, which implies that x = y. Thus, |x| = y = x, which proves that  ·  is absolute. Conversely, suppose that  ·  is absolute and, for convenience, let n = 2. Now, let x, y ∈ F2 be such that |x| ≤≤ |y|. Then, there exist α1, α2 ∈ [0, 1] and θ1, θ2 ∈ R such that x(i) = αi ejθi y(i) for i = 1, 2. Since  ·  is absolute, it follows

598 that

CHAPTER 9

.. .. .. α1ejθ1 y(1) .. . .. . x = .. α2 ejθ2 y(2) .. .. .. .. α1 |y(1) | .. . . .. = .. α2 |y(2) | .. ..   ..    .. .. |y(1) | −|y(1) | |y(1) | .. = .... 12 (1 − α1 ) + 12 (1 − α1 ) + α1 α2 |y(2) | α2 |y(2) | α2 |y(2) | .. .. .. .. 1

.. |y(1) | 1 . .. . ≤ 2 (1 − α1 ) + 2 (1 − α1 ) + α1 .. α2 |y(2) | .. .. .. .. .. |y(1) | .. = .... α2 |y(2) | .. ..   ..    .. 1 |y(1) | .... |y(1) | |y(1) | 1 . . = .. 2 (1 − α2 ) + 2 (1 − α2 ) + α2 −|y(2) | |y(2) | |y(2) | .. .. .. .. |y(1) | .. .. ≤ .... |y(2) | .. = ||y|| .

Thus,  ·  is monotone. As we shall see, there are many different norms. For x ∈ Fn , a useful class of norms consists of the H¨ older norms defined by ⎧ 1/p n ⎪  ⎪ ⎪ p ⎪ |x(i) | , 1 ≤ p < ∞, ⎨  i=1 (9.1.1) xp = ⎪ ⎪ ⎪ ⎪ ⎩ max |x(i) |, p = ∞. i∈{1,...,n}

Note that, for all x ∈ Cn and p ∈ [1, ∞], xp = xp . These norms depend on Minkowski’s inequality given by the following result. Lemma 9.1.3. Let p ∈ [1, ∞], and let x, y ∈ Fn. Then, x + yp ≤ xp + yp .

(9.1.2)

If p = 1, then equality holds if and only if, for all i ∈ {1, . . . , n}, there exists αi ≥ 0 such that either x(i) = αi y(i) or y(i) = αi x(i) . If p ∈ (1, ∞), then equality holds if and only if there exists α ≥ 0 such that either x = αy or y = αx. Proof. See [166, 988] and Fact 1.18.25. Proposition 9.1.4. Let p ∈ [1, ∞]. Then,  · p is a norm on Fn. For p = 1, x1 =

n  i=1

|x(i) |

(9.1.3)

599

NORMS

is the absolute sum norm; for p = 2, 1/2  n  √ |x(i) |2 = x∗x x2 =

(9.1.4)

i=1

is the Euclidean norm; and, for p = ∞, x∞ =

max

i∈{1,...,n}

|x(i) |

(9.1.5)

is the infinity norm. The H¨older norms satisfy the following monotonicity property, which is related to the power-sum inequality given by Fact 1.17.35. Proposition 9.1.5. Let 1 ≤ p ≤ q ≤ ∞, and let x ∈ Fn. Then, x∞ ≤ xq ≤ xp ≤ x1 .

(9.1.6)

Assume, in addition, that 1 < p < q < ∞. Then, x has at least two nonzero components if and only if x∞ < xq < xp < x1 .

(9.1.7)

Proof. If either p = q or x = 0 or x has exactly one nonzero component, then xq = xp . Hence, to prove both (9.1.6) and (9.1.7), it suffices to prove (9.1.7) for the case in which 1 < p < q < ∞ and x has at least two nonzero components. Thus, let n ≥ 2, let x ∈ Fn have at least two nonzero components, and define  f : [1, ∞) → [0, ∞) by f(β) = xβ . Hence, f (β) =

1−β 1 β xβ

n 

γi ,

i=1

where, for all i ∈ {1, . . . , n}, * β   0, |xi | log |x(i) | − log xβ , x(i) =  γi = 0, x(i) = 0. If x(i) = 0, then log |x(i) | < log xβ . It thus follows that f (β) < 0, which implies that f is decreasing on [1, ∞). Hence, (9.1.7) holds. The following result is H¨ older’s inequality. For this result we interpret 1/∞ = 0. Note that, for all x, y ∈ Fn , |xTy| ≤ |x|T |y| = x ◦ y1 . Proposition 9.1.6. Let p, q ∈ [1, ∞] satisfy 1/p + 1/q = 1, and let x, y ∈ Fn. Then, |xTy| ≤ xp yq . (9.1.8) Furthermore, equality holds if and only if |xTy| = |x|T |y| and ⎧ |x| ◦ |y| = y∞ |x|, p = 1, ⎪ ⎪ ⎨ 1/p 1/q ◦1/q ◦1/p yq |x| = xp |y| , 1 < p < ∞, ⎪ ⎪ ⎩ |x| ◦ |y| = x∞ |y|, p = ∞.

(9.1.9)

600

CHAPTER 9

Proof. See [279, p. 127], [728, p. 536], [823, p. 71], Fact 1.18.11, and Fact 1.18.12. The case p = q = 2 is the Cauchy-Schwarz inequality. Corollary 9.1.7. Let x, y ∈ Fn. Then, |xTy| ≤ x2 y2 .

(9.1.10)

Furthermore, equality holds if and only if x and y are linearly dependent. √

 Proof. Suppose that y = 0, and define M = y ∗yI (y ∗y)−1/2 y . Since M ∗M∗  y = yyyI is positive semidefinite, it follows from iii) of Proposition 8.2.4 that ∗ 1

yy ∗ ≤ y ∗yI. Therefore, x∗yy ∗x ≤ x∗xy ∗y, which is equivalent to (9.1.10) with x replaced by x. Now, suppose that x and y are linearly dependent. Then, there exists β ∈ F such that either x = βy or y = βx. In both cases it follows that |x∗ y| = x2 y2.  Conversely, define f : Fn × Fn → [0, ∞) by f(μ, ν) = μ∗μν ∗ν − |μ∗ν|2. Now, suppose that f(x, y) = 0 so that (x, y) minimizes f. Then, it follows that fμ (x, y) = 0, which implies that y ∗yx = y ∗xy. Hence, x and y are linearly dependent. Let x, y ∈ Fn, assume that x and y are both nonzero, let p, q ∈ [1, ∞], and assume that 1/p + 1/q = 1. Since xp = xp , it follows that

and, in particular,

|x∗y| ≤ xp yq ,

(9.1.11)

|x∗y| ≤ x2 y2 .

(9.1.12)

The angle θ ∈ [0, π] between x and y, which is defined by (2.2.20), is thus given by θ = cos−1 

x∗ y . x2 y2

(9.1.13)

The norms  ·  and  ·  on Fn are equivalent if there exist α, β > 0 such that αx ≤ x ≤ βx

(9.1.14)

for all x ∈ F . Note that these inequalities can be written as n

1 β x

≤ x ≤

1 α x .

(9.1.15)

Hence, the word “equivalent” is justified. The following result shows that every pair of norms on Fn is equivalent. Theorem 9.1.8. Let  ·  and  ·  be norms on Fn. Then,  ·  and  ·  are equivalent. Proof. See [728, p. 272].

601

NORMS

9.2 Matrix Norms One way to define norms for matrices is by viewing a matrix A ∈ Fn×m as a vector in Fnm, for example, as vec A. Definition 9.2.1. A norm  ·  on Fn×m is a function  · : Fn×m → [0, ∞) that satisfies the following conditions: i) A ≥ 0 for all A ∈ Fn×m. ii) A = 0 if and only if A = 0. iii) αA = |α|A for all α ∈ F and A ∈ Fn×m. iv) A + B ≤ A + B for all A, B ∈ Fn×m. 

If  ·  is a norm on Fnm, then  ·  defined by A =  vec A is a norm on F . For example, H¨older norms can be defined for matrices by choosing  ·  =  · p . Hence, for all A ∈ Fn×m, define ⎧⎛ ⎞1/p ⎪ m n  ⎪  ⎪ ⎪ ⎝ ⎪ |A(i,j) |p⎠ , 1 ≤ p < ∞, ⎪ ⎨  i=1 j=1 Ap = (9.2.1) ⎪ ⎪ ⎪ ⎪ p = ∞. max |A(i,j) |, ⎪ ⎪ ⎩ i∈{1,...,n} n×m

j∈{1,...,m}

Note that the same symbol  · p is used to denote the H¨older norm for both vectors and matrices. This notation is consistent since, if A ∈ Fn×1, then Ap coincides with the vector H¨older norm. Furthermore, if A ∈ Fn×m and 1 ≤ p ≤ ∞, then Ap =  vec Ap .

(9.2.2)

It follows from (9.1.6) that, if A ∈ Fn×m and 1 ≤ p ≤ q ≤ ∞, then A∞ ≤ Aq ≤ Ap ≤ A1 .

(9.2.3)

If, in addition, 1 < p < q < ∞ and A has at least two nonzero entries, then A∞ < Aq < Ap < A1 .

(9.2.4)

The H¨older norms in the cases p = 1, 2, ∞ are the most commonly used. Let A ∈ Fn×m. For p = 2 we define the Frobenius norm  · F by 

AF = A2 .

(9.2.5)

Since A2 =  vec A2 , it follows that AF = A2 =  vec A2 =  vec AF . It is easy to see that

AF =

√

tr A∗A.

(9.2.6) (9.2.7)

602

CHAPTER 9

Let  ·  be a norm on Fn×m. If S1AS2  = A for all A ∈ Fn×m and for all unitary matrices S1 ∈ Fn×n and S2 ∈ Fm×m, then  ·  is unitarily invariant. Now, let m = n. If A = A∗  for all A ∈ Fn×n, then  ·  is self-adjoint. If In  = 1, then√ ·  is normalized. Note that the Frobenius norm is not normalized since In F = n. If SAS ∗  = A for all A ∈ Fn×n and for all unitary S ∈ Fn×n, then  ·  is weakly unitarily invariant. Matrix norms can be defined in terms of singular values. Let σ1(A) ≥ σ2 (A) ≥ · · · denote the singular values of A ∈ Fn×m. The following result gives a weak majorization condition for singular values. Proposition 9.2.2. Let A, B ∈ Fn×m. Then, for all k ∈ {1, . . . , min{n, m}}, k 

[σi(A) − σi(B)] ≤

i=1

k 

σi(A + B) ≤

i=1

k 

[σi(A) + σi(B)].

(9.2.8)

i=1

In particular, σmax (A) − σmax (B) ≤ σmax (A + B) ≤ σmax (A) + σmax (B) and

tr A − tr B ≤ tr A + B ≤ tr A + tr B.

  Proof. Define A, B ∈ Hn+m by A = A0∗ A0 and B = B0∗ lary 8.6.19 implies that, for all k ∈ {1, . . . , n + m}, k  i=1

λi(A + B) ≤

k 

(9.2.9) (9.2.10)

B 0

. Then, Corol-

[λi(A) + λi(B)].

i=1

Now, consider k ≤ min{n, m}. Then, it follows from Proposition 5.6.5 that, for all i ∈ {1, . . . , k}, λi(A) = σi(A). Setting k = 1 yields (9.2.9), while setting k = min{n, m} and using Fact 8.18.2 yields (9.2.10).

by

Proposition 9.2.3. Let p ∈ [1, ∞], and let A ∈ Fn×m. Then,  · σp defined ⎧⎛ ⎞1/p ⎪ ⎪ min{n,m}  ⎪ ⎨⎝ σip(A)⎠ , 1 ≤ p < ∞,  (9.2.11) Aσp = i=1 ⎪ ⎪ ⎪ ⎩ σmax (A), p = ∞,

is a norm on Fn×m. Proof. Let p ∈ [1, ∞]. Then, it follows from Proposition 9.2.2 and Minkowski’s inequality Fact 1.18.25 that

603

NORMS

⎛



A + Bσp = ⎝ ⎛

σip(A

+ B)⎠

i=1



⎞1/p

min{n,m}

≤⎝ ⎛

⎞1/p

min{n,m}

[σi(A) + σi(B)]p⎠

i=1



min{n,m}

≤⎝

⎞1/p

⎛



min{n,m}

σip(A)⎠ + ⎝

i=1

⎞1/p

σip(B)⎠

i=1

= Aσp + Bσp .



The norm  · σp is a Schatten norm. Let A ∈ Fn×m. Then, for all p ∈ [1, ∞), 1/p

Aσp = (tr Ap )

.

(9.2.12)

Special cases are Aσ1 = σ1(A) + · · · + σmin{n,m}(A) = tr A,  1/2 2 (A) = (tr A∗A)1/2 = AF , Aσ2 = σ12(A) + · · · + σmin{n,m} and

Aσ∞ = σ1(A) = σmax (A),

(9.2.13) (9.2.14) (9.2.15)

which are the trace norm, Frobenius norm, and spectral norm, respectively. By applying Proposition 9.1.5 to the vector obtain the following result.



σ1(A) · · · σmin{n,m}(A)

T , we

Proposition 9.2.4. Let p, q ∈ [1, ∞), where p ≤ q, and let A ∈ Fn×m. Then, Aσ∞ ≤ Aσq ≤ Aσp ≤ Aσ1 .

(9.2.16)

Assume, in addition, that 1 < p < q < ∞ and rank A ≥ 2. Then, Aσ∞ < Aσq < Aσp < Aσ1 .

(9.2.17)

The norms  · σp are not very interesting when applied to vectors. Let x ∈ Fn = Fn×1. Then, σmax (x) = (x∗x)1/2 = x2 , and, since rank x ≤ 1, it follows that, for all p ∈ [1, ∞], xσp = x2 .

(9.2.18)

Proposition 9.2.5. Let A ∈ Fn×m. If p ∈ (0, 2], then If p ≥ 2, then Proof. See [1521, p. 50].

Aσp ≤ Ap .

(9.2.19)

Ap ≤ Aσp .

(9.2.20)

604

CHAPTER 9

Proposition 9.2.6. Let  ·  be a norm on Fn×n, and let A ∈ Fn×n. Then, sprad(A) = lim Ak 1/k.

(9.2.21)

k→∞

Proof. See [728, p. 322].

9.3 Compatible Norms The norms  · ,  ·  , and  ·  on Fn×l, Fn×m, and Fm×l, respectively, are compatible if, for all A ∈ Fn×m and B ∈ Fm×l, AB ≤ A B .

(9.3.1)

For l = 1, the norms  · ,  ·  , and  ·  on Fn, Fn×m, and Fm, respectively, are compatible if, for all A ∈ Fn×m and x ∈ Fm, Ax ≤ A x .

(9.3.2)

Furthermore, the norm  ·  on F is compatible with the norm  ·  on F all A ∈ Fn×n and x ∈ Fn, Ax ≤ A x. n



n×n

if, for

(9.3.3)

is submultiplicative if, for all A, B ∈ Note that In  ≥ 1. The norm  ·  on F Fn×n, AB ≤ AB. (9.3.4) n×n

Hence, the norm  ·  on Fn×n is submultiplicative if and only if  · ,  · , and  ·  are compatible. In this case, In  ≥ 1, while  ·  is normalized if and only if In  = 1. Proposition 9.3.1. Let  ·  be a submultiplicative norm on Fn×n, and let  y ∈ Fn be nonzero. Then, x = xy ∗  is a norm on Fn, and  ·  is compatible with  ·  . Proof. Note that Ax = Axy ∗  ≤ A xy ∗  = A x.



Proposition 9.3.2. Let  ·  be a submultiplicative norm on Fn×n, and let A ∈ Fn×n. Then, sprad(A) ≤ A. (9.3.5) Proof. Use Proposition 9.3.1 to construct a norm · on Fn that is compatible with  · . Furthermore, let A ∈ Fn×n, let λ ∈ spec(A), and let x ∈ Cn be an eigenvector of A associated with λ. Then, Ax = λx implies that |λ|x = Ax ≤ Ax , and thus |λ| ≤ A, which implies (9.3.5). Alternatively, under the additional assumption that  ·  is submultiplicative, it follows from Proposition 9.2.6 that sprad(A) = lim Ak 1/k ≤ lim Ak/k = A.  k→∞

k→∞

605

NORMS

Proposition 9.3.3. Let A ∈ Fn×n, and let ε > 0. Then, there exists a submultiplicative norm  ·  on Fn×n such that sprad(A) ≤ A ≤ sprad(A) + ε.

(9.3.6)

Proof. See [728, p. 297] or [1208, p. 167]. Corollary 9.3.4. Let A ∈ Fn×n, and assume that sprad(A) < 1. Then, there exists a submultiplicative norm  ·  on Fn×n such that A < 1. We now identify some compatible norms. We begin with the H¨ older norms. Proposition 9.3.5. Let A ∈ Fn×m and B ∈ Fm×l. If p ∈ [1, 2], then ABp ≤ Ap Bp .

(9.3.7)

If p ∈ [2, ∞] and q satisfies 1/p + 1/q = 1, then

and

ABp ≤ Ap Bq

(9.3.8)

ABp ≤ Aq Bp .

(9.3.9)

 p/(p−1) ≥ 2. Using H¨ older’s inequality Proof. First let 1 ≤ p ≤ 2 so that q = (9.1.8) and (9.1.6) with p ≤ q yields ⎛ ⎞1/p n,l  ABp = ⎝ |rowi(A)colj (B)|p⎠

i,j=1

⎛ ≤⎝

n,l 

⎞1/p rowi(A)pp colj (B)pq⎠

i,j=1

 =

n 

⎞1/p 1/p⎛ l  rowi(A)pp ⎝ colj (B)pq⎠

i=1

j=1

i=1

j=1

⎞1/p 1/p⎛ l  n   ≤ rowi(A)pp ⎝ colj (B)pp⎠ = Ap Bp . 

Next, let 2 ≤ p ≤ ∞ so that q = p/(p − 1) ≤ 2. Using H¨older’s inequality (9.1.8) and (9.1.6) with q ≤ p yields

606

CHAPTER 9

ABp ≤

 n  i=1

≤

 n 

⎞1/p 1/p⎛ l  rowi(A)pp ⎝ colj (B)pq⎠ j=1

⎞1/q 1/p⎛ l  rowi(A)pp ⎝ colj (B)qq⎠

i=1

j=1

= Ap Bq . Similarly, it can be shown that (9.3.9) holds. Proposition 9.3.6. Let A ∈ Fn×m, B ∈ Fm×l, and p, q ∈ [1, ∞], define 

r=

1 p

1 , + 1q

and assume that r ≥ 1. Then, ABσr ≤ Aσp Bσq .

(9.3.10)

ABσr ≤ Aσ2r Bσ2r .

(9.3.11)

In particular,

Proof. Using Proposition 9.6.2 and H¨ older’s inequality with 1/(p/r) + 1/(q/r) = 1, it follows that ⎛ ⎞1/r min{n,m,l}  ABσr = ⎝ σir(AB)⎠ ⎛

i=1



min{n,m,l}

≤⎝

⎞1/r

σir(A)σir(B)⎠

i=1

⎡⎛ ⎞r/p⎛ ⎞r/q ⎤1/r min{n,m,l} min{n,m,l}   ⎢ ⎥ ≤ ⎣⎝ σip(A)⎠ ⎝ σiq(B)⎠ ⎦ i=1

i=1

= Aσp Bσq .



Corollary 9.3.7. Let A ∈ Fn×m and B ∈ Fm×l. Then, ⎧ ⎫ Aσ∞ Bσ2 ⎪ ⎪ ⎪ ⎪ ⎨ ⎬ Aσ2 Bσ∞ ≤ Aσ2 Bσ2 ABσ∞ ≤ ABσ2 ≤ ⎪ ⎪ ⎪ ⎪ ⎩ ⎭ ABσ1 or, equivalently,

⎧ σmax (A)BF ⎪ ⎪ ⎨ AF σmax (B) σmax (AB) ≤ ABF ≤ ⎪ ⎪ ⎩ tr AB

⎫ ⎪ ⎪ ⎬ ⎪ ⎪ ⎭

≤ AF BF .

(9.3.12)

(9.3.13)

607

NORMS

Furthermore, for all r ∈ [1, ∞], ⎧ Aσr σmax (B) ⎪ ⎪ ⎨ σmax (A)Bσr ABσ2r ≤ ABσr ≤ ⎪ ⎪ ⎩ Aσ2r Bσ2r

⎫ ⎪ ⎪ ⎬ ⎪ ⎪ ⎭

≤ Aσr Bσr .

(9.3.14)

In particular, setting r = ∞ yields σmax (AB) ≤ σmax (A)σmax(B). Corollary 9.3.8. Let A ∈ Fn×m and B ∈ Fm×l. Then, * σmax (A)Bσ1 ABσ1 ≤ Aσ1 σmax (B).

(9.3.15)

(9.3.16)

Note that the inequality ABF ≤ AF BF in (9.3.13) is equivalent to (9.3.7) with p = 2 as well as (9.3.8) and (9.3.9) with p = q = 2. The following result is the matrix version of the Cauchy-Schwarz inequality given by Corollary 9.1.7. Corollary 9.3.9. Let A ∈ Fn×m and B ∈ Fn×m. Then, |tr A∗B| ≤ AF BF .

(9.3.17)

Equality holds if and only if A and B are linearly dependent.

9.4 Induced Norms In this section we consider the case in which there exists a nonzero vector x ∈ Fm such that (9.3.3) holds as an equality. This condition characterizes a special class of norms on Fn×n, namely, the induced norms. Definition 9.4.1. Let  ·  and  ·  be norms on Fm and Fn, respectively. Then,  ·  : Fn×m → F defined by A =

Ax \{0} x

max m

x∈F

(9.4.1)

is an induced norm on Fn×m. In this case,  ·  is induced by  ·  and  · . If m = n and  ·  =  · , then  ·  is induced by  · , and  ·  is an equi-induced norm. The next result confirms that  ·  defined by (9.4.1) is a norm. Theorem 9.4.2. Every induced norm is a norm. Furthermore, every equiinduced norm is normalized. Proof. See [728, p. 293].

608

CHAPTER 9

Let A ∈ Fn×m. It can be seen that (9.4.1) is equivalent to A =

max

x∈{y∈Fm : y =1}

Ax.

(9.4.2)

Theorem 10.3.8 implies that the maximum in (9.4.2) exists. Since, for all x = 0, A =

max

x∈Fm \{0}

Ax Ax ≥ , x x

(9.4.3)

it follows that, for all x ∈ Fm, Ax ≤ A x

(9.4.4)

so that  · ,  ·  , and  ·  are compatible. If m = n and  ·  =  · , then the norm  ·  is compatible with the induced norm  ·  . The next result shows that compatible norms can be obtained from induced norms. Proposition 9.4.3. Let  · ,  ·  , and  ·  be norms on Fl, Fm, and Fn, respectively. Furthermore, let  ·  be the norm on Fm×l induced by  ·  and  ·  , let  ·  be the norm on Fn×m induced by  ·  and  ·  , and let  ·  be the norm on Fn×l induced by  ·  and  ·  . If A ∈ Fn×m and B ∈ Fm×l, then AB ≤ A B .

(9.4.5)

Proof. Note that, for all x ∈ Fl , Bx ≤ B x, and, for all y ∈ Fm , Ay ≤ A y . Hence, for all x ∈ Fl, it follows that

ABx ≤ A Bx ≤ A B x, which implies that AB = max

x∈Fl \{0}

ABx ≤ A B . x



Corollary 9.4.4. Every equi-induced norm is submultiplicative. The following result is a consequence of Corollary 9.4.4 and Proposition 9.3.2. Corollary 9.4.5. Let · be an equi-induced norm on Fn×n, and let A ∈ Fn×n. Then, sprad(A) ≤ A. (9.4.6) By assigning  · p to Fm and  · q to Fn, where p ≥ 1 and q ≥ 1, the H¨ older-induced norm on Fn×m is defined by Axq  . (9.4.7) Aq,p = max x∈Fm \{0} xp Proposition 9.4.6. Let p, q, p , q ∈ [1, ∞], where p ≤ p and q ≤ q , and let A ∈ Fn×m. Then, Aq,p ≤ Aq,p ≤ Aq,p . (9.4.8) Proof. This result follows from Proposition 9.1.5.

609

NORMS

A subtlety of induced norms is that the value of an induced norm may depend on the underlying field. In particular, the value of the induced norm of a real matrix A computed over the complex field may be different from the induced norm of A computed over the real field. Although the chosen field is usually not made explicit, we do so in special cases for clarity. Proposition 9.4.7. Let A ∈ Rn×m, and let Ap,q,F denote the H¨olderinduced norm of A evaluated over the field F, where p ≥ 1 and q ≥ 1. Then, Ap,q,R ≤ Ap,q,C .

(9.4.9)

Ap,1,R = Ap,1,C .

(9.4.10)

If p ∈ [1, ∞], then Finally, if p, q ∈ [1, ∞] satisfy 1/p + 1/q = 1, then A∞,p,R = A∞,p,C .

(9.4.11)

Proof. See [708, p. 716] and [1183].

T and x = x1 x2 . Then, Ax1 =

T so that x∞ = 1, it follows that |x1 − x2 | + |x1 + x2 |. Letting x = 1 j √ A1,∞,C ≥ 2 2. On the other hand, A1,∞,R = 2. Hence, in this case, the inequality (9.4.9) is strict. See [708, p. 716]. Example 9.4.8. Let A =

1 −1 1 1

The following result gives explicit expressions for several H¨older-induced norms. Proposition 9.4.9. Let A ∈ Fn×m. Then, A2,2 = σmax (A).

(9.4.12)

If p ∈ [1, ∞], then Ap,1 =

max

i∈{1,...,m}

coli(A)p .

(9.4.13)

Finally, if p, q ∈ [1, ∞] satisfy 1/p + 1/q = 1, then A∞,p =

max

i∈{1,...,n}

rowi(A)q .

(9.4.14)

Proof. Since A∗A is Hermitian, it follows from Corollary 8.4.2 that, for all x∈F , x∗A∗Ax ≤ λmax(A∗A)x∗x, m

which implies that, for all x ∈ Fm, Ax2 ≤ σmax (A)x2 , and thus A2,2 ≤ σmax (A). Now, let x ∈ Fn×n be an eigenvector associated with λmax(A∗A) so that Ax2 = σmax (A)x2 , which implies that σmax (A) ≤ A2,2 . Hence, (9.4.12) holds.

610

CHAPTER 9

Next, note that, for all x ∈ Fm, Dm D m D D  D D x(i) coli(A)D ≤ |x(i) |coli(A)p ≤ max coli(A)p x1 , Axp = D D D i∈{1,...,m} i=1

i=1

p

and hence Ap,1 ≤ maxi∈{1,...,m} coli(A)p . Next, let j ∈ {1, . . . , m} be such that colj (A)p = maxi∈{1,...,m} coli(A)p . Now, since ej1 = 1, it follows that Aej p = colj (A)p ej1, which implies that max coli(A)p = colj (A)p ≤ Ap,1 ,

i∈{1,...,n}

and hence (9.4.13) holds. Next, for all x ∈ Fm, it follows from H¨ older’s inequality (9.1.8) that Ax∞ =

max |rowi(A)x| ≤

i∈{1,...,n}

max rowi(A)q xp ,

i∈{1,...,n}

which implies that A∞,p ≤ maxi∈{1,...,n} rowi(A)q . Next, let j ∈ {1, . . . , n} be such that rowj (A)q = maxi∈{1,...,n} rowi(A)q , and let nonzero x ∈ Fm be such that |rowj (A)x| = rowj (A)q xp . Hence, Ax∞ =

max |rowi(A)x| ≥ |rowj (A)x| = rowj (A)q xp ,

i∈{1,...,n}

which implies that max rowi(A)q = rowj (A)q ≤ A∞,p ,

i∈{1,...,n}

and thus (9.4.14) holds. Let A ∈ Fn×m. Then, ∗ coli(A)2 = d1/2 max (A A)

max

(9.4.15)

i∈{1,...,m}

and ∗ max rowi(A)2 = d1/2 max (AA ).

i∈{1,...,n}

(9.4.16)

Therefore, it follows from Proposition 9.4.9 that A1,1 = A2,1 =

coli(A)1,

(9.4.17)

∗ coli(A)2 = d1/2 max(A A),

(9.4.18)

max

i∈{1,...,m}

max i∈{1,...,m}

A∞,1 = A∞ = A∞,2 =

max i∈{1,...,n}

A∞,∞ =

max i∈{1,...,n} j∈{1,...,m}

|A(i,j) |,

∗ rowi(A)2 = d1/2 max(AA ),

max rowi(A)1 .

i∈{1,...,n}

(9.4.19)

(9.4.20) (9.4.21)

611

NORMS

For convenience, we define the column norm  Acol = A1,1

(9.4.22)

and the row norm 

Arow = A∞,∞ .

(9.4.23)

AT col = Arow .

(9.4.24)

Note that

The following result follows from Corollary 9.4.5. Corollary 9.4.10. Let A ∈ Fn×n. Then, sprad(A) ≤ σmax (A),

(9.4.25)

sprad(A) ≤ Acol ,

(9.4.26)

sprad(A) ≤ Arow .

(9.4.27)

Proposition 9.4.11. Let p, q ∈ [1, ∞] be such that 1/p + 1/q = 1, and let A ∈ Fn×m. Then, (9.4.28) Aq,p ≤ Aq . Proof. For p = 1 and q = ∞, (9.4.28) follows from (9.4.19). For q < ∞ and x ∈ Fn, it follows from H¨ older’s inequality (9.1.8) that 1/q  n 1/q  n   q q q Axq = |rowi(A)x| ≤ rowi(A)q xp i=1

i=1

⎛ ⎞1/q m n   =⎝ |A(i,j) |q⎠ xp = Aq xp , i=1 j=1

which implies (9.4.28). Next, we specialize Proposition 9.4.3 to the H¨older-induced norms. Corollary 9.4.12. Let p, q, r ∈ [1, ∞], and let A ∈ Fn×m and A ∈ Fm×l. Then, ABr,p ≤ Ar,q Bq,p .

(9.4.29)

ABcol ≤ Acol Bcol,

(9.4.30)

σmax (AB) ≤ σmax (A)σmax (B),

(9.4.31)

ABrow ≤ Arow Brow ,

(9.4.32)

AB∞ ≤ A∞ Bcol ,

(9.4.33)

AB∞ ≤ Arow B∞ ,

(9.4.34)

In particular,

612

CHAPTER 9 ∗ ∗ 1/2 ∗ d1/2 max (B A AB) ≤ dmax (A A)Bcol ,

(9.4.35)

∗ ∗ 1/2 ∗ d1/2 max (B A AB) ≤ σmax (A)dmax (B B),

(9.4.36)

∗ ∗ 1/2 ∗ d1/2 max (ABB A ) ≤ dmax (AA )σmax (B),

(9.4.37)

∗ ∗ 1/2 ∗ d1/2 max (ABB A ) ≤ Brow dmax (BB ).

(9.4.38)

The following result is often useful. Proposition 9.4.13. Let A ∈ Fn×n, and assume that sprad(A) < 1. Then, there exists norm · on Fn×n such that A < 1. Furthermore, a∞submultiplicative k the series k=0 A converges absolutely, and (I − A)−1 =

∞ 

Ak.

(9.4.39)

k=0

Finally, D D 1 1 ≤ D(I − A)−1D ≤ + I − 1. 1 + A 1 − A If, in addition,  ·  is normalized, then D D 1 1 ≤ D(I − A)−1D ≤ . 1 + A 1 − A

(9.4.40)

(9.4.41)

Proof. Corollary 9.3.4 implies that there exists a submultiplicative norm  ·  on Fn×n such that A < 1. It thus follows that D D ∞ ∞ ∞ D  D  1 D kD + I − 1, A D≤ Ak  ≤ I − 1 + Ak = D D D 1 − A k=0 k=0 k=0 ∞ which proves that the series k=0 Ak converges absolutely. Next, we show that I −A is nonsingular. If I −A is singular, then there exists a nonzero vector x ∈ Cn such that Ax = x. Hence, 1 ∈ spec(A), which contradicts sprad(A) < 1. Next, to verify (9.4.39), note that ∞ ∞ ∞ ∞ ∞      (I − A) Ak = Ak − Ak = I + Ak − Ak = I, k=0

k=0

k=1

k=1

k=1

which implies (9.4.39) and thus the right-hand inequality in (9.4.40). Furthermore, 1 ≤ I D D = D(I − A)(I − A)−1D D D ≤ I − A D(I − A)−1D D D ≤ (1 + A) D(I − A)−1D , which yields the left-hand inequality in (9.4.40).

613

NORMS

9.5 Induced Lower Bound We now consider a variation of the induced norm. Definition 9.5.1. Let  ·  and  ·  denote norms on Fm and Fn, respectively, and let A ∈ Fn×m. Then, : Fn×m → R defined by ⎧ y ⎪ max , A = 0, ⎨ min y∈R(A)\{0} x∈{z∈Fm : Az=y} x 

(A) = (9.5.1) ⎪ ⎩0, A = 0, is the lower bound induced by  ·  and  ·  . Equivalently, ⎧ Ax ⎪ max x+z , A = 0, ⎨ min m x∈F \N(A) z∈N(A) 

(A) = ⎪ ⎩0, A = 0.

(9.5.2)

Proposition 9.5.2. Let  ·  and  ·  be norms on Fm and Fn, respectively, let  ·  be the norm induced by  ·  and  ·  , let  ·  be the norm induced by  ·  and  · , and let be the lower bound induced by  ·  and  ·  . Then, the following statements hold: i) (A) exists for all A ∈ Fn×m, that is, the minimum in (9.5.1) is attained. ii) If A ∈ Fn×m, then (A) = 0 if and only if A = 0. iii) For all A ∈ Fn×m there exists a vector x ∈ Fm such that iv) For all A ∈ Fn×m,

(A)x = Ax .

(9.5.3)

(A) ≤ A .

(9.5.4)

v) If A = 0 and B is a (1)-inverse of A, then 1/B ≤ (A) ≤ B .

(9.5.5)

vi) If A, B ∈ Fn×m and either R(A) ⊆ R(A + B) or N(A) ⊆ N(A + B), then

(A) − B ≤ (A + B). vii) If A, B ∈ F

n×m

(9.5.6)

and either R(A + B) ⊆ R(A) or N(A + B) ⊆ N(A), then

(A + B) ≤ (A) + B .

(9.5.7)

viii) If n = m and A ∈ Fn×n is nonsingular, then

(A) = 1/A−1 .

(9.5.8)

Proof. See [596]. Proposition 9.5.3. Let  · ,  ·  , and  ·  be norms on Fl, Fm, and Fn, respectively, let  ·  denote the norm on Fm×l induced by  ·  and  ·  , let  ·  denote the norm on Fn×m induced by  ·  and  ·  , and let  ·  denote the

614

CHAPTER 9

norm on Fn×l induced by  ·  and  ·  . If A ∈ Fn×m and B ∈ Fm×l, then

(A) (B) ≤ (AB).

(9.5.9)

In addition, the following statements hold: i) If either rank B = rank AB or def B = def AB, then

(AB) ≤ A (B).

(9.5.10)

(AB) ≤ (A)B .

(9.5.11)

iii) If rank B = m, then

A (B) ≤ AB .

(9.5.12)

iv) If rank A = m, then

(A)B ≤ AB .

(9.5.13)

ii) If rank A = rank AB, then

Proof. See [596]. By assigning  · p to Fm and  · q to Fn, where p ≥ 1 and q ≥ 1, the H¨ older-induced lower bound on Fn×m is defined by ⎧ yq ⎪ max , A=  0, ⎨ min m : Az=y} xp y∈R(A)\{0} x∈{z∈F 

q,p (A) = (9.5.14) ⎪ ⎩0, A = 0. The following result shows that 2,2 (A) is the smallest positive singular value of A.  Proposition 9.5.4. Let A ∈ Fn×m, assume that A is nonzero, and let r = rank A. Then,

2,2 (A) = σr(A). (9.5.15)

Proof. This result follows from the singular value decomposition. Corollary 9.5.5. Let A ∈ Fn×m. If n ≤ m and A is right invertible, then

2,2 (A) = σmin(A) = σn(A).

(9.5.16)

If m ≤ n and A is left invertible, then

2,2 (A) = σmin(A) = σm(A). Finally, if n = m and A is nonsingular, then    

2,2 A−1 = σmin A−1 =

1 σmax (A)

(9.5.17)

.

(9.5.18)

Proof. Use Proposition 5.6.2 and Fact 6.3.28. In contrast to the submultiplicativity condition (9.4.4) satisfied by the induced norm, the induced lower bound satisfies a supermultiplicativity condition. The following result is analogous to Proposition 9.4.3.

615

NORMS

Proposition 9.5.6. Let  · ,  ·  , and  ·  be norms on Fl, Fm, and Fn, respectively. Let (·) be the lower bound induced by  ·  and  ·  , let (·) be the lower bound induced by  ·  and  ·  , let (·) be the lower bound induced by  ·  and  ·  , let A ∈ Fn×m and B ∈ Fm×l, and assume that either A or B is right invertible. Then,

(A) (B) ≤ (AB). (9.5.19) Furthermore, if 1 ≤ p, q, r ≤ ∞, then

r,q (A) q,p (B) ≤ r,p (AB).

(9.5.20)

σm(A)σl(B) ≤ σl(AB).

(9.5.21)

In particular,

Proof. See [596] and [892, pp. 369, 370].

9.6 Singular Value Inequalities Proposition 9.6.1. Let A ∈ Fn×m and B ∈ Fm×l. Then, for all i ∈ {1, . . . , min{n, m}} and j ∈ {1, . . . , min{m, l}} such that i + j ≤ min{n, l} + 1, σi+j−1(AB) ≤ σi(A)σj (B).

(9.6.1)

In particular, for all i ∈ {1, . . . , min{n, m, l}}, σi(AB) ≤ σmax (A)σi(B)

(9.6.2)

σi(AB) ≤ σi(A)σmax (B).

(9.6.3)

and

Proof. See [730, p. 178]. Proposition 9.6.2. Let A ∈ Fn×m and B ∈ Fm×l. If r ≥ 0, then, for all k ∈ {1, . . . , min{n, m, l}}, k 

σir(AB)

≤

i=1

k 

σir(A)σir(B).

(9.6.4)

i=1

In particular, for all k ∈ {1, . . . , min{n, m, l}}, k 

σi(AB) ≤

i=1

k 

σi(A)σi(B).

(9.6.5)

i=1

If r < 0, n = m = l, and A and B are nonsingular, then n  i=1

σir(AB) ≤

n 

σir(A)σir(B).

(9.6.6)

i=1

Proof. The first statement follows from Proposition 9.6.3 and Fact 2.21.8. For the case r < 0, use Fact 2.21.11. See [201, p. 94] or [730, p. 177].

616

CHAPTER 9

Proposition 9.6.3. Let A ∈ Fn×m and B ∈ Fm×l. Then, for all k ∈ {1, . . . , min{n, m, l}}, k k ! ! σi(AB) ≤ σi(A)σi(B). i=1

i=1

If, in addition, n = m = l, then n !

σi(AB) =

i=1

n !

σi(A)σi(B).

i=1

Proof. See [730, p. 172]. Proposition 9.6.4. Let A ∈ Fn×m and B ∈ Fm×l. If m ≤ n, then, for all i ∈ {1, . . . , min{n, m, l}}, σmin(A)σi(B) = σm(A)σi(B) ≤ σi(AB).

(9.6.7)

If m ≤ l, then, for all i ∈ {1, . . . , min{n, m, l}}, σi(A)σmin(B) = σi(A)σm(B) ≤ σi(AB).

(9.6.8)

2 (A)Im = λmin(A∗A)Im ≤ A∗A, which Proof. Corollary 8.4.2 implies that σm 2 ∗ ∗ ∗ implies that σm(A)B B ≤ B A AB. Hence, it follows from the monotonicity theorem Theorem 8.4.9 that, for all i ∈ {1, . . . , min{n, m, l}}, 2

1/2 1/2 σm(A)σi(B) = λi σm (A)B ∗B ≤ λi (B ∗A∗AB) = σi(AB),

which proves the left-hand inequality in (9.6.7). Similarly, for all i ∈ {1, . . . , min{n, m, l}}, 2

1/2 1/2 σi(A)σm(B) = λi σm (B)AA∗ ≤ λi (ABB ∗A∗ ) = σi(AB).  Corollary 9.6.5. Let A ∈ Fn×m and B ∈ Fm×l. Then, σm(A)σmin{n,m,l}(B) ≤ σmin{n,m,l}(AB) ≤ σmax (A)σmin{n,m,l}(B), σm(A)σmax (B) ≤ σmax (AB) ≤ σmax (A)σmax (B), σmin{n,m,l}(A)σm(B) ≤ σmin{n,m,l}(AB) ≤ σmin{n,m,l}(A)σmax (B), σmax (A)σm(B) ≤ σmax (AB) ≤ σmax (A)σmax (B).

(9.6.9) (9.6.10) (9.6.11) (9.6.12)

Specializing Corollary 9.6.5 to the case in which A or B is square yields the following result. Corollary 9.6.6. Let A ∈ Fn×n and B ∈ Fn×l. Then, for all i ∈ {1, . . . , min{n, l}}, σmin (A)σi(B) ≤ σi(AB) ≤ σmax (A)σi(B).

(9.6.13)

σmin (A)σmax (B) ≤ σmax (AB) ≤ σmax (A)σmax (B).

(9.6.14)

In particular,

617

NORMS

If A ∈ Fn×m and B ∈ Fm×m, then, for all i ∈ {1, . . . , min{n, m}}, σi(A)σmin (B) ≤ σi(AB) ≤ σi(A)σmax (B).

(9.6.15)

σmax (A)σmin (B) ≤ σmax (AB) ≤ σmax (A)σmax (B).

(9.6.16)

In particular,

Corollary 9.6.7. Let A ∈ Fn×m and B ∈ Fm×l. If m ≤ n, then If m ≤ l, then

σmin(A)BF = σm(A)BF ≤ ABF .

(9.6.17)

AF σmin(B) = AF σm(B) ≤ ABF .

(9.6.18)

Proposition 9.6.8. Let A, B ∈ Fn×m. Then, for all i, j ∈ {1, . . . , min{n, m}} such that i + j ≤ min{n, m} + 1,

and

σi+j−1(A + B) ≤ σi(A) + σj (B)

(9.6.19)

σi+j−1(A) − σj (B) ≤ σi(A + B).

(9.6.20)

Proof. See [730, p. 178]. Corollary 9.6.9. Let A, B ∈ Fn×m. Then, σn(A) − σmax (B) ≤ σn(A + B) ≤ σn(A) + σmax (B).

(9.6.21)

If, in addition, n = m, then σmin(A) − σmax (B) ≤ σmin(A + B) ≤ σmin(A) + σmax (B).

(9.6.22)

Proof. This result follows from Proposition 9.6.8. Alternatively, it follows from Lemma 8.4.3 and the Cauchy-Schwarz inequality given by Corollary 9.1.7 that, for all nonzero x ∈ Fn, λmin [(A + B)(A + B)∗ ] ≤

x∗(AA∗ + BB ∗ + AB ∗ + BA∗ )x x∗x

=

x∗AA∗x x∗BB ∗x 2x∗AB ∗x + + Re 2 2 x2 x2 x22

≤

x∗AA∗x (x∗AA∗x)1/2 2 + σmax (B) + 2 σmax (B). 2 x2 x2

Minimizing with respect to x and using Lemma 8.4.3 yields σn2(A + B) = λmin [(A + B)(A + B)∗ ] 2 (B) + 2λmin(AA∗ )σmax (B) ≤ λmin(AA∗ ) + σmax 1/2

= [σn(A) + σmax (B)]2, which proves the right-hand inequality of (9.6.21). Finally, the left-hand inequality follows from the right-hand inequality with A and B replaced by A + B and −B, respectively.

618

CHAPTER 9

9.7 Facts on Vector Norms ◦2

|x|

Fact 9.7.1. Let x, y ∈ Fn. Then, x and y are linearly dependent if and only if and |y|◦2 are linearly dependent and |x∗y| = |x|T |y|.

Remark: This equivalence clarifies the relationship between (9.1.9) with p = 2 and Corollary 9.1.7. Fact 9.7.2. Let x, y ∈ Fn, and let  ·  be a norm on Fn. Then, * . . x + y . x − y . ≤ x − y. Fact 9.7.3. Let x, y ∈ Fn, and let  ·  be a norm on Fn. Then, the following statements hold: i) If there exists β ≥ 0 such that either x = βy or y = βx, then x + y = x + y. ii) If x + y = x + y and x and y are linearly dependent, then there exists β ≥ 0 such that either x = βy or y = βx. iii) If x + y2 = x2 + y2 , then there exists β ≥ 0 such that either x = βy or y = βx. iv) If x and y are linearly independent, then x + y2 < x2 + y2 . Proof: For iii), use v) of Fact 9.7.4.

Remark: Let x = 1 0 and y = 1 Then, x + y∞ = x∞ + y∞ = 2.

1 , which are linearly independent.

Problem: If x and y are linearly independent and p ∈ [1, ∞), then does it follow that x + yp < xp + yp ? Fact 9.7.4. Let x, y, z ∈ Fn. Then, the following statements hold:   i) 12 x + y22 + x − y22 = x22 + y22 . ii) If x and y are nonzero, then 1 2 (x2

D D D x y D D ≤ x − y2 . + y2 )D − D x2 y2 D2

iii) If x and y are nonzero, then D D D D D 1 D D 1 D D D D D x − x y = y − y x 2 D 2 D . D x2 D y2 2 2 iv) If F = R, then v) If F = C, then

4xTy = x + y22 − x − y22 .

  4x∗y = x + y22 − x − y22 + j x + jy22 − x − jy22 .     vi) Re x∗y = 14 x + y22 − x − y22 = 12 x + y22 − x22 − y22 .

619

NORMS

  vii) If F = C, then Im x∗y = 4j x + jy22 − x − jy22 . + viii) x + y2 = x22 + y22 + 2Re x∗y. + ix) x − y2 = x22 + y22 − 2Re x∗y. x) x + y2 x − y2 ≤ x22 + y22 . xi) If x + y2 = x2 + y2 , then Im x∗y = 0 and Re x∗y ≥ 0. xii) |x∗ y| ≤ x2 y2 . xiii) If x + y2 ≤ 2, then (1 − x22 )(1 − y22 ) ≤ |1 − Re x∗y|2. xiv) For all nonzero α ∈ R, x22 y22 − |x∗ y|2 ≤ α−2 αy − x22 x22 . xv) If Re x∗y = 0, then, for all nonzero α ∈ R, 2 2 −2 αy − x22 x22 , x22 y22 − |x∗y|2 ≤ α−2 0 α0 y − x2 x2 ≤ α  where α0 = x∗x/(Re x∗y).

xvi) x, y, z satisfy x + y22 + y + z22 + z + x22 = x22 + y22 + z22 + x + y + z22 and x + y2 + y + z2 + z + x2 ≤ x2 + y2 + z2 + x + y + z2 . xvii) |x∗zz ∗y − 12 x∗yz22| ≤ 12 x2 y2 z22.  xviii) |Re(x∗zz ∗y − 12 x∗yz22)| ≤ 12 z22 x22 y22 − (Im x∗y)2 .  xix) |Im(x∗zz ∗y − 12 x∗yz22)| ≤ 12 z22 x22 y22 − (Re x∗y)2 . Furthermore, the following statements are equivalent: xx) x − y2 = x + y2 . xxi) x + y22 = x22 + y22 . xxii) Re x∗y = 0. Now, let x1, . . . , xk ∈ Fn, and assume that, for all i, j ∈ {1, . . . , n}, x∗i xj = δij . Then, the following statement holds: k ∗ 2 2 xxiii) i=1 |y xi | ≤ y2 . If, in addition, k = n, then the following statement holds: n ∗ 2 2 xxiv) i=1 |y xi | = y2 . Remark: i) is the parallelogram law, which relates the diagonals and the sides of a parallelogram; ii) is the Dunkl-Williams inequality, which compares the distance between x and y with the distance between the projections of x and y onto the unit sphere (see [458], [1035, p. 515], and [1526, p. 28]); iv) and v) are the polarization

620

CHAPTER 9

identity (see [376, p. 54], [1057, p. 276], and Fact 1.20.2); ix) is the cosine law (see Fact 9.9.13 for a matrix version); xiii) is given in [1502] and implies Aczel’s inequality given by Fact 1.18.19; xv) is given in [939]; xvi) is Hlawka’s identity and Hlawka’s inequality (see Fact 1.10.6, Fact 1.20.2, [1035, p. 521], and [1066, p. 100]); xvii) is Buzano’s inequality (see [527] and Fact 1.19.2); xviii) and xix) are given in [1120]; the equivalence of xxi) and xxii) is the Pythagorean theorem; xxiii) is Bessel’s inequality; and xxiv) is Parseval’s identity. Note that xxiv) implies xxiii). Remark: Hlawka’s inequality is called the quadrilateral inequality in [1233], which gives a geometric interpretation. In addition, [1233] provides an extension and geometric interpretation to the polygonal inequalities. See Fact 9.7.7. Remark: When F = R and n = 2 the Euclidean norm of  [ xy ] 2 is equivalent to the absolute value |z| = |x + jy|. See Fact 1.20.2. Remark: δij is the Kronecker delta. Fact 9.7.5. Let x, y ∈ R3 , and let S ⊂ R3 be the parallelogram with vertices 0, x, y, and x + y. Then, area(S) = x × y2 . Remark: See Fact 2.20.13, Fact 2.20.14 and Fact 3.10.1. Remark: The parallelogram associated with the cross product can be interpreted as a bivector. See [436, pp. 86–88] or [620, 895]. Fact 9.7.6. Let x, y ∈ Rn, and assume that x and y are nonzero. Then, xTy (x2 + y2 ) ≤ x + y2 ≤ x2 + y2 . x2 y2 Hence, if xTy = x2 y2 , then x2 + y2 = x + y2 . Proof: See [1035, p. 517]. Remark: This result is a reverse triangle inequality. Problem: Extend this result to complex vectors. Fact 9.7.7. Let x1, . . . , xn ∈ Fn, and let α1, . . . , αn be nonnegative numbers. Then, D D D  D n  n D D n n n D D     D D D D D D αi Dxi − αj xj D ≤ αi xi 2 + αi − 2 D αi xi D . D D D D i=1 j=1 i=1 i=1 i=1 2 2

In particular,

D D D D D n D  n n D D   D n D D D D D ≤ x x  + (n − 2) x D D . j i 2 i D D D D i=1 Dj=1,j=i D i=1 i=1 2 2

Remark: The first inequality is the generalized Hlawka inequality or polygonal inequalities. The second inequality is the Djokovic inequality. See [1285] and Fact 9.7.4.

621

NORMS

Fact 9.7.8. Let x, y ∈ Rn, let α and δ, be positive numbers, and let p, q ∈ (0, ∞) satisfy 1/p + 1/q = 1. Then,  p−1 α δ p ≤ |δ − xTy|p + αp−1 xp2 . α + yq2 q Equality holds if and only if x = [δyq−2 2 /(α + y2 )]y. In particular,

αδ 2 ≤ (δ − xTy)2 + αx22 . α + y22 Equality holds if and only if x = [δ/(α + y22 )]y. Proof: See [1284]. Remark: The first inequality is due to Pecaric. The case p = q = 2 is due to Dragomir and Yang. These results are generalizations of Hua’s inequality. See Fact 1.17.13 and Fact 9.7.9. Fact 9.7.9. Let x1, . . . , xn , y ∈ Rn, and let α be a positive number. Then, D2 D n n D D   α D D 2 y2 ≤ Dy − xi D + α xi 22 . D D α+n i=1

2

i=1

Equality holds if and only if x1 = · · · = xn = [1/(α + n)]y. Proof: See [1284]. Remark: This inequality, which is due to Dragomir and Yang, is a generalization of Hua’s inequality. See Fact 1.17.13 and Fact 9.7.8. Fact 9.7.10. Let x, y ∈ Fn, and assume that x and y are nonzero. Then, . D . D x − y2 − .x2 − y2 . D x y D D D ≤D − min{x2 , y2} x2 y2 D2 .⎫ . ⎧ x − y2 + .x2 − y2 . ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎬ ⎨ max{x2 , y2 } ≤ ⎪ ⎪ ⎪ ⎪ 2x − y2 ⎪ ⎪ ⎭ ⎩ x2 + y2 ⎧ ⎫ 2x − y2 ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎬ max{x2 , y2 } ≤ . . ⎪ 2(x − y2 + .x2 − y2 .) ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ ⎭ x2 + y2 ≤

4x − y2 . x2 + y2

Proof: See Fact 9.7.13 and [1016]. Remark: In the last string of inequalities, the first inequality is the reverse Maligranda inequality, the second and upper third terms constitute the Maligranda inequality, the second and lower third terms constitute the Dunkl-Williams in-

622

CHAPTER 9

equality in an inner product space, the second and upper fourth terms constitute the Massera-Schaffer inequality. Remark: See Fact 1.20.5. Fact 9.7.11. Let x, y ∈ Fn, and let  ·  be a norm on Fn. Then, there exists a unique number α ∈ [1, 2] such that, for all x, y ∈ Fn, at least one of which is nonzero, 2 x + y2 + x − y2 ≤ ≤ 2α. α x2 + y2 Furthermore, if  ·  =  · p , then * α=

2(2−p)/p , 1 ≤ p ≤ 2, 2(p−2)/p , p ≥ 2.

Proof: See [281, p. 258]. Remark: This result is the von Neumann–Jordan inequality. Remark: When p = 2, it follows that α = 2, and this result yields i) of Fact 9.7.4. Fact 9.7.12. Let x, y ∈ Fn, and let  ·  be a norm on Fn. Then, D D  D x y D D ≤ x + y, D + x + y ≤ x + y − min{x, y} 2 − D x y D D D  D x y D D x − y ≤ x + y − min{x, y} 2 − D − D x y D ≤ x + y, D D  D x y D D x + y − max{x, y} 2 − D + D x y D ≤ x + y ≤ x + y, and

D D  D x y D D D ≤ x − y ≤ x + y. x + y − max{x, y} 2 − D − x y D

Proof: See [976]. Fact 9.7.13. Let x, y ∈ Fn, assume that x and y are nonzero, and let  ·  be a norm on Fn. Then, . .  D D D x (x + y) x + y − .x − y. y D 1 D D ≤ 4 (x + y)D + 4 min{x, y} x y D D D D x y D 1 D D ≤ 2 max{x, y}D + x y D   ≤ 12 x + y + max{x, y} − x − y . .  ≤ 1 x + y + .x − y. 2

≤ x + y

623

NORMS

and

. .  D D D x (x + y) x − y − .x − y. y D 1 D D ≤ 4 (x + y)D − 4 min{x, y} x y D D D D x y D 1 D D ≤ 2 max{x, y}D − x y D   ≤ 12 x − y + max{x, y} − x − y . .  ≤ 1 x − y + .x − y. 2

≤ x − y. Furthermore,

. . D D x − y − .x − y. D x y D D D ≤D − min{x, y} x y D . . x − y + .x − y. ≤ max{x, y} ⎫ ⎧ 2x − y ⎪ ⎪ ⎪ ⎪ ⎬ ⎨ max{x, y} . . ≤ 2(x − y + .x − y.) ⎪ ⎪ ⎪ ⎪ ⎭ ⎩ x + y ≤

4x − y . x + y

Proof: This result follows from Fact 9.7.12, [976, 1016] and [1035, p. 516]. Remark: In the last string of inequalities, the first inequality is the reverse Maligranda inequality, the second inequality is the Maligranda inequality, the second and upper fourth terms constitute the Massera-Schaffer inequality, and the second and fifth terms constitute the Dunkl-Williams inequality. See Fact 1.20.2 and Fact 9.7.4 for the case of the Euclidean norm. Remark: Extensions to more than two vectors are given in [816, 1105]. Fact 9.7.14. Let x, y ∈ Fn, and let  ·  be a norm on Fn. Then, x2 + y2 ≤ x + y2 + x − y2 2x2 − 4xy + 2y2 ≤ 2x2 + 4xy + 2y2   ≤ 4 x2 + y2 . Proof: See [544, pp. 9, 10] and [1057, p. 278]. Fact 9.7.15. Let x, y ∈ Fn, let α ∈ [0, 1], and let  ·  be a norm on Fn. Then, x + y ≤ αx + (1 − α)y + (1 − α)x + αy ≤ x + y.

624

CHAPTER 9

Fact 9.7.16. Let x, y ∈ Fn, assume that x and y are nonzero, let  ·  be a norm on Fn, and let p ∈ R. Then, the following statements hold: i) If p ≤ 0, then p p D D Dxp−1 x − yp−1 y D ≤ (2 − p) max{x , y } x − y. max{x, y}

ii) If p ∈ [0, 1], then D D Dxp−1 x − yp−1 y D ≤ (2 − p)

x − y . [max{x, y}]1−p

iii) If p ≥ 1, then D D Dxp−1 x − yp−1 y D ≤ p[max{x, y}]p−1x − y. Proof: See [976]. Fact 9.7.17. Let x, y ∈ Fn, let  ·  be a norm on Fn, assume that x = y, and let p > 0. Then, D D p p D . D . . . .x − y. ≤ . x x − y y . .x − y. ≤ x − y. .xp+1 − yp+1 . Proof: See [1035, p. 516]. Fact 9.7.18. Let x ∈ Fn, and let p, q ∈ [1, ∞] satisfy 1/p + 1/q = 1. Then, + x2 ≤ xp xq . Fact 9.7.19. Let x, y ∈ Rn, assume that x and y are nonnegative, let p ∈ (0, 1], and define 1/p  n   p xp = |x(i) | . i=1

Then,

xp + yp ≤ x + yp .

Remark: The notation is for convenience only since  · p is not a norm for all p ∈ (0, 1). Remark: This result is the reverse Minkowski inequality. Fact 9.7.20. Let x, y ∈ Fn, let  ·  be a norm on Fn, let p and q be real numbers, and assume that 1 ≤ p ≤ q. Then, [ 12 (x +

√ 1 yq q−1

+ x −

√ 1 yq )]1/q q−1

≤ [ 12 (x +

√ 1 yp p−1

+ x −

Proof: See [556, p. 207]. Remark: This result is Bonami’s inequality. See Fact 1.12.16.

√ 1 yp )]1/p. p−1

625

NORMS

Fact 9.7.21. Let x, y ∈ Fn×n. If p ∈ [1, 2], then (xp + yp )p + |xp − yp |p ≤ x + ypp + x − ypp and (x + yp + x − yp )p + |x + yp + x − yp |p ≤ 2p (xpp + ypp ). If p ∈ [2, ∞], then x + ypp + x − ypp ≤ (xp + yσp )p + |xp − yp |p and 2p (xpp + ypp ) ≤ (x + yp + x − yp )p + |x + yp + x − yp |p . Proof: See [120, 932]. Remark: These inequalities are versions of Hanner’s inequality. These vector versions follow from inequalities on Lp by appropriate choice of measure. Remark: Matrix versions are given in Fact 9.9.36. F

n×n

Fact 9.7.22. Let y ∈ Fn, let  ·  be a norm on Fn, let  ·  be the norm on induced by  · , and define 

yD =

max

x∈{z∈Fn: z=1}

|y ∗x|.

Then,  · D is a norm on Fn. Furthermore, y = Hence, for all x ∈ Fn, In addition,

max

x∈{z∈Fn : zD =1}

|y ∗x|.

|x∗y| ≤ xyD. xy ∗  = xyD.

Finally, let p ∈ [1, ∞], and let 1/p + 1/q = 1. Then,  · pD =  · q . Hence, for all x ∈ F , n

and

|x∗y| ≤ xp yq xy ∗ p,p = xp yq .

Proof: See [1261, p. 57]. Remark:  · D is the dual norm of  · . Fact 9.7.23. Let  ·  be a norm on Fn, and define f : Fn → [0, ∞) by f (x) = x. Then, f is convex. Fact 9.7.24. Let x ∈ Rn, and let  ·  be a norm on Rn. Then, xTy > 0 for all y ∈ {z ∈ Rn : z − x < x}.

626

CHAPTER 9

Fact 9.7.25. Let x, y ∈ Rn, assume that x and y are nonzero, assume that x y = 0, and let  ·  be a norm on Rn. Then, x ≤ x + y. T

Proof: If x + y < x, then x + y ∈ Bx(0), and thus y ∈ Bx(−x). By Fact 9.7.24, xTy < 0. Remark: See [222, 927] for related results concerning matrices. Fact 9.7.26. Let x ∈ Fn and y ∈ Fm. Then, σmax (xy ∗ ) = xy ∗ F = x2 y2 and

σmax (xx∗ ) = xx∗ F = x22 .

Remark: See Fact 5.11.16. Fact 9.7.27. Let x ∈ Fn and y ∈ Fm. Then, D D  D  D D D x ⊗ y2 = Dvec x ⊗ yT D2 = Dvec yxT D2 = DyxT D2 = x2 y2 . Fact 9.7.28. Let x ∈ Fn, and let 1 ≤ p, q ≤ ∞. Then, xp = xp,q . Fact 9.7.29. Let x ∈ Fn, and let p, q ∈ [1, ∞), where p ≤ q. Then, xq ≤ xp ≤ n1/p−1/q xq . Proof: See [698], [699, p. 107]. Remark: See Fact 1.17.5 and Fact 9.8.21. Fact 9.7.30. Let A ∈ Fn×n, and assume that A is positive definite. Then,  xA = (x∗Ax)1/2

is a norm on Fn. Fact 9.7.31. Let  ·  and  ·  be norms on Fn, and let α, β > 0. Then, α ·  + β ·  is also a norm on Fn. Furthermore, max{ · ,  ·  } is a norm on Fn. Remark: min{ · ,  ·  } is not necessarily a norm. Fact 9.7.32. Let A ∈ Fn×n, assume that A is nonsingular, and let  ·  be a  norm on Fn. Then, x = Ax is a norm on Fn. Fact 9.7.33. Let x ∈ Fn, and let p ∈ [1, ∞]. Then, xp = xp . n Fact 9.7.34. k Let x1, . . . , xk ∈ F , let α1, . . . , αk be positive numbers, and assume that i=1 αi = 1. Then, k ! |11×n (x1 ◦ · · · ◦ xk )| ≤ xi 1/αi . i=1

Remark: This result is the generalized H¨ older inequality. See [279, p. 128].

627

NORMS

9.8 Facts on Matrix Norms for One Matrix Fact 9.8.1. Let S ⊆ Fm, assume that S is bounded, and let A ∈ Fn×m. Then, AS is bounded. Remark: AS = {Ax: x ∈ S}. Fact 9.8.2. Let A ∈ Fn×n, assume that A is a idempotent, and assume that, for all x ∈ Fn, Ax2 ≤ x2 . Then, A is a projector. Proof: See [550, p. 42]. Fact 9.8.3. Let A, B ∈ Fn×n, and assume that A and B are projectors. Then, the following statements are equivalent: i) A ≤ B. ii) For all x ∈ Fn, Ax2 ≤ Bx2 . iii) R(A) ⊆ R(B). iv) AB = A. v) BA = A. vi) B − A is a projector. Proof: See [550, p. 43] and [1215, p. 24]. Remark: See Fact 3.13.14 and Fact 3.13.17. Fact 9.8.4. Let A ∈ Fn×n, and assume that sprad(A) < 1. Then, there exists a submultiplicative matrix norm  ·  on Fn×n such that A < 1. Furthermore, lim Ak = 0.

k→∞

Fact 9.8.5. Let A ∈ Fn×n, assume that A is nonsingular, and let  ·  be a submultiplicative norm on Fn×n. Then, A−1 ≥ In /A. Fact 9.8.6. Let A ∈ Fn×n, assume that A is nonzero and idempotent, and let  ·  be a submultiplicative norm on Fn×n. Then, A ≥ 1. Fact 9.8.7. Let  ·  be a unitarily invariant norm on Fn×n. Then,  ·  is self-adjoint. 

Fact 9.8.8. Let A ∈ Fn×m, let  ·  be a norm on Fn×m, and define A = A . Then,  ·  is a norm on Fm×n. If, in addition, n = m and  ·  is induced by  ·  , then  ·  is induced by  ·  D . ∗

628

CHAPTER 9

Proof: See [728, p. 309] and Fact 9.8.10. Remark: See Fact 9.7.22 for the definition of the dual norm.  ·  is the adjoint norm of  · . Problem: Generalize this result to nonsquare matrices and norms that are not equi-induced. Fact 9.8.9. Let 1 ≤ p ≤ ∞. Then,  · σp is unitarily invariant. Fact 9.8.10. Let A ∈ Fn×m, and let p, q ∈ [1, ∞] satisfy 1/p + 1/q = 1. Then, A∗ p,p = Aq,q . In particular,

A∗ col = Arow .

Proof: See Fact 9.8.8. Fact 9.8.11. Let A ∈ Fn×m, and let p, q ∈ [1, ∞] satisfy 1/p + 1/q = 1. Then, D D D 0 A D D D D A∗ 0 D = max{Ap,p , Aq,q }. p,p In particular, D D 0 D D A∗

A 0

D D 0 =D D A∗ col

D D D D

A 0

D D D D

= max{Acol , Arow }.

row

Fact 9.8.12. Let A ∈ Fn×m. Then, the following inequalities hold: √ i) AF ≤ A1 ≤ mnAF . ii) A∞ ≤ A1 ≤ mnA∞ . iii) Acol ≤ A1 ≤ mAcol . iv) Arow ≤ A1 ≤ nArow . √ v) σmax (A) ≤ A1 ≤ mn rank A σmax (A). √ vi) A∞ ≤ AF ≤ mnA∞ . √ vii) √1n Acol ≤ AF ≤ mAcol . √ viii) √1m Arow ≤ AF ≤ nArow . √ ix) σmax (A) ≤ AF ≤ rank A σmax (A). x) xi) xii)

1 n Acol

≤ A∞ ≤ Acol .

1 m Arow

≤ A∞ ≤ Arow .

√ 1 σmax (A) mn

≤ Acol ≤ nArow . √ ≤ Acol ≤ nσmax (A). xiv) √ xv) √1n σmax (A) ≤ Arow ≤ mσmax (A).

xiii)

1 m Arow

≤ A∞ ≤ σmax (A).

√1 σmax (A) m

629

NORMS

Proof: See [728, p. 314] and [1538]. Remark: See [699, p. 115] for matrices that attain these bounds. Fact 9.8.13. Let A ∈ Fn×m, and assume that A is normal. Then, 1 √ σmax (A) ≤ A∞ ≤ sprad(A) = σmax (A). mn Proof: Use Fact 5.14.14 and statement xii) of Fact 9.8.12. Fact 9.8.14. Let A ∈ Rn×n, assume that A is symmetric, and assume that every diagonal entry of A is zero. Then, the following conditions are equivalent: i) For all x ∈ Rn such that 11×n x = 0, it follows that xTAx ≤ 0. ii) There exists a positive integer k and vectors x1, . . . , xn ∈ Rk such that, for all i, j ∈ {1, . . . , n}, A(i,j) = xi − xj 22 . Proof: See [20]. Remark: This result is due to Schoenberg. Remark: A is a Euclidean distance matrix. Fact 9.8.15. Let A ∈ Fn×n. Then, AA F ≤ n(2−n)/2 An−1 F . Proof: See [1125, pp. 151, 165]. Fact 9.8.16. Let A ∈ Fn×n, let  ·  and  ·  be norms on Fn, and define the induced norms  A = max Ax m x∈{y∈F : y=1}

and

 A =

max

x∈{y∈Fm : y =1}

Ax .

Then, A A = max A∈{X∈Fn×n : X=0} A A∈{X∈Fn×n : X=0} A max

=

x : y=0} x

max n

x∈{y∈F

x . : y=0} x

max n

x∈{y∈F

Proof: See [728, p. 303]. Remark: This symmetry property is evident in Fact 9.8.12. Fact 9.8.17. Let A ∈ Fn×m, let q, r ∈ [1, ∞], assume that 1 ≤ q ≤ r, define 

p=

1 q

1 , − 1r

and assume that p ≥ 2. Then, Ap ≤ Aq,r .

630

CHAPTER 9

In particular,

A∞ ≤ A∞,∞ .

Proof: See [489]. Remark: This result is due to Hardy and Littlewood. Fact 9.8.18. Let A ∈ Rn×m. Then, D⎡ ⎤D D row1(A)2 D D D √ D⎢ ⎥D .. D⎣ D ≤ 2A1,∞ , ⎦ . D D D rown(A)2 D 1 D⎡ ⎤D D row1(A)1 D D D √ D⎢ ⎥D .. D⎣ D ≤ 2A1,∞ , ⎦ . D D D rown(A)1 D 2 3/4

A4/3 ≤

√ 2A1,∞ .

Proof: See [556, p. 303]. Remark: The first and third results are due to Littlewood, while the second result is due to Orlicz. Fact 9.8.19. Let A ∈ Fn×n, and assume that A is positive semidefinite. Then, A1,∞ =

max

x∈{z∈Fn : z∞ =1}

x∗Ax.

Remark: This result is due to Tao. See [699, p. 116] and [1166]. Fact 9.8.20. Let A ∈ Fn×n. If p ∈ [1, 2], then AF ≤ Aσp ≤ n1/p−1/2 AF . If p ∈ [2, ∞], then Aσp ≤ AF ≤ n1/2−1/p Aσp . Proof: See [204, p. 174]. Fact 9.8.21. Let A ∈ Fn×n, and let p, q ∈ [1, ∞]. Then, ⎧ ⎨n1/p−1/q Aq,q , p ≤ q, Ap,p ≤ ⎩n1/q−1/p A , q ≤ p. q,q Consequently, n1/p−1Acol ≤ Ap,p ≤ n1−1/p Acol , n−|1/p−1/2| σmax (A) ≤ Ap,p ≤ n|1/p−1/2| σmax (A), n−1/pAcol ≤ Ap,p ≤ n1/p Arow . Proof: See [698] and [699, p. 112].

631

NORMS

Remark: See Fact 9.7.29. Problem: Extend these inequalities to nonsquare matrices.  Fact 9.8.22. Let A ∈ Fn×m, p, q ∈ [1, ∞], and α ∈ [0, 1], and let r = pq/[(1 − α)p + αq]. Then, 1−α Ar,r ≤ Aα p,p Aq,q .

Proof: See [698] or [699, p. 113]. Fact 9.8.23. Let A ∈ Fn×m, and let p ∈ [1, ∞]. Then, 1/p

Ap,p ≤ Acol A1−1/p row . In particular,

σmax (A) ≤



Acol Arow .

Proof: Set α = 1/p, p = 1, and q = ∞ in Fact 9.8.22. See [699, p. 113]. To prove the special case p = 2 directly, note that λmax(A∗A) ≤ A∗Acol ≤ A∗ col Acol = Arow Acol . Fact 9.8.24. Let A ∈ Fn×m. Then, A2,1 A∞,2

≤ σmax (A).

Proof: This result follows from Proposition 9.1.5. Fact 9.8.25. Let A ∈ Fn×m, and let p ∈ [1, 2]. Then, 2/p−1 2−2/p σmax (A).

Ap,p ≤ Acol

Proof: Let α = 2/p − 1, p = 1, and q = 2 in Fact 9.8.22. See [699, p. 113]. Fact 9.8.26. Let A ∈ Fn×n, and let p ∈ [1, ∞]. Then, √ Ap,p ≤ |A|p,p ≤ nmin{1/p,1−1/p} Ap,p ≤ nAp,p . Remark: See [699, p. 117]. Fact 9.8.27. Let A ∈ Fn×m, and let p, q ∈ [1, ∞]. Then, Aq,p = Aq,p . Fact 9.8.28. Let A ∈ Fn×m, and let p, q ∈ [1, ∞]. Then, A∗ q,p = Ap/(p−1),q/(q−1) . Fact 9.8.29. Let A ∈ Fn×m, and let p, q ∈ [1, ∞]. Then, * Ap/(p−1) , 1/p + 1/q ≤ 1, Aq,p ≤ Aq , 1/p + 1/q ≥ 1.

632 F

CHAPTER 9

Fact 9.8.30. Let A ∈ Fn×n, and let  ·  be a unitarily invariant norm on . Then, A = A.

n×n

Fact 9.8.31. Let A, S ∈ Fn×n, assume that S is nonsingular, and let  ·  be a unitarily invariant norm on Fn×n. Then, A ≤ 12 SAS −1 + S −∗AS ∗ . Proof: See [64, 250]. Fact 9.8.32. Let A ∈ Fn×n, assume that A is positive semidefinite, and let  ·  be a submultiplicative norm on Fn×n. Then, A1/2 ≤ A1/2 . In particular,



1/2 (A) = σmax (A1/2 ). σmax

n×n , A12 ∈ Fn×m, and A22 ∈ Fm×m, assume that Fact  9.8.33. Let A11 ∈ F ∈ F(n+m)×(n+m) is positive semidefinite, let  ·  and  ·  be unitarily

A11 A12 ∗ A12 A22

invariant norms on Fn×n and Fm×m, respectively, and let p > 0. Then, 2

p Ap22  . A12 p  ≤ A11

Proof: See [732]. Fact 9.8.34. Let A ∈ Fn×n, let  ·  be a norm on Fn, let  · D denote the dual norm on Fn, and let  ·  denote the norm induced by  ·  on Fn×n. Then, A = maxn x,y∈F x,y=0

Re y ∗Ax . yDx

Proof: See [699, p. 115]. Remark: See Fact 9.7.22 for the definition of the dual norm. Problem: Generalize this result to obtain Fact 9.8.35 as a special case. Fact 9.8.35. Let A ∈ Fn×m, and let p, q ∈ [1, ∞]. Then, Aq,p =

max m

x∈F ,y∈Fn x,y=0

|y ∗Ax| . yq/(q−1) xp

Fact 9.8.36. Let A ∈ Fn×m, and let p, q ∈ [1, ∞] satisfy 1/p + 1/q = 1. Then, Ap,p =

max m

x∈F ,y∈Fn x,y=0

|y ∗Ax| = yq xp

max m

x∈F ,y∈Fn x,y=0

|y ∗Ax| . yp/(p−1) xp

Remark: See Fact 9.13.2 for the case p = 2. Fact 9.8.37. Let A ∈ Fn×n, and assume that A is positive definite. Then, √ x∗Ax 2 αβ = min α+β x∈Fn \{0} Ax2 x2

633

NORMS

and min σmax (αA − I) = α≥0



α−β , α+β



where α = λmax(A) and β = λmin(A). Proof: See [624]. Remark: These quantities are antieigenvalues. Fact 9.8.38. Let A ∈ Fn×n, and define nrad(A) = max {|x∗Ax|: x ∈ Cn and x∗x ≤ 1}. 

Then, the following statements hold: i) nrad(A) = max{|z|: z ∈ Θ(A)}.

  ii) sprad(A) ≤ nrad(A) ≤ nrad(|A|) = 12 sprad |A| + |A|T .   1/2  iii) 12 σmax (A) ≤ nrad(A) ≤ 12 σmax (A) + σmax A2 ≤ σmax (A). iv) If A2 = 0, then nrad(A) = σmax (A).   v) If nrad(A) = σmax (A), then σmax A2 = σ2max(A). vi) If A is normal, then nrad(A) = sprad(A).   vii) nrad Ak ≤ [nrad(A)]k for all k ∈ N. viii) nrad(·) is a weakly unitarily invariant norm on Fn×n. ix) nrad(·) is not a submultiplicative norm on Fn×n. 

x)  ·  = αnrad(·) is a submultiplicative norm on Fn×n if and only if α ≥ 4. xi) nrad(AB) ≤ nrad(A)nrad(B) for all A, B ∈ Fn×n such that A and B are normal. xii) nrad(A ◦ B) ≤ αnrad(A)nrad(B) for all A, B ∈ Fn×n if and only if α ≥ 2. xiii) nrad(A ⊕ B) = max{nrad(A), nrad(B)} for all A ∈ Fn×n and B ∈ Fm×m. Proof: See [728, p. 331] and [730, pp. 43, 44]. For iii), see [847]. Remark: nrad(A) is the numerical radius of A. Θ(A) is the numerical range. See Fact 8.14.7. Remark: nrad(·) is not submultiplicative. The example A = [ 00 10 ], B = [ 02 20 ], where B is normal, nrad(A) = 1/2, nrad(B) = 2, and nrad(AB) = 2, shows that xi) is not valid if only one of the matrices A and B is normal, which corrects [730, pp. 43, 73]. Remark: vii) is the power inequality.  Fact 9.8.39. Let A ∈ Fn×m, let γ > σmax (A), and define β = σmax (A)/γ. Then,   AF ≤ − [γ 2/(2π)]log det(I − γ −2A∗A) ≤ β −1 −log(1 − β 2 )AF .

Proof: See [258].

634

CHAPTER 9

Fact 9.8.40. Let  ·  be a unitarily invariant norm on Fn×n. Then, A = 1 for all A ∈ Fn×n such that rank A = 1 if and only if E1,1  = 1. Proof: A = E1,1 σmax (A). Remark: These equivalent normalizations are used in [1261, p. 74] and [201], respectively. Fact 9.8.41. Let  ·  be a unitarily invariant norm on Fn×n. Then, the following statements are equivalent: i) σmax (A) ≤ A for all A ∈ Fn×n. ii)  ·  is submultiplicative. D D iii) DA2 D ≤ A2 for all A ∈ Fn×n. D D iv) DAk D ≤ Ak for all k ≥ 1 and A ∈ Fn×n. v) A ◦ B ≤ AB for all A, B ∈ Fn×n. vi) sprad(A) ≤ A for all A ∈ Fn×n. vii) Ax2 ≤ Ax2 for all A ∈ Fn×n and x ∈ Fn. viii) A∞ ≤ A for all A ∈ Fn×n. ix) E1,1  ≥ 1. x) σmax (A) ≤ A for all A ∈ Fn×n such that rank A = 1. Proof: The equivalence of i)–vii) is given in [729] and [730, p. 211]. Since A = E1,1 σmax (A) for all A ∈ Fn×n such that rank A = 1, it follows that vii) and viii) are equivalent. To prove that ix) =⇒ x), let A ∈ Fn×n satisfy rank A = 1. Then,  A = σmax (A)E1,1  ≥ σmax (A). To show x) =⇒ ii), define  ·  = E1,1 −1  · . Since E1,1  = 1, it follows from [201, p. 94] that  ·  is submultiplicative. Since E1,1 −1 ≤ 1, it follows that  ·  is also submultiplicative. Alternatively, A = σmax (A) for all A ∈ Fn×n having rank 1. Then, Corollary 3.10 of [1261, p. 80] implies that  ·  , and thus  · , is submultiplicative. Fact 9.8.42. Let Φ: Fn → [0, ∞) satisfy the following conditions: i) If x = 0, then Φ(x) > 0. ii) Φ(αx) = |α|Φ(x) for all α ∈ R. iii) Φ(x + y) ≤ Φ(x) + Φ(y) for all x, y ∈ Fn. iv) If A ∈ Fn×n is a permutation matrix, then Φ(Ax) = Φ(x) for all x ∈ Fn. v) Φ(|x|) = Φ(x) for all x ∈ Fn. Furthermore, for A ∈ Fn×m, where n ≤ m, define 

A = Φ[σ1(A), . . . , σn(A)]. Then,  ·  is a unitarily invariant norm on Fn×m. Conversely, if  ·  is a unitarily

635

NORMS

invariant norm on Fn×m, where n ≤ m, then Φ: Fn → [0, ∞) defined by D⎡ ⎤D D x(1) · · · 0 0n×(m−n) D D D ⎥D  D⎢ . .. .. .. Φ(x) = D⎣ .. ⎦D . . . D D D 0 · · · x(n) 0n×(m−n) D satisfies i)–v). Proof: See [1261, pp. 75, 76]. Remark: Φ is a symmetric gauge function. This result is due to von Neumann. See Fact 2.21.13. Fact 9.8.43. Let  ·  and  ·  denote norms on Fm and Fn, respectively, and ˆ Fn×m → R by define : Ax  ˆ , = min

(A) x∈Fm \{0} x or, equivalently,

 ˆ

(A) =

min

Ax .

x∈{y∈Fm : y=1}

Then, for A ∈ Fn×m, the following statements hold: ˆ i) (A) ≥ 0. ˆ ii) (A) > 0 if and only if rank A = m. ˆ iii) (A) = (A) if and only if either A = 0 or rank A = m. Proof: See [892, pp. 369, 370]. Remark: ˆ is a weaker version of . Fact 9.8.44. Let  ·  and  ·  denote norms on Fm and Fn, respectively, let ˆ Fn×m → R by  ·  denote the norm induced by  ·  and  · , and define :

 ˆ =

(A)

If A is nonzero, then

Ax . )\{0} x

min ∗

x∈R(A

1 ˆ ≤ (A). A+ 

If, in addition, rank A = m, then 1 ˆ = (A) = (A). A+  Proof: See [1371]. Fact 9.8.45. Let A ∈ Fn×n, let  ·  be a normalized, submultiplicative norm on F , and assume that I − A < 1. Then, A is nonsingular. n×n

Remark: See Fact 9.9.56.

636

CHAPTER 9

Fact 9.8.46. Let · be a normalized, submultiplicative norm on Fn×n. Then, ·  is equi-induced if and only if A ≤ A for all A ∈ Fn×n and for all normalized submultiplicative norms  ·  on Fn×n. Proof: See [1265]. Remark: As shown in [316, 391], not every normalized submultiplicative norm on Fn×n is equi-induced or induced.

9.9 Facts on Matrix Norms for Two or More Matrices 

Fact 9.9.1.  ·  ∞ = n · ∞ is submultiplicative on Fn×n. Remark: It is not necessarily true that AB∞ ≤ A∞ B∞ . For example, let A = B = [ 11 11 ]. Fact 9.9.2. Let A ∈ Fn×m and B ∈ Fm×l. Then, AB∞ ≤ mA∞ B∞ . Furthermore, if A = 1n×m and B = 1m×l, then AB∞ = mA∞ B∞ . Fact 9.9.3. Let A, B ∈ Fn×n, and let  ·  be a submultiplicative norm on F . Then, AB ≤ AB. Hence, if A ≤ 1 and B ≤ 1, then AB ≤ 1. Finally, if either A < 1 or B < 1, then AB < 1. n×n

Remark: sprad(A) < 1 and sprad(B) < 1 do not imply that sprad(AB) < 1. Let A = BT = [ 00 20 ]. Fact 9.9.4. Let  ·  be a norm on Fm×m, and let   AB m×m : A, B ∈ F , A, B = 0 . δ > sup AB 

Then,  ·  = δ ·  is a submultiplicative norm on Fm×m. Proof: See [728, p. 323]. Fact 9.9.5. Let A, B ∈ Fn×n, let  ·  be a unitarily invariant norm on Fn×n, assume that A and B are Hermitian, and assume that A ≤ B. Then, A ≤ B. Proof: See [219]. Fact 9.9.6. Let A, B ∈ Fn×n, let  ·  be a unitarily invariant norm on Fn×n, and assume that AB is normal. Then, AB ≤ BA. Proof: See [201, p. 253].

637

NORMS

Fact 9.9.7. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite and nonzero, and let  ·  be a submultiplicative unitarily invariant norm on Fn×n. Then, AB A + B ≤ AB A + B and

A + B A ◦ B ≤ . AB A + B

Proof: See [693]. Remark: See Fact 9.8.41.

F

Fact 9.9.8. Let A, B ∈ Fn×n, and let  ·  be a submultiplicative norm on  . Then,  ·  = 2 ·  is a submultiplicative norm on Fn×n and satisfies

n×n

[A, B] ≤ A B . Fact 9.9.9. Let A ∈ Fn×n. Then, the following statements are equivalent: i) There exist projectors Q, P ∈ Rn×n such that A = [P, Q]. ii) σmax (A) ≤ 1/2, A and −A are unitarily similar, and A is skew Hermitian. Proof: See [929]. Remark: Extensions are discussed in [1009]. Remark: See Fact 3.12.16 for the case of idempotent matrices. Remark: In the case F = R, the condition that A is skew symmetric implies that A and −A are orthogonally similar. See Fact 5.9.12. F

Fact 9.9.10. Let A, B ∈ Fn×n, and let  ·  be a unitarily invariant norm on . Then, AB ≤ σmax (A)B

n×n

and

AB ≤ Aσmax (B).

Consequently, if C ∈ F

n×n

, then

ABC ≤ σmax (A)Bσmax (C). Proof: See [844]. F

Fact 9.9.11. Let A, B ∈ Fn×m, and let  ·  be a unitarily invariant norm on . If p > 0, then

m×m

A∗Bp 2 ≤ (A∗A)p (B ∗B)p . In particular, and Furthermore,

(A∗BB ∗A)1/4 2 ≤ AB A∗B = A∗B2 ≤ A∗AB ∗B. tr A∗B ≤ AF BF

638

CHAPTER 9

2  tr (A∗BB ∗A)1/4 ≤ (tr A)(tr B).

and

Proof: See [732] and use Fact 9.8.30. Problem: Noting Fact 9.12.1 and Fact 9.12.2, compare the lower bounds for AF BF given by ⎫ tr A∗B ⎪ ⎬ |tr A∗B| ≤ AF BF .  √ ⎪ ⎭ ∗ 2 ∗ ∗ |tr (A B) | ≤ tr AA BB Fact 9.9.12. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, 1/2  1/2 (2AFBF ) ≤ A2F + B2F 1/2  =  A2 + B 2 F ≤ A + BF √  1/2 ≤ 2 A2F + B2F . Fact 9.9.13. Let A, B ∈ Fn×m. Then, + A + BF = A2F + B2F + 2 tr AB ∗ ≤ AF + BF . In particular, A − BF =

+ A2F + B2F − 2 tr AB ∗.

If, in addition, A is Hermitian and B is skew Hermitian, then tr AB ∗ = 0, and thus A + B2F = A − B2F = A2F + B2F . Remark: The second equality is a matrix version of the cosine law given by ix) of Fact 9.7.4. F

Fact 9.9.14. Let A, B ∈ Fn×n, and let  ·  be a unitarily invariant norm on . Then, AB ≤ 14 (A + B ∗ )2 .

n×n

Proof: See [216]. Fact 9.9.15. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let  ·  be a unitarily invariant norm on Fn×n. Then, AB ≤ 14 (A + B)2 . Proof: See [216] or [1521, p. 77]. Problem: Noting Fact 9.9.12, compare the lower bounds for A + BF given by 1/2  1/2 F ≤ A + BF (2AFBF ) ≤  A2 + B 2 and

1/2

1/2

2ABF ≤ (A + B)2 F ≤ A + BF .

639

NORMS

Fact 9.9.16. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, let  ·  be a unitarily invariant norm on Fn×n, and let p ∈ [0, ∞). If p ∈ [0, 1], then ApB p  ≤ ABp . If p ∈ [1, ∞), then

ABp ≤ ApB p .

Proof: See [207, 537]. Remark: See Fact 8.19.27. Fact 9.9.17. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let  ·  be a unitarily invariant norm on Fn×n. If p ∈ [0, 1], then B pApB p  ≤ (BAB)p . Furthermore, if p ≥ 1, then (BAB)p  ≤ B pApB p . Proof: See [72] and [201, p. 258]. Remark: Extensions and a reverse inequality are given in Fact 8.10.50. Remark: See Fact 8.12.22 and Fact 8.19.27. Fact 9.9.18. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let  ·  be a unitarily invariant norm on Fn×n. Then, A1/2B1/2  ≤ 12A + B. Hence,

AB ≤ 12A2 + B 2 ,

and thus Consequently,

(A + B)2  ≤ 2A2 + B 2 . AB ≤ 14 (A + B)2  ≤ 12A2 + B 2 .

Proof: Let p = 1/2 and X = I in Fact 9.9.49. The last inequality follows from Fact 9.9.15. Remark: See Fact 8.19.13. Fact 9.9.19. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let either p = 1 or p ∈ [2, ∞]. Then, AB1/2 σp ≤ 12A + Bσp . Proof: See [93, 216]. Remark: The inequality holds for all Q-norms. See [201]. Remark: See Fact 8.19.13.

640

CHAPTER 9

Fact 9.9.20. Let A ∈ Fn×m, B ∈ Fm×l, and p, q, q , r ∈ [1, ∞], and assume that 1/q + 1/q = 1. Then, ABp ≤ εpq (n)εpr (l)εqr (m)Aq Br , *

where 

εpq (n) =

1,

p ≥ q,

n1/p−1/q , q ≥ p.

Furthermore, there exist matrices A ∈ Fn×m and B ∈ Fm×l such that equality holds. Proof: See [578]. Remark: Related results are given in [488, 489, 578, 579, 580, 852, 1345]. Fact 9.9.21. Let A, B ∈ Cn×m. Then, there exist unitary matrices S1, S2 ∈ Cm×m such that A + B ≤ S1AS1∗ + S2 BS2∗. Remark: This result is a matrix version of the triangle inequality. See [49, 1302]. Fact 9.9.22. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let p ∈ [1, ∞]. Then, A − B2σ2p ≤ A2 − B 2 σp . Proof: See [837]. Remark: The case p = 1 is due to Powers and Stormer. Fact 9.9.23. Let A, B ∈ Fn×n, and let p ∈ [1, ∞]. Then, A − B2σp ≤ A + Bσ2p A − Bσ2p . Proof: See [851]. Fact 9.9.24. Let A, B ∈ Fn×n. Then, A − B2σ1 ≤ 2A + Bσ1A − Bσ1 . Proof: See [851]. Remark: This result is due to Borchers and Kosaki. See [851]. Fact 9.9.25. Let A, B ∈ Fn×n. Then, A − BF ≤ and

√ 2A − BF

A − B2F + A∗  − B ∗ 2F ≤ 2A − B2F .

If, in addition, A and B are normal, then A − BF ≤ A − BF . Proof: See [49, 73, 836, 851] and [701, pp. 217, 218].

641

NORMS

Fact 9.9.26. Let A, B ∈ Rn×n. Then, √ AB − BAF ≤ 2AF BF . Proof: See [246, 1419]. Remark: The constant

√ 2 holds for all n.

Remark: Extensions to complex matrices are given in [247]. Fact 9.9.27. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, AB − BA2F + (A − B)2 2F ≤ A2 − B 2 2F . Proof: See [844]. Fact 9.9.28. Let A, B ∈ Fn×n, let p be a positive number, and assume that either A is normal and p ∈ [2, ∞], or A is Hermitian and p ≥ 1. Then, AB − BAσp ≤ AB − BAσp . Proof: See [1]. F

Fact 9.9.29. Let · be a unitarily invariant norm on Fn×n, and let A, X, B ∈ . Then, AX − XB ≤ [σmax (A) + σmax (B)]X.

n×n

In particular,

σmax (AX − XA) ≤ 2σmax (A)σmax (X).

Now, assume in addition that A and B are positive semidefinite. Then, AX − XB ≤ max{σmax (A), σmax (B)}X. In particular,

σmax (AX − XA) ≤ σmax (A)σmax (X).

Finally, assume that A and X are positive semidefinite. Then, D D D X 0 D 1 D. D AX − XA ≤ 2 σmax (A) D 0 X D In particular,

σmax (AX − XA) ≤ 12 σmax (A)σmax (X).

Proof: See [218]. Remark:

first inequality is sharp since equality holds for A = B = 0 1The X = −1 0 .

1

0 0 −1

and

Remark:  ·  can be extended to F2n×2n by considering the n largest singular values of matrices in F2n×2n. For details, see [201, pp. 90, 98]. Fact 9.9.30. Let  ·  be a unitarily invariant norm on Fn×n, let A, X ∈ Fn×n, and assume that A is Hermitian. Then, AX − XA ≤ [λmax (A) − λmin (A)]X. Proof: See [218].

642

CHAPTER 9

Remark: λmax (A) − λmin (A) is the spread of A. See Fact 8.15.32 and Fact 9.9.31. Fact 9.9.31. Let  ·  be a unitarily invariant norm on Fn×n, let A, X ∈ Fn×n, assume that A is normal, let spec(A) = {λ1, . . . , λr }, and define 

spd(A) = max{|λi (A) − λj (A)| : i, j ∈ {1, . . . , r}}. Then,

AX − XA ≤

√ 2 spd(A)X.

Furthermore, let p ∈ [1, ∞]. Then, AX − XAσp ≤ 2|2−p|/(2p) spd(A)Xσp . In particular,

AX − XAF ≤ spd(A)XF

and

σmax (AX − XA) ≤

√ 2 spd(A)σmax (X).

Proof: See [218]. Remark: spd(A) is the spread of A. See Fact 8.15.32 and Fact 9.9.30. Fact 9.9.32. Let A, B ∈ Fn×n. Then,   σmax (A) + σmax (B) 2 σmax (A − B). σmax (A − B) ≤ π 2 + log σmax (A − B) Remark: This result is due to Kato. See [851]. Fact 9.9.33. Let A ∈ Fn×m and B ∈ Fm×l, and let r = 1 or r = 2. Then, ABσr = Aσ2r Bσ2r if and only if there exists α ≥ 0 such that AA∗ = αB ∗B. Furthermore, AB∞ = A∞ B∞ if and only if AA∗ and B ∗B have a common eigenvector associated with λ1(AA∗ ) and λ1(B ∗B). Proof: See [1476]. Fact 9.9.34. Let A, B ∈ Fn×n. If p ∈ (0, 2], then 2p−1 (Apσp + Bpσp ) ≤ A + Bpσp + A − Bpσp ≤ 2(Apσp + Bpσp ). If p ∈ [2, ∞), then 2(Apσp + Bpσp ) ≤ A + Bpσp + A − Bpσp ≤ 2p−1 (Apσp + Bpσp ). If p ∈ (1, 2] and 1/p + 1/q = 1, then A + Bqσp + A − Bqσp ≤ 2(Apσp + Bpσp )q/p. If p ∈ [2, ∞) and 1/p + 1/q = 1, then 2(Apσp + Bpσp )q/p ≤ A + Bqσp + A − Bqσp . Proof: See [714].

643

NORMS

Remark: These inequalities are versions of the Clarkson inequalities. See Fact 1.20.2. Remark: See [714] for extensions to unitarily invariant norms. See [217] for additional extensions. Fact 9.9.35. Let A, B ∈ Cn×m. If p ∈ [1, 2], then [A2 + (p − 1)B2 ]1/2 ≤ [ 12 (A + Bp + A − Bp )]1/p. If p ∈ [2, ∞], then [ 12 (A + Bp + A − Bp )]1/p ≤ [A2 + (p − 1)B2 ]1/2. Proof: See [120, 168]. Remark: This result is Beckner’s two-point inequality or optimal 2-uniform convexity. Fact 9.9.36. Let A, B ∈ Fn×n. If either p ∈ [1, 4/3] or both p ∈ (4/3, 2] and A + B and A − B are positive semidefinite, then (Aσp + Bσp )p + |Aσp − Bσp |p ≤ A + Bpσp + A − Bpσp . Furthermore, if either p ∈ [4, ∞] or both p ∈ [2, 4) and A and B are positive semidefinite, then A + Bpσp + A − Bpσp ≤ (Aσp + Bσp )p + |Aσp − Bσp |p. Proof: See [120, 834]. Remark: These inequalities are versions of Hanner’s inequality. Remark: Vector versions are given in Fact 9.7.21. Fact 9.9.37. Let A, B ∈ Cn×n, and assume that A and B are Hermitian. If p ∈ [1, 2], then 1/2 1/2   21/2−1/p  A2 + B 2 p ≤ A + jBσp ≤  A2 + B 2 p and

    21−2/p A2σp + B2σp ≤ A + jB2σp ≤ 22/p−1 A2σp + B2σp .

Furthermore, if p ∈ [2, ∞), then 1/2 1/2   p ≤ A + jBσp ≤ 21/2−1/p  A2 + B 2 p  A2 + B 2 and

    22/p−1 A2σp + B2σp ≤ A + jB2σp ≤ 21−2/p A2σp + B2σp .

Proof: See [215]. Fact 9.9.38. Let A, B ∈ Cn×n, and assume that A and B are Hermitian. If p ∈ [1, 2], then 21−2/p (Apσp + Bpσp ) ≤ A + jBpσp . If p ∈ [2, ∞], then A + jBpσp ≤ 21−2/p (Apσp + Bpσp ).

644

CHAPTER 9

In particular, 1/2 2  F . A + jB2F = A2F + B2F =  A2 + B 2 Proof: See [215, 223]. Fact 9.9.39. Let A, B ∈ Cn×n, and assume that A is positive semidefinite and B is Hermitian. If p ∈ [1, 2], then A2σp + 21−2/p B2σp ≤ A + jB2σp . If p ∈ [2, ∞], then In particular,

A + jB2σp ≤ A2σp + 21−2/p B2σp . A2σ1 + 12 B2σ1 ≤ A + jB2σ1 , A + jB2F = A2F + B2F ,

and

2 2 2 (A + jB) ≤ σmax (A) + 2σmax (B). σmax

In fact,

A2σ1 + B2σ1 ≤ A + jB2σ1.

Proof: See [223]. Fact 9.9.40. Let A, B ∈ Cn×n, and assume that A and B are positive semidefinite. If p ∈ [1, 2], then A2σp + B2σp ≤ A + jB2σp . If p ∈ [2, ∞], then

A + jB2σp ≤ A2σp + B2σp .

Hence,

A2σ2 + B2σ2 = A + jB2σ2 .

In particular, (tr A)2 + B)2 ≤ (tr A + jB)2, 2 2 2 σmax (A + jB) ≤ σmax (A) + σmax (A),

A + jB2F = A2F + B2F . Proof: See [223]. Remark: See Fact 8.19.7. Fact 9.9.41. Let A ∈ Fn×n, let B ∈ Fn×n, assume that B is Hermitian, and let  ·  be a unitarily invariant norm on Fn×n. Then, A − 12 (A + A∗ ) ≤ A − B. In particular,

A − 12 (A + A∗ )F ≤ A − BF

645

NORMS



σmax A − 12 (A + A∗ ) ≤ σmax (A − B).

and

Proof: See [201, p. 275] and [1125, p. 150]. Fact 9.9.42. Let A, M, S, B ∈ Fn×n, assume that A = MS, M is positive semidefinite, and S and B are unitary, and let  ·  be a unitarily invariant norm on Fn×n. Then, A − S ≤ A − B. In particular,

A − SF ≤ A − BF .

Proof: See [201, p. 276] and [1125, p. 150]. Remark: A = MS is the polar decomposition of A. See Corollary 5.6.4. Fact 9.9.43. Let A, B ∈ Fn×n, assume that A and B are Hermitian, let  ·  be a unitarily invariant norm on Fn×n, and let k ∈ N. Then, (A − B)2k+1  ≤ 22kA2k+1 − B 2k+1. Proof: See [201, p. 294] or [781]. F

Fact 9.9.44. Let A, B ∈ Fn×n, and let  ·  be a unitarily invariant norm on . Then,  A − B ≤ 2A + BA − B.

n×n

Proof: See [49]. Remark: This result is due to Kosaki and Bhatia. Fact 9.9.45. Let A, B ∈ Fn×n, and let p ≥ 1. Then, : ;+ A − Bσp ≤ max 21/p−1/2, 1 A + Bσp A − Bσp . Proof: See [49]. Remark: This result is due to Kittaneh, Kosaki, and Bhatia. Fact 9.9.46. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let  ·  be a unitarily invariant norm on F2n×2n. Then, D D D D D D D D A + B 0 D D A 0 D D A1/2B 1/2 0 D+D D≤D D. D 1/2 1/2 D D D D D D 0 B 0 0 0 A B In particular, σmax (A + B) ≤ max{σmax (A), σmax (B)} + σmax (A1/2B 1/2 ) and, for all p ∈ [1, ∞),

1/p  + 21/pA1/2B 1/2 σp . A + Bσp ≤ Apσp + Bpσp

Proof: See [842, 845, 849]. Remark: See Fact 9.14.15 for a tighter upper bound for σmax (A + B).

646

CHAPTER 9

Fact 9.9.47. Let A, X, B ∈ Fn×n, and let  ·  be a unitarily invariant norm on F . Then, A∗XB ≤ 12 AA∗X + XBB ∗ . n×n

In particular,

A∗B ≤ 12 AA∗ + BB ∗ .

Proof: See [64, 206, 213, 539, 839]. Remark: The first result is McIntosh’s inequality. Remark: See Fact 9.14.23. Fact 9.9.48. Let A, X, B ∈ Fn×n, assume that X is positive semidefinite, and let  ·  be a unitarily invariant norm on Fn×n. Then, A∗XB + B ∗XA ≤ A∗XA + B ∗XB. In particular,

A∗B + B ∗A ≤ A∗A + B ∗B.

Proof: See [843]. Remark: See [843] for extensions to the case in which X is not necessarily positive semidefinite. Fact 9.9.49. Let A, X, B ∈ Fn×n, assume that A and B are positive semidefinite, let p ∈ [0, 1], and let  ·  be a unitarily invariant norm on Fn×n. Then, ApXB 1−p + A1−pXB p  ≤ AX + XB and

ApXB 1−p − A1−pXB p  ≤ |2p − 1|AX − XB.

Proof: See [64, 207, 220, 523]. Remark: These results are the Heinz inequalities. Fact 9.9.50. Let A, B ∈ Fn×n, assume that A is nonsingular and B is Hermitian, and let  ·  be a unitarily invariant norm on Fn×n. Then, B ≤ 12 ABA−1 + A−1BA. Proof: See [355, 530]. Fact 9.9.51. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let  ·  be a unitarily invariant norm on Fn×n. If r ∈ [0, 1], then Ar − B r  ≤ A − Br . Furthermore, if r ∈ [1, ∞), then A − Br  ≤ Ar − B r . In particular,

(A − B)2  ≤ A2 − B 2 .

Proof: See [201, pp. 293, 294] and [844].

647

NORMS

Fact 9.9.52. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, let  ·  be a unitarily invariant norm on Fn×n, and let z ∈ F. Then, A − |z|B ≤ A + zB ≤ A + |z|B. In particular,

A − B ≤ A + B.

Proof: See [214]. Remark: Extensions to weak log majorization are given in [1519]. Remark: The special case z = 1 is given in [219]. Fact 9.9.53. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let  ·  be a unitarily invariant norm on Fn×n. If r ∈ [0, 1], then (A + B)r  ≤ Ar + B r . Furthermore, if r ∈ [1, ∞), then Ar + B r  ≤ (A + B)r . In particular, if k is a positive integer, then Ak + B k  ≤ (A + B)k . Proof: See [60]. Fact 9.9.54. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let  ·  be a unitarily invariant norm on Fn×n. Then,  log(I + A) − log(I + B) ≤  log(I + A − B) and

 log(I + A + B) ≤  log(I + A) + log(I + B).

Proof: See [60] and [201, p. 293]. Remark: See Fact 11.16.16. Fact 9.9.55. Let A, X, B ∈ Fn×n, assume that A and B are positive definite, and let  ·  be a unitarily invariant norm on Fn×n. Then, (log A)X − X(log B) ≤ A1/2XB −1/2 − A−1/2XB 1/2. Proof: See [220]. Remark: See Fact 11.16.17. Fact 9.9.56. Let A, B ∈ Fn×n, assume that A is nonsingular, let  ·  be a normalized submultiplicative norm on Fn×n, and assume that A − B < 1/A−1. Then, B is nonsingular. Remark: See Fact 9.8.45.

648

CHAPTER 9

Fact 9.9.57. Let A, B ∈ Fn×n, assume that A is nonsingular, let  ·  be a normalized submultiplicative norm on Fn×n, let γ > 0, and assume that A−1  < γ and A − B < 1/γ. Then, B is nonsingular, γ , B −1  ≤ 1 − γB − A and

A−1 − B −1  ≤ γ 2 A − B.

Proof: See [459, p. 148]. Remark: See Fact 9.8.45. Fact 9.9.58. Let A, B ∈ Fn×n, let λ ∈ C, assume that λI − A is nonsingular, let  ·  be a normalized submultiplicative norm on Fn×n, let γ > 0, and assume that (λI − A)−1  < γ and A − B < 1/γ. Then, λI − B is nonsingular, γ , (λI − B)−1  ≤ 1 − γB − A and (λI − A)−1 − (λI − B)−1  ≤

γ 2 A − B . 1 − γA − B

Proof: See [459, pp. 149, 150]. Remark: See Fact 9.9.57. Fact 9.9.59. Let A, B ∈ Fn×n, assume that A and A + B are nonsingular, and let  ·  be a normalized submultiplicative norm on Fn×n. Then, D D DD D D −1 DA − (A + B)−1D ≤ DA−1D D(A + B)−1D B. If, in addition, A−1B < 1, then −1 −1 D −1 D DA + (A + B)−1D ≤ A A B . −1 1 − A B Furthermore, if A−1B < 1 and B < 1/A−1, then D D −1 DA − (A + B)−1D ≤

A−12 B . 1 − A−1B

Fact 9.9.60. Let A ∈ Fn×n, assume that A is nonsingular, let E ∈ Fn×n, and let  ·  be a normalized norm on Fn×n. Then,  −1 (A + E)−1 = A−1 I + EA−1   = A−1 − A−1EA−1 + O E2 . Fact 9.9.61. Let A ∈ Fn×m and B ∈ Fl×k. Then, A ⊗ Bcol = AcolBcol , A ⊗ B∞ = A∞ B∞ , A ⊗ Brow = ArowBrow .

649

NORMS

Furthermore, if p ∈ [1, ∞], then A ⊗ Bp = Ap Bp . F

Fact 9.9.62. Let A, B ∈ Fn×n, and let  ·  be a unitarily invariant norm on . Then, A ◦ B2 ≤ A∗AB ∗B.

n×n

Proof: See [731]. Fact 9.9.63. Let A, B ∈ Fn×n, assume that A and B are normal, and let  ·  be a unitarily invariant norm on Fn×n. Then, A + B ≤ A + B and

A ◦ B ≤ A ◦ B.

Proof: See [93, 849] and [730, p. 213].

9.10 Facts on Matrix Norms for Partitioned Matrices Fact 9.10.1. Let A ∈ Fn×m be the partitioned matrix ⎡ ⎤ A11 A12 · · · A1k ⎢ A21 A22 · · · A2k ⎥ A=⎢ , .. . .. ⎥ ⎣ ... · ·. · . . ⎦ Ak1 Ak2 · · · Akk where Aij ∈ Fni ×nj for all i, j ∈ {1, . . . , k}. Furthermore, define μ(A) ∈ Rk×k by ⎡ ⎤ σmax (A11 ) σmax (A12 ) · · · σmax (A1k ) ⎢ σmax (A21 ) σmax (A22 ) · · · σmax (A2k ) ⎥  ⎢ ⎥. μ(A) = .. .. . .. ⎣ ⎦ · ·. · . . . σmax (Ak1 ) σmax (Ak2 ) · · ·

σmax (Akk )

Finally, let B ∈ Fn×m be partitioned conformally with A. Then, the following statements hold: i) For all α ∈ F, μ(αA) ≤ |α|μ(A). ii) μ(A + B) ≤ μ(A) + μ(B). iii) μ(AB) ≤ μ(A)μ(B). iv) sprad(A) ≤ sprad[μ(A)]. v) σmax (A) ≤ σmax [μ(A)]. Proof: See [410, 1082, 1236]. Remark: μ(A) is a matricial norm. Remark: This result is a norm-compression inequality.

650

CHAPTER 9

Fact 9.10.2. Let A ∈ Fn×m be the partitioned matrix ⎡ ⎤ A11 A12 · · · A1k ⎢ A21 A22 · · · A2k ⎥ , A=⎢ .. . .. ⎥ ⎣ ... · ·. · . . ⎦ Ak1 Ak2 · · · Akk where Aij ∈ Fni ×nj for all i, j ∈ {1, . . . , k}. Then, the following statements hold: i) If p ∈ [1, 2], then k 

Aij 2σp ≤ A2σp ≤ k 4/p−2

i,j=1

k 

Aij 2σp .

i,j=1

ii) If p ∈ [2, ∞], then k 4/p−2

k 

Aij 2σp ≤ A2σp ≤

i,j=1

k 

Aij 2σp .

i,j=1

iii) If p ∈ [1, 2], then k 

Apσp ≤

Aij pσp ≤ k 2−p Apσp .

i,j=1

iv) If p ∈ [2, ∞), then k 2−p Apσp ≤

k 

Aij pσp ≤ Apσp .

i,j=1

v) A2σ2 =

k

2 i,j=1 Aij σ2 .

vi) For all p ∈ [1, ∞),



k 

1/p Aii pσp

≤ Aσp .

i=1

vii) For all i ∈ {1, . . . , k},

σmax (Aii ) ≤ σmax (A).

Proof: See [133, 212]. Fact 9.10.3. Let A, B ∈ Fn×n, ⎡ A ⎢ ⎢ B ⎢  ⎢ A=⎢ B ⎢ ⎢ .. ⎣ . B Then,

and define A ∈ Fkn×kn by ⎤ B B ··· B ⎥ A B ··· B ⎥ ⎥ .. ⎥ . B ⎥. B A ⎥ .. . . .. ⎥ .. . . . . ⎦ B B ··· A

σmax (A) = max{σmax (A + (k − 1)B), σmax (A − B)}.

651

NORMS

Now, let p ∈ [1, ∞). Then, Aσp = (A + (k − 1)Bpσp + (k − 1)A − Bpσp )1/p . Proof: See [133]. Fact 9.10.4. Let A ∈ Fn×n, and define A ∈ Fkn×kn by ⎤ ⎡ A A A ··· A ⎥ ⎢ A ··· A ⎥ ⎢ −A A ⎥ ⎢ .. ⎥  ⎢ . A ⎥. A = ⎢ −A −A A ⎥ ⎢ . ⎥ .. ⎢ .. .. .. . . .. ⎦ ⎣ . . −A −A −A · · · A ?

Then, σmax (A) =

2 σmax (A). 1 − cos(π/k)

Furthermore, define A0 ∈ Fkn×kn by ⎡ 0 A A ··· ⎢ 0 A ··· ⎢ −A ⎢ ..  ⎢ . A0 = ⎢ −A −A 0 ⎢ .. ⎢ .. .. .. . . ⎣ . . −A −A −A · · ·

A

⎤

⎥ A ⎥ ⎥ ⎥ A ⎥. ⎥ .. ⎥ . ⎦ 0

?

Then, σmax (A0 ) =

1 + cos(π/k) σmax (A). 1 − cos(π/k)

Proof: See [133]. Remark: Extensions to Schatten norms are given in [133]. Fact 9.10.5. Let A, B, C, D ∈ Fn×n. Then, 1 2



max{σmax (A + B + C + D), σmax (A − B − C + D)} ≤ σmax

Now, let p ∈ [1, ∞). Then, 1 2 (A

+B+C +

Proof: See [133].

Dpσp

+ A − B − C +

Dpσp )1/p

D D A ≤D D C

B D

A C

B D

D D D D

σp

.

 .

652

CHAPTER 9

Fact 9.10.6. Let A, B, C ∈ Fn×n, define   A B  , A= B∗ C assume that A is positive semidefinite, let p ∈ [1, ∞], and define   Aσp Bσp  A0 = . Bσp Cσp If p ∈ [1, 2], then

A0 σp ≤ Aσp .

Furthermore, if p ∈ [2, ∞], then Aσp ≤ A0 σp . Hence, if p = 2, then

A0 σp = Aσp .

Finally, if A = C, B is Hermitian, and p is an integer, then Apσp = A + Bpσp + A − Bpσp and

A0 pσp = (Aσp + Bσp )p + |Aσp − Bσp |p.

Proof: See [833]. Remark: This result is a norm-compression inequality. Fact 9.10.7. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m, define   A B  , A= B∗ C assume that A is positive semidefinite, and let p ≥ 1. If p ∈ [1, 2], then Apσp ≤ Apσp + (2p − 2)Bpσp + Cpσp . Furthermore, if p ≥ 2, then Apσp + (2p − 2)Bpσp + Cpσp ≤ Apσp . Finally, if p = 2, then Apσp = Apσp + (2p − 2)Bpσp + Cpσp . Proof: See [89]. Fact 9.10.8. Let A ∈ Fn×m be the partitioned matrix   A11 · · · A1k , A= A21 · · · A2k where Aij ∈ Fni ×nj for all i, j ∈ {1, . . . , k}. Then, the following statements are conjectured to hold:

653

NORMS

i) If p ∈ [1, 2], then D D A11 σp D D D A21 σp ii) If p ≥ 2, then Aσp

···

A1k σp

···

A2k σp

D D A11 σp D ≤D D A21 σp

D D D D D

≤ Aσp .

σp

···

A1k σp

···

A2k σp

D D D D D

.

σp

Proof: See [90]. This result is true when all blocks have rank 1 or when p ≥ 4. Remark: This result is a norm-compression inequality.

9.11 Facts on Matrix Norms and Eigenvalues for One Matrix Fact 9.11.1. Let A ∈ Fn×n. Then, |det A| ≤

n !

rowi (A)2

i=1

and |det A| ≤

n !

coli (A)2 .

i=1

Proof: This result follows from Hadamard’s inequality. See Fact 8.18.11. Fact 9.11.2. Let A ∈ Fn×n, and let mspec(A) = {λ1, . . . , λn }ms . Then, Re tr A ≤ |tr A| ≤

n 

|λi | ≤ Aσ1 = tr A =

i=1

n 

σi(A).

i=1

In addition, if A is normal, then Aσ1 =

n 

|λi |.

i=1

Finally, A is positive semidefinite if and only if Aσ1 = tr A. Proof: See Fact 5.14.14 and Fact 9.13.18. Remark: See Fact 5.11.9 and Fact 5.14.14. Problem: Refine the second statement for necessity and sufficiency. See [763].

654

CHAPTER 9

Fact 9.11.3. Let A ∈ Fn×n, and let mspec(A) = {λ1, . . . , λn }ms . Then, Re tr A2 ≤ |tr A2 | ≤

n 

|λi |2 ≤ A2 σ1 = tr A2  =

i=1

≤

n 

n 

σi(A2 )

i=1

σi2(A) = tr A∗A = tr A2 = A2σ2 = A2F

i=1

and A2F

−

+

∗ n3 −n 12 [A, A ]F

≤

n 

|λi |2 ≤

+ A4F − 12 [A, A∗ ]2F ≤ A2F .

i=1

Consequently, A is normal if and only if A2F

=

n 

|λi |2.

i=1

Furthermore,

n 

|λi |2 ≤

+ A4F − 14 (tr |[A, A∗ ]|)2 ≤ A2F

i=1

and

n 

|λi |2 ≤

+ A4F −

n2 ∗ 2/n 4 |det [A, A ]|

≤ A2F .

i=1

Finally, A is Hermitian if and only if A2F = tr A2. Proof: Use Fact 8.18.5 and Fact 9.11.2. The lower bound involving the commutator is due to Henrici; the corresponding upper bound is given in [871]. The bounds in the penultimate statement are given in [871]. The last statement follows from Fact 3.7.13. Remark: tr (A + A∗ )2 ≥ 0 and tr (A − A∗ )2 ≤ 0 yield |tr A2 | ≤ A2F . n Remark: The result i=1 |λi |2 ≤ A2F is Schur’s inequality. See Fact 8.18.5. Remark: See Fact 5.11.10, Fact 9.11.5, Fact 9.13.16, and Fact 9.13.19. Problem: Merge the first two strings. Fact 9.11.4. Let A ∈ Fn×n. Then, + |tr A2 | ≤ (rank A) A4F − 12 [A, A∗ ]2F . Proof: See [323].

655

NORMS

Fact 9.11.5. Let A ∈ Fn×n, let mspec(A) = {λ1, . . . , λn }ms , and define + 2  A2F − n1 |tr A|2 − 12 [A, A∗ ]2F + n1 |tr A|2. α= Then, n 

|λi |2 ≤ α ≤

+

A4F − 12 [A, A∗ ]2F ≤ A2F ,

i=1 n 

(Re λi )2 ≤ 12 (α + Re tr A2 ),

i=1 n 

(Im λi )2 ≤ 12 (α − Re tr A2 ).

i=1

Proof: See [751]. Remark: The first string of inequalities interpolates the upper bound for in the second string of inequalities in Fact 9.11.3.

n

2 i=1 |λi |

Fact 9.11.6. Let A ∈ Fn×n, let mspec(A) = {λ1, . . . , λn }ms , and let p ∈ (0, 2]. Then, n n   p |λi | ≤ σip (A) = Apσp ≤ App . i=1

i=1

Proof: The left-hand inequality, which holds for all p > 0, follows from Weyl’s inequality in Fact 8.18.5. The right-hand inequality is given by Proposition 9.2.5. Remark: This result is the generalized Schur inequality. Remark: The case of equality is discussed in [763] for p ∈ [1, 2). Fact 9.11.7. Let A ∈ Fn×n, and let mspec(A) = {λ1, . . . , λn }ms . Then,   n n   2 2 ∗ 2 2 1 AF − |λi | = 2  j2 (A − A )F − | Im λi | . i=1

i=1

Proof: See Fact 5.11.22. Remark: This result is an extension of Browne’s theorem. Fact 9.11.8. Let A ∈ Rn×n, and let λ ∈ spec(A). Then, the following inequalities hold: i) |λ| ≤ nA∞ . D D ii) |Re λ| ≤ n2 DA + AT D∞ . √ D D 2 DA − AT D . iii) |Im λ| ≤ 2n√−n ∞ 2 Proof: See [988, p. 140]. Remark: i) and ii) are Hirsch’s theorems, while iii) is Bendixson’s theorem. See Fact 5.11.21.

656

CHAPTER 9

Fact 9.11.9. Let A ∈ Rn×n, and assume that A is nonnegative. Then, min coli (A)1 ≤ sprad(A) ≤ Acol

i=1,...,n

and

min coli (AT )1 ≤ sprad(A) ≤ Arow .

i=1,...,n

Proof: See [1208, pp. 318, 319]. Remark: The upper bounds are given by (9.4.26) and (9.4.27).

9.12 Facts on Matrix Norms and Eigenvalues for Two or More Matrices Fact 9.12.1. Let A, B ∈ Fn×m, let mspec(A∗B) = {λ1, . . . , λm }ms , let p, q ∈  [1, ∞] satisfy 1/p + 1/q = 1, and define r = min{m, n}. Then, |tr A∗B| ≤

m 

m 

|λi | ≤ A∗Bσ1 =

i=1

σi(A∗B) ≤

i=1

In particular,

r 

σi(A)σi(B) ≤ Aσp Bσq .

i=1

|tr A∗B| ≤ AF BF .

Proof: Use Proposition 9.6.2 and Fact 9.11.2. The last inequality in the string of inequalities is H¨older’s inequality. Remark: See Fact 9.9.11. Remark: The result |tr A∗B| ≤

r 

σi(A)σi(B)

i=1

is von Neumann’s trace inequality. See [254]. Fact 9.12.2. Let A, B ∈ Fn×m, and let mspec(A∗B) = {λ1, . . . , λm }ms . Then, m m   |tr (A∗B)2 | ≤ |λi |2 ≤ σi2(A∗B) = tr AA∗BB ∗ = A∗B2F ≤ A2F B2F . i=1

i=1

Proof: Use Fact 8.18.5. Fact 9.12.3. Let A, B ∈ Fn×n, assume that A and B are Hermitian, and let mspec(A + jB) = {λ1, . . . , λn }ms . Then, n 

|Re λi |2 ≤ A2F

i=1

and

n  i=1

Proof: See [1125, p. 146].

|Im λi |2 ≤ B2F .

657

NORMS

Fact 9.12.4. Let A, B ∈ Fn×n, assume that A and B are Hermitian, and let  ·  be a weakly unitarily invariant norm on Fn×n. Then, D⎡ ⎤ ⎡ ⎤D D λ1(A) D 0 0 λ1(B) D D D⎢ D ⎥ ⎢ ⎥ .. .. D⎣ ⎦−⎣ ⎦D ≤ A − B . . D D D 0 λn(A) 0 λn(B) D D⎡ D λ1(A) D D⎢ ≤ D⎣ D D 0

⎤

0 ..

⎡

⎥ ⎢ ⎦−⎣

.

λn(B) ..

λn(A)

⎤D D D ⎥D ⎦D . D λ1(B) D 0

.

0

In particular, |λi(A) − λi(B)| ≤ σmax (A − B) ≤

max

i∈{1,...,n} n 

and

2

[λi(A) − λi(B)] ≤ A − B2F ≤

i=1

max

i∈{1,...,n}

n 

|λi(A) − λn−i+1(B)| 2

[λi(A) − λn−i+1(B)] .

i=1

Proof: See [49], [200, p. 38], [201, pp. 63, 69], [204, p. 38], [818, p. 126], [903, p. 134], [921], or [1261, p. 202]. Remark: The first inequality is the Lidskii-Mirsky-Wielandt theorem. This result can be stated without norms using Fact 9.8.42. See [921]. Remark: See Fact 9.14.29. Remark: The case in which A and B are normal is considered in Fact 9.12.8. Fact 9.12.5. Let A, B ∈ Fn×n, let mspec(A) = {λ1, . . . , λn }ms and mspec(B) = {μ1, . . . , μn }ms , and assume that A and B satisfy at least one of the following conditions: i) A and B are Hermitian. ii) A is Hermitian, and B is skew Hermitian. iii) A is skew Hermitian, and B is Hermitian. iv) A and B are unitary. v) There exist nonzero α, β ∈ C such that αA and βB are unitary. vi) A, B, and A − B are normal. Then,

⎛⎡

⎜⎢ min σmax ⎝⎣

0

λ1 .. 0

. λn

⎤

⎡

⎥ ⎢ ⎦−⎣

0

μσ(1) .. 0

.

⎤⎞ ⎥⎟ ⎦⎠ ≤ σmax (A − B),

μσ(n)

where the minimum is taken over all permutations σ of {1, . . . , n}. Proof: See [204, pp. 52, 152].

658

CHAPTER 9

Fact 9.12.6. Let A, B ∈ Fn×n, let mspec(A) = {λ1, . . . , λn }ms and mspec(B) = {μ1, . . . , μn }ms , and assume that A is normal. Then, D⎡ ⎤ ⎡ ⎤D D D λ1 0 0 μσ(1) D D √ D D⎢ ⎥ ⎢ ⎥ .. .. min D⎣ ⎦−⎣ ⎦D ≤ nA − BF , . . D D D 0 λn 0 μσ(n) D F

where the minimum is taken over all permutations σ of B is normal, then there exists c ∈ (0, 2.9039) such that ⎛⎡ ⎤ ⎡ 0 0 λ1 μσ(1) ⎜⎢ ⎥ ⎢ . . .. .. min σmax ⎝⎣ ⎦−⎣ 0

λn

0

{1, . . . , n}. If, in addition, ⎤⎞ ⎥⎟ ⎦⎠ ≤ cσmax (A − B).

μσ(n)

Proof: See [204, pp. 152, 153, 173]. Remark: Constants c for alternative Schatten norms are given in [204, p. 159]. Remark: If, in addition, A − B is normal, then it follows from Fact 9.12.5 that the last inequality holds with c = 1. Fact 9.12.7. Let A, B ∈ Fn×n, let mspec(A) = {λ1, . . . , λn }ms and mspec(B) = {μ1, . . . , μn }ms , and assume that A is Hermitian. Then, D⎡ ⎤ ⎡ ⎤D D D λ1 0 0 μσ(1) D D √ D D⎢ ⎥ ⎢ ⎥ .. .. min D⎣ ⎦−⎣ ⎦D ≤ 2A − BF , . . D D D 0 λn 0 μσ(n) DF where the minimum is taken over all permutations σ of {1, . . . , n}. Proof: See [204, p. 174]. Fact 9.12.8. Let A, B ∈ Fn×n, assume that A and B are normal, and let spec(A) = {λ1, . . . , λq } and spec(B) = {μ1, . . . , μr }. Then, σmax (A − B) ≤ max{|λi − λj | : i = 1, . . . , q, j = 1, . . . , r}. Proof: See [201, p. 164]. Remark: The case in which A and B are Hermitian is considered in Fact 9.12.4. Fact 9.12.9. Let A, B ∈ Fn×n, and assume that A and B are normal. Then, there exists a permutation σ of 1, . . . , n such that n 

|λσ(i)(A) − λi(B)|2 ≤ A − B2F .

i=1

Proof: See [728, p. 368] or [1125, pp. 160, 161]. Remark: This inequality is the Hoffman-Wielandt theorem. Remark: The case in which A and B are Hermitian is considered in Fact 9.12.4.

659

NORMS

Fact 9.12.10. Let A, B ∈ Fn×n, and assume that A is Hermitian and B is normal. Furthermore, let mspec(B) = {λ1(B), . . . , λn(B)}ms , where Re λn(B) ≤ · · · ≤ Re λ1(B). Then, n 

|λi(A) − λi(B)|2 ≤ A − B2F .

i=1

Proof: See [728, p. 370]. Remark: This result is a special case of Fact 9.12.9. Remark: The left-hand side has the same value for all orderings that satisfy Re λn(B) ≤ · · · ≤ Re λ1(B). Fact 9.12.11. Let A, B ∈ Fn×n, and let  ·  be an induced norm on Fn×n. Then, * n −Bn A − B A A−B , A = B, |det A − det B| ≤ A = B. nA − BAn−1, Proof: See [518]. Remark: See Fact 1.20.2.

9.13 Facts on Matrix Norms and Singular Values for One Matrix Fact 9.13.1. Let A ∈ Fn×m. Then, σmax (A) = and thus



max m

x∈F \{0}

x∗A∗Ax x∗x

1/2 ,

Ax2 ≤ σmax (A)x2 . 

Furthermore, λmin(A∗A) = 1/2

and thus

min n

x∈F \{0}

x∗A∗Ax x∗x

1/2 ,

λmin(A∗A)x2 ≤ Ax2 . 1/2

If, in addition, m ≤ n, then



σm(A) = and thus

min

x∈Fn \{0}

x∗A∗Ax x∗x

1/2 ,

σm(A)x2 ≤ Ax2 .

Finally, if m = n, then

 σmin (A) =

and thus Proof: See Lemma 8.4.3.

min n

x∈F \{0}

x∗A∗Ax x∗x

σmin (A)x2 ≤ Ax2 .

1/2 ,

660

CHAPTER 9

Fact 9.13.2. Let A ∈ Fn×m. Then, σmax (A) = max{|y ∗Ax|: x ∈ Fm, y ∈ Fn, x2 = y2 = 1} = max{|y ∗Ax|: x ∈ Fm, y ∈ Fn, x2 ≤ 1, y2 ≤ 1}. Remark: See Fact 9.8.36. 

Fact 9.13.3. Let x ∈ Fn and y ∈ Fm, and define S = {A ∈ Fn×m : σmax (A) ≤ 1}. Then, √ max x∗Ay = x∗xy∗y. A∈S

Fact 9.13.4. Let  ·  be an equi-induced unitarily invariant norm on Fn×n. Then,  ·  = σmax (·). Fact 9.13.5. Let  ·  be an equi-induced self-adjoint norm on Fn×n. Then,  ·  = σmax (·). Fact 9.13.6. Let A ∈ Fn×n. Then, σmin (A) − 1 ≤ σmin (A + I) ≤ σmin (A) + 1. Proof: Use Proposition 9.6.8. Fact 9.13.7. Let A ∈ Fn×n, assume that A is normal, and let r ∈ N. Then, r (A). σmax (Ar ) = σmax

Remark: that are not normal might also satisfy these conditions. Con 1 0 0Matrices  sider 0 0 0 . 010

Fact 9.13.8. Let A ∈ Fn×n. Then,   2 2 2 (A) − σmax A2 ≤ σmax (A∗A − AA∗ ) ≤ σmax (A) − σmin (A) σmax and

  2 2 2 σmax (A) + σmin (A) ≤ σmax (A∗A + AA∗ ) ≤ σmax (A) + σmax A2 .

If A2 = 0, then

2 (A). σmax (A∗A − AA∗ ) = σmax

Proof: See [844, 848]. Remark: See Fact 8.19.11.

  2 (A) ≤ σmax A2 , although Fact Remark: If A is normal, then it follows that σmax 9.13.7 implies that equality holds. Fact 9.13.9. Let A ∈ Fn×n. Then, the following statements are equivalent: i) sprad(A) = σmax (A). i (A) for all i ∈ P. ii) σmax (Ai ) = σmax n (A). iii) σmax (An ) = σmax

Proof: See [506] and [730, p. 44].

661

NORMS

Remark: The implication iii) =⇒ i) is due to Ptak. Remark: Additional conditions are given in [581]. Fact 9.13.10. Let A ∈ Fn×n. Then, σmax (A) ≤ σmax (|A|) ≤

√ rank Aσmax (A).

Proof: See [699, p. 111]. Fact 9.13.11. Let A ∈ Fn×n, and let p ∈ [2, ∞) be an even integer. Then, Aσp ≤  |A| σp . In particular,

AF ≤  |A| F

and

σmax (A) ≤ σmax (|A|).

Finally, let  ·  be a unitarily invariant norm on Cn×m. Then, AF =  |A| F for all A ∈ Cn×m if and only if  ·  is a constant multiple of  · F . Proof: See [731] and [749].  rank A ≥ 2. If r tr A2 ≤ Fact 9.13.12. Let A ∈ Rn×n, and assume that r = (tr A)2, then ? (tr A)2 − tr A2 ≤ sprad(A). r(r − 1)

If (tr A)2 ≤ r tr A2, then |tr A| + r

?

r tr A2 − (tr A)2 ≤ sprad(A). r2 (r − 1)

If rank A = 2, then equality holds in both cases. Finally, if A is skew symmetric, ? then 3 AF ≤ sprad(A). r(r − 1) Proof: See [737]. Fact 9.13.13. Let A ∈ Rn×n. Then, + 1 (A2F + tr A2 ) ≤ σmax (A). 2(n2 −n) Furthermore, if AF ≤ tr A, then σmax (A) ≤ n1 tr A +

+

n−1 n



A2F − n1 (tr A)2 .

Proof: See [1017], which considers the complex case.

662

CHAPTER 9

Fact 9.13.14. Let A ∈ Fn×n. Then, the polynomial p ∈ R[s] defined by 

p(s) = sn − A2F s + (n − 1)|det A|2/(n−1) has either exactly one or exactly two positive roots 0 < α ≤ β. Furthermore, α and β satisfy α(n−1)/2 ≤ σmin (A) ≤ σmax (A) ≤ β (n−1)/2. Proof: See [1167]. Fact 9.13.15. Let A ∈ Fn×n. Then, tr A = tr A∗ . Fact 9.13.16. Let A ∈ Fn×n. Then, for all k ∈ {1, . . . , n}, k k     σi A2 ≤ σi2(A). i=1

i=1

Hence,

1/2  ≤ tr A∗A, tr A2∗A2

that is,

@ A tr A2 ≤ tr A2.

Proof: Let B = A and r = 1 in Proposition 9.6.2. See also Fact 9.11.3. Fact 9.13.17. Let A ∈ Fn×n, and let k denote the number of nonzero eigenvalues of A. Then, @ A⎫ |tr A2 | ≤ tr A2 ⎪ ⎪ ⎪ ⎬ ∗ tr AA  ≤ tr A2. ⎪ ⎪ ⎪ ⎭ 1 2 k |tr A| 2 Proof: @ 2 A The upper bound for |tr A | is given by Fact 9.11.3. The upper bound for tr A is given by Fact 9.13.16. To prove the center inequality, let A = S1DS2 denote the singular value decomposition of A. Then, tr AA∗  = tr S3∗DS3 D,  where S3 = S1S2 , and tr A∗A = tr D2. The result now follows using Fact 5.12.4. The remaining inequality is given by Fact 5.11.10.

Remark: See Fact 5.11.10 and Fact 9.11.3. Fact 9.13.18. Let A ∈ Fn×n, and let mspec(A) = {λ1, . . . , λn }ms , where λ1, . . . , λn are ordered such that |λ1| ≥ · · · ≥ |λn |. Then, for all k ∈ {1, . . . , n}, k !

|λi |2 ≤

i=1

and

n ! i=1

|λi |2 =

k k !   ! σi A2 ≤ σi2(A) i=1

i=1

n n !   ! σi A2 = σi2(A) = |det A|2. i=1

i=1

663

NORMS

Furthermore, for all k ∈ {1, . . . , n}, . . k k k .  .  . . λi . ≤ |λi | ≤ σi (A), . . . i=1

i=1

and thus |tr A| ≤

k 

i=1

|λi | ≤ tr A.

i=1

Proof: See [730, p. 172], and use Fact 5.11.28. For the last statement, use Fact 2.21.12. Remark: See Fact 5.11.28, Fact 8.19.22, and Fact 9.11.2. Remark: This result is due to Weyl. Fact 9.13.19. Let A ∈ Fn×n, and let mspec(A) = {λ1, . . . , λn }ms , where λ1, . . . , λn are ordered such that |λn | ≤ · · · ≤ |λ1|, and let p ≥ 0. Then, for all k ∈ {1, . . . , n}, . . k k k . .   . . λpi . ≤ |λi |p ≤ σip (A). . . . i=1

i=1

i=1

Proof: See [201, p. 42]. Remark: This result is Weyl’s majorant theorem. Remark: See Fact 9.11.3. Fact 9.13.20. Let A ∈ Fn×n, and define 

ri =

n 

|A(i,j) |,



ci =

j=1 i=1,...,n



rˆi =

|A(i,j) |,

j=1 j=i

and

 min α=

i=1,...,n

|A(j,i) |,

j=1

 min ri 2 , rmin =

n 

n 

 cmin = min ci 2 ,

i=1,...,n



cˆi =

n 

|A(j,i) |,

j=1 j=i



 |A(i,i) | − rˆi ,

 β= min

i=1,...,n

Then, the following statements hold: i) If α > 0, then A is nonsingular and A−1 row < 1/α. ii) If β > 0, then A is nonsingular and A−1 col < 1/β. iii) If α > 0 and β > 0, then A is nonsingular, and  αβ ≤ σmin (A).



 |A(i,i) | − cˆi .

664

CHAPTER 9

iv) σmin (A) satisfies min 1 i=1,...,n 2 v) σmin (A) satisfies 1 i=1,...,n 2

min



2|A(i,i) | − rˆi − cˆi ≤ σmin (A).

  1/2 4|A(i,i) |2 + [ˆ ri − cˆi ]2 − rˆi − cˆi ≤ σmin (A).

vi) σmin (A) satisfies  n−1 (n−1)/2 n

 |det A| max

c r 6nmin , 6nmin c i=1 i i=1 ri

 ≤ σmin (A).

Proof: See Fact 9.8.23, [730, pp. 227, 231], and [725, 786, 796, 1401]. Fact 9.13.21. Let A ∈ Fn×n, and let mspec(A) = {λ1, · · · , λn }ms , where λ1, . . . , λn are ordered such that |λn | ≤ · · · ≤ |λ1|. Then, for all i ∈ {1, . . . , n}, 1/k

lim σi

k→∞

In particular,

(Ak ) = |λi |.

k lim σ1/k max (A ) = sprad(A).

k→∞

Proof: See [730, p. 180]. Remark: This equality is due to Yamamoto. Remark: The expression for sprad(A) is a special case of Proposition 9.2.6. Fact 9.13.22. Let A ∈ Fn×n, and assume that A is nonzero. Then, 1 σmax (A)

=

min

B∈{X∈Fn×n : det(I−AX)=0}

σmax (B).

Furthermore, there exists B0 ∈ Fn×n such that rank B0 = 1, det(I − AB0 ) = 0, and 1 σmax (A)

= σmax (B0 ).

Proof: If σmax (B) < 1/σmax (A), then sprad(AB) ≤ σmax (AB) < 1, and thus I − AB is nonsingular. Hence, 1 σmax (A)

= = ≤

min

B∈{X∈Fn×n : σmax (X)≥1/σmax (A)}

min

σmax (B)

B∈{X∈Fn×n : σmax (X) 0. Then,  2 

1/2 2 (A) + 1 + α−1 σmax (B) σmax (A + B) ≤ (1 + α)σmax and

 2 

1/2 2 (A) + 1 + α−1 σmax (B) . σmin (A + B) ≤ (1 + α)σmin Fact 9.14.17. Let A, B ∈ Fn×n. Then, σmin (A) − σmax (B) ≤ |det(A + B)|1/n n ! ≤ |σi (A) + σn−i+1(B)|1/n i=1

≤ σmax (A) + σmax (B). Proof: See [740, p. 63] and [920]. Fact 9.14.18. Let A, B ∈ Fn×n, and assume that σmax (B) ≤ σmin (A). Then, 0 ≤ [σmin (A) − σmax (B)]n n ! ≤ |σi (A) − σn−i+1(B)| i=1

≤ |det(A + B)| n ! ≤ |σi (A) + σn−i+1(B)| i=1

≤ [σmax (A) + σmax (B)]n.

670

CHAPTER 9

Hence, if σmax (B) < σmin (A), then A is nonsingular and A + αB is nonsingular for all −1 ≤ α ≤ 1. Proof: See [920]. Remark: See Fact 5.12.12 and Fact 11.18.16. Fact 9.14.19. Let A, B ∈ Fn×m. Then, the following statements are equivalent: i) For all k ∈ {1, . . . , min{n, m}}, k 

σi(A) ≤

i=1

k 

σi(B).

i=1

ii) For all unitarily invariant norms  ·  on Fn×m , A ≤ B. Proof: See [730, pp. 205, 206]. Remark: This result is the Fan dominance theorem. Fact 9.14.20. Let A, B ∈ Fn×m. Then, for all k ∈ {1, . . . , min{n, m}}, k 

[σi(A) + σmin{n,m}+1−i(B)] ≤

i=1

k 

σi(A + B) ≤

i=1

k 

[σi(A) + σi(B)].

i=1

Furthermore, if either σmax (A) < σmin (B) or σmax (B) < σmin (A), then, for all k ∈ {1, . . . , min{n, m}}, k 

σi(A + B) ≤

i=1

k 

|σi(A) − σmin{n,m}+1−i(B)|.

i=1

Proof: See Proposition 9.2.2, [730, pp. 196, 197] and [920]. Fact 9.14.21. Let A, B ∈ Fn×m, and let α ∈ [0, 1]. Then, for all i ∈ {1, . . . , min{n, m}},   ⎧ A 0 ⎪ ⎪ σ ⎪ i ⎪ 0 B ⎨ σi [αA + (1 − α)B] ≤   √ ⎪ ⎪ 2αA  0 ⎪ ⎪ σ , ⎩ i 2(1 − α)B 0 and Furthermore,

2σi (AB ∗ ) ≤ σi (A2 + B2 ). αA + (1 − α)B2 ≤ αA2 + (1 − α)B2.

If, in addition, n = m, then, for all i ∈ {1, . . . , n},   A 0 ∗ 1 . σ (A + A ) ≤ σ i 2 i 0 A Proof: See [716]. Remark: See Fact 9.14.23.

671

NORMS

Fact 9.14.22. Let A ∈ Fn×m and B ∈ Fl×m, and let p, q > 1 satisfy 1/p+1/q = 1. Then, for all i ∈ {1, . . . , min{n, m, l}},   σi(AB ∗ ) ≤ σi p1 Ap + 1q Bq . Equivalently, there exists a unitary matrix S ∈ Fm×m such that   AB ∗ 1/2 ≤ S ∗ p1 Ap + 1q Bq S. Proof: See [49, 51, 712] or [1521, p. 28]. Remark: This result is a matrix version of Young’s inequality. See Fact 1.12.32. Fact 9.14.23. Let A ∈ Fn×m and B ∈ Fl×m. Then, for all i ∈ {1, . . . , min{n, m, l}}, σi(AB ∗ ) ≤ 12 σi(A∗A + B ∗B). Proof: Set p = q = 2 in Fact 9.14.22. See [213]. Remark: See Fact 9.9.47 and Fact 9.14.21. Fact 9.14.24. Let A, B, C, D ∈ Fn×m. Then, for all i ∈ {1, . . . , min{n, m}},   √ A B . 2σi (AB ∗ + CD∗ ) ≤ σi C D Proof: See [711]. Fact 9.14.25. Let A, B, C, D, X ∈ Fn×n, assume that A, B, C, D are positive semidefinite, and assume that 0 ≤ A ≤ C and 0 ≤ B ≤ D. Then, for all i ∈ {1, . . . , n}, σi (A1/2XB 1/2 ) ≤ σi (C 1/2XD1/2 ). Proof: See [716, 840]. Fact 9.14.26. Let A1, . . . , Ak ∈ Fn×n. Then, for all l ∈ {1, . . . , n}, ⎛ ⎞ k l k l !  !  σi ⎝ Aj ⎠ ≤ σi (Aj ). i=1

j=1

i=1 j=1

Proof: See [325]. Remark: This result is a weak majorization relation. Fact 9.14.27. Let A, B ∈ Fn×m, and let 1 ≤ l1 < · · · < lk ≤ min{n, m}. Then, k k k    σli(A)σn−i+1(B) ≤ σli(AB) ≤ σli(A)σi(B) i=1

and

i=1 k  i=1

σli(A)σn−li +1(B) ≤

i=1 k  i=1

σi(AB).

672

CHAPTER 9

In particular, k 

σi(A)σn−i+1(B) ≤

k 

i=1

σi(AB) ≤

i=1

Furthermore,

k !

σli (AB) ≤

i=1

k 

σi(A)σi(B).

i=1

k !

σli (A)σi(B)

i=1

with equality for k = n. Furthermore, k !

σli (A)σn−li +1(B) ≤

i=1

k !

σi(AB)

i=1

with equality for k = n. In particular, k !

σi(A)σn−i+1(B) ≤

i=1

k !

σi(AB) ≤

i=1

k !

σi(A)σi(B)

i=1

with equality for k = n. Proof: See [1422]. Remark: See Fact 8.19.20 and Fact 8.19.23. Remark: The left-hand inequalities in the first and third strings are conjectures. See [1422]. Fact 9.14.28. Let A ∈ Fn×m, let k ≥ 1 satisfy k < rank A, and let  ·  be a unitarily invariant norm on Fn×m. Then, min

B∈{X∈Fn×m: rank X≤k}

A − B = A − B0 ,

where B0 is formed by replacing (rank A) − k smallest positive singular values in the singular value decomposition of A by 0’s. Furthermore, σmax (A − B0 ) = σk+1(A) and

/ 0  0 r A − B0 F = 1 σi2(A). i=k+1

Furthermore, B0 is the unique solution if and only if σk+1 (A) < σk (A). 

Proof: This result follows from Fact 9.14.29 with Bσ = diag[σ1(A), . . . , σk(A), 0(n−k)×(m−k) ], S1 = In , and S2 = Im . See [583] and [1261, p. 208]. Remark: This result is known as the Schmidt-Mirsky theorem. For the case of the Frobenius norm, the result is known as the Eckart-Young theorem. See [520] and [1261, p. 210]. Remark: See Fact 9.15.8.

673

NORMS

Fact 9.14.29. Let A, B ∈ Fn×m, define Aσ , Bσ ∈ Fn×m ⎡ σ1(A) ⎢ ..  ⎢ . Aσ = ⎢ ⎣ σr(A) 0(n−r)×(m−r)

by ⎤ ⎥ ⎥ ⎥, ⎦



where r = rank A, and

⎡

⎢  ⎢ Bσ = ⎢ ⎣

⎤

σ1(B) ..

⎥ ⎥ ⎥, ⎦

. σl(B) 0(n−l)×(m−l)



where l = rank B, let S1 ∈ Fn×n and S2 ∈ Fm×m be unitary matrices, and let  ·  be a unitarily invariant norm on Fn×m. Then, Aσ − Bσ  ≤ A − S1BS2  ≤ Aσ + Bσ . In particular, max

i∈{1,...,max{r,l}}

|σi(A) − σi(B)| ≤ σmax (A − B) ≤ σmax (A) + σmax (B).

Proof: See [1424]. Remark: In the case S1 = In and S2 = Im , the left-hand inequality is Mirsky’s theorem. See [1261, p. 204]. Remark: See Fact 9.12.4. Fact 9.14.30. Let A, B ∈ Fn×m, and assume that rank A = rank B. Then, σmax [AA+(I − BB + )] = σmax [BB +(I − AA+ )] ≤ min{σmax (A+ ), σmax (B + )}σmax (A − B). Proof: See [699, p. 400] and [1261, p. 141]. Fact 9.14.31. Let A, B ∈ Fn×m. Then, for all k ∈ {1, . . . , min{n, m}}, k 

σi(A ◦ B) ≤

i=1

k 

di (A∗A)di (BB∗ ) 1/2

1/2

i=1

* k ≤

≤

1/2 ∗ i=1 di (A A)σi (B) k 1/2 ∗ i=1 σi (A)di (BB )

k  i=1

σi(A)σi(B)

-

674

CHAPTER 9

and k 

σi(A ◦ B) ≤

i=1

k 

di (AA∗ )di (B∗B) 1/2

1/2

i=1

* k

1/2 ∗ i=1 di (AA )σi (B) k 1/2 ∗ i=1 σi (A)di (B B)

≤

≤

k 

-

σi(A)σi(B).

i=1

In particular,

*

σmax(A ◦ B) ≤ A2,1 B∞,2 ≤ and

A2,1 σmax(B)

≤ σmax (A)σmax (B)

σmax(A)B∞,2 *

σmax(A ◦ B) ≤ A∞,2 B2,1 ≤

A∞,2 σmax(B)

-

σmax(A)B2,1

≤ σmax (A)σmax (B).

Proof: See [58, 1001, 1517] and [730, pp. 332, 334], and use Fact 2.21.2, Fact 8.18.8, and Fact 9.8.24. Remark: di (A∗A) and di (AA∗ ) are the ith largest Euclidean norms of the columns and rows of A, respectively. 1/2

1/2

Remark: For related results, see [1378]. Remark: The case of equality is discussed in [327]. Fact 9.14.32. Let A, B ∈ Fn×m. Then, √ σmax (A ◦ B) ≤ nA∞ σmax (B). Now, assume in addition that n = m and that either A is positive semidefinite and B is Hermitian or A and B are nonnegative and symmetric. Then, σmax (A ◦ B) ≤ A∞ σmax (B). Next, assume that A and B are real, let β denote the smallest positive entry of |B|, and assume that B is symmetric and positive semidefinite. Then, sprad(A ◦ B) ≤

A∞ B∞ σmax (B) β

and sprad(B) ≤ sprad(|B|) ≤ Proof: See [1107].

B∞ sprad(B). β

675

NORMS

Fact 9.14.33. Let A, B ∈ Fn×m, and let p ∈ [2, ∞) be an even integer. Then, A ◦ B2σp ≤ A ◦ Aσp B ◦ Bσp . In particular, and

A ◦ B2F ≤ A ◦ AF B ◦ BF

2 2 2 (A ◦ B) ≤ σmax (A ◦ A)σmax (B ◦ B) ≤ σmax (A)σmax (B). σmax

Equality holds if B = A. Furthermore, A ◦ Aσp ≤ A ◦ Aσp . In particular, and

A ◦ AF ≤ A ◦ AF σmax (A ◦ A) ≤ σmax (A ◦ A).

Now, assume in addition that n = m. Then, A ◦ AT σp ≤ A ◦ Aσp . In particular, and

A ◦ AT F ≤ A ◦ AF σmax (A ◦ AT ) ≤ σmax (A ◦ A).

Finally, In particular, and

A ◦ A∗ σp ≤ A ◦ Aσp . A ◦ A∗ F ≤ A ◦ AF σmax (A ◦ A∗ ) ≤ σmax (A ◦ A).

Proof: See [730, p. 340] and [731, 1224]. Remark: See Fact 7.6.16 and Fact 9.14.31. Fact 9.14.34. Let A, B ∈ Cn×m. Then, A ◦ B2F =

n 

σi2(A ◦ B)

i=1

= tr (A ◦ B)(A ◦ B)T = tr (A ◦ A)(B ◦ B)T n  ≤ σi [(A ◦ A)(B ◦ B)T ] i=1

≤

n  i=1

Proof: See [749].

σi (A ◦ A)σi (B ◦ B).

676

CHAPTER 9

Fact 9.14.35. Let A, B ∈ Fn×n. Then, (A ◦ B)(A ◦ B)T σ1 ≤ (A ◦ A)(B ◦ B)T σ1 , (A ◦ B)(A ◦ B)T F ≤ (A ◦ A)(B ◦ B)T F , σmax [(A ◦ B)(A ◦ B)T ] ≤ σmax [(A ◦ A)(B ◦ B)T ]. Proof: See [452]. Fact 9.14.36. Let A, B ∈ Rn×n, assume that A and B are nonnegative, and let α ∈ [0, 1]. Then, α 1−α σmax (A◦α ◦ B ◦(1−α) ) ≤ σmax (A)σmax (B).

In particular, Finally,

σmax (A◦1/2 ◦ B ◦1/2 ) ≤



σmax (A)σmax (B).

σmax (A◦1/2 ◦ A◦1/2T ) ≤ σmax (A◦α ◦ A◦(1−α)T ) ≤ σmax (A).

Proof: See [1224]. Remark: See Fact 7.6.17. C

Fact 9.14.37. Let · be a unitarily invariant norm on Cn×n, and let A, X, B ∈ . Then, √ A ◦ X ◦ B ≤ 12 nA ◦ X ◦ A + B ◦ X ◦ B

n×n

and

A ◦ X ◦ B2 ≤ nA ◦ X ◦ AB ◦ X ◦ B.

Furthermore,

A ◦ X ◦ BF ≤ 12 A ◦ X ◦ A + B ◦ X ◦ BF .

Proof: See [749]. Fact 9.14.38. Let A ∈ Fn×m, B ∈ Fl×k, and p ∈ [1, ∞]. Then, A ⊗ Bσp = Aσp Bσp . In particular,

σmax (A ⊗ B) = σmax (A)σmax (B)

and

A ⊗ BF = AF BF .

Proof: See [708, p. 722].

9.15 Facts on Linear Equations and Least Squares Fact 9.15.1. Let A ∈ Rn×n, assume that A is nonsingular, let b ∈ Rn, and let x ˆ ∈ Rn. Then, 1 Aˆ x − b ˆ x − A−1b Aˆ x − b ≤ ≤ κ(A) , κ(A) b A−1b b

677

NORMS 

where κ(A) = AA−1 and the vector and matrix norms are compatible. Equiv  alently, letting ˆb = Aˆ x and x = A−1 b, it follows that ˆ x − x ˆb − b 1 ˆb − b ≤ ≤ κ(A) . κ(A) b x b Remark: This result estimates the accuracy of an approximate solution x ˆ to Ax = b. κ(A) is the condition number of A. Remark: For A = σmax (A), note that κ(A) = κ(A−1 ). Remark: See [1538]. Fact 9.15.2. Let A ∈ Rn×n, assume that A is nonsingular, let A˜ ∈ Rn×n, ˜ < 1, and let b, ˜b ∈ Rn. Furthermore, let x ∈ Rn satisfy Ax = b, assume that A−1A n ˜ x = b + ˜b. Then, and let xˆ ∈ R satisfy (A + A)ˆ   ˜ κ(A) ˆ x − x ˜b A ≤ + , ˜ x b A 1 − A−1A  where κ(A) = AA−1 and the vector and matrix norms are compatible. If, in −1 ˜ < 1, then addition, A A

˜ ˜ ˜b − Ax 1 ˜b − Ax ˆ x − x κ(A) ≤ ≤ . ˜ κ(A) + 1 b x b 1 − A−1A Proof: See [417, 418]. ˆ < 1, let b ∈ R(A), let ˆb ∈ Rn, Fact 9.15.3. Let A, Aˆ ∈ Rn×n satisfy A+A ˆ x = b+ ˆb. ˆ Furthermore, let xˆ ∈ Rn satisfy (A+ A)ˆ and assume that b+ ˆb ∈ R(A+ A).  + + Then, x = A b + (I − A A)ˆ x satisfies Ax = b and   ˆ ˆb A κ(A) ˆ x − x ≤ + , ˆ x b A 1 − A+A 

where κ(A) = AA−1 and the vector and matrix norms are compatible. Proof: See [417]. Remark: See [418] for a lower bound. Fact 9.15.4. Let A ∈ Fn×m and b ∈ Fn, and define f(x) = (Ax − b)∗ (Ax − b) = Ax − b22 , 

where x ∈ Fm. Then, f has a minimizer. Furthermore, x ∈ Fm minimizes f if and only if there exists a vector y ∈ Fm such that x = A+b + (I − A+A)y. In this case,

f(x) = b∗(I − AA+ )b.

Furthermore, if y ∈ Fm is such that (I − A+A)y is nonzero, then + A+b2 < A+b + (I − A+A)y2 = A+b22 + (I − A+A)y22 .

678

CHAPTER 9

Finally, A+b is the unique minimizer of f if and only if A is left invertible. Remark: The minimization of f is the least squares problem. See [16, 230, 1257]. Note that the expression for x is identical to the expression (6.1.13) for solutions of Ax = b. Therefore, x satisfies Ax = b if and only if x is optimal in the least-squares sense. However, unlike Proposition 6.1.7, consistency is not assumed, that is, there need not exist a solution to Ax = b. Remark: This result is a special case of Fact 8.14.15. Fact 9.15.5. Let A ∈ Fn×m and b ∈ Fn, define 

f(x) = (Ax − b)∗ (Ax − b) = Ax − b22 , ˆ = Bb minimizes where x ∈ Fm, and let B ∈ Fm×n be a (1,3)-inverse of A. Then, x f. Furthermore, let z ∈ Fm. Then, the following statements are equivalent: i) z minimizes f. ii) f (z) = b∗ (I − AB)b. iii) Az = ABb. iv) There exists y ∈ Rm such that z = Bb + (I − BA)y. Proof: See [1208, pp. 233–236]. Fact 9.15.6. Let A ∈ Fn×m, B ∈ Fn×l, and define  f(X) = tr[(AX − B)∗(AX − B)] = AX − B2F ,

where X ∈ Fm×l. Then, X = A+B minimizes f. Problem: Determine all minimizers. Problem: Consider f(X) = tr[(AX − B)∗C(AX − B)], where C ∈ Fn×n is positive definite. Fact 9.15.7. Let A ∈ Fn×m and B ∈ Fl×m, and define  f(X) = tr[(XA − B)∗(XA − B)] = XA − B2F ,

where X ∈ Fl×n. Then, X = BA+ minimizes f. Fact 9.15.8. Let A ∈ Fn×m, B ∈ Fn×p, and C ∈ Fq×m, and let k ≥ 1 satisfy k < rank A. Then, min

X∈{Y ∈Fp×q: rank Y ≤k}

A − BXCF = A − BX0 CF ,

where X0 = B +SC + and S is formed by replacing all but the k largest singular values in the singular value decomposition of BB +AC +C by 0’s. Furthermore, X0 is a solution that minimizes XF. Finally, X0 is the unique solution if and only if either rank BB +AC +C ≤ k or both k ≤ BB +AC +C and σk+1 (BB +AC +C) < σk (BB +AC +C).

679

NORMS

Proof: See [520]. Remark: This result generalizes Fact 9.14.28. Fact 9.15.9. Let A, B ∈ Fn×m, and define 

f(X) = tr[(AX − B)∗(AX − B)] = AX − B2F ,



where X ∈ Fm×m is unitary. Then, X = S1S2 minimizes f, where S1 B0ˆ 00 S2 is the singular value decomposition of A∗B. Proof: See [148, p. 224]. See also [996, pp. 269, 270]. Fact 9.15.10. Let A, B ∈ Rn×n, and define

 f(X1, X2 ) = tr (X1AX2 − B)T(X1AX2 − B) = X1AX2 − B2F , n×n are orthogonal. Then, (X1, X2 ) = (V2TU1T, V1T U 2T ) minimizes where X1, X 2 ∈ R

ˆ0 A f, where U1 0 0 V1 is the singular value decomposition of A and U2 B0ˆ 00 V2 is the singular value decomposition of B.

Proof: See [996, p. 270]. Remark: This result is due to Kristof. Remark: See Fact 3.9.5. Problem: Extend this result to C and nonsquare matrices.

A b = Fact 9.15.11. Let A ∈ Rn×m, let b ∈ Rn, and assume that rank

m + 1. Furthermore, consider the singular value decomposition of A b given by  

Σ A b =U V, 0(n−m−1)×(m+1) where U ∈ Rn×n and V ∈ R(m+1)×(m+1) are orthogonal and 

Σ = diag[σ1(A), . . . , σm+1 (A)]. Furthermore, define Aˆ ∈ Rn×m and ˆb ∈ Rn by  

 Σ0 V, Aˆ ˆb = U 0(n−m−1)×(m+1) 

where Σ0 = diag[σ1(A), . . . , σm (A), 0]. Finally, assume that V(m+1,m+1) = 0, and ⎡ ⎤ define V(m+1,1) 1 ⎢ ⎥  .. x ˆ=− ⎣ ⎦. . V(m+1,m+1) V(m+1,m) ˆx = ˆb. Then, Aˆ Remark: x ˆ is the total least squares solution. See [1392].

Remark: The construction of Aˆ ˆb is based on Fact 9.14.28.

680

CHAPTER 9

9.16 Notes The equivalence of absolute and monotone norms given by Proposition 9.1.2 is due to [159]. More general monotonicity conditions are considered in [790]. Induced lower bounds are treated in [892, pp. 369, 370]. See also [1261, pp. 33, 80]. The induced norms (9.4.13) and (9.4.14) are given in [318] and [699, p. 116]. Alternative norms for the convolution operator are given in [318, 1469]. Proposition 9.3.6 is given in [1155, p. 97]. Norm-related topics are discussed in [173]. Spectral perturbation theory in finite and infinite dimensions is treated in [818], where the emphasis is on the regularity of the spectrum as a function of the perturbation rather than on bounds for finite perturbations.

Chapter Ten

Functions of Matrices and Their Derivatives

The norms discussed in Chapter 9 provide the foundation for the development in this chapter of some basic results in topology and analysis.

10.1 Open Sets and Closed Sets Let  ·  be a norm on Fn, let x ∈ Fn, and let ε > 0. Then, define the open ball of radius ε centered at x by  Bε (x) = {y ∈ Fn : x − y < ε}

(10.1.1)

and the sphere of radius ε centered at x by 

Sε (x) = {y ∈ Fn : x − y = ε}.

(10.1.2)

Definition 10.1.1. Let S ⊆ Fn. The vector x ∈ S is an interior point of S if there exists ε > 0 such that Bε (x) ⊆ S. The interior of S is the set 

int S = {x ∈ S: x is an interior point of S}.

(10.1.3)

Finally, S is open if every element of S is an interior point, that is, if S = int S. Definition 10.1.2. Let S ⊆ S ⊆ Fn. The vector x ∈ S is an interior point of S relative to S if there exists ε > 0 such that Bε (x) ∩ S ⊆ S or, equivalently, Bε (x) ∩ S = Bε (x) ∩ S . The interior of S relative to S is the set  intS S = {x ∈ S: x is an interior point of S relative to S }. (10.1.4) Finally, S is open relative to S if S = intS S. As an example, the interval [0, 1) is open relative to the interval [0, 2]. Definition 10.1.3. Let S ⊆ Fn. The vector x ∈ Fn is a closure point of S if, for all ε > 0, the set S ∩ Bε (x) is not empty. The closure of S is the set  cl S = {x ∈ Fn : x is a closure point of S}.

(10.1.5)

Finally, the set S is closed if every closure point of S is an element of S, that is, if S = cl S.

682

CHAPTER 10

Definition 10.1.4. Let S ⊆ S ⊆ Fn. The vector x ∈ S is a closure point of S relative to S if, for all ε > 0, the set S ∩ Bε (x) is not empty. The closure of S relative to S is the set  {x ∈ Fn : x is a closure point of S relative to S }. clS S =

(10.1.6)



Finally, S is closed relative to S if S = clS S. As an example, the interval (0, 1] is closed relative to the interval (0, 2]. It follows from Theorem 9.1.8 on the equivalence of norms on Fn that these definitions are independent of the norm assigned to Fn. Let S ⊆ S ⊆ Fn. Then, clS S = (cl S) ∩ S ,

and In particular, and



(10.1.7)

int S = S \cl(S \S),

(10.1.8)

int S ⊆ intS S ⊆ S ⊆ clS S ⊆ cl S.

(10.1.9)

int S = [cl(S∼ )]∼

(10.1.10)

int S ⊆ S ⊆ cl S.

(10.1.11)

S

The set S is solid if int S is not empty, while S is completely solid if cl(int S) = cl S. If S is completely solid, then S is solid. The boundary of S is the set  cl S\int S, bd S =

(10.1.12)



while the boundary of S relative to S is the set 

bdS S = clS S\intS S.

(10.1.13)

Note that the empty set is both open and closed, although it is not solid. The set S ⊂ Fn is bounded if there exists δ > 0 such that, for all x, y ∈ S, x − y < δ.

(10.1.14)

The set S ⊂ Fn is compact if it is both closed and bounded.

10.2 Limits Definition 10.2.1. The sequence (x1, x2 , . . .) is a tuple with a countably infinite number of components. We write (xi )∞ i=1 for (x1, x2 , . . .). Definition 10.2.2. The sequence (αi )∞ i=1 ⊂ F converges to α ∈ F if, for all ε > 0, there exists a positive integer p such that |αi − α| < ε for all i > p. In this case, we write α = limi→∞ αi or αi → α as i → ∞, where i ∈ P. Finally, the ∞ sequence (αi )∞ i=1 ⊂ F converges if there exists α ∈ F such that (αi )i=1 converges to α.

683

FUNCTIONS OF MATRICES AND THEIR DERIVATIVES

n converges to x ∈ Fn if Definition 10.2.3. The sequence (xi )∞ i=1 ⊂ F n limi→∞ x−xi  = 0, where · is a norm on F . In this case, we write x = limi→∞ xi n or xi → x as i → ∞, where i ∈ P. The sequence (xi )∞ i=1 ⊂ F converges if there n ∞ ∞ exists x ∈ F such that (xi )i=1 converges to x. Similarly, (Ai )i=1 ⊂ Fn×m converges to A ∈ Fn×m if limi→∞ A − Ai  = 0, where  ·  is a norm on Fn×m. In this case, we write A = limi→∞ Ai or Ai → A as i → ∞, where i ∈ P. Finally, the sequence n×m (Ai )∞ converges if there exists A ∈ Fn×m such that (Ai )∞ i=1 ⊂ F i=1 converges to A.

It follows from Theorem 9.1.8 that convergence of a sequence is independent of the choice of norm. Proposition 10.2.4. Let S ⊆ Fn. The vector x ∈ Fn is a closure point of S if and only if there exists a sequence (xi )∞ i=1 ⊆ S that converges to x. Proof. Suppose that x ∈ Fn is a closure point of S. Then, for all i ∈ P, there exists a vector xi ∈ S such that x − xi  < 1/i. Hence, x − xi → 0 as i → ∞. Conversely, suppose that (xi )∞ i=1 ⊆ S is such that xi → x as i → ∞, and let ε > 0. Then, there exists a positive integer p such that x−xi  < ε for all i > p. Therefore, xp+1 ∈ S ∩ Bε (x), and thus S ∩ Bε (x) is not empty. Hence, x is a closure point of S. Theorem 10.2.5. Let S ⊂ Fn be compact, and let (xi )∞ i=1 ⊆ S. Then, ∞ ∞ there exists a subsequence (xij )∞ of (x ) such that (x ) i i j j=1 converges and j=1 i=1 limj→∞ xij ∈ S. Proof. See [1057, p. 145]. Next, we define convergence for the series k x i=1 i .

∞

i=1 xi

in terms of the partial sums

Definition 10.2.6. Let (xi )∞ ⊂ Fn, and let  ·  be a norm on Fn. Then, the ∞ k i=1∞ ∞ series i=1 xi convergesif ( i=1 xi )k=1 converges. Furthermore, i=1 xi converges ∞ absolutely if the series i=1 xi  converges. ⊂ Fn, and assume that the series Proposition 10.2.7. Let (xi )∞ i=1 ∞ converges absolutely. Then, the series i=1 xi converges.

∞

i=1 xi

n×m , and let · be a norm on Fn×m. Then, Definition 10.2.8. Let (Ai )∞ i=1 ⊂ F ∞ k ∞ the series Furthermore, i=1 Ai converges if ( i=1 ∞Ai )k=1 converges.  ∞ A converges absolutely if the series A  converges. i i i=1 i=1 ∞ n×m ∞ Proposition 10.2.9. Let (Ai )i=1 ⊂ F ∞, and assume that the series i=1 Ai converges absolutely. Then, the series i=1 Ai converges.

684

CHAPTER 10

10.3 Continuity Definition 10.3.1. Let D ⊆ Fm, f : D → Fn, and x ∈ D. Then, f is continuous at x if, for every convergent sequence (xi )∞ i=1 ⊆ D such that limi→∞ xi = x, it follows that limi→∞ f(xi ) = f(x). Furthermore, let D0 ⊆ D. Then, f is continuous on D0 if f is continuous at x for all x ∈ D0 . Finally, f is continuous if it is continuous on D. Theorem 10.3.2. Let D ⊆ Fn be convex, and let f : D → F be convex. Then, f is continuous on intaff D D. Proof. See [161, p. 81] and [1161, p. 82].  Corollary 10.3.3. Let A ∈ Fn×m, and define f : Fm → Fn by f(x) = Ax. Then, f is continuous.

Proof. This result is a consequence of Theorem 10.3.2. Alternatively, let m x ∈ Fm, and let (xi )∞ be such that xi → x as i → ∞. Furthermore, let i=1 ⊂ F  · ,  ·  , and  ·  be compatible norms on Fn, Fn×m, and Fm, respectively. Since Ax − Axi  ≤ A x − xi  , it follows that Axi → Ax as i → ∞. Theorem 10.3.4. Let D ⊆ Fm, and let f : D → Fn. Then, the following statements are equivalent: i) f is continuous. ii) For all open S ⊆ Fn, the set f −1(S) is open relative to D. iii) For all closed S ⊆ Fn, the set f −1(S) is closed relative to D. Proof. See [1057, pp. 87, 110]. 

Corollary 10.3.5. Let A ∈ Fn×m and S ⊆ Fn, and define S = {x ∈ Fm : Ax ∈ S}. If S is open, then S is open. If S is closed, then S is closed. The following result is the open mapping theorem. Theorem 10.3.6. Let D ⊆ Fm, let A ∈ Fn×m, assume that D is open, and assume that A is right invertible. Then, AD is open. The following result is the invariance of domain. Theorem 10.3.7. Let D ⊆ Fn, let f: D → Fn , assume that D is open, and assume that f is continuous and one-to-one. Then, f(D) is open. Proof. See [1248, p. 3]. Theorem 10.3.8. Let D ⊂ Fm be compact, and let f: D → Fn be continuous. Then, f(D) is compact.

FUNCTIONS OF MATRICES AND THEIR DERIVATIVES

685

Proof. See [1057, p. 146]. The following corollary of Theorem 10.3.8 shows that a continuous real-valued function defined on a compact set has a minimizer and a maximizer. Corollary 10.3.9. Let D ⊂ Fm be compact, and let f: D → R be continuous. Then, there exist x0 , x1 ∈ D such that, for all x ∈ D, f(x0 ) ≤ f(x) ≤ f(x1 ). The following result is the Schauder fixed-point theorem. Theorem 10.3.10. Let D ⊆ Fm, assume that D is nonempty, closed, and convex, let f : D → D, assume that f is continuous, and assume that f(D) is bounded. Then, there exists x ∈ D such that f(x) = x. Proof. See [1438, p. 167]. The following corollary for the case of a bounded domain is the Brouwer fixed-point theorem. Corollary 10.3.11. Let D ⊆ Fm, assume that D is nonempty, compact, and convex, let f : D → D, and assume that f is continuous. Then, there exists x ∈ D such that f(x) = x. Proof. See [1438, p. 163]. Definition 10.3.12. Let S ⊆ Fn×n. Then, S is pathwise connected if, for all B1, B2 ∈ S, there exists a continuous function f : [0, 1] → S such that f (0) = B1 and f (1) = B2.

10.4 Derivatives Let D ⊆ Fm, and let x0 ∈ D. Then, the variational cone of D with respect to x0 is the set 

vcone(D, x0 ) = {ξ ∈ Fm : there exists α0 > 0 such that, for all α ∈ [0, α0 ), x0 + αξ ∈ D}.

(10.4.1)

Note that vcone(D, x0 ) is a pointed cone, although it may consist of only ξ = 0 as can be seen from the example x0 = 0 and ; : D = x ∈ R2 : 0 ≤ x(1) ≤ 1, x3(1) ≤ x(2) ≤ x2(1) . Now, let D ⊆ Fm and f : D → Fn. If ξ ∈ vcone(D, x0 ), then the one-sided directional differential of f at x0 in the direction ξ is defined by 

D+f (x0 ; ξ) = lim α1 [f (x0 + αξ) − f(x0 )] α↓0

(10.4.2)

if the limit exists. Similarly, if ξ ∈ vcone(D, x0 ) and −ξ ∈ vcone(D, x0 ), then the

686

CHAPTER 10

two-sided directional differential Df(x0 ; ξ) of f at x0 in the direction ξ is defined by 

1 [f (x0 α→0 α

Df (x0 ; ξ) = lim

+ αξ) − f(x0 )]

(10.4.3)

if the limit exists. If ξ = ei so that the direction ξ is one of the coordinate axes, 0) then the partial derivative of f with respect to x(i) at x0 , denoted by ∂f(x ∂x(i) , is given by ∂f(x0 )  = lim α1 [f(x0 + αei ) − f(x0 )], α→0 ∂x(i)

(10.4.4)

∂f(x0 ) = Df(x0 ; ei ), ∂x(i)

(10.4.5)

that is,

when the two-sided directional differential Df(x0 ; ei ) exists. Note that Fn×1.

∂f(x0 ) ∂x(i)

∈

Proposition 10.4.1. Let D ⊆ Fm be a convex set, let f : D → Fn be convex, and let x0 ∈ int D. Then, D+f(x0 ; ξ) exists for all ξ ∈ Fm. Proof. See [161, p. 83]. Note that D+f (x0 ; ξ) = ±∞ is possible if x0 is an element of the boundary of D. For example, consider the convex, continuous function f : [0, ∞) → R given by √ f(x) = 1 − x. In this case, D+f (0; 1) = −∞ and thus does not exist. Next, we consider a stronger form of differentiation. Proposition 10.4.2. Let D ⊆ Fm be solid and convex, let f : D → Fn, and let x0 ∈ D. Then, there exists at most one matrix F ∈ Fn×m satisfying lim

x→x0 x∈D\{x0 }

x − x0 −1[f(x) − f(x0 ) − F (x − x0 )] = 0.

(10.4.6)

Proof. See [1438, p. 170]. In (10.4.6) the limit is taken over all sequences that are contained in D, do not include x0 , and converge to x0 . Note that D is not necessarily open, and x0 may be an element of the boundary of D. Definition 10.4.3. Let D ⊆ Fm be solid and convex, let f : D → Fn, let x0 ∈ D, and assume there exists a matrix F ∈ Fn×m satisfying (10.4.6). Then, f is differentiable at x0 , and the matrix F is the derivative of f at x0 . In this case, we write f (x0 ) = F and lim

x→x0 x∈D\{x0 }

x − x0 −1[f(x) − f(x0 ) − f (x0 )(x − x0 )] = 0.

(10.4.7)

Note that Proposition 10.4.2 and Definition 10.4.3 do not require that x0 lie 0) for f (x0 ). in the interior of D. We alternatively write df(x dx

687

FUNCTIONS OF MATRICES AND THEIR DERIVATIVES

Proposition 10.4.4. Let D ⊆ Fm be solid and convex, let f : D → Fn, let x ∈ D, and assume that f is differentiable at x0 . Then, f is continuous at x0 . Let D ⊆ Fm be solid and convex, and let f : D → Fn. In terms of its scalar

T components, f can be written as f = f1 · · · fn , where fi : D → F for all

T for all x ∈ D. With this notation, i ∈ {1, . . . , n} and f(x) = f1(x) · · · fn(x) if f (x0 ) exists, then it can be written as ⎡ ⎤ f1(x0 ) ⎢ ⎥ .. f (x0 ) = ⎣ (10.4.8) ⎦, . fn (x0 )

where fi (x0 ) ∈ F1×m is the gradient of fi at x0 and f (x0 ) is the Jacobian of f at x0 . Furthermore, if x ∈ int D, then f (x0 ) is related to the partial derivatives of f by   ∂f(x0 ) ∂f(x0 ) ··· f (x0 ) = , (10.4.9) ∂x(1) ∂x(m) where

∂f (x0 ) ∂x(i)

∈ Fn×1 for all i ∈ {1, . . . , m}. Finally, note that the (i, j) entry of the

n × m matrix f (x0 ) is

∂fi(x0 ) ∂x(j) .

For example, if x ∈ Fn and A ∈ Fn×n, then d Ax = A. dx

(10.4.10)

Note that the existence of the partial derivatives of f at x0 does not imply that f (x0 ) exists. That is, f may not be differentiable at x0 since f (x0 ) given by (10.4.9) may not satisfy (10.4.7). Let D ⊆ Fm and f : D → Fn, and assume that f (x) exists for all x ∈ D and f : D → Fn×m is continuous. Then, f is continuously differentiable, or C1. Note that, for all x0 ∈ D, f (x0 ) ∈ Fn×m, and thus f (x0 ) : Fm → Fn. The second derivative of f at x0 ∈ D, denoted by f (x0 ), is the derivative of f : D → Fn×m at x0 ∈ D. By analogy with the first derivative, it follows that f (x0 ): Fm → Fn×m is linear. Therefore, for all η ∈ Fm, it follows that f (x0 )η ∈ Fn×m, and, thus, for  all η, ηˆ ∈ Fm, it follows that [f (x0 )η]ˆ η ∈ Fn. Defining f (x0 )(η, ηˆ) = [f (x0 )η]ˆ η , it m m n follows that f (x0 ): F × F → F is bilinear, that is, for all ηˆ ∈ Fm, the mapping η → f (x0 )(η, ηˆ) is linear, and, for all η ∈ Fm, the mapping ηˆ → f (x0 )(η, ηˆ) is

T linear. Letting f = f1 · · · fn , it follows that ⎡ T ⎤ η f1 (x0 ) ⎢ ⎥ .. f (x0 )η = ⎣ (10.4.11) ⎦, . η Tfn (x0 )

and

⎡

⎤ η Tf1 (x0 )ˆ η ⎢ ⎥ .. f (x0 )(η, ηˆ) = ⎣ ⎦, . η η Tfn (x0 )ˆ

(10.4.12)

688

CHAPTER 10

where, for all i ∈ {1, . . . , n}, the matrix fi (x0 ) is the m × m Hessian of fi at x0 . We write f (2) (x0 ) for f (x0 ) and f (k) (x0 ) for the kth derivative of f at x0 . The function f is Ck if f (k) (x) exists for all x ∈ D and f (k) is continuous on D. The following result is the inverse function theorem [461, p. 185]. Theorem 10.4.5. Let D ⊆ Fn be open, let f : D → Fn, and assume that f is Ck. Furthermore, let x0 ∈ D be such that det f (x0 ) = 0. Then, there exist open sets N ⊂ Fn containing x0 and M ⊂ Fn containing f (x0 ) and a Ck function g : M → N such that, for all x ∈ N, g[f (x)] = x, and, for all y ∈ M, f [g(y)] = y. Let S: [t0 , t1 ] → Fn×m, and assume that every entry of S(t) is differentiable.  ˙ Then, define S(t) = dS(t) ∈ Fn×m for all t ∈ [t0 , t1 ] entrywise, that is, for all dt i ∈ {1, . . . , n} and j ∈ {1, . . . , m}, d S(i,j) (t). (10.4.13) dt If t = t0 or t = t1, then d+/dt or d−/dt (or just d/dt) denotes the right and left ,t one-sided derivatives, respectively. Finally, define t01 S(t) dt entrywise, that is, for all i ∈ {1, . . . , n} and j ∈ {1, . . . , m}, ⎤ ⎡ t t  ˙ [S(t)] (i,j) =

1

1

⎣ S(t) dt⎦ t0

(i,j)



=

[S(t)](i,j) dt.

(10.4.14)

t0

10.5 Functions of a Matrix Let D ⊆ C, let f : D → C, and assume that there exists γ > 0 such that Bγ (0) ⊆ D. Furthermore, assume that there exist complex numbers β0 , β1, . . . such that, for all s ∈ Bγ (0), f (s) is given by the convergent series f(s) =

∞ 

βi si.

(10.5.1)

i=0

Next, let A ∈ Cn×n be such that sprad(A) < γ, and define f (A) by the convergent series ∞   f(A) = βiAi. (10.5.2) i=0 −1

Expressing A as A = SBS , where S ∈ Cn×n is nonsingular and B ∈ Cn×n, it follows that f(A) = Sf(B)S −1. (10.5.3) If, in addition, B = diag(J1, . . . , Jr ) is the Jordan form of A, then f(A) = S diag[f(J1 ), . . . , f(Jr )]S −1.

(10.5.4)

Letting J = λIk + k denote a k × k Jordan block, expanding and rearranging N ∞ the infinite series i=1 βi J i shows that f(J) is the k × k upper triangular Toeplitz

689

FUNCTIONS OF MATRICES AND THEIR DERIVATIVES

matrix 1 f (k−1)(λ)Nkk−1 (k − 1)! ⎤ 1 (k−1) (λ) (k−1)! f ⎥ 1 (k−2) (λ) ⎥ ⎥ (k−2)! f ⎥ 1 ⎥ (k−3) f (λ) (10.5.5) ⎥. (k−3)! ⎥ ⎥ .. ⎥ . ⎦

f(J) = f(λ)Ik + f (λ)Nk + 12 f (λ)Nk2 + · · · + ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ =⎢ ⎢ ⎢ ⎢ ⎣

f(λ)

f (λ)

1 2 f (λ)

···

0

f(λ)

f (λ)

···

0 .. .

0 .. .

f(λ)

···

..

..

0

0

0

.

.

···

f(λ)

Next, we extend the definition f(A) to functions f : D ⊆ C → C that are not necessarily of the form (10.5.1). Definition 10.5.1. Let f : D ⊆ C → C, let A ∈ Cn×n, where spec(A) ⊂ D, and assume that, for all λi ∈ spec(A), f is ki − 1 times differentiable at λi , where  ki = indA(λi ) is the order of the largest Jordan block associated with λi as given by Theorem 5.3.3. Then, f is defined at A, and f(A) is given by (10.5.3) and (10.5.4), where f(Ji ) is defined by (10.5.5) with k = ki and λ = λi . The following result shows that the definition of f (A) in Definition 10.5.1 is uniquely defined in the sense that f (A) is independent of the decomposition A = SBS −1 used to define f (A) in (10.5.3). Theorem 10.5.2. Let A ∈ Fn×n, let spec(A) = {λ1 , . . . , λr }, and, for all i ∈  {1, . . . , r}, let ki = indA(λi ). Furthermore, suppose that f : D ⊆ C → C is defined at A. Then, there exists a polynomial p ∈ F[s] such that f(A) r = p(A). Furthermore, there exists a unique polynomial p of degree less than i=1 ki satisfying f(A) = p(A) and such that, for all i ∈ {1, . . . , r} and j ∈ {0, 1, . . . , ki − 1}, f (j)(λi ) = p(j)(λi ). This polynomial is given by ⎛⎡ ⎤ p(s) =

r 

r !

i=1

j=1 j=i

⎜⎢ ⎜⎢ ⎝⎣

⎥ (s − λj )nj⎥ ⎦

k i −1 k=0

. . . . 1 dk f(s) . 6 r . k k l k! ds l=1 (s − λl ) . l=i .

(10.5.6) ⎞ ⎟ (s − λi )k⎟ ⎠.

(10.5.7)

s=λi

If, in addition, A is simple, then p is given by p(s) =

r  i=1

f(λi )

r ! s − λj . λ − λj j=1 i

(10.5.8)

j=i

Proof. See [367, pp. 263, 264]. The polynomial (10.5.7) is the Lagrange-Hermite interpolation polynomial for f.

690

CHAPTER 10

The following result, which is known as the identity theorem, is a special case of Theorem 10.5.2. Theorem 10.5.3. Let A ∈ Fn×n, let spec(A) = {λ1 , . . . , λr }, and, for all i ∈  {1, . . . , r}, let ki = indA(λi ). Furthermore, let f : D ⊆ C → C and g : D ⊆ C → C be analytic on a neighborhood of spec(A). Then, f(A) = g(A) if and only if, for all i ∈ {1, . . . , r} and j ∈ {0, 1, . . . , ki − 1}, f (j)(λi ) = g (j)(λi ).

(10.5.9)

Corollary 10.5.4. Let A ∈ Fn×n, and let f : D ⊂ C → C be analytic on a neighborhood of mspec(A). Then, mspec[f (A)] = f [mspec(A)].

(10.5.10)

10.6 Matrix Square Root and Matrix Sign Functions Theorem 10.6.1. Let A ∈ Cn×n, and assume that A is group invertible and has no eigenvalues in (−∞, 0). Then, there exists a unique matrix B ∈ Cn×n such that spec(B) ⊂ ORHP ∪ {0} and such that B 2 = A. If, in addition, A is real, then B is real. Proof. See [701, pp. 20, 31]. The matrix B given by Theorem 10.6.1 is the principal square root of A. This matrix is denoted by A1/2. The existence of a square root that is not necessarily the principal square root is discussed in Fact 5.15.19. The following result defines the matrix sign function. Definition 10.6.2. Let A ∈ Cn×n, assume that A has no eigenvalues on the imaginary axis, and let   J1 0 A=S S −1, 0 J2 where S ∈ Cn×n is nonsingular, J1 ∈ Cp×p and J2 ∈ Cq×q are in Jordan canonical form, and spec(J1 ) ⊂ OLHP and spec(J2 ) ⊂ ORHP. Then, the matrix sign of A is defined by   −Ip 0  Sign(A) = S S −1. 0 Iq

10.7 Matrix Derivatives In this section we consider derivatives of differentiable scalar-valued functions with matrix arguments. Consider the linear function f : Fm×n → F given by f(X) = tr AX, where A ∈ Fn×m and X ∈ Fm×n. In terms of vectors x = vec X ∈  Fmn, we can define the linear function fˆ(x) = f (X) = (vec AT )Tx. Consequently,

691

FUNCTIONS OF MATRICES AND THEIR DERIVATIVES

for all Y ∈ Fm×n,

d dX f (X0 ) :

Fm×n → F can be represented by

d f (X0 )Y = fˆ (vec X0 ) vec Y = (vec AT )T vec Y = tr AY . (10.7.1) dX d f (X0 ) with the matrix A, Noting that fˆ (vec X0 ) = (vec AT )T and identifying dX m×n we define the matrix derivative of f : D ⊆ F → F by T  d  f(X) = vec−1 [fˆ (vec X)]T , (10.7.2) dX ∂f(X) . Note the order of indices. which is the n × m matrix A whose (i, j) entry is ∂X (j,i) The matrix derivative is a representation of the derivative in the sense that

lim

X→X0 X∈D\{X0 }

where F denotes

d dX f(X0 )

f(X) − f(X0 ) − tr[F (X − X0 )] = 0, X − X0 

(10.7.3)

and  ·  is a norm on Fm×n.

Proposition 10.7.1. Let x ∈ Fn. Then, the following statements hold: i) If A ∈ Fn×n, then ii) If A ∈ Fn×n

iii) If A ∈ Fn×n

  d T x Ax = xT A + AT . dx is symmetric, then d T x Ax = 2xTA. dx is Hermitian, then d ∗ x Ax = 2x∗A. dx

(10.7.4)

(10.7.5)

(10.7.6)

Proposition 10.7.2. Let A ∈ Fn×m and B ∈ Fl×n. Then, the following statements hold: i) For all X ∈ Fm×n, ii) For all X ∈ Fm×l, iii) For all X ∈ Fl×m,

d tr AX = A. dX

(10.7.7)

d tr AXB = BA. dX

(10.7.8)

d tr AXTB = ATBT. dX

(10.7.9)

iv) For all X ∈ Fm×l and k ≥ 1,

v) For all X ∈ Fm×l,

d tr (AXB)k = kB(AXB)k−1A. dX

(10.7.10)

d det AXB = B(AXB)AA. dX

(10.7.11)

692

CHAPTER 10

vi) For all X ∈ Fm×l such that AXB is nonsingular, d log det AXB = B(AXB)−1A. dX

(10.7.12)

Proposition 10.7.3. Let A ∈ Fn×m and B ∈ Fm×n. Then, the following statements hold: i) For all X ∈ Fm×m and k ≥ 1,  d tr AX kB = X k−1−iBAX i. dX i=0 k−1

(10.7.13)

ii) For all nonsingular X ∈ Fm×m, d tr AX −1B = −X −1BAX −1. dX iii) For all nonsingular X ∈ Fm×m, d det AX −1B = −X −1B(AX −1B)AAX −1. dX

(10.7.14)

(10.7.15)

iv) For all nonsingular X ∈ Fm×m, d log det AX −1B = −X −1B(AX −1B)−1AX −1. dX

(10.7.16)

Proposition 10.7.4. The following statements hold: i) Let A, B ∈ Fn×m. Then, for all X ∈ Fm×n,

ii) Let A ∈ Fn×n

d tr AXBX = AXB + BXA. dX and B ∈ Fm×m. Then, for all X ∈ Fn×m,

d tr AXBXT = BXTA + BTXTAT. dX iii) Let A ∈ Fn×n. Then, for all X ∈ Fn×m, d tr XTAX = XT(A + AT ). dX

(10.7.17)

(10.7.18)

(10.7.19)

iv) Let A ∈ Fk×l, B ∈ Fl×m, C ∈ Fn×l, D ∈ Fl×l, and E ∈ Fl×k. Then, for all X ∈ Fm×n, d tr A(D + BXC)−1E = −C(D + BXC)−1EA(D + BXC)−1B. (10.7.20) dX v) Let A ∈ Fk×l, B ∈ Fl×m, C ∈ Fn×l, D ∈ Fl×l, and E ∈ Fl×k. Then, for all X ∈ Fn×m,  −1 d tr A D + BXTC E dX   −T −T = −BT D + BXTC ATE T D + BXTC CT. (10.7.21)

FUNCTIONS OF MATRICES AND THEIR DERIVATIVES

693

10.8 Facts on One Set Fact 10.8.1. Let x ∈ Fn, and let ε > 0. Then, Bε (x) is completely solid and convex. Fact 10.8.2. Let S ⊂ Fn, let  ·  be a norm on Fn, assume that there exists δ > 0 such that, for all x, y ∈ S, x − y < δ, and let x0 ∈ S. Then, S ⊆ Bδ (x0 ). Fact 10.8.3. Let S ⊆ Fn. Then, cl S is the smallest closed set containing S, and int S is the largest open set contained in S. Fact 10.8.4. Let S ⊆ Fn. If S is (open, closed), then S∼ is (closed, open). Fact 10.8.5. Let S ⊆ S ⊆ Fn. If S is (open relative to S , closed relative to S ), then S \S is (closed relative to S , open relative to S ). Fact 10.8.6. Let S ⊆ Fn. Then, (int S)∼ = cl(S∼ ) and

bd S = bd S∼ = (cl S) ∩ (cl S∼ ) = [(int S) ∪ int(S∼ )]∼ .

Hence, bd S is closed. Fact 10.8.7. Let S ⊆ Fn, and assume that S is either open or closed. Then, int bd S is empty. Proof: See [71, p. 68]. Fact 10.8.8. Let S ⊆ Fn, and assume that S is convex. Then, cl S, int S, and intaff S S are convex. Proof: See [1161, p. 45] and [1162, p. 64]. Fact 10.8.9. Let S ⊆ Fn, and assume that S is convex. Then, the following statements are equivalent: i) S is solid. ii) S is completely solid. iii) dim S = n. iv) aff S = Fn. Fact 10.8.10. Let S ⊆ Fn, and assume that S is solid. Then, co S is completely solid. Fact 10.8.11. Let S ⊆ Fn. Then, cl S ⊆ aff cl S = aff S. Proof: See [243, p. 7].

694

CHAPTER 10

Fact 10.8.12. Let k ≤ n, and let x1, . . . , xk ∈ Fn. Then, int aff {x1, . . . , xk } = ∅. Remark: See Fact 2.9.7. Fact 10.8.13. Let S ⊆ Fn. Then, co cl S ⊆ cl co S. Now, assume in addition that S is either bounded or convex. Then, co cl S = cl co S. Proof: Use Fact 10.8.8 and Fact 10.8.13. Remark: Although

; : S = x ∈ R2 : x2(1) x2(2) = 1 for all x(1) > 0

is closed, co S is not closed. Hence, co cl S ⊂ cl co S. Fact 10.8.14. Let S ⊆ Fn, and assume that S is open. Then, co S is open. Fact 10.8.15. Let S ⊆ Fn, and assume that S is compact. Then, co S is compact. Fact 10.8.16. Let S ⊆ Fn, and assume that S is solid. Then, dim S = n. Fact 10.8.17. Let S ⊆ Fm, assume that S is solid, let A ∈ Fn×m, and assume that A is right invertible. Then, AS is solid. Proof: Use Theorem 10.3.6. Remark: See Fact 2.10.4. Fact 10.8.18. Nn is a closed and completely solid subset of Fn(n+1)/2. Furthermore, int Nn = Pn. Fact 10.8.19. Let S ⊆ Fn, and assume that S is convex. Then, int cl S = int S. Remark: The result follows from Fact 10.9.4. Fact 10.8.20. Let D ⊆ Fn, and let x0 belong to a solid, convex subset of D. Then, dim vcone(D, x0 ) = n. Fact 10.8.21. Let S ⊆ Fn, and assume that S is a subspace. Then, S is closed. Fact 10.8.22. Let S ⊂ Fn, assume that S is symmetric, convex, bounded, solid, and closed, and, for all x ∈ Fn, define  x = min{α ≥ 0: x ∈ αS} = max{α ≥ 0: αx ∈ S}.

FUNCTIONS OF MATRICES AND THEIR DERIVATIVES

695

Then,  ·  is a norm on Fn, and B1 (0) = int S. Conversely, let  ·  be a norm on Fn. Then, B1 (0) is symmetric, convex, bounded, and solid. Proof: See [740, pp. 38, 39]. Remark: In all cases, B1(0) is defined with respect to  · . This result is due to Minkowski. Remark: See Fact 9.7.23. Fact 10.8.23. Let S ⊆ Rn, assume that S is nonempty, closed, and convex, and let E ⊆ S denote the set of elements of S that cannot be represented as nontrivial convex combinations of two distinct elements of S. Then, E is nonempty and satisfies S = co E. If, in addition, n = 2, then E is closed. Proof: See [459, pp. 482–484]. Remark: E is the set of extreme points of S. Remark: E is not necessarily closed for n > 2. See [459, p. 483]. Remark: The last result is the Krein-Milman theorem.

10.9 Facts on Two or More Sets Fact 10.9.1. Let S1 ⊆ S2 ⊆ Fn. Then, cl S1 ⊆ cl S2 and

int S1 ⊆ int S2 . Fact 10.9.2. Let S1, S2 ⊆ Fn. Then, the following statements hold:

i) (int S1 ) ∩ (int S2 ) = int(S1 ∩ S2 ). ii) (int S1 ) ∪ (int S2 ) ⊆ int(S1 ∪ S2 ). iii) (cl S1 ) ∪ (cl S2 ) = cl(S1 ∪ S2 ). iv) bd(S1 ∪ S2 ) ⊆ (bd S1 ) ∪ (bd S2 ). v) If (cl S1 ) ∩ (cl S2 ) = ∅, then bd(S1 ∪ S2 ) = (bd S1 ) ∪ (bd S2 ). Proof: See [71, p. 65]. Fact 10.9.3. Let S1, S2 ⊆ Fn, assume that either S1 or S2 is closed, and assume that int S1 = int S2 = ∅. Then, int(S1 ∪ S2 ) is empty. Proof: See [71, p. 69]. Remark: The set int(S1 ∪ S2 ) is not necessarily empty if neither S1 nor S2 is closed. Consider the sets of rational and irrational numbers.

696

CHAPTER 10

Fact 10.9.4. Let S1, S2 ⊆ Fn, and assume that S1 is open, S2 is convex, and S1 ⊆ cl S2 . Then, S1 ⊆ S2 . Proof: See [1208, pp. 72, 73]. Remark: See Fact 10.8.19. Fact 10.9.5. Let S1, S2 ⊆ Fn, and assume that S1 is closed and S2 is compact. Then, S1 + S2 is closed. Proof: See [454, p. 209]. Fact 10.9.6. Let S1, S2 ⊆ Fn, and assume that S1 and S2 are closed and compact. Then, S1 + S2 is closed and compact. Proof: See [157, p. 34]. Fact 10.9.7. Let S1, S2 , S3 ⊆ Fn, assume that S1, S2 , and S3 are closed and convex, assume that S1 ∩ S2 = ∅, S2 ∩ S3 = ∅, and S3 ∩ S1 = ∅, and assume that S1 ∪ S2 ∪ S3 is convex. Then, S1 ∩ S2 ∩ S3 = ∅. Proof: See [157, p. 32]. Fact 10.9.8. Let S1, S2 , S3 ⊆ Fn, assume that S1 and S2 are convex, S2 is closed, and S3 is bounded, and assume that S1 + S3 ⊆ S2 + S3 . Then, S1 ⊆ S2 . Proof: See [243, p. 5]. Remark: This result is due to Radstrom. Fact 10.9.9. Let S ⊆ Fm, assume that S is closed, let A ∈ Fn×m, and assume that A has full row rank. Then, AS is not necessarily closed. Remark: See Theorem 10.3.6. Fact 10.9.10. Let A be a collection of open subsets of Rn. Then, the union of all elements of A is open. If, in addition, A is finite, then the intersection of all elements of A is open. Proof: See [71, p. 50]. Fact 10.9.11. Let A be a collection of closed subsets of Rn. Then, the intersection of all elements of A is closed. If, in addition, A is finite, then the union of all elements of A is closed. Proof: See [71, p. 50]. Fact 10.9.12. Let A = {A1, A2 , . . .} be a collection of nonempty, closed subsets of Rn such that A1 is bounded and such that, for all i ∈ P, Ai+1 ⊆ Ai . Then, ∩∞ i=1 Ai is closed and nonempty. Proof: See [71, p. 56]. Remark: This result is the Cantor intersection theorem.

697

FUNCTIONS OF MATRICES AND THEIR DERIVATIVES

Fact 10.9.13. Let  ·  be a norm on Fn, let S ⊂ Fn, assume that S is a subspace, let y ∈ Fn, and define 

μ=

max

x∈{z∈S: z=1}

|y ∗x|.

Then, there exists a vector w ∈ S⊥ such that max

x∈{z∈Fn : z=1}

|(y + w)∗x| = μ.

Proof: See [1261, p. 57]. Remark: This result is a version of the Hahn-Banach theorem. Problem: Find a simple interpretation in R2. Fact 10.9.14. Let S ⊂ Rn, assume that S is a convex cone, let x ∈ Rn, and assume that x ∈ int S. Then, there exists a nonzero vector λ ∈ Rn such that λTx ≤ 0 and λTz ≥ 0 for all z ∈ S. Remark: This result is a separation theorem. See [904, p. 37], [1123, p. 443], [1161, pp. 95–101], and [1266, pp. 96–100]. Fact 10.9.15. Let S1, S2 ⊂ Rn, and assume that S1 and S2 are convex. Then, the following statements are equivalent: i) There exist a nonzero vector λ ∈ Rn and α ∈ R such that λTx ≤ α for all x ∈ S1, λTy ≥ α for all y ∈ S2 , and either S1 or S2 is not contained in the affine hyperplane {x ∈ Rn: λTx = α}. ii) intaff S1 S1 and intaff S2 S2 are disjoint. Proof: See [184, p. 82]. Remark: This result is a proper separation theorem. Fact 10.9.16. Let  ·  be a norm on Fn, let y ∈ Fn, let S ⊆ Fn, and assume that S is nonempty and closed. Then, there exists a vector x0 ∈ S such that y − x0  = min y − x. x∈S

Now, assume in addition that S is convex. Then, there exists a unique vector x0 ∈ S such that y − x0  = min y − x. x∈S

In other words, there exists a vector x0 ∈ S such that, for all x ∈ S\{x0 }, y − x0  < y − x. Proof: See [459, pp. 470, 471]. Remark: See Fact 10.9.18.

698

CHAPTER 10

Fact 10.9.17. Let · be a norm on Fn, let y1, y2 ∈ Fn, let S ⊆ Fn, assume that S is nonempty, closed, and convex, and let x1 and x2 denote the unique elements of S that are closest to y1 and y2 , respectively. Then, x1 − x2  ≤ y1 − y2 . Proof: See [459, pp. 474, 475]. Fact 10.9.18. Let S ⊆ Rn, assume that S is a subspace, let A ∈ Fn×n be the projector onto S, and let x ∈ Fn. Then, min x − y2 = A⊥x2 . y∈S

Proof: See [550, p. 41] or [1261, p. 91]. Remark: See Fact 10.9.16. Fact 10.9.19. Let S1, S2 ⊆ Rn, assume that S1 and S2 are subspaces, let A1 and A2 be the projectors onto S1 and S2 , respectively, and define *  max dist(S1, S2 ) =

Then,

max min x − y2 , max min x − y2 .

x∈S1 x =1

y∈S2

y∈S2 y 2 =1

x∈S1

dist(S1, S2 ) = σmax (A1 − A2 ).

If, in addition, dim S1 = dim S2 , then dist(S1, S2 ) = sin θ, where θ is the minimal principal angle defined in Fact 5.11.39. Proof: See [574, Chapter 13] and [1261, pp. 92, 93]. Remark: If  ·  is a norm on Fn×n, then 

dist(S1, S2 ) = A1 − A2 2 defines a metric on the set of all subspaces of Fn, yielding the gap topology. Remark: See Fact 5.12.17.

10.10 Facts on Matrix Functions Fact 10.10.1. Let A ∈ Cn×n, and assume that A is group invertible and has no eigenvalues in (−∞, 0). Then, A1/2 = Proof: See [701, p. 133].

2 πA

∞ 0

(t2I + A)−1 dt.

FUNCTIONS OF MATRICES AND THEIR DERIVATIVES

699

Fact 10.10.2. Let A ∈ Cn×n, and assume that A has no eigenvalues on the imaginary axis. Then, the following statements hold: i) Sign(A) is involutory. ii) A = Sign(A) if and only if A is involutory. iii) [A, Sign(A)] = 0. iv) Sign(A) = Sign(A−1 ). v) If A is real, then Sign(A) is real. vi) Sign(A) = A(A2 )−1/2 = A−1 (A2 )1/2. vii) Sign(A) is given by Sign(A) =

2 πA

∞

(t2I + A2 )−1 dt.

0

Proof: See [701, pp. 39, 40 and Chapter 5] and [826]. Remark: The square root in vi) is the principal square root. Fact 10.10.3. Let A, B ∈ Cn×n, assume that AB has no eigenvalues on the  imaginary axis, and define C = A(BA)−1/2. Then,     0 C 0 A = . Sign C −1 0 B 0 If, in addition, A has no eigenvalues on the imaginary axis, then     0 A1/2 0 A = . Sign I 0 A−1/2 0 Proof: Use vi) of Fact 10.10.2. See [701, p. 108]. Remark: The square root is the principal square root. Fact 10.10.4. Let A, B ∈ Cn×n, and assume that A and B are positive definite. Then,     0 A#B 0 B = . Sign (A#B)−1 A−1 0 0 Proof: See [701, p. 131]. Remark: The geometric mean is defined in Fact 8.10.43.

10.11 Facts on Functions n Fact 10.11.1. Let (xi )∞ i=1 ⊂ F . Then, limi→∞ xi = x if and only if, for all j ∈ {1, . . . , n}, limi→∞ xi(j) = x(j) .

700

CHAPTER 10

Fact 10.11.2. Let p ∈ C[s], where p(s) = sn + an−1sn−1 + · · · + a0 , let roots(p) = {λ1, . . . , λr }, and, for all i ∈ {1, . . . , r}, let αi ∈ R satisfy 0 < αi < minj=i |λi − λj |. Furthermore, for all ε0 , . . . , εn−1 ∈ R, define 

pε0 ,...,εn−1 (s) = sn + (an−1 + εn−1 )sn−1 + · · · + (a1 + ε1 )s + a0 + ε0 . Then, there exists ε > 0 such that, for all ε0 , . . . , εn−1 satisfying |εi | < ε for all i ∈ {1, . . . , n − 1}, it follows that, for all i ∈ {1, . . . , r}, the polynomial pε0 ,...,εn−1 has exactly multp (λi ) roots in the open disk {s ∈ C: |s − λi | < αi }. Proof: See [1030]. Remark: This result shows that the roots of a polynomial are continuous functions of the coefficients. Remark: λ1, . . . , λr are the distinct roots of p. Fact 10.11.3. Let S1 ⊆ Fn, assume that S1 is compact, let S2 ⊂ Fm, let f : S1 × S2 → R, and assume that f is continuous. Then, g : S2 → R defined by  g(y) = maxx∈S1 f(x, y) is continuous. Remark: A related result is given in [454, p. 208]. Fact 10.11.4. Define f : R2 → R by * min{x/y, y/x}, x > 0 and y > 0,  f (x, y) = 0, otherwise.  Furthermore, for all x ∈ R, define gx : R → R by gx (y) = f (x, y), and, for all y ∈ R,  define hy : R → R by hy (x) = f (x, y). Then, the following statements hold:

i) For all x ∈ R, gx is continuous. ii) For all y ∈ R, hy is continuous. iii) f is not continuous at (0, 0). Fact 10.11.5. Let S ⊆ Fn, assume that S is pathwise connected, let f : S → Fn, and assume that f is continuous. Then, f (S) is pathwise connected. Proof: See [1287, p. 65]. Fact 10.11.6. Let f : [0, ∞) → R, assume that f is continuous, and assume that limt→∞ f(t) exists. Then, t 1 t→∞ t

lim

f(τ ) dτ = lim f(t). t→∞

0

Remark: The assumption that f is continuous can be weakened. Fact 10.11.7. Let I ⊆ R be a finite or infinite interval, let f : I → R, assume that f is continuous, and assume that, for all x, y ∈ I, it follows that f [ 12 (x + y)] ≤ 1 2 f (x + y). Then, f is convex. Proof: See [1066, p. 10].

701

FUNCTIONS OF MATRICES AND THEIR DERIVATIVES

Remark: This result is due to Jensen. Remark: See Fact 1.10.4. Fact 10.11.8. Let A0 ∈ Fn×n, let  ·  be a norm on Fn×n, and let ε > 0. Then, there exists δ > 0 such that, if A ∈ Fn×n and A − A0  < δ, then dist[mspec(A) − mspec(A0 )] < ε, where



dist[mspec(A) − mspec(A0 )] = min max |λσ(i)(A) − λi (A0 )| σ

i=1,...,n

and the minimum is taken over all permutations σ of {1, . . . , n}. Proof: See [708, p. 399]. Fact 10.11.9. Let I ⊆ R be an interval, let A: I → Fn×n, and assume that A is continuous. Then, for i = 1, . . . , n, there exist continuous functions λi : I → C such that, for all t ∈ I, mspec(A(t)) = {λ1(t), . . . , λn(t)}ms . Proof: See [708, p. 399]. Remark: The spectrum cannot always be continuously parameterized by more than one variable. See [708, p. 399]. Fact 10.11.10. Let D ⊆ Rm, assume that D is a convex set, and let f : D → R. Then, f is convex if and only if the set {(x, y) ∈ Rn × R: y ≥ f(x)} is convex. Fact 10.11.11. Let D ⊆ Rm, assume that D is a convex set, let f : D → R, and assume that f is convex. Then, f is continuous on intaff D D. Fact 10.11.12. Let D ⊆ Rm, assume that D is a convex set, let f : D → R, and assume that f is convex. Then, f −1((−∞, α]) = {x ∈ D: f(x) ≤ α} is convex.

10.12 Facts on Derivatives Fact 10.12.1. Let p ∈ C[s]. Then, roots(p ) ⊆ co roots(p). Proof: See [459, p. 488]. Remark: p is the derivative of p. Fact 10.12.2. Let f : R2 → R, g : R → R, and h: R → R. Then, assuming each of the following integrals exists, h(α)

d dα

h(α)

∂ f(t, α) dt. ∂α



f(t, α) dt = f(h(α), α)h (α) − f(g(α), α)g (α) +

g(α)

Remark: This equality is Leibniz’s rule.

g(α)

702

CHAPTER 10

Fact 10.12.3. Let D ⊆ Rm, assume that D is open and convex, let f : D → R, and assume that f is C1 on D. Then, the following statements hold: i) f is convex if and only if, for all x, y ∈ D, f (x) + (y − x)Tf (x) ≤ f (y). ii) f is strictly convex if and only if, for all distinct x, y ∈ D, f (x) + (y − x)Tf (x) < f (y). Remark: If f is not differentiable, then these inequalities can be stated in terms of directional differentials of f or the subdifferential of f . See [1066, pp. 29–31, 128–145]. Fact 10.12.4. Let f : D ⊆ Fm → Fn, and assume that D+f(0; ξ) exists. Then, for all β > 0, D+f(0; βξ) = βD+f(0; ξ).  |x|. Then, for all ξ ∈ R, Fact 10.12.5. Define f : R → R by f(x) =

D+f(0; ξ) = |ξ|. √  Now, define f : Rn → Rn by f(x) = xTx. Then, for all ξ ∈ Rn,  D+f(0; ξ) = ξ Tξ. Fact 10.12.6. Let A, B ∈ Fn×n. Then, for all s ∈ F,

Hence,

d (A + sB)2 = AB + BA + 2sB. ds . . d 2. (A + sB) . = AB + BA. ds s=0

Furthermore, for all k ≥ 1,

. k−1  . d k. (A + sB) . = AiBAk−1−i. ds s=0 i=0

 Fact 10.12.7. Let A, B ∈ Fn×n, and let D = {s ∈ F: det(A + sB) = 0}. Then, for all s ∈ D,

d (A + sB)−1 = −(A + sB)−1B(A + sB)−1. ds Hence, if A is nonsingular, then . . d (A + sB)−1 .. = −A−1BA−1. ds s=0

FUNCTIONS OF MATRICES AND THEIR DERIVATIVES

703

Fact 10.12.8. Let D ⊆ F, let A: D −→ Fn×n, and assume that A is differentiable. Then,      n d d A d A 1 det Ai(s), det A(s) = tr A (s) A(s) = n−1 tr A(s) A (s) = ds ds ds i=1 where Ai(s) is obtained by differentiating the entries of the ith row of A(s). If, in addition, A(s) is nonsingular for all s ∈ D, then   d d −1 log det A(s) = tr A (s) A(s) . ds ds If A(s) is positive definite for all s ∈ D, then

  d d 1/n 1/n −1 1 det A (s) = n[det A (s)] tr A (s) A(s) . ds ds

Finally, if A(s) is nonsingular and has no negative eigenvalues for all s ∈ D, then   d d 2 −1 log A(s) = 2 tr [log A(s)]A (s) A(s) ds ds and

d log A(s) = ds

1

[(A(s) − I)t + I]−1

0

d A(s)[(A(s) − I)t + I]−1 dt. ds

Proof: See [367, p. 267], [577], [1039], [1125, pp. 199, 212], [1157, p. 430], and [1214]. Remark: See Fact 11.13.4. Fact 10.12.9. Let D ⊆ F, let A: D −→ Fn×n, assume that A is differentiable, and assume that A(s) is nonsingular for all x ∈ D. Then,   d −1 d A (s) = −A−1(s) A(s) A−1(s) ds ds and

    d −1 d −1 tr A (s) A(s) = − tr A(s) A (s) . ds ds

Proof: See [730, p. 491] and [1125, pp. 198, 212]. Fact 10.12.10. Let A, B ∈ Fn×n. Then, for all s ∈ F,

d det(A + sB) = tr B(A + sB)A . ds . Hence, n    . d i A . det(A + sB). = tr BA = det A ← coli(B) . ds s=0 i=1 Proof: Use Fact 2.16.9 and Fact 10.12.8. Remark: This result generalizes Lemma 4.4.8.

704

CHAPTER 10

Fact 10.12.11. Let A ∈ Fn×n, r ∈ R, and k ≥ 1. Then, for all s ∈ C, dk [det(I + sA)]r = (rtr A)k [det(I + sA)]r. dsk . . dk r. [det(I + sA)] . = (rtr A)k. k ds s=0

Hence,

Fact 10.12.12. Let A ∈ Rn×n, assume that A is symmetric, let X ∈ Rm×n, and assume that XAXT is nonsingular. Then,      −1 d det XAXT = 2 det XAXT ATXT XAXT . dX Proof: See [358].

10.13 Facts on Infinite Series Fact 10.13.1. The following infinite series converge for A ∈ Fn×n with the given bounds on sprad(A): i) For all A ∈ Fn×n, sin A = A − ii) For all A ∈ F

+

1 5 5! A

−

1 7 7! A

+ ··· .

1 2 2! A

+

1 4 4! A

−

1 6 6! A

+ ··· .

n×n

, cos A = I −

iii) For all A ∈ F

1 3 3! A

n×n

such that sprad(A) < π/2,

tan A = A + 13 A3 + iv) For all A ∈ F

n×n

5 2 15 A

+

7 17 315 A

+

9 62 2835 A

+ ··· .

such that sprad(A) < 1, A

e =I +A+

1 2 2! A

+

1 3 3! A

+

1 4 4! A

+ ··· .

v) For all A ∈ Fn×n such that sprad(A − I) < 1,

log A = − I − A + 12 (I − A)2 + 13 (I − A)3 + 14 (I − A)4 + · · · . vi) For all A ∈ Fn×n such that sprad(A) < 1,   log(I − A) = − A + 12 A2 + 13 A3 + 14 A4 + · · · . vii) For all A ∈ Fn×n such that sprad(A) < 1, log(I + A) = A − 12 A2 + 13 A3 − 14 A4 + · · · . viii) For all A ∈ Fn×n such that spec(A) ⊂ ORHP, log A =

∞  i=0

ix) For all A ∈ F

2i+1 2 (A − I)(A + I)−1 . 2i + 1

n×n

,

sinh A = sin jA = A +

1 3 3! A

+

1 5 5! A

+

1 7 7! A

+ ··· .

705

FUNCTIONS OF MATRICES AND THEIR DERIVATIVES

x) For all A ∈ Fn×n, cosh A = cos jA = I + xi) For all A ∈ F

n×n

1 2 2! A

+

1 4 4! A

+

1 6 6! A

+ ··· .

such that sprad(A) < π/2,

tanh A = tan jA = A − 13 A3 + xii) Let α ∈ R. For all A ∈ F

n×n

5 2 15 A

−

7 17 315 A

+

9 62 2835 A

− ··· .

such that sprad(A) < 1,

(I + A)α = I + αA + α(α−1) A2 + α(α−1)(α−2) A3 + 14 A4 + · · · 2! 3!         = I + α1 A + α2 A2 + α3 A3 + α4 A4 + · · · . xiii) For all A ∈ Fn×n such that sprad(A) < 1, (I − A)−1 = I + A + A2 + A3 + A4 + · · · . Proof: See Fact 1.20.8. Remark: The coefficients in iii) can be expressed in terms of Bernoulli numbers. See [772, p. 129].

10.14 Notes An introductory treatment of limits and continuity is given in [1057]. The derivative and the directional differential are typically called the Fr´echet derivative and the Gˆ ateaux differential, respectively [509]. Differentiation of matrix functions is considered in [671, 973, 1000, 1116, 1164, 1213]. In [1161, 1162] the set intaff S S is called the relative interior of S. An extensive treatment of matrix functions is given in Chapter 6 of [730]; see also [735]. The identity theorem is discussed in [762]. A chain rule for matrix functions is considered in [973, 1005]. Differentiation with respect to complex matrices is discussed in [798]. Extensive tables of derivatives of matrix functions are given in [382, pp. 586–593].

Chapter Eleven

The Matrix Exponential and Stability Theory

The matrix exponential function is fundamental to the study of linear ordinary differential equations. This chapter focuses on the properties of the matrix exponential as well as on stability theory.

11.1 Definition of the Matrix Exponential The scalar initial value problem x(t) ˙ = ax(t),

(11.1.1)

x(0) = x0 ,

(11.1.2)

where t ∈ [0, ∞) and a, x(t) ∈ R, has the solution x(t) = eatx0 ,

(11.1.3)

where t ∈ [0, ∞). We are interested in systems of linear differential equations of the form x(t) ˙ = Ax(t), (11.1.4) x(0) = x0 ,

(11.1.5)

˙ denotes dx(t) where t ∈ [0, ∞), x(t) ∈ Rn, and A ∈ Rn×n. Here x(t) dt , where the derivative is one sided for t = 0 and two sided for t > 0. The solution of (11.1.4), (11.1.5) is given by x(t) = etA x0 , (11.1.6) where t ∈ [0, ∞) and etA is the matrix exponential. The following definition is based on (10.5.2). Definition 11.1.1. Let A ∈ Fn×n. Then, the matrix exponential eA ∈ Fn×n or exp(A) ∈ Fn×n is the matrix  eA =

∞  k=0



Note that 0! = 1 and e0n×n = In.

1 k k! A .

(11.1.7)

708

CHAPTER 11

Proposition 11.1.2. Let A ∈ Fn×n. Then, the following statements hold: i) The series (11.1.7) converges absolutely. ii) The series (11.1.7) converges to eA. iii) Let  ·  be a normalized submultiplicative norm on Fn×n. Then, e−A ≤ eA  ≤ eA.

(11.1.8)

Proof. To prove i), let  ·  be a normalized submultiplicative norm on Fn×n. Then, for all k ≥ 1, k k   i i A 1 1 A  ≤ . i! i!A ≤ e  k

i=0

1 i i=0 i!A 

i=0

∞

of partial sums is increasing and bounded,  1 i there exists 0 such that the series ∞ i=0 i!A  converges to α. Hence, the ∞ 1α > i series i=0 i! A converges absolutely. Since the sequence

k=0

Next, ii) follows from i) using Proposition 10.2.9. Next, we have

.. .. ∞ ∞ ∞ ..  ..  . . A i i A 1 i .. 1 1 ≤ , e  = .. i! A .... i! A  ≤ i! A = e .. i=0

i=0

i=0

which proves the second inequality in (11.1.8). Finally, note that 1 ≤ eA e−A ≤ eA eA , and thus

e−A ≤ eA .



The following result generalizes the well-known corresponding scalar result. Proposition 11.1.3. Let A ∈ Fn×n. Then,  k eA = lim I + k1A .

(11.1.9)

k→∞

Proof. It follows from the binomial theorem that k  k  I + k1 A = αi(k)Ai, i=0

  k! 1 1 k . = i αi(k) = i k i k i!(k − i)!

where



For all i ∈ P, it follows that αi(k) → 1/i! as k → ∞. Hence, k ∞    k A 1 i lim I + k1 A = lim αi(k)Ai = i! A = e .

k→∞

k→∞

i=0

i=0



709

THE MATRIX EXPONENTIAL AND STABILITY THEORY

Proposition 11.1.4. Let A ∈ Fn×n. Then, for all t ∈ R, t

e

tA

− I = AeτA dτ

(11.1.10)

0

and

d tA e = AetA. dt

(11.1.11)

Proof. Note that t

Ae

τA

dτ =

t ∞ 

1 k k+1 k! τ A

dτ =

0 k=0

0

∞ 

1 tk+1 k+1 k! k+1 A

= etA − I,

k=0

which yields (11.1.10), while differentiating (11.1.10) with respect to t yields (11.1.11). Proposition 11.1.5. Let A, B ∈ Fn×n. Then, AB = BA if and only if, for all t ∈ [0, ∞), etAetB = et(A+B). (11.1.12) Proof. Suppose that AB = BA. By expanding etA, etB , and et(A+B), it can be seen that the expansions of etAetB and et(A+B) are identical. Conversely, differentiating (11.1.12) twice with respect to t and setting t = 0 yields AB = BA. Corollary 11.1.6. Let A, B ∈ Fn×n, and assume that AB = BA. Then, eAeB = eBeA = eA+B.

(11.1.13)

0 π  The converse of Corollary 11.1.6 is not true. For example, if A = −π 0 and   √  0 (7+4 3)π , then eA = eB = −I and eA+B = I, although AB = BA. B = (−7+4√3)π 0 A partial converse is given by Fact 11.14.1. Proposition 11.1.7. Let A ∈ Fn×n and B ∈ Fm×m. Then, eA⊗Im = eA ⊗ Im ,

(11.1.14)

eIn⊗B = In ⊗ eB,

(11.1.15)

eA⊕B = eA ⊗ eB.

(11.1.16)

Proof. Note that 1 (A ⊗ Im )2 + · · · eA⊗Im = Inm + A ⊗ Im + 2!

= In ⊗ Im + A ⊗ Im + = (In + A + = e ⊗ Im A

1 2 2!A

2 1 2! (A

⊗ Im ) + · · ·

+ · · · ) ⊗ Im

710

CHAPTER 11

and similarly for (11.1.15). To prove (11.1.16), note that (A ⊗ Im )(In ⊗ B) = A ⊗ B and (In ⊗B)(A⊗Im ) = A⊗B, which shows that A⊗Im and In ⊗B commute. Thus, by Corollary 11.1.6,    eA⊕B = eA⊗Im +In⊗B = eA⊗ImeIn⊗B = eA ⊗ Im In ⊗ eB = eA ⊗ eB. 

11.2 Structure of the Matrix Exponential To elucidate the structure of the matrix exponential, recall that, by Theorem  4.6.1, every term Ak in (11.1.7) for k > r = deg μA can be expressed as a linear combination of I, A, . . . , Ar−1. The following result provides an expression for etA in terms of I, A, . . . , Ar−1. Proposition 11.2.1. Let A ∈ Fn×n. Then, for all t ∈ R, E n−1  1 etA = j2π (zI − A)−1etz dz = ψi(t)Ai, C

(11.2.1)

i=0

where, for all i ∈ {0, . . . , n − 1}, ψi(t) is given by E [i+1] χA (z) tz  1 e dz, ψi(t) = j2π C χA(z)

(11.2.2)

where C is a simple, closed contour in the complex plane enclosing spec(A), χA(s) = sn + βn−1sn−1 + · · · + β1s + β0 , [1]

(11.2.3)

[n]

and the polynomials χA , . . . , χA are defined by the recursion [i+1]

sχA [0]  χA =

χA and where ψi(t) satisfies (n)

[i]

(s) = χA (s) − βi ,

[n] χA (s)

i = 0, . . . , n − 1,

= 1. Furthermore, for all i ∈ {0, . . . , n − 1} and t ≥ 0, 

(n−1)

ψi (t) + βn−1ψi

(t) + · · · + β1ψi(t) + β0 ψi(t) = 0,

(11.2.4)

where, for all i, j ∈ {0, . . . , n − 1}, (j)

ψi (0) = δij .

(11.2.5)

Proof. See [583, p. 381], [913, 954], [1490, p. 31], and Fact 4.9.11. The coefficient ψi(t) of Ai in (11.2.1) can be further characterized in terms of the Laplace transform. Define ∞ 

e−stx(t) dt.



x ˆ(s) = L{x(t)} =

(11.2.6)

0

Note that and

L{x(t)} ˙ = sˆ x(s) − x(0)

(11.2.7)

ˆ(s) − sx(0) − x(0). ˙ L{¨ x(t)} = s2x

(11.2.8)

THE MATRIX EXPONENTIAL AND STABILITY THEORY

711

The following result shows that the resolvent of A is the Laplace transform of the exponential of A. See (4.4.23). Proposition 11.2.2. Let A ∈ Fn×n, and define ψ0 , . . . , ψn−1 as in Proposition 11.2.1. Then, for all s ∈ C\spec(A), 7 8 L etA =

∞

e−stetA dt = (sI − A)−1.

(11.2.9)

0

Furthermore, for all i ∈ {0, . . . , n −1}, the Laplace transform ψˆi(s) of ψi(t) is given by [i+1] (s) χ (11.2.10) ψˆi(s) = A χA(s) and (sI − A)−1 =

n−1 

ψˆi(s)Ai.

(11.2.11)

i=0

Proof. Let s ∈ C satisfy Re s > spabs(A) so that A − sI is asymptotically stable. Thus, it follows from Lemma 11.9.2 that 7 8 L etA =

∞

∞

e 0

−st tA

e

et(A−sI) dt = (sI − A)−1.

dt = 0

7 8 By analytic continuation, the expression L etA is given by (11.2.9) for all s ∈ C\spec(A). Comparing (11.2.11) with the expression for (sI −A)−1 given by (4.4.23) shows that there exist B0 , . . . , Bn−2 ∈ Fn×n such that n−1 

sn−2 sn−1 s 1 I+ Bn−2 + · · · + B1 + B0 . ψˆi(s)Ai = χA(s) χA(s) χA(s) χA(s) i=0

(11.2.12)

To further illustrate the structure of etA, where A ∈ Fn×n, let A = SBS −1, where B = diag(B1, . . . , Bk ) is the Jordan form of A. Hence, by Proposition 11.2.8,

where

etA = SetBS −1,

(11.2.13)

  etB = diag etB1, . . . , etBk .

(11.2.14)

The structure of e can thus be determined by considering the block Bi ∈ Fαi ×αi , which, for all i ∈ {1, . . . , k} has the form tB

Bi = λiIαi + Nαi .

(11.2.15)

Since λiIαi and Nαi commute, it follows from Proposition 11.1.5 that etBi = et(λiIαi +Nαi ) = eλi tIαi etNαi = eλitetNαi.

(11.2.16)

712

CHAPTER 11

Since Nααii = 0, it follows that etNαi is a finite sum of powers of tNαi . Specifically, 1 tαi −1Nααii −1, (αi −1)!

etNαi = Iαi + tNαi + 12 t2Nα2i + · · · + and thus

⎡

etNαi

⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ =⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣

1

t

t2 2

···

tαi −2 (αi −2)!

tαi −1 (αi −1)!

..

.

tαi −3 (αi −3)!

tαi −2 (αi −2)!

..

.

tαi −4 (αi −4)!

tαi −3 (αi −3)!

0

1

t

0 .. .

0 .. .

1 .. .

..

.

..

.

.. .

0

0

0

..

.

1

t

0

0

0

···

0

1

(11.2.17)

⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥, ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦

(11.2.18)

which is upper triangular and Toeplitz (see Fact 11.13.1). Alternatively, (11.2.18) follows from (10.5.5) with f (s) = est. Note that (11.2.16) follows from (10.5.5) with f(λ) = eλt. Furthermore, every 1 r λit entry of etBi is of the form r! t e , where r ∈ {0, αi − 1} and λi is an eigenvalue of A. Reconstructing A by means of A = SBS −1 shows that every entry of A is a linear combination of the entries of the blocks etBi. If A is real, then etA is also real. Thus, the term eλit for complex λi = νi + jωi ∈ spec(A), where νi and ωi are real, yields terms of the form eνit cos ωi t and eνit sin ωi t. The following result follows from (11.2.18) or Corollary 10.5.4. Proposition 11.2.3. Let A ∈ Fn×n. Then, 8   7 mspec eA = eλ : λ ∈ mspec(A) ms.

(11.2.19)

Proof. It can be seen that every diagonal entry of the Jordan form of eA is of the form eλ, where λ ∈ spec(A). Corollary 11.2.4. Let A ∈ Fn×n. Then, det eA = etr A.

(11.2.20)

Corollary 11.2.5. Let A ∈ Fn×n, and assume that tr A = 0. Then, det eA = 1. Corollary 11.2.6. Let A ∈ Fn×n. Then, the following statements hold: i) If eA is unitary, then, spec(A) ⊂ jR. ii) spec(eA ) is real if and only if Im spec(A) ⊂ πZ. Proposition 11.2.7. Let A ∈ Fn×n. Then, the following statements hold: i) A and eA have the same number of Jordan blocks of corresponding sizes. ii) eA is semisimple if and only if A is semisimple.

713

THE MATRIX EXPONENTIAL AND STABILITY THEORY

iii) If μ ∈ spec(eA ), then



amult exp(A)(μ) =

amultA(λ)

(11.2.21)

{λ∈spec(A): eλ =μ}

and



gmult exp(A)(μ) =

gmultA(λ).

(11.2.22)

{λ∈spec(A): eλ =μ}

iv) If eA is simple, then A is simple. v) If eA is cyclic, then A is cyclic. vi) eA is a scalar multiple of the identity matrix if and only if A is semisimple and every pair of eigenvalues of A differs by an integer multiple of j2π. vii) eA is a real scalar multiple of the identity matrix if and only if A is semisimple, every pair of eigenvalues of A differs by an integer multiple of j2π, and the imaginary part of every eigenvalue of A is an integer multiple of jπ. Proof. To prove i), note that, for all t = 0, def(etNαi − Iαi ) = 1, and thus the geometric multiplicity of (11.2.18) is 1. Since (11.2.18) has one distinct eigenvalue, it follows that (11.2.18) is cyclic. Hence, by Proposition 5.5.14, (11.2.18) is similar to a single Jordan block. Now, i) follows by setting t = 1 and applying this argument to each Jordan block of A. Statements ii)–v) follow by similar arguments. To prove vi), note that, for all λi , λj ∈ spec(A), it follows that eλi = eλj. Furthermore, since A is semisimple, it follows from ii) that eA is also semisimple. Since all of the eigenvalues of eA are equal, it follows that eA is a scalar multiple of the identity matrix. Finally, vii) is an immediate consequence of vii). Proposition 11.2.8. Let A ∈ Fn×n. Then, the following statements hold:  T T i) eA = eA .   ii) eA = eA.  ∗ ∗ iii) eA = eA .  −1 = e−A. iv) eA is nonsingular, and eA v) If S ∈ Fn×n is nonsingular, then eSAS

−1

= SeAS −1.

ni ×ni for all i ∈ {1, . . . , k}, then vi) If A = diag(A  A 1, . . . ,AAk ), where Ai ∈ F A 1 k e = diag e , . . . , e .

vii) If A is Hermitian, then eA is positive definite. Furthermore, the following statements are equivalent: viii) A is normal. ∗

∗

ix) tr eA eA = tr eA A∗ A

x) e e = e

A∗ +A

.

+A

.

714

CHAPTER 11 ∗

∗

∗

xi) eAeA = eA eA = eA

+A

.

Finally, the following statements hold: xii) If A is normal, then eA is normal. xiii) eA is normal if and only if A is unitarily similar to a block-diagonal matrix diag(A1, . . . , Ak ) such that, for all i ∈ {1, . . . , k}, Ai is semisimple and each pair of eigenvalues of Ai differ by an integer multiple of j2π, and, for all distinct i, j ∈ {1, . . . , k}, spec(eAi ) = spec(eAj ). xiv) If eA is normal and no pair of eigenvalues of A differ by an integer multiple of j2π, then A is normal. xv) A is skew Hermitian if and only if A is normal and eA is unitary. xvi) If F = R and A is skew symmetric, then eA is orthogonal and det eA = 1. xvii) If eA is unitary, then either A is skew Hermitian or at least two eigenvalues of A differ by a nonzero integer multiple of j2π. xviii) eA is unitary if and only if A is unitarily similar to a block-diagonal matrix diag(A1, . . . , Ak ) such that, for all i ∈ {1, . . . , k}, Ai is semisimple, every eigenvalue of Ai is imaginary, and each pair of eigenvalues of Ai differ by an integer multiple of j2π, and, for all distinct i, j ∈ {1, . . . , k}, spec(eAi ) = spec(eAj ). xix) eA is Hermitian if and only if A is unitarily similar to a block-diagonal matrix diag(A1, . . . , Ak ) such that, for all i ∈ {1, . . . , k}, Ai is semisimple, the imaginary part of every eigenvalue of Ai is an integer multiple of πj, and each pair of eigenvalues of Ai differ by an integer multiple of j2π, and, for all distinct i, j ∈ {1, . . . , k}, spec(eAi ) = spec(eAj ). xx) eA is positive definite if and only if A is unitarily similar to a block-diagonal matrix diag(A1, . . . , Ak ) such that, for all i ∈ {1, . . . , k}, Ai is semisimple, the imaginary part of every eigenvalue of Ai is an integer multiple of 2πj, and each pair of eigenvalues of Ai differ by an integer multiple of j2π, and, for all distinct i, j ∈ {1, . . . , k}, spec(eAi ) = spec(eAj ). Proof. The equivalence of viii) and ix) is given in [465, 1239], while the equivalence of viii) and xi) is given in [1202]. Note that xi) =⇒ x) =⇒ ix). Statement xii) follows from the fact that viii) =⇒ xi). Statement xiii) is given in [1503]. Statement xiv) is a consequence of xiii). To prove sufficiency in xv), note that ∗ ∗ eA+A = eAeA = eA (eA )∗ = I = e0. Since A + A∗ is Hermitian, it follows from iii) of Proposition 11.2.9 that A + A∗ = 0. The converse is immediate. To prove xvi), T note that eA (eA )T = eA eA = eA e−A = eA (eA )−1 = I, and, using Corollary 11.2.5, A tr A 0 det e = e = e = 1. To prove xvii), note that it follows from xiii) that, if every block Ai is scalar, then A is skew Hermitian, while, if at least one block Ai is not scalar, then A has at least two eigenvalues that differ by an integer multiple of j2π. Finally, xviii)–xx) are analogous to xiii).

 4π The converse of xii) is false. For example, the matrix A = −2π −2π 2π satisfies

1 A eA = I but is not normal. Likewise, A = jπ = −I but is not 0 −jπ satisfies e ∗ ∗ ∗ ∗ normal. For both matrices, eA eA = eAeA = I, but eA eA = eA +A , which confirms

715

THE MATRIX EXPONENTIAL AND STABILITY THEORY

that xi) does not hold. Both matrices have eigenvalues ±jπ. Proposition 11.2.9. The following statements hold: i) If A, B ∈ Fn×n are similar, then eA and eB are similar. ii) If A, B ∈ Fn×n are unitarily similar, then eA and eB are unitarily similar. iii) If B ∈ Fn×n is positive definite, then there exists a unique Hermitian matrix A ∈ Fn×n such that eA = B. iv) B ∈ Fn×n is Hermitian and nonsingular if and only if there exists a normal matrix A ∈ Cn×n such that, for all λ ∈ spec(A), Im λ is an integer multiple of jπ and eA = B. v) B ∈ Fn×n is normal and nonsingular if and only if there exists a normal matrix A ∈ Fn×n such that eA = B. vi) B ∈ Fn×n is unitary if and only if there exists a normal matrix A ∈ Cn×n such that mspec(A) ⊂ jR and eA = B. vii) B ∈ Fn×n is unitary if and only if there exists a skew-Hermitian matrix A ∈ Cn×n such that eA = B. viii) B ∈ Fn×n is unitary if and only if there exists a Hermitian matrix A ∈ Fn×n such that ejA = B. ix) B ∈ Rn×n is orthogonal and det B = 1 if and only if there exists a skewsymmetric matrix A ∈ Rn×n such that eA = B. x) If A and B are normal and eA = eB, then A + A∗ = B + B ∗. Proof. To prove iii), let B = S diag(b1, . . . , bn )S −1, where S ∈ Fn×n is unitary  and b1, . . . , bn are positive. Then, define A = S diag(log b1, . . . , log bn )S −1. Next, to ˆ prove uniqueness, let A and Aˆ be Hermitian matrices such that B = eA = eA. ˆ ˆ Then, for all t ≥ 0, it follows that etA = (eA )t = (eA )t = etA. Differentiating yields ˆ ˆ tA , while setting t = 0 implies that A = A. ˆ As another proof, it follows AetA = Ae ˆ ˆ ˆ from x) of Fact 11.14.1 that A and A commute. Therefore, I = eA e−A = eA−A, which, since A − Aˆ is Hermitian, implies that A − Aˆ = 0. As yet another proof, the result follows directly from xiii) of Fact 11.14.1. Finally, iii) is given by Theorem 4.4 of [1202]. Statement vii) is given by v) of Proposition 11.6.7. To prove x), note ∗ ∗ that eA+A = eB+B , which, by vii) of Proposition 11.2.8, is positive definite. The result now follows from iii). 



The converse of i) is false. For example, A = [ 00 00 ] and B = A e = eB = I, although A and B are not similar.



0 2π −2π 0

satisfy

11.3 Explicit Expressions In this section we present explicit expressions for the exponential of a general 2 × 2 real matrix A. Expressions are given in terms of both the entries of A and the eigenvalues of A.

716

CHAPTER 11

 Lemma 11.3.1. Let A = a0 db ∈ C2×2. Then,  ⎧  ⎪ 1 b ⎪ a ⎪ e , a = d, ⎪ ⎪ 0 1 ⎪ ⎨ ⎤ eA = ⎡ ⎪ ea −ed a ⎪ b e ⎪ a−d ⎪ ⎣ ⎦, a = d. ⎪ ⎪ ⎩ 0 ed

(11.3.1)

The following result gives an expression for eA in terms of the eigenvalues of A. Proposition 11.3.2. Let A ∈ C2×2, and let mspec(A) = {λ, μ}ms . Then, ⎧ λ = μ, ⎨eλ [(1 − λ)I + A], A e = (11.3.2) ⎩ μeλ −λeμ eμ −eλ I + A, λ =  μ. μ−λ μ−λ Proof. This result follows from Theorem 10.5.2. Alternatively, suppose

that λ = μ. Then, there exists a nonsingular matrix S ∈ C2×2 such that A = S λ0 αλ S −1, where α ∈ C. Hence, eA = eλS[ 10 α1 ]S −1 = eλ [(1 − λ)I + A]. Now, suppose

that λ = μ. Then, there exists a nonsingular matrix S ∈ C2×2 such that A = S λ0 μ0 S −1.

λ λ λ μ −λeμ −eλ λ 0 I + eμ−λ Hence, eA = S e0 e0μ S −1. Then, the equality e0 e0μ = μeμ−λ 0 μ yields the desired result. Next, we give an expression for eA in terms of the entries of A ∈ R2×2. a b   2×2 Corollary 11.3.3. Let A = , and define γ = (a − d)2 + 4bc and c d ∈ R  1 1/2 δ = 2 |γ| . Then, ⎧   b ⎪ cos δ + a−d sin δ sin δ a+d ⎪ 2δ δ ⎪e 2 ⎪ , γ < 0, ⎪ ⎪ c ⎪ cos δ − a−d ⎪ δ sin δ 2δ sin δ ⎪ ⎪ ⎪ ⎪   ⎪ ⎨ a+d b 1 + a−d A 2 e = e 2 (11.3.3) , γ = 0, ⎪ c 1 − a−d ⎪ 2 ⎪ ⎪ ⎪ ⎪   ⎪ ⎪ b ⎪ cosh δ + a−d ⎪ a+d 2δ sinh δ δ sinh δ ⎪ ⎪ 2 , γ > 0. ⎪ ⎩e c cosh δ − a−d δ sinh δ 2δ sinh δ √ √  1  1 Proof. The eigenvalues of A are λ = γ) and μ = γ). 2 (a + d − 2 (a + d + Hence, λ = μ if and only if γ = 0. The result now follows from Proposition 11.3.2. 

ν ω 2×2 Example 11.3.4. Let A = [ −ω . Then, ν] ∈R   cos ωt sin ωt . etA = eνt − sin ωt cos ωt

(11.3.4)

717

THE MATRIX EXPONENTIAL AND STABILITY THEORY 

ω On the other hand, if A = [ ων −ν ], then  cosh δt + νδ sinh δt etA = ω δ sinh δt √  ω 2 + ν 2. where δ =

Example 11.3.5. Let α ∈ F, ⎧ ⎪ 1 ⎪ ⎪ ⎪ ⎪ ⎨ 0 etA =  ⎪ ⎪ ⎪ 1 ⎪ ⎪ ⎩ 0



ω δ sinh δt

cosh δt − νδ sinh δt

,

(11.3.5)

 0 1 [ 0 α ]. Then, and define A =  α−1(eαt − 1) , α = 0, eαt  t , α = 0. 1

0 θ  Example 11.3.6. Let θ ∈ R, and define A = −θ 0 . Then,   cos θ sin θ . eA = − sin θ cos θ  0 π −θ   2 Furthermore, define B = . Then, −π 0 2 +θ   sin θ cos θ B . e = − cos θ sin θ Example 11.3.7. Consider the second-order mechanical vibration equation m¨ q + cq˙ + kq = 0,

(11.3.6)

where m is positive and c and k are nonnegative. Here m, c, and k denote mass, damping, and stiffness parameters, respectively. Equation (11.3.6) can be written in companion form as the system x˙ = Ax, where  x=



q q˙

The inelastic case k = 0 is case, ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ tA e =  ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩



 A=

,



(11.3.7) 0 1 −k/m −c/m

 .

(11.3.8)

the simplest one since A is upper triangular. In this 1 t 0 1 1 0

 ,

m c (1

k = c = 0, − e−ct/m )

e−ct/m



(11.3.9) , k = 0, c > 0,

where c = 0 and c > 0 correspond to a rigid body and a damped rigid body, respectively.

718

CHAPTER 11

Next, we consider the elastic case c ≥ 0 and k > 0. In this case, we define " k c   , ζ= √ , (11.3.10) ωn = m 2 mk where ωn > 0 denotes the (undamped) natural frequency of vibration and ζ ≥ 0 denotes the damping ratio. Now, A can be written as   0 1 A= , (11.3.11) −ωn2 −2ζωn and Corollary 11.3.3 yields (11.3.12) etA  ⎧ 1 ⎪ cos ωn t ⎪ ωn sin ωn t , ⎪ ζ = 0, ⎪ ⎪ −ωn sin ωn t cos ωn t ⎪ ⎪ ⎪ ⎪ ⎪ ⎡ ⎤ ⎪ ⎪ 1 ⎪ sin ω t cos ωd t + √ ζ 2 sin ωd t ⎪ d ⎪ ω d 1−ζ ⎪ ⎥ ⎪e−ζωn t ⎢ ⎪ ⎣ ⎦, 0 < ζ < 1, ⎪ −ω ⎪ ζ d ⎪ cos ωd t − √ 2 sin ωd t ⎪ 1−ζ 2 sin ωd t ⎨ 1−ζ =   ⎪ ⎪ 1 + ωnt t ⎪ ⎪ ⎪e−ωnt , ζ = 1, ⎪ ⎪ 2 ⎪ −ωn t 1 − ωnt ⎪ ⎪ ⎪ ⎪ ⎡ ⎤ ⎪ ⎪ ζ 1 ⎪ √ t + sinh ω t sinh ω t cosh ω ⎪ d d d ⎪ ωd ζ 2 −1 ⎪ ⎢ ⎥ ⎪ ⎪ e−ζωnt ⎣ ⎦, ζ > 1, ⎪ −ω ⎪ ζ d ⎩ √ cosh ωd t − sinh ω t d ζ 2 −1 sinh ωd t 2 ζ −1

where ζ = 0, 0 < ζ < 1, ζ = 1, and ζ > 1 correspond to undamped, underdamped, critically damped, and overdamped oscillators, respectively, and where the damped natural frequency ωd is the positive number ⎧  ⎨ω n 1 − ζ 2 , 0 < ζ < 1,  ωd = (11.3.13)  ⎩ ζ > 1. ωn ζ 2 − 1, Note that m and k are not integers here.

11.4 Matrix Logarithms Definition 11.4.1. Let A ∈ Cn×n. Then, B ∈ Cn×n is a logarithm of A if e = A. B

The following result shows that every complex, nonsingular matrix has a complex logarithm. Proposition 11.4.2. Let A ∈ Cn×n. Then, there exists a matrix B ∈ Cn×n such that A = eB if and only if A is nonsingular. Proof. See [639, pp. 35, 60] or [730, p. 474].

719

THE MATRIX EXPONENTIAL AND STABILITY THEORY

Although does 0 π the real number

not have a real logarithm, the real ma −1 −1 0 B trix B = −π satisfies e = 0 0 −1 . These examples suggest that not all real matrices have a real logarithm. Proposition 11.4.3. Let A ∈ Rn×n. Then, there exists a matrix B ∈ Rn×n such that A = eB if and only if A is nonsingular and, for every negative eigenvalue λ of A and for every positive integer k, the Jordan form of A has an even number of k × k blocks associated with λ. Proof. See [730, p. 475]. Replacing A and B in Proposition 11.4.3 by eA and A, respectively, yields the following result. Corollary 11.4.4. Let A ∈ Rn×n. Then, for every negative eigenvalue λ of eA and for every positive integer k, the Jordan form of eA has an even number of k × k blocks associated with λ.

 4π A Since the matrix A = −2π = I, it follows that a positive−2π 2π satisfies e definite matrix can have a logarithm that is not normal. However, the following result shows that every positive-definite matrix has exactly one Hermitian logarithm. Proposition 11.4.5. The function exp: Hn → Pn is one-to-one and onto. Proof. The result follows from vii) of Proposition 11.2.8 and iii) of Proposition 11.2.9. Let A ∈ Rn×n. If there exists a matrix B ∈ Rn×n such that A = eB, then Corollary 11.2.4 implies that det A = det eB = etr B > 0. However, the converse  0 is not true. Consider, for example, A = −1 0 −2 , which satisfies det A > 0. However, Proposition 11.4.3 implies that there does not exist a matrix B ∈ R2×2 such 0 π  and that A = eB. On the other hand, note that A = eBeC , where B = −π 0 0 0  C = 0 log 2 . While the product of two exponentials of real matrices has positive determinant, the following result shows that the converse is also true. Proposition 11.4.6. Let A ∈ Rn×n. Then, there exist matrices B, C ∈ Rn×n such that A = eBeC if and only if det A > 0. n×n such that A = eBeC. Then, Proof. that  Suppose   there exist B, C ∈ R B C det A = det e det e > 0. Conversely, suppose that det A > 0. If A has no negative eigenvalues, then it follows from Proposition 11.4.3 that there exists B ∈ Rn×n such that A = eB. Hence, A = eBe0n×n. Now, suppose that A has at least one negative eigenvalue. Then, Theorem 5.3.5 on the real Jordan form implies that there exist a nonsingular matrix ∈ Rn×n and matrices A1 ∈ Rn1 ×n1 and

S A1 0 n2 ×n2 −1 A2 ∈ R such that A = S 0 A2 S , where every eigenvalue of A1 is negative and where none of the eigenvalues of A2 are negative. Since det A and det  A2 −A1 0 −In1 0 S −1. are positive, it follows that n1 is even. Now, write A = S 0 In 2

0

A2

720

CHAPTER 11



−In1 0 0 In2



appears in an even number of 1 × 1 Jordan ˆ ∈ Rn×n such blocks, there exists a matrix B  it follows  from Proposition 11.4.3 that

−In1 0 ˆ −A 0 1 that = eB. Furthermore, since has no negative eigenvalues, 0 In2 0 A2 1 0 ˆ C it follows that there exists a matrix Cˆ ∈ Rn×n such that −A 0 A2 = e . Hence, Since the eigenvalue −1 of

ˆ ˆ

ˆ

−1

ˆ

−1

eA = SeBeCS −1 = eSBS eS CS .

Although eAeB may be different from eA+B , the following result, known as the Baker-Campbell-Hausdorff series, provides an expansion for a matrix function C(t) that satisfies eC(t) = etAetB. Proposition 11.4.7. Let A1, . . . , Al ∈ Fn×n. Then, there exists ε > 0 such that, for all t ∈ (−ε, ε), etA1 · · · etAl = eC(t), where 

C(t) =

l  i=1



1 2 2 t [Ai , Aj ] 1≤i 0, there exists a unique C1 solution x: [0, T ) → D satisfying (11.7.1). If xe ∈ D satisfies f(xe ) = 0, then x(t) ≡ xe is an equilibrium of (11.7.1). The following definition concerns the stability of an equilibrium of (11.7.1). Throughout this section,  ·  denotes a norm on Rn. Definition 11.7.1. Let xe ∈ D be an equilibrium of (11.7.1). Then, xe is Lyapunov stable if, for all ε > 0, there exists δ > 0 such that, if x(0) − xe  < δ, then x(t) − xe  < ε for all t ≥ 0. Furthermore, xe is asymptotically stable if it is Lyapunov stable and there exists ε > 0 such that, if x(0) − xe  < ε, then limt→∞ x(t) = xe . In addition, xe is globally asymptotically stable if it is Lyapunov stable, D = Rn, and, for all x(0) ∈ Rn, limt→∞ x(t) = xe . Finally, xe is unstable if it is not Lyapunov stable. Note that, if xe ∈ Rn is a globally asymptotically stable equilibrium, then xe is the only equilibrium of (11.7.1). The following result, known as Lyapunov’s direct method, gives sufficient conditions for Lyapunov stability and asymptotic stability of an equilibrium of (11.7.1). Theorem 11.7.2. Let xe ∈ D be an equilibrium of the dynamical system (11.7.1), and assume there exists a C1 function V : D → R such that V (xe ) = 0,

(11.7.2)

V (x) > 0,

(11.7.3)

V (x)f(x) ≤ 0.

(11.7.4)

such that, for all x ∈ D\{xe }, and such that, for all x ∈ D,

Then, xe is Lyapunov stable. If, in addition, for all x ∈ D\{xe }, V (x)f(x) < 0,

(11.7.5)

then xe is asymptotically stable. Finally, if D = R and n

lim V (x) = ∞,

x→∞

(11.7.6)

then xe is globally asymptotically stable. Proof. To prove Lyapunov stability, let ε > 0 be such that Bε (xe ) ⊆ D. Since Sε (xe ) is compact and V (x) is continuous, it follows from Theorem 10.3.8 that V [Sε (xe )] is compact. Since 0 ∈ Sε (xe ), V (x) > 0 for all x ∈ D\{xe }, and  V [Sε (xe )] is compact, it follows that α = min V [Sε (xe )] is positive. Next, since V is continuous, it follows that there exists δ ∈ (0, ε) such that, for all x ∈ Bδ (xe ), V (x) < α. Now, let x : [0, ∞) → Rn satisfy (11.7.1), where x(0) < δ. Hence,

726

CHAPTER 11

V [x(0)] < α. It thus follows from (11.7.4) that, for all t ≥ 0, t

V [x(s)]f [x(s)] ds ≤ 0,

V [x(t)] − V [x(0)] = 0

and hence, for all t ≥ 0,

V [x(t)] ≤ V [x(0)] < α.

Now, since V (x) ≥ α for all x ∈ Sε (0), it follows that, for all t ≥ 0, x(t) ∈ Sε (xe ). Hence, for all t ≥ 0, x(t) < ε, which proves that xe = 0 is Lyapunov stable. To prove that xe is asymptotically stable, let ε > 0 be such that Bε (xe ) ⊆ D. Since (11.7.5) implies (11.7.4), it follows that there exists δ > 0 such that, if d x(0) < δ, then x(t) < ε for all t ≥ 0. Furthermore, for all t ≥ 0, dt V [x(t)] = V [x(t)]f [x(t)] < 0, and thus V [x(t)] is decreasing and bounded from below by zero. Now, suppose that V [x(t)] does not converge to zero. Therefore, there exists L > 0 such that, for all t ≥ 0, V [x(t)] ≥ L. Now, let δ > 0 be such that, for all x ∈ Bδ (xe ), V (x) < L. Therefore, for all t ≥ 0, x(t) ≥ δ . Next, define γ < 0 by  γ= maxδ ≤x≤ε V (x)f(x). Therefore, since x(t) < ε for all t ≥ 0, it follows that t

V [x(τ )]f [x(τ )] dτ ≤ γt,

V [x(t)] − V [x(0)] = 0

and hence, for all t ≥ 0,

V (x(t)) ≤ V [x(0)] + γt.

However, t > −V [x(0)]/γ implies that V [x(t)] < 0, which is a contradiction. To prove that xe is globally asymptotically stable, let x(0) ∈ Rn, and let β = V [x(0)]. It follows from (11.7.6) that there exists ε > 0 such that, for all x ∈ Rn such that x > ε, V (x) > β. Therefore, x(0) ≤ ε, and, since V [x(t)] is decreasing, it follows that, for all t > 0, x(t) < ε. The remainder of the proof is identical to the proof of asymptotic stability. 

11.8 Linear Stability Theory We now specialize Definition 11.7.1 to the linear system x(t) ˙ = Ax(t),

(11.8.1)

where t ≥ 0, x(t) ∈ Rn, and A ∈ Rn×n. Note that xe = 0 is an equilibrium of (11.8.1), and that xe ∈ Rn is an equilibrium of (11.8.1) if and only if xe ∈ N(A). Hence, if xe is the globally asymptotically stable equilibrium of (11.8.1), then A is nonsingular and xe = 0. We consider three types of stability for the linear system (11.8.1). Unlike Definition 11.7.1, these definitions are stated in terms of the dynamics matrix rather than the equilibrium.

727

THE MATRIX EXPONENTIAL AND STABILITY THEORY

Definition 11.8.1. For A ∈ Cn×n, define the following classes of matrices: i) A is Lyapunov stable if spec(A) ⊂ CLHP and, if λ ∈ spec(A) and Re λ = 0, then λ is semisimple. ii) A is semistable if spec(A) ⊂ OLHP ∪ {0} and, if 0 ∈ spec(A), then 0 is semisimple. iii) A is asymptotically stable if spec(A) ⊂ OLHP. The following result concerns Lyapunov stability, semistability, and asymptotic stability for (11.8.1). Proposition 11.8.2. Let A ∈ Rn×n. Then, the following statements are equivalent: i) xe = 0 is a Lyapunov-stable equilibrium of (11.8.1). ii) At least one equilibrium of (11.8.1) is Lyapunov stable. iii) Every equilibrium of (11.8.1) is Lyapunov stable. iv) A is Lyapunov stable. v) For every initial condition x(0) ∈ Rn, x(t) is bounded for all t ≥ 0. vi) etA  is bounded for all t ≥ 0, where  ·  is a norm on Rn×n. vii) For every initial condition x(0) ∈ Rn, etAx(0) is bounded for all t ≥ 0. The following statements are equivalent: viii) A is semistable. ix) limt→∞ etA exists. x) For every initial condition x(0), limt→∞ x(t) exists. In this case, lim etA = I − AA#.

t→∞

The following statements are equivalent: xi) xe = 0 is an asymptotically stable equilibrium of (11.8.1). xii) A is asymptotically stable. xiii) spabs(A) < 0. xiv) For every initial condition x(0) ∈ Rn, limt→∞ x(t) = 0. xv) For every initial condition x(0) ∈ Rn, etAx(0) → 0 as t → ∞. xvi) etA → 0 as t → ∞. The following definition concerns the stability of a polynomial.

(11.8.2)

728

CHAPTER 11

Definition 11.8.3. Let p ∈ R[s]. Then, define the following terminology: i) p is Lyapunov stable if roots(p) ⊂ CLHP and, if λ is an imaginary root of p, then mp (λ) = 1. ii) p is semistable if roots(p) ⊂ OLHP ∪ {0} and, if 0 ∈ roots(p), then mp (0) = 1. iii) p is asymptotically stable if roots(p) ⊂ OLHP. For the following result, recall Definition 11.8.1. Proposition 11.8.4. Let A ∈ Rn×n. Then, the following statements hold: i) A is Lyapunov stable if and only if μA is Lyapunov stable. ii) A is semistable if and only if μA is semistable. Furthermore, the following statements are equivalent: iii) A is asymptotically stable iv) μA is asymptotically stable. v) χA is asymptotically stable. Next, consider the factorization of the minimal polynomial μA of A given by u μA = μAs μA , (11.8.3) u are monic polynomials such that where μAs and μA

and

roots(μAs ) ⊂ OLHP

(11.8.4)

u ) ⊂ CRHP. roots(μA

(11.8.5)

Proposition 11.8.5. Let A ∈ Rn×n, and let S ∈ Rn×n be a nonsingular matrix such that   A1 A12 S −1, A=S (11.8.6) 0 A2 where A1 ∈ Rr×r is asymptotically stable, A12 ∈ Rr×(n−r), and A2 ∈ R(n−r)×(n−r) satisfies spec(A2 ) ⊂ CRHP. Then,   0 C12s s S −1, (11.8.7) μA(A) = S 0 μAs (A2 ) where C12s ∈ Rr×(n−r) and μAs (A2 ) is nonsingular, and   u μ (A ) C12u u S −1, μA (A) = S A 1 0 0 u (A1 ) is nonsingular. Consequently, where C12u ∈ Rr×(n−r) and μA    Ir s u . N[μA(A)] = R[μA(A)] = R S 0

(11.8.8)

(11.8.9)

729

THE MATRIX EXPONENTIAL AND STABILITY THEORY

If, in addition, A12 = 0, then



μAs (A) = S and

0 0

 0 S −1 μAs (A2 )

 u (A1 ) 0 μA S −1. 0 0    0 s u . R[μA(A)] = N[μA(A)] = R S In−r

(11.8.10)



u (A) = S μA

Consequently,

(11.8.11) (11.8.12)

Corollary 11.8.6. Let A ∈ Rn×n. Then,

and

u (A)] N[μAs (A)] = R[μA

(11.8.13)

u (A)] = R[μAs (A)]. N[μA

(11.8.14)

Proof. It follows from Theorem 5.3.5 that there exists a nonsingular matrix S ∈ Rn×n such that (11.8.6) is satisfied, where A1 ∈ Rr×r is asymptotically stable, A12 = 0, and A2 ∈ R(n−r)×(n−r) satisfies spec(A2 ) ⊂ CRHP. The result now follows from Proposition 11.8.5. In view of Corollary 11.8.6, we define the asymptotically stable subspace Ss (A) of A by  u Ss (A) = N[μAs (A)] = R[μA (A)] (11.8.15) and the unstable subspace Su(A) of A by 

u Su(A) = N[μA (A)] = R[μAs (A)].

Note that u (A) = dim Ss (A) = def μAs (A) = rank μA



amultA(λ)

(11.8.16)

(11.8.17)

λ∈spec(A) Re λ −x∗Rx = (λ + λ)x∗P x, which implies that A is asymptotically stable. Finally, to prove that A is semistable, let jω ∈ spec(A), where ω ∈ R is nonzero, and let x ∈ Cn be an associated eigenvector. Then,   −x∗Rx = x∗ ATP + PA x = x∗[(jωI − A)∗P + P (jωI − A]x = 0. Therefore, Rx = 0, and thus



jωI − A R

 x = 0,

which implies that x = 0, which contradicts x = 0. Consequently, jω ∈ spec(A) for all nonzero ω ∈ R, and thus A is semistable. Equation (11.9.1) is a Lyapunov equation. Converse results for Corollary 11.9.1 are given by Corollary 11.9.4, Proposition 11.9.6, Proposition 11.9.5, and Proposition 11.9.6. The following lemma is useful for analyzing (11.9.1). Lemma 11.9.2. Assume that A ∈ Fn×n is asymptotically stable. Then, ∞

etA dt = −A−1.

(11.9.3)

0

Proof. Proposition 11.1.4 implies that ∞ yields (11.9.3).

,t 0

  eτA dτ = A−1 etA − I . Letting t →

The following result concerns Sylvester’s equation. See Fact 5.10.21 and Proposition 7.2.4.

732

CHAPTER 11

Proposition 11.9.3. Let A, B, C ∈ Rn×n. Then, there exists a unique matrix X ∈ Rn×n satisfying AX + XB + C = 0 if and only if BT ⊕ A is nonsingular. In this case, X is given by   −1 X = − vec−1 BT ⊕ A vec C .

(11.9.4)

(11.9.5)

If, in addition, BT ⊕ A is asymptotically stable, then X is given by ∞

etA CetB dt.

X=

(11.9.6)

0

Proof. The first two statements follow from Proposition 7.2.4. If BT ⊕ A is asymptotically stable, then it follows from (11.9.5) using Lemma 11.9.2 and Proposition 11.1.7 that ∞

  T vec−1 et(B ⊕A) vec C dt =

X=

0 ∞

 T  vec−1 etB ⊗ etA vec C dt

0

  vec−1 vec etA CetB dt =

=

∞

0

∞

etA CetB dt.



0

The following result provides a converse to Corollary 11.9.1 for the case of asymptotic stability. Corollary 11.9.4. Let A ∈ Rn×n, and let R ∈ Rn×n. Then, there exists a unique matrix P ∈ Rn×n satisfying (11.9.1) if and only if A ⊕ A is nonsingular. In this case, if R is symmetric, then P is symmetric. Now, assume in addition that A is asymptotically stable. Then, P ∈ Sn is given by ∞ T

etA RetA dt.

P =

(11.9.7)

0

Finally, if R is (positive semidefinite, positive definite), then P is (positive semidefinite, positive definite). Proof. First note that A⊕A is nonsingular if and only if (A⊕A)T = AT ⊕AT is nonsingular. Now, the first statement follows 11.9.3. To prove the    from Proposition  second statement, note that AT P − P T + P − P T A = 0, which implies that P is symmetric. Now, suppose that A is asymptotically stable. Then, Fact 11.18.33 implies that A ⊕ A is asymptotically stable. Consequently, (11.9.7) follows from (11.9.6). The following results include converse statements. We first consider asymptotic stability.

THE MATRIX EXPONENTIAL AND STABILITY THEORY

733

Proposition 11.9.5. Let A ∈ Rn×n. Then, the following statements are equivalent: i) A is asymptotically stable. ii) For every positive-definite matrix R ∈ Rn×n there exists a positive-definite matrix P ∈ Rn×n such that (11.9.1) is satisfied. iii) There exist a positive-definite matrix R ∈ Rn×n and a positive-definite matrix P ∈ Rn×n such that (11.9.1) is satisfied. Proof. The result i) =⇒ ii) follows from Corollary 11.9.1. The statement ii) =⇒ iii) is immediate. To prove that iii) =⇒ i), note that, since there exists a positive-semidefinite matrix P satisfying (11.9.1), it follows from Proposition 12.4.3 that (A, C) is observably asymptotically

−1Thus, there exists a non 1 stable. 0 singular matrix S ∈ Rn×n such that A = S AA21 and C = C1 0 S −1, A2 S where (C1, A1 ) is observable and A1 is asymptotically stable. Furthermore, since (S −1AS, CS) is detectable, it follows that A2 is also asymptotically stable. Consequently, A is asymptotically stable. Next, we consider the case of Lyapunov stability. Proposition 11.9.6. Let A ∈ Rn×n. Then, the following statements hold: i) If A is Lyapunov stable, then there exist a positive-definite matrix P ∈ Rn×n and a positive-semidefinite matrix R ∈ Rn×n such that rank R = ν− (A) and such that (11.9.1) is satisfied. ii) If there exist a positive-definite matrix P ∈ Rn×n and a positive-semidefinite matrix R ∈ Rn×n such that (11.9.1) is satisfied, then A is Lyapunov stable. Proof. To prove i), suppose that A is Lyapunov stable. Then, it follows from Theorem 5.3.5 and Definition

11.8.1 that there exists a nonsingular matrix S ∈ Rn×n such that A = S A01 A02 S −1 is in real Jordan form, where A1 ∈ Rn1 ×n1 , A2 ∈ Rn2 ×n2 , spec(A1 ) ⊂ jR, A1 is semisimple, and spec(A2 ) ⊂ OLHP. Next, it follows from Fact 5.9.6 that there exists a nonsingular matrix S1 ∈ Rn1 ×n1 such that A1 = S1J1S1−1, where J1 ∈ Rn1 ×n1 is skew symmetric. Then, it follows that  A1TP1 + P1A1 = S1−T(J1 + J1T )S1−1 = 0, where P1 = S1−TS1−1 is positive definite. n2 ×n2 Next, let R2 ∈ R be positive definite, and let P2 ∈ Rn2 ×n2 be the positive T definite solution of A2 P2 + P2 A2 + R2 = 0. Hence, (11.9.1) is satisfied with P =



 P 0 S −T 01 P2 S −1 and R = S −T 00 R02 S −1. To prove ii), suppose there exist a positive-semidefinite matrix R ∈ Rn×n and a positive-definite matrix P ∈ Rn×n such that (11.9.1) is satisfied. Let λ ∈ spec(A), and let x ∈ Cn be an eigenvector of A associated with λ. It thus follows from (11.9.1) that 0 = x∗ATP x + x∗PAx + x∗Rx = (λ + λ)x∗P x + x∗Rx. Therefore, Re λ = −x∗Rx/(2x∗P x), which shows that spec(A) ⊂ CLHP. Now, let jω ∈ spec(A), 2 and suppose that x ∈ Rn satisfies (jωI (jωI − A)y = 0, where  T − A) x = 0. Then, ∗ ∗ y = (jωI − A)x. Computing 0 = y A P + PA y + y Ry yields y ∗Ry = 0 and thus   T ∗ T Ry = 0. Therefore, A P + PA y = 0, and thus y P y = (A + jωI)P y = 0. Since

734

CHAPTER 11

P is positive definite, it follows that y = (jωI − A)x = 0. Therefore, N(jωI − A) =

N (jωI − A)2 . Hence, it follows from Proposition 5.5.8 that jω is semisimple. Corollary 11.9.7. Let A ∈ Rn×n. Then, the following statements hold: i) A is Lyapunov stable if and only if there exists a positive-definite matrix P ∈ Rn×n such that ATP + PA is negative semidefinite. ii) A is asymptotically stable if and only if there exists a positive-definite matrix P ∈ Rn×n such that ATP + PA is negative definite.

11.10 Discrete-Time Stability Theory The theory of difference equations is concerned with solutions of discrete-time dynamical systems of the form x(k + 1) = f [x(k)],

(11.10.1)

where f : Rn → Rn, k ∈ N, x(k) ∈ Rn, and x(0) = x0 is the initial condition. The solution x(k) ≡ xe is an equilibrium of (11.10.1) if xe = f(xe ). A linear discrete-time system has the form x(k + 1) = Ax(k),

(11.10.2)

where A ∈ Rn×n. For k ∈ N, x(k) is given by x(k) = Akx0 .

(11.10.3)

The behavior of the sequence (x(k))∞ k=0 is determined by the stability of A. Definition 11.10.1. For A ∈ Cn×n, define the following classes of matrices: i) A is discrete-time Lyapunov stable if spec(A) ⊂ CUD and, if λ ∈ spec(A) and |λ| = 1, then λ is semisimple. ii) A is discrete-time semistable if spec(A) ⊂ OUD ∪ {1} and, if 1 ∈ spec(A), then 1 is semisimple. iii) A is discrete-time asymptotically stable if spec(A) ⊂ OUD. Proposition 11.10.2. Let A ∈ Rn×n and consider the linear discrete-time system (11.10.2). Then, the following statements are equivalent: i) A is discrete-time Lyapunov stable. ii) For every initial condition x0 ∈ Rn, the sequence (x(k))∞ k=1 is bounded, where  ·  is a norm on Rn. iii) For every initial condition x0 ∈ Rn, the sequence (Akx0 )∞ k=1 is bounded, where  ·  is a norm on Rn. n×n . iv) The sequence (Ak )∞ k=1 is bounded, where  ·  is a norm on R

The following statements are equivalent:

THE MATRIX EXPONENTIAL AND STABILITY THEORY

735

v) A is discrete-time semistable. vi) limk→∞ Ak exists. In fact, limk→∞ Ak = I − (I − A)(I − A)#. vii) For every initial condition x0 ∈ Rn, limk→∞ x(k) exists. The following statements are equivalent: viii) A is discrete-time asymptotically stable. ix) sprad(A) < 1. x) For every initial condition x0 ∈ Rn, limk→∞ x(k) = 0. xi) For every initial condition x0 ∈ Rn, Akx0 → 0 as k → ∞. xii) Ak → 0 as k → ∞. The following definition concerns the discrete-time stability of a polynomial. Definition 11.10.3. For p ∈ R[s], define the following terminology: i) p is discrete-time Lyapunov stable if roots(p) ⊂ CUD and, if λ is an imaginary root of p, then mp (λ) = 1. ii) p is discrete-time semistable if roots(p) ⊂ OUD ∪ {1} and, if 1 ∈ roots(p), then mp (1) = 1. iii) p is discrete-time asymptotically stable if roots(p) ⊂ OUD. Proposition 11.10.4. Let A ∈ Rn×n. Then, the following statements hold: i) A is discrete-time Lyapunov stable if and only if μA is discrete-time Lyapunov stable. ii) A is discrete-time semistable if and only if μA is discrete-time semistable. Furthermore, the following statements are equivalent: iii) A is discrete-time asymptotically stable. iv) μA is discrete-time asymptotically stable. v) χA is discrete-time asymptotically stable. We now consider the discrete-time Lyapunov equation P = ATPA + R.

(11.10.4)

Proposition 11.10.5. Let A ∈ Rn×n. Then, the following statements are equivalent: i) A is discrete-time asymptotically stable. ii) For every positive-definite matrix R ∈ Rn×n there exists a positive-definite matrix P ∈ Rn×n such that (11.10.4) is satisfied. iii) There exist a positive-definite matrix R ∈ Rn×n and a positive-definite matrix P ∈ Rn×n such that (11.10.4) is satisfied.

736

CHAPTER 11

Proposition 11.10.6. Let A ∈ Rn×n. Then, A is discrete-time Lyapunovstable if and only if there exist a positive-definite matrix P ∈ Rn×n and a positivesemidefinite matrix R ∈ Rn×n such that (11.10.4) is satisfied.

11.11 Facts on Matrix Exponential Formulas Fact 11.11.1. Let A ∈ Rn×n. Then, the following statements hold: i) If A2 = 0, then etA = I + tA. ii) If A2 = I, then etA = (cosh t)I + (sinh t)A. iii) If A2 = −I, then etA = (cos t)I + (sin t)A. iv) If A2 = A, then etA = I + (et − 1)A. v) If A2 = −A, then etA = I + (1 − e−t )A. vi) If rank A = 1 and tr A = 0, then etA = I + tA. vii) If rank A = 1 and tr A = 0, then etA = I +

e(tr A)t −1 A. tr A

Remark: See [1112].  Fact 11.11.2. Let A =



0 In In 0

. Then,

etA = (cosh t)I2n + (sinh t)A. Furthermore,

etJ2n = (cos t)I2n + (sin t)J2n .

Fact 11.11.3. Let A ∈ Rn×n, and assume that A is skew symmetric. Then, {e : θ ∈ R} ⊆ SO(n) is a group. If, in addition, n = 2, then θA

Remark: Note that eθJ2

{eθJ2 : θ ∈ R} = SO(2).

θ sin θ = −cos sin θ cos θ . See Fact 3.11.27.

Fact 11.11.4. Let A ∈ Rn×n, ⎡ 0 1 ⎢ 0 0 ⎢ ⎢ 0 0 ⎢  ⎢ . A = ⎢ .. ... ⎢ ⎢ ⎢ 0 0 ⎣ 0 0

where 0 2 0 .. .

0 0 3 .. .

··· ··· ··· .. .

0

0

..

0

0

···

.

0 0 0 .. .

⎤

⎥ ⎥ ⎥ ⎥ ⎥ ⎥. ⎥ ⎥ n −1 ⎥ ⎦ 0

737

THE MATRIX EXPONENTIAL AND STABILITY THEORY

⎡ 0 1 2 3

Then,

0

⎢ ⎢ 0 ⎢ ⎢ 0 ⎢ eA = ⎢ ⎢ .. ⎢ . ⎢ ⎢ 0 ⎣ 0

01 1

02 1 2 2

03 1 3 2

0

0

0

0

0

0

···

.. .

..

.

0 n−1  ⎥ ⎥ 1 n−1 ⎥ ⎥ ⎥ 2 ⎥. .. ⎥ . ⎥  n−1  ⎥ ⎥ n−2 ⎦ n−1

··· ··· .. . .. .

0 .. .

n−1 ⎤

···

n−1

Furthermore, if k ≥ n, then k 

in−1 =



⎡ k  ⎤ 1n−1

2n−1

···

i=1

nn−1

⎢ e−A⎣

1

.. ⎥. ⎦ k.  n

Proof: See [76]. Remark: For related results, see [5], where A is called the creation matrix. See Fact 5.16.3. Fact 11.11.5. Let A ∈ F3×3. If spec(A) = {λ}, then

etA = eλt I + t(A − λI) + 12 t2 (A − λI)2 . If mspec(A) = {λ, λ, μ}ms , where μ = λ, then   μt e − eλt teλt (A − λI)2. etA = eλt [I + t(A − λI)] + − (μ − λ)2 μ −λ If spec(A) = {λ, μ, ν}, then etA =

eμt eλt (A − μI)(A − νI) + (A − λI)(A − νI) (λ − μ)(λ − ν) (μ − λ)(μ − ν) eνt (A − λI)(A − μI). + (ν − λ)(ν − μ)

Proof: See [70]. Remark: Additional expressions are given in [2, 179, 195, 329, 657, 1112, 1115]. 

Fact 11.11.6. Let x ∈ R3, assume that x is nonzero, and define θ = x2 . Then, 1 − cos θ 2 sin θ K(x) + K (x) θ θ2  2 1 sin θ 1 sin( 2 θ) K(x) + 2 K 2 (x) =I+ 1 θ θ 2 sin θ 1 − cos θ T xx . = (cos θ)I + K(x) + θ θ2

eK(x) = I +

Furthermore,

eK(x)x = x,

738

CHAPTER 11

spec[eK(x) ] = {1, ejx2 , e−jx2 }, and

tr eK(x) = 1 + 2cos θ = 1 + 2cos x2 .

Proof: The Cayley-Hamilton theorem or Fact 3.10.1 implies that K 3 (x)+θ2K(x) = 0. Then, every term K k (x) in the expansion of eK(x) can be expressed in terms of K(x) or K 2 (x). Finally, Fact 3.10.1 implies that θ2I + K 2 (x) = xxT. Remark: Fact 11.11.7 shows that, for all z ∈ R3, eK(x)z is the counterclockwise (right-hand-rule) rotation of z about the vector x through the angle θ, which is given by the Euclidean norm of x. In Fact 3.11.29, the cross product is used to construct the pivot vector x for a given pair of vectors having the same length. Fact 11.11.7. Let x, y ∈ R3, and assume that x and y are nonzero. Then, there exists a skew-symmetric matrix A ∈ R3×3 such that y = eAx if and only if xTx = yTy. If x = −y, then one such matrix is A = θK(z), where 

z=

1 x×y x × y2 

and 

−1

θ = cos

 xTy . x2 y2

 ⊥ If x = −y, then one such matrix is A = πK(z), where z = y−1 2 ν × y and ν ∈ {y} T satisfies ν ν = 1.

Proof: This result follows from Fact 3.11.29 and Fact 11.11.6, which provide equivalent expressions for an orthogonal matrix that transforms a given vector into another given vector having the same length. This result thus provides a geometric interpretation for Fact 11.11.6. Remark: Note that z is the unit vector perpendicular to the plane containing x and y, where the direction of z is determined by the right-hand rule. An intuitive proof is to let x be the initial condition to the differential equation w(t) ˙ = K(z)w(t), that is, w(0) = x, where t ∈ [0, θ]. Then, the derivative w(t) ˙ lies in the x, y plane and is perpendicular to w(t) for all t ∈ [0, θ]. Hence, y = w(θ). Remark: Since det eA = etr A = 1, it follows that every pair of vectors in R3 having the same Euclidean length are related by a proper rotation. See Fact 3.9.5 and Fact 3.14.4. This is a linear interpolation problem. See Fact 3.9.5, Fact 3.11.29, and [795]. Remark: See Fact 3.11.29. Remark: Parameterizations of SO(3) are considered in [1226, 1277]. Problem: Extend this result to Rn. See [139, 1194]. Fact 11.11.8. Let A ∈ SO(3), let z ∈ R3 be an eigenvector of A corresponding to the eigenvalue 1 of A, assume that z2 = 1, assume that tr A > −1, and let θ ∈ (−π, π) satisfy tr A = 1 + 2cos θ. Then, A = eθK(z).

739

THE MATRIX EXPONENTIAL AND STABILITY THEORY

Remark: See Fact 5.11.2. Fact 11.11.9. Let x, y ∈ R3, and assume that x and y are nonzero. Then, x x = yTy if and only if θ (yxT −xy T ) x, y = e x×y 2 T

where

  θ = cos−1

 xTy . x2 y2

Proof: Use Fact 11.11.7. Remark: Note that K(x × y) = yxT − xy T. Fact 11.11.10. Let A ∈ R3×3, assume that A ∈ SO(3) and tr A > −1, and let θ ∈ (−π, π) satisfy tr A = 1 + 2cos θ. Then, * 0, θ = 0, log A = θ T (A − A ), θ = 0. 2sin θ Proof: See [767, p. 364] and [1038]. Remark: See Fact 11.15.10. 

Fact 11.11.11. Let x ∈ R3, assume that x is nonzero, and define θ = x2 . Then, K(x) θ K(x) = 2sin − e−K(x) ]. θ [e Proof: Use Fact 11.11.10. Remark: See Fact 3.10.1. Fact 11.11.12. Let A ∈ SO(3), let x, y ∈ R3, and assume that xTx = yTy. Then, Ax = y if and only if, for all t ∈ R, AetK(x)A−1 = etK(y). Proof: See [912]. Fact 11.11.13. Let x, y, z ∈ R3. Then, the following statements are equivalent: i) For every A ∈ SO(3), there exist α, β, γ ∈ R such that A = eαK(x)eβK(y)eγK(z). ii) yTx = 0 and yTz = 0. Proof: See [912]. Remark: This result is due to Davenport. Problem: Given A ∈ SO(3), determine α, β, γ.

740

CHAPTER 11

Fact 11.11.14. Let x, y, z ∈ R3, assume that xT (y × z) = 0, assume that x2 = y2 = z2 = 1, and let α, β, γ ∈ [0, 2π) satisfy

Then,

tan α2 =

z T (x × y) , (x × y)T (x × z)

tan β2 =

xT (y × z) , (y × z)T (y × x)

tan γ2 =

y T (z × x) . (z × x)T (z × y)

eαK(x) eβK(y) eγK(z) = I.

Proof: See [1046, p. 5]. Remark: This result is the Rodrigues-Hamilton theorem. An equivalent result is given by Donkin’s theorem [1046, p. 6]. Remark: See [1170, 1482]. Fact 11.11.15. Let A ∈ R4×4, and assume that A is skew symmetric with mspec(A) = {jω, −jω, jμ, −jμ}ms . If ω = μ, then eA = a3A3 + a2A2 + a1A + a0 I, where

 −1 1  1 sin μ − sin ω , a3 = ω 2 − μ2 μ ω   2 −1 a2 = ω − μ2 (cos μ − cos ω),  −1 ω2  μ2 a1 = ω 2 − μ2 sin μ − sin ω , μ ω    2  −1 ω 2 cos μ − μ2 cos ω . a0 = ω − μ2

If ω = μ, then eA = (cos ω)I +

sin ω A. ω

Proof: See [622, p. 18] and [1115]. Remark: There are errors in [622, p. 18] and [1115]. Remark: See Fact 4.9.21 and Fact 4.10.4. Fact 11.11.16. Let a, b, c ∈ R, define the skew-symmetric matrix A ∈ R4×4, by either ⎤ ⎡ 0 a b c ⎢ −a 0 −c b ⎥  ⎥ A=⎢ ⎣ −b c 0 −a ⎦ −c −b a 0 ⎤ ⎡ or 0 a b c ⎢ −a 0 c −b ⎥  ⎥, A=⎢ ⎣ −b −c 0 a ⎦ −c b −a 0

741

THE MATRIX EXPONENTIAL AND STABILITY THEORY 

and define θ =

√ a2 + b2 + c2 . Then, mspec(A) = {jθ, −jθ, jθ, −jθ}ms .

Furthermore, Ak =

⎧ ⎨(−1)k/2 θkI, ⎩

k even,

(−1)(k−1)/2 θk−1A,

and eA = (cos θ)I +

k odd,

sin θ A. θ

Proof: See [1390]. Remark: (sin 0)/0 = 1. Remark: The skew-symmetric matrix A arises in the kinematic relationship between the angular velocity vector and quaternion (Euler-parameter) rates. See [156, p. 385]. Remark: The two matrices A are similar. To show this, note that Fact 5.9.11 implies that A and −A are similar. Then, apply the similarity transformation S = diag(−1, 1, 1, 1). Remark: See Fact 4.9.21 and Fact 4.10.4.

by

Fact 11.11.17. Let x ∈ R3, and define the skew-symmetric matrix A ∈ R4×4   0 −xT . A= x −K(x)

Then, for all t ∈ R, 1

e 2 tA = cos( 12 xt)I4 +

sin( 12 xt) A. x

Proof: See [753, p. 34]. Remark: The matrix 12 A expresses quaternion rates in terms of the angular velocity vector. Fact 11.11.18. Let a, b ∈ R3, define the skew-symmetric matrix A ∈ R4×4 by   K(a) b , A= −bT 0 √  assume that aTb = 0, and define α = aTa + bTb. Then, eA = I4 +

1 − cos α 2 sin α A+ A. α α2

Proof: See [1366]. Remark: See Fact 4.9.21 and Fact 4.10.4.

742

CHAPTER 11

Fact 11.11.19. Let a, b ∈ Rn−1, define A ∈ Rn×n by   0 aT  , A= b 0(n−1)×(n−1)   and define α = |aT b|. Then, the following statements hold: i) If aT b < 0, then etA = I + ii) If aT b = 0, then

sin α A+ α

 1 2

2 sin(α/2) A2. α/2

etA = I + A + 12 A2.

iii) If aT b > 0, then e

tA

sinh α A+ =I+ α

 1 2

2 sinh(α/2) A2. α/2

Proof: See [1516].

11.12 Facts on the Matrix Sine and Cosine Fact 11.12.1. Let A ∈ Cn×n, and define 

sin A = A − and



cos A = I −

1 3 3! A 1 2 2! A

+

+

1 5 5! A 1 4 4! A

−

−

1 7 7! A 1 6 6! A

+ ···

+ ··· .

Then, the following statements hold: i) sin A =

1 jA j2 (e

− e−jA ).

ii) cos A = 12 (ejA + e−jA ). iii) sin2 A + cos2 A = I. iv) sin(2A) = 2(sin A) cos A. v) cos(2A) = 2(cos2 A) − I. vi) If A is real, then sin A = Re ejA and cos A = Re ejA . vii) sin(A ⊕ B) = (sin A) ⊗ cos B − (cos A) ⊗ sin B. viii) cos(A ⊕ B) = (cos A) ⊗ cos B − (sin A) ⊗ sin B. ix) If A is involutory and k is an integer, then cos(kπA) = (−1)kI. Furthermore, the following statements are equivalent: x) For all t ∈ R, sin[t(A + B)] = [sin(tA)][cos(tB)] + [cos(tA)][sin(tB)]. xi) For all t ∈ R, cos[t(A + B)] = [cos(tA)][cos(tB)] − [sin(tA)][sin(tB)]. xii) AB = BA. Proof: See [701, pp. 287, 288, 300].

743

THE MATRIX EXPONENTIAL AND STABILITY THEORY

11.13 Facts on the Matrix Exponential for One Matrix Fact 11.13.1. Let A ∈ Fn×n, and assume that A is (lower triangular, upper triangular). Then, so is eA. If, in addition, A is Toeplitz, then so is eA. Remark: See Fact 3.18.7. Fact 11.13.2. Let A ∈ Fn×n. Then,   sprad eA = espabs(A). Fact 11.13.3. Let A ∈ Rn×n, and let X0 ∈ Rn×n. Then, the matrix differential equation ˙ X(t) = AX(t), X(0) = X0 , where t ≥ 0, has the unique solution X(t) = etAX0 . Fact 11.13.4. Let A: [0, T ] → Rn×n, assume that A is continuous, and let X0 ∈ Rn×n. Then, the matrix differential equation ˙ X(t) = A(t)X(t), X(0) = X0 has a unique solution X: [0, T ] → Rn×n. Furthermore, for all t ∈ [0, T ], det X(t) = e

,t 0

tr A(τ ) dτ

det X0 .

Therefore, if X0 is nonsingular, then X(t) is nonsingular for all t ∈ [0, T ]. If, in addition, for all t1, t2 ∈ [0, T ], t2

t2

A(t2 ) A(τ ) dτ = t1

A(τ ) dτ A(t2 ), t1

then, for all t ∈ [0, T ], X(t) = e

,t 0

A(τ ) dτ

X0 .

˙ = tr(X AAX) = Proof: It follows from Fact 10.12.8 that (d/dt) det X = tr(X AX) A tr(XX A) = (det X) tr A. This proof is given in [577]. See also [730, pp. 507, 508] and [1179, pp. 64–66]. Remark: See Fact 11.13.4. Remark: The first result is Jacobi’s identity. Remark: If the commutativity assumption does not hold, then the solution is given by the Peano-Baker series. See [1179, Chapter 3]. Alternative expressions for X(t) are given by the Magnus, Fer, Baker-Campbell-Hausdorff-Dynkin, Wei-Norman, Goldberg, and Zassenhaus expansions. See [232, 455, 766, 767, 854, 974, 1083, 1275, 1305, 1448, 1449, 1453] and [636, pp. 118–120].

744

CHAPTER 11

Fact 11.13.5. Let A: [0, T ] → Rn×n, assume that A is continuous, let B: [0, T ] → Rn×m, assume that B is continuous, let X: [0, T ] → Rn×n satisfy the matrix differential equation ˙ X(t) = A(t)X(t), X(0) = I, define

 X(t)X −1(τ ), Φ(t, τ ) =

let u: [0, T ] → Rm, and assume that u is continuous. Then, the vector differential equation x(t) ˙ = A(t)x(t) + B(t)u(t), x(0) = x0 has the unique solution

t

x(t) = X(t)x0 +

Φ(t, τ )B(τ )u(τ )dτ. 0

Remark: Φ(t, τ ) is the state transition matrix. Fact 11.13.6. Let A ∈ Rn×n, let λ ∈ spec(A), and let v ∈ Cn be an eigenvector of A associated with λ. Then, for all t ≥ 0,    x(t) = Re eλtv satisfies x(t) ˙ = Ax(t). Remark: x(t) is an eigensolution. Fact 11.13.7. Let A ∈ Rn×n, let λ ∈ spec(A), and let (v1, . . . , vk ) ∈ (Cn )k be a Jordan chain of A associated with λ. Then, for all i such that 1 ≤ i ≤ k,     1 x(t) = Re eλt (i−1)! ti−1 v1 + · · · + tvi−1 + vi satisfies x(t) ˙ = Ax(t) for all t ≥ 0. Remark: See Fact 5.14.9 for the definition of a Jordan chain. Remark: x(t) is a generalized eigensolution.

β 0 0 1 ˆ Example: Let  = [ 0 0 ], λ = 0, k = 2, v1 = 0 , and v2 = β . Then, x(t) =  A is a generalized eigensolution. Alternatively, choosing kˆ = 1 tv1 + v2 = βt β yields the eigensolution x(t) = v1 = β0 . Note that β represents velocity for the generalized eigensolution and position for the eigensolution. See [1089]. Fact 11.13.8. Let S: [t0 , t1 ] → Rn×n be differentiable. Then, for all t ∈ [t0 , t1 ], d 2 ˙ ˙ S (t) = S(t)S(t) + S(t)S(t). dt Let S1: [t0 , t1 ] → Rn×m and S2 : [t0 , t1 ] → Rm×l be differentiable. Then, for all t ∈ [t0 , t1 ], d S1(t)S2(t) = S˙ 1(t)S2(t) + S1(t)S˙ 2(t). dt

745

THE MATRIX EXPONENTIAL AND STABILITY THEORY 



Fact 11.13.9. Let A ∈ Fn×n, and define A1 = 12 (A + A∗ ) and A2 = 12 (A −A∗ ). Then, A1A2 = A2A1 if and only if A is normal. In this case, eA1eA2 is the polar decomposition of eA. Remark: See Fact 3.7.28. Problem: Obtain the polar decomposition of eA for the case in which A is not normal. Fact 11.13.10. Let A ∈ Fn×m, and assume that rank A = m. Then, ∞ ∗

e−tA AA∗ dt.

+

A = 0

Fact 11.13.11. Let A ∈ Fn×n, and assume that A is nonsingular. Then, ∞ −1

A

∗

e−tA A dtA∗.

= 0



Fact 11.13.12. Let A ∈ Fn×n, and let k = ind A. Then, ∞

e−tA A

k (2k+1)∗ k+1

D

A =

A

dtAkA(2k+1)∗Ak.

0

Proof: See [584]. Fact 11.13.13. Let A ∈ Fn×n, and assume that ind A = 1. Then, ∞

e−tAA

3∗ 2

#

A =

A

dtAA3∗A.

0

Proof: See Fact 11.13.12.  Fact 11.13.14. Let A ∈ Fn×n, and let k = ind A. Then,

t

    eτA dτ = AD etA − I + I − AAD tI +

1 2 2! t A

+ ···+

1 k k−1 k! t A

 .

0

If, in particular, A is group invertible, then t

    eτA dτ = A# etA − I + I − AA# t.

0

Fact 11.13.15. Let A ∈ Fn×n, let mspec(A) = {λ1, . . . , λr , 0, . . . , 0}ms , where λ1, . . . , λr are nonzero, and let t > 0. Then, t

det eτA dτ = tn−r 0

r ! i=1

 λit  λ−1 −1 . i e

746

CHAPTER 11

,t Hence, det 0 eτA dτ = 0 if and only if, for every nonzero integer k, j2kπ/t ∈ spec(A).   ,t Finally, det etA − I = 0 if and only if det A = 0 and det 0 eτA dτ = 0. Fact 11.13.16. Let A ∈ Rn×n, and let σ be a positive number. Then, 1 √ 2πσ

∞

exA e−x /(2σ) dx = e(σ/2)A . 2

2

−∞

Remark: This result is due to Bhat. Remark: See Fact 8.16.1. Fact 11.13.17. Let A ∈ Fn×n, and assume that there exists α ∈ R such that spec(A) ⊂ {z ∈ C: α ≤ Im z < 2π + α}. Then, eA is (diagonal, upper triangular, lower triangular) if and only if A is. Proof: See [957]. Fact 11.13.18. Let A ∈ Fn×n. Then, the following statements hold: i) If A is unipotent, then the series (11.5.1) is finite, log A exists and is nilpotent, and elog A = A. ii) If A is nilpotent, then eA is unipotent and log eA = A. Proof: See [639, p. 60]. Fact 11.13.19. Let B ∈ Rn×n. Then, there exists a normal matrix A ∈ Rn×n such that B = eA if and only if B is normal, nonsingular, and every negative eigenvalue of B has even algebraic multiplicity. Fact 11.13.20. Let C ∈ Rn×n, assume that C is nonsingular, and let k ≥ 1. Then, there exists a matrix B ∈ Rn×n such that C 2k = eB. Proof: Use Proposition 11.4.3 with A = C 2, and note that every negative√eigenvalue −α < 0 of C 2 arises as the square of complex conjugate eigenvalues ±j α of C.

11.14 Facts on the Matrix Exponential for Two or More Matrices Fact 11.14.1. Let A, B ∈ Fn×n, and consider the following conditions: i) A = B. ii) eA = eB. iii) AB = BA. iv) AeB = eBA. v) eAeB = eBeA. vi) eAeB = eA+B. vii) eAeB = eBeA = eA+B.

747

THE MATRIX EXPONENTIAL AND STABILITY THEORY

Then, the following statements hold: viii) iii) =⇒ iv) =⇒ v). ix) iii) =⇒ vii) =⇒ vi). x) If spec(A) is j2π congruence free, then ii) =⇒ iii) =⇒ iv) ⇐⇒ v). xi) If spec(A) and spec(B) are j2π congruence free, then ii) =⇒ iii) ⇐⇒ iv) ⇐⇒ v). xii) If spec(A + B) is j2π congruence free, then iii) ⇐⇒ vii). xiii) If, for all λ ∈ spec(A) and all μ ∈ spec(B), it follows that (λ − μ)/(j2π) is not a nonzero integer, then ii) =⇒ i). xiv) If A and B are Hermitian, then i) ⇐⇒ ii) =⇒ iii) ⇐⇒ iv) ⇐⇒ v) ⇐⇒ vi). Remark: The set S ⊂ C is j2π congruence free if no two elements of S differ by a nonzero integer multiple of j2π. Proof: See [644, pp. 88, 89, 270–272] and [1092, 1199, 1200, 1201, 1239, 1454, 1455]. The assumption of normality in operator versions of some of these statements in [1092, 1201] is not needed in the matrix case. Statement xiii) and the first implication in x) are given in [701, p. 32]. Remark: The matrices A=



0 0

1 j2π



 ,

B=

j2π 0

0 −j2π



satisfy eA = eB = eA+B = I, but AB = BA. Therefore, vii) =⇒ iii) does not hold. Furthermore, since vii) holds and iii) does not hold, it follows from xii) that spec(A + B) is not j2π congruence free. In fact, spec(A + B) = {0, j2π}. The same observation holds for the real matrices √ √ ⎡ ⎤ ⎡ ⎤ 0 0 3/2 0 0 − 3/2 0 −1/2 ⎦, 0 −1/2 ⎦. A = 2π ⎣ √0 B = 2π ⎣ √ 0 − 3/2 1/2 0 3/2 1/2 0 Remark: The matrices  (log 2) + jπ A= 0

0 0



 ,

B=

2(log 2) + j2π 0

1 0



satisfy eAeB = eA+B = eBeA . Therefore, vi) =⇒ vii) does not hold. Furthermore, spec(A + B) = {3(log 2) + j3π, 0}, which is j2π congruence free. Therefore, since vii) does not hold, it follows from xii) that iii) does not hold, that is, AB = BA. In fact,   0 (log 2) + jπ . AB − BA = 0 0 Consequently, under the additional condition that spec(A + B) is j2π congruence free, vi) =⇒ iii) does not hold. However, xiv) implies that vi) =⇒ iii) holds under the additional assumption that A and B are Hermitian, in which case spec(A + B) is j2π congruence free. This example is due to Bourgeois.

748

CHAPTER 11

Fact 11.14.2. Let A ∈ Fn×n, B ∈ Fn×m, and C ∈ Fm×m. Then,   , t (t−τ )A τ C e Be dτ etA t[ A B ] 0 . e 0C = 0 etC Furthermore,

t

e

τA

dτ =



I

 

t[ A I ] 0 0 e 00 . I

0

Remark: The result can be extended to arbitrary upper block-triangular matrices. See [1393]. For an application to sampled-data control, see [1080]. Fact 11.14.3. Let A, B ∈ Rn×n. Then, 1

d A+tB e = dt

eτ (A+tB)Be(1−τ )(A+tB) dτ. 0

Hence,

. d A+tB .. Dexp(A;B) = e . . dt

t=0

Furthermore,

Hence,

1

eτABe(1−τ )A dτ.

= 0

  d tr eA+tB = tr eA+tBB . dt . .  A  d A+tB . tr e . = tr e B . dt t=0

Proof: See [174, p. 175], [454, p. 371], or [906, 1002, 1054]. Fact 11.14.4. Let A, B ∈ Fn×n. Then, .   adA d A+tB .. −I e e (B)eA = . dt adA t=0   I − e−adA (B) = eA adA ∞  1 = adkA(B)eA. (k+1)! k=0

Proof: The second and fourth expressions are given in [106, p. 49] and [767, p. 248], while the third expression appears in [1380]. See also [1400, pp. 107–110]. Remark: See Fact 2.18.6. Fact 11.14.5. Let A, B ∈ Fn×n, and assume that eA = eB. Then, the following statements hold: i) If |λ| < π for all λ ∈ spec(A) ∪ spec(B), then A = B. ii) If λ − μ = j2kπ for all λ ∈ spec(A), μ ∈ spec(B), and k ∈ Z, then [A, B] = 0.

THE MATRIX EXPONENTIAL AND STABILITY THEORY

749

iii) If A is normal and σmax (A) < π, then [A, B] = 0. iv) If A is normal and σmax (A) = π, then [A2, B] = 0. Proof: See [1203, 1239] and [1400, p. 111]. Remark: If [A, B] = 0, then [A2, B] = 0. Fact 11.14.6. Let A, B ∈ Fn×n, and assume that A and B are skew Hermitian. Then, etAetB is unitary, and there exists a skew-Hermitian matrix C(t) such that etAetB = eC(t). Problem: Does (11.4.1) converge in this case? See [231, 471, 1150]. Fact 11.14.7. Let A, B ∈ Fn×n, and assume that A and B are Hermitian. Then,  p 1/p p lim e 2 AepBe 2 A = eA+B. p→0

Proof: See [55]. Remark: This result is related to the Lie-Trotter formula given by Corollary 11.4.8. For extensions, see [10, 547]. Fact 11.14.8. Let A, B ∈ Fn×n, and assume that A and B are Hermitian. Then,   1/p 1 lim 12 epA + epB = e 2 (A+B). p→∞

Proof: See [197]. Fact 11.14.9. Let A, B ∈ Fn×n. Then,  1 1 k2 1 1 lim e k Ae k Be− k Ae− k B = e[A,B] . k→∞

Fact 11.14.10. Let A ∈ Fn×m, X ∈ Fm×l, and B ∈ Fl×n. Then, d tr eAXB = BeAXBA. dX Fact 11.14.11. Let A, B ∈ Fn×n. Then,

. d tA tB −tA −tB .. e e e e . =0 dt t=0

and

. d √tA √tB −√tA −√tB .. e e e e . = AB − BA. dt t=0

Fact 11.14.12. Let A, B, C ∈ Fn×n, assume there exists β ∈ F such that [A, B] = βB + C, and assume that [A, C] = [B, C] = 0. Then, eA+B = eAeφ(β)Beψ(β)C,

750

CHAPTER 11

where 

φ(β) =

 * 1 −β , β = 0, β 1−e 1,

and

* 

ψ(β) =



1 1 β2 − 21 ,

β = 0,  − β − e−β , β = 0, β = 0.

Proof: See [570, 1295]. Fact 11.14.13. Let A, B ∈ Fn×n, and assume there exist α, β ∈ F such that [A, B] = αA + βB. Then, et(A+B) = eφ(t)Aeψ(t)B, where

⎧ ⎪ t, α = β = 0, ⎪ ⎪ ⎨  α = β = 0, 1 + αt > 0, φ(t) = α−1 log(1 + αt), ⎪ ⎪ ,t ⎪ α−β ⎩ dτ, α = β, (α−β)τ 0 −β

αe

and

t 

e−βφ(τ ) dτ.

ψ(t) = 0

Proof: See [1296]. Fact 11.14.14. Let A, B ∈ Fn×n, and assume there exists nonzero β ∈ F such that [A, B] = αB. Then, for all t > 0, −αt

et(A+B) = etAe[(1−e

)/α]B

.

Proof: Apply Fact 11.14.12 with [tA, tB] = αt(tB) and β = αt. Fact 11.14.15. Let A, B ∈ Fn×n, and assume that [[A, B], A] = 0 and [[A, B], B] = 0. Then, for all t ∈ R, 2

etAetB = etA+tB+(t /2)[A,B]. In particular, 1

1

1

eAeB = eA+B+ 2 [A,B] = eA+Be 2 [A,B] = e 2 [A,B] eA+B and

eBe2AeB = e2A+2B .

Proof: See [639, pp. 64–66] and [1465]. Fact 11.14.16. Let A, B ∈ Fn×n , and assume that [A, B] = B 2. Then, eA+B = eA(I + B).

751

THE MATRIX EXPONENTIAL AND STABILITY THEORY

Fact 11.14.17. Let A, B ∈ Fn×n. Then, for all t ∈ [0, ∞), et(A+B) = etAetB +

∞ 

Ck tk,

k=2

where, for all k ∈ N, 

Ck+1 = and

1 k+1 ([A 

Dk+1 =



+ B]Ck + [B, Dk ]),

1 k+1 (ADk

C0 = 0, 

+ Dk B),

D0 = I.

Proof: See [1153]. Fact 11.14.18. Let A, B ∈ Fn×n. Then, for all t ∈ [0, ∞), et(A+B) = etA etBetC2 etC3 · · · , where

 − 21 [A, B], C2 =

 1 1 C3 = 3 [B, [A, B]] + 6 [A, [A, B]].

Remark: This result is the Zassenhaus product formula. See [701, p. 236] and [1206]. Remark: Higher order terms are given in [1206]. Remark: Conditions for convergence do not seem to be available. Fact 11.14.19. Let A ∈ R2n×2n, and assume that A is symplectic and discrete-time Lyapunov stable. Then, spec(A) ⊂ {s ∈ C: |s| = 1}, amultA (1) and amultA (−1) are even, A is semisimple, and there exists a Hamiltonian matrix B ∈ R2n×2n such that A = eB. Proof: Since A is symplectic and discrete-time Lyapunov stable, it follows that the spectrum of A is a subset of the unit circle and A is semisimple. Therefore, the only negative eigenvalue that A can have is −1. Since all nonreal eigenvalues appear in complex conjugate pairs and A has even order, and since, by Fact 3.20.10, det A = 1, it follows that the eigenvalues −1 and 1 (if present) have even algebraic multiplicity. The fact that A has a Hamiltonian logarithm now follows from Theorem 2.6 of [414]. Remark: See xiii) of Proposition 11.6.5. Fact 11.14.20. Let A, B ∈ Fn×n, assume that A is positive definite, and assume that B is positive semidefinite. Then, −1/2

A + B ≤ A1/2eA

BA−1/2 1/2

A

.

Hence,

−1 det(A + B) ≤ etr A B. det A Furthermore, for each inequality, equality holds if and only if B = 0.

Proof: For positive-semidefinite A it follows that I + A ≤ eA.

752

CHAPTER 11

Fact 11.14.21. Let A, B ∈ Fn×n, and assume that A and B are Hermitian. Then,   I ◦ (A + B) ≤ log eA ◦ eB . Proof: See [45, 1521]. Remark: See Fact 8.22.50. Fact 11.14.22. Let A, B ∈ Fn×n, assume that A and B are Hermitian, assume that A ≤ B, let α, β ∈ R, assume that either αI ≤ A ≤ βI or αI ≤ B ≤ βI, and let t > 0. Then, etA ≤ S(t, eβ−α )etB , where, for t > 0 and h > 0,

⎧ t t/(ht −1) ⎪ ⎨ (h − 1)h , h = 1,  etlog h S(t, h) = ⎪ ⎩1, h = 1.

Proof: See [531]. Remark: S(t, h) is Specht’s ratio. See Fact 1.12.22 and Fact 1.17.19. Fact 11.14.23. Let A, B ∈ Fn×n, assume that A and B are Hermitian, let α, β ∈ R, assume that αI ≤ A ≤ βI and αI ≤ B ≤ βI, and let t > 0. Then, tA

1/t 1 αe + (1 − α)etB β−α 1/t β−α S(1, e )S (t, e ) ≤ eαA+(1−α)B

1/t ≤ S(1, eβ−α ) αetA + (1 − α)etB , where S(t, h) is defined in Fact 11.14.22. Proof: See [531]. Fact 11.14.24. Let A, B ∈ Fn×n, and assume that A and B are positive definite. Then, log det A = tr log A and log det AB = tr(log A + log B). Fact 11.14.25. Let A, B ∈ Fn×n, and assume that A and B are positive definite. Then, tr(A − B) ≤ tr[A(log A − log B)] and

(log tr A − log tr B)tr A ≤ tr[A(log A − log B)].

Proof: See [163] and [201, p. 281]. Remark: The first inequality is Klein’s inequality. See [205, p. 118]. Remark: The second inequality is equivalent to the thermodynamic inequality. See Fact 11.14.31.

753

THE MATRIX EXPONENTIAL AND STABILITY THEORY

Remark: tr[A(log A − log B)] is the relative entropy of Umegaki. Fact 11.14.26. Let A, B ∈ Fn×n, assume that A and B are positive definite, and define 1  μ(A, B) = e 2 (log A+log B). Then, the following statements hold: i) μ(A, A−1 ) = I. ii) μ(A, B) = μ(B, A). iii) If AB = BA, then μ(A, B) = AB. Proof: See [77]. Remark: With multiplication defined by μ, the set of n×n positive-definite matrices is a commutative Lie group. See [77]. Fact 11.14.27. Let A, B ∈ Fn×n, assume that A and B are positive definite, and let p > 0. Then, p/2 p p/2 1 )] p tr[Alog(B A B

≤ tr[A(log A + log B)] ≤ p1 tr[Alog(Ap/2B pAp/2 )].

Furthermore, lim p1 tr[Alog(B p/2ApB p/2 )] = tr[A(log A + log B)] = lim p1 tr[Alog(Ap/2B pAp/2 )]. p↓0

p↓0

Proof: See [55, 164, 547, 692]. Remark: This inequality has applications to quantum information theory. Fact 11.14.28. Let A, B ∈ Fn×n, assume that A and B are Hermitian, let  q ≥ p > 0, let h = λmax (eA )/λmin (eB ), and define 

S(1, h) =

(h − 1)h1/(h−1) . e log h

Then, there exist unitary matrices U, V ∈ Fn×n such that A+B ∗ 1 U S(1,h) Ue

≤ e 2 AeBe 2 A ≤ S(1, h)V eA+BV ∗. 1

1

Furthermore, tr eA+B ≤ tr eAeB ≤ S(1, h) tr eA+B, p

p

q

q

tr (epA #epB )2/p ≤ tr eA+B ≤ tr (e 2 BepAe 2 B )1/p ≤ tr (e 2 BeqAe 2 B )1/q , p

p

tr eA+B = lim tr (e 2 BepAe 2 B )1/p, p↓0

eA+B = lim (epA #epB )2/p. p↓0

Moreover, tr eA+B = tr eAeB if and only if AB = BA. Furthermore, for all i ∈ {1, . . . , n}, A+B 1 ) ≤ λi (eAeB ) ≤ S(1, h)λi (eA+B ). S(1,h) λi (e

754

CHAPTER 11

Finally, let α ∈ [0, 1]. Then, lim (epA #α epB )1/p = e(1−α)A+αB p↓0

and

tr (epA #α epB )1/p ≤ tr e(1−α)A+αB .

Proof: See [256]. Remark: The left-hand inequality in the second string of inequalities is the GoldenThompson inequality. See Fact 11.16.4. Remark: Since S(1, h) > 1 for all h > 1, the left-hand inequality in the first string of inequalities does not imply the Golden-Thompson inequality.     Remark: For i = 1, the stronger eigenvalue inequality λmax eA+B ≤ λmax eAeB holds. See Fact 11.16.4. Remark: S(1, h) is Specht’s ratio given by Fact 11.14.22. Remark: The generalized geometric mean is defined in Fact 8.10.45. Fact 11.14.29. Let A, B ∈ Fn×n, and assume that A and B are Hermitian. Then,  A  tr(eAB)/tr eA tr e e ≤ tr eA+B. Proof: See [163]. Remark: This result is the Peierls-Bogoliubov inequality. Remark: This inequality is equivalent to the thermodynamic inequality. See Fact 11.14.31. Fact 11.14.30. Let A, B, C ∈ Fn×n, and assume that A, B, and C are positive definite. Then, ∞

tr e

log A−log B+log C

≤ tr A(B + xI)−1C(B + xI)−1 dx. 0

Proof: See [931, 958]. Remark: −log B is correct. Remark: tr eA+B+C ≤ |tr eAeBeC | is not necessarily true. Fact 11.14.31. Let A, B ∈ Fn×n, and assume that A is positive definite, tr A = 1, and B is Hermitian. Then, tr AB ≤ tr(A log A) + log tr eB. Furthermore, equality holds if and only if  −1 A = tr eB eB. Proof: See [163]. Remark: This result is the thermodynamic inequality. Equivalent forms are given by Fact 11.14.25 and Fact 11.14.29.

755

THE MATRIX EXPONENTIAL AND STABILITY THEORY

Fact 11.14.32. Let A, B ∈ Fn×n, and assume that A and B are Hermitian. Then, 1 1 A − BF ≤  log(e− 2AeBe 2 A )F . Proof: See [205, p. 203]. Remark: This result has a distance interpretation in terms of geodesics. See [205, p. 203] and [211, 1038, 1039]. Fact 11.14.33. Let A, B ∈ Fn×n, and assume that A and B are skew Hermitian. Then, there exist unitary matrices S1, S2 ∈ Fn×n such that −1

eAeB = eS1AS1

+S2BS2−1

.

Proof: See [1241, 1303, 1304]. Fact 11.14.34. Let A, B ∈ Fn×n, and assume that A and B are Hermitian. Then, there exist unitary matrices S1, S2 ∈ Fn×n such that 1

1

−1

e 2AeBe 2 A = eS1AS1

+S2BS2−1

.

Proof: See [1240, 1241, 1303, 1304]. Problem: Determine the relationship between this result and Fact 11.14.33. Fact 11.14.35. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and assume that B ≤ A. Furthermore, let p, q, r, t ∈ R, and assume that r ≥ t ≥ 0, p ≥ 0, p + q ≥ 0, and p + q + r > 0. Then, r A qA+pB r A t/(p+q+r) e2 e e2 ≤ etA. Proof: See [1383]. Fact 11.14.36. Let A ∈ Fn×n and B ∈ Fm×m. Then,    tr eA⊕B = tr eA tr eB . Fact 11.14.37. Let A ∈ Fn×n, B ∈ Fm×m, and C ∈ Fl×l. Then, eA⊕B⊕C = eA ⊗ eB ⊗ eC. Fact 11.14.38. Let A ∈ Fn×n, B ∈ Fm×m, C ∈ Fk×k, and D ∈ Fl×l. Then, tr eA⊗I⊗B⊗I+I⊗C⊗I⊗D = tr eA⊗Btr eC⊗D. Proof: By Fact 7.4.30, a similarity transformation involving the Kronecker permutation matrix can be used to reorder the inner two terms. See [1251]. Fact 11.14.39. Let A, B ∈ Rn×n, and assume that A and B are positive definite. Then, A#B is the unique positive-definite solution X of the matrix equation log(A−1X) + log(B −1X) = 0. Proof: See [1039].

756

CHAPTER 11

11.15 Facts on the Matrix Exponential and Eigenvalues, Singular Values, and Norms for One Matrix Fact 11.15.1. Let A ∈ Fn×n, assume that eA is positive definite, and assume that σmax (A) < 2π. Then, A is Hermitian. Proof: See [876, 1202].  n×n , and define f : [0, ∞) → (0, ∞) by f (t) = Fact  11.15.2. Let A ∈ F At σmax e . Then, f (0) = 12λmax (A + A∗ ).    Hence, there exists ε > 0 such that f (t) = σmax etA is decreasing on [0, ε) if and only if A is dissipative.

Proof: This result follows from iii) of Fact 11.15.7. See [1436]. Remark: The derivative is one sided. Fact 11.15.3. Let A ∈ Fn×n. Then, for all t ≥ 0, ∗ d tA 2 e F = tr etA(A + A∗ )etA . dt

 etA F is decreasing on [0, ∞). Hence, if A is dissipative, then f (t) =

Proof: See [1436]. Fact 11.15.4. Let A ∈ Fn×n. Then, 1/2  . 2A . .tr e . ≤ tr eAeA∗ ≤ tr eA+A∗ ≤ ntr e2(A+A∗) ≤ ∗

n 2

∗

+ 12 tr e2(A+A ).

∗

In addition, tr eAeA = tr eA+A if and only if A is normal. Proof: See [188], [730, p. 515], and [1239]. ∗

∗

Remark: tr eAeA ≤ tr eA+A is Bernstein’s inequality. See [49]. Remark: See Fact 3.7.12. Fact 11.15.5. Let A ∈ Fn×n. Then, for all k ∈ {1, . . . , n}, k k k k  1  ! ! !   ! ∗ ∗ 1 σi eA ≤ λi e 2(A+A ) = eλi[ 2 (A+A )] ≤ eσi (A). i=1

i=1

i=1

i=1

Furthermore, for all k ∈ {1, . . . , n}, k k k k  1        ∗ ∗ 1 σi eA ≤ λi e 2 (A+A ) = eλi[ 2(A+A )] ≤ eσi (A). i=1

In particular,

i=1

i=1

i=1

 1    ∗ ∗ 1 σmax eA ≤ λmax e 2 (A+A ) = e 2 λmax(A+A ) ≤ eσmax (A)

757

THE MATRIX EXPONENTIAL AND STABILITY THEORY

or, equivalently, ∗

∗

∗

λmax (eAeA ) ≤ λmax (eA+A ) = eλmax (A+A

)

≤ e2σmax (A).

. . . . .det eA . = .etr A . ≤ e|tr A| ≤ etr A

Furthermore, and

n @ A  eσi (A). tr eA ≤ i=1

Proof: See [1242], Fact 2.21.12, Fact 8.18.4, and Fact 8.18.5. F

Fact 11.15.6. Let A ∈ Fn×n, and let  ·  be a unitarily invariant norm on . Then, ∗ ∗ eAeA  ≤ eA+A .

n×n

In particular,

∗

∗

λmax (eAeA ) ≤ λmax (eA+A )

and

∗

∗

tr eAeA ≤ tr eA+A .

Proof: See [350]. Fact 11.15.7. Let A, B ∈ Fn×n, let  ·  be the norm on Fn×n induced by the norm  ·  on Fn, let mspec(A) = {λ1, . . . , λn }ms , and define 

μ(A) = lim ε↓0

I + εA − 1 . ε

Then, the following statements hold: 

i) μ(A) = D+f(A; I), where f : Fn×n → R is defined by f(A) = A. ii) μ(A) = limt↓0 t−1 log e tA  = supt>0 t−1 log e tA . . . + + . . iii) μ(A) = ddt etA . = ddt log etA . . t=0

t=0

iv) μ(I) = 1, μ(−I) = −1, and μ(0) = 0. v) spabs(A) = limt→∞ t−1 log e tA  = inf t>0 t−1 log e tA . vi) For all i ∈ {1, . . . , n}, −A ≤ −μ(−A) ≤ Re λi ≤ spabs(A) ≤ μ(A) ≤ A. vii) For all α ∈ R, μ(αA) = |α|μ[(sign α)A]. viii) For all α ∈ F, μ(A + αI) = μ(A) + Re α. ix) max{μ(A) − μ(−B), −μ(−A) + μ(B)} ≤ μ(A + B) ≤ μ(A) + μ(B). x) μ: Fn×n → R is convex. xi) |μ(A) − μ(B)| ≤ max{|μ(A − B)|, |μ(B − A)|} ≤ A − B. xii) For all x ∈ Fn, max{−μ(−A), −μ(A)}x ≤ Ax . xiii) If A is nonsingular, then max{−μ(−A), −μ(A)} ≤ 1/A−1.

758

CHAPTER 11

xiv) For all t ≥ 0 and all i = 1, . . . , n, e−At ≤ e−μ(−A)t ≤ e(Re λi )t ≤ espabs(A)t ≤ etA  ≤ eμ(A)t ≤ eAt. xv) μ(A) = min{β ∈ R: etA  ≤ eβt for all t ≥ 0}. xvi) If  ·  =  · 1 , and thus  ·  =  · col , then ⎛ μ(A) =

⎜ max ⎝Re A(j,j) +

j∈{1,...,n}

⎞ n 

⎟ |A(i,j) |⎠.

i=1 i=j

xvii) If  ·  =  · 2 and thus  ·  = σmax (·), then μ(A) = λmax[ 12 (A + A∗ )]. xviii) If  ·  =  · ∞ , and thus  ·  =  · row , then ⎛ μ(A) =

⎜ Re A(i,i) + max ⎜ i∈{1,...,n}⎝

n  j=1 j=i

⎞ ⎟ |A(i,j) |⎟ ⎠.

Proof: See [408, 412, 1094, 1276], [708, pp. 653–655], and [1348, p. 150]. Remark: μ(·) is the matrix measure, logarithmic derivative, or initial growth rate. For applications, see [708] and [1414]. See Fact 11.18.11 for the logarithmic derivative of an asymptotically stable matrix. Remark: The directional differential D+f(A; I) is defined in (10.4.2). Remark: vi) and xvii) yield Fact 5.11.24. Remark: Higher order logarithmic derivatives are studied in [209]. Fact 11.15.8. Let A ∈ Fn×n, let β > spabs(A), let γ ≥ 1, and let  ·  be a normalized, submultiplicative norm on Fn×n. Then, for all t ≥ 0, D tA D De D ≤ γeβt if and only if, for all k ≥ 1 and α > β, (αI − A)−k  ≤

γ . (α − β)k

Remark: This result is a consequence of the Hille-Yosida theorem. See [369, pp. 26] and [708, p. 672]. Fact 11.15.9. Let A ∈ Rn×n, let β ∈ R, and assume there exists a positivedefinite matrix P ∈ Rn×n such that ATP + PA ≤ 2βP. Then, for all t ≥ 0,

   σmax etA ≤ σmax (P )/σmin (P )eβt.

Remark: See [708, p. 665].

759

THE MATRIX EXPONENTIAL AND STABILITY THEORY

Remark: See Fact 11.18.9. Fact 11.15.10. Let A ∈ SO(3). Then, √  θ = 2 cos−1 ( 12 1 + tr A). Then,

1 θ = σmax (log A) = √  log AF . 2

Remark: See Fact 3.11.31 and Fact 11.11.10. Remark: θ is a Riemannian metric giving the length of the shortest geodesic curve on SO(3) between A and I. See [1038].

11.16 Facts on the Matrix Exponential and Eigenvalues, Singular Values, and Norms for Two or More Matrices Fact 11.16.1. Let A, B ∈ Fn×n. Then, . A+B . . ≤ tr e 12(A+B)e 21 (A+B)∗ .tr e 1

∗

1

∗

≤ tr e 2(A+A

+B+B ∗ ) 1

≤ tr e 2(A+A )e 2(B+B

∗

)

    ∗ 1/2 ∗ 1/2 tr eB+B ≤ tr eA+A   ∗ ∗ ≤ 12 tr eA+A + eB+B and tr eAeB   1 2A + e2B 2 tr e

-

    ∗ ∗ ∗ ∗ ≤ 12 tr eA eA + eB eB ≤ 12 tr eA+A + eB+B .

Proof: See [188, 351, 1102] and [730, p. 514]. Fact 11.16.2. Let A, B ∈ Fn×n. Then, for all p > 0,   1 1 p  1 ≤ 2p σmax ([A, B])eσmax (A)+σmax (B) . σmax eA+B − e p Ae p B Proof: See [701, p. 237] and [1040]. Remark: See Corollary 10.8.8 and Fact 11.16.3. 



Fact 11.16.3. Let A ∈ Fn×n, and define AH = 12 (A+A ∗ ) and AS = 12 (A−A ∗ ). Then, for all p > 0,  p   1 1 ∗ 1 1 σmax eA − e p AHe p AS ≤ 4p σmax ([A∗, A])e 2 λmax (A+A ). Proof: See [1040]. Remark: See Fact 10.8.8.

760

CHAPTER 11

Fact 11.16.4. Let A, B ∈ Fn×n, assume that A and B are Hermitian, and let  ·  be a unitarily invariant norm on Fn×n. Then, D D D D A+B D D 1 1 D D≤D De De 2AeBe 2A D ≤ DeAeB D . If, in addition, p > 0, then D D A+B D D p p D1/p De D≤D De 2 AeBe 2 A D D p D D A+B D p D1/p D = lim D De De 2 AeBe 2 A D .

and

p↓0

Furthermore, for all k ∈ {1, . . . , n}, k k k !  A+B  !  A B !   ≤ λi e λi e e ≤ σi eAeB i=1

i=1

i=1

with equality for k = n, that is, n n n !  A+B  !  A B !     = λi e λi e e = σi eAeB = det eAeB . i=1

i=1

i=1

In fact, det(eA+B ) = =

n !   λi eA+B i=1 n !

eλi (A+B)

i=1

= etr(A+B) = e(tr A)+(tr B) = etr A etr B = det(eA ) det(eB ) = det(eA eB ) n !   = σi eAeB . i=1

Furthermore, for all k ∈ {1, . . . , n}, k k k   A+B    A B    ≤ λi e λi e e ≤ σi eAeB . i=1

In particular,

i=1

i=1

      λmax eA+B ≤ λmax eAeB ≤ σmax eAeB , A @ tr eA+B ≤ tr eAeB ≤ tr eAeB ,

and, for all p > 0,

p

p

tr eA+B ≤ tr(e 2 AeBe 2 A ).

Finally, tr eA+B = tr eAeB if and only if A and B commute.

THE MATRIX EXPONENTIAL AND STABILITY THEORY

761

Proof: See [55], [201, p. 261], Fact 5.11.28, Fact 2.21.12, and Fact 9.11.2. For the last statement, see [1239]. Remark: Note that det(eA+B ) = det(eA ) det(eB ) even though eA+B and eAeB may not be equal. See [701, p. 265] or [730, p. 442]. Remark: 11.14.28.

tr eA+B ≤ tr eAeB is the Golden-Thompson inequality. 1

See Fact

1

Remark: eA+B  ≤ e 2AeBe 2 A  is Segal’s inequality. See [49]. A @ Problem: Compare the upper bound tr eAeB for tr eAeB with the upper bound S(1, h) tr eA+B given by Fact 11.14.28. Fact 11.16.5. Let A, B ∈ Fn×n, assume that A and B are Hermitian, let q, p > 0, where q ≤ p, and let  ·  be a unitarily invariant norm on Fn×n. Then, D D D D D q A qB q A 1/q D D p A pB p A 1/p D D e2 e e2 D ≤ D e2 e e2 D. D D D D Proof: See [55]. Fact 11.16.6. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, A

1/2 1/2 eσmax (AB) − 1 ≤ σmax (e − I)(eB − I) B

1/3 1/3 (e − I)(eA − I)(eB − I) . eσmax (BAB) − 1 ≤ σmax

and

Proof: See [1382]. Remark: See Fact 8.19.31. Fact 11.16.7. Let A, B ∈ Fn×n, and let t ≥ 0. Then, t

e

t(A+B)

=e

tA

e(t−τ )ABeτ (A+B) dτ.

+ 0

Proof: See [701, p. 238]. F

Fact 11.16.8. Let A, B ∈ Fn×n, and let  ·  be a submultiplicative norm on . Then, for all t ≥ 0,   D tA D De − etB D ≤ eAt eA−Bt − 1 .

n×n

Fact 11.16.9. Let A, B ∈ Fn×n, let  ·  be a normalized submultiplicative norm on Fn×n, and let t ≥ 0. Then, etA − etB  ≤ tA − Bet max{A,B}. Proof: See [701, p. 265].

762

CHAPTER 11

Fact 11.16.10. Let A, B ∈ Rn×n, and assume that A is normal. Then, for all t ≥ 0,      σmax etA − etB ≤ σmax etA eσmax (A−B)t − 1 . Proof: See [1454]. Fact 11.16.11. Let A ∈ Fn×n, let  ·  be an induced norm on Fn×n, and let α > 0 and β ∈ R be such that, for all t ≥ 0, etA  ≤ αeβt. Then, for all B ∈ Fn×n and t ≥ 0, et(A+B)  ≤ αe(β+αB)t. Proof: See [708, p. 406]. Fact 11.16.12. Let A, B ∈ Cn×n, assume that A and B are idempotent, assume that A = B, and let  ·  be a norm on Cn×n. Then, ejA − ejB  = |ej − 1|A − B < A − B. Proof: See [1055]. Remark: |ej − 1| ≈ 0.96. Fact 11.16.13. Let A, B ∈ Cn×n, assume that A and B are Hermitian, let X ∈ Cn×n, and let  ·  be a unitarily invariant norm on Cn×n. Then, ejAX − XejB  ≤ AX − XB. Proof: See [1055]. Remark: This result is a matrix version of x) of Fact 1.20.6. Fact 11.16.14. Let A ∈ Fn×n, and, for all i ∈ {1, . . . , n}, define fi : [0, ∞) →    R by fi (t) = log σi etA . Then, A is normal if and only if, for all i ∈ {1, . . . , n}, fi is convex. Proof: See [96] and [465]. Remark: The statement in [96] that convexity holds on R is erroneous. A coun 0  for which log σ1(etA ) = |t| and log σ2 (etA ) = −|t|. terexample is A = 10 −1 Fact 11.16.15. Let A ∈ Fn×n, and, for nonzero x ∈ Fn, define fx : R → R by   fx (t) = log σmax etA x . Then, A is normal if and only if, for all nonzero x ∈ Fn, fx is convex. 

Proof: See [96]. Remark: This result is due to Friedland. Fact 11.16.16. Let A, B ∈ Fn×n, assume that A and B are positive semidefinite, and let  ·  be a unitarily invariant norm on Fn×n. Then, eA−B − I ≤ eA − eB 

763

THE MATRIX EXPONENTIAL AND STABILITY THEORY

and

eA + eB  ≤ eA+B + I.

Proof: See [60] and [201, p. 294]. Remark: See Fact 9.9.54. Fact 11.16.17. Let A, X, B ∈ Fn×n, assume that A and B are Hermitian, and let  ·  be a unitarily invariant norm on Fn×n. Then, AX − XB ≤ e 2 AXe− 2 B − e− 2 BXe 2 A . 1

1

1

1

Proof: See [220]. Remark: See Fact 9.9.55.

11.17 Facts on Stable Polynomials Fact 11.17.1. Let a1, . . . , an be nonzero real numbers, let 

Δ = {i ∈ {1, . . . , n − 1} :

ai+1 ai

< 0},

let b1, . . . , bn be real numbers satisfying b1 < · · · < bn , define f : (0, ∞) → R by f (x) = an xbn + · · · + a1xb1, and define

 {x ∈ (0, ∞) : f (x) = 0}. S=

Furthermore, for all x ∈ S, define the multiplicity of x to be the positive integer m such that f (x) = f (x) = · · · = f (m−1) = 0 and f (m) (x) = 0, and let S denote the multiset consisting of all elements of S counting multiplicity. Then, card(S ) ≤ card(Δ). If, in addition, b1, . . . , bn are nonnegative integers, then card(Δ) − card(S ) is even. Proof: See [863, 1434]. Remark: This result is the Descartes rule of signs. Fact 11.17.2. Let p ∈ R[s], where p(s) = sn + an−1sn−1 + · · · + a0 . If p is asymptotically stable, then a0 , . . . , an−1 are positive. Now, assume in addition that a0 , . . . , an−1 are positive. Then, the following statements hold: i) If n = 1 or n = 2, then p is asymptotically stable. ii) If n = 3, then p is asymptotically stable if and only if a0 < a1a2 . iii) If n = 4, then p is asymptotically stable if and only if a21 + a0 a23 < a1a2 a3 .

764

CHAPTER 11

iv) If n = 5, then p is asymptotically stable if and only if a 2 < a3 a4 , a20 + a1a22 +

a22 + a1a24 < a0 a4 + a2 a3 a4 , a21 a24 + a0 a23 a4 < a0 a2 a3 + 2a0 a1a4

+ a1a2 a3 a4 .

Remark: These results are special cases of the Routh criterion, which provides stability criteria for polynomials of arbitrary degree n. See [309]. Remark: The Jury criterion for stability of continuous-time systems is given by Fact 11.20.1. Fact 11.17.3. Let ε ∈ [0, 1], let n ∈ {2, 3, 4}, let pε ∈ R[s], where pε (s) = sn + an−1sn−1 + · · · + a1s + εa0 , and assume that p1 is asymptotically stable. Then, for all ε ∈ (0, 1], pε is asymptotically stable. Furthermore, p0 (s)/s is asymptotically stable. Remark: This result does not hold for n = 5. A counterexample is p(s) = s5 + 2s4 + 3s3 + 5s2 + 2s + 2.5ε, which is asymptotically stable if and only if ε ∈ (4/5, 1]. This result is another instance of the quartic barrier. See [359], Fact 8.14.7, and Fact 8.15.33.  Fact 11.17.4. Let p ∈ R[s] be monic, and define q(s) = sn p(1/s), where  n = deg p. Then, p is asymptotically stable if and only if q is asymptotically stable.

Remark: See Fact 4.8.1 and Fact 11.17.5. Fact 11.17.5. Let p ∈ R[s] be monic, and assume that p is semistable. Then,   q(s) = p(s)/s and qˆ(s) = sn p(1/s) are asymptotically stable. Remark: See Fact 4.8.1 and Fact 11.17.4. Fact 11.17.6. Let p, q ∈ R[s], assume that p is even, assume that q is odd, and assume that every coefficient of p + q is positive. Then, p + q is asymptotically stable if and only if every root of p and every root of q is imaginary, and the roots of p and the roots of q are interlaced on the imaginary axis. Proof: See [225, 309, 723]. Remark: This result is the Hermite-Biehler or interlacing theorem. Example: s2 + 2s + 5 = (s2 + 5) + 2s. Fact 11.17.7. Let p ∈ R[s] be asymptotically stable, and let p(s) = βnsn + βn−1s + · · · + β1s + β0 , where βn > 0. Then, for all i ∈ {1, . . . , n − 2}, n−1

βi−1 βi+2 < βi βi+1 . Remark: This result is a necessary condition for asymptotic stability, which can be used to show that a given polynomial with positive coefficients is unstable. Remark: This result is due to Xie. See [1509]. For alternative conditions, see [225, p. 68].

765

THE MATRIX EXPONENTIAL AND STABILITY THEORY 

Fact 11.17.8. Let n ∈ P be even, let m = n/2, let p ∈ R[s], where p(s) = βns + βn−1sn−1 + · · · + β1s + β0 and βn > 0, and assume that p is asymptotically stable. Then, for all i ∈ {1, . . . , m − 1}, m (m−i)/m i/m βn ≤ β2i . i β0 n

Remark: This result is a necessary condition for asymptotic stability, which can be used to show that a given polynomial with positive coefficients is unstable. Remark: This result is due to Borobia and Dormido. See [1509, 1510] for extensions to polynomials of odd degree. Fact 11.17.9. Let p, q ∈ R[s], where p(s) = αn sn + αn−1sn−1 + · · · + α1s + α0 and q(s) = βmsm + βm−1sm−1 + · · · + β1s + β0 . If p and q are (Lyapunov,  asymptotically) stable, then r(s) = αl βlsl + αl−1 βl−1sl−1 + · · · + α1 β1s + α0 β0 ,  where l = min{m, n}, is (Lyapunov, asymptotically) stable. Proof: See [557]. Remark: The polynomial r is the Schur product of p and q. See [85, 785]. Fact 11.17.10. Let A ∈ Rn×n, and assume that A is diagonalizable over R. Then, χA has all positive coefficients if and only if A is asymptotically stable. Proof: Sufficiency follows from Fact 11.17.2. For necessity, note that all of the roots of χA are real and that χA(λ) > 0 for all λ ≥ 0. Hence, roots(χA ) ⊂ (−∞, 0). Fact 11.17.11. Let A ∈ Rn×n. Then, the following statements are equivalent: i) χA⊕A has all positive coefficients. ii) χA⊕A is asymptotically stable. iii) A ⊕ A is asymptotically stable. iv) A is asymptotically stable. Proof: If A is not asymptotically stable, then Fact 11.18.32 implies that A ⊕ A has a nonnegative eigenvalue λ. Since χA⊕A(λ) = 0, it follows that χA⊕A cannot have all positive coefficients. See [532, Theorem 5]. Remark: A similar method of proof is used in Proposition 8.2.7. Fact 11.17.12. Let A ∈ Rn×n. Then, the following statements are equivalent: i) χA and χA(2,1) have all positive coefficients. ii) A is asymptotically stable. Proof: See [1274]. Remark: The additive compound A(2,1) is defined in Fact 7.5.17.

766

CHAPTER 11

Fact 11.17.13. For i = 1, . . . , n − 1, let ai , bi ∈ R satisfy 0 < ai ≤ bi , define φ1, φ2 , ψ1, ψ2 ∈ R[s] by φ1 (s) = bn sn + an−2 sn−2 + bn−4 sn−4 + · · · , φ2 (s) = an sn + bn−2 sn−2 + an−4 sn−4 + · · · , ψ1 (s) = bn−1 sn−1 + an−3 sn−3 + bn−5 sn−5 + · · · , ψ2 (s) = an−1 sn−1 + bn−3 sn−3 + an−5 sn−5 + · · · , assume that φ1 + ψ1, φ1 + ψ2 , φ2 + ψ1, and φ2 + ψ2 are asymptotically stable, let p ∈ R[s], where p(s) = βnsn + βn−1sn−1 + · · · + β1s + β0 , and assume that, for all i ∈ {1, . . . , n}, ai ≤ βi ≤ bi . Then, p is asymptotically stable. Proof: See [459, pp. 466, 467]. Remark: This result is Kharitonov’s theorem.

11.18 Facts on Stable Matrices Fact 11.18.1. Let A ∈ Fn×n, and assume that A is semistable. Then, A is Lyapunov stable. Fact 11.18.2. Let A ∈ Fn×n, and assume that A is Lyapunov stable. Then, A is group invertible. Fact 11.18.3. Let A ∈ Fn×n, and assume that A is semistable. Then, A is group invertible. Fact 11.18.4. Let A, B ∈ Fn×n, and assume that A and B are similar. Then, A is (Lyapunov stable, semistable, asymptotically stable, discrete-time Lyapunov stable, discrete-time semistable, discrete-time asymptotically stable) if and only if B is. Fact 11.18.5. Let A ∈ Rn×n, and assume that A is Lyapunov stable. Then, t

eτA dτ = I − AA#.

lim 1 t→∞ t 0 Remark: See Fact 11.18.2.

Fact 11.18.6. Let A ∈ Fn×n, and assume that A is semistable. Then, lim etA = I − AA#,

t→∞

and thus

t

lim 1 t→∞ t 0

eτA dτ = I − AA#.

Remark: See Fact 10.11.6, Fact 11.18.1, and Fact 11.18.2.

767

THE MATRIX EXPONENTIAL AND STABILITY THEORY

Fact 11.18.7. Let A, B ∈ Fn×n. Then, limα→∞ eA+αB exists if and only if B is semistable. In this case,     # # lim eA+αB = e(I−BB )A I − BB # = I − BB # eA(I−BB ). α→∞

Proof: See [292]. Fact 11.18.8. Let A ∈ Fn×n, assume that A is asymptotically stable, let β > spabs(A), and let  ·  be a submultiplicative norm on Fn×n. Then, there exists γ > 0 such that, for all t ≥ 0, D tA D De D ≤ γeβt. Remark: See [572, pp. 201–206] and [808]. Fact 11.18.9. Let A ∈ Rn×n, assume that A is asymptotically stable, let β ∈ (spabs(A), 0), let P ∈ Rn×n be positive definite and satisfy ATP + PA ≤ 2βP, and let  ·  be a normalized, submultiplicative norm on Rn×n. Then, for all t ≥ 0, D tA D  De D ≤ P P −1eβt. Remark: See [707]. Remark: See Fact 11.15.9. Fact 11.18.10. Let A ∈ Fn×n, assume that A is asymptotically stable, let R ∈ Fn×n, assume that R is positive definite, and let P ∈ Fn×n be the positivedefinite solution of A∗P + PA + R = 0. Then, ?  tA  σmax (P ) −tλmin (RP −1 )/2 e σmax e ≤ σmin (P ) and

etAF ≤

 −1 P FP −1F e−tλmin (RP )/2.

If, in addition, A + A∗ is negative definite, then ∗

etAF ≤ e−tλmin(−A−A

)/2

.

Proof: See [977]. Fact 11.18.11. Let A ∈ Rn×n, assume that A is asymptotically stable, let R ∈ Rn×n, assume that R is positive definite, and let P ∈ Rn×n be the positive definite solution of ATP + PA + R = 0. Furthermore, define the vector norm x = √ xTPx on Rn, let  ·  denote the induced norm on Rn×n, and let μ(·) denote the corresponding logarithmic derivative. Then, μ(A) = −λmin (RP −1)/2. Consequently,

etA ≤ e−tλmin (RP

Proof: See [747] and use xiv) of Fact 11.15.7.

−1

)/2

.

768

CHAPTER 11

Remark: See Fact 11.15.7 for the definition and properties of the logarithmic derivative. Fact 11.18.12. Let A ∈ Fn×n. Then, A is similar to a skew-Hermitian matrix if and only if there exists a positive-definite matrix P ∈ Fn×n such that A∗P +PA = 0. Remark: See Fact 5.9.6. Fact 11.18.13. Let A ∈ Rn×n. Then, A and A2 are asymptotically stable if    5π 3π  and only if, for all λ ∈ spec(A), there exist r > 0 and θ ∈ π2 , 3π ∪ such , 4 4 2 that λ = rejθ. / spec(A) Fact 11.18.14. Let A ∈ Rn×n. Then, A is group invertible and j2kπ ∈ for all k ≥ 1 if and only if   # AA# = eA − I eA − I . In particular, if A is semistable, then this equality holds. Proof: Use ii) of Fact 11.21.12 and ix) of Proposition 11.8.2. Fact 11.18.15. Let A ∈ Fn×n. Then, A is asymptotically stable if and only if −1 A is asymptotically stable. Hence, etA → 0 as t → ∞ if and only if etA → 0 as t → ∞. −1

Fact 11.18.16. Let A, B ∈ Rn×n, assume that A is asymptotically stable, and assume that σmax (B ⊕ B) < σmin (A ⊕ A). Then, A + B is asymptotically stable. Proof: Since A ⊕ A is nonsingular, Fact 9.14.18 implies that A ⊕ A + α(B ⊕ B) = (A + αB) ⊕ (A + αB) is nonsingular for all 0 ≤ α ≤ 1. Now, suppose that A + B is not asymptotically stable. Then, there exists α0 ∈ (0, 1] such that A + α0 B has an imaginary eigenvalue, and thus (A + α0 B) ⊕ (A + α0 B) = A ⊕ A + α0 (B ⊕ B) is singular, which is a contradiction. Remark: This result provides a suboptimal solution of a nearness problem. See [697, Section 7] and Fact 9.14.18. Fact 11.18.17. Let A ∈ Cn×n, assume that A is asymptotically stable, let  ·  denote either σmax (·) or  · F , and define 

β(A) = {B: B ∈ Cn×n and A + B is not asymptotically stable}. Then, 1 2 σmin (A ⊗ A)

≤ β(A) = min σmin (A + γjI) γ∈R

≤ min{spabs(A), σmin (A), 12 σmax(A + A∗ )}. Furthermore, let R ∈ Fn×n, assume that R is positive definite, and let P ∈ Fn×n be the positive-definite solution of A∗P + PA + R = 0. Then, 1 2 σmin (R)/P 

≤ β(A).

769

THE MATRIX EXPONENTIAL AND STABILITY THEORY

If, in addition, A + A∗ is negative definite, then − 12 λmin(A + A∗ ) ≤ β(A). Proof: See [697, 1394]. Remark: The analogous problem for real matrices and real perturbations is discussed in [1135]. Fact 11.18.18. Let A ∈ Fn×n, assume that A is asymptotically stable, let V ∈ Fn×n, assume that V is positive definite, and let Q ∈ Fn×n be the positivedefinite solution of AQ + QA∗ + V = 0. Then, for all t ≥ 0, ∗

etA 2F = tr etAetA ≤ κ(Q)tr e−tS

−1

V S −∗

≤ κ(Q)tr e−[t/σmax (Q)]V ,



where S ∈ Fn×n satisfies Q = SS ∗ and κ(Q) = σmax (Q)/σmin (Q). If, in particular, A satisfies AQ + QA∗ + I = 0, then etA 2F ≤ nκ(Q)e−t/σmax (Q). Proof: See [1503]. ∗

∗

Remark: Fact 11.15.4 yields etAetA ≤ et(A+A ). However, this bound is poor in the case in which A + A∗ is not asymptotically stable. See [189]. Remark: See Fact 11.18.19. Fact 11.18.19. Let A ∈ Fn×n, assume that A is asymptotically stable, and let Q ∈ Fn×n be the positive-definite solution of AQ + QA∗ + I = 0. Then, for all t ≥ 0, 2 σmax (etA ) ≤ κ(Q)e−t/σmax (Q),  where κ(Q) = σmax (Q)/σmin (Q).

Proof: See references in [1411, 1412]. √ Remark: Since etA F ≤ nσmax (etA ), it follows that this inequality implies the last inequality in Fact 11.18.18. Fact 11.18.20. Let A ∈ Rn×n, and assume that every entry of A ∈ Rn×n is positive. Then, A is unstable. Proof: See Fact 4.11.4. Fact 11.18.21. Let A ∈ Rn×n. Then, A is asymptotically stable if and only if there exist matrices B, C ∈ Rn×n such that B is positive definite, C is dissipative, and A = BC.   Proof: A = P −1 −ATP − R . Remark: To reverse the order of factors, consider AT.

770

CHAPTER 11

Fact 11.18.22. Let A ∈ Fn×n. Then, the following statements hold: i) All of the real eigenvalues of A are positive if and only if A is the product of two dissipative matrices. ii) A is nonsingular and A = αI for all α < 0 if and only if A is the product of two asymptotically stable matrices. iii) A is nonsingular if and only if A is the product of three or fewer asymptotically stable matrices. Proof: See [130, 1494]. Fact 11.18.23. Let p ∈ R[s], where p(s) = sn + βn−1sn−1 + · · · + β1s + β0 and β0 , . . . , βn > 0. Furthermore, define A ∈ Rn×n by ⎡ ⎤ βn−1 βn−3 βn−5 βn−7 · · · · · · 0 ⎢ 1 βn−2 βn−4 βn−6 · · · · · · 0 ⎥ ⎢ ⎥ ⎢ 0 βn−1 βn−3 βn−5 · · · · · · 0 ⎥ ⎢ ⎥  ⎢ 1 βn−2 βn−4 · · · · · · 0 ⎥ A= ⎢ 0 ⎥. ⎢ .. .. .. ⎥ .. .. .. .. ⎢ . . . . ⎥ . . . ⎢ ⎥ ⎣ 0 0 0 ··· · · · β1 0 ⎦ 0 0 0 ··· · · · β2 β 0 If p is Lyapunov stable, then every subdeterminant of A is nonnegative. Proof: See [85]. Remark: A is totally nonnegative. Furthermore, p is asymptotically stable if and only if every leading principal subdeterminant of A is positive. Remark: The second statement is due to Hurwitz. Remark: The diagonal entries of A are βn−1 , . . . , β0 . Problem: Show that this condition for stability is equivalent to the condition given ˆ in [494, p. 183] in terms of an alternative matrix A. Fact 11.18.24. Let A ∈ Rn×n, assume that A is tridiagonal, and assume that A(i,i) > 0 for all i ∈ {1, . . . , n} and A(i,i+1) A(i+1,i) > 0 for all i = 1, . . . , n − 1. Then, A is asymptotically stable. Proof: See [295]. Remark: This result is due to Barnett and Storey. Fact 11.18.25. Let A ∈ Rn×n, and assume that A is cyclic. Then, there exists  a nonsingular matrix S ∈ Rn×n such that AS = SAS −1 is given by the tridiagonal matrix

771

THE MATRIX EXPONENTIAL AND STABILITY THEORY

⎡

1

0

···

0

0

0

1

0

0

−αn−1 .. .

0 .. .

0 ..

0 .. .

0

0

··· .. . .. . .. .

0

1

0

0

···

−α2

−α1

0

⎢ ⎢ −αn ⎢ ⎢ ⎢ 0 ⎢ AS = ⎢ . ⎢ .. ⎢ ⎢ ⎢ 0 ⎣ 0

.

⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥, ⎥ ⎥ ⎥ ⎥ ⎦

where α1, . . . , αn are real numbers. If α1α2 · · · αn = 0, then the number of eigenvalues of A in the OLHP is equal to the number of positive elements in {α1, α1α2 , . . . , α1α2 · · · αn }ms . Furthermore, AT SP + PAS + R = 0, where 

P = diag(α1α2 · · · αn , α1α2 · · · αn−1 , . . . , α1α2 , α1 )    diag 0, . . . , 0, 2α21 . R=

and

Finally, AS is asymptotically stable if and only if α1, . . . , αn are positive. Proof: See [150, pp. 52, 95]. Remark: AS is in Schwarz form. See Fact 11.18.26 and Fact 11.18.27. Fact 11.18.26. Let α1, . . . , αn be real numbers, and ⎡ 0 1 0 ··· 0 ⎢ ⎢ −αn 0 1 ··· 0 ⎢ ⎢ . .. ⎢ 0 0 −αn−1 0 ⎢ A=⎢ . . . . . ⎢ .. .. .. .. .. ⎢ ⎢ . .. ⎢ 0 0 0 0 ⎣ 0

0

···

0

−α2

define A ∈ Rn×n by ⎤ 0 ⎥ 0 ⎥ ⎥ ⎥ 0 ⎥ ⎥ . .. ⎥ . ⎥ ⎥ ⎥ 1 ⎥ ⎦ α1

Then, spec(A) ⊂ ORHP if and only if α1, . . . , αn are positive. Proof: See [730, p. 111]. Remark: Note the absence of the minus sign in the (n, n) entry compared to the matrix in Fact 11.18.25. This minus sign changes the sign of all eigenvalues of A. Fact 11.18.27. Let α1, α2 , α3 > 0, and define AR , P, R ∈ R3×3 by the tridiagonal matrix ⎡ ⎤ 1/2 −α1 α2 0 ⎢ ⎥  1/2 1/2 AR = ⎢ 0 α3 ⎥ ⎣ −α2 ⎦ 0

1/2

−α3

0

and the diagonal matrices  P = I,

Then, AT RP + PAR + R = 0.

 R= diag(2α1, 0, 0).

772

CHAPTER 11

Remark: The matrix AR is in Routh form. The Routh form AR and the Schwarz −1 form AS are related by AR = SRS AS SRS , where ⎡ ⎤ 1/2 0 0 α1 ⎢ ⎥  SRS = ⎢ 0 −(α1α2 )1/2 0 ⎥ ⎣ ⎦. (α1α2 α3 )1/2 0 0 Remark: See Fact 11.18.25. Fact 11.18.28. Let α1, α2 , α3 > 0, and define AC , P, R ∈ R3×3 by the tridiagonal matrix ⎡ ⎤ 0 0 1/a3 ⎥  ⎢ 0 1/a2 ⎦ AC = ⎣ −1/a2 −1/a1

0

−1/a1

and the diagonal matrices  P = diag(a3 , a2 , a1 ), 



 R= diag(0, 0, 2),



where a1 = 1/α1, a2 = α1/α2 , and a3 = α2 /(α1α3 ). Then, ATC P + PAC + R = 0. Proof: See [321, p. 346]. Remark: The matrix AC is in Chen form. The Schwarz form AS and the Chen −1 form AC are related by AS = SSC AC SSC , where ⎡ ⎤ 0 0 1/(α1α3 ) ⎥  ⎢ 0 1/α2 0 ⎦. SSC = ⎣ 0

0

1/α1

Remark: The Schwarz, Routh, and Chen forms provide the basis for the Routh criterion. See [34, 274, 321, 1100]. Remark: A circuit interpretation of the Chen form is given in [990].

by

Fact 11.18.29. Let α1, . . . , αn > ⎡ −α1 0 ⎢ ⎢ β2 −α2 ⎢ ⎢ ⎢ . .. A = ⎢ .. . ⎢ ⎢ ⎢ 0 0 ⎣ 0

Then,

0

0 and β1, . . . , βn > 0, and define A ∈ Rn×n ⎤ ··· 0 −β1 ⎥ ··· 0 0 ⎥ ⎥ ⎥ .. .. ⎥ .. . . . . ⎥ ⎥ ⎥ .. . −αn−1 0 ⎥ ⎦ −αn ··· βn

χA (s) = (s + α1 )(s + α2 ) · · · (s + αn ) + β1 β2 · · · βn .

Furthermore, if (cos π/n)n < then A is asymptotically stable.

α1 · · · αn , β1 · · · βn

THE MATRIX EXPONENTIAL AND STABILITY THEORY

773

Proof: See [1244]. Remark: If n = 2, then A is asymptotically stable for all positive α1, β1, α2 , β2 . Remark: This result is the secant condition. Fact 11.18.30. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A is asymptotically stable. ii) There exist a negative-definite matrix B ∈ Fn×n, a skew-Hermitian matrix C ∈ Fn×n, and a nonsingular matrix S ∈ Fn×n such that A = B + SCS −1. iii) There exist a negative-definite matrix B ∈ Fn×n, a skew-Hermitian matrix C ∈ Fn×n, and a nonsingular matrix S ∈ Fn×n such that A = S(B+C)S −1. Proof: See [378]. Then, there exist asymptotically Fact 11.18.31. Let A ∈ Rn×n, and let k ≥ 2.  k stable matrices A1, . . . , Ak ∈ Rn×n such that A = i=1 Ai if and only if tr A < 0. Proof: See [769]. Fact 11.18.32. Let A ∈ Cn×n. Then, A is (Lyapunov stable, semistable, asymptotically stable) if and only if A ⊕ A is.  ∗ Proof: Use Fact 7.5.7 and the fact that vec etAVetA = et(A⊕A) vec V. Fact 11.18.33. Let A ∈ Rn×n and B ∈ Rm×m. Then, the following statements hold: i) If A and B are (Lyapunov stable, semistable, asymptotically stable), then so is A ⊕ B. ii) If A ⊕ B is (Lyapunov stable, semistable, asymptotically stable), then so is either A or B. Proof: Use Fact 7.5.7. Fact 11.18.34. Let A ∈ Rn×n, and assume that A is asymptotically stable. Then, ∞ (A ⊕ A)−1 =

(jωI − A)−1 ⊗ (jωI − A)−1 dω

−∞

and

∞

(ω 2I + A2 ) dω = −πA−1. −∞ −1

Proof: Use (jωI − A)

+ (−jωI − A)−1 = −2A(ω 2I + A2 )−1.

Fact 11.18.35. Let A ∈ R2×2. Then, A is asymptotically stable if and only if tr A < 0 and det A > 0.

774

CHAPTER 11

Fact 11.18.36. Let A ∈ Cn×n. Then, there exists a unique asymptotically stable matrix B ∈ Cn×n such that B 2 = −A. Remark: This result is stated in [1262]. The uniqueness of the square root for complex matrices that have no eigenvalues in (−∞, 0] is implicitly assumed in [1263]. Remark: See Fact 5.15.19. Fact 11.18.37. Let A ∈ Rn×n. Then, the following statements hold: i) If A is semidissipative, then A is Lyapunov stable. ii) If A is dissipative, then A is asymptotically stable. iii) If A is Lyapunov stable and normal, then A is semidissipative. iv) If A is asymptotically stable and normal, then A is dissipative. v) If A is discrete-time Lyapunov stable and normal, then A is semicontractive. R

r×r

Fact 11.18.38. Let M ∈ Rr×r, assume that M is positive definite, let C, K ∈ , assume that C and K are positive semidefinite, and consider the equation

Furthermore, define

M q¨ + Cq˙ + Kq = 0.   0 I  . A= −M −1K −M −1C

Then, the following statements hold: i) A is Lyapunov stable if and only if C + K is positive definite. C ] = r. ii) A is Lyapunov stable if and only if rank [ K

iii) A is semistable if and only if (M −1K, C) is observable. iv) A is asymptotically stable if and only if A is semistable and K is positive definite. Proof: See [190]. Remark: See Fact 5.12.21.

11.19 Facts on Almost Nonnegative Matrices Fact 11.19.1. Let A ∈ Rn×n. Then, etA is nonnegative for all t ≥ 0 if and only if A is almost nonnegative. Proof: Let α > 0 be such that αI + A is nonnegative, and consider et(αI+A). See [185, p. 74], [186, p. 146], [194, 373], or [1228, p. 37]. Fact 11.19.2. Let A ∈ Rn×n, and assume that A is almost nonnegative. Then, etA is positive for all t > 0 if and only if A is irreducible.

THE MATRIX EXPONENTIAL AND STABILITY THEORY

775

Proof: See [1215, p. 208]. Fact 11.19.3. Let A ∈ Rn×n, where n ≥ 2, and assume that A is almost nonnegative. Then, the following statements are equivalent: i) There exist α ∈ (0, ∞) and B ∈ Rn×n such that A = B − αI, B is nonnegative, and sprad(B) ≤ α. ii) spec(A) ⊂ OLHP ∪ {0}. iii) spec(A) ⊂ CLHP. iv) If λ ∈ spec(A) is real, then λ ≤ 0. v) Every principal subdeterminant of −A is nonnegative. vi) For every diagonal, positive-definite matrix B ∈ Rn×n, it follows that A−B is nonsingular. Remark: A is an N-matrix if A is almost nonnegative and i)–vi) hold. Remark: This result follows from Fact 4.11.8. Example: A = [ 00 10 ]. Fact 11.19.4. Let A ∈ Rn×n, where n ≥ 2, and assume that A is almost nonnegative. Then, the following conditions are equivalent: i) A is a group-invertible N-matrix. ii) A is a Lyapunov-stable N-matrix. iii) A is a semistable N-matrix. iv) A is Lyapunov stable. v) A is semistable. vi) A is an N-matrix, and there exist α ∈ (0, ∞) and a nonnegative matrix B ∈ Rn×n such that A = B − αI and α−1B is discrete-time semistable. vii) There exists a positive-definite matrix P ∈ Rn×n such that ATP + PA is negative semidefinite. Furthermore, consider the following statements: viii) There exists a positive vector p ∈ Rn such that −Ap is nonnegative. ix) There exists a nonzero nonnegative vector p ∈ Rn such that −Ap is nonnegative. Then, viii) =⇒ [i)–vii)] =⇒ ix). Proof: See [186, pp. 152–155] and [187]. The statement [i)–vii)] =⇒ ix) is given by Fact 4.11.12. 1 Remark: The converse of viii) =⇒ [i)–vii)] does not hold. For example, A = 00 −1 is almost negative and semistable, but there does not exist a positive vector p ∈ R2 such that −Ap is nonnegative. However, note that viii) holds for AT, but not for diag(A, AT ) or its transpose.

776

CHAPTER 11

Remark: A discrete-time semistable matrix is called semiconvergent in [186, p. 152]. Remark: The last statement follows from the fact that the function V (x) = pTx is a Lyapunov function for the system x˙ = −Ax for x ∈ [0, ∞)n with Lyapunov derivative V˙ (x) = −ATp. See [191, 630]. Fact 11.19.5. Let A ∈ Rn×n, where n ≥ 2, and assume that A is almost nonnegative. Then, the following conditions are equivalent: i) A is a nonsingular N-matrix. ii) A is asymptotically stable. iii) A is an asymptotically stable N-matrix. iv) There exist α ∈ (0, ∞) and a nonnegative matrix B ∈ Rn×n such that A = B − αI and sprad(B) < α. v) If λ ∈ spec(A) is real, then λ < 0. vi) If B ∈ Rn×n is nonnegative and diagonal, then A − B is nonsingular. vii) Every principal subdeterminant of −A is positive. viii) Every leading principal subdeterminant of −A is positive. ix) For all i ∈ {1, . . . , n}, the sign of the ith leading principal subdeterminant of A is (−1)i. x) For all k ∈ {1, . . . , n}, the sum of all k × k principal subdeterminants of −A is positive. xi) There exists a positive-definite matrix P ∈ Rn×n such that ATP + PA is negative definite. xii) There exists a positive vector p ∈ Rn such that −Ap is positive. xiii) There exists a nonnegative vector p ∈ Rn such that −Ap is positive. xiv) If p ∈ Rn and −Ap is nonnegative, then p ≥≥ 0 is nonnegative. xv) For every nonnegative vector y ∈ Rn , there exists a unique nonnegative vector x ∈ Rn such that Ax = −y. xvi) A is nonsingular and −A−1 is nonnegative. Proof: See [185, pp. 134–140] or [730, pp. 114–116]. Remark: −A is a nonsingular M-matrix. See Fact 4.11.8. Fact 11.19.6. For i, j = 1, . . . , n, let σij ∈ [0, ∞), and define A ∈ Rn×n by n   A(i,j) = σij for all i = j and A(i,i) = − j=1 σij . Then, the following statements hold: i) A is almost nonnegative.

T is nonnegative. ii) −A1n×1 = σ11 . . . σnn iii) spec(A) ⊂ OLHP ∪ {0}.

THE MATRIX EXPONENTIAL AND STABILITY THEORY

777

iv) A is an N-matrix. v) A is a group-invertible N-matrix. vi) A is a Lyapunov-stable N-matrix. vii) A is a semistable N-matrix. If, in addition, σ11 , . . . , σnn are positive, then A is a nonsingular N-matrix. Proof: It follows from the Gershgorin circle theorem given by  Fact 4.10.17 that every eigenvalue λ of A is an element of a disk in C centered at − nj=1 σij ≤ 0 and n with radius j=1,j=i σij . Hence, if σii = 0, then either λ = 0 or Re λ < 0, whereas, if σii > 0, then Re λ ≤ σii < 0. Thus, iii) holds. Statements iv)–vii) follow from ii) and Fact 11.19.4. The last statement follows from the Gershgorin circle theorem. Remark: AT is a compartmental matrix. See [194, 632, 1421]. Problem: Determine necessary and sufficient conditions on the parameters σij such that A is a nonsingular N-matrix. Fact 11.19.7. Let G = (X, R) be a graph, where X = {x1, . . . , xn }, and let L ∈ Rn×n denote either the inbound Laplacian or the outbound Laplacian of G. Then, the following statements hold: i) −L is semistable. ii) limt→∞ e−Lt exists. Remark: Use Fact 11.19.6. Remark: The spectrum of the Laplacian is discussed in [7]. Fact 11.19.8. Let A ∈ Rn×n, and assume that A is asymptotically stable. Then, at least one of the following statements holds: i) All of the diagonal entries of A are negative. ii) At least one diagonal entry of A is negative and at least one off-diagonal entry of A is negative. Proof: See [519]. Remark: sign stability is discussed in [773].

11.20 Facts on Discrete-Time-Stable Polynomials Fact 11.20.1. Let p ∈ R[s], where p(s) = sn + an−1sn−1 + · · · + a0 . Then, the following statements hold: i) If n = 1, then p is discrete-time asymptotically stable if and only if |a0 | < 1. ii) If n = 2, then p is discrete-time asymptotically stable if and only if |a0 | < 1 and |a1 | < 1 + a0 . iii) If n = 3, then p is discrete-time asymptotically stable if and only if |a0 | < 1, |a0 + a2 | < |1 + a1 |, and |a1 − a0 a2 | < 1 − a20 .

778

CHAPTER 11

Proof: See [140, p. 185], [289, p. 6], [708, p. 355], or [804, pp. 34, 35]. Remark: These results are the Schur-Cohn criterion. Conditions for polynomials of arbitrary degree n follow from the Jury criterion. Remark: The Routh criterion for stability of continuous-time systems is given by Fact 11.17.2. Problem: For n = 3, the conditions given in [289, p. 6] are |a0 + a2 | < 1 + a1, |3a0 −a2 | < 3−a1, and a20 +a1 −a0 a2 < 1. Show that these conditions are equivalent to iii). Fact 11.20.2. Let p ∈ C[s], where p(s) = sn + an−1sn−1 + · · · + a0 , and define pˆ ∈ C[s] by 

pˆ(s) = sn−1 +

an−1 − a0 a1 n−1 an−2 − a0 a2 n−2 a1 − a0 an−1 s + s + ···+ . 1 − |a0 |2 1 − |a0 |2 1 − |a0 |2

Then, p is discrete-time asymptotically stable if and only if |a0 | < 1 and pˆ is discrete-time asymptotically stable. Proof: See [708, p. 354]. Fact 11.20.3. Let p ∈ R[s], where p(s) = sn + an−1sn−1 + · · · + a0 . Then, the following statements hold: i) If a0 ≤ · · · ≤ an−1 ≤ 1, then roots(p) ⊂ {z ∈ C : |z| ≤ 1 + |a0 | − a0 }. ii) If 0 < a0 ≤ · · · ≤ an−1 ≤ 1, then roots(p) ⊂ CUD. iii) If 0 < a0 < · · · < an−1 < 1, then p is discrete-time asymptotically stable. Proof: For i), see [1220]. For ii), see [1029, p. 272]. For iii), use Fact 11.20.2. See [708, p. 355]. Remark: If there exists r > 0 such that 0 < ra0 < · · · < rn−1 an−1 < rn, then roots(p) ⊂ {z ∈ C : |z| ≤ r}. Remark: Statement ii) is the Enestrom-Kakeya theorem. Fact 11.20.4. Let p ∈ C[s], where p(s) = sn + an−1sn−1 + · · · + a0 , assume that a0 , . . . , an−1 are nonzero, and let λ ∈ roots(p). Then, |λ| ≤ max{2|an−1 |, 2|an−2 /an−1 |, . . . , 2|a1/a2 |, |a0 /a1 |}. Remark: This result is due to Bourbaki. See [1030]. Fact 11.20.5. Let p ∈ C[s], where p(s) = sn + an−1sn−1 + · · · + a0 , assume that a0 , . . . , an−1 are nonzero, and let λ ∈ roots(p). Then, |λ| ≤

n−1 

|ai |1/(n−i)

i=1

and |λ + 12 an−1 | ≤ 12 |an−1 | +

n−2 

|ai |1/(n−i).

i=0

Remark: These results are due to Walsh. See [1030].

779

THE MATRIX EXPONENTIAL AND STABILITY THEORY

Fact 11.20.6. Let p ∈ C[s], where p(s) = sn + an−1sn−1 + · · · + a0 , and let λ ∈ roots(p). Then, |a0 | < |λ| ≤ max{|a0 |, 1 + |a1 |, . . . , 1 + |an−1 |}. |a0 | + max{|a1 |, . . . , |an−1 |, 1} Proof: The lower bound is proved in [1030], while the upper bound is proved in [411]. Remark: The upper bound is Cauchy’s estimate. Remark: The weaker upper bound |λ| < 1 +

max

i=0,...,n−1

|ai |

is given in [140, p. 184] and [1030]. Fact 11.20.7. Let p ∈ C[s], where p(s) = sn + an−1sn−1 + · · · + a0 , and let λ ∈ roots(p). Then, " max |ai | + 14 (1 − |an−1 |)2 , |λ| ≤ 12 (1 + |an−1 |) + i=0,...,n−2

|λ| ≤ max{2, |a0 | + |an−1 |, |a1 | + |an−1 |, . . . , |an−2 | + |an−1 |}, |λ| ≤

" 2+

max

i=0,...,n−2

|ai |2 + |an−1 |2 .

Proof: See [411]. Remark: The first inequality is due to Joyal, Labelle, and Rahman. See [1030]. Fact 11.20.8. Let p ∈ C[s], where p(s) = sn + an−1sn−1 + · · · + a0 , assume that a0 , . . . , an−1 are nonzero, define . . . . . . . an−2 . . a0 . . a 1 .  . . . . . . α = max . . , . . , . . . , . a1 a2 an−1 . and

. . . . . . . an−2 . . a1 . . a 2 .  . , β = max .. .. , .. .. , . . . , .. a2 a3 an−1 .

and let λ ∈ roots(p). Then, |λ| ≤ 12 (β + |an−1 |) +

+ α|an−1 | + 14 (β − |an−1 |)2 ,

|λ| ≤ |an−1 | + α,  . . . a0 . |λ| ≤ max .. .. , 2β, 2|an−1 | , a1 |λ| ≤ 2 |λ| ≤

max

i=1,...,n−1

|ai |1/(n−i) ,

 2|an−1 |2 + α2 + β 2 .

780

CHAPTER 11

Proof: See [411, 943]. Remark: The third inequality is Kojima’s bound, while the fourth inequality is Fujiwara’s bound. Fact 11.20.9. Let p ∈ C[s], where p(s) = sn + an−1sn−1 + · · · + a0 , define  2 α = 1 + n−1 i=0 |ai | , and let λ ∈ roots(p). Then, /   0 n−1 0  1 n 1 1 2 2 |ai | − n |an−1 | , |λ| ≤ n |an−1 | + n−1 n − 1 + 

i=0

⎛

⎞ / / 0 0n−2 0 0 ⎜ ⎟ 0 1(|an−1 | − 1)2 + 41 |λ| ≤ 12 ⎜ | + 1 + |ai |2 ⎟ |a n−1 ⎝ ⎠, i=0

⎛ |λ| ≤ 12 ⎝|an−1 | + cos πn

/ ⎞ 0 n−3 0   2 + 1 |an−1 | − cos πn + (|an−2 | + 1)2 + |ai |2 ⎠, / ⎞ 0n−1 0 + 12 ⎝|an−1 | + 1 |ai |2 ⎠, ⎛

π |λ| ≤ cos n+1

i=0

i=0

"  "      1 1 2 2 2 − 4|a |2 . α − ≤ |λ| ≤ α + α − 4|a | α 0 0 2 2

and

Furthermore, ⎛ |Re λ| ≤ 12⎝|Re an−1 | + cos πn

/ ⎞ 0 n−3 0   2 + 1 |Re an−1 | − cos πn + (|an−2 | − 1)2 + |ai |2 ⎠ i=0

and

⎛

|Im λ| ≤ 12⎝|Im an−1 | + cos πn +

/ 0 0 1

 π 2

|Im an−1 | − cos n

+ (|an−2 | + 1)2 +

n−3 

⎞ |ai |2 ⎠.

i=0

Proof: See [527, 846, 850, 943]. Remark: The first bound is due to Linden (see [850]), the fourth bound is due to Fujii and Kubo, and the upper bound in the fifth result, which follows from Fact 5.11.21 and Fact 5.11.30, is due to Parodi, see also [825, 841]. Remark: The Parodi bound is a refinement of the Carmichael-Mason bound. See Fact 11.20.10.

THE MATRIX EXPONENTIAL AND STABILITY THEORY

781

Fact 11.20.10. Let p ∈ C[s], where p(s) = sn + an−1sn−1 + · · · + a0 , let r, q ∈  n−1 (1, ∞), assume that 1/r + 1/q = 1, define α = ( i=0 |ai |r )1/r, and let λ ∈ roots(p). Then, |λ| ≤ (1 + αq )1/q . In particular, if r = q = 2, then  |λ| ≤ 1 + |an−1 |2 + · · · + |a0 |2 . Proof: See [943, 1030]. Remark: Letting r → ∞ yields the upper bound in Fact 11.20.6. Remark: The result for r = q = 2 is due to Carmichael and Mason. Fact 11.20.11. Let p ∈ C[s], where p(s) = sn + an−1sn−1 + · · · + a0 , and let λ ∈ roots(p). Then,  |λ| ≤ 1 + |1 − an−1 |2 + |an−1 − an−2 |2 + · · · + |a1 − a0 |2 + |a0 |2 . Proof: See [1515]. Remark: This result is due to Williams. Fact 11.20.12. Let p ∈ C[s], where p(s) = sn + an−1sn−1 + · · · + a0 , let  mroots(p) = {λ1, . . . , λn }ms , and let r > 0 be the unique positive root of pˆ(s) = sn − |an−1 |sn−1 − · · · − |a0 |. Then, √ n ( 2 − 1)r ≤ max |λi | ≤ r. i=1,...,n

Furthermore,

√ n ( 2 − 1)r ≤

1 n

n 

|λi | < r.

i=1

Finally, the third inequality is an equality if and only if λ1 = · · · = λn . Remark: The first inequality is due to Cohn, the second inequality is due to Cauchy, and the third and fourth inequalities are due to Berwald. See [1030] and [1029, p. 245]. Fact 11.20.13. Let p ∈ C[s], where p(s) = sn + an−1sn−1 + · · · + a0 , define n−1 α = 1 + i=0 |ai |2, and let λ ∈ roots(p). Then, "  "      1 1 2 − 4|a |2 ≤ |λ| ≤ α − α α2 − 4|a0 |2 . 0 2 2 α+ 

Proof: See [847]. This result follows from Fact 5.11.29 and Fact 5.11.30. Fact 11.20.14. Let p ∈ R[s], where p(s) = sn + an−1sn−1 + · · · + a0 , assume that a0 , . . . , an−1 are nonnegative, and let x1, . . . , xm ∈ [0, ∞). Then,  √ p( m x1 · · · xm ) ≤ m p(x1 ) · · · p(xm ). Proof: See [1067]. Remark: This result, which is due to Mihet, extends a result of Huygens for the case p(x) = x + 1.

782

CHAPTER 11

11.21 Facts on Discrete-Time-Stable Matrices Fact 11.21.1. Let A ∈ R2×2. Then, A is discrete-time asymptotically stable if and only if |tr A| < 1 + det A and |det A| < 1. Fact 11.21.2. Let A ∈ Fn×n. Then, A is discrete-time (Lyapunov stable, semistable, asymptotically stable) if and only if A2 is. Fact 11.21.3. Let A ∈ Rn×n, and let χA(s) = sn + an−1sn−1 + · · · + a1s + a0 . Then, for all k ≥ 0, Ak = x1(k)I + x2 (k)A + · · · + xn(k)An−1, where, for all i ∈ {1, . . . , n} and all k ≥ 0, xi : N → R satisfies xi (k + n) + an−1xi (k + n − 1) + · · · + a1xi (k + 1) + a0 xi (k) = 0, with, for all i, j ∈ {1, . . . , n}, the initial conditions xi (j − 1) = δij . Proof: See [878]. Fact 11.21.4. Let A ∈ Rn×n. Then, the following statements hold: i) If A is semicontractive, then A is discrete-time Lyapunov stable. ii) If A is contractive, then A is discrete-time asymptotically stable. iii) If A is discrete-time Lyapunov stable and normal, then A is semicontractive. iv) If A is discrete-time asymptotically stable and normal, then A is contractive. Problem: Prove these results by using Fact 11.15.6. Fact 11.21.5.Let x ∈ Fn , let A ∈ Fn×n, and assume that A is discrete-time i semistable. Then, ∞ i=0 A x exists if and only if x ∈ R(A − I). In this case, ∞ 

Aix = −(A − I)#x.

i=0

Proof: See [755]. Fact 11.21.6. Let A ∈ Fn×n. Then, A is discrete-time (Lyapunov stable, semistable, asymptotically stable) if and only if A ⊗ A is. Proof: Use Fact 7.4.16. Remark: See [755].

783

THE MATRIX EXPONENTIAL AND STABILITY THEORY

Fact 11.21.7. Let A ∈ Rn×n and B ∈ Rm×m. Then, the following statements hold: i) If A and B are discrete-time (Lyapunov stable, semistable, asymptotically stable), then A ⊗ B is discrete-time (Lyapunov stable, semistable, asymptotically stable). ii) If A ⊗ B is discrete-time (Lyapunov stable, semistable, asymptotically stable), then either A or B is discrete-time (Lyapunov stable, semistable, asymptotically stable). Proof: Use Fact 7.4.16. Fact 11.21.8. Let A ∈ Rn×n, and assume that A is (Lyapunov stable, semistable, asymptotically stable). Then, eA is discrete-time (Lyapunov stable, semistable, asymptotically stable). Problem: If B ∈ Rn×n is discrete-time (Lyapunov stable, semistable, asymptotically stable), when does there exist a (Lyapunov-stable, semistable, asymptotically stable) matrix A ∈ Rn×n such that B = eA ? See Proposition 11.4.3. Fact 11.21.9. The following statements hold: 

i) If A ∈ Rn×n is discrete-time asymptotically stable, then B = (A+I)−1(A− I) is asymptotically stable.  ii) If B ∈ Rn×n is asymptotically stable, then A = (I + B)(I − B)−1 is discrete-time asymptotically stable.

iii) If A ∈ Rn×n is discrete-time asymptotically stable, then there exists a unique asymptotically stable matrix B ∈ Rn×n such that A = (I + B)(I − B)−1. In fact, B = (A + I)−1(A − I). iv) If B ∈ Rn×n is asymptotically stable, then there exists a unique discretetime asymptotically stable matrix A ∈ Rn×n such that B = (A + I)−1(A − I). In fact, A = (I + B)(I − B)−1. Proof: See [675]. Remark: For additional results on the Cayley transform, see Fact 3.11.21, Fact 3.11.22, Fact 3.11.23, Fact 3.20.12, and Fact 8.9.31. Problem: Obtain analogous results for Lyapunov-stable and semistable matrices. Fact 11.21.10. Let



P1 P12 T P12



∈ R2n×2n be positive definite, where P1, P12 , P2

P2  T = P1−1P12

∈ Rn×n. If P1 ≥ P2 , then A is discrete-time asymptotically stable, while,  if P2 ≥ P1, then A = P2−1P12 is discrete-time asymptotically stable. T T ≥ P1 − P12 P2−2P12 > 0. See [342]. Proof: If P1 ≥ P2 , then P1 − P12 P1−1P1P1−1P12

784

CHAPTER 11

Fact 11.21.11. Let A ∈ Rn×n, where n ≥ 2, and assume that A is row stochastic. Then, the following statements hold: i) A is discrete-time Lyapunov stable. ii) If A is primitive, then A is discrete-time semistable. Proof: For all k ≥ 1, it follows that Ak 1n×1 = 1n×1 . Since Ak is nonnegative, it follows that every entry of A is bounded. If A is primitive, then the result follows from Fact 4.11.6, which implies that sprad(A) = 1, and viii) and xv) of Fact 4.11.4, which imply that 1 is a simple eigenvalue of A as well as the only eigenvalue of A on the unit circle. Fact 11.21.12. Let A ∈ Rn×n, and let  ·  be a norm on Rn×n. Then, the following statements hold: 7 8∞ i) A is discrete-time Lyapunov stable if and only if Ak  k=0 is bounded. 

ii) A is discrete-time semistable if and only if A∞ = limk→∞ Ak exists.  iii) Assume that A is discrete-time semistable. Then, A∞ = I −(A−I)(A−I)# is idempotent and rank A∞ = amultA(1). If, in addition, rank A = 1, then, for every eigenvector x of A associated with the eigenvalue 1, there exists y ∈ Fn such that y ∗x = 1 and A∞ = xy ∗.

iv) A is discrete-time asymptotically stable if and only if limk→∞ Ak = 0. Remark: A proof of ii) is given in [1023, p. 640]. See Fact 11.21.16. Fact 11.21.13. Let A ∈ Fn×n. Then, A is discrete-time Lyapunov stable if and only if k−1   A∞ = lim k1 Ai k→∞

exists. In this case,

i=0

A∞ = I − (A − I)(A − I)#.

Proof: See [1023, p. 633]. Remark: A is Cesaro summable. Remark: See Fact 6.3.33. Fact 11.21.14. Let A ∈ Fn×n. Then, A is discrete-time asymptotically stable if and only if lim Ak = 0. k→∞

In this case, (I − A)−1 =

∞ 

Ai,

i=1

where the series converges absolutely. Fact 11.21.15. Let A ∈ Fn×n, and assume that A is unitary. Then, A is discrete-time Lyapunov stable.

THE MATRIX EXPONENTIAL AND STABILITY THEORY

785

Fact 11.21.16. Let A, B ∈ Rn×n, assume that A is discrete-time semistable,  limk→∞ Ak. Then, and let A∞ =  k lim A + k1 B = A∞ eA∞BA∞. k→∞

Proof: See [237, 1463]. Remark: If A is idempotent, then A∞ = A. The existence of A∞ is guaranteed by Fact 11.21.12, which also implies that A∞ is idempotent. Fact 11.21.17. Let A ∈ Rn×n. Then, the following statements hold: i) A is discrete-time Lyapunov stable if and only if there exists a positivedefinite matrix P ∈ Rn×n such that P − ATPA is positive semidefinite. ii) A is discrete-time asymptotically stable if and only if there exists a positivedefinite matrix P ∈ Rn×n such that P − ATPA is positive definite. Remark: The discrete-time Lyapunov equation or the Stein equation is P = ATPA+ R. Fact 11.21.18. Let A ∈ Rn×n, assume that A is discrete-time asymptotically stable, let P ∈ Rn×n be positive definite, and assume that P satisfies P = ATPA+I. Then, P is given by π

P =

1 2π

−π

(AT − e−jθI)−1 (A − ejθI) dθ.

Furthermore, 1 ≤ λn (P ) ≤ λ1 (P ) ≤

[σmax (A) + 1]2n−2 . [1 − sprad(A)]2n

Proof: See [659, pp. 167–169]. n×n and, for k ∈ N, consider the discreteFact 11.21.19. Let (Ak )∞ k=0 ⊂ R time, time-varying system x(k + 1) = Ak x(k).

Furthermore, assume there exist real numbers β ∈ (0, 1), γ > 0, and ε > 0 such that ρ(β 2 + ρε2 )2 < 1, where 

ρ=

(γ + 1)2n−2 , (1 − β)2n

and such that, for all k ∈ N, sprad(Ak ) < β, Ak  < γ, Ak+1 − Ak  < ε, where  ·  is an induced norm on Rn×n. Then, x(k) → 0 as k → ∞. Proof: See [659, pp. 170–173].

786

CHAPTER 11

Remark: This result arises from the theory of infinite matrix products. See [79, 234, 235, 383, 623, 722, 886]. Fact 11.21.20. Let A ∈ Fn×n, and define 

r(A) = Then,

|z| − 1 . {z∈C : |z|>1} σmin (zI − A) sup

r(A) ≤ sup σmax (Ak ) ≤ ner(A). k≥0

Hence, if A is discrete-time Lyapunov stable, then r(A) is finite. Proof: See [1447]. Remark: This result is the Kreiss matrix theorem. Remark: The constant ne is the best possible. See [1447]. Fact 11.21.21. Let p ∈ R[s], and assume that p is discrete-time semistable. Then, C(p) is discrete-time semistable, and there exists v ∈ Rn such that lim C k (p) = 1n×1 v T.

k→∞

Proof: Since C(p) is a companion form matrix, it follows from Proposition 11.10.4 that its minimal polynomial is p. Hence, C(p) is discrete-time semistable. Now, it follows from Proposition 11.10.2 that limk→∞ C k (p) exists, and thus the state x(k) of the difference equation x(k + 1) = C(p)x(k) converges for all initial conditions x(0) = x0 . The structure of C(p) shows that all components of x(k) converge to the same value as k → ∞. Hence, all rows of limk→∞ C k (p) are equal.

11.22 Facts on Lie Groups Fact 11.22.1. The groups UT(n), UT+ (n), UT±1 (n), SUT(n), and {In } are Lie groups. Furthermore, ut(n) is the Lie algebra of UT(n), sut(n) is the Lie algebra of SUT(n), and {0n×n } is the Lie algebra of {In }. Remark: See Fact 3.23.11 and Fact 3.23.12. Problem: Determine the Lie algebras of UT+ (n) and UT±1 (n).

11.23 Facts on Subspace Decomposition Fact 11.23.1. Let A ∈ Rn×n, and let S ∈ Rn×n be a nonsingular matrix such   that A1 A12 A=S S −1, 0 A2 where A1 ∈ Rr×r is asymptotically stable, A12 ∈ Rr×(n−r), and A2 ∈ R(n−r)×(n−r).   Then, 0 B12s s S −1, μA(A) = S 0 μAs (A2 )

787

THE MATRIX EXPONENTIAL AND STABILITY THEORY

where B12s ∈ Rr×(n−r), and



u (A) = S μA

 u (A1 ) B12u μA S −1, u 0 μA (A2 )

u (A1 ) is nonsingular. Consequently, where B12u ∈ Rr×(n−r) and μA    I ⊆ Ss (A). R S r 0

If, in addition, A12 = 0, then



μAs (A) = S  u (A) = S μA

0 0

 0 S −1, μAs (A2 )

 u (A1 ) 0 μA S −1, u 0 μA (A2 )

  Su (A) ⊆ R S

0



In−r

.

Proof: This result follows from Fact 4.10.13. Fact 11.23.2. Let A ∈ Rn×n, and let S ∈ Rn×n be a nonsingular matrix such   that A1 A12 A=S S −1, 0 A2 where A1 ∈ Rr×r, A12 ∈ Rr×(n−r), and A2 ∈ R(n−r)×(n−r) satisfies spec(A2 ) ⊂ CRHP. Then,   s C12s μA(A1 ) S −1, μAs (A) = S 0 μAs (A2 ) where C12s ∈ Rr×(n−r) and μAs (A2 ) is nonsingular, and   u μA(A1 ) C12u u S −1, μA (A) = S 0 0 where C12u ∈ Rr×(n−r). Consequently,

   Ir . Ss (A) ⊆ R S 0

If, in addition, A12 = 0, then



μAs (A) = S

 0 μAs (A1 ) S −1, 0 μAs (A2 ) 

u (A) = S μA

  R S

 u (A1 ) 0 μA S −1, 0 0

0 In−r

 ⊆ Su (A).

788

CHAPTER 11

Fact 11.23.3. Let A ∈ Rn×n, and let S ∈ Rn×n be a nonsingular matrix such   that A1 A12 A=S S −1, 0 A2 where A1 ∈ Rr×r satisfies spec(A1 ) ⊂ CRHP, A12 ∈ Rr×(n−r), and A2 ∈ R(n−r)×(n−r). Then,   s B12s μA(A1 ) s S −1, μA(A) = S 0 μAs (A2 ) where μAs (A1 ) is nonsingular and B12s ∈ Rr×(n−r), and   0 B12u u S −1, μA (A) = S u 0 μA (A2 ) where B12u ∈ Rr×(n−r). Consequently,    I ⊆ Su (A). R S r 0 If, in addition, A12 = 0, then



μAs (A)

=S

 0 μAs (A1 ) S −1, 0 μAs (A2 ) 

u (A) = S μA

 0 0 S −1, u 0 μA (A2 )

  Ss (A) ⊆ R S

0 In−r

 .

Fact 11.23.4. Let A ∈ Rn×n, and let S ∈ Rn×n be a nonsingular matrix such   that A1 A12 A=S S −1, 0 A2 where A1 ∈ Rr×r, A12 ∈ Rr×(n−r), and A2 ∈ R(n−r)×(n−r) is asymptotically stable. Then,   s μA(A1 ) C12s s μA(A) = S S −1, 0 0 where C12s ∈ Rr×(n−r), and u (A) μA

 =S

 u (A1 ) C12u μA S −1, u 0 μA (A2 )

u (A2 ) is nonsingular and C12u ∈ Rr×(n−r). Consequently, where μA    I . Su (A) ⊆ R S r 0

If, in addition, A12 = 0, then μAs (A)

 =S

 μAs (A1 ) 0 S −1, 0 0

789

THE MATRIX EXPONENTIAL AND STABILITY THEORY

 u (A) = S μA

  R S

 u (A1 ) 0 μA S −1, u 0 μA (A2 ) 

0

⊆ Ss (A).

In−r

Fact 11.23.5. Let A ∈ Rn×n, and let S ∈ Rn×n be a nonsingular matrix such   that A1 A12 A=S S −1, 0 A2 where A1 ∈ Rr×r satisfies spec(A1 ) ⊂ CRHP, A12 ∈ Rr×(n−r), and A2 ∈ R(n−r)×(n−r) is asymptotically stable. Then,   s μA(A1 ) C12s s S −1, μA(A) = S 0 0 where C12s ∈ Rr×(n−r) and μAs (A1 ) is nonsingular, and   0 C12u u S −1, μA (A) = S u 0 μA (A2 ) u where C12u ∈ Rr×(n−r) and μA (A2 ) is nonsingular. Consequently,    I . Su (A) = R S r 0

If, in addition, A12 = 0, then



μAs (A) = S  u μA (A) = S

 μAs (A1 ) 0 S −1 0 0 0 0

 0 S −1, u μA (A2 )

  Ss (A) = R S

0 In−r

 .

Fact 11.23.6. Let A ∈ Rn×n, and let S ∈ Rn×n be a nonsingular matrix such   that A1 0 A=S S −1, A21 A2 where A1 ∈ Rr×r is asymptotically stable, A21 ∈ R(n−r)×r, and A2 ∈ R(n−r)×(n−r).   Then, 0 0 s S −1, μA(A) = S B21s μAs (A2 ) where B21s ∈ R(n−r)×r, and u (A) μA

 =S

 u (A1 ) 0 μA S −1, u B21u μA (A2 )

790

CHAPTER 11

u where B21u ∈ R(n−r)×r and μA (A1 ) is nonsingular. Consequently,    0 . Su (A) ⊆ R S In−r

If, in addition, A21 = 0, then



μAs (A)

=S 

u (A) = S μA

 0 0 S −1, 0 μAs (A2 )

 u (A1 ) 0 μA S −1, u 0 μA (A2 )

   I ⊆ Ss (A). R S r 0 Fact 11.23.7. Let A ∈ Rn×n, and let S ∈ Rn×n be a nonsingular matrix such   that A1 0 A=S S −1, A21 A2 where A1 ∈ Rr×r, A21 ∈ R(n−r)×r, and A2 ∈ R(n−r)×(n−r) satisfies spec(A2 ) ⊂ CRHP. Then,   s 0 μ (A ) μAs (A) = S A 1 S −1, C21s μAs (A2 ) where C21s ∈ R(n−r)×r and μAs (A2 ) is nonsingular, and   u μA(A1 ) 0 u S −1, μA (A) = S C21u 0 where C21u ∈ R(n−r)×r. Consequently,    0 ⊆ Su (A). R S In−r If, in addition, A21 = 0, then



μAs (A) = S

 0 μAs (A1 ) S −1, 0 μAs (A2 ) 

u (A) = S μA

 u (A1 ) 0 μA S −1, 0 0

   Ir . Ss (A) ⊆ R S 0 Fact 11.23.8. Let A ∈ Rn×n, and let S ∈ Rn×n be a nonsingular matrix such   that A1 0 A=S S −1, A21 A2

THE MATRIX EXPONENTIAL AND STABILITY THEORY

791

where A1 ∈ Rr×r is asymptotically stable, A21 ∈ R(n−r)×r, and A2 ∈ R(n−r)×(n−r) satisfies spec(A2 ) ⊂ CRHP. Then,   0 0 S −1, μAs (A) = S C21s μAs (A2 ) where C21s ∈ R(n−r)×r and μAs (A2 ) is nonsingular, and   u μA(A1 ) 0 u S −1, μA(A) = S C21u 0 u (A1 ) is nonsingular. Consequently, where C21u ∈ R(n−r)×r and μA    0 . Su (A) = R S In−r

If, in addition, A21 = 0, then



μAs (A)

=S 

u μA (A)

=S

0 0

 0 S −1 μAs (A2 )

 u (A1 ) 0 μA S −1, 0 0

   I . Ss (A) = R S r 0 Fact 11.23.9. Let A ∈ Rn×n, and let S ∈ Rn×n be a nonsingular matrix such   that A1 0 A=S S −1, A21 A2 where A1 ∈ Rr×r, A21 ∈ R(n−r)×r, and A2 ∈ R(n−r)×(n−r) is asymptotically stable. Then,   s μ (A ) 0 S −1, μAs (A) = S A 1 B21s 0 where B21s ∈ R(n−r)×r, and



u (A) = S μA

 u (A1 ) 0 μA S −1, u B21u μA (A2 )

u (A2 ) is nonsingular. Consequently, where B21u ∈ R(n−r)×r and μA    0 ⊆ S(A). R S In−r

If, in addition, A21 = 0, then



μAs (A) = S  u (A) μA

=S

 μAs (A1 ) 0 S −1, 0 0

 u (A1 ) 0 μA S −1, u 0 μA (A2 )

792

CHAPTER 11

   I . Su (A) ⊆ R S r 0 Fact 11.23.10. Let A ∈ Rn×n, and let S ∈ Rn×n be a nonsingular matrix   such that A1 0 A=S S −1, A21 A2 where A1 ∈ Rr×r satisfies spec(A1 ) ⊂ CRHP, A21 ∈ R(n−r)×r, and A2 ∈ R(n−r)×(n−r). Then,   s 0 μ (A ) S −1, μAs (A) = S A 1 C12s μAs (A2 ) where C21s ∈ R(n−r)×r and μAs (A1 ) is nonsingular, and   0 0 u S −1, μA(A) = S u C21u μA (A2 ) where C21u ∈ R(n−r)×r. Consequently,

  Ss (A) ⊆ R S

If, in addition, A21 = 0, then μAs (A)

 =S

In−r

 .

 0 μAs (A1 ) S −1, 0 μAs (A2 ) 

u (A) μA

0

=S

 0 0 S −1, u 0 μA (A2 )

   I ⊆ Su (A). R S r 0 Fact 11.23.11. Let A ∈ Rn×n, and let S ∈ Rn×n be a nonsingular matrix   such that A1 0 A=S S −1, A21 A2 where A1 ∈ Rr×r satisfies spec(A1 ) ⊂ CRHP, A21 ∈ R(n−r)×r, and A2 ∈ R(n−r)×(n−r) is asymptotically stable. Then,   s μA(A1 ) 0 S −1, μAs (A) = S C21s 0 where C21s ∈ R(n−r)×r and μAs (A1 ) is nonsingular, and   0 0 u S −1, μA(A) = S u C21u μA (A2 ) u (A2 ) is nonsingular. Consequently, where C21u ∈ R(n−r)×r and μA    0 . Ss (A) = R S In−r

THE MATRIX EXPONENTIAL AND STABILITY THEORY

If, in addition, A21 = 0, then



μAs (A) = S and

 u μA (A) = S

793

 μAs (A1 ) 0 S −1 0 0 0 0

 0 S −1, u μA (A2 )

   Ir . Su (A) = R S 0

11.24 Notes The Laplace transform (11.2.10) is given in [1232, p. 34]. Computational methods are discussed in [701, 1040]. An arithmetic-mean–geometric-mean iteration for computing the matrix exponential and matrix logarithm is given in [1263]. The exponential function plays a central role in the theory of Lie groups, see [172, 303, 639, 743, 761, 1192, 1400]. Applications to robotics and kinematics are given in [1011, 1053, 1097]. Additional applications are discussed in [302]. The real logarithm is discussed in [368, 682, 1075, 1129]. The multiplicity and properties of logarithms are discussed in [475]. An asymptotically stable polynomial is traditionally called Hurwitz. Semistability is defined in [291] and developed in [190, 199]. Stability theory is treated in [635, 910, 1121] and [555, Chapter XV]. Solutions of the Lyapunov equation under weak conditions are considered in [1238]. Structured solutions of the Lyapunov equation are discussed in [815]. Linear and nonlinear difference equations are studied in [8, 289, 875].

Chapter Twelve

Linear Systems and Control Theory

This chapter considers linear state space systems with inputs and outputs. These systems are considered in both the time domain and frequency (Laplace) domain. Some basic results in control theory are also presented.

12.1 State Space and Transfer Function Models Let A ∈ Rn×n and B ∈ Rn×m, and, for t ≥ t0 , consider the state equation x(t) ˙ = Ax(t) + Bu(t),

(12.1.1)

x(t0 ) = x0 .

(12.1.2)

with the initial condition In (12.1.1), x : [0, ∞) → Rn is the state, and u : [0, ∞) → Rm is the input. The function x is called the solution of (12.1.1). The following result give the solution of (12.1.1) known as the variation of constants formula. Proposition 12.1.1. For t ≥ t0 the state x(t) of the dynamical equation (12.1.1) with initial condition (12.1.2) is given by t

x(t) = e

(t−t0 )A

e(t−τ )ABu(τ ) dτ.

x0 + t0

Proof. Multiplying (12.1.1) by e−tA yields ˙ − Ax(t)] = e−tABu(t), e−tA[x(t)

d −tA e x(t) = e−tABu(t). dt

which is equivalent to Integrating over [t0 , t] yields

t

e

−tA

x(t) = e

−t0 A

e−τABu(τ ) dτ.

x(t0 ) + t0

Now, multiplying by e

tA

yields (12.1.3).

(12.1.3)

796

CHAPTER 12

Alternatively, let x(t) be given by (12.1.3). Then, it follows from Leibniz’s rule Fact 10.12.2 that t

d d x(t) ˙ = e(t−t0 )Ax0 + dt dt

e(t−τ )ABu(τ ) dτ t0

t

= Ae(t−t0 )Ax0 +

Ae(t−τ )ABu(τ ) dτ + Bu(t) t0



= Ax(t) + Bu(t).

For convenience, we can reset the clock by replacing t0 by 0, and therefore assume without loss of generality that t0 = 0. In this case, x(t) for all t ≥ 0 is given by t tA

e(t−τ )ABu(τ ) dτ.

x(t) = e x0 +

(12.1.4)

0

If u(t) = 0 for all t ≥ 0, then, for all t ≥ 0, x(t) is given by x(t) = etAx0 .

(12.1.5)

Now, let u(t) = δ(t)v, where δ(t) is the unit impulse at t = 0 and v ∈ Rm. Loosely speaking, the unit impulse at t = 0 is zero for all t = 0 and is infinite at t = 0. More precisely, let a < b. Then, * b 0, a > 0 or b ≤ 0, δ(τ ) dτ = (12.1.6) 1, a ≤ 0 < b. a More generally, if g : D → Rn, where [a, b] ⊆ D ⊆ R, t0 ∈ D, and g is continuous at t0 , then * b 0, a > t0 or b ≤ t0 , (12.1.7) δ(τ − t0 )g(τ ) dτ = g(t ), a ≤ t < b. 0 0 a Consequently, for all t ≥ 0, x(t) is given by x(t) = etAx0 + etABv.

(12.1.8)

The unit impulse has the physical dimensions of 1/time. This convention makes the integral in (12.1.6) dimensionless. Alternatively, let the input u(t) be a step function, that is, u(t) = 0 for all t < 0 and u(t) = v for all t ≥ 0, where v ∈ Rm. Then, by replacing t − τ by τ in the integral in (12.1.4), it follows that, for all t ≥ 0, t tA

eτA dτBv.

x(t) = e x0 + 0

(12.1.9)

797

LINEAR SYSTEMS AND CONTROL THEORY

Using Fact 11.13.14, (12.1.9) can be written for all t ≥ 0 as   A  ind   tA D tA D −1 i i−1 x(t) = e x0 + A e − I + I − AA Bv. (i!) t A

(12.1.10)

i=1

If A is group invertible, then, for all t ≥ 0, (12.1.10) becomes  

x(t) = etAx0 + A# etA − I + t(I − AA# ) Bv.

(12.1.11)

If, in addition, A is nonsingular, then, for all t ≥ 0, (12.1.11) becomes   x(t) = etAx0 + A−1 etA − I Bv.

(12.1.12)

Next, consider the output equation y(t) = Cx(t) + Du(t),

(12.1.13)

where t ≥ 0, y(t) ∈ Rl is the output, C ∈ Rl×n, and D ∈ Rl×m. Then, for all t ≥ 0, the total response of (12.1.1), (12.1.13) is t tA

y(t) = Ce x0 + Ce(t−τ )ABu(τ ) dτ + Du(t).

(12.1.14)

0

If u(t) = 0 for all t ≥ 0, then the free response is given by y(t) = CetAx0 ,

(12.1.15)

while, if x0 = 0, then the forced response is given by t

Ce(t−τ )ABu(τ ) dτ + Du(t).

y(t) =

(12.1.16)

0

Setting u(t) = δ(t)v, where v ∈ Rm, yields, for all t > 0, the total response y(t) = CetAx0 + H(t)v,

(12.1.17)

where, for all t ≥ 0, the impulse response function H(t) is defined by  H(t) = CetAB + δ(t)D.

(12.1.18)

The corresponding forced response is the impulse response y(t) = H(t)v = CetABv + δ(t)Dv.

(12.1.19)

Alternatively, if u(t) = v ∈ Rm for all t ≥ 0, then the total response is t tA

CeτA dτBv + Dv,

y(t) = Ce x0 +

(12.1.20)

0

and the forced response is the step response t

y(t) =

t

CeτA dτBv + Dv.

H(τ ) dτ v = 0

0

(12.1.21)

798

CHAPTER 12

In general, the forced response can be written as the convolution integral t

H(t − τ )u(τ ) dτ.

y(t) =

(12.1.22)

0

Setting u(t) = δ(t)v in (12.1.22) yields (12.1.20) by noting that t

δ(t − τ )δ(τ ) dτ = δ(t).

(12.1.23)

0

Proposition 12.1.2. Let D = 0 and m = 1, and assume that x0 = Bv. Then, the free response and the impulse response are equal and given by y(t) = CetAx0 = CetABv.

(12.1.24)

12.2 Laplace Transform Analysis Now, consider the linear system x(t) ˙ = Ax(t) + Bu(t),

(12.2.1)

y(t) = Cx(t) + Du(t),

(12.2.2)

with state x(t) ∈ Rn, input u(t) ∈ Rm, and output y(t) ∈ Rl, where t ≥ 0 and x(0) = x0 . Taking Laplace transforms yields x(s) + B u ˆ(s), sˆ x(s) − x0 = Aˆ

(12.2.3)

yˆ(s) = C x ˆ(s) + Dˆ u(s),

(12.2.4)

where

∞ 

e−stx(t) dt,

x ˆ(s) = L{x(t)} =

(12.2.5)

0 

and Hence, and thus

u ˆ(s) = L{u(t)},

(12.2.6)

 L{y(t)}. yˆ(s) =

(12.2.7)

ˆ(s), x ˆ(s) = (sI − A)−1x0 + (sI − A)−1B u

(12.2.8)



ˆ(s). yˆ(s) = C(sI − A)−1x0 + C(sI − A)−1B + D u

(12.2.9)

799

LINEAR SYSTEMS AND CONTROL THEORY

We can also obtain (12.2.9) from the time-domain expression for y(t) given by (12.1.14). Using Proposition 11.2.2, it follows from (12.1.14) that ⎧ t ⎫ ⎨ ⎬ 7 tA 8 yˆ(s) = L Ce x0 + L Ce(t−τ )ABu(τ ) dτ + Dˆ u(s) ⎩ ⎭ 0

7 8 7 8 = CL etA x0 + CL etA B u ˆ(s) + Dˆ u(s)

ˆ(s), = C(sI − A)−1x0 + C(sI − A)−1B + D u

(12.2.10)

which coincides with (12.2.9). We define 

G(s) = C(sI − A)−1B + D.

(12.2.11)

Note that G ∈ Rl×m(s), that is, by Definition 4.7.2, G is a rational transfer function. Since L{δ(t)} = 1, it follows that G(s) = L{H(t)}.

(12.2.12)

Using (4.7.2), G can be written as G(s) =

1 C(sI − A)AB + D. χA(s)

(12.2.13)

It follows from (4.7.3) that G is a proper rational transfer function. Furthermore, G is a strictly proper rational transfer function if and only if D = 0, whereas G is an exactly proper rational transfer function if and only if D = 0. Finally, if A is nonsingular, then G(0) = −CA−1B + D. (12.2.14) Let A ∈ Rn×n. If |s| > sprad(A), then Proposition 9.4.13 implies that −1

(sI − A)

∞    1 1 −1 = I − sA = Ak, sk+1 1 s

(12.2.15)

k=0

where the series is absolutely convergent, and thus G(s) = D + 1s CB + ∞  1 = H , sk k

1 s2 CAB

+ ··· (12.2.16)

k=0

where, for k ≥ 0, the Markov parameter Hk ∈ Rl×m is defined by * D, k = 0,  Hk = k−1 CA B, k ≥ 1.

(12.2.17)

It follows from (12.2.15) that lims→∞ (sI − A)−1 = 0, and thus lim G(s) = D.

s→∞

(12.2.18)

Finally, it follows from Definition 4.7.3 that reldeg G = min{k ≥ 0: Hk = 0}.

(12.2.19)

800

CHAPTER 12

12.3 The Unobservable Subspace and Observability Let A ∈ Rn×n and C ∈ Rl×n, and, for t ≥ 0, consider the linear system x(t) ˙ = Ax(t),

(12.3.1)

x(0) = x0 , y(t) = Cx(t).

(12.3.2) (12.3.3)

Definition 12.3.1. The unobservable subspace Utf (A, C) of (A, C) at time tf > 0 is the subspace 

Utf (A, C) = {x0 ∈ Rn : y(t) = 0 for all t ∈ [0, tf ]}.

(12.3.4)

Let tf > 0. Then, Definition 12.3.1 states that x0 ∈ Utf (A, C) if and only if y(t) = 0 for all t ∈ [0, tf ]. Since y(t) = 0 for all t ∈ [0, tf ] is the free response corresponding to x0 = 0, it follows that 0 ∈ Utf (A, C). Now, suppose there exists a nonzero vector x0 ∈ Utf (A, C). Then, with x(0) = x0 , the free response is given by y(t) = 0 for all t ∈ [0, tf ], and thus x0 cannot be determined from knowledge of y(t) for all t ∈ [0, tf ]. The following result provides explicit expressions for Utf (A, C). Lemma 12.3.2. Let tf > 0. Then, the following subspaces are equal: i) Utf (A, C).   F ii) t∈[0,tf ] N CetA .  Fn−1  iii) i=0 N CAi .  C  CA .. . iv) N .n−1 CA

,  T t v) N 0 f etA CTCetA dt .

,t T If, in addition, limtf →∞ 0 f etA CTCetA dt exists, then the following subspace is equal to i)–v): ,  T ∞ vi) N 0 etA CTCetA dt . Proof. The proof is dual to the proof of Lemma 12.6.2. Lemma 12.3.2 shows that Utf (A, C) is independent of tf . We thus write U(A, C) for Utf (A, C), and call U(A, C) the unobservable subspace of (A, C). (A, C) is observable if U(A, C) = {0}. For convenience, define the nl × n observability matrix ⎡ ⎤ C ⎢ CA ⎥  ⎥ .. O(A, C) = ⎢ (12.3.5) ⎣ ⎦ . CAn−1

801

LINEAR SYSTEMS AND CONTROL THEORY

so that Define

U(A, C) = N[O(A, C)].

(12.3.6)

p = n − dim U(A, C) = n − def O(A, C).

(12.3.7)



Corollary 12.3.3. For all tf > 0,

tf

⊥

T

p = dim U(A, C) = rank O(A, C) = rank If, in addition, limtf →∞

, tf 0

etA CTCetA dt.

(12.3.8)

0

e

tAT T

tA

C Ce dt exists, then ∞ T

etA CTCetA dt.

p = rank

(12.3.9)

0

Corollary 12.3.4. U(A, C) is an invariant subspace of A. The following result shows that the unobservable subspace U(A, C) is unchanged by output injection x(t) ˙ = Ax(t) + F y(t).

(12.3.10)

Proposition 12.3.5. Let F ∈ Rn×l. Then, U(A + F C, C) = U(A, C).

(12.3.11)

In particular, (A, C) is observable if and only if (A + F C, C) is observable. Proof. The proof is dual to the proof of Proposition 12.6.5. ˜ Let U(A, C) ⊆ Rn be a subspace that is complementary to U(A, C). Then, ˜ U(A, C) is an observable subspace in the sense that, if x0 = x 0 + x 0 , where x 0 ∈ ˜ U(A, C) is nonzero and x 0 ∈ U(A, C), then it is possible to determine x 0 from knowledge of y(t) for t ∈ [0, tf ]. Using Proposition 3.5.3, let P ∈ Rn×n be the ˜ unique idempotent matrix such that R(P) = U(A, C) and N(P) = U(A, C). Then, x0 = Px0 . The following result constructs P and provides an expression for x 0 in ˜ terms of y(t) for U(A, C) = U(A, C)⊥. In this case, P is a projector. Lemma 12.3.6. Let tf > 0, and define P ∈ Rn×n by ⎛ t ⎞ + t f

P=⎝ e 

f

tAT T

C Ce

tA

dt⎠ etA CTCetA dt.

0

T

(12.3.12)

0 ⊥

Then, P is the projector onto U(A, C) , and P⊥ is the projector onto U(A, C). Hence, R(P) = N(P⊥ ) = U(A, C)⊥, (12.3.13) N(P) = R(P⊥ ) = U(A, C),

(12.3.14)

rank P = def P⊥ = dim U(A, C)⊥ = p,

(12.3.15)

def P = rank P⊥ = dim U(A, C) = n − p.

(12.3.16)

802

CHAPTER 12

If x0 = x 0 + x 0 , where x 0 ∈ U(A, C)⊥ and x 0 ∈ U(A, C), then ⎛ t ⎞ + t f

f

x 0 = Px0 = ⎝ etA CTCetA dt⎠ etA CTy(t) dt. T

T

0

(12.3.17)

0

Finally, (A, C) is observable if and only if P = In . In this case, for all x0 ∈ Rn, ⎛ t ⎞ −1 t f

x0 = ⎝ e

f

tAT T

C Ce

tA

0

dt⎠

T

etA CTy(t) dt.

(12.3.18)

0

Lemma 12.3.7. Let α ∈ R. Then, U(A + αI, C) = U(A, C).

(12.3.19)

The following result uses a coordinate transformation to characterize the observable dynamics of a system. Theorem 12.3.8. There exists an orthogonal matrix S ∈ Rn×n such that  

0 A1 S −1, C = C1 0 S −1, (12.3.20) A=S A21 A2 where A1 ∈ Rp×p, C1 ∈ Rl×p, and (A1, C1 ) is observable. Proof. The proof is dual to the proof of Theorem 12.6.8. Proposition 12.3.9. Let S ∈ Rn×n, and assume that S is orthogonal. Then, the following conditions are equivalent: i) A and C have the form (12.3.20), where A1 ∈ Rp×p, C1 ∈ Rl×p, and (A1, C1 ) is observable.  0  . ii) U(A, C) = R S In−p

  iii) U(A, C)⊥ = R S I0p .   I 0 S T. iv) P = S p 0 0 Proposition 12.3.10. Let S ∈ Rn×n, and assume that S is nonsingular. Then, the following conditions are equivalent: i) A and C have the form (12.3.20), where A1 ∈ Rp×p, C1 ∈ Rl×p, and (A1, C1 ) is observable.    0  ˜ and U(A, C) = R S I0p . ii) U(A, C) = R S In−p

iii) P = S I0p 00 S −1.

803

LINEAR SYSTEMS AND CONTROL THEORY

Definition 12.3.11. Let S ∈ Rn×n, assume that S is nonsingular, and let A and C have the form (12.3.20), where A1 ∈ Rp×p, C1 ∈ Rl×p, and (A1, C1) is observable. Then, the unobservable spectrum of (A, C) is spec(A2 ), while the unobservable multispectrum of (A, C) is mspec(A2 ). Furthermore, λ ∈ C is an unobservable eigenvalue of (A, C) if λ ∈ spec(A2 ). Definition 12.3.12. The observability pencil OA,C (s) is the pencil OA,C = P that is,

 OA,C (s) =

 A I , −C ,[ 0 ]

sI − A C

(12.3.21)  .

(12.3.22)

Proposition 12.3.13. Let λ ∈ spec(A). Then, λ is an unobservable eigenvalue of (A, C) if and only if   λI − A < n. (12.3.23) rank C Proof. The proof is dual to the proof of Proposition 12.6.13. Proposition 12.3.14. Let λ ∈ mspec(A) and F ∈ Rn×m. Then, λ is an unobservable eigenvalue of (A, C) if and only if λ is an unobservable eigenvalue of (A + F C, C). Proof. The proof is dual to the proof of Proposition 12.6.14. Proposition Assume that (A, C) is observable. Then, the Smith   12.3.15. In form of OA,C is 0l×n . Proof. The proof is dual to the proof of Proposition 12.6.15. Proposition 12.3.16. S ∈ Rn×n, assume that S is nonsingular, and let A and C have the form (12.3.20), where A1 ∈ Rp×p, C1 ∈ Rl×p, and (A1, C1) is observable. Furthermore, let p1, . . . , pn−p be the similarity invariants of A2 , where, for all i ∈ {1, . . . , n − p − 1}, pi divides pi+1 . Then, there exist unimodular matrices S1 ∈ R(n+l)×(n+l) [s] and S2 ∈ Rn×n [s] such that, for all s ∈ C, ⎤ ⎡ Ip p1 (s) ⎥ ⎢   ⎥ ⎢ sI − A .. ⎥ ⎢ = S1(s)⎢ (12.3.24) . ⎥S2 (s). C ⎥ ⎢ ⎣ pn−p (s) ⎦ 0l×n Consequently, Szeros(OA,C ) =

n−p 9 i=1

roots(pi ) = roots(χA2 ) = spec(A2 )

(12.3.25)

804

CHAPTER 12

and mSzeros(OA,C ) =

n−p 9

mroots(pi ) = mroots(χA2 ) = mspec(A2 ).

(12.3.26)

i=1

Proof. The proof is dual to the proof of Proposition 12.6.16. Proposition 12.3.17. Let s ∈ C. Then,   sI − A . O(A, C) ⊆ Re R C

(12.3.27)

Proof. The proof is dual to the proof of Proposition 12.6.17. The next result characterizes observability in several equivalent ways. Theorem 12.3.18. The following statements are equivalent: i) (A, C) is observable.

,t T ii) There exists t > 0 such that 0 eτA CTCeτA dτ is positive definite. ,t T iii) 0 eτA CTCeτA dτ is positive definite for all t > 0. iv) rank O(A, C) = n. v) Every eigenvalue of (A, C) is observable. ,t T If, in addition, limt→∞ 0 eτA CTCeτA dτ exists, then the following condition is equivalent to i)–v): ,∞ T vi) 0 etA CTCetA dt is positive definite. Proof. The proof is dual to the proof of Theorem 12.6.18. The following result, which is a restatement of the equivalence of i) and v) of Theorem 12.3.18, is the PBH test for observability. Corollary 12.3.19. The following statements are equivalent: i) (A, C) is observable. ii) For all s ∈ C,

 rank

sI − A C

 = n.

(12.3.28)

The following result implies that arbitrary eigenvalue placement is possible for (12.3.10) if (A, C) is observable. Proposition 12.3.20. The pair (A, C) is observable if and only if, for every polynomial p ∈ R[s] such that deg p = n, there exists a matrix F ∈ Rm×n such that mspec(A + F C) = mroots(p). Proof. The proof is dual to the proof of Proposition 12.6.20.

805

LINEAR SYSTEMS AND CONTROL THEORY

12.4 Observable Asymptotic Stability 

Let A ∈ Rn×n and C ∈ Rl×n, and define p = n − dim U(A, C). Definition 12.4.1. (A, C) is observably asymptotically stable if Su(A) ⊆ U(A, C).

(12.4.1)

Proposition 12.4.2. Let F ∈ Rn×l. Then, (A, C) is observably asymptotically stable if and only if (A + F C, C) is observably asymptotically stable. Proposition 12.4.3. The following statements are equivalent: i) (A, C) is observably asymptotically stable. ii) There exists an orthogonal matrix S ∈ Rn×n such that (12.3.20) holds, where A1 ∈ Rp×p is asymptotically stable and C1 ∈ Rl×p. iii) There exists a nonsingular matrix S ∈ Rn×n such that (12.3.20) holds, where A1 ∈ Rp×p is asymptotically stable and C1 ∈ Rl×p. iv) limt→∞ CetA = 0. v) The positive-semidefinite matrix P ∈ Rn×n defined by ∞ 

T

P = etA CTCetA dt

(12.4.2)

0

exists. vi) There exists a positive-semidefinite matrix P ∈ Rn×n satisfying ATP + PA + CTC = 0.

(12.4.3)

In this case, the positive-semidefinite matrix P ∈ Rn×n defined by (12.4.2) satisfies (12.4.3). Proof. The proof is dual to the proof of Proposition 12.7.3. The matrix P defined by (12.4.2) is the observability Gramian, and (12.4.3) is the observation Lyapunov equation. Proposition 12.4.4. Assume that (A, C) is observably asymptotically stable, let P ∈ Rn×n be the positive-semidefinite matrix defined by (12.4.2), and define P ∈ Rn×n by (12.3.12). Then, the following statements hold: i) PP + = P. ii) R(P ) = R(P) = U(A, C)⊥. iii) N(P ) = N(P) = U(A, C). iv) rank P = rank P = p. v) P is the only positive-semidefinite solution of (12.4.3) whose rank is p.

806

CHAPTER 12

Proof. The proof is dual to the proof of Proposition 12.7.4. Proposition 12.4.5. Assume that (A, C) is observably asymptotically stable, let P ∈ Rn×n be the positive-semidefinite matrix defined by (12.4.2), and let Pˆ ∈ Rn×n. Then, the following statements are equivalent: i) Pˆ is positive semidefinite and satisfies (12.4.3). ii) There exists a positive-semidefinite matrix P0 ∈ Rn×n such that Pˆ = P +P0 and ATP0 + P0 A = 0. In this case, and

rank Pˆ = p + rank P0 rank P0 ≤



gmultA(λ).

(12.4.4) (12.4.5)

λ∈spec(A) λ∈jR

Proof. The proof is dual to the proof of Proposition 12.7.5. Proposition 12.4.6. The following statements are equivalent: i) (A, C) is observably asymptotically stable, every imaginary eigenvalue of A is semisimple, and A has no ORHP eigenvalues. ii) (12.4.3) has a positive-definite solution P ∈ Rn×n. Proof. The proof is dual to the proof of Proposition 12.7.6. Proposition 12.4.7. The following statements are equivalent: i) (A, C) is observably asymptotically stable, and A has no imaginary eigenvalues. ii) (12.4.3) has exactly one positive-semidefinite solution P ∈ Rn×n. In this case, P ∈ Rn×n is given by (12.4.2) and satisfies rank P = p. Proof. The proof is dual to the proof of Proposition 12.7.7. Corollary 12.4.8. Assume that A is asymptotically stable. Then, the positive-semidefinite matrix P ∈ Rn×n defined by (12.4.2) is the unique solution of (12.4.3) and satisfies rank P = p. Proof. The proof is dual to the proof of Corollary 12.7.8. Proposition 12.4.9. The following statements are equivalent: i) (A, C) is observable, and A is asymptotically stable. ii) (12.4.3) has exactly one positive-semidefinite solution P ∈ Rn×n, and P is positive definite. In this case, P ∈ Rn×n is given by (12.4.2).

807

LINEAR SYSTEMS AND CONTROL THEORY

Proof. The proof is dual to the proof of Proposition 12.7.9. Corollary 12.4.10. Assume that A is asymptotically stable. Then, the positive-semidefinite matrix P ∈ Rn×n defined by (12.4.2) exists. Furthermore, P is positive definite if and only if (A, C) is observable.

12.5 Detectability 

Let A ∈ Rn×n and C ∈ Rl×n, and define p = n − dim U(A, C). Definition 12.5.1. (A, C) is detectable if U(A, C) ⊆ Ss(A).

(12.5.1)

Proposition 12.5.2. Let F ∈ Rn×l. Then, (A, C) is detectable if and only if (A + F C, C) is detectable. Proposition 12.5.3. The following statements are equivalent: i) (A, C) is detectable. ii) There exists an orthogonal matrix S ∈ Rn×n such that (12.3.20) holds, where A1 ∈ Rp×p, C1 ∈ Rl×p, (A1, C1 ) is observable, and A2 ∈ R(n−p)×(n−p) is asymptotically stable. iii) There exists a nonsingular matrix S ∈ Rn×n such that (12.3.20) holds, where A1 ∈ Rp×p, C1 ∈ Rl×p, (A1, C1 ) is observable, and A2 ∈ R(n−p)×(n−p) is asymptotically stable. iv) Every CRHP eigenvalue of (A, C) is observable. Proof. The proof is dual to the proof of Proposition 12.8.3. The following result, which is a restatement of the equivalence of i) and iv) of Proposition 12.5.3, is the PBH test for detectability. Corollary 12.5.4. The following statements are equivalent: i) (A, C) is detectable. ii) For all s ∈ CRHP,

 rank

sI − A C

 = n.

Proposition 12.5.5. The following statements are equivalent: i) A is asymptotically stable. ii) (A, C) is observably asymptotically stable and detectable. Proof. The proof is dual to the proof of Proposition 12.8.5.

(12.5.2)

808

CHAPTER 12

Corollary 12.5.6. The following statements are equivalent: i) There exists a positive-semidefinite matrix P ∈ Rn×n satisfying (12.4.3), and (A, C) is detectable. ii) A is asymptotically stable. Proof. The proof is dual to the proof of Proposition 12.8.6.

12.6 The Controllable Subspace and Controllability Let A ∈ Rn×n and B ∈ Rn×m, and, for t ≥ 0, consider the linear system x(t) ˙ = Ax(t) + Bu(t),

(12.6.1)

x(0) = 0.

(12.6.2)

Definition 12.6.1. The controllable subspace Ctf (A, B) of (A, B) at time tf > 0 is the subspace 

Ctf (A, B) = {xf ∈ Rn : there exists a continuous control u: [0, tf ] → Rm such that the solution x(·) of (12.6.1), (12.6.2) satisfies x(tf ) = xf }. (12.6.3) Let tf > 0. Then, Definition 12.6.1 states that xf ∈ Ctf (A, B) if and only if there exists a continuous control u: [0, tf ] → Rm such that tf

e(tf −t)ABu(t) dt.

xf =

(12.6.4)

0

The following result provides explicit expressions for Ctf (A, B). Lemma 12.6.2. Let tf > 0. Then, the following subspaces are equal: i) Ctf (A, B). ⊥  F T tAT . ii) t∈[0,tf ] N B e F ⊥ n−1  T iT  . i=0 N B A 

 iv) R B AB · · · An−1B . ,  T t v) R 0 f etABBTetA dt .

iii)

,t T If, in addition, limtf →∞ 0 f etABBTetA dt exists, then the following subspace is equal to i)–v): ,  T ∞ vi) R 0 etABBTetA dt .

809

LINEAR SYSTEMS AND CONTROL THEORY

Proof. To prove that i) ⊆ ii), let η ∈

F

t∈[0,tf ] N



T

BTetA



so that ηTetAB = 0 ,t for all t ∈ [0, tf ]. Now, let u: [0, tf ] → Rm be continuous. Then, ηT 0 f e(tf −t)ABu(t) dt = 0, which implies that η ∈ Ctf (A, B)⊥.  Fn−1  To prove that ii) ⊆ iii), let η ∈ i=0 N BTAiT so that ηTAiB = 0 for all i ∈ {0, 1, . . . , n − 1}. It follows from the Cayley-Hamilton theorem given by T tA Theorem 4.4.7 that ηTAiB = 0 for all i≥ 0. Now,  let t ∈ [0, tf ]. Then, η e B = ∞ i −1 T i T tAT . i=0 t (i!) η A B = 0, and thus η ∈ N B e 

⊥ B AB · · · An−1B . Then, η ∈  To show that iii) ⊆ iv ),let η ∈ R T n−1 T i , which implies that η A B = 0 for all i ∈ {0, 1, . . . , N B AB · · · A B n − 1}.  , T t To prove that iv ) ⊆ v ), let η ∈ N 0 f etABBTetA dt . Then, tf T

η

T

etABBTetA dtη = 0,

0

which implies that η e B = 0 for all t ∈ [0, tf ]. Differentiating with respect to t and setting t = 0 implies that ηTAiB = 0 for all i ∈ {0, 1, . . . , n − 1}. Hence, 

⊥ η ∈ R B AB · · · An−1B . T tA

,t To prove that v ) ⊆ i), let η ∈ Ctf (A, B)⊥. Then, ηT 0 f e(tf −t)ABu(t) dt = 0 T (tf −t)AT T for all continuous u: [0, tf ] → Rm. Letting η , implies that , u(t) = BT e  , T tf tA T tf tA T tA T tA η 0 e BB e dtη = 0, and thus η ∈ N 0 e BB e dt . Lemma 12.6.2 shows that Ctf (A, B) is independent of tf . We thus write C(A, B) for Ctf (A, B), and call C(A, B) the controllable subspace of (A, B). (A, B) is controllable if C(A, B) = Rn. For convenience, define the m × nm controllability matrix

 B AB · · · An−1B K(A, B) = (12.6.5) so that Define

C(A, B) = R[K(A, B)].

(12.6.6)

q = dim C(A, B) = rank K(A, B).

(12.6.7)



Corollary 12.6.3. For all tf > 0,

tf T

q = dim C(A, B) = rank K(A, B) = rank

etABBTetA dt. 0

(12.6.8)

810

CHAPTER 12

If, in addition, limtf →∞

, tf 0

T

etABBTetA dt exists, then ∞ T

etABBTetA dt.

q = rank

(12.6.9)

0

Corollary 12.6.4. C(A, B) is an invariant subspace of A. The following result shows that the controllable subspace C(A, B) is unchanged by full-state feedback u(t) = Kx(t) + v(t). Proposition 12.6.5. Let K ∈ Rm×n. Then, C(A + BK, B) = C(A, B).

(12.6.10)

In particular, (A, B) is controllable if and only if (A + BK, B) is controllable. Proof. Note that C(A + BK, B) = R[K(A + BK, B)]  = R B AB + BKB

A2B + ABKB + BKAB + BKBKB

= R[K(A, B)] = C(A, B).

···

 

˜ Let C(A, B) ⊆ Rn be a subspace that is complementary to C(A, B). Then, ˜ C(A, B) is an uncontrollable subspace in the sense that, if xf = x f + x f ∈ Rn, where ˜ B) is nonzero, then there exists a continuous control x f ∈ C(A, B) and x f ∈ C(A, m u: [0, tf ] → R such that x(tf ) = x f , but there exists no continuous control such that x(tf ) = xf . Using Proposition 3.5.3, let Q ∈ Rn×n be the unique idempotent ˜ matrix such that R(Q) = C(A, B) and N(Q) = C(A, B). Then, x f = Qxf . The following result constructs Q and a continuous control u(·) that yields x(tf ) = x f  ˜ for C(A, B) = C(A, B)⊥. In this case, Q is a projector. Lemma 12.6.6. Let tf > 0, and define Q ∈ Rn×n by ⎛ t ⎞ + t f

f

 Q = ⎝ etABBTetA dt⎠ etABBTetA dt. T

0

T

(12.6.11)

0

Then, Q is the projector onto C(A, B), and Q⊥ is the projector onto C(A, B)⊥. Hence, R(Q) = N(Q⊥ ) = C(A, B), (12.6.12) N(Q) = R(Q) = C(A, B)⊥, rank Q = def Q⊥ = dim C(A, B) = q,

(12.6.13) (12.6.14)

def Q = rank Q⊥ = dim C(A, B)⊥ = n − q.

(12.6.15)

811

LINEAR SYSTEMS AND CONTROL THEORY

Now, define u: [0, tf ] → Rm by

⎛

⎞ +

tf

T T  u(t) = BTe(tf −t)A ⎝ eτABBTeτA dτ⎠ xf .

(12.6.16)

0

If xf =

x f

+

x f ,

where

x f

∈ C(A, B) and x f ∈ C(A, B)⊥, then tf

x f

e(tf −t)ABu(t) dt.

= Qxf =

(12.6.17)

0

Finally, (A, B) is controllable if and only if Q = In . In this case, for all xf ∈ Rn, tf

e(tf −t)ABu(t) dt,

xf =

(12.6.18)

0

where u: [0, tf ] → R

m

is given by

⎛

⎞ −1

tf

u(t) = BTe(tf −t)A ⎝ eτABBTeτA dτ⎠ xf . T

T

(12.6.19)

0

Lemma 12.6.7. Let α ∈ R. Then, C(A + αI, B) = C(A, B).

(12.6.20)

The following result uses a coordinate transformation to characterize the controllable dynamics of (12.6.1). Theorem 12.6.8. There exists an orthogonal matrix S ∈ Rn×n such that     B1 A1 A12 , (12.6.21) S −1, B=S A=S 0 A2 0 where A1 ∈ Rq×q , B1 ∈ Rq×m, and (A1, B1) is controllable. 

Proof. Let α < 0 be such that Aα = A + αI is asymptotically stable, and let Q ∈ Rn×n be the positive-semidefinite solution of T Aα Q + QAT α + BB = 0

given by

∞ T

Q = etAαBBTetAα dt. 0

It now follows from Lemma 12.6.2 and Lemma 12.6.7 that R(Q) = R[C(Aα , B)] = R[C(A, B)]. Hence,

rank Q = dim C(Aα , B) = dim C(A, B) = q.

(12.6.22)

812

CHAPTER 12

Next, let S ∈ Rn×n be an orthogonal matrix such that Q = S Q01 00 S T, where   1 ˆ1 A ˆ12 −1 Q1 ∈ Rq×q is positive definite. Writing Aα = S AA and B = S B ˆ21 A ˆ2 S B2 , where Aˆ1 ∈ Rq×q and B1 ∈ Rq×m, it follows from (12.6.22) that Aˆ1Q1 + Q1Aˆ1T + B1B1T = 0, Aˆ21Q1 + B2 BT = 0, 1

B2 B2T = 0. Therefore, B2 = 0 and Aˆ21 = 0, and thus   Aˆ1 Aˆ12 S −1, Aα = S 0 Aˆ2 Furthermore, A=S



Aˆ1 0

Aˆ12 Aˆ2

 S

−1

 − αI = S 

Hence, A=S 



A1 0

A12 A2

 B=S

Aˆ1 0

B1 0

Aˆ12 Aˆ2



 .  − αI S −1.

 S −1,



where A1 = Aˆ1 − αIq , A12 = Aˆ12 , and A2 = Aˆ2 − αIn−q . Proposition 12.6.9. Let S ∈ Rn×n, and assume that S is orthogonal. Then, the following conditions are equivalent: i) A and B have the form (12.6.21), where A1 ∈ Rq×q , B1 ∈ Rq×m, and (A1, B1) is controllable.   ii) C(A, B) = R S I0q .  0  iii) C(A, B)⊥ = R S In−q .   Iq 0 S T. iv) Q = S 0 0 Proposition 12.6.10. Let S ∈ Rn×n, and assume that S is nonsingular. Then, the following conditions are equivalent: i) A and B have the form (12.6.21), where A1 ∈ Rq×q , B1 ∈ Rq×m, and (A1, B1) is controllable.  0    ˜ B) = R S In−q . ii) C(A, B) = R S I0q and C(A, Iq 0 −1 iii) Q = S 0 0 S . Definition 12.6.11. Let S ∈ Rn×n, assume that S is nonsingular, and let A and B have the form (12.6.21), where A1 ∈ Rq×q, B1 ∈ Rq×m, and (A1, B1) is controllable. Then, the uncontrollable spectrum of (A, B) is spec(A2 ), while the uncontrollable multispectrum of (A, B) is mspec(A2 ). Furthermore, λ ∈ C is an uncontrollable eigenvalue of (A, B) if λ ∈ spec(A2 ).

813

LINEAR SYSTEMS AND CONTROL THEORY

Definition 12.6.12. The controllability pencil CA,B (s) is the pencil CA,B = P [ A −B ],[ I that is, CA,B (s) =



sI − A

0 ],

(12.6.23)

B .

(12.6.24)

Proposition 12.6.13. Let λ ∈ spec(A). Then, λ is an uncontrollable eigenvalue of (A, B) if and only if

rank λI − A B < n. (12.6.25) Proof. Since (A1, B1) is controllable, it follows from (12.6.21) that  

A12 B1 λI − A1 rank λI − A B = rank 0 λI − A2 0

= rank λI − A1 B1 + rank(λI − A2 )

Hence, rank λI − A B and only if λ ∈ spec(A2 ).

= q + rank(λI − A2 ). < n if and only if rank(λI − A2 ) < n − q, that is, if

Proposition 12.6.14. Let λ ∈ mspec(A) and K ∈ Rn×m. Then, λ is an uncontrollable eigenvalue of (A, B) if and only if λ is an uncontrollable eigenvalue of (A + BK, B). Proof. In the notation of Theorem partition B1 = B11   12.6.8, K1 , where K1 ∈ Rq×m . Then, where B11 ∈ Fq×m, and partition K = K2   A1 + B11 K1 A12 + B12 K2 . A + BK = 0 A2

B12 ,

Consequently, the uncontrollable spectrum of A + BK is spec(A2 ). Proposition that (A, B) is controllable. Then, the Smith

12.6.15. Assume form of CA,B is In 0n×m . Proof. First, note that, if λ ∈ C is not an eigenvalue of A, then n = rank(λI − A) = rank λI − A B = rank CA,B (λ). Therefore, rank CA,B = n, and thus CA,B has n Smith polynomials. Furthermore, since (A, B) is controllable, it follows that (A, B) has no uncontrollable eigenvalues. Therefore, it follows from Proposition 12.6.13 that, for all λ ∈ spec(A), rank λI − A B = n. Consequently, rank CA,B (λ) = n for all λ ∈ C. Thus, every Smith polynomial CA,B is 1.

814

CHAPTER 12

Proposition 12.6.16. Let S ∈ Rn×n, assume that S is nonsingular, and let A and B have the form (12.6.21), where A1 ∈ Rq×q, B1 ∈ Rq×m, and (A1, B1) is controllable. Furthermore, let p1, . . . , pn−q be the similarity invariants of A2 , where, for all i ∈ {1, . . . , n − q − 1}, pi divides pi+1 . Then, there exist unimodular matrices S1 ∈ Rn×n [s] and S2 ∈ R(n+m)×(n+m) [s] such that, for all s ∈ C, ⎤ ⎡ Iq ⎢ p1 (s) 0n×m ⎥

⎥ ⎢ sI − A B = S1(s)⎢ (12.6.26) ⎥S2 (s). .. ⎦ ⎣ . pn−q (s) Consequently, Szeros(CA,B ) =

n−q 9

roots(pi ) = roots(χA2 ) = spec(A2 )

(12.6.27)

i=1

and mSzeros(CA,B ) =

n−q 9

mroots(pi ) = mroots(χA2 ) = mspec(A2 ).

(12.6.28)

i=1

R

Proof. Let S ∈ Rn×n be as in Theorem 12.6.8, let Sˆ1 ∈ Rq×q [s] and Sˆ2 ∈ [s] be unimodular matrices such that

Sˆ1(s) sIq − A1 B1 Sˆ2 (s) = Iq 0q×m ,

(q+m)×(q+m)

and let Sˆ3 , Sˆ4 ∈ R(n−q)×(n−q) be unimodular matrices such that Sˆ3 (s)(sI − A2 )Sˆ4 (s) = Pˆ (s),  where Pˆ = diag(p1, . . . , pn−q ). Then,  −1 

Iq Sˆ1 (s) 0 sI − A B = S 0 0 Sˆ3−1 (s)

⎡

Iq ×⎣ 0 0

0 0 Im

0 Pˆ (s)

⎡ ⎤  Iq −Sˆ1(s)A12  ˆ−1 (s) 0 S 2 ⎣ 0 Sˆ4−1 (s) ⎦ 0 In−q 0 0

0q×m 0

0 0 In−q



⎤ 0q×m  −1 S Im ⎦ 0 0

Proposition 12.6.17. Let s ∈ C. Then, 

 C(A, B) ⊆ Re R sI − A B .

0 Im

 .

(12.6.29)

Proof. Using Proposition 12.6.9 and the notation in the proof of Proposition 12.6.16, it follows that, for all s ∈ C,    −1 

   0 Sˆ1 (s) = R sI − A B . C(A, B) = R S I0q ⊆ R S −1 ˆ ˆ 0 S3 (s)P (s) Finally, (12.6.29) follows from the fact that C(A, B) is a real subspace. The next result characterizes controllability in several equivalent ways.

815

LINEAR SYSTEMS AND CONTROL THEORY

Theorem 12.6.18. The following statements are equivalent: i) (A, B) is controllable.

,t T ii) There exists t > 0 such that 0 eτABBTeτA dτ is positive definite. ,t T iii) 0 eτABBTeτA dτ is positive definite for all t > 0. iv) rank K(A, B) = n. v) Every eigenvalue of (A, B) is controllable. ,t T If, in addition, limt→∞ 0 eτABBTeτA dτ exists, then the following condition is equivalent to i)–v): ,∞ T vi) 0 etABBTetA dt is positive definite. Proof. The equivalence of i)–iv) follows from Lemma 12.6.2. To prove that iv) =⇒ v), suppose that v) does not hold, that is, there exist λ ∈ spec(A) and a nonzero vector x ∈ Cn such that x∗A = λx∗ and x∗B = 0. It thus follows that x∗AB = λx∗B = 0. Similarly, x∗AiB = 0 for all i ∈ {0, 1, . . . , n − 1}. Hence, (Re x)T K(A, B) = 0 and (Im x)T K(A, B) = 0. Since Re x and Im x are not both zero, it follows that dim C(A, B) < n. Conversely, to show that v) implies iv), suppose that rank K(A, B) < n. Then, there exists a nonzero vector x ∈ Rn such that xTAiB = 0 for all i ∈ {0, . . . , n −1}. Now, let p ∈ R[s] be a nonzero polynomial of minimal degree such that xTp(A) = 0. Note that p is not a constant polynomial and that xTμA (A) = 0. Thus, 1 ≤ deg p ≤ deg μA . Now, let λ ∈ C be such that p(λ) = 0, and let q ∈ R[s] be such that p(s) = q(s)(s − λ) for all s ∈ C. Since deg q < deg p, it follows that xTq(A) = 0. Therefore,  η = q(A)x is nonzero. Furthermore, ηT(A − λI) = xTp(A) = 0. Since xTAiB = 0 for all i ∈ {0, . . . , n −1}, it follows that ηTB = xTq(A)B = 0. Consequently, v) does not hold. The following result, which is a restatement of the equivalence of i) and v) of Theorem 12.6.18, is the PBH test for controllability. Corollary 12.6.19. The following statements are equivalent: i) (A, B) is controllable. ii) For all s ∈ C, rank



sI − A B

= n.

(12.6.30)

The following result implies that arbitrary eigenvalue placement is possible for (12.6.1) when (A, B) is controllable. Proposition 12.6.20. The pair (A, B) is controllable if and only if, for every polynomial p ∈ R[s] such that deg p = n, there exists a matrix K ∈ Rm×n such that mspec(A + BK) = mroots(p).

816

CHAPTER 12 



Proof. For the case m = 1 let Ac = C(χA ) and Bc = en as in (12.9.5). Then, Proposition 12.9.3 implies that K(Ac , Bc ) is nonsingular, while Corollary 12.9.9 implies that Ac = S −1AS and Bc = S −1B. Now, let mroots(p) = {λ1, . . . , λn }ms ⊂  −1 C. Letting K = eT it follows that n [C(p) − Ac ]S A + BK = S(Ac + Bc KS)S −1 = S(Ac + En,n [C(p) − Ac ])S −1 = SC(p)S −1. The case m > 1 requires the multivariable controllable canonical form. See [1179, p. 248].

12.7 Controllable Asymptotic Stability 

Let A ∈ Rn×n and B ∈ Rn×m, and define q = dim C(A, C). Definition 12.7.1. (A, B) is controllably asymptotically stable if C(A, B) ⊆ Ss(A).

(12.7.1)

Proposition 12.7.2. Let K ∈ Rm×n. Then, (A, B) is controllably asymptotically stable if and only if (A + BK, B) is controllably asymptotically stable. Proposition 12.7.3. The following statements are equivalent: i) (A, B) is controllably asymptotically stable. ii) There exists an orthogonal matrix S ∈ Rn×n such that (12.6.21) holds, where A1 ∈ Rq×q is asymptotically stable and B1 ∈ Rq×m. iii) There exists a nonsingular matrix S ∈ Rn×n such that (12.6.21) holds, where A1 ∈ Rq×q is asymptotically stable and B1 ∈ Rq×m. iv) limt→∞ etAB = 0. v) The positive-semidefinite matrix Q ∈ Rn×n defined by ∞ 

T

etABBTetA dt

Q=

(12.7.2)

0

exists. vi) There exists a positive-semidefinite matrix Q ∈ Rn×n satisfying AQ + QAT + BBT = 0.

(12.7.3)

In this case, the positive-semidefinite matrix Q ∈ Rn×n defined by (12.7.2) satisfies (12.7.3).

817

LINEAR SYSTEMS AND CONTROL THEORY

Proof. To prove that i) =⇒ ii), assume that (A, B) is controllably asymps u totically stable so that C(A, B) ⊆ Ss (A) = N[μA (A)] = R[μA (A)]. Using Theorem 12.6.8, it follows that there exists an orthogonal matrix S ∈ Rn×n such that is satisfied, where A1 ∈ Rq×q and (A1, B1) is controllable. Thus,  (12.6.21)

 Iq s R S 0 = C(A, B) ⊆ R[μA (A)]. Next, note that

 s (A) μA

=S

s (A1 ) μA

B12s

0

s μA (A2 )

 S −1,

where B12s ∈ Rq×(n−q), and suppose that A1 is not asymptotically stable with s s CRHP eigenvalue λ. Then, λ ∈ / roots(μA ), and thus μA (A1 ) = 0. Let x1 ∈ Rn−q s satisfy μA(A1 )x1 = 0. Then,      Iq x1 ∈R S = C(A, B) 0 0 

and s μA (A)S

x1 0



 =S

s μA (A1 )x1 0

 ,

s and thus [ x01 ] ∈ / N[μA (A)] = Ss (A), which implies that C(A, B) is not contained in Ss (A). Hence, A1 is asymptotically stable.

To prove that iii) =⇒ iv), assume there exists a nonsingular matrix S ∈ Rn×n such that (12.6.21) holds, where A1 ∈ Rk×k is asymptotically stable and  tA B1 ∈ Rk×m. Thus, etAB = e 01B1 S → 0 as t → ∞. Next, to prove that iv ) implies v ), assume that etAB → 0 as t → ∞. Then, every entry of etAB involves exponentials of t, where the coefficients of t have T real part. Hence, so does every entry of etABBTetA , which implies that ,negative T ∞ tA T tA dt exists. 0 e BB e To prove that v) =⇒ vi), note that, since Q = T follows that etABBTetA → 0 as t → ∞. Thus, ∞



T

AQ + QA =

,∞ 0

 T T AetABBTetA + etABBTetA A dt

0 ∞

d tA T tAT e BB e dt dt

= 0

T

= lim etABBTetA − BBT = −BBT, t→∞

which shows that Q satisfies (12.4.3).

T

etABBTetA dt exists, it

818

CHAPTER 12

To prove that vi) =⇒ i), suppose there exists a positive-semidefinite matrix Q ∈ Rn×n satisfying (12.7.3). Then, t

t

T tAT

tA

e BB e

t

  T dτ = − e AQ + QAT etA dτ = −

d τA T e QA dτ dτ

τA

0

0

0 T

= Q − etA QetA ≤ Q. Next, it follows from Theorem 12.6.8 that there exists an orthogonal matrix S ∈ Rn×n such that (12.6.21) is satisfied, where A1 ∈ Rq×q, B1 ∈ Rq×m, and (A1, B1) is controllable. Consequently, we have   T I 0 S eτABBTeτA dτ S T 0 0 0  

I . ≤ I 0 SQS T 0 , ∞ tA  tA1T Thus, it follows from Proposition 8.6.3 that Q1 = e 1B1BT dt exists. Since 1e 0 (A1, B1) is controllable, it follows from vii) of Theorem 12.6.18 that Q1 is positive definite. t

e

τA1T B1BT dτ 1e

τA1

=



I

t

n Now, let λ be an eigenvalue of AT 1 , and let x1 ∈ C be an associated eigen ∗ vector. Consequently, α = x1 Q1x1 is positive, and ∞

α=

x∗1

e

∞ λt

λt BBT 1e

dtx1 =

0

x∗1B1BT 1 x1

e2(Re λ)t dt. 0

,∞ Hence, 0 e2(Re λ)t dt = α/x∗1B1BT 1 x1 exists, and thus Re λ < 0. Consequently, A1 is asymptotically stable, and thus C(A, B) ⊆ Ss (A), that is, (A, B) is controllably asymptotically stable. The matrix Q ∈ Rn×n defined by (12.7.2) is the controllability Gramian, and (12.7.3) is the control Lyapunov equation. Proposition 12.7.4. Assume that (A, B) is controllably asymptotically stable, let Q ∈ Rn×n be the positive-semidefinite matrix defined by (12.7.2), and define Q ∈ Rn×n by (12.6.11). Then, the following statements hold: i) QQ+ = Q. ii) R(Q) = R(Q) = C(A, B). iii) N(Q) = N(Q) = C(A, B)⊥. iv) rank Q = rank Q = q. v) Q is the only positive-semidefinite solution of (12.7.3) whose rank is q. Proof. See [1238] for the proof of v).

819

LINEAR SYSTEMS AND CONTROL THEORY

Proposition 12.7.5. Assume that (A, B) is controllably asymptotically stable, let Q ∈ Rn×n be the positive-semidefinite matrix defined by (12.7.2), and let ˆ ∈ Rn×n. Then, the following statements are equivalent: Q ˆ is positive semidefinite and satisfies (12.7.3). i) Q ˆ = ii) There exists a positive-semidefinite matrix Q0 ∈ Rn×n such that Q Q + Q0 and AQ0 + Q0AT = 0. In this case, and

ˆ = q + rank Q0 rank Q rank Q0 ≤



gmultA(λ).

(12.7.4) (12.7.5)

λ∈spec(A) λ∈jR

Proof. See [1238]. Proposition 12.7.6. The following statements are equivalent: i) (A, B) is controllably asymptotically stable, every imaginary eigenvalue of A is semisimple, and A has no ORHP eigenvalues. ii) (12.7.3) has a positive-definite solution Q ∈ Rn×n. Proof. See [1238]. Proposition 12.7.7. The following statements are equivalent: i) (A, B) is controllably asymptotically stable, and A has no imaginary eigenvalues. ii) (12.7.3) has exactly one positive-semidefinite solution Q ∈ Rn×n. In this case, Q ∈ Rn×n is given by (12.7.2) and satisfies rank Q = q. Proof. See [1238]. Corollary 12.7.8. Assume that A is asymptotically stable. Then, the positive-semidefinite matrix Q ∈ Rn×n defined by (12.7.2) is the unique solution of (12.7.3) and satisfies rank Q = q. Proof. See [1238]. Proposition 12.7.9. The following statements are equivalent: i) (A, B) is controllable, and A is asymptotically stable. ii) (12.7.3) has exactly one positive-semidefinite solution Q ∈ Rn×n, and Q is positive definite. In this case, Q ∈ Rn×n is given by (12.7.2). Proof. See [1238].

820

CHAPTER 12

Corollary 12.7.10. Assume that A is asymptotically stable. Then, the positive-semidefinite matrix Q ∈ Rn×n defined by (12.7.2) exists. Furthermore, Q is positive definite if and only if (A, B) is controllable.

12.8 Stabilizability 

Let A ∈ Rn×n and B ∈ Rn×m, and define q = dim C(A, B). Definition 12.8.1. (A, B) is stabilizable if Su(A) ⊆ C(A, B).

(12.8.1)

Proposition 12.8.2. Let K ∈ Rm×n. Then, (A, B) is stabilizable if and only if (A + BK, B) is stabilizable. Proposition 12.8.3. The following statements are equivalent: i) (A, B) is stabilizable. ii) There exists an orthogonal matrix S ∈ Rn×n such that (12.6.21) holds, where A1 ∈ Rq×q, B1 ∈ Rq×m, (A1, B1) is controllable, and A2 ∈ R(n−q)×(n−q) is asymptotically stable. iii) There exists a nonsingular matrix S ∈ Rn×n such that (12.6.21) holds, where A1 ∈ Rq×q, B1 ∈ Rq×m, (A1, B1) is controllable, and A2 ∈ R(n−q)×(n−q) is asymptotically stable. iv) Every CRHP eigenvalue of (A, B) is controllable. Proof. To prove that i) =⇒ ii), assume that (A, B) is stabilizable so that u s Su (A) = N[μA (A)] = R[μA (A)] ⊆ C(A, B). Using Theorem 12.6.8, it follows that there exists an orthogonal matrix S ∈ Rn×n such that (12.6.21) is satisfied,

  where s A1 ∈ Rq×q and (A1, B1) is controllable. Thus, R[μA (A)] ⊆ C(A, B) = R S I0q . Next, note that

 s μA (A)

=S

s (A1 ) μA 0

B12s s μA (A2 )



S −1,

where B12s ∈ Rq×(n−q), and suppose that A2 is not asymptotically stable with s s CRHP eigenvalue λ. Then, λ ∈ / roots(μA ), and thus μA (A2 ) = 0. Let x2 ∈ Rn−q s satisfy μA(A2 )x2 = 0. Then,        B12s x2 Iq 0 s = C(A, B), ∈ /R S =S μA(A)S s x2 0 μA (A2 )x2 which implies that Su(A) is not contained in C(A, B). Hence, A2 is asymptotically stable. The statement ii) =⇒ iii) is immediate. To prove that iii) =⇒ iv), let λ ∈ spec(A) be a CRHP eigenvalue of A. Since A2 is asymptotically stable, it follows that λ ∈ / spec(A2 ). Consequently, Proposition 12.6.13 implies that λ is not an

821

LINEAR SYSTEMS AND CONTROL THEORY

uncontrollable eigenvalue of (A, B), and thus λ is a controllable eigenvalue of (A, B). To prove that iv) =⇒ i), let S ∈ Rn×n be nonsingular and such that A and B have the form (12.6.21), where A1 ∈ Rq×q, B1 ∈ Rq×m, and (A1, B1) is controllable. Since every CRHP eigenvalue of (A, B) is controllable, it follows from Proposition 12.6.13

 A2 is asymptotically stable. From Fact 11.23.4 it follows  that that Su(A) ⊆ R S I0q = C(A, B), which implies that (A, B) is stabilizable. The following result, which is a restatement of the equivalence of i) and iv) of Proposition 12.8.3, is the PBH test for stabilizability. Corollary 12.8.4. The following statements are equivalent: i) (A, B) is stabilizable. ii) For all s ∈ CRHP, rank



sI − A B

= n.

(12.8.2)

Proposition 12.8.5. The following statements are equivalent: i) A is asymptotically stable. ii) (A, B) is controllably asymptotically stable and stabilizable. Proof. To prove that i) =⇒ ii), assume that A is asymptotically stable. Then, Su(A) = {0}, and Ss (A) = Rn. Thus, Su(A) ⊆ C(A, B), and C(A, B) ⊆ Ss(A). To prove that ii) =⇒ i), assume that (A, B) is stabilizable and controllably asymptotically stable. Then, Su(A) ⊆ C(A, B) ⊆ Ss(A), and thus Su(A) = {0}. As an alternative proof that ii) =⇒ i), note that, since (A, B) is stabilizable, it follows from Proposition 12.5.3 that there exists a nonsingular matrix S ∈ Rn×n such that (12.6.21) holds, where A1 ∈ Rq×q, B1 ∈ Rq×m, (A1, B1) is controllable, and A2 ∈ R(n−q)×(n−q) is asymptotically stable. Then,   ,∞ ∞ tA1 T tA1T e B B e dt 0 T 1 1 0 S −1. etABBTetA dt = S 0 0 0

Since the integral on the left-hand side exists by assumption, the integral on the right-hand side also exists. Since (A1, B1) is controllable, it follows from vii) of ,∞  tA1T Theorem 12.6.18 that Q1 = 0 etA1B1BT dt is positive definite. 1e q Now, let λ be an eigenvalue of AT 1 , and let x1 ∈ C be an associated eigen vector. Consequently, α = x∗1 Q1x1 is positive, and ∞

α=

x∗1 0

e

∞ λt

λt B1BT 1e

dtx1 =

x∗1B1BT 1 x1 0

e2(Re λ)t dt.

,∞ Hence, 0 e2(Re λ)t dt exists, and thus Re λ < 0. Consequently, A1 is asymptotically stable, and thus A is asymptotically stable.

822

CHAPTER 12

Corollary 12.8.6. The following statements are equivalent: i) There exists a positive-semidefinite matrix Q ∈ Rn×n satisfying (12.7.3), and (A, B) is stabilizable. ii) A is asymptotically stable. Proof. This result follows from Proposition 12.7.3 and Proposition 12.8.5.

12.9 Realization Theory Given a proper rational transfer function G, we wish to determine (A, B, C, D) such that (12.2.11) holds. The following terminology is convenient. Definition 12.9.1. Let G ∈ Rl×m(s). If l = m = 1, then G is a singleinput/single-output (SISO) rational transfer function; if l = 1 and m > 1, then G is a multiple-input/single-output (MISO) rational transfer function; if l > 1 and m = 1, then G is a single-input/multiple-output (SIMO) rational transfer function; and, if l > 1 or m > 1, then G is a multiple-input/multiple output (MIMO) rational transfer function. n×n n×m Definition 12.9.2. Let G ∈ Rl×m , prop(s), and assume that A ∈ R  , B ∈ R A B l×n l×m −1 C ∈ R , and D ∈ R satisfy G(s) = C(sI − A) B + D. Then, C D is a realization of G, which is written as   A B G∼ . (12.9.1) C D

The order of the realization (12.9.1) is the order of A. Finally, (A, B, C) is controllable and observable if (A, B) is controllable and (A, C) is observable. Suppose that n = 0. Then, A, B, and C are empty matrices, and G ∈ Rl×m prop(s) is given by G(s) = 0l×0 (sI0×0 − 00×0 )−1 00×m + D = 0l×m + D = D.   00×0 00×m Therefore, the order of the realization 0 is zero. D

(12.9.2)

l×0

Although the realization (12.9.1) is not unique, the matrix D is unique and is given by D = G(∞). (12.9.3)     A B A B Furthermore, note that G ∼ C D if and only if G − D ∼ C 0 . Therefore, it suffices to construct realizations for strictly proper transfer functions. The following result shows that every strictly proper, SISO rational transfer function Ghas a realization. In fact, two realizations are the controllable   A B canonical  Ac Bc o o form G ∼ C and the observable canonical form G ∼ . If G is 0 C 0 c

exactly proper, then a realization can be obtained for G − G(∞).

o

823

LINEAR SYSTEMS AND CONTROL THEORY

Proposition 12.9.3. Let G ∈ Rprop(s) be the SISO strictly proper rational transfer function G(s) = Then, G ∼



0 −β0

 , where Ac , Bc , Cc are defined by ⎤ ⎡ 1 0 ··· 0 0 ⎥ ⎢ 0 0 1 ··· 0 ⎥ ⎢ ⎥  ⎢ . .. .. .. .. , B = ⎥ ⎢ .. c . . . . ⎥ ⎢ ⎦ ⎣ 0 0 0 ··· 1 1 −β1 −β2 · · · −βn−1

α0

α1

Ac Cc

Bc 0

⎡

⎢ ⎢ ⎢ Ac = ⎢ ⎢ ⎣

0 0 .. .



 Cc =  Ao and G ∼ C



Bo 0

o

⎡

⎢ ⎢ ⎢ Ao = ⎢ ⎢ ⎣ 

 Co =

αn−1sn−1 + αn−2 sn−2 + · · · + α1s + α0 . sn + βn−1sn−1 + · · · + β1s + β0



α2

···

···

⎤ ⎥ ⎥ ⎥ ⎥, ⎥ ⎦

(12.9.5)

αn−1 ,

 , where Ao , Bo , Co are defined by ⎤ ⎡ 0 0 ··· 0 −β0 α0 ⎢ α1 1 0 ··· 0 −β1 ⎥ ⎥ ⎢  ⎢ 0 1 ··· 0 −β2 ⎥ ⎥, Bo = ⎢ α2 ⎥ ⎢ .. .. .. .. . . ⎦ ⎣ . . . . . 0 0 · · · 1 −βn−1 αn−1 0 0

(12.9.4)

(12.9.6)

⎤ ⎥ ⎥ ⎥ ⎥, ⎥ ⎦

(12.9.7)

0 1 .

(12.9.8)

Furthermore, (Ac , Bc ) is controllable, and (Ao , Co ) is observable. Finally, the following statements are equivalent: i) The numerator and denominator of G given in (12.9.4) are coprime. ii) (Ac , Cc ) is observable. iii) (Ac , Bc , Cc ) is controllable and observable. iv) (Ao , Bo ) is controllable. v) (Ao , Bo , Co ) is controllable and observable. Proof. The realizations can be verified directly. ⎡ 0 0 0 ⎢ ⎢ 0 0 0 ⎢ ⎢ .. .. . ⎢ . .. . ⎢ K(Ac , Bc ) = O(Ao , Co ) = ⎢ ⎢ 0 0 1 ⎢ ⎢ 1 −βn−1 ⎣ 0 1 −βn−1

−βn−2

Furthermore, note that ⎤ ··· 0 1 ⎥ . .. 1 −βn−1 ⎥ ⎥ ⎥ .. . . . . ⎥ . . . ⎥. ⎥ . . . −β3 −β2 ⎥ ⎥ ⎥ · · · −β2 −β1 ⎦ ···

−β1

−β0

It follows from Fact 2.13.9 that det K(Ac , Bc ) = det O(Ao , Co ) = (−1)n/2 , which implies that (Ac , Bc ) is controllable and (Ao , Co ) is observable.

824

CHAPTER 12

To prove the last statement, let p, q ∈ R[s] denote the numerator and denominator, respectively, of G in (12.9.4). Then, for n = 2,   1 −β1 , K(Ao , Bo ) = OT(Ac , Cc ) = B(p, q)Iˆ 0 1 where B(p, q) is the Bezout matrix of p and q. It follows from ix) of Fact 4.8.6 that B(p, q) is nonsingular if and only if p and q are coprime. The following result shows that every proper rational transfer function has a realization. Theorem 12.9.4. Let G ∈ Rl×m exist A ∈ Rn×n, B ∈ Rn×m, prop(s). Then, there  A B C ∈ Rl×n, and D ∈ Rl×m such that G ∼ C D . 

Aij Cij

Proof. By Proposition 12.9.3, every entry G(i,j) of G has a realization G(i,j) ∼  Bij . Combining these realizations yields a realization of G. D ij

Proposition 12.9.5. Let G ∈ Rl×m prop(s) have the nth-order realization  A B n×n , let S ∈ R , and assume that S is nonsingular. Then, C D   SAS −1 SB . (12.9.9) G∼ CS −1 D     A B SAS −1 SB . If, in addition, C D is controllable and observable, then so is −1 CS D 

Definition 12.9.6. Let G ∈ Rl×m prop(s), and let   ˆ  A B A order realizations of G. Then, C D and Cˆ



A C



ˆ B D −1

B D



 and

ˆ A ˆ C

ˆ B D

 be nth-

are equivalent if there exists

ˆ = SB, and Cˆ = CS −1. a nonsingular matrix S ∈ Rn×n such that Aˆ = SAS , B The following result shows that the Markov parameters of a rational transfer function are independent of the realization.  A B  Proposition 12.9.7. Let G ∈ Rl×m , where (s), and assume that G ∼ prop C D  ˆ ˆ  A B ˆ and, for all k ≥ 0, A ∈ Rn×n, and G ∼ Cˆ Dˆ , where A ∈ Rnˆ ׈n. Then, D = D, ˆ CAkB = CˆAˆkB. (s), assume that G has the nth-order realProposition 12.9.8. Let G ∈ Rl×m     prop A1 B1 A2 B2 izations C and C , and assume that both of these realizations are D D 1 2 controllable and observable. Then, these realizations are equivalent. Furthermore, there exists a unique matrix S ∈ Rn×n such that     A2 B2 SA1S −1 SB1 = . (12.9.10) C2 D C1S −1 D

825

LINEAR SYSTEMS AND CONTROL THEORY

In fact,

−1 T  O2 O1, S = OT 2 O2

  T −1 S −1 = K1 KT , 2 K2 K2

(12.9.11)

  K(Ai , Bi ) and Oi = O(Ai , Ci ). where, for i = 1, 2, Ki =

 A B   A B  1 1 2 2 Proof. By Proposition 12.9.7, the realizations C and D C2 D 1 generate the same Markov parameters.  Hence, O1A1K1 = O2 A2 K2 , O1B1 = O2 B2 , A2 B2 and C1K1 = C2 K2 . Since C is controllable and observable, it follows that D 2

T −1 the n × n matrices K2 KT 2 and O2 O2 are nonsingular. Consequently, A2 = SA1S , −1 B2 = SB1, and C2 = C1S .

To prove uniqueness, assume there exists a matrix Sˆ ∈ Rn×n such that A2 = −1 ˆ 1, and C2 = C1Sˆ−1. Then, it follows that O1Sˆ = O2 . Since ˆ ˆ SA1S , B2 = SB ˆ = 0. Consequently, S = S. ˆ O1S = O2 , it follows that O1(S − S) Corollary 12.9.9. Let G ∈ Rprop(s) be given by (12.9.4), assume that G   A B has the nth-order controllable and observable realization C 0 , and define Ac , Bc , Cc by (12.9.5), (12.9.6) and Ao , Bo , Co by (12.9.7), (12.9.8). Furthermore,  define Sc = [O(A, B)]−1 O(Ac , Bc ). Then, Sc−1 = K(A, B)[K(Ac , Bc )]−1 and



Sc ASc−1 CSc−1

ScB 0



 =

Ac Cc

Bc 0

(12.9.12)  .

(12.9.13)



Furthermore, define So = [O(A, B)]−1O(Ao , Bo ). Then, So−1 = K(A, B)[K(Ao , Bo )]−1 and



So ASo−1 CSo−1

SoB 0



 =

Ao Co

Bo 0

(12.9.14)  .

(12.9.15)

The following result, known as the Kalman decomposition, is useful for constructing controllable and observable realizations. Proposition 12.9.10. Let G ∈ Rl×m prop(s), where G ∼



exists a nonsingular matrix S ∈ Rn×n such that ⎡ ⎡ ⎤ A1 B1 0 A13 0 ⎢ A21 A2 A23 A24 ⎥ −1 ⎢ B2 ⎥S , B = S⎢ A = S⎢ ⎣ 0 ⎣ 0 0 A3 0 ⎦ 0 0 A43 A4 0 C=



C1

0 C3

0 S −1,

A C

B D

 . Then, there

⎤ ⎥ ⎥, ⎦

(12.9.16)

(12.9.17)

826

CHAPTER 12

 A1 0 B1  is controllable, and where, for i ∈ {1, . . . , 4}, Ai ∈ Rni ×ni , A21 A2 , B2  A1 A13  C C 0 A3 , [ 1 3 ] is observable. Furthermore, the following statements hold: i) (A, B) is stabilizable if and only if A3 and A4 are asymptotically stable. ii) (A, B) is controllable if and only if A3 and A4 are empty. iii) (A, C) is detectable if and only if A2 and A4 are asymptotically stable. iv) (A, C) is observable if and only if A2 and A4 are empty.  A B  1 1 . v) G ∼ C D 1   A1 B1 vi) The realization C is controllable and observable. D 1

Proof. Let α ≤ 0 be such that A + αI is asymptotically stable, and let Q ∈ Rn×n and P ∈ Rn×n denote the controllability and observability Gramians of the system (A + αI, B, C). Then, Theorem 8.3.5 implies that there exists a nonsingular matrix S ∈ Rn×n such that ⎡ ⎤ ⎡ ⎤ Q1 P1 0 0 ⎢ ⎥ T ⎢ ⎥ −1 Q2 0 ⎥S , P = S −T⎢ ⎥S , Q = S⎢ ⎣ ⎦ ⎣ ⎦ P2 0 0 0 0 0 where Q1 and P1 are the same order, and where Q1, Q2 , P1, and P2 are positive definite and diagonal. The form of SAS −1, SB, and CS −1 given by (12.9.17) now follows from (12.7.3) and (12.4.3) with A replaced by A + αI, where, as in the proof of Theorem 12.6.8, SAS −1 = S(A + αI)S −1 −αI. Finally,  statements i)–v) are immediate, while it can be verified directly that G.

A1 C1

B1 D1

is a realization of

Note that the uncontrollable multispectrum of (A, B) is given by mspec(A3 )∪ mspec(A4 ), while the unobservable multispectrum of (A, C) is given by mspec(A2 )∪ mspec(A4 ). Likewise, the uncontrollable-unobservable multispectrum of (A, B, C) is given by mspec(A4 ). Let G ∼ Ril×n by



A C

B 0

 . Then, define the i-step observability matrix Oi (A, C) ∈ ⎡

⎤ C ⎢ CA ⎥  ⎥ .. Oi (A, C) = ⎢ ⎣ ⎦ . i−1 CA

and the j-step controllability matrix Kj (A, B) ∈ Rn×jm by

 Kj (A, B) = B AB · · · Aj−1B .

(12.9.18)

(12.9.19)

Note that O(A, C) = On (A, C) and K(A, B) = Kn (A, B). Furthermore, define the Markov block-Hankel matrix Hi,j,k (G) ∈ Ril×jm of G by  Oi (A, C)Ak Kj (A, B). Hi,j,k (G) =

(12.9.20)

827

LINEAR SYSTEMS AND CONTROL THEORY

Note that Hi,j,k (G) is the block-Hankel matrix of Markov parameters given by ⎤ ⎡ CAkB CAk+1B CAk+2B · · · CAk+j−1B ⎥ ⎢ . . . ⎥ ⎢ CAk+1B CAk+2B .. .. .. ⎥ ⎢ ⎥ ⎢ . . . . . . . . k+2 ⎥ . . . . B Hi,j,k (G) = ⎢ CA ⎥ ⎢ ⎥ ⎢ . . . . . .. . . . . ⎥ ⎢ . . . . ⎦ ⎣ . . . .. .. . . CAk+j+i−2B CAk+i−1B ⎡

Hk+1

⎢ ⎢ Hk+2 ⎢ ⎢ =⎢ ⎢ Hk+3 ⎢ . ⎢ .. ⎣ Hk+i

Hk+2 Hk+3 . .. .

..

.

..

Hk+3 . .. ..

.

.

..

.

..

··· . .. ..

.

.

..

.

..

Hk+j . .. .. .

.

..

⎤ ⎥ ⎥ ⎥ ⎥ ⎥. ⎥ ⎥ ⎥ ⎦

(12.9.21)

Hk+j+i−1

Note that Hi,j,0 (G) = Oi (A, C)Kj (A, B)

(12.9.22)

Hi,j,1 (G) = Oi (A, C)AKj (A, B).

(12.9.23)

 Hn,n,0 (G) = O(A, C)K(A, B). H(G) =

(12.9.24)

and

Furthermore, define

The following result provides a MIMO extension of Fact 4.8.8.   A B Proposition 12.9.11. Let G ∼ C 0 , where A ∈ Rn×n. Then, the following statements are equivalent:   A B i) The realization C 0 is controllable and observable. ii) rank H(G) = n. iii) For all i, j ≥ n, rank Hi,j,0 (G) = n. iv) There exist i, j ≥ n such that rank Hi,j,0 (G) = n. Proof. The equivalence of ii), iii), and iv) follows from Fact 2.11.7. To prove that i) =⇒ ii), note that, since the n × n matrices OT(A, C)O(A, C) and K(A, B)KT(A, B) are positive definite, it follows that n = rank OT(A, C)O(A, C)K(A, B)KT(A, B) ≤ rank H(G) ≤ n. Conversely, n = rank H(G) ≤ min{rank O(A, C), rank K(A, B)} ≤ n.

828

CHAPTER 12

  A B Proposition 12.9.12. Let G ∼ C 0 , where A ∈ Rn×n, assume that  A B  is controllable and observable, and let i, j ≥ 1 be such that rank Oi (A, C) C 0  ˆ ˆ  A B = rank Kj (A, B) = n. Then, G ∼ Cˆ 0 , where  + Aˆ = O+ i (A, C)Hi,j,1 (G)Kj (A, B),   Im  ˆ , B = Kj (A, B) 0(j−1)n×m  Cˆ =



0l×(i−1)l

Il

(12.9.25) (12.9.26)

Oi (A, C).

(12.9.27)

 Proposition 12.9.13. Let G ∈ Rl×m prop (s), let i, j ≥ 1, define n = il×n n×jm rank Hi,j,0 (G), and let L ∈ R and R ∈ R be such that Hi,j,0 (G) = LR. Then, the realization ⎡  ⎤  Im + + L H (G)R R i,j,1 ⎢ ⎥ 0(j−1)n×m ⎥ G∼⎢ (12.9.28) ⎣ ⎦

Il 0l×(i−1)l L 0

is controllable and observable. A rational transfer function G ∈ Rl×m prop (s) can have realizations of different orders. For example, letting A = 1, 

and Aˆ =

1 0 0 1

B = 1,



 ,

ˆ= B

1 0

C = 1,  ,

Cˆ =

D=0

1

0 ,

ˆ = 0, D

it follows that 1 . s −1 It is usually desirable to find realizations whose order is as small as possible. ˆ ˆ −1 B ˆ +D ˆ = G(s) = C(sI − A)−1B + D = C(sI − A)

 A B  Definition 12.9.14. Let G ∈ Rl×m . Then, (s), and assume that G ∼ prop C D   A B is a minimal realization of G if its order is less than or equal to the order C D of every realization of G. In this case, we write   A B min G ∼ . (12.9.29) C D Note that the minimality of a realization is independent of D. 

A1 C1

The following result shows that the controllable and observable realization B1 of G in Proposition 12.9.10 is, in fact, minimal. D 1

829

LINEAR SYSTEMS AND CONTROL THEORY



A C

 A Corollary 12.9.15. Let G ∈ Rl×m(s), and assume that G ∼ C  B is minimal if and only if it is controllable and observable. D

B D

 . Then,

  A B Proof. To prove necessity, suppose that C D is either not controllable or not observable. Then, Proposition  12.9.10  can be used to construct a realization A B of G of order less than n. Hence, C D is not minimal.  A B  To prove sufficiency, assume that A ∈ Rn×n, and assume that C D is  ˆ ˆ  A B not minimal. Hence, G has a minimal realization Cˆ D of order n ˆ < n. Since the Markov parameters of G are independent of the realization, it follows from  A B Proposition 12.9.11 that rank H(G) = n ˆ < n. However, since C D is observable and controllable, it follows from Proposition 12.9.11 that rank H(G) = n, which is a contradiction.   A B and A ∈ Rn×n. Theorem 12.9.16. Let G ∈ Rl×m prop(s), where G ∼ C D Then, poles(G) ⊆ spec(A)

(12.9.30)

mpoles(G) ⊆ mspec(A).

(12.9.31)

and

Furthermore, the following statements are equivalent:   A B min . i) G ∼ C D ii) Mcdeg(G) = n. iii) mpoles(G) = mspec(A). Proof. See [1179, p. 319]. min

Definition 12.9.17. Let G ∈ Rl×m prop(s), where G ∼



A C

B D

 . Then, G is

(asymptotically stable, semistable, Lyapunov stable) if A is. Proposition 12.9.18. Let G = p/q ∈ Rprop(s), where p, q ∈ R[s], and assume that p and q are coprime. Then, G is (asymptotically stable, semistable, Lyapunov stable) if and only if q is. Proposition 12.9.19. Let G ∈ Rl×m prop(s). Then, G is (asymptotically stable, semistable, Lyapunov stable) if and only if every entry of G is.   A B min Definition 12.9.20. Let G ∈ Rl×m and A is asymp(s), where G ∼ prop C D  A B  totically stable. Then, the realization C D is balanced if the controllability and observability Gramians (12.7.2) and (12.4.2) are diagonal and equal.

830

CHAPTER 12



min

Proposition 12.9.21. Let G ∈ Rl×m prop(s), where G ∼



A C

B D n×n

asymptotically stable. there exists a nonsingular matrix S ∈ R  Then, SAS −1 SB is balanced. the realization G ∼ CS −1 D

and A is such that

Proof. It follows from Corollary 8.3.8 that there exists a nonsingular matrix S ∈ Rn×n such that SQS T and S −TPS −1 are diagonal, where Q and P are the controllability and   observability Gramians (12.7.2) and (12.4.2). Hence, the realization SAS −1 CS −1

SB D

is balanced.

12.10 Zeros In Section 4.7 the Smith-McMillan decomposition is used to define transmission zeros and blocking zeros of a transfer function G(s). We now define the invariant zeros of a realization of G(s) and relate these zeros to the transmission zeros. These zeros are related to the Smith zeros of a polynomial matrix as well as the spectrum of a pencil. 

Definition 12.10.1. Let G ∈ Rl×m prop(s), where G ∼

A C

B D

 . Then, the

Rosenbrock system matrix Z ∈ R(n+l)×(n+m) [s] is the polynomial matrix   sI − A B  Z(s) = . (12.10.1) C −D   A B Furthermore, z ∈ C is an invariant zero of the realization C D if rank Z(z) < rank Z. Let G ∈ Rl×m prop(s), where G ∼ the pencil Z(s) = P  =s



A C



B D

(12.10.2)

and A ∈ Rn×n, and note that Z is

   A −B , In 0 (s) −C D 0 0

In

0

0

0

Thus,

 Szeros(Z) = spec

and

 −

A −C

 mSzeros(Z) = mspec



A −C

A

−B

−C

D

−B D

 ,

−B D



(12.10.3)

A

In 0

 .

0 0

 , I0n  B by D

Hence, we define the set of invariant zeros of C   A B  = Szeros(Z) izeros C D

(12.10.4)  (12.10.5)

0 0

 .

(12.10.6)

831

LINEAR SYSTEMS AND CONTROL THEORY

 and the multiset of invariant zeros of  A mizeros C Note that P

   A −B , In 0 −C D 0 0

A C B D



B D



by 

= Szeros(Z).

is regular if and only if rank Z = n + min{l, m}.

The following result shows that a strictly proper transfer function with fullstate observation or full-state actuation has no invariant zeros.   A B Proposition 12.10.2. Let G ∈ Rl×m and A ∈ Rn×n. prop(s), where G ∼ C 0 Then, the following statements hold:   A B i) If m = n and B is nonsingular, then rank Z = n + rank C and C 0 has no invariant zeros.   A B ii) If l = n and C is nonsingular, then rank Z = n + rank B and C 0 has no invariant zeros. iii) If m = n and B is nonsingular, then P In

   0 , A −B −C 0 0 0

if rank C = min{l, n}.

iv) If l = n and C is nonsingular, then P In

   0 , A −B −C 0 0 0

if rank B = min{m, n}.

is regular if and only

is regular if and only

It is useful to note that, for all s ∈ spec(A),    sI − A B I 0 Z(s) = 0 −G(s) C(sI − A)−1 I  =

sI − A

0

C

−G(s)



I

(sI − A)−1B

0

I

Proposition 12.10.3. Let G ∈ Rl×m prop(s), where G ∼ then



(12.10.7)

 . A C

(12.10.8) B D

 . If s ∈ spec(A),

rank Z(s) = n + rank G(s).

(12.10.9)

rank Z = n + rank G.

(12.10.10)

Furthermore,

Proof. For s ∈ spec(A), (12.10.9) follows from (12.10.7). Therefore, it follows from Proposition 4.3.6 and Proposition 4.7.8 that rank Z = max rank Z(s) = s∈C

=n+

max s∈C\spec(A)

max s∈C\spec(A)

rank Z(s)

rank G(s) = n + rank G.



Note that the realization in Proposition 12.10.3 is not assumed to be minimal. Therefore, P A −B  , In 0  is (regular, singular) for one realization of G if and only −C D

0 0

832

CHAPTER 12

if it is (regular, singular) for every realization of G. In fact, the following result shows that P A −B  , In 0  is regular if and only if G has full rank. −C D

0 0



Corollary 12.10.4. Let G ∈ Rl×m prop(s), where G ∼ P

   A −B , In 0 −C D 0 0

A C

 . Then,

B D

is regular if and only if rank G = min{l, m}.

In the SISO case, it follows from (12.10.7) and (12.10.8) that, for all s ∈ C\ spec(A), det Z(s) = −[det(sI − A)]G(s). (12.10.11) Consequently, for all s ∈ C, det Z(s) = −C(sI − A)AB − [det(sI − A)]D.

(12.10.12)

The equality (12.10.12) also follows from Fact 2.14.2. In particular, if s ∈ spec(A), then det Z(s) = −C(sI − A)AB.

(12.10.13)

If, in addition, n ≥ 2 and rank(sI − A) ≤ n − 2, then it follows from Fact 2.16.8 that (sI − A)A = 0, and thus det Z(s) = 0.

(12.10.14)

Alternatively, in the case n = 1, it follows that, for all s ∈ C, (sI − A) = 1, and thus, for all s ∈ C, A

det Z(s) = −CB − (sI − A)D.

(12.10.15)

Next, it follows from (12.10.11) and (12.10.12) that G(s) =

C(sI − A)AB + [det(sI − A)]D − det Z(s) = . det(sI − A) det(sI − A)

(12.10.16)

Consequently, G = 0 if and only if det Z = 0. We now have the following result for scalar transfer functions. Corollary 12.10.5. Let G ∈ Rprop(s), where G ∼ lowing statements are equivalent: i) P

   A −B , In 0 −C D 0 0



A C

B D

is regular.

ii) G = 0. iii) rank G = 1. iv) det Z = 0. v) rank Z = n + 1. vi) C(sI − A)AB + [det(sI − A)]D is not the zero polynomial.

 . Then, the fol-

833

LINEAR SYSTEMS AND CONTROL THEORY

In this case,

 mizeros

and



A C

B D

A C

B D



= mroots(det Z)

(12.10.17)



= mtzeros(G) ∪ [mspec(A)\mpoles(G)].   A B min If, in addition, G ∼ , then C D  A B  = mtzeros(G). mizeros C D mizeros

(12.10.18)

(12.10.19)

Now, suppose that G is square, that is, l = m. Then, it follows from (12.10.7) and (12.10.8) that, for all s ∈ C\spec(A), det Z(s) = (−1)l [det(sI − A)] det G(s),

(12.10.20)

and thus det G(s) =

(−1)l det Z(s) . det(sI − A)

(12.10.21)

Furthermore, for all s ∈ C,

  [det(sI − A)]l−1 det Z(s) = (−1)l det C(sI − A)AB + [det(sI − A)]D . (12.10.22)

Hence, for all s ∈ spec(A), it follows that

det C(sI − A)AB = 0.

(12.10.23)

We thus have the following result for square transfer functions G that satisfy det G = 0. Corollary 12.10.6. Let G ∈ Rl×l prop(s), where G ∼ lowing statements are equivalent: i) P

   A −B , In 0 −C D 0 0



A C

B D

 . Then, the fol-

is regular.

ii) det G = 0. iii) rank G = l. iv) det Z = 0. v) rank Z = n + l. vi) det(C(sI − A)AB + [det(sI − A)]D) is not the zero polynomial. In this case,

 mizeros 

mizeros

A C

B D



A C

B D



= mroots(det Z),

= mtzeros(G) ∪ [mspec(A)\mpoles(G)],

(12.10.24) (12.10.25)

834

CHAPTER 12

and

 izeros

A C

min

If, in addition, G ∼



B D



= tzeros(G) ∪ [spec(A)\poles(G)].  A B , then C D   A B = mtzeros(G). mizeros C D

Example 12.10.7. Consider G ∈ R2×2 (s) defined by   s−1 0 s+1  . G(s) = s+1 0 s−1

(12.10.29)

where S1, S2 ∈ R2×2 [s] are the unimodular matrices   −1 (s − 1)2  S1 (s) = − 14 (s + 1)2 (s − 2) 14 (s + 2)  

S2 (s) =

1 4 (s

− 1)2 (s + 2) (s + 1)2 1 4 (s

− 2)

(12.10.27)

(12.10.28)

Then, the Smith-McMillan form of G is given by  1  0  s2 −1 G(s) = S1 (s) S2 (s), 0 s2 − 1

and

(12.10.26)

(12.10.30)

 .

(12.10.31)

1

Thus, mpoles(G) = mtzeros(G) = {1, −1}. Furthermore, a minimal realization of G is given by ⎡ ⎤ −1 0 1 0 0 1 0 1 ⎥ min ⎢ ⎥ G ∼ ⎢ (12.10.32) ⎣ − 2 0 1 0 ⎦. 0

2

0 1

Finally, note that det Z(s) = (−1)2[det(sI − A)] det G = s2 − 1, which confirms (12.10.27).   A B Theorem 12.10.8. Let G ∈ Rl×m . Then, prop(s), where G ∼ C D   A B \spec(A) ⊆ tzeros(G) (12.10.33) izeros C D and

  A B . tzeros(G)\poles(G) ⊆ izeros C D   A B min , then If, in addition, G ∼ C D  A B  \poles(G) = tzeros(G)\poles(G). izeros C D

(12.10.34)

(12.10.35)

835

LINEAR SYSTEMS AND CONTROL THEORY

Proof. To prove (12.10.33), let z ∈ izeros



A C

B D



\spec(A). Since z ∈ /

spec(A) it follows from Theorem 12.9.16 that z ∈ / poles(G). It now follows from Proposition 12.10.3 that n + rank G(z) = rank Z(z) < rank Z = n + rank G, which implies that rank G(z) < rank G. Thus, z ∈ tzeros(G). To prove (12.10.34), let z ∈ tzeros(G)\poles(G). Then, it follows from Proposition 12.10.3 that  rank Z(z)  = n + rank G(z) < n + rank G = rank Z, which implies A

B

that z ∈ izeros C D and Theorem 12.9.16.

. The last statement follows from (12.10.33), (12.10.34),

The following result is a stronger form of Theorem 12.10.8. Theorem 12.10.9. Let G ∈ Rl×m prop(s), where G ∼



A C

B D

 , let S ∈ Rn×n,

assume that S is nonsingular, and let A, B, and B1   A1CAhave

the form  (12.9.16), (12.9.17), 0 1 13 C1 C3 ] is observable. Then, where AA21 , is controllable and , [ A2 B2 0 A3  A B  1 1 (12.10.36) mtzeros(G) = mizeros C D 1

and

 mizeros

A C

B D



= mspec(A2 ) ∪ mspec(A3 ) ∪ mspec(A4 ) ∪ mtzeros(G). (12.10.37)

Proof. Defining Z by (12.10.1), note that, in the notation of Proposition 12.9.10, Z has the same Smith form as ⎡ ⎤ sI − A4 −A43 0 0 0 ⎢ ⎥ ⎢ 0 sI − A3 0 0 0 ⎥ ⎢ ⎥ ⎥  ⎢ ˜= −A23 sI − A2 −A21 B2 ⎥. Z ⎢ −A24 ⎢ ⎥ ⎢ 0 −A13 0 sI − A1 B1 ⎥ ⎣ ⎦ 0 C3 0 C1 −D ˜ = n + r, where Hence, it follows from Proposition 12.10.3 that rank Z = rank Z  ˜ r = rank G. Let p˜1, . . . , p˜n+r be the Smith polynomials of Z. Then, since p˜n+r is the monic greatest common divisor of all (n + r) × (n + r) subdeterminants of ˜ it follows that p˜n+r = χA χA χA pr , where pr is the rth Smith polynomial of Z, 1 2 3 sI−A

1 B1 C1 −D . Therefore,

  1 B1 . mSzeros(Z) = mspec(A2 ) ∪ mspec(A3 ) ∪ mspec(A4 ) ∪ mSzeros sI−A C1 −D Next, using the Smith-McMillan decomposition Theorem 4.7.5, it follows that there exist unimodular matrices S1 ∈ Rl×l [s] and S2 ∈ Rm×m [s] such that G = S1 D0−1N0 S2 , where ⎡ ⎡ ⎤ ⎤ q1 p1 0 0 ⎢ ⎢ ⎥ ⎥ .. .. ⎥ ⎥  ⎢  ⎢ . . D0 = ⎢ ⎥, N0 = ⎢ ⎥. ⎣ ⎣ ⎦ ⎦ qr pr 0 Il−r 0 0(l−r)×(m−r)

836

CHAPTER 12

ˆ ∈ R(n+l)×(n+m) [s] by Now, define the polynomial matrix Z ⎡ ⎤ 0(n−l)×l 0(n−l)×m In−l ⎥  ⎢ ˆ= D0 N0 S2 ⎦. Z ⎣ 0l×(n−l) 0l×(n−l)

S1

0l×m

ˆ is given by Since S1 is unimodular, it follows that the Smith form S of Z   0n×m In . S= 0l×n N0 ˆ = mSzeros(S) = mtzeros(G). Consequently, mSzeros(Z) Next, note that  rank

In−l

0(n−l)×l

0(n−l)×m

0l×(n−l)

D0

N 0 S2

and that G=



S1

0l×(n−l) min

Furthermore, G ∼



A1 C1

0l×m B1 D





⎡

In−l

⎢ = rank ⎣ 0l×(n−l)

0(n−l)×l

0l×(n−l)

D0

−1

 ˆ and sI−A1 , Consequently, Z C1

⎥ ⎦=n

D0

0l×(n−l)

In−l

⎤

0(n−l)×l S1



0(n−l)×m N 0 S2 B1 D

.

have no decou-

pling zeros [1173, pp. 64–70], and it thus follows from Theorem 3.1 of [1173, p.

ˆ and sI−A1 B1 have the same Smith form. Thus, 106] that Z C1 D   sI − A1 B1 ˆ = mtzeros(G). = mSzeros(Z) mSzeros C1 −D Consequently, mizeros



A1 C1

B1 D

 = mSzeros

 sI−A1 C1

B1 −D



= mtzeros(G),

which proves (12.10.36). Finally, to prove (12.10.33) note that   A B mizeros C D = mSzeros(Z)  1 = mspec(A2 ) ∪ mspec(A3 ) ∪ mspec(A4 ) ∪ mSzeros sI−A −C1

B1 −D



= mspec(A2 ) ∪ mspec(A3 ) ∪ mspec(A4 ) ∪ mtzeros(G). Proposition 12.10.10. Equivalent realizations have the same invariant zeros. Furthermore, invariant zeros are not changed by full-state feedback.

837

LINEAR SYSTEMS AND CONTROL THEORY

Proof. Let u = Kx + v, which leads to the rational transfer function   A + BK B . (12.10.38) GK ∼ C + DK D Since



zI − (A + BK) B C + DK −D    A B it follows that C D and



 =

zI − A C 

A + BK C + DK

B D

B −D



I −K

0 I

 ,

(12.10.39)

have the same invariant zeros.

The following result provides an interpretation of condition i) of Theorem 12.17.9.   A B Proposition 12.10.11. Let G ∈ Rl×m , and assume prop(s), where G ∼ C D 

that R = DTD is positive definite. Then, the following statements hold: i) rank Z = n + m.

 A B  ii) z ∈ C is an invariant zero of C D if and only if z is an unobservable 

 eigenvalue of A − BR−1DTC, I − DR−1DT C . Proof. To prove i), assume that rank Z < n + m. Then, for every s ∈ C, there exists a nonzero vector [ xy ] ∈ N[Z(s)], that is,    x sI − A B = 0. y C −D Consequently, Cx − Dy = 0, which DTCx  implies that  − Ry = 0, and thus y = −1 T −1 T R D Cx. Furthermore, since sI − A + BR D C x = 0, choosing s ∈   spec A − BR−1DTC yields x = 0, and thus y = 0, which is a contradiction.  To prove ii), note that z is an invariant zero of

A C

B D

 if and only if

rank Z(z) < n + m, which holds if and only if there exists a nonzero vector [ xy ] ∈ N[Z(z)]. This condition is equivalent to   sI − A + BR−1DTC   x = 0, I − DR−1DT C where x = 0. This last condition

to the fact that z is an unobservable is equivalent eigenvalue of A − BR−1DTC, I − DR−1DT C . 

Corollary 12.10.12. Assume that R = DTD is positive definite, and assume 

 A B that A − BR−1DTC, I − DR−1DT C is observable. Then, C D has no invariant zeros.

838

CHAPTER 12

12.11 H2 System Norm Consider the system x(t) ˙ = Ax(t) + Bu(t),

(12.11.1)

y(t) = Cx(t),

(12.11.2)

where A ∈ Rn×n is asymptotically stable, B ∈ Rn×m, and C ∈ Rl×n. Then, for all t ≥ 0, the impulse response function defined by (12.1.18) is given by H(t) = CetAB.

(12.11.3)

The L2 norm of H(·) is given by ⎛

⎞1/2

∞

HL2 = ⎝ H(t)2F dt⎠ . 

(12.11.4)

0

The following result provides expressions for H(·)L2 in terms of the controllability and observability Gramians. Theorem 12.11.1. Assume that A is asymptotically stable. Then, the L2 norm of H is given by H2L2 = tr CQCT = tr BTPB, where Q, P ∈ R

n×n

(12.11.5)

satisfy AQ + QAT + BBT = 0, T

(12.11.6)

T

A P + PA + C C = 0.

(12.11.7)

Proof. Note that ∞

H2L2

T

tr CetABBTetA CT dt = tr CQCT,

= 0

where Q satisfies (12.11.6). The dual expression (12.11.7) follows in a similar manner or by noting that   tr CQCT = tr CTCQ = − tr ATP + PA Q   = − tr AQ + QAT P = tr BBTP = tr BTPB.  For the following definition, note that G(s)F = [tr G(s)G∗(s)]

1/2

.

(12.11.8)

Definition 12.11.2. The H2 norm of G ∈ Rl×m(s) is the nonnegative number ⎛

∞

 ⎝ 1 GH2 = 2π

−∞

⎞1/2 G(jω)2F dω⎠ .

(12.11.9)

839

LINEAR SYSTEMS AND CONTROL THEORY

The following result is Parseval’s theorem, which relates the L2 norm of the impulse response function to the H2 norm of its transform. Theorem 12.11.3. Let G ∈ Rl×m prop(s), where G ∼



A C

B 0

 , define H by

(12.11.3), and assume that A ∈ Rn×n is asymptotically stable. Then, ∞

∞ T

G(jω)G∗(jω) dω.

1 2π

H(t)H (t) dt =

(12.11.10)

−∞

0

Therefore, HL2 = GH2 . Proof. First note that

(12.11.11)

∞

H(t)e−st dt

G(s) = L{H(t)} = 0

and that

∞

H(t) =

G(jω)ejωt dω.

1 2π −∞

Hence,

⎛

∞

⎝1

H(t)H T(t)e−st dt = 0

2π

∞ 1 2π

1 2π

⎛

⎞

∞

G(jω)⎝ H T(t)e−(s−jω)t dt⎠dω

−∞ ∞

=

G(jω)ejωt dω⎠H T(t)e−st dt

−∞

0

=

⎞

∞

∞

0

G(jω)GT(s − jω) dω.

−∞

Setting s = 0 yields (12.11.7), while taking the trace of (12.11.10) yields (12.11.11). Corollary 12.11.4. Let G ∈ Rl×m prop(s), where G ∼



A C

B 0

 , and assume that

A ∈ Rn×n is asymptotically stable. Then, G2H2 = H2L2 = tr CQCT = tr BTPB,

(12.11.12)

where Q, P ∈ Rn×n satisfy (12.11.6) and (12.11.7), respectively. The following corollary of Theorem 12.11.3 provides a frequency-domain expression for the solution of the Lyapunov equation.

840

CHAPTER 12

Corollary 12.11.5. Let A ∈ Rn×n, assume that A is asymptotically stable, let B ∈ Rn×m, and define Q ∈ Rn×n by ∞

Q=

(jωI − A)−1BBT(jωI − A)−∗ dω.

1 2π

(12.11.13)

−∞

Then, Q satisfies AQ + QAT + BBT = 0.

(12.11.14)

Proof. This result follows directly from Theorem 12.11.3 with H(t) = etAB and G(s) = (sI − A)−1B. Alternatively, it follows from (12.11.14) that ∞

∞ −1

(jωI − A)

∞ −∗

dωQ + Q (jωI − A)

−∞

−∞

(jωI − A)−1BBT(jωI − A)−∗ dω.

dω =

−∞

Assuming that A is diagonalizable with eigenvalues λi = −σi + jωi , it follows that ∞

−∞

dω = jω − λi

∞

−∞

σi − jω σi π − j lim dω = 2 2 r→∞ σi + ω |σi |

r

−r

which implies that

σi2

ω dω = π, + ω2

∞

(jωI − A)−1 dω = πIn , −∞

which yields (12.11.13). See [317] for a proof of the general case. Proposition 12.11.6. Let G1, G2 ∈ Rl×m prop(s) be asymptotically stable rational transfer functions. Then, G1 + G2 H2 ≤ G1 H2 + G2 H2 . min

Proof. Let G1 ∼



A1 C1

B1 0



min

and G2 ∼



A2 C2

B2 0

(12.11.15)  , where A1 ∈ Rn1 ×n1

and A2 ∈ Rn2 ×n2. It follows from Proposition 12.13.2 that ⎡ ⎤ A1 0 B1 G1 + G2 ∼ ⎣ 0 A2 B2 ⎦. C1 C2 0  tr C1Q1C1T and G2 H2 = Now, Theorem 12.11.3 implies that G1H2 =  n1 ×n1 n2 ×n2 T tr C2 Q2 C2 , where Q1 ∈ R and Q2 ∈ R are the unique positive-definite T matrices satisfying A1Q1 + Q1A1T + B1B1T = 0 and A2 Q2 + Q2 AT 2 + B2 B2 = 0.   Furthermore, C1T

2 G2 + G2 H2 = tr C1 C2 Q , C2T where Q ∈ R(n1 +n2 )×(n1 +n2 ) is the unique, positive-semidefinite matrix satisfying    T   T A1 0 A1 0 B1 B1 Q+Q + = 0. 0 A2 0 A2 B2 B2

841

LINEAR SYSTEMS AND CONTROL THEORY

 It can be seen that Q =

Q1 Q12 QT 12 Q2

 , where Q1 and Q2 are as given above and

T where Q12 satisfies A1 Q12 + Q12 AT 2 + B1B2 = 0. Now, using the Cauchy-Schwarz inequality (9.3.17) and iii) of Proposition 8.2.4, it follows that   T T G1 + G2 2H2 = tr C1 Q1 C1T + C2 Q2 C2T + C2 QT 12 C1 + C1 Q12 C2 −1/2

1/2

= G12H2 + G2 2H2 + 2 tr C1Q12 Q2 Q2 C2T     T T T ≤ G12H2 + G2 2H2 + 2 tr C1Q12 Q−1 2 Q12 C1 tr C2 Q2 C2     ≤ G12H2 + G2 2H2 + 2 tr C1Q1 C1T tr C2 Q2 C2T = (G1H2 + G2 H2 )2.



12.12 Harmonic Steady-State Response The following result concerns the response of a linear system to a harmonic input. Theorem 12.12.1. For t ≥ 0, consider the linear system x(t) ˙ = Ax(t) + Bu(t),

(12.12.1)

u(t) = Re u0 ejω0 t ,

(12.12.2)

with harmonic input where u0 ∈ C and ω0 ∈ R is such that jω0 ∈ spec(A). Then, x(t) is given by 



x(t) = etA x(0) − Re (jω0 I − A)−1Bu0 + Re (jω0 I − A)−1Bu0 ejω0 t . (12.12.3) m

Proof. We have t tA

e(t−τ )ABRe(u0 ejω0 τ ) dτ

x(t) = e x(0) + 0

⎡

t

⎤

= etAx(0) + etA Re⎣ e−τA ejω0 τ dτBu0 ⎦ 0

⎡

t

⎤

= etAx(0) + etA Re⎣ eτ (jω0I−A) dτBu0 ⎦ 0

    = e x(0) + e Re (jω0 I − A)−1 et(jω0 I−A) − I Bu0 tA

tA

 

= etAx(0) + Re (jω0 I − A)−1 ejω0 tI − etA Bu0  



= etAx(0) + Re (jω0 I − A)−1 −etA Bu0 + Re (jω0 I − A)−1ejω0 tBu0 



= etA x(0) − Re (jω0 I − A)−1Bu0 + Re (jω0 I − A)−1Bu0 ejω0 t .



842

CHAPTER 12

Theorem 12.12.1 shows that the total response y(t) of the linear system G ∼  B to a harmonic input can be written as y(t) = ytrans (t) + yhss (t), where 0 the transient component 

  ytrans (t) = (12.12.4) CetA x(0) − Re (jω0 I − A)−1Bu0 

A C

depends on the initial condition and the input, and the harmonic steady-state component

yhss (t) = Re G(jω0 )u0 ejω0 t (12.12.5) depends only on the input. If A is asymptotically stable, then limt→∞ ytrans (t) = 0, and thus y(t) approaches its harmonic steady-state component yhss (t) for large t. Since the harmonic steady-state component is sinusoidal, it follows that y(t) does not converge in the usual sense. Finally, if A is semistable, then it follows from Proposition 11.8.2 that

  lim ytrans (t) = C(I − AA# ) x(0) − Re (jω0 I − A)−1Bu0 , (12.12.6) t→∞

which represents a constant offset to the harmonic steady-state component.  In the SISO case, let u0 = a0 (sin φ0 + j cos φ0 ), and consider the input

u(t) = a0 sin(ω0 t + φ0 ) = Re u0 ejω0 t.

(12.12.7)

Then, writing G(jω0 ) = Re Mejθ, it follows that yhss (t) = a0 Msin(ω0 t + φ0 + θ).

(12.12.8)

12.13 System Interconnections ∼ Let G ∈ Rl×m prop(s). We define the parahermitian conjugate G of G by

G∼ = GT(−s). 

(12.13.1)

The following result provides realizations for GT, G∼, and G−1. Proposition 12.13.1. Let Gl×m prop(s), and assume that G ∼   AT CT T G ∼ BT DT and

 G∼ ∼

−AT BT

−CT DT



A C

B D

 . Then, (12.13.2)

 .

Furthermore, if G is square and D is nonsingular, then   −1 −1 C BD A − BD . G−1 ∼ −D−1 C D−1

(12.13.3)

(12.13.4)

843

LINEAR SYSTEMS AND CONTROL THEORY

Proof. Since y = Gu, it follows that G−1 satisfies u = G−1 y. Since x˙ = −1 Ax + Bu y = Cx + Du, Cx + D−1 y, and thus x˙ =  and  it follows−1that  u = −D −1 −1 −1 Ax + B −D Cx + D y = A − BD C x + BD y. Note that, if G ∈ Rprop(s) and G ∼



A C

B D



, then G ∼



AT CT

BT D

 .

1 ×m1 2 ×m2 (s) and G2 ∈ Rlprop (s). If m2 = l2 , then the cascade Let G1 ∈ Rlprop interconnection of G1 and G2 shown in Figure 12.13.1 is the product G2 G1, while the parallel interconnection shown in Figure 12.13.2 is the sum G1 + G2. Note that G2 G1 is defined only if m2 = l1, whereas G1 + G2 requires that m1 = m2 and l1 = l2 .

u1 -

y1 = u 2 -

G1

y2

G2

-

Figure 12.13.1

Cascade Interconnection of Linear Systems

-

G1 + ? f y2 + 6

u1 -

-

G2

Figure 12.13.2

Parallel Interconnection of Linear Systems

Proposition 12.13.2. Let G1 ∼



⎡

A1 C1

A1 G2 G1 ∼ ⎣ B2 C1 D2 C1 and

⎡

A1 G1 + G2 ∼ ⎣ 0 C1

B1 D1

0 A2 C2 0 A2 C2



and G2 ∼



⎤ B1 B2 D1 ⎦ D2 D1

⎤ B1 ⎦. B2 D1 + D2

Proof. Consider the state space equations x˙ 1 = A1x1 + B1u1,

x˙ 2 = A2 x2 + B2 u2 ,

y1 = C1x1 + D1u1,

y2 = C2 x2 + D2 u2 .

A2 C2

B2 D2

 . Then,

(12.13.5)

(12.13.6)

844

CHAPTER 12

Since u2 = y1, it follows that x˙ 2 = A2 x2 + B2 C1x1 + B2 D1u1, y2 = C2 x2 + D2 C1x1 + D2 D1u1, and thus





x˙ 1 x˙ 2

 =

y2 =



A1 B2 C1 D2 C1

0 A2



x1 x2



 +

B1 B2 D1

 u1,

 

x1 C2 + D2 D1u1, x2

which yields the realization (12.13.5) of G2 G1. The realization (12.13.6) for G1 + G2 can be obtained by similar techniques. It is sometimes useful to combine transfer functions by concatenating them into row, column, or block-diagonal transfer functions.    A1 B1 Proposition 12.13.3. Let G1 ∼ C D and G2 ∼ 1 1 ⎡ ⎤ A 0 B1 0 1

G1 G2 ∼ ⎣ 0 A2 0 B2 ⎦, C1 C2 D1 D2 



G1 0

G1 G2

0 G2





⎡

A1 ⎢ 0 ∼⎢ ⎣ C1 0 ⎡

A1 ⎢ 0 ∼⎢ ⎣ C1 0

0 A2 0 C2

⎤ B1 B2 ⎥ ⎥, D1 ⎦ D2

0 A2 0 C2

B1 0 D1 0

⎤ 0 B2 ⎥ ⎥. 0 ⎦ D2

A2 C2

B2 D2

 . Then, (12.13.7)

(12.13.8)

(12.13.9)

Next, we interconnect a pair of systems G1, G2 by means of feedback as shown in Figure 12.13.3. It can be seen that u and y are related by yˆ = (I + G1G2 )−1 G1 u ˆ

(12.13.10)

ˆ. yˆ = G1(I + G2 G1 )−1 u

(12.13.11)

or

The equivalence of (12.13.10) and (12.13.11) follows from the push-through identity given by Fact 2.16.16, (I + G1G2 )−1G1 = G1(I + G2 G1 )−1.

(12.13.12)

A realization of this rational transfer function is given by the following result.

845

LINEAR SYSTEMS AND CONTROL THEORY

u -f − 6

-

y

G1

G2



Figure 12.13.3

Feedback Interconnection of Linear Systems

Proposition 12.13.4. Let G1 ∼



sume that det(I + D1D2 ) = 0. Then,

A1 C1

B1 D1



and G2 ∼



A2 C2

B2 D2

 , and as-

−1

[I + G1G2 ] G1 ⎡ ⎢ ∼⎣

⎤

A1 − B1 (I + D2D1 )−1 D2 C1

−B1 (I + D2D1 )−1C2

B1(I + D2D1 )−1

B2 (I + D1D2 )−1C1

A2 − B2 (I + D1D2 )−1D1C2

B2 (I + D1D2 )−1D1

(I + D1D2 )−1C1

−(I + D1D2 )−1D1C2

(I + D1D2 )−1D1

⎥ ⎦.

(12.13.13)

12.14 Standard Control Problem The standard control problem shown in Figure 12.14.1 involves four distinct signals, namely, an exogenous input w, a control input u, a performance variable z, and a feedback signal y. This system can be written as     w(s) ˆ zˆ(s) , (12.14.1) = G(s) u ˆ(s) yˆ(s) where G(s) is partitioned as  G=

with the realization

⎡



G11 G21

A G ∼ ⎣ E1 C

D1 E0 D2

G12 G22



⎤ B E2 ⎦, D

(12.14.2)

(12.14.3)

which represents the equations x˙ = Ax + D1 w + Bu,

(12.14.4)

z = E1 x + E0 w + E2 u,

(12.14.5)

y = Cx + D2 w + Du.

(12.14.6)

846

CHAPTER 12

Consequently,



G(s) =

E1(sI − A)−1D1 + E0

E1(sI − A)−1B + E2

C(sI − A)−1D1 + D2

C(sI − A)−1B + D

 ,

which shows that G11, G12 , G21, and G22 have the realizations     A D1 A B , G12 ∼ , G11 ∼ E1 E0 E1 E2  G21 ∼

A C

D1 D2



 G22 ∼

,

w -

Gzw

Gzu

u -

Gyw

Gyu

Gc

B D

A C

z

(12.14.7)

(12.14.8)

 .

(12.14.9)

-

y



Figure 12.14.1

Standard Control Problem Letting Gc denote a feedback controller with realization   Ac Bc , Gc ∼ Cc Dc

(12.14.10)

we interconnect G and Gc according to y (s). u ˆ(s) = Gc (s)ˆ

(12.14.11)

˜ satisfying zˆ(s) = G(s) ˜ w(s) The resulting rational transfer function G ˆ is thus given by ˜ = G11 + G12 Gc (I − G22 Gc )−1G21 G (12.14.12) or

˜ = G11 + G12 (I − Gc G22 )−1Gc G21. G

˜ is given by the following result. A realization of G

(12.14.13)

847

LINEAR SYSTEMS AND CONTROL THEORY

Proposition 12.14.1. Let G and Gc have the realizations (12.14.3) and (12.14.10), and assume that det(I − DDc ) = 0. Then, ⎡ ˜∼⎢ G ⎣

A + BDc (I − DDc )−1C

B(I − DcD)−1Cc

D1 + BDc(I + DDc )−1D2

Bc (I − DDc )−1 C

Ac + Bc (I − DDc )−1DCc

Bc(I − DDc )−1D2

E1 + E2Dc(I − DDc )−1C

E2(I − DcD)−1Cc

E0 + E2Dc (I − DDc )−1D2

⎤ ⎥ ⎦.

(12.14.14) The realization (12.14.14) can be simplified when DDc = 0. For example, if D = 0, then ⎡ ⎤ A + BDc C BCc D1 + BDc D2 ˜∼⎣ ⎦, Bc C Ac Bc D2 G (12.14.15) E1 + E2 Dc C E2 Cc E0 + E2 Dc D2 whereas, if Dc = 0, then

⎡

A ˜ ∼ ⎣ Bc C G E1

BCc Ac + Bc DCc E2 Cc

Finally, if both D = 0 and Dc = 0, then ⎡ A BCc ˜ ⎣ B C Ac G∼ c E1 E2 Cc

⎤ D1 Bc D2 ⎦. E0

⎤ D1 Bc D2 ⎦. E0

(12.14.16)

(12.14.17)

The feedback interconnection shown in Figure 12.14.1 forms the basis for the standard control problem in feedback control. For this problem the signal w is an exogenous signal representing a command or a disturbance, while the signal z is the performance variable, that is, the variable whose behavior reflects the performance of the closed-loop system. The performance variable may or may not be physically measured. The controlled input or the control u is the output of the feedback controller Gc , while the measurement signal y serves as the input to the feedback controller Gc . The standard control problem is the following: Given knowledge of w, determine Gc to minimize a performance criterion J(Gc ).

12.15 Linear-Quadratic Control Let A ∈ Rn×n and B ∈ Rn×m, and consider the system x(t) ˙ = Ax(t) + Bu(t), x(0) = x0 ,

(12.15.1) (12.15.2)

where t ≥ 0. Furthermore, let K ∈ Rm×n, and consider the full-state-feedback control law u(t) = Kx(t). (12.15.3)

848

CHAPTER 12

The objective of the linear-quadratic control problem is to minimize the quadratic performance measure ∞



J(K, x0 ) =

xT(t)R1x(t) + 2xT(t)R12 u(t) + uT(t)R2 u(t) dt,

0

where R1 ∈ Rn×n, R12 ∈ Rn×m, and R2 ∈ Rm×m. We assume that positive semidefinite and R2 is positive definite.

(12.15.4) 

R1 R12 RT 12 R2

 is

The performance measure (12.15.4) indicates the desire to maintain the state vector x(t) close to the zero equilibrium without an excessive expenditure of control effort. Specifically, the term xT(t)R1x(t) is a measure of the deviation of the state x(t) from the zero state, where the n×n positive-semidefinite matrix R1 determines how much weighting is associated with each component of the state. Likewise, the m × m positive-definite matrix R2 weights the magnitude of the control input. Finally, the cross-weighting term R12 arises naturally when additional filters are used to shape the system response or in specialized applications. Using (12.15.1) and (12.15.3), the closed-loop dynamic system can be written as x(t) ˙ = (A + BK)x(t)

(12.15.5)

so that ˜

x(t) = etA x0 ,

(12.15.6)

 where A˜ = A + BK. Thus, the performance measure (12.15.4) becomes

∞

∞ ˜T ˜ tA ˜ tA xT Re x0 dt 0e

˜ x (t)Rx(t) dt = T

J(K, x0 ) = 0

0 ∞

∞

˜T ˜ tA ˜ ˜T ˜ tA ˜ tA dtx0 xT Re dtx0 = tr etA Re = tr xT 0 e 0, 0

where

0

 T ˜= R1 + R12 K + K TR12 + K TR2 K. R

Now, consider the standard control ⎡ A G ∼ ⎣ E1 In

(12.15.7)

problem with plant ⎤ D1 B 0 E2 ⎦ 0 0

(12.15.8)

(12.15.9)

and full-state feedback u = Kx. Then, the closed-loop transfer function is given by   D A + BK 1 ˜∼ . (12.15.10) G E1 + E2 K 0 The following result shows that the quadratic performance measure (12.15.4) is equal to the H2 norm of a transfer function.

849

LINEAR SYSTEMS AND CONTROL THEORY

Proposition 12.15.1. Assume that D1 = x0 and     E1T R1 R12

E1 E2 , = T T R12 R2 E2

(12.15.11)

˜ be given by (12.15.10). Then, and let G ˜ 2 . J(K, x0 ) = G H2

(12.15.12)

Proof. This result follows from Proposition 12.1.2. For the following development, we assume that (12.15.11) holds so that R1, R12 , and R2 are given by R1 = E1TE1,

R12 = E1TE2 ,

R2 = E2TE2 .

(12.15.13)

To develop necessary conditions for the linear-quadratic control problem, we restrict K to the set of stabilizing gains 

S = {K ∈ Rm×n : A + BK is asymptotically stable}.

(12.15.14)

Obviously, S is nonempty if and only if (A, B) is stabilizable. The following result gives necessary conditions that characterize a stabilizing solution K of the linearquadratic control problem. Theorem 12.15.2. Assume that (A, B) is stabilizable, assume that K ∈ S solves the linear-quadratic control problem, and assume that (A + BK, D1 ) is controllable. Then, there exists an n × n positive-semidefinite matrix P such that K is given by   T K = −R2−1 BTP + R12 (12.15.15) and such that P satisfies

where and

−1 T ˆ ˆ AˆT RP + PAR + R1 − PBR2 B P = 0,

(12.15.16)

 T A − BR2−1R12 AˆR =

(12.15.17)

 T ˆ1 = R1 − R12 R2−1R12 . R

(12.15.18)

Furthermore, the minimal cost is given by J(K) = tr P V,

(12.15.19)

 D1D1T. where V =

Proof. Since K ∈ S, it follows that A˜ is asymptotically stable. It then follows , ∞ tA˜T tA˜  ˜ e Re dt is positive semidefinite that J(K) is given by (12.15.19), where P = 0 and satisfies the Lyapunov equation ˜ = 0. A˜TP + PA˜ + R

(12.15.20)

850

CHAPTER 12

Note that (12.15.20) can be written as T + K TR2 K = 0. (A + BK)TP + P (A + BK) + R1 + R12 K + K TR12

(12.15.21)

To optimize (12.15.19) subject to the constraint (12.15.20) over the open set S, form the Lagrangian     ˜ , L(K, P, Q, λ0 ) = tr λ0 P V + Q A˜TP + PA˜ + R (12.15.22) where the Lagrange multipliers λ0 ≥ 0 and Q ∈ Rn×n are not both zero. Note that the n × n Lagrange multiplier Q accounts for the n × n constraint equation (12.15.20). The necessary condition ∂L/∂P = 0 implies ˜ + QA˜T + λ0V = 0. AQ

(12.15.23)

Since A˜ is asymptotically stable, it follows from Proposition 11.9.3 that, for all λ0 ≥ 0, (12.15.23) has a unique solution Q and, furthermore, Q is positive semidefinite. In particular, if λ0 = 0, then Q = 0. Since λ0 and Q are not both zero, we can set λ0 = 1 so that (12.15.23) becomes ˜ + QA˜T + V = 0. AQ

(12.15.24)

˜ D1 ) is controllable, it follows from Corollary 12.7.10 that Q is positive Since (A, definite. Next, evaluating ∂L/∂K = 0 yields   T R2 KQ + BTP + R12 Q = 0.

(12.15.25)

Since Q is positive definite, it follows from (12.15.25) that (12.15.15) is satisfied. Furthermore, using (12.15.15), it follows that (12.15.20) is equivalent to (12.15.16). With K given by (12.15.15), the closed-loop dynamics matrix A˜ = A + BK is given by   T A˜ = A − BR2−1 BTP + R12 , (12.15.26) where P is the solution of the Riccati equation (12.15.16).

12.16 Solutions of the Riccati Equation For convenience in the following development, we assume that R12 = 0. With this assumption, the gain K given by (12.15.15) becomes

Defining (12.15.26) becomes

K = −R2−1BTP.

(12.16.1)

 BR2−1BT, Σ=

(12.16.2)

A˜ = A − ΣP,

(12.16.3)

LINEAR SYSTEMS AND CONTROL THEORY

851

while the Riccati equation (12.15.16) can be written as ATP + PA + R1 − PΣP = 0.

(12.16.4)

Note that (12.16.4) has the alternative representation (A − ΣP )TP + P (A − ΣP ) + R1 + PΣP = 0,

(12.16.5)

which is equivalent to the Lyapunov equation

where

˜ = 0, A˜TP + PA˜ + R

(12.16.6)

 ˜= R1 + PΣP. R

(12.16.7)

By comparing (12.15.16) and (12.16.4), it can be seen that the linear-quadratic ˆ 1, 0, R2 ) are equivalent. control problems with (A, B, R1, R12 , R2 ) and (AˆR , B, R Hence, there is no loss of generality in assuming that R12 = 0 in the following ˆ 1, respectively. development, where A and R1 take the place of AˆR and R To motivate the subsequent development, the following examples demonstrate the existence of solutions under various assumptions on (A, B, E1 ). In the following four examples, (A, B) is not stabilizable. Example 12.16.1. Let n = 1, A = 1, B = 0, E1 = 0, and R2 > 0. Hence, (A, B, E1 ) has an ORHP eigenvalue that is uncontrollable and unobservable. In this case, (12.16.4) has the unique solution P = 0. Furthermore, since B = 0, it follows that A˜ = A. Example 12.16.2. Let n = 1, A = 1, B = 0, E1 = 1, and R2 > 0. Hence, (A, B, E1 ) has an ORHP eigenvalue that is uncontrollable and observable. In this case, (12.16.4) has the unique solution P = −1/2 < 0. Furthermore, since B = 0, it follows that A˜ = A. Example 12.16.3. Let n = 1, A = 0, B = 0, E1 = 0, and R2 > 0. Hence, (A, B, E1 ) has an imaginary eigenvalue that is uncontrollable and unobservable. In this case, (12.16.4) has infinitely many solutions P ∈ R. Hence, (12.16.4) has no maximal solution. Furthermore, since B = 0, it follows that, for every solution P, A˜ = A. Example 12.16.4. Let n = 1, A = 0, B = 0, E1 = 1, and R2 > 0. Hence, (A, B, E1 ) has an imaginary eigenvalue that is uncontrollable and observable. In this case, (12.16.4) becomes R1 = 0. Thus, (12.16.4) has no solution. In the remaining examples, (A, B) is controllable. Example 12.16.5. Let n = 1, A = 1, B = 1, E1 = 0, and R2 > 0. Hence, (A, B, E1 ) has an ORHP eigenvalue that is controllable and unobservable. In this case, (12.16.4) has the solutions P = 0 and P = 2R2 > 0. The corresponding closed-loop dynamics matrices are A˜ = 1 > 0 and A˜ = −1 < 0. Hence, the solution P = 2R2 is stabilizing, and the closed-loop eigenvalue 1, which does not depend on R2 , is the reflection of the open-loop eigenvalue −1 across the imaginary axis.

852

CHAPTER 12

Example 12.16.6. Let n = 1, A = 1, B = 1, E1 = 1, and R2 > 0. Hence, (A, B, E1 ) has an ORHP eigenvalue that  is controllable and observable.  In this case, (12.16.4) has the solutions P = R2 − R22 + R2 < 0 and P =  R2 + R22 + R2 > 0. The corresponding closed-loop dynamics matrices are A˜ = 1 + 1/R 2 > 0 and  ˜ A = − 1 + 1/R2 < 0. Hence, the positive-definite solution P = R2 + R22 + R2 is stabilizing. Example 12.16.7. Let n = 1, A = 0, B = 1, E1 = 0, and R2 > 0. Hence, (A, B, E1 ) has an imaginary eigenvalue that is controllable and unobservable. In this case, (12.16.4) has the unique solution P = 0, which is not stabilizing. Example 12.16.8. Let n = 1, A = 0, B = 1, E1 = 1, and R2 > 0. Hence, (A, B, E1 ) has an imaginary eigenvalue that √is controllable and √ observable. In this case, (12.16.4) has the solutions P = − R2 < √ 0 and P = R2 > √ 0. The corresponding closed-loop dynamics matrices√are A˜ = R2 > 0 and A˜ = − R2 < 0. Hence, the positive-definite solution P = R2 is stabilizing. 0 1 Example 12.16.9. Let n = 2, A = −1 0 , B = I2 , E1 = 0, and R2 = 1. Hence, as in Example 12.16.7, both eigenvalues of (A, B, E1 ) are imaginary, controllable, and unobservable. Taking the trace of (12.16.4) yields tr P 2 = 0. Thus, the only symmetric matrix P satisfying (12.16.4) is P = 0, which implies that A˜ = A. Consequently, the open-loop eigenvalues ±j are unaffected by the feedback gain (12.15.15) even though (A, B) is controllable. Example 12.16.10. Let n = 2, A = 0, B = I2 , E1 = I2 , and R2 = I. Hence, as in Example 12.16.8, both eigenvalues of (A, B, E1 ) are imaginary, controllable, and observable. Furthermore, (12.16.4) becomes P 2 = I. Requiring that P be symmetric, it follows that P is a reflector. Hence, P = I is the only positivesemidefinite solution. In fact, P is positive definite and stabilizing since A˜ = −I. Example 12.16.11. Let A = [ 10 02 ], B = [ 11 ], E1 = 0, and R2 = 1 so that (A, B) is controllable, although neither of the states is weighted. In this case, (12.16.4) has four positive-semidefinite solutions, which are given by         18 −24 2 0 0 0 0 0 , P2 = , P3 = , P4 = . P1 = −24 36 0 0 0 4 0 0

6 −12 , The corresponding feedback matrices are given by K1 =





K2 = −2 0 , K3 = 0 −4 , and K4 = 0 0 . Letting A˜i = A − ΣPi , it follows that spec(A˜1 ) = {−1, −2}, spec(A˜2 ) = {−1, 2}, spec(A˜3 ) = {1, −2}, and spec(A˜4 ) = {1, 2}. Thus, P1 is the only solution that stabilizes the closed-loop system, while the solutions P2 and P3 partially stabilize the closed-loop system. Note also that the closed-loop poles that differ from those of the open-loop system are mirror images of the open-loop poles as reflected across the imaginary axis. Finally, note that these solutions satisfy the partial ordering P1 ≥ P2 ≥ P4 and P1 ≥ P3 ≥ P4 , and that “larger” solutions are more stabilizing than “smaller” solutions. Moreover, letting J(Ki ) = tr PiV, it can be seen that larger solutions incur a greater closed-loop cost, with the greatest cost incurred by the stabilizing solution P4 . However, the cost expression J(K) = tr P V does not follow from (12.15.4) when A + BK is not asymptotically stable.

853

LINEAR SYSTEMS AND CONTROL THEORY

The following definition concerns solutions of the Riccati equation. Definition 12.16.12. A matrix P ∈ Rn×n is a solution of the Riccati equation (12.16.4) if P is symmetric and satisfies (12.16.4). Furthermore, P is the stabilizing solution of (12.16.4) if A˜ = A − ΣP is asymptotically stable. Finally, a solution Pmax of (12.16.4) is the maximal solution to (12.16.4) if P ≤ Pmax for every solution P to (12.16.4). Since the ordering “≤” is antisymmetric, it follows that (12.16.4) has at most one maximal solution. The uniqueness of the stabilizing solution is shown in the following section. Next, define the 2n × 2n Hamiltonian   A Σ  H= . R1 −AT

(12.16.8)

Proposition 12.16.13. The following statements hold: i) The Hamiltonian H is a Hamiltonian matrix. ii) χH has a spectral factorization, that is, there exists a monic polynomial p ∈ R[s] such that, for all s ∈ C, χH(s) = p(s)p(−s). iii) χH(jω) ≥ 0 for all ω ∈ R. iv) If either R1 = 0 or Σ = 0, then mspec(H) = mspec(A) ∪ mspec(−A). v) χH is even. vi) λ ∈ spec(H) if and only if −λ ∈ spec(H). vii) If λ ∈ spec(H), then amultH (λ) = amultH (−λ). viii) Every imaginary root of χH has even multiplicity. ix) Every imaginary eigenvalue of H has even algebraic multiplicity. Proof. This result follows from Proposition 4.1.1 and Fact 4.9.24. It is helpful to keep in mind that spectral factorizations are not unique. For example, if χH (s) = (s + 1)(s + 2)(−s + 1)(−s + 2), then χH (s) = p(s)p(−s) = pˆ(s)ˆ p(−s), where p(s) = (s + 1)(s + 2) and pˆ(s) = (s + 1)(s − 2). Thus, the spectral factors p(s) and p(−s) can “trade” roots. These roots are the eigenvalues of H. The following result shows that the Hamiltonian H is closely linked to the Riccati equation (12.16.4). Proposition 12.16.14. Let P ∈ Rn×n be symmetric. Then, the following statements are equivalent: i) P is a solution of (12.16.4). ii) P satisfies



P

I

 H

I −P

 = 0.

(12.16.9)

854

CHAPTER 12



iii) P satisfies H iv) P satisfies  H=

I −P

0 I



I −P



A − ΣP 0

 =

I −P

 (A − ΣP ).

Σ −(A − ΣP )T



I P

(12.16.10)

0 I

 .

(12.16.11)

In this case, the following statements hold: v) mspec(H) = mspec(A − ΣP ) ∪ mspec[−(A − ΣP )]. vi) χH (s) = (−1)n χA−ΣP (s)χA−ΣP (−s).  I  is an invariant subspace of H. vii) R −P Corollary 12.16.15. Assume that (12.16.4) has a stabilizing solution. Then, H has no imaginary eigenvalues. For the next two results, P is not necessarily a solution of (12.16.4). Lemma 12.16.16. Assume that λ ∈ spec(A) is an observable eigenvalue of (A, R1), and let P ∈ Rn×n be symmetric. Then, λ ∈ spec(A) is an observable ˜ R). ˜ eigenvalue of (A,   ˜ A Proof. Suppose that rank λI− < n. Then, there exists a nonzero vector ˜ R n ˜ = λv and Rv ˜ = 0. Hence, v ∗R1v = −v ∗PΣP v ≤ 0, which v ∈ C such that Av implies that R1 v = 0 and PΣP v = 0. Hence, ΣP v = 0, and thus Av = λv. Therefore, rank λI−A < n. R1 Lemma 12.16.17. Assume that (A, R1) is (observable, detectable), and let ˜ R) ˜ is (observable, detectable). P ∈ Rn×n be symmetric. Then, (A, Lemma 12.16.18. Assume that (A, E1) is observable, and assume that (12.16.4) has a solution P. Then, the following statements hold: ˜ = ν+(P ). i) ν−(A) ˜ = ν0 (P ) = 0. ii) ν0 (A) ˜ = ν−(P ). iii) ν+(A) ˜ R) ˜ Proof. Since (A, R1) is observable, it follows from Lemma 12.16.17 that (A, is observable. By writing (12.16.4) as the Lyapunov equation (12.16.6), the result now follows from Fact 12.21.1.

LINEAR SYSTEMS AND CONTROL THEORY

855

12.17 The Stabilizing Solution of the Riccati Equation Proposition 12.17.1. The following statements hold: i) (12.16.4) has at most one stabilizing solution. ii) If P is the stabilizing solution of (12.16.4), then P is positive semidefinite. iii) If P is the stabilizing solution of (12.16.4), then ˜ R). ˜ rank P = rank O(A,

(12.17.1)

Proof. To prove i), suppose that (12.16.4) has stabilizing solutions P1 and P2 . Then, ATP1 + P1A + R1 − P1ΣP1 = 0, ATP2 + P2 A + R1 − P2 ΣP2 = 0. Subtracting these equations and rearranging yields (A − ΣP1 )T(P1 − P2 ) + (P1 − P2 )(A − ΣP2 ) = 0. Since A − ΣP1 and A − ΣP2 are asymptotically stable, it follows from Proposition 11.9.3 and Fact 11.18.33 that P1 − P2 = 0. Hence, (12.16.4) has at most one stabilizing solution. Next, to prove ii), suppose that P is a stabilizing solution of (12.16.4). Then, it follows from (12.16.4) that ∞ T

P = et(A−ΣP ) (R1 + PΣP )et(A−ΣP ) dt, 0

which shows that P is positive semidefinite. Finally, iii) follows from Corollary 12.3.3. Theorem 12.17.2. Assume that (12.16.4) has a positive-semidefinite solution P, and assume that (A, E1) is detectable. Then, P is the stabilizing solution of (12.16.4), and thus P is the only positive-semidefinite solution of (12.16.4). If, in addition, (A, E1) is observable, then P is positive definite. ˜ R) ˜ Proof. Since (A, R1) is detectable, it follows from Lemma 12.16.17 that (A, is detectable. Next, since (12.16.4) has a positive-semidefinite solution P, it follows from Corollary 12.8.6 that A˜ is asymptotically stable. Hence, P is the stabilizing solution of (12.16.4). The last statement follows from Lemma 12.16.18. Corollary 12.17.3. Assume that (A, E1) is detectable. Then, (12.16.4) has at most one positive-semidefinite solution. Lemma 12.17.4. Let λ ∈ C, and assume that λ is either an uncontrollable eigenvalue of (A, B) or an unobservable eigenvalue of (A, E1). Then, λ ∈ spec(H).

856

CHAPTER 12

Proof. Note that

 λI − H =



λI − A

−Σ

−R1

λI + AT

.

If λ is an uncontrollable eigenvalue of (A, B), then the first n rows of λI − H are linearly dependent, and thus λ ∈ spec(H). On the other hand, if λ is an unobservable eigenvalue of (A, E1), then the first n columns of λI − H are linearly dependent, and thus λ ∈ spec(H). The following result is a consequence of Lemma 12.17.4. Proposition 12.17.5. Let S ∈ Rn×n be a nonsingular matrix such that ⎡ ⎡ ⎤ ⎤ A1 B1 0 A13 0 ⎢ A21 A2 A23 A24 ⎥ −1 ⎢ ⎥ ⎥S , B = S⎢ B2 ⎥, A = S⎢ (12.17.2) ⎣ 0 ⎣ 0 ⎦ 0 A3 0 ⎦ 0 0 A43 A4 0

where Then,



E11 0 E13 0 S −1,

B1   A1 0 is controllable and A2 , B2 0

E1 = A1 A21



, [ E11

(12.17.3)  E13 ] is observable.

mspec(A2 ) ∪ mspec(−A2 ) ⊆ mspec(H),

(12.17.4)

mspec(A3 ) ∪ mspec(−A3 ) ⊆ mspec(H), mspec(A4 ) ∪ mspec(−A4 ) ⊆ mspec(H).

(12.17.5) (12.17.6)

A13 A3

Next, we present a partial converse of Lemma 12.17.4. Lemma 12.17.6. Let λ ∈ spec(H), and assume that Re λ = 0. Then, λ is either an uncontrollable eigenvalue of (A, B) or an unobservable eigenvalue of (A, E1). Proof. Suppose that λ = jω is an eigenvalue of H, where ω ∈ R. Then, there exist x, y ∈ Cn such that [ xy ] = 0 and H[ xy ] = jω[ xy ]. Consequently, Ax + Σy = jωx,

R1x − ATy = jωy.

Rewriting these equalities as (A − jωI)x = −Σy, yields

y ∗ (A − jωI)x = −y ∗Σy,

(A − jωI)∗ y = R1x x∗(A − jωI)∗ y = x∗R1x.

Since x∗(A − jωI)∗ y is real, it follows that −y ∗Σy = x∗R1x, and thus y ∗Σy = x∗R1x = 0, which implies that BTy = 0 and E1x = 0. Therefore, (A − jωI)x = 0, and hence



A − jωI E1

 x = 0,

(A − jωI)∗ y = 0, y ∗ A − jωI

B

= 0.

857   <  0, and thus either rank A−jωI Since [ xy ] = 0, it follows that either x = 0 or y = E1

n or rank A − jωI B < n. LINEAR SYSTEMS AND CONTROL THEORY

The following result is a restatement of Lemma 12.17.6. be a nonsingular Proposition 12.17.7. Let S ∈ Rn×n 

B1  matrix such that A1 0 (12.17.2) and (12.17.3) are satisfied, where , B2 is controllable and A A 21 2  A1 A13  E11 E13 ] is observable. Then, , [ 0 A3 mspec(H) ∩ jR ⊆ mspec(A2 ) ∪ mspec(−A2 ) ∪ mspec(A3 ) ∪ mspec(−A3 ) ∪ mspec(A4 ) ∪ mspec(−A4 ).

(12.17.7)

Combining Lemma 12.17.4 and Lemma 12.17.6 yields the following result. Proposition 12.17.8. Let λ ∈ C, assume that Re λ = 0, and let S ∈ Rn×n be a nonsingular matrix such that (12.17.2) and (12.17.3) are satisfied, where (A1, B1, E11 ) is controllable and observable, (A2 , B2 ) is controllable, and (A3 , E13 ) is observable. Then, the following statements are equivalent: i) λ is either an uncontrollable eigenvalue of (A, B) or an unobservable eigenvalue of (A, E1 ). ii) λ ∈ mspec(A2 ) ∪ mspec(A3 ) ∪ mspec(A4 ). iii) λ is an eigenvalue of H. The next result gives necessary and sufficient conditions under which (12.16.4) has a stabilizing solution. This result also provides a constructive characterization of the stabilizing solution. Result ii) of Proposition 12.10.11 shows that the condition in i) that every  eigenvalue of (A, E1) is observable is equivalent to  imaginary the condition that

A E1

B E2

has no imaginary invariant zeros.

Theorem 12.17.9. The following statements are equivalent: i) (A, B) is stabilizable, and every imaginary eigenvalue of (A, E1) is observable. ii) There exists a nonsingular matrix S ∈ Rn×n

such

 that (12.17.2) and A1 0 B1 (12.17.3) are satisfied, where , is controllable, B2 A21 A2  A1 A13  E11 E13 ] is observable, ν0 (A2 ) = 0, and A3 and A4 are asymp, [ 0 A3 totically stable. iii) (12.16.4) has a stabilizing solution. 

In this case, let M=

M1 M21

M12 M2

 ∈ R2n×2n

be a nonsingular matrix such that H = MZM −1, where   Z1 Z12 ∈ R2n×2n Z= 0 Z2

(12.17.8)

(12.17.9)

858

CHAPTER 12

and Z1 ∈ Rn×n is asymptotically stable. Then, M1 is nonsingular, and  P = −M21M1−1

(12.17.10)

is the stabilizing solution of (12.16.4). Proof. The equivalence of i) and ii) is immediate. To prove that i) =⇒ iii), first note that Lemma 12.17.6 implies that H has no imaginary eigenvalues. Hence, since H is Hamiltonian, it follows that there exists a matrix M ∈ R2n×2n of the form (12.17.8) such that M is nonsingular and H = MZM −1, where Z ∈ Rn×n is of the form (12.17.9) and Z1 ∈ Rn×n is asymptotically stable. Next, note that HM = MZ implies that       M1 Z1 M1 = =M Z1. H M21 0 M21 Therefore, 

M1 M21

T   T    M1 M1 M1 = Z1 Jn H Jn M21 M21 M21  

M21 T Z1 = M1T M21 −M1 = LZ1,



T M1. Since Jn H = (Jn H)T, it follows that LZ1 is symmetric, where L = M1TM21 −M21 T T that is, LZ1 = Z1 L . Since, in addition, L is skew symmetric, it follows that 0 = Z1TL + LZ1. Now, since Z1 is asymptotically stable, it follows that L = 0. T T Hence, M1TM21 = M21 M1, which shows that M21 M1 is symmetric.

To show that M1 is nonsingular, note that it follows from the equality    

M1

M1 I 0 H = I 0 Z1 M21 M21 that AM1 + ΣM21 = M1Z1. Now, let x ∈ Rn satisfy M1x = 0. We thus have T (AM1 + ΣM21 )x xTM21ΣM21x = xTM21 T = xTM21 M1Z1x

= xTM1TM21Z1x = 0, which implies that BTM21x = 0. Hence, M1Z1x = (AM1 + ΣM21 )x = 0. Thus, Z1 N(M1 ) ⊆ N(M1 ).

859

LINEAR SYSTEMS AND CONTROL THEORY

Now, suppose that M1 is singular. Since Z1 N(M1 ) ⊆ N(M1 ), it follows that there exist λ ∈ spec(Z1 ) and x ∈ Cn such that Z1x = λx and M1x = 0. Forming    

M1

M1 0 I H x= 0 I Z1x M21 M21   yields −ATM21 x = M21λZ, and thus λI + AT M21x = 0. Since, in addition, as

T shown above, BTM21x = 0, it follows that x∗M21 = 0. Since −λI − A B λ ∈ spec(Z1 ), it follows that Re(−λ) > 0. Furthermore, since, by assumption,

(A, B) is stabilizable, it follows that rank λI − A B = n. Therefore, M21x = 0. M1 Combining this fact with M1x = 0 yields M x = 0. Since x is nonzero, it follows 21 that M is singular, which is a contradiction. Consequently, M1 is nonsingular.  Next, define P = −M21M1−1 and note that, since M1TM21 is symmetric, it follows −T that P = −M1 (M1TM21 )M1−1 is also symmetric. M1 M1 = M21 Z1, it follows that Since H M 21     I I = M1Z1M1−1, H M21M1−1 M21M1−1  H

and thus

Multiplying on the left by 0=



P





I −P P



I H

 =

I

I −P

 M1Z1M1−1.

yields  I = ATP + PA + R1 − PΣP, −P

which shows that P is a solution of (12.16.4). Similarly, multiplying on the left by

I 0 yields A − ΣP = M1Z1M1−1. Since Z1 is asymptotically stable, it follows that A − ΣP is also asymptotically stable. To prove that iii) =⇒ i), note that the existence of a stabilizing solution P implies that (A, B) is stabilizable, and that (12.16.11) implies that H has no imaginary eigenvalues. Corollary 12.17.10. Assume that (A, B) is stabilizable and (A, E1) is detectable. Then, (12.16.4) has a stabilizing solution.

12.18 The Maximal Solution of the Riccati Equation In this section we consider the existence of the maximal solution of (12.16.4). Example 12.16.3 shows that the assumptions of Proposition 12.19.1 are not sufficient to guarantee that (12.16.4) has a maximal solution. Theorem 12.18.1. The following statements are equivalent: i) (A, B) is stabilizable. ii) (12.16.4) has a solution Pmax that is positive semidefinite, maximal, and satisfies spec(A − ΣPmax ) ⊂ CLHP.

860

CHAPTER 12

Proof. The result i) =⇒ ii) is given by Theorem 2.1 and Theorem 2.2 of [575]. See also (i) of Theorem 13.11 of [1535]. The converse result follows from Corollary 3 of [1196]. Proposition 12.18.2. Assume that (12.16.4) has a maximal solution Pmax , let P be a solution of (12.16.4), and assume that spec(A − ΣPmax ) ⊂ CLHP and spec(A − ΣP ) ⊂ CLHP. Then, P = Pmax . Proof. It follows from i) of Proposition 12.16.14 that spec(A−ΣP ) = spec(A− ΣPmax ). Since Pmax is the maximal solution of (12.16.4), it follows that P ≤ Pmax . Consequently, it follows from the contrapositive form of the second statement of Theorem 8.4.9 that P = Pmax . Proposition 12.18.3. Assume that (12.16.4) has a solution P such that spec(A − ΣP ) ⊂ CLHP. Then, P is stabilizing if and only if H has no imaginary eigenvalues. It follows from Proposition 12.18.2 that (12.16.4) has at most one positivesemidefinite solution P such that spec(A − ΣP ) ⊂ CLHP. Consequently, (12.16.4) has at most one positive-semidefinite stabilizing solution. Theorem 12.18.4. The following statements hold: i) (12.16.4) has at most one stabilizing solution. ii) If P is the stabilizing solution of (12.16.4), then P is positive semidefinite. iii) If P is the stabilizing solution of (12.16.4), then P is maximal. Proof. To prove i), assume that (12.16.4) has stabilizing solutions P1 and P2 . Then, (A, B) is stabilizable, and Theorem 12.18.1 implies that (12.16.4) has a maximal solution Pmax such that spec(A − ΣPmax ) ⊂ CLHP. Now, Proposition 12.18.2 implies that P1 = Pmax and P2 = Pmax . Hence, P1 = P2 . Alternatively, suppose that (12.16.4) has the stabilizing solutions P1 and P2 . Then, ATP1 + P1A + R1 − P1ΣP1 = 0, ATP2 + P2 A + R1 − P2 ΣP2 = 0. Subtracting these equations and rearranging yields (A − ΣP1 )T(P1 − P2 ) + (P1 − P2 )(A − ΣP2 ) = 0. Since A − ΣP1 and A − ΣP2 are asymptotically stable, it follows from Proposition 11.9.3 and Fact 11.18.33 that P1 − P2 = 0. Hence, (12.16.4) has at most one stabilizing solution.

861

LINEAR SYSTEMS AND CONTROL THEORY

Next, to prove ii), suppose that P is a stabilizing solution of (12.16.4). Then, it follows from (12.16.4) that ∞ T

P = et(A−ΣP ) (R1 + PΣP )et(A−ΣP ) dt, 0

which shows that P is positive semidefinite. To prove iii), let P be a solution of (12.16.4). Then, it follows that (A − ΣP )T(P − P ) + (P − P )(A − ΣP ) + (P − P )Σ(P − P ) = 0, which implies that P ≤ P. Thus, P is also the maximal solution of (12.16.4). The following result concerns the monotonicity of solutions of the Riccati equation (12.16.4). Proposition 12.18.5. Assume that (A, B) is stabilizable, and let Pmax deˆ 1 ∈ Rn×n be positive note the maximal solution of (12.16.4). Furthermore, let R m×m ˆ ˆ ˆ ∈ Rn×m, semidefinite, let R2 ∈ R be positive definite, let A ∈ Rn×n, let B  ˆ ˆ −1 T ˆ define Σ = BR2 B , assume that     ˆ 1 AˆT R R1 AT , ≤ ˆ A −Σ Aˆ −Σ and let Pˆ be a solution of

Then,

ˆPˆ = 0. ˆ 1 − PˆΣ AˆTPˆ + PˆAˆ + R

(12.18.1)

Pˆ ≤ Pmax .

(12.18.2)

Proof. This result is given by Theorem 1 of [1475]. ˆ 1 ∈ Rn×n be posiCorollary 12.18.6. Assume that (A, B) is stabilizable, let R ˆ 1 ≤ R1, and let Pmax and Pˆmax denote, respectively, tive semidefinite, assume that R the maximal solutions of (12.16.4) and

Then,

ˆ 1 − PΣP = 0. ATP + PA + R

(12.18.3)

Pˆmax ≤ Pmax .

(12.18.4)

Proof. This result follows from Proposition 12.18.5 or Theorem 2.3 of [575].

The following result shows that, if R1 = 0, then the closed-loop eigenvalues of the closed-loop dynamics obtained from the maximal solution consist of the CLHP open-loop eigenvalues and reflections of the ORHP open-loop eigenvalues.

862

CHAPTER 12

Proposition 12.18.7. Assume that (A, B) is stabilizable, assume that R1 = 0, and let P ∈ Rn×n be a positive-semidefinite solution of (12.16.4). Then, P is the maximal solution of (12.16.4) if and only if mspec(A − ΣP ) = [mspec(A) ∩ CLHP] ∪ [mspec(−A) ∩ OLHP].

(12.18.5)

Proof. Sufficiency follows from Proposition 12.18.2. To prove necessity, note that it follows from the definition (12.16.8) of H with R1 = 0 and from iv) of Proposition 12.16.14 that mspec(A) ∪ mspec(−A) = mspec(A − ΣP ) ∪ mspec[−(A − ΣP )]. Now, Theorem 12.18.1 implies that mspec(A − ΣP ) ⊆ CLHP, which implies that (12.18.5) is satisfied. Corollary 12.18.8. Let R1 = 0, and assume that spec(A) ⊂ CLHP. Then, P = 0 is the only positive-semidefinite solution of (12.16.4).

12.19 Positive-Semidefinite and Positive-Definite Solutions of the Riccati Equation The following result gives sufficient conditions under which (12.16.4) has a positive-semidefinite solution. Proposition 12.19.1. Assume that there exists a nonsingular S ∈

matrix 1   Rn×n such that (12.17.2) and (12.17.3) are satisfied, where AA211 A02 , B is B2 

 E11 E13 ] is observable, and A3 is asymptotically stable. controllable, A01 AA13 , [ 3 Then, (12.16.4) has a positive-semidefinite solution. Proof. First, rewrite ⎡ A1 ⎢ 0 A = S⎢ ⎣ A21 0

(12.17.2) and (12.17.3) as ⎡ ⎤ B1 A13 0 0 ⎢ 0 A3 0 0 ⎥ −1 ⎥S , B = S⎢ ⎣ B2 A23 A2 A24 ⎦ A43 0 A4 0

⎤ ⎥ ⎥, ⎦

E1 = E11 E13 0 0 S −1, 

1  

 where AA211 A02 , B is controllable, A01 AA13 , [ E11 E13 ] is observable, and A3 B2  A1 A13 B 3 1 is stabilizable, it follows from Theis asymptotically stable. Since 0 A3 , 0 orem 12.18.1 that there exists a positive-semidefinite matrix Pˆ1 that satisfies    T    ET E  T T E11 E13 B1R−1 0 A1 A13 A1 A13 11 11 2 B1 ˆ ˆ ˆ − P1 Pˆ1 = 0. P1 + P1 0 A3 + T T 0 A3 E13 E11

E13 E13

0

0

 S Tdiag(Pˆ1, 0, 0)S is a positive-semidefinite solution of (12.16.4). Consequently, P =

863

LINEAR SYSTEMS AND CONTROL THEORY

Corollary 12.19.2. Assume that (A, B) is stabilizable. Then, (12.16.4) has a positive-semidefinite solution P. If, in addition, (A, E1) is detectable, then P is the stabilizing solution of (12.16.4), and thus P is the only positive-semidefinite solution of (12.16.4). Finally, if (A, E1) is observable, then P is positive definite. Proof. The first statement is given by Theorem 12.18.1. Next, assume that (A, E1) is detectable. Then, Theorem 12.17.2 implies that P is a stabilizing solution of (12.16.4), which is the only positive-semidefinite solution of (12.16.4). Finally, using Theorem 12.17.2, (A, E1) observable implies that P is positive definite. The next result gives necessary and sufficient conditions under which (12.16.4) has a positive-definite solution. Proposition 12.19.3. The following statements are equivalent: i) (12.16.4) has a positive-definite solution. ii) There exists a nonsingular matrix S ∈ Rn×n

such

 that (12.17.2) and A1 0 B1 (12.17.3) are satisfied, where , is controllable, A21 A2 B2  A1 A13  E E 0 A3 , [ 11 13 ] is observable, A3 is asymptotically stable, −A2 is asymptotically stable, spec(A4 ) ⊂ jR, and A4 is semisimple. In this case, (12.16.4) has exactly one positive-definite solution if and only if A4 is empty, and infinitely many positive-definite solutions if and only if A4 is not empty. Proof. See [1151]. Proposition 12.19.4. Assume that (12.16.4) has a stabilizing solution P, and let S ∈ Rn×n be a nonsingular matrix such that (12.17.2) and (12.17.3) are satisfied, where (A1, B1, E11) is controllable and observable, (A2 , B2 ) is controllable, (A3 , E13 ) is observable, ν0 (A2 ) = 0, and A3 and A4 are asymptotically stable. Then, def P = ν−(A2 ).

(12.19.1)

Hence, P is positive definite if and only if spec(A2 ) ⊂ ORHP.

12.20 Facts on Stability, Observability, and Controllability Fact 12.20.1. Let A ∈ Rn×n, B ∈ Rn×m, and C ∈ Rp×n, and assume that (A, B) is controllable and (A, C) is observable. Then, for all v ∈ Rm, the step response t

CetA dτBv + Dv

y(t) = 0

is bounded on [0, ∞) if and only if A is Lyapunov stable and nonsingular.

864

CHAPTER 12

Fact 12.20.2. Let A ∈ Rn×n and C ∈ Rp×n, assume that (A, C) is detectable, and let x(t) and y(t) satisfy x(t) ˙ = Ax(t) and y(t) = Cx(t) for t ∈ [0, ∞). Then, the following statements hold: i) y is bounded if and only if x is bounded. ii) limt→∞ y(t) exists if and only if limt→∞ x(t) exists. iii) y(t) → 0 as t → ∞ if and only if x(t) → 0 as t → ∞. Fact 12.20.3. Let x(0) = x0 , and let xf − etfAx0 ∈ C(A, B). Then, for all t ∈ [0, tf ], the control u: [0, tf ] → Rm defined by ⎞+ ⎛ t f   T T  u(t) = B Te(tf −t)A ⎝ eτABBTeτA dτ⎠ xf − etfAx0 0

yields x(tf ) = xf . Fact 12.20.4. Let x(0) = x0 , let xf ∈ Rn, and assume that (A, B) is controllable. Then, for all t ∈ [0, tf ], the control u: [0, tf ] → Rm defined by ⎞ ⎛ t −1 f    T (tf −t)AT⎝ τA T τAT u(t) = B e e BB e dτ⎠ xf − etfA x0 0

yields x(tf ) = xf . Fact 12.20.5. Let A ∈ Rn×n, let B ∈ Rn×m, assume that A is skew symmetric, and assume that (A, B) is controllable. Then, for all α > 0, A − αBBT is asymptotically stable. Fact 12.20.6. Let A ∈ Rn×n and B ∈ Rn×m. Then, (A, B) is (controllable, stabilizable) if and only if (A, BBT ) is (controllable, stabilizable). Now, assume in addition that B is positive semidefinite. Then, (A, B) is (controllable, stabilizable) if and only if (A, B 1/2 ) is (controllable, stabilizable). ˆ ˆ ∈ Rn×m , and assume that Fact 12.20.7. Let A ∈ Rn×n, B ∈ Rn×m, and B ˆ ˆ is also (con(A, B) is (controllable, stabilizable) and R(B) ⊆ R(B). Then, (A, B) trollable, stabilizable). ˆ ˆ ∈ Rn×m Fact 12.20.8. Let A ∈ Rn×n, B ∈ Rn×m, and B , and assume that T T ˆ ˆ ˆ is also (con(A, B) is (controllable, stabilizable) and BB ≤ B B . Then, (A, B) trollable, stabilizable).

Proof: Use Lemma 8.6.1 and Fact 12.20.7. ˆ ˆ ˆ ∈ Rn×m Fact 12.20.9. Let A ∈ Rn×n, B ∈ Rn×m, B , and Cˆ ∈ Rm×n , and assume that (A, B) is (controllable, stabilizable). Then,   ˆ C, ˆ [BB T + B ˆB ˆ T ]1/2 A+B

is also (controllable, stabilizable).

865

LINEAR SYSTEMS AND CONTROL THEORY

Proof: See [1490, p. 79]. Fact 12.20.10. Let A ∈ Rn×n and B ∈ Rn×m. Then, the following statements are equivalent: i) (A, B) is controllable. ii) There exists α ∈ R such that (A + αI, B) is controllable. iii) (A + αI, B) is controllable for all α ∈ R. Fact 12.20.11. Let A ∈ Rn×n and B ∈ Rn×m. Then, the following statements are equivalent: i) (A, B) is stabilizable. ii) There exists α ≤ max{0, − spabs(A)} such that (A + αI, B) is stabilizable. iii) (A + αI, B) is stabilizable for all α ≤ max{0, − spabs(A)}. Fact 12.20.12. Let A ∈ Rn×n, assume that A is diagonal, and let B ∈ Rn×1. Then, (A, B) is controllable if and only if the diagonal entries of A are distinct and every entry of B is nonzero. Proof: Note that

⎡

⎢ det K(A, B) = det ⎣  =

n ! i=1

b1

0 ..

0  bi

. bn

⎤⎡

1 a1 ⎥⎢ .. .. ⎦⎣ . . 1 an

··· . · ·. · ···

⎤ an−1 1 .. ⎥ . ⎦ an−1 n

! (ai − aj ). i 0, and define ⎛t ⎞ −1 1

P = ⎝ e−tABBTe−tA dt⎠ . T

0

Then, A − BB P is asymptotically stable. T

Proof: P satisfies

  T (A − BBTP )TP + P (A − BBTP ) + P BBT + et1ABBTet1A P = 0. T

Since (A − BBTP, BBT + et1ABBTet1A ) is observable and P is positive definite, Proposition 11.9.5 implies that A − BBTP is asymptotically stable. Remark: This result is due to Lukes and Kleinman. See [1181, pp. 113, 114]. Fact 12.20.18. Let A ∈ Rn×n and B ∈ Rn×m, assume that A is asymptotically stable, and, for t ≥ 0, consider the linear system x˙ = Ax + Bu. Then, if u is bounded, then x is bounded. Furthermore, if u(t) → 0 as t → ∞, then x(t) → 0 as t → ∞. Proof: See [1243, p. 330]. Remark: These results are consequences of input-to-state stability. Fact 12.20.19. Let A ∈ Rn×n and C ∈ Rp×n, assume that (A, C) is observable, and let k ≥ n. Then,  + 0p×n A= Ok+1 (A, C). Ok (A, C) Remark: This result is due to Palanthandalam-Madapusi.

12.21 Facts on the Lyapunov Equation and Inertia Fact 12.21.1. Let A, P ∈ Fn×n, assume that P is Hermitian, let C ∈ Fl×n, and assume that A∗P + PA + C ∗C = 0. Then, the following statements hold: i) |ν−(A) − ν+(P )| ≤ n − rank O(A, C). ii) |ν+(A) − ν−(P )| ≤ n − rank O(A, C). iii) If ν0 (A) = 0, then |ν−(A) − ν+(P )| + |ν+(A) − ν−(P )| ≤ n − rank O(A, C). If, in addition, (A, C) is observable, then the following statements hold: iv) ν−(A) = ν+(P ). v) ν0 (A) = ν0 (P ) = 0. vi) ν+(A) = ν−(P ). vii) If P is positive definite, then A is asymptotically stable.

LINEAR SYSTEMS AND CONTROL THEORY

867

Proof: See [67, 320, 955, 1471] and [892, p. 448]. Remark: v) does not follow from i)–iii). Remark: For related results, see [1081] and references given in [955]. See also [297, 380]. Fact 12.21.2. Let A, P ∈ Fn×n, assume that P is nonsingular and Hermitian, and assume that A∗P + PA is negative semidefinite. Then, the following statements hold: i) ν−(A) ≤ ν+(P ). ii) ν+(A) ≤ ν−(P ). iii) If P is positive definite, then spec(A) ⊂ CLHP. Proof: See [892, p. 447]. Remark: If P is positive definite, then A is Lyapunov stable, although this result does not follow from i) and ii). Fact 12.21.3. Let A, P ∈ Fn×n, and assume that ν0 (A) = 0, P is Hermitian, and A∗P + PA is negative semidefinite. Then, the following statements hold: i) ν−(P ) ≤ ν+(A). ii) ν+(P ) ≤ ν−(A). iii) If P is nonsingular, then ν−(P ) = ν+(A) and ν+(P ) = ν−(A). iv) If P is positive definite, then A is asymptotically stable. Proof: See [892, p. 447]. Fact 12.21.4. Let A, P ∈ Fn×n, and assume that ν0 (A) = 0, P is nonsingular and Hermitian, and A∗P + PA is negative semidefinite. Then, the following statements hold: i) ν−(A) = ν+(P ). ii) ν+(A) = ν−(P ). Proof: Combine Fact 12.21.2 and Fact 12.21.3. See [892, p. 448]. Remark: This result is due to Carlson and Schneider. Fact 12.21.5. Let A, P ∈ Fn×n, assume that P is Hermitian, and assume that A∗P + PA is negative definite. Then, the following statements hold: i) ν−(A) = ν+(P ). ii) ν0 (A) = 0. iii) ν+(A) = ν−(P ). iv) P is nonsingular. v) If P is positive definite, then A is asymptotically stable.

868

CHAPTER 12

Proof: See [459, pp. 441, 442], [892, p. 445], or [1081]. This result follows from Fact 12.21.1 with positive-definite C = −(A∗P + PA)1/2. Remark: This result is due to Krein, Ostrowski, and Schneider. Remark: These conditions are the classical constraints. An analogous result holds for the discrete-time Lyapunov equation, where the analogous definition of inertia counts the numbers of eigenvalues inside the open unit disk, outside the open unit disk, and on the unit circle. See [287, 401]. Fact 12.21.6. Let A ∈ Fn×n. Then, the following statements are equivalent: i) ν0 (A) = 0. ii) There exists a nonsingular Hermitian matrix P ∈ Fn×n such that A∗P +PA is negative definite. iii) There exists a Hermitian matrix P ∈ Fn×n such that A∗P + PA is negative definite. In this case, the following statements hold for P given by ii) and iii): iv) ν−(A) = ν+(P ). v) ν0 (A) = ν0 (P ) = 0. vi) ν+(A) = ν−(P ). vii) P is nonsingular. viii) If P is positive definite, then A is asymptotically stable. Proof: For the result i) =⇒ ii), see [892, p. 445]. The result iii) =⇒ i) follows from Fact 12.21.5. See [53, 287, 299]. Fact 12.21.7. Let A ∈ Fn×n. Then, the following statements are equivalent: i) A is Lyapunov stable. ii) There exists a positive-definite matrix P ∈ Fn×n such that A∗P + PA is negative semidefinite. Furthermore, the following statements are equivalent: iii) A is asymptotically stable. iv) There exists a positive-definite matrix P ∈ Fn×n such that A∗P + PA is negative definite. v) For every positive-definite matrix R ∈ Fn×n, there exists a positive-definite matrix P ∈ Fn×n such that A∗P + PA is negative definite. Remark: See Proposition 11.9.5 and Proposition 11.9.6. Fact 12.21.8. Let A, P ∈ Fn×n, and assume P is Hermitian. Then, the following statements hold: i) ν+(A∗P + PA) ≤ rank P. ii) ν−(A∗P + PA) ≤ rank P.

869

LINEAR SYSTEMS AND CONTROL THEORY

If, in addition, A is asymptotically stable, then the following statement holds: iii) 1 ≤ ν−(A∗P + PA) ≤ rank P. Proof: See [124, 401]. Fact 12.21.9. Let A, P ∈ Rn×n, assume that ν0 (A) = n, and assume that P is positive semidefinite. Then, exactly one of the following statements holds: i) ATP + PA = 0. ii) ν−(ATP + PA) ≥ 1 and ν+(ATP + PA) ≥ 1. Proof: See [1381]. Fact 12.21.10. Let R ∈ Fn×n, and assume that R is Hermitian and ν+(R) ≥ 1. Then, there exist an asymptotically stable matrix A ∈ Fn×n and a positivedefinite matrix P ∈ Fn×n such that A∗P + PA + R = 0. Proof: See [124]. Fact 12.21.11. Let A ∈ Fn×n, assume that A is cyclic, and let a, b, c, d, e be nonnegative integers such that a + b = c + d + e = n, c ≥ 1, and e ≥ 1. Then, there exists a nonsingular, Hermitian matrix P ∈ Fn×n such that ⎤ ⎡ a In P = ⎣ 0 ⎦ b ⎤ c In(A∗P + PA) = ⎣ d ⎦. e ⎡

and

Proof: See [1230]. Remark: See also [1229]. Fact 12.21.12. Let P, R ∈ Fn×n, and assume that P is positive and R is Hermitian. Then, the following statements are equivalent: i) tr RP −1 > 0. ii) There exists an asymptotically stable matrix A ∈ Fn×n such that A∗P + PA + R = 0. Proof: See [124]. Fact 12.21.13. Let A1 ∈ Rn1 ×n1 , A2 ∈ Rn2 ×n2 , B ∈ Rn1 ×m, and C ∈ Rm×n2 , assume that A1 ⊕ A2 is nonsingular, and assume that rank B = rank C = m. Furthermore, let X ∈ Rn1 ×n2 be the unique solution of A1X + XA2 + BC = 0. Then,

rank X ≤ min{rank K(A1, B), rank O(A2, C)}.

Furthermore, equality holds if m = 1.

870

CHAPTER 12

Proof: See [398]. Remark: Related results are given in [1471, 1477]. Fact 12.21.14. Let A1, A2 ∈ Rn×n, B ∈ Rn, C ∈ R1×n, assume that A1 ⊕ A2 is nonsingular, let X ∈ Rn×n satisfy A1X + XA2 + BC = 0, and assume that (A1, B) is controllable and (A2 , C) is observable. Then, X is nonsingular. Proof: See Fact 12.21.13 and [1477]. Fact 12.21.15. Let A, P, R ∈ Rn×n, and assume that P and R are positive semidefinite, ATP + PA + R = 0, and N[O(A, R)] = N(A). Then, A is semistable. Proof: See [199]. Fact 12.21.16. Let A, V ∈ Rn×n, assume that A is asymptotically stable, assume that V is positive semidefinite, and let Q ∈ Rn×n be the unique, positivedefinite solution to AQ + QAT + V = 0. Furthermore, let C ∈ Rl×n, and assume that CV CT is positive definite. Then, CQCT is positive definite. Fact 12.21.17. Let A, R ∈ Rn×n, assume that A is asymptotically stable, assume that R ∈ Rn×n is positive semidefinite, and let P ∈ Rn×n satisfy ATP + PA + R = 0. Then, for all i, j ∈ {1, . . . , n}, there exist αij ∈ R such that n  P = αij A(i−1)TRAj−1. i,j=1

In particular, for all i, j ∈ {1, . . . , n}, αij = Pˆ(i,j) , where Pˆ ∈ Rn×n satisfies ˆ = E1,1 . ˆ = 0, where Aˆ = C(χA ) and R AˆTPˆ + PˆAˆ + R Proof: See [1235]. Remark: This equality is Smith’s method. See [399, 423, 661, 965] for finite-sum solutions of linear matrix equations. Fact 12.21.18. Let λ1, . . . , λn ∈ C, assume that, for all i ∈ {1, . . . , n}, Re λi <  0, define Λ = diag(λ1, . . . , λn ), let k be a nonnegative integer, and, for all i, j ∈ {1, . . . , n}, define P ∈ Cn×n by 

P =

1 k!

∞

tk eΛt eΛt dt.

0

Then, P is positive definite, P satisfies the Lyapunov equation ΛP + P Λ + I = 0, and, for all i, j ∈ {1, . . . , n},

 P(i,j) =

−1 λi + λj

k+1 .

871

LINEAR SYSTEMS AND CONTROL THEORY

Proof: For all nonzero x ∈ Cn, it follows that ∞

x∗P x =

tk eΛtx22 dt

0

is positive. Hence, P is positive definite. Furthermore, note that ∞

P(i,j) =

tk eλi t eλj t dt =

0

(−1)k+1 k! . (λi + λj )k+1

Remark: See [266] and [730, p. 348]. Remark: See Fact 8.8.16 and Fact 12.21.19. Fact 12.21.19. Let λ1, . . . , λn ∈ C, assume that, for all i ∈ {1, . . . , n}, Re λi <  0, define Λ = diag(λ1, . . . , λn ), let k be a nonnegative integer, let R ∈ Cn×n, assume that R is positive semidefinite, and, for all i, j ∈ {1, . . . , n}, define P ∈ Cn×n by 

P =

1 k!

∞

tk eΛt ReΛt dt.

0

Then, P is positive semidefinite, P satisfies the Lyapunov equation ΛP + P Λ + R = 0, and, for all i, j ∈ {1, . . . , n}, P(i,j) = R(i,j)



−1 λi + λj

k+1 .

If, in addition, I ◦ R is positive definite, then P is positive definite. Proof: Use Fact 8.22.12 and Fact 12.21.18. Remark: See Fact 8.8.16 and Fact 12.21.18. Note that P = Pˆ ◦ R, where Pˆ is the solution to the Lyapunov equation with R = I. Fact 12.21.20. Let A, R ∈ Rn×n, assume that R ∈ Rn×n is positive semidefinite, let q, r ∈ R, where r > 0, and assume that there exists a positive-definite matrix P ∈ Rn×n satisfying [A − (q + r)I]TP + P [A − (q + r)I] + 1r ATPA + R = 0. Then, the spectrum of A is contained in a disk centered at q + j0 with radius r. Remark: The disk is an eigenvalue inclusion region. See [145, 629, 1435] for related results concerning elliptical, parabolic, hyperbolic, sector, and vertical strip regions.

872

CHAPTER 12

12.22 Facts on Realizations and the H2 System Norm Fact 12.22.1. ,Let x: [0, ∞) → Rn and y : [0, ∞) → Rn, assume that ∞ x (t)x(t) dt and 0 y T(t)y(t) dt exist, and let x ˆ : jR → Cn and yˆ: jR → Cn 0 denote the Fourier transforms of x and y, respectively. Then,

,∞

T

∞

∞

xT(t)x(t) dt =

−∞

0

and

∞

x ˆ∗ (jω)ˆ x(jω) dω

xT(t)y(t) dt = Re

∞

x ˆ∗ (jω)ˆ y (jω) dω.

−∞

0

Remark: These equalities are equivalent versions of Parseval’s theorem. The second equality follows from the first equality by replacing x with x + y. min

Fact 12.22.2. Let G ∈ Rl×m prop(s), where G ∼



A C

B D

 , and assume that,

for all i ∈ {1, . . . , l} and j ∈ {1, . . . , m}, G(i,j) = pi,j /qi,j , where pi,j , qi,j ∈ R[s] are coprime. Then, l,m 9 spec(A) = roots(pi,j ). i,j=1

Fact 12.22.3. Let G ∼



 G(as + b). Then, H(s) =

 H∼ 

B D



, let a, b ∈ R, where a = 0, and define

a−1(A − bI) B a−1C D

 .

  , where A is nonsingular, and define H(s) =   −A−1B A−1 . H∼ CA−1 D − CA−1B

Fact 12.22.4. Let G ∼ G(1/s). Then,

A C

A C

B D

Fact 12.22.5. Let G(s) = C(sI − A)−1B. Then, −1 −1   G(jω) = −CA ω 2I + A2 B − jωC ω 2I + A2 B. Fact 12.22.6. Let G ∼



A C

B 0

H∼ Consequently,



 and H(s) = sG(s). Then,  B A . CA CB

sC(sI − A)−1B = CA(sI − A)−1B + CB.

873

LINEAR SYSTEMS AND CONTROL THEORY



Fact 12.22.7. Let G = 1, 2. Then, 

G11 G21

G12 G22



G11 G21

⎡

A11 0 0 0 C11 0

⎢ ⎢ ⎢ ∼⎢ ⎢ ⎢ ⎣

Fact 12.22.8. Let G ∼ Then,



G12 G22

A C

0 A12 0 0 C12 0



[I + GM ]

∼ 

and [I + GM ]−1G ∼ Fact 12.22.9. Let G ∼



A C

DL ∈ Rm×l, then

B D

 GL ∼

0 0 A21 0 0 C21

0 0 0 A22 0 C22

Aij Cij

B11 0 B21 0 D11 D21

Bij Dij

0 B12 0 B22 D12 D22

 for all i, j = ⎤ ⎥ ⎥ ⎥ ⎥. ⎥ ⎥ ⎦

 , where G ∈ Rl×m(s), and let M ∈ Rm×l .

B 0

−1

  , where Gij ∼

A − BMC −C

B I

A − BMC C

B 0



 .

 , where G ∈ Rl×m(s). If D has a left inverse

A − BDL C − DL C

BDL DL



satisfies GL G = I. If D has a right inverse DR ∈ Rm×l, then   A − BDR C BDR R G ∼ − DR C DR satisfies GGR = I.   A B Fact 12.22.10. Let G ∼ C 0 be a SISO rational transfer function, and let λ ∈ C. Then, there exists a rational function H such that G(s) =

1 H(s) (s + λ)r

and such that λ is neither a pole nor a zero of H if and only if the Jordan form of A has exactly one block associated with λ, which is of order r. 

 . Then, G(s) is given by .  . . A − sI B . G(s) = (A − sI). C D .

Fact 12.22.11. Let G ∼

A C

B D

Remark: See [155]. Remark: The vertical bar denotes the Schur complement.

874

CHAPTER 12 min

Fact 12.22.12. Let G ∈ Fn×m(s), where G ∼



A C

B D

 , and, for all i ∈

{1, . . . , n} and j ∈ {1, . . . , m}, let G(i,j) = pij /qij , where pij , qij ∈ F[s] are coprime. Then, n,m 9 roots(qij ) = spec(A). i,j=1

Fact 12.22.13. Let A ∈ Rn×n, B ∈ Rn×m, and C ∈ Rm×n. Then,

det[sI − (A + BC)] = det I − C(sI − A)−1B det(sI − A). If, in addition, n = m = 1, then det[sI − (A + BC)] = det(sI − A) − C(sI − A)AB. Remark: The last expression is used in [1034] to compute the frequency response of a transfer function. Proof: Note that  

sI − A B det I − C(sI − A)−1B det(sI − A) = det C I    I 0 sI − A B = det −C I C I   sI − A − BC B = det 0 I = det(sI − A − BC). Fact 12.22.14. assume that A + BK  A det C

Let A ∈ Rn×n, B ∈ Rn×m, C ∈ Rm×n, and K ∈ Rm×n, and is nonsingular. Then, 

B = (−1)m det(A + BK)det C(A + BK)−1B . 0

A B ] is nonsingular if and only if C(A + BK)−1B is nonsingular. Hence, [ C 0

Proof: Note that det



A C

B 0



 = det

A C

B 0



I K

0 I



 B = det 0

= det(A + BK)det −C(A + BK)−1B . 

A + BK C

Fact 12.22.15. Let A1 ∈ Rn×n, C1 ∈ R1×n, A2 ∈ Rm×m, and B2 ∈ Rm×1, let λ ∈ C, assume that λ is an observable eigenvalue of (A1, C1 ) and a controllable eigenvalue of (A2 , B2 ), and define the dynamics matrix A of the cascaded system by   0 A1  A= . B2 C1 A2 Then, amultA(λ) = amultA1 (λ) + amultA2 (λ)

875

LINEAR SYSTEMS AND CONTROL THEORY

and gmultA(λ) = 1. Remark: The eigenvalue λ is a cyclic eigenvalue of both subsystems as well as the cascaded system. In other words, λ, which occurs in a single Jordan block of each subsystem, occurs in a single Jordan block in the cascaded system. Effectively, the Jordan blocks of the subsystems corresponding to λ are merged. Fact 12.22.16. Let G1 ∈ Rl1 ×m (s) and G2 ∈ Rl2 ×m (s) be strictly proper. Then, D D D G1 D2 2 2 D D D G2 D = G1H2 + G2 H2 . H 2

Fact 12.22.17. Let G1, G2 ∈ Rm×m (s) be strictly proper. Then, D D D D G1 D

D D D G1 G2 D . D D G2 D = H2 H 2

 Fact 12.22.18. Let G(s) =

α s+β ,

where β > 0. Then,

|α| GH2 = √ . 2β  Fact 12.22.19. Let G(s) =

α1s+α0 s2 +β1s+β0 ,

?

GH2 = Fact 12.22.20. Let G1 (s) =

α1 s+β1

where β0 , β1 > 0. Then,

α2 α20 + 1. 2β0 β1 2β1

and G2 (s) =

α2 s+β2 ,

where β1, β2 > 0. Then,

G1G2 H2 ≤ G1H2 G2 H2 if and only if β1 + β2 ≥ 2. Remark: The H2 norm is not submultiplicative.

12.23 Facts on the Riccati Equation Fact 12.23.1. Assume that (A, B) is stabilizable, and assume that H defined by (12.16.8) has an imaginary eigenvalue λ. Then, every Jordan block of H associated with λ has even order. Proof: Let P be a solution of (12.16.4), and let J denote the Jordan form of A − ΣP. Then, there exists a nonsingular 2n × 2n block-diagonal matrix S such that   ˆ   J Σ −1 ˆ ˆ is positive semidefinite. Next, let Jλ = H = S HS = 0 −JT , where Σ λIr + Nr ˆ be a Jordan block of J associated with λ, and consider the submatrix of λI − H T consisting of the rows and columns of λI − Jλ and λI + Jλ . Since (A, B) is stabilizable, it follows that the rank of this submatrix is 2r −1. Hence, every Jordan block of H associated with λ has even order.

876

CHAPTER 12

Remark: Canonical forms for symplectic and Hamiltonian matrices are discussed in [898]. Fact 12.23.2. Let A, B ∈ Cn×n, assume that A and B are positive definite, let S ∈ Cn×n satisfy A = S ∗S, and define X = S −1 (SBS ∗ )1/2 S −∗. 

Then, X satisfies XAX = B. Proof: See [701, p. 52]. Fact 12.23.3. Let A, B ∈ Cn×n, and assume that the 2n × 2n matrix   A −2I 2B − 12 A2 A is simple. Then, there exists a matrix X ∈ Cn×n satisfying X 2 + AX + B = 0. Proof: See [1369]. Fact 12.23.4. Let A, B ∈ Fn×n, and assume that A and B are positive semidefinite. Then, the following statements hold: i) If A is positive definite, then X = A#B is the unique positive-definite solution of XA−1X − B = 0. ii) If A is positive definite, then X = positive-definite solution of

1 2 [−A

+ A#(A + 4B)] is the unique

XA−1X + X − B = 0. iii) If A is positive definite, then X = positive-definite solution of

1 2 [A

+ A#(A + 4B)] is the unique

XA−1X − X − B = 0. iv) If B is positive definite, then X = A#B is the unique positive-definite solution of XB −1X = A. v) If A is positive definite, then X = 12 [A + A#(A + 4BA−1B)] is the unique positive-definite solution of BX −1B − X + A = 0. vi) If A is positive definite, then X = 12 [−A + A#(A + 4BA−1B)] is the unique positive-definite solution of BX −1B − X − A = 0. vii) If 0 < A ≤ B, then X = 12 [A+A#(4B −3A)] is the unique positive-definite solution of XA−1X − X − (B − A) = 0.

877

LINEAR SYSTEMS AND CONTROL THEORY

viii) If 0 < A ≤ B, then X = definite solution of

1 2 [−A

+ A#(4B − 3A)] is the unique positive-

XA−1X + X − (B − A) = 0. ix) If 0 < A < B, X(0) is positive definite, and X(t) satisfies X˙ = −XA−1X + X + (B − A), then

lim X(t) = 12 [A + A#(4B − 3A)].

t→∞

x) If 0 < A < B, X(0) is positive definite, and X(t) satisfies X˙ = −XA−1X − X + (B − A), then

lim X(t) = 12 [A + A#(4B − 3A)].

t→∞

xi) If 0 < A < B, X(0) and Y (0) are positive definite, X(t) satisfies X˙ = −XA−1X + X + (B − A) and Y (t) satisfies then

Y˙ = −Y A−1Y − Y + (B − A), lim X(t)#Y (t) = A#(B − A).

t→∞

Proof: See [936]. Remark: A#B is the geometric mean of A and B. See Fact 8.10.43. Remark: The solution X given by vii) is the golden mean of A and B. In the scalar 2 case with √ A = 1 and B = 2, the solution X of X − X − 1 = 0 is the golden ratio 1 5). See Fact 4.11.13. 2 (1 + R

n×n

Fact 12.23.5. Let P0 ∈ Rn×n, assume that P0 is positive definite, let V ∈ be positive semidefinite, and, for all t ≥ 0, let P (t) ∈ Rn×n satisfy P˙ (t) = ATP (t) + P (t)A + P (t)V P (t), P (0) = P0 .

Then, for all t ≥ 0,

⎡

⎤−1

t

P (t) = etA ⎣P0−1 − eτAVeτA dτ ⎦ etA. T

T

0

Remark: P (t) satisfies a Riccati differential equation.

878

CHAPTER 12

Fact 12.23.6. Let Gc ∼



Ac Cc

Bc 0



denote an nth-order dynamic controller ˜ 2 , then Gc is given by for the standard control problem. If Gc minimizes G  Ac = A + BCc − Bc C − Bc DCc ,    Bc = QC T + V12 V2−1,    −R2−1 BTP + RT Cc = 12 ,

where P and Q are positive-semidefinite solutions to the algebraic Riccati equations −1 T ˆ ˆ AˆT RP + PAR − PBR2 B P + R1 = 0, T −1 AˆE Q + QAˆT CQ + Vˆ1 = 0, E − QC V 2

ˆ 1 are defined by where AˆR and R  AˆR = A − BR2−1RT 12 ,

 ˆ1 = R R1 − R12 R2−1RT 12 ,

and AˆE and Vˆ1 are defined by  AˆE = A − V12V2−1C,

 T Vˆ1 = V1 − V12V2−1V12 .

Furthermore, the eigenvalues of the closed-loop system are given by   A BCc = mspec(A + BCc ) ∪ mspec(A − Bc C). mspec Bc C Ac + Bc DCc Fact 12.23.7. Let Gc ∼



Ac Cc

Bc 0



denote an nth-order dynamic controller ˜ 2 , then Gc is for the discrete-time standard control problem. If Gc minimizes G given by 

Ac = A + BCc − Bc C − Bc DCc ,  −1   Bc = AQC T + V12 V2 + CQC T ,  −1 T   R12 + BTPA , Cc = − R2 + BTPB and the eigenvalues of the closed-loop system are given by   A BCc = mspec(A + BCc ) ∪ mspec(A − Bc C). mspec Bc C Ac + Bc DCc  A B  c c Now, assume in addition that D = 0 and Gc ∼ C D . Then, c

c



Ac = A + BCc − Bc C − BDc C,  −1   AQC T + V12 V2 + CQC T + BDc , Bc =     −1  T RT − R2 + BTPB Cc = 12 + B PA − Dc C,

   −1 T  T T T −1 B PAQC T + RT , Dc = R2 + BTPB 12 QC + B P V12 V2 + CQC and the eigenvalues of the closed-loop system are given by   A + BDc C BCc = mspec(A + BCc ) ∪ mspec(A − Bc C). mspec Bc C Ac

879

LINEAR SYSTEMS AND CONTROL THEORY

In both cases, P and Q are positive-semidefinite solutions to the discrete-time algebraic Riccati equations  −1 T T ˆ ˆT ˆ 1, P = AˆT B PAˆR + R RPAR − ARPB R2 + B PB   −1 T T ˆ ˆ CQAˆT Q = AˆE QAˆT E − AE QC V2 + CQC E + V1, ˆ 1 are defined by where AˆR and R  AˆR = A − BR2−1RT 12 ,

 ˆ1 = R R1 − R12 R2−1RT 12 ,

and AˆE and Vˆ1 are defined by  AˆE = A − V12V2−1C,

 T Vˆ1 = V1 − V12V2−1V12 .

Proof: See [633].

12.24 Notes Linear system theory is treated in [265, 1179, 1368, 1485]. Time-varying linear systems are considered in [375, 1179], while discrete-time systems are emphasized in [678]. The PBH test is due to [673]. Spectral factorization results are given in [345]. Stabilization aspects are discussed in [439]. Observable asymptotic stability and controllable asymptotic stability were introduced and used to analyze Lyapunov equations in [1238]. Zeros are treated in [23, 491, 809, 813, 968, 1101, 1184, 1209]. Matrix-based methods for linear system identification are developed in [1397], while stochastic theory is considered in [648]. Solutions of the LQR problem under weak conditions are given in [558]. Solutions of the Riccati equation are considered in [576, 869, 872, 889, 890, 999, 1151, 1468, 1475, 1480]. Proposition 12.16.16 is based on Theorem 3.6 of [1490, p. 79]. A variation of Theorem 12.18.1 is given without proof by Theorem 7.2.1 of [771, p. 125]. There are numerous extensions to the results given in this chapter relating to various generalizations of (12.16.4). These generalizations include the case in which R1 is indefinite [575, 1472, 1474] as well as the case in which Σ is indefinite [1196]. The latter case is relevant to H∞ optimal control theory [192]. Additional extensions include the Riccati inequality ATP + PA + R1 − PΣP ≥ 0 [1143, 1195, 1196, 1197], the discrete-time Riccati equation [9, 679, 764, 889, 1143, 1479], and fixed-order control [759].

Bibliography

[1] A. Abdessemed and E. B. Davies, “Some Commutator Estimates in the Schatten Classes,” J. London Math. Soc., Vol. s2-39, pp. 299–308, 1989. (Cited on page 641.) [2] R. Ablamowicz, “Matrix Exponential via Clifford Algebras,” J. Nonlin. Math. Phys., Vol. 5, pp. 294–313, 1998. (Cited on page 737.) [3] H. Abou-Kandil, G. Freiling, V. Ionescu, and G. Jank, Matrix Riccati Equations in Control and Systems Theory. Basel: Birkhauser, 2003. (Cited on page xvii.) [4] S. Abramovich, J. Baric, M. Matic, and J. Pecaric, “On Van De Lune-Alzer’s Inequality,” J. Math. Ineq., Vol. 1, pp. 563–587, 2007. (Cited on page 32.) [5] L. Aceto and D. Trigiante, “The Matrices of Pascal and Other Greats,” Amer. Math. Monthly, Vol. 108, pp. 232–245, 2001. (Cited on pages 387, 396, 491, and 737.) [6] S. Afriat, “Orthogonal and Oblique Projectors and the Characteristics of Pairs of Vector Spaces,” Proc. Cambridge Phil. Soc., Vol. 53, pp. 800–816, 1957. (Cited on page 228.) [7] R. Agaev and P. Chebotarev, “On the Spectra of Nonsymmetric Laplacian Matrices,” Lin. Alg. Appl., Vol. 399, pp. 157–168, 2005. (Cited on page 777.) [8] R. P. Agarwal, Difference Equations and Inequalities: Theory, Methods, and Applications, 2nd ed. New York: Marcel Dekker, 2000. (Cited on page 793.) [9] C. D. Ahlbrandt and A. C. Peterson, Discrete Hamiltonian Systems: Difference Equations, Continued Fractions, and Riccati Equations. Dordrecht: Kluwer, 1996. (Cited on page 879.) [10] E. Ahn, S. Kim, and Y. Lim, “An Extended Lie-Trotter Formula and Its Application,” Lin. Alg. Appl., Vol. 427, pp. 190–196, 2007. (Cited on page 749.) [11] A. C. Aitken, Determinants and Matrices, 9th ed. 1956. (Cited on page xix.)

Edinburgh: Oliver and Boyd,

[12] M. Aivazis, Group Theory in Physics—Problems and Solutions. Singapore: World Scientific, 1991. (Cited on page 211.) [13] M. Al-Ahmar, “An Identity of Jacobi,” Amer. Math. Monthly, Vol. 103, pp. 78–79, 1996. (Cited on page 207.) [14] A. A. Albert and B. Muckenhoupt, “On Matrices of Trace Zero,” Michigan Math. J., Vol. 4, pp. 1–3, 1957. (Cited on page 341.)

882

BIBLIOGRAPHY

[15] A. E. Albert, “Conditions for Positive and Nonnegative Definiteness in Terms of Pseudoinverses,” SIAM J. Appl. Math., Vol. 17, pp. 434–440, 1969. (Cited on page 595.) [16] A. E. Albert, Regression and the Moore-Penrose Pseudoinverse. New York: Academic Press, 1972. (Cited on pages 415, 418, 421, 427, and 678.) [17] J. M. Aldaz, “A Refinement of the Inequality between Arithmetic and Geometric Means,” J. Math. Ineq., Vol. 2, pp. 473–477, 2007. (Cited on page 62.) [18] R. Aldrovandi, Special Matrices of Mathematical Physics: Stochastic, Circulant and Bell Matrices. Singapore: World Scientific, 2001. (Cited on pages xvii, 299, and 390.) [19] M. Aleksiejczuk and A. Smoktunowicz, “On Properties of Quadratic Matrices,” Mathematica Pannonica, Vol. 112, pp. 239–248, 2000. (Cited on page 429.) [20] A. Y. Alfakih, “On the Nullspace, the Rangespace and the Characteristic Polynomial of Euclidean Distance Matrices,” Lin. Alg. Appl., Vol. 416, pp. 348–354, 2006. (Cited on page 629.) [21] M. Alic, P. S. Bullen, J. E. Pecaric, and V. Volenec, “On the Geometric-Arithmetic Mean Inequality for Matrices,” Mathematical Communications, Vol. 2, pp. 125–128, 1997. (Cited on page 58.) [22] M. Alic, B. Mond, J. E. Pecaric, and V. Volenec, “Bounds for the Differences of Matrix Means,” SIAM J. Matrix Anal. Appl., Vol. 18, pp. 119–123, 1997. (Cited on page 510.) [23] H. Aling and J. M. Schumacher, “A Nine-Fold Decomposition for Linear Systems,” Int. J. Contr., Vol. 39, pp. 779–805, 1984. (Cited on page 879.) [24] G. Alpargu and G. P. H. Styan, “Some Remarks and a Bibliography on the Kantorovich Inequality,” in Multidimensional Statistical Analysis and Theory of Random Matrices, A. K. Gupta and V. L. Girko, Eds. Utrecht: VSP, 1996, pp. 1–13. (Cited on page 551.) [25] R. C. Alperin, “The Matrix of a Rotation,” College Math. J., Vol. 20, p. 230, 1989. (Cited on page 215.) [26] C. Alsina and R. Ger, “On Some Inequalities and Stability Results Related to the Exponential Function,” J. Ineq. Appl., Vol. 2, pp. 373–380, 1998. (Cited on page 37.) [27] C. Alsina and R. B. Nelsen, “On Candido’s Identity,” Math. Mag., Vol. 80, pp. 226–228, 2007. (Cited on page 35.) [28] S. L. Altmann, Rotations, Quaternions, and Double Groups. University Press, 1986. (Cited on pages xvii, 249, and 251.)

New York: Oxford

[29] S. L. Altmann, “Hamilton, Rodrigues, and the Quaternion Scandal,” Math. Mag., Vol. 62, pp. 291–308, 1989. (Cited on pages 214 and 215.) [30] H. Alzer, “A Lower Bound for the Difference between the Arithmetic and Geometric Means,” Nieuw. Arch. Wisk., Vol. 8, pp. 195–197, 1990. (Cited on page 56.) [31] S. Amghibech, “Problem 11296,” Amer. Math. Monthly, Vol. 114, p. 452, 2007. (Cited on page 536.)

BIBLIOGRAPHY

883

[32] B. D. O. Anderson, “Orthogonal Decomposition Defined by a Pair of SkewSymmetric Forms,” Lin. Alg. Appl., Vol. 8, pp. 91–93, 1974. (Cited on pages xvii and 375.) [33] B. D. O. Anderson, “Weighted Hankel-Norm Approximation: Calculation of Bounds,” Sys. Contr. Lett., Vol. 7, pp. 247–255, 1986. (Cited on page 552.) [34] B. D. O. Anderson, E. I. Jury, and M. Mansour, “Schwarz Matrix Properties for Continuous and Discrete Time Systems,” Int. J. Contr., Vol. 23, pp. 1–16, 1976. (Cited on page 772.) [35] B. D. O. Anderson and J. B. Moore, “A Matrix Kronecker Lemma,” Lin. Alg. Appl., Vol. 15, pp. 227–234, 1976. (Cited on page 511.) [36] G. Anderson, M. Vamanamurthy, and M. Vuorinen, “Monotonicity Rules in Calculus,” Amer. Math. Monthly, Vol. 113, pp. 805–816, 2006. (Cited on pages 25, 30, and 32.) [37] T. W. Anderson and I. Olkin, “An Extremal Problem for Positive Definite Matrices,” Lin. Multilin. Alg., Vol. 6, pp. 257–262, 1978. (Cited on page 504.) [38] W. N. Anderson, “Shorted Operators,” SIAM J. Appl. Math., Vol. 20, pp. 520–525, 1971. (Cited on pages 582 and 583.) [39] W. N. Anderson and R. J. Duffin, “Series and Parallel Addition of Matrices,” J. Math. Anal. Appl., Vol. 26, pp. 576–594, 1969. (Cited on page 582.) [40] W. N. Anderson, E. J. Harner, and G. E. Trapp, “Eigenvalues of the Difference and Product of Projections,” Lin. Multilin. Alg., Vol. 17, pp. 295–299, 1985. (Cited on page 366.) [41] W. N. Anderson and M. Schreiber, “On the Infimum of Two Projections,” Acta Sci. Math., Vol. 33, pp. 165–168, 1972. (Cited on pages 421, 422, and 506.) [42] W. N. Anderson and G. E. Trapp, “Shorted Operators II,” SIAM J. Appl. Math., Vol. 28, pp. 60–71, 1975. (Cited on pages 486, 582, 583, and 595.) [43] W. N. Anderson and G. E. Trapp, “Inverse Problems for Means of Matrices,” SIAM J. Alg. Disc. Meth., Vol. 7, pp. 188–192, 1986. (Cited on page 510.) [44] W. N. Anderson and G. E. Trapp, “Symmetric Positive Definite Matrices,” Amer. Math. Monthly, Vol. 95, pp. 261–262, 1988. (Cited on page 522.) [45] T. Ando, “Concavity of Certain Maps on Positive Definite Matrices and Applications to Hadamard Products,” Lin. Alg. Appl., Vol. 26, pp. 203–241, 1979. (Cited on pages 482, 589, 591, 594, 596, and 752.) [46] T. Ando, “Inequalities for M-Matrices,” Lin. Multilin. Alg., Vol. 8, pp. 291–316, 1980. (Cited on page 588.) [47] T. Ando, “On the Arithmetic-Geometric-Harmonic-Mean Inequalities for Positive Definite Matrices,” Lin. Alg. Appl., Vol. 52/53, pp. 31–37, 1983. (Cited on page 509.) [48] T. Ando, “On Some Operator Inequalities,” Math. Ann., Vol. 279, pp. 157–159, 1987. (Cited on pages 509 and 510.) [49] T. Ando, “Majorizations and Inequalities in Matrix Theory,” Lin. Alg. Appl., Vol. 199, pp. 17–67, 1994. (Cited on pages 520, 539, 640, 645, 657, 671, 756, and 761.)

884

BIBLIOGRAPHY

[50] T. Ando, “Majorization Relations for Hadamard Products,” Lin. Alg. Appl., Vol. 223–224, pp. 57–64, 1995. (Cited on page 589.) [51] T. Ando, “Matrix Young Inequalities,” Oper. Theory Adv. Appl., Vol. 75, pp. 33– 38, 1995. (Cited on page 671.) [52] T. Ando, “Problem of Infimum in the Positive Cone,” in Analytic and Geometric Inequalities and Applications, T. M. Rassias and H. M. Srivastava, Eds. Dordrecht: Kluwer, 1999, pp. 1–12. (Cited on pages 506 and 583.) [53] T. Ando, “Lowner Inequality of Indefinite Type,” Lin. Alg. Appl., Vol. 385, pp. 73–80, 2004. (Cited on pages 503 and 868.) [54] T. Ando and R. Bhatia, “Eigenvalue Inequalities Associated with the Cartesian Decomposition,” Lin. Multilin. Alg., Vol. 22, pp. 133–147, 1987. (Cited on page 566.) [55] T. Ando and F. Hiai, “Log-Majorization and Complementary Golden-Thompson Type Inequalities,” Lin. Alg. Appl., Vol. 197/198, pp. 113–131, 1994. (Cited on pages 512, 749, 753, and 761.) [56] T. Ando and F. Hiai, “H¨ older Type Inequalities for Matrices,” Math. Ineq. Appl., Vol. 1, pp. 1–30, 1998. (Cited on pages 485, 506, and 507.) [57] T. Ando, F. Hiai, and K. Okubo, “Trace Inequalities for Multiple Products of Two Matrices,” Math. Ineq. Appl., Vol. 3, pp. 307–318, 2000. (Cited on pages 524 and 526.) [58] T. Ando, R. A. Horn, and C. R. Johnson, “The Singular Values of a Hadamard Product: A Basic Inequality,” Lin. Multilin. Alg., Vol. 21, pp. 345–365, 1987. (Cited on page 674.) [59] T. Ando, C.-K. Li, and R. Mathias, “Geometric Means,” Lin. Alg. Appl., Vol. 385, pp. 305–334, 2004. (Cited on page 510.) [60] T. Ando and X. Zhan, “Norm Inequalities Related to Operator Monotone Functions,” Math. Ann., Vol. 315, pp. 771–780, 1999. (Cited on pages 566, 647, and 763.) [61] T. Andreescu and D. Andrica, Complex Numbers from A to Z. Boston: Birkhauser, 2006. (Cited on pages 76, 77, 168, 169, 171, 172, and 173.) [62] T. Andreescu and Z. Feng, 103 Trigonometry Problems. 2004. (Cited on page 172.)

Boston: Birkhauser,

[63] G. E. Andrews and K. Eriksson, Integer Partitions. Cambridge: Cambridge University Press, 2004. (Cited on page 391.) [64] E. Andruchow, G. Corach, and D. Stojanoff, “Geometric Operator Inequalities,” Lin. Alg. Appl., Vol. 258, pp. 295–310, 1997. (Cited on pages 632 and 646.) [65] E. Angel, Interactive Computer Graphics, 3rd ed. Reading: Addison-Wesley, 2002. (Cited on pages xvii and 215.) [66] Anonymous, “Like the Carlson’s inequality,” http://www.mathlinks.ro/viewtopic. php?t=151210. (Cited on page 46.) [67] A. C. Antoulas and D. C. Sorensen, “Lyapunov, Lanczos, and Inertia,” Lin. Alg. Appl., Vol. 326, pp. 137–150, 2001. (Cited on page 867.) [68] J. D. Aplevich, Implicit Linear Systems. pages xvii and 331.)

Berlin: Springer, 1991.

(Cited on

885

BIBLIOGRAPHY

[69] T. Apostol, Introduction to Analytic Number Theory. (Cited on page 491.)

New York: Springer, 1998.

[70] T. M. Apostol, “Some Explicit Formulas for the Exponential Matrix,” Amer. Math. Monthly, Vol. 76, pp. 289–292, 1969. (Cited on page 737.) [71] T. M. Apostol, Ed., Mathematical Analysis, 2nd ed. Reading: Addison Wesley, 1974. (Cited on pages xxxv, 14, 68, 74, 693, 695, and 696.) [72] H. Araki, “On an Inequality of Lieb and Thirring,” Lett. Math. Phys., Vol. 19, pp. 167–170, 1990. (Cited on pages 528 and 639.) [73] H. Araki and S. Yamagami, “An Inequality for Hilbert-Schmidt Norm,” Comm. Math. Phys., Vol. 81, pp. 89–96, 1981. (Cited on page 640.) [74] J. Araujo and J. D. Mitchell, “An Elementary Proof that Every Singular Matrix Is a Product of Idempotent Matrices,” Amer. Math. Monthly, Vol. 112, pp. 641–645, 2005. (Cited on page 383.) [75] A. Arimoto, “A Simple Proof of the Classification of Normal Toeplitz Matrices,” Elec. J. Lin. Alg., Vol. 9, pp. 108–111, 2002. (Cited on page 390.) [76] T. Arponen, “A Matrix Approach to Polynomials,” Lin. Alg. Appl., Vol. 359, pp. 181–196, 2003. (Cited on page 737.) [77] V. Arsigny, P. Fillard, X. Pennec, and N. Ayache, “Geometric Means in a Novel Vector Space Structure on Symmetric Positive-Definite Matrices,” SIAM J. Matrix Anal. Appl., Vol. 29, pp. 328–347, 2007. (Cited on page 753.) [78] M. Artin, Algebra. and 252.)

Englewood Cliffs: Prentice-Hall, 1991. (Cited on pages 245

[79] M. Artzrouni, “A Theorem on Products of Matrices,” Lin. Alg. Appl., Vol. 49, pp. 153–159, 1983. (Cited on page 786.) [80] A. Arvanitoyeorgos, An Introduction to Lie Groups and the Geometry of Homogeneous Spaces. Providence: American Mathematical Society, 2003. (Cited on page 252.) [81] H. Aslaksen, “SO(2) Invariants of a Set of 2 × 2 Matrices,” Math. Scand., Vol. 65, pp. 59–66, 1989. (Cited on page 283.) [82] H. Aslaksen, “Laws of Trigonometry on SU(3),” Trans. Amer. Math. Soc., Vol. 317, pp. 127–142, 1990. (Cited on pages 165, 198, and 284.) [83] H. Aslaksen, “Quaternionic Determinants,” Mathematical Intelligencer, Vol. 18, no. 3, pp. 57–65, 1996. (Cited on pages 249 and 251.) [84] H. Aslaksen, “Defining Relations of Invariants of Two 3 × 3 Matrices,” J. Algebra, Vol. 298, pp. 41–57, 2006. (Cited on page 283.) [85] B. A. Asner, “On the Total Nonnegativity of the Hurwitz Matrix,” SIAM J. Appl. Math., Vol. 18, pp. 407–414, 1970. (Cited on pages 765 and 770.) [86] Y.-H. Au-Yeung, “A Note on Some Theorems on Simultaneous Diagonalization of Two Hermitian Matrices,” Proc. Cambridge Phil. Soc., Vol. 70, pp. 383–386, 1971. (Cited on pages 554 and 558.) [87] Y.-H. Au-Yeung, “Some Inequalities for the Rational Power of a Nonnegative Definite Matrix,” Lin. Alg. Appl., Vol. 7, pp. 347–350, 1973. (Cited on page 503.)

886

BIBLIOGRAPHY

[88] Y.-H. Au-Yeung, “On the Semi-Definiteness of the Real Pencil of Two Hermitian Matrices,” Lin. Alg. Appl., Vol. 10, pp. 71–76, 1975. (Cited on page 396.) [89] K. M. R. Audenaert, “A Norm Compression Inequality for Block Partitioned Positive Semidefinite Matrices,” Lin. Alg. Appl., Vol. 413, pp. 155–176, 2006. (Cited on page 652.) [90] K. M. R. Audenaert, “On a Norm Compression Inequality for 2 × N Partitioned Block Matrices,” Lin. Alg. Appl., Vol. 428, pp. 781–795, 2008. (Cited on page 653.) [91] K. M. R. Audenaert, “On the Araki-Lieb-Thirring Inequality,” Int. J. Inform. Sys. Sci., Vol. 4, pp. 78–83, 2008. (Cited on page 528.) [92] J. S. Aujla, “Some Norm Inequalities for Completely Monotone Functions,” SIAM J. Matrix Anal. Appl., Vol. 22, pp. 569–573, 2000. (Cited on page 566.) [93] J. S. Aujla and J.-C. Bourin, “Eigenvalue Inequalities for Convex and Log-Convex Functions,” Lin. Alg. Appl., Vol. 424, pp. 25–35, 2007. (Cited on pages 504, 574, 591, 639, and 649.) [94] J. S. Aujla and F. C. Silva, “Weak Majorization Inequalities and Convex Functions,” Lin. Alg. Appl., Vol. 369, pp. 217–233, 2003. (Cited on pages 565 and 566.) [95] J. S. Aujla and H. L. Vasudeva, “Inequalities Involving Hadamard Product and Operator Means,” Math. Japonica, Vol. 42, pp. 265–272, 1995. (Cited on pages 586, 592, and 595.) [96] B. Aupetit and J. Zemanek, “A Characterization of Normal Matrices by Their Exponentials,” Lin. Alg. Appl., Vol. 132, pp. 119–121, 1990. (Cited on page 762.) [97] M. Avriel, Nonlinear Programming: Analysis and Methods. Englewood Cliffs: Prentice-Hall, 1976, reprinted by Dover, Mineola, 2003. (Cited on page 555.) [98] O. Axelsson, Iterative Solution Methods. Cambridge: Cambridge University Press, 1994. (Cited on page xviii.) [99] L. E. Azar, “On Some Extensions of Hardy-Hilbert’s Inequality and Applications,” J. Ineq. Appl., pp. 1–14, 2008, article ID 546829. (Cited on page 69.) [100] J. C. Baez, “Symplectic, Quaternionic, Fermionic,” http://math.ucr.edu/home /baez/symplectic.html. (Cited on pages 247 and 250.) [101] J. C. Baez, “The Octonions,” Bull. Amer. Math. Soc., Vol. 39, pp. 145–205, 2001. (Cited on page 249.) [102] O. Bagdasar, Inequalities and Applications. Cluj-Napoca: Babes-Bolyai University, 2006, Bachelor’s Degree Thesis, http://rgmia.vu.edu.au/monographs/index.html. (Cited on pages 24, 40, 47, and 59.) [103] Z. Bai and G. Golub, “Bounds for the Trace of the Inverse and the Determinant of Symmetric Positive Definite Matrices,” Ann. Numer. Math., Vol. 4, pp. 29–38, 1997. (Cited on page 523.) [104] D. W. Bailey and D. E. Crabtree, “Bounds for Determinants,” Lin. Alg. Appl., Vol. 2, pp. 303–309, 1969. (Cited on page 294.) [105] H. Bailey and Y. A. Rubinstein, “A Variety of Triangle Inequalities,” College Math. J., Vol. 31, pp. 350–355, 2000. (Cited on page 171.)

BIBLIOGRAPHY

887

[106] A. Baker, Matrix Groups: An Introduction to Lie Group Theory. New York: Springer, 2001. (Cited on pages 198, 239, 248, 252, 379, 724, and 748.) [107] J. K. Baksalary and O. M. Baksalary, “Nonsingularity of Linear Combinations of Idempotent Matrices,” Lin. Alg. Appl., Vol. 388, pp. 25–29, 2004. (Cited on pages 219 and 221.) [108] J. K. Baksalary and O. M. Baksalary, “Particular Formulae for the Moore-Penrose Inverse of a Columnwise Partitioned Matrix,” Lin. Alg. Appl., Vol. 421, pp. 16–23, 2007. (Cited on page 429.) [109] J. K. Baksalary, O. M. Baksalary, and X. Liu, “Further Relationships between Certain Partial Orders of Matrices and their Squares,” Lin. Alg. Appl., Vol. 375, pp. 171–180, 2003. (Cited on page 130.) [110] J. K. Baksalary, O. M. Baksalary, and T. Szulc, “Properties of Schur Complements in Partitioned Idempotent Matrices,” Lin. Alg. Appl., Vol. 397, pp. 303–318, 2004. (Cited on page 425.) [111] J. K. Baksalary, O. M. Baksalary, and G. Trenkler, “A Revisitation of Formulae for the Moore-Penrose Inverse of Modified Matrices,” Lin. Alg. Appl., Vol. 372, pp. 207–224, 2003. (Cited on page 412.) [112] J. K. Baksalary, K. Nordstrom, and G. P. H. Styan, “Lowner-Ordering Antitonicity of Generalized Inverses of Hermitian Matrices,” Lin. Alg. Appl., Vol. 127, pp. 171– 182, 1990. (Cited on pages 337, 503, and 580.) [113] J. K. Baksalary and F. Pukelsheim, “On the Lowner, Minus, and Star Partial Orderings of Nonnegative Definite Matrices,” Lin. Alg. Appl., Vol. 151, pp. 135– 141, 1990. (Cited on pages 514, 576, and 579.) [114] J. K. Baksalary, F. Pukelsheim, and G. P. H. Styan, “Some Properties of Matrix Partial Orderings,” Lin. Alg. Appl., Vol. 119, pp. 57–85, 1989. (Cited on pages 130, 579, and 589.) [115] J. K. Baksalary and G. P. H. Styan, “Generalized Inverses of Bordered Matrices,” Lin. Alg. Appl., Vol. 354, pp. 41–47, 2002. (Cited on page 429.) [116] O. M. Baksalary, “An Inequality Involving a Product of Two Orthogonal Projectors,” IMAGE, Vol. 41, p. 44, 2008. (Cited on page 532.) [117] O. M. Baksalary, A. Mickiewicz, and G. Trenkler, “Rank of a Nonnegative Definite Matrix,” IMAGE, Vol. 39, pp. 27–28, 2007. (Cited on page 578.) [118] O. M. Baksalary and G. Trenkler, “Rank of a Generalized Projector,” IMAGE, Vol. 39, pp. 25–27, 2007. (Cited on page 192.) [119] O. M. Baksalary and G. Trenkler, “Characterizations of EP, Normal, and Hermitian Matrices,” Lin. Multilin. Alg., Vol. 56, pp. 299–304, 2008. (Cited on pages 194, 195, 344, 407, 436, 437, and 438.) [120] K. Ball, E. Carlen, and E. Lieb, “Sharp Uniform Convexity and Smoothness Inequalities for Trace Norms,” Invent. Math., Vol. 115, pp. 463–482, 1994. (Cited on pages 625 and 643.) [121] C. S. Ballantine, “Products of Positive Semidefinite Matrices,” Pac. J. Math., Vol. 23, pp. 427–433, 1967. (Cited on pages 382 and 596.)

888

BIBLIOGRAPHY

[122] C. S. Ballantine, “Products of Positive Definite Matrices II,” Pac. J. Math., Vol. 24, pp. 7–17, 1968. (Cited on page 596.) [123] C. S. Ballantine, “Products of Positive Definite Matrices III,” J. Algebra, Vol. 10, pp. 174–182, 1968. (Cited on page 596.) [124] C. S. Ballantine, “A Note on the Matrix Equation H = AP + PA∗ ,” Lin. Alg. Appl., Vol. 2, pp. 37–47, 1969. (Cited on pages 341 and 869.) [125] C. S. Ballantine, “Products of Positive Definite Matrices IV,” Lin. Alg. Appl., Vol. 3, pp. 79–114, 1970. (Cited on page 596.) [126] C. S. Ballantine, “Products of EP Matrices,” Lin. Alg. Appl., Vol. 12, pp. 257–267, 1975. (Cited on page 192.) [127] C. S. Ballantine, “Some Involutory Similarities,” Lin. Multilin. Alg., Vol. 3, pp. 19–23, 1975. (Cited on page 383.) [128] C. S. Ballantine, “Products of Involutory Matrices I,” Lin. Multilin. Alg., Vol. 5, pp. 53–62, 1977. (Cited on page 383.) [129] C. S. Ballantine, “Products of Idempotent Matrices,” Lin. Alg. Appl., Vol. 19, pp. 81–86, 1978. (Cited on page 383.) [130] C. S. Ballantine and C. R. Johnson, “Accretive Matrix Products,” Lin. Multilin. Alg., Vol. 3, pp. 169–185, 1975. (Cited on page 770.) [131] S. Banerjee, “Revisiting Spherical Trigonometry with Orthogonal Projectors,” College Math. J., Vol. 35, pp. 375–381, 2004. (Cited on page 175.) [132] J. Bang-Jensen and G. Gutin, Digraphs: Theory, Algorithms and Applications. New York: Springer, 2001. (Cited on page xvii.) [133] W. Bani-Domi and F. Kittaneh, “Norm Equalities and Inequalities for Operator Matrices,” Lin. Alg. Appl., Vol. 429, pp. 57–67, 2008. (Cited on pages 650 and 651.) [134] R. B. Bapat, “Two Inequalities for the Perron Root,” Lin. Alg. Appl., Vol. 85, pp. 241–248, 1987. (Cited on pages 73 and 305.) [135] R. B. Bapat, “On Generalized Inverses of Banded Matrices,” Elec. J. Lin. Alg., Vol. 16, pp. 284–290, 2007. (Cited on page 237.) [136] R. B. Bapat and M. K. Kwong, “A Generalization of A ◦ A−1 ≥ I,” Lin. Alg. Appl., Vol. 93, pp. 107–112, 1987. (Cited on page 586.) [137] R. B. Bapat and T. E. S. Raghavan, Nonnegative Matrices and Applications. Cambridge: Cambridge University Press, 1997. (Cited on page 299.) [138] R. B. Bapat and B. Zheng, “Generalized Inverses of Bordered Matrices,” Elec. J. Lin. Alg., Vol. 10, pp. 16–30, 2003. (Cited on pages 134 and 429.) [139] I. Y. Bar-Itzhack, D. Hershkowitz, and L. Rodman, “Pointing in Real Euclidean Space,” J. Guid. Contr. Dyn., Vol. 20, pp. 916–922, 1997. (Cited on pages 212 and 738.) [140] E. J. Barbeau, Polynomials. New York: Springer, 1989. (Cited on pages 34, 43, 46, 51, 52, 171, 778, and 779.) [141] E. R. Barnes and A. J. Hoffman, “On Bounds for Eigenvalues of Real Symmetric Matrices,” Lin. Alg. Appl., Vol. 40, pp. 217–223, 1981. (Cited on page 295.)

BIBLIOGRAPHY

889

[142] S. Barnett, “A Note on the Bezoutian Matrix,” SIAM J. Appl. Math., Vol. 22, pp. 84–86, 1972. (Cited on page 279.) [143] S. Barnett, “Congenial Matrices,” Lin. Alg. Appl., Vol. 41, pp. 277–298, 1981. (Cited on pages 387 and 396.) [144] S. Barnett, “Inversion of Partitioned Matrices with Patterned Blocks,” Int. J. Sys. Sci., Vol. 14, pp. 235–237, 1983. (Cited on page 234.) [145] S. Barnett, Polynomials and Linear Control Systems. New York: Marcel Dekker, 1983. (Cited on pages 396 and 871.) [146] S. Barnett, Matrices in Control Theory, revised ed. (Cited on page xvii.)

Malabar: Krieger, 1984.

[147] S. Barnett, “Leverrier’s Algorithm: A New Proof and Extensions,” SIAM J. Matrix Anal. Appl., Vol. 10, pp. 551–556, 1989. (Cited on page 307.) [148] S. Barnett, Matrices: Methods and Applications. Oxford: Clarendon, 1990. (Cited on pages 158, 386, 396, 411, 430, 499, and 679.) [149] S. Barnett and P. Lancaster, “Some Properties of the Bezoutian for Polynomial Matrices,” Lin. Multilin. Alg., Vol. 9, pp. 99–110, 1980. (Cited on page 279.) [150] S. Barnett and C. Storey, Matrix Methods in Stability Theory. New York: Barnes and Noble, 1970. (Cited on pages xvii, 147, 396, 494, and 771.) [151] W. Barrett, “A Theorem on Inverses of Tridiagonal Matrices,” Lin. Alg. Appl., Vol. 27, pp. 211–217, 1979. (Cited on page 238.) [152] W. Barrett, “Hermitian and Positive Definite Matrices,” in Handbook of Linear Algebra, L. Hogben, Ed. Boca Raton: Chapman & Hall/CRC, 2007, pp. 8–1–8– 12. (Cited on page 592.) [153] W. Barrett, C. R. Johnson, and P. Tarazaga, “The Real Positive Definite Completion Problem for a Simple Cycle,” Lin. Alg. Appl., Vol. 192, pp. 3–31, 1993. (Cited on page 493.) [154] J. Barria and P. R. Halmos, “Vector Bases for Two Commuting Matrices,” Lin. Multilin. Alg., Vol. 27, pp. 147–157, 1990. (Cited on page 348.) [155] H. Bart, I. Gohberg, M. A. Kaashoek, and A. C. M. Ran, “Schur Complements and State Space Realizations,” Lin. Alg. Appl., Vol. 399, pp. 203–224, 2005. (Cited on page 873.) [156] H. Baruh, Analytical Dynamics. Boston: McGraw-Hill, 1999. (Cited on pages 214, 249, and 741.) [157] A. Barvinok, A Course in Convexity. Providence: American Mathematical Society, 2002. (Cited on pages 47, 51, 120, 124, 547, 562, and 696.) [158] S. Barza, J. Pecaric, and L.-E. Persson, “Carlson Type Inequalities,” J. Ineq. Appl., Vol. 2, pp. 121–135, 1998. (Cited on page 64.) [159] F. L. Bauer, J. Stoer, and C. Witzgall, “Absolute and Monotonic Norms,” Numer. Math., Vol. 3, pp. 257–264, 1961. (Cited on page 680.) [160] D. S. Bayard, “An Optimization Result with Application to Optimal Spacecraft Reaction Wheel Orientation Design,” in Proc. Amer. Contr. Conf., Arlington, VA, June 2001, pp. 1473–1478. (Cited on page 364.)

890

BIBLIOGRAPHY

[161] M. S. Bazaraa, H. D. Sherali, and C. M. Shetty, Nonlinear Programming, 2nd ed. New York: Wiley, 1993. (Cited on pages 305, 684, and 686.) [162] N. Bebiano and J. da Providencia, “On the Determinant of Certain Strictly Dissipative Matrices,” Lin. Alg. Appl., Vol. 83, pp. 117–128, 1986. (Cited on page 534.) [163] N. Bebiano, J. da Providencia, and R. Lemos, “Matrix Inequalities in Statistical Mechanics,” Lin. Alg. Appl., Vol. 376, pp. 265–273, 2004. (Cited on pages xvii, 752, and 754.) [164] N. Bebiano, R. Lemos, and J. da Providencia, “Inequalities for Quantum Relative Entropy,” Lin. Alg. Appl., Vol. 401, pp. 159–172, 2005. (Cited on pages xvii and 753.) [165] N. Bebiano and M. E. Miranda, “On a Recent Determinantal Inequality,” Lin. Alg. Appl., Vol. 201, pp. 99–102, 1994. (Cited on page 365.) [166] E. F. Beckenbach and R. Bellman, Inequalities. Berlin: Springer, 1965. (Cited on pages 84 and 598.) [167] R. I. Becker, “Necessary and Sufficient Conditions for the Simultaneous Diagonability of Two Quadratic Forms,” Lin. Alg. Appl., Vol. 30, pp. 129–139, 1980. (Cited on pages 396 and 554.) [168] W. Beckner, “Inequalities in Fourier Analysis,” Ann. Math., Vol. 102, p. 159182, 1975. (Cited on page 643.) [169] L. W. Beineke and R. J. Wilson, Eds., Topics in Algebraic Graph Theory. bridge: Cambridge University Press, 2005. (Cited on page xvii.)

Cam-

[170] T. N. Bekjan, “On Joint Convexity of Trace Functions,” Lin. Alg. Appl., Vol. 390, pp. 321–327, 2004. (Cited on page 549.) [171] P. A. Bekker, “The Positive Semidefiniteness of Partitioned Matrices,” Lin. Alg. Appl., Vol. 111, pp. 261–278, 1988. (Cited on pages 531, 532, and 595.) [172] J. G. Belinfante, B. Kolman, and H. A. Smith, “An Introduction to Lie Groups and Lie Algebras with Applications,” SIAM Rev., Vol. 8, pp. 11–46, 1966. (Cited on page 793.) [173] G. R. Belitskii and Y. I. Lyubich, Matrix Norms and Their Applications. Birkhauser, 1988. (Cited on page 680.)

Basel:

[174] R. Bellman, Introduction to Matrix Analysis, 2nd ed. New York: McGraw-Hill, 1960, reprinted by SIAM, Philadelphia, 1995. (Cited on pages 158, 306, 499, and 748.) [175] R. Bellman, “Some Inequalities for the Square Root of a Positive Definite Matrix,” Lin. Alg. Appl., Vol. 1, pp. 321–324, 1968. (Cited on page 506.) [176] A. Ben-Israel, “A Note on Partitioned Matrices and Equations,” SIAM Rev., Vol. 11, pp. 247–250, 1969. (Cited on page 429.) [177] A. Ben-Israel, “The Moore of the Moore-Penrose Inverse,” Elect. J. Lin. Alg., Vol. 9, pp. 150–157, 2002. (Cited on page 438.) [178] A. Ben-Israel and T. N. E. Greville, Generalized Inverses: Theory and Applications, 2nd ed. New York: Springer, 2003. (Cited on pages 406, 410, 429, 434, 435, 438, and 558.)

BIBLIOGRAPHY

891

[179] B. Ben Taher and M. Rachidi, “Some Explicit Formulas for the Polynomial Decomposition of the Matrix Exponential and Applications,” Lin. Alg. Appl., Vol. 350, pp. 171–184, 2002. (Cited on page 737.) [180] A. Ben-Tal and A. Nemirovski, Lectures on Modern Convex Optimization. Philadelphia: SIAM, 2001. (Cited on pages 119 and 178.) [181] A. T. Benjamin and J. J. Quinn, Proofs That Really Count: The Art of Combinatorial Proof. Washington, DC: Mathematical Association of America, 2003. (Cited on pages 12, 19, and 304.) [182] L. Berg, “Three Results in Connection with Inverse Matrices,” Lin. Alg. Appl., Vol. 84, pp. 63–77, 1986. (Cited on page 141.) [183] C. Berge, The Theory of Graphs. London: Methuen, 1962, reprinted by Dover, Mineola, 2001. (Cited on page xvii.) [184] L. D. Berkovitz, Convexity and Optimization in Rn . (Cited on pages 178 and 697.)

New York: Wiley, 2002.

[185] A. Berman, M. Neumann, and R. J. Stern, Nonnegative Matrices in Dynamic Systems. New York: Wiley, 1989. (Cited on pages 299, 774, and 776.) [186] A. Berman and R. J. Plemmons, Nonnegative Matrices in the Mathematical Sciences. New York: Academic Press, 1979, reprinted by SIAM, Philadelphia, 1994. (Cited on pages 190, 252, 301, 303, 774, 775, and 776.) [187] A. Berman, R. S. Varga, and R. C. Ward, “ALPS: Matrices with Nonpositive OffDiagonal Entries,” Lin. Alg. Appl., Vol. 21, pp. 233–244, 1978. (Cited on page 775.) [188] D. S. Bernstein, “Inequalities for the Trace of Matrix Exponentials,” SIAM J. Matrix Anal. Appl., Vol. 9, pp. 156–158, 1988. (Cited on pages 756 and 759.) [189] D. S. Bernstein, “Some Open Problems in Matrix Theory Arising in Linear Systems and Control,” Lin. Alg. Appl., Vol. 162–164, pp. 409–432, 1992. (Cited on page 769.) [190] D. S. Bernstein and S. P. Bhat, “Lyapunov Stability, Semistability, and Asymptotic Stability of Matrix Second-Order Systems,” ASME Trans. J. Vibr. Acoustics, Vol. 117, pp. 145–153, 1995. (Cited on pages 368, 774, and 793.) [191] D. S. Bernstein and S. P. Bhat, “Nonnegativity, Reducibility, and Semistability of Mass Action Kinetics,” in Proc. Conf. Dec. Contr., Phoenix, AZ, December 1999, pp. 2206–2211. (Cited on page 776.) [192] D. S. Bernstein and W. M. Haddad, “LQG Control with an H∞ Performance Bound: A Riccati Equation Approach,” IEEE Trans. Autom. Contr., Vol. 34, pp. 293–305, 1989. (Cited on page 879.) [193] D. S. Bernstein, W. M. Haddad, D. C. Hyland, and F. Tyan, “Maximum EntropyType Lyapunov Functions for Robust Stability and Performance Analysis,” Sys. Contr. Lett., Vol. 21, pp. 73–87, 1993. (Cited on page 374.) [194] D. S. Bernstein and D. C. Hyland, “Compartmental Modeling and Second-Moment Analysis of State Space Systems,” SIAM J. Matrix Anal. Appl., Vol. 14, pp. 880– 901, 1993. (Cited on pages 252, 774, and 777.) [195] D. S. Bernstein and W. So, “Some Explicit Formulas for the Matrix Exponential,” IEEE Trans. Autom. Contr., Vol. 38, pp. 1228–1232, 1993. (Cited on page 737.)

892

BIBLIOGRAPHY

[196] D. S. Bernstein and C. F. Van Loan, “Rational Matrix Functions and Rank-1 Updates,” SIAM J. Matrix Anal. Appl., Vol. 22, pp. 145–154, 2000. (Cited on page 138.) [197] K. V. Bhagwat and R. Subramanian, “Inequalities between Means of Positive Operators,” Math. Proc. Camb. Phil. Soc., Vol. 83, pp. 393–401, 1978. (Cited on pages 476, 506, and 749.) [198] S. P. Bhat and D. S. Bernstein, “Average-Preserving Symmetries and Energy Equipartition in Linear Hamiltonian Systems,” Math. Contr. Sig. Sys., preprint. (Cited on page 250.) [199] S. P. Bhat and D. S. Bernstein, “Nontangency-Based Lyapunov Tests for Convergence and Stability in Systems Having a Continuum of Equilibria,” SIAM J. Contr. Optim., Vol. 42, pp. 1745–1775, 2003. (Cited on pages 793 and 870.) [200] R. Bhatia, Perturbation Bounds for Matrix Eigenvalues. tific and Technical, 1987. (Cited on page 657.)

Essex: Longman Scien-

[201] R. Bhatia, Matrix Analysis. New York: Springer, 1997. (Cited on pages 176, 177, 201, 355, 356, 476, 479, 482, 484, 485, 486, 526, 528, 560, 561, 565, 573, 595, 596, 615, 634, 636, 639, 641, 645, 646, 647, 657, 658, 663, 752, 761, and 763.) [202] R. Bhatia, “Linear Algebra to Quantum Cohomology: The Story of Alfred Horn’s Inequalities,” Amer. Math. Monthly, Vol. 108, pp. 289–318, 2001. (Cited on pages 562, 565, and 596.) [203] R. Bhatia, “Infinitely Divisible Matrices,” Amer. Math. Monthly, Vol. 113, pp. 221–235, 2006. (Cited on pages 241, 489, 491, and 584.) [204] R. Bhatia, Perturbation Bounds for Matrix Eigenvalues. 2007. (Cited on pages 296, 570, 630, 657, and 658.)

Philadelphia: SIAM,

[205] R. Bhatia, Positive Definite Matrices. Princeton: Princeton University Press, 2007. (Cited on pages 485, 489, 490, 491, 494, 502, 509, 752, and 755.) [206] R. Bhatia and C. Davis, “More Matrix Forms of the Arithmetic-Geometric Mean Inequality,” SIAM J. Matrix Anal. Appl., Vol. 14, pp. 132–136, 1993. (Cited on page 646.) [207] R. Bhatia and C. Davis, “A Cauchy-Schwarz Inequality for Operators with Applications,” Lin. Alg. Appl., Vol. 223/224, pp. 119–129, 1995. (Cited on pages 639 and 646.) [208] R. Bhatia and D. Drissi, “Generalized Lyapunov Equations and Positive Definite Functions,” SIAM J. Matrix Anal. Appl., Vol. 27, pp. 103–114, 2005. (Cited on page 492.) [209] R. Bhatia and L. Elsner, “Higher Order Logarithmic Derivatives of Matrices in the Spectral Norm,” SIAM J. Matrix Anal. Appl., Vol. 25, pp. 662–668, 2003. (Cited on page 758.) [210] R. Bhatia and J. Holbrook, “Frechet Derivatives of the Power Function,” Indiana University Math. J., Vol. 49, pp. 1155–1173, 2000. (Cited on pages 26 and 566.) [211] R. Bhatia and J. Holbrook, “Riemannian Geometry and Matrix Geometric Means,” Lin. Alg. Appl., Vol. 413, pp. 594–618, 2006. (Cited on page 755.)

893

BIBLIOGRAPHY

[212] R. Bhatia and F. Kittaneh, “Norm Inequalities for Partitioned Operators and an Application,” Math. Ann., Vol. 287, pp. 719–726, 1990. (Cited on page 650.) [213] R. Bhatia and F. Kittaneh, “On the Singular Values of a Product of Operators,” SIAM J. Matrix Anal. Appl., Vol. 11, pp. 272–277, 1990. (Cited on pages 504, 646, and 671.) [214] R. Bhatia and F. Kittaneh, “Norm Inequalities for Positive Operators,” Lett. Math. Phys., Vol. 43, pp. 225–231, 1998. (Cited on page 647.) [215] R. Bhatia and F. Kittaneh, “Cartesian Decompositions and Schatten Norms,” Lin. Alg. Appl., Vol. 318, pp. 109–116, 2000. (Cited on pages 643 and 644.) [216] R. Bhatia and F. Kittaneh, “Notes on Matrix Arithmetic-Geometric Mean Inequalities,” Lin. Alg. Appl., Vol. 308, pp. 203–211, 2000. (Cited on pages 638 and 639.) [217] R. Bhatia and F. Kittaneh, “Clarkson Inequalities with Several Operators,” Bull. London Math. Soc., Vol. 36, pp. 820–832, 2004. (Cited on page 643.) [218] R. Bhatia and F. Kittaneh, “Commutators, Pinchings, and Spectral Variation,” Oper. Matrices, Vol. 2, pp. 143–151, 2008. (Cited on pages 641 and 642.) [219] R. Bhatia and F. Kittaneh, “The Matrix Arithmetic-Geometric Mean Inequality,” Lin. Alg. Appl., Vol. 428, pp. 2177–2191, 2008. (Cited on pages 567, 636, and 647.) [220] R. Bhatia and K. R. Parthasarathy, “Positive Definite Functions and Operator Inequalities,” Bull. London Math. Soc., Vol. 32, pp. 214–228, 2000. (Cited on pages 490, 646, 647, and 763.) [221] R. Bhatia and P. Rosenthal, “How and Why to Solve the Operator Equation AX − XB = Y ,” Bull. London Math. Soc., Vol. 29, pp. 1–21, 1997. (Cited on pages 349 and 365.) [222] R. Bhatia and P. Semrl, “Orthogonality of Matrices and Some Distance Problems,” Lin. Alg. Appl., Vol. 287, pp. 77–85, 1999. (Cited on page 626.) [223] R. Bhatia and X. Zhan, “Norm Inequalities for Operators with Positive Real Part,” J. Operator Theory, Vol. 50, pp. 67–76, 2003. (Cited on pages 535 and 644.) [224] R. Bhattacharya and K. Mukherjea, “On Unitary Similarity of Matrices,” Lin. Alg. Appl., Vol. 126, pp. 95–105, 1989. (Cited on page 346.) [225] S. P. Bhattacharyya, H. Chapellat, and L. Keel, Robust Control: The Parametric Approach. Englewood Cliffs: Prentice-Hall, 1995. (Cited on page 764.) [226] M. R. Bicknell, “The Lambda Number of a Matrix: The Sum of Its n2 Cofactors,” Amer. Math. Monthly, Vol. 72, pp. 260–264, 1965. (Cited on page 155.) [227] N. Biggs, Algebraic Graph Theory, 2nd ed. Press, 2000. (Cited on pages xvii and 367.)

Cambridge: Cambridge University

[228] P. Binding, B. Najman, and Q. Ye, “A Variational Principle for Eigenvalues of Pencils of Hermitian Matrices,” Integ. Equ. Oper. Theory, Vol. 35, pp. 398–422, 1999. (Cited on page 396.) [229] K. Binmore, Fun and Games: A Text on Game Theory. Lexington: D. C. Heath and Co., 1992. (Cited on page xvii.) [230] A. Bjorck, Numerical Methods for Least Squares Problems. 1996. (Cited on page 678.)

Philadelphia: SIAM,

894

BIBLIOGRAPHY

[231] S. Blanes and F. Casas, “On the Convergence and Optimization of the BakerCampbell-Hausdorff Formula,” Lin. Alg. Appl., Vol. 378, pp. 135–158, 2004. (Cited on page 749.) [232] S. Blanes, F. Casas, J. A. Oteo, and J. Ros, “Magnus and Fer Expansions for Matrix Differential Equations: The Convergence Problem,” J. Phys. A: Math. Gen., Vol. 31, pp. 259–268, 1998. (Cited on page 743.) [233] E. D. Bloch, Proofs and Fundamentals: A First Course in Abstract Mathematics. Boston: Birkhauser, 2000. (Cited on pages 11 and 84.) [234] V. Blondel and J. N. Tsitsiklis, “When Is a Pair of Matrices Mortal?” Inform. Proc. Lett., Vol. 63, pp. 283–286, 1997. (Cited on page 786.) [235] V. Blondel and J. N. Tsitsiklis, “The Boundedness of All Products of a Pair of Matrices Is Undecidable,” Sys. Contr. Lett., Vol. 41, pp. 135–140, 2000. (Cited on page 786.) [236] L. M. Blumenthal, Theory and Applications of Distance Geometry. ford University Press, 1953. (Cited on page 174.)

Oxford: Ox-

[237] W. Boehm, “An Operator Limit,” SIAM Rev., Vol. 36, p. 659, 1994. (Cited on page 785.) [238] A. W. Bojanczyk and A. Lutoborski, “Computation of the Euler Angles of a Symmetric 3 × 3 Matrix,” SIAM J. Matrix Anal. Appl., Vol. 12, pp. 41–48, 1991. (Cited on page 289.) [239] B. Bollobas, Modern Graph Theory. page xvii.)

New York: Springer, 1998.

(Cited on

[240] J. V. Bondar, “Comments on and Complements to Inequalities: Theory of Majorization and Its Applications,” Lin. Alg. Appl., Vol. 199, pp. 115–129, 1994. (Cited on pages 49 and 66.) [241] A. Borck, Numerical Methods for Least Squares Problems. 1996. (Cited on page 438.)

Philadelphia: SIAM,

[242] J. Borwein, D. Bailey, and R. Girgensohn, Experimentation in Mathematics: Computational Paths to Discovery. Natick: A K Peters, 2004. (Cited on pages 19, 20, 22, 23, 62, 174, and 300.) [243] J. M. Borwein and A. S. Lewis, Convex Analysis and Nonlinear Optimization. New York: Springer, 2000. (Cited on pages 119, 120, 178, 305, 363, 364, 484, 506, 693, and 696.) [244] A. J. Bosch, “The Factorization of a Square Matrix into Two Symmetric Matrices,” Amer. Math. Monthly, Vol. 93, pp. 462–464, 1986. (Cited on pages 382 and 396.) [245] A. J. Bosch, “Note on the Factorization of a Square Matrix into Two Hermitian or Symmetric Matrices,” SIAM Rev., Vol. 29, pp. 463–468, 1987. (Cited on pages 382 and 396.) [246] A. Bottcher and D. Wenzel, “How Big Can the Commutator of Two Matrices Be and How Big Is it Typically?” Lin. Alg. Appl., Vol. 403, pp. 216–228, 2005. (Cited on page 641.) [247] A. Bottcher and D. Wenzel, “The Frobenius Norm and the Commutator,” Lin. Alg. Appl., Vol. 429, pp. 1864–1885, 2008. (Cited on page 641.)

895

BIBLIOGRAPHY

[248] O. Bottema, Z. Djordjovic, R. R. Janic, D. S. Mitrinovic, and P. M. Vasic, Inequalities: Theory of Majorization and Its Applications. Groningen: Wolters-Noordhoff, 1969. (Cited on page 172.) [249] T. L. Boullion and P. L. Odell, Generalized Inverse Matrices. 1971. (Cited on page 438.)

New York: Wiley,

[250] J.-C. Bourin, “Some Inequalities for Norms on Matrices and Operators,” Lin. Alg. Appl., Vol. 292, pp. 139–154, 1999. (Cited on pages 528, 530, 632, and 666.) [251] J.-C. Bourin, “Convexity or Concavity Inequalities for Hermitian Operators,” Math. Ineq. Appl., Vol. 7, pp. 607–620, 2004. (Cited on page 530.) [252] J.-C. Bourin, “Hermitian Operators and Convex Functions,” J. Ineq. Pure Appl. Math., Vol. 6, no. 5, pp. 1–6, 2005, Article 139. (Cited on pages 530 and 531.) [253] J.-C. Bourin, “Reverse Inequality to Araki’s Inequality Comparison of Ap Z p Z p and (AZA)p ,” Math. Ineq. Appl., Vol. 232, pp. 373–378, 2005. (Cited on pages 511 and 512.) [254] J.-C. Bourin, “Matrix Versions of Some Classical Inequalities,” Lin. Alg. Appl., Vol. 407, pp. 890–907, 2006. (Cited on pages 532 and 656.) [255] J.-C. Bourin, “Reverse Rearrangement Inequalities via Matrix Technics,” J. Ineq. Pure Appl. Math., Vol. 7, no. 2, pp. 1–6, 2006, Article 43. (Cited on pages 67 and 504.) [256] J.-C. Bourin and Y. Seo, “Reverse Inequality to Golden-Thompson Type Inequalities: Comparison of eA+B and eA eB ,” Lin. Alg. Appl., Vol. 426, pp. 312–316, 2007. (Cited on page 754.) [257] K. Bourque and S. Ligh, “Matrices Associated with Classes of Arithmetical Functions,” J. Number Theory, Vol. 45, pp. 367–376, 1993. (Cited on page 491.) [258] S. Boyd, “Entropy and Random Feedback,” in Open Problems in Mathematical Systems and Control Theory, V. D. Blondel, E. D. Sontag, M. Vidyasagar, and J. C. Willems, Eds. New York: Springer, 1998, pp. 71–74. (Cited on page 633.) [259] S. Boyd and L. Vandenberghe, Convex Optimization. University Press, 2004. (Cited on pages xvii and 178.)

Cambridge: Cambridge

[260] J. L. Brenner, “A Bound for a Determinant with Dominant Main Diagonal,” Proc. Amer. Math. Soc., Vol. 5, pp. 631–634, 1954. (Cited on page 294.) [261] J. L. Brenner, “Expanded Matrices from Matrices with Complex Elements,” SIAM Rev., Vol. 3, pp. 165–166, 1961. (Cited on pages 166 and 251.) [262] J. L. Brenner and J. S. Lim, “The Matrix Equations A = XY Z and B = ZY X and Related Ones,” Canad. Math. Bull., Vol. 17, pp. 179–183, 1974. (Cited on page 378.) [263] J. W. Brewer, “Kronecker Products and Matrix Calculus in System Theory,” IEEE Trans. Circ. Sys., Vol. CAS-25, pp. 772–781, 1978, correction: CAS-26, p. 360, 1979. (Cited on page 458.) [264] L. Brickman, “On the Field of Values of a Matrix,” Proc. Amer. Math. Soc., Vol. 12, pp. 61–66, 1961. (Cited on page 548.)

896

BIBLIOGRAPHY

[265] R. Brockett, Finite Dimensional Linear Systems. New York: Wiley, 1970. (Cited on page 879.) [266] R. W. Brockett, “Using Feedback to Improve System Identification,” in Control of Uncertain Systems, B. A. Francis, M. C. Smith, and J. C. Willems, Eds. Berlin: Springer, 2006, pp. 45–65. (Cited on pages 562 and 871.) [267] G. Brown, “Convexity and Minkowski’s Inequality,” Amer. Math. Monthly, Vol. 112, pp. 740–742, 2005. (Cited on page 72.) [268] E. T. Browne, Introduction to the Theory of Determinants and Matrices. Chapel Hill: University of North Carolina Press, 1958. (Cited on page 595.) [269] R. Bru, J. J. Climent, and M. Neumann, “On the Index of Block Upper Triangular Matrices,” SIAM J. Matrix Anal. Appl., Vol. 16, pp. 436–447, 1995. (Cited on page 374.) [270] R. A. Brualdi, Combinatorial Matrix Classes. Press, 2006. (Cited on page xvii.)

Cambridge: Cambridge University

[271] R. A. Brualdi, “Combinatorial Matrix Theory,” in Handbook of Linear Algebra, L. Hogben, Ed. Boca Raton: Chapman & Hall/CRC, 2007, pp. 27–1–27–12. (Cited on page 300.) [272] R. A. Brualdi and D. Cvetkovic, A Combinatorial Approach to Matrix Theory and Its Applications. Boca Raton: CRC Press, 2009. (Cited on pages xvii, 178, and 285.) [273] R. A. Brualdi and J. Q. Massey, “Some Applications of Elementary Linear Algebra in Combinatorics,” College Math. J., Vol. 24, pp. 10–19, 1993. (Cited on page 142.) [274] R. A. Brualdi and S. Mellendorf, “Regions in the Complex Plane Containing the Eigenvalues of a Matrix,” Amer. Math. Monthly, Vol. 101, pp. 975–985, 1994. (Cited on pages 293 and 772.) [275] R. A. Brualdi and H. J. Ryser, Combinatorial Matrix Theory. Cambridge: Cambridge University Press, 1991. (Cited on pages xvii, 137, 142, and 550.) [276] R. A. Brualdi and H. Schneider, “Determinantal Identities: Gauss, Schur, Cauchy, Sylvester, Kronecker, Jacobi, Binet, Laplace, Muir, and Cayley,” Lin. Alg. Appl., Vol. 52/53, pp. 769–791, 1983. (Cited on page 140.) [277] D. Buckholtz, “Inverting the Difference of Hilbert Space Projections,” Amer. Math. Monthly, Vol. 104, pp. 60–61, 1997. (Cited on page 228.) [278] D. Buckholtz, “Hilbert Space Idempotents and Involutions,” Proc. Amer. Math. Soc., Vol. 128, pp. 1415–1418, 1999. (Cited on pages 366 and 367.) [279] P. S. Bullen, A Dictionary of Inequalities. Essex: Longman, 1998. (Cited on pages 32, 38, 46, 60, 62, 70, 84, 600, and 626.) [280] P. S. Bullen, Handbook of Means and Their Inequalities. Dordrecht: Kluwer, 2003. (Cited on pages 25, 28, 29, 30, 32, and 84.) [281] P. S. Bullen, D. S. Mitrinovic, and P. M. Vasic, Means and Their Inequalities. Dordrecht: Reidel, 1988. (Cited on pages 41, 56, 84, and 622.) [282] F. Bullo, J. Cortes, and S. Martinez, Distributed Control of Robotic Networks. Princeton: Princeton University Press, 2009. (Cited on pages xvii and 301.)

BIBLIOGRAPHY

897

[283] A. Bultheel and M. Van Barel, Linear Algebra, Rational Approximation and Orthogonal Polynomials. Amsterdam: Elsevier, 1997. (Cited on page 307.) [284] F. Burns, D. Carlson, E. V. Haynsworth, and T. L. Markham, “Generalized Inverse Formulas Using the Schur-Complement,” SIAM J. Appl. Math, Vol. 26, pp. 254– 259, 1974. (Cited on page 429.) [285] P. J. Bushell and G. B. Trustrum, “Trace Inequalities for Positive Definite Matrix Power Products,” Lin. Alg. Appl., Vol. 132, pp. 173–178, 1990. (Cited on page 571.) [286] N. D. Cahill, J. R. D’Errico, D. A. Narayan, and J. Y. Narayan, “Fibonacci Determinants,” College Math. J., Vol. 33, pp. 221–225, 2002. (Cited on page 304.) [287] B. E. Cain, “Inertia Theory,” Lin. Alg. Appl., Vol. 30, pp. 211–240, 1980. (Cited on pages 336 and 868.) [288] D. Callan, “When Is ‘Rank’ Additive?” College Math. J., Vol. 29, pp. 145–147, 1998. (Cited on page 128.) [289] E. Camouzis and G. Ladas, Dynamics of Third-Order Rational Difference Equations with Open Problems and Conjectures. New York: CRC Press, 2007. (Cited on pages 778 and 793.) [290] S. L. Campbell, Singular Systems.

London: Pitman, 1980. (Cited on page 438.)

[291] S. L. Campbell and C. D. Meyer, Generalized Inverses of Linear Transformations. London: Pitman, 1979, reprinted by Dover, Mineola, 1991. (Cited on pages 411, 420, 429, 435, 438, and 793.) [292] S. L. Campbell and N. J. Rose, “Singular Perturbation of Autonomous Linear Systems,” SIAM J. Math. Anal., Vol. 10, pp. 542–551, 1979. (Cited on page 767.) [293] L. Cao and H. M. Schwartz, “A Decomposition Method for Positive Semidefinite Matrices and its Application to Recursive Parameter Estimation,” SIAM J. Matrix Anal. Appl., Vol. 22, pp. 1095–1111, 2001. (Cited on pages 460 and 503.) [294] E. A. Carlen and E. H. Lieb, “A Minkowski Type Trace Inequality and Strong Subadditivity of Quantum Entropy,” Amer. Math. Soc. Transl., Vol. 189, pp. 59– 62, 1999. (Cited on pages 484 and 485.) [295] D. Carlson, “Controllability, Inertia, and Stability for Tridiagonal Matrices,” Lin. Alg. Appl., Vol. 56, pp. 207–220, 1984. (Cited on page 770.) [296] D. Carlson, “What Are Schur Complements Anyway?” Lin. Alg. Appl., Vol. 74, pp. 257–275, 1986. (Cited on page 596.) [297] D. Carlson, “On the Controllability of Matrix Pairs (A, K) with K Positive Semidefinite, II,” SIAM J. Matrix Anal. Appl., Vol. 15, pp. 129–133, 1994. (Cited on page 867.) [298] D. Carlson, E. V. Haynsworth, and T. L. Markham, “A Generalization of the Schur Complement by Means of the Moore-Penrose Inverse,” SIAM J. Appl. Math., Vol. 26, pp. 169–175, 1974. (Cited on pages 424, 485, and 596.) [299] D. Carlson and R. Hill, “Generalized Controllability and Inertia Theory,” Lin. Alg. Appl., Vol. 15, pp. 177–187, 1976. (Cited on page 868.) [300] D. Carlson, C. R. Johnson, D. C. Lay, and A. D. Porter, Eds., Linear Algebra Gems: Assets for Undergraduate Mathematics. Washington, DC: Mathematical Association of America, 2002. (Cited on page 178.)

898

BIBLIOGRAPHY

[301] D. Carlson, C. R. Johnson, D. C. Lay, A. D. Porter, A. E. Watkins, and W. Watkins, Eds., Resources for Teaching Linear Algebra. Washington, DC: Mathematical Association of America, 1997. (Cited on page 178.) [302] P. Cartier, “Mathemagics, A Tribute to L. Euler and R. Feynman,” in Noise, Oscillators and Algebraic Randomness, M. Planat, Ed. New York: Springer, 2000, pp. 6–67. (Cited on page 793.) [303] D. I. Cartwright and M. J. Field, “A Refinement of the Arithmetic Mean-Geometric Mean Inequality,” Proc. Amer. Math. Soc., Vol. 71, pp. 36–38, 1978. (Cited on pages 56 and 793.) [304] N. Castro-Gonzalez and E. Dopazo, “Representations of the Drazin Inverse for a Class of Block Matrices,” Lin. Alg. Appl., Vol. 400, pp. 253–269, 2005. (Cited on page 429.) [305] H. Caswell, Matrix Population Models: Construction, Analysis, and Interpretation, 2nd ed. Sunderland: Sinauer Associates, 2000. (Cited on page xvii.) [306] F. S. Cater, “Products of Central Collineations,” Lin. Alg. Appl., Vol. 19, pp. 251–274, 1978. (Cited on page 380.) [307] Y. Chabrillac and J.-P. Crouzeix, “Definiteness and Semidefiniteness of Quadratic Forms Revisited,” Lin. Alg. Appl., Vol. 63, pp. 283–292, 1984. (Cited on page 554.) [308] N. N. Chan and M. K. Kwong, “Hermitian Matrix Inequalities and a Conjecture,” Amer. Math. Monthly, Vol. 92, pp. 533–541, 1985. (Cited on page 503.) [309] H. Chapellat, M. Mansour, and S. P. Bhattacharyya, “Elementary Proofs of Some Classical Stability Criteria,” IEEE Trans. Educ., Vol. 33, pp. 232–239, 1990. (Cited on page 764.) [310] G. Chartrand, Graphs and Digraphs, 4th ed. Boca Raton: Chapman & Hall, 2004. (Cited on page xvii.) [311] G. Chartrand and L. Lesniak, Graphs and Digraphs, 4th ed. man & Hall/CRC, 2004. (Cited on page xvii.) [312] F. Chatelin, Eigenvalues of Matrices. page xviii.)

Boca Raton: Chap-

New York: Wiley, 1993.

[313] J.-J. Chattot, Computational Aerodynamics and Fluid Dynamics. Springer, 2002. (Cited on page xvii.)

(Cited on Berlin:

[314] N. A. Chaturvedi, N. H. McClamroch, and D. S. Bernstein, “Asymptotic Smooth Stabilization of the Inverted 3D Pendulum,” IEEE Trans. Autom. Contr., 2009. (Cited on page 204.) [315] J.-P. Chehab and M. Raydan, “Geometrical Properties of the Frobenius Condition Number for Positive Definite Matrices,” Lin. Alg. Appl., Vol. 429, pp. 2089–2097, 2008. (Cited on page 523.) [316] V.-S. Chellaboina and W. M. Haddad, “Is the Frobenius Matrix Norm Induced?” IEEE Trans. Autom. Contr., Vol. 40, pp. 2137–2139, 1995. (Cited on page 636.) [317] V.-S. Chellaboina and W. M. Haddad, “Solution to ‘Some Matrix Integral Identities’,” SIAM Rev., Vol. 39, pp. 763–765, 1997. (Cited on page 840.)

BIBLIOGRAPHY

899

[318] V.-S. Chellaboina, W. M. Haddad, D. S. Bernstein, and D. A. Wilson, “Induced Convolution Operator Norms of Linear Dynamical Systems,” Math. Contr. Sig. Sys., Vol. 13, pp. 216–239, 2000. (Cited on page 680.) [319] B. M. Chen, Z. Lin, and Y. Shamash, Linear Systems Theory: A Structural Decomposition Approach. Boston: Birkhauser, 2004. (Cited on pages xvii, 123, 124, 354, 363, and 396.) [320] C. T. Chen, “A Generalization of the Inertia Theorem,” SIAM J. Appl. Math., Vol. 25, pp. 158–161, 1973. (Cited on page 867.) [321] C.-T. Chen, Linear System Theory and Design. New York: Holt, Rinehart, Winston, 1984. (Cited on pages xvii and 772.) [322] H. Chen, “A Unified Elementary Approach to Some Classical Inequalities,” Int. J. Math. Educ. Sci. Tech., Vol. 31, pp. 289–292, 2000. (Cited on pages 56 and 68.) [323] J. Chen, “On Bounds of Matrix Eigenvalues,” Math. Ineq. Appl., Vol. 10, pp. 723– 726, 2007. (Cited on page 654.) [324] J.-Q. Chen, J. Ping, and F. Wang, Group Representation for Physicists, 2nd ed. Singapore: World Scientific, 2002. (Cited on page 244.) [325] L. Chen and C. S. Wong, “Inequalities for Singular Values and Traces,” Lin. Alg. Appl., Vol. 171, pp. 109–120, 1992. (Cited on pages 525 and 671.) [326] S. Chen, “Inequalities for M-Matrices and Inverse M-Matrices,” Lin. Alg. Appl., Vol. 426, pp. 610–618, 2007. (Cited on page 588.) [327] C.-M. Cheng, “Cases of Equality for a Singular Value Inequality for the Hadamard Product,” Lin. Alg. Appl., Vol. 177, pp. 209–231, 1992. (Cited on page 674.) [328] C.-M. Cheng, R. A. Horn, and C.-K. Li, “Inequalities and Equalities for the Cartesian Decomposition of Complex Matrices,” Lin. Alg. Appl., Vol. 341, pp. 219–237, 2002. (Cited on pages 486, 535, and 566.) [329] H.-W. Cheng and S. S.-T. Yau, “More Explicit Formulas for the Matrix Exponential,” Lin. Alg. Appl., Vol. 262, pp. 131–163, 1997. (Cited on page 737.) [330] S. Cheng and Y. Tian, “Moore-Penrose Inverses of Products and Differences of Orthogonal Projectors,” Acta Scientiarum Math., Vol. 69, pp. 533–542, 2003. (Cited on pages 416, 419, and 425.) [331] S. Cheng and Y. Tian, “Two Sets of New Characterizations for Normal and EP Matrices,” Lin. Alg. Appl., Vol. 375, pp. 181–195, 2003. (Cited on pages 192, 194, 406, 407, and 434.) [332] M.-T. Chien and M. Neumann, “Positive Definiteness of Tridiagonal Matrices via the Numerical Range,” Elec. J. Lin. Alg., Vol. 3, pp. 93–102, 1998. (Cited on page 238.) [333] M.-D. Choi, “Tricks or Treats with the Hilbert Matrix,” Amer. Math. Monthly, Vol. 90, pp. 301–312, 1983. (Cited on page 234.) [334] M. D. Choi, T. Y. Lam, and B. Reznick, “Sums of Squares of Real Polynomials,” in K Theory and Algebraic Geometry, B. Jacob and A. Rosenberg, Eds. Providence: American Mathematical Society, 1995, pp. 103–126. (Cited on page 46.)

900

BIBLIOGRAPHY

[335] M.-D. Choi and P. Y. Wu, “Convex Combinations of Projections,” Lin. Alg. Appl., Vol. 136, pp. 25–42, 1990. (Cited on pages 226 and 395.) [336] J. Chollet, “Some Inequalities for Principal Submatrices,” Amer. Math. Monthly, Vol. 104, pp. 609–617, 1997. (Cited on page 522.) [337] A. Choudhry, “Extraction of nth Roots of 2 × 2 Matrices,” Lin. Alg. Appl., Vol. 387, pp. 183–192, 2004. (Cited on page 381.) [338] M. T. Chu, R. E. Funderlic, and G. H. Golub, “A Rank-One Reduction Formula and Its Application to Matrix Factorizations,” SIAM Rev., Vol. 37, pp. 512–530, 1995. (Cited on pages 412 and 425.) [339] M. T. Chu, R. E. Funderlic, and G. H. Golub, “Rank Modifications of Semidefinite Matrices Associated with a Secant Update Formula,” SIAM J. Matrix Anal. Appl., Vol. 20, pp. 428–436, 1998. (Cited on page 503.) [340] X.-G. Chu, C.-E. Zhang, and F. Qi, “Two New Algebraic Inequalities with 2n Variables,” J. Ineq. Pure Appl. Math., Vol. 8, no. 4, pp. 1–6, 2007, Article 102. (Cited on page 19.) [341] J. Chuai and Y. Tian, “Rank Equalities and Inequalities for Kronecker Products of Matrices with Applications,” Appl. Math. Comp., Vol. 150, pp. 129–137, 2004. (Cited on pages 447 and 451.) [342] N. L. C. Chui and J. M. Maciejowski, “Realization of Stable Models with Subspace Methods,” Automatica, Vol. 32, pp. 1587–1595, 1996. (Cited on page 783.) [343] F. R. K. Chung, Spectral Graph Theory. Society, 2000. (Cited on page xvii.)

Providence: American Mathematical

[344] A. Cizmesija and J. Pecaric, “Classical Hardy’s and Carleman’s Inequalities and Mixed Means,” in Survey on Classical Inequalities, T. M. Rassias, Ed. Dordrecht: Kluwer, 2000, pp. 27–65. (Cited on pages 63 and 64.) [345] D. J. Clements, B. D. O. Anderson, A. J. Laub, and J. B. Matson, “Spectral Factorization with Imaginary-Axis Zeros,” Lin. Alg. Appl., Vol. 250, pp. 225–252, 1997. (Cited on page 879.) [346] R. E. Cline, “Representations for the Generalized Inverse of a Partitioned Matrix,” SIAM J. Appl. Math., Vol. 12, pp. 588–600, 1964. (Cited on page 428.) [347] R. E. Cline and R. E. Funderlic, “The Rank of a Difference of Matrices and Associated Generalized Inverses,” Lin. Alg. Appl., Vol. 24, pp. 185–215, 1979. (Cited on pages 128, 129, 131, and 420.) [348] M. J. Cloud and B. C. Drachman, Inequalities with Applications to Engineering. New York: Springer, 1998. (Cited on page 84.) [349] E. S. Coakley, F. M. Dopico, and C. R. Johnson, “Matrices for Orthogonal Groups Admitting Only Determinant One,” Lin. Alg. Appl., Vol. 428, pp. 796–813, 2008. (Cited on page 246.) [350] J. E. Cohen, “Spectral Inequalities for Matrix Exponentials,” Lin. Alg. Appl., Vol. 111, pp. 25–28, 1988. (Cited on page 757.) [351] J. E. Cohen, S. Friedland, T. Kato, and F. P. Kelly, “Eigenvalue Inequalities for Products of Matrix Exponentials,” Lin. Alg. Appl., Vol. 45, pp. 55–95, 1982. (Cited on page 759.)

901

BIBLIOGRAPHY

[352] D. K. Cohoon, “Sufficient Conditions for the Zero Matrix,” Amer. Math. Monthly, Vol. 96, pp. 448–449, 1989. (Cited on page 137.) [353] D. Constales, “A Closed Formula for the Moore-Penrose Generalized Inverse of a Complex Matrix of Given Rank,” Acta Math. Hung., Vol. 80, pp. 83–88, 1998. (Cited on page 405.) [354] J. C. Conway and D. A. Smith, On Quaternions and Octonions: Their Geometry, Arithmetic, and Symmetry. Natick: A. K. Peters, 2003. (Cited on pages 51, 52, 245, and 249.) [355] G. Corach, H. Porta, and L. Recht, “An Operator Inequality,” Lin. Alg. Appl., Vol. 142, pp. 153–158, 1990. (Cited on page 646.) [356] M. J. Corless and A. E. Frazho, Linear Systems and Control: An Operator Perspective. New York: Marcel Dekker, 2003. (Cited on page xvii.) [357] E. B. Corrachano and G. Sobczyk, Eds., Geometric Algebra with Applications in Science and Engineering. Boston: Birkhauser, 2001. (Cited on pages 204 and 249.) [358] P. J. Costa and S. Rabinowitz, “Matrix Differentiation Identities,” SIAM Rev., Vol. 36, pp. 657–659, 1994. (Cited on page 704.) [359] R. W. Cottle, “Quartic Barriers,” Comp. Optim. Appl., Vol. 12, pp. 81–105, 1999. (Cited on pages 546, 556, and 764.) [360] T. M. Cover and J. A. Thomas, “Determinant Inequalities via Information Theory,” SIAM J. Matrix Anal. Appl., Vol. 9, pp. 384–392, 1988. (Cited on pages 485, 536, and 556.) [361] T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. York: Wiley, 2006. (Cited on pages xvii, 485, 536, and 541.)

New

[362] D. E. Crabtree, “The Characteristic Vector of the Adjoint Matrix,” Amer. Math. Monthly, Vol. 75, pp. 1127–1128, 1968. (Cited on page 373.) [363] C. R. Crawford and Y. S. Moon, “Finding a Positive Definite Linear Combination of Two Hermitian Matrices,” Lin. Alg. Appl., Vol. 51, pp. 37–48, 1983. (Cited on page 554.) [364] T. Crilly, “Cayley’s Anticipation of a Generalised Cayley-Hamilton Theorem,” Historia Mathematica, Vol. 5, pp. 211–219, 1978. (Cited on page 284.) [365] M. D. Crossley, Essential Topology. page 212.)

New York: Springer, 2005.

(Cited on

[366] C. G. Cullen, “A Note on Normal Matrices,” Amer. Math. Monthly, Vol. 72, pp. 643–644, 1965. (Cited on page 374.) [367] C. G. Cullen, Matrices and Linear Transformations, 2nd ed. Reading: AddisonWesley, 1972, reprinted by Dover, Mineola, 1990. (Cited on pages 689 and 703.) [368] W. J. Culver, “On the Existence and Uniqueness of the Real Logarithm of a Matrix,” Proc. Amer. Math. Soc., Vol. 17, pp. 1146–1151, 1966. (Cited on page 793.) [369] R. F. Curtain and H. J. Zwart, An Introduction to Infinite-Dimensional Linear Systems Theory. New York: Springer, 1995. (Cited on page 758.) [370] M. L. Curtis, Matrix Groups, 2nd ed. pages 214, 248, and 252.)

New York: Springer, 1984. (Cited on

902

BIBLIOGRAPHY

[371] D. Cvetkovic, P. Rowlinson, and S. Simic, Eigenspaces of Graphs. Cambridge University Press, 1997. (Cited on page xvii.)

Cambridge:

[372] P. J. Daboul and R. Delbourgo, “Matrix Representations of Octonions and Generalizations,” J. Math. Phys., Vol. 40, pp. 4134–4150, 1999. (Cited on page 249.) [373] G. Dahlquist, “On Matrix Majorants and Minorants, with Application to Differential Equations,” Lin. Alg. Appl., Vol. 52/53, pp. 199–216, 1983. (Cited on page 774.) [374] R. D’Andrea, “Extension of Parrott’s Theorem to Nondefinite Scalings,” IEEE Trans. Autom. Contr., Vol. 45, pp. 937–940, 2000. (Cited on page 668.) [375] H. D’Angelo, Linear Time-Varying Systems: Analysis and Synthesis. Allyn and Bacon, 1970. (Cited on page 879.)

Boston:

[376] J. P. D’Angelo, Inequalities from Complex Analysis. Washington, DC: The Mathematical Association of America, 2002. (Cited on pages 77 and 620.) [377] F. M. Dannan, “Matrix and Operator Inequalities,” J. Ineq. Pure. Appl. Math., Vol. 2, no. 3/34, pp. 1–7, 2001. (Cited on pages 528 and 537.) [378] R. Datko and V. Seshadri, “A Characterization and a Canonical Decomposition of Hurwitzian Matrices,” Amer. Math. Monthly, Vol. 77, pp. 732–733, 1970. (Cited on page 773.) [379] B. N. Datta, Numerical Linear Algebra and Applications. Brooks/Cole, 1995. (Cited on page xvii.)

Pacific Grove:

[380] B. N. Datta, “Stability and Inertia,” Lin. Alg. Appl., Vol. 302–303, pp. 563–600, 1999. (Cited on page 867.) [381] B. N. Datta, Numerical Methods for Linear Control Systems. Elsevier Academic Press, 2003. (Cited on page xvii.)

San Diego, CA:

[382] J. Dattorro, Convex Optimization and Euclidean Distance Geometry. Palo Alto: Meboo Publishing, 2005. (Cited on pages xvii, 128, 137, 352, and 705.) [383] I. Daubechies and J. C. Lagarias, “Sets of Matrices all Infinite Products of Which Converge,” Lin. Alg. Appl., Vol. 162, pp. 227–263, 1992. (Cited on page 786.) [384] P. J. Davis, The Mathematics of Matrices, 2nd ed. on page xix.) [385] P. J. Davis, Circulant Matrices, 2nd ed. page 390.)

Boston: Ginn, 1965. (Cited

New York: Chelsea, 1994. (Cited on

[386] R. J. H. Dawlings, “Products of Idempotents in the Semigroup of Singular Endomorphisms of a Finite-Dimensional Vector Space,” Proc. Royal Soc. Edinburgh, Vol. 91A, pp. 123–133, 1981. (Cited on page 383.) [387] J. Day, W. So, and R. C. Thompson, “Some Properties of the Campbell-BakerHausdorff Formula,” Lin. Multilin. Alg., Vol. 29, pp. 207–224, 1991. (Cited on page 720.) [388] J. Day, W. So, and R. C. Thompson, “The Spectrum of a Hermitian Matrix Sum,” Lin. Alg. Appl., Vol. 280, pp. 289–332, 1998. (Cited on page 565.) [389] P. W. Day, “Rearrangement Inequalities,” Canad. J. Math., Vol. 24, pp. 930–943, 1972. (Cited on page 67.)

903

BIBLIOGRAPHY

[390] C. de Boor, “An Empty Exercise,” SIGNUM, Vol. 25, pp. 2–6, 1990. (Cited on page 178.) [391] P. P. N. de Groen, “A Counterexample on Vector Norms and the Subordinate Matrix Norms,” Amer. Math. Monthly, Vol. 97, pp. 406–407, 1990. (Cited on page 636.) [392] F. R. de Hoog, R. P. Speed, and E. R. Williams, “On a Matrix Identity Associated with Generalized Least Squares,” Lin. Alg. Appl., Vol. 127, pp. 449–456, 1990. (Cited on page 413.) [393] W. de Launey and J. Seberry, “The Strong Kronecker Product,” J. Combinatorial Thy., Series A, Vol. 66, pp. 192–213, 1994. (Cited on page 458.) [394] J. de Pillis, “Transformations on Partitioned Matrices,” Duke Math. J., Vol. 36, pp. 511–515, 1969. (Cited on pages 533 and 543.) [395] J. de Pillis, “Inequalities for Partitioned Semidefinite Matrices,” Lin. Alg. Appl., Vol. 4, pp. 79–94, 1971. (Cited on page 543.) [396] J. E. De Pillis, “Linear Operators and their Partitioned Matrices,” Notices Amer. Math. Soc., Vol. 14, p. 636, 1967. (Cited on page 543.) [397] L. G. de Pillis, “Determinants and Polynomial Root Structure,” Int. J. Math. Educ. Sci. Technol., Vol. 36, pp. 469–481, 2005. (Cited on page 277.) [398] E. de Souza and S. P. Bhattacharyya, “Controllability, Observability and the Solution of AX − XB = C,” Lin. Alg. Appl., Vol. 39, pp. 167–188, 1981. (Cited on page 870.) [399] E. de Souza and S. P. Bhattarcharyya, “Controllability, Observability, and the Solution of AX − XB = −C,” Lin. Alg. Appl., Vol. 39, pp. 167–188, 1981. (Cited on page 870.) [400] P. N. De Souza and J.-N. Silva, Berkeley Problems in Mathematics, 3rd ed. York: Springer, 2004. (Cited on pages 122, 127, 163, and 503.)

New

[401] L. M. DeAlba and C. R. Johnson, “Possible Inertia Combinations in the Stein and Lyapunov Equations,” Lin. Alg. Appl., Vol. 222, pp. 227–240, 1995. (Cited on pages 470, 868, and 869.) [402] H. P. Decell, “An Application of the Cayley-Hamilton Theorem to Generalized Matrix Inversion,” SIAM Rev., Vol. 7, pp. 526–528, 1965. (Cited on page 408.) [403] N. Del Buono, L. Lopez, and T. Politi, “Computation of Functions of Hamiltonian and Skew-Symmetric Matrices,” Math. Computers Simulation, Vol. 79, pp. 1284– 1297, 2008. (Cited on page 240.) [404] J. W. Demmel, Applied Numerical Linear Algebra. (Cited on page xviii.)

Philadelphia: SIAM, 1997.

[405] C. Y. Deng, “The Drazin Inverses of Sum and Difference of Idempotents,” Lin. Alg. Appl., Vol. 430, pp. 1282–1291, 2009. (Cited on page 433.) [406] E. D. Denman and A. N. Beavers, “The Matrix Sign Function and Computations in Systems,” Appl. Math. Computation, Vol. 2, pp. 63–94, 1976. (Cited on page 381.) [407] C. R. DePrima and C. R. Johnson, “The Range of A−1A∗ in GL(n, C),” Lin. Alg. Appl., Vol. 9, pp. 209–222, 1974. (Cited on page 377.)

904

BIBLIOGRAPHY

[408] C. A. Desoer and H. Haneda, “The Measure of a Matrix as a Tool to Analyze Computer Algorithms for Circuit Analysis,” IEEE Trans. Circ. Thy., Vol. 19, pp. 480–486, 1972. (Cited on page 758.) [409] D. W. DeTemple, “A Geometric Look at Sequences that Converge to Euler’s Constant,” College Math. J., Vol. 37, pp. 128–131, 2006. (Cited on page 21.) [410] E. Deutsch, “Matricial Norms,” Numer. Math., Vol. 16, pp. 73–84, 1970. (Cited on page 649.) [411] E. Deutsch, “Matricial Norms and the Zeros of Polynomials,” Lin. Alg. Appl., Vol. 3, pp. 483–489, 1970. (Cited on pages 779 and 780.) [412] E. Deutsch and M. Mlynarski, “Matricial Logarithmic Derivatives,” Lin. Alg. Appl., Vol. 19, pp. 17–31, 1978. (Cited on page 758.) [413] P. J. Dhrymes, Mathematics for Econometrics, 3rd ed. New York: Springer, 2000. (Cited on page xvii.) [414] L. Dieci, “Real Hamiltonian Logarithm of a Symplectic Matrix,” Lin. Alg. Appl., Vol. 281, pp. 227–246, 1998. (Cited on pages 724 and 751.) [415] R. Diestel, Graph Theory, 3rd ed.

Berlin: Springer, 2006. (Cited on page xvii.)

[416] L. L. Dines, “On the Mapping of Quadratic Forms,” Bull. Amer. Math. Soc., Vol. 47, pp. 494–498, 1941. (Cited on pages 547 and 548.) [417] J. Ding, “Perturbation of Systems of Linear Algebraic Equations,” Lin. Multilin. Alg., Vol. 47, pp. 119–127, 2000. (Cited on page 677.) [418] J. Ding, “Lower and Upper Bounds for the Perturbation of General Linear Algebraic Equations,” Appl. Math. Lett., Vol. 14, pp. 49–52, 2001. (Cited on page 677.) [419] J. Ding and W. C. Pye, “On the Spectrum and Pseudoinverse of a Special Bordered Matrix,” Lin. Alg. Appl., Vol. 331, pp. 11–20, 2001. (Cited on page 286.) [420] A. Dittmer, “Cross Product Identities in Arbitrary Dimension,” Amer. Math. Monthly, Vol. 101, pp. 887–891, 1994. (Cited on page 204.) [421] G. M. Dixon, Division Algebras: Octonions, Quaternions, Complex Numbers and the Algebraic Design of Physics. Dordrecht: Kluwer, 1994. (Cited on page 249.) [422] J. D. Dixon, “How Good Is Hadamard’s Inequality for Determinants?” Canad. Math. Bull., Vol. 27, pp. 260–264, 1984. (Cited on page 540.) [423] T. E. Djaferis and S. K. Mitter, “Algebraic Methods for the Study of Some Linear Matrix Equations,” Lin. Alg. Appl., Vol. 44, pp. 125–142, 1982. (Cited on page 870.) [424] D. Z. Djokovic, “Product of Two Involutions,” Arch. Math., Vol. 18, pp. 582–584, 1967. (Cited on page 383.) [425] D. Z. Djokovic, “On Some Representations of Matrices,” Lin. Multilin. Alg., Vol. 4, pp. 33–40, 1976. (Cited on pages 163 and 345.) [426] D. Z. Djokovic and O. P. Lossers, “A Determinant Inequality,” Amer. Math. Monthly, Vol. 83, pp. 483–484, 1976. (Cited on page 166.) [427] D. Z. Djokovic and J. Malzan, “Products of Reflections in the Unitary Group,” Proc. Amer. Math. Soc., Vol. 73, pp. 157–160, 1979. (Cited on page 380.)

905

BIBLIOGRAPHY

[428] D. Z. Dokovic, “On the Product of Two Alternating Matrices,” Amer. Math. Monthly, Vol. 98, pp. 935–936, 1991. (Cited on page 375.) [429] D. Z. Dokovic, “Unitary Similarity of Projectors,” Aequationes Mathematicae, Vol. 42, pp. 220–224, 1991. (Cited on pages 342, 343, and 347.) [430] D. Z. Dokovic, F. Szechtman, and K. Zhao, “An Algorithm that Carries a Square Matrix into Its Transpose by an Involutory Congruence Transformation,” Elec. J. Lin. Alg., Vol. 10, pp. 320–340, 2003. (Cited on page 340.) [431] V. Dolotin and A. Morozov, Introduction to Non-Linear Algebra. Singapore: World Scientific, 2007. (Cited on page 458.) [432] W. F. Donoghue, Monotone Matrix Functions and Analytic Continuation. York: Springer, 1974. (Cited on pages 489, 490, 594, and 595.)

New

[433] F. M. Dopico and C. R. Johnson, “Complementary Bases in Symplectic Matrices and a Proof That Their Determinant Is One,” Lin. Alg. Appl., Vol. 419, pp. 772– 778, 2006. (Cited on page 239.) [434] F. M. Dopico, C. R. Johnson, and J. M. Molera, “Multiple LU Factorizations of a Singular Matrix,” Lin. Alg. Appl., Vol. 419, pp. 24–36, 2006. (Cited on page 378.) [435] C. Doran and A. Lasenby, Geometric Algebra for Physicists. bridge University Press, 2005. (Cited on pages 204 and 249.)

Cambridge: Cam-

[436] L. Dorst, D. Fontijne, and S. Mann, Geometric Algebra for Computer Science: An Object-Oriented Approach to Geometry. Amsterdam: Elsevier, 2007. (Cited on pages 249 and 620.) [437] R. G. Douglas, “On Majorization, Factorization, and Range Inclusion of Operators on Hilbert Space,” Proc. Amer. Math. Soc., Vol. 17, pp. 413–415, 1966. (Cited on page 474.) [438] P. G. Doyle and J. L. Snell, Random Walks and Electric Networks. Washington, DC: Mathematical Association of America, 1984. (Cited on page xvii.) [439] V. Dragan and A. Halanay, Stabilization of Linear Systems. 1999. (Cited on page 879.)

Boston: Birkhauser,

[440] S. S. Dragomir, “A Survey on Cauchy-Bunyakovsky-Schwarz Type Discrete Inequalities,” J. Ineq. Pure Appl. Math., Vol. 4, no. 3, pp. 1–142, 2003, Article 3. (Cited on pages 39, 63, 68, 70, 71, 72, 73, and 77.) [441] S. S. Dragomir, Discrete Inequalities of the Cauchy-Bunyakovsky-Schwarz Type. Hauppauge: Nova Science Publishers, 2004. (Cited on pages 39, 63, 68, 70, 71, 72, 73, and 77.) [442] S. S. Dragomir, “Some Reverses of the Generalised Triangle Inequality in Complex Inner Product Spaces,” Lin. Alg. Appl., Vol. 402, pp. 245–254, 2005. (Cited on page 77.) [443] S. S. Dragomir and C. J. Goh, “A Counterpart of Jensen’s Discrete Inequality for Differentiable Convex Mappings and Applications in Information Theory,” Math. Comput. Modelling, Vol. 24, pp. 1–11, 1996. (Cited on page 65.) [444] S. S. Dragomir and C. J. Goh, “Some Bounds on Entropy Measures in Information Theory,” Appl. Math. Lett., Vol. 10, pp. 23–28, 1997. (Cited on page 65.)

906

BIBLIOGRAPHY

[445] S. S. Dragomir, C. E. Pearce, and J. Pecaric, “Some New Inequalities for the Logarithmic Map, with Applications to Entropy and Mutual Information,” Kyungpook Math. J., Vol. 41, pp. 115–125, 2001. (Cited on pages 65 and 70.) [446] M. P. Drazin, “A Note on Skew-Symmetric Matrices,” Math. Gaz., Vol. 36, pp. 253–255, 1952. (Cited on page 282.) [447] D. Drissi, “Sharp Inequalities for Some Operator Means,” SIAM J. Matrix Anal. Appl., Vol. 28, pp. 822–828, 2006. (Cited on page 41.) [448] D. Drivaliaris, S. Karanasios, and D. Pappas, “Factorizations of EP Operators,” Lin. Alg. Appl., Vol. 429, pp. 1555–1567, 2008. (Cited on page 414.) [449] R. Drnovsek, H. Radjavi, and P. Rosenthal, “A Characterization of Commutators of Idempotents,” Lin. Alg. Appl., Vol. 347, pp. 91–99, 2002. (Cited on page 217.) [450] S. W. Drury and G. P. H. Styan, “Normal Matrix and a Commutator,” IMAGE, Vol. 31, p. 24, 2003. (Cited on page 200.) [451] H. Du, C. Deng, and Q. Li, “On the Infimum Problem of Hilbert Space Effects,” Science in China: Series A Math., Vol. 49, pp. 545–556, 2006. (Cited on page 506.) [452] K. Du, “Norm Inequalities Involving Hadamard Products,” Lin. Multilin. Alg., preprint. (Cited on page 676.) [453] F. Dubeau, “Linear Algebra and the Sums of Powers of Integers,” Elec. J. Lin. Alg., Vol. 17, pp. 577–596, 2008. (Cited on page 20.) [454] J. J. Duistermaat and J. A. C. Kolk, Eds., Multidimensional Real Analysis I: Differentiation. Cambridge: Cambridge University Press, 2004. (Cited on pages 696, 700, and 748.) [455] I. Duleba, “On a Computationally Simple Form of the Generalized CampbellBaker-Hausdorff-Dynkin Formula,” Sys. Contr. Lett., Vol. 34, pp. 191–202, 1998. (Cited on page 743.) [456] G. E. Dullerud and F. Paganini, A Course in Robust Control Theory: A Convex Approach, 2nd ed. New York: Springer, 1999. (Cited on page xvii.) [457] D. S. Dummit and R. M. Foote, Abstract Algebra, 3rd ed. New York: Wiley, 2004. (Cited on pages 242, 244, 245, 390, 391, and 396.) [458] C. F. Dunkl and K. S. Williams, “A Simple Inequality,” Amer. Math. Monthly, Vol. 71, pp. 53–54, 1964. (Cited on page 619.) [459] H. Dym, Linear Algebra in Action. Providence: American Mathematical Society, 2006. (Cited on pages 61, 140, 142, 145, 173, 306, 336, 337, 338, 365, 366, 429, 474, 503, 558, 561, 578, 648, 668, 695, 697, 698, 701, 766, and 868.) [460] A. Edelman and G. Strang, “Pascal Matrices,” Amer. Math. Monthly, Vol. 111, pp. 189–197, 2004. (Cited on page 491.) [461] C. H. Edwards, Jr., Advanced Calculus of Several Variables. New York: Academic Press, 1973. (Cited on page 688.) [462] O. Egecioglu, “Parallelogram-Law-Type Identities,” Lin. Alg. Appl., Vol. 225, pp. 1–12, 1995. (Cited on page 77.) [463] H. G. Eggleston, Convexity. Cambridge: Cambridge University Press, 1958. (Cited on page 178.)

BIBLIOGRAPHY

907

[464] L. Elsner, D. Hershkowitz, and H. Schneider, “Bounds on Norms of Compound Matrices and on Products of Eigenvalues,” Bull. London Math. Soc., Vol. 32, pp. 15–24, 2000. (Cited on pages 20 and 454.) [465] L. Elsner and K. D. Ikramov, “Normal Matrices: An Update,” Lin. Alg. Appl., Vol. 285, pp. 291–303, 1998. (Cited on pages 194, 714, and 762.) [466] L. Elsner, C. R. Johnson, and J. A. D. Da Silva, “The Perron Root of a Weighted Geometric Mean of Nonnegative Matrices,” Lin. Multilin. Alg., Vol. 24, pp. 1–13, 1988. (Cited on page 457.) [467] L. Elsner and M. H. C. Paardekooper, “On Measures of Nonnormality of Matrices,” Lin. Alg. Appl., Vol. 92, pp. 107–124, 1987. (Cited on page 194.) [468] L. Elsner and P. Rozsa, “On Eigenvectors and Adjoints of Modified Matrices,” Lin. Multilin. Alg., Vol. 10, pp. 235–247, 1981. (Cited on page 154.) [469] L. Elsner and T. Szulc, “Convex Sets of Schur Stable and Stable Matrices,” Lin. Multilin. Alg., Vol. 48, pp. 1–19, 2000. (Cited on page 293.) [470] A. Engel, Problem-Solving Strategies. New York: Springer, 1998. (Cited on pages 34, 35, 37, 42, 44, 45, 46, 48, 49, 50, 51, 54, 57, 64, 66, and 171.) [471] K. Engo, “On the BCH Formula in so(3),” Numer. Math. BIT, Vol. 41, pp. 629– 632, 2001. (Cited on page 749.) [472] K. Erdmann and M. J. Wildon, Introduction to Lie Algebras. New York: Springer, 2006. (Cited on page 252.) [473] J. A. Erdos, “On Products of Idempotent Matrices,” Glasgow Math. J., Vol. 8, pp. 118–122, 1967. (Cited on page 383.) [474] R. Eriksson, “On the Measure of Solid Angles,” Math. Mag., Vol. 63, pp. 184–187, 1990. (Cited on page 175.) [475] J.-C. Evard and F. Uhlig, “On the Matrix Equation f (X) = A,” Lin. Alg. Appl., Vol. 162–164, pp. 447–519, 1992. (Cited on page 793.) [476] S. Fallat and M. J. Tsatsomeros, “On the Cayley Transform of Positivity Classes of Matrices,” Elec. J. Lin. Alg., Vol. 9, pp. 190–196, 2002. (Cited on page 498.) [477] K. Fan, “Generalized Cayley Transforms and Strictly Dissipative Matrices,” Lin. Alg. Appl., Vol. 5, pp. 155–172, 1972. (Cited on page 498.) [478] K. Fan, “On Real Matrices with Positive Definite Symmetric Component,” Lin. Multilin. Alg., Vol. 1, pp. 1–4, 1973. (Cited on pages 498 and 536.) [479] K. Fan, “On Strictly Dissipative Matrices,” Lin. Alg. Appl., Vol. 9, pp. 223–241, 1974. (Cited on pages 498 and 534.) [480] M. Fang, “Bounds on Eigenvalues of the Hadamard Product and the Fan Product of Matrices,” Lin. Alg. Appl., Vol. 425, pp. 7–15, 2007. (Cited on page 457.) [481] Y. Fang, K. A. Loparo, and X. Feng, “Inequalities for the Trace of Matrix Product,” IEEE Trans. Autom. Contr., Vol. 39, pp. 2489–2490, 1994. (Cited on page 364.) [482] R. W. Farebrother, “A Class of Square Roots of Involutory Matrices,” IMAGE, Vol. 28, pp. 26–28, 2002. (Cited on page 438.)

908

BIBLIOGRAPHY

[483] R. W. Farebrother, J. Gross, and S.-O. Troschke, “Matrix Representation of Quaternions,” Lin. Alg. Appl., Vol. 362, pp. 251–255, 2003. (Cited on page 251.) [484] R. W. Farebrother and I. Wrobel, “Regular and Reflected Rotation Matrices,” IMAGE, Vol. 29, pp. 24–25, 2002. (Cited on page 380.) [485] D. R. Farenick, M. Krupnik, N. Krupnik, and W. Y. Lee, “Normal Toeplitz Matrices,” SIAM J. Matrix Anal. Appl., Vol. 17, pp. 1037–1043, 1996. (Cited on page 390.) [486] A. Fassler and E. Stiefel, Group Theoretical Methods and Their Applications. Boston: Birkhauser, 1992. (Cited on page 252.) [487] A. E. Fekete, Real Linear Algebra. pages 204 and 217.)

New York: Marcel Dekker, 1985. (Cited on

[488] B. Q. Feng, “Equivalence Constants for Certain Matrix Norms,” Lin. Alg. Appl., Vol. 374, pp. 247–253, 2003. (Cited on page 640.) [489] B. Q. Feng and A. Tonge, “Equivalence Constants for Certain Matrix Norms, II,” Lin. Alg. Appl., Vol. 420, pp. 388–399, 2007. (Cited on pages 630 and 640.) [490] R. Fenn, Geometry. 248, 249, and 304.)

New York: Springer, 2001. (Cited on pages 175, 204, 213,

[491] P. G. Ferreira and S. P. Bhattacharyya, “On Blocking Zeros,” IEEE Trans. Autom. Contr., Vol. AC-22, pp. 258–259, 1977. (Cited on page 879.) [492] J. H. Ferziger and M. Peric, Computational Methods for Fluid Dynamics, 3rd ed. Berlin: Springer, 2002. (Cited on page xvii.) [493] M. Fiedler, “A Note on the Hadamard Product of Matrices,” Lin. Alg. Appl., Vol. 49, pp. 233–235, 1983. (Cited on pages 585 and 589.) [494] M. Fiedler, Special Matrices and Their Applications in Numerical Mathematics. Dordrecht: Martinus Nijhoff, 1986, reprinted by Dover, Mineola, 2008. (Cited on pages xvii, 277, 279, 299, 359, 388, 427, 454, and 770.) [495] M. Fiedler, “Some Applications of Matrices and Graphs in Euclidean Geometry,” in Handbook of Linear Algebra, L. Hogben, Ed. Boca Raton: Chapman & Hall/CRC, 2007, pp. 66–1–66–15. (Cited on pages 174 and 585.) [496] M. Fiedler and T. L. Markham, “A Characterization of the Moore-Penrose Inverse,” Lin. Alg. Appl., Vol. 179, pp. 129–133, 1993. (Cited on pages 161 and 410.) [497] M. Fiedler and T. L. Markham, “An Observation on the Hadamard Product of Hermitian Matrices,” Lin. Alg. Appl., Vol. 215, pp. 179–182, 1995. (Cited on page 587.) [498] M. Fiedler and T. L. Markham, “Some Results on the Bergstrom and Minkowski Inequalities,” Lin. Alg. Appl., Vol. 232, pp. 199–211, 1996. (Cited on page 515.) [499] M. Fiedler and V. Ptak, “A New Positive Definite Geometric Mean of Two Positive Definite Matrices,” Lin. Alg. Appl., Vol. 251, pp. 1–20, 1997. (Cited on pages 509 and 510.) [500] J. A. Fill and D. E. Fishkind, “The Moore-Penrose Generalized Inverse for Sums of Matrices,” SIAM J. Matrix Anal. Appl., Vol. 21, pp. 629–635, 1999. (Cited on page 413.)

909

BIBLIOGRAPHY

[501] P. A. Fillmore, “On Similarity and the Diagonal of a Matrix,” Amer. Math. Monthly, Vol. 76, pp. 167–169, 1969. (Cited on page 341.) [502] P. A. Fillmore, “On Sums of Projections,” J. Funct. Anal., Vol. 4, pp. 146–152, 1969. (Cited on page 394.) [503] P. A. Fillmore and J. P. Williams, “On Operator Ranges,” Adv. Math., Vol. 7, pp. 254–281, 1971. (Cited on page 474.) [504] C. H. Fitzgerald and R. A. Horn, “On Fractional Hadamard Powers of Positive Definite Matrices,” J. Math. Anal. Appl., Vol. 61, pp. 633–642, 1977. (Cited on page 584.) [505] H. Flanders, “Methods of Proof in Linear Algebra,” Amer. Math. Monthly, Vol. 63, pp. 1–15, 1956. (Cited on page 233.) [506] H. Flanders, “On the Norm and Spectral Radius,” Lin. Multilin. Alg., Vol. 2, pp. 239–240, 1974. (Cited on page 660.) [507] H. Flanders, “An Extremal Problem in the Space of Positive Definite Matrices,” Lin. Multilin. Alg., Vol. 3, pp. 33–39, 1975. (Cited on page 504.) [508] W. Fleming, Functions of Several Variables, 2nd ed. (Cited on page 543.)

New York: Springer, 1987.

[509] T. M. Flett, Differential Analysis. Cambridge: Cambridge University Press, 1980. (Cited on page 705.) [510] S. Foldes, Fundamental Structures of Algebra and Discrete Mathematics. York: Wiley, 1994. (Cited on page 242.)

New

[511] J. Foley, A. van Dam, S. Feiner, and J. Hughes, Computer Graphics Principles and Practice, 2nd ed. Reading: Addison-Wesley, 1990. (Cited on pages xvii and 215.) [512] E. Formanek, “Polynomial Identities and the Cayley-Hamilton Theorem,” Mathematical Intelligencer, Vol. 11, pp. 37–39, 1989. (Cited on pages 161, 233, and 283.) [513] E. Formanek, The Polynomial Identities and Invariants of n × n Matrices. Providence: American Mathematical Society, 1991. (Cited on pages 161, 233, and 283.) [514] L. R. Foulds, Graph Theory Applications. page xvii.)

New York: Springer, 1992. (Cited on

[515] B. A. Francis, A Course in H∞ Control Theory. New York: Springer, 1987. (Cited on page xvii.) [516] J. Franklin, Matrix Theory. page xix.)

Englewood Cliffs: Prentice-Hall, 1968. (Cited on

[517] M. Frazier, An Introduction to Wavelets through Linear Algebra. Springer, 1999. (Cited on page xvii.)

New York:

[518] S. Friedland, “A Note on a Determinantal Inequality,” Lin. Alg. Appl., Vol. 141, pp. 221–222, 1990. (Cited on page 659.) [519] S. Friedland, D. Hershkowitz, and S. M. Rump, “Positive Entries of Stable Matrices,” Elec. J. Lin. Alg., Vol. 12, pp. 17–24, 2005. (Cited on page 777.)

910

BIBLIOGRAPHY

[520] S. Friedland and A. Torokhti, “Generalized Rank-Constrained Matrix Approximation,” SIAM J. Matrix Anal. Appl., Vol. 29, pp. 656–659, 2007. (Cited on pages 672 and 679.) [521] P. A. Fuhrmann, A Polynomial Approach to Linear Algebra. New York: Springer, 1996. (Cited on pages 208, 209, 277, 279, 280, 307, 336, and 520.) [522] J. I. Fujii and M. Fujii, “Kolmogorov’s Complexity for Positive Definite Matrices,” Lin. Alg. Appl., Vol. 341, pp. 171–180, 2002. (Cited on page 552.) [523] J. I. Fujii, M. Fujii, T. Furuta, and R. Nakamoto, “Norm Inequalities Equivalent to Heinz Inequality,” Proc. Amer. Math. Soc., Vol. 118, pp. 827–830, 1993. (Cited on page 646.) [524] J.-I. Fujii, S. Izumino, and Y. Seo, “Determinant for Positive Operators and Specht’s Theorem,” Scientiae Mathematicae, Vol. 1, pp. 307–310, 1998. (Cited on page 57.) [525] M. Fujii, E. Kamei, and R. Nakamoto, “On a Question of Furuta on Chaotic Order,” Lin. Alg. Appl., Vol. 341, pp. 119–129, 2002. (Cited on pages 507, 513, and 575.) [526] M. Fujii, E. Kamei, and R. Nakamoto, “On a Question of Furuta on Chaotic Order, II,” Math. J. Okayama University, Vol. 45, pp. 123–131, 2003. (Cited on page 513.) [527] M. Fujii and F. Kubo, “Buzano’s Inequality and Bounds for Roots of Algebraic Equations,” Proc. Amer. Math. Soc., Vol. 117, pp. 359–361, 1993. (Cited on pages 74, 620, and 780.) [528] M. Fujii, S. H. Lee, Y. Seo, and D. Jung, “Reverse Inequalities on Chaotically Geometric Mean via Specht Ratio,” Math. Ineq. Appl., Vol. 6, pp. 509–519, 2003. (Cited on page 37.) [529] M. Fujii, S. H. Lee, Y. Seo, and D. Jung, “Reverse Inequalities on Chaotically Geometric Mean via Specht Ratio, II,” J. Ineq. Pure Appl. Math., Vol. 4/40, pp. 1–8, 2003. (Cited on page 57.) [530] M. Fujii and R. Nakamoto, “Rota’s Theorem and Heinz Inequalities,” Lin. Alg. Appl., Vol. 214, pp. 271–275, 1995. (Cited on page 646.) [531] M. Fujii, Y. Seo, and M. Tominaga, “Golden-Thompson Type Inequalities Related to a Geometric Mean via Specht’s Ratio,” Math. Ineq. Appl., Vol. 5, pp. 573–582, 2002. (Cited on page 752.) [532] A. T. Fuller, “Conditions for a Matrix to Have Only Characteristic Roots with Negative Real Parts,” J. Math. Anal. Appl., Vol. 23, pp. 71–98, 1968. (Cited on pages 447, 454, 458, and 765.) [533] W. Fulton and J. Harris, Representation Theory. (Cited on page 244.)

New York: Springer, 2004.

[534] S. Furuichi, “Matrix Trace Inequalities on the Tsallis Entropies,” J. Ineq. Pure Appl. Math., Vol. 9, no. 1, pp. 1–7, 2008, Article 1. (Cited on page 527.) [535] S. Furuichi, K. Kuriyama, and K. Yanagi, “Trace Inequalities for Products of Matrices,” Lin. Alg. Appl., Vol. 430, pp. 2271–2276, 2009. (Cited on page 526.)

BIBLIOGRAPHY

911

[536] T. Furuta, “A ≥ B ≥ 0 Assures (B rAp B r )1/q ≥ B (p+2r)/q for r ≥ 0, p ≥ 0, q ≥ 1 with (1 + 2r)q ≥ p + 2r,” Proc. Amer. Math. Soc., Vol. 101, pp. 85–88, 1987. (Cited on page 476.) [537] T. Furuta, “Norm Inequalities Equivalent to L¨ owner-Heinz Theorem,” Rev. Math. Phys., Vol. 1, pp. 135–137, 1989. (Cited on pages 573 and 639.) [538] T. Furuta, “Two Operator Functions with Monotone Property,” Proc. Amer. Math. Soc., Vol. 111, pp. 511–516, 1991. (Cited on page 507.) [539] T. Furuta, “A Note on the Arithmetic-Geometric-Mean Inequality for Every Unitarily Invariant Matrix Norm,” Lin. Alg. Appl., Vol. 208/209, pp. 223–228, 1994. (Cited on page 646.) [540] T. Furuta, “Extension of the Furuta Inequality and Ando-Hiai Log-Majorization,” Lin. Alg. Appl., Vol. 219, pp. 139–155, 1995. (Cited on pages 507 and 513.) [541] T. Furuta, “Generalizations of Kosaki Trace Inequalities and Related Trace Inequalities on Chaotic Order,” Lin. Alg. Appl., Vol. 235, pp. 153–161, 1996. (Cited on page 529.) [542] T. Furuta, “Operator Inequalities Associated with H¨ older-McCarthy and Kantorovich Inequalities,” J. Ineq. Appl., Vol. 2, pp. 137–148, 1998. (Cited on page 511.) [543] T. Furuta, “Simple Proof of the Concavity of Operator Entropy f (A) = −A log A,” Math. Ineq. Appl., Vol. 3, pp. 305–306, 2000. (Cited on page 485.) [544] T. Furuta, Invitation to Linear Operators: From Matrices to Bounded Linear Operators on a Hilbert Space. London: Taylor and Francis, 2001. (Cited on pages 26, 27, 227, 476, 477, 479, 501, 507, 511, 513, 552, 562, 575, and 623.) ≥ [545] T. Furuta, “Spectral Order A  B if and only if A2p−r  (2p−r)/(p−r) −r/2 p −r/2 A BA for all p > r ≥ 0 and Its Application,” Math. Ineq. Appl., Vol. 4, pp. 619–624, 2001. (Cited on pages 513 and 575.) [546] T. Furuta, “The H¨ older-McCarthy and the Young Inequalities Are Equivalent for Hilbert Space Operators,” Amer. Math. Monthly, Vol. 108, pp. 68–69, 2001. (Cited on page 552.) [547] T. Furuta, “Convergence of Logarithmic Trace Inequalities via Generalized LieTrotter Formulae,” Lin. Alg. Appl., Vol. 396, pp. 353–372, 2005. (Cited on pages 749 and 753.) [548] T. Furuta, “Concrete Examples of Operator Monotone Functions Obtained by an Elementary Method without Appealing to L¨ owner Integral Representation,” Lin. Alg. Appl., Vol. 429, pp. 972–980, 2008. (Cited on page 514.) [549] F. Gaines, “A Note on Matrices with Zero Trace,” Amer. Math. Monthly, Vol. 73, pp. 630–631, 1966. (Cited on pages 199, 200, and 341.) [550] A. Galantai, Projectors and Projection Methods. Dordrecht: Kluwer, 2004. (Cited on pages 221, 252, 343, 344, 358, 366, 627, and 698.) [551] A. Galantai, “Subspaces, Angles and Pairs of Orthogonal Projections,” Lin. Multilin. Alg., Vol. 56, pp. 227–260, 2008. (Cited on pages 122, 227, 228, 343, 344, 358, 366, 367, and 416.)

912

BIBLIOGRAPHY

[552] J. Gallier and D. Xu, “Computing Exponentials of Skew-Symmetric Matrices and Logarithms of Orthogonal Matrices,” Int. J. Robotics Automation, Vol. 17, pp. 1–11, 2002. (Cited on page 214.) [553] L. Gangsong and Z. Guobiao, “Inverse Forms of Hadamard Inequality,” SIAM J. Matrix Anal. Appl., Vol. 23, pp. 990–997, 2002. (Cited on page 563.) [554] F. R. Gantmacher, The Theory of Matrices. (Cited on pages xix and 595.)

New York: Chelsea, 1959, Vol. I.

[555] F. R. Gantmacher, The Theory of Matrices. (Cited on pages xix, 331, 346, and 793.)

New York: Chelsea, 1959, Vol. II.

[556] D. J. H. Garling, Inequalities: A Journey into Linear Analysis. Cambridge: Cambridge University Press, 2007. (Cited on pages 36, 58, 59, 64, 69, 82, 176, 624, and 630.) [557] J. Garloff and D. G. Wagner, “Hadamard Products of Stable Polynomials Are Stable,” J. Math. Anal. Appl., Vol. 202, pp. 797–809, 1996. (Cited on page 765.) [558] T. Geerts, “A Necessary and Sufficient Condition for Solvability of the LinearQuadratic Control Problem without Stability,” Sys. Contr. Lett., Vol. 11, pp. 47– 51, 1988. (Cited on page 879.) [559] I. M. Gelfand, M. M. Kapranov, and A. V. Zelevinsky, Discriminants, Resultants, and Multidimensional Determinants. Boston: Birkhauser, 1994. (Cited on page 458.) [560] M. G. Genton, “Classes of Kernels for Machine Learning: A Statistics Perspective,” J. Machine Learning Res., Vol. 2, pp. 299–312, 2001. (Cited on page 489.) [561] A. George and K. D. Ikramov, “Common Invariant Subspaces of Two Matrices,” Lin. Alg. Appl., Vol. 287, pp. 171–179, 1999. (Cited on pages 373, 391, and 392.) [562] P. Gerdes, Adventures in the World of Matrices. 2007. (Cited on page 390.)

Hauppauge: Nova Publishers,

[563] A. Gerrard and J. M. Burch, Introduction to Matrix Methods in Optics. York: Wiley, 1975. (Cited on page xvii.)

New

[564] A. Gheondea, S. Gudder, and P. Jonas, “On the Infimum of Quantum Effects,” J. Math. Phys., Vol. 46, pp. 1–11, 2005, paper 062102. (Cited on page 506.) [565] F. A. A. Ghouraba and M. A. Seoud, “Set Matrices,” Int. J. Math. Educ. Sci. Technol., Vol. 30, pp. 651–659, 1999. (Cited on page 252.) [566] M. I. Gil, “On Inequalities for Eigenvalues of Matrices,” Lin. Alg. Appl., Vol. 184, pp. 201–206, 1993. (Cited on page 354.) [567] R. Gilmore, Lie Groups, Lie Algebras, and Some of Their Applications. New York: Wiley, 1974, reprinted by Dover, Mineola, 2005. (Cited on page 252.) [568] R. Gilmore, Lie Groups, Physics, and Geometry: An Introduction for Physicists, Engineers, and Chemists. Cambridge: Cambridge University Press, 2008. (Cited on page 252.) [569] P. R. Girard, Quaternions, Clifford Algebras and Relativistic Physics. Birkhauser, 2007. (Cited on pages 204, 213, 214, and 246.)

Boston:

913

BIBLIOGRAPHY

[570] M. L. Glasser, “Exponentials of Certain Hilbert Space Operators,” SIAM Rev., Vol. 34, pp. 498–500, 1992. (Cited on page 750.) [571] C. Godsil and G. Royle, Algebraic Graph Theory. (Cited on page xvii.)

New York: Springer, 2001.

[572] S. K. Godunov, Modern Aspects of Linear Algebra. Providence: American Mathematical Society, 1998. (Cited on pages xix, 229, and 767.) [573] I. Gohberg, P. Lancaster, and L. Rodman, Matrix Polynomials. New York: Academic Press, 1982. (Cited on pages 256, 307, and 396.) [574] I. Gohberg, P. Lancaster, and L. Rodman, Invariant Subspaces of Matrices with Applications. New York: Wiley, 1986. (Cited on pages 366, 396, and 698.) [575] I. Gohberg, P. Lancaster, and L. Rodman, “On Hermitian Solutions of the Symmetric Algebraic Riccati Equation,” SIAM J. Contr. Optim., Vol. 24, pp. 1323–1334, 1986. (Cited on pages 860, 861, and 879.) [576] I. Gohberg, P. Lancaster, and L. Rodman, Indefinite Linear Algebra and Applications. Boston: Birkhauser, 2005. (Cited on pages 396 and 879.) [577] M. A. Golberg, “The Derivative of a Determinant,” Amer. Math. Monthly, Vol. 79, pp. 1124–1126, 1972. (Cited on pages 703 and 743.) [578] M. Goldberg, “Mixed Multiplicativity and lp Norms for Matrices,” Lin. Alg. Appl., Vol. 73, pp. 123–131, 1986. (Cited on page 640.) [579] M. Goldberg, “Equivalence Constants for lp Norms of Matrices,” Lin. Multilin. Alg., Vol. 21, pp. 173–179, 1987. (Cited on page 640.) [580] M. Goldberg, “Multiplicativity Factors and Mixed Multiplicativity,” Lin. Alg. Appl., Vol. 97, pp. 45–56, 1987. (Cited on page 640.) [581] M. Goldberg and G. Zwas, “On Matrices Having Equal Spectral Radius and Spectral Norm,” Lin. Alg. Appl., Vol. 8, pp. 427–434, 1974. (Cited on page 661.) [582] H. Goller, “Shorted Operators and Rank Decomposition Matrices,” Lin. Alg. Appl., Vol. 81, pp. 207–236, 1986. (Cited on page 583.) [583] G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed. Baltimore: Johns Hopkins University Press, 1996. (Cited on pages xviii, 672, and 710.) [584] N. C. Gonzalez, J. J. Koliha, and Y. Wei, “Integral Representation of the Drazin Inverse,” Elect. J. Lin. Alg., Vol. 9, pp. 129–131, 2002. (Cited on page 745.) [585] F. M. Goodman, Algebra: Abstract and Concrete, 2nd ed. Prentice-Hall, 2003. (Cited on page 245.)

Englewood Cliffs:

[586] L. E. Goodman and W. H. Warner, Statics. Belmont: Wadsworth Publishing Company, 1964, reprinted by Dover, Mineola, 2001. (Cited on page 204.) [587] N. Gordon and D. Salmond, “Bayesian Pattern Matching Technique for Target Acquisition,” J. Guid. Contr. Dyn., Vol. 22, pp. 68–77, 1999. (Cited on page 151.) [588] A. Goroncy and T. Rychlik, “How Deviant Can You Be? The Complete Solution,” Math. Ineq. Appl., Vol. 9, pp. 633–647, 2006. (Cited on page 56.) [589] W. Govaerts and J. D. Pryce, “A Singular Value Inequality for Block Matrices,” Lin. Alg. Appl., Vol. 125, pp. 141–145, 1989. (Cited on page 668.)

914

BIBLIOGRAPHY

[590] W. Govaerts and B. Sijnave, “Matrix Manifolds and the Jordan Structure of the Bialternate Matrix Product,” Lin. Alg. Appl., Vol. 292, pp. 245–266, 1999. (Cited on pages 454 and 458.) [591] R. Gow, “The Equivalence of an Invertible Matrix to Its Transpose,” Lin. Alg. Appl., Vol. 8, pp. 329–336, 1980. (Cited on pages 340 and 384.) [592] R. Gow and T. J. Laffey, “Pairs of Alternating Forms and Products of Two SkewSymmetric Matrices,” Lin. Alg. Appl., Vol. 63, pp. 119–132, 1984. (Cited on pages 375 and 384.) [593] A. Graham, Kronecker Products and Matrix Calculus with Applications. ester: Ellis Horwood, 1981. (Cited on page 458.) [594] F. A. Graybill, Matrices with Applications in Statistics, 2nd ed. Wadsworth, 1983. (Cited on page xvii.)

ChichBelmont:

[595] J. Grcar, “Linear Algebra People,” http://seesar.lbl.gov/ccse/people/grcar/index. html. (Cited on page 178.) [596] J. F. Grcar, “A Matrix Lower Bound,” Lawrence Berkeley National Laboratory, Report LBNL-50635, 2002. (Cited on pages 613, 614, and 615.) [597] W. L. Green and T. D. Morley, “Operator Means and Matrix Functions,” Lin. Alg. Appl., Vol. 137/138, pp. 453–465, 1990. (Cited on pages 509 and 582.) [598] D. H. Greene and D. E. Knuth, Mathematics for the Analysis of Algorithms, 3rd ed. Boston: Birkhauser, 1990. (Cited on page 19.) [599] W. H. Greub, Multilinear Algebra. New York: Springer, 1967. (Cited on page 458.) [600] W. H. Greub, Linear Algebra.

New York: Springer, 1981. (Cited on page xix.)

[601] T. N. E. Greville, “Notes on the Generalized Inverse of a Matrix Product,” SIAM Rev., Vol. 8, pp. 518–521, 1966. (Cited on page 415.) [602] T. N. E. Greville, “Solutions of the Matrix Equation XAX = X and Relations between Oblique and Orthogonal projectors,” SIAM J. Appl. Math, Vol. 26, pp. 828–832, 1974. (Cited on pages 228, 416, and 417.) [603] R. Grone, C. R. Johnson, E. M. Sa, and H. Wolkowicz, “Normal Matrices,” Lin. Alg. Appl., Vol. 87, pp. 213–225, 1987. (Cited on pages 194, 205, and 372.) [604] J. Gross, “A Note on a Partial Ordering in the Set of Hermitian Matrices,” SIAM J. Matrix Anal. Appl., Vol. 18, pp. 887–892, 1997. (Cited on pages 576, 578, and 579.) [605] J. Gross, “Some Remarks on Partial Orderings of Hermitian Matrices,” Lin. Multilin. Alg., Vol. 42, pp. 53–60, 1997. (Cited on pages 576 and 578.) [606] J. Gross, “More on Concavity of a Matrix Function,” SIAM J. Matrix Anal. Appl., Vol. 19, pp. 365–368, 1998. (Cited on page 583.) [607] J. Gross, “On Oblique Projection, Rank Additivity and the Moore-Penrose Inverse of the Sum of Two Matrices,” Lin. Multilin. Alg., Vol. 46, pp. 265–275, 1999. (Cited on page 416.) [608] J. Gross, “On the Product of Orthogonal Projectors,” Lin. Alg. Appl., Vol. 289, pp. 141–150, 1999. (Cited on pages 226 and 417.)

BIBLIOGRAPHY

915

[609] J. Gross, “The Moore-Penrose Inverse of a Partitioned Nonnegative Definite Matrix,” Lin. Alg. Appl., Vol. 321, pp. 113–121, 2000. (Cited on page 429.) [610] J. Gross and G. Trenkler, “On the Product of Oblique Projectors,” Lin. Multilin. Alg., Vol. 44, pp. 247–259, 1998. (Cited on page 221.) [611] J. Gross and G. Trenkler, “Nonsingularity of the Difference of Two Oblique Projectors,” SIAM J. Matrix Anal. Appl., Vol. 21, pp. 390–395, 1999. (Cited on pages 219, 220, and 221.) [612] J. Gross and G. Trenkler, “Product and Sum of Projections,” Amer. Math. Monthly, Vol. 111, pp. 261–262, 2004. (Cited on page 366.) [613] J. Gross, G. Trenkler, and S.-O. Troschke, “The Vector Cross Product in C,” Int. J. Math. Educ. Sci. Tech., Vol. 30, pp. 549–555, 1999. (Cited on page 204.) [614] J. Gross, G. Trenkler, and S.-O. Troschke, “Quaternions: Further Contributions to a Matrix Oriented Approach,” Lin. Alg. Appl., Vol. 326, pp. 205–213, 2001. (Cited on page 251.) [615] J. Gross and S.-O. Troschke, “Some Remarks on Partial Orderings of Nonnegative Definite Matrices,” Lin. Alg. Appl., Vol. 264, pp. 457–461, 1997. (Cited on pages 576 and 579.) [616] J. Gross and J. Yellen, Graph Theory and Its Applications, 2nd ed. Chapman & Hall/CRC, 2005. (Cited on page xvii.)

Boca Raton:

[617] I. Grossman and W. Magnus, Groups and Their Graphs. Washington, DC: The Mathematical Association of America, 1992. (Cited on page 244.) [618] L. C. Grove and C. T. Benson, Finite Reflection Groups, 2nd ed. Springer, 1996. (Cited on page 245.)

New York:

[619] K. Guan and H. Zhu, “A Generalized Heronian Mean and Its Inequalities,” University Beogradu Publ. Elektrotehn. Fak., Vol. 17, pp. 1–16, 2006, http://pefmath.etf.bg.ac.yu/to%20appear/rad528.pdf. (Cited on page 41.) [620] S. Gull, A. Lasenby, and C. Doran, “Imaginary Numbers Are not Real: The Geometric Algebra of Spacetime,” Found. Phys., Vol. 23, pp. 1175–1201, 1993. (Cited on pages 204, 249, and 620.) [621] A. K. Gupta and D. K. Nagar, Matrix Variate Distributions. Boca Raton: CRC, 1999. (Cited on page xvii.) [622] K. Gurlebeck and W. Sprossig, Quaternionic and Clifford Calculus for Physicists and Engineers. New York: Chichester, 1997. (Cited on pages 249, 251, and 740.) [623] L. Gurvits, “Stability of Discrete Linear Inclusion,” Lin. Alg. Appl., Vol. 231, pp. 47–85, 1995. (Cited on page 786.) [624] K. E. Gustafson, “Matrix Trigonometry,” Lin. Alg. Appl., Vol. 217, pp. 117–140, 1995. (Cited on page 633.) [625] K. E. Gustafson and D. K. M. Rao, Numerical Range. New York: Springer, 1997. (Cited on page 546.) [626] W. H. Gustafson, P. R. Halmos, and H. Radjavi, “Products of Involutions,” Lin. Alg. Appl., Vol. 13, pp. 157–162, 1976. (Cited on page 383.)

916

BIBLIOGRAPHY

[627] P. W. Gwanyama, “The HM-GM-QM-QM Inequalities,” College Math. J., Vol. 35, pp. 47–50, 2004. (Cited on page 57.) [628] W. M. Haddad and D. S. Bernstein, “Robust Stabilization with Positive Real Uncertainty: Beyond the Small Gain Theorem,” Sys. Contr. Lett., Vol. 17, pp. 191–208, 1991. (Cited on page 539.) [629] W. M. Haddad and D. S. Bernstein, “Controller Design with Regional Pole Constraints,” IEEE Trans. Autom. Contr., Vol. 37, pp. 54–69, 1992. (Cited on page 871.) [630] W. M. Haddad and V. Chellaboina, “Stability and Dissipativity Theory for Nonnegative Dynamical Systems: A Unified Analysis Framework for Biological and Physiological Systems,” Nonlinear Anal. Real World Appl., Vol. 6, pp. 35–65, 2005. (Cited on page 776.) [631] W. M. Haddad and V. Chellaboina, Nonlinear Dynamical Systems and Control. Princeton: Princeton University Press, 2008. (Cited on page xvii.) [632] W. M. Haddad, V. Chellaboina, and S. G. Nersesov, Thermodynamics: A Dynamical Systems Approach. Princeton: Princeton University Press, 2005. (Cited on pages 252 and 777.) [633] W. M. Haddad, V. Kapila, and E. G. Collins, “Optimality Conditions for ReducedOrder Modeling, Estimation, and Control for Discrete-Time Linear Periodic Plants,” J. Math. Sys. Est. Contr., Vol. 6, pp. 437–460, 1996. (Cited on page 879.) [634] W. W. Hager, “Updating the Inverse of a Matrix,” SIAM Rev., Vol. 31, pp. 221– 239, 1989. (Cited on page 178.) [635] W. Hahn, Stability of Motion.

Berlin: Springer, 1967. (Cited on page 793.)

[636] E. Hairer, C. Lubich, and G. Wanner, Geometric Numerical Integration: StructurePreserving Algorithms for Ordinary Differential Equations. Berlin: Springer, 2002. (Cited on page 743.) [637] M. Hajja, “A Method for Establishing Certain Trigonometric Inequalities,” J. Ineq. Pure Appl. Math., Vol. 8, no. 1, pp. 1–11, 2007, Article 29. (Cited on page 171.) [638] A. Hall, “Conditions for a Matrix Kronecker Lemma,” Lin. Alg. Appl., Vol. 76, pp. 271–277, 1986. (Cited on page 511.) [639] B. C. Hall, Lie Groups, Lie Algebras, and Representations: An Elementary Introduction. New York: Springer, 2003. (Cited on pages 239, 718, 720, 722, 723, 724, 746, 750, and 793.) [640] G. T. Halliwell and P. R. Mercer, “A Refinement of an Inequality from Information Theory,” J. Ineq. Pure Appl. Math., Vol. 5, no. 1, pp. 1–3, 2004, Article 3. (Cited on pages 30 and 74.) [641] P. R. Halmos, Finite-Dimensional Vector Spaces. Princeton: Van Nostrand, 1958, reprinted by Springer, New York, 1974. (Cited on pages 341, 371, and 411.) [642] P. R. Halmos, A Hilbert Space Problem Book, 2nd ed. New York: Springer, 1982. (Cited on pages 346, 374, 421, and 422.) [643] P. R. Halmos, “Bad Products of Good Matrices,” Lin. Multilin. Alg., Vol. 29, pp. 1–20, 1991. (Cited on page 382.)

917

BIBLIOGRAPHY

[644] P. R. Halmos, Problems for Mathematicians Young and Old. Washington, DC: Mathematical Association of America, 1991. (Cited on pages 122, 305, 496, and 747.) [645] P. R. Halmos, Linear Algebra Problem Book. Washington, DC: Mathematical Association of America, 1995. (Cited on pages 98, 219, and 374.) [646] M. Hamermesh, Group Theory and its Application to Physical Problems. Reading: Addison-Wesley, 1962, reprinted by Dover, Mineola, 1989. (Cited on page 244.) [647] J. H. Han and M. D. Hirschhorn, “Another Look at an Amazing Identity of Ramanujan,” Math. Mag., Vol. 79, pp. 302–304, 2006. (Cited on page 139.) [648] E. J. Hannan and M. Deistler, The Statistical Theory of Linear Systems. York: Wiley, 1988. (Cited on page 879.)

New

[649] F. Hansen, “Extrema for Concave Operator Mappings,” Math. Japonica, Vol. 40, pp. 331–338, 1994. (Cited on page 531.) [650] F. Hansen, “Operator Inequalities Associated with Jensen’s Inequality,” in Survey on Classical Inequalities, T. M. Rassias, Ed. Dordrecht: Kluwer, 2000, pp. 67–98. (Cited on page 531.) [651] F. Hansen, “Some Operator Monotone Functions,” Lin. Alg. Appl., Vol. 430, pp. 795–799, 2009. (Cited on page 40.) [652] A. J. Hanson, Visualizing Quaternions. pages 249 and 251.)

Amsterdam: Elsevier, 2006. (Cited on

[653] G. Hardy, J. E. Littlewood, and G. Polya, Inequalities. University Press, 1988. (Cited on page 84.)

Cambridge: Cambridge

[654] J. M. Harris, J. L. Hirst, and M. J. Missinghoff, Combinatorics and Graph Theory, 2nd ed. New York: Springer, 2008. (Cited on page xvii.) [655] L. A. Harris, “Factorizations of Operator Matrices,” Lin. Alg. Appl., Vol. 225, pp. 37–41, 1995. (Cited on pages 520 and 521.) [656] L. A. Harris, “The Inverse of a Block Matrix,” Amer. Math. Monthly, Vol. 102, pp. 656–657, 1995. (Cited on page 161.) [657] W. A. Harris, J. P. Fillmore, and D. R. Smith, “Matrix Exponentials–Another Approach,” SIAM Rev., Vol. 43, pp. 694–706, 2001. (Cited on page 737.) [658] G. W. Hart, Multidimensional Analysis: Algebras and Systems for Science and Engineering. New York: Springer, 1995. (Cited on pages xvii and 252.) [659] D. J. Hartfiel, Nonhomogeneous Matrix Products. 2002. (Cited on pages xvii and 785.)

Singapore: World Scientific,

[660] R. Hartwig, X. Li, and Y. Wei, “Representations for the Drazin Inverse of a 2 × 2 Block Matrix,” SIAM J. Matrix Anal. Appl., Vol. 27, pp. 757–771, 2006. (Cited on page 429.) [661] R. E. Hartwig, “Resultants and the Solutions of AX − XB = −C,” SIAM J. Appl. Math., Vol. 23, pp. 104–117, 1972. (Cited on page 870.) [662] R. E. Hartwig, “Block Generalized Inverses,” Arch. Rat. Mech. Anal., Vol. 61, pp. 197–251, 1976. (Cited on page 429.)

918

BIBLIOGRAPHY

[663] R. E. Hartwig, “A Note on the Partial Ordering of Positive Semi-Definite Matrices,” Lin. Multilin. Alg., Vol. 6, pp. 223–226, 1978. (Cited on pages 576, 579, and 580.) [664] R. E. Hartwig, “How to Partially Order Regular Elements,” Math. Japonica, Vol. 25, pp. 1–13, 1980. (Cited on page 129.) [665] R. E. Hartwig and I. J. Katz, “On Products of EP Matrices,” Lin. Alg. Appl., Vol. 252, pp. 339–345, 1997. (Cited on page 418.) [666] R. E. Hartwig and M. S. Putcha, “When Is a Matrix a Difference of Two Idempotents?” Lin. Multilin. Alg., Vol. 26, pp. 267–277, 1990. (Cited on pages 351 and 367.) [667] R. E. Hartwig and M. S. Putcha, “When Is a Matrix a Sum of Idempotents?” Lin. Multilin. Alg., Vol. 26, pp. 279–286, 1990. (Cited on page 395.) [668] R. E. Hartwig and K. Spindelbock, “Matrices for which A∗ and A+ Commute,” Lin. Multilin. Alg., Vol. 14, pp. 241–256, 1984. (Cited on pages 344, 406, 407, and 436.) [669] R. E. Hartwig and G. P. H. Styan, “On Some Characterizations of the “Star” Partial Ordering for Matrices and Rank Subtractivity,” Lin. Alg. Appl., Vol. 82, pp. 145–161, 1989. (Cited on pages 129, 130, 422, and 579.) [670] R. E. Hartwig, G. Wang, and Y. Wei, “Some Additive Results on Drazin Inverse,” Lin. Alg. Appl., Vol. 322, pp. 207–217, 2001. (Cited on page 431.) [671] D. A. Harville, Matrix Algebra from a Statistician’s Perspective. New York: Springer, 1997. (Cited on pages xvii, 216, 219, 405, 411, 413, 417, 420, 535, 557, and 705.) [672] J. Hauke and A. Markiewicz, “On Partial Orderings on the Set of Rectangular Matrices,” Lin. Alg. Appl., Vol. 219, pp. 187–193, 1995. (Cited on page 577.) [673] M. L. J. Hautus, “Controllability and Observability Conditions of Linear Autonomous Systems,” Proc. Koniklijke Akademic Van Wetenshappen, Vol. 72, pp. 443–448, 1969. (Cited on page 879.) [674] J. Havil, Gamma: Exploring Euler’s Constant. Press, 2003. (Cited on page 21.)

Princeton: Princeton University

[675] T. Haynes, “Stable Matrices, the Cayley Transform, and Convergent Matrices,” Int. J. Math. Math. Sci., Vol. 14, pp. 77–81, 1991. (Cited on page 783.) [676] E. V. Haynsworth, “Applications of an Inequality for the Schur Complement,” Proc. Amer. Math. Soc., Vol. 24, pp. 512–516, 1970. (Cited on pages 485, 521, and 596.) [677] E. Hecht, Optics, 4th ed.

Reading: Addison Wesley, 2002. (Cited on page xvii.)

[678] C. Heij, A. Ran, and F. van Schagen, Eds., Introduction to Mathematical Systems Theory: Linear Systems, Identification and Control. Basel: Birkhauser, 2007. (Cited on page 879.) [679] C. Heij, A. Ran, and F. van Schagen, Introduction to Mathematical Systems Theory: Linear Systems, Identification and Control. Basel: Birkhauser, 2007. (Cited on page 879.)

919

BIBLIOGRAPHY

[680] G. Heinig, “Matrix Representations of Bezoutians,” Lin. Alg. Appl., Vol. 223/224, pp. 337–354, 1995. (Cited on page 279.) [681] U. Helmke and P. A. Fuhrmann, “Bezoutians,” Lin. Alg. Appl., Vol. 122–124, pp. 1039–1097, 1989. (Cited on page 279.) [682] B. W. Helton, “Logarithms of Matrices,” Proc. Amer. Math. Soc., Vol. 19, pp. 733–738, 1968. (Cited on page 793.) [683] H. V. Henderson, F. Pukelsheim, and S. R. Searle, “On the History of the Kronecker Product,” Lin. Multilin. Alg., Vol. 14, pp. 113–120, 1983. (Cited on page 458.) [684] H. V. Henderson and S. R. Searle, “On Deriving the Inverse of a Sum of Matrices,” SIAM Rev., Vol. 23, pp. 53–60, 1981. (Cited on pages 157 and 178.) [685] H. V. Henderson and S. R. Searle, “The Vec-Permutation Matrix, The Vec Operator and Kronecker Products: A Review,” Lin. Multilin. Alg., Vol. 9, pp. 271–288, 1981. (Cited on page 458.) [686] J. Herman, R. Kucera, and J. Simsa, Equations and Inequalities. New York: Springer, 2000. (Cited on pages 19, 20, 26, 27, 29, 35, 39, 40, 43, 44, 45, 46, 47, 48, 50, 51, 54, 57, and 66.) [687] D. Hershkowitz, “Positive Semidefinite Pattern Decompositions,” SIAM J. Matrix Anal. Appl., Vol. 11, pp. 612–619, 1990. (Cited on page 522.) [688] D. Hestenes, Space-Time Algebra. on page 249.)

New York: Gordon and Breach, 1966. (Cited

[689] D. Hestenes, New Foundations for Classical Mechanics. Dordrecht: Kluwer, 1986. (Cited on pages 204 and 249.) [690] D. Hestenes and G. Sobczyk, Clifford Algebra to Geometric Calculus: A Unified Language for Mathematics and Physics. Dordrecht: D. Riedel, 1984. (Cited on pages 204 and 249.) [691] F. Hiai and H. Kosaki, Eds., Means of Hilbert Space Operators. 2003. (Cited on pages 32, 38, and 489.)

Berlin: Springer,

[692] F. Hiai and D. Petz, “The Golden-Thompson Trace Inequality Is Complemented,” Lin. Alg. Appl., Vol. 181, pp. 153–185, 1993. (Cited on pages 507 and 753.) [693] F. Hiai and X. Zhan, “Submultiplicativity vs Subadditivity for Unitarily Invariant Norms,” Lin. Alg. Appl., Vol. 377, pp. 155–164, 2004. (Cited on page 637.) [694] D. J. Higham and N. J. Higham, Matlab Guide, 2nd ed. 2005. (Cited on page 178.)

Philadelphia: SIAM,

[695] N. J. Higham, “Newton’s Method for the Matrix Square Root,” Math. Computation, Vol. 46, pp. 537–549, 1986. (Cited on page 381.) [696] N. J. Higham, “Computing Real Square Roots of a Real Matrix,” Lin. Alg. Appl., Vol. 88/89, pp. 405–430, 1987. (Cited on page 381.) [697] N. J. Higham, “Matrix Nearness Problems and Applications,” in Applications of Matrix Theory, M. J. C. Gover and S. Barnett, Eds. Oxford: Oxford University Press, 1989, pp. 1–27. (Cited on pages 666, 768, and 769.) [698] N. J. Higham, “Estimating the Matrix p-Norm,” Numer. Math., Vol. 62, pp. 539– 555, 1992. (Cited on pages 626, 630, and 631.)

920

BIBLIOGRAPHY

[699] N. J. Higham, Accuracy and Stability of Numerical Algorithms, 2nd ed. Philadelphia: SIAM, 2002. (Cited on pages xviii, 234, 241, 357, 360, 626, 629, 630, 631, 632, 661, 666, 673, and 680.) [700] N. J. Higham, “Cayley, Sylvester, and Early Matrix Theory,” Lin. Alg. Appl., Vol. 428, pp. 39–43, 2008. (Cited on page 284.) [701] N. J. Higham, Functions of Matrices: Theory and Computation. Philadelphia: SIAM, 2008. (Cited on pages xviii, 78, 81, 381, 393, 394, 640, 690, 698, 699, 722, 742, 747, 751, 759, 761, 793, and 876.) [702] G. N. Hile and P. Lounesto, “Matrix Representations of Clifford Algebras,” Lin. Alg. Appl., Vol. 128, pp. 51–63, 1990. (Cited on page 249.) [703] R. D. Hill, R. G. Bates, and S. R. Waters, “On Centrohermitian Matrices,” SIAM J. Matrix Anal. Appl., Vol. 11, pp. 128–133, 1990. (Cited on page 242.) [704] R. D. Hill and E. E. Underwood, “On the Matrix Adjoint (Adjugate),” SIAM J. Alg. Disc. Meth., Vol. 6, pp. 731–737, 1985. (Cited on page 154.) [705] C.-J. Hillar, “Advances on the Bessis-Moussa-Villani Trace Conjecture,” Lin. Alg. Appl., Vol. 426, pp. 130–142, 2007. (Cited on page 530.) [706] L. O. Hilliard, “The Case of Equality in Hopf’s Inequality,” SIAM J. Alg. Disc. Meth., Vol. 8, pp. 691–709, 1987. (Cited on page 306.) [707] D. Hinrichsen, E. Plischke, and F. Wirth, “Robustness of Transient Behavior,” in Unsolved Problems in Mathematical Systems and Control Theory, V. D. Blondel and A. Megretski, Eds. Princeton: Princeton University Press, 2004. (Cited on page 767.) [708] D. Hinrichsen and A. J. Pritchard, Mathematical Systems Theory I: Modelling, State Space Analysis, Stability and Robustness. Berlin: Springer, 2005. (Cited on pages 347, 355, 356, 609, 676, 701, 758, 762, and 778.) [709] M. W. Hirsch and S. Smale, Differential Equations, Dynamical Systems, and Linear Algebra. San Diego: Academic Press, 1974. (Cited on pages xvii and 339.) [710] M. W. Hirsch, S. Smale, and R. L. Devaney, Differential Equations, Dynamical Systems and an Introduction to Chaos, 2nd ed. New York: Elsevier, 2003. (Cited on page xvii.) [711] O. Hirzallah, “Inequalities for Sums and Products of Operators,” Lin. Alg. Appl., Vol. 407, pp. 32–42, 2005. (Cited on page 671.) [712] O. Hirzallah and F. Kittaneh, “Matrix Young Inequalities for the Hilbert-Schmidt Norm,” Lin. Alg. Appl., Vol. 308, pp. 77–84, 2000. (Cited on page 671.) [713] O. Hirzallah and F. Kittaneh, “Commutator Inequalities for Hilbert-Schmidt Norm,” J. Math. Anal. Appl., Vol. 268, pp. 67–73, 2002. (Cited on pages 42 and 77.) [714] O. Hirzallah and F. Kittaneh, “Non-Commutative Clarkson Inequalities for Unitarily Invariant Norms,” Pac. J. Math., Vol. 202, pp. 363–369, 2002. (Cited on pages 642 and 643.) [715] O. Hirzallah and F. Kittaneh, “Norm Inequalities for Weighted Power Means of Operators,” Lin. Alg. Appl., Vol. 341, pp. 181–193, 2002. (Cited on page 552.)

BIBLIOGRAPHY

921

[716] O. Hirzallah and F. Kittaneh, “Inequalities for Sums and Direct Sums of Hilbert Space Operators,” Lin. Alg. Appl., Vol. 424, pp. 71–82, 2007. (Cited on pages 670 and 671.) [717] A. Hmamed, “A Matrix Inequality,” Int. J. Contr., Vol. 49, pp. 363–365, 1989. (Cited on page 595.) [718] J. B. Hoagg, J. Chandrasekar, and D. S. Bernstein, “On the Zeros, Initial Undershoot, and Relative Degree of Lumped-Parameter Structures,” ASME J. Dyn. Sys. Meas. Contr., Vol. 129, pp. 493–502, 2007. (Cited on page 238.) [719] K. Hoffman and R. Kunze, Linear Algebra, 2nd ed. Hall, 1971. (Cited on page xix.)

Englewood Cliffs: Prentice-

[720] L. Hogben, Ed., Handbook of Linear Algebra. Boca Raton: Chapman & Hall/CRC, 2007. (Cited on page xvii.) [721] R. R. Holmes and T. Y. Tam, “Group Representations,” in Handbook of Linear Algebra, L. Hogben, Ed. Boca Raton: Chapman & Hall/CRC, 2007, pp. 68–1– 68–11. (Cited on page 244.) [722] O. Holtz, “On Convergence of Infinite Matrix Products,” Elec. J. Lin. Alg., Vol. 7, pp. 178–181, 2000. (Cited on page 786.) [723] O. Holtz, “Hermite-Biehler, Routh-Hurwitz, and Total Positivity,” Lin. Alg. Appl., Vol. 372, pp. 105–110, 2003. (Cited on page 764.) [724] Y. Hong and R. A. Horn, “The Jordan Canonical Form of a Product of a Hermitian and a Positive Semidefinite Matrix,” Lin. Alg. Appl., Vol. 147, pp. 373–386, 1991. (Cited on page 382.) [725] Y. P. Hong and C.-T. Pan, “A Lower Bound for the Smallest Singular Value,” Lin. Alg. Appl., Vol. 172, pp. 27–32, 1992. (Cited on page 664.) [726] K. J. Horadam, Hadamard Matrices and Their Applications. Princeton: Princeton University Press, 2007. (Cited on page 391.) [727] A. Horn, “Doubly Stochastic Matrices and the Diagonal of a Rotation Matrix,” Amer. J. Math., Vol. 76, pp. 620–630, 1954. (Cited on page 562.) [728] R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge: Cambridge University Press, 1985. (Cited on pages 151, 201, 205, 233, 276, 295, 297, 298, 299, 300, 306, 307, 319, 342, 348, 374, 377, 378, 392, 394, 454, 470, 474, 490, 522, 541, 555, 558, 561, 562, 563, 564, 585, 591, 592, 595, 600, 604, 605, 607, 628, 629, 633, 636, 658, 659, and 667.) [729] R. A. Horn and C. R. Johnson, “Hadamard and Conventional Submultiplicativity for Unitarily Invariant Norms on Matrices,” Lin. Multilin. Alg., Vol. 20, pp. 91–106, 1987. (Cited on page 634.) [730] R. A. Horn and C. R. Johnson, Topics in Matrix Analysis. Cambridge: Cambridge University Press, 1991. (Cited on pages 176, 177, 209, 302, 355, 356, 373, 380, 444, 447, 457, 474, 486, 517, 546, 547, 555, 560, 615, 616, 617, 633, 634, 649, 660, 663, 664, 670, 674, 675, 703, 705, 718, 719, 722, 743, 756, 759, 761, 771, 776, and 871.) [731] R. A. Horn and R. Mathias, “An Analog of the Cauchy-Schwarz Inequality for Hadamard Products and Unitarily Invariant Norms,” SIAM J. Matrix Anal. Appl., Vol. 11, pp. 481–498, 1990. (Cited on pages 649, 661, and 675.)

922

BIBLIOGRAPHY

[732] R. A. Horn and R. Mathias, “Cauchy-Schwarz Inequalities Associated with Positive Semidefinite Matrices,” Lin. Alg. Appl., Vol. 142, pp. 63–82, 1990. (Cited on pages 593, 632, and 638.) [733] R. A. Horn and R. Mathias, “Block-Matrix Generalizations of Schur’s Basic Theorems on Hadamard Products,” Lin. Alg. Appl., Vol. 172, pp. 337–346, 1992. (Cited on page 458.) [734] R. A. Horn and I. Olkin, “When Does A∗A = B ∗B and Why Does One Want to Know?” Amer. Math. Monthly, Vol. 103, pp. 470–482, 1996. (Cited on page 348.) [735] R. A. Horn and G. G. Piepmeyer, “Two Applications of the Theory of Primary Matrix Functions,” Lin. Alg. Appl., Vol. 361, pp. 99–106, 2003. (Cited on page 705.) [736] R. A. Horn and F. Zhang, “Basic Properties of the Schur Complement,” in The Schur Complement and Its Applications, F. Zhang, Ed. New York: Springer, 2004, pp. 17–46. (Cited on pages 362, 430, 507, 539, and 542.) [737] B. G. Horne, “Lower Bounds for the Spectral Radius of a Matrix,” Lin. Alg. Appl., Vol. 263, pp. 261–273, 1997. (Cited on page 661.) [738] H.-C. Hou and H.-K. Du, “Norm Inequalities of Positive Operator Matrices,” Integ. Eqns. Operator Theory, Vol. 22, pp. 281–294, 1995. (Cited on pages 516, 573, and 668.) [739] S.-H. Hou, “A Simple Proof of the Leverrier-Faddeev Characteristic Polynomial Algorithm,” SIAM Rev., Vol. 40, pp. 706–709, 1998. (Cited on page 307.) [740] A. S. Householder, The Theory of Matrices in Numerical Analysis. New York: Blaisdell, 1964, reprinted by Dover, New York, 1975. (Cited on pages xviii, 412, 669, and 695.) [741] A. S. Householder, “Bezoutiants, Elimination and Localization,” SIAM Rev., Vol. 12, pp. 73–78, 1970. (Cited on page 279.) [742] A. S. Householder and J. A. Carpenter, “The Singular Values of Involutory and of Idempotent Matrices,” Numer. Math., Vol. 5, pp. 234–237, 1963. (Cited on page 358.) [743] R. Howe, “Very Basic Lie Theory,” Amer. Math. Monthly, Vol. 90, pp. 600–623, 1983. (Cited on pages 252 and 793.) [744] J. M. Howie, Complex Analysis.

New York: Springer, 2003. (Cited on page 84.)

[745] R. A. Howland, Intermediate Dynamics: A Linear Algebraic Approach. New York: Springer, 2006. (Cited on page xvii.) [746] P.-F. Hsieh and Y. Sibuya, Basic Theory of Ordinary Differential Equations. New York: Springer, 1999. (Cited on pages xvii and 339.) [747] G.-D. Hu and G.-H. Hu, “A Relation between the Weighted Logarithmic Norm of a Matrix and the Lyapunov Equation,” Numer. Math. BIT, Vol. 40, pp. 606–610, 2000. (Cited on page 767.) [748] S. Hu-yun, “Estimation of the Eigenvalues of AB for A > 0 and B > 0,” Lin. Alg. Appl., Vol. 73, pp. 147–150, 1986. (Cited on pages 138 and 572.) [749] R. Huang, “Some Inequalities for Hadamard Products of Matrices,” Lin. Multilin. Alg., Vol. 56, pp. 543–554, 2008. (Cited on pages 661, 675, and 676.)

BIBLIOGRAPHY

923

[750] R. Huang, “Some Inequalities for the Hadamard Product and the Fan Product of Matrices,” Lin. Alg. Appl., Vol. 428, pp. 1551–1559, 2008. (Cited on page 457.) [751] T.-Z. Huang and L. Wang, “Improving Bounds for Eigenvalues of Complex Matrices Using Traces,” Lin. Alg. Appl., Vol. 426, pp. 841–854, 2007. (Cited on pages 56 and 655.) [752] X. Huang, W. Lin, and B. Yang, “Global Finite-Time Stabilization of a Class of Uncertain Nonlinear Systems,” Automatica, Vol. 41, pp. 881–888, 2005. (Cited on page 42.) [753] P. C. Hughes, Spacecraft Attitude Dynamics. pages xvii and 741.)

New York: Wiley, 1986. (Cited on

[754] M. Huhtanen, “Real Linear Kronecker Product Operations,” Lin. Alg. Appl., Vol. 418, pp. 347–361, 2006. (Cited on page 77.) [755] Q. Hui and W. M. Haddad, “H2 Optimal Semistable Stabilisation for Linear Discrete-Time Dynamical Systems with Applications to Network Consensus,” Int. J. Contr., Vol. 82, pp. 456–469, 2009. (Cited on page 782.) [756] S. Humphries and C. Krattenthaler, “Trace Identities from Identities for Determinants,” Lin. Alg. Appl., Vol. 411, pp. 328–342, 2005. (Cited on page 143.) [757] C. H. Hung and T. L. Markham, “The Moore-Penrose Inverse of a Partitioned Matrix,” Lin. Alg. Appl., Vol. 11, pp. 73–86, 1975. (Cited on page 429.) [758] J. J. Hunter, “Generalized Inverses and Their Application to Applied Probability Problems,” Lin. Alg. Appl., Vol. 45, pp. 157–198, 1982. (Cited on page 438.) [759] D. C. Hyland and D. S. Bernstein, “The Optimal Projection Equations for FixedOrder Dynamic Compensation,” IEEE Trans. Autom. Contr., Vol. AC-29, pp. 1034–1037, 1984. (Cited on page 879.) [760] D. C. Hyland and E. G. Collins, “Block Kronecker Products and Block Norm Matrices in Large-Scale Systems Analysis,” SIAM J. Matrix Anal. Appl., Vol. 10, pp. 18–29, 1989. (Cited on page 458.) [761] N. H. Ibragimov, Elementary Lie Group Analysis and Ordinary Differential Equations. Chichester: Wiley, 1999. (Cited on page 793.) [762] Y. Ikebe and T. Inagaki, “An Elementary Approach to the Functional Calculus for Matrices,” Amer. Math. Monthly, Vol. 93, pp. 390–392, 1986. (Cited on page 705.) [763] K. D. Ikramov, “A Simple Proof of the Generalized Schur Inequality,” Lin. Alg. Appl., Vol. 199, pp. 143–149, 1994. (Cited on pages 653 and 655.) [764] V. Ionescu, C. Oar, and M. Weiss, Generalized Riccati Theory and Robust Control. Chichester: Wiley, 1999. (Cited on pages xvii and 879.) [765] I. Ipsen and C. Meyer, “The Angle between Complementary Subspaces,” Amer. Math. Monthly, Vol. 102, pp. 904–911, 1995. (Cited on pages 122, 228, 358, 366, and 416.) [766] A. Iserles, “Solving Linear Ordinary Differential Equations by Exponentials of Iterated Commutators,” Numer. Math., Vol. 45, pp. 183–199, 1984. (Cited on page 743.)

924

BIBLIOGRAPHY

[767] A. Iserles, H. Z. Munthe-Kaas, S. P. Norsett, and A. Zanna, “Lie-Group Methods,” Acta Numerica, Vol. 9, pp. 215–365, 2000. (Cited on pages 204, 739, 743, and 748.) [768] T. Ito, “Mixed Arithmetic and Geometric Means and Related Inequalities,” J. Ineq. Pure Appl. Math., Vol. 9, no. 3, pp. 1–21, 2008, Article 65. (Cited on page 46.) [769] Y. Ito, S. Hattori, and H. Maeda, “On the Decomposition of a Matrix into the Sum of Stable Matrices,” Lin. Alg. Appl., Vol. 297, pp. 177–182, 1999. (Cited on page 773.) [770] S. Izumino, H. Mori, and Y. Seo, “On Ozeki’s Inequality,” J. Ineq. Appl., Vol. 2, pp. 235–253, 1998. (Cited on pages 71 and 551.) [771] D. H. Jacobson, D. H. Martin, M. Pachter, and T. Geveci, Extensions of Linear Quadratic Control Theory. Berlin: Springer, 1980. (Cited on page 879.) [772] A. Jeffrey, Handbook of Mathematical Formulas and Integrals. demic Press, 1995. (Cited on pages 81, 82, 84, and 705.)

San Diego: Aca-

[773] C. Jeffries, V. Klee, and P. van der Driessche, “Quality Stability of Linear Systems,” Lin. Alg. Appl., Vol. 87, pp. 1–48, 1987. (Cited on page 777.) [774] A. Jennings and J. J. McKeown, Matrix Computation, 2nd ed. New York: Wiley, 1992. (Cited on page xviii.) [775] G. A. Jennings, Modern Geometry with Applications. (Cited on page 175.)

New York: Springer, 1997.

[776] S. T. Jensen and G. P. H. Styan, “Some Comments and a Bibliography on the Laguerre-Samuelson Inequality with Extensions and Applications in Statistics and Matrix Theory,” in Analytic and Geometric Inequalities and Applications, T. M. Rassias and H. M. Srivastava, Eds. Dordrecht: Kluwer, 1999, pp. 151–181. (Cited on pages 56 and 499.) [777] J. Ji, “Explicit Expressions of the Generalized Inverses and Condensed Cramer Rules,” Lin. Alg. Appl., Vol. 404, pp. 183–192, 2005. (Cited on page 141.) [778] G. Jia and J. Cao, “A New Upper Bound of the Logarithmic Mean,” J. Ineq. Pure Appl. Math., Vol. 4, no. 4, pp. 1–4, 2003, Article 80. (Cited on pages 30 and 41.) [779] C.-C. Jiang, “On Products of Two Hermitian Operators,” Lin. Alg. Appl., Vol. 430, pp. 553–557, 2009. (Cited on page 340.) [780] Y. Jiang, W. W. Hager, and J. Li, “The Geometric Mean Decomposition,” Lin. Alg. Appl., Vol. 396, pp. 373–384, 2005. (Cited on page 344.) [781] D. Jocic and F. Kittaneh, “Some Perturbation Inequalities for Self-Adjoint Operators,” J. Operator Theory, Vol. 31, pp. 3–10, 1994. (Cited on page 645.) [782] C. R. Johnson, “An Inequality for Matrices Whose Symmetric Part Is Positive Definite,” Lin. Alg. Appl., Vol. 6, pp. 13–18, 1973. (Cited on page 536.) [783] C. R. Johnson, “Inequalities for a Complex Matrix whose Real Part Is Positive Definite,” Trans. Amer. Math. Soc., Vol. 212, pp. 149–154, 1975. (Cited on page 534.) [784] C. R. Johnson, “The Inertia of a Product of Two Hermitian Matrices,” J. Math. Anal. Appl., Vol. 57, pp. 85–90, 1977. (Cited on page 336.)

BIBLIOGRAPHY

925

[785] C. R. Johnson, “Closure Properties of Certain Positivity Classes of Matrices under Various Algebraic Operations,” Lin. Alg. Appl., Vol. 97, pp. 243–247, 1987. (Cited on page 765.) [786] C. R. Johnson, “A Gersgorin-Type Lower Bound for the Smallest Singular Value,” Lin. Alg. Appl., Vol. 112, pp. 1–7, 1989. (Cited on page 664.) [787] C. R. Johnson and L. Elsner, “The Relationship between Hadamard and Conventional Multiplication for Positive Definite Matrices,” Lin. Alg. Appl., Vol. 92, pp. 231–240, 1987. (Cited on page 587.) [788] C. R. Johnson, M. Neumann, and M. J. Tsatsomeros, “Conditions for the Positivity of Determinants,” Lin. Multilin. Alg., Vol. 40, pp. 241–248, 1996. (Cited on page 238.) [789] C. R. Johnson and M. Newman, “A Surprising Determinantal Inequality for Real Matrices,” Math. Ann., Vol. 247, pp. 179–186, 1980. (Cited on page 143.) [790] C. R. Johnson and P. Nylen, “Monotonicity Properties of Norms,” Lin. Alg. Appl., Vol. 148, pp. 43–58, 1991. (Cited on page 680.) [791] C. R. Johnson, K. Okubo, and R. Beams, “Uniqueness of Matrix Square Roots,” Lin. Alg. Appl., Vol. 323, pp. 51–60, 2001. (Cited on page 381.) [792] C. R. Johnson and L. Rodman, “Convex Sets of Hermitian Matrices with Constant Inertia,” SIAM J. Alg. Disc. Meth., Vol. 6, pp. 351–359, 1985. (Cited on pages 335 and 337.) [793] C. R. Johnson and R. Schreiner, “The Relationship between AB and BA,” Amer. Math. Monthly, Vol. 103, pp. 578–582, 1996. (Cited on page 378.) [794] C. R. Johnson and H. M. Shapiro, “Mathematical Aspects of the Relative Gain Array A ◦ A−T ,” SIAM J. Alg. Disc. Meth., Vol. 7, pp. 627–644, 1986. (Cited on pages 456 and 585.) [795] C. R. Johnson and R. L. Smith, “Linear Interpolation Problems for Matrix Classes and a Transformational Characterization of M-Matrices,” Lin. Alg. Appl., Vol. 330, pp. 43–48, 2001. (Cited on pages 200, 212, and 738.) [796] C. R. Johnson and T. Szulc, “Further Lower Bounds for the Smallest Singular Value,” Lin. Alg. Appl., Vol. 272, pp. 169–179, 1998. (Cited on page 664.) [797] C. R. Johnson and F. Zhang, “An Operator Inequality and Matrix Normality,” Lin. Alg. Appl., Vol. 240, pp. 105–110, 1996. (Cited on page 552.) [798] M. Jolly, “On the Calculus of Complex Matrices,” Int. J. Contr., Vol. 61, pp. 749–755, 1995. (Cited on page 705.) [799] T. F. Jordan, Quantum Mechanics in Simple Matrix Form. New York: Wiley, 1986, reprinted by Dover, Mineola, 2005. (Cited on page 251.) [800] E. A. Jorswieck and H. Boche, “Majorization and Matrix Monotone Functions in Wireless Communications,” Found. Trends Comm. Inform. Theory, Vol. 3, pp. 553–701, 2006. (Cited on page xvii.) [801] E. A. Jorswieck and H. Boche, “Performance Analysis of MIMO Systems in Spatially Correlated Fading Using Matrix-Monotone Functions,” IEICE Trans. Fund., Vol. E89-A, pp. 1454–1472, 2006. (Cited on pages xvii and 176.)

926

BIBLIOGRAPHY

[802] A. Joseph, A. Melnikov, and R. Rentschler, Eds., Studies in Memory of Issai Schur. Boston: Birkhauser, 2002. (Cited on page 596.) [803] D. Joyner, Adventures in Group Theory. Baltimore: Johns Hopkins University Press, 2002. (Cited on pages 139, 231, 245, 246, and 252.) [804] E. I. Jury, Inners and Stability of Dynamic Systems, 2nd ed. 1982. (Cited on pages 454, 458, and 778.)

Malabar: Krieger,

[805] W. J. Kaczor and M. T. Nowak, Problems in Mathematical Analysis II. American Mathematical Society, 2001. (Cited on pages 26, 30, 32, 37, and 72.) [806] R. V. Kadison, “Order Properties of Bounded Self-Adjoint Operators,” Proc. Amer. Math. Soc., Vol. 2, pp. 505–510, 1951. (Cited on page 505.) [807] A. Kagan and P. J. Smith, “A Stronger Version of Matrix Convexity as Applied to Functions of Hermitian Matrices,” J. Ineq. Appl., Vol. 3, pp. 143–152, 1999. (Cited on page 531.) [808] J. B. Kagstrom, “Bounds and Perturbation Bounds for the Matrix Exponential,” Numer. Math. BIT, Vol. 17, pp. 39–57, 1977. (Cited on page 767.) [809] T. Kailath, Linear Systems. Englewood Cliffs: Prentice-Hall, 1980. (Cited on pages 259, 307, 331, 386, and 879.) [810] D. Kalman and J. E. White, “Polynomial Equations and Circulant Matrices,” Amer. Math. Monthly, Vol. 108, pp. 821–840, 2001. (Cited on page 390.) [811] T. R. Kane, P. W. Likins, and D. A. Levinson, Spacecraft Dynamics. McGraw-Hill, 1983. (Cited on page xvii.)

New York:

[812] I. Kaplansky, Linear Algebra and Geometry: A Second Course. New York: Chelsea, 1974, reprinted by Dover, Mineola, 2003. (Cited on pages xix and 346.) [813] N. Karcanias, “Matrix Pencil Approach to Geometric System Theory,” Proc. IEE, Vol. 126, pp. 585–590, 1979. (Cited on pages 396 and 879.) [814] S. Karlin and F. Ost, “Some Monotonicity Properties of Schur Powers of Matrices and Related Inequalities,” Lin. Alg. Appl., Vol. 68, pp. 47–65, 1985. (Cited on page 457.) [815] E. Kaszkurewicz and A. Bhaya, Matrix Diagonal Stability in Systems and Computation. Boston: Birkhauser, 2000. (Cited on page 793.) [816] M. Kato, K.-S. Saito, and T. Tamura, “Sharp Triangle Inequality and Its Reverse in Banach Space,” Math. Ineq. Appl., Vol. 10, pp. 451–460, 2007. (Cited on page 623.) [817] T. Kato, “Spectral Order and a Matrix Limit Theorem,” Lin. Multilin. Alg., Vol. 8, pp. 15–19, 1979. (Cited on page 506.) [818] T. Kato, Perturbation Theory for Linear Operators. Berlin: Springer, 1980. (Cited on pages 657 and 680.) [819] H. Katsuura, “Generalizations of the Arithmetic-Geometric Mean Inequality and a Three Dimensional Puzzle,” College Math. J., Vol. 34, pp. 280–282, 2003. (Cited on pages 51, 58, and 59.) [820] M. Kauderer, Symplectic Matrices: First Order Systems and Special Relativity. Singapore: World Scientific, 1994. (Cited on page xvii.)

927

BIBLIOGRAPHY

[821] S. Kayalar and H. L. Weinert, “Oblique Projections: Formulas, Algorithms, and Error Bounds,” Math. Contr. Sig. Sys., Vol. 2, pp. 33–45, 1989. (Cited on page 344.) [822] J. Y. Kazakia, “Orthogonal Transformation of a Trace Free Symmetric Matrix into One with Zero Diagonal Elements,” Int. J. Eng. Sci., Vol. 26, pp. 903–906, 1988. (Cited on page 341.) [823] N. D. Kazarinoff, Analytic Inequalities. New York: Holt, Rinehart, and Winston, 1961, reprinted by Dover, Mineola, 2003. (Cited on pages 40 and 600.) [824] M. G. Kendall, A Course in the Geometry of n Dimensions. 1961, reprinted by Dover, Mineola, 2004. (Cited on page 199.)

London: Griffin,

[825] C. Kenney and A. J. Laub, “Controllability and Stability Radii for Companion Form Systems,” Math. Contr. Sig. Sys., Vol. 1, pp. 239–256, 1988. (Cited on pages 357 and 780.) [826] C. Kenney and A. J. Laub, “Rational Iteration Methods for the Matrix Sign Function,” SIAM J. Matrix Anal. Appl., Vol. 12, pp. 273–291, 1991. (Cited on page 699.) [827] H. Kestelman, “Eigenvectors of a Cross-Diagonal Matrix,” Amer. Math. Monthly, Vol. 93, p. 566, 1986. (Cited on page 371.) [828] N. Keyfitz, Introduction to the Mathematics of Population. Wesley, 1968. (Cited on pages xvii and 299.)

Reading: Addison-

[829] W. Khalil and E. Dombre, Modeling, Identification, and Control of Robots. York: Taylor and Francis, 2002. (Cited on page xvii.)

New

[830] C. G. Khatri, “A Note on Idempotent Matrices,” Lin. Alg. Appl., Vol. 70, pp. 185–195, 1985. (Cited on page 408.) [831] C. G. Khatri and S. K. Mitra, “Hermitian and Nonnegative Definite Solutions of Linear Matrix Equations,” SIAM J. Appl. Math., Vol. 31, pp. 579–585, 1976. (Cited on page 421.) [832] S. Kim and Y. Lim, “A Converse Inequality of Higher Order Weighted Arithmetic and Geometric Means of Positive Definite Operators,” Lin. Alg. Appl., Vol. 426, pp. 490–496, 2007. (Cited on pages 58 and 510.) [833] C. King, “Inequalities for Trace Norms of 2 × 2 Block Matrices,” Comm. Math. Phys., Vol. 242, pp. 531–545, 2003. (Cited on page 652.) [834] C. King and M. Nathanson, “New Trace Norm Inequalities for 2 × 2 Blocks of Diagonal Matrices,” Lin. Alg. Appl., Vol. 389, pp. 77–93, 2004. (Cited on page 643.) [835] M. K. Kinyon, “Revisited: arctan 1 + arctan 2 + arctan 3 = π,” College Math. J., Vol. 37, pp. 218–219, 2006. (Cited on page 83.) [836] F. Kittaneh, “Inequalities for the Schatten p-Norm III,” Commun. Math. Phys., Vol. 104, pp. 307–310, 1986. (Cited on page 640.) [837] F. Kittaneh, “Inequalities for the Schatten p-Norm. IV,” Commun. Math. Phys., Vol. 106, pp. 581–585, 1986. (Cited on page 640.) [838] F. Kittaneh, “On Zero-Trace Matrices,” Lin. Alg. Appl., Vol. 151, pp. 119–124, 1991. (Cited on page 341.) [839] F. Kittaneh, “A Note on the Arithmetic-Geometric-Mean Inequality for Matrices,” Lin. Alg. Appl., Vol. 171, pp. 1–8, 1992. (Cited on page 646.)

928

BIBLIOGRAPHY

[840] F. Kittaneh, “Norm Inequalities for Fractional Powers of Positive Operators,” Lett. Math. Phys., Vol. 27, pp. 279–285, 1993. (Cited on pages 566 and 671.) [841] F. Kittaneh, “Singular Values of Companion Matrices and Bounds on Zeros of Polynomials,” SIAM J. Matrix Anal. Appl., Vol. 16, pp. 333–340, 1995. (Cited on pages 357 and 780.) [842] F. Kittaneh, “Norm Inequalities for Certain Operator Sums,” J. Funct. Anal., Vol. 143, pp. 337–348, 1997. (Cited on pages 568, 569, and 645.) [843] F. Kittaneh, “Some Norm Inequalities for Operators,” Canad. Math. Bull., Vol. 42, pp. 87–96, 1999. (Cited on page 646.) [844] F. Kittaneh, “Commutator Inequalities Associated with the Polar Decomposition,” Proc. Amer. Math. Soc., Vol. 130, pp. 1279–1283, 2001. (Cited on pages 637, 641, 646, and 660.) [845] F. Kittaneh, “Norm Inequalities for Sums of Positive Operators,” J. Operator Theory, Vol. 48, pp. 95–103, 2002. (Cited on pages 568, 645, 668, and 669.) [846] F. Kittaneh, “Bounds for the Zeros of Polynomials from Matrix Inequalities,” Arch. Math., Vol. 81, pp. 601–608, 2003. (Cited on pages 361 and 780.) [847] F. Kittaneh, “A Numerical Radius Inequality and an Estimate for the Numerical Radius of the Frobenius Companion Matrix,” Studia Mathematica, Vol. 158, pp. 11–17, 2003. (Cited on pages 633 and 781.) [848] F. Kittaneh, “Norm Inequalities for Sums and Differences of Positive Operators,” Lin. Alg. Appl., Vol. 383, pp. 85–91, 2004. (Cited on pages 567, 568, 569, and 660.) [849] F. Kittaneh, “Norm Inequalities for Sums of Positive Operators II,” Positivity, Vol. 10, pp. 251–260, 2006. (Cited on pages 645 and 649.) [850] F. Kittaneh, “Bounds for the Zeros of Polynomials from Matrix Inequalities–II,” Lin. Multilin. Alg., Vol. 55, pp. 147–158, 2007. (Cited on page 780.) [851] F. Kittaneh and H. Kosaki, “Inequalities for the Schatten p-Norm. V,” Publications of RIMS Kyoto University, Vol. 23, pp. 433–443, 1987. (Cited on pages 640 and 642.) [852] A.-L. Klaus and C.-K. Li, “Isometries for the Vector (p, q) Norm and the Induced (p, q) Norm,” Lin. Multilin. Alg., Vol. 38, pp. 315–332, 1995. (Cited on page 640.) [853] J. A. Knox and H. J. Brothers, “Novel Series-Based Approximations to e,” College Math. J., Vol. 30, pp. 269–275, 1999. (Cited on page 28.) [854] C. T. Koch and J. C. H. Spence, “A Useful Expansion of the Exponential of the Sum of Two Non-Commuting Matrices, One of Which is Diagonal,” J. Phys. A: Math. Gen., Vol. 36, pp. 803–816, 2003. (Cited on page 743.) [855] D. Koks, Explorations in Mathematical Physics. New York: Springer, 2006. (Cited on pages 249 and 338.) [856] J. J. Koliha, “A Simple Proof of the Product Theorem for EP Matrices,” Lin. Alg. Appl., Vol. 294, pp. 213–215, 1999. (Cited on page 418.) [857] J. J. Koliha and V. Rakocevic, “Invertibility of the Sum of Idempotents,” Lin. Multilin. Alg., Vol. 50, pp. 285–292, 2002. (Cited on page 219.)

BIBLIOGRAPHY

929

[858] J. J. Koliha and V. Rakocevic, “Invertibility of the Difference of Idempotents,” Lin. Multilin. Alg., Vol. 51, pp. 97–110, 2003. (Cited on page 221.) [859] J. J. Koliha and V. Rakocevic, “The Nullity and Rank of Linear Combinations of Idempotent Matrices,” Lin. Alg. Appl., Vol. 418, pp. 11–14, 2006. (Cited on page 219.) [860] J. J. Koliha, V. Rakocevic, and I. Straskraba, “The Difference and Sum of Projectors,” Lin. Alg. Appl., Vol. 388, pp. 279–288, 2004. (Cited on pages 219, 220, 221, and 222.) [861] N. Komaroff, “Bounds on Eigenvalues of Matrix Products with an Applicaton to the Algebraic Riccati Equation,” IEEE Trans. Autom. Contr., Vol. 35, pp. 348–350, 1990. (Cited on page 363.) [862] N. Komaroff, “Rearrangement and Matrix Product Inequalities,” Lin. Alg. Appl., Vol. 140, pp. 155–161, 1990. (Cited on pages 67 and 570.) [863] V. Komornik, “Another Short Proof of Descartes’s Rule of Signs,” Amer. Math. Monthly, Vol. 113, pp. 829–830, 2006. (Cited on page 763.) [864] R. H. Koning, H. Neudecker, and T. Wansbeek, “Block Kronecker Products and the vecb Operator,” Lin. Alg. Appl., Vol. 149, pp. 165–184, 1991. (Cited on page 458.) [865] T. Koshy, Fibonacci and Lucas Numbers with Applications. 2001. (Cited on pages xvii, 238, and 304.)

New York: Wiley,

[866] J. Kovac-Striko and K. Veselic, “Trace Minimization and Definiteness of Symmetric Pencils,” Lin. Alg. Appl., Vol. 216, pp. 139–158, 1995. (Cited on page 396.) [867] O. Krafft, “An Arithmetic-Harmonic-Mean Inequality for Nonnegative Definite Matrices,” Lin. Alg. Appl., Vol. 268, pp. 243–246, 1998. (Cited on page 582.) [868] C. Krattenthaler, “Advanced Determinant Calculus: A Complement,” Lin. Alg. Appl., Vol. 411, pp. 68–166, 2005. (Cited on page xvii.) [869] W. Kratz, Quadratic Functionals in Variational Analysis and Control Theory. New York: Wiley, 1995. (Cited on page 879.) [870] E. Kreindler and A. Jameson, “Conditions for Nonnegativeness of Partitioned Matrices,” IEEE Trans. Autom. Contr., Vol. AC-17, pp. 147–148, 1972. (Cited on page 595.) [871] R. Kress, H. L. de Vries, and R. Wegmann, “On Nonnormal Matrices,” Lin. Alg. Appl., Vol. 8, pp. 109–120, 1974. (Cited on page 654.) [872] V. Kucera, “On Nonnegative Definite Solutions to Matrix Quadratic Equations,” Automatica, Vol. 8, pp. 413–423, 1972. (Cited on page 879.) [873] A. Kufner, L. Maligranda, and L.-E. Persson, “The Prehistory of the Hardy Inequality,” Amer. Math. Monthly, Vol. 113, pp. 715–732, 2006. (Cited on pages 64 and 69.) [874] J. B. Kuipers, Quaternions and Rotation Sequences: A Primer with Applications to Orbits, Aerospace, and Virtual Reality. Princeton: Princeton University Press, 1999. (Cited on pages xvii, 249, and 251.) [875] M. R. S. Kulenovic and G. Ladas, Dynamics of Second Order Rational Difference Equations with Open Problems and Conjectures. New York: Chapman & Hall/CRC Press, 2002. (Cited on page 793.)

930

BIBLIOGRAPHY

[876] S. Kurepa, “A Note on Logarithms of Normal Operators,” Proc. Amer. Math. Soc., Vol. 13, pp. 307–311, 1962. (Cited on page 756.) [877] K. Kwakernaak and R. Sivan, Linear Optimal Control Systems. New York: Wiley, 1972. (Cited on page xvii.) [878] M. Kwapisz, “The Power of a Matrix,” SIAM Rev., Vol. 40, pp. 703–705, 1998. (Cited on page 782.) [879] M. K. Kwong, “Some Results on Matrix Monotone Functions,” Lin. Alg. Appl., Vol. 118, pp. 129–153, 1989. (Cited on pages 242 and 506.) [880] I. I. Kyrchei, “Analogs of the Adjoint Matrix for Generalized Inverses and Corresponding Cramer Rules,” Lin. Multilin. Alg., Vol. 56, pp. 453–469, 2008. (Cited on page 141.) [881] K. R. Laberteaux, “Hermitian Matrices,” Amer. Math. Monthly, Vol. 104, p. 277, 1997. (Cited on page 195.) [882] T. J. Laffey, “Products of Skew-Symmetric Matrices,” Lin. Alg. Appl., Vol. 68, pp. 249–251, 1985. (Cited on page 384.) [883] T. J. Laffey, “Products of Matrices,” in Generators and Relations in Groups and Geometries, A. Barlotti, E. W. Ellers, P. Plaumann, and K. Strambach, Eds. Dordrecht: Kluwer, 1991, pp. 95–123. (Cited on page 384.) [884] T. J. Laffey and S. Lazarus, “Two-Generated Commutative Matrix Subalgebras,” Lin. Alg. Appl., Vol. 147, pp. 249–273, 1991. (Cited on pages 347 and 348.) [885] T. J. Laffey and E. Meehan, “A Refinement of an Inequality of Johnson, Loewy and London on Nonnegative Matrices and Some Applications,” Elec. J. Lin. Alg., Vol. 3, pp. 119–128, 1998. (Cited on page 306.) [886] J. C. Lagarias and Y. Wang, “The Finiteness Conjecture for the Generalized Spectral Radius of a Set of Matrices,” Lin. Alg. Appl., Vol. 214, pp. 17–42, 1995. (Cited on page 786.) [887] S. Lakshminarayanan, S. L. Shah, and K. Nandakumar, “Cramer’s Rule for NonSquare Matrices,” Amer. Math. Monthly, Vol. 106, p. 865, 1999. (Cited on page 141.) [888] P. Lancaster, Lambda-Matrices and Vibrating Systems. Oxford: Pergamon, 1966, reprinted by Dover, Mineola, 2002. (Cited on page xvii.) [889] P. Lancaster and L. Rodman, “Solutions of the Continuous and Discrete Time Algebraic Riccati Equations: A Review,” in The Riccati Equation, S. Bittanti, J. C. Willems, and A. Laub, Eds. New York: Springer, 1991, pp. 11–51. (Cited on page 879.) [890] P. Lancaster and L. Rodman, Algebraic Riccati Equations. 1995. (Cited on pages xvii and 879.)

Oxford: Clarendon,

[891] P. Lancaster and L. Rodman, “Canonical Forms for Hermitian Matrix Pairs under Strict Equivalence and Congruence,” SIAM Rev., Vol. 47, pp. 407–443, 2005. (Cited on pages 331 and 554.) [892] P. Lancaster and M. Tismenetsky, The Theory of Matrices, 2nd ed. Orlando: Academic Press, 1985. (Cited on pages 349, 371, 447, 615, 635, 680, 867, and 868.)

931

BIBLIOGRAPHY

[893] L. Larson, Problem Solving Through Problems. New York: Springer, 1983. (Cited on pages 27, 32, 36, 38, 42, 44, 45, 46, 47, 48, 49, 53, 168, and 171.) [894] L. Larsson, L. Maligranda, J. Pecaric, and L.-E. Persson, Multiplicative Inequalities of Carlson Type and Interpolation. Singapore: World Scientific, 2006. (Cited on page 64.) [895] J. Lasenby, A. N. Lasenby, and C. J. L. Doran, “A Unified Mathematical Language for Physics and Engineering in the 21st Century,” Phil. Trans. Math. Phys. Eng. Sci., Vol. 358, pp. 21–39, 2000. (Cited on pages 204, 249, and 620.) [896] J. B. Lasserre, “A Trace Inequality for Matrix Product,” IEEE Trans. Autom. Contr., Vol. 40, pp. 1500–1501, 1995. (Cited on page 363.) [897] A. J. Laub, Matrix Analysis for Scientists and Engineers. 2004. (Cited on pages xix, 122, 331, 332, and 369.)

Philadelphia: SIAM,

[898] A. J. Laub and K. Meyer, “Canonical Forms for Symplectic and Hamiltonian Matrices,” Celestial Mechanics, Vol. 9, pp. 213–238, 1974. (Cited on page 876.) [899] C. Laurie, B. Mathes, and H. Radjavi, “Sums of Idempotents,” Lin. Alg. Appl., Vol. 208/209, pp. 175–197, 1994. (Cited on page 395.) [900] J. L. Lavoie, “A Determinantal Inequality Involving the Moore-Penrose Inverse,” Lin. Alg. Appl., Vol. 31, pp. 77–80, 1980. (Cited on page 430.) [901] C. L. Lawson, Solving Least Squares Problems. Englewood Cliffs: Prentice-Hall, 1974, reprinted by SIAM, Philadelphia, 1995. (Cited on page 438.) [902] J. D. Lawson and Y. Lim, “The Geometric Mean, Matrices, Metrics, and More,” Amer. Math. Monthly, Vol. 108, pp. 797–812, 2001. (Cited on pages 474 and 509.) [903] P. D. Lax, Linear Algebra. 502, and 657.)

New York: Wiley, 1997. (Cited on pages 173, 282,

[904] S. R. Lay, Convex Sets and Their Applications. on pages 101, 178, and 697.)

New York: Wiley, 1982. (Cited

[905] B. Leclerc, “On Identities Satisfied by Minors of a Matrix,” Adv. Math., Vol. 100, pp. 101–132, 1993. (Cited on page 140.) [906] K. J. LeCouteur, “Representation of the Function Tr(exp(A-λB)) as a Laplace Transform with Positive Weight and Some Matrix Inequalities,” J. Phys. A: Math. Gen., Vol. 13, pp. 3147–3159, 1980. (Cited on pages 571 and 748.) [907] A. Lee, “On S-Symmetric, S-Skewsymmetric, and S-Orthogonal Matrices,” Periodica Math. Hungar., Vol. 7, pp. 71–76, 1976. (Cited on page 340.) [908] A. Lee, “Centrohermitian and Skew-Centrohermitian Matrices,” Lin. Alg. Appl., Vol. 29, pp. 205–210, 1980. (Cited on pages 242 and 410.) [909] J. M. Lee and D. A. Weinberg, “A Note on Canonical Forms for Matrix Congruence,” Lin. Alg. Appl., Vol. 249, pp. 207–215, 1996. (Cited on page 396.) [910] S. H. Lehnigk, Stability Theorems for Linear Motions. Englewood Cliffs: PrenticeHall, 1966. (Cited on page 793.) [911] T.-G. Lei, C.-W. Woo, and F. Zhang, “Matrix Inequalities by Means of Embedding,” Elect. J. Lin. Alg., Vol. 11, pp. 66–77, 2004. (Cited on pages 498, 531, 532, 542, 551, 555, 556, 559, and 561.)

932

BIBLIOGRAPHY

[912] F. S. Leite, “Bounds on the Order of Generation of so(n, r) by One-Parameter Subgroups,” Rocky Mountain J. Math., Vol. 21, pp. 879–911, 1183–1188, 1991. (Cited on pages 380 and 739.) [913] E. Leonard, “The Matrix Exponential,” SIAM Rev., Vol. 38, pp. 507–512, 1996. (Cited on page 710.) [914] G. Letac, “A Matrix and Its Matrix of Reciprocals Both Positive Semi-Definite,” Amer. Math. Monthly, Vol. 82, pp. 80–81, 1975. (Cited on page 584.) [915] J. S. Lew, “The Cayley Hamilton Theorem in n Dimensions,” Z. Angew. Math. Phys., Vol. 17, pp. 650–653, 1966. (Cited on pages 283 and 284.) [916] A. S. Lewis, “Convex Analysis on the Hermitian Matrices,” SIAM J. Optim., Vol. 6, pp. 164–177, 1996. (Cited on pages 363 and 364.) [917] D. C. Lewis, “A Qualitative Analysis of S-Systems: Hopf Bifurcations,” in Canonical Nonlinear Modeling, E. O. Voit, Ed. New York: Van Nostrand Reinhold, 1991, pp. 304–344. (Cited on page 456.) [918] C.-K. Li, “Positive Semidefinite Matrices and Tensor Product,” IMAGE, Vol. 41, p. 44, 2008. (Cited on page 590.) [919] C.-K. Li and R.-C. Li, “A Note on Eigenvalues of Perturbed Hermitian Matrices,” Lin. Alg. Appl., Vol. 395, pp. 183–190, 2005. (Cited on page 296.) [920] C.-K. Li and R. Mathias, “The Determinant of the Sum of Two Matrices,” Bull. Austral. Math. Soc., Vol. 52, pp. 425–429, 1995. (Cited on pages 669 and 670.) [921] C.-K. Li and R. Mathias, “The Lidskii-Mirsky-Wielandt Theorem–Additive and Multiplicative Versions,” Numer. Math., Vol. 81, pp. 377–413, 1999. (Cited on page 657.) [922] C.-K. Li and R. Mathias, “Extremal Characterizations of the Schur Complement and Resulting Inequalities,” SIAM Rev., Vol. 42, pp. 233–246, 2000. (Cited on pages 479, 485, 586, and 596.) [923] C.-K. Li and R. Mathias, “Inequalities on Singular Values of Block Triangular Matrices,” SIAM J. Matrix Anal. Appl., Vol. 24, pp. 126–131, 2002. (Cited on page 357.) [924] C.-K. Li and S. Nataraj, “Some Matrix Techniques in Game Theory,” Math. Ineq. Appl., Vol. 3, pp. 133–141, 2000. (Cited on page xvii.) [925] C.-K. Li and E. Poon, “Additive Decomposition of Real Matrices,” Lin. Multilin. Alg., Vol. 50, pp. 321–326, 2002. (Cited on page 394.) [926] C.-K. Li and L. Rodman, “Some Extremal Problems for Positive Definite Matrices and Operators,” Lin. Alg. Appl., Vol. 140, pp. 139–154, 1990. (Cited on pages 505 and 506.) [927] C.-K. Li and H. Schneider, “Orthogonality of Matrices,” Lin. Alg. Appl., Vol. 347, pp. 115–122, 2002. (Cited on page 626.) [928] J.-L. Li and Y.-L. Li, “On the Strengthened Jordan’s Inequality,” J. Ineq. Appl., pp. 1–8, 2007, article ID 74328. (Cited on page 32.) [929] Q. Li, “Commutators of Orthogonal Projections,” Nihonkai. Math. J., Vol. 15, pp. 93–99, 2004. (Cited on page 637.)

933

BIBLIOGRAPHY

[930] X. Li and Y. Wei, “A Note on the Representations for the Drazin Inverse of 2 × 2 Block Matrices,” Lin. Alg. Appl., Vol. 423, pp. 332–338, 2007. (Cited on page 429.) [931] E. H. Lieb, “Convex Trace Functions and the Wigner-Yanase-Dyson Conjecture,” Adv. Math., Vol. 11, pp. 267–288, 1973. (Cited on pages 484, 549, 596, and 754.) [932] E. H. Lieb and M. Loss, Analysis. 2001. (Cited on page 625.)

Providence: American Mathematical Society,

[933] E. H. Lieb and M. B. Ruskai, “Some Operator Inequalities of the Schwarz Type,” Adv. Math., Vol. 12, pp. 269–273, 1974. (Cited on pages 521 and 596.) [934] E. H. Lieb and R. Seiringer, “Equivalent Forms of the Bessis-Moussa-Villani Conjecture,” J. Stat. Phys., Vol. 115, pp. 185–190, 2004. (Cited on page 530.) [935] E. H. Lieb and W. E. Thirring, “Inequalities for the Moments of the Eigenvalues of the Schrodinger Hamiltonian and Their Relation to Sobolev Inequalities,” in Studies in Mathematical Physics, E. H. Lieb, B. Simon, and A. Wightman, Eds. Princeton: Princeton University Press, 1976, pp. 269–303. (Cited on page 571.) [936] Y. Lim, “The Matrix Golden Mean and its Applications to Riccati Matrix Equations,” SIAM J. Matrix Anal. Appl., Vol. 29, pp. 54–66, 2006. (Cited on page 877.) [937] C.-S. Lin, “On Halmos’s Sharpening of Reid’s Inequality,” Math. Reports Acad. Sci. Canada, Vol. 20, pp. 62–64, 1998. (Cited on page 552.) [938] C.-S. Lin, “Inequalities of Reid Type and Furuta,” Proc. Amer. Math. Soc., Vol. 129, pp. 855–859, 2000. (Cited on page 553.) [939] C.-S. Lin, “Heinz-Kato-Furuta Inequalities with Bounds and Equality Conditions,” Math. Ineq. Appl., Vol. 5, pp. 735–743, 2002. (Cited on page 620.) [940] C.-S. Lin, “On Operator Order and Chaotic Operator Order for Two Operators,” Lin. Alg. Appl., Vol. 425, pp. 1–6, 2007. (Cited on pages 512 and 575.) [941] T.-P. Lin, “The Power Mean and the Logarithmic Mean,” Amer. Math. Monthly, Vol. 81, pp. 879–883, 1974. (Cited on page 40.) [942] W.-W. Lin, “The Computation of the Kronecker Canonical Form of an Arbitrary Symmetric Pencil,” Lin. Alg. Appl., Vol. 103, pp. 41–71, 1988. (Cited on page 396.) [943] H. Linden, “Numerical Radii of Some Companion Matrices and Bounds for the Zeros of Polynomials,” in Analytic and Geometric Inequalities and Applications, T. M. Rassias and H. M. Srivastava, Eds. Dordrecht: Kluwer, 1999, pp. 205–229. (Cited on pages 780 and 781.) [944] L. Lipsky, Queuing Theory: A Linear Algebraic Approach, 2nd ed. Springer, 2008. (Cited on page xvii.)

New York:

[945] B. Liu and H.-J. Lai, Matrices in Combinatorics and Graph Theory. Springer, 2000. (Cited on page xvii.)

New York:

[946] H. Liu and L. Zhu, “New Strengthened Carleman’s Inequality and Hardy’s Inequality,” J. Ineq. Appl., pp. 1–7, 2007, article ID 84104. (Cited on page 28.) [947] J. Liu and J. Wang, “Some Inequalities for Schur Complements,” Lin. Alg. Appl., Vol. 293, pp. 233–241, 1999. (Cited on pages 485 and 596.)

934

BIBLIOGRAPHY

[948] J. Liu and Q. Xie, “Inequalities Involving Khatri-Rao Products of Positive Semidefinite Hermitian Matrices,” Int. J. Inform. Sys. Sci., Vol. 4, pp. 30–40–135, 2008. (Cited on page 458.) [949] R.-W. Liu and R. J. Leake, “Exhaustive Equivalence Classes of Optimal Systems with Separable Controls,” SIAM Rev., Vol. 4, pp. 678–685, 1966. (Cited on page 201.) [950] S. Liu, “Matrix Results on the Khatri-Rao and Tracy-Singh Products,” Lin. Alg. Appl., Vol. 289, pp. 267–277, 1999. (Cited on pages 458, 586, 589, and 593.) [951] S. Liu, “Several Inequalities Involving Khatri-Rao Products of Positive Semidefinite Matrices,” Lin. Alg. Appl., Vol. 354, pp. 175–186, 2002. (Cited on page 458.) [952] S. Liu and H. Neudecker, “Several Matrix Kantorovich-Type Inequalities,” Math. Anal. Appl., Vol. 197, pp. 23–26, 1996. (Cited on pages 63 and 551.) [953] S. Liu and G. Trenkler, “Hadamard, Khatri-Rao, Kronecker and Other Matrix Products,” Int. J. Inform. Sys. Sci., Vol. 4, pp. 160–177, 2008. (Cited on page 458.) [954] E. Liz, “A Note on the Matrix Exponential,” SIAM Rev., Vol. 40, pp. 700–702, 1998. (Cited on page 710.) [955] R. Loewy, “An Inertia Theorem for Lyapunov’s Equation and the Dimension of a Controllability Space,” Lin. Alg. Appl., Vol. 260, pp. 1–7, 1997. (Cited on page 867.) [956] D. O. Logofet, Matrices and Graphs: Stability Problems in Mathematical Ecology. Boca Raton: CRC Press, 1993. (Cited on page xvii.) [957] R. Lopez-Valcarce and S. Dasgupta, “Some Properties of the Matrix Exponential,” IEEE Trans. Circ. Sys. Analog. Dig. Sig. Proc., Vol. 48, pp. 213–215, 2001. (Cited on page 746.) [958] M. Loss and M. B. Ruskai, Eds., Inequalities: Selecta of Elliott H. Lieb. York: Springer, 2002. (Cited on pages 549 and 754.)

New

[959] P. Lounesto, Clifford Algebras and Spinors, 2nd ed. Cambridge: Cambridge University Press, 2001. (Cited on pages 204 and 249.) [960] D. G. Luenberger, Optimization by Vector Space Methods. 1969. (Cited on page xvii.)

New York: Wiley,

[961] D. G. Luenberger, Introduction to Linear and Nonlinear Programming, 2nd ed. Reading: Addison-Wesley, 1984. (Cited on page 555.) [962] M. Lundquist and W. Barrett, “Rank Inequalities for Positive Semidefinite Matrices,” Lin. Alg. Appl., Vol. 248, pp. 91–100, 1996. (Cited on page 134.) [963] M. Lundquist and W. Barrett, “Rank Inequalities for Positive Semidefinite Matrices,” Lin. Alg. Appl., Vol. 248, pp. 91–100, 1996. (Cited on pages 488 and 541.) [964] H. Lutkepohl, Handbook of Matrices. Chichester: Wiley, 1996. (Cited on page xix.) [965] E.-C. Ma, “A Finite Series Solution of the Matrix Equation AX −XB = C,” SIAM J. Appl. Math., Vol. 14, pp. 490–495, 1966. (Cited on page 870.) [966] Y. Ma, S. Soatto, J. Kosecka, and S. S. Sastry, An Invitation to 3-D Vision. New York: Springer, 2004. (Cited on page xvii.)

935

BIBLIOGRAPHY

[967] C. C. MacDuffee, The Theory of Matrices. pages 447, 451, 454, and 458.)

New York: Chelsea, 1956. (Cited on

[968] A. G. J. Macfarlane and N. Karcanias, “Poles and Zeros of Linear Multivariable Systems: A Survey of the Algebraic, Geometric, and Complex-Variable Theory,” Int. J. Contr., Vol. 24, pp. 33–74, 1976. (Cited on page 879.) [969] D. S. Mackey, N. Mackey, and F. Tisseur, “Structured Tools for Structured Matrices,” Elec. J. Lin. Alg., Vol. 10, pp. 106–145, 2003. (Cited on page 252.) [970] J. H. Maddocks, “Restricted Quadratic Forms, Inertia Theorems, and the Schur Complement,” Lin. Alg. Appl., Vol. 108, pp. 1–36, 1988. (Cited on pages 338 and 554.) [971] J. R. Magnus, “A Representation Theorem for (tr Ap )1/p ,” Lin. Alg. Appl., Vol. 95, pp. 127–134, 1987. (Cited on pages 523, 525, and 530.) [972] J. R. Magnus, Linear Structures. 428, and 458.)

London: Griffin, 1988. (Cited on pages xvii,

[973] J. R. Magnus and H. Neudecker, Matrix Differential Calculus with Applications in Statistics and Econometrics. Chichester: Wiley, 1988. (Cited on pages xvii, 405, 421, 425, 429, 438, 458, 523, 525, 553, and 705.) [974] W. Magnus, “On the Exponential Solution of Differential Equations for a Linear Operator,” Commun. Pure Appl. Math., Vol. VII, pp. 649–673, 1954. (Cited on page 743.) [975] K. N. Majindar, “On Simultaneous Hermitian Congruence Transformations of Matrices,” Amer. Math. Monthly, Vol. 70, pp. 842–844, 1963. (Cited on page 559.) [976] L. Maligranda, “Simple Norm Inequalities,” Amer. Math. Monthly, Vol. 113, pp. 256–260, 2006. (Cited on pages 622, 623, and 624.) [977] A. N. Malyshev and M. Sadkane, “On the Stability of Large Matrices,” J. Comp. Appl. Math., Vol. 102, pp. 303–313, 1999. (Cited on page 767.) [978] O. Mangasarian, Nonlinear Programming. New York: reprinted by Krieger, Malabar, 1982. (Cited on page xvii.)

McGraw-Hill, 1969,

[979] S. M. Manjegani, “H¨ older and Young Inequalities for the Trace of Operators,” Positivity, Vol. 11, pp. 239–250, 2007. (Cited on page 525.) [980] H. B. Mann, “Quadratic Forms with Linear Constraints,” Amer. Math. Monthly, Vol. 50, pp. 430–433, 1943. (Cited on page 555.) [981] L. E. Mansfield, Linear Algebra with Geometric Application. Dekker, 1976. (Cited on page xix.)

New York: Marcel-

[982] M. Marcus, “An Eigenvalue Inequality for the Product of Normal Matrices,” Amer. Math. Monthly, Vol. 63, pp. 173–174, 1956. (Cited on page 363.) [983] M. Marcus, Finite Dimensional Multilinear Algebra, Part I. Marcel Dekker, 1973. (Cited on pages 454 and 458.) [984] M. Marcus, Finite Dimensional Multilinear Algebra, Part II. Marcel Dekker, 1975. (Cited on page 458.) [985] M. Marcus, “Two Determinant Condensation Formulas,” Lin. Multilin. Alg., Vol. 22, pp. 95–102, 1987. (Cited on page 148.)

936

BIBLIOGRAPHY

[986] M. Marcus and S. M. Katz, “Matrices of Schur Functions,” Duke Math. J., Vol. 36, pp. 343–352, 1969. (Cited on page 543.) [987] M. Marcus and N. A. Khan, “A Note on the Hadamard Product,” Canad. Math. J., Vol. 2, pp. 81–83, 1959. (Cited on pages 444, 458, and 590.) [988] M. Marcus and H. Minc, A Survey of Matrix Theory and Matrix Inequalities. Boston: Prindle, Weber, and Schmidt, 1964, reprinted by Dover, New York, 1992. (Cited on pages xix, 60, 84, 252, 354, 598, and 655.) [989] M. Marcus and W. Watkins, “Partitioned Hermitian Matrices,” Duke Math. J., Vol. 38, pp. 237–249, 1971. (Cited on pages 495, 532, 533, 543, and 564.) [990] M. Margaliot and G. Langholz, “The Routh-Hurwitz Array and Realization of Characteristic Polynomials,” IEEE Trans. Autom. Contr., Vol. 45, pp. 2424–2427, 2000. (Cited on page 772.) [991] T. L. Markham, “An Application of Theorems of Schur and Albert,” Proc. Amer. Math. Soc., Vol. 59, pp. 205–210, 1976. (Cited on page 595.) [992] T. L. Markham, “Oppenheim’s Inequality for Positive Definite Matrices,” Amer. Math. Monthly, Vol. 93, pp. 642–644, 1986. (Cited on page 588.) [993] G. Marsaglia and G. P. H. Styan, “Equalities and Inequalities for Ranks of Matrices,” Lin. Multilin. Alg., Vol. 2, pp. 269–292, 1974. (Cited on pages 131, 178, 424, and 426.) [994] J. E. Marsden and R. S. Ratiu, Introduction to Mechanics and Symmetry: A Basic Exposition of Classical Mechanical Systems, 2nd ed. New York: Springer, 1994. (Cited on page 214.) [995] J. E. Marsden and T. S. Ratiu, Introduction to Mechanics and Symmetry. York: Springer, 1994. (Cited on page xvii.)

New

[996] A. W. Marshall and I. Olkin, Inequalities: Theory of Majorization and Its Applications. New York: Academic Press, 1979. (Cited on pages 49, 66, 67, 84, 171, 172, 175, 176, 177, 178, 201, 355, 356, 364, 365, 454, 485, 486, 561, 595, 596, and 679.) [997] A. W. Marshall and I. Olkin, “Matrix Versions of the Cauchy and Kantorovich Inequalities,” Aequationes Math., Vol. 40, pp. 89–93, 1990. (Cited on page 500.) [998] A. W. Marshall, I. Olkin, and B. Arnold, Inequalities: Theory of Majorization and Its Applications, 2nd ed. New York: Springer, 2009. (Cited on page 178.) [999] K. Martensson, “On the Matrix Riccati Equation,” Inform. Sci., Vol. 3, pp. 17–49, 1971. (Cited on page 879.) [1000] A. M. Mathai, Jacobians of Matrix Transformations and Functions of Matrix Argument. Singapore: World Scientific, 1997. (Cited on page 705.) [1001] R. Mathias, “The Spectral Norm of a Nonnegative Matrix,” Lin. Alg. Appl., Vol. 131, pp. 269–284, 1990. (Cited on page 674.) [1002] R. Mathias, “Evaluating the Frechet Derivative of the Matrix Exponential,” Numer. Math., Vol. 63, pp. 213–226, 1992. (Cited on page 748.) [1003] R. Mathias, “Matrices with Positive Definite Hermitian Part: Inequalities and Linear Systems,” SIAM J. Matrix Anal. Appl., Vol. 13, pp. 640–654, 1992. (Cited on pages 484 and 563.)

937

BIBLIOGRAPHY

[1004] R. Mathias, “An Arithmetic-Geometric-Harmonic Mean Inequality Involving Hadamard Products,” Lin. Alg. Appl., Vol. 184, pp. 71–78, 1993. (Cited on page 492.) [1005] R. Mathias, “A Chain Rule for Matrix Functions and Applications,” SIAM J. Matrix Anal. Appl., Vol. 17, pp. 610–620, 1996. (Cited on page 705.) [1006] R. Mathias, “Singular Values and Singular Value Inequalities,” in Handbook of Linear Algebra, L. Hogben, Ed. Boca Raton: Chapman & Hall/CRC, 2007, pp. 17–1–17–16. (Cited on page 362.) [1007] M. Matic, C. E. M. Pearce, and J. Pecaric, “Shannon’s and Related Inequalities in Information Theory,” in Survey on Classical Inequalities, T. M. Rassias, Ed. Dordrecht: Kluwer, 2000, pp. 127–164. (Cited on pages 65 and 74.) [1008] T. Matsuda, “An Inductive Proof of a Mixed Arithmetic-Geometric Mean Inequality,” Amer. Math. Monthly, Vol. 102, pp. 634–637, 1995. (Cited on page 63.) [1009] V. Mazorchuk and S. Rabanovich, “Multicommutators and Multianticommutators of Orthogonal Projectors,” Lin. Multilin. Alg., Vol. 56, pp. 639–646, 2008. (Cited on page 637.) [1010] J. E. McCarthy, “Pick’s Theorem—What’s the Big Deal?” Amer. Math. Monthly, Vol. 110, pp. 36–45, 2003. (Cited on page 494.) [1011] J. M. McCarthy, Geometric Design of Linkages. New York: Springer, 2000. (Cited on page 793.) [1012] J. P. McCloskey, “Characterizations of r-Potent Matrices,” Math. Proc. Camb. Phil. Soc., Vol. 96, pp. 213–222, 1984. (Cited on page 231.) [1013] A. R. Meenakshi and C. Rajian, “On a Product of Positive Semidefinite Matrices,” Lin. Alg. Appl., Vol. 295, pp. 3–6, 1999. (Cited on page 583.) [1014] C. L. Mehta, “Some Inequalities Involving Traces of Operators,” J. Math. Phys., Vol. 9, pp. 693–697, 1968. (Cited on page 549.) [1015] Y. A. Melnikov, Influence Functions and Matrices. 1998. (Cited on page xvii.)

New York: Marcel Dekker,

[1016] P. R. Mercer, “The Dunkl-Williams Inequality in an Inner Product Space,” Math. Ineq. Appl., Vol. 10, pp. 447–450, 2007. (Cited on pages 621 and 623.) [1017] J. K. Merikoski, H. Sarria, and P. Tarazaga, “Bounds for Singular Values Using Traces,” Lin. Alg. Appl., Vol. 210, pp. 227–254, 1994. (Cited on page 661.) [1018] R. Merris, “Laplacian Matrices of Graphs: A Survey,” Lin. Alg. Appl., Vol. 198, pp. 143–176, 1994. (Cited on pages 301 and 550.) [1019] R. Merris, Multilinear Algebra. on pages 458 and 590.)

Amsterdam: Gordon and Breach, 1997. (Cited

[1020] R. Merris and S. Pierce, “Monotonicity of Positive Semidefinite Hermitian Matrices,” Proc. Amer. Math. Soc., Vol. 31, pp. 437–440, 1972. (Cited on pages 507 and 516.) [1021] C. D. Meyer, “The Moore-Penrose Inverse of a Bordered Matrix,” Lin. Alg. Appl., Vol. 5, pp. 375–382, 1972. (Cited on page 429.)

938

BIBLIOGRAPHY

[1022] C. D. Meyer, “Generalized Inverses and Ranks of Block Matrices,” SIAM J. Appl. Math, Vol. 25, pp. 597–602, 1973. (Cited on pages 390 and 429.) [1023] C. D. Meyer, Matrix Analysis and Applied Linear Algebra. Philadelphia: SIAM, 2000. (Cited on pages 174, 190, 191, 231, 265, 305, 409, and 784.) [1024] C. D. Meyer and N. J. Rose, “The Index and the Drazin Inverse of Block Triangular Matrices,” SIAM J. Appl. Math., Vol. 33, pp. 1–7, 1977. (Cited on pages 374, 429, and 435.) [1025] J.-M. Miao, “General Expressions for the Moore-Penrose Inverse of a 2 × 2 Block Matrix,” Lin. Alg. Appl., Vol. 151, pp. 1–15, 1991. (Cited on page 429.) [1026] L. Mihalyffy, “An Alternative Representation of the Generalized Inverse of Partitioned Matrices,” Lin. Alg. Appl., Vol. 4, pp. 95–100, 1971. (Cited on page 429.) [1027] K. S. Miller, Some Eclectic Matrix Theory. pages 276, 451, 452, and 557.)

Malabar: Krieger, 1987. (Cited on

[1028] G. A. Milliken and F. Akdeniz, “A Theorem on the Difference of the Generalized Inverses of Two Nonnegative Matrices,” Commun. Statist. Theory Methods, Vol. 6, pp. 73–79, 1977. (Cited on page 579.) [1029] G. V. Milovanovic, D. S. Mitrinovic, and T. M. Rassias, Topics in Polynomials: Extremal Problems, Inequalities, Zeros. Singapore: World Scientific, 1994. (Cited on pages 778 and 781.) [1030] G. V. Milovanovic and T. M. Rassias, “Inequalities for Polynomial Zeros,” in Survey on Classical Inequalities, T. M. Rassias, Ed. Dordrecht: Kluwer, 2000, pp. 165– 202. (Cited on pages 700, 778, 779, and 781.) [1031] N. Minamide, “An Extension of the Matrix Inversion Lemma,” SIAM J. Alg. Disc. Meth., Vol. 6, pp. 371–377, 1985. (Cited on page 413.) [1032] H. Miranda and R. C. Thompson, “A Trace Inequality with a Subtracted Term,” Lin. Alg. Appl., Vol. 185, pp. 165–172, 1993. (Cited on page 595.) [1033] L. Mirsky, An Introduction to Linear Algebra. Oxford: Clarendon, 1972, reprinted by Dover, Mineola, 1990. (Cited on pages xix and 211.) [1034] P. Misra and R. V. Patel, “A Determinant Identity and its Application in Evaluating Frequency Response Matrices,” SIAM J. Matrix Anal. Appl., Vol. 9, pp. 248–255, 1988. (Cited on page 874.) [1035] D. S. Mitrinovic, J. E. Pecaric, and A. M. Fink, Classical and New Inequalities in Analysis. Dordrecht: Kluwer, 1993. (Cited on pages 25, 77, 84, 295, 619, 620, 623, and 624.) [1036] D. S. Mitrinovic, J. E. Pecaric, and V. Volenec, Recent Advances in Geometric Inequalities. Dordrecht: Kluwer, 1989. (Cited on page 171.) [1037] B. Mityagin, “An Inequality in Linear Algebra,” SIAM Rev., Vol. 33, pp. 125–127, 1991. (Cited on pages 504 and 595.) [1038] M. Moakher, “Means and Averaging in the Group of Rotations,” SIAM J. Matrix Anal. Appl., Vol. 24, pp. 1–16, 2002. (Cited on pages 207, 394, 739, 755, and 759.) [1039] M. Moakher, “A Differential Geometric Approach to the Geometric Mean of Symmetric Positive-Definite Matrices,” SIAM J. Matrix Anal. Appl., Vol. 26, pp. 735– 747, 2005. (Cited on pages 510, 703, and 755.)

BIBLIOGRAPHY

939

[1040] C. Moler and C. F. Van Loan, “Nineteen Dubious Ways to Compute the Exponential of a Matrix, Twenty-Five Years Later,” SIAM Rev., Vol. 45, pp. 3–49, 2000. (Cited on pages 759 and 793.) [1041] B. Mond, “Generalized Inverse Extensions of Matrix Inequalities,” Lin. Alg. Appl., Vol. 2, pp. 393–399, 1969. (Cited on page 551.) [1042] B. Mond and J. E. Pecaric, “Reverse Forms of a Convex Matrix Inequality,” Lin. Alg. Appl., Vol. 220, pp. 359–364, 1995. (Cited on page 502.) [1043] B. Mond and J. E. Pecaric, “Inequalities for the Hadamard Product of Matrices,” SIAM J. Matrix Anal. Appl., Vol. 19, pp. 66–70, 1998. (Cited on page 591.) [1044] B. Mond and J. E. Pecaric, “On Inequalities Involving the Hadamard Product of Matrices,” Elec. J. Lin. Alg., Vol. 6, pp. 56–61, 2000. (Cited on pages 590 and 592.) [1045] V. V. Monov, “On the Spectrum of Convex Sets of Matrices,” IEEE Trans. Autom. Contr., Vol. 44, pp. 1009–1012, 1992. (Cited on page 293.) [1046] A. Morawiec, Orientations and Rotations: Computations in Crystallographic Textures. New York: Springer, 2003. (Cited on pages 249 and 740.) [1047] T. Moreland and S. Gudder, “Infima of Hilbert Space Effects,” Lin. Alg. Appl., Vol. 286, pp. 1–17, 1999. (Cited on page 506.) [1048] T. Mori, “Comments on ‘A Matrix Inequality Associated with Bounds on Solutions of Algebraic Riccati and Lyapunov Equation’,” IEEE Trans. Autom. Contr., Vol. 33, p. 1088, 1988. (Cited on page 595.) [1049] Y. Moschovakis, Notes on Set Theory, 2nd ed. New York: Springer, 2005. (Cited on page 15.) [1050] T. Muir, The Theory of Determinants in the Historical Order of Development. New York: Dover, 1966. (Cited on page 178.) [1051] W. W. Muir, “Inequalities Concerning the Inverses of Positive Definite Matrices,” Proc. Edinburgh Math. Soc., Vol. 19, pp. 109–113, 1974–75. (Cited on pages 485 and 596.) [1052] I. S. Murphy, “A Note on the Product of Complementary Principal Minors of a Positive Definite Matrix,” Lin. Alg. Appl., Vol. 44, pp. 169–172, 1982. (Cited on page 541.) [1053] R. M. Murray, Z. Li, and S. S. Sastry, A Mathematical Introduction to Robotic Manipulation. Boca Raton: CRC, 1994. (Cited on pages xvii and 793.) [1054] I. Najfeld and T. F. Havel, “Derivatives of the Matrix Exponential and Their Computation,” Adv. Appl. Math., Vol. 16, pp. 321–375, 1995. (Cited on page 748.) [1055] R. Nakamoto, “A Norm Inequality for Hermitian Operators,” Amer. Math. Monthly, Vol. 110, pp. 238–239, 2003. (Cited on page 762.) [1056] Y. Nakamura, “Any Hermitian Matrix is a Linear Combination of Four Projections,” Lin. Alg. Appl., Vol. 61, pp. 133–139, 1984. (Cited on page 396.) [1057] A. W. Naylor and G. R. Sell, Linear Operator Theory in Engineering and Science. New York: Springer, 1986. (Cited on pages 77, 84, 620, 623, 683, 684, 685, and 705.) [1058] T. Needham, Visual Complex Analysis. (Cited on page 84.)

Oxford: Oxford University Press, 1997.

940

BIBLIOGRAPHY

[1059] C. N. Nett and W. M. Haddad, “A System-Theoretic Appropriate Realization of the Empty Matrix Concept,” IEEE Trans. Autom. Contr., Vol. 38, pp. 771–775, 1993. (Cited on page 178.) [1060] M. G. Neubauer, “An Inequality for Positive Definite Matrices with Applications to Combinatorial Matrices,” Lin. Alg. Appl., Vol. 267, pp. 163–174, 1997. (Cited on page 540.) [1061] M. F. Neuts, Matrix-Geometric Solutions in Stochastic Models. Baltimore: Johns Hopkins University Press, 1981, reprinted by Dover, Mineola, 1994. (Cited on page xvii.) [1062] R. W. Newcomb, “On the Simultaneous Diagonalization of Two Semi-Definite Matrices,” Quart. Appl. Math., Vol. 19, pp. 144–146, 1961. (Cited on page 595.) [1063] M. Newman, “Lyapunov Revisited,” Lin. Alg. Appl., Vol. 52/53, pp. 525–528, 1983. (Cited on page 336.) [1064] M. Newman, W. So, and R. C. Thompson, “Convergence Domains for the Campbell-Baker-Hausdorff Formula,” Lin. Multilin. Alg., Vol. 24, pp. 301–310, 1989. (Cited on page 720.) [1065] D. W. Nicholson, “Eigenvalue Bounds for of AB + BA for A, B Positive Definite Matrices,” Lin. Alg. Appl., Vol. 24, pp. 173–183, 1979. (Cited on page 572.) [1066] C. Niculescu and L.-E. Persson, Convex Functions and Their Applications: A Contemporary Approach. New York: Springer, 2006. (Cited on pages 23, 24, 38, 40, 46, 59, 60, 62, 69, 143, 531, 620, 700, and 702.) [1067] C. P. Niculescu, “Convexity According to the Geometric Mean,” Math. Ineq. Appl., Vol. 3, pp. 155–167, 2000. (Cited on pages 43, 59, 171, and 781.) [1068] C. P. Niculescu and F. Popovici, “A Refinement of Popoviciu’s Inequality,” Bull. Math. Soc. Sci. Math. Roumanie, Vol. 49(97), pp. 285–290, 2006. (Cited on page 24.) [1069] M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information. Cambridge: Cambridge University Press, 2000. (Cited on page xvii.) [1070] M. Niezgoda, “Laguerre-Samuelson Type Inequalities,” Lin. Alg. Appl., Vol. 422, pp. 574–581, 2007. (Cited on pages 56 and 499.) [1071] K. Nishio, “The Structure of a Real Linear Combination of Two Projections,” Lin. Alg. Appl., Vol. 66, pp. 169–176, 1985. (Cited on page 217.) [1072] B. Noble and J. W. Daniel, Applied Linear Algebra, 3rd ed. Prentice-Hall, 1988. (Cited on pages xix and 178.)

Englewood Cliffs:

[1073] K. Nomakuchi, “On the Characterization of Generalized Inverses by Bordered Matrices,” Lin. Alg. Appl., Vol. 33, pp. 1–8, 1980. (Cited on page 429.) [1074] K. Nordstrom, “Some Further Aspects of the Lowner-Ordering Antitonicity of the Moore-Penrose Inverse,” Commun. Statist. Theory Meth., Vol. 18, pp. 4471–4489, 1989. (Cited on pages 503 and 580.) [1075] J. Nunemacher, “Which Real Matrices Have Real Logarithms?” Vol. 62, pp. 132–135, 1989. (Cited on page 793.)

Math. Mag.,

BIBLIOGRAPHY

941

[1076] H. Ogawa, “An Operator Pseudo-Inversion Lemma,” SIAM J. Appl. Math., Vol. 48, pp. 1527–1531, 1988. (Cited on page 413.) [1077] I. Olkin, “An Inequality for a Sum of Forms,” Lin. Alg. Appl., Vol. 52–53, pp. 529–532, 1983. (Cited on page 502.) [1078] J. M. Ortega, Matrix Theory, A Second Course. New York: Plenum, 1987. (Cited on page xix.) [1079] B. Ortner and A. R. Krauter, “Lower Bounds for the Determinant and the Trace of a Class of Hermitian Matrices,” Lin. Alg. Appl., Vol. 236, pp. 147–180, 1996. (Cited on page 523.) [1080] S. L. Osburn and D. S. Bernstein, “An Exact Treatment of the Achievable ClosedLoop H2 Performance of Sampled-Data Controllers: From Continuous-Time to Open-Loop,” Automatica, Vol. 31, pp. 617–620, 1995. (Cited on page 748.) [1081] A. Ostrowski and H. Schneider, “Some Theorems on the Inertia of General Matrices,” J. Math. Anal. Appl., Vol. 4, pp. 72–84, 1962. (Cited on pages 867 and 868.) [1082] A. M. Ostrowski, “On Some Metrical Properties of Operator Matrices and Matrices Partitioned into Blocks,” J. Math. Anal. Appl., Vol. 2, pp. 161–209, 1961. (Cited on page 649.) [1083] J. A. Oteo, “The Baker-Campbell-Hausdorff Formula and Nested Commutator Identities,” J. Math. Phys., Vol. 32, pp. 419–424, 1991. (Cited on page 743.) [1084] D. V. Ouellette, “Schur Complements and Statistics,” Lin. Alg. Appl., Vol. 36, pp. 187–295, 1981. (Cited on pages 515, 535, and 596.) [1085] D. A. Overdijk, “Skew-Symmetric Matrices in Classical Mechanics,” Eindhoven University, Memorandum COSOR 89-23, 1989. (Cited on page 204.) [1086] C. C. Paige and M. Saunders, “Towards a Generalized Singular Value Decomposition,” SIAM J. Numer. Anal., Vol. 18, pp. 398–405, 1981. (Cited on page 344.) [1087] C. C. Paige, G. P. H. Styan, B. Y. Wang, and F. Zhang, “Hua’s Matrix Equality and Schur Complements,” Int. J. Inform. Sys. Sci., Vol. 4, pp. 124–135, 2008. (Cited on pages 336, 520, 539, and 553.) [1088] C. C. Paige and M. Wei, “History and Generality of the CS Decomposition,” Lin. Alg. Appl., Vol. 208/209, pp. 303–326, 1994. (Cited on page 344.) [1089] H. Palanthandalam-Madapusi, D. S. Bernstein, and R. Venugopal, “Dimensional Analysis of Matrices: State-Space Models and Dimensionless Units,” IEEE Contr. Sys. Mag., Vol. 27, no. December, pp. 100–109, 2007. (Cited on pages 252 and 744.) [1090] H. Palanthandalam-Madapusi, T. Van Pelt, and D. S. Bernstein, “Matrix Pencils and Existence Conditions for Quadratic Programming with a Sign-Indefinite Quadratic Equality Constraint,” J. Global Optimization, preprint. (Cited on page 548.) [1091] H. Palanthandalam-Madapusi, T. Van Pelt, and D. S. Bernstein, “Parameter Consistency and Quadratically Constrained Errors-in-Variables Least-Squares Identification,” preprint. (Cited on page 548.) [1092] F. C. Paliogiannis, “On Commuting Operator Exponentials,” Proc. Amer. Math. Soc., Vol. 131, pp. 3777–3781, 2003. (Cited on page 747.)

942 [1093] B. P. Palka, An Introduction to Complex Function Theory. 1991. (Cited on page 84.)

BIBLIOGRAPHY

New York: Springer,

[1094] C. V. Pao, “Logarithmic Derivatives of a Square Matrix,” Lin. Alg. Appl., Vol. 6, pp. 159–164, 1973. (Cited on page 758.) [1095] J. G. Papastavridis, Tensor Calculus and Analytical Dynamics. CRC, 1998. (Cited on page xvii.)

Boca Raton:

[1096] J. G. Papastavridis, Analytical Mechanics. Oxford: Oxford University Press, 2002. (Cited on pages xvii and 496.) [1097] F. C. Park, “Computational Aspects of the Product-of-Exponentials Formula for Robot Kinematics,” IEEE Trans. Autom. Contr., Vol. 39, pp. 643–647, 1994. (Cited on page 793.) [1098] P. Park, “On the Trace Bound of a Matrix Product,” IEEE Trans. Autom. Contr., Vol. 41, pp. 1799–1802, 1996. (Cited on page 364.) [1099] D. F. Parker, Fields, Flow and Waves: An Introduction to Continuum Models. London: Springer, 2003. (Cited on page 396.) [1100] P. C. Parks, “A New Proof of the Routh-Hurwitz Stability Criterion Using the Second Method of Liapunov,” Proc. Camb. Phil. Soc., Vol. 58, pp. 694–702, 1962. (Cited on page 772.) [1101] R. V. Patel, “On Blocking Zeros in Linear Multivariable Systems,” IEEE Trans. Autom. Contr., Vol. AC-31, pp. 239–241, 1986. (Cited on page 879.) [1102] R. V. Patel and M. Toda, “Trace Inequalities Involving Hermitian Matrices,” Lin. Alg. Appl., Vol. 23, pp. 13–20, 1979. (Cited on pages 532, 533, and 759.) [1103] C. Pearcy, “A Complete Set of Unitary Invariants for Operators Generating Finite W ∗ -Algebras of Type I,” Pac. J. Math., Vol. 12, pp. 1405–1414, 1962. (Cited on page 346.) [1104] M. C. Pease, Methods of Matrix Algebra. New York: Academic Press, 1965. (Cited on page 252.) [1105] J. Pecaric and R. Rajic, “The Dunkl-Williams Inequality with n Elements in Normed Linear Spaces,” Math. Ineq. Appl., Vol. 10, pp. 461–470, 2007. (Cited on page 623.) [1106] J. E. Pecaric, S. Puntanen, and G. P. H. Styan, “Some Further Matrix Extensions of the Cauchy-Schwarz and Kantorovich Inequalities, with Some Statistical Applications,” Lin. Alg. Appl., Vol. 237/238, pp. 455–476, 1996. (Cited on page 551.) [1107] J. E. Pecaric, S. Puntanen, and G. P. H. Styan, “Some Further Matrix Extensions of the Cauchy-Schwarz and Kantorovich Inequalities, with Some Statistical Applications,” Lin. Alg. Appl., Vol. 237/238, pp. 455–476, 1996. (Cited on page 674.) [1108] S. Perlis, Theory of Matrices. Reading: Addison-Wesley, 1952, reprinted by Dover, New York, 1991. (Cited on pages 178, 255, 256, 259, 307, and 396.) [1109] T. Peter, “Maximizing the Area of a Quadrilateral,” College Math. J., Vol. 34, pp. 315–316, 2003. (Cited on page 172.) [1110] I. R. Petersen and C. V. Hollot, “A Riccati Equation Approach to the Stabilization of Uncertain Systems,” Automatica, Vol. 22, pp. 397–411, 1986. (Cited on page 553.)

943

BIBLIOGRAPHY

[1111] D. Petz and R. Temesi, “Means of Positive Numbers and Matrices,” SIAM J. Matrix Anal. Appl., Vol. 27, pp. 712–720, 2005. (Cited on pages 510 and 514.) [1112] L. A. Pipes, “Applications of Laplace Transforms of Matrix Functions,” J. Franklin Inst., Vol. 285, pp. 436–451, 1968. (Cited on pages 736 and 737.) [1113] A. O. Pittenger and M. H. Rubin, “Convexity and Separation Problem of Quantum Mechanical Density Matrices,” Lin. Alg. Appl., Vol. 346, pp. 47–71, 2002. (Cited on page xvii.) [1114] I. Polik and T. Terlaky, “A Survey of the S-Lemma,” SIAM Rev., Vol. 49, pp. 371–418, 2007. (Cited on page 548.) [1115] T. Politi, “A Formula for the Exponential of a Real Skew-Symmetric Matrix of Order 4,” Numer. Math. BIT, Vol. 41, pp. 842–845, 2001. (Cited on pages 737 and 740.) [1116] D. S. G. Pollock, “Tensor Products and Matrix Differential Calculus,” Lin. Alg. Appl., Vol. 67, pp. 169–193, 1985. (Cited on page 705.) [1117] B. T. Polyak, “Convexity of Quadratic Transformations and Its Use in Control and Optimization,” J. Optim. Theory Appl., Vol. 99, pp. 553–583, 1998. (Cited on pages 547 and 548.) [1118] B. Poonen, “A Unique (2k+1)-th Root of a Matrix,” Amer. Math. Monthly, Vol. 98, p. 763, 1991. (Cited on page 381.) [1119] B. Poonen, “Positive Deformations of the Cauchy Matrix,” Amer. Math. Monthly, Vol. 102, pp. 842–843, 1995. (Cited on page 491.) [1120] D. Popa and I. Rasa, “Inequalities Involving the Inner Product,” J. Ineq. Pure Appl. Math., Vol. 8, no. 3, pp. 1–4, 2007, Article 86. (Cited on page 620.) [1121] V. M. Popov, Hyperstability of Control Systems. on pages xvii and 793.)

Berlin: Springer, 1973. (Cited

[1122] G. J. Porter, “Linear Algebra and Affine Planar Transformations,” College Math. J., Vol. 24, pp. 47–51, 1993. (Cited on page 215.) [1123] B. H. Pourciau, “Modern Multiplier Rules,” Amer. Math. Monthly, Vol. 87, pp. 433–452, 1980. (Cited on page 697.) [1124] C. R. Pranesachar, “Ratio Circum-to-In-Radius,” Amer. Math. Monthly, Vol. 114, p. 648, 2007. (Cited on page 171.) [1125] V. V. Prasolov, Problems and Theorems in Linear Algebra. Providence: American Mathematical Society, 1994. (Cited on pages xix, 155, 162, 199, 200, 209, 225, 232, 249, 277, 279, 282, 295, 299, 340, 341, 349, 358, 364, 366, 371, 372, 374, 377, 381, 383, 392, 430, 454, 485, 521, 535, 537, 542, 558, 559, 587, 629, 645, 656, 658, 666, 703, and 724.) [1126] U. Prells, M. I. Friswell, and S. D. Garvey, “Use of Geometric Algebra: Compound Matrices and the Determinant of the Sum of Two Matrices,” Proc. Royal Soc. Lond. A, Vol. 459, pp. 273–285, 2003. (Cited on page 454.) [1127] J. S. Przemieniecki, Theory of Matrix Structural Analysis. Hill, 1968. (Cited on page xvii.)

New York: McGraw-

944

BIBLIOGRAPHY

[1128] P. J. Psarrakos, “On the mth Roots of a Complex Matrix,” Elec. J. Lin. Alg., Vol. 9, pp. 32–41, 2002. (Cited on page 381.) [1129] N. J. Pullman, Matrix Theory and Its Applications: Selected Topics. Marcel Dekker, 1976. (Cited on page 793.)

New York:

[1130] S. Puntanen and G. P. H. Styan, “Historical Introduction: Issai Schur and the Early Development of the Schur Complement,” in The Schur Complement and Its Applications, F. Zhang, Ed. New York: Springer, 2004, pp. 1–16. (Cited on page 423.) [1131] F. Qi, “Inequalities between the Sum of Squares and the Exponential of Sum of a Nonnegative Sequence,” J. Ineq. Pure Appl. Math., Vol. 8, no. 3, pp. 1–5, 2007, Article 78. (Cited on page 66.) [1132] F. Qi and L. Debnath, “Inequalities for Power-Exponential Functions,” J. Ineq. Pure. Appl. Math., Vol. 1, no. 2/15, pp. 1–5, 2000. (Cited on page 42.) [1133] C. Qian and J. Li, “Global Finite-Time Stabilization by Output Feedback for Planar Systems without Observation Linearization,” IEEE Trans. Autom. Contr., Vol. 50, pp. 885–890, 2005. (Cited on page 42.) [1134] R. X. Qian and C. L. DeMarco, “An Approach to Robust Stability of Matrix Polytopes through Copositive Homogeneous Polynomials,” IEEE Trans. Autom. Contr., Vol. 37, pp. 848–852, 1992. (Cited on page 293.) [1135] L. Qiu, B. Bernhardsson, A. Rantzer, E. J. Davison, P. M. Young, and J. C. Doyle, “A Formula for Computation of the Real Stability Radius,” Automatica, Vol. 31, pp. 879–890, 1995. (Cited on page 769.) [1136] L. Qiu and X. Zhan, “On the Span of Hadamard Products of Vectors,” Lin. Alg. Appl., Vol. 422, pp. 304–307, 2007. (Cited on page 455.) [1137] J. F. Queiro and A. Kovacec, “A Bound for the Determinant of the Sum of Two Normal Matrices,” Lin. Multilin. Alg., Vol. 33, pp. 171–173, 1993. (Cited on page 365.) [1138] V. Rabanovich, “Every Matrix Is a Linear Combination of Three Idempotents,” Lin. Alg. Appl., Vol. 390, pp. 137–143, 2004. (Cited on page 395.) [1139] H. Radjavi, “Decomposition of Matrices into Simple Involutions,” Lin. Alg. Appl., Vol. 12, pp. 247–255, 1975. (Cited on page 380.) [1140] H. Radjavi and P. Rosenthal, Simultaneous Triangularization. Springer, 2000. (Cited on page 392.)

New York:

[1141] H. Radjavi and J. P. Williams, “Products of Self-Adjoint Operators,” Michigan Math. J., Vol. 16, pp. 177–185, 1969. (Cited on pages 347, 380, and 410.) [1142] V. Rakocevic, “On the Norm of Idempotent Operators in a Hilbert Space,” Amer. Math. Monthly, Vol. 107, pp. 748–750, 2000. (Cited on pages 228 and 367.) [1143] A. C. M. Ran and R. Vreugdenhil, “Existence and Comparison Theorems for Algebraic Riccati Equations for Continuous- and Discrete-Time Systems,” Lin. Alg. Appl., Vol. 99, pp. 63–83, 1988. (Cited on page 879.) [1144] A. Rantzer, “On the Kalman-Yakubovich-Popov Lemma,” Sys. Contr. Lett., Vol. 28, pp. 7–10, 1996. (Cited on page 348.)

BIBLIOGRAPHY

945

[1145] C. R. Rao and S. K. Mitra, Generalized Inverse of Matrices and Its Applications. New York: Wiley, 1971. (Cited on pages 438, 582, and 595.) [1146] C. R. Rao and M. B. Rao, Matrix Algebra and Its Applications to Statistics and Econometrics. Singapore: World Scientific, 1998. (Cited on pages xvii, 304, and 458.) [1147] J. V. Rao, “Some More Representations for the Generalized Inverse of a Partitioned Matrix,” SIAM J. Appl. Math., Vol. 24, pp. 272–276, 1973. (Cited on page 429.) [1148] U. A. Rauhala, “Array Algebra Expansion of Matrix and Tensor Calculus: Part 1,” SIAM J. Matrix Anal. Appl., Vol. 24, pp. 490–508, 2003. (Cited on page 458.) [1149] P. A. Regalia and S. K. Mitra, “Kronecker Products, Unitary Matrices and Signal Processing Applications,” SIAM Rev., Vol. 31, pp. 586–613, 1989. (Cited on page 458.) [1150] M. W. Reinsch, “A Simple Expression for the Terms in the Baker-CampbellHausdorff Series,” J. Math. Physics, Vol. 41, pp. 2434–2442, 2000. (Cited on page 749.) [1151] T. J. Richardson and R. H. Kwong, “On Positive Definite Solutions to the Algebraic Riccati Equation,” Sys. Contr. Lett., Vol. 7, pp. 99–104, 1986. (Cited on pages 863 and 879.) [1152] D. S. Richeson, Euler’s Gem: The Polyhedron Formula and the Birth of Topology. Princeton: Princeton University Press, 2008. (Cited on page 16.) [1153] A. N. Richmond, “Expansions for the Exponential of a Sum of Matrices,” in Applications of Matrix Theory, M. J. C. Gover and S. Barnett, Eds. Oxford: Oxford University Press, 1989, pp. 283–289. (Cited on page 751.) [1154] K. S. Riedel, “A Sherman-Morrison-Woodbury Identity for Rank Augmenting Matrices with Application to Centering,” SIAM J. Matrix Anal. Appl., Vol. 13, pp. 659–662, 1992. (Cited on page 413.) [1155] J. R. Ringrose, Compact Non-Self-Adjoint Operators. Reinhold, 1971. (Cited on page 680.)

New York: Van Nostrand

[1156] R. S. Rivlin, “Further Remarks on the Stress Deformation Relations for Isotropic Materials,” J. Rational Mech. Anal., Vol. 4, pp. 681–702, 1955. (Cited on pages 283 and 284.) [1157] J. W. Robbin, Matrix Algebra Using MINImal MATlab. Wellesley: A. K. Peters, 1995. (Cited on pages 84, 125, 178, 229, 252, 307, 309, 348, 377, 378, and 703.) [1158] J. W. Robbin and D. A. Salamon, “The Exponential Vandermonde Matrix,” Lin. Alg. Appl., Vol. 317, pp. 225–226, 2000. (Cited on page 456.) [1159] D. W. Robinson, “Nullities of Submatrices of the Moore-Penrose Inverse,” Lin. Alg. Appl., Vol. 94, pp. 127–132, 1987. (Cited on page 237.) [1160] P. Robinson and A. J. Wathen, “Variational Bounds on the Entries of the Inverse of a Matrix,” IMA J. Numerical Anal., Vol. 12, pp. 463–486, 1992. (Cited on page 523.) [1161] R. T. Rockafellar, Convex Analysis. Princeton: Princeton University Press, 1990. (Cited on pages 178, 684, 693, 697, and 705.)

946

BIBLIOGRAPHY

[1162] R. T. Rockafellar and R. J. B. Wets, Variational Analysis. Berlin: Springer, 1998. (Cited on pages 693 and 705.) [1163] L. Rodman, “Products of Symmetric and Skew Symmetric Matrices,” Lin. Multilin. Alg., Vol. 43, pp. 19–34, 1997. (Cited on page 384.) [1164] G. S. Rogers, Matrix Derivatives. page 705.)

New York: Marcel Dekker, 1980. (Cited on

[1165] C. A. Rohde, “Generalized Inverses of Partitioned Matrices,” SIAM J. Appl. Math., Vol. 13, pp. 1033–1035, 1965. (Cited on page 429.) [1166] J. Rohn, “Computing the Norm ||A||∞,1 Is NP-Hard,” Lin. Multilin. Alg., Vol. 47, pp. 195–204, 2000. (Cited on page 630.) [1167] O. Rojo, “Further Bounds for the Smallest Singular Value and the Spectral Condition Number,” Computers Math. Appl., Vol. 38, pp. 215–228, 1999. (Cited on page 662.) [1168] O. Rojo, “Inequalities Involving the Mean and the Standard Deviation of Nonnegative Real Numbers,” J. Ineq. Appl., Vol. 2006, pp. 1–15, 2006, article ID 43465. (Cited on page 56.) [1169] J. Rooin, “On Ky Fan’s Inequality and its Additive Analogues,” Math. Ineq. Appl., Vol. 6, pp. 595–604, 2003. (Cited on page 60.) [1170] T. G. Room, “The Composition of Rotations in Euclidean Three-Space,” Amer. Math. Monthly, Vol. 59, pp. 688–692, 1952. (Cited on page 740.) [1171] D. J. Rose, “Matrix Identities of the Fast Fourier Transform,” Lin. Alg. Appl., Vol. 29, pp. 423–443, 1980. (Cited on page 390.) [1172] K. H. Rosen, Ed., Handbook of Discrete and Combinatorial Mathematics. Raton: CRC, 2000. (Cited on pages xvii and xix.)

Boca

[1173] H. H. Rosenbrock, State-Space and Multivariable Theory. New York: Wiley, 1970. (Cited on page 836.) [1174] M. Rosenfeld, “A Sufficient Condition for Nilpotence,” Amer. Math. Monthly, Vol. 103, pp. 907–909, 1996. (Cited on pages xvii and 233.) [1175] A. Rosoiu, “Triangle Radii Inequality,” Amer. Math. Monthly, Vol. 114, p. 640, 2007. (Cited on page 171.) [1176] W. Rossmann, Lie Groups: An Introduction Through Linear Groups. Oxford University Press, 2002. (Cited on page 252.)

Oxford:

[1177] U. G. Rothblum, “Nonnegative Matrices and Stochastic Matrices,” in Handbook of Linear Algebra, L. Hogben, Ed. Boca Raton: Chapman & Hall/CRC, 2007, pp. 9–1–9–25. (Cited on pages 298, 299, 306, and 438.) [1178] J. J. Rotman, An Introduction to the Theory of Groups, 4th ed. Springer, 1999. (Cited on pages 245 and 391.)

New York:

[1179] W. J. Rugh, Linear System Theory, 2nd ed. Upper Saddle River: Prentice Hall, 1996. (Cited on pages 274, 275, 743, 816, 829, and 879.) [1180] A. M. Russell and C. J. F. Upton, “A Class of Positive Semidefinite Matrices,” Lin. Alg. Appl., Vol. 93, pp. 121–126, 1987. (Cited on page 493.)

947

BIBLIOGRAPHY

[1181] D. L. Russell, Mathematics of Finite-Dimensional Control Systems. Marcel Dekker, 1979. (Cited on page 866.) [1182] A. Saberi, P. Sannuti, and B. M. Chen, H2 Optimal Control. Prentice-Hall, 1995. (Cited on page xvii.)

New York:

Englewood Cliffs:

[1183] N. Sabourova, Real and Complex Operator Norms. Lulea, Sweden: Lulea University of Technology, 2007, Ph.D. Dissertation, http://epubl.ltu.se/14021757/2007/09/LTU-LIC-0709-SE.pdf. (Cited on page 609.) [1184] M. K. Sain and C. B. Schrader, “The Role of Zeros in the Performance of Multiinput, Multioutput Feedback Systems,” IEEE Trans. Educ., Vol. 33, pp. 244–257, 1990. (Cited on page 879.) [1185] J. Sandor, “On Certain Inequalities for Means, II,” J. Math. Anal. Appl., Vol. 199, pp. 629–635, 1996. (Cited on page 40.) [1186] J. Sandor, “Inequalities for Generalized Convex Functions with Applications I,” Studia University Babes-Bolyai, Mathematica, Vol. XLIV, pp. 93–107, 1999. (Cited on page 24.) [1187] J. Sandor, “On Certain Limits Related to the Number e,” Libertas Mathematica, Vol. XX, pp. 155–159, 2000. (Cited on page 28.) [1188] J. Sandor, “Inequalities for Generalized Convex Functions with Applications II,” Studia University Babes-Bolyai, Mathematica, Vol. XLVI, pp. 79–91, 2001. (Cited on page 24.) [1189] J. Sandor, “On an Inequality of Ky Fan,” Int. J. Math. Educ. Sci. Tech., Vol. 32, pp. 133–160, 2001. (Cited on page 60.) [1190] J. Sandor and L. Debnath, “On Certain Inequalities Involving the Constant e and Their Applications,” J. Math. Anal. Appl., Vol. 249, pp. 569–582, 2000. (Cited on pages 28, 29, and 73.) [1191] R. A. Satnoianu, “General Power Inequalities between the Sides and the Circumscribed and Inscribed Radii Related to the Fundamental Triangle Inequality,” Math. Ineq. Appl., Vol. 5, pp. 745–751, 2002. (Cited on page 171.) [1192] D. H. Sattinger and O. L. Weaver, Lie Groups and Algebras with Applications to Physics, Geometry, and Mechanics. New York: Springer, 1986. (Cited on pages 162, 187, 720, 724, and 793.) [1193] A. H. Sayed, Fundamentals of Adaptive Filtering. New York: Wiley, 2003. (Cited on page xvii.) [1194] H. Schaub, P. Tsiotras, and J. L. Junkins, “Principal Rotation Representations of Proper N × N Orthogonal Matrices,” Int. J. Eng. Sci., Vol. 33, pp. 2277–2295, 1995. (Cited on page 738.) [1195] C. W. Scherer, “The Solution Set of the Algebraic Riccati Equation and the Algebraic Riccati Inequality,” Lin. Alg. Appl., Vol. 153, pp. 99–122, 1991. (Cited on page 879.) [1196] C. W. Scherer, “The Algebraic Riccati Equation and Inequality for Systems with Uncontrollable Modes on the Imaginary Axis,” SIAM J. Matrix Anal. Appl., Vol. 16, pp. 1308–1327, 1995. (Cited on pages 860 and 879.)

948

BIBLIOGRAPHY

[1197] C. W. Scherer, “The General Nonstrict Algebraic Riccati Inequality,” Lin. Alg. Appl., Vol. 219, pp. 1–33, 1995. (Cited on page 879.) [1198] P. Scherk, “On the Decomposition of Orthogonalities into Symmetries,” Proc. Amer. Math. Soc., Vol. 1, pp. 481–491, 1950. (Cited on page 379.) [1199] C. Schmoeger, “Remarks on Commuting Exponentials in Banach Algebras,” Proc. Amer. Math. Soc., Vol. 127, pp. 1337–1338, 1999. (Cited on page 747.) [1200] C. Schmoeger, “Remarks on Commuting Exponentials in Banach Algebras, II,” Proc. Amer. Math. Soc., Vol. 128, pp. 3405–3409, 2000. (Cited on page 747.) [1201] C. Schmoeger, “On Normal Operator Exponentials,” Proc. Amer. Math. Soc., Vol. 130, pp. 697–702, 2001. (Cited on page 747.) [1202] C. Schmoeger, “On Logarithms of Linear Operators on Hilbert Spaces,” Demonstratio Math., Vol. XXXV, pp. 375–384, 2002. (Cited on pages 714, 715, and 756.) [1203] C. Schmoeger, “On the Operator Equation eA = eB ,” Lin. Alg. Appl., Vol. 359, pp. 169–179, 2003. (Cited on page 749.) [1204] H. Schneider, “Olga Taussky-Todd’s Influence on Matrix Theory and Matrix Theorists,” Lin. Multilin. Alg., Vol. 5, pp. 197–224, 1977. (Cited on page 293.) [1205] B. Scholkopf and A. J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. Cambridge: MIT Press, 2001. (Cited on page 489.) [1206] D. Scholz and M. Weyrauch, “A Note on the Zassenhaus Product Formula,” J. Math. Phys., Vol. 47, pp. 033 505/1–7, 2006. (Cited on page 751.) [1207] J. R. Schott, “Kronecker Product Permutation Matrices and Their Application to Moment Matrices of the Normal Distribution,” J. Multivariate Analysis, Vol. 87, pp. 177–190, 2003. (Cited on page 19.) [1208] J. R. Schott, Matrix Analysis for Statistics, 2nd ed. New York: Wiley, 2005. (Cited on pages xvii, 121, 229, 236, 391, 411, 413, 414, 415, 419, 420, 428, 449, 470, 486, 536, 557, 564, 565, 569, 570, 571, 585, 586, 587, 589, 605, 656, 678, and 696.) [1209] C. B. Schrader and M. K. Sain, “Research on System Zeros: A Survey,” Int. J. Contr., Vol. 50, pp. 1407–1433, 1989. (Cited on page 879.) [1210] B. S. W. Schroder, Ordered Sets: An Introduction. (Cited on page 84.)

Boston: Birkhauser, 2003.

[1211] A. J. Schwenk, “Tight Bounds on the Spectral Radius of Asymmetric Nonnegative Matrices,” Lin. Alg. Appl., Vol. 75, pp. 257–265, 1986. (Cited on page 456.) [1212] S. R. Searle, Matrix Algebra Useful for Statistics. New York: Wiley, 1982. (Cited on page xvii.) [1213] P. Sebastian, “On the Derivatives of Matrix Powers,” SIAM J. Matrix Anal. Appl., Vol. 17, pp. 640–648, 1996. (Cited on page 705.) [1214] P. Sebastiani, “On the Derivatives of Matrix Powers,” SIAM J. Matrix Anal. Appl., Vol. 17, pp. 640–648, 1996. (Cited on page 703.) [1215] G. A. F. Seber, A Matrix Handbook for Statisticians. New York: Wiley, 2008. (Cited on pages 74, 121, 127, 151, 167, 168, 198, 224, 226, 231, 236, 241, 252, 293, 296, 408, 409, 414, 431, 535, 550, 579, 588, 627, 665, and 775.)

949

BIBLIOGRAPHY

[1216] J. M. Selig, Geometric Fundamentals of Robotics, 2nd ed. 2005. (Cited on pages xvii, 213, 249, and 252.) [1217] D. Serre, Matrices: Theory and Applications. on pages 187, 239, 283, and 427.)

New York: Springer,

New York: Springer, 2002. (Cited

[1218] J.-P. Serre and L. L. Scott, Linear Representations of Finite Groups . New York: Springer, 1996. (Cited on pages 244 and 245.) [1219] C. Shafroth, “A Generalization of the Formula for Computing the Inverse of a Matrix,” Amer. Math. Monthly, Vol. 88, pp. 614–616, 1981. (Cited on pages 151 and 207.) [1220] W. M. Shah and A. Liman, “On the Zeros of a Class of Polynomials,” Math. Ineq. Appl., Vol. 10, pp. 733–744, 2007. (Cited on page 778.) [1221] H. Shapiro, “A Survey of Canonical Forms and Invariants for Unitary Similarity,” Lin. Alg. Appl., Vol. 147, pp. 101–167, 1991. (Cited on page 346.) [1222] H. Shapiro, “Notes from Math 223: Olga Taussky Todd’s Matrix Theory Course, 1976–1977,” Mathematical Intelligencer, Vol. 19, no. 1, pp. 21–27, 1997. (Cited on page 383.) [1223] R. Shaw and F. I. Yeadon, “On (a × b) × c,” Amer. Math. Monthly, Vol. 96, pp. 623–629, 1989. (Cited on page 204.) [1224] S.-Q. Shen and T.-Z. Huang, “Several Inequalities for the Largest Singular Value and the Spectral Radius of Matrices,” Math. Ineq. Appl., Vol. 10, pp. 713–722, 2007. (Cited on pages 299, 457, 458, 675, and 676.) [1225] G. E. Shilov, Linear Algebra. Englewood Cliffs: Prentice-Hall, 1971, reprinted by Dover, New York, 1977. (Cited on page xix.) [1226] M. D. Shuster, “A Survey of Attitude Representations,” J. Astron. Sci., Vol. 41, pp. 439–517, 1993. (Cited on pages 249 and 738.) [1227] M. D. Shuster, “Attitude Analysis in Flatland: The Plane Truth,” J. Astronautical Sci., Vol. 52, pp. 195–209, 2004. (Cited on page 214.) [1228] D. D. Siljak, Large-Scale Dynamic Systems: Stability and Structure. North-Holland, 1978. (Cited on pages xvii, 252, and 774.)

New York:

[1229] F. C. Silva and R. Simoes, “On the Lyapunov and Stein Equations,” Lin. Alg. Appl., Vol. 420, pp. 329–338, 2007. (Cited on page 869.) [1230] F. C. Silva and R. Simoes, “On the Lyapunov and Stein Equations, II,” Lin. Alg. Appl., Vol. 426, pp. 305–311, 2007. (Cited on page 869.) [1231] S. F. Singer, Symmetry in Mechanics: A Gentle, Modern Introduction. Birkhauser, 2001. (Cited on page xvii.)

Boston:

[1232] R. E. Skelton, T. Iwasaki, and K. Grigoriadis, A Unified Algebraic Approach to Linear Control Design. London: Taylor and Francis, 1998. (Cited on pages xvii and 793.) [1233] D. M. Smiley and M. F. Smiley, “The Polygonal Inequalities,” Amer. Math. Monthly, Vol. 71, pp. 755–760, 1964. (Cited on page 620.) [1234] O. K. Smith, “Eigenvalues of a Symmetric 3 × 3 Matrix,” Comm. ACM, Vol. 4, p. 168, 1961. (Cited on page 289.)

950

BIBLIOGRAPHY

[1235] R. A. Smith, “Matrix Calculations for Lyapunov Quadratic Forms,” J. Diff. Eqns., Vol. 2, pp. 208–217, 1966. (Cited on page 870.) [1236] A. Smoktunowicz, “Block Matrices and Symmetric Perturbations,” Lin. Alg. Appl., Vol. 429, pp. 2628–2635, 2008. (Cited on pages 200 and 649.) [1237] R. Snieder, A Guided Tour of Mathematical Methods for the Physical Sciences, 2nd ed. Cambridge: Cambridge University Press, 2004. (Cited on page 404.) [1238] J. Snyders and M. Zakai, “On Nonnegative Solutions of the Equation AD + DA = C,” SIAM J. Appl. Math., Vol. 18, pp. 704–714, 1970. (Cited on pages 793, 818, 819, and 879.) [1239] W. So, “Equality Cases in Matrix Exponential Inequalities,” SIAM J. Matrix Anal. Appl., Vol. 13, pp. 1154–1158, 1992. (Cited on pages 194, 524, 714, 747, 749, 756, and 761.) [1240] W. So, “The High Road to an Exponential Formula,” Lin. Alg. Appl., Vol. 379, pp. 69–75, 2004. (Cited on page 755.) [1241] W. So and R. C. Thompson, “Product of Exponentials of Hermitian and Complex Symmetric Matrices,” Lin. Multilin. Alg., Vol. 29, pp. 225–233, 1991. (Cited on page 755.) [1242] W. So and R. C. Thompson, “Singular Values of Matrix Exponentials,” Lin. Multilin. Alg., Vol. 47, pp. 249–258, 2000. (Cited on pages 560 and 757.) [1243] E. D. Sontag, Mathematical Control Theory: Deterministic Finite-Dimensional Systems, 2nd ed. New York: Springer, 1998. (Cited on pages xvii and 866.) [1244] E. D. Sontag, “Passivity Gains and the ‘Secant Condition’ for Stability,” Sys. Contr. Lett., Vol. 55, pp. 177–183, 2006. (Cited on page 773.) [1245] A. R. Sourour, “A Factorization Theorem for Matrices,” Lin. Multilin. Alg., Vol. 19, pp. 141–147, 1986. (Cited on page 383.) [1246] A. R. Sourour, “Nilpotent Factorization of Matrices,” Lin. Multilin. Alg., Vol. 31, pp. 303–308, 1992. (Cited on page 383.) [1247] E. Spiegel, “Sums of Projections,” Lin. Alg. Appl., Vol. 187, pp. 239–249, 1993. (Cited on page 395.) [1248] M. Spivak, A Comprehensive Introduction to Differential Geometry, 3rd ed. Houston: Publish or Perish, 1999. (Cited on pages 249 and 684.) [1249] R. P. Stanley, Enumerative Combinatorics, Volume 1, 2nd ed. Cambridge: Cambridge University Press, 2000. (Cited on page 12.) [1250] W.-H. Steeb, Matrix Calculus and Kronecker Product with Applications and C++ Programs. Singapore: World Scientific, 2001. (Cited on page 458.) [1251] W.-H. Steeb and F. Wilhelm, “Exponential Functions of Kronecker Products and Trace Calculation,” Lin. Multilin. Alg., Vol. 9, pp. 345–346, 1981. (Cited on page 755.) [1252] J. M. Steele, The Cauchy-Schwarz Master Class. Washington, DC: Mathematical Association of America, 2004. (Cited on page 84.) [1253] R. F. Stengel, Flight Dynamics. (Cited on page xvii.)

Princeton: Princeton University Press, 2004.

951

BIBLIOGRAPHY

[1254] C. Stepniak, “Ordering of Nonnegative Definite Matrices with Application to Comparison of Linear Models,” Lin. Alg. Appl., Vol. 70, pp. 67–71, 1985. (Cited on page 579.) [1255] H. J. Stetter, Numerical Polynomial Algebra. on page xviii.)

Philadelphia: SIAM, 2004. (Cited

[1256] G. W. Stewart, Introduction to Matrix Computations. New York: Academic Press, 1973. (Cited on page xviii.) [1257] G. W. Stewart, “On the Perturbation of Pseudo-Inverses, Projections and Linear Least Squares Problems,” SIAM Rev., Vol. 19, pp. 634–662, 1977. (Cited on page 678.) [1258] G. W. Stewart, Matrix Algorithms Volume I: Basic Decompositions. Philadelphia: SIAM, 1998. (Cited on page xviii.) [1259] G. W. Stewart, “On the Adjugate Matrix,” Lin. Alg. Appl., Vol. 283, pp. 151–164, 1998. (Cited on pages xxxvi and 178.) [1260] G. W. Stewart, Matrix Algorithms Volume II: Eigensystems. Philadelphia: SIAM, 2001. (Cited on page xviii.) [1261] G. W. Stewart and J. Sun, Matrix Perturbation Theory. Boston: Academic Press, 1990. (Cited on pages 331, 332, 344, 367, 391, 392, 554, 595, 625, 634, 635, 657, 672, 673, 680, 697, and 698.) [1262] E. U. Stickel, “Fast Computation of Matrix Exponential and Logarithm,” Analysis, Vol. 5, pp. 163–173, 1985. (Cited on page 774.) [1263] E. U. Stickel, “An Algorithm for Fast High Precision Computation of Matrix Exponential and Logarithm,” Analysis, Vol. 10, pp. 85–95, 1990. (Cited on pages 774 and 793.) [1264] L. Stiller, “Multilinear Algebra and Chess Endgames,” in Games of No Chance, R. Nowakowski, Ed. Berkeley: Mathematical Sciences Research Institute, 1996, pp. 151–192. (Cited on page xvii.) [1265] J. Stoer, “On the Characterization of Least Upper Bound Norms in Matrix Space,” Numer. Math, Vol. 6, pp. 302–314, 1964. (Cited on page 636.) [1266] J. Stoer and C. Witzgall, Convexity and Optimization in Finite Dimensions I. Berlin: Springer, 1970. (Cited on pages 178 and 697.) [1267] K. B. Stolarsky, “Generalizations of the Logarithmic Mean,” J. Math. Anal. Appl., Vol. 48, pp. 87–92, 1975. (Cited on pages 40 and 41.) [1268] M. G. Stone, “A Mnemonic for Areas of Polygons,” Amer. Math. Monthly, Vol. 93, pp. 479–480, 1986. (Cited on page 173.) [1269] G. Strang, Linear Algebra and Its Applications, 3rd ed. Brace, Jovanovich, 1988. (Cited on pages xvii and xix.)

San Diego: Harcourt,

[1270] G. Strang, “The Fundamental Theorem of Linear Algebra,” Amer. Math. Monthly, Vol. 100, pp. 848–855, 1993. (Cited on page 178.) [1271] G. Strang, “Every Unit Matrix is a LULU,” Lin. Alg. Appl., Vol. 265, pp. 165–172, 1997. (Cited on page 379.)

952

BIBLIOGRAPHY

[1272] G. Strang and K. Borre, Linear Algebra, Geodesy, and GPS. Wellesley-Cambridge Press, 1997. (Cited on page xvii.) [1273] G. Strang and T. Nguyen, “The Interplay of Ranks of Submatrices,” SIAM Rev., Vol. 46, pp. 637–646, 2004. (Cited on pages 135 and 237.) [1274] S. Strelitz, “On the Routh-Hurwitz Problem,” Amer. Math. Monthly, Vol. 84, pp. 542–544, 1977. (Cited on page 765.) [1275] R. S. Strichartz, “The Campbell-Baker-Hausdorff-Dynkin Formula and Solutions of Differential Equations,” J. Funct. Anal., Vol. 72, pp. 320–345, 1987. (Cited on page 743.) [1276] T. Strom, “On Logarithmic Norms,” SIAM J. Numer. Anal., Vol. 12, pp. 741–753, 1975. (Cited on page 758.) [1277] J. Stuelpnagel, “On the Parametrization of the Three-Dimensional Rotation Group,” SIAM Rev., Vol. 6, pp. 422–430, 1964. (Cited on page 738.) [1278] G. P. H. Styan, “On Lavoie’s Determinantal Inequality,” Lin. Alg. Appl., Vol. 37, pp. 77–80, 1981. (Cited on page 430.) [1279] R. P. Sullivan, “Products of Nilpotent Matrices,” Lin. Multilin. Alg., Vol. 56, pp. 311–317, 2008. (Cited on page 383.) [1280] K. N. Swamy, “On Sylvester’s Criterion for Positive-Semidefinite Matrices,” IEEE Trans. Autom. Contr., Vol. AC-18, p. 306, 1973. (Cited on page 595.) [1281] P. Szekeres, A Course in Modern Mathematical Physics: Groups, Hilbert Space and Differential Geometry. Cambridge: Cambridge University Press, 2004. (Cited on page 249.) [1282] G. Szep, “Simultaneous Triangularization of Projector Matrices,” Acta Math. Hung., Vol. 48, pp. 285–288, 1986. (Cited on page 392.) [1283] T. Szirtes, Applied Dimensional Analysis and Modeling. New York: McGraw-Hill, 1998. (Cited on page xvii.) [1284] H. Takagi, T. Miura, T. Kanzo, and S.-E. Takahasi, “A Reconsideration of Hua’s Inequality,” J. Ineq. Appl., Vol. 2005, pp. 15–23, 2005. (Cited on pages 56 and 621.) [1285] Y. Takahashi, S.-E. Takahasi, and S. Wada, “A General Hlawka Inequality and Its Reverse Inequality,” Math. Ineq. Appl., Vol. 12, pp. 1–10, 2009. (Cited on page 620.) [1286] Y. Tao, “More Results on Singular Value Inequalities of Matrices,” Lin. Alg. Appl., Vol. 416, pp. 724–729, 2006. (Cited on page 567.) [1287] K. Tapp, Matrix Groups for Undergraduates. Providence: American Mathematical Society, 2005. (Cited on pages 214, 245, 247, 248, 249, and 700.) [1288] O. Taussky, “Positive-Definite Matrices and Their Role in the Study of the Characteristic Roots of General Matrices,” Adv. Math., Vol. 2, pp. 175–186, 1968. (Cited on page 396.) [1289] O. Taussky, “The Role of Symmetric Matrices in the Study of General Matrices,” Lin. Alg. Appl., Vol. 5, pp. 147–154, 1972. (Cited on page 396.) [1290] O. Taussky, “How I Became a Torchbearer for Matrix Theory,” Amer. Math. Monthly, Vol. 95, pp. 801–812, 1988. (Cited on page 178.)

BIBLIOGRAPHY

953

[1291] O. Taussky and J. Todd, “Another Look at a Matrix of Mark Kac,” Lin. Alg. Appl., Vol. 150, pp. 341–360, 1991. (Cited on page 360.) [1292] O. Taussky and H. Zassenhaus, “On the Similarity Transformation between a Matrix and Its Transpose,” Pac. J. Math., Vol. 9, pp. 893–896, 1959. (Cited on page 396.) [1293] W. Tempelman, “The Linear Algebra of Cross Product Operations,” J. Astron. Sci., Vol. 36, pp. 447–461, 1988. (Cited on page 204.) [1294] G. ten Have, “Structure of the nth Roots of a Matrix,” Lin. Alg. Appl., Vol. 187, pp. 59–66, 1993. (Cited on page 381.) [1295] R. E. Terrell, “Solution to ‘Exponentials of Certain Hilbert Space Operators’,” SIAM Rev., Vol. 34, pp. 498–500, 1992. (Cited on page 750.) [1296] R. E. Terrell, “Matrix Exponentials,” SIAM Rev., Vol. 38, pp. 313–314, 1996. (Cited on page 750.) [1297] R. C. Thompson, “On Matrix Commutators,” J. Washington Acad. Sci., Vol. 48, pp. 306–307, 1958. (Cited on page 199.) [1298] R. C. Thompson, “A Determinantal Inequality for Positive Definite Matrices,” Canad. Math. Bull., Vol. 4, pp. 57–62, 1961. (Cited on page 543.) [1299] R. C. Thompson, “Some Matrix Factorization Theorems,” Pac. J. Math., Vol. 33, pp. 763–810, 1970. (Cited on page 384.) [1300] R. C. Thompson, “Dissipative Matrices and Related Results,” Lin. Alg. Appl., Vol. 11, pp. 155–169, 1975. (Cited on page 534.) [1301] R. C. Thompson, “A Matrix Inequality,” Comment. Math. Univ. Carolinae, Vol. 17, pp. 393–397, 1976. (Cited on page 539.) [1302] R. C. Thompson, “Matrix Type Metric Inequalities,” Lin. Multilin. Alg., Vol. 5, pp. 303–319, 1978. (Cited on pages 75, 76, and 640.) [1303] R. C. Thompson, “Proof of a Conjectured Exponential Formula,” Lin. Multilin. Alg., Vol. 19, pp. 187–197, 1986. (Cited on page 755.) [1304] R. C. Thompson, “Special Cases of a Matrix Exponential Formula,” Lin. Alg. Appl., Vol. 107, pp. 283–292, 1988. (Cited on page 755.) [1305] R. C. Thompson, “Convergence Proof for Goldberg’s Exponential Series,” Lin. Alg. Appl., Vol. 121, pp. 3–7, 1989. (Cited on page 743.) [1306] R. C. Thompson, “Pencils of Complex and Real Symmetric and Skew Matrices,” Lin. Alg. Appl., Vol. 147, pp. 323–371, 1991. (Cited on page 396.) [1307] R. C. Thompson, “High, Low, and Quantitative Roads in Linear Algebra,” Lin. Alg. Appl., Vol. 162–164, pp. 23–64, 1992. (Cited on page 178.) [1308] Y. Tian, “Eight Expressions for Generalized Inverses of a Bordered Matrix,” Lin. Multilin. Alg., preprint. (Cited on page 429.) [1309] Y. Tian, “EP Matrices Revisited,” preprint. (Cited on pages 192 and 339.) [1310] Y. Tian, “The Moore-Penrose Inverses of m × n Block Matrices and Their Applications,” Lin. Alg. Appl., Vol. 283, pp. 35–60, 1998. (Cited on pages 422 and 429.)

954

BIBLIOGRAPHY

[1311] Y. Tian, “Matrix Representations of Octonions and Their Applications,” Adv. Appl. Clifford Algebras, Vol. 10, pp. 61–90, 2000. (Cited on page 249.) [1312] Y. Tian, “Commutativity of EP Matrices,” IMAGE, Vol. 27, pp. 25–27, 2001. (Cited on pages 417 and 418.) [1313] Y. Tian, “How to Characterize Equalities for the Moore-Penrose Inverse of a Matrix,” Kyungpook Math. J., Vol. 41, pp. 1–15, 2001. (Cited on pages 165, 219, 406, and 409.) [1314] Y. Tian, “Some Equalities for Generalized Inverses of Matrix Sums and Block Circulant Matrices,” Archivum Mathematicum (BRNO), Vol. 37, pp. 301–306, 2001. (Cited on pages 160, 166, 422, 423, and 431.) [1315] Y. Tian, “Some Inequalities for Sums of Matrices,” Scientiae Mathematicae Japonicae, Vol. 54, pp. 355–361, 2001. (Cited on page 511.) [1316] Y. Tian, “How to Express a Parallel Sum of k Matrices,” J. Math. Anal. Appl., Vol. 266, pp. 333–341, 2002. (Cited on page 582.) [1317] Y. Tian, “The Minimal Rank of the Matrix Expression A − BX − Y C,” Missouri J. Math. Sci., Vol. 14, pp. 40–48, 2002. (Cited on page 424.) [1318] Y. Tian, “Upper and Lower Bounds for Ranks of Matrix Expressions Using Generalized Inverses,” Lin. Alg. Appl., Vol. 355, pp. 187–214, 2002. (Cited on page 425.) [1319] Y. Tian, “A Range Equality for Idempotent Matrix,” IMAGE, Vol. 30, pp. 26–27, 2003. (Cited on page 216.) [1320] Y. Tian, “A Range Equality for the Difference of Orthogonal Projectors,” IMAGE, Vol. 30, p. 36, 2003. (Cited on page 415.) [1321] Y. Tian, “On Mixed-Type Reverse Order Laws for the Moore-Penrose Inverse of a Matrix Product,” Int. J. Math. Math. Sci., Vol. 58, pp. 3103–3116, 2004. (Cited on pages 415, 416, and 428.) [1322] Y. Tian, “Rank Equalities for Block Matrices and Their Moore-Penrose Inverses,” Houston J. Math., Vol. 30, pp. 483–510, 2004. (Cited on page 423.) [1323] Y. Tian, “Using Rank Formulas to Characterize Equalities for Moore-Penrose Inverses of Matrix Products,” Appl. Math. Comp., Vol. 147, pp. 581–600, 2004. (Cited on page 415.) [1324] Y. Tian, “A Range Equality for the Commutator with Two Involutory Matrices,” IMAGE, Vol. 35, pp. 32–33, 2005. (Cited on page 231.) [1325] Y. Tian, “A Range Equality for the Kronecker Product of Matrices,” IMAGE, Vol. 34, pp. 26–27, 2005. (Cited on page 447.) [1326] Y. Tian, “Similarity of Two Block Matrices,” IMAGE, Vol. 34, pp. 27–32, 2005. (Cited on page 350.) [1327] Y. Tian, “The Schur Complement in an Orthogonal Projector,” IMAGE, Vol. 35, pp. 34–36, 2005. (Cited on page 424.) [1328] Y. Tian, “Two Characterizations of an EP Matrix,” IMAGE, Vol. 34, p. 32, 2005. (Cited on page 406.) [1329] Y. Tian, “Inequalities Involving Rank and Kronecker Product,” Amer. Math. Monthly, Vol. 113, p. 851, 2006. (Cited on page 447.)

BIBLIOGRAPHY

955

[1330] Y. Tian, “On the Inverse of a Product of Orthogonal Projectors,” Amer. Math. Monthly, Vol. 113, pp. 467–468, 2006. (Cited on page 417.) [1331] Y. Tian, “Two Commutativity Equalities for the Regularized Tikhonov Inverse,” IMAGE, Vol. 41, pp. 37–39, 2008. (Cited on page 500.) [1332] Y. Tian, “Two Equalities for the Moore-Penrose Inverse of a Row Block Matrix,” IMAGE, Vol. 41, pp. 39–42, 2008. (Cited on page 428.) [1333] Y. Tian and S. Cheng, “Some Identities for Moore-Penrose Inverses of Matrix Products,” Lin. Multilin. Alg., Vol. 52, pp. 405–420, 2004. (Cited on page 414.) [1334] Y. Tian and S. Cheng, “The Moore-Penrose Inverse for Sums of Matrices under Rank Additivity Conditions,” Lin. Multilin. Alg., Vol. 53, pp. 45–65, 2005. (Cited on pages 419, 422, and 584.) [1335] Y. Tian and Y. Liu, “Extremal Ranks of Some Symmetric Matrix Expressions with Applications,” SIAM J. Matrix Anal. Appl., Vol. 28, pp. 890–905, 2006. (Cited on page 424.) [1336] Y. Tian and G. P. H. Styan, “Universal Factorization Inequalities for Quaternion Matrices and their Applications,” Math. J. Okayama University, Vol. 41, pp. 45–62, 1999. (Cited on page 251.) [1337] Y. Tian and G. P. H. Styan, “How to Establish Universal Block-Matrix Factorizations,” Elec. J. Lin. Alg., Vol. 8, pp. 115–127, 2001. (Cited on pages 146 and 251.) [1338] Y. Tian and G. P. H. Styan, “Rank Equalities for Idempotent and Involutory Matrices,” Lin. Alg. Appl., Vol. 335, pp. 101–117, 2001. (Cited on pages 219, 220, 221, 228, and 405.) [1339] Y. Tian and G. P. H. Styan, “A New Rank Formula for Idempotent Matrices with Applications,” Comment. Math. University Carolinae, Vol. 43, pp. 379–384, 2002. (Cited on pages 134 and 218.) [1340] Y. Tian and G. P. H. Styan, “When Does rank(ABC) = rank(AB) + rank(BC) − rank(B) Hold?” Int. J. Math. Educ. Sci. Tech., Vol. 33, pp. 127–137, 2002. (Cited on pages 127, 134, 220, 225, and 426.) [1341] Y. Tian and G. P. H. Styan, “Rank Equalities for Idempotent Matrices with Applications,” J. Comp. Appl. Math., Vol. 191, pp. 77–97, 2006. (Cited on pages 219, 220, and 228.) [1342] Y. Tian and Y. Takane, “Schur Complements and Banachiewicz-Schur Forms,” Elec. J. Lin. Alg., Vol. 13, pp. 405–418, 2005. (Cited on page 429.) [1343] T. Toffoli and J. Quick, “Three-Dimensional Rotations by Three Shears,” Graph. Models Image Process., Vol. 59, pp. 89–96, 1997. (Cited on page 379.) [1344] M. Tominaga, “A Generalized Reverse Inequality of the Cordes Inequality,” Math. Ineq. Appl., Vol. 11, pp. 221–227, 2008. (Cited on page 572.) [1345] A. Tonge, “Equivalence Constants for Matrix Norms: A Problem of Goldberg,” Lin. Alg. Appl., Vol. 306, pp. 1–13, 2000. (Cited on page 640.) [1346] G. E. Trapp, “Hermitian Semidefinite Matrix Means and Related Matrix Inequalities—An Introduction,” Lin. Multilin. Alg., Vol. 16, pp. 113–123, 1984. (Cited on page 509.)

956 [1347] L. N. Trefethen and D. Bau, Numerical Linear Algebra. 1997. (Cited on page xviii.)

BIBLIOGRAPHY

Philadelphia: SIAM,

[1348] L. N. Trefethen and M. Embree, Spectra and Pseudospectra. Princeton: Princeton University Press, 2005. (Cited on page 758.) [1349] D. Trenkler, G. Trenkler, C.-K. Li, and H. J. Werner, “Square Roots and Additivity,” IMAGE, Vol. 29, p. 30, 2002. (Cited on page 504.) [1350] G. Trenkler, “A Trace Inequality,” Amer. Math. Monthly, Vol. 102, pp. 362–363, 1995. (Cited on page 524.) [1351] G. Trenkler, “The Moore-Penrose Inverse and the Vector Product,” Int. J. Math. Educ. Sci. Tech., Vol. 33, pp. 431–436, 2002. (Cited on pages 204 and 427.) [1352] G. Trenkler, “A Matrix Related to an Idempotent Matrix,” IMAGE, Vol. 31, pp. 39–40, 2003. (Cited on page 409.) [1353] G. Trenkler, “On the Product of Orthogonal Projectors,” IMAGE, Vol. 31, p. 43, 2003. (Cited on pages 227 and 417.) [1354] G. Trenkler, “An Extension of Lagrange’s Identity to Matrices,” Int. J. Math. Educ. Sci. Technol., Vol. 35, pp. 245–249, 2004. (Cited on page 68.) [1355] G. Trenkler, “A Property for the Sum of a Matrix A and its Moore-Penrose Inverse A+ ,” IMAGE, Vol. 35, pp. 38–40, 2005. (Cited on page 406.) [1356] G. Trenkler, “Factorization of a Projector,” IMAGE, Vol. 34, pp. 33–35, 2005. (Cited on page 383.) [1357] G. Trenkler, “On the Product of Orthogonal Projectors,” IMAGE, Vol. 35, pp. 42–43, 2005. (Cited on page 504.) [1358] G. Trenkler, “Projectors and Similarity,” IMAGE, Vol. 34, pp. 35–36, 2005. (Cited on pages 409 and 429.) [1359] G. Trenkler, “Property of the Cross Product,” IMAGE, Vol. 34, pp. 36–37, 2005. (Cited on page 204.) [1360] G. Trenkler, “A Range Equality for Idempotent Hermitian Matrices,” IMAGE, Vol. 36, pp. 34–35, 2006. (Cited on page 226.) [1361] G. Trenkler, “Necessary and Sufficient Conditions for A + A∗ to be a Nonnegative Definite Matrix,” IMAGE, Vol. 37, pp. 28–30, 2006. (Cited on page 577.) [1362] G. Trenkler, “Another Property for the Sum of a Matrix A and its Moore-Penrose Inverse A+ ,” IMAGE, Vol. 39, pp. 23–25, 2007. (Cited on page 406.) [1363] G. Trenkler, “Characterization of EP-ness,” IMAGE, Vol. 39, pp. 30–31, 2007. (Cited on page 406.) [1364] G. Trenkler and S. Puntanen, “A Multivariate Version of Samuelson’s Inequality,” Lin. Alg. Appl., Vol. 410, pp. 143–149, 2005. (Cited on pages 56 and 499.) [1365] G. Trenkler and D. Trenkler, “The Sherman-Morrison Formula and Eigenvalues of a Special Bordered Matrix,” Acta Math. University Comenianae, Vol. 74, pp. 255–258, 2005. (Cited on page 286.)

957

BIBLIOGRAPHY

[1366] G. Trenkler and D. Trenkler, “The Vector Cross Product and 4×4 Skew-Symmetric Matrices,” in Recent Advances in Linear Models and Related Areas, Shalabh and C. Heumann, Eds. Berlin: Springer, 2008, pp. 95–104. (Cited on pages 204, 287, and 741.) [1367] G. Trenkler and H. J. Werner, “Partial Isometry and Idempotent Matrices,” IMAGE, Vol. 29, pp. 30–32, 2002. (Cited on page 224.) [1368] H. L. Trentelman, A. A. Stoorvogel, and M. L. J. Hautus, Control Theory for Linear Systems. New York: Springer, 2001. (Cited on pages xvii and 879.) [1369] P. Treuenfels, “The Matrix Equation X 2 − 2AX + B = 0,” Amer. Math. Monthly, Vol. 66, pp. 145–146, 1959. (Cited on page 876.) [1370] B. Tromborg and S. Waldenstrom, “Bounds on the Diagonal Elements of a Unitary Matrix,” Lin. Alg. Appl., Vol. 20, pp. 189–195, 1978. (Cited on page 207.) [1371] J. A. Tropp, A. C. Gilbert, and M. J. Strauss, “Algorithms for Simultaneous Sparse Approximation, Part I: Greedy Pursuit,” Sig. Proc., Vol. 86, pp. 572–588, 2006. (Cited on page 635.) [1372] M. Tsatsomeros, J. S. Maybee, D. D. Olesky, and P. Van Den Driessche, “Nullspaces of Matrices and Their Compounds,” Lin. Multilin. Alg., Vol. 34, pp. 291–300, 1993. (Cited on page 369.) [1373] N.-K. Tsing and F. Uhlig, “Inertia, Numerical Range, and Zeros of Quadratic Forms for Matrix Pencils,” SIAM J. Matrix Anal. Appl., Vol. 12, pp. 146–159, 1991. (Cited on pages 396 and 554.) [1374] P. Tsiotras, “Further Passivity Results for the Attitude Control Problem,” IEEE Trans. Autom. Contr., Vol. 43, pp. 1597–1600, 1998. (Cited on page 204.) [1375] P. Tsiotras, J. L. Junkins, and H. Schaub, “Higher-Order Cayley Transforms with Applications to Attitude Representations,” J. Guid. Contr. Dyn., Vol. 20, pp. 528– 534, 1997. (Cited on page 212.) [1376] S. H. Tung, “On Lower and Upper Bounds of the Difference between the Arithmetic and the Geometric Mean,” Math. Comput., Vol. 29, pp. 834–836, 1975. (Cited on page 56.) [1377] D. A. Turkington, Matrix Calculus and Zero-One Matrices. bridge University Press, 2002. (Cited on page 458.)

Cambridge: Cam-

[1378] R. Turkmen and D. Bozkurt, “A Note for Bounds of Norms of Hadamard Product of Matrices,” Math. Ineq. Appl., Vol. 9, pp. 211–217, 2006. (Cited on page 674.) [1379] H. W. Turnbull, The Theory of Determinants, Matrices and Invariants. Blackie, 1950. (Cited on page 178.)

London:

[1380] G. M. Tuynman, “The Derivation of the Exponential Map of Matrices,” Amer. Math. Monthly, Vol. 102, pp. 818–820, 1995. (Cited on page 748.) [1381] F. Tyan and D. S. Bernstein, “Global Stabilization of Systems Containing a Double Integrator Using a Saturated Linear Controller,” Int. J. Robust Nonlin. Contr., Vol. 9, pp. 1143–1156, 1999. (Cited on page 869.) [1382] M. Uchiyama, “Norms and Determinants of Products of Logarithmic Functions of Positive Semi-Definite Operators,” Math. Ineq. Appl., Vol. 1, pp. 279–284, 1998. (Cited on pages 574 and 761.)

958

BIBLIOGRAPHY

[1383] M. Uchiyama, “Some Exponential Operator Inequalities,” Math. Ineq. Appl., Vol. 2, pp. 469–471, 1999. (Cited on page 755.) [1384] F. E. Udwadia and R. E. Kalaba, Analytical Dynamics: A New Approach. bridge: Cambridge University Press, 1996. (Cited on page xvii.)

Cam-

[1385] F. Uhlig, “A Recurring Theorem About Pairs of Quadratic Forms and Extensions: A Survey,” Lin. Alg. Appl., Vol. 25, pp. 219–237, 1979. (Cited on pages 396, 554, 558, and 559.) [1386] F. Uhlig, “On the Matrix Equation AX = B with Applications to the Generators of a Controllability Matrix,” Lin. Alg. Appl., Vol. 85, pp. 203–209, 1987. (Cited on page 126.) [1387] F. Uhlig, “Constructive Ways for Generating (Generalized) Real Orthogonal Matrices as Products of (Generalized) Symmetries,” Lin. Alg. Appl., Vol. 332–334, pp. 459–467, 2001. (Cited on page 379.) [1388] F. A. Valentine, Convex Sets. New York: McGraw-Hill, 1964. (Cited on page 178.) [1389] M. Van Barel, V. Ptak, and Z. Vavrin, “Bezout and Hankel Matrices Associated with Row Reduced Matrix Polynomials, Barnett-Type Formulas,” Lin. Alg. Appl., Vol. 332–334, pp. 583–606, 2001. (Cited on page 279.) [1390] R. van der Merwe, Sigma-Point Kalman Filters for Probabilistic Inference in Dynamic State-Space Models. Portland: Oregon Health and Science University, 2004, Ph.D. Dissertation. (Cited on page 741.) [1391] P. Van Dooren, “The Computation of Kronecker’s Canonical Form of a Singular Pencil,” Lin. Alg. Appl., Vol. 27, pp. 103–140, 1979. (Cited on page 396.) [1392] S. Van Huffel and J. Vandewalle, The Total Least Squares Problem: Computational Aspects and Analysis. Philadelphia: SIAM, 1987. (Cited on page 679.) [1393] C. F. Van Loan, “Computing Integrals Involving the Matrix Exponential,” IEEE Trans. Autom. Contr., Vol. AC-23, pp. 395–404, 1978. (Cited on page 748.) [1394] C. F. Van Loan, “How Near Is a Stable Matrix to an Unstable Matrix,” Contemporary Math., Vol. 47, pp. 465–478, 1985. (Cited on page 769.) [1395] C. F. Van Loan, Computational Frameworks for the Fast Fourier Transform. Philadelphia: SIAM, 1992. (Cited on pages xvii and 390.) [1396] C. F. Van Loan, “The Ubiquitous Kronecker Product,” J. Comp. Appl. Math., Vol. 123, pp. 85–100, 2000. (Cited on page 458.) [1397] P. Van Overschee and B. De Moor, Subspace Identification for Linear Systems: Theory, Implementation, Applications. Dordrecht: Kluwer, 1996. (Cited on page 879.) [1398] R. Vandebril and M. Van Barel, “A Note on the Nullity Theorem,” J. Comp. Appl. Math., Vol. 189, pp. 179–190, 2006. (Cited on page 135.) [1399] R. Vandebril, M. Van Barel, and N. Mastronardi, Matrix Computations and Semiseparable Matrices. Baltimore: Johns Hopkins University Press, 2008. (Cited on pages 135, 136, and 140.) [1400] V. S. Varadarajan, Lie Groups, Lie Algebras, and Their Representations. York: Springer, 1984. (Cited on pages 252, 720, 748, 749, and 793.)

New

BIBLIOGRAPHY

959

[1401] J. M. Varah, “A Lower Bound for the Smallest Singular Value of a Matrix,” Lin. Alg. Appl., Vol. 11, pp. 3–5, 1975. (Cited on page 664.) [1402] A. I. G. Vardulakis, Linear Multivariable Control: Algebraic Analysis and Synthesis Methods. Chichester: Wiley, 1991. (Cited on pages xvii and 307.) [1403] R. S. Varga, Matrix Iterative Analysis. (Cited on pages xviii and 299.)

Englewood Cliffs: Prentice-Hall, 1962.

[1404] R. S. Varga, Gersgorin and His Circles. page 293.)

New York: Springer, 2004. (Cited on

[1405] P. R. Vein, “A Short Survey of Some Recent Applications of Determinants,” Lin. Alg. Appl., Vol. 42, pp. 287–297, 1982. (Cited on page 178.) [1406] R. Vein and P. Dale, Determinants and Their Applications in Mathematical Physics. New York: Springer, 1999. (Cited on pages xvii and 178.) [1407] D. Veljan, “The Sine Theorem and Inequalities for Volumes of Simplices and Determinants,” Lin. Alg. Appl., Vol. 219, pp. 79–91, 1995. (Cited on page 174.) [1408] D. Veljan and S. Wu, “Parametrized Klamkin’s Inequality and Improved Euler’s Inequality,” Math. Ineq. Appl., Vol. 11, pp. 729–737, 2008. (Cited on pages 46 and 171.) [1409] J. Vermeer, “Orthogonal Similarity of a Real Matrix and Its Transpose,” Lin. Alg. Appl., Vol. 428, pp. 382–392, 2008. (Cited on page 377.) [1410] K. Veselic, “On Real Eigenvalues of Real Tridiagonal Matrices,” Lin. Alg. Appl., Vol. 27, pp. 167–171, 1979. (Cited on page 359.) [1411] K. Veselic, “Estimating the Operator Exponential,” Lin. Alg. Appl., Vol. 280, pp. 241–244, 1998. (Cited on page 769.) [1412] K. Veselic, “Bounds for Exponentially Stable Semigroups,” Lin. Alg. Appl., Vol. 358, pp. 309–333, 2003. (Cited on page 769.) [1413] W. J. Vetter, “Matrix Calculus Operations and Taylor Expansions,” SIAM Rev., Vol. 15, pp. 352–369, 1973. (Cited on page 458.) [1414] M. Vidyasagar, “On Matrix Measures and Convex Liapunov Functions,” J. Math. Anal. Appl., Vol. 62, pp. 90–103, 1978. (Cited on page 758.) [1415] G. Visick, “A Weak Majorization Involving the Matrices A ◦ B and AB,” Lin. Alg. Appl., Vol. 223/224, pp. 731–744, 1995. (Cited on page 589.) [1416] G. Visick, “Majorizations of Hadamard Products of Matrix Powers,” Lin. Alg. Appl., Vol. 269, pp. 233–240, 1998. (Cited on pages 589 and 594.) [1417] G. Visick, “A Quantitative Version of the Observation that the Hadamard Product Is a Principal Submatrix of the Kronecker Product,” Lin. Alg. Appl., Vol. 304, pp. 45–68, 2000. (Cited on pages 585, 591, 592, and 593.) [1418] G. Visick, “Another Inequality for Hadamard Products,” IMAGE, Vol. 29, pp. 32–33, 2002. (Cited on page 585.) [1419] S.-W. Vong and X.-Q. Jin, “Proof of Bottcher and Wenzel’s Conjecture,” Oper. Matrices, Vol. 2, pp. 435–442, 2008. (Cited on page 641.)

960

BIBLIOGRAPHY

[1420] S. Wada, “On Some Refinement of the Cauchy-Schwarz Inequality,” Lin. Alg. Appl., Vol. 420, pp. 433–440, 2007. (Cited on pages 66 and 70.) [1421] G. G. Walter and M. Contreras, Compartmental Modeling with Networks. Boston: Birkhauser, 1999. (Cited on pages xvii and 777.) [1422] B. Wang and F. Zhang, “Some Inequalities for the Eigenvalues of the Product of Positive Semidefinite Hermitian Matrices,” Lin. Alg. Appl., Vol. 160, pp. 113–118, 1992. (Cited on pages 570, 571, and 672.) [1423] B.-Y. Wang and M.-P. Gong, “Some Eigenvalue Inequalities for Positive Semidefinite Matrix Power Products,” Lin. Alg. Appl., Vol. 184, pp. 249–260, 1993. (Cited on page 571.) [1424] B.-Y. Wang, B.-Y. Xi, and F. Zhang, “Some Inequalities for Sum and Product of Positive Semidefinite Matrices,” Lin. Alg. Appl., Vol. 293, pp. 39–49, 1999. (Cited on pages 580 and 673.) [1425] B.-Y. Wang and F. Zhang, “A Trace Inequality for Unitary Matrices,” Amer. Math. Monthly, Vol. 101, pp. 453–455, 1994. (Cited on page 209.) [1426] B.-Y. Wang and F. Zhang, “Trace and Eigenvalue Inequalities for Ordinary and Hadamard Products of Positive Semidefinite Hermitian Matrices,” SIAM J. Matrix Anal. Appl., Vol. 16, pp. 1173–1183, 1995. (Cited on pages 571 and 587.) [1427] B.-Y. Wang and F. Zhang, “Schur Complements and Matrix Inequalities of Hadamard Products,” Lin. Multilin. Alg., Vol. 43, pp. 315–326, 1997. (Cited on page 595.) [1428] C.-L. Wang, “On Development of Inverses of the Cauchy and H¨ older Inequalities,” SIAM Rev., Vol. 21, pp. 550–557, 1979. (Cited on pages 63 and 71.) [1429] D. Wang, “The Polar Decomposition and a Matrix Inequality,” Amer. Math. Monthly, Vol. 96, pp. 517–519, 1989. (Cited on page 476.) [1430] G. Wang, Y. Wei, and S. Qiao, Generalized Inverses: Theory and Computations. Beijing/New York: Science Press, 2004. (Cited on pages 141, 219, 412, and 438.) [1431] J.-H. Wang, “The Length Problem for a Sum of Idempotents,” Lin. Alg. Appl., Vol. 215, pp. 135–159, 1995. (Cited on page 395.) [1432] L.-C. Wang and C.-L. Li, “On Some New Mean Value Inequalities,” J. Ineq. Pure Appl. Math., Vol. 8, no. 3, pp. 1–8, 2007, Article 87. (Cited on page 63.) [1433] Q.-G. Wang, “Necessary and Sufficient Conditions for Stability of a Matrix Polytope with Normal Vertex Matrices,” Automatica, Vol. 27, pp. 887–888, 1991. (Cited on page 295.) [1434] X. Wang, “A Simple Proof of Descartes’s Rule of Signs,” Amer. Math. Monthly, Vol. 111, pp. 525–526, 2004. (Cited on page 763.) [1435] Y. W. Wang and D. S. Bernstein, “Controller Design with Regional Pole Constraints: Hyperbolic and Horizontal Strip Regions,” AIAA J. Guid. Contr. Dyn., Vol. 16, pp. 784–787, 1993. (Cited on page 871.) [1436] Y. W. Wang and D. S. Bernstein, “L2 Controller Synthesis with L∞ -Bounded Closed-Loop Impulse Response,” Int. J. Contr., Vol. 60, pp. 1295–1306, 1994. (Cited on page 756.)

961

BIBLIOGRAPHY

[1437] A. J. B. Ward and F. Gerrish, “A Constructive Proof by Elementary Transformations of Roth’s Removal Theorems in the Theory of Matrix Equations,” Int. J. Math. Educ. Sci. Tech., Vol. 31, pp. 425–429, 2000. (Cited on page 349.) [1438] J. Warga, Optimal Control of Differential and Functional Equations. Academic Press, 1972. (Cited on pages 685 and 686.)

New York:

[1439] W. E. Waterhouse, “A Determinant Identity with Matrix Entries,” Amer. Math. Monthly, Vol. 97, pp. 249–250, 1990. (Cited on pages 148 and 149.) [1440] W. Watkins, “Convex Matrix Functions,” Proc. Amer. Math. Soc., Vol. 44, pp. 31–34, 1974. (Cited on pages 537 and 590.) [1441] W. Watkins, “Generating Functions,” Coll. Math. J., Vol. 18, pp. 195–211, 1987. (Cited on page 304.) [1442] W. Watkins, “A Determinantal Inequality for Correlation Matrices,” Lin. Alg. Appl., Vol. 104, pp. 59–63, 1988. (Cited on page 586.) [1443] G. S. Watson, G. Alpargu, and G. P. H. Styan, “Some Comments on Six Inequalities Associated with the Inefficiency of Ordinary Least Squares with One Regressor,” Lin. Alg. Appl., Vol. 264, pp. 13–54, 1997. (Cited on pages 63 and 72.) [1444] J. R. Weaver, “Centrosymmetric (Cross-Symmetric) Matrices, Their Basic Properties, Eigenvalues, and Eigenvectors,” Amer. Math. Monthly, Vol. 92, pp. 711–717, 1985. (Cited on pages 242 and 342.) [1445] J. H. Webb, “An Inequality in Algebra and Geometry,” College Math. J., Vol. 36, p. 164, 2005. (Cited on page 49.) [1446] R. Webster, Convexity. page 178.)

Oxford: Oxford University Press, 1994.

(Cited on

[1447] E. Wegert and L. N. Trefethen, “From the Buffon Needle Problem to the Kreiss Matrix Theorem,” Amer. Math. Monthly, Vol. 101, pp. 132–139, 1994. (Cited on page 786.) [1448] J. Wei and E. Norman, “Lie Algebraic Solution of Linear Differential Equations,” J. Math. Phys., Vol. 4, pp. 575–581, 1963. (Cited on page 743.) [1449] J. Wei and E. Norman, “On Global Representations of the Solutions of Linear Differential Equations as a Product of Exponentials,” Proc. Amer. Math. Soc., Vol. 15, pp. 327–334, 1964. (Cited on page 743.) [1450] M. Wei, “Reverse Order Laws for Generalized Inverses of Multiple Matrix Products,” Lin. Alg. Appl., Vol. 293, pp. 273–288, 1999. (Cited on page 415.) [1451] Y. Wei, “A Characterization and Representation of the Drazin Inverse,” SIAM J. Matrix Anal. Appl., Vol. 17, pp. 744–747, 1996. (Cited on page 431.) [1452] Y. Wei, “Expressions for the Drazin Inverse of a 2 × 2 Block Matrix,” Lin. Multilin. Alg., Vol. 45, pp. 131–146, 1998. (Cited on page 429.) [1453] G. H. Weiss and A. A. Maradudin, “The Baker-Hausdorff Formula and a Problem in Crystal Physics,” J. Math. Phys., Vol. 3, pp. 771–777, 1962. (Cited on page 743.) [1454] E. M. E. Wermuth, “Two Remarks on Matrix Exponentials,” Lin. Alg. Appl., Vol. 117, pp. 127–132, 1989. (Cited on pages 747 and 762.)

962

BIBLIOGRAPHY

[1455] E. M. E. Wermuth, “A Remark on Commuting Operator Exponentials,” Proc. Amer. Math. Soc., Vol. 125, pp. 1685–1688, 1997. (Cited on page 747.) [1456] H.-J. Werner, “On the Matrix Monotonicity of Generalized Inversion,” Lin. Alg. Appl., Vol. 27, pp. 141–145, 1979. (Cited on page 579.) [1457] H. J. Werner, “On the Product of Orthogonal Projectors,” IMAGE, Vol. 32, pp. 30–37, 2004. (Cited on pages 227, 409, and 416.) [1458] H. J. Werner, “Solution 34-4.2 to ‘A Range Equality for the Commutator with Two Involutory Matrices’,” IMAGE, Vol. 35, pp. 32–33, 2005. (Cited on page 217.) [1459] J. R. Wertz and W. J. Larson, Eds., Space Mission Analysis and Design. drecht: Kluwer, 1999. (Cited on page 175.) [1460] P. Wesseling, Principles of Computational Fluid Dynamics. 2001. (Cited on page xvii.)

Dor-

Berlin: Springer,

[1461] J. R. Westlake, A Handbook of Numerical Matrix Inversion and Solution of Linear Equations. New York: Wiley, 1968. (Cited on page xviii.) [1462] N. A. Wiegmann, “Normal Products of Matrices,” Duke Math. J., Vol. 15, pp. 633–638, 1948. (Cited on page 374.) [1463] Z. Wiener, “An Interesting Matrix Exponent Formula,” Lin. Alg. Appl., Vol. 257, pp. 307–310, 1997. (Cited on page 785.) [1464] E. P. Wigner and M. M. Yanase, “On the Positive Semidefinite Nature of a Certain Matrix Expression,” Canad. J. Math., Vol. 16, pp. 397–406, 1964. (Cited on page 502.) [1465] R. M. Wilcox, “Exponential Operators and Parameter Differentiation in Quantum Physics,” J. Math. Phys., Vol. 8, pp. 962–982, 1967. (Cited on pages xviii and 750.) [1466] J. B. Wilker, J. S. Sumner, A. A. Jagers, M. Vowe, and J. Anglesio, “Inequalities Involving Trigonometric Functions,” Amer. Math. Monthly, Vol. 98, pp. 264–266, 1991. (Cited on page 32.) [1467] J. H. Wilkinson, The Algebraic Eigenvalue Problem. Press, 1965. (Cited on page xviii.)

London: Oxford University

[1468] J. C. Willems, “Least Squares Stationary Optimal Control and the Algebraic Riccati Equation,” IEEE Trans. Autom. Contr., Vol. AC-16, pp. 621–634, 1971. (Cited on page 879.) [1469] D. A. Wilson, “Convolution and Hankel Operator Norms for Linear Systems,” IEEE Trans. Autom. Contr., Vol. AC-34, pp. 94–97, 1989. (Cited on page 680.) [1470] P. M. H. Wilson, Curved Spaces: From Classical Geometries to Elementary Differential Geometry. Cambridge: Cambridge University Press, 2008. (Cited on pages 175 and 214.) [1471] H. K. Wimmer, “Inertia Theorems for Matrices, Controllability, and Linear Vibrations,” Lin. Alg. Appl., Vol. 8, pp. 337–343, 1974. (Cited on pages 867 and 870.) [1472] H. K. Wimmer, “The Algebraic Riccati Equation without Complete Controllability,” SIAM J. Alg. Disc. Math., Vol. 3, pp. 1–12, 1982. (Cited on page 879.) [1473] H. K. Wimmer, “On Ostrowski’s Generalization of Sylvester’s Law of Inertia,” Lin. Alg. Appl., Vol. 52/53, pp. 739–741, 1983. (Cited on page 337.)

BIBLIOGRAPHY

963

[1474] H. K. Wimmer, “The Algebraic Riccati Equation: Conditions for the Existence and Uniqueness of Solutions,” Lin. Alg. Appl., Vol. 58, pp. 441–452, 1984. (Cited on page 879.) [1475] H. K. Wimmer, “Monotonicity of Maximal Solutions of Algebraic Riccati Equations,” Sys. Contr. Lett., Vol. 5, pp. 317–319, 1985. (Cited on pages 861 and 879.) [1476] H. K. Wimmer, “Extremal Problems for H¨ older Norms of Matrices and Realizations of Linear Systems,” SIAM J. Matrix Anal. Appl., Vol. 9, pp. 314–322, 1988. (Cited on pages 505 and 642.) [1477] H. K. Wimmer, “Linear Matrix Equations, Controllability and Observability, and the Rank of Solutions,” SIAM J. Matrix Anal. Appl., Vol. 9, pp. 570–578, 1988. (Cited on page 870.) [1478] H. K. Wimmer, “On the History of the Bezoutian and the Resultant Matrix,” Lin. Alg. Appl., Vol. 128, pp. 27–34, 1990. (Cited on page 279.) [1479] H. K. Wimmer, “Normal Forms of Symplectic Pencils and the Discrete-Time Algebraic Riccati Equation,” Lin. Alg. Appl., Vol. 147, pp. 411–440, 1991. (Cited on page 879.) [1480] H. K. Wimmer, “Lattice Properties of Sets of Semidefinite Solutions of ContinuousTime Algebraic Riccati Equations,” Automatica, Vol. 31, pp. 173–182, 1995. (Cited on page 879.) [1481] A. Witkowski, “A New Proof of the Monotonicity of Power Means,” J. Ineq. Pure Appl. Math., Vol. 5, no. 1, pp. 1–2, 2004, Article 6. (Cited on pages 25 and 30.) [1482] J. Wittenburg and L. Lilov, “Decomposition of a Finite Rotation into Three Rotations about Given Axes,” Multibody Sys. Dyn., Vol. 9, pp. 353–375, 2003. (Cited on page 740.) [1483] H. Wolkowicz and G. P. H. Styan, “Bounds for Eigenvalues Using Traces,” Lin. Alg. Appl., Vol. 29, pp. 471–506, 1980. (Cited on pages 56, 351, 361, and 362.) [1484] H. Wolkowicz and G. P. H. Styan, “More Bounds for Eigenvalues Using Traces,” Lin. Alg. Appl., Vol. 31, pp. 1–17, 1980. (Cited on page 361.) [1485] W. A. Wolovich, Linear Multivariable Systems. New York: Springer, 1974. (Cited on page 879.) [1486] M. J. Wonenburger, “A Decomposition of Orthogonal Transformations,” Canad. Math. Bull., Vol. 7, pp. 379–383, 1964. (Cited on page 383.) [1487] M. J. Wonenburger, “Transformations Which are Products of Two Involutions,” J. Math. Mech., Vol. 16, pp. 327–338, 1966. (Cited on page 383.) [1488] C. S. Wong, “Characterizations of Products of Symmetric Matrices,” Lin. Alg. Appl., Vol. 42, pp. 243–251, 1982. (Cited on page 382.) [1489] E. Wong, Ed., Stochastic Processes in Information and Dynamical Systems. New York: McGraw-Hill, 1971. (Cited on page 489.) [1490] W. M. Wonham, Linear Multivariable Control: A Geometric Approach, 2nd ed. New York: Springer, 1979. (Cited on pages xvii, 285, 710, 865, and 879.) [1491] C.-F. Wu, “On Some Ordering Properties of the Generalized Inverses of Nonnegative Definite Matrices,” Lin. Alg. Appl., Vol. 32, pp. 49–60, 1980. (Cited on page 579.)

964

BIBLIOGRAPHY

[1492] P. Y. Wu, “Products of Nilpotent Matrices,” Lin. Alg. Appl., Vol. 96, pp. 227–232, 1987. (Cited on page 383.) [1493] P. Y. Wu, “Products of Positive Semidefinite Matrices,” Lin. Alg. Appl., Vol. 111, pp. 53–61, 1988. (Cited on pages 382 and 596.) [1494] P. Y. Wu, “The Operator Factorization Problems,” Lin. Alg. Appl., Vol. 117, pp. 35–63, 1989. (Cited on pages 382, 383, 384, and 770.) [1495] P. Y. Wu, “Sums of Idempotent Matrices,” Lin. Alg. Appl., Vol. 142, pp. 43–54, 1990. (Cited on pages 394 and 395.) [1496] S. Wu, “A Further Generalization of Aczel’s Inequality and Popoviciu’s Inequality,” Math. Ineq. Appl., Vol. 10, pp. 565–573, 2007. (Cited on page 70.) [1497] S. Wu, “A Sharpened Version of the Fundamental Triangle Inequality,” Math. Ineq. Appl., Vol. 11, pp. 477–482, 2008. (Cited on page 171.) [1498] S. Wu, “Note on a Conjecture of R. A. Satnoianu,” Math. Ineq. Appl., Vol. 12, pp. 147–151, 2009. (Cited on page 65.) [1499] Y.-D. Wu, Z.-H. Zhang, and V. Lokesha, “Sharpening on Mircea’s Inequality,” J. Ineq. Pure Appl. Math., Vol. 8, no. 4, pp. 1–6, 2007, Article 116. (Cited on page 171.) [1500] Z.-G. Xiao and Z.-H. Zhang, “The Inequalities G ≤ L ≤ I ≤ A in n Variables,” J. Ineq. Pure. Appl. Math., Vol. 4, no. 2/39, pp. 1–6, 2003. (Cited on page 59.) [1501] C. Xu, “Bellman’s Inequality,” Lin. Alg. Appl., Vol. 229, pp. 9–14, 1995. (Cited on page 526.) [1502] C. Xu, Z. Xu, and F. Zhang, “Revisiting Hua-Marcus-Bellman-Ando Inequalities on Contractive Matrices,” Lin. Alg. Appl., Vol. 430, pp. 1499–1508, 2009. (Cited on pages 76, 157, 520, 539, and 620.) [1503] H. Xu, “Two Results About the Matrix Exponential,” Lin. Alg. Appl., Vol. 262, pp. 99–109, 1997. (Cited on pages 714 and 769.) [1504] A. Yakub, “Symmetric Inequality,” Amer. Math. Monthly, Vol. 114, p. 649, 2007. (Cited on page 49.) [1505] T. Yamazaki, “An Extension of Specht’s Theorem via Kantorovich Inequality and Related Results,” Math. Ineq. Appl., Vol. 3, pp. 89–96, 2000. (Cited on page 57.) [1506] T. Yamazaki, “Further Characterizations of Chaotic Order via Specht’s Ratio,” Math. Ineq. Appl., Vol. 3, pp. 259–268, 2000. (Cited on page 575.) [1507] B. Yang, “On a New Inequality Similar to Hardy-Hilbert’s Inequality,” Math. Ineq. Appl., Vol. 6, pp. 37–44, 2003. (Cited on page 69.) [1508] B. Yang and T. M. Rassias, “On the Way of Weight Coefficient and Research for the Hilbert-Type Inequalities,” Math. Ineq. Appl., Vol. 6, pp. 625–658, 2003. (Cited on page 69.) [1509] X. Yang, “Necessary Conditions of Hurwitz Polynomials,” Lin. Alg. Appl., Vol. 359, pp. 21–27, 2003. (Cited on pages 764 and 765.) [1510] X. Yang, “Some Necessary Conditions for Hurwitz Stability,” Automatica, Vol. 40, pp. 527–529, 2004. (Cited on page 765.)

BIBLIOGRAPHY

965

[1511] Z. P. Yang and X. X. Feng, “A Note on the Trace Inequality for Products of Hermitian Matrix Power,” J. Ineq. Pure. Appl. Math., Vol. 3, no. 5/78, pp. 1–12, 2002. (Cited on pages 524 and 526.) [1512] S. F. Yau and Y. Bresler, “A Generalization of Bergstrom’s Inequality and Some Applications,” Lin. Alg. Appl., Vol. 161, pp. 135–151, 1992. (Cited on pages 588 and 592.) [1513] D. M. Young, Iterative Solution of Large Linear Systems. New York: Academic Press, 1971, reprinted by Dover, Mineola, 2003. (Cited on page xviii.) [1514] N. J. Young, “Matrices which Maximise any Analytic Function,” Acta Math. Hung., Vol. 34, pp. 239–243, 1979. (Cited on page 665.) [1515] R. Zamfir, “Refining Some Inequalities,” J. Ineq. Pure Appl. Math., Vol. 9, no. 3, pp. 1–5, 2008, Article 77. (Cited on page 781.) [1516] A. Zanna and H. Z. Munthe-Kaas, “Generalized Polar Decompositions for the Approximation of the Matrix Exponential,” SIAM J. Matrix Anal. Appl., Vol. 23, pp. 840–862, 2002. (Cited on page 742.) [1517] X. Zhan, “Inequalities for the Singular Values of Hadamard Products,” SIAM J. Matrix Anal. Appl., Vol. 18, pp. 1093–1095, 1997. (Cited on page 674.) [1518] X. Zhan, “Inequalities for Unitarily Invariant Norms,” SIAM J. Matrix Anal. Appl., Vol. 20, pp. 466–470, 1998. (Cited on page 492.) [1519] X. Zhan, “Singular Values of Differences of Positive Semidefinite Matrices,” SIAM J. Matrix Anal. Appl., Vol. 22, pp. 819–823, 2000. (Cited on pages 567 and 647.) [1520] X. Zhan, “Span of the Orthogonal Orbit of Real Matrices,” Lin. Multilin. Alg., Vol. 49, pp. 337–346, 2001. (Cited on page 394.) [1521] X. Zhan, Matrix Inequalities. New York: Springer, 2002. (Cited on pages 177, 356, 477, 485, 492, 560, 561, 562, 573, 582, 583, 584, 589, 591, 592, 594, 595, 603, 638, 671, and 752.) [1522] X. Zhan, “On Some Matrix Inequalities,” Lin. Alg. Appl., Vol. 376, pp. 299–303, 2004. (Cited on page 567.) [1523] F. Zhang, Linear Algebra: Challenging Problems for Students. Baltimore: Johns Hopkins University Press, 1996. (Cited on pages 142, 199, and 200.) [1524] F. Zhang, “Quaternions and Matrices of Quaternions,” Lin. Alg. Appl., Vol. 251, pp. 21–57, 1997. (Cited on page 251.) [1525] F. Zhang, “A Compound Matrix with Positive Determinant,” Amer. Math. Monthly, Vol. 105, p. 958, 1998. (Cited on page 166.) [1526] F. Zhang, Matrix Theory: Basic Results and Techniques. New York: Springer, 1999. (Cited on pages 136, 137, 235, 340, 373, 390, 502, 511, 518, 524, 525, 538, 595, and 619.) [1527] F. Zhang, “Schur Complements and Matrix Inequalities in the Lowner Ordering,” Lin. Alg. Appl., Vol. 321, pp. 399–410, 2000. (Cited on page 539.) [1528] F. Zhang, “Matrix Inequalities by Means of Block Matrices,” Math. Ineq. Appl., Vol. 4, pp. 481–490, 2001. (Cited on pages 497, 516, 542, 560, 578, 586, 587, 592, and 593.)

966

BIBLIOGRAPHY

[1529] F. Zhang, “Inequalities Involving Square Roots,” IMAGE, Vol. 29, pp. 33–34, 2002. (Cited on page 503.) [1530] F. Zhang, “Block Matrix Techniques,” in The Schur Complement and Its Applications, F. Zhang, Ed. New York: Springer, 2004, pp. 83–110. (Cited on pages 498, 531, 532, 542, 551, 555, 556, 559, and 561.) [1531] F. Zhang, “A Matrix Identity on the Schur Complement,” Lin. Multilin. Alg., Vol. 52, pp. 367–373, 2004. (Cited on page 430.) [1532] L. Zhang, “A Characterization of the Drazin Inverse,” Lin. Alg. Appl., Vol. 335, pp. 183–188, 2001. (Cited on page 431.) [1533] Q.-C. Zhong, “J-Spectral Factorization of Regular Para-Hermitian Transfer Matrices,” Automatica, Vol. 41, pp. 1289–1293, 2005. (Cited on page 223.) [1534] Q.-C. Zhong, Robust Control of Time-delay Systems. (Cited on page 223.)

London: Springer, 2006.

[1535] K. Zhou, Robust and Optimal Control. Upper Saddle River: Prentice-Hall, 1996. (Cited on pages xvii, 307, 664, 668, and 860.) [1536] L. Zhu, “A New Simple Proof of Wilker’s Inequality,” Math. Ineq. Appl., Vol. 8, pp. 749–750, 2005. (Cited on page 32.) [1537] L. Zhu, “On Wilker-Type Inequalities,” Math. Ineq. Appl., Vol. 10, pp. 727–731, 2005. (Cited on page 32.) [1538] G. Zielke, “Some Remarks on Matrix Norms, Condition Numbers, and Error Estimates for Linear Equations,” Lin. Alg. Appl., Vol. 110, pp. 29–41, 1988. (Cited on pages 629 and 677.) [1539] S. Zlobec, “An Explicit Form of the Moore-Penrose Inverse of an Arbitrary Complex Matrix,” SIAM Rev., Vol. 12, pp. 132–134, 1970. (Cited on page 400.) [1540] D. Zwillinger, Standard Mathematical Tables and Formulae, 31st ed. Boca Raton: Chapman & Hall/CRC, 2003. (Cited on pages 82, 171, 175, 277, and 391.)

Author Index Abdessemed, A. 641 Ablamowicz, R. 737 Abou-Kandil, H. xvii Abramovich, S. 32 Aceto, L. 387, 396, 491, 737 Afriat, S. 228 Agaev, R. 777 Agarwal, R. P. 793 Ahlbrandt, C. D. 879 Ahn, E. 749 Aitken, A. C. xix Aivazis, M. 211 Akdeniz, F. 579 Al-Ahmar, M. 207 Albert, A. A. 341 Albert, A. E. 415, 418, 421, 427, 595, 678 Aldaz, J. M. 62 Aldrovandi, R. xvii, 299, 390 Aleksiejczuk, M. 429 Alfakih, A. Y. 629 Alic, M. 58, 510 Aling, H. 879 Alpargu, G. 63, 72, 551 Alperin, R. C. 215 Alsina, C. 35, 37 Altmann, S. L. xvii, 214, 215, 249, 251 Alzer, H. 56 Amghibech, S. 536 Anderson, B. D. O. xvii, 375, 511, 552, 772, 879 Anderson, G. 25, 30, 32 Anderson, T. W. 504 Anderson, W. N. 366, 421, 422, 486, 506, 510, 522, 582, 583, 595 Ando, T. 482, 485, 503, 506, 507, 509, 510, 512, 520, 524, 526, 539, 566, 583, 588, 589, 591, 594,

596, 640, 645, 647, 657, 671, 674, 749, 752, 753, 756, 761, 763, 868 Andreescu, T. 76, 77, 168, 169, 171–173 Andrews, G. E. 391 Andrica, D. 76, 77, 168, 169, 171–173 Andruchow, E. 632, 646 Angel, E. xvii, 215 Anglesio, J. 32 Anonymous 46 Antoulas, A. C. 867 Aplevich, J. D. xvii, 331 Apostol, T. 491 Apostol, T. M. 737 Araki, H. 528, 639, 640 Araujo, J. 383 Arimoto, A. 390 Arnold, B. 178 Arponen, T. 737 Arsigny, V. 753 Artin, M. 245, 252 Artzrouni, M. 786 Arvanitoyeorgos, A. 252 Aslaksen, H. 165, 198, 249, 251, 283, 284 Asner, B. A. 765, 770 Au-Yeung, Y.-H. 396, 503, 554, 558 Audenaert, K. M. R. 528, 652, 653 Aujla, J. S. 504, 565, 566, 574, 586, 591, 592, 595, 639, 649 Aupetit, B. 762 Avriel, M. 555 Axelsson, O. xviii Ayache, N. 753 Azar, L. E. 69 Baez, J. C. 247, 249, 250

Bagdasar, O. 24, 40, 47, 59 Bai, Z. 523 Bailey, D. 19, 20, 22, 23, 62, 174, 300 Bailey, D. W. 294 Bailey, H. 171 Baker, A. 198, 239, 248, 252, 379, 724, 748 Baksalary, J. K. 130, 219, 221, 337, 412, 425, 429, 503, 514, 576, 579, 580, 589 Baksalary, O. M. 130, 192, 194, 195, 219, 221, 344, 407, 412, 425, 429, 436–438, 532, 578 Ball, K. 625, 643 Ballantine, C. S. 192, 341, 382, 383, 596, 770, 869 Banerjee, S. 175 Bang-Jensen, J. xvii Bani-Domi, W. 650, 651 Bapat, R. B. 73, 134, 237, 299, 305, 429, 586 Bar-Itzhack, I. Y. 212, 738 Barbeau, E. J. 34, 43, 46, 51, 52, 171, 778, 779 Baric, J. 32 Barnes, E. R. 295 Barnett, S. xvii, 147, 158, 234, 279, 307, 386, 387, 396, 411, 430, 494, 499, 679, 771, 871 Barrett, W. 134, 238, 488, 493, 541, 592 Barria, J. 348 Bart, H. 873 Baruh, H. 214, 249, 741 Barvinok, A. 47, 51, 120, 124, 547, 562, 696 Barza, S. 64 Bates, R. G. 242

968 Bau, D. xviii Bauer, F. L. 680 Bayard, D. S. 364 Bazaraa, M. S. 305, 684, 686 Beams, R. 381 Beavers, A. N. 381 Bebiano, N. xvii, 365, 534, 752–754 Beckenbach, E. F. 84, 598 Becker, R. I. 396, 554 Beckner, W. 643 Bekjan, T. N. 549 Bekker, P. A. 531, 532, 595 Belinfante, J. G. 793 Belitskii, G. R. 680 Bellman, R. 84, 158, 306, 499, 506, 598, 748 Ben-Israel, A. 406, 410, 429, 434, 435, 438, 558 Ben Taher, B. 737 Ben-Tal, A. 119, 178 Benjamin, A. T. 12, 19, 304 Benson, C. T. 245 Berg, L. 141 Berge, C. xvii Berkovitz, L. D. 178, 697 Berman, A. 190, 252, 299, 301, 303, 774–776 Bernhardsson, B. 769 Bernstein, D. S. 138, 204, 238, 250, 252, 368, 374, 539, 548, 680, 737, 744, 748, 756, 759, 769, 774, 776, 777, 793, 869–871, 879 Bhagwat, K. V. 476, 506, 749 Bhat, S. P. 250, 368, 774, 776, 793, 870 Bhatia, R. 26, 176, 177, 201, 241, 296, 349, 355, 356, 365, 476, 479, 482, 484–486, 489–492, 494, 502, 504, 509, 526, 528, 535, 560–562, 565–567, 570, 573, 584, 595, 596, 615, 626, 630, 634, 636, 638, 639, 641–647, 650, 657, 658, 663, 671, 752, 755, 758, 761, 763 Bhattacharya, R. 346 Bhattacharyya, S. P. 764, 870, 879

AUTHOR INDEX

Bhattarcharyya, S. P. 870 Bhaya, A. 793 Bicknell, M. R. 155 Biggs, N. xvii, 367 Binding, P. 396 Binmore, K. xvii Bjorck, A. 678 Blanes, S. 743, 749 Bloch, E. D. 11, 84 Blondel, V. 786 Blumenthal, L. M. 174 Boche, H. xvii, 176 Boehm, W. 785 Bojanczyk, A. W. 289 Bollobas, B. xvii Bondar, J. V. 49, 66 Borck, A. 438 Borre, K. xvii Borwein, J. 19, 20, 22, 23, 62, 174, 300 Borwein, J. M. 119, 120, 178, 305, 363, 364, 484, 506, 693, 696 Bosch, A. J. 382, 396 Bottcher, A. 641 Bottema, O. 172 Boullion, T. L. 438 Bourin, J.-C. 67, 504, 511, 512, 528, 530–532, 574, 591, 632, 639, 649, 656, 666, 754 Bourque, K. 491 Boyd, S. xvii, 178, 633 Bozkurt, D. 674 Brenner, J. L. 166, 251, 294, 378 Bresler, Y. 588, 592 Brewer, J. W. 458 Brickman, L. 548 Brockett, R. 879 Brockett, R. W. 562, 871 Brothers, H. J. 28 Brown, G. 72 Browne, E. T. 595 Bru, R. 374 Brualdi, R. A. xvii, 137, 140, 142, 178, 285, 293, 300, 550, 772 Buckholtz, D. 228, 366, 367 Bullen, P. S. 25, 28–30, 32, 38, 41, 46, 56, 58, 60, 62, 70, 84, 600, 622, 626 Bullo, F. xvii, 301 Bultheel, A. 307

Burch, J. M. xvii Burns, F. 429 Bushell, P. J. 571 Cahill, N. D. 304 Cain, B. E. 336, 868 Callan, D. 128 Camouzis, E. 778, 793 Campbell, S. L. 411, 420, 429, 435, 438, 767, 793 Cao, J. 30, 41 Cao, L. 460, 503 Carlen, E. 625, 643 Carlen, E. A. 484, 485 Carlson, D. 424, 429, 485, 596, 770, 867, 868 Carpenter, J. A. 358 Cartier, P. 793 Cartwright, D. I. 56, 793 Casas, F. 743, 749 Castro-Gonzalez, N. 429 Caswell, H. xvii Cater, F. S. 380 Chabrillac, Y. 554 Chan, N. N. 503 Chandrasekar, J. 238 Chapellat, H. 764 Chartrand, G. xvii Chatelin, F. xviii Chattot, J.-J. xvii Chaturvedi, N. A. 204 Chebotarev, P. 777 Chehab, J.-P. 523 Chellaboina, V. xvii, 252, 776, 777 Chellaboina, V.-S. 636, 680, 840 Chen, B. M. xvii, 123, 124, 354, 363, 396 Chen, C. T. 867 Chen, H. 56, 68 Chen, J. 654 Chen, J.-Q. 244 Chen, L. 525, 671 Chen, S. 588 Cheng, C.-M. 486, 535, 566, 674 Cheng, H.-W. 737 Cheng, S. 192, 194, 406, 407, 414, 416, 419, 422, 425, 434, 584 Chien, M.-T. 238 Choi, M.-D. 226, 234, 395 Chollet, J. 522

969

AUTHOR INDEX

Choudhry, A. 381 Chu, M. T. 412, 425, 503 Chu, X.-G. 19 Chuai, J. 447, 451 Chuang, I. L. xvii Chui, N. L. C. 783 Chung, F. R. K. xvii Cizmesija, A. 63, 64 Clements, D. J. 879 Climent, J. J. 374 Cline, R. E. 128, 129, 131, 420, 428 Cloud, M. J. 84 Coakley, E. S. 246 Cohen, J. E. 757, 759 Cohoon, D. K. 137 Collins, E. G. 458, 879 Constales, D. 405 Contreras, M. xvii, 777 Conway, J. C. 51, 52, 245, 249 Corach, G. 632, 646 Corless, M. J. xvii Cortes, J. xvii, 301 Costa, P. J. 704 Cottle, R. W. 546, 556, 764 Cover, T. M. xvii, 485, 536, 541, 556 Crabtree, D. E. 294, 373 Crawford, C. R. 554 Crilly, T. 284 Crossley, M. D. 212 Crouzeix, J.-P. 554 Cullen, C. G. 374, 689, 703 Culver, W. J. 793 Curtain, R. F. 758 Curtis, M. L. 214, 248, 252 Cvetkovic, D. xvii, 178, 285 da Providencia, J. xvii, 534, 752–754 Da Silva, J. A. D. 457 Daboul, P. J. 249 Dahlquist, G. 774 Dale, P. xvii, 178 D’Andrea, R. 668 D’Angelo, H. 879 D’Angelo, J. P. 77, 620 Daniel, J. W. xix, 178 Dannan, F. M. 528, 537 Dasgupta, S. 746 Datko, R. 773 Datta, B. N. xvii, 867

Dattorro, J. xvii, 128, 137, 352, 705 Daubechies, I. 786 Davies, E. B. 641 Davis, C. 639, 646 Davis, P. J. xix, 390 Davison, E. J. 769 Dawlings, R. J. H. 383 Day, J. 565, 720 Day, P. W. 67 de Boor, C. 178 de Groen, P. P. N. 636 de Hoog, F. R. 413 de Launey, W. 458 De Moor, B. 879 de Pillis, J. 533, 543 De Pillis, J. E. 543 de Pillis, L. G. 277 de Souza, E. 870 De Souza, P. N. 122, 127, 163, 503 de Vries, H. L. 654 DeAlba, L. M. 470, 868, 869 Debnath, L. 28, 29, 42, 73 Decell, H. P. 408 Deistler, M. 879 Del Buono, N. 240 Delbourgo, R. 249 DeMarco, C. L. 293 Demmel, J. W. xviii Deng, C. 506 Deng, C. Y. 433 Denman, E. D. 381 DePrima, C. R. 377 D’Errico, J. R. 304 Desoer, C. A. 758 DeTemple, D. W. 21 Deutsch, E. 649, 758, 779, 780 Devaney, R. L. xvii Dhrymes, P. J. xvii Dieci, L. 724, 751 Diestel, R. xvii Dines, L. L. 547, 548 Ding, J. 286, 677 Dittmer, A. 204 Dixon, G. M. 249 Dixon, J. D. 540 Djaferis, T. E. 870 Djokovic, D. Z. 163, 166, 345, 380, 383 Djordjovic, Z. 172

Dokovic, D. Z. 340, 342, 343, 347, 375 Dolotin, V. 458 Dombre, E. xvii Donoghue, W. F. 489, 490, 594, 595 Dopazo, E. 429 Dopico, F. M. 239, 246, 378 Doran, C. 204, 249, 620 Doran, C. J. L. 204, 249, 620 Dorst, L. 249, 620 Douglas, R. G. 474 Doyle, J. C. 769 Doyle, P. G. xvii Drachman, B. C. 84 Dragan, V. 879 Dragomir, S. S. 39, 63, 65, 68, 70–73, 77 Drazin, M. P. 282 Drissi, D. 41, 492 Drivaliaris, D. 414 Drnovsek, R. 217 Drury, S. W. 200 Du, H. 506 Du, H.-K. 516, 573, 668 Du, K. 676 Dubeau, F. 20 Duffin, R. J. 582 Duleba, I. 743 Dullerud, G. E. xvii Dummit, D. S. 242, 244, 245, 390, 391, 396 Dunkl, C. F. 619 Dym, H. 61, 140, 142, 145, 173, 306, 336–338, 365, 366, 429, 474, 503, 558, 561, 578, 648, 668, 695, 697, 698, 701, 766, 868 Edelman, A. 491 Edwards, C. H., Jr. 688 Egecioglu, O. 77 Eggleston, H. G. 178 Elsner, L. 20, 154, 194, 293, 454, 457, 587, 714, 758, 762 Embree, M. 758 Engel, A. 34, 35, 37, 42, 44–46, 48–51, 54, 57, 64, 66, 171 Engo, K. 749 Erdmann, K. 252 Erdos, J. A. 383

970 Eriksson, K. 391 Eriksson, R. 175 Evard, J.-C. 793 Fallat, S. 498 Fan, K. 498, 534, 536 Fang, M. 457 Fang, Y. 364 Farebrother, R. W. 251, 380, 438 Farenick, D. R. 390 Fassler, A. 252 Feiner, S. xvii, 215 Fekete, A. E. 204, 217 Feng, B. Q. 630, 640 Feng, X. 364 Feng, X. X. 524, 526 Feng, Z. 172 Fenn, R. 175, 204, 213, 248, 249, 304 Ferreira, P. G. 879 Ferziger, J. H. xvii Fiedler, M. xvii, 161, 174, 277, 279, 299, 359, 388, 410, 427, 454, 509, 510, 515, 585, 587, 589, 770 Field, M. J. 56, 793 Fill, J. A. 413 Fillard, P. 753 Fillmore, J. P. 737 Fillmore, P. A. 341, 394, 474 Fink, A. M. 25, 77, 84, 295, 619, 620, 623, 624 Fishkind, D. E. 413 Fitzgerald, C. H. 584 Flanders, H. 233, 504, 660 Fleming, W. 543 Flett, T. M. 705 Foldes, S. 242 Foley, J. xvii, 215 Fontijne, D. 249, 620 Foote, R. M. 242, 244, 245, 390, 391, 396 Formanek, E. 161, 233, 283 Foulds, L. R. xvii Francis, B. A. xvii Franklin, J. xix Frazho, A. E. xvii Frazier, M. xvii Freiling, G. xvii Friedland, S. 659, 672, 679, 759, 777 Friswell, M. I. 454

AUTHOR INDEX

Fuhrmann, P. A. 208, 209, 277, 279, 280, 307, 336, 520 Fujii, J. I. 552, 646 Fujii, M. 37, 57, 74, 507, 513, 552, 575, 620, 646, 752, 780 Fuller, A. T. 447, 454, 458, 765 Fulton, W. 244 Funderlic, R. E. 128, 129, 131, 412, 420, 425, 503 Furuichi, S. 526, 527 Furuta, T. 26, 27, 227, 476, 477, 479, 485, 501, 507, 511, 513, 514, 529, 552, 562, 573, 575, 623, 639, 646, 749, 753 Gaines, F. 199, 200, 341 Galantai, A. 122, 221, 227, 228, 252, 343, 344, 358, 366, 367, 416, 627, 698 Gallier, J. 214 Gangsong, L. 563 Gantmacher, F. R. xix, 331, 346, 595, 793 Garling, D. J. H. 36, 58, 59, 64, 69, 82, 176, 624, 630 Garloff, J. 765 Garvey, S. D. 454 Geerts, T. 879 Gelfand, I. M. 458 Genton, M. G. 489 George, A. 373, 391, 392 Ger, R. 37 Gerdes, P. 390 Gerrard, A. xvii Gerrish, F. 349 Geveci, T. 879 Gheondea, A. 506 Ghouraba, F. A. A. 252 Gil, M. I. 354 Gilbert, A. C. 635 Gilmore, R. 252 Girard, P. R. 204, 213, 214, 246 Girgensohn, R. 19, 20, 22, 23, 62, 174, 300 Glasser, M. L. 750 Godsil, C. xvii Godunov, S. K. xix, 229, 767 Goh, C. J. 65

Gohberg, I. 256, 307, 366, 396, 698, 860, 861, 873, 879 Golberg, M. A. 703, 743 Goldberg, M. 640, 661 Goller, H. 583 Golub, G. H. xviii, 412, 425, 503, 672, 710 Golub, G.H. 523 Gong, M.-P. 571 Gonzalez, N. C. 745 Goodman, F. M. 245 Goodman, L. E. 204 Gordon, N. 151 Goroncy, A. 56 Govaerts, W. 454, 458, 668 Gow, R. 340, 375, 384 Graham, A. 458 Graybill, F. A. xvii Grcar, J. 178 Grcar, J. F. 613–615 Green, W. L. 509, 582 Greene, D. H. 19 Greub, W. H. xix, 458 Greville, T. N. E. 228, 406, 410, 415–417, 429, 434, 435, 438, 558 Grigoriadis, K. xvii, 793 Grone, R. 194, 205, 372 Gross, J. xvii, 204, 219–221, 226, 251, 366, 416, 417, 429, 576, 578, 579, 583 Grossman, I. 244 Grove, L. C. 245 Guan, K. 41 Gudder, S. 506 Gull, S. 204, 249, 620 Guobiao, Z. 563 Gupta, A. K. xvii Gurlebeck, K. 249, 251, 740 Gurvits, L. 786 Gustafson, K. E. 546, 633 Gustafson, W. H. 383 Gutin, G. xvii Gwanyama, P. W. 57 Haddad, W. M. xvii, 178, 252, 374, 539, 636, 680, 776, 777, 782, 840, 871, 879 Hager, W. W. 178, 344 Hahn, W. 793 Hairer, E. 743

971

AUTHOR INDEX

Hajja, M. 171 Halanay, A. 879 Hall, A. 511 Hall, B. C. 239, 718, 720, 722–724, 746, 750, 793 Halliwell, G. T. 30, 74 Halmos, P. R. 98, 122, 219, 305, 341, 346, 348, 371, 374, 382, 383, 411, 421, 422, 496, 747 Hamermesh, M. 244 Han, J. H. 139 Haneda, H. 758 Hannan, E. J. 879 Hansen, F. 40, 531 Hanson, A. J. 249, 251 Hardy, G. 84 Harner, E. J. 366 Harris, J. 244 Harris, J. M. xvii Harris, L. A. 161, 520, 521 Harris, W. A. 737 Hart, G. W. xvii, 252 Hartfiel, D. J. xvii, 785 Hartwig, R. 429 Hartwig, R. E. 129, 130, 344, 351, 367, 395, 406, 407, 418, 422, 429, 431, 436, 576, 579, 580, 870 Harville, D. A. xvii, 216, 219, 405, 411, 413, 417, 420, 535, 557, 705 Hattori, S. 773 Hauke, J. 577 Hautus, M. L. J. xvii, 879 Havel, T. F. 748 Havil, J. 21 Haynes, T. 783 Haynsworth, E. V. 424, 429, 485, 521, 596 Hecht, E. xvii Heij, C. 879 Heinig, G. 279 Helmke, U. 279 Helton, B. W. 793 Henderson, H. V. 157, 178, 458 Herman, J. 19, 20, 26, 27, 29, 35, 39, 40, 43–48, 50, 51, 54, 57, 66 Hershkowitz, D. 20, 212, 454, 522, 738, 777 Hestenes, D. 204, 249

Hiai, F. 485, 506, 507, 512, 524, 526, 637, 749, 753, 761 Higham, D. J. 178 Higham, N. J. xviii, 78, 81, 178, 234, 241, 284, 357, 360, 381, 393, 394, 626, 629–632, 640, 661, 666, 673, 680, 690, 698, 699, 722, 742, 747, 751, 759, 761, 768, 769, 793, 876 Hile, G. N. 249 Hill, R. 868 Hill, R. D. 154, 242 Hillar, C.-J. 530 Hilliard, L. O. 306 Hinrichsen, D. 347, 355, 356, 609, 676, 701, 758, 762, 767, 778 Hirsch, M. W. xvii, 339 Hirschhorn, M. D. 139 Hirst, J. L. xvii Hirzallah, O. 42, 77, 552, 642, 643, 670, 671 Hmamed, A. 595 Hoagg, J. B. 238 Hoffman, A. J. 295 Hoffman, K. xix Holbrook, J. 26, 566, 755 Hollot, C. V. 553 Holmes, R. R. 244 Holtz, O. 764, 786 Hong, Y. 382 Hong, Y. P. 664 Horadam, K. J. 391 Horn, A. 562 Horn, R. A. 151, 176, 177, 201, 205, 209, 233, 276, 295, 297–300, 302, 306, 307, 319, 342, 348, 355, 356, 362, 373, 374, 377, 378, 380, 382, 392, 394, 430, 444, 447, 454, 457, 458, 470, 474, 486, 490, 507, 517, 522, 535, 539, 541, 542, 546, 547, 555, 558, 560–564, 566, 584, 585, 591–593, 595, 600, 604, 605, 607, 615–617, 628, 629, 632–634, 636, 638, 649, 658–661, 663, 664, 667, 670, 674, 675, 703, 705, 718, 719, 722,

743, 756, 759, 761, 771, 776, 871 Horne, B. G. 661 Hou, H.-C. 516, 573, 668 Hou, S.-H. 307 Householder, A. S. xviii, 279, 358, 412, 669, 695 Howe, R. 252, 793 Howie, J. M. 84 Howland, R. A. xvii Hsieh, P.-F. xvii, 339 Hu, G.-D. 767 Hu, G.-H. 767 Hu-yun, S. 138, 572 Huang, R. 457, 661, 675, 676 Huang, T.-Z. 56, 299, 457, 458, 655, 675, 676 Huang, X. 42 Hughes, J. xvii, 215 Hughes, P. C. xvii, 741 Huhtanen, M. 77 Hui, Q. 782 Humphries, S. 143 Hung, C. H. 429 Hunter, J. J. 438 Hyland, D. C. 252, 374, 458, 774, 777, 879 Ibragimov, N. H. 793 Ikebe, Y. 705 Ikramov, K. D. 194, 373, 391, 392, 653, 655, 714, 762 Inagaki, T. 705 Ionescu, V. xvii, 879 Ipsen, I. 122, 228, 358, 366, 416 Iserles, A. 204, 739, 743, 748 Ito, T. 46 Ito, Y. 773 Iwasaki, T. xvii, 793 Izumino, S. 57, 71, 551 Jacobson, D. H. 879 Jagers, A. A. 32 Jameson, A. 595 Janic, R. R. 172 Jank, G. xvii Jeffrey, A. 81, 82, 84, 705 Jeffries, C. 777 Jennings, A. xviii Jennings, G. A. 175

972 Jensen, S. T. 56, 499 Ji, J. 141 Jia, G. 30, 41 Jiang, C.-C. 340 Jiang, Y. 344 Jin, X.-Q. 641 Jocic, D. 645 Johnson, C. R. 143, 151, 176, 177, 194, 200, 201, 205, 209, 212, 233, 238, 239, 246, 276, 295, 297–300, 302, 306, 307, 319, 335–337, 342, 348, 355, 356, 372–374, 377, 378, 380, 381, 392, 394, 444, 447, 454, 456, 457, 470, 474, 486, 490, 493, 517, 522, 534, 536, 541, 546, 547, 552, 555, 558, 560–564, 585, 587, 591, 592, 595, 600, 604, 605, 607, 615–617, 628, 629, 633, 634, 636, 649, 658–660, 663, 664, 667, 670, 674, 675, 680, 703, 705, 718, 719, 722, 738, 743, 756, 759, 761, 765, 770, 771, 776, 868, 869, 871 Jolly, M. 705 Jonas, P. 506 Jordan, T. F. 251 Jorswieck, E. A. xvii, 176 Joyner, D. 139, 231, 245, 246, 252 Jung, D. 37, 57 Junkins, J. L. 212, 738 Jury, E. I. 454, 458, 772, 778 Kaashoek, M. A. 873 Kaczor, W. J. 26, 30, 32, 37, 72 Kadison, R. V. 505 Kagan, A. 531 Kagstrom, J. B. 767 Kailath, T. 259, 307, 331, 386, 879 Kalaba, R. E. xvii Kalman, D. 390 Kamei, E. 507, 513, 575 Kane, T. R. xvii Kanzo, T. 56, 621 Kapila, V. 879

AUTHOR INDEX

Kaplansky, I. xix, 346 Kapranov, M. M. 458 Karanasios, S. 414 Karcanias, N. 396, 879 Karlin, S. 457 Kaszkurewicz, E. 793 Kato, M. 623 Kato, T. 506, 657, 680, 759 Katsuura, H. 51, 58, 59 Katz, I. J. 418 Katz, S. M. 543 Kauderer, M. xvii Kayalar, S. 344 Kazakia, J. Y. 341 Kazarinoff, N. D. 40, 600 Keel, L. 764 Kelly, F. P. 759 Kendall, M. G. 199 Kenney, C. 357, 699, 780 Kestelman, H. 371 Keyfitz, N. xvii, 299 Khalil, W. xvii Khan, N. A. 444, 458, 590 Khatri, C. G. 408, 421 Kim, S. 58, 510, 749 King, C. 643, 652 Kinyon, M. K. 83 Kittaneh, F. 42, 77, 341, 357, 361, 504, 552, 566–569, 633, 636–647, 649–651, 660, 668–671, 780, 781 Klaus, A.-L. 640 Klee, V. 777 Knox, J. A. 28 Knuth, D. E. 19 Koch, C. T. 743 Koks, D. 249, 338 Koliha, J. J. 219–222, 418, 745 Kolman, B. 793 Komaroff, N. 67, 363, 570 Komornik, V. 763 Koning, R. H. 458 Kosaki, H. 640, 642 Kosecka, J. xvii Koshy, T. xvii, 238, 304 Kovac-Striko, J. 396 Kovacec, A. 365 Krafft, O. 582 Krattenthaler, C. xvii, 143 Kratz, W. 879 Krauter, A. R. 523 Kreindler, E. 595

Kress, R. 654 Krupnik, M. 390 Krupnik, N. 390 Kubo, F. 74, 620, 780 Kucera, R. 19, 20, 26, 27, 29, 35, 39, 40, 43–48, 50, 51, 54, 57, 66 Kucera, V. 879 Kufner, A. 64, 69 Kuipers, J. B. xvii, 249, 251 Kulenovic, M. R. S. 793 Kunze, R. xix Kurepa, S. 756 Kuriyama, K. 526 Kwakernaak, K. xvii Kwapisz, M. 782 Kwong, M. K. 242, 503, 506, 586 Kwong, R. H. 863, 879 Kyrchei, I. I. 141 Laberteaux, K. R. 195 Ladas, G. 778, 793 Laffey, T. J. 306, 347, 348, 375, 384 Lagarias, J. C. 786 Lai, H.-J. xvii Lakshminarayanan, S. 141 Lam, T. Y. 46 Lancaster, P. xvii, 256, 279, 307, 331, 349, 366, 371, 396, 447, 554, 615, 635, 680, 698, 860, 861, 867, 868, 879 Langholz, G. 772 Larson, L. 27, 32, 36, 38, 42, 44–49, 53, 168, 171 Larsson, L. 64 Lasenby, A. 204, 249, 620 Lasenby, A. N. 204, 249, 620 Lasenby, J. 204, 249, 620 Lasserre, J. B. 363 Laub, A. J. xix, 122, 331, 332, 357, 369, 699, 780, 876, 879 Laurie, C. 395 Lavoie, J. L. 430 Lawson, C. L. 438 Lawson, J. D. 474, 509 Lax, P. D. 173, 282, 502, 657 Lay, S. R. 101, 178, 697

973

AUTHOR INDEX

Lazarus, S. 347, 348 Leake, R. J. 201 Leclerc, B. 140 LeCouteur, K. J. 571, 748 Lee, A. 242, 340, 410 Lee, J. M. 396 Lee, S. H. 37, 57 Lee, W. Y. 390 Lehnigk, S. H. 793 Lei, T.-G. 498, 531, 532, 542, 551, 555, 556, 559, 561 Leite, F. S. 380, 739 Lemos, R. xvii, 752–754 Leonard, E. 710 Lesniak, L. xvii Letac, G. 584 Levinson, D. A. xvii Lew, J. S. 283, 284 Lewis, A. S. 119, 120, 178, 305, 363, 364, 484, 506, 693, 696 Lewis, D. C. 456 Li, C.-K. xvii, 296, 357, 394, 479, 485, 486, 504–506, 510, 535, 566, 586, 590, 596, 626, 640, 657, 669, 670 Li, C.-L. 63 Li, J. 42, 344 Li, J.-L. 32 Li, Q. 506, 637 Li, R.-C. 296 Li, X. 429 Li, Y.-L. 32 Li, Z. xvii, 793 Lieb, E. 625, 643 Lieb, E. H. 484, 485, 521, 530, 549, 571, 596, 625, 754 Ligh, S. 491 Likins, P. W. xvii Lilov, L. 740 Lim, J. S. 378 Lim, Y. 58, 474, 509, 510, 749, 877 Liman, A. 778 Lin, C.-S. 512, 552, 553, 575, 620 Lin, T.-P. 40 Lin, W. 42 Lin, W.-W. 396 Lin, Z. xvii, 123, 124, 354, 363, 396

Linden, H. 780, 781 Lipsky, L. xvii Littlewood, J. E. 84 Liu, B. xvii Liu, H. 28 Liu, J. 458, 485, 596 Liu, R.-W. 201 Liu, S. 63, 458, 551, 586, 589, 593 Liu, X. 130 Liu, Y. 424 Liz, E. 710 Loewy, R. 867 Logofet, D. O. xvii Lokesha, V. 171 Loparo, K. A. 364 Lopez, L. 240 Lopez-Valcarce, R. 746 Loss, M. 625 Lossers, O. P. 166 Lounesto, P. 204, 249 Lubich, C. 743 Luenberger, D. G. xvii, 555 Lundquist, M. 134, 488, 541 Lutkepohl, H. xix Lutoborski, A. 289 Lyubich, Y. I. 680 Ma, E.-C. 870 Ma, Y. xvii MacDuffee, C. C. 447, 451, 454, 458 Macfarlane, A. G. J. 879 Maciejowski, J. M. 783 Mackey, D. S. 252 Mackey, N. 252 Maddocks, J. H. 338, 554 Maeda, H. 773 Magnus, J. R. xvii, 405, 421, 425, 428, 429, 438, 458, 523, 525, 530, 553, 705 Magnus, W. 244, 743 Majindar, K. N. 559 Maligranda, L. 64, 69, 622–624 Malyshev, A. N. 767 Malzan, J. 380 Mangasarian, O. xvii Manjegani, S. M. 525 Mann, H. B. 555 Mann, S. 249, 620 Mansfield, L. E. xix

Mansour, M. 764, 772 Maradudin, A. A. 743 Marcus, M. xix, 60, 84, 148, 252, 354, 363, 444, 454, 458, 495, 532, 533, 543, 564, 590, 598, 655 Margaliot, M. 772 Markham, T. L. 161, 410, 424, 429, 485, 515, 587, 588, 595, 596 Markiewicz, A. 577 Marsaglia, G. 131, 178, 424, 426 Marsden, J. E. xvii, 214 Marshall, A. W. 49, 66, 67, 84, 171, 172, 175–178, 201, 355, 356, 364, 365, 454, 485, 486, 500, 561, 595, 596, 679 Martensson, K. 879 Martin, D. H. 879 Martinez, S. xvii, 301 Massey, J. Q. 142 Mastronardi, N. 135, 136, 140 Mathai, A. M. 705 Mathes, B. 395 Mathias, R. 357, 362, 458, 479, 484, 485, 492, 510, 563, 586, 593, 596, 632, 638, 649, 657, 661, 669, 670, 674, 675, 705, 748 Matic, M. 32, 65, 74 Matson, J. B. 879 Matsuda, T. 63 Maybee, J. S. 369 Mazorchuk, V. 637 McCarthy, J. E. 494 McCarthy, J. M. 793 McClamroch, N. H. 204 McCloskey, J. P. 231 McKeown, J. J. xviii Meehan, E. 306 Meenakshi, A. R. 583 Mehta, C. L. 549 Mellendorf, S. 293, 772 Melnikov, Y. A. xvii Mercer, P. R. 30, 74, 621, 623 Merikoski, J. K. 661 Merris, R. 301, 458, 507, 516, 550, 590 Meyer, C. 122, 228, 358, 366, 416

974 Meyer, C. D. 174, 190, 191, 231, 265, 305, 374, 390, 409, 411, 420, 429, 435, 438, 784, 793 Meyer, K. 876 Miao, J.-M. 429 Mickiewicz, A. 578 Mihalyffy, L. 429 Miller, K. S. 276, 451, 452, 557 Milliken, G. A. 579 Milovanovic, G. V. 700, 778, 779, 781 Minamide, N. 413 Minc, H. xix, 60, 84, 252, 354, 598, 655 Miranda, H. 595 Miranda, M. E. 365 Mirsky, L. xix, 211 Misra, P. 874 Missinghoff, M. J. xvii Mitchell, J. D. 383 Mitra, S. K. 421, 438, 458, 582, 595 Mitrinovic, D. S. 25, 41, 56, 77, 84, 171, 172, 295, 619, 620, 622–624, 778, 781 Mitter, S. K. 870 Mityagin, B. 504, 595 Miura, T. 56, 621 Mlynarski, M. 758 Moakher, M. 207, 394, 510, 703, 739, 755, 759 Moler, C. 759, 793 Molera, J. M. 378 Mond, B. 502, 510, 551, 590–592 Monov, V. V. 293 Moon, Y. S. 554 Moore, J. B. 511 Morawiec, A. 249, 740 Moreland, T. 506 Mori, H. 71, 551 Mori, T. 595 Morley, T. D. 509, 582 Morozov, A. 458 Moschovakis, Y. 15 Muckenhoupt, B. 341 Muir, T. 178 Muir, W. W. 485, 596 Mukherjea, K. 346 Munthe-Kaas, H. Z. 204, 739, 742, 743, 748

AUTHOR INDEX

Murphy, I. S. 541 Murray, R. M. xvii, 793 Nagar, D. K. xvii Najfeld, I. 748 Najman, B. 396 Nakamoto, R. 507, 513, 575, 646, 762 Nakamura, Y. 396 Nandakumar, K. 141 Narayan, D. A. 304 Narayan, J. Y. 304 Nataraj, S. xvii Nathanson, M. 643 Naylor, A. W. 77, 84, 620, 623, 683–685, 705 Needham, T. 84 Nelsen, R. B. 35 Nemirovski, A. 119, 178 Nersesov, S. G. 252, 777 Nett, C. N. 178 Neubauer, M. G. 540 Neudecker, H. xvii, 63, 405, 421, 425, 429, 438, 458, 523, 525, 551, 553, 705 Neumann, M. 238, 299, 374, 774, 776 Neuts, M. F. xvii Newcomb, R. W. 595 Newman, M. 143, 336, 720 Nguyen, T. 135, 237 Nicholson, D. W. 572 Niculescu, C. 23, 24, 38, 40, 46, 59, 60, 62, 69, 143, 531, 620, 700, 702 Niculescu, C. P. 24, 43, 59, 171, 781 Nielsen, M. A. xvii Niezgoda, M. 56, 499 Nishio, K. 217 Noble, B. xix, 178 Nomakuchi, K. 429 Nordstrom, K. 337, 503, 580 Norman, E. 743 Norsett, S. P. 204, 739, 743, 748 Nowak, M. T. 26, 30, 32, 37, 72 Nunemacher, J. 793 Nylen, P. 680 Oar, C. xvii, 879 Odell, P. L. 438

Ogawa, H. 413 Okubo, K. 381, 524, 526 Olesky, D. D. 369 Olkin, I. 49, 66, 67, 84, 171, 172, 175–178, 201, 348, 355, 356, 364, 365, 454, 485, 486, 500, 502, 504, 561, 595, 596, 679 Ortega, J. M. xix Ortner, B. 523 Osburn, S. L. 748 Ost, F. 457 Ostrowski, A. 867, 868 Ostrowski, A. M. 649 Oteo, J. A. 743 Ouellette, D. V. 515, 535, 596 Overdijk, D. A. 204 Paardekooper, M. H. C. 194 Pachter, M. 879 Paganini, F. xvii Paige, C. C. 336, 344, 520, 539, 553 Palanthandalam-Madapusi, H. 252, 548, 744 Paliogiannis, F. C. 747 Palka, B. P. 84 Pan, C.-T. 664 Pao, C. V. 758 Papastavridis, J. G. xvii, 496 Pappas, D. 414 Park, F. C. 793 Park, P. 364 Parker, D. F. 396 Parks, P. C. 772 Parthasarathy, K. R. 490, 646, 647, 763 Patel, R. V. 532, 533, 759, 874, 879 Pearce, C. E. 65, 70 Pearce, C. E. M. 65, 74 Pearcy, C. 346 Pease, M. C. 252 Pecaric, J. 32, 63–65, 70, 74, 623 Pecaric, J. E. 25, 58, 77, 84, 171, 295, 502, 510, 551, 590–592, 619, 620, 623, 624, 674 Pennec, X. 753 Peric, M. xvii

975

AUTHOR INDEX

Perlis, S. 178, 255, 256, 259, 307, 396 Persson, L.-E. 23, 24, 38, 40, 46, 59, 60, 62, 64, 69, 143, 531, 620, 700, 702 Peter, T. 172 Petersen, I. R. 553 Peterson, A. C. 879 Petz, D. 507, 510, 514, 753 Piepmeyer, G. G. 705 Pierce, S. 507, 516 Ping, J. 244 Pipes, L. A. 736, 737 Pittenger, A. O. xvii Plemmons, R. J. 190, 252, 301, 303, 774–776 Plischke, E. 767 Polik, I. 548 Politi, T. 240, 737, 740 Pollock, D. S. G. 705 Polya, G. 84 Polyak, B. T. 547, 548 Poon, E. 394 Poonen, B. 381, 491 Popa, D. 620 Popov, V. M. xvii, 793 Popovici, F. 24 Porta, H. 646 Porter, G. J. 215 Pourciau, B. H. 697 Pranesachar, C. R. 171 Prasolov, V. V. xix, 155, 162, 199, 200, 209, 225, 232, 249, 277, 279, 282, 295, 299, 340, 341, 349, 358, 364, 366, 371, 372, 374, 377, 381, 383, 392, 430, 454, 485, 521, 535, 537, 542, 558, 559, 587, 629, 645, 656, 658, 666, 703, 724 Prells, U. 454 Pritchard, A. J. 347, 355, 356, 609, 676, 701, 758, 762, 778 Pryce, J. D. 668 Przemieniecki, J. S. xvii Psarrakos, P. J. 381 Ptak, V. 279, 509, 510 Pukelsheim, F. 130, 458, 514, 576, 579, 589 Pullman, N. J. 793 Puntanen, S. 56, 423, 499, 551, 674

Putcha, M. S. 351, 367, 395 Pye, W. C. 286 Qi, F. 19, 42, 66 Qian, C. 42 Qian, R. X. 293 Qiao, S. 141, 219, 412, 438 Qiu, L. 455, 769 Queiro, J. F. 365 Quick, J. 379 Quinn, J. J. 12, 19, 304 Rabanovich, S. 637 Rabanovich, V. 395 Rabinowitz, S. 704 Rachidi, M. 737 Radjavi, H. 217, 347, 380, 383, 392, 395, 410 Raghavan, T. E. S. 299 Rajian, C. 583 Rajic, R. 623 Rakocevic, V. 219–222, 228, 367 Ran, A. 879 Ran, A. C. M. 873, 879 Rantzer, A. 348, 769 Rao, C. R. xvii, 304, 438, 458, 582, 595 Rao, D. K. M. 546 Rao, J. V. 429 Rao, M. B. xvii, 304, 458 Rasa, I. 620 Rassias, T. M. 69, 700, 778, 779, 781 Ratiu, R. S. 214 Ratiu, T. S. xvii Rauhala, U. A. 458 Raydan, M. 523 Recht, L. 646 Regalia, P. A. 458 Reinsch, M. W. 749 Reznick, B. 46 Richardson, T. J. 863, 879 Richeson, D. S. 16 Richmond, A. N. 751 Riedel, K. S. 413 Ringrose, J. R. 680 Rivlin, R. S. 283, 284 Robbin, J. W. 84, 125, 178, 229, 252, 307, 309, 348, 377, 378, 456, 703 Robinson, D. W. 237 Robinson, P. 523

Rockafellar, R. T. 178, 684, 693, 697, 705 Rodman, L. xvii, 212, 256, 307, 331, 335, 337, 366, 384, 396, 505, 506, 554, 698, 738, 860, 861, 879 Rogers, G. S. 705 Rohde, C. A. 429 Rohn, J. 630 Rojo, O. 56, 662 Rooin, J. 60 Room, T. G. 740 Ros, J. 743 Rose, D. J. 390 Rose, N. J. 374, 429, 435, 767 Rosenbrock, H. H. 836 Rosenfeld, M. xvii, 233 Rosenthal, P. 217, 349, 365, 392 Rosoiu, A. 171 Rossmann, W. 252 Rothblum, U. G. 298, 299, 306, 438 Rotman, J. J. 245, 391 Rowlinson, P. xvii Royle, G. xvii Rozsa, P. 154 Rubin, M. H. xvii Rubinstein, Y. A. 171 Rugh, W. J. 274, 275, 743, 816, 829, 879 Rump, S. M. 777 Ruskai, M. B. 521, 596 Russell, A. M. 493 Russell, D. L. 866 Rychlik, T. 56 Ryser, H. J. xvii, 137, 142, 550 Sa, E. M. 194, 205, 372 Saberi, A. xvii Sabourova, N. 609 Sadkane, M. 767 Sain, M. K. 879 Saito, K.-S. 623 Salamon, D. A. 456 Salmond, D. 151 Sandor, J. 24, 28, 29, 40, 60, 73 Sannuti, P. xvii Sarria, H. 661 Sastry, S. S. xvii, 793 Satnoianu, R. A. 171

976 Sattinger, D. H. 162, 187, 720, 724, 793 Saunders, M. 344 Sayed, A. H. xvii Schaub, H. 212, 738 Scherer, C. W. 860, 879 Scherk, P. 379 Schmoeger, C. 714, 715, 747, 749, 756 Schneider, H. 20, 140, 293, 454, 626, 867, 868 Scholkopf, B. 489 Scholz, D. 751 Schott, J. R. xvii, 19, 121, 229, 236, 391, 411, 413–415, 419, 420, 428, 449, 470, 486, 536, 557, 564, 565, 569–571, 585–587, 589, 605, 656, 678, 696 Schrader, C. B. 879 Schreiber, M. 421, 422, 506 Schreiner, R. 378 Schroder, B. S. W. 84 Schumacher, J. M. 879 Schwartz, H. M. 460, 503 Schwenk, A. J. 456 Scott, L. L. 244, 245 Searle, S. R. xvii, 157, 178, 458 Sebastian, P. 705 Sebastiani, P. 703 Seber, G. A. F. 74, 121, 127, 151, 167, 168, 198, 224, 226, 231, 236, 241, 252, 293, 296, 408, 409, 414, 431, 535, 550, 579, 588, 627, 665, 775 Seberry, J. 458 Seiringer, R. 530 Selig, J. M. xvii, 213, 249, 252 Sell, G. R. 77, 84, 620, 623, 683–685, 705 Semrl, P. 626 Seo, Y. 37, 57, 71, 551, 752, 754 Seoud, M. A. 252 Serre, D. 187, 239, 283, 427 Serre, J.-P. 244, 245 Seshadri, V. 773 Shafroth, C. 151, 207 Shah, S. L. 141 Shah, W. M. 778

AUTHOR INDEX

Shamash, Y. xvii, 123, 124, 354, 363, 396 Shapiro, H. 346, 383 Shapiro, H. M. 456, 585 Shaw, R. 204 Shen, S.-Q. 299, 457, 458, 675, 676 Sherali, H. D. 305, 684, 686 Shetty, C. M. 305, 684, 686 Shilov, G. E. xix Shuster, M. D. 214, 249, 738 Sibuya, Y. xvii, 339 Sijnave, B. 454, 458 Siljak, D. D. xvii, 252, 774 Silva, F. C. 565, 566, 869 Silva, J.-N. 122, 127, 163, 503 Simic, S. xvii Simoes, R. 869 Simsa, J. 19, 20, 26, 27, 29, 35, 39, 40, 43–48, 50, 51, 54, 57, 66 Singer, S. F. xvii Sivan, R. xvii Skelton, R. E. xvii, 793 Smale, S. xvii, 339 Smiley, D. M. 620 Smiley, M. F. 620 Smith, D. A. 51, 52, 245, 249 Smith, D. R. 737 Smith, H. A. 793 Smith, O. K. 289 Smith, P. J. 531 Smith, R. A. 870 Smith, R. L. 200, 212, 738 Smoktunowicz, A. 200, 429, 649 Smola, A. J. 489 Snell, J. L. xvii Snieder, R. 404 Snyders, J. 793, 818, 819, 879 So, W. 194, 524, 560, 565, 714, 720, 737, 747, 749, 755–757, 761 Soatto, S. xvii Sobczyk, G. 204, 249 Sontag, E. D. xvii, 773, 866 Sorensen, D. C. 867 Sourour, A. R. 383 Speed, R. P. 413 Spence, J. C. H. 743

Spiegel, E. 395 Spindelbock, K. 344, 406, 407, 436 Spivak, M. 249, 684 Sprossig, W. 249, 251, 740 Stanley, R. P. 12 Steeb, W.-H. 458, 755 Steele, J. M. 84 Stengel, R. F. xvii Stepniak, C. 579 Stern, R. J. 299, 774, 776 Stetter, H. J. xviii Stewart, G. W. xviii, xxxvi, 178, 331, 332, 344, 367, 391, 392, 554, 595, 625, 634, 635, 657, 672, 673, 678, 680, 697, 698 Stickel, E. U. 774, 793 Stiefel, E. 252 Stiller, L. xvii Stoer, J. 178, 636, 680, 697 Stojanoff, D. 632, 646 Stolarsky, K. B. 40, 41 Stone, M. G. 173 Stoorvogel, A. A. xvii, 879 Storey, C. xvii, 147, 396, 494, 771 Strang, G. xvii, xix, 135, 178, 237, 379, 491 Straskraba, I. 219–222 Strauss, M. J. 635 Strelitz, S. 765 Strichartz, R. S. 743 Strom, T. 758 Stuelpnagel, J. 738 Styan, G. P. H. 56, 63, 72, 127, 129–131, 134, 146, 178, 200, 218–221, 225, 228, 251, 336, 337, 351, 361, 362, 405, 422–424, 426, 429, 430, 499, 503, 520, 539, 551, 553, 579, 580, 589, 674 Subramanian, R. 476, 506, 749 Sullivan, R. P. 383 Sumner, J. S. 32 Sun, J. 331, 332, 344, 367, 391, 392, 554, 595, 625, 634, 635, 657, 672, 673, 680, 697, 698 Swamy, K. N. 595 Szechtman, F. 340 Szekeres, P. 249

977

AUTHOR INDEX

Szep, G. 392 Szirtes, T. xvii Szulc, T. 293, 425, 664 Takagi, H. 56, 621 Takahashi, Y. 620 Takahasi, S.-E. 56, 620, 621 Takane, Y. 429 Tam, T. Y. 244 Tamura, T. 623 Tao, Y. 567 Tapp, K. 214, 245, 247–249, 700 Tarazaga, P. 493, 661 Taussky, O. 178, 360, 396 Temesi, R. 510, 514 Tempelman, W. 204 ten Have, G. 381 Terlaky, T. 548 Terrell, R. E. 750 Thirring, W. E. 571 Thomas, J. A. xvii, 485, 536, 541, 556 Thompson, R. C. 75, 76, 178, 199, 384, 396, 534, 539, 543, 560, 565, 595, 640, 720, 743, 755, 757 Tian, Y. 127, 134, 146, 160, 165, 166, 192, 194, 216, 218–221, 225, 228, 231, 249, 251, 339, 350, 405–407, 409, 414–419, 422–426, 428, 429, 431, 434, 447, 451, 500, 511, 582, 584 Tismenetsky, M. 349, 371, 447, 615, 635, 680, 867, 868 Tisseur, F. 252 Toda, M. 532, 533, 759 Todd, J. 360 Toffoli, T. 379 Tominaga, M. 572, 752 Tonge, A. 630, 640 Torokhti, A. 672, 679 Trapp, G. E. 366, 486, 509, 510, 522, 582, 583, 595 Trefethen, L. N. xviii, 758, 786 Trenkler, D. 204, 286, 287, 504, 741 Trenkler, G. 56, 68, 192, 194, 195, 204, 219–221, 224, 226, 227, 251, 286,

287, 344, 366, 383, 406, 407, 409, 412, 417, 427, 429, 436–438, 458, 499, 504, 524, 577, 578, 741 Trentelman, H. L. xvii, 879 Treuenfels, P. 876 Trigiante, D. 387, 396, 491, 737 Tromborg, B. 207 Tropp, J. A. 635 Troschke, S.-O. 204, 251, 576, 579 Trustrum, G. B. 571 Tsatsomeros, M. 369 Tsatsomeros, M. J. 238, 498 Tsing, N.-K. 396, 554 Tsiotras, P. 204, 212, 738 Tsitsiklis, J. N. 786 Tung, S. H. 56 Turkington, D. A. 458 Turkmen, R. 674 Turnbull, H. W. 178 Tuynman, G. M. 748 Tyan, F. 374, 869 Uchiyama, M. 574, 755, 761 Udwadia, F. E. xvii Uhlig, F. 126, 379, 396, 554, 558, 559, 793 Underwood, E. E. 154 Upton, C. J. F. 493 Valentine, F. A. 178 Vamanamurthy, M. 25, 30, 32 Van Barel, M. 135, 136, 140, 279, 307 van Dam, A. xvii, 215 Van Den Driessche, P. 369 van der Driessche, P. 777 van der Merwe, R. 741 Van Dooren, P. 396 Van Huffel, S. 679 Van Loan, C. F. xvii, xviii, 138, 390, 458, 672, 710, 748, 759, 769, 793 Van Overschee, P. 879 Van Pelt, T. 548 van Schagen, F. 879 Vandebril, R. 135, 136, 140 Vandenberghe, L. xvii, 178 Vandewalle, J. 679

Varadarajan, V. S. 252, 720, 748, 749, 793 Varah, J. M. 664 Vardulakis, A. I. G. xvii, 307 Varga, R. S. xviii, 293, 299, 775 Vasic, P. M. 41, 56, 84, 172, 622 Vasudeva, H. L. 586, 592, 595 Vavrin, Z. 279 Vein, P. R. 178 Vein, R. xvii, 178 Veljan, D. 46, 171, 174 Venugopal, R. 252, 744 Vermeer, J. 377 Veselic, K. 359, 396, 769 Vetter, W. J. 458 Vidyasagar, M. 758 Visick, G. 585, 589, 591–594 Volenec, V. 58, 171, 510 Vong, S.-W. 641 Vowe, M. 32 Vreugdenhil, R. 879 Vuorinen, M. 25, 30, 32 Wada, S. 66, 70, 620 Wagner, D. G. 765 Waldenstrom, S. 207 Walter, G. G. xvii, 777 Wang, B. 570, 571, 672 Wang, B.-Y. 209, 571, 580, 587, 595, 673 Wang, C.-L. 63, 71 Wang, D. 476 Wang, F. 244 Wang, G. 141, 219, 412, 431, 438 Wang, J. 485, 596 Wang, J.-H. 395 Wang, L. 56, 655 Wang, L.-C. 63 Wang, Q.-G. 295 Wang, X. 763 Wang, Y. 786 Wang, Y. W. 756, 871 Wanner, G. 743 Wansbeek, T. 458 Ward, A. J. B. 349 Ward, R. C. 775 Warga, J. 685, 686 Warner, W. H. 204

978 Waterhouse, W. E. 148, 149 Waters, S. R. 242 Wathen, A. J. 523 Watkins, W. 304, 495, 532, 533, 537, 543, 564, 586, 590 Watson, G. S. 63, 72 Weaver, J. R. 242, 342 Weaver, O. L. 162, 187, 720, 724, 793 Webb, J. H. 49 Webster, R. 178 Wegert, E. 786 Wegmann, R. 654 Wei, J. 743 Wei, M. 344, 415 Wei, Y. 141, 219, 412, 429, 431, 438, 745 Weinberg, D. A. 396 Weinert, H. L. 344 Weiss, G. H. 743 Weiss, M. xvii, 879 Wenzel, D. 641 Wermuth, E. M. E. 747, 762 Werner, H.-J. 579 Wesseling, P. xvii Westlake, J. R. xviii Wets, R. J. B. 693, 705 Weyrauch, M. 751 White, J. E. 390 Wiegmann, N. A. 374 Wiener, Z. 785 Wigner, E. P. 502 Wilcox, R. M. xviii, 750 Wildon, M. J. 252 Wilhelm, F. 755 Wilker, J. B. 32 Wilkinson, J. H. xviii Willems, J. C. 879 Williams, E. R. 413 Williams, J. P. 347, 380, 410, 474 Williams, K. S. 619 Wilson, D. A. 680 Wilson, P. M. H. 175, 214

AUTHOR INDEX

Wimmer, H. K. 279, 337, 505, 642, 861, 867, 870, 879 Wirth, F. 767 Witkowski, A. 25, 30 Wittenburg, J. 740 Witzgall, C. 178, 680, 697 Wolkowicz, H. 56, 194, 205, 351, 361, 362, 372 Wolovich, W. A. 879 Wonenburger, M. J. 383 Wong, C. S. 382, 525, 671 Wonham, W. M. xvii, 285, 710, 865, 879 Woo, C.-W. 498, 531, 532, 542, 551, 555, 556, 559, 561 Wrobel, I. 380 Wu, C.-F. 579 Wu, P. Y. 226, 382–384, 394, 395, 596, 770 Wu, S. 46, 65, 70, 171 Wu, Y.-D. 171 Xi, B.-Y. 580, 673 Xiao, Z.-G. 59 Xie, Q. 458 Xu, C. 76, 157, 520, 526, 539, 620 Xu, D. 214 Xu, H. 714, 769 Xu, Z. 76, 157, 520, 539, 620 Yakub, A. 49 Yamagami, S. 640 Yamazaki, T. 57, 575 Yanagi, K. 526 Yanase, M. M. 502 Yang, B. 42, 69 Yang, X. 764, 765 Yang, Z. P. 524, 526 Yau, S. F. 588, 592 Yau, S. S.-T. 737 Ye, Q. 396 Yeadon, F. I. 204 Yellen, J. xvii

Young, D. M. xviii Young, N. J. 665 Young, P. M. 769 Zakai, M. 793, 818, 819, 879 Zamfir, R. 781 Zanna, A. 204, 739, 742, 743, 748 Zassenhaus, H. 396 Zelevinsky, A. V. 458 Zemanek, J. 762 Zhan, X. 177, 356, 394, 455, 477, 485, 492, 535, 560–562, 566, 567, 573, 582–584, 589, 591, 592, 594, 595, 603, 637, 638, 644, 647, 671, 674, 752, 763 Zhang, C.-E. 19 Zhang, F. 76, 136, 137, 142, 157, 166, 199, 200, 209, 235, 251, 336, 340, 362, 373, 390, 430, 497, 498, 502, 503, 507, 511, 516, 518, 520, 524, 525, 531, 532, 538, 539, 542, 551–553, 555, 556, 559–561, 570, 571, 578, 580, 586, 587, 592, 593, 595, 619, 620, 672, 673 Zhang, L. 431 Zhang, Z.-H. 59, 171 Zhao, K. 340 Zheng, B. 134, 429 Zhong, Q.-C. 223 Zhou, K. xvii, 307, 664, 668, 860 Zhu, H. 41 Zhu, L. 28, 32 Zielke, G. 629, 677 Zlobec, S. 400 Zwart, H. J. 758 Zwas, G. 661 Zwillinger, D. 82, 171, 175, 277, 391

Index Symbols 0n×m n × m zero matrix definition, 90 1n×m n × m ones matrix definition, 92 2 × 2 matrices commutator Fact 2.18.1, 161 2 × 2 matrix discrete-time asymptotically stable matrix Fact 11.21.1, 782 eigenvalue inequality Fact 8.18.1, 559 singular value Fact 5.11.31, 357 2×2 positive-semidefinite matrix square root Fact 8.9.6, 496 2 × 2 trace Fact 2.12.9, 137 3 × 3 matrix equality trace Fact 4.9.5, 283 3 × 3 symmetric matrix eigenvalue Fact 4.10.3, 288 A⊕B Kronecker sum

definition, 443 A#B geometric mean definition, 508 A#α B generalized geometric mean definition, 510 A−1 inverse matrix definition, 109

parallel sum definition, 581 Aˆ∗ reverse complex conjugate transpose definition, 96 A◦α Schur power definition, 444

GL

A+ generalized inverse definition, 397

rs

A1/2 positive-semidefinite matrix square root definition, 474

A ≤ B generalized L¨ owner partial ordering definition, 577 A≤B rank subtractivity partial ordering definition, 129 ∗

A≤B star partial ordering definition, 130 i

A←b column replacement definition, 87 A◦B Schur product definition, 444 A⊗B Kronecker product definition, 440 A:B

A# group generalized inverse definition, 403 AA adjugate definition, 114 AD Drazin generalized inverse definition, 401 AL left inverse definition, 106 AR right inverse definition, 106

980

AT

AT transpose definition, 94 ˆ

J

0 I −I 0

definition, 271

definition, 183

AT reverse transpose definition, 96

J 2n 0

A[i;j] submatrix definition, 114

K(x) cross-product matrix definition, 90

A⊥ complementary idempotent matrix definition, 190 complementary projector definition, 190

N standard nilpotent matrix definition, 92

B(p, q) Bezout matrix definition, 277 C(p) companion matrix definition, 309 C∗ complex conjugate transpose definition, 95 D|A Schur complement definition, 401 Ei,j,n×m n × m matrix with a single unit entry definition, 92

In −In 0

definition, 183

Nn n × n standard nilpotent matrix definition, 92 PA,B pencil definition, 330 Pn,m Kronecker permutation matrix definition, 442 V (λ1, . . . , λn ) Vandermonde matrix definition, 387 [A, B] commutator definition, 89 Bε (x) open ball definition, 681

Ei,j matrix with a single unit entry definition, 92

Cn×m n × m complex matrices definition, 86

H(g) Hankel matrix definition, 279

F real or complex numbers definition, 4

In identity matrix definition, 91

F(s) rational functions

F[s] polynomials with coefficients in F definition, 253 Fn×m n × m real or complex matrices definition, 86 Fn×m [s] polynomial matrices with coefficients in Fn×m definition, 256 Fn×m(s) n × m rational transfer functions definition, 271 Fn×m prop (s) n × m proper rational transfer functions definition, 271 Fprop(s) proper rational functions definition, 271 R complex numbers definition, 3 real numbers definition, 3 Rn×m n × m real matrices definition, 86 Sε (x) sphere definition, 681 Hn n × n Hermitian matrices definition, 459 Nn

co S n × n positivesemidefinite matrices definition, 459 Pn n × n positive-definite matrices definition, 459 Im x imaginary part definition, 4 In A inertia definition, 267 Re x real part definition, 4

L{x(t)} Laplace transform definition, 710 N(A) null space definition, 102

Aq,p H¨ older-induced norm definition, 608

R(A) range definition, 101

xp H¨ older norm definition, 598

S⊥ orthogonal complement definition, 99

yD dual norm definition, 625

C(A, B) controllable subspace definition, 809 H Hamiltonian definition, 853

Su(A) unstable subspace definition, 729

H(G) Markov block-Hankel matrix definition, 827

U(A, C) unobservable subspace definition, 800

Hi,j,k (G) Markov block-Hankel matrix definition, 826 Jl(q) real Jordan matrix definition, 315 K(A, B) controllability matrix definition, 809

Aσp Schatten norm definition, 602

O(A, C) observability matrix definition, 800

Ss (A) asymptotically stable subspace definition, 729

Hl(q) hypercompanion matrix definition, 314

row norm definition, 611

X∼ complement definition, 3 Y\X relative complement definition, 2 Ap H¨ older norm definition, 601 AF Frobenius norm definition, 601 Acol column norm definition, 611 Arow

adA adjoint operator definition, 89 aff S affine hull definition, 98 bd S boundary definition, 682 bdS S relative boundary definition, 682 χA characteristic equation definition, 262 χA,B characteristic polynomial definition, 332 cl S closure definition, 681 clS S relative closure definition, 682 co S convex hull

981

982

co S

definition, 98 coco S convex conical hull definition, 98 coli (A) column definition, 87 cone S conical hull definition, 98 dcone S dual cone definition, 99 def A defect definition, 104 deg p degree definition, 253 det A determinant definition, 112 diag(A1 , . . . , Ak ) block-diagonal matrix definition, 181 diag(a1, . . . , an ) diagonal matrix definition, 181 dim S dimension of a set definition, 98 (A) lower bound definition, 613 q,p (A) H¨ older-induced lower bound definition, 614 Iˆn reverse permutation matrix definition, 91

ind A index of a matrix definition, 190 indA(λ) index of an eigenvalue definition, 321 int S interior definition, 681 intS S relative interior definition, 681 λ1(A) maximum eigenvalue definition, 262 minimum eigenvalue definition, 262 λi(A) eigenvalue definition, 262 log(A) matrix logarithm definition, 718 mroots(p) multiset of roots definition, 254 mspec(A) multispectrum definition, 262

dual cone definition, 99 rank A rank definition, 104 rank G normal rank for a rational transfer function definition, 271 rank P normal rank for a polynomial matrix definition, 257 reldeg G relative degree definition, 271 revdiag(a1, . . . , an ) reverse diagonal matrix definition, 181 dmax(A) maximum diagonal entry definition, 87 dmin(A) minimum diagonal entry definition, 87

μA minimal polynomial definition, 269

di(A) diagonal entry definition, 87

ν−(A), ν0 (A) inertia definition, 267

roots(p) set of roots definition, 254

C complex conjugate definition, 95

rowi(A) row definition, 87

π prime numbers Fact 1.9.11, 23

sig A signature definition, 267

polar S

σmax (A)

Abelian group maximum singular value definition, 328

ith column of the identity matrix definition, 92

σmin (A) minimum singular value definition, 328

ei,n ith column of the n × n identity matrix definition, 92

σi(A) singular value definition, 328 sign x sign definition, 97 sign α sign definition, xxi spabs(A) spectral abscissa definition, 267 spec(A) spectrum definition, 262 sprad(A) spectral abscissa definition, 267 tr A trace definition, 94 vcone(D, x0 ) variational cone definition, 685 vec A column-stacking operator definition, 439 |x| absolute value definition, 96 eA matrix exponential definition, 707 ei

f

(k)

(x0 ) kth derivative definition, 688

f (x0 ) derivative definition, 686 kth derivative definition, 688 n-tuple definition, 3 x >> 0 positive vector definition, 86 x ≥≥ 0 nonnegative vector definition, 86 SO(3) logarithm Fact 11.15.10, 759 SO(n) eigenvalue Fact 5.11.2, 350 amultA(λ) algebraic multiplicity definition, 262 circ(a0 , . . . , an−1) circulant matrix definition, 388 exp(A) matrix exponential definition, 707 glb(S) greatest lower bound definition, 8

983

gmultA geometric multiplicity definition, 267 inf (S) infimum definition, 9 lub(S) least upper bound definition, 8 multp(λ) multiplicity definition, 254 sh(A, B) shorted operator definition, 582 sup(S) supremum definition, 9 D+f (x0 ; ξ) one-sided directional differential definition, 685 (1)-inverse definition, 398 determinant Fact 6.5.28, 430 left inverse Proposition 6.1.3, 398 right inverse Proposition 6.1.2, 398 (1,2)-inverse definition, 398

A Abel quintic polynomial Fact 3.23.4, 243 Abelian group definition Definition 3.3.3, 186 equivalence relation Proposition 3.4.2, 188

984

absolute norm

absolute norm monotone norm Proposition 9.1.2, 597 absolute sum norm definition, 599 absolute value Frobenius norm Fact 9.13.11, 661 H¨ older-induced norm Fact 9.8.26, 631 inequality Fact 1.13.24, 49 Fact 1.13.25, 50 irreducible matrix Fact 3.22.6, 241 matrix, 96 maximum singular value Fact 9.13.10, 661 reducible matrix Fact 3.22.6, 241 scalar inequality Fact 1.13.1, 42 Fact 1.13.12, 47 Fact 1.14.3, 51 Schatten norm Fact 9.13.11, 661 spectral radius Fact 4.11.17, 305 vector, 96 absolute-value function Niculescu’s inequality Fact 1.12.19, 36 absolute-value matrix positive-semidefinite matrix Fact 8.9.1, 495 absolutely convergent sequence convergent sequence Proposition 10.2.7, 683 Proposition 10.2.9, 683 absolutely convergent series

definition Definition 10.2.6, 683 Definition 10.2.8, 683 acyclic graph definition Definition 1.6.3, 10 Aczel’s inequality norm inequality Fact 9.7.4, 618 quadratic inequality Fact 1.18.19, 70 additive compound asymptotically stable polynomial Fact 11.17.12, 765 additive decomposition diagonalizable matrix Fact 5.9.5, 339 Hermitian matrix Fact 3.7.29, 198 nilpotent matrix Fact 5.9.5, 339 orthogonal matrix Fact 5.19.2, 394 Fact 5.19.3, 394 unitary matrix Fact 5.19.1, 394 adjacency matrix definition Definition 3.2.1, 184 graph of a matrix Proposition 3.2.5, 185 inbound Laplacian matrix Theorem 3.2.2, 185 Laplacian matrix Theorem 3.2.2, 185 Theorem 3.2.3, 185 Fact 4.11.7, 300 outbound Laplacian matrix Theorem 3.2.2, 185 outdegree matrix Fact 3.21.2, 240 row-stochastic matrix

Fact 3.21.2, 240 symmetric graph Fact 3.21.1, 240 adjacent Definition 1.6.1, 9 adjoint norm definition Fact 9.8.8, 627 dual norm Fact 9.8.8, 627 H¨ older-induced norm Fact 9.8.10, 628 adjoint operator commutator Fact 2.18.5, 162 Fact 2.18.6, 162 adjugate basic properties, 115 characteristic polynomial Fact 4.9.8, 284 cross product Fact 6.5.16, 427 defect Fact 2.16.7, 155 definition, 114 derivative Fact 10.12.8, 703 Fact 10.12.10, 703 determinant Fact 2.14.27, 151 Fact 2.16.3, 153 Fact 2.16.5, 154 Fact 2.16.6, 155 diagonalizable matrix Fact 5.14.4, 370 eigenvalue Fact 4.10.9, 291 eigenvector Fact 5.14.25, 373 elementary matrix Fact 2.16.1, 153 factor Fact 2.16.9, 155 Frobenius norm

almost nonnegative matrix Fact 9.8.15, 629 generalized inverse Fact 6.3.6, 404 Fact 6.3.7, 405 Fact 6.5.16, 427 Hermitian matrix Fact 3.7.10, 193 iterated Fact 2.16.5, 154 matrix powers Fact 4.9.8, 284 matrix product Fact 2.16.10, 155 nilpotent matrix Fact 6.3.6, 404 null space Fact 2.16.7, 155 outer-product perturbation Fact 2.16.3, 153 partitioned matrix Fact 2.14.27, 151 range Fact 2.16.7, 155 rank Fact 2.16.7, 155 Fact 2.16.8, 155 scalar factor Fact 2.16.5, 154 singular value Fact 5.11.36, 358 skew-Hermitian matrix Fact 3.7.10, 193 Fact 3.7.11, 194 skew-symmetric matrix Fact 4.9.21, 286 spectrum Fact 4.10.9, 291 trace Fact 4.9.8, 284 transpose Fact 2.16.5, 154 affine closed half space closed half space Fact 2.9.6, 120 definition, 99

affine function definition, 88 affine hull closure Fact 10.8.11, 693 constructive characterization Theorem 2.3.5, 100 convex hull Fact 2.9.3, 119 convex set Theorem 10.3.2, 684 Fact 10.8.8, 693 definition, 98 linear mapping Fact 2.10.4, 125 affine hyperplane affine subspace Fact 2.9.6, 120 definition, 99 determinant Fact 2.20.3, 167 affine mapping Hermitian matrix Fact 3.7.14, 195 normal matrix Fact 3.7.14, 195 affine open half space definition, 99 open half space Fact 2.9.6, 120 affine subspace affine hull of image Fact 2.9.26, 123 affine hyperplane Fact 2.9.6, 120 definition, 97 image under linear mapping Fact 2.9.26, 123 left inverse Fact 2.9.26, 123 span Fact 2.9.7, 120 Fact 2.20.4, 167 Fact 10.8.12, 694 subspace

985

Fact 2.9.8, 120 Afriat spectrum of a product of projectors Fact 5.12.15, 365 Akers maximum singular value of a product of elementary projectors Fact 9.14.1, 665 algebraic multiplicity block-triangular matrix Proposition 5.5.13, 324 definition Definition 4.4.4, 262 geometric multiplicity Proposition 5.5.3, 322 index of an eigenvalue Proposition 5.5.6, 322 orthogonal matrix Fact 5.11.2, 350 outer-product matrix Fact 5.14.3, 370 almost nonnegative matrix asymptotically stable matrix Fact 11.19.5, 776 definition Definition 3.1.4, 182 group-invertible matrix Fact 11.19.4, 775 irreducible matrix Fact 11.19.2, 774 Lyapunov-stable matrix Fact 11.19.4, 775 matrix exponential Fact 11.19.1, 774 Fact 11.19.2, 774

986

almost nonnegative matrix

N-matrix Fact 11.19.3, 775 Fact 11.19.5, 776 nonnegative matrix Fact 11.19.1, 774 positive matrix Fact 11.19.2, 774 alternating group definition Proposition 3.3.6, 187 group Fact 3.23.4, 243 Alzer’s inequality sum of integers Fact 1.11.31, 32 Amemiya’s inequality Schur product Fact 8.22.41, 593 Anderson rank of a tripotent matrix Fact 2.10.23, 127 Ando convex function Proposition 8.6.17, 596 inertia of congruent, normal matrices Fact 5.10.17, 348 angle definition, 93 angle between complex vectors definition, 94 angular velocity vector quaternions Fact 11.11.16, 740 antieigenvalue definition Fact 9.8.37, 632 antisymmetric graph Laplacian Fact 3.21.1, 240 antisymmetric relation definition

Definition 1.5.8, 8 one-sided cone induced by Proposition 2.3.6, 101 positive-semidefinite matrix Proposition 8.1.1, 460 aperiodic graph Definition 1.6.3, 10 nonnegative matrix Fact 4.11.4, 298 Araki positive-semidefinite matrix inequality Fact 8.12.23, 528 Araki-Lieb-Thirring inequality positive-semidefinite matrix inequality Fact 8.12.22, 528 arc definition, 9 area parallelogram Fact 2.20.17, 173 Fact 9.7.5, 620 polygon Fact 2.20.14, 173 triangle Fact 2.20.7, 168 Fact 2.20.8, 168 Fact 2.20.10, 169 arithmetic mean Carleman’s inequality Fact 1.17.41, 64 geometric mean Fact 1.12.37, 40 Fact 1.17.21, 58 Fact 1.17.23, 58 Fact 1.17.24, 59 Fact 1.17.25, 59 Fact 1.17.26, 59 Fact 1.17.27, 59 identric mean Fact 1.12.37, 40

logarithmic mean Fact 1.17.26, 59 mixed arithmeticgeometric mean inequality Fact 1.17.40, 63 Muirhead’s theorem Fact 1.17.25, 59 positive-definite matrix Fact 8.10.34, 506 scalar inequality Fact 1.13.6, 43 Fact 1.13.7, 43 Fact 1.13.8, 44 Fact 1.13.9, 44 Fact 1.13.10, 46 arithmetic-mean inequality harmonic mean Fact 1.17.16, 57 Fact 1.17.17, 57 arithmetic-mean– geometric-mean inequality alternative form Fact 1.17.34, 62 difference Fact 1.17.29, 60 harmonic mean Fact 1.17.15, 56 Jensen’s inequality Fact 1.10.4, 24 main form Fact 1.17.14, 56 Fact 1.17.28, 59 Popoviciu Fact 1.17.29, 60 positive-definite matrix Fact 8.13.8, 535 quartic equality Fact 1.14.5, 51 Rado Fact 1.17.29, 60 ratio Fact 1.17.29, 60

asymptotically stable matrix refined weighted arithmetic-mean– geometric-mean inequality Fact 1.17.33, 62 reverse inequality Fact 1.17.18, 57 Fact 1.17.19, 57 sextic equality Fact 1.15.1, 52 variation Fact 1.12.13, 35 weighted arithmetic-mean– geometric-mean inequality Fact 1.17.32, 61 arithmetic-mean– harmonic-mean inequality scalar inequality Fact 1.17.38, 63 associative equalities definition, 89 associativity composition Proposition 1.4.1, 5 asymptotic stability eigenvalue Proposition 11.8.2, 727 input-to-state stability Fact 12.20.18, 866 linear dynamical system Proposition 11.8.2, 727 Lyapunov equation Corollary 11.9.1, 730 matrix exponential Proposition 11.8.2, 727 nonlinear system Theorem 11.7.2, 725 asymptotically stable equilibrium definition Definition 11.7.1, 725

asymptotically stable matrix 2 × 2 matrix Fact 11.18.35, 773 almost nonnegative matrix Fact 11.19.5, 776 asymptotically stable polynomial Proposition 11.8.4, 728 Cayley transform Fact 11.21.9, 783 compartmental matrix Fact 11.19.6, 776 controllability Fact 12.20.5, 864 controllability Gramian Proposition 12.7.9, 819 Corollary 12.7.10, 820 controllable pair Proposition 12.7.9, 819 Corollary 12.7.10, 820 controllably asymptotically stable Proposition 12.8.5, 821 cyclic matrix Fact 11.18.25, 770 definition Definition 11.8.1, 727 detectability Proposition 12.5.5, 807 Corollary 12.5.6, 807 diagonalizable over R Fact 11.17.10, 765 discrete-time asymptotically stable matrix Fact 11.21.9, 783 dissipative matrix Fact 11.18.21, 769 Fact 11.18.37, 774 factorization Fact 11.18.22, 770 integral Lemma 11.9.2, 731 inverse matrix

987

Fact 11.18.15, 768 Kronecker sum Fact 11.18.32, 773 Fact 11.18.33, 773 Fact 11.18.34, 773 linear matrix equation Proposition 11.9.3, 731 logarithmic derivative Fact 11.18.11, 767 Lyapunov equation Proposition 11.9.5, 733 Corollary 11.9.4, 732 Corollary 11.9.7, 734 Corollary 12.4.4, 806 Corollary 12.5.6, 807 Corollary 12.7.4, 819 Corollary 12.8.6, 822 Fact 12.21.7, 868 Fact 12.21.17, 870 matrix exponential Lemma 11.9.2, 731 Fact 11.18.8, 767 Fact 11.18.9, 767 Fact 11.18.10, 767 Fact 11.18.15, 768 Fact 11.18.18, 769 Fact 11.18.19, 769 Fact 11.21.8, 783 minimal realization Definition 12.9.17, 829 negative-definite matrix Fact 11.18.30, 773 nonsingular N-matrix Fact 11.19.5, 776 normal matrix Fact 11.18.37, 774 observability Gramian Corollary 12.4.10, 807 observable pair Proposition 12.4.9, 806 Corollary 12.4.10, 807 observably asymptotically stable

988

asymptotically stable matrix

Proposition 12.5.5, 807 perturbation Fact 11.18.16, 768 positive-definite matrix Proposition 11.9.5, 733 Proposition 12.4.9, 806 Corollary 11.9.7, 734 Fact 11.18.21, 769 secant condition Fact 11.18.29, 772 sign of entry Fact 11.19.5, 777 sign stability Fact 11.19.5, 777 similar matrices Fact 11.18.4, 766 skew-Hermitian matrix Fact 11.18.30, 773 spectrum Fact 11.18.13, 768 square root Fact 11.18.36, 774 stability radius Fact 11.18.17, 768 stabilizability Proposition 12.8.5, 821 Corollary 12.8.6, 822 subdeterminant Fact 11.19.1, 776 trace Fact 11.18.31, 773 tridiagonal matrix Fact 11.18.24, 770 Fact 11.18.25, 770 Fact 11.18.26, 771 Fact 11.18.27, 771 Fact 11.18.28, 772 asymptotically stable polynomial additive compound Fact 11.17.12, 765 asymptotically stable matrix Proposition 11.8.4, 728 definition Definition 11.8.3, 728 even polynomial

Fact 11.17.6, 764 Hermite-Biehler theorem Fact 11.17.6, 764 interlacing theorem Fact 11.17.6, 764 Kharitonov’s theorem Fact 11.17.13, 766 Kronecker sum Fact 11.17.11, 765 odd polynomial Fact 11.17.6, 764 polynomial coefficients Fact 11.17.2, 763 Fact 11.17.3, 764 Fact 11.17.7, 764 Fact 11.17.8, 765 Fact 11.17.10, 765 Fact 11.17.11, 765 Fact 11.17.12, 765 reciprocal argument Fact 11.17.4, 764 Schur product of polynomials Fact 11.17.9, 765 subdeterminant Fact 11.18.23, 770 asymptotically stable subspace definition, 729 asymptotically stable transfer function minimal realization Proposition 12.9.18, 829 SISO entry Proposition 12.9.19, 829 average positive-semidefinite matrix Fact 5.19.5, 395 averaged limit integral Fact 10.11.6, 700

B Baker-CampbellHausdorff series matrix exponential Proposition 11.4.7, 720 Baker-CampbellHausdorff-Dynkin expansion time-varying dynamics Fact 11.13.4, 743 balanced realization definition Definition 12.9.20, 829 minimal realization Proposition 12.9.21, 830 balancing transformation existence Corollary 8.3.4, 465 Bandila’s inequality triangle Fact 2.20.11, 169 Barnett asymptotic stability of a tridiagonal matrix Fact 11.18.24, 770 Barnett factorization Bezout matrix Fact 4.8.6, 277 barycentric coordinates conjugate parameters Fact 1.18.11, 68 definition, 97 basis definition, 98 Beckner’s two-point inequality powers

binomial series Fact 1.12.15, 36 Fact 9.9.35, 643 Bellman quadratic form inequality Fact 8.15.8, 551 Ben-Israel generalized inverse Fact 6.3.34, 411 Bencze arithmetic-mean– geometric-mean– logarithmic-mean inequality Fact 1.17.26, 59 Bendixson’s theorem eigenvalue bound Fact 5.11.21, 354 Fact 9.11.8, 655 Berezin trace of a convex function Fact 8.12.34, 531 Bergstrom positive-definite matrix determinant Fact 8.13.16, 536 Bergstrom’s inequality quadratic form Fact 8.11.3, 514 Fact 8.15.19, 553 Bernoulli matrix Vandermonde matrix Fact 5.16.3, 387 Bernoulli’s inequality scalar inequality Fact 1.11.1, 25 Fact 1.11.2, 25 Bernstein matrix Vandermonde matrix Fact 5.16.3, 387 Bernstein’s inequality matrix exponential Fact 11.15.4, 756

Berwald polynomial root bound Fact 11.20.12, 781 Bessel’s inequality norm inequality Fact 9.7.4, 618 Bessis-Moussa-Villani trace conjecture derivative of a matrix exponential Fact 8.12.32, 530 power of a positivesemidefinite matrix Fact 8.12.31, 530 Bezout equation coprime polynomials Fact 4.8.5, 277 Bezout identity right coprime polynomial matrices Theorem 4.7.14, 274 Bezout matrix coprime polynomials Fact 4.8.6, 277 Fact 4.8.7, 279 Fact 4.8.8, 279 definition Fact 4.8.6, 277 distinct roots Fact 4.8.9, 280 factorization Fact 5.15.24, 381 polynomial roots Fact 4.8.9, 280 Bhat integral of a Gaussian density Fact 11.13.16, 746 Bhatia Schatten norm inequality Fact 9.9.45, 645

989

unitarily invariant norm inequality Fact 9.9.44, 645 bialternate product compound matrix Fact 7.5.17, 452 Kronecker product, 458 bidiagonal matrix singular value Fact 5.11.47, 362 biequivalent matrices congruent matrices Proposition 3.4.5, 189 definition Definition 3.4.3, 188 Kronecker product Fact 7.4.12, 445 rank Proposition 5.1.3, 309 similar matrices Proposition 3.4.5, 189 Smith form Theorem 5.1.1, 309 Corollary 5.1.2, 309 unitarily similar matrices Proposition 3.4.5, 189 bijective function definition, 84 bilinear function definition, 687 Binet-Cauchy formula determinant Fact 2.13.5, 140 Binet-Cauchy theorem compound of a matrix product Fact 7.5.17, 452 binomial equality sum Fact 1.9.1, 16 Fact 1.9.2, 19 binomial series infinite series

990

binomial series

Fact 1.20.8, 79 Birkhoff doubly stochastic matrix Fact 3.11.3, 205 bivector parallelogram Fact 9.7.5, 620 block definition, 87 block decomposition Hamiltonian Proposition 12.17.5, 856 minimal realization Proposition 12.9.10, 825 block-circulant matrix circulant matrix Fact 3.18.3, 234 Drazin generalized inverse Fact 6.6.1, 431 generalized inverse Fact 6.5.2, 423 inverse matrix Fact 2.17.6, 160 block-diagonal matrix companion matrix Proposition 5.2.8, 312 Lemma 5.2.2, 311 definition Definition 3.1.3, 181 geometric multiplicity Proposition 5.5.13, 324 Hermitian matrix Fact 3.7.8, 193 least common multiple Lemma 5.2.7, 312 matrix exponential Proposition 11.2.8, 713 maximum singular value Fact 5.11.33, 357

minimal polynomial Lemma 5.2.7, 312 normal matrix Fact 3.7.8, 193 shifted-unitary matrix Fact 3.11.8, 205 similar matrices Theorem 5.3.2, 314 Theorem 5.3.3, 315 singular value Fact 8.19.9, 567 Fact 8.19.10, 567 Fact 9.14.21, 670 Fact 9.14.25, 671 skew-Hermitian matrix Fact 3.7.8, 193 unitary matrix Fact 3.11.8, 205 block-Hankel matrix definition Definition 3.1.3, 181 Hankel matrix Fact 3.18.3, 234 Markov block-Hankel matrix definition, 826 block-Kronecker product Kronecker product, 458 block-Toeplitz matrix definition Definition 3.1.3, 181 Toeplitz matrix Fact 3.18.3, 234 block-triangular matrix algebraic multiplicity Proposition 5.5.13, 324 controllable dynamics Theorem 12.6.8, 811 controllable subspace Proposition 12.6.9, 812

Proposition 12.6.10, 812 controllably asymptotically stable Proposition 12.7.3, 816 detectability Proposition 12.5.3, 807 determinant Fact 2.14.8, 145 index of a matrix Fact 5.14.31, 374 Fact 6.6.14, 435 inverse matrix Fact 2.17.1, 159 maximum singular value Fact 5.11.32, 357 minimal polynomial Fact 4.10.13, 292 observable dynamics Theorem 12.3.8, 802 observably asymptotically stable Proposition 12.4.3, 805 spectrum Proposition 5.5.13, 324 stabilizability Proposition 12.8.3, 820 unobservable subspace Proposition 12.3.9, 802 Proposition 12.3.10, 802 blocking zero definition Definition 4.7.10, 273 rational transfer function Definition 4.7.4, 271 Smith-McMillan form Proposition 4.7.11, 273 Blundon triangle inequality Fact 2.20.11, 169 blunt cone

Cartesian decomposition definition, 97 Bonami’s inequality powers Fact 1.12.16, 36 Fact 9.7.20, 624

Fact 2.20.13, 172 Brauer spectrum bounds Fact 4.10.22, 295

Borchers trace norm of a matrix difference Fact 9.9.24, 640

Brouwer fixed-point theorem image of a continuous function Corollary 10.3.11, 685

Borobia asymptotically stable polynomial Fact 11.17.8, 765

Brown trace of a convex function Fact 8.12.34, 531

both definition, 1

Browne’s theorem eigenvalue bound Fact 5.11.21, 354 Fact 5.11.22, 354 Fact 9.11.7, 655

boundary definition, 682 interior Fact 10.8.7, 693 union Fact 10.9.2, 695 boundary relative to a set definition, 682 bounded set continuous function Theorem 10.3.10, 685 Corollary 10.3.11, 685 definition, 682 image under linear mapping Fact 9.8.1, 627 open ball Fact 10.8.2, 693 Bourbaki polynomial root bound Fact 11.20.4, 778 Bourin spectral radius of a product Fact 8.19.26, 572 Brahmagupta’s formula quadrilateral

Brownian motion positive-semidefinite matrix Fact 8.8.3, 489 Buzano’s inequality Cauchy-Schwarz inequality Fact 1.19.2, 74 norm inequality Fact 9.7.4, 618

C Callan determinant of a partitioned matrix Fact 2.14.15, 147 Callebaut monotonicity Fact 1.18.1, 66 Callebaut’s inequality refined Cauchy-Schwarz inequality Fact 1.18.16, 70 canonical form

991

definition, 5 canonical mapping definition, 5 Cantor intersection theorem intersection of closed sets Fact 10.9.12, 696 Cardano’s trigonometric solution cubic polynomial Fact 4.10.3, 288 eigenvalue Fact 4.10.3, 288 cardinality definition, 2 inclusion-exclusion principle Fact 1.7.5, 12 union Fact 1.7.5, 12 Carleman’s inequality arithmetic mean Fact 1.17.41, 64 Carlson inertia of a Hermitian matrix Fact 12.21.4, 867 Carlson inequality sum of powers Fact 1.17.42, 64 Carmichael polynomial root bound Fact 11.20.10, 781 Cartesian decomposition determinant Fact 8.13.4, 534 Fact 8.13.11, 534 eigenvalue Fact 5.11.21, 354 Hermitian matrix Fact 3.7.27, 197

992

Cartesian decomposition

Fact 3.7.28, 197 Fact 3.7.29, 198 positive-semidefinite matrix Fact 9.9.40, 644 Schatten norm Fact 9.9.37, 643 Fact 9.9.38, 643 Fact 9.9.39, 644 Fact 9.9.40, 644 singular value Fact 8.19.7, 566 skew-Hermitian matrix Fact 3.7.27, 197 Fact 3.7.28, 197 Fact 3.7.29, 198 spectrum Fact 5.11.21, 354 Cartesian product definition, 3 cascade interconnection definition, 843 transfer function Proposition 12.13.2, 843 cascaded systems geometric multiplicity Fact 12.22.15, 874 Cauchy polynomial root bound Fact 11.20.12, 781 Cauchy interlacing theorem Hermitian matrix eigenvalue Lemma 8.4.4, 468 Cauchy matrix determinant Fact 3.22.9, 241 Fact 3.22.10, 242 positive-definite matrix

Fact 8.8.16, 494 Fact 12.21.18, 870 positive-semidefinite matrix Fact 8.8.7, 491 Fact 8.8.9, 492 Fact 12.21.19, 871 Cauchy’s estimate polynomial root bound Fact 11.20.6, 779 Cauchy-Schwarz inequality Buzano’s inequality Fact 1.19.2, 74 Callebaut’s inequality Fact 1.18.16, 70 De Bruijn’s inequality Fact 1.18.20, 70 determinant Fact 8.13.23, 537 Frobenius norm Corollary 9.3.9, 607 inner product bound Corollary 9.1.7, 600 McLaughlin’s inequality Fact 1.18.17, 70 Milne’s inequality Fact 1.18.15, 69 Ozeki’s inequality Fact 1.18.23, 71 Polya-Szego inequality Fact 1.18.21, 71 positive-semidefinite matrix Fact 8.11.14, 517 Fact 8.11.15, 517 Fact 8.15.9, 551 vector case Fact 1.18.9, 68 Cayley transform asymptotically stable matrix Fact 11.21.9, 783

cross product Fact 3.11.29, 210 cross-product matrix Fact 3.10.1, 202 definition Fact 3.11.21, 208 discrete-time asymptotically stable matrix Fact 11.21.9, 783 Hamiltonian matrix Fact 3.20.12, 239 Hermitian matrix Fact 3.11.21, 208 orthogonal matrix Fact 3.11.22, 208 Fact 3.11.23, 209 Fact 3.11.29, 210 positive-definite matrix Fact 8.9.31, 498 skew-Hermitian matrix Fact 3.11.22, 208 skew-symmetric matrix Fact 3.11.22, 208 Fact 3.11.23, 209 Fact 3.11.29, 210 symplectic matrix Fact 3.20.12, 239 unitary matrix Fact 3.11.22, 208 Cayley-Hamilton theorem characteristic polynomial Theorem 4.4.7, 265 generalized version Fact 4.9.7, 284 center subgroup commutator Fact 2.18.10, 163 centralizer commutator Fact 2.18.9, 163 Fact 7.5.2, 450 commuting matrices

circulant matrix Fact 5.14.21, 373 Fact 5.14.23, 373 centrohermitian matrix complex conjugate transpose Fact 3.22.11, 242 definition Definition 3.1.2, 180 generalized inverse Fact 6.3.30, 410 matrix product Fact 3.22.12, 242 centrosymmetric matrix definition Definition 3.1.2, 180 matrix product Fact 3.22.12, 242 matrix transpose Fact 3.22.11, 242 Cesaro summable discrete-time Lyapunov-stable matrix Fact 11.21.13, 784 chain definition Definition 1.6.3, 10 chaotic order matrix logarithm Fact 8.20.1, 574 characteristic equation definition, 262 characteristic polynomial 2 × 2 matrix Fact 4.9.1, 282 3 × 3 matrix Fact 4.9.2, 283 adjugate Fact 4.9.8, 284 Cayley-Hamilton theorem Theorem 4.4.7, 265 companion matrix Proposition 5.2.1, 310

Corollary 5.2.4, 312 Corollary 5.2.5, 312 cross-product matrix Fact 4.9.20, 286 Fact 4.9.21, 286 cyclic matrix Proposition 5.5.14, 325 definition Definition 4.4.1, 261 degree Proposition 4.4.3, 262 derivative Lemma 4.4.8, 266 eigenvalue Proposition 4.4.6, 264 equalities Proposition 4.4.5, 263 generalized inverse Fact 6.3.19, 408 Hamiltonian matrix Fact 4.9.22, 287 Fact 4.9.24, 287 inverse matrix Fact 4.9.9, 284 Leverrier’s algorithm Proposition 4.4.9, 266 matrix product Proposition 4.4.10, 266 Corollary 4.4.11, 267 minimal polynomial Fact 4.9.25, 288 monic Proposition 4.4.3, 262 outer-product matrix Fact 4.9.17, 285 Fact 4.9.19, 286 output feedback Fact 12.22.13, 874 partitioned matrix Fact 4.9.15, 285 Fact 4.9.16, 285 Fact 4.9.18, 286 Fact 4.9.19, 286 Fact 4.9.23, 287 Fact 4.9.24, 287 similar matrices Fact 4.9.10, 284 similarity invariant

993

Proposition 4.4.2, 262 Proposition 4.6.2, 270 skew-Hermitian matrix Fact 4.9.14, 285 skew-symmetric matrix Fact 4.9.13, 285 Fact 4.9.20, 286 Fact 4.9.21, 286 Fact 5.14.33, 375 sum of derivatives Fact 4.9.11, 284 upper block-triangular matrix Fact 4.10.12, 291 upper triangular matrix Fact 4.10.10, 291 Chebyshev’s inequality rearrangement Fact 1.18.3, 66 Chen form tridiagonal matrix Fact 11.18.27, 772 child Definition 1.6.1, 9 Cholesky decomposition existence Fact 8.9.38, 500 circle complex numbers Fact 2.20.12, 172 circulant matrix block-circulant matrix Fact 3.18.3, 234 companion matrix Fact 5.16.7, 388 cyclic permutation matrix Fact 5.16.7, 388 Fourier matrix Fact 5.16.7, 388

994

circulant matrix

group Fact 3.23.4, 245 primary circulant matrix Fact 5.16.7, 388 spectrum Fact 5.16.7, 388 Clarkson inequalities complex numbers Fact 1.20.2, 75 Schatten norm Fact 9.9.34, 642 CLHP closed left half plane definition, 4 Clifford algebra real matrix representation Fact 3.24.1, 247 Cline factorization expression for the group generalized inverse Fact 6.6.13, 435 generalized inverse of a matrix product Fact 6.4.14, 414 closed half space affine closed half space Fact 2.9.6, 120 definition, 99 closed relative to a set continuous function Theorem 10.3.4, 684 definition Definition 10.1.4, 682 closed set complement Fact 10.8.4, 693 continuous function Theorem 10.3.10, 685 Corollary 10.3.5, 684 Corollary 10.3.11, 685 definition

Definition 10.1.3, 681 image under linear mapping Fact 10.9.9, 696 intersection Fact 10.9.11, 696 Fact 10.9.12, 696 polar Fact 2.9.4, 119 positive-semidefinite matrix Fact 10.8.18, 694 subspace Fact 10.8.21, 694 union Fact 10.9.11, 696 closed-loop spectrum detectability Lemma 12.16.17, 854 Hamiltonian Proposition 12.16.14, 853 maximal solution of the Riccati equation Proposition 12.18.2, 860 observability Lemma 12.16.17, 854 observable eigenvalue Lemma 12.16.16, 854 Riccati equation Proposition 12.16.14, 853 Proposition 12.18.2, 860 Proposition 12.18.3, 860 Proposition 12.18.7, 862 closure affine hull Fact 10.8.11, 693 complement Fact 10.8.6, 693 convex hull Fact 10.8.13, 694

convex set Fact 10.8.8, 693 Fact 10.8.19, 694 definition Definition 10.1.3, 681 smallest closed set Fact 10.8.3, 693 subset Fact 10.9.1, 695 union Fact 10.9.2, 695 closure point definition Definition 10.1.3, 681 closure point relative to a set definition Definition 10.1.4, 682 closure relative to a set definition Definition 10.1.4, 682 Cochran’s theorem sum of projectors Fact 3.13.25, 229 codomain definition, 4 cofactor definition, 114 determinant expansion Proposition 2.7.5, 114 cogredient diagonalization commuting matrices Fact 8.17.1, 558 definition, 465 diagonalizable matrix Fact 8.17.2, 558 Fact 8.17.3, 558 positive-definite matrix Theorem 8.3.1, 465 Fact 8.17.5, 558 positive-semidefinite matrix

commutator Theorem 8.3.5, 466 unitary matrix Fact 8.17.1, 558 cogredient diagonalization of positive-definite matrices Weierstrass Fact 8.17.2, 558 cogredient transformation Hermitian matrix Fact 8.17.4, 558 Fact 8.17.6, 559 simultaneous diagonalization Fact 8.17.4, 558 Fact 8.17.6, 559 simultaneous triangularization Fact 5.17.9, 392 Cohn polynomial root bound Fact 11.20.12, 781 colinear determinant Fact 2.20.1, 167 Fact 2.20.5, 168 Fact 2.20.9, 169 colleague form definition, 396 column definition, 86 column norm definition, 611 H¨ older-induced norm Fact 9.8.21, 630 Fact 9.8.23, 631 Kronecker product Fact 9.9.61, 648 partitioned matrix Fact 9.8.11, 628 row norm Fact 9.8.10, 628

spectral radius Corollary 9.4.10, 611 column vector definition, 85 column-stacking operator, see vec column-stochastic matrix definition Definition 3.1.4, 182 common divisor definition, 255 common eigenvector commuting matrices Fact 5.14.27, 374 norm equality Fact 9.9.33, 642 simultaneous triangularization Fact 5.17.1, 391 subspace Fact 5.14.26, 373 common multiple definition, 256 commutant commutator Fact 2.18.9, 163 Fact 7.5.2, 450 commutator 2 × 2 matrices Fact 2.18.1, 161 adjoint operator Fact 2.18.5, 162 Fact 2.18.6, 162 center subgroup Fact 2.18.10, 163 centralizer Fact 2.18.9, 163 Fact 7.5.2, 450 convergent sequence Fact 11.14.9, 749 definition, 89 derivative of a matrix Fact 11.14.11, 749

995

determinant Fact 2.18.7, 162 dimension Fact 2.18.9, 163 Fact 2.18.10, 163 Fact 2.18.11, 163 Fact 7.5.2, 450 equalities Fact 2.12.19, 138 Fact 2.18.4, 162 factorization Fact 5.15.33, 383 Frobenius norm Fact 9.9.26, 641 Fact 9.9.27, 641 Hermitian matrix Fact 3.8.1, 199 Fact 3.8.3, 199 Fact 9.9.30, 641 idempotent matrix Fact 3.12.16, 217 Fact 3.12.17, 217 Fact 3.12.30, 221 Fact 3.12.31, 221 Fact 3.12.32, 222 Fact 3.15.6, 217 infinite product Fact 11.14.18, 751 involutory matrix Fact 3.15.6, 231 lower triangular matrix Fact 3.17.11, 233 matrix exponential Fact 11.14.9, 749 Fact 11.14.11, 749 Fact 11.14.12, 749 Fact 11.14.13, 750 Fact 11.14.14, 750 Fact 11.14.15, 750 Fact 11.14.16, 750 Fact 11.14.17, 751 Fact 11.14.18, 751 maximum eigenvalue Fact 9.9.30, 641 Fact 9.9.31, 642 maximum singular value Fact 9.9.29, 641

996

commutator

Fact 9.14.9, 667 nilpotent matrix Fact 3.12.16, 217 Fact 3.17.11, 233 Fact 3.17.12, 233 Fact 3.17.13, 233 normal matrix Fact 3.8.6, 200 Fact 3.8.7, 200 Fact 9.9.31, 642 power Fact 2.18.2, 161 powers Fact 2.18.3, 161 projector Fact 3.13.23, 228 Fact 6.4.18, 415 Fact 9.9.9, 637 range Fact 6.4.18, 415 rank Fact 3.12.31, 221 Fact 3.13.23, 228 Fact 5.17.5, 392 Fact 6.3.9, 405 Schatten norm Fact 9.9.27, 641 series Fact 11.14.17, 751 simultaneous triangularization Fact 5.17.5, 392 Fact 5.17.6, 392 skew-Hermitian matrix Fact 3.8.1, 199 Fact 3.8.4, 200 skew-symmetric matrix Fact 3.8.5, 200 spectrum Fact 5.12.14, 365 spread Fact 9.9.30, 641 Fact 9.9.31, 642 submultiplicative norm Fact 9.9.8, 637 subspace

Fact 2.18.9, 163 Fact 2.18.10, 163 Fact 2.18.12, 163 sum Fact 2.18.12, 163 trace Fact 2.18.1, 161 Fact 2.18.2, 161 Fact 5.9.20, 341 triangularization Fact 5.17.5, 392 unitarily invariant norm Fact 9.9.29, 641 Fact 9.9.30, 641 Fact 9.9.31, 642 upper triangular matrix Fact 3.17.11, 233 zero diagonal Fact 3.8.2, 199 zero trace Fact 2.18.11, 163 commutator realization Shoda’s theorem Fact 5.9.20, 341 commuting matrices centralizer Fact 5.14.21, 373 Fact 5.14.23, 373 cogredient diagonalization Fact 8.17.1, 558 common eigenvector Fact 5.14.27, 374 cyclic matrix Fact 5.14.21, 373 diagonalizable matrix Fact 5.17.8, 392 dimension Fact 5.10.15, 347 Fact 5.10.16, 347 Drazin generalized inverse Fact 6.6.4, 431 Fact 6.6.5, 431 eigenvector

Fact 5.14.24, 373 generalized Cayley-Hamilton theorem Fact 4.9.7, 284 Hermitian matrix Fact 5.14.28, 374 idempotent matrix Fact 3.16.5, 231 Kronecker sum Fact 7.5.4, 450 matrix exponential Proposition 11.1.5, 709 Corollary 11.1.6, 709 Fact 11.14.1, 746 Fact 11.14.5, 748 nilpotent matrix Fact 3.17.9, 233 Fact 3.17.10, 233 normal matrix Fact 3.7.28, 197 Fact 3.7.29, 198 Fact 5.14.28, 374 Fact 5.17.7, 392 Fact 11.14.5, 748 polynomial representation Fact 5.14.21, 373 Fact 5.14.22, 373 Fact 5.14.23, 373 positive-definite matrix Fact 8.9.41, 500 positive-semidefinite matrix Fact 8.20.4, 514, 576 projector Fact 6.4.36, 419 Fact 8.10.23, 504 Fact 8.10.25, 504 range-Hermitian matrix Fact 6.4.29, 417 Fact 6.4.30, 418 rank subtractivity partial ordering Fact 8.20.4, 576 simple matrix Fact 5.14.22, 373

complementary subspaces simultaneous diagonalization Fact 8.17.1, 558 simultaneous triangularization Fact 5.17.4, 392 spectral radius Fact 5.12.11, 365 spectrum Fact 5.12.14, 365 square root Fact 5.18.1, 393 Fact 8.10.25, 504 star partial ordering Fact 2.10.36, 130 time-varying dynamics Fact 11.13.4, 743 triangularization Fact 5.17.4, 392 compact set continuous function Theorem 10.3.8, 684 convergent subsequence Theorem 10.2.5, 683 convex hull Fact 10.8.15, 694 definition, 682 existence of minimizer Corollary 10.3.9, 685 companion form matrix discrete-time semistable matrix Fact 11.21.21, 786 companion matrix block-diagonal matrix Proposition 5.2.8, 312 Lemma 5.2.2, 311 bottom, right, top, left Fact 5.16.1, 385 characteristic polynomial Proposition 5.2.1, 310 Corollary 5.2.4, 312

Corollary 5.2.5, 312 circulant matrix Fact 5.16.7, 388 cyclic matrix Fact 5.16.5, 388 definition, 309 elementary divisor Theorem 5.2.9, 313 example Example 5.3.6, 316 Example 5.3.7, 317 hypercompanion matrix Corollary 5.3.4, 315 Lemma 5.3.1, 314 inverse matrix Fact 5.16.2, 386 minimal polynomial Proposition 5.2.1, 310 Corollary 5.2.4, 312 Corollary 5.2.5, 312 nonnegative matrix Fact 4.11.14, 304 oscillator Fact 5.14.34, 375 similar matrices Fact 5.16.5, 388 singular value Fact 5.11.30, 356 Vandermonde matrix Fact 5.16.4, 387 compartmental matrix asymptotically stable matrix Fact 11.19.6, 776 Lyapunov-stable matrix Fact 11.19.6, 776 semistable matrix Fact 11.19.6, 776 compatible norm induced norm Proposition 9.4.3, 608 compatible norms definition, 604 H¨ older norm Proposition 9.3.5, 605 Schatten norm

997

Proposition 9.3.6, 606 Corollary 9.3.7, 606 Corollary 9.3.8, 607 submultiplicative norm Proposition 9.3.1, 604 trace norm Corollary 9.3.8, 607 complement closed set Fact 10.8.4, 693 closure Fact 10.8.6, 693 definition, 3 interior Fact 10.8.6, 693 open set Fact 10.8.4, 693 relatively closed set Fact 10.8.5, 693 relatively open set Fact 10.8.5, 693 complement of a graph Definition 1.6.2, 9 complement of a relation definition Definition 1.5.4, 7 complementary relation relation Proposition 1.5.5, 7 complementary submatrix defect Fact 2.11.20, 135 complementary subspaces complex conjugate transpose Fact 3.12.1, 215 definition, 98 group-invertible matrix Corollary 3.5.8, 191 idempotent matrix

998

complementary subspaces

Proposition 3.5.3, 190 Proposition 3.5.4, 190 Fact 3.12.1, 215 Fact 3.12.33, 223 index of a matrix Proposition 3.5.7, 191 partitioned matrix Fact 3.12.33, 223 projector Fact 3.13.24, 228 simultaneous Fact 2.9.23, 122 stable subspace Proposition 11.8.8, 729 sum of dimensions Corollary 2.3.2, 99 unstable subspace Proposition 11.8.8, 729 completely solid set convex set Fact 10.8.9, 693 definition, 682 open ball Fact 10.8.1, 693 positive-semidefinite matrix Fact 10.8.18, 694 solid set Fact 10.8.9, 693 complex conjugate determinant Fact 2.19.8, 166 Fact 2.19.9, 166 matrix exponential Proposition 11.2.8, 713 partitioned matrix Fact 2.19.9, 166 similar matrices Fact 5.9.33, 345 complex conjugate of a matrix definition, 95 complex conjugate of a vector definition, 93 complex conjugate transpose

complementary subspaces Fact 3.12.1, 215 definition, 95 determinant Fact 9.11.1, 653 diagonalizable matrix Fact 5.14.4, 370 equality Fact 2.10.33, 129 Fact 2.10.34, 129 factorization Fact 5.15.23, 381 generalized inverse Fact 6.3.9, 405 Fact 6.3.10, 405 Fact 6.3.12, 406 Fact 6.3.15, 407 Fact 6.3.16, 407 Fact 6.3.17, 407 Fact 6.3.21, 408 Fact 6.3.26, 409 Fact 6.3.27, 410 Fact 6.4.8, 413 Fact 6.4.9, 414 Fact 6.4.10, 414 Fact 6.6.17, 436 Fact 6.6.18, 437 Fact 6.6.19, 437 group generalized inverse Fact 6.6.11, 434 Hermitian matrix Fact 3.7.13, 195 Fact 5.9.10, 340 Fact 6.6.19, 437 idempotent matrix Fact 5.9.23, 342 Kronecker product Proposition 7.1.3, 440 left inverse Fact 2.15.1, 152 Fact 2.15.2, 152 matrix exponential Proposition 11.2.8, 713 Fact 11.15.4, 756 Fact 11.15.6, 757

maximum singular value Fact 8.18.3, 559 Fact 8.19.11, 567 Fact 8.22.10, 586 nonsingular matrix Fact 2.16.30, 158 norm Fact 9.8.8, 627 normal matrix Fact 5.14.29, 374 Fact 6.3.15, 407 Fact 6.3.16, 407 Fact 6.6.11, 434 Fact 6.6.18, 437 partitioned matrix Fact 6.5.3, 423 positive-definite matrix Fact 8.9.40, 500 projector Fact 3.13.1, 223 range Fact 6.5.3, 423 Fact 8.7.2, 486 range-Hermitian matrix Fact 3.6.4, 192 Fact 6.3.10, 405 Fact 6.6.17, 436 Schur product Fact 8.22.9, 586 similarity transformation Fact 5.9.10, 340 Fact 5.15.4, 377 singular value Fact 5.11.34, 357 subspace Fact 2.9.28, 123 trace Fact 8.12.4, 524 Fact 8.12.5, 524 Fact 9.13.15, 662 unitarily invariant norm Fact 9.8.30, 632

cone unitarily left-equivalent matrices Fact 5.10.18, 348 Fact 5.10.19, 348 unitarily right-equivalent matrices Fact 5.10.18, 348 unitarily similar matrices Fact 5.9.22, 342 Fact 5.9.23, 342 complex conjugate transpose of a vector definition, 93 complex inequality Petrovich Fact 1.20.2, 75 complex matrix block 2 × 2 representation Fact 2.19.3, 164 complex conjugate Fact 2.19.4, 165 determinant Fact 2.19.3, 164 Fact 2.19.10, 166 partitioned matrix Fact 2.19.4, 165 Fact 2.19.5, 165 Fact 2.19.6, 165 Fact 2.19.7, 165 Fact 3.11.10, 206 positive-definite matrix Fact 3.7.9, 193 positive-semidefinite matrix Fact 3.7.9, 193 rank Fact 2.19.3, 164 complex numbers 2 × 2 representation Fact 2.19.1, 164 circle Fact 2.20.12, 172

Clarkson inequalities Fact 1.20.2, 75 Dunkl-Williams inequality Fact 1.20.5, 77 equalities Fact 1.20.1, 74 Fact 1.20.2, 75 equality Fact 1.20.4, 77 equilateral triangle Fact 2.20.6, 168 exponential function Fact 1.20.6, 78 inequalities Fact 1.20.1, 74 Fact 1.20.2, 75 Fact 1.20.5, 77 inequality Fact 1.14.4, 51 infinite series Fact 1.20.8, 79 logarithm function Fact 1.20.7, 79 Maligranda inequality Fact 1.20.5, 77 Massera-Schaffer inequality Fact 1.20.5, 77 parallelogram law Fact 1.20.2, 75 polarization identity Fact 1.20.2, 75 quadratic formula Fact 1.20.3, 77 trigonometric function Fact 1.21.4, 84

999

Fact 5.9.24, 342 T-congruent diagonalization Fact 5.9.24, 342 unitary matrix Fact 5.9.24, 342 component definition, 85 composition associativity Proposition 1.4.1, 5 definition, 4 composition of functions one-to-one function Fact 1.7.17, 15 onto function Fact 1.7.17, 15 compound matrix matrix product Fact 7.5.17, 452 compound of a matrix product Binet-Cauchy theorem Fact 7.5.17, 452 comrade form definition, 396 concave function definition Definition 8.6.14, 479 function composition Lemma 8.6.16, 480 nonincreasing function Lemma 8.6.16, 480

complex symmetric Jordan form similarity transformation Fact 5.15.2, 377 Fact 5.15.3, 377

condition number linear system solution Fact 9.15.1, 676 Fact 9.15.2, 677 Fact 9.15.3, 677

complex-symmetric matrix T-congruence

cone blunt definition, 97

1000

cone

cone of image Fact 2.9.26, 123 constructive characterization Theorem 2.3.5, 100 definition, 97 dictionary ordering Fact 2.9.31, 124 image under linear mapping Fact 2.9.26, 123 intersection Fact 2.9.9, 120 left inverse Fact 2.9.26, 123 lexicographic ordering Fact 2.9.31, 124 one-sided definition, 97 pointed definition, 97 quadratic form Fact 8.14.11, 547 Fact 8.14.13, 548 Fact 8.14.14, 548 sum Fact 2.9.9, 120 variational definition, 685 confederate form definition, 396 congenial matrix definition, 396 congruence equivalence relation Fact 5.10.3, 345 generalized inverse Fact 8.21.5, 578 congruence transformation normal matrix Fact 5.10.17, 348 congruent matrices biequivalent matrices Proposition 3.4.5, 189 definition

Definition 3.4.4, 188 Hermitian matrix Proposition 3.4.5, 189 Corollary 5.4.7, 321 inertia Corollary 5.4.7, 321 Fact 5.8.22, 338 Kronecker product Fact 7.4.13, 446 matrix classes Proposition 3.4.5, 189 positive-definite matrix Proposition 3.4.5, 189 Corollary 8.1.3, 461 positive-semidefinite matrix Proposition 3.4.5, 189 Corollary 8.1.3, 461 range-Hermitian matrix Proposition 3.4.5, 189 Fact 5.9.8, 339 skew-Hermitian matrix Proposition 3.4.5, 189 skew-symmetric matrix Fact 3.7.34, 198 Fact 5.9.18, 341 Sylvester’s law of inertia, 320 symmetric matrix Fact 5.9.18, 341 unit imaginary matrix Fact 3.7.34, 198 conical hull definition, 98 conjugate parameters barycentric coordinates Fact 1.18.11, 68 connected graph Definition 1.6.3, 10 irreducible matrix Fact 4.11.3, 297 walk

Fact 4.11.2, 297 constant polynomial definition, 253 contained definition, 3 continuity spectrum Fact 10.11.8, 701 Fact 10.11.9, 701 continuity of roots coefficients polynomial Fact 10.11.2, 700 continuous function bounded set Theorem 10.3.10, 685 Corollary 10.3.11, 685 closed relative to a set Theorem 10.3.4, 684 closed set Theorem 10.3.10, 685 Corollary 10.3.5, 684 Corollary 10.3.11, 685 compact set Theorem 10.3.8, 684 convex function Theorem 10.3.2, 684 Fact 10.11.11, 701 convex set Theorem 10.3.10, 685 Corollary 10.3.11, 685 definition Definition 10.3.1, 684 differentiable function Proposition 10.4.4, 687 existence of minimizer Corollary 10.3.9, 685 fixed-point theorem Theorem 10.3.10, 685 Corollary 10.3.11, 685 jointly continuous function Fact 10.11.4, 700 linear function

controllable dynamics Corollary 10.3.3, 684 maximization Fact 10.11.3, 700 open relative to a set Theorem 10.3.4, 684 open set Corollary 10.3.5, 684 open set image Theorem 10.3.7, 684 pathwise-connected set Fact 10.11.5, 700 continuous-time control problem LQG controller Fact 12.23.6, 878 continuously differentiable function definition, 687 contractive matrix complex conjugate transpose Fact 3.22.7, 241 definition Definition 3.1.2, 180 partitioned matrix Fact 8.11.24, 520 positive-definite matrix Fact 8.11.13, 517 contradiction definition, 1 contragredient diagonalization definition, 465 positive-definite matrix Theorem 8.3.2, 465 Corollary 8.3.4, 465 positive-semidefinite matrix Theorem 8.3.6, 466 Corollary 8.3.8, 466 contrapositive definition, 1

controllability asymptotically stable matrix Fact 12.20.5, 864 cyclic matrix Fact 12.20.13, 865 diagonal matrix Fact 12.20.12, 865 final state Fact 12.20.4, 864 geometric multiplicity Fact 12.20.14, 865 Gramian Fact 12.20.17, 865 input matrix Fact 12.20.15, 865 PBH test Theorem 12.6.19, 815 positive-semidefinite matrix Fact 12.20.6, 864 positive-semidefinite ordering Fact 12.20.8, 864 range Fact 12.20.7, 864 shift Fact 12.20.10, 865 shifted dynamics Fact 12.20.9, 864 skew-symmetric matrix Fact 12.20.5, 864 stabilization Fact 12.20.17, 865 Sylvester’s equation Fact 12.21.14, 870 transpose Fact 12.20.16, 865 controllability Gramian asymptotically stable matrix Proposition 12.7.9, 819 Corollary 12.7.10, 820 controllably asymptotically stable Proposition 12.7.3, 816

1001

Proposition 12.7.4, 818 Proposition 12.7.5, 819 Proposition 12.7.6, 819 Proposition 12.7.7, 819 frequency domain Corollary 12.11.5, 840 H2 norm Corollary 12.11.4, 839 Corollary 12.11.5, 840 L2 norm Theorem 12.11.1, 838 controllability matrix controllable pair Theorem 12.6.18, 815 definition, 809 rank Corollary 12.6.3, 809 Sylvester’s equation Fact 12.21.13, 869 controllability pencil definition Definition 12.6.12, 813 Smith form Proposition 12.6.15, 813 Smith zeros Proposition 12.6.16, 814 uncontrollable eigenvalue Proposition 12.6.13, 813 uncontrollable spectrum Proposition 12.6.16, 814 controllable canonical form definition, 822 equivalent realizations Corollary 12.9.9, 825 realization Proposition 12.9.3, 822 controllable dynamics block-triangular matrix

1002

controllable dynamics

Theorem 12.6.8, 811 orthogonal matrix Theorem 12.6.8, 811 controllable eigenvalue controllable subspace Proposition 12.6.17, 814 controllable pair asymptotically stable matrix Proposition 12.7.9, 819 Corollary 12.7.10, 820 controllability matrix Theorem 12.6.18, 815 cyclic matrix Fact 5.14.7, 370 eigenvalue placement Proposition 12.6.20, 815 equivalent realizations Proposition 12.9.8, 824 Markov block-Hankel matrix Proposition 12.9.11, 827 minimal realization Proposition 12.9.10, 825 Corollary 12.9.15, 829 positive-definite matrix Theorem 12.6.18, 815 rank Fact 5.14.8, 371 controllable subspace block-triangular matrix Proposition 12.6.9, 812 Proposition 12.6.10, 812 controllable eigenvalue Proposition 12.6.17, 814 definition

Definition 12.6.1, 808 equivalent expressions Lemma 12.6.2, 808 final state Fact 12.20.3, 864 full-state feedback Proposition 12.6.5, 810 identity-matrix shift Lemma 12.6.7, 811 invariant subspace Corollary 12.6.4, 810 nonsingular matrix Proposition 12.6.10, 812 orthogonal matrix Proposition 12.6.9, 812 projector Lemma 12.6.6, 810 controllably asymptotically stable asymptotically stable matrix Proposition 12.8.5, 821 block-triangular matrix Proposition 12.7.3, 816 controllability Gramian Proposition 12.7.3, 816 Proposition 12.7.4, 818 Proposition 12.7.5, 819 Proposition 12.7.6, 819 Proposition 12.7.7, 819 definition Definition 12.7.1, 816 full-state feedback Proposition 12.7.2, 816 Lyapunov equation Proposition 12.7.3, 816 orthogonal matrix Proposition 12.7.3, 816 rank Proposition 12.7.4, 818 Proposition 12.7.5, 819 stabilizability Proposition 12.8.5, 821 convergent sequence

absolutely convergent sequence Proposition 10.2.7, 683 Proposition 10.2.9, 683 closure point Proposition 10.2.4, 683 commutator Fact 11.14.9, 749 discrete-time semistable matrix Fact 11.21.16, 785 generalized inverse Fact 6.3.34, 411 Fact 6.3.35, 411 Hermitian matrix Fact 11.14.7, 749 Fact 11.14.8, 749 inverse matrix Fact 2.16.29, 158 matrix exponential Proposition 11.1.3, 708 Fact 11.14.7, 749 Fact 11.14.8, 749 Fact 11.14.9, 749 Fact 11.21.16, 785 matrix sign function Fact 5.15.21, 381 spectral radius Fact 9.8.4, 627 square root Fact 5.15.21, 381 Fact 8.9.33, 499 unitary matrix Fact 8.9.34, 499 vectors Fact 10.11.1, 699 convergent sequence of matrices definition Definition 10.2.3, 683 convergent sequence of scalars definition Definition 10.2.2, 682 convergent sequence of vectors definition

convex function Definition 10.2.3, 683 convergent series definition Definition 10.2.6, 683 Definition 10.2.8, 683 inverse matrix Fact 4.10.7, 290 matrix exponential Proposition 11.1.2, 708 spectral radius Fact 4.10.7, 290 convergent subsequence compact set Theorem 10.2.5, 683 converse definition, 1 convex combination definition, 97 determinant Fact 8.13.17, 537 norm inequality Fact 9.7.15, 623 positive-semidefinite matrix Fact 5.19.6, 395 Fact 8.13.17, 537 convex cone definition, 97 induced by transitive relation Proposition 2.3.6, 101 inner product Fact 10.9.14, 697 intersection Fact 2.9.9, 120 polar Fact 2.9.4, 119 positive-semidefinite matrix, 459 quadratic form Fact 8.14.11, 547 Fact 8.14.13, 548 Fact 8.14.14, 548 separation theorem Fact 10.9.14, 697 sum

Fact 2.9.9, 120 union Fact 2.9.10, 120 convex conical hull constructive characterization Theorem 2.3.5, 100 convex hull Fact 2.9.3, 119 definition, 98 dual cone Fact 2.9.3, 119 convex function constant function Fact 1.10.3, 23 continuous function Theorem 10.3.2, 684 Fact 10.11.11, 701 convex set Fact 10.11.10, 701 Fact 10.11.11, 701 Fact 10.11.12, 701 definition Definition 1.4.3, 6 Definition 8.6.14, 479 derivative Fact 10.12.3, 702 determinant Proposition 8.6.17, 480 Fact 2.13.17, 143 directional differential Fact 10.12.3, 702 eigenvalue Corollary 8.6.19, 486 Fact 8.19.5, 565 function composition Lemma 8.6.16, 480 Hermite-Hadamard inequality Fact 1.10.6, 24 Hermitian matrix Fact 8.12.33, 530 Fact 8.12.34, 531 increasing function Theorem 8.6.15, 479 Jensen Fact 10.11.7, 700

1003

Jensen’s inequality Fact 1.10.4, 24 Fact 1.17.36, 62 Kronecker product Proposition 8.6.17, 480 log majorization Fact 2.21.11, 177 logarithm Fact 11.16.14, 762 Fact 11.16.15, 762 logarithm of determinant Proposition 8.6.17, 480 logarithm of trace Proposition 8.6.17, 480 matrix exponential Fact 8.14.18, 549 Fact 11.16.14, 762 Fact 11.16.15, 762 matrix functions Proposition 8.6.17, 480 matrix logarithm Proposition 8.6.17, 480 midpoint convex Fact 10.11.7, 700 minimizer Fact 8.14.15, 548 Niculescu’s inequality Fact 1.10.5, 24 nondecreasing function Lemma 8.6.16, 480 one-sided directional differential Proposition 10.4.1, 686 Popoviciu’s inequality Fact 1.10.6, 24 positive-definite matrix Fact 8.14.17, 549 positive-semidefinite matrix Fact 8.14.15, 548 Fact 8.21.20, 583 reverse inequality Fact 8.10.9, 502 scalar inequality

1004

convex function

Fact 1.10.1, 23 Schur complement Proposition 8.6.17, 480 Lemma 8.6.16, 480 singular value Fact 11.16.14, 762 Fact 11.16.15, 762 strong log majorization Fact 2.21.8, 176 strong majorization Fact 2.21.7, 176 Fact 2.21.10, 177 subdifferential Fact 10.12.3, 702 trace Proposition 8.6.17, 480 Fact 8.14.17, 549 transformation Fact 1.10.2, 23 weak majorization Fact 2.21.7, 176 Fact 2.21.8, 176 Fact 2.21.9, 177 Fact 2.21.10, 177 Fact 8.19.5, 565 convex hull affine hull Fact 2.9.3, 119 closure Fact 10.8.13, 694 compact set Fact 10.8.15, 694 constructive characterization Theorem 2.3.5, 100 definition, 98 Hermitian matrix diagonal Fact 8.18.8, 561 open set Fact 10.8.14, 694 simplex Fact 2.20.4, 167 solid set Fact 10.8.10, 693 spectrum Fact 8.14.7, 546 Fact 8.14.8, 547

strong majorization Fact 3.9.6, 201 convex polyhedron volume Fact 2.20.20, 174 convex set affine hull Theorem 10.3.2, 684 Fact 10.8.8, 693 closure Fact 10.8.8, 693 Fact 10.8.19, 694 completely solid set Fact 10.8.9, 693 continuous function Theorem 10.3.10, 685 Corollary 10.3.11, 685 convexity of image Fact 2.9.26, 123 definition, 97 distance from a point Fact 10.9.16, 697 Fact 10.9.17, 698 extreme point Fact 10.8.23, 695 image under linear mapping Fact 2.9.26, 123 interior Fact 10.8.8, 693 Fact 10.8.19, 694 Fact 10.9.4, 695 intersection Fact 2.9.9, 120 Fact 10.9.7, 696 left inverse Fact 2.9.26, 123 norm Fact 9.7.23, 625 open ball Fact 10.8.1, 693 positive-semidefinite matrix Fact 8.14.2, 544 Fact 8.14.3, 544 Fact 8.14.4, 544 Fact 8.14.5, 545 Fact 8.14.6, 545

quadratic form Fact 8.14.2, 544 Fact 8.14.3, 544 Fact 8.14.4, 544 Fact 8.14.5, 545 Fact 8.14.6, 545 Fact 8.14.9, 547 Fact 8.14.11, 547 Fact 8.14.12, 548 Fact 8.14.13, 548 Fact 8.14.14, 548 set cancellation Fact 10.9.8, 696 solid set Fact 10.8.9, 693 sublevel set Fact 8.14.1, 543 sum Fact 2.9.9, 120 sum of sets Fact 2.9.1, 119 Fact 2.9.2, 119 Fact 10.9.5, 696 Fact 10.9.6, 696 Fact 10.9.8, 696 union Fact 10.9.8, 696 convex sets proper separation theorem Fact 10.9.15, 697 convolution integral definition, 798 coplanar determinant Fact 2.20.2, 167 copositive matrix nonnegative matrix Fact 8.15.33, 556 positive-semidefinite matrix Fact 8.15.33, 556 quadratic form Fact 8.15.33, 556 coprime polynomial Fact 4.8.3, 276

cycle Fact 4.8.4, 277 coprime polynomials Bezout matrix Fact 4.8.6, 277 Fact 4.8.7, 279 Fact 4.8.8, 279 definition, 255 resultant Fact 4.8.4, 277 Smith-McMillan form Fact 4.8.15, 282 Sylvester matrix Fact 4.8.4, 277 coprime right polynomial fraction description Smith-McMillan form Proposition 4.7.16, 275 unimodular matrix Proposition 4.7.15, 275 Copson inequality sum of powers Fact 1.17.44, 64 Cordes inequality maximum singular value Fact 8.19.27, 572 corollary definition, 1 cosine law vector equality Fact 9.7.4, 618 cosine rule triangle Fact 2.20.11, 169 CPP closed punctured plane definition, 4 Crabtree Schur complement of a partitioned matrix

Fact 6.5.29, 430 Crabtree-Haynsworth quotient formula Schur complement of a partitioned matrix Fact 6.5.29, 430 Cramer’s rule linear system solution Fact 2.13.7, 140 creation matrix upper triangular matrix Fact 11.11.4, 736 CRHP closed right half plane definition, 4 Crimmins product of projectors Fact 6.3.31, 410 cross product adjugate Fact 6.5.16, 427 Cayley transform Fact 3.11.29, 210 equalities Fact 3.10.1, 202 matrix exponential Fact 11.11.7, 738 Fact 11.11.8, 738 Fact 11.11.9, 739 orthogonal matrix Fact 3.10.2, 204 Fact 3.10.3, 204 Fact 3.11.29, 210 outer-product matrix Fact 3.11.29, 210 parallelogram Fact 9.7.5, 620 triangle Fact 2.20.10, 169 cross-product matrix Cayley transform

1005

Fact 3.10.1, 202 characteristic polynomial Fact 4.9.20, 286 Fact 4.9.21, 286 diagonalization Fact 5.9.1, 338 Fact 5.9.2, 338 equalities Fact 3.10.1, 202 matrix exponential Fact 11.11.6, 737 Fact 11.11.12, 739 Fact 11.11.13, 739 Fact 11.11.14, 740 Fact 11.11.17, 741 Fact 11.11.18, 741 orthogonal matrix Fact 11.11.12, 739 Fact 11.11.13, 739 Fact 11.11.14, 740 spectrum Fact 4.9.20, 286 CS decomposition unitary matrix Fact 5.9.31, 344 cube root equality Fact 2.12.23, 138 cubes equality Fact 2.12.24, 139 cubic scalar inequality Fact 1.13.14, 47 Fact 1.13.15, 47 Fact 1.13.16, 47 cubic polynomial Cardano’s trigonometric solution Fact 4.10.3, 288 CUD closed unit disk definition, 4 cycle

1006

cycle

definition Definition 1.6.3, 10 graph Fact 1.8.4, 15 symmetric graph Fact 1.8.5, 15 cyclic eigenvalue definition Definition 5.5.4, 322 eigenvector Fact 5.14.1, 369 semisimple eigenvalue Proposition 5.5.5, 322 simple eigenvalue Proposition 5.5.5, 322 cyclic group definition Proposition 3.3.6, 187 group Fact 3.23.4, 243 cyclic inequality scalar inequality Fact 1.13.11, 47 cyclic matrix asymptotically stable matrix Fact 11.18.25, 770 campanion matrix Fact 5.16.5, 388 characteristic polynomial Proposition 5.5.14, 325 commuting matrices Fact 5.14.21, 373 controllability Fact 12.20.13, 865 controllable pair Fact 5.14.7, 370 definition Definition 5.5.4, 322 determinant Fact 5.14.7, 370 identity-matrix perturbation Fact 5.14.15, 372 linear independence

Fact 5.14.7, 370 matrix power Fact 5.14.7, 370 minimal polynomial Proposition 5.5.14, 325 nonsingular matrix Fact 5.14.7, 370 rank Fact 5.11.1, 350 semisimple matrix Fact 5.14.10, 371 similar matrices Fact 5.16.5, 388 simple matrix Fact 5.14.10, 371 tridiagonal matrix Fact 11.18.25, 770 cyclic permutation matrix definition, 91 determinant Fact 2.13.2, 139 involutory matrix Fact 3.15.5, 230 irreducible matrix Fact 3.22.5, 241 permutation matrix Fact 5.16.8, 390 primary circulant matrix Fact 5.16.7, 388 reverse permutation matrix Fact 3.15.5, 230

D damped natural frequency definition, 718 Fact 5.14.34, 375 damping definition, 718 damping matrix partitioned matrix Fact 5.12.21, 368 damping ratio

definition, 718 Fact 5.14.34, 375 Davenport orthogonal matrices and matrix exponentials Fact 11.11.13, 739 De Bruijn’s inequality refined Cauchy-Schwarz inequality Fact 1.18.20, 70 De Morgan’s laws logical equivalents Fact 1.7.1, 11 Decell generalized inverse Fact 6.4.34, 418 decreasing function definition Definition 8.6.12, 478 defect adjugate Fact 2.16.7, 155 definition, 104 equality Fact 2.10.20, 127 geometric multiplicity Proposition 4.5.2, 268 group-invertible matrix Fact 3.6.1, 191 Hermitian matrix Fact 5.8.7, 335 Fact 8.9.7, 496 Kronecker sum Fact 7.5.2, 450 partitioned matrix Fact 2.11.3, 131 Fact 2.11.8, 132 Fact 2.11.11, 133 powers Proposition 2.5.8, 106 product Proposition 2.6.3, 107

detectability product of matrices Fact 2.10.14, 126 rank Corollary 2.5.5, 105 semisimple eigenvalue Proposition 5.5.8, 323 submatrix Fact 2.11.20, 135 Sylvester’s law of nullity Fact 2.10.15, 126 transpose Corollary 2.5.3, 105 defective eigenvalue definition Definition 5.5.4, 322 defective matrix definition Definition 5.5.4, 322 identity-matrix perturbation Fact 5.14.15, 372 nilpotent matrix Fact 5.14.17, 372 outer-product matrix Fact 5.14.3, 370 deflating subspace pencil Fact 5.13.1, 369 degree graph Definition 1.6.3, 10 degree matrix definition Definition 3.2.1, 184 symmetric graph Fact 3.21.1, 240 degree of a polynomial definition, 253 degree of a polynomial matrix definition, 256 derivative

adjugate Fact 10.12.8, 703 Fact 10.12.10, 703 convex function Fact 10.12.3, 702 definition Definition 10.4.3, 686 determinant Proposition 10.7.3, 692 Fact 10.12.8, 703 Fact 10.12.10, 703 Fact 10.12.11, 704 Fact 10.12.12, 704 inverse matrix Proposition 10.7.2, 691 Fact 10.12.7, 702 Fact 10.12.8, 703 logarithm of determinant Proposition 10.7.3, 692 matrix definition, 688 matrix exponential Fact 8.12.32, 530 Fact 11.14.3, 748 Fact 11.14.4, 748 Fact 11.14.10, 749 Fact 11.15.2, 756 matrix power Proposition 10.7.2, 691 maximum singular value Fact 11.15.2, 756 realization Fact 12.22.6, 872 squared matrix Fact 10.12.6, 702 trace Proposition 10.7.4, 692 Fact 11.14.3, 748 transfer function Fact 12.22.6, 872 uniqueness Proposition 10.4.2, 686 derivative of a matrix commutator Fact 11.14.11, 749 matrix exponential Fact 11.14.11, 749

1007

matrix product Fact 11.13.8, 744 derivative of a matrix exponential Bessis-MoussaVillani trace conjecture Fact 8.12.32, 530 derivative of an integral Leibniz’s rule Fact 10.12.2, 701 derogatory eigenvalue definition Definition 5.5.4, 322 derogatory matrix definition Definition 5.5.4, 322 identity-matrix perturbation Fact 5.14.15, 372 Descartes rule of signs polynomial Fact 11.17.1, 763 detectability asymptotically stable matrix Proposition 12.5.5, 807 Corollary 12.5.6, 807 block-triangular matrix Proposition 12.5.3, 807 closed-loop spectrum Lemma 12.16.17, 854 definition Definition 12.5.1, 807 Lyapunov equation Corollary 12.5.6, 807 observably asymptotically stable Proposition 12.5.5, 807 orthogonal matrix Proposition 12.5.3, 807 output convergence Fact 12.20.2, 864 output injection

1008

detectability

Proposition 12.5.2, 807 PBH test Theorem 12.5.4, 807 Riccati equation Corollary 12.17.3, 855 Corollary 12.19.2, 863 state convergence Fact 12.20.2, 864 determinant (1)-inverse Fact 6.5.28, 430 adjugate Fact 2.14.27, 151 Fact 2.16.3, 153 Fact 2.16.5, 154 affine hyperplane Fact 2.20.3, 167 basic properties Proposition 2.7.2, 112 Binet-Cauchy formula Fact 2.13.5, 140 block-triangular matrix Fact 2.14.8, 145 Cartesian decomposition Fact 8.13.4, 534 Fact 8.13.11, 534 Cauchy matrix Fact 3.22.9, 241 Fact 3.22.10, 242 Cauchy-Schwarz inequality Fact 8.13.23, 537 cofactor expansion Proposition 2.7.5, 114 colinear Fact 2.20.1, 167 Fact 2.20.5, 168 Fact 2.20.9, 169 column interchange Proposition 2.7.2, 112 commutator Fact 2.18.7, 162 complex conjugate Fact 2.19.8, 166 Fact 2.19.9, 166

complex conjugate transpose Proposition 2.7.1, 112 Fact 9.11.1, 653 complex matrix Fact 2.19.3, 164 Fact 2.19.10, 166 convex combination Fact 8.13.17, 537 convex function Proposition 8.6.17, 480 Fact 2.13.17, 143 coplanar Fact 2.20.2, 167 cyclic matrix Fact 5.14.7, 370 cyclic permutation matrix Fact 2.13.2, 139 definition, 112 derivative Proposition 10.7.3, 692 Fact 10.12.8, 703 Fact 10.12.10, 703 Fact 10.12.11, 704 Fact 10.12.12, 704 dissipative matrix Fact 8.13.2, 533 Fact 8.13.11, 534, 535 Fact 8.13.32, 539 eigenvalue Fact 5.11.28, 356 Fact 5.11.29, 356 Fact 8.13.1, 533 elementary matrix Fact 2.16.1, 153 equality Fact 2.13.10, 141 Fact 2.13.11, 141 Fact 2.13.12, 141 Fact 2.13.13, 142 factorization Fact 5.15.8, 378 Fact 5.15.34, 383 Fibonacci numbers Fact 4.11.13, 303 Frobenius norm Fact 9.8.39, 633 full-state feedback

Fact 12.22.14, 874 generalized inverse Fact 6.5.26, 429 Fact 6.5.27, 430 Fact 6.5.28, 430 geometric mean Fact 8.10.43, 508 group Proposition 3.3.6, 187 Hadamard’s inequality Fact 8.13.34, 540 Fact 8.13.35, 540 Hankel matrix Fact 3.18.4, 234 Hermitian matrix Corollary 8.4.10, 470 Fact 3.7.21, 197 Fact 8.13.7, 535 Hua’s inequalities Fact 8.13.26, 538 induced norm Fact 9.12.11, 659 inequality Fact 8.13.25, 538 Fact 8.13.26, 538 Fact 8.13.27, 539 Fact 8.13.28, 539 Fact 8.13.29, 539 Fact 8.13.31, 539 Fact 8.22.20, 587, 588 integral Fact 11.13.15, 745 invariant zero Fact 12.22.14, 874 inverse Fact 2.13.6, 140 inverse function theorem Theorem 10.4.5, 688 involutory matrix Fact 3.15.1, 230 Fact 5.15.32, 383 Kronecker product Proposition 7.1.11, 442 Kronecker sum Fact 7.5.11, 451 linear combination Fact 8.13.19, 537

determinant lower block-triangular matrix Proposition 2.7.1, 112 lower reverse-triangular matrix Fact 2.13.9, 141 lower triangular matrix Fact 3.22.1, 240 matrix exponential Proposition 11.4.6, 719 Corollary 11.2.4, 712 Corollary 11.2.5, 712 Fact 11.13.15, 745 Fact 11.15.5, 756 matrix logarithm Fact 8.13.8, 535 Fact 8.19.31, 574 Fact 9.8.39, 633 Fact 11.14.24, 752 maximum singular value Fact 9.14.17, 669 Fact 9.14.18, 669 minimum singular value Fact 9.14.18, 669 multiplicative commutator Fact 5.15.34, 383 nilpotent matrix Fact 3.17.9, 233 nonsingular matrix Corollary 2.7.4, 113 Lemma 2.8.6, 117 normal matrix Fact 5.12.12, 365 ones matrix Fact 2.13.3, 139 ones matrix perturbation Fact 2.16.6, 155 orthogonal matrix Fact 3.11.16, 207 Fact 3.11.17, 207 Ostrowski-Taussky inequality

Fact 8.13.2, 533 outer-product perturbation Fact 2.16.3, 153 output feedback Fact 12.22.13, 874 partitioned matrix Corollary 2.8.5, 116 Lemma 8.2.6, 463 Fact 2.14.2, 144 Fact 2.14.3, 144 Fact 2.14.4, 145 Fact 2.14.5, 145 Fact 2.14.6, 145 Fact 2.14.7, 145 Fact 2.14.9, 145 Fact 2.14.10, 146 Fact 2.14.11, 146 Fact 2.14.13, 147 Fact 2.14.14, 147 Fact 2.14.15, 147 Fact 2.14.16, 148 Fact 2.14.17, 148 Fact 2.14.18, 148 Fact 2.14.19, 149 Fact 2.14.20, 149 Fact 2.14.21, 149 Fact 2.14.22, 149 Fact 2.14.23, 150 Fact 2.14.24, 150 Fact 2.14.25, 150 Fact 2.14.26, 151 Fact 2.14.28, 151 Fact 2.17.5, 160 Fact 2.19.3, 164 Fact 2.19.9, 166 Fact 5.12.21, 368 Fact 6.5.26, 429 Fact 6.5.27, 430 Fact 6.5.28, 430 Fact 8.13.36, 541 Fact 8.13.37, 541 Fact 8.13.39, 541 Fact 8.13.40, 542 Fact 8.13.41, 542 Fact 8.13.42, 542 Fact 8.13.43, 543

1009

partitioned positivesemidefinite matrix Proposition 8.2.3, 462 positive-definite matrix Proposition 8.4.14, 471 Fact 8.12.1, 523 Fact 8.13.6, 534 Fact 8.13.7, 535 Fact 8.13.8, 535 Fact 8.13.9, 535 Fact 8.13.10, 535 Fact 8.13.13, 536 Fact 8.13.14, 536 Fact 8.13.15, 536 Fact 8.13.16, 536 Fact 8.13.18, 537 Fact 8.13.20, 537 Fact 8.13.22, 537 Fact 8.13.24, 538 positive-semidefinite matrix Corollary 8.4.15, 472 Fact 8.13.12, 536 Fact 8.13.17, 537 Fact 8.13.19, 537 Fact 8.13.21, 537 Fact 8.13.22, 537 Fact 8.13.25, 538 Fact 8.13.30, 539 Fact 8.13.36, 541 Fact 8.13.37, 541 Fact 8.13.39, 541 Fact 8.13.40, 542 Fact 8.13.41, 542 Fact 8.13.42, 542 Fact 8.18.11, 562 Fact 8.19.31, 574 Fact 8.22.8, 586 Fact 8.22.20, 587, 588 Fact 8.22.21, 588 Fact 9.8.39, 633 product Proposition 2.7.3, 113 rank-deficient matrix Fact 2.13.4, 140 reverse permutation matrix

1010

determinant

Fact 2.13.1, 139 row interchange Proposition 2.7.2, 112 Schur complement Proposition 8.2.3, 462 semidissipative matrix Fact 8.13.3, 534 Fact 8.13.4, 534 Fact 8.13.11, 534, 535 singular value Fact 5.11.28, 356 Fact 5.11.29, 356 Fact 8.13.1, 533 Fact 9.13.22, 664 singular values Fact 5.12.13, 365 skew-Hermitian matrix Fact 3.7.11, 194 Fact 3.7.16, 196 Fact 8.13.6, 534 skew-symmetric matrix Fact 3.7.15, 195 Fact 3.7.33, 198 Fact 4.8.14, 282 Fact 4.9.21, 286 Fact 4.10.4, 289 strongly increasing function Proposition 8.6.13, 478 subdeterminant Fact 2.13.5, 140 Fact 2.14.12, 146 subdeterminant expansion Corollary 2.7.6, 115 submatrix Fact 2.14.1, 144 sum of Kronecker product Fact 7.5.12, 451 Fact 7.5.13, 452 sum of matrices Fact 5.12.12, 365 Fact 9.14.18, 669 sum of orthogonal matrices

Fact 3.11.17, 207 Sylvester’s identity Fact 2.14.1, 144 symplectic matrix Fact 3.20.10, 239 Fact 3.20.11, 239 time-varying dynamics Fact 11.13.4, 743 Toeplitz matrix Fact 2.13.13, 142 Fact 3.18.9, 235 trace Proposition 8.4.14, 471 Corollary 11.2.4, 712 Corollary 11.2.5, 712 Fact 2.13.16, 143 Fact 8.12.1, 523 Fact 8.13.21, 537 Fact 11.14.20, 751 transpose Proposition 2.7.1, 112 tridiagonal matrix Fact 3.18.9, 235 Fact 3.19.2, 237 Fact 3.19.3, 238 Fact 3.19.4, 238 Fact 3.19.5, 238 unimodular matrix Proposition 4.3.7, 260 unitary matrix Fact 3.11.15, 207 Fact 3.11.18, 207 Fact 3.11.19, 207 Fact 3.11.20, 208 upper bound Fact 2.13.14, 142 Fact 2.13.15, 143 Fact 8.13.33, 540 Fact 8.13.34, 540 Fact 8.13.35, 540 upper triangular matrix Fact 3.22.1, 240 Vandermonde matrix Fact 5.16.3, 387 determinant equalities Magnus Fact 2.13.16, 143

determinant inequality Hua’s inequalities Fact 8.11.21, 519 determinant lower bound nonsingular matrix Fact 4.10.19, 294 determinant of a partitioned matrix Hadamard’s inequality Fact 6.5.26, 429 determinant of an outer-product perturbation Sherman-MorrisonWoodbury formula Fact 2.16.3, 153 determinantal compression partitioned matrix Fact 8.13.43, 543 diagonal eigenvalue Fact 8.12.3, 523 positive-semidefinite matrix Fact 8.12.3, 523 zero Fact 5.9.20, 341 diagonal dominance rank Fact 4.10.24, 295 diagonal dominance theorem nonsingular matrix Fact 4.10.18, 293 Fact 4.10.19, 294 diagonal entries of a unitary matrix Schur-Horn theorem Fact 3.11.14, 207 Fact 8.18.10, 562 diagonal entry

difference equation definition, 87 eigenvalue Fact 8.18.8, 561 Hermitian matrix Corollary 8.4.7, 469 Fact 8.18.8, 561 Fact 8.18.9, 562 Fact 8.18.13, 563 positive-semidefinite matrix Fact 8.10.16, 503 similar matrices Fact 5.9.15, 340 strong majorization Fact 8.18.8, 561 unitarily similar matrices Fact 5.9.19, 341 Fact 5.9.21, 341 unitary matrix Fact 3.11.14, 207 Fact 8.18.10, 562 diagonal matrix controllability Fact 12.20.12, 865 definition Definition 3.1.3, 181 Hermitian matrix Corollary 5.4.5, 320 Lemma 8.5.1, 473 Kronecker product Fact 7.4.4, 445 matrix exponential Fact 11.13.17, 746 orthogonally similar matrices Fact 5.9.17, 341 unitary matrix Theorem 5.6.3, 329 Lemma 8.5.1, 473 diagonalizable matrix additive decomposition Fact 5.9.5, 339 adjugate Fact 5.14.4, 370 cogredient diagonalization

Fact 8.17.2, 558 Fact 8.17.3, 558 commuting matrices Fact 5.17.8, 392 complex conjugate transpose Fact 5.14.4, 370 eigenvector Fact 5.14.5, 370 example Example 5.5.17, 326 factorization Fact 5.15.27, 382 involutory matrix Fact 5.14.19, 373 Jordan-Chevalley decomposition Fact 5.9.5, 339 positive-definite matrix Corollary 8.3.3, 465 S-N decomposition Fact 5.9.5, 339 simultaneous diagonalization Fact 8.17.2, 558 Fact 8.17.3, 558 transpose Fact 5.14.4, 370 diagonalizable over C definition Definition 5.5.4, 322 diagonalizable over F identity-matrix perturbation Fact 5.14.15, 372 diagonalizable over R asymptotically stable matrix Fact 11.17.10, 765 definition Definition 5.5.4, 322 factorization Proposition 5.5.12, 324 Corollary 5.5.21, 327 similar matrices Proposition 5.5.12, 324 Corollary 5.5.21, 327

1011

diagonalization cross-product matrix Fact 5.9.1, 338 Fact 5.9.2, 338 diagonally dominant matrix nonsingular matrix Fact 4.10.18, 293 diagonally located block definition, 87 Diaz-Goldman-Metcalf inequality H¨ older’s inequality Fact 1.18.22, 71 dictionary ordering cone Fact 2.9.31, 124 total ordering Fact 1.7.8, 13 difference Drazin generalized inverse Fact 6.6.6, 432 Frobenius norm Fact 9.9.25, 640 generalized inverse Fact 6.4.42, 420 idempotent matrix Fact 5.12.19, 367 maximum singular value Fact 8.19.8, 566 Fact 9.9.32, 642 projector Fact 3.13.24, 228 Fact 5.12.17, 366 Fact 6.4.23, 416 Schatten norm Fact 9.9.23, 640 singular value Fact 8.19.9, 567 Fact 8.19.10, 567 trace norm Fact 9.9.24, 640 difference equation

1012

difference equation

golden ratio Fact 4.11.13, 303 nonnegative matrix Fact 4.11.13, 303 difference of idempotent matrices Makelainen Fact 5.12.19, 367 Styan Fact 5.12.19, 367 difference of matrices idempotent matrix Fact 3.12.26, 220 Fact 3.12.27, 220 Fact 3.12.28, 220 Fact 3.12.30, 221 Fact 3.12.32, 222 differentiable function continuous function Proposition 10.4.4, 687 definition Definition 10.4.3, 686 digraph definition, 84 dihedral group definition Proposition 3.3.6, 187 group Fact 3.23.4, 243 Klein four-group Fact 3.23.4, 243 dimension commuting matrices Fact 5.10.15, 347 Fact 5.10.16, 347 product of matrices Fact 2.10.14, 126 rank inequality Fact 2.10.4, 125 solid set Fact 10.8.16, 694 subspace Fact 2.10.4, 125 subspace dimension theorem Theorem 2.3.1, 98

subspace intersection Fact 2.9.20, 122 Fact 2.9.21, 122 Fact 2.9.22, 122 variational cone Fact 10.8.20, 694 zero trace Fact 2.18.11, 163 dimension of a subspace definition, 98 dimension of an affine subspace definition, 98 dimension of an arbitrary set definition, 98 dimension theorem rank and defect Corollary 2.5.5, 105 directed cut definition Definition 1.6.3, 10 graph Fact 4.11.3, 297 directed graph definition, 84 direction cosines Euler parameters Fact 3.11.31, 212 orthogonal matrix Fact 3.11.31, 212 directional differential convex function Fact 10.12.3, 702 discrete Fourier analysis circulant matrix Fact 5.16.7, 388 discrete-time asymptotic stability eigenvalue Proposition 11.10.2, 734

linear dynamical system Proposition 11.10.2, 734 matrix exponential Proposition 11.10.2, 734 discrete-time asymptotically stable matrix 2 × 2 matrix Fact 11.21.1, 782 asymptotically stable matrix Fact 11.21.9, 783 Cayley transform Fact 11.21.9, 783 definition Definition 11.10.1, 734 discrete-time asymptotically stable polynomial Proposition 11.10.4, 735 dissipative matrix Fact 11.21.4, 782 Kronecker product Fact 11.21.6, 782 Fact 11.21.7, 783 Lyapunov equation Proposition 11.10.5, 735 matrix exponential Fact 11.21.8, 783 matrix limit Fact 11.21.14, 784 matrix power Fact 11.21.2, 782 normal matrix Fact 11.21.4, 782 partitioned matrix Fact 11.21.10, 783 positive-definite matrix Proposition 11.10.5, 735 Fact 11.21.10, 783 Fact 11.21.17, 785 Fact 11.21.18, 785

discrete-time semistable matrix similar matrices Fact 11.18.4, 766 discrete-time asymptotically stable polynomial definition Definition 11.10.3, 735 discrete-time asymptotically stable matrix Proposition 11.10.4, 735 Jury criterion Fact 11.20.1, 777 polynomial coefficients Fact 11.20.1, 777 Fact 11.20.2, 778 Fact 11.20.3, 778 Schur-Cohn criterion Fact 11.20.1, 777 discrete-time control problem LQG controller Fact 12.23.7, 878 discrete-time dynamics matrix power Fact 11.21.3, 782 discrete-time Lyapunov equation discrete-time asymptotically stable matrix Fact 11.21.17, 785 Fact 11.21.18, 785 discrete-time Lyapunov-stable matrix Proposition 11.10.6, 735 Stein equation Fact 11.21.17, 785 Fact 11.21.18, 785 discrete-time Lyapunov stability eigenvalue

Proposition 11.10.2, 734 linear dynamical system Proposition 11.10.2, 734 matrix exponential Proposition 11.10.2, 734 discrete-time Lyapunov-stable matrix definition Definition 11.10.1, 734 discrete-time Lyapunov equation Proposition 11.10.6, 735 discrete-time Lyapunov-stable polynomial Proposition 11.10.4, 735 group generalized inverse Fact 11.21.13, 784 Kreiss matrix theorem Fact 11.21.20, 786 Kronecker product Fact 11.21.6, 782 Fact 11.21.7, 783 logarithm Fact 11.14.19, 751 matrix exponential Fact 11.21.8, 783 matrix limit Fact 11.21.13, 784 matrix power Fact 11.21.2, 782 Fact 11.21.12, 784 maximum singular value Fact 11.21.20, 786 normal matrix Fact 11.21.4, 782 positive-definite matrix

1013

Proposition 11.10.6, 735 positive-semidefinite matrix Fact 11.21.17, 785 row-stochastic matrix Fact 11.21.11, 784 semicontractive matrix Fact 11.21.4, 782 semidissipative matrix Fact 11.21.4, 782 similar matrices Fact 11.18.4, 766 unitary matrix Fact 11.21.15, 784 discrete-time Lyapunov-stable polynomial definition Definition 11.10.3, 735 discrete-time Lyapunov-stable matrix Proposition 11.10.4, 735 discrete-time semistability eigenvalue Proposition 11.10.2, 734 linear dynamical system Proposition 11.10.2, 734 matrix exponential Proposition 11.10.2, 734 discrete-time semistable matrix companion form matrix Fact 11.21.21, 786 convergent sequence Fact 11.21.16, 785 definition

1014

discrete-time semistable matrix

Definition 11.10.1, 734 discrete-time semistable polynomial Proposition 11.10.4, 735 idempotent matrix Fact 11.21.12, 784 Kronecker product Fact 11.21.5, 782 Fact 11.21.6, 782 Fact 11.21.7, 783 limit Fact 11.21.12, 784 matrix exponential Fact 11.21.8, 783 Fact 11.21.16, 785 row-stochastic matrix Fact 11.21.11, 784 similar matrices Fact 11.18.4, 766 discrete-time semistable polynomial definition Definition 11.10.3, 735 discrete-time semistable matrix Proposition 11.10.4, 735 discrete-time time-varying system state convergence Fact 11.21.19, 785 discriminant compound matrix Fact 7.5.17, 452 disjoint definition, 3 dissipative matrix asymptotically stable matrix Fact 11.18.21, 769 Fact 11.18.37, 774 definition Definition 3.1.1, 179

determinant Fact 8.13.2, 533 Fact 8.13.11, 534, 535 Fact 8.13.32, 539 discrete-time asymptotically stable matrix Fact 11.21.4, 782 Frobenius norm Fact 11.15.3, 756 inertia Fact 5.8.12, 336 Kronecker sum Fact 7.5.8, 451 matrix exponential Fact 11.15.3, 756 maximum singular value Fact 8.18.12, 563 nonsingular matrix Fact 3.22.8, 241 normal matrix Fact 11.18.37, 774 positive-definite matrix Fact 8.18.12, 563 Fact 11.18.21, 769 range-Hermitian matrix Fact 5.14.30, 374 semidissipative matrix Fact 8.13.32, 539 spectrum Fact 8.13.32, 539 strictly dissipative matrix Fact 8.9.32, 498 unitary matrix Fact 8.9.32, 498 distance from a point set Fact 10.9.16, 697 Fact 10.9.17, 698 distance to singularity nonsingular matrix Fact 9.14.7, 666 distinct eigenvalues

eigenvector Proposition 4.5.4, 268 distinct roots Bezout matrix Fact 4.8.9, 280 distributive equalities definition, 89 divides definition, 255 division of polynomial matrices quotient and remainder Lemma 4.2.1, 256 Dixmier projectors and unitarily similar matrices Fact 5.10.12, 347 Djokovic maximum singular value of a product of elementary projectors Fact 9.14.1, 665 rank of a Kronecker product Fact 8.22.16, 587 Schur product of positive-definite matrices Fact 8.22.13, 586 Djokovic inequality Euclidean norm Fact 9.7.7, 620 domain definition, 4 Dormido asymptotically stable polynomial Fact 11.17.8, 765 double cover orthogonal matrix parameterization

eigenvalue Fact 3.11.31, 212 spin group Fact 3.11.31, 212 doublet definition Fact 2.10.24, 128 outer-product matrix Fact 2.10.24, 128 Fact 2.12.6, 136 spectrum Fact 5.11.13, 352 doubly stochastic matrix Birkhoff Fact 3.11.3, 205 definition Definition 3.1.4, 182 permutation matrix Fact 3.9.6, 201 Fact 3.11.3, 205 strong majorization Fact 3.9.6, 201 Douglas-FillmoreWilliams lemma factorization Theorem 8.6.2, 474 Dragomir’s inequality harmonic mean Fact 1.18.24, 72 Dragomir-Yang inequalities Euclidean norm Fact 9.7.8, 621 Fact 9.7.9, 621 Drazin real eigenvalues Fact 5.14.12, 371 Drazin generalized inverse block-circulant matrix Fact 6.6.1, 431 commuting matrices Fact 6.6.4, 431

Fact 6.6.5, 431 definition, 401 idempotent matrix Proposition 6.2.2, 402 Fact 6.6.6, 432 integral Fact 11.13.12, 745 Fact 11.13.14, 745 Kronecker product Fact 7.4.33, 449 matrix difference Fact 6.6.6, 432 matrix exponential Fact 11.13.12, 745 Fact 11.13.14, 745 matrix limit Fact 6.6.12, 435 matrix product Fact 6.6.3, 431 Fact 6.6.4, 431 matrix sum Fact 6.6.5, 431 Fact 6.6.6, 432 null space Proposition 6.2.2, 402 partitioned matrix Fact 6.6.1, 431 Fact 6.6.2, 431 positive-semidefinite matrix Fact 8.21.2, 577 range Proposition 6.2.2, 402 sum Fact 6.6.1, 431 tripotent matrix Proposition 6.2.2, 402 uniqueness Theorem 6.2.1, 402 dual cone convex conical hull Fact 2.9.3, 119 definition, 99 intersection Fact 2.9.5, 120 sum of sets Fact 2.9.5, 120 dual norm

1015

adjoint norm Fact 9.8.8, 627 definition Fact 9.7.22, 625 induced norm Fact 9.7.22, 625 quadratic form Fact 9.8.34, 632 Dunkl-Williams inequality complex numbers Fact 1.20.5, 77 norm Fact 9.7.10, 621 Fact 9.7.13, 622 dynamic compensator LQG controller Fact 12.23.6, 878 Fact 12.23.7, 878

E Eckart-Young theorem fixed-rank approximation Fact 9.14.28, 672 eigensolution eigenvector Fact 11.13.6, 744 Fact 11.13.7, 744 eigenvalue, see spectrum, multispectrum SO(n) Fact 5.11.2, 350 adjugate Fact 4.10.9, 291 asymptotic spectrum Fact 4.10.29, 296 asymptotic stability Proposition 11.8.2, 727 bound Fact 4.10.23, 295 Fact 5.11.22, 354 Fact 5.11.23, 354 Fact 9.11.7, 655 bounds

1016

eigenvalue

Fact 4.10.17, 293 Fact 4.10.21, 294 Cardano’s trigonometric solution Fact 4.10.3, 288 Cartesian decomposition Fact 5.11.21, 354 convex function Corollary 8.6.19, 486 Fact 8.19.5, 565 definition, 262 determinant Fact 5.11.28, 356 Fact 8.13.1, 533 diagonal entry Fact 8.12.3, 523 Fact 8.18.8, 561 discrete-time asymptotic stability Proposition 11.10.2, 734 discrete-time Lyapunov stability Proposition 11.10.2, 734 discrete-time semistability Proposition 11.10.2, 734 Frobenius norm Fact 9.11.3, 654 Fact 9.11.5, 655 generalized eigenvector Fact 5.14.9, 371 generalized Schur inequality Fact 9.11.6, 655 Hermitian matrix Theorem 8.4.5, 468 Theorem 8.4.9, 469 Theorem 8.4.11, 470 Corollary 8.4.2, 467 Corollary 8.4.6, 469 Corollary 8.4.7, 469 Corollary 8.6.19, 486 Lemma 8.4.3, 467

Lemma 8.4.4, 468 Fact 8.10.4, 501 Fact 8.15.21, 553 Fact 8.15.32, 555 Fact 8.18.8, 561 Fact 8.18.9, 562 Fact 8.18.13, 563 Fact 8.18.15, 563 Fact 8.18.16, 563 Fact 8.19.4, 565 Fact 8.19.17, 569 Fact 8.19.18, 569 Fact 8.22.29, 590 Hermitian part Fact 5.11.24, 355 H¨ older matrix norm Fact 9.11.6, 655 Kronecker product Proposition 7.1.10, 442 Fact 7.4.14, 446 Fact 7.4.16, 446 Fact 7.4.22, 446 Fact 7.4.28, 447 Fact 7.4.34, 449 Kronecker sum Proposition 7.2.3, 443 Fact 7.5.5, 450 Fact 7.5.7, 450 Fact 7.5.16, 452 lower triangular matrix Fact 4.10.10, 291 Lyapunov stability Proposition 11.8.2, 727 matrix logarithm Theorem 11.5.2, 721 matrix sum Fact 5.12.2, 362 Fact 5.12.3, 363 normal matrix Fact 5.14.14, 372 orthogonal matrix Fact 5.11.2, 350 partitioned matrix Proposition 5.6.5, 330 Fact 5.12.20, 367 Fact 5.12.21, 368 Fact 5.12.22, 369

positive-definite matrix Fact 8.10.24, 504 Fact 8.15.21, 553 Fact 8.15.30, 555 Fact 8.15.31, 555 Fact 8.19.30, 574 Fact 8.22.22, 589 positive-semidefinite matrix Fact 8.12.3, 523 Fact 8.15.12, 551 Fact 8.19.6, 565 Fact 8.19.20, 570 Fact 8.19.21, 570 Fact 8.19.23, 571 Fact 8.19.24, 572 Fact 8.19.25, 572 Fact 8.19.28, 573 Fact 8.21.17, 580 Fact 8.22.18, 587 Fact 8.22.21, 588 quadratic form Lemma 8.4.3, 467 Fact 8.15.21, 553 root locus Fact 4.10.29, 296 Schatten norm Fact 9.11.6, 655 Schur product Fact 8.22.18, 587 Schur’s inequality Fact 8.18.5, 560 Fact 9.11.3, 654 semistability Proposition 11.8.2, 727 singular value Fact 8.18.5, 560 Fact 8.18.6, 561 Fact 9.13.21, 664 skew-Hermitian matrix Fact 5.11.6, 350 skew-symmetric matrix Fact 4.10.4, 289 spectral abscissa Fact 5.11.24, 355 strong majorization

eigenvector Corollary 8.6.19, 486 Fact 8.18.8, 561 Fact 8.19.4, 565 Fact 8.19.30, 574 subscript convention, 262 symmetric matrix Fact 4.10.3, 288 trace Proposition 8.4.13, 471 Fact 5.11.11, 351 Fact 8.18.5, 560 Fact 8.19.19, 570 upper triangular matrix Fact 4.10.10, 291 weak log majorization Fact 8.19.28, 573 weak majorization Fact 8.18.5, 560 Fact 8.19.5, 565 Fact 8.19.6, 565 Fact 8.19.28, 573 eigenvalue bound Bendixson’s theorem Fact 5.11.21, 354 Fact 9.11.8, 655 Browne’s theorem Fact 5.11.21, 354 Frobenius norm Fact 9.12.3, 656 Henrici Fact 9.11.3, 654 Hermitian matrix Fact 9.12.3, 656 Hirsch’s theorem Fact 5.11.21, 354 Hirsch’s theorems Fact 9.11.8, 655 H¨ older norm Fact 9.11.8, 655 trace Fact 5.11.45, 361 eigenvalue bounds ovals of Cassini Fact 4.10.22, 295

eigenvalue characterization minimum principle Fact 8.18.15, 563 eigenvalue inclusion region Lyapunov equation Fact 12.21.20, 871 eigenvalue inequality 2 × 2 matrix Fact 8.18.1, 559 Hermitian matrix Lemma 8.4.1, 467 Fact 8.19.3, 565 Poincar´ e separation theorem Fact 8.18.16, 563 eigenvalue of Hermitian part maximum singular value Fact 5.11.25, 355 minimum singular value Fact 5.11.25, 355 singular value Fact 5.11.27, 355 Fact 8.18.4, 560 weak majorization Fact 5.11.27, 355 eigenvalue perturbation Frobenius norm Fact 9.12.4, 657 Fact 9.12.9, 658 Fact 9.12.10, 659 Hermitian matrix Fact 4.10.28, 296 maximum singular value Fact 9.12.4, 657 Fact 9.12.8, 658 normal matrix Fact 9.12.8, 658 partitioned matrix Fact 4.10.28, 296

1017

unitarily invariant norm Fact 9.12.4, 657 eigenvalue placement controllable pair Proposition 12.6.20, 815 observable pair Proposition 12.3.20, 804 eigenvalues of a product of positive-semidefinite matrices Lidskii Fact 8.19.23, 571 eigenvector adjugate Fact 5.14.25, 373 commuting matrices Fact 5.14.24, 373 cyclic eigenvalue Fact 5.14.1, 369 definition, 267 diagonalizable matrix Fact 5.14.5, 370 distinct eigenvalues Proposition 4.5.4, 268 eigensolution Fact 11.13.6, 744 Fact 11.13.7, 744 generalized eigensolution Fact 11.13.7, 744 Kronecker product Proposition 7.1.10, 442 Fact 7.4.22, 446 Fact 7.4.34, 449 Kronecker sum Proposition 7.2.3, 443 Fact 7.5.16, 452 M-matrix Fact 4.11.12, 302 normal matrix Proposition 4.5.4, 268 Lemma 4.5.3, 268

1018

eigenvector

similarity transformation Fact 5.14.5, 370 Fact 5.14.6, 370 upper triangular matrix Fact 5.17.1, 391 either definition, 1 element definition, 2 elementary divisor companion matrix Theorem 5.2.9, 313 definition, 313 factorization Fact 5.15.37, 384 hypercompanion matrix Lemma 5.3.1, 314

Definition 3.1.1, 179 elementary reflector Fact 3.13.7, 224 Fact 3.14.3, 229 hyperplane Fact 3.13.8, 225 maximum singular value Fact 9.14.1, 665 reflector Fact 5.15.13, 379 spectrum Proposition 5.5.20, 326 trace Fact 5.8.11, 335 unitarily similar matrices Proposition 5.5.22, 327

elementary polynomial matrix definition, 258

elementary reflector definition Definition 3.1.1, 179 elementary projector Fact 3.13.7, 224 Fact 3.14.3, 229 hyperplane Fact 3.14.5, 229 null space Fact 3.13.7, 224 orthogonal matrix Fact 5.15.15, 379 range Fact 3.13.7, 224 rank Fact 3.13.7, 224 reflection theorem Fact 3.14.4, 229 reflector Fact 5.15.14, 379 spectrum Proposition 5.5.20, 326 trace Fact 5.8.11, 335 unitarily similar matrices Proposition 5.5.22, 327

elementary projector definition

elementary symmetric function

elementary matrix definition Definition 3.1.2, 180 inverse matrix Fact 3.7.20, 196 nonsingular matrix Fact 5.15.12, 379 properties and matrix types Fact 3.7.19, 196 semisimple matrix Fact 5.14.16, 372 spectrum Proposition 5.5.20, 326 unitarily similar matrices Proposition 5.5.22, 327 elementary multicompanion form definition, 313

Schur concave function Fact 1.17.20, 58 elementary symmetric mean Newton’s inequality Fact 1.17.11, 55 elementary symmetric polynomial inequality Fact 1.17.11, 54 Newton’s identities Fact 4.8.2, 276 ellipsoid positive-definite matrix Fact 3.7.35, 199 volume Fact 3.7.35, 199 Embry commuting matrices Fact 5.12.14, 365 empty matrix definition, 90 empty set definition, 2 Enestrom-Kakeya theorem polynomial root locations Fact 11.20.3, 778 entropy logarithm Fact 1.17.46, 65 Fact 1.17.47, 65 Fact 1.17.48, 65 Fact 1.18.30, 73 Schur concave function Fact 2.21.6, 176 strong majorization Fact 2.21.6, 176 entry definition, 86

Euclidean norm EP matrix, see range-Hermitian matrix definition, 252 equality cube root Fact 2.12.23, 138 equi-induced norm definition Definition 9.4.1, 607 normalized norm Theorem 9.4.2, 607 spectral radius Corollary 9.4.5, 608 submultiplicative norm Corollary 9.4.4, 608 Fact 9.8.46, 636 equi-induced self-adjoint norm maximum singular value Fact 9.13.5, 660 equi-induced unitarily invariant norm maximum singular value Fact 9.13.4, 660 equilateral triangle complex numbers Fact 2.20.6, 168 equilibrium definition, 725 equivalence equivalence relation Fact 5.10.3, 345 equivalence class equivalent matrices Fact 5.10.4, 346 induced by equivalence relation Theorem 1.5.6, 8 similar matrices Fact 5.10.4, 346

unitarily similar matrices Fact 5.10.4, 346 equivalence class induced by definition, 7 equivalence hull definition Definition 1.5.4, 7 relation Proposition 1.5.5, 7 equivalence relation Abelian group Proposition 3.4.2, 188 congruence Fact 5.10.3, 345 definition Definition 1.5.2, 6 equivalence Fact 5.10.3, 345 equivalence class Theorem 1.5.6, 8 group Proposition 3.4.1, 188 Proposition 3.4.2, 188 intersection Proposition 1.5.3, 7 left equivalence Fact 5.10.3, 345 partition Theorem 1.5.7, 8 right equivalence Fact 5.10.3, 345 similarity Fact 5.10.3, 345 unitary biequivalence Fact 5.10.3, 345 unitary left equivalence Fact 5.10.3, 345 unitary right equivalence Fact 5.10.3, 345 unitary similarity Fact 5.10.3, 345 equivalent matrices

1019

equivalence class Fact 5.10.4, 346 equivalent norms equivalence Theorem 9.1.8, 600 norms Fact 9.8.12, 628 equivalent realizations controllable canonical form Corollary 12.9.9, 825 controllable pair Proposition 12.9.8, 824 invariant zero Proposition 12.10.10, 836 observable canonical form Corollary 12.9.9, 825 observable pair Proposition 12.9.8, 824 similar matrices Definition 12.9.6, 824 ergodic theorem unitary matrix limit Fact 6.3.33, 411 essentially nonnegative matrix definition, 252 Euclidean distance matrix negative-semidefinite matrix Fact 9.8.14, 629 Schoenberg Fact 9.8.14, 629 Euclidean norm Cauchy-Schwarz inequality Corollary 9.1.7, 600 definition, 599 Djokovic inequality Fact 9.7.7, 620 Dragomir-Yang inequalities Fact 9.7.8, 621

1020

Euclidean norm

Fact 9.7.9, 621 generalized Hlawka inequality Fact 9.7.7, 620 inequality Fact 9.7.4, 618 Fact 9.7.6, 620 Fact 9.7.7, 620 Fact 9.7.8, 621 Fact 9.7.9, 621 Fact 9.7.18, 624 Kronecker product Fact 9.7.27, 626 outer-product matrix Fact 9.7.27, 626 projector Fact 9.8.2, 627 Fact 9.8.3, 627 Fact 10.9.18, 698 reverse triangle inequality Fact 9.7.6, 620 Euler characteristic Euler’s polyhedron formula Fact 1.8.7, 16 planar graph Fact 1.8.7, 16 Euler constant logarithm Fact 1.9.5, 20 Euler parameters direction cosines Fact 3.11.31, 212 orthogonal matrix Fact 3.11.31, 212 Fact 3.11.32, 214 Rodrigues’s formulas Fact 3.11.32, 214 Euler product formula prime numbers Fact 1.9.11, 23 zeta function Fact 1.9.11, 23 Euler totient function

positive-semidefinite matrix Fact 8.8.5, 491 Euler’s inequality triangle Fact 2.20.11, 169 Euler’s polyhedron formula Euler characteristic Fact 1.8.7, 16 even permutation matrix definition Definition 3.1.1, 179 even polynomial asymptotically stable polynomial Fact 11.17.6, 764 definition, 254 Everitt determinant of a partitioned positivesemidefinite matrix Fact 8.13.39, 541 exactly proper rational function definition Definition 4.7.1, 271 exactly proper rational transfer function definition Definition 4.7.2, 271 existence of transformation Hermitian matrix Fact 3.9.2, 200 orthogonal matrix Fact 3.9.5, 201 outer-product matrix Fact 3.9.1, 200 skew-Hermitian matrix

Fact 3.9.4, 201 existential statement definition, 2 logical equivalents Fact 1.7.4, 12 exogenous input definition, 845 exponent scalar inequality Fact 1.11.1, 25 exponential, see matrix exponential inequality Fact 1.17.49, 65 matrix logarithm Fact 11.14.26, 753 positive-definite matrix Fact 11.14.26, 753 exponential function complex numbers Fact 1.20.6, 78 convex function Fact 1.12.26, 37 inequality Fact 1.12.27, 37 limit Fact 1.11.17, 28 scalar inequalities Fact 1.12.28, 38 scalar inequality Fact 1.11.14, 27 Fact 1.11.15, 27 Fact 1.11.16, 28 Fact 1.11.18, 28 exponential inequality scalar case Fact 1.11.13, 27 extended infinite interval definition, xxxv extreme point convex set Fact 10.8.23, 695

Fan Krein-Milman theorem Fact 10.8.23, 695

F fact definition, 1 factorial bounds Fact 1.11.19, 29 inequality Fact 1.11.31, 32 Stirling’s formula Fact 1.11.20, 29 factorization asymptotically stable matrix Fact 11.18.22, 770 Bezout matrix Fact 5.15.24, 381 commutator Fact 5.15.33, 383 complex conjugate transpose Fact 5.15.23, 381 determinant Fact 5.15.8, 378 Fact 5.15.34, 383 diagonalizable matrix Fact 5.15.27, 382 diagonalizable over R Proposition 5.5.12, 324 Corollary 5.5.21, 327 Douglas-FillmoreWilliams lemma Theorem 8.6.2, 474 elementary divisor Fact 5.15.37, 384 full rank Fact 5.15.40, 384 generalized inverse Fact 6.5.25, 429 group generalized inverse Fact 6.6.13, 435

Hermitian matrix Fact 5.15.17, 380 Fact 5.15.25, 382 Fact 5.15.26, 382 Fact 5.15.41, 384 Fact 8.17.1, 558 idempotent matrix Fact 5.15.28, 383 Fact 5.15.30, 383 involutory matrix Fact 5.15.18, 380 Fact 5.15.31, 383 Fact 5.15.32, 383 Jordan form Fact 5.15.6, 378 lower triangular matrix Fact 5.15.10, 378 LULU decomposition Fact 5.15.11, 379 nilpotent matrix Fact 5.15.29, 383 nonsingular matrix Fact 5.15.12, 379 Fact 5.15.36, 384 orthogonal matrix Fact 5.15.15, 379 Fact 5.15.16, 380 Fact 5.15.31, 383 Fact 5.15.35, 384 partitioned matrix, 462 Proposition 2.8.3, 116 Proposition 2.8.4, 116 Fact 2.14.9, 145 Fact 2.16.2, 153 Fact 2.17.3, 159 Fact 2.17.4, 159 Fact 2.17.5, 160 Fact 6.5.25, 429 Fact 8.11.25, 520 Fact 8.11.26, 521 positive-definite matrix Fact 5.15.26, 382 Fact 5.18.4, 393 Fact 5.18.5, 393 Fact 5.18.6, 393

1021

Fact 5.18.8, 394 positive-semidefinite matrix Fact 5.15.22, 381 Fact 5.15.26, 382 Fact 5.18.2, 393 Fact 5.18.3, 393 Fact 5.18.7, 394 Fact 8.9.37, 499 Fact 8.9.38, 500 projector Fact 5.15.13, 379 Fact 5.15.17, 380 Fact 6.3.31, 410 range Theorem 8.6.2, 474 reflector Fact 5.15.14, 379 reverse-symmetric matrix Fact 5.9.14, 340 rotation-dilation Fact 2.19.2, 164 shear factor Fact 5.15.11, 379 similar matrices Fact 5.15.7, 378 skew-symmetric matrix Fact 5.15.37, 384 Fact 5.15.38, 384 symmetric matrix Corollary 5.3.9, 318 Fact 5.15.24, 381 ULU decomposition Fact 5.15.11, 379 unitary matrix Fact 5.15.9, 378 Fact 5.18.6, 393 upper triangular matrix Fact 5.15.9, 378 Fact 5.15.10, 378 Fan convex function Proposition 8.6.17, 596 trace of a Hermitian matrix product Fact 5.12.4, 363

1022

Fan

trace of a product of orthogonal matrices Fact 5.12.10, 365 Fan constant definition Fact 8.10.49, 511 Fan dominance theorem singular value Fact 9.14.19, 670 Farkas theorem linear system solution Fact 4.11.15, 305 fast Fourier transform circulant matrix Fact 5.16.7, 388 feedback interconnection realization Proposition 12.13.4, 845 Proposition 12.14.1, 847 Fact 12.22.8, 873 transfer function Fact 12.22.8, 873 feedback signal definition, 845 Fejer’s theorem positive-semidefinite matrix Fact 8.22.37, 592 Fer expansion time-varying dynamics Fact 11.13.4, 743 Fermat’s last theorem scalar equality Fact 1.13.18, 48 Fibonacci numbers determinant Fact 4.11.13, 303 generating function

Fact 4.11.13, 303 nonnegative matrix Fact 4.11.13, 303 field of values spectrum of convex hull Fact 8.14.7, 546 Fact 8.14.8, 547 final state controllability Fact 12.20.4, 864 controllable subspace Fact 12.20.3, 864 finite group group Fact 3.23.3, 243 Fact 3.23.4, 243 representation Fact 3.23.5, 245 finite interval definition, xxxv finite set definition, 2 pigeonhole principle Fact 1.7.14, 14 finite-sum solution Lyapunov equation Fact 12.21.17, 870 Finsler’s lemma positive-definite linear combination Fact 8.15.25, 554 Fact 8.15.26, 554 Fischer’s inequality positive-semidefinite matrix determinant Fact 8.13.36, 541 Fact 8.13.37, 541 positive-semidefinite matrix determinant reverse inequality Fact 8.13.42, 542 fixed-point theorem continuous function Theorem 10.3.10, 685

Corollary 10.3.11, 685 fixed-rank approximation Eckart-Young theorem Fact 9.14.28, 672 Frobenius norm Fact 9.14.28, 672 Fact 9.15.8, 678 least squares Fact 9.14.28, 672 Fact 9.15.8, 678 Schmidt-Mirsky theorem Fact 9.14.28, 672 singular value Fact 9.14.28, 672 Fact 9.15.8, 678 unitarily invariant norm Fact 9.14.28, 672 forced response definition, 797 forest definition Definition 1.6.3, 10 symmetric graph Fact 1.8.5, 15 Fourier matrix circulant matrix Fact 5.16.7, 388 Vandermonde matrix Fact 5.16.7, 388 Fourier transform Parseval’s theorem Fact 12.22.1, 872 Frame finite sequence for inverse matrix Fact 2.16.28, 158 Franck maximum singular value lower bound on distance to singularity Fact 9.14.6, 666

full row rank ´ Frechet derivative definition, 705 free response definition, 797 frequency domain controllability Gramian Corollary 12.11.5, 840 frequency response imaginary part Fact 12.22.5, 872 real part Fact 12.22.5, 872 transfer function Fact 12.22.5, 872 Friedland matrix exponential and singular value Fact 11.16.15, 762 Frobenius similar to transpose Corollary 5.3.8, 317 singular value Corollary 9.6.7, 617 symmetric matrix factorization Fact 5.15.24, 381 Frobenius canonical form, see multicompanion form definition, 396 Frobenius inequality rank of partitioned matrix Fact 2.11.14, 134 Fact 6.5.15, 426 Frobenius matrix definition, 396 Frobenius norm absolute value Fact 9.13.11, 661 adjugate Fact 9.8.15, 629

Cauchy-Schwarz inequality Corollary 9.3.9, 607 commutator Fact 9.9.26, 641 Fact 9.9.27, 641 definition, 601 determinant Fact 9.8.39, 633 dissipative matrix Fact 11.15.3, 756 eigenvalue Fact 9.11.3, 654 Fact 9.11.5, 655 eigenvalue bound Fact 9.12.3, 656 eigenvalue perturbation Fact 9.12.4, 657 Fact 9.12.9, 658 Fact 9.12.10, 659 fixed-rank approximation Fact 9.14.28, 672 Fact 9.15.8, 678 Hermitian matrix Fact 9.9.41, 644 inequality Fact 9.9.25, 640 Kronecker product Fact 9.14.38, 676 matrix difference Fact 9.9.25, 640 matrix exponential Fact 11.14.32, 755 Fact 11.15.3, 756 maximum singular value bound Fact 9.13.13, 661 normal matrix Fact 9.12.9, 658 outer-product matrix Fact 9.7.26, 626 polar decomposition Fact 9.9.42, 645 positive-semidefinite matrix Fact 9.8.39, 633

1023

Fact 9.9.12, 638 Fact 9.9.15, 638 Fact 9.9.27, 641 rank Fact 9.11.4, 654 Fact 9.14.28, 672 Fact 9.15.8, 678 Schatten norm, 603 Fact 9.8.20, 630 Schur product Fact 9.14.33, 675 Fact 9.14.35, 676 Schur’s inequality Fact 9.11.3, 654 spectral radius Fact 9.13.12, 661 trace Fact 9.11.3, 654 Fact 9.11.4, 654 Fact 9.11.5, 655 Fact 9.12.2, 656 trace norm Fact 9.9.11, 637 triangle inequality Fact 9.9.13, 638 unitarily invariant norm Fact 9.14.33, 675 unitary matrix Fact 9.9.42, 645 Fujii-Kubo polynomial root bound Fact 11.20.9, 780 Fujiwara’s bound polynomial Fact 11.20.8, 779 full column rank definition, 104 equivalent properties Theorem 2.6.1, 107 nonsingular equivalence Corollary 2.6.6, 110 full rank definition, 104 full row rank

1024

full row rank

definition, 104 equivalent properties Theorem 2.6.1, 107 nonsingular equivalence Corollary 2.6.6, 110 full-rank factorization generalized inverse Fact 6.4.12, 414 idempotent matrix Fact 3.12.23, 219

union Fact 1.7.11, 13 Fact 1.7.12, 13 function composition matrix multiplication Theorem 2.1.3, 88 fundamental theorem of algebra definition, 254

full-state feedback controllable subspace Proposition 12.6.5, 810 controllably asymptotically stable Proposition 12.7.2, 816 determinant Fact 12.22.14, 874 invariant zero Proposition 12.10.10, 836 Fact 12.22.14, 874 stabilizability Proposition 12.8.2, 820 uncontrollable eigenvalue Proposition 12.6.14, 813 unobservable eigenvalue Proposition 12.3.14, 803 unobservable subspace Proposition 12.3.5, 801

fundamental triangle inequality Ramus Fact 2.20.11, 169 triangle Fact 2.20.11, 169 Wu Fact 2.20.11, 169

function definition, 4 graph Fact 1.8.1, 15 Fact 1.8.2, 15 Fact 1.8.3, 15 intersection Fact 1.7.11, 13 Fact 1.7.12, 13 relation Proposition 1.5.1, 6

gamma logarithm Fact 1.9.5, 20

Furuta inequality positive-definite matrix Fact 8.10.51, 512 positive-semidefinite matrix inequality Proposition 8.6.7, 476 spectral order Fact 8.20.3, 575

G Galois quintic polynomial Fact 3.23.4, 243

gap topology minimal principal angle Fact 10.9.19, 698 subspace Fact 10.9.19, 698 Gastinel

distance to singularity of a nonsingular matrix Fact 9.14.7, 666 ˆ Gateaux differential definition, 705 Gaussian density integral Fact 8.16.1, 556 Fact 8.16.2, 556 Fact 8.16.3, 556 Fact 8.16.4, 557 Fact 8.16.5, 557 Fact 11.13.16, 746 positive-definite matrix Fact 8.16.6, 557 generalized algebraic multiplicity definition, 331 generalized Cayley-Hamilton theorem commuting matrices Fact 4.9.7, 284 generalized eigensolution eigenvector Fact 11.13.7, 744 generalized eigenvalue definition, 330 pencil Proposition 5.7.3, 332 Proposition 5.7.4, 333 regular pencil Proposition 5.7.3, 332 Proposition 5.7.4, 333 singular pencil Proposition 5.7.3, 332 generalized eigenvector eigenvalue Fact 5.14.9, 371 generalized Furuta inequality

generalized inverse positive-definite matrix inequality Fact 8.10.53, 512 generalized geometric mean positive-definite matrix Fact 8.10.45, 510 generalized geometric multiplicity definition, 331 ¨ generalized Holder inequality vector Fact 9.7.34, 626 generalized inverse (1,3) inverse Fact 6.3.13, 406 (1,4) inverse Fact 6.3.13, 406 adjugate Fact 6.3.6, 404 Fact 6.3.7, 405 Fact 6.5.16, 427 basic properties Proposition 6.1.6, 399 block-circulant matrix Fact 6.5.2, 423 centrohermitian matrix Fact 6.3.30, 410 characteristic polynomial Fact 6.3.19, 408 characterization Fact 6.4.1, 411 complex conjugate transpose Fact 6.3.9, 405 Fact 6.3.10, 405 Fact 6.3.12, 406 Fact 6.3.15, 407 Fact 6.3.16, 407 Fact 6.3.17, 407 Fact 6.3.21, 408 Fact 6.3.26, 409

Fact 6.3.27, 410 Fact 6.4.8, 413 Fact 6.4.9, 414 Fact 6.4.10, 414 Fact 6.6.17, 436 Fact 6.6.18, 437 Fact 6.6.19, 437 congruence Fact 8.21.5, 578 convergent sequence Fact 6.3.34, 411 Fact 6.3.35, 411 definition, 397 determinant Fact 6.5.26, 429 Fact 6.5.27, 430 Fact 6.5.28, 430 difference Fact 6.4.36, 419 equality Fact 6.3.32, 411 factorization Fact 6.5.25, 429 full-rank factorization Fact 6.4.12, 414 group generalized inverse Fact 6.6.8, 434 Hermitian matrix Fact 6.3.20, 408 Fact 6.4.4, 413 Fact 8.21.12, 579 idempotent matrix Fact 5.12.18, 367 Fact 6.3.21, 408 Fact 6.3.22, 408 Fact 6.3.23, 408 Fact 6.3.24, 409 Fact 6.3.25, 409 Fact 6.3.26, 409 Fact 6.4.21, 416 Fact 6.4.22, 416 Fact 6.4.23, 416 Fact 6.4.25, 417 Fact 6.4.28, 417 induced lower bound Fact 9.8.44, 635 inertia

1025

Fact 6.3.20, 408 Fact 8.21.12, 579 integral Fact 11.13.10, 745 Jordan canonical form Fact 6.6.10, 434 Kronecker product Fact 7.4.32, 449 least squares Fact 9.15.4, 677 Fact 9.15.5, 678 Fact 9.15.6, 678 Fact 9.15.7, 678 left inverse Corollary 6.1.4, 398 Fact 6.4.44, 421 Fact 6.4.45, 421 left-inner matrix Fact 6.3.8, 405 left-invertible matrix Proposition 6.1.5, 398 linear matrix equation Fact 6.4.43, 421 linear system Proposition 6.1.7, 400 lower bound Fact 9.8.44, 635 matrix difference Fact 6.4.42, 420 matrix exponential Fact 11.13.10, 745 matrix inversion lemma Fact 6.4.5, 413 matrix limit Fact 6.3.18, 408 matrix product Fact 6.4.6, 413 Fact 6.4.7, 413 Fact 6.4.11, 414 Fact 6.4.12, 414 Fact 6.4.14, 414 Fact 6.4.15, 415 Fact 6.4.16, 415 Fact 6.4.17, 415 Fact 6.4.19, 416 Fact 6.4.20, 416

1026

generalized inverse

Fact 6.4.24, 417 Fact 6.4.25, 417 Fact 6.4.26, 417 Fact 6.4.33, 418 Fact 6.4.34, 418 matrix sum Fact 6.4.37, 419 Fact 6.4.38, 419 Fact 6.4.39, 420 Fact 6.4.40, 420 Fact 6.4.41, 420 maximum singular value Fact 9.14.8, 666 Fact 9.14.30, 673 Newton-Raphson algorithm Fact 6.3.34, 411 normal matrix Proposition 6.1.6, 399 Fact 6.3.15, 407 Fact 6.3.16, 407 null space Proposition 6.1.6, 399 Fact 6.3.23, 408 observability matrix Fact 12.20.19, 866 outer-product matrix Fact 6.3.2, 404 outer-product perturbation Fact 6.4.2, 412 Fact 6.4.3, 413 partial isometry Fact 6.3.27, 410 partitioned matrix Fact 6.3.29, 410 Fact 6.5.1, 422 Fact 6.5.2, 423 Fact 6.5.3, 423 Fact 6.5.4, 423 Fact 6.5.13, 426 Fact 6.5.17, 427 Fact 6.5.18, 427 Fact 6.5.19, 428 Fact 6.5.20, 428 Fact 6.5.21, 428 Fact 6.5.22, 428

Fact 6.5.23, 429 Fact 6.5.24, 429 Fact 8.21.22, 583 positive-definite matrix Proposition 6.1.6, 399 Fact 6.4.9, 414 Fact 6.4.10, 414 positive-semidefinite matrix Proposition 6.1.6, 399 Fact 6.4.5, 413 Fact 8.21.1, 577 Fact 8.21.2, 577 Fact 8.21.3, 578 Fact 8.21.4, 578 Fact 8.21.6, 578 Fact 8.21.7, 578 Fact 8.21.8, 579 Fact 8.21.9, 579 Fact 8.21.10, 579 Fact 8.21.11, 579 Fact 8.21.13, 580 Fact 8.21.15, 580 Fact 8.21.16, 580 Fact 8.21.17, 580 Fact 8.21.18, 581 Fact 8.21.19, 582 Fact 8.21.20, 583 Fact 8.21.22, 583 Fact 8.21.23, 583 projector Fact 6.3.3, 404 Fact 6.3.4, 404 Fact 6.3.5, 404 Fact 6.3.25, 409 Fact 6.3.26, 409 Fact 6.3.31, 410 Fact 6.4.18, 415 Fact 6.4.19, 416 Fact 6.4.20, 416 Fact 6.4.21, 416 Fact 6.4.22, 416 Fact 6.4.24, 417 Fact 6.4.26, 417 Fact 6.4.27, 417 Fact 6.4.28, 417 Fact 6.4.36, 419 Fact 6.4.46, 421

Fact 6.4.51, 422 Fact 6.5.10, 425 range Proposition 6.1.6, 399 Fact 6.3.23, 408 Fact 6.4.47, 421 Fact 6.4.48, 421 Fact 6.5.3, 423 range-Hermitian matrix Proposition 6.1.6, 399 Fact 6.3.10, 405 Fact 6.3.11, 406 Fact 6.3.15, 407 Fact 6.3.16, 407 Fact 6.4.13, 414 Fact 6.4.29, 417 Fact 6.4.30, 418 Fact 6.4.31, 418 Fact 6.4.32, 418 rank Fact 6.3.9, 405 Fact 6.3.21, 408 Fact 6.3.35, 411 Fact 6.4.2, 412 Fact 6.4.8, 413 Fact 6.4.49, 421 Fact 6.5.6, 423 Fact 6.5.8, 424 Fact 6.5.9, 425 Fact 6.5.12, 425 Fact 6.5.13, 426 Fact 6.5.14, 426 rank subtractivity partial ordering Fact 6.5.30, 431 right inverse Corollary 6.1.4, 398 right-inner matrix Fact 6.3.8, 405 right-invertible matrix Proposition 6.1.5, 398 sequence Fact 6.3.35, 411 singular value Fact 6.3.28, 410 singular value decomposition

golden ratio Fact 6.3.14, 407 square root Fact 8.21.4, 578 star partial ordering Fact 8.20.7, 576 star-dagger matrix Fact 6.3.12, 406 sum Fact 6.5.1, 422 Fact 6.5.2, 423 trace Fact 6.3.21, 408 uniqueness Theorem 6.1.1, 397 unitary matrix Fact 6.3.33, 411 Urquhart Fact 6.3.13, 406 ¨ generalized Lowner partial ordering definition Fact 8.20.9, 577 generalized multispectrum definition, 330 generalized projector range-Hermitian matrix Fact 3.6.4, 192 generalized Schur inequality eigenvalues Fact 9.11.6, 655 generalized spectrum definition, 330 generating function Fibonacci numbers Fact 4.11.13, 303 geometric mean arithmetic mean Fact 1.17.21, 58 Fact 1.17.23, 58 Fact 1.17.24, 59 Fact 1.17.25, 59 Fact 1.17.26, 59 Fact 1.17.27, 59

determinant Fact 8.10.43, 508 matrix exponential Fact 8.10.44, 510 matrix logarithm Fact 11.14.39, 755 Muirhead’s theorem Fact 1.17.25, 59 nondecreasing function Fact 8.10.43, 508 Fact 8.10.44, 510 positive-definite matrix Fact 8.10.43, 508 Fact 8.10.46, 510 Fact 8.10.47, 511 Fact 8.22.53, 595 positive-semidefinite matrix Fact 8.10.43, 508 Riccati equation Fact 12.23.4, 876 scalar inequality Fact 1.13.6, 43 Schur product Fact 8.22.53, 595 geometric multiplicity algebraic multiplicity Proposition 5.5.3, 322 block-diagonal matrix Proposition 5.5.13, 324 cascaded systems Fact 12.22.15, 874 controllability Fact 12.20.14, 865 defect Proposition 4.5.2, 268 definition Definition 4.5.1, 267 partitioned matrix Proposition 5.5.13, 324 rank Proposition 4.5.2, 268 similar matrices Proposition 5.5.10, 324

1027

geometric-mean decomposition unitary matrix Fact 5.9.32, 344 Gershgorin circle theorem eigenvalue bounds Fact 4.10.17, 293 Fact 4.10.21, 294 Gerstenhaber dimension of the algebra generated by two commuting matrices Fact 5.10.21, 347 Gibson diagonal entries of similar matrices Fact 5.9.15, 340 Givens rotation orthogonal matrix Fact 5.15.16, 380 global asymptotic stability nonlinear system Theorem 11.7.2, 725 globally asymptotically stable equilibrium definition Definition 11.7.1, 725 Gohberg-Semencul formulas Bezout matrix Fact 4.8.6, 277 golden mean positive-definite solution of a Riccati equation Fact 12.23.4, 876 Riccati equation Fact 12.23.4, 876 golden ratio difference equation Fact 4.11.13, 303 Riccati equation

1028

golden ratio

Fact 12.23.4, 876 Golden-Thompson inequality matrix exponential Fact 11.14.28, 753 Fact 11.16.4, 760 Gordan’s theorem positive vector Fact 4.11.16, 305 gradient definition, 687 Gram matrix positive-semidefinite matrix Fact 8.9.37, 499 Gram-Schmidt orthonormalization upper triangular matrix factorization Fact 5.15.5, 377 Gramian controllability Fact 12.20.17, 865 stabilization Fact 12.20.17, 865 Graph definition, 4 graph antisymmetric graph Fact 3.21.1, 240 cycle Fact 1.8.4, 15 definition, 9 directed cut Fact 4.11.3, 297 function Fact 1.8.1, 15 Fact 1.8.2, 15 Fact 1.8.3, 15 Hamiltonian cycle Fact 1.8.6, 16 irreducible matrix Fact 4.11.3, 297 Laplacian matrix Fact 8.15.1, 550

spanning path Fact 1.8.6, 16 symmetric graph Fact 3.21.1, 240 tournament Fact 1.8.6, 16 walk Fact 4.11.1, 297 graph of a matrix adjacency matrix Proposition 3.2.5, 185 definition Definition 3.2.4, 185 greatest common divisor definition, 255 greatest lower bound projector Fact 6.4.46, 421 greatest lower bound for a partial ordering definition Definition 1.5.9, 8 Gregory’s series infinite series Fact 1.20.8, 79 Greville generalized inverse of a matrix product Fact 6.4.14, 414 Fact 6.4.16, 415 generalized inverse of a partitioned matrix Fact 6.5.17, 427 group alternating group Fact 3.23.4, 243 circulant matrix Fact 3.23.4, 245 classical Proposition 3.3.6, 187 cyclic group Fact 3.23.4, 243 definition Definition 3.3.3, 186

dihedral group Fact 3.23.4, 243 equivalence relation Proposition 3.4.1, 188 Proposition 3.4.2, 188 finite group Fact 3.23.3, 243 Fact 3.23.4, 243 Fact 3.23.5, 245 icosahedral group Fact 3.23.4, 243 isomorphism Proposition 3.3.5, 186 Lagrange Fact 3.23.6, 246 Lie group Definition 11.6.1, 722 Proposition 11.6.2, 722 matrix exponential Proposition 11.6.7, 724 octahedral group Fact 3.23.4, 243 orthogonal matrix Fact 3.23.8, 246 pathwise connected Proposition 11.6.8, 724 permutation group Fact 3.23.4, 243 real numbers Fact 3.23.1, 242 rotation matrix Proposition 3.3.6, 187 subgroup Fact 3.23.4, 243 Fact 3.23.6, 246 symmetric group Fact 3.23.4, 243 symmetry groups Fact 3.23.4, 243 tetrahedral group Fact 3.23.4, 243 transpose Fact 3.23.7, 246 unipotent matrix Fact 3.23.12, 247 Fact 11.22.1, 786 unit sphere Fact 3.23.9, 246

H2 norm upper triangular matrix Fact 3.23.12, 247 Fact 11.22.1, 786 group generalized inverse complex conjugate transpose Fact 6.6.11, 434 definition, 403 discrete-time Lyapunov-stable matrix Fact 11.21.13, 784 factorization Fact 6.6.13, 435 generalized inverse Fact 6.6.8, 434 idempotent matrix Proposition 6.2.3, 403 integral Fact 11.13.13, 745 Fact 11.13.14, 745 irreducible matrix Fact 6.6.21, 438 Kronecker product Fact 7.4.33, 449 limit Fact 6.6.15, 435 matrix exponential Fact 11.13.13, 745 Fact 11.13.14, 745 Fact 11.18.5, 766 Fact 11.18.6, 766 normal matrix Fact 6.6.11, 434 null space Proposition 6.2.3, 403 positive-semidefinite matrix Fact 8.21.1, 577 range Proposition 6.2.3, 403 range-Hermitian matrix Fact 6.6.9, 434 singular value decomposition Fact 6.6.16, 435

trace Fact 6.6.7, 434 group-invertible matrix almost nonnegative matrix Fact 11.19.4, 775 complementary subspaces Corollary 3.5.8, 191 definition Definition 3.1.1, 179 equivalent characterizations Fact 3.6.1, 191 Hermitian matrix Fact 6.6.19, 437 idempotent matrix Proposition 3.1.6, 183 Proposition 3.5.9, 191 Proposition 6.2.3, 403 Fact 5.11.8, 351 index of a matrix Proposition 3.5.6, 190 Corollary 5.5.9, 323 Fact 5.14.2, 369 inertia Fact 5.8.5, 334 Jordan canonical form Fact 6.6.10, 434 Kronecker product Fact 7.4.17, 446 Fact 7.4.33, 449 Lyapunov-stable matrix Fact 11.18.2, 766 matrix exponential Fact 11.18.14, 768 matrix power Fact 3.6.2, 192 Fact 6.6.20, 438 N-matrix Fact 11.19.4, 775 normal matrix Fact 6.6.18, 437 outer-product matrix Fact 5.14.3, 370

1029

positive-definite matrix Fact 8.10.12, 502 positive-semidefinite matrix Fact 8.10.12, 502 projector Fact 3.13.21, 227 range Fact 5.14.2, 369 range-Hermitian matrix Proposition 3.1.6, 183 Fact 6.6.17, 436 rank Fact 5.8.5, 334 Fact 5.14.2, 369 semistable matrix Fact 11.18.3, 766 similar matrices Proposition 3.4.5, 189 Fact 5.9.7, 339 spectrum Proposition 5.5.20, 326 square root Fact 5.15.20, 381 stable subspace Proposition 11.8.8, 729 tripotent matrix Proposition 3.1.6, 183 unitarily similar matrices Proposition 3.4.5, 189

H H2 norm controllability Gramian Corollary 12.11.4, 839 Corollary 12.11.5, 840 definition Definition 12.11.2, 838 observability Gramian Corollary 12.11.4, 839 Parseval’s theorem Theorem 12.11.3, 839

1030

H2 norm

partitioned transfer function Fact 12.22.16, 875 Fact 12.22.17, 875 quadratic performance measure Proposition 12.15.1, 849 submultiplicative norm Fact 12.22.20, 875 sum of transfer functions Proposition 12.11.6, 840 transfer function Fact 12.22.16, 875 Fact 12.22.17, 875 Fact 12.22.18, 875 Fact 12.22.19, 875 Hadamard matrix orthogonal matrix Fact 5.16.9, 391 Hadamard product, see Schur product Hadamard’s inequality determinant Fact 8.13.34, 540 Fact 8.13.35, 540 determinant bound Fact 9.11.1, 653 determinant of a partitioned matrix Fact 6.5.26, 429 positive-semidefinite matrix determinant Fact 8.18.11, 562 Hadamard-Fischer inequality positive-semidefinite matrix Fact 8.13.37, 541 Hahn-Banach theorem inner product inequality Fact 10.9.13, 696

half-vectorization operator Kronecker product, 458 Hamiltonian block decomposition Proposition 12.17.5, 856 closed-loop spectrum Proposition 12.16.14, 853 definition, 853 Hamiltonian matrix Proposition 12.16.13, 853 Jordan form Fact 12.23.1, 875 Riccati equation Theorem 12.17.9, 857 Proposition 12.16.14, 853 Corollary 12.16.15, 854 spectral factorization Proposition 12.16.13, 853 spectrum Theorem 12.17.9, 857 Proposition 12.16.13, 853 Proposition 12.17.5, 856 Proposition 12.17.7, 857 Proposition 12.17.8, 857 Lemma 12.17.4, 855 Lemma 12.17.6, 856 stabilizability Fact 12.23.1, 875 stabilizing solution Corollary 12.16.15, 854 uncontrollable eigenvalue Proposition 12.17.7, 857 Proposition 12.17.8, 857 Lemma 12.17.4, 855 Lemma 12.17.6, 856

unobservable eigenvalue Proposition 12.17.7, 857 Proposition 12.17.8, 857 Lemma 12.17.4, 855 Lemma 12.17.6, 856 Hamiltonian cycle definition Definition 1.6.3, 10 graph Fact 1.8.6, 16 tournament Fact 1.8.6, 16 Hamiltonian graph definition Definition 1.6.3, 10 Hamiltonian matrix Cayley transform Fact 3.20.12, 239 characteristic polynomial Fact 4.9.22, 287 Fact 4.9.24, 287 definition Definition 3.1.5, 183 equality Fact 3.20.1, 238 Hamiltonian Proposition 12.16.13, 853 inverse matrix Fact 3.20.5, 239 matrix exponential Proposition 11.6.7, 724 matrix logarithm Fact 11.14.19, 751 matrix sum Fact 3.20.5, 239 orthogonal matrix Fact 3.20.13, 240 orthosymplectic matrix Fact 3.20.13, 240 partitioned matrix Proposition 3.1.7, 184 Fact 3.20.6, 239

Hermite-Biehler theorem Fact 3.20.8, 239 Fact 4.9.23, 287 Fact 5.12.21, 368 skew reflector Fact 3.20.3, 238 skew-involutory matrix Fact 3.20.2, 238 Fact 3.20.3, 238 skew-symmetric matrix Fact 3.7.34, 198 Fact 3.20.3, 238 Fact 3.20.8, 239 spectrum Proposition 5.5.20, 326 symplectic matrix Fact 3.20.2, 238 Fact 3.20.12, 239 Fact 3.20.13, 240 symplectic similarity Fact 3.20.4, 239 trace Fact 3.20.7, 239 unit imaginary matrix Fact 3.20.3, 238 Hamiltonian path definition Definition 1.6.3, 10 Hankel matrix block-Hankel matrix Fact 3.18.3, 234 definition Definition 3.1.3, 181 Hilbert matrix Fact 3.18.4, 234 Markov block-Hankel matrix definition, 826 rank Fact 3.18.8, 235 rational function Fact 4.8.8, 279 symmetric matrix Fact 3.18.2, 234 Toeplitz matrix

Fact 3.18.1, 234 trigonometric matrix Fact 3.18.8, 235 Hanner inequality H¨ older norm Fact 9.7.21, 625 Schatten norm Fact 9.9.36, 643 Hansen trace of a convex function Fact 8.12.34, 531 Hardy H¨ older-induced norm Fact 9.8.17, 629 Hardy inequality sum of powers Fact 1.17.43, 64 Hardy-Hilbert inequality sum of powers Fact 1.18.13, 69 Fact 1.18.14, 69 Hardy-Littlewood rearrangement inequality sum of products Fact 1.18.4, 66 sum of products inequality Fact 1.18.5, 67 Hardy-Littlewood-Polya theorem doubly stochastic matrix Fact 3.9.6, 201 harmonic mean arithmetic-mean inequality Fact 1.17.16, 57 Fact 1.17.17, 57 arithmetic-mean– geometric-mean inequality Fact 1.17.15, 56

1031

Dragomir’s inequality Fact 1.18.24, 72 harmonic steady-state response linear system Theorem 12.12.1, 841 Hartwig rank of an idempotent matrix Fact 3.12.27, 220 Haynsworth positive-semidefinite matrix Fact 5.14.12, 371 Schur complement of a partitioned matrix Fact 6.5.29, 430 Haynsworth inertia additivity formula Schur complement Fact 6.5.5, 423 Heinz inequality unitarily invariant norm Fact 9.9.49, 646 Heinz mean scalar inequality Fact 1.12.39, 41 Heisenberg group unipotent matrix Fact 3.23.12, 247 Fact 11.22.1, 786 upper triangular matrix Fact 3.23.12, 247 Fact 11.22.1, 786 Henrici eigenvalue bound Fact 9.11.3, 654 Hermite-Biehler theorem asymptotically stable polynomial

1032

Hermite-Biehler theorem

Fact 11.17.6, 764 Hermite-Hadamard inequality convex function Fact 1.10.6, 24 Hermitian matrix, see symmetric matrix additive decomposition Fact 3.7.29, 198 adjugate Fact 3.7.10, 193 affine mapping Fact 3.7.14, 195 block-diagonal matrix Fact 3.7.8, 193 Cartesian decomposition Fact 3.7.27, 197 Fact 3.7.28, 197 Fact 3.7.29, 198 cogredient transformation Fact 8.17.4, 558 Fact 8.17.6, 559 commutator Fact 3.8.1, 199 Fact 3.8.3, 199 Fact 9.9.30, 641 commuting matrices Fact 5.14.28, 374 complex conjugate transpose Fact 3.7.13, 195 Fact 5.9.10, 340 Fact 6.6.19, 437 congruent matrices Proposition 3.4.5, 189 Corollary 5.4.7, 321 convergent sequence Fact 11.14.7, 749 Fact 11.14.8, 749 convex function Fact 8.12.33, 530 Fact 8.12.34, 531 convex hull Fact 8.18.8, 561

defect Fact 5.8.7, 335 Fact 8.9.7, 496 definition Definition 3.1.1, 179 determinant Corollary 8.4.10, 470 Fact 3.7.21, 197 Fact 8.13.7, 535 diagonal Fact 8.18.8, 561 diagonal entry Corollary 8.4.7, 469 Fact 8.18.8, 561 Fact 8.18.9, 562 Fact 8.18.13, 563 diagonal matrix Corollary 5.4.5, 320 Lemma 8.5.1, 473 eigenvalue Theorem 8.4.5, 468 Theorem 8.4.9, 469 Theorem 8.4.11, 470 Corollary 8.4.2, 467 Corollary 8.4.6, 469 Corollary 8.4.7, 469 Corollary 8.6.19, 486 Lemma 8.4.3, 467 Lemma 8.4.4, 468 Fact 8.10.4, 501 Fact 8.15.21, 553 Fact 8.15.32, 555 Fact 8.18.8, 561 Fact 8.18.9, 562 Fact 8.18.15, 563 Fact 8.18.16, 563 Fact 8.19.4, 565 Fact 8.19.17, 569 Fact 8.19.18, 569 Fact 8.22.29, 590 eigenvalue bound Fact 9.12.3, 656 eigenvalue inequality Lemma 8.4.1, 467 Fact 8.19.3, 565 eigenvalue perturbation Fact 4.10.28, 296 eigenvalues

Fact 8.18.13, 563 existence of transformation Fact 3.9.2, 200 factorization Fact 5.15.17, 380 Fact 5.15.25, 382 Fact 5.15.26, 382 Fact 5.15.41, 384 Fact 8.17.1, 558 Frobenius norm Fact 9.9.41, 644 generalized inverse Fact 6.3.20, 408 Fact 6.4.4, 413 Fact 8.21.12, 579 group-invertible matrix Fact 6.6.19, 437 inequality Fact 8.9.13, 496 Fact 8.9.15, 497 Fact 8.9.20, 497 Fact 8.13.27, 539 Fact 8.13.31, 539 inertia Theorem 8.4.11, 470 Proposition 5.4.6, 320 Fact 5.8.6, 334 Fact 5.8.8, 335 Fact 5.8.12, 336 Fact 5.8.13, 336 Fact 5.8.14, 336 Fact 5.8.15, 336 Fact 5.8.16, 336 Fact 5.8.17, 336 Fact 5.8.18, 337 Fact 5.8.19, 337 Fact 5.12.1, 362 Fact 6.3.20, 408 Fact 8.10.15, 503 Fact 8.21.12, 579 Fact 8.21.14, 580 Fact 12.21.1, 866 Fact 12.21.2, 867 Fact 12.21.3, 867 Fact 12.21.4, 867 Fact 12.21.5, 867 Fact 12.21.6, 868

Hermitian matrix Fact 12.21.7, 868 Fact 12.21.8, 868 Fact 12.21.10, 869 Fact 12.21.11, 869 Fact 12.21.12, 869 involutory matrix Fact 3.14.2, 229 Kronecker product Fact 7.4.17, 446 Fact 8.22.29, 590 Kronecker sum Fact 7.5.8, 451 limit Fact 8.10.1, 501 linear combination Fact 8.15.25, 554 Fact 8.15.26, 554 Fact 8.15.27, 554 linear combination of projectors Fact 5.19.10, 395 matrix exponential Proposition 11.2.8, 713 Proposition 11.2.9, 715 Proposition 11.4.5, 719 Corollary 11.2.6, 712 Fact 11.14.7, 749 Fact 11.14.8, 749 Fact 11.14.21, 752 Fact 11.14.28, 753 Fact 11.14.29, 754 Fact 11.14.31, 754 Fact 11.14.32, 755 Fact 11.14.34, 755 Fact 11.15.1, 756 Fact 11.16.4, 760 Fact 11.16.5, 761 Fact 11.16.13, 762 Fact 11.16.17, 763 maximum eigenvalue Lemma 8.4.3, 467 Fact 5.11.5, 350 Fact 8.10.3, 501 maximum singular value Fact 5.11.5, 350 Fact 9.9.41, 644 minimum eigenvalue Lemma 8.4.3, 467

Fact 8.10.3, 501 normal matrix Proposition 3.1.6, 183 outer-product matrix Fact 3.7.18, 196 Fact 3.9.2, 200 partitioned matrix Fact 3.7.27, 197 Fact 4.10.28, 296 Fact 5.8.19, 337 Fact 5.12.1, 362 Fact 6.5.5, 423 positive-definite matrix Fact 5.15.41, 384 Fact 8.10.13, 502 Fact 8.13.7, 535 positive-semidefinite matrix Fact 5.15.41, 384 Fact 8.9.11, 496 Fact 8.10.13, 502 product Example 5.5.18, 326 projector Fact 3.13.2, 224 Fact 3.13.13, 225 Fact 3.13.20, 227 Fact 5.15.17, 380 Fact 8.9.24, 497 properties of < and ≤ Proposition 8.1.2, 460 quadratic form Fact 3.7.6, 193 Fact 3.7.7, 193 Fact 8.15.25, 554 Fact 8.15.26, 554 Fact 8.15.27, 554 Fact 8.15.32, 555 quadratic matrix equation Fact 5.11.4, 350 range Lemma 8.6.1, 474 rank Fact 3.7.22, 197 Fact 3.7.30, 198

1033

Fact 5.8.6, 334 Fact 5.8.7, 335 Fact 8.9.7, 496 Rayleigh quotient Lemma 8.4.3, 467 reflector Fact 3.14.2, 229 Schatten norm Fact 9.9.27, 641 Fact 9.9.39, 644 Schur decomposition Corollary 5.4.5, 320 Schur product Fact 8.22.29, 590 Fact 8.22.34, 591 signature Fact 5.8.6, 334 Fact 5.8.7, 335 Fact 8.10.17, 503 similar matrices Proposition 5.5.12, 324 simultaneous diagonalization Fact 8.17.1, 558 Fact 8.17.4, 558 Fact 8.17.6, 559 skew-Hermitian matrix Fact 3.7.9, 193 Fact 3.7.28, 197 skew-symmetric matrix Fact 3.7.9, 193 spectral abscissa Fact 5.11.5, 350 spectral radius Fact 5.11.5, 350 spectral variation Fact 9.12.5, 657 Fact 9.12.7, 658 spectrum Proposition 5.5.20, 326 Lemma 8.4.8, 469 spread Fact 8.15.32, 555 strong majorization Fact 8.18.8, 561 submatrix Theorem 8.4.5, 468

1034

Hermitian matrix

Corollary 8.4.6, 469 Lemma 8.4.4, 468 Fact 5.8.8, 335 symmetric matrix Fact 3.7.9, 193 trace Proposition 8.4.13, 471 Corollary 8.4.10, 470 Lemma 8.4.12, 471 Fact 3.7.13, 195 Fact 3.7.22, 197 Fact 8.12.18, 526 Fact 8.12.40, 532 trace of a product Fact 8.12.6, 524 Fact 8.12.7, 524 Fact 8.12.8, 524 Fact 8.12.16, 526 trace of product Fact 5.12.4, 363 Fact 5.12.5, 363 Fact 8.19.19, 570 tripotent matrix Fact 3.16.3, 231 unitarily invariant norm Fact 9.9.5, 636 Fact 9.9.41, 644 Fact 9.9.43, 645 Fact 11.16.13, 762 unitarily similar matrices Proposition 3.4.5, 189 Proposition 5.5.22, 327 Corollary 5.4.5, 320 unitary matrix Fact 3.11.21, 208 Fact 8.17.1, 558 Fact 11.14.34, 755 Hermitian matrix eigenvalue Cauchy interlacing theorem Lemma 8.4.4, 468 inclusion principle Theorem 8.4.5, 468 monotonicity theorem Theorem 8.4.9, 469

Fact 8.10.4, 501 Weyl’s inequality Theorem 8.4.9, 469 Fact 8.10.4, 501 Hermitian matrix inertia equality Styan Fact 8.10.15, 503 Hermitian part eigenvalue Fact 5.11.24, 355 Hermitian perturbation Lidskii-MirskyWielandt theorem Fact 9.12.4, 657 Heron mean logarithmic mean Fact 1.12.38, 41 Heron’s formula triangle Fact 2.20.11, 169 Hessenberg matrix lower or upper Definition 3.1.3, 181 Hessian definition, 688 hidden convexity quadratic form Fact 8.14.11, 547 Hilbert matrix Hankel matrix Fact 3.18.4, 234 positive-definite matrix Fact 3.18.4, 234 Hille-Yosida theorem matrix exponential bound Fact 11.15.8, 758 Hirsch’s theorem eigenvalue bound Fact 5.11.21, 354 Fact 9.11.8, 655

Hlawka’s equality norm equality Fact 9.7.4, 618 Hlawka’s inequality Euclidean norm Fact 9.7.7, 620 norm inequality Fact 9.7.4, 618 Hoffman eigenvalue perturbation Fact 9.12.9, 658 Hoffman-Wielandt theorem eigenvalue perturbation Fact 9.12.9, 658 ¨ Holder norm compatible norms Proposition 9.3.5, 605 complex conjugate Fact 9.7.33, 626 definition, 598 eigenvalue Fact 9.11.6, 655 eigenvalue bound Fact 9.11.8, 655 Hanner inequality Fact 9.7.21, 625 H¨ older-induced norm Proposition 9.4.11, 611 Fact 9.7.28, 626 Fact 9.8.12, 628 Fact 9.8.17, 629 Fact 9.8.18, 630 Fact 9.8.19, 630 Fact 9.8.29, 631 inequality Proposition 9.1.5, 599 Proposition 9.1.6, 599 Fact 9.7.18, 624 Fact 9.7.21, 625 Fact 9.7.29, 626 Kronecker product Fact 9.9.61, 648 matrix

Hurwitz polynomial definition, 601 Minkowski’s inequality Lemma 9.1.3, 598 monotonicity Proposition 9.1.5, 599 power-sum inequality Fact 1.17.35, 62 Schatten norm Proposition 9.2.5, 603 Fact 9.11.6, 655 submultiplicative norm Fact 9.9.20, 640 vector Fact 9.7.34, 626 vector norm Proposition 9.1.4, 598 ¨ Holder’s inequality Diaz-GoldmanMetcalf inequality Fact 1.18.22, 71 positive-semidefinite matrix Fact 8.12.12, 525 positive-semidefinite matrix trace Fact 8.12.11, 525 reversal Fact 1.18.22, 71 scalar case Fact 1.18.11, 68 Fact 1.18.12, 68 vector inequality Proposition 9.1.6, 599 ¨ Holder-induced lower bound definition, 614 ¨ Holder-induced norm absolute value Fact 9.8.26, 631 adjoint norm Fact 9.8.10, 628 column norm Fact 9.8.21, 630 Fact 9.8.23, 631

Fact 9.8.25, 631 complex conjugate Fact 9.8.27, 631 complex conjugate transpose Fact 9.8.28, 631 definition, 608 field Proposition 9.4.7, 609 formulas Proposition 9.4.9, 609 Hardy Fact 9.8.17, 629 H¨ older norm Proposition 9.4.11, 611 Fact 9.7.28, 626 Fact 9.8.12, 628 Fact 9.8.17, 629 Fact 9.8.18, 630 Fact 9.8.19, 630 Fact 9.8.29, 631 inequality Fact 9.8.21, 630 Fact 9.8.22, 631 Littlewood Fact 9.8.17, 629 Fact 9.8.18, 630 maximum singular value Fact 9.8.21, 630 monotonicity Proposition 9.4.6, 608 Orlicz Fact 9.8.18, 630 partitioned matrix Fact 9.8.11, 628 quadratic form Fact 9.8.35, 632 Fact 9.8.36, 632 row norm Fact 9.8.21, 630 Fact 9.8.23, 631 Fact 9.8.25, 631 ¨ Holder-McCarthy inequality quadratic form Fact 8.15.15, 552 Hopf’s theorem

1035

eigenvalues of a positive matrix Fact 4.11.21, 306 Horn diagonal entries of a unitary matrix Fact 8.18.10, 562 Householder matrix, see elementary reflector definition, 252 Householder reflector, see elementary reflector definition, 252 Hsu orthogonally similar matrices Fact 5.9.17, 341 Hua’s inequalities determinant Fact 8.13.26, 538 determinant inequality Fact 8.11.21, 519 positive-semidefinite matrix Fact 8.11.21, 519 Hua’s inequality scalar inequality Fact 1.17.13, 56 Hua’s matrix equality positive-semidefinite matrix Fact 8.11.21, 519 Hurwitz matrix, see asymptotically stable matrix Hurwitz polynomial, see asymptotically stable polynomial asymptotically stable polynomial Fact 11.18.23, 770

1036

Huygens

Huygens polynomial bound Fact 11.20.14, 781

icosahedral group group Fact 3.23.4, 243

Huygens’s inequality trigonometric inequality Fact 1.11.29, 30

idempotent matrix commutator Fact 3.12.16, 217 Fact 3.12.17, 217 Fact 3.12.30, 221 Fact 3.12.31, 221 Fact 3.12.32, 222 Fact 3.15.6, 217 commuting matrices Fact 3.16.5, 231 complementary idempotent matrix Fact 3.12.12, 216 complementary subspaces Proposition 3.5.3, 190 Proposition 3.5.4, 190 Fact 3.12.1, 215 Fact 3.12.33, 223 complex conjugate Fact 3.12.7, 216 complex conjugate transpose Fact 3.12.7, 216 Fact 5.9.23, 342 definition Definition 3.1.1, 179 difference Fact 3.12.26, 220 Fact 3.12.30, 221 Fact 5.12.19, 367 difference of matrices Fact 3.12.27, 220 Fact 3.12.28, 220 Fact 3.12.32, 222 discrete-time semistable matrix Fact 11.21.12, 784 Drazin generalized inverse Proposition 6.2.2, 402 Fact 6.6.6, 432 equalities Fact 3.12.18, 217 factorization

hyperbolic equalities Fact 1.21.3, 83 hyperbolic inequality scalar Fact 1.11.29, 30 Fact 1.12.29, 38 hypercompanion form existence Theorem 5.3.2, 314 Theorem 5.3.3, 315 hypercompanion matrix companion matrix Corollary 5.3.4, 315 Lemma 5.3.1, 314 definition, 314 elementary divisor Lemma 5.3.1, 314 example Example 5.3.6, 316 Example 5.3.7, 317 real Jordan form Fact 5.10.1, 345 similarity transformation Fact 5.10.1, 345 hyperellipsoid volume Fact 3.7.35, 199 hyperplane definition, 99 elementary projector Fact 3.13.8, 225 elementary reflector Fact 3.14.5, 229

I

Fact 5.15.28, 383 Fact 5.15.30, 383 full-rank factorization Fact 3.12.23, 219 generalized inverse Fact 5.12.18, 367 Fact 6.3.21, 408 Fact 6.3.22, 408 Fact 6.3.23, 408 Fact 6.3.24, 409 Fact 6.3.25, 409 Fact 6.3.26, 409 Fact 6.4.21, 416 Fact 6.4.22, 416 Fact 6.4.23, 416 Fact 6.4.25, 417 Fact 6.4.28, 417 group generalized inverse Proposition 6.2.3, 403 group-invertible matrix Proposition 3.1.6, 183 Proposition 3.5.9, 191 Proposition 6.2.3, 403 Fact 5.11.8, 351 identity-matrix perturbation Fact 3.12.13, 216 inertia Fact 5.8.1, 334 involutory matrix Fact 3.15.2, 230 Kronecker product Fact 7.4.17, 446 left inverse Fact 3.12.10, 216 linear combination Fact 3.12.25, 219 Fact 3.12.28, 220 Fact 5.19.9, 395 matrix exponential Fact 11.11.1, 736 Fact 11.16.12, 762 matrix product Fact 3.12.21, 218 Fact 3.12.23, 219 matrix sum

identity-matrix perturbation Fact 3.12.25, 219 Fact 5.19.7, 395 Fact 5.19.8, 395 Fact 5.19.9, 395 maximum singular value Fact 5.11.38, 358 Fact 5.11.39, 358 Fact 5.12.18, 367 nilpotent matrix Fact 3.12.16, 217 nonsingular matrix Fact 3.12.11, 216 Fact 3.12.25, 219 Fact 3.12.28, 220 Fact 3.12.32, 222 norm Fact 11.16.12, 762 normal matrix Fact 3.13.3, 224 null space Fact 3.12.3, 215 Fact 3.15.6, 217 Fact 6.3.23, 408 onto a subspace along another subspace definition, 190 outer-product matrix Fact 3.7.18, 196 Fact 3.12.6, 216 partitioned matrix Fact 3.12.14, 217 Fact 3.12.20, 218 Fact 3.12.33, 223 Fact 5.10.22, 349 positive-definite matrix Fact 5.15.30, 383 positive-semidefinite matrix Fact 5.15.30, 383 power Fact 3.12.3, 215 product Fact 3.12.29, 221 projector Fact 3.13.3, 224

Fact 3.13.13, 225 Fact 3.13.20, 227 Fact 3.13.24, 228 Fact 5.10.13, 347 Fact 5.12.18, 367 Fact 6.3.25, 409 Fact 6.4.21, 416 Fact 6.4.22, 416 Fact 6.4.23, 416 Fact 6.4.28, 417 quadratic form Fact 3.13.11, 225 range Fact 3.12.3, 215 Fact 3.12.4, 216 Fact 3.15.6, 217 Fact 6.3.23, 408 range-Hermitian matrix Fact 3.13.3, 224 Fact 6.3.26, 409 rank Fact 3.12.6, 216 Fact 3.12.9, 216 Fact 3.12.19, 218 Fact 3.12.20, 218 Fact 3.12.22, 218 Fact 3.12.24, 219 Fact 3.12.26, 220 Fact 3.12.27, 220 Fact 3.12.31, 221 Fact 5.8.1, 334 Fact 5.11.7, 350 right inverse Fact 3.12.10, 216 semisimple matrix Fact 5.14.20, 373 similar matrices Proposition 3.4.5, 189 Proposition 5.5.22, 327 Corollary 5.5.21, 327 Fact 5.10.9, 346 Fact 5.10.13, 347 Fact 5.10.14, 347 Fact 5.10.22, 349 singular value Fact 5.11.38, 358 skew-Hermitian matrix

1037

Fact 3.12.8, 216 skew-idempotent matrix Fact 3.12.5, 216 spectrum Proposition 5.5.20, 326 Fact 5.11.7, 350 stable subspace Proposition 11.8.8, 729 submultiplicative norm Fact 9.8.6, 627 sum Fact 3.12.22, 218 trace Fact 5.8.1, 334 Fact 5.11.7, 350 transpose Fact 3.12.7, 216 tripotent matrix Fact 3.16.1, 231 Fact 3.16.5, 231 unitarily similar matrices Proposition 3.4.5, 189 Fact 5.9.23, 342 Fact 5.9.28, 343 Fact 5.9.29, 343 Fact 5.10.10, 347 unstable subspace Proposition 11.8.8, 729 idempotent matrix onto a subspace along another subspace definition, 190 identity function definition, 5 identity matrix definition, 91 symplectic matrix Fact 3.20.3, 238 identity theorem matrix function evaluation Theorem 10.5.3, 690 identity-matrix perturbation

1038

identity-matrix perturbation

cyclic matrix Fact 5.14.15, 372 defective matrix Fact 5.14.15, 372 derogatory matrix Fact 5.14.15, 372 diagonalizable over F Fact 5.14.15, 372 inverse matrix Fact 4.8.12, 281 semisimple matrix Fact 5.14.15, 372 simple matrix Fact 5.14.15, 372 spectrum Fact 4.10.14, 292 Fact 4.10.15, 292

definition Definition 4.7.2, 271 impulse function definition, 796 impulse response definition, 797 impulse response function definition, 797 inbound Laplacian matrix adjacency matrix Theorem 3.2.2, 185 definition Definition 3.2.1, 184

identity-matrix shift controllable subspace Lemma 12.6.7, 811 unobservable subspace Lemma 12.3.7, 802

incidence matrix definition Definition 3.2.1, 184 Laplacian matrix Theorem 3.2.2, 185 Theorem 3.2.3, 185

identric mean arithmetic mean Fact 1.12.37, 40 logarithmic mean Fact 1.12.37, 40

inclusion principle Hermitian matrix eigenvalue Theorem 8.4.5, 468

image definition, 4 imaginary part frequency response Fact 12.22.5, 872 transfer function Fact 12.22.5, 872 imaginary vector definition, 93 implication definition, 1 improper rational function definition Definition 4.7.1, 271 improper rational transfer function

inclusion-exclusion principle cardinality Fact 1.7.5, 12 increasing function convex function Theorem 8.6.15, 479 definition Definition 8.6.12, 478 log majorization Fact 2.21.11, 177 logarithm Proposition 8.6.13, 478 matrix functions Proposition 8.6.13, 478 positive-definite matrix Fact 8.10.55, 513 Schur complement Proposition 8.6.13, 478

weak majorization Fact 2.21.9, 177 increasing sequence positive-semidefinite matrix Proposition 8.6.3, 475 indecomposable matrix, see irreducible matrix definition, 252 indegree graph Definition 1.6.3, 10 indegree matrix definition Definition 3.2.1, 184 index of a matrix block-triangular matrix Fact 5.14.31, 374 Fact 6.6.14, 435 complementary subspaces Proposition 3.5.7, 191 definition Definition 3.5.5, 190 group-invertible matrix Proposition 3.5.6, 190 Corollary 5.5.9, 323 Fact 5.14.2, 369 Kronecker product Fact 7.4.27, 447 outer-product matrix Fact 5.14.3, 370 partitioned matrix Fact 5.14.31, 374 Fact 6.6.14, 435 range Fact 5.14.2, 369 rank Proposition 5.5.2, 321 index of an eigenvalue algebraic multiplicity Proposition 5.5.6, 322

inertia definition Definition 5.5.1, 321 Jordan block Proposition 5.5.3, 322 minimal polynomial Proposition 5.5.14, 325 rank Proposition 5.5.2, 321 semisimple eigenvalue Proposition 5.5.8, 323 induced lower bound definition Definition 9.5.1, 613 Proposition 9.5.2, 613 generalized inverse Fact 9.8.44, 635 lower bound Fact 9.8.43, 635 Fact 9.8.44, 635 maximum singular value Corollary 9.5.5, 614 minimum singular value Corollary 9.5.5, 614 properties Proposition 9.5.2, 613 Proposition 9.5.3, 613 singular value Proposition 9.5.4, 614 supermultiplicativity Proposition 9.5.6, 615 induced norm compatible norm Proposition 9.4.3, 608 definition Definition 9.4.1, 607 determinant Fact 9.12.11, 659 dual norm Fact 9.7.22, 625 field Example 9.4.8, 609 maximum singular value Fact 9.8.24, 631

norm Theorem 9.4.2, 607 quadratic form Fact 9.8.34, 632 spectral radius Corollary 9.4.5, 608 Corollary 9.4.10, 611 induced norms symmetry property Fact 9.8.16, 629 inequality elementary symmetric function Fact 1.17.20, 58 sum of products Fact 1.17.20, 58 inertia congruent matrices Corollary 5.4.7, 321 Fact 5.8.22, 338 definition, 267 dissipative matrix Fact 5.8.12, 336 generalized inverse Fact 6.3.20, 408 Fact 8.21.12, 579 group-invertible matrix Fact 5.8.5, 334 Hermitian matrix Theorem 8.4.11, 470 Proposition 5.4.6, 320 Fact 5.8.6, 334 Fact 5.8.8, 335 Fact 5.8.12, 336 Fact 5.8.13, 336 Fact 5.8.14, 336 Fact 5.8.15, 336 Fact 5.8.16, 336 Fact 5.8.17, 336 Fact 5.8.18, 337 Fact 5.8.19, 337 Fact 5.12.1, 362 Fact 6.3.20, 408 Fact 8.10.15, 503 Fact 8.21.12, 579 Fact 8.21.14, 580 Fact 12.21.1, 866

1039

Fact 12.21.2, 867 Fact 12.21.3, 867 Fact 12.21.4, 867 Fact 12.21.5, 867 Fact 12.21.6, 868 Fact 12.21.7, 868 Fact 12.21.8, 868 Fact 12.21.10, 869 Fact 12.21.11, 869 Fact 12.21.12, 869 idempotent matrix Fact 5.8.1, 334 inequalities Fact 5.8.16, 336 involutory matrix Fact 5.8.2, 334 Lyapunov equation Fact 12.21.1, 866 Fact 12.21.2, 867 Fact 12.21.3, 867 Fact 12.21.4, 867 Fact 12.21.5, 867 Fact 12.21.6, 868 Fact 12.21.7, 868 Fact 12.21.8, 868 Fact 12.21.9, 869 Fact 12.21.10, 869 Fact 12.21.11, 869 Fact 12.21.12, 869 nilpotent matrix Fact 5.8.4, 334 normal matrix Fact 5.10.17, 348 partitioned matrix Fact 5.8.19, 337 Fact 5.8.20, 337 Fact 5.8.21, 337 Fact 5.12.1, 362 Fact 6.5.5, 423 positive-definite matrix Fact 5.8.10, 335 positive-semidefinite matrix Fact 5.8.9, 335 Fact 5.8.10, 335 Fact 12.21.9, 869 rank Fact 5.8.5, 334

1040

inertia

Fact 5.8.18, 337 Riccati equation Lemma 12.16.18, 854 Schur complement Fact 6.5.5, 423 skew-Hermitian matrix Fact 5.8.4, 334 skew-involutory matrix Fact 5.8.4, 334 submatrix Fact 5.8.8, 335 tripotent matrix Fact 5.8.3, 334 inertia matrix positive-definite matrix Fact 8.9.5, 495 rigid body Fact 8.9.5, 495 infinite interval definition, xxxv infinite matrix product convergence Fact 11.21.19, 785

Mercator’s series Fact 1.20.8, 79 spectral radius Fact 10.13.1, 704 infinity norm definition, 599 Kronecker product Fact 9.9.61, 648 submultiplicative norm Fact 9.9.1, 636 Fact 9.9.2, 636 injective function definition, 84 inner product convex cone Fact 10.9.14, 697 inequality Fact 2.12.1, 136 open ball Fact 9.7.24, 625 separation theorem Fact 10.9.14, 697 Fact 10.9.15, 697 subspace Fact 10.9.13, 696

infinite product commutator Fact 11.14.18, 751 convergence Fact 11.21.19, 785 equality Fact 1.9.9, 22 Fact 1.9.10, 23 matrix exponential Fact 11.14.18, 751

inner product of complex matrices definition, 96

infinite series binomial series Fact 1.20.8, 79 complex numbers Fact 1.20.8, 79 equality Fact 1.9.6, 21 Fact 1.9.7, 21 Fact 1.9.8, 22 Gregory’s series Fact 1.20.8, 79

inner product of real vectors definition, 93

inner product of complex vectors definition, 93 inner product of real matrices definition, 95

inner-product minimization positive-definite matrix Fact 8.15.13, 552 input matrix controllability

Fact 12.20.15, 865 stabilizability Fact 12.20.15, 865 input-to-state stability asymptotic stability Fact 12.20.18, 866 integer equality Fact 1.12.1, 33 number of partitions Fact 5.16.8, 390 integers equality Fact 1.12.2, 33 integral asymptotically stable matrix Lemma 11.9.2, 731 averaged limit Fact 10.11.6, 700 determinant Fact 11.13.15, 745 Drazin generalized inverse Fact 11.13.12, 745 Fact 11.13.14, 745 Gaussian density Fact 8.16.1, 556 Fact 8.16.2, 556 Fact 8.16.3, 556 Fact 8.16.4, 557 Fact 8.16.5, 557 Fact 11.13.16, 746 generalized inverse Fact 11.13.10, 745 group generalized inverse Fact 11.13.13, 745 Fact 11.13.14, 745 inverse matrix Fact 11.13.11, 745 matrix definition, 688 matrix exponential Proposition 11.1.4, 709 Lemma 11.9.2, 731 Fact 11.13.10, 745

invariant zero Fact 11.13.11, 745 Fact 11.13.12, 745 Fact 11.13.13, 745 Fact 11.13.14, 745 Fact 11.13.15, 745 Fact 11.13.16, 746 Fact 11.14.2, 748 Fact 11.18.5, 766 Fact 11.18.6, 766 positive-definite matrix Fact 8.16.1, 556 Fact 8.16.2, 556 Fact 8.16.3, 556 Fact 8.16.4, 557 Fact 8.16.5, 557 positive-semidefinite matrix Proposition 8.6.10, 476 quadratic form Fact 8.16.3, 556 Fact 8.16.4, 557 Fact 8.16.5, 557 integral of a Gaussian density Bhat Fact 11.13.16, 746

Fact 10.9.1, 695 union Fact 10.9.2, 695 Fact 10.9.3, 695 interior point definition Definition 10.1.1, 681 interior point relative to a set definition Definition 10.1.2, 681 interior relative to a set definition Definition 10.1.2, 681 interlacing singular value Fact 9.14.10, 667 interlacing theorem asymptotically stable polynomial Fact 11.17.6, 764 intermediate node definition Definition 1.6.3, 10

integral representation Kronecker sum Fact 11.18.34, 773

interpolation polynomial Fact 4.8.11, 281

interior boundary Fact 10.8.7, 693 complement Fact 10.8.6, 693 convex set Fact 10.8.8, 693 Fact 10.8.19, 694 Fact 10.9.4, 695 definition Definition 10.1.1, 681 intersection Fact 10.9.2, 695 largest open set Fact 10.8.3, 693 simplex Fact 2.20.4, 167 subset

intersection closed set Fact 10.9.11, 696 Fact 10.9.12, 696 convex set Fact 10.9.8, 696 definition, 2 dual cone Fact 2.9.5, 120 equivalence relation Proposition 1.5.3, 7 interior Fact 10.9.2, 695 open set Fact 10.9.10, 696 reflexive relation Proposition 1.5.3, 7 span

1041

Fact 2.9.12, 121 symmetric relation Proposition 1.5.3, 7 transitive relation Proposition 1.5.3, 7 intersection of closed sets Cantor intersection theorem Fact 10.9.12, 696 intersection of ranges projector Fact 6.4.46, 421 intersection of subspaces subspace dimension theorem Theorem 2.3.1, 98 interval definition, xxxv invariance of domain open set image Theorem 10.3.7, 684 invariant subspace controllable subspace Corollary 12.6.4, 810 definition, 102 lower triangular matrix Fact 5.9.4, 339 matrix representation Fact 2.9.25, 122 stable subspace Proposition 11.8.8, 729 unobservable subspace Corollary 12.3.4, 801 unstable subspace Proposition 11.8.8, 729 upper triangular matrix Fact 5.9.4, 339 invariant zero definition Definition 12.10.1, 830

1042

invariant zero

determinant Fact 12.22.14, 874 equivalent realizations Proposition 12.10.10, 836 full actuation Definition 12.10.2, 831 full observation Definition 12.10.2, 831 full-state feedback Proposition 12.10.10, 836 Fact 12.22.14, 874 observable pair Corollary 12.10.12, 837 pencil Corollary 12.10.4, 832 Corollary 12.10.5, 832 Corollary 12.10.6, 833 regular pencil Corollary 12.10.4, 832 Corollary 12.10.5, 832 Corollary 12.10.6, 833 transmission zero Theorem 12.10.8, 834 Theorem 12.10.9, 835 uncontrollable spectrum Theorem 12.10.9, 835 uncontrollableunobservable spectrum Theorem 12.10.9, 835 unobservable eigenvalue Proposition 12.10.11, 837 unobservable spectrum Theorem 12.10.9, 835 inverse determinant Fact 2.13.6, 140 left-invertible matrix Proposition 2.6.5, 110 polynomial matrix definition, 257

positive-definite matrix Fact 8.11.10, 516 rank Fact 2.11.21, 136 Fact 2.11.22, 136 right-invertible matrix Proposition 2.6.5, 110 subdeterminant Fact 2.13.6, 140 inverse function definition, 5 uniqueness Theorem 1.4.2, 5 inverse function theorem determinant Theorem 10.4.5, 688 existence of local inverse Theorem 10.4.5, 688 inverse image definition, 5 subspace intersection Fact 2.9.30, 124 subspace sum Fact 2.9.30, 124 inverse matrix 2×2 Fact 2.16.12, 155 2 × 2 block triangular Lemma 2.8.2, 116 3×3 Fact 2.16.12, 155 asymptotically stable matrix Fact 11.18.15, 768 block-circulant matrix Fact 2.17.6, 160 block-triangular matrix Fact 2.17.1, 159 characteristic polynomial Fact 4.9.9, 284

companion matrix Fact 5.16.2, 386 convergent sequence Fact 2.16.29, 158 convergent series Fact 4.10.7, 290 definition, 109 derivative Proposition 10.7.2, 691 Fact 10.12.8, 703 elementary matrix Fact 2.16.1, 153 Fact 3.7.20, 196 equality Fact 2.16.13, 156 Fact 2.16.14, 156 Fact 2.16.15, 156 Fact 2.16.16, 156 Fact 2.16.17, 156 Fact 2.16.18, 156 Fact 2.16.19, 156 Fact 2.16.20, 157 Fact 2.16.21, 157 Fact 2.16.22, 157 Fact 2.16.23, 157 Fact 2.16.24, 157 Fact 2.16.25, 157 Fact 2.16.26, 158 Fact 2.16.27, 158 finite sequence Fact 2.16.28, 158 Hamiltonian matrix Fact 3.20.5, 239 Hankel matrix Fact 3.18.4, 234 identity-matrix perturbation Fact 4.8.12, 281 integral Fact 11.13.11, 745 Kronecker product Proposition 7.1.7, 441 lower bound Fact 8.9.17, 497 matrix exponential Proposition 11.2.8, 713 Fact 11.13.11, 745 matrix inversion lemma

involutory matrix Corollary 2.8.8, 117 matrix sum Corollary 2.8.10, 119 maximum singular value Fact 9.14.8, 666 Newton-Raphson algorithm Fact 2.16.29, 158 normalized submultiplicative norm Fact 9.8.45, 635 Fact 9.9.56, 647 Fact 9.9.57, 648 Fact 9.9.58, 648 Fact 9.9.59, 648 outer-product perturbation Fact 2.16.3, 153 partitioned matrix Fact 2.16.4, 154 Fact 2.17.2, 159 Fact 2.17.3, 159 Fact 2.17.4, 159 Fact 2.17.5, 160 Fact 2.17.6, 160 Fact 2.17.8, 161 Fact 5.12.21, 368 perturbation Fact 9.9.60, 648 polynomial representation Fact 4.8.13, 281 positive-definite matrix Proposition 8.6.6, 476 Lemma 8.6.5, 475 Fact 8.9.17, 497 Fact 8.9.42, 500 positive-semidefinite matrix Fact 8.10.37, 507 product Proposition 2.6.9, 111 rank Fact 2.17.10, 161 Fact 6.5.11, 425 series

Proposition 9.4.13, 612 similar matrices Fact 5.15.31, 383 similarity transformation Fact 5.15.4, 377 spectral radius Proposition 9.4.13, 612 spectrum Fact 5.11.14, 353 sum Fact 2.17.6, 160 Toeplitz matrix Fact 3.18.10, 236 Fact 3.18.11, 236 tridiagonal matrix Fact 3.18.10, 236 Fact 3.19.4, 238 Fact 3.19.5, 238 upper block-triangular matrix Fact 2.17.7, 161 Fact 2.17.9, 161 inverse operation composition Fact 1.7.10, 13 iterated Fact 1.7.9, 13 inverse trigonometric equalities Fact 1.21.2, 82 invertible function definition, 5 involutory matrix commutator Fact 3.15.6, 231 cyclic permutation matrix Fact 3.15.5, 230 definition Definition 3.1.1, 179 determinant Fact 3.15.1, 230 Fact 5.15.32, 383 diagonalizable matrix

1043

Fact 5.14.19, 373 equality Fact 3.15.3, 230 factorization Fact 5.15.18, 380 Fact 5.15.31, 383 Fact 5.15.32, 383 Hermitian matrix Fact 3.14.2, 229 idempotent matrix Fact 3.15.2, 230 inertia Fact 5.8.2, 334 Kronecker product Fact 7.4.17, 446 matrix exponential Fact 11.11.1, 736 normal matrix Fact 5.9.11, 340 Fact 5.9.12, 340 null space Fact 3.15.6, 231 partitioned matrix Fact 3.15.7, 231 range Fact 3.15.6, 231 reflector Fact 3.14.2, 229 reverse permutation matrix Fact 3.15.4, 230 semisimple matrix Fact 5.14.18, 372 signature Fact 5.8.2, 334 similar matrices Proposition 3.4.5, 189 Corollary 5.5.21, 327 Fact 5.15.31, 383 spectrum Proposition 5.5.20, 326 symmetric matrix Fact 5.15.36, 384 trace Fact 5.8.2, 334 transpose Fact 5.9.9, 340 tripotent matrix Fact 3.16.2, 231

1044

involutory matrix

unitarily similar matrices Proposition 3.4.5, 189 unitary matrix Fact 3.14.2, 229 irreducible matrix absolute value Fact 3.22.6, 241 almost nonnegative matrix Fact 11.19.2, 774 connected graph Fact 4.11.3, 297 cyclic permutation matrix Fact 3.22.5, 241 definition Definition 3.1.1, 179 graph Fact 4.11.3, 297 group generalized inverse Fact 6.6.21, 438 M-matrix Fact 4.11.12, 302 permutation matrix Fact 3.22.5, 241 perturbation matrix Fact 4.11.5, 300 positive matrix Fact 4.11.4, 298 row-stochastic matrix Fact 11.21.11, 784 spectral radius convexity Fact 4.11.19, 306 spectral radius monotonicity Fact 4.11.19, 306 upper block-triangular matrix Fact 4.11.5, 300 irreducible polynomial definition, 255 isomorphic groups

symplectic group and unitary group Fact 3.23.10, 247 isomorphism definition Definition 3.3.4, 186 group Proposition 3.3.5, 186

J Jacobi identity commutator Fact 2.18.4, 162 Jacobi’s identity determinant Fact 2.14.28, 151 matrix differential equation Fact 11.13.4, 743 Jacobian definition, 687 Jacobson nilpotent commutator Fact 3.17.12, 233 Jensen convex function Fact 10.11.7, 700 Jensen’s inequality arithmetic-mean– geometric-mean inequality Fact 1.10.4, 24 convex function Fact 1.10.4, 24 Fact 1.17.36, 62

Jordan block index of an eigenvalue Proposition 5.5.3, 322 Jordan canonical form generalized inverse Fact 6.6.10, 434 group-invertible matrix Fact 6.6.10, 434 Jordan form existence Theorem 5.3.3, 315 factorization Fact 5.15.6, 378 Hamiltonian Fact 12.23.1, 875 minimal polynomial Proposition 5.5.14, 325 normal matrix Fact 5.10.6, 346 real Jordan form Fact 5.10.2, 345 Schur decomposition Fact 5.10.6, 346 square root Fact 5.15.19, 380 transfer function Fact 12.22.10, 873 Jordan matrix example Example 5.3.6, 316 Example 5.3.7, 317 Jordan structure logarithm Corollary 11.4.4, 719 matrix exponential Corollary 11.4.4, 719

JLL inequality trace of a matrix power Fact 4.11.23, 306

Jordan’s inequality trigonometric inequality Fact 1.11.29, 30

jointly continuous function continuous function Fact 10.11.4, 700

Jordan-Chevalley decomposition diagonalizable matrix

Kronecker product Fact 5.9.5, 339 nilpotent matrix Fact 5.9.5, 339 Joyal polynomial root bound Fact 11.20.7, 779 Jury criterion discrete-time asymptotically stable polynomial Fact 11.20.1, 777

K Kalman decomposition minimal realization Proposition 12.9.10, 825 Kantorovich inequality positive-semidefinite matrix Fact 8.15.10, 551 quadratic form Fact 8.15.10, 551 scalar case Fact 1.17.37, 63 Kato maximum singular value of a matrix difference Fact 9.9.32, 642 kernel function positive-semidefinite matrix Fact 8.8.1, 488 Fact 8.8.4, 490 Kharitonov’s theorem asymptotically stable polynomial Fact 11.17.13, 766 Khatri-Rao product Kronecker product, 458 Kittaneh

Schatten norm inequality Fact 9.9.45, 645 Klamkin’s inequality triangle Fact 2.20.11, 169 Klein four-group dihedral group Fact 3.23.4, 243 Klein’s inequality trace of a matrix logarithm Fact 11.14.25, 752 Kleinman stabilization and Gramian Fact 12.20.17, 865 Kojima’s bound polynomial Fact 11.20.8, 779 Kosaki Schatten norm inequality Fact 9.9.45, 645 trace norm of a matrix difference Fact 9.9.24, 640 trace of a convex function Fact 8.12.34, 531 unitarily invariant norm inequality Fact 9.9.44, 645 Krein inertia of a Hermitian matrix Fact 12.21.5, 867 Krein-Milman theorem extreme points of a convex set Fact 10.8.23, 695 Kreiss matrix theorem maximum singular value Fact 11.21.20, 786

1045

Kristof least squares and unitary biequivalence Fact 9.15.10, 679 Kronecker canonical form pencil Theorem 5.7.1, 331 regular pencil Proposition 5.7.2, 331 Kronecker permutation matrix definition, 442 Kronecker product Fact 7.4.30, 447 orthogonal matrix Fact 7.4.30, 447 projector Fact 7.4.31, 449 trace Fact 7.4.30, 447 transpose Proposition 7.1.13, 442 vec Fact 7.4.30, 447 Kronecker product biequivalent matrices Fact 7.4.12, 445 column norm Fact 9.9.61, 648 complex conjugate transpose Proposition 7.1.3, 440 congruent matrices Fact 7.4.13, 446 convex function Proposition 8.6.17, 480 definition Definition 7.1.2, 440 determinant Proposition 7.1.11, 442 Fact 7.5.12, 451 Fact 7.5.13, 452 diagonal matrix Fact 7.4.4, 445

1046

Kronecker product

discrete-time asymptotically stable matrix Fact 11.21.6, 782 Fact 11.21.7, 783 discrete-time Lyapunov-stable matrix Fact 11.21.6, 782 Fact 11.21.7, 783 discrete-time semistable matrix Fact 11.21.5, 782 Fact 11.21.6, 782 Fact 11.21.7, 783 Drazin generalized inverse Fact 7.4.33, 449 eigenvalue Proposition 7.1.10, 442 Fact 7.4.14, 446 Fact 7.4.16, 446 Fact 7.4.22, 446 Fact 7.4.28, 447 Fact 7.4.34, 449 eigenvector Proposition 7.1.10, 442 Fact 7.4.22, 446 Fact 7.4.34, 449 Euclidean norm Fact 9.7.27, 626 Frobenius norm Fact 9.14.38, 676 generalized inverse Fact 7.4.32, 449 group generalized inverse Fact 7.4.33, 449 group-invertible matrix Fact 7.4.17, 446 Fact 7.4.33, 449 Hermitian matrix Fact 7.4.17, 446 Fact 8.22.29, 590 H¨ older norm Fact 9.9.61, 648 idempotent matrix Fact 7.4.17, 446

index of a matrix Fact 7.4.27, 447 infinity norm Fact 9.9.61, 648 inverse matrix Proposition 7.1.7, 441 involutory matrix Fact 7.4.17, 446 Kronecker permutation matrix Fact 7.4.30, 447 Kronecker sum Fact 11.14.37, 755 left-equivalent matrices Fact 7.4.12, 445 lower triangular matrix Fact 7.4.4, 445 matrix exponential Proposition 11.1.7, 709 Fact 11.14.37, 755 Fact 11.14.38, 755 matrix multiplication Proposition 7.1.6, 440 matrix power Fact 7.4.5, 445 Fact 7.4.11, 445 Fact 7.4.22, 446 matrix sum Proposition 7.1.4, 440 maximum singular value Fact 9.14.38, 676 nilpotent matrix Fact 7.4.17, 446 normal matrix Fact 7.4.17, 446 orthogonal matrix Fact 7.4.17, 446 outer-product matrix Proposition 7.1.8, 441 partitioned matrix Fact 7.4.19, 446 Fact 7.4.20, 446 Fact 7.4.25, 447

positive-definite matrix Fact 7.4.17, 446 positive-semidefinite matrix Fact 7.4.17, 446 Fact 8.22.16, 587 Fact 8.22.23, 589 Fact 8.22.24, 589 Fact 8.22.25, 589 Fact 8.22.27, 589 Fact 8.22.28, 590 Fact 8.22.30, 590 Fact 8.22.32, 590 projector Fact 7.4.17, 446 range Fact 7.4.23, 447 range-Hermitian matrix Fact 7.4.17, 446 rank Fact 7.4.24, 447 Fact 7.4.25, 447 Fact 7.4.26, 447 Fact 8.22.16, 587 reflector Fact 7.4.17, 446 right-equivalent matrices Fact 7.4.12, 445 row norm Fact 9.9.61, 648 Schatten norm Fact 9.14.38, 676 Schur product Proposition 7.3.1, 444 semisimple matrix Fact 7.4.17, 446 similar matrices Fact 7.4.13, 446 singular matrix Fact 7.4.29, 447 skew-Hermitian matrix Fact 7.4.18, 446 spectral radius Fact 7.4.15, 446 square root

Lagrange identity Fact 8.22.30, 590 Fact 8.22.31, 590 submatrix Proposition 7.3.1, 444 trace Proposition 7.1.12, 442 Fact 11.14.38, 755 transpose Proposition 7.1.3, 440 triple product Proposition 7.1.5, 440 Fact 7.4.8, 445 tripotent matrix Fact 7.4.17, 446 unitarily similar matrices Fact 7.4.13, 446 unitary matrix Fact 7.4.17, 446 upper triangular matrix Fact 7.4.4, 445 vec Fact 7.4.6, 445 Fact 7.4.7, 445 Fact 7.4.9, 445 vector Fact 7.4.1, 445 Fact 7.4.2, 445 Fact 7.4.3, 445 Fact 7.4.21, 446 Kronecker sum associativity Proposition 7.2.2, 443 asymptotically stable matrix Fact 11.18.32, 773 Fact 11.18.33, 773 Fact 11.18.34, 773 asymptotically stable polynomial Fact 11.17.11, 765 commuting matrices Fact 7.5.4, 450 defect Fact 7.5.2, 450 definition Definition 7.2.1, 443 determinant

Fact 7.5.11, 451 dissipative matrix Fact 7.5.8, 451 eigenvalue Proposition 7.2.3, 443 Fact 7.5.5, 450 Fact 7.5.7, 450 Fact 7.5.16, 452 eigenvector Proposition 7.2.3, 443 Fact 7.5.16, 452 Hermitian matrix Fact 7.5.8, 451 integral representation Fact 11.18.34, 773 Kronecker product Fact 11.14.37, 755 linear matrix equation Proposition 11.9.3, 731 linear system Fact 7.5.15, 452 Lyapunov equation Corollary 11.9.4, 732 Lyapunov-stable matrix Fact 11.18.32, 773 Fact 11.18.33, 773 matrix exponential Proposition 11.1.7, 709 Fact 11.14.36, 755 Fact 11.14.37, 755 matrix power Fact 7.5.1, 450 nilpotent matrix Fact 7.5.3, 450 Fact 7.5.8, 451 normal matrix Fact 7.5.8, 451 positive matrix Fact 7.5.8, 451 positive-semidefinite matrix Fact 7.5.8, 451 range-Hermitian matrix Fact 7.5.8, 451 rank

1047

Fact 7.5.2, 450 Fact 7.5.9, 451 Fact 7.5.10, 451 semidissipative matrix Fact 7.5.8, 451 semistable matrix Fact 11.18.32, 773 Fact 11.18.33, 773 similar matrices Fact 7.5.9, 451 skew-Hermitian matrix Fact 7.5.8, 451 spectral abscissa Fact 7.5.6, 450 trace Fact 11.14.36, 755

L L2 norm controllability Gramian Theorem 12.11.1, 838 definition, 838 observability Gramian Theorem 12.11.1, 838 ¨ Lowner-Heinz inequality positive-semidefinite matrix inequality Corollary 8.6.11, 477 Labelle polynomial root bound Fact 11.20.7, 779 Laffey simultaneous triangularization Fact 5.17.5, 392 Lagrange subgroup Fact 3.23.6, 246 Lagrange identity

1048

Lagrange identity

product equality Fact 1.18.8, 67 Lagrange interpolation formula polynomial interpolation Fact 4.8.11, 281 Lagrange-Hermite interpolation polynomial matrix function Theorem 10.5.2, 689

lattice definition Definition 1.5.9, 8 positive-semidefinite matrix Fact 8.10.30, 505 Fact 8.10.31, 505 leading principal submatrix definition, 88 leaf Definition 1.6.1, 9

Laguerre-Samuelson inequality mean Fact 1.17.12, 55 Fact 8.9.36, 499

least common multiple block-diagonal matrix Lemma 5.2.7, 312 definition, 256

Lancaster’s formulas quadratic form integral Fact 8.16.3, 556

least squares fixed-rank approximation Fact 9.14.28, 672 Fact 9.15.8, 678 generalized inverse Fact 9.15.4, 677 Fact 9.15.5, 678 Fact 9.15.6, 678 Fact 9.15.7, 678 singular value decomposition Fact 9.14.28, 672 Fact 9.15.8, 678 Fact 9.15.9, 679 Fact 9.15.10, 679

Laplace transform matrix exponential Proposition 11.2.2, 711 resolvent Proposition 11.2.2, 711 Laplacian symmetric graph Fact 3.21.1, 240 Laplacian matrix adjacency matrix Theorem 3.2.2, 185 Theorem 3.2.3, 185 Fact 4.11.7, 300 definition Definition 3.2.1, 184 incidence matrix Theorem 3.2.2, 185 Theorem 3.2.3, 185 quadratic form Fact 8.15.1, 550 spectrum Fact 11.19.7, 777 symmetric graph Fact 8.15.1, 550

least squares and unitary biequivalence Kristof Fact 9.15.10, 679 least upper bound projector Fact 6.4.46, 422 least upper bound for a partial ordering definition Definition 1.5.9, 8

left divides definition, 256 left equivalence equivalence relation Fact 5.10.3, 345 left inverse (1)-inverse Proposition 6.1.3, 398 affine subspace Fact 2.9.26, 123 complex conjugate transpose Fact 2.15.1, 152 Fact 2.15.2, 152 cone Fact 2.9.26, 123 convex set Fact 2.9.26, 123 definition, 5 generalized inverse Corollary 6.1.4, 398 Fact 6.4.44, 421 Fact 6.4.45, 421 idempotent matrix Fact 3.12.10, 216 left-inner matrix Fact 3.11.6, 205 matrix product Fact 2.15.5, 153 positive-definite matrix Fact 3.7.25, 197 representation Fact 2.15.3, 152 subspace Fact 2.9.26, 123 uniqueness Theorem 1.4.2, 5 left-equivalent matrices definition Definition 3.4.3, 188 group-invertible matrix Fact 3.6.1, 191 Kronecker product Fact 7.4.12, 445 null space Proposition 5.1.3, 309

limit positive-semidefinite matrix Fact 5.10.19, 348 left-inner matrix definition Definition 3.1.2, 180 generalized inverse Fact 6.3.8, 405 left inverse Fact 3.11.6, 205 left-invertible function definition, 5 left-invertible matrix definition, 106 equivalent properties Theorem 2.6.1, 107 generalized inverse Proposition 6.1.5, 398 inverse Proposition 2.6.5, 110 matrix product Fact 2.10.3, 125 nonsingular equivalence Corollary 2.6.6, 110 unique left inverse Proposition 2.6.2, 107 Lehmer matrix positive-semidefinite matrix Fact 8.8.5, 491 Lehmer mean power inequality Fact 1.12.36, 40 Leibniz’s rule derivative of an integral Fact 10.12.2, 701 lemma definition, 1 length definition Definition 1.6.3, 10 Leslie matrix definition, 396

1049

Leverrier’s algorithm characteristic polynomial Proposition 4.4.9, 266

Lie algebra of a Lie group matrix exponential Proposition 11.6.3, 723

lexicographic ordering cone Fact 2.9.31, 124 total ordering Fact 1.7.8, 13

Lie group definition Definition 11.6.1, 722 group Proposition 11.6.2, 722 Lie algebra Proposition 11.6.4, 723 Proposition 11.6.5, 723 Proposition 11.6.6, 724

Lidskii eigenvalues of a product of positivesemidefinite matrices Fact 8.19.23, 571 Lidskii-Mirsky-Wielandt theorem Hermitian perturbation Fact 9.12.4, 657 Lidskii-Wielandt inequalities eigenvalue inequality for Hermitian matrices Fact 8.19.3, 565 Lie algebra classical examples Proposition 3.3.2, 185 definition Definition 3.3.1, 185 Lie group Proposition 11.6.4, 723 Proposition 11.6.5, 723 Proposition 11.6.6, 724 matrix exponential Proposition 11.6.7, 724 strictly upper triangular matrix Fact 3.23.11, 247 Fact 11.22.1, 786 upper triangular matrix Fact 3.23.11, 247 Fact 11.22.1, 786

Lie-Trotter formula matrix exponential Fact 11.14.7, 749 Lie-Trotter product formula matrix exponential Corollary 11.4.8, 720 Fact 11.16.2, 759 Fact 11.16.3, 759 Lieb concavity theorem, 596 Lieb-Thirring inequality positive-semidefinite matrix Fact 8.12.17, 526 Fact 8.19.21, 570 limit discrete-time semistable matrix Fact 11.21.12, 784 Drazin generalized inverse Fact 6.6.12, 435 Hermitian matrix Fact 8.10.1, 501 matrix exponential Fact 11.18.5, 766 Fact 11.18.6, 766 Fact 11.18.7, 767 matrix logarithm Proposition 8.6.4, 475 positive-definite matrix

1050

limit

Fact 8.10.48, 511 positive-semidefinite matrix Proposition 8.6.3, 475 Fact 8.10.48, 511 projector Fact 6.4.46, 421 Fact 6.4.51, 422 semistable matrix Fact 11.18.7, 767

linear dependence of two vectors definition, 86 linear dependence of vectors definition, 98

linear combination determinant Fact 8.13.19, 537 Hermitian matrix Fact 8.15.25, 554 Fact 8.15.26, 554 Fact 8.15.27, 554 idempotent matrix Fact 5.19.9, 395 positive-semidefinite matrix Fact 8.13.19, 537

linear dynamical system asymptotically stable Proposition 11.8.2, 727 discrete-time asymptotically stable Proposition 11.10.2, 734 discrete-time Lyapunov stable Proposition 11.10.2, 734 discrete-time semistable Proposition 11.10.2, 734 Lyapunov stable Proposition 11.8.2, 727 semistable Proposition 11.8.2, 727

linear combination of projectors Hermitian matrix Fact 5.19.10, 395

linear function continuous function Corollary 10.3.3, 684 definition, 88

linear combination of two vectors definition, 86

linear independence cyclic matrix Fact 5.14.7, 370 definition, 98 outer-product matrix Fact 2.12.8, 137

Linden polynomial root bound Fact 11.20.9, 780

linear constraint quadratic form Fact 8.14.10, 547 linear dependence absolute value Fact 9.7.1, 618 triangle inequality Fact 9.7.3, 618 linear dependence of two matrices definition, 88

linear interpolation strong majorization Fact 3.9.6, 201 linear matrix equation asymptotically stable matrix Proposition 11.9.3, 731 existence of solutions

Fact 5.10.20, 348 Fact 5.10.21, 349 generalized inverse Fact 6.4.43, 421 Kronecker sum Proposition 11.9.3, 731 matrix exponential Proposition 11.9.3, 731 rank Fact 2.10.16, 126 skew-symmetric matrix Fact 3.7.3, 193 solution Fact 6.4.43, 421 Sylvester’s equation Proposition 7.2.4, 443 Proposition 11.9.3, 731 Fact 5.10.20, 348 Fact 5.10.21, 349 Fact 6.5.7, 424 symmetric matrix Fact 3.7.3, 193 linear system generalized inverse Proposition 6.1.7, 400 harmonic steady-state response Theorem 12.12.1, 841 Kronecker sum Fact 7.5.15, 452 right inverse Fact 6.3.1, 404 solutions Proposition 6.1.7, 400 Fact 2.10.6, 125 linear system solution Cramer’s rule Fact 2.13.7, 140 nonnegative vector Fact 4.11.15, 305 norm Fact 9.15.1, 676 Fact 9.15.2, 677 Fact 9.15.3, 677 rank Theorem 2.6.4, 108

lower Hessenberg matrix Corollary 2.6.7, 110 right-invertible matrix Fact 2.13.8, 141 total least squares Fact 9.15.1, 679 linear-quadratic control problem definition, 848 Riccati equation Theorem 12.15.2, 849 solution Theorem 12.15.2, 849 linearly independent rational functions definition, 272 Littlewood H¨ older-induced norm Fact 9.8.17, 629 Fact 9.8.18, 630 Ljance minimal principal angle and subspaces Fact 5.11.39, 358 log majorization convex function Fact 2.21.11, 177 increasing function Fact 2.21.11, 177 positive-semidefinite matrix Fact 8.11.9, 516 logarithm, see matrix logarithm SO(3) Fact 11.15.10, 759 convex function Fact 11.16.14, 762 Fact 11.16.15, 762 determinant Fact 8.13.8, 535 determinant and convex function Proposition 8.6.17, 480 entropy

Fact 1.17.46, 65 Fact 1.17.47, 65 Fact 1.17.48, 65 Fact 1.18.30, 73 Euler constant Fact 1.9.5, 20 gamma Fact 1.9.5, 20 increasing function Proposition 8.6.13, 478 inequality Fact 1.17.46, 65 Fact 1.17.47, 65 Fact 1.17.48, 65 Jordan structure Corollary 11.4.4, 719 orthogonal matrix Fact 11.15.10, 759 rotation matrix Fact 11.15.10, 759 scalar inequalities Fact 1.11.21, 29 Fact 1.11.22, 29 Fact 1.11.23, 29 Fact 1.11.24, 29 Fact 1.11.25, 30 Fact 1.12.24, 37 Fact 1.12.25, 37 Fact 1.12.41, 42 Shannon’s inequality Fact 1.18.30, 73 trace and convex function Proposition 8.6.17, 480 logarithm function complex numbers Fact 1.20.7, 79 principal branch Fact 1.20.7, 79 scalar inequalities Fact 1.11.26, 30 Fact 1.11.27, 30 Fact 1.11.28, 30 logarithmic derivative asymptotically stable matrix Fact 11.18.11, 767 Lyapunov equation

1051

Fact 11.18.11, 767 properties Fact 11.15.7, 757 logarithmic mean arithmetic mean Fact 1.17.26, 59 Heron mean Fact 1.12.38, 41 identric mean Fact 1.12.37, 40 Polya’s inequality Fact 1.12.37, 40 logical equivalents De Morgan’s laws Fact 1.7.1, 11 existential statement Fact 1.7.4, 12 implication Fact 1.7.1, 11 Fact 1.7.2, 12 Fact 1.7.3, 12 universal statement Fact 1.7.4, 12 loop Definition 1.6.1, 9 lower block-triangular matrix definition Definition 3.1.3, 181 determinant Proposition 2.7.1, 112 lower bound generalized inverse Fact 9.8.44, 635 induced lower bound Fact 9.8.43, 635 Fact 9.8.44, 635 minimum singular value Fact 9.13.20, 663 lower bound for a partial ordering definition Definition 1.5.9, 8 lower Hessenberg matrix

1052

lower Hessenberg matrix

definition Definition 3.1.3, 181 submatrix Fact 3.19.1, 237 lower reverse-triangular matrix definition Fact 2.13.9, 141 determinant Fact 2.13.9, 141 lower triangular matrix, see upper triangular matrix characteristic polynomial Fact 4.10.10, 291 commutator Fact 3.17.11, 233 definition Definition 3.1.3, 181 determinant Fact 3.22.1, 240 eigenvalue Fact 4.10.10, 291 factorization Fact 5.15.10, 378 invariant subspace Fact 5.9.4, 339 Kronecker product Fact 7.4.4, 445 matrix exponential Fact 11.13.1, 743 Fact 11.13.17, 746 matrix power Fact 3.18.7, 235 matrix product Fact 3.22.2, 240 nilpotent matrix Fact 3.17.11, 233 similar matrices Fact 5.9.4, 339 spectrum Fact 4.10.10, 291 Toeplitz matrix Fact 3.18.7, 235 Fact 11.13.1, 743 LQG controller

continuous-time control problem Fact 12.23.6, 878 discrete-time control problem Fact 12.23.7, 878 dynamic compensator Fact 12.23.6, 878 Fact 12.23.7, 878 LU decomposition existence Fact 5.15.10, 378 Lucas numbers nonnegative matrix Fact 4.11.13, 303 Lukes stabilization and Gramian Fact 12.20.17, 865 LULU decomposition factorization Fact 5.15.11, 379 Lyapunov equation asymptotic stability Corollary 11.9.1, 730 asymptotically stable matrix Proposition 11.9.5, 733 Corollary 11.9.4, 732 Corollary 11.9.7, 734 Corollary 12.4.4, 806 Corollary 12.5.6, 807 Corollary 12.7.4, 819 Corollary 12.8.6, 822 Fact 12.21.7, 868 Fact 12.21.17, 870 controllably asymptotically stable Proposition 12.7.3, 816 detectability Corollary 12.5.6, 807 discrete-time asymptotically stable matrix

Proposition 11.10.5, 735 eigenvalue inclusion region Fact 12.21.20, 871 finite-sum solution Fact 12.21.17, 870 inertia Fact 12.21.1, 866 Fact 12.21.2, 867 Fact 12.21.3, 867 Fact 12.21.4, 867 Fact 12.21.5, 867 Fact 12.21.6, 868 Fact 12.21.7, 868 Fact 12.21.8, 868 Fact 12.21.9, 869 Fact 12.21.10, 869 Fact 12.21.11, 869 Fact 12.21.12, 869 Kronecker sum Corollary 11.9.4, 732 logarithmic derivative Fact 11.18.11, 767 Lyapunov stability Corollary 11.9.1, 730 Lyapunov-stable matrix Proposition 11.9.6, 733 Corollary 11.9.7, 734 matrix exponential Corollary 11.9.4, 732 Fact 11.18.18, 769 Fact 11.18.19, 769 null space Fact 12.21.15, 870 observability matrix Fact 12.21.15, 870 observably asymptotically stable Proposition 12.4.3, 805 positive-definite matrix Fact 12.21.16, 870 Fact 12.21.18, 870 positive-semidefinite matrix

Mann Fact 12.21.15, 870 Fact 12.21.19, 871 Schur power Fact 8.8.16, 494 semistability Corollary 11.9.1, 730 semistable matrix Fact 12.21.15, 870 skew-Hermitian matrix Fact 11.18.12, 768 stabilizability Corollary 12.8.6, 822 Lyapunov stability eigenvalue Proposition 11.8.2, 727 linear dynamical system Proposition 11.8.2, 727 Lyapunov equation Corollary 11.9.1, 730 matrix exponential Proposition 11.8.2, 727 nonlinear system Theorem 11.7.2, 725 Lyapunov’s direct method stability theory Theorem 11.7.2, 725 Lyapunov-stable equilibrium definition Definition 11.7.1, 725 Lyapunov-stable matrix almost nonnegative matrix Fact 11.19.4, 775 compartmental matrix Fact 11.19.6, 776 definition Definition 11.8.1, 727 group-invertible matrix Fact 11.18.2, 766 Kronecker sum Fact 11.18.32, 773

Fact 11.18.33, 773 Lyapunov equation Proposition 11.9.6, 733 Corollary 11.9.7, 734 Lyapunov-stable polynomial Proposition 11.8.4, 728 matrix exponential Fact 11.18.5, 766 Fact 11.21.8, 783 minimal realization Definition 12.9.17, 829 N-matrix Fact 11.19.4, 775 normal matrix Fact 11.18.37, 774 positive-definite matrix Proposition 11.9.6, 733 Corollary 11.9.7, 734 semidissipative matrix Fact 11.18.37, 774 semistable matrix Fact 11.18.1, 766 similar matrices Fact 11.18.4, 766 step response Fact 12.20.1, 863 Lyapunov-stable polynomial definition Definition 11.8.3, 728 Lyapunov-stable matrix Proposition 11.8.4, 728 subdeterminant Fact 11.18.23, 770 Lyapunov-stable transfer function minimal realization Proposition 12.9.18, 829 SISO entry Proposition 12.9.19, 829

1053

M M-matrix definition Fact 4.11.8, 301 determinant Fact 4.11.10, 302 eigenvector Fact 4.11.12, 302 inverse Fact 4.11.10, 302 irreducible matrix Fact 4.11.12, 302 nonnegative matrix Fact 4.11.8, 301 rank Fact 8.7.8, 488 Schur product Fact 7.6.15, 457 submatrix Fact 4.11.9, 302 Z-matrix Fact 4.11.8, 301 Fact 4.11.10, 302 Magnus determinant equalities Fact 2.13.16, 143 Magnus expansion time-varying dynamics Fact 11.13.4, 743 Makelainen difference of idempotent matrices Fact 5.12.19, 367 Maligranda inequality complex numbers Fact 1.20.5, 77 norm Fact 9.7.10, 621 Fact 9.7.13, 622 Mann positivity of a quadratic form on a subspace

1054

Mann

Fact 8.15.28, 554 Marcus quadratic form inequality Fact 8.15.20, 553 similar matrices and nonzero diagonal entry Fact 5.9.16, 341 Markov block-Hankel matrix controllable pair Proposition 12.9.11, 827 definition, 826, 827 minimal realization Proposition 12.9.12, 828 observable pair Proposition 12.9.11, 827 rational transfer function Proposition 12.9.11, 827 Proposition 12.9.12, 828 Proposition 12.9.13, 828 Markov parameter definition, 799 rational transfer function Proposition 12.9.7, 824 Martins’s inequality sum of integers Fact 1.11.31, 32 Mason polynomial root bound Fact 11.20.10, 781 mass definition, 718 mass matrix partitioned matrix Fact 5.12.21, 368

mass-spring system spectrum Fact 5.12.21, 368 stability Fact 11.18.38, 774 Massera-Schaffer inequality complex numbers Fact 1.20.5, 77 norm Fact 9.7.10, 621 Fact 9.7.13, 622 matricial norm partitioned matrix Fact 9.10.1, 649 matrix definition, 86 matrix cosine matrix exponential Fact 11.12.1, 742 matrix sine Fact 11.12.1, 742 matrix derivative definition, 691 matrix differential equation Jacobi’s identity Fact 11.13.4, 743 matrix exponential Fact 11.13.3, 743 Riccati differential equation Fact 12.23.5, 877 time-varying dynamics Fact 11.13.4, 743 Fact 11.13.5, 744 matrix exponential 2 × 2 matrix Proposition 11.3.2, 716 Corollary 11.3.3, 716 Lemma 11.3.1, 715 Example 11.3.4, 716 Example 11.3.5, 717 3 × 3 matrix Fact 11.11.5, 737

3 × 3 orthogonal matrix Fact 11.11.10, 739 Fact 11.11.11, 739 3 × 3 skew-symmetric matrix Fact 11.11.6, 737 Fact 11.11.10, 739 Fact 11.11.11, 739 4 × 4 skew-symmetric matrix Fact 11.11.15, 740 Fact 11.11.16, 740 Fact 11.11.17, 741 Fact 11.11.18, 741 SO(n) Fact 11.11.3, 736 almost nonnegative matrix Fact 11.19.1, 774 Fact 11.19.2, 774 asymptotic stability Proposition 11.8.2, 727 asymptotically stable matrix Lemma 11.9.2, 731 Fact 11.18.8, 767 Fact 11.18.9, 767 Fact 11.18.10, 767 Fact 11.18.15, 768 Fact 11.18.18, 769 Fact 11.18.19, 769 Fact 11.21.8, 783 block-diagonal matrix Proposition 11.2.8, 713 commutator Fact 11.14.9, 749 Fact 11.14.11, 749 Fact 11.14.12, 749 Fact 11.14.13, 750 Fact 11.14.14, 750 Fact 11.14.15, 750 Fact 11.14.16, 750 Fact 11.14.17, 751 Fact 11.14.18, 751 commuting matrices Proposition 11.1.5, 709 Corollary 11.1.6, 709

matrix exponential Fact 11.14.1, 746 Fact 11.14.5, 748 complex conjugate Proposition 11.2.8, 713 complex conjugate transpose Proposition 11.2.8, 713 Fact 11.15.4, 756 Fact 11.15.6, 757 convergence in time Proposition 11.8.7, 729 convergent sequence Proposition 11.1.3, 708 Fact 11.14.7, 749 Fact 11.14.8, 749 Fact 11.14.9, 749 Fact 11.21.16, 785 convergent series Proposition 11.1.2, 708 convex function Fact 8.14.18, 549 Fact 11.16.14, 762 Fact 11.16.15, 762 cross product Fact 11.11.7, 738 Fact 11.11.8, 738 Fact 11.11.9, 739 cross-product matrix Fact 11.11.6, 737 Fact 11.11.12, 739 Fact 11.11.13, 739 Fact 11.11.14, 740 Fact 11.11.17, 741 Fact 11.11.18, 741 definition Definition 11.1.1, 707 derivative Fact 8.12.32, 530 Fact 11.14.3, 748 Fact 11.14.4, 748 Fact 11.14.10, 749 Fact 11.15.2, 756 derivative of a matrix Fact 11.14.11, 749 determinant Proposition 11.4.6, 719 Corollary 11.2.4, 712 Corollary 11.2.5, 712

Fact 11.13.15, 745 Fact 11.15.5, 756 diagonal matrix Fact 11.13.17, 746 discrete-time asymptotic stability Proposition 11.10.2, 734 discrete-time asymptotically stable matrix Fact 11.21.8, 783 discrete-time Lyapunov stability Proposition 11.10.2, 734 discrete-time Lyapunov-stable matrix Fact 11.21.8, 783 discrete-time semistability Proposition 11.10.2, 734 discrete-time semistable matrix Fact 11.21.8, 783 Fact 11.21.16, 785 dissipative matrix Fact 11.15.3, 756 Drazin generalized inverse Fact 11.13.12, 745 Fact 11.13.14, 745 eigenstructure Proposition 11.2.7, 712 Frobenius norm Fact 11.14.32, 755 Fact 11.15.3, 756 generalized inverse Fact 11.13.10, 745 geometric mean Fact 8.10.44, 510 Golden-Thompson inequality Fact 11.14.28, 753 Fact 11.16.4, 760 group Proposition 11.6.7, 724

1055

group generalized inverse Fact 11.13.13, 745 Fact 11.13.14, 745 Fact 11.18.5, 766 Fact 11.18.6, 766 group-invertible matrix Fact 11.18.14, 768 Hamiltonian matrix Proposition 11.6.7, 724 Hermitian matrix Proposition 11.2.8, 713 Proposition 11.2.9, 715 Proposition 11.4.5, 719 Corollary 11.2.6, 712 Fact 11.14.7, 749 Fact 11.14.8, 749 Fact 11.14.21, 752 Fact 11.14.28, 753 Fact 11.14.29, 754 Fact 11.14.31, 754 Fact 11.14.32, 755 Fact 11.14.34, 755 Fact 11.15.1, 756 Fact 11.16.4, 760 Fact 11.16.5, 761 Fact 11.16.13, 762 Fact 11.16.17, 763 idempotent matrix Fact 11.11.1, 736 Fact 11.16.12, 762 infinite product Fact 11.14.18, 751 integral Proposition 11.1.4, 709 Lemma 11.9.2, 731 Fact 11.13.10, 745 Fact 11.13.11, 745 Fact 11.13.12, 745 Fact 11.13.13, 745 Fact 11.13.14, 745 Fact 11.13.15, 745 Fact 11.13.16, 746 Fact 11.14.2, 748 Fact 11.16.7, 761 Fact 11.18.5, 766 Fact 11.18.6, 766 inverse matrix

1056

matrix exponential

Proposition 11.2.8, 713 Fact 11.13.11, 745 involutory matrix Fact 11.11.1, 736 Jordan structure Corollary 11.4.4, 719 Kronecker product Proposition 11.1.7, 709 Fact 11.14.37, 755 Fact 11.14.38, 755 Kronecker sum Proposition 11.1.7, 709 Fact 11.14.36, 755 Fact 11.14.37, 755 Laplace transform Proposition 11.2.2, 711 Lie algebra Proposition 11.6.7, 724 Lie algebra of a Lie group Proposition 11.6.3, 723 Lie-Trotter formula Fact 11.14.7, 749 Lie-Trotter product formula Corollary 11.4.8, 720 Fact 11.16.2, 759 Fact 11.16.3, 759 limit Fact 11.18.5, 766 Fact 11.18.6, 766 Fact 11.18.7, 767 linear matrix equation Proposition 11.9.3, 731 logarithm Fact 11.14.21, 752 lower triangular matrix Fact 11.13.1, 743 Fact 11.13.17, 746 Lyapunov equation Corollary 11.9.4, 732 Fact 11.18.18, 769 Fact 11.18.19, 769 Lyapunov stability Proposition 11.8.2, 727 Lyapunov-stable matrix

Fact 11.18.5, 766 Fact 11.21.8, 783 matrix cosine Fact 11.12.1, 742 matrix differential equation Fact 11.13.3, 743 matrix logarithm Theorem 11.5.2, 721 Proposition 11.4.2, 718 Fact 11.13.18, 746 Fact 11.14.31, 754 matrix power Fact 11.13.20, 746 matrix sine Fact 11.12.1, 742 maximum eigenvalue Fact 11.16.4, 760 maximum singular value Fact 11.15.1, 756 Fact 11.15.2, 756 Fact 11.15.5, 756 Fact 11.16.6, 761 Fact 11.16.10, 762 nilpotent matrix Fact 11.11.1, 736 Fact 11.13.18, 746 nondecreasing function Fact 8.10.44, 510 norm Fact 11.16.9, 761 Fact 11.16.11, 762 Fact 11.16.12, 762 norm bound Fact 11.18.10, 767 normal matrix Proposition 11.2.8, 713 Fact 11.13.19, 746 Fact 11.14.5, 748 Fact 11.16.10, 762 orthogonal matrix Proposition 11.6.7, 724 Fact 11.11.6, 737 Fact 11.11.7, 738 Fact 11.11.8, 738 Fact 11.11.9, 739 Fact 11.11.12, 739

Fact 11.11.13, 739 Fact 11.11.14, 740 Fact 11.15.10, 759 outer-product matrix Fact 11.11.1, 736 partitioned matrix Fact 11.11.2, 736 Fact 11.14.2, 748 Peierls-Bogoliubov inequality Fact 11.14.29, 754 polar decomposition Fact 11.13.9, 745 polynomial matrix Proposition 11.2.1, 710 positive-definite matrix Proposition 11.2.8, 713 Proposition 11.2.9, 715 Fact 11.14.20, 751 Fact 11.14.22, 752 Fact 11.14.23, 752 Fact 11.15.1, 756 positive-semidefinite matrix Fact 11.14.20, 751 Fact 11.14.35, 755 Fact 11.16.6, 761 Fact 11.16.16, 762 quaternions Fact 11.11.16, 740 rank-two matrix Fact 11.11.19, 742 resolvent Proposition 11.2.2, 711 rotation matrix Fact 11.11.13, 739 Fact 11.11.14, 740 Schur product Fact 11.14.21, 752 semisimple matrix Proposition 11.2.7, 712 semistability Proposition 11.8.2, 727 semistable matrix Fact 11.18.6, 766 Fact 11.18.7, 767 Fact 11.21.8, 783

matrix logarithm series Proposition 11.4.7, 720 Fact 11.14.17, 751 similar matrices Proposition 11.2.9, 715 singular value Fact 11.15.5, 756 Fact 11.16.14, 762 Fact 11.16.15, 762 skew-Hermitian matrix Proposition 11.2.8, 713 Proposition 11.2.9, 715 Fact 11.14.6, 749 Fact 11.14.33, 755 skew-involutory matrix Fact 11.11.1, 736 skew-symmetric matrix Example 11.3.6, 717 Fact 11.11.3, 736 Fact 11.11.6, 737 Fact 11.11.7, 738 Fact 11.11.8, 738 Fact 11.11.9, 739 Fact 11.11.16, 740 Specht’s ratio Fact 11.14.28, 753 spectral abscissa Fact 11.13.2, 743 Fact 11.15.8, 758 Fact 11.15.9, 758 Fact 11.18.8, 767 Fact 11.18.9, 767 spectral radius Fact 11.13.2, 743 spectrum Proposition 11.2.3, 712 Corollary 11.2.6, 712 stable subspace Proposition 11.8.8, 729 state equation Proposition 12.1.1, 795 strong log majorization Fact 11.16.4, 760 submultiplicative norm

Proposition 11.1.2, 708 Fact 11.15.8, 758 Fact 11.15.9, 758 Fact 11.16.8, 761 Fact 11.18.8, 767 Fact 11.18.9, 767 sum of integer powers Fact 11.11.4, 736 symplectic matrix Proposition 11.6.7, 724 thermodynamic inequality Fact 11.14.31, 754 trace Corollary 11.2.4, 712 Corollary 11.2.5, 712 Fact 8.14.18, 549 Fact 11.11.6, 737 Fact 11.14.3, 748 Fact 11.14.10, 749 Fact 11.14.28, 753 Fact 11.14.29, 754 Fact 11.14.30, 754 Fact 11.14.31, 754 Fact 11.14.36, 755 Fact 11.14.38, 755 Fact 11.15.4, 756 Fact 11.15.5, 756 Fact 11.16.1, 759 Fact 11.16.4, 760 transpose Proposition 11.2.8, 713 unipotent matrix Fact 11.13.18, 746 unitarily invariant norm Fact 11.15.6, 757 Fact 11.16.4, 760 Fact 11.16.5, 761 Fact 11.16.13, 762 Fact 11.16.16, 762 Fact 11.16.17, 763 unitary matrix Proposition 11.2.8, 713 Proposition 11.2.9, 715 Proposition 11.6.7, 724 Corollary 11.2.6, 712 Fact 11.14.6, 749

1057

Fact 11.14.33, 755 Fact 11.14.34, 755 upper triangular matrix Fact 11.11.4, 736 Fact 11.13.1, 743 Fact 11.13.17, 746 vibration equation Example 11.3.7, 717 weak majorization Fact 11.16.4, 760 Z-matrix Fact 11.19.1, 774 Zassenhaus product formula Fact 11.14.18, 751 matrix function definition, 688 Lagrange-Hermite interpolation polynomial Theorem 10.5.2, 689 spectrum Corollary 10.5.4, 690 matrix function defined at a point definition Definition 10.5.1, 689 matrix function evaluation identity theorem Theorem 10.5.3, 690 matrix inequality matrix logarithm Proposition 8.6.4, 475 matrix inversion lemma generalization Fact 2.16.21, 157 generalized inverse Fact 6.4.5, 413 inverse matrix Corollary 2.8.8, 117 matrix logarithm chaotic order Fact 8.20.1, 574 complex matrix

1058

matrix logarithm

Definition 11.4.1, 718 convergent series Theorem 11.5.2, 721 convex function Proposition 8.6.17, 480 determinant Fact 8.19.31, 574 Fact 9.8.39, 633 Fact 11.14.24, 752 determinant and derivative Proposition 10.7.3, 692 discrete-time Lyapunov-stable matrix Fact 11.14.19, 751 eigenvalues Theorem 11.5.2, 721 exponential Fact 11.14.26, 753 geometric mean Fact 11.14.39, 755 Hamiltonian matrix Fact 11.14.19, 751 Klein’s inequality Fact 11.14.25, 752 limit Proposition 8.6.4, 475 matrix exponential Theorem 11.5.2, 721 Proposition 11.4.2, 718 Fact 11.13.18, 746 Fact 11.14.21, 752 Fact 11.14.31, 754 matrix inequality Proposition 8.6.4, 475 maximum singular value Fact 8.19.31, 574 nonsingular matrix Proposition 11.4.2, 718 norm Theorem 11.5.2, 721 positive-definite matrix Proposition 8.6.4, 475 Proposition 11.4.5, 719 Fact 8.9.44, 501 Fact 8.13.8, 535

Fact 8.19.30, 574 Fact 8.20.1, 574 Fact 9.9.55, 647 Fact 11.14.24, 752 Fact 11.14.25, 752 Fact 11.14.26, 753 Fact 11.14.27, 753 positive-semidefinite matrix Fact 9.9.54, 647 quadratic form Fact 8.15.16, 552 real matrix Proposition 11.4.3, 719 Fact 11.14.19, 751 relative entropy Fact 11.14.25, 752 Schur product Fact 8.22.49, 594 Fact 8.22.50, 594 spectrum Theorem 11.5.2, 721 symplectic matrix Fact 11.14.19, 751 trace Fact 11.14.24, 752 Fact 11.14.25, 752 Fact 11.14.27, 753 Fact 11.14.31, 754 unitarily invariant norm Fact 9.9.54, 647 matrix measure properties Fact 11.15.7, 757 matrix polynomial definition, 256 matrix power outer-product perturbation Fact 2.12.18, 138 positive-definite matrix inequality Fact 8.10.53, 512 Fact 8.20.2, 575 positive-semidefinite matrix Fact 8.12.31, 530

Fact 8.15.17, 552 matrix product lower triangular matrix Fact 3.22.2, 240 normal matrix Fact 9.9.6, 636 strictly lower triangular matrix Fact 3.22.2, 240 strictly upper triangular matrix Fact 3.22.2, 240 unitarily invariant norm Fact 9.9.6, 636 upper triangular matrix Fact 3.22.2, 240 matrix sign function convergent sequence Fact 5.15.21, 381 definition Definition 10.6.2, 690 partitioned matrix Fact 10.10.3, 699 positive-definite matrix Fact 10.10.4, 699 properties Fact 10.10.2, 699 square root Fact 5.15.21, 381 matrix sine matrix cosine Fact 11.12.1, 742 matrix exponential Fact 11.12.1, 742 maximal solution Riccati equation Definition 12.16.12, 853 Theorem 12.18.1, 859 Theorem 12.18.4, 860 Proposition 12.18.2, 860

maximum singular value Proposition 12.18.7, 862 maximal solution of the Riccati equation closed-loop spectrum Proposition 12.18.2, 860 stabilizability Theorem 12.18.1, 859 maximization continuous function Fact 10.11.3, 700 maximum eigenvalue commutator Fact 9.9.30, 641 Fact 9.9.31, 642 Hermitian matrix Lemma 8.4.3, 467 Fact 5.11.5, 350 Fact 8.10.3, 501 matrix exponential Fact 11.16.4, 760 positive-semidefinite matrix Fact 8.19.11, 567 Fact 8.19.13, 568 Fact 8.19.14, 568 quadratic form Lemma 8.4.3, 467 spectral abscissa Fact 5.11.5, 350 unitarily invariant norm Fact 9.9.30, 641 Fact 9.9.31, 642 maximum singular value absolute value Fact 9.13.10, 661 block-diagonal matrix Fact 5.11.33, 357 block-triangular matrix Fact 5.11.32, 357 bound Fact 5.11.35, 358

commutator Fact 9.9.29, 641 Fact 9.14.9, 667 complex conjugate transpose Fact 8.18.3, 559 Fact 8.19.11, 567 Fact 8.22.10, 586 Cordes inequality Fact 8.19.27, 572 derivative Fact 11.15.2, 756 determinant Fact 9.14.17, 669 Fact 9.14.18, 669 discrete-time Lyapunov-stable matrix Fact 11.21.20, 786 dissipative matrix Fact 8.18.12, 563 eigenvalue of Hermitian part Fact 5.11.25, 355 eigenvalue perturbation Fact 9.12.4, 657 Fact 9.12.8, 658 elementary projector Fact 9.14.1, 665 equi-induced self-adjoint norm Fact 9.13.5, 660 equi-induced unitarily invariant norm Fact 9.13.4, 660 generalized inverse Fact 9.14.8, 666 Fact 9.14.30, 673 Hermitian matrix Fact 5.11.5, 350 Fact 9.9.41, 644 H¨ older-induced norm Fact 9.8.21, 630 idempotent matrix Fact 5.11.38, 358 Fact 5.11.39, 358

1059

Fact 5.12.18, 367 induced lower bound Corollary 9.5.5, 614 induced norm Fact 9.8.24, 631 inequality Proposition 9.2.2, 602 Corollary 9.6.5, 616 Corollary 9.6.9, 617 Fact 9.9.32, 642 Fact 9.14.16, 669 inverse matrix Fact 9.14.8, 666 Kreiss matrix theorem Fact 11.21.20, 786 Kronecker product Fact 9.14.38, 676 matrix difference Fact 8.19.8, 566 Fact 9.9.32, 642 matrix exponential Fact 11.15.1, 756 Fact 11.15.2, 756 Fact 11.15.5, 756 Fact 11.16.6, 761 Fact 11.16.10, 762 matrix logarithm Fact 8.19.31, 574 matrix power Fact 8.19.27, 572 Fact 9.13.7, 660 Fact 9.13.9, 660 normal matrix Fact 5.14.14, 372 Fact 9.8.13, 629 Fact 9.12.8, 658 Fact 9.13.7, 660 Fact 9.13.8, 660 Fact 9.14.5, 666 Fact 11.16.10, 762 outer-product matrix Fact 5.11.16, 353 Fact 5.11.18, 353 Fact 9.7.26, 626 partitioned matrix Fact 8.18.3, 559 Fact 8.18.14, 563

1060

maximum singular value

Fact 8.19.1, 564 Fact 8.19.2, 564 Fact 9.10.1, 649 Fact 9.10.3, 650 Fact 9.10.4, 651 Fact 9.10.5, 651 Fact 9.14.12, 668 Fact 9.14.13, 668 Fact 9.14.14, 668 positive-definite matrix Fact 8.19.26, 572 positive-semidefinite matrix Fact 8.19.1, 564 Fact 8.19.2, 564 Fact 8.19.8, 566 Fact 8.19.12, 568 Fact 8.19.13, 568 Fact 8.19.14, 568 Fact 8.19.15, 569 Fact 8.19.16, 569 Fact 8.19.26, 572 Fact 8.19.27, 572 Fact 8.19.29, 573 Fact 8.19.31, 574 Fact 8.19.32, 574 Fact 8.21.9, 579 Fact 11.16.6, 761 power Fact 11.21.20, 786 product Fact 9.14.2, 665 projector Fact 5.11.38, 358 Fact 5.12.17, 366 Fact 5.12.18, 367 Fact 9.14.1, 665 Fact 9.14.30, 673 quadratic form Fact 9.13.1, 659 Fact 9.13.2, 660 Schur product Fact 8.22.10, 586 Fact 9.14.31, 673 Fact 9.14.32, 674 Fact 9.14.35, 676 Fact 9.14.36, 676 spectral abscissa

Fact 5.11.26, 355 spectral radius Corollary 9.4.10, 611 Fact 5.11.5, 350 Fact 5.11.26, 355 Fact 8.19.26, 572 Fact 9.8.13, 629 Fact 9.13.9, 660 square root Fact 8.19.14, 568 Fact 9.8.32, 632 Fact 9.14.15, 669 sum of matrices Fact 9.14.15, 669 trace Fact 5.12.7, 364 Fact 9.14.4, 665 trace norm Corollary 9.3.8, 607 unitarily invariant norm Fact 9.9.10, 637 Fact 9.9.29, 641 maximum singular value bound Frobenius norm Fact 9.13.13, 661 minimum singular value bound Fact 9.13.14, 662 polynomial root Fact 9.13.14, 662 trace Fact 9.13.13, 661 maximum singular value of a matrix difference Kato Fact 9.9.32, 642 maximum singular value of a partitioned matrix Parrott’s theorem Fact 9.14.13, 668 Tomiyama Fact 9.14.12, 668 McCarthy inequality

positive-semidefinite matrix Fact 8.12.30, 529 McCoy simultaneous triangularization Fact 5.17.5, 392 McIntosh’s inequality unitarily invariant norm Fact 9.9.47, 646 McLaughlin’s inequality refined Cauchy-Schwarz inequality Fact 1.18.17, 70 McMillan degree Definition 4.7.10, 273 minimal realization Theorem 12.9.16, 829 mean inequality Fact 1.18.18, 70 Laguerre-Samuelson inequality Fact 1.17.12, 55 Fact 8.9.36, 499 variance inequality Fact 1.17.12, 55 Fact 8.9.36, 499 mean-value inequality product of means Fact 1.17.39, 63 Fact 1.17.45, 64 Mercator’s series infinite series Fact 1.20.8, 79 Metzler matrix definition, 252 Mihet polynomial bound Fact 11.20.14, 781 Milne’s inequality

minimum singular value refined Cauchy-Schwarz inequality Fact 1.18.15, 69 Milnor simultaneous diagonalization of symmetric matrices Fact 8.17.6, 559 MIMO transfer function definition Definition 12.9.1, 822 minimal polynomial block-diagonal matrix Lemma 5.2.7, 312 block-triangular matrix Fact 4.10.13, 292 characteristic polynomial Fact 4.9.25, 288 companion matrix Proposition 5.2.1, 310 Corollary 5.2.4, 312 Corollary 5.2.5, 312 cyclic matrix Proposition 5.5.14, 325 definition, 269 existence Theorem 4.6.1, 269 index of an eigenvalue Proposition 5.5.14, 325 Jordan form Proposition 5.5.14, 325 null space Corollary 11.8.6, 729 partitioned matrix Fact 4.10.13, 292 range Corollary 11.8.6, 729 similar matrices Proposition 4.6.3, 270 Fact 11.23.3, 788 Fact 11.23.4, 788 Fact 11.23.5, 789 Fact 11.23.6, 789

Fact 11.23.7, 790 Fact 11.23.8, 790 Fact 11.23.9, 791 Fact 11.23.10, 792 Fact 11.23.11, 792 spectrum Fact 4.9.26, 288 stable subspace Proposition 11.8.5, 728 Fact 11.23.1, 786 Fact 11.23.2, 787 upper block-triangular matrix Fact 4.10.13, 292 minimal realization asymptotically stable matrix Definition 12.9.17, 829 asymptotically stable transfer function Proposition 12.9.18, 829 balanced realization Proposition 12.9.21, 830 block decomposition Proposition 12.9.10, 825 controllable pair Proposition 12.9.10, 825 Corollary 12.9.15, 829 definition Definition 12.9.14, 828 Kalman decomposition Proposition 12.9.10, 825 Lyapunov-stable matrix Definition 12.9.17, 829 Lyapunov-stable transfer function Proposition 12.9.18, 829 Markov block-Hankel matrix

1061

Proposition 12.9.12, 828 McMillan degree Theorem 12.9.16, 829 observable pair Proposition 12.9.10, 825 Corollary 12.9.15, 829 pole Fact 12.22.2, 872 Fact 12.22.12, 874 rational transfer function Fact 12.22.12, 874 semistable matrix Definition 12.9.17, 829 semistable transfer function Proposition 12.9.18, 829 minimal-rank equality partitioned matrix Fact 6.5.7, 424 minimum eigenvalue Hermitian matrix Lemma 8.4.3, 467 Fact 8.10.3, 501 nonnegative matrix Fact 4.11.11, 302 quadratic form Lemma 8.4.3, 467 Z-matrix Fact 4.11.11, 302 minimum principle eigenvalue characterization Fact 8.18.15, 563 minimum singular value determinant Fact 9.14.18, 669 eigenvalue of Hermitian part Fact 5.11.25, 355 induced lower bound Corollary 9.5.5, 614 inequality

1062

minimum singular value

Corollary 9.6.6, 616 Fact 9.13.6, 660 lower bound Fact 9.13.20, 663 quadratic form Fact 9.13.1, 659 spectral abscissa Fact 5.11.26, 355 spectral radius Fact 5.11.26, 355 minimum singular value bound maximum singular value bound Fact 9.13.14, 662 polynomial root Fact 9.13.14, 662 Minkowski set-defined norm Fact 10.8.22, 694 Minkowski inequality reverse inequality Fact 9.7.19, 624 Minkowski’s determinant theorem positive-semidefinite matrix determinant Corollary 8.4.15, 472 Minkowski’s inequality H¨ older norm Lemma 9.1.3, 598 positive-semidefinite matrix Fact 8.12.30, 529 scalar case Fact 1.18.25, 72 minor, see subdeterminant Mircea’s inequality triangle Fact 2.20.11, 169 Mirsky singular value trace bound Fact 5.12.6, 364

Mirsky’s theorem singular value perturbation Fact 9.14.29, 673

Moore-Penrose generalized inverse, see generalized inverse

MISO transfer function definition Definition 12.9.1, 822

Muirhead’s theorem Schur convex function Fact 1.17.25, 59 strong majorization Fact 2.21.5, 176

mixed arithmetic-geometric mean inequality arithmetic mean Fact 1.17.40, 63 ML-matrix definition, 252 Moler regular pencil Fact 5.17.3, 392 monic polynomial definition, 253 monic polynomial matrix definition, 256 monotone norm absolute norm Proposition 9.1.2, 597 definition, 597 monotonicity Callebaut Fact 1.18.1, 66 power inequality Fact 1.12.33, 39 power mean inequality Fact 1.17.30, 60 Riccati equation Proposition 12.18.5, 861 Corollary 12.18.6, 861 monotonicity theorem Hermitian matrix eigenvalue Theorem 8.4.9, 469 Fact 8.10.4, 501

multicompanion form definition, 311 existence Theorem 5.2.3, 311 similar matrices Corollary 5.2.6, 312 similarity invariant Corollary 5.2.6, 312 multigraph definition, 9 multinomial theorem power of sum Fact 1.17.1, 52 multiple definition, 255 multiplication definition, 89 function composition Theorem 2.1.3, 88 Kronecker product Proposition 7.1.6, 440 multiplicative commutator realization Fact 5.15.34, 383 reflector realization Fact 5.15.35, 384 multiplicative perturbation small-gain theorem Fact 9.13.22, 664 multiplicity of a root definition, 254 multirelation

nilpotent matrix definition, 6 multiset definition, 2 multispectrum, see eigenvalue, spectrum definition Definition 4.4.4, 262 properties Proposition 4.4.5, 263

N N-matrix almost nonnegative matrix Fact 11.19.3, 775 Fact 11.19.5, 776 asymptotically stable matrix Fact 11.19.5, 776 definition Fact 11.19.3, 775 group-invertible matrix Fact 11.19.4, 775 Lyapunov-stable matrix Fact 11.19.4, 775 nonnegative matrix Fact 11.19.3, 775 Nanjundiah mixed arithmeticgeometric mean inequality Fact 1.17.40, 63 natural frequency definition, 718 Fact 5.14.34, 375 necessity definition, 1 negation definition, 1 negative-definite matrix

asymptotically stable matrix Fact 11.18.30, 773 definition Definition 3.1.1, 179 negative-semidefinite matrix definition Definition 3.1.1, 179 Euclidean distance matrix Fact 9.8.14, 629 Nesbitt’s inequality scalar inequality Fact 1.13.21, 48 Newcomb simultaneous cogredient diagonalization, 595 Newton’s identities elementary symmetric polynomial Fact 4.8.2, 276 polynomial roots Fact 4.8.2, 276 spectrum Fact 4.10.8, 290 Newton’s inequality elementary symmetric polynomial Fact 1.17.11, 55 Newton-Raphson algorithm generalized inverse Fact 6.3.34, 411 inverse matrix Fact 2.16.29, 158 square root Fact 5.15.21, 381 Niculescu’s inequality absolute-value function Fact 1.12.19, 36 convex function

1063

Fact 1.10.5, 24 square-root function Fact 1.12.20, 36 nilpotent matrix additive decomposition Fact 5.9.5, 339 adjugate Fact 6.3.6, 404 commutator Fact 3.12.16, 217 Fact 3.17.11, 233 Fact 3.17.12, 233 Fact 3.17.13, 233 commuting matrices Fact 3.17.9, 233 Fact 3.17.10, 233 defective matrix Fact 5.14.17, 372 definition Definition 3.1.1, 179 determinant Fact 3.17.9, 233 example Example 5.5.16, 326 factorization Fact 5.15.29, 383 idempotent matrix Fact 3.12.16, 217 identity-matrix perturbation Fact 3.17.7, 232 Fact 3.17.8, 233 inertia Fact 5.8.4, 334 Jordan-Chevalley decomposition Fact 5.9.5, 339 Kronecker product Fact 7.4.17, 446 Kronecker sum Fact 7.5.3, 450 Fact 7.5.8, 451 lower triangular matrix Fact 3.17.11, 233 matrix exponential Fact 11.11.1, 736 Fact 11.13.18, 746

1064

nilpotent matrix

matrix sum Fact 3.17.10, 233 null space Fact 3.17.1, 232 Fact 3.17.2, 232 Fact 3.17.3, 232 outer-product matrix Fact 5.14.3, 370 partitioned matrix Fact 3.12.14, 217 Fact 5.10.23, 349 range Fact 3.17.1, 232 Fact 3.17.2, 232 Fact 3.17.3, 232 rank Fact 3.17.4, 232 Fact 3.17.5, 232 S-N decomposition Fact 5.9.5, 339 similar matrices Proposition 3.4.5, 189 Fact 5.10.23, 349 simultaneous triangularization Fact 5.17.6, 392 spectrum Proposition 5.5.20, 326 Toeplitz matrix Fact 3.18.6, 235 trace Fact 3.17.6, 232 triangular matrix Fact 5.17.6, 392 unitarily similar matrices Proposition 3.4.5, 189 upper triangular matrix Fact 3.17.11, 233 node definition, 9 nondecreasing function convex function Lemma 8.6.16, 480 definition

Definition 8.6.12, 478 function composition Lemma 8.6.16, 480 geometric mean Fact 8.10.43, 508 Fact 8.10.44, 510 matrix exponential Fact 8.10.44, 510 matrix functions Proposition 8.6.13, 478 Schur complement Proposition 8.6.13, 478 nonderogatory eigenvalue definition Definition 5.5.4, 322 nonderogatory matrix definition Definition 5.5.4, 322 nonempty set definition, 2 nonincreasing function concave function Lemma 8.6.16, 480 definition Definition 8.6.12, 478 function composition Lemma 8.6.16, 480 nonnegative matrix almost nonnegative matrix Fact 11.19.1, 774 aperiodic graph Fact 4.11.4, 298 companion matrix Fact 4.11.14, 304 copositive matrix Fact 8.15.33, 556 definition, 88 Definition 3.1.4, 182 difference equation Fact 4.11.13, 303 eigenvalue Fact 4.11.4, 298 Fibonacci numbers Fact 4.11.13, 303

limit of matrix powers Fact 4.11.22, 306 Lucas numbers Fact 4.11.13, 303 M-matrix Fact 4.11.8, 301 matrix power Fact 4.11.23, 306 minimum eigenvalue Fact 4.11.11, 302 N-matrix Fact 11.19.3, 775 spectral radius Fact 4.11.4, 298 Fact 4.11.8, 301 Fact 4.11.17, 305 Fact 4.11.18, 305 Fact 7.6.13, 456 Fact 9.11.9, 656 Fact 11.19.3, 775 spectral radius convexity Fact 4.11.20, 306 spectral radius monotonicity Fact 4.11.19, 306 trace Fact 4.11.23, 306 nonnegative matrix eigenvalues Perron-Frobenius theorem Fact 4.11.4, 298 nonnegative vector definition, 86 linear system solution Fact 4.11.15, 305 null space Fact 4.11.16, 305 nonsingular matrix complex conjugate Proposition 2.6.8, 110 complex conjugate transpose Proposition 2.6.8, 110 Fact 2.16.30, 158

norm controllable subspace Proposition 12.6.10, 812 cyclic matrix Fact 5.14.7, 370 definition, 109 determinant Corollary 2.7.4, 113 Lemma 2.8.6, 117 determinant lower bound Fact 4.10.19, 294 diagonal dominance theorem Fact 4.10.18, 293 Fact 4.10.19, 294 diagonally dominant matrix Fact 4.10.18, 293 dissipative matrix Fact 3.22.8, 241 distance to singularity Fact 9.14.7, 666 elementary matrix Fact 5.15.12, 379 factorization Fact 5.15.12, 379 Fact 5.15.36, 384 group Proposition 3.3.6, 187 idempotent matrix Fact 3.12.11, 216 Fact 3.12.25, 219 Fact 3.12.28, 220 Fact 3.12.32, 222 inverse matrix Fact 3.7.1, 192 matrix logarithm Proposition 11.4.2, 718 norm Fact 9.7.32, 626 normal matrix Fact 3.7.1, 192 perturbation Fact 9.14.6, 666 Fact 9.14.18, 669 range-Hermitian matrix

Proposition 3.1.6, 183 similar matrices Fact 5.10.11, 347 simplex Fact 2.20.4, 167 skew Hermitian matrix Fact 3.7.1, 192 spectral radius Fact 4.10.30, 296 submultiplicative norm Fact 9.8.5, 627 Sylvester’s equation Fact 12.21.14, 870 transpose Proposition 2.6.8, 110 unitary matrix Fact 3.7.1, 192 unobservable subspace Proposition 12.3.10, 802 weak diagonal dominance theorem Fact 4.10.20, 294 nonsingular matrix transformation Smith polynomial Proposition 4.3.8, 260 nonsingular polynomial matrix Definition 4.2.5, 257 regular polynomial matrix Proposition 4.2.5, 257 nonzero diagonal entry similar matrices Fact 5.9.16, 341 norm absolute definition, 597 absolute sum definition, 599 column definition, 611 compatible

1065

definition, 604 complex conjugate transpose Fact 9.8.8, 627 convex set Fact 9.7.23, 625 Dunkl-Williams inequality Fact 9.7.10, 621 Fact 9.7.13, 622 equi-induced Definition 9.4.1, 607 equivalent Theorem 9.1.8, 600 Euclidean definition, 599 Euclidean-norm inequality Fact 9.7.4, 618 Fact 9.7.18, 624 Frobenius definition, 601 H¨ older-norm inequality Fact 9.7.18, 624 idempotent matrix Fact 11.16.12, 762 induced Definition 9.4.1, 607 induced norm Theorem 9.4.2, 607 inequality Fact 9.7.2, 618 Fact 9.7.4, 618 Fact 9.7.10, 621 Fact 9.7.13, 622 Fact 9.7.16, 624 Fact 9.7.17, 624 infinity definition, 599 linear combination of norms Fact 9.7.31, 626 linear system solution Fact 9.15.1, 676 Fact 9.15.2, 677 Fact 9.15.3, 677

1066

norm

Maligranda inequality Fact 9.7.10, 621 Fact 9.7.13, 622 Massera-Schaffer inequality Fact 9.7.10, 621 Fact 9.7.13, 622 matrix Definition 9.2.1, 601 matrix exponential Fact 11.16.9, 761 Fact 11.16.11, 762 Fact 11.16.12, 762 matrix logarithm Theorem 11.5.2, 721 monotone definition, 597 nonsingular matrix Fact 9.7.32, 626 normalized definition, 602 partitioned matrix Fact 9.10.1, 649 Fact 9.10.2, 650 Fact 9.10.8, 652 positive-definite matrix Fact 9.7.30, 626 quadratic form Fact 9.7.30, 626 row definition, 611 self-adjoint definition, 602 set-defined Fact 10.8.22, 694 spectral definition, 603 spectral radius Proposition 9.2.6, 604 submultiplicative definition, 604 trace definition, 603 triangle inequality Definition 9.1.1, 597 unitarily invariant definition, 602

vector Definition 9.1.1, 597 weakly unitarily invariant definition, 602 norm bound matrix exponential Fact 11.18.10, 767 norm equality common eigenvector Fact 9.9.33, 642 Hlawka’s equality Fact 9.7.4, 618 polarization identity Fact 9.7.4, 618 Pythagorean theorem Fact 9.7.4, 618 Schatten norm Fact 9.9.33, 642 norm inequality Aczel’s inequality Fact 9.7.4, 618 Bessel’s inequality Fact 9.7.4, 618 Buzano’s inequality Fact 9.7.4, 618 convex combination Fact 9.7.15, 623 Hlawka’s inequality Fact 9.7.4, 618 H¨ older norm Fact 9.7.21, 625 orthogonal vectors Fact 9.7.25, 626 Parseval’s inequality Fact 9.7.4, 618 polygonal inequalities Fact 9.7.4, 618 quadrilateral inequality Fact 9.7.4, 618 Schatten norm Fact 9.9.34, 642 Fact 9.9.36, 643 Fact 9.9.37, 643 Fact 9.9.38, 643

unitarily invariant norm Fact 9.9.47, 646 Fact 9.9.48, 646 Fact 9.9.49, 646 Fact 9.9.50, 646 vector inequality Fact 9.7.11, 622 Fact 9.7.12, 622 Fact 9.7.14, 623 Fact 9.7.15, 623 von Neumann–Jordan inequality Fact 9.7.11, 622 norm monotonicity power-sum inequality Fact 1.12.30, 39 Fact 1.17.35, 62 norm-compression inequality partitioned matrix Fact 9.10.1, 649 Fact 9.10.8, 652 positive-semidefinite matrix Fact 9.10.6, 652 normal matrix affine mapping Fact 3.7.14, 195 asymptotically stable matrix Fact 11.18.37, 774 block-diagonal matrix Fact 3.7.8, 193 characterizations Fact 3.7.12, 194 commutator Fact 3.8.6, 200 Fact 3.8.7, 200 Fact 9.9.31, 642 commuting matrices Fact 3.7.28, 197 Fact 3.7.29, 198 Fact 5.14.28, 374 Fact 5.17.7, 392

normal matrix Fact 11.14.5, 748 complex conjugate transpose Fact 5.14.29, 374 Fact 6.3.15, 407 Fact 6.3.16, 407 Fact 6.6.11, 434 Fact 6.6.18, 437 congruence transformation Fact 5.10.17, 348 definition Definition 3.1.1, 179 determinant Fact 5.12.12, 365 discrete-time asymptotically stable matrix Fact 11.21.4, 782 discrete-time Lyapunov-stable matrix Fact 11.21.4, 782 dissipative matrix Fact 11.18.37, 774 eigenvalue Fact 5.14.14, 372 eigenvalue perturbation Fact 9.12.8, 658 eigenvector Proposition 4.5.4, 268 Lemma 4.5.3, 268 example Example 5.5.16, 326 Frobenius norm Fact 9.12.9, 658 generalized inverse Proposition 6.1.6, 399 Fact 6.3.15, 407 Fact 6.3.16, 407 group generalized inverse Fact 6.6.11, 434 group-invertible matrix Fact 6.6.18, 437 Hermitian matrix Proposition 3.1.6, 183

idempotent matrix Fact 3.13.3, 224 inertia Fact 5.10.17, 348 involutory matrix Fact 5.9.11, 340 Fact 5.9.12, 340 Jordan form Fact 5.10.6, 346 Kronecker product Fact 7.4.17, 446 Kronecker sum Fact 7.5.8, 451 Lyapunov-stable matrix Fact 11.18.37, 774 matrix exponential Proposition 11.2.8, 713 Fact 11.13.19, 746 Fact 11.14.5, 748 Fact 11.16.10, 762 matrix power Fact 9.13.7, 660 matrix product Fact 9.9.6, 636 maximum singular value Fact 5.14.14, 372 Fact 9.8.13, 629 Fact 9.12.8, 658 Fact 9.13.7, 660 Fact 9.13.8, 660 Fact 9.14.5, 666 Fact 11.16.10, 762 orthogonal eigenvectors Corollary 5.4.8, 321 partitioned matrix Fact 3.12.14, 217 Fact 8.11.12, 517 polar decomposition Fact 11.13.9, 745 positive-semidefinite matrix Fact 8.9.22, 497 Fact 8.10.11, 502 Fact 8.11.12, 517 projector Fact 3.13.3, 224

1067

Fact 3.13.20, 227 Putnam-Fuglede theorem Fact 5.14.29, 374 range-Hermitian matrix Proposition 3.1.6, 183 reflector Fact 5.9.11, 340 Fact 5.9.12, 340 Schatten norm Fact 9.9.27, 641 Fact 9.14.5, 666 Schur decomposition Corollary 5.4.4, 319 Fact 5.10.6, 346 Schur product Fact 9.9.63, 649 semidissipative matrix Fact 11.18.37, 774 semisimple matrix Proposition 5.5.11, 324 shifted-unitary matrix Fact 3.11.26, 209 similar matrices Proposition 5.5.11, 324 Fact 5.9.11, 340 Fact 5.9.12, 340 Fact 5.10.7, 346 similarity transformation Fact 5.15.3, 377 singular value Fact 5.14.14, 372 skew-Hermitian matrix Proposition 3.1.6, 183 spectral decomposition Fact 5.14.13, 372 spectral radius Fact 5.14.14, 372 spectral variation Fact 9.12.5, 657 Fact 9.12.6, 658 spectrum Fact 4.10.25, 295

1068

normal matrix

Fact 8.14.7, 546 Fact 8.14.8, 547 square root Fact 8.9.28, 498 Fact 8.9.29, 498 Fact 8.9.30, 498 trace Fact 3.7.12, 194 Fact 8.12.5, 524 trace of product Fact 5.12.4, 363 transpose Fact 5.9.11, 340 Fact 5.9.12, 340 unitarily invariant norm Fact 9.9.6, 636 unitarily similar matrices Proposition 3.4.5, 189 Corollary 5.4.4, 319 Fact 5.10.6, 346 Fact 5.10.7, 346 unitary matrix Proposition 3.1.6, 183 Fact 3.11.5, 205 Fact 5.15.1, 377 normal rank definition for a polynomial matrix Definition 4.2.4, 257 definition for a rational transfer function Definition 4.7.4, 271 rational transfer function, 307 normalized norm definition, 602 equi-induced norm Theorem 9.4.2, 607 normalized submultiplicative norm inverse matrix Fact 9.8.45, 635 Fact 9.9.56, 647 Fact 9.9.57, 648

Fact 9.9.58, 648 Fact 9.9.59, 648 null space adjugate Fact 2.16.7, 155 definition, 102 Drazin generalized inverse Proposition 6.2.2, 402 equality Fact 2.10.20, 127 generalized inverse Proposition 6.1.6, 399 Fact 6.3.23, 408 group generalized inverse Proposition 6.2.3, 403 group-invertible matrix Fact 3.6.1, 191 idempotent matrix Fact 3.12.3, 215 Fact 3.15.6, 217 Fact 6.3.23, 408 inclusion Fact 2.10.5, 125 Fact 2.10.7, 125 inclusion for a matrix power Corollary 2.4.2, 102 inclusion for a matrix product Lemma 2.4.1, 102 Fact 2.10.2, 124 intersection Fact 2.10.9, 125 involutory matrix Fact 3.15.6, 231 left-equivalent matrices Proposition 5.1.3, 309 Lyapunov equation Fact 12.21.15, 870 matrix sum Fact 2.10.10, 125 minimal polynomial Corollary 11.8.6, 729 nilpotent matrix Fact 3.17.1, 232

Fact 3.17.2, 232 Fact 3.17.3, 232 outer-product matrix Fact 2.10.11, 126 partitioned matrix Fact 2.11.3, 131 positive-semidefinite matrix Fact 8.7.4, 487 Fact 8.7.5, 487 Fact 8.15.2, 550 quadratic form Fact 8.15.2, 550 range Corollary 2.5.6, 105 Fact 2.10.1, 124 range inclusions Theorem 2.4.3, 103 range-Hermitian matrix Fact 3.6.3, 192 semisimple eigenvalue Proposition 5.5.8, 323 skew-Hermitian matrix Fact 8.7.5, 487 symmetric matrix Fact 3.7.4, 193 nullity, see defect nullity theorem defect of a submatrix Fact 2.11.20, 135 partitioned matrix Fact 9.14.11, 667 numerical radius weakly unitarily invariant norm Fact 9.8.38, 633 numerical range spectrum of convex hull Fact 8.14.7, 546 Fact 8.14.8, 547

octahedral group

O oblique projector, see idempotent matrix observability closed-loop spectrum Lemma 12.16.17, 854 PBH test Theorem 12.3.19, 804 Riccati equation Lemma 12.16.18, 854 Sylvester’s equation Fact 12.21.14, 870 observability Gramian asymptotically stable matrix Corollary 12.4.10, 807 H2 norm Corollary 12.11.4, 839 L2 norm Theorem 12.11.1, 838 observably asymptotically stable Proposition 12.4.3, 805 Proposition 12.4.4, 805 Proposition 12.4.5, 806 Proposition 12.4.6, 806 Proposition 12.4.7, 806 observability matrix definition, 800 generalized inverse Fact 12.20.19, 866 Lyapunov equation Fact 12.21.15, 870 observable pair Theorem 12.3.18, 804 Fact 12.20.19, 866 rank Corollary 12.3.3, 801 Sylvester’s equation Fact 12.21.13, 869 observability pencil definition Definition 12.3.12, 803 Smith form

Proposition 12.3.15, 803 Smith zeros Proposition 12.3.16, 803 unobservable eigenvalue Proposition 12.3.13, 803 unobservable spectrum Proposition 12.3.16, 803 observable canonical form definition, 822 equivalent realizations Corollary 12.9.9, 825 realization Proposition 12.9.3, 822 observable dynamics block-triangular matrix Theorem 12.3.8, 802 orthogonal matrix Theorem 12.3.8, 802 observable eigenvalue closed-loop spectrum Lemma 12.16.16, 854 observable subspace Proposition 12.3.17, 804 observable pair asymptotically stable matrix Proposition 12.4.9, 806 Corollary 12.4.10, 807 eigenvalue placement Proposition 12.3.20, 804 equivalent realizations Proposition 12.9.8, 824 invariant zero Corollary 12.10.12, 837

1069

Markov block-Hankel matrix Proposition 12.9.11, 827 minimal realization Proposition 12.9.10, 825 Corollary 12.9.15, 829 observability matrix Theorem 12.3.18, 804 Fact 12.20.19, 866 positive-definite matrix Theorem 12.3.18, 804 observable subspace observable eigenvalue Proposition 12.3.17, 804 observably asymptotically stable asymptotically stable matrix Proposition 12.5.5, 807 block-triangular matrix Proposition 12.4.3, 805 definition Definition 12.4.1, 805 detectability Proposition 12.5.5, 807 Lyapunov equation Proposition 12.4.3, 805 observability Gramian Proposition 12.4.3, 805 Proposition 12.4.4, 805 Proposition 12.4.5, 806 Proposition 12.4.6, 806 Proposition 12.4.7, 806 orthogonal matrix Proposition 12.4.3, 805 output injection Proposition 12.4.2, 805 rank Proposition 12.4.4, 805 octahedral group

1070

octahedral group

group Fact 3.23.4, 243 octonions inequality Fact 1.16.1, 52 real matrix representation Fact 3.24.1, 247 odd permutation matrix definition Definition 3.1.1, 179 odd polynomial asymptotically stable polynomial Fact 11.17.6, 764 definition, 254 off-diagonal entry definition, 87 off-diagonally located block definition, 87 OLHP open left half plane definition, 4 one-sided cone definition, 97 induced by antisymmetric relation Proposition 2.3.6, 101 positive-semidefinite matrix, 459 quadratic form Fact 8.14.14, 548 one-sided directional differential convex function Proposition 10.4.1, 686 definition, 685 example Fact 10.12.5, 702 homogeneity Fact 10.12.4, 702 one-to-one definition, 5

inverse function Theorem 1.4.2, 5 one-to-one and onto function Schroeder-Bernstein theorem Fact 1.7.18, 15 one-to-one function composition of functions Fact 1.7.17, 15 equivalent conditions Fact 1.7.15, 14 finite domain Fact 1.7.13, 14 onto function Fact 1.7.18, 15 one-to-one matrix equivalent properties Theorem 2.6.1, 107 nonsingular equivalence Corollary 2.6.6, 110 ones matrix definition, 92 rank Fact 2.10.18, 127 onto inverse function Theorem 1.4.2, 5 onto function composition of functions Fact 1.7.17, 15 definition, 5 equivalent conditions Fact 1.7.16, 14 finite domain Fact 1.7.13, 14 one-to-one function Fact 1.7.18, 15 onto matrix equivalent properties Theorem 2.6.1, 107 nonsingular equivalence

Corollary 2.6.6, 110 open ball bounded set Fact 10.8.2, 693 completely solid set Fact 10.8.1, 693 convex set Fact 10.8.1, 693 inner product Fact 9.7.24, 625 open ball of radius ε definition, 681 open half space affine open half space Fact 2.9.6, 120 definition, 99 open mapping theorem open set image Theorem 10.3.6, 684 open relative to a set continuous function Theorem 10.3.4, 684 definition Definition 10.1.2, 681 open set complement Fact 10.8.4, 693 continuous function Theorem 10.3.7, 684 Corollary 10.3.5, 684 convex hull Fact 10.8.14, 694 definition Definition 10.1.1, 681 intersection Fact 10.9.10, 696 invariance of domain Theorem 10.3.7, 684 right-invertible matrix Theorem 10.3.6, 684 union Fact 10.9.10, 696 OPP

orthogonal matrix open punctured plane definition, 4 Oppenheim’s inequality determinant inequality Fact 8.22.20, 588 optimal 2-uniform convexity powers Fact 1.12.15, 36 Fact 9.9.35, 643 order definition, 86 Definition 12.9.2, 822 ORHP open right half plane definition, 4 Orlicz H¨ older-induced norm Fact 9.8.18, 630 orthogonal complement definition, 99 intersection Fact 2.9.15, 121 projector Proposition 3.5.2, 190 subspace Proposition 3.5.2, 190 Fact 2.9.16, 121 Fact 2.9.27, 123 sum Fact 2.9.15, 121 orthogonal eigenvectors normal matrix Corollary 5.4.8, 321 orthogonal matrices and matrix exponentials Davenport Fact 11.11.13, 739

orthogonal matrix, see unitary matrix 2×2 parameterization Fact 3.11.27, 209 3 × 3 skew-symmetric matrix Fact 11.11.10, 739 Fact 11.11.11, 739 additive decomposition Fact 5.19.2, 394 algebraic multiplicity Fact 5.11.2, 350 Cayley transform Fact 3.11.22, 208 Fact 3.11.23, 209 Fact 3.11.29, 210 controllable dynamics Theorem 12.6.8, 811 controllable subspace Proposition 12.6.9, 812 controllably asymptotically stable Proposition 12.7.3, 816 convex combination Fact 5.19.3, 394 cross product Fact 3.10.2, 204 Fact 3.10.3, 204 Fact 3.11.29, 210 cross-product matrix Fact 11.11.12, 739 Fact 11.11.13, 739 Fact 11.11.14, 740 definition Definition 3.1.1, 179 detectability Proposition 12.5.3, 807 determinant Fact 3.11.16, 207 Fact 3.11.17, 207 direction cosines Fact 3.11.31, 212 eigenvalue Fact 5.11.2, 350 elementary reflector

1071

Fact 5.15.15, 379 Euler parameters Fact 3.11.31, 212 Fact 3.11.32, 214 existence of transformation Fact 3.9.5, 201 factorization Fact 5.15.15, 379 Fact 5.15.16, 380 Fact 5.15.31, 383 Fact 5.15.35, 384 group Proposition 3.3.6, 187 Hadamard matrix Fact 5.16.9, 391 Hamiltonian matrix Fact 3.20.13, 240 Kronecker permutation matrix Fact 7.4.30, 447 Kronecker product Fact 7.4.17, 446 logarithm Fact 11.15.10, 759 matrix exponential Proposition 11.6.7, 724 Fact 11.11.6, 737 Fact 11.11.7, 738 Fact 11.11.8, 738 Fact 11.11.9, 739 Fact 11.11.10, 739 Fact 11.11.11, 739 Fact 11.11.12, 739 Fact 11.11.13, 739 Fact 11.11.14, 740 Fact 11.15.10, 759 observable dynamics Theorem 12.3.8, 802 observably asymptotically stable Proposition 12.4.3, 805 orthosymplectic matrix Fact 3.20.13, 240 parameterization Fact 3.11.30, 212 Fact 3.11.31, 212

1072

orthogonal matrix

partitioned matrix Fact 3.11.10, 206 permutation matrix Proposition 3.1.6, 183 quaternions Fact 3.11.31, 212 reflector Fact 3.11.30, 212 Fact 5.15.31, 383 Fact 5.15.35, 384 Rodrigues Fact 3.11.31, 212 Rodrigues’s formulas Fact 3.11.32, 214 rotation matrix Fact 3.11.29, 210 Fact 3.11.30, 212 Fact 3.11.31, 212 Fact 3.11.32, 214 Fact 3.11.33, 215 skew-symmetric matrix Fact 3.11.22, 208 Fact 3.11.23, 209 SO(3) Fact 3.11.28, 210 square root Fact 8.9.27, 498 stabilizability Proposition 12.8.3, 820 subspace Fact 3.11.1, 205 Fact 3.11.2, 205 trace Fact 3.11.12, 206 Fact 3.11.13, 206 Fact 5.12.9, 364 Fact 5.12.10, 365 unobservable subspace Proposition 12.3.9, 802 orthogonal projector, see projector orthogonal vectors norm inequality Fact 9.7.25, 626 unitary matrix Fact 3.11.7, 205

vector sum and difference Fact 2.12.2, 136 orthogonality single complex matrix Lemma 2.2.4, 95 single complex vector Lemma 2.2.2, 93 single real matrix Lemma 2.2.3, 95 single real vector Lemma 2.2.1, 93 orthogonality of complex matrices definition, 96 orthogonality of complex vectors definition, 93 orthogonality of real matrices definition, 95 orthogonality of real vectors definition, 93 orthogonally complementary subspaces definition, 99 orthogonal complement Proposition 2.3.3, 99 orthogonally similar matrices definition Definition 3.4.4, 188 diagonal matrix Fact 5.9.17, 341 skew-symmetric matrix Fact 5.14.32, 375 symmetric matrix Fact 5.9.17, 341

upper block-triangular matrix Corollary 5.4.2, 319 upper triangular matrix Corollary 5.4.3, 319 orthosymplectic matrix group Proposition 3.3.6, 187 Hamiltonian matrix Fact 3.20.13, 240 orthogonal matrix Fact 3.20.13, 240 oscillator companion matrix Fact 5.14.34, 375 definition, 718 Ostrowski inertia of a Hermitian matrix Fact 12.21.5, 867 quantitative form of Sylvester’s law of inertia Fact 5.8.17, 336 Ostrowski-Taussky inequality determinant Fact 8.13.2, 533 OUD open unit disk definition, 4 outbound Laplacian matrix adjacency matrix Theorem 3.2.2, 185 definition Definition 3.2.1, 184 outdegree graph Definition 1.6.3, 10 outdegree matrix adjacency matrix Fact 3.21.2, 240

ovals of Cassini definition Definition 3.2.1, 184 row-stochastic matrix Fact 3.21.2, 240 outer-product matrix algebraic multiplicity Fact 5.14.3, 370 characteristic polynomial Fact 4.9.17, 285 Fact 4.9.19, 286 cross product Fact 3.11.29, 210 defective matrix Fact 5.14.3, 370 definition, 94 Definition 3.1.2, 180 doublet Fact 2.10.24, 128 Fact 2.12.6, 136 equality Fact 2.12.3, 136 Fact 2.12.5, 136 Euclidean norm Fact 9.7.27, 626 existence of transformation Fact 3.9.1, 200 Frobenius norm Fact 9.7.26, 626 generalized inverse Fact 6.3.2, 404 group-invertible matrix Fact 5.14.3, 370 Hermitian matrix Fact 3.7.18, 196 Fact 3.9.2, 200 idempotent matrix Fact 3.7.18, 196 Fact 3.12.6, 216 index of a matrix Fact 5.14.3, 370 Kronecker product Proposition 7.1.8, 441 linear independence Fact 2.12.4, 136 Fact 2.12.8, 137

matrix exponential Fact 11.11.1, 736 matrix power Fact 2.12.7, 136 maximum singular value Fact 5.11.16, 353 Fact 5.11.18, 353 Fact 9.7.26, 626 nilpotent matrix Fact 5.14.3, 370 null space Fact 2.10.11, 126 partitioned matrix Fact 4.9.19, 286 positive-definite matrix Fact 3.9.3, 201 positive-semidefinite matrix Fact 8.9.2, 495 Fact 8.9.3, 495 Fact 8.9.4, 495 Fact 8.15.3, 550 Fact 8.15.4, 550 quadratic form Fact 9.13.3, 660 range Fact 2.10.11, 126 rank Fact 2.10.19, 127 Fact 2.10.24, 128 Fact 3.7.17, 196 Fact 3.12.6, 216 semisimple matrix Fact 5.14.3, 370 singular value Fact 5.11.17, 353 skew-Hermitian matrix Fact 3.7.17, 196 Fact 3.9.4, 201 spectral abscissa Fact 5.11.13, 352 spectral radius Fact 5.11.13, 352 spectrum Fact 4.10.1, 288 Fact 5.11.13, 352

1073

sum Fact 2.10.24, 128 trace Fact 5.14.3, 370 unitarily invariant norm Fact 9.8.40, 634 outer-product perturbation adjugate Fact 2.16.3, 153 determinant Fact 2.16.3, 153 elementary matrix Fact 3.7.19, 196 generalized inverse Fact 6.4.2, 412 Fact 6.4.3, 413 inverse matrix Fact 2.16.3, 153 matrix power Fact 2.12.18, 138 rank Fact 2.10.25, 128 Fact 6.4.2, 412 unitary matrix Fact 3.11.20, 208 output convergence detectability Fact 12.20.2, 864 output equation definition, 797 output feedback characteristic polynomial Fact 12.22.13, 874 determinant Fact 12.22.13, 874 output injection detectability Proposition 12.5.2, 807 observably asymptotically stable Proposition 12.4.2, 805 ovals of Cassini

1074

ovals of Cassini

spectrum bounds Fact 4.10.22, 295 Ozeki’s inequality reversed Cauchy-Schwarz inequality Fact 1.18.23, 71

P parallel affine subspaces definition, 98 parallel interconnection definition, 843 transfer function Proposition 12.13.2, 843 parallel sum definition Fact 8.21.18, 581 parallelepiped volume Fact 2.20.16, 173 Fact 2.20.17, 173 parallelogram area Fact 2.20.17, 173 Fact 9.7.5, 620 bivector Fact 9.7.5, 620 cross product Fact 9.7.5, 620 parallelogram law complex numbers Fact 1.20.2, 75 vector equality Fact 9.7.4, 618 parent Definition 1.6.1, 9 Parker equal diagonal entries by unitary similarity Fact 5.9.19, 341

Parodi polynomial root bound Fact 11.20.9, 780 Parrott’s theorem maximum singular value of a partitioned matrix Fact 9.14.13, 668 Parseval’s inequality norm inequality Fact 9.7.4, 618 Parseval’s theorem Fourier transform Fact 12.22.1, 872 H2 norm Theorem 12.11.3, 839 partial derivative definition, 686 partial isometry generalized inverse Fact 6.3.27, 410 partial ordering definition Definition 1.5.8, 8 generalized L¨ owner ordering Fact 8.20.9, 577 planar case Fact 1.7.7, 13 positive-semidefinite matrix Proposition 8.1.1, 460 rank subtractivity Fact 2.10.32, 129 partition definition, 3 equivalence relation Theorem 1.5.7, 8 integer Fact 5.16.8, 390 partitioned matrix adjugate Fact 2.14.27, 151

characteristic polynomial Fact 4.9.15, 285 Fact 4.9.16, 285 Fact 4.9.18, 286 Fact 4.9.19, 286 Fact 4.9.23, 287 Fact 4.9.24, 287 column norm Fact 9.8.11, 628 complementary subspaces Fact 3.12.33, 223 complex conjugate Fact 2.19.9, 166 complex conjugate transpose Proposition 2.8.1, 115 Fact 6.5.3, 423 complex matrix Fact 2.19.4, 165 Fact 2.19.5, 165 Fact 2.19.6, 165 Fact 2.19.7, 165 Fact 3.11.10, 206 contractive matrix Fact 8.11.24, 520 damping matrix Fact 5.12.21, 368 defect Fact 2.11.3, 131 Fact 2.11.8, 132 Fact 2.11.11, 133 definition, 87 determinant Proposition 2.8.1, 115 Corollary 2.8.5, 116 Lemma 8.2.6, 463 Fact 2.14.2, 144 Fact 2.14.3, 144 Fact 2.14.4, 145 Fact 2.14.5, 145 Fact 2.14.6, 145 Fact 2.14.7, 145 Fact 2.14.9, 145 Fact 2.14.10, 146 Fact 2.14.11, 146 Fact 2.14.13, 147 Fact 2.14.14, 147

partitioned matrix Fact 2.14.15, 147 Fact 2.14.16, 148 Fact 2.14.17, 148 Fact 2.14.18, 148 Fact 2.14.19, 149 Fact 2.14.20, 149 Fact 2.14.21, 149 Fact 2.14.22, 149 Fact 2.14.23, 150 Fact 2.14.24, 150 Fact 2.14.25, 150 Fact 2.14.26, 151 Fact 2.14.28, 151 Fact 2.17.5, 160 Fact 2.19.3, 164 Fact 2.19.9, 166 Fact 5.12.21, 368 Fact 6.5.26, 429 Fact 6.5.27, 430 Fact 6.5.28, 430 Fact 8.13.36, 541 Fact 8.13.37, 541 Fact 8.13.39, 541 Fact 8.13.40, 542 Fact 8.13.41, 542 Fact 8.13.42, 542 Fact 8.13.43, 543 determinant of block 2×2 Proposition 2.8.3, 116 Proposition 2.8.4, 116 discrete-time asymptotically stable matrix Fact 11.21.10, 783 Drazin generalized inverse Fact 6.6.1, 431 Fact 6.6.2, 431 eigenvalue Proposition 5.6.5, 330 Fact 5.12.20, 367 Fact 5.12.21, 368 Fact 5.12.22, 369 eigenvalue perturbation Fact 4.10.28, 296 factorization, 462 Fact 2.14.9, 145

Fact 2.16.2, 153 Fact 2.17.3, 159 Fact 2.17.4, 159 Fact 2.17.5, 160 Fact 6.5.25, 429 Fact 8.11.25, 520 Fact 8.11.26, 521 factorization of block 2×2 Proposition 2.8.3, 116 Proposition 2.8.4, 116 generalized inverse Fact 6.3.29, 410 Fact 6.5.1, 422 Fact 6.5.2, 423 Fact 6.5.3, 423 Fact 6.5.4, 423 Fact 6.5.13, 426 Fact 6.5.17, 427 Fact 6.5.18, 427 Fact 6.5.19, 428 Fact 6.5.20, 428 Fact 6.5.21, 428 Fact 6.5.22, 428 Fact 6.5.23, 429 Fact 6.5.24, 429 Fact 8.21.22, 583 geometric multiplicity Proposition 5.5.13, 324 Hamiltonian matrix Proposition 3.1.7, 184 Fact 3.20.6, 239 Fact 3.20.8, 239 Fact 4.9.23, 287 Fact 5.12.21, 368 Hermitian matrix Fact 3.7.27, 197 Fact 4.10.28, 296 Fact 5.8.19, 337 Fact 5.12.1, 362 Fact 6.5.5, 423 H¨ older-induced norm Fact 9.8.11, 628 idempotent matrix Fact 3.12.14, 217 Fact 3.12.20, 218 Fact 3.12.33, 223

1075

Fact 5.10.22, 349 index of a matrix Fact 5.14.31, 374 Fact 6.6.14, 435 inertia Fact 5.8.19, 337 Fact 5.8.20, 337 Fact 5.8.21, 337 Fact 5.12.1, 362 Fact 6.5.5, 423 inverse matrix Fact 2.16.4, 154 Fact 2.17.2, 159 Fact 2.17.3, 159 Fact 2.17.4, 159 Fact 2.17.5, 160 Fact 2.17.6, 160 Fact 2.17.8, 161 Fact 2.17.9, 161 Fact 5.12.21, 368 inverse of block 2 × 2 Proposition 2.8.7, 117 Corollary 2.8.9, 118 involutory matrix Fact 3.15.7, 231 Kronecker product Fact 7.4.19, 446 Fact 7.4.20, 446 Fact 7.4.25, 447 mass matrix Fact 5.12.21, 368 matricial norm Fact 9.10.1, 649 matrix exponential Fact 11.11.2, 736 Fact 11.14.2, 748 matrix sign function Fact 10.10.3, 699 maximum eigenvalue Fact 5.12.20, 367 maximum singular value Fact 8.18.3, 559 Fact 8.18.14, 563 Fact 8.19.1, 564 Fact 8.19.2, 564 Fact 9.10.1, 649 Fact 9.10.3, 650 Fact 9.10.4, 651

1076

partitioned matrix

Fact 9.10.5, 651 Fact 9.14.12, 668 Fact 9.14.13, 668 Fact 9.14.14, 668 minimal polynomial Fact 4.10.13, 292 minimal-rank equality Fact 6.5.7, 424 minimum eigenvalue Fact 5.12.20, 367 multiplicative equalities, 90 nilpotent matrix Fact 3.12.14, 217 Fact 5.10.23, 349 norm Fact 9.10.1, 649 Fact 9.10.2, 650 Fact 9.10.8, 652 norm-compression inequality Fact 9.10.1, 649 Fact 9.10.8, 652 normal matrix Fact 3.12.14, 217 Fact 8.11.12, 517 null space Fact 2.11.3, 131 orthogonal matrix Fact 3.11.10, 206 outer-product matrix Fact 4.9.19, 286 polynomial Fact 4.10.11, 291 positive-definite matrix Proposition 8.2.4, 462 Proposition 8.2.5, 462 Lemma 8.2.6, 463 Fact 8.9.18, 497 Fact 8.11.1, 514 Fact 8.11.2, 514 Fact 8.11.5, 515 Fact 8.11.8, 516 Fact 8.11.10, 516 Fact 8.11.13, 517 Fact 8.11.29, 522

Fact 8.11.30, 522 Fact 8.13.22, 537 Fact 8.18.14, 563 Fact 8.22.6, 585 Fact 11.21.10, 783 positive-semidefinite matrix Proposition 8.2.3, 462 Proposition 8.2.4, 462 Corollary 8.2.2, 461 Lemma 8.2.1, 461 Lemma 8.2.6, 463 Fact 5.12.22, 369 Fact 8.7.6, 487 Fact 8.7.7, 487 Fact 8.9.18, 497 Fact 8.11.1, 514 Fact 8.11.2, 514 Fact 8.11.5, 515 Fact 8.11.6, 516 Fact 8.11.7, 516 Fact 8.11.8, 516 Fact 8.11.9, 516 Fact 8.11.11, 516 Fact 8.11.12, 517 Fact 8.11.13, 517 Fact 8.11.14, 517 Fact 8.11.15, 517 Fact 8.11.17, 518 Fact 8.11.18, 518 Fact 8.11.19, 518 Fact 8.11.20, 519 Fact 8.11.21, 519 Fact 8.11.30, 522 Fact 8.11.31, 522 Fact 8.12.37, 531 Fact 8.12.41, 532 Fact 8.12.42, 532 Fact 8.12.43, 532 Fact 8.13.22, 537 Fact 8.13.36, 541 Fact 8.13.37, 541 Fact 8.13.39, 541 Fact 8.13.40, 542 Fact 8.13.41, 542 Fact 8.13.42, 542 Fact 8.13.43, 543 Fact 8.15.5, 550 Fact 8.18.14, 563

Fact 8.19.1, 564 Fact 8.19.2, 564 Fact 8.19.29, 573 Fact 8.21.22, 583 Fact 8.22.41, 593 Fact 8.22.42, 593 Fact 8.22.45, 594 Fact 8.22.46, 594 Fact 9.8.33, 632 Fact 9.10.6, 652 Fact 9.10.7, 652 power Fact 2.12.21, 138 product Fact 2.12.22, 138 projector Fact 3.13.12, 225 Fact 3.13.22, 228 Fact 3.13.23, 228 Fact 6.5.13, 426 quadratic form Fact 8.15.6, 550 Fact 8.15.7, 550 range Fact 2.11.1, 130 Fact 2.11.2, 131 Fact 6.5.3, 423 rank Corollary 2.8.5, 116 Fact 2.11.6, 132 Fact 2.11.8, 132 Fact 2.11.9, 132 Fact 2.11.10, 132 Fact 2.11.11, 133 Fact 2.11.12, 133 Fact 2.11.13, 133 Fact 2.11.14, 134 Fact 2.11.15, 134 Fact 2.11.16, 134 Fact 2.11.18, 135 Fact 2.11.19, 135 Fact 2.14.4, 145 Fact 2.14.5, 145 Fact 2.14.11, 146 Fact 2.17.5, 160 Fact 2.17.10, 161 Fact 3.12.20, 218 Fact 3.13.12, 225 Fact 3.13.22, 228

PBH test Fact 5.12.21, 368 Fact 6.3.29, 410 Fact 6.5.6, 423 Fact 6.5.7, 424 Fact 6.5.8, 424 Fact 6.5.9, 425 Fact 6.5.10, 425 Fact 6.5.12, 425 Fact 6.5.13, 426 Fact 6.5.14, 426 Fact 6.5.15, 426 Fact 6.6.2, 431 Fact 8.7.6, 487 Fact 8.7.7, 487 Fact 8.7.8, 488 rank of block 2 × 2 Proposition 2.8.3, 116 Proposition 2.8.4, 116 row norm Fact 9.8.11, 628 Schatten norm Fact 9.10.2, 650 Fact 9.10.3, 650 Fact 9.10.4, 651 Fact 9.10.5, 651 Fact 9.10.6, 652 Fact 9.10.7, 652 Fact 9.10.8, 652 Schur complement Fact 6.5.4, 423 Fact 6.5.5, 423 Fact 6.5.6, 423 Fact 6.5.8, 424 Fact 6.5.12, 425 Fact 6.5.29, 430 Fact 8.22.41, 593 Schur product Fact 8.22.6, 585 Fact 8.22.41, 593 Fact 8.22.42, 593 semicontractive matrix Fact 8.11.6, 516 Fact 8.11.22, 520 similar matrices Fact 5.10.21, 349 Fact 5.10.22, 349 Fact 5.10.23, 349 singular value

Proposition 5.6.5, 330 Fact 9.14.11, 667 Fact 9.14.24, 671 skew-Hermitian matrix Fact 3.7.27, 197 skew-symmetric matrix Fact 3.11.10, 206 spectrum Fact 2.19.3, 164 Fact 4.10.26, 296 Fact 4.10.27, 296 stability Fact 11.18.38, 774 stiffness matrix Fact 5.12.21, 368 Sylvester’s equation Fact 5.10.20, 348 Fact 5.10.21, 349 Fact 6.5.7, 424 symmetric matrix Fact 3.11.10, 206 symplectic matrix Fact 3.20.9, 239 trace Proposition 2.8.1, 115 Fact 8.12.37, 531 Fact 8.12.41, 532 Fact 8.12.42, 532 Fact 8.12.43, 532 Fact 8.12.44, 533 transpose Proposition 2.8.1, 115 unitarily invariant norm Fact 9.8.33, 632 unitarily similar matrices Fact 5.9.25, 342 unitary matrix Fact 3.11.9, 206 Fact 3.11.10, 206 Fact 3.11.19, 207 Fact 8.11.22, 520 Fact 8.11.23, 520 Fact 8.11.24, 520 Fact 9.14.11, 667

1077

partitioned positive-semidefinite matrix determinant Proposition 8.2.3, 462 rank Proposition 8.2.3, 462 partitioned transfer function H2 norm Fact 12.22.16, 875 Fact 12.22.17, 875 realization Proposition 12.13.3, 844 Fact 12.22.7, 873 transfer function Fact 12.22.7, 873 Pascal matrix positive-semidefinite matrix Fact 8.8.5, 491 Vandermonde matrix Fact 5.16.3, 387 path definition Definition 1.6.3, 10 pathwise connected continuous function Fact 10.11.5, 700 definition Definition 10.3.12, 685 group Proposition 11.6.8, 724 Pauli spin matrices quaternions Fact 3.24.6, 250 PBH test controllability Theorem 12.6.19, 815 detectability Theorem 12.5.4, 807 observability Theorem 12.3.19, 804 stabilizability Theorem 12.8.4, 821

1078

Pecaric

Pecaric Euclidean norm inequality Fact 9.7.8, 621 Pedersen trace of a convex function Fact 8.12.34, 531 Peierls-Bogoliubov inequality matrix exponential Fact 11.14.29, 754 pencil definition, 330 deflating subspace Fact 5.13.1, 369 generalized eigenvalue Proposition 5.7.3, 332 Proposition 5.7.4, 333 invariant zero Corollary 12.10.4, 832 Corollary 12.10.5, 832 Corollary 12.10.6, 833 Kronecker canonical form Theorem 5.7.1, 331 Weierstrass canonical form Proposition 5.7.3, 332 Penrose generalized inverse of a matrix sum Fact 6.4.39, 420 period definition Definition 1.6.3, 10 graph Definition 1.6.3, 10 permutation definition, 112 permutation group definition Proposition 3.3.6, 187 group Fact 3.23.4, 243

permutation matrix Birkhoff Fact 3.11.3, 205 cyclic permutation matrix Fact 5.16.8, 390 definition Definition 3.1.1, 179 doubly stochastic matrix Fact 3.9.6, 201 Fact 3.11.3, 205 irreducible matrix Fact 3.22.5, 241 orthogonal matrix Proposition 3.1.6, 183 spectrum Fact 5.16.8, 390 transposition matrix Fact 3.23.2, 242 Perron-Frobenius theorem nonnegative matrix eigenvalues Fact 4.11.4, 298 perturbation asymptotically stable matrix Fact 11.18.16, 768 inverse matrix Fact 9.9.60, 648 irreducible matrix Fact 4.11.5, 300 nonsingular matrix Fact 9.14.18, 669 reducible matrix Fact 4.11.5, 300 perturbed matrix spectrum Fact 4.9.12, 285 Fact 4.10.5, 290 Pesonen simultaneous diagonalization of symmetric matrices Fact 8.17.6, 559 Petrovich

complex inequality Fact 1.20.2, 75 Pfaff’s theorem determinant of a skew-symmetric matrix Fact 4.8.14, 282 Pfaffian skew-symmetric matrix Fact 4.8.14, 282 Pick matrix positive-semidefinite matrix Fact 8.8.17, 494 pigeonhole principle finite set Fact 1.7.14, 14 planar graph Euler characteristic Fact 1.8.7, 16 plane rotation orthogonal matrix Fact 5.15.16, 380 Poincare´ separation theorem eigenvalue inequality Fact 8.18.16, 563 pointed cone definition, 97 induced by reflexive relation Proposition 2.3.6, 101 positive-semidefinite matrix, 459 polar closed set Fact 2.9.4, 119 convex cone Fact 2.9.4, 119 definition, 99 polar cone definition, 178 polar decomposition

polynomial existence Corollary 5.6.4, 330 Frobenius norm Fact 9.9.42, 645 matrix exponential Fact 11.13.9, 745 normal matrix Fact 5.18.8, 394 Fact 11.13.9, 745 uniqueness Fact 5.18.2, 393 Fact 5.18.3, 393 Fact 5.18.4, 393 Fact 5.18.5, 393 Fact 5.18.6, 393 Fact 5.18.7, 394 unitarily invariant norm Fact 9.9.42, 645 unitary matrix Fact 5.18.8, 394 polarization identity complex numbers Fact 1.20.2, 75 norm equality Fact 9.7.4, 618 vector equality Fact 9.7.4, 618 polarized Cayley-Hamilton theorem trace Fact 4.9.3, 283 triple product equality Fact 4.9.4, 283 Fact 4.9.6, 284 pole minimal realization Fact 12.22.2, 872 Fact 12.22.12, 874 rational transfer function Definition 4.7.4, 271 Smith-McMillan form Proposition 4.7.11, 273

pole of a rational function definition Definition 4.7.1, 271 pole of a transfer function definition Definition 4.7.10, 273 Polya’s inequality logarithmic mean Fact 1.12.37, 40 Polya-Szego inequality reversed Cauchy-Schwarz inequality Fact 1.18.21, 71 polygon area Fact 2.20.14, 173 polygonal inequalities Euclidean norm Fact 9.7.4, 618 Fact 9.7.7, 620 polyhedral convex cone definition, 98 polynomial asymptotically stable Definition 11.8.3, 728 Bezout matrix Fact 4.8.6, 277 Fact 4.8.8, 279 bound Fact 11.20.14, 781 continuity of roots Fact 10.11.2, 700 coprime Fact 4.8.3, 276 Fact 4.8.4, 277 Fact 4.8.5, 277 definition, 253 Descartes rule of signs Fact 11.17.1, 763

1079

discrete-time asymptotically stable Definition 11.10.3, 735 discrete-time Lyapunov stable Definition 11.10.3, 735 discrete-time semistable Definition 11.10.3, 735 Fujiwara’s bound Fact 11.20.8, 779 greatest common divisor Fact 4.8.5, 277 interpolation Fact 4.8.11, 281 Kojima’s bound Fact 11.20.8, 779 least common multiple Fact 4.8.3, 276 Lyapunov stable Definition 11.8.3, 728 partitioned matrix Fact 4.10.11, 291 root bound Fact 11.20.4, 778 Fact 11.20.5, 778 Fact 11.20.6, 779 Fact 11.20.7, 779 Fact 11.20.8, 779 Fact 11.20.9, 780 Fact 11.20.10, 781 Fact 11.20.11, 781 root bounds Fact 11.20.12, 781 Fact 11.20.13, 781 roots Fact 4.8.1, 276 Fact 4.8.2, 276 roots of derivative Fact 10.12.1, 701 semistable Definition 11.8.3, 728 spectrum Fact 4.9.27, 288 Fact 4.10.11, 291 Vandermonde matrix

1080

polynomial

Fact 5.16.6, 388 polynomial bound Huygens Fact 11.20.14, 781 Mihet Fact 11.20.14, 781 polynomial coefficients asymptotically stable polynomial Fact 11.17.2, 763 Fact 11.17.3, 764 Fact 11.17.7, 764 Fact 11.17.8, 765 Fact 11.17.10, 765 Fact 11.17.11, 765 Fact 11.17.12, 765 discrete-time asymptotically stable polynomial Fact 11.20.1, 777 Fact 11.20.2, 778 Fact 11.20.3, 778 polynomial division quotient and remainder Lemma 4.1.2, 255 polynomial matrix definition, 256 matrix exponential Proposition 11.2.1, 710 Smith form Proposition 4.3.4, 259 polynomial matrix division linear divisor Corollary 4.2.3, 257 Lemma 4.2.2, 257 polynomial multiplication Toeplitz matrix Fact 4.8.10, 280 polynomial representation commuting matrices Fact 5.14.21, 373 Fact 5.14.22, 373

Fact 5.14.23, 373 inverse matrix Fact 4.8.13, 281 polynomial root maximum singular value bound Fact 9.13.14, 662 minimum singular value bound Fact 9.13.14, 662 polynomial root bound Berwald Fact 11.20.12, 781 Bourbaki Fact 11.20.4, 778 Carmichael Fact 11.20.10, 781 Cauchy Fact 11.20.12, 781 Cohn Fact 11.20.12, 781 Fujii-Kubo Fact 11.20.9, 780 Joyal Fact 11.20.7, 779 Labelle Fact 11.20.7, 779 Linden Fact 11.20.9, 780 Mason Fact 11.20.10, 781 Parodi Fact 11.20.9, 780 Rahman Fact 11.20.7, 779 Walsh Fact 11.20.5, 778 Williams Fact 11.20.11, 781 polynomial root locations Enestrom-Kakeya theorem Fact 11.20.3, 778 polynomial roots Bezout matrix Fact 4.8.9, 280

Newton’s identities Fact 4.8.2, 276 polytope definition, 98 Popoviciu arithmetic-mean– geometric-mean inequality Fact 1.17.29, 60 Popoviciu’s inequality convex function Fact 1.10.6, 24 positive diagonal upper triangular matrix Fact 5.15.5, 377 positive matrix almost nonnegative matrix Fact 11.19.2, 774 definition, 88 Definition 3.1.4, 182 eigenvalue Fact 4.11.21, 306 Kronecker sum Fact 7.5.8, 451 Schur product Fact 7.6.13, 456 Fact 7.6.14, 456 spectral radius Fact 7.6.14, 456 spectrum Fact 5.11.12, 351 unstable matrix Fact 11.18.20, 769 positive vector definition, 86 null space Fact 4.11.16, 305 positive-definite matrix arithmetic mean Fact 8.10.34, 506 arithmetic-mean– geometric-mean inequality Fact 8.13.8, 535

positive-definite matrix asymptotically stable matrix Proposition 11.9.5, 733 Proposition 12.4.9, 806 Corollary 11.9.7, 734 Fact 11.18.21, 769 Cauchy matrix Fact 8.8.16, 494 Fact 12.21.18, 870 Cayley transform Fact 8.9.31, 498 cogredient diagonalization Theorem 8.3.1, 465 Fact 8.17.5, 558 commuting matrices Fact 8.9.41, 500 complex conjugate transpose Fact 8.9.40, 500 complex matrix Fact 3.7.9, 193 congruent matrices Proposition 3.4.5, 189 Corollary 8.1.3, 461 contractive matrix Fact 8.11.13, 517 contragredient diagonalization Theorem 8.3.2, 465 Corollary 8.3.4, 465 controllable pair Theorem 12.6.18, 815 convex function Fact 8.14.17, 549 definition Definition 3.1.1, 179 determinant Proposition 8.4.14, 471 Fact 8.12.1, 523 Fact 8.13.6, 534 Fact 8.13.7, 535 Fact 8.13.8, 535 Fact 8.13.9, 535 Fact 8.13.10, 535 Fact 8.13.13, 536 Fact 8.13.14, 536 Fact 8.13.15, 536 Fact 8.13.16, 536

Fact 8.13.18, 537 Fact 8.13.20, 537 Fact 8.13.22, 537 Fact 8.13.24, 538 diagonalizable matrix Corollary 8.3.3, 465 discrete-time asymptotically stable matrix Proposition 11.10.5, 735 Fact 11.21.10, 783 Fact 11.21.17, 785 Fact 11.21.18, 785 discrete-time Lyapunov-stable matrix Proposition 11.10.6, 735 dissipative matrix Fact 8.18.12, 563 Fact 11.18.21, 769 eigenvalue Fact 8.10.24, 504 Fact 8.15.21, 553 Fact 8.15.30, 555 Fact 8.15.31, 555 Fact 8.19.30, 574 Fact 8.22.22, 589 ellipsoid Fact 3.7.35, 199 equality Fact 8.10.6, 501 Fact 8.10.7, 502 exponential Fact 11.14.26, 753 factorization Fact 5.15.26, 382 Fact 5.18.4, 393 Fact 5.18.5, 393 Fact 5.18.6, 393 Fact 5.18.8, 394 Furuta inequality Fact 8.10.51, 512 Gaussian density Fact 8.16.6, 557 generalized geometric mean

1081

Fact 8.10.45, 510 generalized inverse Proposition 6.1.6, 399 Fact 6.4.9, 414 Fact 6.4.10, 414 geometric mean Fact 8.10.43, 508 Fact 8.10.46, 510 Fact 8.10.47, 511 Fact 8.22.53, 595 group-invertible matrix Fact 8.10.12, 502 Hermitian matrix Fact 5.15.41, 384 Fact 8.10.13, 502 Fact 8.13.7, 535 Hilbert matrix Fact 3.18.4, 234 idempotent matrix Fact 5.15.30, 383 increasing function Fact 8.10.55, 513 inequality Fact 8.9.42, 500 Fact 8.9.43, 500 Fact 8.10.8, 502 Fact 8.10.9, 502 Fact 8.10.19, 503 Fact 8.10.20, 503 Fact 8.10.21, 503 Fact 8.10.22, 504 Fact 8.10.28, 504 Fact 8.10.40, 507 Fact 8.10.49, 511 Fact 8.10.53, 512 Fact 8.11.27, 521 Fact 8.15.22, 553 Fact 8.15.23, 553 Fact 8.20.2, 575 Fact 8.22.44, 593 inertia Fact 5.8.10, 335 inertia matrix Fact 8.9.5, 495 inner-product minimization Fact 8.15.13, 552 integral

1082

positive-definite matrix

Fact 8.16.1, 556 Fact 8.16.2, 556 Fact 8.16.3, 556 Fact 8.16.4, 557 Fact 8.16.5, 557 inverse Fact 8.11.10, 516 inverse matrix Proposition 8.6.6, 476 Lemma 8.6.5, 475 Fact 8.9.17, 497 Fact 8.9.42, 500 Kronecker product Fact 7.4.17, 446 left inverse Fact 3.7.25, 197 limit Fact 8.10.48, 511 Lyapunov equation Fact 12.21.16, 870 Fact 12.21.18, 870 Lyapunov-stable matrix Proposition 11.9.6, 733 Corollary 11.9.7, 734 matrix exponential Proposition 11.2.8, 713 Proposition 11.2.9, 715 Fact 11.14.20, 751 Fact 11.14.22, 752 Fact 11.14.23, 752 Fact 11.15.1, 756 matrix logarithm Proposition 8.6.4, 475 Proposition 11.4.5, 719 Fact 8.9.44, 501 Fact 8.13.8, 535 Fact 8.19.30, 574 Fact 8.20.1, 574 Fact 9.9.55, 647 Fact 11.14.24, 752 Fact 11.14.25, 752 Fact 11.14.26, 753 Fact 11.14.27, 753 matrix power Fact 8.10.41, 507 Fact 8.10.42, 507 matrix product Corollary 8.3.7, 466

matrix sign function Fact 10.10.4, 699 maximum singular value Fact 8.19.8, 566 Fact 8.19.26, 572 norm Fact 9.7.30, 626 observable pair Theorem 12.3.18, 804 outer-product matrix Fact 3.9.3, 201 partitioned matrix Proposition 8.2.4, 462 Proposition 8.2.5, 462 Lemma 8.2.6, 463 Fact 8.9.18, 497 Fact 8.11.1, 514 Fact 8.11.2, 514 Fact 8.11.5, 515 Fact 8.11.8, 516 Fact 8.11.10, 516 Fact 8.11.13, 517 Fact 8.11.29, 522 Fact 8.11.30, 522 Fact 8.13.22, 537 Fact 8.18.14, 563 Fact 8.22.6, 585 Fact 11.21.10, 783 positive-semidefinite matrix Fact 8.8.13, 492 Fact 8.8.14, 493 Fact 8.10.27, 504 Fact 8.12.26, 529 power Fact 8.9.43, 500 Fact 8.10.38, 507 Fact 8.10.39, 507 Fact 8.10.49, 511 power inequality Fact 8.10.52, 512 Fact 8.10.54, 513 product of Gaussian densities Fact 8.16.6, 557 properties of < and ≤

Proposition 8.1.2, 460 quadratic form Fact 8.15.25, 554 Fact 8.15.26, 554 Fact 8.15.27, 554 Fact 8.15.30, 555 Fact 8.15.31, 555 Fact 9.8.37, 632 quadratic form inequality Fact 8.15.5, 550 regularized Tikhonov inverse Fact 8.9.41, 500 Riccati equation Fact 12.23.4, 876 Schur product Fact 8.22.4, 584 Fact 8.22.5, 585 Fact 8.22.6, 585 Fact 8.22.7, 586 Fact 8.22.13, 586 Fact 8.22.14, 587 Fact 8.22.15, 587 Fact 8.22.22, 589 Fact 8.22.35, 591 Fact 8.22.36, 592 Fact 8.22.38, 592 Fact 8.22.40, 592 Fact 8.22.44, 593 Fact 8.22.49, 594 Fact 8.22.51, 595 Fact 8.22.52, 595 Fact 8.22.53, 595 simultaneous diagonalization Fact 8.17.5, 558 skew-Hermitian matrix Fact 8.13.6, 534 Fact 11.18.12, 768 spectral order Fact 8.20.3, 575 spectral radius Fact 8.10.5, 501 Fact 8.19.26, 572 spectrum Proposition 5.5.20, 326

positive-semidefinite matrix strictly convex function Fact 8.14.15, 548 Fact 8.14.16, 549 subdeterminant Proposition 8.2.8, 464 Fact 8.13.18, 537 submatrix Proposition 8.2.8, 464 Fact 8.11.28, 522 Toeplitz matrix Fact 8.13.14, 536 trace Proposition 8.4.14, 471 Fact 8.9.16, 497 Fact 8.10.46, 510 Fact 8.10.47, 511 Fact 8.11.10, 516 Fact 8.12.1, 523 Fact 8.12.2, 523 Fact 8.12.25, 528 Fact 8.12.28, 529 Fact 8.12.39, 532 Fact 8.13.13, 536 Fact 11.14.24, 752 Fact 11.14.25, 752 Fact 11.14.27, 753 tridiagonal matrix Fact 8.8.18, 494 unitarily similar matrices Proposition 3.4.5, 189 Proposition 5.5.22, 327 upper bound Fact 8.10.29, 505 positive-definite matrix inequality generalized Furuta inequality Fact 8.10.53, 512 positive-definite matrix product inequality Fact 8.10.43, 508 Fact 8.10.45, 510 positive-definite solution Riccati equation

Theorem 12.17.2, 855 Proposition 12.19.3, 863 Corollary 12.19.2, 863 positive-semidefinite function positive-semidefinite matrix Fact 8.8.1, 488 positive-semidefinite matrix absolute-value matrix Fact 8.9.1, 495 antisymmetric relation Proposition 8.1.1, 460 average Fact 5.19.5, 395 Brownian motion Fact 8.8.3, 489 Cartesian decomposition Fact 9.9.40, 644 Cauchy matrix Fact 8.8.7, 491 Fact 8.8.9, 492 Fact 12.21.19, 871 Cauchy-Schwarz inequality Fact 8.11.14, 517 Fact 8.11.15, 517 Fact 8.15.9, 551 closed set Fact 10.8.18, 694 cogredient diagonalization Theorem 8.3.5, 466 commuting matrices Fact 8.20.4, 514, 576 completely solid set Fact 10.8.18, 694 complex matrix Fact 3.7.9, 193 congruent matrices Proposition 3.4.5, 189 Corollary 8.1.3, 461

1083

contragredient diagonalization Theorem 8.3.6, 466 Corollary 8.3.8, 466 controllability Fact 12.20.6, 864 convex combination Fact 5.19.6, 395 Fact 8.13.17, 537 convex cone, 459 convex function Fact 8.14.15, 548 Fact 8.21.20, 583 convex set Fact 8.14.2, 544 Fact 8.14.3, 544 Fact 8.14.4, 544 Fact 8.14.5, 545 Fact 8.14.6, 545 copositive matrix Fact 8.15.33, 556 cosines Fact 8.8.15, 493 definition Definition 3.1.1, 179 determinant Corollary 8.4.15, 472 Fact 8.13.12, 536 Fact 8.13.17, 537 Fact 8.13.19, 537 Fact 8.13.21, 537 Fact 8.13.22, 537 Fact 8.13.25, 538 Fact 8.13.30, 539 Fact 8.13.36, 541 Fact 8.13.37, 541 Fact 8.13.39, 541 Fact 8.13.40, 542 Fact 8.13.41, 542 Fact 8.13.42, 542 Fact 8.18.11, 562 Fact 8.19.31, 574 Fact 8.22.8, 586 Fact 8.22.20, 587, 588 Fact 8.22.21, 588 Fact 9.8.39, 633 diagonal entry Fact 8.9.8, 496 Fact 8.9.9, 496

1084

positive-semidefinite matrix

Fact 8.10.16, 503 Fact 8.12.3, 523 discrete-time Lyapunov-stable matrix Fact 11.21.17, 785 Drazin generalized inverse Fact 8.21.2, 577 eigenvalue Fact 8.12.3, 523 Fact 8.15.12, 551 Fact 8.19.6, 565 Fact 8.19.20, 570 Fact 8.19.21, 570 Fact 8.19.23, 571 Fact 8.19.24, 572 Fact 8.19.25, 572 Fact 8.19.28, 573 Fact 8.21.17, 580 Fact 8.22.18, 587 Fact 8.22.21, 588 equality Fact 8.11.16, 517 Fact 8.20.5, 576 Euler totient function Fact 8.8.5, 491 factorization Fact 5.15.22, 381 Fact 5.15.26, 382 Fact 5.18.2, 393 Fact 5.18.3, 393 Fact 5.18.7, 394 Fact 8.9.37, 499 Fact 8.9.38, 500 Fejer’s theorem Fact 8.22.37, 592 Frobenius norm Fact 9.8.39, 633 Fact 9.9.12, 638 Fact 9.9.15, 638 Fact 9.9.27, 641 Furuta inequality Proposition 8.6.7, 476 generalized inverse Proposition 6.1.6, 399 Fact 6.4.5, 413 Fact 8.21.1, 577

Fact 8.21.2, 577 Fact 8.21.3, 578 Fact 8.21.4, 578 Fact 8.21.6, 578 Fact 8.21.7, 578 Fact 8.21.8, 579 Fact 8.21.9, 579 Fact 8.21.10, 579 Fact 8.21.11, 579 Fact 8.21.13, 580 Fact 8.21.15, 580 Fact 8.21.16, 580 Fact 8.21.17, 580 Fact 8.21.18, 581 Fact 8.21.19, 582 Fact 8.21.20, 583 Fact 8.21.22, 583 Fact 8.21.23, 583 geometric mean Fact 8.10.43, 508 group generalized inverse Fact 8.21.1, 577 group-invertible matrix Fact 8.10.12, 502 Hadamard-Fischer inequality Fact 8.13.37, 541 Hermitian matrix Fact 5.15.41, 384 Fact 8.9.11, 496 Fact 8.10.13, 502 H¨ older’s inequality Fact 8.12.11, 525 Fact 8.12.12, 525 Hua’s inequalities Fact 8.11.21, 519 Hua’s matrix equality Fact 8.11.21, 519 idempotent matrix Fact 5.15.30, 383 increasing sequence Proposition 8.6.3, 475 inequality Proposition 8.6.7, 476 Corollary 8.6.8, 476 Corollary 8.6.9, 476

Fact 8.9.10, 496 Fact 8.9.19, 497 Fact 8.9.21, 497 Fact 8.9.39, 500 Fact 8.10.19, 503 Fact 8.10.20, 503 Fact 8.10.21, 503 Fact 8.10.28, 504 Fact 8.10.32, 506 Fact 8.10.33, 506 Fact 8.15.22, 553 Fact 8.15.23, 553 Fact 8.22.44, 593 Fact 9.14.22, 671 inertia Fact 5.8.9, 335 Fact 5.8.10, 335 Fact 12.21.9, 869 integral Proposition 8.6.10, 476 inverse matrix Fact 8.10.37, 507 Kantorovich inequality Fact 8.15.10, 551 kernel function Fact 8.8.1, 488 Fact 8.8.4, 490 Kronecker product Fact 7.4.17, 446 Fact 8.22.16, 587 Fact 8.22.23, 589 Fact 8.22.24, 589 Fact 8.22.25, 589 Fact 8.22.27, 589 Fact 8.22.28, 590 Fact 8.22.30, 590 Fact 8.22.32, 590 Kronecker sum Fact 7.5.8, 451 lattice Fact 8.10.30, 505 Fact 8.10.31, 505 left-equivalent matrices Fact 5.10.19, 348 Lehmer matrix Fact 8.8.5, 491 limit

positive-semidefinite matrix Proposition 8.6.3, 475 Fact 8.10.48, 511 linear combination Fact 8.13.19, 537 log majorization Fact 8.11.9, 516 Lyapunov equation Fact 12.21.15, 870 Fact 12.21.19, 871 matrix exponential Fact 11.14.20, 751 Fact 11.14.35, 755 Fact 11.16.6, 761 Fact 11.16.16, 762 matrix logarithm Fact 9.9.54, 647 matrix power Corollary 8.6.11, 477 Fact 8.9.14, 496 Fact 8.10.36, 507 Fact 8.10.50, 512 Fact 8.12.31, 530 Fact 8.15.14, 552 Fact 8.15.15, 552 Fact 8.15.16, 552 Fact 8.15.17, 552 Fact 9.9.17, 639 matrix product Corollary 8.3.7, 466 maximum eigenvalue Fact 8.19.14, 568 maximum singular value Fact 8.19.1, 564 Fact 8.19.2, 564 Fact 8.19.11, 567 Fact 8.19.12, 568 Fact 8.19.13, 568 Fact 8.19.14, 568 Fact 8.19.15, 569 Fact 8.19.16, 569 Fact 8.19.26, 572 Fact 8.19.27, 572 Fact 8.19.29, 573 Fact 8.19.31, 574 Fact 8.19.32, 574 Fact 8.21.9, 579 Fact 11.16.6, 761 McCarthy inequality

Fact 8.12.30, 529 Minkowski’s inequality Fact 8.12.30, 529 norm-compression inequality Fact 9.10.6, 652 normal matrix Fact 8.9.22, 497 Fact 8.10.11, 502 Fact 8.11.12, 517 null space Fact 8.7.4, 487 Fact 8.7.5, 487 Fact 8.15.2, 550 Fact 8.15.24, 554 one-sided cone, 459 outer-product Fact 8.9.3, 495 outer-product matrix Fact 8.9.2, 495 Fact 8.9.4, 495 Fact 8.15.3, 550 Fact 8.15.4, 550 partial ordering Proposition 8.1.1, 460 Fact 8.20.8, 576 partitioned matrix Proposition 8.2.3, 462 Proposition 8.2.4, 462 Corollary 8.2.2, 461 Lemma 8.2.1, 461 Lemma 8.2.6, 463 Fact 5.12.22, 369 Fact 8.7.6, 487 Fact 8.7.7, 487 Fact 8.9.18, 497 Fact 8.11.1, 514 Fact 8.11.2, 514 Fact 8.11.5, 515 Fact 8.11.6, 516 Fact 8.11.7, 516 Fact 8.11.8, 516 Fact 8.11.9, 516 Fact 8.11.11, 516 Fact 8.11.12, 517 Fact 8.11.13, 517 Fact 8.11.14, 517

1085

Fact 8.11.15, 517 Fact 8.11.17, 518 Fact 8.11.18, 518 Fact 8.11.19, 518 Fact 8.11.20, 519 Fact 8.11.21, 519 Fact 8.11.30, 522 Fact 8.11.31, 522 Fact 8.12.37, 531 Fact 8.12.41, 532 Fact 8.12.42, 532 Fact 8.12.43, 532 Fact 8.13.22, 537 Fact 8.13.36, 541 Fact 8.13.37, 541 Fact 8.13.39, 541 Fact 8.13.40, 542 Fact 8.13.41, 542 Fact 8.13.42, 542 Fact 8.13.43, 543 Fact 8.15.5, 550 Fact 8.18.14, 563 Fact 8.19.1, 564 Fact 8.19.2, 564 Fact 8.19.29, 573 Fact 8.21.22, 583 Fact 8.22.41, 593 Fact 8.22.42, 593 Fact 8.22.45, 594 Fact 8.22.46, 594 Fact 9.8.33, 632 Fact 9.10.6, 652 Fact 9.10.7, 652 Pascal matrix Fact 8.8.5, 491 Pick matrix Fact 8.8.17, 494 pointed cone, 459 positive-definite matrix Fact 8.8.13, 492 Fact 8.8.14, 493 Fact 8.10.27, 504 Fact 8.12.26, 529 positive-semidefinite function Fact 8.8.1, 488 power Fact 8.10.38, 507

1086

positive-semidefinite matrix

Fact 8.10.39, 507 projector Fact 3.13.4, 224 properties of < and ≤ Proposition 8.1.2, 460 quadratic form Fact 8.14.2, 544 Fact 8.14.3, 544 Fact 8.14.4, 544 Fact 8.14.5, 545 Fact 8.14.6, 545 Fact 8.15.2, 550 Fact 8.15.10, 551 Fact 8.15.11, 551 Fact 8.15.12, 551 Fact 8.15.14, 552 Fact 8.15.15, 552 Fact 8.15.16, 552 Fact 8.15.17, 552 Fact 8.15.18, 552 Fact 8.15.19, 553 Fact 8.15.24, 554 quadratic form inequality Fact 8.15.5, 550 Fact 8.15.8, 551 range Theorem 8.6.2, 474 Corollary 8.2.2, 461 Fact 8.7.1, 486 Fact 8.7.2, 486 Fact 8.7.3, 486 Fact 8.7.4, 487 Fact 8.7.5, 487 Fact 8.10.2, 501 Fact 8.21.7, 578 Fact 8.21.8, 579 Fact 8.21.10, 579 Fact 8.21.11, 579 range-Hermitian matrix Fact 8.21.21, 583 rank Fact 5.8.9, 335 Fact 8.7.1, 486 Fact 8.7.4, 487 Fact 8.7.6, 487 Fact 8.7.7, 487

Fact 8.7.8, 488 Fact 8.10.2, 501 Fact 8.10.14, 503 Fact 8.21.11, 579 Fact 8.22.16, 587 rank subtractivity partial ordering Fact 8.20.4, 576 Fact 8.21.7, 578 Fact 8.21.8, 579 real eigenvalues Fact 5.14.12, 371 reflexive relation Proposition 8.1.1, 460 reproducing kernel space Fact 8.8.4, 490 right inverse Fact 3.7.26, 197 Schatten norm Fact 9.9.22, 640 Fact 9.9.39, 644 Fact 9.9.40, 644 Fact 9.10.6, 652 Fact 9.10.7, 652 Schur complement Corollary 8.6.18, 486 Fact 8.11.3, 514 Fact 8.11.4, 515 Fact 8.11.17, 518 Fact 8.11.18, 518 Fact 8.11.19, 518 Fact 8.11.20, 519 Fact 8.11.27, 521 Fact 8.21.19, 582 Fact 8.22.11, 586 Schur inverse Fact 8.22.1, 584 Schur power Fact 8.22.2, 584 Fact 8.22.3, 584 Fact 8.22.26, 589 Schur product Fact 8.22.4, 584 Fact 8.22.7, 586 Fact 8.22.11, 586 Fact 8.22.12, 586 Fact 8.22.14, 587 Fact 8.22.17, 587

Fact 8.22.18, 587 Fact 8.22.21, 588 Fact 8.22.23, 589 Fact 8.22.24, 589 Fact 8.22.33, 590 Fact 8.22.37, 592 Fact 8.22.39, 592 Fact 8.22.41, 593 Fact 8.22.42, 593 Fact 8.22.43, 593 Fact 8.22.44, 593 Fact 8.22.45, 594 Fact 8.22.46, 594 Fact 8.22.47, 594 Fact 8.22.48, 594 semicontractive matrix Fact 8.11.6, 516 Fact 8.11.13, 517 semisimple matrix Corollary 8.3.7, 466 shorted operator Fact 8.21.19, 582 signature Fact 5.8.9, 335 singular value Fact 8.19.7, 566 Fact 9.14.27, 671 singular values Fact 8.11.9, 516 skew-Hermitian matrix Fact 8.9.12, 496 spectral order Fact 8.20.3, 575 spectral radius Fact 8.19.26, 572 Fact 8.21.8, 579 spectrum Proposition 5.5.20, 326 Fact 8.21.16, 580 square root Fact 8.9.6, 496 Fact 8.10.18, 503 Fact 8.10.26, 504 Fact 8.22.30, 590 Fact 9.8.32, 632 stabilizability Fact 12.20.6, 864

positive-semidefinite square root star partial ordering Fact 8.20.7, 576 Fact 8.21.8, 579 structured matrix Fact 8.8.2, 489 Fact 8.8.3, 489 Fact 8.8.4, 490 Fact 8.8.5, 491 Fact 8.8.6, 491 Fact 8.8.7, 491 Fact 8.8.8, 491 Fact 8.8.9, 492 Fact 8.8.10, 492 Fact 8.8.11, 492 Fact 8.8.12, 492 subdeterminant Proposition 8.2.7, 463 Fact 8.13.12, 536 submatrix Proposition 8.2.7, 463 Fact 8.7.8, 488 Fact 8.13.37, 541 submultiplicative norm Fact 9.9.7, 637 Szasz’s inequality Fact 8.13.37, 541 trace Proposition 8.4.13, 471 Fact 2.12.16, 138 Fact 8.12.3, 523 Fact 8.12.9, 524 Fact 8.12.10, 524 Fact 8.12.11, 525 Fact 8.12.12, 525 Fact 8.12.13, 525 Fact 8.12.17, 526 Fact 8.12.18, 526 Fact 8.12.19, 527 Fact 8.12.20, 527 Fact 8.12.21, 527 Fact 8.12.22, 528 Fact 8.12.23, 528 Fact 8.12.24, 528 Fact 8.12.25, 528 Fact 8.12.27, 529 Fact 8.12.29, 529 Fact 8.12.30, 529 Fact 8.12.35, 531

Fact 8.12.36, 531 Fact 8.12.37, 531 Fact 8.12.40, 532 Fact 8.12.41, 532 Fact 8.12.42, 532 Fact 8.12.43, 532 Fact 8.13.21, 537 Fact 8.19.16, 569 Fact 8.19.21, 570 Fact 8.21.3, 578 Fact 8.21.17, 580 trace norm Fact 9.9.15, 638 transitive relation Proposition 8.1.1, 460 triangle inequality Fact 9.9.21, 640 unitarily invariant norm Fact 9.9.7, 637 Fact 9.9.14, 638 Fact 9.9.15, 638 Fact 9.9.16, 639 Fact 9.9.17, 639 Fact 9.9.27, 641 Fact 9.9.46, 645 Fact 9.9.51, 646 Fact 9.9.52, 647 Fact 9.9.53, 647 Fact 9.9.54, 647 Fact 11.16.16, 762 Fact 11.16.17, 763 unitarily left-equivalent matrices Fact 5.10.18, 348 Fact 5.10.19, 348 unitarily right-equivalent matrices Fact 5.10.18, 348 unitarily similar matrices Proposition 3.4.5, 189 Proposition 5.5.22, 327 upper bound Fact 8.10.35, 506 upper triangular matrix

1087

Fact 8.9.38, 500 weak majorization Fact 8.19.6, 565 Young’s inequality Fact 8.12.12, 525 zero matrix Fact 8.10.10, 502 positive-semidefinite matrix determinant Fischer’s inequality Fact 8.13.36, 541 Fact 8.13.37, 541 Minkowski’s determinant theorem Corollary 8.4.15, 472 reverse Fischer inequality Fact 8.13.42, 542 positive-semidefinite matrix inequality Araki Fact 8.12.23, 528 Araki-Lieb-Thirring inequality Fact 8.12.22, 528 positive-semidefinite matrix root definition, 474 positive-semidefinite matrix square root definition, 474 positive-semidefinite solution Riccati equation Theorem 12.17.2, 855 Theorem 12.18.4, 860 Proposition 12.17.1, 855 Proposition 12.19.1, 862 Corollary 12.17.3, 855 Corollary 12.18.8, 862 Corollary 12.19.2, 863 positive-semidefinite square root

1088

positive-semidefinite square root

definition, 474 positivity quadratic form on a subspace Fact 8.15.28, 554 Fact 8.15.29, 555 power adjugate Fact 4.9.8, 284 cyclic matrix Fact 5.14.7, 370 derivative Proposition 10.7.2, 691 discrete-time asymptotically stable matrix Fact 11.21.2, 782 discrete-time dynamics Fact 11.21.3, 782 discrete-time Lyapunov-stable matrix Fact 11.21.12, 784 discrete-time semistable matrix Fact 11.21.2, 782 equalities Fact 7.6.11, 455 group-invertible matrix Fact 3.6.2, 192 Fact 6.6.20, 438 idempotent matrix Fact 3.12.3, 215 inequality Fact 1.11.7, 26 Fact 1.12.12, 35 Fact 1.12.31, 39 Fact 1.17.2, 53 Fact 1.17.4, 53 Fact 1.17.5, 53 Fact 1.17.6, 53 Fact 1.17.7, 53 Fact 1.17.8, 54 Fact 1.17.9, 54 Fact 1.17.11, 54, 55 Fact 1.17.22, 58

Kronecker product Fact 7.4.5, 445 Fact 7.4.11, 445 Fact 7.4.22, 446 Kronecker sum Fact 7.5.1, 450 lower triangular matrix Fact 3.18.7, 235 matrix classes Fact 3.7.32, 198 matrix exponential Fact 11.13.20, 746 maximum singular value Fact 8.19.27, 572 Fact 9.13.7, 660 Fact 9.13.9, 660 Fact 11.21.20, 786 nonnegative matrix Fact 4.11.23, 306 normal matrix Fact 9.13.7, 660 outer-product matrix Fact 2.12.7, 136 positive-definite matrix Fact 8.10.41, 507 Fact 8.10.42, 507 positive-semidefinite matrix Corollary 8.6.11, 477 Fact 8.9.14, 496 Fact 8.10.36, 507 Fact 8.10.50, 512 Fact 9.9.17, 639 scalar inequality Fact 1.11.3, 25 Fact 1.11.4, 26 Fact 1.11.5, 26 Fact 1.11.8, 26 Fact 1.11.9, 26 Fact 1.11.10, 27 Fact 1.12.18, 36 Fact 1.13.5, 43 Schur product Fact 7.6.11, 455 similar matrices

Fact 5.9.3, 339 singular value inequality Fact 9.13.18, 662 Fact 9.13.19, 663 skew-Hermitian matrix Fact 8.9.14, 496 strictly lower triangular matrix Fact 3.18.7, 235 strictly upper triangular matrix Fact 3.18.7, 235 symmetric matrix Fact 3.7.4, 193 trace Fact 2.12.13, 137 Fact 2.12.17, 138 Fact 4.10.23, 295 Fact 4.11.23, 306 Fact 5.11.9, 351 Fact 5.11.10, 351 Fact 8.12.4, 524 Fact 8.12.5, 524 unitarily invariant norm Fact 9.9.17, 639 upper triangular matrix Fact 3.18.7, 235 power difference expansion Fact 2.12.20, 138 power function scalar inequalities Fact 1.12.23, 37 power inequality Lehmer mean Fact 1.12.36, 40 monotonicity Fact 1.12.33, 39 Fact 1.12.34, 39 Fact 1.12.35, 40 Fact 1.12.36, 40 positive-definite matrix Fact 8.10.52, 512

problem Fact 8.10.54, 513 scalar case Fact 1.11.11, 27 Fact 1.11.12, 27 Fact 1.12.42, 42 sum inequality Fact 1.18.28, 73 Fact 1.18.29, 73 two-variable Fact 1.12.21, 36 Fact 1.12.22, 37 power mean monotonicity Fact 1.17.30, 60 power of a positive-semidefinite matrix Bessis-MoussaVillani trace conjecture Fact 8.12.31, 530 power-sum inequality H¨ older norm Fact 1.17.35, 62 norm monotonicity Fact 1.12.30, 39 Fact 1.17.35, 62 Powers Schatten norm for positivesemidefinite matrices Fact 9.9.22, 640 powers Beckner’s two-point inequality Fact 1.12.15, 36 Fact 9.9.35, 643 inequality Fact 1.12.8, 34 Fact 1.12.9, 35 Fact 1.12.10, 35 Fact 1.12.14, 36 Fact 1.12.15, 36 Fact 1.12.16, 36 Fact 9.7.20, 624 Fact 9.9.35, 643

optimal 2-uniform convexity Fact 1.12.15, 36 Fact 9.9.35, 643 primary circulant matrix circulant matrix Fact 5.16.7, 388 cyclic permutation matrix Fact 5.16.7, 388 prime numbers Euler product formula Fact 1.9.11, 23 factorization for π Fact 1.9.11, 23 primitive matrix definition Fact 4.11.4, 298 row-stochastic matrix Fact 11.21.11, 784 principal angle gap topology Fact 10.9.19, 698 subspace Fact 2.9.19, 122 Fact 5.11.39, 358 Fact 5.12.17, 366 Fact 10.9.19, 698 principal angle and subspaces Ljance Fact 5.11.39, 358 principal branch logarithm function Fact 1.20.7, 79 principal logarithm definition Definition 11.5.1, 721 principal square root definition, 690 integral formula Fact 10.10.1, 698

1089

square root Theorem 10.6.1, 690 principal submatrix definition, 87 problem absolute value inequality Fact 1.13.1, 42 Fact 1.13.12, 47 Fact 1.14.3, 51 adjoint norm Fact 9.8.8, 627 adjugate of a normal matrix Fact 3.7.10, 193 asymptotic stability of a compartmental matrix Fact 11.19.6, 776 bialternate product and compound matrix Fact 7.5.17, 452 Cayley transform of a Lyapunov-stable matrix Fact 11.21.9, 783 commutator realization Fact 3.8.2, 199 commuting projectors Fact 3.13.20, 227 convergence of the Baker-CampbellHausdorff series Fact 11.14.6, 749 convergent sequence for the generalized inverse Fact 6.3.34, 411 cross product of complex vectors Fact 3.10.1, 202 determinant lower bound Fact 8.13.32, 539

1090

problem

determinant of a partitioned matrix Fact 2.14.13, 147 determinant of the geometric mean Fact 8.22.20, 587, 588 diagonalization of the cross-product matrix Fact 5.9.2, 338 dimension of the centralizer Fact 7.5.2, 450 discrete-time asymptotic stability Fact 11.20.1, 777 discrete-time Lyapunov-stable matrix and the matrix exponential Fact 11.21.4, 782 entries of an orthogonal matrix Fact 3.11.30, 212 equality in the triangle inequality Fact 9.7.3, 618 exponential representation of a discrete-time Lyapunov-stable matrix Fact 11.21.8, 783 factorization of a nonsingular matrix by elementary matrices Fact 5.15.12, 379 factorization of a partitioned matrix Fact 6.5.25, 429 factorization of a unitary matrix Fact 5.15.16, 380 factorization of an orthogonal matrix by reflectors Fact 5.15.31, 383

Frobenius norm lower bound Fact 9.9.11, 637 Fact 9.9.15, 638 generalized inverse least squares solution Fact 9.15.6, 678 generalized inverse of a partitioned matrix Fact 6.5.24, 429 geometric mean and generalized inverses Fact 8.10.43, 508 Hahn-Banach theorem interpretation Fact 10.9.13, 696 H¨ older-induced norm inequality Fact 9.8.21, 630 Hurwitz stability test Fact 11.18.23, 770 inequalities involving the trace and Frobenius norm Fact 9.11.3, 654 inverse image of a subspace Fact 2.9.30, 124 inverse matrix Fact 2.17.8, 161 Kronecker product of positivesemidefinite matrices Fact 8.22.23, 589 least squares and unitary biequivalence Fact 9.15.10, 679 Lie algebra of upper triangular Lie groups Fact 11.22.1, 786

lower bounds for the difference of complex numbers Fact 1.20.2, 75 Lyapunov-stable matrix and the matrix exponential Fact 11.18.37, 774 majorization and singular values Fact 8.18.5, 560 matrix exponential and proper rotation Fact 11.11.7, 738 Fact 11.11.8, 738 Fact 11.11.9, 739 matrix exponential formula Fact 11.14.34, 755 maximum eigenvalue of the difference of positivesemidefinite matrices Fact 8.19.14, 568 maximum singular value of an idempotent matrix Fact 5.11.38, 358 modification of a positivesemidefinite matrix Fact 8.8.14, 493 orthogonal complement Fact 2.9.15, 121 orthogonal matrix Fact 3.9.5, 201 polar decomposition of a matrix exponential Fact 11.13.9, 745 Popoviciu’s inequality and Hlawka’s inequality Fact 1.10.6, 24 positive-definite matrix

product equality Fact 8.8.9, 492 positive-semidefinite matrix trace upper bound Fact 8.12.22, 528 power inequality Fact 1.11.2, 25 Fact 1.12.42, 42 Fact 1.17.7, 53 quadrilateral with an inscribed circle Fact 2.20.13, 172 rank of a positivesemidefinite matrix Fact 8.8.4, 490 reflector Fact 3.14.7, 230 reverse triangle inequality Fact 9.7.6, 620 simisimple imaginary eigenvalues of a partitioned matrix Fact 5.12.21, 368 singular value of a partitioned matrix Fact 9.14.14, 668 singular values of a normal matrix Fact 9.11.2, 653 special orthogonal group and matrix exponentials Fact 11.11.13, 739 spectrum of a partitioned positivesemidefinite matrix Fact 5.12.22, 369 spectrum of a sum of outer products Fact 5.11.13, 352 spectrum of the Laplacian matrix Fact 4.11.7, 300 sum of commutators Fact 2.18.12, 163

trace of a positive-definite matrix Fact 8.12.28, 529 upper bounds for the trace of a product of matrix exponentials Fact 11.16.4, 760 product adjugate Fact 2.16.10, 155 characteristic polynomial Corollary 4.4.11, 267 compound matrix Fact 7.5.17, 452 Drazin generalized inverse Fact 6.6.3, 431 Fact 6.6.4, 431 equalities Fact 2.12.19, 138 generalized inverse Fact 6.4.6, 413 Fact 6.4.7, 413 Fact 6.4.11, 414 Fact 6.4.12, 414 Fact 6.4.14, 414 Fact 6.4.15, 415 Fact 6.4.16, 415 Fact 6.4.17, 415 Fact 6.4.19, 416 Fact 6.4.20, 416 Fact 6.4.24, 417 Fact 6.4.25, 417 Fact 6.4.26, 417 Fact 6.4.33, 418 idempotent matrix Fact 3.12.29, 221 induced lower bound Proposition 9.5.3, 613 left inverse Fact 2.15.5, 153 left-invertible matrix Fact 2.10.3, 125 maximum singular value Fact 9.14.2, 665

1091

positive-definite matrix Corollary 8.3.7, 466 positive-semidefinite matrix Corollary 8.3.7, 466 projector Fact 3.13.18, 226 Fact 3.13.20, 227 Fact 3.13.21, 227 Fact 6.4.19, 416 Fact 6.4.20, 416 Fact 6.4.24, 417 Fact 6.4.26, 417 Fact 8.10.23, 504 quadruple Fact 2.16.11, 155 rank Fact 3.7.30, 198 right inverse Fact 2.15.6, 153 right-invertible matrix Fact 2.10.3, 125 singular value Proposition 9.6.1, 615 Proposition 9.6.2, 615 Proposition 9.6.3, 616 Proposition 9.6.4, 616 Fact 8.19.22, 571 Fact 9.14.26, 671 singular value inequality Fact 9.13.16, 662 Fact 9.13.17, 662 skew-symmetric matrix Fact 5.15.37, 384 trace Fact 5.12.6, 364 Fact 5.12.7, 364 Fact 8.12.14, 525 Fact 8.12.15, 526 Fact 9.14.3, 665 Fact 9.14.4, 665 vec Fact 7.4.7, 445 product equality Lagrange identity

1092

product equality

Fact 1.18.8, 67 product of Gaussian densities positive-definite matrix Fact 8.16.6, 557 product of matrices definition, 89 product of projectors Crimmins Fact 6.3.31, 410 product of sums inequality Fact 1.18.10, 68 projection of a set onto a subspace definition, 190 projector commutator Fact 3.13.23, 228 Fact 6.4.18, 415 Fact 9.9.9, 637 commuting matrices Fact 6.4.36, 419 Fact 8.10.23, 504 Fact 8.10.25, 504 complementary subspaces Fact 3.13.24, 228 complex conjugate transpose Fact 3.13.1, 223 controllable subspace Lemma 12.6.6, 810 definition Definition 3.1.1, 179 difference Fact 3.13.24, 228 Fact 5.12.17, 366 Fact 6.4.36, 419 elementary reflector Fact 5.15.13, 379 equality Fact 3.13.9, 225 Euclidean norm Fact 9.8.2, 627

Fact 9.8.3, 627 Fact 10.9.18, 698 factorization Fact 5.15.13, 379 Fact 5.15.17, 380 Fact 6.3.31, 410 generalized inverse Fact 6.3.3, 404 Fact 6.3.4, 404 Fact 6.3.5, 404 Fact 6.3.25, 409 Fact 6.3.26, 409 Fact 6.3.31, 410 Fact 6.4.18, 415 Fact 6.4.19, 416 Fact 6.4.20, 416 Fact 6.4.21, 416 Fact 6.4.22, 416 Fact 6.4.24, 417 Fact 6.4.26, 417 Fact 6.4.27, 417 Fact 6.4.28, 417 Fact 6.4.36, 419 Fact 6.4.46, 421 Fact 6.4.51, 422 Fact 6.5.10, 425 greatest lower bound Fact 6.4.46, 421 group-invertible matrix Fact 3.13.21, 227 Hermitian matrix Fact 3.13.2, 224 Fact 3.13.13, 225 Fact 3.13.20, 227 Fact 5.15.17, 380 Fact 8.9.24, 497 idempotent matrix Fact 3.13.3, 224 Fact 3.13.13, 225 Fact 3.13.20, 227 Fact 3.13.24, 228 Fact 5.10.13, 347 Fact 5.12.18, 367 Fact 6.3.25, 409 Fact 6.4.21, 416 Fact 6.4.22, 416 Fact 6.4.23, 416 Fact 6.4.28, 417

inequality Fact 8.9.23, 497 intersection of ranges Fact 6.4.46, 421 Kronecker permutation matrix Fact 7.4.31, 449 Kronecker product Fact 7.4.17, 446 least upper bound Fact 6.4.46, 422 matrix difference Fact 3.13.24, 228 Fact 6.4.23, 416 matrix limit Fact 6.4.46, 421 Fact 6.4.51, 422 matrix product Fact 3.13.18, 226 Fact 3.13.20, 227 Fact 3.13.21, 227 Fact 6.4.19, 416 Fact 6.4.20, 416 Fact 6.4.24, 417 Fact 6.4.26, 417 matrix sum Fact 5.19.4, 394 maximum singular value Fact 5.11.38, 358 Fact 5.12.17, 366 Fact 5.12.18, 367 Fact 9.14.1, 665 Fact 9.14.30, 673 normal matrix Fact 3.13.3, 224 Fact 3.13.20, 227 onto a subspace definition, 190 orthogonal complement Proposition 3.5.2, 190 partitioned matrix Fact 3.13.12, 225 Fact 3.13.22, 228 Fact 3.13.23, 228 Fact 6.5.13, 426

quadratic positive-semidefinite matrix Fact 3.13.4, 224 product Fact 3.13.24, 228 Fact 5.12.16, 366 Fact 6.4.22, 416 Fact 8.10.23, 504 quadratic form Fact 3.13.10, 225 Fact 3.13.11, 225 range Proposition 3.5.1, 190 Fact 3.13.5, 224 Fact 3.13.14, 226 Fact 3.13.15, 226 Fact 3.13.17, 226 Fact 3.13.18, 226 Fact 3.13.19, 227 Fact 3.13.20, 227 Fact 6.4.46, 421 Fact 6.4.50, 422 Fact 6.4.51, 422 range-Hermitian matrix Fact 3.13.3, 224 Fact 3.13.20, 227 rank Fact 3.13.9, 225 Fact 3.13.12, 225 Fact 3.13.22, 228 Fact 3.13.23, 228 Fact 5.12.17, 366 Fact 8.12.38, 532 reflector Fact 3.13.16, 226 Fact 3.14.1, 229 right inverse Fact 3.13.6, 224 similar matrices Corollary 5.5.21, 327 Fact 5.10.13, 347 simultaneous triangularization Fact 5.17.6, 392 skew-Hermitian matrix Fact 9.9.9, 637 spectrum

Proposition 5.5.20, 326 Fact 5.12.15, 365 Fact 5.12.16, 366 square root Fact 8.10.25, 504 subspace Proposition 3.5.2, 190 Fact 9.8.3, 627 Fact 10.9.18, 698 sum Fact 3.13.23, 228 Fact 3.13.25, 229 Fact 5.12.17, 366 trace Fact 5.8.11, 335 Fact 8.12.38, 532 tripotent matrix Fact 6.4.36, 419 union of ranges Fact 6.4.46, 422 unitarily similar matrices Fact 5.10.12, 347 unobservable subspace Lemma 12.3.6, 801 projector onto a subspace definition, 190 proper rational function definition Definition 4.7.1, 271 proper rational transfer function definition Definition 4.7.2, 271 realization Theorem 12.9.4, 824 proper rotation matrix exponential Fact 11.11.7, 738 Fact 11.11.8, 738 Fact 11.11.9, 739 proper separation theorem convex sets Fact 10.9.15, 697

1093

proper subset definition, 3 proposition definition, 1 Ptak maximum singular value Fact 9.13.9, 660 Ptolemy’s inequality quadrilateral Fact 2.20.13, 172 Ptolemy’s theorem quadrilateral Fact 2.20.13, 172 Purves similar matrices and nonzero diagonal entry Fact 5.9.16, 341 Putnam-Fuglede theorem normal matrix Fact 5.14.29, 374 Pythagorean theorem norm equality Fact 9.7.4, 618 vector equality Fact 9.7.4, 618 Pythagorean triples quadratic equality Fact 1.12.1, 33

Q QR decomposition existence Fact 5.15.9, 378 quadratic equality Fact 1.13.2, inequality Fact 1.12.4, Fact 1.12.5, Fact 1.12.6,

43 34 34 34

1094

quadratic

Fact 1.12.7, 34 Fact 1.13.3, 43 Fact 1.13.4, 43 quadratic form cone Fact 8.14.11, 547 Fact 8.14.13, 548 Fact 8.14.14, 548 convex cone Fact 8.14.11, 547 Fact 8.14.13, 548 Fact 8.14.14, 548 convex set Fact 8.14.2, 544 Fact 8.14.3, 544 Fact 8.14.4, 544 Fact 8.14.5, 545 Fact 8.14.6, 545 Fact 8.14.9, 547 Fact 8.14.11, 547 Fact 8.14.12, 548 Fact 8.14.13, 548 Fact 8.14.14, 548 copositive matrix Fact 8.15.33, 556 definition, 180 dual norm Fact 9.8.34, 632 eigenvalue Lemma 8.4.3, 467 Fact 8.15.21, 553 field Fact 3.7.7, 193 Hermitian matrix Fact 3.7.6, 193 Fact 3.7.7, 193 Fact 8.15.25, 554 Fact 8.15.26, 554 Fact 8.15.27, 554 Fact 8.15.32, 555 hidden convexity Fact 8.14.11, 547 H¨ older-induced norm Fact 9.8.35, 632 Fact 9.8.36, 632 idempotent matrix Fact 3.13.11, 225 induced norm

Fact 9.8.34, 632 inequality Fact 8.15.8, 551 Fact 8.15.9, 551 Fact 8.15.14, 552 Fact 8.15.15, 552 Fact 8.15.16, 552 Fact 8.15.17, 552 Fact 8.15.19, 553 Fact 8.15.20, 553 Fact 8.15.22, 553 Fact 8.15.23, 553 integral Fact 8.16.3, 556 Fact 8.16.4, 557 Fact 8.16.5, 557 Kantorovich inequality Fact 8.15.10, 551 Laplacian matrix Fact 8.15.1, 550 linear constraint Fact 8.14.10, 547 matrix logarithm Fact 8.15.16, 552 maximum eigenvalue Lemma 8.4.3, 467 maximum singular value Fact 9.13.1, 659 Fact 9.13.2, 660 minimum eigenvalue Lemma 8.4.3, 467 minimum singular value Fact 9.13.1, 659 norm Fact 9.7.30, 626 null space Fact 8.15.2, 550 Fact 8.15.24, 554 one-sided cone Fact 8.14.14, 548 outer-product matrix Fact 9.13.3, 660 partitioned matrix Fact 8.15.6, 550 Fact 8.15.7, 550

positive-definite matrix Fact 8.15.25, 554 Fact 8.15.26, 554 Fact 8.15.27, 554 Fact 8.15.30, 555 Fact 8.15.31, 555 Fact 9.8.37, 632 positive-definite matrix inequality Fact 8.15.5, 550 positive-semidefinite matrix Fact 8.14.2, 544 Fact 8.14.3, 544 Fact 8.14.4, 544 Fact 8.14.5, 545 Fact 8.14.6, 545 Fact 8.15.2, 550 Fact 8.15.10, 551 Fact 8.15.11, 551 Fact 8.15.12, 551 Fact 8.15.14, 552 Fact 8.15.15, 552 Fact 8.15.16, 552 Fact 8.15.17, 552 Fact 8.15.18, 552 Fact 8.15.19, 553 positive-semidefinite matrix inequality Fact 8.15.5, 550 Fact 8.15.8, 551 projector Fact 3.13.10, 225 Fact 3.13.11, 225 quadratic minimization lemma Fact 8.14.15, 548 Rayleigh quotient Lemma 8.4.3, 467 Reid’s inequality Fact 8.15.19, 552 skew-Hermitian matrix Fact 3.7.6, 193 skew-symmetric matrix Fact 3.7.5, 193

Rahman spectrum Fact 8.14.7, 546 Fact 8.14.8, 547 subspace Fact 8.15.28, 554 Fact 8.15.29, 555 symmetric matrix Fact 3.7.5, 193 vector derivative Proposition 10.7.1, 691 quadratic form inequality Marcus Fact 8.15.20, 553 quadratic form on a subspace positivity Fact 8.15.28, 554 Fact 8.15.29, 555 quadratic formula complex numbers Fact 1.20.3, 77 quadratic inequality Aczel’s inequality Fact 1.18.19, 70 sum Fact 1.12.17, 36 sum of squares Fact 1.14.4, 51 Fact 1.16.1, 52 quadratic matrix equation spectrum Fact 5.11.3, 350 Fact 5.11.4, 350 quadratic minimization lemma quadratic form Fact 8.14.15, 548 quadratic performance measure definition, 848 H2 norm Proposition 12.15.1, 849

quadrilateral Brahmagupta’s formula Fact 2.20.13, 172 Ptolemy’s inequality Fact 2.20.13, 172 Ptolemy’s theorem Fact 2.20.13, 172 semiperimeter Fact 2.20.13, 172 quadrilateral inequality Euclidean norm Fact 9.7.4, 618 quadruple product trace Fact 7.4.10, 445 vec Fact 7.4.10, 445

complex matrix representation Fact 3.24.7, 251 inequality Fact 1.16.1, 52 matrix exponential Fact 11.11.16, 740 orthogonal matrix Fact 3.11.31, 212 Pauli spin matrices Fact 3.24.6, 250 real matrix representation Fact 3.24.1, 247 Fact 3.24.8, 251 Rodrigues’s formulas Fact 3.11.32, 214 unitary matrix Fact 3.24.9, 252

quantum information matrix logarithm Fact 11.14.27, 753

quintic inequality Fact 1.12.11, 35

quartic arithmetic-mean– geometric-mean inequality Fact 1.14.5, 51 equality Fact 1.12.3, 33 inequality Fact 1.19.1, 74

quintic polynomial Abel Fact 3.23.4, 243 Galois Fact 3.23.4, 243

quaternion group symplectic group Fact 3.24.4, 249 quaternions 2 × 2 matrix representation Fact 3.24.6, 250 4 × 4 matrix representation Fact 3.24.3, 249 angular velocity vector Fact 11.11.16, 740 complex decomposition Fact 3.24.2, 249

1095

quotient definition, 255

R Rado arithmetic-mean– geometric-mean inequality Fact 1.17.29, 60 convex hull interpretation of strong majorization Fact 3.9.6, 201 Radstrom set cancellation Fact 10.9.8, 696 Rahman

1096

Rahman

polynomial root bound Fact 11.20.7, 779 Ramanujan cube equality Fact 2.12.24, 139 Ramus fundamental triangle inequality Fact 2.20.11, 169 range adjugate Fact 2.16.7, 155 commutator Fact 6.4.18, 415 complex conjugate transpose Fact 6.5.3, 423 Fact 8.7.2, 486 controllability Fact 12.20.7, 864 Drazin generalized inverse Proposition 6.2.2, 402 equality Fact 2.10.8, 125 Fact 2.10.12, 126 Fact 2.10.20, 127 factorization Theorem 8.6.2, 474 generalized inverse Proposition 6.1.6, 399 Fact 6.3.23, 408 Fact 6.4.47, 421 Fact 6.4.48, 421 Fact 6.5.3, 423 group generalized inverse Proposition 6.2.3, 403 group-invertible matrix Fact 3.6.1, 191 Fact 5.14.2, 369 Hermitian matrix Lemma 8.6.1, 474 idempotent matrix Fact 3.12.3, 215 Fact 3.12.4, 216

Fact 3.15.6, 217 Fact 6.3.23, 408 inclusion Fact 2.10.7, 125 Fact 2.10.8, 125 inclusion for a matrix power Corollary 2.4.2, 102 inclusion for a matrix product Lemma 2.4.1, 102 Fact 2.10.2, 124 index of a matrix Fact 5.14.2, 369 involutory matrix Fact 3.15.6, 231 Kronecker product Fact 7.4.23, 447 minimal polynomial Corollary 11.8.6, 729 nilpotent matrix Fact 3.17.1, 232 Fact 3.17.2, 232 Fact 3.17.3, 232 null space Corollary 2.5.6, 105 Fact 2.10.1, 124 null space inclusions Theorem 2.4.3, 103 outer-product matrix Fact 2.10.11, 126 partitioned matrix Fact 2.11.1, 130 Fact 2.11.2, 131 Fact 6.5.3, 423 positive-semidefinite matrix Theorem 8.6.2, 474 Corollary 8.2.2, 461 Fact 8.7.1, 486 Fact 8.7.2, 486 Fact 8.7.3, 486 Fact 8.7.4, 487 Fact 8.7.5, 487 Fact 8.10.2, 501 Fact 8.21.7, 578 Fact 8.21.8, 579 Fact 8.21.10, 579

Fact 8.21.11, 579 projector Proposition 3.5.1, 190 Fact 3.13.14, 226 Fact 3.13.15, 226 Fact 3.13.17, 226 Fact 3.13.18, 226 Fact 3.13.19, 227 Fact 3.13.20, 227 Fact 6.4.46, 421 Fact 6.4.50, 422 Fact 6.4.51, 422 rank Fact 2.11.5, 131 right-equivalent matrices Proposition 5.1.3, 309 Schur product Fact 7.6.9, 455 skew-Hermitian matrix Fact 8.7.5, 487 square root Fact 8.7.2, 486 stabilizability Fact 12.20.7, 864 subspace Fact 2.9.24, 122 symmetric matrix Fact 3.7.4, 193 range of a function definition, 4 range of a matrix definition, 101 range-Hermitian matrix commuting matrices Fact 6.4.29, 417 Fact 6.4.30, 418 complex conjugate transpose Fact 3.6.4, 192 Fact 6.3.10, 405 Fact 6.6.17, 436 congruent matrices Proposition 3.4.5, 189 Fact 5.9.8, 339 definition Definition 3.1.1, 179

rank dissipative matrix Fact 5.14.30, 374 generalized inverse Proposition 6.1.6, 399 Fact 6.3.10, 405 Fact 6.3.11, 406 Fact 6.3.15, 407 Fact 6.3.16, 407 Fact 6.4.13, 414 Fact 6.4.29, 417 Fact 6.4.30, 418 Fact 6.4.31, 418 Fact 6.4.32, 418 generalized projector Fact 3.6.4, 192 group generalized inverse Fact 6.6.9, 434 group-invertible matrix Proposition 3.1.6, 183 Fact 6.6.17, 436 idempotent matrix Fact 3.13.3, 224 Fact 6.3.26, 409 Kronecker product Fact 7.4.17, 446 Kronecker sum Fact 7.5.8, 451 nonsingular matrix Proposition 3.1.6, 183 normal matrix Proposition 3.1.6, 183 null space Fact 3.6.3, 192 positive-semidefinite matrix Fact 8.21.21, 583 product Fact 6.4.32, 418 projector Fact 3.13.3, 224 Fact 3.13.20, 227 rank Fact 3.6.3, 192 Fact 3.6.5, 192 right-equivalent matrices Fact 3.6.3, 192

Schur decomposition Corollary 5.4.4, 319 unitarily similar matrices Proposition 3.4.5, 189 Corollary 5.4.4, 319 rank additivity Fact 2.11.4, 131 Fact 6.4.35, 418 adjugate Fact 2.16.7, 155 Fact 2.16.8, 155 biequivalent matrices Proposition 5.1.3, 309 commutator Fact 3.12.31, 221 Fact 3.13.23, 228 Fact 5.17.5, 392 Fact 6.3.9, 405 complex conjugate transpose Fact 2.10.21, 127 complex matrix Fact 2.19.3, 164 controllability matrix Corollary 12.6.3, 809 controllable pair Fact 5.14.8, 371 controllably asymptotically stable Proposition 12.7.4, 818 Proposition 12.7.5, 819 cyclic matrix Fact 5.11.1, 350 defect Corollary 2.5.5, 105 definition, 104 diagonal dominance Fact 4.10.24, 295 difference Fact 2.10.31, 129 dimension inequality Fact 2.10.4, 125 equalities with defect Corollary 2.5.1, 104 equality

1097

Fact 2.10.12, 126 Fact 2.10.13, 126 Fact 2.10.17, 127 Fact 2.10.20, 127 Fact 2.10.23, 127 factorization Fact 5.15.40, 384 Frobenius norm Fact 9.11.4, 654 Fact 9.14.28, 672 Fact 9.15.8, 678 generalized inverse Fact 6.3.9, 405 Fact 6.3.21, 408 Fact 6.3.35, 411 Fact 6.4.2, 412 Fact 6.4.8, 413 Fact 6.4.35, 418 Fact 6.4.49, 421 Fact 6.5.6, 423 Fact 6.5.8, 424 Fact 6.5.9, 425 Fact 6.5.12, 425 Fact 6.5.13, 426 Fact 6.5.14, 426 geometric multiplicity Proposition 4.5.2, 268 group-invertible matrix Fact 3.6.1, 191 Fact 5.8.5, 334 Fact 5.14.2, 369 Hankel matrix Fact 3.18.8, 235 Hermitian matrix Fact 3.7.22, 197 Fact 3.7.30, 198 Fact 5.8.6, 334 Fact 5.8.7, 335 Fact 8.9.7, 496 idempotent matrix Fact 3.12.6, 216 Fact 3.12.9, 216 Fact 3.12.19, 218 Fact 3.12.20, 218 Fact 3.12.22, 218 Fact 3.12.24, 219 Fact 3.12.26, 220

1098

rank

Fact 3.12.27, 220 Fact 3.12.31, 221 Fact 5.8.1, 334 Fact 5.11.7, 350 inequality Fact 2.10.22, 127 inertia Fact 5.8.5, 334 Fact 5.8.18, 337 inverse Fact 2.11.21, 136 Fact 2.11.22, 136 inverse matrix Fact 2.17.10, 161 Fact 6.5.11, 425 Kronecker product Fact 7.4.24, 447 Fact 7.4.25, 447 Fact 7.4.26, 447 Fact 8.22.16, 587 Kronecker sum Fact 7.5.2, 450 Fact 7.5.9, 451 Fact 7.5.10, 451 linear matrix equation Fact 2.10.16, 126 linear system solution Theorem 2.6.4, 108 Corollary 2.6.7, 110 lower bound for product Proposition 2.5.9, 106 Corollary 2.5.10, 106 M-matrix Fact 8.7.8, 488 matrix difference Fact 2.10.27, 128 Fact 2.10.30, 128 matrix power Fact 2.10.22, 127 matrix powers Corollary 2.5.7, 105 Fact 3.17.5, 232 matrix sum Fact 2.10.27, 128 Fact 2.10.28, 128 Fact 2.10.29, 128

Fact 2.11.4, 131 nilpotent matrix Fact 3.17.4, 232 Fact 3.17.5, 232 nonsingular submatrices Proposition 2.7.7, 115 observability matrix Corollary 12.3.3, 801 observably asymptotically stable Proposition 12.4.4, 805 ones matrix Fact 2.10.18, 127 outer-product matrix Fact 2.10.19, 127 Fact 2.10.24, 128 Fact 3.12.6, 216 outer-product perturbation Fact 2.10.25, 128 Fact 6.4.2, 412 partitioned matrix Corollary 2.8.5, 116 Fact 2.11.7, 132 Fact 2.11.8, 132 Fact 2.11.9, 132 Fact 2.11.10, 132 Fact 2.11.11, 133 Fact 2.11.12, 133 Fact 2.11.13, 133 Fact 2.11.14, 134 Fact 2.11.15, 134 Fact 2.11.16, 134 Fact 2.11.18, 135 Fact 2.11.19, 135 Fact 2.14.4, 145 Fact 2.14.5, 145 Fact 2.14.11, 146 Fact 2.17.5, 160 Fact 2.17.10, 161 Fact 3.12.20, 218 Fact 3.13.12, 225 Fact 3.13.22, 228 Fact 5.12.21, 368 Fact 6.3.29, 410 Fact 6.5.6, 423

Fact 6.5.7, 424 Fact 6.5.8, 424 Fact 6.5.9, 425 Fact 6.5.10, 425 Fact 6.5.12, 425 Fact 6.5.13, 426 Fact 6.5.14, 426 Fact 6.5.15, 426 Fact 6.6.2, 431 Fact 8.7.6, 487 Fact 8.7.7, 487 Fact 8.7.8, 488 partitioned positivesemidefinite matrix Proposition 8.2.3, 462 positive-semidefinite matrix Fact 5.8.9, 335 Fact 8.7.1, 486 Fact 8.7.4, 487 Fact 8.7.6, 487 Fact 8.7.7, 487 Fact 8.7.8, 488 Fact 8.10.2, 501 Fact 8.10.14, 503 Fact 8.21.11, 579 Fact 8.22.16, 587 powers Proposition 2.5.8, 106 product Proposition 2.6.3, 107 Fact 3.7.30, 198 product of matrices Fact 2.10.14, 126 Fact 2.10.26, 128 projector Fact 3.13.9, 225 Fact 3.13.12, 225 Fact 3.13.22, 228 Fact 3.13.23, 228 Fact 5.12.17, 366 Fact 8.12.38, 532 range Fact 2.11.5, 131 range-Hermitian matrix Fact 3.6.3, 192 Fact 3.6.5, 192

rational transfer function rational transfer function Definition 4.7.4, 271 Riccati equation Proposition 12.19.4, 863 Rosenbrock system matrix Proposition 12.10.3, 831 Proposition 12.10.11, 837 Schur complement Proposition 8.2.3, 462 Fact 6.5.6, 423 Fact 6.5.8, 424 Fact 6.5.11, 425 Schur product Fact 7.6.10, 455 Fact 8.22.16, 587 simple matrix Fact 5.11.1, 350 singular value Proposition 5.6.2, 329 Fact 9.14.28, 672 Fact 9.15.8, 678 skew-Hermitian matrix Fact 3.7.17, 196 Fact 3.7.30, 198 Smith form Proposition 4.3.5, 259 Proposition 4.3.6, 260 Smith-McMillan form Proposition 4.7.7, 272 Proposition 4.7.8, 272 subapce dimension theorem Fact 2.11.10, 132 submatrix Proposition 4.3.5, 259 Proposition 4.7.7, 272 Fact 2.11.6, 131 Fact 2.11.17, 134 Fact 2.11.20, 135 Fact 2.11.21, 136 Fact 2.11.22, 136 Fact 3.19.1, 237

subspace dimension theorem Fact 2.11.9, 132 subtractivity Fact 2.10.30, 128 Fact 2.10.31, 129 Sylvester’s equation Fact 12.21.13, 869 totally positive matrix Fact 8.7.8, 488 trace Fact 5.11.10, 351 Fact 9.11.4, 654 transpose Corollary 2.5.3, 105 trigonometric matrix Fact 3.18.8, 235 tripotent matrix Fact 2.10.23, 127 Fact 3.16.3, 231 Fact 3.16.4, 231 unitarily invariant norm Fact 9.14.28, 672 upper bound for product Corollary 2.5.10, 106 upper bound on rank of a product Lemma 2.5.2, 104 upper bound with dimensions Corollary 2.5.4, 105 rank of a polynomial matrix definition Definition 4.2.4, 257 Definition 4.3.3, 259 submatrix Proposition 4.2.7, 258 rank of a rational function linearly independent columns Proposition 4.7.6, 272 Proposition 4.7.9, 273 rank subtractivity

1099

equivalent conditions Fact 2.10.30, 128 transitivity Fact 2.10.31, 129 rank subtractivity partial ordering commuting matrices Fact 8.20.4, 576 definition Fact 2.10.32, 129 generalized inverse Fact 6.5.30, 431 positive-semidefinite matrix Fact 8.20.4, 576 Fact 8.20.8, 576 Fact 8.21.7, 578 Fact 8.21.8, 579 rank-deficient matrix determinant Fact 2.13.4, 140 rank-two matrix matrix exponential Fact 11.11.19, 742 ratio of powers scalar inequalities Fact 1.12.40, 42 rational canonical form, see multicompanion form or elementary multicompanion form rational function complex conjugate Fact 4.8.17, 282 definition Definition 4.7.1, 271 Hankel matrix Fact 4.8.8, 279 imaginary part Fact 4.8.17, 282 spectrum Fact 5.11.15, 353 rational transfer function blocking zero

1100

rational transfer function

Definition 4.7.4, 271 definition Definition 4.7.2, 271 Markov block-Hankel matrix Proposition 12.9.11, 827 Proposition 12.9.12, 828 Proposition 12.9.13, 828 Markov parameter Proposition 12.9.7, 824 minimal realization Fact 12.22.12, 874 normal rank Definition 4.7.4, 271 poles Definition 4.7.4, 271 rank Definition 4.7.4, 271 realization Fact 12.22.11, 873 Rayleigh quotient Hermitian matrix Lemma 8.4.3, 467 quadratic form Lemma 8.4.3, 467 real eigenvalues positive-semidefinite matrix Fact 5.14.12, 371 real hypercompanion form definition, 315 real Jordan form existence Theorem 5.3.5, 316 hypercompanion matrix Fact 5.10.1, 345 Jordan form Fact 5.10.2, 345 similarity transformation Fact 5.10.1, 345

Fact 5.10.2, 345 real Jordan matrix definition, 315 real normal form existence Corollary 5.4.9, 321 real part frequency response Fact 12.22.5, 872 transfer function Fact 12.22.5, 872 real Schur decomposition definition, 319 existence Corollary 5.4.2, 319 Corollary 5.4.3, 319 real symplectic group special orthogonal group Fact 3.24.5, 250 real vector definition, 93 realization controllable canonical form Proposition 12.9.3, 822 definition Definition 12.9.2, 822 derivative Fact 12.22.6, 872 feedback interconnection Proposition 12.13.4, 845 Proposition 12.14.1, 847 Fact 12.22.8, 873 observable canonical form Proposition 12.9.3, 822 partitioned transfer function Proposition 12.13.3, 844 Fact 12.22.7, 873

proper rational transfer function Theorem 12.9.4, 824 rational transfer function Fact 12.22.11, 873 similar matrices Proposition 12.9.5, 824 transfer function Proposition 12.13.1, 842 Fact 12.22.3, 872 Fact 12.22.4, 872 Fact 12.22.6, 872 Fact 12.22.7, 873 Fact 12.22.8, 873 rearrangement inequality Chebyshev’s inequality Fact 1.18.3, 66 product of sums Fact 1.18.4, 66 reverse inequality Fact 1.18.6, 67 sum of differences Fact 1.18.4, 66 sum of products Fact 1.18.4, 66 sum of products inequality Fact 1.18.5, 67 reciprocal scalar equality Fact 1.13.18, 48 scalar inequality Fact 1.13.13, 47 Fact 1.13.19, 48 Fact 1.13.20, 48 reciprocal argument transfer function Fact 12.22.4, 872 reciprocal powers inequality Fact 1.18.26, 72 Fact 1.18.27, 73 reciprocals

relative entropy scalar inequality Fact 1.13.23, 49 Fact 1.13.26, 50 Walker’s inequality Fact 1.13.22, 49 reducible matrix absolute value Fact 3.22.6, 241 definition Definition 3.1.1, 179 perturbation matrix Fact 4.11.5, 300 upper block-triangular matrix Fact 4.11.5, 300 zero entry Fact 3.22.3, 241 Fact 3.22.4, 241 redundant assumptions definition, 2 refined weighted arithmetic-mean– geometric-mean inequality arithmetic-mean– geometric-mean inequality Fact 1.17.33, 62 reflection theorem elementary reflector Fact 3.14.4, 229 reflector definition Definition 3.1.1, 179 elementary reflector Fact 5.15.14, 379 equality Fact 3.14.8, 230 factorization Fact 5.15.14, 379 Hermitian matrix Fact 3.14.2, 229 involutory matrix Fact 3.14.2, 229 Kronecker product

Fact 7.4.17, 446 normal matrix Fact 5.9.11, 340 Fact 5.9.12, 340 orthogonal matrix Fact 3.11.30, 212 Fact 5.15.31, 383 Fact 5.15.35, 384 projector Fact 3.13.16, 226 Fact 3.14.1, 229 rotation matrix Fact 3.11.30, 212 similar matrices Corollary 5.5.21, 327 skew reflector Fact 3.14.7, 230 spectrum Proposition 5.5.20, 326 trace Fact 5.8.11, 335 tripotent matrix Proposition 3.1.6, 183 unitary matrix Fact 3.14.2, 229 reflexive hull definition Definition 1.5.4, 7 relation Proposition 1.5.5, 7 reflexive relation definition Definition 1.5.2, 6 graph Definition 1.6.2, 9 intersection Proposition 1.5.3, 7 pointed cone induced by Proposition 2.3.6, 101 positive-semidefinite matrix Proposition 8.1.1, 460 regular pencil definition, 330 generalized eigenvalue Proposition 5.7.3, 332

1101

Proposition 5.7.4, 333 invariant zero Corollary 12.10.4, 832 Corollary 12.10.5, 832 Corollary 12.10.6, 833 Kronecker canonical form Proposition 5.7.2, 331 Moler Fact 5.17.3, 392 simultaneous triangularization Fact 5.17.2, 391 Stewart Fact 5.17.3, 392 upper Hessenberg matrix Fact 5.17.3, 392 upper triangular matrix Fact 5.17.3, 392 regular polynomial matrix definition, 256 nonsingular polynomial matrix Proposition 4.2.5, 257 regularized Tikhonov inverse positive-definite matrix Fact 8.9.41, 500 Reid’s inequality quadratic form Fact 8.15.19, 552 relation definition, 6 function Proposition 1.5.1, 6 relative complement definition, 2 relative degree definition Definition 4.7.1, 271 Definition 4.7.3, 271 relative entropy

1102

relative entropy

matrix logarithm Fact 11.14.25, 752 relative gain array definition Fact 8.22.4, 584 relatively closed set complement Fact 10.8.5, 693 relatively open set complement Fact 10.8.5, 693 remainder definition, 255 reproducing kernel space positive-semidefinite matrix Fact 8.8.4, 490 resolvent definition, 265 Laplace transform Proposition 11.2.2, 711 matrix exponential Proposition 11.2.2, 711 resultant coprime polynomials Fact 4.8.4, 277 reversal of a graph Definition 1.6.2, 9 reversal of a relation definition Definition 1.5.4, 7 reverse arithmetic-mean– geometric-mean inequality Specht Fact 1.17.19, 57 Specht’s ratio Fact 1.17.19, 57 reverse complex conjugate transpose definition, 96

reverse inequality arithmetic-mean– geometric-mean inequality Fact 1.17.18, 57 Fact 1.17.19, 57 Euclidean norm triangle inequality Fact 9.7.6, 620 Fischer’s inequality Fact 8.13.42, 542 Young inequality Fact 1.12.22, 37

reverse-symmetric matrix definition Definition 3.1.1, 179 factorization Fact 5.9.14, 340 similar matrices Fact 5.9.13, 340 Toeplitz matrix Fact 3.18.5, 234

reverse Minkowski inequality p < 1 case Fact 9.7.19, 624

Riccati differential equation matrix differential equation Fact 12.23.5, 877

reverse permutation matrix cyclic permutation matrix Fact 3.15.5, 230 definition, 91 determinant Fact 2.13.1, 139 involutory matrix Fact 3.15.4, 230 spectrum Fact 5.9.26, 342 symplectic matrix Fact 3.20.3, 238 reverse transpose definition, 96 similar matrices Fact 5.9.13, 340 reverse-diagonal entry definition, 87 reverse-diagonal matrix definition Definition 3.1.3, 181 semisimple matrix Fact 5.14.11, 371 reverse-Hermitian matrix definition Definition 3.1.1, 179

reversed relation relation Proposition 1.5.5, 7

Riccati equation closed-loop spectrum Proposition 12.16.14, 853 Proposition 12.18.2, 860 Proposition 12.18.3, 860 Proposition 12.18.7, 862 detectability Corollary 12.17.3, 855 Corollary 12.19.2, 863 existence Fact 12.23.3, 876 geometric mean Fact 12.23.4, 876 golden mean Fact 12.23.4, 876 golden ratio Fact 12.23.4, 876 Hamiltonian Theorem 12.17.9, 857 Proposition 12.16.14, 853 Corollary 12.16.15, 854 inertia Lemma 12.16.18, 854 linear-quadratic control problem

rigid-body rotation Theorem 12.15.2, 849 maximal solution Definition 12.16.12, 853 Theorem 12.18.1, 859 Theorem 12.18.4, 860 Proposition 12.18.2, 860 Proposition 12.18.7, 862 monotonicity Proposition 12.18.5, 861 Corollary 12.18.6, 861 observability Lemma 12.16.18, 854 positive-definite matrix Fact 12.23.4, 876 positive-definite solution Theorem 12.17.2, 855 Proposition 12.19.3, 863 Corollary 12.19.2, 863 positive-semidefinite solution Theorem 12.17.2, 855 Theorem 12.18.4, 860 Proposition 12.17.1, 855 Proposition 12.19.1, 862 Corollary 12.17.3, 855 Corollary 12.18.8, 862 Corollary 12.19.2, 863 rank Proposition 12.19.4, 863 solution Definition 12.16.12, 853 Fact 12.23.2, 876 stabilizability Theorem 12.17.9, 857 Theorem 12.18.1, 859 Corollary 12.19.2, 863 stabilizing solution

Definition 12.16.12, 853 Theorem 12.17.2, 855 Theorem 12.17.9, 857 Theorem 12.18.4, 860 Proposition 12.17.1, 855 Proposition 12.18.3, 860 Proposition 12.19.4, 863 Corollary 12.16.15, 854 right divides definition, 256 right coprime polynomial matrices Bezout identity Theorem 4.7.14, 274 right equivalence equivalence relation Fact 5.10.3, 345 right inverse (1)-inverse Proposition 6.1.2, 398 definition, 5 generalized inverse Corollary 6.1.4, 398 idempotent matrix Fact 3.12.10, 216 linear system Fact 6.3.1, 404 matrix product Fact 2.15.6, 153 positive-semidefinite matrix Fact 3.7.26, 197 projector Fact 3.13.6, 224 representation Fact 2.15.4, 152 right-inner matrix Fact 3.11.6, 205 transfer function Fact 12.22.9, 873 uniqueness Theorem 1.4.2, 5

1103

right-equivalent matrices definition Definition 3.4.3, 188 group-invertible matrix Fact 3.6.1, 191 Kronecker product Fact 7.4.12, 445 range Proposition 5.1.3, 309 range-Hermitian matrix Fact 3.6.3, 192 right-inner matrix definition Definition 3.1.2, 180 generalized inverse Fact 6.3.8, 405 right inverse Fact 3.11.6, 205 right-invertible function definition, 5 right-invertible matrix definition, 106 equivalent properties Theorem 2.6.1, 107 generalized inverse Proposition 6.1.5, 398 inverse Proposition 2.6.5, 110 linear system solution Fact 2.13.8, 141 matrix product Fact 2.10.3, 125 nonsingular equivalence Corollary 2.6.6, 110 open set Theorem 10.3.6, 684 unique right inverse Proposition 2.6.2, 107 rigid body inertia matrix Fact 8.9.5, 495 rigid-body rotation

1104

rigid-body rotation

matrix exponential Fact 11.11.6, 737 Rodrigues orthogonal matrix Fact 3.11.31, 212 Rodrigues’s formulas Euler parameters Fact 3.11.32, 214 orthogonal matrix Fact 3.11.32, 214 quaternions Fact 3.11.32, 214 ¨ Rogers-Holder inequality scalar case Fact 1.18.12, 68 root Definition 1.6.1, 9 polynomial Fact 4.8.1, 276 Fact 4.8.2, 276 Fact 11.20.4, 778 Fact 11.20.5, 778 Fact 11.20.6, 779 Fact 11.20.7, 779 Fact 11.20.8, 779 Fact 11.20.9, 780 Fact 11.20.10, 781 Fact 11.20.11, 781 root bounds polynomial Fact 11.20.12, 781 Fact 11.20.13, 781 root locus eigenvalue Fact 4.10.29, 296 roots of polynomial convex hull Fact 10.12.1, 701 Rosenbrock system matrix definition Definition 12.10.1, 830 rank

Proposition 12.10.3, 831 Proposition 12.10.11, 837 rotation vector Fact 3.11.34, 215 rotation matrix definition, 187 group Proposition 3.3.6, 187 logarithm Fact 11.15.10, 759 matrix exponential Fact 11.11.13, 739 Fact 11.11.14, 740 orthogonal matrix Fact 3.11.29, 210 Fact 3.11.30, 212 Fact 3.11.31, 212 Fact 3.11.32, 214 Fact 3.11.33, 215 reflector Fact 3.11.30, 212 trace Fact 3.11.12, 206 rotation-dilation factorization Fact 2.19.2, 164 Roth solutions of Sylvester’s equation Fact 5.10.20, 348 Fact 5.10.21, 349 Roup positive-definite matrix Fact 8.8.13, 492 Routh criterion asymptotically stable polynomial Fact 11.17.2, 763 Routh form tridiagonal matrix Fact 11.18.27, 771

row definition, 86 row norm column norm Fact 9.8.10, 628 definition, 611 H¨ older-induced norm Fact 9.8.21, 630 Fact 9.8.23, 631 Kronecker product Fact 9.9.61, 648 partitioned matrix Fact 9.8.11, 628 spectral radius Corollary 9.4.10, 611 row-stochastic matrix adjacency matrix Fact 3.21.2, 240 definition Definition 3.1.4, 182 discrete-time Lyapunov-stable matrix Fact 11.21.11, 784 discrete-time semistable matrix Fact 11.21.11, 784 irreducible matrix Fact 11.21.11, 784 outdegree matrix Fact 3.21.2, 240 primitive matrix Fact 11.21.11, 784 spectral radius Fact 4.11.6, 300 spectrum Fact 4.10.2, 288

S S-N decomposition diagonalizable matrix Fact 5.9.5, 339 nilpotent matrix Fact 5.9.5, 339

Schoenberg scalar inequality arithmetic mean Fact 1.13.6, 43 Bernoulli’s inequality Fact 1.11.1, 25 Cauchy-Schwarz inequality Fact 1.18.9, 68 exponential function Fact 1.11.14, 27 Fact 1.11.15, 27 Fact 1.11.16, 28 Fact 1.11.18, 28 geometric mean Fact 1.13.6, 43 H¨ older’s inequality Fact 1.18.11, 68 Fact 1.18.12, 68 Hua’s inequality Fact 1.17.13, 56 Kantorovich inequality Fact 1.17.37, 63 logarithm Fact 1.17.46, 65 Fact 1.17.47, 65 Fact 1.17.48, 65 Minkowski’s inequality Fact 1.18.25, 72 rearrangement inequality Fact 1.18.7, 67 reciprocal powers Fact 1.18.26, 72 Fact 1.18.27, 73 reversal of H¨ older’s inequality Fact 1.18.22, 71 Rogers-H¨ older inequality Fact 1.18.12, 68 Schweitzer’s inequality Fact 1.17.38, 63 Wang’s inequality Fact 1.17.13, 56 Young inequality

Fact 1.12.21, 36 Young’s inequality Fact 1.12.32, 39 Fact 1.17.31, 61 Schatten norm absolute value Fact 9.13.11, 661 Cartesian decomposition Fact 9.9.37, 643 Fact 9.9.39, 644 Fact 9.9.40, 644 Clarkson inequalities Fact 9.9.34, 642 commutator Fact 9.9.27, 641 compatible norms Proposition 9.3.6, 606 Corollary 9.3.7, 606 Corollary 9.3.8, 607 definition Proposition 9.2.3, 602 eigenvalue Fact 9.11.6, 655 equality Fact 9.9.33, 642 Frobenius norm Fact 9.8.20, 630 Hanner inequality Fact 9.9.36, 643 Hermitian matrix Fact 9.9.27, 641 Fact 9.9.39, 644 H¨ older matrix norm Fact 9.11.6, 655 H¨ older norm Proposition 9.2.5, 603 inequality Fact 9.9.34, 642 Fact 9.9.36, 643 Fact 9.9.37, 643 Fact 9.9.38, 643 Fact 9.9.45, 645 Kronecker product Fact 9.14.38, 676 matrix difference Fact 9.9.23, 640 monotonicity Proposition 9.2.4, 603

1105

normal matrix Fact 9.9.27, 641 Fact 9.14.5, 666 partitioned matrix Fact 9.10.2, 650 Fact 9.10.3, 650 Fact 9.10.4, 651 Fact 9.10.5, 651 Fact 9.10.6, 652 Fact 9.10.7, 652 Fact 9.10.8, 652 positive-semidefinite matrix Fact 9.9.22, 640 Fact 9.9.39, 644 Fact 9.9.40, 644 Fact 9.10.6, 652 Fact 9.10.7, 652 Schur product Fact 9.14.33, 675 trace Fact 9.12.1, 656 unitarily invariant norm Fact 9.8.9, 628 Schauder fixed-point theorem image of a continuous function Theorem 10.3.10, 685 Schinzel determinant upper bound Fact 2.13.15, 143 Schmidt-Mirsky theorem fixed-rank approximation Fact 9.14.28, 672 Schneider inertia of a Hermitian matrix Fact 12.21.4, 867 Fact 12.21.5, 867 Schoenberg Euclidean distance matrix

1106

Schoenberg

Fact 9.8.14, 629 Schott’s theorem Schur product of positivesemidefinite matrices Fact 8.22.12, 586 Schroeder-Bernstein theorem one-to-one and onto function Fact 1.7.18, 15 Schur dimension of the algebra generated by commuting matrices Fact 5.10.15, 347 Schur complement convex function Proposition 8.6.17, 480 Lemma 8.6.16, 480 definition Definition 6.1.8, 401 determinant Proposition 8.2.3, 462 increasing function Proposition 8.6.13, 478 inequality Fact 8.11.17, 518 inertia Fact 6.5.5, 423 nondecreasing function Proposition 8.6.13, 478 partitioned matrix Fact 6.5.4, 423 Fact 6.5.5, 423 Fact 6.5.6, 423 Fact 6.5.8, 424 Fact 6.5.12, 425 Fact 6.5.29, 430 Fact 8.22.41, 593 positive-semidefinite matrix Corollary 8.6.18, 486 Fact 8.11.3, 514

Fact 8.11.4, 515 Fact 8.11.18, 518 Fact 8.11.19, 518 Fact 8.11.20, 519 Fact 8.11.27, 521 Fact 8.21.19, 582 Fact 8.22.11, 586 rank Proposition 8.2.3, 462 Fact 6.5.6, 423 Fact 6.5.8, 424 Fact 6.5.11, 425 Schur product Fact 8.22.11, 586 Schur concave function definition Definition 2.1.2, 86 elementary symmetric function Fact 1.17.20, 58 entropy Fact 2.21.6, 176 strong majorization Fact 2.21.6, 176 Schur convex function definition Definition 2.1.2, 86 Muirhead’s theorem Fact 1.17.25, 59 strong majorization Fact 2.21.4, 176 Fact 2.21.5, 176 Schur decomposition Hermitian matrix Corollary 5.4.5, 320 Jordan form Fact 5.10.6, 346 normal matrix Corollary 5.4.4, 319 Fact 5.10.6, 346 range-Hermitian matrix Corollary 5.4.4, 319 Schur inverse positive-semidefinite matrix Fact 8.22.1, 584

Schur power definition, 444 Lyapunov equation Fact 8.8.16, 494 positive-semidefinite matrix Fact 8.22.2, 584 Fact 8.22.3, 584 Fact 8.22.26, 589 Schur product associative equalities, 444 commutative equalities, 444 complex conjugate transpose Fact 8.22.9, 586 definition, 444 distributive equalities, 444 eigenvalue Fact 8.22.18, 587 Frobenius norm Fact 9.14.33, 675 Fact 9.14.35, 676 geometric mean Fact 8.22.53, 595 Hermitian matrix Fact 8.22.29, 590 Fact 8.22.34, 591 Kronecker product Proposition 7.3.1, 444 lower bound Fact 8.22.14, 587 M-matrix Fact 7.6.15, 457 matrix equality Fact 7.6.3, 454 Fact 7.6.4, 454 Fact 7.6.8, 455 matrix exponential Fact 11.14.21, 752 matrix logarithm Fact 8.22.49, 594 Fact 8.22.50, 594 matrix power Fact 7.6.11, 455 maximum singular value

secant condition Fact 8.22.10, 586 Fact 9.14.31, 673 Fact 9.14.32, 674 Fact 9.14.35, 676 Fact 9.14.36, 676 nonnegative matrix Fact 7.6.13, 456 normal matrix Fact 9.9.63, 649 partitioned matrix Fact 8.22.6, 585 Fact 8.22.41, 593 Fact 8.22.42, 593 positive matrix Fact 7.6.14, 456 positive-definite matrix Fact 8.22.4, 584 Fact 8.22.5, 585 Fact 8.22.6, 585 Fact 8.22.7, 586 Fact 8.22.13, 586 Fact 8.22.14, 587 Fact 8.22.15, 587 Fact 8.22.22, 589 Fact 8.22.35, 591 Fact 8.22.36, 592 Fact 8.22.38, 592 Fact 8.22.40, 592 Fact 8.22.44, 593 Fact 8.22.49, 594 Fact 8.22.51, 595 Fact 8.22.52, 595 Fact 8.22.53, 595 positive-semidefinite matrix Fact 8.22.4, 584 Fact 8.22.7, 586 Fact 8.22.11, 586 Fact 8.22.12, 586 Fact 8.22.14, 587 Fact 8.22.17, 587 Fact 8.22.18, 587 Fact 8.22.21, 588 Fact 8.22.23, 589 Fact 8.22.24, 589 Fact 8.22.33, 590 Fact 8.22.37, 592 Fact 8.22.39, 592

Fact 8.22.41, 593 Fact 8.22.42, 593 Fact 8.22.43, 593 Fact 8.22.44, 593 Fact 8.22.45, 594 Fact 8.22.46, 594 Fact 8.22.47, 594 Fact 8.22.48, 594 quadratic form Fact 7.6.5, 454 Fact 7.6.6, 455 range Fact 7.6.9, 455 rank Fact 7.6.10, 455 Fact 8.22.16, 587 Schatten norm Fact 9.14.33, 675 Schur complement Fact 8.22.11, 586 singular value Fact 9.14.31, 673 Fact 9.14.32, 674 Fact 9.14.34, 675 spectral radius Fact 7.6.13, 456 Fact 7.6.14, 456 Fact 7.6.16, 457 Fact 7.6.17, 458 Fact 9.14.32, 674 submultiplicative norm Fact 9.8.41, 634 trace Fact 7.6.7, 455 Fact 8.22.17, 587 Fact 9.14.34, 675 trace norm Fact 9.14.35, 676 transpose Fact 7.6.12, 456 unitarily invariant norm Fact 9.8.41, 634 Fact 9.9.62, 649 Fact 9.9.63, 649 Fact 9.14.37, 676 vector equality Fact 7.6.1, 454

1107

Fact 7.6.2, 454 weak majorization Fact 9.14.31, 673 Schur product of polynomials asymptotically stable polynomial Fact 11.17.9, 765 Schur’s formulas determinant of partitioned matrix Fact 2.14.13, 147 Schur’s inequality eigenvalue Fact 8.18.5, 560 eigenvalues and the Frobenius norm Fact 9.11.3, 654 Schur’s theorem eigenvalue inequality Fact 8.18.8, 561 Schur product of positivesemidefinite matrices Fact 8.22.12, 586 Schur-Cohn criterion discrete-time asymptotically stable polynomial Fact 11.20.1, 777 Schur-Horn theorem diagonal entries of a unitary matrix Fact 3.11.14, 207 Fact 8.18.10, 562 Schwarz form tridiagonal matrix Fact 11.18.25, 770 Fact 11.18.26, 771 Schweitzer’s inequality scalar inequality Fact 1.17.38, 63 secant condition

1108

secant condition

asymptotically stable matrix Fact 11.18.29, 772 second derivative definition, 687 Seiler determinant inequality Fact 8.13.31, 539 self-adjoint norm definition, 602 unitarily invariant norm Fact 9.8.7, 627 self-conjugate set definition, 254 semicontractive matrix complex conjugate transpose Fact 3.22.7, 241 definition Definition 3.1.2, 180 discrete-time Lyapunov-stable matrix Fact 11.21.4, 782 partitioned matrix Fact 8.11.6, 516 Fact 8.11.22, 520 positive-semidefinite matrix Fact 8.11.6, 516 Fact 8.11.13, 517 unitary matrix Fact 8.11.22, 520 semidissipative matrix definition Definition 3.1.1, 179 determinant Fact 8.13.3, 534 Fact 8.13.4, 534 Fact 8.13.11, 534, 535 discrete-time Lyapunov-stable matrix Fact 11.21.4, 782

dissipative matrix Fact 8.13.32, 539 Kronecker sum Fact 7.5.8, 451 Lyapunov-stable matrix Fact 11.18.37, 774 normal matrix Fact 11.18.37, 774 semiperimeter quadrilateral Fact 2.20.13, 172 triangle Fact 2.20.11, 169

outer-product matrix Fact 5.14.3, 370 positive-semidefinite matrix Corollary 8.3.7, 466 reverse-diagonal matrix Fact 5.14.11, 371 similar matrices Proposition 5.5.11, 324 Fact 5.9.6, 339 Fact 5.10.5, 346 simple matrix Fact 5.14.10, 371 skew-involutory matrix Fact 5.14.18, 372

semisimple eigenvalue cyclic eigenvalue Proposition 5.5.5, 322 defect Proposition 5.5.8, 323 definition Definition 5.5.4, 322 index of an eigenvalue Proposition 5.5.8, 323 null space Proposition 5.5.8, 323 simple eigenvalue Proposition 5.5.5, 322

semistability eigenvalue Proposition 11.8.2, 727 linear dynamical system Proposition 11.8.2, 727 Lyapunov equation Corollary 11.9.1, 730 matrix exponential Proposition 11.8.2, 727

semisimple matrix cyclic matrix Fact 5.14.10, 371 definition Definition 5.5.4, 322 elementary matrix Fact 5.14.16, 372 idempotent matrix Fact 5.14.20, 373 identity-matrix perturbation Fact 5.14.15, 372 involutory matrix Fact 5.14.18, 372 Kronecker product Fact 7.4.17, 446 matrix exponential Proposition 11.2.7, 712 normal matrix Proposition 5.5.11, 324

semistable matrix compartmental matrix Fact 11.19.6, 776 definition Definition 11.8.1, 727 group-invertible matrix Fact 11.18.3, 766 Kronecker sum Fact 11.18.32, 773 Fact 11.18.33, 773 limit Fact 11.18.7, 767 Lyapunov equation Fact 12.21.15, 870 Lyapunov-stable matrix Fact 11.18.1, 766 matrix exponential

sign of entry Fact 11.18.6, 766 Fact 11.18.7, 767 Fact 11.21.8, 783 minimal realization Definition 12.9.17, 829 semistable polynomial Proposition 11.8.4, 728 similar matrices Fact 11.18.4, 766 unstable subspace Proposition 11.8.8, 729 semistable polynomial definition Definition 11.8.3, 728 reciprocal argument Fact 11.17.5, 764 semistable matrix Proposition 11.8.4, 728 semistable transfer function minimal realization Proposition 12.9.18, 829 SISO entry Proposition 12.9.19, 829 separation theorem convex cone Fact 10.9.14, 697 inner product Fact 10.9.14, 697 Fact 10.9.15, 697 sequence definition Definition 10.2.1, 682 generalized inverse Fact 6.3.35, 411 series commutator Fact 11.14.17, 751 definition Definition 10.2.6, 683 Definition 10.2.8, 683 inverse matrix Proposition 9.4.13, 612 matrix exponential

Fact 11.14.17, 751 set definition, 2 distance from a point Fact 10.9.16, 697 Fact 10.9.17, 698 finite set definition, 2 set cancellation convex set Fact 10.9.8, 696 Radstrom Fact 10.9.8, 696 set equality intersection Fact 1.7.6, 12 union Fact 1.7.6, 12 sextic arithmetic-mean– geometric-mean inequality Fact 1.15.1, 52 Shannon’s inequality logarithm Fact 1.18.30, 73 shear factor factorization Fact 5.15.11, 379 Shemesh common eigenvector Fact 5.14.26, 373 Sherman-MorrisonWoodbury formula determinant of an outer-product perturbation Fact 2.16.3, 153 shift controllability Fact 12.20.10, 865 stabilizability Fact 12.20.11, 865

1109

shifted argument transfer function Fact 12.22.3, 872 shifted-orthogonal matrix definition Definition 3.1.1, 179 shifted-unitary matrix block-diagonal matrix Fact 3.11.8, 205 definition Definition 3.1.1, 179 normal matrix Fact 3.11.26, 209 spectrum Proposition 5.5.20, 326 unitary matrix Fact 3.11.25, 209 Shoda factorization Fact 5.15.8, 378 Fact 5.15.34, 383 Shoda’s theorem commutator realization Fact 5.9.20, 341 zero trace Fact 5.9.20, 341 shortcut of a relation definition Definition 1.5.4, 7 shorted operator definition Fact 8.21.19, 582 positive-semidefinite matrix Fact 8.21.19, 582 sign matrix, 97 vector, 97 sign of entry asymptotically stable matrix Fact 11.19.5, 777

1110

sign stability

sign stability asymptotically stable matrix Fact 11.19.5, 777 signature definition, 267 Hermitian matrix Fact 5.8.6, 334 Fact 5.8.7, 335 Fact 8.10.17, 503 involutory matrix Fact 5.8.2, 334 positive-semidefinite matrix Fact 5.8.9, 335 tripotent matrix Fact 5.8.3, 334 signed volume simplex Fact 2.20.15, 173 similar matrices asymptotically stable matrix Fact 11.18.4, 766 biequivalent matrices Proposition 3.4.5, 189 block-diagonal matrix Theorem 5.3.2, 314 Theorem 5.3.3, 315 campanion matrix Fact 5.16.5, 388 characteristic polynomial Fact 4.9.10, 284 complex conjugate Fact 5.9.33, 345 cyclic matrix Fact 5.16.5, 388 definition Definition 3.4.4, 188 diagonal entry Fact 5.9.15, 340 diagonalizable over R Proposition 5.5.12, 324 Corollary 5.5.21, 327

discrete-time asymptotically stable matrix Fact 11.18.4, 766 discrete-time Lyapunov-stable matrix Fact 11.18.4, 766 discrete-time semistable matrix Fact 11.18.4, 766 equivalence class Fact 5.10.4, 346 equivalent realizations Definition 12.9.6, 824 example Example 5.5.19, 326 factorization Fact 5.15.7, 378 geometric multiplicity Proposition 5.5.10, 324 group-invertible matrix Proposition 3.4.5, 189 Fact 5.9.7, 339 Hermitian matrix Proposition 5.5.12, 324 idempotent matrix Proposition 3.4.5, 189 Proposition 5.5.22, 327 Corollary 5.5.21, 327 Fact 5.10.9, 346 Fact 5.10.13, 347 Fact 5.10.14, 347 Fact 5.10.22, 349 inverse matrix Fact 5.15.31, 383 involutory matrix Proposition 3.4.5, 189 Corollary 5.5.21, 327 Fact 5.15.31, 383 Kronecker product Fact 7.4.13, 446 Kronecker sum Fact 7.5.9, 451 lower triangular matrix

Fact 5.9.4, 339 Lyapunov-stable matrix Fact 11.18.4, 766 matrix classes Proposition 3.4.5, 189 matrix exponential Proposition 11.2.9, 715 matrix power Fact 5.9.3, 339 minimal polynomial Proposition 4.6.3, 270 Fact 11.23.3, 788 Fact 11.23.4, 788 Fact 11.23.5, 789 Fact 11.23.6, 789 Fact 11.23.7, 790 Fact 11.23.8, 790 Fact 11.23.9, 791 Fact 11.23.10, 792 Fact 11.23.11, 792 multicompanion form Corollary 5.2.6, 312 nilpotent matrix Proposition 3.4.5, 189 Fact 5.10.23, 349 nonsingular matrix Fact 5.10.11, 347 nonzero diagonal entry Fact 5.9.16, 341 normal matrix Proposition 5.5.11, 324 Fact 5.9.11, 340 Fact 5.9.12, 340 Fact 5.10.7, 346 partitioned matrix Fact 5.10.21, 349 Fact 5.10.22, 349 Fact 5.10.23, 349 projector Corollary 5.5.21, 327 Fact 5.10.13, 347 realization Proposition 12.9.5, 824 reflector Corollary 5.5.21, 327 reverse transpose

simplex Fact 5.9.13, 340 reverse-symmetric matrix Fact 5.9.13, 340 semisimple matrix Proposition 5.5.11, 324 Fact 5.9.6, 339 Fact 5.10.5, 346 semistable matrix Fact 11.18.4, 766 similarity invariant Theorem 4.3.10, 261 Corollary 5.2.6, 312 simultaneous diagonalization Fact 5.17.8, 392 skew-Hermitian matrix Fact 5.9.6, 339 Fact 11.18.12, 768 skew-idempotent matrix Corollary 5.5.21, 327 skew-involutory matrix Proposition 3.4.5, 189 skew-symmetric matrix Fact 5.15.39, 384 Sylvester’s equation Corollary 7.2.5, 444 Fact 7.5.14, 452 symmetric matrix Fact 5.15.39, 384 transpose Proposition 5.5.12, 324 Corollary 4.3.11, 261 Corollary 5.3.8, 317 Corollary 5.5.21, 327 Fact 5.9.11, 340 Fact 5.9.12, 340 tripotent matrix Proposition 3.4.5, 189 Corollary 5.5.21, 327 unitarily invariant norm Fact 9.8.31, 632 unitarily similar matrices

Fact 5.10.7, 346 upper triangular matrix Fact 5.9.4, 339 Vandermonde matrix Fact 5.16.5, 388 similarity equivalence relation Fact 5.10.3, 345 similarity invariant characteristic polynomial Proposition 4.4.2, 262 Proposition 4.6.2, 270 definition Definition 4.3.9, 261 multicompanion form Corollary 5.2.6, 312 similar matrices Theorem 4.3.10, 261 Corollary 5.2.6, 312 similarity transformation complex conjugate transpose Fact 5.9.10, 340 Fact 5.15.4, 377 complex symmetric Jordan form Fact 5.15.2, 377 Fact 5.15.3, 377 eigenvector Fact 5.14.5, 370 Fact 5.14.6, 370 hypercompanion matrix Fact 5.10.1, 345 inverse matrix Fact 5.15.4, 377 normal matrix Fact 5.15.3, 377 real Jordan form Fact 5.10.1, 345 Fact 5.10.2, 345 symmetric matrix Fact 5.15.2, 377 Fact 5.15.3, 377

1111

SIMO transfer function definition Definition 12.9.1, 822 Simon determinant inequality Fact 8.13.31, 539 normal product and Schatten norm Fact 9.14.5, 666 simple eigenvalue cyclic eigenvalue Proposition 5.5.5, 322 definition Definition 5.5.4, 322 semisimple eigenvalue Proposition 5.5.5, 322 simple graph definition Definition 1.6.3, 10 simple matrix commuting matrices Fact 5.14.22, 373 cyclic matrix Fact 5.14.10, 371 definition Definition 5.5.4, 322 identity-matrix perturbation Fact 5.14.15, 372 rank Fact 5.11.1, 350 semisimple matrix Fact 5.14.10, 371 simplex convex hull Fact 2.20.4, 167 definition, 98 interior Fact 2.20.4, 167 nonsingular matrix Fact 2.20.4, 167 signed volume Fact 2.20.15, 173 volume Fact 2.20.19, 174

1112

simultaneous diagonalization

simultaneous diagonalization cogredient transformation Fact 8.17.4, 558 Fact 8.17.6, 559 commuting matrices Fact 8.17.1, 558 definition, 465 diagonalizable matrix Fact 8.17.2, 558 Fact 8.17.3, 558 Hermitian matrix Fact 8.17.1, 558 Fact 8.17.4, 558 Fact 8.17.6, 559 positive-definite matrix Fact 8.17.5, 558 similar matrices Fact 5.17.8, 392 unitarily similar matrices Fact 5.17.7, 392 unitary matrix Fact 8.17.1, 558 simultaneous diagonalization of symmetric matrices Milnor Fact 8.17.6, 559 Pesonen Fact 8.17.6, 559 simultaneous orthogonal biequivalence transformation upper Hessenberg matrix Fact 5.17.3, 392 upper triangular matrix Fact 5.17.3, 392 simultaneous triangularization cogredient transformation

Fact 5.17.9, 392 common eigenvector Fact 5.17.1, 391 commutator Fact 5.17.5, 392 Fact 5.17.6, 392 commuting matrices Fact 5.17.4, 392 nilpotent matrix Fact 5.17.6, 392 projector Fact 5.17.6, 392 regular pencil Fact 5.17.2, 391 simultaneous unitary biequivalence transformation Fact 5.17.2, 391 unitarily similar matrices Fact 5.17.4, 392 Fact 5.17.6, 392 simultaneous unitary biequivalence transformation simultaneous triangularization Fact 5.17.2, 391 sine rule triangle Fact 2.20.11, 169 singular matrix definition, 110 Kronecker product Fact 7.4.29, 447 spectrum Proposition 5.5.20, 326 singular pencil definition, 330 generalized eigenvalue Proposition 5.7.3, 332 singular polynomial matrix Definition 4.2.5, 257 singular value

2 × 2 matrix Fact 5.11.31, 357 adjugate Fact 5.11.36, 358 bidiagonal matrix Fact 5.11.47, 362 block-diagonal matrix Fact 8.19.9, 567 Fact 8.19.10, 567 Fact 9.14.21, 670 Fact 9.14.25, 671 Cartesian decomposition Fact 8.19.7, 566 companion matrix Fact 5.11.30, 356 complex conjugate transpose Fact 5.11.20, 353 Fact 5.11.34, 357 convex function Fact 11.16.14, 762 Fact 11.16.15, 762 definition Definition 5.6.1, 328 determinant Fact 5.11.28, 356 Fact 5.11.29, 356 Fact 5.12.13, 365 Fact 8.13.1, 533 Fact 9.13.22, 664 eigenvalue Fact 8.18.5, 560 Fact 8.18.6, 561 Fact 9.13.21, 664 eigenvalue of Hermitian part Fact 5.11.27, 355 Fact 8.18.4, 560 Fan dominance theorem Fact 9.14.19, 670 fixed-rank approximation Fact 9.14.28, 672 Fact 9.15.8, 678 Frobenius Corollary 9.6.7, 617

skew reflector generalized inverse Fact 6.3.28, 410 homogeneity Fact 5.11.19, 353 idempotent matrix Fact 5.11.38, 358 induced lower bound Proposition 9.5.4, 614 inequality Proposition 9.2.2, 602 Corollary 9.6.5, 616 Fact 9.14.23, 671 Fact 9.14.24, 671 interlacing Fact 9.14.10, 667 matrix difference Fact 8.19.9, 567 Fact 8.19.10, 567 matrix exponential Fact 11.15.5, 756 Fact 11.16.14, 762 Fact 11.16.15, 762 matrix power Fact 9.13.18, 662 Fact 9.13.19, 663 matrix product Proposition 9.6.1, 615 Proposition 9.6.2, 615 Proposition 9.6.3, 616 Proposition 9.6.4, 616 Fact 8.19.22, 571 Fact 9.13.16, 662 Fact 9.13.17, 662 Fact 9.14.26, 671 matrix sum Proposition 9.6.8, 617 Fact 9.14.20, 670 Fact 9.14.21, 670 Fact 9.14.25, 671 normal matrix Fact 5.14.14, 372 outer-product matrix Fact 5.11.17, 353 partitioned matrix Proposition 5.6.5, 330 Fact 9.14.11, 667 Fact 9.14.24, 671 perturbation

Fact 9.14.6, 666 positive-semidefinite matrix Fact 8.11.9, 516 Fact 8.19.7, 566 Fact 9.14.27, 671 rank Proposition 5.6.2, 329 Fact 9.14.28, 672 Fact 9.15.8, 678 Schur product Fact 9.14.31, 673 Fact 9.14.32, 674 Fact 9.14.34, 675 strong log majorization Fact 9.13.18, 662 submatrix Fact 9.14.10, 667 trace Fact 5.12.6, 364 Fact 8.18.2, 559 Fact 9.12.1, 656 Fact 9.13.15, 662 Fact 9.14.3, 665 Fact 9.14.34, 675 unitarily biequivalent matrices Fact 5.10.18, 348 unitarily invariant norm Fact 9.14.28, 672 unitary matrix Fact 5.11.37, 358 Fact 9.14.11, 667 weak log majorization Proposition 9.6.2, 615 weak majorization Proposition 9.2.2, 602 Proposition 9.6.3, 616 Fact 5.11.27, 355 Fact 8.18.5, 560 Fact 8.19.7, 566 Fact 8.19.22, 571 Fact 9.13.16, 662 Fact 9.13.17, 662 Fact 9.13.19, 663

1113

Fact 9.14.19, 670 Fact 9.14.20, 670 Fact 9.14.31, 673 Weyl majorant theorem Fact 9.13.19, 663 singular value decomposition existence Theorem 5.6.3, 329 generalized inverse Fact 6.3.14, 407 group generalized inverse Fact 6.6.16, 435 least squares Fact 9.14.28, 672 Fact 9.15.8, 678 Fact 9.15.9, 679 Fact 9.15.10, 679 unitary similarity Fact 5.9.30, 343 Fact 6.3.14, 407 Fact 6.6.16, 435 singular value perturbation unitarily invariant norm Fact 9.14.29, 673 SISO transfer function definition Definition 12.9.1, 822 size definition, 86 skew reflector Hamiltonian matrix Fact 3.20.3, 238 reflector Fact 3.14.7, 230 skew-Hermitian matrix Fact 3.14.6, 230 skew-involutory matrix Fact 3.14.6, 230 spectrum Proposition 5.5.20, 326

1114

skew reflector

unitary matrix Fact 3.14.6, 230 skew-Hermitian matrix, see skew-symmetric matrix adjugate Fact 3.7.10, 193 Fact 3.7.11, 194 asymptotically stable matrix Fact 11.18.30, 773 block-diagonal matrix Fact 3.7.8, 193 Cartesian decomposition Fact 3.7.27, 197 Fact 3.7.28, 197 Fact 3.7.29, 198 Cayley transform Fact 3.11.22, 208 characteristic polynomial Fact 4.9.14, 285 commutator Fact 3.8.1, 199 Fact 3.8.4, 200 complex conjugate Fact 3.12.8, 216 congruent matrices Proposition 3.4.5, 189 definition Definition 3.1.1, 179 determinant Fact 3.7.11, 194 Fact 3.7.16, 196 Fact 8.13.6, 534 eigenvalue Fact 5.11.6, 350 existence of transformation Fact 3.9.4, 201 Hermitian matrix Fact 3.7.9, 193 Fact 3.7.28, 197 inertia Fact 5.8.4, 334 Kronecker product Fact 7.4.18, 446

Kronecker sum Fact 7.5.8, 451 Lyapunov equation Fact 11.18.12, 768 matrix exponential Proposition 11.2.8, 713 Proposition 11.2.9, 715 Fact 11.14.6, 749 Fact 11.14.33, 755 matrix power Fact 8.9.14, 496 normal matrix Proposition 3.1.6, 183 null space Fact 8.7.5, 487 outer-product matrix Fact 3.7.17, 196 Fact 3.9.4, 201 partitioned matrix Fact 3.7.27, 197 positive-definite matrix Fact 8.13.6, 534 Fact 11.18.12, 768 positive-semidefinite matrix Fact 8.9.12, 496 projector Fact 9.9.9, 637 quadratic form Fact 3.7.6, 193 range Fact 8.7.5, 487 rank Fact 3.7.17, 196 Fact 3.7.30, 198 similar matrices Fact 5.9.6, 339 Fact 11.18.12, 768 skew reflector Fact 3.14.6, 230 skew-involutory matrix Fact 3.14.6, 230 skew-symmetric matrix Fact 3.7.9, 193 spectrum

Proposition 5.5.20, 326 symmetric matrix Fact 3.7.9, 193 trace Fact 3.7.24, 197 trace of a product Fact 8.12.6, 524 unitarily similar matrices Proposition 3.4.5, 189 Proposition 5.5.22, 327 unitary matrix Fact 3.11.22, 208 Fact 3.14.6, 230 Fact 11.14.33, 755 skew-idempotent matrix idempotent matrix Fact 3.12.5, 216 similar matrices Corollary 5.5.21, 327 skew-involutory matrix definition Definition 3.1.1, 179 Hamiltonian matrix Fact 3.20.2, 238 Fact 3.20.3, 238 inertia Fact 5.8.4, 334 matrix exponential Fact 11.11.1, 736 semisimple matrix Fact 5.14.18, 372 similar matrices Proposition 3.4.5, 189 size Fact 3.15.8, 231 skew reflector Fact 3.14.6, 230 skew-Hermitian matrix Fact 3.14.6, 230 skew-symmetric matrix Fact 3.20.3, 238 spectrum Proposition 5.5.20, 326 symplectic matrix

Smith zeros Fact 3.20.2, 238 unitarily similar matrices Proposition 3.4.5, 189 unitary matrix Fact 3.14.6, 230 skew-symmetric matrix, see skew-Hermitian matrix adjugate Fact 4.9.21, 286 Cayley transform Fact 3.11.22, 208 Fact 3.11.23, 209 Fact 3.11.29, 210 characteristic polynomial Fact 4.9.13, 285 Fact 4.9.20, 286 Fact 4.9.21, 286 Fact 5.14.33, 375 commutator Fact 3.8.5, 200 congruent matrices Fact 3.7.34, 198 Fact 5.9.18, 341 controllability Fact 12.20.5, 864 definition Definition 3.1.1, 179 determinant Fact 3.7.15, 195 Fact 3.7.33, 198 Fact 4.8.14, 282 Fact 4.9.21, 286 Fact 4.10.4, 289 eigenvalue Fact 4.10.4, 289 factorization Fact 5.15.37, 384 Fact 5.15.38, 384 Hamiltonian matrix Fact 3.7.34, 198 Fact 3.20.3, 238 Fact 3.20.8, 239 Hermitian matrix Fact 3.7.9, 193

linear matrix equation Fact 3.7.3, 193 matrix exponential Example 11.3.6, 717 Fact 11.11.3, 736 Fact 11.11.6, 737 Fact 11.11.7, 738 Fact 11.11.8, 738 Fact 11.11.9, 739 Fact 11.11.10, 739 Fact 11.11.11, 739 Fact 11.11.15, 740 Fact 11.11.16, 740 Fact 11.11.17, 741 Fact 11.11.18, 741 matrix product Fact 5.15.37, 384 orthogonal matrix Fact 3.11.22, 208 Fact 3.11.23, 209 Fact 11.11.10, 739 Fact 11.11.11, 739 orthogonally similar matrices Fact 5.14.32, 375 partitioned matrix Fact 3.11.10, 206 Pfaffian Fact 4.8.14, 282 quadratic form Fact 3.7.5, 193 similar matrices Fact 5.15.39, 384 skew-Hermitian matrix Fact 3.7.9, 193 skew-involutory matrix Fact 3.20.3, 238 spectrum Fact 4.9.21, 286 Fact 4.10.4, 289 Fact 5.14.32, 375 symmetric matrix Fact 5.9.18, 341 Fact 5.15.39, 384 trace Fact 3.7.23, 197

1115

Fact 3.7.31, 198 unit imaginary matrix Fact 3.7.34, 198 small-gain theorem multiplicative perturbation Fact 9.13.22, 664 Smith form biequivalent matrices Theorem 5.1.1, 309 Corollary 5.1.2, 309 controllability pencil Proposition 12.6.15, 813 existence Theorem 4.3.2, 259 observability pencil Proposition 12.3.15, 803 polynomial matrix Proposition 4.3.4, 259 rank Proposition 4.3.5, 259 Proposition 4.3.6, 260 submatrix Proposition 4.3.5, 259 unimodular matrix Proposition 4.3.7, 260 Smith polynomial nonsingular matrix transformation Proposition 4.3.8, 260 Smith polynomials definition Definition 4.3.3, 259 Smith zeros controllability pencil Proposition 12.6.16, 814 definition Definition 4.3.3, 259 observability pencil Proposition 12.3.16, 803 uncontrollable spectrum

1116

Smith zeros

Proposition 12.6.16, 814 unobservable spectrum Proposition 12.3.16, 803 Smith’s method finite-sum solution of Lyapunov equation Fact 12.21.17, 870 Smith-McMillan form blocking zero Proposition 4.7.11, 273 coprime polynomials Fact 4.8.15, 282 coprime right polynomial fraction description Proposition 4.7.16, 275 existence Theorem 4.7.5, 272 poles Proposition 4.7.11, 273 rank Proposition 4.7.7, 272 Proposition 4.7.8, 272 submatrix Proposition 4.7.7, 272 SO(2) parameterization Fact 3.11.27, 209 solid angle circular cone Fact 2.20.22, 175 Fact 2.20.23, 175 cone Fact 2.20.21, 174 solid set completely solid set Fact 10.8.9, 693 convex hull Fact 10.8.10, 693 convex set Fact 10.8.9, 693 definition, 682 dimension Fact 10.8.16, 694

image Fact 10.8.17, 694 solution Riccati equation Definition 12.16.12, 853 span affine subspace Fact 2.9.7, 120 Fact 2.20.4, 167 Fact 10.8.12, 694 constructive characterization Theorem 2.3.5, 100 convex conical hull Fact 2.9.3, 119 definition, 98 intersection Fact 2.9.12, 121 union Fact 2.9.12, 121 spanning path graph Fact 1.8.6, 16 tournament Fact 1.8.6, 16 spanning subgraph Definition 1.6.3, 10 Specht reverse arithmetic-mean– geometric-mean inequality Fact 1.17.19, 57 Specht’s ratio matrix exponential Fact 11.14.28, 753 power of a positive-definite matrix Fact 11.14.22, 752 Fact 11.14.23, 752 reverse arithmetic-mean– geometric-mean inequality

Fact 1.17.19, 57 reverse Young inequality Fact 1.12.22, 37 special orthogonal group real symplectic group Fact 3.24.5, 250 spectral abscissa definition, 267 eigenvalue Fact 5.11.24, 355 Hermitian matrix Fact 5.11.5, 350 Kronecker sum Fact 7.5.6, 450 matrix exponential Fact 11.13.2, 743 Fact 11.15.8, 758 Fact 11.15.9, 758 Fact 11.18.8, 767 Fact 11.18.9, 767 maximum eigenvalue Fact 5.11.5, 350 maximum singular value Fact 5.11.26, 355 minimum singular value Fact 5.11.26, 355 outer-product matrix Fact 5.11.13, 352 spectral radius Fact 4.10.6, 290 Fact 11.13.2, 743 spectral decomposition normal matrix Fact 5.14.13, 372 spectral factorization definition, 254 Hamiltonian Proposition 12.16.13, 853 polynomial roots Proposition 4.1.1, 254

spectrum spectral norm definition, 603 spectral order positive-definite matrix Fact 8.20.3, 575 positive-semidefinite matrix Fact 8.20.3, 575 spectral radius bound Fact 4.10.23, 295 column norm Corollary 9.4.10, 611 commuting matrices Fact 5.12.11, 365 convergent sequence Fact 9.8.4, 627 convergent series Fact 4.10.7, 290 convexity for nonnegative matrices Fact 4.11.20, 306 definition, 267 equi-induced norm Corollary 9.4.5, 608 Frobenius norm Fact 9.13.12, 661 Hermitian matrix Fact 5.11.5, 350 induced norm Corollary 9.4.5, 608 Corollary 9.4.10, 611 infinite series Fact 10.13.1, 704 inverse matrix Proposition 9.4.13, 612 Kronecker product Fact 7.4.15, 446 lower bound Fact 9.13.12, 661 matrix exponential Fact 11.13.2, 743 matrix sum Fact 5.12.2, 362 Fact 5.12.3, 363

maximum singular value Corollary 9.4.10, 611 Fact 5.11.5, 350 Fact 5.11.26, 355 Fact 8.19.26, 572 Fact 9.8.13, 629 Fact 9.13.9, 660 minimum singular value Fact 5.11.26, 355 monotonicity for nonnegative matrices Fact 4.11.19, 306 nonnegative matrix Fact 4.11.8, 301 Fact 4.11.17, 305 Fact 4.11.18, 305 Fact 7.6.13, 456 Fact 9.11.9, 656 Fact 11.19.3, 775 nonsingular matrix Fact 4.10.30, 296 norm Proposition 9.2.6, 604 normal matrix Fact 5.14.14, 372 outer-product matrix Fact 5.11.13, 352 perturbation Fact 9.14.6, 666 positive matrix Fact 7.6.14, 456 positive-definite matrix Fact 8.10.5, 501 Fact 8.19.26, 572 positive-semidefinite matrix Fact 8.19.26, 572 Fact 8.21.8, 579 row norm Corollary 9.4.10, 611 row-stochastic matrix Fact 4.11.6, 300 Schur product

1117

Fact 7.6.13, 456 Fact 7.6.14, 456 Fact 7.6.16, 457 Fact 7.6.17, 458 Fact 9.14.32, 674 spectral abscissa Fact 4.10.6, 290 Fact 11.13.2, 743 submultiplicative norm Proposition 9.3.2, 604 Proposition 9.3.3, 605 Corollary 9.3.4, 605 Fact 9.8.4, 627 Fact 9.9.3, 636 trace Fact 4.10.23, 295 Fact 5.11.46, 361 Fact 9.13.12, 661 spectral radius of a product Bourin Fact 8.19.26, 572 spectral variation Hermitian matrix Fact 9.12.5, 657 Fact 9.12.7, 658 normal matrix Fact 9.12.5, 657 Fact 9.12.6, 658 spectrum, see eigenvalue, multispectrum adjugate Fact 4.10.9, 291 asymptotic eigenvalue Fact 4.10.29, 296 asymptotically stable matrix Fact 11.18.13, 768 block-triangular matrix Proposition 5.5.13, 324 bounds Fact 4.10.17, 293 Fact 4.10.21, 294 Fact 4.10.22, 295

1118

spectrum

Cartesian decomposition Fact 5.11.21, 354 circulant matrix Fact 5.16.7, 388 commutator Fact 5.12.14, 365 commuting matrices Fact 5.12.14, 365 continuity Fact 10.11.8, 701 Fact 10.11.9, 701 convex hull Fact 8.14.7, 546 Fact 8.14.8, 547 cross-product matrix Fact 4.9.20, 286 definition Definition 4.4.4, 262 dissipative matrix Fact 8.13.32, 539 doublet Fact 5.11.13, 352 elementary matrix Proposition 5.5.20, 326 elementary projector Proposition 5.5.20, 326 elementary reflector Proposition 5.5.20, 326 group-invertible matrix Proposition 5.5.20, 326 Hamiltonian Theorem 12.17.9, 857 Proposition 12.16.13, 853 Proposition 12.17.5, 856 Proposition 12.17.7, 857 Proposition 12.17.8, 857 Lemma 12.17.4, 855 Lemma 12.17.6, 856 Hamiltonian matrix Proposition 5.5.20, 326 Hermitian matrix Proposition 5.5.20, 326 Lemma 8.4.8, 469

idempotent matrix Proposition 5.5.20, 326 Fact 5.11.7, 350 identity-matrix perturbation Fact 4.10.14, 292 Fact 4.10.15, 292 inverse matrix Fact 5.11.14, 353 involutory matrix Proposition 5.5.20, 326 Laplacian matrix Fact 11.19.7, 777 lower triangular matrix Fact 4.10.10, 291 mass-spring system Fact 5.12.21, 368 matrix exponential Proposition 11.2.3, 712 Corollary 11.2.6, 712 matrix function Corollary 10.5.4, 690 matrix logarithm Theorem 11.5.2, 721 minimal polynomial Fact 4.9.26, 288 nilpotent matrix Proposition 5.5.20, 326 normal matrix Fact 4.10.25, 295 Fact 8.14.7, 546 Fact 8.14.8, 547 outer-product matrix Fact 4.10.1, 288 Fact 5.11.13, 352 partitioned matrix Fact 2.19.3, 164 Fact 4.10.26, 296 Fact 4.10.27, 296 permutation matrix Fact 5.16.8, 390 perturbed matrix Fact 4.9.12, 285 Fact 4.10.5, 290 polynomial Fact 4.9.27, 288 Fact 4.10.11, 291

positive matrix Fact 5.11.12, 351 positive-definite matrix Proposition 5.5.20, 326 positive-semidefinite matrix Proposition 5.5.20, 326 Fact 8.21.16, 580 projector Proposition 5.5.20, 326 Fact 5.12.15, 365 Fact 5.12.16, 366 properties Proposition 4.4.5, 263 quadratic form Fact 8.14.7, 546 Fact 8.14.8, 547 quadratic matrix equation Fact 5.11.3, 350 Fact 5.11.4, 350 rational function Fact 5.11.15, 353 reflector Proposition 5.5.20, 326 reverse permutation matrix Fact 5.9.26, 342 row-stochastic matrix Fact 4.10.2, 288 shifted-unitary matrix Proposition 5.5.20, 326 singular matrix Proposition 5.5.20, 326 skew reflector Proposition 5.5.20, 326 skew-Hermitian matrix Proposition 5.5.20, 326 skew-involutory matrix Proposition 5.5.20, 326 skew-symmetric matrix Fact 4.9.21, 286 Fact 4.10.4, 289

stability Fact 5.14.32, 375 subspace decomposition Proposition 5.5.7, 323 Sylvester’s equation Corollary 7.2.5, 444 Fact 7.5.14, 452 symplectic matrix Proposition 5.5.20, 326 Toeplitz matrix Fact 4.10.16, 293 Fact 5.11.43, 360 Fact 5.11.44, 360 Fact 8.9.35, 499 trace Fact 4.10.8, 290 tridiagonal matrix Fact 5.11.40, 359 Fact 5.11.41, 359 Fact 5.11.42, 359 Fact 5.11.43, 360 Fact 5.11.44, 360 tripotent matrix Proposition 5.5.20, 326 unipotent matrix Proposition 5.5.20, 326 unit imaginary matrix Fact 5.9.27, 343 unitary matrix Proposition 5.5.20, 326 upper triangular matrix Fact 4.10.10, 291 spectrum bounds Brauer Fact 4.10.22, 295 ovals of Cassini Fact 4.10.22, 295 spectrum of convex hull field of values Fact 8.14.7, 546 Fact 8.14.8, 547 numerical range Fact 8.14.7, 546 Fact 8.14.8, 547 sphere of radius ε

definition, 681 spin group double cover Fact 3.11.31, 212 spread commutator Fact 9.9.30, 641 Fact 9.9.31, 642 Hermitian matrix Fact 8.15.32, 555 square definition, 86 trace Fact 8.18.7, 561 square root 2 × 2 positivesemidefinite matrix Fact 8.9.6, 496 asymptotically stable matrix Fact 11.18.36, 774 commuting matrices Fact 5.18.1, 393 Fact 8.10.25, 504 convergent sequence Fact 5.15.21, 381 Fact 8.9.33, 499 definition, 474 equality Fact 8.9.25, 497 Fact 8.9.26, 497 generalized inverse Fact 8.21.4, 578 group-invertible matrix Fact 5.15.20, 381 Jordan form Fact 5.15.19, 380 Kronecker product Fact 8.22.30, 590 Fact 8.22.31, 590 matrix sign function Fact 5.15.21, 381 maximum singular value Fact 8.19.14, 568

1119

Fact 9.8.32, 632 Fact 9.14.15, 669 Newton-Raphson algorithm Fact 5.15.21, 381 normal matrix Fact 8.9.28, 498 Fact 8.9.29, 498 Fact 8.9.30, 498 orthogonal matrix Fact 8.9.27, 498 positive-semidefinite matrix Fact 8.10.18, 503 Fact 8.10.26, 504 Fact 8.22.30, 590 Fact 9.8.32, 632 principal square root Theorem 10.6.1, 690 projector Fact 8.10.25, 504 range Fact 8.7.2, 486 scalar inequality Fact 1.11.6, 26 Fact 1.14.1, 50 Fact 1.14.2, 50 submultiplicative norm Fact 9.8.32, 632 sum of squares Fact 2.18.8, 163 unitarily invariant norm Fact 9.9.18, 639 Fact 9.9.19, 639 unitary matrix Fact 8.9.27, 498 square-root function Niculescu’s inequality Fact 1.12.20, 36 squares scalar inequality Fact 1.13.21, 48 stability mass-spring system Fact 11.18.38, 774

1120

stability

partitioned matrix Fact 11.18.38, 774 stability radius asymptotically stable matrix Fact 11.18.17, 768 stabilizability asymptotically stable matrix Proposition 12.8.5, 821 Corollary 12.8.6, 822 block-triangular matrix Proposition 12.8.3, 820 controllably asymptotically stable Proposition 12.8.5, 821 definition Definition 12.8.1, 820 full-state feedback Proposition 12.8.2, 820 Hamiltonian Fact 12.23.1, 875 input matrix Fact 12.20.15, 865 Lyapunov equation Corollary 12.8.6, 822 maximal solution of the Riccati equation Theorem 12.18.1, 859 orthogonal matrix Proposition 12.8.3, 820 PBH test Theorem 12.8.4, 821 positive-semidefinite matrix Fact 12.20.6, 864 positive-semidefinite ordering Fact 12.20.8, 864 range Fact 12.20.7, 864 Riccati equation Theorem 12.17.9, 857 Theorem 12.18.1, 859 Corollary 12.19.2, 863

shift Fact 12.20.11, 865 stabilization controllability Fact 12.20.17, 865 Gramian Fact 12.20.17, 865 stabilizing solution Hamiltonian Corollary 12.16.15, 854 Riccati equation Definition 12.16.12, 853 Theorem 12.17.2, 855 Theorem 12.17.9, 857 Theorem 12.18.4, 860 Proposition 12.17.1, 855 Proposition 12.18.3, 860 Proposition 12.19.4, 863 Corollary 12.16.15, 854 stable subspace complementary subspaces Proposition 11.8.8, 729 group-invertible matrix Proposition 11.8.8, 729 idempotent matrix Proposition 11.8.8, 729 invariant subspace Proposition 11.8.8, 729 matrix exponential Proposition 11.8.8, 729 minimal polynomial Proposition 11.8.5, 728 Fact 11.23.1, 786 Fact 11.23.2, 787 standard control problem definition, 847 standard nilpotent matrix definition, 92

star partial ordering characterization Fact 6.4.52, 422 commuting matrices Fact 2.10.36, 130 definition Fact 2.10.35, 130 Fact 8.20.6, 576 generalized inverse Fact 8.20.7, 576 positive-semidefinite matrix Fact 8.20.7, 576 Fact 8.20.8, 576 Fact 8.21.8, 579 star-dagger matrix generalized inverse Fact 6.3.12, 406 state convergence detectability Fact 12.20.2, 864 discrete-time time-varying system Fact 11.21.19, 785 state equation definition, 795 matrix exponential Proposition 12.1.1, 795 variation of constants formula Proposition 12.1.1, 795 state transition matrix time-varying dynamics Fact 11.13.5, 744 statement definition, 1 Stein equation discrete-time Lyapunov equation Fact 11.21.17, 785 Fact 11.21.18, 785 step function, 796 step response

strongly decreasing function definition, 797 Lyapunov-stable matrix Fact 12.20.1, 863 step-down matrix resultant Fact 4.8.4, 277 Stephanos eigenvector of a Kronecker product Fact 7.4.22, 446 Stewart regular pencil Fact 5.17.3, 392 stiffness definition, 718 stiffness matrix partitioned matrix Fact 5.12.21, 368 Stirling matrix Vandermonde matrix Fact 5.16.3, 387 Stirling’s formula factorial Fact 1.11.20, 29 Storey asymptotic stability of a tridiagonal matrix Fact 11.18.24, 770

definition Definition 8.6.14, 479 positive-definite matrix Fact 8.14.15, 548 Fact 8.14.16, 549 trace Fact 8.14.16, 549 transformation Fact 1.10.2, 23 strictly dissipative matrix dissipative matrix Fact 8.9.32, 498 strictly lower triangular matrix definition Definition 3.1.3, 181 matrix power Fact 3.18.7, 235 matrix product Fact 3.22.2, 240 strictly proper rational function definition Definition 4.7.1, 271 strictly proper rational transfer function definition Definition 4.7.2, 271

strengthening definition, 2

strictly upper triangular matrix definition Definition 3.1.3, 181 Lie algebra Fact 3.23.11, 247 Fact 11.22.1, 786 matrix power Fact 3.18.7, 235 matrix product Fact 3.22.2, 240

strictly concave function definition Definition 8.6.14, 479

strong Kronecker product Kronecker product, 458

strictly convex function

strong log majorization

Stormer Schatten norm for positivesemidefinite matrices Fact 9.9.22, 640

1121

convex function Fact 2.21.8, 176 definition Definition 2.1.1, 85 matrix exponential Fact 11.16.4, 760 singular value inequality Fact 9.13.18, 662 strong majorization convex function Fact 2.21.7, 176 Fact 2.21.10, 177 convex hull Fact 3.9.6, 201 definition Definition 2.1.1, 85 diagonal entry Fact 8.18.8, 561 doubly stochastic matrix Fact 3.9.6, 201 eigenvalue Corollary 8.6.19, 486 Fact 8.19.4, 565 Fact 8.19.30, 574 entropy Fact 2.21.6, 176 Hermitian matrix Fact 8.18.8, 561 linear interpolation Fact 3.9.6, 201 Muirhead’s theorem Fact 2.21.5, 176 ones vector Fact 2.21.1, 175 Schur concave function Fact 2.21.6, 176 Schur convex function Fact 2.21.4, 176 Fact 2.21.5, 176 strongly connected graph definition, 84 strongly decreasing function

1122

strongly decreasing function

definition Definition 8.6.12, 478 strongly increasing function definition Definition 8.6.12, 478 determinant Proposition 8.6.13, 478 matrix functions Proposition 8.6.13, 478 structured matrix positive-semidefinite matrix Fact 8.8.2, 489 Fact 8.8.3, 489 Fact 8.8.4, 490 Fact 8.8.5, 491 Fact 8.8.6, 491 Fact 8.8.7, 491 Fact 8.8.8, 491 Fact 8.8.9, 492 Fact 8.8.10, 492 Fact 8.8.11, 492 Fact 8.8.12, 492 Styan difference of idempotent matrices Fact 5.12.19, 367 Hermitian matrix inertia equality Fact 8.10.15, 503 rank of a tripotent matrix Fact 2.10.23, 127 rank of an idempotent matrix Fact 3.12.27, 220 SU(2) quaternions Fact 3.24.6, 250 subdeterminant asymptotically stable matrix Fact 11.19.1, 776 asymptotically stable polynomial

Fact 11.18.23, 770 definition, 114 inverse Fact 2.13.6, 140 Lyapunov-stable polynomial Fact 11.18.23, 770 positive-definite matrix Proposition 8.2.8, 464 Fact 8.13.18, 537 positive-semidefinite matrix Proposition 8.2.7, 463 Fact 8.13.12, 536 subdiagonal definition, 87 subdifferential convex function Fact 10.12.3, 702 subgraph Definition 1.6.3, 10 subgroup definition Definition 3.3.3, 186 group Fact 3.23.4, 243 Fact 3.23.6, 246 Lagrange Fact 3.23.6, 246 sublevel set convex set Fact 8.14.1, 543 submatrix complementary Fact 2.11.20, 135 defect Fact 2.11.20, 135 definition, 87 determinant Fact 2.14.1, 144 Hermitian matrix Theorem 8.4.5, 468 Corollary 8.4.6, 469 Lemma 8.4.4, 468 Fact 5.8.8, 335

inertia Fact 5.8.8, 335 Kronecker product Proposition 7.3.1, 444 lower Hessenberg matrix Fact 3.19.1, 237 M-matrix Fact 4.11.9, 302 positive-definite matrix Proposition 8.2.8, 464 positive-semidefinite matrix Proposition 8.2.7, 463 Fact 8.7.8, 488 Fact 8.13.37, 541 rank Proposition 4.3.5, 259 Proposition 4.7.7, 272 Fact 2.11.6, 131 Fact 2.11.17, 134 Fact 2.11.20, 135 Fact 2.11.21, 136 Fact 2.11.22, 136 Fact 3.19.1, 237 singular value Fact 9.14.10, 667 Smith form Proposition 4.3.5, 259 Smith-McMillan form Proposition 4.7.7, 272 tridiagonal Fact 3.19.1, 237 Z-matrix Fact 4.11.9, 302 submultiplicative norm commutator Fact 9.9.8, 637 compatible norms Proposition 9.3.1, 604 definition, 604 equi-induced norm Corollary 9.4.4, 608 Fact 9.8.46, 636 H2 norm Fact 12.22.20, 875 H¨ older norm

subspace dimension theorem Fact 9.9.20, 640 idempotent matrix Fact 9.8.6, 627 infinity norm Fact 9.9.1, 636 Fact 9.9.2, 636 matrix exponential Proposition 11.1.2, 708 Fact 11.15.8, 758 Fact 11.15.9, 758 Fact 11.16.8, 761 Fact 11.18.8, 767 Fact 11.18.9, 767 matrix norm Fact 9.9.4, 636 nonsingular matrix Fact 9.8.5, 627 positive-semidefinite matrix Fact 9.9.7, 637 Schur product Fact 9.8.41, 634 spectral radius Proposition 9.3.2, 604 Proposition 9.3.3, 605 Corollary 9.3.4, 605 Fact 9.8.4, 627 Fact 9.9.3, 636 square root Fact 9.8.32, 632 unitarily invariant norm Fact 9.8.41, 634 Fact 9.9.7, 637 subset closure Fact 10.9.1, 695 definition, 3 interior Fact 10.9.1, 695 subset operation induced partial ordering Fact 1.7.19, 15 transitivity Fact 1.7.19, 15 subspace affine

definition, 97 affine subspace Fact 2.9.8, 120 closed set Fact 10.8.21, 694 common eigenvector Fact 5.14.26, 373 complementary Fact 2.9.18, 121 Fact 2.9.23, 122 complex conjugate transpose Fact 2.9.28, 123 definition, 97 dimension Fact 2.9.20, 122 Fact 2.9.21, 122 Fact 2.9.22, 122 dimension inequality Fact 2.10.4, 125 gap topology Fact 10.9.19, 698 image under linear mapping Fact 2.9.26, 123 inclusion Fact 2.9.11, 120 Fact 2.9.14, 121 Fact 2.9.28, 123 inclusion and dimension ordering Lemma 2.3.4, 99 inner product Fact 10.9.13, 696 intersection Fact 2.9.9, 120 Fact 2.9.16, 121 Fact 2.9.17, 121 Fact 2.9.29, 123 Fact 2.9.30, 124 left inverse Fact 2.9.26, 123 minimal principal angle Fact 5.11.39, 358 Fact 5.12.17, 366 Fact 10.9.19, 698 orthogonal complement

1123

Proposition 3.5.2, 190 Fact 2.9.14, 121 Fact 2.9.16, 121 Fact 2.9.18, 121 Fact 2.9.27, 123 orthogonal matrix Fact 3.11.1, 205 Fact 3.11.2, 205 principal angle Fact 2.9.19, 122 projector Proposition 3.5.1, 190 Proposition 3.5.2, 190 Fact 9.8.3, 627 Fact 10.9.18, 698 quadratic form Fact 8.15.28, 554 Fact 8.15.29, 555 range Proposition 3.5.1, 190 Fact 2.9.24, 122 span Fact 2.9.13, 121 span of image Fact 2.9.26, 123 sum Fact 2.9.9, 120 Fact 2.9.13, 121 Fact 2.9.16, 121 Fact 2.9.17, 121 Fact 2.9.29, 123 Fact 2.9.30, 124 union Fact 2.9.11, 120 Fact 2.9.13, 121 unitary matrix Fact 3.11.1, 205 Fact 3.11.2, 205 subspace decomposition spectrum Proposition 5.5.7, 323 subspace dimension theorem dimension Theorem 2.3.1, 98 rank Fact 2.11.9, 132

1124

subspace dimension theorem

Fact 2.11.10, 132 subspace intersection inverse image Fact 2.9.30, 124 subspace sum inverse image Fact 2.9.30, 124 sufficiency definition, 1 sum Drazin generalized inverse Fact 6.6.5, 431 Fact 6.6.6, 432 eigenvalue Fact 5.12.2, 362 Fact 5.12.3, 363 generalized inverse Fact 6.4.37, 419 Fact 6.4.38, 419 Fact 6.4.39, 420 Fact 6.4.40, 420 Fact 6.4.41, 420 Hamiltonian matrix Fact 3.20.5, 239 outer-product matrix Fact 2.10.24, 128 projector Fact 3.13.25, 229 Fact 5.12.17, 366 singular value Fact 9.14.20, 670 Fact 9.14.21, 670 Fact 9.14.25, 671 spectral radius Fact 5.12.2, 362 Fact 5.12.3, 363 sum inequality power inequality Fact 1.18.28, 73 Fact 1.18.29, 73 sum of integer powers inequality Fact 1.11.31, 32 matrix exponential

Fact 11.11.4, 736 sum of matrices determinant Fact 5.12.12, 365 Fact 9.14.18, 669 idempotent matrix Fact 3.12.22, 218 Fact 3.12.25, 219 Fact 5.19.7, 395 Fact 5.19.8, 395 Fact 5.19.9, 395 inverse matrix Corollary 2.8.10, 119 Kronecker product Proposition 7.1.4, 440 nilpotent matrix Fact 3.17.10, 233 projector Fact 3.13.23, 228 Fact 5.19.4, 394 sum of orthogonal matrices determinant Fact 3.11.17, 207 sum of powers Carlson inequality Fact 1.17.42, 64 Copson inequality Fact 1.17.44, 64 Hardy inequality Fact 1.17.43, 64 sum of products Hardy-Hilbert inequality Fact 1.18.13, 69 Fact 1.18.14, 69 inequality Fact 1.17.20, 58 sum of products inequality Hardy-Littlewood rearrangement inequality Fact 1.18.4, 66 Fact 1.18.5, 67 sum of projectors

Cochran’s theorem Fact 3.13.25, 229 sum of sets convex set Fact 2.9.1, 119 Fact 2.9.2, 119 Fact 10.9.5, 696 Fact 10.9.6, 696 Fact 10.9.8, 696 dual cone Fact 2.9.5, 120 sum of squares square root Fact 2.18.8, 163 sum of subspaces subspace dimension theorem Theorem 2.3.1, 98 sum of transfer functions H2 norm Proposition 12.11.6, 840 sum-of-squares inequality square-of-sum inequality Fact 1.17.14, 53 summation equality Fact 1.9.3, 20 Fact 1.9.4, 20 Fact 1.9.5, 20 superdiagonal definition, 87 supermultiplicativity induced lower bound Proposition 9.5.6, 615 support of a relation definition Definition 1.5.4, 7 surjective function definition, 84 Sylvester matrix

symmetric set coprime polynomials Fact 4.8.4, 277 Sylvester’s equation controllability Fact 12.21.14, 870 controllability matrix Fact 12.21.13, 869 linear matrix equation Proposition 7.2.4, 443 Proposition 11.9.3, 731 Fact 5.10.20, 348 Fact 5.10.21, 349 Fact 6.5.7, 424 nonsingular matrix Fact 12.21.14, 870 observability Fact 12.21.14, 870 observability matrix Fact 12.21.13, 869 partitioned matrix Fact 5.10.20, 348 Fact 5.10.21, 349 Fact 6.5.7, 424 rank Fact 12.21.13, 869 similar matrices Corollary 7.2.5, 444 Fact 7.5.14, 452 spectrum Corollary 7.2.5, 444 Fact 7.5.14, 452 Sylvester’s identity determinant Fact 2.14.1, 144 Sylvester’s inequality rank of a product, 106 Sylvester’s law of inertia definition, 320 Ostrowski Fact 5.8.17, 336 Sylvester’s law of nullity defect

Fact 2.10.15, 126 symmetric cone induced by symmetric relation Proposition 2.3.6, 101 symmetric gauge function unitarily invariant norm Fact 9.8.42, 634 weak majorization Fact 2.21.13, 177 symmetric graph adjacency matrix Fact 3.21.1, 240 cycle Fact 1.8.5, 15 degree matrix Fact 3.21.1, 240 forest Fact 1.8.5, 15 Laplacian Fact 3.21.1, 240 Laplacian matrix Fact 8.15.1, 550 symmetric group group Fact 3.23.4, 243 symmetric hull definition Definition 1.5.4, 7 relation Proposition 1.5.5, 7 symmetric matrix, see Hermitian matrix congruent matrices Fact 5.9.18, 341 definition Definition 3.1.1, 179 eigenvalue Fact 4.10.3, 288 factorization Corollary 5.3.9, 318 Fact 5.15.24, 381 Hankel matrix Fact 3.18.2, 234

1125

Hermitian matrix Fact 3.7.9, 193 involutory matrix Fact 5.15.36, 384 linear matrix equation Fact 3.7.3, 193 matrix power Fact 3.7.4, 193 matrix transpose Fact 3.7.2, 192 maximum eigenvalue Fact 5.12.20, 367 minimum eigenvalue Fact 5.12.20, 367 orthogonally similar matrices Fact 5.9.17, 341 partitioned matrix Fact 3.11.10, 206 quadratic form Fact 3.7.5, 193 similar matrices Fact 5.15.39, 384 similarity transformation Fact 5.15.2, 377 Fact 5.15.3, 377 skew-Hermitian matrix Fact 3.7.9, 193 skew-symmetric matrix Fact 5.9.18, 341 Fact 5.15.39, 384 trace Fact 5.12.8, 364 symmetric relation definition Definition 1.5.2, 6 graph Definition 1.6.2, 9 intersection Proposition 1.5.3, 7 symmetric cone induced by Proposition 2.3.6, 101 symmetric set

1126

symmetric set

definition, 97

Fact 3.20.3, 238

symmetry groups group Fact 3.23.4, 243

symplectic similarity Hamiltonian matrix Fact 3.20.4, 239

symplectic group determinant Fact 3.20.11, 239 quaternion group Fact 3.24.4, 249 special orthogonal group Fact 3.24.5, 250 unitary group Fact 3.23.10, 247

Szasz’s inequality positive-semidefinite matrix Fact 8.13.37, 541

symplectic matrix Cayley transform Fact 3.20.12, 239 definition Definition 3.1.5, 183 determinant Fact 3.20.10, 239 Fact 3.20.11, 239 equality Fact 3.20.1, 238 group Proposition 3.3.6, 187 Hamiltonian matrix Fact 3.20.2, 238 Fact 3.20.12, 239 Fact 3.20.13, 240 identity matrix Fact 3.20.3, 238 matrix exponential Proposition 11.6.7, 724 matrix logarithm Fact 11.14.19, 751 partitioned matrix Fact 3.20.9, 239 reverse permutation matrix Fact 3.20.3, 238 skew-involutory matrix Fact 3.20.2, 238 spectrum Proposition 5.5.20, 326 unit imaginary matrix

T T-congruence complex-symmetric matrix Fact 5.9.24, 342 T-congruent diagonalization complex-symmetric matrix Fact 5.9.24, 342 T-congruent matrices definition Definition 3.4.4, 188 Tao H¨ older-induced norm Fact 9.8.19, 630 Taussky-Todd factorization Fact 5.15.8, 378 tautology definition, 1 tetrahedral group group Fact 3.23.4, 243 tetrahedron volume Fact 2.20.15, 173 theorem definition, 1 thermodynamic inequality matrix exponential

Fact 11.14.31, 754 relative entropy Fact 11.14.25, 752 Tian idempotent matrix and similar matrices Fact 5.10.22, 349 range of a partitioned matrix Fact 6.5.3, 423 Tikhonov inverse positive-definite matrix Fact 8.9.41, 500 time-varying dynamics commuting matrices Fact 11.13.4, 743 determinant Fact 11.13.4, 743 matrix differential equation Fact 11.13.4, 743 Fact 11.13.5, 744 state transition matrix Fact 11.13.5, 744 trace Fact 11.13.4, 743 Toeplitz matrix block-Toeplitz matrix Fact 3.18.3, 234 definition Definition 3.1.3, 181 determinant Fact 2.13.13, 142 Fact 3.18.9, 235 Hankel matrix Fact 3.18.1, 234 inverse matrix Fact 3.18.10, 236 Fact 3.18.11, 236 lower triangular matrix Fact 3.18.7, 235 Fact 11.13.1, 743

trace nilpotent matrix Fact 3.18.6, 235 polynomial multiplication Fact 4.8.10, 280 positive-definite matrix Fact 8.13.14, 536 reverse-symmetric matrix Fact 3.18.5, 234 spectrum Fact 4.10.16, 293 Fact 5.11.43, 360 Fact 5.11.44, 360 Fact 8.9.35, 499 tridiagonal matrix Fact 3.18.9, 235 Fact 3.18.10, 236 Fact 5.11.43, 360 Fact 5.11.44, 360 upper triangular matrix Fact 3.18.7, 235 Fact 11.13.1, 743 Tomiyama maximum singular value of a partitioned matrix Fact 9.14.12, 668 total least squares linear system solution Fact 9.15.1, 679 total ordering definition Definition 1.5.9, 8 dictionary ordering Fact 1.7.8, 13 lexicographic ordering Fact 1.7.8, 13 planar example Fact 1.7.8, 13 total response, 797 totally nonnegative matrix

definition Fact 11.18.23, 770 totally positive matrix rank Fact 8.7.8, 488 tournament graph Fact 1.8.6, 16 Hamiltonian cycle Fact 1.8.6, 16 spanning path Fact 1.8.6, 16 trace 2 × 2 matrices Fact 2.12.9, 137 2 × 2 matrix equality Fact 4.9.3, 283 Fact 4.9.4, 283 3 × 3 matrix equality Fact 4.9.5, 283 Fact 4.9.6, 284 adjugate Fact 4.9.8, 284 asymptotically stable matrix Fact 11.18.31, 773 commutator Fact 2.18.1, 161 Fact 2.18.2, 161 Fact 5.9.20, 341 complex conjugate transpose Fact 8.12.4, 524 Fact 8.12.5, 524 Fact 9.13.15, 662 convex function Proposition 8.6.17, 480 Fact 8.14.17, 549 definition, 94 derivative Proposition 10.7.4, 692 Fact 11.14.3, 748 determinant Proposition 8.4.14, 471 Corollary 11.2.4, 712 Corollary 11.2.5, 712 Fact 2.13.16, 143 Fact 8.12.1, 523

1127

Fact 8.13.21, 537 Fact 11.14.20, 751 dimension Fact 2.18.11, 163 eigenvalue Proposition 8.4.13, 471 Fact 5.11.11, 351 Fact 8.18.5, 560 Fact 8.19.19, 570 eigenvalue bound Fact 5.11.45, 361 elementary projector Fact 5.8.11, 335 elementary reflector Fact 5.8.11, 335 equalities, 95 Frobenius norm Fact 9.11.3, 654 Fact 9.11.4, 654 Fact 9.11.5, 655 Fact 9.12.2, 656 generalized inverse Fact 6.3.21, 408 group generalized inverse Fact 6.6.7, 434 Hamiltonian matrix Fact 3.20.7, 239 Hermitian matrix Proposition 8.4.13, 471 Corollary 8.4.10, 470 Lemma 8.4.12, 471 Fact 3.7.13, 195 Fact 3.7.22, 197 Fact 8.12.18, 526 Fact 8.12.40, 532 Hermitian matrix product Fact 5.12.4, 363 Fact 5.12.5, 363 Fact 8.12.6, 524 Fact 8.12.7, 524 Fact 8.12.8, 524 Fact 8.12.16, 526 Fact 8.19.19, 570 idempotent matrix Fact 5.8.1, 334 Fact 5.11.7, 350 inequality

1128

trace

Fact 5.12.9, 364 involutory matrix Fact 5.8.2, 334 Klein’s inequality Fact 11.14.25, 752 Kronecker permutation matrix Fact 7.4.30, 447 Kronecker product Proposition 7.1.12, 442 Fact 11.14.38, 755 Kronecker sum Fact 11.14.36, 755 matrix exponential Corollary 11.2.4, 712 Corollary 11.2.5, 712 Fact 8.14.18, 549 Fact 11.11.6, 737 Fact 11.14.3, 748 Fact 11.14.10, 749 Fact 11.14.28, 753 Fact 11.14.29, 754 Fact 11.14.30, 754 Fact 11.14.31, 754 Fact 11.14.36, 755 Fact 11.14.38, 755 Fact 11.15.4, 756 Fact 11.15.5, 756 Fact 11.16.1, 759 Fact 11.16.4, 760 matrix logarithm Fact 11.14.24, 752 Fact 11.14.25, 752 Fact 11.14.27, 753 Fact 11.14.31, 754 matrix power Fact 2.12.13, 137 Fact 2.12.17, 138 Fact 4.10.23, 295 Fact 4.11.23, 306 Fact 5.11.9, 351 Fact 5.11.10, 351 Fact 8.12.4, 524 Fact 8.12.5, 524 matrix product Fact 2.12.16, 138 Fact 5.12.6, 364 Fact 5.12.7, 364 Fact 8.12.14, 525

Fact 8.12.15, 526 Fact 9.14.3, 665 Fact 9.14.4, 665 matrix squared Fact 5.11.9, 351 Fact 5.11.10, 351 maximum singular value Fact 5.12.7, 364 Fact 9.14.4, 665 maximum singular value bound Fact 9.13.13, 661 nilpotent matrix Fact 3.17.6, 232 nonnegative matrix Fact 4.11.23, 306 normal matrix Fact 3.7.12, 194 Fact 8.12.5, 524 normal matrix product Fact 5.12.4, 363 orthogonal matrix Fact 3.11.12, 206 Fact 3.11.13, 206 Fact 5.12.9, 364 Fact 5.12.10, 365 outer-product matrix Fact 5.14.3, 370 partitioned matrix Fact 8.12.37, 531 Fact 8.12.41, 532 Fact 8.12.42, 532 Fact 8.12.43, 532 Fact 8.12.44, 533 polarized Cayley-Hamilton theorem Fact 4.9.3, 283 positive-definite matrix Proposition 8.4.14, 471 Fact 8.9.16, 497 Fact 8.10.46, 510 Fact 8.10.47, 511 Fact 8.11.10, 516 Fact 8.12.1, 523

Fact 8.12.2, 523 Fact 8.12.25, 528 Fact 8.12.28, 529 Fact 8.12.39, 532 Fact 8.13.13, 536 Fact 11.14.24, 752 Fact 11.14.25, 752 Fact 11.14.27, 753 positive-semidefinite matrix Proposition 8.4.13, 471 Fact 8.12.3, 523 Fact 8.12.9, 524 Fact 8.12.10, 524 Fact 8.12.11, 525 Fact 8.12.12, 525 Fact 8.12.13, 525 Fact 8.12.17, 526 Fact 8.12.18, 526 Fact 8.12.19, 527 Fact 8.12.20, 527 Fact 8.12.21, 527 Fact 8.12.22, 528 Fact 8.12.23, 528 Fact 8.12.24, 528 Fact 8.12.25, 528 Fact 8.12.27, 529 Fact 8.12.29, 529 Fact 8.12.30, 529 Fact 8.12.35, 531 Fact 8.12.36, 531 Fact 8.12.37, 531 Fact 8.12.40, 532 Fact 8.12.41, 532 Fact 8.12.42, 532 Fact 8.12.43, 532 Fact 8.13.21, 537 Fact 8.19.16, 569 Fact 8.19.21, 570 Fact 8.21.3, 578 Fact 8.21.17, 580 projector Fact 5.8.11, 335 Fact 8.12.38, 532 quadruple product Fact 7.4.10, 445 rank Fact 5.11.10, 351 Fact 9.11.4, 654

transfer function reflector Fact 5.8.11, 335 rotation matrix Fact 3.11.12, 206 Schatten norm Fact 9.12.1, 656 Schur product Fact 8.22.17, 587 Fact 9.14.34, 675 singular value Fact 5.12.6, 364 Fact 8.18.2, 559 Fact 9.12.1, 656 Fact 9.13.15, 662 Fact 9.14.3, 665 Fact 9.14.34, 675 skew-Hermitian matrix Fact 3.7.24, 197 skew-Hermitian matrix product Fact 8.12.6, 524 skew-symmetric matrix Fact 3.7.23, 197 Fact 3.7.31, 198 spectral radius Fact 4.10.23, 295 Fact 5.11.46, 361 Fact 9.13.12, 661 spectrum Fact 4.10.8, 290 square Fact 8.18.7, 561 strictly convex function Fact 8.14.16, 549 symmetric matrix Fact 5.12.8, 364 time-varying dynamics Fact 11.13.4, 743 trace norm Fact 9.11.2, 653 triple product Fact 2.12.11, 137 Fact 7.4.8, 445 tripotent matrix Fact 3.16.4, 231

Fact 5.8.3, 334 unitarily similar matrices Fact 5.10.8, 346 unitary matrix Fact 3.11.11, 206 Fact 3.11.24, 209 vec Proposition 7.1.1, 439 Fact 7.4.8, 445 Fact 7.4.10, 445 zero matrix Fact 2.12.14, 137 Fact 2.12.15, 137 trace and singular value von Neumann’s trace inequality Fact 9.12.1, 656 trace norm compatible norms Corollary 9.3.8, 607 definition, 603 Frobenius norm Fact 9.9.11, 637 matrix difference Fact 9.9.24, 640 maximum singular value Corollary 9.3.8, 607 positive-semidefinite matrix Fact 9.9.15, 638 Schur product Fact 9.14.35, 676 trace Fact 9.11.2, 653 trace of a convex function Berezin Fact 8.12.34, 531 Brown Fact 8.12.34, 531 Hansen Fact 8.12.34, 531 Kosaki Fact 8.12.34, 531 Pedersen

1129

Fact 8.12.34, 531 trace of a Hermitian matrix product Fan Fact 5.12.4, 363 trace of a product Fan Fact 5.12.10, 365 traceable graph definition Definition 1.6.3, 10 Tracy-Singh product Kronecker product, 458 trail definition Definition 1.6.3, 10 transfer function cascade interconnection Proposition 12.13.2, 843 derivative Fact 12.22.6, 872 feedback interconnection Fact 12.22.8, 873 frequency response Fact 12.22.5, 872 H2 norm Fact 12.22.16, 875 Fact 12.22.17, 875 Fact 12.22.18, 875 Fact 12.22.19, 875 imaginary part Fact 12.22.5, 872 Jordan form Fact 12.22.10, 873 parallel interconnection Proposition 12.13.2, 843 partitioned transfer function Fact 12.22.7, 873 real part

1130

transfer function

Fact 12.22.5, 872 realization Fact 12.22.3, 872 Fact 12.22.4, 872 Fact 12.22.6, 872 Fact 12.22.7, 873 Fact 12.22.8, 873 realization of inverse Proposition 12.13.1, 842 realization of parahermitian conjugate Proposition 12.13.1, 842 realization of transpose Proposition 12.13.1, 842 reciprocal argument Fact 12.22.4, 872 right inverse Fact 12.22.9, 873 shifted argument Fact 12.22.3, 872 transitive hull definition Definition 1.5.4, 7 relation Proposition 1.5.5, 7 transitive relation convex cone induced by Proposition 2.3.6, 101 definition Definition 1.5.2, 6 graph Definition 1.6.2, 9 intersection Proposition 1.5.3, 7 positive-semidefinite matrix Proposition 8.1.1, 460 transmission zero definition Definition 4.7.10, 273 Definition 4.7.13, 274 invariant zero

Theorem 12.10.8, 834 Theorem 12.10.9, 835 null space Fact 4.8.16, 282 rank Proposition 4.7.12, 273 transpose controllability Fact 12.20.16, 865 diagonalizable matrix Fact 5.14.4, 370 involutory matrix Fact 5.9.9, 340 Kronecker permutation matrix Proposition 7.1.13, 442 Kronecker product Proposition 7.1.3, 440 matrix exponential Proposition 11.2.8, 713 normal matrix Fact 5.9.11, 340 Fact 5.9.12, 340 similar matrices Proposition 5.5.12, 324 Corollary 4.3.11, 261 Corollary 5.3.8, 317 Corollary 5.5.21, 327 Fact 5.9.11, 340 Fact 5.9.12, 340 transpose of a matrix definition, 94 transpose of a vector definition, 92 transposition matrix definition Definition 3.1.1, 179 permutation matrix Fact 3.23.2, 242 tree definition Definition 1.6.3, 10 triangle area Fact 2.20.7, 168

Fact 2.20.8, 168 Fact 2.20.10, 169 Bandila’s inequality Fact 2.20.11, 169 cosine rule Fact 2.20.11, 169 cross product Fact 2.20.10, 169 Euler’s inequality Fact 2.20.11, 169 fundamental triangle inequality Fact 2.20.11, 169 Heron’s formula Fact 2.20.11, 169 inequality Fact 1.13.17, 47 Klamkin’s inequality Fact 2.20.11, 169 Mircea’s inequality Fact 2.20.11, 169 semiperimeter Fact 2.20.11, 169 sine rule Fact 2.20.11, 169 triangle inequality Blundon Fact 2.20.11, 169 definition Definition 9.1.1, 597 equality Fact 9.7.3, 618 Frobenius norm Fact 9.9.13, 638 linear dependence Fact 9.7.3, 618 positive-semidefinite matrix Fact 9.9.21, 640 Satnoianu Fact 2.20.11, 169 triangular matrix nilpotent matrix Fact 5.17.6, 392 triangularization commutator Fact 5.17.5, 392 commuting matrices

uncontrollable eigenvalue Fact 5.17.4, 392 tridiagonal submatrix Fact 3.19.1, 237 tridiagonal matrix asymptotically stable matrix Fact 11.18.24, 770 Fact 11.18.25, 770 Fact 11.18.26, 771 Fact 11.18.27, 771 Fact 11.18.28, 772 cyclic matrix Fact 11.18.25, 770 definition Definition 3.1.3, 181 determinant Fact 3.18.9, 235 Fact 3.19.2, 237 Fact 3.19.3, 238 Fact 3.19.4, 238 Fact 3.19.5, 238 inverse matrix Fact 3.18.10, 236 Fact 3.19.4, 238 Fact 3.19.5, 238 positive-definite matrix Fact 8.8.18, 494 Routh form Fact 11.18.27, 771 Schwarz form Fact 11.18.25, 770 Fact 11.18.26, 771 spectrum Fact 5.11.40, 359 Fact 5.11.41, 359 Fact 5.11.42, 359 Fact 5.11.43, 360 Fact 5.11.44, 360 Toeplitz matrix Fact 3.18.9, 235 Fact 3.18.10, 236 Fact 5.11.43, 360 Fact 5.11.44, 360 trigonometric equalities Fact 1.21.2, 82

Fact 1.21.1, 81 trigonometric inequality Huygens’s inequality Fact 1.11.29, 30 Jordan’s inequality Fact 1.11.29, 30 scalar Fact 1.11.29, 30 Fact 1.11.30, 32 Fact 1.12.29, 38 trigonometric matrix rank Fact 3.18.8, 235 triple product equality Fact 2.12.10, 137 Kronecker product Proposition 7.1.5, 440 Fact 7.4.8, 445 trace Fact 4.9.4, 283 Fact 4.9.6, 284 Fact 7.4.8, 445 vec Proposition 7.1.9, 441 tripotent matrix definition Definition 3.1.1, 179 Drazin generalized inverse Proposition 6.2.2, 402 group-invertible matrix Proposition 3.1.6, 183 Hermitian matrix Fact 3.16.3, 231 idempotent matrix Fact 3.16.1, 231 Fact 3.16.5, 231 inertia Fact 5.8.3, 334 involutory matrix Fact 3.16.2, 231 Kronecker product Fact 7.4.17, 446 projector

1131

Fact 6.4.36, 419 rank Fact 2.10.23, 127 Fact 3.16.3, 231 Fact 3.16.4, 231 reflector Proposition 3.1.6, 183 signature Fact 5.8.3, 334 similar matrices Proposition 3.4.5, 189 Corollary 5.5.21, 327 spectrum Proposition 5.5.20, 326 trace Fact 3.16.4, 231 Fact 5.8.3, 334 unitarily similar matrices Proposition 3.4.5, 189 tuple definition, 3 Turan’s inequalities spectral radius bound Fact 4.10.23, 295 two-sided directional differential definition, 686

U ULU decomposition factorization Fact 5.15.11, 379 Umegaki relative entropy Fact 11.14.25, 752 uncontrollable eigenvalue controllability pencil Proposition 12.6.13, 813 full-state feedback Proposition 12.6.14, 813

1132

uncontrollable eigenvalue

Hamiltonian Proposition 12.17.7, 857 Proposition 12.17.8, 857 Lemma 12.17.4, 855 Lemma 12.17.6, 856 uncontrollable multispectrum definition Definition 12.6.11, 812 uncontrollable spectrum controllability pencil Proposition 12.6.16, 814 definition Definition 12.6.11, 812 invariant zero Theorem 12.10.9, 835 Smith zeros Proposition 12.6.16, 814 uncontrollableunobservable spectrum invariant zero Theorem 12.10.9, 835 unimodular matrix coprime right polynomial fraction description Proposition 4.7.15, 275 definition Definition 4.3.1, 258 determinant Proposition 4.3.7, 260 Smith form Proposition 4.3.7, 260 union boundary Fact 10.9.2, 695 cardinality Fact 1.7.5, 12 closed set Fact 10.9.11, 696 closure

Fact 10.9.2, 695 convex cone Fact 2.9.10, 120 convex set Fact 10.9.8, 696 definition, 2 interior Fact 10.9.2, 695 Fact 10.9.3, 695 open set Fact 10.9.10, 696 span Fact 2.9.12, 121 union of ranges projector Fact 6.4.46, 422 unipotent matrix definition Definition 3.1.1, 179 group Fact 3.23.12, 247 Fact 11.22.1, 786 Heisenberg group Fact 3.23.12, 247 Fact 11.22.1, 786 matrix exponential Fact 11.13.18, 746 spectrum Proposition 5.5.20, 326 unit imaginary matrix congruent matrices Fact 3.7.34, 198 definition, 183 Hamiltonian matrix Fact 3.20.3, 238 skew-symmetric matrix Fact 3.7.34, 198 spectrum Fact 5.9.27, 343 symplectic matrix Fact 3.20.3, 238 unit impulse function definition, 796 unit sphere group Fact 3.23.9, 246

unit-length quaternions Sp(1) Fact 3.24.1, 247 unitarily biequivalent matrices definition Definition 3.4.3, 188 singular values Fact 5.10.18, 348 unitarily invariant norm commutator Fact 9.9.29, 641 Fact 9.9.30, 641 Fact 9.9.31, 642 complex conjugate transpose Fact 9.8.30, 632 definition, 602 fixed-rank approximation Fact 9.14.28, 672 Frobenius norm Fact 9.14.33, 675 Heinz inequality Fact 9.9.49, 646 Hermitian matrix Fact 9.9.5, 636 Fact 9.9.41, 644 Fact 9.9.43, 645 Fact 11.16.13, 762 Hermitian perturbation Fact 9.12.4, 657 inequality Fact 9.9.11, 637 Fact 9.9.44, 645 Fact 9.9.47, 646 Fact 9.9.48, 646 Fact 9.9.49, 646 Fact 9.9.50, 646 matrix exponential Fact 11.15.6, 757 Fact 11.16.4, 760 Fact 11.16.5, 761 Fact 11.16.13, 762 Fact 11.16.16, 762 Fact 11.16.17, 763 matrix logarithm

unitarily similar matrices Fact 9.9.54, 647 matrix power Fact 9.9.17, 639 matrix product Fact 9.9.6, 636 maximum eigenvalue Fact 9.9.30, 641 Fact 9.9.31, 642 maximum singular value Fact 9.9.10, 637 Fact 9.9.29, 641 McIntosh’s inequality Fact 9.9.47, 646 normal matrix Fact 9.9.6, 636 outer-product matrix Fact 9.8.40, 634 partitioned matrix Fact 9.8.33, 632 polar decomposition Fact 9.9.42, 645 positive-semidefinite matrix Fact 9.9.7, 637 Fact 9.9.14, 638 Fact 9.9.15, 638 Fact 9.9.16, 639 Fact 9.9.17, 639 Fact 9.9.27, 641 Fact 9.9.46, 645 Fact 9.9.51, 646 Fact 9.9.52, 647 Fact 9.9.53, 647 Fact 9.9.54, 647 Fact 11.16.16, 762 Fact 11.16.17, 763 properties Fact 9.8.41, 634 rank Fact 9.14.28, 672 Schatten norm Fact 9.8.9, 628 Schur product Fact 9.8.41, 634 Fact 9.9.62, 649 Fact 9.9.63, 649

Fact 9.14.37, 676 self-adjoint norm Fact 9.8.7, 627 similar matrices Fact 9.8.31, 632 singular value Fact 9.14.28, 672 singular value perturbation Fact 9.14.29, 673 square root Fact 9.9.18, 639 Fact 9.9.19, 639 submultiplicative norm Fact 9.8.41, 634 Fact 9.9.7, 637 symmetric gauge function Fact 9.8.42, 634 unitarily left-equivalent matrices complex conjugate transpose Fact 5.10.18, 348 Fact 5.10.19, 348 definition Definition 3.4.3, 188 positive-semidefinite matrix Fact 5.10.18, 348 Fact 5.10.19, 348 unitarily right-equivalent matrices complex conjugate transpose Fact 5.10.18, 348 definition Definition 3.4.3, 188 positive-semidefinite matrix Fact 5.10.18, 348 unitarily similar matrices biequivalent matrices Proposition 3.4.5, 189

1133

complex conjugate transpose Fact 5.9.22, 342 Fact 5.9.23, 342 definition Definition 3.4.4, 188 diagonal entry Fact 5.9.19, 341 Fact 5.9.21, 341 elementary matrix Proposition 5.5.22, 327 elementary projector Proposition 5.5.22, 327 elementary reflector Proposition 5.5.22, 327 group-invertible matrix Proposition 3.4.5, 189 Hermitian matrix Proposition 3.4.5, 189 Proposition 5.5.22, 327 Corollary 5.4.5, 320 idempotent matrix Proposition 3.4.5, 189 Fact 5.9.23, 342 Fact 5.9.28, 343 Fact 5.9.29, 343 Fact 5.10.10, 347 involutory matrix Proposition 3.4.5, 189 Kronecker product Fact 7.4.13, 446 matrix classes Proposition 3.4.5, 189 nilpotent matrix Proposition 3.4.5, 189 normal matrix Proposition 3.4.5, 189 Corollary 5.4.4, 319 Fact 5.10.6, 346 Fact 5.10.7, 346 partitioned matrix Fact 5.9.25, 342 positive-definite matrix Proposition 3.4.5, 189 Proposition 5.5.22, 327 positive-semidefinite matrix

1134

unitarily similar matrices

Proposition 3.4.5, 189 Proposition 5.5.22, 327 projector Fact 5.10.12, 347 range-Hermitian matrix Proposition 3.4.5, 189 Corollary 5.4.4, 319 similar matrices Fact 5.10.7, 346 simultaneous diagonalization Fact 5.17.7, 392 simultaneous triangularization Fact 5.17.4, 392 Fact 5.17.6, 392 skew-Hermitian matrix Proposition 3.4.5, 189 Proposition 5.5.22, 327 skew-involutory matrix Proposition 3.4.5, 189 trace Fact 5.10.8, 346 tripotent matrix Proposition 3.4.5, 189 unitary matrix Proposition 3.4.5, 189 Proposition 5.5.22, 327 upper triangular matrix Theorem 5.4.1, 318 unitary biequivalence equivalence relation Fact 5.10.3, 345 unitary group symplectic group Fact 3.23.10, 247 unitary left equivalence equivalence relation Fact 5.10.3, 345 unitary matrix, see orthogonal matrix additive decomposition

Fact 5.19.1, 394 block-diagonal matrix Fact 3.11.8, 205 Cayley transform Fact 3.11.22, 208 cogredient diagonalization Fact 8.17.1, 558 complex-symmetric matrix Fact 5.9.24, 342 convergent sequence Fact 8.9.34, 499 CS decomposition Fact 5.9.31, 344 definition Definition 3.1.1, 179 determinant Fact 3.11.15, 207 Fact 3.11.18, 207 Fact 3.11.19, 207 Fact 3.11.20, 208 diagonal entry Fact 3.11.14, 207 Fact 8.18.10, 562 diagonal matrix Theorem 5.6.3, 329 Lemma 8.5.1, 473 discrete-time Lyapunov-stable matrix Fact 11.21.15, 784 dissipative matrix Fact 8.9.32, 498 equalities Fact 3.11.4, 205 factorization Fact 5.15.9, 378 Fact 5.18.6, 393 Frobenius norm Fact 9.9.42, 645 geometric-mean decomposition Fact 5.9.32, 344 group Proposition 3.3.6, 187 group generalized inverse

Fact 6.3.33, 411 Hermitian matrix Fact 3.11.21, 208 Fact 8.17.1, 558 Fact 11.14.34, 755 involutory matrix Fact 3.14.2, 229 Kronecker product Fact 7.4.17, 446 matrix exponential Proposition 11.2.8, 713 Proposition 11.2.9, 715 Proposition 11.6.7, 724 Corollary 11.2.6, 712 Fact 11.14.6, 749 Fact 11.14.33, 755 Fact 11.14.34, 755 matrix limit Fact 6.3.33, 411 normal matrix Proposition 3.1.6, 183 Fact 3.11.5, 205 Fact 5.15.1, 377 orthogonal vectors Fact 3.11.7, 205 outer-product perturbation Fact 3.11.20, 208 partitioned matrix Fact 3.11.9, 206 Fact 3.11.10, 206 Fact 3.11.19, 207 Fact 8.11.22, 520 Fact 8.11.23, 520 Fact 8.11.24, 520 Fact 9.14.11, 667 polar decomposition Fact 5.18.8, 394 quaternions Fact 3.24.9, 252 reflector Fact 3.14.2, 229 semicontractive matrix Fact 8.11.22, 520 shifted-unitary matrix Fact 3.11.25, 209

upper block-triangular matrix simultaneous diagonalization Fact 8.17.1, 558 singular value Fact 5.11.37, 358 Fact 9.14.11, 667 skew reflector Fact 3.14.6, 230 skew-Hermitian matrix Fact 3.11.22, 208 Fact 3.14.6, 230 Fact 11.14.33, 755 skew-involutory matrix Fact 3.14.6, 230 spectrum Proposition 5.5.20, 326 square root Fact 8.9.27, 498 subspace Fact 3.11.1, 205 Fact 3.11.2, 205 sum Fact 3.11.18, 207 trace Fact 3.11.11, 206 Fact 3.11.24, 209 unitarily similar matrices Proposition 3.4.5, 189 Proposition 5.5.22, 327 upper triangular matrix Fact 5.15.9, 378 unitary right equivalence equivalence relation Fact 5.10.3, 345 unitary similarity equivalence relation Fact 5.10.3, 345 singular value decomposition Fact 5.9.30, 343 Fact 6.3.14, 407 Fact 6.6.16, 435 universal statement

definition, 2 logical equivalents Fact 1.7.4, 12 unobservable eigenvalue definition Definition 12.3.11, 803 full-state feedback Proposition 12.3.14, 803 Hamiltonian Proposition 12.17.7, 857 Proposition 12.17.8, 857 Lemma 12.17.4, 855 Lemma 12.17.6, 856 invariant zero Proposition 12.10.11, 837 observability pencil Proposition 12.3.13, 803 unobservable multispectrum definition Definition 12.3.11, 803 unobservable spectrum definition Definition 12.3.11, 803 invariant zero Theorem 12.10.9, 835 observability pencil Proposition 12.3.16, 803 Smith zeros Proposition 12.3.16, 803 unobservable subspace block-triangular matrix Proposition 12.3.9, 802 Proposition 12.3.10, 802 definition Definition 12.3.1, 800

1135

equivalent expressions Lemma 12.3.2, 800 full-state feedback Proposition 12.3.5, 801 identity-matrix shift Lemma 12.3.7, 802 invariant subspace Corollary 12.3.4, 801 nonsingular matrix Proposition 12.3.10, 802 orthogonal matrix Proposition 12.3.9, 802 projector Lemma 12.3.6, 801 unstable equilibrium definition Definition 11.7.1, 725 unstable matrix positive matrix Fact 11.18.20, 769 unstable subspace complementary subspaces Proposition 11.8.8, 729 definition, 729 idempotent matrix Proposition 11.8.8, 729 invariant subspace Proposition 11.8.8, 729 semistable matrix Proposition 11.8.8, 729 upper block-triangular matrix characteristic polynomial Fact 4.10.12, 291 definition Definition 3.1.3, 181 inverse matrix Fact 2.17.7, 161 Fact 2.17.9, 161 irreducible matrix Fact 4.11.5, 300 minimal polynomial Fact 4.10.13, 292

1136

upper block-triangular matrix

orthogonally similar matrices Corollary 5.4.2, 319 power Fact 2.12.21, 138 reducible matrix Fact 4.11.5, 300 upper bound positive-definite matrix Fact 8.10.29, 505 upper bound for a partial ordering definition Definition 1.5.9, 8 upper Hessenberg matrix definition Definition 3.1.3, 181 regular pencil Fact 5.17.3, 392 simultaneous orthogonal biequivalence transformation Fact 5.17.3, 392 upper triangular matrix, see lower triangular matrix characteristic polynomial Fact 4.10.10, 291 commutator Fact 3.17.11, 233 definition Definition 3.1.3, 181 determinant Fact 3.22.1, 240 eigenvalue Fact 4.10.10, 291 factorization Fact 5.15.9, 378 Fact 5.15.10, 378 group Fact 3.23.12, 247 Fact 11.22.1, 786 Heisenberg group

Fact 3.23.12, 247 Fact 11.22.1, 786 invariant subspace Fact 5.9.4, 339 Kronecker product Fact 7.4.4, 445 Lie algebra Fact 3.23.11, 247 Fact 11.22.1, 786 matrix exponential Fact 11.11.4, 736 Fact 11.13.1, 743 Fact 11.13.17, 746 matrix power Fact 3.18.7, 235 matrix product Fact 3.22.2, 240 nilpotent matrix Fact 3.17.11, 233 orthogonally similar matrices Corollary 5.4.3, 319 positive diagonal Fact 5.15.5, 377 positive-semidefinite matrix Fact 8.9.38, 500 regular pencil Fact 5.17.3, 392 similar matrices Fact 5.9.4, 339 simultaneous orthogonal biequivalence transformation Fact 5.17.3, 392 spectrum Fact 4.10.10, 291 Toeplitz matrix Fact 3.18.7, 235 Fact 11.13.1, 743 unitarily similar matrices Theorem 5.4.1, 318 unitary matrix Fact 5.15.9, 378 Urquhart generalized inverse Fact 6.3.13, 406

V Vandermonde matrix companion matrix Fact 5.16.4, 387 determinant Fact 5.16.3, 387 Fourier matrix Fact 5.16.7, 388 polynomial Fact 5.16.6, 388 similar matrices Fact 5.16.5, 388 variance Laguerre-Samuelson inequality Fact 1.17.12, 55 Fact 8.9.36, 499 variance inequality mean Fact 1.17.12, 55 Fact 8.9.36, 499 variation of constants formula state equation Proposition 12.1.1, 795 variational cone definition, 685 dimension Fact 10.8.20, 694 vec definition, 439 Kronecker permutation matrix Fact 7.4.30, 447 Kronecker product Fact 7.4.6, 445 Fact 7.4.7, 445 Fact 7.4.9, 445 matrix product Fact 7.4.7, 445 quadruple product Fact 7.4.10, 445 trace Proposition 7.1.1, 439 Fact 7.4.8, 445 Fact 7.4.10, 445

weak majorization triple product Proposition 7.1.9, 441 vector definition, 85 H¨ older norm Fact 9.7.34, 626 vector derivative quadratic form Proposition 10.7.1, 691 vector equality cosine law Fact 9.7.4, 618 parallelogram law Fact 9.7.4, 618 polarization identity Fact 9.7.4, 618 Pythagorean theorem Fact 9.7.4, 618 vector inequality H¨ older’s inequality Proposition 9.1.6, 599 norm inequality Fact 9.7.11, 622 Fact 9.7.12, 622 Fact 9.7.14, 623 Fact 9.7.15, 623 vibration equation matrix exponential Example 11.3.7, 717 volume convex polyhedron Fact 2.20.20, 174 ellipsoid Fact 3.7.35, 199 hyperellipsoid Fact 3.7.35, 199 parallelepiped Fact 2.20.16, 173 Fact 2.20.17, 173 simplex Fact 2.20.19, 174 tetrahedron Fact 2.20.15, 173 transformed set Fact 2.20.18, 174

von Neumann symmetric gauge function and unitarily invariant norm Fact 9.8.42, 634 von Neumann’s trace inequality trace and singular value Fact 9.12.1, 656 von Neumann–Jordan inequality norm inequality Fact 9.7.11, 622

W walk connected graph Fact 4.11.2, 297 definition Definition 1.6.3, 10 graph Fact 4.11.1, 297 Walker’s inequality scalar inequality Fact 1.13.22, 49 Walsh polynomial root bound Fact 11.20.5, 778 Wang’s inequality scalar inequality Fact 1.17.13, 56 weak diagonal dominance theorem nonsingular matrix Fact 4.10.20, 294 weak log majorization definition Definition 2.1.1, 85 eigenvalue Fact 8.19.28, 573 singular value

1137

Proposition 9.6.2, 615 weak majorization Fact 2.21.12, 177 weak majorization convex function Fact 2.21.7, 176 Fact 2.21.8, 176 Fact 2.21.9, 177 Fact 2.21.10, 177 Fact 8.19.5, 565 definition Definition 2.1.1, 85 eigenvalue Fact 8.18.5, 560 Fact 8.19.5, 565 Fact 8.19.6, 565 Fact 8.19.28, 573 eigenvalue of Hermitian part Fact 5.11.27, 355 increasing function Fact 2.21.9, 177 matrix exponential Fact 11.16.4, 760 positive-semidefinite matrix Fact 8.19.6, 565 powers Fact 2.21.13, 177 scalar inequality Fact 2.21.2, 175 Fact 2.21.3, 175 Schur product Fact 9.14.31, 673 singular value Proposition 9.2.2, 602 Proposition 9.6.3, 616 Fact 5.11.27, 355 Fact 8.18.5, 560 Fact 8.19.7, 566 Fact 9.14.19, 670 Fact 9.14.20, 670 singular value inequality Fact 8.19.22, 571 Fact 9.13.16, 662 Fact 9.13.17, 662 Fact 9.13.19, 663 Fact 9.14.31, 673

1138

weak majorization

symmetric gauge function Fact 2.21.13, 177 weak log majorization Fact 2.21.12, 177 Weyl majorant theorem Fact 9.13.19, 663 Weyl’s inequalities Fact 8.18.5, 560 weakly unitarily invariant norm definition, 602 matrix power Fact 9.8.38, 633 numerical radius Fact 9.8.38, 633 Wei-Norman expansion time-varying dynamics Fact 11.13.4, 743 Weierstrass cogredient diagonalization of positive-definite matrices Fact 8.17.2, 558 Weierstrass canonical form pencil Proposition 5.7.3, 332 weighted arithmetic-mean– geometric-mean inequality arithmetic-mean– geometric-mean inequality Fact 1.17.32, 61 refined weighted arithmetic-mean– geometric-mean inequality Fact 1.17.33, 62 Weyl, 470

singular value inequality Fact 5.11.28, 356 singular values and strong log majorization Fact 9.13.18, 662 Weyl majorant theorem singular values and weak majorization Fact 9.13.19, 663 Weyl’s inequalities weak majorization and singular values Fact 8.18.5, 560 Weyl’s inequality Hermitian matrix eigenvalue Theorem 8.4.9, 469 Fact 8.10.4, 501 Wielandt eigenvalue perturbation Fact 9.12.9, 658 positive power of a primitive matrix Fact 4.11.4, 298 Wielandt inequality quadratic form inequality Fact 8.15.30, 555 Williams polynomial root bound Fact 11.20.11, 781

X Xie asymptotically stable polynomial Fact 11.17.7, 764

Y

Yamamoto singular value limit Fact 9.13.21, 664 Young inequality positive-definite matrix Fact 8.9.43, 500 Fact 8.10.46, 510 Fact 8.10.47, 511 reverse inequality Fact 1.12.22, 37 scalar inequality Fact 1.12.21, 36 Specht’s ratio Fact 1.12.22, 37 Young’s inequality positive-semidefinite matrix Fact 8.12.12, 525 positive-semidefinite matrix inequality Fact 9.14.22, 671 scalar case Fact 1.12.32, 39 Fact 1.17.31, 61

Z Z-matrix definition Definition 3.1.4, 182 M-matrix Fact 4.11.8, 301 Fact 4.11.10, 302 M-matrix inequality Fact 4.11.10, 302 matrix exponential Fact 11.19.1, 774 minimum eigenvalue Fact 4.11.11, 302 submatrix Fact 4.11.9, 302 Zassenhaus expansion time-varying dynamics Fact 11.13.4, 743

zeta function Zassenhaus product formula matrix exponential Fact 11.14.18, 751 zero blocking Definition 4.7.10, 273 invariant Definition 12.10.1, 830 invariant and determinant Fact 12.22.14, 874 invariant and equivalent realizations Proposition 12.10.10, 836 invariant and full-state feedback Proposition 12.10.10, 836 Fact 12.22.14, 874

invariant and observable pair Corollary 12.10.12, 837 invariant and transmission Theorem 12.10.8, 834 invariant and unobservable eigenvalue Proposition 12.10.11, 837 transmission Definition 4.7.10, 273 Proposition 4.7.12, 273 transmission and invariant Theorem 12.10.8, 834 zero diagonal commutator Fact 3.8.2, 199 zero entry reducible matrix Fact 3.22.3, 241

1139

Fact 3.22.4, 241 zero matrix definition, 90 maximal null space Fact 2.12.12, 137 positive-semidefinite matrix Fact 8.10.10, 502 trace Fact 2.12.14, 137 Fact 2.12.15, 137 zero of a rational function definition Definition 4.7.1, 271 zero trace Shoda’s theorem Fact 5.9.20, 341 zeta function Euler product formula Fact 1.9.11, 23