1,812 322 13MB
English Pages 432 [449] Year 1997
Table of contents :
0. Systems of Equations and Matrices 
1. Vector Spaces 
2. Linear Operators 
3. Inner Product Spaces 
4. Diagonalizable Linear Operators 
5. The Structure of Normal Operators 
6. Bilinear and Quadratic Forms 
7. Small Oscillations 
8. Factorizations and Canonical Forms.
International Séries in Pure and Applied Mathematics
/ •
J. T. Scheick
'

■.״V
* .
Linear Algebra . י with Applications
צ
ר
Linear Algebra with Applications
International Series in Pure and A pplied M athem atics Ahlfors: Complex Analysis Bender and Orszag: Advanced Mathematical Methods for Scientists and Engineers Bilodeau and Thie: An Introduction to Analysis Boas: Invitation to Complex Analysis Brown and Churchill: Complex Variables and Applications Brown and Churchill: Fourier Series and BoundaryValue Problems Buchanan and Turner: Numerical Methods and Analysis Buck: Advanced Calculus Burton: Elementary’ Number Theory Burton: The History of Mathematics: An Introduction Chartrand and Oellermann: Applied and Algorithmic Graph Theory Colton: Partial Differential Equations Conte and de Boor: Elementary Numerical Analysis: An Algorithmic Approach EdelsteinKeshet: Mathematical Models in Biology Farlow: An Introduction to Differential Equations and Their Applications Goldberg: Matrix Theory with Applications Gulick: Encounters with Chaos Hill: Experiments in Computational Matrix Algebra Keisler and Robbin: Mathematical Logic and Computability Kurtz: Foundations of Abstract Mathematics Lewin and Lewin: An Introduction to Mathematical Analysis Malik, Mordeson, and Sen: Fundamentals of Abstract Algebra Morash: Bridge to Abstract Mathematics: Mathematical Proof and Structures Parzynski and Zipse: Introduction to Mathematical Analysis Pinsky: Partial Differential Equations and BoundaryValue Problems Pinter: A Book of Abstract Algebra Ralston and Rabinowitz: A First Course in Numerical Analysis Ritger and Rose: Differential Equations with Applications Robertson: Engineering Mathematics with Maple Robertson: Engineering Mathematics with Mathematica Rudin: Functional Analysis Rudin: Principles of Mathematical Analysis Rudin: Real and Complex Analysis Scheick: Linear Algebra with Applications Simmons: Differential Equations with Applications and Historical Notes Small and Hosack: Calculus: An Integrated Approach Small and Hosack: Explorations in Calculus with a Computer Algebra System Vanden Eynden: Elementary Number Theory Walker: Introduction to Abstract Algebra
A lso available from M cG raw H ill Schaum ’s O utline Series in M athem atics and Statistics Most outlines include basic theory, definitions, hundreds of example problems solved in stepbystep detail, and supplementary problems with answers. Related titles on the current list include: Advanced Calculus Advanced Mathematics for Engineers & Scientists Analytic Geometry Basic Mathematics for Electricity & Electronics Basic Mathematics with Applications to Science & Technology Beginning Calculus Boolean Algebra & Switching Circuits Calculus Calculus for Business, Economics, & the Social Sciences College Algebra College Mathematics Combinatorics Complex Variables Descriptive Geometry Differential Equations Differential Geometry Discrete Mathematics Elementary Algebra Finite Differences & Difference Equations Finite Element Analysis Finite Mathematics Fourier Analysis General Topology Geometry Group Theory Laplace Transforms Linear Algebra Mathematical Handbook of Formulas & Tables Mathematical Methods for Business & Economics Mathematics for Nurses Matrix Operations Modem Abstract Algebra Numerical Analysis Partial Differential Equations
Probability Probability & Statistics Real Variables Review of Elementary Mathematics Set Theory & Related Topics Statistics Technical Mathematics Tensor Calculus Trigonometry Vector Analysis
Sch aum ’s Solved Problem s Series Each title in this series is a complete and expert source of solved problems with solutions worked out in stepbystep detail. Related titles on the current list include: 3000 Solved Problems 2500 Solved Problems 2000 Solved Problems 3000 Solved Problems 2000 Solved Problems 3000 Solved Problems
in Calculus in Differential Equations in Discrete Mathematics in Linear Algebra in Numerical Analysis in Precalculus
Bob M iller’s Math Helpers Bob Miller’s Calc I Helper Bob Miller’s Calc 11 Helper Bob Miller’s Precalc Helper McGrawHill Paperbacks Arithmetic and Algebra . . . Again How to Solve Word Problems in Algebra Mind Over Math Available at most college bookstores, or, for a complete list of titles and prices, write to: Schaum Division The McGrawHill Companies, Inc. 11 West 19th Street New York, NY 10011
Linear Algebra with Applications
John T. Scheick Duke University
THE McGRAWHILL COMPANIES, INC. New York St. Louis San Francisco Auckland Bogotá Caracas London Madrid Mexico City Milan Montreal New Delhi San Juan Singapore Sydney Tokyo Toronto
Lisbon
This book was set in Times Roman by Publication Services, Inc. The Editors were Jack Shira, Karen M. Minette, and John M. Morriss; the production supervisor was Kathryn Porzio. Project supervision was done by Publication Services, Inc. R. R. Donnelley & Sons Company was printer and binder.
McGrawHill A Division of The M cGrawHill Companies
LINEAR ALGEBRA WITH APPLICATIONS Copyright ©1997 by The McGrawHill Companies, Inc. All rights reserved. Printed in the United States of America. Except as permitted under the United States Copyright Act of 1976, no part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system, without the prior written permission of the publisher. This book is printed on acidfree paper. 1 2 3 4 5 6 7 8 9 0
DOC DOC 9 0 9 8 7 6
ISBN 0070551847־ Library of Congress CataloginginPublication Data Scheick, John T. Linear algebra with applications / John T. Scheick. p. cm.—(International series in pure and applied mathematics) Includes index. ISBN 0070551847 1. Algebras, Linear. I. Title. II. Series. QA184.S34 1997 512' .5—dc20 9627488 http://www.mhcollege.com
ABOUT THE AUTHOR
J. T. SC H EIC K received his Ph.D. in mathematics, in approximation theory, from Syracuse University in 1966. His interests turned to applied mathematics and numerical analysis in the mid 1970s. He has done consulting for General Motors for seven years and has worked on research projects with the Electroscience Laboratory at Ohio State University for several years. Professor Scheick has a strong interest in teaching engineering students at the undergraduate and graduate levels and has served on hundreds of Ph.D. examination and dissertation committees in the College of Engineering at Ohio State University. He is also a member of the Society for Industrial and Applied Mathematics (SIAM). He recently retired from Ohio State University and now teaches at Duke University.
TA BLE OF C O N T E N T S AND O U T L IN E
Preface
0
1
XIII
Systems of Equations and Matrices
1
0.1
Systems of Linear Equations and Matrices
1
0.2
Solution of Homogeneous Systems Gauss elimination / Rowreduced echelon form / Existence of nontrivial solutions and description of the solution of Ax = 0
4
0.3
Solution of Inhomogeneous Systems Gauss elimination revisited / Existence and uniqueness of solutions
10
0.4
Matrix Algebra Matrix multiplication, transposition, and conjugation / Identities and rules
16
0.5
Systems of Equations and Matrix Inverses Conditions for the existence of an inverse / Computation of the inverse / Elementary matrices
23
0.6
Fields
29
Vector Spaces
32
1.1
Vector Spaces Axioms / Principal examples
32
1.2
Subspaces Subspaces / Linear combinations and span / Connection to matrices
37
1.3
Linear Independence Examples / Connection to matrices
43
1.4
Basis and Dimension Basic theory / Examples / Applications of the theory to some fundamental vector spaces, matrices, and systems of equations
53
1.5
Coordinates with Respect to a Basis Change of basis and change of coordinates
62
Linear Operators
73
2.1
73
Preliminary Topics
Functions / Linear operators / Null space and range / Examples IX
Table of Contents and Outline
3
4
2.2
The Rank and Nullity Theorem Rank and nullity theorem / Operator inverses / Application to matrix theory / Computation of the range and null space o f a matrix
80
2.3
Linear Operators and Matrices Matrix of an operator / Operator algebra / Change of basis and similar matrices / Applications
91
Inner Product Spaces
103
3.1
Preliminaries Definitions and examples / Norms; angle between vectors
103
3.2
Orthogonal Sets Computational advantages of orthogonal sets / Fourier coefficients and Parseval’s identity / GramSchmidt process / QR factorization
108
3.3
Approximation and Orthogonal Projection Equivalence of the problems / Computations using orthogonal and nonorthogonal sets / Normal equations / Projection operator / M 1 /Examples in many settings
120
3.4
Applications of Projection Theory Projections in Fm / Weighted linear regression / Data fitting with polynomials, weighted and unweighted
133
3.5
Orthogonal Complements 151 Decomposition of the vector space /Applications to approximation and matrix theory / Fredholm alternative
3.6
The Gram Matrix and Orthogonal Change of Basis Matrix representation of inner products / Orthogonal change of basis / Rank of a Gram matrix
158
Diagonalizable Linear Operators
164
4.1
Eigenvalues and Eigenvectors Definitions / Spectrum and eigenspaces of an operator / Theoretical computations using determinants / Properties of the characteristic polynomial / Geometric and algebraic multiplicities / Remarks on numerical calculations
164
4.2
Linear Operators with an Eigenbasis Diagonalizable operators and their computational advantages / Similarity to a diagonal matrix
173
4.3
Functions of Diagonalizable Operators Two competing definitions / Functions of matrices / General properties of functions of diagonalizable operators / Minimal polynomial
186
Table of Contents and Outline
4.4
FirstOrder Matrix Differential Equations Decoupling the differential equations: Two viewpoints for diagonalizable matrices / eAl
196
4.5
Estimates of Eigenvalues: Gershgorin’s Theorems
202
4.6
Application to Finite Difference Equations Biological models, Markov chain examples, and finite difference equations
207
Appendix:
5
xi
Review of determinants
216
The Structure of Normal Operators
219
5.1
Adjoints and Classification of Operators Definitions / Normal, Hermitian, and unitary operators / Matrix characterization
219
5.2
The Spectral Theorem Spectral theorem and resolution / Functions of normal operators / Simultaneous diagonalization of normal operators
228
5.3
Applications to Matrix Theory Functions of normal matrices / Generalized eigenvalue problem
237
5.4
Extremum Principles for Hermitian Operators The Rayleigh quotient and its extremal properties / CourantFischer theorem / Interlacing theorem for bordered matrices
250
5.5
The Power Method Estimating the dominant eigenvalue and eigenvector / Sharpening estimates / Approximation of secondary eigenvalues and eigenvectors / Inverse power method / Subspace methods
257
5.6
The RayleighRitz Method Approximation of a finite number of eigenvalues and eigenvectors of a Hermitian operator defined on an infinite dimensional space
269
6 Bilinear and Quadratic Forms
282
6.1
Preliminaries Definitions and examples / Elementary properties / Polar identities / The matrix of a bilinear form and change of basis; congruent matrices
282
6.2
Classification of Hermitian Quadratic Forms Classification of quadratic forms, operators, and matrices / Diagonalization and the law of inertia / Minimization problems
291
xii
Table of Contents and Outline
7
8
6.3
Orthogonal Diagonalization Orthogonal diagonalization and the Principal Axis theorem / Applications: level sets and the strain ellipsoid
301
6.4
Other Methods of Diagonalization Completion of squares, simultaneous row and column operations, a method of Jacobi
307
6.5
Simultaneous Diagonalization of Quadratic Forms Simultaneous diagonalization of two and three quadratic forms and the connection to the generalized eigenvalue problem
317
Small Oscillations
324
7.1
Differential Equations of Small Oscillations Derivation of the differential equations by Newton’s laws, Kirchoff’s laws, Lagrange’s equations of motion, and the Ritz method for continuous systems
324
7.2
Undamped Small Oscillations Solution of the differential equations using two viewpoints / The language of small oscillations / Application of the power method / Examples, including the Ritz method / Response to harmonic excitation
331
7.3
Damped Small Oscillations Conditions in order that the differential equations can be decoupled / Solution of the differential equations / Response to harmonic excitation
354
7.4
Galerkin’s Method for Partial Differential Equations Illustration of the method for the wave equation
363
Factorizations and Canonical Forms
373
8.1
The Singular Value and Polar Decompositions The SVD and its interpretations / Polar decomposition
373
8.2
Applications of the SVD Solutions of Ax = y and the pseudoinverse / Applications in numerical analysis / Applications in pattern recognition
380
8.3
Schur’s Theorem Schur s theorem / Matrix norms and the sequence A" / Iterative solution of equations
389
8.4
Jordan Canonical Form Applications to differential equations / eAt
395
Answers to Selected Problems
405
Bibliography
423
Index
425
PREFACE
Linear algebra is a tool used extensively in a wide variety of mathematical, science, and engineering fields. Most textbooks focus on an elementary introduction to the topic, an abstract approach with emphasis on the algebraic aspects of the field, or on technical aspects of the subject. The purpose of this book is to present the core topics of linear algebra that are essential to its use in contemporary mathematics, applied mathematics, engineering, and the sciences. Also, in an effort to illustrate the applicability of this core material, many applications are woven into the text in a wide variety of settings. The book is designed for strong upperlevel undergraduates or beginning graduate students. It can be used in several ways: • A s a onesemester course for advanced undergraduates • A s a topics course for undergraduates • As a onesemester course for beginning mathematics graduate students, possibly followed by a more algebraically oriented course • As a onesemester course for graduate engineers with emphasis on application of the theory and computations A wide variety of exercises allows ample selection of problems for each category of students. There are sections and chapters o f the book that focus on applications. These may be used as special topics in lecture or as projects for students. Often students will read them out o f curiosity. Many good engineering students will find material in the book that is a valuable resource for their work. The book is selfcontained and rigorous, but not designed to be covered one section per lecture. It is thematically arranged around the important concepts and language of abstract vector spaces and operator theory (e.g., the concepts o f basis, dimension, inner product, orthogonal expansion, projection, eigenvectors, Hermitian operators, and unitary operators). Many of these ideas will carry over to subsequent courses, and references are cited for followup reading. An essential premise is that students need to know about function spaces as well as «tuple spaces (matrix theory), since many applications involve function spaces. The theory is given in an abstract setting, but is enhanced by many examples taken from vector spaces indispensable to the sciences and mathematics. This shows the value of an abstract theory, as well as illustrating the ideas behind the theory. Real and complex scalars, «tuple spaces, and function spaces are stressed from the beginning. The use of several examples to illustrate one abstract idea will help the student to see the general principles and to learn to think abstractly. The theory for some serious contemporary applications is developed. For example, the linear algebra needed by a finite elements specialist is included. The interplay between matrix theory and vector spaces is constantly stressed, and the dual viewpoints are frequently used to show complementary methods of problem solving and thinking. Students should be adept at the use o f matrices and matrix methods as well as the use of vector space and operator methods after studying this book. Xlll
xiv
Preface
The text is flexible enough to allow the instructor to choose which chapters to cover and in which order. For example, there are sections and chapters that can be omitted, if the instructor so chooses, without interfering with the main ideas presented later in the book. For example, Chapter 7 can be used without Chapter 6, or both Chapters 6 and 7 can be omitted. The following sections could be considered optional: 3.4, 3.5, 3 .6 ,4 .4 ,4 .5 ,4 .6 , 5.5, 5.6, 6.4, 6.5, 7.4, and 8.3. If students have had an elementary course, then Chapter 0 could be skipped or quickly reviewed, or its essential facts could be mentioned by the instructor as Chapter 1 is traversed. Perhaps some or all of Section 4.1 could be assumed to be known as well. If Chapter 0 is skipped, the main results that will be used later in the book are Theorem 3 of Section 0.2, Theorem 2 of Section 0.3, the matrix identities (2a)(2f) and (3) in Section 0.4, and Theorem 4 and Corollary 5 of Section 0.5. Some sample courses are given in the following table. Recommended sections are listed in each column under the student category. Chapter
Undergraduate
0 1 2 3 4 5 6 7 8
15 all all 13 13
Math graduate all all 13, 56 13,5 14 13,5 all
Engineering graduate review as needed all all 15 14 15 13 (4, 5 optional) 13 12
The sections that may be considered primarily applications are 3 .4 ,4 .4 ,4 .6 ,5 .6 , and Chapter 7. These sections and some sections not typically found in undergraduate courses could serve as topics in a reading course for a second course or as a topics course for advanced undergraduates.
ACKNOWLEDGMENTS I would like to thank the following reviewers for their perceptive and valuable comments: Michael Ecker Hans Engler Larry Grove Murli Gupta Vyron M. Klassen Glen Ledder Justin Peters Stewart M. Robinson Alexander Soifer Horace Wiser
Pennsylvania State University Georgetown University University of Arizona George Washington University California State University University of Nebraska Iowa State University Cleveland State University University of Colorado Washington State University
Preface
xv
I would also like to thank Jack Shira at McGrawHill for his encouragement and unflagging efforts to improve the book during review process. Finally, I would like to thank the many energetic and thoughtful engineering students at Ohio State University who have read previous drafts of the book and who offered many insightful and valuable suggestions for its improvement, both while taking the course and later, during the various stages of their research. Their enthusiasm contributed immensely to the pleasure of writing the book and to developing the contents of the book.
Linear Algebra with Applications
CHAPTER 0
Systems of Equations and Matrices
A s a prelude to the study of vector spaces in general, we briefly consider systems of equations, procedures for solving them, the basic rules o f matrix algebra, and matrix inverses. Most of our examples involve real or complex numbers, but our considerations are valid for any field. The last section of this chapter contains a brief overview of fields. In this chapter, only the essentials o f systems o f equations and matrices will be developed. In later chapters, especially Chapters 1 and 2, the theory o f systems of equations and matrices will be extended to more sophisticated results as a byproduct o f the study o f vector spaces and linear operators.
0.1
SYSTEMS OF LINEAR EQUATIONS AND MATRICES A system of linear equations is a set of m equations in n unknowns:
ã\\X\ + Ü\2X2 + *• • + a \nxn = b x #21 X\ + Ü22%2 + * • י+ ainXn = b2
(1)
am\X\ + am2X2 + * ־1 ־QmnXn = b!n. tion, this is more compactly written as n
^ L aijXj = bj
i  1 , 2 , . . . , m.
(2)
y=1 A simple example of a system of two equations in three unknowns is
x\ + 3x2 5 ־־x3 = 1 (*) 2X — X2 + 4x3 = 7•
2
chapter
0: Systems of Equations and Matrices
System (1) is said to be homogeneous iff all b !• = 0 and inhomogeneous, or nonhomogeneous, otherwise. In discussing procedures to solve a system o f linear equations, it is convenient to introduce the concepts of a matrix, a column, and the multiplication of a matrix and a column. The matrix involved consists of the coefficients of the unknowns x!, and the columns consist of the unknowns and the bt. This results in an even more compact notation for system (1) and assists in the discussion of solving the system. In addition, procedures for solving a system of equations often are given solely in terms of the matrix of coefficients and the numbers bj. A column of length n is an n X 1 array *1
X = (x 1,X2, . . . , X n)T =
x2
X״. where the superscript T means transpose, i.e., convert the row (*!, x 2, . . . , xn) to a column. An m X n matrix is an array of objects consisting of m rows and n columns
A = [au ] =
־a 11 a2\
a \2 a 22
 0 0 3.
1 1 1
1 0 0
R2 *—/?2
1 1 2
r 1 2.
2/? ן
/?3 + R\
/?3
Next add row 2 to row 1 and twice row 2 to row 3. Tidy up the result by multiplying row 2 by —1. Schematically,
־1 0 0
1 0 0
1 1 2
 1־ 1 2
־1 0 0
1 0 0
־1  2־ 1 > 0 0. 0
0 1 0
/?! ־ (e) (AB) = ' ־B' ־A ' ־
B are invertible, then so are A1 , AT, A, AH, and AB. Fur
24
chapter
0: Systems of Equations and Matrices
Proof. To prove (a ), note that from the symmetry of the requirements AB = / and BA = /, both A = B~l and B = A 1 ־hold. Thus /J1 = (,41 )1 . The remainder of the assertions involve verifying the requirements of the definition. To prove (b), one has to show that B = (A_1)T satisfies BAJ = Ar B = /. But taking the transposes of AA1 = A_1A = l and using the properties of the transpose yields BAT = AJB = I. Statement (c) is proved similarly, and (d) follows from (b) and (c). To prove (e), just verify that (B1 ־A1 )AB = / = AB{B~XA~]). The next theorem discusses the issue of uniqueness. LA = I and AR = /, then L = R = /41 . Hence the inverse of A is unique, when it exists.
t h e o r e m 2. If
Proof One has L = LI = L(AR) = (LA)R = Rl = R.
Matrix Inverses and A x
=
y
Next, we turn to the connection between matrix inverses and systems of equations. 3. If A is invertible, then the system of equations Ax = y has the unique solution x = A~]y.
p r o p o s it io n
Proof Suppose .4x = y.T\\cnA~xAx = l x = x = A 1 ־y. This shows thatx is unique. On the other hand, A(A~*y) = l y = y, which shows that A~' y is a solution to Ax = y.
The next theorem is very important in matrix theory. It asserts the logical equivalency of a number of statements. This means that, if any one statement is true, then all are true, or equivalently, if any one statement is false, then all are false. The proof also contains a method for the construction of A~]. 4. L e t/lb e a square matrix, and let r denote the row rank of A. The following statements are logically equivalent:
theorem
(n) A is rowreducible to /. (b) The only solution of ^4x = 0 is x = 0. (c) r = n. (d) The system Ax = y is consistent for every y. (e) There is a matrix B such that AB = /. ( / ) A has an inverse.
Proof The scheme of the proof is to show that (a) => (b) => (c) => (d) => (e) => ( / ) . (0)
(b ): Assume (a) is true. Since A rowreduces to /, Ax = 0 and lx — 0 have the same solution, namely, x = 0. (b) (c): Let R be the rowreduced echelon form of A. If r < n, then there would be free variables in the equations Rx = 0, and therefore a nonzero solution to Ax = 0. Therefore r = n must hold. (c) (d): By Theorem 2 of Section 0.3, Ax = y is solvable for each y since r =
m = n. (d) (