Introduction to Linear Algebra, Fifth Edition [5 ed.] 0980232775, 9780980232776

Table of contents :
Cover
Table of Contents
Preface
1 Introduction to Vectors
1.1 Vectors and Linear Combinations
1.2 Lengths and Dot Products
1.3 Matrices
2 Solving Linear Equations
2.1 Vectors and Linear Equations
2.2 The Idea of Elimination
2.3 Elimination Using Matrices
2.4 Rules for Matrix Operations
2.5 Inverse Matrices
2.6 Elimination = Factorization: A = LU
2.7 Transposes and Permutations
3 Vector Spaces and Subspaces
3.1 Spaces of Vectors
3.2 The Nullspace of A : Solving Ax= 0 and Rx = 0
3.3 The Complete Solution to Ax = b
3.4 Independence, Basis and Dimension
3.5 Dimensions of the Four Subspaces
4 Orthogonality
4.1 Orthogonality of the Four Subspaces
4.2 Projections
4.3 Least Squares Approximations
4.4 Orthonormal Bases and Gram-Schmidt
5 Determinants
5.1 The Properties of Determinants
5.2 Permutations and Cofactors
5.3 Cramer's Rule, Inverses, and Volumes
6 Eigenvalues and Eigenvectors
6.1 Introduction to Eigenvalues
6.2 Diagonalizing a Matrix
6.3 Systems of Differential Equations
6.4 Symmetric Matrices
6.5 Positive Definite Matrices
7 The Singular Value Decomposition (SVD)
7.1 Image Processing by Linear Algebra
7.2 Bases and Matrices in the SVD
7.3 Principal Component Analysis (PCA by the SVD)
7.4 The Geometry of the SVD
8 Linear Transformations
8.1 The Idea of a Linear Transformation
8.2 The Matrix of a Linear Transformation
8.3 The Search for a Good Basis
9 Complex Vectors and Matrices
9.1 Complex Numbers
9.2 Hermitian and Unitary Matrices
9.3 The Fast Fourier Transform
10 Applications
10.1 Graphs and Networks
10.2 Matrices in Engineering
10.3 Markov Matrices, Population, and Economics
10.4 Linear Programming
10.5 Fourier Series: Linear Algebra for Functions
10.6 Computer Graphics
10.7 Linear Algebra for Cryptography
11 Numerical Linear Algebra
11.1 Gaussian Elimination in Practice
11.2 Norms and Condition Numbers
11.3 Iterative Methods and Preconditioners
12 Linear Algebra in Probability & Statistics
12.1 Mean, Variance, and Probability
12.2 Covariance Matrices and Joint Probabilities
12.3 Multivariate Gaussian and Weighted Least Squares
Matrix Factorizations
Index
Six Great Theorems of Linear Algebra/Linear Algebra in a Nutshell

Citation preview

Table of Contents 1

1 2

Introduction to Vectors 1.1 Vectors and Linear Combinations. 1.2 Lengths and Dot Products. 1.3 Matrices .........

22

2 Solving Linear Equations 2.1 Vectors and Linear Equations . 2.2 The Idea of Elimination ... 2.3 Elimination Using Matrices . 2.4 Rules for Matrix Operations 2.5 Inverse Matrices ....... 2.6 Elimination= Factorization: A= LU 2.7 Transposes and Permutations . . . . . .

31 31 46 58 70 83 97 109

3 Vector Spaces and Subspaces 3.1 Spaces of Vectors ....... . . . . . . . . . . . . 3.2 The Nullspace of A: Solving Ax = 0 and Rx = 0 3.3 The Complete Solution to Ax = b 3.4 Independence, Basis and Dimension 3.5 Dimensions of the Four Subspaces

123 123 135 150 164 181

4 Orthogonality 4.1 Orthogonality of the Four Subspaces 4.2 Projections . . . . . . . . . . . . . 4.3 Least Squares Approximations ... 4.4 Orthonormal Bases and Gram-Schmidt.

194 194 206 219 233

5 Determinants 5.1 The Properties of Determinants . 5.2 Permutations and Cofactors ... 5.3 Cramer's Rule, Inverses, and Volumes

247 247 258 273

iii

11

iv

Table of Contents

6 Eigenvalues and Eigenvectors 6.1 Introduction to Eigenvalues.. 6.2 Diagonalizing a Matrix .... 6.3 Systems of Differential Equations 6.4 Symmetric Matrices........ 6.5 Positive Definite Matrices.....

288 2 88 30 4 319 338 350

7 The Singular Value Decomposition (SVD) 7.1 Image Processing by Linear Algebra .... 7.2 Bases and Matrices in the SVD ....... 7.3 Principal Component Analysis (PCA by the SVD). 7.4 The Geometry of the SVD .............

364 36 4 371 382 39 2

8 Linear Transformations 8.1 The Idea of a Linear Transformation 8.2 The Matrix of a Linear Transformation. 8.3 The Search for a Good Basis ......

401 40 1 411 42 1

9 Complex Vectors and Matrices 9.1 Complex Numbers ...... 9.2 Hermitian and Unitary Matrices 9.3 The Fast Fourier Transform.

430 431 438 445

10 Applications 10.1 Graphs and Networks ............. 10.2 Matrices in Engineering............ 10.3 Markov Matrices, Population, and Economics 10.4 Linear Programming ............. 10.5 Fourier Series: Linear Algebra for Functions. 10.6 Computer Graphics ........ 10.7 Linear Algebra for Cryptography.......

452 452 46 2 474 483 49 0 49 6 50 2

11 Numerical Linear Algebra 11.1 Gaussian Elimination in Practice 11.2 Norms and Condition Numbers. 11.3 Iterative Methods and Preconditioners

508 50 8 518 52 4

12 Linear Algebra in Probability & Statistics 12.1 Mean, Variance, and Probability ...... 12.2 Covariance Matrices and Joint Probabilities 12.3 Multivariate Gaussian and Weighted Least Squares

535 535 546 555

Matrix Factorizations

563

Index

565

Six Great Theorems/ Linear Algebra in a Nutshell

574

m+4m- UJ = m· 2

5

angle with v less than 90 °

! u. i

= cos 0

, v+w

u-

[-i l

X1 [ -

~

0

v-

l

+ X2

[

Ul ~

-1

w - [

l ~l +

X3 [

1

n-

= [ :~ X3 -

X1 X2

l·

(1)

= b1

X1

- x1+x2

=b2

-x2 +x3 = b3

b

~

m

gives x

~

m

Cx

=

1 01 [ -1 0 -1

- 10 1

i[ l X1

x2 X3

=

rn

V

1

right weights :r1, .'.C2, x3, can produce weights A- 1 b.

1 0 ~1 0 0 ~1

1 0 0l [ 1 1 0 1

1

1

[ Y2 Y1 l

=

Y3

[ C1C2 l

•

C3

-

1 1 ol [ 321 74c

[11 1 o 0cl 011

[c2 3

·

--

--

The column picture combines the column vectors on the left side to produce the vector b on the right side.

''

3[

t ]+ [ -~ ] = [

1~ ] ·

=

o [ ~ ] +o [

j]

+2 [ ; ] = [

i].

1

ol o0

0

0

I = [ 0

1

l

After ~

~

=l =8

Before elimination

8y

=8

2

3

X

l

X

X

X

X

X

X

X

X

X X

rn rn =

1

=

~] ~l m~ m.

P23

1

o ol

0

1 0

= [0 0 1 .

0

0 0 -4

1

3

1 E = [a

ol ol0

b O l

o ol

1 F= [ 0 1 0 . 0

C

1

0 0 1 0

2 1 3

3

1 [3

X

X

X]

4

X

X

X

31 2

21 21 3 1

1

1

2

l

2 0

0 2

0 0 0 0

AB

= [~ : :] [~ : :] = [ 0 00x

00x

00

l·

Multiply

Ax = b by A - 1 .

Then

x = A - 1 Ax = A - 1 b.

FE=

[-t 20

0 1 -4

(6)

2

2

2

~ 1 -~

~]

➔ [~ ➔ [~

-~J

3

1

1 1 1 3

A= [ 1 3 1

l

2

1

1 1 1 3

B= [ 1 2 1

l

C = [ 11 11 11 1

1

3

l

6 [ 6

G] ()

,-1

C

[ 6.·

6 ]

6 6

1 [ () 6

= :36

2

0

3

0

0 6 0

7

ll

2 1 A= [ 1 2 1 1

1

2

1

o ol

0

0

A= [ 2 1 3

1

and

ll

A= [ 11 21 2 . 1

2

3

-~i

1 -11 0 1 -1 0 0 1

1

T = [ -l

0

-1 2 -1

ol

-1 2

r- 1 =

ll

3 [2

2

2

1

1

1

1

1

,,"/ /

/

/

/

'

''

a

~

H]

and

b

~

U]

and

c

~

H] .

1 - 1

1 1

-~]

1 - 1 - 1

-1 -1

1

=

=

AT A=

[99 189] = [11 10] [9 9] [10 11] .

v12 -v12

-1 -1]

-1 -1 1 -1 -1 1

.

A=

o o [1 0

1]

A=

l [1

o

1 1]

1

1

B = [1 0

0 0 1 0

1

2

ll

2

1 2 3

1 0

A=

C=

l [1 1

[l 2 3] 2

2

3

3

3

3

1 1 1

.

ll

1 . 1

-1 = 0

-

::::-

-

.:;;.;:;;;;

-

A

=

[01 21 ]

and

A-

1

=

[-1112/2 10 ].

l

A= [ 31 62 3 4 8 4

P=

and

B = P APT =

[L

0 0

[682 431 43]1 .

....

Ax1

\

I

=

x1

,\1

x - 1 Ax

= A = [

[~~] [~i] A

X

=

>-n

[1 5] = [1 l][1 ][1-lJ= [1 0

6

k

0

1

6k

O

1

0

6k 6k

] =XA.

l]=

Ak

.

1

------- -

Forward

(llF)

Yn+l = Yn + flt Zn Zn+I = Zn - flt Yn

[ 1 becomes U n+l = - flt

-Yn-1 - Yn - Yn + l

flt][z: 1

Y, ]

= AUn ·

(12)

Backward

Yn+ l Zn+l

= Yn + .6.t Zn+ l = Zn - .6.t Yn+ l

--

--

al [o1 0o0o0ol [alb [ dt ~ = ~ ~ ~ ~ ~ db

d [ y] dt y'

[

0 1] [y'y] .

= -9 6

u

=

[t]

B

o = [o

-4] 0

. Trapezoidal

[

1 6;.t/ 2

-b;.t/2 ] [ Yn+l ] _ [ 1 1 Zn+l - b;.t/2

6;.t/2 ] [ Yn ] 1 Zn ·

A3

=

2 [ -1 0

-1 2 - 1

o

-1 2

l

1

...

9

B = [ 12

1

1

~1

-1

0

S

2

= [- 1 0

ol

- 1 2 - 1 -1 2

T

bl

2 -1 2 - 1 . b - 1 2

= [-1

ol [- O11 - o11 o]01

O -l

O

O - 1

X

x+y =X

J2

S4

10] . = [ 101 101

2

2

o]

0

3

8

S= [ 2 5 3 .

5

=

C

[

1 1

1 1] C

1

1

C

T=

[1 d2 3] 2 3

4

4 5

.

'

'

t 3 ol T= [ 3 t 4 . 0 4 t

S

=

9 [0 0

o1 ol 2 2

8

1 S= [ 1 21 2ll .

1

2

7

101 - •

• 10-1 < - - - - - ~_ _ _ _ ___,____ _ _ 0 10 20 30

_ _ _ J L __ _ ____,

40

10-20 < - - - - - - ' - - - - - - ~ - - - - - " - - - - - '

0

10

20

30

40

AT A= [ 20 10

10 ]

5

~ [1] Av2

-

~

[

-f ]

xTATAx xTx

xTSx xTx ·

2

(/1

· (J/ . 0/ . 2 2 + ··· + (JR2 1s 99.10 or 99.9.10 of