Mathematical Control Theory - An Introduction [2 ed.] 9783030447762, 9783030447786

Table of contents :
Preface to the first edition
Preface to the second edition
Contents
Introduction
0.1 Problems of mathematical control theory
0.2 Specific models
Bibliographical notes
Part I Elements of Classical Control Theory
1 Controllability and observability
1.1 Linear differential equations
1.2 The controllability matrix
1.3 Rank condition
1.4 A classification of control systems
1.5 Kalman decomposition
1.6 Observability
Bibliographical notes
2 Stability and stabilizability
2.1 Stable linear systems
2.2 Stable polynomials
2.3 The Routh theorem
2.4 Stability, observability and Lyapunov equation
2.5 Stabilizability and controllability
2.6 Hautus lemma
2.7 Detectability and dynamical observers
Bibliographical notes
3 Controllability with vanishing energy
3.1 Characterization theorems
3.2 Orbital Transfer Problem
3.3 NCVE and generalized Liouville's theorem
Bibliographical notes
4 Systems with constraints
4.1 Bounded sets of parameters
4.2 Positive systems
Bibliographical notes
5 Realization theory
5.1 Impulse response and transfer functions
5.2 Realizations of the impulse response function
5.3 The characterization of transfer functions
Bibliographical notes
Part II Nonlinear Control Systems
6 Controllability and observability of nonlinear systems
6.1 Nonlinear differential equations
6.2 Controllability and linearization
6.3 Lie brackets
6.4 The openness of attainable sets
6.5 Observability
Bibliographical notes
7 Stability and stabilizability
7.1 Differential inequalities
7.2 The main stability test
7.3 Linearization
7.4 The Lyapunov function method
7.5 La Salle's theorem
7.6 Topological stability criteria
7.7 Exponential stabilizability and the robustness problem
7.8 Necessary conditions for stabilizability
7.9 Stabilization of the Euler equations
Bibliographical notes
8 Realization theory
8.1 Input–output maps
8.2 Partial realizations
Bibliographical notes
Part III Optimal Control
9 Dynamic programming
9.1 Introductory comments
9.2 Bellman equation and the value function
9.3 The linear regulator problem and the Riccati equation
9.4 The linear regulator and stabilization
Bibliographical notes
10 Viscosity solutions of Bellman equations
10.1 Viscosity solution
10.2 Dynamic programming principle
10.3 Regularity of the value function
10.4 Existence of the viscosity solution
10.5 Uniqueness of the viscosity solution
Bibliographical notes
11 Dynamic programming for impulse control
11.1 Impulse control problems
11.2 An optimal stopping problem
11.3 Iterations of convex mappings
11.4 The proof of Theorem 11.1
Bibliographical notes
12 The maximum principle
12.1 Control problems with fixed terminal time
12.2 An application of the maximum principle
12.3 The maximum principle for impulse control problems
12.4 Separation theorems
12.5 Time-optimal problems
Bibliographical notes
13 The existence of optimal strategies
13.1 A control problem without an optimal solution
13.2 Filippov's theorem
Bibliographical notes
Part IV Infinite-Dimensional Linear Systems
14 Linear control systems
14.1 Introduction
14.2 Semigroups of operators
14.3 The Hille–Yosida theorem
14.4 Phillips' theorem
14.5 Important classes of generators
14.6 Specific examples of generators
14.7 The integral representation of linear systems
14.8 Delay systems
14.9 Existence of solutions to delay equation
14.10 Semigroup approach to delay systems
14.11 State space representation of delay systems
Bibliographical notes
15 Controllability
15.1 Images and kernels of linear operators
15.2 The controllability operator
15.3 Various concepts of controllability
15.4 Systems with self-adjoint generators
15.5 Controllability of the wave equation
Bibliographical notes
16 Stability and stabilizability
16.1 Various concepts of stability
16.2 Spectrum determined growth assumptions
16.3 Stability of delay equations
16.4 Lyapunov's equation
16.5 Stability of abstract hyperbolic systems
16.6 Stabilizability and controllability
Bibliographical notes
17 Linear regulators in Hilbert spaces
17.1 Introduction
17.2 The operator Riccati equation
17.3 The finite horizon case
17.4 Regulator problem for delay systems
17.5 The infinite horizon case: Stabilizability and detectability
Bibliographical notes
18 Boundary control systems
18.1 Semigroup approach to boundary control systems
18.2 Boundary systems with general controls
18.3 Heating systems
18.4 Approximate controllability of the heating systems
18.5 Null controllability for a boundary control system
Bibliographical notes
Appendix
A.1 Measurable spaces
A.2 Metric spaces
A.3 Banach spaces
A.4 Hilbert spaces
A.5 Bochner's integral
A.6 Spaces of continuous functions
A.7 Spaces of measurable functions
References
Index

Citation preview

Systems & Control: Foundations & Applications

Jerzy Zabczyk

Mathematical Control Theory An Introduction Second Edition

Systems & Control: Foundations & Applications Series Editor Tamer Başar, University of Illinois at Urbana-Champaign, Urbana, IL, USA

Editorial Board Karl Johan Åström, Lund Institute of Technology, Lund, Sweden Han-Fu Chen, Academia Sinica, Beijing, China Bill Helton, University of California, San Diego, CA, USA Alberto Isidori, Sapienza University of Rome, Rome, Italy Miroslav Krstic, University of California, San Diego, La Jolla, CA, USA H. Vincent Poor, Princeton University, Princeton, NJ, USA Mete Soner, ETH Zürich, Zürich, Switzerland; Swiss Finance Institute, Zürich, Switzerland Former Editorial Board Member Roberto Tempo, (1956–2017), CNR-IEIIT, Politecnico di Torino, Italy

More information about this series at http://www.springer.com/series/4895

Jerzy Zabczyk

Mathematical Control Theory An Introduction Second Edition

Jerzy Zabczyk Institute of Mathematics Polish Academy of Sciences Warsaw, Poland

ISSN 2324-9749 ISSN 2324-9757 (electronic) Systems & Control: Foundations & Applications ISBN 978-3-030-44776-2 ISBN 978-3-030-44778-6 (eBook) https://doi.org/10.1007/978-3-030-44778-6 Mathematics Subject Classification (2010): 49N05, 49J15, 49J20 1st edition: © Birkhäuser Boston 2008 2nd edition: © Springer Nature Switzerland AG 2020 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This book is published under the imprint Birkhäuser, www.birkhauser-science.com by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

To Zofia, Veronika, Johannes, and Maximilian my grandchildren

PREFACE TO THE FIRST EDITION

Control theory originated around 150 years ago when the performance of mechanical governors started to be analysed in a mathematical way. Such governors act in a stable way if all the roots of some associated polynomials are contained in the left half of the complex plane. One of the most outstanding results of the early period of control theory was the Routh algorithm, which allowed one to check whether a given polynomial had this property. Questions of stability are present in control theory today, and, in addition, to technical applications, new ones of economical and biological nature have been added. Control theory has been strongly linked with mathematics since World War II. It has had considerable influence on the calculus of variations, the theory of differential equations and the theory of stochastic processes. The aim of Mathematical Control Theory is to give a self-contained outline of mathematical control theory. The work consciously concentrates on typical and characteristic results, presented in four parts preceded by an introduction. The introduction surveys basic concepts and questions of the theory and describes typical, motivating examples. Part I is devoted to structural properties of linear systems. It contains basic results on controllability, observability, stability and stabilizability. A separate chapter covers realization theory. Toward the end more special topics are treated: linear systems with bounded sets of control parameters and the so-called positive systems. Structural properties of nonlinear systems are the content of Part II, which is similar in setting to Part I. It starts from an analysis of controllability and observability and then discusses in great detail stability and stabilizability. It also presents typical theorems on nonlinear realizations. Part III concentrates on the question of how to find optimal controls. It discusses Bellman’s optimality principle and its typical applications to the linear regulator problem and to impulse control. It gives a proof of Pontryagin’s maximum principle for classical problems with fixed control intervals as well as for time-optimal and impulse control problems. Existence problems vii

viii

Preface to the first edition

are considered in the final chapters, which also contain the basic Filippov theorem. Part IV is devoted to infinite dimensional systems. The course is limited to linear systems and to the so-called semigroup approach. The first chapter treats linear systems without control and is, in a sense, a concise presentation of the theory of semigroups of linear operators. The following two chapters concentrate on controllability, stability and stabilizability of linear systems and the final one on the linear regulator problem in Hilbert spaces. Besides classical topics the book also discusses less traditional ones. In particular great attention is paid to realization theory and to geometrical methods of analysis of controllability, observability and stabilizability of linear and nonlinear systems. One can find here recent results on positive, impulsive and infinite dimensional systems. To preserve some uniformity of style discrete systems as well as stochastic ones have not been included. This was a conscious compromise. Each would be worthy of a separate book. Control theory is today a separate branch of mathematics, and each of the topics covered in this book has an extensive literature. Therefore the book is only an introduction to control theory. Knowledge of basic facts from linear algebra, differential equations and calculus is required. Only the final part of the book assumes familiarity with more advanced mathematics. Several unclear passages and mistakes have been removed due to remarks of Professor L. Mikołajczyk and Professor W. Szlenk. The presentation of the realization theory owes much to discussions with Professor B. Jakubczyk. I thank them very much for their help. Finally some comments about the arrangement of the material. Successive number of paragraph, theorem, lemma, formula, example, exercise is preceded by the number of the chapter. When referring to a paragraph from some other parts of the book a Latin number of the part is added. Numbers of paragraphs, formulae and examples from Introduction are preceded by 0 and those from Appendix by letter A. Warsaw and Coventry 1992

Jerzy Zabczyk

PREFACE TO THE SECOND EDITION

As the first edition, the second one is divided into four parts: Part I, Elements of the classical control theory; Part II, Nonlinear control systems; Part III, Optimal control; Part IV, Infinite-dimensional linear systems. The new elements of the book can be described as follows. Into Part I a new chapter on controllability with vanishing energy is added. The concept was introduced by E. Priola and the author. Applications to the orbital transfer problem are elaborated. A short proof of the Routh stability criteria is presented. Part II is essentially as before. New chapter Viscosity solutions of Bellman’s equation is added to Part III. It includes detailed proofs of the existence and uniqueness theorems. Boundary control systems systems are discussed in a new chapter in Part IV. Approximate controllability of one-dimensional heating system is established and null controllability of multidimensional heating system is presented. In addition delayed systems are investigated in some detail. In particular their stability, semigroup representation and the linear regulator are covered. Improved stability results for general hyperbolic systems are presented. The index was substantially enlarged. Several misprints and mistakes were corrected. Some of them were noticed by Dr. J. Kowalski, who also typed the new material. I thank him for his help. I thank Professors B. Gołdys, A. Święch and R. Triggiani for comments on some parts of the book. I thank the reviewers of the second edition for constructive suggestions. Writing of the second edition took more time than expected and I thank the editors for their patience. Warsaw 2020

Jerzy Zabczyk

ix

CONTENTS

Preface to the first edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Preface to the second edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ix

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §0.1. Problems of mathematical control theory . . . . . . §0.2. Specific models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xvii xvii xix xxvi

Part I. Elements of Classical Control Theory Chapter 1.

Controllability and observability . . . . . . . . . . . . . . . . . §1.1. Linear differential equations . . . . . . . . . . . . . . . . . §1.2. The controllability matrix . . . . . . . . . . . . . . . . . . . §1.3. Rank condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . §1.4. A classification of control systems . . . . . . . . . . . . §1.5. Kalman decomposition . . . . . . . . . . . . . . . . . . . . . . §1.6. Observability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 3 6 10 14 16 18 20

Chapter 2.

Stability and stabilizability . . . . . . . . . . . . . . . . . . . . . . §2.1. Stable linear systems . . . . . . . . . . . . . . . . . . . . . . . §2.2. Stable polynomials . . . . . . . . . . . . . . . . . . . . . . . . . §2.3. The Routh theorem . . . . . . . . . . . . . . . . . . . . . . . . . §2.4. Stability, observability and the Lyapunov equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §2.5. Stabilizability and controllability . . . . . . . . . . . . . §2.6. Hautus lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §2.7. Detectability and dynamical observers . . . . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

21 21 25 27 31 34 37 39 41 xi

xii

Contents

Chapter 3.

Controllability with vanishing energy . . . . . . . . . . . . §3.1. Characterization theorems . . . . . . . . . . . . . . . . . . . §3.2. Orbital Transfer Problem . . . . . . . . . . . . . . . . . . . . §3.3. NCVE and generalized Liouville’s theorem . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

43 43 49 52 53

Chapter 4.

Systems with constraints . . . . . . . . . . . . . . . . . . . . . . . . §4.1. Bounded sets of parameters . . . . . . . . . . . . . . . . . . §4.2. Positive systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

55 55 57 64

Chapter 5.

Realization theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §5.1. Impulse response and transfer functions . . . . . . . §5.2. Realizations of the impulse response function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §5.3. The characterization of transfer functions . . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

65 65 69 75 76

Part II. Nonlinear Control Systems Chapter 6.

Chapter 7.

Controllability and observability of nonlinear systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §6.1. Nonlinear differential equations . . . . . . . . . . . . . . §6.2. Controllability and linearization . . . . . . . . . . . . . . §6.3. Lie brackets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §6.4. The openness of attainable sets . . . . . . . . . . . . . . §6.5. Observability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Stability and stabilizability . . . . . . . . . . . . . . . . . . . . . . §7.1. Differential inequalities . . . . . . . . . . . . . . . . . . . . . . §7.2. The main stability test . . . . . . . . . . . . . . . . . . . . . . §7.3. Linearization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §7.4. The Lyapunov function method . . . . . . . . . . . . . . §7.5. La Salle’s theorem . . . . . . . . . . . . . . . . . . . . . . . . . . §7.6. Topological stability criteria . . . . . . . . . . . . . . . . . §7.7. Exponential stabilizability and the robustness problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §7.8. Necessary conditions for stabilizability . . . . . . . . §7.9. Stabilization of the Euler equations . . . . . . . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

79 79 83 86 90 94 96 97 97 100 105 108 110 113 117 121 122 125

Contents

Chapter 8.

xiii

Realization theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §8.1. Input–output maps . . . . . . . . . . . . . . . . . . . . . . . . . §8.2. Partial realizations . . . . . . . . . . . . . . . . . . . . . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

127 127 129 133

Part III. Optimal Control Chapter 9.

Dynamic programming . . . . . . . . . . . . . . . . . . . . . . . . . . §9.1. Introductory comments . . . . . . . . . . . . . . . . . . . . . §9.2. Bellman equation and the value function . . . . . . §9.3. The linear regulator problem and the Riccati equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §9.4. The linear regulator and stabilization . . . . . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

143 146 151

Chapter 10. Viscosity solutions of Bellman equations . . . . . . . . . §10.1. Viscosity solution . . . . . . . . . . . . . . . . . . . . . . . . . . §10.2. Dynamic programming principle . . . . . . . . . . . . . . §10.3. Regularity of the value function . . . . . . . . . . . . . . §10.4. Existence of the viscosity solution . . . . . . . . . . . . §10.5. Uniqueness of the viscosity solution . . . . . . . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

153 153 155 157 158 160 166

Chapter 11. Dynamic programming for impulse control . . . . . . §11.1. Impulse control problems . . . . . . . . . . . . . . . . . . . . §11.2. An optimal stopping problem . . . . . . . . . . . . . . . . §11.3. Iterations of convex mappings . . . . . . . . . . . . . . . . §11.4. The proof of Theorem 11.1 . . . . . . . . . . . . . . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

167 167 169 170 171 176

Chapter 12. The maximum principle . . . . . . . . . . . . . . . . . . . . . . . . . §12.1. Control problems with fixed terminal time . . . . . §12.2. An application of the maximum principle . . . . . . §12.3. The maximum principle for impulse control problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §12.4. Separation theorems . . . . . . . . . . . . . . . . . . . . . . . . §12.5. Time-optimal problems . . . . . . . . . . . . . . . . . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

177 177 180

Chapter 13. The existence of optimal strategies . . . . . . . . . . . . . . §13.1. A control problem without an optimal solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §13.2. Filippov’s theorem . . . . . . . . . . . . . . . . . . . . . . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

137 137 138

182 187 189 194 195 195 196 200

xiv

Contents

Part IV. Infinite-Dimensional Linear Systems Chapter 14. Linear §14.1. §14.2. §14.3. §14.4. §14.5. §14.6. §14.7.

control systems . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Semigroups of operators . . . . . . . . . . . . . . . . . . . . . The Hille–Yosida theorem . . . . . . . . . . . . . . . . . . . Phillips’ theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . Important classes of generators . . . . . . . . . . . . . . . Specific examples of generators . . . . . . . . . . . . . . . The integral representation of linear systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §14.8. Delay systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §14.9. Existence of solutions to delay equation . . . . . . . §14.10. Semigroup approach to delay systems . . . . . . . . . §14.11. State space representation of delay systems . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

229 234 235 240 246 248

Chapter 15. Controllability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §15.1. Images and kernels of linear operators . . . . . . . . . §15.2. The controllability operator . . . . . . . . . . . . . . . . . . §15.3. Various concepts of controllability . . . . . . . . . . . . §15.4. Systems with self-adjoint generators . . . . . . . . . . §15.5. Controllability of the wave equation . . . . . . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

249 249 252 255 256 261 262

Chapter 16. Stability and stabilizability . . . . . . . . . . . . . . . . . . . . . . §16.1. Various concepts of stability . . . . . . . . . . . . . . . . . §16.2. Spectrum determined growth assumptions . . . . . §16.3. Stability of delay equations . . . . . . . . . . . . . . . . . . §16.4. Lyapunov’s equation . . . . . . . . . . . . . . . . . . . . . . . . §16.5. Stability of abstract hyperbolic systems . . . . . . . §16.6. Stabilizability and controllability . . . . . . . . . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

265 265 270 271 272 273 277 281

Chapter 17. Linear §17.1. §17.2. §17.3. §17.4. §17.5.

283 283 285 287 291

regulators in Hilbert spaces . . . . . . . . . . . . . . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The operator Riccati equation . . . . . . . . . . . . . . . The finite horizon case . . . . . . . . . . . . . . . . . . . . . . Regulator problem for delay systems . . . . . . . . . . The infinite horizon case: Stabilizability and detectability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

203 203 204 212 215 217 221

293 297

Contents

xv

Chapter 18. Boundary control systems . . . . . . . . . . . . . . . . . . . . . . . §18.1. Semigroup approach to boundary control systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §18.2. Boundary systems with general controls . . . . . . §18.3. Heating systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . §18.4. Approximate controllability of the heating systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §18.5. Null controllability for a boundary control system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

299

306 313

Chapter A. Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §A.1. Measurable spaces . . . . . . . . . . . . . . . . . . . . . . . . . §A.2. Metric spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §A.3. Banach spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §A.4. Hilbert spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . §A.5. Bochner’s integral . . . . . . . . . . . . . . . . . . . . . . . . . §A.6. Spaces of continuous functions . . . . . . . . . . . . . . . §A.7. Spaces of measurable functions . . . . . . . . . . . . . .

315 315 315 316 319 320 322 322

299 301 303 306

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332

INTRODUCTION

In the first part of the Introduction, in a schematic way, basic questions of control theory are formulated. The second part describes several specific models of control systems giving physical interpretation of the introduced parameters. The second part is not necessary for the understanding of the following considerations, although mathematical versions of the discussed models will often appear.

§0.1. Problems of mathematical control theory A departure point of control theory is the differential equation y˙ = f (y, u),

y(0) = x ∈ Rn ,

(0.1)

with the right hand side depending on a parameter u from a set U ⊂ Rm . The set U is called the set of control parameters. Differential equations depending on a parameter have been objects of the theory of differential equations for a long time. In particular an important question of continuous dependence of the solutions on parameters has been asked and answered under appropriate conditions. Problems studied in mathematical control theory are, however, of different nature, and a basic role in their formulation is played by the concept of control. One distinguishes controls of two types: open and closed loop. An open-loop control can be basically an arbitrary function u(·): [0, +∞) −→ U , for which the equation y(t) ˙ = f (y(t)), u(t)),

t ≥ 0, y(0) = x,

(0.2)

has a well defined solution. A closed-loop control can be identified with a mapping k: Rn −→ U , which may depend on t ≥ 0, such that the equation xvii

xviii

Introduction

y(t) ˙ = f (y(t), k(y(t))),

t ≥ 0, y(0) = x,

(0.3)

has a well defined solution. The mapping k(·) is called feedback . Controls are called also strategies or inputs, and the corresponding solutions of (0.2) or (0.3) are outputs of the system. One of the main aims of control theory is to find a strategy such that the corresponding output has desired properties. Depending on the properties involved one gets more specific questions. Controllability. One says that a state z ∈ Rn is reachable from x in time T , if there exists an open-loop control u(·) such that, for the output y(·), y(0) = x, y(T ) = z. If an arbitrary state z is reachable from an arbitrary state x in a time T , then the system (0.1) is said to be controllable. In several situations one requires a weaker property of transferring an arbitrary state into a given one, in particular into the origin. A formulation of effective characterizations of controllable systems is an important task of control theory only partially solved. Stabilizability. An equally important issue is that of stabilizability. Assume ¯ ∈ U , f (¯ x, u ¯) = 0. A function k : Rn −→ U , such that for some x ¯ ∈ Rn and u that k(¯ x) = u ¯, is called a stabilizing feedback if x ¯ is a stable equilibrium for the system y(t) ˙ = f (y(t), k(y(t))), t ≥ 0, y(0) = x. (0.4) In the theory of differential equations there exist several methods to determine whether a given equilibrium state is a stable one. The question of whether, in the class of all equations of the form (0.4), there exists one for which x ¯ is a stable equilibrium is of a new qualitative type. Observability. In many situations of practical interest one observes not the state y(t) but its function h(y(t)), t ≥ 0. It is therefore often necessary to investigate the pair of equations y˙ = f (y, u), w = h(y).

y(0) = x,

(0.5) (0.6)

Relation (0.6) is called an observation equation. The system (0.5)–(0.6) is said to be observable if, knowing a control u(·) and an observation w(·), on a given interval [0, T ], one can determine uniquely the initial condition x. Stabilizability of partially observable systems. The constraint that one can use only a partial observation w complicates considerably the stabilizability problem. Stabilizing feedback should be a function of the observation only, and therefore it should be “factorized” by the function h(·). This way one is led to a closed-loop system of the form y˙ = f (y, k(h(y))),

y(0) = x.

(0.7)

0.2 Specific models

xix

There exists no satisfactory theory which allows one to determine when there exists a function k(·) such that a given x ¯ is a stable equilibrium for (0.7). Realization. In connection with the full system (0.5)–(0.6) one poses the problem of realization. For a given initial condition x ∈ Rn , system (0.5)–(0.6) defines a mapping which transforms open-loop controls u(·) onto outputs given by (0.6): w(t) = h(y(t)), t ∈ [0, T ]. Denote this transformation by R. What are its properties? What conditions should a transformation R satisfy to be given by a system of the type (0.5)–(0.6)? How, among all the possible “realizations” (0.5)–(0.6) of a transformation R, do we find the simplest one? The transformation R is called an input–output map of the system (0.5)–(0.6). Optimality. Besides the above problems of structural character, in control theory, with at least the same intensity, one asks optimality questions. In the so-called time-optimal problem one is looking for a control which not only transfers a state x onto z but does it in the minimal time T . In other situations the time T > 0 is fixed and one is looking for a control u(·) which minimizes the integral  0

T

g(y(t), u(t)) dt + G(y(T )),

in which g and G are given functions. A related class of problems consists of optimal impulse control. They require however a modified concept of strategy. Systems on manifolds. Difficulties of a different nature arise if the state space is not Rn or an open subset of Rn but a differential manifold. This is particularly so if one is interested in the global properties of a control system. The language and methods of differential geometry in control theory are starting to play a role similar to the one they used to play in classical mechanics. Infinite-dimensional systems. The problems mentioned above do not lose their meanings if, instead of ordinary differential equations, one takes, as a description of a model, a partial differential equation of parabolic or hyperbolic type. The methods of solutions, however become, much more complicated.

§0.2. Specific models The aim of the examples introduced in this paragraph is to show that the models and problems discussed in control theory have an immediate real meaning. Example 0.1 (Electrically heated oven). Let us consider a simple model of an electrically heated oven, which consists of a jacket with a coil directly heating

xx

Introduction

the jacket and of an interior part. Let T0 denote the outside temperature. We make a simplifying assumption, that at an arbitrary moment t ≥ 0, temperatures in the jacket and in the interior part are uniformly distributed and equal to T1 (t), T2 (t). We assume also that the flow of heat through a surface is proportional to the area of the surface and to the difference of temperature between the separated media. Let u(t) be the intensity of the heat input produced by the coil at moment t ≥ 0. Let moreover a1 , a2 denote the area of exterior and interior surfaces of the jacket, c1 , c2 denote heat capacities of the jacket and the interior of the oven and r1 , r2 denote radiation coefficients of the exterior and interior surfaces of the jacket. An increase of heat in the jacket is equal to the amount of heat produced by the coil reduced by the amount of heat which entered the interior and exterior of the oven. Therefore, for the interval [t, t + Δt],we have the following balance: c1 (T1 (t + Δt) − T1 (t)) ≈ u(t)Δt − (T1 (t) − T2 (t))a1 r1 Δt − (T1 (t) − T0 )a2 r2 Δt. Similarly, an increase of heat in the interior of the oven is equal to the amount of heat radiated by the jacket: c2 (T2 (t + Δt) − T2 (t)) = (T1 (t) − T2 (t))a1 r2 Δt. Dividing the obtained identities by Δt and taking the limit, as Δt ↓ 0, we obtain dT1 = u − (T1 − T2 )a1 r1 − (T1 − T0 )a2 r2, dt dT2 = (T1 − T2 )a1 r1 . c2 dt

c1

Let us remark that, according to the physical interpretation, u(t) ≥ 0 for t ≥ 0. Introducing new variables x1 = T1 − T0 and x2 = T2 − T0 , we have    r1 a1 +r2 a2 r1 a1     −1  − d x1 x1 c c1 c1 = + 1 u. r1 a1 r1 a1 x − 0 dt x2 2 c2 c2 It is natural to limit the considerations to the case when x1 (0) ≥ 0 and x2 (0) ≥ 0. It is physically obvious that if u(t) ≥ 0 for t ≥ 0, then also x1 (t) ≥ 0, x2 (t) ≥ 0, t ≥ 0. One can prove this mathematically; see § I.4.2. Let us assume that we want to obtain, in the interior part of the oven, a temperature T and keep it at this level infinitely long. Is this possible? Does the answer depend on initial temperatures T1 ≥ T0 , T2 ≥ T0 ? The obtained model is a typical example of a positive control system discussed in detail in § I.4.2. Example 0.2 (Rigid body). When studying the motion of a spacecraft it is convenient to treat it as a rigid body rotating around a fixed point O. Let

0.2 Specific models

xxi

us regard this point as the origin of an inertial system. In the inertial system the motion is described by H˙ = F, where H denotes the total angular momentum and F the torque produced by 2r gas jets located symmetrically with respect to O. The torque F is equal to u1 b1 + . . . + ur br , where b1 , . . . , br are vectors fixed to the spacecraft and u1 , . . . , ur — are thrusts of the jets. Let {e1 , e2 , e3 } and {r1 , r2 , r3 } be orthonormal bases of the inertial system and the rotating one. There exists exactly one matrix R such that ri = Rei , i = 1, 2, 3. It determines completely the position of the body. Let Ω be the angular velocity measured in the inertial system, given in the moving system by the formula ω = RΩ. It is not difficult to show (see [2]) that ⎤ ⎡ 0 ω3 −ω2 R˙ = SR, where S = ⎣ −ω3 0 ω1 ⎦ . ω2 −ω1 0 Let J be the inertia matrix. Then the total angular momentum H is given by H = R−1 J. Inserting this expression into the equation of motion we obtain

 d −1 R Jω = R−1 u i bi . dt i=1 r

After an elementary transformation we arrive at the Euler equation J ω˙ = SJω +

r

u i bi .

i=1

This equation together with

R˙ = SR

characterizes completely the motion of the rotating object. From a practical point of view the following questions are of interest: Is it possible to transfer a given pair (R0 , ω0 ) to a desirable pair (R1 , 0), using some steering functions u1 (·), . . . , ur (·)? Can one find feedback controls which slow down the rotational movement independently on the initial angular velocity? While the former question was concerned with controllability, the latter one was about stabilizability. Example 0.3 (Watt’s governor). Let J be the moment of inertia of the flywheel of a steam engine with the rotational speed ω(t), t ≥ 0. Then J ω˙ = u − p,

xxii

Introduction

where u and p are torques produced by the action of steam and the weight of the cage, respectively. This is the basic movement equation, and Watt’s governor is a technical implementation of a feedback whose aim is a stable action of the steam engine. It consists of two arms with weights m at their ends, fixed symmetrically to a vertical rod, rotating with speed l, where l is a positive number. Let ϕ denote the angle between the arms and the rod, equal to the angle between the bars supporting the arms and attached to the sleeve moving along the rod and to the rod itself. The position of the sleeve determines the intensity of the steam input. If b is the coefficient of the frictional force in the hinge joints and g the gravitational acceleration, then (see [78]) ˙ mϕ¨ = ml2 ω 2 sin ϕ cos ϕ − mg sin ϕ − bϕ. One is looking for a feedback of the form u=u ¯ + k(cos ϕ − cos ϕ), ¯ where u ¯, ϕ¯ and k are fixed numbers. Introducing a new variable ψ = ϕ, ˙ one obtains a system of three equations ϕ˙ = ψ, b ψ˙ = l2 ω 2 sin ϕ cos ϕ − g sin ϕ − ψ, m ω˙ = k cos ϕ + (¯ u − p − k cos ϕ). ¯ Assume that p is constant and let (ϕ0 , ψ0 , ω0 )∗ be an equilibrium state of the above system. For what parameters of the system is the equilibrium position stable for values p from a given interval? This question has already been asked by Maxwell. An answer is provided by stability theory. Example 0.4 (Electrical filter). An electrical filter consists of a capacitor with capacity C, a resistor with resistance R, two inductors with inductance L and a voltage source [78]. Let U (t) be the voltage of the input and I(t) the current in one of the inductors at a moment t ∈ R. From Kirchhoff’s law L2 C

d(3) I d2 I dI + RI = U. + RLC + 2L dt2 dt dt(3)

The basic issue here is a relation between the voltage U (·) and current I(·). They can be regarded as the control and the output of the system. Let us assume that the voltage U (·) is a periodic function of a period ω. Is it true that the current I(·) is also periodic? Let α(ω) be the amplitude of I(·). For what range of ω is the amplitude α(ω) very large or conversely very close to zero? In the former case we deal with the amplification of the frequencies ω and in the latter one the frequencies ω are “filtered” out.

0.2 Specific models

xxiii

Example 0.5 (Soft landing). Let us consider a spacecraft of total mass M moving vertically with the gas thruster directed toward the landing surface. Let h be the height of the spacecraft above the surface, u the thrust of its engine produced by the expulsion of gas from the jet. The gas is a product of the combustion of the fuel. The combustion decreases the total mass of the spacecraft, and the thrust u is proportional to the speed with which the mass decreases. Assuming that there is no atmosphere above the surface and that g is gravitational acceleration, one arrives at the following equations [37]: ¨ = −gM + u, Mh M˙ = −ku,

(0.8) (0.9)

˙ = h1 ; k a positive with the initial conditions M (0) = M0 , h(0) = h0 , h(0) constant. One imposes additional constraints on the control parameter of the type 0 ≤ u ≤ α and M ≥ m, where m is the mass of the spacecraft without fuel. Let us fix T > 0. The soft landing problem consists of finding a control u(·) such that for the solutions M (·), h(·) of equation (0.8) M (t) ≥ m,

h(t) ≥ 0,

t ∈ [0, T ],

and

˙ ) = 0. h(T ) = h(T

The problem of the existence of such a control is equivalent to the controllability of the system (0.8)–(0.9). A natural optimization question arises when the moment T is not fixed and one is minimizing the landing time. The latter problem can be formulated equivalently as the minimum fuel problem. In fact, let v = h˙ denote the velocity of the spacecraft, and let M (t) > 0 for t ∈ [0, T ]. Then M˙ (t) = −k v(t) ˙ − gk, M (t)

t ∈ [0, T ].

Therefore, after integration, M (T ) = e−v(T )k−gkT +v(0)k M (0). Thus a soft landing is taking place at a moment T > 0 (v(T ) = 0) if and only if M (T ) = e−gkT ev(0)k M (0). Consequently, the minimization of the landing time T is equivalent to the minimization of the amount of fuel M (0) − M (T ) needed for landing. Example 0.6 (Orbital transfer problem). A spacecraft is orbiting around a space station moving around the Earth. The spacecraft should enter a new orbit using as small amount of energy as possible. In a frame attached to the station the linearized movement of the spacecraft satisfies the so-called Hill’s equation [83, pp. 139–152], [90]:

xxiv

Introduction

x ¨ = 2ω y˙ + 3ω 2 x + u1 y¨ = −2ω x˙ + u2 , where ω = 2π T and T is the time of one rotation of the station. The control functions describe the force exercised by the jet engine of the spacecraft. In §3.2, following [90], a derivation of Hill’s equation is sketched and it is shown that the transfer to a new orbit can t be done with a negligible amount of the energy measured by the integral 0 (u21 (s) + u22 (s)) ds. However, the time t of the transfer might be rather large. Example 0.7 (Optimal consumption). The capital y(t) ≥ 0 of an economy at any moment t is divided into two parts: u(t)y(t) and (1 − u(t))y(t), where u(t) is a number from the interval [0, 1]. The first part goes for investments and contributes to the increase in capital according to the formula y˙ = uy,

y(0) = x > 0.

The remaining part is for consumption evaluated by the satisfaction  JT (x, u(·)) =

0

T

((1 − u(t))y(t))α dt + ay α (T ).

(0.10)

In definition (0.10), the number a is nonnegative and α ∈ (0, 1). In the described situation one is trying to divide the capital to maximize the satisfaction. Example 0.8 (Heating a rod). To fix ideas let us denote by y(t, ξ) the temperature of a rod at the moment t ≥ 0 and at the point ξ ∈ [0, L], heated with the intensity u(t)b(ξ), where the value u(t) can be arbitrarily chosen: u(t) ∈ R. Assuming additionally that the ends of the rods are insulated: u(t, 0) = u(t, L), t ≥ 0, and applying elementary laws of the heat conduction, one arrives at the following parabolic equation describing the evolution of the temperature in the rod: ∂2y ∂y (t, ξ) = σ 2 2 (t, ξ) + u(t)b(ξ), ∂t ∂ξ y(t, 0) = y(t, L) = 0, t > 0, y(0, ξ) = x(ξ), ξ ∈ [0, L].

t > 0, ξ ∈ (0, L),

(0.11) (0.12) (0.13)

The parameter σ in (0.11) is the heat capacity of the rod, and the function x(·) is the initial distribution of the temperature. Let us assume that x ˆ(ξ), ξ ∈ [0, L], is the required distribution of the temperature and T > 0 a fixed moment. The question whether one can heat the rod in such a way to obtain y(T, ξ) = x ˆ(ξ), ξ ∈ [0, L], is identical to asking whether, for system (0.11)–(0.12), the initial state x(·) can be transferred ˆ(·) in time T . A practical situation may impose additional constraints onto x

0.2 Specific models

xxv

on the control parameter u of the type u ∈ [0, u ¯]. Under such a constraint, the problem of finding control transferring x(·) and x ˆ(·) in minimal time is mathematically well posed and has a clear physical meaning. Example 0.9 (Stabilizing elastic structure). A simplified model of an elastic structure like a suspension bridge can be described through a hyperbolic equation ∂2y ∂2y (t, ξ) = a (t, ξ), ∂t2 ∂ξ 2 ∂y (t, 1) = u(t), y(t, 0) = 0, ∂ξ ∂y y(0, ξ) = x1 (ξ), (0, ξ) = x2 (ξ), ∂ξ

ξ ∈ (0, 1), t > 0,

(0.14)

t > 0,

(0.15)

ξ ∈ (0, 1),

(0.16)

with positive constant a and a (boundary) control function u. Functions x1 and x2 are equal to the initial displacement of the structure and the initial velocity of its points. Can one choose a function u in such a way that for a given T and all ξ ∈ (0, 1), y(T, ξ) = 0 and ∂u ∂ξ (T, ξ) = 0, ξ ∈ (0, 1)? Can one choose control u a feedback form  1  1 ∂y (0.17) k1 (ξ)y(t, ξ) dξ + k2 (ξ) (t, ξ) dξ, t > 0, u(t) = ∂t 0 0 such that the solution to (0.14), (0.15) and (0.16), with u replaced by (0.17), and its time derivative tend to the zero (rest) state? The control u acts on one end of the bridge as a force modifying the tension of the structure. A stabilizer in the form (0.17) should be accompanied by sensor devices monitoring the shape y(t, ξ) and its velocity ∂y ∂t (t, ξ) for some (better all) ξ ∈ (0, 1). Example 0.10 (Kalecki delayed model of economic cycles). The capital y(t) of a country at a moment t satisfies a delay equation y  (t) = ay(t) + by(t − 1) + u(t),

t ≥ 0,

as one assumes that only a fraction b of the capital goes to investments and produces new capital with delay 1. The constant a is equal to the rate with which the economy grows. The control u could be interpreted as exterior input at the capital and/or the outflow of the labour force. The asymptotic behaviour of the capital, even when u ≡ 0, depends in a capricious way on a and b, see N.D. Hayes [45]. The model exhibits economical cycles.

xxvi

Introduction

Bibliographical notes In the development of mathematical control theory the following works played an important rôle: J.C. Maxwell, On governers [69], N. Wiener, Cybernetics or control and communication in the animal and the machine [98], R. Bellman, Dynamic Programming [7], L.S. Pontryagin, V.G. Boltyanskii, R.V. Gamkrelidze and E.F. Mishchenko, Matematicheskaya teoriya optimal’nykh processov [77], P.E. Kalman, On the general theory of control systems [55], T. Ważewski, Systèmes de commande et équations au contingent [97], J.-L. Lions, Contrôle optimale de systèmes par des équations aux dérivées partielles [65], W.M. Wonham, Linear multivariable control: A geometric approach [101], M.G. Crandall and P.-L. Lions, Viscosity solutions of Hamilton–Jacobi equations [21], I. Lasiecka and R. Triggiani, Control Theory for Partial Differential Equations: Continuous and Approximation Theories [62]. The model of an electrically heated oven is borrowed from S. Barnett [6]. The dynamics of a rigid body is discussed in V.I. Arnold [2]. A detailed derivation of the equations describing Watt’s governor can be found in L.S. Pontryagin [78]. The electrical filter is a typical example of an automatic system. The soft landing and optimal consumption models are extensively discussed in W.H. Fleming and R.W. Rishel [37]. The orbital transfer problem is solved in M. Shibata and A. Ichikawa [90]. The model of economic cycles was introduced by M. Kalecki in 1935 in M. Kalecki [53], see also his book [54]. For recent investigation of the model see P. Kriesler and J.W. Nevile [61]. Interesting information about heating processes can be found in S. Rolewicz [86] and A.G. Butkovski [14]. Recently, control applications to biological sciences have been intensively studied. The interested reader is referred to the conference proceedings [19].

Part I

Elements of Classical Control Theory

Chapter 1

Controllability and observability

In this chapter basic information about linear differential equations are recalled and the main concepts of control theory, controllability and observability, are studied. Specific formulae for controls transferring one state onto another as well as algebraic characterizations of controllable and observable systems are obtained. A complete classification of controllable systems with one-dimensional input is given.

§1.1. Linear differential equations The basic object of classical control theory is a linear system described by a differential equation dy = Ay(t) + Bu(t), dt

y(0) = x ∈ Rn ,

(1.1)

and an observation relation w(t) = Cy(t),

t ≥ 0.

(1.2)

Linear transformations A : Rn −→ Rn , B : Rm −→ Rn , C : Rm −→ Rk in (1.1) and (1.2) will be identified with representing matrices and elements of Rn , Rm , Rk with one column matrices. The set of all matrices with n rows and m columns will be denoted by M(n, m) and the identity transformation as well as the identity matrix by I. The scalar product x, y and the norm |x|, of elements x, y ∈ Rn with coordinates ξ1 , . . . , ξn and η1 , . . . , ηn , are defined by n n 1/2   ξj ηj , |x| = ξj2 . x, y = j=1

j=1

© Springer Nature Switzerland AG 2020 J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_1

3

4

1 Controllability and observability

The adjoint transformation of a linear transformation A as well as the transpose matrix of A are denoted by A∗ . A matrix A ∈ M(n, n) is called symmetric if A = A∗ . The set of all symmetric matrices is partially ordered by the relation A1 ≥ A2 if A1 x, x ≥ A2 x, x for arbitrary x ∈ Rn . If A ≥ 0 then one says that matrix A is nonnegative definite and if, in addition, Ax, x > 0 for x = 0 that A is positive definite. Treating x ∈ Rn as an element of M(n, 1) we have x∗ ∈ M(1, n). In particular we can write x, y = x∗ y and |x|2 = x∗ x. The inverse transformation of A and the inverse matrix of A will be denoted by A−1 . If F (t) = [fij (t); i = 1, . . . , n, j = 1, . . . , m] ∈ M(n, m), t ∈ [0, T ], then, by definition, 



T

0

F (t) dt =

T

0

 fij (t) dt, i = 1, . . . , n; j = 1, . . . , n ,

(1.3)

under the condition that elements of F (·) are integrable. Derivatives of the first and second order of a function y(t), t ∈ R, are (n) d2 y denoted by dy ˙ y¨ and the nth order derivative, by ddt(n)y . dt , dt2 or by y, We will need some basic results on linear equations dq = A(t)q(t) + a(t), dt

q(t0 ) = q0 ∈ Rn ,

(1.4)

on a fixed interval [0, T ]; t0 ∈ [0, T ], where A(t) ∈ M(n, n), A(t) = [aij (t); i = 1, . . . , n, j = 1, . . . , m], a(t) ∈ Rn , a(t) = (ai (t); i = 1, . . . , n), t ∈ [0, T ]. Theorem 1.1. Assume that elements of the function A(·) are locally integrable. Then there exists exactly one function S(t), t ∈ [0, T ] with values in M(n, n) and with absolutely continuous elements such that d S(t) = A(t)S(t) dt S(0) = I.

for almost all t ∈ [0, T ],

(1.5) (1.6)

In addition, a matrix S(t) is invertible for an arbitrary t ∈ [0, T ], and the unique solution of the equation (1.4) is of the form q(t) = S(t)S

−1



t

(t0 )q0 +

S(t)S −1 (s)a(s) ds,

t ∈ [0, T ].

t0

Because of its importance we will sketch a proof of the theorem. Proof. Equation (1.4) is equivalent to the integral equation 

t

A(s)q(s) ds + t0

The formula



t

q(t) = a0 +

a(s) ds, t0

t ∈ [0, T ].

(1.7)

1.1 Linear differential equations

 Ly(t) = a0 +

5



t

t

a(s)ds + t0

A(s)y(s)ds,

t ∈ [0, T ],

t0

defines a continuous transformation from the space of continuous functions C[0, T ; Rn ] into itself, such that for arbitrary y(·), y˜(·) ∈ C[0, T ; Rn ] sup |Ly(t) − L˜ y (t)| ≤

t∈[0,T ]



T

0

|A(s)| ds



sup |y(t) − y˜(t)|.

t∈[0,T ]

T If 0 |A(s) ds < 1, then by Theorem A.1 (the contraction mapping principle) the equation q = Lq has exactly one solution in C[0, T ; Rn ] which is the T solution of the integral equation. The case 0 |A(s)| ds ≥ 1 can be reduced to the previous one by considering the equation on appropriately shorter intervals. In particular we obtain the existence and uniqueness of a matrix valued function satisfying (1.5) and (1.6). To prove the second part of the theorem let us denote by ψ(t), t ∈ [0, T ], the matrix solution of d ψ(t) = −ψ(t)A(t), dt

ψ(0) = I, t ∈ [0, T ].

Assume that, for some t ∈ [0, T ], det S(t) = 0. Let T0 = min{t ∈ [0, T ]; det S(t) = 0}. Then T0 > 0, and for t ∈ [0, T0 )

d d d S(t) S −1 (t) + S(t) S −1 (t). 0 = (S(t)S −1 (t)) = dt dt dt Thus −A(t) = S(t)

d −1 S (t), dt

and consequently d −1 S (t) = −S −1 (t)A(t), dt

t ∈ [0, T0 ),

so S −1 (t) = ψ(t), t ∈ [0, T0 ). The function det ψ(t), t ∈ [0, T ], is continuous and det ψ(t) =

1 , det S(t)

t ∈ [0, T0 ),

therefore there exists a finite lim det ψ(t). This way det S(T0 ) = lim S(t) = 0, t ↑T0

t↑T0

a contradiction. The validity of (1.6) follows by elementary calculation.

2

The function S(t), t ∈ [0, T ], will be called the fundamental solution of equation (1.4). It follows from the proof that the fundamental solution of the “adjoint” equation

6

1 Controllability and observability

dp = −A∗ (t)p(t), dt

t ∈ [0, T ],

is (S ∗ (t))−1 , t ∈ [0, T ]. Exercise 1.1. Show that for A ∈ M(n, n) the series +∞  An n t , n! n=1

t ∈ R,

is uniformly convergent, with all derivatives, on an arbitrary finite interval. The sum of the series from Exercise 1.1 is often denoted by exp(tA) or etA , t ∈ R. We check easily that etA esA = e(t+s)A , in particular

(etA )−1 = e−tA ,

t, s ∈ R, t ∈ R.

Therefore the solution of (1.1) has the form 

t

tA

y(t) = e x +

0

 = S(t)x +

0

e(t−s)A Bu(s)ds t

S(t − s)Bu(s)ds,

(1.8) t ∈ [0, T ],

where S(t) = exp tA, t ≥ 0. The majority of the concepts and results from Part I discussed for systems (1.1)–(1.2) can be extended to time dependent matrices A(t) ∈ M(n, n), B(t) ∈ M(n, m), C(t) ∈ M(k, n), t ∈ [0, T ], and therefore for systems dy =A(t)y(t) + B(t)u(t), y(0) = x ∈ Rn , dt w(t) = C(t)y(t), t ∈ [0, T ].

(1.9) (1.10)

Some of these extensions will appear in the exercises. Generalizations to arbitrary finite-dimensional spaces of states and control parameters E and U are immediate.

§1.2. The controllability matrix An arbitrary function u(·) defined on [0, +∞) locally integrable and with values in Rm will be called a control , strategy or input of the system (1.1)– (1.2). The corresponding solution of equation (1.1) will be denoted by y x,u (·),

1.2 The controllability matrix

7

to underline the dependence on the initial condition x and the input u(·). Relationship (1.2) can be written in the following way: w(t) = Cy x,u (t),

t ∈ [0, T ].

The function w(·) is the output of the controlled system. We will assume now that C = I or equivalently that w(t) = y x,u (t), t ≥ 0. We say that a control u transfers a state a to a state b at the time T > 0 if y a,u (T ) = b.

(1.11)

We then also say that the state a can be steered to b at time T or that the state b is reachable or attainable from a at time T . The proposition below gives a formula for a control transferring a to b. In this formula the matrix QT , called the controllability matrix or controllability Gramian, appears:  QT =

0

T

S(r)BB ∗ S ∗ (r) dr,

T > 0.

We check easily that QT is symmetric and nonnegative definite (see the beginning of §1.1). Proposition 1.1. Assume that for some T > 0 the matrix QT is nonsingular. Then (i) for arbitrary a, b ∈ Rn the control u ˆ(s) = −B ∗ S ∗ (T − s)Q−1 T (S(T )a − b),

s ∈ [0, T ],

(1.12)

transfers a to b at time T ; (ii) among all controls u(·) steering a to b at time T the control u ˆ miniT mizes the integral 0 |u(s)|2 ds. Moreover,  0

T

 |ˆ u(s)|2 ds = Q−1 T (S(T )a − b), S(T )a − b .

(1.13)

Proof. It follows from (1.12) that the control u ˆ is smooth or even analytic. From (1.8) and (1.12) we obtain y a,ˆu (T ) = S(T )a −

 0

T

 S(T − s)BB ∗ S ∗ (T − s) ds (Q−1 T (S(T )a − b))

= S(T )a − QT (Q−1 T (S(T )a − b)) = b. This shows (i). To prove (ii) let us remark that the formula (1.13) is a consequence of the following simple calculations:

8

1 Controllability and observability



 T

∗ ∗

B S (T − s)Q−1 (S(T )a − b) 2 ds |ˆ u(s)|2 ds = T 0 0  T  −1 = S(T − s)BB ∗ S ∗ (T − s)(Q−1 (S(T )a − b)) ds, Q (S(T )a − b) T T T

0

−1 = QT Q−1 T (S(T )a − b), QT (S(T )a − b)

= Q−1 T (S(T )a − b), S(T )a − b. Now let u(·) be an arbitrary control transferring a to b at time T . We can assume that u(·) is square integrable on [0,T ]. Then  0

T



T

u(s), u ˆ(s) ds = − u(s), B ∗ S ∗ (T − s)Q−1 T (S(T )b − a) ds 0  T  −1 =− S(T − s)Bu(s) ds, QT (S(T )a − b) 0

= S(T )a − b, Q−1 T (S(T )a − b). Hence



T

0

 u(s), u ˆ(s) ds =

From this we obtain   T 2 |u(s)| ds = 0

0

T

2

0

T

ˆ u(s), u ˆ(s) ds.

|ˆ u(s)| ds +

 0

T

|u(s) − u ˆ(s)|2 ds 2

and consequently the desired minimality property. Exercise 1.2. Write the equations d2 y = u, dt2

y(0) = ξ1 ,

dy (0) = ξ2 , dt



ξ1 ξ2



∈ R2 ,

as a first-order system. Prove that for the new system, the matrix   QT is  ξ1 0 nonsingular, T > 0. Find the control u transferring the state to ξ2 0 T at time T > 0 and minimizing the functional 0 |u(s)|2 ds. Determine the minimal value m of the functional. Consider ξ1 = 1, ξ2 = 0. Answer. The required control is of the form

ξ2 T 2 sT ξ2 12 ξ1 T + − − sξ1 , u ˆ(s) = − 3 T 2 3 2

s ∈ [0, T ],

and the minimal value m of the functional is equal to

12 2T 2 2 2 (ξ2 ) . m = 3 (ξ1 ) + ξ1 ξ2 T − T 3

1.2 The controllability matrix

9

In particular, when ξ1 = 1, ξ2 = 0, u ˆ(s) =

T 12  s − , T3 2

s ∈ [0, T ], m =

12 . T3

We say that a state b is attainable or reachable from a ∈ Rn if it is attainable or reachable at some time T > 0. System (1.1) is called controllable if an arbitrary state b ∈ Rn is attainable from any state a ∈ Rn at some time T > 0. Instead of saying that system (1.1) is controllable we will frequently say that the pair (A, B) is controllable. If for arbitrary a, b ∈ Rn the attainability takes place at a given time T > 0, we say that the system is controllable at time T . Proposition 1.1 gives a sufficient condition for the system (1.1) to be controllable. It turns out that this condition is also a necessary one. The following result holds. Proposition 1.2. If an arbitrary state b ∈ Rn is attainable from 0, then the matrix QT is nonsingular for an arbitrary T > 0. Proof. Let, for a control u and T > 0,  LT u =

0

T

S(r)Bu(T − r) dr.

(1.14)

The formula (1.14) defines a linear operator from UT = L1 [0, T ; Rm ] into Rn . Let us remark that LT u = y 0,u (T ).

(1.15)

Let ET = LT (UT ), T > 0. It follows from (1.14) family of the linear  that the ET = Rn , taking into account spaces ET is nondecreasing in T > 0. Since T >0

the dimensions of ET , we have that ET˜ = Rn for some T˜. Let us remark that, for arbitrary T > 0, v ∈ Rn and u ∈ UT ,  T    T ∗ ∗ S(r)BB S (r) dr v, v = |B ∗ S ∗ (r)v|2 dr, (1.16) QT v, v =  LT u, v =

0

0

T

u(r), B ∗ S ∗ (T − r)v dr.

0

(1.17)

From identities (1.16) and (1.17) we obtain QT v = 0 for some v ∈ Rn if the space ET is orthogonal to v or if the function B ∗ S ∗ (·)v is identically equal to zero on [0, T ]. It follows from the analyticity of this function that it is equal to zero everywhere. Therefore if QT v = 0 for some T > 0 then QT v = 0 for all T > 0 and in particular QT˜ v = 0. Since ET˜ = Rn we have that v = 0, 2 and the nonsingularity of QT follows.

10

1 Controllability and observability

A sufficient condition for controllability is that the rank of B is equal to n. This follows from the next exercise. Exercise 1.3. Assume rank B = n and let B + be a matrix such that BB + = I. Check that the control u(s) =

1 + (s−T )A B e (b − eT A a), T

s ∈ [0, T ],

transfers a to b at time T ≥ 0.

§1.3. Rank condition We now formulate an algebraic condition equivalent to controllability. For matrices A ∈ M(n, n), B ∈ M(n, m) denote by [A|B] the matrix [B, AB, . . . , An−1 B] ∈ M(n, nm) which consists of consecutively written columns of matrices B, AB, . . . , An−1 B. Theorem 1.2. The following conditions are equivalent: (i) An arbitrary state b ∈ Rn is attainable from 0. (ii) System (1.1) is controllable. (iii) System (1.1) is controllable at a given time T > 0. (iv) Matrix QT is nonsingular for some T > 0. (v) Matrix QT is nonsingular for an arbitrary T > 0. (vi) rank[A|B] = n. Condition (vi) is called the Kalman rank condition, or the rank condition for short. The proof will use the Cayley–Hamilton theorem. Let us recall that a characteristic polynomial p(·) of a matrix A ∈ M(n, n) is defined by p(λ) = det(λI − A),

λ ∈ C.

(1.18)

Let p(λ) = λn + a1 λn−1 + . . . + an ,

λ ∈ C.

(1.19)

The Cayley–Hamilton theorem has the following formulation ([58, 219]): Theorem 1.3. For arbitrary A ∈ M(n, n), with the characteristic polynomial (1.19), An + a1 An−1 + . . . + an I = 0. Symbolically, p(A) = 0. Proof of Theorem 1.2 Equivalences (i)–(v) follow from the proofs of Propositions 1.1 and 1.2 and the identity

1.3 Rank condition

11

y a,u (T ) = LT u + S(T )a. To show the equivalences to condition (vi) it is convenient to introduce a linear mapping ln from the Cartesian product of n copies Rm into Rn : ln (u0 , . . . , un−1 ) =

n−1 

Aj Buj ,

uj ∈ Rm , j = 0, . . . , n − 1.

j=0

We prove first the following lemma. Lemma 1.1. The transformation LT , T > 0, has the same image as ln . In particular LT is onto if and only if ln is onto. Proof. For arbitrary v ∈ Rn , u ∈ L1 [0, T ; Rm ], uj ∈ Rm , j = 0, . . . , n − 1:  LT u, v =

0

T

u(s), B ∗ S ∗ (T − s)v ds,

ln (u0 , . . . , un−1 ), v = u0 , B ∗ v + . . . + un−1 , B ∗ (A∗ )n−1 v. Suppose that ln (u0 , . . . , un−1 ), v = 0 for arbitrary u0 , . . . , un−1 ∈ Rm . Then B ∗ v = 0, . . . , B ∗ (A∗ )n−1 v = 0. From Theorem 1.3, applied to matrix A∗ , it follows that for some constants c0 , . . . , cn−1 (A∗ )n =

n−1 

ck (A∗ )k .

k=0

Thus, by induction, for arbitrary l = 0, 1, . . . there exist constants cl,0 , . . . , cl,n−1 such that n−1  cl,k (A∗ )k . (A∗ )n+l = k=0 ∗

∗ k

Therefore B (A ) v = 0 for k = 0, 1, . . . . Taking into account that B ∗ S ∗ (t)v =

+∞ 

B ∗ (A∗ )k v

k=0

tk , k!

t ≥ 0,

we deduce that for arbitrary T > 0 and t ∈ [0, T ] B ∗ S ∗ (t)v = 0, so LT u, v = 0 for arbitrary u ∈ L1 [0, T ; Rm ]. Assume, conversely, that for arbitrary u ∈ L1 [0, T ; Rn ], LT u, v = 0. Then ∗ ∗ B S (t)v = 0 for t ∈ [0, T ]. Differentiating the identity

12

1 Controllability and observability +∞ 

B ∗ (A∗ )k v

k=0

tk = 0, k!

t ∈ [0, T ],

0, 1, . . . , (n − 1)-times and inserting each time t = 0, we obtain B ∗ (A∗ )k v = 0 for k = 0, 1, . . . , n − 1. And therefore ln (u0 , . . . , un−1 ), v = 0 for arbitrary u0 , . . . , un−1 ∈ Rm . 2

This implies the lemma.

Assume that the system (1.1) is controllable. Then the transformation LT is onto Rn for arbitrary T > 0 and, by the above lemma, the matrix [A|B] has rank n. Conversely, if the rank of [A|B] is n then the mapping ln is onto Rn and also, therefore, the transformation LT is onto Rn and the controllability of (1.1) follows. If the rank condition is satisfied then the control u ˆ(·) given by (1.12) transfers a to b at time T . We now give a different, more explicit, formula for the transfer control involving the matrix [A|B] instead of the controllability matrix QT . Note that if rank[A|B] = n then there exists a matrix K ∈ M(mn, n) such that [A|B]K = I ∈ M(n, n) or equivalently there exist matrices K1 , K2 , . . . , Kn ∈ M(m, n) such that BK1 + ABK2 + . . . + An−1 BKn = I.

(1.20)

Let, in addition, ϕ be a function of class C n−1 from [0, T ] into R such that d(j) ϕ d(j) ϕ (0) = (T ) = 0, j = 0, 1, . . . , n − 1, ds(j) ds(j)  T ϕ(s) ds = 1.

(1.21) (1.22)

0

Proposition 1.3. Assume that rank[A|B] = n and (1.20)–(1.22) hold. Then the control u ˜(s) = K1 ψ(s) + K2

dψ d(n−1) ψ (s) + . . . + Kn (n−1) (s), ds ds

s ∈ [0, T ],

where ψ(s) = S(s − T )(b − S(T )a)ϕ(s),

s ∈ [0, T ],

(1.23)

transfers a to b at time T ≥ 0. Proof. Taking into account (1.21) and integrating by parts (j − 1) times, we have

1.3 Rank condition

 0

T

S(T − s)BKj  =

0

T

13

d(j−1) ψ(s) ds = ds(j−1)



T

0

eA(T −s) Aj−1 BKj ψ(s) ds =

eA(T −s) BKj



T

0

d(j−1) ψ(s) ds ds(j−1)

S(T − s)Aj−1 BKj ψ(s) ds, j = 1, 2, . . . , n.

Consequently  0

T

 S(T − s)B u ˜(s) ds =

0

 =

T

0

T

S(t − s)[A|B]Kψ(s) ds S(T − s)ψ(s) ds.

By the definition of ψ and by (1.22) we finally have  y a,˜u (T ) = S(T )a +

0

T

S(T − s)(S(s − T )(b − S(T )a))ϕ(s) ds  = S(T )a + (b − S(T )a)

T 0

ϕ(s) ds = b.

2

Remark. Note that Proposition 1.3 is a generalization of Exercise 1.3. Exercise 1.4. Assuming that U = R prove that the system describing the electrically heated oven from Example 0.1 is controllable. Exercise 1.5. Let L0 be a linear subspace dense in L1 [0, T ; Rm ]. If system (1.1) is controllable then for arbitrary a, b ∈ Rn there exists u(·) ∈ L0 transferring a to b at time T . Hint. Use the fact that the image of the closure of a set under a linear continuous mapping is contained in the closure of the image of the set. Exercise 1.6. If system (1.1) is controllable then for arbitrary T > 0 and arbitrary a, b ∈ Rn there exists a control u(·) of class C ∞ transferring a to b at time T and such that d(j) u d(j) u (0) = (T ) = 0 for dt(j) dt(j)

j = 0, 1, . . . .

Exercise 1.7. Assuming that the pair (A, B) is controllable, show that the system y˙ = Ay + Bv v˙ = u, with the state space Rn+m and the set of control parameters Rm , is also controllable. Deduce that for arbitrary a, b ∈ Rn , u0 , u1 ∈ Rm and T > 0

14

1 Controllability and observability

there exists a control u(·) of class C ∞ transferring a to b at time T and such that u(0) = u0 , u(T ) = u1 . Hint. Use Exercise 1.6 and the Kalman rank condition. Exercise 1.8. Suppose that A ∈ M(n, n), B ∈ M(n, m). Prove that the system d2 y dy (0) ∈ Rn , = Ay + Bu, y(0) ∈ Rn , 2 dt dt is controllable in R2n if and only if the pair (A, B) is controllable. Exercise 1.9. Consider system (1.9) on [0, T ] with integrable matrix valued functions A(t), B(t), t ∈ [0, T ]. Let S(t), t ∈ [0, T ], be the fundamental solution of the equation q˙ = Aq. Assume that the matrix  QT =

0

T

S(T )S −1 (s)B(s)B ∗ (s)(S −1 (s))∗ S ∗ (T ) ds

is positive definite. Show that the control u ˆ(s) = B ∗ (S −1 (s))∗ S ∗ (T )Q−1 T (b − S(T )a),

s ∈ [0, T ],

transfers a to b at time T minimizing the functional u −→

T 0

|u(s)|2 ds.

§1.4. A classification of control systems Let y(t), t ≥ 0, be a solution of the equation (1.1) corresponding to a control u(t), t ≥ 0, and let P ∈ M(n, n) and S ∈ M(m, m) be nonsingular matrices. Define y˜(t) = P y(t),

u ˜(t) = Su(t),

t ≥ 0.

Then d d y˜(t) = P y(t) = P Ay(t) + P Bu(t) dt dt ˜y (t) + B ˜u = P AP −1 y˜(t) + P BS −1 u ˜(t) = A˜ ˜(t), where

A˜ = P AP −1 ,

˜ = P BS −1 . B

t ≥ 0, (1.24)

˜ B) ˜ are called equivalent The control systems described by (A, B) and (A, if there exist nonsingular matrices P ∈ M(n, n), S ∈ M(m, m), such that (1.24) holds. Let us remark that P −1 and S −1 can be regarded as transition matrices from old to new bases in Rn and Rm , respectively. The introduced

1.4 A classification of control systems

15

concept is an equivalence relation. It is clear that a pair (A, B) is controllable ˜ B) ˜ is controllable. if and only if (A, We now give a complete description of equivalent classes of the introduced relation in the case when m = 1. Let us first consider a system d(n) d(n−1) z + a z + . . . + an z = u, 1 dt(n) dt(n−1)

(1.25)

with initial conditions z(0) = ξ1 ,

dz d(n−1) z (0) = ξ2 , . . . , (0) = ξn . dt dt(n−1)

(1.26)

(n−1)

d z Let z(t), dz dt (t), . . . , dt(n−1) (t), t ≥ 0, be coordinates of a function y(t), t ≥ 0, and ξ1 , . . . , ξn coordinates of a vector x. Then

˜ + Bu, ˜ y˙ = Ay

y(0) = x ∈ Rn ,

˜ are of the form where matrices A˜ and B ⎡ 0 1 ... 0 ⎢ 0 0 ... 0 ⎢ ⎢ . .. . . .. A˜ = ⎢ .. . . . ⎢ ⎣ 0 0 ... 0 −an −an−1 . . . −a2

0 0 .. .

⎤

⎥ ⎥ ⎥ ⎥, ⎥ 1 ⎦ −a1

⎡ ⎤ 0 ⎢ .. ⎥ ⎥ ˜=⎢ B ⎢ . ⎥. ⎣0⎦ 1

(1.27)

(1.28)

˜ B] ˜ there are We easily check that on the main diagonal of the matrix [A| ˜ ˜ = n and, only ones and above the diagonal only zeros. Therefore rank[A|B] ˜ ˜ by Theorem 1.2, the pair (A, B) is controllable. Interpreting this result in terms of the initial system (1.21)–(1.22) we can say that for two arbitrary sequences of n numbers ξ1 , . . . , ξn and η1 , . . . , ηn and for an arbitrary positive number T there exists an analytic function u(t), t ∈ [0, T ], such that for the corresponding solution z(t), t ∈ [0, T ], of the equation (1.25)–(1.26) z(T ) = η1 ,

dz (T ) = η2 , dt

...,

d(n−1) z (T ) = ηn . dt(n−1)

Theorem 1.4 states that an arbitrary controllable system with the onedimensional space of control parameters is equivalent to a system of the form (1.25)–(1.26). Theorem 1.4. If A ∈ M(n, n), b ∈ M(n, 1) and the system y˙ = Ay + bu,

y(0) = x ∈ Rn

(1.29)

is controllable then it is equivalent to exactly one system of the form (1.28). Moreover the numbers a1 , . . . , an in the representation (1.24) are identical to

16

1 Controllability and observability

the coefficients of the characteristic polynomial of the matrix A: p(λ) = det[λI − A] = λn + a1 λn−1 + . . . + an ,

λ ∈ C.

(1.30)

Proof. By the Cayley–Hamilton theorem, An + a1 An−1 + . . . + an I = 0. In particular An b = −a1 An−1 b − . . . − an b. Since rank[A|b] = n, therefore vectors e1 = An−1 b, . . . , en = b are linearly independent and form a basis in Rn . Let ξ1 (t), . . . , ξn (t), be coordinates of the vector y(t) in this basis, t ≥ 0. Then ⎡ ⎤ ⎡ ⎤ 0 −a1 1 0 . . . 0 0 ⎢0⎥ ⎢ −a2 0 1 . . . 0 0 ⎥ ⎢ ⎥ ⎥ dξ ⎢ ⎢ .. .. . . .. .. ⎥ ξ + ⎢ .. ⎥ u. (1.31) = ⎢ ... ⎢.⎥ ⎥ . . . ⎥ . . ⎢ ⎥ ⎢ dt ⎣0⎦ ⎣−an−1 0 0 . . . 0 1 ⎦ 1 −an 0 0 . . . 0 0 Therefore an arbitrary controllable system (1.29) is equivalent to (1.31) and the numbers a1 , . . . , an are the coefficients of the characteristic polynomial ˜ of A. On the other hand, direct calculation of the determinant of [λI − A] gives ˜ = λn + a1 λn−1 + . . . + an = p(λ), λ ∈ C. det(λI − A) ˜ B) ˜ is equivalent to the system (1.31) and consequently Therefore the pair (A, also to the pair (A, b). 2 Remark. The problem of an exact description of the equivalence classes in the case of arbitrary m is much more complicated, see Bibliographical Notes.

§1.5. Kalman decomposition Theorem 1.2 gives several characterizations of controllable systems. Here we deal with uncontrollable ones. Theorem 1.5. Assume that rank[A|B] = l < n. There exists a nonsingular matrix P ∈ M(n, n) such that     A11 A12 B1 P AP −1 = , , PB = 0 A22 0 where A11 ∈ M(l, l), A22 ∈ M(n − l, n − l), B1 ∈ M(l, m). In addition the pair

1.5 Kalman decomposition

17

(A11 , B1 ) is controllable. The theorem states that there exists a basis in Rn such that system (1.1) written with respect to that basis has a representation ξ˙1 = A11 ξ1 + A12 ξ2 + B1 u, ξ˙2 = A22 ξ2 ,

ξ1 (0) ∈ Rl , ξ2 (0) ∈ Rn−l ,

in which (A11 , B1 ) is a controllable pair. The first equation describes the socalled controllable part and the second the completely uncontrollable part of the system. Proof. It follows from Lemma 1.1 that the subspace E0 = LT (L1 [0, T ; Rm ]) is identical with the image of the transformation ln . Therefore it consists of all elements of the form Bu1 + ABu1 + . . . + An−1 Bun , u1 , . . . , un ∈ Rm and is of dimension l. In addition it contains the image of B and by the Cayley– Hamilton theorem, it is invariant with respect to the transformation A. Let E1 be any linear subspace of Rn complementing E0 and let e1 , . . . , el and el+1 , . . . , en be bases in E0 and E1 and P the transition matrix from the new ˜ = P B, to the old basis. Let A˜ = P AP −1 , B       ξ1 A11 ξ1 + A12 ξ2 B1 u ˜ ˜ A , = , B= ξ2 A21 ξ1 + A22 ξ2 B2 u ξ1 ∈ Rl , ξ2 ∈ Rn−l , u ∈ Rm . Since the space E0 is invariant with respect to A,     ξ A11 ξ1 A˜ 1 = , ξ1 ∈ Rl . 0 0 Taking into account that B(Rm ) ⊂ E0 , B2 u = 0

for u ∈ Rm .

Consequently the elements of the matrices A22 and B2 are zero. This finishes the proof of the first part of the theorem. To prove the final part, let us remark that for the nonsingular matrix P ˜ B]. ˜ rank[A|B] = rank(P [A|B]) = rank[A| 

Since ˜ B] ˜ = [A|

 B1 A11 B1 . . . An−1 11 B1 , 0 0 ... 0

so ˜ B] ˜ = rank[A11 |B1 ]. l = rank[A| Taking into account that A11 ∈ M(l, l), one gets the required property.

2

18

1 Controllability and observability

Remark. Note that the subspace E0 consists of all points attainable from 0. It follows from the proof of Theorem 1.5 that E0 is the smallest subspace of Rn invariant with respect to A and containing the image of B, and it is identical to the image of the transformation represented by [A|B]. Exercise 1.10. Give a complete classification of controllable systems when m = 1 and the dimension of E0 is l < n.

§1.6. Observability Assume that B = 0. Then system (1.1) is identical with the linear equation z˙ = Az,

z(0) = x.

(1.32)

The observation relation (1.2) is left unchanged: (1.33)

w = Cz. The solution to (1.32) will be denoted by z x (t), t ≥ 0. Obviously z x (t) = S(t)x,

x ∈ Rn .

The system (1.32)–(1.33), or the pair (A, C), is said to be observable if for arbitrary x ∈ Rn , x = 0, there exists a t > 0 such that w(t) = Cz x (t) = 0. If for a given T > 0 and for arbitrary x = 0 there exists t ∈ [0, T ] with the above property, then the system (1.32)–(1.33) or the pair (A, C) are said to be observable at time T. Let us introduce the so-called observability matrix :  RT =

0

T

S ∗ (r)C ∗ CS(r) dr.

The following theorem, dual to Theorem 1.2, holds. Theorem 1.6. The following conditions are equivalent: (i) System (1.32)–(1.33) is observable. (ii) System (1.32)–(1.33) is observable at a given time T > 0. (iii) The matrix RT is nonsingular for some T > 0. (iv) The matrix RT is nonsingular for arbitrary T > 0. (v) rank[A∗ |C ∗ ] = n. Proof. Analysis of the function w(·) implies the equivalence of (i) and (ii). Besides,

1.6 Observability

 0

T

19

|w(r)|2 dr =

 0

 =

T

0

T

|Cz x (r)|2 dr S ∗ (r)C ∗ CS(r)x, x dr = RT x, x.

Therefore observability at time T ≥ 0 is equivalent to RT x, x = 0 for arbitrary x = 0 and consequently to nonsingularity of the nonnegative, symmetric matrix RT . The remaining equivalences are consequences of Theorem 1.2 and the observation that the controllability matrix corresponding to (A∗ , C ∗ ) is 2 exactly RT . Example 1.1. Let us consider the equation d(n) z d(n−1) z + a + . . . + an z = 0 1 dt(n) dt(n−1)

(1.34)

and assume that w(t) = z(t),

t ≥ 0.

(1.35)

Matrices A and C corresponding to (1.34)–(1.35) are of the form ⎡ ⎤ 0 1 ··· 0 0 ⎢ 0 0 ··· 0 0 ⎥ ⎢ ⎥ ⎢ .. . . .. ⎥ , C = [1, 0, . . . , 0] . .. . .. A=⎢ . ⎥ . . . ⎢ ⎥ ⎣ 0 0 ··· 0 1 ⎦ −an −an−1 · · · −a2 −a1 We check directly that rank[A∗ |C ∗ ] = n and thus the pair (A, C) is observable. The next theorem is analogous to Theorem 1.5 and gives a decomposition of system (1.32)–(1.33) into observable and completely unobservable parts. Theorem 1.7. Assume that rank[A∗ |C ∗ ] = l < n. Then there exists a nonsingular matrix P ∈ M(n, n) such that   A11 0 −1 , CP −1 = [C1 , 0], P AP = A21 A22 where A11 ∈ M(l, l), A22 ∈ M(n − l, n − l) and C1 ∈ M(k, l) and the pair (A11 , C1 ) is observable. Proof. The theorem follows directly from Theorem 1.5 and from the observation that a pair (A, C) is observable if and only if the pair (A∗ , C ∗ ) is controllable. 2 Remark. It follows from the above theorem that there exists a basis in Rn such that the system (1.1)–(1.2) has representation

20

1 Controllability and observability

ξ˙1 = A11 ξ1 + B1 u, ξ˙2 = A21 ξ1 + A22 ξ2 + B2 u, η = C1 ξ1 , and the pair (A11 , C1 ) is observable.

Bibliographical notes Basic concepts of the chapter are due to R. Kalman [55]. He is also the author of Theorems 1.2, 1.5, 1.6 and 1.7. Exercise 1.3 as well as Proposition 1.3 are due to R. Triggiani [94]. A generalization of Theorem 1.4 to arbitrary m leads to the so-called controllability indices discussed in W.A. Wolvich [99], W.M. Wonham [101], R.W. Brockett [12].

Chapter 2

Stability and stabilizability

In this chapter stable linear systems are characterized in terms of associated characteristic polynomials and Lyapunov equations. A proof of the Routh theorem on stable polynomials is given as well as a complete description of completely stabilizable systems. Luenberger’s observer is introduced and used to illustrate the concept of detectability.

§2.1. Stable linear systems Let A ∈ M(n, n) and consider linear systems z˙ = Az,

z(0) = x ∈ Rn .

(2.1)

Solutions of equation (2.1) will be denoted by z x (t), t ≥ 0. In accordance with earlier notations we have z x (t) = S(t)x = (exp tA)x,

t ≥ 0.

The system (2.1) is called stable if for arbitrary x ∈ Rn z x (t) −→ 0 as t ↑ +∞. Instead of saying that (2.1) is stable we will often say that the matrix A is stable. Let us remark that the concept of stability does not depend on the choice of the basis in Rn . Therefore if P is a nonsingular matrix and A is a stable one, then the matrix P AP −1 is stable. In what follows we will need the Jordan theorem [6] on canonical representation of matrices. Denote by M(n, m; C) the set of all matrices with n rows and m columns and with complex elements. Let us recall that a number λ ∈ C is called an eigenvalue of a matrix A ∈ M(n, n; C) if there exists a vector a ∈ Cn , a = 0, such that Aa = λa. The set of all eigenvalues of a © Springer Nature Switzerland AG 2020

J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_2

21

22

2 Stability and stabilizability

matrix A will be denoted by σ(A) and called spectrum of A. Since λ ∈ σ(A) if and only if the matrix λI − A is singular, therefore λ ∈ σ(A) if and only if p(λ) = 0, where p is a characteristic polynomial of A: p(λ) = det[λI − A], λ ∈ C. The set σ(A) consists of at most n elements and is nonempty. A vector v ∈ Cn , Av = λv, is called an eigenvector corresponding to the eigenvalue λ. If for some k ≥ 2, (λ − A)k v = 0, then v is called a generalized eigenvector. Theorem 2.1. For an arbitrary matrix A ∈ M(n, n; C) there exists a nonsingular matrix P ∈ M(n, n; C) such that ⎤ ⎡ J1 0 ··· 0 0 ⎢ 0 0 ⎥ J2 · · · 0 ⎥ ⎢ ⎢ .. . . .. ⎥ = A, −1 . ˜ .. . . .. (2.2) P AP = ⎢ . . ⎥ ⎥ ⎢ ⎦ ⎣ 0 0 · · · Jr−1 0 0 0 ··· 0 Jr where J1 , J2, . . . , Jr ⎡ λk γk ⎢ 0 λk ⎢ ⎢ .. .. Jk = ⎢ . . ⎢ ⎣ 0 0 0 0

are the so-called Jordan blocks ⎤ ··· 0 0 ··· 0 0 ⎥ ⎥ . .. ⎥ , γ = 0 or J = [λ ], .. .. k k k . . ⎥ ⎥ · · · λk γk ⎦ ··· 0 λk

k = 1, . . . , r.

The set of all eigenvectors and generalized eigenvectors is linearly dense in C. In the representation (2.2) at least one Jordan block corresponds to an eigenvalue λk ∈ σ(A). Selecting matrix P properly one can obtain a representation with numbers γk = 0 given in advance. For matrices with real elements the representation theorem has the following form: Theorem 2.2. For an arbitrary matrix A ∈ M(n, n) there exists a nonsingular matrix P ∈ M(n, n) such that (2.2) holds with “real” blocks Ik . Blocks Ik , k = 1, . . . , r, corresponding to real eigenvalues λk = αk ∈ R are of the form ⎤ ⎡ αk λk · · · 0 0 ⎢ 0 αk · · · 0 0 ⎥ ⎥ ⎢ ⎥ ⎢ [αk ] or ⎢ ... ... . . . ... ... ⎥ , γk = 0, γk ∈ R, ⎥ ⎢ ⎣ 0 0 · · · αk γk ⎦ 0 0 · · · 0 αk and corresponding to complex eigenvalues λk = αk + iβk , βk = 0, αk , βk ∈ R,

2.1 Stable linear systems

⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣

Kk 0 .. . 0 0

Lk Kk .. .

··· ··· .. .

0 0 .. .

23

0 0 .. .

⎤

⎥ ⎥ ⎥ ⎥ ⎥ 0 · · · Kk Lk ⎦ 0 · · · 0 Kk



where

 αk βk γk 0 Kk = , Lk = , −βk αk 0 γk

compare [6]. We now prove the following theorem. Theorem 2.3. Assume that A ∈ M(n, n). The following conditions are equivalent: (i) z x (t) −→ 0 as t ↑ +∞, for arbitrary x ∈ Rn . (ii) z x (t) −→ 0, exponentially as t ↑ +∞, for arbitrary x ∈ Rn . (iii) ω(A) = sup{Re λ; λ ∈ σ(A)} < 0.

+∞ (iv) 0 |z x (t)|2 dt < +∞, for arbitrary x ∈ Rn . For the proof we will need the following lemma. Lemma 2.1. Let ω > ω(A). For arbitrary norm  ·  on Rn there exist constants M such that z x (t) ≤ M eωt x

for t ≥ 0 and x ∈ Rn .

Proof. Let us consider (2.1) with the matrix A in the Jordan form (2.2) w(0) = x ∈ Cn .

˜ x˙ = Aw,

For a = a1 + ia2 , where a1 , a2 ∈ Rn set a = a1  + a2 . Let us decompose vector w(t), t ≥ 0, and the initial state x into sequences of vectors w1 (t), . . . , wr (t), t > 0, and x1 , . . . , xr according to the decomposition (2.2). Then w˙ k = Jk wk , wk (0) = xk , k = 1, . . . , r. Let j1 , . . . , jr denote the dimensions of the matrices J1 , . . . , Jr , j1 + j2 + . . . + jr = n. If jk = 1 then wk (t) = eλk t xk , t ≥ 0. So wk (t) = e(Re λk )t xk , t ≥ 0. If jk > 1, then ⎡ λk t

wk (t) = e

⎢ ⎢ ⎢ ⎢ ⎣

j k −1 ⎢ l=0

0 γk . . . 0 0 ... .. .. . . . . . 0 0 ... 0 0 ...

⎤l 0 0 0 0⎥ ⎥ l .. .. ⎥ x t . k . . ⎥ ⎥ l! 0 γk ⎦ 0 0

24

2 Stability and stabilizability

So wk (t) ≤ e(Re λk )t xk 

j k −1 l=0

tl (Mk )l , l!

t ≥ 0,

where Mk is the norm of the transformation represented by ⎡ ⎤ 0 γk . . . 0 0 ⎢ 0 0 ... 0 0 ⎥ ⎢ ⎥ ⎢ .. .. . . .. .. ⎥ . ⎢ . . . . . ⎥ ⎢ ⎥ ⎣ 0 0 . . . 0 γk ⎦ 0 0 ... 0 0 Setting ω0 = ω(A) we get r 

wk (t) ≤ eω0 t q(t)

k=1

r 

xk ,

t ≥ 0,

k=1

where q is a polynomial of order at most max(jk − 1), k = 1, . . . , r. If ω > ω0 and

M0 = sup q(t)e(ω0 −ω)t ; t ≥ 0 , then M0 < +∞ and r 

ωt

wk (t) ≤ M0 e

k=1

r 

xk ,

t ≥ 0.

k=1

Therefore for a new constant M1 w(t) ≤ M1 eωt x,

t ≥ 0.

Finally z x (t) = P w(t)P −1  ≤ M1 eωt P  P −1  x, and this is enough to define M = M1 P  P −1 .

t ≥ 0, 2

Proof of the theorem. Assume ω0 ≥ 0. There exist λ = α + iβ, Re λ = α ≥ 0 and a vector a = 0, a = a1 + ia2 , a1 , a2 ∈ Rn such that A(a1 + ia2 ) = (α + iβ)(a1 + ia2 ). The function

z(t) = z1 (t) + iz2 (t) = e(α+iβ)t a,

t ≥ 0,

as well as its real and imaginary parts, is a solution of (2.1). Since a = 0, either a1 = 0 or a2 = 0. Let us assume, for instance, that a1 = 0 and β = 0. Then

2.2 Stable polynomials

25

z1 (t) = eαt [(cos βt)a1 − (sin βt)a2 ],

t ≥ 0.

Inserting t = 2πk/β, we have |z1 (t)| = eαt |a1 | and, taking k ↑ +∞, we obtain z1 (t) → 0. Now let ω0 < 0 and α ∈ (0, −ω0 ). Then by the lemma |z x (t)| ≤ M e−αt |x|

for t ≥ 0 and x ∈ Rn .

This implies (ii) and therefore also (i). It remains to consider (iv). It is clear that it follows from (ii) and thus also from (iii). Let us assume that condition (iv) holds and ω0 ≥ 0. Then |z1 (t)| = eαt |a1 |, t ≥ 0, and therefore  0

+∞

|z1 (t)|2 dt = +∞, 2

a contradiction. The proof is complete. Exercise 2.1. The matrix



0 1 A= −2 −2



corresponds to the equation z¨ + 2z˙ + 2z = 0. Calculate ω(A). For ω > ω(A) find the smallest constant M = M (ω) such that |S(t)| ≤ M eωt , t ≥ 0. Hint. Prove that |S(t)| = ϕ(t)e−t , where ϕ(t) =

1/2 1 2 + 5 sin2 t + (20 sin2 t + 25 sin4 t)1/2 , 2

t ≥ 0.

§2.2. Stable polynomials Theorem 2.3 reduces the problem of determining whether a matrix A is stable to the question of finding out whether all roots of the characteristic polynomial of A have negative real parts. Polynomials with this property will be called stable. Because of its importance, several efforts have been made to find necessary and sufficient conditions for the stability of an arbitrary polynomial (2.3) p(λ) = λn + a1 λn−1 + . . . + an , λ ∈ C,

26

2 Stability and stabilizability

with real coefficients, in term of the coefficients a1 , . . . , an . Since there is no general formula for roots of polynomials of order greater than 4, the existence of such conditions is not obvious. Therefore their formulation in the nineteenth century by Routh was a kind of a sensation. Before formulating and proving a version of the Routh theorem we will characterize stable polynomials of degree smaller than or equal to 4 using only the fundamental theorem of algebra. We deduce also a useful necessary condition for stability. Theorem 2.4. (1) Polynomials with real coefficients: (i) λ + a, (ii) λ2 + aλ + b, (iii) λ3 + aλ2 + bλ + c, (iv) λ4 + aλ3 + bλ2 + cλ + d are stable if and only if, respectively, ∗ (i) a > 0, ∗ (ii) a > 0, b > 0, ∗ (iii) a > 0, b > 0, c > 0 and ab > c, ∗ (iv) a > 0, b > 0, c > 0, d > 0 and abc > c2 + a2 d. (2) If polynomial (2.3) is stable then all its coefficients a1 , . . . , an are positive. Proof. (1) Equivalence (i) ⇐⇒ (i)∗ is obvious. To prove (ii) ⇐⇒ (ii)∗ assume that the roots of the polynomial are of the form λ1 = −α + iβ, λ2 = −α − iβ, β = 0. Then p(λ) = λ2 + 2αλ + α2 + β 2 , λ ∈ C, and therefore the stability conditions are a > 0 and b > 0. If the roots λ1 , λ2 of the polynomial p are real then a = −(λ1 + λ2 ), b = λ1 λ2 . Therefore they are negative if and only if a > 0, b > 0. To show that (iii) ⇐⇒ (iii)∗ let us remark that the fundamental theorem of algebra, see [58], p.14, implies the following decomposition of the polynomial, with real coefficients α, β, γ: p(λ) = λ3 + aλ2 + bλ + c = (λ + α)(λ2 + βλ + γ),

λ ∈ C.

It therefore follows from (i) and (ii) that the polynomial p is stable if only if α > 0, β > 0 and γ > 0. Comparing the coefficients gives a = α + β,

b = γ + αβ,

c = αγ,

and therefore ab − c = β(α2 + γ + αβ) = β(α2 + b). Assume that a > 0, b > 0, c > 0 and ab − c > 0. It follows from b > 0 and ab − c > 0 that β > 0. Since c = αγ, α and γ are either positive or negative. They cannot, however, be negative because then b = γ + αβ < 0. Thus α > 0 and γ > 0 and consequently α > 0, β > 0, γ > 0. It is clear from the above formulae that the positivity of α, β, γ implies inequalities (iii)∗ . To prove (iv) ⇐⇒ (iv)∗ we again apply the fundamental theorem of algebra to obtain the representation

2.3 The Routh theorem

27

λ4 + aλ3 + bλ2 + cλ + d = (λ2 + αλ + β)(λ2 + γλ + δ) and the stability condition α > 0, β > 0, γ > 0, δ > 0. From the decomposition a = α + γ,

b = αγ + β + δ,

c = αδ + βγ,

d = βδ,

we check directly that abc − c2 − a2 d = αγ((β − δ)2 + ac). It is therefore clear that α > 0, β > 0, γ > 0 and δ > 0, and then (iv)∗ holds. Assume now that the inequalities (iv)∗ are true. Then αγ > 0, and, since a = α + γ > 0, therefore α > 0 and δ > 0. Since, in addition, d = βδ > 0 and c = αδ + βγ > 0, so β > 0, δ > 0. Finally α > 0, β > 0, γ > 0, δ > 0 and the polynomial p is stable. (2) By the fundamental theorem of algebra, the polynomial p is a product of polynomials of degrees at most 2 which, by (1), have positive coefficients. This implies the result. 2 Exercise 2.2. Find necessary and sufficient conditions for the polynomial λ2 + aλ + b with complex coefficients a and b to have both roots with negative real parts. Hint. Consider the polynomial (λ2 + aλ + b)(λ2 + a ¯λ + ¯b) and apply Theorem 2.4. Exercise 2.3. Equation L2 C z¨ + RLC z¨ + 2Lz˙ + Rz = 0,

R > 0, L > 0, C > 0,

describes the action of the electrical filter from Example 0.4. Check that the associated characteristic polynomial is stable.

§2.3. The Routh theorem We now present the theorem, mentioned earlier, which allows to check, in a finite number of steps, that a given polynomial p(λ) = λn +a1 λn−1 +. . .+an , λ ∈ C, with real coefficients is stable. As we already know, a stable polynomial has all coefficients positive, but this condition is not sufficient for stability if n > 3. Let U and V be polynomials with real coefficients given by U (x) + iV (x) = p(ix),

x ∈ R.

28

2 Stability and stabilizability

Let us remark that deg U = n, deg V = n − 1 if n is an even number and deg U = n − 1, deg V = n, if n is an odd number. Denote f1 = U , f2 = V if deg U = n, deg V = n − 1, and f1 = V , f2 = U if deg V = n, deg U = n − 1. Let f3 , f4 , . . . , fm be polynomials obtained from f1 , f2 by an application of the Euclid algorithm. Thus deg fk+1 < deg fk , k = 2, . . . , m − 1 and there exist polynomials κ1 , . . . , κm such that fk−1 = κk fk − fk+1 ,

fm−1 = κm fm .

Moreover the polynomial fm is equal to the largest common divisor of f1 , f2 multiplied by a constant. The following theorem is due to F.J. Routh [87]. Theorem 2.5. A polynomial p is stable if m = n + 1 and the leading coefficients of the polynomials f1 , . . . , fn+1 are all either strictly positive or strictly negative. Proof. The proof follows that of Gantmacher [40] and uses some geometrical ideas from complex analysis. We start from a result expressing the number of roots with positive and with negative real parts of a polynomial in terms of the change of the argument of the function p(iω) as ω changes between −∞ and +∞.

iω z1 z1 iω Note that if Re z1 < 0 then the argument of p(iω) = iω−z1 increases and if Re z1 > 0 the argument of p(iω) decreases as a function of ω ∈ (−∞, ∞). The total increase of the argument Δ+∞ −∞ p(iω) between −∞ and +∞ is equal to π in the former case and −π in the latter case. Since the sum of the arguments of two complex numbers different from zero is an argument of the product of the numbers we have the following observation. Lemma 2.2. If p is a polynomial of degree n without roots on the imaginary axis then Δ+∞ −∞ p(iω) = (n − 2k)π where k is the number of roots of p, counting multiplicity, with positive real parts.

2.3 The Routh theorem

29

Assume that f1 , f2 are polynomials with real coefficients, deg f1 ≥ deg f2 with distinct roots. Then the index Iω˜ ff21 of the rational function ff21 at the point ω ˜ is equal to 1, Iω˜ ff21 = 1, if lim

ω↑˜ ω

f2 (ω) = −∞ and f1 (ω)

lim

f2 (ω) = +∞. f1 (ω)

lim

f2 (ω) = −∞. f1 (ω)

ω↓˜ ω

Similarly, Iω˜ ff21 = −1 if lim

ω↑˜ ω

f2 (ω) = +∞ and f1 (ω)

ω↓˜ ω

f2 +∞ f2 The Cauchy index I−∞ f1 is the sum of all indexes Iωk f1 corresponding to all roots ωk of f1 . +∞ f2 Lemma 2.3. If I−∞ f1 = n then p has no roots on the imaginary axis and all its roots have negative real parts. If all roots of p have negative real parts +∞ f2 then I−∞ f1 = n. +∞ f2 Proof. If I−∞ f1 = n then f1 should have n distinct roots and they are different from the roots of f2 . Since f1 , f2 do not have a common root, p has no roots on the imaginary axis. By Lemma 2.2 it is enough to show that Δ+∞ −∞ arg p(iω) = n. We assume that n is an even number, the reasoning if n is odd is similar. +∞ −V −V −V ˜ of U should be If I−∞ ˜ U of U at each root ω U = n then the index Iω equal to 1. If U (˜ ω ) = 0 then U (ω) = (ω − ω ˜ )H(ω), for some polynomial H such that H(˜ ω ) = U . If H(˜ ω ) > 0 then U (ω) < 0 for ω < ω ˜ close to ω ˜ . Thus V (˜ ω ) < 0. Let us enumerate anticlockwise the quadrants {x > 0, y > 0}, {x < 0, y > 0}, {x < 0, y < 0}, {x > 0, y < 0}, as respectively, first, second, third and fourth. Thus if H(˜ ω ) > 0 the arc p(i · ) crosses the imaginary axis passing from the third quadrant to the fourth. If H(˜ ω ) < 0 then U (ω) > 0 for ω < ω ˜ and close to ω ˜ and U (ω) < 0 for ω > ω ˜ and close to ω ˜ and V (˜ ω ) > 0. Thus the arc p(i · ) crosses the imaginary axis passing from the first quadrant to the second. Since the arc p(i · ) crosses the imaginary axis exactly n times, the arc rotates around the origin anticlockwise. This is only possible if arg p(iω), ω ∈ R, is an increasing function. Since

lim

ω→+∞

V (ω) V (ω) = 0 = lim , ω→−∞ U (ω) U (ω)

the increase of arg p(iω) for ω between −∞ and the smallest root of U is π2 and similarly the increase of arg p(iω) for ω between the largest root of U and +∞ is π2 . Consequently the total increase of the argument of p(iω), ω ∈ R, is nπ as required. To establish the second part of the lemma note that arg p(iω), ω ∈ R, must be an increasing function and p(i·) will cross the imaginary axis from

30

2 Stability and stabilizability

the first to the second or from the third to the fourth quadrants. Each crossing will result in a jump of −V U from −∞ to +∞. As there will be n crossings, +∞ f2 Ind−∞ f1 = n. 2 Proof of the theorem. If p is a stable polynomial then by Lemma 2.3 f2 Ind+∞ −∞ f1 = n. If in the sequence f1 , . . . , fm , m < n + 1, then deg fm ≥ 1 and the polynomial fm would divide both f1 and f2 and the degree of the denominator in ff21 would be less than n which is impossible. Thus m = ω ) = 0 and 1 < k < n + 1, then fk−1 (˜ ω ) = 0, n + 1. Note that if fk (˜ ω ) = 0, as in the opposite case, by a simple induction argument one fk+1 (˜ ω ) = 0 and this is impossible as f1 , f2 do not have common would get fn+1 (˜ ω ) = −fk+1 (˜ ω ) and therefore fk−1 (˜ ω )fk+1 (˜ ω ) < 0. roots. In addition fk−1 (˜ Consequently if Z(ω), ω ∈ R1 , is the number of sign changes in the sequence (f1 (ω), . . . , fn+1 (ω)), then Z cannot change its value crossing roots of any f2 , . . . , fn . However, when ω crosses a zero of f1 , Z looses one sign as ff21 passes from −∞ to +∞. Thus Z(ω) = n for ω smaller than the roots of all polynomials f1 , . . . , fn+1 . Denote by bk the leading coefficient of the polynomial fk , k = 1, . . . , n + 1. If |ω| is large then sign fk (ω) = sign(bk ω n+1−k ). Thus for ω negative with |ω| large the signs in the sequence (f1 (ω), . . . , fn+1 (ω)) are the same as in the sequence b1 (−1)n , b2 (−1)n−1 , b3 (−1)n−2 , . . . , bn+1 (−1)0 . There are exactly n sign changes if and only if b1 , . . . , bn+1 are either all positive or all negative. This proves the result in one direction. To prove the f2 2. opposite it is enough to note that Ind+∞ −∞ f1 = n and apply Lemma 2.3. Example 2.1. If p(z) = z 2 + az + b, then U (ω) = −ω 2 + b, Thus

f1 (ω) = −ω 2 + b,

V (ω) = aω,

f2 (ω) = −aω,

ω ∈ R1 . f3 (ω) = −b,

and therefore Z(−∞) is equal to the number of sign changes in (−, sign a, sgn(−b)). Consequently Z(−∞) = 2 if and only if a > 0, b > 0. The leading coefficients of (f1 , f2 , f3 ) are (−1, −a, −b) and are of the same sign if a > 0, b > 0. Example 2.2. If p(z) = z 3 + az 2 + bz + c, then U (ω) = −aω 2 + c, V (ω) = −ω 3 + bω and f1 (ω) = −ω 2 + bω, f2 (ω) = −aω 2 + c, f3 (ω) = (b − ac )ω, f4 (ω) = −c. The leading coefficients of the polynomials f1 , . . . , f4 are −, −a, b − ac , −c. They are of the same sign iff a > 0, b − ac > 0, c > 0, and we arrive at the known stability condition.

2.4 Stability, observability and Lyapunov equation

31

Example 2.3. If p(z) = z 4 + az 3 + bz 2 + cz + d, then f1 (ω) = ω 4 − bω 2 + d,

f2 (ω) = −aω 3 + cω,

f4 (ω) = −(c − ad(b − ac )−1 )ω,

f3 (ω) = (b − ac )ω 2 + d, f5 (ω) = d.

The signs of their leading coefficients are +, sgn(−a), sgn(b − ac ), sgn(−(c − ad(b − ac )−1 )), sgn d, and they are all positive if and only if a > 0, b − ac > 0, c − ad(b − ac )−1 < 0, d > 0. We arrive again at the known stability condition. We leave as an exercise the proof that the Routh theorem leads to an explicit stability algorithm. To formulate it we have to define the so-called Routh array. For arbitrary sequences (αk ), (βk ), the Routh Sequences (γk ) is defined by  1 α α γk = − det 1 k+1 , k = 1, 2, . . . β1 βk+1 β1 If a1 , . . . , an are coefficients of a polynomial p, we set additionally ak = 0 for k > n = deg p. The Routh array is a matrix with infinite rows obtained from the first two rows: 1, a2 , a4 , a6 , . . . , a1 , a3 , a5 , a7 , . . . , by consecutive calculations of the Routh sequences from the two preceding rows. The calculations stop if 0 appears in the first column. The Routh algorithm can be now stated as the theorem Theorem 2.6. A polynomial p of degree n without roots on the imaginary axis is stable if and only if the n + 1 first elements of the first columns of the Routh array are positive. Exercise 2.4. Show that, for an arbitrary polynomial p(λ) = λn + a1 λn−1 + . . . + an , λ ∈ C, with complex coefficients a1 , . . . , an , the polynomial (λn + ¯1 λn−1 + . . . + a ¯n ) has real coefficients. Formulate a1 λn−1 + . . . + an )(λn + a necessary and sufficient conditions for the polynomial p to have all roots with negative real parts.

§2.4. Stability, observability and the Lyapunov equation An important role in stability considerations is played by the so-called Lyapunov equation of the form A∗ Q + QA = −R,

(2.4)

32

2 Stability and stabilizability

in which are given a symmetric matrix R ∈ M(n, n) and a matrix A ∈ M(n, n) and unknown is a symmetric matrix Q ∈ M(n, n). A connection between the equation (2.4) and the stability question is illustrated by the following theorem: Theorem 2.7. (1) Assume that the pair (A, C) is observable and R = C ∗ C. If there exists a nonnegative definite matrix Q satisfying (2.4) then the matrix A is stable. (2) If the matrix A is stable then for an arbitrary symmetric matrix R the equation (2.4) has exactly one symmetric solution Q. This solution is positive (nonnegative) definite if the matrix R is positive (nonnegative) definite. Proof. (1) It follows from (2.4) that d ∗ S (t)QS(t) = −S ∗ (t)RS(t), dt Let (see §1.6)  RT =

T

0

∗



S (r)RS(r) dr =

0

T

t ≥ 0.

S ∗ (r)C ∗ CS(r) dr,

(2.5)

T ≥ 0.

Integrating equation (2.5) from 0 to T we obtain Q = RT + S ∗ (T )QS(T ). It follows from the observability of (A, C) that the matrix RT is positive definite and therefore the matrix Q is also positive definite. Let x ∈ Rn be a fixed vector different from 0. Define v(t) = Qz x (t), z x (t) ,

t > 0.

It is enough to show that v(t) −→ 0 as t ↑ +∞. Let us remark that dv (t) = − Rz x (t), z x (t) = −|Cz x (t)|2 ≤ 0, dt

t ≥ 0.

The function v is nondecreasing on [0, +∞), and therefore the trajectory z x (t), t ≥ 0, is bounded and sup(|z x (t)|t ≥ 0) < +∞ for arbitrary x ∈ Rn . This implies easily that for arbitrary λ ∈ σ(A), Re λ ≤ 0. Assume that for an ω ∈ R, iω ∈ σ(A). If ω = 0 then, for a vector a = 0 with real elements, Aa = 0. Then 0 = (A∗ Q+QA)a, a = −|Ca|2 , a contradiction with Theorem 1.6(v). If ω = 0 and a1 , a2 ∈ Rn are vectors such that a1 + ia2 = 0 and iω(a1 + ia2 ) = Aa1 + iAa2 , then the function z a1 (t) = a1 cos ωt − a2 sin ωt,

t ≥ 0,

(2.6)

2.4 Stability, observability and Lyapunov equation

33

is a periodic solution of the equation z˙ = Az. Periodicity and the formula d Qz a1 (t), z a1 (t) = −|Cz a1 (t)|2 ≤ 0, dt

t ≥ 0,

imply that for a constant γ ≥ 0 Qz a1 (t), z a1 (t) = γ,

t ≥ 0.

Hence |Cz a1 (t)| = 0 for t ≥ 0. However, by the observability of (A, C) and (2.6), a1 = a2 = 0, a contradiction. So Re λ < 0 for all λ ∈ σ(A) and the matrix A is stable. (2) Assume that the matrix A is stable. Then the matrix  +∞ S ∗ (r)RS(r) dr = lim RT Q= T ↑+∞

0

is well defined. Moreover A∗ Q + QA =

 0



+∞

(A∗ S ∗ (r)RS(r) + S ∗ (r)RS(r)A) dr

+∞

d ∗ S (r)RS(r) dr dr 0 = lim (S ∗ (T )RS(T ) − R) = −R,

=

T ↑+∞

and therefore Q satisfies the Lyapunov equation. It is clear that if the matrix R is positive (nonnegative) definite then also the matrix Q is positive (nonnegative) definite. ˜ is also a symmetric It remains to show the uniqueness. Assume that Q solution of the equation (2.4). Then d ∗ ˜ − Q)S(t)) = 0 for t ≥ 0. (S (t)(Q dt Consequently

˜ − Q)S(t) = Q ˜ − Q for t ≥ 0. S ∗ (t)(Q

˜ Since S(t) −→ 0, S ∗ (t) −→ 0 as t ↑ +∞, 0 = Q−Q. The proof of the theorem is complete. 2 Remark. Note that the pair (A, I) is always observable, so if there exists a nonnegative solution Q of the equation A∗ Q + QA = −I, then A is a stable matrix.

34

2 Stability and stabilizability

As a corollary we deduce the following result: Theorem 2.8. If a pair (A, C) is observable and for arbitrary x ∈ Rn w(t) = Cz x (t) −→ 0,

as t ↑ +∞,

then the matrix A is stable and consequently z x (t) −→ 0 as t ↑ +∞ for arbitrary x ∈ Rn Proof. For arbitrary β > 0 a nonnegative matrix  +∞ Qβ = e−2βt S ∗ (t)C ∗ CS(t) dt 0

is well defined and satisfies the equation (A − β)∗ Q + Q(A − β) = −C ∗ C. For sufficiently small β > 0, the pair ((A − β), C) is also observable, therefore for arbitrary β > 0, sup{Re λ; λ ∈ σ(A)} ≤ β. One has only to show that the matrix A has no eigenvalues on the imaginary axis. For this purpose it is enough to repeat the reasoning from the final part of the proof of Theorem 2.7(1). 2 Exercise 2.5. Deduce from Theorem 2.8 the following result: If for arbitrary initial conditions z(0) = ξ1 ,

dz (0) = ξ2 , dt

...,

d(n−1) z (0) = ξn dt(n−1)

solutions z(t), t > 0, of the equation (1.30) tend to 0 as t −→ +∞ then for arbitrary k = 1, 2, . . . , n − 1 d(k) z −→ 0, dt(k)

as t ↑ +∞.

§2.5. Stabilizability and controllability We say that the system y˙ = Ay + Bu,

y(0) = x ∈ Rn ,

(2.7)

is stabilizable or that the pair (A, B) is stabilizable if there exists a matrix K ∈ M(m, n) such that the matrix A + BK is stable. So if the pair (A, B) is stabilizable and a control u(·) is given in the feedback form u(t) = K y(t),

t ≥ 0,

2.5 Stabilizability and controllability

35

then all solutions of the equation y(t) ˙ = Ay(t) + BKy(t) = (A + BK)y(t),

y(0) = x,

t ≥ 0,

(2.8)

tend to zero as t ↑ +∞. We say that system (2.7) is completely stabilizable if and only if for arbitrary ω > 0 there exist a matrix K and a constant M > 0 such that for an arbitrary solution y x (t), t ≥ 0, of (2.8) |y x (t)| ≤ M e−ωt |x|,

t ≥ 0.

(2.9)

By pK we will denote the characteristic polynomial of the matrix A + BK. One of the most important results in the linear control theory is given by the Wonham theorem. Theorem 2.9. The following conditions are equivalent: (i) System (2.7) is completely stabilizable. (ii) System (2.7) is controllable. (iii) For arbitrary polynomial p(λ) = λn + α1 λn−1 + . . . + αn , λ ∈ C, with real coefficients, there exists a matrix K such that p(λ) = pK (λ)

for

λ ∈ C.

Proof. We start with the implication (ii)=⇒(iii) and prove it in three steps. Step 1. The dimension of the space of control parameters m = 1. It follows from §1.4 that we can limit our considerations to systems of the form d(n−1) z d(n) z (t) + a (t) + . . . + an z(t) = u(t), 1 dt(n) dt(n−1)

t ≥ 0.

In this case, however, (iii) is obvious: it is enough to define the control u in the feedback form, u(t) = (a1 − α1 )

d(n−1) z (t) + . . . + (an − αn )z(t), dt(n−1)

t ≥ 0,

and use the result (see §1.4) that the characteristic polynomial of the equation d(n) z d(n−1) z + α + . . . + αn z = 0, 1 dt(n) dt(n−1) or, equivalently, of the matrix ⎡ 0 1 ⎢ 0 0 ⎢ ⎢ .. .. ⎢ . . ⎢ ⎣ 0 0 −αn −αn−1

⎤ ··· 0 0 ··· 0 0 ⎥ ⎥ . .. ⎥ , .. . .. . ⎥ ⎥ ··· 0 1 ⎦ · · · −α2 −α1

36

2 Stability and stabilizability

is exactly

p(λ) = λn + α1 λn−1 + . . . + αn λ,

λ ∈ C.

Step 2. The following lemma allows us to reduce the general case to m = 1. Note that in its formulation and proof its vectors from Rn are treated as one-column matrices. Lemma 2.4. If a pair (A, B) is controllable then there exist a matrix L ∈ M(m, n) and a vector v ∈ Rm such that the pair (A+BL, Bv) is controllable. Proof of the lemma. It follows from the controllability of (A, B) that there exists v ∈ Rm such that Bv = 0. We show first that there exist vectors u1 , . . . , un−1 in Rm such that the sequence e1 , . . . , en defined inductively e1 = Bv, el+1 = Ael + Bul

for l = 1, 2, . . . , n − 1

(2.10)

is a basis in Rn . Assume that such a sequence does not exist. Then for some k ≥ 0 vectors e1 , . . . , ek , corresponding to some u1 , . . . , uk are linearly independent, and for arbitrary u ∈ Rm the vector Aek + Bu belongs to the linear space E0 spanned by e1 , . . . , ek . Taking u = 0 we obtain Aek ∈ E0 . Thus Bu ∈ E0 for arbitrary u ∈ Rm and consequently Aej ∈ E0 for j = 1, . . . , k. This way we see that the space E0 is invariant for A and contains the image of B. Controllability of (A, B) implies now that E0 = Rn , and compare the remark following Theorem 1.5. Consequently k = n and the required sequences e1 , . . . , en and u1 , . . . , un−1 exist. Let un be an arbitrary vector from Rm . We define the linear transformation L setting Lel = ul , for l = 1, . . . , n. We have from (2.10) el+1 = Ael + BLel = (A + BL)el = (A + BL)l e1 = (A + BL)l Bv,

l = 0, 1, . . . , n − 1.

Since [A + BL|Bv] = [e1 , e2 , . . . , en ], the pair (A + BL, Bv) is controllable. 2 Step 3. Let a polynomial p be given and let L and v be the matrix and vector constructed in the Step 2. The system y˙ = (A + BL)y + (Bv)u, in which u(·) is a scalar control function, is controllable. It follows from Step 1 that there exists k ∈ Rn such that the characteristic polynomial of (A + BL) + (Bv)k ∗ = A + B(L + vk ∗ ) is identical with p. The required feedback K can be defined as K = L + vk ∗ .

2.6 Hautus lemma

37

We proceed to the proofs of the remaining implications. To show that (iii) =⇒ (ii) assume that (A, B) is not controllable, that rank[A|B] = l < n and that K is a linear feedback. Let P ∈ M(n, n) be a nonsingular matrix from Theorem 1.5. Then pK (λ) = det[λI − (A + BK)] = det[λI − (P AP −1 + P BKP −1 )]  −A12 (λI − (A11 + B1 K1 )) = det 0 (λI − A22 ) = det[λI − (A11 + B1 K1 )] det[λI − A22 ],

λ ∈ C,

where K1 ∈ M(m, n). Therefore for arbitrary K ∈ M(m, n) the polynomial pK has a nonconstant divisor, equal to the characteristic polynomial of A22 , and therefore pK cannot be arbitrary. This way the implication (iii) =⇒ (ii) is true. Assume now that condition (i) holds but that the system is not controllable. By the above argument we have for arbitrary K ∈ M(m, n) that σ(A22 ) ⊂ σ(A + BK). So if for some M > 0, ω > 0 condition (2.9) holds then ω ≤ − sup{Re λ; λ ∈ σ(A22 )}, which contradicts complete stabilizability. Hence (i)=⇒(ii). Assume now that (ii) and therefore (iii) hold. Let p(λ) = λn + a1 λn−1 + . . . + an , λ ∈ C be a polynomial with all roots having real parts smaller than −ω (e.g. p(λ) = (λ + ω + ε)n , ε > 0). We see from (iii) that there exists a matrix K such that pK (·) = p(·). Consequently all eigenvalues of A + BK have real parts smaller than −ω. By Theorem 2.3, condition (i) holds. The proof of Theorem 2.9 is complete. 2

§2.6. Hautus lemma The following theorem, called Hautus lemma, is of frequent use in control theory, see Bibliographical notes. Theorem 2.10. A control system (A, B) is controllable if and only if (i) for each λ ∈ C the matrix [λI − A, B] has full rank and if and only if (ii) the matrix [λI − A, B] has full rank for all λ ∈ σ(A). Proof. Assume that the pair (A, B), where A, B are, respectively, n × n and n×m matrices, is controllable but for some λ the rank of the matrix [λI−A, B] is less than n. Then there exists a vector p = 0 such that p, (λI − A)u1 + Bu0 = 0 for all u0 , u1 ∈ Cn . Equivalently for all u0 , u1 ∈ Cn

38

2 Stability and stabilizability

p, Au1 = p, λu1 and p, Bu0 = 0. We claim that then, for all k = 0, 1, . . . , n − 1, p, Ak Buk = p, λk Buk = 0,

uk ∈ Cn .

(2.11)

In fact, if (2.11) holds for some k then for all uk+1 ∈ Cn p, Ak+1 Buk+1 = p, AAk Buk+1 = p, λAk Buk+1 = λ p, Ak Buk+1 = λ p, λk Buk+1 = p, λk+1 Buk+1 = 0. Thus the matrix [A|B] cannot be of rank n. Conversely, assume that the pair (A, B) is not controllable, then by Theorem 1.5 one can assume that   A11 A12 B1 A= , B= 0 A22 0 and (A11 , B1 ) is controllable. One can always find in Cn−l a vector p2 = 0 and λ ∈ C such that     0 0 , = 0 for all u2 ∈ Cn−l . λu2 − A22 u2 p2 However, for all

u1  u2

∈ Cn ,

   1     1    0 u 0 u A11 u1 + A12 u2 , (λI − A) = , λ − p2 p2 u2 u2 A22 u2     0 0 = , = 0. 2 2 p λu − A22 u2 Obviously

   1      0 u 0 B1 u2 , B = , = 0. p2 p2 u2 0  1   Consequently p02 = 0 and for some λ and all uu2    1  1  0 u u , (λI − A) 2 + B 2 =0 p2 u u contrary to the assumptions. If λ = σ(A) then the matrix λI − A is of full rank and (i) is equivalent to (ii). The proof of the theorem is complete. 2

2.7 Detectability and dynamical observers

39

§2.7. Detectability and dynamical observers We say that a pair (A, C) is detectable if there exists a matrix L ∈ M(n, k) such that the matrix A + LC is stable. It follows from Theorem 2.9 that controllability implies stabilizability. Similarly, observability implies detectability. For if the pair (A, C) is observable then (A∗ , C ∗ ) is controllable and there exists a matrix K such that A∗ +C ∗ K is a stable matrix. Therefore the matrix A + K ∗ C is stable and it is enough to set L = K ∗ . We illustrate the importance of the concept of detectability by discussing the dynamical observer introduced by Luenberger. Let us consider the system (1.1)–(1.2). Since the observation ω = Cy, it is natural to define an output stabilizing feedback as K ∈ M(m, k) such that the matrix A + BKC is stable. It turns out, however, that even imposing conditions like controllability of (A, B) and observability of (A, C) it is not possible, in general, to construct the desired K. In short, the output stabilizability is only very rarely possible. Consider for instance the system d(n−1) z d(n) z (t) + (n−1) (t) + . . . + an z = u (n) dt dt

(2.12)

with the observation relation w(t) = z(t),

t ≥ 0.

(2.13)

We know from earlier considerations that if system (2.12)–(2.13) is written in the form (1.1)–(1.2) then pairs (A, B), (A, C) are, respectively, controllable and observable. Let a feedback strategy u be of the form u = kw where k is a constant. Then the system (2.12) becomes d(n) z d(n−1) z + a + . . . + an z = −kz, 1 dt(n) dt(n−1) and the corresponding characteristic polynomial is of the form p(λ) = λn + a1 λn−1 + . . . + an−1 λ + an − k,

λ ∈ C.

So if some of the coefficients a1 , . . . , an−1 are negative then there exists no k such that the obtained system is stable. Therefore it was necessary to extend the concept of stabilizability allowing feedback based on dynamical observers which we define as follows: We say that matrices F ∈ M(r, r), H ∈ M(r, n), D ∈ M(r, m) define an r-dimensional dynamical observer if for arbitrary control u(·), solution z(·) of the equation z(t) ˙ = F z(t) + Hw(t) + Du(t),

t ≥ 0,

(2.14)

40

2 Stability and stabilizability

and the estimator t ≥ 0,

(2.15)

as t ↑ +∞.

(2.16)

yˆ(t) = V w(t) + W z(t), one has y(t) − yˆ(t) −→ 0,

The equation (2.14) and the number r are called, respectively, the equation and the dimension of the observer. The following theorem gives an example of an n-dimensional observer and shows its applicability to stabilizability. Theorem 2.11. (1) Assume that the pair (A, C) is detectable and let L be a matrix such that the matrix A + LC is stable. Then the equations z˙ = (A + LC)z − Lw + Bu, yˆ = z define a dynamical observer of order n. (2) If, in addition, the pair (A, B) is stabilizable, K is a matrix such that A + BK is stable and a control u is given by (2.17)

u = kˆ y, then y(t) −→ 0 as t ↑ +∞. Proof. (1) From the definition of the observer

d (ˆ y − y) = (A + LC)ˆ y + Bu − LCy − Ay − Bu = (A + LC)(ˆ y − y). dt Since the matrix A + LC is stable, therefore yˆ(t) − y(t) −→ 0 exponentially as t ↑ +∞. (2) If the control u is of the form (2.17), then yˆ˙ = (A + BK)ˆ y + LC(ˆ y − y). Therefore  yˆ(t) = SK (t)y(0) +

0

t

SK (t − s)ϕ(s) ds,

where SK (t) = exp t(A + BK), ϕ(t) = LC(ˆ y (t) − y(t)), t ≥ 0. There exist constants M > 0, ω > 0, such that |SK (t)| ≤ M e−ωt ,

t ≥ 0.

t ≥ 0,

2.7 Detectability and dynamical observers

41

Consequently for arbitrary T > 0 and t ≥ T  t   T   M −ωt   SK (t − s)ϕ(s) ds ≤ M e eωs |ϕ(s)| ds + sup |ϕ(s)| .  ω T ≤s≤t 0 0 So we see that yˆ(t) −→ 0 as t ↑ +∞ and by (1), y(t) −→ 0 as t ↑ +∞.

2

Remark. The obtained result can be summarized as follows. If the pair (A, C) is detectable and the pair (A, B) is stabilizable then the system (1.1)– (1.2) is output stabilizable with respect to the modified output z(t), t ≥ 0, based on the original observation w(t), t ≥ 0.

Bibliographical notes In the proof of the Routh theorem we basically follow F.R. Gantmacher [40]. There exist proofs which do not use analytic function theory. In particular, in P.C. Parks [75] the proof is based on Theorem 2.7 from §2.4. For numerous modifications of the Routh algorithm we refer to W.M. Wonham [101]. The proof of Theorem 2.9 is due to W.M. Wonham [100]. The Hautus lemma was discovered by V.M. Popov, see [79]. Its applicability was a subject of several papers by M.L.J. Hautus, see, e.g. [43], [44]. The main aim of §2.7 was to illustrate the concept of detectability.

Chapter 3

Controllability with vanishing energy

A new class of systems controllable with vanishing energy is introduced and characterized. An interplay between the class of null controllable systems with vanishing energy and Liouville’s theorem on bounded harmonic functions is discussed. Applications to orbital transfer problem for a satellite system are presented.

§3.1. Characterization theorems We consider again a linear control system y˙ = Ay + Bu,

y(0) = x.

(3.1)

The energy required to produce control action u(t), t ∈ [0, T ], is by definition equal to  T |u(s)|2 ds. 0

This definition is often used in the literature, although more adequate seems to be the quantity  T |u(s)| ds. 0

We keep the traditional definition and for comments on the interplay between these two see M. Shibata, A. Ichikawa [90] and S. Ivanov [50]. We say that the system (3.1) is controllable with vanishing energy (CVE) if for arbitrary two states a, b ∈ Rn and arbitrary ε > 0 there exist T > 0 and control u ∈ L2 ([0, T ], Rm ) such that y

a,u

 (T ) = b

and

© Springer Nature Switzerland AG 2020

0

T

|u(s)|2 ds < ε.

J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_3

43

44

3 Controllability with vanishing energy

We say that the system (3.1) is null controllable with vanishing energy (NCVE) if for arbitrary a ∈ Rn and arbitrary ε > 0 there exist T > 0 and u ∈ L2 ([0, T ]; Rm ) such that y

a,u

 (T ) = 0 and

T

0

|u(s)|2 ds < ε.

Theorem 3.1. The system (3.1) is (NCVE) if and only if (i) the system is controllable, (ii) P = 0 is the only non negative solution of the algebraic Riccati equation A∗ P + P A − P BB ∗ P = 0. Proof. It follows from Proposition 1.1 that  t  inf |u(s)|2 ds; y x,u (t) = 0 = Pt x, x,

(3.2)

0

where ∗

tA Pt = etA Q−1 t e ,

 Qt =

0

t

∗

esA BB ∗ esA ds,

t ≥ 0.

Note that Pt is a positive definite operator. We check now that it does satisfy the following matrix Riccati equation: d Pt = A∗ Pt + Pt A − Pt BB ∗ Pt . dt

(3.3)

In fact  d ∗ d tA tA∗ d tA Pt = etA (Q−1 (Q−1 t e )+e t e ) dt dt  dt    d −1 tA −1 d tA ∗ tA∗ −1 tA tA∗ e + Qt Q e = A e Qt e + e dt t dt d  ∗ ∗ tA etA + etA Q−1 Q−1 = A∗ Pt + etA t t e A. dt

(3.4)

However, 0=

d  d  d d , I = (Qt Q−1 Qt Q−1 Q−1 t )= t + Qt t dt dt dt dt

and

d  d −1 −1 tA ∗ tA∗ −1 Qt = −Q−1 Qt Q−1 Qt . t t = −Qt e BB e dt dt Therefore from (3.4) and (3.5) d Pt = A∗ Pt − Pt BB ∗ Pt + Pt A, dt

(3.5)

3.1 Characterization theorems

45

as required. It is clear from (3.2) that Pt , t > 0, is decreasing in the sense that Pt ≤ Ps for t ≥ s. Moreover, system (3.1) is (NCVE) if and only if lim Pt = 0. To t→+∞

prove Theorem 3.1 assume that (3.1) is (NCVE) but there exists a solution P ≥ 0 of (E) different from 0. Then the constant function Pt = P,

t ≥ 0,

satisfies (3.3) with the initial condition P . However, from Theorem III.9.3  t

2 x,u x,u inf |u(s)| ds + P y (t), y (t) = P t x, x = P x, x. 0

For each control u such that y x,u (t) = 0, one has  0

t

2

|u(s)| ds + P y

x,u

(t), y

x,u

 (t) =

t

0

|u(s)|2 ds,

therefore P x, x = P t x, x ≤ P t x, x, and if (3.1) is (NCVE), lim Pt x, x = 0, one gets P = 0, a contradiction. t→+∞

On the other hand, assume that the only non negative solution to (E) is 0 and consider the limit P˜ of the decreasing function Pt , t ≥ 0. Since Pt , t ≥ 0, satisfies (3.3), one can easily prove that P˜ ≥ 0 satisfies (E). Thus P˜ = 0 and the (NCVE) follows. 2 Theorem 3.2. System (3.1) is (NCVE) if and only if (i) the system is controllable, (ii) Re λ ≤ 0 for λ ∈ σ(A). Proof. Assume that (3.1) is (NCVE) and there exists μ ∈ σ(A) such that Re μ > 0. Denote the state space of the system by E. Using the representation Theorem 2.2 one can decompose E = E0 ⊕ E1 , as the direct sum of two subspaces invariant for A with the restrictions to E0 , E1 denoted by A0 , A1 and σ(A0 ) = {μ}. Let us split (3.1) into two systems: y0 (t) = A0 y0 (t) + B0 u, y1 (t) = A1 y1 (t) + B1 u,

y0 (0) = x0 ∈ E0 , y1 (0) = x1 ∈ E1 ,

where B0 , B1 transform U into E0 , E1 , respectively. It is clear that the systems (A0 , B0 ), (A1 , B1 ), as well as (−A0 , B0 ) are controllable. The family e−tA0 is exponentially stable (on E0 ) and the controllability operators  Rt =

0

t

∗

e−sA0 B0 B0∗ e−sA ds

46

3 Controllability with vanishing energy

are well defined isomorphisms on E0 . The operator  +∞ ∗ R= e−sA0 B0 B0∗ e−sA ds 0

is the unique solution of the Lyapunov equation (−A0 )R + R(−A0 )∗ = −B0 B0∗ . Thus R−1 = P is a non negative, different from 0, solution of the equation A∗0 P + P A0 − P B0 B0∗ P = 0. Since (A0 , B0 ) is (NCVE) we arrive at a contradiction with Theorem 3.1. To prove the opposite implication we will show that if Re λ ≤ 0 for all λ ∈ σ(A) and P is a non negative solution to (E) then for all eigenvectors and generalized eigenvectors v of A, P v = 0. By Theorem 2.1 this will complete the proof. The following result needed in the proof is a consequence of Proposition 1.2 and formulae (1.16). If the system (A, B) is controllable then for arbitrary T > 0 there exists a constant CT > 0 such that for all v  T  ∗ A∗ s 2 B e v  ds ≥ CT |v|2 . (3.6) 0

Assume that for all λ ∈ σ(A), one has Re λ ≤ 0 and that P is a solution to A∗ P + P A = P BB ∗ P. Step 1. Assume that Av = λv, and Re λ ≤ 0. Then A∗ P v, v + P Av, v = |B P v|2 and ∗

A∗ P v, v + λP v, v = P v, Av + λP v, v = P v, λv + λP v, v = 2 Re λ · P v, v = |B ∗ P v|2 . Since Re λ ≤ 0 one gets B∗P v = 0

(3.7)

A∗ P v+P Av = A∗ P v + λP v = P BB ∗ P v = 0 and A∗ P v = −λP v. ∗

Consequently eA s P v = e−λs P v and by (3.7)  0

 B ∗ eA∗ s P v 2 ds =

T

 0

 e−λs B ∗ P v 2 ds = 0.

T

(3.8)

3.1 Characterization theorems

47

From (3.6) and (3.8)  0=

0

 B ∗ eA∗ s P v 2 ds ≥ CT |P v|2

T

and thus P v = 0. Step 2. We argue by induction. Assume that for some k and w such that (λ − A)k w = 0 one has P w = 0. If (λ − A)k+1 v = 0 then (λ − A)k (λ − A)v = 0 and therefore P (λ − A)v = 0. Note that A∗ P v, v + P Av, v = |B ∗ P v|2 , P v, Av + Av, P v = |B ∗ P v|2 . But v, P Av + P Av, v = 2 ReP Av, v and therefore 2 ReP Av, v = |B ∗ P v|2 = 2 ReλP v, v = 2 Re λP v, v. Thus B ∗ P v = 0. Repeating the argument from Step 1 one gets P v = 0.

2

Theorem 3.3. System (3.1) is (CVE) if and only if (i) the system is controllable, (ii) Re λ = 0 for all λ ∈ σ(A). (A,B)

Proof. For a control system (A, B) and states a, b define Et (a, b), the minimal energy needed to transform the state a to b in time t, by the formula  t  (A,B) Et (a, b) = inf |y a,u (s)|2 ds; y a,u (t) = b . 0

By Proposition 1.1, for controllable systems  −1/2 At 2 (A,B) (a, b) = Qt (e a − b) . Et Moreover, there exists and optimal control u ˆ, given by the formula u ˆ(s) = −B ∗ eA

∗

(t−s)

At Q−1 t (e a − b),

s ∈ (0, t).

To prove the theorem, assuming a priori that the system (A, B) is controllable, we have to show the following equivalence: (A,B)

lim Et

t→+∞

(a, b) = 0 for all a, b ⇐⇒ σ(A) ⊂ {iλ; λ ∈ R}.

Note that if u(s), s ∈ (0, t), is a control for which the system (A, B) transfers a to b in time t, then for the control v(s) = u(t − s), s ∈ (0, t), the system (−A, −B) transfers b to a and obviously with the same energy. Thus

48

3 Controllability with vanishing energy (A,B)

Et

(−A,−B)

(a, b) = Et

(b, a). (A,B)

To show the implication =⇒ remark that, in particular, lim Et t→+∞

(a, 0) = 0

for all a, and by Theorem 3.2, for all λ ∈ σ(A), Re λ ≤ 0. However also (A,B) (−A,−B) lim Et (0, b) = lim Et (b, 0) = 0 for all b. Again by Theorem

t→+∞

t→+∞

3.2, for all λ ∈ σ(−A), Re λ ≤ 0 and the implication has been established. To prove the opposite implication note that for all a, b, t > 0, (A,B)

Et

(A,B)

(a, b) ≤ Et/2

(A,B)

≤ Et/2

(A,B)

(a, 0) + Et/2

(0, b)

(−A,−B)

(a, 0) + Et/2

(3.9)

(b, 0).

Both systems (A, B) and (−A, −B) are null controllable with vanishing energy and hence by (3.9), (A, B) is controllable with vanishing energy. 2 The next theorem shows how to construct feedback strategy which has small energy and steers a state asymptotically to 0. Theorem 3.4. Assume that system (3.1) is (NCVE). (1) For arbitrary ε > 0 there exists exactly one non negative solution Pε of the equation A∗ P + P A − P BB ∗ P + εI = 0 and Pε decreases to 0 as ε ↓ 0. (2) If uε (t) = −B ∗ Pε yε (t), where

t ≥ 0,

y˙ ε = Ayε + Buε (t) = (A − BB ∗ Pε )yε ,

then

 lim yε (t) = 0

t→∞

and 0

+∞

yε (0) = a,

|uε (s)|2 ds ≤ Pε a, a.

Proof. We use results on the so-called linear-quadratic control problems established in Part III, §9.4. In fact by Theorem III.9.4 with Q = εI, R − I, the equation A∗ Pε + Pε A + εI − Pε BB ∗ Pε = 0, has the unique solution Pε and    +∞ x,u

2 2 ε|y (s)| +|u(s)| ds = Pε x, x = inf u

 +∞

0

0

Pε ≥ 0,

+∞

(3.10)

ε|yε (s)|2 +|uε (s)|2 ds.

Thus 0 |uε (s)|2 ds ≤ Pε x, x. Since Pε decreases as ε ↓ 0 there exists P = lim Pε and passing with ε to 0 in (3.10) one gets ε↓0

3.2 Orbital Transfer Problem

49

A∗ P + P A − P BB ∗ P = 0. By NCVE, P = 0. Since  +∞  2 ε|yε (s)| ds = ε 0

 es(A−BB ∗ Pε ) x2 ds < +∞,

+∞  0

so the matrix A − BB ∗ Pε is stable and lim yε (t) = 0. t→+∞



Remark. As for each a ∈ Rn , limPε a, a = 0, the energy required by conε↓0

trol uε can be made arbitrarily small by choosing ε small enough.

§3.2. Orbital Transfer Problem We discuss now an application of the above material to the Orbital Transfer Problem introduced in Example 0.6. We follow M. Shibata and A. Ichikawa [90].

Fig. 3.1 Target and chaser

Consider an orbital station moving around the Earth and a spacecraft in its neighbourhood. Let us call them the target and the chaser, respectively. It is of interest to find a control which moves the chaser from a given orbit to a new one using as small amount of energy as possible. If the final orbit coincides with the position of the target one arrives at the optimal landing problem. Assume that the station moves around the Earth on the circle of radius R0 . If M denotes the mass of the Earth, G is the gravitational constant, T the time length of one rotation of the station around the Earth, and μ = GM , then

50

3 Controllability with vanishing energy

 ω=

μ R03

1/2 =

2π T

is the station rate. Let us consider the right-handed coordinate system (x, y) fixed at the centre of mass of the target. If x axis is along the radial direction and y axis is along the flight direction of the target then the Newton equations for the two-dimensional motion are of the form μ(R0 + x) ((R0 + x)2 + y 2 )1/2 μy y¨ = −2ω x˙ + ω 2 y − . ((R0 + x)2 + y 2 )1/2

x ¨ = 2ω y˙ + ω 2 (R0 + x) −

Linearization around the origin of the system leads to the equations x ¨ = 2ω y˙ + 3ω 2 x + u1 y¨ = −2ω x˙ + u2 with initial position x0 , x˙ 0 , y0 , y˙ 0 . We added the steering functions u1 , u2 . ˙ x3 = y, x4 = y˙ and z is the state vector with coordinates If x1 = x, x2 = x, x1 , x2 , x3 , x4 , then ⎛ ⎛ ⎞ ⎞ 0 1 0 0 0 0   ⎜3ω 2 0 ⎜ 1 0 ⎟ u1 0 2ω ⎟ ⎜ ⎜ ⎟ ⎟ z˙ = ⎝ . (3.11) z+⎝ 0 0 0 1⎠ 0 0 ⎠ u2 0 −2ω 0 0 0 1 We propose to solve the following two exercises devoted to properties of the control system. Exercise 3.1. Taking into account that x(t) = x1 (t), y(t) = x3 (t), and x˙ 1 = x3

x1 (0) = x0

x˙ 2 = x4

x2 (0) = x˙ 0 2

x˙ 3 = 2ωx4 + 3ω x1

x3 (0) = y0

x˙ 4 = −2ωx3

x4 (0) = y˙ 0

show that

 2  1 y˙ 0 − 3x0 + y˙ 0 cos ωt + x˙ 0 sin ωt ω ω ω  2 2 4  x˙ 0 + x˙ 0 cos ωt + 6x0 + y˙ 0 sin ωt − (6ωx0 + 3y˙ 0 )t. y(t) = y0 − ω ω ω

x(t) = 4x0 +

2

Exercise 3.2. Deduce that the solution (x(t), y(t)), t ≥ 0, is periodic if and only if y˙ 0 = −2ωx0 . Check that the trajectory of the periodic solution is an

3.2 Orbital Transfer Problem

ellipse:

51

x2 (y − d)2 + =1 a2 (2a)2 

where a=

x20 +

 x˙ 2 0

ω

,

d = y0 −

2 ω

x˙ 0 .

Thus free motion is periodic if and only if y˙ 0 = −2ωx0 and it is given by the formulae 1 x(t) = x0 cos ωt + x˙ 0 sin ωt ω 2 2 x˙ 0 + x˙ 0 cos ωt − 2x0 sin ωt. y(t) = y0 − ω ω The eigenvalues of the matrix A of (3.11) are {0, ωi, −ωi} and lie on the imaginary axis. Moreover, the system (3.11) is controllable as the corresponding matrix [A|B] has rank 4. Thus the linear approximation of the real system satisfies all conditions of Theorem 3.3 and therefore it is controllable with vanishing energy.

Fig. 3.2 Orbit transfer

We will sketch now an approximate solution of the problem of transferring with small energy the chase vehicle to a selected orbit which coincides with the orbit of a virtual vehicle. Assume that at time 0 the chase vehicle is at the state x10 and uses control u(t), t ≥ 0. The virtual vehicle starts from the state x20 . If y1 and y2 are the trajectories of the chase vehicle and the virtual one, then

52

3 Controllability with vanishing energy

y˙ 1 = Ay1 + Bu,

y1 (0) = x10 = (x0 , x˙ 0 , y0 , y˙ 0 ),

y˙ 2 = Ay2 ,

y2 (0) = x20 .

Consequently for the trajectory y(t) = y1 (t) − y2 (t), t ≥ 0, one has y˙ = Ay + Bu,

y(0) = x10 − x20 .

Using Theorem 3.4 we see that if the control u (t) = −B ∗ Pε y(t), t ≥ 0, then  lim y(t) = 0

t→+∞

+∞

and 0

  | u(t)|2 dt ≤ Pε (x10 − x20 ), x10 − x20 .

Thus for large t > 0, y1 (t) is almost y2 (t). Choosing ε > 0 small and x10 ,  will x20 minimizing the form Pε (x10 − x20 ), x10 − x20  the feedback control u result in almost exact transfer with very small energy indeed. Note that the control u (t) = −B ∗ Pε (y1 (t) − y2 (t)), t ≥ 0, is an affine function of the state and the chase vehicle will move according to the equation y˙ 1 (t) = Ay1 (t) + B u(t),

y1 (0) = x10 .

§3.3. NCVE and generalized Liouville’s theorem There exists a connection between controllable systems with vanishing energy and a generalization of Liouville’s theorem on harmonic functions. Let us recall that a function h : Rn → R1 is harmonic if Δh(x) =

n  ∂2h (x) = 0, ∂x2k

x ∈ Rn ,

k=1

and the classical Liouville theorem states: Bounded harmonic functions are constant. The extension is concerned with Ornstein–Uhlenbeck operators L on Rn of the form Lh(x) = Ax, Dh(x) + Tr B ∗ D2 h(x)B, x ∈ Rn .

(3.12)

Here Dh and D2 h denote, respectively, the gradient and the Hessian of h:    2  ∂h ∂ h 2 Dh(x) = (x), k = 1, . . . , n , D h(x) = (x), k, l = 1, . . . , n . ∂xk ∂xk ∂xl If A = 0, B = I, then L = Δ. If A = a0 10 , B = 01 , then

3.3 NCVE and generalized Liouville’s theorem

Lh(x) = x2

53

∂h ∂h ∂2h (x) + ax1 (x) + (x), ∂x1 ∂x2 ∂x22

x = (x1 , x2 ) ∈ R2 ,

(3.13)

and is called Kolmogorov’s operator. The mentioned extension of Liouville’s theorem is the following result of E. Priola and the author [80]. Theorem 3.5. Bounded solutions h of the equation Lh(x) = 0,

x ∈ Rn ,

where L is the Ornstein–Uhlenbeck operator (3.12) are constant if and only if the control system (A, B) is NCVE. The proof is in E. Priola and J. Zabczyk [80]. Note that if L = Δ, then the pair (A, B) is controllable and σ(A) = {0} and thus the system (A, B) is NCVE and Liouville’s theorem holds. In the case of Kolmogorov’s operator one easily checks that (A, B) is controllable for arbitrary a ∈ R1 and √ √ σ(A) = { a, − a} if a > 0, √ √ σ(A) = {−i a, i a} if a ≤ 0. Consequently bounded functions h for which Lh = 0, where L is the Kolmogorov operator (3.13) are constant if and only if a ≤ 0.

Bibliographical notes The concept of controllability with vanishing energy as well as main theorems are due to E. Priola and the author [80]. They were obtained in greater generality, for infinite-dimensional systems. Generalization to boundary systems is the subject of the paper by L. Pandolfi, E. Priola, and J. Zabczyk [74]. The orbit transfer problem is introduced and solved by M. Shibata, A. Ichikawa in [90]. The two figures in §3.2 are based on Fig. 3.1 and Fig. 3.2 in M. Shibata, A. Ichikawa [90] and were prepared by Johannes Utri. More on the surprising connection between NCVE systems and Liouville’s theorem on bounded harmonic functions, see E. Priola and J. Zabczyk [80, 81].

Chapter 4

Systems with constraints

This chapter illustrates complications which arise in control theory when control parameters have to satisfy various constraints. Controllable systems with the set of controls being either a bounded neighbourhood of 0 ∈ Rm or the positive cone Rm + are characterized.

§4.1. Bounded sets of parameters In the preceding chapters sets of states and of control parameters were identical with Rn and Rm . In several situations, however, it is necessary to consider controls taking values in subsets of Rm or to require that the system evolves in a given subset of Rn . We will discuss two types of constraints for linear systems. y˙ = Ay + Bu, y(0) = x. (4.1) We start from the null controllability. Proposition 4.1. Assume that 0 ∈ Rm is an interior point of a control set U ⊂ Rm and (A, B) is a controllable pair. Then there exists a neighbourhood V of 0 ∈ Rn such that all its elements can be transferred to 0 by controls taking values in U . Proof. Let T > 0 be a given number. By Proposition 1.1 the control u ˆ(s) = −B ∗ S ∗ (T − s)Q−1 T S(T )x,

s ∈ [0, T ],

transfers the state x to 0 at the time T . Since the function t −→ S ∗ (t) is continuous, therefore, for a constant M > 0, |ˆ u(s)| ≤ M |x| for all s ∈ [0, T ],

x ∈ Rn .

So the result follows.

2

© Springer Nature Switzerland AG 2020

55

J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_4

56

4 Systems with constraints

A possibility of transferring to 0 all points from Rn is discussed in the following theorem. Theorem 4.1. Assume that U is a bounded set. (i) If 0 ∈ Rm is an interior point of U , (A, B) is a controllable pair and the matrix A is stable, then arbitrary x ∈ Rn can be transferred to 0 by controls with values in U only. (ii) If the matrix A has at least one eigenvalue with a positive real part then there are states in Rn which cannot be transferred to 0 by controls with values in U only. Proof. (i) If the matrix A is stable then the control u(t) = 0, for all t ≥ 0, transfers an arbitrary state, in finite time, to a given in advance neighbourhood U of 0 ∈ Rn . It follows from Proposition 4.1 that if the neighbourhood V is sufficiently small then all its elements can be transferred to 0 by controls with values in U . Hence (i) holds. (ii) Let λ = α + iβ be an eigenvalue of A∗ such that α > 0. Assume that β = 0. Then there exist vectors e1 , e2 ∈ Rn , |e1 | + |e2 | = 0, such that A∗ e1 + iA∗ e2 = (α + iβ)(e1 + ie2 ) or equivalently A∗ e1 = αe1 − βe2 ,

A∗ e2 = βe1 + αe2 .

Let u(·) be a control taking values in U and y(·) the corresponding output: y˙ = Ay + Bu,

y(0) = x.

Let v be the absolutely continuous function given by the formula v(t) = y(t), e1 2 + y(t), e2 2 ,

t ≥ 0.

Then, for almost all t ≥ 0, v(t) ˙ = 2(y(t), ˙ e1 y(t), e1 + y(t), ˙ e2 y(t), e2 ) ∗ = 2(y(t), A e1 y(t), e1 + y(t), A∗ e2 y(t), e2 ) + 2(Bu(t), e1 y(t), e1 + Bu(t), e2 y(t), e2 ) = I1 (t) + I2 (t). From the definition of e1 and e2 , I1 (t) = 2αv(t). The set U is bounded, therefore, for a constant K > 0, independent of u(·) and x,  |I2 (t)| ≤ K v(t). Consequently, for almost all t ≥ 0, v(t) ˙ ≥ Ψ(v(t)),

4.2 Positive systems

57

 K 2 √ where Ψ(ξ) = 2αξ − K ξ, ξ ≥ 0. We have Ψ(ξ) > 0 for ξ > ξ0 = 2α therefore if v(0) = x, e1 2 + x, e2 2 > ξ0 the function v is increasing and v(t) → 0 as t ↑ +∞. Hence (ii) holds if β = 0. The proof of the result if β = 0 is similar. 2 Exercise 4.1. System d(n−1) d(n) z + a1 (n−1) z + . . . + an z = u, (n) dt dt studied in §1.4, is controllable if U = R. It follows from Theorem 4.1 that it is controllable to 0 also when U = [−1, 1] and the polynomial p(λ) = λn + a1 λn−1 + . . . + an , λ ∈ C, is stable. In particular, the system d2 z dz + z = u, + dt2 dt

|u| ≤ 1,

is controllable to 0 ∈ R2 . Note however that the system d2 z = u, dt2

|u| ≤ 1,

is also controllable to 0 ∈ R2 , see Example III. 12.1, although the corresponding polynomial p is not stable.

§4.2. Positive systems Denote by E+ and U+ the sets of all vectors with nonnegative coordinates from Rn and Rm , respectively. System (4.1) is said to be positive if for an arbitrary, locally integrable control u(·), taking values in U+ and for an arbitrary initial condition x ∈ E+ , the output y u,x takes values in E+ . Positive systems are of practical interest and are often seen in applications to heat processes. In particular, see the system modelling the electrically heated oven of Example 0.1. We have the following: Theorem 4.2. System (4.1) is positive if and only if all elements of the matrix B and all elements of the matrix A outside of the main diagonal are nonnegative. Proof. Assume that system (4.1) is positive. Then for arbitrary vector u ¯ ∈ U+ and t ≥ 0 1 t

 0

t

S(s)B u ¯ ds ∈ E+ ,

where

S(s) = eAs , s ≥ 0.

(4.2)

58

4 Systems with constraints

Letting t tend to 0 in (4.2) to the limit as t ↓ 0 we get B u ¯ ∈ E+ . Since u ¯ was an arbitrary element of U+ , all elements of B have to be nonnegative. Positivity of (4.1) also implies that for arbitrary x ¯ ∈ E+ and t ≥ 0, S(t)¯ x ∈ E+ . Consequently all elements of the matrices S(t), t ≥ 0, are nonnegative. Assume that for some i = j the element aij of A is negative. Then, for sufficiently small t ≥ 0, the element in the i-th row and j-th column of the matrix     +∞ 1 Ak k−1 1 S(t) = I +A +t t t t k! k=2

is also negative. This is a contradiction with the fact that all elements of S(t), t > 0, are nonnegative. Assume conversely that all elements of the matrix B and all elements of the matrix A outside of the main diagonal are nonnegative. Let x ∈ E+ and let u(·) be a locally integrable function with values in U+ . There exists a number λ > 0 such that the matrix λI + A has only nonnegative elements and therefore also the matrix e(λI+A)t has all nonnegative elements. But e(λI+A)t = eλt S(t),

t ≥ 0,

and we see that all elements of S(t), t ≥ 0, are nonnegative as well. Since  y x,u (t) = S(t)x +

0

t

S(t − s)Bu(s) ds,

t ≥ 0,

vectors y x,u (t), t ≥ 0, belong to E+ . This finishes the proof.

2

To proceed further we introduce the following sets of attainable points: Ot+ = {x ∈ E+ ; x = y 0,u (s) for some s ∈ [0, t] and u ∈ L1 [0, s; U+ ]}, t ≥ 0 Ot+ . O+ = t≥0

System (4.1) is called positively controllable at time t > 0 if the set Ot+ is dense in E + . System (4.1) is called positively controllable if the set O+ is dense in E + . Theorem 4.3. Let e1 , . . . , en and e˜1 , . . . , e˜m be standard bases in Rn and Rm , respectively. (i) A positive system (4.1) is positively controllable at time t > 0 if and only if for arbitrary i = 1, 2, . . . , n there exists j = 1, 2, . . . , m and a constant μ > 0 such that ej . ei = μB˜ (ii) A positive system (4.1) is positively controllable if and only if for arbitrary i = 1, 2, . . . , n there exists j = 1, 2, . . . , m such that either

4.2 Positive systems

59

ei = μB˜ ej or ei = lim

k↑+∞

for some μ > 0

S(tk )B˜ ej |S(tk )B˜ ej |

for some tk ↑ +∞.

We will need several lemmas. Lemma 4.1. A positive system (4.1) is positively controllable, respectively, positively controllable at time t > 0 if the cone generated by S(s)B˜ ej ,

s ≥ 0, j = 1, 2, . . . , m,

respectively, generated by s ∈ [0, t], j = 1, 2, . . . , m,

S(s)B˜ ej , are dense in E+ .

Proof of the lemma. Fix s > 0. Then S(s)B˜ ej = lim y 0,uδ (s), δ↓0



where uδ (r) =

δB˜ ej for r ∈ [s − δ, s], 0 for r ∈ [0, s − δ].

On the other hand, piecewise constant functions are dense in L1 [0, s; U+ ], and vectors y 0,u (s), where u(·) ∈ L1 [0, s; U+ ], are limits of finite sums of the form  δ  S(kδ) S(r)Buδk dr, 0

k

where uδk ∈ U+ , for δ > 0, k = 1, 2, . . . . Consequently, y 0,u (s) is a limit of finite sums m   γδk S(kδ)B˜ ej , j=1 k

with nonnegative numbers γδk . Hence the conclusion of the lemma follows.2 For arbitrary x ∈ E+ define s(x) = {z ∈ E+ ; x, z = 0}. The set s(x) will be called the side of U+ determined by x. Lemma 4.2. Let K be a compact subset of E+ and let x ∈ E+ , x = 0. If s(x) contains a nonzero element of the convex cone C(K), spanned by K, then s(x) contains at least one element of K.

60

4 Systems with constraints

Proof of the lemma. We can assume that 0 ∈ / K. Suppose that z ∈ (C(K)) ∩ (s(x)), Then lim l

k(l)  j=1

z = 0.

αjl xjl = z for some αjl > 0, xjl ∈ K, j = 1, . . . , k(l), l =

1, 2, . . . . Setting yjl = xjl |xjl |−1 ,

βjl = αjl |xjl | > 0, we obtain 0 = x, z = lim l

k(l) 

αjl x, xjl = lim l

j=1

k(l) 

βjl x, yjl .

j=1

Moreover k(l)  j=1

βjl =

k(l) 

k(l)  βjl |yjl | ≥ βjl yjl −→ |z|,

j=1

l ↑ +∞,

j=1

and we can assume that for l = 1, 2, . . . k(l)



 |f | , βjl x, yjl < 2l j=1

k(l) 

βjl ≥

j=1

|f | . 2

Therefore for arbitrary l = 1, 2, . . . there is j(l) such that x, yj(l),l < 1/l. Since the set K is compact, there exists a subsequence (xj(l),l ) convergent to x0 ∈ K. Consequently there exists a subsequence (yl ) of the sequence (yj(l),l ) convergent to x0 |x0 |−1 . But x, x0 |x0 |−1 limx, yl = 0, l

so finally x0 ∈ s(x) as required.

2

Proof of the theorem. Sufficiency follows from Lemma 4.1. To show necessity we can assume, without any loss of generality, that B˜ ej = 0 for j = 1, . . . , m. (i) The set {S(s)B˜ ej ; s ∈ [0, t], j = 1, . . . , m} does not contain 0 and ej ; s ∈ [0, t], j = 1, . . . , m} is is compact, therefore the cone C1 = C{S(s)B˜ closed. So if system (4.1) is positively controllable at time t > 0 then the cone C1 must be equal to E+ . Taking into account Lemma 4.2 we see in particular that for arbitrary i = 1, 2, . . . , n there exist s ∈ [0, t], j = 1, 2, . . . , m and ej . μ > 0 such that ei = μS(s)B˜ Let λ > 0 be a number such that the matrix λI + A has all elements nonnegative. Then

4.2 Positive systems

61

eλs S(s)B˜ ej = e(λI+A)s B˜ ej = B˜ ej +

+∞  1 (λI + A)k B˜ ej ≥ B˜ ej . k!

k=1

Consequently for a positive ν > 0 S(s)B˜ ej = νB˜ ej , or equivalently ei = μνB˜ ej . (ii) Assume that the system (4.1) is positive by controllable and denote by K the closure of the set {S(s)Bej /|S(s)Bej |; s ≥ 0, j = 1, 2, . . . }. It follows from the positive controllability of the system that C(K) = E+ . Therefore, by Lemma 4.2, for arbitrary i = 1, 2, . . . , n there exists j = 1, 2, . . . , m such that either or

ei = μS(s)B˜ ej for some μ > 0, s ≥ 0, ei = lim k

S(tk )B˜ ej |S(tk )Bej |

for some

tk ↑ +∞.

Repeating now the arguments from the proof of (i) we deduce that the conditions of the theorem are necessary. 2 We propose now to solve the following exercise. 

 −α β Exercise 4.2. Assume that n = 2, β, γ ≥ 0 and that A = , see γ −δ Example 0.1. Let S(t) = eAt , t ≥ 0. Prove that for arbitrary x ∈ E+ , x = 0, there exists S(t)x =e lim t↑+∞ |S(t)x| and e is an eigenvector of A. Hint. Show that the eigenvalues λ1 , λ2 of A are real. Consider separately the cases λ1 = λ2 and λ1 = λ2 . Example 4.1. Continuation of Exercise 4.2. Let     −α β 1 A= , B= , β, γ ≥ 0. γ −δ 0

(4.3)

It follows from Theorem 4.3 that system (4.1) is not positively controllable at any t > 0. By the same theorem and Exercise 4.2, the system is positively controllable if and only if

62

4 Systems with constraints

  1   S(t) 0 1   = . lim 0 t↑+∞ S(t) 1 0

(4.4)

  1 In particular, see Exercise 4.2, the vector must be an eigenvector of A 0 and so β = 0. Consequently  ⎧ ⎪ e−αt 0 ⎪ ⎪ if α = δ, ⎪ ⎨ γte−αt e−δt ,  S(t) =  ⎪ ⎪ −e−αt 0 ⎪ ⎪ , if α = δ. ⎩ (γe−αt − e−δt )(δ − α)−1 e−δt We see that (4.4) holds if γ > 0, β = 0 and δ ≤ α. These conditions are necessary and sufficient for positive controllability of (4.3). Let us remark that the pair (A, B) is controllable under much the weaker condition γ = 0. The above example shows that positive controllability is a rather rare property. In particular, the system from Example 0.1 is not positively controllable. However in situations of practical interest one does not need to reach arbitrary elements from E+ but only states in which the system can rest arbitrarily long. This way we are led to the concept of a positive stationary pair . We say that (¯ x, u ¯) ∈ E+ ×U+ is a positive stationary pair for (4.1) if A¯ x + Bu ¯ = 0.

(4.5)

It follows from the considerations below that a transfer to a state x ¯ for which (4.5) holds with some u ¯ ∈ U+ can be achieved for a rather general class of positive systems. Theorem 4.4. Assume that system (4.1) is positive and the matrix A is stable. Then ¯ = 0 there exists exactly one vector (i) for arbitrary u ¯ ∈ U+ such that B u x ¯ ∈ E+ such that A¯ x + Bu ¯ = 0; (ii) if (¯ x, u ¯) is a positive stationary pair for (4.1) and u ˜(t) = u ¯ for t ≥ 0 then, for arbitrary x ∈ E+ y x,˜u (t) −→ x ¯,

t ↑ +∞.

Proof. (i) If A is a stable matrix then for some ω > 0 and M > 0 |eAt | ≤ M e−ωt ,

t ≥ 0.

4.2 Positive systems

63

Therefore the integral elements. Since

 +∞ 0

eAt dt is a well-defined matrix with nonnegative  +∞ −1 eAt dt, −A = 0

−1

the matrix −A

has nonnegative elements. Consequently the pair (¯ x, u ¯) = (−A−1 B u ¯, u ¯)

is the only positive stationary pair corresponding to u ¯. (ii) If u ˜(t) = u ¯ for t ≥ 0 and y˙ = Ay + B u ¯ = Ay − A¯ x,

y(0) = x,

d then dt (y(t) − x ¯) = A(y(t) − x ¯) and y(t) − x ¯ = S(t)(x − x ¯), t ≥ 0. Since the matrix A is stable we finally have S(t)(x − x ¯) −→ 0 as t ↑ +∞ and y(t) −→ x ¯ as t ↑ +∞. 2

Conversely, the existence of positive stationary pairs implies, under rather weak assumptions, the stability of A. Theorem 4.5. Assume that for a positive system (4.1) there exists a stationary pair (¯ x, u ¯) such that vectors x ¯ and B u ¯ have all coordinates positive. Then matrix A is stable. Proof. It is enough to show that S(t)¯ x −→ 0 as t ↑ +∞. Since  t+r  t+r d S(s)¯ x ds = S(t)¯ x+ S(s)A¯ x ds S(t + r)¯ x = S(t)¯ x+ ds r r  t+r S(s)B u ¯ ds ≤ S(t)¯ x, t, r ≥ 0, = S(t)¯ x− r

the coordinates of the function t −→ S(t)¯ x are nondecreasing. Consequently lim S(t)¯ x = z exists and S(t)z = z for all t ≥ 0. On the other hand, for t↑+∞ t ¯ dr has all coordinates positive. Since arbitrary t > 0, the vector 0 S(r)B u  t x ¯−z ≥x ¯ − S(t)¯ x= S(r)B u ¯ dr, 0

for a number μ > 0, Finally

x ¯ ≤ μ(¯ x − z). 0 ≤ S(t)¯ x ≤ μ(S(t)¯ x − z) −→ 0,

and S(t)¯ x −→ 0 as t ↑ +∞. This finishes the proof of the theorem.

2

The question of attainability of a positive stationary pair is answered by the following basic result:

64

4 Systems with constraints

Theorem 4.6. Assume that (¯ x, u ¯) is a positive stationary pair for (4.1) such that all coordinates of u ¯ are positive, and let (A, B) be a controllable pair. ¯ there For an arbitrary x ∈ E+ and an arbitrary neighbourhood V ⊂ U+ of u exists a bounded control u ˜(·) with values in V and a number t0 > 0 such that y x,˜u (s) = x ¯

for

s > t0 .

Proof. Let δ > 0 be a number such that {u; |¯ u − u| < δ} ⊂ V . It follows from formula (1.12) that for arbitrary t1 > 0 there exists γ > 0 and that for arbitrary b ∈ Rn , |b| < γ, there exists a control v b (·), |v b (t)| < δ, t ∈ [0, t1 ], ¯| < γ and let x − x ¯ = b, transferring b to 0 at time t1 . Assume first that |x − x ¯ and y(t) = y x,˜u (t), t ∈ [0, t1 ]. Then u ˜(t) = v b (t) + u d d (y(t) − x ¯) = y(t) = Ay(t) + Bv b (t) + B u ¯ dt dt = A(y(t) − x ¯) + Bv b (t), t ∈ [0, t1 ], and y(0) − x ¯ = x−x ¯. Hence y(t1 ) − x ¯ = 0 or y(t1 ) = x ¯. It is also obvious ˜(t) = u ¯ for t > t1 we get the required that u ˜(t) ∈ V for t ∈ [0, t1 ]. Setting u control. If x is an arbitrary element of E+ then, by Theorem 4.5, there exists t2 > 0 ˜˜(t) = u such that the constant control u ¯, t ∈ [0, t2 ], transfers x into the ball {z; |¯ x − z| < γ} at the time t2 . By the first part of the proof all points from the ball can be transferred to x ¯ in the required way. 2 Example 4.1 (continuation). Let α, β, γ, δ > 0. The matrix A is stable if and only if αδ − βγ > 0. A positive stationary pair is proportional to      1 δ (¯ x, u ¯) = ,1 . γ αδ − βγ     δ 0 So if αδ > βγ then the state is attainable from using only nonnegγ 0 ative controls.

Bibliographical notes Results on positive systems are borrowed from the paper [89] by T. Schanbacher. In particular, he is the author of Theorem 4.3 and its generalizations to infinite-dimensional systems.

Chapter 5

Realization theory

This chapter is devoted to the input–output map generated by a linear control system. The input–output map is characterized in terms of the impulse response function and the transfer function.

§5.1. Impulse response and transfer functions The connection between a control u(·) and the observation w(·) of the system (1.1)–(1.2) is given by  w(t) = CS(t)x +

0

t

CS(t − s)Bu(s) ds

= CS(t)x + Ru(t),

(5.1)

t ≥ 0.

The operator R defined above is called the input–output map of (1.1)–(1.2). Thus  t Ru(t) = Ψ(t − s)u(s) ds, (5.2) 0

where

Ψ(t) = CS(t)B = CeAt B,

t ≥ 0.

(5.3)

The function Ψ is called the impulse response function of the system (1.1)– (1.2). Obviously there exists a one-to-one correspondence between input– output mappings and impulse response functions. However, different triplets (A, B, C) may define the same impulse response functions. Each of them is called a realization of Ψ or, equivalently, a realization of R. The realization theory of linear systems is concerned with the structural properties of input–output mappings R or impulse response functions Ψ as well as with constructions of their realizations. The theory has practical motivations which we will discuss briefly. © Springer Nature Switzerland AG 2020

J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_5

65

66

5 Realization theory

In various applications, matrices A, B and C defining system (1.1)–(1.2) can be obtained from a consideration of physical laws. It happens, however, that the actual system is too complicated to get its description in such a way. Then the system is tested by applying special inputs and observing the corresponding outputs, and the matrices A, B, C are chosen to match the experimental results in the best way. Observed outputs corresponding to the impulse inputs lead directly to the impulse response function defined above. This follows from Proposition 5.1. Let v be a fixed element of Rm , uk (·), k = 1, 2, . . . , controls of the form  kv for t ∈ [0, k1 ), uk (t) = 0 for t ≥ k1 , and yk (·) the corresponding solutions of the equations d yk = Ayk + Buk , dt

yk (0) = x.

Then the sequence (yk ) converges almost uniformly on (0, +∞) to the function y(t) = S(t)x + S(t)Bv,

t ≥ 0.

Proof. Let us remark that  t S(t)x + k 0 S(r)Bv dr  1/k yk (t) = S(t)x + S(t − k1 )k 0 S(r)Bv dr

(5.4)

for t ∈ [0, k1 ], for t ≥ k1 .

Therefore, for t ≥ k1 ,    1/k  1    1  k Bv . |yk (t) − y(t)| = S t − S(r)Bv dr − S k k 0 Moreover    k

0

1/k

S(r)Bv dr − S

1 k

  Bv  −→ 0,

as k ↑ +∞, 2

and the proposition easily follows. The function w(t) = CS(t)x + CS(t)Bv,

t ≥ 0,

can be regarded as an output corresponding to the impulse input u(·) = vδ{0} (·), where δ{0} (·) is the Dirac distribution. If x = 0 then w(t) = Ψ(t)v, t ≥ 0, and this is the required relation between the observation w(·) and the impulse response function Ψ.

5.1 Impulse response and transfer functions

67

It is convenient also to introduce the transfer function of (1.1)–(1.2) defined by (5.5) Φ(λ) = C(λI − A)−1 B, λ ∈ C \ σ(A). Let us notice that if Re λ > sup{Re μ; μ ∈ σ(A)}, then Φ(λ) = C



+∞

0

 e−λt S(t) dt B =

+∞

0

e−λt Ψ(t) dt.

Hence the transfer function is the Laplace transform of the impulse response function, and they are in a one-to-one correspondence. The transfer function can be constructed from the observations of outputs resulting from periodic, called also harmonic, inputs u(·): u(t) = eitω v, v ∈ Rm , ω ∈ R, t ≥ 0.

(5.6)

For them w(t) = CS(t)x + eiωt



t

e−iωr CS(r)Bv dr,

0

t ≥ 0.

(5.7)

Practical methods are based on the following result. Proposition 5.2. Assume that A ∈ M(n, n) is a stable matrix. Let u : R −→ Rm be a bounded, periodic function with a period γ and y x (t), t ≥ 0, the solution of the equation y(t) ˙ = Ay(t) + Bu(t),

t ≥ 0, y(0) = x.

There exists exactly one periodic function yˆ : R −→ Rn such that |y x (t) − yˆ(t)| −→ 0,

as t ↑ +∞.

Moreover, the period of yˆ is also γ and  +∞ yˆ(t) = S(r)Bu(t − r) dr, 0

t ∈ R.

(5.8)

The function yˆ(·) is called sometimes a periodic component of the solution y. Proof. The function yˆ(·) given by (5.8) is well defined, periodic with period γ. We have

68

5 Realization theory

y x (t) = S(t)x +





t

t

S(t − s)Bu(s) ds = S(t)x + S(r)Bu(t − r) dr 0 0  +∞  +∞ = S(t)x + S(r)Bu(t − r) dr − S(r)Bu(t − r) dr 0 t  +∞ S(r)Bu(−r) dr = S(t)x + yˆ(t) − S(t) 0

= S(t)x + yˆ(t) − S(t)ˆ y (0),

t ≥ 0,

therefore |y x (t) − yˆ(t)| ≤ |S(t)||x − yˆ(0)| −→ 0,

as t ↑ +∞.

If yˆ1 is an arbitrary function with the properties specified in the proposition, then y (t) − y x (t)| + |y x (t) − yˆ1 (t)|, t ≥ 0. |ˆ y (t) − yˆ1 (t)| ≤ |ˆ Consequently |ˆ y (·) − yˆ1 (·)| −→ 0 as t ↑ +∞. By the periodicity of yˆ, 2 yˆ(t) − yˆ1 (t) = 0 for all t ≥ 0. It follows from Proposition 5.2 that the observed outputs of a stable system, corresponding to harmonic controls eiωt v, t ≥ 0, are approximately equal, with an error tending to zero, to  +∞ w(t) ˆ =C S(r)eiω(t−r) dr Bv 0  +∞ e−iωr S(r) dr Bv, t ≥ 0. = eiωt C 0

Taking into account that  +∞ e−iωr S(r) dr = (iωI − A)−1 , 0

ω ∈ R,

we see that the periodic component of the output w(·) corresponding to the periodic input (5.6) is of the form w(t) ˆ = eiωt Φ(iω)v,

t ≥ 0.

(5.9)

Assume that m = k = 1. Then the transfer function is meromorphic. Denote it by ϕ and consider for arbitrary ω ∈ R the trigonometric representation ϕ(iω) = |ϕ(iω)|eiδ(ω) , δ(ω) ∈ [0, 2π). If v = 1 then the output w(·) ˆ given by (5.9) is exactly w(t) ˆ = |ϕ(iω))|ei(δ(ω)+ωt) ,

t ≥ 0.

5.2 Realizations of the impulse response function

69

Thus, if |ϕ(iω)| is large then system (1.1)–(1.2) amplifies the periodic oscillation of the period ω and, if |ϕ(iω)| is small, filters them off. Exercise 5.1. Find the transfer function for system (2.12)–(2.13). Answer. ϕ(λ) = p−1 (λ), when p(λ) = λn + a1 λn−1 + . . . + an = 0. Exercise 5.2. Consider the equation of an electrical filter, Example 0.4, L2 C

d(3) z d2 z dz + RLC + 2L + Rz = u, dt2 dt dt(3)

where C, L, R are positive numbers. Find ϕ(iω), ω ∈ R. Show that lim |ϕ(iω)| = 1,

lim |ϕ(iω)|

ω↓0

ω↑+∞

CL2 ω 3 = 1. R

§5.2. Realizations of the impulse response function Not an arbitrary smooth function Ψ(·) with values in M(k, m), defined on [0, +∞), is the impulse response function of a linear system (1.1)–(1.2). Similarly, a not arbitrary operator R of the convolution type (5.2) is an input– output map of a linear system. In this paragraph we give two structural descriptions of the impulse response function and we show that among all triplets (A, B, C) defining the same function Ψ there exists, in a sense, a “best” one. Moreover, from Theorem 5.1 below, the following result will be obtained: An operator R of the convolution type (5.2) is an input–output map of a linear system (1.1)–(1.2) if and only if it is of the form  Ru(t) = H(t)

t

G(s)u(s) ds, 0

t ≥ 0,

(5.10)

where H(·), G(·) are functions with values in M(k, n), M(n, m), for some n. Let us remark that operator (5.10) is the product of an integration and a multiplication operator. Before formulating the next theorem we propose first to solve the following exercise. Exercise 5.3. Continuous, real functions g, h and r defined on [0, +∞) satisfy the functional equation g(t − s) = h(t)r(s),

t ≥ s ≥ 0,

if and only if either g and one of the functions h or r are identically zero, or if there exist numbers α, β = 0, γ = 0 such that

70

5 Realization theory

g(t) = γeαt ,

h(t) = βeαt ,

r(t) =

γ −αt e , β

t ≥ 0.

Theorem 5.1. A function Ψ of class C 1 defined on [0, +∞) and with values in M(k, m) is the impulse response function of a system (A, B, C) if and only if there exist a natural number n and functions G : [0, +∞) −→ M(m, n), H : [0, +∞) −→ M(n, k) of class C 1 such that Ψ(t − s) = H(t)G(s)

for t ≥ s ≥ 0.

(5.11)

Proof. If Ψ(t) = CeAt B, t ≥ 0, then it is sufficient to define H(t) = CeAt ,

t ≥ 0,

and

G(s) = e−As B,

s ≥ 0.

The converse implication requires a construction of a number n and matrices A, B, C in terms of the functions G, H. We show first the validity of the following lemma: Lemma 5.1. Assume that for some functions Ψ, G, H of class C 1 , with values, respectively, in M(k, m), M(n, m), M(k, n), relation (5.11) holds. Then ˜ ˜ for arbitrary T > 0 there exist a natural number n ˜ and functions G(t), H(t), 1 n, m) and M(k, n ˜ ) such that t ≥ 0 of class C with values in M(˜ ˜ G(s), ˜ Ψ(t − s) = H(t) and the matrix



T

s ≤ t, 0 ≤ s ≤ T,

˜ G ˜ ∗ (s) ds G(s)

0

is nonsingular and thus positive definite. Proof of the lemma. Let n ˜ be the rank of the matrix  W =

0

T

G(s)G∗ (s) ds.

If W is nonsingular then we define n ˜ = n, and the lemma is true. Assume therefore that n ˜ < n. Then there exists an orthogonal matrix P such that the diagonal of the matrix P W P −1 = P W P ∗ is composed of eigenvalues of the matrix W . Therefore there exists a diagonal nonsingular matrix D such that

 I˜ 0 V = = (DP )W (DP )∗ = P˜ W P˜ ∗ , 0 0 where I˜ ∈ M(˜ n, n ˜ ) is the identity matrix and P˜ is a nonsingular one. Consequently, for Q = (DP )−1 , QV Q∗ = W. This and the definition of W imply that

5.2 Realizations of the impulse response function



T

0

71

(QV Q−1 G(s) − G(s))(QV Q−1 G(s) − G(s))∗ ds = 0.

Therefore QV Q−1 G(s) = G(s), for s ∈ [0, T ]. ˜ = Q−1 G(t), H(t) ˜ Define G(t) − H(t)Q, t ≥ 0. Then ˜ ˜ Ψ(t − s) = H(t)V G(s)

for t ≥ s and s ∈ [0, T ].

It follows from the representation ˜ ˜ 1 (t), H ˜ 2 (t)], H(t) = [H

 ˜ 1 (s) G ˜ G(s) = ˜ , G2 (s)

˜ 1 (t) ∈ M(k, n ˜ 2 (t) ∈ M(k, n − n ˜ 2 (s) ∈ where H ˜ ), H ˜ ), G1 (s) ∈ M(˜ n, m), G 2 M(n − n ˜ , m), t, s ≥ 0, and from the identity V = V that

 ˜ 1 (s) G ˜ ˜ ˜ Ψ(t − s) = (H(t)V )(V G(s)) = [H1 (t), 0] 0 ˜ 1 (s), t ≥ s, s ∈ [0, T ]. ˜ 1 (t)G =H Hence

 V =

0

T

(Q−1 G(s))(Q−1 G(s))∗ ds =

and thus I˜ =

 0

T



T

˜ G(s) ˜ ∗ ds, G(s)

0

˜ 1 (s)G ˜ ∗ (s) ds G 1

is a nonsingular matrix of the dimension n ˜ . The proof of the lemma is complete. 2 Continuation of the proof of Theorem 5.1 We can assume that functions H and G have the properties specified in the lemma. For arbitrary t ≥ s, 0≤s≤T 0=

∂Ψ ∂Ψ ˙ ˙ (t − s) + (t − s) = H(t)G(s) + H(t)G(s). ∂t ∂s

In particular ∗ ∗ ˙ ˙ H(t)G(s)G (s) = −H(t)G(s)G (S),

and thus for t ≥ T we obtain  T  ∗ ˙ H(t) G(s)G (s) ds = −H(t) 0

Define

t ≥ s, s ∈ [0, T ],

T 0

∗ ˙ G(s)G (s) ds.

72

5 Realization theory

 A=−

T

0

∗ ˙ G(s)G (s) ds



T

0

G(s)G∗ (s) ds

−1

.

˙ Then H(t) = H(t)A, t ≥ T , and therefore H(t) = H(T )S(t − T ), t ≥ T , where S(r) = erA , r ≥ 0. However, for arbitrary r ≥ 0, Ψ(r) = Ψ((r + T ) − T ) = H(r + T )G(T ) = H(T )S(r)G(T ). Consequently, defining B = G(T ) and C = H(T ), we obtain Ψ(r) = BerA C,

r ≥ 0, 2

the required representation.

We now give a description of impulse response functions in terms of their Taylor expansions. Theorem 5.2. A function Ψ : [0, +∞) −→ M(k, m) given in the form of an absolutely convergent series Ψ(t) =

+∞ j=0

Wj

tj , j!

t ≥ 0,

is the impulse response function of a certain system (A, B, C) if and only if there exist numbers a1 , a2 , . . . , ar such that Wj+r = a1 Wj+r−1 + . . . + ar W j,

j = 0, 1, . . . .

(5.12)

Proof. (Necessity) If Ψ(t) = C(exp tA)B, t ≥ 0, then Ψ(t) =

+∞ j=0

CAj B

tj , j!

t ≥ 0,

and therefore Wj = CAj B, j = 0, 1, . . . . If p(λ) = λr − a1 λr−1 − . . . − a1 , λ ∈ C, is the characteristic polynomial of A then, by the Cayley–Hamilton theorem, Ar = a1 Ar−1 + . . . + ar I and

(5.13)

Ar+j = a1 Ar+j−1 + . . . + ar Aj .

Therefore CAr+j B = a1 CAr+j−1 B + . . . + ar CAj B, the required relationship.

j = 0, . . . ,

5.2 Realizations of the impulse response function

(Sufficiency) Define ⎡ 0 I ... ⎢ 0 0 . .. ⎢ ⎢ .. . . . .. A=⎢ . . ⎢ ⎣ 0 0 ... ar I ar−1 I . . .

0 0 .. .

0 0 .. .

⎤

⎥ ⎥ ⎥ ⎥, ⎥ 0 I ⎦ a2 I a1 I

73

⎡

⎤ W0 ⎢ ⎥ B = ⎣ ... ⎦ , Wr−1

C = [I, 0, . . . , 0],

where I is the identity matrix of dimension k. Permuting rows and columns of the matrix A properly we can assume that it is of the form ⎡ ⎤ ⎡ ˜ ⎤ A 0 ... 0 0 0 1 ... 0 0 ⎢ 0 0 ... 0 0 ⎥ ⎢ 0 A˜ . . . 0 0 ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ .. .. . . .. .. ⎥ .. .. . . .. .. ⎥ . ˜=⎢ where A (5.14) ⎢ ⎥ ⎢ . . . . . . ⎥ . . ⎥ ⎢ . . ⎢ ⎥ ⎣ 0 0 ... 0 1 ⎦ ⎣ 0 0 . . . A˜ 0 ⎦ a1 a2 . . . ar−1 ar 0 0 . . . 0 A˜ The characteristic polynomial of A˜ is λr − a1 λr−1 − . . . − ar , λ ∈ C, therefore A˜r = a1 A˜r−1 + . . . + ar I. The same equation holds for matrix (5.14) and consequently for A. We also check that CAj B = Wj

for

j = 0, 1, . . . , r − 1.

By (5.12), CAr B = a1 CAr−1 B + . . . + ar CB = a1 Wr−1 + . . . + ar Wr = Wr . Similarly, CAj B = Wj for j ≥ r.

2

The next theorem shows that there exists a realization of an impulse response function with additional regularity properties. Theorem 5.3. For arbitrary matrices A ∈ M(n, n), B ∈ M(n, m), C ∈ ˜ ∈ M(k, n) there exist a natural number n ˜ and matrices A˜ ∈ (˜ n, n ˜ ), B ˜ B) ˜ and (A, ˜ C) ˜ are, respecM(˜ n, m), C˜ ∈ M(k, n ˜ ) such that the pairs (A, tively, controllable and observable and ˜ B ˜ CS(t)B = C˜ S(t)

for t ≥ 0,

˜ = exp tA, ˜ t ≥ 0. where S(t) = exp tA, S(t) Proof. Assume that the pair (A, B) is not controllable and let l = rank[A|B]. Let P, A11 , A12 , A22 and B1 be matrices given by Theorem 1.5 and let S(t) = exp tA, S11 (t) = exp(A11 t), S22 (t) = exp(A22 t), t ≥ 0. There exists, in addition, a function S12 (t), t ≥ 0, with values in M(l, n − l), such that

74

5 Realization theory

P S(t)P −1 = Therefore,





S11 (t) S12 (t) , 0 S22 (t)

t ≥ 0.

 S11 (t) S12 (t) CS(t)B = CP PB 0 S22 (t) 



B1 S11 S12 (t) = CP −1 0 0 S22 (t) 

S11 (t)B1 = CP −1 = C1 S11 (t)B1 , 0 −1



t ≥ 0,

for some matrix C1 . So we can assume, without any loss of generality, that the pair (A, B) is controllable. If the pair (A, C) is not observable and rank[A|C] = l < n then, by Theorem 1.7, for a nonsingular matrix P , an observable pair (A11 , C1 ) ∈ M(l, l) × M(k, l) and matrices B1 ∈ M(l, n), B2 ∈ M(n − l, m),



  A11 0 B1 P AP −1 = , CP −1 = [C1 , 0], P B = . B2 A21 A22 It follows from the controllability of (A, B) that the pair (A11 , B1 ) is controllable as well. If S(t) = exp tA, S11 (t) = exp tA11 , S22 (t) = exp tA22 , t ≥ 0, then for an M(n − l, l)-valued function S21 (t), t ≥ 0, 

 B1 S11 (t) 0 −1 CS(t)B = [C1 , 0]P S(t)P P B = [C1 , 0] B2 S21 (t) S22 (t) = C1 S11 (t)B1 ,

t ≥ 0. 2

This finishes the proof of the theorem. The following basic result is now an easy consequence of Theorem 5.3.

Theorem 5.4. For any arbitrary impulse response function there exists a realization (A, B, C) such that the pairs (A, B), (A, C) are, respectively, controllable and observable. The dimension of the matrix A is the same for all controllable and observable realizations. Moreover, for an arbitrary realization ˜ B, ˜ C), ˜ dim A˜ ≥ dim A. (A, Proof. The first part of the theorem is a direct consequence of Theorem ˜ B, ˜ C) ˜ are two controllable and observ5.3. Assume that (A, B, C) and (A, able realizations of the same impulse response function and that dim A˜ = n ˜, dim A = n. Since +∞

+∞

j tj ˜t , C˜ A˜j B CA B = Ψ(t) = j! j! j=0 j=0 j

t ≥ 0,

5.3 The characterization of transfer functions

so

˜ = CAj B C˜ A˜j B

75

for j = 0, 1, . . . .

(5.15)

Moreover, the pair (A, B) is controllable and (A, C) is observable, and therefore the ranks of the matrices ⎡ ⎤ C ⎢ CA ⎥ ⎢ ⎥ ⎢ ⎥ , [B, AB, . . . , Ar−1 B] .. ⎣ ⎦ . CAr−1

are n for arbitrary r ≥ n. Consequently the ranks of the matrices ⎡ ⎡ ⎤ ⎤ CB CAB . . . CAr−1 B C ⎢ CAB CA2 B . . . CAr B ⎥ ⎢ CA ⎥ ⎢ ⎢ ⎥ ⎥ r−1 [B, AB, . . . , A B] = ⎢ ⎢ ⎥ ⎥ (5.16) .. .. .. .. .. ⎣ ⎣ ⎦ ⎦ . . . . . r−1 r−1 r 2r−2 CA B CA B CA B . . . CA are, for r ≥ n, also equal to n. On the other hand, by (5.15), (5.16), ⎡

⎡ ⎤ ⎤ C˜ C ⎢ C˜ A˜ ⎥  ⎥  ⎢  ⎢ ⎢ CA ⎥  ⎥ ˜ ˜˜ n ˜ −1 ˜ B, A B, . . . , A B = ⎢ ⎢ ⎥ ⎥ B, AB, . . . , An˜ −1 B , .. .. ⎣ ⎣ ⎦ ⎦ . . n ˜ −1 n ˜ −1 ˜ ˜ CA CA and therefore n ˜ ≤ n. By the same argument n ≤ n ˜ , so n = n ˜ . The proof that ˜ B, ˜ C), ˜ is analogous. n ˜ ≥ n for arbitrary realization (A, 2

§5.3. The characterization of transfer functions The transfer function of a given triplet (A, B, C) was defined by the formula Φ(λ) = C(λI − A)−1 B, λ ∈ C \ σ(A). Let Φ(λ) = [ϕqr (λ), q = 1, . . . , k, r = 1, 2, . . . , m], λ ∈ C\σ(A). It follows, from the formulae for the inverse matrix (λI − A)−1 , that functions ϕqr are rational with real coefficients for which the degree of the numerator is greater than the degree of the denominator. So they vanish as |λ| −→ +∞. Moreover, Φ(λ) =

+∞ j=0

1 CAj B λj+1

for |λ| > |A|,

and the series in (5.17) is convergent with its all derivatives.

(5.17)

76

5 Realization theory

Theorem 5.5. If Φ(·) is an M(k, m; C)-valued function, defined outside of a finite subset of C, with rational elements having real coefficients vanishing as |λ| −→ +∞, then the function Φ(·) is of the form (5.9). Consequently, Φ(·) is the transfer function of a system (A, B, C). Proof. There exists a polynomial p(λ) = λr + a1 λr−1 + . . . + ar , λ ∈ C, with real coefficients, such that for arbitrary elements ϕqs , q = 1, . . . , k, s = 1, . . . , m, of the matrix Φ, functions pϕqs , are polynomials. Since Φ(·) is rational for some γ > 0 and arbitrary λ, |λ| > γ, Φ(λ) =

+∞ j=0

1 λj+1

Wj .

The coefficients by λ−1 , λ−2 , . . . of the expansion of p(λ)Φ(λ), λ ∈ C, are zero, therefore ar W0 + ar−1 W1 + . . . + a1 Wr−1 + Wr = 0, ar W1 + ar−1 W2 + . . . + a1 Wr + Wr+1 = 0, .. . ar Wj + ar−1 Wj+1 + . . . + a1 Wr+j−1 + Wr+j = 0. Hence the sequence W0 , W1 ,. . . satisfies (5.12). Arguing as in the proof of Theorem 5.2, we deduce that there exist matrices A, B, C such that Wj = CAj B,

j = 0, 1, . . . . 2

The proof is complete. Exercise 5.4. Assume that k = m = 1 and let 1 1 ϕ(λ) = + , λ ∈ C \ {a, b}, λ − a (λ − b)2

where a and b are some real numbers. Find the impulse response function Ψ(·) corresponding to ϕ. Construct its realization. Answer. Ψ(t) = eat + tebt , t ≥ 0.

Bibliographical notes Realization theory was created by R.E. Kalman [56]. He is the author of Theorems 5.2, 5.3 and 5.4. The proof of Theorem 5.1 is borrowed from R.W. Brockett [12].

Part II

Nonlinear Control Systems

Chapter 6

Controllability and observability of nonlinear systems

This chapter begins by recalling basic results of nonlinear differential equations. Controllability and observability of nonlinear control systems are studied next. Two approaches to problems are illustrated: one based on linearization and the other one on concepts of differential geometry.

§6.1. Nonlinear differential equations The majority of the notions and results of Part I have been generalized to nonlinear systems of the form y˙ = f (y, u), w = h(y).

y(0) = x ∈ Rn ,

(6.1) (6.2)

Functions f and h in (6.1) and (6.2) are defined on Rn × Rm and Rn and take values in Rn and Rk , respectively. In the present chapter we discuss generalizations concerned with controllability and observability. The following two chapters are devoted to stability and stabilizability as well as to the problem of realizations. As in Part I by control, strategy or input we will call an arbitrary locally integrable function u(·) from [0, +∞) into Rm . The corresponding solution y(·) of the equation (6.1) will be denoted by y x,u (t) or y u (t, x), t ≥ 0. The function h(y(·)) is called the output or the response of the system. To proceed further we will need some results on general differential equations (6.3) z˙ = f (z(t), t), z(t0 ) = x ∈ Rn , where t0 is a nonnegative number and f a mapping from Rn × R into Rn . A solution z(t), t ∈ [0, T ], of equation (6.3) on the interval [0, T ], t0 ≤ T , is an arbitrary absolutely continuous function z(·) : [0, T ] −→ Rn satisfying (6.3) © Springer Nature Switzerland AG 2020 J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_6

79

80

6 Controllability and observability of nonlinear systems

for almost all t ∈ [0, T ]. A local solution of (6.3) is an absolutely continuous function defined on an interval [t0 , τ ), τ > t0 , satisfying (6.3) for almost all [t0 , τ ). If a local solution defined on [t0 , τ ) cannot be extended to a local solution on a larger interval [t0 , τ1 ), τ1 > τ , then it is called a maximal solution and the interval [t0 , τ ) is the maximal interval of existence. An arbitrary local solution has an extension to a maximal one. The following classical results on the existence and uniqueness of solutions to (6.3) hold. Theorem 6.1. Assume that f is a continuous mapping from Rn ×R into Rn . Then for arbitrary t0 ≥ 0 and x ∈ Rn there exists a local solution to (6.3). If z(t), t ∈ [t0 , τ ), is a maximal solution and τ < +∞ then lim |z(t)| = +∞. t↑τ

Theorem 6.2. Assume that for arbitrary x ∈ Rn the function f (·, x) : [0, T ] −→ Rn is Borel measurable and for a nonnegative, integrable on [0, T ] function c(·) |f (x, t)| ≤ c(t)(|x| + 1), |f (x, t) − f (y, t)| ≤ c(t)|x − y|,

(6.4) n

x, y ∈ R , t ∈ [0, T ].

(6.5)

Then equation (6.3) has exactly one solution z(·, x). Moreover, for arbitrary t ∈ [0, T ] the mapping x −→ z(t, x) is a homeomorphism of Rn into Rn . Proofs of the existence and uniqueness of the solutions, similar to those for linear equations (see [78]) will be omitted. The existence of explosion, formulated in the second part of Theorem 6.1, is left as an exercise. The above-formulated definitions and results can be directly extended to the complex case, when the state space Rn is replaced by Cn . The following result is a corollary of Theorem 6.2. Theorem 6.3. Assume that the transformation f : Rn × Rm −→ Rn is continuous and, for a number c > 0, |f (x, u)| ≤ c(|x| + |u| + 1), |f (x, u) − f (y, u)| ≤ c|x − y|,

x, y ∈ Rn , u ∈ Rm .

(6.6) (6.7)

Then for an arbitrary control u(·) there exists exactly one solution of the equation (6.1). Proof. If u(·) : [0, T ] −→ Rn is an integrable function then f (x, u(t)), x ∈ Rn , t ∈ [0, T ], satisfies the assumptions of Theorem 6.2 and the result follows. 2 Corollary 6.1. If conditions (6.6), (6.7) hold then the control u(·) and the initial condition x uniquely determine the output y u (·, x).

6.1 Nonlinear differential equations

81

In several proofs of control theory a central role is played by theorems on the regular dependence of solutions on the initial data. Theorem 6.4. Assume that the conditions of Theorem 6.2 hold, that for arbitrary t ∈ [0, T ] the function f (·, t) has a continuous derivative fx (·, t) and that the functions f (·, ·), fx (·, ·) are bounded on bounded subsets of [0, T ]×Rn . Then the mapping x −→ z(·, x) acting from Rn into the space of continuous functions C(0, T ; Rn ) is Fréchet differentiable at an arbitrary point x0 and the directional derivative in the direction v ∈ Rn is a solution ξ(·) of the linear equation (6.8) ξ˙ = fx (z(t, x0 ), t)ξ, ξ(t0 ) = v. In particular, the function t −→ zx (t, x0 ) is absolutely continuous and satisfies the equation d zx (t, x0 ) = fx (z(t, x0 ), t)zx (t, x0 ) dt zx (t0 , x0 ) = I.

for a.a. t ∈ [0, T ],

(6.9) (6.10)

Theorem 6.4 is often formulated under stronger conditions, therefore we will prove it here. We will need the classical, implicit function theorem, the proof of which, following easily from the contraction principle, will be omitted. See also Lemma IV.17.3. Lemma 6.1. Let X and Z be Banach spaces and F a mapping from a neighbourhood O of a point (x0 , z0 ), with the following properties: (i) F (x0 , z0 ) = 0, (ii) there exist Gateaux derivatives Fx (·, ·), Fz (·, ·) continuous at (0, 0), and the operator Fx (x0 , z0 ) has a continuous inverse Fz−1 (x0 , z0 ). Then there exist balls O(x0 ) ⊂ X and O(z0 ) ⊂ Z with centres, respectively, at x0 , z0 such that for arbitrary x ∈ O(x0 ) there exists exactly one z ∈ O(z0 ), denoted by z(x), such that (iii) F (x, z) = 0. Moreover the function z(·) is differentiable in the neighbourhood O(x0 ) and zx (x) = −Fz−1 (x, z(x))Fx (x, z(x)). Proof of Theorem 6.4. Let X = Rn , Z = C(0, T ; Rn ) and define the mapping F : X × Z −→ Z by the formula  t F (x, z(·))(t) = x + f (z(s), s) ds − z(t), t ∈ [0, T ]. t0

The mapping Fx (x, z) associates with an arbitrary vector v ∈ Rn the function on [0, T ] with a constant value v. The Gateaux derivative Fz (x, z) in the direction ξ ∈ Z is equal to

82

6 Controllability and observability of nonlinear systems

 (Fz (x, z)ξ)(t) =

t

t0

fx (z(s), s)ξ(s) ds − ξ(t),

t ∈ [0, T ].

(6.11)

To prove (6.11), remark that if h > 0 then by Theorem A.6 (the mean value theorem)  t  1   fx (z(s), s)hξ(s) ds F (x, z + hξ)(t) − F (x, z)(t) − t∈[0,T ] h t0   1 T  ≤ f (z(s) + hξ(s), s) − f (z(s), s) − fx (z(s), s)hξ(s) ds h 0  T   fx (η, s) − fx (z(s), s) ds. ≤ sup sup

0

η∈I(z(s),z(s)+hξ(s))

By the assumptions and the Lebesgue dominated convergence theorem we obtain (6.11). From the estimate  Fz (x, z)ξ − Fz (¯ x, z¯)ξ ≤

0

T

fx (z(s), s) − fx (¯ z (s), s) ds ≤ ξ

valid for x, x ¯ ∈ Rn and z, z¯ ∈ Z, ξ ∈ Z and again by the Lebesgue dominated convergence theorem the continuity of the derivative Fz (·, ·) follows. Assume that  t f (z0 (s), s) ds = z0 (t) = z(t, x0 ), t ∈ [0, T ]. x0 + t0

To show the invertibility of Fz (x0 , z0 ) assume η(·) ∈ Z and consider the equation  t x0 + fy (z0 (s), s)ξ(s) ds − ξ(t) = η(t), t ∈ [0, T ]. t0

Putting ζ(t) = η(t) + ξ(t), t ∈ [0, T ], we obtain the following equivalent differential equation on ζ(·): ζ˙ = fz (z0 (t), t)ζ − fz (z0 (t), t)η(t),

t ∈ [0, T ],

(6.12)

ζ(t0 ) = x0 , which, by Theorem I.1.1, has exactly one solution. Therefore, the transformation F satisfies all the assumptions of Lemma 6.1, and, consequently, for all x ∈ Rn sufficiently close to x0 , the equation (iii) — equivalent to (6.3) — has the unique solution z(·, x) ∈ Z, with the Fréchet derivative at x0 given by ξ = zx (x0 )v = −Fz−1 (x0 , z0 )Fx (x0 , z0 )v.

6.2 Controllability and linearization

83

Since η(t) = Fx (x0 , z0 )v = v, t ∈ [0, T ], by (I.1.7) and (1.12): ˙ = ζ(t) ˙ = fz (z0 (t), t)(ξ(t) + v) − fz (z0 (t), t)v ξ(t) = fz (z0 (t), t)ξ(t), ξ(t0 ) = v.



Let us finally consider equation (6.3) which depends on a parameter γ ∈ Rl and denote by z(·, γ, x), γ ∈ Rl , a solution of the equation z˙ = f (z, γ, t),

(6.13)

z(t0 ) = x.

We have the following: Theorem 6.5. Assume that a function f (x, γ, t), (x, γ) ∈ Rn × Rl , t ∈ [0, T ], satisfies the assumptions of Theorem 6.4 with Rn replaced by Rn × Rl . Then for arbitrary t ∈ [0, T ], γ0 ∈ Rl , x0 ∈ Rn , the function γ −→ z(t, γ, x0 ) is Fréchet differentiable at γ0 and d zγ (t, γ0 , x0 ) = fx (z(t, γ0 , x0 ), γ0 , t)zγ (t, γ0 , x0 ) dt + fγ (z(t, γ0 , x0 ), γ0 , t), t ∈ [0, T ].

(6.14)

Proof. Consider the following system of equations: z˙ = f (z, γ, t),

γ˙ = 0,

z(t0 ) = x,

γ(t0 ) = γ.

(6.15)

The function (x, γ, t) −→ (f (x, γ, t), 0) acting from Rn ×Rl ×[0, T ] into Rn ×Rl satisfies the assumptions of Theorem 6.4 with Rn replaced by Rn × Rl . Consequently the solution t −→ (z(t, γ, x), γ) of (6.15) is Fréchet differentiable with respect to (x, γ) and an arbitrary point (x0 , γ0 ). It is now enough to apply (6.9). 2

§6.2. Controllability and linearization Let B(a, r) denote the open ball of radius r and centre a contained in Rn . We say that the system (6.1) is locally controllable at x ¯ and at time T if for arbitrary ε > 0 there exists δ ∈ (0, ε) such that for arbitrary a, b ∈ B(¯ x, δ) there exists a control u(·) defined on an interval [0, t] ⊂ [0, T ] for which y a,u (t) = b, y

a,u

(s) ∈ K(¯ x, ε),

for all s ∈ [0, t].

(6.16) (6.17)

We say also that the point b for which (6.16) holds, is attainable from a at time t.

84

6 Controllability and observability of nonlinear systems

Exercise 6.1. Assume that the pair (A, B), A ∈ M(n, n), B ∈ M(n, m), is controllable, and show that the system y˙ = Ay + Bu,

(6.18)

y(0) = x,

is locally controllable at 0 ∈ Rn at arbitrary time T > 0. Hint. Apply Proposition I.1.1 and Theorem I.1.2. The result formulated as Exercise 6.1 can be extended to nonlinear systems (6.1) using the concept of linearization. Assume that the mapping f is differentiable at (¯ x, u ¯) ∈ Rn × Rm ; then the system (6.18) with A = fx (¯ x, u ¯),

B = fu (¯ x, u ¯)

(6.19)

is called the linearization of (6.1) at (¯ x, u ¯). Theorem 6.6. Assume that the mapping f is continuously differentiable in a neighbourhood of a point (¯ x, u ¯) for which (6.20)

f (¯ x, u ¯) = 0.

If the linearization (6.19) is controllable then the system (6.1) is locally controllable at the point x ¯ and at arbitrary time T > 0. Proof. Without any loss of generality we can assume that x ¯ = 0, u ¯ = 0. Let us first consider the case when the initial condition a = 0. Controllability of (A, B) = (fx (0, 0), fu (0, 0)) implies, see Chapter I.1.1, that there exist smooth controls u1 (·), . . . , un (·) with values in Rm and defined on a given interval [0, T ], such that for the corresponding outputs y 1 (·), . . . , y n (·) of the equation (6.1), with the initial condition 0, vectors y 1 (T ), . . . , y n (T ) are linearly independent. For an arbitrary γ = (γ1 , . . . , γn )∗ ∈ Rn we set u(t, γ) = u1 (t)γ1 + . . . + un (t)γn ,

t ∈ [0, T ].

Let y(t, γ, x), t ∈ [0, T ], γ ∈ Rn , x ∈ Rn , be the output of (6.1) corresponding to u(·, γ). Then d y(t, γ, x) = f (y(t, γ, x), u(t, γ)), dt y(t, γ, x) = x.

t ∈ [0, T ],

(6.21)

By Theorem 6.5 applied to (6.21), with x0 = 0, γ0 = 0, t ∈ [0, T ], we obtain

6.2 Controllability and linearization

85

d ∂f yγ (t, 0, 0) = (y(t, 0, 0), u(t, 0))yγ (t, 0, 0) dt ∂x ∂u ∂f (y(t, 0, 0), u(t, 0)) (t, 0) + ∂u ∂γ = Ayγ (t, 0, 0) + B [u1 (t), . . . , un (t)]. Consequently the columns of the matrix yγ (t, 0, 0) are identical to the vectors y 1 (t), . . . , y n (t), t ∈ [0, T ], and the matrix yγ (T, 0, 0) is nonsingular. By Lemma 6.1, the transformation γ −→ y(T, γ, 0) maps an arbitrary neighbourhood of 0 ∈ Rn onto a neighbourhood of 0 ∈ Rn , so for ε > 0 and τ > 0 there exists δ ∈ (0, ε) such that for arbitrary b ∈ B(0, δ) there exists γ ∈ Rn , |γ| < r, and y(T, γ, 0) = b. This proves local controllability if a = 0. To complete the proof let us consider the system y˜˙ = −f (˜ y, u ˜), y˜(0) = 0, (6.22) and repeat the above arguments. If u(t) = u ˜(T − t), then

y(t) = y˜(T − t),

y(t) ˙ = f (y(t), u(t)),

t ∈ [0, T ],

t ∈ [0, T ],

and y(0) = y˜(T ). Taking into account that the state a = y(T ) could be an arbitrary element from a neighbourhood of 0 we conclude that system (6.1) is locally controllable at 0 and at time 2T , the required property. 2 Condition (6.20) is essential for the validity of Theorem 6.5 even for linear systems. Theorem 6.7. System (6.18) is locally controllable at x ¯ ∈ Rn if and only if the pair (A,B) is controllable, and A¯ x + Bu ¯ = 0 for some u ¯ ∈ Rm .

(6.23) (6.24)

Proof. Taking into account Theorem 6.6 it remains only to prove the necessity of conditions (6.23)–(6.24). To prove the necessity of (6.23) assume that A¯ x + Bu = 0 for arbitrary x + Bu with the minimal norm. Then u ∈ Rm . Let v be a vector of the form A¯ x + Bu − v, v = 0. Consequently, for |v| > 0 and, for arbitrary u ∈ Rm , A¯ x ∈ Rn , u ∈ Rm , Ax + Bu, v = A¯ x + Bu + A(x − x ¯), v ¯), v ≥ |v|2 − |A| |x − x ¯| |v|. = |v|2 + A(x − x If δ is a number such that 0 < δ < |v|2 , then

86

6 Controllability and observability of nonlinear systems

Ax + Bu, v ≥ δ for x ∈ B(¯ x, r¯),

u ∈ Rm ,

where r¯ = (|v|2 − δ)/(|A| |v|) > 0. Therefore for arbitrary control u(·) and the output y x¯,u we have d x¯,u y (t), v ≥ δ, provided t ∈ [0, t¯], dt where t¯ = inf{t; |y x¯,u (t) − x ¯| ≥ r¯}. So an arbitrary state b = x ¯ − βv/|v|, β ∈ (0, r¯), can be attained from x ¯ only after the output exits from B(¯ x, r¯). The system (6.1) cannot be locally controllable at x ¯. Assume now that the system (6.1) is controllable at x ¯ ∈ Rn and (6.24) holds. If y(·, x ¯) is the output corresponding to u(·) then y˜(·, 0) = y(·, x ¯) − x ¯ is the response of (6.1) to u ˜(·) = u(·) − u ¯. Therefore we obtain that the system (6.1) is locally controllable at 0 and that the pair (A, B) is controllable, see Section I.1.2. 2 Exercise 6.2. Let a function g(ξ1 , ξ2 , . . . , ξn, ξn+1 ), ξ1 , . . . , ξn+1 ∈ R, be of class C 1 and satisfy the condition: g(0, . . . , 0) = 0, ∂g/∂ξn+1 (0, . . . , 0) = 0. Show that the system  (n−1)  d z d(n) z (t) = g (t), . . . , z(t), u(t) , t ≥ 0, (6.25) dt(n) dt(n−1) treated as a system in Rn , is locally controllable at 0 ∈ Rn . Hint. Show first that the linearization (6.2) is of the form d(n) z ∂g d(n−1) z ∂g = (0, . . . , 0) (n−1) (t) + . . . + (0, . . . , 0)z(t) (n) ∂ξ1 ∂ξn dt dt ∂g + (0, . . . , 0)u(t), t ≥ 0. ∂ξn+1

(6.26)

There exist important locally controllable systems whose linearizations are not controllable. A proper tool to discuss such cases is the Lie bracket introduced in the next section.

§6.3. Lie brackets We define a vector field of class C k as an arbitrary mapping f , from an open set D ⊂ Rn into Rn , whose coordinates are k-times continuously differentiable in D. Let f and g be vector fields of class C 1 , defined on D, with coordinates f 1 (x), . . . , f n (x) and g 1 (x), . . . , g n (x), x ∈ D, respectively. The Lie bracket of f, g is a vector field denoted by [f, g] and given by

6.3 Lie brackets

87

[f, g](x) = fx (x)g(x) − gx (x)f (x),

x ∈ D.

Thus, the i-th coordinate, [f, g]i (x), of the Lie bracket is of the form  n   ∂f i ∂g i i k k (x)g (x) − (x)f (x) , (6.27) [f, g] (x) = ∂xk ∂xk k=1

⎤ x1 ⎢ ⎥ x = ⎣ ... ⎦ ∈ D. ⎡

xn A subset P of Rn is called a k-dimensional surface, k = 1, 2, . . . , n, in short, a surface, if there exists an open set V ⊂ Rk and a homeomorphism q mapping V onto P with the properties: ⎡ 1⎤ q ⎢ .. ⎥ q = ⎣ . ⎦ is of class C 1 , (6.28) qn

⎤ v1 ∂q ∂q ⎢ ⎥ vectors (v), . . . , (v) are linearly independent, v = ⎣ ... ⎦ ∈ V. (6.29) ∂v1 ∂vk vk ⎡

The transformation q is called a parametrization of the surface and the ∂q ∂q (v), . . . , ∂v (v) is called the linear subspace of Rn spanned by vectors ∂v 1 k tangent space to the surface P at the point q(v). We will need the following proposition. Proposition 6.1. (i) If q(·) is a mapping of class C 1 from an open set V ∈ Rk into a Banach v ) : Rk −→ E at a point v¯ is one-to-one space E such that the derivative qv (¯ then there exists a neighbourhood W of v¯ such that the mapping q restricted to W is a homeomorphism. (ii) If, in addition, E = Rk , then the image q(W ) of an arbitrary neighbourhood W of v¯ contains a ball with the centre q(¯ v ). Proof. (i) Without any loss of generality we can assume that v¯ = 0, q(¯ v) = 0. Let A = qv (¯ v ). Since the linear mapping A is defined on a finite-dimensional linear space and is one-to-one, there exists c > 0 such that Av ≥ c|v| for v ) = 0, and by Theorem v ∈ Rk . Define q¯(v) = q(v) − Av, v ∈ Rk . Then q¯v (¯ A.6 (the mean value theorem), for arbitrary ε > 0 there exists δ > 0 such that ¯ q (u) − q¯(v) ≤ ε|u − v|, if |u|, |v| < δ. Let ε ∈ (0, c) and W = B(0, δ). For u, v ∈ W

88

6 Controllability and observability of nonlinear systems

c|u − v| − q(u) − q(v) ≤ A(u − v) − q(u) − q(v) ≤ ¯ q (u) − q¯(v) ≤ ε|u − v|. So (c − ε)|u − v| ≤ q(u) − q(v) ,

u, v ∈ W,

and we see that q restricted to W is one-to-one, and the inverse transformation satisfies the Lipschitz condition with the constant (c−ε)−1 . This finishes the proof of (i). Part (ii) of the proposition follows from Lemma 6.1. 2 As far as Lie brackets are concerned, we will need the following basic result. Theorem 6.8. Let q(v), v ∈ V , be a parametrization of class C 2 of the surface P in Rn . Let f and g be vector fields of class C 1 defined in a neighbourhood of P. If for arbitrary v ∈ V the vectors f (q(v)), g(q(v)) belong to the tangent space to P at q(v), then for arbitrary v ∈ V the Lie bracket [f, g](q(v)) belongs to the tangent space to P at q(v). Proof. Let q 1 (v), . . . , q n (v), f 1 (q(v)), . . . , f n (q(v)), g 1 (q(v)), . . . , g n (q(v)) be coordinates, respectively, of q(v), f (q(v)), g(q(v)), v ∈ V . By (6.29) and Cramer’s formulae, there exist functions α1 (v), . . . , αk (v), β 1 (v), . . . , β k (v), v ∈ V , of class C 1 such that f (q(v)) =

k  ∂q (v)αj (v), ∂v j j=1

g(q(v)) =

k  ∂q (v)β j (v), ∂v j j=1

(6.30) v ∈ V.

(6.31)

Therefore   k ∂ ∂  ∂q i i j (f (q(v))) = (v)α (v) ∂vl ∂vl j=1 ∂vj =

k k   ∂ 2 qi ∂q i ∂αj (v)αj (v) + (v) (v), ∂vl ∂vj ∂vj ∂vl j=1 j=1

k k   ∂ i ∂ 2 qi ∂q i ∂β j (g (q(v))) = (v)β j (v) + (v) (v). ∂vl ∂vl ∂vj ∂vj ∂vl j=1 j=1

On the other hand

(6.32)

(6.33)

6.3 Lie brackets

89 n  ∂ ∂f i ∂q r (f i (q(v))) = (q(v)) (v), ∂vl ∂xk ∂vl r=1 n  ∂g i ∂ i ∂q r (g (q(v))) = (q(v)) (v), ∂vl ∂xr ∂vl r=1

(6.34) v ∈ V.

(6.35)

Taking into account definition (6.27) i

[f, g] (q(v)) =

 ∂g i r (q(v))g (q(v)) − (q(v))f (q(v)) , ∂xr ∂xr

n   ∂f i r=1

r

v ∈ V.

From (6.30), (6.31) and (6.34), (6.35) we have [f, g]i (q(v)) =

  k ∂q r (q(v)) (v)β l (v) ∂xr ∂vl r=1 l=1   k ∂g i ∂q r − (q(v)) (v)αl (v) ∂xr ∂vl n  ∂f i

l=1

=

k 

k

β l (v)

l=1

 ∂ ∂ i (f i (q(v)))− αl (v) (g (q(v))). ∂vl ∂vl l=1

Hence by (6.32), (6.33) i

[f, g] (q(v)) =

k  l=1

−

  k  ∂ 2 qi ∂q i ∂αj j β (v) (v)α (v) + (v) (v) ∂vl ∂vj ∂vj ∂vl j=1 l

k  l=1

  k  ∂ 2 qi ∂q i ∂β j j α (v) (v)β (v) + (v) (v) . ∂vl ∂vj ∂vj ∂vl j=1 l

Since the functions q i , i = 1, . . . , n, are of class C 2 we obtain finally   k k  ∂q i ∂αj ∂β j l l [f, g] (q(v)) = (v) (v)β (v) − (v)α (v) , ∂vj ∂vl ∂vl j=1 i

l=1

i = 1, 2, . . . , n,

v ∈ V.

Therefore the vector [f, g](q(v)) is in the tangent space to P at q(v). The proof of the theorem is complete. 2 Exercise 6.3. Let A1 , A2 ∈ M(n, n), a1 , a2 ∈ Rn and f (x) = A1 x + a1 , g(x) = A2 x + a2 , x ∈ Rn . Show that [f, g](x) = (A1 A2 − A2 A1 )x + A1 a2 − A2 a1 ,

x ∈ Rn .

90

6 Controllability and observability of nonlinear systems

§6.4. The openness of attainable sets Theorem 6.9 of the present section gives algebraic conditions for the set of all attainable points to be open. The property is slightly weaker than that of local controllability. Under additional conditions, formulated in Theorem 6.10, the local controllability follows. Let U be a fixed subset of Rm and assume that mappings f (·, u), u ∈ U , from Rn into Rn , defining system (6.1), are of class C k . By elementary control we understand a right continuous piecewise constant function u : R+ −→ U , taking on only a finite number of values. Denote by L0 the set of all vector fields defined on Rn of the form f (·, u), u ∈ U , and by Lj , j = 1, 2, . . . , the set of all vector fields which can be obtained from vector fields in L0 by an application of the Lie brackets at most j times: L0 = {f (·, u); u ∈ U }, Lj = Lj−1 ∪ {[f, g](·); f (·) ∈ Lj−1 and g(·) ∈ L0 or f (·) ∈ L0 and g(·) ∈ Lj−1 }, j = 1, 2, . . . . Let us define additionally for x ∈ Rn and j = 0, 1, 2, . . . Lj (x) = {f (x); f ∈ Lj } , and let dim Lj (x) be the maximum number of linearly independent vectors in Lj (x). The following result holds. Theorem 6.9. Assume that the fields f (·, u), u ∈ U , are of class C k for some k ≥ 2 and that for some x ∈ Rn and j ≤ k dim Lj (x) = n.

(6.36)

Then the set of all points attainable from x by elementary controls, which keep the system in a given in advance neighbourhood D of x, has a nonempty interior. Proof. Since the fields f (·, u), u ∈ U , are smooth, we can assume that condition (6.36) is satisfied for any x ∈ D. For arbitrary parameters u1 , . . . , ul ∈ U , for a fixed element u+ ∈ U and for positive numbers v1 , v2 , . . . , vl , l = 1, 2, . . . , denote by u(t; u1 , . . . , ul ; v1 , . . . , vl ), t ≥ 0, a control u(·) defined by: ⎧ ⎪ if 0 ≤ t < v1 , ⎨u1 , u(t) = ur , if v1 + . . . + vr−1 ≤ t < v1 + . . . + vr , r = 2, . . . , l, (6.37) ⎪ ⎩ + u , if t ≥ v1 + . . . + vl . Let us fix parameters u1 , . . . , ul and element x ∈ Rn and let q(v1 , . . . , vl ) = y u(·;u1 ,...,ul ;v1 ,...,vl ) (v1 + . . . + vl , x),

v1 , . . . , vl > 0.

6.4 The openness of attainable sets

91

Note that   q(v1 , . . . , vl ) = z ul vl , z ul−1 (vl−1 , z ul−2 (. . . , z u1 (v1 , x), . . . )) ,

(6.38)

where z u (t, y), t ≥ 0, y ∈ Rn , denote the solution of the equation z˙ = f (z, u),

z(0) = y.

(6.39)

Let l ≤ n be the largest natural number for which there exist control parameters u1 , . . . , ul ∈ U and positive numbers α1 < β1 , . . . , αl < βl such that the formula (6.38) defines a parametric representation ⎧⎡ ⎤ ⎫ ⎪ ⎪ ⎨ v1 ⎬ ⎢ ⎥ V = ⎣ ... ⎦ ; αi < vi < βi , i = 1, 2, . . . , l , ⎪ ⎪ ⎩ ⎭ vl of a surface of class C k , contained in D. We show first that l ≥ 1. Note that there exists u1 ∈ U , such that f (x, u1 ) = 0. For if f (x, u) = 0 for all u ∈ U , then the definition of the Lie bracket implies that g(x) = 0 for arbitrary g ∈ L0 and more generally g(x) = 0 for arbitrary g ∈ Lj , j = 1, 2, . . . . This contradicts the assumption that dim Lj (x) = n. If, now, f (x, u1 ) = 0 then, by Proposition 6.1(i), for small α1 > 0 the function z u1 (v, x), v ∈ (0, α1 ), is a parametric representation of a one-dimensional surface contained in D. Assume that l < n and let P be a surface given by (6.38). There exist ⎡ ⎤ v1 ⎢ .. ⎥  v = ⎣ . ⎦ ∈ V, u ∈ U, vl

such that the vector f (q(v  ), u ) is not included in the tangent space to P at q(v  ). This follows from Theorem 6.8 and from the assumption that dim Lj (q(v)) = n > l for v ∈ V . Since the function f (·, u ) is continuous, there exists δ > 0 such that if αj − δ < vj < αj + δ, j = 1, . . . , l, then the vector f (q(v), u ) is not in the tangent space to P at the point q(v). It follows from Proposition 6.1(i) that for some positive numbers αl+1 < βl+1 the transformation q  (·), 

q  (v1 , . . . , vl , vl+1 ) = z u (vl+1 , q(v1 , . . . , vl )), αj − δ < vj < αj + δ,

j = 1, . . . , l,

αl+1 < vl+1 < βl+1 ,

is a parametric representation of an (l+1)-dimensional surface included in D. By the regular dependence of solutions of differential equations on initial conditions (see Theorem 6.4), the transformation q  (·) is of the same class as q(·) and consequently of the same class C k as fields f (·, u), u ∈ U . This

92

6 Controllability and observability of nonlinear systems

contradicts the definition of the number l so l = n. Hence, by Proposition 6.1(ii), the image q(v) has a nonempty interior. This finishes the proof of the result. 2 Example 6.1. Consider the system y˙ 1 = 1, y˙ 2 = u(y1 )2 with the initial condition x = 0 and U = R. In this case 



1 0 , u ∈ R, f (x, u) = +u 0 (x1 )2 and the fields f (x) =

 1 , 0

g(x) =

0 (x1 )2

belong to L0 . By direct calculations,

 0 [f, g](x) = − = f1 (x), 2x1



,

x=

[f, f1 ](x) =

x1 x2



 0 , 2

∈ R2 ,

x ∈ R2 .

Hence dim L2 (x) = 2 for arbitrary x ∈ R2 . By Theorem 6.9, the set of all

 0 attainable points from has a nonempty interior. However, since y˙ 1 > 0, 0



 x1 0 points . Therefore, in general, , x1 < 0, cannot be attained from 0 x2 the conditions of Theorem 6.9 do not imply local controllability at x. Example 6.2. Let f (x, u) = Ax + Bu, x ∈ Rn , u ∈ Rm . Then (see Exercise 6.3), [f (·, u), f (·, v)] = AB(v − u). We easily see that L0 (x) = {Ax + Bu; u ∈ Rm } ,   Lk (x) = L0 (x) ∪ Aj Buj ; j = 1, . . . , k, uj ∈ Rm . So dim Ln−1 (0) = n if and only if the pair (A, B) is controllable. If the pair (A, B) is controllable then dim Ln−1 (x) = n

for arbitary x ∈ Rn .

This way condition (6.36) generalizes Kalman’s condition from Theorem I.1.2. Under additional assumptions, condition (6.36) does imply local controllability. We say that the system (6.1), with U ⊂ Rm , is symmetric if for arbitrary u ∈ U there exists u ∈ U such that f (x, u) = −f (x, u ),

x ∈ Rn .

(6.40)

6.4 The openness of attainable sets

93

Theorem 6.10. If system (6.1) is symmetric and conditions of Theorem 6.9 hold, then system (6.1) is locally controllable at x. Proof. Let us fix a neighbourhood D of x. By Theorem 6.7, there exists a ball B(˜ x, r) ∈ D such that arbitrary b ∈ B(˜ x, r) can be attained from x at a time τ (b) using elementary controls ub (t), t ∈ [0, τ (b)], and keeping the system in D. In particular there are numbers vi > 0 and parameters ui ∈ U , i = 1, 2, . . . , l, such that strategies ux (t) = u(t, u1 , . . . , ul ; v1 , . . . , vl ), t ≥ 0, transfer x to x ˜ at time τ (˜ x) = v1 + v2 + . . . + vl . Let u1 , . . . , ul ∈ U be parameters such that f (x, ui ) = −f (x, ui ) and Z i = Rn −→ Rn — transformations given by 

Z i (x) = z ui (vi , x),

x ∈ Rn , i = 1, 2, . . . , l,

compare (6.39). It follows from Theorem 6.2 that these transformations are homeomorphic. For b ∈ B(˜ x, r) we define new controls ⎧ b ⎪ ⎨u (t), if t ∈ [0, τ (b)], b u ˜ (t) = ui , if τ (b) + vi+1 + . . . + vl < t ≤ τ (b) + vi + vi+1 + . . . + vl , ⎪ ⎩ i = 1, 2, . . . , l, vl+1 = 0. Then the set

 u˜b  y (τ (b) + v1 + . . . + vl , x); b ∈ K(˜ x, r)

consists of states attainable from x and is equal to the image of the ball B(˜ x, r) by the transformation composed of Z 1 , Z l−1 , . . . , Z 1 . Therefore, this set contains a nonempty neighbourhood of x and local controllability of (6.1) at x follows. 2 Example 6.3. Consider the following system (compare also §7.8): y˙ 1 = u, y˙ 2 = v, y˙ 3 = y1 v − y2 u,

 u ∈ U = R2 . v

Let us remark that the linearization of the system at (x, 0) is of the form ⎤ ⎡ ⎡ ⎤ 1 0 0 0 0 1 ⎦, A = ⎣0 0 0⎦, B = ⎣ 0 0 0 0 −x2 x1 so the linearization is not controllable and Theorem 6.6 is not applicable. On the other hand, the system is symmetric. Let

94

6 Controllability and observability of nonlinear systems

⎤ 1 f (x) = ⎣ 0 ⎦ , −x2 ⎡

Then f (x), g(x) ∈ L0 (x) and

⎡

⎤ 0 [f, g](x) = ⎣ 0 ⎦ , −2

⎡

⎤ 0 g(x) = ⎣ 1 ⎦ . x1 ⎡

⎤ x1 x = ⎣ x2 ⎦ ∈ R3 . x3

Therefore dim L1 (x) = 3, x ∈ R3 , and the system is locally controllable at arbitrary point of R3 .

§6.5. Observability We will now extend the concept of observability from §I.1.6 to nonlinear systems. Let us assume that the stated equation is independent of the control parameter (6.41) z˙ = f (z), z(0) = x ∈ Rn , and that the observation is of the form w(t) = h(z(t)),

t ≥ 0.

(6.42)

We say that the system (6.41)–(6.42) is observable at a point x if there exists a neighbourhood D of x such that for arbitrary x1 ∈ D, x1 = x, there exists t > 0 for which h(z(t, x)) = h(z(t, x1 )). If, in addition, t ≤ T then the system (6.41)–(6.42) is said to be observable at point x and at the time T. We start from a sufficient condition for observability at an equilibrium state, based on linearization. To simplify notation we assume that the equilibrium state is 0 ∈ Rn . Theorem 6.11. Assume that transformations f : Rn −→ Rn and h : Rn −→ Rk are of class C 1 . If the pair (fx (0), hx (0)) is observable then the system (6.41)–(6.42) is observable at 0 and at any time T > 0. Proof. Let us fix T > 0 and define a transformation K from Rn into C(0, T ; Rn ) by the formula (Kx)(t) = h(z(t, x)),

x ∈ Rn , t ∈ [0, T ].

It follows from Theorem 6.4 that the transformation K is Fréchet differen¯ tiable at arbitrary x ¯ ∈ Rn . In addition the directional derivative of K at x and at the direction v ∈ Rn is equal to

6.5 Observability

95

(Kx (¯ x; v))(t) = hx (z(t, x ¯))zx (t, x ¯)v, and d zx (t, x ¯) = fx (z(t, x ¯))zx (t, x ¯), dt zx (0, x ¯) = I.

t ∈ [0, T ],

In particular, for x ¯ = 0, v ∈ Rn , Kx (0; v)(t) = hx (0)efx (0)t v,

t ∈ [0, T ].

Since the pair (fx (0), hx (0)) is observable therefore the derivative Kx (0, ·) is a one-to-one mapping onto a finite-dimensional subspace of C(0, T ; Rn ). By Proposition 6.1, the transformation K is one-to-one in a neighbourhood of 0 and the system (6.41)–(6.42) is observable at 0 at time T . 2 We will now formulate a different sufficient condition for observability, often applicable if the linearization is not observable. Assume that the system (6.41)–(6.42) is of class C r or equivalently that the mappings f and h are of class at least C r , r ≥ 1. Let ⎡ 1 ⎤ ⎤ ⎡ 1 h (x) f (x) ⎢ ⎥ ⎥ ⎢ f (x) = ⎣ ... ⎦ , h(x) = ⎣ ... ⎦ , x ∈ Rn , f n (x)

hk (x)

and let H0 be the family of functions h1 , . . . , hk : H0 = {h1 , . . . , hk }. Define families Hj , 1 ≤ j ≤ k, as follows: Hj = Hj−1 ∪

n 

fi

i=1

 ∂g ; g ∈ Hj−1 , ∂xi

and set ⎧⎡ ⎪ ⎨ ⎢ dHj (x) = ⎣ ⎪ ⎩

⎤ ∂g ∂x1 (x) .. .

∂g ∂xn (x)

⎥ ⎦ ; g ∈ Hj

⎫ ⎪ ⎬ ⎪ ⎭

,

x ∈ Rn , j = 0, . . . , r − 1.

Let dim dHj (x) be the maximal number of linearly independent vectors in dHj (x). Example 6.4. Assume that f (x) = Ax, h(x) = Cx, x ∈ Rn , where A ∈ M(n, n), C ∈ M(k, n). Then f i (x) = a∗i , x , hj (x) = c∗j , x , i = 1, 2, . . . , n, j = 1, 2, . . . , k, x ∈ Rn , where a1 , . . . , an , c1 , . . . ck are the rows of matrices A

96

6 Controllability and observability of nonlinear systems

and C, respectively. Thus for arbitrary x ∈ Rn dH0 (x) = {c∗1 , . . . , c∗k }, dH1 (x) = dH0 (x) ∪ {A∗ c∗1 , . . . , A∗ c∗k }, .. . dHj (x) = dHj−1 (x) ∪ {(A∗ )j c1 , . . . , (A∗ )j ck , j = 1, 2, . . . }. Theorem 6.12. Assume that system (6.41)–(6.42) is of class C r , r ≥ 1, and dim dHr−1 (x) = n.

(6.43)

Then the system is observable at x at arbitrary time T > 0. Proof. Assume that condition (6.43) holds but the system (6.41)–(6.42) is not observable at x and at a time T > 0. Then, in an arbitrary neighbourhood D of x there exists a point x1 such that g(z x (t)) = g(z x1 (t)),

t ∈ [0, T ], g ∈ H0 .

(6.44)

By an easy induction argument (6.44) holds for arbitrary g ∈ Hr−1 . Since 1 n dim dHr−1 (x) = n, there ⎡ exist ⎤ in Hr−1 functions g˜ , . . . , g˜ such that the g˜1 (·) ⎢ .. ⎥ derivative of G(·) = ⎣ . ⎦ at x is nonsingular. In particular, for arbi-

g˜n (·) trary x1 sufficiently close to x, G(x) = G(x1 ) and, for some j < n, 2 g˜j (z x (0)) = g˜j (z x1 (0)), a contradiction with (6.44). Example 6.4. (Continuation) Condition dim dHn−1 (x) = n is equivalent to the rank condition of Theorem I.1.6.

Bibliographical notes Theorem 6.6 is due to E.B. Lee and L. Markus [63] and Theorem 6.9 to H.J. Sussmann and V. Jurdjevic [91]. The proof of Theorem 6.9 follows that of A.J. Krener [60]. Results of §6.4 and 6.5 are typical for geometric control theory based on differential geometry. More on this topic can be found in the monograph by A. Isidori [51]. Example 6.3 was introduced by R.W. Brockett [13].

Chapter 7

Stability and stabilizability

Three types of stability and stabilizability are studied: exponential, asymptotic and Lyapunov. Discussions are based on linearization and Lyapunov’s function approaches. When analysing a relationship between controllability and stabilizability topological methods are used.

§7.1. Differential inequalities The basic method of studying asymptotic properties of solutions of differential equations in Rn consists of analysing one-dimensional images of the solutions by appropriately chosen transformations from Rn into R. These images do not satisfy, in general, differential equations but very often are solutions of differential or integral inequalities. This is why we start with such inequalities. We consider first the Gronwall lemma. Lemma 7.1. Let k be a nonnegative, bounded, Borel measurable function on an interval [t0 , t1 ] and l a nondecreasing one. Let v be a function integrable on [t0 , t1 ] such that for almost all t ∈ [t0 , t1 ]  v(t) ≤ l(t)

t

k(s)v(s) ds.

(7.1)

t0

Then for those t ∈ [t0 , t1 ] for which (7.1) holds one has  t  v(t) ≤ exp k(s) ds l(t).

(7.2)

t0

In particular, (7.2) holds almost surely on [t0 , t1 ]. Proof. Assume first that l(t) = l(t0 ) for t ∈ [t0 , t1 ] and define © Springer Nature Switzerland AG 2020 J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_7

97

98

7 Stability and stabilizability



t

u(t) = l(t0 ) +  w(t) = exp −

k(s)v(s) ds, t0  t

 k(s) ds u(t),

t ∈ [t0 , t1 ].

t0

Functions u and w are absolutely continuous and u(t) ˙ = k(t)v(t) ≤ k(t)u(t),   t    t  w(t) ˙ = u(t) ˙ exp − k(s) ds − k(t)u(t) exp − k(s) ds ≤ 0, t0

t0

almost surely on [t0 , t1 ]. Therefore, w(t) ≤ w(t0 ) ≤ u(t0 ) ≤ l(t0 ),  t   t  u(t) = ω(t) exp k(s) ds ≤ l(t0 ) exp k(s) ds , t0

and, consequently,



v(t) ≤ u(t) ≤ exp

t0



t

k(s) ds l(t0 ) almost surely on [t0 , t1 ]. t0

If l is an arbitrary nondecreasing function on [t0 , t1 ] and t2 ∈ (t0 , t1 ), then l(t) ≤ l(t2 ) on [t0 , t2 ]. Moreover  t k(s)v(s) ds almost surely on [t0 , t2 ]. (7.3) v(t) ≤ l(t2 ) + t0

Assume additionally that (7.1) holds for t = t2 . Then (7.3) holds for t = t2 , as well. By the above reasoning,  t2  k(s) ds l(t2 ). v(t2 ) ≤ exp t0

2

The proof of the lemma is complete. The following result will play an important role in what follows.

Theorem 7.1. Assume that ϕ : [t0 , t1 ] × R −→ R is a continuous function such that for some M > 0 |ϕ(t, x) − ϕ(t, y)| ≤ M |x − y|,

t ∈ [t0 , t1 ], x, y ∈ R.

(7.4)

If v is an absolutely continuous function on [t0 , t1 ] such that v(t) ˙ ≤ ϕ(t, v(t)) or

a.s. on [t0 , t1 ],

(7.5)

7.1 Differential inequalities

99

v(t) ˙ ≥ ϕ(t, v(t))

a.s. on [t0 , t1 ],

(7.6)

then v(t) ≤ u(t)

for all t ∈ [t0 , t1 ]

(7.7)

v(t) ≥ u(t)

for all t ∈ [t0 , t1 ],

(7.8)

or, respectively, where u is a solution of the equation u(t) ˙ = ϕ(t, u(t)),

t ∈ [t0 , t1 ],

(7.9)

with the initial condition u(t0 ) ≥ v(t0 ) or, respectively, u(t0 ) ≤ v(t0 ). Proof. By Theorem 6.2 there exists exactly one solution of the equation (7.9) on [t0 , t1 ]. Assume that v(t) ˙ ≤ ϕ(t, v(t)) a.s. on [t0 , t1 ] and consider a sequence un (·), n = 1, 2, . . . , of functions satisfying 1 , t ∈ [t0 , t1 ], n n = 1, 2, . . . .

u˙ n (t) = ϕ(t, un (t)) + un (t0 ) = u(t0 ),

Let K > 0 be a number such that |ϕ(s, 0)| ≤ K, s ∈ [t0 , t1 ]. From (7.4), |ϕ(s, x)| ≤ |ϕ(s, x) − ϕ(s, 0)| + |ϕ(s, 0)| ≤ M |x| + K, s ∈ [t0 , t1 ], x ∈ R.

(7.10)

Since un (t) ≤ u(t0 ) + (t − t0 )

1 + n



t

ϕ(s, un (s)) ds,

t ∈ [t0 , t1 ],

(7.11)

t0

we have 

  t 1 |un (s)| ds, |un (t)| ≤ |u(t0 )| + K + (t − t0 ) + M n t0

t ∈ [t0 , t1 ].

Hence, by Lemma 7.1   1 |un (t)| ≤ |u(t0 )| + K + (t1 − t0 )eM (t−t0 ) , n

t ∈ [t0 , t1 ],

and functions un (t), t ∈ [t0 , t1 ], n = 1, 2, . . . , are uniformly bounded. They are also equicontinuous as

100

7 Stability and stabilizability

|un (t) − un (s)| ≤

t s

 |ϕ(r, un (r))| dr +

1 n (t

− s) ≤

 L+

1 n

(t − s),

t0 ≤ s ≤ t ≤ t1 , for a constant L > 0, independent of n. It follows from Theorem A.8 (the Ascoli theorem) that there exists a subsequence of (un (·)) uniformly continuous to a continuous function u ˜(·). Letting n in (7.11) tend to +∞ we see that the function u ˜(·) is an absolutely continuous solution of the equation (7.9) and therefore it is identical with u(·). To prove (7.7) it is sufficient to show that un (t) ≥ v(t) for

n = 1, 2, . . . , t ∈ [t0 , t1 ].

Assume that for a natural number m and for some t2 ∈ (t0 , t1 ), um (t2 ) < v(t2 ). Then for some t˜ ∈ [t0 , t2 ), um (t˜) = v(t˜) and um (t) < v(t) for t ∈ [t˜, t2 ). Taking into account the definitions of v(·) and um (·) and the continuity of ϕ(·, ·) we see that for some δ > 0 and arbitrary t ∈ (t˜, t˜ + δ)  t v(t) − v(t˜) 1 1 , ≤ ϕ(s, v(s)) ds ≤ ϕ(t˜, v(t˜)) + 3m t − t˜ t − t˜ t˜ um (t) − v(t˜) 1 . ≥ ϕ(t˜, v(t˜)) + 2m t − t˜ Therefore um (t) > v(t), t ∈ (t˜, t˜ + δ), a contradiction. This proves (7.7). In a similar way (7.6) implies (7.8). 2 Corollary 7.1. If v(·) is an absolutely continuous function on [t0 , t1 ] such that for some α ∈ R v(t) ˙ ≤ αv(t) then

a.s. on [t0 , t1 ],

v(t) ≤ eα(t−t0 ) v(t0 ),

t ∈ [t0 , t1 ].

Similarly if v(t) ˙ ≥ αv(t) a.s. on [t0 , t1 ] then v(t) ≥ eα(t−t0 ) v(t0 ),

t ∈ [t0 , t1 ].

§7.2. The main stability test In the present section we study the asymptotic behaviour of solutions of the equation

7.2 The main stability test

101

(7.12)

z˙ = Az + h(t, z), z(0) = x ∈ C , n

where A is a linear transformation from Cn into Cn , identified with an element of M(n, n; C), and h is a continuous mapping from R+ ×Cn onto Cn satisfying sup t≥0

|h(t, x)| −→ 0, |x|

if

|x| −→ 0.

(7.13)

As in Part I (see §I.2.1), we set ω(A) = max{Re λ; λ ∈ σ(A)}, and if x, y are vectors with complex coordinates ξ1 , . . . , ξn and η1 , . . . , ηn , x, y =

n  i=1

ξi η¯i ,

|x|2 =

n 

|ξi |2 .

i=1

Under the imposed conditions, for arbitrary x ∈ Cn there exists a maximal solution z(t), t ∈ [0, τ ), z(0) = x of (7.12). The following theorem is of basic importance in the theory of nonlinear systems. Theorem 7.2. (i) If (7.13) holds and ω(A) < 0, then for arbitrary ω > ω(A) there exist M > 0, δ > 0 such that an arbitrary maximal solution z(·) of (7.12), with initial condition z(0) satisfying |z(0)| < δ, is defined on [0, +∞) and |z(t)| ≤ M eωt |z(0)|, t ≥ 0. (ii) If (7.13) holds and ω(A) > 0 then there exists r > 0 such that for arbitrary δ > 0 one can find a solution z(t), t ∈ [0, τ ), with an initial condition z(0), |z(0)| < δ, and a number s ∈ [0, τ ), such that |z(s)| > r. If, in addition, transformations A and h(t, ·), t ≥ 0, restricted to Rn take values in Rn then there exists a solution z(·) with values in Rn having the described properties. Before proving the result we prove a lemma on Jordan blocks. Let us recall (see Theorem I.2.1) that a complex Jordan block corresponding to a number λ = α + iβ, β = 0, or λ = α, α, β ∈ R, is a square matrix of arbitrary dimension m = 1, 2, . . . or of dimension 1, of the form

102

7 Stability and stabilizability

⎡ ⎢ ⎢ ⎢ ⎢ ⎢ J =⎢ ⎢ ⎢ ⎢ ⎣

λ 0 0 .. .

γ λ 0 .. .

0 γ λ .. .

··· ··· ··· .. .

0 0 0 .. .

0 0 0 .. .

0 0 0

0 0 0

0 0 0

··· ··· ···

λ 0 0

γ λ 0

0 0 0 .. .

⎤

⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ 0 ⎥ ⎥ γ ⎦ λ

or J = [λ],

respectively, where γ is a number different from 0. Lemma 7.2. For an arbitrary complex Jordan block J and for arbitrary x ∈ Cn (Re λ − |γ|)|x|2 ≤ Re Jx, x ≤ (Re λ + |γ|)|x|2 . Proof. It is sufficient to consider the case of matrix J with the parameter γ = 0. Then Jx, x = λ|x|2 + γ(ξ2 ξ¯1 + . . . + ξm ξ¯m−1 ) and therefore Re Jx, x = Re λ|x|2 + Re (γ(ξ2 ξ¯1 + . . . + ξm ξ¯m−1 )). By the Schwartz inequality

Re (γ(ξ2 ξ¯1 + . . . + ξm ξ¯m−1 )) ≤ |γ||x|2 , 2

and the required inequality follows.

Proof of the theorem. (i) By Theorem I.7.1 there exists a nonsingular matrix P such that the matrix A˜ = P AP −1 consists of only complex Jordan blocks J1 , . . . , Jr corresponding to the eigenvalues of A and to each eigenvalue λ ∈ σ (A) corresponds at least one block. Parameters γ = 0 can be chosen in advance. The function z˜(t) = P z(t), t ∈ [0, τ ), is a solution of the equation ˜ z˜), ˜z + h(t, z˜˙ = A˜

z˜(0) = P z(0),

˜ x) = P h(t, P −1 x), t ≥ 0, x ∈ Cn , satisin which the transformation h(t, fies also the conditions of the theorem. Since the matrix P is nonsingular, therefore, without any loss of generality, we can assume that the matrix A consists of Jordan blocks only with parameters γ = 0 chosen arbitrarily. Since ω(A) < 0 we can assume that ω < 0. It follows from Lemma 7.2 that Re Ax, x ≤ (ω(A) + γ)|x|2 ,

x ∈ Cn .

For an arbitrary maximal solution z(t), t ∈ I = [0, τ ), define v(t) = |z(t)|2 ,

t ∈ I.

7.2 The main stability test

103

Then     d v(t) = 2Re Az(t), z(t) + 2Re h(t, z(t)), z(t) dt ≤ 2(ω(A) + γ)|z(t)|2 + 2|h(t, z(t))| |z(t)|, t ∈ I. By (7.13) there exists δ > 0 such that |h(t, x)| ≤ ε|x| for t ≥ 0, |x| ≤ δ. Let |z(0)| < δ and τ¯ = inf{t ∈ [0, r); |z(t)| ≥ δ} be the first exit time of the solution from the ball B(0, δ), (inf ∅ = +∞). Then (see Theorem 6.1) either τ¯ = +∞ or τ¯ < τ and d v(t) ≤ 2(ω(A) + γ + ε)v(t), dt

t ∈ I¯ = [0, τ¯).

By Corollary 7.1 of Theorem 7.1 v(t) < e2ωt v(0),

¯ t ∈ I,

or equivalently |z(t)| ≤ eωt |z(0)|, If τ¯ < +∞, then

¯ t ∈ I.

(7.14)

δ ≤ lim|z(t)| ≤ eωτ¯ |z(0)| < δ, t↑¯ τ

a contradiction. Thus τ¯ = +∞ and inequality (7.14) completes the proof of part (i). (ii) We can assume that the matrix A is of the form   B 0 A= , (7.15) 0 C where matrices B ∈ M(k, k, C), C ∈ M(l, l, C), k + l = n, consist of Jordan blocks corresponding to all eigenvalues of A with positive and nonpositive real parts. Let α = min{Re λ; λ ∈ σ(B)}. Identifying Cn with the Cartesian product of Ck and Cl , we denote by z1 (·), z2 (·) and h1 , h2 projections of the solution z(·) and the function h on Ck and Cl , respectively. Define v1 (t) = |z1 (t)|2 , Then for t ∈ I

v2 (t) = |z2 (t)|2 ,

t ∈ [0, τ ) = I.

104

7 Stability and stabilizability

d v1 (t) = 2Re Bz1 (t), z1 (t) + 2Re h1 (t, z), z1 (t), dt d v2 (t) = 2Re Cz2 (t), z2 (t) + 2Re h2 (t, z), z2 (t). dt It follows from Lemma 7.2 and the Schwartz inequality that d 1/2 v1 (t) ≥ 2(α − γ)v1 (t) − 2v1 (t)|h1 (t, z(t))|, dt d 1/2 v2 (t) ≤ 2γv2 (t) + 2v2 (t)|h2 (t, z(t))|, t ∈ I. dt We can assume that α > γ > 0. For arbitrary ε > 0 there exists δ > 0, compare (7.13), such that for x ∈ Cn , |x| < δ, |h1 (t, x)| ≤ ε|x|,

|h2 (t, x)| ≤ ε|x|.

Assume that |z(0)| < δ and let τ¯ = inf{t ∈ [0, τ ); |z(t)| ≥ δ}. For t ∈ I¯ = [0, τ¯) |h1 (t, z(t))| ≤ ε(v1 (t) + v2 (t))1/2 , |h2 (t, z(t))| ≤ ε(v1 (t) + v2 (t))1/2 , 1/2

This and the inequality (v1

1/2

+ v2 ) ≤

¯ t ∈ I.

√ 2(v1 + v2 )1/2 imply

d (v1 (t) − v2 (t)) ≥ 2(α − γ)v1 (t) − 2γv2 (t) dt 1/2 1/2 − ε(v1 (t) + v2 (t))(v1 (t) + v2 (t))1/2 √ ≥ 2(α − γ)v1 (t) − 2γv2 (t) − 2ε 2(v1 (t) + v2 (t)),

¯ for t ∈ I.

Selecting ε > 0 and γ > 0 properly, we can find c > 0 such that for all t ∈ I¯ d (v1 (t) − v2 (t)) ≥ c(v1 (t) − v2 (t)). dt By Theorem 7.1 v1 (t) − v2 (t) ≥ ect (v1 (0) − v2 (0)),

¯ t ∈ I.

Therefore, if v1 (0) − v2 (0) = |z1 (0)|2 ≥ |z2 (0)|2 then τ¯ < +∞ and |z(t)| ≥ δ for some t ∈ (0, τ ).

and

|z(0)| < δ,

(7.16)

7.3 Linearization

105

This way part (ii) of the theorem has been proved for the complex case. To show the final part of (ii), we apply the representation theorem from §I.2.1. Since the matrix A is real, one can find invertible matrices P0 ∈ M(n, n),  P1 0 −1 ˜ P1 ∈ M(k, k; C), P2 ∈ M(l, l; C) such that A = P AP and P = , 0 P2 P0 is of the form (7.15). It is therefore clear, for  arbitrary δ > 0, one can z (0) 1 find x ∈ Rn such that (7.16) holds for = P x. The proof of (ii) is z2 (0) complete. 2 Exercise 7.1. Find a matrix A ∈ M(n, n) with ω(A) = 0, n ≥ 3 and vectors x1 , x2 , x3 ∈ Rn , such that lim |eAt x1 | = +∞,

t↑+∞

0 < lim |eAt x2 | < +∞, t↑+∞

lim |eAt x3 | = 0.

t↑+∞

Exercise 7.2. A continuous mapping f : Rn −→ Rn is called dissipative if f (x) − f (y), x − y ≤ 0

for arbitrary x, y ∈ Rn ,

(7.17)

where ·, · is the scalar product on Rn . Show that if f is a dissipative mapping then an arbitrary maximal solution of the equation z˙ = f (z), z(0) = x, (7.18) is defined on [0, +∞), and for arbitrary x ∈ Rn equation (7.12) has exactly one solution z x (t), t ≥ 0. Moreover |z x (t) − z y (t)| ≤ |x − y| for

t ≥ 0,

x, y ∈ Rn .

(7.19)

Conversely, if f : Rn −→ Rn is a continuous mapping and equation (7.18) has for arbitrary x ∈ Rn exactly one solution z x (·) for which (7.19) holds, then f is dissipative. Hint. If z(·) is a solution to (7.19) defined on [0, τ ) then function v(t) = |z(t)|2 , t ∈ [0, τ ), satisfies inequality v˙ ≤ |f (0)|v 1/2 on [0, τ ). For two arbitrary solutions z1 (t), t ∈ [0, τ1 ), and z2 (t), t ∈ [0, τ2 ), of (7.18), investigate the monotonic character of the function |z1 (t) − z2 (t)|2 , t ∈ [0, τ1 ∧ τ2 ).

§7.3. Linearization Let us consider a differential equation: z˙ = f (z),

z(0) = x ∈ Rn .

(7.20)

Assume that f (¯ x) = 0 or equivalently that x ¯ is an equilibrium state for (7.20). We say that the state x ¯ is exponentially stable for (7.20) if there exist ω < 0,

106

7 Stability and stabilizability

M > 0, δ > 0 such that arbitrary maximal solution z(·) of (7.20), with the initial condition z(0), |z(0) − x ¯| < δ, is defined on [0, +∞) and satisfies |z(t) − x ¯| ≤ M eωt |z(0) − x ¯|,

t ≥ 0.

(7.21)

The infimum of all those numbers ω < 0 for which (7.21) holds will be called the exponent of x ¯. The next theorem gives an effective characterization of exponentially stable equilibria and their exponents. Theorem 7.3. Assume that a continuous function f is differentiable at an equilibrium state x ¯. Then x ¯ is exponentially stable for (7.20) if and only if the Jacobi matrix x) A = fx (¯ is stable. Moreover the exponent of x ¯ is ω(A). Proof. We can assume, without any loss of generality, that x ¯ = 0. Define h(x) = f (x) − Ax,

x ∈ Rn .

It follows from the assumptions that |h(x)| −→ 0, |x|

when

|x| → 0.

If matrix A is stable then the equation z˙ = f (z) = Az + h(z),

z(0) = x,

satisfies the conditions of Theorem 7.2 (i). Therefore the state 0 is exponentially stable and its exponent is at least equal to ω(A). The following lemma completes the proof of the theorem. Lemma 7.3. Assume that condition (7.21) holds for some ω < 0, δ > 0, M > 0 and x ¯ = 0. Then for arbitrary N > M and γ > ω |eAt | ≤ N eγt ,

t ≥ 0.

(7.22)

Proof. We show first that (7.22) holds for some and then for arbitrary N > M . We can assume that γ < 0. Let η be an arbitrary number from the interval (ω, γ) and let y(t) = e−ηt z(t), Then We have

t ≥ 0.

y(t) ˙ = (−η + A)y(t) + e−ηt h(eηt y(t)),

t ≥ 0.

7.3 Linearization

sup t≥0

107

|e−ηt h(eηt x)| |h(eηt x)| = sup −→ 0, |x| |eηt x| t≥0

|x| −→ 0,

if

therefore, by Theorem 7.2 (ii), ω(−ηI + A) ≤ 0 and consequently ω(A) ≤ η. There exists N1 > 0 such that, for arbitrary t ≥ 0, |eAt | ≤ N1 eγt . Moreover, for the solution z(·) of (7.20),  z(t) = eAt z(0) +

t

eA(t−s) h(z(s)) ds,

0

t ≥ 0,

and, assuming that x = z(0),  |eAt x| ≤ |z(t)| +

0

t

|eA(t−s) | |h(z(s))| ds,

t ≥ 0.

For arbitrary ε > 0 there exists δ1 > 0 such that if |x| = |z(0)| < δ1 then |h(z(t))| ≤ ε|z(t)|, Hence, for |x| < δ1



t ≥ 0.

t



|e x| ≤ M e |x| + M N1 ε e e 0   1 eγt |x| ≤ M eωt + M N1 ε γ−ω   M N1 ε γt ≤ M+ e |x|, t ≥ 0. γ−ω At

ωt

γ(t−s) ωs

ds |x|

The number ε > 0 was arbitrary and the lemma follows.

2

Theorem 7.3 reduces the problem of stability of an equilibrium state x ¯ ∈ Rn to the question of stability of the matrix A obtained by the linearization of the mapping f at x ¯. The content of Theorem 7.3 is called the linearization method or the first method of Lyapunov of stability theory. The practical algorithm due to Routh allowing one to determine whether a given matrix A is stable was given in §I.7.3. Exercise 7.3. Watt’s regulator (see Example 0.3) is modelled by the equations x˙ = a cos y − b, 2

x(0) = x0 ,

˙ y¨ = cx sin y cos y − d sin y − ey,

y(0) = y0 , y(0) = z0 ,

where a, b, c, d, e are some positive constants, a > b. Show that there exists exactly one equilibrium state for the system, with coordinates x ¯ > 0,

108

7 Stability and stabilizability

√ √ y¯ ∈ (0, 12 π), z¯ = 0. Prove that if e da > 2 cb3 then the equilibrium state √ √ is exponentially stable and if e da < 2 cb3 then it is not stable in the Lyapunov sense; see §7.4. Hint. Apply Theorems 7.3, 7.2 and I.2.4.

§7.4. The Lyapunov function method We say that a state x ¯ is stable in the Lyapunov sense for (7.20), or that x ¯ is Lyapunov stable, if for arbitrary r > 0 there is δ > 0 such that if |z(0)− x ¯| < δ then |z(t) − x ¯| < r for t ≥ 0. It is clear that exponentially stable equilibria are stable in the Lyapunov sense. Assume that a mapping f : Rn −→ Rn is continuous and G ⊂ Rn is a neighbourhood of a state x ¯ for which f (¯ x) = 0. A real function V differentiable on G is said to be a Lyapunov function at the state x ¯ for equation (7.20) if V (¯ x) = 0, V (x) > 0 for x ∈ G, x = x ¯, n  ∂V Vf (x) = (x)f j (x) ≤ 0, x ∈ G. ∂x j j=1

(7.23) (7.24)

The function Vf defined by (7.24) is called the Lyapunov derivative of V with respect to f . Theorem 7.4. (i) If there exists a Lyapunov function at x ¯ for (7.20), then the state x ¯ is Lyapunov stable for (7.20). (ii) If x ¯ is exponentially stable at x ¯ and f is differentiable at x ¯, then there exists a Lyapunov function at x ¯ for (7.20), being a quadratic form. For the proof of part (i) it is convenient to introduce the concept of an invariant set. We say that a set K ⊂ Rn is invariant for (7.20), if for arbitrary solution z(t), t ∈ [0, r), then z(0) ∈ K, z(t) ∈ K for all t ∈ [0, γ). We will need the following lemma: Lemma 7.4. If for an open set G0 ⊂ G and for an α > 0 the set K0 = {x ∈ G0 ; V (x) ≤ α} is closed, then it is also invariant for (7.20). Proof. Assume that the lemma is not true and that there exists a point x ∈ K0 and a solution z(t), t ∈ [0, τ ), of the equation (7.20), with values in G, z(0) = x such that for some s ∈ [0, τ ), z(s) ∈ K0c .

7.4 The Lyapunov function method

109

Let t0 = inf{s ≥ 0; s < τ, z(s) ∈ K0c } < +∞. Since the set K0 is closed z(t0 ) ∈ K0 and t0 < τ . For some s ∈ (t0 , τ ), V (z(s)) > V (z(t0 )). Since dV (z(t)) = Vf (z(t)) ≤ 0 for t < τ, dt the function V (z(·)) is nonincreasing on [0, τ ), a contradiction.

2

Proof of the theorem. x, r) = (i) Let r > 0 be a number such that the closure of G0 = B(¯ {x; |x − x ¯| < r} is contained in G. Let β be the minimal value of the function V on the boundary S of the ball B(¯ x, r). Let us fix α ∈ (0, β) and define x, r); V (x) ≤ α}. K0 = {x ∈ B(¯ Let x ˆ be the limit of a sequence (xm ) of elements from K0 . Then V (ˆ x) ≤ α < β and consequently x ˆ ∈ B(¯ x, r) and x ˆ ∈ K0 . The set K0 is closed and, by Lemma 7.4, is also invariant. Since the function V is continuous, V (¯ x) = 0 and there exists δ ∈ (0, r) such that V (x) ≤ α for x ∈ B(¯ x, δ). This implies (i). x) is stable by Theorem 7.3. It follows from The(ii) The matrix A = fx (¯ orem I.2.7 that there exists a positive definite matrix Q satisfying the Lyapunov equation A∗ Q + QA = −I. To simplify the notation we set x ¯ = 0 and define V (x) = Qx, x,

h(x) = f (x) − Ax,

x ∈ Rn .

It is clear that the function V satisfies (7.23). To show that (7.24) holds as well remark that Vf (x) = 2Qx, f (x) = 2Qx, Ax + h(x) = (A∗ Q + QA)x, x + 2Qx, h(x) ≤ −|x|2 + 2|Q| |x| |h(x)|,

x ∈ Rn .

Since f is differentiable at 0, there exists a number δ > 0 such that |h(x)| ≤ 1 4|Q| |x|, provided |x| < δ. Consequently 1 Vf (x) < − |x|2 , 2

for

|x| 0 such that, for arbitrary maximal solution z(t), t ∈ [0, r), of the equation (7.20), one can find t0 ∈ [0, r) such that, for all t ∈ [t0 , τ ), |z(t)| ≥ r. Hint. The matrix −A is stable. Follow the proof of part (ii) of Theorem 7.4. Exercise 7.5. Let a matrix A ∈ M(n, n) be stable and a continuous mapping F : Rn −→ Rn be bounded. Then, for arbitrary solution z(t), t ∈ [0, r), of the equation z˙ = Az + F (z), there exists M > 0 such that |z(t)| ≤ M for t ∈ [0, r). Hint. Examine the behaviour of V (x) = Qx, x, z ∈ Rn , where A∗ Q + QA = −I, along the trajectories of the equation. A closed invariant set K for the equation (7.20) is called nonattainable if one cannot find a solution z(t), t ∈ [0, r), of (7.20) such that z(0) ∈ / K and z(t) ∈ K for some t ∈ [0, τ ). Exercise 7.6. (i) Assume that the mapping F : Rn −→ Rn satisfies the Lipschitz conx) = 0. Show that the set dition in a neighbourhood of a point x ¯ ∈ Rn , f (¯ K = {¯ x} is nonattainable for (7.20). (ii) Construct a mapping f : Rn −→ Rn satisfying the Lipschitz condition and an invariant compact set K for (7.20) which is not nonattainable. (iii) Assume that f : Rn −→ Rn satisfies the Lipschitz condition and K is a closed invariant set for (7.20). Let moreover for arbitrary y ∈ K there be x ∈ K and a solution z(t), t ≥ 0, of (7.20) such that z(0) = x and z(s) = y for some s > 0. Prove that K is nonattainable for (7.20). (iv) Let z(t), t ≥ 0, be a solution of the following prey-predator equation in R2 : z˙1 = αz1 − βz1 z2 , z˙2 = −γz2 + δz1 z2 , in which α, β, γ, δ are positive numbers. Show that if z1 (0) > 0, z2 (0) > 0 then z1 (t) > 0, z2 (t) > 0 for all t > 0.

§7.5. La Salle’s theorem Besides exponential stability and stability in the sense of Lyapunov it is also interesting to study asymptotic stability. An equilibrium state x ¯ for (7.20) is asymptotically stable if it is stable in the sense of Lyapunov and there exists δ > 0 such that, for any maximal solution z(t), t ∈ [0, r), of (7.20), with |z(0) − x ¯| < δ, ¯. one has τ = ∞ and lim z(t) = x t↑+∞

(7.25)

7.5 La Salle’s theorem

111

For the linear system on R2 z˙1 = z2 ,

z˙2 = −z1

(7.26)

the origin is stable in the sense of Lyapunov but it is not asymptotically stable. The function V (x) = |x|2 , x ∈ R2 , is a Lyapunov function for (7.26) and Vf (x) = 0, x ∈ R2 . Hence the existence of a Lyapunov function does not necessarily imply asymptotic stability. Additional conditions, which we discuss now, are needed. We will limit our considerations to equations (7.26) with the function f : Rn −→ Rn satisfying the local Lipschitz condition. Let us recall that a function f : G −→ Rn satisfies the Lipschitz condition on a set G ⊂ Rn if there exists c > 0 such that |f (x) − f (y)| ≤ c|x − y| for all x, y ∈ G. If the Lipschitz condition is satisfied on an arbitrary bounded subset of Rn then we say that it is satisfied locally. Exercise 7.7. If a function f : Rn −→ Rn satisfies the local Lipschitz condition then for arbitrary r > 0 the function ⎧ (x),  for |x| ≤ r, ⎨f  fˆ(x) = x ⎩f r , for |x| ≥ r, |x| satisfies the Lipschitz condition on Rn . It follows from Theorems 6.1 and 6.2 that if a mapping f : Rn −→ Rn satisfies the local Lipschitz condition then for arbitrary x ∈ Rn there exists exactly one maximal solution z(t, x) = z x (t), t ∈ [0, τ (x)), of the equation (7.20). The maximal solution (see Exercise 7.7 and Theorem 7.2) depends continuously on the initial state: If lim xk = x and t ∈ [0, τ (x)) k↑+∞

then for all large k, τ (xk ) > t and lim z(t, xk ) = z(t, x). k↑+∞

If τ (x) = +∞, then the orbit O(x) and the limit set K (x) of x are given by the formulae O(x) = {y ∈ Rn ; y = z(t, x), t ≥ 0}, K(x) = {y ∈ Rn ; lim z(tk , x) = y for a sequence tk ↑ +∞}. k

It is clear that the limit set K(x) is contained in the closure of O(x). Lemma 7.5. If O(x) is a bounded set then (i) the set K(x) is invariant for (7.20), (ii) the set K(x) is compact, (iii) lim (z(t, x), K(x)) = 0, where (·, ·) denotes the distance between a t↑+∞

point and a set.

112

7 Stability and stabilizability

Proof. (i) Assume that y ∈ K(x) and t > 0. There exists a sequence tm ↑ +∞ such that ym = z(tm x) −→ y. The uniqueness and the continuous dependence of solutions on initial data imply z(t + tm , x) = z(ym , t) −→ z(y, t),

when

m −→ +∞.

Hence z(y, t) ∈ K(x). (ii) Assume that (ym ) is a sequence of elements from K(x) converging to y ∈ Rn . There exists a sequence tm ↑ +∞ such that |z(tm , x) − ym | −→ 0. Consequently z(tm , x) −→ y and y ∈ K(x). (iii) Assume that (z(t, x), K(x)) → 0 when t ↑ +∞. Then there exists ε > 0 and a sequence (tm ), tm ↑ +∞, such that |z(tm , x) − y| ≥ 0 for arbitrary y ∈ K(x),

m = 1, 2, . . . .

(7.27)

Since the set O(x) is bounded, one can find a subsequence (t˜k ) of (tm ) and an element y˜ ∈ G such that z(t˜k , x) −→ y˜. Hence y˜ ∈ K(x) and, by (7.27), |y − y˜| ≥ ε for arbitrary y ∈ K(x). In particular for y = y˜, 0 = |˜ y − y˜| ≥ ε, a contradiction. This way the proof of (iii) is complete. 2 We are now in a position to prove the La Salle theorem. Theorem 7.5. Assume that a mapping f : Rn −→ Rn is locally Lipschitz, f (¯ x) = 0 and there exists a Lyapunov function V for (7.20) in a neighbourhood G of x ¯. If, for an x ∈ G, O(x) ⊂ G, then K(x) ⊂ {y ∈ G; Vf (y) = 0}. Proof. Let v(t) = V (z(t, x)), t ≥ 0. Then v(·) is a nonnegative, decreasing function and lim v(t) = vˆ ≥ 0. t↑+∞

If y ∈ K(x) then for a sequence tm ↑ +∞, z(tm , x) −→ y. Thus V (y) = vˆ. It follows from Lemma 7.5(i) that for an arbitrary y ∈ K(x) and t ≥ 0, V (z(t, y)) = V (y). Hence 1 Vf (y) = lim (V (y) − V (z(t, y))) = 0. t↑0 t

2

We will now formulate a version of Theorem 7.5 useful in specific applications. Let L = {y ∈ G; Vf (y) = 0} (7.28) and K be the maximal, invariant for (7.20), subset of L.

(7.29)

7.6 Topological stability criteria

113

Since the sum of invariant subsets for (7.20) is also invariant for (7.20) as well as the set {¯ x} is invariant, therefore K is a well-defined nonempty subset of G. Theorem 7.6. (i) Under conditions of Theorem 7.5, (z(t, x), K) −→ 0,

for t ↑ +∞.

(ii) If, in addition, Vf (y) < 0 for y ∈ G \ {¯ x}, then x ¯ is an asymptotically stable equilibrium for (7.20). Proof. (i) Since K(x) ⊂ K, it is enough to apply Lemma 7.5 (iii). (ii) In this case L = {¯ x} = K.

2

Exercise 7.8. Show that maximal solutions of the Liénard equation x ¨ + (2a + (x) ˙ 2 )x˙ + ν 2 x = 0,

ν = 0,

exist on [0, +∞). Find the unique equilibrium state for this system and prove that (i) if a ≥ 0, then it is asymptotically stable, and (ii) if a < 0, then it is not stable. d V (x, x) ˙ ≤ |a|2 . To Hint. Let V (x, y) = 12 (ν 2 x2 + y 2 ), x, y ∈ R. Show that dt prove (i) apply Theorem 7.5. Base the proof of (ii) on Theorem 7.3.

§7.6. Topological stability criteria We proceed now to stability criteria based on the concept of the degree of a vector field. They will be applied to stabilizability of control systems in §7.8. Let S be the unit sphere, i.e. the boundary of the unit ball B(0, 1) = {y ∈ Rn ; |y| < 1}. With an arbitrary continuous mapping F : S −→ S one associates, in topology, an integer called the degree of F , denoted by deg F , see N. Dunford and J. Schwartz [32]. Intuitively, if n = 1, the degree of F is the number of times the image F (x) rotates around S when x performs one oriented rotation. We will not give here a precise definition of the degree of a map but gather its basic properties in the following proposition. For the proofs we refer to I.V. Girsanov [41]. Let us recall that if F0 and F1 are two continuous mappings from S into S and there exists a continuous function H : [0, 1] × S −→ S such that H(0, x) = F0 (x),

H(1, x) = F1 (x),

x ∈ S,

then F0 and F1 are called homotopic and H is a homotopy which deforms F0 to F1 . If F0 and F1 are homotopic we symbolically write F0  F1 .

114

7 Stability and stabilizability

Proposition 7.1. (i) For arbitrary continuous mappings F0 , F1 from S onto S: If

F0  F 1 ,

then

deg F = deg F1 .

(ii) If a mapping F is constant: F (x) = x ˜ for some x ˜ ∈ S and all x ∈ S, then deg F = 0. (iii) If F is an antipodal mapping: F (x) = −x, x ∈ S, then deg F = (−1)n . Assume that f is a continuous mapping from a ball B(¯ x, δ) ⊂ Rn onto Rn , f (¯ x) = 0 and f (x) = 0 for x = x ¯. For arbitrary r ∈ (0, δ) define a mapping Fr from S onto S by the formula Fr (x) =

f (rx + x ¯) , |f (rx + x ¯)|

x ∈ S.

If 0 < r0 ≤ r1 < δ then the transformation H(s, x) =

¯) f ((r0 + s(r1 − r0 ))x + x , |f ((r0 + s(r1 − r0 ))x + x ˜)|

s ∈ [0, 1],

x ∈ S,

defines a homotopy deforming Fr0 to Fr1 . By Proposition 7.1(i), all the map¯ pings Fr , r ∈ (0, δ), have the same degree, which is called the index of f at x and denoted by Indx¯ f . Thus Indx¯ f = deg Fr ,

0 < r < δ.

The following theorem concerns asymptotically stable equilibria and attracting points. A point x ¯ is attracting for (7.20) if an arbitrary maximal ¯. solution z(·) of (7.20) is defined on [0, +∞) and lim z(t) = x t↑+∞

Exercise 7.9. Construct an equation with an attracting point which is not asymptotically stable. Hint. See E.A. Coddington and N. Levinson [18, page 59, picture 1.7.9]. Theorem 7.7. If a point x ¯ is either asymptotically stable or attracting for equation (7.20) with the right-hand side satisfying the local Lipschitz condition then Indx¯ f = (−1)n . Proof. Without any loss of generality we can assume that x ¯ = 0. It follows from the assumptions that f (0) = 0 and f (x) = 0 for all x = 0 in a neighbourhood of 0. We show first that there exist numbers R > r > 0 and t0 > 0 such that for all solutions z(·, x), |x| = R, of (7.20) |z(t0 , x)| < r.

7.6 Topological stability criteria

115

Moreover, in the case of an asymptotically stable point, the numbers R > 0 and r > 0 can be chosen arbitrarily small. It follows from the assumptions of the theorem that there exist numbers R1 > r1 > 0 such that for all x ∈ Rn , |x| = R1 : Tr1 (x) = inf{t ≥ 0; |z(t, x)| ≤ r1 } < +∞. Fix r2 ∈ (r1 , R1 ). If |x| = R1 , then Tr2 (x) < Tr1 (x). Since the solutions of (7.20) depend continuously on initial data, for arbitrary x, |x| = R1 one can find δ > 0 such that Tr2 (y) < Tr1 (x) < +∞ provided |y| = R1 and |y − x| < δ. Consequently the function Tr2 (·) is locally bounded on a compact set {y; |y| = R1 } and therefore it is globally bounded: Tˆ = sup{Tr2 (y); |y| = R1 } < +∞.

(7.30)

If O is an asymptotically stable equilibrium, the numbers R = R1 > r2 > r1 and r ∈ (r2 , R) can be chosen arbitrarily small and such that |z(t, x)| < r

for

t≥0

and

|x| = r2 .

It is therefore enough to take as to an arbitrary number greater than Tˆ. If O is an attracting point then for arbitrary R1 > 0 the set-theoretic sum K of all orbits O(x), |x| = R1 , is bounded and invariant. To see this choose a number T > T˜. Then K ⊂ {y; |y| ≤ r2 } ∪ {y; y = z(s, x), s ≤ T, |x| = R1 }. Since the sets K1 and K2 are compact, the boundedness of K follows. It is also obvious that the set K is invariant. Let R be an arbitrary number such that K ⊂ {y; |x| < R} and let r = sup{|y|; y ∈ K}. Then Tˆ = sup{TR1 (x); |x| = R} < +∞. It is therefore clear that it is enough to choose as t0 an arbitrary number greater than Tˆ. The proof of the theorem follows immediately from the following lemma. Lemma 7.6. Assume that for some R > r > 0, t0 > 0, lim |z(t, x)| < R,

t↑+∞

|z(t0 , x)| < r,

where |x| ≤ R,

where |x| = R.

Then Ind0 f = (−1)n . Proof of the lemma. Define for t ∈ (0, t0 ]

(7.31) (7.32)

116

7 Stability and stabilizability

Ft (y) =

z(t, Ry) − Ry , |z(t, Ry) − Ry|

y ∈ S.

(7.33)

It follows from (7.31) that solutions z(·, x), |x| = R, are not periodic functions. Consequently the formula (7.33) defines continuous transformations from S onto S which are clearly homotopic. Taking into account the following homotopy: H(α, y) =

αz(t0 , Ry) − Ry , |αz(t0 , Ry) − Ry|

α ∈ [0, 1], y ∈ S,

we see that the transformation H(1, ·) = Ft0 is homotopic to the antipodal map H(0, y) = −y, y ∈ S. So deg Ft0 = (−1)n . On the other hand, 1 t (z(t, Ry) − Ry) −→ f (Ry) uniformly on S, when t ↓ 0 so for sufficiently small t > 0, Ft  f (R·)/|f (R·)|. Consequently by Proposition 7.1 the result follows. 2 As a corollary from Theorem 7.7 we obtain that the vector field f defining a stable system is “rich in directions”. Theorem 7.8. Assume that the conditions of Theorem 7.7 are satisfied. Then for sufficiently small r > 0, the transformation Fr : S −→ S given by Fr (x) =

f (rx + x ¯) , |f (rx + x ¯)|

x ∈ S, r > 0,

transforms S onto S. In addition, the mapping f transforms an arbitrary neighbourhood of x ¯ onto a neighbourhood of 0. Proof. It follows from Theorem 7.7 that deg Fr = 0 for sufficiently small ¯∈S r > 0. Assume that the transformation Fr is not onto S and that a point x is not in the image of Fr . One can easily construct a homotopy deforming S \{¯ x} to the antipodal point x ¯. Consequently the transformation Fr is homotopic to a constant transformation and, by Proposition 7.1.(ii), deg Fr = 0, a contradiction. Thus Fr is a transformation onto S. To prove the final statement of the theorem assume that x ¯ = 0. Let r > 0 be a number such that f (x) = 0 for all x ∈ B(0, r) \ {0}. If 0 < δ < min(|f (x)|, |x| = r) then for arbitrary λ ∈ [0, 1] and y ∈ Rn , |y| < δ, |λ(f (x) − y) + (1 − λ)f (x)| = |f (x) − λy| ≥ |f (x)| − |y| ≥ 0. Let us fix y ∈ Rn , 0 < |y| < δ and define H(λ, x) =

(1 − λ)f (rx) + λ(f (rx) − y) , |(1 − λ)f (rx) + λ(f (rx) − y)|

λ ∈ [0, 1], x ∈ S.

(rx) Then H is a homotopy deforming G(x) = |ff (rx)| to Gr (x) = |ff (rx)−y (rx)−y| , x ∈ S. Since deg G = 0, deg Gr = 0 as well. Assume that for arbitrary s ∈ (0, r]

7.7

Exponential stabilizability and the robustness problem

117

and arbitrary x, |x| ≤ s, f (x) = y. Then the transformation Gr is homotopic to all, defined analogically, transformations Gs , s ∈ (0, r]. Since f (0) = 0 and f is continuous, for sufficiently small s > 0 the image Gs is included in a given in advance neighbourhood of −y/|y|. Hence for such s > 0 the transformations Gs cannot be onto S and therefore deg Gs = 0, contrary to 2 deg Gs = deg Gr = 0. This finishes the proof.

§7.7. Exponential stabilizability and the robustness problem In this section we begin our discussion of the stabilizability of nonlinear systems y˙ = f (y, u), y(0) = x. (7.34) Let U ⊂ Rm be the set of control parameters and assume that the continuous x, u ¯) = 0 for some x ¯ ∈ Rn function f : Rn × U −→ Rn is such that f (¯ and u ¯ ∈ U . Then a feedback is defined as an arbitrary continuous function x) = u ¯. Feedbacks determine closed-loop systems v : Rn −→ U such that v(¯ z˙ = g(z), where

z(0) = x ∈ Rn ,

g(x) = f (x, v(x)),

x ∈ Rn .

(7.35) (7.36)

A feedback v is stabilizing if x ¯ is a stable equilibrium for (7.35). A feedback v stabilizes (7.34) exponentially or asymptotically or in the sense of Lyapunov if the state x ¯ is, respectively, exponentially stable or asymptotically stable or stable in the sense of Lyapunov for the closed-loop system. If for system (7.34) there exists a feedback with one of the specified properties, then system (7.34) is called stabilizable exponentially, asymptotically or in the sense of Lyapunov. The stabilization problem consists of formulating checkable conditions on the right-hand side of (7.34) implying stabilizability and of constructing stabilizing feedback. To simplify notation we assume that x ¯ = 0, u ¯ = 0. We first examine exponential stabilizability and restrict considerations to the case when 0 ∈ Rm is an interior point of U and the function f as well as admissible feedbacks are differentiable, respectively, at (0, 0) ∈ Rn × Rm and 0 ∈ Rn . According to our definitions, system (7.34) is exponentially stabilizable if there exists a feedback v and numbers ω < 0, δ > 0 and M such that for arbitrary solution z(t), t ∈ [0, τ ), of (7.34), with |z(0)| < δ, |z(t)| ≤ M eωt |z(0)|,

t ∈ [0, τ ).

(7.37)

The infimum of all ω < 0 from (7.37) is called the exponent of stabilizability of (7.34). Let us recall that the linearization of the system (7.34) is of the form

118

7 Stability and stabilizability

(7.38)

y˙ = Ay + Bu, where A = fx (0, 0),

(7.39)

B = fu (0, 0).

The following theorem gives a complete solution to the question of exponential stabilizability. Theorem 7.9. (i) System (7.34) is exponentially stabilizable if and only if the linear system (7.38)–(7.39) is stabilizable. (ii) An exponentially stabilizing feedback can always be found to be linear. (iii) System (7.34) and its linearization (7.38)–(7.39) have the same stabilizability exponents. Proof. Assume that v is a feedback such that (7.37) holds. It follows from the differentiability of f and v that g(x) = (fx (0, 0) + fu (0, 0)vx (0))x + h(x), where

|h(x)| −→ 0, |x|

x ∈ Rn ,

if |x| −→ 0.

By Lemma 7.3 the matrix A˜ = fx (0, 0) + fu (0, 0)vx (0) is stable, and, for arbitrary N > M and ω < γ < 0, ˜

|eAt | ≤ N eγt ,

t ≥ 0.

(7.40)

Therefore the linearization (7.38)–(7.39) is exponentially stabilizable by the linear feedback x −→ vx (0)x and the stabilizability exponent of the linearization is not greater than the stabilizability exponent of (7.34). Assume that a feedback v is of the form v(x) = Kx, x ∈ K, and that (7.40) holds for the matrix A˜ = fx (0, 0) + fu (0, 0)K. Then the derivative ˜ By Theorem 7.3 the state 0 has of g˜(·) = f (·, v(·)) at 0 is identical with A. identical exponent stability with respect to the following linear and nonlinear systems: ˜ and z˙ = g˜(z). z˙ = Az Consequently the stabilizability exponent for (7.34) is not greater than the stabilizability exponent of its linearization, and a linear feedback which stabilizes the linearization of (7.34) also stabilizes the system (7.34). This way the proof of the theorem is complete. 2 It follows from the proof of Theorem 7.9 that Corollary 7.2. If the pair (fx (0, 0), fu (0, 0)) is stabilizable and for a matrix K the matrix fx (0, 0) + fu (0, 0)K is stable then the linear feedback v(x) = Kx, x ∈ Rn , stabilizes (7.34) exponentially.

7.7

Exponential stabilizability and the robustness problem

119

Corollary 7.3. If the pair (fx (0, 0), fu (0, 0)) is not stabilizable then for arbitrary (differentiable at 0) feedback there exist trajectories of the closed-loop system which start arbitrarily close to 0 but do not tend to 0 exponentially fast. We will now discuss the related problem of the robustness of exponentially stabilizable systems. Assume that (7.37) holds for the closed-loop system (7.35)–(7.36) and let r be a transformation from Rn into Rn of class C 1 and such that r(0) = 0. We ask for what “perturbations” r the system z˙ = g(z) + r(z),

z(0) = x,

(7.41)

remains exponentially stable. The following lemma holds. Lemma 7.7. Assume that lim

x−→0

|r(x)| |ω| < . |x| M

(7.42)

Then system (7.41) is exponentially stable. Proof. Let A˜ = gx (0), C = rx (0). The linearization of (7.41) is of the form z˙ = (A˜ + C)z. Note that |C| ≤ lim

x−→0

|r(x)| |x|

(7.43)

and therefore |C|
ω and N > M are numbers sufficiently close to ω and M respectively. By Lemma 7.3 ˜

|eAt | ≤ N eβt

for t ≥ 0.

It follows from (7.44) and (7.45) that for a solution z(·) of (7.43) ˜

z(t) = eAt z(0) +



t

˜

eA(t−s) Cz(s) ds,

0

t ≥ 0, x ∈ Rn ,

and  t eβ(t−s) |z(s)| ds, |z(t)| ≤ N eβt |z(0)| + N |C| 0  t e−βs |z(s)| ds, t ≥ 0. e−βt |z(t)| ≤ N |z(0)| + N |C| 0

(7.45)

120

7 Stability and stabilizability

Taking into account Lemma 7.1 we see that e−βt |z(t)| ≤ N eN |C|t |z(0)|, |z(t)| ≤ N e(β+N |C|)t |z(0)|,

t ≥ 0.

Since β + N |C| < 0, system (7.43) is exponentially stable. The supremum |ω|/M , where ω and M are arbitrary numbers from the definition (7.37) of exponential stabilizability, will be called the robustness index of system (7.34). The following theorem gives an upper bound on the robustness index. In its formulation, δ = sup{(x, Im A); x ∈ Im B, |x| = 1}, where Im A and Im B are the images of the linear transformations A = fx (0, 0) and B = fu (0, 0). Theorem 7.10. If, for a feedback v, relation (7.37) holds, then |ω| |A| ≤ . M δ Proof. Taking into account Lemma 7.3, we can assume that system (7.34) and the feedback v are linear, v(x) = Kx, x ∈ Rn . The theorem is trivially true if δ = 0. Assume that δ = 0 and let x ¯ be a vector such that A¯ x + Bu = 0 ¯(·) by the formula for all u ∈ Rm . Define the control u u ¯(t) = K(y(t) − x ¯), where

y˙ = Ay + B u ¯,

t ≥ 0,

y(0) = x ¯.

Let y¯(t) = y(t) − x ¯ and A˜ = A + BK. Then y¯˙ = (A + BK)¯ y + A¯ x ˜y + A¯ = A¯ x, y¯(0) = 0. It follows from (7.37) and (7.46) that

 t

M ˜ A(t−s)

|y(t) − x ¯| = |¯ |A¯ x|, y (t)| = e |A¯ x| ds

≤ |ω| 0

(7.46)

t ≥ 0.

(7.47)

On the other hand, repeating the arguments from the first part of the proof of Theorem 6.7, we obtain that for an arbitrary control u(·) and the corresponding output y u (·) satisfying y˙ = Ay + Bu, one has

y(0) = x ¯,

7.8

Necessary conditions for stabilizability

121

 sup |y u (t) − x ¯| ≥ t≥o

 γ |a| − |A|−1 , |a|

(7.48)

where a is the vector of the form A¯ x + Bu with the minimal norm and γ ∈ (0, |a|2 ). Comparing formulae (7.47) and (7.48) we have   γ M |A¯ x| |A|. |a| − ≤ |a| |ω| Taking into account constraints on γ and x ¯ and the definition of δ we easily obtain the required estimate. 2 It follows from the theorem that the robustness index is in general finite. Hence, if all solutions of the closed-loop system tend to zero fast then some of the solutions will deviate far from the equilibrium on initial time intervals. Corollary 7.4. If Im A is not contained in Im B then |ω| |A| ≤ < +∞. M δ Corollary 7.5. The robustness index of (7.34) is bounded from above by |A|/δ.

§7.8. Necessary conditions for stabilizability We will now deduce necessary conditions for stabilizability. The following theorem is a direct consequence of the result from §7.6 and is concerned with systems (7.34), (7.35) with f and v satisfying local Lipschitz conditions. Theorem 7.11. If system (7.34) is asymptotically stabilizable then the mapping f (·, ·) transforms arbitrary neighbourhoods of (0, 0) ∈ Rn × Rm onto neighbourhoods 0 ∈ Rn . Proof. If v is a feedback stabilizing (7.34) asymptotically then the closedloop system (7.35)–(7.36) satisfies all the assumptions of Theorem 7.8. Hence the mapping x −→ f (x, v(x)) transforms an arbitrary neighbourhood of 2 0 ∈ Rn onto a neighbourhood of 0 ∈ Rn . Taking into account Theorem 7.11 one can show that for nonlinear systems local controllability does not imply, in general, asymptotic stabilizability. Such implication is of course true for linear systems, see Theorem I.2.9. Example 7.1. Consider again the system y˙ 1 = u, y˙ 2 = v, y˙ 3 = y1 v − y2 u,

  u ∈ U = R2 . v

122

7 Stability and stabilizability

We know, see §6.4, Example 6.3, that the system is locally controllable at an arbitrary point, in particular, at 0 ∈ R3 . Let us remark, ⎡ ⎤ however, that 0 the image of f (·, ·) does not contain vectors of the form ⎣ 0 ⎦, γ = 0, so the γ necessary condition for stabilizability is not satisfied. Hence the system is not asymptotically stabilizable. This is a surprising result as it concerns a system which differs only slightly from a linear one. The following theorem is also an immediate consequence of Theorem 7.8. Theorem 7.12. Assume that there exists a feedback which either asymptotically stabilizes (7.34) or for which 0 ∈ Rn is an attracting point for the closed-loop system (7.35)–(7.36). Then, for sufficiently small r > 0, transformations Fr : S × U −→ S given by Fr (x, u) =

f (rx, u) , |f (rx, u)|

x ∈ S, u ∈ U,

(7.49)

are onto S. Example 7.2. It is not difficult to show that the following system on R2 y˙ 1 = (4 − y22 )u,

y˙ 2 = y2 − e−y1 v, (7.50)    u with U = ∈ R2 ; u ≥ 0, −1 ≤ v ≤ 1 is controllable to 0 ∈ R2 from v an arbitrary state in R2 . Let us also remark that the first coordinate of the right side of (7.50) is nonnegative for state vectors close to 0 ∈ R2 . Therefore, for sufficiently small r > 0, transformation (7.49) cannot be onto S. Thus, although system (7.50) is exactly controllable to 0 ∈ R2 , one cannot find a feedback v such that the state 0 is attracting for the closed-loop system determined by v.

§7.9. Stabilization of the Euler equations We will now apply the obtained results to an analysis of an important control system described by the Euler equations from Example 0.2. Let us recall that the stated equation was of the form J ω˙ = S(ω)Jω + Bu, where

ω(0) ∈ R3 ,

(7.51)

7.9

Stabilization of the Euler equations

⎡

⎤

I1 0 0 J = ⎣ 0 I2 0 ⎦ , 0 0 I3 B = [b1 , b2 , b3 ],

123

⎤

⎡

0 ω3 −ω2 S(ω) = ⎣ −ω3 0 ω1 ⎦ , ω ∈ R3 , ω2 −ω1 0 ⎡ ⎤ b1i bi = ⎣ b2i ⎦ ∈ R3 , i = 1, 2, 3. b3i

The control set is U = R3 . We assume that Ii = 0, Ii = Ij , i, j = 1, 2, 3, i = j. Vectors bi , i = 1, 2, 3, will be called the control axes. We will look for conditions under which system (7.51) is stabilizable and for formulae defining stabilizing feedbacks v : R3 −→ R3 , v(0) = 0. The closed-loop system corresponding to v is given by the equation (7.52)

ω˙ = F (ω), where

F (ω) = J −1 S(ω)Jω + J −1 Bv(ω).

The question of exponential stabilizability has, in the present case, a simple solution. The linearization of (7.51) is given by ω˙ = J −1 Bu.

(7.53)

The pair (0, J −1 B) is stabilizable if and only if the matrix J −1 B is invertible, therefore, by Theorem 7.9, system (7.51) is exponentially stabilizable if and only if det B = 0. Let 1 1 W (ω) = ω ∗ Jω = (I1 ω12 + I2 ω22 + I3 ω33 ), ω ∈ R3 . 2 2 Then W is the kinetic energy of the system and the Lyapunov derivative WF of W , see (7.24), is given by WF (ω) = ω ∗ JJ −1 S(ω)Jω + ω ∗ JJ −1 Bv(ω) = det[Jω, ω, ω] + ω ∗ Bv(ω) = ω ∗ Bv(ω),

ω ∈ R3 .

If v ≡ 0 then WF ≤ 0 and W is a Lyapunov function for (7.52). By Theorem 7.4, the constant feedback v ≡ 0 stabilizes system (7.51) in the sense of Lyapunov. Let us remark that if a feedback v is given by v(ω) = −B ∗ ω, then

ω ∈ R3 .

WF (ω) = −ω ∗ BB ∗ ω = −|B ∗ ω|2 ≤ 0,

(7.54) ω ∈ R3

and W is a Lyapunov function for (7.52) also in this case.

(7.55)

124

7 Stability and stabilizability

The following theorem shows that for a large class of systems (7.51) feedback (7.54) is asymptotically stabilizing. Theorem 7.13. Feedback v(ω) = −B ∗ ω, ω ∈ R3 , stabilizes the system (7.51) asymptotically if and only if every row of the matrix B has a nonzero element. Proof. By (7.55) the closed-loop system ω˙ = J −1 S(ω)Jω − J −1 BB ∗ ω

(7.56)

is stable in the sense of Lyapunov. It follows from the La Salle theorem that an arbitrary trajectory (7.56) converges, as t ↑ +∞, to the maximal invariant set K contained in L = {ω ∈ R3 ; WF (ω) = 0} = ker B ∗ . In particular the set M of all stationary points for (7.56) is in L: M = {ω ∈ R3 ; F (ω) = 0} = {ω ∈ L; S(ω)Jω = 0} = N ∩ L, where N = {ω ∈ R3 ; S(ω)Jω = 0}. Let us denote by qi the i-th coordinate axis and by Pj the hyperplane 3  orthogonal to the j-th control axis bj , i, j = 1, 2, 3. Then L = ker B ∗ = Pj j=1

and M=

3  j=1

Moreover

Pj ∩

3 

3  3   qi = Pj ∩ qi .

i=1

 {0}, Pj ∩ qi = qi ,

j=1 i=1

if bi,j = 0, if bi,j = 0,

Therefore, if bi,j = 0 for j = 1, 2, 3, then M = qi . Consequently M = {0} if and only if for arbitrary i = 1, 2, 3 there exists j = 1, 2, 3 such that bi,j = 0. If system (7.56) is asymptotically stable then M = {0} and therefore every row of B has a nonzero element. This proves the theorem in one direction. To prove the converse implication assume that M = {0}. We will consider three cases. (i) rank B = 3. Then L = {0} and since K ⊂ L we have K = {0}. (ii) rank B = 2. In this case L is a straight line. If ω(·) is an arbitrary trajectory of (7.56), completely contained in K, then, by taking into account that K ⊂ L, d W (ω(t)) = −|B ∗ ω(t)|2 = 0, t ≥ 0, dt and, consequently, for a constant c W (ω(t)) = c = W (ω(0)),

t ≥ 0.

7.9

Stabilization of the Euler equations

125

Since the trajectory ω(·) is contained in the line L and in the ellipsoid {ω ∈ R3 ; W (ω) = c}, it has to be a stationary solution: ω(t) = ω(0), t ≥ 0. Finally we get c = 0 and M ⊂ K ⊂ L ∩ {0} = {0}. We see that K = {0}. (iii) rank B = 1. Without any loss of generality we can assume that b1 = 0, b2 = b3 = 0. Note that K ⊂ P = {ω ∈ L; F (ω) ∈ L} = {ω ∈ L; b∗1 (J −1 S(ω)Jω − J −1 BB ∗ ω) = 0} = {ω ∈ R3 ; b∗1 J −1 S(ω)Jω = 0 and b∗1 ω = 0}. Therefore the set P is an intersection of a cone, defined by the equation b11 I23 ω2 ω3 + b21 I31 ω3 ω1 + b31 I12 ω1 ω2 = 0, where I23 ≤ (I2 − I3 )/I1 , I31 = (I3 − I1 )/I2 , I12 = (I1 − I2 )/I3 , and of the plane b11 ω1 + b21 ω2 + b31 ω3 = 0, ω ∈ R3 . Consequently P can be either a point or a sum of two, not necessarily different, straight lines. As in case (ii), we show that K ⊂ P ∩ {ω ∈ R3 ; W (ω) = c} for and finally K = {0}.

c = 0, 2

Bibliographical notes The main stability test is taken from E.A. Coddington and N. Levinson [18]. The content of Theorem 7.4 is sometimes called the second Lyapunov method of studying stability. The first one is the linearization. Theorem 7.7 is due to M.A. Krasnoselski and P.P. Zabreiko [59]; see also J. Zabczyk [114]. A similar result was obtained independently by R. Brockett, and Theorem 7.11 is due to him, see R.W. Brockett [13]. He applied it to show that, in general, for nonlinear systems, local controllability does not imply asymptotic stabilizability. A converse to the Brockett theorem is studied in a recent paper [42] by R. Gupta, F. Jafari, R.J. Kipka and B.S. Mordukhovich. Theorem 7.9 is taken from J. Zabczyk [114], as is Theorem 7.10. Theorem 7.13 is due to M. Szafrański [92], see D. Aeyels and M. Szafrański [1] for some extensions, and Exercise 7.9 to H. Sussmann.

Chapter 8

Realization theory

It is shown in this chapter how, by knowing the input–output map of a control system, one can determine the impulse-response function of its linearization. A control system with a given input–output map is also constructed.

§8.1. Input–output maps Let us consider a control system y˙ = f (y, u), w = h(y),

y(0) = x ∈ Rn ,

(8.1) (8.2)

with an open set of control parameters U ⊂ Rm . Assume that the function f satisfies the conditions of Theorem 6.3 and that h is a continuous function with values in Rk . Let us fix T and denote by B(0, T ; U ) and C(0, T ; Rk ) the spaces of bounded, Borel and continuous functions, respectively, defined on [0, T ] and with values in U and in Rk . For an arbitrary function u(·) ∈ B(0, T ; U ) there exists exactly one solution y u (t, x), t ∈ [0, T ], of the equation (8.1), and thus for arbitrary x ¯ ∈ Rn the formula Ru(t) = w(t) = h(y u (t, x ¯)),

t ∈ [0, T ],

(8.3)

defines an operator from B(0, T ; U ) into C(0, T ; Rk ). As in the linear case (compare §I.5.1, formula (8.2)), the transformation R will be called the input– output map of the system (8.1)–(8.2). Realization theory is concerned with the construction of a system (8.1)–(8.2) assuming the knowledge of its input– output map. The linear theory was discussed in Chapter I.5. The nonlinear case is much more complicated, and we limit our presentation to two typical results.

© Springer Nature Switzerland AG 2020

J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_8

127

128

8 Realization theory

The same input–output map can be defined by different linear systems (8.1)–(8.2). On the other hand, input–output maps determine uniquely several important characteristics of control systems. One of them is the impulseresponse function of the linearization we now show. ¯ and a parameter u ¯ ∈ Rm form a Assume that U = Rm and that a state x stationary pair, f (¯ x, u ¯) = 0. Let us recall (see (6.19) and Theorem 6.11) that if the mappings f and h are differentiable at (¯ x, u ¯) and x ¯, respectively, then the linearization of (8.1)–(8.2) at (¯ x, u ¯) is of the form y˙ = Ay + Bu, w = Cy,

(8.4) (8.5)

y(0) = x,

where A = fx (¯ x, u ¯), B = fu (¯ x, u ¯), C = hx (¯ x). The function Ψ (t) = CetA B = hx (¯ x)etfx (¯x,¯u) fu (¯ x, u ¯),

t ≥ 0,

(8.6)

is the impulse-response function of the linearization (8.4)–(8.5). Theorem 8.1. If mappings f : Rn × Rm −→ Rn , h : Rn −→ Rk are of class C 1 and f satisfies the linear growth condition (6.6), then the input–output map R of (8.1)–(8.2) is well defined and uniquely determines the impulseresponse function (8.6) of the linearization (8.4)–(8.5). Proof. Let u : [0, T ] → Rm be a continuous function such that u(0) = 0. For an arbitrary number γ and t ∈ [0, T ] R(¯ u + γu)(t) = h(y γu (t, x ¯)) = h(x(t, γ)), where z(t) = z(t, γ), t ∈ [0, T ], is a solution to the equation z(t) ˙ = f (z(t), u ¯ + γu(t)),

t ∈ [0, T ], z(0) = x ¯.

By Theorem 6.5, the solution z(·, ·) is differentiable with respect to γ. Moreover, the derivatives zγ (t) = zγ (t, 0), t ∈ [0, T ], satisfy the linear equation d zγ (t) = fx (¯ x, u ¯)zγ (t) + fu (¯ x, u ¯)u(t), dt

zγ (0) = 0.

Consequently lim

γ→0

1 (R(¯ u + γu)(t) − R(¯ u)(t)) = hx (¯ x)zγ (t) γ  t hx (¯ x)e(t−s)fx (¯x,¯u) fu (¯ x, u ¯)u(s) ds, = 0

and we see that R determines Ψ(·) uniquely.

t ∈ [0, T ], 2

8.2 Partial realizations

129

§8.2. Partial realizations Given an input–output map R, we construct here a system of the type (8.1)–(8.2), which generates, at least on the same time intervals, the mapping R. To simplify the exposition, we set k = 1. Let (8.7)

u(t) = u(t; v1 , . . . , vl ; u1 , . . . , ul ), 

u (t) =

u(t; v1 , . . . , vl ;

u1 , . . . , ul ),

t ≥ 0,

(8.8)

be inputs defined by (6.37); see §6.4. We define a new control function u ◦ u setting  u(t) for t ∈ [0, v1 + . . . + vl ), u ◦ u(t) =  for t ≥ v1 + . . . + vl . u (t − (v1 + . . . + vl )) Hence u ◦ u(t) = u(t; v1 , . . . , vl , v1 , . . . , vl ; u1 , . . . , ul , u1 , . . . , ul ),

t ≥ 0.

Let us fix parameters (u1 , . . . , ul ) and inputs uj (·) = u(·; v1j , . . . , vljj ; uj1 , . . . , ujlj ),

j = 1, . . . , r.

(8.9)

∗ Define, on the set ⎡ Δ⎤= {(v1 , . . . , vl ) ; v1 + . . . + vl < T , v1 > 0, . . . , vl > 0}, g1 ⎢ ⎥ a mapping G = ⎣ ... ⎦ with values in Rr in the following way:

gr gj (v1 , . . . , vl ) = R(uj ◦ u)(v1 + . . . + vl + v1j + . . . + vljj ) j

¯)), = h(y u ◦u (v1 + . . . + vl + v1j + . . . + vljj , x ⎡ ⎤ v1 ⎢ .. ⎥ v = ⎣ . ⎦ ∈ Δ, j = 1, . . . , r. vl The mapping G is a composition G = G2 G1 of transformations G1 :Δ → Rn and G2 : Rn −→ Rr , given by ⎡ ⎤ v1 ⎢ .. ⎥ 1 u(·,v1 ,...,vl ;u1 ,...,ul ) G (v) = y (v1 + . . . + v1 , x ¯), v = ⎣ . ⎦ ∈ Δ, (8.10) 2



G (x) = h(y

uj

(v1j

+ ... +

 vljj , x)) j=1,2,...,r ,

vl x ∈ Rn .

(8.11)

130

8 Realization theory

The following lemma holds. Lemma 8.1. If h and f are of class C 1 , then ⎡ ⎤ v1 ⎢ .. ⎥ rank Gv (v) ≤ n, v = ⎣ . ⎦ ∈ Δ. vl Proof. The derivative Gv (v) of G at v ∈ Δ is the composition of the derivatives G1v (v) and G2x (G1 (v)) which have ranks not greater than the dimension  of Rn . An input–output map R is said to be regular if there exist parameters u1 , . . . , un ∈ U , controls u1 , . . . , un of the form (8.9) and v˜ ∈ Rn such that rank Gv (˜ v ) = n. In addition, coordinates v˜1 , . . . , v˜n of v˜ should satisfy the constraints v˜1 + . . . + v˜n + v1j + . . . + vljj < T, j = 1, . . . , n. Controls u ˜(·) = u(·; v˜1 , . . . , v˜n ; u1 , . . . , un ), u1 (·), . . . , un (·) and the sequence (˜ v1 , . . . , v˜n ) are called input!reference and a reference sequence. Let D be a bounded open subset of Rn , v˜ ∈ Rn , f˜ a continuous mapping ˜ a real function on D. Assume that there exists a from D × U in Rn and h constant c > 0 such that |f˜(v, u)| ≤ c(|v| + |u| + 1), |f˜(v, u) − f˜(w, u)| ≤ c|v − w|, v, w ∈ D, u ∈ U . Let T˜ > 0 be a positive number. Then for an arbitrary bounded control u(t), t ≥ 0, there exists a unique maximal in D solution y˜(t) = y˜u (t, v), t ∈ [T˜, τ (u)), of y˜˙ = f˜(˜ y (t), u(t)), t ∈ [T˜, τ (u)). y˜(T˜) =v;

(8.12)

see Theorem 6.3 and Theorem 6.1. If there exist a bounded control u ˜(t), t ≥ 0, a vector v˜ ∈ D and a number T˜ < T such that, for all controls u(·) ∈ B(0, T ; U ) which coincide with u ˜(·) on [0, T˜], one has ˜ y u (t, v˜)), R(u)(t) = h(˜

t ∈ [T , min(T, τ (u)),

(8.13)

t ∈ [0, τ (u)),

(8.14)

then one says that the system y˜˙ = f˜(˜ y , u) ˜ w(t) ˜ = h(˜ y (t)),

is a partial realization of the input–output map R.

8.2 Partial realizations

131

If a partial realization of an input–output map R can be constructed, then one says that the map R is partially realizable. The following theorem holds. Theorem 8.2. Assume that all transformations f (·, u), u ∈ U , are of class C 2 and that an input–output map R is regular. Then the map R is partially realizable. Proof. Assume that R is regular. Let u ˜(·), u1 , . . . , un and (˜ v1 , . . . , v˜n ) be the corresponding reference inputs and the reference sequence. Define ⎡ ⎤ v˜1 ⎢ .. ⎥ v˜ = ⎣ . ⎦ , T˜ = v˜1 + . . . + v˜n v˜n and fix a neighbourhood D of v˜ in which the mapping G is a homeomorphism. For x = G1 (v), v ∈ D, u ∈ U and sufficiently small η > 0 the equation y u (t, x) = G1 (˜ y u (t, v)),

t ∈ [0, η),

determines a function y˜u (·, v) which satisfies the differential equation d u y˜ (t, v) = [G1v (˜ y u (t, v))]−1 f (˜ y u (t, v), u), dt

t ∈ [0, η),

(8.15)

We claim that the formulae f˜(v, u) = [G1v (v)]−1 f (G1 (v), u), ˜ h(v) = h(G1 (v)), v ∈ D, u ∈ U,

(8.16) (8.17)

define a partial realization of R. To show this note first that h(G1 (v)) = h(y u(·;v1 ,...,vn ;u1 ,...,un ) (v1 . . . + vn , x ¯)) = R(u(·; v1 , . . . , vn ; u1 , . . . , un ))(v1 + . . . + vn ),

⎤ v1 ⎢ ⎥ v = ⎣ ... ⎦ . ⎡

vn ˜ can be defined in terms of the mapping R only. Hence the function h Let uj (v; s; u)(·) be a control identical to u(·; u1 , . . . , un ; v1 , . . . , vn ) on the interval [0, v1 + . . . + vn ), and to u, uj1 , . . . , ujlj , j = 1, 2, . . . , n, on consecutive intervals of lengths s, v1j , . . . , vljj , respectively. Let us remark that

R(uj (v; s; u))(v1 + . . . + vn + s + v1j + . . . + vljj ) 

j

¯)) . = h y u v1j + . . . + vljj , y u (s, y u(·;v1 ,...,vn ;u1 ,...,un ) (v1 + . . . + vn , x

132

8 Realization theory

Therefore, for sufficiently small s > 0 and all u ∈ U , the transformations

 G2 y u (s, G1 (v)) , v ∈ D, are also given in terms of R. Moreover, denoting by G−1 , (G1 )−1 the inverses of G and G1 , respectively, we obtain



  (8.18) G−1 G2 y u (s, G1 (v)) = (G1 )−1 y u (s, G1 (v)) . The derivative at s = 0 of the function defined by (8.18) is identical with f˜. Hence the function f˜ is expressible in terms of the transformation R. It ˜ define a partial follows from (8.15) and (8.17) that the functions f˜ and h realization of R.  We finally show that the family of all regular input–output maps is rather large. Proposition 8.1. Assume that h(·) and f (·, u), u ∈ U , are of class C 1 and C k , k ≥ 2, respectively, and that for x) = n. If

some j ≤ k, dim Lj (¯ x, u ¯), hx (¯ x) is observable, then f (¯ x, u ¯) = 0 for some u ¯ ∈ U and the pair fx (¯ the input–output map R corresponding to the system (8.1)–(8.2), with x = x ¯, is regular. Proof. We see, using similar arguments to those in the proof of Theorem that for arbitrary neighbourhood ¯, there exist parameters ⎡ V ⎤of the point x v1 ⎢ ⎥ u1 , . . . , un ∈ U and a point v = ⎣ ... ⎦, such that the derivative G1v (v) of the vn transformation G1 defined by (8.10) is nonsingular and G1 (v) ∈ V . Define ¯; v j ), t ≥ 0 and j = 1, . . . , n. It is enough to show that the uj (t) = u(t; u ¯ is nonsingular as well. However this derivative is of derivative of G2 (·) at x ⎤ ⎡ the form x)ev1 fx (¯x,¯u) hx (¯ ⎥ ⎢ .. x) = ⎣ G2x (¯ ⎦. . x)evn fx (¯x,¯u) hx (¯ Consequently, to complete the proof of the theorem, it remains to establish the following lemma. Lemma 8.2. If a pair (A, C), A ∈ M(n, n), C ∈ M(n, k), is observable, then for arbitrary δ > 0 there exist numbers v j , 0 < v j < δ, j = 1, . . . , n, such that ⎡ v1 A ⎤ Ce ⎢ . ⎥ rank ⎣ .. ⎦ = n. Cev

n

A

Proof. Taking into account that the pair (A∗ , C ∗ ) is controllable, it is enough to show that for an arbitrary controllable pair (A, B), A ∈ M(n, n), B ∈ M(n, m), there exist numbers v j , 0 < v j < δ, j = 1, . . . , n, such that

8.2 Partial realizations 1

133

n

rank[ev A B, . . . , ev A B] = n. Denote by by b1 , . . . , bm the columns of the matrix B. It easily follows from the controllability of (A, B) that the smallest linear subspace containing {esA br ; s ∈ (0, δ), r = 1, . . . , m} is identical with Rn . Hence this set contains n linearly independent vectors.

Bibliographical notes Theorem 8.2 is a special case of a theorem due to B. Jakubczyk [52]. The construction introduced in its proof applies also to nonregular systems, however, that proof is much more complicated.

Part III

Optimal Control

Chapter 9

Dynamic programming

This chapter starts from a derivation of the dynamic programming equations called Bellman equations. They are used to solve the linear regulator problem on a finite time interval. A fundamental role is played here by the Riccati-type matrix differential equations. The stabilization problem is reduced to an analysis of an algebraic Riccati equation.

§9.1. Introductory comments Part III is concerned with controls which are optimal, according to a given criterion. Our considerations will be devoted mainly to control systems y˙ = f (y, u), y(0) = x, (9.1) and to criteria, called also cost functionals,  JT (x, u(·)) =

T

0

g(y(t), u(t)) dt + G(y(T )),

(9.2)

when T < +∞. If the control interval is [0, +∞], then the cost functional  J(x, u(·)) =

+∞

g(y(t), u(t)) dt.

(9.3)

0

Our aim will be to find a control u ˆ(·) such that for all admissible controls u(·) JT (x, u ˆ(·)) ≤ JT (x, u(·)) (9.4) or J(x, u ˆ(·)) ≤ J(x, u(·)).

(9.5)

The function © Springer Nature Switzerland AG 2020

J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_9

137

138

9 Dynamic programming

W (t, x) = inf Jt (x, u(·)) u(·)

where the infimum is taken with respect to all admissible controls u(·) is called value function. In Chapter 11 and §12.3 we will also consider the so-called impulse control problems. There are basically two methods for finding controls minimizing cost functionals (9.2) or (9.3). One of them embeds a given minimization problem into a parametrized family of similar problems. The embedding should be such that the minimal value, as a function of the parameter, satisfies an analytic relation. If the selected parameter is the initial state and the length of the control interval, then the minimal value of the cost functional is called the value function, and the analytical relation the Bellman equation. Knowing the solutions to the Bellman equation one can find the optimal strategy in the form of a closed-loop control. The other method leads to necessary conditions on the optimal, open-loop, strategy formulated in the form of the so-called maximum principle discovered by L. Pontryagin and his collaborators. They can be obtained (in the simplest case) by considering a parametrized family of controls and the corresponding values of the cost functional (9.2) and by an application of classical calculus.

§9.2. Bellman equation and the value function Assume that the state space E of a control system is an open subset of Rn and let the set U of control parameters be included in Rm . We assume that the functions f , g and G are continuous on E × U and E, respectively, and that g is nonnegative. Theorem 9.1. Assume that a real function W (·, ·), defined and continuous on [0, T ] × E, is of class C 1 on (0, T ) × E and satisfies the equation ∂W (t, x) = inf (g(x, u) + Wx (t, x), f (x, u)), u∈U ∂t

(t, x) ∈ (0, T ) × E, (9.6)

with the boundary condition W (0, x) = G(x),

x ∈ E.

(9.7)

(i) If u(·) is a control and y(·) the corresponding absolutely continuous, E-valued, solution of (9.1) then JT (x, u(·)) ≥ W (T, x). (ii) Assume that for a certain function vˆ : [0, T ] × E −→ U

(9.8)

9.2 Bellman equation and the value function

139

g(x, vˆ(t, x)) + Wx (t, x), f (x, vˆ(t, x)) ≤ g(x, u) + Wx (t, x), f (x, u),

t ∈ (0, T ), x ∈ E, u ∈ U,

(9.9)

and that yˆ is an absolutely continuous, E-valued solution of the equation d yˆ(t) = f (ˆ y (t), vˆ(T − t, yˆ(t))), dt yˆ(0) = x.

t ∈ [0, T ],

(9.10)

Then, for the control u ˆ(t) = vˆ(T − t, yˆ(t)), t ∈ [0, T ], JT (x, u ˆ(·)) = W (T, x). Proof. (i) Let w(t) = W (T − t, y(t)), t ∈ [0, T ]. Then w(·) is an absolutely continuous function on an arbitrary interval [α, β] ⊂ (0, T ) and   dw ∂W dy (t) = − (T − t, y(t)) + Wx (T − t, y(t)), (t) dt ∂t dt   ∂W =− (T − t, y(t)) + Wx (T − t, y(t)), f (y(t), u(t)) (9.11) ∂t for almost all t ∈ [0, T ]. Hence, from (9.6) and (9.7)



β

dw (t) dt α dt   β   ∂W (T − t, y(t)) + Wx (T − t, y(t)), f (y(t), u(t)) dt = − ∂t α  β g(y(t), u(t)) dt. ≥−

W (T − β, y(β)) − W (T − α, y(α)) = w(β) − w(α) =

α

Letting α and β tend to 0 and T , respectively, we obtain  T g(y(t), u(t)) dt. G(y(T )) − W (T, x) ≥ − 0

This proves (i). (ii) In a similar way, taking into account (9.9), for the control u ˆ and the output yˆ, G(ˆ y (T )) − W (T, x)   T   ∂W (T − t, yˆ(t)) + Wx (T − t, yˆ(t)), f (ˆ = y (t), u ˆ(t)) dt − ∂t 0  T g(ˆ y (t), u ˆ(t)) dt. =− 0

140

9 Dynamic programming

Therefore

 G(ˆ y (T )) +

0

T

g(ˆ y (s), u ˆ(s)) ds = W (T, x), 

the required identity.

Remark. Equation (9.6) is called the Bellman equation. It follows from Theorem 9.1 that, under appropriate conditions, W (T, x) is the minimal value of the functional JT (x, ·). Hence W is the value function for the problem of minimizing (9.2). Let U (t, x) be the set of all control parameters u ∈ U for which the infimum on the right-hand side of (9.6) is attained. The function vˆ(·, ·) from part (ii) of the theorem is a selector of the multivalued function U (·, ·) in the sense that vˆ(t, x) ∈ U (t, x),

(t, x) ∈ [0, T ] × E.

Therefore, for the conditions of the theorem to be fulfilled, such a selector not only should exist, but the closed-loop equation (9.10) should have a well defined, absolutely continuous solution. Remark. A similar result holds for a more general cost functional  T JT (x, u(·)) = e¯αt g(y(t), u(t)) dt + e−αT G(y(T )).

(9.12)

0

In this direction we propose to solve the following exercise. Exercise 9.1. Taking into account a solution W (·, ·) of the equation ∂W (t, x) = inf (g(x, u) − αW (t, x) + Wx (t, x), f (x, u)), u∈U ∂t W (0, x) = G(x), x ∈ E, t ∈ (0, T ), and a selector vˆ of the multivalued function U (t, x) = u ∈ U ; g(x, u) + Wx (t, x), f (x, u)

= inf (g(x, u) + Wx (t, x), f (x, u)) , u∈U

generalize Theorem 9.1 to the functional (9.12). We will now describe an intuitive derivation of equation (9.6). Similar reasoning often helps to guess the proper form of the Bellman equation in situations different from the one covered by Theorem 9.1. Let W (t, x) be the minimal value of the functional Jt (x, ·). For arbitrary h > 0 and arbitrary parameter v ∈ U denote by uv (·) a control which is constant and equal to v on [0, h) and is identical with the optimal strategy for the minimization problem on [h, t + h]. Let z x,v (t), t ≥ 0, be the solution of the equation z˙ = f (z, u), z(0) = x. Then

9.2 Bellman equation and the value function

Jt+h (x, uv (·)) =

 0

h

141

g(z x,v (s), v) ds + W (t, z x,v (h))

and, approximately, W (t + h, x) ≈ inf Jt+h (x, uv (·)) ≈ inf v∈U



v∈U

0

h

g(z x,v (s), v) ds + W (t, z x,v (h)).

Subtracting W (t, x) we obtain 1 (W (t + h, x) − W (t, x)) h    h 1 1 ≈ inf g(z x,v (s), v) ds + (W (t, z x,v (h)) − W (t, x)) . v∈U h 0 h Assuming that the function W is differentiable and taking the limits as h ↓ 0 we arrive at (9.6).  For the control problem on the infinite time interval we have the following easy consequence of Theorem 9.1. Theorem 9.2. Let g be a nonnegative, continuous function and assume that there exists a nonnegative function W , defined on E and of class C 1 , which satisfies the equation  inf g(x, u) + Wx (x), f (x, u) = 0, x ∈ E. (9.13) u∈U

If for a strategy u(·) and the corresponding output y, lim W (y(t)) = 0, then t↑+∞

J(x, u(·)) ≥ W (x).

(9.14)

If vˆ : E −→ U is a mapping such that g(x, vˆ(x)) + Wx (x), f (x, vˆ(x)) = 0 for x ∈ E, and yˆ is an absolutely continuous, E-valued, solution of the equation d yˆ(t) = f (ˆ y (t), vˆ(ˆ y (t))), dt

t ≥ 0,

for which lim W (ˆ y (t)) = 0, then t↑+∞

J(x, u ˆ(·)) = W (x), Proof. We apply Theorem 9.1 with G(x) = W (x),

x ∈ E.

(9.15)

142

9 Dynamic programming

(t, x) = W (x), t ∈ [0, T ], x ∈ E, satisfies all the conSince the function W ditions of Theorem 9.1 we see that, for an arbitrary admissible control u(·) and the corresponding output, 

T

0

g(y(t), u(t)) dt + W (y(T )) ≥ W (x),

T > 0.

Letting T tend to +∞ and taking into account that lim W (y(T )) = 0 we T ↑+∞

obtain the required inequality. On the other hand, for the strategy u ˆ(·),  0

T

g(ˆ y (t), u ˆ(t)) dt + W (ˆ y (T )) = W (x),

T ≥ 0.

Hence, if lim W (ˆ y (T )) = 0, then T ↑+∞



J(x, u ˆ) = W (x).

Let us remark that if a function W is a nonnegative solution of equation (9.13) then function W + c, where c is a nonnegative constant, is also a nonnegative solution to (9.13). Therefore, without additional conditions of the type lim W (y(t)) = 0, the estimate (9.14) cannot hold. t↑+∞

Theorem 9.2 can be generalized (compare Exercise 9.1) to the cost functional  +∞ e−αt g(y(t), u(t)) dt. J(x, u(·)) = 0

The corresponding Bellman equation (9.13) is then of the form inf ((g(x, u) + Wx (x), f (x, u)) = αW (x),

u∈U

x ∈ E.

Exercise 9.2. Show that the solution of the Bellman equation corresponding to the optimal consumption model of Example 0.7, with α ∈ (0, 1), is of the form W (t, x) = p(t)xα , t ≥ 0, x ≥ 0, where the function p(·) is the unique solution of the following differential equation:  1, for p ≤ 1, p˙ = 1 α/(1−α) αp + (1 − α)( p ) , for p ≥ 1, p(0) = a. Find the optimal strategy. Hint. First prove the following lemma.

9.3 The linear regulator problem and the Riccati equation

143

Lemma 9.1. Let ψp (u) = αup + (1 − u)α , p ≥ 0, u ∈ [0, 1]. The maximal value m(p) of the function ψp (·) is attained at  0, if p > 1, u(p) = ( p1 )1/(1−α) , if p ∈ [0, 1]. Moreover

 1, m(p) = αp + (1 − α)( p1 )α/(1−α) ,

if p ≥ 1, if p ∈ [0, 1].

§9.3. The linear regulator problem and the Riccati equation We now consider a special case of problems (9.1) and (9.4) when the system equation is linear y˙ = Ay + Bu,

y(0) = x ∈ Rn ,

(9.16)

A ∈ M(n, n), B ∈ M(n, m), the state space E = Rn and the set of control parameters U = Rm . We assume that the cost functional is of the form  JT =

0

T

Qy(s), y(s) + Ru(s), u(s) ds + P0 y(T ), y(T ),

(9.17)

where Q ∈ M(n, n), R ∈ M(m, m), P0 ∈ M(n, n) are symmetric, nonnegative matrices and the matrix R is positive definite; see §I.1.1. The problem of minimizing (9.17) for a linear system (9.16) is called the linear regulator problem or the linear-quadratic problem. The form of an optimal solution to (9.16) and (9.17) is strongly connected with the following matrix Riccati equation: P˙ = Q + P A + A∗ P − P BR−1 B ∗ P,

P (0) = P0 ,

(9.18)

in which P (s), s ∈ [0, T ], is the unknown function with values in M(n, n). The following theorem takes place. Theorem 9.3. Equation (9.18) has a unique global solution P (s), s ≥ 0. For arbitrary s ≥ 0 the matrix P (s) is symmetric and nonnegative definite. The minimal value of the functional (9.17) is equal to P (T )x, x and the optimal control is of the form u ˆ(t) = −R−1 B ∗ P (T − t)ˆ y (t), where

t ∈ [0, T ],

(9.19)

144

9 Dynamic programming

y (t), yˆ˙ (t) = (A − BR−1 B ∗ P (T − t))ˆ

t ∈ [0, T ],

yˆ(0) = x.

(9.20)

Proof. The proof will be given in several steps. Step 1. For an arbitrary symmetric matrix P0 equation (9.18) has exactly one local solution and the values of the solution are symmetric matrices. Equation (9.18) is equivalent to a system of n2 differential equations for elements pij (·), i, j = 1, 2, . . . , n, of the matrix P (·). The right-hand sides of these equations are polynomials of order 2, and therefore the system has a unique local solution being a smooth function of its argument. Let us remark that the same equation is also satisfied by P ∗ (·). This is because matrices Q, R and P0 are symmetric. Since the solution is unique, P (·) = P ∗ (·), and the values of P (·) are symmetric matrices. Step 2. Let P (s), s ∈ [0, T0 ), be a symmetric solution of (9.18) and let T < T0 . The function W (s, x) = P (s)x, x, s ∈ [0, T ], x ∈ Rn , is a solution of the Bellman equation (9.6)–(9.7) associated with the linear regular problem (9.16)–(9.17). The condition (9.7) follows directly from the definitions. Moreover, for arbitrary x ∈ Rn and t ∈ [0, T ]  infm Qx, x + Ru, u + 2P (t)x, Ax + Bu (9.21) u∈R    ∗ = Qx, x + (A P (t) + P (t)A)x, x + infm Ru, u + u, 2B ∗ P (t)x . u∈R

We need now the following lemma, the proof of which is left as an exercise. Lemma 9.2. If a matrix R ∈ M(m, m) is positive definite and a ∈ Rm , then for arbitrary u ∈ Rm 1 Ru, u + a, u ≥ − R−1 a, a. 4 Moreover, the equality holds if and only if 1 u = − R−1 a. 2 It follows from the lemma that the expression (9.21) is equal to   Q + A∗ P (t) + P (t)A∗ − P (t)BR−1 B ∗ P (A)x, x and that the infimum in formula (9.21) is attained at exactly one point given by −R−1 B ∗ P (t)x, t ∈ [0, T ]. Since P (t), t ∈ [0, T0 ), satisfies equation (9.18), the function W is a solution to the problem (9.6)–(9.7). Step 3. The control u ˆ given by (9.19) on [0, T ], T < T0 , is optimal with respect to the functional JT (x, ·).

9.3 The linear regulator problem and the Riccati equation

145

This fact is a direct consequence of Theorem 9.1. Step 4. For arbitrary t ∈ [0, T ], T < T0 , the matrix P (t) is nonnegative definite and  t Q˜ y x (s), y˜x (s) ds + P0 y˜x (t), y˜x (t), (9.22) P (t)x, x ≤ 0

where y˜x (·) is the solution to the equation y˜˙ = A˜ y,

y˜(0) = x.

Applying Theorem 9.1 to the function Jt (x, ·) we see that its minimal value is equal to P (t)x, x. For arbitrary control u(·), Jt (x, u) ≥ 0, the matrix P (t) is nonnegative definite. In addition, estimate (9.22) holds because its righthand side is the value of the functional Jt (x, ·) for the control u(s) = 0, s ∈ [0, t]. Step 5. For arbitrary t ∈ [0, T0 ) and x ∈ Rn 0 ≤ P (t)x, x ≤



t

0

  S ∗ (r)QS(r) dr + S ∗ (t)P0 S(t) x, x ,

where S(r) = eAr , r ≥ 0. This result is an immediate consequence of the estimate (9.22). Exercise 9.3. Show that if, for some symmetric matrices P = (pij ) ∈ M(n, n) and S = (sij ) ∈ M(n, n), 0 ≤ P x, x ≤ Sx, x, then

x ∈ Rn ,

1 1 − (sii + sjj ) ≤ pij ≤ sij + (sii + sjj ), 2 2

i, j = 1, . . . , n.

It follows from Step 5 and Exercise 9.3 that solutions of (9.18) are bounded in M(n, n) and therefore an arbitrary maximal solution P (·) in M(n, n) exists for all t ≥ 0; see Theorem II.6.1. The proof of the theorem is complete. 2 Exercise 9.4. Solve the linear regulator problem with a more general cost functional  T (Q(y(t) − a), y(t) − a + Ru(t), u(t)) dt + P0 y(T ), y(T ), 0

where a ∈ Rn is a given vector. Answer. Let P (t), q(t), r(t), t ≥ 0, be solutions of the following matrix, vector and scalar equations, respectively,

146

9 Dynamic programming

P˙ = Q + A∗ P + P A − P BR−1 B ∗ P, ∗

−1

q˙ = A q − P BR q − 2Qa, 1 r˙ = − R−1 q, q + Qa, a, 4

P (0) = P0 ,

q(0) = 0, r(0) = 0.

The minimal value of the functional is equal to r(T ) + q(T ), x + P (T )x, x, and the optimal, feedback strategy is of the form 1 u(t) = − R−1 q(T − t) − R−1 B ∗ P (T − t)y(t), 2

t ∈ [0, T ].

§9.4. The linear regulator and stabilization The obtained solution of the linear regulator problem suggests an important way to stabilize linear systems. It is related to the algebraic Riccati equation (9.23) Q + P A + A∗ P − P BR−1 B ∗ P = 0, P ≥ 0, in which the unknown is a nonnegative definite matrix P . If P is a solution to (9.23) and P ≤ P for all the other solutions P , then P is called a minimal solution of (9.23). For arbitrary control u(·) defined on [0, +∞) we introduce the notation  +∞ (Qy(s), y(s) + Ru(s), u(s)) ds. (9.24) J(x, u) = 0

Theorem 9.4. If there exists a nonnegative solution P of equation (9.23) then there also exists a unique minimal solution P˜ of (9.23), and the control u ˜ given in the feedback form u ˆ(t) = −R−1 B ∗ Py(t),

t ≥ 0,

minimizes functional (9.24). Moreover the minimal value of the cost functional is equal to Px, x. Proof. Let us first remark that if P1 (t), P2 (t), t ≥ 0, are solutions of (9.18) and P1 (0) ≤ P2 (0) then P1 (t) ≤ P2 (t) for all t ≥ 0. This is because the minimal value of the functional  t (Qy(s), y(s) + Ru(s), u(s)) ds + P1 (0)y(t), y(t) Jt1 (x, u) = 0

is not greater than the minimal value of the functional

9.4 The linear regulator and stabilization



Jt2 (x, u) =

t

0

147

(Qy(s), y(s) + Ru(s), u(s)) ds + P2 (0)y(t), y(t)

and by Theorem 9.3 the minimal values are P1 (t)x, x and P2 (t)x, x respectively. If, in particular, P1 (0) = 0 and P2 (0) = P then P2 (t) = P and therefore P1 (t) ≤ P for all t ≥ 0. It also follows from Theorem 9.3 that the function P1 (·) is nondecreasing with respect to the natural order existing in the space of symmetric matrices; see §I.1.1. This easily implies that for arbitrary i, j = pij (t)) = P1 (t), 1, 2, . . . , n there exist finite limits p˜ij = lim p˜ij (t), where (˜ t↑+∞

t ≥ 0. Taking into account equation (9.18) we see that there exist finite limits lim

t↑+∞

d p˜ij (t) = γij , dt

i, j = 1, . . . , n.

These limits have to be equal to zero, for if γi,j > 0 or γi,j < 0 then lim p˜ij (t) = +∞. But lim p˜ij (t) = −∞, a contradiction. Hence the matrix

t↑+∞

t↑+∞

P = (˜ pij ) satisfies equation (9.23). It is clear that P ≤ P . Now let y˜(·) be the output corresponding to the input u ˜(·). By Theorem 9.3, for arbitrary T ≥ 0 and x ∈ Rn ,  P˜ x, x = and

T

0

 0

(Q˜ y (t), y˜(t) + R˜ u(t), u ˜(t)) dt + P˜ y˜(T ), y˜(T ),

T

(9.25)

(Q˜ y (t), y˜(t) + R˜ u(t), u ˜(t)) dt ≤ P˜ x, x.

Letting T tend to +∞ we obtain J(x, u ˜) ≤ Px, x. On the other hand, for arbitrary T ≥ 0 and x ∈ Rn  T P1 (T )x, x ≤ (Q˜ y (t), y˜(t) + R˜ u(t), u ˜(t)) dt ≤ J(x, u ˜), 0

consequently, Px, x ≤ J(x, u ˜) and finally J(x, u ˜) = Px, x. 

The proof is complete. Exercise 9.5. For the control system y¨ = u, find the strategy which minimizes the functional

148

9 Dynamic programming

 0

+∞

(y 2 + u2 ) dt

and the minimal value of this functional.



0 0

   1 0 ,B = , 0 1

Answer. The solution of equation (9.23) in which A =   √  1 0 2 √1 Q = , R = [1] is matrix P = . The optimal strategy 0 0 1 2 √ ˙ and the minimal value of the functional is is of the form u = −y −√ 2(y) √ 2 2(y(0))2 + 2y(0)y(0) ˙ + 2(y(0)) ˙ . For stabilizability the following result is essential.

Theorem 9.5. (i) If the pair (A, B) is stabilizable then equation (9.23) has at least one solution. (ii) If Q = C ∗ C and the pair (A, C) is detectable then equation (9.23) has at most one solution, and if P is the solution then the matrix A − BR−1 B ∗ P is stable. Proof. (i) Let K be a matrix such that the matrix A + BK is stable. Consider a feedback control u(t) = Ky(t), t ≥ 0. It follows from the stability of A + BK that y(t) −→ 0, and therefore u(t) −→ 0 exponentially as t ↑ +∞. Thus for arbitrary x ∈ Rn ,  +∞ J(x, u(·)) = (Qy(t), y(t) + Ru(t), u(t)) dt < +∞. 0

Since

P1 (T )x, x ≤ J(x, u(·)) < +∞,

T ≥ 0,

for the solution P1 (t), t ≥ 0, of (9.18) with the initial condition P1 (0) = 0, there exists lim P1 (T ) = P which satisfies (9.23). (Compare the proof of T ↑+∞

the previous theorem.) (ii) We prove first the following lemma. Lemma 9.3. (i) Assume that for some matrices M ≥ 0 and K of appropriate dimensions, (9.26) M (A − BK) + (A − BK)∗ M + C ∗ C + K ∗ RK = 0. If the pair (A, C) is detectable, then the matrix A − BK is stable. (ii) If, in addition, P is a solution to (9.23), then P ≤ M . Proof. (i) Let S1 (t) = e(A−BK)t , S2 (t) = e(A−LC)t , where L is a matrix such that A − LC is stable and let y(t) = S1 (t)x, t ≥ 0. Since

9.4 The linear regulator and stabilization

149

A − BK = (A − LC) + (LC − BK), therefore

 y(t) = S2 (t)x +

t

0

We show now that  +∞ |Cy(s)|2 ds < +∞

S2 (t − s)(LC − BK)y(s) ds. 

and

0

+∞

0

(9.27)

|Ky(s)|2 ds < +∞.

(9.28)

Let us remark that, for t ≥ 0, d M y(t), y(t) = 2M y(t), ˙ y(t). dt

y(t) ˙ = (A − BK)y(t) and It therefore follows from (9.26) that

d M y(t), y(t) + Cy(t), Cy(t) + RKy(t), Ky(t) = 0. dt Hence, for t ≥ 0, M y(t), y(t) +

 0

t

|Cy(s)|2 ds +

 0

t

RKy(s), Ky(s) ds = M x, x. (9.29)

Since the matrix R is positive definite, (9.29) follows from (9.28). By (9.29),  |y(t)| ≤ |S2 (t)x| + N

0

t

|S2 (t − s)|(|Cy(s)| + |Ky(s)|) ds,

where N = max(|L|, |B|), t ≥ 0. By Theorem A.8 (Young’s inequality) and by (9.28),  0

+∞

|y(s)|2 ds ≤ N



+∞

0

 +

0

|S2 (s)| ds

+∞

2



|S2 (s)| ds

0

+∞

1/2

(|Cy(s)| + |Ky(s)|2 ) ds

1/2

|x| < +∞.

It follows from Theorem I.2.3(iv) that y(t) → 0 as t → ∞. This proves the required result. Let us also remark that  +∞ M= S1∗ (s)(C ∗ C + K ∗ RK)S1 (s) ds. (9.30) 0

(ii) Define K0 = R quently,

−1

B ∗ P then RK0 = −B ∗ P,

P B = −K0∗ R. Conse-

P (A − BK) + (A − BK)∗ P + K ∗ RK = −C ∗ C + (K − K0 )∗ R(K − K0 )

150

and

9 Dynamic programming

M (A − BK) + (A − BK)∗ M + K ∗ RK = −C ∗ C.

Hence if V = M − P then V (A − BK) + (A − BK)∗ V + (K − K0 )∗ R(K − K0 ) = 0. Since the matrix A − BK is stable,  +∞ V = S1∗ (s)(K − K0 )∗ R(K − K0 )S1 (s) ds ≥ 0, 0

by Theorem I.2.7, and therefore M ≥ P . The proof of the lemma is complete.  To prove part (ii) of Theorem 9.5 assume that matrices P ≥ 0, P1 ≥ 0 are solutions of (9.23). Define K = R−1 B ∗ P . Then P (A − BK) + (A − BK)∗ P + C ∗ C + K ∗ RK = P A + A∗ P + C ∗ C − P BR−1 B ∗ P = 0.

(9.31)

Therefore, by Lemma 9.3(ii), P1 ≤ P . In the same way P1 ≥ P . Hence P1 = P . Identity (9.31) and Lemma 9.3(i) imply the stability of A − BK.  Remark. Lemma 9.3(i) implies Theorem I.2.7(1). It is enough to define B = 0, K = 0 and remark that observable pairs are detectable; see §I.2.7. As a corollary from Theorem 9.5 we obtain Theorem 9.6. If the pair (A, B) is controllable, Q = C ∗ C and the pair (A, C) is observable, then equation (9.23) has exactly one solution, and if P is this unique solution, then the matrix A − BR−1 B ∗ P is stable. Theorem 9.6 indicates an effective way of stabilizing linear system (9.16). Controllability and observability tests in the form of the corresponding rank conditions are effective, and equation (9.23) can be solved numerically using methods similar to those for solving polynomial equations. The uniqueness of the solution of (9.23) is essential for numerical algorithms. The following examples show that equation (9.23) does not always have a solution and that in some cases it may have many solutions. Example 9.1. If, in (9.23), B = 0, then we arrive at the Lyapunov equation P A + A∗ P = Q,

P ≥ 0.

(9.32)

If Q is positive definite, then equation (9.32) has at most one solution, and if, in addition, matrix A is not stable, then it does not have any solutions; see §I.2.4.

9.4 The linear regulator and stabilization

151

Exercise 9.6. If Q is a singular matrix then equation (9.23) may have many solutions. For if P is a solution to (9.23) and     = 0 0 , Q = 0 0 , A  ∈ M(k, k), k > n, A 0 A 0 Q then, for an arbitrary nonnegative matrix R ∈ M(k − n, k − n), the matrix   R 0 P˜ = 0 P satisfies the equation

+A  ∗ P = Q.  PA

Bibliographical notes Dynamic programming ideas are presented in the monograph by R. Bellman [7]. The results of the linear regulator problem are classic. Theorem 9.5 is due to W.M. Wonham [101]. In the proof of Lemma 9.3(i) we follow J. Zabczyk [106].

Chapter 10

Viscosity solutions of Bellman equations

The value function of a control problem does not satisfy, in general, the corresponding Bellman equation in the classical sense. In this chapter we show that under rather general conditions it is a unique viscosity solution of the equation, the concept introduced and developed by Crandall and Lions [21].

§10.1. Viscosity solution If a solution W of the Bellman equation (9.6)–(9.7) exists and is of class C 1 , then by Theorem 9.1 it is equal to the value function and allows constructing the optimal solution of the problem. However, the value function is often not of class C 1 and does not satisfy the regularity assumption of the theorem. Let us consider the simplest possible situation of the system on R1 : y  (t) = 0,

y(0) = x,

independent of the control parameter and with the final cost G(y(T )), only. Then y(t) = x + t, t ≥ 0, and the value function is of the form W (t, x) = G(x + t),

x ∈ R1 , t ≥ 0.

Thus W is of class C 1 if and only if G is continuously differentiable. The corresponding Bellman equation looks as follows: ∂W (t, x) = Wx (t, x), ∂t

W (0, x) = G(x), x ∈ R1 ,

and does not have a solution in the classical sense.

© Springer Nature Switzerland AG 2020

J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_10

153

154

10 Viscosity solutions of Bellman equations

It turns out however that under rather mild conditions the value function is the unique solution of (9.6)–(9.7) in the so-called viscosity sense, introduced by Crandall and Lions. We will be concerned with more general control systems for which the terminal cost T is fixed and the initial time t can be arbitrary from [0, T ]. Thus dy = f (y(s), u(s)), y(t) = x, T ≥ s ≥ t ≥ 0, (10.1) ds and the cost functional  T g(y(s), u(s)) ds + G(y(T )). (10.2) Jt (x, u(·)) = t

The initial state is x ∈ Rn and the control functions U are defined on [t, T ] and take values in a bounded set U ∈ Rm . They are assumed to be Borel measurable and form the set denoted by Ut . The solution to (10.1) will be denoted by ytx,U (s), s ∈ [t, T ], and the value function is given by the formula  V (t, x) = inf

U ∈Ut

t

T

    g ytx,U (s), U (s) ds + G ytx,U (t)

= inf Jt,T (x, U ). U ∈Ut

Note that if the initial time t is 0, we are in the framework of the previous chapter. Moreover, V (t, x) − W (T − t, x),

t ∈ [0, T ], x ∈ Rn .

(10.3)

When is clear from the context, we drop upper indices x, U and the lower index t. Formal arguments lead to the following Bellman equation for V : −

  ∂V (t, x) = inf g(x, u) + Vx (t, x), f (x, u) , u∈U ∂t V (T, x) = G(x), x ∈ Rn .

t ∈ (0, T ),

(10.4) (10.5)

It differs in sign on the left-hand side, due to the “change of variables” in (10.3). If we denote by H(x, p), x ∈ Rn , p ∈ Rn , the so-called hamiltonian of the optimization problem:   H(x, p) = inf g(x, u) + p, f (x, u) , u∈U

the Bellman equation can be written in the form ∂V (t, x) + H(x, Vx (t, x)) = 0, t ∈ (0, T ), x ∈ Rn , ∂t V (T, x) = G(x), x ∈ Rn .

(10.6) (10.7)

10.2 Dynamic programming principle

155

Define Q = [0, T ) × Rn , Q = [0, T ] × Rn . A uniformly continuous on Q function v is a viscosity subsolution to (10.6)–(10.7) if v(T, x) ≤ G(x), x ∈ Rn , and if v −ϕ, where ϕ is a C ∞ function on Q, attains a maximum at (t, x) ∈ Q, then ∂ϕ (t, x) + H(x, ϕx (t, x)) ≥ 0. (10.8) ∂t A uniformly continuous on Q function v is a viscosity supersolution to (10.6)–(10.7) if v(T, x) ≥ G(x), x ∈ Rn , and if v−ϕ, where ϕ is a C ∞ function on Q, attains a minimum at (t, x) ∈ Q, then ∂ϕ (t, x) + H(x, ϕx (t, x)) ≤ 0. ∂t

(10.9)

If a function v is both viscosity sub- and supersolution to (10.6)–(10.7) then it is called a viscosity solution to (10.6) and (10.7). We assume that the set U ⊂ Rm is bounded and T > 0 is fixed, and that f, g, G are bounded and uniformly continuous. More precisely, we will require that (A1) f (x, u), x ∈ Rn , u ∈ Rm , is bounded and continuous and for a constant Mf |f (x, u) − f (x, u)| ≤ Mf |x − x|,

x, x ∈ Rn , u ∈ U,

(A2) g(x, u), x ∈ Rn , u ∈ U , G(x), x ∈ Rn , are bounded and continuous on Rn × U and Rn , respectively, with the following modulae of continuity Mg , MG :   Mg (r) = sup |g(x, u) − g(x, u)|; |x − x| ≤ r, u ∈ U ,  MG (r) = sup |G(x) − G(x)|; |x − x| ≤ r}, r ≥ 0, finite and continuous at 0. By K we denote the constant by which |f |, |g|, |G| are bounded. Theorem 10.1. Under the above assumptions (A1), (A2), the value function V is bounded and uniformly continuous on [0, T ] × Rn and is the unique viscosity solution of the Bellman equation (10.6)–(10.7). The proof will be divided into several steps and sections. First we prove the so-called dynamic programming principle (DPP) being an important tool of the proof. Then we establish first regularity of the value function V , and then, in the final two sections, that V is the unique viscosity solution.

§10.2. Dynamic programming principle Our aim is to prove the following result.

156

10 Viscosity solutions of Bellman equations

Theorem 10.2. Under assumptions (A1), (A2), for arbitrary 0 ≤ t ≤ r ≤ T and x ∈ Rn ,

 r g(y(s), U (s)) ds + V (r, y(r)) , (10.10) V (t, x) = inf U ∈U

t

where y are solutions starting at t from x and corresponding to controls U (s), s ∈ [t, T ]. Proof. Denote the left-hand side and the right-hand side of (10.10) by L and P . For arbitrary ε > 0, there exists control U (s), s ∈ [t, T ], such that  L+ε≥

t



r

g(y(s), U (s)) ds + 

and thus L+ε≥

r

t

T

r

g(y(s), U (s)) ds + G(y(T ))

g(y(s), U (s)) ds + V (r, y(r)) ≥ P.

Since ε > 0 was arbitrary, L ≥ P . To show the opposite inequality note that for a given ε > 0 one can find control U (s), s ∈ [t, r], such that  r P +ε≥ g(y(s), U (s)) ds + V (r, y(r)). t

For x = y(r) one can find control U (s), s ∈ [r, T ] such that for the corresponding y(s), s ∈ [r, T ], y(r) = x,  V (r, x) + ε ≥ Thus

 P +ε≥

where

U

T

r

g(y(s), U (s)) ds + G(y(T )).

T

g(y(s), U (s)) ds + G(y(T ))

t

is a control such that  U

=

U (s), U (s),

for s ∈ [t, r[, for s ∈ [r, T ]

and y the corresponding controlled trajectory. Consequently P + 2ε ≥ V (t, x) = L and since ε > 0 was arbitrary, This completes the proof.

P ≥ L. 

10.3 Regularity of the value function

157

§10.3. Regularity of the value function Proposition 10.1. Function V is uniformly continuous and bounded on Q. Proof. The boundedness of V on [0, T ] × Rn follows from the boundedness of f, g and G. We pass to the proof of the uniform continuity of V . Fix t ∈ [0, T ], x, x ∈ Rn and let y(s) = ytx,U (s),

y(s) = ytx¯,U (s),

s ∈ [t, T ],

be trajectories of the control system corresponding to a control function U . Then |V (t, x) − V (t, x)| = | inf Jt (x, U ) − inf Jt (x, U )| U ∈Ut U ∈Ut

 T  g(y(s), U (s)) ds + G(y(T )) = inf U ∈Ut

t

− inf



U ∈Ut

 ≤ sup U ∈Ut

t

≤ sup

U ∈Ut

 g(y(s), U (s)) ds + G(y(T ))

 g(y(s), U (s)) − g(y(s), U (s)) ds + G(y(T )) − G(y(T ))

T

t

 ≤ sup C U ∈Ut

T



T

t T

t

|y(s) − y(s)| ds + C|y(T ) − y(T )|

 Mg (|y(s) − y(s)|) ds + MG (y(T ) − y(T )) .

However, for s ∈ [t, T ],  s   |y(s) − y(s)| = (x − x) + f (y(r),U (r)) − f (y(r), u(r)) dr t  s ≤ |x − x| + Mf |y(r) − y(r)| dr, t

and by Gronwall’s lemma |y(s) − y(s)| ≤ eMf (s−t) |x − x|. Thus we arrive at the estimate |V (t, x) − V (t, x)|  T     Mg eMf (s−t) |x − x| ds + MG eMf (T −t) |x − x| . (10.11) ≤ t

158

10 Viscosity solutions of Bellman equations

If now r ∈ [t, T ] and U ∈ Ut , denote by U the restriction of U to the interval [r, T ]. Then U ∈ Ur and |V (t, x)−V (r, x)| = | inf Jt (x, U )− inf Jr (x, U )| ≤ sup |Jt (x, U )−Jr (x, U )|. U ∈Ut

U ∈Ut

U ∈Ut

However,  |Jt (x, U ) − Jr (x, U )| =

t

≤

f (y(s), U (s)) ds + Jr (y(r), U ) − Jr (x, U )

r



r

t

|f (y(s), U (s))| ds + |Jr (y(r), u) − Jr (x, U )|.

By the first part of the proof and since |y(r) − x| ≤ K|r − t|, sup |Jr (y(r), U ) − Jr (x, U )|

U ∈Ut

 ≤

T

r

    Mg eMf (s−t) |y(r) − x| ds + MG eMf (T −r) |y(r) − x|  ≤

r

T

    Mg eMf (s−t) Kf |r − t| ds + MG eMf (s−t) Kf |r − t| .

Thus  |V (t, x) − V (r, x)| ≤ Kf |r − t| +

r

T

  Mg eMf (s−t) Kf |r − t| ds   + MG eMf (s−t) Kf |r − t| . (10.12)

But V (t, x) − V (r, x)| ≤ |V (t, x) − V (r, x)| + |V (r, x) − V (r, x)| and it is clear by (10.11) and (10.12) that   lim sup |V (t, x) − V (r, x)|; |t − r| + |x − x| ≤ s = 0. s↓0



§10.4. Existence of the viscosity solution Proposition 10.2. Function V is a viscosity solution of the Bellman equation. Proof. V is a viscosity subsolution of the Bellman equation. Assume that at (t, x), t ∈ [0, T ], V − ϕ attains maximum. For r ∈ (t, T ) and arbitrary u ∈ U and solution y of the equation

10.4 Existence of the viscosity solution

159

dy (s) = f (y(s), u), ds

y(t) = x,

1 (V (t, x) − ϕ(t, x)) − (V (r, y(r)) − ϕ(r, y(r))) ≥ 0, r−t and



(10.13)

r

d ϕ(s, y(s)) ds ds t

 r ∂ϕ = ϕ(t, x) + (s, y(s)) + ϕx (s, y(s)), f (y(s), u) ds. ∂s t (10.14)

ϕ(r, y(r)) = ϕ(t, x) +

By the dynamic programming principle  r V (t, x) ≤ g(y(s), u) ds + V (r, y(r)) t

and thus by (10.13) and (10.14)  r 1 g(y(s), u) ds r−t t



 t ∂u (s, y(s)) + ϕx (s, y(s)), f (y(s), u) ds ≥ 0. (10.15) + ∂s r Passing in (10.15) with r ↓ t one gets for all u ∈ U f (x, u) + Therefore

∂ϕ (t, x) + ϕx (t, x), f (x, u) ≥ 0. ∂t

  ∂ϕ (t, x) + inf g(x, u) + ϕx (t, x), f (x, u) ≥ 0 u∈U ∂t

and thus V is a subsolution to (10.6)–(10.7). Proof that V is a supersolution. To show that V is also a supersolution assume that V − ϕ attains its minimum at (t, x). By the dynamic programming principle for any ε > 0 and r ∈ (t, T ) there exists a control U (s), s ∈ [t, T ], such that for the corresponding y(s), s ∈ [t, T ], y(t) = x,  r V (t, x) ≥ −ε(r − t) + g(y(s), U (s)) ds + V (r, y(r)). t

However,     V (t, x) − ϕ(t, x) − V (r, y(r)) − ϕ(r, y(r)) ≥ 0 and thus

160

10 Viscosity solutions of Bellman equations

 0 ≥ −ε(r − t) +

r

t r

g(y(s), U (s)) ds − ϕ(t, x) + ϕ(r, y(r))

g(y(s), U (s)) ds ≥ −ε(r − t) + t

 r ∂ϕ (s, y(s)) + ϕx (s, y(s)), f (y(s), U (s)) ds + ∂s t  r  ∂ϕ (s, y(s)) + inf g(y(s), u) ≥ −ε(r − t) + u ∂s t

 + ϕx (s, y(s)), f (y(s), u) ds. Consequently 1 ε≥ r−t

 r t

  ∂ϕ (s, y(s)) + H y(s), ϕx (s, y(s)) ds. ∂s

But, for s ∈ [t, r],

 y(s) = x +

t

(10.16)

s

f (y(v), U (v)) dv

and |y(s) − x| ≤ K|s − t|,

s ∈ [t, T ].

And, although the solution y depends on ε > 0 and r, we can pass in (10.16) with r ↓ t to obtain ε≥

∂ϕ (t, x) + H(x, ϕx (t, x)). ∂t

Since ε > 0 was arbitrary, we arrive at ∂ϕ (t, x) + H(x, ϕx (t, x)) ≤ 0, ∂t as required.



§10.5. Uniqueness of the viscosity solution The uniqueness of the viscosity solution is a consequence of the following proposition. Proposition 10.3. Under the assumptions (A1), (A2), if W and V are, respectively, viscosity sub- and supersolutions of the Bellman equation (10.6)– (10.7), then W (t, x) ≤ V (t, x), t ∈ [0, T ], x ∈ Rn .

10.5 Uniqueness of the viscosity solution

161

In fact, if V1 and V2 are two viscosity solutions then each of them is both sub- and supersolutions, thus, by the proposition, V1 ≤ V2 and V2 ≤ V1 , consequently V1 = V2 . The proof of Proposition 10.3 is based on the so-called dubling method introduced by Crandall, Ishii and Lions. Define, for arbitrary positive numbers ε, δ, β and (t, x), (s, z) ∈ [0, T ] × Rn the function φ(t, x; s, z) = W (t, x) − V (s, z) −

1 1 |x − z|2 − |t − s|2 − β(T − s), 2ε 2δ

and for γ > 0 let (tγ , xγ ), (sγ , zγ ) be such that φ(tγ , xγ ; sγ , zγ ) ≥ sup φ − γ. Q×Q

In addition define on Q × Q: φγ (t, x; s, z) = φ(t, x; s, z) −

 γ |t − tγ |2 + |s − sγ |2 + |x − xγ |2 + |z − zγ |2 . 2

The function φ may not attain its supremum value contrary to the function φγ . Let ((t, x); (s, z)) be a point where φγ attains its maximal value. Since φγ (tγ , xγ ; sγ , zγ ) = φ(tγ , xγ ; sγ , zγ ) ≥ sup φ − γ, Q×Q

2

2

2

2

and if |t−tγ | +|s−sγ | +|x−xγ | +|z−zγ | > 2, then φγ (t, x; s, z) ≤ sup φ−γ, Q×Q

therefore

|t − tγ |2 + |s − sγ |2 + |x − xγ |2 + |z − zγ |2 ≤ 2.

(10.17)

We have the obvious relation sup φγ ≤ sup φ ≤ sup φγ − γ.

Q×Q

Q×Q

Q×Q

Let MW and MV be the modulae of continuity of functions W and V , and K the supremum value of MW . Thus for r ≥ 0   MW (r) = sup |W (t, x) − W (s, z)|; |t − s| + |x − z| ≤ r, (t, x), (s, z) ∈ Q ,   MV (r) = sup |V (t, x) − V (s, z)|; |t − s| + |x − z| ≤ r, (t, x), (s, z) ∈ Q , K = sup{MW (r); t ≥ 0}, and by absolute continuity of W and V and boundedness of W lim MW (r) = lim MV (r) = 0, r↓0

r↓0

We assume that ε, δ and γ are smaller than estimates:

K < +∞. 1 2

and show the following

162

10 Viscosity solutions of Bellman equations

 |t − s| ≤ 2 δ(K + 1)      |x − z| ≤ 2 ε γ + MW 2 ε(K + 1) .

(10.18) (10.19)

Since φγ (t, x; s, z) ≥ φγ (s, x; s, z), we have 1 γ |t − s|2 ≤ W (t, x) − W (s, x) + [|s − tγ |2 − |t − tγ |2 ]. 2δ 2 In addition, |s−tγ |2 −|t−tγ |2 = |(s−t)+(t−tγ )|2 −|t−tγ |2 ≤ 2|s−t|2 +|t−tγ |2 and thus, compare (10.2): 1 γ |t − s|2 ≤ W (t, x) − W (s, x) + γ|s − t|2 + |t − tγ |2 2δ 2 ≤ K + γ|s − t|2 + γ ≤ (K + 1) + γ|s − t|2 1 1 |t − s|2 ≤ δ(K + 1) + δγ|t − s|2 ≤ δ(K + 1) + |t − s|2 . 2 4 We arrive at (10.18). Similarly starting from φγ (t, x; s, z) ≥ φγ (t, z; s, z) we have 1 γ |x − z|2 ≤ W (t, x) − W (t, z) + [|z − xγ |2 − |x − zg |2 ] 2ε 2 γ ≤ MW (|x − z|) + γ|x − z|2 + |x − xγ |2 2 ≤ MW (|x − z|) + γ|x − z|2 + γ ≤ MW (|x − z|) + 1 + γ|x − z|2 . Thus 1 |x − z|2 ≤ ε(MW (|x − z|) + 1) + εγ|x − z|2 2 1 ≤ ε(MW (|x − z|) + 1) + |x − z|2 4 and

 |x − z| ≤ 2 ε(K + 1) .

Thus

 MW (|x − z|) ≤ MW (2 ε(K + 1))

and repeating again the above estimations we arrive at    1 |x − z|2 ≤ MW (2 ε(K + 1)) + γ + γ|x − z|2 . 2ε Finally

    |x − z| ≤ 2 ε γ + MW (2 ε(K + 1))

10.5 Uniqueness of the viscosity solution

163 2

and (10.19) holds. It will be used later that |x−z| can be made small if γ ε and ε are small. The next step is to see the convergence when t = T or s = T , that is, when maxima are attained on the upper boundary of Q. If t = T , then W (t, x) − V (t, x) ≤ 0, and Φγ (t, x; s, z) ≤ W (t, x) − V (s, z) ≤ V (t, x) − V (s, z) + W (t, x) − V (t, x) ≤ MV (|t − s| + |x − z|). Thus by (i) and (ii)  √ √ √  φγ (t, x; s, z) ≤ MV 2 K + 1( δ + ε) . Similarly, if s = T φγ (t, x; s, z) ≤ MW (|t − s| + |x − z|) √  √ √  ≤ MW 2 K + 1( δ + ε) . Finally we consider the case when t < T and s < T . If ϕ(t, x) =

 1 1 γ |t − s|2 + |x − z|2 + |t − tγ |2 + |x − xγ |2 , 2δ 2ε 2

then the function W − ϕ attains its maximal value at (t, x). In fact 1 1 W (t, x) − ϕ(t, x) = W (t, x) − |t − s|2 − |x − z|2 2δ 2ε  γ − |t − tγ |2 + |s − sγ |2 + |x − xγ |2 + |y − yγ |2 2 and  γ |t − tγ |2 + |s − sγ |2 + |x − xγ |2 + |z − zγ |2 2 1 1 = W (t, x) − V (s, z) − |x − z|2 − |t − s|2 − β(T − s) 2ε 2δ  γ − |t − tγ |2 + |s − sγ |2 + |x − xγ |2 + |y − yγ |2 2 = W (t, x) − ϕ(t, x) + constant.

φγ (t, x; s, z) = φ(t, x; s, z) −

Since the maximal value of φγ (t, x; s, z) is attained at (t, x), the same is true for W − ϕ. By the definition of viscosity subsolution ∂ϕ (t, x) + H(x, ϕx (t, x)) ≥ 0. ∂t

164

10 Viscosity solutions of Bellman equations

We have ∂ϕ 1 (t, x) = (t − s) + γ(t − tγ ) ∂t δ 1 ϕx (t, x) = (x − z) + γ(x − xγ ), ε therefore

   1 1 (t−s)+γ(t−tγ ) = inf g(x, u)+ f (x, u), (x−z)+γ(x−xγ ) . (10.20) u∈U δ ε

Similarly if V is a viscosity supersolution, define ϕ(s, z) = −

1 1 γ γ |t − s|2 − |x − z|2 − |s − sγ |2 − |z − zγ |2 − β(T − s), 2δ 2ε 2 2

then V (s, z) − ϕ(s, z) attains its minimum at (s, z). Thus ∂ϕ (s, z) + H(z, ϕz (s, z)) ≤ 0 ∂s and consequently 1 (t − s) − γ(s − sγ ) + β δ    1 ≤0 + inf g(z, u) + f (z, u), (x − z) − γ(z − zγ ) u∈U ε or equivalently 1 β ≤ − (t − s) + γ(s − sγ ) δ   1 ≤0 − inf g(z, u) + f (z, u), (x − z) − γ(z − zγ ) u∈U ε Adding (10.20) and (10.21): β ≤ γ(s − sγ ) + γ(t − tγ )    1 + inf g(x, u) + f (x, u), (x − z) + γ(x − xγ ) u ε    1 − inf g(z, u) + f (z, u), (x − z) − γ(z − zγ ) . u ε Thus

(10.21)

10.5 Uniqueness of the viscosity solution

165

β ≤ γ(s − sγ ) + γ(t − tγ )  1 + sup g(x, u) − g(z, u) + f (x, u) − f (z, u), x − z ε u∈U

    + γ f (x, u), (x − xγ ) + f (z, u), z − zγ and finally 1 β ≤ γ(|s − sγ | + |t − tγ |) + Mg (|x − z|) + Mf |x − z|2 + 2γKf ε

(10.22)

where Kf is the bound on f and by (10.17) |x − xγ | + |z − zγ | ≤

√  1/2 2 |x − xγ |2 + |z − zγ |2 ≤ 2.

For fixed β > 0 and ε > 0 one can find δ0 > 0 and γ0 > 0 such that for γ ∈ (0, γ0 ), δ ∈ (0, δ0 ) the inequality is not satisfied and thus one of the inequalities √ √ √ φγ (t, x; s, z) ≤ MW (2 K + 1( δ + ε)) √ √ √ φγ (t, x; s, z) ≤ MV (2 K + 1( δ + ε)) must hold. By the definition of φγ we see that for arbitrary (t, x), (s, z) ∈ Q φ(t, x; s, z) ≤ φg (t, x; s, z) − γ. Consequently φ(t, x; t, x) = W (t, x) − V (t, x) − β(T − t) √ √  √  √ √  √  ≤ γ + MW 2 K + 1( δ + ε) + MV 2 K + 1( δ + ε) . (10.23) Choosing initially ε > 0 close to 0, we can make the right-hand side of (10.23) close to 0 by the choice of δ and γ. This way we get W (t, x) − V (t, x) − β(T − t) ≤ 0. Since β was an arbitrary positive number and t ∈ [0, T ] we see that W (t, x) − V (t, x) ≤ 0 for all (t, x) ∈ Q, as required.



166

10 Viscosity solutions of Bellman equations

Bibliographical notes The notion of a viscosity solution for first order Hamilton–Jacobi equation was introduced by M.G. Crandall and P.-L. Lions in [21] and extensively studied there. P.-L. Lions in [67] applied it to deterministic control problems. In our presentation we follow J. Yong and X.Y. Zhou [102] and W.H. Fleming and H.M. Soner [38]. For a connection between the theory of viscosity solution and the nonsmooth analysis see the book [17] by F.H. Clarke, Y.S. Ledyaev, R.J. Stern and P.R. Wolenski.

Chapter 11

Dynamic programming for impulse control

The dynamic programming approach is applied to impulse control problems. The existence of optimal impulse strategy is deduced from general results on fixed points for monotonic and concave transformations.

§11.1. Impulse control problems Let us consider a differential equation z(0) = x ∈ Rn ,

z˙ = f (z),

(11.1)

with the right-hand side f satisfying the Lipschitz condition, and let us denote the solution to (11.1) by z x (t), t ≥ 0. Assume that for arbitrary x ∈ Rn a nonempty subset Γ(x) ⊂ Rn is given such that Γ(w) ⊂ Γ(x)

for arbitrary w ∈ Γ(x).

(11.2)

Let, in addition, c(·, ·) be a positive bounded function defined on the graph {(x, v); x ∈ Rn , v ∈ Γ(x)} of the multifunction Γ(·), satisfying the following conditions: c(x, v) + c(v, w) ≥ c(x, w), x ∈ Rn , v ∈ Γ(x), w ∈ Γ(v), there exists γ > 0, such that c(x, w) ≥ γ for x ∈ Rn , w ∈ Γ(x).

(11.3) (11.4)

Elements of the set Γ(x) can be interpreted as those states of Rn to which one can shift immediately the state x and c(x, v), the cost of the shift from x to v. An impulse strategy π will be defined as a set of three sequences: a nondecreasing sequence (tm ) of numbers from [0, ∞) and two sequences (xm ) and (wm ) of elements of Rn ∪ {Δ} (where Δ is an additional point outside of Rn ), satisfying the following conditions: © Springer Nature Switzerland AG 2020

J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_11

167

168

11 Dynamic programming for impulse control

if tm < +∞,

then tm+1 > tm ;

(11.5)

if tm = +∞, then xm = zm = Δ; if t1 = 0, then x1 = x and w1 ∈ Γ(x);

(11.6) (11.7)

if 0 < t1 + ∞, then x1 = z x (t1 ), w1 ∈ Γ(x1 ) and if m > 1, and tm < +∞, then xm = z wm−1 (tm − tm−1 ), wm ∈ Γ(xm ).

(11.8)

The number tm should be interpreted as the moment of the m-th impulse, identical with an immediate shift from xm to wm . An arbitrary impulse trajectory π determines, together with the initial state x, the output strategy y π,x (t), t ≥ 0, by the formulae y π,x (t) = z x (t) for t ∈ [0, t1 ), y π,x (t) = z wm−1 (t − tm−1 ) for t ∈ [tm−1 , tm ), y π,x (+∞) = Δ.

m = 2, 3, . . . ,

With the strategy π we associate the following cost functional :  JT (π, x) =

0

T

e−αt g(y π,x (t)) dt +



e−αtm c(xm , wm ),

α > 0,

tm ≤T

where g is a nonnegative function, α is a positive constant and T ≤ +∞. We assume also that c(Δ, Δ) = 0. It is our aim to find a strategy minimizing JT (·, x). We consider first the case of the infinite control interval, T = +∞. The case of finite T < +∞ will be analysed in §12.3 by the maximum principle approach. Instead of JT we write simply J. An important class of impulse strategies form the stationary strategies. Optimal strategies often are stationary. To define a stationary strategy we start from a closed set K ⊂ Rn and a mapping S : K −→ Rn such that S(x) ∈ Γ(x)\K, x ∈ K, and define sequences (tm ), (xm ) and (wm ) as follows. Let the starting point be x. If x ∈ k then t1 = 0, x1 = x and w1 = S(x1 ). If x ∈ / K then t1 = inf{t ≥ 0; z x (t) ∈ K} and if t1 < +∞ then x1 = z x (t1 ) and w1 = S(x1 ); if t1 = +∞ then we define x1 = Δ, w1 = Δ. Suppose that sequences t1 , . . . , tm , x1 , . . . , xm and w1 , . . . , wm have been defined. We can assume that tm < +∞. Let us define t˜m = inf{t ≥ 0; z wm (t) ∈ K}, tm+1 = tm + t˜m , and if t˜m < +∞ then xm+1 = z wm (t˜m ), wm+1 = S(xm+1 ); if t˜m = +∞ then we set tm+1 = +∞, xm+1 = Δ = wm+1 . To formulate the basic existence result it is convenient to introduce two more conditions. For an arbitrary nonnegative and bounded continuous function v,

11.2 An optimal stopping problem

169

M v(x) = inf{v(z) + c(x, z); z ∈ Γ(x)}, x ∈ Rn , is continuous.

(11.9)

n

For arbitrary x ∈ R , the set Γ(x) is compact and the function c(x, ·) is continuous on Γ(x).

(11.10)

Theorem 11.1. Assume that g is a nonnegative, continuous and bounded function and that f satisfies the Lipschitz condition and conditions (11.9) and (11.10). Then (i) The equation  v(x) = inf

t≥0

0

t

e−αs g(z x (s)) ds + e−αt M v(z x (t)),

x ∈ Rn ,

(11.11)

has exactly one nonnegative, continuous and bounded solution v = vˆ. (ii) For arbitrary x in the set ˆ = {x; vˆ(x) = M vˆ(x)}, K ˆ ˆ such that M vˆ(x) = vˆ(x) = c(x, S(x))+ ˆ there exists an element S(x) ∈ Γ(ˆ x)\K ˆ ˆ and Sˆ is an optimal vˆ(S(x)), and the stationary strategy π ˆ determined by K one. Remark. Equation (11.11) is the Bellman equation corresponding to the problem of the impulse control on the infinite time interval. In a heuristic way one derives it as follows. Assume that v(x) is the minimal value of the function J(x, ·), and consider a strategy which consists first of the shift at moment t ≥ 0 from the current state z x (t) to a point w ∈ Γ(z x (t)) and then of an application of the optimal strategy corresponding to the new initial state w. The total cost associated with this strategy is equal to  t e−αs g(z x (s)) ds + e−αt [c(z x (t), w) + v(w)]. 0

Minimizing the above expression with respect to t and w we obtain v(x), hence equation (11.11) follows. These arguments are incomplete because in particular they assume the existence of optimal strategies for an arbitrary initial condition x ∈ Rn . To prove the theorem we will need several results of independent interest.

§11.2. An optimal stopping problem Let us assume that a continuous and bounded function ϕ on Rn is given, and consider, for arbitrary x ∈ Rn , the question of finding a number t ≥ 0, at which the infimum

170

11 Dynamic programming for impulse control

inf{e−αt ϕ(z x (t)); t ≥ 0} = Φ(x),

(11.12)

is attained as well as characterizing the minimal value Φ(x). The following result holds. Lemma 11.1. Assume that f is Lipschitz continuous, ϕ continuous and bounded and α a positive number. Then the function Φ defined by (11.12) is continuous. Let K = {x ∈ Rn ; Φ(x) = ϕ(x)} and  inf{t ≥ 0; z x (t) ∈ K}, t(x) = / K for all t ≥ 0. +∞, if z x (t) ∈ If t(x) < +∞ then Φ(x) = e−αt(x) ϕ(z x (t(x))); otherwise ϕ(z x (t)) > 0 for t > 0, and Φ(x) = 0. Proof. Note that for x, w ∈ Rn     Φ(x) − Φ(w) ≤ sup e−αt |ϕ(z x (t)) − ϕ(z w (t))| . t≥0

If xm −→ w, then z xm −→ z w uniformly on an arbitrary finite interval [0, T ], T < +∞. The continuity of ϕ and positivity of α imply that Φ(xm ) −→ Φ(w). Therefore Φ is a continuous function and the set K is closed. Note also that if t ≤ t(x) < +∞ then (11.13) Φ(x) = e−αt Φ(z x (t)). This follows from the identities Φ(z x (t)) = inf e−αs Φ(z x (s + t)) s≥0

= eαt inf e−α(s+t) Φ(z x (s + t)) = eαt Φ(x). s≥0

The formula (11.13) implies the remaining parts of the lemma.

2

Lemma 11.1 states that the solution to the problem (11.12) is given by the first hitting time of the coincidence set K = {x; Φ(x) = ϕ(x)}.

§11.3. Iterations of convex mappings The existence of a solution to equation (11.11) will be a corollary of the general results on convex transformations. Let K be a convex cone included in a linear space L. Hence if x, y ∈ K and α, β ≥ 0 then αx + βy ∈ K. In L we introduce a relation ≥:

11.4 The proof of Theorem 11.1

171

x ≥ y if and only if x − y ∈ K. Let A be a transformation from K into K. We say that A is monotonic if for arbitrary x ≥ y, A(x) ≥ A(y). In a similar way the transformation A is called concave if for arbitrary x, y ∈ K and α, β ≥ 0, α + β = 1, A(αx + βy) ≥ αA(x) + βA(y). Lemma 11.2. Assume that a mapping A : K −→ K is monotonic and concave. If for an element h ∈ K, h ≥ A(h), and for some δ > 0, δh ≤ A(0), then there exists γ ∈ (0, 1) such that 0 ≤ Am−1 (h) − Am (h) ≤ γ m Am−1 (h)

for m = 1, 2, . . . .

(11.14)

Proof. We can assume that δ ∈ (0, 1). We show that (11.14) holds with γ = 1 − δ. For m = 1 inequalities (11.14) take the forms 0 ≤ h − A(h) ≤ γh, and therefore (11.14) follows from the assumptions. If (11.14) holds for some m then also (1 − γ m )Am−1 (h) + γ m 0 ≤ Am (h). It follows, from the monotonicity and concavity of A, that Am+1 (h) ≥ (1 − γ m )Am (h) + γ m A(0) or, equivalently, that Am (h) − Am+1 (h) ≤ γ m Am (h) − γ m A(0). It suffices therefore to show that γ m Am (h) − γ m A(0) ≤ γ m+1 Am (h), or that

(1 − γ)Am (h) ≤ A(0).

(11.15)

Since (1 − γ) = δ and Am (h) ≤ h, the inequality (11.15) holds. This finishes the proof of the lemma. 

§11.4. The proof of Theorem 11.1 (i) We have to prove the existence of a solution to (11.11) in the class of all nonnegative continuous and bounded functions v. For this purpose, define L as the space of all continuous and bounded functions on Rn and K ⊂ L as

172

11 Dynamic programming for impulse control

the cone of all nonnegative functions from L. For arbitrary v ∈ K we set  t Av(x) = inf e−αs g(z x (s)) ds + e−αt M v(z x (t)) . t≥0

0

Then equation (11.11) is equivalent to v(x) = Av(x),

x ∈ Rn .

Since g ≥ 0, the solution v has to be nonnegative, and a function h defined by the formula  +∞ h(x) = e−αs g(z x (s)) ds, x ∈ Rn , 0

is in K. We check easily that, for x ∈ Rn , Av(x) = h(x) + inf (e−αt (M v − h)(z x (t))). t≥0

(11.16)

Therefore, by (11.9) and Lemma 11.1, the function A(v) is bounded and continuous for arbitrary v ∈ K. Hence A transforms K into K. Monotonicity of M implies that A is monotonic. To show that the transformation A is concave, remark that for nonnegative numbers β, γ, β + γ = 1 and for functions v1 , v2 ∈ K, M (βv1 + γv2 )(x) ≥ βM v1 (x) + γM v2 (x),

x ∈ Rn .

Therefore, for arbitrary t ≥ 0  t e−αs g(z x (s)) ds + e−αt M (βv1 + γv2 )(z x (t)) 0 

 t e−αs g(z x (s)) ds + e−αt M v1 (z x (t)) ≥β 0 

 t e−αs g(z x (s)) ds + e−αt M v2 (z x (t)) +γ 0

≥ βAv1 (x) + γAv2 (x),

x ∈ Rn .

By the definition of A, A(βv1 + γv2 )(x) ≥ βAv1 (x) + γAv2 (x),

x ∈ Rn ,

the required concavity. It follows, from the definition of h, that A(h) ≤ h. On the other hand, for v = 0 and arbitrary t ≥ 0,

11.4 The proof of Theorem 11.1



t

0

173

e−αs g(z x (s)) ds + e−αT M (0)(x)  t e−αs g(z x (s)) ds + e−αt γ ≥ h(x) + e−αt (γ − h)(z x (t)), ≥ 0

where γ is a positive lower bound for c(·, ·). Let δ ∈ (0, 1) be such that δh ≤ γ. Since h(x) ≥ e−αt h(z x (t)), t ≥ 0, so finally  0

t

e−αs g(z x (s)) ds + e−αt M (0)(x) ≥ h(x) + (δ − 1)e−αt h(z x (t)) ≥ δh(x),

and A(0) ≥ δh. All the conditions of Lemma 11.2 are satisfied and therefore the sequence (Am (h)) is uniformly convergent to a continuous, bounded and nonnegative function vˆ. It is not difficult to see that if v2 ≥ v1 ≥ 0 then 0 ≤ A(v2 ) − A(v1 ) ≤ v2 − v1 . In a similar way, for m = 1, 2, . . . , 0 ≤ Am+1 (h) − A(ˆ v ) ≤ Am h − vˆ , and therefore (ˆ v ) = A(ˆ v ). This way we have shown that there exists a solution of (11.11) in the set K. This finishes the proof of (i). (ii) Since vˆ ∈ K, the set ˆ = {x; vˆ(x) = M vˆ(x)} K is closed by condition (11.9). ˆ with the properties required We show first that there exists a function S(·) by the theorem. By (11.10), function c(x, ·) + vˆ(·) attains its minimum at the point w1 ∈ Γ(x). Hence c(x, w1 ) + vˆ(w1 ) = vˆ(x). ˆ then there exists an element w2 ∈ Γ(w1 ) such that If w1 ∈ K, c(w1 , w2 ) + vˆ(w2 ) = vˆ(w1 ). More generally, assume that w2 , . . . , wm are elements such that c(wi , wi+1 ) + vˆ(wi+1 ) = vˆ(wi ) ˆ i = 1, 2, . . . , m − 1. Adding the above identities we and wi+1 ∈ Γ(wi ) ∩ K, obtain c(x, w1 ) + c(w1 , w2 ) + . . . + c(wm−1 , wm ) + vˆ(wm ) = vˆ(w1 ).

174

11 Dynamic programming for impulse control

Since vˆ is nonnegative and c(·, ·) ≥ γ > 0, γm ≤ vˆ(w1 ). ˆ In addition Therefore for some m, wm ∈ Γ(wm−1 ) and wm ∈ / K. c(wm−1 , wm ) + vˆ(wm ) = vˆ(wm−1 ). Since wm ∈ Γ(x), by (11.3), c(x, w1 ) + . . . + c(wm−1 , wm ) ≥ c(x, wm ), c(x, wm ) + vˆ(wm ) ≤ vˆ(x) and c(x, wm ) + vˆ(wm ) = vˆ(x). Hence a function Sˆ with the required properties exists. We finally show that if vˆ ∈ K is a solution to (11.11) then, for arbitrary ˆ S, ˆ ˆ determined by K, x ∈ Rn and the stationary strategy π J(ˆ π , x) = vˆ(x). Let tˆ1 , tˆ2 , . . . and (ˆ x1 , w ˆ1 ), (ˆ x2 , w ˆ2 ), . . . be elements of the stationary strategy. Then for arbitrary finite tˆm  vˆ(x) =

tˆm

0

e−αs g(y πˆ ,x (s)) ds +

m 

ˆ

ˆ

e−αti c(ˆ xi , w ˆi ) + e−αtm vˆ(w ˆm ). (11.17)

i=1

ˆ To check that (11.17) holds assume first that m = 1. If tˆ1 = 0, then x = x ˆ1 ∈ K ˆ x1 ). Hence and w ˆ1 = S(ˆ vˆ(x) = vˆ(ˆ x1 ) = c(ˆ x1 , w ˆ1 ) + vˆ(w ˆ1 ). ˆ and tˆ1 = inf{t ≥ 0; z x (t) ∈ K}. ˆ It follows from If 0 < tˆ1 < +∞ then x ∈ /K equation (11.11) and Lemma 11.1 that  vˆ(x) =

0

tˆ1

ˆ e−αs g(z x (s)) ds + e−αt0 M vˆ(z x (tˆ1 )).

ˆ x1 ) and ˆ1 = S(ˆ However x ˆ1 = z x (tˆ1 ), w vˆ(ˆ x1 ) = M vˆ(ˆ x1 ) = c(ˆ x1 , w ˆ1 ) + vˆ(w ˆ1 ); consequently,  vˆ(x) =

0

tˆ1

ˆ

ˆ

e−αs g(z x (s)) ds + e−αt1 c(ˆ x, zˆ) + e−αt1 vˆ(ˆ z1 ).

11.4 The proof of Theorem 11.1

175

Similarly, if (11.17) holds for some m and tˆm+1 < +∞ then ˆ = tˆ(w tˆm+1 − tˆm = inf{t ≥ 0; z wˆm (t) ∈ K} ˆm ) and, by the Bellman equation (11.11),  vˆ(w ˆm ) =

tˆ(w ˆm )

0

ˆ

e−αs g(z wˆm (s)) ds + e−α(t(wˆm )) M vˆ(z wˆm (tˆ(w ˆm ))).

Since y wˆm (tˆ(w ˆm )) = x ˆm+1 and vˆ(ˆ xm+1 ) = M vˆ(ˆ xm+1 ) = c(ˆ xm+1 , w ˆm+1 ) + vˆ(w ˆm+1 ), therefore  vˆ(x) =

tˆm

0

e−αs g(y πˆ ,x (s)) ds + ˆ

+ e−αtm

m 

ˆ

e−αti c(ˆ xi , w ˆi )

i=0



tˆm+1 −tˆm

0

e−αs g(z wˆm (s)) ds

 ˆ ˆ  xm+1 , w ˆm+1 ) + vˆ(w ˆm+1 ) . + e−α(tm+1 −tm ) c(ˆ By the very definition of the strategy π ˆ we see that (11.17) holds for m + 1 and thus for all m = 1, 2, . . . . In exactly the same way we show that for arbitrary strategy π and finite tm  vˆ(x) ≤

tm

0

e−αs g(y π,x (s)) ds +

m 

e−αti c(xi , wi ) + e−αtm vˆ(wm ).

i=0

To prove that vˆ(x) = J(ˆ π , x), assume, for instance, that tˆm < +∞ for m = 1, 2, . . . . Since +∞ > vˆ(x) ≥

m 

ˆ

e−αti c(ˆ xi , w ˆi ) ≥ γ

i=0

m 

ˆ

e−αti ,

i=0

tˆm ↑ +∞. Letting m tend to +∞ in (11.17), we obtain  vˆ(x) =

0

+∞

e−αs g(y πˆ ,x (s)) ds +

+∞ 

ˆ

e−αti c(ˆ xi , w ˆi ).

i=0

ˆ for arbitrary t ≥ 0. Lemma 11.1 and Similarly, if tˆ1 = +∞ then z x (t) ∈ / K equation (11.11) imply

176

11 Dynamic programming for impulse control

 vˆ(x) =

+∞

0

e−αs g(z x (s)) ds = J(ˆ π , x).

ˆ for all t ≥ 0, If, finally, tˆ1 < tˆ2 < . . . < tˆm < tˆm+1 = +∞, then z wˆm (t) ∈ /K and  +∞ e−αs g(z wˆm (s)) ds. vˆ(w ˆm ) = 0

By the identity (11.17) we obtain  vˆ(x) =

0

tˆm

e−αs g(y πˆ ,x (s)) ds + ˆ

+ e−αtm

m 

ˆ

e−αti c(ˆ xi , w ˆi )

i=0



+∞

0

e−αs g(z wˆm (s)) ds = J(ˆ π , x).

Arguments of a similar type allow one to show that, for arbitrary strategy π, J(π, x) ≥ vˆ(x). This way we have shown that the strategy π ˆ is optimal and at the same time that vˆ is the unique solution of (11.11). 

Bibliographical notes There is a vast literature devoted to impulse control problems. We refer the reader to the monograph by A. Bensoussan and J.-L. Lions [8] and to the author survey [113] for more information and references. Theorem 11.1 is a special case of a result due to M. Robin and Lemma 11.2 to B. Hanouzet and J.L. Joly, see, e.g. J. Zabczyk [113]. The abstract approach to dynamic programming equations based on monotonicity and concavity is due to the author, see J. Zabczyk [103], [104], [108] and references there.

Chapter 12

The maximum principle

The maximum principle is formulated and proved first for control problems with fixed terminal time. As an application, a new derivation of the solution to the linear regulator problem is given. Next the maximum principle for impulse control problems and for time-optimal problems are discussed. In the latter case an essential role is played by the separation theorems for convex sets.

§12.1. Control problems with fixed terminal time We start our discussion of the second approach to the optimality questions (see §9.1) by considering a control system y˙ = f (y, u),

y(0) = x ∈ Rn ,

(12.1)

and the cost functional  JT (x, u(·)) =

T

g(y(t), u(t)) dt + G(y(T ))

(12.2)

0

with T > 0, a fixed, finite number. The set of control parameters will be denoted, as before, by U ⊂ Rm , the derivative of f , with respect to the state variable, by fx and derivatives of g and G by gx and Gx . Here is a version of the maximum principle. Theorem 12.1. Assume that functions f , g, G and fx , gx , Gx are continuous and that a bounded control u ˆ(·) and the corresponding absolutely continuous solution of (12.1) maximize the functional (12.2). For arbitrary t ∈ (0, T ), − such that the left derivative ddt yˆ(t) exists and is equal to f (ˆ y (t), u ˆ(t)), the following inequality holds: © Springer Nature Switzerland AG 2020

J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_12

177

178

12 The maximum principle

p(t), f (ˆ y (t), u ˆ(t)) + g(ˆ y (t), u ˆ(t)) ≥ max(p(t), f (ˆ y (t), u) + g(ˆ y (t), u)). u∈U

(12.3)

In (12.3), p(t), t ∈ [0, T ], stands for the absolutely continuous solution of the equation y, u ˆ)p − gx∗ (ˆ y, u ˆ) (12.4) p˙ = −fx∗ (ˆ with the end condition p(T ) = Gx (ˆ y (T )).

(12.5)

Remark. Since an arbitrary absolutely continuous function is differentiable almost everywhere, the inequality (12.3) holds for almost all t ∈ [0, T ]. Proof. Assume first that g(x, y) = 0 for x ∈ E, u ∈ U . Let, for some t0 ∈ (0, T ), d− yˆ(t0 ) = f (ˆ y (t0 ), u ˆ(t0 )). dt For arbitrary control parameter v ∈ U and sufficiently small h ≥ 0 define the needle variation of u ˆ by the formula ⎧ ⎪ ˆ(t) if t ∈ [0, t0 − h), ⎨u u(t, h) = v if t ∈ [t0 − h, t0 ), ⎪ ⎩ u ˆ(t) if t ∈ [t0 , T ]. Let y(·, h) be the output corresponding to u(·, h). It is clear that d+ G(y(T, 0)) ≤ 0, dh

(12.6)

provided that the right derivative in (12.6) exists. To prove that the derivative exists and to find its value, let us remark first that  t0 y(t0 , h) = yˆ(t0 − h) + f (y(s, h), v) ds, t0 −h t0

 yˆ(t0 ) = yˆ(t0 − h) +

f (ˆ y (s), u ˆ(s)) ds.

t0 −h

Taking this into account, we obtain y(t0 , h) − yˆ(t0 ) d+ y(t0 , 0) = lim h↓0 dh h y(t0 , h) − yˆ(t0 − h) yˆ(t0 ) − yˆ(t0 − h) − lim = lim h↓0 h↓0 h h y (t0 ), u ˆ(t0 )). = f (ˆ y (t0 ), v) − f (ˆ

(12.7)

12.1 Control problems with fixed terminal time

179

d Moreover, dt y(t, h) = f (y(t, h), u ˆ(t)) for almost all t ∈ [t0 , T ], so by Theorem II.6.4, Theorem I.1.1 and by (12.7)

  d+ y(T, 0) = S(T )S −1 (t0 ) f (ˆ y (t0 ), v) − f (ˆ y (t0 ), u ˆ(t0 )) , dh where S(·) is the fundamental solution of q˙ = fx (ˆ y, u ˆ)q. Therefore, the right derivative in (12.6) exists and

d+ d+ G(y(T, 0)) = Gx (ˆ y(T, 0) y (T )), dh dh  = Gx (ˆ y (T )), S(T )S −1 (t0 )(f (ˆ y (t0 ), v) − f (ˆ y (t0 ), u ˆ(t0 )))  y (T )), (f (ˆ y (t0 ), v) − f (ˆ y (t0 ), u ˆ(t0 ))) . = S ∗ (t0 )−1 S ∗ (T )Gx (ˆ Since the fundamental solution of the equation p˙ = −fx∗ (ˆ y, u ˆ)p is equal to (S (t))−1 , t ∈ [0, T ], formula (I.1.7) and inequality (12.6) finish the proof of the theorem in the case when g(x, u) = 0, x ∈ Rn , u ∈ U . The case of the general function g can be reduced to the considered one. To do so we introduce an additional variable w ∈ R1 and define

         x f (x, u) x ˜ x = G(x) + w, f˜ ,u = , G ∈ Rn+1 , u ∈ U. w g(x, u) w w ∗

It is easy to see that the problem of maximization of the function (12.2) for the system (12.1) is equivalent to the problem of maximization of   y(T ) ˜ G w(T ) for the following control system on Rn+1 ,  

     d y y f (y, u) ˜ =f ,u = w g(y, u) dt w with the initial condition



   y(0) x = ∈ Rn+1 . w(0) 0

Applying the simpler version of the theorem, which has just been proved, we obtain the statement of the theorem in the general case. 2 The obtained result can be reformulated in terms of the Hamiltonian function: H(x, p, u) = p, f (x, u) + g(x, u),

x ∈ Rn , u ∈ U, p ∈ Rn .

(12.8)

180

12 The maximum principle

Equations (12.1) and (12.4) are of the following form: ∂H (y, p, u), y(0) = x, ∂p ∂H p˙ = − (y, p, u), p(T ) = Gx (y(T )), ∂x

y˙ =

(12.9) (12.10)

and condition (12.3) is equivalent to max H(ˆ y , pˆ, u) = H(ˆ y , pˆ, u ˆ) u∈U

(12.11)

at appropriate points t ∈ (0, T ).

§12.2. An application of the maximum principle We will illustrate the maximum principle by applying it to the regulator problem of §9.3. Assume that u ˆ is an optimal control for the linear regulator problem and yˆ the corresponding output. Since f (x, u) = Ax + Bu, g(x, u) = Qx, x + Ru, u, G(x) = P0 x, x, x ∈ Rn , u ∈ Rm , equation (12.4) and the end condition (12.5) are of the form p˙ = −A∗ p − 2Qˆ y in [0, T ], p(T ) = 2P0 yˆ(T ).

(12.12) (12.13)

In the considered case we minimize the cost functional (12.2) rather than maximize, and, therefore, inequality (12.3) has to be reversed, and the maximization operation should be changed to the minimization one. Hence for almost all t ∈ [0, T ] we have   y (t) + Bu(t) + Qˆ y (t), yˆ(t) + Ru, u minm p(t), Aˆ u∈R

= p(t), Aˆ y (t) + B u ˆ(t) + Qˆ y (t), yˆ(t) + Rˆ u(t), u ˆ(t). It follows from Lemma 9.2 that the minimum in the above expression is attained at exactly one point − 12 R−1 B ∗ p(t). Consequently, 1 u ˆ(t) = − R−1 B ∗ p(t) for almost all t ∈ [0, T ]. 2 Taking into account Theorem 12.1 we see that the optimal trajectory yˆ and the function p satisfy the following system of equations:

12.2 An application of the maximum principle

181

1 yˆ˙ = Aˆ y − BR−1 B ∗ p, yˆ(0) = x, 2 p˙ = −2Qˆ y − A∗ p, p(T ) = 2P0 yˆ(T ).

(12.14) (12.15)

System (12.14)–(12.15) has a unique solution and therefore determines the optimal trajectory. The exact form of the solution to (12.14)–(12.15) is given by Lemma 12.1. Let P (t), t ≥ 0, be the solution to the Riccati equation (9.18). Then p(t) = 2P (T − t)ˆ y (t), ˙yˆ(t) = (A − BR−1 B ∗ P (T − t))ˆ y (t),

(12.16) t ∈ [0, T ].

(12.17)

Proof. We show that (12.14) and (12.15) follow from (12.16) and (12.17). One obtains (12.14) by substituting (12.16) into (12.17). Moreover, taking account of the form of the Riccati equation we have p(t) ˙ = −2P˙ (T − t)ˆ y (t) + 2P (T − t)yˆ˙ (t)   ∗ = −2 A P (T − t) + P (T − t)A + Q − P (T − t)BR−1 B ∗ P (T − t) yˆ(t) y (t) = −A∗ p(t) − 2Qˆ y (t) + 2P (T − t)(A − BR−1 B ∗ P (T − t))ˆ for almost all t ∈ [0, T ].

2

Remark. It follows from the maximum principle that u ˆ = − 12 R−1 B ∗ p. But, by (12.16), y (t), t ∈ [0, T ]. u ˆ(t) = −R−1 B ∗ P (T − t)ˆ This way we arrive at the optimal control in the feedback form derived earlier using the dynamic programming approach (see Theorem 9.3). Exercise 12.1. We will solve system (12.14)–(12.15) in the case n = m = 1, A = a ∈ R, B = 1, Q = R = 1, P0 = 0. Then 1 yˆ˙ = aˆ y − p, 2 p˙ = −2ˆ y − ap,

x(0) = x ∈ R, p(T ) = 0.



 a − 21 Let us remark that the matrix has two eigenvectors −2 −a     1 √1 √ , , 2(a − 1 + a2 ) 2(a + 1 + a2 ) √ √ corresponding to the eigenvalues 1 + a2 and − 1 + a2 . Therefore, for some constants ξ1 , ξ2 , t ∈ [0, T ]       √ √ 1 1 yˆ(t) t 1+a2 −t 1+a2 √ √ =e ξ +e ξ . p(t) 2(a − 1 + a2 ) 1 2(a + 1 + a2 ) 2

182

12 The maximum principle

Since p(T ) = 0 and yˆ(0) = x,

√ a + 1 + a2 √ ξ2 , ξ1 = −e a − 1 + a2 √

−1 √ 1 + a2 2 a + √ ξ2 = 1 − e−2T 1+a x. a − 1 + a2 −2T

√

1+a2

From the obtained formulae and the relation u ˆ = − 12 p, u ˆ(t) = =

p(t) u ˆ(t) yˆ(t) = − yˆ(t) yˆ(t) 2ˆ y (t) (a +



1+

√

1+a2 −1 √ 2 2 2(T −t) 1+a a )e +

e2(T −t)

√ yˆ(t), 1 + a2 − a

t ∈ [0, T ].

On the other hand, by Theorem 9.3, u ˆ(t) = −P (T − t)ˆ y (t), t ∈ [0, T ], where P˙ = 1 + 2aP − P 2 ,

P (0) = 0.

Hence we arrive at the well known formula √

2

e2t 1+a − 1 √ √ √ , P (t) = (a + 1 + a2 )e2t 1+a2 + 1 + a2 − a

t ≥ 0,

for the solution of the above Riccati equation.

§12.3. The maximum principle for impulse control problems We show now how the idea of the needle variation can be applied to impulse control problems. We assume, as in §11.1, that the evolution of a control system between impulses is described by the equation z˙ = f (z),

z(0) = x ∈ Rn ,

(12.18)

the solution of which will be denoted by z(t, x), t ∈ R. It is convenient to assume that there exists a set U and a function ϕ : Rn × U → Rn such that x+ϕ(x, U ) = Γ(x), x ∈ Rn , and to identify a shift (impulse) from x to y ∈ Γ(x) with a choice of u ∈ U such that y = ϕ(x, u). An admissible impulse strategy on I = [0, T ] is then completely determined by specifying a sequence of moments (t1 , . . . , tm ), 0 ≤ t1 < t2 < . . . < tm < T and a sequence (u1 , . . . , um ) of elements in U , m = 1, 2, . . . . We will denote it by π(t1 , . . . , tm ; u1 , . . . , um ) and the corresponding output trajectory starting from x by y(t) = y π,x (t), t ∈ I. Thus on each of the intervals I0 = [t0 , t1 ],

12.3 The maximum principle for impulse control problems

183

Ik = (tk , tk+1 ], k = 1, 2, . . . , m, t0 = 0, tm+1 = T the function y(·) satisfies equation (12.18) and boundary conditions y(t+ k ) = y(tk ) + ϕ(y(tk ), uk ),

y(0) = x, k = 1, 2, . . . , m.

To simplify formulations, in contrast to the definition in §11.1, we assume that output trajectories are left (not right) continuous. The theorem below formulates necessary conditions which a strategy maximizing a performance functional JT (x, π) = G(y π,x (T ))

(12.19)

has to satisfy. The case of a more general functional  JT (x, π) =

T

e−αt g(y π,x (t)) dt

0

+

m 

e−αtj c(y π,x (tj ), uj ) + e−αT G(y π,x (T ))

(12.20)

j=1

can be sometimes reduced to the one given by (12.19) (see comments after Theorem 12.2). To be in agreement with the setting of §12.1 we do maximize rather than minimize the performance functional. Theorem 12.2. Assume that mappings f (·) and ϕ(·, u), u ∈ U , and the function G(·) are of class C 1 and that for arbitrary x ∈ Rn the set ϕ(x, U ) u1 , . . . , u ˆm )) be a strategy maximizing is convex. Let π ˆ = π ˆ ((tˆ1 , . . . , tˆm ), (ˆ (12.19) and yˆ(·) the corresponding output trajectory. Then, for arbitrary k = 1, 2, . . . , m, y (tˆk ), u ˆk ) ≥ maxp(t+ y (tˆk ), u), p(t+ k ), ϕ(ˆ k ), ϕ(ˆ

(12.21)

ˆ y (tˆ+ y (tˆk )), p(tˆ+ k ), f (ˆ k )) = p(tk ), f (ˆ

(12.22)

u∈U

if tk > 0.

If tˆk = 0 then = in (12.22) has to be replaced by ≥. The function p(·) : I −→ Rn in (12.21) and (12.22) is left continuous and satisfies the equation y )p, (12.23) p˙ = −fx∗ (ˆ in all intervals Iˆ0 = [0, tˆ1 ], Iˆk = (tˆk , tˆk+1 ], k = 1, 2, . . . , m, with the end conditions y (T )), p(T ) = Gx (ˆ p(tˆk ) =

p(tˆ+ k)

+

ϕ∗x (ˆ y (tk ), u ˆk )p(tˆ+ k ),

(12.24) k = 1, 2, . . . , m.

(12.25)

Proof. We start from the following lemma which is a direct consequence of Theorem II.6.4 and Theorem I.1.1. The latter also imply that the derivative zx (t, x) exists for arbitrary t ∈ R and x ∈ Rn .

184

12 The maximum principle

Lemma 12.2. Let Φ(t, s) = zx (t − s, yˆ(s)), t, s ∈ [0, T ]. Then p(s) = Φ∗ (tˆk+1 , s)p(tˆk+1 ), s ∈ Iˆk , ∗ ˆ ˆ+ ˆ p(tˆ+ k = 0, 1, . . . , m. k ) = Φ (tk+1 , tk )p(tk+1 ), To prove the theorem we fix u ∈ U , a natural number k ≤ m and a number ε > 0 such that tˆk−1 + ε < tˆk < tˆk+1 − ε. It follows from the convexity of the set ϕ(ˆ y (tˆk ), U ) that for an arbitrary number β ∈ [0, 1] there exists u(β) ∈ U such that ϕ(ˆ y (tˆk ), u ˆk ) + β(ϕ(ˆ y (tˆk ), u) − ϕ(ˆ y (tˆk ), u ˆk )) = ϕ(ˆ y (tˆk ), u(β)). We can assume that u(0) = u ˆk and u(1) = u. For arbitrary α ∈ (tˆk − ε, tˆk + ε) and β ∈ [0, 1] we define new strategies π k−1 (α, β), π k (α, β), . . . , π m (α, β) as follows: π k−1 (α, β) = π((tˆ1 , . . . , tˆk−1 ), (ˆ u1 , . . . , u ˆk−1 )), k π (α, β) = π((tˆ1 , . . . , tˆk−1 , α), (ˆ u1 , . . . , u ˆk−1 , u(β))),  j ˆ ˆ ˆ π (α, β) = π (t1 , . . . , tk−1 , α, tk+1 , . . . , tˆj ),  (ˆ u1 , . . . , u ˆk−1 , u(β), u ˆk+1 , . . . , u ˆj ) , j = k + 1, . . . , m. Let y j (t; α, β) = y π Then

j

(α,β),x

(t),

t ∈ [0, T ], j = k − 1, . . . , m.

yˆ(t) = y m (t; tˆk , 0),

t ∈ [0, T ].

For arbitrary α ∈ (tˆk − ε, tˆk + ε) and β ∈ [0, 1] ω(α, β)) = G(y m (T ; α, β)) ≤ G(y m (T ; tˆk , 0)) ≤ ω(tˆk , 0). Hence

∂ω ˆ (tk , 0) = 0, if tˆk > 0, ∂α ∂ω ˆ (tk , 0) ≤ 0, if tˆk = 0, ∂α ∂ω ˆ (tk , 0) ≤ 0, ∂β

(12.26)

(12.27)

under the condition that the function ω(·, ·) has partial derivatives at (tˆk , 0). The essence of the proof consists in showing that the partial derivatives exist and in calculating their values; it turns out that (12.26) and (12.27) imply (12.22) and (12.21), respectively.

12.3 The maximum principle for impulse control problems

185

To prove (12.21) define χ(β) = ω(tˆk , β) = G(y m (T ; tˆk , β)),

β ∈ [0, 1],

and note that for β ∈ [0, 1] ∂χ (β) = Gx (y m (T ; tˆk , β))yβm (T ; tˆk , β) ∂β ∂ = Gx (y m (T ; tˆk , β))zx (T − tˆm , δ1 (β)) δ1 (β), ∂β where δ1 (β) = y m−1 (tˆm ; tˆk , β) + ϕ(y m−1 (tˆm ; tˆk , β), u ˆm ). Thus  ∂ω ˆ m−1 ˆ (tk , 0) = χβ (0) = Φ∗ (T, tˆ+ (tm ; tˆk , 0) m )p(T ), yβ ∂β ˆm )yβm−1 (tˆm ; tˆk , 0) . + ϕx (y m−1 (tˆm ; tˆk , 0), u It follows from Lemma 12.2 and condition (12.25) that ˆ+ Φ∗ (T, tˆ+ m )p(T ) = p(tm ), ˆ ˆ+ ϕx (y m−1 (tˆm ; tˆk , 0), u ˆm )p(tˆ+ m ) = p(tm ) − p(tm ). Therefore

χβ (0) = p(tˆm ), yβm−1 (tˆm ; tˆk , 0).

By induction, we obtain

Moreover

χβ (0) = p(tˆk+1 , yβk (tˆk+1 ; tˆk , 0).

(12.28)

y k (tˆk+1 ; tˆk , β) = z(tˆk+1 − tˆk ; δk (β)),

(12.29)

where δk (β) = y k−1 (tˆk ; tˆk , β) + ϕ(y k−1 (tˆk ; tˆk , β), u(β)) = yˆ(tˆk ) + ϕ(ˆ y (tˆk ), u ˆk ) + β(ϕ(ˆ y (tˆk ), u) − ϕ(ˆ y (tˆk ), u ˆk )). Taking into account (12.28), (12.29) and (12.30) we finally see that ˆ χβ (0) = Φ∗ (tˆk+1 , tˆ+ y (tˆk ), u) − ϕ(ˆ y (tˆk ), u ˆk ) k )p(tk+1 ), ϕ(ˆ + = p(tˆk ), ϕ(ˆ y (tˆk ), u) − ϕ(ˆ y (tˆk ), u ˆk ). Hence (12.21) follows. To prove (12.22) define ψ(α) = ω(α, 0) = G(y m (T ; α, 0)), Then

α ∈ (tˆk − ε, tˆk + ε).

(12.30)

186

12 The maximum principle

∂ψ ∂y m (α) = Gx (y 1 (T ; α, 0)) (T ; α, 0) ∂α ∂α = Gx (y 1 (T ; α, 0))zx (T − tˆm ; γ1 (α)) where

∂γ1 (α), ∂α

γ1 (α) = y m−1 (tˆm ; α, 0) + ϕ(y m−1 (tˆm ; α, 0)).

Consequently, by Lemma 12.2 and formula (12.25), ∂ψ m−1 ˆ = p(tˆ+ (tm ; m ), yα ∂α = p(tˆm ), yαm−1 (tˆm ;

m−1 ˆ tˆk , 0) + p(tˆm ) − p(tˆ+ (tm ; tˆk , 0) m ), yα

tˆk , 0),

and, again by induction,  ∂ψ (tk ) = p(tˆk+1 ), yαk (tˆk+1 ; tˆk , 0) . ∂α Moreover

y k (tˆk+1 ; α, 0) = z(tˆk+1 − α; γk (α)),

where γk (α) = y k−1 (α; α, 0) + ϕ(y k−1 (α; α, 0), u ˆk ). Hence ∂ψ (tk ) = −p(tˆk+1 ), f (ˆ y (tˆk+1 )) + p(tˆ+ y (tˆk )) k ), f (ˆ ∂α + p(tˆ+ y (tˆk ), u ˆk )f (ˆ y (tˆk )). k ), ϕx (ˆ Taking into account equation (12.25), we arrive at ∂ψ (tk ) = −p(tˆk+1 ), f (ˆ y (tˆk+1 )) + p(tˆk ), f (ˆ y (tˆk )). ∂α On the other hand, the function p(·), f (ˆ y (·)) is constant on (tˆk , tˆk+1 ] and therefore  + ˆ y (tˆk+1 )) . p(tˆk ), f (y(tˆ+ k )) = p(tk+1 ), f (ˆ In addition,   ∂ψ (tk ) = p(tˆk ), f (ˆ y (tˆk )) − p(tˆ+ y (tˆ+ k ), f (ˆ k )) , ∂α 2

and by (12.26) we finally obtain (12.22).

Remark. Theorem 12.2 can be reformulated in terms of appropriately defined Hamiltonians H1 (x, p) = p, f (x),

H2 (x, p, u) = p, ϕ(x, u),

x, p ∈ Rn ,

u ∈ U.

12.4 Separation theorems

187

The conjugate equation (12.23) is of the form ∗ p˙ = −H1x (p, yˆ),

and condition (12.21) expresses the fact that the function H2 (p(tˆ+ ˆ(tˆk ), ·) k ), y attains its maximal value at u ˆk . Remark. To cover the case of general functional (12.20) one has to replace state space Rn by Rn+2 and define ⎤ ⎛⎡ ⎡ ⎤ ⎡ ⎤ ⎞ ⎡ ⎤ x x f (x) ϕ(x, u) ⎦ , ϕ˜ ⎝⎣ xn+1 ⎦ , u⎠ = ⎣ 0 ⎦ , 1 f˜ ⎣ xn+1 ⎦ = ⎣ xn+2 xn+2 e−αxn+1 g(x) c(x, u) ⎡ ⎤ x ˜ ⎣ xn+1 ⎦ = xn+2 + e−αxn+1 G(x), x ∈ Rn , xn+1 , xn+2 ∈ R. (12.31) G xn+2 ˜ we Formulating conditions from Theorem 12.2 for new functions f˜, ϕ˜ and G obtain the required maximum principle. The most stringent assumption here is the convexity of ϕ(x, ˜ U ). It can, however, be omitted, see [84].

§12.4. Separation theorems An important role in the following section and in Part IV will be played by separation theorems for convex sets. Assume that E is a Banach space with the norm · . The space of all linear and continuous functionals ϕ on E with the norm ϕ ∗ = sup{|ϕ(x)|; x ≤ 1} will be denoted by E ∗ . Theorem 12.3. Let K and L be disjoint convex subsets of E. If the interior of K is nonempty then there exists a functional ϕ ∈ E ∗ different from zero such that ϕ(x) ≤ ϕ(y) for x ∈ K, y ∈ L. (12.32) Proof. Let x0 be an interior point of K and y0 an arbitrary point of L. Then the set M = K − L + y0 − x0 = {x − y + y0 − x0 ; x ∈ K, y ∈ L} is convex and 0 is its interior point. Define   x (12.33) p(x) = inf t > 0; ∈ M . t It is easy to check that p(x + y) ≤ p(x) + p(y),

p(αx) = αp(x),

x, y ∈ E, α ≥ 0,

p(x) ≤ c x for some c > 0 and all x ∈ E.

(12.34) (12.35)

188

12 The maximum principle

Define on the one-dimensional linear space {αz0 ; α ∈ R}, where z0 = y0 − x0 , / M , for all α > 0 a functional ϕ0 by setting ϕ(αz0 ) = α, α ∈ R. Since z0 ∈   α p(αz0 ) = inf t > 0; z0 ∈ M ≥ α = ϕ0 (αz0 ). t If α ≤ 0 then

ϕ0 (αz0 ) ≤ 0 ≤ p(αz0 ).

By Theorem A.4 (the Hahn–Banach theorem), there exists a linear functional ϕ such that ϕ(x) ≤ p(x), x ∈ E. (12.36) It follows from (12.35) and (12.36) that ϕ ∈ E ∗ . Moreover, for x ∈ M   x (12.37) ϕ(x) ≤ p(x) = inf t > 0; ∈ M ≤ 1. t Hence from (12.37) and the identity ϕ(z0 ) = 1, for x ∈ K and y ∈ L, ϕ(x − y + y0 − x0 ) = ϕ(x − y) + ϕ(z0 ) ≤ 1 ≤ ϕ(z0 ), 2

(12.32) follows.

If (12.32) holds we say that the functional ϕ ∈ E ∗ separates the sets K and L. Theorem 12.4. Let M ⊂ E be a convex set. (i) If the interior Int M of M is nonempty and x0 ∈ / Int M then there exists a functional ϕ ∈ E ∗ , ϕ = 0, such that ϕ(x) ≤ ϕ(x0 ),

x ∈ M.

(ii) If a point x0 ∈E does not belong to the closure of M then there exist a functional ϕ ∈ E ∗ , ϕ = 0, and a number δ > 0 such that ϕ(x) + δ ≤ ϕ(x0 ),

x ∈ M.

Proof. (i) Define K = Int M and L = {x0 }. By Theorem 12.3 there exists ϕ ∈ E ∗ , ϕ = 0, such that (12.38) ϕ(x) ≤ ϕ(x0 ) for x ∈ Int M. Since the set M is contained in the closure of Int M , inequality (12.38) holds for all x ∈ M . (ii) For some r > 0 the closed ball L = {y ∈ E; y − x0 ≤ r} is disjoint from M . Setting K = M we have from Theorem 12.3 that for some ϕ ∈ E ∗ , ϕ = 0, (12.32) holds. Since ϕ = 0 and m = inf{ϕ(y); y − x0 ≤ r} < ϕ(x0 ), it suffices to define δ as arbitrary positive number smaller than ϕ(x0 ) − m.2

12.5 Time-optimal problems

189

Theorem 12.5. Let M be a convex subset of E = Rn . If a point x0 ∈ M is not in the interior of M then there exists a vector p ∈ Rn , p = 0, such that p, x ≤ p, x0 ,

x ∈ M.

Proof. If M has a nonempty interior then the result follows from Theorem 12.4 (i). If the interior of M is empty then the smallest linear space E0 containing all vectors x − x0 , x ∈ M , is different from E. Therefore there exists a linear functional ϕ, different from zero and such that ϕ(y) = 0 for y ∈ E0 . In particular, ϕ(x) = ϕ(x0 ) for x ∈ M . The functional ϕ can be represented 2 in the form ϕ(z) = p, z, z ∈ Rn , for some p ∈ Rn .

§12.5. Time-optimal problems There are important optimization problems in control theory which cannot be solved using the idea of the needle variation as these variations are not admissible controls. In such cases it is often helpful to reformulate the problem as a geometric question of finding a supporting hyperplane to a properly defined convex set. We will apply this approach to prove the maximum principle for the so-called time-optimal problem. Let us consider again a linear system y˙ = Ay + Bu,

y(0) = x

(12.39)

ˆ be a given and assume that the set U ⊂ Rm is convex and compact. Let x element of Rn different from x. We say that a control u(·) : [0, +∞) −→ U transfers x to x ˆ at time T > 0 if for the corresponding solution y x,u (·) of x,u ˆ. (12.39), y (T ) = x The time-optimal problem consists of finding a control which transfers x to x ˆ in the minimal time. The following theorem is the original formulation of the maximum principle and is due to L. Pontryagin and his collaborators. Theorem 12.6. (i) If there exists a control transferring x to x ˆ then the time-optimal problem has a solution. (ii) If u ˆ(·) is a control transferring x to x ˆ in the minimal time Tˆ > 0 n then there exists a nonzero vector λ ∈ R such that for the solution p(·) of the equation (12.40) p˙ = −A∗ p, p(Tˆ) = λ, the identity

B ∗ p(t), u ˆ(t) = maxB ∗ p(t), u u∈U

holds for almost all t ∈ [0, Tˆ].

(12.41)

190

12 The maximum principle

Remark. For a vector b ∈ Rm , b = 0, there exists exactly one (n − 1)dimensional hyperplane L(b) orthogonal to b, supporting U and such that the set U is situated below L(b), the orientation being determined by b. The maximum principle (12.41) says that hyperplane L(B ∗ p(t)) rolling over U touches the set U at the values u ˆ(t), t ∈ [0, Tˆ], of the optimal control. The adjoint equation (12.40) indicates that the rolling hyperplanes are of a special form. It is therefore clear that the maximum principle provides important information about optimal control. Proof. (i) Let S(t) = eAt , t ∈ R. The solution to (12.9) is of the form  t  y(t) = S(t) x + S −1 (r)Bu(r) dr , t ≥ 0. 0

For arbitrary t ≥ 0 define a set R(t) ⊂ Rn : ! t " R(t) = S −1 (r)Bu(r) dr; u(r) ∈ U, r ∈ [0, t] . 0

We will need several properties of the multivalued function R(t), t ≥ 0, formulated in the following lemma. Lemma 12.3. (i) For arbitrary t ≥ 0 the set R(t) is convex and compact. (ii) If xm → a and tm ↓ t as m ↑ +∞ and xm ∈ R(tm ) for m = 1, 2, . . . , then a ∈ R(t). (iii) If a ∈ Int R(t) for some t > 0 then a ∈ R(s) for all s < t sufficiently close to t. Proof. In the proof we need a classic lemma about weak convergence. We recall that a sequence (hm ) of elements from a Hilbert space H is weakly convergent to h ∈ H if for arbitrary x ∈ H, limhm , xH = h, xH , where ·, ·H denotes the scalar product in H.

m

Lemma 12.4. An arbitrary bounded sequence (hm ) of elements of a separable Hilbert space H contains a subsequence weakly convergent to an element of H. Proof of Lemma 12.4. Let us consider linear functionals ϕm (x) = hm , xH ,

x ∈ H, m = 1, 2, . . .

It easily follows from the separability of H that there exists a subsequence (ϕmk ) and a linear subspace H0 ⊂ H, dense in H, such that the sequence (ϕmk (x)) is convergent for arbitrary x ∈ H0 to its limit ϕ(x), x ∈ H0 . Since the sequence (hm ) is bounded, there exists a constant c > 0 such that |ϕ(x)| ≤ c|x|H ,

x ∈ H0 .

12.5 Time-optimal problems

191

Therefore the linear functional ϕ has a unique extension to a linear, continuous functional ϕ˜ on H. By Theorem A.7 (the Riesz representation theorem) there exists h ∈ H such that ϕ˜ = h, xH x ∈ H. The element h is the weak 2 limit of (hmk ). Proof of Lemma 12.3. Denote by U (t) the set of all inputs defined on [0, t] with values in U . It is a bounded, convex and closed subset of the Hilbert space H = L2 (0, t; Rn ). The linear transformation from H into Rn  t u(·) −→ S −1 (r)Bu(r) dr 0

maps the convex set U (t) onto a convex set. Hence the set R(t) is convex. It also maps weakly convergent sequences onto weakly convergent ones, and, by Lemma 12.4, the set R(t) is closed. Since R(t) is obviously a bounded set, the proof of (i) is complete. To prove (ii), remark that  t  tm −1 S (r)Bum (r) dr + S −1 (r)Bum (r) dr = x1m + x2m xm = 0

t

for some um (·) ∈ U (tm ), m = 1, 2, . . . . Since x1m ∈ R(t), x2m → 0 and R(t) is a closed set, a ∈ R(t). To show (iii), assume, to the contrary, that a ∈ Int R(t) but for sequences / tm ↑ t and xm −→ α, xm ∈R(t m ). Since the set R(tm ) is convex, by Theorem 12.5 there exists a vector pm ∈ R, |pm | = 1, such that x − am , pm  ≤ 0

for all x ∈ R(tm ).

(12.42)

Without any loss of generality we can assume that pm −→ p, |p| = 1. Letting m tend to infinity # in (12.42), we see that x − a, p ≤ 0 for arbitrary x from the closure of R(tm ) and thus for all x ∈ R(t). Since α ∈ R(t), p has to be m

2

zero, a contradiction. We go back to the proof of Theorem 12.6. Let z(t) = S −1 (t)ˆ x − x,

t ≥ 0,

and let Tˆ be the infimum of all t > 0 such that z(t) ∈ R(t). There exists a decreasing sequence tm ↓ Tˆ such that xm = z(tm ) ∈ R(tm ), m = 1, 2, . . . . By Lemma 12.3(ii), z(Tˆ) ∈ R(Tˆ), and therefore there exists an optimal solution to the problem. To show (ii) define a = z(Tˆ). Then a ∈ / Int R(Tˆ). For, if a ∈ Int R(Tˆ), then, by Lemma 12.3(iii), there exists a number t < Tˆ such that z(t) ∈ R(t), a contradiction with the definition of Tˆ. Consequently, by Theorem 12.5, there exists λ ∈ Rn , λ = 0, such that x − z(Tˆ), λ ≤ 0,

x ∈ R(Tˆ).

(12.43)

192

12 The maximum principle

Let u ˆ(·) be an optimal strategy and u(·) an arbitrary control taking values in U . It follows from (12.43) that $

Tˆ

0

% S −1 (r)(Bu(r) − B u ˆ(r)) dr, λ ≤ 0,

or, equivalently,  0

Tˆ 

B ∗ (S ∗ (r))−1 (r)λ, u ˆ(r) − u(r) dr ≥ 0.

(12.44)

Since the function p(t) = (S ∗ (t))−1 λ, t ∈ [0, T ], is a solution to (12.40), see §I.1.1, and (12.44) holds for arbitrary admissible strategy u(·), relation (12.41) holds. 2 Example 12.1. Consider the system z¨ = u

(12.45)

with the initial conditions z(0) = x1 ,

z(0) ˙ = x2 .

(12.46)

We will find a control u(·) satisfying the constrains −1 ≤ u(t) ≤ 1, t ≥ 0, and transferring, in the minimal time, initial state (12.46) to z(Tˆ) = 0,

z( ˙ Tˆ) = 0.

Transforming (12.45)–(12.46) into form (12.39), we obtain     0 1 0 A= , B= . 0 0 1 The initial and the final states are, respectively,     x1 0 x= , x ˆ= x2 0 and the set of control parameters U = [−1, 1]. Since the pair (A, B) is controllable, an arbitrary initial state x close to x ˆ can be transferred to x ˆ by controls with values in [−1, 1], see Theorem I.4.1. In fact, as we will see soon, all states x can be transferred to x ˆ this way. An arbitrary solution of the adjoint equation   0 0 p˙ = p −1 0

12.5 Time-optimal problems

193



 −a is of the form p(t) = , t ≥ 0, where a and b are some constants. at + b Assume that x = x ˆ = 0 and u ˆ is an optimal control transferring x to 0 in the minimal time Tˆ. Then, by Theorem 12.6, there exist constants a and b, at least one different from zero, such that for almost all t ∈ [0, Tˆ]: u ˆ(t) = sgn B ∗ p(t) = sgn(at + b)

(12.47)

where sgn r is equal 1, 0 or −1 according to whether r is positive, zero or negative. Without any loss of generality we can assume that (12.47) holds for all t ∈ [0, Tˆ]. It follows from (12.47) that the optimal control changes its value at most once and therefore u ˆ should be of one of the following forms: (i) u ˆ(t) = 1, t ∈ [0, Tˆ], (ii) u ˆ(t) = −1, t ∈ [0, Tˆ], (iii) u ˆ(t) = 1, t ∈ [0, s), u ˆ(t) = −1, t ∈ (−s, Tˆ], ˆ(t) = 1, t ∈ (s, Tˆ], (iv) u ˆ(t) = −1, t ∈ [0, s), u where s is a number from [0, Tˆ]. Denote by Γ+ and Γ− sets of these states x ∈ R2 which can be transferred to x ˆ by the constant controls taking values 1 and −1, respectively. Solutions to the equations           0 1 0 0 1 0 x1 y˙ + = y+ + , y˙ − = y− + , y(0) = x2 0 0 1 0 0 −1 are given by the formulae t2 , 2 t2 y1− (t) = x1 + x2 t − , 2 y1+ (t) = x1 + x2 t +

Hence

y2+ (t) = x2 + t, y2− (t) = x2 − t,

t ≥ 0.



  x2 x1 ; x1 = 2 , x2 ≤ 0 , x2 2    x22 x1 Γ− = ; x1 = − , x2 ≥ 0 . x2 2 Γ+ =

The sets Γ+ and Γ− contain exactly those states which can be transferred to x ˆ by optimal strategies (i) and (ii), and the minimal time to reach 0 is exactly x2 . Taking into account the shapes of the solutions y + and y − , it is easy to see that if the initial state x is situated below the curve Γ+ ∪ Γ− , then the optimal strategy is of type (iii) with s being equal to the moment of hitting Γ− . If the initial state is situated above Γ+ ∪ Γ− , then the optimal strategy is of the type (iv). We see also that in the described way an arbitrary state x can be transferred to x ∈ R2 .

194

12 The maximum principle

Bibliographical notes Numerous applications of the maximum principle can be found in W.H. Fleming and R.W. Rishel [37], and in E.B. Lee and L. Markus [63]. The proof of the maximum principle for more general control problems is given, for instance, in W.H. Fleming and R.W. Rishel [37], in R. Vinter [96] and B.S. Mordukhovich [71]. A functional analytic approach is presented in I.V. Girsanov [41]. Nonlinear optimization problems can be discussed in the framework of nonsmooth analysis, see F.H. Clarke [16]. Theorem 12.2 is borrowed from A. Blaquiere [11], and its proof follows the paper by R. Rempala and J. Zabczyk [85]. Applications can be found in A. Blaquiere [11].

Chapter 13

The existence of optimal strategies

This chapter starts with an example of a simple control problem without an optimal solution showing a need for existence results. A proof of the classic Filippov theorem on existence is also presented.

§13.1. A control problem without an optimal solution An important role in control theory is played by results on existence of optimal strategies. In particular, all the formulations of the maximum principle in Chapter 12 required the existence of an optimal strategy. The need for existence theorems is also more practical as natural control problems may not have optimal solutions. Let us consider a control system on R1 y˙ = u,

y(0) = 0

(13.1)

with the (two-element) control set U = {−1, 1}. It is easy to see that a strategy minimizing the quadratic cost functional JT1 (0, u)

 =

0

T

(y 2 + u2 ) dt

(13.2)

exists and is either of the form u ˆ(t) = 1, t ∈ [0, T ], or u ˆ(t) = −1, t ∈ [0, T ]. It turns out however that for a similar functional  T (y 2 − u2 ) dt JT2 (0, u) = 0

the optimal strategy does not exist. To see that this is really the case, let us remark first that for an arbitrary admissible strategy u(·), y 2 −u2 ≥ −1, hence JT2 (0, u) ≥ −T . On the other hand, if we define strategies um (·), m = 1, 2, . . . , © Springer Nature Switzerland AG 2020

J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_13

195

196

13 The existence of optimal strategies

 1, um (t) = −1,

if if

1 T 1 T

j−1 1 2j−1 m ≤ t < T 2m , 2j−1 1 j 2m ≤ t < T m , j

= 1, . . . , m,

then for the corresponding solutions ym (·) of (13.1), 0 ≤ ym (t) ≤

1 , 2Tm

t ∈ [0, T ],

and therefore JT2 (0, um ) −→ −T as m ↑ +∞. So the infimum of the functional JT2 (0, ·) is −T . If, on the other hand, for a control u(·): JT2 (0, u(·)) = −T , then, for almost all t ∈ [0, T ], y 2 (t) − u2 (t) = y 2 (t) − 1 = −1, and thus y(t) = 0, and, consequently, u(t) = 0, for almost all t ∈ [0, T ]. But this strategy is not an admissible one, a contradiction. Before going to the main result of the chapter let us remark that some existence results were formulated in connection with the dynamic programming in §9.2 and §9.3, with the impulse control in §11.1 and for the time-optimal problem in §12.5.

§13.2. Filippov’s theorem Let us consider a control system y˙ = f (y, u),

y(0) = x ∈ Rn

(13.3)

with the cost functional J(x, u) = G(T, y(T )).

(13.4)

Let us assume in addition that given are a set U ⊂ Rm of control parameters and a set K ⊂ Rn+1 of the final constraints. Instead of fixing T > 0 we will require that (T, y(T )) ∈ K. (13.5) Let C > 0 be a positive number such that |f (x, u)| ≤ C(1 + |x| + |u|), |f (x, u) − f (z, u)| ≤ C|x − z|(1 + |u|)

(13.6) (13.7)

for arbitrary x, z ∈ Rn and u ∈ U see §II.6.1. An admissible control is an arbitrary Borel measurable function u(·) with values in U defined on an interval [0, T ] depending on u(·) and such that the solution y(·) of (13.3) satisfies condition (13.5). The following theorem holds. Theorem 13.1. Assume that sets U and K are compact and the set f (x, U ) is convex for arbitrary x ∈ Rn . Let moreover the function f satisfy (13.6)–

13.2 Filippov’s theorem

197

(13.7) and G be a continuous function on K. If there exists an admissible control for (13.3) and (13.5) then there also exists an admissible control minimizing (13.4). Proof. It follows from the boundedness of the set K that there exists a number T˜ > 0 such that admissible controls u(·) are defined on intervals [0, T ] ⊂ [0, T˜]. Let us extend an admissible control u(·) onto [0, T˜] by setting u(t) = u+ for t ∈ [T, T˜], where u+ is an arbitrary selected element from U . Denote by K the set of all solutions of (13.3), defined on [0, T˜] and corresponding to the extended controls. By (13.6), for y ∈ K and t ∈ [0, T˜],  t  t     γ(t) = |y(t)| = x + f (y(s), u(s)) ds ≤ |x| + C (1 + |y(s)| + |u(s)|) ds. 0

0

Hence, taking into account Gronwall’s Lemma II.7.1,  t   |u(s)| ds , t ∈ [0, T˜]. |y(t)| ≤ eCt |x| + Ct + 0

Since the set U is bounded, for some constant C1 > 0 |y(t)| ≤ C1

for y ∈ K,

t ∈ [0, T˜].

(13.8)

Similarly, for y(·) ∈ K, and [t, s] ⊂ [0, T˜],  s  s |y(t) − y(s)| ≤ |f (y(r), u(r))| dr ≤ C (1 + |y(r)| + |u(r)|) dr. t

t

By the boundedness of U by (13.8), for an appropriate constant C2 > 0 |y(t) − y(s)| ≤ C2 |t − s| for y(·) ∈ K,

t, s ∈ [0, T˜].

(13.9)

Conditions (13.8) and (13.9) imply that K is a bounded subset of C(0, T˜; Rn ) and that its elements are equicontinuous. By Theorem A.8 (the Ascoli theorem), the closure of K is compact in C(0, T˜; Rn ). Let Jˆ denote the minimal value of (13.4) and let (ym ) be a sequence of elements from K corresponding to controls (um ) such that ˆ J(x, um ) = G(Tm , ym (Tm )) −→ J.

(13.10)

Since K and K are compact and the function G is continuous, we can assume, without any loss of generality, that the sequence (ym ) is uniformly convergent on [0, T˜] to a continuous function yˆ(·) and Tm −→ Tˆ. We will show that the function yˆ(·) is an optimal trajectory. Since yˆ(·) satisfies the Lipschitz condition (13.9), it has finite derivatives dˆ y (t) for almost all t ∈ [0, T˜]. Let t be a fixed number from (0, T˜) for which the dt derivative exists. The continuity of f (·, ·) implies that for arbitrary ε > 0 there exists δ > 0 such that if |z − yˆ(t)| < δ then the set f (z, U ) is contained in the

198

13 The existence of optimal strategies

δ-neighbourhood fδ (ˆ y (t), U ) of the set f (ˆ y (t), U ). By uniform convergence of the sequence (ym ), there exist η > 0 and N > 0 such that |ym (s) − yˆ(t)| < δ, for s ∈ (t, t + η) and m > N . Let us remark that  s ym (s) − ym (t) 1 = f (ym (r), um (r)) dr (13.11) s−t s−t t t 1 and that the integral s−t (y (r), um (r)) dr is in the closure fδ (ˆ y (t), U ) of s m y (t), U ). Letting m tend to +∞ in (13.10) we obtain the convex set fδ (ˆ yˆ(s) − yˆ(t) ∈ fδ (ˆ y (t), U ), s−t and consequently dˆ y (t) ∈ fδ (ˆ y (t), U ). dt Since δ > 0 was arbitrary and the set f (ˆ y (t), U ) is convex and closed, dˆ y (t) ∈ f (ˆ y (t), U ). dt

(13.12)

y This way we have shown that if for some t ∈ (0, T˜) the derivative dˆ dt (t) exists, then (13.12) holds. From this and the lemma below on measurable selectors, we will deduce that there exists a control u ˆ(·) with values in U such that dˆ y (t) = f (ˆ y (t), u ˆ(t)) for almost all t ∈ [0, T˜]. (13.13) dt The control u ˆ is an optimal strategy and yˆ is an optimal output. For, from the convergence Tm −→ Tˆ, ym −→ yˆ in C(0, T ; Rn )) as m → +∞, the compactness of K and the continuity of G, it follows, see (13.10), that Jˆ = G(Tˆ, yˆ(Tˆ)) and (Tˆ, yˆ(T )) ∈ K. We proceed now to the already mentioned lemma on measurable selectors.

Lemma 13.1. Assume that D is a compact subset of Rp × Rm and let Δ be the projection of D on Rp . There exists a Borel function v : Δ −→ Rm such that (w, v(w)) ∈ D

f orarbitrary w ∈ Δ.

Proof. The set D is compact as an image of a compact set by a continuous transformation, and thus also Borel. If m = 1 then we define v(w) = min{z; (w, z) ∈ D},

w ∈ Δ.

(13.14)

The function v is lower-semicontinuous and thus measurable. Hence the lemma is true for m = 1. If m is arbitrary then we can regard the set D is a subset of Rp+m−1 × R1 and use the fact that the lemma is true if m = 1. This way we obtain that

13.2 Filippov’s theorem

199

there exists a measurable function v1 defined on the projection D1 ⊂ Rp+m−1 of the set D such that (w1 , v1 (w1 )) ∈ D for arbitrary w1 ∈ Δ1 . So if we assume that the lemma is true for m − 1 and regard the set Δ1 as a subset Rp × Rm−1 we obtain that there exists a measurable function v2 : Δ −→ Rm−1 such that (w, v2 (w)) ∈ Δ1 ,

for all w ∈ Δ.

The function v(w) = (v2 (w), v1 (w, v2 (w))),

w ∈ Δ,

has the required property. Hence the lemma follows by induction.



We will now complete the proof of Theorem 13.1. It follows from Theorem A.10 (Luzin’s theorem) that there exists an increasing sequence (Δk ) of  compact subsets of [0, T˜] such that the Lebesgue measure of Δk is T˜ and, k

on each Δk , the function dˆ y /dt is continuous. We check easily that sets

dˆ y (t) = f (ˆ y (t), u) ⊆ R×Rm , k = 1, 2, . . . , (13.15) Dk = (t, u) ∈ Δk ×U : dt are compact, and by Lemma 13.1 there exist Borel measurable functions u ˆk : Δk −→ U such that dˆ y (t) = f (ˆ y (t), u ˆk (t)), dt

t ∈ Δk ,

k = 1, 2, . . . .

The required control u ˆ satisfying (13.13) can finally be defined by the formula  u ˆ1 (t) for t ∈ Δ1 u ˆ(t) = u ˆk (t) for t ∈ Δk \ Δk−1 , k = 2, . . . . The proof of Theorem 13.1 is complete.



We deduce from Theorem 13.1 several corollaries. Theorem 13.2. Assume that a function f and a set U satisfy the conditions of Theorem 13.1 and that for system (13.3) there exists a control transferring x to x ˆ in a finite time. Then there exists a control transferring x to x ˆ at the minimal time. Proof. Let us assume that there exists a control transferring x to x ˆ at time T˜ > 0 and define K = [0, T˜] × {ˆ x}, G(T, x ˆ) = T , T ∈ [0, T˜]. The set K is compact and the function G continuous. Therefore all the assumptions of Theorem 13.1 are satisfied and there exists a strategy minimizing functional (13.4). It is clear that this is a strategy which transfers x to x at the minimal time. 

200

13 The existence of optimal strategies

In particular, assumptions of Theorem 13.2 are satisfied if system (13.3) is linear and the set U compact and convex. Hence the theorem implies the existence of an optimal solution to the time-optimal problem for linear systems, see Theorem 12.6. As a corollary of Theorem 13.1 we also have the following result. Theorem 13.3. Assume that a function f and a set U satisfy the assumptions of Theorem 13.1 and that G is a continuous function on Rn . Then, for arbitrary T > 0, there exists a strategy minimizing the functional JT (x, u) = G(y(T )).

(13.16)

Proof. As in the proof of Theorem 13.1 we show that there exists C > 0 such that |y(T )| ≤ C for arbitrary output trajectory y(·). It is enough now to apply Theorem 13.1 with K = {T } × {z; |z| ≤ C} and G(T, z) = G(z), |z| ≤ C.  Remark. Theorem 13.3 is not true for more general functionals  J(x, u) =

0

T

g(y(s), u(s)) ds + G(y(T )),

(13.17)

as an example from §13.1 shows. The method of introducing an additional variable, used at the end of the proof of Theorem II.8.1, cannot be applied here. This is because the new system is of the form y˙ = f (y, u), w˙ = g(y, u),

y(0) = x, W (0) = 0,

and even if the sets f (x, U ), x ∈ Rn , are convex subsets of Rn , the set 

f (x, u) ; u∈U g(x, u) is not, in general, a convex subset of Rn+1 . Theorems on the existence of optimal controls for functionals (13.17) require more sophisticated methods than those used in the proof of Theorem 13.1.

Bibliographical notes Theorem 13.1 is due to A.F. Filippov [39]. In its proof we followed. W.H. Fleming and R.W. Rishel [37]. The proof contains essential ingredients—in a simple form—of stronger existence results which can be found in the book by I. Cesari [15] or in the paper by C. Olech [72].

Part IV

Infinite-Dimensional Linear Systems

Chapter 14

Linear control systems

This chapter starts with basic results on semigroups of linear operators on Banach spaces. Characterizations of the generators of the semigroups due to Hille and Yosida and to Lions are given. Abstract material is illustrated by self-adjoint and differential operators. Non-homogeneous differential equations in Banach spaces, which are the mathematical models of infinitedimensional control systems, are studied. Their weak and strong solutions are examined. The final sections are devoted to controlled delay equations: existence of solutions and the semigroup representations.

§14.1. Introduction The control systems considered in the preceding chapters were described by ordinary differential equations and the state space was Rn . There is, however, a large number of systems which cannot be represented by a finite number of parameters. If we identify, for instance, the state of a heated bar with its temperature distribution, then the state is an element of an infinitedimensional function space. Parametrizing points of the bar by numbers from the interval [0, L] and denoting by y(t, ξ) the temperature at moment t ≥ 0 and at point ξ ∈ [0, L], we arrive at the parabolic equation (0.11) with boundary conditions (0.12)–(0.13), see Example 0.8, see also Example 0.9. The variety of situations described by partial differential equations is enormous and quite often control questions, similar to those considered in Parts I and III, appear. Control theory of infinite-dimensional systems requires more sophisticated methods than those of finite dimension. The difficulties increase to the same extent as passing from ordinary differential equations to equations of parabolic and hyperbolic types. A complete theory exists only for a linear system. We will limit our discussion to several fundamental questions of linear theory and use the so-called semigroup approach. © Springer Nature Switzerland AG 2020

J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_14

203

204

14 Linear control systems

§14.2. Semigroups of operators Assume for a while that E and U are finite-dimensional linear spaces and A : E −→ E and B : U −→ E are linear operators. Solutions to the linear differential equation y(0) = x ∈ E, t ≥ 0,

y˙ = Ay(t) + Bu(t),

are given, see § I.1.1, by the formula  t y(t) = S(t)x + S(t − s)Bu(s) ds, 0

t ≥ 0,

(14.1)

(14.2)

where S(t) = eAt ,

t ≥ 0,

(14.3)

is the fundamental solution of the equation z˙ = Az,

z(0) = x ∈ E.

(14.4)

The theory of linear semigroups extends the concept of fundamental solutions to arbitrary Banach spaces E and gives a precise meaning to (14.2) and (14.3) in more general situation. It turns out that a number of control systems modelled by partial differential equations can be treated as special cases of the general theory. Uncontrolled systems (14.4) on Banach spaces are discussed in §§14.2– 14.6 and basic properties of controlled systems are the object of §14.7. The following considerations are valid for both real and complex Banach spaces. The norm on the Banach space E will be denoted by  · , and, if the state space is Hilbert, usually by | · |. Let us remark that a function S(t), t ≥ 0, with values in M(n, n) is the fundamental solution of the equation (14.4) if and only if it is a continuous solution of the operator equation S(t + s) = S(t)S(s),

t, s ≥ 0, S(0) = I.

(14.5)

This leads us to the following generalization. Let E be a Banach space. A semigroup of operators is an arbitrary family of bounded linear operators S(t) : E −→ E, t ≥ 0, satisfying (14.5) and such that (14.6) lim S(t)x = x for arbitrary x ∈ E. t↓0

Because of the continuity condition (14.6), the full name of the concept is C0 -semigroups of operators; we will however say, in short, semigroups of operators or even semigroups. In the case of finite-dimensional spaces E the operator A in (14.3) and (14.4) is identical with the derivative of S(·) at 0:

14.2 Semigroups of operators

205

dS S(h) − I (0) = lim = A. h↓0 dt h

(14.7)

In the general case, the counterpart of A is called the infinitesimal operator or the generator of S(t), t ≥ 0. It is not, in general, an operator defined on the whole of E, but its domain D(A) consists of all those x ∈ E for which there exists the limit S(h)x − x , (14.8) lim h↓0 h and the operator A itself is given by the formula Ax = lim h↓0

S(h)x − x , h

x ∈ D(A).

(14.9)

Example 14.1. If A is a bounded linear operator on E then the family S(t) = etA =

+∞ m  t Am , m! m=0

t ≥ 0,

is a semigroup with generator A. Example 14.2. Let H be a Hilbert space with a complete and orthonormal basis (em ) and let (λm ) be a sequence of real numbers diverging to −∞. Define +∞  S(t)x = eλm t x, em em , x ∈ H, t ≥ 0. (14.10) m=1

It is not difficult to prove that the family given by (14.10) is a semigroup. We will show later, see page 224, that the domain of its generator A is given by +∞    D(A) = x ∈ H; (λm x, em )2 < +∞ , m=1

and the generator itself by Ax =

+∞ 

λm x, em ,

x ∈ D(A).

m=1

Example 14.3. Let H = L2 [0, π], Ax = d2 x/dξ 2 and the domain D(A) of the operator A consist of absolutely continuous functions x(·) defined on [0, π], equal to zero at 0 and π, and such that the first derivative dx/dξ is absolutely continuous on [0, π] and the second derivative d2 x/dξ 2 belongs to H. We will show later, see page 225, that the operator A defined  this way is the generator of a semigroup given by (14.10) with em (ξ) = 2

2 π

sin mξ,

ξ ∈ [0, π], λm = −m , m = 1, 2, . . . . This semigroup is sometimes denoted by

206

14 Linear control systems 2

S(t) = etd

/dξ 2

t ≥ 0.

,

It is not difficult to guess that the formula  t y(t) = S(t)x + S(t − s)bu(s) ds, 0

t ≥ 0,

defines the solution to the parabolic equation (0.11)–(0.13) of Example 0.8 in which or σ 2 = 1 and L = π. Example 14.4. Let us consider the wave equation ∂2y ∂2y (t, ξ) = 2 (t, ξ), 2 ∂t ∂ξ

t ∈ R, ξ ∈ [0, π],

with initial and boundary conditions y(t, 0) = y(t, π) = 0, t ∈ R, ∂y (0, ξ) = b(ξ), y(0, ξ) = a(ξ), ∂t

ξ ∈ (0, π).

We identify functions a(·) and b(·) with their Fourier expansions a(ξ) = b(ξ) =

+∞  m=1 +∞ 

(14.11)

αm sin mξ, βm sin mξ,

ξ ∈ (0, π).

(14.12)

m=1

If one assumes that

+∞  m=1

(m2 |αm | + m|βm |) < +∞ then is it easy to check

that the Fourier expansion of the solution y(·, ·) of the wave equation is of class C 2 with respect to both variables and y(t, ξ) =

+∞   βm sin mt sin mξ. αm cos mt + m m=1

(14.13)

We have also +∞  ∂y (t, ξ) = (−mαm sin mt + βm cos mt) sin mξ, ∂t m=1

ξ ∈ (0, π).

(14.14)

 a Define E to be the set of all pairs of functions with expansions (14.11) b and (14.12) such that

14.2 Semigroups of operators +∞ 

207

m2 |αm |2 + |βm |2 < +∞.

m=1

This is a Hilbert space with the following scalar product:  

 +∞ a ˜ a , ˜ = (m2 αm α ˜ m + βm β˜m ). b b m=1

Formulae (14.13) and (14.14) suggest that the semigroup of solutions to the wave equation should be defined as follows: S(t)

  +∞

a cos mt = b −m sin mt

sin mt m



cos mt

m=1

 αm sin m(·), βm

t ≥ 0.

We check directly that the above formula defines a semigroup of operators. The formula is meaningful for all t ∈ R and S ∗ (t) = S −1 (t) = S(−t),

t ∈ R.

Hence solutions of the wave equations define a continuous group of linear, unitary transformations of E into E (see page 318). It follows from Examples 14.2 and 14.3 that infinitesimal generators are, in general, noncontinuous and not everywhere defined operators. Therefore a complete description of the class of all generators, due to Hille and Yosida, is rather complicated. We devote §14.3 to it and the rest of the present section will be on elementary properties of the semigroups. In our considerations we will often integrate E-valued functions F defined β on [α, β] ⊂ [0, +∞). The integral α F (s) ds should be understood in the Bochner sense, see §A.5; in particular, an arbitrary continuous function is Bochner integrable and its integral is equal to the limit of Riemann’s sums 

β

F (s) ds = lim n

α



F (snk )(tn,k+1 − tnk ),

0≤k 0 and ω such that

208

14 Linear control systems

S(t) ≤ M eωt ,

t ≥ 0;

(ii) for arbitrary x ∈ E, the function S(·)x, x ∈ E, is continuous on [0, +∞). Proof. (i) By Theorem A.5 (the Banach–Steinhaus theorem) and condition (14.6), for some numbers M > 0, m > 0 and arbitrary t ∈ [0, m], S(t) ≤ M. Let t ≥ 0 be arbitrary and k = 0, 1, . . . and s ∈ [0, m) such that t = km + s. Then S(t) = S(km)S(s) ≤ M M k ≤ M M t/m ≤ M eωt , 1 where ω = m log M . (ii) For arbitrary t ≥ 0, h ≥ 0 and x ∈ E

S(t + h)x − S(t)x ≤ M eωt S(h)x − x. If, in addition, t − h ≥ 0, then S(t − h)x − S(t)x ≤ M eω(t−h) ||S(h)x − x. Hence the continuity of S(·)x at t ≥ 0 follows from (14.6).



Theorem 14.2. Assume that an operator A with the domain D(A) generates a semigroup S(t), t ≥ 0. Then for arbitrary x ∈ D(A) and t ≥ 0, S(t)x ∈ D(A), d S(t)x = AS(t)x = S(t)Ax, dt  t S(t)x − x = S(r)Ax dr.

(i) (ii) (iii)

0

Moreover, (iv) the domain D(A) is dense in E and the operator A is closed (i.e. its graph {(x, A(x)); x ∈ D(A)} is a closed subset of E × E). Proof. For h > 0, t > 0 S(t + h)x − S(t)x S(h)x − x S(h) − I = S(t) = S(t)x. h h h If x ∈ D(A), then lim S(t) h↓0

Hence, S(t)x ∈ D(A) and

S(h)x − x = S(t)Ax. h

(14.15)

14.2 Semigroups of operators

209

d+ S(t)x = S(t)Ax + AS(t)x. dt If h > 0, t > 0 and t − h > 0 then S(t − h)x − S(t)x S(h)x − x = −S(t − h) h h and therefore

d− S(t)x = S(t)Ax. dt This way the proofs of (i) and (ii) are complete. To prove (iii) note that the right-hand side of (iii) is well defined as continuous functions are Bochner integrable. Since the function S(·)x is continuously differentiable and (ii) holds, we have for all ϕ ∈ E ∗  t  t  d ϕ(S(r)x) dr = ϕ ϕ(S(t)x − x) = S(r)Ax dr , 0 dr 0

and (iii) follows by Theorem A.4 (the Hahn–Banach theorem). For h > 0, t > 0 and x ∈ E   1 h S(h) − I  t S(r)x dr = S(r)(S(t)x − x) dt, h h 0 0

(14.16)

t hence, letting h in (14.16) tend to 0 we obtain 0 S(r)x dr ∈ D(A). But t lim 1t 0 S(r)x dr = x and thus the set D(A) is dense in E. t↓0

To prove that the operator A is closed assume that (xm ) ∈ D(A), m = 1, 2, . . . , xm → x and Axm → y as m ↑ +∞. Since, for arbitrary r ≥ 0, m = 1, 2, . . . , S(r)Axm − S(r)y ≤ M eωr Axm − y, S(·)Axm −→ S(·)y uniformly on an arbitrary interval [0, t], t ≥ 0. On the other hand,  t S(r)Axm dr. (14.17) S(t)xm − xm = 0

Letting m in (14.17) tend to +∞ we obtain  S(t)x − x = and consequently lim t↓0

t

S(r)y dr, 0

S(t)x − x = y. t

Hence x ∈ D(A) and Ax = y. The proof of Theorem 14.2 is complete. As a corollary we have the following important proposition.



210

14 Linear control systems

Proposition 14.1. A given operator A can be a generator of at most one semigroup. ˜ Proof. Let S(t), S(t), t ≥ 0, be semigroups with the generator A. Fix ˜ x ∈ D(A), t ≥ 0, and define a function z(s) = S(t − s)S(s)x, s ∈ [0, t]. The function z(·) has a continuous first derivative and d   z(s) = −AS(t − s)S(s)x + S(t − s)AS(s)x ds   = −S(t − s)AS(s)x + S(t − s)AS(s)x = 0. Hence  S(t)x − S(t)x = z(0) − z(t) =

 0

t

d z(s) ds = 0. ds



We will need a result on the family S ∗ (t), t ≥ 0, of the adjoint operators on E ∗ , see § III.4.4 and §A.3. In the theorem below, E is a Hilbert space and we identify E with E ∗ . Theorem 14.3. Assume that E is a Hilbert space and S(t), t ≥ 0, a semigroup on E. Then the adjoint operators S ∗ (t), t ≥ 0, form a semigroup on E as well. Proof. By the definition of the adjoint operator, (S(t+s))∗ = (S(t)S(s))∗ = S ∗ (s)S ∗ (t), t, s ≥ 0, and therefore (14.5) holds in the present situation. To show that lim S ∗ (t)x = x for x ∈ E ∗ , t↓0

we need the following lemma. Lemma 14.1. If f is a function with values in a Banach space E, defined and Bochner integrable on an interval (α, β + h0 ), h0 > 0, β > α, then 

β

lim h↓0

f (t + h) − f (t) dt = 0.

α

Proof. The lemma is true if E is one-dimensional, by Theorem A.4, and hence it is true for simple functions f which take on only a finite number of values. By the very definition of Bochner integrable functions, see §A.5, there exists a sequence (fn ) of simple functions such that 

β+h0

lim

n→∞

α

It follows from the estimate

fn (t) − f (t) dt = 0.

14.2 Semigroups of operators



β

α

211

f (t + h) − f (t) dt



β

≤

fn (t + h) − f (t + h) dt

α



β

+  ≤2

 fn (t + h) − fn (t)|| dt +

α β+h0

 fn (t) − f (t) dt +

β

fn (t) − f (t) dt

α β

fn (t + h) − fn (t) dt,

α

α

valid for h ∈ (0, h0 ) and n = 1, 2, . . . , that the lemma is true in general.



For arbitrary s ≥ 0, S(s) = S ∗ (s). Without any loss of generality we can assume that for some c > 0 and all s ≥ 0, S ∗ (s) ≤ c. Moreover, for arbitrary z, y ∈ E the function S ∗ (s)z, y = z, S(s)y, s ≥ 0, is continuous. This easily implies that S ∗ (·)z is a bounded, Borel measurable function with values in a separable Hilbert space, and, therefore, it is Bochner integrable on arbitrary finite interval. To complete the proof of Theorem 14.3 we first fix t > 0 and an element x ∈ E and show that (14.18) lim S ∗ (t + h)x = S ∗ (t)x. h↓0

If γ ∈ (0, t) then  γ    −1  S (t + h)x − S (t)x = γ (S ∗ (t + h)x − S ∗ (t)x) du 0 γ    −1  = γ S ∗ (u)(S ∗ (t + h − u)x − S ∗ (t − u)x) du 0  γ −1 ≤γ M S ∗ (t + h − u)x − S ∗ (t − u)x du, ∗

∗

0

and, by Lemma 14.1, formula (14.18) holds. Let us finally consider an arbitrary sequence (tl ) of positive numbers convergent to zero and an arbitrary element x0 of E and denote by M the closure of the convex combinations of the elements S ∗ (tl )x0 , l = 1, 2, . . . Since S ∗ (tl )x0 → x0 weakly as l → +∞, so, by Theorem III.12.4(ii), x0 ∈ M . Therefore, the space E0 ⊂ E of all linear combinations of S ∗ (tl )x, l = 1, 2, . . . , x ∈ E, is dense in E. By (14.18), lim S ∗ (tl )x = x for all x ∈ E0 . l

Moreover, sup S ∗ (tl ) < +∞ and therefore l

lim S ∗ (tl )x = x for all x ∈ E. l



212

14 Linear control systems

Exercise 14.1. Give an example of a separable Banach space E and of a semigroup S ∗ (t), t ≥ 0, on E such that for some x ∈ E ∗ the function S ∗ (t)x, t ≥ 0, is not continuous at 0.

§14.3. The Hille–Yosida theorem Of great importance in applications are theorems which give sufficient conditions for an operator A to be the infinitesimal generator of a semigroup. A fundamental result in this respect is due to Hille and Yosida. Conditions formulated in their theorem are at the same time sufficient and necessary. The proof of the necessity part (not needed in the book) is rather easy and is left as an exercise for the interested reader. Theorem 14.4. Let A be a closed linear operator defined on a dense set D(A) contained in a Banach space E. If there exist ω ∈ R and M ≥ 1 such that, for arbitrary λ > ω, the operator λI − A has an inverse R(λ) = (λI − A)−1 satisfying Rm (λ) ≤

M (λ − ω)m

for m = 1, 2, . . . ,

(14.19)

then A is the infinitesimal generator of a semigroup S(t), t ≥ 0, on E such that (14.20) S(t) ≤ M eωt , t > 0. Proof. For λ > ω, define linear bounded operators Aλ = λ(λR(λ) − I) and semigroups +∞  (λ2 t)m m R (λ). S λ (t) = etAλ = e−λt m! m=0 We will prove that the limit of the semigroups S λ (·) as λ → +∞ is the required semigroup. Let us show first that Aλ x −→ Ax for all x ∈ D(A).

(14.21)

If x ∈ D(A), then R(λ)(λx − Ax) = x and therefore λR(λ)x − x = R(λ)Ax ≤ Hence

M Ax||, λ−ω

lim λR(λ)x = x for x ∈ D(A).

λ↑+∞

λ > ω. (14.22)

Since λR(λ) ≤ |λ|M λ−ω , λ > ω and the set D(A) is dense in E, the formula (14.22) is true for all x ∈ E by Theorem A.5(ii) (the Banach–Steinhaus theorem). Taking into account that

14.3 The Hille–Yosida theorem

213

Aλ x = λR(λ)Ax,

λ > ω,

we arrive at (14.21). Let us remark that S λ (t) ≤ e−λt

+∞  λ2 ωλ M (λ2 t)m ≤ M e−λt et λ−ω = M e λ−ω t , m m! (λ − ω) m=0

λ > ω, t > 0.

(14.23)

Since Aλ Aμ = Aμ Aλ , S λ (t)Aμ = Aμ S λ (t), t ≥ 0, λ, μ > ω, so, for x ∈ D(A),  S λ (t)x − S μ (t)x =

t

d μ S (t − s)S λ (s)x ds 0 ds  t S μ (t − s)(Aλ − Aμ )S λ (s)x ds = 0  t S μ (t − s)S λ (s)(Aλ − Aμ ) = x ds. = 0

Therefore, by (14.23), S λ (t)x − S μ (t)x ≤ Aλ x − Aμ xM 2

 0

t

ωμ

ωλ

e μ−ω (t−s)+ λ−ω s ds

ωμ

≤ M 2 tAλ x − Aμ xe μ−ω t . Consequently, by (14.21), S λ (t)x − S μ (t)x −→ 0,

for λ, μ ↑ +∞ and x ∈ D(A),

(14.24)

uniformly in t ≥ 0 from an arbitrary bounded set. It follows from Theorem A.5(ii) (the Banach–Steinhaus theorem) and from (14.23) that the convergence in (14.24) holds for all x ∈ E. Define (14.25) S(t)x = lim S λ (t)x, x ∈ E. λ↑+∞

We check easily that S(t), t ≥ 0, is a semigroup of operators and that for x ∈ D(A),  t S(t)x − x = S(r)Ax dr, t ≥ 0. (14.26) 0

Let B be the infinitesimal generator of S(t), t ≥ 0. By (14.26), D(A) ⊂ D(B) and Ax = Bx for x ∈ D(A). It is therefore enough to show that D(A) = D(B). Since S(t) ≤ M eωt , the operator

214

14 Linear control systems

 ˜ R(λ)x =

+∞

e−λt S(t)x dt,

x∈E

0

(λ > ω)

is well defined and continuous. For arbitrary λ > ω, y ∈ E and h > 0 1 ˜ ˜ [S(h)R(λ)x − R(λ)x] h   1 +∞ −λ(t+h) 1 +∞ −λt = eλh e S(t + h)x dt − e S(t)x dt h h h 0  1 h −λt eλh − 1 ˜ R(λ)x − eλh e S(t)x dt. = h h 0 ˜ Hence R(λ)y ∈ D(B) and ˜ (λ − B)R(λ)y = y, If x ∈ D(B) and λ > ω, then  +∞  ˜ R(λ)Bx = e−λt S(t)Bx dt = 0

and therefore

+∞

y ∈ E.

e−λt

0

˜ R(λ)(λ − B)x = x,

(14.27)

d ˜ S(t)x dt = −x + λR(λ)x, dt

x ∈ D(B).

(14.28)

λ > ω.

(14.29)

From (14.27) and (14.28) ˜ R(λ) = (λ − B)−1 ,

In particular, for arbitrary y ∈ E, the equation λx − Bx = y,

x ∈ D(B),

(14.30)

has exactly one solution in D(B). Since x = R(λ)y ∈ D(A) is a solution of this ˜ ˜ equation, R(λ) = R(λ). In particular, D(B) = R(λ)E = R(λ)E = D(A).  In the proof we implicitly assumed that the Banach space E is over the field of real numbers R however the result is true also if E is a Banach space over the field of complex numbers C and the operators A and S(t) are operators acting in such spaces. If this is the case then one defines the resolvent set ρ(A) as the set of all complex numbers λ ∈ C such that for every y ∈ E there exists unique x ∈ D(A) such that λx − Ax = y. The operator Rλ = (λI − A)−1 which transforms elements y onto x is called the resolvent of A corresponding to λ. The complement of the set ρ(A) is called the spectrum of A and denoted by σ(A). It follows from the proof of Theorem 14.4 that

14.4 Phillips’ theorem

215

Proposition 14.2. If for the generator A of a semigroup S(t), t ≥ 0, condition (14.19) is satisfied or if a semigroup S(t) satisfies (14.20) and Re λ > ω, then λ ∈ ρ(A) and  +∞ e−λt S(t)x dt, x ∈ E. (14.31) (λ − A)−1 x = R(λ)x = 0

If one deals with operators and semigroups acting on a Banach space E over reals than one can extend them to the complexification Ec consisting of elements a + ib, a, b ∈ E equipped with the norm ||a + ib||c = ||a|| + ||b|| and multiplication: (α + iβ)(a + ib) = (αa − βb) + i(βa + αb). The extensions Sc (t) and Ac are given by the formulae: Sc (t)(a + ib) = S(t)a + iS(t)b, Ac (a + ib) = Aa + iAb.

§14.4. Phillips’ theorem For control theory, and for the stabilizability question in particular, of great importance is a result saying that the sum of a generator and a bounded linear operator is a generator. It is due to Phillips. Theorem 14.5. If an operator A generates a semigroup on a Banach space E and K : E −→ E is a bounded linear operator then the operator A + K with the domain identical to D(A) is also a generator. Proof. Let S(t), t ≥ 0, be the semigroup generated by A. Then for some constants M ≥ 1 and ω ∈ R S(t) ≤ M eωt ,

t ≥ 0.

It follows from Proposition 14.2 that the operator R(λ) = (λI − A)−1 is given by  +∞

R(λ)x =

e−λt S(t)x dt,

0

x ∈ E, λ > ω.

Since the semigroup e−ωt S(t), t ≥ 0, is generated by A − ωI, without any loss of generality we can assume that S(t) ≤ M,

t ≥ 0.

Let us introduce a new norm  · 0 on E, x0 = sup S(t)x. t≥0

216

14 Linear control systems

Then x ≤ x0 ≤ M x,

x ∈ E.

Hence the norms  · 0 and  ·  are equivalent, and we check easily that S(t)0 ≤ 1, t ≥ 0, 1 R(λ)0 ≤ , λ > 0. λ Assume that λ > K0 . Since KR(λ)0 ≤ K0 R(λ)0 < 1, the operator I − KR(λ) is invertible. Define ˜ R(λ) = R(λ)(I − KR(λ))−1 =

+∞ 

λ > K0 .

R(λ)(KR(λ))m ,

(14.32)

m=0

We will show that 1 , λ − K0

(i)

˜ R(λ) 0 ≤

(ii)

˜ R(λ) = (λ − A − K)−1 .

From (14.32) ˜ R(λ) 0 ≤

+∞ 

R(λ)0 KR(λ)m 0

m=0

≤

1 1 1 ≤ , λ 1 − KR(λ)||0 λ − K0

λ > K0 ,

so the inequality (i) holds. To prove (ii) notice that ˜ ˜ ˜ (λ − A − K)R(λ) = (λ − A)R(λ) − K R(λ) = (I − KR(λ))−1 − KR(λ)(I − KR(λ))−1 = I. Since ˜ R(λ)(λ − A − K) = R(λ)(λ − A − K) +

+∞ 

R(λ)(KR(λ))m (λ − A − K)

m=1

= I − R(λ) +

+∞  m=1

(ii) takes place as well.

(R(λ)K)m −

+∞  m=2

(R(λ)K)m = I,

14.5 Important classes of generators

217

It follows from (i) that m ˜ (R(λ)) 0 ≤

1 , (λ − K0 )m

m = 1, 2, . . . .

Moreover, the operator A + K is closed and the theorem follows from Theorem 14.4.  ˜ Remark. It follows from the proof of the theorem that if S(t), t ≥ 0, is the semigroup generated by A + K then ˜ S(t) ≤ M e(ω+ K )t ,

t ≥ 0.

§14.5. Important classes of generators We will now deduce some consequences of Hille–Yosida’s theorem which help to prove that a given operator is the generator of a semigroup. If A is a closed, linear operator with a dense domain D(A) then the domain D(A∗ ) of the adjoint to operator A consists of all functionals f ∈ E ∗ for which the transformation x −→ f (Ax) has a continuous extension from D(A) to E. If the extension exists then it is unique, belongs to E ∗ and by the very definition it is the value A∗ f of the adjoint operator A∗ on f . The following theorem holds. Theorem 14.6. Let A be a closed operator with a dense domain D(A). If for ω ∈ R and all λ > ω (λI − A)x ≥ (λ − ω)||x

for all x ∈ D(A),

(14.33)

(λI − A∗ )f  ≥ (λ − ω)f 

for all f ∈ D(A∗ ),

(14.34)

then the operator A generates a semigroup of operators S(t), t ≥ 0, such that S(t) ≤ eωt ,

t ≥ 0.

Proof. It follows from (14.33) that Ker(λI − A) = {0}, for λ > ω and that the image of λI − A is a closed, linear space E0 ⊂ E. If E0 = E, then, by Theorem A.4 (the Hahn–Banach theorem), there exists a functional f ∈ E ∗ , f = 0, such that f (λx − Ax) = 0, x ∈ D(A). (14.35) Hence f ∈ D(A∗ ) and A∗ f = λf . By (14.34), f = 0, so, necessarily, E0 = E. The operator λ − A is invertible and, taking into account (14.33), R(λ) ≤ Therefore

1 . λ−ω

(14.36)

218

14 Linear control systems

Rm (λ) ≤

1 , (λ − ω)m

λ > ω, m = 1, 2, . . . ,

(14.37) 

and the result follows by the Hille–Yosida theorem.

Conditions (14.33) and (14.34) have particularly simple form if the space E is Hilbert (over real or complex numbers). Theorem 14.7. Assume that a closed operator A with a dense domain D(A) is defined on a Hilbert space H. If there exists ω ∈ R such that ReAx, x ≤ ωx2 ∗

ReA x, x ≤ ωx||

2

f or x ∈ D(A), ∗

f or x ∈ D(A ),

(14.38) (14.39)

then the operator A generates a semigroup S(t), t ≥ 0, such that S(t) ≤ eωt ,

t ≥ 0.

(14.40)

Proof. We apply Theorem 14.6 and check that (14.33) holds; condition (14.34) can be proved in the same way. Let λ > ω, x ∈ D(A), then λx − Ax2 = (λ − ω)x + (ωx − Ax)2 = (λ − ω)2 x2 + ωx − Ax2 + 2(λ − ω) Reωx − Ax, x. From (14.38) Reωx − Ax, x = ωx||2 − ReAx, x ≥ 0, 

and (14.33) follows.

A linear operator A on a Hilbert space H, densely defined, is said to be self-adjoint if D(A) = D(A∗ ) and A = A∗ . We have the following corollary of Theorem 14.7 Corollary 14.1. A self-adjoint operator A such that ReAx, x ≤ ωx2

for x ∈ D(A),

(14.41)

generates a semigroup, S(t), t ≥ 0. Since the semigroups S λ (·), λ > ω, generated by Aλ = λ(λR(λ) − I), are self-adjoint, the semigroup S(t) = lim S λ (t), t ≥ 0, consists of self-adjoint operators as well.

λ↑+∞

The following way of defining semigroups is due to J.-L. Lions, see, e.g. J. Kisynski [57]. Let us consider a pair of Hilbert spaces V ⊂ H, with scalar products and norms ((·, ·)), ·, ·, ·, |·| such that V is a dense subset of H and the embedding of V into H is continuous. Let a(·, ·) be a continuous, bilinear function defined on V . In the case of complex Hilbert spaces V and H instead of linearity with respect to the second variable we require from a(·, ·) antilinearity

14.5 Important classes of generators

¯ a(u, λv) = λa(u, v),

219

u, v ∈ V,

λ ∈ C.

Define D(A) = {u ∈ V ; a(u, ·) is continuous in norm | · |}.

(14.42)

Since V is dense in H, for u ∈ D(A) there exists exactly one element Au ∈ H such that Au, v = a(u, v), v ∈ V. (14.43) The following theorem is due to J.-L. Lions. Theorem 14.8. If for some constants ω ∈ R, α > 0, Re a(u, u) + αu2 ≤ ω|u|2 ,

u ∈ V,

(14.44)

then the operator A defined by (14.42), (14.43) generates a semigroup S(t), t ≥ 0, on H and |S(t)| ≤ eωt , t ≥ 0. Proof. Let us fix λ > ω and define a new bilinear functional aλ (u, v) = λu, v − a(u, v),

u, v ∈ V.

From (14.44), αu2 ≤ λu, u − Re a(u, u) ≤ Re(λu, u − a(u, u)) ≤ Re aλ (u, u) ≤ |aλ (u, u)|,

u ∈ V.

We will need the following abstract result. Proposition 14.3. (The Lax–Milgram theorem). Let a(·, ·) be a continuous bilinear functional defined on a Hilbert space H such that, for some α > 0, |a(u, u)| ≥ α|u|2 ,

u ∈ H.

Then there exists a continuous, invertible, linear operator F from H onto H such that F u, v = a(u, v), u, v ∈ H. Proof. For arbitrary u, the functional v −→ a(u, v) is linear and continuous, and by Theorem A.7 (the Riesz representation theorem) there exists exactly one element in H, which we denote by F u, such that a(u, v) = v, F u,

v ∈ V.

Hence F u, v = a(u, v), and it is easy to check that F is a linear, continuous transformation. Moreover,

220

14 Linear control systems

|F u| = sup{|F u, v|; |v| ≤ 1} = sup{|a(u, v)|; |v| ≤ 1}     u   ≥ α|u|, u ∈ H. ≥ a u, |u|  Consequently the image F (H) of the transformation F is a closed linear subspace of H. If F (H) = H, then by the Hahn–Banach theorem there exists v ∈ H, v = 0, such that F u, v = 0 for all u ∈ H. In particular,

F v, v = 0.

But

F v, v = a(v, v) = 0, 

a contradiction.

It follows from Proposition 14.3 that there exists a linear, invertible operator A˜λ : V −→ V such that aλ (u, v) = ((A˜λ u, v))

for u, v ∈ V.

Let Aλ be defined by (14.42)–(14.43) with a(·, ·) replaced by aλ (·, ·). Then Aλ = λI − A, D(Aλ ) = D(A). Let, moreover, an operator J : H −→ V be defined by the relation h, v = ((Jh, v)),

h ∈ H, v ∈ V.

We directly check that the image J(H) is dense in V . For h ∈ H we set u = A˜−1 λ J(h). Then A˜λ u = Jh, aλ (u, v) = ((A˜λ u, v)) = ((Jh, v)) = h, v, hence u ∈ D(Aλ ) and Aλ u = h. On the other hand, if u ∈ D(Aλ ) and Aλ u = h then for v ∈ V aλ (u, v) = Aλ u, v = ((JAλ u, v)) = ((A˜λ u, v)). Consequently u = A˜−1 λ Jh. This way we have shown that D(Aλ ) = A˜−1 λ J(H), −1 ˜ A = A−1 J. λ

For arbitrary λ > ω the operator

λ

(14.45)

14.6 Specific examples of generators

221

R(λ) = (λI − A)−1 = A−1 λ is a well-defined linear operator. In particular, operators Aλ and A are closed. It follows from (14.45) and the denseness of J(H) in V that D(A) is dense in H and, moreover, for u ∈ D(A) |(λI − A)u|2 = |(λ − ω)u + (ω − A)u|2 = (λ − ω)2 |u|2 + 2(λ − ω)[ω|u|2 − ReAu, u] + |(ω − A)u|2 ≥ (λ − ω)2 |u|2 . m

(λ)|≤1 1 Therefore, for λ > ω, we have |R(λ)| ≤ λ−ω and thus |R(λ−ω) m , m = 1, 2, . . . . The conditions of the Hille–Yosida theorem are satisfied. The proof of the theorem is complete. 

Remark. In the paper [111] a class of decomposable generators of the form P A, AP where A is a generator and P a bounded operator, was introduced. It is shown that they can be used to study partial differential equations with non-local boundary conditions as well as some delay equations.

§14.6. Specific examples of generators We will illustrate the general theory by showing that specific differential operators are generators. We will also complete examples introduced earlier. Example 14.5. Let (a, b) ⊂ R be a bounded interval and H = L2 (a, b). Let us consider an operator d A= dξ with the following domain:   dx D(A) = x ∈ H; x is absolutely continuous on [a, b], ∈ H, x(b) = 0 . dξ We will show that the operator A is the generator of the so-called left-shift semigroup S(t), t ≥ 0:  x(t + ξ), if t + ξ ∈ (a, b), S(t)x(ξ) = (14.46) 0, if t + ξ ∈ / (a, b). We will give two proofs, first a direct one and then a longer one based on Theorem 14.7. Let λ > 0 and y ∈ H. The equation λx − Ax = y,

x ∈ D(A),

222

14 Linear control systems

is equivalent to a differential equation dx (ξ) = λx(ξ) − y(ξ), dξ x(b) = 0, and has a solution x = R(λ)y of the form  b x(ξ) = eλ(ξ−s) y(s) ds,

ξ ∈ (a, b),

ξ ∈ [a, b].

ξ

On the other hand, for the semigroup S(t), t ≥ 0, given by (14.46)  b−ξ  +∞  e−λt S(t) dt y(ξ) = e−λt y(ξ + t) dt 0

0

 =

b

eλ(ξ−s) y(s) ds,

ξ ∈ [a, b], λ > 0.

ξ

So the resolvent of the generator S(t), t ≥ 0, is identical with the resolvent of the operator A and therefore A is the generator of S(t), t ≥ 0 (see Proposition 14.1 and Proposition 14.2). To give a proof based on Theorem 14.7, let us remark that for x ∈ D(A)  b  b dx dx (ξ)x(ξ) dξ = −x2 (a) − (ξ)x(ξ) dξ, Ax, x = ds a a dξ hence

1 Ax, x = − x2 (a) ≤ 0. 2 We find now the adjoint operator A∗ . If y ∈ D(A∗ ), then there exists z ∈ H such that Ax, y = x, z for all x ∈ D(A). Therefore  b  b dx y dξ = x(ξ)z(ξ) dξ (14.47) a dξ a  b  s  b  b dx dx ds dξ = z(ξ) z(ξ) dξ ds. =− a ξ ds a ds a To proceed further we need the following lemma, which will also be used in some other examples. Lemma 14.2. Assume that ψ is an integrable function on an interval [a, b] such that  b (m) d ϕ (ξ)ψ(ξ) dξ = 0 (14.48) (m) a dξ

14.6 Specific examples of generators

223

for some m = 0, 1, . . . and arbitrary ϕ ∈ C0∞ (a, b). Then ψ is identical, almost everywhere, with a polynomial of the order m − 1. Proof. If m = 0 the identity (14.48) is of the form 

b

(14.49)

ϕ(ξ)ψ(ξ) dξ = 0. a

By a standard limit argument we show first that (14.49) holds for all continuous and bounded functions and therefore also for all bounded and measurable functions ϕ. Let ϕn = (sgn ψ)χ(−n,n) (ψ), 

Then



b

0=

n = 1, 2, . . . .

ϕn (ξ)ψ(ξ) dξ = a

Since ψ is integrable,

 |ψ|≥n



|ψ|≥n

ψ(ξ) dξ +

|ψ| 0, ξ ∈ O, ∂t2 ∂z (0, ξ) = y(ξ), ξ ∈ O, z(0, ξ) = x(ξ), ∂t ∂ z(t, ξ) = 0, t > 0, ξ ∈ Γ, ∂n where

∂ ∂n

is the normal derivative calculated at the points of Γ .

§14.7. The integral representation of linear systems Let us consider a linear control system y˙ = Ay + Bu,

y(0) = x,

(14.61)

on a Banach space E. We will assume that the operator A generates a semigroup of operators S(t), t ≥ 0, on E and that B is a linear, bounded operator acting from a Banach space U into E. We will study first the case when U = E and the equation (14.61) is replaced by y˙ = Ay + f,

y(0) = x,

with f an E-valued function. Any continuous function y: [0, T ] −→ E such that

(14.62)

230

14 Linear control systems

(i) y(0) = x, y(t) ∈ D(A), t ∈ [0, T ], (ii) y is differentiable at any t ∈ [0, T ] and dy (t) = Ay(t) + f (t), dt

t ∈ [0, T ]

is called a strong solution of (14.62) on [0, T ]. Theorem 14.9. Assume that x ∈ D(A), f (·) is a continuous function on [0, T ] and y(·) is a strong solution of (14.62) on [0, T ]. Then  y(t) = S(t)x +

0

t

S(t − s)f (s) ds,

t ∈ [0, T ].

(14.63)

Proof. Let us fix t > 0 and s ∈ (0, t). It follows from the elementary properties of generators that d dy(s) S(t − s)y(s) = − AS(t − s)y(s) + S(t − s) ds ds = − S(t − s)Ay(s) + S(t − s)[Ay(s) + f (s)] = S(t − s)f (s). Integrating the above identity from 0 to t we obtain  t y(t) − S(t)y(0) = S(t − s)f (s) ds. 0



The formula does not always define a strong solution to (14.62). Additional conditions either on f or on the semigroup S(t), t ≥ 0, are needed. Here is a typical result in this direction. Theorem 14.10. Assume that f (·) is a function with continuous first derivatives on [0, +∞] and x ∈ D(A). Then equation (14.62) has a strong solution. Proof. The function S(·)x is, by Theorem 14.2, a strong solution of equation (14.62) with f = 0. Therefore we can assume that x = 0. For all, t ≥ 0,  t  t  s   df y(t) = (r) dr ds S(t − s)f (s) ds = S(t − s) f (0) + 0 0 0 dr  t  t  t  df (14.64) = S(t − s)f (0) ds + S(t − s) (r) ds dr. dr 0 0 r We will show that for x ∈ E and r ≥ 0  r  S(s)x ds ∈ D(A) and S(r)x − x = A 0

r

 S(s)x ds .

(14.65)

0

It follows from Theorem 14.2 that (14.65) holds for all x ∈ D(A). For an arbitrary x ∈ E let (xn ) be a sequence of elements from D(A) converging to x. Then

14.7 The integral representation of linear systems

 S(r)xn − xn = A

 S(s)xn ds −→ S(r)x − x

r

0



and

r

0

231

 S(s)xn ds −→

r

S(r)x ds. 0

Since the operator A is closed, (14.65) holds for all x. For arbitrary t ≥ r ≥ 0, we have, by (14.65), 

t

S(t − s)f (0) ds ∈ D(A)   t S(t − s)f (0) ds , S(t)f (0) − f (0) = A

(14.66)

0



0

t

df S(t − s) (r) ds ∈ D(A) dr 0  t  df df (S(t − r) − I) (r) = A S(t − s) (r) ds . dr dr r

(14.67)

Let us remark that for an arbitrary closed operator A, if functions ψ(·) and Aψ(·) are continuous, then for arbitrary r ≥ 0  r  r   r ψ(s) ds ∈ D(A) and A ψ(s) ds = Aψ(s) ds. 0

0

0

Therefore (14.64), (14.66) and (14.67) imply that y(t) ∈ D(A) and  Ay(t) = (S(t) − I)f (0) +

t

0

(S(t − r) − I)

df (r) dr, dr

t ≥ 0.

Since  y(t) = dy (t) = S(t)f (0) + dt



t

S(r) 0

0

t

S(s)f (t − s) ds,

df (t − r) dr = S(t)f (0) + dr

 0

t

S(t − r)

df (r) dr. dr

Summarizing,  t dy df (t) − Ay(t) = S(t)f (0) − f (0) + (S(t − r) − I) (r) dr dt dr 0  t df − S(t)f (0) + S(t − r) (r) dr = 0. dr 0 The proof of the theorem is complete.



232

14 Linear control systems

Corollary 14.2. If a function f is of class C 1 and x ∈ D(A), then equation (14.62) has exactly one strong solution, and the solution is given by  y(t) = S(t)x +

0

t

S(t − s)f (s) ds,

t ≥ 0.

(14.68)

The function y(t), t ≥ 0, given by the formula (14.68) is well defined for an arbitrary Bochner integrable function u(·). It is easy to show that it is a continuous function. The following proposition, the proof of which is left as an exercise, shows that the function y given by (14.68) is a uniform limit of strong solutions. Proposition 14.4. Assume that f is a Bochner integrable function on [0, T ] and x ∈ E. If (fk (·)) and (xk ) are sequences such that (i) xk ∈ D(A), for all k = 1, 2, . . . and lim xk = x, k

(ii) fk (·) are C 1 functions from [0, T ] −→ E, n = 1, 2, . . . , and  T fk (s) − f (s) ds = 0, lim k

0

then the sequence of functions  t yk (t) = S(t)xk + S(t − s)fk (s) ds, 0

k = 1, 2, . . . , t ∈ [0, T ],

converges uniformly to y(·) given by (14.68). The function y(t), t ≥ 0, given by the formula (14.68) can be identified with more classical concept of weak solution. Namely a continuous function y is a weak solution to the equation (14.62) if for arbitrary x∗ ∈ D(A∗ ) and t≥0  t  t y(s), A∗ x∗  ds + f (s), x∗  ds. (14.69) y(t), x∗  = x, x∗  + 0

0

We have the result: Proposition 14.5. Assume that E is a Hilbert space. If a function f is Bochner integrable then equation (14.62) has exactly one weak solution, and the solution is given by (14.68). Proof. To show that y given by (14.68) is a weak solution of (14.62) one uses the fact that for x∗ ∈ D(A∗ ) the function S ∗ (s)x∗ , s ≥ 0, is continuously differentiable and d ∗ S (s)x∗ = S ∗ (s)A∗ x∗ , ds Changing the order of integration one obtains

s ≥ 0.

14.7 The integral representation of linear systems

 0

t

233

y(s), A∗ x∗  ds = y(t), x∗  −

 0

t

f (s)ds, x∗



and thus y is a weak solution of the equation. To show the converse one first generalizes (14.69) to 

t

y(t), g(t) = x, g(0) + y(s), g (s) + A∗ g(s) ds 0  t f (s), g(s) ds, + 0

valid for functions g continuously differentiable as functions with values in D(A∗ ). To do so one starts with functions g of the form g(s) = aψ(s) where a ∈ D(A∗ ) and ψ is a real continuously differentiable function and then by approximating general g by linear combinations of such functions. To complete the proof one inserts into the formula function g(s) = S ∗ (t − s)x∗ , s ∈ [0, t] and arrives at the identity ∗

∗

y(t), x  = S(t)x, x  +

 0

t

 S(t − s)f (s) ds, x∗ .

Since it is valid for all functionals x∗ in D(A∗ ) and the set D(A∗ ) is dense  in E ∗ , the required identity follows. Remark. To infer that the set D(A∗ ) is dense, the assumption that E is Hilbert, see Theorem 14.3, was used. It follows from the proof that in the definition of the weak solution one can consider a smaller set K of functionals x∗ ∈ D(A∗ ) which is dense in E ∗ . Such sets are called cores of the semigroups. Strong, and therefore weak, solutions of (14.62) are not in general identical with classical solutions of partial differential equations of which they are abstract models. To see the difference let us go back to heat equation (14.10). Physical arguments which led to its derivation did not determine which function space should be taken as the state space E. There are always several choices possible. Let us assume, for instance, that the state space is the Hilbert space H = L2 (0, L) and that the generator A is given by Ax = σ 2

d2 x , dξ 2

with the domain D(A) described in Example 14.3. Assume that b(·) ∈ H and let f = ub, u ∈ R. Equation (14.62) is then a version of (0.11), (0.12) and (0.13). If a function u(·) is of class C 1 and x ∈ D(A) then, by Theorem 14.10,

234

14 Linear control systems

 y(t) = S(t)x +

t

0

S(t − s)bu(s) ds ∈ D(A),

(14.70)

dy (t) = Ay(t) + bu(t) for t ≥ 0. dt A solution y(·) given by (14.70) is nonclassical because, for each t ≥ 0, y(t) is a class of functions and the inclusion y(t) ∈ D(A) implies only that there exists a representative of the class which is sufficiently smooth and satisfies the Dirichlet boundary conditions. Whether one could find a representative y(t, ξ), ξ ∈ [0, L], which is a smooth function of (t, ξ) ∈ [0, +∞) × [0, L] satisfying (0.11)–(0.13), requires an additional analysis. Moreover, the continuity of the strong solution and its differentiability are in the sense of the space L2 (0, L) which is again different from the classical one. However, despite the mentioned limitations, the abstract theory is a powerful instrument for studying evolutionary problems. It gives a first orientation of the problem. Moreover, the strong solution often has a classical interpretation, and the classical solution always is the strong one.

§14.8. Delay systems An important role in applications is played by delay systems whose typical example is the following: y(t) ˙ =

l 

Ak y(t + θk ) + Bu(t)

(14.71)

k=0

y(0) = x ∈ Rn ,

y(θ) = ϕ(θ),

θ ∈ [−h, 0[,

where h is a positive constant, A0 , . . . , Al are n × n matrices, B is n × m matrix, θ0 = 0 > θ1 > . . . > θl = −h and ϕ is a function defined on [−h, 0[. If t + θk < 0 for some t ≥ 0 and k = 1, 2, . . . , then the term y(t + θk ) is replaced by ϕ(t + θk ). One can find solutions to (14.71) by the so-called step method . If 0 ≤ t < −θ1 then one should have y(t) ˙ = A0 y(t) +

l  k=1

Ak ϕ(t + θk ) + Bu(t)

(14.72)

y(0) = x and the equation (14.72) determines the solution on the interval [0, −θ1 [, which we denote by y. In the following interval [−θ1 , −θ2 [ one has to solve the equation:

14.9 Existence of solutions to delay equation

y(t) ˙ = A0 y(t) + A1 y(t + θ1 ) +

l 

235

Ak ϕ(t + θk ) + Bu(t)

k=2

(14.73)

y(−θ1 ) = y(−θ1 ), which obviously has a unique solution denoted again by y. Repeating the procedure on consecutive intervals [−θ2 , −θ3 [, . . . , [−θl−1 , h[, one finds solution y to (14.71) on [0, h[. In a similar way one solves (14.71) on intervals [h, 2h[, [2h, 3h[.

§14.9. Existence of solutions to delay equation In this section we consider general delay system  0 A(ds) y(t + s) + Bu(t), y(t) ˙ =

(14.74)

−h

y(0) = x ∈ R , n

where

y(θ) = ϕ(θ),

−h ≤ θ < 0,

  A(ds) = Aij (ds) i,j=1,...,n

(14.75)

is a matrix of finite measures on [−h, 0]. As far as the function ϕ is concerned we assume only that ϕ ∈ L2 (−h, 0; Rn ). If y(t) = (yi (t))i=1,...,n , then (14.74) should be understood as a system of equations y˙ i (t) =

n   j=1

0

−h

Aij (ds) yj (t + s) +

n 

bik uk (t),

t ≥ 0,

(14.76)

k=1

where bik , i = 1, . . . , n, k = 1, . . . , m, are elements of the matrix B and u1 (t), . . . , um (t), t ≥ 0, are coordinates of the function u(t) ∈ Rn , t ≥ 0. Note that if l  A(ds) = Ak δ{θk } (ds) k=0

then (14.74) becomes (14.71). In this section we show that (14.74) has a unique solution. In the next one we prove that delay systems can be regarded as linear control systems (14.61) with values in a Hilbert space H being the product of Rn and L2 (−h, 0; Rn ), Rn × H= L2 (−h, 0; Rn ) equipped with the scalar product

(14.77)

236

14 Linear control systems

    0  x x h(θ), h(θ) n dθ, = x, xn + , h h −h H n where a, bn = i=1 ai bi , for a = (a1 , . . . , an ), b = (b1 , . . . , bn ). This interpretation allows us to apply the results on the general linear systems, which will be developed in the following chapters, to the delay control systems. This approach was originated by M. C. Delfour and S. K. Mitter in the paper [27]. In the first reading we recommend to consider the scalar case, when n = m = 1. To formulate the existence result let us rewrite (14.74) in the integral form:  t  y(t) = x + y(θ) = ϕ(θ),

0

0

−h

 t  A(dr) y(r + s) ds + f (s) ds, 0

t ∈ [0, T ],

−h ≤ θ < 0.

(14.78) (14.79)

Theorem 14.11. For arbitrary T > 0, x ∈ Rn , f ∈ L1 (0, T ; Rn ), ϕ ∈ L2 (−h, 0; Rn ) there exists a unique function y, continuous on [0, T ], satisfying (14.78)–(14.79). For the proof we need two lemmas. Recall that if ν is a finite scalar measure on I = [−h, 0] then by the Jordan decomposition (see §A.1) ν = ν+ − ν− where ν+ and ν− are nonnegative measures with disjoint supports. Define a nonnegative measure ν by the formula ν = ν+ + ν− . Then for measurable functions ψ:       ψ(s) ν(ds) ≤ |ψ(s)| ν(ds). I

I

Similarly, for the matrix A of measures (14.75), we define a nonnegative measure A as follows: n  Aij . A = i,j=1

We have the following: Lemma 14.3. For ψ taking values in Rn we have   √     A(ds) ψ(s) ≤ n A(ds) ψ(s). I

(14.80)

I

Proof. Let Ψj (s), j = 1, 2, . . . , n, s ∈ I be coordinate functions of ψ(i), s ∈ I. Then

14.9 Existence of solutions to delay equation

237

n  n     2   Aij (ds) ψj (s)  A(ds) ψ(s) = I

i=1

n  n  

≤

i=1

≤

|ψj (s)| Aij (ds)

j=1 I n  

|ψ(s)|

n 

I

i=1

I

j=1

2

≤

n  n   i=1

2 |ψ(s)| Aij (ds)

I

j=1

2  2 Aij (ds) ≤ n |ψ(s)| A(ds) .

2

I

j=1

Lemma 14.4. For arbitrary ϕ ∈ L2 (−h, 0; Rn ) the function 

−s

A(dr) ϕ(r + s),

−h

s ∈ [0, h],

is integrable and  h    0

−s

−h

 √   A(dr) ϕ(r + s) ds ≤ h

0

−h

A(dr)



0

−h

|ϕ(θ)|2 dθ

1/2

. (14.81)

Proof. Note that by Lemma 14.3  h    0

−s

−h

   A(dr) ϕ(r + s) ds ≤  ≤

≤



0

−h

 A(dr)

0

−h 0

−h



0

−h  h

A(dr)

|ϕ(θ)| dθ ≤

0

−s

−h

A(dr) |ϕ(r + s)| ds

1[−h,−s] (r)|ϕ(r + s)| ds



0

−h

 √  A(dr) h

0

−h



|ϕ(θ)|2 dθ

1/2

. 2

Proof of Theorem 14.11 Let us note that if 0 ≤ t ≤ h then (14.78)–(14.79) is equivalent to  t  t  −s  y(t) = x + f (s) ds + A(dr) ϕ(r + s) ds 0

 t  +

0

 t =x+

0

0

0

−s

−h

 A(dr) y(r + s) ds

f (s) +

For h ≤ t ≤ T we have



−s

−h

 t   A(dr) ϕ(r + s) ds + 0

0

−s

 A(dr) y(r + s) ds.

238

14 Linear control systems

 y(t) = x +

 h 

t

f (s) ds +

0

 t  +

−h

h t

 =x+

0

0

0

 =x+

0

−s

0

−s

−h

 h 

f (s) ds + 0

−s

 A(dr) ϕ(r + s) ds

 t   A(dr) y(r + s) ds +

t

0

+

 h 

0

 h 

−h

 A(dr) y(r + s) ds

 A(dr) y(r + s) ds

f (s) ds +

 h  +

0

0

0

−h

h

−s

−h

0

 A(dr) y(r + s) ds

 A(dr) ϕ(r + s) ds

 t   A(dr) y(r + s) ds +

0

−h

h

 A(dr) y(r + s) ds.

Without loss of generality we can assume that T > h. For functions g ∈ C([0, T ]; Rn ) define the function Kg(t), t ∈ [0, T ], by the formula ⎧ t  0  ⎪ ⎪ A(dr) g(r + s) ds, ⎨  t  Kg(t) = 0 h −s0  ⎪ ⎪ ⎩ A(dr) g(r + s) ds + 0

−s

h

if t ≤ h, 0

−h

 A(dr) g(r + s) ds,

if t ≥ h.

(14.82) It is easy to see that K is a linear operator from C([0, T ]; Rn ) into C([0, T ]; Rn ). We fix β > 0, equip C([0, T ]; Rn ) with the norm gβ = sup eβt |g(t)|, 0≤t≤T

and estimate the norm of K. If t ∈ [0, h], using Lemma 14.3: e−βt |Kg(t)| ≤ e−βt √ ≤ e−βt n √ ≤ e−βt n

 t 

0

0

−s 0

0

−s 0

 t 

 t  √ −βt ≤e n 0

√ ≤ n gβ



−s

t

 t     0

e

Similarly for h ≤ t ≤ T :

−s

  A(dr) g(r + s)  ds

 A(dr)|g(r + s)| ds  A(dr)eβ(r+s) |g(r + s)|e−β(r+s) ds  A(dr)eβ(r+s) ds · gβ

−β(t−s)

0

0

√  0 n ds A(dr) ≤ A(dr) gβ . β −h −h 

0

14.9 Existence of solutions to delay equation

√ e−βt |Kg(t)| ≤ e−βt n

 h  0

−s

 t 

√ + e−βt n

h

√ ≤ e−βt n

 h  0

≤

 n gβ

−s

h 0

−h

 A(dr) |g(r + s)| ds 0

−h 0

 t 

√ + e−βt n √

0

 A(dr) g(r + s) ds

 A(dr) eβ(r+s) ds · gβ

0

−h

 A(dr) eβ(r+s) ds · gβ

  −βt A(dr) e

√  0 n A(dr) gβ . ≤ β −h Consequently

239

h

βs

e

0

−βt



t

ds + e

eβs ds



h

√  0 n A(dr). Kβ ≤ β −h

The equation (14.78)–(14.79) can be treated as a fixed point problem in C([0, T ]; Rn ): y(t) = φ(t) + Ky(t), t ∈ [0, T ], (14.83) where ⎧  t  −s  ⎪ ⎪ A(dr) ϕ(r + s) ds, if t ≤ h, ⎨x + 0 t −h  h  −s  φ(t) = (14.84)  ⎪ ⎪ ⎩x + A(dr) ϕ(r + s) ds, if t ≥ h. f (s) ds + 0

0

−h

By Lemma 14.4, the function is well defined, it is also clear that it is continuous. The problem (14.83) has a unique solution because if β is chosen so that √  0 n cβ = A(dr) < 1. (14.85) 1 − cβ −h Then K is a contraction and the Banach fixed point theorem applies. Remark. If (14.85) holds then we get an estimate for the solution y: yβ ≤

1 φβ . 1 − cβ

2

240

14 Linear control systems

§14.10. Semigroup approach to delay systems For arbitrary t ≥ 0, x ∈ Rn , ϕ ∈ L2 (−h, 0; Rn ) define the operator St by the formula   x y(t) St = (14.86) ϕ y(t + ·) where y is the unique solution of the equation  0 A(dr) y(s + r), y(0) = x, y(θ) = ϕ(θ), θ ∈ [−h, 0[. y(s) ˙ =

(14.87)

−h

Proposition 14.6. The operators (St ) form a C0 -semigroup on the space H. Proof. Let us fix x and ϕ and thus the function y as well. Define the function z(v), v ≥ −h, as the unique solution of the equation  0 A(dr) z(v + r), z(0) = y(t), z(θ) = y(t + θ), θ ∈ [−h, 0[. (14.88) z(v) ˙ = −h

Then z is given by the formula z(v) = y(t + v),

v ≥ −h.

(14.89)

In fact, from (14.89) 



0

z(v) ˙ = y(t ˙ + v) =

A(dr) y(t + v + r) = h

0

−h

A(dr) z(v + r),

and z satisfies the appropriate initial conditions. Thus      x z(s) y(t + s) x = = = St+s , Ss S t ϕ z(s + ·) y(t + s + ·) ϕ and the semigroup property follows. To see that operators St , t ≥ 0, are continuous in H it is enough to consider t < h, and for larger times r ≥ h, use the semigroup property Sr = (Sr/k )k , where natural k is such that kr < h. Thus fix t < h and T ∈ (t, h). Let β > 0 chosen in the proof of Theorem 14.11 be such that (14.85) holds and the Remark at the end of §14.9 can be applied. Thus   sup e−βt |y(t)| ≤

0≤t≤T

where

 t  φ(t) = x +

By Lemma 14.4, for s ∈ [0, T ]:

0

1 sup e−βt |φ(t)| 1 − cβ 0≤t≤T −s

−h

 A(dr) ϕ(r + s) ds.

14.10 Semigroup approach to delay systems

√  |φ(s)| ≤ |x| + h

0

−h

241

 A(dr) · ϕL2 (−h,0;Rn ) .

Thus |y(s)| ≤ eβs and  T 0

|y(s)|2 ds ≤

√  1  |x| + h 1 − cβ

2 (1 − cβ )2

 0

T



0

−h

  A(dr) · ϕL2 (−h,0;Rn )

  e2βs ds |x|2 +h

0

−h

 2 A(dr) ·ϕ2L2 (−h,0;Rn ) .

It is now clear  that there exists a constant C > 0 such that for all t ≤ T , arbitrary ϕx ∈ H          S t x  ≤ C  x  .    ϕ H ϕ H It remains to show that the semigroup is strongly continuous at 0. To see this note that for t ∈ (0, h)    2  0   St x − x  = |y(t) − x|2 + |y(t + θ) − ϕ(θ)|2 dθ  ϕ ϕ  −h  −t  0 |ϕ(t + θ) − ϕ(θ)|2 dθ + |y(t + θ) − ϕ(θ)|2 dθ. = |y(t) − x|2 + −h

−t

All the three terms converge to 0 as t ↓ 0. The first one, because y is continuous, the second by Theorem A.10 and the third one, because y is continuous and ϕ ∈ L2 (−h, 0; Rn ). 2 The next theorem describes the form at the infinitesimal generator of the semigroup (St ). Denote by W 1,2 (−h, 0; Rn ) the space of Rn -valued functions f defined on the interval [−h, 0] such that  f (θ) = a +

θ

g(s) ds, −h

θ ∈ [−h, 0],

for some a ∈ Rn and g ∈ L2 (−h, 0; Rn ). That is the space of absolutely continuous functions whose first derivative is square integrable. Theorem 14.12. The domain D(A) of the infinitesimal generator A of the semigroup (St ) is of the form   x D(A) = ; z ∈ W 1,2 (−h, 0; Rn ), x = z(0) . z   For each xz ∈ D(A):

242

14 Linear control systems

A

 0

 x = z

−h

A(dθ) z(θ) dz dθ

.

Proof.   Let λ be a positive number such that for some M > 0, all t ≥ 0 and all ϕx ∈ H:          St x  ≤ M eλt  x  .    ϕ H ϕ H   As we know the domain of the generator A consists of all elements az of the form   +∞   a x x e−λt St = dt, ∈ H. z ϕ ϕ 0    y(t)  , where Since St ϕx = y(t+·)  y(t) ˙ =

0

y(0) = x, y(θ) = ϕ(θ), θ ∈ [−h, 0[,

A(dr) y(t + r),

−h

   +∞ y(t) a −λt e dt = y(t + ·) z 0

we have

and consequently  a=

+∞

−λt

e

 y(t) dt,

z(θ) =

0

+∞

0

e−λt y(t + θ) dt.

Thus a = z(0) and 

+∞

z(θ) =

−λ(s−θ)

e θ



= eλθ

0

+∞

 λθ

+∞

y(s) ds = e

e−λs y(s) ds + eλθ 

= eλθ z(0) + eλθ

0

θ  0

e−λs y(s) ds

e−λs ϕ(s) ds

θ

e−λs ϕ(s) ds.

θ

We see that z ∈ W 1,2 (−h, 0; Rn ) and if z ∈ W 1,2 (−h, 0; Rn ) then there exists unique ϕ ∈ L2 (−h, 0; Rn ) such that  z(θ) = eλθ z(0) + eλθ

0

e−λs ϕ(s) ds,

θ ∈ [−h, 0].

θ

Moreover, from (14.90) dz = λeλ0 z(0) + λeλθ dθ

 θ

0

e−λs ϕ(s) ds − eλθ e−λθ ϕ(θ)

(14.90)

14.10 Semigroup approach to delay systems

243

and thus

dz = λz(θ) − ϕ(θ), θ ∈ [−h, 0[. dθ We can assume that λ was chosen large enough that

(14.91)

lim e−λt |y(t)| = 0.

t→+∞

Note that  z(0) =

+∞

−λt

e 0

 y(t) dt =

+∞  −λt

0 +∞

e −λ

y(t) dt

+∞   e−λt

e−λt y(t) y (t) dt + λ λ 0 0  +∞ −λt  0  e x A(dθ) y(t + θ) dt = + λ λ 0 −h  0 1 x A(dθ) z(θ), = + λ λ −h =−

and consequently

 λz(0) −

0

−h

(14.92)

A(dθ) z(θ) = x.

Summarizing (14.91) and (14.92) gives   0  μ(dθ) z(θ) z(0) x −h λ − = dz z ϕ dθ 2

and the proof is complete. Theorem 14.13. The domain D(A∗ ) consists of all pairs ϕ(s) = g(s) + A∗ [−h, s] x,

x ϕ

such that

s ∈ [−h, 0],

where g ∈ W 1,2 (−h, 0; Rd ), g(−h) = 0 and   ∗ A [−h, 0] x + g(0) x . A∗ = ϕ − dϕ dθ

(14.93)

    Proof. An element ϕx ∈ D(A∗ ) if and only if there exists ϕx ∈ H such that   for all ϕx ∈ D(A):     x x x x = , . A , ϕ ϕ ϕ H ϕ H If this is the case then

(14.94)

244

14 Linear control systems

  x ∗ x A = . ϕ ϕ

(14.95)

Note that %  &   ϕ(0) x x x 0 = , , ϕ ϕ H ϕ ϕ(0) − · dϕ ds H

 0  0  0 dϕ (s), ϕ(r) dt ds = ϕ(0), xd + ϕ(0), ϕ(r)d dr − 1[r,0] (s) ds −h −h −h d

 s  0  0   dϕ (s), = ϕ(0), x + ϕ(r) dr − ϕ(r) dr . d −h −h ds −h d Similarly  

 0  0  x dϕ x (r), ϕ(r) dr A , = A(dr) ϕ(r), x + ϕ ϕ H d −h −h dr d 

 0  0  0 dϕ dϕ ds , x + (r), ϕ(r) dr A(dr) ϕ(0) − = ds −h r −h dr d d  0

 0  dϕ dϕ ∗ (r), ϕ(r) ds 1[r,0] (s) (s), x + = ϕ(0), A [−h, 0] x − ds −h −h dr d d

 0  dϕ (s), −A∗ [−h, s] x + ϕ(s) ds. = ϕ(0), A∗ [−h, 0] x + −h ds d Thus (14.94) holds for all   0 = ϕ(0), x +

0

−

−h

ϕ

∈ D(A) if and only if

ϕ(r) dr − A∗ [−h, 0] x

−h 0 



ϕ(0)

dϕ (s), ds



s −h

 d

 ϕ(r) dr − A∗ [−h, s] x + ϕ(s) ds. (14.96) d

2 d Since dϕ ds might be an arbitrary element of L (−h, 0; R ) and ϕ(0) an arbitrary d element of R , we see that (14.96) holds if and only if

 x+ 

s

−h

0

−h

ϕ(r) dr − A∗ [−h, 0] x = 0,

ϕ(r) dr − A∗ [−h, s] x + ϕ(s) = 0,

s ∈ [−h, 0].

(14.97) (14.98)

    Consequently ϕx ∈D(A∗ ) if and only if (14.97), (14.98) hold for some ϕx ∈H. Taking into account that the function

14.10 Semigroup approach to delay systems

 g(s) = −

245

s

s ∈ [−h, 0],

ϕ(r) dr, −h

is an arbitrary element of W 1,2 (−h, 0; Rd ), such that g(−h) = 0, the description of D(A∗ ) follows. From (14.97) one gets x = A∗ [−h, 0] x + g(0) and this gives (14.93). 2 The following result will be used when the rate of growth of the delay equations will be discussed in Section 16.3. Proposition 14.7. For arbitrary t ≥ 2h, the operators St are compact as operators acting on H. Proof. According to the definition one has to show that the unit ball B1 in H is transformed by St into a relatively compact subset of H. Assume that  0 |x|2 + |ϕ(θ)|2 dθ ≤ 1 (14.99) −h

and let y be the solution of the delay equation (14.87). Then y(s) = φ(s) + Ky(s),

t ∈ [0, T ],

when we choose and fix T ≥ 2h, and function φ and operator K are given by formulae (14.84) and (14.82), respectively. Note that  h  φ(s) = x + and by Lemma 14.4 √  |φ(s)| ≤ |x| + h

0

0

−h

−u

−h

 A(dr) ϕ(r + u) du,

 A(dr)

0

−h

2

|ϕ(θ)| dθ

1/2

h ≤ s ≤ 2h,

√  ≤1+ h

0

−h

A(dr).

It follows from Remark (page 239) that there exists a constant C1 > 0 such that sup |y(s)| ≤ C1 sup |φ(s)|. 0≤s≤2h

0≤s≤2h

One can easily check, using again Lemma 14.4, that for a constant C2 : sup |y(s)| ≤ C2 .

0≤s≤2h

For h ≤ s ≤ 2h  h  Ky(s) = 0

0

−u

 s   A(dr) y(r + u) du + h

Then Ky is an arbitrary continuous function,

0

−h

 A(dr) y(r + u) du.

246

14 Linear control systems

d Ky(s) = ds



0

−h

A(dr) y(r + s),

for s ∈ [h, 2h],

 0  d   A(dr). sup  Ky(s) ≤ C2 h≤s≤2h ds −h

(14.100)

Since the set K of all functions y corresponding to x, ϕ satisfying (14.99) is bounded in C([h, 2h], Rn ) and (14.100) holds then by the Ascoli Theorem A.8 the set K is relatively compact in C([h, 2h], Rn ) and therefore in L2 ([h, 2h], Rn ). Consequently   0   x S2h |ϕ(θ)|2 dθ ≤ 1 ; |x| ≤ 1, ϕ −h is relatively compact in H.



§14.11. State space representation of delay systems We go back to the control system with delay:  0 m  y(t) ˙ = A(dr) y(t + r) + bk uk (t) −h

k=1

y(0) = x ∈ R , n

(14.101)

y(θ) = ϕ(θ), −h ≤ θ ≤ 0,

and show that it can be treated as a linear control system in the space H defined by (14.77). Let (St ) be the semigroup of operators with the generator A and let bk , k = 1, 2, . . . , m be given by the formula  bk bk = ∈ H, k = 1, 2, . . . , m. 0 Theorem 14.14. Let z(t), t ≥ 0, be the weak solution in H of the equation z(t) ˙ = Az(t) +

m  k=1

 x z(0) = ∈ H, ϕ

bk uk (t) (14.102)

where uk (t), t ≥ 0, k = 1, 2, . . . , m, are locally integrable functions. Then  y(t) z(t) = , t ≥ 0, y(t + ·) where y is a solution to (14.101).

14.11 State space representation of delay systems

247

Proof. The weak solution to (14.102) is of the form   m  t x z(t) = St St−s bk uk (s) ds, + ϕ 0

t ≥ 0.

(14.103)

k=1

If v(t), t ≥ 0, is the Rn -coordinate of z(t), then it follows from (14.103) and the definition (14.86) of the semigroup (St ) that  v(t) z(t) = , t ≥ 0. v(t + ·) Moreover v(t) = w(t) +

m  

t

0

k=1

wk (t − s)uk (s) ds,

(14.104)

where w(t) and wk (t), t ≥ 0, k = 1, 2, . . . , m, are solutions of the following problems on [0, +∞):  w(t) ˙ = w˙ k (t) =

0

−h  0 −h

w(0) = x, w(θ) = ϕ(θ), θ ∈ [−h, 0],

A(dr) w(t + r),

wk (0) = bk , wk (θ) = 0, θ ∈ [−h, 0].

A(dr) wk (t + r),

The functions w, wk , k = 1, 2, . . . , m, are absolutely continuous, therefore  m   d t k w (t − s)uk (s) ds dt 0 k=1 m m  t   k w (0)uk (t) + w˙ k (t − s)uk (s) ds = w(t) ˙ +

v(t) ˙ = w(t) ˙ +

 =

k=1 0

−h

+  =

A(dr) w(t + r) + m  t   k=1

0

+

0

−h

0

A(dr) −h

bk uk (t)

 A(dr) wk (t − s + r) uk (s) ds

A(dr) w(t + r) + 

0

k=1

0

−h

k=1 m 

m   k=1

m 

k=1 t+r

0

Taking into account (14.104) we see that

bk uk (t)

wk (t + r − s)uk (s) ds.

248

14 Linear control systems

 v(t) ˙ =

0

−h

A(dr) v(t + r) +

m 

bk uk (t),

t ≥ 0,

k=1

since v(0) = x, v(θ) = ϕ(θ), θ ∈ [−h, 0[, we see that v(t) = y(t), t ≥ 0, as required. 2 A consequence of the theorem to the linear regulator problem will be discussed in §17.4.

Bibliographical notes Our presentation of the material on semigroups was often based on elegant sources like A. Pazy [76] and J. Kisynski [57]. The general result on weak solutions is due to J. Ball [5]. The semigroup approach to control theory of infinite-dimensional systems is the subject of the monographs by R.F. Curtain and A.J. Pritchard [22], R.F. Curtain and H. Zwart [23], see also the author’s paper of survey character [109]. This approach was initiated by A.V. Balakrishnan [4] and M.O. Fattorini [33]. The semigroup approach to delay systems, in the product space H, was initiated by M.C. Delfour and S.K. Mitter in [27] and developed by many authors, see, e.g. R.F. Curtain and H. Zwart [23] and references therein. Important contributions are due to A. Manitius and R. Triggiani, see, e.g. [68]. In the author’s paper J. Zabczyk [111] some delay equations were treated using the concept of decomposable generators.

Chapter 15

Controllability

This chapter is devoted to the controllability of linear systems. The analysis is based on characterizations of images of linear operators in terms of their adjoint operators. The abstract results lead to specific descriptions of approximately controllable and exactly controllable systems which are applicable to parabolic and hyperbolic equations. Formulae for controls which transfer one state to another are given as well.

§15.1. Images and kernels of linear operators Assume that U is a separable Hilbert space. For arbitrary numbers b > a ≥ 0 denote by L2 (a, b; U ) the space of functions, more precisely, equivalence classes of functions, u(·) : [a, b] → U , Bochner integrable (see §A.5 and §A.7), on [a, b] and such that 

b

|u(s)|2 ds < +∞.

a

It is also a Hilbert space with the scalar product  u(·), v(·) =

b

u(s), v(s) ds,

u(·), v(·) ∈ L2 (a, b; U ),

a

where ·, · is the scalar product in U . Norms in U and in L2 (a, b; H) will be denoted by | · | and  · , respectively. As in Part I, in the study of controllability of infinite-dimensional systems y˙ = Ay + Bu,

y(0) = x ∈ E,

(15.1)

an important role will be played by the operator © Springer Nature Switzerland AG 2020

J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_15

249

250

15 Controllability

 LT u =

0

T

u(·) ∈ L2 (0, T ; U )

S(T − s)Bu(s) ds,

(15.2)

(see §14.7), acting from UT = L2 (0, T ; U ) into E. Note that y(T ) = S(T )x + LT u,

u(·) ∈ L2 (0, T ; U ),

where y(·) is the weak solution to (15.1). To analyse the operator LT we will need several results from operator theory which we discuss now. Let X, Y and Z be Banach spaces and F and G linear, bounded operators from X into Z and Y into Z. The adjoint spaces and adjoint operators will be denoted, as before, by X ∗ , Y ∗ and Z ∗ and by F ∗ and G∗ , respectively. The image F (X) and the kernel F −1 {0} of the transformation F will be denoted by Im F and Ker F . Our main aim in this section is to give characterizations of the inclusions Im F ⊂ Im G,

Im F ⊂ Im G

in terms of the adjoint operators F ∗ and G∗ . Theorem 15.1. The following two conditions are equivalent: Im F ⊂ Im G, ∗

∗

Ker F ⊃ Ker G .

(15.3) (15.4)

Proof. Assume that (15.3) holds and that for some f ∈ Z ∗ , G∗ f = 0 and F ∗ f = 0. Then for arbitrary y ∈ Y , f (G(y)) = 0 and f = 0 on Im G. It follows from (15.3) that f = 0 on Im F and f (F (x)) = 0 for x ∈ X. Hence F ∗ f = 0, a contradiction. This way we have shown that (15.3) implies (15.4). Assume now that (15.4) holds and that there exists z ∈ Im F \ Im G. There exists a functional f ∈ Z ∗ such that f (z) = 0 and f = 0 on Im G. Moreover, for a sequence of elements xm ∈ X, m = 1, 2, . . . , F (xm ) −→ z. For sufficiently large m, F ∗ f (xm ) = 0. Therefore F ∗ f = 0, and, at the same 2 time, G∗ f = 0, which contradicts (15.4). Under rather general conditions, the inclusion Im F ⊂ Im G

(15.5)

takes place if there exists c > 0 such that F ∗ f  ≤ cG∗ f ||

for all f ∈ Z ∗ .

(15.6)

We will prove first Lemma 15.1. Inclusion (15.5) holds if and only if for some c > 0 {F (x); x ≤ 1} ⊂ {Gy; y ≤ c}.

(15.7)

15.1 Images and kernels of linear operators

251

Moreover (15.6) holds if and only if {F (x); x ≤ 1} ⊂ {Gy; y ≤ c}.

(15.8)

Proof. Assume that (15.5) holds, and, in addition, Ker G = {0}. Then the operator G−1 F is well defined, closed and, by Theorem A.2 (the closed graph theorem), continuous. Hence there exists a constant c > 0 such that G−1 (F (x)) ≤ c,

provided x ≤ 1,

ˆ and (15.7) holds. If Ker G = {0}, consider the induced transformation G ˆ from the quotient space Y = Y /Ker G into X. We recall that the norm of the equivalence class [y] ∈ Yˆ of an element y ∈ Y is given by [y] = inf{y + y˜||; G(˜ y ) = 0}. ˆ inclusion (15.5) implies Im F ⊂ Im G, ˆ and, by the proven Since Im G = Im G, part of the lemma, ˆ {F (x); x ≤ 1} ⊂ {G([y]); [y] ≤ c} ⊂ {G(y + y˜); y + y˜ ≤ c + 1}. Hence (15.5) implies (15.7) for arbitraryG. It is obvious that (15.7) implies (15.5). To prove the second part assume that (15.8) holds. Then for arbitrary f ∈ Z∗ F ∗ f  = sup |f (F (x))| ≤ c sup |f (G(y))| ≤ cG∗ f , x||≤1

y≤1

and (15.6) is true. Finally assume that (15.6) holds and for some x0 ∈ X, x0  ≤ 1, / {Gy; y ≤ c}. By Theorem III.12.4(ii) there exists f ∈ Z ∗ for F (x0 ) ∈ which f (F (x0 )) > 1 and for arbitrary y ∈ Y, y ≤ 1, c|f (G(y))| ≤ 1. Hence at the same time F ∗ f  > 1 and cG∗ f  ≤ 1. The obtained contradiction with the condition (15.6) completes the proof of the lemma. 2 Corollary 15.1. If the set {Gy; y ≤ c} is closed, then inclusion (15.5) holds if and only if there exists c > 0 such that (15.6) holds. In particular we have the following theorem. Theorem 15.2. If Y is a separable Hilbert space then inclusion (15.5) holds if and only if, for some c > 0, (15.6) holds. Proof. Assume that Y is a Hilbert space and z ∈ {Gy; y ≤ c}. Then there exists a sequence of elements ym ∈ Y , m = 1, 2, . . . , such that G(ym ) −→ z. y  ≤ c, see We can assume that (ym ) converges weakly to y˜ ∈ Y, ˜ Lemma III.3.4. For arbitrary f ∈ Z ∗ , f (G(ym )) −→ f (z) and f (G(ym )) = G∗ f (ym ) −→ G∗ f (˜ y ), hence f (z) = f (G(˜ y )). Since f ∈ Z ∗ is arbitrary,

252

15 Controllability

z = G(˜ y ) ∈ {G(y); y ≤ c} and the set {G(y); ||y ≤ c} is closed. Corollary 15.1 implies the result. 2 Remark. It follows from the proof that Theorem 15.2 is valid for Banach spaces such that an arbitrary bounded sequence has a weakly convergent subsequence. As was shown by Banach, this property characterizes the reflexive spaces in which, by the very definition, an arbitrary linear, bounded functional f on Y ∗ is of the form f −→ f (y), for some y ∈ Y . Example 15.1. Let Y = C[0, 1], X = Z = L2 [0, 1] and let F x = x, Gy = y, x ∈ X, y ∈ Y . Then condition (15.8) is satisfied but not (15.7). So, without additional assumptions on the space Y , Theorem 15.2 is not true. We will often meet, in what follows, inverses of operators which are not one-to-one and onto. For simplicity, assume that X and Z are Hilbert spaces and F : X −→ Z is a linear continuous map. Then X0 = Ker F is a closed subspace of X. Denote by X1 the orthogonal complement of X0 : X1 = {x ∈ X; x, y = 0 for all y ∈ X0 }. The subspace X1 is also closed and the restriction F1 of F to X1 is one-to-one. Moreover Im F1 = Im F. We define

F −1 (z) = F1−1 (z),

z ∈ Im F.

−1

The operator F : Im F −→ Y defined this way is called the pseudoinverse of F . It is linear and closed but in general noncontinuous. Note that for arbitrary z ∈ Im F inf{x; F (x) = z} = F −1 (z).

§15.2. The controllability operator We start our study of the controllability of (15.1) by extending Proposition I.1.1 and give an explicit formula for a control transferring state a ∈ E to b ∈ E. Generalizing the definition of the controllability matrix we introduce the controllability operator by the formula  QT x =

0

T

S(r)BB ∗ S ∗ (r)x dr,

x ∈ E.

(15.9)

It follows from Theorems 14.1 and 14.3 that for arbitrary x ∈ E the function S(r)BB ∗ S ∗ (r)x, r ∈ [0, T ], is continuous, and the Bochner integral in (15.9) is well defined. Moreover, for a constant c > 0,

15.2 The controllability operator

 0

T

253

|S(r)BB ∗ S ∗ (r)x| dr ≤ c|x|,

x ∈ E.

Hence the operator QT is linear and continuous. It is also self-adjoint and nonnegative definite:  QT x, x =

T

0

|B ∗ S ∗ (r)x|2 dr ≥ 0,

x ∈ E.

(15.10)

1/2

We denote in what follows by QT the unique self-adjoint and nonnegative operator whose square is equal QT . There exists exactly one such operator. 1/2 −1 Well defined are operators Q−1 (see §15.1); the latter will also T and (QT ) −1/2 be denoted by QT . In connection with these definitions we propose to solve the following. Exercise 15.1. Let (em ) be an orthonormal, not necessarily complete, sequence in a Hilbert space E and (γm ) a bounded sequence of positive numbers. Define +∞  Qx = γm x, em em , x ∈ E. m=1

Find formulae for Q1/2 , Q−1 , (Q1/2 )−1 . Theorem 15.3. (i) There exists a strategy u(·) ∈ UT transferring a ∈ E to b ∈ E in time T if and only if 1/2

S(T )a − b ∈ Im QT . (ii) Among the strategies transferring a to b in time T there exists exactly T one strategy u ˆ which minimizes the functional JT (u) = 0 |u(s)|2 ds. Moreover, u) = |Q−1/2 (S(T )a − b)|2 . JT (ˆ T (iii) If S(T )a − b ∈ Im QT , then the strategy u ˆ is given by u ˆ(t) = −B ∗ S ∗ (T − t)Q−1 T (S(T )a − b),

t ∈ [0, T ].

Proof. (i) Let us remark that a control u ∈ UT transfers a to b in time T if and only if b = S(T )a + LT (u). It is therefore sufficient to prove that b − S(T )a ∈ Im LT , Let x ∈ E and u(·) ∈ UT . Then

1/2

Im LT = Im QT .

(15.11)

254

15 Controllability

u(·), L∗T x = LT u, x =



T

0

 =

T

0

S(T − r)Bu(r), x dr u(r), B ∗ S ∗ (T − r)x dr,

and, by the very definition of the scalar product in UT , L∗T x(r) = B ∗ S ∗ (T − r)x, for almost all r ∈ (0, T ).

(15.12)

Since L∗T x2

 =

T

0

|B ∗ S ∗ (T − r)x|2 dr 1/2

= QT x, x = |QT x|2 ,

x ∈ E,

part (i) follows immediately from Theorem 15.2. (ii) The unique optimal strategy is of the form L−1 T (a−S(T )b). The formula for the minimal cost is an immediate consequence of the following result. Proposition 15.1. If F and G are linear bounded operators acting between separable Hilbert spaces X, Z and Y, Z such that F ∗ f  = G∗ f  for f ∈ Z ∗ , then Im F = Im G and F −1 z = |G−1 z| for z ∈ Im F . Proof. The identity Im F = Im G follows from Theorem 15.2. To show the latter identity, one can assume that operators are injective as otherwise one could consider their restrictions to the orthogonal complements of their kernels. Assume that for some z ∈ Z, z = 0, z = F x1 = Gy1 , x1 ∈ X, y1 ∈ Y and x1  > y1 ||. Then yz1  = G( yy11  ) ∈ {Gy; ||y ≤ 1}. By Lemma 15.1 {Gy; y ≤ 1} = {F x; x|| ≤ 1}. Consequently, yz1  ∈ {F x; x ≤ 1}.   / But ||yz1  = F ( yx11|| ). Since  yx11||  > 1 and F is injective, F ( yx11 ) ∈ {F (x); x ≤ 1}. A contradiction. Thus x1 || = y1 . To prove (iii), let us remark that for arbitrary x ∈ E, LT L∗T x = QT x.

(15.13)

Let us fix y ∈ Im QT and let u ˆ = L−1 T y,

−1/2

z = QT

−1/2

y,

QT

z = Q−1 T y.

It follows from (15.13) that −1/2

LT L∗T QT

−1/2

z = QT

z = y.

If LT u = 0, u ∈ UT , then −1/2

L∗T QT

−1/2

z, u = QT

z, LT U  = 0.

(15.14)

15.3 Various concepts of controllability

255

Taking into account the definition of L−1 T and identity (15.14), we obtain −1/2

u ˆ = L∗T QT

(15.15)

z,

and, by (15.12), u ˆ(t) = B ∗ S ∗ (T − t)Q−1 T y,

t ∈ [0, T ].

Defining y = b − S(T )a, we arrive at (ii).

2

§15.3. Various concepts of controllability The results of the two preceding sections lead to important analytical characterizations of various concepts of controllability. Let RT (a) be the set of all states attainable from a in time T ≥ 0. We have the obvious relation RT (a) = S(T )a + Im LT . We say that system (15.1) is exactly controllable from a in time T if RT (a) = E. If RT (a) = E, then we say that system (15.1) is approximately controllable from a in time T . We say that system (15.1) is null controllable in time T if an arbitrary state can be transferred to 0 in time T or, equivalently, if and only if Im S(T ) ⊂ Im LT .

(15.16)

We have the following characterizations. Theorem 15.4. The following conditions are equivalent: (i) System (15.1) is exactly controllable from an arbitrary state in time T > 0. (ii) There exists c > 0 such that for arbitrary x ∈ E  T |B ∗ S ∗ (t)x|2 dt ≥ c|x|2 . (15.17) 0

(iii)

1/2

Im QT

= E.

(15.18)

Proof. If a system is exactly controllable from an arbitrary state, it is controllable in particular from 0. However, the exact controllability from 0 in time T is equivalent to

256

15 Controllability

Im LT ⊃ E. Applying Theorem 15.2 to G = LT and F = I, we deduce that condition (15.17) is equivalent to the exact controllability of (15.1) from 0. If, however, Im LT = E, then, for arbitrary a ∈ E, RT (a) = E, and (15.1) is exactly controllable from arbitrary state. The final part of the theorem follows from Theorem 15.3(i). 2 Theorem 15.5. The following conditions are equivalent: (i) System (15.1) is approximately controllable in time T > 0 from an arbitrary state. (15.19) (ii) If B ∗ S ∗ (r)x = 0 for almost all r ∈ [0, T ], then x = 0. 1/2

(iii)

Im QT is dense in E.

(15.20)

Proof. It is clear that the set RT (a) is dense in E if and only if the set Im LT is dense in E. Hence (i) is equivalent to (ii) by Theorem 15.1. Moreover (i) is equivalent to (iii) by Theorem 15.3. 2 Theorem 15.6. The following conditions are equivalent: (i) System (15.1) is null controllable in time T > 0. (ii) There exists c > 0 such that for all x ∈ E  T |B ∗ S ∗ (r)x|2 dr ≥ c|S ∗ (T )x|2 .

(15.21)

0

(iii)

1/2

Im QT

⊃ Im S(T ).

(15.22)

Proof. Since null controllability is equivalent to (15.16), characterizations and (ii) and characterizations (i) and (iii) are equivalent by Theorem 15.2 and Theorem 15.3, respectively. 2

§15.4. Systems with self-adjoint generators Let us consider system (15.1) in which the generator A is a self-adjoint operator on a Hilbert space H, such that for an orthonormal and complete basis (em ) and for a decreasing to −∞ sequence (λm ), Aem = λm em , m = 1, 2, . . . . (See Example 14.2 with E = H, pages 205 and 224). We will consider two cases, B as the identity operator and B as onedimensional. In the former case, we have the system y˙ = Ay + u,

y(0) = a,

(15.23)

with the set U of control parameters identical to H. In the latter case, y˙ = Ay + uh,

y(0) = a,

(15.24)

15.4 Systems with self-adjoint generators

257

U = R1 and h is a fixed element in H. Lemma 15.2. Sets Im LT , for system (15.23), are identical for all T > 0. Moreover, state b is reachable from 0 in a time T > 0 if and only if +∞ 

|λm | |b, em |2 < +∞.

(15.25)

m=1

Proof. In the present situation the operator QT is of the form  T +∞  T   QT x = S(2t)x dt = e2λm t dt x, em em 0

=

0 m=1  +∞  2λm T

e

m=1

Therefore 1/2 QT x

−1



2λm

x, em em .

1/2 +∞  2λm T  |e − 1| = x, em em . 2|λm | m=1

Let us remark that for arbitrary T > 0 |e2λm T − 1| −→ 1,

as m ↑ +∞.

+∞ 1/2 Hence b = m=1 b, em em ∈ Im QT if and only if condition (15.25) holds. 1/2 Since Im LT = Im QT the result follows. 2 Lemma 15.3. Under the conditions of Lemma 15.2, for arbitrary T > 0, 1/2

Im S(T ) ⊂ Im QT .

(15.26)

Proof. If a ∈ H, then S(T )a =

+∞ 

eλm T a, em em .

m=1

Since

 m

 |λm |e2λm T |a, em |2 ≤ sup |λm |e2λm T |a|2 < +∞, m

(15.26) holds.

2

As an immediate corollary of Lemma 15.2, Lemma 15.3 and results of §15.3, we obtain the following theorem. Theorem 15.7. System (15.23) has the following properties: (i) It is not exactly controllable from any state and at any moment T > 0. (ii) It is approximately controllable from arbitrary a and in arbitrary T > 0. (iii) The set RT (a) is characterized by (15.25).

258

15 Controllability

We proceed now to system (15.24). It follows from Theorem 15.6 that (15.24) is not exactly controllable. We will now formulate a result giving a characterization of the set of all attainable points showing at the same time the applicability of theorems from §15.1. An explicit and complete characterization of the set is not known (see [86]). First of all we have the following. Theorem 15.8. System (15.24) is approximately controllable from an arbitrary state at a time T > 0 if and only if λn = λm for n = m, h, em  = 0 for m = 1, 2, . . . .

(15.27) (15.28)

Proof. We apply Theorem 15.5. For arbitrary x ∈ H B ∗ S ∗ (t)x =

+∞ 

eλm t h, em x, em .

m=1

Since

+∞ 

|h, em x, em | ≤ |hx| < +∞

m=1

and λm −→ −∞ as m ↑ +∞, the function ϕ(t) =

+∞ 

eλm t h, em x, em ,

t > 0,

m=1

is well defined and analytic. Hence ϕ(t) = 0 for almost all t ∈ [0, T ] if and only if ϕ(t) = 0 for all t ≥ 0. It is clear that conditions (15.27) and (15.28) are necessary for the approximate controllability of (15.24). To prove that they also are sufficient we can assume, without any loss of generality, that the sequence (λm ) is decreasing. Then lim e−λ1 t ϕ(t) = h, e1 x, e1  = 0.

t↑+∞

Since h, e1  = 0, x, e1  = 0. Moreover, ϕ(t) =

+∞ 

eλm t h, em x, em ,

t > 0.

m=2

In a similar way lim e−λ2 t ϕ(t) = h, e2 x, e2  = 0,

t↑+∞

and thus x, e2  = 0. By induction x, em  = 0 for all m = 1, 2, . . . , and, since the basis (em ) is complete, x = 0. This completes the proof of the theorem.2

15.4 Systems with self-adjoint generators

259

Let (αm ) be a bounded sequence of positive numbers. Let H0 be a subspace of H given by

 +∞  x, em 2 H0 = x ∈ H; < +∞ . (15.29) 2 αm m=1 We say that the space H0 is reachable at a moment T > 0 if 0 can be transferred to an arbitrary element of H0 in time T by the proper choice of a control from L2 (0, T ). It turns out that one can give necessary conditions, which are very close to sufficient conditions, for the space H0 to be reachable at time T > 0. To formulate them consider functions fm (t) = eλm t , t ∈ [0, T ], and closed subspaces Zm of Z = L2 (0, T ), m = 1, 2, . . . , generated by all fk , k = m. Let δm be the distance, in the sense of the space Z, from fm to Zm , m = 1, 2, . . . . Theorem 15.9. (i) If H0 is reachable at time T , then there exists γ > 0 such that 0 < αm ≤ γδm |h, em |, m = 1, 2, . . . . (ii) If, for some γ > 0, 0 < αm ≤

γ δm |h, em |, m

m = 1, 2, . . . ,

then H0 is reachable at time T . Proof. Let F : H −→ H be an operator given by Fx =

+∞ 

αm x, em em ,

x ∈ H.

m=1

Then

Im F = H0 .

Hence the space H0 is reachable at T > 0 if and only if Im F ⊂ Im LT .

(15.30)

It follows from Theorem 15.2 that (15.30) holds if and only if for a number γ>0  T 1 |S(t)h, x|2 dt ≥ 2 |F x|2 , x ∈ H. (15.31) γ 0 Denoting the norm in Z by  ·  we can reformulate (15.31) equivalently as k k   1  βj   fj ξj  ≥ provided ξj2 = 1,  α γ j j=1 j=1

k = 1, 2, . . . .

(15.32)

Let us fix m and ε > 0 choose k ≥ m and numbers ηj , j ≤ k, j = m such that

260

15 Controllability k    βj αm   δm + ε ≥ fm + fj ηj . αj βm

(15.33)

j=m

Defining ηm = 1, δ =

k 

j=1

ηj2

1/2

≥ 1, we obtain from (15.32) and (15.33)

k k    1 |βm | 1 βj βj ηj      (δm + ε) ≥  fj ηj  ≥ δ  fj  ≥ δ ≥ . αm α α δ γ γ j=1 j j=1 j

Since ε > 0 was arbitrary, (i) follows. To prove (ii) denote by fˆm the orthogonal projection of fm onto Zm and let 1 f˜m = 2 (fm − fˆm ), m = 1, 2, . . . . δm Let us remark that 1 fm , f˜m  = 2 fm , fm − fˆm  = 1, δm 1 f˜m  = , k = m, δm

fm , f˜k  = 0,

(15.34)

where ·, · denotes the scalar product on Z. The sequence (f˜m ) with the property (15.34) is called biorthogonal to (fm ). Therefore, if, for a sequence (ξm ), +∞  |ξm | 1 < +∞, |βm | δm m=1

then u(·) = and

 0

T

+∞  ξm ˜ fm (·) ∈ Z β m=1 m

eλm t u(t) dt = fm , u =

ξm , βm

m = 1, 2, . . . .

The idea to construct u as a solution of the above sequence of equations is called the moment method (see, e.g. the book by S.A. Avdonin and S.A. Ivanov [3]). Consequently, +∞  |x, em | < +∞ |βm |δm m=1

implies x ∈ Im LT . On the other hand,

15.5 Controllability of the wave equation

261

+∞ +∞  |x, em |  1 m = |x, em | |β |δ |β |δ m m m m m m=1 m=1

+∞ +∞  |x, em |2 m2 1/2   1 1/2 ≤ . 2 δ2 βm m2 m m=1 m=1

Hence, if 1 |βm |δm , m = 1, 2, . . . , m then H0 ⊂ Im LT . The proof of the theorem is complete. 0 < αm ≤

2

§15.5. Controllability of the wave equation As another application of the abstract results, we discuss the approximate controllability of the wave equation, see Example 14.4. Let h be a fixed function from L2 (0, π) with the Fourier expansion +∞ 

h(ξ) =

γm sin mξ,

ξ ∈ (0, π),

m=1

+∞ 

2 γm < +∞.

m=1

Let us consider the equation ∂2y ∂2y = 2 + hu(t), 2 ∂t ∂ξ

t ≥ 0, ξ ∈ (0, π),

(15.35)

with initial and boundary conditions as in Example 14.4 and with real-valued u(·). Equation (15.35) is equivalent to the system of equations ∂y = v, ∂t ∂v ∂2y = 2 + hu(t), ∂t ∂ξ

(15.36) t ≥ 0, ξ ∈ (0, π).

(15.37)

Let E be the Hilbert space and S(t), t ∈ R, the group of the unitary operators introduced in Example 14.4. In accordance with the considerations of §14.7, the solution to (15.36)–(15.37) with the initial conditions u(0) = a, v(0) = b will be identified with the function Y (t), t ≥ 0, given by        t 0 y(t) a S(t − s) u(s). (15.38) Y (t) = = S(t) + h v(t) b 0 Theorem 15.10. If T ≥ 2π,

γm = 0

for m = 1, 2, . . . ,

262

15 Controllability

then system (15.36)–(15.37) is approximately controllable in time T from an arbitrary state. Proof. In the present situation U = R and the operator B : R −→ E is given by   0 Bu = u, u ∈ R. h Since S ∗ (t) = S(−t), t ≥ 0,       a 0 a ∗ ∗ B S (t) = , S(−t) b h b =

+∞ 

γm (mαm sin mt + βm cos mt),

t ≥ 0.

m=1

Taking into account that +∞ 

|mγm αm | < +∞,

m=1

+∞ 

|γm βm | < +∞,

m=1

we see that the formula ϕ(t) =

+∞ 

γm (mαm sin mt + βm cos mt),

t ≥ 0,

m=1

defines a continuous, periodic function with the period 2π. Moreover,  1 2π mγm αm = ϕ(t) cos mt dt, π 0  1 2π γm βm = ϕ(t) sin mt dt, m = 1, 2, . . . . π 0 Hence if T ≥ 2π and ϕ(t) = 0 for t ∈ [0, T ], mγm αm = 0 and βm γm = 0, m = 1, 2, . . . . Since γm = 0 for all m = 1, 2, . . . , αm = βm = 0, m = 1, 2, . . . , and we deduce that a = 0, b = 0. By Theorem 15.6 the theorem follows. 2

Bibliographical notes Theorems from §15.1 are taken from the paper by S. Dolecki and D.L. Russell [28] as well as from the author’s paper J. Zabczyk [109]. They are generalizations of a classical result due to R. Douglas [29].

15.5 Controllability of the wave equation

263

An explicit description of reachable spaces in the case of finite-dimensional control operators B has not yet been obtained; see an extensive discussion of the problem in the survey paper by D.L. Russell [88] and in H.O. Fattorini and D.L. Russell [36]. Asymptotic properties of the sequences (fm ) and (δm ) were an object of intensive studies, see D.L. Russell [88] and S.A. Avdonin and S.A. Ivanov [3]. Theorem 15.9 is from author’s paper J. Zabczyk [109]. Part (i) is in the spirit of D.L. Russell [88]. For more recent results we refer to J.-M. Coron [20] and I. Lasiecka and R. Triggiani [62].

Chapter 16

Stability and stabilizability

We will show first that the asymptotic stability of an infinite-dimensional linear system does not imply its exponential stability and is not determined by the spectrum of the generator. Spectral stability conditions for delay systems are derived. Then stable systems are characterized by their corresponding Lyapunov equations. It is also proved that null controllability implies stabilizability and that under additional conditions a converse implication takes place. Stability of hyperbolic systems is examined as well.

§16.1. Various concepts of stability Characterizations of stable or stabilizable infinite-dimensional systems are much more complicated then the finite-dimensional one. We will restrict our discussion to some typical results underlying specific features of the infinite-dimensional situation. Let A be the infinitesimal generator of a semigroup S(t), t ≥ 0, on a Banach space E. If x ∈ D(A) then a strong solution, see §14.7, of the equation z˙ = Az, is the function

z(0) = x ∈ E,

z x (t) = S(t)x,

(16.1)

t ≥ 0.

For arbitrary x ∈ E, z x (·) is the limit of strong solutions to (16.1) and is also called a weak solution to (16.1), see §14.7. If the space E is finite-dimensional then, by Theorem I.2.3, the following conditions are equivalent:

© Springer Nature Switzerland AG 2020

J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_16

265

266

16 Stability and stabilizability

For some N > 0, ν > 0 and all t ≥ 0 S(t) ≤ N e−νt . x

(16.2)

For arbitrary x ∈ E, z (t) −→ 0 exponentially as t → +∞.  +∞ For arbitrary x ∈ E, z x (t)2 dt < +∞.

(16.4)

For arbitrary x ∈ E, z (t) −→ 0 as t → +∞. sup{Re λ; λ ∈ σ(A)} < 0.

(16.5) (16.6)

0 x

(16.3)

In general, in the infinite-dimensional situation the above conditions are not equivalent, as the theorem below shows. Let us recall that for linear operators A on an infinite-dimensional space E, λ ∈ σ(A) if and only if for some z ∈ E the equation λx − Ax = z either does not have a solution x ∈ D(A) or has more than one solution. Theorem 16.1. Let E be an infinite-dimensional Banach space. Then (i) Conditions (16.2), (16.3) and (16.4) are all equivalent. (ii) Conditions (16.2), (16.3) and (16.4) are essentially stronger than (16.5) and (16.6). (iii) Condition (16.5) does not imply, in general, condition (16.6), even if E is a Hilbert space. (iv) Condition (16.6) does not imply, in general, condition (16.5), even if E is a Hilbert space. Proof. (i) It is obvious that (16.2) implies (16.3) and (16.4). Assume that (16.4) holds. Then the transformation x −→ S(·)x from E into L2 (0, +∞; E) is everywhere defined and closed. Therefore, by Theorem A.2 (the closed graph theorem), it is continuous. Consequently, there exists a constant K > 0 such that  +∞ S(t)x2 dt < Kx2 , for all x ∈ E. 0

Moreover, by Theorem 14.1, for some M ≥ 1 and ω > 0, S(t) ≤ M eωt for all t ≥ 0. For arbitrary t > 0, x ∈ E, 1 − e−2ωt S(t)x2 = 2ω

 0



t

e−2ωr S(t)x2 dr

t

e−2ωr S(r)2 S(t − r)x2 dr  t 2 S(s)x2 ds ≤ M 2 Kx2 . ≤M ≤

0

0

Therefore, for a number L > 0,

16.1 Various concepts of stability

267

S(t) ≤ L,

t ≥ 0.

Consequently, tS(t)x2 =



t

0 2

S(t)x2 ds ≤

≤ L Kx2 ,



t

0

S(s)2 S(t − s)X2 ds

t ≥ 0, x ∈ E, 

and

S(t) ≤ L

k , t

t > 0.

Hence, there exists t¯ > 0 such that S(t¯) < 1. For arbitrary t ≥ 0 there exist m = 0, 1, . . . and s ∈ [0, t¯) such that t = mt¯+s. Therefore ¯ ¯ S(t) ≤ S(t¯)m M eωs ≤ S(t¯)t/t M S(t¯)−1 eωt ,

and (16.2) follows. (ii) It is clear that (16.2) implies (16.5). Moreover, if (16.2) holds and Re λ > −ν then  +∞ R(λ) = (λI − A)−1 = e−λt S(t) dt, 0

see Proposition 14.2. Consequently sup{Re λ; λ ∈ σ(A)} ≤ −ν < 0, and (16.6) holds as well. To see that (16.5) does not imply, in general, (16.2), we define in E = l2 a semigroup S(t)x = (e−γm t ξm ), x = (ξm ) ∈ l2 , where (γm ) is a sequence monotonically decreasing to zero. Then 2

S(t)x =

+∞ 

e−2γm t |ξm |2 −→ 0,

as t ↑ +∞.

m=1

However

     S 1  ≥ e−1 ,  γm 

m = 1, 2, . . . ,

and (16.2) is not satisfied. An example showing that (16.6) does not imply (16.5) and (16.2) will be constructed in the proof of (iv).

268

16 Stability and stabilizability

(iii) Note that the semigroup S(t), t ≥ 0, constructed in the proof of (ii) satisfies (16.5). Since −γm ∈ σ(A), m = 1, 2, . . . , sup{Re λ; λ ∈ σ(A)} ≥ 0 and (16.6) does not hold. (iv) We will prove first a proposition. Proposition 16.1. There exists a semigroup S(t), t ≥ 0, on E, the complex Hilbert space lC2 such that S(t) = et , t ≥ 0 and σ(A) = {iλm ; m = 1, 2, . . . } where (λm ) is an arbitrary sequence of real numbers such that |λm | −→ +∞. Proof. Let us represent an arbitrary element x ∈ lC2 , regarded as an infinite column, in the form x = (xm ), where xm ∈ Cm . The required semigroup S(t), t ≥ 0, is given by the formula S(t)x = (eiλm t eAm t xm ),

(16.7)

where matrices Am = (am i,j ) ∈ M(n, n) are such that  1 for j = i + 1, i = 1, 2, . . . , m − 1, am ij = 0 for the remaining i, j. It is easy to show that lim S(t)x = x for all x ∈ E, and we leave this as an t↓0

exercise. To calculate S(t) remark that S(t) = sup eiλm t eAm t  ≤ sup eAm t  ≤ sup etAm  ≤ et , m

m

m

t ≥ 0,

because Am  = 1, m = 1, 2, . . . . On the other hand, for the vector em ∈ Cm with all elements equal to √1m , Am t

e

 em  =

  2 2 1/2 1 2 t tm−1 + ... + 1 + ... + 1 + 1+ m 1! (m − 1)! −→ et ,

as m ↑ +∞.

Hence S(t) = et . For arbitrary λ ∈ / {iλm ; m = 1, 2, . . . } let W (λ) be the following operator: W (λ)x = ((λI − (iλm + Am ))−1 xm ). An elementary calculation shows that W (λ) is a bounded operator such that +∞   D = Im W (λ) = (xm ) ∈ l2 ; |λm |2 |xm |2 < +∞ . m=1

16.1 Various concepts of stability

269

Moreover, if D0 is the set of all sequences x = (xm ) such that xm = 0 for a finite number of m, then for x ∈ D0 , A(xm ) = ((λm i + Am )xm ) and W (λ)x = R(λ)x, where R(λ), λ ∈ / σ(A) is the resolvent of S(t), t ≥ 0. Consequently D(A) = D and for λ ∈ / σ(A), R(λ) = W (λ). One checks that for λ ∈ {iλm ; m = 1, 2, . . . }, (λI − A) W (λ) = I on l2 , W (λ)(λI − A) = I on D(A). Hence σ(A) = {iλm ; m = 1, 2, . . . }. The proof is complete. 2 We go back to the proof of (iv). For arbitrary α ∈ R1 and β > 0 define a ˜ new semigroup S(·), ˜ = eαt S(βt), t ≥ 0. S(t) ˜ is given by The generator A˜ of the semigroup S(·) A˜ = αI + βA, Moreover

˜ = D(A). D(A)

˜ ˜ = α. S(t) = e(α+β)t and sup{Re λ; λ ∈ σ(A)}

Taking in particular α = − 12 and β = 1 we have 1

˜ S(t) = e 2 t , t ≥ 0, Since

and

˜ = −1. sup{Re λ; λ ∈ σ(A)} 2

˜ lim S(t) = +∞, there exists, by Theorem A.5(i) (the Banach–

t→+∞

˜ Steinhaus theorem), x ∈ E such that lim S(t)x = +∞. t→+∞

Hence (16.6) does not imply (16.5) in the case of E being the complex Hilbert space lC2 . The case of E being a real Hilbert space follows from the following exercise. 2 Exercise 16.1. Identify the complex Hilbert space lC2 with the Cartesian product l2 × l2 of two real l2 spaces and the semigroup S(t), t ≥ 0, from Proposition 16.1 with its image S(t), t ≥ 0, after the identification. Show that S(t) = et , t ≥ 0 and σ(A) = {±iλn ; n = 1, 2, . . . }, where A is the generator of S(t), t ≥ 0. If a semigroup S(t), t ≥ 0, satisfies one of the conditions (16.2), (16.3), (16.4) then it is called exponentially stable, and its generator an exponentially stable generator.

270

16 Stability and stabilizability

§16.2. Spectrum determined growth assumptions The growth rate ω(S) of a semigroup S(t) is defined as the infimum of those ω for which (16.8) |S(t)| ≤ M eωt holds for some M and all t ≥ 0. Note that if (16.8) holds for sufficiently large t then it does hold for all t ≥ 0 with a larger M . It is not difficult to see that ln |S(t)| . (16.9) ω(S) = inf t>0 t In fact let ω0 = inf t>0

ln |S(t)| . t

If (16.8) holds then

ln |S(t)| ln M ≤ +ω t t and thus ω0 ≤ ω. Consequently ω0 ≤ ω(S). Assume that ω ¯ ≥ ω0 . Then for a number t¯ > 0 ln |S(t¯)| ω0 and sufficiently large t. This proves (16.9). It is also clear that ln |S(t)| ln |S(t)| = lim . t→+∞ t>0 t t inf

If Re λ > ω and (16.8) holds then  +∞  −λt |e S(t)| dt ≤ M 0

+∞

0

e(− Re λ+ω)t dt < +∞.

It does follow from the proof of the Hille–Yosida theorem that λ ∈ ρ(A) and  +∞ e−λt S(t) dt. (λI − A)−1 = 0

Consequently

16.3 Stability of delay equations

271

sup{Re λ; λ ∈ σ(A)} ≤ ω(S).

(16.10)

If for a semigroup S one has equality in (16.10) then one says that semigroup satisfies the spectrum determination growth assumption. The concept was introduced by R. Triggiani in [93]. The assumption is satisfied for systems in finite-dimensional spaces, see Theorem 2.1 in Part I. R. Triggiani showed in [93] that it does hold if the operator A is bounded or the trajectories S(t)x, t ≥ 0, x ∈ E, are differentiable. J. Zabczyk in [106] proved that the assumption is satisfied, if for some t > 0, the operator S(t) is compact.

§16.3. Stability of delay equations Explicit conditions for stability of delay equations can be obtained by analysing the spectrum of the associated semigroup of operators St which is described in the proposition below. That the spectrum determines the growth rate of the solutions follows from the fact that operators St are compact for sufficiently large t, see Proposition 14.7 and discussion in the previous sections. In the case of the simplest delay equation, see Kalecki’s Example 0.9 in the Introduction: y(t) ˙ = ay(t) + by(t − 1),

y(0) = x, y(θ) = φ(θ),

θ ∈ [−1, 0),

all solutions tend to 0 exponentially if and only if all roots λ of the equation λ − be−λ − a = 0

(16.11)

have negative real parts. The nontrivial analysis of the equation (16.11) was performed in paper N.D. Hayes [45]. Proposition 16.2. If A is the generator of the semigroup St corresponding to the delay equation then  0  

(16.12) σ(A) = λ ∈ C; det λI − A(ds) eλs . −h

  Proof. A complex number λ ∈ ρ(A) if and only if for arbitrary gz ∈ H the resolvent equation       x x z λ −A = (16.13) f f g   has a unique solution fx ∈ D(A). Equation (16.13) is equivalent to two equations:

272

16 Stability and stabilizability

 λf (0) − λf (s) −

0

−h

(16.14)

A(ds) f (s) = z

df (s) = g(s), ds

s ∈ [−h, 0].

The solution to (16.15) is of the form  0 f (s) = eλs f (0) + eλs e−λu g(u) du, s

(16.15)

−h ≤ s ≤ 0,

and equation (16.14) becomes an equation on f (0):  0  0

 λf (0) − A(ds) eλs f (0) + eλs e−λu g(u) du = z

(16.16)

or equivalently  0 

 λs λI − A(ds)e f (0) = z +

(16.17)

−h

s

−h

0

−h

 A(ds)

0

 eλ(s−u) g(u) du .

s

The solution f (0) to (16.17) exists for arbitrary z ∈ Cn and is unique if and only if  0 

A(ds) eλs = 0, det λI − −h

2

and the result follows.

§16.4. Lyapunov’s equation As we know from §I.2.4, a matrix A is stable if and only if the Lyapunov equation A∗ Q + QA = −I, Q ≥ 0, (16.18) has a nonnegative solution Q. A generalization of this result to the infinitedimensional case is the subject of the following theorem. Theorem 16.2. Assume that E is a real Hilbert space. The infinitesimal generator A of the semigroup is exponentially stable if and only if there exists a nonnegative, linear and continuous operator Q such that 2 QAx, x = −|x|2

for all x ∈ D(A).

(16.19)

If the generator A is exponentially stable then equation (16.19) has exactly one nonnegative, linear and continuous solution Q. Proof. Assume that Q ≥ 0 is a linear and continuous operator solving (16.19). If x ∈ D(A) then the function v(t) = Qz x (t), z x (t), t ≥ 0, is

16.5 Stability of abstract hyperbolic systems

273

differentiable, see Theorem 14.2. Moreover     d dz x dz x v(t) = Q (t), z x (t) + Qz x (t), (t) dt dt dt = 2 QAz x (t), z x (t),

t ≥ 0.

Therefore, for arbitrary t ≥ 0,  t

Qz x (t), z x (t) − Qx, x = − |z x (s)|2 ds, 0  t |z x (s)|2 ds + Qz x (t), z x (t) = Qx, x.

(16.20)

0

Letting t tend to +∞ in (16.20) we obtain  +∞ |S(s)x|2 ds ≤ Qx, x, 0

x ∈ D(A).

(16.21)

Since the subspace D(A) is dense in E, estimate (16.21) holds for x ∈ E. By Theorem 16.1, the semigroup S(t), t ≥ 0, is exponentially stable. If the semigroup S(t), t ≥ 0, is exponentially stable, then z x (t) −→ 0 as t → +∞, and, letting t tend to +∞ in (16.20), we obtain  +∞

S ∗ (t)S(t)x, x dt = Qx, x, x ∈ E. (16.22) 0

Therefore, there exists a solution Q to (16.19) that is given by  +∞ S ∗ (t)S(t)x dt, x ∈ E. 2 Qx =

(16.23)

0

§16.5. Stability of abstract hyperbolic systems We consider now stability of a system introduced in Example 14.8. The solution to (14.59)–(14.60) is given by the formula



z(t) z(0) = S(t) , t ≥ 0. z(t) ˙ z(0) ˙ Let ω(A) = sup{λ; λ ∈ σ(A)} < 0 where σ(A) is the spectrum of the operator A. Theorem 16.3. 1) The operator

274

16 Stability and stabilizability

P =

I−

α2 −1 2 A α 2I

− α2 A−1 I



is the unique nonnegative solution of the Lyapunov equation  2    x  x ∈ D(A) x x  2 PA , = −α  y  , y ∈ D(−A)1/2 . y y 2) The following estimate holds:  2    2  x   x   ≤ P x , x  ,  γ−  ≤ γ +  y  y  y y where

√

1+a √ γ− = , 1+ 1+a

γ+ = 1 +

1+



x ∈ H, y

(16.24)

(16.25)

√

1+a a

and

|4ω(A)| . α2 Moreover, γ− and γ+ are the best constants in (16.25). 3) For arbitrary t ≥ 0    γ+ α exp − t . S(t) ≤ γ− 2γ+ a=

Proof of 1). In the following calculations we use elementary properties of self-adjoint operators, see N. Dunford and J. Schwartz [32]. To check that P is symmetric take [ xy ], [ xy ] ∈ H. Then   x x P , y y 

 2 I − α2 A−1 − α2 A−1 x = α y I 2I 

 α2 −1 α −1 x (I − 2 A )x − 2 A y = , α y x + y 2        2 = (−A)1/2 I − α2 A−1 x − (−A)1/2 α2 A−1 y , (−A)1/2 x + α2 x + y, y   2 = (−A)1/2 x + α2 (−A)1/2 (−A)−1 x + (−A)1/2 α2 (−A)−1 y, (−A)1/2 x   + α2 x + y, y     2 = (−A)1/2 x + α2 (−A)−1/2 x + α2 (−A)−1/2 y, (−A)1/2 x + α2 x + y, y   2 = (−A)1/2 x, (−A)1/2 x + α2 x, x + α2 y, x + α2 x, y + y, y. This implies symmetricity of P .

16.5 Stability of abstract hyperbolic systems

275

The operator P is nonnegative as it follows from the identities   2 x x P , = |(−A)1/2 x|2 + α2 |x|2 + α x, y + |y|2 y y = |(−A)1/2 x|2 + 12 [|αx|2 + 2 αx, y + |y|2 ] + 12 |y|2 = |(−A)1/2 x|2 + 12 (|αx + y|2 + |y|2 ) ≥ 0. To prove that P solves the Lyapunov equation (16.24) note that for [ xy ] ∈ D(A)   

 x x y x PA , = P , y y Az − αy y   2 = (−A)1/2 y, (−A)1/2 x + α2 y, x + α2 Ax − αy, x + α2 y, y + Ax − αy, y 2

= (−A)1/2 y, (−A)1/2 x + α2 y, x + α2 Ax, x − + α2 y, y + Ax, y − α y, y α2 2 y, x

α2 2 y, x

+ α2 y, y + Ax, y − α y, y  2  x     = − α2 (−A)x, x − α2 |y|2 = − α2 |(−A)1/2 x|2 + |y|2 = − α2   y  .

= − y, Ax +

+ α2 Ax, x −

α2 2 y, x

The proof that P is a nonnegative solution to (16.24) is therefore complete. The uniqueness will follow from the fact that S(t) is exponentially stable  +∞ as then S ∗ (t)S(t) dt. P =α 0

The exponential stability of S(t) will follow from 3). We pass to the proof of 2). We will find the maximal γ ≥ 0 such that for all [ xy ] ∈ H   2 (16.26) |(−A)1/2 x|2 + α2 |x|2 + α x, y + |y|2 ≥ γ |(−A)1/2 x|2 + |y|2 . If γ = 1, inequality (16.26) becomes 2 α2 2 |x|

equivalent to

+ α x, y ≥ 0,

√ 2  √ 2     α   √ x + 2 y  −  2 y  ≥ 0,   2  2 2 

which, obviously, does not hold for all [ xy ] ∈ H. Therefore γ ∈ [0, 1[ and (16.26) becomes (1 − γ)|(−A)1/2 x|2 +

2 α2 2 |x|

+ α x, y + (1 − γ)|y|2 ≥ 0.

For fixed x the minimal value, with respect to y, of the expression

(16.27)

276

16 Stability and stabilizability

  2 α x, y + (1 − γ)|y| = (1 − γ)y + is −

2  α α2 x − |x|2 2(1 − δ) 4(1 − γ)

α2 |x|2 . 4(1 − γ)

Therefore the required γ ∈ [0, 1[ should be such that for x ∈ D(−A)1/2 ,   1 α2 1/2 2 (1 − γ)|(−A) x| ≥ − 1 |x|2 . 2 2(1 − γ) Equivalently, one is looking for the maximal γ ∈ [0, 1[ such that   1 |(−A)1/2 x|2 α2 1 = |ω(A)| ≥ −2 inf , x=0 |x|2 4 (1 − γ)2 1−γ or a≥

1 1 . −2 (1 − γ)2 1−γ

This easily gives

√

γ− =

1+a √ . 1+ 1+a

In a similar way the expression for γ+ can be obtained.   This time one is looking for a minimal number γ > 0 such that for all xy ∈ H |(−A)1/2 x|2 +

  α2 2 |x| + α x, y + |y|2 ≤ γ |(−A)1/2 x|2 + |y|2 . 2

(16.28)

It is clear that one should look for γ > 1. For fixed x ∈ D(−A)1/2 , the maximal value, with respect to y, of the expression 2    α α2 2  −(γ − 1)|y| + α x, y = −(γ − 1)y − x + |x|2 2(γ − 1) 4(γ − 1) is

α2 |x|2 . 4(γ − 1)

Therefore, the required γ > 1 should be such that   α2 1 1/2 2 (γ − 1)|(−A) x| ≥ 1+ |x|2 . 2 2(1 − γ) Equivalently, one is looking for minimal γ > 1 such that   1 |(−A)1/2 x|2 α2 1 inf = |ω(A)| ≥ + 2 , x=0 |x|2 4 (γ − 1)2 γ−1

16.6 Stabilizability and controllability

or a≥

277

1 1 . +2 2 (γ − 1) 1−γ

This easily gives

√

1+a . a To prove the final part of the theorem denote by z(t), t ≥ 0, the strong solution of the problem

d x z(t) = Az(t), z(0) = ∈ D(A). y dt γ+ = 1 +

1+

Then, from the Lyapunov equation, d

P z(t), z(t) = −αz(t)2 , dt

t ≥ 0.

On the other hand γ− z(t)2 ≤ P z(t), z(t) ≤ γ+ z(t)2 , and therefore d α

P z(t), z(t) = −αz(t)2 ≤ − P z(t), z(t). dt γ+ Consequently

P z(t), z(t) ≤ eαt/γ+ P z(0), z(0) ≤ eαt/γ+ z(0)2 . Finally, z(t)2 ≤

1 γ+ −αt/γ+

P z(t), z(t) ≤ e z(0) γ− γ− 

and S(t) ≤

γ+ −αt/(2γ+ ) e , γ−

t ≥ 0.

§16.6. Stabilizability and controllability A linear control system y˙ = Ay + Bu,

y(0) = x

(16.29)

is said to be exponentially stabilizable if there exists a linear, continuous operator K : E −→ U such that the operator AK ,

278

16 Stability and stabilizability

with the domain D(AK ) = D(A),

AK = A + BK

(16.30)

generates an exponentially stable semigroup SK (t), t ≥ 0. That the operator AK generates a semigroup follows from the Phillips theorem, see §14.4. The operator K should be interpreted as the feedback law. If, in (16.29), u(t) = Ky(t), t ≥ 0, then y˙ = (A + BK)y, y(0) = x, (16.31) and y(t) = SK (t)x,

t ≥ 0.

(16.32)

One of the main results of finite-dimensional linear control theory was Theorem I.2.9 stating that controllable systems are stabilizable. In the infinitedimensional theory there are many concepts of controllability and stabilizability, and therefore relationships are much more complicated. Here is a typical result. Theorem 16.4. (i) Null controllable systems are exponentially stabilizable. (ii) There are approximate controllable systems which are not exponentially stabilizable. Proof. (i) Let ux (·) be a control transferring x to 0 in time T x > 0 and ux (t) = 0 for t > T x . If y x (·) is the output corresponding to ux (·) then y x (t) = 0 for t ≥ T x . Therefore  +∞ x J(x, u (·)) = (|y x (t)|2 + |ux (t)|2 ) dt < +∞, (16.33) 0

and the assumptions of Theorem 17.3, from Chapter 17, are satisfied with Q = I and R = I. Consequently, there exists a nonnegative continuous oper˜ y˜(t) = −B ∗ P˜ y˜(t) ator P˜ on H such that for the feedback control u ˜(t) = K and the corresponding solution y˜(t), t ≥ 0, of ˜ y˜(t) = (A + B K)˜ ˜ y (t), y˜˙ (t) = A˜ y (t) + B K

t ≥ 0,

one has  J(x, u ˜(·)) =

0

+∞

(|˜ y (t)|2 + |˜ u(t)|2 ) dt ≤ J(x, ux (·)) < +∞,

x ∈ E.

But y˜(t) = SK˜ (t)x, t ≥ 0, where SK˜ (·) is the semigroup generated by AK˜ . Consequently,  +∞  +∞ |SK˜ (t)x|2 dt = |˜ y (t)|2 dt < +∞ for all x ∈ E, 0

0

16.6 Stabilizability and controllability

279

and it follows from Theorem 16.1(i) that the semigroup SK˜ (·) is exponentially stable. (ii) Let us consider a control system of the form y˙ = Ay + hu,

(16.34)

y(0) = x,

in a separable Hilbert space E with an orthonormal and complete basis (em ). Let A be a linear and bounded operator on E given by Ax =

+∞ 

λm x, em em ,

x ∈ E,

m=1

with (λm ) a strictly increasing sequence converging to 0. Let h ∈ E be an element such that h, em  = 0 for all m = 1, 2, . . . and +∞  | h, em |2 < +∞. |λm |2 m=1

Then (16.34) is approximately controllable, see Theorem 15.8. To show that (16.34) is not exponentially stabilizable, consider an arbitrary bounded linear operator K : E −→ U = R1 . It is of the form Kx = k, x,

x ∈ E,

where k is an element in E. We prove that 0 ∈ σ(A + BK). Let us fix z ∈ E and consider the following linear equation: Ax + BKx = z,

x ∈ E.

(16.35)

Put

h, em  = γm ,

x, em  = ξm ,

z, em  = ηm ,

m = 1, 2, . . . .

Then (16.35) is equivalent to an infinite system of equations λm ξm + γm k, x = ηm , from which it follows that ηm γm ξm = −

k, x, λm λm Since

 +∞    γm 2    λm  < +∞ and

m=1

we have

m = 1, 2, . . . ,

m = 1, 2, . . . . +∞  m=1

|ξm |2 < +∞,

280

16 Stability and stabilizability

 +∞    ηm 2    λm  < +∞,

m=1

and z ∈ E cannot be an arbitrary element of E. Hence 0 ∈ σ(A + BK). It follows from Theorem 16.1(ii) that (16.34) is not exponentially stabilizable. 2 It follows from Theorem 16.4 that all exactly controllable systems are exponentially stabilizable. We will show that in a sense a converse statement is true. We say that system (16.29) is completely stabilizable if for arbitrary ω ∈ R there exist a linear, continuous operator K : E −→ U and a constant M > 0 such that (16.36) |SK (t)| ≤ M eωt for t ≥ 0. The following theorem together with Theorem 16.2(i) are an infinitedimensional version of Wonham’s theorem (Theorem I.2.9). Theorem 16.5. If system (16.29) is completely stabilizable and the operator A generates a group of operators S(t), t ∈ R, then system (16.29) is exactly controllable in some time T > 0. Proof. By Theorem 14.1 there exist N > 0 and ν ∈ R such that |S(−t)| ≤ N eνt ,

t ≥ 0.

(16.37)

Assume that for a feedback K : E −→ U and some constants M > 0 and ω ∈ R1 |SK (t)| ≤ M eωt , t ≥ 0. Since  S(t)x = SK (t)x −

0

t

S(t − s)BKSK (s)x ds,

x ∈ E, t ≥ 0,

therefore, for arbitrary f ∈ E ∗ , t ≥ 0, x ∈ E,  t ∗ ∗ ∗ |S (t)f | ≤ |SK (t)f | + |SK (s)K ∗ B ∗ S ∗ (t − s)f | ds 0  t eω(t−s) |B ∗ S ∗ (s)f | ds ≤ M eωt |f | + |K|M 0  t 1/2  t 1/2 e2ωs ds |B ∗ S ∗ (s)f | ds . ≤ M eωt |f | + |K|M 0

0

If |f | = 1, then |S ∗ (−t)S ∗ (t)f | = 1 and |S ∗ (−t)|−1 = |S(−t)|−1 ≤ |S ∗ (−t)f |, The above estimates imply that if |f | = 1, then

t ≥ 0.

16.6 Stabilizability and controllability

|S(−t)|−1 ≤ M eωt + |K|M

281



t

e2ωs ds

1/2 

0

0

t

|B ∗ S ∗ (s)f |2 ds

1/2

.

(16.38)

Assume that system (16.29) is not exactly controllable at a time t > 0. It follows from the results of §15.1 that for arbitrary c > 0 there exists f ∈ E ∗ such that  t

0

|B ∗ S ∗ (s)f |2 ds ≤ c and |f | = 1.

Since c is arbitrary and from (16.38) |S(−t)|−1 ≤ M eωt .

(16.39)

If (16.29) is not exactly controllable at any time t > 0 then by (16.39) and (16.37) M −1 e−ωt ≤ |S(−t)| ≤ N eνt for all t > 0. This way we have ω ≥ −ν, and (16.29) cannot be completely stabilizable. 2

Bibliographical notes Theorem 16.1(i) and Theorem 16.1(iii) are due to R. Datko [25, 26]. Proposition 16.1 is taken from J. Zabczyk [105]. Theorem 16.2 is due to R. Datko [26]. As noticed in J. Zabczyk [106], Theorem 16.4(i) is implicitly contained in R. Datko [26]. Theorem 16.5 is due to M. Megan [70]. For its generalization, see J. Zabczyk [107]. In condition (16.4) function z x (·)2 can be replaced by much more general functions N (z x (·)); see J. Zabczyk [103] and S. Rolewicz [86]. The result on stability of the abstract hyperbolic systems is taken from the preprint J. Zabczyk [115] which improves a result from A. Pritchard and J. Zabczyk [82]. Delay systems are treated also in R.F. Curtain and H. Zwart [23]. For a review of stability and stabilizability results we refer the reader to A. Pritchard and J. Zabczyk [82] and D.L. Russell [88]. For more recent results we refer to J.-M. Coron [20] and I. Lasiecka and R. Triggiani [62].

Chapter 17

Linear regulators in Hilbert spaces

In this chapter the existence of a solution to an operator Riccati equation related to the linear regulator problem in separable Hilbert spaces is shown first. Then a formula for the optimal solution on an arbitrary finite time interval is given. The existence of an optimal solution on the infinite time interval is discussed as well. Some applications to the stabilizability problem are also included. Regulator problem for delay systems is treated as well.

§17.1. Introduction Our discussion of optimal control problems in infinite dimensions will be limited to the linear regulator problem. In what follows we will assume that the stated space denoted here by H as well as the space U of the control parameters are real, separable Hilbert spaces. The control system is given by y˙ = Ay + Bu,

y(0) = x ∈ H,

(17.1)

where A is the infinitesimal generator of a semigroup S(t), t ≥ 0, and B is a linear and continuous operator from U into H. By a solution to (17.1) we mean the weak solution, see §14.7, given by  t y(t) = S(t)x + S(t − s)Bu(s) ds, r ≥ 0. (17.2) 0

Admissible controls are required to be Borel measurable and locally square integrable  T |u(s)|2 ds < +∞ for T > 0. 0

We will also consider closed-loop controls of the form © Springer Nature Switzerland AG 2020

J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_17

283

284

17 Linear regulators in Hilbert spaces

u(t) = K(t)y(t),

t ≥ 0,

(17.3)

where K(·) is a function with values in the space L(H, U ) of all linear and continuous operators from H into U . Let Q ∈ L(H, H), P0 ∈ L(H, H), R ∈ L(U, U ) be fixed self-adjoint continuous operators, and let R be, in addition, an invertible operator with the continuous inverse R−1 . The linear regulator problem with finite horizon T (compare §III.9.3), consists of finding a control u(·) minimizing the functional JT (x, ·) :  JT (x, u) =

0

T

(Qy(s), y(s) + Ru(s), u(s)) ds + P0 y(T ), y(T ).

(17.4)

The linear regulator problem with infinite horizon consists of minimizing the functional  +∞ (Qy(s), y(s) + Ru(s), u(s)) ds. (17.5) J(x, u) = 0

We present solutions to both problems. We are particularly interested in optimal controls given in the closed-loop form (17.3). Then y(t) ˙ = Ay(t) + BK(t)y(t), y(0) = x. Thus controls define outputs y(·) only indirectly as solutions of the following integral equation:  t y(t) = S(t)x + S(t − s)BK(s)y(s) ds, t ≥ 0, (17.6) 0

and we start by giving conditions implying existence of a unique continuous solutions to (17.6). We say that K : [0, T ] → L(H, U ) is strongly continuous if for arbitrary h ∈ H the function K(t)h, t ∈ [0, T ], is continuous. In the obvious way this definition generalizes to the function K(·) defined on [0, +∞). We check easily that if K(·) is strongly continuous on [0, T ] and an H-valued function h(t), t ∈ [0, T ], is continuous, then also K(t)h(t), t ∈ [0, T ], is continuous. There also exists a constant M1 > 0 such that |K(t)| ≤ M1 , t ∈ [0, T ]. Lemma 17.1. If an operator valued function K(·) is strongly continuous on [0, T ] then equation (17.6) has exactly one continuous solution y(t), t ∈ [0, T ]. Proof. We apply the contraction mapping principle. Let us denote by CT = C(0, T ; H) the space of all continuous, H-valued functions defined on [0, T ] with the norm h T = sup{|h(t); t ∈ [0, T ]|}. Let us remark that if h ∈ CT then the function

17.2 The operator Riccati equation

 AT h(t) =

t

0

285

S(t − s)K(s)h(s) ds,

t ∈ [0, T ],

(17.7)

is well defined and continuous. Let M > 0 and ω > 0 be constants such that |S(t)| ≤ M eωt , Then Therefore, if

t ≥ 0.

|AT h(t)| ≤ ω −1 M M1 (eωT − 1) h T ,

h ∈ CT .

M M1 (eωT0 − 1) < ω,

then AT0 < 1, and, by the contraction principle, see §A.2, the equation y(t) = S(t)x + AT0 y(t),

t ∈ [0, T0 ],

has exactly one solution y(·) in CT0 . Let k be a natural number such that kT0 ≥ T . By the above argument, equation (17.6) has exactly one solution on the arbitrary interval [jT0 , (j + 1)T0 ], j = 0, 1, . . . , k − 1, therefore also on [0, T ]. 2 Let us assume in particular that K(t) = K for all t ≥ 0. By the Phillips theorem, see §14.4, the operator AK = A + K, D(AK ) = D(A), defines a semigroup of operators SK (t), t ≥ 0. Its relation with the solution y(·) (17.6) is given by the following proposition, the proof of which is left as an exercise. Proposition 17.1. If K(t) = K, t ≥ 0, and SK (t), t ≥ 0, is the semigroup generated by AK = A + K, D(AK ) = D(A), then the solution y(·) to (17.5) is given by y(t) = SK (t)x, t ≥ 0.

§17.2. The operator Riccati equation To solve the linear regulator problem in Hilbert spaces we proceed similarly to the finite-dimensional case, see §III.9.3, and start from an analysis of an infinite-dimensional version of the Riccati equation P˙ = A∗ P + P A + Q − P BR−1 B ∗ P,

P (0) = P0 .

(17.8)

Taking into account that operators A and A∗ are not everywhere defined, the concept of a solution of (17.8) requires special care. We say that a function P (t), t ≥ 0, with values in L(H, H), P (0) = P0 , is a solution to (17.8) if, for arbitrary g, h ∈ D(A), the function P (t)h, g, t ≥ 0, is absolutely continuous and

286

17 Linear regulators in Hilbert spaces

d P (t)h, g = P (t)h, Ag + P (t)Ah, g dt + Qh, g − P BR−1 B ∗ P h, g

for almost all t ≥ 0. (17.9)

Theorem 17.1. A strongly continuous operator valued function P (t), t ≥ 0, is a solution to (17.8) if and only if, for arbitrary h ∈ H, P (t), t ≥ 0, is a solution to the following integral equation: P (t)h = S ∗ (t)P0 S(t)h  t + S ∗ (t − s)(Q − P (s)BR−1 B ∗ P (s))S(t − s)h ds, 0

(17.10) t ≥ 0.

The proof of the theorem follows immediately from Lemma 17.2 below concerned with linear operator equations P˙ (t) = A∗ P + P A + Q(t),

t ≥ 0, P (0) = P0 .

(17.11)

In equation (17.11) values Q(t), t ≥ 0, are continuous, self-adjoint operators. An operator valued function P (t), t ≥ 0, P (0) = P0 is a solution to (17.11) if for arbitrary g, h ∈ D(A), the function P (t)h, g, t ≥ 0, is absolutely continuous and d P (t)h, g = P (t)h, Ag + P (t)Ah, g+Q(t)h, g dt for almost all t ≥ 0.

(17.12)

Lemma 17.2. If Q(t), t ≥ 0, is strongly continuous then a solution to (17.11) exists and is given by  t P (t)h = S ∗ (t)P0 S(t)h + S ∗ (t − s)Q(s)S(t − s) ds, 0

t ≥ 0, h ∈ H. (17.13) Proof. It is not difficult to see that function P (·) given by (17.13) is well defined and strongly continuous. In addition,  t P (t)h, g = P0 S(t)h, S(t)g + Q(s)S(t − s)h, S(t − s)g ds, t ≥ 0. 0

Since for h, g ∈ D(A) d P0 S(t)h, S(t)g = P0 S(t)Ah, S(t)g + P0 S(t)h, S(t)Ag, dt and

t ≥ 0,

17.3 The finite horizon case

287

d Q(s)S(t − s)h, S(t − s)g = Q(s)S(t − s)Ah, S(t − s)g dt + Q(s)S(t − s)h, S(t − s)Ag,

t ≥ s ≥ 0,

hence d P (t)h, g = S ∗ (t)P0 S(t)Ah, g + S ∗ (t)P0 S(t)h, Ag dt  t  ∗ + Q(t)h, g + S (t − s)Q(s)S(t − s)Ah, g 0  + S ∗ (t − s)Q(s)S(t − s)h, Ag ds. Taking into account (17.13) we obtain (17.12). Conversely, let us assume that (17.12) holds. For s ∈ [0, t] d P (s)S(t − s)h, S(t − s)g ds = P (s)S(t − s)h, S(t − s)Ag + P (s)S(t − s)Ah, S(t − s)g + Q(s)S(t − s)h, S(t − s)g − P (s)S(t − s)Ah, S(t − s)g − P (s)S(t − s)h, S(t − s)Ag = Q(s)S(t − s)h, S(t − s)g. Integrating the above identity over [0, t], we obtain ∗



P (t)h, g − S (t)P0 S(t)h, g =

0

t

S ∗ (t − s)Q(s)S(t − s)h, g ds. (17.14)

Since D(A) is dense in H, equality (17.14) holds for arbitrary h, g ∈ H.

2

§17.3. The finite horizon case The following theorem extends the results of §III.9.3 to Hilbert state spaces. Theorem 17.2. (i) Equation (17.8) has exactly one global solution P (s), s ≥ 0. For arbitrary s ≥ 0 the operator P (s) is self-adjoint and nonnegative definite. (ii) The minimal value of functional (17.4) is equal to P (T )x, x and the optimal control u ˆ(·) is given in the feedback form ˆ y (t), u ˆ(t) = K(t)ˆ ˆ K(t) = −R−1 B ∗ P (T − t),

t ∈ [0, T ].

288

17 Linear regulators in Hilbert spaces

Proof. We prove first the existence of a local solution to (17.10). We apply the following version of the contraction mapping theorem (compare Theorem A.1). Lemma 17.3. Let A be a transformation from a Banach space C into C, v an element of C and α a positive number. If A(0) = 0, v ≤ 12 α and A(p1 ) − A(p2 ) ≤

1 p1 − p2 , 2

then the equation

if p1 ≤ α, p2 ≤ α,

A(p) + v = p

has exactly one solution p satisfying p ≤ α. ˜ Proof. Let A(p) = A(p) + v, p ∈ C. By an elementary induction argument   k  1 A˜k (0) ≤ α 1 − , 2

k = 1, 2, . . . .

Hence A˜k (0) ≤ α, k = 1, 2, . . . . From the Lipschitz condition it follows ˜ that the sequence (A˜k (0)) converges to a fixed point of A. Let us denote by CT the Banach space of all strongly continuous functions P (t), t ∈ [0, T ], having values being self-adjoint operators on H, with the norm P (·) T = sup{|P (t)|; t ∈ [0, T ]}. For P (·) ∈ CT we define AT (P )(t)h = −



t

S ∗ (t − s)P (s)BR−1 B ∗ P (s)S(t − s)h ds

0

and v(t)h = S ∗ (t)P0 S(t)h +

 0

t

S ∗ (t − s)QS(t − s)h ds,

h ∈ H, t ∈ [0, T ].

Equation (17.10) is equivalent to P = v + AT (P ).

(17.15)

It is easy to check that AT maps CT into CT and v(·) ∈ CT . If P1 T ≤ α, P2 T ≤ α, then |P1 (s)BR−1 B ∗ P1 (s) − P2 (s)BR−1 B ∗ P2 (s)| ≤ |(P1 (s) − P2 (s))BR−1 B ∗ P1 (s)| + |(P2 (s)BR−1 B ∗ (P2 (s) − P1 (s))| ≤ 2α|BR−1 B ∗ | P1 − P2 T , Therefore

s ∈ [0, T ].

17.3 The finite horizon case

289

AT (P1 ) − AT (P2 ) T ≤ 2α|BR−1 B ∗ |M 2



T

0

e2ωs ds P1 − P2 T

≤ β(α, T ) P1 − P2 T , where

β(α, T ) = αω −1 M 2 |BR−1 B ∗ |(e2ωT − 1).

(17.16)

At the same time, v T ≤ M 2 e2ωT |P0 | + M 2

 0

T

e2ωs ds|Q| ≤ M 2 e2ωT |P0 | +

M 2 (e2ωT − 1) |Q|. 2ω

Consequently, if for numbers α > 0 and T > 0, 1 , 2 α M 2 (e2ωT − 1) M 2 e2ωT |P0 | + |Q| < , 2ω 2 αω −1 M 2 |BR−1 B ∗ |(e2ωT − 1)
0 and M > 0, one can find α > 0 such that M 2 |P0 | +

α M2 |Q| < . 2ω 2

One can therefore find T = T (α) > 0 such that both conditions (17.17) and (17.18) are satisfied. By Lemma 17.3, equation (17.10) has a solution on the interval [0, T (α)]. 2 To proceed further we will need an interpretation of the local solution. Lemma 17.4. Let us assume that a function P (t), t ∈ [0, T ], is a solution to (17.10). Then for arbitrary control u(·) and the corresponding output y(·), JT0 (x, u) = P (T0 )x, x  T0 |R1/2 u(s) + R−1/2 B ∗ P (T0 − s)y(s)|2 ds. +

(17.19)

0

Proof. Assume first that x ∈ D(A) and that u(·) is of class C 1 . Then y(·) is the strong solution of the equation d y(t) = Ay(t) + Bu(t), dt

y(0) = x.

290

17 Linear regulators in Hilbert spaces

It follows from (17.9) that for arbitrary z ∈ D(A) function P (T0 − t)z, z, t ∈ [0, T0 ], is of class C 1 , and its derivative is given by d P (T0 − t)z, z = −P (T0 − t)z, Az − P (T0 − t)Az, z − Qz, z dt + P (T0 − t)BR−1 B ∗ P (T0 − t)z, z, t ∈ [0, T0 ]. Hence d P (T0 − t)y(t), y(t) dt = −P (T0 − t)y(t), Ay(t) − P (T0 − t)Ay(t), y(t) − Qy(t), y(t) + P (T0 − t)BR−1 B ∗ P (T0 − t)y(t), y(t) + P (T0 − t)y(t), ˙ y(t) + P (T0 − t)y(t), y(t) ˙ = −Qy(t), y(t) + P (T0 − t)BR−1 B ∗ P (T0 − t)y(t), y(t) + P (T0 − t)Bu(t), y(t) + P (T0 − t)y(t), Bu(t) = −Qy(t), y(t) + |R1/2 u(t) + R−1/2 B ∗ P (T0 − t)y(t)|2 − Ru(t), u(t),

t ∈ [0, T0 ].

Integrating this equality over [0, T0 ] we obtain  T0 |R1/2 u(t) + R−1/2 B ∗ P (T0 − t)y(t)|2 dt, −P (T0 )x, x = −JT0 (x, u) + 0

and therefore (17.19) holds. Since the solutions y(·) depend continuously on the initial state x and on the control u(·), equality (17.19) holds for all x ∈ H and all u(·) ∈ L2 (0, T0 ; U ). 2 It follows from Lemma 17.4 that for arbitrary u(·) ∈ L2 (0, T0 ; U ), P (T0 )x, x ≤ JT0 (x, u). By Lemma 17.1, the equation  t yˆ(t) = S(t)x − S(t − s)BR−1 B ∗ P (T0 − s)ˆ y (s)ds, 0

t ∈ [0, T0 ],

has exactly one solution yˆ(t), t ∈ [0, T ]. Define control u ˆ(·): u ˆ(t) = −R−1 B ∗ P (T0 − t)ˆ y (t),

t ∈ [0, T0 ].

Then yˆ(·) is the output corresponding to u ˆ(·) and by (17.19), JT0 (x, u ˆ(·)) = P (T0 )x, x.

(17.20)

Therefore u ˆ(·) is the optimal control. It follows from (17.20) that P (T0 ) ≥ 0. Setting u(·) = 0 in (17.19) we obtain

17.4 Regulator problem for delay systems

291

 0 ≤ P (T0 )x, x ≤ 0 ≤ P (T0 )x, x ≤

T0

0

 0

QS(t)x, S(t)x dt,

T0

 S ∗ (t)QS(t) dt x, x .

Consequently  |P (T )| ≤

T0

0

|S ∗ (t)QS(t)| dt

for T ≤ T0 .

Let now α be a number such that M2

 0

T˜

|S ∗ (t)QS(t)| dt +

α M2 |Q| < 2ω 2

where T˜ is a positive number given in advance. We can assume that T1 < T (α). It follows from the proof of the local existence of a solution to (17.10), that the solution to (17.10) exists on [0, T1 ]. Repeating the proof, on consecutive intervals [T1 , 2T1 ], [2T1 , 3T1 ], . . . , we deduce that there exists a unique global solution (17.8). Nonnegativeness of P (·) as well as the latter part of the theorem follow from Lemma 17.4. 2

§17.4. Regulator problem for delay systems We will apply the abstract results to the problem of minimizing the cost functional     T

x Qy(s), y(s) + |u(s)|2 ds + P0 y(T ), y(T ) (17.21) , u(·) = JT ϕ 0 for controlled delay systems  0 y(t) ˙ = A(dr) y(t + r) + Bu(t), −h

(17.22)

y(0) = x ∈ Rn , y(θ) = ϕ(θ), θ ∈ [−h, 0[, where controls u(·) take values in Rm and the initial segment ϕ belongs to  y(t)  , t ≥ 0, then z is the weak solution of the L2 (−h, 0; Rn ). Define z(t) = y(t+·) equation   x z(t) ˙ = Az(t) + Bu(t), z(0) = , t ≥ 0, (17.23) ϕ   and Bu = Bu ∈ H, for u ∈ Rm . The cost functional can be rewritten as 0

292

17 Linear regulators in Hilbert spaces

    T

x Qz(s), z(s)H + |u(s)|2 ds + P0 z(T ), z(T )H JT , u(·) = ϕ 0 w  where, for ψ ∈ H         w Qw w P0 w Q = , P0 = . ψ 0 ψ 0

(17.24)

(17.25)

By Theorem 17.2 the problem of minimizing (17.21) when the control system is (17.22) has a solution given in the feedback form:  z (t), u (t) = K(t)  K(t) = −B ∗ P(T − t),

t ∈ [0, T ],

where P(t), t ≥ 0, is the solution of the operator Riccati equation P˙ = A∗ P + PA + Q − PBB ∗ P,

P(0) = P0 .

(17.26)

P(t), t ≥ 0, are positive, self-adjoint operators acting on the space H. Thus   P00 (t) P01 (t) P(t) = , t ≥ 0, (17.27) P10 (t) P11 (t) where P00 (t) ∈ L(Rn , Rn ), P01 (t) ∈ L(L2 (−h, 0; Rn ), Rn ), P10 (t) ∈ L(Rn , L2 (−h, 0; Rn )), P11 (t) ∈ L(L2 (−h, 0; Rn ), L2 (−h, 0; Rn )). In particular, P00 (t), t ≥ 0, can be identified with an n × n-matrix valued function and  P01 (t)ψ =

0

−h

P01 (t, r)ψ(r) dr

where for each t, P01 (t, r) is an n × n-matrix valued function such that 0 |P01 (t, r)|2 dr < +∞. Note that −h       BB ∗ w w ∗ w BB = , ∈ H. ϕ 0 ψ The optimal solution z solves the equation    t x St−s BB ∗ P(T − s) z (s) ds. (17.28) − z(t) = St ϕ 0   For arbitrary t ≥ s, and w ψ ∈ H:   w St−s BB ∗ P(T − s) ψ  0  ∗ BB P00 (T − s)w + −h P01 (T − s, r)ψ(r) dr = St−s . 0

17.5

The infinite horizon case: Stabilizability and detectability

293

However, for arbitrary v ≥ 0, w ∈ Rn    w  w y (v) Sv = 0 y w (v + ·) where y w is the solution of the delay equation  0 A(r)y w (t + r) dr, y˙ w (t) = −h

t ≥ 0,

with the initial condition y w (0) = w, y w (θ) = 0, θ ∈ [−h, 0]. It therefore   ∈ H, then z0 (t) = z0 (t + ·), t ≥ 0, the second follows that if z(t) = zz01 (t) (t) coordinate of the optimal solution is the segment of the first coordinate. However z is the weak solution of the equation     x d ∗ z(t) = A − BB P(T − t) z(t), z(0) = . ϕ dt Therefore d z0 (t) = dt



0

−h

A(dr) z0 (t + r) ∗



z0 (t) + − BB P00 (T − t) and thus d z0 (t) = dt



0

−h

 P01 (T − t, r) z0 (t + r) dr ,

0

−h

A(t, dr) z0 (t + r) dr,

where A(t, dr) = A(dr) − BB ∗ P0 (T − t)δ{0} (dr) + P01 (T − t, r) dr. We see that the optimal solution solves a delay equation with the new, time dependent measure acting on delays.

§17.5. The infinite horizon case: Stabilizability and detectability We proceed to the regulator problem on [0, +∞) (compare to (17.5)). An operator algebraic Riccati equation is of the form

294

17 Linear regulators in Hilbert spaces

2P Ax, x − P BR−1 B ∗ P x, x + Qx, x = 0,

x ∈ D(A),

(17.29)

where a nonnegative operator P ∈ L(H, H) is unknown. Theorem 17.3. Assume that for arbitrary x ∈ H there exists a control ux (t), t ≥ 0, such that (17.30) J(x, ux (·)) < +∞. Then there exists a nonnegative operator P˜ ∈ L(H, H) satisfying (17.29) such that P˜ ≤ P for an arbitrary nonnegative solution P of (17.29). Moreover, the control u ¯(·) given in the feedback form u ˜(t) = −R−1 B ∗ P˜ y(t),

t ≥ 0,

minimizes functional (14.5). The minimal value of this functional is equal to P˜ x, x. Proof. Let P (t), t ≥ 0, be the solution of the Riccati equation (17.9) with the initial condition P0 = 0. It follows from Theorem 17.2(ii) that, for arbitrary x ∈ H, function P (t)x, x, t ≥ 0, is nondecreasing. Moreover, P (t)x, x ≤ J(x, ux (·)) < +∞,

x ∈ H.

Therefore there exists a finite lim P (t)x, x, x ∈ H. On the other hand t↑+∞

P (t)x, y =

1 (P (t)(x + y), x + y − P (t)(x − y), x − y) 2

(17.31)

for arbitrary x, y ∈ H. Applying Theorem A.5 (Banach–Steinhaus theorem), first to the family of functionals P (t)x, ·, t ≥ 0, for arbitrary x ∈ H and then to the family of operators P (t), t ≥ 0, we see that sup |P (t)| = c < +∞. Hence for arbitrary t≥0

x, y ∈ H, there exists a finite limit a(x, y) = lim P (t)x, y and t→+∞

|a(x, y)| ≤ (sup |P (t)|)|x| |y| ≤ c|x| |y|, t≥0

x, y ∈ H.

Therefore there exists P˜ ∈ L(H, H) such that a(x, y) = P˜ x, y,

x, y ∈ H.

The operator P˜ is self-adjoint, nonnegative definite, because a(x, y) = a(y, x), a(x, x) ≥ 0, x, y ∈ H.

17.5

The infinite horizon case: Stabilizability and detectability

295

To show that P˜ satisfies (17.29) let us fix x ∈ D(A) and consider (17.9) with h = y = x. Then d P (t)x, x = P (t)x, Ax + P (t)Ax, x dt + Qx, x − P BR−1 B ∗ P x, x.

(17.32)

Letting t tend to +∞ in (17.32) and arguing as in the proof of Theorem III.9.4 we easily show that P˜ is a solution to (17.29). The proof of the final part of the theorem is completely analogous to that of Theorem III.9.4. 2 Let A be the infinitesimal generator of a C0 -semigroup S(t), t ≥ 0, and let B ∈ L(U, H). The pair (A, B) is said to be exponentially stabilizable if there exists K ∈ L(H, U ) such that the operator AK = A + BK, D(AK ) = D(A) generates an exponentially stable semigroup (compare to §12.3). Note that in particular if the pair (A, B) is null controllable then it is exponentially stabilizable. Let C ∈ L(H, V ) where V is another separable Hilbert space. The pair (A, C) is said to be exponentially detectable if the pair (A∗ , C ∗ ) is exponentially stabilizable. The following theorem is a generalization of Theorem III.9.5. Theorem 17.4. (i) If the pair (A, B) is exponentially stabilizable then the equation (17.29) has at least one nonnegative solution P ∈ L(H, H). (ii) If Q = C ∗ C and the pair (A, C) is exponentially detectable then equation (17.29) has at most one solution and if P is the solution of (17.29) then the operator A − BR−1 B ∗ P is exponentially stable and the feedback K = −R−1 B ∗ P exponentially stabilizes (17.1). Proof. (i) If the pair (A, B) is exponentially stabilizable then the assumptions of Theorem 17.3 are satisfied and therefore (17.29) has at least one solution. (ii) We prove first a generalization of Lemma III.9.3. Lemma 17.5. Assume that for a nonnegative operator M ∈ L(H, H) and K ∈ L(H, U ) 2M (A + BK)x, x + C ∗ Cx, x + K ∗ RKx, x = 0,

x ∈ D(A). (17.33)

(i) If the pair (A, C) is exponentially detectable then the operator AK = A + BK, D(AK ) = D(A) is exponentially stable. (ii) If in addition P ∈ L(H, H), P ≥ 0 is a solution to (17.29) with Q = C ∗ C, then P ≥ M. (17.34)

296

17 Linear regulators in Hilbert spaces

Proof. (i) Let S1 (·) be the semigroup generated by AK . Since the pair (A, C) is exponentially detectable, there exists an operator L ∈ L(V, H) such that the operator A∗L∗ = A∗ + C ∗ L∗ , D(A∗L∗ ) = D(A∗ ), is exponentially stable. ˜ = D(A) generates Hence the operator A˜ = (A∗ + C ∗ L∗ )∗ = A + LC, D(A) an exponentially stable semigroup S2 (·). Let y(t) = S1 (t)x, t ≥ 0. Since A + BK = (A + LC) + (LC + BK), therefore, by Proposition 17.1,  t y(t) = S2 (t)x + S2 (t − s)(LC + BK)y(s) ds.

(17.35)

We show that   +∞ 2 |Cy(s)| ds < +∞ and

(17.36)

0

0

+∞

0

|Ky(s)|2 ds < +∞.

Assume first that x ∈ D(A). Then y(t) ˙ = (A + BK)y and

d M y(t), y(t) = 2M y(t), ˙ y(t), dt

t ≥ 0.

It follows from (17.33) that d M y(t), y(t) + C ∗ y(t), Cy(t) + RKy(t), Ky(t) = 0. dt Hence



M y(t), y(t) +

0

t

|Cy(s)|2 ds +

 0

t

RKy(s), Ky(s) ds = M x, x. (17.37)

By a standard limit argument, (17.37) holds for all x ∈ H. Applying Theorem A.8 (Young’s inequality) in the same way as in the proof of Lemma III.9.3, we obtain  +∞  +∞ |S1 (t)x|2 dt = |y(t)|2 dt < +∞. 0

0

By Theorem 16.1(i), the semigroup S1 (·) is exponentially stable. Thus the proof of (i) is completed. (ii) Put M − P = W . Then for x ∈ D(A) 2W (A + BK)x, x = −C ∗ Cx, x − K ∗ RKx, x − 2P Ax, x − 2P BKx, x. Since P satisfies (17.29) with Q = C ∗ C,

17.5

The infinite horizon case: Stabilizability and detectability

297

2W (A + BK)x, x = −K ∗ RKx, x − P BR−1 B ∗ P x, x

(17.38)

− 2P BKx, x. Let K0 = −R−1 B ∗ P , then RK0 = −B ∗ P, (17.38) we easily obtain

P B = −K0∗ R, and from

2W (A + BK)x, x = −(K − K0 )∗ R(K − K0 )x, x,

x ∈ D(A).

Since A+BK is the exponentially stable generator, operator W is nonnegative by Theorem 16.2. 2 We return now to the proof of part (ii) of Theorem 17.4 and assume that operators P ≥ 0 and P1 ≥ 0 are solutions of (17.29). Let K = −R−1 B ∗ P , then 2P (A + BK)x, x + K ∗ RKx, x + C ∗ Cx, x = 2P Ax, x − P BR−1 B ∗ P x, x + C ∗ Cx, x = 0.

(17.39)

Consequently, by Lemma 17.5(ii), P1 ≤ P . In the same way P ≤ P1 . Hence P = P1 . Moreover, by Lemma 17.5(i) and (17.39) the operator A + BK is exponentially stable. 2

Bibliographical notes The direct proof of the existence of the solution P (·) to the Riccati equation (17.9) based on Lemma 17.3 is due to G. Da Prato [24]. Theorem 17.4 is taken from the author’s paper J. Zabczyk [106]. More information on the subject can be found in the monographs by R.F. Curtain and A.J. Pritchard [22] and I. Lasiecka and R. Triggiani [62]. A specific application to delay systems can be found in R.F. Curtain and H. Zwart [23, pp. 288–298].

Chapter 18

Boundary control systems

In this chapter the semigroup theory is applied to systems when the control action is exercised on the boundary of the spatial domain where the system evolves. The general theory is illustrated by one-dimensional heating systems and the null controllability of the multidimensional heating systems.

§18.1. Semigroup approach to boundary control systems Of real interests are control systems described by parabolic or hyperbolic equations with control action applied on the boundary of a spatial region rather than inside. Typical examples can be described as follows. Let Ω ⊂ Rn be a fixed, spatial region and A an operator with partial derivatives acting from a function space H0 (Ω) into, say, H = L2 (Ω), and τ a linear operator associating with elements of H0 (Ω) their boundary values or the boundary values of some of their derivatives, which are functions defined on the boundary ∂Ω = Γ and elements of a function space H1 (Γ ), embedded into L2 (Γ ). The state y(t, ξ), (t, ξ) ∈ (0, T ) × Ω, of a boundary control system satisfies the equation ∂y (t, ξ) = (Ay(t))(ξ) ∂t τ y(t)(η) = u(t, η) y(0) = x

(t, ξ) ∈ (0, T ) × Ω (t, η) ∈ (0, T ) × Γ, boundary condition initial condition.

(18.1)

Here u(t), t ≥ 0, is a U -valued function where U is a set of control parameters and a subset of H1 (Γ ). It turns out that the semigroup treatment, used for systems with distributed controls, can be adjusted to cover also the boundary systems. Assume that a restriction A of A to the set © Springer Nature Switzerland AG 2020

J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6_18

299

300

18 Boundary control systems

D(A) = {ω ∈ H0 (Ω); τ (ω) = 0} generates a C0 -semigroup S(t), t ≥ 0 on L2 (Ω). Assume that the number λ is regular for the operator A, that is, the operator λ − A has a continuous inverse. Assume also that for the number λ the problem λω − Aω = 0,

τ (ω) = u,

(18.2)

has a unique solution ω = D(u) for arbitrary element u ∈ H1 (Γ ) and that D is a continuous mapping from u ∈ H1 (Γ ) into Lp (Ω). We then have the following result: Proposition 18.1. Assume that the operators A, A, D satisfy the above conditions and that u(·) is a twice continuously differentiable function with values in U and x − Du(0) ∈ D(A). Then  t S(t − s)Du(s) ds, t ∈ [0, T ], (18.3) y(t) = S(t)x + (λ − A) 0

is a solution of the boundary problem (18.1). Note that formally inserting the operator λ − A under the integral sign we get a familiar expression as treated earlier with the operator B = (λ − A)D. However the definition of B cannot be made precise and one should work with more complicated formula (18.3). Proof. By Theorem 14.10 there exists a strong solution z(t), t ≥ 0, of the equation dz du = Az(t) + λDu(t) − D (t), t ≥ 0, dt dt z(0) = x − Du(0), and in particular τ (z(t)) = 0 for all t ≥ 0. Define y(t) = z(t) + Du(t),

t ≥ 0.

We check first that y is a solution to (18.1) and then identify it with (18.3). By elementary calculations: τ (y(t)) = τ (Du(t)) = u(t) d (y(t) − Du(t)) = (A − λ)(y(t) − Du(t)) dt du (t) dt du (t). = (A − λ)(y(t) − Du(t)) + λy(t) − D dt + λ(y(t) − Du(t)) + λDu(t) − D

Thus taking into account that (A − λ)Du(t) = 0 we see that y(t), t ≥ 0, solves the required boundary problem. On the other hand, it follows from

18.2 Boundary systems with general controls

301

the proof of Theorem 14.10 that  t  t du (s)ds + S(t)Du(0) − Du(t). A S(t − s)Du(s) ds = S(t − s)D ds 0 0 From the definition of z we have  z(t) = S(t)(x − Du(0)) +

t

0

  du (s) ds. S(t − s) λDu(s) − D ds

Since y(t) = z(t) + Du(t), by combining the above two formulae we arrive at (18.3). 2

§18.2. Boundary systems with general controls We will work with systems given by (18.3). The following two propositions formulate conditions on A and D under which the formula (18.3) defines a continuous function for all u ∈ Lp (0, T ; U ), extending the abstract definition of the boundary control system to more general controls. Proposition 18.2. Let A be a self-adjoint operator with a complete set of eigenvectors (en ) corresponding to eigenvalues λn ≤ 0, n = 1, 2, . . . . If p > 1 and +∞  |λn |2/p |D∗ en |2 < +∞, (18.4) n=1

then y given by (18.3) is well defined and continuous for u ∈ Lp [0, T ; U ]. Proof. For smooth u(·): 

t

A 0

+∞ 

S(t − s)Du(s) ds =

 λn en

n=1

Consequently, defining q =

t

0

eλn (t−s) D∗ en , u(s) ds.

p p−1

+∞   t  t 2  2     S(t − s)Du(s) ds = λ2n  eλn (t−s) D∗ en , u(s) ds A 0

≤

+∞  n=1

2

|λn |



t

λn sq

e 0

0

n=1

ds

2/q

∗

2

|D en |

 0

t

|u(s)|p ds

 2/q  +∞  T 2/p 1 ≤ |λn |2/p |D∗ en |2 |u(s)|p ds . q 0 n=1

2/p

302

18 Boundary control systems

The estimate shows that the mapping from the functions u ∈ Lp [0, T ; U ] to functions y ∈ C[0, T ; U ] is continuous so it can be extended from regular functions to the whole Lp [0, T ; U ]. In fact, for an arbitrary u ∈ Lp [0, T ; U ] let un be a sequence of smooth functions converging to u in the sense of Lp [0, T ; U ] and  yn (t) = A

t

0

S(t − s)Dun (s) ds,

t ∈ [0, T ], n = 1, 2, . . . ,

the corresponding sequence of the state trajectories. Then, by the obtained estimate, yn converges to a continuous function y. The closedness of the operator A implies that  t y(t) = A S(t − s)Du(s) ds, t ∈ [0, T ]. 0

2

The proof is complete.

The following prop will be helpful for a study of multidimensional null controllability problem. Proposition 18.3. Let A be a self-adjoint, negative definite generator of a C0 semigroup S(t), t ≥ 0, and D a bounded operator from a Hilbert space U into H. If for some numbers α, p, such that 0 < 1/p < α < 1 the operator (−A)α D is bounded then the function  t  y(t) = A S(t − s)Du(s) ds , 0

t ∈ [0, T ],

is continuous for arbitrary u ∈ Lp [0, T ; U ]. Proof. Note that (−A) = (−A)α (−A)1−α and for arbitrary T > 0 there exists K > 0 such that |(−A)1−α S(t)| ≤ Ktα−1 ,

t ∈ [0, T ].

Therefore  y(t) =

0

t

(−A)1−α S(t − s)(−A)α Du(s) ds,

t ∈ [0, T ].

If the operator (−A)α D is bounded with the norm K1 , we get by the Hölder inequality:

18.3 Heating systems



303 t

|(−A)1−α S(t − s)| |(−A)α D| |u(s)| ds  t 1/q  t 1/p |(−A)1−α S(t − s)|q ds |u(s)|p ds ≤ K1 0 0  t 1/q  t 1/p 1 p ≤ KK1 ds |u(s)| ds . q(1−α) 0 s 0

|y(t)| ≤

0

Since 0 < 1/p < α < 1, we have q(1 − α) < 1 and  0

t

1/p

1

ds

sq(1−α)

= K2 < +∞.

Consequently |y(t)| ≤ KK1 K2



T

0

|u(s)|p ds

1/p

,

t ∈ [0, T ],

and by the same argument as at the end of the previous proof, the result follows. 2

§18.3. Heating systems In this section we present the abstract description of two boundary control systems describing heating of a rod with controls applied at one end of the rod, determining the temperature or the heat flow at the end and keeping, respectively, the temperature or the heat flow equal 0 at the other end. In the next section we will prove that they are approximately controllable. Example 18.1. Control of the temperature ∂2y ∂y (t, ξ) = 2 (t, ξ) on (0, T ) × (0, π) ∂t ∂ξ y(t, 0) = u(t), y(t, π) = 0 for t ∈ (0, T ) y(0, ξ) = x(ξ),

x ∈ H = L2 (0, π).

Example 18.2. Control of the heat flow ∂y ∂2y (t, ξ) = 2 (t, ξ) on (0, T ) × (0, π) ∂t ∂ξ ∂y ∂y (t, 0) = u(t) (t, π) = 0 for t ∈ (0, T ) ∂ξ ∂ξ y(0, ξ) = x(ξ), x ∈ H = L2 (0, π).

304

18 Boundary control systems

In Example 18.1 we define A as the operator of the second derivative with the Dirichlet boundary conditions and take λ = 0. In Example 18.2 A is again the operator of the second derivative but with the Neumann boundary conditions and λ = 1. In the both cases H = L2 (0, π), U = R1 . For Example 18.1 we have π−ξ Du(ξ) = u, ξ ∈ [0, π], π 2 en (ξ) = sin nξ, λn = −n2 , n = 1, 2 . . . , π π 1 D∗ h = (π − ξ)h(ξ) dξ, h ∈ L2 (0, π) π 0 4 |D∗ en |2 = , n = 1, 2, . . . πn2 For Example 18.2: − ch(π − ξ) u, Du(ξ) = sh π  π 1 D∗ h = − ch(π − ξ)h(ξ) dξ, h ∈ L2 (0, π) sh π 0 2 1 cos nξ, λn = −n2 , n = 1, . . . λ0 = 0, e0 = √ , en (ξ) = π π 1 2 |D∗ en |2 = , n = 1, . . . . π (1 + n2 )2 We will derive the formulae for the latter case leaving the former and easier to the reader. We need the following lemmas which will be useful also for a study of a multidimensional control system in Section 18.5. Lemma 18.1. The equation d Φ(ξ) = a2 Φ(ξ), dξ

ξ ∈ (0, π),

(18.5)

with the boundary conditions: d Φ(0) = α, dξ

d Φ(π) = β, dξ

has a solution Φa , for a = 0, given by the formula

 1 β β sh(aξ), φa (ξ) = − α− ch a(π − ξ) + a sh(aπ) ch(aπ) a ch(aπ) In particular, if α = 1 and β = 0,

ξ ∈ [0, π].

18.3 Heating systems

305

φa (ξ) = −

ch a(π − ξ) , a sh(aπ)

ξ ∈ [0, π].

Proof. The result follows from the definitions of the hyperbolic functions: ch ξ =

1 ξ (e + e−ξ ), 2

sh ξ =

1 ξ (e − e−ξ ), 2

ξ ∈ R,

and obvious formulae for their derivatives. d sh(ξ) = ch ξ, dξ

d ch(ξ) = sh ξ, dξ

ξ ∈ R.



Integrating by parts we arrive at the following formulae: Lemma 18.2. If 1 e0 = √ , en (ξ) = π



2 cos nξ, π

n = 1, . . . ,

then  0

π



1 2 Φa (ξ)en (ξ)dξ = − , 2 + n2 π a  π 1 1 Φa (ξ)e0 (ξ)dξ = − , π a2 0

for n = 1, . . . , if n = 0.

Going back to the latter system, the formula for the transformation D acting from R1 to L2 (0, π) follows from (18.5) and the formula for D∗ follows from the very definition. Moreover from Lemma 18.2 and for n = 1, . . .  π 1 2 ∗ Φ1 (ξ)en (ξ)dξ = − , D en = π 1 + n2 0 1 2 |D∗ en |2 = . π (1 + n2 )2 Taking into account Proposition 18.2 and the above formulae we arrive at the following regularity result. Proposition 18.4. The trajectories of the system from Example 18.1 are continuous for controls u(·) ∈ Lp (0, T ), p > 6. Similarly the trajectories of the system from Example 18.2 are continuous for controls u(·) ∈ Lp (0, T ), p > 43 , and in particular when p = 2.

306

18 Boundary control systems

§18.4. Approximate controllability of the heating systems The following theorem is a version of Theorem 15.8 valid for boundary control systems:  t y(t) = S(t)x + (λ − A) S(t − s)gu(s) ds, t ∈ [0, T ], u ∈ Lp [0, T ] (18.6) 0

where g ∈ H and A is a self-adjoint generator with distinct eigenvalues λn , n = 1, 2, . . . diverging to −∞ and complete system of eigenvectors en , n = 1, 2, . . . . Theorem 18.1. If for each n = 1, 2, . . . , g, en  = 0, and p is a number greater than 1, then the boundary control system (18.6) is approximately controllable. Proof. Fix r ∈ (0, T ). Since the operator (λ − A)S(r) is bounded, for functions u such that u(s) = 0, s ∈ [0, r], we have:  (λ − A)

T 0

 S(s)gu(s) ds =

0

T

(T − r)S(s)(λ − A)S(r)gu(s + r) ds.

Thus putting h = (λ − A)S(r)g: h, en  = (λ − λn )erλn g, en  = 0, and the proof of Theorem 15.8 is valid in the present situation.



By direct considerations the theorem implies the following result: Theorem 18.2. The boundary control systems defined as Example 18.1 and Example 18.2 are approximately controllable on arbitrary time interval (0, T ).

§18.5. Null controllability for a boundary control system Using abstract formulation we will prove null controllability for a heat equation on a parallelepipedon Ω = (0, π)d with Neumann boundary conditions and control applied on a portion of the boundary. The control system is as follows:

18.5 Null controllability for a boundary control system

⎧ ∂y ⎪ ⎪ (t, ξ) = Δy(t, ξ), ⎪ ⎪ ∂t ⎪ ⎪ ⎪ ⎪ ∂y ⎪ ⎪ (t, ξ) = u(t, ξ), ⎪ ⎪ ⎪ ⎨ ∂ξd ∂y (t, ξ) = 0, ⎪ ⎪ ∂ξd ⎪ ⎪ ⎪ ⎪ ∂y ⎪ ⎪ (t, ξ) = 0, j < d, ⎪ ⎪ ∂ξj ⎪ ⎪ ⎪ ⎩y(0, ξ) = x(ξ),

307

t > 0, ξ ∈ Ω, t > 0, ξ = (ξ1 , . . . , ξd−1 , 0) ∈ ∂Ω, t > 0, ξ = (ξ1 , . . . , ξd−1 , π) ∈ ∂Ω,

(18.7)

t > 0, ξ = (ξ1 , . . . , ξd ) ∈ ∂Ω, ξj = 0 or π, ξ ∈ Ω, x ∈ H = L2 (Ω).

We denote by Γ = ∂Ω the boundary of Ω and by Γ1 , Γ1 = {(ξ1 , . . . , ξd ) ∈ ∂Ω; ξd = 0}. The part of the boundary to which the control action is applied and set U = L2 (Γ1 ). Theorem 18.3. Control system (18.7) is null controllable in arbitrary time T > 0 with control processes u(·) ∈ L2 (0, T ; U ). We first construct the abstract version of the above boundary control system in the form  t y(t) = S(t)x + (1 − A) S(t − s)Du(s) ds, t ∈ [0, T ], (18.8) 0

with λ = 1 and show that the square integrable controls produce continuous trajectories y. As far as the generator A is concerned we take the Laplace operator with the Neumann boundary condition:  ∂x D(A) = {x ∈ H 2 (Ω); ∂γ (ξ) = 0, ξ ∈ ∂Ω} (18.9) Ax = Δx for x ∈ D(A), where H 2 (Ω) is the classical Sobolev space of order 2. The partial derivatives, in the definition of the domain of A, are taken in the directions γ orthogonal to the boundary Γ . We will find now the eigenvectors and eigenvalues of the operator A. The system of eigenvectors turns out to be complete and one can therefore write an explicit formula for the corresponding semigroup. For arbitrary u ∈ U = L2 (Γ1 ) we denote by Du = ω ∈ L2 (Ω) a solution of the following problem:

308

18 Boundary control systems

⎧ ⎪ Δω − ω = 0 on Ω ⎪ ⎪ ⎪ ⎪ ⎪ ∂ω ⎪ ⎪ for ξ = (ξ1 , . . . , ξd−1 , 0) ∈ ∂Ω, ⎪ ⎨ ∂ξd (ξ) = u(ξ) ∂ω ⎪ (ξ) = 0 for ξ = (ξ1 , . . . , ξd−1 , π) ∈ ∂Ω, ⎪ ⎪ ⎪ ∂ξ d ⎪ ⎪ ⎪ ∂ω ⎪ ⎪ ⎩ (ξ) = 0, j < d, for ξ = (ξ1 , . . . , ξd ) ∈ ∂Ω, ξj = 0 or π. ∂ξj

(18.10)

To find an explicit formula for the operator D, define λ0 = 0, λn = −n2 , n = 1, . . . , 2 1 cos(ξ), ξ ∈ (0, π). en (ξ) = e0 (ξ) = √ , π π Then (λn ) and (en ) consist of eigenvalues and eigenfunctions of the operad2 dx dx 2 2 tor dξ 2 on L (0, π) with the domain {x ∈ H (0, 1); dξ (0) = dξ (π) = 0}. The sequence (en ) is complete in L2 (0, π). Moreover the numbers λn1 n2 ...nd = −(n21 + . . . + n2d ),

ni ∈ N ∪ {0},

i = 1, 2, . . . , d,

and the function en1 n2 ...nd (ξ) = en1 (ξ1 ) . . . end (ξd ),

ξ = (ξ1 , . . . , ξd ) ∈ Ω,

form, respectively, the sequence of eigenvalues and eigenfunction corresponding to the operator A given by (18.9). An arbitrary function u ∈ U = L2 (Γ1 ) can be uniquely represented by its expansion with respect to the orthonormal and complete basis em1 m2 ...md−1 (ξ1 , . . . , ξd−1 ) = em1 (ξ1 ) . . . emd−1 (ξd−1 ),

(ξ1 , . . . , ξd−1 ) ∈ Γ1 ,

where mi ∈ N ∪ {0},

i = 1, 2, . . . , d − 1.

2

Thus, in the U = L (Γ1 ) sense: u(ξ1 , . . . , ξd−1 )  =

um1 ...md−1 em1 ...md−1 (ξ1 , . . . , ξd−1 ), (ξ1 , . . . , ξd−1 ) ∈ Γ1 .

m1 ,...,md−1

It turns out that trajectories of the abstract version of the system are continuous. Namely we have: Proposition 18.5. The trajectories y of the boundary system (18.8) are continuous for arbitrary controls u(·) ∈ Lp (0, T ; U ), p > 43 , in particular they are continuous for u(·) ∈ L2 (0, T ; U ).

18.5 Null controllability for a boundary control system

309

The results is a consequence of Proposition 18.3 and the following two lemmas of independent interest. Lemma 18.3. The solution ω = Du of (18.10) is of the form Du(ξ1 , . . . , ξd−1 , ξd )  =

um1 m2 ...md−1 em1 ...md−1 (ξ1 . . . ξd−1 )Φ˜m1 ...md−1 (ξd )

m1 ,...,md−1

˜m ...m where Φ are functions of the variable ξd ∈ [0, π]: 1 d−1 ˜m ...m (ξd ) = Φ(1+m2 +...+m2 )1/2 (ξd ), Φ 1 d−1 1 d−1 Φ˜m1 ...md−1 (ξd ) = −

ch[(1 + m21 + . . . + m2d−1 ) 1/2

(1 + m21 + . . . + m2d−1 )

1/2

(ξd − π)]

sh[(1 + m21 + . . . + m2d−1 )

1/2

. π]

Proof. Assume that the boundary function u is given by the formula u(ξ1 , . . . , ξd−1 ) = em1 m2 ...md−1 (ξ) =em1 (ξ1 ) . . . emd−1 (ξd ), ξ = (ξ1 , . . . , ξd−1 ) ∈ Rd−1 . We look for the solution ω in the form ω(ξ1 , . . . , ξd−1 , ξd ) = em1 (ξ1 ) . . . emd−1 (ξd−1 )Φ(ξd ). Then Δω(ξ) =

d−1  1

 d2 λmj ω(ξ) + em1 (ξ1 ) . . . emd−1 (ξd−1 ) 2 Φ(ξd ). dξd

Taking into account the fact that Δω = ω and the boundary conditions the function ω should satisfy, we deduce that Φ satisfies (18.5) with a2 = 1 −

d−1 

λ mj ,

α = 1, β = 0.

1

Taking into account the form of the solution to (18.5) we get the formula for Du for the boundary conditions u which form an orthonormal and complete basis in U = L2 (Γ1 ) and the result follows by the linear dependence of the solution on the boundary data.  The following lemma is a special case of a general result due to J.-L. Liones and E. Magenes [66], but we provide a direct proof. Lemma 18.4. For arbitrary α ∈ (0, 34 ) the operator (−A)α D, where D is given by (18.10), is bounded from U = L2 (Γ1 ) into H.

310

18 Boundary control systems

Proof. We will show that for a constant C and arbitrary u ∈ U |(−A)α Du|2 ≤ C|u|2 . For arbitrary h ∈ D(A):  −Ah = (n21 + . . . + n2d )h, en1 ...nd en1 ...nd . n1 ...nd

Moreover, for u ∈ U 

u=

um1 ...md−1 em1 ...md−1 ,

m1 ,...,md−1

we have Du, en1 ...nd  = un1 ...nd−1 Φ(1+n21 +...n2d−1 )1/2 , end . Therefore



|(−A)α Du|2 = =



(n21 + . . . + n2d )2α u2n1 ...nd−1 Φ(1+n21 +...n2d−1 )1/2 , end 2

n1 ...nd

n1 ...nd−1

u2n1 ...nd−1

  (n21 + . . . + n2d )2α Φ(1+n21 +...n2d−1 )1/2 , end 2 . nd

Moreover, by direct computation, if nd ≥ 1: 1 2 )2 , Φ(1+n21 +...n2d−1 )1/2 , end 2 = ( π 1 + n21 + . . . n2d and if nd = 0: 1 Φ(1+n21 +...n2d−1 )1/2 , e0  = π 2



1 1 + n21 + . . . n2d−1

2 .

Thus, for arbitrary n1 , . . . , nd−1 ,   (n21 + . . . + n2d )2α Φ(1+n21 +...n2d−1 )1/2 , end 2 nd

 2 1 2  2 (n1 + . . . + n2d )2α π 1 + n21 + . . . n2d nd ≥1  2 1 1 + (n21 + . . . + n2d−1 )2α π 1 + n21 + . . . n2d−1 1 2  1 ≤ + . 4(1−α) π π m

=

m≥1

Consequently

18.5 Null controllability for a boundary control system

|(−A)α Du|2 ≤



2  1 4(1−α) π m m≥1

311

 1 + |u|2 , π

and the operator (−A)α D is bounded if α < 34 . This implies the result.



Proof of Theorem 18.3. Let A be the operator given by (18.9). One has to show that for arbitrary x ∈ H = L2 (Ω) there exists u(·) ∈ L2 [0, T ; L2 (U )] such that  eAT x = (I − A)

T

eA(T −s) Du(s) ds.

(18.11)

0

To show this we will use the moments method, see §15.4. The functions en1 n2 ...nd , ni ∈ N ∪ {0},

i = 1, 2, . . . , d,

form an orthonormal and complete basis in L2 (Ω), therefore (18.11) is equivalent to the system of equalities  e

AT

x, en1 ...nd  =

0

T

 u(s), D∗ eA(T −s) (I − A)en1 ...nd ds.

(18.12)

If we take into account that en1 ...nd are eigenfunctions of the operator −A corresponding to the eigenvalues n21 + . . . + n2d , the formula (18.12) can be rewritten as follows: 2

2

e−(n1 +...+nd )T x, en1 ...nd   T 2 2 e−(n1 +...+nd )(T −s) (1 + n21 + . . . + n2d )u(s), D∗ en1 ...nd  ds. = 0

It follows from (18.5) and Lemma 18.2 that if nd ≥ 1, n1 , . . . , nd−1 arbitrary, then 1 2 D∗ en1 ...nd = − en1 ...nd−1 , (18.13) 2 π 1 + n1 + . . . + n2d and if nd = 0, n1 , . . . , nd−1 arbitrary, then 1 1 D∗ en1 ...nd = − en1 ...nd−1 . π 1 + n21 + . . . + n2d

(18.14)

Defining ε(0) = − π1 and ε(n) = − π2 for n = 1, 2, . . . we arrive at the equations 2

2

e−(n1 +...+nd )T x, en1 ...nd   T 2 2 1 + n21 + . . . + n2d = e−(n1 +...+nd )(T −s) ε(nd ) u(s), en1 ...nd−1  ds. 1 + n21 + . . . + n2d 0 It is therefore enough to show that there exists a family of functions un1 ...nd−1 (s) = u(s), en1 ...nd−1 , s ∈ [0, T ], such that

312

18 Boundary control systems

 (i)

T

0

|u(s)|2 ds =





T

0



2 un1 ...nd−1 (s) ds < +∞.

n1 ,...,nd−1

(ii) For an arbitrary sequence n1 , . . . , nd−1 the following identities, indexed by n = 0, 1, 2, . . . , are satisfied: 2

2

2

e−n T e−(n1 +...+nd−1 )T x, en1 ...nd−1 n (ε(n))−1  T  −n2 (T −s) −(n2 +...+n2 )(T −s)  d−1 e = e 1 un1 ...nd−1 (s) ds.

(18.15)

0

Thus, for an arbitrary sequence n1 , . . . , nd−1 = 0 of numbers we have to solve simultaneously problems (18.15) and to show that the solutions satisfy (i). Using similar notation as in the section on the moments problem we denote by gn , n = 0, 1 . . ., the biorthogonal sequence, see §15.4, to 2 fn (s) = e−n , s ∈ [0, T ], u = 0, 1, . . . . Then the solution to (18.15) is given by 2

2

e−(n1 +...+nd−1 )(T −s) un1 ...nd−1 (s) =

+∞ 

2

2

2

e−(n1 +...+nd−1 )T e−n

T

x, en1 ...nd−1 n (ε(n))−1 gn (s)

n=0

and 2

2

un1 ...nd−1 (s) = e−(n1 ...nd−1 )(T +s)

+∞ 

2

e−n

T

x, en1 ...nd−1 n (ε(n))−1 gn (s).

n=0

Consequently |un1 ...nd−1 (s)|2 ≤ π 2

+∞ 

2

e−2n

n=0



T

0

+∞  

T 2 gn (s)

|un1 ...nd−1 (s)|2 ds ≤ π 2

+∞ 

x, en1 ...nd−1 n 2

n=0 2

e−2n

T

||gn ||2

+∞  

n=0



x, en1 ...nd−1 n 2



n=0

and therefore  0

T

+∞    2 |un1 ...nd−1 (s)|2 ds ≤ π 2 x 2 e−2n T ||gn ||2 .

 n1 ...nd−1

n=0

It is known, see Lemma 3.1 in [36], that for some constants c1 > 0, c2 > 0 gn = so condition (i) is satisfied.

1 ≤ c1 ec2 n , σn

n = 0, 1, . . . , 

18.5 Null controllability for a boundary control system

313

Bibliographical notes The semigroup approach was started by A.V. Balakrishnan [4] and H.O. Fattorini [34]. The stability and stabilizability problems can be studied using this approach as well and for initial reading see J. Zabczyk [110], also Exercise 5.25 in R.F. Curtain and H. Zwart [23], page 262. The main result on null controllability was proved by H.O. Fattorini in the paper [35]. The moment method is discussed in S.A. Avdonin and S.A. Ivanov [3] in Chapters I, II and VII. Control theory for partial differential equations is comprehensively treated by I. Lasiecka and R. Triggiani [62]. We recommend also books by A. Bensoussan, G. Da Prato, M.C. Delfour and S.K. Mitter [9] and J.-M. Coron [20].

Appendix

§A.1. Measurable spaces A σ-field or σ-algebra in a space E is a nonempty collection E of subsets of E closed under countable unions and intersections as well as under taking complements. The pair (E, E) is called measurable space. If E is a metric space and E is the smallest σ-field containing all closed sets then the elements of E are called Borel sets and E is denoted by B(E). A measure on (E, E) is a function μ : E → R+ such that μ(∅) = 0 and +∞  +∞    μ An = μ(An ) n=1

(A.1)

n=1

for disjoint subsets An of E, n = 1, 2, . . . , belonging to E. A signed measure on (E, E) is a bounded function μ : E → R such that (A.1) holds. The following Jordan decomposition holds: Any signed measure μ can be uniquely written as a difference of two (nonnegative) measures μ+ and μ− : μ(A) = μ+ (A) − μ− (A),

A ∈ E,

which are mutually singular, that is, have disjoint supports. A set B ∈ E is a support of a measure μ if for all A ∈ E μ(A) = μ(A ∩ B).

§A.2. Metric spaces A metric space is a pair composed of a set E and a function  : E×E −→ R+ such that © Springer Nature Switzerland AG 2020 J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6

315

316

Appendix

(x, y) = 0

if and only if

x=y

(x, y) = (y, x) for all x, y ∈ E (x, y) ≤ (x, z) + (z, y), for arbitrary

x, y, z ∈ E.

The function  is called a metric. For any r ≥ 0 and a ∈ E the set B(a, r) = {x ∈ E; (a, x) < r}

(A.2)

is called a ball of radius r and centre a. A union of any family of balls is an open subset of E and the complement Ac of any open set A ⊂ E is called a closed subset of E. The smallest closed set containing a set A ⊂ E is denoted by A¯ or cl A and called the closure of A. If there exists a countable set E0 ⊂ E such that E 0 = E then E is a separable metric space. A metric space (E, ) is complete if for arbitrary sequence (xn ) of elements of E such that lim (xn , xm ) = 0 there exists x ∈ E for which n,m→∞

lim (xn , x) = 0. n→∞ If E1 , E2 are two metric spaces and F : E1 → E2 a transformation such that for any open set Γ2 ⊂ E2 the set F −1 (Γ2 ) = {x ∈ E1 ; F (x) ∈ Γ2 } is open in E1 , then F is called a continuous mapping from E1 onto E2 . A transformation F : E −→ E is called a contraction if there exists c ∈ (0, 1) such that (F (x), F (y)) ≤ c(x, y), for all x, y ∈ E.

(A.3)

Theorem A.1 (The contraction mapping principle) Let E be a complete metric space and F : E −→ E a contraction. Then the equation F (x) = x,

x∈E

(A.4)

has a unique solution x ˆ. Moreover, for arbitrary x0 ∈ E, lim (xn , x ˆ) = 0,

n→∞

where xn+1 = F (xn ), n = 0, 1, . . . .

§A.3. Banach spaces A complete, metric space (E, ) which is also a linear space over R (or C) and such that

A.3 Banach spaces

317

(x, y) = (x − y, 0), x, y ∈ E (αx, 0) = |α|(x, 0), x ∈ E, α ∈ R (α ∈ C) is called a Banach space. The function x = (x, 0),

x∈E

is called the norm of E. An operator A acting from a linear subspace D(A) of a Banach space E onto a Banach space E2 such that A(αx + βy) = αAx + βAy,

x, y ∈ D(A), α, β ∈ R (α, β ∈ C)

is called linear. If D(A) = E1 and the linear operator A is continuous then it is a bounded operator and the number A = sup{ Ax 2 ; x 1 ≤ 1}

(A.5)

is its norm. Here · 1 , · 2 stand for the norms on E1 and E2 , respectively. A linear operator A is closed if the set {(x, Ax); x ∈ D(A)} is a closed subset of E1 × E2 . Equivalently, an operator A is closed if, for an arbitrary sequence of elements (xn ) from D(A) such that for some x ∈ E1 , y ∈ E2 , lim xn − x 1 = 0, lim Axn − y 2 = 0, one has x ∈ D(A) and Ax = y.

n→∞

n

Theorem A.2 (The closed graph theorem) If a closed operator A : E1 −→ E2 is defined everywhere on E1 (D(A) = E1 ), then A is bounded. Theorem A.3 (The inverse mapping theorem) If a bounded operator A : E1 −→ E2 is one-to-one and onto E2 , then the inverse operator A−1 is bounded. Theorem A.4 (The Hahn–Banach theorem) Assume that E0 ⊂ E is a linear subspace of E and p : E −→ R+ a functional such that p(a + b) ≤ p(a) + p(b), a, b ∈ E p(αa) = αp(a), α ≥ 0, a ∈ E. If ϕ : E0 → R is a linear functional such that ϕ0 (a) ≤ p(a), a ∈ E0 , then there exists a linear functional ϕ : E → R such that ϕ(x) = ϕ0 (x),

x ∈ E0

ϕ(x) ≤ p(x),

x ∈ E.

and The space of all linear, continuous functionals on a Banach space E, with values in R (in C) equipped with the norm · ∗ ,

318

Appendix

ϕ ∗ = sup{|ϕ(x)|; x ≤ 1}, is a Banach space over R (or C) denoted by E ∗ and called the dual or adjoint space to E. Assume that A is a linear mapping from a Banach space E1 onto a Banach space E2 with the dense domain D(A). Denote by D(A∗ ) the set of all elements ϕ ∈ E2∗ such that the linear functional x ∈ D(A) −→ ϕ(Ax) has a continuous extension Ψ to E1 . Then D(A∗ ) is a linear subspace of E2∗ and the extension Ψ is unique and denoted by A∗ ϕ. The operator A∗ with the domain D(A∗ ) is called the adjoint operator of A. If E is a Hilbert space and A = A∗ then A is called self-adjoint. A selfadjoint operator A is nonnegative if Ax, x ≥ 0 for x ∈ E. If for a constant c > 0 and all x ∈ E, Ax, x ≥ c|x|2 , then A is called positive. If a bounded operator A from a Hilbert space E onto E is such that |Ax| = |x| for all x ∈ E, then A is called a unitary operator. A bounded operator A is unitary if and only if A∗ = A−1 . Theorem A.5 (The Banach–Steinhaus theorem) (i) If (An ) is a sequence of bounded operators from a Banach space E1 onto E2 such that sup An x 2 < +∞ n

for all x ∈ E1

then sup An < +∞. n

(ii) If, in addition, sequence (An x) is convergent for all x ∈ E0 , where E0 is a dense subset of E, then there exists the limit lim An x = Ax,

n→∞

for all x ∈ E and A is a bounded operator. Let X and Z be Banach spaces and F : U → Z a transformation from an open set U ⊂ X into Z. The Gateaux derivative of F at x0 ∈ U is a linear bounded operator A from X into Z such that for arbitrary h ∈ X 1 lim (F (x0 + th) − F (x0 )) = Ah. t

t→0

If, in addition, lim

h→0

F (x0 + h) − F (x0 ) − Ah = 0, h

then F is called Fréchet differentiable at x0 and A the Fréchet derivative of F at x0 denoted by Fx (x0 ) or dF (x0 ). The vector Ah = Fx (x0 )h = dF (x0 ; h) is the directional derivative of F at x0 and at the direction h ∈ X. Theorem A.6 (The mean value theorem) If a continuous mapping F : U −→ Z has the Gateaux derivative Fx (z) at an arbitrary point z in the

A.4 Hilbert spaces

319

open set U , then F (b) − F (a) − Fx (a)(b − a) ≤

(A.6)

sup Fx (η) − Fx (a) b − a ,

η∈I(a,b)

where I(a, b) = {a + s(b − a); s ∈ [0, 1]} is the interval with the endpoints a, b.

§A.4. Hilbert spaces If E is a Banach space over R (or C) equipped with the transformation

·, · : E × E −→ R (or, C) such that

a, b = b, a,

(= b, a)

a, a ≥ 0 and a, a = 0 if and only if a = 0

αa + βb, c = α a, c + β b, c for a, b ∈ E and scalars α, β a = a, a1/2 , a ∈ E then E is called a Hilbert space and the form ·, · is the scalar product on E. Theorem A.7 (The Riesz representation theorem) If E is a Hilbert space then for arbitrary ϕ ∈ E ∗ there exists exactly one element a ∈ E such that ϕ(x) = x, a,

x ∈ E.

(A.7)

Moreover, ϕ ∗ = a . This way E ∗ can be identified with E. Norms on Hilbert spaces are also denoted by | · |. Typical examples of Hilbert spaces are l2 and lC2 . The space l2 consists of +∞  |ξn |2 < +∞ and the scalar product is all real sequences (ξn ) such that n=1

defined by

(ξn ), (ηn ) =



ξn ηn ,

(ξn ), (ηn ) ∈ l2 .

n

The space lC2 consists of all complex sequences (ξn ) such that and the scalar product is given by

(ξn ), (ηn ) =

+∞  n=1

ξn η¯n ,

(ξn ), (ηn ) ∈ lC2 .

+∞  n=1

|ξn |2 < +∞

320

Appendix

A sequence of elements (en ) in a Hilbert space E is called an orthonormal basis if |en | = 1, en , em  = 0 for m, n = 1, 2, . . . , m = n. A basis (en ) is complete if their linear combinations are dense in E. Then for any x |x|2 =

+∞ 

| x, en |2

and

x=

n=1

+∞ 

x, en en .

n=1

§A.5. Bochner’s integral Let E be a metric space. The smallest family E of subsets of a metric space E such that (i) all open sets are in E, (ii) if Γ ∈ E then Γc ∈ E, +∞  Γn ∈ E, (iii) if Γ1 , Γ2 , . . . ∈ E then n=1

is called the Borel σ-field of E and is denoted by B(E). Elements of B(E) are called Borel sets of E. A transformation F : E1 −→ E2 is called Borel if for arbitrary Γ2 ∈ B(E2 ), F −1 (Γ2 ) ∈ B(F1 ). If E2 = R then F is called a Borel function. Continuous mappings are Borel. Assume that E is a separable Banach space and let E1 = (α, β) ⊂ (−∞, +∞) and E2 = E. If f : (α, β) −→ E is a Borel transformation then also function F (t) , t ∈ (α, β), is Borel measurable. The transformation F is called Bochner integrable if 

β

α

F (t) dt < +∞,

where the integral is the usual Lebesgue integral of the real-valued function F (·) . Assume, in addition, that F is a simple transformation, i.e. there m  Γj = (α, β) and vectors exist Borel, disjoint subsets Γ1 , . . . , Γm of (α, β), j=1

a1 , . . . , am such that F (t) = ai ,

t ∈ Γi .

Then the sum m  j=1

 aj

β

α

χΓj (t) dt

is well defined and by definition equal to the Bochner integral of f over (α, β) It is denoted by

A.5 Bochner’s integral

321



β

F (t) dt.

α

Note that for simple transformations F 

β

α

 F (t) dt ≤

β

F (t) dt.

α

(A.8)

Let {en } be a dense and a countable subset of E and F a Borel mapping from (α, β) into E. Define γm (t) = min{ f (t) − ek ; k = 1, . . . , m}, km (t) = min{k ≤ m; γm (t) = f (t) − ek } and Fm (t) = ekm (t) , t ∈ (α, β), m = 1, 2, . . . . Then Fm is a Borel, simple transformation and Fm (t) − F (t) , t ∈ (α, β), decreases monotonically to 0 as m → +∞. In addition, Fm (t) − F (t) ≤ e1 = F (t) ≤ e1 + F (t) ,

t ∈ (α, β), m = 1, 2, . . . .

Consequently, if F is Bochner’s integrable  β  β lim Fm (t) dt − Fn (t) dt n,m→∞ α α  β  ≤ lim Fm (t) − F (t) dt + lim m→∞

n→∞

α

and therefore there exists the limit  lim n→∞

β

α

β

α

Fn (t) − F (t) dt = 0,

Fn (t) dt,

β which is also denoted by α F (t) dt and called the Bochner integral of F . The estimate (A.8) holds for arbitrary Bochner integrable functions. Moreover, if (Gm ) is any sequence of Borel transformations such that  β lim Gm (t) − F (t) dt = 0 m→∞

α

then  lim

m→∞

β

α

 Gm (t) dt −

β

α

F (t) dt = 0.

One can easily show, using Theorem A.9, that for an arbitrary Bochner integrable transformation F (·) there exists a sequence of continuous transformations (Gm ) such that

322

Appendix

 lim

m→∞

β

α

Gm (t) − F (t) dt = 0.

§A.6. Spaces of continuous functions Let −∞ < α < β < +∞ and E be a Banach space. The space of all continuous functions from [α, β] into E with the norm f = sup{ f (t) ; t ∈ [α, β]} is a Banach space, denoted by C(α, β; E). If the space E is separable, the space C(α, β; E) is separable as well. We have also the following result: Theorem A.8 (The Ascoli theorem) Let K be a closed subset of C(α, β; Rn ). Every sequence of elements from K contains a convergent subsequence if and only if (i) sup{ f (t) ; t ∈ [α, β], f ∈ K} < +∞ (ii) for arbitrary ε > 0 there exists δ > 0 such that f (t) − f (s) ≤ ε

for all f ∈ K

and all t, s ∈ [α, β] such that |t − s| < δ.

§A.7. Spaces of measurable functions Let p ≥ 1. The space of all equivalence classes of Bochner’s integrable functions f from (α, β) ⊂ (−∞, +∞) into E such that 

β

α

f (t) p dt < +∞

is denoted by Lp (α, β; E). It is a Banach space equipped with the norm f | =



β

f (t) p dt

α

1/p

.

If E is a Hilbert space and p = 2 then L2 (α, β; E) is a Hilbert space as well and the scalar product

·, · is given by 

f, g =

β

α

f (t), g(t)E dt.

If E = R or E = C we write Lp (α, β) for short.

A.7 Spaces of measurable functions

323

Theorem A.9 (Young’s inequality) If 1 ≤ p, q < +∞, 1r = p1 + 1q − 1 and f ∈ Lp (0, +∞), g ∈ Lq (0, +∞), then f ∗ g ∈ Lr (0, +∞) (f ∗ g(t) =

t f (t − s)g(s) ds, t ≥ 0) and 0  0

+∞

r

|f ∗ g(t)| dt

1/r

≤



+∞

0

p

|f (t)| dt

1/p 

+∞

0

|g(t)|q dt

1/q

.

Theorem A.10. If f ∈ Lp (−∞, +∞) then  lim

δ→0

+∞

−∞

|f (t + δ) − f (t)| dt = 0.

Theorem A.11 (Luzin’s theorem) If f : [α, β] → R is a Borel function then for arbitrary ε > 0 there exists a closed set Γ ⊂ [α, β] such that the Lebesgue measure of [α, β] \ Γ is smaller than ε and f restricted to Γ is a continuous function. A function f : [α, β] → Rn is called absolutely continuous if for arbitrary ε > 0 one can find δ > 0 such that if α ≤ t0 < t1 < t2 < . . . < t2k < t2k+1 ≤ β and k 

(t2j+1 − tj ) < δ

j=0

then k 

|f (t2j+1 ) − f (tj )| < ε.

j=0

A function f : [α, β] → Rn is absolutely continuous if and only if there exists Ψ ∈ L1 (α, β; Rn ) such that  f (t) = f (α) +

t

α

Ψ(s) ds,

t ∈ [α, β].

Moreover, f is differentiable at almost all t ∈ [α, β], and df (t) = Ψ(t). dt

References

[1] D. Aeyels, M. Szafranski, Comments on the stability of the angular velocity of a rigid body. Syst. Control Lett. 10, 35–39 (1988) [2] V.I. Arnold, Mathematical Methods of Classical Mechanics (Springer, New York, 1978) [3] S.A. Avdonin, S.A. Ivanov, Families of Exponentials. The Method of Moments in Controllability Problems for Distributed Parameter Systems (Cambridge University Press, Cambridge, 1995) [4] A.V. Balakrishnan, Semigroup theory and control theory, in Proceedings of IFIP Congress, Tokyo (1965) [5] J. Ball, Strongly continuous semigroups, weak solutions and the variation of constant formula. Proc. Am. Math. Soc. 63, 370–373 (1977) [6] S. Barnett, Introduction to Mathematical Control Theory (Clarendon Press, Oxford, 1975) [7] R. Bellman, Dynamic Programming (Princeton University Press, Upper Sadddle River, 1977) [8] A. Bensoussan, J.-L. Lions, Contrôle impulsionnel et inéquations quasivariationnelles (Dunod, Paris, 1982) [9] A. Bensoussan, G. Da Prato, M.C. Delfour, S.K. Mitter, Representation and Control of Infinite Dimensional Systems, Vol. I–II, Systems & Control: Foundations & Applications (Birkhäuser, Boston, MA, 1992– 1993) [10] N.P. Bhatia, G.P. Szegö, Stability Theory of Dynamical Systems (Springer, New York, 1970) [11] A. Blaquiere, Impulsive optimal control with finite or infinite time horizon. J. Optim. Theory Appl. 46, 431–439 (1985) [12] R.W. Brockett, Finite Dimensional Linear Systems (Wiley, New York, 1970) [13] R.W. Brockett, Asymptotic stability and feedback stabilization, in Differential Geometric Control Theory, ed. by R.W. Brockett, R.S. Millman, H.J. Sussmann (Basel, Birkhäuser, 1983), pp. 181–191 © Springer Nature Switzerland AG 2020 J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6

324

References

325

[14] A.G. Butkovski, Teoriya upravleniya sistemami s raspredelennymi parametrami (Nauka, Moscow, 1975) [15] L. Cesari, Optimization Theory and Applications (Springer, New York, 1963) [16] F.H. Clarke, Optimization and Nonsmooth Analysis (Wiley Interscience, New York, 1983) [17] F.H. Clarke, Y.S. Ledyaev, R.J. Stern, P.R. Wolenski, Nonsmooth Analysis and Control Theory, vol. 178, Graduate Texts in Mathematics (Springer, New York, 1998) [18] E.A. Coddington, N. Levinson, Theory of Ordinary Differential Equations (McGraw-Hill, New York, 1955) [19] Y. Cohen (ed.), Applications of Control Theory in Ecology, Lecture Notes in Biomathematics (Springer, New York, 1988) [20] J.-M. Coron, Control and Nonlinearity, Mathematical Surveys and Monographs 136 (American Mathematical Society, Providence, 2007) [21] M.G. Crandall, P.-L. Lions, Viscosity solutions of Hamilton-Jacobi equations. Trans. Am. Math. Soc. 277, 1–42 (1983) [22] R.F. Curtain, A.J. Pritchard, Infinite Dimensional Linear Systems Theory, Lecture Notes in Control and Information Sciences (Springer, New York, 1978) [23] R.F. Curtain, H. Zwart, An Introduction to Infinite-Dimensional Linear System Theory (Springer, Berlin, 1995) [24] G. Da Prato, Quelques résultats d’existence et régularité pour un problème non linéaire de la théorie du controle. J. Math. Pures et Appl. 52, 353–375 (1973) [25] R. Datko, Extending a theorem of A.M. Lapunov to Hilbert spaces. J. Math. Anal. Appl. 32, 610–616 (1970) [26] R. Datko, Uniform asymptotic stability of evolutionary processes in a Banach space. SIAM J. Math. Anal. 3, 428–445 (1972) [27] M.C. Delfour, S.K. Mitter, Controllability, observability and optimal feedback control of affine hereditary differential systems. SIAM J. Control 10, 298–328 (1972) [28] S. Dolecki, D.L. Russell, A general theory of observation and control. SIAM J. Control 15, 185–221 (1977) [29] R. Douglas, On majorization, factorization and range inclusion of operators in Hilbert spaces. Proc. Am. Math. Soc. 17, 413–415 (1966) [30] J. Dugundji, Topology (Allyn and Bacon, Boston, 1966) [31] N. Dunford, J. Schwartz, Linear Operators, Part I (Interscience Publishers, New York, 1958) [32] N. Dunford, J. Schwartz, Linear Operators, Part II (Interscience Publishers, New York, 1963) [33] H.O. Fattorini, Some remarks on complete controllability. SIAM J. Control 4, 686–694 (1966) [34] H.O. Fattorini, Boundary control systems. SIAM J. Control 6, 349–385 (1966)

326

References

[35] H.O. Fattorini, Boundary control of temperature distributions in a parallellopipedon. SIAM J. Control 13, 1–13 (1975) [36] H.O. Fattorini, D.L. Russell, Exact controllability theorems for linear parabolic equations in one space dimension. Arch. Ration. Mech. Anal. 43, 272–292 (1971) [37] W.H. Fleming, R.W. Rishel, Deterministic and Stochastic Optimal Control (Springer, Berlin, 1975) [38] W.H. Fleming, H.M. Soner, Controlled Markov Processes and Viscosity Solutions, vol. 25, 2nd edn., Stochastic Modelling and Applied Probability (Springer, New York, 2006) [39] A.F. Filippov, On certain questions in the theory of optimal regulation. Vestn. Mosk. Univ. Ser. Mat. Mekh. Astr. Fiz. Khim., 25–32 (1959) (translation in J. SIAM Control. Ser A, 1, 76–84 (1962)) [40] F.R. Gantmacher, Applications of the Theory of Matrices (Interscience Publishers Inc., New York, 1959) [41] I.V. Girsanov, Lectures on Mathematical Theory of Extremum Problems (Springer, New York, 1972) [42] R. Gupta, F. Jafari, R.J. Kipka, B.S. Mordukhovich, Linear openness and feedback stabilization of nonlinear control systems. Discret. Contin. Dyn. Syst. Ser. S 11, 1103–1119 (2018) [43] M.L.J. Hautus, Controllability and observability conditions of linear autonomous systems. Nederl. Akad. Wetenschappen Proc. Ser. A 72, 443–448 (1969) [44] M.L.J. Hautus, Stabilization, controllability, and observability of linear autonomous systems. Nederl. Akad. Wetenschappen Proc. Ser. A 73, 448–455 (1970) [45] N.D. Hayes, Roots of the transcendental equation associated with a certain difference-differential equation. J. Lond. Math. Soc. 25, 226– 232 (1950) [46] R. Herman, A. Krener, Nonlinear controllability and observability. IEEE Trans. Autom. Control AC-22, 728–740 (1977) [47] A. Ichikawa, Null controllability with a vanishing energy for discretetime systems in Hilbert spaces. SIAM J. Control Optim. 46, 683–693 (2007) [48] A. Ichikawa, Null controllability with vanishing energy for discrete-time systems. Control Lett. 57, 34–38 (2008) [49] A. Ichikawa, Minimum energy control and its applications. Int. J. Pure Appl. Math. 42, 475–480 (2008) [50] S. Ivanov, Control norms for large control times, ESAIM: Control. Optim. Calc. Var. 4, 405–418 (1999) [51] A. Isidori, Nonlinear Control Systems: An Introduction, vol. 72, Lecture Notes in Control and Information Sciences (Springer, New York, 1985) [52] B. Jakubczyk, Local realization theory for nonlinear systems (Wroclaw, In Geometric Theory of nonlinear control systems, Polytechnic of Wroclaw, 1985), pp. 83–104

References

327

[53] M. Kalecki, Macrodynamic theory of business cycles. Econometrica 3, 327–344 (1935) [54] M. Kalecki, Theory of Economic Dynamics (George Allen, London, 1954) [55] R.E. Kalman, On the general theory of control systems, Automatic and Remote Control, in Proceedings of First International Congress of IFAC, vol. 1, Moscow (1960), pp. 481–492 [56] R.E. Kalman, Mathematical description of linear dynamical systems. SIAM J. Control Optim. 1, 152–192 (1963) [57] J. Kisynski, Semigroups of operators and some of their applications to partial differential equations, in Control Theory and Topics in Functional Analysis, vol. 3 (IAEA, Vienna, 1976), pp. 305–405 [58] A.W. Knapp, Basic Algebra (Birkhauser, Boston, Basel, Berlin, 2006) [59] M.A. Krasnoselski, P.P. Zabreiko, Geometrical Methods of Nonlinear Analysis (Springer, Berlin, New York, 1984) [60] A.J. Krener, A generalization of Chow’s theorem and the bang-bang theorem to nonlinear control systems. SIAM J. Control 12, 43–52 (1974) [61] P. Kriesler, J.W. Nevile, Dynamic Keynesian economics: cycling forward with Harrod and Kalecki. Camb. J. Econ. 36, 405–417 (2012) [62] I. Lasiecka, R. Triggiani, Control Theory for Partial Differential Equations: Continuous and Approximation Theories, vol. I–II, Encyclopedia of Mathematics and its Applications (Cambridge University Press, Cambridge, 2000), pp. 74–75 [63] E.B. Lee, L. Markus, Foundations of Optimal Control Theory (Wiley, New York, 1967) [64] G. Leitman, An Introduction to Optimal Control (Mc Graw-Hill, New York, 1966) [65] J.-L. Lions, Contrôle optimale de systèmes gouvernés par des équations aux dérivées partielles (Dunod, Paris, 1968) [66] J.-L. Lions, E. Magenes, Problèmes aux limites non homogènes (Dunod, Paris, 1968) [67] P.-L. Lions, Generalized Solutions of Hamilton-Jacobi Equations, Pitman Research Notes in Mathematics, vol. 69 (Pitman, Boston, 1982) [68] A. Manitius, R. Triggiani, Function space controllability of linear retarded systems: a derivation from abstract operator conditions. SIAM J. Control Optim. 16, 599–645 (1978) [69] J.C. Maxwell, On Governers (Royal Society, Proc, 1868) [70] M. Megan, On the stabilizability and controllability of linear dissipative systems in Hilbert space, S. E. F., Universitatea din Timisoara, 32 (1975) [71] B.S. Mordukhovich, Variational Analysis and Generalized Differentiation. II: Applications, Grundlehren der Mathematischen Wissenschaften, vol. 331 (Springer, Berlin, 2006)

328

References

[72] C. Olech, Existence Theory in Optimal Control, in Control Theory and Topics in Functional Analysis, vol. 1 (IAEA, 1976), pp. 291–328 [73] R. Pallu de la Barriere, Cours d’automatique théorique (Dunod, Paris, 1966) [74] L. Pandolfi, E. Priola, J. Zabczyk, Linear operator inequality and null controllability with vanishing energy for unbounded control systems. SIAM J. Control Optim. 51, 629–659 (2013) [75] P.C. Parks, A new proof of the Routh-Hurwitz stability criterion using the second method of Lyapunov. Proc. Cambridge Philos. Soc. 58, 694– 702 (1962) [76] A. Pazy, Semigroups of Linear Operators and Applications to Partial Differential Equations (Springer, New York, 1983) [77] L.S. Pontryagin, V.G. Boltyanskii, R.V. Gamkrelidze, E.F. Mishchenko, Matematicheskaya teoriya optimal’nykh processov (Nauka, Moscow, 1969) [78] L.S. Pontryagin, Ordinary Differential Equations (Addison-Wesley, Mass, 1962) [79] V.M. Popov, Hyperstability of Control Systems, Grundlehren der Mathematischen Wissenschaften 204 (Springer, Berlin, 1973) [80] E. Priola, J. Zabczyk, Null controllability with vanishing energy. SIAM J. Control Optim. 42, 1013–1032 (2003) [81] E. Priola, J. Zabczyk, Liouville theorems for non-local operators. J. Functl. Anal. 216, 455–490 (2004) [82] A. Pritchard, J. Zabczyk, Stability and stabilizability of infinite dimensional systems. SIAM Rev. 23, 25–52 (1981) [83] J.E. Prussing, B.A. Conway, Orbital Mechanics (Oxford University Press, New York, 1993) [84] R. Rempala, Impulsive control problems, in Proceedings of the 14th IFIP Conference on System Modeling and Optimization (Springer, New York, Leipzig, 1989) [85] R. Rempala, J. Zabczyk, On the maximum principle for deterministic impulse control problems. J. Optim. Theory Appl. 59, 281–288 (1988) [86] S. Rolewicz, Functional analysis and control theory, Polish Scientific Publisher, Warszawa, and D (Reidel Publishing Company, Dordrecht, Boston, Lancaster, Tokyo, 1987) [87] E.J. Routh, Treatise on the Stability of a Given State of Motion (Macmillan and Co., London, 1877) [88] D.L. Russell, Controllability and stabilizability theory for linear partial differential equations: recent progress and open questions. SIAM Rev. 20, 639–739 (1978) [89] T. Schanbacher, Aspects of positivity in control theory. SIAM J. Control Optim. 27, 457–475 (1989) [90] M. Shibata, A. Ichikawa, Orbital Rendezvous and Flyaround based on null controllability with vanishing energy. J. Guid. Control Dyn. 30(4). Accessed July–August 2007

References

329

[91] H.J. Sussmann, V. Jurdjevic, Controllability of nonlinear systems. J. Differ. Equ. 12, 95–116 (1972) [92] M. Szafranski, Stabilization of Euler’s equation, University of Warsaw, Dept. of Mathematics, Informatics and Mechanics, Master Thesis (1987), in Polish [93] R. Triggiani, On the stabilization problem in Banach spaces. J. Math. Anal. Appl. 52, 383–403 (1975) [94] R. Triggiani, Constructive steering control functions for linear systems and abstract rank condition. J. Optim. Theory Appl. 74, 347–367 (1992) [95] J. Van Neerven, Null controllability and the algebraic Riccati equation in Banach spaces. SIAM J. Control Optim. 43, 1313–1327 (2005) [96] R. Vinter, Optimal Control (Birkhäuser, Boston, 2010) [97] T. Wazewski, Systèmes de commande et équations au contigent. Bull. Acad. Pol. Sc. 9, 151–155 (1961) [98] N. Wiener, Cybernetics or Control and Communication in the Animal and the Machine (The MIT Press, Cambridge, 1948) [99] W.A. Wolvich, Linear Multivariable Systems (Springer, New York, 1974) [100] W.M. Wonham, On pole assignment in multi-input controllable linear systems. IEEE Trans. Autom. Control AC-12, 660–665 (1967) [101] W.M. Wonham, Linear Multivariable Control: A Geometric Approach (Springer, New York, 1979) [102] J. Yong, X.Y. Zhou, Stochastic Controls. Hamiltonian Systems and HJB Equations, vol. 43, Applications of Mathematics (Springer, New York, 1999) [103] J. Zabczyk, Remarks on the control of discrete-time distributed parameter systems. SIAM J. Control 12, 161–176 (1974) [104] J. Zabczyk, On optimal stochastic control of discrete-time parameter systems in Hilbert spaces. SIAM J. Control 13, 1217–1234 (1975) [105] J. Zabczyk, A note on C0 -semigroups. Bull. Acad. Pol. Sc. Ser. Math. 23, 895–898 (1975) [106] J. Zabczyk, Remarks on the algebraic Riccati equation in Hilbert space. Appl. Math. Optim. 3, 383–403 (1976) [107] J. Zabczyk, Complete stabilizability implies exact controllability, in Seminarul Ecuati Functionale, vol. 38 (Romania, Timisoara, 1976), pp. 1–7 [108] J. Zabczyk, Stability properties of the discrete Riccati operator equation. Kybernetika 13, 1–10 (1977) [109] J. Zabczyk, Infinite dimensional systems in optimal control, Bulletin of the International Statistical Institute, vol. XLII, Book II, Invited papers (1977), pp. 286–310

330

References

[110] J. Zabczyk, Stabilization of boundary control systems, in International Symposium on Systems Optimization and Analysis, Springer Lecture Notes in Control and Information Sciences, , vol. 14 ed. by A. Bensoussan, J.L. Lions (1978), pp. 321–322 [111] J. Zabczyk, On decomposition of generators. SIAM J. Control Optim. 16, 523–534 (1978) [112] J. Zabczyk, Erratum: on decomposition of generators. SIAM J. Control Optim. 18, 325 (1980) [113] J. Zabczyk, Stopping problems in stochastic control, in Proceedings of the International Congress of Mathematicians, Warsaw (1984), pp. 1425–1437 [114] J. Zabczyk, Some comments on stabilizability. Appl. Math. Optim. 19, 1–9 (1988) [115] J. Zabczyk, A note on the paper “Stability and stabilizability of infinite dimensional systems” by A.J. Pritchard and J. Zabczyk, SIAM Review, 23(1981), 25–52. Preprint of the Institute of Mathematics Polish Academy of Sciences 740(2012), 1–5

Notation

R: The set of all real numbers. R+ : The set of all nonnegative real numbers. C: The set of all complex numbers.

En: M(n, m): M(n, m, C): l2 :

The Cartesian product of n copies of E. The set of all n × m matrices with real elements. The set of all n × m matrices with complex elements.  Hilbert space of all real sequences (ξn ) satisfying |ξn |2 < +∞ with the scalar product  ξn ηn . (ξn ), (ηn ) =

n

n

2 : Hilbert space of all complex sequences (ξ ) satisfying lC n  |ξn |2 < +∞ with the scalar product n

(ξn ), (ηn ) =



ξn η n .

n

χΓ : Indicator function of the set Γ : χΓ (x) = 1 if x ∈ Γ , χΓ (x) = 0 if x ∈ Γ c . Lp (α, β; E): The space of all E-valued, Borel functions integrable in power p on the interval (α, β). L(E, F ): The space of all linear bounded operators from E into F . C0∞ (α, β): The space of all infinite differentiable real functions with compact supports included in (α, b). A, a: Norm of an operator A, norm of an element a. |a|: Norm of an element of a Hilbert space. a, b: Scalar product of elements a, b. Q1/2 : If Q is a nonnegative operator then Q1/2 is the unique nonnegative operator such that S 2 = Q.

© Springer Nature Switzerland AG 2020 J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6

331

Index

Symbols C0 -semigroup of operators, 204 A adjoint equation, 5, 192 adjoint operator, 250 adjoint space, 250 antipodal mapping, 114 B basis complete, 205, 320 orthonormal, 205, 320 Bellman equation, 138, 140, 154 for impulse control infinite time, 169 biorthogonal, 260 Bochner’s integral, 320 Borel set, 315 boundary control, xxv boundary system, 301 C canonical representation, 21 characteristic polynomial, 10, 22 coincidence set, 170 complex Banach space, 204 complexification, 215 contraction mapping principle, 316 control, xvii admissible, 196 closed loop, xvii elementary, 90 impulse, 167 open loop, xvii optimal, xix, 137

control parameters, 138 control system completely stabilizable, 280 exponentially stabilizable, 277 infinite-dimensional, xix, 203 null controllable, 278 controllability, xviii approximate, 255 exact, 255 local, 83 null, 44 positive, 58 rank condition, 10 wave equation, 261 with vanishing energy, 43 controls admissible, 137 closed loop, 283 convex cone, 170 cost functional, 137 D delay controlled system, 234 delay equation existence of solutions, 235 Kalecki, xxv semigroup approach, 240 stability, 271 delayed model, xxv detectable exponentially, 295 differential equation invariant set, 108 nonlinear, 79 solution, 79 dissipative mapping, 105 domain of adjoint operator, 217

© Springer Nature Switzerland AG 2020 J. Zabczyk, Mathematical Control Theory, Systems & Control: Foundations & Applications, https://doi.org/10.1007/978-3-030-44778-6

332

Index domain of generator, 205 dubling method, 161 dynamic programming principle, 155 E electrical filter, xxii electrically heated oven, xix embedding, 138 equation closed loop, 140 differential, 79 linear, 3 nonlinear, 79 Hill’s, xxiii Liénard, 113 Lyapunov, 31 matrix Riccati existence, 143 of observation, xviii Riccati, 143 algebraic, 146 minimal solution, 146 uniqueness, 150 equilibrium state, 105 Euler equation, xxi F feedback, xviii, 117 in Watt’s regulator, xxii stabilizing, xviii, 117 asymptotically, 117 exponentially, 117 Fréchet derivative, 318 function absolutely continuous, 323 multivalued, 140 strongly continuous, 284 fundamental soluion, 204 G Gateaux derivative, 318 generator, 205 exponentially stable, 269 Gronwall lemma, 97 group of unitary transformations, 207 H hamiltonian, 154, 179 Hautus lemma, 37 heated oven, 13 heating a rod, xxiv heating system approximate controllability, 306

333 controlling heat flow, 303 controlling temperature, 303 multidimensional, 306 null controllable, 306 homotopic mapping, 113 hyperbolic system, 228 abstract, 273 stability, 273 I image of an operator, 250 implicit function theorem, 81 impulse control maximum principle, 182 impulse response function, 65 impulse strategy, 167 impulse-response function of linearization, 128 index, 114 inequalities differential, 97 integral, 97 input harmonic, 67 impulse, 66 periodic, 67 input–output map, xix, 65, 69, 127 partial realizations, 130 realizations, 129 inputs, xviii inverse of an operator, 252 J Jordan decomposition, 315 K Kalman rank condition, 10 kernel of an operator, 250 L left-shift semigroup, 221 lemma Hautus, 37 Lie bracket, 86 limit set, 111 linear control system, 229 linear regulator problem, 143 linear semigroup, 204 linearization, 107 Lipschitz condition, 111 Lyapunov equation, 272 Lyapunov function, 108 Lyapunov’s method first, 107 second, 125

334 M mapping concave, 171 monotonic, 171 matrix characteristic polynomial, 10 controllability, 7 eigenvalue, 21 eigenvector, 22 generalized eigenvector, 22 Jordan block, 22 Lyapunov equation, 31 nonnegative, 4 observability, 18 positive, 4 spectrum, 22 stable, 21 maximal interval of existence, 80 maximal solution, 80 maximum principle, 177 regulator problem, 180 time-optimal problem, 189 measurable selectors, 198 measure, 315 signed, 315 moment method, 260, 313 N needle variation, 178 Neumann boundary conditions, 228 nonlinear differential equation existence of solution, 80 uniqueness of solution, 80 nonlinear system control, 79 linearization, 84 local controllability, 85 observable, 94 output, 79 response, 79 strategy, 79 norm, 204 O observability, xviii, 18 dynamical, 39 operator adjoint, 318 bounded, 317 closed, 208, 317 controllability, 252 infinitesimal, 205 Kolmogorov, 53 linear, 317

Index nonnegative, 318 positive, 318 pseudoinverse, 252 self-adjoint, 218, 318 optimal landing, 49 optimal consumption, xxiv optimality, xix optimality criteria, 137 orbit, 111 output periodic component, 67 outputs, xviii P pair controllable, 9, 150 detectable, 39, 148 equivalent, 14 observable, 18, 150 positive stationary, 62 stabilizable, 34, 148 pair of operators exponentially detectable, 295 exponentially stabilizable, 295 partial realization, 131 performance functional, 183 polynomials stable, 25, 26 problem of minimum fuel, xxiii orbital transfer, xxiii, xxvi, 49 time-optimal, 189 R real block, 22 realization, xix, 65 of an impulse response function, 73 regulator, 137 general, 145 regulator problem delay systems, 291 finite horizon, 287 infinite horizon, 293 linear, 284 solution, 180 with infinite horizon, 284 resolvent, 214 resolvent set, 214 Riccati equation algebraic, 293 operator, 285 operator algebraic, 293 solution, 285

Index rigid body, xx robustness index, 120 Routh sequence, 31 S selector, 140 semigroup adjoint, 210 exponentially stable, 269 semigroup of operators, 204 set of control parameters, xvii soft landing, xxiii solution fundamental, 5 local, 80 minimal, 146 strong, 230 viscosity, 154 weak, 246 space Banach, 317 Hilbert, 319 reachable, 259 reflexive, 252 spacecraft, xxiii chaser, 49 target, 49 spectrum, 214 stability asymptotic, 110 criteria, 26 exponential, 106 Lyapunov sense, 108 test, 100 stabilizability, xviii complete, 35 exponent, 117 exponential, 117 output feedback, 39 stabilization elastic structure, xxv Euler equations, 122 state asymptotically stable, 110 attainable, 7, 9, 255 exponentially stable, 105 reachable, xviii, 7, 9 step method, 234 stopping optimal, 169 strategies, xviii surface, 87 parametrization, 87

335 system boundary control, 299 completely stabilizable, 35 control, 6 controllable, xviii, 9 vanishing energy, 43 controllable part, 17 CVE, 43 decomposition, 16 delay, 234 detectable, 39 equivalent, 14 exactly controllable, 255 feedback for NCVE, 48 hyperbolic, 299 input, 6 Kalman decomposition, 16 linear, 3 local controllable, 83 NCVE, 44 nonlinear, 79 null controllable, 255 vanishing energy, 44 observable, xviii, 18 output, 7 parabolic, 299 partially observable, xviii positive, 57 positively controllable, 58 second order, 228 semigroup approach, 240 stabilizable, 34 stable, 21 strategy, 6 symmetric, 92 uncontrollable part, 17 with self-adjoint generators, 256 systems on manifolds, xix T tangent space, 87 theorem Ascoli, 322 Banach–Steinhaus, 318 Brockett, 125 Cayley–Hamilton, 10 closed graph, 317 Datko, 281 exponential stabilizability, 118 Filippov, 196 Hahn–Banach, 317 Hille–Yosida, 212 Jordan, 101

336 Kalman, 10 La Salle, 112 Lax–Milgram, 219 Lions, 218 Liouville, 52 Luzin, 323 mean value, 318 Phillips, 215 regular dependence, 81 Riesz, 319 Routh, 27, 28 separation, 187 Wonham, 35 time-optimal problem minimal time, 192 transfer function, 75 Transfer problem, xxiii, 49 transformation M , 169 convex, 170

Index V value function, 138, 140 regularity, 153, 157 vector field, 86 degree, 113 of class C k , 86 vehicle chase, 51 virtual, 51 viscosity solution existence, 158 uniqueness, 160 viscosity subsolution, 155, 158 viscosity supersolution, 155, 159 W Watt’s governor, xxi weak convergence, 190 weak solution, 232