Dynamic Optimization of Path-Constrained Switched Systems 3031234278, 9783031234279

Table of contents :
Acknowledgements
Contents
1 Introduction
1.1 Motivation
1.2 Dynamic Optimization: Basic Results and Solution Methods
1.2.1 Mathematical Formulation
1.2.2 First-Order Necessary Conditions
1.2.3 Numerical Solution Methods
1.3 Dynamic Optimization Problems with Path Constraints
1.3.1 Mathematical Formulation
1.3.2 CVP-Based Methods for Path Constraints
1.4 Dynamic Optimization of Switched Systems
1.4.1 Background on Hybrid Systems
1.4.2 Dynamic Optimization of Time-Dependent Switched Systems
1.5 Summary
References
2 Bi-level Dynamic Optimization of Path-Constrained Switched Systems
2.1 Definitions and Preliminary Results
2.1.1 Differentiability of Functionals in Normed Spaces
2.1.2 Perturbation Analysis of Constrained Optimization Problems
2.1.3 Semi-infinite Program
2.2 Problem Statement
2.3 Dimensionality Reduction of the Path Constraint
2.4 Directional Differentiability of the Optimal Value Function
2.5 Algorithm Development
2.6 Convergence Analysis
2.7 Numerical Case Studies
2.7.1 Effectiveness of the Bi-level Algorithm
2.7.2 Computational Performance
2.8 Summary
References
3 Single-Level Dynamic Optimization of Path-Constrained Switched Systems
3.1 Gradient Information
3.2 Algorithm Development
3.3 Convergence Analysis
3.4 Numerical Case Studies
3.4.1 Effectiveness of the Single-Level Algorithm
3.4.2 Computational Efficiency
3.5 Summary
References
4 Dynamic Optimization of Switched Systems with Free Switching Sequences
4.1 Problem Statement
4.2 Master Problem Construction
4.2.1 Switching Sequence Characterization
4.2.2 Support Function Construction
4.3 Algorithm Development
4.4 Convergence Analysis
4.5 Numerical Case Studies
4.5.1 Switch Scheduling
4.5.2 Feasibility Verification
4.6 Summary
References
5 Multi-objective Dynamic Optimization of Path-Constrained Switched Systems
5.1 Definitions and Preliminary Results
5.1.1 Multi-objective Optimization
5.1.2 ε-Constraint Method
5.2 Problem Statement
5.3 Algorithm Development
5.4 Convergence Analysis
5.5 Numerical Case Studies
5.5.1 Pareto Solutions
5.5.2 Feasibility Verification
5.6 Summary
References
6 Conclusions and Future Work
References

Citation preview

Studies in Systems, Decision and Control 459

Jun Fu Chi Zhang

Dynamic Optimization of Path-Constrained Switched Systems

Studies in Systems, Decision and Control Volume 459

Series Editor Janusz Kacprzyk, Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland

The series “Studies in Systems, Decision and Control” (SSDC) covers both new developments and advances, as well as the state of the art, in the various areas of broadly perceived systems, decision making and control–quickly, up to date and with a high quality. The intent is to cover the theory, applications, and perspectives on the state of the art and future developments relevant to systems, decision making, control, complex processes and related areas, as embedded in the fields of engineering, computer science, physics, economics, social and life sciences, as well as the paradigms and methodologies behind them. The series contains monographs, textbooks, lecture notes and edited volumes in systems, decision making and control spanning the areas of Cyber-Physical Systems, Autonomous Systems, Sensor Networks, Control Systems, Energy Systems, Automotive Systems, Biological Systems, Vehicular Networking and Connected Vehicles, Aerospace Systems, Automation, Manufacturing, Smart Grids, Nonlinear Systems, Power Systems, Robotics, Social Systems, Economic Systems and other. Of particular value to both the contributors and the readership are the short publication timeframe and the world-wide distribution and exposure which enable both a wide and rapid dissemination of research output. Indexed by SCOPUS, DBLP, WTI Frankfurt eG, zbMATH, SCImago. All books published in the series are submitted for consideration in Web of Science.

Jun Fu · Chi Zhang

Dynamic Optimization of Path-Constrained Switched Systems

Jun Fu State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang, China

Chi Zhang State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang, China

ISSN 2198-4182 ISSN 2198-4190 (electronic) Studies in Systems, Decision and Control ISBN 978-3-031-23427-9 ISBN 978-3-031-23428-6 (eBook) https://doi.org/10.1007/978-3-031-23428-6 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Acknowledgements

This book is a revised version of Chi Zhang’s Ph.D. thesis under the supervision of Prof. Jun Fu at State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University, China. During our joint research, we would like to express our deepest gratitude to Prof. Tianyou Chai at Northeastern University, Prof. Wei Lin at Case Western Reserve University, and Prof. Wei Zhang at Southern University of Science and Technology, for their in-depth discussions, constructive criticism and valuable suggestions. Special thanks are due to our families for their unconditional love and constant encouragement over the years. Finally, we would like to thank the editors at Springer for their professional and efficient handling of this project. The writing of this book was supported in part by National Natural Science Foundation of China (61825301), and the National Key Research and Development Program of China (2018AAA0101603). Shenyang, China August 2022

Jun Fu Chi Zhang

v

Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Dynamic Optimization: Basic Results and Solution Methods . . . . . . 1.2.1 Mathematical Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.2 First-Order Necessary Conditions . . . . . . . . . . . . . . . . . . . . . . 1.2.3 Numerical Solution Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Dynamic Optimization Problems with Path Constraints . . . . . . . . . . 1.3.1 Mathematical Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.2 CVP-Based Methods for Path Constraints . . . . . . . . . . . . . . . 1.4 Dynamic Optimization of Switched Systems . . . . . . . . . . . . . . . . . . . 1.4.1 Background on Hybrid Systems . . . . . . . . . . . . . . . . . . . . . . . . 1.4.2 Dynamic Optimization of Time-Dependent Switched Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Bi-level Dynamic Optimization of Path-Constrained Switched Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Definitions and Preliminary Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.1 Differentiability of Functionals in Normed Spaces . . . . . . . . 2.1.2 Perturbation Analysis of Constrained Optimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.3 Semi-infinite Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Dimensionality Reduction of the Path Constraint . . . . . . . . . . . . . . . . 2.4 Directional Differentiability of the Optimal Value Function . . . . . . . 2.5 Algorithm Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Numerical Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7.1 Effectiveness of the Bi-level Algorithm . . . . . . . . . . . . . . . . . 2.7.2 Computational Performance . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 1 3 4 6 8 11 11 12 16 16 17 23 25 31 32 32 33 36 37 39 41 46 49 53 53 58

vii

viii

Contents

2.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

59 59

3 Single-Level Dynamic Optimization of Path-Constrained Switched Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Gradient Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Algorithm Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Numerical Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 Effectiveness of the Single-Level Algorithm . . . . . . . . . . . . . 3.4.2 Computational Efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

61 62 62 64 67 67 71 71 72

4 Dynamic Optimization of Switched Systems with Free Switching Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Master Problem Construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Switching Sequence Characterization . . . . . . . . . . . . . . . . . . . 4.2.2 Support Function Construction . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Algorithm Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Numerical Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.1 Switch Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.2 Feasibility Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

73 74 75 75 77 81 82 83 83 86 87 88

5 Multi-objective Dynamic Optimization of Path-Constrained Switched Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 5.1 Definitions and Preliminary Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 5.1.1 Multi-objective Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . 90 5.1.2 -Constraint Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 5.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 5.3 Algorithm Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 5.4 Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 5.5 Numerical Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.5.1 Pareto Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.5.2 Feasibility Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 6 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

Chapter 1

Introduction

1.1 Motivation Optimization is ubiquitous in the natural world and human society-light takes the path which requires the shortest time, a system is stable at its lowest energy state, an investment strategy is expected to maximize the revenue while minimizing the risk, decision makers in manufacturing processes aim to improve the production efficiency while maintaining the quality. The essence of optimization is to make the best choice according to a specified expectation. With the globalization of the economy and the development of computer technologies, including high-speed parallel processors and efficient solvers, optimization has become an effective tool for decision making in the fields of economics, management and engineering, and also has a significant impact on energy conservation, resources utilization and environment improvement. Optimization problems consist of three basic elements: objective functions, decision variables and constraints. An objective function can be any index that describes the performance of the system, e.g., benefit, time, energy consumption. The value of these indices depends on certain parameters of the system-that are, decision variables. Typically, the system operation also requires the satisfaction of certain constraints that determine the range in which decision variables can be chosen. In other words, the main goal of optimization is to find the values of decision variables that optimize the objective function under the satisfaction of constraints. Some common decisionmaking problems in real life can be optimized by simply employing empirical knowledge rather than systematic theoretical approaches. However, as the system structure becomes complex and, the number of decision variables and constraints increases, the optimal solution is difficult to be obtained directly by employing only priori knowledge. In this case, it is more effective to describe the optimization problem in mathematical form and to solve it by employing systematic optimization methods. This is the importance of studying optimization theories and numerical solution methods.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Fu and C. Zhang, Dynamic Optimization of Path-Constrained Switched Systems, Studies in Systems, Decision and Control 459, https://doi.org/10.1007/978-3-031-23428-6_1

1

2

1 Introduction

In 1696, Bernoulli challenged the mathematical community with the Brachistochrone Problem. Calculus of variations was born as the first systematic optimization method, and played an important role in the development of multiple integrals, differential equations and control theory. Although calculus of variation is mainly concerned with the analytic solution, it provides the basic idea for solving optimization problems. With the increasing application of optimization in practice, numerical solution methods have raised a lot of interest over the course of the years. In 1947, Dantzig proposed the simplex algorithm for solving linear programming problems, and in the 1850s, Kuhn and Tucker introduced the necessary conditions of optimality for nonlinear programming problems. Today, optimization theories and numerical solution methods have become two important parts of both theoretical science and practical engineering. Many complex systems and industrial processes with nonlinearity and dynamics are usually described by ordinary differential equations (ODEs), differentialalgebraic equations (DAEs) and partial differential equations (PDEs). With the development of dynamic process modeling and simulation software, e.g., Aspen, Ansys and gPROMS, dynamic systems are increasingly being used to describe complex objects and manufacturing processes such as robotics [1–4], microbiological fermentation [5–7], batch processes [8, 9] and chemical vapor deposition reactors [10], etc. Optimization problems of dynamic systems naturally arise. Such problems are also known as open-loop optimal control problems. Unlike steady state optimization problems in a finite dimensional space, dynamic optimization problems are in the functional (infinite dimensional) space. Conventional methods cannot be directly applied to such problems. Therefore, optimization methods of dynamic systems deserve exploration, and have been develop into a complete research field with a wide range of practical applications. Switched systems arise as models of processes regulated by switching control mechanisms and/or affected by abrupt changes in the dynamics [11]. A switched system consists of several subsystems and a switching law specifying the active subsystems at each time instant. On the one hand, this system description can decompose a complex nonlinear dynamic system into several simpler subsystems. Due to the convenience of this description in optimization and control synthesis, numerous applications are frequently modeled as switched systems, including automotive systems [12, 13], drug schedules [14, 15], the above mentioned robotics [1, 4] and microbiological reactions [5, 7]. On the other hand, dynamic optimization problems of switched systems consist of both continuous and discrete decision variables-that are, the input, switch times and the switching sequence-that interact with each other. Therefore, although switched systems describe complex system dynamics in a relatively simple and direct version, it is still not possible to solve optimization problems of switched systems by employing only continuous or discrete optimization methods. Besides, switch times and the switching sequence are not explicitly expressed in the dynamic equations describing system dynamics, this also makes it difficult to calculate the gradient information during the optimization process. All these factors make the dynamic optimization of switched systems more challenging than general systems.

1.2 Dynamic Optimization: Basic Results and Solution Methods

3

In industrial processes, to enforce product quality or to guarantee safety, path constraints that are usually described as functions of the state and/or the input over the entire time horizon are practically important [16, 17]. Thus as far we have outlined our interest here as being in dynamic optimization of switched systems with path constraints. Existing works on such topics only enforce path constraints at a finite number of time points, which results in a potential risk of constraint violation at any point other than those where the point constraints are enforced, otherwise, an infinite number of iteration. However, a dynamic optimization algorithm for path-constrained switched systems should account for the optimality, guaranteed feasibility and finite convergence simultaneously. This book is meant to be sufficiently self-contained; we have therefore devoted this chapter to an introduction of dynamic optimization, dynamic path-constrained optimization and dynamic optimization of switched systems. This introduction is intended to present an theoretical background of these topics and relevant methods, including underlying their features and motivations of further studies. Chapters 2–5 present four state-of-the-art methods for dynamic optimization problems of pathconstrained switched systems, viz. a bi-level and an efficient single-level dynamic optimization method for problems with a fixed switching sequence, a mixed-integer dynamic optimization (MIDO) method for problems with free switching sequences, and a multi-objective dynamic optimization (MODO) method for problems with several conflicting and incommensurable objectives. All of these algorithms can locate optimal input, switch times and switching sequence (if considered), and can converge finitely with a rigorous satisfaction of path constraints. Each of the 4 chapters collects theoretical results, algorithm descriptions and illustrative examples. In Chap. 6, we conclude the book by briefly summarizing the main theoretical findings, and proposing unsolved problems for further investigations.

1.2 Dynamic Optimization: Basic Results and Solution Methods In this section, we first introduce the mathematical formulation of dynamic optimization problems. Then we present necessary conditions for the solution to a dynamic optimization problem based on calculus of variations. Finally, we discuss various solution methods for solving such problems. Note that the problems considered in this section are embedded with ODEs and does not involve input constraints. Problems with path constraints on both the state and the input are discussed in the next section.

4

1 Introduction

1.2.1 Mathematical Formulation Nonlinear programming (NLP) problems, calculus of variations, optimal control problems and dynamic optimization problems are concepts that are often confused. This subsection introduces the evolution of these concepts, and then presents the mathematical formulation of dynamic optimization problems.

1.2.1.1

From NLP Problems to Calculus of Variations

In an NLP problem, our goal is to find a real variable or real vector variable x to optimize a certain objective function: min x l(x) s.t. x ∈ Rn x

(1.1)

Next, we consider an N stages generalization of the above problem over a discrete time horizon: N  min l(k, xk ) (1.2) x1 ,x2 ,...,x N k=1 s.t. xk ∈ Rn x , k = 1, 2, . . . , N Obviously, problem (1.2) is an (N × n x )-dimensional generalization of problem (1.1). xk only affects the value of l(k, xk ) and has no effect on other stages. Therefore, problem (1.2) is a finite-dimensional NLP problem. If the decision variable affects not only its current stage, but also the next one, then the problem becomes dynamic: N 

min

x1 ,x2 ,...,x N

s.t.

l(k, xk , xk−1 )

k=1

x0 given, xk ∈ Rn x , k = 1, 2, . . . , N

(1.3)

Note that x0 must be specified since it affects the value of x1 . In this case, decision variables of all stages should be solved simultaneously. So far, the generalization from problem (1.2) to problem (1.3) can be regarded as an extension from a steady and finite-dimensional NLP problem to a dynamical one. Now we generalize problem (1.3) to the continuous time horizon T = [t0 , t f ]: min x(t)

 tf t0

l(k, xk )dt

s.t. x(t) ∈ Rn x , ∀t ∈ T

(1.4)

where x(t) is a real-valued function and is of infinite dimension. Similar to problem (1.2), for a given time t¯, x(t¯) only affects the value of l(x, x(t)) at t = t¯.

1.2 Dynamic Optimization: Basic Results and Solution Methods

5

Therefore, problem (1.4) has no dynamics in the sense of continuous time and is an infinite-dimensional NLP problem. So far, the generalization from problem (1.2) to problem (1.4) can be regarded as an extension from a finite-dimensional NLP problem to an infinite-dimensional one. Finally, we apply the generalization, from problem (1.2) to problem (1.3), to problem (1.4). Different form the case of discrete time, we introduce the concept of derivative to describe the effect of “previous stage” to “current stage”-that is, the objective function depends on the decision variable and its derivative. Similarly, x0 must be specified. Then the dynamic formulation of problem (1.4) can be described as follows: t min t0 f l(t, x(t), x(t))dt ˙ x(t) (1.5) s.t. x(t0 ) = x0 , nx x(t) ∈ R , ∀t ∈ T which is a basic mathematical formulation of calculus of variations.

1.2.1.2

From Calculus of Variations to Dynamic Optimization Problems

Calculus of variations are concerned with seeking for n x curves, such that the objective function is minimized and the initial conditions are satisfied. Dynamic optimization considers the same problem but from a more “dynamic” viewpoint: Let us imagine n x moving particle and their trajectories in the (t, x)-space. If we consider the derivative x(t) ˙ of the point on the trajectory as a infinite-dimensional decision ˙ = y(t). Then probvariable y(t) : T → Rn y , we then obtain a dynamic system x(t) lem (1.5) can be equivalently described as follows: min

y(t)∈Rn y

s.t.

 tf t0

l(t, x(t), y(t))dt

x(t) ˙ = y(t), ∀t ∈ T, x(t0 ) = x0

(1.6)

Furthermore, if there is equality constraints of the form M(t, x(t), x(t)) ˙ = 0, ∀t ∈ T

(1.7)

We can solve Eq. (1.7) for x(t) ˙ and consider x(t) ˙ as a function of t and x(t). Generally, the number of constraints is less than the dimension of x(t). ˙ Therefore we introduce a free variable u(t) : T → Rn u : x(t) ˙ = f (t, x(t), u(t)), ∀t ∈ T where f : T × Rn x × Rn u . In this case, the calculus of variations becomes

(1.8)

6

1 Introduction

minn

u(t)∈R

s.t.

u

J (u(t)) =

 tf t0

l(t, x(t), u(t))dt

x(t) ˙ = f (t, x(t), u(t)), ∀t ∈ T, x(t0 ) = x0

(1.9)

Problem (1.9) is a dynamic optimization problem which does not contain input constraints and the sate is only governed by the ODE system. x and u refer to the state variable and control input, respectively. Problem (1.6) is a special case of problem (1.9). The solution x(t; x0 , u(t)) of the ODE system depends on the input and the initial condition of the system. We write J (u(t)) for simplicity to reflect the fact that the objective function is minimized over the space of u(t), since x(t) also depends on u(t). It can be seen that the formulation of dynamic optimization problems is more applicable for practical problems with constraints. In problem (1.9), the integral function l : T × Rn x × Rn u is referred to as Lagrangian, and is usually used to describe dynamic performance indices regarding the system operation process. For different optimization problem, there are also other forms of the objective function. For example, the Mayer form (terminal form) ϕ(x(t f ), t f ) : Rn x × R is used to describe performance indices regarding the terminal state of the system. In addition, the Bolza form consists of both integral and terminal terms. Objective functions of the above three forms can be converted to each other [18].

1.2.2 First-Order Necessary Conditions To solve a dynamic optimization problem, conditions that must be satisfied at optimal solutions should be given. Below we shall give first-order necessary conditions for unconstrained dynamic optimization problems (i.e., Euler-Lagrange equations), based on concepts of functional and its variations. For problem (1.9), we form the augmented functional to include the differential equation constraints J¯ =



tf

[l(t, x(t), u(t)) + λT (t)( f (t, x(t), u(t)) − x)]dt ˙

(1.10)

t0

by introducing the adjoint multiplier λ(t). For simplify, we define a scalar function H (the Hamiltonian) as follows: H = l(t, x(t), u(t)) + λT (t) f (t, x(t), u(t))

(1.11)

Integrating the right-hand side of (1.11) by parts yields J¯ = − λT (t f )x(t f ) + λT (t0 )x(t0 )  tf (H (t, x(t), u(t), λ(t)) + λ˙ T (t)x(t))dt + t0

(1.12)

1.2 Dynamic Optimization: Basic Results and Solution Methods

7

Now we consider the variation in J¯ due to variations in the control input u(t) for fixed times t0 and t f : δ J¯ = −λT δx |t=t f + λT δx |t=t0 +





tf t0

  ∂H ∂H T ˙ + λ δx + δu dt ∂x ∂u

(1.13)

Since it would be tedious to derive the variation δx, we choose the following adjoint multiplier λ(t) to make δx vanished: ∂l ∂f ∂H =− − λT , ∀t ∈ T, λ˙ T = − ∂x ∂x ∂x T λ (t f ) = 0

(1.14)

Substituting (1.14) into (1.13) then yields δ J¯ = λT (t0 )δx(t0 ) +



tf

t0

∂H δudt ∂u

(1.15)

For an optimal solution of problem (1.9), δ J must be 0 for arbitrary δu, that is, ∂H =0 ∂u

(1.16)

Equations (1.14) and (1.16) are also known as Euler-Lagrange equations. For the case that the objective function is of Mayer form, then the terminal condition of the adjoint multiplier should be λT (t f ) = ∂ϕ . ∂x In conclusion, by solving the following two-point boundary-value problem, we can obtain an input satisfying the first-order necessary conditions of problem (1.9): x˙ = f (t, x, u), ∀t ∈ T,

λ˙ −



∂l ∂x

T



∂f −λ ∂x

(1.17)

T , ∀t ∈ T,

(1.18)

∂H = 0, ∂u

(1.19)

x(t0 ) = x0 ,

(1.20)

λ(t f ) = 0

(1.21)

8

1 Introduction

1.2.3 Numerical Solution Methods After introducing necessary conditions that must be satisfied for dynamic optimization problems, we shall focus on two numerical techniques for solving such problems, namely, indirect methods and direct methods. Indirect methods, also known as variational methods, attempt to locate the optimal solution by solving the twopoint boundary-value problem (1.17)–(1.21) based on the first-order necessary conditions [19–25]. Direct methods, also know as discretization methods, transform the infinite-dimensional dynamic optimization problem into finite-dimensional mathematical program such as NLP, and then construct a sequence that converges to the optimal solution [26–28]. Other numerical solution methods not discussed herein include dynamic programming [29–31] and stochastic optimization methods [32].

1.2.3.1

Indirect Methods

Indirect methods attempt to locate the optimal solution “indirectly” by solving the two-point boundary-value problem (1.17)–(1.21). Such methods are generally based on iterative procedures-an initial value is estimated to find a solution that satisfies parts of problem (1.17)–(1.21), then the solution is modified to successively satisfy all the necessary conditions. Next, we shall introduce three common indirect methods for solving dynamic optimization problems. Steepest descent [33–35], first proposed by Cauchy in 1847 [36], is the most common algorithm for solving dynamic optimization problems. The application of steepest descent in calculus of variations dates back to the work published by Hadamard in 1907 [37]. The clear statement and standard solving procedure was given for the first time by Courant [38]. In dynamic optimization problems, the steepest descent first estimates an initial value u k (t) of the control input, and solves the state and adjoint equations by forward and backward integration respectively, to obtain x k (t) and λk (t). Then according to Eq. (1.11) and by specific methods for determining iteration steps, e.g., Armijo-Goldstein criterion, Wolfe-Powell criterion, etc., new control input is generated until Eq. (1.11) is satisfied. The convergence of steepest descent does not depend heavily on the estimate of the initial value, but the convergence rate becomes slower and slower as the iterative procedure approaches to the optimal solution. In addition to estimating the control input, one can also estimate boundary conditions of the state or adjoint equations. Such approaches are referred to as boundary iterative methods, including variation of extremals for problems with fixed initial time, terminal time and initial state [35], and indirect shooting methods for problems with fixed initial time, terminal state and unfixed terminal time [22]. The solutions generated at each iterations of these approaches satisfy Eqs. (1.17)–(1.19), then the estimates of boundary conditions are adjusted by calculating the transfer matrix representing the sensitivity of the terminal value to the initial value. Boundary iterative methods have a faster convergence rate compared with steepest descent when the

1.2 Dynamic Optimization: Basic Results and Solution Methods Table 1.1 Comparison of three indirect methods [35] Steepest descent Quasilinearization Initial guess Conditions to be satisfied Convergence Convergence rate

u(t), t ∈ T ∂H ∂u = 0 Does not depend on the initial guess Slows down as approaching the optimal solution

x(t), λ(t), t ∈ T State and adjoint equations Poor initial guess may lead to divergence Quadratic once converging

9

Variation of extremals λ(t) λ(t f ) = 0 Poor initial guess may lead to divergence Quadratic once converging

iterative process is close to the neighborhood of the optimal solution. However, the convergence depends heavily on the initial guess-if the initial guess is very poor, the algorithm may not convergence at all. This is due to the fact that Euler-Lagrange equations are essentially influence function equations [22], and the stability of the state equation and the adjoint equation is opposite. Regardless of the direction from which the integration begins, the difference in order of magnitude between x(t) and λ(t) becomes larger as time grows, which leads to loss of computational accuracy. This loss of accuracy may cause the transfer matrix to become pathological (the elements with larger values are all equal and the elements with smaller values are all zero), which results in non-convergence of the algorithm. Therefore, the initial guess is the key to the boundary iterative method, and a good initial guess should generate a solution that satisfies the given boundary conditions as much as possible. Quasilinearization is another commonly used indirect method for dealing with nonlinear differential equations that is difficult to solve. Different from steepest descent and boundary iterative methods, quasilinearization first estimates the initial solutions x k (t) and λk (t) of the state and adjoint equations, and then determines the expression of u k (t) by the first-order necessary conditions. Finally the solutions of the state and adjoint equations are adjusted by solving a linearized two-point boundary-value problem. McGill and Kenneth [39] proved that the solution sequence of the linearized two-point boundary-value problem converges to the solution of the original nonlinear two-point boundary-value problem. It can be seen from the solution procedure that quasilinearization is more suitable for problems where estimating the state is easier than estimating the input. As with boundary iterative methods, quasilinearization converges faster than steepest descent in the vicinity of the optimal solution. However, divergence may also result from a poor guess. Table 1.1 provides a comparison of the above three indirect methods. It can be seen that these solution approaches may not all converge, and even if they do, they may only converge to a local optimal solution. Therefore, a hybrid scheme and different initial guesses may be useful for practical problems. For example, steepest descent can be employed to generate a solution at the beginning of the algorithm, then this solution can be used as the initial guess for the indirect shooting method.

10

1 Introduction

Indirect methods is characterized by dealing with dynamic optimization problems in an infinite-dimensional space, which does not require transformation of the problem structure. Therefore, the exact solution of the original problem can be located theoretically. Especially for convex problems, indirect methods can even locate the analytical solution. However, for dynamic systems with high-dimensional state variables, indirect methods suffer from the difficulty of finding the solution of EulerLagrange equations. Moreover, for the case where state variables are with inequality constraints, adjoint equations are usually discontinuous, and the first-order necessary conditions become a multi-point boundary-value problem. Prior knowledge of the structure are required for the optimal solution when solving such problems (i.e., the number of intervals with active constraints) [40, 41]. This indeed makes the application of indirect methods to constrained dynamic optimization problems difficult.

1.2.3.2

Direct Methods

In order to perform an algorithm on computers, even in the indirect method, we usually need to approximate infinite-dimensional decision variables by using discretization techniques (e.g., iterations for control inputs in steepest descent). If we can accept the fact that decision variables will inevitably be discretized during the numerical solution procedure, we can use direct methods to solve dynamic optimization problems. The basic idea of direct methods is to discretize the infinite-dimensional dynamic optimization problem into a finite-dimensional problem, and then solve it by NLP methods. According to the degree of discretization, direct methods can be divided into two categories: sequential methods and simultaneous methods. Sequential methods discretize the control variables into polynomial functions over finite elements. The discretization process is usually realized by control vector parameterization (CVP), and the polynomial coefficients are then considered as decision variables of the discretized NLP problem [42–44]. State variables can be obtained by solving ODEs. In simultaneous methods, both the state and control variables are discretized as polynomial functions over finite elements, and the ODEs are treated as equality constraints. The discretization is usually implemented by collocation approaches [45–51]. Sequential methods only discretizes control variables and generates NLP problems with fewer decision variables, however, ODEs are required to be solved at each iterations. Simultaneous methods do not involve solving ODEs, however, transforming them into equality constraints will further increase the complexity of the NLP problems. Since the existing techniques for solving ODEs are much more established than those for solving NLPs, Sequential methods have more advantages than Simultaneous methods in large-scale dynamic optimization problems. In fact, collocation is equivalent to performing a fully implicit Runge-Kutta integration, which is not as efficient as the backward differentiation formula (BDF) method [52]. Compared with indirect methods, direct methods have many advantages, although it can only locate the approximate solution of the original problem. For example,

1.3 Dynamic Optimization Problems with Path Constraints

11

direct methods only require calculating control and state variables, and do not require calculating adjoint multipliers, avoiding difficulties in solving Euler-Lagrange equations when dealing with large-scale dynamic optimization problems. As pointed in [53], direct methods can be used to solve problems with thousands of state variables, which is much higher than the scale that indirect methods apply to. Besides, well-established solution methods for NLPs can be used to solve the discretized finite-dimensional optimization problem, and therefore direct methods is more suitable for the case with constraints.

1.3 Dynamic Optimization Problems with Path Constraints In the previous subsection, we mainly introduced related theories and numerical solutions of unconstrained dynamic optimization problems, while constrained dynamic optimization problems are more common in real life. In this subsection, we shall give the mathematical description of dynamic optimization problems with path constraints, and will introduce some common numerical solutions and their limitations.

1.3.1 Mathematical Formulation Constraints determine the optional range of state and/or input variables of a system, which are generally divided into two categories: (1) point constraint: ψ(t¯, x(t¯), u(t¯)) ≤ 0

(1.22)

where t¯ ∈ [t0 , t f ], ψ : R × Rn x × Rn u → R. Point constraints require state and/or input variables to be satisfied at a certain time point, e.g., xtLf ≤ x(t f ) ≤ xtUf

(1.23)

ensures that the terminal state of a system stabilized within the range of [xtLf , xtUf ]. Such constraints often occur in system stabilization or state transformation problems. (2) path constraint: g(t, x(t), u(t)) ≤ 0, ∀t ∈ [t0 , t f ]

(1.24)

where g : R × Rn x × Rn u → R. Path constraints require state and/or input variables to be satisfied over the entire time horizon, e.g.,

12

1 Introduction

x L ≤ x(t) ≤ x U ∀t ∈ [t0 , t f ]

(1.25)

guarantees that a system runs within the range of [x L , x U ] over the entire time horizon. Such constraints often occur in system safety problems. It can be seen that a path constraint indicate an infinite number of point constraints over continuous time. Point constraints can be regarded as a special case of path constraints. As shown in Eq. (1.24), a path constraint is a continuous function in x(t) and u(t). In particular, a path constraint that is only in x(t) is called a pure state path constraint. Pure state path constraints are more difficult to handle than path constraints that are both in x(t) and u()t. This is because pure state path constraints do not explicitly depend on u(t), and x(t) can only be determined indirectly by differential equations [54]. Therefore, a general path constraint of the form of Eq. (1.24) can be obtained only when the pure state path constraint is successively differentiated with respect to t until u(t) appears. Unless otherwise specified, Eq. (1.24) shall be used to represent path constraints hereinafter. Pure state constraints can be regarded as a special case of general path constraints. So far, we can give the mathematical formulation of dynamic problems with path constraints: min J (u(t)) = u(t)

 tf t0

l(t, x(t), u(t))dt

s.t. g(t, x(t), u(t)) ≤ 0, ∀t ∈ T, x(t) ˙ = f (t, x(t), u(t)), ∀t ∈ T, x(t0 ) = x0 .

(1.26)

1.3.2 CVP-Based Methods for Path Constraints At the end of the previous section, we introduced the advantages of direct methods compared with indirect methods in handling constrained dynamic optimization problems. Therefore, in this subsection, we shall mainly focus on several numerical methods for path constrained dynamic optimization problems based on CVP. It is worth mentioning that Pontryagin’s maximum principle (PMP) [55], as the theoretical basis of indirect method for solving constrained dynamic optimization problems, is also one of the most important theories in dynamic optimization. Interested readers may refer to the review article by Hartl et al. [54].

1.3.2.1

Slack Variable Approach

Slack variable approach, proposed by Jacobson and Lele in 1969 [56], is a method to convert inequality path constraints into ODEs by introducing slack variables. Reference [56] considers the pure state path constraint as follows:

1.3 Dynamic Optimization Problems with Path Constraints

g(t, x(t)) ≤ 0

13

(1.27)

First, by introducing a slack variable a(t), Eq. (1.27) is appended to 1 g(t, x(t)) + a 2 (t) = 0 2

(1.28)

It can be seen that Eq. (1.28) holds only if Eq. (1.27) is satisfied, regardless of the value of a(t). Next, we differentiate both sides of Eq. (1.28) with respect to t for k times: g (1) + aa (1) = 0, g (2) + (a (1) )2 + aa (2) = 0, .. . g (k) +

1 dk 2 a =0 2 dt k

(1.29)

where g (i) = ddtgi , a (i) = ddtai , i = 1, 2, . . . , k, until u(t) appears. The pure state path constraint (1.27) is then transformed into an ODE system (1.29). The decision variables of the transformed optimization problem are u(t) and x(t), which can be solved by an NLP solver after CVP is performed. We give the example in [56] to further elaborate the specific steps of the slack variable approach: i

i

Example 1.1 Consider the following dynamic optimization problem: min J (u(t)) = u(t)

1 0

(x12 + x22 + 0.005u 2 )dt

s.t. x˙1 = x2 , x1 (0) = 0, x˙2 = −x2 + u, x2 (0) = −1, x2 − 8(t − 0.5)2 + 0.5 ≤ 0, t ∈ [0, 1]

(1.30)

First, we introduce the slack variable a(t) 1 x2 − 8(t − 0.5)2 + 0.5 + a 2 = 0 2

(1.31)

Differentiating both sides of Eq. (1.31) then yields −x2 + u − 16(t − 0.5) + aa1 = 0

(1.32)

˙ Substituting Eq. (1.32) into ODEs and the objective function, then where a1 = a. problem (1.30) becomes

14

1 Introduction

1 mina1 J (a1 ) = 0 (x12 + x22 + 0.005(x2 + 16t − 8 − aa1 )2 )dt s.t. x˙1 = x2 , x1 (0) = 0, x˙2 = −aa1 + 16t√− 8, x2 (0) = −1, a˙ = a1 , a(0) = 5, t ∈ [0, 1]

(1.33)

Several problems with the slack variable approach was noted in [53]. First of all, the dynamic optimization problem contains a singular arc whenever a(t) = 0. In CVP, this singular arc has the effect of providing no gradient information to the optimizer whenever a(t) = 0, and thus significantly slowing down or preventing convergence of gradient-based NLP methods. Moreover, the optimality of the slack variable problem is only a necessary condition of the original problem. Therefore, an optimal solution of the slack variable problem may not be the optimal solution of the original problem.

1.3.2.2

Penalty Function Approach

The penalty function approach [22, 57] is a method that transforms a constrained optimization problem into an unconstrained one. In the penalty function approach, the objective function is augmented to the following form: J¯ = J + K



tf

r T (t, x(t), u(t))r (t, x(t), u(t))dt

(1.34)

t0

or J¯ = J +

ng  i=1

 Ki

tf

ri (t, x(t), u(t))dt

(1.35)

t0

where K ∈ R+ , n g denotes the number of path constraints. r (t, x(t), u(t)) ∈ Rn g is called a penalty function, denotes the measure of constraint violation: ri (t, x(t), u(t)) = max[gi (t, x(t), u(t)), 0], i = 1, 2, . . . , n g

(1.36)

There are two main difficulties in the application of the penalty function method: first, the penalty factor K needs to be large enough, i.e., K → ∞ to ensure the rigorous satisfaction of the path constraint. Second, the operator max in Eq. (1.36) leads to an implicit discontinuity in numerical simulation [53], which requires special treatment during integration [58] and calculation of sensitivities [59].

1.3 Dynamic Optimization Problems with Path Constraints

1.3.2.3

15

Terminal Constraint Approach

In addition to penalizing the path constraint violation into the objective function, it can also be transformed into a terminal constraint [60] of the form: 

tf

ϕi (t, x(t), u(t)) =

ri (t, x(t), u(t))dt, i = 1, 2, . . . , n g

(1.37)

t0

Like the penalty function approach, the terminal constraint approach also has the problem of discontinuity in numerical simulation. Besides, Eq. (1.36) is nondifferentiable at gi = 0. To solve this problem, a smooth technique was proposed in [26]: ⎧ ⎪ if gi (t, x, u) ≤ − ⎨0, ri (t, x, u) = (−gi (t, x, u) − )2 /4, if −  ≤ gi (t, x, u) ≤ , i = 1, 2, . . . , n g ⎪ ⎩ gi (t, x, u), if gi (t, x, u) >  (1.38) This smooth technique makes r (t, x, u) differentiable at gi = 0 and involves no implicit discontinuity.

1.3.2.4

Point-Wise Discretization

The basic idea of point-wise discretization is to successively add the constraint violation to the next iteration as discrete point constraints [44, 52, 61]: g(ti , x(ti ), u(ti )) ≤ 0, i = 1, 2, . . . , n p

(1.39)

where n p denotes the number of discrete point constraints. For example, the unconstrained optimization problem is first solved to locate the time period where the path constraint is violated. Then a time point constraint at the midpoint of this period is added to the next iteration. Repeat the above steps until the violation of the path constraint satisfies a specified tolerance. Point-wise discretization can guarantee that the violation of path constraints is within a specified tolerance within a finite number of iterations. However, it still requires n p → ∞, i.e., the number of discrete point constraints is large enough to ensure the rigorous satisfaction of path constraints over the entire time horizon. Now, we have introduced several common approaches for handling path constraints. However, none of these approaches can guarantee the rigorous satisfaction of path constraints within a finite number of iterations. The reason why this goal is difficult to achieve is that-on one hand, path constraints indicate an infinite number of time point constraints in continuous time, the iterative number of discretizationbased approaches needs to be infinite large to guarantee the rigorous satisfaction of

16

1 Introduction

path constraints. On the other hand, due to inevitable errors during the numerical computation, path constraints may still be violated in practice even if the feasibility of the obtained solution is theoretically guaranteed.

1.4 Dynamic Optimization of Switched Systems With the rapid development of control theory and computer technology, the systems that scholars study, analyze and design have become more and more complex. In practice, many systems have both continuous dynamics and discrete logic, which cannot be described by a unified mathematical model, so the concept of hybrid systems is proposed. Furthermore, we can simplify the discrete logic to the concept of “switching”, and study the switched system to design the control and optimization methods of hybrid systems. In this section, we refer to the review of optimal control of hybrid switched systems published by Feng Zhu and Panos J. Antsaklis [62], and introduce the current research progress on dynamic optimization of hybrid systems, followed by the mathematical description of two categories of dynamic optimization problems of switched systems. Finally, we summarize the existing dynamic optimization methods for time-dependent switched systems that are the main focus of this book.

1.4.1 Background on Hybrid Systems A hybrid system consists of both continuous dynamics describing physical characteristics and discrete information portraying modes and decisions. The state evolution of the system is driven by both time and events that are closely interconnected, thus expressing a complex dynamic behavior. Hybrid systems are almost ubiquitous in real world. Take the manufacturing production process as an example, different production chains are run in different manufacturing shops, and each chain has its own dynamic model. Only when the current chain is finished, the next chain begins. In other words, the manufacturing production process consists of event-driven dynamics associated with switching between different chains and the respective time-driven dynamics of each chain [63]. The modeling of hybrid system also facilitate the control and optimization of engine systems. For example, a four-stroke (also known as four-cycle) gasoline engine is a system with natural hybrid characteristics. On the one hand, the power train and energy conversion are continuous processes. On the other hand, the pistons completes four separate strokes while turning the crankshaft: intake, compression, combustion and exhaust. A four-stroke gasoline engine completes an operating period after four strokes, in which the piston moves up and down reciprocally for four strokes and the corresponding crankshaft rotates for two cycles. Other examples include hierarchical systems. The lower layers contain multiple independent dynamic subsystems whose states are indirectly controlled by discrete logic

1.4 Dynamic Optimization of Switched Systems

17

signals from the upper layers [64]. An aircraft also has different requirements for engine thrust in different phases, and the engine is required to continuously switched between different modes during the flight [65]. Employing the modeling of hybrid systems can more accurately describe the phenomena and processes driven by the interplay of time and events, and can better meet the control and optimization requirements of these systems. In the past 30 years, hybrid systems have been the focus in various fields such as automatic control theory and computer science. In 1993, Springer-Verlag published the first series of book on hybrid systems [66], which included outstanding articles from the symposiums on hybrid systems held at Cornell University in 1991 and Technical University of Denmark in 1992. Since then, this series has regularly featured the state-of-the-art research on hybrid systems [67–70] and organized the special issue “Hybrid Systems: Computation and Control series” [71–74]. International influential journals on automatic control, such as IEEE Transactions on Automatic Control and Automatica, have also published special issues on hybrid systems [75, 76]. Since the concept of dynamic optimization for hybrid systems was proposed, numerous relevant theoretical results and numerical solution methods have been developed. On the one hand, based on dynamic programming, Branicky and Borkar et al. proposed a theoretical framework for dynamic optimization of general hybrid systems, and provided the optimal necessary conditions for the existence of optimal and suboptimal solutions for hybrid systems. In addition, generalized quasi-variational inequalities need to be satisfied for the optimal value function were derived [77]. Hedlund and Rantzer calculated the upper and lower bounds of the optimal value and obtained the approximate optimal solution by employing convex dynamic programming method [78, 79]. This method was also extended to the case of piecewise affine systems by Rantzer and Johanson [80]. On the other hand, based on maximum principle, Sussmann [81] and Piccoli [82] gave the optimal necessary conditions for dynamic optimization of hybrid systems with fixed switching sequences. Riedinger and Kratz et al., aiming at dynamic optimization problems of hybrid system with quadratic objective function, gave the optimal necessary conditions for cases of autonomous switching and controlled switching [83]. Shaikh and Caines also gave the necessary conditions for finite time hybrid dynamic optimization problems with fixed switching sequences [84], and extended these results to the case of unfixed switching sequences [85] and the feedback control law design of finite time linear quadratic regulator problems [86, 87].

1.4.2 Dynamic Optimization of Time-Dependent Switched Systems Switched systems are a special class of hybrid systems that consist of several subsystems and a switching law that determines when each subsystem is activated. Consider a switched system that can run in m different modes indexed by M = {1, 2, . . . , m},

18

1 Introduction

Fig. 1.1 Sketch of time-dependent switched systems

Fig. 1.2 Sketch of state-dependent switched systems

and undergoes n switches at s1 , s2 , . . . , sn in the time interval T = [t0 , t f ]. We define the feasible set of switch times as S:={s ∈ Rn : 0 ≤ s1 ≤ s2 . . . ≤ sn ≤ t f }

(1.40)

and s0 = t0 , sn+1 = t f . The system dynamic is governed by ODEs of a vector field f w : Rn x × Rn u → nx R : x(t) ˙ = f w(t) (x(t), u(t)), ∀t ∈ T, x(t0 ) = x0

(1.41)

where the variables x : T → X ⊆ Rn x and u : T → U ⊆ Rn u denote the state and input of the dynamic system, with X and U nonempty and compact. The schedule function w ∈ W := {w : T → {1, 2, . . . , m}} is a piecewise function with s1 , s2 , . . . , sn as its piecewise points, and determines the active subsystem at time t. According to the mechanism of switching signals, switched systems can be divided into two categories: time-dependent and state-dependent switched systems. As shown in Fig. 1.1, a time-dependent switched system determines how to switch by on human intervention, and the switching signal is one of the input signals similar to the continuous control input, i.e. the external inputs of the system contains both u(t) and s. As shown in Fig. 1.2, the switching signal of a state-dependent switched system depends on the state and the currently active subsystem, i.e. the switching of the system depends on the evolution of the system. In this book, we focus on dynamic optimization problems of time-dependent switched systems, which can be mathematically formulated as follows:

1.4 Dynamic Optimization of Switched Systems

min

(w,u)∈W×U

J (x(t), u(t)) =

 tf t0

19

l(t, x(t), u(t))dt + ϕ(x(t f ))

x(t) ˙ = f w(t) (t, x(t), u(t)), ∀t ∈ T, x(t0 ) = x0

(1.42)

where l : Rn x × Rn u → R is referred to as the running cost. In certain cases, l may be dependent on the subsystem activated at the moment. ϕ : Rn x → R is referred to as the terminal cost. Notice that the schedule function w characterizes the switching sequence and switch times, therefore, decision variable of problem (1.42) are essentially the input, switch times and switching sequence. We define the switching sequence σ := {σ1 , σ2 , . . . , σn+1 } ∈ {1, 2, . . . , m}n+1 , that is, the system switches from subsystem f σi to subsystem f σi+1 at the switch time si for i = 1, 2, . . . , n. We define ∇s J (s) as the derivative of the objective function with respect to the switch time vector. Dynamic optimization methods for time-dependent switched systems are divided into two main categories: two-stage optimization and embedding transformation.

1.4.2.1

Two-Stage Optimization

The two-stage optimization algorithm was first proposed by Xu and Antsaklis [88]. The algorithm first locates the optimal input and switch times with a fixed switching sequence, then updates the switching sequence to minimize the objective function. The optimal solution of the dynamic optimization problem is obtained by alternating iterations of the two optimization stages. The solution procedure can be described as follows: Stage 1 For a given switching sequence σ, Locate the optimal input u and switch time vector s. Stage 2 Consider the objective function J as a function of the switching sequence σ, and optimize the switching sequence to σ. ˜ Repeat Stage 1. Notice that Stage 1 and 2 are two separate optimization problems, and the solution procedure are independent of each other. Therefore, many existing results assume that the switching sequence is pre-fixed, and mainly study optimization methods for Stage 1. (1) Fixed switching sequences For autonomous switched systems, only switch times needs to be optimized for Stage 1. Egerstedt et al. derived the formula to calculate ∇s J (s) for autonomous nonlinear switched systems, and employed Armijo’s step sizes to locate the optimal switch times [89, 90]. Furthermore, they also considered the case where the complete information of the system is not available in real time but only partially known during the

20

1 Introduction

practical optimization process [91, 92], and proposed a series of online optimization methods [93–96]. Unlike the aforementioned results that utilize the constrained Lagrange multiplier method to calculate ∇s J (s), Johnson and Murphey proposed a method that utilizes calculus of variations to calculate ∇s J (s) and extended the method to compute ∇s2 J (s) [97, 98]. Caldwell and Murphey compared the convergence performance of optimizing the switch times using first-order method (steepest descent) and second-order method (Newton method) [99]. The results show that the application of both first and second-order gradients significantly reduces the number of iterations required for the algorithm to converge, compared to algorithms with only first-order gradients. Besides, the quadratic convergence of the second-order method is more applicable to online optimization scenarios. In addition to the above systems, Giua et al. considered a class of autonomous switched affine systems with state jumps. The optimal switch times take the form of state feedback and appear at the instant when the system state enters a specific domain [100, 101]. Xu and Antsaklis proposed a series of bi-level optimization methods for Stage 1 of non-autonomous nonlinear switched systems [90, 104–106]. They treat the value of the objective function at a given input as a function of the switch time vector, and then employed a gradient-based NLP algorithm to locate optimal switch times. The solution procedure can be described as follows: Inner level Fix the switching sequence and switch times, locate the optimal input, and consider the objective function as a function of the switch time vector, i.e., v(s). Outer level Solve the optimization problem: min v(s) s

s.t. s ∈ S

(1.43)

to update the switching sequence. Repeat the inner level algorithm. Where v(s) := inf u∈U J (s, u) is referred to as the optimal value function. At the inner level of the algorithm, although different active subsystems are running on different time intervals, the corresponding optimization problem can still be solved by traditional unconstrained optimization methods, since these time intervals are fixed. At the outer level, the optimization problem needs to be solved by constrained optimization methods, such as gradient projection and constrained Newton methods, because the switch time vector are constrained. All these methods require the calculation of gradient information for the optimal value function. Now we present the bi-level optimization algorithm as follows. As shown in Algorithm 1, the calculation of ∇s v(s k ) and ∇s2 v(s k ) is the key element since they are not directly available in step 3. Xu and Antsaklis proposed a method to calculate the approximate value of ∇s v(s k ) by direct differentiation of the optimal value function [102]. They also proposed a method to calculate the exact

1.4 Dynamic Optimization of Switched Systems

21

Algorithm 1 Bi-level dynamic optimization algorithm for switched systems with a fixed switching sequence. Input: iteration counter k = 0; feasible switch time vector s 0 ; switching sequence σ Output: (s k , u k ) 1: loop 2: Solve problem (1.42) at (s k , σ), obtain the optimal solution u k 3: Calculate ∇s v(s k ) (and ∇s2 v(s k ) for second-order methods) 4: if Termination conditions are satisfied (e.g., the norm of the projection of ∇s v(s k ) in all feasible directions is less than a sufficiently small positive real number ) then 5: Terminate 6: else 7: Update switch times using the feasible direction method (where the step sizes are obtained by rules such as Armijo’s step sizes, and the feasible direction is obtained by the gradient information of the objective function with respect to the switch time vector) 8: end if 9: k ← k + 1 10: end loop

value of ∇s v(s k ) by transforming the original problem into an equivalent problem with the switch times as parameters, and solving the two-point boundary-value differential algebraic equations [103]. Notice that even though there have been many results on the calculation of the derivative of the optimal value function, whether and under what conditions the optimal value function is differentiable had not been analyzed. Xu and Antsaklis only mentioned that the optimal value function and its approximation are differentiable as shown from simulation results, but this conclusion still lacks theoretical proof. Until 2012, [104] has made an attempt towards this by using perturbation analysis theory, but only considered the case without any constraints. Nevertheless, this result provides theoretical support for the application of the bi-level algorithm to unconstrained dynamic optimization problems of non-autonomous switched systems. (1) Free switching sequences Stage 2 of the two-stage algorithm can be considered as a discrete optimization problem, however, searching for all possible switching sequences is an exponentially growing process. To reduce the computational complexity, Axelsson et al. proposed the mode insertion technique, which updates the switching sequence by inserting new subsystems [108, 109]. For autonomous nonlinear switched systems, the calculation of the gradient information was investigated by Egerstedt et al. [90], and a complete description of the algorithm was presented by Axelsson et al. [105, 106]. The algorithm first optimizes switching times in Stage 1 and then updates the switching sequence in Stage 2 by inserting a new mode into the current system. To address the problem that updating the switching sequence leads to an increase in the dimensionality of the search space, Axelsson et al. defined and proved a specific local optimality and convergence in [107]. For non-autonomous nonlinear switched systems with state constraints, Gonzalez et al. proposed a hierarchical algorithm similar to the two-stage algorithm [108, 109]. First, the inner level of the algorithm

22

1 Introduction

assumes a fixed switching sequence, and locates the optimal input and switch times. The state constraints are satisfied by enforcing the maximum of the system state. Then the switching sequence is updated in the outer level using the mode insertion technique to optimize the objective function. It is worth mentioning that although the mode insertion technique realizes the update of the switching sequence in the two-stage algorithm, however, Wardi and Egerstedt pointed out in [110] that this method may lead to an infinite loop. In addition to the mode insertion technique, a master-slave iterative algorithm was introduced in [100, 101, 111, 112] for autonomous switched linear systems. This algorithm assumes a fixed switching sequence in the slave problem and determines optimal switch times according to the instants when the state entries into a specific domain. Then the optimal switching sequence is obtained in the master problem by solving mixed-integer quadratic programs. In addition, Caldwell and Merphey also proposed a dynamic optimization algorithm for switched systems based on the gradient projection method [113, 114]. This algorithm first transforms the original problem into an infinite-dimensional optimal control problem with integer constraints on the switching decision variables, and then utilizes the gradient projection method to deal with the integer constraints.

1.4.2.2

Embedding Transformation

The basic idea of embedding transformation is to describe the switched system as a family of nonlinear systems, which in turn can be solved by using conventional nonlinear dynamic optimization methods. Das and Mukherjee proposed an embedding transformation technique for autonomous linear switched systems and employed the maximum principle to find the optimal solution by solving a two-point boundary-value problem [114]. References [115] and [116] proposed an embedding transformation technique for autonomous nonlinear switched systems, which transformed the switched system into an equivalent polynomial system. However, due to non-convex constraints and discontinuous feasible domains in the transformed problem, it is difficult to be solved by conventional optimization solvers. So they further relaxed the transformed problem and then solved it by using convex programming methods. References [117] and [118] considered applying embedding transformation to non-autonomous nonlinear switching systems, and transformed the switched system into the following form: x(t) ˙ =

n+1 

νi (t) f i (x(t), u(t))

(1.44)

i=1

n+1 where νi (t) ∈ [0, 1] and i=1 νi (t) = 1. The transformed optimization problem (1.44) can be solved by using conventional dynamic optimization methods. Bengea and DeCarlo proved that the trajectory of the embedded optimization prob-

1.5 Summary

23

lem can be approximated arbitrarily close by the trajectory of the switched systems generated by an appropriate switching control [117]. The solution to the embedded optimization problem (if exists) is either an optimal solution (bang-bang form) of the original problem, or can be utilized to construct a sub-optimal solution. Wei et al. further combined this method with the sequential quadratic programming (SQP) method to reduce the computational complexity [118]. In addition to the two-stage optimization and embedding transformation technique discussed above, time-scale scaling [26, 119–121] is also an efficient method for dynamic optimization of switched systems. Algorithms based on mixed-integer programming have also emerged in recent years [122–124], which consider both switching costs and jumps, and deal with path constraints by a simultaneous adaptive collocation approach.

1.5 Summary From the introduction and background presented in the previous sections, it can seen that although dynamic optimization problems of both path-constrained and switched systems have been well studied, there are still many problems that have not been well addressed, especially when both of them occur simultaneously. In this section, we will analyze open problems in dynamic optimization of path-constrained switched systems, which are also motivations of this book. We first discuss the related research from the perspective of indirect and direct methods. Indirect methods for switched systems mainly utilizes variation techniques or maximum principle. However, indirect methods suffer from difficulties in solving Euler-Lagrange equations when dealing with path constraints or ODEs. For example, it is difficult to find an appropriate initial guess of the adjoint equation. In addition, for the case with path constraints, the adjoint equation becomes discontinuous, and the necessary conditions then become a multi-point boundary-value problem. Indirect methods requires prior knowledge of the structure of the optimal solution (e.g., the number of time intervals where constraints are active) when solving such problems [40, 41]. Therefore, compared to indirect methods, direct methods is more suitable for solving dynamic optimization problems of path-constrained switched systems. However, there are still many difficulties in solving dynamic optimization problems of path-constrained switching systems by employing direct methods. For example, for the most commonly used two-stage optimization method [88, 102, 103, 125], in order to find the optimal switch times, the method requires analyzing the differentiability of the optimal value function in Stage 1 and calculating its derivatives. Existing results only consider the unconstrained case [104]. However, as pointed out in [62], it is difficult to investigate differentiability properties of the optimal value function when the problem has state and/or input constraints, because the feasible set is unfixed and subject to perturbations [126]. Thus, questions naturally arise: under what conditions, if exist, is the optimal value function differentiable for path-constrained switched systems, and how to calculate its derivative when differ-

24

1 Introduction

entiable? These questions have not yet been addressed. Apart from the two-stage optimization framework, time scaling transformations [26, 119–121] are also capable of optimizing switch times, but they optimize the input under the discretization criterion of corresponding switch times, which may cause loss of the search space. How to find the optimal switching sequence has been one of the most challenging task due to the fact that the switching sequence is a discrete decision variable and is not expressed explicitly in the system dynamic. In two-stage optimization, existing methods of switching sequence optimization for Stage 2 may be inefficient due to a possible infinite-loop procedure at each step of the algorithms [110]. Apart from the two-stage optimization framework, infinite termination also occurs when the transformed problem does not have bang-bang type solutions in embedding techniques [114–118]. Mixed-integer programming approaches based on the decomposition framework [122–124] can find the optimal input, switch times, and switching sequence, but its master problem is a MIDO problem, which requires relaxation of ODEs coming along with an expensive computational complexity. In addition to the optimality, finite convergence and guaranteed feasibility should also be considered for the dynamic optimization of path-constrained switched systems. Most of existing results are based on methods such as simultaneous adaptive collocation, which only enforce path constraints to hold at collocation points [122– 124]. These methods require enough discrete time point constraints to approximate the path constraint, and there is no guarantee on the approximation error. Therefore, constraint violation can occur at any point other than those where the point constraints are enforced, which is a common occurrence in many path-constrained optimization problems. Moreover, multiple conflicting decisions often have to be made in practice, Pareto solutions are required to be obtained to provide references for decision makers. Most of existing deterministic multi-objective optimization methods are based on the assumption that a global solution can be located for the single-objective optimization problem [127, 128]. However, in practice even the most advanced solvers cannot handle the global optimization of path-constrained dynamic systems. Ideally, only an approximate KKT (Karush-Kuhn -Tucker, KKT) solution satisfying specified tolerances can be located. Thus how to solve MODO problems when global optimality cannot be attained? How optimal is the Pareto solution is when only approximate KKT solutions satisfying specified tolerances can be located? What is the approximation error in the sense of Pareto optimality? These are also the questions we want to address. Another challenge in deterministic multi-objective optimization is how to avoid generating weak Pareto solutions. For weak Pareto solutions, there does not exist other solutions that optimize all the objective functions at the same time, but there may exist solutions that optimize some of the objective functions while keeping others unchanged. So weak Pareto solutions can provide wrong references to decision makers. For deterministic multi-objective optimization methods, many factors may lead to the occurrence of weak Pareto solutions. Taking the -constraint method as an example, weak Pareto solutions may be generated if (1) the optimal set of the

References

25

single-objective optimization problem is not a singleton or, (2) the  constraint is inactive at the optimal solution of the single-objective optimization problem. These circumstances are very common in non-convex optimization problems.

References 1. Lee, T.C., Jiang, Z.P.: Uniform asymptotic stability of nonlinear switched systems with an application to mobile robots. IEEE Trans. Autom. Control 53(5), 1235–1252 (2008) 2. Liang, M., Wang, B., Yan, T.: Dynamic optimization of robot arm based on flexible multi-body model. J. Mech. Sci. Technol. 31(8), 3747–3754 (2017) 3. Li, Z., Yuan, W., Chen, Y., Ke, F., Chu, X., Philip Chen, C.L.: Neural-dynamic optimizationbased model predictive control for tracking and formation of nonholonomic multirobot systems. IEEE Transactions on Neural Networks and Learning Systems 29(12), 6113–6122 (2018) 4. Sheng, Z., Sun, Z., Molazadeh, V., Sharma, N.: Switched control of an n-degree-of-freedom input delayed wearable robotic system. Automatica 125, 109455 (2021) 5. Benmansour, K., Benalia, A., Djemai, M., De Leon, J.: Hybrid control of a multicellular converter. Nonlinear Anal. Hybrid Syst 1(1), 16–29 (2007) 6. Liu, C., Gong, Z., Lay Teo, K., Feng, E.: Multi-objective optimization of nonlinear switched time-delay systems in fed-batch process. Appl. Math. Model. 40(23–24), 10533–10548 (2016) 7. Venayak, N., Raj, K., Jaydeep, R., Mahadevan, R.: An optimized bistable metabolic switch to decouple phenotypic states during anaerobic fermentation. ACS Synth. Biol. 7(12), 2854– 2866 (2018) 8. Srinivasan, B., Palanki, S., Bonvin, D.: Dynamic optimization of batch processes: I. characterization of the nominal solution. Computers Chemical Engineering 27(1), 1–26 (2003) 9. Rippin, D.W.T.: Simulation of single-and multiproduct batch chemical plants for optimal design and operation. Computers & Chemical Engineering 7(3), 137–156 (1983) 10. Raja, L.L., Kee, R.J., Serban, R., Petzold, L.R.: Computational algorithm for dynamic optimization of chemical vapor deposition processes in stagnation flow reactors. J. Electrochem. Soc. 147(7), 2718 (2000) 11. Liberzon, D., Tempo, R.: Common lyapunov functions and gradient algorithms. IEEE Trans. Autom. Control 49(6), 990–994 (2004) 12. Vargas, A.N., Costa, E.F., Acho, L., do Val, J.B.R.: Switching stochastic nonlinear systems with application to an automotive throttle. IEEE Trans. Autom. Control 63(9), 3098–3104 (2017) 13. Kirches, C., Sager, S., Bock, H.G., Schlöder, J.P.: Time-optimal control of automobile test drives with gear shifts. Optimal Control Applications and Methods 31(2), 137–153 (2010) 14. Martin, R.B.: Optimal control drug scheduling of cancer chemotherapy. Automatica 28(6), 1113–1123 (1992) 15. Goto, H., Endo, K., Suzuki, M., Sakai, Y., Kanao, T., Hamakawa, Y., Hidaka, R., Yamasaki, M., Tatsumura, K.: High-performance combinatorial optimization based on classical mechanics. Sci. Adv. 7(6) (2021) 16. Feehery, W.F., Barton, P.I.: Dynamic optimization with state variable path constraints. Computers & Chemical Engineering 22(9), 1241–1256 (1998) 17. Srinivasan, B., Palanki, S., Bonvin, D.: Dynamic optimization of batch processes: I. characterization of the nominal solution. Comput. Chem. Eng. 27(1), 1–26 (2003) 18. Hull, D.G.: Optimal control theory for applications. Springer, New York, NY, USA (2003) 19. Dreyfus, S.: Variational problems with state variable inequality constraints. Technical report, RAND CORP SANTA MONICA CALIF (1962) 20. Kelly, H., Kopp, R., Moyer, H.: A trajectory optimization technique based upon the theory of the second variation. In: Astrodynamics Conference, pp. 415 (1963)

26

1 Introduction

21. Denham, W.F., Bryson, A.E., Jr.: Optimal programming problems with inequality constraints. ii-solution by steepest-ascent. AIAA J. 2(1), 25–34 (1964) 22. Bryson, A.E., Ho, Y-C.: Applied optimal control: optimization, estimation, and control. Routledge (2018) 23. Dixon, L.C.W., Bartholomew-Biggs, M.C.: Adjoint-control transformations for solving practical optimal control problems. Optimal Control Applications and Methods 2(4), 365–381 (1981) 24. Miele, A., Mohanty, B.P., Venkataraman, P., Kuo, Y.M.: Numerical solution of minimax problems of optimal control, part 2. J. Optim. Theory Appl. 38(1), 111–135 (1982) 25. Pesch, H.J.: Real-time computation of feedback controls for constrained optimal control problems. part 2: A correction method based on multiple shooting. Optimal Control Applications and Methods 10(2), 147–171 (1989) 26. Teo, K.L., Goh, C., Wong, K.: A unified computational approach to optimal control problems. Longman Scientific & Technical, New York, NY, USA (1991) 27. Vassiliadis, V.S., Sargent, R.W.H., Pantelides, C.C.: Solution of a class of multistage dynamic optimization problems. 1. problems without path constraints. Ind. Eng. Chem. Res. 33(9), 2111–2122 (1994) 28. Biegler, L.T., Cervantes, A.M., Wächter, A.: Advances in simultaneous strategies for dynamic process optimization. Chem. Eng. Sci. 57(4), 575–593 (2002) 29. Bellman, R.: Dynamic programming. Science 153(3731), 34–37 (1966) 30. Dadebo, S.A., McAuley, K.B.: Dynamic optimization of constrained chemical engineering problems using dynamic programming. Comput. Chem. Eng. 19(5), 513–525 (1995) 31. Luus, R.: Iterative dynamic programming. Chapman and Hall/CRC (2019) 32. Banga, J.R., Irizarry-Rivera, R., Seider, W.D.: Stochastic optimization for optimal and modelpredictive control. Comput. Chem. Eng. 22(4–5), 603–612 (1998) 33. Levenberg, K.: A method for the solution of certain non-linear problems in least squares. Q. Appl. Math. 2(2), 164–168 (1944) 34. Curry, H.B.: The method of steepest descent for non-linear minimization problems. Q. Appl. Math. 2(3), 258–261 (1944) 35. Kirk, D.E.: Optimal Control Theory: An Introduction. Prentice-Hall networks series. Prentice Hall, first printing edition (1970) 36. Cauchy, A., et al.: Méthode générale pour la résolution des systemes d’équations simultanées. C. R. Hebd. Seances Acad. Sci. 25(1847), 536–538 (1847) 37. Hadamard, J.: Mémoire sur le problème d’analyse relatif à l’équilibre des plaques élastiques encastrées, vol. 33. Imprimerie nationale, Paris, France (1908) 38. Courant, R.: Variational methods for the solution of problems of equilibrium and vibrations. Bull. Am. Math. Soc. 49(1), 1–23 (1943) 39. McGill, R., Kenneth, P.: A convergence theorem on the iterative solution of nonlinear twopoint boundary value systems. In: Proceedings of XIV International Astronautical Congress, Paris, IV, vol. 12, pp. 173–188 (1963) 40. Chachuat, B.: Nonlinear and dynamic optimization: From theory to practice. Technical report, Ècole Polytechnique Fédérale de Lausanne (2007) 41. Büskens, C., Maurer, H.: Sqp-methods for solving optimal control problems with control and state constraints: adjoint variables, sensitivity analysis and real-time control. J. Comput. Appl. Math. 120(1–2), 85–108 (2000) 42. Brusch, R.G., Schappelle, R.H.: Solution of highly constrained optimal control problems using nonlinear programing. AIAA J. 11(2), 135–136 (1973) 43. Morison, K.R., Sargent, R.W.H.: Optimization of multistage processes described by differential-algebraic equations. In: Numerical Analysis, pp. 86–102. Springer (1986) 44. Vassiliadis, V.S., Sargent, R.W.H., Pantelides, C.C.: Solution of a class of multistage dynamic optimization problems. 2. problems with path constraints. Ind. Eng. Chem. Res. 33(9), 2123– 2133 (1994) 45. Lynn, L.L., Parkin, E.S., Zahradnik, R.L.: Near-optimal control by trajectory approximation. Tubular reactors with axial dispersion. Ind. Eng. Chem. Fundam. 9(1), 58–63 (1970)

References

27

46. Neuman, C.P., Sen, A.: A suboptimal control algorithm for constrained problems using cubic splines. Automatica 9(5), 601–613 (1973) 47. Tsang, T.H., Himmelblau, D.M., Edgar, T.F.: Optimal control via collocation and non-linear programming. Int. J. Control 21(5), 763–768 (1975) 48. Leis, J.R., Kramer, M.A.: The simultaneous solution and sensitivity analysis of systems described by ordinary differential equations. ACM Trans. Math. Software (TOMS) 14(1), 45–60 (1988) 49. Yen, V., Nagurka, M.: Optimal control of linearly constrained linear systems via state parametrization. Optim. Control Appl. Methods 13(2), 155–167 (1992) 50. Gill, P.E., Murray, W., Saunders, M.A.: Large-scale sqp methods and their application in trajectory optimization. In: Computational optimal control, pp. 29–42. Springer (1994) 51. Tanartkit, P., Biegler, L.T.: Stable decomposition for dynamic optimization. Ind. Eng. Chem. Res. 34(4), 1253–1266 (1995) 52. Vassiliadis, V.: Computational solution of dynamic optimization problems with general differential-algebraic constraints. PhD thesis, Imperial College London (University of London) (1993) 53. Feehery, W.F.: Dynamic optimization with path constraints. PhD thesis, Massachusetts Institute of Technology (1998) 54. Hartl, R.F., Sethi, S.P., Vickson, R.G.: A survey of the maximum principles for optimal control problems with state constraints. SIAM Rev. 37(2), 181–218 (1995) 55. Pontryagin, L.S.: The mathematical theory of optimal processes. L.S. Pontryagin selected works 4 Classics of Soviet mathematics [v. 1]. Gordon and Breach Science Publishers, English ed edition (1986) 56. Jacobson, D., Lele, M.: A transformation technique for optimal control problems with a state variable inequality constraint. IEEE Trans. Autom. Control 14(5), 457–464 (1969) 57. Xing, A.-Q., Wang, C.-L.: Applications of the exterior penalty method in constrained optimal control problems. Optim. Control Appl. Methods 10(4), 333–345 (1989) 58. Park, T., Barton, P.I.: State event location in differential-algebraic models. ACM Trans. Modeling Comput. Simul. (TOMACS) 6(2), 137–165 (1996) 59. Galán, S., Feehery, W.F., Barton, P.I.: Parametric sensitivity functions for hybrid discrete/continuous systems. Appl. Numer. Math. 31(1), 17–47 (1999) 60. Sargent, R.W.H., Sullivan, G.R.: The development of an efficient optimal control package. In: Optimization Techniques, pp. 158–168. Springer (1978) 61. Chen, T.W.C., Vassiliadis, V.S.: Inequality path constraints in optimal control: a finite iteration ε-convergent scheme based on pointwise discretization. J. Process Control 15(3), 353–362 (2005) 62. Zhu, F., Antsaklis, P.J.: Optimal control of hybrid switched systems: A brief survey. Discrete Event Dynam. Syst. 25(3), 345–364 (2015) 63. Antsaklis, P.J.: A brief introduction to the theory and applications of hybrid systems. In: Proc IEEE, Special Issue on Hybrid Systems: Theory and Applications. Citeseer (2000) 64. Borrelli, F.: Constrained Optimal Control of Linear and Hybrid Systems. Lecture Notes in Control and Information Sciences 290. Springer, 1 edition (2003) 65. Samar, R.: Robust multi-mode control of high performance aero-engines. PhD thesis, University of Leicester (1995) 66. Grossman, R.L., Nerode, A., Ravn, A.P., Rischel, H.: Hybrid Systems. Lecture Notes in Computer Science. Springer, Berlin, Heidelberg (1993) 67. Antsaklis, P., Kohn, W., Nerode, S.S.A.: Hybrid Systems II. Lecture Notes in Computer Science. Springer, Berlin, Heidelberg (1995) 68. Alur, R., Henzinger, T.A., Sontag, E.D.: Hybrid Systems III. Lecture Notes in Computer Science. Springer, Berlin, Heidelberg (1996) 69. Antsaklis, P., Kohn, W., Nerode, S.S.A.: Hybrid Systems IV. Lecture Notes in Computer Science. Springer, Berlin, Heidelberg (1997) 70. Antsaklis, P., Lemmon, M., Kohn, W., Nerode, S.S.A.: Hybrid Systems V. Lecture Notes in Computer Science. Springer, Berlin, Heidelberg (1999)

28

1 Introduction

71. Henzinger, T.A., Sastry, S.: Hybrid Systems: Computation and Control. Lecture Notes in Computer Science. Springer, Berlin, Heidelberg (1998) 72. Bemporad, A., Bicchi, A., Buttazzo, G.: Hybrid Systems: Computation and Control. Lecture Notes in Computer Science. Springer, Berlin, Heidelberg (2007) 73. Egerstedt, M., Mishra, B.: Hybrid Systems: Computation and Control. Lecture Notes in Computer Science. Springer, Berlin, Heidelberg (2008) 74. Majumdar, R., Tabuada, P.: Hybrid Systems: Computation and Control. Lecture Notes in Computer Science. Springer, Berlin, Heidelberg (2009) 75. Antsaklis, P.J., Nerode, A.: Hybrid control systems: An introductory discussion to the special issue. IEEE Trans. Autom. Control 43(4), 457–460 (1998) 76. Morse, A.S., Pantelides, C.C., Sastry, S.S., (Hans) Schumacher, J.M.: Editorial. Automatica, 35(3):347–348 (1999) 77. Branicky, M.S., Borkar, V.S., Mitter, S.K.: A unified framework for hybrid control: Model and optimal control theory. IEEE Trans. Autom. Control 43(1), 31–45 (1998) 78. Hedlund, S., Rantzer, A.: Optimal control of hybrid systems. In: Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No. 99CH36304), vol. 4, pp. 3972–3977. IEEE (1999) 79. Hedlund, S., Rantzer, A.: Convex dynamic programming for hybrid systems. IEEE Trans. Autom. Control 47(9), 1536–1540 (2002) 80. Rantzer, A., Johansson, M.: Piecewise linear quadratic optimal control. IEEE Trans. Autom. Control 45(4), 629–637 (2000) 81. Sussmann, H.J.: Set-valued differentials and the hybrid maximum principle. In: Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No. 00CH37187), vol. 1, pp. 558–563. IEEE (2000) 82. Piccoli, B.: Necessary conditions for hybrid optimization. In: Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No. 99CH36304), vol. 1, pp. 410–415. IEEE (1999) 83. Riedinger, P., Kratz, F., Iung, C., Zanne, C.: Linear quadratic optimization for hybrid systems. In: Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No. 99CH36304), vol. 3, pp. 3059–3064. IEEE (1999) 84. Shaikh, M.S., Caines, P.E.: On trajectory optimization for hybrid systems: Theory and algorithms for fixed schedules. In: Proceedings of the 41st IEEE Conference on Decision and Control, 2002, vol. 2, pp. 1997–1998. IEEE (2002) 85. Shahid Shaikh, M., Caines, P.E.: On the optimal control of hybrid systems: Optimization of trajectories, switching times, and location schedules. In: International Workshop on Hybrid Systems: Computation and Control, pp. 466–481. Springer (2003) 86. Caines P.E., Shahid Shaikh, M.: Optimality zone algorithms for hybrid systems computation and control: From exponential to linear complexity. In: Proceedings of the 2005 IEEE International Symposium on, Mediterrean Conference on Control and Automation Intelligent Control, 2005, pp. 1292–1297. IEEE (2005) 87. Caines, P.E., Shahid Shaikh, M.: Optimality zone algorithms for hybrid systems: Efficient algorithms for optimal location and control computation. In: International Workshop on Hybrid Systems: Computation and Control, pp. 123–137. Springer (2006) 88. Xu, X., Antsaklis, P.J.: Optimal control of switched systems: new results and open problems. In Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No. 00CH36334), vol. 4, pp. 2683–2687. IEEE (2000) 89. Egerstedt, M., Wardi, Y., Delmotte, F.: Optimal control of switching times in switched dynamical systems. In: 42nd Ieee International Conference on Decision and Control (ieee Cat. No. 03ch37475), vol. 3, pp. 2138–2143. IEEE (2003) 90. Egerstedt, M., Wardi, Y., Axelsson, H.: Transition-time optimization for switched-mode dynamical systems. IEEE Trans. Autom. Control 51(1), 110–115 (2006) 91. Azuma, S., Egerstedt, M., Wardi, Y.: Output-based optimal timing control of switched systems. In: International Workshop on Hybrid Systems: Computation and Control, pp. 64–78. Springer (2006)

References

29

92. Egerstedt, M., Azuma, S., Wardi, Y.: Optimal timing control of switched linear systems based on partial information. Nonlinear Anal.: Theory, Methods Appl. 65(9), 1736–1750 (2006) 93. Wardi, Y., Ding, X.C., Egerstedt, M., Azuma, S.: On-line optimization of switched-mode systems: Algorithms and convergence properties. In: 2007 46th IEEE Conference on Decision and Control, pp. 5005–5010. IEEE (2007) 94. Ding, X.C., Wardi, Y., Egerstedt, M.: On-line adaptive optimal timing control of switched systems. In: Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference, pp. 5305–5310. IEEE (2009) 95. Ding, X.-C., Wardi, Y., Egerstedt, M.: On-line optimization of switched-mode dynamical systems. IEEE Trans. Autom. Control 54(9), 2266–2271 (2009) 96. Wardi, Y., Twu, P., Egerstedt, M.: On-line optimal timing control of switched systems. In: 49th IEEE Conference on Decision and Control (CDC), pp. 2137–2142. IEEE(2010) 97. Johnson, E.R., Murphey, T.D.: Second order switching time optimization for time-varying nonlinear systems. In: Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference, pp. 5281–5286. IEEE (2009) 98. Johnson, E.R., Murphey, T.D.: Second-order switching time optimization for nonlinear timevarying dynamic systems. IEEE Trans. Autom. Control 56(8), 1953–1957 (2011) 99. Caldwell, T.M., Murphey, T.D.: An adjoint method for second-order switching time optimization. In: 49th IEEE Conference on Decision and Control (CDC), pp. 2155–2162. IEEE (2010) 100. Giua, A., Seatzu, C., Van Der Mee, C.: Optimal control of switched autonomous linear systems. In: Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No. 01CH37228), vol. 3, pp. 2472–2477. IEEE (2001) 101. Giua, A., Seatzu, C., Van Der Mee, C.: Optimal control of autonomous linear systems switched with a pre-assigned finite sequence. In: Proceeding of the 2001 IEEE International Symposium on Intelligent Control (ISIC’01)(Cat. No. 01CH37206), pp. 144–149. IEEE (2001) 102. Xu, X., Antsaklis, P.J.: Optimal control of switched systems via non-linear optimization based on direct differentiations of value functions. Int. J. Control 75(16–17), 1406–1426 (2002) 103. Xu, X., Antsaklis, P.J.: Optimal control of switched systems based on parameterization of the switching instants. IEEE Trans. Autom. Control 49(1), 2–16 (2004) 104. Kamgarpour, M., Tomlin, C.: On optimal control of non-autonomous switched systems with a fixed mode sequence. Automatica 48(6), 1177–1181 (2012) 105. Axelsson, H., Wardi, Y., Egerstedt, M.: Transition-time optimization for switched systems. IFAC Proc. Vol. 38(1), 453–458 (2005) 106. Axelsson, H., Wardi, Y., Egerstedt, M., Verriest, E.I.: Gradient descent approach to optimal mode scheduling in hybrid dynamical systems. J. Optim. Theory Appl. 136(2), 167–186 (2008) 107. Axelsson, H., Wardi, Y., Egerstedt, M., Verriest, E.: A provably convergent algorithm for transition-time optimization in switched systems. In: Proceedings of the 44th IEEE Conference on Decision and Control, pp. 1397–1402. IEEE (2005) 108. Gonzalez, H., Vasudevan, R., Kamgarpour, M., Sastry, S.S., Bajcsy, R., Tomlin, C.: A numerical method for the optimal control of switched systems. In: 49th IEEE Conference on Decision and Control (CDC), pp. 7519–7526. IEEE (2010) 109. Gonzalez, H., Vasudevan, R., Kamgarpour, M., Sastry, S.S., Bajcsy, R., Tomlin, C.J.: A descent algorithm for the optimal control of constrained nonlinear switched dynamical systems. In: Proceedings of the 13th ACM international conference on Hybrid systems: computation and control, pp. 51–60 (2010) 110. Wardi, Y., Egerstedt, M.: Algorithm for optimal mode scheduling in switched systems. In: 2012 American Control Conference (ACC), pp. 4546–4551. IEEE (2012) 111. Bemporad, A., Giua, A., Seatzu, C.: An iterative algorithm for the optimal control of continuous-time switched linear systems. In: Sixth International Workshop on Discrete Event Systems, 2002. Proceedings, pp. 335–340. IEEE (2002) 112. Bemporad, A., Giua, A., Seatzu, C.: A master-slave algorithm for the optimal control of continuous-time switched affine systems. In: Proceedings of the 41st IEEE Conference on Decision and Control, 2002, vol. 2, pp. 1976–1981. IEEE (2002)

30

1 Introduction

113. Caldwell, T.M., Murphey, T.D.: Projection-based switched system optimization. In: 2012 American Control Conference (ACC), pp. 4552–4557. IEEE (2012) 114. Das, T., Mukherjee, R.: Optimally switched linear systems. Automatica 44(5), 1437–1441 (2008) 115. Mojica-Nava, E., Meziat, R., Quijano, N., Gauthier, A., Rakoto-Ravalontsalama, N.: Optimal control of switched systems: A polynomial approach. IFAC Proc. Vol. 41(2), 7808–7813 (2008) 116. Mojica-Nava, E., Quijano, N., Rakoto-Ravalontsalama, N.: A polynomial approach for optimal control of switched nonlinear systems. Int. J. Robust Nonlinear Control 24(12), 1797– 1808 (2014) 117. Bengea, S.C., DeCarlo, R.A.: Optimal control of switching systems. Automatica 41(1), 11–27 (2005) 118. Wei, S., Uthaichana, K., Žefran, M., DeCarlo, R.A., Bengea, S.: Applications of numerical optimal control to nonlinear hybrid systems. Nonlinear Anal. Hybrid Syst 1(2), 264–279 (2007) 119. Feng, Z.G., Teo, K.L., Rehbock, V.: A discrete filled function method for the optimal control of switched systems in discrete time. Optim. Control Appl. Methods 30(6), 585–593 (2009) 120. Xiang, W., Zhang, K., Sun, C.: Numerical algorithm for a class of constrained optimal control problems of switched systems. Numer. Algorithms 67(4), 771–792 (2014) 121. Xiang, W., Zhang, K., Cheng, M.: Computational method for optimal control of switched systems with input and state constraints. Nonlinear Anal. Hybrid Syst 26, 1–18 (2017) 122. Bestehorn, F., Hansknecht, C., Kirches, C., Manns, P.: Mixed-integer optimal control problems with switching costs: a shortest path approach. Math. Program. 188(2), 621–652 (2021) 123. Bock, H.G., Kirches, C., Meyer, A., Potschka, A.: Numerical solution of optimal control problems with explicit and implicit switches. Optim. Methods Softw. 33(3), 450–474 (2018) 124. Kirches, C., Kostina, E., Meyer, A., Schlöder, M., PN SPP1962.: Numerical solution of optimal control problems with switches, switching costs and jumps. Optim. Online 6888 (2018) 125. Xu, X., Antsaklis, P.J.: Results and perspectives on computational methods for optimal control of switched systems. In International Workshop on Hybrid Systems: Computation and Control, pp. 540–555. Springer (2003) 126. Bonnans, J.F., Shapiro, A.: Perturbation analysis of optimization problems. Springer Science & Business Media, New York (2013) 127. Ehrgott, M., Ruzika, S.: Improved ε-constraint method for multiobjective programming. J. Optim. Theory Appl. 138(3), 375–396 (2008) 128. Mavrotas, G.: Effective implementation of the ε-constraint method in multi-objective mathematical programming problems. Appl. Math. Comput. 213(2), 455–465 (2009)

Chapter 2

Bi-level Dynamic Optimization of Path-Constrained Switched Systems

This chapter considers the dynamic optimization problem of path-constrained switched systems with a fixed switching sequence. Our main goal is to design a dynamic optimization algorithm, which can find the optimal input and switch times with guaranteed satisfaction of path constraints within a finite number of iterations. Such kind of problem is quite common during the acceleration process of vehicles, where the switching sequence of the gear is pre-defined and all we need is to find the best fuel feeding and shift timing. Meanwhile, the engine speed is required to be ensured within the safe range. In the previous section, we mentioned that bilevel algorithm is an effective way to solve such problem. However, under what conditions, if exist, is the optimal value function differentiable for path-constrained switched systems, and how to calculate its derivative when differentiable? These are the limitations to the application of bi-level algorithms in the dynamic optimization problem of constrained switched systems. From the motivations above, first, a method is proposed to reduce path constraints into a family of abstract constraints in terms of the input and the switch time vector, such that the path-constrained dynamic the path-constrained problem is transformed into an approximate parameterized optimization problem depending on switch time vector. Then differentiability of the optimal value function is analyzed by using perturbation analysis of optimization problems, and under certain conditions the derivative of the optimal value function is proved equal to the derivative of the cost function evaluated at the optimal input for a given switch time vector. Based on the theoretical results above, a bi-level algorithm is proposed to find the optimal input and switch times with guaranteed feasibility of path constraints, followed by a finite convergence proof of the bi-level algorithm. Finally, a numerical example is provided to illustrate the effectiveness of the proposed algorithm.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Fu and C. Zhang, Dynamic Optimization of Path-Constrained Switched Systems, Studies in Systems, Decision and Control 459, https://doi.org/10.1007/978-3-031-23428-6_2

31

32

2 Bi-level Dynamic Optimization of Path-Constrained Switched Systems

2.1 Definitions and Preliminary Results The dynamic optimization of path-constrained switched systems involves perturbation analysis of non-convex optimization problems. To facilitate understanding, general concepts and theories of functional analysis in normed spaces are presented in this section. The definition of SIP is also presented, which is an important kind of optimization to which path-constrained optimization problems belong. For some conclusions we do not give the detailed proofs, the interested reader is referred to the references cited.

2.1.1 Differentiability of Functionals in Normed Spaces Let X and Y be vector (linear) normed spaces and consider a mapping g : X → Y . Definition 2.1 ([1]) A mapping g : X → Y is said to be directionally differentiable at a point x along a direction h if the limit g  (x; h) := lim t↓0

g(x + th) − g(x) t

(2.1)

exists; this limit, denote g  (x; h), is called the directional derivative of g. If g is directionally differentiable at x in every direction h ∈ X , we say that g is directionally differentiable at x. The directional derivative g  (x; h), when it exists, is positive homogeneous in h, i.e., g  (x; th) = t f  (x; h) for any t > 0. Furthermore, g is said to be Gäteaux differentiable at a point x if g is directionally differentiable at a point x and the directional derivative g  (x; h) is linear and continuous in h. Another relatively strong definition of differentiability is in the Hadamard sense. Definition 2.2 ([1]) A mapping g : X → Y is said to be directionally differentiable at a point x in the Hadamard sense if the directional derivative g  (x; h) exists for all h and, moreover, g  (x; h) := lim

t↓0 h  →h

g(x + th  ) − g(x) t

(2.2)

If in addition g  (x; h) is linear in h, g is Hadamard differentiable at x. Equation (2.2) can be formulated in the following equivalent form, for any sequences h n → h and tn ↓ 0: g(x + tn h n ) − g(x) n→∞ tn

g  (x; h) = lim

(2.3)

2.1 Definitions and Preliminary Results

33

Another frequently used definition of differentiability is in the Fréchet sense. Definition 2.3 ([1]) A mapping g : X → Y is said to be directionally differentiable at x in the Fréchet sense if g is directionally differentiable at x and g(x + h) = g(x) + g  (x; h) + o(h), h ∈ X

(2.4)

If in addition, g  (x; h) is linear and continuous, it is said that g is Fréchet differentiable at x. The relationship between these differential maps is as follows: For continuously differentiable mappings, Fréchet differentiability is equivalent to Gäteaux differentiability. If the space X is finite dimensional, then Hadamard directionally differentiable implies Fréchet directionally differentiable. If g is Fréchet directionally differentiable and g  (x; ) is continuous, then the converse is also true. In addition, if X is finite dimensional and g is locally Lipschitz continuous, then the Fréchet directionally differentiability and Hadamard directionally differentiability are equivalent [2].

2.1.2 Perturbation Analysis of Constrained Optimization Problems To analyze differentiability of the optimal value function of the optimal control problem for path-constrained switched systems, we briefly review the material of [1] that introduce the stability and sensitivity analysis of constrained systems with maps between Banach Spaces in this section. First, we consider a parameterized optimization problem of the form: min J (s, u) u∈U

s.t. G(s, u) ∈ K

(2.5)

depending on the parameter vector s ∈ S. We assume here that S, U and Y are Banach Spaces, K is a closed convex set of Y. J : S × U → R and G : S × U → Y are continuous mappings. We define ˜ Φ(s) := {u ∈ U : G(s, u) ∈ K}

(2.6)

˜ Ω(s) := arg min J (s, u)

(2.7)

and the associated set ˜ u∈Φ(s)

For a given point s¯ , in the parameter space S, we view the corresponding problem as an unperturbed problem (the feasible set is fixed), and investigate continuity and differentiability properties of the optimal value function

34

2 Bi-level Dynamic Optimization of Path-Constrained Switched Systems

v(s) ˜ := inf J (s, u) ˜ u∈Φ(s)

(2.8)

of optimal solutions in the vicinity of the given point s¯ . Definition 2.4 ([1, p. 147]) uˆ is said to be an ε-optimal solution of problem (2.5) if uˆ is feasible, and J (s, u) ˆ ≤ v(s) ˜ + ε. Then we discuss stability of feasible sets defined by abstract constraints in the form of (2.2). For s ∈ S, consider the multifunction Fs (u) := G(s, u) − K associated with ˜ the mapping G(s, u). Obviously, a point u is in Φ(s) iff 0 ∈ Fs (u). Assume that G(s, u) is Fréchet differentiable. This implies that G(s, u) is differentiable in u and that Du G(s, u) is continuous (in the operator norm topology) jointly in s and u. Multifunction Fs is approximated by linearizing the mapping G(¯s , ·) at ˜ s ): the point u¯ ∈ Φ(¯ ¯ + Du G(¯s , u)(u ¯ − u) ¯ −K F ∗ (u) = G(¯s , u)

(2.9)

Definition 2.5 (Robinson’s constraint qualification) [1, p. 64] We say that Robinson’s constraint qualification holds at a point u¯ ∈ U such that G(¯s , u) ¯ ∈ K, with respect to the mapping G(¯s , ·) and the set K, if ¯ − K} 0 ∈ int{G(¯s , u) ¯ + Du G(¯s , u)U

(2.10)

is satisfied. For a given (direction) d in the parameter space S, we consider a path s(δ) := s¯ + δd + o(δ), with δ ∈ R+ . We investigate the problem of constructing a feasible ˜ path u(δ) ∈ Φ(s(δ)). Definition 2.6 ([1, p. 266]) We say that h ∈ U is a first order feasible direction at s¯ , relative to the direction d ∈ S, if for any path s(δ) := s¯ + δd + o(δ) in the parameter ˜ space S there exists r (δ) = o(δ), δ ≥ 0, such that u¯ + δh + r (δ) ∈ Φ(s(δ)). Now a directional regularity condition that depends on the perturbation direction d is stated as follows. ˜ s ). We say Definition 2.7 (Directional regularity condition) ([1, p. 267]) Let u¯ ∈ Φ(¯ that the directional regularity condition holds at u¯ in a direction d ∈ S if Robinson’s constraint qualification is satisfied at the point (0, u) ¯ for the mapping ¯ G(u, t) := (G(¯s + td, u), t) : U × R → Y × R

(2.11)

and with respect to the set K × R+ ⊂ Y × R. The characterizations below are useful for understanding and verifying directional regularity in particular situations.

2.1 Definitions and Preliminary Results

35

Lemma 2.8 ([1, p. 267]) The following statements hold. (1) The directional regularity condition is equivalent to: 0 ∈ int{G(¯s , u) ¯ + DG(¯s , u)(R ¯ + (d), U) − K}

(2.12)

where R+ (d) := {δd : δ ≥ 0}. (2) Robinson’s constraint qualification (2.10) implies the directional regularity condition for any direction d. (3) If the directional regularity condition holds, then h ∈ U is a feasible direction iff it satisfies ¯ DG(¯s , u)(d, ¯ h) ∈ TK (G(¯s , u))

(2.13)

¯ denotes the contingent cone to the set K at the point where TK (G(¯s , u)) G(¯s , u) ¯ ∈ K. Remark 2.9 The concept of directional regularity condition depends on the chosen direction d, while Robinson’s constraint qualification does not. Moreover, as indicated in [1, p. 71], if Φ˜ is defined by a finite number of linearly independent equality and active inequality constraints, then Robinson’s constraint qualification is equivalent to the Mangasarian-Fromovitz constraint qualification (MFCQ). This theoretical result is supported by the following lemma. Let U and Y be Banach Spaces, G : U → Y a continuously differentiable mapping, and K a closed convex subset of Y. Then for a point u¯ ∈ U, Robinson’s constraint qualification implies that 0 ∈ int{G(u) ¯ + DG(u)U ¯ − K}. We now present the constraints in a product form, i.e., Y is the Cartesian product of two Banach Spaces Y1 and Y2 , and K = K1 × K2 ⊂ Y1 × Y2 , where K1 and K2 are close convex subset of Y1 and Y2 , respectively. Then G(u) = (G 1 (u), G 2 (u)), with G i (u) ∈ Yi , i = 1, 2. Then we obtain the following result. ¯ is onto and that K 2 has a nonempty Lemma 2.10 ([1, p. 70]) Suppose that DG 1 (u) interior. Then Robinson’s constraint qualification is equivalent to existence of h ∈ U satisfying ¯ + DG 1 (u)h ¯ ∈ K1 , G 1 (u) G 2 (u) ¯ + DG 2 (u)h ¯ ∈ int(K2 )

(2.14)

We define Φ by a finite number of linearly independent equality and inequality constraints as Φ = {u : gi (u) = 0, i = 1, . . . , q; gi (u) ≤ 0, i = q + 1, . . . , p}

(2.15)

If K1 = {0}, i.e., if the first constraint is an equality type constraint, and K2 has a nonempty interior, then Robinson’s constraint qualification is equivalent to the MFCQ:

36

2 Bi-level Dynamic Optimization of Path-Constrained Switched Systems

Dgi (u), ¯ i = 1, . . . , q are linearly independent, ∃h ∈ U : Dgi (u)h ¯ = 0, i = 1, . . . , q, Dgi (u)h ¯ < 0, ∀i ∈ A(u) ¯

(2.16)

¯ = 0 | i ∈ {q + 1, . . . , p}}. where A(u) ¯ = {i : gi (u) For a given direction d ∈ S, we consider the following linearization of problem (2.5): min D J (¯s , u)(d, ¯ h) h∈U (2.17) ¯ s.t. DG(¯s , u)(d, ¯ h) ∈ TK (G(¯s , u)) We denote val(P L d ) and val(DL d ) by the optimal values of the linearization and its dual program of problem (2.5). Lemma 2.11 ([1, p. 278]) Let the directional regularity condition (2.12) hold. Then val(P L d ) = val(DL d ) < +∞, and the common value of val(P L d ) = val(DL d ) is finite iff the set Λ(¯s , u) ¯ of Lagrange multipliers is nonempty. In that case, the set of optimal solutions of the linearization dual program is a nonempty, bounded and weakly-star compact subset of Λ(¯s , u). ¯ Remark 2.12 From Lemma 2.11 above, we can obtain that if the directional regularity condition holds, then val(P L d ) = val(DL d ) < +∞. This implies that the strong duality holds for the linearization and its dual programs of problem (2.5). ˜ s ), the set of dual multipliers coincides with the set of Lagrange Hence for any u ∈ Ω(¯ multipliers of problem (2.17). Clearly, let h be a feasible point of (2.17), if directional regularity qualification holds, then by (2.9) we have that there exists r (δ) = o(δ), δ ≥ 0, such that ˜ u(δ) := u¯ + δh + r (δ) ∈ Φ(s(δ))

(2.18)

Therefore, h is a feasible direction.

2.1.3 Semi-infinite Program SIPs are optimization problems consisting of finite-dimensional decision variables and infinite-dimensional constraints. For the sake of brevity, we consider only one inequality constraint here: min J (a) a

s.t. g(a, b) ≤ 0, ∀b ∈ B, a ∈ A ⊂ Rna where a = (a1 , a2 , . . . , an ) ∈ A ⊂ Rna , B is an infinite-dimensional set.

(2.19)

2.2 Problem Statement

37

Definition 2.13 ([3]) A point a ∈ A is called an SIP-Slater point if g(a s , b) < 0, ∀b ∈ B

(2.20)

From continuity of g and compactness of B, if a s is an SIP-Slater point, there exists s > 0, such that g(a s , b) ≤ −s , ∀b ∈ B

(2.21)

2.2 Problem Statement Consider a switched system consisting of a set of subsystems indexed by Q = {1, 2, . . . , n q }. The system undergoes n switches at switch times s1 , s2 , . . . , sn in the time interval T = [t0 , t f ]. We define the feasible set of switch times as: S := {s ∈ Rn : 0 ≤ s1 ≤ s2 . . . ≤ sn ≤ t f }

(2.22)

and s0 = t0 , sn+1 = t f . The system dynamic for each mode q ∈ Q is described by vector field f q : Rn x × Rn u → Rn x . The dynamic process of the state x ∈ Rn x is described by  si , si+1 ) , if i = n  x˙ = f ρi (x, u), t ∈  , sn , t f , if i = n

(2.23)

x(t0 ) = x0 where ρi ∈ Q, i = 0, 1, . . . , n. The variables x : T → X ⊆ Rn x and u : T → U ⊆ Rn u denote the state and input of the dynamic system, with X and U nonempty and compact. we denote by u the piecewise function (i.e., the element of the set U), and by u(t) the value of the piecewise function at a specific time t. Since the number of path constraints do not pose any further complication for the results herein, we only consider problems with one path constraint g : Rn x × Rn u → R defined as g(x, u) ≤ 0, ∀t ∈ [t0 , t f ]

(2.24)

Meanwhile, we define Φ(s) := {u ∈ U : g(x, u) ≤ 0} as the feasible set of the current optimization problem at a given point s ∈ S. We consider the path-constrained optimal control problem of the form:

38

2 Bi-level Dynamic Optimization of Path-Constrained Switched Systems

 si+1 lρi (x, u)dt i=0 si

n

min

J (s, u) =

s.t.

g(x, u) ≤ 0, ∀t ∈ [t0 , t f ],  si , si+1 ) , if i = n  x˙ = f ρi (x, u), t ∈  , sn , t f , if i = n x(t0 ) = x0 , i = 0, 1, . . . , n

s∈S,u∈U

+ ϕ(x(t f )) (2.25)

where lρi : Rn x × Rn u → R, i = 0, 1, . . . , n are referred to as mode-dependent running costs and ϕ : Rn x → R is referred to as the final cost. Obviously, S and U are Banach Spaces. We define Js (u) := J (s, u), the optimal value function v(s) := inf u∈Φ(s) Js (u), and its associated set Ω(s) := arg minu∈Φ(s) J (s, u). Du J (s, u) denotes the partial derivative of the mapping J : S × U → R with respect to u at the point (s, u) ∈ S × U. Ds J (s, u) denotes the partial derivative of the mapping J : S × U → R with respect to s at the point (s, u) ∈ S × U. Similarly, Dv(s) denotes the derivative of the mapping v : S → R at the point s ∈ S. Problem (2.25) is a typical optimal control problem in Bolza form. As in the usual practice of formulating optimal control problems, we make following assumptions on the dynamics, cost, and constraint of our optimal control problem: Assumption 2.14 For all q ∈ Q, J (a, u), f q (x, u) and lq (x, u) are Lipschitz and differentiable in their respective arguments. In addition, the derivatives of these functions in their respective arguments are also Lipschitz. Remark 2.15 Assumption 2.14 along with the measurability of the input u is sufficient to ensure the existence and uniqueness of the solution to the differential equations governed by f w(t) in each bounded interval [si , si+1 ], i = 0, 1, . . . , n [4]. The state of system (2.23) can be obtained by piecing together the solutions from the sub-intervals in which each single mode is active. In addition, this assumption ensures that the costate is well-defined and continuous. Assumption 2.16 x(t) and u(t) are continuous at switch times, respectively. Remark 2.17 The dynamic optimization algorithm proposed in this book is gradientbased, therefore, Ds Jσ (x, u) is required in the switch time optimization process. Assumption 2.16 implies that limt↑si x(t) = limt↓si x(t) and limt↑si u(t) = limt↓si u(t) for i = 0, 1, . . . , n, which are sufficient conditions for the existence of Ds Jσ (x, u) [5]. Assumption 2.18 For any fixed (s, u) ∈ S × U and the corresponding x, g(x, u) is Lipschitz and differentiable in t in each time interval [si , si+1 ], i = 0, 1, . . . , n. In addition, g(x, u) has a uniform lower bound, that is ∃M ∈ R such that g(x, u) ≥ M. Remark 2.19 Since the system dynamics are described with differential equations with discontinuous right-hand side, the left and right derivatives of g(x, u) with respect to t are different at switch times. So we assume the Lipschitz condition and differentiability of g(x, u) in each time interval [si , si+1 ]. In this case, there are Lipschitz constants for g(x, u) in each time interval [si , si+1 ]. Roughly speaking, if the input u is bounded and f q and g are Lipschitz in each time interval [si , si+1 ], then g is uniformly bounded.

2.3 Dimensionality Reduction of the Path Constraint

39

2.3 Dimensionality Reduction of the Path Constraint Notice that the path constraint is described in terms of the state and the input instead of the switch time vector. By uniqueness of the solution to the differential equation (2.23), once the input and the switch time vector are fixed, we can redescribe the path constraint by an abstract constraint in terms of the input and the switch time vector. In this case, the abstract constraint is infinite-dimensional. We can analyze differentiability of v(s) in Banach Space by using perturbation analysis theory. However, as can be obtained from the next subsection, the formula to calculate the derivative of v(s) is derived from the finite-dimensional KKT conditions and differentiability of the abstract constraints, since a calculatable formula is difficult to be derived from the infinite-dimensional KKT conditions [6] and differentiability of the abstract constraints in Banach Spaces. Hence, we first propose a method to reduce the path constraint into a finite number of discrete abstract constraints in advance, which are parameterized by the switch time vector and the input. In this way, we can study the transformed parameterized optimization problem that is easier to analyze than the original problem, and derive a calculatable formula by finite-dimensional KKT conditions. The difference between the transformed problem and the original problem can be controlled via the restriction parameters introduced in the following. Since the path constraint contains an infinite number of time point constraints over the entire time horizon, the dynamic optimization can be treated as a semi-infinite program (SIP) [7] if the number of decision variables is finite and the time variable is viewed as a parameter of the SIP. We can use the finite-dimensional representation of KKT conditions of SIP to locate the optimal solutions of problem (2.25) after the switch time vector is fixed and the control vector parameterized technique is performed. Thus, for the path constraint transformation and finite termination of the algorithm proposed later, we make the following assumption regarding the existence of a Slater point satisfying the first-order KKT conditions of problem (2.25) to specified tolerances, which is first introduced in [7]. in s Assumption 2.20 For each s ∈ S, given tolerances in stat , act > 0, there exist u ∈ U, s in a positive constant  ≤ act , nonnegative and uniformly bounded multipliers λis for i = 1, 2 . . . , m, and a finite set {t1s , . . . , tms } ⊂ Tinact (u s ), such that

g(x, u s ) ≤ −s , ∀t ∈ [t0 , t f ], ∇u Js (u s ) +

m 

λis ∇u g(x(tis ), u s (tis )) ≤ in stat ,

i=1

λis g(x(tis ), u s (tis )) ∈ [−λis in act , 0], i = 1, 2, . . . , m

(2.26)

in where Tinact (u s ) = {t ∈ [t0 , t f ] | g(x, u s ) ∈ [−in act , 0]} denotes the act -active index set.

Remark 2.21 There are two types of KKT conditions for semi-infinite programmings (SIPs), namely infinite-dimensional representation [6, 8] and finite-

40

2 Bi-level Dynamic Optimization of Path-Constrained Switched Systems

dimensional representation [9, 10]. Under extended Mangasarian-Fromovitz constraint qualification, the theorem in [10] coincides with the finite-dimensional representation of KKT conditions [11, 12], which states that for a local minimizer of SIP there exist at most m active indices of the minimizer to characterize its KKT (first-oder) necessary optimality conditions of SIP. Moreover, as indicated in [11], in the case when less than m active indices are sufficient for the KKT conditions, one can satisfy the second and third conditions of (2.26) by artificially listing enough inactive points with zero multipliers. In Assumption 2.20, we assume by analogy that for a local minimizer u s of problem (2.25), there exist at most m in act -active indices associated with u s to characterize its KKT conditions with specified tolerances. Remark 2.22 Assumption 2.20 is a relaxation of the notions of primal feasibility, stationarity and complementary slackness of KKT conditions, and leads to approximately optimal solutions. Once problem (2.25) has suboptimal solutions, in Assumption 2.20 holds if in stat and act are sufficient small. For a given (s, u) ∈ S × U, we denote L i by the Lipschitz constant of g(x, u) with respect to t in the time interval [si , si+1 ]. Theorem 2.23 Under Assumptions 2.14–2.20, for a given positive constant  and (s, u) ∈ S × U, suppose that (1) the time interval is subdivided into c() + 1 segments: t0 < t1 < . . . < tc() < tc()+1 = t f such that max

i∈{0,1,...,c()}

(ti+1 − ti ) ≤

 L

(2.27)

where L = maxs∈S,u∈U {L 0 , L 1 , . . . , L n }, (2) the discrete point constraints hold: g(x(ti ), u(ti )) ≤ −, i = 1, 2, . . . , c().

(2.28)

Then the path constraint (2.24) holds. Proof Consider ti ∈ [t0 , t f ], i = 1, 2, . . . , c(), for a given positive constant  and (s, u) ∈ S × U, we have g(x(ti ), u(ti )) ≤ −. By the Mean Value Theorem and continuity of g(x, u), we obtain then from (2.28) that, for any t ∈ [ti−1 , ti+1 ], i = 1, 2, . . . , c(), g(x(t), u(t)) − g(x(ti ), u(ti )) ≤ Lt − ti  ≤L ≤

max

(ti+1 − ti )

i∈{0,1,...,c()}

(2.29)

and hence g(x(t), u(t)) ≤ g(x(ti ), u(ti )) +  ≤ 0, which completes the proof.



2.4 Directional Differentiability of the Optimal Value Function

41

In Theorem 2.23, it is assumed that a positive constant  is given previously to satisfy assumption (2) in Theorem 2.23. Indeed,  need to be small enough to make sure that there exists a feasible u to satisfy the discrete constraints (2.28). For different switch time vector s, the corresponding optimization problem and the largest  that can be chosen is also different. An alternative selection of  is  = inf s∈S s . In this case, according to Assumption 2.20, there always exists a feasible u to satisfy condition (2.28) for all s ∈ S. This selection of  over the whole set S implies that the selection of  and c() (i.e., the satisfaction of condition (2.28)) can be independent from s. Based on the theorem above, for a given positive constant  ≤ s , we can subdivide the time interval into c() + 1 segments: t0 < t1 < . . . < tc() < tc()+1 = t f under the satisfaction of (2.27). Furthermore, the point constraints approach to the path constraint when  → 0 and the number of discrete points tends to infinity. Under this dimensionality reduction, the path constraint can be represented in the form: g(x(ti ), u(ti )) ∈ K, i = 1, 2, . . . , c()

(2.30)

by taking K = {k ∈ R | M ≤ k ≤ −} (recall that M is the uniform lower bound of g(x, u)), clearly K is a closed convex set. By uniqueness of the solution to the differential equation (2.23), we define G i : S × U → R as G i (s, u) = g(x(ti ), u(ti )), i = 1, 2, . . . , c()

(2.31)

then the path constraint (2.24) can be reduced into a family of abstract constraints of the form: G i (s, u) ∈ K. i = 1, 2, . . . , c().

(2.32)

2.4 Directional Differentiability of the Optimal Value Function In this subsection we study the transformed parameterized optimization problem under Assumptions 2.14–2.20 of the form: min J (s, u) u∈U

s.t. G i (s, u) ∈ K, i = 1, 2, . . . , c()

(2.33)

depending on the parameter vector s ∈ S. The dynamic of the system is governed by (2.23). We assume that J (s, u) and G i (s, u) are Lipschitz and differentiable. In ˜ any case we denote by Φ(s) the feasible set of the current optimization problem. In particular, for the above problem we have

42

2 Bi-level Dynamic Optimization of Path-Constrained Switched Systems

˜ Φ(s) := {u ∈ U : G i (s, u) ∈ K, i = 1, 2, . . . , c()}

(2.34)

and the associated set ˜ Ω(s) := arg min J (s, u). ˜ u∈Φ(s)

(2.35)

For a given point s¯ , in the parameter space S, we view the corresponding problem as an unperturbed problem, and investigate continuity and differentiability properties of the approximately optimal value function v(s) ˜ := inf J (s, u) ˜ u∈Φ(s)

(2.36)

in the vicinity of the given point s¯ . The result below closely follows the content in [1]. However, we first use perturbation analysis theory to analyze the optimal value function of the optimal control problem for path-constrained switched systems. Furthermore, based on the dimensionality reduction of the path constraint, the derivative of the optimal value function is proved equal to the derivative of the cost function evaluated at the optimal input under some conditions. First, conditions under which the approximately optimal value function is continuous are given as follows. Proposition 2.24 Consider problem (2.33) under Assumptions 2.14–2.20, suppose that (1) there exist α ∈ R and a compact set C ⊂ U such that for every s in a neighborhood of s¯ , the level set ˜ : J (s, u) ≤ α} levα J (s, ·) := {u ∈ Φ(s)

(2.37)

is nonempty and contained in C, ˜ s ). (2) Robinson constraint qualification holds at (¯s , u), ¯ where u¯ ∈ Ω(¯ Then the approximately optimal value function v(s) ˜ is continuous at s¯ . Proof The proof follows from Proposition 4.4 in [1]. Then we analyze the differentiability of the approximately optimal value function. We define the Lagrange function as: L(s, λ, u) = J (s, u) +

c() 

λi G i (s, u)

i=1

where λi are nonnegative and uniformly bounded multipliers.

(2.38)

2.4 Directional Differentiability of the Optimal Value Function

43

Theorem 2.25 Consider problem (2.33) under Assumptions 2.14–2.20, suppose that ˜ s ) of optimal solutions, (1) problem (2.33) has nonempty set Ω(¯ ˜ s ), (2) the directional regularity condition holds in the direction d for all u¯ ∈ Ω(¯ (3) for any s(δ) := s¯ + δd + o(δ) and δ > 0 small enough, problem (2.33) pos˜ s )) = O(δ), sesses an o(δ)-optimal solution u(δ) ˆ such that dist(u(δ), ˆ Ω(¯ ˜ s ). ˆ n )} has a limit point u¯ ∈ Ω(¯ (4) for any δn ↓ 0 the sequence {u(δ Then the approximately optimal value function v(s) ˜ is Hadamard directionally differentiable at s¯ in the direction d, and v˜  (¯s , d) = inf Ds J (¯s , u)d ˜ s) u∈Ω(¯

(2.39)

Proof Let h be a feasible point of the linearization problem of problem (2.33). Then we have that DG(¯s , u)(d, ¯ h) ∈ TK (G(¯s , u)). ¯ By (2.13) we obtain that, under the directional regularity condition, h is then a feasible direction. This implies that, ˜ there exists r (δn ) = o(δn ), δn ≥ 0, such that u(δn ) := u¯ + δn h + r (δn ) ∈ Φ(s(δ n )). It follows that ¯ + δn h + r (δn ), s(δn )) v(s(δ ˜ n )) ≤ J (u = J (¯s , u) ¯ + δn D J (¯s , u)(d, ¯ h) + o(δn )

(2.40)

Since J (¯s , u) ¯ = v(¯ ˜ s ), we have lim

n→∞

v(s(δ ˜ ˜ s) n )) − v(¯ ≤ D J (¯s , u)(d, ¯ h) δn

(2.41)

Since h is an arbitrary feasible point, by Lemma 2.11 we obtain v(s(δ ˜ ˜ s) n )) − v(¯ δn ≤ inf sup Ds L(¯s , λ, u)d

v˜  (¯s , d) = lim

n→∞

˜ s ) λ∈(¯s ,u) u∈Ω(¯

(2.42)

which implies the upper bound. We proceed as follows to verify the lower bound. Let s n := s¯ + δn d + o(δn ) and n ˜ s ). By assumption 1), we have that there ˆ n ) be convergent to some u¯ ∈ Ω(¯ u := u(δ n n n ˜ exists u¯ ∈ Ω(¯s ) such that u − u¯  = O(δn ). Let λ¯ n be the optimal solution of the linearization dual problem of problem (2.33) corresponding to the point u¯ n , i.e., λ¯ n = arg max n Ds L(¯s , λ, u¯ n )d λ∈(¯s ,u¯ )

(2.43)

Proposition 4.43 in [1] gives the result that λ¯ n is uniformly bounded. Clearly n ¯ λ , G(s n , u n ) − G(¯s , u¯ n ) ≤ 0, we have

44

2 Bi-level Dynamic Optimization of Path-Constrained Switched Systems

J (s n , u n )− J (¯s , u¯ n ) ≥ L(s n , λ¯ n , u n )− L(¯s , λ¯ n , u¯ n )

(2.44)

Since it is assumed thatu n − u¯ n  = O(δn ). By the Taylor expansion we have Δ1 = L(¯s , λ¯ n , u n ) − L(¯s , λ¯ n , u¯ n ) = u n − u¯ n Du L(¯s , λ¯ n , u¯ n ) + o(δn )

(2.45)

By the first order necessary conditions Du L(¯s , λ¯ n , u¯ n ) = 0, and hence Δ1 = o(δn )

(2.46)

We also have that Δ2 = L(s n , λ¯ n , u n ) − L(¯s , λ¯ n , u n ) = L(¯s , λ¯ n , u¯ n ) + δn Ds L(¯s , λ¯ n , u¯ n ) + o(δn ) + u n − u¯ n Du L(¯s , λ¯ n , u¯ n ) + o(δn )−[L(¯s , λ¯ n , u¯ n ) + u n − u¯ n Du L(¯s , λ¯ n , u¯ n ) + o(δn )] = δn Ds L(¯s , λ¯ n , u¯ n ) + o(δn )

(2.47)

and hence L(s n , λ¯ n , u n ) = L(¯s , λ¯ n , u n ) + δn Ds L(¯s , λ¯ n , u¯ n ) + o(δn )

(2.48)

˜ s ) = J (¯s , u¯ n ), it follows from (2.44), (2.46) Since v(s ˜ n ) = J (s n , u n ) + o(δn ) and v(¯ and (2.48) that ˜ s ) ≥ δn Ds L(¯s , λ¯ n , u¯ n ) + o(δn ) v(s ˜ n ) − v(¯

(2.49)

and hence v(s ˜ n ) − v(¯ ˜ s) n→∞ δn ≥ inf sup Ds L(¯s , λ, u)d

v˜  (¯s , d) = lim

˜ s ) λ∈(¯s ,u¯ n ) u∈Ω(¯

(2.50)

Together with (2.42) this implies that v˜  (¯s , d) = inf

sup Ds L(¯s , λ, u)d

˜ s ) λ∈(¯s ,u) u∈Ω(¯

(2.51)

2.4 Directional Differentiability of the Optimal Value Function

45

Notice that v˜  (¯s , d) = inf

sup Ds L(¯s , λ, u)d

˜ s ) λ∈(¯s ,u) u∈Ω(¯

= inf Ds J (¯s , u)d ˜ s) u∈Ω(¯

+ inf

sup

˜ s ) λi ∈i (¯s ,u) u∈Ω(¯

c() 

λi Ds G i (¯s , u)d

(2.52)

i=1

We consider two cases of the constraints. ˜ s ). Case 1: G i (¯s , u) < −, where u ∈ Ω(¯ From [1, p. 278] we can obtain that the effective domain of the dual program is exactly the set of Lagrange multipliers of problem (2.33) at point (¯s ). Since the KKT ˜ s ), by the complementary slackness condition, we have conditions hold at u ∈ Ω(¯ λi = 0 and thus λi Ds G i (¯s , u) = 0. ˜ s ). Case 2: G i (¯s , u) = −, where u ∈ Ω(¯ In this case, for a s ∈ S in a neighborhood of s¯ , it follows that G i (s, u) ≤ −. By differentiability of G i (s, u), we can obtain that Ds G i (¯s , u) = 0, hence λi Ds G i (¯s , u) = 0. Then we obtain that formula (2.39) holds, which completes the proof.  As we mentioned in the previous subsection, when  → 0 and the number of discrete points tends to infinity, the transformed problem approaches to the original problem, and the method to calculate the derivative of the approximately optimal value function can be extended to the optimal value function. Corollary 2.26 Suppose that assumptions in Theorem 2.23 hold. Then, as  → 0, the optimal value function v(s) is Hadamard directionally differentiable at s¯ in the direction d, and v  (¯s , d) = inf Ds J (¯s , u)d u∈Ω(¯s )

(2.53)

Proof Since when  → 0 and the number of discrete points tends to infinity, the trans˜ = formed problem approaches pointwise to the original problem, that is, lim→0 v(s) v(s). Thus, the convergence of the directional derivative lim→0 v˜  (¯s , d) = v  (¯s , d) ˜ s ) = v(¯s ) for all s¯ ∈ S by taking the directional is directly derived from lim→0 v(¯ ˜ s ) = v(¯s ) in the direction d. Furthermore, derivatives from both sides of lim→0 v(¯ the optimal set of the transformed problem also converges pointwise to that of the ˜ = Ω(s) (see the definition of pointwise conoriginal problem, that is, lim→0 Ω(s) ˜ s ) = Ω(¯s ) for all vergence for set-valued maps in [13, 14]), and thus lim→0 Ω(¯ s¯ ∈ S. Then by taking the limit of both sides of (2.39) we have that (2.53) holds, which completes the proof.  Remark 2.27 Note that in Corollary 2.26 the optimal set Ω(¯s ) may not be a singleton, hence v(·) is directional differentiable in the sense of Hadamard. In particular,

46

2 Bi-level Dynamic Optimization of Path-Constrained Switched Systems

v(·) is Fréchet differentiable at s¯ if the optimal set Ω(¯s ) = {u} ¯ is a singleton, in ¯ which case Dv(¯s ) = Ds J (¯s , u). Next, a method to calculate the derivative of the cost function with a nominal input is introduced by adapting the method in [5]. For any ω : R → Rn we define ω(t − ) = limτ ↑t ω(τ ) and ω(t + ) = limτ ↓t ω(τ ) as the left and right limits of the signal ω at the time t, respectively, and define at a switch time si  f (si− ) = f ρi−1 (x(si ), u(si− )) − f ρi (x(si ), u(si− )),  f (si+ ) = f ρi−1 (x(si ), u(si+ )) − f ρi (x(si ), u(si+ ))

(2.54)

Similarly, we define l(si− ) and l(si+ ). To compute the derivative of the cost function with respect to s for all u ∈ Ω(¯s ), we need the following lemma. Lemma 2.28 ([5]) The directional derivative of the cost function in the direction di for i = 1, 2, . . . , n exists and is given as: J  (s, u, di ) = p T (si ) f (si+ ) + l(si+ )

(2.55)

where the costate p(t), t ∈ [t0 , t f ] satisfies the following switched system dynamics: ∂lρi T ∂ f ρi T (x, u) p − (x, u), t ∈ [si , si+1 ], ∂x ∂x p(t f ) = Dx ϕ(x(t f ))T

p˙ = −

(2.56)

and p(si− ) = p(si+ ) at the switch time si . Remark 2.29 Since we assume that J (s, u) is continuously differentiable, it follows that for i = 1, 2, . . . , n, the partial derivative of the cost function with respect to si exists and is given as: ∂J = p T (si ) f (si+ ) + l(si+ ) ∂si

(2.57)

Meanwhile, the derivative of the cost function with respect to s exists and is given ∂J ∂J , . . . , ∂s ) for all u ∈ Ω(¯s ). as Ds J (s, u) = ( ∂s 1 n

2.5 Algorithm Development Based on the theoretical results above, a bi-level algorithm (graphically illustrated in Fig. 2.1) is proposed to solve the optimal control problem for path-constrained switched systems with guaranteed feasibility, and the convergence analysis is given in

2.5 Algorithm Development

47

Fig. 2.1 Graphic illustration of the bi-level algorithm. The inner and outer level optimization problems are solved by Algorithms 2 and 3, respectively

this section. At the inner level, for any fixed switch time vector s ∈ S, problem (2.25) becomes a semi-infinite-dimensional, inequality-path-constrained dynamic optimization problem after the control vector parameterization technique is performed. We locate a guaranteed feasible input satisfying the KKT conditions to specified tolerances by iteratively solving approximate dynamic optimization problems of the path-constrained switched system (see Algorithm 2), which simply adapts our previous method [7] to the switched systems case. At the outer level, we first calculate the gradient of v(s) by using the method in Sect. 2.4, then update the switch time vector by sequentially solving quadratic programs related to the optimal value function (see Algorithm 3). Before we present the bi-level algorithm, for a fixed switch time vector s ∈ S we introduce the inner optimization problem (IOP) and its approximate optimization problem as follows: min Js (u) = J (s, u) u∈U

s.t. g(x, u) ≤ 0, ∀t ∈ [t0 , t f ],  si , si+1 ) , if i = n  x˙ = f ρi (x, u), t ∈  , sn , t f , if i = n x(t0 ) = x0 , i = 0, 1, . . . , n

(2.58)

min Js (u) = J (s, u) u∈U

s.t. g(x, u) ≤ − p , ∀t ∈ T p ,  si , si+1 ) , if i = n  x˙ = f ρi (x, u), t ∈  , sn , t f , if i = n x(t0 ) = x0 , i = 0, 1, . . . , n

(2.59)

48

2 Bi-level Dynamic Optimization of Path-Constrained Switched Systems

where  p > 0 and T p ⊂ [t0 , t f ] denote the restriction parameter and (finite) discretized constraint points at iteration p. Unlike the stationarity and complementary slackness of KKT conditions, the feasibility needs to be verified over the entire time horizon for the path constraint. The feasibility of a given input u¯ ∈ U for problem (2.58) is established via globally solving problem (2.60) below and determining the largest violation of the path constraint over T as ¯ := max g(x(t), u(t)) ¯ g max (u)

 si , si+1 ) , if i = n  x(t) ˙ = f ρi (x(t), u(t)), ¯ t∈  , sn , t f , if i = n x(t0 ) = x0 , i = 0, 1, . . . , n t∈T

s.t.

(2.60)

Problem (2.60) can be addressed by integrating the dynamic system with available ODE solvers and then determining the maximal value of g(x(t), u(t)) ¯ based on the intermediate state values. However, sufficiently many points are required in approximating the path constraint and no direct error control is provided. A more efficient way of controlling the error level involves formulating and solving a hybrid discrete-continuous dynamic system [15, 16]. Specifically, an extra variable γ(t, u) ¯ representing the maximal constraint violation up to t is appended to the dynamic system and state events are defined ensuring that the activations/deactivations of the path constraint function are precisely located:  γ(t, ˙ u) ¯ =

0, if γ(t, u) ¯ ≥ g(x, u) ¯ or g(x, ˙ u) ¯ 0; in reduction parameter r > 1; tolerances in stat ≥ act > 0; iteration counter p = 0 p Output: u 1: loop 2: Solve problem (2.59) to local optimality 3: if feasible then p p 4: Set u p equal to the optimal solution point. Obtain m gradients ∇u g(x(ti ), u p (ti )), i = 1, 2, . . . , m and m multipliers, which consists of all (at most m) linearly independent gradients of active constraints at u p with their respective multipliers, and enough inactive constraint gradients (if needed) with zero 5: Obtain the maximum violation of g(x, u p ) as g max (x, u p ) and the corresponding time point t max (u p ) by using the method in [7] 6: if g max (x, u p ) ≤ 0 then m p p p p p p 7: if ∇u Js (u p ) + i=0 λi ∇u g(x(ti ), u p (ti )) ≤ in and λi g(x(ti ), u p (ti )) ∈ stat p in [−λi act , 0], i = 1, 2, . . . , m then 8: terminate 9: else 10: Set  p+1 ←  p /r and T p+1 ← T p (Case 3) 11: end if 12: else 13: Set T p+1 ← T p ∪ t max (u p ) (Case 2) 14: end if 15: else 16: Set  p+1 ←  p /r and T p+1 ← T p (Case 1) 17: end if 18: p ← p + 1 19: end loop

Remark 2.30 Noticing that the constraints of the outer optimization problem are h i (s) = si − si+1 ≤ 0, i = 0, 1, . . . , n, we can also optimize the total number of switches as any two switch times merge into one, i.e., h i (s) = 0. Remark 2.31 The outer level optimization algorithm locates an optimal switch time vector by satisfying the KKT conditions to a specified tolerance. The algorithm is in the framework of sequential quadratical programming, since the quadratic programs and line search are also used to generate iteration steps and step sizes, respectively.

2.6 Convergence Analysis Notice that problem (2.59) can be solved by a nonlinear program (NLP) solver, however, a local NLP solver may fail to locate a feasible point, even if feasible points do indeed exist [7]. We make following assumptions to rule out these possibilities and to prove the finite convergence. Assumption 2.32 A local NLP solver is available that generates a KKT point of problem (2.59) at each iteration, whenever it is feasible.

50

2 Bi-level Dynamic Optimization of Path-Constrained Switched Systems

Algorithm 3 Bi-level algorithm for the dynamic optimization problem of pathconstrained switched systems with guaranteed feasibility. Input: total number of switches n; iteration counter k = 0; feasible switch time vector s 0 ; initial out Lagrange multiplier μ0 ; tolerances out stat ≥ act > 0 k k Output: (s , u ) 1: loop 2: Solve IOP by Algorithm 2 (Inner level) 3: Set u k equal to the optimal solution, and calculate Dsi J (sik , u k ), i = 1, 2, . . . , n ∂v ← Dsi J (sik , u k ) as the partial derivative of the optimal value function with respect to 4: Set ∂s i

5: 6: 7: 8: 9:

10: 11: 12:

∂v ∂v T si , and obtain the gradient ∇s v(s k ) = ( ∂s , . . . , ∂s ) , (since problem (2.62) contains linear n 1 2 v(s k ) (or be its Hessian approximation) constraints only) let L(s k , μk ) be ∇ss k Obtain n gradients n ∇s hki (s ) k k k out k if ∇s v(s k ) + i=0 μi ∇s h i (s ) ≤ out stat and μi h i (s ) ∈ [−μi act , 0], i = 1, 2, . . . , n then terminate else s k+1 ← s k + αk dk and μk+1 ← μ¯ k , where dk solves min{∇s v(s k )T d + 21 d T L(s k , μk )d : h i (s k ) + ∇h i (s k )T d ≤ 0, i = 0, 1, . . . , n}, μ¯ k is its associatedLagrange multiplier of n the and αk solves min{v(s k + αdk ) + i=0 pi |h i (s k + αdk )| + n quadratic problem, k i=0 pi max(h i (s + αdk ), 0) : α > 0} where pi > 0 are chosen penalty parameters. (Case 4) end if Set k ← k + 1 end loop

Assumption 2.33 The optimal value function of problem (2.25) has uniformly continuous first and second derivatives. Assumption 2.34 All KKT multipliers are nonnegative and uniformly bounded with respect to all iterations at both the inner level and the outer level optimizations. Assumption 2.35 Assume that the outer optimization problem (2.62) has a Slater point satisfying the KKT condition to specified tolerance. Namely, for given tolerout l ances out stat ≥ act > 0, assume that there exist s ∈ S, and uniformly bounded multil pliers μi ≥ 0, for i = 1, 2, . . . , n, such that h i (s l ) ≤ 0, i = 0, 1, 2, . . . , n, ∇s v(s l ) +

n 

μli ∇s h i (s l ) ≤ out stat ,

i=0

μli h i (s l ) ∈ [−μli out act , 0], i = 0, 1, 2, . . . , n

(2.63)

Remark 2.36 Practically, NLP solvers return points satisfying KKT conditions within some tolerances, we can adjust the specified tolerances of Algorithm 2 to take into account these points. Roughly speaking the tolerances of NLP solvers should be substantially tighter than the specified tolerances in Algorithm 2.

2.6 Convergence Analysis

51

Remark 2.37 In order to ensure that the KKT multipliers are all in a compact set, which is used in the exclusion of Case 3, we make the same assumption as did in [7] and [11]. In [7], it is also assumed that the KKT multipliers are nonnegative and uniformly bounded. In [11], it was assumed that all multipliers belong to a standard simplex. Now the finite convergence result is given as follows. Theorem 2.38 Under Assumptions 2.20–2.35, the bi-level optimization algorithm out (Algorithm 3) terminates finitely and generates a feasible (in stat , stat )-approximate in out KKT point (i.e., a KKT point with stat -approximate in u and stat -approximate in s) out of path-constrained switched systems optimization problem (2.25) with (in act , act )active indices. Proof In the bi-level algorithm, there are four potential outcomes. The inner level includes Case 1: problem (2.59) is infeasible; Case 2: problem (2.59) is feasible but generates an IOP-infeasible point; and Case 3: problem (2.59) is feasible and generate an IOP-feasible point, but the approximate KKT conditions of IOP are not satisfied, and the out level includes Case 4: the approximate KKT conditions of problem (2.58) are not satisfied. We will prove the infinite occurrence of the four cases is impossible. The finite termination of Cases 1–3 is proved by using the similar idea in [3, 7], and proof of the finite convergence of Case 4 bears similarities with the proof idea in [11, 18]. Case 1: problem (2.59) is feasible within finite iterations. By Assumption 2.20 there exists at least one Slater point of problem (2.25) for any s ∈ S. This point is also feasible for problem (2.59) when  p ∈ [0, s ] since T p ⊂ [t0 , t f ]. Therefore, infinite Case 1 is excluded after at most N1 = logr 0 /s  updates. Case 2: there is no more violation of the path constraint after finite iterations. After Case 1 is excluded and from Assumption 2.20, for all iterations k ≥ min  = s /r hold. We first consider a sequence of solutions to problem (2.59). Since U is compact, we can select a converging subsequence {u m } with the limit point u. ˆ Consider the corresponding optimal time point t m := t max (u m ) for largest violation of the path constraint over [t0 , t f ]. By construction of problem (2.59) at iteration l, we have g(x(t m , u l ), u l ) ≤ −min < 0, ∀l, m ∈ N with l > m. By continuity of g and compactness of X × U, we know that g is uniformly continuous on X × U. Because x(t, u) ∈ X , ∀(t, u) ∈ T × U, g is essentially uniformly continuous on T × U. Define g(t, ˜ u) := g(x(t, u), u). Thus, for all ¯ > 0 there exists a δ¯ > 0 (independent of any t ∈ T and any u ∈ U due to the uniform continuity of g(t, ˜ u) jointly on T × U) such that for all u and the u l that l ¯ we have |g(t ˜ m , u) − g(t ˜ m , u l )| < ¯, for all l, m ∈ N with satisfies |u − u | < δ, min m m l ˜ , u ). Taking ¯ =  2 > 0 and noting that l > m. It follows that g(t ˜ , u) < ¯ + g(t min ˜ m , u) < −  2 < 0. Since u m → u, ˆ for any δ¯ g(t ˜ m , u l ) ≤ −min < 0, we have g(t m l ¯ there exists K such that |u − u | < δ, for all l, m ∈ N with l > m > K . Then min g(x(t m , u m ), u m ) < −  2 < 0. Thus, after a finite K , the points given by problem (2.59) in this case are IOP-feasible.

52

2 Bi-level Dynamic Optimization of Path-Constrained Switched Systems

Case 3: the approximate KKT conditions of (2.58) textbfare satisfied within finite iterations. Assume that the algorithm does not terminate. Then there exists a subsequence {u pl } of the returned solution sequence of problem (2.59) such that for each pl ∈ N, u pl is a KKT point for problem (2.59), but the termination criterion is not satisfied for any pl ∈ N. Since u pl is a KKT point for problem (2.59), we have g(x, u pl ) ≤ − pl , ∀t ∈ T pl , ∇u Js (u pl ) +

m 

p

p

p

λi l ∇u g(x(ti l ), u pl (ti l )) = 0,

i=1 pl pl p λi [g(x(ti ), u pl (ti l )) p

+  pl ] = 0, i = 1, 2, . . . , m p

(2.64)

p

According to Assumption 2.34, {λ1 l , λ2 l , . . . , λml } belongs to a compact set, say p p p p p p  pl , and {u pl , λ1 l , λ2 l , . . . , λml , t1 l , t2 l , . . . , tml } is contained in the compact set U × l × (T )m , then it possesses an accumulation point (u ∗ , λ∗1 , λ∗2 , . . . , λ∗m , t1∗ , t2∗ , . . . , tm∗ ). Thus, by the continuity the second equation of (2.64) yields ∇u Js (u ∗ ) +

m 

λi∗ ∇u g(x(ti∗ ), u ∗ (ti∗ )) = 0

(2.65)

i=1

From the assumption that there are at most m in act -active indices to characterize the approximately KKT conditions (see Remark 2.21), for some p¯l ∈ N we have ∇u Js (u p¯l ) +

m 

p¯

p¯

p¯

λi l ∇u g(x(ti l ), u p¯l (ti l )) ≤ in stat

(2.66)

i=1

After at most N3 = max{ p¯l , logr 0 /s } updates, we have  N3 ≤ s . Thus the third equation of (2.64) gives λiN3 [g(x(tiN3 ), u N3 (tiN3 )) +  N3 ] = 0, i = 1, 2, . . . , m

(2.67)

Since  N3 ≤ s , s ≤ in act , we have −λiN3 g(x(tiN3 ), u N3 (tiN3 )) ∈ [−λiN3 in act , 0], i = 1, 2, . . . , m

(2.68)

Therefore, (2.66) and (2.68) imply that after at most N3 iterations while Case 1 and Case 2 are excluded, the finite-dimensional KKT conditions are satisfied. Case 4: the approximate KKT conditions of (2.58) are satisfied within finite iterations. Since 0 ≤ si ≤ t f , S is a compact set. From Assumption 2.34, we have that {s k , μk } belongs to a compact set. According to Assumption 2.35, similar to the argumentation of Case 3, for some k¯ we have

2.7 Numerical Case Studies

53 ¯

∇s v(s k ) +

n 

¯

¯

μik ∇s h i (s k ) ≤ out stat ,

i=0 ¯ ¯ μik h i (s k )

¯

∈ [−μik out act , 0], i = 1, 2, . . . , n

This implies that the infinite Case 4 is excluded, which completes the proof.

(2.69) 

Remark 2.39 As remarked in [7], if problem (2.58) does not have Slater points, problem (2.59) cannot generate feasible points of (2.58), and thus Algorithm 2 loops infinitely; if problem (2.25) has Slater points which are not locally approximate KKT points, Algorithm 2 may not terminate finitely either, since Algorithm 2 is designed not to terminate just for feasible points.

2.7 Numerical Case Studies In this section, we illustrate the effectiveness of the proposed algorithms, numerically verify the derivative of the optimal value function, analyze computational performance affected by the algorithm parameters. The implementation is carried out in MATLAB Version 9.5.0.944444(R2018b, win64) on an Intel Core i7-7700 CPU @ 3.60 GHz with 32 GB of RAM. Du J (s, u) is calculated by solving sensitivity equations with ode45. Ds J (s, u) is calculated by using the method in Sect. 2.4 through backward integration.

2.7.1 Effectiveness of the Bi-level Algorithm Consider a switched system consisting of 

Subsystem 1: Subsystem 2:

 0.6 1.2 1 x+ u −0.8 3.4 1   0.2 3 2 x˙ = x+ u −1 0 −1 x˙ =

(2.70a) (2.70b)

with the initial condition x(0) = [0, 2]T . Suppose that the system undergoes one switch at t = s from subsystem 1 to 2 in the time interval t ∈ [0.2, 1.2]. We want to find an optimal input u and an optimal switch time s such that the cost function 1 J = (x1 (t f ) − 4) + 2



2

2

u 2 dt 0

(2.71)

54

2 Bi-level Dynamic Optimization of Path-Constrained Switched Systems

Table 2.1 Summary table for numerical case studies of the bi-level algorithm Opt-cost Opt-switch-time g max Computational time(s) 0.1030

0.3947

–0.05

207.0938

Fig. 2.2 Optimal input. The time horizon is subdivided into 30 equidistant subintervals, and the input is parameterized by constants on each subinterval

is minimized subject to the inequality constraints of states and the input over the entire time horizon: x2 − 7 ≤ 0, ∀t ∈ [0, 2], − 10 ≤ u ≤ 10, ∀t ∈ [0, 2]

(2.72)

We implement the proposed bi-level algorithm to this example. At the inner level, −3 in −3 0 the path constraint is guaranteed satisfied with in stat = 10 , act = 10 ,  = 5 × −3 0 10 , r = 4 and T = ∅. The input is parameterized by using CVP with piecewiseconstant basis functions, we consider 30 equidistant subintervals resulting in 30 decision variables. At the outer level, we update the switch time vector from the −6 initial nominal value s 0 = 0.7 with out stat = 10 . For this example, the bi-level algorithm terminates after 15 iterations of the outer level. The results obtained are reported in Table 2.1. These results confirm that the path constraint is rigorously satisfied over the entire time horizon. The optimal switch time is 0.3947 and the corresponding optimal cost is 0.1030. The optimal input and the corresponding state trajectory are shown in Figs. 2.2 and 2.3.

2.7 Numerical Case Studies

55

8 6 4 2 0 -2 -4 -6 -8 -10 0

2

4

6

8

10

12

14

Fig. 2.3 Optimal state trajectory

Figure 2.4 shows the corresponding path constraint profile under the optimal control, where the red star indicates that guaranteed feasibility is achieved over the entire time horizon [0, 2] at the last iteration, and the point at which the maximal value of the path constraint function occurs over the entire time horizon. Figure 2.5 gives a zoomed-in Fig. 2.4 around this point. The above observations agree to what we expect from the properties of the algorithm. The optimal value function and its zoomed-in view around the optimal point is shown in Figs. 2.6 and 2.7. It can be verified that the optimal cost and the corresponding optimal switch time are consistent with our numerical results. In addition, we customize the step-size to describe the the derivative of the optimal value function shown in Fig. 2.8. It should be pointed out that the profile is nonsmooth since that the derivative is computed with the discrete nominal input. Due to the inevitable numerical computational error from our customized step-size, the derivative of the optimal value function varies from negative to positive around the optimal switch time between 0.39 to 0.43. It can be verified that the variation of the derivative coincides with the variation of the optimal value function. Remark 2.40 The tolerance of the input obtained at the inner level can be controlled in via adjusting the parameters (see in stat , act in Algorithm 2), and thus the tolerance of the derivative of the optimal value function can also be controlled indirectly. In addition, the implementation of CVP also has an affect on these tolerances. Higher-order polynomials, more subintervals, and adaptive step size can give lower tolerance of the optimal solutions, which may lead to an increase on the computational complexity.

56

2 Bi-level Dynamic Optimization of Path-Constrained Switched Systems 2 0 -2 -4 -6 -8 -10 -12 -14 -16 0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Fig. 2.4 Path constraint profile. The red star indicates rigorous satisfaction of the path constraint over the entire time horizon [0, 2] at the last iteration, and the point at which the maximal value of the path constraint function occurs over the entire time horizon 0.04

0

-0.04 -0.05

-0.08

-0.12 0.394

0.39468

0.396

0.398

Fig. 2.5 Zoomed-in Fig. 2.4 around the maximum value of the path constraint function

2.7 Numerical Case Studies

57

30

25

20

15

10

5

0 0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Fig. 2.6 Optimal value function. The switch is specified to occur in [0.2, 1.2] 1.2

1

0.8

0.6

0.4

0.2 0.1030 0 0.3

0.37

0.3947

0.44

Fig. 2.7 Zoomed-in view of the optimal value function around the optimal point

0.51

58

2 Bi-level Dynamic Optimization of Path-Constrained Switched Systems 90 80 70 60 50 40 30 20 10 0 -10 0

0.2 0.39 0.43

1

1.2

2

Fig. 2.8 Derivative of the optimal value function. The red star indicates the optimal switch time at which the derivative of the optimal value function varies from negative to positive

2.7.2 Computational Performance As indicated in [3] and [7], the restriction parameter 0 and the strategy employed for its decrease can have a large influence on the computational performance of the bi-level algorithm. Problem (2.29) may become infeasible or generate significantly suboptimal points if 0 is too large. With too small a value for 0 on the other hand, problem (2.29) yields an outer-approximation of problem (2.25) for each s k until T p closely approximates T , thereby resulting in many computational demanding iterations due to a large number of interior-point constraints. As a consequence, too rapid or too slow a decrease of  (small value of r ) can result in a large number of iterations. For a fixed restriction parameter 0 and a fixed r , the computational performance also varies with the cardinality of T 0 . Since the inner level algorithm needs to populate enough cuts before feasible points can be generated, a too small cardinality usually results in a long computational time. On the other hand, if T 0 is very large, the cost per iteration is initially very large due to the presence of many interior point constraints in problem (2.59).

References

59

2.8 Summary A bi-level algorithm is proposed to find optimal switch times and the optimal input of switched systems with guaranteed feasibility of path constraints, a distinguishing feature compared to the existing methods in the literature. It is shown that this algorithm finitely converges to a feasible point satisfying the KKT conditions of the optimization problem to specified tolerances. It is also shown that, for a given switch time vector, the derivative of the optimal value function is proved equal to the derivative of the cost function evaluated at the optimal input. We have compared relevant state-of-art approaches with ours to show the novelty of our method. An efficient method to compute optimal switch times for switched systems was proposed in [19]. Reference [20] provided an approach to find the optimal sequence of modes and optimal switch times. Nevertheless, both problems considered in [19] and [20] were autonomous. Reference [5] presented a method for computing the derivative of the optimal value function such that the optimal switch time vector and input can be located for non-autonomous systems. However, the method can only deal with the problem that the optimal input is known analytically. Also note that none of the literature mentioned above considered path constraints. Although path constraints were considered in [21], as indicated in [22], the algorithm may have the potential of an infinite-loop. Reference [23] presented a framework for the numerical solution of the optimal control problem with switches, switching costs and jumps. By applying a direct and simultaneous adaptive collocation approach, path constraints are enforced to hold at collocation points such that the rigorous satisfaction of path constraints cannot be guaranteed over the entire time horizon. In addition, no convergence result is provided for the approach theoretically. In this end, to our best knowledge, our method is the first result to solve the optimal control problem for path-constrained switched systems with guaranteed feasibility within a finite number of iterations. Since the proposed method in this chapter is based on a bi-level optimization framework, which suffers from some drawbacks, e.g., each iteration requires solving two levels of optimization problems, and the algorithm is more suitable for problems where the inner level problem is convex or an optimal input can be located analytically. Therefore, in the next chapter, we propose an efficient single-level dynamic optimization algorithm for path-constrained switching systems.

References 1. Bonnans, J., Shapiro, A.: Perturbation analysis of optimization problems. Springer Science & Business Media, 2013 2. Cheng’en, Yu.: Four kinds of differentiable maps. Int. J. Pure Appl. Math. 83(3), 465–475 (2013) 3. Mitsos, A.: Global optimization of semi-infinite programs via restriction of the right-hand side. Optimization 60(10–11), 1291–1308 (2011)

60

2 Bi-level Dynamic Optimization of Path-Constrained Switched Systems

4. Polak, E.: Optimization: algorithms and consistent approximations, vol. 124. Springer Science & Business Media (2012) 5. Kamgarpour, M., Tomlin, C.: On optimal control of non-autonomous switched systems with a fixed mode sequence. Automatica 48(6), 1177–1181 (2012) 6. Cánovas, M., López, M.A., Mordukhovich, B.S., Parra, J.: Variational analysis in semi-infinite and infinite programming, ii: Necessary optimality conditions. SIAM J. Optim. 20(6), 2788– 2806 (2010) 7. Fu, J., Faust, J.M.M., Chachuat, B., Mitsos, A.: Local optimization of dynamic programs with guaranteed satisfaction of path constraints. Automatica 62, 184–192 (2015) 8. Jahn, J.: Introduction to the theory of nonlinear optimization. Springer Science & Business Media, (2007) 9. Bertsekas, D.P.: Nonlinear programming. Athena scientific, Belmont (1999) 10. John, F.: Extremum Problems with Inequalities as Subsidiary Conditions, pp. 197–215. Springer, Basel (2014) 11. Floudas, C.A., Stein, O.: The adaptive convexification algorithm: a feasible point method for semi-infinite programming. SIAM J. Optim. 18(4), 1187–1208 (2007) 12. Stein, O., Steuermann, P.: The adaptive convexification algorithm for semi-infinite programming with arbitrary index sets. Math. Program. 136(1), 183–207 (2012) ˇ Kwieci´nska, G.: Pointwise topological convergence and topological graph convergence 13. Holá, L, of set-valued maps. Filomat 31(9), 2779–2785 (2017) ˇ Graph convergence of set-valued maps and its relationship 14. Del Prete, I., Di Iorio, M., Holá, L.: to other convergences. J. Appl. Anal. 6(2), 213–226 (2000) 15. Barton, P.I., Kun Lee, C.: Modeling, simulation, sensitivity analysis, and optimization of hybrid systems. ACM Trans. Modeling Comput. Simulation (TOMACS) 12(4), 256–289 (2002) 16. Branicky, M.S. Erik Mattsson, S.: Simulation of hybrid systems. In: International Hybrid Systems Workshop, pp. 31–56. Springer (1996) 17. Park, T., Barton, P.I.: State event location in differential-algebraic models. ACM Trans. Modeling Comput. Simulation (TOMACS) 6(2), 137–165 (1996) 18. Dutta, J., Deb, K., Tulshyan, R., Arora, R.: Approximate kkt points and a proximity measure for termination. J. Global Optim. 56(4), 1463–1499 (2013) 19. Stellato, B., Ober-Blöbaum, S., Goulart, P.J.: Second-order switching time optimization for switched dynamical systems. IEEE Trans. Autom. Control 62(10), 5407–5414 (2017) 20. Azhmyakov, V., Juarez, R.: A first-order numerical approach to switched-mode systems optimization. Nonlinear Anal. Hybrid Syst 25, 126–137 (2017) 21. Gonzalez, H., Vasudevan, R., Kamgarpour, M., Sastry, S., Bajcsy, R., Tomlin, C.: Computable optimal control of switched systems with constraints. In: Proceedings of the 13th International Conference on Hybrid Systems, Stockholm, Sweden, pp. 51–60 (2010) 22. Wardi, Y., Egerstedt, M., Hale, M.: Switched-mode systems: gradient-descent algorithms with armijo step sizes. Discrete Event Dynamic Systems 25(4), 571–599 (2015) 23. Kirches, C., Kostina, E., Meyer, A., Schlöder, M., PN SPP1962.: Numerical solution of optimal control problems with switches, switching costs and jumps. Optimization Online, 6888 (2018)

Chapter 3

Single-Level Dynamic Optimization of Path-Constrained Switched Systems

Bi-level optimization strategy optimizes decision variables in two stages at each iteration, resulting in more NLPs to be solved, which are the most expensive cost at each iteration of the algorithms. As one can observe from the numerical simulation in the previous chapter, the computational efficiency of the bi-level algorithm is relatively low, even for a simple linear quadratic example. Practically, less expensive dynamic optimization algorithms are desired to meet operational requirements of systems. From the motivations above, this chapter presents an efficient single-level algorithm for dynamic optimization problems of path-constrained switched systems with a fixed switching sequence. This algorithm can also locate the optimal input and switch times with rigorous satisfaction of path constraints within a finite number of iterations. First, at each iteration, gradients of the objective function with respect to the system input and switch times are evaluated by solving sensitivity equations and adjoint systems, respectively. Then the optimization of the input is performed at the same iteration with that of the switch time vector, which greatly reduces the number of NLPs and computational burden compared with multi-levels algorithms. The feasibility of the optimal solution is guaranteed by the semi-infinite programming technique and the right-hand restriction approach. It is mathematically proven that the proposed algorithm terminates finitely, and converges to a solution which satisfies the KKT conditions to specified tolerances. Numerical case studies are provided to illustrate that the proposed algorithm has less expensive computational time than the bi-level algorithm. Compared with multi-level algorithms, the single-level algorithm reduces the number of NLPs leading to fewer function evaluations and Hessian approximations that result in less computational time. Specifically, it does not require any convexity assumptions or an analytical solution of the input.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Fu and C. Zhang, Dynamic Optimization of Path-Constrained Switched Systems, Studies in Systems, Decision and Control 459, https://doi.org/10.1007/978-3-031-23428-6_3

61

62

3 Single-Level Dynamic Optimization of Path-Constrained Switched Systems

3.1 Gradient Information Since the problem considered in this chapter are the same as that in Chap. 2, we make the same Assumptions 2.14, 2.16 and 2.20. This chapter no longer distinguishes between inner and outer optimization problems, therefore, we rewrite Assumption 2.20 as follows. Assumption 3.1 For each s ∈ S, given tolerances stat , act > 0, there exist u s ∈ U, nonnegative and uniformly bounded multipliers λi for i = 1, 2 . . . , m, a positive constant s ≤ act , and a finite set {t1s , . . . , tms } ⊂ Tact (u s ), such that g(x, u s ) ≤ −s , ∀t ∈ [t0 , t f ], ∇u J (u s ) +

m 

λi ∇u g(x(tis ), u s (tis )) ≤ stat ,

i=1

λi g(x(tis ), u s (tis )) ∈ [−λi act , 0], i = 1, 2, . . . , m

(3.1)

where Tact (u s ) = {t ∈ [t0 , t f ] | g(x, u s ) ∈ [−act , 0]} denotes the act -active index set. In order to optimize the input and switch time vector at the meantime, gradient information is required to employ an NLP solver. Du J (s, u) can be calculated by solving sensitivity equations. Note that the switch time vector is not explicitly expressed in the ODEs of the system dynamic. We can employ the approach described ∂J ∂J , . . . , ∂s ), where in Sect. 2.4 to calculate Ds J (s, u), i.e., Ds J (s, u) = ( ∂s 1 n ∂J = p T (si ) f (si+ ) + l(si+ ), ∂si ∂lρi T ∂ f ρi T (x, u) p − (x, u), t ∈ [si , si+1 ], p˙ = − ∂x ∂x p(t f ) = Dx ϕ(x(t f ))T , p(si− ) = p(si+ ), i = 1, 2, . . . , n

(3.2)

3.2 Algorithm Development With the gradient information of the objective function with respect to the switch time vector, we no longer need to decompose problem (2.25) into inner and outer problems, and can directly solve its approximate problem. As the iteration proceeds, convergence is achieved until an SIP-Slater point satisfies the finite-dimensional KKT conditions of problem (2.25) (see Fig. 3.1).

3.2 Algorithm Development

63

Fig. 3.1 Graphic illustration of the single-level algorithm

We introduce the approximate problem of problem (2.25) as follows. min

J (s, u)

s.t.

∈ T p, g(x, u) ≤ − p , ∀t   si , si+1 ) , if i = n  x˙ = f ρi (x, u), t ∈  , sn , t f , if i = n x(t0 ) = x0 , i = 0, 1, . . . , n,

s∈S,u∈U

(3.3)

where  p > 0 and T p ⊂ [t0 , t f ] denote the restriction parameter and (finite) discretized constraint points at iteration p. We use the sequential method to solve problem (3.3). As shown in Fig. 3.2, the input is discretized as piecewise polynomials by CVP. The ODEs are solved in an inner loop and the polynomial coefficients and switch time vector are updated by an NLP solver. The single-level algorithm (Algorithm 4) is as follows.

64

3 Single-Level Dynamic Optimization of Path-Constrained Switched Systems

Fig. 3.2 Sketch of the sequential method [1]. Only the input is discretized by CVP. The system dynamic is treated as an initial value problem that can be solved by state-of-the-art ODE solvers. Gradients are calculated either from sensitivity equations or adjoint systems

Remark 3.2 According to Assumption 3.1, for each s ∈ S, there always exists a feasible u satisfying the Slater point conditions. Thus, the satisfaction of the path constraint is only related to the input. In other words, the introduction of the path constraint does not affect the feasible region of s. Therefore, the approximate KKT conditions is only verified for u at step 8 in Algorithm 4. The optimality conditions of s is already satisfied after step 3. Remark 3.3 The optimization strategy in this chapter populates time points corresponding to the maximum violation of the path constraint into existing time point constraints as a “feedback”. Such time point populated can be considered as a punishment and added into the next iteration.

3.3 Convergence Analysis To analyze the convergence of Algorithm 4, we make following assumptions similar to Sect. 2.6. Assumption 3.4 A local NLP solver is available that generates a KKT point of problem (3.3) at each iteration, whenever it is feasible. Assumption 3.5 The KKT multipliers obtained during the solution procedure are nonnegative and uniformly bounded. Now we give the finite convergence result of Algorithm 4.

3.3 Convergence Analysis

65

Algorithm 4 Single-level algorithm for path-constrained switched systems with guaranteed feasibility. Input: initial set T 0 ⊂ [t0 , t f ] (finite or empty); right-hand side restriction parameter 0 > 0; reduction parameter r > 1; termination tolerances stat ≥ act > 0; iteration counter p = 0 Output: s p , u p 1: loop 2: Calculate the gradient information of the objective function by solving sensitivity equations and using the method in Sect. 3.1 3: Solve problem (3.3) to local optimality by using the gradient information 4: if feasible then p p 5: Set (s p , u p ) equal to the optimal solution point. Obtain m gradients ∇u g(x(ti ), u p (ti )), i = 1, 2, . . . , m and m multipliers, consisting of all (up to m) linearly independent gradients of active constraints at u p with their respective multipliers, and a sufficient number of inactive constraint gradients (if needed) with zero 6: Obtain the maximum violation of g(x, u p ) as g max (x, u p ) and the corresponding time point t max (u p ) 7: if g max (x, u p ) ≤ 0 then p p p p p p m 8: if ∇u Js (s p , u p ) + i=0 λi ∇u g(x(ti ), u p (ti )) ≤ stat and λi g(x(ti ), u p (ti )) ∈ p [−λi act , 0], i = 1, 2, . . . , m then 9: terminate 10: else 11: Set  p+1 ←  p /r and T p+1 ← T p (Case 3) 12: end if 13: else 14: Set T p+1 ← T p ∪ t max (u p ) (Case 2) 15: end if 16: else 17: Set  p+1 ←  p /r and T p+1 ← T p (Case 1) 18: end if 19: p ← p + 1 20: end loop

Theorem 3.1 Algorithm 4 converges finitely to a feasible point that satisfies stat approximate KKT conditions (i.e., a KKT point with stat -approximate in u) of problem (2.25) with act -active indices, if Assumptions 2.14, 2.16, and 3.1–3.5 hold. Proof In Algorithm 4, there are three potential outcomes. Case 1: problem (3.3) is infeasible; Case 2: (3.3) is feasible but generates an (2.25)-infeasible point; and Case 3: (3.3) is feasible and generate an (2.25)-feasible point, but the approximate KKT conditions of (2.25) are not satisfied. We prove the infinite occurrence of the three cases is impossible. The finite termination of Cases 1–3 is proved by using the similar idea in [2, 3]. Case 1: problem (3.3) is feasible within finite iterations. By Assumption 3.1 there exists at least one Slater point of problem (2.25) for any s ∈ S. This point is also feasible for problem (3.3) when  p ∈ [0, s ] since T p ⊂ [t0 , t f ]. Therefore, infinite Case 1 is excluded after at most N1 = logr 0 /s updates.

66

3 Single-Level Dynamic Optimization of Path-Constrained Switched Systems

Case 2: there is no more violation of the path constraint after finite iterations. After Case 1 is excluded and from Assumption 3.1, for all iterations k ≥ min = s  /r hold. We first consider a sequence of solutions to (3.3). Since U is compact, we can select a converging subsequence {u m } with the limit point u. ˆ Consider the corresponding optimal time point t m := t max (u m ) for largest violation of the path constraint over [t0 , t f ]. By construction of (3.3) at iteration l, we have g(x(t m , u l ), u l ) ≤ −min < 0, ∀l, m ∈ N with l > m. By continuity of g and compactness of X × U, we know that g is uniformly continuous on X × U. Because x(t, u) ∈ X , ∀(t, u) ∈ T × U, g is essentially uniformly continuous on T × U. Define g(t, ˜ u) := g(x(t, u), u). Thus, for all ¯ > 0 there exists a δ¯ > 0 (independent of any t ∈ T and any u ∈ U due to the uniform continuity of g(t, ˜ u) jointly on T × U) such that for all u and ¯ we have |g(t ˜ m , u) − g(t ˜ m , u l )| < ¯, for all l, m ∈ N the u l that satisfies |u − u l | < δ, min m m l ˜ , u ). Taking ¯ =  2 > 0 and noting with l > m. It follows that g(t ˜ , u) < ¯ + g(t min that g(t ˜ m , u l ) ≤ −min < 0, we have g(t ˜ m , u) < −  2 < 0. Since u m → u, ˆ for any ¯ for all l, m ∈ N with l > m > N2 . Then δ¯ there exists N2 such that |u m − u l | < δ, min g(x(t m , u m ), u m ) < −  2 < 0. Thus, after a finite N2 , the points given by (3.3) in this case are (2.25)-feasible. Case 3: the approximate KKT conditions of (2.25) are satisfied within finite iterations. Assume that the algorithm does not terminate. Then there exists a subsequence {s pl , u pl } of the returned solution sequence of problem (3.3) such that for each pl ∈ N, (s pl , u pl ) is a KKT point for problem (3.3), but the termination criterion is not satisfied for any pl ∈ N. Since (s pl , u pl ) is a KKT point for problem (3.3), then g(x, u pl ) ≤ − pl , ∀t ∈ T pl , ∇u Js (s pl , u pl ) + p

p

m 

p

p

p

λi l ∇u g(x(ti l ), u pl (ti l )) = 0,

i=1 p

λi l [g(x(ti l ), u pl (ti l )) +  pl ] = 0, i = 1, 2, . . . , m p

p

(3.4)

p

According to Assumption 3.5, {λ1 l , λ2 l , . . . , λml } belongs to a compact set, say p p p p p p  pl , and {s pl ,u pl ,λ1 l ,λ2 l ,. . . ,λml ,t1 l ,t2 l ,. . . ,tml } is contained in the compact set l m S × U ×  × (T ) , then it possesses an accumulation point (s ∗ , u ∗ , λ∗1 , λ∗2 , . . . , λ∗m , t1∗ , t2∗ , . . . , tm∗ ). Thus, by the continuity the second equation of (3.4) yields ∇u Js (s ∗ , u ∗ ) +

m 

λi∗ ∇u g(x(ti∗ ), u ∗ (ti∗ )) = 0

(3.5)

i=1

so that for some p¯l ∈ N we have ∇u J (s p¯l , u p¯l ) +

m  i=1

p¯

p¯

p¯

λi l ∇u g(x(ti l ), u p¯l (ti l )) ≤ stat

(3.6)

3.4 Numerical Case Studies

67

After at most N3 = max{ p¯l , logr 0 /s } updates, we have  N3 ≤ s . Thus the third equation of (3.4) gives λiN3 [g(x(tiN3 ), u N3 (tiN3 )) +  N3 ] = 0, i = 1, 2, . . . , m

(3.7)

Since  N3 ≤ s , s ≤ act , we have −λiN3 g(x(tiN3 ), u N3 (tiN3 )) ∈ [−λiN3 act , 0], i = 1, 2, . . . , m

(3.8)

Therefore, (3.6) and (3.8) imply that after at most N3 iterations while Case 1 and Case 2 are excluded, the finite-dimensional KKT conditions are satisfied. This is a contradiction. Hence, the approximate KKT conditions are satisfied within finite iterations.  Remark 3.6 If there does not exist a Slater point of problem (2.25), then solutions of the approximate problem (3.3) are infeasible for problem (2.25). Also, if Slater points of problem (2.25) do not satisfy the approximate KKT conditions, Algorithm 4 may not converge finitely either, since the termination criterion of Algorithm 4 is not only feasible.

3.4 Numerical Case Studies Next, we illustrate the effectiveness of the single-level algorithm. To better compare it to our previous bi-level algorithm in Chap. 2, we utilize the same simulation software and platform, consider the same numerical example and make the same initial settings for the algorithm parameters. We code all of Algorithm 4 in MATLAB Version 9.5.0.944444 (R2018b, win64). The computation is performed on an Intel Core i7-7700 CPU @ 3.60 GHz with 32 GB of RAM.

3.4.1 Effectiveness of the Single-Level Algorithm Consider the same switched system consisting of 

Subsystem 1: Subsystem 2: with x(0) = [0, 2]T .

   0.6 1.2 1 x+ u −0.8 3.4 1     0.2 3 2 x˙ = x+ u −1 0 −1 x˙ =

(3.9a) (3.9b)

68

3 Single-Level Dynamic Optimization of Path-Constrained Switched Systems

Table 3.1 Summary table for the numerical case studies of the single-level algorithm Opt-cost Opt-switch-time g max Computational time(s) 0.0936

0.3958

−2.5308 × 10−4

184.2222

Suppose that the system undergoes one switch at t = s from subsystem 1 to 2 in the time interval t ∈ [0.2, 1.2]. We want to find an optimal input and an optimal switch time s such that the cost function J = (x1 (t f ) − 4)2 +

1 2



2

u 2 dt

(3.10)

0

is minimized subject to the inequality constraints of states and the input over the entire time horizon: x2 − 7 ≤ 0, ∀t ∈ [0, 2], − 10 ≤ u ≤ 10, ∀t ∈ [0, 2].

(3.11)

The initial parameters are set to be the same as that of the bi-level algorithm, i.e., stat = 10−3 , act = 10−3 , 0 = 5 × 10−3 , r = 4 and T 0 = ∅. The input is parameterized by using the control parameterization technique, we consider 30 equidistant subintervals resulting in 30 decision variables. The switch time vector is updated from the initial nominal value s 0 = 0.7. For this example, the algorithm terminates after 5 iterations. The optimal switch time is 0.3958 and the corresponding optimal cost is 0.0936. The maximum of the path constraint is −2.5308 × 10−4 , which confirms that the path constraint is rigorously satisfied over the entire time horizon (see Table 3.1). The optimal input and the corresponding state trajectory are shown in Figs. 3.3 and 3.4. The feasibility result is provided in Fig. 3.5, where the red star indicates the maximum of the path constraint function. As shown in Fig. 3.6, which is a zoomedin figure, the path constraint is rigorously satisfied for all t ∈ [0, 2]. The above results are consistent with the feature that we expect from the algorithm. Remark 3.7 The tolerance of the optimal input obtained can be controlled via adjusting the parameters (see stat , act in Algorithm 4). In addition, the implementation of the control parameterization technique also has an affect on these tolerances. Higher-order polynomials, more subintervals, and adaptive step size can give lower tolerance of the optimal solutions, which may lead to an increase on the computational complexity.

3.4 Numerical Case Studies

69

0.3 0.2 0.1 0 -0.1 -0.2 -0.3 -0.4 -0.5 -0.6 0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Fig. 3.3 Optimal input. The time horizon is subdivided into 30 equidistant subintervals, and the input is parameterized by constants on each subinterval 8 6 4 2 0 -2 -4 -6 -8 -10 0

2

Fig. 3.4 Optimal state trajectory

4

6

8

10

12

14

70

3 Single-Level Dynamic Optimization of Path-Constrained Switched Systems 2 0 -2 -4 -6 -8 -10 -12 -14 -16 0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Fig. 3.5 Path constraint profile. The red star indicates the maximum of g(t), and the path constraint is rigorously satisfied over T = [0, 2] 5

10-4

0 -2.5308 -5

-10

-15

-20 0.3957

0.39576

0.3959

0.3961

Fig. 3.6 Zoomed-in Fig. 3.5 around the maximum value of the path constraint function

3.5 Summary

71

Table 3.2 Summary table for the comparisons between the bi-level algorithm and the single-level algorithm Subsystem 1 → 2 Algorithm 2 Algorithm 4 Optimal cost 0.1030 0.0936 Optimal switch time 0.3947 0.3958 g max −0.0500 −2.5308 × 10−4 Times(s) 207.09 184.22

Subsystem 2 → 1 Algorithm 2 Algorithm 4 1.2503 × 10−13 2.5093 × 10−9 0.6427 0.6427 −5 −5 105.22 68.61

3.4.2 Computational Efficiency We use the same algorithm parameters to solve the same problem by employing the bi-level and single-level algorithm, respectively. The comparison results of computational performance between the two algorithms are shown in Table 3.2. Remark 3.8 The bi-level algorithm proceed as follows: At the inner level, for fixed switch times, we locate a guaranteed feasible input satisfying the KKT conditions to specified tolerances (similar to Algorithm 4). At the outer level, the gradient of the optimal value function with respect to the switch times is calculated. Then the switch times are updated by sequentially solving quadratic programs related to the optimal value function. In the bi-level algorithm, the switch times are optimized in the different level from the that of the input, which results in more redundant nonlinear programs to be solved (e.g., updates of the input at intermediate switch time vectors). Additional NLP solving may also result in loss of precision and increase of computational time during procedures such as function evaluations and Hessian approximations. As can be seen in Table 3.2, the computational efficiency of Algorithm 4 is 11.04% higher for the case of subsystem 1 to 2, and 34.79% for the case of subsystem 2 to 1.

3.5 Summary Considering that the bi-level algorithm is more suitable for problems where the inner level problem is convex or an optimal input can be located analytically, an efficient single-level algorithm is proposed in this chapter, to locate the optimal input and switch times of path-constrained switched systems. This algorithm also converges to a guaranteed feasible solution satisfying the KKT conditions to specified tolerances, but requires no convexity assumptions or analytical solutions. In addition, the singlelevel algorithm has less computational complexity and higher solution efficiency than the bi-level algorithm.

72

3 Single-Level Dynamic Optimization of Path-Constrained Switched Systems

Similar to the bi-level algorithm, the single-level algorithm is also for dynamic optimization problems of switched systems with a fixed switching sequence, and can only locate the optimal input and switch times. Therefore, in the next chapter, we shall remove the restriction on the switching sequence, and consider the case where the switching sequence also needs to be optimized.

References 1. Chachuat, B.: Nonlinear and dynamic optimization: From theory to practice. Technical report, Ècole Polytechnique Fédérale de Lausanne (2007) 2. Mitsos, A.: Global optimization of semi-infinite programs via restriction of the right-hand side. Optimization 60(10–11), 1291–1308 (2011) 3. Fu, J., Faust, J.M.M., Chachuat, B., Mitsos, A.: Local optimization of dynamic programs with guaranteed satisfaction of path constraints. Automatica 62, 184–192 (2015)

Chapter 4

Dynamic Optimization of Switched Systems with Free Switching Sequences

After handling the difficulty of optimizing the input and switch times, in this chapter, we shall remove the restriction of the fixed switching sequence, and consider dynamic optimization problems with free switching sequences. Seeking the best combinatorial optimization among the system input, switch times and the switching sequence has always been a challenging task. In two-stage optimization, existing methods of switching sequence optimization for Stage 2 may be inefficient due to a possible infinite-loop procedure at each step of the algorithms [1]. Apart from the two-stage optimization framework, infinite termination also occurs when the transformed problem does not have bang-bang type solutions in embedding techniques [2–6]. Mixedinteger programming approaches based on the decomposition framework [7–9] can find the optimal input, switch times, and switching sequence, but its master problem is a MIDO problem, which requires relaxation of ODEs coming along with an expensive computational complexity. In this chapter, for the first time, we employ the variant 2 of generalized Benders decomposition (v-2 GBD) framework to separate the optimization of the input and switch times, and that of the switching sequence. In the primal problem, the input and switch times are optimized by using the single-level algorithm proposed in the previous chapter. In the master problem, a support function is constructed by solving a modified primal problem, making the master problem a simple 0-1 mixedinteger linear program without dynamics. The proposed algorithm also guarantees the rigorous satisfaction of path constraints within a finite number of iterations. Numerical case studies are provided to illustrate the effectiveness of the proposed method.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Fu and C. Zhang, Dynamic Optimization of Path-Constrained Switched Systems, Studies in Systems, Decision and Control 459, https://doi.org/10.1007/978-3-031-23428-6_4

73

74

4 Dynamic Optimization of Switched Systems with Free Switching Sequences

4.1 Problem Statement We first present a generalized description of a path-constrained switched system and the corresponding optimization problem to conduct some necessary assumptions and notifications. The switched system considered can run in m different modes indexed by M = {1, 2, . . . , m}, and undergoes n switches at s1 , s2 , . . . , sn in the time interval T = [t0 , t f ]. We define the feasible set of switch times as S = {s ∈ Rn : 0 ≤ s1 ≤ s2 . . . ≤ sn ≤ t f }

(4.1)

and s0 = t0 , sn+1 = t f . The system dynamic is governed by ODEs of a vector field f w : Rn x × Rn u → nx R : x(t) ˙ = f w(t) (x(t), u(t)), ∀t ∈ T x(t0 ) = x0

(4.2)

where the variables x : T → X ⊆ Rn x and u : T → U ⊆ Rn u denote the state and input of the dynamic system, with X and U nonempty and compact. The schedule function w ∈ W := {w : T → {1, 2, . . . , m}} is a piecewise function with s1 , s2 , . . . , sn as its piecewise points, and determines the active subsystem at time t. The dynamic optimization problem for path-constrained switched systems then takes the following form:  min

(w,u)∈W×U

s.t.

tf

J (x(t), u(t)) =

l(x(t), u(t))dt t0

g(x(t), u(t)) ≤ 0, ∀t ∈ T x(t) ˙ = f w(t) (x(t), u(t)), ∀t ∈ T x(t0 ) = x0

(4.3)

For the sake of brevity, we denote l : Rn x × Rn u → R as the running cost regarding the system operation process. Performance indices dependent on the active subsystem and regarding the terminal state can be found in Chaps. 2 and 3. Similar to the previous chapters, the number of path constraints do not pose any further complication for the results herein, we only consider problems with one path constraint g : Rn x × Rn u → R. Notice that the schedule function w characterizes the switching sequence and switch times, therefore, decision variable of problem (4.3) are essentially the input, switch times and switching sequence. We define the switching sequence σ := {σ1 , σ2 , . . . , σn+1 } ∈ {1, 2, . . . , m}n+1 , that is, the system switches from subsystem f σi to subsystem f σi+1 at the switch time si for i = 1, 2, . . . , n. We define Jσ (x, u) as the cost function for a given switching sequence σ. Ds Jσ (x, u) denotes the derivative of Jσ (x, u) with respect to s evaluated at the point (x, u). Similarly, we define Du Jσ (x, u).

4.2 Master Problem Construction

75

Similar to the previous chapters, to ensure the existence and uniqueness of the ODEs and the well-definition of the costate, we make the following assumptions. Assumption 4.1 For all q ∈ M, f q , g and l are Lipschitz and differentiable in their respective arguments. In addition, the derivatives of these functions in their respective arguments are also Lipschitz. Assumption 4.2 x(t) and u(t) are continuous at switch times, respectively.

4.2 Master Problem Construction We conduct GBD that is a decomposition approach such that fixing the switching sequence in problem (4.3) renders a primal problem in continuous variables-that is, the input and switch time vector. In the primal problem, our goal is to optimize the input and switch times under fixed switching sequence such that the switching sequence is not required to be explicitly characterized. Therefore, we employ a piecewise formulation to facilitate the use of perturbation analysis of differential equations in calculating Ds Jσ (x, u). The primal problem can be solved by using the dynamic optimization method in Chap. 3. To optimize the switching sequence, we first introduce a binary variable to explicitly characterize the switching sequence in the system dynamic. Then based on this system description, we solve a modified primal problem to construct the support function of the master problem. This construction of support function makes the master problem a simple 0-1 mixed linear program without dynamics. Finally, we solve the master problem to update the switching sequence and design an integer cut to exclude previously found 0-1 combinations. The different GBD algorithms are distinguished by the way in which the master problems are constructed. The mixed-integer dynamic optimization (MIDO) algorithm proposed in this book is in a v-2 GBD scheme. This variant of GBD is based on the assumption that we can use the optimal solution of the primal problem along with the multiplier vectors for the determination of the support function. The construction of the master problem is discussed in the following.

4.2.1 Switching Sequence Characterization To explicitly represent the switching sequence in the ODEs, we reformulate system (4.2) by introducing a binary variable v(t) := (v1 (t), v2 (t), . . . , vm−1 (t)) ⊂ {0, 1}m−1 . We define J1 = {2, 3, . . . , m − 1}. The reformulated system is defined as

76

4 Dynamic Optimization of Switched Systems with Free Switching Sequences

x(t) ˙ =

m 

α j (t) f j (x(t), u(t)), ∀t ∈ T

j=1

x(t0 ) = x0

(4.4)

where α j (t), j = 1, 2, . . . , m are defined by α1 (t) = v1 (t) α j (t) = (1 − v1 (t)) · · · (1 − v j−1 (t))v j (t), j ∈ J1 = {2, 3, . . . , m − 1}, (4.5) αm (t) = (1 − v1 (t))(1 − v2 (t)) · · · (1 − vm−1 (t)). Notice that the construction of the reformulated problem is a convex combination such that mj=1 αi (t) = 1 for all t ∈ T . Particularly, if m = 2, then the problem is described by x(t) ˙ = v1 (t) f 1 (x(t), u(t)) +(1 − v1 (t)) f 2 (x(t), u(t)), ∀t ∈ T (4.6) x(t0 ) = x0 How the value of v(t) determine the switching sequence is detailed in an example below. Example 4.3 Consider a switched system which can run in 3 different modes: x(t) ˙ = v1 (t) f 1 (x(t), u(t)) + (1 − v1 (t))v2 (t) f 2 (x(t), u(t)) +(1 − v1 (t))(1 − v2 (t)) f 3 (x(t), u(t)), ∀t ∈ T x(t0 ) = x0

(4.7)

For a given time t¯ ∈ T , the relationship between v(t¯) and the active subsystem at time t¯ is shown in Table 4.1. We can obtain that if v1 (t¯) = 1, then the active subsystem at time t¯ is subsystem 1. If v1 (t¯) = 0 and v2 (t¯) = 1, then the active subsystem at time t¯ is subsystem 2. If v1 (t¯) = 0 and v2 (t¯) = 0, then the active subsystem at time t¯ is subsystem 3. To deal with time-dependent binary variable v(t), we subdivide the time interval T into n + 1 sub-intervals according to switch times, and parameterize v(t) by means of a piecewise constant function: v(t) = ν(i), t ∈ [si−1 , si ], i = 1, 2, . . . , n + 1 Table 4.1 Relationship between v(t¯) and the active subsystem at t¯

v(t¯)

Active subsystem

(1, ·) (0, 1) (0, 0)

1 2 3

(4.8)

4.2 Master Problem Construction

77

Fig. 4.1 Example case. Switch times and parameterization points of u are not required to be consistent

In other words, the value of v(t) during the ith sub-interval is given by ν(i). Obviously, ν is of (n + 1) × (m − 1) dimensions. Remark 4.4 The formulation of system (4.4) is similar to the embedding technique proposed in [5]. However, our formulation is a combination of m subsystems instead of n + 1 sub-intervals. Moreover, the parameterization of v is similar to the CVP of u. For a given input and switch times, the subdivision criterion of ν is known, in which case, the solution of system (4.4) can be obtained by piecing together the solutions from the sub-intervals in which each single mode is active. Remark 4.5 The advantage of this system formulation method is that neither the parameterization criterion of v nor s are dependent on that of u. This ensures that the optimization of decision variables is in the whole search space. In other words, our method allows that the system switches not only at parameterization points of u (see Fig. 4.1). The parameterization criterion of v is dependent on s, which is in line with the practical situation.

4.2.2 Support Function Construction The result below closely follows the content in [10]. However, we first use this construction method in the dynamic optimization problem for path-constrained switched systems. Furthermore, the original problem considered here does not contain integer variables explicitly. The successful application of this construction method benefits from the proposed system formulation method for free switch sequences. Based on the embedded system formulation (4.4), we now consider ν as a decision variable of the master problem. The qth primal problem then takes the form:

78

4 Dynamic Optimization of Switched Systems with Free Switching Sequences

min J (x(t), u(t)) s,u

s.t. g(x(t), u(t)) ≤ 0, ∀t ∈ T  q x(t) ˙ = mj=1 α j (t) f j (x(t), u(t)), ∀t ∈ T

(4.9)

x(t0 ) = x0 According to the standard v-2 GBD algorithm, the support function is given by the Lagrange function evaluated at the optimal solution of the primal problem. Notice that we use the sequential method to solve dynamic optimization problems in this book. The system dynamic is not considered as constraints and is solved in an inner loop. Therefore, the ODEs are not contained in the Lagrange function and are kept in the master problem, which leads to another computationally expensive MIDO problem to be solved. In order to construct a master problem without dynamic, we substitute h a (x˙ q (t), x q (t), u q (t), ν) = 0 and h b (x q (t0 ), x0 ) = 0 for the ODEs and treat them as equality constraints. Then the support function of problem (4.9) is given by: q

ξ1 (s q , u q (t), ν) =J (x q (t), u q (t)) + μqg T g(x q (t), u q (t)) + μqa T h a (x˙ q (t), x q (t), u q (t), ν) qT

+ μb h b (x q (t0 ), x0 ) q

(4.10)

q

where μqg , μa and μb are dual multipliers associated with g, h a and h b . q q Notice that Algorithm 4 does not involve calculating μa and μb (i.e., the multipliers associated with the ODEs). Besides, the dual multipliers associated with path constraints obtained from Algorithm 5 are of finite dimensions and are with tolerances. To avoid calculating these dual multipliers, an alternative approach proposed in [10] is adapted, which leads to an equivalent but much more simple master problem. In the sequel, we consider a modification of the primal problem of the form: min J (x(t), u(t)) s,u,ν¯

s.t. g(x(t), u(t)) ≤ 0, ∀t ∈ T h a (x(t), ˙ x(t), u(t), ν) ¯ = 0, ∀t ∈ T

(4.11)

h b (x(t0 ), x0 ) = 0 ν(i) ¯ − ν q (i) = 0, i = 1, 2, . . . , n + 1 where ν q (i) denotes the fixed switching sequence of the primal problem (4.9) at the qth iteration. Remark 4.6 ν¯ is now a continuous search variable. Notice that the differences between problems (4.9) and (4.11) are the relaxed search variables and the additional

4.2 Master Problem Construction

79

integer constraint. In other words, the original binary variables are relaxed but still forced to take integer values by the additional constraint. Obviously, problems (4.9) and (4.11) are identical and the optimization of problem (4.11) will converge in one iteration if the same values are chosen. Then the support function of problem (4.11) is given by: q

¯ =J (x q (t), u q (t)) ξ2 (s q , u q (t), ν) +

n+1 

μq T (i)(ν q (i) − ν(i)) ¯

(4.12)

i=1

¯ − ν q (i) = 0, i = 1, 2, . . . , n + 1. where μq is the set of dual multipliers for ν(i) q q ¯ are stated in the The equivalence of ξ1 and ξ2 (even for a continuous variable ν) following theorem. Theorem 4.7 Let ν¯ ∈ R(n+1)×(m−1) , (s q , u q ) be an optimal solution (not necessarily q q ¯ = ξ2 (s q , u q (t), ν). ¯ global) of problem (4.9), then ξ1 (s q , u q (t), ν) Proof Since (s q , u q , ν q ) is a stationary point for the Lagrange function of problem (4.11), let (s, u) = (s q , u q ) and by the optimality condition we have      ∂ J  q T ∂g  q T ∂h a  + μg + μa 0=   ∂ ν¯ ν q ∂ ν¯ ν q ∂ ν¯ ν q  n+1  q T ∂h b  + μb + μq T (i) I ∂ ν¯ ν q i=1

(4.13)

Substituting (4.13) into (4.12) then gives  ∂ J  ∂ ν¯ ν q   q T ∂h a  + μa ∂ ν¯ ν q  (ν¯ − ν q ) 

q

¯ =J (x q (t), u q (t)) + ξ2 (s q , u q (t), ν) +

μqg T qT

+ μb

 ∂g  ∂ ν¯ ν q  ∂h b  ∂ ν¯ ν q

(4.14)

Since J (x q (t), u q (t)), g(x q (t), u q (t)) and h b (x q (t0 ), x0 ) are implicitly dependent on ν. ¯ It follows that q

¯ =J (x q (t), u q (t)) ξ2 (s q , u q (t), ν)  ∂h a  (ν¯ − ν q ) + μqa T ∂ ν¯ ν q

(4.15)

80

4 Dynamic Optimization of Switched Systems with Free Switching Sequences q

We proceed as follows to verify ξ1 (s q , u q (t), ν). ¯ It is obvious that the dual multipliers associated with g, h a and h b are identical for problems (4.9) and (4.11). Notice that (x q , u q ) is a KKT point of problem (4.9), by the complementary slackness of KKT conditions we have that μqg T g(x q (t), u q (t)) = 0. Together with h b (x q (t0 ), x0 ) = 0, (4.10) implies that q

¯ =J (x q (t), u q (t)) ξ1 (s q , u q (t), ν) + μqa T h a (x˙ q (t), x q (t), u q (t), ν) ¯

(4.16)

From (4.5) we can obtain that ν¯ participates linearly in h a , and thus ¯ = h a (·, ·, ·, ν q ) + h a (·, ·, ·, ν)

 ∂h a  (ν¯ − ν q ) ∂ ν¯ ν q

(4.17)

Together with (4.16) this implies that q

¯ =J (x q (t), u q (t)) ξ1 (s q , u q (t), ν) + μqa T h a (x˙ q (t), x q (t), u q (t), ν q )  ∂h a  + (ν¯ − ν q ) ∂ ν¯ ν q

(4.18)

We also have that h a (x˙ q (t), x q (t), u q (t), ν q ) = 0, and hence q

¯ =J (x q (t), u q (t)) ξ1 (s q , u q (t), ν)  ∂h a  + (ν¯ − ν q ) ∂ ν¯ ν q q

(4.19)

q

Then we obtain that ξ1 (s q , u q (t), ν) ¯ = ξ2 (s q , u q (t), ν) ¯ holds, which completes the proof.  Remark 4.8 For problems with implicit or linearly separable complicating variables, ODEs can be considered as equality constraints and be vanished in the support function. This owes to the dual information of g, h a and h b can be transferred to μq by Theorem 4.7. This comes with the expense of solving the modified primal problem (4.11) which, nevertheless, converges in only one iteration at the primal solution. The master problem then takes the following form: min η ν,η

q

s.t. ξ2 (s q , u q (t), ν) ≤ η, q ∈ Q feas ν(i) ∈ {0, 1}m−1 , i = 1, 2, . . . , n + 1 where Q feas denotes the set of all feasible primal problems to the iterations.

(4.20)

4.3 Algorithm Development

81

Obviously, the master problem (4.20) is a mixed 0-1 linear programming problem that can be solved by standard branch and bound algorithms. Additional constraints, called integer cuts, are required to be added to exclude previously found 0-1 combinations. We design the following integer cuts to exclude the qth integer solution:  (i, j)∈B q



ν j (i) −

ν j (i) ≤ |B q | − 1

(i, j)∈N B q q

B q = {(i, j) : ν j (i) = 1} q

N B q = {(i, j) : ν J (i) = 0}

(4.21)

with i = 1, 2, . . . , n + 1 and j = 1, 2, . . . , m − 1, where |B q | denotes the cardinality of B q .

4.3 Algorithm Development Now, we can state the algorithm procedure of dynamic optimization for pathconstrained switched systems as follows: Algorithm 5 MIDO algorithm for path-constrained switched systems with free switching sequences. Input: termination tolerance ; iteration counter q = 0; lower bound L B = −∞; upper bound U B = ∞; feasible switching sequence variables ν 0 Output: (ν q , s q , u q ) 1: loop 2: For fixed values of ν = ν q , obtain the switching sequence of the system, solve the qth primal problem (2.25) to obtain an optimal value J q and the corresponding solution (s q , u q ) Algorithm 4 3: Set U B = min(U B, J q ) 4: Solve problem (4.11) at the solution (s q , u q ). Obtain the multipliers μq to construct the qth master problem (4.20) fmincon-SQP 5: Solve the qth master problem (4.20) with integer cuts to obtain a solution η q BARON 6: Update the lower bound L B = η q 7: if U B − L B ≤  or the mater problem is infeasible then 8: terminate 9: else 10: Set ν q+1 equal to the integer solution of the qth master problem 11: end if 12: Set q = q + 1 13: end loop

82

4 Dynamic Optimization of Switched Systems with Free Switching Sequences

Remark 4.9 Sometimes a particular ν may render the primal problem infeasible at step 2 and 4, then a feasibility problem (e.g., the l1 -minimization) is solved to determine the multipliers. We omit this case for the sake of brevity, more details can be found in [11, 12]. Remark 4.10 Notice that Algorithm 5 can also optimize the total number of switches as any two switch times or modes merge into one (i.e., si = si+1 or ν(i) = ν(i + 1)). In such a case, the total number of switches only has the opportunity to be smaller. Therefore, if the maximum number of switches is known or pre-specified as Nmax , we can set Nmax as the initial switch times of Algorithms 4 and 5.

4.4 Convergence Analysis At step 2, the primal problems are solved at different 0-1 combinations of ν and yield a sequence of upper bounds. The best/least upper bound (i.e., U B = min(U B, J q )) is kept at step 2 so that the sequence for the iterative upper bounds is monotonically nonincreasing. At step 4, as the iteration proceed, the number of constraints in the master problem is increasing whenever a feasible primal problem is solved. Therefore, the sequence of lower bounds obtained from the master problems is nondecreasing. As the iterations proceed, the upper and lower bounds converge within  or an infeasible master problem is met (i.e., there is no 0-1 combination that makes it feasible). These two termination criterion can be met within a finite number of iterations. Therefore, the finite termination of Algorithm 5 depends on Algorithm 4. Note that GBD has a possibility to exclude portions of the feasible set, when the primal problem is non-convex and the complicating variables are continuous. Nevertheless, the following definition ensures that GBD converges to a local optimum for the mixed-integer case. Definition 4.11 ([13]) A local optimum of an MIDO problem is one which, for fixed binary variables, satisfies local optimality conditions for the resulting primal problem. The concept of local optimum for MIDO problems is not the same as continuous optimization problems, since it makes no sense to speak about a neighborhood in discrete cases. Therefore, [10, 14] also suggest the above definition. For the convergence of Algorithm 5, we carry over this definition to problem (4.3) with KKT conditions. This conservativeness can be vanished in the future by applying a global optimization algorithm.

4.5 Numerical Case Studies

83

4.5 Numerical Case Studies We test our algorithms by implementing them on a numerical example. We encode the algorithms in MATLAB Version 9.9.0.1467703 (R2020b, win64), and perform the computations on an Intel(R) Core i7-7700 CPU @ 3.60 GHz processor. The derivatives of the cost function with respect to s and u are calculated by solving sensitivity equations with ode45, which is also used to solve the system dynamic. The local solution of problem (4.9) and dual multipliers of problem (4.11) are generated by the SQP algorithm encoded in fmincon. The master problems are solved by using BARON Version 17.8.9 [15, 16].

4.5.1 Switch Scheduling We consider a switched system formulated as follows: 

Subsystem 1: Subsystem 2:

   0.6 1.2 1 x+ u −0.8 3.4 1     0.2 3 2 x˙ = x+ u −1 0 −1 x˙ =

(4.22a) (4.22b)

with the initial condition x(0) = [0, 2]T , and can undergo at most one switch at t = s in the time interval t ∈ [0, 2]. The cost function is defined by 1 J = (x1 (t f ) − 4) + 2



2

2

u 2 dt

(4.23)

0

with both the state and input constrained over the entire time horizon: x2 (t) − 7 ≤ 0, ∀t ∈ [0, 2] − 10 ≤ u(t) ≤ 10, ∀t ∈ [0, 2]

(4.24)

We employ the same CVP settings to parameterize the input into 30 decision variables, and solve the problem at an initial point u 0 = 0, s 0 = 0.5 and ν 0 = [1, 0]T , which implies that the system switches from subsystem 1 to 2. The initial parameters and termination tolerances are listed in Table 4.2.

Table 4.2 Parameter settings for Algorithms 4 and 5 Parameter stat act 0 Value

0.001

0.001

0.01

T0

r



∅

4

0.1

84

4 Dynamic Optimization of Switched Systems with Free Switching Sequences 1.5

1

0.5

0

-0.5 0

0.4

0.6427

1.2

1.6

2

Fig. 4.2 Optimal switching sequence. The system switches from subsystem 2 to 1 at s = 0.6427 10-5

10

6

2

-2

-6 0

0.4

0.8

1.2

1.6

2

Fig. 4.3 Optimal input trajectory. The time horizon is subdivided into 30 equidistant subintervals, and the input is parameterized by constants on each subinterval

Algorithm 5 terminates after 3 iterations in 208.0156 s, and the optimal cost is 1.4115 × 10−9 . The system switches from subsystem 2 to 1 at s = 0.6427, which can be seen in Fig. 4.2. Figures 4.3 and 4.4 shows the optimal input and the corresponding state trajectory. The path constraint function is presented in Fig. 4.5. As indicated by the red triangle, the maximum point of the path constraint function is (0, −5) at the last iteration. The detailed iterative procedure of Algorithm 5 is reported in Table 4.3.

4.5 Numerical Case Studies

85

Fig. 4.4 Optimal state trajectory. The initial state is x(0) = [0, 2]T , and x1 (t f ) is optimized to 4 according to the (x1 (t f ) − 4)2 term in the cost function 2 0 -2 -6 -10 -14 -18 -22 0

0.4

0.8

1.2

1.6

2

Fig. 4.5 Path constraint profile. The maximum point of the path constraint function is (0, −5)

86

4 Dynamic Optimization of Switched Systems with Free Switching Sequences

Table 4.3 Iterative procedure of Algorithm 5. Three cases of the switching sequence are evaluated. At iteration 2, the system only runs in subsystem 1, and therefore no switch time optimization is required. At iteration 3, the upper and lower bound converge within 0.1 and the iterative procedure terminates Iteration 1 2 3 Mode sequence Optimal switch time Optimal cost (Upper bound) g max Computational time(s)

[1, 0]T 0.3957 0.0937

[1, 1]T – 546.0955

[0, 1]T 0.6427 1.4115 × 10−9

−5.4552 × 10−4 111.6094

−6.2500 × 10−4 17.5317

−5 69.4531

4.5.2 Feasibility Verification Notice that for ν = [0, 1]T , the maximum of the path constraint function happens to be the initial value of x2 (0) and the state is decreasing. Thus we provide the feasibility result of the first iteration at ν = [1, 0]T to further elaborate on how the path constraint is enforced. Figure 4.6 gives a zoomed-in figure around the maximum of the path constraint function, which confirms that the path constraint is rigorously satisfied over the entire time horizon. Furthermore, we provide the iterative procedure of Algorithm 5 at this switching sequence. For clarification, we conduct the optimization under the initial parameters listed in Table 4.4, and a CVP of 20 equidistant subintervals. As can be seen in Fig. 4.7, the maximizer of the path constraint function at the current iteration is added successively to the next iteration as a time point constraint, until the path constraint is rigorously satisfied.

Fig. 4.6 Path constraint profile at ν = [1, 0]T . The red triangle indicates the point at which the maximum of the path constraint function occurs. As shown in the zoomed-in figure around the maximum of the path constraint function, the path constraint is rigorously satisfied over the entire time horizon

2 0

10-4

-2

-5.4547

-6

0.3957

-10

-14

-18 0

0.4

0.8

1.2

1.6

2

4.6 Summary

87

Table 4.4 Parameter settings for the iterative procedure of Algorithm 5 at ν = [1, 0]T Parameter stat act 0 T0 r Value

0.1

0.1

2

∅

0.01

4

0 1

0

2

0

-2 1 2

-6

-10

-14

-18 0

0.4

0.8

1.2

1.6

2

Fig. 4.7 Iterative procedure of Algorithm 5 at ν = [1, 0]T . The path constraint is not satisfied at iterations p = 0 and p = 1, then the time point constraints of iteration p = 2 contains the maximizers of p = 0 and p = 1

4.6 Summary In this chapter, we propose an MIDO method for dynamic optimization of pathconstrained switched systems in a v-2 GBD algorithm framework. The algorithm terminates finitely with a certification of guaranteed satisfaction for path constraints, and can optimize the switching sequence. In contrast to outer-approximation methods, generalized Benders decomposition can handle binary variables that participate explicitly in the system dynamic [10, 17]. This decomposition optimization scheme also leads to an MILP instead of a mixed-integer dynamic program in the master problem, and therefore requires no relaxations of the ODEs, which is a prohibitively expensive task in branch and bound methods. This benefits from the adoption of two different system formulations in the primal and master problem construction, respectively. The piecewise one is for calculating the derivative of the cost function with respect to the input and switch time vector. The embedded one is for explicitly characterizing the switching sequence by a binary variable, such that the ODEs can be considered as equality constraints and be vanished in the support function.

88

4 Dynamic Optimization of Switched Systems with Free Switching Sequences

In the future research, we can employ the proposed algorithm to problems with multiple conflicting objectives. Another aspect that deserve further studies is to replace the local optimization algorithm with a global one. Furthermore, we can consider the cases of problems with switching costs, switching jumps, system dynamics described by PDEs, and nonsmooth objectives or constraints. In addition, we can apply the proposed method to applications that requires a rigorous satisfaction of path constraints over the entire time horizon, such as obstacle avoidance robots and microbiological fermentation reactions under high-precision conditions.

References 1. Wardi, Y., Egerstedt, M.: Algorithm for optimal mode scheduling in switched systems. In: 2012 American Control Conference (ACC), pp. 4546–4551. IEEE (2012) 2. Das, T., Mukherjee, R.: Optimally switched linear systems. Automatica 44(5), 1437–1441 (2008) 3. Mojica-Nava, E., Meziat, R., Quijano, N., Gauthier, A., Rakoto-Ravalontsalama, N.: Optimal control of switched systems: A polynomial approach. IFAC Proc. Vol. 41(2), 7808–7813 (2008) 4. Mojica-Nava, E., Quijano, N., Rakoto-Ravalontsalama, N.: A polynomial approach for optimal control of switched nonlinear systems. Int. J. Robust Nonlinear Control 24(12), 1797–1808 (2014) 5. Bengea, S.C., DeCarlo, R.A.: Optimal control of switching systems. Automatica 41(1), 11–27 (2005) 6. Wei, S., Uthaichana, K., Žefran, M., DeCarlo, R.A., Bengea, S.: Applications of numerical optimal control to nonlinear hybrid systems. Nonlinear Analysis: Hybrid Syst. 1(2), 264–279 (2007) 7. Bestehorn, F., Hansknecht, C., Kirches, C., Manns, P.: Mixed-integer optimal control problems with switching costs: a shortest path approach. Math. Program. 188(2), 621–652 (2021) 8. Georg Bock, H., Kirches, C., Meyer, A., Potschka, A.: Numerical solution of optimal control problems with explicit and implicit switches. Optim. Methods Softw. 33(3), 450–474 (2018) 9. Kirches, C., Kostina, E., Meyer, A., Schlöder, M., PN SPP1962.: Numerical solution of optimal control problems with switches, switching costs and jumps. Optimization Online, 6888 (2018) 10. Bansal, V., Sakizlis, V., Ross, R., Perkins, J.D., Pistikopoulos, E.N.: New algorithms for mixedinteger dynamic optimization. Comput. Chem. Eng. 27(5), 647–668 (2003) 11. Geoffrion, A.M.: Generalized benders decomposition. J. Optim. Theory Appl. 10(4), 237–260 (1972) 12. Floudas, C.A.: Nonlinear and mixed-integer optimization: fundamentals and applications. Oxford University Press (1995) 13. Ross, R., Bansal, V., Perkins, J.D.: A mixed-interger dynamic optimization approach to simultaneous design and control. American Institute of Chemical Engineers (1998) 14. Sager, S.: Numerical methods for mixed-integer optimal control problems. Der Andere Verlag Tönning (2005) 15. Tawarmalani, M., Sahinidis, N.V.: A polyhedral branch-and-cut approach to global optimization. Math. Program. 103, 225–249 (2005) 16. Sahinidis, N.V.: BARON 17.8.9: Global Optimization of Mixed-Integer Nonlinear Programs, User’s Manual (2017) 17. Schweiger, C.A., Floudas, C.A.: Interaction of design and control: optimization with dynamic models. In: Optimal Control, pp. 388–435. Springer (1998)

Chapter 5

Multi-objective Dynamic Optimization of Path-Constrained Switched Systems

Conflicting decisions have to be made in practical problems, therefore we consider multi-objective dynamic optimization problems (MDOPs) of path-constrained switched systems in this chapter. To guarantee the rigorous satisfaction of path constraints and the finite convergence of the algorithm, we combine the single-objective optimization method proposed in Chap. 3 with -constraint method. Conventional -constraint method may generate weak Pareto solutions if the constraints are not all active [1, 2]. In [3, 4], the modified -constraint method, also known as the augmented -constraint method has been developed, by which solutions generated are guaranteed to be Pareto optimal; however, the theoretical proof of this property is derived from the global optimality. In practice, numerical optimization algorithms usually locate solutions that are not global optimal. Thus, one may ask whether it is local Pareto optimal when a local solution is located for the modified -constraint problem? From the motivations above and following the results in the previous content, we propose a multi-objective dynamic optimization algorithm for path-constrained switched systems, which locates solutions with specified tolerances of local Pareto optimality. The core advantage of this algorithm is that path constraints are guaranteed to be rigorously satisfied over the whole time period within a finite number of iterations. Furthermore, our algorithm is based on the modified -constraint method to avoid generating weak Pareto solutions. Theoretically, the approximated local Pareto optimality is analyzed and the approximation error is given when a solution is located with specified tolerances of KKT conditions. We also demonstrate this algorithm in a numerical example consisting of two subsystems.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Fu and C. Zhang, Dynamic Optimization of Path-Constrained Switched Systems, Studies in Systems, Decision and Control 459, https://doi.org/10.1007/978-3-031-23428-6_5

89

90

5 Multi-objective Dynamic Optimization of Path-Constrained Switched Systems

5.1 Definitions and Preliminary Results In this section, we shall provide the general formulation of multi-objective optimization problems (MOPs) and present, both globally and locally, basic concepts of the corresponding optimal solution, namely Pareto optimality. Subsequently, we briefly review the -constraint method and related theories.

5.1.1 Multi-objective Optimization The MOP consists of multiple and conflicting objective functions that are to be optimized simultaneously. Such problem can be formulated as follows: min J (x) = (J1 (x), J2 (x), . . . , Jm (x)) x

s.t.

x ∈X

(5.1)

where J (x) is a vector-valued objective function composed of m objective functions. Since there are conflicting objective functions in the MOP, the decrease in one objective function may lead to an increase in the other. Therefore, we use Pareto optimality as a substitute for the concept of optimality. Definition 5.1 We say that x ∗ ∈ X is Pareto optimal on X if x ∈ X such that Jk (x) ≤ Jk (x ∗ ), ∀k = 1, 2, . . . , m with Jk (x) < Jk (x ∗ ) for at least one k. Definition 5.2 We say that x ∗ ∈ X is weak Pareto optimal on X if x ∈ X such that Jk (x) < Jk (x ∗ ), ∀k = 1, 2, . . . , m. For a Pareto solution, there exists no other solution that optimizes at least one objective function without deteriorating another one. For a weak Pareto solution, there does not exist other solutions that optimize all the objective functions at the same time, but there may exist solutions that optimize some of the objective functions while keeping others unchanged. Consequently, a solution which is Pareto optimal is also weak Pareto optimal, but not vice versa. The above definitions are in the sense of global optimality, however, verifying whether a point is global optimal is intractable, especially for nonlinear programs and with local gradient-based solvers. For this reason, we further introduce the definition of Pareto optimality in the local sense. Definition 5.3 We say that x ∗ ∈ X is locally Pareto optimal if there exists a neighborhood N (x ∗ ; α) ⊆ {x ∈ X : x − x ∗  < α, α > 0}, such that x ∗ is Pareto optimal on x ∈ X ∩ N (x ∗ ; α). Another important concept is the approximated local Pareto optimality, which is often used in cases where only approximated solutions are available.

5.2 Problem Statement

91

Definition 5.4 We say that x ∗ is δ-approximated local Pareto optimal if for δ > 0 (1) ∃N (x ∗ ; α) ⊆ {x ∈ X : x − x ∗  < α, α > 0}, (2) x ∈ X ∩ N (x ∗ ; α) such that Jk (x) ≤ Jk (x ∗ ) − δ, ∀k = 1, 2, . . . , m with Jk (x) < Jk (x ∗ ) − δ for at least one k.

5.1.2 -Constraint Method The -constraint method is one of the most common solutions to MOPs, it solves a sequence of single-objective optimization problems, in which the rest of the objective functions are restricted by additional constraints. Therefore, it can be easily combined with our previously proposed single-objective optimization algorithm. The -constraint problem is formulated as follows: min Jk (x) x

s.t. J j (x) ≤  j , ∀ j = 1, 2, . . . , m, j = k, x ∈X

(5.2)

where k ∈ {1, 2, . . . , m}.  ∈ Rm−1 denotes restriction parameter of the additional right-hand constraints. Below are two main properties of the -constraint problem. Lemma 5.5 ([2]) For any  ∈ Rm−1 , the following statements hold. (1) If x ∗ is an optimal solution of problem (5.2) on X , then x ∗ is weak Pareto optimal, (2) x ∗ is Pareto optimal on X iff x ∗ solves problem (5.2) for every k = 1, 2, . . . , m, where J j (x) =  j , ∀ j = 1, 2, . . . , m, j = k. We can obtain from the above lemma that the solutions of problem (5.2) are only guaranteed to be weak Pareto optimal, not Pareto optimal. This is due to the fact that the inequality constraints may be inactive, which is quite possible for the non-convex case.

5.2 Problem Statement Consider a switched system which can run in n q different modes indexed by Q = {1, 2, . . . , n q }, and undergoes n switches at s1 , s2 , . . . , sn in the time interval T = [t0 , t f ]. We define the feasible set of switch times as S = {s ∈ Rn : 0 ≤ s1 ≤ s2 . . . ≤ sn ≤ t f } and s0 = t0 , sn+1 = t f .

(5.3)

92

5 Multi-objective Dynamic Optimization of Path-Constrained Switched Systems

The system dynamic is governed by ordinary differential equations of the vector field f ρi : Rn x × Rn u → Rn x :  si , si+1 ) , if i = n  x(t) ˙ = f ρi (x(t), u(t)), t ∈  , sn , t f , if i = n x(t0 ) = x0

(5.4)

where ρi ∈ Q, i = 0, 1, . . . , n denote the active system indices in each sub-interval. The variables x : T → X ⊆ Rn x and u : T → U ⊆ Rn u denote the state and input of the dynamic system, with X and U nonempty and compact. Throughout the chapter, it is assumed that the mode sequence has been pre-specified. The MDOP for path-constrained switched systems then takes the following form: min J = (J1 , J2 , . . . , Jm ) s,u

s.t. g(x(t), u(t)) ≤ 0, ∀t ∈ T,

 si , si+1 ) , if i = n, x(t) ˙ = f ρi (x(t), u(t)), ∀t ∈   sn , t f , if i = n, x(t0 ) = x0 , s ∈ S, u ∈ U

(5.5)

For k = 1, 2, . . . , m,  Jk (s, u) =

tf

lk (x(t), u(t))dt

(5.6)

t0

denotes the kth objective functions with the running cost lk : Rn x × Rn u → R. g : Rn x × Rn u → R denotes the path constraint. We define Ds Jk (s, u) as the derivative of Jk (s, u) with respect to s evaluated at the point (s, u). Similarly, we define Du Jk (s, u). Similar to Chap. 3, we make Assumptions 2.14, 2.16 and 3.1 for k = 1, 2, . . . , m.

5.3 Algorithm Development In the -constraint method, weak Pareto solutions arises in two cases. One case is when there are multiple optimal solutions for the single-optimization problem, and the other case is when the constraint is inactive at the optimal solution. In this section, we formulate a modified -constrained problem for the path-constrained switched system. Then we propose an algorithm to locate guaranteed feasible solutions satisfying the KKT conditions to specified tolerances within a finite number of iterations. Finally, we mathematically analyze the Pareto optimality of the obtained solutions.

5.3 Algorithm Development

93

In order to guarantee the Pareto optimality of the solutions, we employ a modified -constraint method [3], in which, the original -constraints are allowed to be violated and the violation is penalized in the objective function: min Jk − ω

s,u,σ j

m−1 j =k

σj

s.t. J j + σ j ≤  j , ∀ j = 1, 2, . . . , m, j = k, g(x(t), u(t)) ≤ 0, ∀t ∈ T, x(t) ˙ = f ρi (x(t), u(t)), ∀t ∈ T,

(5.7)

x(t0 ) = x0 , s ∈ S, u ∈ U, σ j ∈ R+ , ∀ j = 1, 2, . . . , m, j = k where ω > 0 is the positive weight, and σ j , j = k are slack variables for the original -constraints. As shown in Algorithm 6, the multi-objective dynamic optimization algorithm for the path-constrained switched system is designed as follows: first, we calculate the range of each objective function from the pay-off table, that is, the results from the individual optimization of the m objective functions [5]. Then we divide the range of each objective function. Finally, we lexicographically solve the modified -constraint problem of each objective function, and obtain Pareto solutions with different grid points as their parameters of the -constraints. The rigorous satisfaction of the path constraint guaranteed by employing the method proposed in Chap. 3. Algorithm 6 Multi-objective optimization for the path-constrained switched system. Initialize: number of grid points d 1: Solve single-objective optimization problems of problem (5.5) and create the pay-off table Algorithm 4 2: For each objective function, calculate the ranges and obtain the grid point values (k1 , k2 , . . . , kd ), k = 1, 2, . . . , m 3: for k = 1, 2, . . . , m do 4: for p = 1, 2, . . . , d do p 5: Solve the modified -constraint problem (5.7) with k as the parameter of the -constraint Algorithm 4 6: end for 7: end for

94

5 Multi-objective Dynamic Optimization of Path-Constrained Switched Systems

5.4 Convergence Analysis Many of existing literatures only analyze the modified -constraint method under the assumption of global optimality. Here, we investigate the solution properties in a KKT version. This is due to the fact that while the numerical solutions are, in general, not always global optimal, the local Pareto optimality can be guaranteed once KKT points are available. Also note that at step 5, the solutions generated are with tolerances. Thus, we quantitatively analyze the approximated local Pareto optimality under specified tolerances of KKT conditions. Recall that at step 5 of Algorithm 6, the optimization problem becomes min Jk − ω

s,u,σ j

m−1  j =k

σj p

s.t. J j + σ j ≤ k , ∀ j = 1, 2, . . . , m, j = k, g(x(t), u(t)) ≤ 0, ∀t ∈ T, x(t) ˙ = f ρi (x(t), u(t)), ∀t ∈ T,

(5.8)

x(t0 ) = x0 , s ∈ S, u ∈ U, σ j ∈ R+ , ∀ j = 1, 2, . . . , m, j = k In the following analysis, μ j , j = 1, 2, . . . , m, j = k denote the KKT multipliers of the -constraints. tq and λq , q = 1, 2, . . . , h denote the discrete time points and their KKT multipliers (see Algorithm 4 in Chap. 3). Recall that n and h refers to the dimensions of the switch time vector and the parameterized input, respectively. stat and act denote the specified tolerance of the finite-dimensional KKT conditions corresponding to Algorithm 4. h Theorem 5.6 Let μ = max[stat + ω, 1] and λ = q=1 λq . Under Assumptions 2.14, 2.16 and 3.1–3.5. If (s ∗ , u ∗ ) is a solution generated by Algorithm 6, then (s ∗ , u ∗ ) is δ-approximated local Pareto optimal for some neighborhood N ((s ∗ , u ∗ ); α) ⊆ {(s, u) ∈ S × U : (s, u) − (s ∗ , u ∗ )2 < α, α > 0}

(5.9)

with δ=

α(n + h)stat + λact + o(α) mμ

(5.10)

Proof Since the generated solution (s ∗ , u ∗ ) satisfies the finite-dimensional KKT conditions with stat and act as the tolerances, we have

5.4 Convergence Analysis

95

 ⎡ ⎤  ∇ J j (s ∗ , u ∗ ) 1   ⎢ ⎥ 2 0 ⎡ ⎢ ⎥ ⎤  ⎢ ⎥ .. ..  ∇ Jk (s ∗ , u ∗ ) 1 ⎢ ⎥ . . ⎢ ⎢ ⎥ ⎥2  −ω ⎢ ⎢ ⎥j − 1 0 ⎥ m−1 ⎢ ⎢ ⎥ ⎥ .. + j =k μ j ⎢ .. ⎣ ⎥ j 1 ⎦. .  ⎢ ⎥  ⎢ ⎥j + 1 0 −ω m  ⎢ ⎥  ⎢ ⎥ . . ..  ⎣ ⎦ ..   m 0  ⎡ ⎤ ∇g(x ∗ (tq ), u ∗ (tq )) 1   ⎢ ⎥2 h 0  ⎢ ⎥  + λq ⎢ ⎥ ..  ≤ stat .. ⎣ ⎦. q=1 .  m 0 λq g(x ∗ (tq ), u ∗ (tq )) ∈ [−λq act , 0], q = 1, 2, . . . , h

(5.11)

(5.12)

The stationarity condition (5.11) implies that   m−1   μ j ∇ J j (s ∗ , u ∗ ) ∇ Jk (s ∗ , u ∗ ) +  j =k   h   + λq ∇g(x ∗ (tq ), u ∗ (tq )) ≤ stat ,  q=1 −ω + μ j ≤ stat

(5.13)

(5.14)

Since g is essentially uniformly continuous on S × U, from the first-order Taylor expansion, for (s, u) ∈ S × U ∩ N ((s ∗ , u ∗ ); α) we have Jk (s, u) +

m−1  j =k

−Jk (s ∗ , u ∗ ) −

μ j J j (s, u) + m−1  j =k

∗

= ∇ Jk (s , u ) + +

h  q=1

λq g(x(tq ), u(tq ))

q=1

 ∗

h 

∗

μ j J j (s ∗ , u ∗ ) −

m−1  j =k

h 

λq g(x ∗ (tq ), u ∗ (tq ))

q=1

(5.15) ∗

∗

μ j ∇ J j (s , u ) T  ∗

λq ∇g(x (tq ), u (tq ))

 s − s∗ + o(α) u − u∗

Note that s is n-dimensional and u is h-dimensional after the CVP technique is performed. Let μk = 1 and substitute (5.13) into (5.15), we can obtain that

96

5 Multi-objective Dynamic Optimization of Path-Constrained Switched Systems h     μ j J j (s, u) − J j (s ∗ , u ∗ ) + λq g(x(tq ), u(tq ))

m  j=1

∗

q=1



∗

(5.16)

−g(x (tq ), u (tq )) ≥ −α(n + h)stat + o(α) Since μ = max[stat + ω, 1], then (5.16) becomes h     μ J j (s, u) − J j (s ∗ , u ∗ ) + λq g(x(tq ), u(tq ))

m  j=1

∗

q=1



∗

(5.17)

−g(x (tq ), u (tq )) ≥ −α(n + h)stat + o(α) In the second term of (5.17), λq is the nonnegative KKT multipliers, thus we consider two possible cases: For λq = 0, obviously 

  λq g(x(tq ), u(tq )) − g(x ∗ (tq ), u ∗ (tq )) = 0

(5.18)

q|λq =0

For λq > 0, (s, u) is feasible and g(x(tq ), u(tq )) ≤ 0, then λq g(x(tq ), u(tq )) ≤ 0. Meanwhile, from (5.12) we have that λq g(x ∗ (tq ), u ∗ (tq )) ≥ −λq act . Hence 

   λq g(x(tq ), u(tq )) − g(x ∗ (tq ), u ∗ (tq )) ≤ λq act

q|λq >0

(5.19)

q|λq >0

Together with (5.18) this implies that h 



∗

∗



λq g(x(tq ), u(tq )) − g(x (tq ), u (tq )) ≤

λq act

(5.20)

  μ J j (s, u) − J j (s ∗ , u ∗ ) ≥ − α(n + h)stat − λact + o(α)

(5.21)

q=1

Substituting (5.20) and λ = m 

h  q=1

h q=1

λq into (5.17) then gives

j=1

Our claim is that (s ∗ , u ∗ ) is δ-approximated local Pareto optimal with stat +λact + o(α) for some neighborhood N ((s ∗ , u ∗ ); α) ⊆ {(s, u) : δ = α(n+h)mμ ∗ ∗ (s, u) − (s , u )2 < α}. On the contrary we assume that there exist (s, u) ∈ S × U ∩ N ((s ∗ , u ∗ ); α) such that J j (s, u) ≤ J j (s ∗ , u ∗ ) − δ for all j = 1, 2, . . . , m, with at least one strict inequality. Now consider the case that J j (s, u) ≤ J j (s ∗ , u ∗ ) −

α(n + h)stat + λact + o(α) mμ

(5.22)

5.5 Numerical Case Studies

97

Moving J j (s ∗ , u ∗ ) to the left-hand side and multiplying through μ yield   α(n + h)stat + λact μ J j (s, u) − J j (s ∗ , u ∗ ) ≤ − + o(α) m

(5.23)

Since (5.23) holds for each j = 1, 2, . . . , m with at least one strict inequality, we can obtain that m 

  μ J j (s, u) − J j (s ∗ , u ∗ ) < − α(n + h)stat − λact + o(α)

(5.24)

j=1



which is a contradiction to (5.21).

Remark 5.7 In practice, the tolerance stat and the weight parameter ω are sufficiently small such that μ = 1. From the form of the term δ, which essentially implies the maximal approximation error of the generated solution, we can obtain the following conclusions. First of all, δ can be directly controlled by user-specified tolerances stat and act . Another factor that affects δ is λ, which reflects the influence of the path constraint on the optimal problem. Moreover, more objective functions leads to a smaller δ.

5.5 Numerical Case Studies We validate the effectiveness of our algorithm on a numerical example, which consists of two subsystems with both state and input constraints. We encode the algorithm in MATLAB Version 9.9.0.1467703 (R2020b, win64), and perform the computations on an Intel(R) Core i7-7700 CPU. The derivatives of the cost function with respect to s and u are calculated with ode45, which is also used to solve the system dynamic. The related optimization problems are solved by the sequential quadratic programming method encoded in fmincon.

5.5.1 Pareto Solutions Consider a switched system formulated as follows: 

Subsystem 1: Subsystem 2:

   0.6 1.2 1 x˙ = x+ u. −0.8 3.4 1     0.2 3 2 x˙ = x+ u −1 0 −1

(5.25a) (5.25b)

98

5 Multi-objective Dynamic Optimization of Path-Constrained Switched Systems

Table 5.1 Parameter settings for Algorithm 4 Parameter stat act Value

0.001

0.001

0

T0

r

0.05

∅

4

with the initial condition x(0) = [0, 2]T , and can undergo at most one switch at t = s in the time interval t ∈ [0, 2], where s ∈ [0.2, 1.2]. The objective functions are defined by J1 = (x2 (t f ) − 4)2 +

1 2



2

u 2 dt, 0

J2 = x1 (t f )

(5.26)

with both the state and input constraints: x2 (t) − 3 ≤ 0, ∀t ∈ [0, 2], − 10 ≤ u(t) ≤ 10, ∀t ∈ [0, 2], 0 ≤ x1 (t f ) + x2 (t f ) ≤ 10

(5.27)

As can be seen from the objective functions J1 , J2 and the constraint 0 ≤ x1 (t f ) + x2 (t f ) ≤ 10, there is a trade-off between the two objective functions. The state x2 and the input u are path-constrained during the whole time period t ∈ [0, 2]. Our goal is to generate Pareto solutions (i.e., optimal inputs and switch times) while guaranteeing the rigorous satisfaction of the path constraints. For this example, the initial parameters and termination tolerances of Algorithm 4 are listed in Table 5.1. During the optimization of the input, a piece-wise constant CVP with 30 equidistant sub-intervals is performed resulting in 30 decision variables. We divide the range of J2 into 9 segments, that is, the number of grid points is set to be 10. The nonnegative weight ω is set to be 10−4 . We solve the -constraint problems all at u 0 = 0 and s 0 = 0.7. Figure 5.1 shows the obtained 10 Pareto points, each of which corresponds to an optimal input and switch time. The ranges of J1 and J2 are [11.4707, 13.0704] and [−2.9992, 0.0998], respectively. The computational time is 478.7 s.

5.5.2 Feasibility Verification To verify the satisfaction of the path constraint, we provide results of the Pareto point A in Fig. 5.1 for example. The optimal value of point A is [J1 , J2 ] = [11.5906, −1.2775]. The corresponding optimal input and state is shown in Figs. 5.2

5.5 Numerical Case Studies

99

0.5 0 -0.5 -1

A

-1.5 -2 -2.5 -3 -3.5 11.4

11.6

11.8

12

12.2

12.4

12.6

12.8

13

13.2

Fig. 5.1 Pareto points generated by Algorithm 6 4 3 2 1 0 -1 -2 -3 -4 -5 0

0.5

1

1.5

2

Fig. 5.2 Optimal input trajectory. The time horizon is subdivided into 30 equidistant subintervals, and the input is parameterized by constants on each subintervals

and 5.3, and the optimal switch time is 0.2. The path constraint profile is presented in Fig. 5.4. As indicated by the red triangle, the maximum point of the path constraint function is (0.2, −0.1994), which confirms that the path constraint is rigorously satisfied over the entire time horizon.

100

5 Multi-objective Dynamic Optimization of Path-Constrained Switched Systems 3

2.5

2

1.5

1

0.5

0

-0.5 -2

-1

0

1

2

3

4

5

Fig. 5.3 Optimal state trajectory. The initial state is x(0) = [0, 2]T 0.5 0 -0.5 -1 -1.5 -2 -2.5 -3 -3.5 0

0.5

1

1.5

2

Fig. 5.4 Path constraint profile. The maximum point of the path constraint function is (0.2000, −0.1994), which confirms that the path constraint is rigorously satisfied over the entire time horizon

References

101

5.6 Summary This chapter presents a multi-objective dynamic optimization algorithm for pathconstrained switched systems. Within a finite number of iterations, the algorithm locates local Pareto solutions with specified tolerances and guarantees the rigorous satisfaction of path constraints. In addition, the approximated local optimality of the Pareto solutions is quantitatively analyzed with specified tolerances of the KKT conditions of single-objective modified -constraint dynamic optimization problems. In the proposed algorithm, we divided the ranges of objective functions into equidistant segments to obtain -constraint values, which leads to an uneven distribution of solutions (see Fig. 5.1). It deserves further studies on how to adaptively select appropriate -constraint values during the run to generate solutions with an even distribution.

References 1. Chankong, V., Haimes, Y.Y.: Multiobjective decision making: theory and methodology. Courier Dover Publications (2008) 2. Kaisa, M.: Nonlinear multiobjective optimization. In: International Series in Operations Research & Management Science, vol. 12. Springer Science & Business Media, New York (1998) 3. Ehrgott, M., Ruzika, S.: Improved ε-constraint method for multiobjective programming. J. Optim. Theory Appl. 138(3), 375–396 (2008) 4. Mavrotas, G.: Effective implementation of the ε-constraint method in multi-objective mathematical programming problems. Appl. Math. Comput. 213(2), 455–465 (2009) 5. Cohon, J.L.: Multiobjective Programming and Planning. Academic, New York (1978)

Chapter 6

Conclusions and Future Work

Many practical systems and industrial processes involves dynamic optimization. For hybrid systems, the simultaneous appearance of switching mechanisms and path constraints poses great challenges to both theoretical studies and numerical solutions of optimization problems. On the one hand, there is an urgent need to develop a general theoretical framework for dynamic optimization to systems with the underlying characteristics. On the other hand, we need to design numerical solution algorithms that can meet the practical requirements. Therefore, in this book, we address the dynamic optimization problem of path-constrained switched systems, considering two basic requirements in practice, that are, finite iteration of the algorithm and rigorous satisfaction of path constraints, and conduct the following research. (1) Dynamic optimization of path-constrained switched systems with a fixed switching sequence. In Chaps. 2 and 3, we consider the dynamic optimization problems= of pathconstrained switched systems with a fixed switching sequence. In Chap. 2, we propose a bi-level dynamic optimization method. The main contributions of this work include three aspects: first, we propose a dimensionality reduction method for path constraints and gives sufficient conditions under which the path constraint can be replaced by a finite number of discrete point constraints. In addition to facilitating the subsequent analysis of the differentiability of the optimal value function, this method can also be used to deal with general path constrained dynamic optimization problems. Secondly, we analyze the differentiability of the optimal value function in the dynamic optimization problem of path-constrained switched systems by using perturbation analysis theory, and also give the derivative calculation formula. In this way, we can use the gradient information of the optimal value function in the outer level to update the switch times. In contrast, existing results only considered the unconstrained case. Finally, we design a dynamic optimization algorithm that can locate guaranteed feasible input and © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 J. Fu and C. Zhang, Dynamic Optimization of Path-Constrained Switched Systems, Studies in Systems, Decision and Control 459, https://doi.org/10.1007/978-3-031-23428-6_6

103

104

6 Conclusions and Future Work

switch times satisfying the KKT conditions within a specified tolerance, and mathematically prove the finite convergence of the algorithm. In Chap. 3, we propose an efficient single-level dynamic optimization method. Different from the bi-level method, this method can optimize the system input and switch times simultaneously, which greatly reduce the number of NLPs leading to fewer function evaluations and Hessian approximations that result in less computational burden. In Chap. 3, we propose an efficient single-layer dynamic optimization method. Unlike the two-layer dynamic optimization algorithm, the single-layer algorithm can optimize both system inputs and switching moments, which somewhat reduces the number of NLPs to be solved, and thus the number of function estimations and Hessian approximations. While improving the solution efficiency and accuracy, the single-level dynamic optimization algorithm can also locate guaranteed feasible input and switch times satisfying the KKT conditions within a specified tolerance, and converges within a finite number of iterations. (2) Dynamic optimization of path-constrained switched systems with free switching sequences. Furthermore, we remove the restriction on the switching sequence, and consider a more general case where the switching sequence also needs to be optimized. In Chap. 4, we propose a v-2 GBD-based solution framework, and design an MIDO algorithm for path-constrained switched systems. This algorithm terminates finitely with a certification of guaranteed satisfaction for path constraints, and can optimize the switching sequence. The core advantage of this algorithm is that it leads to an MILP instead of a mixed-integer dynamic program in the master problem, and therefore requires no relaxations of the ODEs, which is a prohibitively expensive task in branch and bound methods. This benefits from the adoption of two different system formulations in the primal and master problem construction, respectively. The piecewise one is for calculating the derivative of the cost function with respect to the input and switch time vector. The embedded one is for explicitly characterizing the switching sequence by a binary variable, such that the ODEs can be considered as equality constraints and be vanished in the support function. For different requirements in practice, different optimization algorithms can be employed to solve the primal and master problem. For example, for the case that there are no requirements of the rigorous satisfaction of path constraints and finite convergence of the algorithm, we can employ heuristic algorithms to deal with problems with unknown models, and to increase the convergence speed. (3) Multi-objective dynamic optimization of path-constrained switched systems Considering multiple conflicting decisions in practical problems, we propose a multi-objective dynamic optimization algorithm in Chap. 5. Within a finite number of iterations, this algorithm locates local Pareto solutions with specified tolerance and guarantees the rigorous satisfaction of path constraints. The upper bound of the approximation error can be controlled via user-specified tolerances. The main contribution of this work is that, for the first time, we quantitatively analyzed the approximated local optimality of the Pareto solutions with speci-

6 Conclusions and Future Work

105

fied tolerances of the KKT conditions of single-objective modified -constraint dynamic optimization problems. Moreover, since the definition of Pareto optimality is in the sense of objective function rather than optimal condition, we further give the calculation formula of the upper bound of the approximation error on the objective function. Although a series of theoretical results and numerical solution methods are proposed and are validated in numerical simulations, there are still many aspects that need further exploration, for example: (1) In practice, many dynamic optimization problems exhibit sub-optimal local minima almost pathologically due to nonconvexity of the functions participating in most system models [1]. Implementing a sub-optimal operating policy on a real process can have direct economic, safety, and environmental impacts. For example, the sub-optimal solution is not the exact parameter that we are seeking, then the whole system may fail. This has motivated us to study global optimization algorithms for path-constrained switched systems. (2) In many industrial applications, parts of the problem/models are not known accurately and only input-output data is available. In this case, deterministic optimization methods does not apply and heuristic optimization methods or machine learning motivated approaches are required to be developed. Therefore, we shall study hybrid dynamic optimization methods for path-constrained switched systems. As mentioned in earlier, algorithms in the primal and master problems of the v-2 GBD solution framework can be replaced by heuristic algorithms to deal with problems with unknown models, and to improve the convergence speed. (3) In this book, all of the proposed method is only verified in numerical simulations. Since there is a significant gap between practical systems and numerical cases, further verification is needed to validate whether the proposed method can achieve the desired effect in practical applications. In particular, in order to ensure the convergence and feasibility of the algorithm, the proposed algorithms have limited computational complexity and can only be implemented off-line for existing computing platforms. Therefore, in the subsequent research, we need to gradually remove the assumptions that restrict the application of the proposed methods to practical systems, such as the assumptions of differentiability of objective functions, constraints, and differential equations of the system, etc., and consider problems with state jumps and switching costs that are more practical [2–4], and explore the possibility of real-time online optimization on these bases.

106

6 Conclusions and Future Work

References 1. Chachuat, B., Singer, A.B., Barton, P.I.: Global methods for dynamic optimization and mixedinteger dynamic optimization. Ind. Eng. Chem. Res. 45(25), 8373–8392 (2006) 2. Bestehorn, F., Hansknecht, C., Kirches, C., Manns, P.: Mixed-integer optimal control problems with switching costs: a shortest path approach. Math. Program. 188(2), 621–652 (2021) 3. Georg Bock, H., Kirches, C., Meyer, A., Potschka, A.: Numerical solution of optimal control problems with explicit and implicit switches. Optim. Methods Softw. 33(3), 450–474 (2018) 4. Kirches, C., Kostina, E., Meyer, A., Schlöder, M., PN SPP1962.: Numerical solution of optimal control problems with switches, switching costs and jumps. Optimization Online, 6888 (2018)