Iterative Learning Control for Systems with Iteration-Varying Trial Lengths: Synthesis and Analysis [1st ed.] 978-981-13-6135-7, 978-981-13-6136-4

Table of contents :
Front Matter ....Pages i-xiv
Introduction (Dong Shen, Xuefang Li)....Pages 1-14
Front Matter ....Pages 15-15
Averaging Techniques for Linear Discrete-Time Systems (Dong Shen, Xuefang Li)....Pages 17-32
Averaging and Lifting Techniques for Linear Discrete-Time Systems (Dong Shen, Xuefang Li)....Pages 33-47
Moving Averaging Techniques for Linear Discrete-Time Systems (Dong Shen, Xuefang Li)....Pages 49-65
Switching System Techniques for Linear Discrete-Time Systems (Dong Shen, Xuefang Li)....Pages 67-80
Two-Dimensional Techniques for Linear Discrete-Time Systems (Dong Shen, Xuefang Li)....Pages 81-99
Front Matter ....Pages 101-101
Moving Averaging Techniques for Nonlinear Continuous-Time Systems (Dong Shen, Xuefang Li)....Pages 103-117
Modified Lambda-Norm Techniques for Nonlinear Discrete-Time Systems (Dong Shen, Xuefang Li)....Pages 119-134
Sampled-Data Control for Nonlinear Continuous-Time Systems (Dong Shen, Xuefang Li)....Pages 135-161
CEF Techniques for Parameterized Nonlinear Continuous-Time Systems (Dong Shen, Xuefang Li)....Pages 163-192
CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems (Dong Shen, Xuefang Li)....Pages 193-224
CEF Techniques for Uncertain Systems with Partial Structure Information (Dong Shen, Xuefang Li)....Pages 225-254
Back Matter ....Pages 255-256

Citation preview

Dong Shen · Xuefang Li

Iterative Learning Control for Systems with IterationVarying Trial Lengths Synthesis and Analysis

Iterative Learning Control for Systems with Iteration-Varying Trial Lengths

Dong Shen Xuefang Li •

Iterative Learning Control for Systems with Iteration-Varying Trial Lengths Synthesis and Analysis

123

Dong Shen College of Information Science and Technology Beijing University of Chemical Technology Beijing, China

Xuefang Li Department of Electrical and Electronic Engineering Imperial College London London, UK

ISBN 978-981-13-6135-7 ISBN 978-981-13-6136-4 https://doi.org/10.1007/978-981-13-6136-4

(eBook)

Library of Congress Control Number: 2018967742 © Springer Nature Singapore Pte Ltd. 2019 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

To Prof. Jian-Xin Xu National University of Singapore

Preface

As an effective control approach, iterative learning control (ILC) is designed to improve the current control performance of repetitive processes by utilizing past control information. Specially, ILC generates/synthesizes the current control input from previous control inputs and tracking errors, which thus is able to learn from past control experience and improve the current tracking performance from iteration to iteration. In fact, ILC mimics the learning process of human being. In contrast to most existing control methods aiming at achieving asymptotic convergence, ILC is focusing on control systems working on a finite time interval and targets at perfect tracking performance at every time instant. During the past decades, ILC has been intensively investigated in theory and applied in real-time applications. In traditional ILC, it is well known that many perfectly repeating conditions, such as fixed time interval, identical initial state, identical tracking target, identical operation environments, etc., are required, which hinder practical applications of ILC. In literature, many efforts have been devoted to relax one or more of these constraints in ILC. As a monograph, this book is dedicated to the application of ILC to systems with iteration-varying trial lengths. In detail, the iteration/trial length is assumed to be randomly varying in the iteration domain. For the case that the trial lengths are equal to or longer than the desired length, the redundant operating/ control information can be discarded directly since they are useless for learning, and the ILC design for such a case is therefore identical to standard ILC design. If the trial length is shorter than the desired one, it implies that the operation ends earlier than the desired case, and thus some of the tracking information are missing, which may be unavailable for learning. For such a case, how to design suitable learning algorithms to compensate the missing tracking/control information and how to analyze their convergence properties with effective mathematical tools become challenging problems. This book provides detailed ILC design and analysis approaches for systems with iteration-varying trial lengths in a systematic manner. The main content can be classified into two parts following the linear and nonlinear systems: Chaps. 2–6 focus on linear control systems, while Chaps. 7–10 concentrate on nonlinear control systems.

vii

viii

Preface

This monograph is self-contained. Prior knowledge from other literatures is not required when reading the main body of the monograph. The book is written for students, engineers, and researchers working in the field of ILC. Both authors would like to express their sincere appreciation to all their collaborators and students supervised since some of the chapters in this monograph originate from the joint articles with them. The authors would like to thank all ILC experts worldwide as they benefited a lot from the previous foundational works when preparing this book. Finally, the authors are very thankful to their family for their unconditional love, encouragement, and support. This monograph is dedicated to the late Prof. Jian-Xin Xu from National University of Singapore, an IEEE Fellow, a leading expert in various fields of systems and control, and an excellent model with his passion and dedication to science and engineering for us. Beijing, China London, UK November 2018

Dong Shen Xuefang Li

Acknowledgements

We are pleased to thank the support of the National Natural Science Foundation of China under grants 61673045 and 61304085, and Beijing Natural Science Foundation under grant 4152040. We sincerely appreciate Elsevier, IEEE, and John Wiley & Sons for granting us the permission to reuse materials in the papers copyrighted by these publishers.

ix

Contents

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

1 1 3 3 6 7 10 12 13

2

Averaging Techniques for Linear Discrete-Time Systems 2.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . 2.2 ILC Design and Convergence Analysis . . . . . . . . . . 2.3 Extension to Time-Varying Systems . . . . . . . . . . . . 2.4 Illustrative Simulations . . . . . . . . . . . . . . . . . . . . . . 2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

17 17 19 25 26 32 32

3

Averaging and Lifting Techniques for Linear Discrete-Time Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 ILC Design and Convergence Analysis . . . . . . . . . . . . . 3.3 Extension to Time-Varying Systems . . . . . . . . . . . . . . . 3.4 Illustrative Simulations . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

33 33 36 40 41 47 47

1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Iterative Learning Control . . . . . . . . . . . . . . 1.2 Basic Formulation of ILC . . . . . . . . . . . . . . 1.2.1 Discrete-Time Case . . . . . . . . . . . . 1.2.2 Continuous-Time Case . . . . . . . . . . 1.3 ILC for Systems with Varying Trial Lengths 1.4 Structure of this Monograph . . . . . . . . . . . . 1.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Part I

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

Linear Systems

xi

xii

4

5

6

Contents

Moving Averaging Techniques for Linear Discrete-Time Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Controller Design I and Convergence Analysis . . . . . 4.3 Controller Design II and Convergence Analysis . . . . 4.4 Illustrative Simulations . . . . . . . . . . . . . . . . . . . . . . 4.4.1 Simulations for ILC Law (I) . . . . . . . . . . . . 4.4.2 Simulations for ILC Law (II) . . . . . . . . . . . 4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

49 49 50 57 60 60 61 64 65

Switching System Techniques for Linear Discrete-Time Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . 5.2 ILC Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Strong Convergence Properties . . . . . . . . . . . . . . . 5.4 Illustrative Simulations . . . . . . . . . . . . . . . . . . . . . 5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

67 67 70 74 78 80 80

Two-Dimensional Techniques for Linear Discrete-Time Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . 6.2 Learning Gain Matrix Design . . . . . . . . . . . . . . . . 6.3 Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . 6.4 Alternative Scheme with Distribution Estimation . . 6.5 Illustrative Simulations . . . . . . . . . . . . . . . . . . . . . 6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

81 81 83 87 91 95 99 99

Moving Averaging Techniques for Nonlinear Continuous-Time Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 ILC Design and Convergence Analysis . . . . . . . . . . . . . . . 7.3 Extension to Non-affine Nonlinear Systems . . . . . . . . . . . . 7.4 Illustrative Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

103 103 104 111 113 116 117

Part II 7

8

Nonlinear Systems

Modified Lambda-Norm Techniques for Nonlinear Discrete-Time Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 8.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 8.2 ILC Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

Contents

8.3 Convergence Analysis . 8.4 Illustrative Simulations 8.5 Summary . . . . . . . . . . References . . . . . . . . . . . . . . 9

xiii

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

124 130 133 134

Sampled-Data Control for Nonlinear Continuous-Time Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2 Sampled-Data ILC Design and Convergence Analysis . . . 9.2.1 Generic PD-type ILC Scheme . . . . . . . . . . . . . . 9.2.2 The Modified ILC Scheme . . . . . . . . . . . . . . . . 9.3 Sampled-Data ILC Design with Initial Value Fluctuation 9.3.1 Generic PD-type ILC Scheme . . . . . . . . . . . . . . 9.3.2 The Modified ILC Scheme . . . . . . . . . . . . . . . . 9.4 Illustrative Simulations . . . . . . . . . . . . . . . . . . . . . . . . . 9.4.1 Generic PD-type ILC Scheme . . . . . . . . . . . . . . 9.4.2 The Modified ILC Scheme . . . . . . . . . . . . . . . . 9.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

135 135 139 139 145 149 150 152 154 154 157 160 161

. . . . . . . . .

. . . . . . . . .

163 164 165 174 174 175 178 179 181

. . . .

. . . .

181 186 192 192

. . . . .

. . . . .

193 193 196 196 204

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

10 CEF Techniques for Parameterized Nonlinear Continuous-Time Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2 ILC Algorithm and Its Convergence . . . . . . . . . . . . . . . . . . 10.3 Effect of Random Trial Lengths and Parameters . . . . . . . . . . 10.4 Extensions and Discussions . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.1 Unknown Lower Bound of the Input Gain . . . . . . . . 10.4.2 Iteration-Varying Tracking References . . . . . . . . . . . 10.4.3 High-Order Systems . . . . . . . . . . . . . . . . . . . . . . . . 10.4.4 Multi-input-Multi-output Systems . . . . . . . . . . . . . . 10.4.5 Parametric Systems with Nonparametric Uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5 Illustrative Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Robust ILC Algorithms and Their Convergence Analysis 11.2.1 Norm-Bounded Uncertainty Case . . . . . . . . . . . 11.2.2 Variation-Norm-Bounded Uncertainty Case . . . . 11.2.3 Norm-Bounded Uncertainty with Unknown Coefficient Case . . . . . . . . . . . . . . . . . . . . . . . .

. . . . .

. . . . .

. . . . .

. . . . . 208

xiv

Contents

11.3 Extension to MIMO System . . . . . . . . . . . . . . . . . 11.3.1 Norm-Bounded Uncertainty Case . . . . . . . 11.3.2 Variation-Norm-Bounded Uncertainty Case 11.4 Illustrative Simulations . . . . . . . . . . . . . . . . . . . . . 11.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

210 211 217 218 222 224

12 CEF Techniques for Uncertain Systems with Partial Structure Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2 Time-Invariant and Time-Varying Mixing Scheme . . . . . . . 12.3 Differential-Difference Hybrid Scheme . . . . . . . . . . . . . . . . 12.4 Illustrative Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

225 225 228 237 245 252 254

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255

Chapter 1

Introduction

This chapter provides a rudimentary introduction of iterative learning control (ILC) and its basic formulation for both discrete-time and continuous-time systems, which is followed by a review on recent developments of ILC with iteration-varying trial lengths. At the end of this chapter, the structure/organization of the whole monograph is also presented.

1.1 Iterative Learning Control In our daily life, almost every task is conducted by trial and error, which is the inherent concept of learning. Indeed, it is learning that helps us to survive from severe conditions in the ancient times and become stronger in handling with various problems nowadays. The underlying philosophy of the human being learning process is “practice makes perfect”. Human being can do things better and better after several practices. For example, when we learn to write a Chinese character in primary school, our teacher always asks us to repeat the character many times. While repeating the character, we are adjusting the writing positions step by step and the writing performance will be improved gradually. Another example is the basketball shooting. Consider to learn to shoot the basketball from a fixed position, it might be difficult for us to hit the basket at the first several trials since we have little knowledge about the correct shooting angle and force. The hit ratio will be definitely increased if we can learn from the failures and correct our behavior. Learning is an important concept to human being, and it is interesting to find that such a fundamental principle can be applied to control systems, which is the origin of iterative learning control (ILC). ILC is designed for systems repeating a certain task over a fixed time interval. In ILC, the current control input signal is generated by previous input and output information as well as the tracking objective. As a result, the control performance can be gradually improved as the iteration number increases. © Springer Nature Singapore Pte Ltd. 2019 D. Shen and X. Li, Iterative Learning Control for Systems with Iteration-Varying Trial Lengths, https://doi.org/10.1007/978-981-13-6136-4_1

1

2 Fig. 1.1 ILC in iteration domain

1 Introduction yk

uk

k th iteration

Plant

uk+1

yd

Controller

(k+1)th iteration

yk+1

uk+1 Plant

uk+2 (k+2)th iteration

Fig. 1.2 Framework of ILC

yd

Controller

Memory uk

yd

Controller

uk+1

yk+1 Plant

—

Note that the term “iteration” may be replaced with “trial”, “batch”, and “cycle” in the literature due to different backgrounds. In contrast to other control methods, such as adaptive control and robust control, ILC has several distinct features, which include (1) finite time horizon, (2) accurate resetting condition, and (3) perfect repeating conditions including the system plant and control objective. The block diagram of ILC is shown in Fig. 1.1, where yd denotes the reference trajectory. At the kth iteration, the input u k is fed to the plant so that the corresponding output is yk and the tracking error is ek = yd − yk . This is a causal process. Since the tracking error ek is nonzero, implying that the input u k is not good enough, the input signal of the next iteration (i.e., the (k + 1)th iteration) should then be updated. The control input signal u k+1 at the (k + 1)th iteration usually is constructed as a function of u k and ek . In other words, the control input signal u k+1 is generated by previous control information. Then, the newly generated input is fed to the plant for the next iteration. Meanwhile, the input u k+1 is also stored into the memory for updating the input for the (k + 2)th iteration. As a result, a closed-loop feedback is formed in the iteration domain. A concise block diagram of the ILC principle is shown in Fig. 1.2, which is common in many ILC papers and monographs. As can be seen from the above figures, the main difference between ILC and the conventional control methodologies is that ILC improves the control performance along the iteration axis rather than the time axis. In other words, the control performance is gradually improved as the iteration number increases to infinity, while the

1.1 Iterative Learning Control

3

transient performance within an iteration is usually ignored. Meanwhile, ILC can be combined with the conventional control methods to further enhance performance. That is, we can employ the conventional control method as an inner control loop to achieve an acceptable performance of the systems such as stability and then add ILC as an outer control loop to further improve the tracking precision. This topic has also been investigated in the literature. As discussed before, ILC synthesizes the current control input signal from previous input and output information, while the exact system plant information is not required. Actually, this is one of the major advantages of ILC. In other words, ILC is a data-driven control method. ILC has been proposed for a long time which can be tracked back to a US patent [1] in 1967. In 1978, Uchiyama initialized a “repeating” method of correcting the reference function by trial [2]. However, this paper was written in Japanese and failed to attract wide attention in the community. The paper published by Arimoto et al. in 1984 opened the research of ILC [3]. Since then, numerous articles were published on ILC. Detailed surveys on ILC can be found in [4–9].

1.2 Basic Formulation of ILC Now, let us go through the basic formulation of ILC for both discrete-time and continuous-time systems, which is followed by the conventional convergence analysis.

1.2.1 Discrete-Time Case Consider the following discrete time-invariant linear system xk (t + 1) = Axk (t) + Bu k (t), yk (t) = C xk (t),

(1.1)

where x ∈ Rn , u ∈ R p , and y ∈ Rq denote the system state, input, and output, respectively. Matrices A, B, and C are system matrices with appropriate dimensions. t denotes an arbitrary time instant in an operation iteration, t = 0, 1, . . . , N , where N is the length of the operation iteration. For simplicity, t ∈ [0, N ] is used in the following. k = 0, 1, 2, . . . denote different iterations. Since it is required that a given tracking task should be repeated, the initial state needs to be reset at each iteration. The following is a basic reset condition, called identical initialization condition (i.i.c.), which is common in ILC theory. xk (0) = x0 , ∀k.

(1.2)

4

1 Introduction

The reference trajectory is denoted by yd (t), t ∈ [0, N ]. With regard to the reset condition, it is usually required that yd (0) = y0  C x0 . The control objective of ILC is to design a proper update law for the input u k (t), so that the corresponding output yk (t) can track yd (t) as closely as possible. To this end, for any t in [0, N ], we define the tracking error as (1.3) ek (t) = yd (t) − yk (t). Then, the update law is a function of u k (t) and ek (t) to generate u k+1 (t), whose general form is as follows: u k+1 (t) = h(u k (·), . . . , u 0 (·), ek (·), . . . , e0 (·)).

(1.4)

When the above relationship depends only on the last iteration, it is called first-order ILC update law; otherwise, it is called high-order ILC update law. Generally, to achieve the algorithm simplicity, most update laws are of first order, i.e., u k+1 (t) = h(u k (·), ek (·)).

(1.5)

Additionally, the update law is usually linear. The simplest update law is as follows: u k+1 (t) = u k (t) + K ek (t + 1),

(1.6)

where K is the learning gain matrix to be designed. In (1.6), u k (t) is the input of current iteration, while K ek (t + 1) is the innovation term. The update law (1.6) is called P-type ILC update law. If the innovation term is replaced by K (ek (t + 1) − ek (t)), the update law is called D-type. For system (1.1) and update law (1.6), a basic convergence condition is that K satisfies I − C B K  < 1. (1.7) Then, one has ek (t) −−−→ 0, where  ·  denotes the matrix or vector norm. k→∞

From this condition, one can deduce that the design of K needs no information with regard to the system matrix A, but requires information of the coupling matrix C B. This fact demonstrates the advantage of ILC from the perspective that ILC has little dependence on the system information A. Thus, ILC can handle tracking problems that have more uncertainties. Remark 1.1 From the formulation of ILC, one can see that the model takes the classic features of a 2D system. That is, the system dynamics (1.1) and the up date law (1.6) evolve along time and iteration axes, respectively. Many scholars have made contributions from this point of view and developed a 2D system-based approach, which is one of the principal techniques for ILC design and analysis.

1.2 Basic Formulation of ILC

5

Note that the operation length is limited by N , and is then repeated multiple times. Thus, one could use the so-called lifting technique for discrete-time systems, which implies lifting all of the inputs and outputs as supervectors, Uk = [u kT (0), u kT (1), . . . , u kT (N − 1)]T , Yk = Denote

⎡

[ykT (1), ykT (2), . . . , ykT (N )]T .

(1.8) (1.9)

⎤ 0 ··· 0 ⎢ 0 ··· 0 ⎥ ⎢ ⎥ G=⎢ .. . . .. ⎥ , ⎣ . . ⎦ . C A N −1 B C A N −2 B · · · · · · C B

(1.10)

Yk = GUk + d,

(1.11)

d = [(C x0 )T , (C Ax0 )T , . . . , (C A N −1 x0 )T ]T .

(1.12)

CB C AB .. .

0 CB .. .

then, we have

where

Similar to (1.8) and (1.9), define Yd = [ydT (1), ydT (2), . . . , ydT (N )]T , E k = (ekT (1), ekT (2), . . . , ekT (N ))T , then it leads to Uk+1 = Uk + KE k ,

(1.13)

where K = diag{K , K , . . . , K }. By simple calculation, we have E k+1 = Yd − Yk+1 = Yd − GUk+1 − d = Yd − GUk − GKE k − d = E k − GKE k = (I − GK)E k . Therefore, we obtain the condition (1.7) that is sufficient to guarantee the convergence of ILC. Actually, the lifting technique does not only help us to obtain the convergence condition but also provides us with an intrinsic understanding of ILC. In the lifted model (1.11), the time-domain evolutionary process within the operating iteration has been integrated into G, whereas the relationship between adjacent iterations is highlighted. That is, the lifted model (1.11) depends on the iteration axis only. Remark 1.2 Note that the focus of ILC is how to improve the tracking performance gradually along the iteration axis, as one can see from the design of the update law

6

1 Introduction

(1.13) and lifted model (1.11). Therefore, it would not cause additional difficulties when the system is extended from the linear time-invariant case to the linear timevarying case. This is because, for any fixed time, the updating process along the iteration axis is a time-invariant case. It is usually assumed that the reference trajectory yd (t) is realizable. That is, there exists an appropriate initial state x0 and input u d (t) such that the expression (1.1) still holds with the subscript k being replaced by d. In other words, Yd = GUd + d, where Ud is defined in a similar manner as (1.8). Then, the discussion that the system output converges to the reference trajectory, i.e., limk→∞ Yk = Yd , is equivalent to the one that the system input converges to the objective input, i.e., limk→∞ Uk = Ud . For the system with stochastic noises, this transformation of proof objective is more convenient for convergence analysis. Remark 1.3 One may be interested in the case that the reference trajectory is not realizable. In other words, there is no control input producing the reference trajectory; thus, entirely accurate tracking is impossible. Then, the design objective of the ILC algorithm is no longer to guarantee asymptotically accurate tracking, but to converge to the nearest trajectory of the given reference. Consequently, the tracking problem has become an optimization problem. On the other hand, from the viewpoint of practical applications, the reference trajectory is usually realizable; thus, the assumption is not rigorous.

1.2.2 Continuous-Time Case Let us consider the following linear continuous-time system: x˙k (t) = Axk (t) + Bu k (t), yk (t) = C xk (t),

(1.14)

where notations have similar meanings to the discrete-time case. The control task is to drive the output yk to track the desired reference yd on a fixed time interval t ∈ [0, T ] as the iteration number k increases. If the relative degree of the system is one, an ILC scheme of Arimoto type can be given as u k+1 = u k + Γ e˙k ,

(1.15)

where ek (t) = yd (t) − yk (t) and Γ is the diagonal learning gain matrix. Similarly, if the learning gain matrix satisfies I − C BΓ  < 1,

(1.16)

then the control objective can be achieved, i.e., limk→∞ yk (t) → yd (t). Note that the basic formula for selecting the learning gain matrix given in (1.16) requires

1.2 Basic Formulation of ILC

7

no information about the system matrix A, which implies that ILC is effective for uncertain system matrices. Moreover, a “PID-like” update law can be formulated as  u k+1 = u k + Φek + Γ e˙k + Ψ

ek dt,

(1.17)

where Φ, Γ , and Ψ are learning gain matrices. The high-order PID-like update law can be formulated as u k+1 =

n

(I − )Pi u k+1−i + u 0 +

i=1

where

n i=1

  n

Φi ek+1−i + Γi e˙k+1−i + Ψi ek+1−i dt ,

(1.18)

i=1

Pi = I .

1.3 ILC for Systems with Varying Trial Lengths In traditional ILC, to achieve perfect tracking performance, some exactly repeating conditions, such as identical trial length, identical initial condition, and iterationinvariant learning target, are required. However, these iteration-invariant conditions will often be violated in real-time applications due to unknown uncertainties or unpredictable factors, which hinders practical applications of conventional ILC, and thus motivate scholars to relax/remove the perfect repeating conditions in ILC. This monograph will focus on ILC design when control systems have iteration-varying trial lengths. In practice, sometimes it is difficult to ensure that the control system repeats on a fixed time interval. For instance, when applying ILC in a functional electrical stimulation (FES) for upper limb movement and gait assistance, it is found that the operation processes end early for at least the first few passes due to safety considerations [10]. The FES-induced foot motion and the associated variable-length-trial problem are detailed in [11, 12], which clearly illustrate the violation of the identicaltrial-length assumption. Another example can be seen in the analysis of humanoid and biped walking robots, which is characterized by periodic or quasi-periodic gaits [13]. For analysis purpose, these gaits are divided into phases that are defined by the time when the foot strikes the ground, and the duration of the resulting phases are usually not the same from iteration to iteration. Furthermore, as can be found in [14], a trajectory-tracking problem of a lab-scale gantry crane was investigated under the framework of ILC. In this example, the trial lengths at different iterations might be varying since authors defined that the learning process should be terminated if the system output drift far away from the desired trajectory. Based on these observations, it is interesting and valuable to investigate ILC with iteration-varying trial lengths. To clarify the effect of varying trial lengths, one can refer to Fig. 1.3, where Fig. 1.3a illustrates the complete trial length with Td being the desired iteration length while Fig. 1.3b–d demonstrates possible incomplete trial lengths. In other words, the varying trial length problem here indicates that the iteration may end before its desired

8

1 Introduction

0

Td

(a) complete trial

(b) incomplete trial

Td

0

Td

0

(c) incomplete trial

Td

0

(d) incomplete trial

Fig. 1.3 Illustration of varying trial lengths

time length but the tracking objective remains the same for all iterations. Therefore, the major influence of this setting is that the latter part of the tracking information is lost if the iteration ends early. As a result, emphasis of analysis should go to the inherent effect of uncompleted operation part. Moreover, in this monograph, we consider the varying length to be random with or without statistical knowledge, and thus the specific convergence sense should be carefully considered. There were some early research aiming to provide a suitable design and analysis framework for varying-iteration-length ILC that formed the groundwork for subsequent investigations [10–14]. For example, based on the experimental verifications and primary analysis of the convergence property given in [10–12], a systematic proof of the monotonic convergence in different norm senses was elaborated in [15] for linear systems with nonuniform trial lengths. In that paper, the necessary and sufficient conditions for monotonic convergence were discussed, as well as other issues including the controller design guidelines and influence of disturbances. However, it is worthwhile to mention that authors did not provide an uniform framework on ILC with iteration-varying trial lengths. The first random model of varying-length iterations was proposed in [16] for discrete-time systems, and it was then extended to continuous-time systems in [17]. In [16] and [17], a stochastic variable was used to represent the occurrence of the output at each time instant and iteration, and it was then multiplied to the tracking error, which denoted the actual information of the updating process. To compensate for the information loss caused by randomly varying trial lengths, an iteration-average operator of all historical data was introduced to the ILC algorithm in [16], whereas in [17], this average operator was replaced by an iteration-moving-average operator

1.3 ILC for Systems with Varying Trial Lengths

9

to reduce the influence of very old data. Moreover, a lifted framework of ILC for a discrete-time linear system was provided in [18] to avoid conservatism of the conventional λ-norm-based analysis in [16, 17]. Note that all of the results in [16–18] obtained asymptotical convergence with respect to the expected value, which is very weak for the control of stochastic models, and thus motivates scholars to seek a stronger convergence result in [19]. In detail, the discrete-time linear system was revisited and the traditional P-type ILC law was employed. The authors transformed the error evolution along the iteration axis by modeling it as a switching system and then established the input error’s statistical properties (i.e., the mathematical expectations and covariances) in a recursive form. The convergence in the mathematical expectation, mean square, and almost sure senses was derived simultaneously. The results were then extended to a class of affine nonlinear systems in [20] using different analysis techniques. A recent work [21] further proposed two novel and improved ILC schemes based on the iterationmoving-average operator, in which a random searching mechanism was additionally introduced to collect useful information while avoiding redundant tracking information from the past. In addition, some extensions have also been reported. Nonlinear stochastic systems were taken into account in [22] with bounded disturbances. Nevertheless, a Gaussian distribution of the variable pass length was required, which limits the possible application range. In [23], the authors extended the method to discrete-time linear systems with a vector relative degree. In this case, one needs to carefully select the output data for the learning algorithms to function. The issue was also extended to stochastic impulse differential equations in [24] and fractional order systems in [25]. We would like to note that the convergence analyses derived in these papers were primarily based on the mature contraction mapping method similar to [16]. A recent progress [26] presented a deterministic convergence analysis if the full-length learning occur any adjacent finite iterations. In short, we can observe the following facts from the above literature. First, most papers have focused on discrete-time linear systems such as those in [15, 16, 18, 19, 21, 23], mainly owing to the beneficial system structure and mature analysis techniques for discrete random variables. The results on continuous-time systems, originated from practical systems, are rather limited. Although nonlinear systems were considered in [17, 20, 22], the globally Lipschitz continuous condition was imposed on the nonlinear functions in these papers, which effectively transforms the system to a linear system. Therefore, it is significant to consider removing the globally Lipschitz continuous condition for continuous-time systems. This advance was presented in a recent paper [27], where continuous-time parameterized nonlinear systems with nonlinear functions being locally Lipschitz continuous were taken into account. Adaptive learning controller consisting of a stabilization feedback term and a compensation feedforward term was proposed. A novel modified composite energy function (CEF) was defined with the new concept of a virtual tracking error for the untrodden part of each iteration. This CEF allowed one to present an explicit difference between adjacent iterations and thus facilitated the analysis. Moreover, if partial structure information is available, the paper [28] presented two

10

1 Introduction

types of learning schemes: a mixing-type adaptive learning scheme and a hybridtype differential-difference learning scheme. The convergence was conducted using similar idea of [27] with novel virtual tracking errors.

1.4 Structure of this Monograph In this monograph, we concentrate on ILC for systems with varying trial lengths. Our primary objective is to provide a systematic framework of the synthesis and analysis of ILC algorithms. To this end, we will clarify the following aspects: the controller design, the convergence analysis, and the influence evaluation of nonuniform trial lengths. The investigation of this monograph would greatly help to understand ILC with varying trial lengths, which is a specific type of incomplete information. The visual structure of this monograph is shown in Fig. 1.4, where three different divisions of the chapters can be observed. First division: The main materials in this monograph are placed into two parts. Part I including Chaps. 2–6 focuses on linear systems, for which the conventional P-type algorithm can behave well. Part II including Chaps. 7–12 aims to provide fruitful results for nonlinear systems, where direct and indirect learning schemes are proposed. Second division: Readers can also refer to the monograph according to discretetime and continuous-time types of system dynamics. In particular, Chaps. 2–6 and 8 present the design and analysis techniques for discrete-time systems and Chaps. 7, 8–12 are the counterpart for continuous-time systems. It should be mentioned that the analysis for discrete time and continuous time are fairly different.

Start

Discrete-time

Part I Linear Syst.

Part II Nonlin. Syst.

Continuous-time End Averaging

Switching

Fig. 1.4 Structure of this monograph

2D

λ-norm

CEF

1.4 Structure of this Monograph

11

Third division: The monograph also provides a technical angle of the results. Specifically, Chaps. 2–4 and 7 deeply discuss the averaging technique in controller design. Chapter 5 gives a novel switching system approach for convergence analysis of the conventional P-type scheme. Chapter 6 presents the two-dimensional Kalmanfiltering-based approach for stochastic systems. Chapters 8 and 9 provide an in-depth application of the λ-norm technique for analysis. Chapters 10–12 comprehensively show the definition and employment of CEF techniques for continuous-time nonlinear systems. The contents of each chapter can be summarized as follows: Chapter 2 presents a novel formulation and idea to address the tracking control problem for discrete-time linear systems with randomly varying trial lengths. An ILC scheme with an iteration-average operator is introduced, which thus mitigates the requirement on classic ILC that all trial lengths must be identical. Chapter 3 also considers a class of discrete-time linear systems with randomly varying trial lengths. However, in contrast to Chap. 2, this chapter aims to avoid using the traditional λ-norm in convergence analysis which may lead to a non-monotonic convergence. Chapter 4 proposes two novel ILC schemes for discrete-time linear systems with randomly varying trial lengths. In contrast to Chaps. 2 and 3 that advocate to replace the missing control information by zero, the proposed learning algorithms in this chapter are equipped with a random searching mechanism to collect useful but avoid redundant past tracking information, which could expedite the learning speed. Chapter 5 proceeds to a novel analysis technique for linear discrete-time systems, which is called the switching system technique. In this technique, the iteration evolution of the input error is formulated as a switching system. Then, the mean and covariance of the associated random matrices can be recursively computed along the iteration axis, which paves a novel way for convergence analysis. Chapter 6 presents the two-dimensional technique for addressing the tracking problem of linear discrete-time stochastic systems with varying trial lengths. The Kalman filtering technique is applied to derive the recursive learning gain matrix which guarantees the mean square convergence of the input error to zero. As a consequence, the tracking error will converge asymptotically in mean square sense. Chapter 7 extends the idea on ILC design with randomly varying trial lengths to nonlinear continuous-time dynamic systems. Different from Chaps. 2 and 3, this chapter will employ an iteratively moving average operator with fixed window length into the ILC scheme. Chapter 8 considers the discrete-time nonlinear systems, which is different from the continuous-time case in the previous chapter. In particular, the affine nonlinear system is taken into account, where the nonlinear functions satisfy globally Lipschitz continuous condition. A novel technical lemma is also provided for the strict convergence analysis in pointwise sense. Chapter 9 provides the first result on sampled-data control for continuous-time nonlinear systems with varying trial lengths. To deal with the iteration-varying length problem, we propose two sampled-data ILC schemes, a generic PD-type scheme and a modified version with moving average operator, based on the modified tracking errors

12

1 Introduction

that have been redefined when the trial length is shorter or longer than the desired one. Sufficient conditions are derived rigorously to guarantee the convergence of the nonlinear system at each sampling instant. Chapter 10 proposes a novel method for parameterized nonlinear continuoustime systems with varying trial lengths. As opposed to the previous chapters, this chapter is applicable to nonlinear systems that do not satisfy the globally Lipschitz continuous condition. To solve the problem, the adaptive ILC schemes are adopted in this chapter to learn the parameters and ensure an asymptotical convergence. Moreover, this chapter introduces a novel CEF using newly defined virtual tracking errors for proving the convergence. Chapter 11 proceeds to consider the continuous-time nonlinear systems with nonparametric uncertainties, differing the parameterized systems in the previous chapter, under nonuniform trial length circumstances. Three common types of nonparametric uncertainties are taken into account in sequence: norm-bounded uncertainty, variation-norm-bounded uncertainty, and norm-bounded uncertainty with unknown coefficients. The CEF defined in the previous chapter is employed for the asymptotical convergence of the proposed schemes. Chapter 12 applies the CEF technique proposed in Chaps. 10 and 11 to uncertain systems with two specific types of partial structure information. First, we consider the case that the system uncertainty consists of two parts, a time-invariant part and a time-varying part. A mixing-type adaptive learning scheme is derived, where the time-invariant part and the time-varying part are learned in differential and difference forms. Next, we move to consider the case that time-invariant and time-varying system uncertainties cannot be directly separated. A hybrid form of the differential and difference learning laws is proposed, where both differential and difference learning mechanisms are integrated in a unified adaptive learning scheme to derive the estimation of unknown parameters.

1.5 Summary In this chapter, the introduction of ILC is provided first, which is followed by the basic formulation of ILC for both discrete-time and continuous-time control systems. In addition, a brief review of ILC with iteration-varying trial lengths is then given. Lastly, the structure of the whole monograph is also presented.

References

13

References 1. Garden M (1967) Learning control of actuators in control systems. US Patent, Appl No 637769 2. Uchiyama M (1978) Formulation of high-speed motion pattern of a mechanical arm by trial. Trans Soc Instrum Control Eng 14(6):706–712 3. Arimoto S, Kawamura S, Miyazaki F (1984) Bettering operation of robots by learning. J Robotic Syst 1(2):123–140 4. Bristow DA, Tharayil M, Alleyne AG (2006) A survey of iterative learning control: a learningbased method for high-performance tracking control. IEEE Control Syst Mag 26(3):96–114 5. Ahn H-S, Chen YQ, Moore KL (2007) Iterative learning control: survey and categorization from 1998 to 2004. IEEE Trans Syst Man Cybern Part C 37(6):1099–1121 6. Wang Y, Gao F, Doyle FJ III (2009) Survey on iterative learning control, repetitive control and run-to-run control. J Process Control 19(10):1589–1600 7. Shen D, Wang Y (2014) Survey on stochastic iterative learning control. J Process Control 24(12):64–77 8. Shen D (2018) Iterative learning control with incomplete information: a survey. IEEE/CAA J Autom Sin 5(5):885–901 9. Shen D (2018) A technical overview of recent progresses on stochastic iterative learning control. Unmanned Syst 6(3):147–164 10. Seel T, Schauer T, Raisch J (2011) Iterative learning control for variable pass length systems. In: Proceedings of the 18th IFAC World congress. Milano, Italy, pp 4880–4885, 28 Aug–2 Sept 2011 11. Seel T, Werner C, Schauer T (2016) The adaptive drop foot stimulator - multivariable learning control of foot pitch and roll motion in paretic gait. Med Eng Phys 38(11):1205–1213 12. Seel T, Werner C, Raisch J, Schauer T (2016) Iterative learning control of a drop foot neuroprosthesis - generating physiological foot motion in paretic gait by automatic feedback control. Control Eng Pract 48:87–97 13. Longman RW, Mombaur KD (2014) Investigating the use of iterative learning control and repetitive control to implement periodic gaits. Lect Notes Control Inf Sci 340:189–218 14. Guth M, Seel T, Raisch J (2013) Iterative learning control with variable pass length applied to trajectory tracking on a crane with output constraints. In: Proceedings of the 52nd IEEE conference on decision and control. Florence, Italy, pp 6676–6681 15. Seel T, Schauer T, Raisch J (2017) Monotonic convergence of iterative learning control systems with variable pass length. Int J Control 90(3):393–406 16. Li X, Xu J-X, Huang D (2014) An iterative learning control approach for linear systems with randomly varying trial lengths. IEEE Trans Autom Control 59(7):1954–1960 17. Li X, Xu J-X, Huang D (2015) Iterative learning control for nonlinear dynamic systems with randomly varying trial lengths. Int J Adapt Control Signal Process 29(11):1341–1353 18. Li X, Xu J-X (2015) Lifted system framework for learning control with different trial lengths. Int J Autom Comput 12(3):273–280 19. Shen D, Zhang W, Wang Y, Chien C-J (2016) On almost sure and mean square convergence of P-type ILC under randomly varying iteration lengths. Automatica 63:359–365 20. Shen D, Zhang W, Xu J-X (2016) Iterative learning control for discrete nonlinear systems with randomly iteration varying lengths. Syst Control Lett 96:81–87 21. Li X, Shen D (2017) Two novel iterative learning control schemes for systems with randomly varying trial lengths. Syst Control Lett 107:9–16 22. Shi J, He X, Zhou D (2016) Iterative learning control for nonlinear stochastic systems with variable pass length. J Frankl Inst 353:4016–4038 23. Wei Y-S, Li X-D (2017) Varying trail lengths-based iterative learning control for linear discretetime systems with vector relative degree. Int J Syst Sci 48(10):2146–2156 24. Liu S, Debbouche A, Wang J (2017) On the iterative learning control for stochastic impulsive differential equations with randomly varying trial lengths. J Comput Appl Math 312:47–57 25. Liu S, Wang J (2017) Fractional order iterative learning control with randomly varying trial lengths. J Frankl Inst 354:967–992

14

1 Introduction

26. Meng D, Zhang J (2018) Deterministic convergence for learning control systems over iterationdependent tracking intervals. IEEE Trans Neural Netw Learn Syst 29(8):3885–3892 27. Shen D, Xu J-X (2018) Adaptive learning control for nonlinear systems with randomly varying iteration lengths. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2018. 2861216 28. Zeng C, Shen D, Wang J (2018) Adaptive learning tracking for uncertain systems with partial structure information and varying trial lengths. J Frankl Inst 355(15):7027–7055

Part I

Linear Systems

Chapter 2

Averaging Techniques for Linear Discrete-Time Systems

This chapter presents a novel formulation and idea to addresses the tracking control problem for discrete-time linear systems with randomly varying trial lengths. An ILC scheme with an iteration-average operator is introduced, which thus mitigates the requirement on classic ILC that all trial lengths must be identical. In addition, the identical initialization condition can be absolutely removed. The learning convergence condition of ILC in mathematical expectation is derived through rigorous analysis. As a result, the proposed ILC scheme is applicable to more practical systems. In the end, two illustrative examples are presented to demonstrate the performance and the effectiveness of the averaging ILC scheme for both time-invariant and time-varying linear systems.

2.1 Problem Formulation First of all, some notations are presented. Throughout this chapter, denote  ·  the Euclidean norm or any consistent norm, and  f (t)λ = supt∈{0,1,...,N } α −λt  f (t) the λ-norm of a vector function f (t) with λ > 0 and α > 1. Denote N as the set of natural numbers and I as the identity matrix. Moreover, define Id  {0, 1, . . . , Nd }, where Nd is the desired trial length and Ik  {0, 1, . . . , Nk }, where Nk is the trial length of the kth iteration. When Nk < Nd , it follows that Ik ⊂ Id . Define Id /Ik  / Ik } as the complementary set of Ik in Id . Given two integers M1 and M2 {t ∈ Id : t ∈ satisfying 0 ≤ M1 < Nd and M2 ≥ 0, respectively. Set I N  {0, 1, . . . , Nd + M2 } and it may be divided into two subsets, Ia  {0, 1, . . . , Nd − M1 − 1} and Ib  {Nd − M1 , . . . , Nd + M2 }. On the set Ia , the control system is deterministic, whereas on the set Ib the system trial length is randomly varying. Denote τm  Nd − M1 + m, m ∈ {0, 1, . . . , M1 + M2 }, which implies τm ∈ Ib .

© Springer Nature Singapore Pte Ltd. 2019 D. Shen and X. Li, Iterative Learning Control for Systems with Iteration-Varying Trial Lengths, https://doi.org/10.1007/978-981-13-6136-4_2

17

18

2 Averaging Techniques for Linear Discrete-Time Systems

Fig. 2.1 Randomly varying trial lengths

Consider a class of linear time-invariant systems xk (t + 1) = Axk (t) + Bu k (t), yk (t) = C xk (t),

(2.1)

where k ∈ N and t ∈ Ik denote the iteration index and discrete time, respectively. Meanwhile, xk (t) ∈ Rn , u k (t) ∈ R p , and yk (t) ∈ Rr denote state, input, and output of the system (2.1), respectively. Further, A, B, and C are constant matrices with appropriate dimensions, and C B is full rank. Let yd (t), t ∈ Id be the desired output trajectory. Assume that, for any realizable output trajectory yd (t), there exists a unique control input u d (t) ∈ R p such that xd (t + 1) = Axd (t) + Bu d (t), yd (t) = C xd (t),

(2.2)

where u d (t) is uniformly bounded for all t ∈ Id . The main difficulty in designing ILC scheme for the system (2.1) is that the actual trial length Nk is iteration varying and different from the desired trial length Nd . Here, a simple example is illustrated in Fig. 2.1 to show the variation of the trial lengths in the iteration domain. Assume that the desired trial length is 10, namely, Nd = 10, and M1 = 3, M2 = 2. Clearly, we have Id = {0, 1, . . . , 10}, I N = {0, 1, . . . , 12}, Ia = {0, 1, . . . , 6} and Ib = {7, 8, . . . , 12}. The span of curve k, k ∈ {1, 2, . . . , 5} in Fig. 2.1 represents the trial length of the control process at the kth iteration, and the dashed line stands for the possible values, {Nd − M1 , . . . , Nd + M2 }, of the stochastic variable Nk . As can be seen from Fig. 2.1, N1 = 7, N2 = 11, N3 = 9, N4 = 10, and N5 = 12. For k = 1, 3, we observe Nk < Nd , namely, Ik ⊂ Id . It is easy to verify that Id /I1 = {8, 9, 10} and Id /I3 = {10}. Figure 2.1 shows that the trial lengths randomly vary between 7 and 12 and they are likely different from the desired trial length. Before addressing the ILC design problem with nonuniform trial lengths, let us give some notations and assumptions that would be useful in the derivation of our main result.

2.1 Problem Formulation

19

Definition 2.1 E{ f } represents for the expectation of the stochastic variable f . P[ f ] means the occurrence probability of the event f . Assumption 2.1 Assume that Nk ∈ Ib is a stochastic variable with P[Nk = τm ] = pm , m ∈ {0, 1, . . . , M1 + M2 }, where τm = Nd − M1 + m, and 0 ≤ pm < 1 is a known constant. Assumption 2.2 E{xk (0)} = xd (0). Remark 2.1 The contraction-mapping-based ILC usually requires the identical initial condition in each iteration. In Assumption 2.2, the condition is extended clearly. The initial states of system could change randomly with E{xk (0)} = xd (0) and there are no limitations to the variance of xk (0). If the control process (2.1) repeats with the same trial length Nd , namely, Nk = Nd , and under the identical initial condition, a simple and effective ILC [1] for the linear system (2.1) is u k+1 (t) = u k (t) + Lek (t + 1),

(2.3)

where ek (t + 1)  yd (t + 1) − yk (t + 1) and L ∈ R p×r is an appropriate learning gain matrix. However, when the trial length Nk is iteration varying, which corresponds to a non-standard ILC process, the learning control scheme (2.3) has to be redesigned.

2.2 ILC Design and Convergence Analysis In this section, based on the assumptions and notations that are given in Sect. 2.1, ILC design and convergence analysis are addressed, respectively. In practice, for one scenario that the kth trial ends before the desired trial length, namely, Nk < Nd , both the output yk (t) and the tracking error ek (t) on the time interval Id /Ik are missing, which thus cannot be used for learning. For the other scenario that the kth trial is still running after the time instant at which we want it to stop, i.e., Nk > Nd , the signals yk (t) and ek (t) after the time instant Nd are redundant and useless for learning. In order to cope with those missing signals or redundant signals in different scenarios, a sequence of stochastic variables satisfying Bernoulli distribution is defined. Using those stochastic variables, a newly defined tracking error ek∗ (t) is introduced to facilitate the modified ILC design. The main procedure for deriving a modified ILC scheme can be described as follows: (1) Define a stochastic variable γk (t) in the kth iteration. Let γk (t), t ∈ I N be a stochastic variable satisfying Bernoulli distribution and taking binary values 0 and 1. On the one hand, the relationship γk (t) = 1 represents the event that the control process (2.1) can continue to the time instant t in the kth iteration, which occurs with a probability of p(t), where 0 < p(t) ≤ 1 is a prespecified function of time t. On the other hand, the relationship γk (t) = 0 denotes the

20

2 Averaging Techniques for Linear Discrete-Time Systems

event that the control process (2.1) cannot continue to the time instant t in the kth iteration, which occurs with a probability of 1 − p(t). (2) Compute the probability P[γk (t) = 1]. Since the control process (2.1) will not stop within the time interval Ia , the event that γk (t) = 1 surely occurs when t ∈ Ia , which implies that p(t) = 1, ∀t ∈ Ia . While for the scenario of t ∈ Ib , denote Am the event that the control process (2.1) stops at τm , where τm = Nd − M1 + m, m ∈ {0, 1, . . . , M1 + M2 }. Then, it follows from Assumption 2.1 that P[Am ] = pm and the events Am , m ∈ {0, 1, . . . , M1 + M2 } are mutually exclusive clearly. For t ∈ Ib , the event γk (t) = 1 corresponds to the statement that the control process (2.1) stops at or after the time instant t. Thus, ⎡

M 1 +M2

P[γk (t) = 1] = P ⎣

⎤ Am ⎦

m=t−Nd +M1 M 1 +M2

=

P[Am ]

m=t−Nd +M1 M 1 +M2

=

pm .

(2.4)

t ∈ Ia , pm , t ∈ Ib .

(2.5)

m=t−Nd +M1

Therefore, it follows that  p(t) =

1, M1 +M2

m=t−Nd +M1

Further, there has been that 0 < p(t) ≤ 1. In order to demonstrate the calculation of the probability P[γk (t) = 1] more clearly, a simple example is illustrated in Fig. 2.2. In addition, since γk (t) satisfies Bernoulli distribution, the expectation E{γk (t)} = 1 · p(t) + 0 · (1 − p(t)) = p(t). (3) Define a modified tracking error. Denote ek∗ (t)  γk (t)ek (t), t ∈ Id

(2.6)

as a modified tracking error, which renders to ek∗ (t) =



ek (t), t ∈ Ik , 0, t ∈ Id /Ik ,

(2.7)

when Nk < Nd , and ek∗ (t) = ek (t), t ∈ Id , when Nk ≥ Nd .

(2.8)

2.2 ILC Design and Convergence Analysis

21

Fig. 2.2 Similarly as Fig. 2.1, set Nd = 10, M1 = 3 and M2 = 2, the stochastic variable Nk has six possible values τm = 7 + m, m ∈ {0, 1, . . . , 5}. All of the possible outcomes are shown in the table and the probability of the event γk (t) = 1 is related to the number of the character 1 in the corresponding column. It is easy to verify the formulation (2.4). For instance, when t = 9, there are



four 1s in its corresponding column. Then, P[γk (9) = 1] = P[A2 · · · A5 ] = 5m=2 P[Am ] = 5 there are only two 1s in its corresponding column. Thus, it m=2 pm . Similarly, when t = 11,

follows that P[γk (11) = 1] = P[A4 A5 ] = 5m=4 P[Am ] = 5m=4 pm

Remark 2.2 Since the absent signals are unavailable, and the redundant signals are useless for learning, it is reasonable to define a modified tracking error ek∗ (t) as in (2.6), or equivalently (2.7) and (2.8). In the modified tracking error ek∗ (t), the redundant signals in ek (t) are cut off when Nk > Nd , and the unavailable signals in ek (t) are set as zero, when Nk < Nd . (4) The modified ILC scheme. Introduce an iteration-average operator [2], 1  f j (·), k + 1 j=0 k

A{ f k (·)} 

(2.9)

for a sequence f 0 (·), f 1 (·), . . . , f k (·), which plays a pivotal role in the proposed controller. The modified ILC scheme is given as follows:

22

2 Averaging Techniques for Linear Discrete-Time Systems

k+2  ∗ e (t + 1), t ∈ Id , L k + 1 j=0 j k

u k+1 (t) = A{u k (t)} +

(2.10)

for all k ∈ N, where the learning gain matrix L will be determined in the following. Remark 2.3 As a matter of fact, the second term on the right-hand side of (2.10) can be rewritten as (k + 2)LA{ek∗ (t + 1)}. In A{ek∗ (t + 1)}, the error profiles e∗j (t + 1), j = 0, 1, 2, . . . , k, have been reduced by (k + 1) times. Nevertheless, by multiplying the factor (k + 2) in the feedback loop, their magnitudes can be retained even when k → ∞. The following theorem presents the first main result of this chapter. Theorem 2.1 For the discrete-time linear system (2.1) and the ILC scheme (2.10), choose the learning gain matrix L such that for any constant 0 ≤ ρ < 1, sup I − p(t)LC B ≤ ρ,

(2.11)

t∈Id

then the expectation of the error, E{ek (t)}, t ∈ Id , will converge to zero asymptotically as k → ∞. Remark 2.4 In practice, the probability distribution of the trial length Nk could be estimated in advance based on previous multiple experiments or by experience. In consequence, the probability pm in Assumption 2.1 is known. Finally, p(t) can be calculated by (2.5), thus is available for controller design. Remark 2.5 From the convergence condition (2.11) and Remark 2.4, it can be found that the only system knowledge needed for ILC is the system gradient information C B. In Chap. 7, it will be discussed that the accurate mathematical expression of p(t) is not actually required, and we only need its upper and lower bounds when designing ILC law. Proof The proof consists of two parts. Part I proves the convergence of the input error in iteration average and expectation using the λ-norm. Part II proves the convergence of the tracking error in expectation. Part I. Let Δu k (t)  u d (t) − u k (t) and Δxk (t)  xd (t) − xk (t) be the input and state errors, respectively, then Δxk (t + 1) = AΔxk (t) + BΔu k (t), ek (t) = CΔxk (t).

(2.12)

By the definition of iteration-average operator (2.9), A{Δu k+1 (t)} can be rewritten as A{Δu k+1 (t)} =

1 [Δu k+1 (t) + (k + 1)A{Δu k (t)}]. k+2

(2.13)

2.2 ILC Design and Convergence Analysis

23

In addition, subtracting u d (t) from both sides of the ILC law (2.10) implies k+2  ∗ Δu k+1 (t) = A{Δu k (t)} − L e (t + 1). k + 1 j=0 k k

(2.14)

Then, substituting (2.14) into the right-hand side of (2.13) and applying the operator E{·} on both sides of (2.13) yield E{A{Δu k+1 (t)}} = E{A{Δu k (t)}} − LE{A{ek∗ (t + 1)}}.

(2.15)

Since both E{·} and A{·} are linear operators, the operation orders of E{·} and A{·} can be exchanged, yielding E{A{ek∗ (t + 1)}} = p(t + 1)E{A{ek (t + 1)}},

(2.16)

where E{γ j (t + 1)e j (t + 1)} = p(t + 1)E{e j (t + 1)} is applied as γ j (t + 1) and e j (t + 1) are independent with each other. Meanwhile, from (2.12), it follows that ek (t + 1) = C AΔxk (t) + C BΔu k (t).

(2.17)

Then, combining (2.16) and (2.17) gives E{A{ek∗ (t + 1)}} = p(t + 1)C AE{A{Δxk (t)}} + p(t + 1)C BE{A{Δu k (t)}}.

(2.18)

In consequence, substituting (2.18) into (2.15) yields E{A{Δu k+1 (t)}} = [I − p(t + 1)LC B]E{A{Δu k (t)}} − p(t + 1)LC AE{A{Δxk (t)}}.

(2.19)

Further, since the solution of the reference system (2.2) is xd (t) = A xd (0) + t

t−1 

At−i−1 Bu d (k),

(2.20)

i=0

it can be obtained similarly from (2.12) that Δxk (t) = At (xd (0) − xk (0)) +

t−1 

At−1−i BΔu k (k).

(2.21)

i=0

Applying both operators E{·} and A{·} on both sides of (2.21) and noticing Assumption 2.2, it concludes that

24

2 Averaging Techniques for Linear Discrete-Time Systems

E{A{Δxk (t)}} =

t−1 

At−1−i BE{A{Δu k (k)}}.

(2.22)

i=0

Then, substituting (2.22) into (2.19) and taking the norm  ·  on both sides lead to E{A{Δu k+1 (t)}} ≤ I − p(t + 1)LC BE{A{Δu k (t)}} t−1  +β α t−i E{A{Δu k (i)}},

(2.23)

i=0

where the parameter α satisfies α ≥ A and β  supt∈Id  p(t + 1)LCB. Multiplying both sides of (2.23) by α −λt , and taking the supremum over Id , we have sup α −λt E{A{Δu k+1 (t)}} ≤ ρ sup α −λt E{A{Δu k (t)}} t∈Id

(2.24)

t∈Id

+ β sup α

−λt

t−1 

t∈Id

α t−i E{A{Δu k (i)}},

i=0

where the constant ρ is chosen such that (2.11) holds. From the definition of λ-norm, it follows that sup α

−λt

t−1 

t∈Id

α t−i E{A{Δu k (i)}}

i=0

= sup α −(λ−1)t t∈Id

t−1 

α −λk E{A{Δu k (i)}}α (λ−1)i

i=0

≤ E{A{Δu k (t)}}λ sup α −(λ−1)t t∈Id

≤

−(λ−1)Nd

1−α α λ−1 − 1

t−1 

α (λ−1)i

i=0

E{A{Δu k (t)}}λ .

(2.25)

Then, combining (2.24) and (2.25), we finally have E{A{Δu k+1 (t)}}λ ≤ ρ0 E{A{Δu k (t)}}λ ,

(2.26)

−(λ−1)Nd

. Since 0 ≤ ρ < 1 by the condition (2.11), it is possible where ρ0  ρ + β 1−α α λ−1 −1 to choose a sufficiently large λ such that ρ0 < 1. Therefore, (2.26) implies that lim E{A{Δu k (t)}}λ = 0.

k→∞

(2.27)

Part II: Now prove the convergence of ek (t) in expectation. Multiplying both sides of (2.26) by (k + 2), it follows that

2.2 ILC Design and Convergence Analysis

25

⎫ ⎫  ⎧  ⎧  ⎨ k+1  ⎨ k ⎬ ⎬        E  Δu j (t)  ≤ ρ0 E Δu j (t)   + ρ0 E {A{Δu k (t)}}λ .(2.28)  ⎩ ⎩ ⎭ ⎭     j=0 j=0 λ

λ

According to the boundedness of E{A{Δu k (t)}}λ from (2.26), (2.27) and Lemma 1 in [2], limk→∞ E{ kj=0 Δu j (t)}λ = 0 is further derived, thus ⎫ ⎧ ⎫⎤ ⎡ ⎧ k k−1 ⎨ ⎬ ⎨ ⎬ lim E{Δu k (t)} = lim ⎣E Δu j (t) − E Δu j (t) ⎦ = 0. (2.29) k→∞ k→∞ ⎩ ⎭ ⎩ ⎭ j=0

j=0

Pre-multiplying the matrix C on both sides of (2.21) and taking the operator E{·} on both sides yield E{ek (t)} =

t−1 

C At−1−i BE{Δu k (i)},

(2.30)

i=0

where Assumption 2.2 is applied. Finally, since (2.29) holds for any t ∈ Id , it is proved that limk→∞ E{ek (t)} = 0, t ∈ Id . The proof is thus completed.

2.3 Extension to Time-Varying Systems In this section, the proposed ILC scheme is extended to time-varying systems xk (t + 1) = At xk (t) + Bt u k (t), yk (t) = Ct xk (t),

(2.31)

where At , Bt , and Ct are time-varying matrices with appropriate dimensions and Ct Bt is full rank. The result is summarized in the following theorem. Theorem 2.2 For the discrete-time linear time-varying system (2.31) and the ILC algorithm (2.10), choose the learning gain matrix L such that for any constant 0 ≤ ρ < 1, sup I − p(t)L(t)Ct Bt  ≤ ρ,

(2.32)

t∈Id

the expectation of the error, E{ek (t)}, t ∈ Id , will converge to zero asymptotically as k → ∞. Proof The proof can be performed similarly as in the proof of Theorem 2.1. Considering the desired dynamics that corresponds to (2.31), namely, (2.2) with the matrices A, B, andC replaced by At , Bt , and Ct , respectively, we have

26

2 Averaging Techniques for Linear Discrete-Time Systems

xd (t) =

 t−1 

 Ai

xd (0) +

i=0

t−1 

t−i−2 

i=0

 At−1−l

Bi u d (i).

(2.33)

l=0

Since a similar relationship also holds at the kth iteration, it follows that Δxk (t) =

t−1 

 Ai (xd (0) − xk (0)) +

i=0

t−1 

t−i−2 

i=0

 At−1−l

Bi )Δu k (i).

(2.34)

l=0

Now, replacing (2.20) and (2.21) in the proof of Theorem 2.1 with (2.33) and (2.34), respectively, we can obtain that the inequality (2.26) holds, where the parameter α satisfies α ≥ supt∈Id At  and β  supt∈Id  p(t + 1)L(t)Ct  · supt∈Id Bt . By choosing a sufficient large λ and noticing the condition (2.32), it follows that ρ0 < 1. Hence, limk→∞ E{A{Δu k (t)}}λ = 0 can be obtained similarly. Following the second part of the proof of Theorem 2.1, it gives that limk→∞ E{ek (t)} = 0, t ∈ Id . This completes the proof. Remark 2.6 In Theorems 2.1 and 2.2, the identical initialization condition is replaced by E{xk (0)} = xd (0). According to (2.21), it has ek (t) = C At (xd (0) − xk (0)) +

t−1 

C At−1−i BΔu k (i).

i=0

So, other than deriving the convergence of tracking error, its expectation converges asymptotically which is proved using the expectation operator and the proposed iteration-average-based ILC scheme. Remark 2.7 The proposed ILC law (2.10) can be extended to the following mth (m ≥ 2) order ILC scheme, u k+1 (t) =

m 

α j u i− j+1 (t) +

j=1

m 

∗ β j ei− j+1 (t + 1), t ∈ Id ,

(2.35)

j=1

where α j and β j are design parameters. Similarly, as the proofs of Theorems 2.1 and 2.2, the convergence of the expectation of tracking error, E{ek (t)}, can be derived by mapping method, and the learning convergence condi the contraction tions are mj=1 α j = 1 and mj=1 γ j < 1, where γ j  supt∈Id α j · I − β j p(t)C B. In (2.35), only the tracking information of the last m trials are adopted.

2.4 Illustrative Simulations In order to show the effectiveness of the proposed ILC scheme, two examples are considered.

2.4 Illustrative Simulations

27

2 1.5 1

Reference

0.5 0 −0.5 −1 −1.5 −2 0

10

20

30

40

50

Time axis

Fig. 2.3 The reference yd with desired trial length Nd = 50

Example 2.1 (Time-invariant system) Consider the following discrete-time linear time-invariant system ⎛

⎞ ⎛ ⎞ 0.50 0 1.00 0 xk (t + 1) = ⎝ 0.15 0.30 0 ⎠ xk (t) + ⎝ 0 ⎠ u k (t), −0.75 0.25 −0.25 1.00   yk (t) = 0 0 1.00 xk (t),

(2.36)

where xk (0) = [0, 0, 0]T , k ∈ N. Let the desired trajectory be yd (t) = sin(2π t/50) + sin(2π t/5) + sin(50π t), t ∈ Id  {0, 1, . . . , 50}, as shown in Fig. 2.3, and thus, Nd = 50. Without loss of generality, set u 0 (t) = 0, t ∈ Id in the first iteration. Moreover, assume that M1 = M2 = 5 and that Nk is a stochastic variable satisfying discrete uniform distribution. Then, Nk ∈ {45, 46, . . . , 55} and P[Nk = τm ] = 1/11, where τm = 45 + m, m ∈ {0, 1, . . . , 10}. Further, the learning gain is set as L = 0.5, which renders to supt∈Id I − p(t)LC B ≈ 0.7273 < 1. The performance of the maximal tracking error, ek s  supt∈Id ek , is presented in Fig. 2.4. It shows that the maximal tracking error ek s decreases from 1.801 to 0.0098 within 42 iterations. Moreover, Fig. 2.5 gives the tracking error profiles for 10, 20, 40, and 80th iterations, respectively. The ends of these trials are marked with the dots A, B, C, and D, respectively. To demonstrate the effects of M1 and M2 on the convergence speed of the tracking error, the learning gain is fixed as L = 0.5, and it is assumed M1 = M2 = 30. Here, Nk ∈ {20, 21, . . . , 80} and P[Nk = τm ] = 1/61, where τm = 20 + m, m ∈

2 Averaging Techniques for Linear Discrete-Time Systems

Maximal tracking error ||e ||

k s

28 10

2

10

0

10

−2

10

−4

10

−6

10

−8

0

20

40

60

80

100

Iteration axis

Fig. 2.4 Maximal tracking error profile of ILC with nonuniform trial length: M1 = M2 = 5 1.5

e10

e20

e80

A

1

k

Tracking error e (t)

e40

0.5

C D

0

B −0.5

−1

0

10

20

30

40

50

60

Time axis

Fig. 2.5 Tracking error profiles of ILC with nonuniform trial length: M1 = M2 = 5

2.4 Illustrative Simulations 1

10

0

Maximal tracking error ||e ||

k s

10

10

−1

10

−2

10

−3

10

−4

10

−5

10

−6

0

29

20

40

60

80

100

Iteration axis

Fig. 2.6 Maximal tracking error profile of ILC with nonuniform trial length: M1 = M2 = 30 0.2 0.15 0.1 A B

E{ek(t)}

0.05

C

0

D

−0.05 −0.1 −0.15 −0.2

E{e10} 0

10

20

E{e20}

30

40

E{e40} 50

E{e80} 60

Time axis

Fig. 2.7 The expectation of tracking errors when the proposed ILC scheme is applied in (2.36)

{0, 1, . . . , 60}, then it follows that supt∈Id I − p(t)LC B ≈ 0.7417 < 1. It can be seen from Fig. 2.6 that more than 60 iterations are needed to decrease the ek s from 1.801 to 0.0097. The convergence speed is obviously slower than the case M1 = M2 = 5. To show the effectiveness of the proposed ILC scheme with randomly varying initial states, it is assumed that the learning gain L = 0.5 and M1 = M2 = 5. Assume

30

2 Averaging Techniques for Linear Discrete-Time Systems 1.5

e100

e200

e300

e500

Tracking errors

1

0.5

0

−0.5

−1

0

10

20

30

40

50

Time axis

Fig. 2.8 Tracking error profiles when the proposed ILC scheme is applied in (2.36)

xk (0) is a stochastic variable with probability P[xk (0) = v1 ] = 1/3, P[xk (0) = v2 ] = 1/3 and P[xk (0) = v3 ] = 1/3, where v1 = [0, 0, −1]T , v2 = [0, 0, 0]T , v3 = [0, 0, 1]T . Figure 2.7 shows that the expectation of the tracking error E{ek (t)} will converge to zero within 80 iterations. The tracking error profiles of the proposed ILC scheme and the ILC scheme in [3] are illustrated in Figs. 2.8 and 2.9, respectively. It is obvious that the performance of the proposed ILC scheme is superior to that of the ILC scheme in [3] under the situation of randomly varying initial states. Similarly, in [4, 5], the identical initialization condition is also indispensable. Example 2.2 (Time-varying system) In order to show effectiveness of our proposed ILC algorithm for time-varying systems, the following discrete-time linear timevarying system is considered ⎛

⎛ ⎞ ⎞ 0.2e−t/100 −0.6 0 1.3 0 0.5 sin(t) ⎠ xk (t) + ⎝ 0.5 ⎠ u k (t), xk (t + 1) = ⎝ 0.6 0 0 0.7   yk (t) = −0.5 1.5 0 xk (t),

(2.37)

where xk (0) = [0, 0, 0]T , k ∈ N. Similarly as Example 2.1, let the desired trajectory be yd (t) = sin(2π t/50) + sin(2π t/5) + sin(50π t), t ∈ Id = {0, 1, . . . , 50}. Set u 0 (t) = 0, t ∈ Id in the first iteration. Assume that M1 = M2 = 5 and Nk satism m p (1 − p)11−m and p = 0.5, fies the binomial distribution with P[Nk = τm ] = C11 where τm = 45 + m, m ∈ {0, 1, . . . , 10}. Set the learning gain as L = 2, then it follows that supt∈Id I − p(t)LC B = 1 − 0.5 · 0.2 = 0.9 < 1. The performance

2.4 Illustrative Simulations

31

1.5

e100

e200

e300

e500

Tracking errors

1

0.5

0

−0.5

−1

0

10

20

30

40

50

Time axis

Fig. 2.9 Tracking error profiles when the ILC scheme in [3] is applied in (2.36) 2

10

k

Maximal tracking error ||e ||s

0

10

−2

10

−4

10

−6

10

−8

10

0

20

40

60

80

100

Iteration axis

Fig. 2.10 Maximal tracking error profile of ILC with nonuniform trial length: M1 = M2 = 5

of the maximal tracking error ek s is presented in Fig. 2.10, where ek s decreases from 1.553 to 0.0058 within 80 iterations. Remark 2.8 When xk (0) is a stochastic variable, the tracking error ek (t) is also a stochastic variable, and satisfies the same probability distribution with xk (0). If xk (0)

32

2 Averaging Techniques for Linear Discrete-Time Systems

is fixed, E{ek (t)} = ek (t), and plotting ||ek ||s would be a rational and clear way to demonstrate the efficacy of the proposed ILC scheme.

2.5 Summary This chapter presents the ILC design and analysis results for discrete-time linear timeinvariant or time-varying systems with nonuniform trial lengths. Due to the variation of the trial lengths, a modified ILC scheme is developed by applying an iterationaverage operator. The learning condition of ILC that guarantees the convergence of tracking error in expectation is derived through rigorous analysis. The proposed ILC scheme mitigates the requirement on classic ILC that each trial must end in a fixed time of duration. In addition, the identical initialization condition might be removed. Therefore, the proposed ILC scheme is applicable to more repetitive control processes. The formulation of ILC with nonuniform trial lengths is novel and could be extended to other control problems that are perturbed by random factors, for instance, control systems with random factors in communication channels. The results in this chapter are mainly based on [6].

References 1. Bien Z, Xu J-X (1998) Iterative learning control: analysis, design, integration and applications. Kluwer, Boston 2. Park K-H (2005) An average operator-based PD-type iterative learning control for variable initial state error. IEEE Trans Autom Control 50(6):865–869 3. Seel T, Schauer T, Raisch J (2011) Iterative learning control for variable pass length systems. In: Proceedings of 18th IFAC world congress, pp 4880–4885 4. Longman RW, Mombaur KD (2006) Investigating the use of iterative learning control and repetitive control to implement periodic gaits. Lect Notes Control Inf Sci 340:189–218 5. Moore KL (2000) A non-standard iterative learning control approach to tracking periodic signals in discrete-time non-linear systems. Int J Control 73(10):955–967 6. Li X, Xu J-X, Huang D (2014) An iterative learning control approach for linear systems with randomly varying trial lengths. IEEE Trans Autom Control 59(7):1954–1960

Chapter 3

Averaging and Lifting Techniques for Linear Discrete-Time Systems

Similar to Chap. 2, this chapter also considers a class of discrete-time linear systems with randomly varying trial lengths. However, in contrast to Chap. 2, this chapter aims to avoid using the traditional λ-norm in convergence analysis which may lead to a non-monotonic convergence. Compared to Chap. 2, the main contributions of the chapter can be summarized as follows: (i) A new formulation is presented for ILC of discrete-time systems with randomly varying trial lengths by defining a stochastic matrix. Comparing with Chap. 2, the introduction of the stochastic matrix is more straightforward, and the calculation of its probability distribution is less complex. (ii) Different from Chap. 2, we investigate ILC for systems with nonuniform trial lengths under the framework of lifted system and the utilization of λ-norm is avoided.

3.1 Problem Formulation First of all, notations are defined the same as in Chap. 2. In particular, denote N as the set of natural numbers and I as the identity matrix. Moreover, define Id  {0, 1, . . . , Nd }, where Nd is the desired trial length, and Ik  {0, 1, . . . , Nk }, where Nk is the trial length of the kth iteration. When Nk < Nd , it follows that / Ik } as the complementary set of Ik in Id . Ik ⊂ Id . Define Id /Ik  {t ∈ Id : t ∈ Given two integers M1 and M2 satisfying 0 ≤ M1 < Nd and M2 ≥ 0, respectively. Set I N  {0, 1, . . . , . . . , Nd + M2 } and it may be divided into two subsets, Ia  {0, 1, . . . , Nd − M1 − 1} and Ib  {Nd − M1 , . . . , Nd + M2 }. On the set Ia , the control system is deterministic, whereas on the set Ib the system trial length is randomly varying. Denote τm  Nd − M1 + m, m ∈ {0, 1, . . . , M1 + M2 }, which implies τm ∈ Ib .

© Springer Nature Singapore Pte Ltd. 2019 D. Shen and X. Li, Iterative Learning Control for Systems with Iteration-Varying Trial Lengths, https://doi.org/10.1007/978-981-13-6136-4_3

33

34

3 Averaging and Lifting Techniques for Linear Discrete-Time Systems

Consider a class of linear discrete-time systems xk (t + 1) = Axk (t) + Bu k (t), yk (t) = C xk (t),

(3.1)

where t ∈ {0, 1, 2, . . . , Nk } denotes discrete time. Meanwhile, xk (t) ∈ Rn , u k (t) ∈ R, and yk (t) ∈ R denote state, input, and output of the system (3.1), respectively. Further, A, B, and C are constant matrices with appropriate dimensions, and C B = 0. This state-space system is equivalent to yk (t) = C(q I − A)−1 B u k (t) + C At xk (0), k ∈ Ik ,      

(3.2)

dk (t)

P(q)

where q is the forward time-shift operator q x(t) = x(t + 1). This system can be written equivalently as the Nk × Nk -dimensional lifted system ⎡ ⎢ ⎢ ⎢ ⎣

yk (1) yk (2) .. .

⎤

⎡

⎥ ⎥ ⎥= ⎦

⎢ ⎢ ⎢ ⎣

yk (Nk )

p1 p2 .. .

··· ··· .. .

0 p1 .. .

0 0 .. .

⎤ ⎥ ⎥ ⎥ ⎦

p Nk p Nk −1 · · · p1 ⎡ ⎤ ⎡ ⎤ dk (1) u k (0) ⎢ u k (1) ⎥ ⎢ dk (2) ⎥ ⎢ ⎥ ⎢ ⎥ ×⎢ ⎥ + ⎢ .. ⎥ , .. ⎣ ⎦ ⎣ . ⎦ . u k (Nk − 1) dk (Nk )

(3.3)

where pt = C At−1 B, t ∈ Ik are Markov parameters. Let yd (t), t ∈ {0, 1, 2, . . . , Nd } be the desired output trajectory. Assume that, for any realizable output trajectory yd (t), there exists a unique control input u d (t) such that xd (t + 1) = Axd (t) + Bu d (t), (3.4) yd (t) = C xd (t). In addition, system (3.4) can be rewritten as follows: ⎡ ⎢ ⎢ ⎢ ⎣ 

yd (1) yd (2) .. .

⎤

⎥ ⎢ ⎥ ⎢ ⎥=⎢ ⎦ ⎣

yd (Nd )   yd

⎡



p1 p2 .. .

0 p1 .. .

··· ··· .. .

0 0 .. .

⎤ ⎥ ⎥ ⎥ ⎦

p Nd p Nd −1 · · · p1   P

3.1 Problem Formulation

35

⎡ ⎢ ⎢ ×⎢ ⎣ 

⎤

u d (0) u d (1) .. .

⎡

⎥ ⎢ ⎥ ⎢ ⎥+⎢ ⎦ ⎣

u d (Nd − 1)  



dd (1) dd (2) .. .

⎤ ⎥ ⎥ ⎥, ⎦

(3.5)

dd (Nd )  

Ud

dd

where dd (t) = C At xd (0), t ∈ Id . The main difficulty in designing ILC scheme for the system (3.1) is that the actual trial length Nk is iteration varying and different from the desired trial length Nd . Before addressing the ILC design problem with nonuniform trial lengths, let us give some notations and assumptions that would be useful in the derivation of our main result. Definition 3.1 E{ f } stands for the mathematical expectation of the stochastic variable f . P[ f ] means the occurrence probability of the event f . Assumption 3.1 Assume that Nk ∈ Ib is a stochastic variable with P[Nk = τm ] = qm , τm ∈ Ib , where 0 ≤ qm < 1 is a known constant. Assumption 3.2 E{xk (0)} = xd (0). Remark 3.1 The contraction-mapping-based ILC usually requires the identical initial condition in each iteration. In Assumption 3.2, the condition is extended clearly. The initial states of system could change randomly with E{xk (0)} = xd (0) and there are no limitations to the variance of xk (0). If the control process (3.1) repeats with the same trial length Nd , namely, Nk = Nd , and under the identical initial condition, a simple and effective ILC [1] for the linear system (3.1) is uk+1 = uk + L ek ,

(3.6)

where uk  [u k (0), u k (1), . . . , u k (Nd − 1)]T , L is an appropriate learning gain matrix, and the tracking error is defined as follows: ⎡ ⎢ ⎢ ⎢ ⎣ 

ek (1) ek (2) .. .

⎤

⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎦ ⎣

ek (Nd )   ek

⎡



yd (1) yd (2) .. .

⎤

⎥ ⎢ ⎥ ⎢ ⎥−⎢ ⎦ ⎣

yd (Nd )   yd

⎡



yk (1) yk (2) .. .

⎤ ⎥ ⎥ ⎥. ⎦

yk (Nd )   yk

(3.7)

36

3 Averaging and Lifting Techniques for Linear Discrete-Time Systems

However, when the trial length Nk is iteration varying, which corresponds to a nonstandard ILC process, the learning control scheme (3.6) has to be redesigned.

3.2 ILC Design and Convergence Analysis In this section, based on the assumptions and notations that are given in Sect. 3.1, ILC design and convergence analysis are addressed, respectively. In practice, for one scenario that the kth trial ends before the desired trial length, namely, Nk < Nd , both the output yk (t) and the tracking error ek (t) on the time interval Id /Ik are missing, that is, unavailable for learning. For the other scenario that the kth trial is still running after the time instant, we want it to stop, i.e., Nk > Nd , the signals yk (t) and ek (t) after the time instant Nd are redundant and useless for learning. In order to cope with those missing signals or redundant signals in different scenarios, we define a stochastic matrix. Using the stochastic variables, a newly defined tracking error e∗k is introduced to facilitate the modified ILC design. The main procedure for deriving a modified ILC scheme can be described as follows: (I) Define a stochastic matrix Γk . Let Γk be a stochastic matrix with possible values

D(τm ) 

⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩

Nd

   diag {1, . . . , 1, 0, . . . , 0}, τm < Nd ,   

(3.8)

τm

I Nd ×Nd , τm ≥ Nd .

The relationship Γk = D(τm ) represents the event that the kth trial length of control process (3.1) is τm , which, as shown in Assumption 3.1, occurs with a probability of qm , where 0 < qm ≤ 1 is a prespecified constant. (II) Compute the mathematical expectation E{Γk }. The mathematical expectation of the stochastic matrix Γk is E{Γk } =

M 1 +M2

D(τm )qm

m=0

=

M 1 −1  m=0

D(τm )qm + I Nd ×Nd

 qm

m=M1

⎧ ⎨

= diag 1, 1, . . . , 1, ⎩   Nd −M1

 D.

 M +M 1 2 

M 1 +M2 m=1

qm , . . . ,

M 1 +M2 m=M1

⎫ ⎬ qm

⎭ (3.9)

3.2 ILC Design and Convergence Analysis

37

(III) Define a modified tracking error. Denote e∗k  Γk ek

(3.10)

as a modified tracking error, which renders to e∗k

 =

[e1 (1), . . . , ek (Nk ), 0, . . . , 0]T , Nk < Nd , Nk ≥ Nd . ek ,

(3.11)

Remark 3.2 Since the absent signals are unavailable, and the redundant signals are useless for learning, it is reasonable to define a modified tracking error e∗k as in (3.10), or equivalently (3.11). In the modified tracking error e∗k , the redundant signals are cut off when Nk > Nd , and the unavailable signals are set as zero, when Nk < Nd . (IV) The modified ILC scheme. Introduce an iteration-average operator A{·} [2], 1  f j (·), k + 1 j=0 i

A{ f k (·)} 

(3.12)

for a sequence f 0 (·), f 1 (·), . . . , f k (·), which plays a pivotal role in the proposed controller. The modified ILC scheme is given as follows: k+2  ∗ L e , k + 1 j=0 j i

uk+1 = A{uk } +

(3.13)

for all k ∈ N, where the learning gain matrix L will be determined in the following. The following theorem presents the first main result of the chapter. Theorem 3.1 For the discrete-time linear system (3.1) and the ILC scheme (3.13), choose the learning gain matrix L such that for any constant 0 ≤ ρ < 1, sup I − L D P ≤ ρ,

(3.14)

t∈Id

then the mathematical expectation of the error, E{ek (t)}, t ∈ Id , will converge to zero asymptotically as k → ∞. Remark 3.3 In practice, the probability distribution of the trial length Nk could be estimated in advance based on previous multiple experiments or by experience. In consequence, the probability qm in Assumption 3.1 is known. Finally, D can be calculated by (3.9), thus is available for controller design.

38

3 Averaging and Lifting Techniques for Linear Discrete-Time Systems

Proof The proof consists of two parts. Part I proves the convergence of the input error in iteration average and mathematical expectation using contraction mapping. Part II proves the convergence of the tracking error in mathematical expectation. Part I. Let Δuk  ud − uk be the input error. By the definition of iteration-average operator (3.12), we can rewrite A{Δuk+1 } as A{Δuk+1 } =

1 [Δuk+1 + (k + 1)A{Δuk }]. k+2

(3.15)

In addition, subtracting ud from both sides of the ILC law (3.13), we have k+2  ∗ L e . k + 1 j=0 j k

Δuk+1 = A{Δuk } −

(3.16)

Then, substituting (3.16) into the right-hand side of (3.15) and applying the operator E{·} on both sides of (3.15), we obtain E{A{Δuk+1 }} = E{A{Δuk }} − LE{A{e∗k }}.

(3.17)

Since both E{·} and A{·} are linear operators, we can exchange the operation orders of E{·} and A{·} yielding E{A{e∗k }} = DE{A{ek }},

(3.18)

where E{Γ j e j } = DE{e j } is applied as Γ j and e j are independent from each other. Meanwhile, from (3.3), (3.5), and (3.7), it follows that ek = PΔuk + (dd − dk ),

(3.19)

where dk  [dk (1), dk (2), . . . , dk (Nd )]T . Then, combining (3.18) and (3.19) gives E{A{e∗k }} = D PE{A{Δuk }} + DE{A{dd − dk }}.

(3.20)

By virtue of Assumption 3.2, we can obtain that E{dd − dk } = E{dd } − E{dk } = 0,

(3.21)

E{A{dd − dk }} = 0.

(3.22)

which yields

Then, the relationship (3.20) becomes E{A{e∗k }} = D PE{A{Δuk }}.

(3.23)

3.2 ILC Design and Convergence Analysis

39

In consequence, substituting (3.23) into (3.17), we have E{A{Δuk+1 }} = [I − L D P]E{A{Δuk }}.

(3.24)

Taking the norm  ·  on both sides leads to E{A{Δuk+1 }} ≤ I − L D PE{A{Δuk }} ≤ ρE{A{Δuk }}.

(3.25)

According to the condition (3.14) and 0 ≤ ρ < 1, (3.25) implies that lim E{A{Δuk }} = 0.

(3.26)

k→∞

Part II: Now we prove the convergence of ek in mathematical expectation. Multiplying both sides of (3.25) by (k + 2), it follows that  ⎧ ⎫ ⎫  ⎧  ⎨ k  ⎨ k+1 ⎬ ⎬          E E ≤ ρ Δu Δu j  j  + ρE{A{Δuk }}.   ⎩ ⎭ ⎭  ⎩ j=0  j=0

(3.27)

According to the boundedness of E{A{Δu  k }} from (3.25), (3.26) and Lemma 1 in [2], we can further derive limk→∞ E{ kj=0 Δu j } = 0, thus ⎫ ⎧ ⎫⎤ ⎡ ⎧ k k−1 ⎨ ⎬ ⎨ ⎬ lim E{Δuk } = lim ⎣E Δu j − E Δu j ⎦ = 0. k→∞ k→∞ ⎩ ⎭ ⎩ ⎭ j=0

(3.28)

j=0

Applying the operator E{·} on both sides of (3.19) yields E{ek } = PE{Δuk },

(3.29)

where Assumption 3.1 is applied. Finally, it is proved that limk→∞ E{ek } = 0. The proof is completed. Remark 3.4 In Assumption 3.1, it is assumed that the probability distribution is known and then the mathematical expectation matrix D can be calculated directly. Whereas, if qm is unknown, we know its lower and upper bounds, i.e., 0 ≤ α1 ≤ qm ≤ α2 ≤ 1 (α1 , α2 are known constants), then, according to (3.9), we have diag{1, 1, . . . , 1, (M1 + M2 )α1 , . . . , (M2 + 1)α1 }    Nd −M1

≤ D ≤ diag{1, 1, . . . , 1, (M1 + M2 )α2 , . . . , (M2 + 1)α2 },    Nd −M1

40

3 Averaging and Lifting Techniques for Linear Discrete-Time Systems

where “≤” means every corresponding diagonal element of the left matrix is less than that of the right one. Based on the lower and upper bounds of D and convergence condition (3.14), the controller can be designed similarly.

3.3 Extension to Time-Varying Systems In this section, the proposed ILC scheme is extended to time-varying systems xk (t + 1) = At xk (t) + Bt u k (t), yk (t) = Ct xk (t),

(3.30)

where At , Bt , and Ct are time-varying matrices with appropriate dimensions and Ct Bt = 0. This system can be written equivalently as the Nk × Nk -dimensional lifted system ⎡ ⎢ ⎢ ⎢ ⎣

yk (1) yk (2) .. .

⎤

⎡

⎥ ⎢ ⎥ ⎢ ⎥ = Pk ⎢ ⎦ ⎣

yk (Nk ) where wk (t) = Ct

t−1 j=0

u k (0) u k (1) .. .

⎤

⎡

⎥ ⎢ ⎥ ⎢ ⎥+⎢ ⎦ ⎣

u k (Nk − 1)

wk (1) wk (2) .. .

⎤ ⎥ ⎥ ⎥, ⎦

(3.31)

wk (Nk )

A j xk (0) and ⎡

⎢ Pk = ⎣ C Nk

C1 B0 .. .  Nk −1 k=1

⎤ ··· 0 ⎥ .. .. ⎦. . . At B0 · · · C Nk B Nk −1

The desired trajectory yd is generated by yd = P ud + dd ,

(3.32)

where ⎡ ⎢ P=⎣ C Nd

C1 B0 .. .  Nd −1 k=1

⎤ ··· 0 ⎥ .. .. ⎦ . . At B0 · · · C Nd B Nd −1

 and dd = [dd (1), dd (2), . . . , dd (Nd )]T , dd (t) = Ct t−1 j=0 A j x d (0). The result is summarized in the following theorem. Theorem 3.2 For the discrete-time linear time-varying system (3.30) and the ILC algorithm (3.13), choose the learning gain matrix L such that for any constant 0 ≤ ρ < 1,

3.3 Extension to Time-Varying Systems

sup I − L t D P ≤ ρ,

41

(3.33)

t∈Id

the mathematical expectation of the error, E{ek (t)}, t ∈ Id , will converge to zero asymptotically as k → ∞. Proof The proof can be performed similarly as in the proof of Theorem 3.1. Considering the desired dynamics and the lifted system (3.31), we have ek = PΔuk + (dd − wk ),

(3.34)

where wk  [wk (1), wk (2), . . . , wk (Nd )]T . Similar as (3.20)–(3.23), it follows that E{A{Δuk+1 }} = [I − L t D P]E{A{Δuk }}.

(3.35)

Now, following the procedure of the proof of Theorem 3.1, we can conclude that limk→∞ E{ek } = 0. This completes the proof. Remark 3.5 In Theorems 3.1 and 3.2, the identical initialization condition is replaced by E{xk (0)} = xd (0). So, other than deriving the convergence of tracking error, we prove its mathematical expectation converges asymptotically using the mathematical expectation operator and the proposed iteration-average-based ILC scheme.

3.4 Illustrative Simulations In order to show the effectiveness of the proposed ILC scheme, two examples are considered. Example 3.1 (Time-invariant system) Consider the following discrete-time linear time-invariant system ⎛

⎞ ⎛ ⎞ 0.50 0 1.00 0 xk (t + 1) = ⎝ 0.15 0.30 0 ⎠ xk (t) + ⎝ 0 ⎠ u k (t), −0.75 0.25 −0.25 1.00   yk (t) = 0 0 1.00 xk (t),

(3.36)

where xk (0) = [0, 0, 0]T , k ∈ N. Let the desired trajectory be yd (t) = sin(2π t/50) + sin(2π t/5) + sin(50π t), t ∈ Id  {1, 2, . . . , 50}, as shown in Fig. 3.1, and thus, Nd = 50. Without loss of generality, set u 0 (t) = 0, t ∈ Id in the first iteration. Moreover, assume that M1 = M2 = 5 and that Nk is a stochastic variable satisfying discrete uniform distribution. Then, Nk ∈ {45, 46, . . . , 55} and P[Nk = τm ] = 1/11, where τm = 45 + m, m ∈ {0, 1, . . . , 10}. Further, the learning gain is set as L = 0.5I50×50 . The performance of the tracking error, ek , is presented in Fig. 3.2. It shows that the tracking error ek  will converge within 42 iterations.

42

3 Averaging and Lifting Techniques for Linear Discrete-Time Systems 2 1.5 1

Reference

0.5 0 −0.5 −1 −1.5 −2

0

10

20

30

40

50

80

100

Time axis

Fig. 3.1 The reference yd with desired trial length Nd = 50 45 40 35

25

k

||e ||

30

20 15 10 5 0

0

20

40

60

Iteration axis

Fig. 3.2 Norm of tracking error in each iteration of ILC with nonuniform trial length: M1 = M2 = 5

Moreover, Fig. 3.3 gives the tracking error profiles for 10, 20, 80, and 100th iterations, respectively. To demonstrate the effect of M1 and M2 on the convergence speed of the tracking error, we fix the learning gain L = 0.5I50×50 and set M1 = M2 = 30. Here, Nk ∈

3.4 Illustrative Simulations

43

1.5

e

e

10

20

e

e

80

100

Tracking errors

1

0.5

0

−0.5

−1

0

10

20

30

40

50

Time axis

Fig. 3.3 Tracking error profiles of ILC with nonuniform trial length: M1 = M2 = 5 45 40 35

25

k

||e ||

30

20 15 10 5 0

0

20

40

60

80

100

Iteration axis

Fig. 3.4 Norm of tracking error in each iteration of ILC with nonuniform trial length: M1 = M2 = 30

{20, 21, . . . , 80} and P[Nk = τm ] = 1/61, where τm = 20 + m, m ∈ {0, 1, . . . , 60}. It can be seen from Fig. 3.4 that the convergence will be achieved after more than 60 iterations, namely, the convergence speed is obviously slower than the case M1 = M2 = 5.

44

3 Averaging and Lifting Techniques for Linear Discrete-Time Systems 1.5

{e10}

{e20}

{e40}

{e80}

1

k

{e }

0.5

0

−0.5

−1

−1.5

0

10

20

30

40

50

Time axis

Fig. 3.5 The mathematical expectation of tracking errors when the proposed ILC scheme is applied to (3.36) 1.5

e

100

e

200

e

300

e

500

Tracking errors

1

0.5

0

−0.5

−1

0

10

20

30

40

50

Time axis

Fig. 3.6 Tracking error profiles when the proposed ILC scheme is applied to (3.36)

To show the effectiveness of the proposed ILC scheme with randomly varying initial states, we fix the learning gain L = 0.5I50×50 and M1 = M2 = 5. Assume xk (0) is a stochastic variable with probability P[xk (0) = v1 ] = 1/3, P[xk (0) = v2 ] = 1/3, and P[xk (0) = v3 ] = 1/3, where v1 = [0, 0, −1]T , v2 = [0, 0, 0]T , and v3 = [0, 0, 1]T . Figure 3.5 shows that the mathematical expectation of the tracking

3.4 Illustrative Simulations

45

1.5 e100

e200

e300

e500

Tracking errors

1

0.5

0

−0.5

−1

0

10

20

30

40

50

Time axis

Fig. 3.7 Tracking error profiles when the ILC scheme in [3] is applied to (3.36)

error E{ek } will converge to zero within 80 iterations. The tracking error profiles of the proposed ILC scheme and the ILC scheme without average operator in [3] are illustrated in Figs. 3.6 and 3.7, respectively. It is obvious that the performance of the proposed ILC scheme is superior to that of the ILC scheme in [3] under the situation of randomly varying initial states. Similarly, in [4, 5], the identical initialization condition is also indispensable. Example 3.2 (Time-varying system) In order to show effectiveness of our proposed ILC algorithm for time-varying systems, we consider the following discrete-time linear time-varying system ⎛

⎛ ⎞ ⎞ 0.2e−t/100 −0.6 0 1.3 0 0.5 sin(t) ⎠ xk (t) + ⎝ 0.5 ⎠ u k (t), xk (t + 1) = ⎝ 0.6 0 0 0.7   yk (t) = −0.5 1.5 0 xk (t),

(3.37)

where xk (0) = [0, 0, 0]T , k ∈ N. Similarly as Example 3.1, let the desired trajectory be yd (t) = sin(2π t/50) + sin(2π t/5) + sin(50π t), t ∈ Id = {1, 2, . . . , 50}. Set u 0 (t) = 0, k ∈ Id in the first iteration. Assume that Nk satisfies the Gaussian distribution with mean 50 and standard deviation 10, namely, Nk ∼ N (50, 100). Since Nk is integer in this example, it is generated approximately by the MATLAB command “round(50 + 10 ∗ randn(1, 1))”. Further, set the learning gain as L = 2I50×50 . The performance of the tracking error ek  is presented in Fig. 3.8, where ek  will converge within 50 iterations. In addition, Fig. 3.9 gives tracking error profiles at 10th, 20th, 80th and 100th iterations.

46

3 Averaging and Lifting Techniques for Linear Discrete-Time Systems 35 30 25

||ek||

20 15 10 5 0

0

20

40

60

80

100

Iteration axis

Fig. 3.8 Norm of tracking error in each iteration of ILC with Nk ∼ N (50, 100) 0.4 e

10

e

20

e

e

80

100

0.3

Tracking errors

0.2 0.1 0 −0.1 −0.2 −0.3 −0.4

0

10

20

30

Time axis

Fig. 3.9 Tracking error profiles of ILC with Nk ∼ N (50, 100)

40

50

3.5 Summary

47

3.5 Summary This chapter presents the ILC design and analysis results for systems with nonuniform trial lengths under the framework of lifted systems. Due to the variation of the trial lengths, a modified ILC scheme is developed by applying an iteration-average operator and introducing a stochastic matrix. Owing to the application of lifted systems, the traditional λ-norm is avoided and the monotonic convergence of the tracking error is derived in the sense of mathematical expectation. The effectiveness of the proposed approach is verified by two illustrative examples. The results in this chapter are mainly based on [6].

References 1. Bien Z, Xu J-X (1998) Iterative learning control: analysis, design, integration and applications. Kluwer, Boston 2. Park K-H (2005) An average operator-based PD-type iterative learning control for variable initial state error. IEEE Trans Autom Control 50(6):865–869 3. Seel T, Schauer T, Raisch J (2011) Iterative learning control for variable pass length systems. In: Proceedings of 18th IFAC world congress, pp 4880-4885 4. Longman RW, Mombaur KD (2006) Investigating the use of iterative learning control and repetitive control to implement periodic gaits. Lect Notes Control Inf Sci 340:189–218 5. Moore KL (2000) A non-standard iterative learning control approach to tracking periodic signals in discrete-time non-linear systems. Int J Control 73(10):955–967 6. Li X, Xu J-X (2015) Lifted system framework for learning control with different trial lengths. Int J Autom Comput 12(3):273–280

Chapter 4

Moving Averaging Techniques for Linear Discrete-Time Systems

To further improve the learning performance, this chapter will propose two novel ILC schemes for discrete-time linear systems with randomly varying trial lengths. In contrast to Chaps. 2 and 3 that advocate to replace the missing control information by zero, the proposed learning algorithms in this chapter are equipped with a random searching mechanism to collect useful but avoid redundant past tracking information, which could expedite the learning speed. The searching mechanism is realized by using the newly defined stochastic variables and an iteratively moving average operator. The convergence of the proposed learning schemes is strictly proved based on the contraction mapping methodology. Two illustrative examples are provided to show the superiorities of the proposed approaches.

4.1 Problem Formulation Consider the following discrete-time linear system: xk (t + 1) = Axk (t) + Bu k (t), yk (t) = C xk (t),

(4.1)

where k ∈ N is the iteration index, t ∈ {0, 1, 2, . . . , Nk } denotes the time instant, and Nk is the trial length at the kth iteration. Moreover, xk (t) ∈ Rn , u k (t) ∈ R p , and yk (t) ∈ Rr denote the state, input, and output of the system (4.1), respectively. Furthermore, A, B, and C are constant matrices with appropriate dimensions, and C B = 0, i.e., the relative degree is assumed to be one. It is worth to point out that the results and convergence analysis in this chapter can be extended to linear timevarying systems straightforwardly, and thus we just consider the time-invariant case to clarify our idea. Let yd (t), t ∈ {0, 1, 2, . . . , Nd } be the desired output trajectory. © Springer Nature Singapore Pte Ltd. 2019 D. Shen and X. Li, Iterative Learning Control for Systems with Iteration-Varying Trial Lengths, https://doi.org/10.1007/978-981-13-6136-4_4

49

50

4 Moving Averaging Techniques for Linear Discrete-Time Systems

Assume that, for any realizable output trajectory yd (t), there exists a unique control input u d (t) ∈ R p such that xd (t + 1) = Axd (t) + Bu d (t), yd (t) = C xd (t),

(4.2)

where u d (t) is uniformly bounded for all t ∈ {0, 1, 2, . . . , Nd } with Nd being the desired trial length. The control objective is to track the desired reference yd (t), t ∈ {0, 1, 2, . . . , Nd } by determining a sequence of control inputs u k such that the tracking error converges as the iteration number k increases. Before addressing the controller design problem, the following assumptions are imposed. Assumption 4.1 The coupling matrix C B is of full-column rank. Assumption 4.2 The initial states satisfy xd (0) − xk (0) ≤ ,  > 0. Remark 4.1 The initial state resetting problem is one of the fundamental issues in ILC field as it is a standard assumption to ensure the perfect tracking performance. In the past three decades, some papers have devoted to remove this condition, such as [1–3], by developing additional control mechanisms. Under Assumption 4.2, since the initial state is different from the desired initial state, it is impossible to achieve the perfect tracking. The ILC algorithms should force the output trajectory to be as close as possible to the target. Remark 4.2 It is worthy noting that, unlike the classic ILC theory that requires control tasks repeat on a fixed time interval, the trial lengths Nk , k ∈ N are iterationvarying and may be different from the desired trial length Nd . For the case that the kth trial length is shorter than the desired trial length, both the system output and the tracking error information will be missing and cannot be used for learning. Thus, this chapter aims to redesign ILC schemes to make up the missing signals by making full use of the previously available tracking information, and thus expedite the learning speed. Although some previous works have been reported, such as the Chaps. 2 and 3, a basic assumption is that the probability distribution of Nk is known prior. In this work, the proposed ILC algorithms will be equipped with an automatic searching mechanism, where the probability distribution of randomly varying trial lengths is no longer required.

4.2 Controller Design I and Convergence Analysis In this section, a novel ILC algorithm will be developed to reduce the effect of redundant tracking information in the design of ILC algorithms in Chaps. 2 and 3 and thus could expedite a faster convergence speed.

4.2 Controller Design I and Convergence Analysis

51

Fig. 4.1 Illustration of Sm t,k : set m = 5 and t = 9, then Sm t,k = {Nk , Nk−3 , Nk−4 }, which implies that n kt = 3 and m − n kt = 2

Recall that Nk is the trial length at kth iteration. It varies in the iteration domain randomly. Denote Sm t,k  {Nk+1− j | t > Nk+1− j , j = 1, 2, . . . , m} m where m > 1 is an integer. Let n kt = |Sm t,k | be the amount of elements in St,k . That is, k for a given time instant t > 0 and a given iteration k, there are only m − n t iterations with the available tracking information in the past m iterations, and the tracking information at other iterations are missing. To show the definition of the set Sm t,k and the number n kt , a simple example is illustrated in Fig. 4.1. It is worthy pointing out that the number n kt is a random variable due to the randomness of the trial lengths. If we denote the probability of the occurrence of the output at time instant t as p(t), the mathematical expectation of n kt can be calculated as E{n kt } = p(t)m. Therefore, m − n kt would increase to infinity as m goes to infinity in the sense of mathematical expectation. This property guarantees the reasonability of the following assumption.

Assumption 4.3 For a given iteration number k and a time instant t ∈ {0, 1, . . . , Nk }, the number m − n kt ≥ 1. That is, there exists at least one iteration whose trial length is not shorter than t in the past m consecutive iterations. Remark 4.3 Assumption 4.3 is imposed to guarantee the learning effectiveness. If m − n kt = 0, it gives Nk+1− j < t for j = 1, 2, . . . , m, namely, all trial lengths of the adjacent m iterations before the (k + 1)th iteration are shorter than the given time t. This further means that at time instant t, there is no output information available and nothing can be learned from the past m iterations. By assuming m − n kt ≥ 1, the effective learning process can be guaranteed. However, this assumption is not restrictive as m − n kt would increase to infinity as m goes to infinity. In addition, Assumption 4.3 implies that iteration-varying trial lengths are not totally stochastic in this section. This assumption would be further relaxed in the Sect. 4.3. Similar to Chap. 2, we introduce a stochastic variable γk (t), t ∈ {0, 1, . . . , Nd }, satisfying Bernoulli distribution and taking binary values 0 and 1. The relationship γk (t) = 1 represents the event that the control process can continue to the time instant

52

4 Moving Averaging Techniques for Linear Discrete-Time Systems

t at the kth iteration, while γk (t) = 0 denotes the event that the control process cannot continue to the time instant t. Based on the notations above and Assumption 4.3, the first proposed ILC law is presented as follows (I) :

u k+1 (t) =

m  1 γk+1− j (t)u k+1− j (t) m − n kt j=1

+

m  1 Γ γk+1− j (t)ek+1− j (t + 1), m − n kt j=1

(4.3)

where Γ is the learning gain matrix to be determined, and ek  yd − yk represents the tracking error. Remark 4.4 From (4.3), we can see that the stochastic variable γk (t) is also adopted in the control input part, i.e., the first term on the right-hand side of (4.3). It implies that if the trial length is shorter than the given time t, both the corresponding input and tracking error signals will not be involved in the updating. The major difference between ILC law (4.3) and the ones in Chaps. 2 and 3 lies in that the average operator will not incorporate redundant control information into the learning law, and thus the convergence speed could be expedited. Remark 4.5 The initial m input and tracking error signals in the searching algorithm can be determined by using other control methods such as the classic feedback control that can stabilize the controlled system, and they will not affect the final convergence performance. The convergence of the proposed ILC scheme (4.3) can be summarized in the following theorem. Theorem 4.1 Consider the system (4.1) and the ILC law (4.3). Assume that the Assumptions 4.1–4.3 hold. If the following condition holds, I − Γ C B ≤ ρ < 1,

(4.4)

then the tracking error ek will converge to the δ-neighborhood of zero asymptotically in the sense of λ-norm as k goes to infinity, where δ > 0 is a suitable constant to be defined later. Proof Denote Δxk = xd − xk the state error, Δu k = u d − u k the input error, and ek = yd − yk the tracking error. Subtracting both sides of the updating law (4.3) from u d , we have

4.2 Controller Design I and Convergence Analysis

Δu k+1 (t) =

53

m  1 γk+1− j (t)Δu k+1− j (t) m − n kt j=1

−

m  1 Γ γk+1− j (t)ek+1− j (t + 1). m − n kt j=1

(4.5)

From (4.1) and (4.2), we have ek (t + 1) = C AΔxk (t) + C BΔu k (t).

(4.6)

Substituting (4.6) into (4.5) implies Δu k+1 (t) =

m  1 γk+1− j (t)Δu k+1− j (t) m − n kt j=1

−

=

m   1 Γ γk+1− j (t) C AΔxk+1− j (t) k m − nt j=1  + C BΔu k+1− j (t)

m  1 γk+1− j (t)(I − Γ C B)Δu k+1− j (t) m − n kt j=1

 1 − Γ C A γk+1− j (t)Δxk+1− j (t). m − n kt j=1 m

(4.7)

Since Δxk (t) = At Δxk (0) +

t−1 

At−n−1 BΔu k (n),

(4.8)

n=0

it follows that Δu k+1 (t) =

m  1 γk+1− j (t)[I − Γ C B]Δu k+1− j (t) m − n kt j=1

−

 1 Γ C At+1 γk+1− j (t)Δxk+1− j (0) k m − nt j=1

−

m t−1   1 Γ C A γ (t) At−n−1 BΔu k+1− j (n). k+1− j m − n kt n=0 j=1

m

(4.9)

54

4 Moving Averaging Techniques for Linear Discrete-Time Systems

Taking norm to both sides of (4.9), we can obtain that Δu k+1 (t) m  1 γk+1− j (t)I − Γ C BΔu k+1− j (t) m − n kt j=1

≤

+ κα t+1  m t−1   1 + κβ γk+1− j (t) α t−n Δu k+1− j (n)), m − n kt n=0 j=1

where

1 m−n kt

m j=1

(4.10)

γk+1− j (t) = 1, α ≥ A, β ≥ B, and κ ≥ Γ C are applied.

Multiplying both sides of (4.10) by α −λt , and taking the supremum with respect to the time t, we have Δu k+1 (t)λ ≤

m  1 γk+1− j (t)I − Γ C BΔu k+1− j (t)λ m − n kt j=1

+ κα  1 κβ γk+1− j (t) m − n kt j=1   t−1  −λt t−n α Δu k+1− j (n) . × sup α m

+

t

(4.11)

n=0

Note that sup α

−λt

t

 t−1 

 α

n=0

= sup α

−(λ−1)t

t

t−n

Δu k+1− j (n)

 t−1 

 α

−λn

n=0

≤ sup α −(λ−1)t

 t−1 

t

n=0

 sup(α −λn Δu k+1− j (n))α (λ−1)n n

= Δu k+1− j (t)λ sup α t

≤

Δu k+1− j (n)α

(λ−1)n

−(λ−1)Nd

1−α α λ−1 − 1

−(λ−1)t

t−1 

α (λ−1)n

n=0

Δu k+1− j (t)λ ,

(4.12)

4.2 Controller Design I and Convergence Analysis

55

thus, (4.11) becomes Δu k+1 (t)λ ≤

m  1 γk+1− j (t)ρ0 Δu k+1− j (t)λ + κα m − n kt j=1

≤ ρ0

max

j=1,2,...,m

Δu k+1− j (t)λ + κα,

(4.13)

where  κβ(1 − α −(λ−1)Nd ) , ρ0 Δq I − Γ C B + α λ−1 − 1 and the equation Define

1 m−n kt

m j=1

γk+1− j (t) = 1 is applied.

Q k+1 = Δu k+1 (t)λ −

κα . 1 − ρ0

From (4.13), it follows that Q k+1 ≤ ρ0

max

j=1,2,...,m

Q k+1− j .

(4.14)

κα -neighborhood of zero and If Q k+1 ≤ 0, it means Δu k+1 (t)λ has entered the 1−ρ 0 will stay in the neighborhood. Thus, to show the bounded convergence, it is sufficient to analyze the scenario with Q k+1 > 0. Similar to (4.14), we have the following relations:

Q k+2 ≤ ρ0

max

j=1,2,...,m

Q k+2− j .

(4.15)

Note that max

j=1,2,...,m

Q k+2− j

≤ max{ max Q k+2− j , Q k+1 } j=2,...,m

≤ max{ max Q k+1− j , ρ0 j=1,...,m

max

j=1,2,...,m

Q k+1− j }

= max Q k+1− j , j=1,...,m

and it follows that Q k+2 ≤ ρ0 max Q k+1− j . j=1,...,m

(4.16)

56

4 Moving Averaging Techniques for Linear Discrete-Time Systems

By induction, we can obtain that Q k+ p ≤ ρ0 max Q k+1− j , p = 1, 2, . . . , m.

(4.17)

max Q k+ p ≤ ρ0 max Q k+1− j ,

(4.18)

j=1,...,m

Therefore, it gives p=1,...,m

j=1,...,m

which implies the convergence of max p=1,...,m Q k+ p , i.e., lim Δu(t)λ =

k→∞

κα . 1 − ρ0

From (4.8), it is not difficult to obtain ek (t) = C A Δxk (0) + C t

t−1 

At−n−1 BΔu k (n).

(4.19)

1 − α −(λ−1)Nd Δu k (t)λ , αλ − α

(4.20)

n=0

Taking λ-norm on both sides of (4.19) gives ek (t)λ ≤ c + cβ

where c = C. Due to the convergence of Δu k (t)λ , we can obtain the convergence of ek (t)λ to a neighborhood of zero where the bound is proportional to . In other words, there exists an appropriate δ > 0 such that lim ek (t) ≤ δ.

k→∞

This completes the proof. Remark 4.6 It is noted that the convergence condition given in Theorem 4.1, i.e., (4.4), is the same as the one in classic ILC, which is irrelevant with the probability distribution of the randomly varying trial lengths. This is one of the advantages of the proposed ILC scheme since it is shown that the same convergence condition can be applied to deal with more complex control problems. Although the probability distribution of the trial length is not involved in (4.4), different probability distribution will lead to different convergence speed. In details, for a given time instant t, the greater probability of the event Nk ≥ t, the faster the convergence speed. This can be verified from (4.18). For a greater probability of Nk ≥ t, we can select a smaller m, which indicates (4.18) will converge faster. However, due to the lack of analysis

4.2 Controller Design I and Convergence Analysis

57

tools, currently, it is difficult to present an analytic expression for the probability distribution and the convergence speed. This is an interesting problem and should be addressed in future work. Remark 4.7 The choice of m in the controller (4.3) depends on the length of the random interval for the trial length Nk . If the random interval is long, it implies that the trial length varies drastically in the iteration domain. In such a case, more previous trials will expedite the convergence speed because some of the missing information can be made up. While if the random interval is short, which means that the trial length in each iteration changes slightly and is close to the desired trial length, it is better to use a small number of previous trials. When the randomness is low, a large number of past trials may adversely weaken the learning effect because the large averaging operation would reduce the corrective action from the most recent trials.

4.3 Controller Design II and Convergence Analysis In this section, we will give a new ILC law to make full use of the previous control information, which thus could expedite the learning speed. In order to facilitate the controller design, the following assumption is first imposed. Assumption 4.4 For a given iteration number k > m and a time instant t ∈ {0, 1, . . . , Nk }, we can find m past iterations such that Nk+1−rk, j > t, j = 1, 2, . . . , m, where rk, j , j = 1, 2, . . . , m is an increasing sequence with 1 ≤ rk, j ≤ k being an integer. Remark 4.8 Assumption 4.4 is reasonable since we can always find enough past iterations satisfying the assumption after a sufficiently large number of iterations. Otherwise, the learning process can not be guaranteed. In practical, only the first few iterations may not satisfy Assumption 4.4. For these iterations, we can adopt the control algorithm in Sect. 4.2 or the ones in Chaps. 2 and 3 if necessary, which will not affect the convergence of the learning algorithm. A simple example of Assumption 4.4 is illustrated in Fig. 4.2. Based on Assumption 4.4, the second proposed ILC law is given as follows (II) :

u k+1 (t) =

m 1  u k+1−rk, j (t) m j=1

+

m 1  Γ ek+1−rk, j (t + 1). m j=1

(4.21)

Remark 4.9 From (4.21), it can be found that rk, j , j = 1, 2, . . . , m are random variables because of the randomness of the trial lengths. The introduction of these

58

4 Moving Averaging Techniques for Linear Discrete-Time Systems

Fig. 4.2 Illustration of Assumption 4.4: set m = 4 and t = 9, then we can find that N j > t, j = k − 1, k − 2, k − 5, k − 6, which implies that rk,1 = 2, rk,2 = 3, rk,3 = 6, rk,4 = 7

random variables actually forms the searching mechanism in the control algorithm. By fully searching and utilizing the available tracking information, (4.21) is able to increase the convergence speed. Remark 4.10 In this work, the ILC laws (4.3) and (4.21) are totally different. Based on Assumption 4.3, the searching mechanism in ILC law (4.3) is restricted in the last m iterations. Within the last m iterations, (4.3) incorporates all the available information into the controller. The main advantage for this controller is some of “too old” tracking information, which may weaken the correction from the latest iterations, can be avoided. However, the drawback is that the available historical information may be too scanty to improve the learning process if the probability of the occurrence of full trial length is small. While for the ILC law (4.21), it keeps searching until m available output signals are found. This controller is good at collecting all useful past control information, but the information far away from the current iteration may degrade the learning performance. The comparison of these two ILC laws will be presented in the numerical examples. The second main result of this chapter is summarized in the following theorem. Theorem 4.2 Consider system (4.1) and ILC law (4.21). Assume Assumptions 4.1, 4.2, and 4.4 hold. If the following condition holds, I − Γ C B ≤ ρ < 1,

(4.22)

then the tracking error ek will converge to the δ-neighborhood of zero asymptotically in the sense of λ-norm as k goes to infinity, where δ > 0 is a suitable constant. Proof For a given time instant t, let G t  {Nk | Nk > t, k = 1, 2, . . .}. Define a new sequence 1 ≤ σ1 < σ2 < · · · < σi < . . . and assume Nσi is the ith elements of G t . Then G t can be represented as G t = {Nσ1 , Nσ2 , . . . , Nσi , . . .}. For a given iteration number k, if σi < k + 1 ≤ σi+1 and i ≥ m, by the definition of G t , the ILC law (4.21) can be rewritten as follows

4.3 Controller Design II and Convergence Analysis

u k+1 (t) =

59

m m 1  1  u σi+1− j (t) + Γ eσ (t + 1). m j=1 m j=1 i+1− j

(4.23)

Moreover, the control input will not be updated from iteration σi + 1 to σi+1 , namely, u σi +1 (t) = · · · = u k+1 (t) = · · · = u σi+1 (t).

(4.24)

Therefore, (4.23) and (4.24) imply that u σi+1 (t) =

m m 1  1  u σi+1− j (t) + Γ eσ (t + 1). m j=1 m j=1 i+1− j

(4.25)

Hence, it is sufficient to prove the convergence of the input sequence u σi , i = 1, 2, . . .. Similar to the proof of Theorem 4.1, the following inequality can be obtained Δu σi+1 (t)λ ≤ ρ0

max

j=1,2,...,m

Δu σi+1− j (t)λ + κα.

(4.26)

By following the same procedure as the latter part proof of Theorem 4.1, the convergence of the input sequence can be derived lim Δu σi (t)λ =

κα , 1 − ρ0

lim Δu k (t)λ =

κα . 1 − ρ0

i→∞

which further gives

k→∞

Finally, the convergence of the tracking error can be proved similarly as the proof of Theorem 4.1, i.e., lim ek (t) ≤ δ.

k→∞

The proof is thus completed. Remark 4.11 The learning algorithm (4.21) is stochastic due to the randomness of rk, j . It seems that the algorithm (4.21) is deterministic but it is essentially stochastic because of the stochastic selection of suitable iterations, which can be seen from the subscripts of inputs and tracking errors. In addition, due to the introduction of randomly varying iteration length, the convergence proof in this chapter uses a sequential contraction mapping as can be seen from (4.18) and (4.26). The major difference between the two recursions lies in that the sequential contraction in (4.18) is deterministic while in (4.26) is stochastic.

60

4 Moving Averaging Techniques for Linear Discrete-Time Systems

Remark 4.12 Similar results and convergence analysis can be extended to linear time-varying systems, namely, A = At , B = Bt and C = Ct , and nonlinear systems with globally Lipschitz continuous uncertainties without significant efforts. For nonlinear systems without Lipschitz conditions, composite energy function (CEF) would be an optional approach. Remark 4.13 In ILC field, 2D approach is another preferable analysis tools. For instance, in [4], 2D approach is applied to analyze the stability property of an inferential ILC, and in [5], a systematic procedure for ILC design by using 2D approach is developed. Therefore, investigating ILC with nonuniform trial lengths by 2D method would be an interesting research topic. As will be detailed in Chap. 6, it is not difficult to reformulate the problem addressed in this chapter into 2D framework. Due to the variation of the trial lengths, the stochastic variable is still needed to modify the 2D variables when they are unavailable/missing. However, for nonlinear systems, it is difficult to apply 2D approach.

4.4 Illustrative Simulations In order to show the effectiveness of the proposed ILC schemes, the same discretetime linear system in Chap. 2 is considered ⎛

⎞ ⎛ ⎞ 0.50 0 1.00 0 xk (t + 1) = ⎝ 0.15 0.30 0 ⎠ xk (t) + ⎝ 0 ⎠ u k (t), −0.75 0.25 −0.25 1.00   yk (t) = 0 0 1.00 xk (t).

(4.27)

Let the desired trajectory be yd (t) = sin(2π t/50) + sin(2π t/5) + sin(50π t), t ∈ Id  {0, 1, . . . , 50}, and thus Nd = 50. Without loss of generality, set u 0 (t) = 0, t ∈ Id in the first iteration. Moreover, assume that the trial length Nk varies from 30 to 50 satisfying discrete uniform distribution. This assumption is just a simple illustration. For other kinds of probability distribution, the proposed ILC scheme still works well since the probability distribution of trial lengths is not involved in the convergence conditions (4.4) and (4.22), and the influence of the probability distribution has also been discussed in Remark 4.6.

4.4.1 Simulations for ILC Law (I) In this subsection, let m = 4 and the learning gain is set as L = 0.5, which renders to I − LC B = 0.5 < 1. First, we consider the case with identical initial condition, i.e., xk (0) = [0, 0, 0]T , k ∈ N. The performance of the maximal tracking error, ei s  supt∈Id ei , is presented in Fig. 4.3. It shows that the maximal tracking error

4.4 Illustrative Simulations

61

1

10

The proposed ILC (I) The ILC in Chapter 2 0

Tracking error

10

−1

10

−2

10

−3

10

−4

10

0

20

40

60

80

100

Iteration axis

Fig. 4.3 The maximal tracking error profiles of the proposed ILC scheme (I) and the one in Chap. 2 under identical initial condition

ei s decreases from 1.945 to 0.0004 within 100 iterations. Meanwhile, to show the effectiveness of the proposed ILC scheme, the comparisons with the ILC law in Chap. 2 is also given in Fig. 4.3. It is obvious that by removing the redundant control input signal in the control laws, the proposed ILC scheme outperforms the one in Chap. 2. In detail, it can be seen that the convergence of ILC law (I) is much faster and smoother, which could be more desirable to practical applications. It is noted that oscillations in the tracking error profiles are due to the variation of the trial lengths. The tracking performance of the ILC law (I) at different iterations is shown in Fig. 4.4, where we can see that after 50 iterations, the difference between y50 and yd is almost invisible. Furthermore, we consider the case with iteration-varying initial condition, namely, xk (0) = k [1, 1, 1]T , k ∈ N with k = 0.05 sin(k). The convergence of ei s is given in Fig. 4.5. It is can been seen that after 50 iterations ei s cannot been reduced further, and the final convergence bound is proportional to 0.05 which is the magnitude of xk (0).

4.4.2 Simulations for ILC Law (II) This subsection will demonstrate the effectiveness of the proposed ILC scheme (II). Similar to Sect. 4.4.1, we select the learning gain L = 0.5. Let m = 2 and xk (0) = [0, 0, 0]T , k ∈ N. Figure 4.6 shows the convergence of the maximal tracking error

62

4 Moving Averaging Techniques for Linear Discrete-Time Systems 2 yd

y1

y5

y50

1.5 1

Outputs

0.5 0 −0.5 −1 −1.5 −2

5

10

15

20

25

30

35

40

45

50

Time axis

Fig. 4.4 The system outputs at the 1st, 5th, and 50th iterations. The reference yd is given for comparison 1

10

0

Tracking error

10

−1

10

−2

10

−3

10

0

20

40

60

80

100

Iteration axis

Fig. 4.5 The convergence of the maximal tracking error of ILC law (I) without identical initial condition

4.4 Illustrative Simulations

63

1

10

The proposed ILC (II) The ILC in Chapter 2 0

10

−1

Tracking error

10

−2

10

−3

10

−4

10

−5

10

−6

10

0

20

40

60

80

100

Iteration axis

Fig. 4.6 The maximal tracking error profiles of the proposed ILC scheme (II) and the one in Chap. 2 under identical initial condition 1

10

The proposed ILC (I) The proposed ILC (II) 0

10

−1

Tracking error

10

−2

10

−3

10

−4

10

−5

10

−6

10

0

20

40

60

Iteration axis

Fig. 4.7 The comparison of the proposed ILC algorithms (I) and (II)

80

100

64

4 Moving Averaging Techniques for Linear Discrete-Time Systems 1

10

Tracking error

0

10

−1

10

−2

10

0

20

40

60

80

100

Iteration axis

Fig. 4.8 The convergence of the maximal tracking error of ILC law (II) without identical initial condition

ek s . By comparing with the ILC law in Chap. 2, we can see that the proposed ILC algorithm (II) is able to expedite the convergence speed a hundredfold. Moreover, Fig. 4.7 shows the comparison between the proposed ILC schemes (I) and (II). It finds that the ILC (I) presents a smoother tracking performance, while the ILC (II) wins by a faster convergence speed. The reason is that the ILC (II) incorporates more historic learning information into updating and expedites the convergence speed. However, utilizing some older control information in the control algorithm may lead to oscillation in tracking performance. Therefore, which algorithm should be chosen is entirely dependent on the control targets. If the identical initial condition is not satisfied, e.g., xk (0) = k [1, 1, 1]T , k ∈ N, the ILC (4.21) still works well, as shown in Fig. 4.8, by sacrificing the convergence accuracy.

4.5 Summary In this chapter, two novel improved ILC schemes for systems with randomly varying trial lengths are presented. To improve the control performance under iterationvarying trial lengths, the proposed learning algorithms are equipped with a searching mechanism to collect useful but avoid redundant past control information, which is able to expedite the learning speed. The searching mechanism is realized by introducing newly defined stochastic variables and an iteratively moving average operator. The convergence of the proposed learning schemes is analyzed according to

4.5 Summary

65

the contraction mapping methodology. Moreover, the efficiency of the proposed ILC schemes is verified by two numerical examples. The results in this chapter are mainly based on [6].

References 1. Chen Y, Wen C, Gong Z, Sun M (1999) An iterative learning controller with initial state learning. IEEE Trans Autom Control 44(2):371–375 2. Sun M, Wang D (2002) Iterative learning control with initial rectifying action. Automatica 38(7):1177–1182 3. Xu J-X, Yan R (2005) On initial conditions in iterative learning control. IEEE Trans Autom Control 50(9):1349–1354 4. Bolder J, Oomen T (2016) Inferential iterative learning control: a 2D-system approach. Automatica 71:247–253 5. Paszke W, Rogers E, Galkowski K (2016) Experimentally verified generalized KYP lemma based iterative learning control design. Control Eng Pract 53:57–67 6. Li X, Shen D (2017) Two novel iterative learning control schemes for systems with randomly varying trial lengths. Syst Control Lett 107:9–16

Chapter 5

Switching System Techniques for Linear Discrete-Time Systems

In this chapter, we proceed to a novel analysis technique for linear discrete-time systems, which is called the switching system technique. In this technique, the iteration evolution of the input error is formulated as a switching system. Then, the mean and covariance of the associated random matrices can be recursively computed along the iteration axis, which paves a novel way for convergence analysis. This chapter differs from the previous chapters in the following aspects: (1) no average operator is introduced and the conventional P-type update law is taken into account; (2) a switching system model of the iteration evolution is established; (3) no prior information is required on the probability distribution of randomly iteration-varying lengths; and (4) both almost sure and mean square convergence of the proposed algorithms are strictly proved. Noting that the extension from linear systems to nonlinear systems is not straightforward for this technique, the related results are still open for nonlinear systems.

5.1 Problem Formulation Consider the following linear discrete-time system: xk (t + 1) = At xk (t) + Bt u k (t), yk (t) = Ct xk (t),

(5.1)

where xk (t) ∈ Rn , u k (t) ∈ R p , and yk (t) ∈ Rq denote state, input, and output, respectively. The superscripts n, p, and q are the dimension of the state, input, and output, respectively. k = 0, 1, . . . denotes the iteration number, and t = 0, 1, . . . , N denotes the time index with N being the maximum of iteration lengths. At , Bt , and Ct are

© Springer Nature Singapore Pte Ltd. 2019 D. Shen and X. Li, Iterative Learning Control for Systems with Iteration-Varying Trial Lengths, https://doi.org/10.1007/978-981-13-6136-4_5

67

68

5 Switching System Techniques for Linear Discrete-Time Systems

time-varying system matrices with appropriate dimensions. It is assumed that Ct+1 Bt is of full-column rank, implying that the relative degree is 1 and q ≥ p. To facilitate the convergence analysis, we formulate the lifted framework of the systems similar to Chap. 2. In particular, we first assume the operation lengths of all iterations are identical to the maximum N , then the system (5.1) can be lifted as yk = Huk + y◦k , ⎡

where

⎢ ⎢ H=⎢ ⎣

C1 B0 C2 A1 B0 .. .

0 C2 B1 .. .

(5.2) ··· ··· .. .

0 0 .. .

⎤ ⎥ ⎥ ⎥ ⎦

(5.3)

C N A(N − 1, 1)B0 · · · · · · C N B N −1 with A( j, i)  A j A j−1 . . . Ai for j ≥ i and ⎡ ⎢ ⎢ yk = ⎢ ⎣

yk (1) yk (2) .. .

yk (N )

⎤

⎡

⎢ ⎥ ⎢ ⎥ ⎥ , uk = ⎢ ⎣ ⎦

u k (0) u k (1) .. .

⎤

⎡

⎢ ⎥ ⎥ ◦ ⎢ ⎥ , yk = ⎢ ⎣ ⎦

u k (N − 1)

C 1 A0 C1 A(1, 0) .. .

⎤ ⎥ ⎥ ⎥ xk (0). ⎦

C N A(N − 1, 0)

Denote the desired trajectory by yd (t), t = 0, 1, . . . , N . We assume the desired trajectory is realizable; that is, for yd (t), there exists a unique control input u d (t) and suitable initial state xd (0) such that yd = Hud + y◦d ,

(5.4)

where yd , ud , and y◦d are defined similar to yk , uk , and y◦k , respectively, by replacing yk (t), u k (t), and xk (0) with yd (t), u d (t), and xd (0) in the specific forms. To make the main idea of chapter easy to follow, we assume the identical initialization condition throughout this chapter, i.e., y◦k = y◦d . Clearly, under this assumption, the effect of the state initialization will be canceled in the learning update law. The conventional control objective is to design an ILC algorithm such that the desired trajectory yd can be asymptotically tracked as the iteration number k increases, i.e., yk → yd , provided that the iteration length is fixed as N . However, the actual iteration length may vary from iteration to iteration randomly due to safety consideration and other factors. In such case, it is rational to have a lower bound of actual iteration lengths, named by Nmin , Nmin < N , such that the actual iteration length varies in {Nmin , Nmin + 1, . . . , N } randomly (this is because the iteration length must be a nonnegative integer). In other words, for the first Nmin time instants the actual output can be always available, while for the remaining time instants the system output occurs randomly. Thus, there are N − Nmin + 1 possible output trajectories. For the kth iteration, it ends at the Nk th time instant; that

5.1 Problem Formulation

69

is, the actual iteration length is Nk . In other words, only the first Nk outputs can be received, Nmin ≤ Nk ≤ N . In order to make it clear, a cutting operator · Nk is introduced to yk which means the last N − Nk outputs of yk are removed, i.e., · Nk : R N q → R Nk q . Therefore, the control objective of this chapter is to design ILC update law such that  yk  Nk →  yd  Nk as k approaches to infinity. For notations concise, let h  N − Nmin + 1 throughout this chapter. Clearly, there are h = N − Nmin + 1 possible iteration lengths in total, which occur randomly in the repetition. In order to cope with these different scenarios, we provide a probability model of the random iteration lengths. In particular, let the probability that the iteration length is of Nmin , Nmin + 1, . . . , N be p1 , p2 , . . . , ph , respectively. That is, P(A Nmin ) = p1 , P(A Nmin +1 ) = p2 , . . . , P(A N ) = ph , ∀k, where Ai denotes the event that the iteration length is i, i.e., Yk  Nk = Yk i , Nmin ≤ i ≤ N . Obviously, pi > 0, 1 ≤ i ≤ h, and p1 + p2 + · · · + ph = 1.

(5.5)

It is worthy pointing out that no specific probability distribution is assumed on pi , thus the above expression on randomly varying iteration length is general. In other words, the design and analysis in this chapter do not depend on the specific information of the random iteration lengths. Remark 5.1 In Chap. 2, we introduce a sequence of random variables satisfying Bernoulli distribution based on p1 , . . . , ph to model the probability of the occurrence of the last N − Nmin outputs. Then, the random variable is multiplied to the corresponding tracking error to define a modified tracking error. The analysis objective becomes to show that the mathematical expectation of these N − Nmin modified tracking errors converges to zero. In this chapter, we prove the convergence both in almost sure sense and mean square sense straightforward according to the iteration length probabilities p1 , . . . , ph by direct calculations. In other words, this chapter provides a novel way for establishing a stronger convergence. Besides, it is worth pointing out that these probabilities are only used for the analysis, whereas the design condition of learning algorithm does not require any prior information on these probabilities, which is more suitable for practical applications. Remark 5.2 In many practical applications, the actual iteration length may be larger than the desired length N because of continuous operation. In this case, we may denote the available maximum iteration length by Nmax with Nmax > N . However, the redundant information at the time instants N < i ≤ Nmax cannot help to improve the tracking performance. In other words, these data are usually discarded in the algorithm design. As a consequence, this case can be regarded equivalent to the case that the iteration length N .

70

5 Switching System Techniques for Linear Discrete-Time Systems

5.2 ILC Design Note that, the iteration length cannot exceed the desired tracking length N , thus only two cases of tracking error need to be considered. The first is full iteration length case; that is, the iteration length equals the maximum length, whence the tracking error is a normal one with dimension N q. The second is uncompleted iteration length case; that is, the iteration length is shorter than the maximum length, whence the tracking error at the absent time instants are missing. This lost information cannot be used for updating the input signal. In such a case, we can append zeros to the absent time instants so that the tracking error is again transformed to be a normal one with dimension N q. In other words, when the kth actual output length is not up to the maximum N , namely, Nk < N , the tracking error is defined as  ek (t) =

yd (t) − yk (t), 1 ≤ t ≤ Nk , 0, Nk < t ≤ N .

(5.6)

Denote the lifted vector the tracking error by

T ek = ekT (1), . . . , ekT (N ) .

(5.7)

The control update law is thus defined as follows uk+1 = uk + Lek ,

(5.8)

where L is the learning gain matrix to be designed later. Noting that ek = yd − yk if Nk < N , we have to fill the gap by introducing the following matrix:  M Nk =

0 I N k ⊗ Iq , 0 0(N −Nk ) ⊗ Iq

Nmin ≤ Nk ≤ N ,

(5.9)

where Il and 0l denote unit matrix and zero matrix with dimension of l × l, respectively. Then we have ek = M Nk ( yd − yk ) = M Nk H(ud − uk ). Now, (5.8) leads to uk+1 = uk + Lek = uk + LM Nk H(ud − uk ). Subtracting both sides of last equation from ud , we have

5.2 ILC Design

71

ud − uk+1 = ud − uk − LM Nk H(ud − uk ) = (I − LM Nk H)(ud − uk ). That is, Δuk+1 = (I − LM Nk H)Δuk ,

(5.10)

where Δuk  ud − uk . Notice that Nk is a random variable taking values in the set {Nmin , . . . , N }; therefore, M Nk is a random matrix, which further leads to that I − LM Nk H is also a random matrix. Thus, one could introduce h binary random variables γi , 1 ≤ i ≤ h with h = N − Nmin + 1, such that γi ∈ {0, 1}, γ1 + γ2 + · · · + γh = 1, and P(γi = 1) = P(A Nmin −1+i ) = pi , 1 ≤ i ≤ h. Then (5.10) could be reformulated as Δuk+1 = [γ1 (I − LM Nmin H) + γ2 (I − LM Nmin +1 H) + · · · + γh (I − LM N H)]Δuk ,

(5.11)

where M N = I N q corresponds to the full-length case. Consequently, the zero convergence of the original update law (5.10) could be achieved by analyzing zero convergence of (5.11). Denote Ξi = I − LM Nmin −1+i H, 1 ≤ i ≤ h. Then (5.11) could be simplified as Δuk+1 = (γ1 Ξ1 + γ2 Ξ2 + · · · + γh Ξh )Δuk .

(5.12)

It is easy to see that all γi are dependent, since whenever one of them values 1, then all the others have to value 0. In order to give the design condition of learning gain matrix L, we first calculate the mean and covariance along the sample path. Let S = {Ξi , 1 ≤ i ≤ h}, and denote Z k = X k X k−1 . . . X 1 X 0 ,

(5.13)

where X k is a random matrix taking values in S with P(X k = Ξi ) = P(γi = 1) = pi , 1 ≤ i ≤ h, ∀k. Now, we have the following equation from (5.12): Δuk+1 = Z k Δu0 . The following two lemmas are given for further analysis.

(5.14)

72

5 Switching System Techniques for Linear Discrete-Time Systems

Lemma 5.1 Let S k = {Z k : taken over all sample paths}, then the mean of the S, denoted by K k , is given recursively by Kk =

h 

 pi Ξi

K k−1 .

(5.15)

i=1

Proof Let Sik = {Z k ∈ S k : X k = Ξi }, i = 1, 2, . . . , h. It is obvious that S k is the disjoint union of Sik . Recalling the definition of mathematical expectation, we have Kk =



P(Z k )Z k .

(5.16)

Z k ∈S k

By the independence of X j , one can decompose the above sum as follows Kk =



P(Z k )Z k

Z k ∈S k

=

h 



P(X k = Ξi )P(Z k−1 )Ξi Z k−1

Z k−1 ∈S k−1 i=1

=

 Z k−1

=

h 

h 

∈S k−1

P(γi = 1)P(Z k−1 )Ξi Z k−1

i=1

P(γi = 1)Ξi

h 

P(γi = 1)Ξi K k−1

i=1

=

h 

P(Z k−1 )Z k−1

Z k−1 ∈S k−1

i=1

=



pi Ξi K k−1 =

i=1

h 

 pi Ξi

K k−1 .

i=1

Thus the proof is completed. Lemma 5.2 Let S k = {Z k : taken over all sample paths}, then the covariance of the S, denoted by Rk , is given by Rk = Jk − K k K kT ,

(5.17)

where Jk is generated recursively as Jk =

h  i=1

pi Ξi Jk−1 ΞiT .

(5.18)

5.2 ILC Design

73

Proof The covariance is calculated as Rk =

 Zk

P(Z k )(Z k − K k )(Z k − K k )T .

∈S k

Then by decomposition, it leads to the following derivation: h    P(X k = Ξi )P(Z k−1 ) × (Ξi Z k−1 − K k )(Ξi Z k−1 − K k )T



Rk =

Z k−1 ∈S k−1 i=1 h    P(γi = 1)P(Z k−1 ) × (Ξi Z k−1 − K k )(Ξi Z k−1 − K k )T



=

Z k−1 ∈S k−1 i=1

⎡

=

h 



⎢ pi ⎣

i=1

Z k−1

∈S k−1



−

Z k−1

T ΞT − P(Z k−1 )Ξi Z k−1 Z k−1 i

Z k−1

P(Z k−1 )Ξi Z k−1 K kT +

∈S k−1

=



⎢ pi ⎣

 Z k−1

⎡

h 



T ΞT P(Z k−1 )K k Z k−1 i

∈S k−1

⎤

⎥ P(Z k−1 )K k K kT ⎦

∈S k−1

⎤

T ΞT − K KT ΞT − Ξ K T T⎥ P(Z k−1 )Ξi Z k−1 Z k−1 k k−1 i i k−1 K k + K k K k ⎦ . i

Z k−1 ∈S k−1

i=1

From Lemma 5.1, it is noticed that h 

T pi K k K k−1 ΞiT = K k K kT ,

i=1 h 

pi Ξi K k−1 K kT = K k K kT .

i=1

Thus we have Rk =

h  i=1

⎛ pi Ξi ⎝

 Z k−1

⎞ T ⎠ P(Z k−1 )Z k−1 Z k−1 ΞiT − K k K kT .

∈S k−1

On the other hand Rk = E(Z k − K k )(Z k − K k )T = EZ k Z kT − K k K kT  = P(Z k )Z k Z kT − K k K kT . Z k ∈S k

74

Let Jk =

5 Switching System Techniques for Linear Discrete-Time Systems

 Z k ∈S k

P(Z k )Z k Z kT , then it is obvious that Jk =

h 

pi Ξi Jk−1 ΞkT ,

i=1

by combing the last two expressions of Rk . This completes the proof. Now, let us return back to the iterative recursion (5.12) and find the suitable form of the learning gain matrix L. Notice that H is a block lower triangular matrix, and Mi is a block diagonal matrix, Nmin ≤ i ≤ N . Thus, there is a great degree of freedom on the selection of learning gain L. As a matter of fact, L could be partitioned as L = [L i, j ], 1 ≤ i, j ≤ N , where L i, j denotes the submatrix of p × q. Two types of L are considered as follows. • Arimoto-like gain: The diagonal blocks of L, i.e., L i,i , 1 ≤ i ≤ N , are valued, while the other blocks are set 0 p×q . • Causal gain: The blocks in the lower triangular part of L, i.e., L i, j , i ≥ j, are valued, while the other blocks are set 0 p×q . No matter which type of L mentioned above is adopted, it is clear that the matrix LM N K H is still a block lower triangular matrix whose diagonal blocks are L t,t Ct Bt−1 , 1 ≤ t ≤ Nk or 0 p , Nk + 1 ≤ t ≤ N . As a consequence, we can simply design L t,t such that (5.19) 0 < I − L t,t Ct Bt−1 < I. Remark 5.3 If the Arimoto-like gain is selected, we find that the update law (5.8) is formulated on each time instant as u k+1 (t) = u k (t) + L t+1,t+1 ek (t + 1). In other words, the update of the input signal is conducted for each time instant rather than for the whole iteration simultaneously. This treatment further reduces the computational burden brought by high dimension of the lifted model (5.2). However, on the other hand, the causal gain may offer more flexibilities for us, since no special condition is required on the block L i, j , i > j. Remark 5.4 Different from (5.19), the condition in Chap. 2 is that sup I − p(t)LC B ≤ θ where 0 ≤ θ < 1 and p(t) is the occurrence probability of output at t. Thus it is required in Chap. 2 that the probability of each iteration length is known. In this chapter, the requirement on prior knowledge of random trial lengths is removed to facilitate practical applications.

5.3 Strong Convergence Properties Based on Lemma 5.1, the following theorem establishes the convergence in mathematical expectation sense.

5.3 Strong Convergence Properties

75

Theorem 5.1 Consider system (5.2) with randomly iteration-varying length and apply control update law (5.8). The mathematical expectation of tracking error ek , i.e., Eek , converges to zero if the learning gain L satisfies (5.19). Proof By (5.14), we have EΔuk+1 = K k EΔu0 . Then, by recurrence of the mean K k , i.e., (5.15), it is obvious to have EΔuk =

h 

k pi Ξi

EΔu0 .

i=1

  h Thus, it is sufficient to show that ρ i=1 pi Ξi < 1, where ρ(·) denote the spectral radius of a matrix. By (5.19) we have the consequence that each eigenvalue of I − LM N K H, denoting as λ j (I − LM N K H), satisfies 0 < λ j (I − LM N K H) ≤ 1, 1 ≤ j ≤ N p, Nmin ≤ Nk ≤ N . An eigenvalue λ j (I − LM N K H) is equal to 1 if and only if there are some outputs missed, i.e., Nk < N . Note that, Ξi is a block lower triangular matrix, thus the eigenvalues actually are a collection of eigenvalues of all the diagonal blocks. Take the lth diagonal blocks from top to bottom of each alternative matrix Ξi into account. For the case 1 ≤ l ≤ Nmin , it is observed that all the eigenvalues of the lth diagonal block are positive and less than 1. While for the case Nmin + 1 ≤ l ≤ N , all the eigenvalues of the lth diagonal block are positive and not larger than 1. Meanwhile, not all eigenvalues of the lth diagonal blocks are equal to 1 because of the existence of Ξh = I − LM N H = I − LH, whose eigenvalues are all less than 1, Nmin + 1 ≤ l ≤ N . By noticing that  h i=1 pi = 1, it is obvious that, for any 1 ≤ j ≤ N p, 0
0, and then it leads to P(limk→∞ Δuk = 0) = 1. That is, Δuk converges to zero almost surely. Noting that ek = HΔuk  Nk ≤ HΔuk , the proof is completed. To show the mean square convergence, it is sufficient to show EΔuk ΔukT → 0. That is, Jk → 0, because we have already shown that K k → 0 in Theorem 5.1. It is first noted that the matrix Jk recursively defined in (5.18) is positive definite. Then, by the recursion (5.18), we have the following theorem. Theorem 5.3 Consider system (5.2) with randomly iteration-varying length and use control update law (5.8). The tracking error ek converges to zero in mean square sense if the learning gain L satisfies (5.19). Proof Following similar steps of the proof of Theorem 5.2, there is a suitable constant 0 < η < 1 such that h  pi Ξi 2 < η. 0< i=1

Then we have   h     Jk =  pi Ξi Jk−1 Ξi    i=1

≤

h 

pi Ξi Jk−1 Ξi

i=1

≤

h 

pi Ξi 2 Jk−1

i=1

=

h 

 pi Ξi

2

Jk−1

i=1

k then we define kn=m [item] = I . From Assumptions 6.3–6.4, there is no correlation among Δxk (0), Δu 0 (t), ωk (0 ≤ m ≤ t − 1), ω0≤m≤k−1 (t) and ν0≤m≤k−1 (t + 1). Moreover, these terms are uncorrelated with Δu k (0 ≤ m ≤ t − 1) and Δx0≤m≤k−1 (t). Because Δu k (0 ≤ m ≤ t − 1) and Δx0≤m≤k−1 (t) cannot be represented by each other, they are uncorrelated. Then, we have H12,k = 0. We can rewrite (6.13) as T 2 2 T −1 ¯ K k = θ(t)H 11,k M [(θ¯ (t) + σθ˜ (t) )M H11,k M + D1,k ] , k

(6.14)

T where M = C + B, D1,k = (θ¯ 2 (t) + σθ˜2 (t) )[(C + A)H22,k (C + A)T + C + Q 11,t C + + k Q 22,t+1 ]. Furthermore, from (6.9), we have T (t)] H11,k+1 = E[Δu k+1 (t)Δu k+1

=V3 H V3T + (σ 2˜

θk (t)

+ (θ¯ 2 (t) + σ 2˜

T T T ¯ ¯ + θ¯ 2 (t))K k V1 H V1T K kT − θ(t)K 3 H V1 K k k V1 H V3 − θ(t)V

)K k (C + Q 11,t C + + Q 22,t+1 )K kT T

θk (t)

T T ¯ ¯ =H11,k − θ(t)K k M H11,k − θ(t)H 11,k M K k

+ K k [(θ¯ 2 (t) + σ 2˜

θk (t)

)M H11,k M T + D1,k ]K kT

=(I − θ¯ (t)K k M)H11,k .

(6.15)

By denoting S = C + AH22,k (C + A)T + C + Q 11,t C + + Q 22,t+1 , it is clear that T

I − θ¯ (t)K k M = I − θ¯ 2 (t)H11,k M T (θ¯ 2 (t) + σθ˜2 (t) )−1 (M H11,k M T + S)−1 M. k

Noticing θ¯ 2 (t)/(θ¯ 2 (t) + σθ˜2 (t) ) = θ¯ (t) and applying (M H11,k M T + S)−1 = k

−1 −1 ) M T S −1 , we can rewrite the above equation as S −1 − S −1 M(M T S −1 M + H11,k T −1 ¯ ¯ M I − θ(t)K k M =I − θ (t)H11,k M S −1 −1 T −1 ¯ ) M T S −1 M. + θ (t)H11,k M S M(M T S −1 M + H11,k

Denoting G = M T S −1 M and W = H11,k G leads to −1 −1 ¯ I − θ¯ (t)K k M =I − θ¯ (t)H11,k G + θ(t)H 11,k G(G + H11,k ) G ¯ =I − θ¯ (t)W + θ(t)W (I + W −1 )−1

=((1 − θ¯ (t))I + W −1 )(I + W −1 )−1 .

(6.16)

Note that, S is a positive-definite matrix and C + B is full-column rank, then G is positive definite. According to Assumption 6.4, H11,k is positive definite. Moreover,

6.3 Convergence Analysis

89

W and W −1 have positive real eigenvalues since W is a product of two positivedefinite matrices. Lemma 6.1 The spectral radius of ((1 − θ¯ (t))I + W −1 )(I + W −1 )−1 is less than 1. Proof Noting the formulation of W , it is clear that W −1 is diagonalizable [1]. Let V be an eigenvector matrix of (1 − θ¯ (t))I + W −1 and Λ = diag{αi } the associated diagonal eigenvalue matrix with αi being the real eigenvalues. V ((1 − θ¯ (t))I + W −1 )V −1 = Λ, V (I + W −1 )V −1 = diag{αi + θ¯ (t)}, V (I + W −1 )−1 V −1 = diag{αi + θ¯ (t)}−1 . Then, we obtain I − θ¯ (t)K k M =((1 − θ¯ (t))I + W −1 )(I + W −1 )−1 =V diag{αi }V −1 V diag{αi + θ¯ (t)}−1 V −1   αi V −1 . =V diag (αi + θ¯ (t)) Noticing that all eigenvalues of W are positive real numbers (Theorem 7.6.3 of [1]) ¯ and (1 − θ(t))I is a positive-definite diagonal matrix, it is clear that all eigenvalues ¯ of (1 − θ (t))I + W −1 are positive. That is, αi > 0. Therefore, the spectral radius of ((1 − θ¯ (t))I + W −1 )(I + W −1 )−1 is less than 1. Theorem 6.1 Assume that Assumptions 6.1–6.5 hold for system (6.1) and apply the learning algorithm presented by (6.3), (6.13) and (6.15), then H11,k → 0 and ⎡ H22,k → ⎣

t−1 

⎡

⎤T

A Tj ⎦ E[Δxk (0)ΔxkT (0)] ⎣

j=0

+

 t−2 t−1  

m=0

⎤ A Tj ⎦

j=0

T T An+1

t−1 

Q 11,m

n=m

 t−2 t−1   m=0

 T An+1

n=m

as k → ∞. ¯ Proof According to the above analysis, the spectral radius of I − θ(t)K k M is less than 1. Thus, it is easy to obtain limk→∞ H11,k = limk→∞ E[Δu k (t)Δu kT (t)] = 0 from (6.15). Moreover, ⎡ Δxk (t) = ⎣

t−1  j=0

⎤T A Tj ⎦

Δxk (0) +

 t−2 t−1   m=0

n=m

T T An+1

[Bm Δu k (m) − ωk (m)]

90

6 Two-Dimensional Techniques for Linear Discrete-Time Systems

Noticing that xd (0) − xk (0) is uncorrelated with u d (t) − u 0 (t), ωk (0) and νk (0) in Assumption 6.4, we obtain ⎡ E[Δxk (t)ΔxkT (t)] = ⎣

t−1 

A Tj ⎦ E[Δxk (0)ΔxkT (0)] ⎣

j=0

+

⎡

⎤T

t−1 

⎡ ⎣

T ⎦ E {[B Δu (m) − ω (m)] An+1 m k k

n=m

m=0

t−1 

[Bm Δu k (m) − ωk (m)]T ⎡ =⎣

t−1 

+

⎣

m=0

⎤T

t−1  m=0

⎡ ⎣

⎤T

t−2 

T ⎦ An+1

⎡





⎣

0≤i= j≤t−1

T ⎦ An+1

n=m

⎡

⎤

t−1 

A Tj ⎦

T Bm E[Δu k (m)Δu kT (m)]Bm

t−1  m=0

t−2 

⎤

t−2 

j=0

n=m

+E[ωk (m)ωkT (m)]

+

⎡

A Tj ⎦ E[Δxk (0)ΔxkT (0)] ⎣

j=0

A Tj ⎦

j=0

⎤T

t−2 

⎤

t−1 

⎡ ⎣

t−2 

⎤ T ⎦ An+1

n=m

⎤T

⎡

T ⎦ B E[Δu (i)Δu T ( j)]B ⎣ An+1 i k j k

n=i

t−2 

⎤ T ⎦. An+1

n= j

By applying the property limk→∞ E[Δu k (m)Δu kT (m)] = 0 and limk→∞ E[Δu k (i) Δu kT ( j)] = 0 (as a consequence of the previous property), it leads to ⎡ E[Δxk (t)ΔxkT (t)] = ⎣

t−1 

⎡

⎤T

A Tj ⎦ E[Δxk (0)ΔxkT (0)] ⎣

j=0

+

 t−2 t−1  

m=0

n=m

⎤ A Tj ⎦

j=0

T T An+1

t−1 

Q 11,m

 t−2 t−1   m=0

 T An+1

.

n=m

The proof is thus completed. The specific algorithm is given as follows. Remark 6.3 In Theorem 6.1, we know that H22,k converges to a fixed value as the iteration number increases. If there is no initial resetting error and stochastic system noises at every iteration, then we have Q 11,t → 0 and H22,k → 0 as k → ∞. Moreover, from (6.8), we have H22,k (t + 1) = Bt H11,k (t)BtT + At H22,k (t)AtT + Q 11,t .

(6.17)

6.3 Convergence Analysis

91

Algorithm 1 The procedure of the proposed scheme with optimal learning gain matrix 1: The initial input error covariance H11,0 (t) is set as s I with s > 0, ∀t and the initial state error covariance is set as H22,k (0) = diag{0, 0, 0}, ∀k; 2: using Eq. (6.13), compute learning gain K k ; 3: using Eq. (6.17), compute H22,k (t + 1); 4: using Eq. (6.3), update the control u k+1 (t); 5: using Eq. (6.15), update H11,k+1 (t); 6: k = k + 1, repeat the whole process.

Remark 6.4 For the kth iteration, if the actual iteration length is equal to the maxi¯ = 1, mum of iteration lengths, i.e., Nk = N , then we have θ¯ (t) = 1, ∀t. When θ(t) ¯ the spectral radius of I − θ (t)K k M is smallest. Roughly speaking, the fastest convergence speed is achieved. When considering randomly varying trial lengths, the actual iteration length can still achieve the maximum N with a positive probability, ¯ thus θ¯ (t) = 0 and the spectral radius of I − θ(t)K k M is smaller than 1, ∀t. Thus, the convergence is guaranteed by this inherent mechanism.

6.4 Alternative Scheme with Distribution Estimation In this section, we consider the case that the distribution pm is unknown and we provide a recursive-estimation-based algorithm. Because θk (t) satisfies Bernoulli distribution, we can divide θk (t) = θ¯ (t) + θ˜k (t), where θ˜k (t) is a scalar zero-mean ¯ ¯ θ(t). Due to the condition that random variable with variance σθ˜2 (t) = (1 − θ(t)) k ¯ The estimation of θ¯ (t) is defined as pm is unknown, we need to estimate θ(t). k θi (t). Then, denote θ˜k (t) = θ¯ (t) − θˆ¯k (t). Clearly, θ˜k (t) is a scalar θˆ¯k (t) = k1 i=1 ¯ ¯ θ(t). However, this zero-mean random variable with variance σ 2 = (1 − θ(t)) θ˜k (t)

information cannot be used in the algorithm because θ¯ (t) is unavailable. To this end, we give its estimation σˆ θ˜2 (t) = (1 − θˆ¯k (t))θˆ¯k (t). Now, we can present the “modk ified” or suboptimal learning control algorithm and its convergence analysis in the rest of this section. Denote Kˆ k and Hˆ as the counterparts of K k and H for the modified algorithm. Consider the “modified” learning gain matrix given by ∗ M T Ξ ∗ −1 , Kˆ k = θˆ¯k (t) Hˆ 11,k

(6.18)

T ∗ ∗ M T + (C + A) Hˆ 22,k (C + A)T + C + Q 11,t C + + where Ξ ∗ = (σˆ θ˜2 (t) + θˆ¯k2 (t))(M Hˆ 11,k k

∗ ∗ and Hˆ 22,k are defined later in (6.23) and (6.26), respectively. Q 22,t+1 ), Hˆ 11,k

92

6 Two-Dimensional Techniques for Linear Discrete-Time Systems

Then, we can rewrite (6.3) and (6.9) as u k+1 (t) = u k (t) + θk (t) Kˆ k (t)ek (t + 1), 

    ¯ Kˆ k (t)C(t + 1)B(t) −θ(t) ¯ Kˆ k (t)C(t + 1)A(t) Δuˆ k (t) Δuˆ k+1 (t) I − θ(t) = Δxˆk (t + 1) Δxˆk (t) B(t) A(t)    −θ˜k (t) Kˆ k (t)C(t + 1)B(t) −θ˜k (t) Kˆ k (t)C(t + 1)A(t) Δuˆ k (t) + Δxˆk (t) 0 0    ωk (t) θ˜ (t) Kˆ k (t)C(t + 1) θ˜k (t) Kˆ k (t) + k νk (t + 1) 0 0    ωk (t) θ¯ (t) Kˆ k (t)C(t + 1) θ¯ (t) Kˆ k (t) + . νk (t + 1) −I 0

(6.19)

(6.20)

Let   Hˆ 11,k Hˆ 12,k T ˆ ˆ ˆ , H = E[ X X ] = ˆ T ˆ H12,k H22,k where Hˆ 11,k = E[Δuˆ k (t)Δuˆ kT (t)], Hˆ 12,k = E[Δuˆ k (t)ΔxˆkT (t)], Hˆ 22,k = E[Δxˆk (t) ΔxˆkT (t)]. We have Hˆ 12,k = 0 using the same method given in Sect. 6.3. Then, we can rewrite (6.18) as ∗ ∗ ∗ −1 M T [(σˆ θ˜2 (t) + θˆ¯k2 (t))M Hˆ 11,k M T + D1,k ] , Kˆ k = θ¯ˆk (t) Hˆ 11,k

(6.21)

k

∗ ∗ where M = C + B and D1,k = (σˆ θ˜2 (t) + θˆ¯k2 (t))[(C + A) Hˆ 22,k (C + A)T + C + Q 11,t C + T k + Q 22,t+1 ]. Furthermore, from (6.20), we obtain T (t)] Hˆ 11,k+1 = E[Δuˆ k+1 (t)Δuˆ k+1

= Hˆ 11,k − θ¯ (t) Kˆ k M Hˆ 11,k − θ¯ (t) Hˆ 11,k M T Kˆ kT + Kˆ k [(θ¯ 2 (t) + σθ˜2 (t) )M Hˆ 11,k M T + D1,k ] Kˆ kT . k

(6.22)

∗ . Moreover, assume Accordingly, we denote the estimation of Hˆ 11,k by Hˆ 11,k = Hˆ 11,0 without loss of generality. Similar to the derivations in previous sections, we have ∗ Hˆ 11,0

6.4 Alternative Scheme with Distribution Estimation

93

∗ ∗ ∗ ∗ = Hˆ 11,k − θˆ¯k (t) Kˆ k M Hˆ 11,k − θˆ¯k (t) Hˆ 11,k M T Kˆ kT Hˆ 11,k+1 ∗ ∗ + Kˆ k [(σˆ θ˜2 (t) + θˆ¯k2 (t))M Hˆ 11,k M T + D1,k ] Kˆ kT k

∗ =(I − θˆ¯k (t) Kˆ k M) Hˆ 11,k .

(6.23)

Note that, θ¯ 2 (t) + σθ˜2 (t) = θ¯ (t) and σˆ θ˜2 (t) + θˆ¯k2 (t) = θˆ¯k (t). Thus, (6.22) yields k

k

T (t)] Hˆ 11,k+1 = E[Δuˆ k+1 (t)Δuˆ k+1

¯ Kˆ k M Hˆ 11,k − θ(t) ¯ Hˆ 11,k M T Kˆ kT = Hˆ 11,k − θ(t) + Kˆ k (θ¯ (t)M Hˆ 11,k M T + D1,k ) Kˆ kT = (I − θ¯ (t) Kˆ k M) Hˆ 11,k ¯ ¯ Hˆ 11,k M T ] Kˆ kT . + [ Kˆ k (θ(t)M Hˆ 11,k M T + D1,k ) − θ(t)

(6.24)

Note that, θi (t), i = 1, 2, . . . , k, is a sequence of independent and identically distributed random variables with expectation being E(θi (t)) = θ¯ (t), then we have P

ˆ ¯ ¯ lim θk (t) = θ (t) = 1

k→∞

by the strong law of large numbers in probability theory. Therefore, we have that ∗ −→ Hˆ 11,k as k → ∞, Hˆ 11,k

Kˆ k (θ¯ (t)M Hˆ 11,k M T + D1,k ) −→ θ¯ (t) Hˆ 11,k M T as k → ∞. In other words, the second term on the right-hand side of (6.24) tends to 0 as k → ∞. ∗ (C + A)T + C + Q 11,t C + T + Q 22,t+1 , G ∗ = Moreover, denote S ∗ = C + A Hˆ 22,k ∗ G ∗ . Following similar steps of Sect. 6.3, the first term M T S ∗ −1 M and W ∗ = Hˆ 11,k on the right-hand side of (6.24) can be modified as (I − θ¯ (t) Kˆ k M) = ((1 − θ¯ (t))I + W ∗ −1 )(I + W ∗ −1 )−1 as k → ∞. ∗ are positive-definite matrices, while C + B is of full-column Note that, S ∗ and Hˆ 11,k ∗ rank. Then, G is positive definite. The matrices W ∗ and W ∗ −1 have positive real eigenvalues since W ∗ is a product of two positive-definite matrices. Lemma 6.2 The spectral radius of I − θ¯ (t) Kˆ k M is less than 1.

Proof The proof is completely the same as that of Lemma 6.1 and thus are omitted for saving space. By Lemma 6.2, the first term on the right-hand side of (6.24) ensures a contraction mapping. The convergence property of the modified algorithm with asymptotical estimation is presented in the following theorem.

94

6 Two-Dimensional Techniques for Linear Discrete-Time Systems

Theorem 6.2 Assume that Assumptions 6.1–6.4 hold for system (6.1) and apply the learning algorithm presented by (6.18), (6.19) and (6.24), then Hˆ 11,k → 0 and ⎡ Hˆ 22,k → ⎣

t−1 

⎤T

A T ( j)⎦ E[Δxk (0)ΔxkT (0)] ⎣

j=0

+

⎡

t−1 

 t−2 

m=0

n=m

⎤ A T ( j)⎦

j=0

T A T (n + 1)

t−1 

Q 11,m

t−1 

 t−2 

m=0

n=m

 A T (n + 1)

as k → ∞. Proof The proof of this theorem can be conducted by following the similar steps to that of Theorem 6.1, where the difference between these (6.15) and (6.24) is that an additional term (i.e., the second term on the right-hand side of (6.24)) appears in (6.24). However, this term tends to zero as k goes to infinity because the estimation θˆ¯k (t) converges to the precise expectation asymptotically. Lemma 6.2 guarantees a contraction mapping, which further implies the asymptotical convergence to zero of (6.24). The rest of the proof can be completed similar to that of Theorem 6.1. The proof is completed. Moreover, from (6.20), we have Hˆ 22,k (t + 1) = B Hˆ 11,k (t)B T + A Hˆ 22,k (t)A T + Q 11,t .

(6.25)

However, both Hˆ 11,k (t) and Hˆ 22,k (t) are unavailable for computation. To this end, ∗ ∗ we denote the estimation of Hˆ 22,k as Hˆ 22,k . Moreover, assume Hˆ 22,k (0) = Hˆ 22,k (0) ∗ ˆ without loss of generality. The recursive estimation of H22,k is defined as ∗ ∗ ∗ (t + 1) = B Hˆ 11,k (t)B T + A Hˆ 22,k (t)A T + Q 11,t . Hˆ 22,k

(6.26)

The specific algorithm is given as follows. Algorithm 2 The procedure of the estimation-based learning algorithm 1: The initial input error covariance H11,0 (t) is set as s I with s > 0, ∀t and the initial state error covariance is set as Hˆ 22,k (0) = diag{0, 0, 0}, ∀k; 2: using Eq. (6.18), compute “modified” learning gain Kˆ k ; ∗ 3: using Eq. (6.26), compute Hˆ 22,k (t + 1); 4: using Eq. (6.19), update the control u k+1 (t); ∗ 5: using Eq. (6.23), update Hˆ 11,k+1 (t); 6: k = k + 1, repeat the whole process.

6.5 Illustrative Simulations

95

6.5 Illustrative Simulations In this section, a numerical multi-input-multi-output system model is presented to illustrate the effectiveness of the proposed algorithm of Theorem 6.1 and 6.2. Then, a permanent magnet linear motor (PMLM) model is also simulated to demonstrate the effectiveness for practical systems. Example 6.1 (MIMO Systems) Consider a time-varying linear system (At , Bt , Ct ), where the system matrices are given as follows ⎡

⎤ 0.05 sin(0.2t) −0.2 0.02t 0.1 −0.01t −0.02 cos(0.5t) ⎦ , At = ⎣ 0.1 0.1 0.2 + 0.05 cos(0.2t) ⎡ ⎤ 2 1 − 0.2 sin (0.5tπ ) 0 ⎦, 0.01t 0.01t Bt = ⎣ 0 1 + 0.1 sin(0.5tπ )   −0.1 0.2 + 0.1 sin2 (0.5tπ ) 0.1 . Ct = 0 0.1 0.2 − 0.1 sin(0.5tπ ) The desired trajectory is described as follows  yd (t) =

 ) sin( 2tπ 50 , t ∈ Z50 . ) − sin( tπ 20

The stochastic system noise satisfies ωk (t) ∼ N (0, 0.022 ) and the measurement noise satisfies νk (t) ∼ N (0, 0.012 ). In addition, the maximum of iteration length is N = 50. The iteration length varies from 35 to 50 satisfying discrete uniform distribution, which is carried out by using the MATLAB command “randi.” Moreover, we have P(Nk = m) = 1/16, where m ∈ {35, 36, . . . , 50}. The algorithm runs for 40 iterations for Algorithms 1 and 2. The first and second dimension of the system output yk (t) at the 30th and 40th iterations and the desired trajectory are shown in Figs. 6.1 and 6.3 for Algorithms 1 and 2, respectively. As can be seen, y30 (t) and y40 (t) coincide with the desired trajectory perfectly, which clearly demonstrates the effectiveness of the proposed algorithms. Moreover, the actual trial lengths for the 30th and 40th iterations are less than 50 (the desired trial length), which shows the nonuniform trial length problem. In addition, Figs. 6.2 and 6.4 demonstrate a fact that the evolution of avgt∈[0,50] ρ(H11,k (t)) tends to 0 as k → ∞, where avgt∈[0,50] ρ(H11,k (t)) denotes the average value of the spectral radius for each iteration. All the results have verified theoretical conclusions. Example 6.2 (PMLM Model) The discretized model of permanent magnet linear motor (PMLM) is given as follows [2]

96

6 Two-Dimensional Techniques for Linear Discrete-Time Systems yd(t)

trajectories (first output)

1

y30(t) 0

y40(t)

−1 0

10

20

30

40

50

40

50

time axis trajectories (second output)

1

yd(t)

0

y30(t) y40(t)

−1 0

10

20

30

time axis

Fig. 6.1 The desired trajectory and tracking profiles for 30th and 40th iterations (Algorithm 1) 1

10

avg

t∈[0,50]

ρ(H

11,k

)

0

value

10

−1

10

−2

10

0

5

10

15

20

25

30

35

iterations

Fig. 6.2 The evolution of avgt∈[0,50] ρ(H11,k (t)) with respect to k (Algorithm 1)

40

6.5 Illustrative Simulations

97 y (t)

trajectories (first output)

1

d

y (t) 30

0

y (t) 40

−1 0

10

20

30

40

50

40

50

time axis trajectories (second output)

1

yd(t)

0

y30(t) y40(t)

−1 0

10

20

30

time axis

Fig. 6.3 The desired trajectory and tracking profiles for 30th and 40th iterations (Algorithm 2) 1

10

value

avg

t∈[0,50]

ρ(H ) 11,k

0

10

−1

10

0

5

10

15

20

25

30

35

iterations

Fig. 6.4 The evolution of avgt∈[0,50] ρ( Hˆ 11,k (t)) with respect to k (Algorithm 2)

40

98

6 Two-Dimensional Techniques for Linear Discrete-Time Systems 1.5

y (t) d

y30(t)

1

y (t) 40

trajectories

0.5

0

−0.5

−1

−1.5

0

10

20

30

40

50

time axis

Fig. 6.5 The desired trajectory and tracking profiles for 30th and 40th iterations (Algorithm 1)

⎧ ⎪ ⎨x(t + 1) = x(t) + v(t)Δ 2 k1 k2 ψ k2 ψ f v(t + 1) = v(t) − Δ Rm f v(t) + Δ Rm u(t) ⎪ ⎩ y(t) = v(t) where x and v denote the motor position and rotor velocity, respectively. Δ = 0.01s, R = 8.6, m = 1.635 kg, and ψ f = 0.35 Wb are the sampling period, the resistance of stator, the rotor mass, and the flux linkage, respectively. k1 = π/τ and k2 = 1.5π/τ with τ = 0.031 m are the pole pitches. In addition, the desired trajectory is given by yd (t) = − sin(0.05tπ ) + 0.5 − 0.5 cos(0.02t), 0 ≤ t ≤ 0.5. The maximum of iteration length is N = 50. The iteration length varies from 35 to 50 satisfying discrete uniform distribution. For simplicity, we applying the proposed approach (Algorithm 1) to the discretized model of PMLM in this section. The system output yk (t) at the 30th and 40th iterations and the desired trajectory are shown in Fig. 6.5. Clearly, the output at the 30th and 40th iterations almost coincide with the desired reference, which implies a good tracking performance of the proposed algorithm. Moreover, the actual trial lengths for the 30th and 40th iterations are observed to be less than the desired trial length. In addition, Fig. 6.6 demonstrates the evolution of avgt∈[0,50] ρ(H11,k (t)) along the iteration axis, which tends to 0 as k → ∞. These figures demonstrate the effectiveness of the proposed learning algorithm under randomly varying trial lengths.

6.6 Summary

99 1

10

avg

t∈[0,50]

ρ(H

11,k

)

0

value

10

−1

10

−2

10

0

5

10

15

20

25

30

35

40

iterations

Fig. 6.6 The evolution of avgt∈[0,50] ρ(H11,k (t)) with respect to k (Algorithm 1)

6.6 Summary This chapter presents a 2D approach for designing and analyzing ILC for linear stochastic systems with randomly varying trial lengths. Two algorithms are proposed for the cases with and without prior distribution information. The first algorithm is established by optimizing the trace of input error covariance matrix with respect to the learning gain matrix. It leads to an optimal design of the learning gain matrix provided that the prior distribution information of varying trial lengths is available. The second algorithm removes the requirement on trial length distribution by adding an asymptotical estimation mechanism. The convergence properties of both algorithms are carefully calculated. The results in this chapter are mainly based on [3].

References 1. Horn RA, Johnson CR (1985) Matrix analysis. Cambridge University Press, New York 2. Zhou W, Yu M, Huang D (2015) A high-order internal model based iterative learning control scheme for discrete linear time-varying systems. Int J Autom Comput 12(3):330–336 3. Liu C, Shen D, Wang J (2018) A two-dimensional approach to iterative learning control with randomly varying trial lengths. J Syst Sci Complex

Part II

Nonlinear Systems

Chapter 7

Moving Averaging Techniques for Nonlinear Continuous-Time Systems

In this chapter, we will extend the idea on ILC design with randomly varying trial lengths to nonlinear dynamic systems. Different from Chaps. 2 and 3, this chapter will employ an iteratively moving average operator into the ILC scheme. The main contributions of this chapter can be summarized as: (i) A new formulation is presented for continuous-time nonlinear dynamic systems with randomly varying trial lengths, where the trial lengths satisfy a continuous probability distribution; (ii) Different from the previous chapters that consider linear systems, ILC for nonlinear affine and nonaffine dynamic systems with nonuniform trial lengths is investigated; (iii) Instead of using the iteration-average operator that includes all the past tracking information as in Chaps. 2 and 3, an iteratively moving average operator that incorporates the most recent few trials are introduced. With the ILC convergence, it is clear that, the latest trials could provide more accurate control information than those “older” trials.

7.1 Problem Formulation Consider a nonlinear dynamical system x˙k (t) = f (xk (t), t) + bu k (t), yk (t) = cT xk (t),

(7.1)

where k ∈ N and t ∈ [0, Tk ] denote the iteration index and time, respectively. Meanwhile, xk (t) ∈ Rn , u k (t) ∈ R, and yk (t) ∈ R denote state, input, and output of the system (7.1), respectively. f (x, t) is Lipschitz continuous with respect to x, i.e.,  f (x1 , t) − f (x2 , t) ≤ f 0 x1 − x2 . Further, b ∈ Rn and c ∈ Rn are constant vectors, and cT b = 0. Let yd (t) ∈ R, t ∈ [0, Td ] be the desired output trajectory. Assume that, for any realizable output trajectory yd (t), there exists a unique control input u d (t) ∈ R such that © Springer Nature Singapore Pte Ltd. 2019 D. Shen and X. Li, Iterative Learning Control for Systems with Iteration-Varying Trial Lengths, https://doi.org/10.1007/978-981-13-6136-4_7

103

104

7 Moving Averaging Techniques for Nonlinear Continuous-Time Systems

x˙d (t) = f (xd (t), t) + bu d (t), yd (t) =cT xd (t),

(7.2)

where u d (t) is uniformly bounded for all t ∈ [0, Td ]. The main difficulty in designing ILC scheme for the system (7.1) is that the actual trial length Tk is iteration-varying and may be different from the desired trial length Td . Before addressing the ILC design problem with nonuniform trial lengths, let us give some notations and assumptions that would be useful in the derivation of our main result. Definition 7.1 E{η} stands for the expectation of the stochastic variable η. P[η ≤ t] means the occurrence probability of the event η ≤ t with a given t. Assumption 7.1 Assume that Tk is a stochastic variable and its probability distribution function is ⎧ t ∈ [0, Td − N1 ), ⎨ 0, FTk (t)  P[Tk ≤ t] = p(t), t ∈ [Td − N1 , Td + N2 ], (7.3) ⎩ 1, t > Td + N2 , where 0 ≤ p(t) ≤ 1 is a known function, and 0 ≤ N1 < Td and N2 ≥ 0 are two given constants. Assumption 7.2 xk (0) = xd (0). If the control process (7.1) repeats with the same trial length Td , namely, Tk = Td , and under the identical initial condition, a simple and effective ILC for system (7.1) is u k+1 (t) = u k (t) + L e˙k (t),

(7.4)

where ek (t)  yd (t) − yk (t) is the tracking error on the interval [0, Td ], e˙k (t) is the derivative of ek (t) on [0, Td ], and L ∈ R is the learning gain. However, when the trial length Tk is iteration-varying, which corresponds to a nonstandard ILC process, the learning control scheme (7.4) has to be redesigned.

7.2 ILC Design and Convergence Analysis In this section, based on the assumptions and notations that are given in Sect. 7.1, ILC design and convergence analysis are addressed, respectively. In practice, for one scenario that the ith trial ends before the desired trial length, namely, Tk < Td , both the output yk (t) and the derivative of tracking error e˙k (t) on the time interval (Tk , Td ] are missing, which thus cannot be used for learning. For the other scenario that the ith trial is still running after the time instant we want it to

7.2 ILC Design and Convergence Analysis

105

stop, i.e., Tk > Td , the signals yk (t) and e˙k (t) after the time instant Td are redundant and useless for learning. In order to cope with those missing signals or redundant signals in different scenarios, a sequence of stochastic variables satisfying Bernoulli distribution is defined. By using those stochastic variables, a newly defined tracking error e˙k∗ (t) is introduced to facilitate the modified ILC design. The main procedure for deriving a modified ILC scheme can be described as follows: (1) Define a stochastic variable γk (t) at the kth iteration. Let γk (t), t ∈ [0, Td + N2 ] be a stochastic variable satisfying Bernoulli distribution and taking binary values 0 and 1. The relationship γk (t) = 1 represents the event that the control process (7.1) can continue to the time instant t in the kth iteration, which occurs with a probability of q(t), where 0 < q(t) ≤ 1 is a prespecified function of time t. The relationship γk (t) = 0 denotes the event that the control process (7.1) cannot continue to the time instant t in the kth iteration, which occurs with a probability of 1 − q(t). (2) Compute the probability P[γk (t) = 1]. Since the control process (7.1) will not stop within the time interval [0, Td − N1 ), the event that γk (t) = 1 surely occurs when t ∈ [0, Td − N1 ), which implies that q(t) = 1, ∀t ∈ [0, Td − N1 ). While for the scenario of t ∈ [Td − N1 , Td + N2 ], the event γk (t) = 1 corresponds to the statement that the control process (7.1) stops at or after the time instant t, which means that Tk ≥ t. Thus, P[γk (t) = 1] = P[Tk ≥ t] = 1 − P[Tk < t] = 1 − P[Tk < t] − P[Tk = t] = 1 − P[Tk ≤ t] = 1 − FTk (t),

(7.5)

where P[Tk = t] = 0 is applied. Thus, it follows that ⎧ t ∈ [0, Td − N1 ), ⎨ 1, q(t) = 1 − FTk (t) = 1 − p(t), t ∈ [Td − N1 , Td + N2 ], ⎩ 0, t > Td + N2

(7.6)

Since γk (t) satisfies Bernoulli distribution, the expectation E{γk (t)} = 1 · q(t) + 0 · (1 − q(t)) = q(t). (3) Define a modified tracking error. Denote ek∗ (t)  γk (t)ek (t), t ∈ [0, Td ]

(7.7)

as a modified tracking error, and e˙k∗ (t)  γk (t)e˙k (t), t ∈ [0, Td ],

(7.8)

106

7 Moving Averaging Techniques for Nonlinear Continuous-Time Systems

which renders to e˙k∗ (t) =



e˙k (t), t ∈ [0, Tk ], 0, t ∈ (Tk , Td ],

(7.9)

when Tk < Td , and e˙k∗ (t) = e˙k (t), t ∈ [0, Td ],

(7.10)

when Tk ≥ Td . (4) The ILC scheme. Different from the previous chapters, an iteratively moving average operator is introduced, 1  f k− j (·), m + 1 j=0 m

A{ f k (·)} 

(7.11)

for a sequence f k−m (·), f k−m+1 (·), . . . , f k (·) with m ≥ 1 being the size of the moving window, which includes only the last m + 1 trials since the recent trials could provide more accurate control information for learning. The ILC scheme is given as follows u k+1 (t) = A{u k (t)} +

m 

∗ β j e˙k− j (t), t ∈ [0, Td ],

(7.12)

j=0

for all k ∈ N, where the learning gains β j ∈ R, j = 0, 1, . . . , m, will be determined in the following and u −1 (t) = u −2 (t) = · · · = u −m (t) = 0. The following theorem presents the first main result of this chapter. Theorem 7.1 For the nonlinear system (7.1) and the ILC scheme (7.12), choose the learning gains β j , j = 0, 1, 2, . . . , m, such that, for any constant 0 ≤ ρ < 1, m 

η j ≤ ρ,

(7.13)

j=0

where     1  1 − q(t) T   − β j c b q(t) + , η j  sup  m+1 m+1 t∈[0,Td ] then the tracking error ek (t), t ∈ [0, Tk ], will converge to zero asymptotically as k → ∞. Remark 7.1 In practice, the probability distribution of the trial length Tk could be estimated in advance based on previous multiple experiments or by experience. In

7.2 ILC Design and Convergence Analysis

107

consequence, the probability distribution function FTk (t) in Assumption 7.1 is known. Thus, q(t) is available for controller design and can be calculated by (7.6). Proof. Denote Δu k (t)  u d (t) − u k (t) and Δxk (t)  xd (t) − xk (t) the input and state errors, respectively, then there has e˙k (t) = y˙d (t) − y˙k (t) = cT (x˙d (t) − x˙k (t)) = cT ( f (xd (t), t) − f (xk (t), t)) + cT bΔu k (t).

(7.14)

From (7.12), the following relationship can be obtained: Δu k+1 (t) = A{Δu k (t)} −

m 

∗ β j e˙k− j (t)

j=0

= A{Δu k (t)} −

m 

β j γk− j (t)e˙k− j (t)

j=0

= A{Δu k (t)} − cT b

m 

γk− j (t)β j Δu k− j (t)

j=0 m 

−cT =

m  j=0

1 − γk− j (t)β j cT b Δu k− j (t) m+1

m 

−cT

γk− j (t)β j [ f (xd , t) − f (xk− j , t)]

j=0

γk− j (t)β j [ f (xd , t) − f (xk− j , t)].

(7.15)

j=0

Taking norm on both sides of (7.15) yields |Δu k+1 (t)| ≤

m 

1 j=0 m+1

+c f 0

 − γk− j (t)β j cT b |Δu k− j (t)|

m

j=0

γk− j (t)|β j |Δxk− j ,

(7.16)

where c ≥ cT . According to Assumption 7.2, the system (7.1) can be rewritten as Δxk (t) =

t

[ f (xd (τ ), τ ) − f (xk (τ ), τ ) + bΔu k (τ )]dτ,

(7.17)

0

then Δxk (t) ≤ f 0 0

t



t

Δxk (τ )dt + b 0

|Δu k (τ )|dτ,

(7.18)

108

7 Moving Averaging Techniques for Nonlinear Continuous-Time Systems

where b ≥ b. By applying Gronwall Lemma, it gives Δxk (t) ≤ be

t

f0 t

|Δu k (τ )|dτ

0

≤ be f0 t = be f0 t

t

eλτ dτ |Δu k (t)|λ

0 λt

e −1 |Δu k (t)|λ . λ

(7.19)

Substituting (7.19) into (7.16) implies that |Δu k+1 (t)| ≤

 m    1  T   − γ (t)β c b k− j j m + 1  |Δu k− j (t)| j=0

eλt − 1  γk− j (t)|β j ||Δu k− j (t)|λ . λ j=0 m

+bc f 0 e f0 t

(7.20)

Applying the expectation operator E on both sides of (7.20) and noting that γk (t) is the only stochastic variable, which is independent of Δu k (t), imply   m   1  T   − γk− j (t)β j c b |Δu k− j (t)| E  |Δu k+1 (t)| ≤ m+1 j=0

eλt − 1  E{γk− j (t)}|β j ||Δu k− j (t)|λ . λ j=0 m

+bc f 0 e f0 t

(7.21)

According to the definition of mathematical expectation, it follows    1  T   − 1 · β c b j m + 1  q(t)    1  +  − 0 · β j cT b (1 − q(t)) m+1    1  1 − q(t) =  − β j cT b q(t) + (7.22) m+1 m+1

    1 − γk− j (t)β j cT b = E  m+1

and 

m    1  1 − q(t) T   |Δu k+1 (t)| ≤  m + 1 − β j c b q(t) + m + 1 |Δu k− j (t)| j=0 eλt − 1  |β j ||Δu k− j (t)|λ . λ j=0 m

+bc f 0 q(t)e f0 t

(7.23)

7.2 ILC Design and Convergence Analysis

109

From (7.23) and the definition of λ-norm, there has     1  1 − q(t) T   |Δu k+1 |λ ≤ − β j c b q(t) + |Δu k− j |λ sup  m+1 m+1 j=0 t∈[0,Td ] m 

+bc f 0 e f0 Td

m 1 − e−λTd  |β j ||Δu k− j |λ . λ j=0

(7.24)

Define     1  1 − q(t) T   q(t) + − β c b j m + 1  m+1 t∈[0,Td ]

η j  sup

(7.25)

and δ  bc f 0 e f0 Td

m 1 − e−λTd  |β j |. λ j=0

(7.26)

Then (7.24) can be rewritten as |Δu k+1 |λ ≤ (

m 

η j + δ) max{|Δu k |λ , |Δu k−1 |λ , . . . , |Δu k−m+1 |λ }, (7.27)

j=0

where 0 < q(t) ≤ 1 is applied. Since δ can be made  sufficiently small with a sufficiently large λ and is independent of k, noting that mj=0 η j ≤ ρ < 1, it follows that m j=0 η j + δ ≤ ρ + δ < 1. That is, as k goes to infinity, it has Δu k → 0, namely, uk → ud . According to the convergence of Δu k and the inequality (7.19), it is obvious that lim Δxk (t) = 0.

(7.28)

|ek (t)| = |cT Δxk (t)| ≤ cT Δxk (t) ≤ cΔxk (t),

(7.29)

k→∞

Since

it follows that limk→∞ ek (t) = 0. The proof is completed. Remark 7.2 In Assumption 7.1, it is assumed that the probability distribution is known and the expectation of q(t) can be calculated directly. While, if p(t) is unknown, but its lower and upper bounds 0 ≤ α1 ≤ p(t) ≤ α2 < 1 for t ∈ [Td − N1 , Td ] are available, where α1 , α2 are known constants, then according to (7.6), there has 1 − α2 ≤ q(t) ≤ 1 − α1 , t ∈ [Td − N1 , Td ].

110

7 Moving Averaging Techniques for Nonlinear Continuous-Time Systems

Based on the lower and upper bounds of q(t) and convergence condition (7.13), the learning gains can be selected as follows. ρ , j = 0, 1, 2, . . . , m. Since One sufficient condition of (7.13) is η j ≤ m+1     1  1 − q(t) T   η j = sup  − β j c b q(t) + m+1 m+1 t∈[0,Td ]       1  1 1 T   − − β q(t) + = sup c b j m + 1  m+1 m +1 t∈[0,Td ]  

  1  1 1 − β j cT b − (1 − α2 ) + , =  m+1 m+1 m+1 the inequality η j ≤

ρ m+1

(7.30)

yields

1 + ρ − 2α2 1−ρ ≤ β j cT b ≤ , (m + 1)(1 − α2 ) (m + 1)(1 − α2 )

(7.31)

where ρ ≥ α2 is required. Without loss of generality, it is assumed that cT b > 0. From (7.31), the learning gain β j satisfies 1−ρ 1 + ρ − 2α2 ≤ βj ≤ . (m + 1)(1 − α2 )(cT b) (m + 1)(1 − α2 )(cT b)

(7.32)

Further, if cT b is unknown, but its lower and upper bounds are known, i.e., b ≤ cT b ≤ b, then from (7.31), it gives 1−ρ 1 + ρ − 2α2 ≤ β j b ≤ β j cT b ≤ β j b ≤ . (m + 1)(1 − α2 ) (m + 1)(1 − α2 )

(7.33)

Therefore, β j should be selected as 1−ρ (m + 1)(1 − α2 )b

≤ βj ≤

1 + ρ − 2α2 (m + 1)(1 − α2 )b

In such case, ρ should satisfy that ρ≥

b + 2α2 b − b b+b

,

where b + 2α2 b − b b+b


0 is the water resistance coefficient, and F is the forward thrust generated by tail motion of the robotic fish. Dividing M on both sides of the first equation in (7.40), the system can be rewritten as v˙ = − αv2 + u, y =v,

(7.41)

7.4 Illustrative Simulations

115

0.04

Maximal tracking error |e |

ks

0.035 0.03 0.025 0.02 0.015 0.01 0.005 0

0

5

10

15

20

25

30

Iteration axis

Fig. 7.4 Maximal tracking error profile of ILC for speed control of robotic fish with N1 = 6, N2 = 8

where α  μ/M > 0 and by Least Square Method, its estimation value is α = 31.2485. u  F/M is viewed as the control input of the system. Because of the term −αv2 , system (7.41) is locally Lipschitz continuous. However, in real world the velocity of the robotic fish is bounded, namely, there exists a constant v¯ > 0 such that |v| ≤ v¯ . As such, for any v1 , v2 , there has | − αv12 − (−αv22 )| = α|v22 − v12 | = α|v2 + v1 ||v2 − v1 | ≤ 2α v¯ |v2 − v1 |,

(7.42)

which implies the globally Lipschitz continuity of system (7.41) and the applicability of the proposed ILC scheme. Let the desired velocity trajectory be vd (t) = 48/505 t 2 (t − 50)2 , t ∈ [0, 50]. To improve the control performance, the following PD-type ILC is adopted: u k+1 (t) =

m m  1  ∗ ∗ u k− j (t) + ( β j e˙k− j (t) − Lek (t)), m + 1 j=0 j=0

(7.43)

where m = 2, β0 = β1 = β2 = 1/3, and ek∗ (t)  γk (t)ek (t). Based on the results in [1], the P-type learning gain L should be a negative value. Without loss of generality, set L = −1 and u 0 (t) = 0.05, t ∈ [0, 50]. Moreover, due to the random disturbances in the external environment, the trial length Tk in each experiment is randomly vary-

116

7 Moving Averaging Techniques for Nonlinear Continuous-Time Systems 0.04

Tracking error ek(t)

0.03 0.02 0.01 D

0

C

−0.01

B A

−0.02 e1 −0.03

0

10

20

e5

e10 30

e20 40

50

Time axis

Fig. 7.5 Tracking error profiles of ILC for speed control of robotic fish with N1 = 6, N2 = 8

ing. Based on multiple experiments and estimation, Tk approximately satisfies an uniform distribution U (44, 58). Consequently, Fig. 7.4 presents the convergence of the maximal tracking error ek s , which shows that ek s has been decreased more than 90% within 30 iterations, and Fig. 7.5 gives the tracking error profiles for 1st, 5th, 10th, and 20th iterations, respectively.

7.5 Summary This chapter extends the ILC design and analysis with nonuniform trial lengths to nonlinear dynamic systems. Similar to the previous chapters, a stochastic variable is introduced to unify the iteration-varying trial lengths, based on which a modified ILC scheme is developed by introducing an iteratively moving average operator that only incorporates the control information from the most recent few trials and may provide more accurate learning experience. The efficiency of the proposed ILC scheme is verified by both an numerical example and an practical application. The results in this chapter are mainly based on [2].

References

117

References 1. Lee HS, Bien Z (1996) Study on robustness of iterative learning control with non-zero initial error. Int J Control 64(3):345–359 2. Li X, Xu J-X, Huang D (2015) Iterative learning control for nonlinear dynamic systems with randomly varying trial lengths. Int J Adapt Control Signal Process 29:1341–1353

Chapter 8

Modified Lambda-Norm Techniques for Nonlinear Discrete-Time Systems

In this chapter, we will consider the discrete-time nonlinear systems, which is different from the continuous-time case in the previous chapter. In particular, the affine nonlinear system is taken into account, where the nonlinear functions satisfy globally Lipschitz continuous condition. Because of the nonlinearity, the lifting techniques in Chaps. 3 and 5 are no longer available. The λ-norm technique in Chap. 2 is revisited and modified with expectations to remove the requirements on prior distribution knowledge of the random trial lengths. A novel technical lemma is provided for the strict convergence analysis in pointwise sense. The initial state variation problem is also discussed in this chapter and a bounded convergence can be guaranteed with the upper bound being a function of the initial state deviation. This chapter differs from the chapters in Part I that nonlinear systems are considered and from other chapters in Part II that the discrete-time case is considered. The techniques derived in this chapter can be applied to other problems, for example, the sampled-data control of continuous-time nonlinear system (in the next chapter).

8.1 Problem Formulation Consider the following discrete-time affine nonlinear system: xk (t + 1) = f (xk (t)) + Bu k (t), yk (t) = C xk (t),

(8.1)

where k = 0, 1, . . . denotes iteration index, t is time instant, t ∈ {0, 1, . . . , N }, and N is the desired iteration length. xk (t) ∈ Rn , u k (t) ∈ R p , and yk (t) ∈ Rq are system state, input, and output, respectively. f (·) : Rn → Rn is the nonlinear function. C

© Springer Nature Singapore Pte Ltd. 2019 D. Shen and X. Li, Iterative Learning Control for Systems with Iteration-Varying Trial Lengths, https://doi.org/10.1007/978-981-13-6136-4_8

119

120

8 Modified Lambda-Norm Techniques for Nonlinear Discrete-Time Systems

and B are matrices with appropriate dimensions. Without loss of generality, it is assumed that C B is of full-column rank. Remark 8.1 Matrices B and C are assumed time-invariant in system (8.1) to make the expressions concise. They can be extended to the time-varying case, Bt and Ct (similar to the one given in Chap. 5), and/or state dependent case, B(x(t)), without making any further effort (see the analysis details below). Moreover, it will be shown that the convergence condition is independent of f (·) in the following. This is the major advantage of ILC; that is, ILC focuses on the convergence property along the iteration axis and requires little system information. In addition, it is evident the nonlinear function could be time-varying. Let yd (t), t ∈ {0, 1, . . . , N } be the desired trajectory. yd (t) is assumed to be realizable; that is, there is a suitable initial state xd (0) and unique input u d (t) such that xd (t + 1) = f (xd (t)) + Bu d (t), yd (t) = C xd (t).

(8.2)

The following assumptions are required for the technical analysis. Assumption 8.1 The nonlinear function f (·) : Rn → Rn satisfies globally Lipschitz continuous condition; that is, ∀x1 , x2 ∈ Rn ,  f (x1 ) − f (x2 ) ≤ k f x1 − x2 ,

(8.3)

where k f > 0 is the Lipschitz constant. Indeed, the globally Lipschitz continuity on nonlinear functions is somewhat strong, although it is common in the ILC field for nonlinear systems and widely satisfied by many practical systems. However, we should point out that this assumption is mainly imposed to facilitate the convergence derivations using the λ-norm technique. With novel analysis techniques such as mathematical induction method, the assumption could be extended to locally Lipschitz case. Assumption 8.2 The identical initialization condition is fulfilled, i.e., xk (0) = xd (0), ∀k. The initial state may not be reset precisely every iteration in practical applications, but the bias is usually bounded. Thus one would relax Assumption 8.2 to the following one. Assumption 8.3 The initial state could shift from xd (0) but should be bounded, i.e., xd (0) − xk (0) ≤  where  is a positive constant. Note that N is the desired trial length, which is also the length of the desired reference. The actual length, Nk , varies in different iterations randomly. Thus, two cases need to be taken into account, i.e., Nk < N and Nk ≥ N . For the latter case, it is observed that

8.1 Problem Formulation

121

only the data at the first N time instants are used for input updating. As a consequence, without loss of generality, one could regard the latter case as Nk = N . From another point of view, we can regard N as the maximum length of actual lengths. For the former case, the outputs at the time instant Nk + 1, . . . , N are missing, and therefore, they are not available for updating. In other words, only input signals for the former Nk time instants can be updated. The control objective of this chapter is to design an ILC algorithm to track the desired trajectory yd (t), t ∈ {0, 1, . . . , N }, based on the available output yk (t), t ∈ {0, 1, . . . , Nk }, Nk ≤ N , such that the tracking error ek (t), ∀t, converges to zero with probability one as the iteration number k goes to infinity. The following lemma is needed for the following analysis. Lemma 8.1 Let η be a Bernoulli binary random variable with P(η = 1) = η and P(η = 0) = 1 − η. M is a positive-definite matrix. Then, the equality EI − ηM = I − ηM holds if and only if one of the following conditions is satisfied: (1) η = 0; (2) η = 1; and (3) 0 < η < 1 and 0 < M ≤ I . Proof. We first prove the sufficiency. It is clear that the equation EI − ηM = I − ηM is valid if η = 0 or η = 1, which means η ≡ 0 and η ≡ 1, respectively. Moreover, the equality also holds if M = I . Thus, it is sufficient to verify the equation for the case 0 < η < 1 and 0 < M < I . According to the definition of mathematical expectation for discrete random variables, we are easy to derive that EI − ηM = P(η = 0) · I − 0 · M + P(η = 1) · I − 1 · M = (1 − η) + ηI − M = 1 + η(I − M − 1). By noticing that M is a positive-definite matrix and 0 < M < I , I − M is a positive-definite matrix. Moreover, for a positive-definite matrix, the Euclidean norm (the largest singular value of the associated matrix) is equal to its maximal eigenvalue, i.e., I − M = σmax (I − M), and therefore, I − M = 1 − σmin (M), where σ (·) denotes the singular value. This property further leads to EI − ηM = 1 − ησmin (M). On the other hand, noting 0 < η < 1, we have I − ηM = σmax (I − ηM) = 1 − σmin (ηM) = 1 − ησmin (M).

122

8 Modified Lambda-Norm Techniques for Nonlinear Discrete-Time Systems

It is clear that the equation EI − ηM = I − ηM is valid. Next, we proceed to prove the necessity. It suffices to show that the equality EI − ηM = I − ηM is not valid if M > I and 0 < η < 1. In this case, it is easy to find EI − ηM = 1 + η(I − M − 1) = 1 + η(σmax (M − I ) − 1) = 1 + ησmax (M) − 2η, while the norm I − ηM is complex as three cases should be discussed, respectively. The details are given as follows. (a) If I − ηM is negative-definite, i.e., I − ηM < 0, then I − ηM = ησmax (M) − 1; (b) If I − ηM is positive-definite, i.e., I − ηM > 0, then I − ηM = 1 − ησmin (M); (c) If I − ηM is indefinite, then I − ηM = max{ησmax (M) − 1, 1 − ησmin (M)}. Thus, it is sufficient to verify that EI − ηM is equal to neither ησmax (M) − 1 nor 1 − ησmin (M). Suppose EI − ηM = ησmax (M) − 1, then we have 1 + ησmax (M) − 2η = ησmax (M) − 1, which means η = 1, and this contradicts with 0 < η < 1. Suppose EI − ηM = 1 − ησmin (M), then we have 1 + ησmax (M) − 2η = 1 − ησmin (M), which means σmax (M) + σmin (M) = 2, and this contradicts with M > I . The proof is thus completed. Remark 8.2 This lemma presents a description of the commutativity between the expectation operator and norm operator, which is not evidently valid. The conclusion of scalar case, i.e., the case E|1 − ηm| = |1 − ηm| with m being a constant, is easy to verify. This lemma provides an extension to the square matrix, where we limit the discussions to positive-definite matrix M. It is still open what the conclusion is for other types of matrices.

8.2 ILC Design In this chapter, the minimum trial length is denoted by Nmin , 0 < Nmin < N . The actual trial length varies among the discrete integer set {Nmin , . . . , N }; that is, the outputs at time instants t = 0, 1, . . . , Nmin are always available for input updating, while the availability of outputs at time instants t = Nmin + 1, . . . , N is random. To describe the randomness of iteration length, we denote the probability of the occurrence of the output at time instant t by p(t). From the above explanations, it is clear that p(t) = 1, 0 ≤ t ≤ Nmin , and 0 < p(t) < 1, Nmin + 1 ≤ t ≤ N . Moreover, when the output at time instant t0 is available in an iteration, the outputs at any time instant t where t < t0 are definitely available in the same iteration. This further

8.2 ILC Design

123

implies that p(Nmin ) > p(Nmin + 1) > · · · > p(N ). It is worthy pointing out that the probability is defined on time instant directly instead of on iteration length, differing from the definition given in Chaps. 2–5. We denote the kth iteration length by Nk , which is a random variable taking values in the set {Nmin , . . . , N }. Let A Nk be the event that the kth iteration length is Nk . Moreover, when an iteration length is Nk , it implies that the outputs at time instants 0 ≤ t ≤ Nk are available and the outputs at time instants Nk + 1 ≤ t ≤ N are missing. Therefore, the probability of the kth iteration length being Nk is calculated N P(At ) = 1. as P(A Nk ) = p(Nk ) − p(Nk+1 ). In turn, t=N min Remark 8.3 In Chap. 2, the probability of random iteration length is first given and then the probability of the output occurrence at each time instant is calculated. In this chapter, the calculation order is exchanged; that is, the probability of the output occurrence at each time instant is first given and then calculate the probability of random iteration length. However, the internally logical relationships are identical. As long as Nk < N , the actual output information is not complete. That is, only the data of the former Nk time instants are used to calculate tracking error for input updating. While for the left time instants, the input updating has to be suspended until the corresponding output information is available. In this case, we simply set the tracking error to be zero because none knowledge is obtained. In other words, a modified tracking error is defined as follows  ek∗ (t)

=

ek (t), 0 ≤ t ≤ Nk , 0, Nk + 1 ≤ t ≤ N ,

(8.4)

where ek (t)  yd (t) − yk (t) is the original tracking error. To make a more concise expression, let us introduce an indicator function 1{event} of an event, which is defined as  1, if the event holds 1{event} = 0, otherwise Then (8.4) could be reformulated as ek∗ (t) = 1{t≤Nk } ek (t).

(8.5)

Remark 8.4 We give a detailed understanding of the newly introduced indicator function or random variable. For arbitrary given time instant t0 with t0 ≤ Nmin , the event {t0 ≤ Nk } holds with probability one. For arbitrary given time instant t0 with t0 > Nmin , the event {t0 ≤ Nk } is a union of events {Nk = t0 }, {Nk = t0 + 1}, . . ., {Nk = N }. Thus the probability of the event {1{t0 ≤Nk } = 1} is calculated as N P(Ai ) = p(t0 ), t0 > Nmin . Combining these two scenarios, P(1{t0 ≤Nk } = 1) = i=t 0 we have P(1{t0 ≤Nk } = 1) = p(t0 ), ∀t. In addition, E1{t0 ≤Nk } = P(1{t0 ≤Nk } = 1) × 1 + P(1{t0 ≤Nk } = 0) × 0 = p(t0 ).

124

8 Modified Lambda-Norm Techniques for Nonlinear Discrete-Time Systems

With the modified tracking error, we can give the following update law for input signal now: (8.6) u k+1 (t) = u k (t) + Lek∗ (t + 1), where L is the learning gain matrix to be defined later, L ∈ R p×q .

8.3 Convergence Analysis The following theorem gives the zero-error convergence of the proposed ILC algorithm for the case that initial state is accurately reset. Theorem 8.1 Consider discrete-time affine nonlinear system (8.1) and ILC algorithm (8.6), and Assumptions 8.1 and 8.2 hold. If the learning gain matrix L satisfies that 0 < LC B < I , then the tracking error would converge to zero as iteration number k goes to infinity, i.e., limk→∞ ek (t) = 0, t = 1, . . . , N . Proof. Subtracting both sides of (8.6) from u d (t), we have Δu k+1 (t) = Δu k (t) − Lek∗ (t + 1),

(8.7)

where Δu k (t)  u d (t) − u k (t) is the input error. Noticing (8.1) and (8.2), it follows that Δxk (t + 1) = ( f (xd (t)) − f (xk (t))) + BΔu k (t), ek (t) = CΔxk (t),

(8.8)

where Δxk (t)  xd (t) − xk (t), which further leads to ek (t + 1) = CΔxk (t + 1) = C BΔu k (t) + C ( f (xd (t)) − f (xk (t))) .

(8.9)

Substituting (8.9) and (8.5) into (8.7), we have Δu k+1 (t) = Δu k (t) − 1{t≤Nk } Lek (t + 1) = Δu k (t) − 1{t≤Nk } L[C BΔu k (t) + C( f (xd (t)) − f (xk (t)))] = (I − 1{t≤Nk } LC B)Δu k (t) − 1{t≤Nk } LC( f (xd (t)) − f (xk (t))). Taking Euclidean norm of both sides of last equation, we have

8.3 Convergence Analysis

125

Δu k+1 (t) ≤ (I − 1{t≤Nk } LC B)Δu k (t) + 1{t≤Nk } LC( f (xd (t)) − f (xk (t))) ≤ (I − 1{t≤Nk } LC B) · Δu k (t) + 1{t≤Nk } LC · ( f (xd (t)) − f (xk (t))) ≤ I − 1{t≤Nk } LC B · Δu k (t) + k f 1{t≤Nk } LC · Δxk (t). Noticing that the event t ≤ Nk is independent of Δu k (t) and Δxk (t). Thus, by taking mathematical expectation of the last inequality, it follows EΔu k+1 (t) ≤ E(I − 1{t≤Nk } LC B · Δu k (t)) + k f E(1{t≤Nk } LC · Δxk (t)) ≤ I − p(t)LC BEΔu k (t) + k f  p(t)LCEΔxk (t),

(8.10)

where for the last inequality, Lemma 8.1 is used by noticing that 0 < LC B < I . On the other hand, taking Euclidean norm of both sides of the first equation in (8.8) yields Δxk (t + 1) ≤ B · Δu k (t) +  f (xd (t)) − f (xk (t)) ≤ B · Δu k (t) + k f xd (t) − xk (t) = B · Δu k (t) + k f Δxk (t),

(8.11)

and then, by taking mathematical expectation, it follows that EΔxk (t + 1) ≤ kb EΔu k (t) + k f EΔxk (t),

(8.12)

where kb ≥ B. Based on the recursion of (8.12) and noting Assumption 8.2, we have EΔxk (t + 1) ≤ kb EΔu k (t) + k f kb EΔu k (t − 1) + k 2f EΔxk (t − 1) ≤ kb EΔu k (t) + kb k f EΔu k (t − 1) + · · · t + kb k t−1 f EΔu k (1) + kb k f EΔu k (0)

+ k tf EΔxk (0) = kb

t  i=0

which further infers

k t−i f EΔu k (i),

(8.13)

126

8 Modified Lambda-Norm Techniques for Nonlinear Discrete-Time Systems

EΔxk (t) ≤ kb

t−1 

k t−1−i EΔu k (i). f

(8.14)

i=0

Then, substituting (8.14) into (8.10) yields that EΔu k+1 (t) ≤ I − p(t)LC BEΔu k (t) + kb  p(t)LC

t−1 

k t−i f EΔu k (i).

(8.15)

i=0

Apply the λ-norm to both sides of last inequality; that is, multiply both sides of last inequality with α −λt and take supremum according to all time instants t, then we have sup α −λt EΔu k+1 (t) t

≤ sup I − p(t)LC B · sup α −λt EΔu k (t) t t   t−1  t−i −λt k f EΔu k (i) . + kb · sup  p(t)LC · sup α t

t

i=0

Let α > k f , then it is observed that sup α

−λt

t

≤ sup t

≤ sup t

≤ sup t

 k t−i f EΔu k (i)

i=0

≤ sup α t

 t−1 

−λt

 t−1 

 t−1 

 α

t−i

EΔu k (i)

i=0

α

−(λ−1)t−i

 EΔu k (i)

i=0

 t−1 

 α −λi EΔu k (i) · α −(λ−1)(t−i)

i=0

 t−1  i=0

  sup α −λi EΔu k (i) α −(λ−1)(t−i) t



 t−1    −λi  −(λ−1)(t−i) ≤ sup α EΔu k (i) sup α t

t

i=0 −(λ−1)Nd

 1−α  ≤ sup α −λi EΔu k (i) × α λ−1 − 1 t

.

(8.16)

8.3 Convergence Analysis

127

Therefore, from (8.16), Δu k+1 (t)λ ≤ sup I − p(t)LC B · Δu k (t)λ t

1 − α −(λ−1)Nd + kb · sup  p(t)LC · Δu k (t)λ × α λ−1 − 1 t

1 − α −(λ−1)Nd Δu k (t)λ , ≤ ρ + kb ϕ α λ−1 − 1 where ρ and ϕ are defined as ρ = sup (I − p(t)LC B), t

ϕ = sup  p(t)LC. t

Noticing that the learning gain matrix L satisfies 0 < LC B < I and 0 < p(t) ≤ 1, ∀t, it is evident that I − p(t)LC B < 1, ∀t. Since 0 ≤ t ≤ N has only finite values, we have 0 < ρ < 1. Let α > max{1, k f }, then there always exists a λ large −(λ−1)Nd enough such that kb ϕ 1−α < 1 − ρ, which further yields α λ−1 −1 ρ  ρ + kb ϕ

1 − α −(λ−1)Nd < 1. α λ−1 − 1

(8.17)

This property implies lim Δu k (t)λ = 0, ∀t.

k→∞

Again, by the finiteness of t, we have lim EΔu k (t) = 0, ∀t.

k→∞

(8.18)

Notice that Δu k (t) ≥ 0, thus it could be concluded from (8.18) that lim Δu k (t) = 0, ∀t.

k→∞

(8.19)

By directly applying mathematical induction method along time axis t, it is easy to show that limk→∞ Δxk (t) = 0 and limk→∞ ek (t) = 0, ∀t. The proof is thus completed. Remark 8.5 One may argue whether it is conservative to design L such that 0 < LC B < I . In our opinion, there is a tradeoff between algorithm design and scope of application. In Chap. 2, the condition on L is somewhat loose; however, the occurrence probability of randomly varying length is required to be estimated prior

128

8 Modified Lambda-Norm Techniques for Nonlinear Discrete-Time Systems

because the convergence condition depends on it. While in this chapter, the requirements on L is a little restrictive, but no information on probability is requested prior. Thus, it is more suitable for practical applications. Here, two simple alternatives are referential if knowledge of C B is available. The first is that design L ∗ such that L ∗ C B > 0 and then multiply a constant μ small enough such that μL ∗ C B < I , (C B)T whence L = μL ∗ . The second is L = β+C , where β > 0. B2 Remark 8.6 The operation length varies from iteration to iteration, thus one may doubt why Theorem 8.1 claims that the tracking error would converge to zero for all time instants. In other words, the influence of random varying length is not revealed. We have some explanations for this issue. On the one hand, in the proof, we introduce the so-called λ-norm, which is similar to the conventional λ-norm in earlier publications but is modified with an additional expectation to deal with the randomness, eliminate the random indicator function 1{t≤Nk } and convert the original expression into the deterministic case. On the other hand, there is a positive probability that the iteration length achieves the maximal length Nd , thus the input at each time instant would be updated more or less. To be specific, the input at time instant t ≤ Nmin would be updated for all iterations, while the input at time instant Nmin < t ≤ Nd is updated for part iterations. Therefore, the input at different time instants may have different convergence speed but they all converge to the desired one. Remark 8.7 The time-invariant model (8.1) is studied in this chapter. However, the system is easy to extend to the time-varying case, where the design condition is slightly modified as 0 < L t Ct+1 Bt < I . The convergence analysis could be completely same. Moreover, the P-type update law (8.6) could be extended to other types of ILC such as PD-type ILC with slight modifications to the proof and convergence conditions (a PD-type algorithm is applied in the next chapter). Furthermore, the considered system is of relative degree one; that is, C B is not zero and of full-column rank. This observation leads us to design the P-type update law (8.6) and convergence condition 0 < LC B < I in the theorem. Under some circumstances, the system τ −1 may be of high relative degree τ ; that is, C ∂ f ( f∂u(x)+Bu) is of full-column rank and

= 0, 0 ≤ i ≤ τ − 2, where f i (x) = f i−1 ◦ f (x) and ◦ denotes the C ∂ f ( f (x)+Bu) ∂u composite operator of functions [1]. For this case, the analysis is still valid provided that the update law is modified as u k+1 (t) = u k (t) + Lek∗ (t + τ ) and the convergence τ −1 condition becomes 0 < LC ∂ f ( f∂u(x)+Bu) < I . i

Remark 8.8 We provide a convergence analysis in modified λ-norm sense above for the ILC problem for discrete nonlinear systems under randomly iteration-varying length situation. One may be interested in monotonic convergence in vector norm sense. To this end, define the lifted vector ψk  [EΔu k (0)T , EΔu k (1)T , . . ., EΔu k (N − 1)T ]T and the associated matrix Γ from (8.15) as a block lower triangular matrix with its elements being the parameters of (8.15), then we have ψk+1 1 ≤ Γ ∞ ψk 1 from (8.15) directly, where  · 1 and  · ∞ denote the 1norm of a vector and the ∞-norm of a matrix, respectively. Consequently, we draw a conclusion that the input error converges to zero monotonically if one can design L such that Γ ∞ < 1.

8.3 Convergence Analysis

129

The identical initialization condition (i.i.c.) in Assumption 8.2 is required to make the analysis more concise. However, it is of great interest to consider the case that the i.i.c. is no longer valid. To be specific, the initial state may vary in a small zone, then it could be proven that the tracking error would converge into a small zone, of which the bound is in proportion to initial state error. This is given in the next theorem. Theorem 8.2 Consider discrete-time affine nonlinear system (8.1) and ILC algorithm (8.6), and Assumptions 8.1 and 8.3 hold. If the learning gain matrix L satisfies that 0 < LC B < I , then the tracking error would converge to small zone, whose bound is in proportion to , as iteration number k goes to infinity, i.e., lim supk→∞ Eek (t) ≤ γ , t = 1, . . . , N , where γ is a suitable constant. Proof. The proof is similar to that of Theorem 8.1 with minor technical modifications. The derivations from (8.7) to (8.12) remain unchanged. While (8.13) is replaced by EΔxk (t + 1) ≤ kb

t 

t k t−i f EΔu k (i) + k f EΔx k (0).

(8.20)

i=0

Combining with Assumption 8.3 leads to EΔxk (t) ≤ kb

t−1 

k t−1−i EΔu k (i) + k t−1 f f .

(8.21)

i=0

Then, substituting (8.21) into (8.10) yields that EΔu k+1 (t) ≤ (I − p(t)LC B)EΔu k (t) + kb  p(t)LC

t−1 

t k t−i f EΔu k (i) +  p(t)LCk f .

(8.22)

i=0

By applying the λ-norm to both sides of last inequality and similar steps to the proof of Theorem 8.1, we are clear to obtain Δu k+1 (t)λ ≤ ρΔu k (t)λ + sup α −λt  p(t)LCk tf .

(8.23)

t

By the finiteness of t (i.e., 0 ≤ t ≤ N ), there exists a positive constant ϑ such that supt α −λt  p(t)LCk tf < ϑ and then Δu k+1 (t)λ ≤ ρΔu k (t)λ + ϑ, which further implies lim sup Δu k+1 (t)λ ≤ k→∞

ϑ . 1−ρ

(8.24)

(8.25)

130

8 Modified Lambda-Norm Techniques for Nonlinear Discrete-Time Systems

By the definition of λ-norm, we have lim sup EΔu k+1 (t) ≤ k→∞

α λt ϑ . 1−ρ

Then, combining with (8.21), we are clear to have that EΔxk (t) is bounded in proportion to , and thus, a suitable γ exists such that lim supk→∞ Eek  ≤ γ . This completes the proof.

8.4 Illustrative Simulations In order to show the effectiveness of the proposed ILC algorithm and verify the convergence analysis, consider the following affine nonlinear system: xk(1) (t + 1) = cos(xk(1) (t)) + 0.3xk(2) (t)xk(1) (t), xk(2) (t + 1) = 0.4 sin(xk(1) (t)) + cos(xk(2) (t)) + u k (t), yk (t) = xk(2) (t), which means B = [0 1]T and C = [0 1] in (8.1). xk (t) = [xk(1) (t), xk(2) (t)]T denotes the state vector of two dimensions. The desired tracking trajectory is

2π t yd (t) = 0.8 sin 50





2π t + 2 sin 25





πt + sin 5

, 0 ≤ t ≤ 50.

The desired iteration length is N = 50. To simulate the randomly iteration-varying length, let Nmin = 40. In other words, the iteration length Nk varies from 40 to 50. As a simple case for illustration, we let Nk satisfy discrete uniform distribution during the discrete set {40, 41, . . . , 50}. It should be noted that the probability distribution is not required for the control design. However, the probability distribution would alter the convergence speed. This is because, generally speaking, larger probability means more updates to the corresponding input, which therefore leads to faster convergence. If P(N ) is very close to 1; that is, most iterations could complete the maximum length, then the behavior along iteration axis would be very close to the iteration-length-invariant tracking case and thus a fast convergence speed could be obtained. The initial state is set to be xk (0) = [0, 0]T . Without loss of generality, the input of the initial iteration is zero, i.e., u 0 (t) = 0, 0 ≤ t ≤ N . It is obvious that C B is equal to 1, thus we set the learning gain in (8.6) as 0.5. The algorithm runs for 50 iterations. The desired trajectory and output at 50th iteration are shown in Fig. 8.1, where the red solid line denotes the desired trajectory, and the blue dashed line marked with

8.4 Illustrative Simulations

131

Desired trajectory vs Output of last iteration

3 desired trajectory last output 2

1

0

−1

−2

−3 5

10

15

20

25

30

35

40

45

50

Time axis

Fig. 8.1 Tracking Performance of the output at the 50th iteration

circles denotes the output at the 50th iteration. As one could see, the system output achieves perfect tracking performance. The tracking error profiles of the whole time interval at the 15th, 20th, 30th, and 40th iterations are shown in Fig. 8.2. It is observed that the error at the 15th iteration has been small. At the 20th iteration, the tracking error is already acceptable. Meanwhile, the error profiles of different iterations end at different time instants, which demonstrates the random varying iteration-length circumstance. The convergence property along the iteration axis is shown in Fig. 8.3, illustrated by the blue line, where the maximal tracking error is defined by maxt |ek (t)| for the kth iteration. As commented in Remark 8.7, the proposed algorithm could be extended to PD-type algorithm. Here, we also make simulations based on PD-type update law u k+1 (t) = u k (t) + L p ek∗ (t + 1) + K d (ek∗ (t + 1) − ek∗ (t)) with learning gain L p = 0.4 and K d = 0.3. The maximal tracking error profile along iteration axis is illustrated by the red dashed line. As one could see from Fig. 8.3, the maximal tracking error reduces to zero fast. To verify the convergence under varying initial states, we let each dimension of the initial state obeys a uniform distribution in [−, ] with different scales  being 0.01, 0.05, and 0.2. The tracking performance would be inferior to the identical initial condition case. However, the algorithm still maintains a robust performance, as shown in Fig. 8.4, where one could find that large initial bias leads to large bound of tracking errors.

132

8 Modified Lambda-Norm Techniques for Nonlinear Discrete-Time Systems 0.08 15th iteration 20th iteration 30th iteration 40th iteration

0.06

Tracking error profiles

0.04

0.02

0

−0.02

−0.04

−0.06

−0.08

0

5

10

15

20

25

30

35

40

45

50

Time axis

Fig. 8.2 Tracking error profiles at the 15th, 20th, 30th, and 40th iterations 0

P−type PD−type

10

−1

Maximal tracking error

10

−2

10

−3

10

−4

10

−5

10

5

10

15

20

25

30

Iterations

Fig. 8.3 Maximal errors along iterations

35

40

45

50

8.4 Illustrative Simulations

133 ε=0.01 ε=0.05 ε=0.2

0

Maximal tracking error

10

−1

10

−2

10

−3

10

5

10

15

20

25

30

35

40

45

50

Iterations

Fig. 8.4 Maximal errors for the case with iteration-varying initial value

We conclude this section with several remarks. The simulations have shown that the conventional P-type update law has a good performance against randomly iteration-varying lengths for discrete-time affine nonlinear systems. Although the convergence in λ-norm sense does not imply the monotonic decreasing naturally, the simulations show that the tracking performance is sustainedly improved. Moreover, when encountering other practical issues, the proposed P-type algorithm can be modified by incorporating other design techniques.

8.5 Summary This chapter proposes the convergence analysis of ILC for discrete-time affine nonlinear systems with randomly iteration-varying lengths. A random variable is introduced to describe the random length. Then the tracking error is modified to facilitate the practical situations. The traditional P-type update law is taken as the control algorithm for our research and it can be extended to other schemes. If the identical initialization condition is satisfied, the tracking error is proved to converge to zero as the iteration number goes to infinity by using modified λ-norm technique. If the initial state shifts in a small bound, then it is shown that the tracking error is also bounded. It is worth pointing out that the probability of random length is not required prior for control design. Due to the usage of modified λ-norm technique, the nonlinear function in this chapter is required to satisfy globally Lipschitz continuous condition. For further study, the case of general nonlinear systems, especially those allow nonlinearities in the actuators and/or sensors, are of great interest. The results in this chapter are mainly based on [2].

134

8 Modified Lambda-Norm Techniques for Nonlinear Discrete-Time Systems

References 1. Sun M, Wang D (2001) Analysis of nonlinear discrete-time systems with higher-order iterative learning control. Dyn Control 11:81–96 2. Shen D, Zhang W, Xu J-X (2016) Iterative learning control for discrete nonlinear systems with randomly iteration varying lengths. Syst Control Lett 96:81–87

Chapter 9

Sampled-Data Control for Nonlinear Continuous-Time Systems

In this chapter, we will provide the results on sampled-data control for continuoustime nonlinear systems with varying trial lengths. To deal with the iteration-varying length problem, we propose two sampled-data ILC schemes, a generic PD-type scheme and a modified version with moving average operator, based on the modified tracking errors that have been redefined when the trial length is shorter or longer than the desired one. Sufficient conditions are derived rigorously to guarantee the convergence of the nonlinear system at each sampling instant. The effectiveness of the proposed schemes are illustrated by simulations. Comparing with the previous chapters, this chapter contributes the following novelties: (1) a first result on sampleddata ILC for continuous-time nonlinear systems with iteration-varying lengths, (2) an in-depth convergence analysis of the generic and iteration-moving-averaged PDtype ILC update laws with sufficient conditions for asymptotical convergence, and (3) the general relative degree for nonlinear systems. In addition, the impact of initial state deviations are also discussed similarly to those in the previous chapter.

9.1 Problem Formulation Consider the following continuous-time nonlinear system: x˙k (t) = f (xk (t)) + B(xk (t))u k (t), yk (t) = g(xk (t)),

(9.1)

where k = 0, 1, . . . denotes the iteration index, t ∈ [0, Tk ] denotes the time index, and Tk is the actual trial length of the kth iteration. Moreover, xk (t) ∈ Rn , u k (t) ∈ R p , yk (t) ∈ Rq are the state, the control input, and the output of the system (9.1), respectively. The nonlinear functions f (·) ∈ Rn , B(·) ∈ Rn× p , and g(·) = [g1 (·), g2 (·), . . . , gq (·)]T ∈ Rq are smooth in their domain of definition.

© Springer Nature Singapore Pte Ltd. 2019 D. Shen and X. Li, Iterative Learning Control for Systems with Iteration-Varying Trial Lengths, https://doi.org/10.1007/978-981-13-6136-4_9

135

136

9 Sampled-Data Control for Nonlinear Continuous-Time Systems

Let h be the sampling period of the sampler and Nk = [Tk / h] be the actual sampling number in the kth iteration. The notation [m] means the largest integer less or equal to the real constant m. Further, the desired trajectory is denoted by yd (t) and assumed to be realizable, where t ∈ [0, T ], T is the desired length of each iteration, and N = [T / h] is the largest number of desired sampling instants. The control input is derived from the ILC law which is designed by using the sampling signal. In order to generate a continuous control input, the zero-order holder device is adopted. Then, the continuous-time control signal is taken piecewise constant between the sampling instants, u k (t) = u k (i h),

(9.2)

where t ∈ [i h, i h + h), 0 ≤ i ≤ Nk − 1. The control objective in this chapter is to design a sampled-data ILC law u k (i h) such that the output error at each sampling instant satisfies limk→∞ yd (i h) − yk (i h) = 0, 0 ≤ i ≤ Nk . To describe the input–output causal relationship of system (9.1), we need the derivative notations which are defined as ∂g(x) f (x) ∂x

L f g(x) = and

j−1

j

j−1

L f g(x) = L f (L f g(x)) =

∂(L f g(x)) ∂x

f (x)

∂ L f g(x) b(x) ∂x

L b L f g(x) =

with L 0f g(x) = g(x), where the superscript  0 means no derivative operation. Definition 9.1 ([1]) The continuous-time nonlinear system (9.1) has extended relative degree {β1 , β2 , . . . , βq } for x(t) and the following conditions hold: (a)



i h+h

L br gm (x(t1 ))dt1 = 0, 1 ≤ r ≤ p, 1 ≤ m ≤ q;

ih

(b)



i h+h

ih



t1

ih

 ···

tj

ih

j

L br L f gm (x(t j+1 ))dt j+1 · · · dt1 = 0

where 1 ≤ j ≤ βm − 2; (c) the q × p matrix is of full-column rank.

9.1 Problem Formulation

137

⎡ ⎤  tβ −1 β −1 β −1 i h+h  t1 · · · i h 1 [L b1 L f 1 g1 (x(tβ1 )), . . . , L b p L f 1 g1 (x(tβ1 ))dtβ1 · · · dt1 i h i h ⎢ ⎥ ⎢ ⎥ .. ⎢ ⎥. . ⎣ ⎦   t β −1 β −1 β −1 q q q i h+h t1 · · · [L L g (x(t )), . . . , L L g (x(t ))dt · · · dt q q β β β 1 b b q q q p 1 f ih ih ih f

Here, bi denotes the ith column of the matrix B(·), 1 ≤ i ≤ p. From system (9.1) and Definition 9.1, we can derive that the mth component of system output at the sampling instant i h + h of the kth iteration is evaluated as ym,k (i h + h) = ym,k (i h) + h L f gm (xk (i h)) + · · · +  + +

i h+h

ih  i h+h



t1

ih  t1

 ··· ···

tβm −1

ih  tβm −1

h βm −1 β −1 L m gm (xk (i h)) (βm − 1)! f

β

L f m gm (xk (tβm ))dtβm · · · dt1 β −1

[L b1 L f m

ih ih ih βm −1 L b p L f gm (xk (tβm ))]dtβm

gm (xk (tβm )), . . . ,

· · · dt1 · u k (i h).

(9.3)

It indicates that the output ym,k (i h + h) is obtained from the control input u k (i h). Thus, {u k (i h), ym,k (i h + h), 1 ≤ m ≤ q} is a pair of dynamically related cause and effect. Assumptions are as follows: Assumption 9.1 For any realizable reference trajectory yd (t), there exist a suitable initial state xd (0) and unique input u d (t) ∈ R p such that x˙d (t) = f (xd (t)) + B(xd (t))u d (t), yd (t) = g(xd (t)),

(9.4)

where t ∈ [0, T ], and u d (t) is uniformly bounded for all t ∈ [0, T ]. Assumption 9.2 For each fixed xk (0), the mappings S (a mapping from (xk (0), u k (t), t ∈ [0, Tk ]) to (xk (t), t ∈ [0, Tk ])) and O (a mapping from (xk (0), u k (t), t ∈ [0, Tk ]) to (yk (t), t ∈ [0, Tk ])) are one to one. Assumption 9.3 The system has an extended relative degree {β1 , β2 , . . . , βq } for x(t), t ∈ [0, T ]. j

β −1

Assumption 9.4 The functions f (·), g(·), B(·), L f gm (·), and L br L f m gm (·), 0 ≤ j ≤ βm , 1 ≤ m ≤ q, 1 ≤ r ≤ p are globally Lipschitz continuous in x on [0, T ]. The Lipschitz constants are denoted by l f , l g , l B , l L f , L b f , respectively. Remark 9.1 Assumption 9.1 presents the realizability of the reference trajectory, which is widely used in the existing literature for nonlinear systems. Indeed, this assumption can be guaranteed by 9.2, while 9.2 further implies the existence and

138

9 Sampled-Data Control for Nonlinear Continuous-Time Systems

uniqueness of the solution to system (9.1). Assumption 9.3 describes the extended relative degree, which is defined above. Assumption 9.4 is imposed to limit the nonlinearities so that the Gronwall inequality and contraction mapping method can be applied to derive the strict convergence analysis. Assumption 9.5 The initial state conditions are identical in each iteration, i.e., xk (0) = xd (0), ∀k. Assumption 9.6 The initial states for any iteration are bounded i.e., xd (0) − xk (0) ≤  where  is a positive constant. Remark 9.2 Assumption 9.5 is imposed to ensure perfect tracking performance. However, in many practical applications, the identical initial states in each iteration cannot hold, because the value of initial state xk (0) may not be reset accurately in each iteration. Thus, Assumption 9.6 is given to relax such assumption to the case that the initial state deviations are bounded. Clearly, Assumption 9.6 holds generally for most practical systems. Assumptions 9.5 and 9.6 will be addressed in Sections 9.2 and 9.3, respectively. In this chapter, we consider the continuous-time nonlinear system with iterationvarying lengths. In order to improve the performance by ILC, it is necessary to suitably address the randomness of the actual trial length Tk in each iteration. Without loss of generality, assume that there exist a minimal trial length Tmin and a maximal trial length Tmax , and the actual trial length in each iteration varies among [Tmin , Tmax ]. Besides, the desired trial length T satisfies the condition Tmin ≤ T ≤ Tmax . Let Tk be a stochastic variable and the probability of the output occurrence at time t be p(t), then the probability distribution function of Tk is ⎧ ⎪ t ∈ [0, Tmin ) ⎨0, P(Tk ≤ t) = p(t), t ∈ [Tmin , Tmax ] ⎪ ⎩ 1, t ∈ (Tmax , +∞] where 0 ≤ p(t) ≤ 1. Besides, the output at any time t < t  is available in an iteration if the output at time t  is available in the same iteration. We can define the general form of this probability distribution without prior information,  p(t) = 

Tmax

1, t ∈ [0, Tmin ) T pmax + t max ς (τ )dτ, t ∈ [Tmin , Tmax ]

ς (τ )dτ = 1 − pmax ,

Tmin

where ς (τ ) is a probability density function, pmax > 0 is the probability of the event that the trial length is Tmax , and thus the probability p(t) satisfies the condition 0 < pmax ≤ p(t) ≤ 1. We should point out that p(t) is of no prior form in this chapter.

9.1 Problem Formulation

139

Obviously, there are two cases for sampling numbers to be addressed, i.e., Nk < N and Nk ≥ N . For the former case, the kth iteration would end before the desired trial length is achieved, and the outputs on the interval (Nk , N ] are missing, which are not available for updating. For the latter case, the kth iteration will still run up to the time instant Nk instead of stopping at N . It is observed that the data after the time instant N are redundant and useless for learning. Without loss of generality, we could simply let the latter case be Nk = N . Then, let 1{i h≤Nk h} , i ∈ [0, N ] be a stochastic variable taking binary values 0 and 1. Here, 1{i h≤Nk h} = 1 denotes the event that the control process can last beyond the sample time instant i h, which occurs with a probability of p(i h), where 0 < p(i h) < 1, while 1{i h≤Nk h} = 0 denotes the event that the control process can not continue to the sample time instant i h, which occurs with a probability of 1 − p(i h). It is apparent that 1{i h≤Nk h} satisfies the Bernoulli distribution, thus we can derive the expectation E{1{i h≤Nk h} } = 1 · p(i h) + 0 · (1 − p(i h)) = p(i h). Therefore, we can define a modified tracking error as follows:  ek∗ (i h)

=

ek (i h), 0 ≤ i ≤ Nk 0, (Nk + 1) ≤ i ≤ N

(9.5)

where ek (i h)  yd (i h) − yk (i h) is the original tracking error. Then (9.5) could be reformulated as ek∗ (i h) = 1{i h≤Nk h} ek (i h).

(9.6)

Before concluding this section, we remind that the technique lemma (Lemma 8.1 in the previous chapter) will also be used to derive some corollaries in the following.

9.2 Sampled-Data ILC Design and Convergence Analysis Two ILC laws with the modified tracking error are introduced in this section, and convergence analyses are addressed.

9.2.1 Generic PD-type ILC Scheme The generic PD-type ILC is given by u k+1 (i h) = u k (i h) + K P ek∗ ((i + 1)h) + K D (ek∗ ((i + 1)h) − ek∗ (i h)),

(9.7)

where 0 ≤ i ≤ N − 1, and K P ∈ R p×q and K D ∈ R p×q are proportional and derivative learning gains, respectively. These gains will be defined in the following.

140

9 Sampled-Data Control for Nonlinear Continuous-Time Systems

Theorem 9.1 Consider the continuous-time nonlinear system (9.1) with Assumptions 9.1–9.5. Let the PD-type ILC law (9.7) be applied with learning gains K P and K D satisfying sup sup EI − G k (i h)k (i h)≤ θ < 1, (9.8) k

i

where G k (i h) = (K P + K D )1{(i+1)h≤Nk h} , and ⎡  i h+h  t1

 tβ1 −1

⎤ β −1 g1 (x(tβ1 )), . . . , L b p L f 1 g1 (x(tβ1 ))dtβ1 · · · dt1 ⎥ ⎢ .. ⎥. k (i h) = ⎢ ⎦ ⎣ .  i h+h  t1  tβq −1 βq −1 βq −1 [L b1 L f gq (x(tβq )), . . . , L b p L f gq (x(tβq ))dtβq · · · dt1 ih ih · · · ih ih

ih

···

ih

β −1

[L b1 L f 1

If the sampling period h is chosen small enough, then the system output yk (i h) converges to yd (i h) for all i ∈ [1, N ] as k → ∞. Proof Define Δu k (·) = u d (·) − u k (·) and Δxk (·) = xd (·) − xk (·). It follows from (9.6) and (9.7) that Δu k+1 (i h) = Δu k (i h) − K P ek∗ ((i + 1)h) − K D (ek∗ ((i + 1)h) − ek∗ (i h)) = Δu k (i h) − (K P + K D )ek∗ ((i + 1)h) + K D ek∗ (i h) = Δu k (i h) − G k (i h)ek ((i + 1)h) + Hk (i h)ek (i h),

(9.9)

where G k (i h) = (K P + K D )1{(i+1)h≤Nk h} and Hk (i h) = K D 1{i h≤Nk h} . From (9.3), the mth component of tracking error at the sampling time instant (i + 1)h can be expressed as em,k ((i + 1)h) = ym,d ((i + 1)h) − ym,k ((i + 1)h) = em,k (i h) + υm,k (i h) + ωm,k (i h) + m,k (i h)Δu k (i h),

(9.10)

where em,k (i h) = gm (xd (i h)) − gm (xk (i h)), υm,k (i h) = h[L f gm (xd (i h)) − L f gm (xk (i h))] + · · · h βm −1 β −1 β −1 [L m gm (xd (i h)) − L f m gm (xk (i h))], (βm − 1)! f  i h+h  t1  tβ −1 m β β ··· [L f m gm (xd (tβm )) − L f m gm (xk (tβm ))]dtβm · · · dt1 ωm,k (i h) = +

ih



ih i h+h  t1

ih



tβm −1

β −1 β −1 [L b1 L f m gm (xd (tβm )), . . . , L b p L f m gm (xd (tβm ))] ih ih β −1 β −1 − [L b1 L f m gm (xk (tβm )), . . . , L b p L f m gm (xk (tβm ))]dtβm · · · dt1 u d (i h),

+

···

ih

 m,k (i h) =

i h+h

ih



t1

ih



···

tβm −1

ih

β −1

[L b1 L f m

β −1

gm (xk (tβm )), . . . , L b p L f m

gm (xk (tβm ))]dtβm · · · dt1 .

9.2 Sampled-Data ILC Design and Convergence Analysis

141

The tracking error at time instant (i + 1)h can be written as ek ((i + 1)h) = ek (i h) + υk (i h) + ωk (i h) + k (i h)Δu k (i h),

(9.11)

where ek (i h) = [e1,k (i h), . . . , eq,k (i h)]T , υk (i h) = [υ1,k (i h), . . . , υq,k (i h)]T , ωk (i h) = [ω1,k (i h), . . . , ωq,k (i h)]T , T T k (i h) = [1,k (i h), . . . , q,k (i h)]T .

Substituting (9.11) into (9.9) yields Δu k+1 (i h) = (I − G k (i h)k (i h))Δu k (i h) + (Hk (i h) − G k (i h))ek (i h) − G k (i h)(υk (i h) + ωk (i h)). (9.12) Taking norms to both sides of (9.12) and applying the Lipschitz condition in Assumption 9.4, we have Δu k+1 (i h) ≤ I − G k (i h)k (i h)Δu k (i h) + l g γm Δxk (i h) + γn (υk (i h) + ωk (i h))

(9.13)

and υk (i h) ≤ γ1 Δxk (i h),  tβ1 −1 ⎡  i h+h  t1 ⎤  Δxk (tβ1 )dtβ1 · · · dt1  ih · · · ih  ih  ⎢ ⎥ .. ωk (i h) ≤ γ2 ⎣ , ⎦ .     i h+h  t1 · · ·  tβq −1 Δx (t )dt · · · dt  ih

ih

ih

k

βq

βq

1

where h βm −1 h + ··· + }l L f , 1≤m≤q 1! (βm − 1)! γ2 = l L f + plb f ρud , ρud = sup u d (i h), γ1 = max {

0≤i≤n k −1

and γm is the norm bound for (Hk (i h) − G k (i h)), γn is the norm bound for G k (i h).

142

9 Sampled-Data Control for Nonlinear Continuous-Time Systems

From (9.1) and (9.4) we can obtain  Δxk (t) =

t



t

[ f (xd (τ )) − f (xk (τ ))]dτ +

ih

[B(xd (τ ))u d (τ ) − B(xk (τ ))u k (τ )]dτ,

ih

(9.14) where t ∈ [i h, i h + h]. Then, taking the norms and applying Bellman–Gronwall’s lemma to (9.14) results in that Δxk (t) ≤ Δxk (i h)eγ3 (t−i h) +



t

eγ3 (t−s) γ B Δu k (s)ds,

ih

Δxk (t) ≤ γ4 Δxk (i h) + γ5 Δu k (i h), where γ3 = l f + l B ρud , γ4 = eγ3 h , γ5 = B(xk (t)). Moreover, we can obtain

γ B (γ4 −1) , γ3

(9.15)

and γ B is the norm bound for

Δxk (i h) ≤ γ4 Δxk ((i − 1)h) + γ5 Δu k ((i − 1)h).

(9.16)

According to the Assumption 9.5, i.e., xk (0) = xd (0) for all k, it has Δxk (i h) ≤ γ5

i−1 

i−1− j

γ4

Δu k ( j h).

(9.17)

j=0

Then, (9.13) can be rewritten as Δu k+1 (i h) ≤ ρ¯k (i h)Δu k (i h) + γ6 Δxk (i h), Δu k+1 (i h) ≤ ρ¯k (i h)Δu k (i h) + γ5 γ6

i−1 

(9.18)

i−1− j

γ4

Δu k ( j h),

(9.19)

j=0

where ρ¯k (i h) = ρk (i h) + γ2 γ5 γh γn , ρk (i h) = I − G k (i h)k (i h), γh = max βq β1 { hβ1 ! , . . . , hβq ! } and γ6 = (γ1 + γ2 γ4 γh )γn + l g γm . Clearly, sufficiently small sampling period h yields arbitrarily small γh . Furthermore, taking mathematical expectation to both sides of (9.19), we have ⎛ EΔu k+1 (i h) ≤ E {ρ¯k (i h)Δu k (i h)} + E ⎝γ5 γ6

i−1 

⎞ i−1− j

γ4

Δu k ( j h)⎠

j=0

≤ E{ρ¯k (i h)}EΔu k (i h) + γ5 γ6

i−1 

i−1− j

γ4

EΔu k ( j h), (9.20)

j=0

where E{ρ¯k (i h)} = E{ρk (i h)} + γ2 γ5 γh γn , and E{ρk } = EI − G k (i h)k (i h).

9.2 Sampled-Data ILC Design and Convergence Analysis

143

Multiplying both sides of (9.20) with α −λi , and taking supremum for all time instants i yields sup α −λi EΔu k+1 (i h) ≤ sup E{ρ¯k } sup α −λi EΔu k (i h) i

i

i

+ γ5 γ6 sup α −λi i

i−1 

i−1− j

γ4

EΔu k ( j h).

(9.21)

j=0

Let α > γ4 , then we can drive that sup α −λi i

i−1 

i−1− j

γ4

EΔu k ( j h) ≤ sup α −λi i

j=0

≤α

i−1 

α i−1− j EΔu k ( j h)

j=0

−1

sup i

i−1 





sup α

j=0

−λj



EΔu k ( j h) α

 −(λ−1)(i− j)

j

≤ α −1 Δu k (i h)λ sup

i−1 

i

α −(λ−1)(i− j)

j=0

≤ ηd · Δu k (i h)λ ,

(9.22)

−(λ−1)n d

where ηd = 1−ααλ −α . Substituting (9.22) into (9.21) implies that Δu k+1 (i h)λ ≤ sup E{ρ¯k (i h)}Δu k (i h)λ + γ5 γ6 ηd Δu k (i h)λ .

(9.23)

i

Let μ = sup sup E{ρ¯k (i h)} k

i

and κ = γ5 γ 6 . We have Δu k+1 (i h)λ ≤ (μ + κηd )Δu k (i h)λ .

(9.24)

Let α > max{1, γ4 }, then it is possible to choose a sufficiently small sampling period h and a sufficiently large λ such that κηd = κ

1 − α −(λ−1)n d αλ − α

(9.25)

is arbitrarily small. Thus, if the (9.8) holds ∀i, there exist a sufficiently small h and a sufficiently large λ such that μ + κηd ≤ ζ < 1.

144

9 Sampled-Data Control for Nonlinear Continuous-Time Systems

Then, it is guaranteed that lim Δu k (i h)λ = 0, ∀i.

k→∞

According to the finiteness of i, it follows lim EΔu k (i h) = 0, ∀i.

k→∞

Noticing Δu k (i h) ≥ 0 we obtain lim Δu k (i h) = 0, ∀i.

k→∞

Then, it is easy to conclude the results limk→∞ Δxk (i h) = 0 and limk→∞ ek (i h) = 0, ∀i. This completes the proof. Theorem 9.1 presents an explicit sufficient condition guaranteeing the asymptotical convergence of the tracking errors at sampling time instants for the generic PD-type sampled-data ILC for nonlinear systems with iteration-varying lengths. The sufficient condition (9.8) indicates that gains of both P-part and D-part of the update law would affect the convergence. It is worthwhile to mention that the proposed sampled-data ILC is able to work well without an accurate system model. The learning gains can be determined by some approximations while the sampling period is sufficiently small. Noting that mathematical expectation is involved in the convergence condition, we can remove this operator by strengthening the design of learning gains as in the following corollary. Corollary 9.1 Consider the continuous-time nonlinear system (9.1) with Assumptions 9.1–9.5. Let the PD-type ILC law (9.7) be applied with learning gains K P and K D satisfying 0 < (K P + K D )k (i h) < I, sup sup I − (K P + K D ) p((i + 1)h)k (i h) < 1. k

(9.26)

i

Then the system output yk (i h) converges to yd (i h) for all i ∈ [1, N ] as k → ∞ if the sampling period h is chosen small enough. Proof Applying the results in Lemma 8.1 to the condition (9.8) in Theorem 9.1, we can complete the proof of this corollary. Remark 9.3 In Theorem 9.1, we depict a qualitative expression of the sampling period h to guarantee the asymptotical convergence that the sampling period should be small enough. One may be interested in how is the explicit range of the sampling period. Indeed, it is difficult to determine the period because of the nonlinearities in the system and controller. Generally, it is seen from (9.24) that the sampling period

9.2 Sampled-Data ILC Design and Convergence Analysis

145

should ensure that γ2 γ5 γh γn + κηd < 1 − θ . Since we can always select sufficiently large α to make κηd arbitrarily small, we should still ensure γ2 γ5 γh γn < 1 − θ . In practical applications, we find that a small sampling period implies a better control performance. From this point of view, it is suggested to select the possible smallest sampling period to guarantee the convergence condition and improve the actual tracking performance in the meantime.

9.2.2 The Modified ILC Scheme An iteratively moving average operator is used in this section to solve the problem of sampled-data ILC for nonlinear systems with iteration-varying lengths. The iteratively moving average operator in this chapter only use the information of several previous trials to compensate the absent tracking information. The inherent reason lies in that the data from early iterations may be useless for current input updating, while the information of adjacent iterations would be helpful in correcting the input signals. Definition 9.2 For a sequence f k−r (·), f k−r +1 (·), . . . , f k (·) with r ≥ 0, an iteratively moving average operator is defined as 1  f k− j (·), r + 1 j=0 r

A{ f k (·)} 

(9.27)

where r + 1 is the size of the moving window. As a special case, the iteratively moving average operator of the mth component of the vector sequence is represented as 1  m f (·). r + 1 j=0 k− j r

A{ f km (·)} 

(9.28)

Design an iteratively moving average operator-based PD-type ILC law as follows u k+1 (i h) = A{u k (i h)} + K P  A{ek∗ ((i + 1)h)} + K D  A{ek∗ ((i + 1)h) − ek∗ (i h)}, (9.29) where K P  ∈ R p×q and K D  ∈ R p×q are proportional and derivative learning gains, respectively. These gains will be determined in the following. In addition, we assume that u −1 (i h) = u −2 (i h) = · · · = u −z (i h) = 0 without loss of any generality. Theorem 9.2 Consider the continuous-time nonlinear system (9.1) with Assumptions 9.1–9.5. Let the PD-type ILC law (9.29) be applied with learning gains K P  and K D  satisfying

146

9 Sampled-Data Control for Nonlinear Continuous-Time Systems

0≤

r 

ϑw ≤ θ < 1,

(9.30)

w=0

where ϑw 

1 sup sup EI − G k−w  (i h)k−w (i h), r +1 k i

G k−w  (i h) = (K P  + K D  )1{(i+1)h≤n k−w h} . If the sampling period h is chosen small enough, then the system output yk (i h) converges to yd (i h) for all i ∈ [1, N ] as k → ∞. Proof Substituting (9.6), (9.27) into (9.29) yields to 1  1  u k−w (i h) + G k−w  (i h)ek−w ((i + 1)h) r + 1 w=0 r + 1 w=0 r

u k+1 (i h) =

r

1  Hk−w  (i h)ek−w (i h), r + 1 w=0 r

−

(9.31)

where Hk−w  (i h) = K D  1{i h≤n k−w h} and G k−w  (i h) = (K P  + K D  )1{(i+1)h≤n k−w h} . Then, it follows that Δu k+1 (i h) =

r 

1 (I − G k−w  (i h)k−w (i h))Δu k−w (i h) r + 1 w=0 + −

r 

1 (Hk−w  (i h) − G k−w  (i h))ek−w (i h) r + 1 w=0

r 

1 G k−w  (i h)(υk−w (i h) + ωk−w (i h)). r + 1 w=0

(9.32)

Taking norms to both sides of (9.32) and applying the Lipschitz condition in Assumption 9.4 yields that  r    1     Δu k+1 (i h) ≤  r + 1 (I − G k−w (i h)k−w (i h)) Δu k−w (i h) w=0 +

r  1 l g γm  Δxk−w (i h) r +1 w=0

+

r  1 γn  (υk−w (i h) + ωk−w (i h)) , r + 1 w=0

(9.33)

9.2 Sampled-Data ILC Design and Convergence Analysis

147

where γm  is the norm bound for (Hk−w  (i h) − G k−w  (i h)) and γn  is the norm bound for G k−w  (i h). Then from (9.17), we have Δxk−w (i h) ≤ γ5

i−1 

i−1− j

γ4

Δu k−w ( j h),

(9.34)

j=0

υk−w (i h) + ωk−w (i h) ≤ (γ1 + γ2 γ4 γh )Δxk−w (i h) + γ2 γ5 γh Δu k−w (i h). (9.35) Combing (9.33), (9.34) and (9.35), we can obtain 1  ρ¯k−w  (i h)Δu k−w (i h) r + 1 w=0 r

Δu k+1 (i h) ≤

1   i−1− j γ Δu k−w ( j h), r + 1 w=0 j=0 4 r

+ γ5 γ 7 ·

i−1

(9.36)

where ρ¯k−w  (i h) = ρk−w  (i h) + γ2 γ5 γh γn  , ρk−w  (i h) = I − G k−w  (i h)k−w (i h), and γ7 = (γ1 + γ2 γ4 γh )γn  + l g γm  . Taking mathematical expectation to both sides of (9.36), we can conclude that 

 r 1  EΔu k+1 (i h) ≤ E ρ¯k−w  (i h)Δu k−w (i h) r + 1 w=0 ⎧ ⎫ r i−1 ⎨ ⎬ 1   i−1− j + E γ5 γ 7 · γ4 Δu k−w ( j h) ⎩ ⎭ r + 1 w=0 j=0 1  E{ρ¯k−w  (i h)}EΔu k−w (i h) r + 1 w=0 r

≤

1   i−1− j γ EΔu k−w ( j h), r + 1 w=0 j=0 4 r

+ γ5 γ 7 ·

i−1

(9.37)

where E{ρ¯k−w  (i h)} = E{ρk−w  (i h)} + γ2 γ5 γh γn  , E{ρk−w  (i h)} = EI − G k−w  (i h)k−w (i h). Multiplying both sides of (9.37) with α −λi , and taking supremum for all time instants i, we have

148

9 Sampled-Data Control for Nonlinear Continuous-Time Systems

1  sup E{ρ¯k−w  (i h)} sup α −λi EΔu k−w (i h) r + 1 w=0 i i r

sup α −λi EΔu k+1 (i h) ≤ i

  i−1− j 1 γ4 EΔu k−w ( j h), sup α −λi r +1 i w=0 j=0 r

+ γ5 γ 7 ·

i−1

(9.38) and sup α −λi i

r  i−1 

i−1− j

γ4

EΔu k−w ( j h) ≤ ηd

w=0 j=0

r 

Δu k−w (i h)λ .

(9.39)

w=0

Thus, (9.38) can be rewritten as 1  sup E{ρ¯k−w  (i h)}Δu k−w (i h)λ r + 1 w=0 i r

Δu k+1 (i h)λ ≤

+ γ5 γ7 · Define ϑw 

r  1 ηd Δu k−w ( j h)λ . r + 1 w=0

1 sup sup E{ρk−w  (i h)} r +1 k i

and θ0  γ5 γ7 ηd ·

1 . r +1

(9.40)

(9.41)

(9.42)

Then, we have Δu k+1 (i h)λ ≤

 r 



  (ϑw + ) + θ0 max Δu k (i h)λ , Δu k−1 (i h)λ , . . . , Δu k−r +1 (i h)λ

w=0

(9.43) 1 where  = γ2 γ5 γh γn  · r +1 . If we choose a λ large enough and the sampling period h small enough, then θ0 and  can be made sufficiently small and be independent of k.  From (9.30), it follows that rw=0 (ϑw + ) + θ0 < 1. This further implies that

lim Δu k (i h)λ = 0, ∀i.

k→∞

Therefore, lim Δu k (i h) = 0, ∀i.

k→∞

9.2 Sampled-Data ILC Design and Convergence Analysis

149

It is apparent that limk→∞ Δxk (i h) = 0 and limk→∞ ek (i h) = 0, ∀i. This completes the proof. Theorem 9.2 presents a parallel result to Theorem 9.1 for the iteration-movingaverage-operator-based algorithm. Since we have employed the information of the previous r iterations, it can be seen from the condition (9.30) that the convergence depends on the jointed contraction of the involved iterations. Noting that an average (i.e., 1/(r + 1)) is added to ϑw , the convergence condition of this theorem is generally not stricter than that of Theorem 9.1. Corollary 9.2 Consider the continuous-time nonlinear system (9.1) with Assumptions 9.1–9.5. Let the PD-type ILC law (9.29) be applied with learning gains K P  and K D  satisfying that 0 < (K P  + K D  )k−w (i h) < I, 0≤

r 

ϑw  < 1,

w=0

where ϑw  

1 sup sup I − (K P  + K D  ) p((i + 1)h)k−w (i h). r +1 k i

Then the system output yk (i h) converges to yd (i h) for all i ∈ [1, N ] as k → ∞ if the sampling period h is chosen small enough. Remark 9.4 In many practical applications, there may be stochastic disturbances and measurement noises in the process of control. Such disturbances and noises would lead to a large deviation between the actual output and the desired trajectory in some iterations. In such case, if we only use the information from the last iteration, the computed signals may have remarkable deviations. Meanwhile, when considering nonlinear systems, the nonlinearity may further involve a complex updating process. In this chapter, we adopt the tracking information from several iterations and make a combination of such information. This is the iteratively moving operator mechanism proposed above. It is believed that such mechanism would have certain advantages in dealing with the disturbances, noises, and uncertainties.

9.3 Sampled-Data ILC Design with Initial Value Fluctuation In practical applications, the value of initial state xk (0) may not be set precisely in each iteration, which leads to an observation that Assumption 9.5 does not hold. In this section, we will replace Assumption 9.5 with a relaxed condition, Assumption 9.6, and propose the corresponding stable convergence results.

150

9 Sampled-Data Control for Nonlinear Continuous-Time Systems

9.3.1 Generic PD-type ILC Scheme Theorem 9.3 Consider the continuous-time nonlinear system (9.1) with Assumptions 9.1–9.4 and 9.6. Let the PD-type ILC law (9.7) be applied. If the sampling period h is chosen small enough and the learning gains K P and K D satisfy sup sup EI − G k (i h)k (i h) ≤ θ < 1, k

i

then the tracking error would converge to a small zone, whose upper bound is in proportion to , for all i ∈ [1, N ] as k → ∞, i.e., limk→∞ supi Eek (i h) ≤ γe . Proof From (9.16), it follows Δxk (i h) ≤ γ5

i−1 

i−1− j

γ4

Δu k ( j h) + γ4i .

(9.44)

j=0

Thus, we can obtain that ⎛ Δu k+1 (i h) ≤ ρ¯k (i h)Δu k (i h) + γ6 ⎝γ5

i−1 

⎞ i−1− j

γ4

Δu k ( j h) + γ4i  ⎠

j=0

≤ ρ¯k (i h)Δu k (i h) + γ5 γ6

i−1 

i−1− j

γ4

Δu k ( j h) + γ6 γ4i . (9.45)

j=0

Taking mathematical expectation to both sides of (9.45) yields ⎧ ⎨

⎫ i−1 ⎬  i−1− j EΔu k+1 (i h) ≤ E{ρ¯k (i h)Δu k (i h)} + E γ5 γ6 γ4 Δu k ( j h) + γ6 γ4i  ⎩ ⎭ j=0

i−1  i−1− j γ4 EΔu k ( j h) + γ6 γ4i . ≤ E{ρ¯k (i h)}EΔu k (i h) + γ5 γ6 j=0

(9.46) Multiplying both sides of (9.46) with α −λi , and taking supremum for all time instants i, we can get sup α −λi EΔu k+1 (i h) ≤ sup E{ρ¯k (i h)} sup α −λi EΔu k (i h) i

i

i

+ γ5 γ6 sup α −λi i

i−1  j=0

i−1− j

γ4

EΔu k ( j h)

9.3 Sampled-Data ILC Design with Initial Value Fluctuation

+ γ6  sup α −(λ−1)i .

151

(9.47)

i

From (9.22), we obtain that Δu k+1 (i h)λ ≤ (μ + κηd )Δu k (i h)λ + 1 ,

(9.48)

where 1 = γ6  supi α −(λ−1)i . Then, it follows that lim Δu k (i h)λ ≤

k→∞

1 . 1 − (μ + κηd )

(9.49)

Moreover, from the relationship among Δu k (i h), Δxk (i h) and ek (i h), we have lim ek (i h)λ ≤

k→∞

1 l g γ 5 η d + l g sup α −(λ−1)i . 1 − (μ + κηd ) i

(9.50)

It can be further obtained that lim sup Eek (i h) ≤ γe ,

k→∞

where γe =

supi α i l g γ5 γ6 ηd 1−(μ+κηd )

(9.51)

i

+ l g supi α i . This completes the proof.

Generally, Theorem 9.3 shows that the initial state deviations linearly constrain the final tracking performance. Consequently, as  → 0 (i.e., the identically resetting condition holds), the tracking errors at sampling instants would converge to zero. This result coincides with our intuitive knowledge of the effect of initial states on the entire operation interval. In practical applications, we may design suitable initial learning mechanisms to achieve an asymptotically precise initialization. Corollary 9.3 Consider the continuous-time nonlinear system (9.1) with Assumptions 9.1–9.4 and 9.6. Let the PD-type ILC law (9.7) be applied. If the sampling period h is chosen small enough and the learning gains K P and K D satisfy that 0 < (K P + K D )k (i h) < I, sup sup I − (K P + K D ) p((i + 1)h)k (i h) < 1, k

(9.52)

i

then the tracking error would converge to a small zone, whose upper bound us in proportion to , for all i ∈ [1, N ] as k → ∞, i.e.,limk→∞ supi Eek (i h) ≤ γe  .

152

9 Sampled-Data Control for Nonlinear Continuous-Time Systems

9.3.2 The Modified ILC Scheme Theorem 9.4 Consider the continuous-time nonlinear system (9.1) with Assumptions 9.1–9.4 and 9.6. Let the PD-type ILC law (9.29) be applied. If the sampling period h is chosen small enough and the learning gains K P  and K D  satisfy that 0≤

r 

ϑw ≤ θ < 1,

w=0

then the tracking error would converge to a small zone, whose upper bound is in proportion to , for all i ∈ [1, N ] as k → ∞, i.e., limk→∞ supi Eek (i h) ≤ γ˜e . Proof From (9.34), we have Δxk−w (i h) ≤ γ5

i−1 

i−1− j

γ4

Δu k−w ( j h) + γ4i .

(9.53)

j=0

Substituting (9.53) into (9.33) implies that 1  ρ¯k−w  (i h)Δu k−w (i h) r + 1 w=0 r

Δu k+1 (i h) ≤

1   i−1− j 1 · γ7 γ4i . γ4 Δu k−w ( j h) + r + 1 w=0 j=0 r +1 r

+ γ5 γ 7 ·

i−1

(9.54) Taking mathematical expectation on both sides of (9.54), we can obtain that 

 r 1   EΔu k+1 (i h) ≤ E ρ¯k−w (i h)Δu k−w (i h) r +1 w=0 ⎧ ⎫ r i−1 ⎨ ⎬ 1   i−1− j 1 · γ7 γ4i  + E γ5 γ7 · γ4 Δu k−w ( j h) + ⎩ ⎭ r +1 r +1 w=0 j=0

≤

1 r +1

r 

E{ρ¯k−w  (i h)}EΔu k−w (i h)

w=0

+ γ5 γ7 ·

r i−1 1   i−1− j 1 · γ7 γ4i . γ4 EΔu k−w ( j h) + r +1 r +1 w=0 j=0

(9.55) Multiplying both sides of (9.55) with α −λi , and taking supremum for all time instants i, we can get

9.3 Sampled-Data ILC Design with Initial Value Fluctuation

153

1  sup E{ρ¯k−w  (i h)} sup α −λi EΔu k−w (i h) r + 1 w=0 i i r

sup α −λi EΔu k+1 (i h) ≤ i

+ γ5 γ 7 ·

i−1 r   1 i−1− j γ4 EΔu k−w ( j h) sup α −λi r +1 i w=0 j=0

1 sup α −(λ−1)i . r +1 i

+ γ7  ·

(9.56)

Then, 1  sup E(ρ¯k−w  (i h))Δu k−w (i h)λ r + 1 w=0 i r

Δu k+1 (i h)λ ≤

+ γ5 γ7 ·

r  1 1 Δu k−w (i h)λ + γ7  · ηd sup α −(λ−1)i . r + 1 w=0 r +1 i

(9.57) Therefore, Δu k+1 (i h)λ ≤

 r 

 ϑw + θ0 max{Δu k (i h)λ , . . . , Δu k−r +1 (i h)λ } + 2 ,

w=0

(9.58) where 2 = γ7  ·

1 r +1

supi α −(λ−1)i . It follows that lim Δu k (i h)λ ≤

k→∞

1−(

r

2

w=0

ϑw + θ0 )

.

(9.59)

Moreover, from the relationship between Δu k (i h), Δxk (i h) and ek (i h), we have lim ek (i h)λ ≤

k→∞

1−(

2 l g γ5 η d r + l g  sup α −(λ−1)i . ϑ + θ ) w 0 i w=0

(9.60)

It can be further obtained that lim sup Eek (i h) ≤ γ˜e 

k→∞

where γ˜e =

1 r +1

·

supi α i l g γ5 γ7 ηd  1−( rw=0 ϑw +θ0 )

(9.61)

i

+ l g supi α i . This completes the proof.

Similar to Theorem 9.2, this theorem extends previous results to the iterationmoving-average-operator-based ILC algorithm and provides the sufficient condition for convergence. The dependence of the final tracking error on the initial state error is also described.

154

9 Sampled-Data Control for Nonlinear Continuous-Time Systems

Corollary 9.4 Consider the continuous-time nonlinear system (9.1) with Assumptions 9.1–9.4 and 9.6. Let the PD-type ILC law (9.29) be applied. If the sampling period h is chosen small enough and the learning gains K P  and K D  satisfy that 0 < (K P  + K D  )k−w (i h) < I, 0≤

r 

ϑw  < 1,

(9.62)

w=0

then the tracking error would converge to a small zone, whose upper bound is in proportion to , for all i ∈ [1, N ] as k → ∞, i.e.,limk→∞ supi Eek (i h) ≤ γ˜e  .

9.4 Illustrative Simulations In this section, an illustration example is presented to show the effectiveness of two proposed ILC schemes. Consider the following continuous-time nonlinear system: x˙1 (t) = 1.8x2 (t), x˙2 (t) = 2.2 cos(x2 (t)) + 2.2u(t) + 0.1 sin(x1 (t)u(t)), y(t) = x1 (t).

(9.63)

The length of desired trajectory is T = 1. In order to simulate the randomly iterationvarying length, let the actual length Tk vary among [0.9, 1]. We choose h = 0.005 as the sampling period, then the expected sampling number is N = 200. Thus, Nk varies among [180, 200]. For a simple simulation, we assume that Nk obeys the uniform distribution on [180, 200] for illustration. The desired reference trajectory is 1 yd (i h) = 3π(i h)3 − π(i h)7 . 7

9.4.1 Generic PD-type ILC Scheme The learning law (9.7) is applied with the learning gains given as K P = 0.1 and K D = 190. It is numerically computed that the condition (9.8) is satisfied. The initial state for each iteration is first set to be xk (0) = 0 (according to Assumption 9.5). The algorithm runs for 100 iterations. Figure 9.1 shows that the output trajectory converges to the desired trajectory at all sampling instants for the last iteration, i.e., the 100th iteration. It is seen that the output at the last iteration almost coincides with

9.4 Illustrative Simulations

155

Reference trajectory and output trajectory

9

output of the last iteration desired trajectory

8 7 6 5 4 3 2 1 0 0

50

100

150

200

Time axis

Fig. 9.1 Desired trajectory and output of the last iteration

the desired reference, which shows the well-tracking performance of the generic PD-type ILC. The performance of the maximal tracking error is presented in Fig. 9.2, where the maximal tracking error is defined as the worst tracking error of each iteration. We can observe from Fig. 9.2 that the maximal tracking error decreases fast at the first few iterations and then converges to zero asymptotically along the iteration axis. Moreover, to show the tracking performance and the iteration-varying lengths, we plot the tracking error of the whole iteration in Fig. 9.3, where the 60th, 70th, and 80th iterations are illustrated, respectively. As one can see, the magnitude of the tracking error is rather small for the referred iterations. Meanwhile, the lengths of the 60th, 70th, and 80th iterations are 195, 191, and 196, respectively. This observation demonstrates the fact that the iteration length can vary from iteration to iteration. In addition, we also find that the tracking error of the latter part of the time interval is distinctly larger than that of the former part of the time interval. This is because the tracking error from previous time instants will also affect the tracking error at the latter time instants. Thus it is reasonable that the tracking error from the former part would converge faster than that from the latter part. In addition, to show the robustness of the proposed algorithm against randomly varying initial states, we let  in Assumption 9.6 be 0.02, 0.06, and 0.12, respectively. Then, the algorithm still runs for 100 iterations according to each case and the maximal tracking error profiles along the iteration axis are plotted in Fig. 9.4. It can be seen that the upper bounds of the maximal tracking errors are strongly related to

156

9 Sampled-Data Control for Nonlinear Continuous-Time Systems 6

Maximal tracking error

5

4

3

2

1

0 0

20

40

60

80

100

Iteration axis

Fig. 9.2 Maximal tracking error along iterations 0.03

60th iteration 70th iteration 80th iteration

0.02

Tracking error e

0.01

0

−0.01

−0.02

−0.03

−0.04

0

0.2

0.4

0.6

Time axis

Fig. 9.3 Tracking errors at the 60th, 70th and 80th iterations

0.8

1

9.4 Illustrative Simulations

157

1

10

Maximal tracking error

0.12 0.06 0.02

0

10

−1

10

−2

10

0

20

40

60

80

100

Iteration axis

Fig. 9.4 Maximal tracking errors along iterations

the value of ; that is, a larger  leads to a larger bound of maximal tracking errors profiles. This verifies the theoretical analysis.

9.4.2 The Modified ILC Scheme The modified ILC scheme (9.29) is also simulated for 100 iterations. The parameters of the learning algorithm are same to the generic ILC (9.7), except that there exists an averaging operator with the moving window size being four. That is, in the modified ILC scheme, we still retain the learning gains as K P = 0.1 and K D = 190 and let r = 3. Thus, the signals from the kth, (k − 1)th, (k − 2)th, and (k − 3)th iterations are used in generating the input signal for the (k + 1)th iteration. We first simulate the identical initialization case, i.e., xk (0) = 0 (according to Assumption 9.5). The output tracking performance of the last iteration is same to Fig. 9.1, and thus we omit this figure to avoid repetition. The maximal tracking error profile along the iteration axis and the illustrated tracking error profiles along the time axis for selected iterations are plotted in Figs. 9.5 and 9.6, respectively. Then we simulate the varying initial states case following the same setting in the last subsection. That is, we let  in Assumption 9.6 be 0.02, 0.06, and 0.12, respectively. The maximal tracking error profiles along the iteration axis are plotted in Fig. 9.7. It is observed that the conclusions for the generic ILC scheme still hold for the modified ILC scheme.

158

9 Sampled-Data Control for Nonlinear Continuous-Time Systems 6

Maximal tracking error

5

4

3

2

1

0 0

20

40

60

80

100

Iteration axis

Fig. 9.5 Maximal tracking error along iterations 0.015 60th iteration 70th iteration 80th iteration

Tracking error e

0.01

0.005

0

−0.005

−0.01

−0.015

0

0.2

0.4

0.6

Time axis

Fig. 9.6 Tracking errors at the 60th, 70th, and 80th iterations

0.8

1

9.4 Illustrative Simulations 10

159

1

Maximal tracking error

0.12 0.06 0.02

10

10

10

0

−1

−2

0

20

40

60

80

100

Iteration axis

Fig. 9.7 Maximal tracking errors along iterations

Some interesting observations are noted by comparing the related figures for the generic ILC algorithm and the modified one. First of all, both of them are effective for achieving the precise tracking performance with sampled data as shown in Fig. 9.1. This demonstrates the effectiveness of both algorithms. Moreover, by comparing Figs. 9.2 and 9.5, we find that the convergence speed of the modified scheme is a little slower than the generic scheme. This is because that the generic algorithm (9.7) is more sensitive to the latest information as it only use the information from the last iteration for its updating, while the modified algorithm (9.29) would make an average to the information coming from adjacent iterations. However, as shown in Figs. 9.3 and 9.6, within the same iterations and for the varying time interval (i.e., 180 ≤ Nk ≤ 200), the magnitude of the tracking error profiles of the modified algorithm (9.29) is generally smaller than that of the generic algorithm (9.7). The reason lies in the fact that the average mechanism in the modified algorithm would bring us a robustness against the varying length problem, which makes a successive improvement of the tracking performance along the iteration axis. On the other hand, without such mechanism, the generic algorithm is more possible to be affected a lot when encountering bad situations. Similar performance also exists in the varying initial state case, as shown in Figs. 9.4 and 9.7, where the modified algorithm provides a more attractive improvement of the tracking performance than the generic algorithm.

160

9 Sampled-Data Control for Nonlinear Continuous-Time Systems

In addition, one may be interested in how the moving window size will affect the tracking performance in the modified algorithm (9.29). This is an important issue for further study. Generally, the design of the moving window size, i.e., r + 1, depends on the system dynamics, the nonlinearity of the system, the varying length range, the distribution of the varying length, among other factors. Thus, it is a hard work to give an explicit expression of the moving size r + 1 according to the system information and process environments. However, we may give some general guidelines for the selection of the moving window size for practical applications. First, we usually select the size from three to five. The algorithm would behaves less well when the size is too small or too large. Moreover, if the random interval of Nk is long, we usually select a large size because this case implies that the iteration length varies drastically in the iteration domain and more previous information is required to make up the missing data. Otherwise if the random interval of Nk is short, a small size is preferable to avoid redundancy of historical information. In short, the selection of the moving window size is a trade off among various factors.

9.5 Summary In this chapter, the sampled-data ILC problem for continuous-time nonlinear systems with iteration-varying lengths and higher relative degree is discussed. To achieve the control objective, two sampled-data ILC schemes are proposed with modified tracking errors, namely, the generic PD-type ILC scheme and the PD-type ILC algorithm incorporated with an iteratively moving average operator. Moreover, the probability distribution of the random trial length is not required prior in this chapter. For the identical initial state case, if the sampling period is set to be small enough and certain conditions are satisfied for the learning gains, the system output at each sampling instant has been shown to converge to the desired trajectory as the iteration number goes to infinity for both algorithms. For the varying initial state case, both algorithms are also effective in the sense that the tracking errors converge to a small zone with its upper bound being in proportion to the initial state magnitude. For further research, it is of great interest to make a deep investigation on the relationship between the moving window size and the operation environments. The results in this chapter are mainly based on [2].

References

161

References 1. Sun M, Wang D (2001) Sampled-data iterative learning control for nonlinear systems with arbitrary relative degree. Automatica 37:283–289 2. Wang L, Li X, Shen D (2018) Sampled-data iterative learning control for continuous-time nonlinear systems with iteration-varying lengths. Int J Robust Nonlinear Control 28(8):3073– 3091

Chapter 10

CEF Techniques for Parameterized Nonlinear Continuous-Time Systems

In this chapter, we proceed to propose a novel method for parameterized nonlinear continuous-time systems with varying trial lengths. As opposed to the previous chapters, this chapter is applicable to nonlinear systems that do not satisfy the globally Lipschitz continuous condition. To solve the problem, the adaptive ILC schemes are adopted in this chapter to learn the parameters and ensure an asymptotical convergence. Moreover, this chapter introduces a novel composite energy function (CEF) using newly defined virtual tracking errors for proving the convergence. Both an original update algorithm and a projection-based update algorithm for estimating the unknown parameters are proposed. Strict convergence analysis is conducted by investigating the decreasing trend of the CEF along the iteration axis. Extensions to cases with unknown input gains, iteration-varying tracking references, high-order nonlinear systems, and multi-input-multi-output systems are all elaborated upon. The general nonparameterized nonlinear continuous-time systems will be conducted in the next chapter. The contributions of this chapter covering system formulation, controller design, and analysis techniques are summarized as follows: (1) the random iteration length problem is addressed for various types of continuous-time parameterized nonlinear systems without the globally Lipschitz condition; (2) Adaptive learning controller consisting of a stabilization feedback term and a compensation feedforward term is proposed, where the estimated parameters in the controller are iteratively updated utilizing the available tracking information only; and (3) A novel modified CEF is defined with the new concept of a virtual tracking error for the untrodden part of each iteration and the newly defined CEF allows us to present an explicit difference between adjacent iterations and thus facilitates the analysis.

© Springer Nature Singapore Pte Ltd. 2019 D. Shen and X. Li, Iterative Learning Control for Systems with Iteration-Varying Trial Lengths, https://doi.org/10.1007/978-981-13-6136-4_10

163

164

10 CEF Techniques for Parameterized …

10.1 Problem Formulation To clearly demonstrate the main idea, we consider the following parametric nonlinear uncertain system x˙ = θ ◦T (t)ξ ◦ (x, t) + b(t)u(t), (10.1) x(0) = x0 , where t denotes time index, t ∈ [0, T ] with T being the desired iteration length. x ∈ R is the measurable system state, u ∈ R is the system control input, b(t) ∈ C 1 ([0, T ]) is the perturbed gain of the system input, θ ◦ (t) ∈ Rn 1 is a vector of unknown timevarying parameters, of which each entry belongs to C 1 ([0, T ]), and ξ ◦ (x, t) ∈ Rn 1 is a known vector-valued function whose elements are assumed to be locally Lipschitz continuous with respect to the state x, where C 1 ([a, b]) denotes the set of all differentiable functions over the interval [a, b]. The positive integer n 1 denotes the dimension of the vectors. The tracking reference is denoted by xr (t) ∈ C 1 ([0, T ]). As a result, the system state is fully measurable. The following assumptions on the system are required. Assumption 10.1 The input gain b(t) is unknown but the control direction (i.e., the sign of b(t)) is known and invariant. That is, b(t) is either positive or negative and nonsingular for all t ∈ [0, T ]. Without loss of generality, we assume that b(t) ≥ b > 0, ∀t ∈ [0, T ]. Here we assume that the lower bound b is known. In this chapter, we consider the iterative learning problem of system (10.1). Therefore, we add the subscript k, which denotes the iteration number, to the state and input in (10.1). Our control objective is to drive the system state xk (t) to track the desired reference xr (t) asymptotically as the iteration number k goes to infinity. Moreover, we focus on the randomly varying operation length problem. That is, the main difficulty in designing and analyzing the ILC scheme for system (10.1) is that the actual operation length Tk is iteration-varying and may be different from the desired length T . Obviously, the actual length Tk of different iterations varies randomly; thus, two cases need to be taken into account: Tk < T and Tk ≥ T . For the latter case, it is observed that only the data in the time interval [0, T ] will be used for further updating, whereas the data from (T, Tk ] will be immediately discarded. Consequently, without loss of generality, we can regard such a case as Tk = T . In other words, we can only consider the problem that Tk is not greater than T . Moreover, it is reasonable to assume that there exists a minimum of Tk , denoted by Tmin in which Tmin > 0. In short, we concentrate our discussion on the case 0 < Tmin ≤ Tk ≤ Tmax  T in this chapter. We need the following assumption on the randomly nonuniform iteration lengths Tk . Assumption 10.2 Assume that Tk is a random variable, and its probability distribution function is

10.1 Problem Formulation

165

⎧ ⎪ t ∈ [0, Tmin ), ⎨0, FTk (t)  P[Tk < t] = p(t), t ∈ [Tmin , Tmax ], ⎪ ⎩ 1, t > Tmax ,

(10.2)

where 0 ≤ p(t) ≤ 1 is a continuous function. This assumption describes the random variable of the varying-length iterations. According to the model, we have p(Tmin ) = 0 and p(Tmax ) < 1, where the latter implies that the trial length take the full length with a positive probability. We should emphasize that the probability distribution function is not required to be known a prior. In other words, no specific description is imposed on the randomly varying iteration lengths. Based on this assumption, we can further define a sequence of random variables that satisfies the Bernoulli distribution and then modify the tracking error signal to facilitate the design of ILC algorithms (see the next section). Furthermore, to gradually improve the tracking performance along the iteration axis, we need the following re-initialization assumption. Assumption 10.3 The identical initial condition, i.e., xk (0) = xr (0), ∀k, is satisfied. Assumption 10.3 is a basic requirement in ILC that has been widely used in ILC papers. This assumption implies that the system operation can be repeated. In practice, this assumption may be difficult to realize due to various reasons. The initial rectifying or learning mechanisms can be applied to solve this problem. However, this issue is beyond the scope of this chapter; thus, we simply use Assumption 10.3. Now we can formulate our problem as follows. Problem statement: The control objective of this chapter is to design suitable learning control algorithms for nonlinear system (10.1) when the operation length varies randomly for different iterations. Based on the available information of the previous iterations, the ILC algorithms should enable the system state to track the desired reference as the iteration number goes to infinity, that is, the tracking error will converge to zero along the iteration axis.

10.2 ILC Algorithm and Its Convergence Define the tracking error ek (t) = xk (t) − xr (t), t ≤ Tk .

(10.3)

Then, the error dynamics at the kth iteration is given by e˙k = x˙k − x˙r = θ ◦T ξ ◦ (xk , t) + bu k − x˙r = b(u k + b−1 θ ◦T ξ ◦k − b−1 x˙r ) = b(u k + θ T ξ k ),

(10.4)

166

10 CEF Techniques for Parameterized …

where ek (0) = 0, t ≤ Tk , ξ ◦k = ξ ◦ (xk , t), θ = [b−1 θ ◦T , −b−1 ]T , and ξ k = [ξ ◦Tk , x˙r ]T . Hereafter, we omit the arguments from the variables when no confusion is caused. The proposed learning control algorithm at the kth iteration is given by θ k ξ k , t ≤ Tk , u k = −b−1 μek −  T

(10.5)

where μ > 0 is the feedback gain and  θ k is the estimation of the unknown timevarying system uncertainty θ defined in (10.4). As the system restarts after the time instant Tk , we are not required to design the input signal for t > Tk . However, we should update the estimation of the time-varying system uncertainty θ for the entire time interval [0, T ]. Specifically, the updating law for  θ k is given by   θ k−1 + ηξ k ek , t ≤ Tk ,  θk =  Tk < t ≤ T, θ k−1 ,

(10.6)

with  θ −1 = 0, ∀t ∈ [0, T ], where η > 0 is the learning gain. Before proceeding to the convergence analysis of the proposed ILC algorithms, we first introduce a random variable γk (t) and modify the tracking error for the missing section Tk < t ≤ T . Let γk (t) be a random variable that satisfies a Bernoulli distribution and takes binary values of 0 or 1. The relation γk (t) = 1 represents the event that the operation of system (10.1) can continue until the time instant t in the kth iteration, i.e., Tk ≥ t. The probability of this event is q(t), where 0 < q(t) ≤ 1 is a predefined function of time t. The event γk (t) = 0 denotes the event that the operation of system (10.1) ends before the time instant t, which occurs with a probability of 1 − q(t). Remark 10.1 Although we do not actually require detailed information of the random variable γk (t) in the following design and analysis of ILC algorithms, we can calculate the probability P[γk (t) = 1] to clarify the inherent relation between the random iteration length Tk and the newly defined variable γk (t). From Assumption 10.2, it is evident that γk (t) is equal to 1 when 0 ≤ t < Tmin because the operation of system (10.1) will not stop within [0, Tmin ). Moreover, when t is located in [Tmin , T ], the event γk (t) = 1 implies that the operation will end at or after the time instant t. Therefore, P[γk (t) = 1] = P[Tk ≥ t] = 1 − P[Tk < t] = 1 − FTk (t). In short, we have q(t) = 1 − FTk (t). Moreover, q(Tmax ) > 0 implies that the trial interval can take the full length T with a positive probability. For the kth iteration, the operation only runs during the time interval [0, Tk ], after which time the system returns to its initial position and begins another trial. Therefore, we only have the tracking information for 0 ≤ t ≤ Tk . In addition, Tk varies randomly for different iterations. To ensure a reasonable formulation of the ILC algorithms, most existing papers have padded the missing tracking error with zeros. Contrary to the existing papers, we introduce a new complement of the missing tracking error, which will be called the virtual tracking error k (t), 0 ≤ t ≤ T , for

10.2 ILC Algorithm and Its Convergence

167

the reminder of this chapter. Specifically, the virtual tracking error k (t) is defined as follows:  ek (t), 0 ≤ t ≤ Tk , k (t) = (10.7) ek (Tk ), Tk < t ≤ T. That is, k (t) = γk (t)ek (t) + (1 − γk (t))ek (Tk ), 0 ≤ t ≤ T . We can now derive the convergence property of the proposed algorithms (10.5)– (10.6) in the following theorem. Theorem 10.1 For system (10.1), under Assumptions 10.1–10.3, the learning control law (10.5) and the updating law (10.6) guarantee that the tracking error converges to zero pointwisely over [0, T ] with probability 1 as the iteration number k goes to infinity. Proof To show the convergence, define the following CEF for the kth iteration 1 1 E k (t) = k2 (t) + 2 2η

 0

t

T b(τ ) θ k (τ ) θ k (τ )dτ,

(10.8)

θ k − θ denotes the estimation error. Note that in the CEF, the first term where θk   is a quadratic type scalar function of the virtual tracking error and the second term is actually an L2 -norm of the parameter estimation error. The proof will be carried out in two steps. In the first step, we will show that the CEF decreases as the iteration number increases, i.e., the difference of the CEF along the iteration axis is strictly negative. In the second step, we will show the boundedness of the initial CEF and then the pointwise convergence of the tracking error. Step A: Difference of the CEF. Consider the difference of E k (t) at the kth iteration: 1 1 2 ΔE k (t) =E k (t) − E k−1 (t) = k2 (t) − k−1 (t) 2 2  t 1 T T b( θk θ k−1 + θk − θ k−1 )dτ. 2η 0

(10.9)

θ k (t) have different formulations for 0 ≤ t ≤ Tk and Note that both k (t) and Tk < t ≤ T , respectively. Thus, we should separately discuss the expression of (10.9) for these two cases. We first consider the case t ≤ Tk . For this case, let us examine the first term on the right-hand-side (RHS) of (10.9). According to the identical initial condition, i.e., Assumption 10.3, the error dynamics (10.4), and the control law (10.5), we have

168

10 CEF Techniques for Parameterized …

 t 1 2 1 1 k (t) = ek2 (t) = ek2 (0) + ek e˙k dτ 2 2 2 0  t ek b(u k + θ T ξ k )dτ = 0  t T ek b(−b−1 μek −  θ k ξ k + θ T ξ k )dτ = 0  t  t T ek2 dτ − ek b θ k ξ k dτ, ≤−μ 0

(10.10)

0

where in the last inequality the fact b(t)/b ≥ 1 is applied. For the last term on the RHS of (10.9), we notice that 1 T T (θ θ k − θ k−1 θ k−1 ) 2η k 1 θk − θ k−1 )T ( θk + θ k−1 ) = ( 2η 1 θk − θ k−1 )T ( θk + θ k−1 − 2θ ) = ( 2η 1 θk − θ k−1 )T ( θ k−1 −  θ k + 2 θ k − 2θ ) = ( 2η 1   θk − θ k−1 ) =− (θ k − θ k−1 )T ( 2η 1 θk − θ k−1 )T ( θ k − θ) + ( η η T = − ξ k 2 ek2 + ek θk ξk, 2

(10.11)

where in the last equality the updating law (10.6) is applied and  ·  denotes the Euclidean norm of a vector. Thus, 1 2η



t

T T b( θk θ k−1 θk − θ k−1 )dτ 0  t  t η T 2 2 ≤− bξ k  ek dτ + bek θ k ξ k dτ. 2 0 0

(10.12)

Then, substituting (10.10) and (10.12) into (10.9) leads to 

 η t 1 2 bξ k 2 ek2 dτ − k−1 (t) 2 0 2 0  t 1 2 ek2 dτ − k−1 (t). ≤−μ 2 0

ΔE k (t) ≤ − μ

t

ek2 dτ −

(10.13)

10.2 ILC Algorithm and Its Convergence

169

Next, we proceed to the case where Tk < t ≤ T if Tk < T . For this case, the first term on the RHS of (10.9) becomes 1 2 1  (t) = ek2 (Tk ) 2 k 2   Tk ek2 dτ − ≤−μ 0

Tk 0

T ek b θ k ξ k dτ.

(10.14)

For the last term on the RHS of (10.9), we have  t 1 T T b( θk θ k−1 θk − θ k−1 )dτ 2η 0  Tk 1 T T = b( θk θ k−1 θk − θ k−1 )dτ 2η 0  t 1 T T + b( θk θ k−1 θk − θ k−1 )dτ 2η Tk  Tk 1 T T = b( θk θ k−1 θk − θ k−1 )dτ 2η 0  Tk  η Tk T ≤− bξ k 2 ek2 dτ + bek θ k ξ k dτ. 2 0 0

(10.15)

Therefore, for the second case, we derive that 

Tk

ΔE k (t) ≤ −μ 0

1 2 ek2 dτ − k−1 (t). 2

(10.16)

Combining (10.13) and (10.16) results in 

Tk ∨t

ΔE k (t) ≤ −μ 0

1 2 ek2 dτ − k−1 (t), 2

(10.17)

where Tk ∨ t  min{Tk , t}. Note that Tk is a random variable, thus the difference E k (t) is not deterministic (which is different from the existing results). Step B: Convergence of the Tracking Error. According to (10.17), it can be derived that the finiteness of E k (t) is guaranteed for any iteration provided that E 0 (t) is finite. As a consequence, we will show the finiteness of E 0 (t). To this end, we note that E 0 (t) = Hence, the derivation of E 0 (t) is

1 2 1  + 2 0 2η

 0

t

T b θ0 θ 0 dτ.

(10.18)

170

10 CEF Techniques for Parameterized …

1 T 1 T bθ 0 θ 0 = γ0 e0 e˙0 + bθ θ 0 , E˙ 0 (t) = γ02 e0 e˙0 + 2η 2η 0

(10.19)

where the second equality holds because γ0 is binary. From the error dynamics, we have

 b 2 T γ0 e0 e˙0 = γ0 − μe0 − be0 θ 0 ξ 0 b T ≤ − γ0 μe02 − γ0 be0 θ 0 ξ 0.

(10.20)

From the fact that  θ −1 = 0, the last term on the RHS of (10.19) becomes 1 T 1 T T 1 T bθ 0 θ 0 = b(θ 0 θ 0 − θ −1 θ −1 ) + bθ θ −1 2η 2η 2η −1 η 1 T T bθ θ , = − γ0 bξ 0 2 e02 + γ0 e0 b θ0 ξ0 + 2 2η

(10.21)

θ k−1 + γk ηξ k ek from (10.6) is applied. where, in the second equality, the fact  θk =  Consequently, it is evident from (10.19)–(10.21) that 1 T bθ θ . E˙ 0 (t) ≤ 2η

(10.22)

Note that b(t) and θ(t) are continuous, so (10.22) is bounded over the time interval [0, T ]. Therefore, there exists a constant such that

M = max

t∈[0,T ]

1 T bθ θ 2η

 < ∞.

Taking e0 (0) = 0 into account, we are left with  t ˙ E 0 (t) ≤ |E 0 (0)| + E 0 (τ )dτ 0  t  t E˙ 0 (τ ) dτ ≤ ≤ Mdτ ≤ M T < ∞. 0

(10.23)

0

In other words, we have shown the finiteness of E 0 (t), ∀t ∈ [0, T ]. This finiteness further implies the finiteness of E k (t), ∀k ≥ 1, ∀t ∈ [0, T ]. Hence, k (t), xk (t), and

t 2  θ  k dτ are all bounded for all k ∈ N. 0 From the expression of the difference of the CEF given in (10.17), we have E k (t) ≤ E 0 (t) − μ

k   j=1

0

T j ∨t

1 2 −  (t), 2 j=1 j−1 k

e2j dτ

(10.24)

10.2 ILC Algorithm and Its Convergence

171

or equivalently, ∀k, by the positiveness of E k (t), μ

k   j=1

T j ∨t 0

1 ek2 dτ + 2

k 

 2j−1 (t)

j=1

≤ E 0 (t) − E k (t) ≤ E 0 (t).

(10.25)

 2 2 Therefore, ∞ j=1  j (t) converges, which further implies that lim k→∞ k (t) = 0, ∀t ∈ [0, T ]. When t ∈ [0, Tmin ], it is evident that k (t) = ek (t) always holds, thus limk→∞ ek2 (t) = 0, ∀t ∈ [0, Tmin ]. When t ∈ (Tmin , T ], k (t) = ek (t) only holds for a portion of iterations with Tk ≥ t, whereas for the remaining iterations with Tk < t, we have k (t) = ek (Tk ). Therefore, ∀t ∈ (Tmin , T ], ∞  k=1

γk (t)ek2 (t) =

∞  k=1

γk (t)k2 (t) ≤

∞ 

k2 (t) < ∞,

(10.26)

k=1

which implies that limk→∞ γk (t)ek2 (t) = 0, ∀t ∈ (Tmin , T ]. Because p(t) > 0, ∀t ∈ (Tmin , T ], it is evident that there are infinite iterations for which γk (t) = 1; thus, it is concluded that limk→∞ ek2 (t) = 0. In other words, the pointwise convergence to zero of the actual tracking error ek (t) is guaranteed as the iteration number approaches to infinity. Finally, as ξ k is continuous with respect to xk , the boundedness of xk yields Lipschitz continuous condition of ξ (x, t). By the boundedness of ξ k by the

t locally θ k 2 dτ and the control law (10.5), we conclude that noting the boundedness of 0  the control signal u k is bounded with respect to the L2 -norm. This theorem implies that the available tracking performance can be gradually improved along the iteration axis even though the iteration length randomly varies over different iterations (see Case 1 in the simulations). That is, when the operation ends at time instant Tk , the energy E k (t) is decreased for t ∈ [0, Tk ] compared with the previous iteration. As can be seen from the control law (10.5) and the updating law (10.6), the improvement occurs during the actual operation interval [0, Tk ], whereas for the left section, no updating is imposed. In other words, for the time instants located in (Tmin , T ], the updating frequency is lower than that in [0, Tmin ] (depending on the probability distribution of Tk ), but the updating will continue to occur. As a consequence, zero-error tracking performance can be achieved. It can be noticed from the proof that we can primarily guarantee the L2 -norm boundedness of the parameter estimation. In other words, we do not have knowledge on the boundedness of  θ k . On the one hand, this can be viewed as an advantage because we do not need to require prior knowledge of the system parameter θ . On the other hand, we are interested in whether stronger convergence can be obtained if we incorporate additional system bounding information for the parameter. Indeed, if we know the upper and lower bounds of the unknown system parameters, we

172

10 CEF Techniques for Parameterized …

can modify the original updating law (10.6) into a projection-based update law. Specifically, the modified version of (10.6) is 

P( θ k−1 ) + ηξ k ek , t ≤ Tk ,  θk = Tk < t ≤ T, P( θ k−1 ),

(10.27)

where the operator P(φ) for a vector φ = [φ1 , . . . , φn ]T is defined as P(φ) = [P(φ1 ), . . . , P(φn )]T ,  φi , |φi | ≤ φi∗ , P(φi ) = ∗ φi · sign(φi ), |φi | > φi∗ , where sign(·) is the sign function and φi∗ (i = 1, . . . , n) are the known projection bounds. In this case, the parameter estimation is naturally bounded and then the boundedness of other associated variables is guaranteed. Consequently, the Barbalat lemma can be applied to derive the uniform convergence. This idea is shown in the following theorem for the control law (10.5) and the new updating law (10.27). Theorem 10.2 For system (10.1), under Assumptions 10.1–10.3, the learning control law (10.5) and the updating law (10.27) guarantee that the tracking error converges to zero uniformly over [0, T ] with probability 1 as the iteration number k goes to infinity. Proof We apply the same CEF defined in (10.8), and the derivations in the proof of Theorem 10.1 still hold except for the difference of the parameter estimation error

t 1 T part (i.e., 2η 0 b(τ )θ k (τ )θ k (τ )dτ ). To show the validity of (10.12) and (10.15), T T θ θ we must check the difference 1 ( θk − θ k−1 ). Specifically, noting the basic 2η

k

k−1

property [φ − ψ]2 ≥ [φ − P(ψ)]2 for any suitable vectors φ and ψ, we have T θ k−1 − θ]T [ θ k−1 − θ ] θ k−1 = [ θ k−1  T   ≥ P( θ k−1 ) − θ P( θ k−1 ) − θ ,

which further implies that T T θ k−1 θk − θ k−1 θk  T   T P( θ k−1 ) − θ ≤ θk θ k−1 ) − θ θ k − P(  T   = θ k − P( θ k + P( θ k−1 ) + θ θ k−1 ) − θ  T   =  θ k − P( θ k + P( θ k−1 )  θ k−1 ) − 2θ  T   =−  θ k − P( θ k−1 )  θ k−1 ) θ k − P(  T   +2  θ k − P( θk − θ . θ k−1 ) 

10.2 ILC Algorithm and Its Convergence

173

Therefore, for the case of 0 ≤ t ≤ Tk , we can easily show the validity of (10.12) when the updating law (10.27) is applied. For the case when Tk < t ≤ T , the relation (10.15) can be also verified by noting that  t 1 T T b( θk θ k−1 θk − θ k−1 )dτ 2η Tk  t   T   1 P( θ k−1 ) − θ b P( θ k−1 ) − θ = 2η Tk T     θ k−1 − θ dτ ≤ 0. −  θ k−1 − θ Consequently, a similar result as (10.17) is obtained: 

Tk ∨t

ΔE k (t) ≤ − μ 0

ek2 dτ −

1 2 − k−1 (t) ≤ 0. 2

η 2

 0

Tk ∨t

bξ k 2 ek2 dτ (10.28)

With this result, the pointwise convergence of ek can be proved according to Theorem 10.1. The boundedness of ek (t) implies the boundedness of xk (t) and, therefore, ensures θ k , u k (t), and x˙k (t) according to the locally Lipschitz the boundedness of ξ k (t),  continuity of ξ k (t) with respect to xk , the updating law (10.27), the control law (10.5), and the system dynamics (10.1), respectively. Moreover, the boundedness of x˙k (t) implies the uniform continuity of xk (t) and, thereafter, the uniform continuity of the tracking error ek (t). In other words, the uniform convergence is guaranteed. This completes the proof. This theorem implies that if we know a prior bound information of the unknown parameters, then the projection-based type updating law will ensure boundedness of all involved quantities. This property further leads to uniform convergence of the tracking error over the operation interval by the Barbalat lemma, which is stronger than the pointwise convergence obtained by the original algorithms. This indicates the inherent connection between Theorems 10.1 and 10.2; that is, Theorem 10.1 is for non-projection-based algorithms (with pointwise convergence) while Theorem 10.2 is for projection-based algorithms (with uniform convergence). Observing the problem formulation, we find that several issues such as input gain information, tracking references, and systems can be further extended to a wide variety of applications. These extensions will be detailed in Sect. 10.4.

174

10 CEF Techniques for Parameterized …

10.3 Effect of Random Trial Lengths and Parameters Although we have defined the probability distribution function for the random trial length, the specific distribution is not much involved in the convergence proofs. Thus, the effect of the random trial length to the tracking performance is not clear. In this section, we make a brief clarification on this issue. We have shown the convergence following the main procedures of CEF approach, where a direct difference of the CEF for two adjacent iterations is derived. Although we establish a decreasing trend of the CEF, it does not necessarily imply the successive decrease of the tracking error. This is because the decrease of the CEF may be caused by a short trial length. However, generally speaking, the tracking performance is heavily influenced by the update frequency, whereas the update frequency is strongly related with the probability distribution of the trial length variable. We notice a general formulation of random trial lengths is given in Sect. 10.1. Two basic observations can be found. First, the smaller the time instant t, the higher the update frequency for the input at time instant t. Second, the tracking performance of the whole trial can be evaluated from the occurrence probability of full length (i.e., q(Tmax )). Generally, a larger value of q(Tmax ) implies a higher frequency of full trial length, which leads to a better tracking performance within same iterations. Other quantities involved with the tracking performance include the lower bound of the unknown input gain (i.e., b), the feedback gain in control laws (i.e., μ), and the learning gain in parameter update laws (i.e., η). Generally, a more precise lower bound b leads to a more efficient learning process as the input exhibit better performance. If the lower bound b is far away from the true value, the feedback term main dominate the control so that the learning part exhibit a slow improvement to the tracking performance. Moreover, the feedback gain μ tunes the effect of the feedback term together with b as can be seen from control laws. If a prior estimate of b is small, we may select a suitably small μ to balance the effect. Further, a larger selection of the learning gain η would generally yield a faster convergence speed of the parameter estimation process; however, the fast convergence is not always good for the performance improvement. Thus, the selection of the learning gain indicates a trade-off between the tracking performance and parameter estimation.

10.4 Extensions and Discussions In this section, we will outline the extensions of our results. First, in Assumption 10.1, the input gain b(t) is assumed to have its lower bound as b. The removal of this information is detailed in Sect. 10.4.1 (Theorem 10.3). Moreover, we require the desired reference to be invariant along the iteration axis, while in many practical applications, the desired reference may vary from iteration to iteration. Such a case will be discussed in Sect. 10.4.2. Further, system (10.1) is a first-order system and we would like to provide some discussions on high-order systems, which are provided

10.4 Extensions and Discussions

175

in Sect. 10.4.3. The extension to general multi-input-multi-output systems is given in Sect. 10.4.4. Finally, in system (10.1), only parametric uncertainties are taken into account, so the extension to nonparametric systems will be given in Sect. 10.4.5 (Theorem 10.4). Clearly, both Theorems 10.3 and 10.4 present similar results to Theorem 10.1 for relaxed system formulations.

10.4.1 Unknown Lower Bound of the Input Gain In Assumption 10.1 of Sect. 10.1, the lower bound of the input gain function b(t), i.e., b, is involved in the control law (10.5). The lower-bound information and associated control law ensure the demonstration of a concise design and analysis for the tracking problem. However, in some practical applications, prior knowledge of the lower bound is unavailable. In such a case, we may provide an additional estimate of the input gain or, equivalently, its inverse. As a consequence, the convergence analysis involves extra steps for the derivative of the input gain. The details are given below. Specifically, the proposed learning control law (10.5) at the kth iteration is replaced by the following law T ϑ k ξ k , t ≤ Tk , (10.29) u k = −μek −  where μ > 0 is the feedback gain,  ϑ k is the estimation of the time-varying parameter ˙ T , and ξ k  [ξ ◦Tk , x˙r , ek /2]T . ϑ  [b−1 θ ◦T , −b−1 , −b−2 b] The updating law for parameter estimation  ϑ k is given by  ϑk =  ϑ k−1 + γk ηξ k ek ,

(10.30)

with  ϑ −1 = 0, ∀t ∈ [0, T ]. The convergence is summarized in the following theorem. Theorem 10.3 For system (10.1), under Assumptions 10.1–10.3, where the lower bound information of b(t) is removed, the learning control law (10.29) and the updating law (10.30) guarantee that the tracking error converges to zero pointwisely over [0, T ] with probability 1 as the iteration number k goes to infinity. Proof Because the derivative of the input gain function is involved, we need to modify the CEF as follows: 1 −1 1 E k (t) = b (t)k2 (t) + 2 2η



t 0

T ϑ k (τ ) ϑ k (τ )dτ,

(10.31)

where b(t) is modified from b(t), b(t) = γk (t)b(t) + (1 − γk (t))b(Tk ), 0 ≤ t ≤ T . The proof can be carried out similar to that given for Theorem 10.1. In the following, we mainly present the differences between the two proofs, where the same steps are omitted for brevity.

176

10 CEF Techniques for Parameterized …

Step A: Difference of the CEF. Consider the difference of E k (t): ΔE k (t) =E k (t) − E k−1 (t) 1 −1 1 −1 2 = b (t)k2 (t) − b (t)k−1 (t) 2 2  t 1 T T ϑk − ϑ k−1 )dτ. ( ϑk ϑ k−1 + 2η 0

(10.32)

We first consider the case 0 ≤ t ≤ Tk . In this case, the first term on the RHS of (10.32) can be rewritten as 1 −1 1 b (t)k2 (t) = b−1 (t)ek2 (t) 2 2  t  1 t −2 ˙ 2 1 −1 b ek e˙k dτ − b bek dτ + b−1 (0)ek2 (0) = 2 2 0 0  t  1 t −2 ˙ 2 −1 ◦T ◦ ek (u k + b θ ξ k − b−1 x˙r )dτ − b bek dτ = 2 0 0  t  t  t T ϑ k ξ k ek dτ, ek (u k + ϑ T ξ k )dτ = −μ ek2 dτ − = 0

0

(10.33)

0

where ϑk   ϑ k − ϑ. Similar to the proof of Theorem 10.1, it is easy to derive that 

1 2η

t

T T ϑk − ϑ k−1 )dτ ( ϑk ϑ k−1 0  t  t η T 2 2 ϑ k ξ k ek dτ. =− ξ k  ek dτ + 2 0 0

(10.34)

Therefore, for 0 ≤ t ≤ Tk , 

t

ΔE k (t) = −μ 0

ek2 dτ −

η 2



t 0

1 −1 2 ξ k 2 ek2 dτ − b k−1 . 2

(10.35)

Next, we consider the case Tk < t ≤ T . For this case, the first term on the RHS of (10.32) is 1 −1 1 b (t)k2 (t) = b−1 (Tk )ek2 (Tk ) 2 2   Tk 2 ek dτ − =−μ 0

Tk 0

T ϑ k ξ k ek dτ.

(10.36)

10.4 Extensions and Discussions

177

Moreover, we have  t 1 T T ϑk − ϑ k−1 )dτ ( ϑ ϑ k−1 2η 0 k  Tk 1 T T ϑk − ϑ k−1 )dτ = ( ϑk ϑ k−1 2η 0  Tk  Tk η T 2 2 ϑ k ξ k ek dτ. =− ξ k  ek dτ + 2 0 0

(10.37)

where the relationship  ϑ k (t) =  ϑ k−1 (t) for Tk < t ≤ T from (10.30) is applied to the second equality. This further leads to 

Tk

ΔE k (t) = − μ 0

ek2 dτ −

η 2

 0

Tk

ξ k 2 ek2 dτ

1 −1 2 − b k−1 , Tk < t ≤ T. 2

(10.38)

Combining (10.35) and (10.38), we have 

Tk ∨t

ΔE k (t) = − μ 0

ek2 dτ −

η 2

 0

Tk ∨t

ξ k 2 ek2 dτ

1 −1 2 − b (t)k−1 (t). 2

(10.39)

Step B: Convergence of the Tracking Error. The proof can be completed following similar derivations to those given in Theorem 10.1, provided that we can show the finiteness of E 0 (t), ∀t ∈ [0, T ]. To demonstrate this point, we note from the definition of E k (t) that 1 −1 1 E 0 (t) = b (t)02 (t) + 2 2η



t 0

T ϑ 0 (τ ) ϑ 0 (τ )dτ,

(10.40)

and 1 T ˙ 02 + 1 ϑ ϑ 0. E˙ 0 (t) = γ0 b−1 e0 e˙0 − γ0 b−2 be 2 2η 0

(10.41)

From the error dynamics (10.4) and the control law (10.29), we have   1 T ˙ 02 = γ0 −μe02 − γ0 b−1 e0 e˙0 − γ0 b−2 be ϑ 0 ξ 0 e0 . 2

(10.42)

178

10 CEF Techniques for Parameterized …

From the fact that ϑ −1 = 0, it can be derived that  1 T 1 T 1  T T ϑ0 ϑ0 = ϑ0 ϑ0 − ϑ ϑ −1 ϑ −1 + ϑ −1 2η 2η 2η −1 η 1 T T ϑ 0 ξ 0 e0 + = − γ0 ξ 0 2 e02 + γ0 ϑ ϑ. 2 2η

(10.43)

Consequently, 1 T ϑ ϑ. E˙ 0 (t) ≤ 2η

(10.44)

Therefore, E 0 (t) is finite for all t ∈ [0, T ]. The reminder of the proof is the same as that in Theorem 10.1. Thus, the proof is completed. As can be seen from the proof, when the prior information about the lower bound of the input gain function is unknown, we must introduce an additional estimate of the corresponding signal (see the definition of ϑ) and an associated compensation in the control law (see (10.29)). Accordingly, the convergence analysis would involve extra terms (see (10.33) for example). However, the inherent convergence principle is unchanged (see the verification from Case 2 in simulations). Moreover, the pointwise convergence is obtained in the above theorem. The convergence can be improved if we can use a projection-type updating law for ϑ similar to (10.27), provided that the bound information of unknown parameters is known. Here, we omit the repetition for brevity.

10.4.2 Iteration-Varying Tracking References In the previous chapters, we assumed that the tracking reference xr is invariant along the iteration axis, which is the typical ILC problem. Generally, in ILC, the invariance of the tracking reference makes it possible to learn the tracking objective based on the extra updating activity in the iteration domain. In this chapter, we formulate the problem according to a parametric system (10.1) and provide an indirect control strategy to solve the problem. That is, the control law (10.5) comprises a feedback T θ k ξ k , while the unknown time-varying but term −b−1 μek and a compensation term − iteration-invariant system parameter θ is inherently learned using trial information (as shown in the updating law (10.6)). In short, it is the intrinsic system information θ rather than the tracking reference xr that is learned by the proposed iterative learning algorithms. Thus, we are interested in whether the invariant reference xr can be extended. In fact, from the learning control algorithms, it is seen that the tracking reference xr is incorporated into ξ k as a part of the known information. Therefore, it is possible to learn even if the tracking reference varies from iteration to iteration. The

10.4 Extensions and Discussions

179

essential reason can be understood from two aspects. First, the references have been transformed as a piece of known information for correcting the unknown parameters (see ξ k defined later). Second, the indirect adaptive learning scheme proposed in this chapter can well learn the unknown parameter θ , which is invariant in the iteration domain. With these two points, the feedback-type control law (10.5) with a compensation term is effective in dealing with the varying references. Now, we provide the necessary modifications to clarify our idea. As the tracking reference could vary from iteration to iteration, the reference for the kth iteration is xr,k ∈ C 1 ([0, T ]). Then, the tracking error for the kth iteration is denoted as ek = xk − xr,k . The error dynamics at the kth iteration is given by e˙k =x˙k − x˙r,k = θ ◦T ξ ◦k + bu k − x˙r,k = b(u k + θ T ξ k ), where ξ k = [ξ ◦k T , x˙r,k ]T . Then the proposed control law is u k = −b−1 μek −  θ k ξ k , t ≤ Tk . T

(10.45)

The updating law for  θ k is   θ k−1 + ηξ k ek , t ≤ Tk ,  θk =  Tk < t ≤ T, θ k−1 ,

(10.46)

with  θ −1 = 0, ∀t ∈ [0, T ]. By following the exact same steps as in Sect. 10.2, we can demonstrate the pointwise convergence of the tracking error along the iteration axis for iteration-varying tracking references. Moreover, if we apply the projection-type updating law 

P( θ k−1 ) + ηξ k ek , t ≤ Tk ,  θk = Tk < t ≤ T, P( θ k−1 ), where the projection P(·) is defined as before, then the uniform convergence can be obtained similarly to the derivations in Sect. 10.2. The details of the steps are omitted here for brevity. The illustration of varying references is given by Case 3 in the simulation section.

10.4.3 High-Order Systems To clearly express our basic idea, we considered a first-order system (10.1) in Sect. 10.1. In this subsection, we will demonstrate that the results can be easily extended to the following class of high-order systems

180

10 CEF Techniques for Parameterized …

x˙ j =x j+1 , x˙n =θ

◦T

j = 1, . . . , n − 1, ◦

(t)ξ (x, t) + b(t)u(t),

where x = [x1 , . . . , xn ]T ∈ Rn denotes the state vector and x j is the jth dimension of the state x. To indicate the iteration index for the reminder of this subsection, we rewrite x j as x j,k for the jth dimension of the state x k at the kth iteration. Other notations are defined similarly by adding the subscript k for indicating iteration index. It is evident that the high-order system given above is in a parametric, strictfeedback form. The reference signal is defined as x r = [xr , xr(1) , . . . , xr(n−1) ]T , where ( j) ( j−1) , j ≥ 1, with xr(0) = xr . xr  x˙r To handle such high-order systems, we define the extended tracking error (ETE) σk (t) =

n 

c j e j,k (t), cn = 1,

(10.47)

j=1

where c j , j = 1, . . . , n, are the coefficients of a Hurwitz polynomial and e j,k (t) is ( j−1) . the tracking error for the jth dimension at the kth iteration, e j,k (t) = x j,k − xr The dynamics of the ETE σk (t) can be calculated as follows: σ˙ k (t) =

n 

c j e˙ j,k (t) =

j=1

=

n−1 

n−1 

c j e j+1,k (t) + e˙n,k (t)

j=1

c j e j+1,k (t) + x˙n,k (t) − x˙r(n−1)

j=1

=

n−1 

c j e j+1,k (t) + θ ◦T ξ ◦ (x k , t) + b(t)u k (t) − xr(n)

j=1 n−1   −1 = b(t) u k + b c j e j+1,k (t) j=1

 ξ (x k , t) − b−1 xr(n) .

−1 ◦T ◦

+b θ

(10.48)

In short, a similar formulation as (10.4) is obtained. Based on this ETE dynamics (10.48), we can propose the control law and the parameter updating law similar to that in (10.5) and (10.6), respectively. The details are not provided here for brevity. The zero-error convergence of x(t) to the desired reference x r is guaranteed by the zero-error convergence of σ (t).

10.4 Extensions and Discussions

181

10.4.4 Multi-input-Multi-output Systems The high-order system given in Sect. 10.4.3 actually is a single-input-multi-output system. In this subsection, we brief the possibility of extensions to multi-input-multioutput system. In this case, the input gain has become a distribution matrix rather than a scalar. Consider the following dynamic system, x˙ k (t) = Ξ (x k , t)ϕ(t) + Bk uk (t),

(10.49)

where x k (t) ∈ Rn is the measurable state, Ξ (x k , t) ∈ Rn×n 1 is a known matrix of nonlinear functions of the state, ϕ(t) ∈ Rn 1 is a vector of unknown time-varying parameters, Bk  B(x k , t) is a known and invertible input distribution matrix, and uk (t) ∈ Rn is the input. Denote the tracking reference x r . Then, we obtain the error dynamics: e˙ k = x˙ k (t) − x˙ r (t) = Ξ (x k , t)ϕ(t) + Bk uk (t) − x˙ r (t). Provided that Bk is known and invertible, we can propose the following control law: ϕ k (t) + x˙ r (t)], uk (t) = Bk−1 [−Γ ek (t) − Ξ (x k , t) where  ϕ k (t) is the estimation of unknown parameters ϕ(t), recursively defined by   ϕ k−1 (t) + ηΞ T (x k , t)ek (t), t ≤ Tk ,  ϕ k (t) =  Tk < t ≤ T. ϕ k−1 (t), The convergence proof is similar to the previous sections. Note that the input and state in (10.49) have the same dimension. In fact, we can consider a general case: x˙ k (t) = Ξ (x k , t)ϕ(t) + Bk [(I + Hk )uk (t) + d k ], where uk (t) ∈ Rm , m ≤ n is the input. Bk ∈ Rn×m is a known left invertible input distribution matrix, Hk  H (x k , t) ∈ Rm×m represents uncertainties in the input distribution matrix, and d k is the system uncertainty. For this general case, the conditions and techniques in [1, 2] can be borrowed to complete the convergence incorporating with the newly proposed CEF. We omit the details to save space.

10.4.5 Parametric Systems with Nonparametric Uncertainty In this subsection, we consider a natural extension to the original system (10.1) that the nonparametric uncertainty is taken into account. Specifically, in (10.1), only

182

10 CEF Techniques for Parameterized …

the parametric uncertainty is taken into account. In practical applications, some uncertainties cannot be parameterized; therefore, we need to consider nonparametric uncertainty. We note that the nonparameterized systems will be elaborated in the next chapter and thus, in this subsection, we mainly focus on the main idea of the application of adaptive ILC scheme. A promising approach for controlling general nonlinear systems is the so-called function-approximation-based approach such as adaptive fuzzy control or adaptive neural networks control, where the general nonlinear functions are approximated by a weighted-summation of base functions (i.e., fuzzy rules or artificial neurons). In such a case, the approximation part is parametric while the approximation error is nonparametric. To consider the case with nonparametric uncertainty, we consider the following model of the system, x˙k (t) = θ ◦T (t)ξ ◦ (xk , t) + ρ(xk , t) + b(t)u k (t),

(10.50)

where xk (0) = x0 , ρ(xk , t) ∈ R denotes the time-varying, state-dependent, and unknown lumped system uncertainty, while the remaining notations are the same as in (10.1). We require the following assumption on the lumped system uncertainty ρ(x, t). Assumption 10.4 The nonlinear function ρ(x, t) satisfies the following condition: |ρ(z 1 , t) − ρ(z 2 , t)| ≤ φ(z)|z 1 − z 2 |, t ∈ [0, T ],

(10.51)

where z = [z 1 , z 2 ]T and φ : R2 → R≥0 is a known continuous function that is radially unbounded. We should note that Assumption 10.4 is much weaker than the widely applied globally Lipschitz condition because φ(·) can approach infinity when the state of the system is unbounded. Unlike the parametric part, where we establish an updating mechanism to learn the unknown parameters, for this nonparametric part, we must introduce a compensation or robust term to the control law. The details are given as follows. The tracking error is defined as ek = xk − xr . The error dynamics is e˙k = x˙k − x˙r = θ ◦T ξ ◦ (xk , t) + ρ(xk , t) + bu k − x˙r = θ ◦T ξ ◦ (xk , t) + ρ(xr , t) + bu k − x˙r + ρ(xk , t) − ρ(xd , t) = b[u k +  T ψ k ] + ρ(xk , t) − ρ(xd , t),

(10.52)

where  = [b−1 ξ ◦T , −b−1 , b−1 ρ(xr , t)]T , ψ k = [ξ

◦T

, x˙r , 1] . T

(10.53) (10.54)

10.4 Extensions and Discussions

183

It can be seen that the above error dynamics contain both parametric and nonparametric uncertainties. By a suitable transformation, it is possible to incorporate adaptive ILC with a robust term to handle both the parametric part  T ψ k and the nonparametric part ρ(xk , t) − ρ(xd , t). The control law is given by  kT ψ k − b−1 φ(xk )ek , u k = −b−1 μek − 

(10.55)

 k is the estimation of  at the kth iteration and φ(xk ) is defined as where  φ(xk )  sup φ(xk , z), |z|≤m r

with m r  supt |xr (t)|. Substituting the above control law (10.55) into the error dynamics (10.52) leads to e˙k = b[u k +  T ψ k ] + ρ(xk , t) − ρ(xd , t) b kT ψ k + ρ(xk , t) − ρ(xd , t). = − (μ + φ(xk ))ek − b b

(10.56)

Moreover, the updating law for the unknown parameters is similar to (10.6) given by  k = 

 k−1 + ηψ k ek , t ≤ Tk ,   k−1 , Tk < t ≤ T, 

(10.57)

 −1 = 0, ∀t ∈ [0, T ]. with the initial value  Now we can present the following convergence theorem. Theorem 10.4 For system (10.50), under Assumptions 10.1–10.4, the learning control law (10.55) and the updating law (10.57) guarantee that the tracking error converges to zero pointwisely over [0, T ] with probability 1 as the iteration number k goes to infinity. Proof We employ the same CEF as given in (10.8), i.e., E k (t) =

1 2 1  (t) + 2 k 2η



t 0

kT (τ ) k (τ )dτ, b(τ )

(10.58)

 k −  is the estimation error of the unknown parameters. k   where  The proof comprises two steps. In Step A, the difference of the CEF (10.58) between two consecutive iterations is derived. In Step B, the boundedness of the related signals and the convergence is further proved.

184

10 CEF Techniques for Parameterized …

Step A: Difference of the CEF. From (10.58), we have  kT (τ ) k (τ ) b(τ )  0  T k−1 (τ ) dτ. k−1 (τ ) −

1 1 2 1 (t) + ΔE k (t) = k2 (t) − k−1 2 2 2η



t

(10.59)

We first consider the case 0 ≤ t ≤ Tk . According to the identical initial condition (10.3), we have 1 2 1 1 k (t) = ek2 (t) = ek2 (0) + 2 2 2



t

 ek e˙k dτ =

0

t

ek e˙k dτ,

(10.60)

0

while the error dynamics (10.56) leads to b kT ψ k ek ek e˙k = − (μ + φ(xk ))ek2 − b b + [ρ(xk , t) − ρ(xd , t)]ek .

(10.61)

By taking into account Assumption 10.4, we have |[ρ(xk , t) − ρ(xd , t)]ek | ≤ φ(xk , xr )|xk − xr | · |ek | ≤ φ(xk )ek2 .

(10.62)

Substituting (10.62) into (10.61) yields kT ψ k ek . ek e˙k ≤ −μek2 − b

(10.63)

Following derivations similar to those used for (10.11) and (10.12), we have that 1 2η



t

T kT (τ ) k (τ ) −  k−1 k−1 (τ )]dτ b(τ )[ (τ ) 0  t  η t kT ψ k dτ. ≤− bψ2 ek2 dτ + bek  2 0 0

(10.64)

Combining (10.59) to (10.64), for 0 ≤ t ≤ Tk , 

t

ΔE k (t) ≤ −μ 0

ek2 dτ −

η 2



t 0

1 2 bψ k 2 ek2 dτ − k−1 (t). 2

(10.65)

10.4 Extensions and Discussions

185

Similarly, for the case Tk < t ≤ T , we can derive that 

Tk

ΔE k (t) ≤ −μ 0

ek2 dτ

η − 2



Tk 0

1 2 bψ k 2 ek2 dτ − k−1 (t). 2

(10.66)

In short, we can arrive at the conclusion that 

Tk ∨t

ΔE k (t) ≤ − μ 0

ek2 dτ −

η 2



Tk ∨t 0

bψ k 2 ek2 dτ

1 2 − k−1 (t). 2

(10.67)

Step B: Convergence of the Tracking Error. The proof can be completed following similar derivations to those given in Theorem 10.1, provided that the finiteness of the initial CEF, i.e., E 0 (t), t ∈ [0, T ] is valid. To this end, we investigate the derivative of E 0 (t), 1 0T  0. b E˙ 0 (t) =γ0 e0 e˙0 + 2η

(10.68)

Applying the error dynamics (10.56) leads to  b 0T ψ 0 γ0 e0 e˙0 = γ0 e0 − (μ + φ(x0 ))e0 − b b  + ρ(x0 , t) − ρ(xr , t) 0T ψ 0 . ≤ − γ0 μe02 − γ0 e0 b

(10.69)

 −1 = 0, we have On the contrary, from the fact  1 1 1 T 0T  0T  −1 0 = b[ 0 −  −1 ] + b b T   2η 2η 2η 1 0T ψ 0 + ≤ γ0 e0 b b T  . 2η

(10.70)

Combining (10.68)–(10.70) yields E˙ 0 (t) ≤ 1/(2η) · b T  , ∀t ∈ [0, T ]. The finiteness of E 0 (t), ∀t ∈ [0, T ] is then guaranteed. Thus, the proof is completed following similar derivations to those for the proof of Theorem 10.1. This theorem reveals that a robust term can be introduced to compensate the nonparametric lumped system uncertainty. Thus the control law (10.55) is a com kT ψ k and a robust bination of a feedback term bμek , a learning correction term  −1 term b φ(xk )ek to ensure accurate tracking, learn parametric uncertainty, and reject lumped uncertainty, respectively. Guiding by this design idea, we are able to deal with more types of uncertainties. The results will be reported in the next chapter. In

186

10 CEF Techniques for Parameterized …

addition, the extension to the projection-based learning law case, which ensures uniform convergence along the iteration axis, is completely identical to the discussions in Sect. 10.2; thus, the details are omitted here for brevity. The illustration is given by Case 4 in the next section.

10.5 Illustrative Simulations In this section, to verify the effectiveness of the proposed schemes, we consider their application to the following one-link robotic manipulator: 

x˙1 x˙2





01 = 00



   0 x1 + [u − gl cos x1 + η1 ] , 1 x2 ml 2 +I

where x1 is the joint angle, x2 is the angular velocity, m is the mass, l is the length, I is the moment of inertia, u is the joint input and η1 = 5x12 sin3 (5t) is a disturbance. As a result, the input gain is b = 1/(ml 2 + I ). In this simulation, we let m = (3 + 0.1 sin t) kg, l = 1 m, I = 0.5 kg·m2 , and g be the gravitational acceleration. The ETE is defined as δ = 3e1 + e2 , where e1 and e2 are the tracking errors for the first and second dimensions of the state, respectively. Obviously, δ˙k = 3e˙1,k + e˙2,k = b(3b−1 e2,k + b−1 x˙r,2 − u k + gl cos x1,k − η1,k ). We first consider the iteration-invariant tracking case in which desired trajectories for the first and second states are xr,1 = 0.05e−t (2π t 3 − t 4 )(2π − t) and xr,2 = 0.05e−t [12π 2 t 2 − (16π + 4π 2 )t 3 + (4π + 5)t 4 − t 5 ], respectively. The initial state is set to be 0. The desired operation length is 2π s. However, the actual operation length varies randomly for different iterations. In this simulation, we assume that the operation length obeys a uniform distribution on the time interval [2π − 3, 2π ] with a probability of P[Tk = T ] = 0.4. Case 1: the original scheme (10.5)–(10.6). In this case, the estimated parameter θ = [3b−1 , b−1 , gl, −5 sin3 (5t)]T and the associated information is ξ k = [e2,k , x˙r,2 , 2 T ] . For the algorithms, we choose η = 20, μ = 2, and b = 0.25. The cos x1,k , x1,k convergence of the maximum and virtual average ETE is shown in Fig. 10.1. Here, the maximum ETE is defined as max0≤t≤Tk |δk (t)|, and it can be seen from the plot that the maximum ETE quickly decreases as the iteration number increases. However, the maximum ETE for each iteration demonstrates the worst tracking performance of the scheme over the entire time interval. To display more properties of the scheme, we introduce the virtual average ETE as follows: we sample the ETE with a period ΔT = 0.001 s and then take the mean of the absolute values of all sampled ETEs. In other words, the virtual mean is defined as 1/ 2π/0.001 · 1≤ j≤ 2π/0.001 |δk ( j · ΔT )| with · being the floor-rounding function. From Fig. 10.1, it is apparent that the average ETE converges to zero very quickly. We have also simulated different parameters (μ and η) to verify their effect on tracking performance. It is found that larger μ and η would lead to a faster convergence speed, as explained in Sect. 10.3. Thus, one may select suitably large parameter to accelerate the learning process

10.5 Illustrative Simulations

187

0.45 maximum ETE average ETE

0.4

max|δ(t)| and ave|δ(t)|

0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 5

10

15

20

25

30

35

40

45

50

Iteration axis

Fig. 10.1 Maximum and average ETE for Case 1

provided with acceptable transient performance. Due to limited space, the detailed simulation figures are not copied here. To further demonstrate the randomly varying iteration length problem and verify the trajectory-based error performance, we plot the ETE profiles for the 1st, 3rd, 5th, 10th, and 50th iterations in Fig. 10.2. Two facts can be observed from the figure. The first one is that the error profiles converge to zero very quickly after only a few learning iterations. The second one is that the iteration ends at different time instants for different iterations, as demonstrated by the colored solid circles. The tracking performance of the illustrative iterations are shown in Fig. 10.3. It can be seen that the proposed scheme can enable asymptotically precise tracking in the iteration domain. Two points are noted before concluding this case. The first one is that we actually have the precise information 5 sin3 (5t) in the parameter vector θ . Thus, the estimation of such a component can be replaced with its true value to save the learning burden and increase convergence speed. The second one is that the projection-based scheme ((10.5) and (10.27)) performs similarly as above; thus, we omit the results for brevity. Case 2: unknown lower bound of the input gain. In this case, the lower bound of the input gain b = 0.25 is removed. Then, according to (10.29) and (10.30), the param˙T eter and associated information turn into ϑ = [3b−1 , b−1 , gl, −5 sin3 (5t), −b−2 b] 2 T and ξ k = [e2,k , x˙r,2 , cos x1,k , x1,k , δk /2] , respectively. Then, the scheme given by (10.29) and (10.30) is applied with η = 20 and μ = 2. The convergence of the ETE is demonstrated in Fig. 10.4. A similar quick reduction of the ETE is observed. Case 3: iteration-varying tracking reference. In this case, we consider the nonuniform tracking problem. To this end, we first generate a random coefficient κk from the

188

10 CEF Techniques for Parameterized … 0.05 0 −0.05 −0.1

δ(t)

−0.15 −0.2 −0.25 −0.3 1st iteration 3rd iteration 5th iteration 10th iteration 50th iteration

−0.35 −0.4 −0.45

0

1

2

3

4

5

6

Time axis

Fig. 10.2 ETE profiles of illustrative iterations for Case 1

interval of [−1, 0) ∪ (0, 1]. Then, for odd iterations, the desired trajectories are given as xr,1,k = κk sin3 (0.5t) and xr,2,k = 1.5κk sin2 (0.5t) cos(0.5t), whereas for even iterations the desired trajectories are given as xr,1,k = 0.05κk e−t (2π t 3 − t 4 )(2π − t) and xr,2,k = 0.05κk e−t [12π 2 t 2 − (16π + 4π 2 )t 3 + (4π + 5)t 4 − t 5 ]. An illustrative example of the desired trajectories of the first four iterations of xr,2 are shown in Fig. 10.5. Even for the iteration-varying references tracking problem, the proposed scheme with η = 20 and μ = 2 still presents a favourable learning ability. This conclusion is verified by Fig. 10.6, in which the maximum and virtual average ETE profile along the iteration axis are plotted. As can be seen, the convergence property holds for this case. Case 4: with nonparametric uncertainty. In this case, we assume that there exists a nonparametric part in the system model and then apply the scheme given by (10.55) and (10.57). For simplicity, we use the primary model. The parameter and associated information for this case are ψ = [3b−1 , b−1 , gl, −5 sin3 (5t), −b−1 ρ(xr,2 , t)]T and 2 , 1]T , respectively. The bounding function is given by  = [e2,k , x˙r,2 , cos x1,k , x1,k φ(z) = cos(z 1 − z 2 ) + 2 and φ(·) = 3. The learning gains are again set to η = 20 and μ = 2. The convergence of the ETE is shown in Fig. 10.7. The simulation results demonstrate that the learning abilities of the different schemes are similar and that

10.5 Illustrative Simulations

189

1.2 xr,1 1st iteration 10th iteration 20th iteration 30th iteration 50th iteration

1

Tracking performance

0.8

0.6

0.4

0.2

0

−0.2

0

1

2

3

4

5

6

Time axis

(a) First dimension 1 xr,2 1st iteration 10th iteration 20th iteration 30th iteration 50th iteration

0.8

Tracking performance

0.6

0.4

0.2

0

−0.2

−0.4

−0.6

0

1

2

3

4

Time axis

(b) Second dimension Fig. 10.3 Tracking performance of illustrative iterations for Case 1

5

6

190

10 CEF Techniques for Parameterized … 0.7 maximum ETE average ETE

max|δ(t)| and ave|δ(t)|

0.6

0.5

0.4

0.3

0.2

0.1

0 5

10

15

20

25

30

35

40

45

50

Iteration axis

Fig. 10.4 Maximum and average ETE for Case 2 0.5 0.4 0.3 0.2

x˙ r

0.1 0 −0.1 −0.2 1st iteration 2nd iteration 3rd iteration 4th iteration

−0.3 −0.4 −0.5

0

1

2

3

4

Time axis

Fig. 10.5 Desired trajectories of the first four iterations for Case 3

5

6

10.5 Illustrative Simulations

191

0.45 maximum ETE average ETE

0.4

max|δ(t)| and ave|δ(t)|

0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 5

10

15

20

25

30

35

40

45

50

Iteration axis

Fig. 10.6 Maximum and average ETE for Case 3 0.25 maximum ETE average ETE

max|δ(t)| and ave|δ(t)|

0.2

0.15

0.1

0.05

0

5

10

15

20

25

30

Iteration axis

Fig. 10.7 Maximum and average ETE for Case 4

35

40

45

50

192

10 CEF Techniques for Parameterized …

their performances also heavily depend on the random generation of the varyinglength iterations. Therefore, we omit the tracking performance figures for Cases 2–4 for brevity.

10.6 Summary In this chapter, we attempt to apply ILC to continuous-time parametric nonlinear systems in which the nonlinear functions do not satisfy the globally Lipschitz continuous condition and the iteration lengths vary randomly. To handle this type of nonlinear system, an iterative learning estimation scheme of the unknown parameters is incorporated with a feedback controller to ensure the stability along the time axis and asymptotical convergence along the iteration axis. To demonstrate the convergence, a virtual tracking error is defined for the missing section of the operation process and a novel CEF is introduced for the nonuniform trial length problem. In contrast with the common compensation mechanism of the missing tracking error with zero, we replace the latter with the iteration ending value in this chapter. This compensation scheme enables us to make an explicit difference of the CEF between adjacent iterations and show the asymptotical convergence. The results in this chapter show that the adaptive ILC scheme is effective for parametric nonlinear systems in which the operation lengths vary randomly. The results are then extended to cases with unknown input gain, iteration-varying tracking reference, nonparametric uncertainty, and highorder systems, where slight modifications to the control and estimation laws were employed when necessary. For further research, it is of interest to investigate general nonlinear systems and design more effective ILC algorithms with faster convergence speeds. The results in this chapter are mainly based on [3].

References 1. Xu JX, Jin X, Huang D (2014) Composite energy function-based iterative learning control for systems with nonparametric uncertainties. Int J Adapt Control Signal Process 28(1):1–13 2. Li X, Huang D, Chu B, Xu JX (2016) Robust iterative learning control for systems with normbounded uncertainties. Int J Robust Nonlinear Control 26(4):697–718 3. Shen D, Xu JX.: Adaptive learning control for nonlinear systems with randomly varying iteration lengths. IEEE Trans Neural Netw Learn Syst (2018). https://doi.org/10.1109/TNNLS.2018. 2861216

Chapter 11

CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems

In this chapter, we proceed to consider continuous-time nonlinear systems with nonparametric uncertainties, in contrast to parameterized systems in the previous chapter, under nonuniform trial length circumstances. Three common types of nonparametric uncertainties are taken into account in sequence: norm-bounded uncertainty, variation-norm-bounded uncertainty, and norm-bounded uncertainty with unknown coefficients. The composite energy function defined in the previous chapter is employed for the asymptotical convergence of the proposed schemes. The singleinput-single-output case is first detailed to clarify the main idea and the extension to multi-input-multi-output case is then presented. Comparing to the previous chapters, the major contributions of this chapter are as follows: (1) continuous-time nonlinear systems with three types of nonparametric uncertainties, namely, normbounded uncertainty, variation-norm-bounded uncertainty, and norm-bounded uncertainty with unknown coefficients, are elaborated; (2) various robust ILC schemes associated with piecewise iterative estimations of the unknown components are proposed to deal with the nonparametric uncertainties and nonuniform trial lengths, where the adaptive design techniques are combined with the robust control so as to produce a favorable approximation of the unknown uncertainties; and (3) the convergence in L 2 norm sense is strictly derived for all types of uncertainties in presence of nonuniform trial lengths.

11.1 Problem Formulation Consider the following nonlinear system: x˙k = f (xk , t) + b(xk , t)u k (t),

(11.1)

where the subscript k is the iteration index, k ∈ N, and the indicated t denotes time instant, t ∈ [0, T ] with T being the desired iteration length. xk  xk (t) ∈ R denotes the measurable system state, u k (t) ∈ R is the system input, b(xk , t) ∈ R is the input © Springer Nature Singapore Pte Ltd. 2019 D. Shen and X. Li, Iterative Learning Control for Systems with Iteration-Varying Trial Lengths, https://doi.org/10.1007/978-981-13-6136-4_11

193

194

11 CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems

gain function, and f (xk , t) ∈ R is a lumped uncertainty, which is the main uncertainty considered in this chapter. In the following, the argument t may be omitted for concise notations when no confusion arises. We denote f k  f (xk , t) and bk  b(xk , t). The desired tracking reference xr is generated by the following model: x˙r (t) = wr  w(xr , t), t ∈ [0, T ],

(11.2)

where w(xr , t) ∈ R is a known function specified the tracking reference. It is continuous with respect to all its arguments. Define the tracking error ek (t)  xk (t) − xr (t). The error dynamics at the kth iteration can be derived from (11.1) and (11.2), e˙k (t) = f k + bk u k (t) − wr .

(11.3)

The primary control objective is to track the desired reference xr (t), t ∈ [0, T ] by generating an input sequence u k (t), such that the tracking error converges as the iteration number approaches to infinity. The following assumptions are required for the convergence analysis. Assumption 11.1 The input gain function bk is unknown but the control direction (i.e., the sign of bk ) is known and invariant. That is, bk is either positive or negative and nonsingular for all t ∈ [0, T ]. Without loss of any generality, we assume that bk ≥ b > 0, ∀t ∈ [0, T ], where the lower bound b is known. Assumption 11.1 is widely satisfied by practical systems such as industrial systems and mechanical systems. The control direction is generally unchanged in these systems. We can make primary tests on the system to probe the right control direction in advance. If the control direction is unknown, the mature technique of Nussbaum gain can be applied to solve the problem. If the system includes deadzone-type nonlinearity in the input, where the deadzone part may not provide a positive input gain, the techniques presented in [1] can be borrowed to solve the problem under the framework of this chapter. Assumption 11.2 In each iteration of ILC, the initial states of the system (11.1) and its desired reference (11.2) satisfy the identical initial condition: xk (0) = xr (0), ∀k ∈ N. Assumption 11.2 is a common condition in ILC due to the essential requirement of perfect repetition. This assumption implies that the operation should start from the same position so that the learning algorithms can gradually improve the tracking performance for a given reference. Although some papers have addressed learning or rectifying mechanisms for varying initial states, this issue is out of the scope of this chapter and we apply Assumption 11.2 to concentrate our discussions. In this chapter, we focus on two practical system issues arising in applications: the unknown nonparametric uncertainties and nonuniform trial lengths. We will make brief explanations on such two problems in turn, before giving the detailed design and

11.1 Problem Formulation

195

analysis of the update schemes, to clarify our main contributions. First, we restrict our discussions to nonparametric uncertainty f k in this chapter. Specifically, three common models on the uncertainty will be elaborated in turn, namely, • norm-bounded uncertainty,  f (x, t) ≤ ρ(x, t)

(11.4)

where ρ(x, t) is a known locally Lipschitz continuous (LLC) function; • variation-norm-bounded uncertainty,  f (x, t) − f (y, t) ≤ ρ(x, y)x − y

(11.5)

where ρ(x, y, t) is a known function; • and norm-bounded uncertainty with unknown coefficient,  f (x, t) ≤ θρ(x, t)

(11.6)

where ρ(x, t) is a known LLC function and θ is a unknown coefficient. The above three types of uncertainties cover most common nonparametric uncertainties in real systems. For example, when considering the bounded disturbances and noises in systems, we can depict it using the first model with ρ(x, t) being some constant. Moreover, when considering the structure uncertainty that depends on the state, we can model this uncertainty with a suitable ρ(x, t) with or without the parameter θ . In addition, if the derivative of unknown signals is bounded such as deadzone nonlinearity, the second model is applicable. In short, the proposed three typical models have a strong connection with real systems. The kernel idea for treating these nonparametric uncertainties is to parameterize the bounding functions and then propose robust terms in the update law. Therefore, the robust compensation term can well cancel the unknown uncertainty and an additional feedback term is designed in the control law to stabilize the system. Moreover, we consider the nonuniform trial length problem. That is, the actual trial length Tk is iteration-varying and thus different from the desired length T . Obviously, the actual trial length Tk varies in different iterations randomly. Consequently, two cases need to be taken into account: Tk < T and Tk ≥ T . For the latter case, it is observed that only the data in the time interval [0, T ] will be used for further updating, while the data from (T, Tk ] will be discarded directly. Thus, without loss of any generality, we can regard this case as Tk = T . In other words, we can only consider the case that Tk is not beyond T . Moreover, it is reasonable to assume that there exists a minimum of Tk , denoted by Tmin with Tmin > 0. In sum, we concentrate our discussions on the case that 0 < Tmin ≤ Tk ≤ Tmax  T in the following. We need the following assumption on the nonuniform trial length variable Tk . Assumption 11.3 Assume that Tk is a random variable, and its probability distribution function is given by

196

11 CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems

⎧ ⎪ t ∈ [0, Tmin ) ⎨0, FTk (t)  P[Tk < t] = p(t), t ∈ [Tmin , Tmax ] ⎪ ⎩ 1, t > Tmax

(11.7)

where 0 ≤ p(t) < 1 is a continuous function. Moreover, we have p(Tmax ) > 0 implying that the trial length take the full length with a positive probability. The above assumption is a description of the nonuniform trial lengths, which vary randomly during different iterations. We emphasize that the probability distribution function is not required to be known prior. In other words, the specific information of the nonuniform trial length is not involved in the controller design. Now we can specify our problem as follows. The control objective of this chapter is to provide a framework for the design and analysis of ILC for systems with nonparametric uncertainties and nonuniform trial lengths. Based on the available information of the previous iterations, the ILC algorithms should ensure the system state to track the given reference as the iteration number increases. The convergence is in a certain sense for the actual operation interval [0, Tk ] rather than [0, T ] for the kth iteration. The design and analysis of these algorithms will be specified in the next section. The extensions to MIMO system case will be presented in Sect. 11.3.

11.2 Robust ILC Algorithms and Their Convergence Analysis In this section, we will present the robust ILC algorithms for three types of nonparametric uncertainties in turn and show the convergence in L 2 norm sense.

11.2.1 Norm-Bounded Uncertainty Case In this subsection, we first consider the norm-bounded uncertainty case, i.e.,  f (x, t) ≤ ρ(x, t). Note that such condition on the uncertainty implies that the upper bound of the uncertainty is known, and thus can be employed as the robust compensation term. The controller is formulated as θk (t)ρ(xk , t)sgn(ek (t)) u k (t) = b−1 [ − μek (t) −  − |wr |sgn(ek (t))], t ∈ [0, Tk ],

(11.8)

where μ > 0 is a control gain and sgn(·) denotes the sign function. The parameter estimation  θk (t) is updated by

11.2 Robust ILC Algorithms and Their Convergence Analysis

  θk−1 (t) + γρ(xk , t)|ek (t)|, t ∈ [0, Tk ]  θk (t) =  θk−1 (t), t ∈ (Tk , T ]  θ0 (t) = 0, ∀t ∈ [0, T ],

197

(11.9) (11.10)

where γ > 0 is the learning gain and | · | represents the absolute value of its indicated variable. Remark 11.1 From the condition on the unknown uncertainty, we are evident that the uncertainty is bounded by a known function of the state, i.e., ρ(xk , t). Thus one may argue that why we do not apply the upper bound ρ(xk , t) directly in (11.8) but introduce an additional estimation  θk (t). The reason is that, if we directly apply the θk (t), then the control law (11.8) is upper bound ρ(xk , t) and omit the parameter  complete a robust feedback law without using any knowledge or experience from previous iterations. In other words, we have learned nothing from the previous iterations. However, it is much conservative to directly use the upper bound function. Instead, we make an iteratively estimation of the bound of the uncertainty so that the performance can be gradually improved along the iteration axis. In short, one may regard  θk (t) as a time-varying estimation of 1, which does not appear in the uncertainty condition (11.4). Remark 11.2 Under nonuniform trial length circumstances, the operation interval for the kth iteration is Tk , which can be either longer or shorter than that of the θk is (k − 1)th iteration Tk−1 . As a consequence, the updating of the estimation  divided into two parts: during the operation interval, the estimation is updated, and out of the operation length, the estimation retains its current value (since we cannot operate during this interval and nothing can be learned). Then, the convergence proof has to be separated into two subcases accordingly. In order to facilitate the convergence analysis of the proposed robust ILC scheme, we employ the following CEF: E k (t) =

1 2 1 k (t) + 2 2γ

 0

t

φk2 (τ )dτ,

(11.11)

where k (t) is the virtual tracking error, defined as k (t) = ek (t), ∀t ∈ [0, Tk ] and k (t) = ek (Tk ), ∀t ∈ (Tk , T ]. We should point out that the virtual tracking error k (t) is introduced only to facilitate the convergence analysis. This signal is not used for θk (t) − 1 is the virtual the design of controllers and estimation algorithms. φk (t)   estimation error. The convergence property of the proposed scheme is summarized in the following theorem. Theorem 11.1 For the nonlinear system (11.1) with its uncertainty satisfying the norm-bounded condition, under Assumptions 11.1–11.3, the robust ILC algorithms (11.8)–(11.9) guarantee that the tracking error converges to zero in the L 2 -norm sense as the iteration number goes to infinity.

198

11 CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems

Proof The proof consists of three parts. First, we check the difference of the novel CEF to show its decreasing trend. Then, we move to show the convergence of tracking errors. Last, we check the boundedness property of all involved signals. Part I: The difference of E k (t). Consider the difference of E k (t), ∀k ≥ 2, ΔE k (t)  E k (t) − E k−1 (t)



1 2 1 1 (t) + = k2 (t) − k−1 2 2 2γ

0

t

2

2 φk − φk−1 dτ,

(11.12)

θk (t) − 1 is the virtual estimation error. Note that where we remind that φk (t) =  θk (t) are piecewise defined, thus we need to consider two the notations k (t) and  scenarios, 0 ≤ t ≤ Tk and Tk < t ≤ T , separately. We first consider the case t ≤ Tk . For this case, let us examine the first term on the right-hand side (RHS) of (11.12). According to the error dynamics (11.3), the identical initial condition, Assumption 11.2, and the control law (11.8), we have  t 1 2 1 2 1 2  (t) = ek (t) = ek (0) + ek e˙k dτ 2 k 2 2 0  t ek ( f k + bk u k − wr )dτ = 0  t  ek f k − bk b−1 μek − wr − bk b−1 |wr |sgn(ek ) = 0 − bk b−1 θk (τ )ρ(xk , τ )sgn(ek ) dτ.

(11.13)

From the norm-bounded condition, we have ek f k ≤ ρ(xk , t)|ek (t)|.

(11.14)

On the other hand, noting that b is the lower bound of unknown input gain bk and bk b−1 ≥ 1, we have θk (t)ρ(xk , t)sgn(ek ) ≤ − θk (t)ρ(xk , t)|ek (t)|, − ek bk b−1 −1

− bk b |wr ek | ≤ −ek wr ,

(11.15) (11.16)

where the positiveness of  θk (t) coming from the updating law (11.9). Consequently, for all t ∈ [0, Tk ], 1 2  (t) ≤ − 2 k



t 0

 μek2 (τ )dτ

− 0

t

φk ρ(xk , τ )|ek (τ )|dτ.

(11.17)

11.2 Robust ILC Algorithms and Their Convergence Analysis

199

In addition, for the last term on the RHS of (11.12), we notice that  t 1 2 (φ 2 − φk−1 )dτ 2γ 0 k  t 1 = (φk + φk−1 )(φk − φk−1 )dτ 2γ 0  t 1 = [2φk + (φk−1 − φk )](φk − φk−1 )dτ 2γ 0  t  t 1 = φk ρ(xk , τ )|ek (τ )|dτ − (φk − φk−1 )2 dτ. 2γ 0 0

(11.18)

Substituting (11.17) and (11.18) into (11.12) yields that, ∀t ∈ [0, Tk ], 

t

1 2 μek2 (τ )dτ − k−1 (t) 2 0  t 1 (φk − φk−1 )2 dτ. − 2γ 0

ΔE k (t) ≤ −

(11.19)

Next, we proceed to derive the counterpart case Tk < t ≤ T when Tk = T . For this case, the first term on the RHS of (11.12) becomes 1 2 1  (t) = ek2 (Tk ) 2 k 2   Tk μek2 (τ )dτ − ≤− 0

Tk

φk ρ(xk , τ )|ek (τ )|dτ.

(11.20)

0

Moreover, for the last term on the RHS of (11.12), we have  t 1 2 (φ 2 − φk−1 )dτ 2γ 0 k  Tk  t 1 1 2 2 2 = (φk − φk−1 )dτ + (φ 2 − φk−1 )dτ 2γ 0 2γ Tk k  Tk  Tk 1 φk ρ(xk , τ )|ek (τ )|dτ − (φk − φk−1 )2 dτ. = 2γ 0 0

(11.21)

Therefore, for the second case, we can derive that 

Tk

1 2 μek2 (τ )dτ − k−1 (t) 2 0  Tk 1 (φk − φk−1 )2 dτ. − 2γ 0

ΔE k (t) ≤ −

(11.22)

200

11 CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems

Combining (11.19) and (11.22), we come to 

Tk ∨t

ΔE k (t) ≤ − 0

1 2 μek2 (τ )dτ − k−1 (t), 2

(11.23)

where Tk ∨ t  min{Tk , t} denotes the smaller value between Tk and t, and the addiT tional term 2γ1 0 k (φk − φk−1 )2 dτ is omitted. Note that Tk is a random variable, thus the difference of E k (t) is not deterministic. Part II: Convergence of the tracking error. According to (11.23), it can be derived that the finiteness of E k (t) is ensured for any iteration k provided that E 1 (t) is finite and then we can prove the asymptotical convergence of the tracking error. To this end, we first show the finiteness of E 1 (t). Note the definition of E 1 (t), 1 1 E 1 (t) = 12 (t) + 2 2γ



t 0

φ12 dτ.

(11.24)

Therefore, for t ∈ [0, T1 ], we have 1 2 φ E˙ 1 (t) = e1 (t)e˙1 (t) + 2γ 1 θ1 (t)ρ(x1 , t)sgn(e1 )] = e1 (t)[ f 1 − b1 b−1 − e1 (t)[wr + b1 b−1 |wr |sgn(e1 )] 1 2 φ − b1 b−1 μe12 (t) + 2γ 1 ≤ ρ(x1 , t)|e1 (t)| −  θ1 (t)ρ(x1 , t)|e1 (t)| 1 2 φ , − μe12 (t) + 2γ 1

(11.25)

where the error dynamics in the first iteration and the relationship e1 (t) f 1 ≤ θ0 = 0, ∀t, we have  θ1 (t) = γρ(x1 , t)|e1 (t)|, or ρ(x1 , t)|e1 (t)| are applied. Since  equivalently, ρ(x1 , t)|e1 (t)| =

 θ1 (t) . γ

Substituting (11.26) into (11.25) leads to 1 1 2 1 2 E˙ 1 (t) ≤ − μe12 (t) +  θ1 −  θ1 (t) + φ γ γ 2γ 1 1 1 2 1 θ1 −  θ (t) + ( θ1 (t) − 1)2 = − μe12 (t) +  γ γ 1 2γ

(11.26)

11.2 Robust ILC Algorithms and Their Convergence Analysis

= − μe12 (t) −

201

1 2 1  θ1 (t) + . 2γ 2γ

(11.27)

Then, E˙ 1 (t) ≤ 1/(2γ ) for t ∈ [0, T1 ]. By the boundedness of the trial length Tk , we obtain the finiteness of E 1 (t). θ1 (t) = 0; therefore, ∀t ∈ (T1 , T ], In addition, when t > T1 , k (t) is a constant and  1 2 1 E˙ 1 (t) = φ1 = , 2γ 2γ

(11.28)

and the finiteness of E 1 (t) is obtained. Now we come back to difference of the CEF (11.23). The finiteness of E 1 (t) implies that the finiteness of E k (t), ∀k ≥ 2, ∀t ∈ [0, T ]. Moreover, from (11.23) we have  Tk 1 2 μek2 (τ )dτ − k−1 (T ), (11.29) ΔE k (T ) ≤ − 2 0 or equivalently, k  

1 2  (T ), 2 j=2 j−1

(11.30)

e2j (τ )dτ ≤ E 1 (T ) − E k (T ) ≤ E 1 (T ).

(11.31)

E k (T ) ≤ E 1 (T ) − μ

j=2

Tj 0

k

e2j (τ )dτ −

which further implies that μ

k   j=2

0

Tj

T This inequality implies that 0 k ek2 (τ )dτ → 0 as k → ∞. In other words, the convergence of the tracking ek (t) error in the sense of L 2 norm is proved. In addition, by the assumption of the random trial length, we have that the probability of full trial length, namely, Tk = T , is positive; that is, P[Tk = T ] > 0. In other words, there are infinite iterations satisfy Tk = T and for these iterations it holds that

T 2 0 ei k (τ )dτ → 0 as k → ∞, where {i k } is a subset of N denoting those iterations with trial length being equal to T . Part III: Boundedness of the involved variables. In this part, we check the boundedness property of the system state xk (t), the estimation  θk , and the control input u k (t). As we have shown in Part II, the CEF is bounded for all k and t. As a result, k (t) (equivalently, ek (t) for t ∈ [0, Tk ]) E k (t)

t t θk (τ )dτ and 0 φk2 dτ are bounded for all iterations. Then, it follows that xk (t) and 0  are bounded.

202

11 CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems

Because ρ(xk , t) is LLC with respect to its state xk (t), the boundedness of xk (t) ensures the boundedness of ρ(xk , t). Then, according to the control law (11.8), it is evident that u k (t) is also bounded in L 2 -norm. The proof is completed. This theorem implies that the available tracking performance can be gradually improved along the iteration axis, although the iteration length may vary randomly for different iterations. That is, when the operation ends at time instant Tk , the energy E k (t) is decreased for t ∈ [0, Tk ] comparing with its previous iteration. As can be seen from the control law (11.8) and the updating law (11.9), the improvement works during the actual operation interval [0, Tk ], while for the remaining part, no updating is imposed. Remark 11.3 In Theorem 11.1, we have proved the convergence of the tracking error in L 2 -norm sense by deriving the difference of the CEF. In fact, from the difference (11.23), we implies  (11.23)  can obtain the pointwise convergence directly, because 2 that 1/2 kj=2  2j (t) ≤ E k (t) − E 1 (t) ≤ E 1 (t), ∀k, t. That is, ∞ j=2  j (t) < ∞ for arbitrary time instant t, which further leads to k (t) → 0 as k → ∞, ∀t. Again, based on the positive probability of the event Tk = T , the pointwise convergence of the tracking error can be guaranteed with probability 1. It is noticed from the proof that we can mainly guarantee the L 2 -norm boundedness of the parameter estimation. In other words, we do not establish the direct boundedness of  θk (t). As a consequence, we could not ensure the uniform convergence of the tracking error to zero since the estimation is involved into the control signal. However, as we have explained before, the estimation  θk (t) is to approximate the constant 1; in other words, we may have some prior knowledge of the estimation process. Then, we can incorporate this additional bounding information into the learning process. In particular, the parameter updating law (11.9) can be modified as a projection-based type,  P( θk−1 (t)) + γρ(xk , t)|ek (t)|, t ∈ [0, Tk ]  θk (t) =  t ∈ (Tk , T ] P(θk−1 (t)),

(11.32)

with  θ0 = 0, ∀t ∈ [0, T ], where the operator P(α) is defined as  P(α) =

α, |α| ≤ α ∗ ∗ α · sgn(α), |α| > α ∗

In this case, we let the saturation bound for  θk be 1. By the boundedness of ek (t) and xk (t) shown in Theorem 11.1, we are able to verify that the control input u k (t) is bounded and thus the error dynamics e˙k (t) is bounded, which further implies the uniform continuity of the tracking error ek (t). Therefore, the uniform convergence of the tracking error is guaranteed. The results are summarized in the following theorem.

11.2 Robust ILC Algorithms and Their Convergence Analysis

203

Theorem 11.2 For the nonlinear system (11.1) with its uncertainty satisfying the norm-bounded condition, under Assumptions 11.1–11.3, the robust ILC algorithms (11.8) and (11.32) guarantee that the tracking error converges to zero uniformly as the iteration number goes to infinity. Proof We apply the same CEF defined in (11.11). The derivations in the proof of Theorem 11.1 still hold except for the difference of the virtual estimation error part t (i.e., 2γ1 0 φk2 dτ in (11.12)). To show the validity of (11.18), we must check the

t

2 dτ . In particular, using the basic property (a − b)2 ≥ difference 2γ1 0 φk2 − φk−1 [a − P(b)]2 for any suitable scalars a and b, we have 2 = ( θk−1 − 1)2 ≥ [P( θk−1 ) − 1]2 . φk−1

(11.33)

This further leads to  t 1 2 (φ 2 − φk−1 )dτ 2γ 0 k  t 2

1 ≤ φk − [P( θk−1 ) − 1]2 dτ 2γ 0  t



1 = φk + P( θk−1 ) − 1 φk − P( θk−1 ) + 1 dτ 2γ 0  t



1  = θk + P( θk−1 ) − 2  θk − P( θk−1 ) dτ 2γ 0  t 

2



1 1 t   =− θk − P( θk − P( θk−1 ) dτ + θk−1 )  θk − 1 dτ 2γ 0 γ 0  t  t

2 1  = θk − P( φk ρ(xk , τ )|ek (τ )|dτ − θk−1 ) dτ, (11.34) 2γ 0 0 where in the last equality (11.32) is applied. Then, (11.19) becomes 

t

1 2 μek2 (τ )dτ − k−1 (t) 2 0  t

2 1  θk − P( θk−1 ) dτ. − 2γ 0

ΔE k (t) ≤ −

Similarly, we can revise (11.21) for the case Tk < t ≤ T ,  t 1 2 (φ 2 − φk−1 )dτ 2γ 0 k  Tk  t 1 1 2 2 2 = (φk − φk−1 )dτ + (φ 2 − φk−1 )dτ 2γ 0 2γ Tk k

(11.35)

204

11 CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems



Tk

= 0

1 φk ρ(xk , τ )|ek (τ )|dτ − 2γ



Tk



2  θk − P( θk−1 ) dτ.

(11.36)

0

Consequently, (11.22) will change accordingly. Then, the last inequality (11.23) is still valid. In other words, Part I in the proof of Theorem 11.1 holds. Meanwhile, Part II in the proof of Theorem 11.1 can be copied here without change. Now, we proceed to check the boundedness properties of involved quantities. From the boundedness of the CEF, we know that ek (t) is bounded, and therefore, xk (t) is bounded. The latter further implies the boundedness of ρ(xk , t). Noticing (11.32), it is clear that the estimated value  θk is bounded for all k. In such case, u k (t) is bounded over time interval [0, Tk ] for all k because all its associated quantities are bounded (c.f. (11.8)). Consequently, from (11.1) we notice that x˙k (t) is bounded. The boundedness of x˙k (t) implies the uniform continuity of xk (t), and therefore, the uniform convergence of the tracking error ek (t). In other words, the uniform convergence is guaranteed. The proof is thus completed. This theorem reveals that if we have the boundedness information of the unknowns, then the projection-based scheme can ensure the boundedness of all involved quantities. This property further leads to a uniform convergence of the tracking error over the operation interval with the help of Barbalat lemma. The projection-based scheme is also effective for the other robust control algorithms in the rest schemes; however, we will not repeat the design and analysis in the following to save space.

11.2.2 Variation-Norm-Bounded Uncertainty Case In this subsection, we proceed to consider the variation-norm-bounded uncertainty case, i.e.,  f (x, t) − f (y, t) ≤ ρ(x, y)x − y. Note that such condition is on the variation rather than the uncertainty itself, thus we need to reformulate the uncertainty into a variation form. To this end, we rewrite the error dynamics (11.3) as follows: e˙k (t) = f (xk , t) + b(xk , t)u k (t) − wr = [ f (xk , t) − f (xr , t)] + f (xr , t) · 1 + b(xk , t)u k (t) − wr .

(11.37)

From this equation, the variation f (xk , t) − f (xr , t) can be compensated by a robust term while the newly introduced f (xr , t) can be regarded as an unknown time-varying parameter θ (t) so that it can be iteratively estimated (i.e., θ (t)  f (xr , t)). The controller is formulated as θk (t)sgn( θk ek ) u k (t) = − b−1 μek (t) − b−1 − b−1 ρ(xk )ek − b−1 |wr |sgn(ek ),

(11.38)

where  θk (t) is the estimation of θ (t) for the kth iteration and ρ(xk ) is defined as

11.2 Robust ILC Algorithms and Their Convergence Analysis

ρ(xk )  sup ρ(xk , z) |z|≤m r

205

(11.39)

with m r  supt |xr (t)|. The updating law for the unknown parameter θ (t) is given by   θk−1 (t) + γ ek (t), t ∈ [0, Tk ]  θk (t) =  θk−1 (t), t ∈ (Tk , T ]  θ0 = 0, t ∈ [0, T ],

(11.40) (11.41)

where γ > 0 is the learning gain. Recall the definition of k (t) and still let φk (t) be the estimation error, φk (t) =  θk (t) − θ (t), for saving notations. Moreover, the CEF in (11.11) is also employed θk (t) are with new meanings. Then, we can for this case with noticing that φk (t) and  present the following convergence theorem for the variation-norm-bounded uncertainty case. Theorem 11.3 For the nonlinear system (11.1) with its uncertainty satisfying the variation-norm-bounded condition, under Assumptions 11.1–11.3, the robust ILC algorithms (11.38)–(11.40) guarantee that the tracking error converges to zero in the L 2 -norm sense as the iteration number goes to infinity. Proof Similar to the proof of Theorem 11.1, we will carry out the proof by three parts. The first part is to derive the difference of the CEF along the iteration axis. The second part will proceed to the convergence analysis. The last part presents the boundedness property of involved signals, which is completely the same to that of Theorem 11.1 and thus we omit it in this proof. Part I: The difference of E k (t). Consider the difference of E k (t) given by (11.11) and remind that ΔE k (t) =

1 2 1 2 1 (t) + k (t) − k−1 2 2 2γ



t 0

2

2 φk − φk−1 dτ.

(11.42)

We conduct the analysis for two cases, t ≤ Tk and t > Tk , respectively. First, for the case t ≤ Tk , substituting the controller (11.38) into the error dynamics (11.37) yields that e˙k (t) = [ f (xk , t) − f (xr , t)] + f (xr , t) · 1 bk bk θk (t)sgn( θk ek ) − (μ + ρ(xk ))ek (t) −  b b bk − wr − |wr |sgn(ek ). b

(11.43)

Then, substituting the above equation into the derivation of the first term on RHS of (11.42), we obtain

206

11 CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems

 t 1 2 1 k (t) = ek2 (t) = ek (τ )e˙k (τ )dτ 2 2 0  t ek (τ )[ f (xk , τ ) − f (xr , τ )]dτ = 0  t  t ek (τ ) θk (τ )dτ − ek (τ )φk (τ )dτ + 0 0  t bk (μ + ρ(xk ))ek2 (τ )dτ − 0 b  t bk  θk (τ )ek (τ )sgn( − θk ek )dτ 0 b  t  t bk − ek (τ )wr dτ − |wr ek |dτ. 0 0 b

(11.44)

Notice that the following inequalities are true: ek (τ )[ f (xk , τ ) − f (xr , τ )] − ≤ ρ(xk , xr )ek2 (τ ) −

bk ρ(xk )ek2 (τ ) b

bk ρ(xk )ek2 (τ ) ≤ 0, b

bk  θk (τ )ek (τ )sgn( θk ek ) b bk θk (τ ) − | θk (τ )ek (τ )| ≤ 0, ≤ ek (τ ) b bk − ek (τ )wr (τ ) − |wr (τ )ek (τ )| b bk ≤ |ek (τ )wr (τ )| − |wr (τ )ek (τ )| ≤ 0, b

(11.45)

θk (τ ) − ek (τ )

(11.46)

(11.47)

where Assumption 11.1 and the uncertainty condition are applied. Then, the Eq. (11.44) becomes 1 2  (t) ≤ − μ 2 k



t 0

 ek2 (τ )dτ −

t

ek (τ )φk (τ )dτ.

0

Moreover, for the last term on the RHS of (11.42), we have  t 1 2 (φ 2 − φk−1 )dτ 2γ 0 k  t 1 = [2φk + (φk−1 − φk )](φk − φk−1 )dτ 2γ 0

(11.48)

11.2 Robust ILC Algorithms and Their Convergence Analysis



t

=

φk ek dτ −

0



1 2γ

t

(φk − φk−1 )2 dτ,

207

(11.49)

0

where the parameter updating law (11.40) is applied. Substituting (11.48) and (11.49) into (11.42) yields that, ∀t ∈ [0, Tk ], 

t

ΔE k (t) ≤ − μ

1 2 ek2 (τ )dτ − k−1 (t) 2

0



1 − 2γ

t

(φk − φk−1 )2 dτ.

(11.50)

0

Similar to the steps in the proof of Theorem (11.1), for the case t > Tk , we obtain 

Tk

ΔE k (t) ≤ − μ

1 2 ek2 (τ )dτ − k−1 (t) 2

0

1 − 2γ



Tk

(φk − φk−1 )2 dτ.

(11.51)

0

Further, combining (11.50) and (11.51) leads to a same result to (11.23) 

Tk ∨t

ΔE k (t) ≤ −μ 0

1 2 ek2 (τ )dτ − k−1 (t). 2

(11.52)

Part II: Convergence of the tracking error. This part can be completed following similar derivations to that of Theorem (11.1), provided that we can show the finiteness of E 1 (t), ∀t ∈ [0, T ]. To show this point, we note from the definition of E k (t) that E 1 (t) =

1 2 1  (t) + 2 1 2γ



t 0

φ12 (τ )dτ.

Therefore, for t ∈ [0, T1 ], we have 1 2 φ (t) E˙ 1 (t) = e1 (t)e˙1 (t) + 2γ 1 = e1 (t)[ f (x1 , t) − f (xr , t)] + e1 (t) f (xr , t) b1 b1 θ1 (t)e1 (t)sgn( θk ek ) − (μ + ρ(x1 ))e12 (t) −  b b b1 1 2 φ1 (t) − e1 (t)wr − |wr e1 (t)| + 2γ b ≤ − μe12 (t) − e1 (t)φ1 (t) 1 2 + [φ + 2φ1 (φ1 − φ0 ) − (φ1 − φ0 )2 ] 2γ 0

(11.53)

208

11 CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems

≤ − μe12 (t) − e1 (t)φ1 (t) + ≤

1 2 1 φ0 + φ1 ( θ1 −  θ0 ) 2γ 2γ

1 2 θ (t), 2γ

(11.54)

whence the finiteness of E 1 (t) is guaranteed for t ∈ [0, T1 ] due to the boundedness of T1 . θ1 (t) =  θ0 (t) = 0 Moreover, for the case t > T1 , noting that k (t) is a constant and  in the interval (T1 , T ], we are evident to conclude that 1 2 1 2 E˙ 1 (t) = φ1 (t) = θ (t), t ∈ (T1 , T ], 2γ 2γ

(11.55)

and then, the finiteness of E 1 (t) is also guaranteed for t ∈ (T1 , T ]. The rest of the proof can be completed similarly to the proof of Theorem 11.1, so we omit the details to save space. The proof is thus completed. This theorem demonstrates that the convergence is guaranteed for the variationnorm-bounded uncertainty. Different from the last subsection in which the upper bound of the uncertainty is known prior, the variation-norm-bounded uncertainty implies little information about the uncertainty itself but its derivative. Therefore, in the design of the controller, the uncertainty is compensated by two terms: one is to compensate a variation from a virtual signal and the other one is to estimate the newly introduced virtual signal. Remark 11.4 It should be pointed out that, in the error dynamics (11.37), we formulate the newly introduced signal f (xr , t) as f (xr , t) · 1, which is in the parametric form. This is why we only employ the tracking error ek (t) in the parameter updating law (11.40) because the known information (i.e., 1) can be omitted.

11.2.3 Norm-Bounded Uncertainty with Unknown Coefficient Case In this subsection, we proceed to the last type of unknown uncertainty, i.e., the norm-bounded uncertainty with unknown coefficients ( f (x, t) ≤ θρ(x, t)). It is apparent that the difference between this type and the first norm-bounded type lies in the fact that an additional unknown coefficient is involved. To this end, an estimation process is necessary for the learning control. As a result, the controller design in Sect. 11.2.1 still applies. We rewrite the controller here for smooth readability θk (t)ρ(xk , t)sgn(ek (t)) u k (t) = b−1 [ − μek (t) −  − |wr |sgn(ek (t))],

t ∈ [0, Tk ],

(11.56)

11.2 Robust ILC Algorithms and Their Convergence Analysis

209

where the parameter estimation  θk (t) is to estimate the unknown coefficient θ and is also updated as   θk−1 (t) + γρ(xk , t)|ek (t)|, t ∈ [0, Tk ]  θk (t) =  θk−1 (t), t ∈ (Tk , T ]  θ0 (t) = 0, ∀t ∈ [0, T ].

(11.57) (11.58)

The result is summarized in the following theorem. Theorem 11.4 For the nonlinear system (11.1) with its uncertainty satisfying the norm-bounded with unknown coefficient condition, under Assumptions 11.1–11.3, the robust ILC algorithms (11.56)–(11.57) guarantee that the tracking error converges to zero in the L 2 -norm sense as the iteration number goes to infinity. Proof The proof can be carried out following similar steps to the proof of Theorem 11.1; therefore, we mainly present the differences. We again use the CEF defined in (11.11), in which φk   θk (t) − θ with θ being defined in (11.6), denoting the estimation error of parametric uncertainty embedded in the bounding function. When t ≤ Tk , it is apparent that  t 1 2 (φ 2 − φk−1 )dτ 2γ 0 k  t 1 = [2( θk − θ ) + ( θk−1 −  θk )]( θk −  θk−1 )dτ 2γ 0  t  t 1 = φk ρ(xk , τ )|ek (τ )|dτ − ( θk −  θk−1 )2 dτ. 2γ 0 0

(11.59)

Moreover, from the condition  f (xk , t) ≤ θρ(xk , t), we obtain that ek f (xk , t) ≤ θρ(xk , t)|ek (t)|.

(11.60)

Then, substituting (11.60) into (11.13) and using (11.59), we have 

t

1 2 μek2 (τ )dτ − ek−1 (t) 2 0  t 1 ( θk −  θk−1 )2 dτ. − 2γ 0

ΔE k (t) ≤ −

(11.61)

Similarly, when Tk < t ≤ T , we must have 

Tk

1 2 μek2 (τ )dτ − k−1 (t) 2 0  Tk 1 ( θk −  θk−1 )2 dτ. − 2γ 0

ΔE k (t) ≤ −

(11.62)

210

11 CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems

In other words, the relationship (11.23) also holds for this type of uncertainty. Further, to show the finiteness of the CEF at the first iteration E 1 (t), we notice from substituting (11.60) with k = 1 into (11.25) that 1 2 φ θ1 (t)ρ(x1 , t)|e1 (t)| − μe12 (t) + E˙ 1 (t) ≤ θρ(x1 , t)|e1 (t)| −  2γ 1 θ2 θ θ1  1 2 − 1 − μe12 (t) + φ = γ γ 2γ 1  θ2 1 2 θ , = − 1 − μe12 (t) + (11.63) 2γ 2γ and therefore, E˙ 1 (t) ≤ θ/(2γ ) is bounded for t ∈ [0, T1 ]. Then, the finiteness of E 1 (t) can be obtained. The convergence of the tracking error can then be proved and the boundedness of the involved quantities can be obtained similarly as in the proof of Theorem 11.1. In this section, we have established the robust ILC schemes for nonparametric uncertain systems, where the kernel idea is to design suitable compensation term for the unknown uncertainties. In particular, for three types of common nonparametric uncertainties, the known upper bound is fully investigated for the compensation. If we have additional structure information, it may provide benefits for the design and analysis. As an illustration, the parameterized nonlinear system with unknown coefficients and known nonlinear functions is taken into account in the previous chapter, and the learning estimation-based control scheme is presented. In this case, the learning process for the nonlinearities may be accelerated.

11.3 Extension to MIMO System To show the applicability of the proposed ILC scheme, we consider a more general MIMO uncertain dynamic system, x˙ k (t) = f k + Bk (uk (t) + d k ),

(11.64)

where x k (t) ∈ Rn is the measurable state, f k  f (x k , t) ∈ Rn is the lumped uncertainty, Bk  B(x k , t) ∈ Rn×n is a known and invertible control input distribution matrix, uk (t) ∈ Rn is the system input, and d k = d(x k , t) ∈ Rn is the state-dependent input disturbance. The desired reference x r is given by x˙ r (t) = wr , t ∈ [0, T ], where wr  w(x r , t) ∈ Rn is a known vector function.

(11.65)

11.3 Extension to MIMO System

211

Then, we define the tracking error ek (t) = x k (t) − x r (t). We note the error dynamics at the kth iteration, e˙ k = f k + Bk (uk (t) + d k ) − wr .

(11.66)

For the MIMO system, Assumption 11.1 is no longer required as we have assumed the availability of the input distribution matrix. Additional relaxation of the input distribution matrix will be clarified in Remark 11.9. Assumption 11.2 is assumed with a slight modification on the notation. It is rewritten as follows. Assumption 11.4 In each iteration of ILC, the initial states of the system (11.1) and its desired reference (11.2) satisfy the identical initial condition: x k (0) = x r (0), ∀k ∈ N. Further, we assume the input disturbance is norm-bounded, namely, for any x k ∈ Rn , d(x k , t) ≤ αk , where αk  α(x k , t) > 0 is a known locally Lipschitz continuous (LLC) function.

11.3.1 Norm-Bounded Uncertainty Case We first consider the norm-bounded type of the lumped uncertainty, i.e.,  f k  ≤ ρ(x k , t) with ρ(x k , t) is a known LLC function. The controller is formulated as   θk (t)ρ(x k , t)sgn(ek (t)) + wr uk (t) = Bk−1 −Γ ek (t) −  − ηk (t)α(x k , t)

BkT ek BkT ek 

(11.67)

where Γ ∈ Rn×n is a symmetric and positive-definite matrix denoting the control gain, Bk−1  B(x k , t)−1 is the inverse of the input distribution matrix, and sgn(ek (t))  [sgn(e1,k (t)), sgn(e2,k (t)), . . . , sgn(en,k (t))]T with ek (t) = [e1,k (t), θk (t) and  ηk (t) are updated by e2,k (t), . . . , en,k (t)]T . The parameter estimations   n  θk−1 (t) + γ1 ρ(x k , t) i=1 |ei,k |, t ∈ [0, Tk ]  θk (t) =  θk−1 (t), t ∈ (Tk , T ]  θ0 (t) = 0, t ∈ [0, T ],

(11.68) (11.69)

and   ηk−1 (t) + γ2 α(x k , t)ekT Bk , t ∈ [0, Tk ]  ηk (t) = t ∈ (Tk , T ]  ηk−1 (t),  η0 (t) = 0, t ∈ [0, T ],

(11.70) (11.71)

212

11 CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems

where γ1 > 0 and γ2 > 0 are the learning gains. In order to facilitate the convergence of the proposed ILC scheme for the MIMO system, we introduce the following CEF: 1 1 E k (t) = ξkT (t)ξk (t) + 2 2γ1

 0

t

φk2 dτ

1 + 2γ2



t

ψk2 dτ,

0

(11.72)

where ξk (t) is defined as ξk (t) = ek (t), ∀t ∈ [0, Tk ] and ξk (t) = ek (Tk ), ∀t ∈ (Tk , T ], θk (t) − 1 and ψk (t)   ηk (t) − 1 are the estimation error. The main result and φk (t)   is presented in the following theorem. Theorem 11.5 For the nonlinear system (11.64) with its uncertainty satisfying the norm-bounded condition, under Assumptions 11.3 and 11.4, the robust ILC algorithms (11.67), (11.68), and (11.70) guarantee that the tracking error converges to zero in the L 2 -norm sense as the iteration number goes to infinity. Proof The proof consists of three parts, similarly to the previous proofs, namely, the difference of the CEF, the asymptotical convergence of the tracking error, and the boundedness of the involved quantities, respectively. These three parts will be derived in turn in the following. Part I: The difference of E k (t). Consider the difference of E k (t), k ≥ 2, ΔE k (t)  E k (t) − E k−1 (t) 1 T 1 1 (t)ξk−1 (t) + = ξkT (t)ξk (t) − ξk−1 2 2 2γ1  t 1 2 + (ψ 2 − ψk−1 )dτ, 2γ2 0 k

 0

t

2 (φk2 − φk−1 )dτ

(11.73)

θk − 1 and ψk =  ηk − 1. where φk =  We first consider the case t ≤ Tk . According to the error dynamics (11.66), the identical initial condition, Assumption 11.4, and the control law (11.67), we have 1 T 1 ξ (t)ξk (t) = ekT (t)ek (t) 2 k 2  t 1 ekT (τ )˙ek (τ )dτ = ekT (0)ek (0) + 2 0  t ekT (τ )[ f k + Bk (uk (τ ) + d k ) − wr ]dτ. =

(11.74)

0

By the norm-bounded condition on the uncertainty, we have the relationships ekT f k ≤ ρ(x k , t)

n  i=1

|ei,k |,

(11.75)

11.3 Extension to MIMO System

213

ekT Bk d k ≤ αk ekT Bk .

(11.76)

For the second term on RHS of (11.74), by substituting the control law (11.67), we have ekT Bk uk (t)   θk (t)ρ(x k , t)sgn(ek (t)) + wr = ekT Bk Bk−1 −Γ ek (t) −  BkT ek BkT ek  n  = − ekT Γ ek (t) −  θk (t)ρ(x k , t) |ei,k | + ekT wr − ekT Bk  ηk (t)α(x k , t)

i=1

− ηk (t)α(x k , t)ekT Bk .

(11.77)

Substituting (11.75)–(11.77) into (11.74) leads to 1 T e (t)ek (t) ≤ 2 k

 t 0

− ekT Γ ek − ( θk (τ ) − 1)ρ(x k , τ )

n 

|ei,k |

i=1

− ( ηk (τ ) − 1)α(x k , τ )ekT Bk  dτ.

(11.78)

Moreover, we have  t 1 2 (φ 2 − φk−1 )dτ 2γ1 0 k  t 

 1 = 2( θk − 1) + ( θk−1 −  θk )  θk −  θk−1 dτ 2γ1 0  t  t n 

2 1   = θk −  (θk − 1)ρ(x k , τ ) |ei,k |dτ − θk−1 dτ, 2γ 1 0 0 i=1  t 1 2 (ψ 2 − ψk−1 )dτ 2γ2 0 k  t 

 1 = 2( ηk − 1) + ( ηk−1 −  ηk )  ηk −  ηk−1 dτ 2γ1 0  t  t

2 1 =  ηk −  ( ηk − 1)α(x k , τ )ekT Bk dτ − ηk−1 dτ. 2γ1 0 0

(11.79)

(11.80)

214

11 CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems

Substituting (11.78)–(11.80) into (11.73) yields, for t ≤ Tk ,  t 1 T ΔE k (t) ≤ − ξk−1 (t)ξk−1 (t) − ekT (τ )Γ ek (τ )dτ 2 0  t  t

2

2 1 1   θk − θk−1 dτ −  ηk −  ηk−1 dτ. − 2γ1 0 2γ1 0

(11.81)

Next, we proceed to consider the case Tk < t ≤ T when Tk = T . For this case, the counterparts of (11.74), (11.79), (11.80) are  t  Tk 1 T ξk (t)ξk (t) = ξkT (τ )ξ˙k (τ )dτ = ekT (τ )˙ek (τ )dτ 2 0 0  Tk T ek (τ )[ f k + Bk (uk (τ ) + d k )]dτ =  ≤

0 Tk



0

− ekT Γ ek − ( θk (τ ) − 1)ρ(x k , τ )

n 

|ei,k |

i=1

− ( ηk (τ ) − 1)α(x k , τ )ekT Bk  dτ,  t  Tk 1 1 2 2 2 (φ − φk−1 )dτ = (φk2 − φk−1 )dτ 2γ1 0 k 2γ1 0  Tk n  = ( θk − 1)ρ(x k , τ ) |ei,k |dτ 0

1 2γ2

1 − 2γ1



t 0

i=1



t



2  θk −  θk−1 dτ,

0

2 (ψk2 − ψk−1 )dτ =

=



(11.82)

1 2γ2

 0

Tk

(11.83) 2 (ψk2 − ψk−1 )dτ

Tk

( ηk − 1)α(x k , τ )ekT Bk dτ 0  t

2 1 ηk−1 dτ.  ηk −  − 2γ1 0

(11.84)

Substituting (11.82)–(11.84) into (11.73) leads to, for t > Tk ,  Tk 1 T ΔE k (t) ≤ − ξk−1 (t)ξk−1 (t) − ekT (τ )Γ ek (τ )dτ 2 0  Tk  Tk

2

2 1 1  θk −   ηk −  θk−1 dτ − ηk−1 dτ. − 2γ1 0 2γ1 0

(11.85)

11.3 Extension to MIMO System

215

Combining (11.81) and (11.85), we come to 1 T ΔE k (t) ≤ − ξk−1 (t)ξk−1 (t) − 2



Tk ∨t 0

ekT (τ )Γ ek (τ )dτ,

(11.86)

where Tk ∨ t  min{Tk , t} denotes the smaller one. Part II: Convergence of the tracking error. In order to derive the convergence of the tracking error, we first derive the boundedness of E 1 (t). The derivative of E 1 (t) is 1 2 1 2 E˙ 1 (t) = ξ1T (t)ξ˙1 (t) + φ + ψ . 2γ1 k 2γ2 k

(11.87)

For the case t ∈ [0, T1 ], we have 1 2 1 2 E˙ 1 (t) = e1T (t)[ f k + B1 (u1 + d 1 ) − wr ] + φ1 + ψ 2γ1 2γ2 1 n  ≤ ρ(x 1 , t) |ei,1 (t)| − e1T (t)Γ e1 (t) i=1

− θ1 (t)ρ(x 1 , t)

n 

|ei,1 (t)| −  η1 (t)α(x 1 , t)e1T (t)B1 

i=1

+

α(x 1 , t)e1T (t)B1 

+

1 2 1 2 φ + ψ . 2γ1 1 2γ2 1

(11.88)

Because  θ0 = 0 and  η0 = 0, we have  θ1 = γ1 ρ(x 1 , t)

n 

|ei,1 (t)|, i=1  η1 = γ2 α(x 1 , t)e1T (t)B1 .

(11.89) (11.90)

Then, from (11.88) we have  θ 2 (t)  θ1 (t) + E˙ 1 (t) ≤ − e1T (t)Γ e1 (t) − 1 γ1 γ1  η12 (t)  η1 (t) 1 2 1 2 − + + φ1 + ψ γ2 γ2 2γ1 2γ2 1  θ 2 (t)  η2 (t) 1 1 = − e1T (t)Γ e1 (t) − 1 − 1 + + 2γ1 2γ2 2γ1 2γ2 1 1 ≤ + . 2γ1 2γ2

(11.91)

216

11 CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems

Then, by the boundedness of the trial length T1 , we obtain the finiteness of E 1 (t) for t ∈ [0, T1 ]. θ1 (t) =  θ0 (t) = 0, For the case t ∈ (T1 , T ], we find that ξk (t) is a constant vector,  and  η1 (t) =  η0 (t) = 0. Thus, when t ∈ (T1 , T ], we have E˙ 1 (t) = 1/(2γ1 ) + 1/(2γ2 ) and the finiteness of E 1 (t) is obtained. Therefore, we can derive the conclusion  1 T ξ j−1 (t)ξ j−1 (t) − 2 j=2 j=2 k

E k (T ) ≤ E 1 (T ) − ≤ E 1 (T ) −

1 2

− λmin (Γ )

k 

k



Tj 0

eTj (τ )Γ e j (τ )dτ

T ξ j−1 (t)ξ j−1 (t)

j=2 k   j=2

Tj

e j (τ )2 dτ,

(11.92)

0

where λmin (Γ ) denotes the smallest eigenvalue of the positive-definite matrix Γ . This inequality implies the asymptotical convergence ek in the sense of L 2 -norm. Part III: Boundedness of all involved quantities. Now, we are in the position of checking the boundedness property of the system state, the parameter estimation, and the control signal. From the finiteness of E 1 (t) and the negativeness of the difference between adjacent iterations, the CEF E k (t)

T

T is bounded for all k and t. And therefore, the boundedness of 0 k φk2 dτ , 0 k ψk2 dτ ,

T 2 θk dτ , and and ekT ek are bounded for all iterations. Then, it follows that x k (t), 0 k 

Tk 2 ηk dτ are bounded. 0  Moreover, because ρ(x k , t) and α(x k , t) are LLC with respect to x k , the boundedness of x k implies the boundedness of ρ(x k , t) and α(x k , t). Then, the boundedness of the input signal can be ensured according to the control law. The proof is thus completed. Remark 11.5 In order to deal with the input disturbance d k in (11.64), an additional term is introduced to compensate this uncertainty.

t Accordingly, a corresponding term is also added to the CEF (i.e., the last term 0 ψk2 dτ ) for deriving the convergence analysis. Remark 11.6 As shown in Sect. 11.2.3, the proposed scheme (11.67) for the MIMO system (11.64) can also apply to address the third type of unknown uncertainty, namely, the norm-bounded uncertainty with unknown coefficient ( f k  ≤ θρ(x k , t)). In the latter case, the parameter learning process  θk is used for estimating the unknown coefficient θ . The convergence can be conducted parallel to the above proof and thus we omit the repetition for this case. Remark 11.7 In the controller, since we have the knowledge of input distribution matrix Bk , we directly add the signal wr to compensate the term −wr in the error

11.3 Extension to MIMO System

217

dynamics (11.66). This compensation is sensitive to the input distribution matrix. In certain applications, to enhance the robustness, we can replace the direct compensation with −wr sgn(ek (t)). The convergence analysis can be conducted by using similar steps in the proof of Theorem 11.1.

11.3.2 Variation-Norm-Bounded Uncertainty Case In this subsection, we proceed to consider the variation-norm-bounded uncertainty case for the lumped uncertainty f k . In this case, the uncertainty condition is formulated as  f (x, t) − f ( y, t) ≤ ρ(x, y)x − y. Similar to Sect. 11.2.2, we should first derive the error dynamics (11.66) as follows: e˙ k (t) = f (x k , t) + Bk (uk (t) + d k ) − wr = [ f (x k , t) − f (x r , t)] + f (x r , t) − wr + Bk (uk (t) + d k ).

(11.93)

Then we can regard the unknown vector f (x r , t) as an time-varying vector parameter Θ(t), i.e., Θ(t)  f (x r , t). The controller is formulated as k (t)) + ρ(x k )ek (t) k (t)sgn(ekT Θ uk (t) = −Bk−1 [Γ ek (t) + Θ + wr ] −  ηk (t)α(x k , t)

BkT ek , BkT ek 

(11.94)

k (t) is the estimation of Θ(t) for the kth iteration and the scalar function where Θ ρ(x k ) is defined as ρ(x k )  sup ρ(x k , z) z≤m r

(11.95)

with m r  supt x r . The updating law for the parameter Θ(t) is given by 

 k (t) = Θk−1 (t) + γ1 ek (t), t ∈ [0, Tk ] Θ k−1 (t), Θ t ∈ (Tk , T ]

(11.96)

0 (t) = 0, t ∈ [0, T ], Θ

(11.97)

where γ1 > 0 is the learning gain and 0 is the zero vector. The updating law for the estimation  ηk (t) is the same to (11.70).

218

11 CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems

In order to facilitate the convergence analysis, the CEF (11.72) is rewritten as 1 1 E k (t) = ξkT (t)ξk (t) + 2 2γ1

 0

t

ΦkT Φk dτ, +

1 2γ2

 0

t

ψk2 dτ

(11.98)

k (t) − Θ(t) is the vector estimation error of the unknown vector where Φk (t)  Θ Θ(t) = f (x r , t), while ψk is a scalar estimation that approximates 1. The result is summarized in the following theorem. Theorem 11.6 For the nonlinear system (11.64) with its uncertainty satisfying the variation-norm-bounded condition, under Assumptions 11.3 and 11.4, the robust ILC algorithms (11.94), (11.96), and (11.70) guarantee that the tracking error converges to zero in the L 2 -norm sense as the iteration number goes to infinity. Proof The proof can be conducted following the same steps as the proof of Theorem 11.3, where the necessary modifications for the vector variables can be made similar to that given in the proof of Theorem 11.5. To save space, we omit the detailed derivations. Remark 11.8 In the controller design (11.94), we also introduce the desired trajectory dynamics wr to compensate the same term in the error dynamics (11.93). In fact, we can also cancel this term from (11.94), that is, uk (t) = −Bk−1 [Γ ek (t) + BT e k (t)sgn(ekT Θ k (t)) + ρ(x k )ek (t)] −  Θ ηk (t)α(x k , t) kT k . In this case, the parameBk ek 

k (t) actually estimates f (x r , t) − wr instead of f (x r , t). Therefore, the newly ter Θ defined controller still works. Remark 11.9 In this section, to illustrative the promising extension of our robust ILC algorithms for nonparametric nonlinear systems with nonuniform trial lengths, we consider the MIMO system with square input distribution matrices, which means that the input and the state are of the same dimensions. Indeed, the results of this section can be further extended to a general case x˙ k (t) = f k + Bk (I + Hk )(uk (t) + d k ), where Bk is a known left invertible input distribution matrix and Hk represents uncertainties in the input distribution matrix. For this case, the design techniques of the conventional CEF method such as those in [2, 3] can be borrowed to complete the modifications of the controller. For MIMO systems, the control direction issue is important for achieving good performance. This issue is far from perfect and more effort is expected. To make our subject concentrated and highlighted, we make necessary simplifications on the input gain matrix.

11.4 Illustrative Simulations In this section, to verify the effectiveness of the proposed schemes for nonparametric nonlinear continuous-time systems, we consider the following MIMO nonlinear systems:

11.4 Illustrative Simulations

219

x˙ k (t) = f k + Bk (uk (t) + d k ),

(11.99)

where x k = [x1,k , x2,k ]T ∈ R2 is the state vector, 





 x1,k + sin(x2,k ) + 4π cos(8π t) fk = = , 2 sin(x1,k + x2,k ) + 4π sin(16π t)   0.1 sin(x1,k ) , dk = 0.2 sin(x2,k ) f 1,k f 2,k

and Bk = diag{2 + sin(x1,k ) + 0.1 sin(2π t), 1 + sin(x2,k ) + 0.1 sin(4π t)}. Let the desired trajectory be 

 1 + 0.5 sin(8π t) , t ∈ [0, 0.5]. x r (t) = −0.5 − 0.25 cos(16π t) The algorithms for each case are run for 50 iterations. To simulate the nonuniform trial lengths, we implement the following randomness. The probability of the case that the trial length achieves the desired length, namely, Tk = 0.5, is 0.6. In other words, the trial would end early with a probability of 0.4. Moreover, Tk varies in the interval [0.3, 0.5] with a uniform distribution for illustration. Two cases are discussed here: norm-bounded uncertainty case, where the algorithms (11.67), (11.68), and (11.70) are applied, and variation-norm-bounded uncertainty case, where the algorithms (11.94) and (11.96) are applied. Case 1: norm-bounded uncertainty case. The feedback gain is set to be Γ = diag{3, 2}. The bounding function is 2 2 2 f 1,k + f 2,k . The learning gains for (11.68) and (11.70) are set to be γ1 = 2 and γ2 = 5. The upper bound of the disturbances select αk = 0.4. The maximal tracking error for both states, maxt∈[0,0.5] |e1,k (t)| and maxt∈[0,0.5] |e2,k (t)|, are plotted in Fig. 11.1. It can be seen from the figure that the maximal tracking errors decrease as the iteration number increases, which shows the inherent learning ability of the proposed schemes for systems with norm-bounded uncertainties. To see the tracking performance, the outputs at the 1st, 5th, 10th, 20th, and 50th iterations as well as the desired trajectory are plotted in Fig. 11.2. The left and right plots demonstrate the first and second states, respectively. It can be seen that the tracking performance of the first iteration is deflected from the desired trajectory but the derivations are corrected after a few iterations. Moreover, the illustrative iterations end with different lengths, for example, the 10 and 20th iterations end early comparing with the desired length. This phenomenon indicates the nonuniform trial length problem. As a results, these plots show an acceptable learning ability of the proposed schemes against nonuniform trial lengths.

220

11 CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems −1

max e1 (t)

10

−2

10

−3

10

−4

10

5

10

15

20

25

30

35

40

45

50

35

40

45

50

Iteration axis −1

max e2 (t)

10

−2

10

−3

10

−4

10

5

10

15

20

25

30

Iteration axis

Fig. 11.1 Maximal tracking errors of both states for Case 1

Case 2: variation-norm-bounded uncertainty case. For this case, we make a slight modification to the uncertainty f k = [x1,k , sin(x2,k )]T so that the upper bound of the uncertainty variation can be selected as 1. The feedback gain is set to be Γ = diag{3, 4}. The learning gains for (11.96) is γ1 = 4. Now we apply the proposed scheme (11.94)–(11.96). The maximal tracking errors for both states are demonstrated in Fig. 11.3. Similar to Case 1, the profiles of maximal tracking errors also possess continuously decreasing trend as the iteration number increases, which surely show the effectiveness of the established schemes. Moreover, we also plot the tracking performance of illustrative iterations in Fig. 11.4. It is notable that the 1st, 10th, and 50th iterations end before completing the entire trial length. Indeed, the fluctuations in the maximal tracking error profiles in Fig. 11.3 mainly root in the nonuniform trial length problem. In addition, this figure clearly shows that the tracking performance can be effectively improved by the proposed schemes within a few iterations.

11.4 Illustrative Simulations

221

1.8 x

Tracking performance: first dimension

r,1

1st iteration 5th iteration 10th iteration 20th iteration 50th iteration

1.6

1.4

1.2

1

0.8

0.6

0.4 0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Time axis

(a) First state −0.1

Tracking performance: second dimension

xr,2 1st iteration 5th iteration 10th iteration 20th iteration 50th iteration

−0.2

−0.3

−0.4

−0.5

−0.6

−0.7

−0.8 0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Time axis

(b) Second state Fig. 11.2 Tracking performance of illustrative iterations for Case 1

0.4

0.45

0.5

222

11 CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems 0

maxe1 (t)

10

−1

10

−2

10

−3

10

5

10

15

20

25

30

35

40

45

50

35

40

45

50

Iteration axis −1

maxe2 (t)

10

−2

10

−3

10

5

10

15

20

25

30

Iteration axis

Fig. 11.3 Maximal tracking errors of both states for Case 2

To conclude this section, we remark that if we have prior knowledge of the elements in f k , we may apply the estimation-based techniques for the tracking problem (c.f. Chap. 10). However, in practical applications, it is difficult to obtain the precise structure information, thus the robust schemes in this chapter can well handle the tracking problem as illustrated in this section.

11.5 Summary In this chapter, we have designed robust ILC schemes to continuous-time nonlinear systems with various types of nonparametric uncertainties in presence of the nonuniform trial lengths. Robust terms are designed and incorporated into the controller to compensate the unknown uncertainties. The unknown parameters are iteratively updated by using the concept of iterative learning. To demonstrate the convergence, a virtual tracking error is defined and a novel CEF is introduced for the nonuniform trial length problem. The results in this chapter clearly show the effectiveness of the proposed schemes against various types of nonparametric uncertainties. The extensions to a more general MIMO system are also elaborated. For further research, our idea can be extended along the following directions: extension to nonlinear systems with non-square uncertain input distribution matrix and extension to output or state tracking with input or state constraints. The results in this chapter are mainly based on [4].

11.5 Summary

223 2 xr,1 1st iteration 5th iteration 10th iteration 20th iteration 50th iteration

Tracking performance: first dimension

1.8

1.6

1.4

1.2

1

0.8

0.6

0.4 0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Time axis

(a) First state −0.2

Tracking performance: second dimension

xr,2 1st iteration 5th iteration 10th iteration 20th iteration 50th iteration

−0.3

−0.4

−0.5

−0.6

−0.7

−0.8

−0.9 0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Time axis

(b) Second state Fig. 11.4 Tracking performance of illustrative iterations for Case 2

0.4

0.45

0.5

224

11 CEF Techniques for Nonparameterized Nonlinear Continuous-Time Systems

References 1. Wei J, Hu Y, Sun M (2014) Adaptive iterative learning control for a class of nonlinear timevarying systems with unknown delays and input dead-zone. IEEE/CAA J Autom Sin 1(3):302– 314 2. Xu JX, Jin X, Huang D (2014) Composite energy function-based iterative learning control for systems with nonparametric uncertainties. Int J Adapt Control Signal Process 28(1):1–13 3. Li X, Huang D, Chu B, Xu JX (2016) Robust iterative learning control for systems with normbounded uncertainties. Int J Robust Nonlinear Control 26(4):697–718 4. Shen D, Xu JX (2018) Robust learning control for nonlinear systems with nonparametric uncertainties and nonuniform trial lengths. Int J Robust Nonlinear Control. https://doi.org/10.1002/ rnc.4437

Chapter 12

CEF Techniques for Uncertain Systems with Partial Structure Information

In this chapter, we apply the CEF techniques proposed in previous two chapters to uncertain systems with partial structure information, where the iteration-varying trial length problem is solved by adaptive ILC. In particular, we consider two specific types of partial structure information in this chapter. First, we consider the case that the system uncertainty consists of two parts, a time-invariant part and a time-varying part. In such case, the two parts can be learned by different types of adaptive learning law, and therefore, a mixing-type adaptive learning scheme is derived. In other words, the time-invariant part and the time-varying part are learned in differential and difference form, respectively, and their learning laws are directly combined or mixed to generate the corresponding control signal. Next, we move to consider the case that time-invariant and time-varying system uncertainties cannot be directly separated. In such case, a hybrid form of the differential and difference learning laws is proposed, where both differential and difference learning mechanisms are integrated in a unified adaptive learning scheme to derive the estimation of unknown parameters. Therefore, the hybrid design of adaptive learning laws is expected to be able to deal with time-invariant and time-varying hybrid uncertainties. For the two cases, suitable compensation mechanisms for the lost section of tracking error profile. The convergence analysis under nonuniform trial length environments is strictly derived using the proposed CEF. In short, we are motivated to propose new control schemes when partial structure information of the systems is available and the compensation mechanisms for the lost information are deeply discussed.

12.1 Problem Formulation Consider the following nonlinear dynamic system: x˙ = θ ◦T (t)ξ ◦ (x, t) + bu(t) x(0) = x0 © Springer Nature Singapore Pte Ltd. 2019 D. Shen and X. Li, Iterative Learning Control for Systems with Iteration-Varying Trial Lengths, https://doi.org/10.1007/978-981-13-6136-4_12

(12.1) 225

226

12 CEF Techniques for Uncertain Systems with Partial Structure Information

where t denotes the time, t ∈ [0, T ]. x ∈ R is the measurable system state, u ∈ R is the system control input, the constant b is the perturbed gain of the system input, θ ◦ (t) ∈ C(Rn , [0, T ]) is a vector of unknown time-varying parameters, and ξ ◦ (x, t) ∈ Rn is a known vector-valued function whose elements are assumed to be locally Lipschitz continuous with respect to x. Here n is a positive integer specifying the dimension. In addition, C(Rn , [a, b]) denotes the set of all differentiable functions over the interval [a, b] within the n-dimensional space Rn and it is abbreviated as C([a, b]) if the functions belong to R. The reference trajectory is denoted by xr (t) ∈ C([0, T ]). The following assumptions are made for the system: Assumption 12.1 The input gain b is unknown but its sign is known. Without loss of generality, we assume that b is positive and no less than b whose value is known, i.e., b ≥ b > 0. In this chapter, we consider the input gain b as a timeinvariant constant so that the unknown parameters include both time-varying part θ ◦ (t) and time-invariant part b, which motivates us to propose the mixing scheme in the next section. If the input gain is also time-varying, i.e., b(t), with prior knowledge on its lower bound, all the parameters in the system are time-varying. The treatment of this case can be regarded as a special case of the proposed scheme in the next section. In addition, the time-varying parameters make the conventional adaptive algorithms unsuitable for this system. In this chapter, we consider the iterative learning problem of the system (12.1). Thus, we add subscript k denoting the iteration number to the state and input. Our control objective is to drive the system state xk (t) to track the desired reference xr (t) asymptotically as the iteration number k goes to infinity. In order to gradually improve the tracking performance along the iteration direction, we need the following initialization assumption. Assumption 12.2 The identical initial condition, i.e., xk (0) = xr (0), ∀k, is satisfied. Assumption 12.2 is a natural and specific formulation in the ILC field, which has been widely used in numerous papers. This assumption implies that the system operation can be repeated. In practice, a perfect initial resetting may not be easy because of various situations. Motivated by this observation, some papers such as [1] have provided possible initial rectifying mechanisms. However, this issue is beyond the scope of this chapter, and we simply use Assumption 12.2 to make our discussions concentrated on the novel schemes. Moreover, we focus on the random iteration-varying operation length problem. That is, the main difficulty in designing and analyzing the ILC scheme for the system (12.1) is that the actual operation length Tk is iteration-varying and therefore may be different from the desired length T . Obviously, the actual length Tk varies in different iterations randomly and thus two cases need to be taken into account, i.e., Tk < T and Tk ≥ T . For the latter case, it is observed that only the data in the time interval [0, T ] will be used for further updating, while the data from (T, Tk ] will be discarded directly. Consequently, without loss of generality, we can regard such case

12.1 Problem Formulation

227

as Tk = T . In other words, we can only consider the problem that Tk is not beyond T . Moreover, it is reasonable to assume that there exists a minimum of Tk , denoted by Tmin with Tmin > 0. In the following, we concentrate our discussions on the case that 0 < Tmin ≤ Tk ≤ Tmax  T . We need the following assumption on the randomly nonuniform iteration lengths Tk . Assumption 12.3 Assume that Tk is a random variable, and its probability distribution function is ⎧ ⎪ t ∈ [0, Tmin ] ⎨0, (12.2) FTk (t)  P[Tk < t] = p(t), t ∈ (Tmin , Tmax ] ⎪ ⎩ 1, t > Tmax where 0 ≤ p(t) ≤ 1 is a continuous function. This assumption describes the random variable of iteration-varying lengths. From the definition we note FTk (Tmin ) = 0 (indicating the fact that the trial length cannot be shorter than Tmin ), but we should point out that the distribution function p(t) need not approach 0 as t approaches Tmin . In other words, the limitation of p(t) + + p(t) can be a positive constant. In this case, it means )  limt→Tmin from right p(Tmin that the trial length can be equal to the minimum length Tmin with a positive probability, i.e., P[Tk = Tmin ] > 0. Similarly, p(t) need not approach 1 as t approaches Tmax . In other words, p(Tmax ) can be less than 1. If p(Tmax ) < 1, it indicates that the trial length has a positive probability to achieve the full length Tmax . That is, P[Tk = Tmax ] > 0, which actually is 1 − p(Tmax ) according to the definition. It is evident that the above definition of probability distribution function satisfies the leftcontinuous property. We should emphasize that the probability distribution function is not required to be known prior, because the design of control laws and parameter update laws given below is independent of the distribution function. In other words, no specific description is imposed to the randomly iteration-varying lengths. Therefore, Assumption 12.3 provides a general formulation which satisfies most practical applications. Besides, from this viewpoint, we can conclude that the distribution function can vary from trial to trial as long as the above conditions are satisfied. Moreover, based on this assumption, we can further define a sequence of random variables satisfying Bernoulli distribution (see γk (t) defined in the next section) and then modify the tracking error signal to facilitate the design of ILC algorithms (see the next sections). Now we can make the following statement about our problem: Problem statement: The control objective of this chapter is to design suitable learning control algorithms based on the partial structure information for the nonlinear system (12.1) with randomly varying iteration lengths. Using the available information of previous iterations, the ILC algorithms can guarantee the system state to track the desired reference as the iteration number goes to infinity; that is, the tracking error will converge to zero along the iteration axis.

228

12 CEF Techniques for Uncertain Systems with Partial Structure Information

12.2 Time-Invariant and Time-Varying Mixing Scheme In this section, we consider the first type of partial structure information. Generally, to model an unknown system, we usually apply the time-varying parameters so that the time-varying uncertainties can be included. However, in many practical applications, we may have a prior knowledge of the structure separation that some unknown parameters are time-varying and the rest are time-invariant. In such case, we can design different learning laws for time-invariant and time-varying parameters, respectively, and then combine them to generate the control signal. In order to facilitate the learning of time-varying parameters and time-invariant parameters, respectively, we separate θ ◦T (t)ξ ◦ (x, t) into θ ◦T (t)ξ ◦ (x, t) = θ ◦1 T (t)ξ ◦1 (x, t) + θ ◦2 T ξ ◦2 (x, t), where θ ◦1 (t) ∈ C(Rn 1 , [0, T ]) is an unknown time-varying parameter vector (but should be iteration-invariant), θ ◦2 ∈ Rn 2 is an unknown timeinvariant parameter vector and both ξ ◦1 (x, t) ∈ Rn 1 and ξ ◦2 (x, t) ∈ Rn 2 are known continuous vector-valued functions whose elements are locally Lipschitz continuous with respect to x. n 1 and n 2 are positive integers specifying dimensions. Define the tracking error ek (t) = xk (t) − xr (t), t ≤ Tk .

(12.3)

Then, the error dynamics at the kth iteration is e˙k = b[u k + b−1 θ ◦1 T (t)ξ ◦1,k + b−1 θ ◦2 T ξ ◦2,k − b−1 x˙r ] = b[u k + θ 1T (t)ξ 1,k + θ 2T ξ 2,k ]

(12.4)

where θ 1 (t) = b−1 θ ◦1 (t), ξ 1,k = ξ ◦1 (xk , t), θ 2 = [b−1 θ ◦2 , −b−1 ]T , and ξ 2,k = [ξ ◦2 (xk , t)T , x˙r ]T . Note that θ 1 (t) denotes all the time-varying parameters, whereas θ 2 presents all the time-invariant parameters. We observe that the input gain b has been involved in both θ 1 (t) and θ 2 . Since b is a time-invariant constant, θ 2 denotes the time-invariant part of unknown parameters. If b is of time-varying type, i.e., b(t), then both θ 1 and θ 2 are time-varying. This case can be treated by using the difference adaptive learning law given below. The learning control law at the kth iteration is constructed as T

u k = −b−1 μek −  θ 1,k (t)ξ 1,k −  θ 2,k ξ 2,k T

T

(12.5)

θ 2,k is to learn θ 2 . where μ > 0 is the feedback gain,  θ 1,k (t) is to learn θ 1 (t), and  For the purpose of learning these two types of parameters, we apply the mixingtype parameter updating laws. That is, for the time-varying parameter θ 1 (t) which is iteratively invariant, the difference adaptive learning law is employed  θ 1,k

  θ 1,k−1 + η1 ξ 1,k ek , t ≤ Tk =  Tk < t ≤ T θ 1,k−1 ,

(12.6)

12.2 Time-Invariant and Time-Varying Mixing Scheme

229

with  θ1,−1 (t) = 0, ∀t ∈ [0, T ], where η1 > 0 is the learning gain. For the constant part θ 2 , the differential adaptation law is used  η2 ξ 2,k ek , t ≤ Tk ˙  θ 2,k = 0, Tk < t ≤ T

(12.7)

with  θ2,k (0) =  θ2,k−1 (T ), and  θ2,0 (0) = 0, where η2 > 0 is the learning gain. In order to facilitate the convergence analysis of the proposed ILC algorithms, we introduce a random variable γk (t) and compensate the absent control information for Tk < t ≤ T . Let γk (t) be a random variable that satisfies the Bernoulli distribution and takes binary values 0 or 1. The relation γk (t) = 1 represents the event that the operation of system (12.1) can continue until the time instant t in the kth iteration. The probability of this event is q(t), where 0 < q(t) ≤ 1 is a predefined function of time t. The case γk (t) = 0 denotes the event that the operation of system (12.1) ends before the time instant t, which occurs with a probability of 1 − q(t). Remark 12.1 Although we do not actually require detailed information of the random variable γk (t) in the following design and analysis of ILC algorithms, we can calculate the probability P[γk (t) = 1] to clarify the inherent relationship between the random iteration length Tk and the newly defined variable γk (t). From Assumption 12.3, it is evident that γk (t) is equal to 1 when 0 ≤ t < Tmin because the operation of system (12.1) will not stop within [0, Tmin ). Moreover, when t is located in [Tmin , T ], the event γk (t) = 1 implies that the operation will end at or after the time instant t, therefore, P[γk (t) = 1] = P[Tk ≥ t] = 1 − P[Tk < t] = 1 − P[Tk ≤ t] = 1 − FTk (t), where P[Tk = t] = 0 is employed. In short, we have q(t) = 1 − FTk (t). For the kth iteration, the operation only runs during the time interval [0, Tk ], whereafter the system returns to its initial position and starts the next trial. Therefore, we only have the tracking information for 0 ≤ t ≤ Tk . In addition, Tk varies randomly for different iterations. To ensure a reasonable formulation of the ILC algorithms, the missing tracking error have been compensated with zero in most existing studies. Different from those papers, we introduce a new complement of the missing tracking error, which will be called the virtual tracking error k (t), 0 ≤ t ≤ T in the sequel. Specifically, the virtual tracking error k (t) is defined as follows:  k (t) =

ek (t), 0 ≤ t ≤ Tk ek (Tk ), Tk < t ≤ T

(12.8)

That is, k (t) = γk (t)ek (t) + (1 − γk (t))ek (Tk ), 0 ≤ t ≤ T . The compensation mechanism in the virtual tracking error is mainly for the convergence analysis, and they are not used for the controller design and parameter update. In other words, the compensation mechanism in (12.8) will not influence the practical implementation of the control law (12.5) and parameter update laws (12.6)–(12.7).

230

12 CEF Techniques for Uncertain Systems with Partial Structure Information

Now we can derive the convergence property of the ILC scheme (12.5)–(12.7) in the following theorem. The L2Tk -norm of the tracking error ek (t) is defined as T 1 ek L2T  ( 0 k ek2 dt) 2 , where Tk is the actual operation length. k

Theorem 12.1 For system (12.1), under Assumptions 12.1–12.3, the ILC scheme consisting of learning control law (12.5) and updating laws (12.6)–(12.7) ensures that the tracking error converges to zero in L2Tk -norm over [0, T ] with a probability of 1 as the iteration number k approaches to infinity. Proof To show the learning property, define a CEF as 1 1 E k (t) = k2 (t) + 2 2η1



t

0

1 T

T b

θ 1,k (τ )

θ 1,k (τ )dτ + bθ θ 2,k 2η2 2,k

(12.9)

θ 1,k (t) − θ 1 (t) and

θ 2,k   θ 2,k − θ 2 are the estimation errors. where

θ 1,k (t)   The proof will be carried out in three steps. In Step A, we derive the difference of the CEF. In Step B, we will prove the convergence of the tracking error. In Step C, we will show the boundedness of the system state and the control signal. Step A: Difference of the CEF. Consider the CEF at the time instant t = T E k (T ) =

1 2 1 k (T ) + 2 2η1

0

T

1 T T b

θ 1,k

bθ (T )

θ 2,k (T ), θ 1,k dτ + 2η2 2,k

(12.10)

whose difference is ΔE k (T ) = E k (T ) − E k−1 (T ) T 1 1 T T b(

θ 1,k

θ 1,k−1

θ 1,k −

θ 1,k−1 )dτ = k2 (T ) + 2 2η1 0 1 T T b[θ 2,k (T )

θ 2,k (T ) −

θ 2,k−1 (T )

θ 2,k−1 (T )] + 2η2 1 2 (T ). − k−1 2

(12.11)

Let us examine the first term on the right-hand side (RHS) of (12.11). According to the identical initial condition Assumption 12.2, the error dynamics (12.4) and the control law (12.5), we can obtain that 1 1 2 k (T ) = ek2 (Tk ) 2 2 Tk 1 2 ek e˙k dτ = ek (0) + 2 0 Tk ek b(u k + θ 1T (t)ξ 1,k + θ 2T ξ 2,k )dτ = 0

12.2 Time-Invariant and Time-Varying Mixing Scheme



Tk

T T ek b(−b−1 μek −

θ 1,k ξ 1,k −

θ 2,k ξ 2,k )dτ 0 Tk Tk T 2 ek dτ − bek

≤ −μ θ 1,k ξ 1,k dτ 0 0 Tk T bek

− θ 2,k ξ 2,k dτ.

=

231

(12.12)

0

For the second term on the RHS of (12.11), according to the updating law (12.6), we have 1 T

T (θ θ 1,k −

θ 1,k−1

θ 1,k−1 ) 2η1 1,k 1

(θ 1,k −

θ 1,k−1 )T (

θ 1,k +

θ 1,k−1 ) = 2η1 1  (θ 1,k −  θ 1,k−1 )T ( θ 1,k +  θ 1,k−1 − 2θ 1 ) = 2η1 1  (θ 1,k −  θ 1,k−1 )T ( θ 1,k −  θ 1,k−1 ) =− 2η1 1 θ 1,k −  θ 1,k−1 )T ( θ 1,k − θ 1 ) + ( η1 η1 T = − ξ 1,k 2 ek2 + ek

θ 1,k ξ 1,k 2

(12.13)

where  ·  denotes the Euclidean norm of a vector. Thus, T 1 T T b(

θ 1,k

θ 1,k−1

θ 1,k −

θ 1,k−1 )dτ 2η1 0 Tk 1 T T = b(

θ 1,k

θ 1,k−1

θ 1,k −

θ 1,k−1 )dτ 2η1 0 Tk η1 Tk T 2 2 =− bξ 1,k  ek dτ + bek

θ 1,k ξ 1,k dτ. 2 0 0

(12.14)

From the updating law (12.7), the third term on the RHS of (12.11) becomes 1 T T b[θ 2,k (T )

θ 2,k (T ) −

θ 2,k−1 (T )

θ 2,k−1 (T )] 2η2 T b b T T ˙

θ 2,k dτ + = θ 2,k (0) θ 2,k

θ (0)

η2 0 2η2 2,k b T − (T )

θ 2,k−1 (T ) θ 2η2 2,k−1

232

12 CEF Techniques for Uncertain Systems with Partial Structure Information

=

Tk

b T T bek

θ 2,k (0) θ 2,k ξ 2,k dτ + θ 2,k (0)

2η 2 0 b T − (T )

θ 2,k−1 (T ). θ 2η2 2,k−1

(12.15)

Then, substituting (12.12)–(12.15) back into (12.11) leads to

Tk

ΔE k (T ) ≤ − μ 0

ek2 dτ

η1 − 2



Tk 0

bξ 1,k 2 ek2 dτ

b T b T θ 2,k (0) − (T )

θ 2,k−1 (T ) θ (0)

θ 2η2 2,k 2η2 2,k−1 1 2 (T ). − k−1 2

+

(12.16)

Considering the fact that  θ2,k (0) =  θ2,k−1 (T ), we have

θ 2,k (0) =

θ 2,k−1 (T ), thus

Tk

ΔE k (T ) ≤ − μ 0

ek2 dτ −

η1 2



Tk 0

bξ 1,k 2 ek2 dτ

1 2 − k−1 (T ). 2

(12.17)

Step B: Convergence of the Tracking Error. According to (12.17), it can be derived that the finiteness of E k (T ) is guaranteed for any iteration provided E 0 (T ) is finite. In the following, we will show the finiteness of E 0 (t). Note that 1 1 E 0 (t) = 02 (t) + 2 2η1

0

t

1 T

T b

θ 1,0

bθ θ 2,0 θ 1,0 dτ + 2η2 2,0

whose derivative is 1 T

1 T

bθ θ 1,0 + b

θ θ˙ 2,0 . E˙ 0 (t) = γ0 e0 e˙0 + 2η1 1,0 η2 2,0

(12.18)

From (12.4)–(12.5), we can obtain that b T T γ0 e0 e˙0 = γ0 (− μe02 − be0

θ 1,0 ξ 1,0 − be0

θ 2,0 ξ 2,0 ) b T T ≤ −γ0 μe02 − γ0 be0

θ 1,0 ξ 1,0 − γ0 be0

θ 2,0 ξ 2,0 .

For the second term on the RHS of (12.18), from the fact that  θ 1,−1 (t) = 0, we have

12.2 Time-Invariant and Time-Varying Mixing Scheme

233

1 T

1 T

1 T

T bθ 1,0 θ 1,0 = b(θ 1,0 θ 1,0 −

θ 1,−1

bθ θ 1,−1 ) + θ 1,−1 2η1 2η1 2η1 1,−1 η1 b T T = − bγ0 ξ 1,0 2 e02 + γ0 be0

θ θ1 θ 1,0 ξ 1,0 + 2 2η1 1 where  θ 1,k =  θ 1,k−1 + γk η1 ξ 1,k ek from (12.6) is employed. According to the updating law  θ˙ 2,k = γk η2 ξ 2,k ek from (12.7) and using θ˙ 2 = 0, the last term on the RHS of (12.18) can be expressed as 1 T

1 T  T bθ θ˙ 2,0 = b

θ (θ˙ 2,0 − θ˙ 2 ) = γ0 be0

θ 2,0 ξ 2,0 . η2 2,0 η2 2,0 Therefore, η1 b T E˙ 0 (t) ≤ − γ0 μe02 − bγ0 ξ 1,0 2 e02 + θ θ1 2 2η1 1 b T ≤ θ θ 1. 2η1 1 Note that θ 1 (t) is continuous, i.e., it is bounded over the time interval [0, T ]. Hence, there exists a constant M such that  b T θ 1 θ 1 < ∞. M = max t∈[0,T ] 2η1 Considering e0 (0) = 0,  θ 2,0 (0) = 0 and the boundedness of θ 2 , it is clear that



E 0 (t) ≤|E 0 (0)| +



˙ E 0 (τ )dτ

0 t

1 T

E˙ 0 (τ ) dτ

≤ bθ 2,0 (0)θ 2,0 (0) + 2η2 0 t b T ≤ θ θ2 + Mdτ 2η2 2 0 b T θ θ 2 + M T < ∞. ≤ 2η2 2 t

Apparently, we have shown the finiteness of E 0 (T ), which further implies the finiteness of E k (T ), ∀k ∈ N.

234

12 CEF Techniques for Uncertain Systems with Partial Structure Information

From (12.17), it can be derived that E k (T ) ≤E 0 (T ) − μ

k  j=1

lim E k (T ) ≤E 0 (T ) − μ lim

k→∞

k→∞

Tj

0

e2j dτ,

k  j=1

0

Tj

e2j dτ.

T Since E 0 (T ) is finite and E k (T ) is positive, limk→∞ 0 k ek2 dτ = 0 is ensured. Hence T 1 ek converges to zero in L2Tk -norm, which is defined as ek L2T  ( 0 k ek2 dt) 2 . Note k that Tk is a random variable, thus we can only claim the convergence of the available output or tracking error. Step C: Boundedness Property. Next, we will examine the boundedness property of the system state xk and the control signal u k . Note that, we have proved the boundedness of E k (T ), from which we need to further derive the boundedness of E k (t) for any t ∈ [0, T ]. According to the definition of E k (t) and the finiteness of E k (T ), the boundedness T T T θ 2,k (T )

θ 2,k (T ) are ensured for any k. Therefore, ∀k ∈ N, there θ 1,k

θ 1,k dτ and

of 0

exist finite constants L 1 and L 2 such that T 1 T T b

θ 1,k

b

θ 1,k

θ 1,k dτ ≤ θ 1,k dτ ≤ L 1 < ∞, 2η1 0 0 1 T 1 T bθ (0)

θ 2,k+1 (0) = bθ (T )

θ 2,k (T ) ≤ L 2 < ∞. 2η2 2,k+1 2η2 2,k 1 2η1



t

Hence, from the CEF (12.9), we can obtain E k (t) ≤

1 2 1 T k (t) + L 1 + bθ (t)

θ 2,k (t). 2 2η2 2,k

(12.19)

On the other hand, analogous to the derivation of (12.16), we have 1 T bθ (0)

θ 2,k+1 (0) 2η2 2,k+1 1 1 T bθ (t)

θ 2,k (t) − k2 (t) − 2η2 2,k 2 1 T 1 bθ (t)

θ 2,k (t) − k2 (t). ≤L 2 − 2η2 2,k 2

ΔE k+1 (t) ≤

(12.20)

Adding (12.19)–(12.20) yields E k+1 (t) = E k (t) + ΔE k+1 (t) ≤ L 1 + L 2 .

(12.21)

12.2 Time-Invariant and Time-Varying Mixing Scheme

235

From (12.21), we can derive that E k (t) is finite for all k ∈ N since E 0 (t) is bounded. t θ 1,k 2 dτ and  θ 2,k (t) are all bounded. Considering that ξ 1,k and Hence, both xk , 0  ξ 2,k are local Lipschitz continuous with respect to xk , then, the boundedness of xk guarantees the boundedness of ξ 1,k and ξ 2,k . Thereafter, from learning control law (12.5), it is clear that u k is bounded in L2Tk -norm. This theorem implies that the available tracking performance can be gradually improved along the iteration axis, even though the trial length varies randomly for different iterations. To be specific, when the operation ends at time instant Tk , the CEF E k (t) is decreased for t ∈ [0, Tk ] compared with the previous iteration. From the control law (12.5) and updating laws (12.6)–(12.7), we can see that the improvement works during the actual operation interval [0, Tk ]; however, for the left part (Tk , T ], no updating is imposed. Remark 12.2 The inherent idea of the virtual tracking error εk (t) can be understood from Eq. (12.12). First, the virtual tracking error should be a constant during the missing interval (Tk , T ] so that its derivative is zero. In this case, the virtual tracking error will not affect the integral of involved quantities. Moreover, if the virtual tracking error is defined as zero, which is adopted in most existing papers, the derivations in (12.12) will no longer hold, and therefore, the uncertain terms in (12.14) and (12.15) cannot be canceled accordingly. Remark 12.3 This theorem mainly provides the asymptotical convergence of the proposed scheme as k approaches to infinity. However, in practical applications, one may be also interested in the possible convergence speed of the proposed scheme. Unlike the contraction mapping method, where an explicit expression of the convergence speed can be formulated, here we propose a modified version of the CEF method for continuous-time nonlinear systems, which makes it difficult to obtain a precise description of the speed. However, we may give a rough estimate of the required iterations for converging into a predefined zone of zero from (12.17) using the techniques in [2]. For example, the tracking error ek L2T will enter the -zone k

of zero after at most E 0 (T )/(μ 2 ). We should note that an accurate estimation of the convergence speed is still open and can be deeply investigated in the next. From the proof, we observe that we can primarily ensure the L2 -norm boundedness  of θ 1,k . In other words, we do not know the upper and lower bounds of time-varying parameter θ 1,k (t). However, in many control systems, these knowledge are known a priori. In this case, we want to know whether the control performance can be improved if we incorporate additional system bounding information in the learning control. Specifically, we can modify the updating law (12.6) as follows: 

P( θ 1,k−1 ) + η1 ξ 1,k ek , t ≤ Tk  θ 1,k =  P(θ 1,k−1 ), Tk < t ≤ T where the operator P(φ) for a vector φ = [φ1 , . . . , φn ]T is defined as

(12.22)

236

12 CEF Techniques for Uncertain Systems with Partial Structure Information

P(φ) = [P(φ1 ), . . . , P(φn )]T ,  φi , |φi | ≤ φi∗ P(φi ) = ∗ φi · sign(φi ), |φi | > φi∗ where sign(·) is the sign function and φi∗ (i = 1, . . . , n) are the known projection bounds. The ILC scheme consisting of control law (12.5) and the updating law (12.22)– (12.7) can guarantee the following uniform convergence property. Theorem 12.2 For system (12.1), under Assumptions 12.1–12.3, the ILC scheme consisting of learning control law (12.5) and updating laws (12.22) and (12.7) ensures that the tracking error converges to zero uniformly over [0, T ] with a probability of 1 as the iteration number k approaches to infinity. Proof We apply the same CEF defined in (12.9), the relations (12.11), (12.12)– (12.15) still hold. But the relation (12.14) may be different. The property [φ − ψ]2 ≥ [φ − P(ψ)]2 for any suitable vectors φ and ψ can be verified. Using the new updating law (12.22), and comparing with (12.13), we can obtain 1 T

T (θ θ 1,k −

θ 1,k−1

θ 1,k−1 ) 2η1 1,k  1  T

θ 1,k−1 − θ 1 )T ( θ 1,k−1 − θ 1 ) = θ 1,k θ 1,k − ( 2η1 T    1  T

θ 1,k θ 1,k − P( P( θ 1,k−1 ) − θ 1 θ 1,k−1 ) − θ 1 ≤ 2η1 T   1 

θ 1,k − P( θ 1,k + P( θ 1,k−1 ) + θ 1

θ 1,k−1 ) − θ 1 = 2η1  T  1  θ 1,k − P( θ 1,k + P( θ 1,k−1 )  θ 1,k−1 ) − 2θ 1 = 2η1 T   1  θ 1,k − P( θ 1,k − P( θ 1,k−1 )  θ 1,k−1 ) =− 2η1  T  1  θ 1,k − P( θ 1,k − θ 1 θ 1,k−1 )  + η1 which further implies that 1 2η1

0

T

T T b(

θ 1,k

θ 1,k−1

θ 1,k −

θ 1,k−1 )dτ

T   T   1 ≤− θ 1,k − P( b  θ 1,k − P( θ 1,k−1 )  θ 1,k−1 ) dτ 2η1 0  T  1 T  + θ 1,k − θ 1 dτ b θ 1,k − P( θ 1,k−1 )  η1 0

12.2 Time-Invariant and Time-Varying Mixing Scheme

237

Tk   T   1 θ 1,k − P( =− b  θ 1,k − P( θ 1,k−1 )  θ 1,k−1 ) dτ 2η1 0  T  1 Tk  + θ 1,k − θ 1 dτ b θ 1,k − P( θ 1,k−1 )  η1 0 Tk η1 Tk T 2 2 =− bξ 1,k  ek dτ + bek

θ 1,k ξ 1,k dτ. 2 0 0

(12.23)

Note that the relation (12.23) is the same as (12.14). Consequently, the same result as (12.17) can be obtained

Tk

ΔE k (T ) ≤ − μ 0

ek2 dτ

η1 − 2



Tk 0

bξ 1,k 2 ek2 dτ

1 2 − k−1 (T ) 2 ≤ 0.

(12.24)

Then, the pointwise convergence of ek can be derived according to Theorem 12.1. In addition, the boundedness of ek (t) leads to the boundedness of xk (t), further θ 1,k ,  θ 2,k , u k (t) and x˙k (t). Moreover, the boundensures the boundedness of ξ 1,k , ξ 2,k ,  edness of x˙k (t) implies the uniform continuity of xk (t) and, thereafter, the uniform continuity of the tracking error ek (t). In other words, the uniform convergence is guaranteed. This theorem implies that if the bound information of the time-varying parameter θ 1 (t) is known a priori, the projection-based type updating law will guarantee the boundedness of all related signals. Therefore, the uniform convergence of the tracking error over the operation interval can be ensured with the help of Barbalat lemma.

12.3 Differential-Difference Hybrid Scheme In the last section, we propose a mixing-type ILC scheme to combine both timeinstant- and time-varying parameters. However, in certain systems, the clear parameter separation is difficult to obtain if possible. If the time-invariant and time-varying parameters are involved together, a natural question is whether we could still apply an ILC scheme to suitably cope with such case. In fact, we will introduce a hybrid differential-difference adaptive scheme in this section. In particular, the differential and difference learning are integrated in one learning law. In order to make a strict convergence analysis, we consider the system parameters to be time-invariant, which is a special case of the last section; however, we propose a hybrid mechanism that the designer can tune the regulating factor according to the ratio of the time-invariant and time-varying parameters.

238

12 CEF Techniques for Uncertain Systems with Partial Structure Information

The considered nonlinear dynamic system is x˙ = θ T ξ (x, t) + bu(t) x(0) = x0

(12.25)

where θ ∈ Rn is an unknown constant vector. Now, the error dynamics at the kth iteration is (12.26) e˙k = x˙k − x˙r = θ T ξ k + bu k − x˙r with t ≤ Tk . The proposed learning control algorithm at the kth iteration is T T u k (t) = b−1 [− θ k ξ k · sign(ek θ k ξ k ) − μek − x˙r · sign(ek x˙r )]

(12.27)

where μ > 0 is the feedback gain to be designed, sign(·) denotes the sign function, and  θ k is the estimation. The updating law for  θ k is θ k (t) + α θ k−1 (t) + r (xk (t), t) (1 − α) θ˙ k (t) = −α 

with r (xk (t), t) =

ηξ k ek , t ≤ Tk 0, Tk < t ≤ T

(12.28)

(12.29)

θ k (0) =  θ k−1 (T ). η > 0 is the learning where α ∈ [0, 1],  θ−1 (t) = 0. For α ∈ [0, 1),  gain. We should emphasize that the factor α is a man-tuned factor according to the time-invariance degree of parameters. That is, all parameters are more likely timeinvariant, we can set a much smaller α; otherwise, we can select a larger α. As a matter of fact, if α = 0, the update law (12.28) turns into a completely differential one (for time-invariant parameters); otherwise if α = 1, the update law (12.28) turns into a completely difference one (for time-varying parameters). Since α varies in [0, 1], we call (12.28) a hybrid-type learning law, differing from the mixing-type learning laws given in the last section. Different from (12.8), in this section, we use the following traditional method to compensate the absent tracking error:  ek (t), 0 ≤ t ≤ Tk , k (t) = 0, Tk < t ≤ T.

(12.30)

Now, we can present our result in the following convergence theorem: Theorem 12.3 For system (12.25), under Assumptions 12.1–12.3, the ILC scheme consisting of learning control law (12.27) and hybrid law (12.28) ensures that the

12.3 Differential-Difference Hybrid Scheme

239

modified tracking error k (t) converges to zero pointwisely over [0, T ] and the actual tracking error ek (t) converges to zero in L2Tk -norm sense, with a probability of 1 as the iteration number k approaches to infinity. Proof From (12.28), we can easily observe that the updating law is different for α ∈ [0, 1) and α = 1, respectively. Therefore, we should discuss the two cases separately. Different from the previous section, here, the proof comprises two steps. In the first step, the difference of the CEF between two consecutive iterations is derived. In the second step, the boundedness of the related signals and the convergence of the tracking error are proved. We first consider the case α ∈ [0, 1). For this case, to show the convergence, we modify the CEF as follows: E k (t) =

1 2 1 T

 (t) + (1 − α)

θ θk 2 k 2η k

(12.31)

θk − θ. where

θk   Step A: Difference of the CEF. The time derivative of (12.31) is given by 1 T

θ θ˙ k E˙ k = k ˙k + (1 − α)

η k 1 T = k ˙k + (1 − α)

θ θ˙ k . η k

(12.32)

When t ≤ Tk , in view of (12.26)–(12.29), we have 1 T E˙ k = ek (θ T ξ k + bu k − x˙r ) +

θ k + α θ k−1 + ηξ k ek ) θ (−α η k α T   T = θ k ξ k ek + ek bu k − ek x˙r −

θ (θ k − θ k−1 ) η k b b T T T = θ k ξ k ek − ek θ k ξ k · sign(ek θ k ξ k ) − μek2 b b b α T   − ek x˙r · sign(ek x˙r ) − ek x˙r − θ k (θ k − θ k−1 ) b η α T   2 ≤ − μek − θ k (θ k − θ k−1 ) η α T   α T

≤ − θ k (θ k − θ k−1 ) = −

θ (θ k − θ k−1 ). η η k When Tk < t ≤ T if Tk < T , according to (12.29)–(12.30), we can obtain

(12.33)

240

12 CEF Techniques for Uncertain Systems with Partial Structure Information

1 T θ k + α θ k−1 + 0) E˙ k = 0 +

θ (−α η k α T

=−

θ (θ k − θ k−1 ). η k

(12.34)

Combining (12.33)–(12.34), we come to α T

E˙ k ≤ −

θ (θ k − θ k−1 ). η k

(12.35)

 2  2 T  T θ k  + 14 

θ k−1  , we derive that θ k−1 ≤ 

Using Young’s inequality

θk

 2 α T

α α   T 2 

θk  + E˙ k ≤ −

θ k θ k + 

θ k−1  η η 4η 2 α   

≤ θ k−1 . 4η

(12.36)

θ 0 (0) =  θ −1 (T ), we can get that E 0 (t) and hence 0 (t) Considering  θ −1 (t) = 0 and 

and θ 0 (t) are bounded for any t ∈ [0, T ]. Now, we will use another positive definite function as E¯ k (t) = E k (t) + α

0

t

1 T

θ θ k dτ. 2η k

The difference of E¯ k (T ) is Δ E¯ k (T ) = E k (T ) − E k−1 (T ) +

α 2η



T 0

T T (

θk

θ k−1

θk −

θ k−1 )dτ

1 − α T 1 θ k (T ) = k2 (T ) + θ (T )

2 2η k 1 − α T 1 2 (T ) − θ k−1 (T ) − k−1 θ (T )

2 2η k−1 T α T T + (

θk

θ k−1

θk −

θ k−1 )dτ 2η 0 T 1 2 1 = − k−1 (T ) + k2 (0) + k ˙k dτ 2 2 0 1 − α T

1 − α T + θ k−1 (T ) θ (0)θ k (0) − θ (T )

2η k 2η k−1 T 1 − α T

+ θ k θ˙ k dτ η 0 T α T T + (

θk

θ k−1

θk −

θ k−1 )dτ 2η 0

(12.37)

12.3 Differential-Difference Hybrid Scheme

241

1 2 1 = − k−1 (T ) + k2 (0) 2 2 Tk T α T T

(ek e˙k + θ k ξ k ek )dτ − + θ¯ k θ¯ k dτ 2η 0 0  1 − α  T

T θ k (0)θ k (0) −

θ k−1 (T )

θ k−1 (T ) + 2η Tk T α 1 T 2 ≤−μ ek dτ − θ¯ k θ¯ k dτ + k2 (0) 2η 2 0 0  1 − α  T

T θ k (0)θ k (0) −

θ k−1 (T )

θ k−1 (T ) + 2η 1 2 − k−1 (T ) 2

(12.38)

T T 1 T

1 ¯T ¯ where 2η (θ k θ k −

θ k−1

θ k (t) −  θ k−1 (t). θ k θ k + η1

θ k−1 ) = − 2η θ k θ¯ k with θ¯ k (t) =  Taking k (0) = 0 and  θ k (0) =  θ k−1 (T ) into account, we can obtain that

Δ E¯ k (T ) ≤ −μ



T 0

k2 dτ −

α 2η

0

T

T θ¯ k θ¯ k dτ ≤ 0.

(12.39)

Step B: Convergence of the Tracking Error. From (12.39), combining the fact that E¯ 0 (T ) is bounded due to the boundedness of E 0 (t) over [0, T ], we can get that E¯ k (T ) is bounded for all k ∈ N, which implies T 1 T T

θ k 2 dτ is the boundedness of E k (T ) and 0 2η θk

θ k dτ , thus it concludes that 0 

bounded for all k ∈ N. From (12.36), considering k (0) = 0, we are evident to have

t

E k (t) = E k (0) +

E˙ k (τ )dτ

0

t 2 α  1 T

1 

≤ k2 (0) + (1 − α)

θ k (0)θ k (0) + θ k−1  dτ 2 2η 0 4η T 2  α 1 T 

≤ (1 − α) θ k−1 (T )

θ k−1 (T ) + θ k−1  dτ 2η 0 4η

(12.40)

θ k (t) and u k (t) are which implies that E k (t) is bounded for all k ∈ N, hence, xk (t),

all bounded for all k ∈ N. From (12.39), it is clear that E¯ k (T ) = E¯ 0 (T ) +

k 

Δ E¯ j (T )

j=1

≤ E¯ 0 (T ) − μ

k  j=1

0

T

 2j dτ −

k α  T ¯T ¯ θ θ j dτ. 2η j=1 0 j

(12.41)

242

12 CEF Techniques for Uncertain Systems with Partial Structure Information

That is, μ

k  j=1

T 0

 2j dτ +

k α  T ¯T ¯ θ θ j dτ ≤ E¯ 0 (T ) − E¯ k (T ). 2η j=1 0 j

(12.42)

It is evident that E¯ k (t) is bounded since E k (t) is bounded. Hence, it concludes T T T that limk→∞ 0 k2 dτ = 0 and, if α = 0, limk→∞ 0 θ¯ k θ¯ k dτ = 0. Moreover, x˙k (t) θ k (t) and u k (t). Consequently, it can be is finite due to the boundedness of xk (t),

derived that limk→∞ k2 (t) = 0, ∀t ∈ [0, T ]. Hence limk→∞ k (t) = 0, ∀t ∈ [0, T ]. Next, we proceed to the case α = 1. For this case, we employ the following CEF to evaluate the learning property: 1 E k (t) = k2 (t) + 2

0

t

1 T

θ θ k dτ, 2η k

(12.43)

θk − θ. where

θk   Step A: Difference of the CEF. The derivative of (12.43) is 1 T

E˙ k (t) = k ˙k +

θ θk 2η k 1 T

= k ˙k +

θ θk − 2η k 1 ¯T ¯ = k ˙k − θ θk + 2η k 1 ¯T ¯ ≤ − μk2 − θ θk 2η k 1 T

≤

θ θ k−1 . 2η k−1

1 T

1 T

θ k−1 θ k−1 +

θ θ k−1 2η 2η k−1 1 T ¯ 1 T

θ θk +

θ θ k−1 η k 2η k−1 1 T

+

θ θ k−1 2η k−1 (12.44)

θ 0 (t) = r (x0 (t), t). Hence, it can Since α = 1 and  θ −1 (t) = 0, from (12.28), there is   be derived that θ 0 (0) is bounded since x0 (0) is bounded. Hereafter, from (12.44), one can conclude that E 0 (t) is bounded for all t ∈ [0, T ]. Consider the difference of E k (T ) 1 1 2 ΔE k (T ) = k2 (T ) − k−1 (T ) 2 2 T 1 T

1 T

+ θk θk −

θ k−1 θ k−1 dτ 2η 2η 0 Tk 1 2 1 2 = − k−1 (T ) + k (0) + ek e˙k dτ 2 2 0

12.3 Differential-Difference Hybrid Scheme

243

1 T

1 T

+ θ θk − θ θ k−1 dτ 2η k 2η k−1 0 Tk Tk 1 T ≤ −μ ek2 dτ − θ¯ k θ¯ k dτ 2η 0 0 1 2 1 2 − k−1 (T ) + k (0). 2 2

Tk



(12.45)

Combining the fact that k (0) = 0, we can obtain

T

ΔE k (T ) ≤ −μ 0

k2 dτ

1 − 2η

0

T

T θ¯ k θ¯ k dτ ≤ 0.

(12.46)

Step B: Convergence of the Tracking Error. According to (12.46), it can be derived that E k (T ) is bounded for all k ∈ N t 1 T

θk

θ k dτ is guaransince E 0 (T ) is bounded. Thereafter the boundedness of 0 2η t 1 T

teed. In other words, there exists a finite constant L satisfying 0 2η θk

θ k dτ ≤ T 1 T



θ θ k dτ ≤ L < ∞. Hence, we can obtain 0 2η k

E k (t) =

1 2  (t) + 2 k



t 0

and E k−1 (t) ≤

1 T

1 θ k θ k dτ ≤ k2 (t) + L 2η 2

(12.47)

1 2  (t) + L . 2 k−1

(12.48)

Moreover, from (12.45), we have ΔE k (t) = E k (t) − E k−1 (t) t t 1 1 2 1 T k2 dτ − (t) + k2 (0) θ¯ k θ¯ k dτ − k−1 ≤ −μ 2η 0 2 2 0 1 2 1 2 (12.49) ≤ k (0) − k−1 (t). 2 2 Combining (12.48), (12.49) and k (0) = 0, one can conclude that E k (t) ≤

1 2  (0) + L = L . 2 k

(12.50)

θ k (t) and u k (t) are all bounded The finiteness of E k (t) implies that both xk (t),

∀k ∈ N, ∀t ∈ [0, T ].

244

12 CEF Techniques for Uncertain Systems with Partial Structure Information

Analogous to the derivation of (12.42), we can get μ

k 

T 0

j=1

k 1  T ¯T ¯ + θ θ j dτ ≤ E 0 (T ) − E k (T ), 2η j=1 0 j

 2j dτ

(12.51)

T which leads to the fact limk→∞ 0 k2 dτ = 0 (which is equivalent to ek L2T ) and k T T limk→∞ 0 θ¯ k θ¯ k dτ = 0. In fact, tacking k (0) = 0 into account, (12.45) becomes

t

ΔE k (t) ≤ −μ 0

k2 dτ

1 − 2η



t 0

1 2 T (t) ≤ 0. θ¯ k θ¯ k dτ − k−1 2

(12.52)

Therefore, k  1 j=1

2

2 (t) + μ k−1

k  j=1

+

t

0

 2j dτ

k 1  t ¯T ¯ θ θ j dτ ≤ E 0 (T ) − E k (T ). 2η j=1 0 j

(12.53)

It is clear that limk→∞ k2 (t) = 0 since E k (t) is bounded. consequently, limk→∞ k (t) = 0, ∀t ∈ [0, T ]. This theorem implies that the hybrid differential-difference learning law can guarantee the convergence of the tracking error, even if the type of parameters to be learned is unknown. Specifically, the value of the scalar α can be manually tuned according to the time-invariance degree of unknown parameters. That is, the update algorithm (12.28) becomes a strictly differential adaptation if α = 0, a strictly difference adaptation if α = 1, and an integration of both if α ∈ (0, 1). Remark 12.4 To clearly illustrate our basic idea, we only consider a first-order system in this paper. Actually, the results can be easily extended to the following highorder systems: x˙ j = x j+1 , x˙n = θ

◦T

j = 1, . . . , n − 1 ◦

(t)ξ (x, t) + bu(t)

where x = [x1 , . . . , xn ]T ∈ Rn denotes the state vector and x j is the jth dimension of x. To deal with such class of systems, we define the extended tracking error σ (t) =

n  j=1

c j e j (t), cn = 1

(12.54)

12.3 Differential-Difference Hybrid Scheme

245

where c j , j = 1, . . . , n, are the coefficients of a Hurwitz polynomial and e j (t) is the ( j−1) . tracking error for the jth dimension, e j (t) = x j − xr The time derivative of σ (t) can be calculated as follows: σ˙ (t) =

n 

c j e˙ j (t)

j=1

=

n−1 

c j e j+1 (t) + e˙n (t)

j=1

=

n−1 

c j e j+1 (t) + x˙n (t) − x˙r(n−1)

j=1

=

n−1 

c j e j+1 (t) + θ ◦T ξ ◦ (x, t) + bu(t) − xr(n)

j=1

⎡

= b ⎣u + b−1

n−1 

⎤ c j e j+1 (t) + b−1 θ ◦T ξ ◦ (x, t) − b−1 xr(n) ⎦ .

(12.55)

j=1

It is clear that a similar formulation as (12.4) and (12.26) is obtained. Therefore, the control laws and the parameter updating laws proposed in Sects. 12.2 and 12.3 can be applied directly and the convergence of x(t) to the desired reference x r is ensured by the convergence of σ (t).

12.4 Illustrative Simulations In order to demonstrate the effectiveness of our results, we consider the following one-link robotic manipulator system: 

x˙1 x˙2





01 = 00



     0 x1 u − gl cos x1 + η1 + 1 x2 ml 2 +I

where x1 is the joint angle, x2 is the angular velocity, m is the mass, g is the gravitational acceleration, l is the length, I is the moment of inertia, u is the joint input and η1 = 5x12 sin3 (5t) is a disturbance. As a result, the input gain is b = 1/(ml 2 + I ). The extended tracking error is defined as σ = 3e1 + e2 , whose dynamics at the kth iteration is σ˙ k = 3e˙1,k + e˙2,k = 3e2,k + b(u k − gl cos x1,k + η1,k ) − x˙r,k = b(u k + η1,k + 3b−1 e2,k − b−1 x˙r,k − gl cos x1,k ).

246

12 CEF Techniques for Uncertain Systems with Partial Structure Information

In our simulation, the target trajectories for x1 and x2 are as follows: xr,1 = 0.05e−t (2π t 3 − t 4 )(2π − t), xr,2 = 0.05e−t [12π 2 t 2 − (16π + 4π 2 )t 3 + (4π + 5)t 4 − t 5 ]. The initial states are x1,k (0) = 0 and x2,k (0) = 0. The initial input is set to be 0, i.e., u 0 (t) ≡ 0, ∀t ∈ [0, T ]. The desired operation length is T = 2π s. However, the actual operation length Tk varies randomly for each iteration. Case 1: time-invariant- and time-varying mixing-type scheme. In this case, we let m = 3 kg, l = 1 m and I = 0.5 kg·m2 . It can be seen that b is time-invariant. Therefore, θ 1 (t) = [5 sin3 (5t)] and θ 2 = [3b−1 , −b−1 , −gl]T are unknown time2 ] and ξ 2,k = varying- and time-invariant parameter vectors, respectively. ξ 1,k = [x1,k [e2,k , x˙r,k , cos x1,k ]T are associated uncertainties. Applying the proposed algorithms (12.5)–(12.7), where η1 = η2 = 40, μ = 4 and b = 0.25, the convergence of σ (t) is shown in Fig. 12.1. The vertical is the maximum of σ (t) defined as max0≤t≤Tk |σk (t)|. It is clear that σ (t) reduces to a small value after only a few learning iterations and continues to decline as the iteration number increases. Figure 12.2 shows the extended tracking error profiles at the 1st, 3rd, 5th, 10th, and 50th iterations, where the nonuniform trial length problem is clearly observed. It should be noticed that the tracking performance becomes acceptable after few iterations.

0.7

0.6

max |σ(t)|

0.5

0.4

0.3

0.2

0.1

0 5

10

15

20

25 30 Iteration axis

35

Fig. 12.1 Convergence of the extended tracking error σk in Case 1

40

45

50

12.4 Illustrative Simulations

247

0.2

0.1

0

σ(t)

−0.1

−0.2

−0.3 1st iteration 3rd iteration 5th iteration 10th iteration 50th iteration

−0.4

−0.5

−0.6 0

1

2

3

4

5

6

Time axis

Fig. 12.2 The error profiles in Case 1

The estimation of the parameters θ 1 (t) and θ 2 are demonstrated in Fig. 12.3 for the last iteration. Clearly, the estimation of the time-invariant parameters almost coincide with the true values, while the estimation of the time-varying part has some deviation from the true curve. We should remark that in the adaptive control, it is unnecessary to require the convergence of the parameter estimation to the true values. This is also why our schemes are effective without precise estimation of the involved parameters. Moreover, even if the learning process of parameters involves random noises, the tracking performance is still retained; however, the estimation of the time-varying parameters is contaminated. The detailed plots are omitted due to similarity. The tracking performance of the first and second states are given in Fig. 12.4. It can be seen that the profiles end at different time instants for different iterations, which display the randomness of nonuniform trial lengths. Moreover, the tracking performance for the first few iterations has already been acceptable. Case 2: differential-difference hybrid-type scheme. In this case, we assume that the type of estimated parameters are unknown. The system parameters and initial condition are the same as Case 1. Applying the control law given by (12.27)–(12.28) and choosing η = 1.5, μ = 3 and α = 0.5, we can get the convergence of σ (t) shown in Fig. 12.5. One apparent conclusion is verified by this figure that the hybrid adaptive law is effective for both time-varying- and time-invariant parameters.

248

12 CEF Techniques for Uncertain Systems with Partial Structure Information 15

θ(1) ˆ θ(1) θ(2) ˆ θ(2) θ(3) ˆ θ(3) θ(4) ˆ θ(4)

Estimation of parameters

10

5

0

−5

−10 0

1

2

3 Time axis

4

5

6

Fig. 12.3 Estimation of parameters in Case 1

Similar to Fig. 12.2, we plot the extended tracking error profiles in Fig. 12.6. Figure 12.7 shows the tracking situation of the system outputs. The fact that the actual operation length varies randomly with the iteration number k can be observed. From Case 1, we find that there are three time-varying parameters and one timeinvariant parameter in the robotic manipulator system. Let the disturbance η1 = 5x12 . Then, all the unknown parameters are constant. We apply the hybrid differentialdifference adaptive law, and choose α = 0. η and μ remain unchanged. Figure 12.8 depicts the learning result. It is clear that the convergence speed is much faster than that in Fig. 12.5. The system outputs are shown in Fig. 12.9, and it can be seen that a better tracking performance is achieved. Remark 12.5 Note that the values of controller parameters are artificially selected; that is, η, μ and α should be tuned to guarantee a certain convergence speed and tracking performance. In the tuning process, three facts are observed. First, a larger η will result in faster convergence speed. Notice, however, that excessive learning gains will bring about unnecessary oscillations and even an uptrend in error profiles as the iteration number increases, especially for the differential learning mechanism. Second, appropriate feedback gain μ will help reduce the tracking error and improve convergence speed, which also explains why there is already a good tracking performance for the first iteration. Third, in Case 2, the closer the parameter α approximates to 0, the fewer the iterations are needed to drive the tracking error to enter a

12.4 Illustrative Simulations

249

1.2

x

0.8 Tracking performance

r

1st iteration 3rd iteration 5th iteration 10th iteration 50th iteration

1

0.6

0.4

0.2

0

−0.2

0

1

2

3

4

5

6

Time t

(a) First dimension 1.2 x

r

1

1st iteration 3rd iteration 5th iteration 10th iteration 50th iteration

0.8

Tracking performance

0.6 0.4 0.2 0 −0.2 −0.4 −0.6

0

1

2

3

4

Time axis

(b) Second dimension

Fig. 12.4 The tracking performance of the system outputs in Case 1

5

6

250

12 CEF Techniques for Uncertain Systems with Partial Structure Information 0.7

0.6

max |σ(t)|

0.5

0.4

0.3

0.2

0.1

0 5

10

15

20

25

30

35

40

45

50

Iteration axis

Fig. 12.5 Convergence of the extended tracking error σk in Case 2 0.1

0

−0.1

σ(t)

−0.2

−0.3

−0.4 1st iteration 3rd iteration 5th iteration 10th iteration 50th iteration

−0.5

−0.6

−0.7 0

1

Fig. 12.6 The error profiles in Case 2

2

3 Time axis

4

5

6

12.4 Illustrative Simulations

251

1.2 x

r

1st iteration 3rd iteration 5th iteration 10th iteration 50th iteration

1

Tracking performance

0.8

0.6

0.4

0.2

0

−0.2 0

1

2

3 Time axis

4

5

6

(a) First dimension 1 x

r

1st iteration 3rd iteration 5th iteration 10th iteration 50th iteration

0.8

Tracking performance

0.6 0.4 0.2 0 −0.2 −0.4 −0.6 −0.8 0

1

2

3 Time axis

4

(b) Second dimension

Fig. 12.7 The tracking performance of the system outputs in Case 2

5

6

252

12 CEF Techniques for Uncertain Systems with Partial Structure Information 0.8

0.7

0.6

max |σ(t)|

0.5

0.4

0.3

0.2

0.1

0 5

10

15

20

25

30

35

40

45

50

Iteration axis

Fig. 12.8 Convergence of the extended tracking error σk : θ is constant (α = 0)

pre-defined zone of zero. This is because the parameters in Case 2 are time-invariant. The above observations actually provide some guidelines for the selection of parameters in practical applications.

12.5 Summary In this chapter, we consider a continuous-time parametric nonlinear system in which the operation lengths vary randomly. Based on the partial structure information that can be obtained, two ILC learning schemes are proposed. Specifically, when the estimated parameters are known to include both time-varying- and time-invariant types, the mixing-type learning mechanism is applied. When all the unknown parameters are constant, a hybrid-type adaptive law is employed, which is also effective even if certain time-varying uncertainties are involved. To demonstrate the convergence, we define two compensation schemes for the missing tracking error and introduce a novel CEF for the nonuniform trial length problem. Through theoretical analysis and simulation examples, the effectiveness of the adaptive ILC law for parametric nonlinear systems with iteration-varying trial lengths is shown. The results in this chapter are mainly based on [3].

12.5 Summary

253

1.2 x 1

0.8 Tracking performance

r

1st iteration 3rd iteration 5th iteration 10th iteration 50th iteration

0.6

0.4

0.2

0

−0.2 0

1

2

3 Time axis

4

5

6

(a) First dimension 1 x

0.6 Tracking performance

r

1st iteration 3rd iteration 5th iteration 10th iteration 50th iteration

0.8

0.4

0.2

0

−0.2

−0.4

−0.6 0

1

2

3 Time axis

4

5

(b) Second dimension

Fig. 12.9 The tracking performance of the system outputs: θ is constant (α = 0)

6

254

12 CEF Techniques for Uncertain Systems with Partial Structure Information

References 1. Sun M, Wang D (2002) Iterative learning control with initial rectifying action. Automatica 38(7):1177–1182 2. Xu JX, Yan R (2005) On initial conditions in iterative learning control. IEEE Trans Autom Control 50(9):1349–1354 3. Zeng C, Shen D, Wang J (2018) Adaptive learning tracking for uncertain systems with partial structure information and varying trial lengths. J Frankl Inst 355(15):7027–7055

Index

A Affine nonlinear, 119 Almost sure convergence, 75 Arimoto-like gain, 74 Arimoto-type update law, 6

First-order ILC, 4

B Barbalat lemma, 172 Bernoulli distribution, 20

H High-order ILC, 4 High-order systems, 179, 244 Hybrid learning law, 238

C Causal gain, 74 Composite energy function, 9, 167 Continuous-time systems, 6, 10 Control direction, 164 Convergence condition, 5, 56 Convergence speed, 56, 235, 248 Coupling matrix, 4 D Data-driven control, 3 Desired trial length, 18 Difference adaptive learning, 228 Differential adaptive learning, 229 Discrete-time systems, 3, 10 D-type update law, 4 E Extended relative degree, 136 Extended tracking error, 180, 244 F Feedback-type update law, 166

G Globally Lipschitz continuous, 9, 60, 120

I Identical initialization condition, 3, 19, 50, 104, 120 Indicator function, 123 Initial state error, 50 Inner control loop, 3 Input distribution matrix, 181, 210 Iteration-average operator, 21 Iterative learning control, 1 L Learning, 1 Learning effectiveness, 51 Learning gain matrix, 4, 19, 70 Lifting technique, 5, 34 L 2 -norm, 202 M Markov inequality, 77 Mean square convergence, 77 Mean value theorem, 112 Monotonic convergence, 128 Moving average operator, 106, 145

© Springer Nature Singapore Pte Ltd. 2019 D. Shen and X. Li, Iterative Learning Control for Systems with Iteration-Varying Trial Lengths, https://doi.org/10.1007/978-981-13-6136-4

255

256 N Nonparametric uncertainty, 182, 193 Norm-bounded uncertainty, 195

O Outer control loop, 3

P PD-type update law, 139 PID update law, 7 Positive-definite, 121 Probability density function, 138 Probability distribution, 22, 39 Probability distribution function, 107, 138 Projection-based update law, 172, 202 P-type update law, 4

R Random matrix, 71 Realizable, 6, 18, 68, 120 Relative degree, 6, 128 Robotic manipulator, 186 Robust compensation, 195

Index S Sampled-data control, 135 Sign function, 172, 196 Singular value, 121 Stochastic variable, 104, 138 Strong convergence, 69 Structure information, 225 Switching system, 67

T 2D approach, 60 2D system, 4

U Uniform convergence, 202 Update law, 4

V Variation-norm-bounded uncertainty, 195 Varying trial lengths, 7

Z Zero-order holder, 136