Stability of the Turnpike Phenomenon in Discrete-Time Optimal Control Problems 3319080334, 978-3-319-08033-8, 978-3-319-08034-5, 3319080342

Table of contents :
Front Matter....Pages i-x
Introduction....Pages 1-7
Optimal Control Problems with Singleton Turnpikes....Pages 9-45
Optimal Control Problems with Discounting....Pages 47-63
Optimal Control Problems with Nonsingleton Turnpikes....Pages 65-103
Back Matter....Pages 105-109

Citation preview

SpringerBriefs in Optimization

Editors Panos M. Pardalos Industrial & Systems Engineering University of Florida Gainesville, Florida USA János D. Pintér Pinter Consulting Services, Inc. Halifax, Nova Scotia Canada Stephen Robinson Industrial and Systems Engineering University of Wisconsin Madison, Wisconsin USA Tamás Terlaky Industrial & Systems Engineering Lehigh University Bethlehem, Pennsylvania USA My T. Thai Computer and Information Science and Engineering University of Florida Gainesville, Florida USA

SpringerBriefs in Optimization showcases algorithmic and theoretical techniques, case studies, and applications within the broad-based field of optimization. Manuscripts related to the ever-growing applications of optimization in applied mathematics, engineering, medicine, economics, and other applied sciences are encouraged.

More information about this series at http://www.springer.com/series/8918

Alexander J. Zaslavski

Stability of the Turnpike Phenomenon in Discrete-Time Optimal Control Problems

2123

Alexander J. Zaslavski Department of Mathematics Technion- Israel Institute of Techn Haifa Israel

ISSN 2190-8354 ISSN 2191-575X (electronic) ISBN 978-3-319-08033-8 ISBN 978-3-319-08034-5 (eBook) DOI 10.1007/978-3-319-08034-5 Springer Cham Heidelberg New York Dordrecht London Library of Congress Control Number: 2014943649 © The Author 2014 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

Preface

The monograph is devoted to the study of the structure of approximate solutions of nonconvex (nonconcave) discrete-time optimal control problems. It contains new results on properties of approximate solutions which are independent of the length of the interval, for all sufficiently large intervals. These results deal with the socalled turnpike property of optimal control problems. The term was first coined by P. Samuelson in 1948 when he showed that an efficient expanding economy would spend most of the time in the vicinity of a balanced equilibrium path (also called a von Neumann path). To have the turnpike property means, roughly speaking, that the approximate solutions of the problems are determined mainly by the objective function and are essentially independent of the choice of interval and endpoint conditions, except in regions close to the endpoints. Now it is well-known that the turnpike property is a general phenomenon which holds for large classes of variational and optimal control problems. Using the Baire category (generic) approach, it was shown that the turnpike property holds for a generic (typical) variational problem [45] and for a generic optimal control problem [56]. According to the generic approach we say that a property holds for a generic (typical) element of a complete metric space (or the property holds generically) if the set of all elements of the metric space possessing this property contains a G? everywhere dense subset of the metric space which is a countable intersection of open everywhere dense sets. In [55] we were interested in individual (non-generic) turnpike results and in sufficient and necessary conditions for the turnpike phenomenon in the calculus of variations. In our recent research [46-51, 54] we were are also interested in individual turnpike results but for discrete-time optimal control problems which, in particular, describe a general model of economic dynamics. For these problems we established the turnpike property for approximate solutions with a singleton-turnpike and studied the stability of the turnpike phenomenon under small perturbations of objective functions. In this book we continue to study the discrete-time optimal control problems considered in [46-51, 54]. Some results of these works are discussed in Chap. 1. In Chaps. 2 and 3 we show the stability of the turnpike phenomenon under small perturbations of objective functions and under small perturbations of control maps. The optimal control problems without discounting are studied in Chap. 2 while the discount case is considered in Chap. 3. In Chap. 4 we establish the turnpike property v

vi

Preface

and its stability for discrete-time problems with nonsingleton-turnpikes. Note that the stability of the turnpike property is crucial in practice. One reason is that in practice we deal with a problem which consists of a perturbation of the problem we wish to consider. Another reason is that the computations introduce numerical errors. Rishon LeZion December 30, 2013

Alexander J. Zaslavski

Contents

1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 The Turnpike Phenomenon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Discrete-Time Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 1 3 6

2

Optimal Control Problems with Singleton Turnpikes . . . . . . . . . . . . . . . 2.1 Preliminaries and Stability Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Three Lemmata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Auxiliary Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Proofs of Theorems 2.2 and 2.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 Proofs of Theorems 2.4 and 2.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Proof of Theorem 2.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8 Proof of Theorems 2.7 and 2.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9 9 14 16 20 34 37 40 43

3

Optimal Control Problems with Discounting . . . . . . . . . . . . . . . . . . . . . . 3.1 Stability of the Turnpike Phenomenon . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Auxiliary Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Proofs of Theorems 3.2 and 3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Proof of Theorem 3.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

47 47 50 57 61

4

Optimal Control Problems with Nonsingleton Turnpikes . . . . . . . . . . . . 4.1 Discrete-Time Optimal Control Systems . . . . . . . . . . . . . . . . . . . . . . . 4.2 The Turnpike Property . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Auxiliary Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Proof of Theorem 4.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

65 65 67 70 76 96

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

vii

List of Symbols

A, 65 a(x), 65 A, 67 ¯ 67 A, B(M), 65–69, 79 B(x, r), 9 Card, 6, 13, 14, 39, 40, 44 c(f ), 66 cl (E), 66 C(M), 65 c, ¯ 3, 10, 17 d1 , 66, 67 dist, 66, 68, 70 ¯ 66 E, E(λ), 12, 13 H(f ), 68 (K, d), 65 M, 9, 12, 65 M0 , 47 Mreg , 67, 68 r¯ , 5, 11 U f (q, y, z), 70 2 −1 U ({fi }Ti=T , y, z), 66 1 Vf , 67 w, 6 X, 3 XM , 4

ix

x

List of Symbols

x, ¯ 4 2 −1 Y ({t }Tt=T , T1 , T2 ), 12 1 T2 −1 ¯ Y ({t }t=T , T1 , T2 ), 12 1

f

zj , 67 Z, 65 Zp , 65 q Zp , 65 γ (f ), 67, 69 ¯ 11 λ, μ(f ), 67 ρ, 1 , 3 ρ1 , 9 σ (w, T , x, y), 10 2 −1 2 −1 σ ({ut }Tt=T , {t }Tt=T , T1 , T2 ), 11 1 1

2 −1 2 −1 , {t }Tt=T , T1 , T2 , x), 11 σ ({ut }Tt=T 1 1

2 −1 2 −1 , {t }Tt=T , T1 , T2 , x, y), 11 σ ({ut }Tt=T 1 1 ∞ ({xi }i=0 ), 66 ω({xi }∞ i=0 ), 66

Chapter 1

Introduction

The study of the existence and the structure of solutions of optimal control problems defined on infinite intervals and on sufficiently large intervals has recently been a rapidly growing area of research. See, for example, [3, 4, 7–11, 13, 14, 16, 18–22, 25, 27, 32–34, 36, 44, 45, 52, 55] and the references mentioned therein. These problems arise in engineering [1, 23, 57], in models of economic growth [2, 5, 12, 13, 17, 22, 26, 31, 35, 38, 39, 40, 45], in infinite discrete models of solid-state physics related to dislocations in one-dimensional crystals [6, 41] and in the theory of thermodynamical equilibrium for materials [15, 24, 28–30]. In this chapter we discuss the structure of solutions of a discrete-time optimal control system describing a general model of economic dynamics.

1.1 The Turnpike Phenomenon We study the structure of approximate solutions of an autonomous discrete-time control system with a compact metric space of states X equipped with a metric ρ. This control system is described by a bounded upper semicontinuous function v : X × X → R 1 which determines an optimality criterion and by a nonempty closed set Ω ⊂ X × X which determines a class of admissible trajectories (programs). In models of economic growth the set X is the space of states, v is a utility function and v (xt , xt+1 ) evaluates consumption at moment t. Consider the problem T −1 

−1 v (xi , xi+1 ) → max, {(xi , xi+1 )}Ti=0 ⊂ Ω, x0 = z, xT = y,

(1.1)

i=0

where T ≥ 1 is an integer and the points y, z ∈ X. We are interested in the turnpike property of approximate solutions which is independent of the length of the interval, for all sufficiently large intervals. To have this property means, roughly speaking, that approximate solutions of optimal control problems on an interval [0, T ] with given values y, z at the endpoints 0 and T , A. J. Zaslavski, Stability of the Turnpike Phenomenon in Discrete-Time Optimal Control Problems, SpringerBriefs in Optimization, DOI 10.1007/978-3-319-08034-5_1, © The Author 2014

1

2

1 Introduction

corresponding to the pair (v, Ω), are determined mainly by the objective function v, and are essentially independent of T , y and z. In the classical turnpike theory the objective function v possesses the turnpike property (TP) if there exists a point x¯ ∈ X (a turnpike) such that the following condition holds: For each positive number there exists an integer L ≥ 1 such that for each integer T ≥ 2 L and each solution {xi }Ti=0 ⊂ X of the problem (P) the inequality ρ(xi , x) ¯ ≤ is true for all i = L, . . . , T − L. It should be mentioned that the constant L depends neither on T nor on y, z. The turnpike phenomenon has the following interpretation. If one wishes to reach a point A from a point B by a car in an optimal way, then one should turn to a turnpike, spend most of time on it and then leave the turnpike to reach the required point. In the classical turnpike theory [17, 31, 38, 40] the space X is a compact convex subset of a finite-dimensional Euclidean space, the set Ω is convex and the function v is strictly concave. Under these assumptions the turnpike property can be established and the turnpike x¯ is a unique solution of the maximization problem v(x, x) → max, (x, x) ∈ Ω. In this situation admissible sequence T −1 it is shown that for each ∞ {xt }∞ − T v x, ¯ x)} ¯ either the sequence { v , x ) ( (x t t+1 t=0 T =1 is bounded (in this t=0 case the sequence {xt }∞ is called (v)-good) or it diverges to −∞. Moreover, it is t=0 also established that any (v)-good admissible sequence converges to the turnpike x. ¯ In the sequel this property is called as the asymptotic turnpike property. Recently it was shown that the turnpike property is a general phenomenon which holds for large classes of variational and optimal control problems without convexity assumptions. (See, for example, [45, 55, 56] and the references mentioned therein). For these classes of problems a turnpike is not necessarily a singleton but may instead be an nonstationary trajectory (in the discrete time nonautonomous case) or an absolutely continuous function on the interval [0, ∞) (in the continuous time nonautonomous case) or a compact subset of the space X (in the autonomous case). For classes of problems considered in [45, 56], using the Baire category approach, it was shown that the turnpike property holds for a generic (typical) problem. In this book we are interested in individual (non-generic) turnpike results and in stability of the turnpike phenomenon under small perturbations of the objective function v and the set Ω. As we have mentioned before in general a turnpike is not necessarily a singleton. Nevertheless problems of the type (P) for which the turnpike is a singleton are of great importance because of the following reasons: there are many models of economic growth for which a turnpike is a singleton; if a turnpike is a singleton, then approximate solutions of (P) have very simple structure and this is very important for applications; if a turnpike is a singleton, then it can be easily calculated as a solution of the problem v(x, x) → max, (x, x) ∈ Ω. The turnpike property is very important for applications. Suppose that our objective function v has the turnpike property and we know a finite number of “approximate” solutions of the problem (P). Then we know the turnpike x, ¯ or at least its approximation, and the constant L (see the definition of (TP)) which is an estimate for the time period required to reach the turnpike. This information can

1.2 Discrete-Time Problems

3

be useful if we need to find an “approximate” solution of the problem (P) with a new time interval [m1 , m2 ] and the new values z, y ∈ X at the end points m1 and m2 . Namely instead of solving this new problem on the “large” interval [m1 , m2 ] we can find an “approximate” solution of the problem (P) on the “small” interval [m1 , m1 + L] with the values z, x¯ at the end points and an “approximate” solution of ¯ y at the end the problem (P) on the “small” interval [m2 − L, m2 ] with the values x, points. Then the concatenation of the first solution, the constant sequence xi = x, ¯ i = m1 + L, . . . , m2 − L and the second solution is an “approximate” solution of the problem (P) on the interval [m1 , m2 ] with the values z, y at the end points. Sometimes as an “approximate” solution of the problem (P) we can choose any admissible 2 sequence {xi }m i=m1 satisfying xm1 = z, xm2 = y and xi = x¯ for all i = m1 + L, . . . , m2 − L.

1.2

Discrete-Time Problems

Let (X, ρ) be a compact metric space, Ω be a nonempty closed subset of X × X and let v : X × X → R 1 be a bounded upper semicontinuous function. A sequence {xt }∞ t=0 ⊂ X is called program if (xt , xt+1 ) ∈ Ω for all nonnegative integers t. A sequence {xt }Tt=0 where T ≥ 1 is an integer is called a program if (xt , xt+1 ) ∈ Ω for all integers t ∈ [0, T − 1]. We consider the problems T −1 

−1 v (xi , xi+1 ) → max, {(xi , xi+1 )}Ti=0 ⊂ Ω, x0 = y,

i=0

and

T −1 

−1 v (xi , xi+1 ) → max, {(xi , xi+1 )}Ti=0 ⊂ Ω, x0 = y, xT = z,

i=0

where T ≥ 1 is an integer and the points y, z ∈ X. We suppose that there exist a point x¯ ∈ X and a positive number c¯ such that the following assumptions hold: (i) (x, ¯ x) ¯ is an interior point of Ω and the function v is continuous at the point ( x, ¯ x); ¯ T −1 (ii) ¯ x) ¯ + c¯ for any natural number T and any program t=0 v(xt , xt+1 ) ≤ T v(x, {xt }Tt=0 . The property (ii) implies that for each program {xt }∞ t=0 either the sequence ∞ T −1  v(xt , xt+1 ) − T v(x, ¯ x) ¯ T =1

t=0

is bounded or limT →∞



T −1 t=0

 v(xt , xt+1 ) − T v(x, ¯ x) ¯ = −∞.

4

1 Introduction

A program {xt }∞ t=0 is called (v)-good if the sequence T −1 ∞  ¯ x) ¯ v (xt , xt+1 ) − T v (x, T =1

t=0

is bounded. Suppose that the following assumption holds. (iii) (the asymptotic turnpike property) For any (v)-good program {xt }∞ t=0 , ¯ = 0. limt→∞ ρ(xt , x) Note that the properties (i)–(iii) hold for models of economic dynamics considered in the classical turnpike theory. For each positive number M denote by XM the set of all points x ∈ X for which there exists a program {xt }∞ t=0 such that x0 = x and that for all natural numbers T the following inequality holds: T −1 

v(xt , xt+1 ) − T v(x, ¯ x) ¯ ≥ −M.

t=0

It is not difficult to see that ∪ {XM : M ∈ (0, ∞)} is the set of all points x ∈ X for 1 which there exists a (v)-good program {xt }∞ t=0 satisfying x0 = x. Let u : X×X → R , T be a bounded function, an integer T ≥ 1 and Δ ≥ 0. A program {xi }i=0 ⊂ X is  T called (u, Δ)-optimal if for any program xi i=0 satisfying x0 = x0 , the inequality T −1 

u (xi , xi+1 ) ≥

i=0

T −1  



−Δ u xi , xi+1 i=0

holds. The following turnpike result describes the structure of approximate solutions of our first optimization problem stated above. Theorem 1.1 Let , M be positive numbers. Then there exist a natural number L and a positive number δ such that for each integer T > 2 L and each (v, δ)-optimal program {xt }Tt=0 which satisfies x0 ∈ XM there exist nonnegative integers τ1 , τ2 ≤ L such that ρ(xt , x) ¯ ≤ for all t = τ1 , . . . , T − τ2 and if ρ(x0 , x) ¯ ≤ δ, then τ1 = 0. An analogous turnpike result also holds for approximate solutions of our second optimization problem. The following notion of an overtaking optimal program was introduced in [5, 17, 40]. ∞ A program {xt }∞ t=0 is called (v)-overtaking optimal if for each program {yt }t=0 satisfying y0 = x0 the inequality lim sup T →∞

T −1 

v (yt , yt+1 ) − v(xt , xt+1 ) ≤ 0

t=0

holds. The following result establishes the existence of an overtaking optimal program.

1.2 Discrete-Time Problems

5

Theorem 1.2 Assume that x ∈ X and that there exists a (v)-good program {xt }∞ t=0 such that x0 = x. Then there exists a (v)-overtaking optimal program {xt∗ }∞ t=0 such that x0∗ = x. The following result provides necessary and sufficient conditions for overtaking optimality. Theorem 1.3 Let {xt }∞ t=0 be a program such that x0 ∈ ∪{XM : M ∈ (0, ∞)}. Then the program {xt }∞ t=0 is (v)-overtaking optimal if and only if the following conditions hold: (1) limt→∞ ρ(xt , x) ¯ = 0; (2) for each natural number T and each program {yt }Tt=0 satisfying y0 = x0 ,  −1  −1 v(yt , yt+1 ) ≤ Tt=0 v(xt , xt+1 ) holds. yT = xT the inequality Tt=0 The next two theorems establish uniform convergence of overtaking optimal programs to x. ¯ Theorem 1.4 Assume that the function v is continuous and let be a positive number. Then there exists a positive number δ such that for each (v)-overtaking optimal program {xt }∞ ¯ ≤ δ the inequality ρ(xt , x) ¯ ≤ holds for t=0 satisfying ρ(x0 , x) all nonnegative integers t. Theorem 1.5 Assume that the function v is continuous and let M, be positive numbers. Then there exists an integer L ≥ 1 such that for each (v)-overtaking optimal program {xt }∞ ¯ ≤ holds for all t=0 satisfying x0 ∈ XM the inequality ρ(xt , x) integers t ≥ L. Theorems 1.1–1.5 were obtained in [46]. Example 1.6 Let (X, ρ) be a compact metric space, Ω be a nonempty closed subset of X × X, x¯ ∈ X, (x, ¯ x) ¯ be an interior point of Ω, π : X → R 1 be a continuous function, α be a real number and L : X × X → [0, ∞) be a continuous function such that for each (x, y) ∈ X × X the equality L(x, y) = 0 holds if and only if (x, y) = (x, ¯ x). ¯ Set v (x, y) = α − L(x, y) + π(x) − π (y) for all x, y ∈ X. It is not difficult to see that assumptions (i), (ii) and (iii) hold. Example 1.7 Let X be a compact convex subset of the Euclidean space R n with the norm | · | induced by the scalar product ·, · , let ρ(x, y) = |x − y|, x, y ∈ R n , Ω be a nonempty closed subset of X × X, a point x¯ ∈ X, (x, ¯ x) ¯ be an interior point of Ω and let v : X × X → R 1 be a strictly concave continuous function such that v(x, ¯ x) ¯ = sup{v(z, z) : z ∈ X and (z, z) ∈ Ω}. We assume that there exists a positive constant r¯ such that  (x, y) ∈ R n × R n : |x − x|, ¯ |y − x| ¯ ≤ r¯ ⊂ Ω.

6

1 Introduction

It is a well-known fact of convex analysis [37] that there exists a point l ∈ R n such that v(x, y) ≤ v(x, ¯ x) ¯ + l, x − y for any point (x, y) ∈ X × X. Set L(x, y) = v(x, ¯ x) ¯ + l, x − y − v(x, y) for all (x, y) ∈ X × X. It is not difficult to see that this example is a particular case of Example 1.6. Therefore assumptions (i), (ii) and (iii) hold. Denote by M the set of all bounded functions u : X ×X → R 1 . For each function w ∈ M we set w = sup{|w(x, y)| : x, y ∈ X}. The following two theorems obtained in [48, 49] respectively show that the turnpike property is stable under perturbations of the objective function. Theorem 1.8 Let M0 , > 0. Then there exist a positive number δ and an integer L∗ ≥ 1 such that for each function u ∈ M satisfying u − v ≤ δ, each integer T > 2L∗ and each (u, δ)-optimal program {xt }Tt=0 which satisfies x0 ∈ XM0 there exist integers τ1 ∈ [0, L∗ ], τ2 ∈ [T − L∗ , T ] such that ρ(xt , x) ¯ ≤ , t = τ1 , . . . , τ2 . Moreover if ρ(x0 , x) ¯ ≤ δ, then τ1 = 0. Denote by Card(A) the cardinality of a set A. Theorem 1.9 Let M0 , M1 , be positive numbers. Then there exist a positive number δ and an integer L∗ ≥ 1 such that for each function u ∈ M satisfying u − v ≤ δ, each integer T > L∗ and each (u, M1 )-optimal program {xt }Tt=0 which satisfies x0 ∈ XM0 the following inequality holds: Card({t ∈ {0, . . . , T } : ρ(xt , x) ¯ > }) ≤ L∗ .

1.3

Examples

 Example 1.10 Let X = [0, 1], Ω = (x, y) ∈ [0, 1] × [0, 1] : y ≤ x 1/2 , v(x, y) = x 1/2 − y 2 , x, y ∈ X. It is not difficult to see that the set Ω is convex, the function v is strictly concave, the optimization

problem v(z, z) → max, z ∈ X and (z, z) ∈ Ω has a unique solution 16−1/3 and 16−1/3 , 16−1/3 is an interior point of Ω. Therefore this example is a particular case of Example 1.7 and assumptions (i), (ii) and (iii) hold. Example 1.11 Let X = [0, 1], Ω = {(x, y) ∈ [0, 1] × [0, 1] : y ≤ x 1/2 }, v(x, y) = x 1/2 − y, x, y ∈ X. It is not difficult to see that the set Ω is convex, the function v is concave but not strictly concave, the optimization problem v(z, z) → max, z ∈ X

1.3 Examples

7

and (z, z) ∈ Ω has a unique solution 4−1 and (4−1 , 4−1 ) is an interior point of Ω. Since the function v is concave for all x, y ∈ X, v (x, y) ≤ v(4−1 , 4−1 ) + x − y = 4−1 + x − y and

2

4−1 + x − y − v(x, y) = x 1/2 − 2−1

is equal zero if and only if x = 4−1 . Now it is not difficult to see that assumptions (i), (ii) and (iii) hold. Example 1.12 Consider the sets X, Ω and the function v defined in Example 1.11 and set u(x, y) = x 1/2 − x 2 − y + y 2 , x, y ∈ X. The function u is strictly convex with respect to the variable y. Nevertheless assumptions (i), (ii) and (iii) hold for the function u because for any integer T and any program {xt }Tt=0 , T −1  t=0

u (xt , xt+1 ) =

T −1  t=0

v (xt , xt+1 ) + xT2 − x02 .

Chapter 2

Optimal Control Problems with Singleton Turnpikes

In this chapter we study the structure of solutions of a discrete-time control system with a compact metric space of states X which arises in economic dynamics. This control system is described by a nonempty closed set Ω ⊂ X×X which determines a class of admissible trajectories (programs) and by a bounded upper semicontinuous objective function v : X × X → R 1 which determines an optimality criterion. We show the stability of the turnpike phenomenon under small perturbations of the objective function v and the set Ω.

2.1

Preliminaries and Stability Results

Let (X, ρ) be a compact metric space. For each x ∈ X and each nonempty set C ⊂ X set ρ(x, C) = inf{ρ(x, y) : y ∈ C}. For each x ∈ X and each r > 0 set B(x, r) = {y ∈ X : ρ(x, y) ≤ r}. We equip the space X × X with the metric ρ1 defined by ρ1 ((x1 , x2 ), (y1 , y2 )) = ρ(x1 , y1 ) + ρ(x2 , y2 ), x1 , x2 , y1 , y2 ∈ X. For each (x1 , x2 ) ∈ X × X and each nonempty set C ⊂ X × X set ρ1 ((x1 , x2 ), C) = inf{ρ1 ((x1 , x2 ), (y1 , y2 )) : (y1 , y2 ) ∈ C}. Denote by M the set of all bounded functions u : X × X → R 1 . For each w ∈ M set w = sup{|w(x, y)| : (x, y) ∈ X × X}. Let Ω be a nonempty closed subset of X × X.

A. J. Zaslavski, Stability of the Turnpike Phenomenon in Discrete-Time Optimal Control Problems, SpringerBriefs in Optimization, DOI 10.1007/978-3-319-08034-5_2, © The Author 2014

9

10

2 Optimal Control Problems with Singleton Turnpikes

A sequence {xt }∞ t=0 ⊂ X is called an (Ω)-program if (xt , xt+1 ) ∈ Ω for all integers 2 t ≥ 0. A sequence {xt }Tt=T ⊂ X where integers T1 , T2 satisfy 0 ≤ T1 < T2 is called 1 an (Ω)-program if (xt , xt+1 ) ∈ Ω for all integers t ∈ [T1 , T2 − 1]. Let v ∈ M be an upper semicontinuous function. We suppose that there exist x¯ ∈ X and a constant c¯ > 0 such that the following assumptions hold. (A1) (x, ¯ x) ¯ is an interior point of Ω (there is > 0 such that {(x, y) ∈ X × X : ρ(x, x), ¯ ρ(y, x) ¯ ≤ } ⊂ Ω) and v is continuous at (x, ¯ x). ¯ (A2) For any integer T ≥ 1 and any (Ω)-program {xt }Tt=0 , T −1 

v(xt , xt+1 ) ≤ T v(x, ¯ x) ¯ + c. ¯

t=0

Assumption (A2) implies the following result. Proposition 2.1 For each (Ω)-program {xt }∞ t=0 either the sequence T −1 

∞ v(xt , xt+1 ) − T v(x, ¯ x) ¯

t=0

is bounded or limT →∞



T −1 t=0



T =1

v(xt , xt+1 ) − T v(x, ¯ x) ¯ = −∞.

An (Ω)-program {xt }∞ t=0 is called (v, Ω)-good if the sequence T −1  t=0

∞ v(xt , xt+1 ) − T v(x, ¯ x) ¯ T =1

is bounded [13, 17, 45, 55, 56]. In this chapter we suppose that the following assumption holds. (A3) (the asymptotic turnpike property) For any (v, Ω)-good program {xt }∞ t=0 , limt→∞ ρ(xt , x) ¯ = 0. Note that (A3) holds for many important infinite horizon optimal control problems. In particular, (A3) holds for a general model of economic dynamics considered in Example 1.7. For each x, y ∈ X, each integer T ≥ 1 and each w ∈ M set σ (w, T , x, y) T −1   = sup w(xi , xi+1 ) : {xi }Ti=0 is an (Ω) − program and x0 = x, xT = y . i=0

(Here we use the convention that the supremum of an empty set is −∞).

2.1 Preliminaries and Stability Results

11

In Chap. 1 we considered the turnpike properties of approximate solutions of the problems T −1  −1 v(xi , xi+1 ) → max, {(xi , xi+1 )}Ti=0 ⊂ Ω, x0 = y, i=0

and

T −1 

−1 v(xi , xi+1 ) → max, {(xi , xi+1 )}Ti=0 ⊂ Ω, x0 = y, xT = z,

i=0

where T ≥ 1 is an integer and the points y, z ∈ X. In this chapter we show that these turnpike properties are stable under small perturbations of the objective function v and the set Ω. In order to meet this goal we introduce the following definitions. By assumption (A1) there exists r¯ ∈ (0, 1) such that B(x, ¯ r¯ ) × B(x, ¯ r¯ ) ⊂ Ω.

(2.1)

λ¯ ∈ (0, r¯ ).

(2.2)

Fix

For each λ > 0 denote by E(λ) the collection of all nonempty sets Ω ⊂ X × X such that ρ1 (z, Ω) ≤ λ for each z ∈ Ω ,

(2.3)

¯ × B(x, ¯ ⊂Ω. B(x, ¯ λ) ¯ λ)

(2.4)



Let integers T1 , T2 satisfy 0 ≤ T1 < T2 and let Ωt , t = T1 , . . . , T2 −1 be nonempty subsets of X × X. 2 2 −1 A sequence {xt }Tt=T ⊂ X is called an ({Ωt }Tt=T )-program if (xt , xt+1 ) ∈ Ωt for 1 1 all integers t ∈ [T1 , T2 − 1]. 2 −1 For each x, y ∈ X and each finite sequence {ut }Tt=T ⊂ M set 1 2 −1 2 −1 , {Ωt }Tt=T , T1 , T2 , x) σ ({ut }Tt=T 1 1

= sup

T −1 2 

ut (xt , xt+1 ) :

t=T1

 T2 −1 2 {xt }Tt=T is an ({Ω } ) − program and x = x , t T 1 t=T 1 1 T −1 2    2 −1 2 −1 σ {ut }Tt=T , {Ωt }Tt=T , T1 , T2 , x, y = sup ut (xt , xt+1 ) : 1 1

(2.5)

t=T1

 2 2 −1 {xt }Tt=T is an ({Ωt }Tt=T ) − program, xT1 = x and xT2 = y , 1 1

(2.6)

12

2 Optimal Control Problems with Singleton Turnpikes 2 −1 2 −1 σ ({ut }Tt=T , {Ωt }Tt=T , T1 , T2 ) 1 1

= sup

T −1 2 

ut (xt , xt+1 ) :

t=T1

 T2 −1 2 is an ({Ω } ) − program . {xt }Tt=T t t=T1 1

(2.7)

(Here we use the convention that the supremum of an empty set is −∞). 2 −1 Denote by Y ({Ωt }Tt=T , T1 , T2 ) the set of all x ∈ X for which there exists an 1 T2 −1 T2 ({Ωt }t=T1 )-program {xt }t=T1 such that xT1 = x¯ and xT2 = x. 2 −1 , T1 , T2 ) the set of all x ∈ X for which there exists an Denote by Y¯ ({Ωt }Tt=T 1 T2 −1 T2 ({Ωt }t=T1 )-program {xt }t=T1 such that xT1 = x and xT2 = x. ¯ For sufficiently small positive numbers δ, we study the structure of approximate solutions of the problems T −1 

ui (xi , xi+1 ) → max,

i=0 −1 {xi }Ti=0 is an ({Ωt }Tt=0 ) − program and x0 = y,

and

T −1 

ui (xi , xi+1 ) → max,

i=0 −1 {xi }Ti=0 is an ({Ωt }Tt=0 ) − program and x0 = y, xT = z,

where T ≥ 1 is an integer, y, z ∈ X and for all t = 0, . . . , T − 1, we have Ωt ∈ E(δ), ut ∈ M and ut − v ≤ δ. In this chapter we prove the following four stability results. Theorem 2.2 Let be a positive number and let l1 , l2 be natural numbers. Then there exist δ > 0 and a natural number L > l1 + l2 such that for each integer T > 2 L, each Ωt ∈ E(δ), t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ δ, t = 0, . . . , T − 1 −1 )-program {xt }Tt=0 which satisfies and each ({Ωt }Tt=0 −1 1 −1 x0 ∈ Y¯ ({Ωt }lt=0 , 0, l1 ), xT ∈ Y ({Ωt }Tt=T −l2 , T − l2 , T ),

−1 −1 σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0 , xT ) ≤

T −1  t=0

ut (xt , xt+1 ) + δ

2.1 Preliminaries and Stability Results

13

there exist integers τ1 ∈ [0, L], τ2 ∈ [T − L, T ] such that ρ(xt , x) ¯ ≤ for all t = τ1 , . . . , τ2 . Moreover if ρ(x0 , x) ¯ ≤ δ, then τ1 = 0 and if ρ(xT , x) ¯ ≤ δ, then τ2 = T . Theorem 2.3 Let be a positive number and let l1 be a natural number. Then there exist δ > 0 and a natural number L > l1 such that for each integer T > 2 L, each Ωt ∈ E(δ), t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ δ, t = 0, . . . , T − 1 −1 and each ({Ωt }Tt=0 )-program {xt }Tt=0 which satisfies 1 −1 , 0, l1 ), x0 ∈ Y¯ ({Ωt }lt=0

−1 −1 σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0 )

≤

T −1 

ut (xt , xt+1 ) + δ

t=0

there exist integers τ1 ∈ [0, L], τ2 ∈ [T − L, T ] such that ρ(xt , x) ¯ ≤ for all t = τ1 , . . . , τ2 . Moreover if ρ(x0 , x) ¯ ≤ δ, then τ1 = 0 and if ρ(xT , x) ¯ ≤ δ, then τ2 = T . Denote by Card(B) the cardinality of a set B. Theorem 2.4 Let , M be positive numbers and let l1 , l2 be natural numbers. Then there exist δ > 0 and a natural number L > l1 + l2 such that for each integer T > L, each Ωt ∈ E(δ), t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ δ, t = 0, . . . , T − 1 −1 and each ({Ωt }Tt=0 )-program {xt }Tt=0 which satisfies −1 1 −1 , 0, l1 ), xT ∈ Y ({Ωt }Tt=T x0 ∈ Y¯ ({Ωt }lt=0 −l2 , T − l2 , T ), −1 −1 , {Ωt }Tt=0 , 0, T , x0 , xT ) ≤ σ ({ut }Tt=0

T −1 

ut (xt , xt+1 ) + M

t=0

the inequality Card({t ∈ {0, . . . , T } : ρ(xt , x) ¯ > }) ≤ L holds.

14

2 Optimal Control Problems with Singleton Turnpikes

Theorem 2.5 Let , M be positive numbers and let l1 be a natural number. Then there exist δ > 0 and a natural number L > l1 such that for each integer T > L, each Ωt ∈ E(δ), t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ δ, t = 0, . . . , T − 1 −1 and each ({Ωt }Tt=0 )-program {xt }Tt=0 which satisfies 1 −1 x0 ∈ Y¯ ({Ωt }lt=0 , 0, l1 ),

−1 −1 σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0 )

≤

T −1 

ut (xt , xt+1 ) + M

t=0

the inequality ¯ > }) ≤ L Card({t ∈ {0, . . . , T } : ρ(xt , x) holds.

2.2

Extensions

We use the notation, definitions, and assumptions introduced in Sect. 2.1. In this section we state the extensions of the turnpike results of the previous section. In these extensions we describe the structure of programs defined on an interval [0, T ] with sufficiently large T which are approximate solutions of the corresponding optimal problems on subintervals of the length L, where L is a constant which does not depend on T . ¯ and M be a positive number. Then there exist γ ∈ (0, ) Theorem 2.6 Let ∈ (0, λ) and a natural number L0 such that for each integer L1 ≥ L0 there exists a positive number δ < γ such that the following assertion holds. Assume that an integer T > 3L1 , Ωt ∈ E(δ), t = 0, . . . , T − 1, ut ∈ M, t = 0, . . . , T − 1 satisfy ut − v ≤ δ, t = 0, . . . , T − 1 −1 and that an ({Ωt }Tt=0 )-program {xt }Tt=0 and a finite sequence of integers {Si }i=0 satisfy q

S0 = 0, Si+1 − Si ∈ [L0 , L1 ], i = 0, . . . , q − 1, Sq > T − L1 ,

2.2 Extensions

15 Si+1 −1



Si+1 −1

ut (xt , xt+1 ) ≥

t=Si



ut (x, ¯ x) ¯ −M

t=Si

for each integer i ∈ [0, q − 1], Si+2 −1



S

−1

S

−1

i+2 i+2 ut (xt , xt+1 ) ≥ σ ({ut }t=S , {Ωt }t=S , Si , Si+2 , xSi , xSi+2 ) − γ i i

t=Si

for each integer i ∈ [0, q − 2] and T −1 

−1 −1 ut (xt , xt+1 ) ≥ σ ({ut }Tt=S , {Ωt }Tt=S , Sq−2 , T , xSq−2 , xT ) − γ . q−2 q−2

t=Sq−2

Then there exist integers τ1 ∈ [0, L1 ], τ2 ∈ [T − 2L1 , T ] such that ρ(xt , x) ¯ ≤ for all t = τ1 , . . . , τ2 . Moreover if ρ(x0 , x) ¯ ≤ γ , then τ1 = 0 and if ρ(xT , x) ¯ ≤ γ , then τ2 = T . Theorem 2.7 Let , M be positive numbers and let l1 , l2 be natural numbers. Then there exist δ > 0 and a natural number L > l1 + l2 such that for each integer T > 2 L, each Ωt ∈ E(δ), t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ δ, t = 0, . . . , T − 1 −1 and each ({Ωt }Tt=0 )-program {xt }Tt=0 which satisfies −1 1 −1 x0 ∈ Y¯ ({Ωt }lt=0 , 0, l1 ), xT ∈ Y ({Ωt }Tt=T −l2 , T − l2 , T ), −1 −1 , {Ωt }Tt=0 , 0, T , x0 , xT ) ≤ σ ({ut }Tt=0

T −1 

ut (xt , xt+1 ) + M

t=0

and τ +L−1 

+L−1 +L−1 ut (xt , xt+1 ) ≥ σ ({ut }τt=τ , {Ωt }τt=τ , τ , τ + L, xτ , xτ +L ) − δ

t=τ

for each integer τ ∈ [0, T − L], there exist integers τ1 ∈ [0, L], τ2 ∈ [T − L, T ] such that ρ(xt , x) ¯ ≤ for all t = τ1 , . . . , τ2 . Moreover if ρ(x0 , x) ¯ ≤ δ, then τ1 = 0 and if ρ(xT , x) ¯ ≤ δ, then τ2 = T .

16

2 Optimal Control Problems with Singleton Turnpikes

Theorem 2.8 Let , M be positive numbers and let l1 be a natural number. Then there exist δ > 0 and a natural number L > l1 such that for each integer T > 2 L, each Ωt ∈ E(δ), t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ δ, t = 0, . . . , T − 1 −1 and each ({Ωt }Tt=0 )-program {xt }Tt=0 which satisfies 1 −1 , 0, l1 ), x0 ∈ Y¯ ({Ωt }lt=0

−1 −1 σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0 ) ≤

T −1 

ut (xt , xt+1 ) + M

t=0

and τ +L−1 

+L−1 +L−1 ut (xt , xt+1 ) ≥ σ ({ut }τt=τ , {Ωt }τt=τ , τ , τ + L, xτ , xτ +L ) − δ

t=τ

for each integer τ ∈ [0, T − L], there exist integers τ1 ∈ [0, L], τ2 ∈ [T − L, T ] such that ρ(xt , x) ¯ ≤ for all t = τ1 , . . . , τ2 . ¯ ≤ δ, then τ1 = 0 and if ρ(xT , x) ¯ ≤ δ, then τ2 = T . Moreover if ρ(x0 , x)

2.3 Three Lemmata In order to prove our stability results we need the following useful lemmas. Lemmas 2.9 and 2.10 were obtained in [46] while Lemma 2.11 was proved in [47]. Lemma 2.9 Let > 0 and M0 > 0. Then there exists a natural number T such that for each (Ω)-program {xt }Tt=0 which satisfies T −1 

v(xt , xt+1 ) ≥ T v(x, ¯ x) ¯ − M0

t=0

the relation ¯ : i = 1, . . . , T } ≤ min{ρ(xi , x) holds. Proof Let us assume the contrary. Then for each natural number k there exists an (Ω)-program {xt(k) }kt=0 which satisfies k−1  t=0

(k) v(xt(k) , xt+1 ) ≥ kv(x, ¯ x) ¯ − M0 ,

(2.8)

2.3 Three Lemmata

17

ρ(xt(k) , x) ¯ > for all integers t = 1, . . . , k.

(2.9)

Let k ≥ 1 be an integer. By (2.8) and (A2) for each integer j satisfying 0 < j < k j −1 

(k) v(xt(k) , xt+1 )=

t=0

k−1 

(k) v(xt(k) , xt+1 )−

t=0

k−1 

(k) v(xt(k) , xt+1 )

t=j

≥ kv(x, ¯ x) ¯ − M0 −

k−1 

(k) v(xt(k) , xt+1 )

t=j

≥ kv(x, ¯ x) ¯ − M0 − (k − j )v(x, ¯ x) ¯ − c. ¯ Together with (2.8) this inequality implies that for each integer k ≥ 1 and each j ∈ {1, . . . , k} j −1 

(k) v(xt(k) , xt+1 ) ≥ j v(x, ¯ x) ¯ − c¯ − M0 .

(2.10)

t=0

There exists a strictly increasing sequence of natural numbers {ki }∞ i=1 such that for each integer t ≥ 0 there exists xt = lim xt(ki ) . i→∞

(2.11)

Clearly, {xt }∞ t=0 is an (Ω)-program. In view of (2.11) and (2.9) ¯ ≥ for all integers t ≥ 1. ρ(xt , x)

(2.12)

It follows from (2.11) and (2.10) that for each integer T ≥ 1 we have T −1 

v(xt , xt+1 ) ≥ T v(x, ¯ x) ¯ − M0 − c. ¯

t=0

¯ = 0. This implies that {xt }∞ t=0 is a (v, Ω)-good program. By (A3) lim t→∞ ρ(xt , x) This equality contradicts (2.12). The contradiction we have reached proves Lemma 2.9. Lemma 2.10 Let > 0. Then there exists δ > 0 such that for each integer T ≥ 1 and each (Ω)-program {xt }Tt=0 which satisfies ¯ ρ(xT , x) ¯ ≤ δ, ρ(x0 , x), T −1 

v(xt , xt+1 ) ≥ σ (v, T , x0 , xT ) − δ

t=0

¯ ≤ holds for all t = 0, . . . , T . the inequality ρ(xt , x)

(2.13) (2.14)

18

2 Optimal Control Problems with Singleton Turnpikes

Proof Since v is continuous at (x, ¯ x) ¯ for each natural number k there exists δk ∈ (0, 2−k r¯ )

(2.15)

|v(x, y) − v(x, ¯ x)| ¯ ≤ 2−k

(2.16)

ρ(x, x), ¯ ρ(y, x) ¯ ≤ δk .

(2.17)

such that

for each x, y ∈ X satisfying

Assume that the lemma is wrong. Then for each natural number k there exist an k integer Tk ≥ 1 and an (Ω)-program {xt(k) }Tt=0 such that     ρ x0(k) , x¯ , ρ xT(k) , x¯ ≤ δk , (2.18) k T k −1 t=0

    (k) ≥ σ v, Tk , x0(k) , xT(k) − δk , v xt(k) , xt+1 k

(2.19)

  max{ρ xt(k) , x¯ : t = 0, . . . , Tk } > .

(2.20)

k ⊂ X as follows: Let k ≥ 1 be an integer. Define a sequence {zt }Tt=0

, zt = x, ¯ t ∈ {0, . . . , Tk } \ {0, Tk }. z0 = x0(k) , zTk = xT(k) k

(2.21)

k is an (Ω)-program. It follows from (2.19) By (2.21), (2.18), (2.15), and (2.1), {zt }Tt=0 and (2.21) that

T k −1

T k −1     (k) ≥ σ v, Tk , x0(k) , xT(k) − δ v xt(k) , xt+1 ≥ v(zt , zt+1 ) − δk . k k

t=0

(2.22)

t=0

In view of (2.18), (2.21), and the choice of δk (see (2.15)–(2.17)) |v(z0 , z1 ) − v(x, ¯ x)| ¯ ≤ 2−k , |v(zTk−1 , zTk ) − v(x, ¯ x)| ¯ ≤ 2−k , v(zt , zt+1 ) = v(x, ¯ x), ¯ t ∈ {0, . . . , Tk−1 } \ {0, Tk − 1}.

(2.23)

Relations (2.23) and (2.22) imply that T k −1

  (k) ≥ Tk v(x, v xt(k) , xt+1 ¯ x) ¯ − 2 · 2−k − δk .

(2.24)

t=0

Set S0 = 0, Sk =

k  i=1

(Ti + 1) − 1 for all integers k ≥ 1.

(2.25)

2.3 Three Lemmata

19

Define a sequence {xt }∞ t=0 ⊂ X as follows: xt = xt(1) , t = 0, . . . , T1 , xt = xi(k+1)

(2.26)

for each integer k ≥ 1, each i ∈ {0, . . . , Tk+1 } and t = Sk + i + 1. It follows from (2.26), (2.18), (2.15), and (2.1) that {xt }∞ t=0 is an (Ω)-program. Relations (2.25), (2.26), (2.18), and (2.15) imply that for each integer k ≥ 1 |v(xSk , xSk +1 ) − v(x, ¯ x)| ¯ ≤ 2 · 2−k .

(2.27)

By (2.25), (2.26), (2.24), (2.21), and the choice of δj , j = 1, 2, . . . (see (2.15)–(2.18)) for any integer k ≥ 2 ⎛ ⎞ Tj −1 S k k −1    (j ) (j )  ⎝ v(xt , xt+1 ) − Sk v(x, ¯ x) ¯ = [v xt , xt+1 − v(x, ¯ x)] ¯ ⎠ j =1

t=0

+

k−1 

t=0

k k−1     (j ) (j +1) − v(x, ¯ x)] ¯ ≥− [v xTj , x0 (2 · 2−j + δj ) − 2 2−j .

j =1

j =1

(2.28)

j =1

In view of (2.28) and (2.15), for any integer k ≥ 2 we have S k −1

v(xt , xt+1 ) − Sk v(x, ¯ x) ¯ ≥ −5

k 

2−j ≥ −10.

(2.29)

j =1

t=0

It follows from (2.29) and Proposition 2.1 that {xt }∞ t=0 is a (v, Ω)-good program. In view of (A3) limt→∞ ρ(xt , x) ¯ = 0. On the other hand (2.20), (2.25), and (2.26) imply ¯ ≥ . The contradiction we have reached proves Lemma 2.10. that lim supt→∞ ρ(xt , x) Lemma 2.11 Let > 0 and M0 > 0. Then there exists a natural number T0 such that for each integer T ≥ T0 , each (Ω)-program {xt }Tt=0 which satisfies T −1 

v(xt , xt+1 ) ≥ T v(x, ¯ x) ¯ − M0

(2.30)

t=0

and each integer s ∈ [0, T − T0 ] the inequality min{ρ(xi , x) ¯ : i = s + 1, . . . , s + T0 } ≤ holds. Proof By Lemma 2.9 there is a natural number T0 such that the following property holds:

20

2 Optimal Control Problems with Singleton Turnpikes 0 (P0) For each (Ω)-program {xt }Tt=0 which satisfies

T 0 −1

v(xt , xt+1 ) ≥ T0 v(x, ¯ x) ¯ − M0 − 2c¯

t=0

the relation ¯ : i = 1, . . . , T0 } ≤ min{ρ(xi , x) holds. Let an integer T ≥ T0 , let an (Ω)-program {xt }Tt=0 satisfy (2.30) and let an integer s ∈ [0, T − T0 ]. It follows from (2.30) and (A2) that s+T 0 −1 

v(xt , xt+1 ) − T0 v(x, ¯ x) ¯ ≥ −M0 − 2c. ¯

t=s

By the inequality above and (P0) ¯ : i = s + 1, . . . , s + T0 } ≤ . min{ρ(xi .x) Lemma 2.11 is proved.

2.4 Auxiliary Results We use all the notation, definitions, and assumptions of Sect. 2.1. Lemma 2.12 Let be a positive number and let L be a natural number. Then there exists δ > 0 such that for each Ωt ∈ E(δ), t = 0, . . . , L − 1, each ut ∈ M, t = 0, . . . , L − 1 satisfying ut − v ≤ δ, t = 0, . . . , L − 1 L L and each ({Ωt }L−1 t=0 )-program {xt }t=0 there exists an (Ω)-program {yt }t=0 such that

ρ(xt , yt ) ≤ for all t = 0, . . . , L and

L−1  t=0

v(yt , yt+1 ) ≥

L−1 

ut (xt , xt+1 ) − .

t=0

Proof Assume that the lemma does not hold. Then for each natural number k there exist sets Ωt(k) ∈ E(k −1 ), t = 0, . . . , L − 1,

(2.31)

2.4 Auxiliary Results

21

functions ut(k) ∈ M, t = 0, . . . , L − 1 satisfying ut(k) − v ≤ k −1 , t = 0, . . . , L − 1

(2.32)

(k) L and an ({Ωt(k) }L−1 t=0 )-program {xt }t=0 such that the following property holds: (P1) if an (Ω)-program {yt }Lt=0 satisfies   ρ xt(k) , yt ≤ for all t = 0, . . . , L,

then

L−1 

v(yt , yt+1 )
k0 such that 16k1−1 L < .

(2.36)

22

2 Optimal Control Problems with Singleton Turnpikes

Assume that an integer k ≥ k1 . Then (2.34) and (2.35) hold. In view of (2.32) and (2.35), L−1 

v(xt , xt+1 ) ≥

t=0

L−1 

(k) v(xt(k) , xt+1 ) − /4

t=0

≥

L−1 

(k) ) − /4 − L max{ut − v : t = 0, . . . , L − 1} ut (xt(k) , xt+1

t=0

≥

L−1 

(k) ut (xt(k) , xt+1 ) − /4 − Lk1−1 ≥

t=0

L−1 

(k) ut (xt(k) , xt+1 ) − /2.

t=0

When combined with (2.34) this contradicts property (P1). The contradiction we have reached proves Lemma 2.12. Lemma 2.13 Let , M be positive numbers. Then there exists a natural number L such that for each integer L˜ ≥ L there exists δ > 0 such that the following assertion holds. ˜ each For each integer T ∈ [L, L], Ωt ∈ E(δ), t = 0, . . . , T − 1,

(2.37)

each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ δ, t = 0, . . . , T − 1

(2.38)

−1 and each ({Ωt }Tt=0 )-program {xt }Tt=0 satisfying T −1 

ut (xt , xt+1 ) ≥

t=0

T −1 

ut (x, ¯ x) ¯ −M

(2.39)

t=0

the inequality ¯ : t = 1, . . . , T } ≤ min{ρ(xt , x) holds. Proof We may assume without loss of generality that < 1. By Lemma 2.11, there exists a natural number L such that the following property holds: (P2) for each integer T ≥ L, each (Ω)-program {xt }Tt=0 which satisfies T −1 

v(xt , xt+1 ) ≥ T v(x, ¯ x) ¯ −M −4

t=0

and each integer S ∈ [0, T − L], we have min{ρ(xt , x) ¯ : t = S + 1, . . . , S + L} ≤ /4. Let an integer L˜ ≥ L. By Lemma 2.12, there exists δ0 > 0 such that the following property holds:

2.4 Auxiliary Results

23

˜ each (P3) for each integer T ∈ [L, L], Ωt ∈ E(δ0 ), t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ δ0 , t = 0, . . . , T − 1 −1 )-program {zt }Tt=0 there exists an (Ω)-program {yt }Tt=0 such that and each ({Ωt }Tt=0

ρ(zt , yt ) ≤ /4for allt = 0, . . . , T and

T −1 

v(yt , yt+1 ) ≥

t=0

T −1 

ut (zt , zt+1 ) − /4

t=0

Set δ = min{δ0 , L˜ −1 }.

(2.40)

˜ (2.37) holds, functions ut ∈ M, t = Assume that an integer T ∈ [L, L], −1 0, . . . , T − 1 satisfy (2.38) and ({Ωt }Tt=0 )-program {xt }Tt=0 satisfies (2.39). By (2.37), (2.38), (2.40), and property (P3), there exists an (Ω)-program {yt }Tt=0 such that ρ(xt , yt ) ≤ /4 for all t = 0, . . . , T

(2.41)

and T −1 

v(yt , yt+1 ) ≥

T −1 

ut (xt , xt+1 ) − /4.

(2.42)

t=0

t=0

In view of (2.38), (2.39), (2.40), and (2.42), T −1 

v(yt , yt+1 ) ≥

t=0

T −1 

ut (xt , xt+1 ) − /4

t=0

≥

T −1 

ut (x, ¯ x) ¯ − M − /4

t=0

≥ T v(x, ¯ x) ¯ − T δ − M − /4 ≥ T v(x, ¯ x) ¯ − M − 2 ≥ T v(x, ¯ x) ¯ − M − 2. It follows from (2.43) and property (P2) that ¯ : t = 1, . . . , L} ≤ /4. min{ρ(yt , x) When combined with (2.41) this implies that ¯ : t = 1, . . . , T } ≤ /2. min{ρ(xt , x) Lemma 2.13 is proved.

(2.43)

24

2 Optimal Control Problems with Singleton Turnpikes

Lemma 2.14 For each natural number T , σ (v, T , x, ¯ x) ¯ = T v(x, ¯ x). ¯ Proof Let T be a natural number. Clearly, σ (v, T , x, ¯ x) ¯ ≥ T v(x, ¯ x). ¯ Assume that an (Ω)-program {yt }Tt=0 satisfies y0 = x, ¯ yT = x. ¯

(2.44)

For all integers i > T define yi ∈ X such that yi+T = yi for all integers i ≥ 0.

(2.45)

It is clear that {yt }∞ t=0 is an (Ω)-program. By (2.45) and assumption (A2), for each natural number k, T kv(x, ¯ x) ¯ + c¯ ≥

kT −1 

v(yt , yt+1 ) = k

t=0

and k(−T v(x, ¯ x) ¯ +

T −1 

v(yt , yt+1 )

t=0 T −1 

v(yt , yt+1 )) ≤ c¯

t=0

for all natural numbers k. This implies that T −1 

v(yt , yt+1 ) ≤ T v(x, ¯ x) ¯

t=0

and σ (v, T , x, ¯ x) ¯ ≤ T v(x, ¯ x). ¯ Lemma 2.14 is proved. Lemma 2.15 Let > 0. Then there exists δ ∈ (0, r¯ ) such that for each natural number T and each z0 , z1 ∈ X satisfying ρ(zi , x) ¯ ≤ δ, i = 0, 1

(2.46)

the inequality |σ (v, T , z0 , z1 ) − T v(x, ¯ x)| ¯ ≤ holds. Proof In view of (A1), there exists a positive number δ < r¯ such that for each y, z ∈ X satisfying ρ(y, x), ¯ ρ(z, x) ¯ ≤ δ, we have |v(y, z) − v(x, ¯ x)| ¯ ≤ /8.

(2.47)

2.4 Auxiliary Results

25

Assume that T is a natural number and that z0 , z1 ∈ X satisfy (2.46). Define y0 = z0 , yi = x¯ for all integers i satisfying 1 ≤ i < T , yT = z1 . By (2.1), (2.46), and (2.48),

{yt }∞ t=0

(2.48)

is an (Ω)-program and

σ (v, T , z0 , z1 ) ≥

T −1 

v(yt , yt+1 ).

(2.49)

t=0

It follows from (2.46), (2.47), and (2.48) that for i = 0, T − 1, ¯ x)| ¯ ≤ /8. |v(yi , yi+1 ) − v(x, By the inequality above, (2.48) and (2.49), σ (v, T , z0 , z1 ) ≥ T v(x, ¯ x) ¯ − /4. Let an (Ω)-program

{y˜t }∞ t=0

(2.50)

satisfy y˜0 = z0 , y˜T = z1 .

(2.51)

 y0 = x, ¯  yi = y˜i−1 , i = 1, . . . , T + 1,  yT +2 = x. ¯

(2.52)

Set +2 In view of (2.1), (2.46), (2.51), and (2.52), { yt }Tt=0 is an (Ω)-program. It follows from Lemma 2.14, (2.51), and (2.52) that

(T + 2)v(x, ¯ x) ¯ ≥

T +1 

v( yt ,  yt+1 )

t=0

=

T −1 

v(y˜t , y˜t+1 ) + v(x, ¯ z0 ) + v(z1 , x). ¯

(2.53)

t=0

By (2.46) and (2.47), ¯ x)| ¯ ≤ /8, |v(x, ¯ z0 ) − v(x, ¯ − v(x, ¯ x)| ¯ ≤ /8. |v(z1 , x)

(2.54)

It follows from (2.53) and (2.54) that (T + 2)v(x, ¯ x) ¯ ≥

T −1 

v(y˜t , y˜t+1 ) + 2v(x, ¯ x) ¯ − /4

t=0

and

T −1 

v(y˜t , y˜t+1 ) ≤ T v(x, ¯ x) ¯ + /4.

t=0

Since this inequality holds for any (Ω)-program {y˜t }Tt=0 satisfying (2.51) we have σ (v, T , z0 , z1 ) ≤ T v(x, ¯ x) ¯ + /4. When combined with (2.50) this inequality completes the proof of Lemma 2.15.

26

2 Optimal Control Problems with Singleton Turnpikes

¯ such that Lemma 2.16 Let be a positive number. Then there exists δ ∈ (0, λ) for each natural number L there exists δ0 ∈ (0, δ) such that the following assertion holds. For each natural number T ≤ L, each Ωt ∈ E(δ0 ), t = 0, . . . , T − 1,

(2.55)

each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ δ0 , t = 0, . . . , T − 1

(2.56)

−1 and each ({Ωt }Tt=0 )-program {xt }Tt=0 satisfying

ρ(x0 , x), ¯ ρ(xT , x) ¯ ≤δ

(2.57)

and T −1 

−1 −1 ut (xt , xt+1 ) ≥ σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0 , xT ) − δ

(2.58)

t=0

¯ ≤ holds for all t = 0, . . . , T . the inequality ρ(xt , x) Proof By Lemma 2.10, there exists a positive number γ < min{ /4, λ¯ /4} such that the following property holds: (P4) for each integer T ≥ 1 and each (Ω)-program {xt }Tt=0 which satisfies ρ(x0 , x), ¯ ρ(xT , x) ¯ ≤ 2γ , T −1 

v(xt , xt+1 ) ≥ σ (v, T , x0 , xT ) − 2γ

t=0

¯ ≤ /4 holds for all t = 0, . . . , T . the inequality ρ(xt , x) By Lemma 2.15, there exists δ ∈ (0, γ /4) such that the following property holds: (P5) for each natural number T and each z0 , z1 ∈ X satisfying ρ(zi , x) ¯ ≤ 2δ, i = 0, 1, we have |σ (v, T , z0 , z1 ) − T v(x, ¯ x)| ¯ ≤ γ /4. Let L be a natural number. By Lemma 2.12, there exists δ1 ∈ (0, δ) such that the following property holds:

2.4 Auxiliary Results

27

(P6) for each integer T ∈ [1, L], each Ωt ∈ E(δ1 ), t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ δ1 , t = 0, . . . , T − 1 −1 )-program {xt }Tt=0 there exists an (Ω)-program {yt }Tt=0 such that and each ({Ωt }Tt=0

ρ(xt , yt ) ≤ δ/4 for all t = 0, . . . , T and

T −1 

v(yt , yt+1 ) ≥

t=0

T −1 

ut (xt , xt+1 ) − δ/4.

t=0

Set δ0 = (2 L)−1 δ1 .

(2.59)

Assume that an integer T ∈ [1, L], (2.55) holds, functions ut ∈ M, t = −1 0, . . . , T − 1 satisfy (2.56) and that an ({Ωt }Tt=0 )-program {xt }Tt=0 satisfies (2.57) and (2.58). By (2.55), (2.56), (2.59), and property (P6), there exists an (Ω)-program {yt }Tt=0 such that ρ(xt , yt ) ≤ δ/4 for all t = 0, . . . , T

(2.60)

and T −1 

v(yt , yt+1 ) ≥

t=0

T −1 

ut (xt , xt+1 ) − δ/4.

(2.61)

t=0

In view of (2.58) and (2.61), T −1 

 −1 −1 v(yt , yt+1 ) ≥ σ {ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0 , xT − δ − δ/4.

(2.62)

t=0

Set x˜0 = x0 , x˜t = x¯ for all integers t satisfying 1 ≤ i < T , x˜T = xT .

(2.63)

−1 )-program. It In view of (2.4), (2.55), (2.57), and (2.63), {x˜t }Tt=0 is an ({Ωt }Tt=0 follows from (2.57), (2.63), and property (P5) that

¯ x)| ¯ ≤ γ /4, i = 0, T − 1. |v(x˜i , x˜i+1 ) − v(x,

(2.64)

28

2 Optimal Control Problems with Singleton Turnpikes

By (2.56), (2.59), (2.62), and (2.63), T −1 

T −1 

v(yt , yt+1 ) ≥

t=0

ut (x˜t , x˜t+1 ) − δ − δ/4

t=0

≥ −δ − δ/4 +

T −1 

v(x˜t , x˜t+1 )

t=0

− T max{ut − v : t = 0, . . . , T − 1} ≥

T −1 

v(x˜t , x˜t+1 ) − δ − δ/4 − Lδ0

t=0

≥

T −1 

v(x˜t , x˜t+1 ) − δ − δ/4 − δ/2.

(2.65)

t=0

It follows from (2.63), (2.64), and (2.65) that T −1 

v(yt , yt+1 ) ≥ T v(x, ¯ x) ¯ − γ /2 − 2δ.

(2.66)

t=0

In view of (2.60), for i = 0, T , ¯ ≤ ρ(yi , xi ) + ρ(xi , x) ¯ ≤ δ/4 + δ. ρ(yi , x)

(2.67)

By (2.67) and property (P5), |σ (v, T , y0 , yT ) − T v(x, ¯ x)| ¯ ≤ γ /4.

(2.68)

It follows from (2.66) and (2.68) that T −1 

v(yt , yt+1 ) ≥ σ (v, T , y0 , yT ) − γ /2 − γ /4 − 2δ

t=0

≥ σ (v, T , y0 , yT ) − 2γ .

(2.69)

In view of (2.67), (2.69), and property (P4), ρ(yt , x) ¯ ≤ /4 for all t = 0, . . . , T . By (2.60) and (2.70), for all t = 0, . . . , T , ρ(xt , x) ¯ ≤ ρ(xt , yt ) + ρ(yt , x) ¯ ≤ δ/4 + /4 < . This completes the proof of Lemma 2.16. In order to prove our stability results we need the following two lemmas.

(2.70)

2.4 Auxiliary Results

29

Lemma 2.17 Let , M0 be positive numbers and l1 , l2 be natural numbers. Then there exist δ > 0 and a natural number L > l1 + l2 such that for each integer T ≥ L, each Ωt ∈ E(δ), t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ δ, t = 0, . . . , T − 1, −1 each ({Ωt }Tt=0 )-program {xt }Tt=0 satisfying −1 xT ∈ Y ({Ωt }Tt=T −l2 , T − l2 , T )

and −1 −1 σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0 , xT )

≤

T −1 

ut (xt , xt+1 ) + M0

t=0

and each integer S ∈ [0, T − L] satisfying 1 −1 xS ∈ Y¯ ({Ωt }S+l , S, S + l1 ) t=S

the inequality min{ρ(xt , x) ¯ : t = S + 1, . . . , S + L} ≤ holds. Lemma 2.18 Let , M0 be positive numbers and l1 be a natural number. Then there exists δ > 0 and a natural number L > l1 such that for each integer T ≥ L, each Ωt ∈ E(δ), t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ δ, t = 0, . . . , T − 1, −1 each ({Ωt }Tt=0 )-program {xt }Tt=0 satisfying −1 −1 σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0 ) ≤

T −1 

ut (xt , xt+1 ) + M0

t=0

and each integer S ∈ [0, T − L] satisfying 1 −1 xS ∈ Y¯ ({Ωt }S+l , S, S + l1 ) t=S

the inequality min{ρ(xt , x) ¯ : t = S + 1, . . . , S + L} ≤ holds.

30

2 Optimal Control Problems with Singleton Turnpikes

Proof We prove Lemmas 2.17 and 2.18 simultaneously. We may assume without loss of generality that ¯ < λ.

(2.71)

By Lemma 2.13, there exists a natural number L0 and δ ∈ (0, ) such that the following property holds: (P7) for each Ωt ∈ E(δ), t = 0, . . . , L0 − 1, each ut ∈ M, t = 0, . . . , L0 − 1 satisfying ut − v ≤ δ, t = 0, . . . , L0 − 1 0 −1 0 and each ({Ωt }Lt=0 )-program {xt }Lt=0 satisfying

L 0 −1

ut (xt , xt+1 ) ≥

t=0

L 0 −1

ut (x, ¯ x) ¯ −1

t=0

the inequality ¯ : t = 1, . . . , L0 } ≤ min{ρ(xt , x) holds. In the case of Lemma 2.18 set l2 = 1. Choose a natural number k0 such that k0 > 2(v + 1)(L0 + l1 + l2 + 1) + M0 + 1.

(2.72)

L = L 0 k0 .

(2.73)

Ωt ∈ E(δ), t = 0, . . . , T − 1,

(2.74)

Set

Let an integer T ≥ L,

functions ut ∈ M, t = 0, . . . , T − 1 satisfy ut − v ≤ δ, t = 0, . . . , T − 1,

(2.75)

−1 ({Ωt }Tt=0 )-program {xt }Tt=0 satisfies −1 xT ∈ Y ({Ωt }Tt=T −l2 , T − l2 , T )

(2.76)

and −1 −1 σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0 , xT ) ≤

T −1  t=0

ut (xt , xt+1 ) + M0

(2.77)

2.4 Auxiliary Results

31

in the case of Lemma 2.17 and satisfies −1 −1 , {Ωt }Tt=0 , 0, T , x0 ) ≤ σ ({ut }Tt=0

T −1 

ut (xt , xt+1 ) + M0

(2.78)

t=0

in the case of Lemma 2.18. Assume that an integer

satisfies

S ∈ [0, T − L]

(2.79)

  1 −1 , S, S + l1 . xS ∈ Y¯ {Ωt }S+l t=S

(2.80)

In order to complete the proof of Lemmas 2.17 and 2.18 it is sufficient to show that ¯ : t = S + 1, . . . , S + L} ≤ . min{ρ(xt , x) Assume the contrary. Then ¯ > for all t = S + 1, . . . , S + L. ρ(xt , x)

(2.81)

There are two cases: (1) There is an integer S0 ∈ (S, T ] such that ¯ ≤ . ρ(xS0 , x)

(2.82)

¯ > for all t = S + 1, . . . , T . ρ(xt , x)

(2.83)

(2)

Assume that the case (1) holds. In view of (2.81) and (2.82), S0 > S + L.

(2.84)

We may assume without loss of generality that ¯ > for all t = S + 1, . . . , S0 − 1. ρ(xt , x)   1 −1 1 -program {yt }S+l In view of (2.80), there exists an {Ωt }S+l t=S such that t=S

(2.85)

¯ yS = xS , yS+l1 = x.

(2.86)

¯ t = S + l1 + 1, . . . , S0 − 1. yS0 = xS0 , yt = x,

(2.87)

Set

By (2.4), (2.71), (2.82), (2.86), and (2.87),   0 0 −1 is an {Ωt }St=S —program. {yt }St=S

(2.88)

32

2 Optimal Control Problems with Singleton Turnpikes

It follows from (2.75), (2.77), (2.78), (2.86), (2.87), and (2.88) that S 0 −1

ut (xt , xt+1 ) ≥

t=S

S 0 −1

ut (yt , yt+1 ) − M0

t=S

≥

S 0 −1

ut (x, ¯ x) ¯ −2

S+l 1 −1 

t=S

≥

S 0 −1

ut  − 2uS0 −1  − M0

t=S

ut (x, ¯ x) ¯ − M0 − 2(l1 + 1)(v + 1).

(2.89)

t=S

There exists a natural number k such that S0 − S ∈ (kL0 , (k + 1)L0 ].

(2.90)

By (2.73), (2.84), and (2.90), k ≥ k0 .

(2.91)

It follows from (2.74), (2.75), (2.85), (2.90), (2.91), and property (P7) that for each integer j ∈ [0, k − 1], S+(j +1)L0 −1



S+(j +1)L0 −1

ut (xt , xt+1 )
2 L, Ωt ∈ E(δ), t = 0, . . . , T − 1,

(2.105)

ut ∈ M, t = 0, . . . , T − 1 satisfy ut − v ≤ δ, t = 0, . . . , T − 1,

(2.106)

−1 and that an ({Ωt }Tt=0 )-program {xt }Tt=0 satisfies 1 −1 x0 ∈ Y¯ ({Ωt }lt=0 , 0, l1 ).

(2.107)

Moreover, assume that in the case of Theorem 2.2 −1 xT ∈ Y ({Ωt }Tt=T −l2 , T − l2 , T ),

−1 −1 , {Ωt }Tt=0 , 0, T , x0 , xT ) ≤ σ ({ut }Tt=0

T −1 

ut (xt , xt+1 ) + δ

(2.108)

(2.109)

t=0

and that in the case of Theorem 2.3, −1 −1 , {Ωt }Tt=0 , 0, T , x0 ) ≤ σ ({ut }Tt=0

T −1 

ut (xt , xt+1 ) + δ.

(2.110)

t=0

¯ applying by induction By (2.1), (2.4), (2.105)–(2.109), and the inequality γ < λ, property (i) in the case of Theorem 2.2 and applying by induction property (ii) in the case of Theorem 2.3, we obtain a finite sequence of integer Si , i = 0, . . . , q such that 0 ≤ S0 ≤ L, T − L < Sq ≤ T , 1 ≤ Si+1 − Si ≤ L, i = 0, . . . , q − 1, ¯ ≤ γ , i = 0, . . . , q. ρ(xSi , x) ¯ ≤ δ, we may assume that S0 = 0 and if ρ(xT , x) ¯ ≤ δ, we may assume If ρ(x0 , x) that Sq = T . Set τ1 = S0 , τ2 = Sq . Let an integer t ∈ [τ1 , τ2 ]. Then there exists an integer i ∈ [0, q − 1] such that t ∈ [Si , Si+1 ]. By the inclusion above, the choice of Si , i = 0, . . . , q, (2.105), (2.106), (2.109), (2.110), and property (P9), ρ(xt , x) ¯ ≤ . This completes the proof of Theorems 2.2 and 2.3.

2.6 Proofs of Theorems 2.4 and 2.5

2.6

37

Proofs of Theorems 2.4 and 2.5

We prove Theorems 2.4 and 2.5 simultaneously. ¯ such that the following property holds: By Lemma 2.16, there exists γ ∈ (0, λ) (P10) For each natural number L there exists γL ∈ (0, γ ) such that for each natural number T ≤ L, each Ωt ∈ E(γL ), t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ γL , t = 0, . . . , T − 1 −1 )-program {yt }Tt=0 satisfying and each ({Ωt }Tt=0

¯ ρ(yT , x) ¯ ≤γ ρ(y0 , x), and

T −1 

−1 −1 ut (yt , yt+1 ) ≥ σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , y0 , yT ) − γ

t=0

the inequality ρ(yt , x) ¯ ≤ holds for all t = 0, . . . , T . In the case of Theorem 2.5 set l2 = 1. By Lemmas 2.17 and 2.18 (with = γ and M0 = M + 1), there exist γ˜ ∈ (0, γ ) and a natural number L0 > l1 + l2 such that the following properties hold: (i) For each integer T ≥ L0 , each Ωt ∈ E(γ˜ ), t = 0, . . . , T − 1,

(2.111)

each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ γ˜ , t = 0, . . . , T − 1,

(2.112)

−1 )-program {zt }Tt=0 satisfying each ({Ωt }Tt=0 −1 zT ∈ Y ({Ωt }Tt=T −l2 , T − l2 , T )

and −1 −1 σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , z0 , zT ) ≤

T −1 

ut (zt , zt+1 ) + 1 + M

t=0

and each integer S ∈ [0, T − L0 ] satisfying 1 −1 , S, S + l1 ) zS ∈ Y¯ ({Ωt }S+l t=S

(2.113)

38

2 Optimal Control Problems with Singleton Turnpikes

the inequality min{ρ(zt , x) ¯ : t = S + 1, . . . , S + L0 } ≤ γ

(2.114)

holds. (ii) For each integer T ≥ L0 , each Ωt ⊂ X × X, t = 0, . . . , T − 1, satisfying −1 (2.111), each ut ∈ M, t = 0, . . . , T − 1 satisfying (2.112), each ({Ωt }Tt=0 )-program T {zt }t=0 satisfying −1 −1 σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , z0 ) ≤

T −1 

ut (zt , zt+1 ) + 1 + M

t=0

and each integer S ∈ [0, T − L0 ] satisfying (2.113) inequality (2.114) holds. By (P10) there exists δ ∈ (0, γ˜ ) such that the following property holds: (P11) For each natural number T ≤ 2L0 + 4, each Ωt ∈ E(δ), t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ δ, t = 0, . . . , T − 1 −1 )-program {yt }Tt=0 satisfying and each ({Ωt }Tt=0

ρ(y0 , x), ¯ ρ(yT , x) ¯ ≤γ and

T −1 

−1 −1 ut (yt , yt+1 ) ≥ σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , y0 , yT ) − γ

t=0

¯ ≤ holds for all t = 0, . . . , T . the inequality ρ(yt , x) Choose a natural number L > (4 + γ −1 (1 + M))(L0 + 1).

(2.115)

Assume that an integer T > L, Ωt ∈ E(δ), t = 0, . . . , T − 1,

(2.116)

ut ∈ M, t = 0, . . . , T − 1 satisfy ut − v ≤ δ, t = 0, . . . , T − 1,

(2.117)

−1 and that an ({Ωt }Tt=0 )-program {xt }Tt=0 satisfies 1 −1 x0 ∈ Y¯ ({Ωt }lt=0 , 0, l1 ).

(2.118)

2.6 Proofs of Theorems 2.4 and 2.5

39

Moreover, assume that in the case of Theorem 2.4 −1 xT ∈ Y ({Ωt }Tt=T −l2 , T − l2 , T ),

−1 −1 , {Ωt }Tt=0 , 0, T , x0 , xT ) ≤ σ ({ut }Tt=0

T −1 

(2.119)

ut (xt , xt+1 ) + M

(2.120)

t=0

and that in the case of Theorem 2.5 −1 −1 σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0 ) ≤

T −1 

ut (xt , xt+1 ) + M.

(2.121)

t=0

By (2.115), (2.116)–(2.121), and the inequality δ < γ˜ , applying by induction property (i) in the case of Theorem 2.4 and applying by induction propert (ii) in the case of Theorem 2.5, we obtain a finite sequence of integers Si , i = 0, . . . , q such that 0 ≤ S0 ≤ L0 , T − L0 < Sq ≤ T ,

(2.122)

1 ≤ Si+1 − Si ≤ L0 , i = 0, . . . , q − 1,

(2.123)

ρ(xSi , x) ¯ ≤ γ , i = 0, . . . , q.

(2.124)

By (2.120), (2.121), and (2.122), −1 −1 M ≥ σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0 , xT ) −

≥

q−1 

T −1 

ut (xt , xt+1 )

t=0

⎡

Si+1 −1 i+1 −1 ⎣σ ({ut }St=S , {Ωt }t=S , Si , Si+1 , xSi , xSi+1 ) − i

Si+1 −1



i

i=0

⎤ ut (xt , xt+1 )⎦ .

t=Si

(2.125) Set E = {i ∈ {0, . . . , q − 1} : S

−1

S

−1

Si+1 −1

i+1 i+1 σ ({ut }t=S , {Ωt }t=S , Si , Si+1 , xSi , xSi+1 ) − i i



ut (xt , xt+1 ) > γ }.

(2.126)

t=Si

By (2.125) and (2.126), M ≥ γ Card(E) and Card(E) ≤ γ −1 M.

(2.127)

40

2 Optimal Control Problems with Singleton Turnpikes

Let j ∈ {0, . . . , q − 1} \ E.

(2.128)

In view of (2.126) and (2.128), Sj +1 −1



S

−1

S

−1

j +1 j +1 ut (xt , xt+1 ) ≥ σ ({ut }t=S , {Ωt }t=S , Sj , Sj +1 , xSj , xSj +1 ) − γ . j j

(2.129)

t=Sj

It follows from (2.116), (2.117), (2.123), (2.124), (2.129), and property (P11) that ρ(xt , x) ¯ ≤ , t = Sj , . . . , Sj +1 .

(2.130)

Since (2.130) holds for any integer j satisfying (2.128) we conclude that {t ∈ {0, . . . , T } : ρ(xt , x) ¯ > } ⊂ ∪{{Si , . . . , Si+1 } : i ∈ E} ∪ {0, . . . , S0 } ∪ {Sq , . . . , T }. By the inclusion above, (2.115), (2.122), (2.123), and (2.127), Card({t ∈ {0, . . . , T } : ρ(xt , x) ¯ > }) ≤ (L0 + 1)(Card(E) + 2) ≤ (L0 + 1)(2 + Mγ −1 ) < L. This completes the proof of Theorems 2.4 and 2.5.

2.7

Proof of Theorem 2.6

By Lemma 2.16, there exists ¯ γ ∈ (0, min{ , λ}) such that the following property holds: (P12) For each natural number L there exists γL ∈ (0, γ ) such that for each natural number T ≤ L, each Ωt ∈ E(γL ), t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ γL , t = 0, . . . , T − 1 −1 and each ({Ωt }Tt=0 )-program {yt }Tt=0 satisfying

ρ(y0 , x), ¯ ρ(yT , x) ¯ ≤γ and

T −1 

−1 −1 ut (yt , yt+1 ) ≥ σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , y0 , yT ) − γ

t=0

¯ ≤ holds for all t = 0, . . . , T . the inequality ρ(yt , x)

2.7 Proof of Theorem 2.6

41

By Lemma 2.13 (with = γ ), there exists a natural number L0 such that for each integer L1 ≥ L0 there exists γ0 ∈ (0, γ ) such that the following property holds: (P13) For each integer τ ∈ [L0 , L1 ], each Ωt ∈ E(γ0 ), t = 0, . . . , τ − 1, each ut ∈ M, t = 0, . . . , τ − 1 satisfying ut − v ≤ γ0 , t = 0, . . . , τ − 1 −1 )-program {xt }τt=0 satisfying and each ({Ωt }τt=0 τ −1 

ut (xt , xt+1 ) ≥

t=0

τ −1 

ut (x, ¯ x) ¯ −M −1

t=0

the inequality ¯ : t = 1, . . . , τ } ≤ γ min{ρ(xt , x) holds. Let an integer L1 ≥ L0 and let γ0 ∈ (0, γ ) be as guaranteed by (P13). By (P12) there exists δ ∈ (0, γ0 ) such that the following property holds: (P14) For each natural number T ≤ 2L1 + 4, each Ωt ∈ E(δ), t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ δ, t = 0, . . . , T − 1 −1 )-program {yt }Tt=0 satisfying and each ({Ωt }Tt=0

¯ ρ(yT , x) ¯ ≤γ ρ(y0 , x), and

T −1 

−1 −1 ut (yt , yt+1 ) ≥ σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , y0 , yT ) − γ

t=0

¯ ≤ holds for all t = 0, . . . , T . the inequality ρ(yt , x) Assume that an integer T > 3L1 , Ωt ∈ E(δ), t = 0, . . . , T − 1,

(2.131)

ut ∈ M, t = 0, . . . , T − 1 satisfy ut − v ≤ δ, t = 0, . . . , T − 1

(2.132)

−1 )-program {xt }Tt=0 and a finite sequence of integers {Si }i=0 and that an ({Ωt }Tt=0 satisfy q

S0 = 0, Si+1 − Si ∈ [L0 , L1 ], i = 0, . . . , q − 1, Sq > T − L1 ,

(2.133)

42

2 Optimal Control Problems with Singleton Turnpikes Si+1 −1



Si+1 −1

ut (xt , xt+1 ) ≥

t=Si



ut (x, ¯ x) ¯ −M

(2.134)

t=Si

for each integer i ∈ [0, q − 1], Si+2 −1



  Si+2 −1 Si+2 −1 ut (xt , xt+1 ) ≥ σ {ut }t=S , {Ω } , S , S , x , x −γ t i i+2 S S i i+2 t=Si i

(2.135)

t=Si

for each integer i ∈ [0, q − 2] and T −1 

−1 −1 ut (xt , xt+1 ) ≥ σ ({ut }Tt=S , {Ωt }Tt=S , Sq−2 , T , xSq−2 , xT ) − γ . q−2 q−2

(2.136)

t=Sq−2

Let an integer i ∈ [0, q −1]. By (2.131)–(2.134), the inequality δ < γ0 , the choice of γ0 and property (P13), there exists an integer τi such that ¯ ≤ γ. τi ∈ [Si + 1, Si+1 ] and ρ(xτi , x)

(2.137)

Thus for each integer i ∈ [0, q − 1] there exists an integer τi satisfying (2.137). Clearly, τ0 ≤ L1 , τq−1 > T − 2L1 . Let an integer i ∈ [0, q − 2]. By (2.133) and (2.137), 1 ≤ τi+1 − τi ≤ 2L1 , τi , τi+1 ∈ [Si , Si+2 ].

(2.138)

By (2.136) and (2.138), τi+1 −1



−1

i+1 −1 ut (xt , xt+1 ) ≥ σ ({ut }t=τ , {Ωt }ττi+1 , τi , τi+1 , xτi , xτi+1 ) − γ . i i

τ

(2.139)

t=τi p

Thus we have shown that there exists a finite sequence of integers {τi }i=0 such that ¯ ≤ γ , i = 0, . . . , p, ρ(xτi , x) 1 ≤ τi+1 − τi ≤ 2L1 for all integers i satisfying 0 ≤ i < p

(2.140)

and (2.139) holds for all integers i satisfying 0 ≤ i < p. ¯ ≤ γ and τp = T if ρ(xT , x) ¯ ≤ γ. Clearly, we may assume that τ0 = 0 if ρ(x0 , x) Let an integer i satisfies 0 ≤ i < p. By (2.131), (2.132), (2.137), (2.139), (2.140), and property (P14), ρ(xt , x) ¯ ≤ , t = τi , . . . , τi+1 . This implies that ¯ ≤ , t = τ0 , . . . , τp ρ(xt , x) and completes the proof of Theorem 2.6.

2.8 Proof of Theorems 2.7 and 2.8

2.8

43

Proof of Theorems 2.7 and 2.8

We prove Theorems 2.7 and 2.8 simultaneously. In the case of Theorem 2.8 set l2 = 1. ¯ We may assume without loss of generality that < λ. By Theorem 2.2, there exist δ1 ∈ (0, ) and a natural number L1 > l1 + l2 such that the following property holds: (P15) For each integer T > 2L1 , each Ωt ∈ E(δ1 ), t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ δ1 , t = 0, . . . , T − 1 −1 and each ({Ωt }Tt=0 )-program {xt }Tt=0 which satisfies

ρ(x0 , x) ¯ ≤ δ1 , ρ(xT , x) ¯ ≤ δ1 , −1 −1 σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0 , xT ) ≤

T −1 

ut (xt , xt+1 ) + δ1

t=0

we have ρ(xt , x) ¯ ≤ for all t = 0, . . . , T . By Theorems 2.4 and 2.5, there exist a natural number L2 > 2L1 and δ ∈ (0, δ1 )

(2.141)

such that the following property holds: (P16) For each integer T > L2 , each Ωt ∈ E(δ), t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ δ, t = 0, . . . , T − 1 −1 and each ({Ωt }Tt=0 )-program {xt }Tt=0 which satisfies at least one of the following conditions below: (i) −1 1 −1 x0 ∈ Y¯ ({Ωt }lt=0 , 0, l1 ), xT ∈ Y ({Ωt }Tt=T −l2 , T − l2 , T ), −1 −1 σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0 , xT ) ≤

T −1 

ut (xt , xt+1 ) + M;

t=0

(ii)

1 −1 x0 ∈ Y¯ ({Ωt }lt=0 , 0, l1 ),

−1 −1 σ ({ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0 ) ≤

T −1  t=0

the inequality

ut (xt , xt+1 ) + M

44

2 Optimal Control Problems with Singleton Turnpikes

Card({t ∈ {0, . . . , T } : ρ(xt , x) ¯ > δ1 }) ≤ L2 holds. Set L = 4L2 . Assume that an integer T > 2 L, Ωt ∈ E(δ), t = 0, . . . , T − 1,

(2.142)

ut ∈ M, t = 0, . . . , T − 1 satisfies ut − v ≤ δ, t = 0, . . . , T − 1

(2.143)

−1 )-program {xt }Tt=0 satisfies and that an ({Ωt }Tt=0 τ +L−1 

+L−1 +L−1 ut (xt , xt+1 ) ≥ σ ({ut }τt=τ , {Ωt }τt=τ , τ , τ + L, xτ , xτ +L ) − δ

(2.144)

t=τ

for each integer τ ∈ [0, T − L] and at least one of the conditions (i) and (ii) hold. Set ¯ > δ1 }. E0 = {t ∈ {0, . . . , T } : ρ(xt , x)

(2.145)

In view of (2.142), (2.143), (2.145), (P16), and conditions (i) and (ii), Card(E0 ) ≤ L2 .

(2.146)

¯ ≤ δ1 }, τ1 = min{t ∈ {0, . . . , T } : ρ(xt , x)

(2.147)

τ2 = max{t ∈ {0, . . . , T } : ρ(xt , x) ¯ ≤ δ1 },

(2.148)

Set

By (2.145) and (2.148), τ1 ≤ L2 , τ2 ≥ T − L2 .

(2.149)

In order to complete the proof of Theorems 2.7 and 2.8 it is sufficient to show that ¯ ≤ for all t = τ1 , . . . , τ2 . ρ(xt , x) Let an integer t ∈ [τ1 , τ2 ].

(2.150)

In view of (2.149), (2.150), and the inequality T > 2 L, at least one of the following inequalities holds: t − τ1 > 2L2 ; τ2 − t > 2L2 .

(2.151)

2.8 Proof of Theorems 2.7 and 2.8

45

Define integers τ˜1 , τ˜2 as follows: ift − τ1 ≤ 2L2 setτ˜1 = τ1 ;

(2.152)

if t − τ1 > 2L2 , then by (2.145) and (2.146) there exists an integer τ˜1 such that τ˜1 ∈ [t − 2L2 , t − L2 ], ρ(xτ˜1 , x) ¯ ≤ δ1 ;

(2.153)

if τ2 − t ≤ 2L2 set τ˜2 = τ2 ;

(2.154)

if τ2 − t > 2L2 , then by (2.145) and (2.146) there exists an integer τ˜2 such that τ˜2 ∈ [t + L2 , t + 2L2 ], ρ(xτ˜2 , x) ¯ ≤ δ1 .

(2.155)

By (2.151)–(2.155), τ˜2 − τ˜1 = τ˜2 − t + t − τ˜1 ≤ 4L2 = L, τ˜2 − τ˜1 = τ˜2 − t + t − τ˜1 ≥ max{τ˜2 − t, t − τ˜1 } ≥ L2 .

(2.156) (2.157)

It follows from (2.144), (2.156), and (2.157) that τ ˜2 −1

˜2 −1 τ˜2 −1 ut (xt , xt+1 ) ≥ σ ({ut }τt= τ˜1 , {Ωt }t=τ˜1 , τ˜1 , τ˜2 , xτ˜1 , xτ˜2 ) − δ.

(2.158)

t=τ˜1

In view of (2.141)–(2.143), (2.147), (2.148), (2.152)–(2.158), and property (P15), ¯ ≤ . ρ(xt , x) This completes the proof of Theorems 2.7 and 2.8.

Chapter 3

Optimal Control Problems with Discounting

In this chapter we continue our study of the structure of approximate solutions of the discrete-time optimal control problems with a compact metric space of states X and with a singleton turnpike. These problems are described by a nonempty closed set Ω ⊂ X × X which determines a class of admissible trajectories (programs) and by a bounded upper semicontinuous objective function v : X × X → R 1 which determines an optimality criterion. We show the stability of the turnpike phenomenon under small perturbations of the objective function v and the set Ω in the case with discounting. The results of the chapter generalize the results obtained in [54] for the discounting case with a perturbation only on the objective function.

3.1

Stability of the Turnpike Phenomenon

We use the notation, definitions, and assumptions introduced in Sect. 2.1. Denote by M0 the set of all upper semicontinuous functions u ∈ M. It is not difficult to see that the following result holds. Proposition 3.1 Let l be a natural number, integers T1 , T2 satisfy 0 ≤ T1 ≤ T2 − l, 2 −1 ⊂ M0 , for any integer t ∈ {T1 , . . . , T2 − 1}, let Ωt be a closed subset of {ut }Tt=T 1 X × X such that (x, ¯ x) ¯ ∈ Ωt and let   1 +l−1 x ∈ Y¯ {Ωt }Tt=T , 0, l . 1 2 −1 2 Then there exists an ({Ωt }Tt=T )-program {xt }Tt=T such that x0 = x and 1 1

T 2 −1

  T2 −1 2 −1 } {Ω ut (xt , xt+1 ) = σ {ut }Tt=T , , T , T , x . t 1 2 t=T1 1

t=T1

In this chapter we prove the following result which shows the stability of the turnpike phenomenon in the case of discounting.

A. J. Zaslavski, Stability of the Turnpike Phenomenon in Discrete-Time Optimal Control Problems, SpringerBriefs in Optimization, DOI 10.1007/978-3-319-08034-5_3, © The Author 2014

47

48

3 Optimal Control Problems with Discounting

 Theorem 3.2 Let ∈ 0, λ¯ and let l be a natural number. Then there exist δ ∈ (0, ), a natural number L > l and λ ∈ (0, 1) such that for each integer T > 2L, each Ωt ∈ E(δ), t = 0, . . . , T − 1, each ut ∈ M0 , t = 0, . . . , T − 1 satisfying ut − v ≤ δ, t = 0, . . . , T − 1, −1 ⊂ (0, 1] such that each sequence {αt }Tt=0

αi αj−1 ≥ λ for each i, j ∈ {0, . . . , T − 1} satisfying |i − j | ≤ L,

 −1 and each {Ωt }Tt=0 -program {xt }Tt=0 which satisfies

 x0 ∈ Y¯ {Ωt }l−1 t=0 , 0, l , T −1

  −1 −1 σ {αt ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0 = αt ut (xt , xt+1 ) t=0

there exist integers τ1 ∈ [0, L], τ2 ∈ [T − L, T ] such that ¯ ≤ for all t = τ1 , . . . , τ2 . ρ(xt , x) ¯ ≤ δ, then τ1 = 0 and if ρ(xT , x) ¯ ≤ δ, then τ2 = T . Moreover if ρ(x0 , x) −1 Roughly speaking, the turnpike property holds if discount coefficients {αt }Tt=0 ⊂ (0, 1] are changed rather slowly. Let Ωt ⊂ X × X be a nonempty set for integers t.

all∞nonnegative  ⊂ X is called an {Ω } A sequence {xt }∞ -program if (xt , xt+1 ) ∈ Ωt for all t t=0 t=0 integers t ≥ 0.

 {ut }∞ {xt }∞ M be given. An {Ωt }∞ -program t=0 ⊂ t=0 is called t=0

Let

 ∞ ∞ ∞ {ut }t=0 , {Ωt }t=0 -overtaking optimal if for each {Ωt }t=0 -program {yt }∞ t=0 satisfying x0 = y0 , we have T −1  T −1   ut (yt , yt+1 ) − ut (xt , xt+1 ) ≤ 0. lim sup T →∞

t=0

t=0

The following result establishes the stability of turnpike phenomenon for overtaking optimal programs.

 Theorem 3.3 Let ∈ 0, λ¯ and let l be a natural number. Then there exist δ ∈ (0, ), a natural number L > l and λ ∈ (0, 1) such that for each Ωt ∈ E(δ), t = 0, 1, . . . , each ut ∈ M0 , t = 0, 1, . . . , satisfying ut − v ≤ δ, t = 0, 1, . . . ,

3.1 Stability of the Turnpike Phenomenon

49

each sequence {αt }∞ t=0 ⊂ (0, 1] such that αi αj−1 ≥ λ for each pair of nonnegative integers i, j satisfying |i − j | ≤ L

 ∞ ∞ and each {αt ut }∞ t=0 , {Ωt }t=0 -overtaking optimal program {xt }t=0 which satisfies

 x0 ∈ Y¯ {Ωt }l−1 t=0 , 0, l the following inequality holds: ρ(xt , x) ¯ ≤ for all integers t ≥ L. Moreover, if ρ(x0 , x) ¯ ≤ δ, then ¯ ≤ for all integers t ≥ 0. ρ(xt , x) In this chapter we prove the following existence result. Theorem 3.4 Let l ≥ 1 be an integer, = λ¯ /4, and let δ ∈ (0, ), an integer L > l and λ ∈ (0, 1) be as guaranteed by Theorem 3.2. Let ut ∈ M0 and ut − v ≤ δ, t = 0, 1, . . . , Ωt ∈ E(δ), t = 0, 1, . . . be closed subsets of X × X for all nonnegative integers t, and let {αt }∞ t=0 ⊂ (0, 1] satisfy the relations lim αt = 0, t→∞

αi αj−1

≥ λ for all nonnegative integers i, j satisfying |i − j | ≤ L.  ∞



 (z) ∞ Then for each z ∈ Y¯ {Ωt }l−1 t=0 , 0, l there exists an {Ωt }t=0 -program xt

t=0

such that x0(z) = z and that the following property holds: For each real number γ > 0 there exists an integer n0 ≥ 1 such that for each  integer T ≥ n0 and each point z ∈ Y¯ {Ωt }l−1 t=0 , 0, l the inequality T −1  

  (z) −1 −1 |σ {αt ut }Tt=0 |≤γ , {Ωt }Tt=0 , 0, T , z − αt ut xt(z) , xt+1 t=0

holds. It is clear that Theorem 3.4 establishes the existence of ({αt ut }∞ t=0 )-overtaking optimal program when the sequence of the discount coefficients {αt }∞ t=0 tends to zero slowly.

 Note that the existence of an {αt ut }∞ t=0 -overtaking optimal program when the discount coefficients {αt }∞ t=0 tends to zero rapidly is a well known fact.

50

3 Optimal Control Problems with Discounting

3.2 Auxiliary Results In the proof of Theorems 3.2 and 3.3 we use the following two lemmas.

 Lemma 3.5 Let ∈ 0, λ¯ and l be a natural number. Then there exist δ ∈ (0, ), a natural number L > l and λ ∈ (0, 1) such that for each integer T ≥ L, each Ωt ∈ E(δ), t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ δ, t = 0, . . . , T − 1, −1 each finite sequence {αt }Tt=0 ⊂ (0, 1] which satisfies

αi αj−1 ≥ λ for each i, j ∈ {0, 1, . . . , T − 1} satisfying |i − j | ≤ L, 

−1 -program {xt }Tt=0 such that each {Ωt }Tt=0 T −1 



−1 −1 αt ut (xt , xt+1 ) = σ {αt ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0

t=0

and each integer S ∈ [0, T − L] satisfying

 xS ∈ Y¯ {Ωt }S+l−1 t=S , S, S + l

(3.1)

the inequality min{ρ(xt , x) ¯ : t = S + 1, . . . , S + L} ≤

(3.2)

holds.

 Lemma 3.6 Let ∈ 0, λ¯ and l be a natural number. Then there exist δ ∈ (0, ), a natural number L and λ ∈ (0, 1) such that for each Ωt ∈ E(δ), t = 0, 1, . . . , each ut ∈ M, t = 0, 1, . . . , satisfying ut − v ≤ δ, t = 0, 1, . . . , each sequence {αt }∞ t=0 ⊂ (0, 1] which satisfies αi αj−1 ≥ λ for each pair of integers i, j ≥ 0 satisfying |i − j | ≤ L,

 ∞ ∞ each {αt ut }∞ t=0 , {Ωt }t=0 -overtaking optimal program {xt }t=0 and each integer S ≥ 0 satisfying (3.1), inequality (3.2) holds.

3.2 Auxiliary Results

51

Proof We prove Lemmas 3.5 and 3.6 simultaneously. By Lemma 2.13, there exist a natural number L0 and δ ∈ (0, ) such that the following property holds: (P17) For each Ωt ∈ E(δ), t = 0, . . . , L0 − 1, each ut ∈ M, t = 0, . . . , L0 − 1 satisfying ut − v ≤ δ, t = 0, . . . , L0 − 1 0 −1 0 )-program {xt }Lt=0 satisfying and each ({Ωt }Lt=0

L 0 −1

ut (xt , xt+1 ) ≥

t=0

L 0 −1

¯ x) ¯ −1 ut (x,

t=0

the inequality min{ρ(xt , x) ¯ : t = 1, . . . , L0 } ≤ holds. Choose a natural number k0 such that k0 > 8(v + 1)(L0 + l + 1).

(3.3)

L = L 0 k0 .

(3.4)

Set

Choose a number λ ∈ (0, 1) such that λl > 2−1 ,

(3.5) −1

8(v + 1)L0 |1 − λ| < 2 ,

(3.6)

λk0 +l+1 k0 > 8(v + 1)(L0 + l + 1).

(3.7)

We suppose that ∞ + x = ∞ for any x ∈ R 1 and that the sum over empty set is zero. Let T ∈ {1, 2 . . . , } ∪ {∞} and T ≥ L, for all integers t satisfying 0 ≤ t < T ,

(3.8)

52

3 Optimal Control Problems with Discounting

Ωt ∈ E(δ), ut ∈ M and ut − v ≤ δ, αt ∈ (0, 1]

(3.9) (3.10) (3.11)

satisfy αi αj−1 ≥ λ for each pair of integers i, j ≥ 0 satisfying i, j < T and |i − j | ≤ L, (3.12)

 −1 and let an {Ωt }Tt=0 -program {xt }Tt=0 satisfy T −1 

 −1 −1 αt ut (xt , xt+1 ) = σ {αt ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0

(3.13)

t=0

in the case of Lemma 3.5, and let

 ∞ ∞ {xt }∞ t=0 be an {αt ut }t=0 , {Ωt }t=0 -overtaking optimal program

(3.14)

in the case of Lemma 3.6. Assume that an integer S ≥ 0 satisfies

 S ≤ T − L, xS ∈ Y¯ {Ωt }S+l−1 t=S , S, S + l .

(3.15)

In order to complete the proof of Lemmas 3.5 and 3.6, it is sufficient to show that min {ρ(xt , x) ¯ : t = S + 1, . . . , S + L} ≤ . Assume the contrary. Then ρ(xt , x) ¯ > for all t = S + 1, . . . , S + L.

(3.16)

There are two cases: (1) There is an integer S0 > S such that S ≤ T and

 ρ xS0 , x¯ ≤ .

(3.17)

(2) ρ(xt , x) ¯ > for all integers t satisfying S + 1 ≤ t ≤ T .

(3.18)

Assume that the case (1) holds. Thus (3.17) holds. In view of (3.16) and (3.17), S0 > S + L.

(3.19)

We may assume without loss of generality that ρ (xt , x) ¯ > for all t = S + 1, . . . , S0 − 1.

(3.20)

3.2 Auxiliary Results

53

 In view of (3.15), there exists an {Ωt }S+l−1 -program {yt }S+l t=S t=S such that yS = xS , yS+l = x. ¯

(3.21)

yS0 = xS0 , yt = x, ¯ t = S + l + 1, . . . , S0 − 1.

(3.22)

Set

0 0 −1 ¯ {yt }St=S is an {Ωt }St=S By (2.4), (3.17), (3.21), (3.22), and the inequality < λ, program. It follows from (3.10), (3.12), (3.13), (3.14), (3.21), and (3.22) that

S 0 −1

αt ut (xt , xt+1 ) ≥

t=S

S 0 −1

αt ut (yt , yt+1 )

t=S

≥

S 0 −1

¯ x) ¯ −2 αt ut (x,

t=S

≥

S+l−1 

αt ut  − 2uS0 −1 αS0 −1

t=S

S 0 −1

¯ x) ¯ − 2αS λ−1 l(v + 1) − 2αS0 −1 (v + 1). αt ut (x,

t=S

(3.23) There exists a natural number k such that S0 − S ∈ (kL0 , (k + 1)L0 ].

(3.24)

By (3.4), (3.19), and (3.24), k ≥ k0 . It follows from (3.9), (3.10), (3.20), (3.24), and property (P17) that for each integer j ∈ [0, k − 1], S+(j +1)L0 −1



S+(j +1)L0 −1

ut (xt , xt+1 )
0, a contradiction. The contradiction we have reached proves that case (1) does not hold. Therefore, case (2) holds and (3.18) is true.

 In view of (3.15), there exists an {Ωt }S+l−1 -program {yt }S+l t=S t=S such that yS = xS , yS+l = x. ¯

(3.28)

Set yt = x¯ for all integers t satisfying S + l < t ≤ T . (3.29)

 −1 By (3.28) and (3.29), {yt }Tt=S is an {Ωt }Tt=S -program. It follows from (3.9), (3.10), (3.18), and property (P17) that that for each integer j ≥ 0 satisfying S + (j + 1)L0 ≤ T , S+(j +1)L0 −1



S+(j +1)L0 −1

ut (xt , xt+1 )
0, a contradiction. The contradiction we have reached proves Lemma 3.6. Let us complete the proof of Lemma 3.5. There exists a natural number k such that T − S ∈ [kL0 , (k + 1)L0 ) .

(3.30)

By (3.4), (3.15), and (3.30), k ≥ k0 . By the inequality above, (3.7), (3.10), (3.12), (3.13), (3.28)–(3.30), and (3.27) which holds for each integer j ≥ 0 satisfying S + (j + 1)L0 ≤ T , 0≥

T −1 

αt ut (yt , yt+1 ) −

t=S

≥

T −1 

αt ut (x, ¯ x) ¯ −

T −1 

=

k−1 

αt ut (xt , xt )

t=S

S+l−1  t=S

αt ut (xt , xt )

t=S

t=S

+

T −1 

αt ut (yt , yt+1 ) −

S+l−1 

αt ut (x, ¯ x) ¯

t=S

⎡

S+(j +1)L0 −1



⎣

j =0

t=S+j L0

S+(j +1)L0 −1

αt ut (x, ¯ x) ¯ −



⎤ αt ut (xt , xt )⎦

t=S+j L0

 + {αt ut (x, ¯ x) ¯ − αt ut (xt , xt ) : t is an integer and S + kL0 ≤ t < T } − 2(v + 1)lαS λ−l ≥ 2−1

k−1 

αS+j L0 − 2L0 (v + 1)αS+kL0 λ−1 − 2(v + 1)αS λ−l l

j =0

≥ 4−1 αS

k 0 −1 j =0

λj − 2(v + 1)αS λ−l l

3.3 Proofs of Theorems 3.2 and 3.3 k−1 

+ 4−1

57

αS+kL0 λk−j − 2L0 (v + 1)αS+kL0 λ−1

j =k−k0

 ≥ αS 4−1 k0 λk0 − 2(v + 1)λ−l l

 + αS+kL0 4−1 k0 λk0 − 2L0 (v + 1)λ−1 > 0,

a contradiction. The contradiction we have reached proves Lemma 3.5.

3.3

Proofs of Theorems 3.2 and 3.3

We prove Theorems 3.2 and 3.3 simultaneously. ¯ }) such that the following property By Lemma 2.16, there exists γ ∈ (0, min{λ, holds: (P18) For each natural number L there exists γL ∈ (0, γ ) such that for each natural number S ≤ L, each Ωt ∈ E(γL ), t = 0, . . . , S − 1, each ut ∈ M, t = 0, . . . , S − 1 satisfying ut − v ≤ γL , t = 0, . . . , S − 1

 S and each {Ωt }S−1 t=0 -program {yt }t=0 satisfying ρ(y0 , x), ¯ ρ(yS , x) ¯ ≤γ and S−1 

 S−1 ut (yt , yt+1 ) ≥ σ {ut }S−1 t=0 , {Ωt }t=0 , 0, S, y0 , yS − γ

t=0

¯ ≤ holds for all t = 0, . . . , S. the inequality ρ(yt , x) By Lemmas 3.5 and 3.6 (with = γ ), there exist γ˜ ∈ (0, γ ), a natural number L > l and λ˜ ∈ (0, 1) such that the following properties hold: (i) For each integer T ≥ L, each Ωt ∈ E (γ˜ ) , t = 0, . . . , T − 1, each ut ∈ M, t = 0, . . . , T − 1 satisfying ut − v ≤ γ˜ , t = 0, . . . , T − 1,

58

3 Optimal Control Problems with Discounting

−1 each finite sequence {αt }Tt=0 ⊂ (0, 1] which satisfies

αi αj−1 ≥ λ˜ for each i, j ∈ {0, 1, . . . , T − 1} satisfying |i − j | ≤ L, 

−1 -program {xt }Tt=0 such that each {Ωt }Tt=0 T −1 

 −1 −1 αt ut (xt , xt+1 ) = σ {αt ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0

t=0

and each integer S ∈ [0, T − L] satisfying

 xS ∈ Y¯ {Ωt }S+l−1 t=S , S, S + l

(3.31)

the inequality min {ρ(xt , x) ¯ : t = S + 1, . . . , S + L} ≤ γ

(3.32)

holds. (ii) For each Ωt ∈ E(γ˜ ), t = 0, 1, . . . , each ut ∈ M, t = 0, 1, . . . , satisfying ut − v ≤ γ˜ , t = 0, 1, . . . , each sequence {αt }∞ t=0 ⊂ (0, 1] which satisfies αi αj−1 ≥ λ˜ for each pair of integers i, j ≥ 0 satisfying |i − j | ≤ L,

 ∞ ∞ each {αt ut }∞ t=0 , {Ωt }t=0 -overtaking optimal program {xt }t=0 and each integer S ≥ 0 satisfying (3.31) inequality (3.32) holds. Choose λ ∈ (0, 1) such that ˜ λ < λ, 2|1 − λ|λ−1 (v + 1)L < γ /2.

(3.33)

By property (P18), there exists δ ∈ (0, γ˜ ) such that the following property holds: (P19) For each natural number S ≤ 2 L + 4, each Ωt ∈ E(δ), t = 0, . . . , S − 1, each ut ∈ M, t = 0, . . . , S − 1 satisfying ut − v ≤ δ, t = 0, . . . , S − 1

3.3 Proofs of Theorems 3.2 and 3.3

59

 S and each {Ωt }S−1 t=0 -program {yt }t=0 satisfying ρ(y0 , x), ¯ ρ(yS , x) ¯ ≤γ and S−1 



S−1 ut (yt , yt+1 ) ≥ σ {ut }S−1 t=0 , {Ωt }t=0 , 0, S, y0 , yS − γ

t=0

¯ ≤ holds for all t = 0, . . . , S. the inequality ρ(yt , x) In the case of Theorem 3.2 assume that T > 2 L is an integer. In the case of Theorem 3.3 put T = ∞. Assume that for all integers t satisfying 0 ≤ t < T , Ωt ∈ E(δ),

(3.34)

ut − v ≤ δ,

(3.35)

αt ∈ (0, 1]

(3.36)

ut ∈ M0 satisfies

satisfy αi αj−1 ≥ λ for each pair of integers i, j satisfying 0 ≤ i, j < T , |i − j | ≤ L

(3.37)

 −1 and that an {Ωt }Tt=0 -program {xt }Tt=0 satisfies

 x0 ∈ Y¯ {Ωt }l−1 t=0 , 0, l .

(3.38)

Assume that in the case of Theorem 3.2 T −1

  −1 −1 σ {αt ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0 = αt ut (xt , xt+1 ) .

(3.39)

t=0

In the case of Theorem 3.3 assume that

 ∞ ∞ {xt }∞ t=0 is an {αt ut }t=0 , {Ωt }t=0 − overtaking optimal program.

(3.40)

In the case of Theorem 3.3 using (3.34)–(3.38), (3.40), and applying by induction property (ii) we obtain a sequence of integers Si , i = 0, 1, . . . , such that 0 ≤ S0 ≤ L and for each integer i ≥ 0,

(3.41)

60

3 Optimal Control Problems with Discounting

1 ≤ Si+1 − Si ≤ L,

 ρ xSi , x¯ ≤ γ .

(3.42) (3.43)

In the case of Theorem 3.2 using (3.34)–(3.39) and applying by induction property (i) we obtain a sequence of integers Si , i = 0, 1, . . . , q such that 0 ≤ S0 ≤ L, T − L < Sq ≤ T ,

(3.44)

1 ≤ Si+1 − Si ≤ L, i = 0, . . . , q − 1,

 ρ xSi , x¯ ≤ γ , i = 0, . . . , q.

(3.45) (3.46)

In the case of Theorem 3.3 set q = ∞. If ρ(x0 , x) ¯ ≤ γ , we may assume that S0 = 0. Let an integer i ≥ 0 satisfy i + 1 ≤ q. By (3.40) and (3.39), Si+1 −1



  Si+1 −1 Si+1 −1 αt ut (xt , xt+1 ) = σ {αt ut }t=S , {Ω } , S , S , x , x t t=Si i i+1 Si Si+1 . i

(3.47)

t=Si S

−1

S

i+1 i+1 )-program {yt }t=S , In view of (3.35)–(3.37) and (3.45), for each ({Ωt }t=S i i

  Si+1 −1  Si+1 −1     −1  ut (yt , yt+1 ) − αSi αt ut (yt , yt+1 )   t=Si  t=Si Si+1 −1

≤ (v + 1)



|1 − αS−1 αt | ≤ (v + 1)L|λ − 1|λ−1 . i

(3.48)

t=Si

It follows from (3.33), (3.47), and (3.48) that   Si+1 −1   

 

 Si+1 −1 Si+1 −1  ut xt , xt+1 − σ {ut }t=Si , {Ωt }t=Si , Si , Si+1 , xSi , xSi+1    t=Si  ≤ 2(v + 1)L|λ − 1|λ−1 < γ .

(3.49)

By (3.34), (3.35), (3.44), (3.46), (3.49), and property (P19), ρ(xt , x) ¯ ≤ for all integers t ∈ [Si , Si+1 ].

(3.50)

Since (3.50) holds for any nonnegative integer i satisfying i + 1 < q, Theorems 3.2 and 3.3 are proved.

3.4 Proof of Theorem 3.4

3.4

61

Proof of Theorem 3.4

We recall that δ ∈ (0, ), λ ∈ (0, 1) and the integer L > l be as guaranteed by Theorem 3.2, ut ∈ M0 and ut − v ≤ δ, t = 0, 1, . . . ,

(3.51)

Ωt ∈ E(δ), t = 0, 1, . . .

(3.52)

and that {αt }∞ t=0 ⊂ (0, 1] satisfy the relations lim αt = 0,

t→∞

αi αj−1 ≥ λ for all nonnegative integers i, j satisfying |i − j | ≤ L.

(3.53) (3.54)

In the proof we use the following auxiliary result. Lemma 3.7 Let γ > 0. Then there is a natural number n0 such that for each pair −1 of integers T > S ≥ n0 and each {Ωt }Tt=0 -program {xt }Tt=0 satisfying

 x0 ∈ Y¯ {Ωt }l−1 (3.55) t=0 , 0, l , T −1 

 −1 −1 αt ut (xt , xt+1 ) = σ {αt ut }Tt=0 , {Ωt }Tt=0 , 0, T , x0

(3.56)

t=0

the following inequality holds: S−1 

 S−1 αt ut (xt , xt+1 ) ≥ σ {αt ut }S−1 t=0 , {Ωt }t=0 , 0, S, x0 − γ .

(3.57)

t=0

Proof Since limt→∞ αt = 0 there exists a natural number n0 > 4L + 4 such that for all integers t > n0 − L − 4, αt ≤ γ (8L + 8)−1 (v + 1)−1 .

(3.58)

−1 Assume that integers T > S ≥ n0 and that an {Ωt }Tt=0 -program {xt }Tt=0 satisfies (3.55) and (3.56). By (3.51), (3.52), (3.55), and Proposition 3.1, there is an {Ωt }S−1 t=0 program {x˜t }St=0 such that

x˜0 = x0 , S−1  t=0

 S−1 αt ut (x˜t , x˜t+1 ) = σ {αt ut }S−1 t=0 , {Ωt }t=0 , 0, S, x0 .

(3.59) (3.60)

62

3 Optimal Control Problems with Discounting

By the choice of δ and L, Theorem 3.2, (3.51), (3.52), (3.54), (3.55), (3.56), (3.59), and (3.60), ¯ ¯ ≤ λ/4, t = L, . . . , T − L, ρ(xt , x)

(3.61)

¯ ¯ ≤ λ/4, t = L, . . . , S − L. ρ(x˜t , x)

(3.62)

−1 -program {yt }Tt=0 such that By (2.4), (3.61), and (3.62) there is an {Ωt }Tt=0

yt = x˜t , t = 0, . . . , S − L, yt = xt , t = S − L + 1, . . . , T .

(3.63)

In view of (3.56), (3.63), (3.60), (3.51), and (3.58), 0≤

T −1 

αt ut (xt , xt+1 ) −

T −1 

t=0 S−L 

=

αt ut (xt , xt+1 ) −

t=0

≤

S−L−1 

αt ut (yt , yt+1 )

t=0 S−L 

αt ut (yt , yt+1 )

t=0

αt ut (xt , xt+1 ) −

S−L−1 

t=0

αt ut (x˜t , x˜t+1 ) + 2αS−L (v + 1)

t=0

≤

S−1 

αt ut (xt , xt+1 ) + (v + 1)

t=0

S−1 

αt

t=S−L

S−1  

S−1 αt + 2αS−L (v + 1) −σ {αt ut }S−1 t=0 , {Ωt }t=0 , 0, S, x0 + (v + 1) t=S−L

≤

S−1 



S−1 αt ut (xt , xt+1 ) − σ {αt ut }S−1 t=0 , {Ωt }t=0 , 0, S, x0 + γ .

t=0

Lemma 3.7 is proved. Completion of the proof of Theorem 3.4 Let 

z ∈ Y¯ {Ωt }l−1 t=0 , 0, l .

(3.64)

By (2.4), (3.51), (3.52), (3.64), and Proposition 3.1, for each integer T ≥ 1 there is −1 -program {xt(z,T ) }Tt=0 such that an {Ωt }Tt=0 x0(z,T ) = z, T −1  t=0

 

 (z,T ) −1 −1 = σ {αt ut }Tt=0 αt ut xt(z,T ) , xt+1 , {Ωt }Tt=0 , 0, T , z .

(3.65) (3.66)

3.4 Proof of Theorem 3.4

63

Clearly there exists a strictly increasing sequence of natural numbers {Tj }∞ j =1 such that for any integer t ≥ 0 there exists (z,Tj )

xt(z) = lim xt j →∞

.

(3.67)

∞ Clearly, {xt(z) }∞ t=0 is an {Ωt }t=0 - program and

x0(z) = z,

(3.68)

Let γ > 0. By Lemma 3.7 there is a natural number n0 such that the following property holds: −1 (P20) For each pair of integers T > S ≥ n0 and each {Ωt }Tt=0 -program {xt }Tt=0 satisfying (3.55) and (3.56), Eq. (3.57) holds. Let S ≥ n0 be an integer. By (P20), (3.64), (3.65), and (3.66), for each natural number j satisfying Tj > S, S−1 

 

 (z,T ) (z,T ) S−1 αt ut xt j , xt+1j ≥ σ {αt ut }S−1 t=0 , {Ωt }t=0 , 0, S, z − γ .

t=0

Together with (3.57) this implies that S−1 

 

 (z) S−1 ≥ σ {αt ut }S−1 αt ut xt(z) , xt+1 t=0 , {Ωt }t=0 , 0, S, z − γ

t=0

for all integers S ≥ n0 . Theorem 3.4 is proved.

Chapter 4

Optimal Control Problems with Nonsingleton Turnpikes

In this chapter we study stability of the turnpike phenomenon for approximate solutions for a general class of discrete-time optimal control problems with nonsingleton turnpikes and with a compact metric space of states. This class of optimal control problems is identified with a complete metric space of objective functions. We show that the turnpike phenomenon is stable under perturbations of an objective function if the corresponding infinite horizon optimal control problem possesses an asymptotic turnpike property.

4.1

Discrete-Time Optimal Control Systems

Let (K, d) be a compact metric space equipped with a metric d and with the topology induced by this metric, and let M be a nonempty closed subset of K × K equipped with the product topology. Set A = {x ∈ K : {x} × K ∩ M  = ∅}

(4.1)

a(x) = {y ∈ K : (x, y) ∈ M} for all x ∈ A.

(4.2)

and

Denote by Z the set of all integers. For each pair of integers q > p set Zp = {i ∈ Z : i ≥ p} and Zqp = [p, q] ∩ Z.

(4.3) q

A sequence xi ∈ K, i ∈ I , where I is either Z or Zp or Zp (with p, q ∈ Z satisfying p < q) is called a program if (xi , xi+1 ) ∈ M for each integer i ∈ I such that i + 1 ∈ I . Denote by C(M) the space of all continuous functions v : M → R 1 and by B(M) the space of all bounded functions v : M → R 1 with the topology of the uniform convergence (||v|| = sup{|v(x, y)| : (x, y) ∈ M}).

A. J. Zaslavski, Stability of the Turnpike Phenomenon in Discrete-Time Optimal Control Problems, SpringerBriefs in Optimization, DOI 10.1007/978-3-319-08034-5_4, © The Author 2014

65

66

4 Optimal Control Problems with Nonsingleton Turnpikes

In this chapter we consider the problem T −1 

v(xi , xi+1 ) → min

(P)

i=0 −1 ⊂ M, s.t. {(xi , xi+1 )}Ti=0

where T is a natural number and v ∈ B(M) is a lower semicontinuous function. This discrete-time optimal control system describes a general model of economic dynamics, where the set K is the space of states, −v is a utility function and −v(xt , xt+1 ) evaluates consumption at moment t. For each f ∈ B(M), each y, z ∈ K and each integer q ≥ 1 we set U f (q, y, z) = inf

q−1 

f (xi , xi+1 ) :

i=0 q

{xi }i=0 is a program such that x0 = y, xq = z}.

(4.4)

Let T1 ≥ 0, T2 > T1 be integers, y, z ∈ K, and let fi ∈ B(M), i = T1 , . . . , T2 −1. Set 2 −1 U ({fi }Ti=T , y, z) = inf 1

2 −1  T

fi (xi , xi+1 ) :

i=T1 2 is a program such thatxT1 = y, xT2 = z}. {xi }Ti=T 1

(4.5)

(We suppose that infimum over an empty set is ∞.) For any subset E of a metric space the closure of E is denoted by cl(E) and also ¯ by E. 

∞ For any sequence {xi }∞ i=0 ⊂ K denote by Ω {xi }i=0 the set of all points (z1 , z2 ) ∈ K×K such that some subsequence {(xik , xik +1 )}∞ k=1 converges to (z1 , z2 ) and denote by ω({xi }∞ ) the set of all points z ∈ K such that some subsequence {xik }∞ i=0 k=1 converges to z. Define a metric d1 on K × K by d1 ((x1 , x2 ), (y1 , y2 )) = d(x1 , y1 ) + d(x2 , y2 ), x1 , x2 , y1 , y2 ∈ K. Put d(x, B) = inf{d(x, y) : y ∈ B} for x ∈ K, B ⊂ K and d1 ((x1 , x2 ), E) = inf{d1 ((x1 , x2 ), (y1 , y2 )) : (y1 , y2 ) ∈ E} for (x1 , x2 ) ∈ K × K and E ⊂ K × K. We denote by dist(B1 , B2 ) the Hausdorff metric for two sets B1 , B2 ⊂ K (respectively B1 , B2 ⊂ K × K) and denote by Card(B) the cardinality of a set B.

4.2 The Turnpike Property

67

4.2 The Turnpike Property Denote by Mreg the set of all lower semicontinuous functions f ∈ B(M) which satisfy the following assumption. (A) There exist a program {zj }∞ j =0 ⊂ K, constants c(f ) > 0, γ (f ) > 0, and μ(f ) ∈ R 1 and an open set Vf ⊂ M in the relative topology such that:  −1 f f (i) | N j =0 [f (zj , zj +1 ) − μ(f )]| ≤ c(f ) for all integers N ≥ 1. (ii) For each integer N ≥ 1 and each program {zj }N j =1 ⊂ K f

N−1 





f zj , zj +1 − μ(f ) ≥ −c(f ).

j =0 2 1 (iii) [ω({zj }∞ j =0 )] ⊂ Vf and the restriction f |Vf : Vf → R is a continuous function. (iv) For each integer j ≥ 0 and each (x, y) ∈ M satisfying    f f d1 (x, y) , zj , zj +1 ≤ γ (f ) f

f

f

the inclusions (x, zj +1 ), (zj , y) ∈ M hold.

f

(v) For each integer j ≥ 0 and each (x, y) ∈ M satisfying d(x, zj +1 ) ≤ γ (f ) f

the inclusion (zj , x) ∈ M holds. (vi) For each x, y, z ∈ ω({zj }∞ j =0 ) which satisfy (x, y) ∈ M and d(x, z) ≤ γ (f ) the inclusion (z, y) ∈ M holds. f

Let f ∈ B(M). Clearly, (A)(iii) holds if the function f is continuous. Assumptions (A)(iv)–(A)(vi) hold if there is γ0 > 0 such that all the closed balls in f 2 K × K with radius γ0 and with centers belonging to M ∩ [cl({zj }∞ j =0 )] are contained in M. Assumptions (A)(i) and (A)(ii) imply that for any natural number T T −1 

   f f f zj , zj +1 ≤ inf U f (T , y, z) : y, z ∈ K + 2c(f ).

j =0

It means that the program {zj }∞ j =0 is an approximate (up to 2c(f )) solution of problem (P) with any natural number T . It should be mentioned that a program which possesses this property usually exists for optimal control problems with the turnpike property [45]. Let A be either Mreg or Mreg ∩ C(M). ¯ the closure of A in B(M) and consider the topological space A ¯ with Denote by A the relative topology. ¯ which is a countable intersection In [53] we proved the existence of a set F ⊂ A ¯ of open everywhere dense sets in A and for which the following theorem is true. f

68

4 Optimal Control Problems with Nonsingleton Turnpikes

Theorem 4.1 Let f ∈ F. Then there exists a nonempty closed set H (f ) ⊂ M such that for each S > 0 and each > 0 there exist a neighborhood U of f in B(M) and integers l, L, Q ≥ 1 such that the following assertion holds: For each g ∈ U, each integer T ≥ L + lQ. and each program {xi }Ti=0 ⊂ K which satisfies T −1 

g (xi , xi+1 ) ≤ inf{U g (T , y, z) : y, z ∈ K} + S,

i=0 q

q

there exist sequences of integers {bi }i=1 , {ci }i=1 ⊂ [0, T ] such that q ≤ Q, 0 ≤ ci − bi ≤ l, i = 1, . . . q, dist(H (f ), {(xi , xi+1 ) : i = p, . . . , p + L − 1} ) ≤ q

for each integer p ∈ [0, T − L] \ ∪i=1 [bi , ci ]. This result shows that any f ∈ F possesses the turnpike property with the turnpike H (f ) which is not necessarily a singleton and that this turnpike property is stable under perturbations of the objective function. Example 4.2 Suppose that M = K × K. It follows from the results of Leizarowitz [22] that C(K × K) ⊂ Mreg . Then there exists a set F ⊂ C(K × K) which is a countable intersection of open everywhere dense subsets of C(K × K) and for which Theorem 4.1 is valid. Let x0 , y0 ∈ K and x0  = y0 . Define a function g ∈ C(K × K) by g(x, y) = d(x, x0 )d(x, y0 ), x, y ∈ K. Clearly, the function g does not have the turnpike property. It means that Theorem 4.1 cannot be improved in principle. Example 4.3 Let K be a compact convex subset of R n and let M ⊂ K × K be a convex compact subset of R n × R n with an nonempty interior denoted by int(M). We assume that d(x, y) = x − y, x, y ∈ K where  ·  is the Euclidean norm in R n induced by an inner product ·, · . Denote by Mconv the set of all convex lower semicontinuous functions f ∈ B(M) for which there exists (zf , zf ) ∈ int(M) such that f (z, z) ≥ f (zf , zf ) for all (z, z) ∈ M.

(4.6)

Let f ∈ Mconv . It is a well-known fact of convex analysis [37] that there exists η ∈ R n such that f (x, y) ≥ f (zf , zf ) + < η, x − y > for all (x, y) ∈ M.

(4.7)

It follows from (4.6) and (4.7) that Mconv ⊂ Mreg . ¯ be a countable intersection Let A be either Mreg or Mreg ∩ C(M), and let F ⊂ A ¯ of open everywhere dense subsets of A for which Theorem 4.1 is true. It follows

4.2 The Turnpike Property

69

from the construction of the set F (see [53]) that the set F ∩ cl(Mconv ∩ A) is a countable intersection of open everywhere dense sets in cl(Mconv ∩ A). Therefore, a generic function f ∈ cl(Mconv ∩ A) has the turnpike property. f Let f ∈ Mreg . There exist a program {zj }∞ j =0 ⊂ K, constants c(f ) > 0, γ (f ) > 0, and μ(f ) ∈ R 1 , and an open set Vf ⊂ M in the relative topology such that assumption (A) holds. A program {yj }∞ j =0 ⊂ K is called (f )-good [22, 45] if  ⎫ ⎧  ⎬ ⎨N−1   [f (yj , yj +1 ) − μ(f )] : N = 1, 2, . . . < ∞. sup  ⎭ ⎩  j =0

We can easily deduce the following result. ∞ Proposition 4.4 Let {yj }∞ j =0 ⊂ K be a program. Then either {yj }j =0 is an (f )-good program or N−1  [f (yj , yj +1 ) − μ(f )] → ∞ as N → ∞. j =0

{yj }∞ j =0

Moreover, if is an (f )-good program, then there exists an integer N0 ≥ 1 such that for each pair of integers q > p ≥ N0 , q−1 

[f (yj , yj +1 ) − μ(f )] ≤ c(f ) + 1.

j =p

We say that f possesses the asymptotic turnpike property or, briefly, (ATP) if for

∞ each pair of (f )-good programs {yj }∞ j =0 , {yj }j =0 ,     ∞ ∞ Ω yj j =0 = Ω yj j =0 . In this case there exists a nonempty compact set H (f ) ⊂ M depending only on f such that   ∞ Ω yj j =0 = H (f ) for any (f )-good program {yj }∞ j =0 . In this chapter we prove the following result which shows that the turnpike phenomenon holds and is stable under perturbation of the objective function f if it possesses the asymptotic turnpike property. Theorem 4.5 Assume that f ∈ Mreg and that there exists a nonempty closed set H (f ) ⊂ M such that   ∞ Ω yj j =0 = H (f ) for any (f )-good program {yj }∞ j =0 . Let , M > 0. Then there exist a neighborhood U of f in B(M) and integers l, L, Q ≥ 1 such that the following assertion holds:

70

4 Optimal Control Problems with Nonsingleton Turnpikes

For each integer T ≥ L + lQ, each gi ∈ U, i = 0, . . . , T − 1 and each program {xi }Ti=0 which satisfies T −1 



 −1 gi (xi , xi+1 ) ≤ inf U {gi }Ti=0 , y, z : y, z ∈ K + M

i=0 q

q

there exist sequences of integers {bi }i=1 , {ci }i=1 ⊂ [0, T ] such that q ≤ Q, 0 ≤ ci − bi ≤ l, i = 1, . . . q, dist(H (f ), {(xi , xi+1 ) : i = p, . . . , p + L − 1}) ≤ q

for each integer p ∈ [0, T − L] \ ∪i=1 [bi , ci ].

4.3

Preliminaries

Let f ∈ Mreg . There exist a program {zj }∞ j =0 ⊂ K, constants c(f ) > 0, γ (f ) > 0 1 and μ(f ) ∈ R and an open set Vf ⊂ M in the relative topology such that assumption (A) holds. We can easily deduce the following three results. f

Proposition 4.6 Assume that x, y ∈ K, q ≥ 1 is an integer, and that U f (q, x, y) < q +∞. Then there exists a program {zj }j =0 ⊂ K such that z0 = x, zq = y,

q−1 

f (zj , zj +1 ) = U f (q, x, y).

j =0

Proposition 4.7 For each integer q ≥ 1 the function (x, y) → U f (q, x, y), x, y ∈ K is lower semicontinuous. q A program xj , j ∈ I , where I is either Z or Zp or Zp (with p, q ∈ Z satisfying p < q) is (f )-minimal [6, 24, 41, 45] if for each n, m ∈ I satisfying m < n n−1 

f (xj , xj +1 ) = U f (n − m, xm , xn ).

j =m

Proposition 4.8 1. For each (x, y) ∈ M and each (x, ˜ y) ˜ ∈ Ω({zi }∞ i=0 ) satisfying ˜ y)) ˜ < γ (f ) the inclusions (x, y), ˜ (x, ˜ y) ∈ M hold. d1 ((x, y), (x, f 2. For each (x, y) ∈ M and each (x, ˜ y) ˜ ∈ Ω({zi }∞ ˜ x) < γ (f ) i=0 ) satisfying d(y, the inclusion (x, ˜ x) ∈ M holds. f

Proposition 4.9 Let {zi }∞ i=0 ⊂ K be an (f )-good program and let

 (y0 , y1 ) ∈ Ω {zi }∞ i=0 .

4.3 Preliminaries

71

Then there exists a program {yi }∞ i=−∞ ⊂ K such that



 yj , yj +1 ∈ Ω {zi }∞ i=0 for all j ∈ Z, q−1 



f yj , yj +1 − μ(f ) ≤ c(f ) + 1 for all integers q > p.

(4.8) (4.9)

j =p ∞ ∞ Moreover, if Ω({zi }∞ i=0 ) ⊂ Ω({zi }i=0 ), then {yi }i=−∞ is an (f )-minimal program. f

Proof There exists a subsequence (zik , zik +1 ) → (y0 , y1 ) as k → ∞.

(4.10)

By Proposition 4.4 we may assume without loss of generality that q−1 



f zj , zj +1 − μ(f ) ≤ c(f ) + 1 for all nonnegative integers q > p. (4.11) j =p

For each integer k ≥ 1 we set yjk = zj +ik for all integers j ≥ −ik .

(4.12)

k

There exist a subsequence of programs {yi j : i ≥ −ikj }, j = 1, 2, . . . and a sequence {yi }∞ i=−∞ such that k

yi j → yi as j → ∞ for each integer i.

(4.13)

It is easy to see that the sequence {yi }∞ i=−∞ is a program. Equations (4.8) and (4.9) follow from (4.11), (4.12), (4.13), and the lower semicontinuity of f . Assume that  ∞   ∞  f ⊂ Ω zi . (4.14) Ω zi i=0

i=0

{yi }∞ i=−∞

is an (f )-minimal program. We will show that Let us assume the contrary. Then there exist r > 0 and integers p < q such that q−1 

f (yi , yi+1 ) > U f (q − p, yp , yq ) + r.

(4.15)

i=p q

By Proposition 4.6 there exists a program {xi }i=p ⊂ K such that xi = yi , i = p, q,

q−1 

f (xi , xi+1 ) = U f (q − p, yp , yq ).

(4.16)

i=p

By assumption (A)(iii), (4.14), and (4.8) there exists ∈ (0, 8−1 γ (f ))

(4.17)

72

4 Optimal Control Problems with Nonsingleton Turnpikes

such that for each i ∈ {p − 1, . . . , q} and each (x, y) ∈ M satisfying d1 ((x, y), (yi , yi+1 )) ≤ 4

(4.18)

the inequality |f (x, y) − f (yi , yi+1 )| ≤ 8−1 r(q − p + 2)−1 holds. Since {zi }∞ i=0 is an (f )-good program there exists an integer N0 ≥ 1 such that n−1 

f (zi , zi+1 ) ≤ U (n − m, zm , zn ) + 32−1 r for all integers n > m ≥ N0 . (4.19)

i=m

By (4.12) and (4.13) there exists an integer k ≥ 1 such that ik − |p| − |q| ≥ 2N0 + 2, d(zj +ik , yj ) ≤ 8−1 , j = p − 1, . . . , q + 1.

(4.20)

It follows from (4.20), (4.8), (4.14), (4.17), and Proposition 4.8 that (zp−1+ik , yp ), (yq , zq+1+ik ) ∈ M.

(4.21)

We define vp−1+ik = zp−1+ik vj = xj −ik , j = p + ik , . . . , q + ik , vq+1+ik = zq+1+ik . (4.22) Equations (4.21), (4.16), and (4.22) imply that {vj : j = p − 1 + ik , . . . , q + 1 + ik } is a program. We will estimate 

q+ik

[f (zj , zj +1 ) − f (vj , vj +1 )].

j =p−1+ik

By (4.22), (4.19), and (4.20), 

q+ik

[f (zj , zj +1 ) − f (vj , vj +1 )] ≤ 32−1 r.

(4.23)

j =p−1+ik

On the other hand it follows from (4.22), (4.16), and (4.15) that 

q+ik

[f (zj , zj +1 ) − f (vj , vj +1 )] = f (zp−1+ik , zp+ik ) − f (zp+ik −1 , yp )

j =p−1+ik q+ik −1

+



j =p+ik

f (zj , zj +1 ) −

q−1 

f (yj , yj +1 ) +

j =p

q−1  j =p

f (yj , yj +1 ) −

q−1 

f (xj , xj +1 )

j =p

+f (zq+ik , zq+ik +1 ) − f (yq , zq+ik +1 ) ≥ r + f (zp−1+ik , zp+ik ) − f (zp+ik −1 , yp ) +f (zq+ik , zq+ik +1 ) − f (yq , zq+ik +1 ) +

q−1  j =p

[f (zj +ik , zj +ik +1 ) − f (yj , yj +1 )].

4.3 Preliminaries

73

By the equation above, (4.20), (4.21), and the definition of (see (4.17), (4.18)), 

q+ik

[f (zj , zj +1 ) − f (vj , vj +1 )] ≥ r − 2(q − p + 2)8−1 r(q − p − 2)−1 ≥ 2−1 r.

j =p−1+ik

This contradicts (4.23). Therefore {yi }∞ −∞ is an (f )-minimal trajectory. The proposition is proved. Assume that f possesses (ATP) and that H (f ) is a nonempty closed subset of M such that ∞ H (f ) = Ω({zi }∞ i=0 ) for any(f ) − good program{zi }i=0 .

(4.24)

By Proposition 4.9 there exists an (f )-minimal program {xj }∞ j =−∞ ⊂ K such that f

f

f

(xj , xj +1 ) ∈ H (f ) for each integerj ,

(4.25)

q−1      f f f xj , xj +1 − μ(f ) ≤ c(f ) + 1 for each pair of integers q > p. (4.26) j =p

It follows from (ATP), assumption (A) and (4.26) that &  ' &  ' ∞ ∞ f f Ω xj = H (f ) = Ω zj . j =0

j =0

(4.27)

Set f

H0 = {x ∈ K : there exists y ∈ K such that (x, y) ∈ H (f )}.

(4.28)

f

For any x ∈ H0 we define N−1   [f (yi , yi+1 ) − μ(f )] : π f (x) = inf lim inf N→∞

i=0

 f a program {yi }∞ ⊂ H and y = x . 0 i=0 0

(4.29)

f

Proposition 4.10 π f : H0 → R 1 is a continuous function. Proof It follows from assumption (A)(ii), Proposition 4.9, and (4.25)–(4.28) that f

π f (x) ∈ [ − c(f ), c(f ) + 1], x ∈ H0 . f

By assumption (A)(vi) for each x, y, z ∈ H0 which satisfy (x, y) ∈ M, d(z, x) ≤ γ (f )

(4.30)

74

4 Optimal Control Problems with Nonsingleton Turnpikes

the inclusion (z, y) ∈ M holds. By assumption (A)(iii) and (4.27) there exists δ0 ∈ f f (0, 2−1 γ (f )) such that for each (x1 , y1 ) ∈ H0 ×H0 and each (x2 , y2 ) ∈ M satisfying d1 ((x1 , y1 ), (x2 , y2 )) ≤ δ0 the inclusion (x2 , y2 ) ∈ Vf holds. Set     f f Q = (x, y) ∈ M : d1 (x, y), H0 × H0 ≤ δ0 .

(4.31)

It follows from the choice of δ0 that Q ⊂ Vf . Let be a positive number. Since f |Vf : Vf → R 1 is a continuous function (see assumption (A)(iii)) there exists δ ∈ (0, 2−1 δ0 ) such that for each (x1 , y1 ), (x2 , y2 ) ∈ Q satisfying d1 ((x1 , y1 ), (x2 , y2 )) ≤ δ

(4.32)

the following inequality holds: |f (x1 , y1 ) − f (x2 , y2 )| ≤ 8−1 .

(4.33)

Suppose that f

x, y ∈ H0 , d(x, y) ≤ δ.

(4.34)

We show that |π f (x) − π f (y)| ≤ . There exists a program {xj }∞ =1 ⊂ H0 such that f

x0 = x, π f (x) ≥ lim inf N→∞

N−1 

[f (xi , xi+1 ) − μ(f )] − 8−1 .

(4.35)

i=0

It follows from (4.35), (4.34), (4.30), and the choice of δ0 , δ (see (4.31)–(4.33)) that (y, x1 ) ∈ M, |f (y, x1 ) − f (x, x1 )| ≤ 8−1 .

(4.36)

y0 = y, yi = xi , i = 1, 2, . . . .

(4.37)

Define

By (4.36), (4.37), (4.34), and (4.35), {yi }∞ i=0 is a program and π f (y) ≤ lim inf N→∞

≤ lim inf N →∞

N −1 

N−1 

[f (yi , yi+1 ) − μ(f )]

i=0

[f (xi , xi+1 ) − μ(f )] + 8−1 ≤ π f (x) + 4−1 .

i=0

This completes the proof of the proposition.

4.3 Preliminaries

75

Proposition 4.11 Define θ f (x, y) = f (x, y) − μ(f ) + π f (y) − π f (x) f M∩(H0

f ×H0 ). Then θ f f

f M∩(H0

(4.38)

f ×H0 )

for each (x, y) ∈ : → R 1 is a continuous nonnegative function such that θ (x, y) = 0 for each (x, y) ∈ H (f ). Proof The continuity of θ f follows from Proposition 4.10, (4.27), (4.28), and assumption (A)(iii). It follows from the definition of π f , θ f that θ f (x, y) ≥ 0 f

f

for each (x, y) ∈ (H0 × H0 ) ∩ M. Let (x, y) ∈ H (f ). We show that θ f (x, y) = 0. By (4.25) and (4.26) for each integer N ≥ 1, c(f ) + 1 ≥

  N−1        f f  f f f f θ f xi , xi+1 −π f xN +π f x0 , f xi , xi+1 − μ(f ) =

N−1  i=0

i=0

θ

f

f f (xi , xi+1 )

→ 0 as i → ∞.

(4.39)

It follows from the continuity of θ f , (4.39) and (4.27) that θ f (x, y) = 0. The proposition is proved. Proposition 4.12 Let j ∈ Z. Then f

π f (xj ) = lim inf N→∞

N−1 

   f f f xi , xi+1 − μ(f ) ,

(4.40)

i=j f

sup{π f (y) : y ∈ H0 } = 0. Proof It follows from Propositions 4.10 and 4.11, (4.25), (4.27), and (4.28) that lim inf N→∞

N −1  

        f f f f f xi , xi+1 − μ(f ) = lim inf π f xj − π f xN N→∞

i=j f

f

= π f (xj ) − sup{π f (y) : y ∈ H0 }.

(4.41)

Assume that a program {yi }∞ i=0 ⊂ H0 , f

f

y0 = xj , lim inf N→∞

N−1 

 

f f (yi , yi+1 ) − μ(f ) ≤ π f xj + 1.



i=0

By Proposition 4.11 and (4.42) lim inf N→∞

N −1  i=0

 

f f (yi , yi+1 ) − μ(f ) ≥ lim inf [π f xj − π f (yN )]



N→∞

    f f ≥ π f xj − sup π f (y) : y ∈ H0 .

(4.42)

76

4 Optimal Control Problems with Nonsingleton Turnpikes

Since the equation above holds for each program {yi }∞ i=0 satisfying (4.42) we conclude that       f f f π f xj ≥ π f xj − sup π f (y) : y ∈ H0 . Together with (4.41) this implies (4.40). The proposition is proved.

4.4 Auxiliary Results Assume that f ∈ Mreg possesses (ATP) and that H (f ) is a nonempty closed subset of M such that ∞ H (f ) = Ω({zi }∞ i=0 ) for any (f ) − good program {zi }i=0 .

It follows from assumption (A) and the results of Sect. 4.3 (see (4.25)–(4.27), f Proposition 4.8) that there exist an (f )-minimal program {xi }∞ i=−∞ , constants c(f ) > 1 0, γ (f ) > 0, μ(f ) ∈ R , an open set Vf ⊂ M in the relative topology such that the following properties hold: q−1 f f (a) j =p [f (xj , xj +1 ) − μ(f )] ≤ c(f ) for each pair of integers q > p. (b) For each integer N ≥ 1 and each program {zj }N j =0 ⊂ K, N−1 





f zj , zj +1 − μ(f ) ≥ −c(f ).

j =0 2 1 (c) [ω({xi }∞ i=0 )] ⊂ Vf and the restriction f |Vf : Vf → R is a continuous function; (d) For each (x1 , y1 ) ∈ H (f ) and each (x2 , y2 ) ∈ M satisfying f

d1 ((x1 , y1 ), (x2 , y2 )) < γ (f ) the relations (x1 , y2 ), (x2 , y1 ) ∈ M hold. (e) For each (x1 , y1 ) ∈ H (f ) and (x2 , y2 ) ∈ M satisfying d(y1 , x2 ) < γ (f ) the inclusion (x1 , x2 ) ∈ M holds. f (f) For each x, y, z ∈ ω({xi }∞ i=0 ) which satisfy (x, y) ∈ M and d(x, z) ≤ γ (f ) the inclusion (z, y) ∈ M holds. f f f (g) (xj , xj +1 ) ∈ H (f ) for each integer j , Ω({xj }∞ j =0 ) = H (f ). f

Consider the set H0 ⊂ K defined by (4.28) and the continuous functions π f : f f → R 1 and θ f : (H0 × H0 ) ∩ M → [0, ∞) defined by (4.29) and (4.38).

f H0

Lemma 4.13 Let > 0. Then there exists an integer L ≥ 1 such that for each (f )-good program {xi }∞ i=0 ⊂ K, dist(H (f ), {(xi , xi+1 ) : i ∈ [p, p + L]}) ≤ for all sufficiently large integers p.

4.4 Auxiliary Results

77

Proof Let us assume the contrary. Then for each integer N ≥ 1 there exists an (f )-good program {xiN }∞ i=0 ⊂ K such that  



N lim sup dist H (f ), xiN , xi+1 : i ∈ [p, p + N ] ≥ . (4.43) p→∞

By Proposition 4.4, we may assume that for each integer N ≥ 1 and each pair of integers q > p ≥ 0, q−1  N N 

f xi , xi+1 − μ(f ) ≤ c(f ) + 2.

(4.44)

i=p

Properties (c) and (g) and (4.28) imply that there exists an open set V ⊂ M in the relative topology for which f f H0 × H0 ⊂ V and V¯ ⊂ Vf .

(4.45)

It follows from (ATP) that for each integer N ≥ 1 N N Ω({xiN }∞ i=0 ) = H (f ), d1 ((xi , xi+1 ), H (f )) → 0 as i → ∞.

(4.46)

By (4.46), (4.45), (4.28), and property (g) we may assume without loss of generality that for each pair of integers N ≥ 1 and i ≥ 0, N N d1 ((xiN , xi+1 ), H (f )) ≤ 4−1 , (xiN , xi+1 ) ∈ V.

(4.47)

By (4.43) for each integer N ≥ 1 there exists an integer TN ≥ 1 such that N dist(H (f ), {(xiN , xi+1 ) : i ∈ [TN , TN + N ]}) ≥ (7/8).

(4.48)

Together with (4.47) this implies that for each integer N ≥ 1 there exists hN ∈ H (f ) which satisfies N d1 (hN , {(xiN , xi+1 ) : i ∈ [TN , TN + N ]}) ≥ 2−1 .

(4.49)

For each integer N ≥ 1 we define a program {viN }∞ i=0 ⊂ K by N viN = xi+T , i = 0, 1, . . . N

(4.50)

It follows from (4.49) and (4.50) that N d1 (hN , {(viN , vi+1 ) : i = 0, . . . , N }) ≥ 2−1 for each integer N ≥ 1.

(4.51)

We may assume by extracting a subsequence and reindexing that hN → h ∈ H (f ) as N → ∞

(4.52)

and that there exists a sequence {vi∗ }∞ i=0 ⊂ K such that viN → vi∗ as N → ∞ for each integer i ≥ 0.

(4.53)

78

4 Optimal Control Problems with Nonsingleton Turnpikes

Equations (4.51)–(4.53) imply that ∗ d1 (h, (vi∗ , vi+1 )) ≥ 4−1 for each integer i ≥ 0.

(4.54)

On the other hand it follows from the lower semicontinuity of f , (4.44), (4.50), and (4.53) that {vi∗ }∞ i=0 is an (f )-good program. Combined with (ATP) and (4.52) this implies that h ∈ H (f ) = Ω({vi∗ }∞ i=0 ). This contradicts (4.54). The obtained contradiction proves the lemma. Lemma 4.14 Let 0 , M0 > 0 and let l ≥ 1 be an integer such that for each (f )-good program {xi }∞ i=0 ⊂ K, dist(H (f ), {(xi , xi+1 ) : i ∈ [p, p + l]}) ≤ 8−1 0

(4.55)

for all large integers p (the existence of l follows from Lemma 4.13). Then there l exists an integer N ≥ 10 such that for each program {xi }N i=0 ⊂ K which satisfies Nl−1 

f (xi , xi+1 ) ≤ N lμ(f ) + M0

(4.56)

i=0

there exists an integer j0 ∈ [0, N − 8] such that dist(H (f ), {(xi , xi+1 ) : i ∈ [T , T + l]}) ≤ 0

(4.57)

for each integer T ∈ [j0 l, (j0 + 7)l]. Proof Let us assume the contrary. Then for each integer N ≥ 10 there exists a trajectory {xiN }Nl i=0 ⊂ K such that Nl−1 

N f (xiN , xi+1 ) ≤ N lμ(f ) + M0

(4.58)

i=0

and that for each integer j ∈ [0, N − 8] there exists an integer T (j ) ∈ [j l, (j + 7)l] for which N ) : i ∈ [T (j ), T (j ) + l]}) > 0 . dist(H (f ), {(xiN , xi+1

(4.59)

kl There exist a subsequence of programs {xiNk }N i=0 , k = 1, 2, . . . and a sequence ∞ {yi }i=0 ⊂ K such that

xiNk → yi as k → ∞ for each integer i ≥ 0.

(4.60)

It follows from (4.58) and property (b) that for each integer N ≥ 1 and each pair of integers q, p ∈ [0, N ] satisfying p < q, q−1  i=p

N ) ≤ (q − p)μ(f ) + M0 + 2c(f ). f (xiN , xi+1

4.4 Auxiliary Results

79

Together with (4.60) and lower semicontinuity of f this implies that {yi }∞ i=0 is an (f )-good program. Therefore by the definition of l (see (4.55)) there exists an integer Q ≥ 1 such that for each integer T ≥ Ql, dist(H (f ), {(yi , yi+1 ) : i ∈ [T , T + l]}) ≤ 8−1 0 .

(4.61)

By (4.60) there exists an integer k such that k ≥ 3Q + 30, d(xiNk , yi ) ≤ 64−1 0 , i = 0, . . . , (2Q + 20)l.

(4.62)

It follows from (4.61) and (4.62) that for each integer T ∈ [Ql, (Q + 10)l], Nk ) : i ∈ [T , T + l]}) ≤ 4−1 0 . dist(H (f ), {(xiNk , xi+1

(4.63)

kl On the other hand it follows from the definition of {xiNk }N i=0 and (4.62) that there exists an integer T (Q) ∈ [Ql, (Q + 7)l] for which

Nk ) : i ∈ [T (Q), T (Q) + l]}) > 0 . dist(H (f ), {(xiNk , xi+1

This contradicts (4.63) which holds for each integer T ∈ [Ql, (Q + 10)l]. The obtained contradiction proves the lemma. Lemma 4.14 and property (a) imply the following result. Lemma 4.15 Let 0 , M0 > 0 and let l ≥ 1 be an integer such that each (f )-good program {xi }∞ i=0 ⊂ K satisfies (4.55) for all sufficiently large integers p. Then there exist an integer N ≥ 10 and a neighborhood U of f in B(M) such that for each l gi ∈ U, i = 0, . . . , N l − 1 and each program {xi }N i=0 ⊂ K which satisfies Nl−1 

gi (xi , x+1 ) ≤ inf{U ({gi }Nl−1 i=0 , y, z) : y, z ∈ K} + M0

i=0

there exists an integer j0 ∈ [0, N − 8] such that (4.57) holds for each integer T ∈ [j0 l, (j0 + 7)l]. Lemma 4.16 Let 0 , M0 > 0. Then there exist an integer q0 ≥ 2 and δ > 0 such that for each integer T ≥ q0 , each program {xi }Ti=0 which satisfies T −1 

f (xi , xi+1 ) ≤ T μ(f ) + M0 + δT

(4.64)

i=0

the following inequality holds: min{d1 ((xi , xi+1 ), H (f )) : i = 1, . . . , T − 1} ≤ 0 .

(4.65)

Proof By Lemma 4.14, there exist integers l0 ≥ 1 and N0 ≥ 10 such that for each 0 l0 program {yi }N i=0 ⊂ K satisfying N 0 l0 −1 i=0

f (yi , yi+1 ) ≤ N0 l0 μ(f ) + 1

(4.66)

80

4 Optimal Control Problems with Nonsingleton Turnpikes

there exists an integer j0 ∈ [0, N0 − 8] such that dist(H (f ), {(yi , yi+1 ) : i ∈ [τ , τ + l0 ]}) ≤ 0

(4.67)

for each integer τ ∈ [j0 l0 , (j0 + 7)l0 ]. Choose a positive number δ < (2N0 l0 )−1

(4.68)

q0 > (2N0 l0 )[4 + M0 + N0 l0 (f  + |μ(f )|)].

(4.69)

and a natural number

Assume that an integer T ≥ q0 and that a program {xi }Ti=0 satisfies (4.64). In order to complete the proof it is sufficient to show that (4.65) holds. Assume the contrary. Then d1 ((xi , xi+1 ), H (f )) > 0 , i = 1, . . . , T − 1.

(4.70)

There is a natural number p such that pN0 l0 ≤ T < (p + 1)N0 l0 .

(4.71)

In view of (4.69) and (4.71), p ≥ 4.

(4.72)

By (4.70) and the choice of l0 , N0 (see (4.66), (4.67)), for each integer j ∈ [0, p − 1], (j +1)N0 l0 −1



f (xi , xi+1 ) > N0 l0 μ(f ) + 1.

i=j N0 l0

It follows from (4.64), (4.71), and (4.73) that δT + M0 ≥

T −1 

(f (xi , xi+1 ) − μ(f ))

i=0

= +

p−1 



j =0

(j +1)N0 l0 −1

(



f (xi , xi+1 ) − μ(f ))

i=j N0 l0

{f (xi , xi+1 ) − μ(f ) : i is an integer and pN0 l0 ≤ i < T }

≥ p − N0 l0 (f  + |μ(f )|) ≥ T (N0 l0 )−1 − 1 − N0 l0 (f  + |μ(f )|). Together with (4.68) this implies that q0 (2N0 l0 )−1 ≤ T (2N0 l0 )−1 ≤ T ((N0 l0 )−1 − δ) ≤ 1 + M0 + N0 l0 (f  + |μ(f )|)

(4.73)

4.4 Auxiliary Results

81

and q0 ≤ (2N0 l0 )[1 + M0 + N0 l0 (f  + |μ(f )|)]. This contradicts (4.69). The contradiction we have reached completes the proof of Lemma 4.16. Lemma 4.17 Let 0 , M0 > 0. Then there exist an integer Q ≥ 1 and a neighborhood U of f in B(M) such that for each integer q ≥ 8Q + 8, each gi ∈ U, q i = 0, . . . , q − 1, each program {xi }i=0 ⊂ K which satisfies q−1 

q−1

gi (xi , xi+1 ) ≤ inf{U ({gi }i=0 , y, z) : y, z ∈ K} + M0

(4.74)

i=0

and each integer p ∈ [0, q − Q] the following inequality holds: min{d1 ((xi , xi+1 ), H (f )) : i ∈ [p, p + Q − 1]} ≤ 0 .

(4.75)

Proof Fix a number δ0 ∈ (0, 8−1 γ (f )) ∩ (0, 1)

(4.76)

(recall γ (f ) in properties (d)–(f)) such that {(x1 , x2 ) ∈ M : d(xk , ω({xi }∞ i=0 )) ≤ 8δ0 , k = 1, 2} ⊂ Vf . f

(4.77)

By property (c) and (4.77) we may assume without loss of generality that 0 ∈ (0, 8−1 δ0 )

(4.78)

|f (x1 , x2 ) − f (x3 , x4 )| ≤ 4−1

(4.79)

and

for each (x1 , x2 ), (x3 , x4 ) ∈ M which satisfy d(xk , ω({xi }∞ i=0 )) ≤ 8δ0 , k = 1, 2, 3, 4, d1 ((x1 , x2 ), (x3 , x4 )) ≤ 4 0 . f

(4.80)

It follows from property (g) that there exist integers N1 , N2 ≥ 1 such that dist({(xi , xi+1 ) : i = 0, . . . , N1 − 1}, H (f )) ≤ 8−1 0 , f

f

dist({(xi , xi+1 ) : i = 2N1 + 4, . . . , 2N1 + 3 + N2 }, H (f )) ≤ 8−1 0 . f

f

(4.81)

By Lemma 4.16 there exists δ1 ∈ (0, 16−1 ) and a natural number Q0 > 4 such that the following property holds:

(4.82)

82

4 Optimal Control Problems with Nonsingleton Turnpikes

(Pi) for each integer τ ≥ Q0 and each program {xi }τi=0 which satisfies τ −1 

f (xi , xi+1 ) ≤ τ μ(f ) + 2δ1 τ + M0 + 4 + 3c(f ) + 7(f  + 1) + |μ(f )|

i=0

we have min{d1 ((xi , xi+1 ), H (f )) : i = 1, . . . , τ − 1} ≤ 0 . We may assume without loss of generality that δ1 Q0 < 1.

(4.83)

Q > 4(Q0 + N1 + N2 + 4).

(4.84)

U = {g ∈ B(M) : ||g − f || < 2−1 δ1 }.

(4.85)

Fix an integer

Set

Assume that an integer q ≥ 8Q + 8, gi ∈ U, i = 0, . . . , q − 1 and that a program q {xi }i=0 ⊂ K satisfies (4.74). We show that for each pair of integers k, s ∈ [0, q] satisfying s − k ≥ Q0 , k(q − s) = 0

(4.86)

the following inequality holds: min{d1 ((xi , xi+1 ), H (f )) : i ∈ [k, s − 1]} ≤ 0 .

(4.87)

Let us assume the contrary. Then there exist integers k, s ∈ [0, q] which satisfy (4.86) and such that min{d1 ((xi , xi+1 ), H (f )) : i ∈ [k, s − 1]} > 0 .

(4.88)

We may assume without loss of generality that for each integer p ∈ [0, q − 1] ∩ {k − 1, s}

(4.89)

the following inequality holds: d1 ((xp , xp+1 ), H (f )) ≤ 0 .

(4.90)

By (4.81) and (4.90) for any p ∈ [0, q − 1] ∩ {k − 1, s} there exists an integer j (p) ∈ {0, , . . . , N1 − 1} for which d1 ((xp , xp+1 ), (xj (p) , xj (p)+1 )) ≤ 0 (1 + 6−1 ). f

f

(4.91)

4.4 Auxiliary Results

83

It follows from (4.91), (4.78), (4.76), and properties (d) and (g) that for any p ∈ [0, q − 1] ∩ {k − 1, s}     f f xp , xj (p)+1 , xj (p) , xp+1 ∈ M. (4.92) By the definition of 0 (see (4.78)–(4.80)), (4.85), (4.92), and (4.91), for any p ∈ [0, q − 1] ∩ {k − 1, s} the following equation holds:       f f f f gp xj (p) , xj (p)+1 , gp xp , xj (p)+1 , gp xj (p) , xp+1 ∈ [gp (xp , xp+1 ) − 4, gp (xp , xp+1 ) + 4].

(4.93)

There are three cases: (i) k = 0, s = q; (ii) k = 0, s < q; (iii) k > 0, s = q. In q all these cases we define a program {yi }i=0 ⊂ K and estimate q−1 

[f (yi , yi+1 ) − f (xi , xi+1 )].

i=0

In the case (i) we set f

yi = xi , i = 0, . . . , q.

(4.94)

yi = xi+j (s)−s , i = 0, . . . , s, yi = xi , i = s + 1, . . . , q.

(4.95)

In the case (ii) we set f

In the case (iii) we set f

yi = xi , i = 0, . . . , k − 1, yi = xi+j (k−1)−k+1 , i = k, . . . , q.

(4.96)

q

By (4.92), the program {yi }i=0 ⊂ K is well defined. It follows from (4.74), (4.93)– (4.96), (4.85), and the inclusion gi ∈ U, i = 0, . . . , q − 1 that M0 ≥

q−1 

[gi (xi , xi+1 ) − gi (yi , yi+1 )] ≥

s−1 

[gi (xi , xi+1 ) − gi (yi .yi+1 )] − 4

i=k

i=0

≥

s−1 

[f (xi , xi+1 ) − f (yi , yi+1 )] − (s − k)δ1 − 4.

(4.97)

i=k

Property (a) and (4.94)–(4.96) imply that s−1  i=k

f (yi , yi+1 ) ≤ (s − k)μ(f ) + c(f ).

(4.98)

84

4 Optimal Control Problems with Nonsingleton Turnpikes

By (4.97) and (4.98), s−1 

f (xi , xi+1 ) ≤ M0 + (s − k)δ1 + 4 + (s − k)μ(f ) + c(f ).

i=k

It follows from the inequality above, (4.86) and property (Pi) that min{d1 ((xi , xi+1 ), H (f )) : i = k, . . . , s − 1} ≤ 0 . This contradicts (4.88). The obtained contradiction proves the following assertion: (B) Eq. (4.87) holds for each pair of integers k, s ∈ [0, q] which satisfy (4.86). Assume that an integer p ∈ [0, q − Q]. We show that (4.75) holds. Let us assume the contrary. Then d1 ((xi , xi+1 ), H (f )) > 0 for each integer i ∈ [p, p + Q − 1].

(4.99)

For each integer j ∈ [0, q − 1] we set Dj = {i ∈ [0, j ] ∩ Z : d1 ((xi , xi+1 ), H (f )) ≤ 0 }, Cj = {i ∈ [j , q − 1] ∩ Z : d1 ((xi , xi+1 ), H (f )) ≤ 0 }.

(4.100)

It follows from assertion (B), (4.99), (4.86), and (4.87) that p > 0, Dp−1  = ∅, p + Q < q, Cp+Q  = ∅.

(4.101)

k = sup{i : i ∈ Dp−1 }, s = inf{i : i ∈ Cp+Q }.

(4.102)

Set

By (4.99)–(4.102) 0 ≤ k ≤ p − 1, p + Q ≤ s ≤ q − 1, d1 ((xi , xi+1 ), H (f )) ≤ 0 , i = k, s, d1 ((xi , xi+1 ), H (f )) > 0 , i ∈ [k + 1, . . . , s − 1].

(4.103)

Set m = inf{i : i ∈ Dk }, n = sup{i : i ∈ Cs }.

(4.104)

Assertion (B), (4.104) and (4.100) imply that m ≤ Q0 , n ≥ q − Q 0 .

(4.105)

It follows from (4.104) and (4.100) that d1 ((xi , xi+1 ), H (f )) ≤ 0 , i = m, n.

(4.106)

4.4 Auxiliary Results

85

By (4.103), (4.106), and (4.81) there exist integers i(k) ∈ [0, N1 − 1], i(s) ∈ [2N1 + 4, . . . , 2N1 + 3 + N2 ], i(n) ∈ [2N1 + 4, . . . , 2N1 + 3 + N2 ]

(4.107)

d1 ((xl , xl+1 ), (xi(l) , xi(l)+1 )) ≤ 0 + 6−1 0 , l = k, s, n.

(4.108)

such that f

f

It follows from (4.108), (4.78), (4.76), and properties (d) and (g) that f

f

f

(xk , xi(k)+1 ) ∈ M, (xi(s) , xs+1 ) ∈ M, (xn , xi(n)+1 ) ∈ M.

(4.109)

q

We define a program {yi }i=0 and estimate q−1 

[gi (xi , xi+1 ) − gi (yi , yi+1 )].

i=0

Set f

yi = xi , i = 0, . . . , k, yi = xi−k+i(k) , i = k + 1, . . . , k + i(s) − i(k), τ = k + i(s) − i(k).

(4.110)

If n = s, then we set f

yi = xi−k+i(k) , i = k + i(s) − i(k) + 1, . . . , q.

(4.111)

Otherwise we put yi = xi+s−τ , i = k + i(s) − i(k) + 1, . . . , n + k + i(s) − i(k) − s,

(4.112)

f

yi = xi+i(n)−n−τ +s , i = n + k + i(s) − i(k) − s + 1, . . . , q. q

It follows from (4.109)–(4.112), (4.107), (4.103), and (4.104) that {yi }i=0 is a program. By (4.74), (4.85), (4.103), (4.110)–(4.112), (4.83), and the inclusion gi ∈ U, i = 0, . . . , q − 1, M0 ≥

q−1 

(gi (xi , xi+1 ) − gi (yi , yi+1 )) =

i=0

(gi (xi , xi+1 ) − gi (yi , yi+1 ))

i=k

≥ −||f || − 1 +

s−1 

gi (xi , xi+1 ) +

i=k+1

− (||f || + 1 +

q−1 

q−1  i=k+1

gi (yi , yi+1 )).

q−1 

gi (xi , xi+1 )

i=s

(4.113)

86

4 Optimal Control Problems with Nonsingleton Turnpikes

It follows from (4.85) and the inclusion gi ∈ U, i = 0, . . . , q − 1 that s−1 

s−1 

gi (xi , xi+1 ) ≥ −δ1 (s − k − 1) +

i=k+1

f (xi , xi+1 ).

(4.114)

i=k+1

q−1 q−1 We estimate i=s gi (xi , xi+1 ) and i=k+1 gi (yi , yi+1 ). There are two cases: (1) n = s; (2) n > s. Consider the case (1). Then it follows from (4.85), (4.105), property (b), and the inclusion gi ∈ U, i = 0, . . . , q − 1 that q−1 

gi (xi , xi+1 ) ≥ −δ1 Q0 +

i=s

q−1 

f (xi , xi+1 ) ≥ −δ1 Q0 − c(f ) + (q − s)μ(f ).

i=s

(4.115) By (4.110), (4.111), (4.85), the inclusion gi ∈ U, i = 0, . . . , q − 1, and properties (a) and (g), q−1 



q−k+i(k)−1

gi (yi , yi+1 ) ≤ δ1 (q − k − 1) +

i=k+1

f

f

f (xi , xi+1 )

i=i(k)+1

≤ δ1 (q − k − 1) + (q − k − 1)μ(f ) + c(f ).

(4.116)

Together with (4.115) this implies that q−1 

gi (xi , xi+1 ) −

i=s

q−1 

gi (yi , yi+1 ) ≥ (k + 1 − s)μ(f ) − δ1 Q0 − 2c(f )

i=k+1

− δ1 (q − k − 1).

(4.117)

Combining this equation with (4.113), (4.114), and the equality n = s we obtain that M0 ≥ −2f  − 2 − δ1 (s − k − 1) +

s−1 

f (xi , xi+1 )

i=k+1

+μ(f )(k + 1 − s) − δ1 Q0 − 2c(f ) − δ1 (s − k − 1) − δ1 (q − n).

(4.118)

Consider the case (2). It follows from (4.110), (4.85), (4.107), (4.103), (4.105), (4.112), (4.83), the inclusion gi ∈ U, i = 0, . . . , q − 1, and properties (a) and (g) that q−1  i=k+1

gi (yi , yi+1 ) ≤

τ −1 

gi (yi , yi+1 ) + f  + 1 +

i=τ +1

i=k+1

+

n−s+τ 

q−1  i=n−s+τ +1

gi (yi , yi+1 )

gi (yi , yi+1 )

4.4 Auxiliary Results

87 i(s)−1 

≤ (i(s) − i(k))δ1 +

f

f

f (xi , xi+1 ) + f  + 1 +

i=i(k)+1

n 

gi (xi , xi+1 )

i=s+1

q+i(n)−n−τ +s−1



+2 + 2f  +

f

f

f (xi , xi+1 ) + δ1 (q − n − τ + s − 1)

i=i(n)+1

≤

n 

gi (xi , xi+1 ) + 3f 

i=s+1

+3 + δ1 (q − n − k + s − 1) + μ(f )(i(s) − i(k) − 1) + c(f ) +μ(f )(q − n − τ + s − 1) + c(f ) ≤

n 

gi (xi , xi+1 ) + 3f  + 3 + 2c(f )

i=s+1

+δ1 (q − n − k + s − 1) + μ(f )(q − n − k + s − 2).

(4.119)

By (4.85), the inclusion gi ∈ U, i = 0, . . . , q − 1 and property (b) q−1 

gi (xi , xi+1 ) ≥ −δ1 (q − n) + μ(f )(q − n) − c(f ).

i=n

Together with (4.119), (4.114), (4.113), (4.85), and (4.105) this implies that M0 ≥ −2f  − 2 +

s−1 

f (xi , xi+1 ) − δ1 (s − k − 1)

i=k+1

+

n−1 

gi (xi , xi+1 ) +

i=s

q−1 

gi (xi , xi+1 ) −

i=n

≥ −2f  − 2 +

s−1 

q−1 

gi (yi , yi+1 )

i=k+1

f (xi , xi+1 ) − δ1 (s − k − 1)

i=k+1

+

n−1 

gi (xi , xi+1 ) − δ1 (q − n) + μ(f )(q − n) − c(f )

i=s

−

n 

gi (xi , xi+1 ) − 3f  − 3

i=s+1

−2c(f ) − δ1 (q − n − k + s − 1) − μ(f )(q − n − k + s − 2) ≥

s−1 

f (xi , xi+1 ) − 7(f  + 1)

i=k+1

−2δ1 (s − k − 1) − 2δ1 (q − n) − μ(f )(s − k − 2) − 3c(f )

88

4 Optimal Control Problems with Nonsingleton Turnpikes

≥

s−1 

f (xi , xi+1 ) − 2δ1 (s − k − 1)

i=k+1

−μ(f )(s − k − 1) − 2δ1 Q0 − 7(f  + 1) − 3c(f ) − |μ(f )|. It follows from the relation above, (4.118), (4.105), and (4.83) that in both cases we have s−1 

f (xi , xi+1 ) ≤ μ(f )(s − k − 1)

i=k+1

+2δ1 (s − k − 1) + M0 + 3c(f ) + 7(f  + 1) + 2δ1 Q0 + |μ(f )| ≤ μ(f )(s − k − 1) + 2δ1 (s − k − 1) + M0 +3c(f ) + 7(f  + 1) + 2 + |μ(f )|. By the relation above, (4.84), (4.103), and property (Pi) there exists i ∈ {k + 1, . . . , s − 1} such that d1 ((xi , xi+1 ), H (f )) ≤ 0 . This contradicts (4.103). The obtained contradiction proves Lemma 4.17. Lemma 4.18 Let M0 > 0. Then there exist numbers M1 > M0 , > 0, an integer N0 ≥ 1, and a neighborhood U of f in B(M) such that for each each integer q ≥ 1, q each gi ∈ U, i = 0, . . . , q − 1, each program {xi }i=0 ⊂ K which satisfies q−1 

q−1

gi (xi , xi+1 ) ≤ inf{U ({gi }i=0 , y, z) : y, z ∈ K} + M0

(4.120)

i=0

and each pair of integers p1 , p2 ∈ [0, q − 1] which satisfy p2 > p1 + N0 , d1 ((xi , xi+1 ), H (f )) ≤ , i = p1 , p2

(4.121)

the following inequality holds: p2 −1



p −p1 −1

2 gi (xi , xi+1 ) ≤ inf{U ({gi }i=0

, y, z) : y, z ∈ K} + M1 .

(4.122)

i=p1

Proof By properties (c) and (d) there exist an open set V ⊂ M in the relative topology and a number δ0 ∈ (0, 8−1 ) such that: f 2 ¯ [ω({xi }∞ i=0 )] ⊂ V ⊂ V ⊂ Vf ;

{(x1 , x2 ) ∈ M : d(xj , ω({xi }∞ i=0 )) ≤ δ0 , j = 1, 2} ⊂ V ; f

(4.123)

4.4 Auxiliary Results

89

for each (x1 , y1 ) ∈ H (f ) and each (x2 , y2 ) ∈ M which satisfy d1 ((x1 , y1 ), (x2 , y2 )) ≤ δ0 we have (x1 , y2 ), (x2 , y1 ) ∈ M;

(4.124)

for each (xi , yi ) ∈ V¯ , i = 1, 2 satisfying d1 ((x1 , y1 ), (x2 , y2 )) ≤ δ0

(4.125)

the inequality |f (x1 , y1 ) − f (x2 , y2 )| ≤ 1 holds. Fix ∈ (0, 64−1 δ0 ).

(4.126)

It follows from property (g) that there exist integers N1 , N2 ≥ 1 such that dist({(xi , xi+1 ) : i = 0, . . . , N1 − 1}, H (f )) ≤ 8−1 , f

f

dist({(xi , xi+1 ) : i = 2N1 + 4, . . . , 2N1 + 3 + N2 }, H (f )) ≤ 8−1 . f

f

(4.127)

By Lemma 4.17 there exist an integer Q1 ≥ 1 and a neighborhood U1 of f in B(M) such that for each integer q ≥ 8Q1 + 8, each gi ∈ U1 , i = 0, . . . , q − 1, each q program {xi }i=0 ⊂ K which satisfies q−1 

q−1

gi (xi , xi+1 ) ≤ inf{U ({gi }i=0 , y, z) : y, z ∈ K} + 4M0 + 8

(4.128)

i=0

and each integer p ∈ [0, q − Q1 ] the following equation holds: inf{d1 ((xi , xi+1 ), H (f )) : i ∈ [p, p + Q1 − 1]} ≤ 4−1 .

(4.129)

Fix integers N3 and N0 such that N3 ≥ 10, N0 ≥ 64(Q1 + N1 + N2 + N3 + 4), 64−1 N0 ∈ Z,

(4.130)

a number M1 > M0 + (1 + f )(N1 + N2 + N0 + Q1 + N3 + 4)10 + 4

(4.131)

and a neighborhood U of f in B(M) such that U ⊂ U1 ∩ {g ∈ B(M) : g − f  < 1}.

(4.132) q

Assume that an integer q ≥ 1, gi ∈ U, i = 0, . . . , q − 1, a program {xi }i=0 ⊂ K satisfies (4.120) and that integers p1 , p2 ∈ [0, q − 1] satisfy (4.121). We show that (4.122) holds.

90

4 Optimal Control Problems with Nonsingleton Turnpikes

Let us assume the contrary. Then p2 −1



p −p1 −1

2 gi (xi , xi+1 ) > inf{U ({gi }i=0

, y, z) : y, z ∈ K} + M1 .

(4.133)

i=p1 p

2 There exists a program {zi }i=p ⊂ K such that 1

p2 −1



gi (zi , zi+1 )

i=p1 p2 −1

< min{



p −p1 −1

2 gi (xi , xi+1 ) − M1 , inf{U ({gi }i=0

, y1 , y2 ) : y1 , y2 ∈ K} + 1}.

i=p1

(4.134) It follows from (4.134) and the definition of Q1 and U1 (see (4.128), (4.129)), (4.121), and (4.130) that there exist integers j1 ∈ [p1 + Q1 + 16−1 N0 , p1 + 2Q1 + 16−1 N0 ], j2 ∈ [p2 − 2Q1 − 16−1 N0 , p2 − Q1 − 16−1 N0 ]

(4.135)

such that d1 ((zi , zi+1 ), H (f )) ≤ 2−1 , i = j1 , j2 .

(4.136)

p3 = sup{i ∈ [p2 , q − 1] : d1 ((xi , xi+1 ), H (f )) ≤ }.

(4.137)

Set

By (4.121), (4.120), the definition of Q1 , U1 (see (4.128), (4.129)), and (4.130), p3 ≥ p2 , p3 ≥ q − Q1 .

(4.138)

Equations (4.136), (4.121), (4.127), (4.138), and (4.137) imply that there exist integers i1 , s2 ∈ [0, N1 − 1], i3 , s1 , i2 ∈ [2N1 + 4, 2N1 + 3 + N2 ]

(4.139)

such that d1 ((xik , xik +1 ), (xpk , xpk +1 )) ≤ + 6−1 , k = 1, 2, 3, f

f

d1 ((xsfk , xsk +1 ), (zjk , zjk +1 )) ≤ + 6−1 , k = 1, 2. f

(4.140)

4.4 Auxiliary Results

91 q

We define a program {yi }i=0 and estimate q−1 

[gi (xi , xi+1 ) − gi (yi , yi+1 )].

i=0

Set τ (1) = p1 − i1 , τ (2) = j1 − (s1 − i1 + p1 ), τ (3) = s2 − (j2 − j1 + s1 − i1 + p1 ), τ (4) = p2 − (i2 − s2 + j2 − j1 + s1 − i1 + p1 ), τ (5) = i3 − (p3 − p2 + i2 − s2 + j2 − j1 + s1 − i1 + p1 ), yi = xi , i = 0, . . . , p1 , yi =

f xi−τ (1) ,

(4.141)

i = p1 + 1, . . . , s1 − i1 + p1 ,

yi = zi+τ (2) , i = s1 − i1 + p1 + 1, . . . s1 − i1 + p1 + j2 − j1 , f

yi = xi+τ (3) , i = s1 − i1 + p1 + j2 − j1 + 1, . . . , i2 − s2 + j2 − j1 +s1 −i1 +p1 , (4.142) if p3 − p2 ≤ N3 , then we set f

yi = xi+τ (3) , i = i2 − s2 + j2 − j1 + s1 − i1 + p1 + 1, . . . , q,

(4.143)

otherwise we set yi = xi+τ (4) , i = i2 − s2 + j2 − j1 + s1 − i1 + p1 + 1, . . . , p3 − p2 + i2 − s2 + j2 − j1 + s1 − i1 + p1, f

yi = xi+τ (5) , i = p3 − p2 + i2 − s2 + j2 − j1 + s1 − i1 + p1 + 1, . . . , q.

(4.144)

By (4.141)–(4.144), (4.139), (4.121), (4.130),(4.135), (4.140), (4.126), and the q choice of δ0 (see (4.124)), {yi }i=0 is a program. By (4.120) and (4.142), M0 ≥

q−1 

(gi (xi , xi+1 ) − gi (yi , yi+1 ))

i=0

=

q−1 

[gi (xi , xi+1 ) − gi (yi , yi+1 )].

i=p1

By (4.141), (4.142), and (4.139), q−1 

gi (yi , yi+1 ) ≤ 5f  + 5 +

gi (yi , yi+1 )

i=p1 +1

i=p1 j2 −τ (2)−1

+

s1 +τ (1)−1 



i=s1 +τ (1)+1

gi (yi , yi+1 ) +

q−1  i=j2 −τ (2)+1

gi (yi , yi+1 )

(4.145)

92

4 Optimal Control Problems with Nonsingleton Turnpikes j2 −1

s 1 −1

≤ 5f  + 5 +

f

f

gi (xi , xi+1 ) +

i=i1 +1



q−1 

gi (zi , zi+1 ) +

i=j1 +1 j2 −1

≤ (f  + 1)(2N1 + 8 + N2 ) +



q−1 

gi (zi , zi+1 ) +

i=j1 +1

gi (yi , yi+1 )

i=j2 −τ (2)+1

gi (yi , yi+1 ).

i=j2 −τ (2)+1

(4.146) It follows from (4.134) and (4.135) that p2 −1

j2 −1



gi (zi , zi+1 ) ≤

i=j1 +1



gi (zi , zi+1 ) + 2(1 + f )(2Q1 + 16−1 N0 + 4)

i=p1 p2 −1



≤

gi (xi , xi+1 ) − M1 + 2(1 + f )(2Q1 + 16−1 N0 + 4).

(4.147)

i=p1

There two cases: (i) p3 − p2 ≤ N3 ; (ii) p3 − p2 > N3 . Consider the case (i). Equations (4.135), (4.138), (4.139), (4.140), (4.141), and (4.142) imply that q − (j2 − τ (2)) ≤ q − p2 + 4Q1 + 8−1 N0 ≤ Q1 + N3 + 4Q1 + 8−1 N0 . Together with (4.146) and (4.148) this implies that q−1 

p2 −1

gi (yi , yi+1 ) ≤ (1 + f ) (2N1 + N2 + 8) +

i=p1



gi (xi , xi+1 ) − M1

i=p1

+(1 + f )(2Q1 + 16−1 N0 + 4)2 + (1 + f )(5Q1 + N3 + 8−1 N0 ) p2 −1

≤



gi (xi , xi+1 ) − M1 + (1 + f ) (2N1 + N2 + 9Q1 + N3 + 4−1 N0 + 16).

i=p1

(4.148) It follows from (4.138) that q−1  i=p1

p2 −1

gi (xi , xi+1 ) ≥



gi (xi , xi+1 ) − (1 + f )(Q1 + N3 ).

i=p1

Combining this equation with (4.145) and (4.148) we obtain that M0 ≥ M1 − (1 + f )[Q1 + N3 + 2N1 + N2 + 9Q1 + N3 + 4−1 N0 + 16]. (4.149) Consider the case (ii). By (4.141), (4.142), (4.144), (4.139), (4.138), (4.135), and the inclusion gi ∈ U, i = 0, . . . , q − 1, q−1  i=j2 −τ (2)+1

p3 −τ (4)−1

gi (yi , yi+1 ) ≤ (1 + f )(i2 − s2 + 2) +



i=p2 −τ (4)+1

gi (yi , yi+1 )

4.4 Auxiliary Results

93

+(1 + f )(q − (p3 − p2 + i2 − s2 + j2 − j1 + s1 − i1 + p1 ) + 2) p3 −1

≤ (1 + f )(2N1 + N2 + 5) +



gi (xi , xi+1 )

i=p2 +1

+(1 + f )(Q1 + 4Q1 + 8−1 N0 + 2) p3 −1

≤



gi (xi , xi+1 ) + (1 + f )(2N1 + N2 + 5Q1 + 8−1 N0 + 7).

i=p2 +1

Together with (4.146) and (4.147) this implies that q−1 

gi (yi , yi+1 )

i=p1 p2 −1

≤ (1 + f )(2N1 + N2 + 8) +



gi (xi , xi+1 ) − M1

i=p1

+2(1 + f )(2Q1 + 16−1 N0 + 4) p3 −1



+

gi (xi , xi+1 ) + (1 + f )(2N1 + N2 + 5Q1 + 8−1 N0 + 7)

i=p2 +1 p3 −1

≤



gi (xi , xi+1 ) − M1 + (1 + f )(4N1 + 2N2 + 9Q1 + 4−1 N0 + 28).

i=p1

Combining the equation above with (4.145) and (4.138) we obtain that M0 ≥ −(Q1 + 2)(1 + f ) + M1 −(1 + f )(4N1 + 2N2 + 4−1 N0 + 9Q1 + 28).

(4.150)

It both cases in view of (4.150), (4.149), and (4.132) M0 ≥ M1 − (1 + f )(4N1 + 2N2 + 4−1 N0 + 10Q1 + 2N3 + 30). This contradicts (4.131). The obtained contradiction proves Lemma 4.18. Lemmas 4.17 and 4.18 imply the following result. Lemma 4.19 Let M0 > 0. Then there exist a number M1 > 0 and a neighborhood U of f in B(M) such that for each integer q ≥ 1, each gi ∈ U, i = 0, . . . , q − 1, q each program {xi }i=0 ⊂ K which satisfies q−1  i=0

q−1

gi (xi , xi+1 ) ≤ inf{U ({gi }i=0 , y, z) : y, z ∈ K} + M0

(4.151)

94

4 Optimal Control Problems with Nonsingleton Turnpikes

and each pair of integers p1 ∈ [0, q −1], p2 ∈ (p1 , q] the following inequality holds: p2 −1



p −p1 −1

2 gi (xi , xi+1 ) ≤ inf{U ({gi }i=0

, y, z) : y, z ∈ K} + M1 .

i=p1

Lemmas 4.15 and 4.19 imply the following result. Lemma 4.20 Let 0 , M0 > 0 and let l ≥ 1 be an integer such that for each (f )-good program {xi }∞ i=0 ⊂ K, dist(H (f ), {(xi , xi+1 ) : i ∈ [p, p + l]}) ≤ 8−1 0 for all sufficiently large natural numbers p (the existence of l follows from Lemma 4.13). Then there exist an integer N ≥ 10 and a neighborhood U of f in B(M) such q that for each integer q ≥ 1, each gi ∈ U, i = 0, . . . , q −1, each program {xi }i=0 ⊂ K which satisfies (4.151) and each integer p satisfying 0 ≤ p ≤ q − N l there exists an integer j0 ∈ [0, N − 8] such that for each integer T ∈ [p + j0 l, p + (j0 + 7)l] the following inequality holds: dist(H (f ), {(xi , xi+1 ) : i ∈ [T , T + l]}) ≤ 0 . Lemma 4.21 Let > 0 and let L ≥ 1 be an integer such that each (f )-good program {xi }∞ i=0 ⊂ K satisfies dist(H (f ), {(xi , xi+1 ) : i ∈ [p, p + L]}) ≤

(4.152)

for all sufficiently large integers p ≥ 0 (the existence of L follows from Lemma 4.13). Then there exists a number δ > 0 such that for each triplet of integers T ≥ L, s, q and each program {xi }Ti=0 ⊂ K which satisfies x0 = xsf , xT = xqf ,

T −1 

[f (xi , xi+1 ) − μ(f )] − π f (xsf ) + π f (xqf ) ≤ δ

i=0

inequality (4.152) holds for all integers p ∈ [0, T − L] (recall π f in (4.29)). Proof Assume the contrary. Then for each integer k ≥ 1 there exist integers T (k) ≥ (k) L, s(k), q(k), p(k), and a program {xik }Ti=0 ⊂ K such that f

f

x0k = xs(k) , xTk (k) = xq(k) , T (k)−1

k [f (xik , xi+1 ) − μ(f )] − π f (xs(k) ) + π f (xq(k) ) ≤ 2−k , f

f

i=0

p(k) ∈ [0, T (k) − L], k dist(H (f ), {(xik , xi+1 ) : i ∈ [p(k), p(k) + L]}) > .

(4.153)

4.4 Auxiliary Results

95

We construct an (f )-good program {yi }∞ i=0 . Let k ≥ 1 be an integer. By property (g) (xs(k+1)−1 , xs(k+1) ) ∈ Ω({xi }∞ i=0 ). f

f

f

(4.154)

It follows from (4.154), properties (c) and (d) and Proposition 4.10 that there exists an integer j (k) such that j (k) > 2q(k) + 4, |π f (xj (k) ) − π f (xs(k+1)−1 )| ≤ 2−k−1 , f

f

|π f (xj (k)+1 ) − π f (xs(k+1) )| ≤ 2−k−1 , f

f

(xj (k) , xs(k+1) ) ∈ M, |f (xj (k) , xs(k+1) ) − f (xs(k+1)−1 , xs(k+1) )| ≤ 2−k−1 , f

f

f

f

f

f

|f (xj (k) , xj (k)+1 ) − f (xs(k)+1)−1 , xs(k+1 )| ≤ 2−k−1 . f

f

f

f

(4.155)

Set τ (1) = T (1), τ (k) =

k 

(1 + T (i)) +

i=1

k−1 

[j (i) − q(i)], k ∈ Z2 .

i=1

We define f

yi = xi1 , i = 0, . . . , τ (1), yτ (k)+i = xq(k)+i , i = 1, . . . , j (k) − q(k), k+1 yτ (k)+j (k)−q(k)+i = xi−1 , i = 1 . . . , 1 + T (k + 1), k = 1, 2, 3, . . .

(4.156)

It follows from (4.153), (4.155), (4.156), and Proposition 4.12 that f

yτ (k) = xTk (k) , yτ (k)+j (k)−q(k) = xj (k) , k = 1, 2, . . . , {yi }∞ i=0 is a program and that for each integer k ≥ 1, τ (k+1)−1 

[f (yi , yi+1 ) − μ(f )] − π f (yτ (k) ) + π f (yτ (k+1) )

i=τ (k)



τ (k)+j (k)−q(k)−1

=

[f (yi , yi+1 ) − μ(f )] − π f (yτ (k) ) + π f (yτ (k)+j (k)−q(k) )

i=τ (k)

+[f (yτ (k)+j (k)−q(k) , yτ (k)+j (k)−q(k)+1 ) − μ(f ) −π f (yτ (k)+j (k)−q(k) ) + π f (yτ (k)+j (k)−q(k)+1 )] +

τ (k+1)−1 

[f (yi , yi+1 ) − μ(f )] − π f (yτ (k)+j (k)−q(k)+1 ) + π f (yτ (k+1) )

i=τ (k)+j (k)−q(k)+1



j (k)−1

=

i=q(k)

f

f

f

f

[f (xi , xi+1 ) − μ(f )] − π f (xq(k) ) + π f (xj (k) )

96

4 Optimal Control Problems with Nonsingleton Turnpikes f

f

f

f

+[f (xj (k) , xs(k+1) ) − μ(f ) − π f (xj (k) ) + π f (xs(k+1) )] +

T (k+1)−1 

k+1 [f (xik+1 , xi+1 ) − μ(f )] − π f (x0k+1 ) + π f (xTk+1 (k+1) )

i=0

≤ 2−k + f (xs(k+1)−1 , xs(k+1) ) − μ(f ) f

f

−π f (xs(k+1)−1 ) + π f (xs(k+1) ) + 2−k−1 ≤ 2−k+1 . f

f

This implies that {yi }∞ i=0 is an (f )-good program. Therefore, it follows from the choice of L that dist(H (f ), {(yi , yi+1 ) : i ∈ [p, p + L]}) ≤ for all large enough integersp. (4.157) On the other hand by (4.153) and (4.156) for each integer k ≥ 1, dist(H (f ), {(yi , yi+1 ) : i ∈ [τ (k) + j (k) − q(k) + 1 + p(k) + 1), τ (k) + j (k) − q(k) + 1 + p(k) + L]}) > . This contradicts (4.157). The contradiction we have reached proves the lemma.

4.5

Proof of Theorem 4.5

There exist an (f )-minimal program {xi }∞ i=−∞ , constants c(f ) > 0, γ (f ) > 0, μ(f ) ∈ R 1 , an open set Vf ⊂ M in the relative topology such that properties (a)–(g) (see Sect. 4.4) hold. Note that we can use all the results of Sects. 4.3 and 4.4. By Lemma 4.13 there exists an integer L ≥ 2 such that for each (f )-good program {xi }∞ i=0 ⊂ K the equation f

dist(H (f ), {(xi , xi+1 ) : i ∈ [s, s + L − 1]}) ≤ 4−1

(4.158)

holds for all large enough integers s ≥ 1. By Lemma 4.21 there exists δ0 ∈ (0, 8−1 )

(4.159)

such that for each integer T ≥ L and each program {xi }Ti=0 ⊂ K which satisfies f

x0 , xT ∈ {xi : i ∈ Z},

T −1 

[f (xi , xi+1 ) − μ(f )] − π f (x0 ) + π f (xT ) ≤ δ0

i=0

(4.160) the Eq. (4.158) holds for all integers s ∈ [0, T − L].

4.5 Proof of Theorem 4.5

97

Let an open set Vf ⊂ M in the relative topology be as guaranteed by property (c). By properties (c),(d), and (e) we may assume without loss of generality that {(x1 , x2 ) ∈ M : d(xj , ω({xi }∞ i=0 )) ≤ 8δ0 , j = 1, 2} ⊂ Vf , f

(4.161)

for each (x1 , y1 ) ∈ H (f ) and each (x2 , y2 ) ∈ M satisfying d1 ((x1 , y1 ), (x2 , y2 )) ≤ 8δ0 we have (x1 , y2 ), (x2 , y1 ) ∈ M,

(4.162)

and that for each (x1 , y1 ) ∈ H (f ) and each (x2 , y2 ) ∈ M satisfying d(y1 , x2 ) ≤ 8δ0 we have (x1 , x2 ) ∈ M.

(4.163)

Set f

π f  = sup{|π f (x)| : x ∈ H0 }.

(4.164)

Fix a natural number Q > 24 + 2δ0−1 (M + 2 + 4(f  + c(f ) + |μ(f )| + 1 + + 2π f ))

(4.165)

(recall c(f ) in property (a)). It follows from property (c), (4.161), and Proposition 4.10 that there exists δ1 ∈ (0, 8−1 δ0 )

(4.166)

such that: for each 2 (y1 , y2 ), (z1 , z2 ) ∈ M ∩ {x ∈ K : d(x, ω({xi }∞ i=0 )) ≤ 8δ0 } f

(4.167)

d(yi , zi ) ≤ 8δ1 , i = 1, 2

(4.168)

|f (y1 , y2 ) − f (z1 , z2 )| ≤ (16Q)−1 δ0 ;

(4.169)

which satisfies

we have

f

for each y, z ∈ H0 which satisfy d(y, z) ≤ 8δ1 the equation |π f (y) − π f (z)| ≤ (16Q)−1 δ0

(4.170)

holds. By property (g) there exist integers N1 , N2 ≥ 1 for which dist(H (f ), {(xi , xi+1 ) : i = 0, . . . , N1 − 1}) ≤ 16−1 δ1 , f

f

(4.171)

98

4 Optimal Control Problems with Nonsingleton Turnpikes

dist(H (f ), {(xi , xi+1 ) : i = 2N1 + 4, . . . , 2N1 + 3 + N2 }) ≤ 16−1 δ1 . f

f

By Lemma 4.13 there exists an integer L1 ≥ 1 such that for each (f )-good program {xi }∞ i=0 ⊂ K the equation dist(H (f ), {(xi , xi+1 ) : i ∈ [S, S + L1 ]}) ≤ 8−1 δ1

(4.172)

holds for all large enough natural numbers S. We may assume that L1 ≥ 8(L + N1 + N2 + 4).

(4.173)

By Lemma 4.20 there exist an integer N ≥ 10 and a neighborhood U1 of f in B(M) such that for each integer P ≥ 1, each gi ∈ U1 , i = 0, . . . , P − 1, each program {xi }Pi=0 ⊂ K which satisfies P −1 

−1 gi (xi , xi+1 ) ≤ inf{U ({gi }Pi=0 , y, z) : y, z ∈ K} + M + 4

(4.174)

i=0

and each integer p satisfying 0 ≤ p ≤ P −N L1 there exists an integer j0 ∈ [0, N −8] such that for each integer s ∈ [p + j0 L1 , p + (j0 + 7)L1 ] the following equation holds: dist(H (f ), {(xi , xi+1 ) : i ∈ [s, s + L1 ]}) ≤ δ1 .

(4.175)

l = 50N L1 .

(4.176)

δ2 ∈ (0, (6400QN (L1 + l))−1 δ1 )

(4.177)

Set

Fix a number

and a neighborhood U of f in B(M) such that U ⊂ U1 ∩ {g ∈ B(M) : g − f  < δ2 }.

(4.178)

Assume that an integer T ≥ L + lQ, gi ∈ U, i = 0, . . . , T − 1 and that a program {xi }Ti=0 ⊂ K satisfies T −1 

−1 gi (xi , xi+1 ) ≤ inf{U ({gi }Ti=0 , y, z) : y, z ∈ K} + M.

(4.179)

i=0

Set E = {s ∈ Z : 10N L1 ≤ s ≤ T − 10N L1 and dist(H (f ), {(xi , xi+1 ) : i ∈ [s, s + L − 1]}) > }.

(4.180)

4.5 Proof of Theorem 4.5

99

If E = ∅, then the assertion of the theorem is valid. Hence we may assume that E = ∅. Set h1 = inf{h : h ∈ E}.

(4.181)

It follows from the choice of U1 , N (see (4.174), (4.175)) and (4.179) that there are integers i1 , i2 ∈ {0, . . . , N − 8} such that (4.175) holds for any integer s ∈ [h1 − (2 N − i1 )L1 , h1 − (2 N − i1 − 7)L1 ] ∪ [h1 + (N + i2 )L1 , h1 +(N + i2 + 7)L1 ].

(4.182)

b1 = h1 − (2 N − i1 )L1 , c1 = h1 + (N + i2 )L1 .

(4.183)

Set

By induction we define sequences of integers bq , cq , q ≥ 1 such that (B) NL1 ≤ cq − bq ≤ 4N L1 , bq ≥ cq−1 if q ≥ 2, q

[bq , cq − N L1 ] ∩ E  = ∅, E \ ∪j =1 [bj , cj ] ⊂ (cq , T ]; (C) for h ∈ {bq , cq } Eq. (4.175) holds for each s ∈ [h, h + 7L1 ]. It is easy to see that for q = 1 properties (B) and (C) hold. Assume that sequences of integers {bq }kq=1 , {cq }kq=1 have been defined and properties (B) and (C) hold for q = 1, . . . , k where k is a natural number. If E \ ∪kq=1 [bq , cq ] = ∅, then the construction of the sequences is completed and bk , ck are their last elements. Let us assume that E \ ∪kq=1 [bq , cq ]  = ∅ and set h2 = inf{h : h ∈ E \ ∪kq=1 [bq , cq ]}. It follows from (4.179), (4.180), and the definition of U1 (see (4.174), (4.175)) that there are integers j1 , j2 ∈ [0, N − 8] such that (4.175) holds for any integer s ∈ [h2 −(2 N −j1 )L1 , h2 −(2 N −j1 −7)L1 ]∪[h2 +(N +j2 )L1 , h2 +(N +j2 +7)L1 ]. Set ck+1 = h2 + (N + j2 )L1 , bk+1 = max{ck , h2 − 2N L1 + j1 L1 }. It is easy to see that properties (B) and (C) hold with q = k + 1. Evidently the construction of the sequence will be completed in a finite number of steps. Let bQ˜ , cQ˜ be the last elements of the sequences. Clearly, ˜

E ⊂ ∪Q q=1 [bq , cq ].

(4.184)

100

4 Optimal Control Problems with Nonsingleton Turnpikes

˜ By property (C) Let i ∈ {1, . . . , Q}. d1 ((xbi , xbi +1 ), H (f )) ≤ δ1 , d1 ((xci , xci +1 ), H (f )) ≤ δ1 . It follows from these inequalities and the choice of N1 , N2 (see (4.171)) that there exist p(i) ∈ [0, N1 − 1], s(i) ∈ [2N1 + 4, . . . , 2N1 + 3 + N2 ]

(4.185)

such that d1 ((xbi , xbi +1 ), (xp(i) , xp(i)+1 )) ≤ δ1 + 15−1 δ1 , f

f

d1 ((xci , xci +1 ), (xs(i) , xs(i)+1 )) ≤ δ1 + 15−1 δ1 . f

f

(4.186)

We show that c i −1

[f (xj , xj +1 ) − μ(f )] + π f (xs(i) ) − π f (xp(i) ) ≥ δ0 (1 − (8Q)−1 ). f

f

(4.187)

j =bi +1 Define a sequence {wj }jci=b ⊂ K by i f

f

wbi = xp(i) , wj = xj , j ∈ [bi + 1, . . . , ci ], wci +1 = xs(i+1) .

(4.188)

+1 By (4.188), (4.186), (4.166), (4.162), and (4.163) the sequence {wj }jci=b is a program. i It follows from property (B), the choice of δ0 (see (4.160)), (4.186), and (4.188) that ci 

[f (wj , wj +1 ) − μ(f )] − π f (wbi ) + π f (wci +1 ) > δ0 .

j =bi

By the choice of δ1 (see (4.167)–(4.169)), (4.186), (4.188), and (4.166), |f (wj , wj +1 ) − f (xj , xj +1 )| ≤ (16Q)−1 δ0 , j = bi , ci , |f (wci , wci +1 ) − f (xs(i) , xs(i)+1 )| ≤ (16Q)−1 δ0 . f

f

Together with (4.189) and (4.188) these inequalities imply that f

f

δ0 < π f (xs(i)+1 ) − π f (xp(i) ) +

c i −1

[f (xj , xj +1 ) − μ(f )]

j =bi

+[f (wbi , wbi +1 ) − f (xbi , xbi +1 )] + f (wci , wci +1 ) − μ(f ) ≤

c i −1 j =bi

[f (xj , xj +1 ) − μ(f )]

(4.189)

4.5 Proof of Theorem 4.5

101

+(16Q)−1 δ0 + (16Q)−1 δ0 + f (xs(i) , xs(i)+1 ) − μ(f ) f

f

f

f

+π f (xs(i)+1 ) − π f (xp(i) ). Equation (4.187) follows from the equation above. By Proposition 4.12, (4.187), (4.178), and property (B), c i −1

f

f

[gj (xj , xj +1 ) − μ(f )] + π f (xs(i) ) − π f (xp(i) )

j =bi

≥ δ0 (1 − (8Q)−1 ) − δ2 4N L1 . s(i)−p(i)

Define a sequence {uji }j =0

(4.190)

⊂ K by

f

i u0i = xbi , uji = xp(i)+j , j = 1, . . . , s(i) − p(i) − 1, us(i)−p(i) = xci .

(4.191)

It follows from (4.191), (4.186), (4.166), (4.162), and (4.163) that the sequence s(i)−p(i) {uji }j =0 is a program. By (4.191), Proposition 4.12, (4.186), and (4.166)–(4.169) 

s(i)−p(i)−1 f

f

[f (uji , uji +1 ) − μ(f )] − π f (xp(i) ) + π f (xs(i) )

j =0 f

f

f

f

f

f

= f (xbi , xp(i)+1 ) − f (xp(i) , xp(i+1) ) + f (xs(i)−1 , xci ) − f (xs(i)−1 , xs(i) ) +

s(i)−1 

[f (xj , xj +1 ) − μ(f )] − π f (xp(i) ) + π f (xs(i) ) ≤ (8Q)−1 δ0 . f

f

f

f

j =p(i)

This equation, (4.178), (4.185), and (4.173) imply that 

s(i)−p(i)−1

[gi (uji , uji +1 ) − μ(f )] − π f (xp(i) ) + π f (xs(i) ) ≤ (8Q)−1 δ0 + δ2 L1 . f

f

j =0

(4.192) ¯ c(i),i ˜ by By induction we define sequences of integers b(i), ¯ = 1, . . . , Q ¯ = b1 , c(i) ¯ + s(i) − p(i), i = 1, . . . , Q, ˜ b(1) ¯ = b(i) ¯ + 1) = bi+1 + c(i) ˜ − 1. b(i ¯ − ci for all integers i such that i ≤ Q

(4.193)

By (4.193), property (B), (4.185), and (4.173), ˜ = cQ˜ − c( ¯ Q)

˜ Q 

˜ 1 (N − 1), 4QL ˜ 1 N ]. [ci − bi − (s(i) − p(i))] ∈ [QL

i=1 c˜

Define a sequence {yj }jQ=0 ⊂ K by ¯ ˜ ¯ i = 1, . . . , Q, yj = xj , j = 0, . . . , b1 , yj = uji −b(i) ¯ , j ∈ [b(i), c(i)],

(4.194)

102

4 Optimal Control Problems with Nonsingleton Turnpikes

¯ + 1)] for all natural numbers i ≤ Q ˜ − 1, yj = xj −c(i)+c ¯ b(i ¯ i , j ∈ [c(i), f ˜ + 1, . . . , cQ˜ . yj = xj −c(¯ Q)+s( ¯ Q) ˜ ˜ , j = c( Q)

(4.195) c˜

It follows from (4.195), (4.193), (4.191), (4.186), (4.162), and (4.163) that {yj }jQ=0 is well defined and is a program. Evidently, f

ycQ˜ = xc

˜

˜

¯ Q)+s(Q) ˜ −c( Q

(4.196)

.

Equation (4.171) implies that there exists an integer q ∈ [0, . . . , N1 − 1] for which d1 ((xc(Q)− ), (xqf , xq+1 )) ≤ 15−1 δ1 . ˜ c( ˜ ˜ , xc(Q)− ˜ c( ˜ ˜ ¯ Q)+s( Q) ¯ Q)+s( Q)+1 f

f

f

(4.197)

Together with (4.196), (4.162), (4.163), and (4.166) this implies that f

(ycQ˜ , xq+1 ) ∈ M.

(4.198)

˜ and (4.162), (4.163), and (4.166) By (4.186), which holds with i = Q, f

(xs(Q) ˜ , xcQ˜ +1 ) ∈ M.

(4.199)

We set f ˜ − q, ycQ˜ +j = xq+s , j = 1, . . . , s(Q)

˜ + q. yc ˜ +s(Q)−q+j = xcQ˜ +j , j = 1, . . . , T − cQ˜ − s(Q) ˜ Q

(4.200)

Equations (4.200), (4.198), and (4.199) imply that {yj }Tj=0 ⊂ K is a program. It follows from (4.179), (4.195), (4.193), (4.190), and (4.192) that M≥

T −1 

[gi (xi , xi+1 ) − μ(f )] −

i=0

=

˜ cj −1 Q  

bj +1 −1





[gi (xi , xi+1 ) − μ(f )] +

[gi (xi , xi+1 ) − μ(f )]

i=cj ˜ j ∈Z: 1≤j ≤Q−1

T −1 

[gi (xi , xi+1 ) − μ(f )] −



˜ c(j ¯ )−1 Q  

[gi (yi , yi+1 ) − μ(f )]

¯ ) j =1 i=b(j

i=cQ˜

−

[gi (yi , yi+1 ) − μ(f )]

i=0

j =1 i=bj

+

T −1 

¯ +1)−1 b(j



[gi (yi , yi+1 ) − μ(f )] −

i=c(j ¯ ) ˜ j ∈Z: 1≤j ≤Q−1

T −1 

[gi (yi , yi+1 ) − μ(f )]

i=c¯Q˜

˜ 0 (1 − (8Q)−1 ) − δ2 4N L1 − δ0 (8Q)−1 − δ2 L1 ) ≥ Q(δ +

T −1  i=cQ˜

[gi (xi , xi+1 ) − μ(f )] −

T −1  ˜ i=c( ¯ Q)

[gi (yi , yi+1 ) − μ(f )].

(4.201)

4.5 Proof of Theorem 4.5

We estimate and (4.178) T −1 

T −1

˜ i=c( ¯ Q)

103

[gi (yi , yi+1 − μ(f )]. In view of (4.195), (4.191), (4.200), ˜ ˜ cQ˜ −c( ¯ Q)+s( Q)−1



[gi (yi , yi+1 ) − μ(f )] ≤ 1 + f  + |μ(f )| +

˜ i=c( ¯ Q)

f

f

[gi (xi , xi+1 ) − μ(f )]

˜ i=s(Q)+1 ˜ s( Q)−1

+1 + f  + |μ(f )| +

f

f

[gi (xi , xi+1 ) − μ(f )] + 1 + f  + |μ(f )|

i=q+1 ˜ T −s(Q)+q−1

+



[gi (xi , xi+1 ) − μ(f )] ≤ 4(1 + f  + |μ(f )|)

i=cQ +1

˜ + +δ2 (cQ˜ − c( ¯ Q))

˜ ˜ cQ˜ −c( ¯ Q)+s( Q)−1



˜ [f (xi , xi+1 ) − μ(f )] + δ2 s(Q) f

f

˜ i=s(Q)+1

+

˜ s( Q)−1

f

f

[f (xi , xi+1 ) − μ(f )]

i=q+1

+

T −1 

˜ + 1) − [gi (xi , xi+1 ) − μ(f )] + δ2 (s(Q)

T −1 

[f (xi , xi+1 ) − μ(f )].

˜ i=T −s(Q)−q

i=cQ˜

It follows from this equation, (4.194), (4.185), (4.173), and properties (a) and (b) that T −1 

[gi (yi , yi+1 − μ(f )]

˜ i=c( ¯ Q)

≤

T −1 

˜ 1 N + 3c(f ). [gi (xi , xi+1 ) − μ(f )] + 4(1 + f  + |μ(f )|) + δ2 8QL

i=cQ˜

Together with (4.178), (4.201), (4.177), and (4.166) this implies that ˜ 0 (1 − (4Q)−1 ) − δ2 8N L1 ] − (1 + f  + |μ(f )|)4 − δ2 8QL ˜ 1 N − 3c(f ) M ≥ Q[δ ˜ 0 − (1 + f  + |μ(f )|)4 − 3c(f ). ≥ 2−1 Qδ ˜ ≤ Q − 24. This completes the proof of Theorem By this equation and (4.165), Q 4.5.

References

1. Anderson, B.D.O., Moore, J.B.: Linear Optimal Control. Prentice-Hall, Englewood Cliffs (1971) 2. Arkin, V.I., Evstigneev, I.V.: Stochastic Models of Control and Economic Dynamics. Academic Press, London (1987) 3. Aseev, S.M., Kryazhimskiy, A.V.: The Pontryagin maximum principle and transversality conditions for a class of optimal control problems with infinite time horizons. SIAM J. Control Optim. 43(3), 1094–1119 (2004) 4. Aseev, S.M., Veliov, V.M.: Maximum principle for infinite-horizon optimal control problems with dominating discount. Dyn. Cont. Discrete Impuls. Syst. Ser. B 19(1–2), 43–63 (2012) 5. Atsumi, H.: Neoclassical growth and the efficient program of capital accumulation. Rev. Econ. Stud. 32(2), 127–136 (1965) 6. Aubry, S., Le Daeron, P.Y.: The discrete Frenkel–Kontorova model and its extensions I. Phys. D: Nonlinear Phenom. 8, 381–422 (1983) 7. Baumeister, J., Leitao, A., Silva, G.N.: On the value function for nonautonomous optimal control problem with infinite horizon. Syst. Control Lett. 56(3), 188–196 (2007) 8. Blot, J.: Infinite-horizon Pontryagin principles without invertibility. J. Nonlinear Convex Anal. 10(2), 177–189 (2009) 9. Blot, J., Cartigny, P.: Optimality in infinite-horizon variational problems under sign conditions. J. Optim. Theory Appl. 106(2), 411–419 (2000) 10. Blot, J., Hayek, N.: Sufficient conditions for infinite-horizon calculus of variations problems. ESAIM Control Optim. Calc. Var. 5, 279–292 (2000) 11. Bright, I.: A reduction of topological infinite-horizon optimization to periodic optimization in a class of compact 2-manifolds. J. Math. Anal. Appl. 394(1), 84–101 (2012) 12. Carlson, D.A.: The existence of catching-up optimal solutions for a class of infinite horizon optimal control problems with time delay. SIAM J. Control Optim. 28(2), 402–422 (1990) 13. Carlson, D.A., Haurie, A., Leizarowitz, A.: Infinite Horizon Optimal Control. Springer-Verlag, Berlin (1991) 14. Cartigny, P., Michel, P.: On a sufficient transversality condition for infinite horizon optimal control problems. Automatica 39(6), 1007–1010 (2003) 15. Coleman, B.D., Marcus, M., Mizel, V.J.: On the thermodynamics of periodic phases. Arch. Rational Mech. Anal. 117(4), 321–347 (1992) 16. Gaitsgory, V., Rossomakhine, S., Thatcher, N.: Approximate solution of the HJB inequality related to the infinite horizon optimal control problem with discounting. Dyn. Cont. Discrete Impuls. Syst. Ser. B 19, 65–92 (2012) 17. Gale, D.: On optimal development in a multi-sector economy. Rev. Econ. Stud. 34, 1–18 (1967) 18. Guo, X., Hernandez-Lerma, O.: Zero-sum continuous-time Markov games with unbounded transition and discounted payoff rates. Bernoulli 11, 1009–1029 (2005)

A. J. Zaslavski, Stability of the Turnpike Phenomenon in Discrete-Time Optimal Control Problems, SpringerBriefs in Optimization, DOI 10.1007/978-3-319-08034-5, © The Author 2014

105

106

References

19. Hayek, N.: Infinite horizon multiobjective optimal control problems in the discrete time case. Optimization 60, 509–529 (2011) 20. Jasso-Fuentes, H., Hernandez-Lerma, O.: Characterizations of overtaking optimality for controlled diffusion processes. Appl. Math. Optim. 57, 349–369 (2008) 21. Kolokoltsov, V., Yang, W.: The turnpike theorems for Markov games. Dyn. Games Appl. 2(3), 294–312 (2012) 22. Leizarowitz, A.: Infinite horizon autonomous systems with unbounded cost. Appl. Math. Optim. 13(1), 19–43 (1985) 23. Leizarowitz, A.: Tracking nonperiodic trajectories with the overtaking criterion. Appl. Math. Optim. 14, 155–171 (1986) 24. Leizarowitz, A., Mizel, V.J.: One dimensional infinite horizon variational problems arising in continuum mechanics. Arch. Ration. Mech. Anal. 106(2), 161–194 (1989) 25. Lykina, V., Pickenhain, S., Wagner, M.: Different interpretations of the improper integral objective in an infinite horizon control problem. J. Math. Anal. Appl. 340, 498–510 (2008) 26. Makarov, V.L., Rubinov, A.M.: Mathematical Theory of Economic Dynamics and equilibria. Springer-Verlag, New York (1977) 27. Malinowska, A.B., Martins, N., Torres, D.F.M.: Transversality conditions for infinite horizon variational problems on time scales. Optim. Lett. 5, 41–53 (2011) 28. Marcus, M., Zaslavski, A.J.: On a class of second order variational problems with constraints. Israel J. Math. 111, 1–28 (1999) 29. Marcus, M., Zaslavski, A.J.: The structure of extremals of a class of second order variational problems. Ann. Inst. H. Poincaré, Anal. non linéaire Anal. 16(5), 593–629 (1999) 30. Marcus, M., Zaslavski, A.J.: The structure and limiting behavior of locally optimal minimizers. Ann. Inst. H. Poincaré, Anal. non linéaire Anal. 19(3), 343–370 (2002) 31. McKenzie, L.W.: Turnpike theory. Econometrica 44, 841–866 (1976) 32. Mordukhovich, B.S.: Minimax design for a class of distributed parameter systems. Automat. Remote Control 50, 1333–1340 (1990) 33. Mordukhovich, B.S.: Optimal control and feedback design of state-constrained parabolic systems in uncertainly conditions. Appl. Anal. 90, 1075–1109 (2011) 34. Mordukhovich, B.S., Shvartsman, I.: Optimization and feedback control of constrained parabolic systems under uncertain perturbations. Optimal Control, Stabilization and Nonsmooth Analysis. Lecture notes control information science, vol. 301 Springer, 121–132 (2004) 35. Ocana Anaya, E., Cartigny, P., Loisel, P.: Singular infinite horizon calculus of variations. Applications to fisheries management. J. Nonlinear Convex Anal. 10(2), 157–176 (2009) 36. Pickenhain, S., Lykina, V., Wagner, M.: On the lower semicontinuity of functionals involving Lebesgue or improper Riemann integrals in infinite horizon optimal control problems. Control Cybernet. 37, 451–468 (2008) 37. Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970) 38. Rubinov, A.M.: Economic dynamics. J. Soviet Math. 26, 1975–2012 (1984) 39. Samuelson, P.A.: A catenary turnpike theorem involving consumption and the golden rule. Amer. Econom. Rev. 55, 486–496 (1965) 40. von Weizsacker, C.C.: Existence of optimal programs of accumulation for an infinite horizon. Rev. Econ. Stud. 32, 85–104 (1965) 41. Zaslavski, A.J.: Ground states in Frenkel–Kontorova model. Math. USSR Izvestiya 29, 323– 354 (1987) 42. Zaslavski, A.J.: Optimal programs on infinite horizon 1. SIAM J. Control Optim. 33(6), 1643– 1660 (1995) 43. Zaslavski, A.J.: Optimal programs on infinite horizon 2. SIAM J. Control and Optim. 33(6), 1661–1686 (1995) 44. Zaslavski, A.J.: Turnpike property for dynamic discrete time zero-sum games. Abstr. Appl. Anal. 4(1), 21–48 (1999)

References

107

45. Zaslavski, A.J.: Turnpike Properties in the Calculus of Variations and Optimal Control. Springer, New York (2006) 46. Zaslavski, A.J.: Turnpike results for a discrete-time optimal control systems arising in economic dynamics. Nonlinear Anal. Theory, Meth. Appl. 67(7), 2024–2049 (2007) 47. Zaslavski, A.J.: Two turnpike results for a discrete-time optimal control systems. Nonlinear Anal. 71(12), 902–909 (2009) 48. Zaslavski, A.J.: Stability of a turnpike phenomenon for a discrete-time optimal control systems. J. Optim. Theory Appl. 145(3), 597–612 (2010) 49. Zaslavski, A.J.: Turnpike properties of approximate solutions for discrete-time control systems. Commun. Math. Anal. 11, 36–45 (2011) 50. Zaslavski, A.J.: Structure of approximate solutions for a class of optimal control systems. J. Math. Appl. 34, 1–14 (2011) 51. Zaslavski, A.J.: Stability of a turnpike phenomenon for a class of optimal control systems in metric spaces. Numerical Algebra Control Optim. 1, 245–260 (2011) 52. Zaslavski, A.J.: The existence and structure of approximate solutions of dynamic discrete time zero-sum games. J. Nonlinear Convex Anal. 12, 49–68 (2011) 53. Zaslavski, A.J.: A generic turnpike result for a class of discrete-time optimal control systems. Dyn. Contin. Discrete Impuls. Syst. Ser. B 19, 225–265 (2012) 54. Zaslavski, A.J.: Existence and structure of solutions for a class of optimal control systems with discounting arising in economic dynamics. Nonlinear Anal. Real World Appl. 13(4), 1749–1760 (2012) 55. Zaslavski, A.J.: Structure of Solutions of Variational Problems. SpringerBriefs in Optimization, New York (2013) 56. Zaslavski, A.J.: Structure of Approximate Solutions of Optimal Control Problems. SpringerBriefs in Optimization, NewYork (2013) 57. Zaslavski, A.J., Leizarowitz, A.: Optimal solutions of linear control systems with nonperiodic integrands. Math. Op. Res. 22(3), 726–746 (1997)

Index

A Admissible program, 1 trajectory, 9, 47 Approximate solution, 1–4, 11, 12, 47 Asymptotic turnpike property, 2, 4, 10, 65, 69 Autonomous discrete-time control system, 1 B Baire category approach, 2 Bounded function, 4, 6, 9, 65 sequence, 4 C Cardinality of a set, 6, 13, 66 Compact metric space, 1, 3, 9, 47, 65 Concave function, 5–7 Convex function, 2, 6 set, 2, 68 D Discrete -time optimal control system, 1, 65, 66 -time problem, 3–6 E Economic dynamics, 1, 4, 9, 10, 66 Euclidean space, 2, 5 G Generic problem, 2 Good program, 4, 10, 79, 96 sequence, 2, 4, 10 H Hausdorff metric, 66

I Infinite interval, 1 Interior point, 3, 5–7, 10 L Lower semicontinuous function, 66–68 M Metric, 1, 3, 5, 9, 47, 65, 66 Model of economic growth, 1, 2 N Nonstationary trajectory, 2 O Objective function, 2, 6, 9, 11, 47, 65, 68, 69 Optimal control problem, 1, 2, 9, 47, 65, 67 Optimality criterion, 1, 9, 47 Overtaking optimal program, 4, 5, 48, 49 P Program, 3, 4, 10, 19, 69, 70 R Relative topology, 67, 69, 70, 76, 96 S Singleton, 2, 47, 68 Strictly concave function, 2, 5 T Turnpike phenomenon, 2, 9, 47, 65, 69 property, 1, 2, 6, 65, 67–69 result, 4 theory, 2, 4 U Uniform convergence, 5, 65 Upper semicontinuous function, 1, 3, 10 Utility function, 1, 66

A. J. Zaslavski, Stability of the Turnpike Phenomenon in Discrete-Time Optimal Control Problems, SpringerBriefs in Optimization, DOI 10.1007/978-3-319-08034-5, © The Author 2014

109