Optimal control problems arising in forest management 9783030235864, 9783030235871

Table of contents :
Preface......Page 5
Contents......Page 7
1.1 Convex Discrete-Time Problems......Page 9
1.2 The Turnpike Phenomenon......Page 16
2.1 The Forest Management Problem......Page 18
2.2 Infinite Horizon Problems Without Discounting......Page 23
2.3 Auxiliary Results......Page 28
2.4 Proofs of Theorems 2.6 and 2.7......Page 30
2.5 Proof of Theorem 2.9......Page 31
2.6 Proof of Theorem 2.10......Page 33
2.7 Infinite Horizon Problems with Discounting......Page 34
2.8 An Auxiliary Result for Theorem 2.13......Page 35
2.10 An Application to the Forest Management Problem......Page 37
3.1 Preliminaries and Main Results......Page 39
3.2 Auxiliary Results......Page 44
3.3 Proof of Theorem 3.2......Page 50
3.4 Proof of Theorem 3.3......Page 51
3.5 Proof of Theorem 3.4......Page 53
3.6 Proof of Theorem 3.5......Page 56
3.7 Proof of Theorem 3.6......Page 57
3.8 Stability Results......Page 59
3.9 Agreeable Programs......Page 64
4.1 Preliminaries......Page 69
4.2 Equivalence of the Turnpike Properties......Page 70
4.3 Generic Results......Page 73
5.1 Preliminaries......Page 78
5.2 Lagrange Problems......Page 83
5.3 An Auxiliary Result for Theorem 5.13......Page 88
5.4 Proof of Theorem 5.13......Page 90
5.5 The First Class of Bolza Problems......Page 94
5.6 An Auxiliary Result for Theorem 5.15......Page 95
5.7 Proof of Theorem 5.15......Page 98
5.8 The Second Class of Bolza Problems......Page 103
5.9 Auxiliary Results for Theorem 5.17......Page 104
5.10 Proof of Theorem 5.17......Page 107
6.1 Preliminaries......Page 114
6.2 Auxiliary Results......Page 117
6.3 Turnpike Results......Page 123
6.4 Generic Results......Page 127
References......Page 136
Index......Page 140

Citation preview

SPRINGER BRIEFS IN OPTIMIZATION

Alexander J. Zaslavski

Optimal Control Problems Arising in Forest Management

123

SpringerBriefs in Optimization Series Editors Sergiy Butenko Mirjam Dür Panos M. Pardalos János D. Pintér Stephen M. Robinson Tamás Terlaky My T. Thai

SpringerBriefs in Optimization showcases algorithmic and theoretical techniques, case studies, and applications within the broad-based field of optimization. Manuscripts related to the ever-growing applications of optimization in applied mathematics, engineering, medicine, economics, and other applied sciences are encouraged.

More information about this series at http://www.springer.com/series/8918

Alexander J. Zaslavski

Optimal Control Problems Arising in Forest Management

123

Alexander J. Zaslavski Department of Mathematics The Technion – Israel Institute of Techn Rishon LeZion, Israel

ISSN 2190-8354 ISSN 2191-575X (electronic) SpringerBriefs in Optimization ISBN 978-3-030-23586-4 ISBN 978-3-030-23587-1 (eBook) https://doi.org/10.1007/978-3-030-23587-1 © The Author(s), under exclusive license to Springer Nature Switzerland AG 2019 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG. The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

The growing importance of optimal control has been recognized in recent years. This is due not only to impressive theoretical developments but also because of numerous applications to engineering, economics, life sciences, etc. This book is devoted to the study of a class of optimal control problems arising in forest management. The forest management problem is an important and interesting topic in mathematical economics that was studied by many researchers including Nobel laureate P.A. Samuelson [68]. As usual, for this problem, the existence of optimal solutions over infinite horizon and the structure of solutions on finite intervals are under consideration. In our books [84, 86], we study a class of discrete-time optimal control problems which describe many models of economic dynamics except for the model of forest management. This happens because some assumptions posed in [84, 86], which are true for many models of economic dynamics, do not hold for the model of forest management. By this reason, the forest management problem is not a particular case of general models of economic dynamics and is studied separately in the literature. In this book, we study the forest management problem using the approach introduced and employed in our research [80, 81, 83]. Namely, we analyze a class of optimal control problems which contains, as a particular case, the forest management problem. For this class of problems, we show the existence of optimal solutions over infinite horizon and study the structure of approximate solutions on finite intervals and their turnpike properties, the stability of the turnpike phenomenon, and the structure of approximate solutions on finite intervals in the regions close to the endpoints. In Chap. 1, we provide some preliminary knowledge on turnpike properties. The forest management problem is discussed in Chap. 2, which also contains existence results for infinite horizon problems. In Chap. 3, we establish the turnpike properties of approximate solutions. Chapter 4 contains generic turnpike results. We consider a class of optimal control problems which is identified with a complete metric space of objective functions and show the existence of a Gδ everywhere dense subset of the metric space, which is a countable intersection of open everywhere dense sets, such that the turnpike property holds for any of its element. Chapter 5 is devoted to

v

vi

Preface

the study of the structure of approximate solutions on finite intervals in the regions close to the endpoints. In Chap. 6, we again consider the forest management problem and show that the results of Chaps. 3 and 5 are true for it. Rishon LeZion, Israel October 30, 2018

Alexander J. Zaslavski

Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Convex Discrete-Time Problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 The Turnpike Phenomenon. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 1 8

2

Infinite Horizon Optimal Control Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 The Forest Management Problem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Infinite Horizon Problems Without Discounting . . . . . . . . . . . . . . . . . . . . . 2.3 Auxiliary Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Proofs of Theorems 2.6 and 2.7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Proof of Theorem 2.9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 Proof of Theorem 2.10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Infinite Horizon Problems with Discounting . . . . . . . . . . . . . . . . . . . . . . . . . 2.8 An Auxiliary Result for Theorem 2.13 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9 Proof of Theorem 2.13 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10 An Application to the Forest Management Problem . . . . . . . . . . . . . . . . .

11 11 16 21 23 24 26 27 28 30 30

3

Turnpike Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Preliminaries and Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Auxiliary Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Proof of Theorem 3.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Proof of Theorem 3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Proof of Theorem 3.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Proof of Theorem 3.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 Proof of Theorem 3.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8 Stability Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.9 Agreeable Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

33 33 38 44 45 47 50 51 53 58

4

Generic Turnpike Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Equivalence of the Turnpike Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Generic Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

63 63 64 67

vii

viii

Contents

5

Structure of Solutions in the Regions Close to the Endpoints . . . . . . . . . . 73 5.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 5.2 Lagrange Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 5.3 An Auxiliary Result for Theorem 5.13 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 5.4 Proof of Theorem 5.13 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 5.5 The First Class of Bolza Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 5.6 An Auxiliary Result for Theorem 5.15 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 5.7 Proof of Theorem 5.15 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 5.8 The Second Class of Bolza Problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5.9 Auxiliary Results for Theorem 5.17 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 5.10 Proof of Theorem 5.17 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

6

Applications to the Forest Management Problem . . . . . . . . . . . . . . . . . . . . . . . 6.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Auxiliary Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Turnpike Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Generic Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

109 109 112 118 122

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

Chapter 1

Introduction

The study of optimal control problems and variational problems defined on infinite intervals and on sufficiently large intervals has been a rapidly growing area of research [9, 10, 16, 18, 24, 37, 43, 48, 57, 59, 66, 76, 84–86] which has various applications in engineering [1, 44, 87], in models of economic growth [2, 5, 14, 20, 23, 28, 38–40, 47, 52–55, 60, 61, 63, 65, 67, 68, 70, 77, 80, 81, 83], in the game theory [29, 32, 42, 74, 82], in infinite discrete models of solid-state physics related to dislocations in one-dimensional crystals [6, 71], and in the theory of thermodynamical equilibrium for materials [19, 45, 50, 51]. Discrete-time problems were considered in [7, 8, 13, 21, 26, 30, 33, 72, 73, 78, 79] while continuous-time problems were studied in [3, 4, 11, 12, 15, 17, 22, 25, 27, 31, 41, 46, 49, 56, 58, 62, 69, 75]. In this chapter we discuss turnpike properties for a class of simple convex dynamic optimization problems.

1.1 Convex Discrete-Time Problems Let R n be the n-dimensional Euclidean space with the inner product ·, · which induces the norm |x| =

 n 

1/2 xi2

, x = (x1 , . . . , xn ) ∈ R n .

i=1

Let v : R n × R n → R 1 be bounded from below function. We consider the minimization problem

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2019 A. J. Zaslavski, Optimal Control Problems Arising in Forest Management, SpringerBriefs in Optimization, https://doi.org/10.1007/978-3-030-23587-1_1

1

2

1 Introduction T −1

v(xi , xi+1 ) → min,

(P0 )

i=0

such that {xi }Ti=0 ⊂ R n and x0 = z, xT = y, where T is a natural number and the points y, z ∈ R n . The interest in discrete-time optimal problems of type (P0 ) stems from the study of various optimization problems which can be reduced to it, e.g., continuous-time control systems which are represented by ordinary differential equations whose cost integrand contains a discounting factor [43], tracking problems in engineering [1, 44, 87], the study of Frenkel-Kontorova model [6, 71], and the analysis of a long slender bar of a polymeric material under tension in [19, 45, 50, 51]. Optimization problems of the type (P0 ) were considered in [72, 73]. In this section we suppose that the function v : R n × R n → R 1 is strictly convex and differentiable and satisfies the growth condition v(y, z)/(|y| + |z|) → ∞ as |y| + |z| → ∞.

(1.1)

We intend to study the behavior of solutions of the problem (P0 ) when the points y, z and the real number T vary and T is sufficiently large. Namely, we are interested to study a turnpike property of solutions of (P0 ) which is independent of the length of the interval T , for all sufficiently large intervals. To have this property means, roughly speaking, that solutions of the optimal control problems are determined mainly by the objective function v and are essentially independent of T , y, and z. Turnpike properties are well known in mathematical economics. The term was first coined by Samuelson in 1948 (see [67]) where he showed that an efficient expanding economy would spend most of the time in the vicinity of a balanced equilibrium path (also called von Neumann path). This property was further investigated for optimal trajectories of models of economic dynamics (see, for example, [47, 53, 65] and the references mentioned there). Many turnpike results are collected in [76, 84, 86]. In order to meet our goal we consider the auxiliary optimization problem v(x, x) → min, x ∈ R n .

(P1 )

It follows from the strict convexity of v and (1.1) that the problem (P1 ) has a unique solution x. ¯ Let ∇v(x, ¯ x) ¯ = (l1 , l2 ),

(1.2)

where l1 , l2 ∈ R n . Since x¯ is a solution of (P1 ) it follows from (1.2) that for each h ∈ Rn, l1 , h + l2 , h = (l1 , l2 ), (h, h) = lim t −1 [v(x¯ + th, x¯ + th) − v(x, ¯ x)] ¯ ≥ 0. t→0+

1.1 Convex Discrete-Time Problems

3

Thus l1 + l2 , h ≥ 0 for all h ∈ R n , l2 = −l1 and ∇v(x, ¯ x) ¯ = (l1 , −l1 ),

(1.3)

For each (y, z) ∈ R n × R n set L(y, z) = v(y, z) − v(x, ¯ x) ¯ − ∇v(x, ¯ x), ¯ (y − x, ¯ z − x) ¯ = v(y, z) − v(x, ¯ x) ¯ − l1 , y − z.

(1.4)

It is not difficult to verify that the function L : R n × R n → R 1 is differentiable and strictly convex. It follows from (1.1) and (1.4) that L(y, z)/(|y| + |z|) → ∞ as |y| + |z| → ∞.

(1.5)

Since the functions v and L are both strictly convex it follows from (1.4) that L(y, z) ≥ 0 for all (y, z) ∈ R n × R n

(1.6)

L(y, z) = 0 if and only if y = x, ¯ z = x. ¯

(1.7)

and

We claim that the function L : R n × R n → R 1 has the following property: n n (C) If a sequence {(yi , zi )}∞ i=1 ⊂ R × R satisfies the equality lim L(yi , zi ) = 0,

i→∞

then ¯ x). ¯ lim (yi , zi ) = (x,

i→∞

n n Assume that a sequence {(yi , zi )}∞ i=1 ⊂ R × R satisfies limi→∞ L(yi , zi ) = 0. In view of (1.5) the sequence {(yi , zi )}∞ is bounded. Let (y, z) be its limit point. i=1 Then it is easy to see that the equality

L(y, z) = lim L(yi , zi ) = 0 i→∞

holds and by (1.7) (y, z) = (x, ¯ x). ¯ This implies that (x, ¯ x) ¯ = limi→∞ (yi , zi ). Thus the property (C) holds, as claimed.

4

1 Introduction

Consider an auxiliary minimization problem T −1

L(xi , xi+1 ) → min,

(P2 )

i=0

such that {xi }Ti=0 ⊂ R n and x0 = z, xT = y, where T is a natural number and the points y, z ∈ R n . It follows from (1.4) that for any integer T ≥ 1 and any sequence {xi }Ti=0 ⊂ R n , we have T −1

L(xi , xi+1 ) =

T −1

i=0

v(xi , xi+1 ) − T v(x, ¯ x) ¯ −

i=0

=

T −1

T −1

l1 , xi − xi+1 

i=0

v(xi , xi+1 ) − T v(x, ¯ x) ¯ − l1 , x0 − xT .

(1.8)

i=0

Relation (1.8) implies that the problems (P0 ) and (P2 ) are equivalent. Namely, {xi }Ti=0 ⊂ R n is a solution of the problem (P0 ) if and only if it is a solution of the problem (P2 ). Let T be a natural number and Δ ≥ 0. A sequence {xi }Ti=0 ⊂ R n is called (Δ)optimal if for any sequence {xi }Ti=0 ⊂ R n satisfying xi = xi , i = 0, T the inequality T −1

v(xi , xi+1 ) ≤

T −1

i=0

v(xi , xi+1 )+Δ

i=0

holds. Clearly, if a sequence {xi }Ti=0 ⊂ R n is (0)-optimal, then it is a solution of the problems (P0 ) and (P2 ) with z = x0 and y = xT . We prove the following existence result. Proposition 1.1 Let T > 1 be an integer and y, z ∈ R n . Then the problem (P0 ) has a solution. Proof It is sufficient to show that the problem (P2 ) has a solution. Consider a sequence {xi }Ti=0 ⊂ R n such that x0 = z, xT = y. Set M1 =

T −1

L(xi , xi+1 )

i=0

and M2 = inf

T −1  i=0

 L(xi , xi+1 ) :

{xi }Ti=0

⊂ R , x0 = z, xT = y . n

(1.9)

1.1 Convex Discrete-Time Problems

5

Clearly, 0 ≤ M2 ≤ M1 . We may assume without loss of generality that M2 < M1 .

(1.10)

(k)

There exists a sequence {xi }Ti=0 ⊂ R n , k = 1, 2, . . . such that for any natural number k, (k)

(k)

x0 = z, xT = y

(1.11)

and lim

k→∞

T −1

(k)

(k)

L(xi , xi+1 ) = M2 .

(1.12)

i=0

In view of (1.10), (1.11), and (1.12) we may assume that T −1

(k)

(k)

L(xi , xi+1 ) < M1 for all integers k ≥ 1.

(1.13)

i=0

By (1.13) and (1.5) there is M3 > 0 such that |xi(k) | ≤ M3 for all i = 0, . . . , T and all integers k ≥ 1.

(1.14)

In view of (1.14), extracting subsequences, using diagonalization process and reindexing, if necessary, we may assume without loss of generality that for each i ∈ {0, . . . , T } there exists (k)

 xi = lim xi .

(1.15)

xT = y.  x0 = z, 

(1.16)

k→∞

By (1.15) and (1.11),

It follows from (1.15) and (1.12) that T −1 i=0

L( xi ,  xi+1 ) = M2 .

6

1 Introduction

Together with (1.16) and (1.9) this implies that { xi }Ti=0 is a solution of the problem (P2 ). This completes the proof of Proposition 1.1. Denote by Card (A) the cardinality of a set A. The following result establishes a turnpike property for approximate solutions of the problem (P0 ). Proposition 1.2 Let M1 , M2 ,  be positive numbers. Then there exists a natural number k0 such that for each integer T > 1 and each (M1 )-optimal sequence {xi }Ti=0 ⊂ R n satisfying |x0 | ≤ M2 , |xT | ≤ M2 ,

(1.17)

the following inequality holds: ¯ + |xi+1 − x| ¯ > }) ≤ k0 . Card({i ∈ {0, . . . , T − 1} : |xi − x| Proof By condition (C) there is δ > 0 such that for each (y, z) ∈ R n ×R n satisfying L(y, z) ≤ δ,

(1.18)

|y − x| ¯ + |z − x| ¯ ≤ .

(1.19)

we have

Set ¯ + M2 }. M3 = sup{L(y, z) : y, z ∈ R n and |y| + |z| ≤ |x|

(1.20)

Choose a natural number k0 > δ −1 (M1 + 2M3 ).

(1.21)

Assume that an integer T > 1 and that an (M1 )-optimal sequence {xi }Ti=0 ⊂ R n satisfies (1.17). Set ¯ i = 1, . . . , T − 1. y0 = x0 , yT = xT , yi = x, Since the sequence {xi }Ti=0 is (M1 )-optimal it follows from (1.22) that T −1 i=0

v(xi , xi+1 ) ≤

T −1 i=0

v(yi , yi+1 ) + M1 .

(1.22)

1.1 Convex Discrete-Time Problems

7

Together with (1.7), (1.8), and (1.22) this implies that T −1

L(xi , xi+1 ) ≤

i=0

T −1

L(yi , yi+1 ) + M1 = L(x0 , x) ¯ + L(x, ¯ xT ) + M1 .

i=0

Combined with (1.17) and (1.20) this implies that T −1

L(xi , xi+1 ) ≤ M1 + 2M3 .

i=0

It follows from the choice of δ (see (1.18) and (1.19)), (1.21), and the inequality above that ¯ + |xi+1 − x| ¯ > }) Card({i ∈ {0, . . . , T − 1} : |xi − x| ≤ Card({i ∈ {0, . . . , T − 1} : L(xi , xi+1 ) > δ}) ≤ δ −1

T −1

L(xi , xi+1 ) ≤ δ −1 (M1 + 2M3 ) ≤ k0 .

i=0

Proposition 1.2 is proved. Proposition 1.2 implies the following turnpike result for exact solutions of the problem (P0 ). Proposition 1.3 Let M,  be positive numbers. Then there exists a natural number k0 such that for each integer T > 1, each y, z ∈ R n satisfying |y|, |z| ≤ M, and each optimal solution {xi }Ti=0 ⊂ R n of the problem (P0 ) the following inequality holds: ¯ + |xi+1 − x| ¯ > }) ≤ k0 . Card({i ∈ {0, . . . , T − 1} : |xi − x| It is easy now to see that the optimal solution {xi }Ti=0 of the problem (P0 ) spends most of the time in an -neighborhood of x. ¯ By Proposition 1.3 the number of all integers i ∈ {0, . . . , T − 1}, such that xi does not belong to this -neighborhood, does not exceed the constant k0 which depends only on M,  and does not depend on T . Following the tradition, the point x¯ is called the turnpike. Moreover we can show that the set {i ∈ {0 . . . , T } : |xi − x| ¯ > } is contained in the union of two intervals [0, k1 ]∪[T −k1 , T ], where k1 is a constant depending only on M, .

8

1 Introduction

1.2 The Turnpike Phenomenon In the previous section we proved the turnpike result for rather simple class of discrete-time problems. The problems of this class are unconstrained and their objective functions are convex and differentiable. It should be mentioned that, using the methods of convex analysis [64], this result can be extended to the class of objective functions which are merely convex. In the turnpike theory and its applications to the mathematical economics our goal is to establish the turnpike property for constrained optimal control problems. In particular, in this book and in [84, 86] we study the structure of approximate solutions of an autonomous discretetime control system with a compact metric space of states X. This control system is described by a nonempty closed set Ω ⊂ X × X which determines a class of admissible trajectories (programs) and a bounded upper semicontinuous function v : Ω → R 1 which determines an optimality criterion. We study the problem T −1

T −1 v(xi , xi+1 ) → max, {(xi , xi+1 )}i=0 ⊂ Ω, x0 = z, xT = y,

(P )

i=0

where T ≥ 1 is an integer and the points y, z ∈ X. Clearly, these constrained problems are more difficult and less understood than their unconstrained prototypes. They are also more realistic from the point of view of mathematical economics. In the turnpike theory the objective function v possesses the turnpike property (TP) if there exists a point x¯ ∈ X (a turnpike) such that the following condition holds: For each positive number  there exists an integer L ≥ 1 such that for each integer T ≥ 2L and each solution {xi }Ti=0 ⊂ X of the problem (P ) the inequality ρ(xi , x) ¯ ≤  is true for all i = L, . . . , T − L. It should be mentioned that the constant L depends neither on T nor on y, z. The turnpike phenomenon has the following interpretation. If one wishes to reach a point A from a point B by a car in an optimal way, then one should turn to a turnpike, spend most of the time on it, and then leave the turnpike to reach the required point. It should be mentioned that in general a turnpike is not necessarily a singleton [76]. Nevertheless problems of the type (P ) for which the turnpike is a singleton are of great importance because of the following reasons: there are many models of economic growth for which a turnpike is a singleton; if a turnpike is a singleton, then approximate solutions of (P ) have very simple structure and this is very important for applications; if a turnpike is a singleton, then it can be easily calculated as a solution of the problem v(x, x) → max, (x, x) ∈ Ω. The turnpike property is very important for applications. Suppose that our objective function v has the turnpike property and we know a finite number of “approximate” solutions of the problem (P ). Then we know the turnpike x, ¯ or at least its approximation, and the constant L (see the definition of (TP)) which is an estimate for the time period required to reach the turnpike. This information can

1.2 The Turnpike Phenomenon

9

be useful if we need to find an “approximate” solution of the problem (P ) with a new time interval [m1 , m2 ] and the new values z, y ∈ X at the end points m1 and m2 . Namely instead of solving this new problem on the “large” interval [m1 , m2 ] we can find an “approximate” solution of the problem (P ) on the “small” interval [m1 , m1 + L] with the values z, x¯ at the end points and an “approximate” solution of the problem (P ) on the “small” interval [m2 − L, m2 ] with the values x, ¯ y at the end points. Then the concatenation of the first solution, the constant sequence xi = x, ¯ i = m1 + L, . . . , m2 − L and the second solution is an “approximate” solution of the problem (P ) on the interval [m1 , m2 ] with the values z, y at the end points. Sometimes as an “approximate” solution of the problem (P ) we can choose 2 any admissible sequence {xi }m i=m1 satisfying xm1 = z, xm2 = y and xi = x¯ for all i = m1 + L, . . . , m2 − L. In this book we study a class of optimal control problems arising in the forest management. The forest management problem is an important and interesting topic in mathematical economics studies by many researchers. We consider the discrete time model for the optimal management of a forest of total area S occupied by k species I = {1, . . . , k} with maturity ages of n1 , . . . , nk years respectively. This model and its versions were studied in [20, 38–40, 54, 55, 61, 68, 80, 81, 83]. Here the optimal harvesting of a mixed forest composed of multiple species is studied, each one having a different maturity age, where only mature trees can be harvested. As usual, for this problem the existence of optimal solutions over infinite horizon and the structure of solutions on finite intervals are under consideration. In our books [84, 86] we study a class of discrete-time optimal control problems which describe many models of economic dynamics except of the model of the forest management. This happens because some assumptions posed in [84, 86], which are true for many models of economic dynamics, do not hold for the model of the forest management. Namely, in [84, 86] we studied optimal control problems of type (P ) for which the turnpike property holds with the turnpike x¯ such that (x, ¯ x) ¯ is an interior point of the set Ω ⊂ X × X. This assumption, which plays an important role in [84, 86], holds for many models of economic dynamics except of the forest management problem considered in this book. By this reason, the forest management problem is not a particular case of general models of economic dynamics and is studied separately in the literature. In this book we study the forest management problem using the approach introduced and employed in our research [80, 81, 83]. Namely, we analyze a class of optimal control problems which contains, as a particular case, the forest management problem. For this class of problems we show the existence of optimal solutions over infinite horizon, study the structure of approximate solutions on finite intervals and their turnpike properties, the stability of the turnpike phenomenon, and the structure of approximate solutions on finite intervals in the regions close to the end points.

Chapter 2

Infinite Horizon Optimal Control Problems

In this chapter we present the forest management problem and study some properties of its solutions. This problem is a particular case of a general optimal control problem which is also introduced in this chapter. The corresponding infinite horizon problems without discounting as well as with discounting are considered. The results of the chapter were obtained from [80, 81].

2.1 The Forest Management Problem We consider a discrete time model for the optimal management of a forest of total area S occupied by k species I = {1, . . . , k} with maturity ages of n1 , . . . , nk years respectively. This model and its versions were studied in [20, 38–40, 54, 55, 61, 68, 80, 81, 83]. Here the optimal harvesting of a mixed forest composed of multiple species is studied, each one having a different maturity age, where only mature trees can be harvested. j For each period t = 0, 1, . . . we denote xi (t) ≥ 0 the area covered by trees of species i that are j years old with j = 1, . . . , ni and x¯i (t) ≥ 0 the area occupied by over-mature trees (older than ni ). We must decide how much land vi (t) ≥ 0 to harvest and how to reallocate this land to new seedlings. Assuming that only mature trees can be harvested we must have vi (t) ≤ x¯i (t) + xini (t),

(2.1)

and then the area not harvested in that period will comprise the over-mature trees at the next step, namely x¯i (t + 1) = x¯i (t) + xini (t) − vi (t).

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2019 A. J. Zaslavski, Optimal Control Problems Arising in Forest Management, SpringerBriefs in Optimization, https://doi.org/10.1007/978-3-030-23587-1_2

(2.2)

11

12

2 Infinite Horizon Optimal Control Problems

The fact that immature trees cannot be harvested is represented by j +1

xi

j

(t + 1) = xi (t), j = 1, . . . , ni − 1.

(2.3)

 The total harvested area i∈I vi (t) is allocated to new seedlings which is expressed by the equation   xi1 (t + 1) = vi (t). (2.4) i∈I

i∈I

In the sequel we use the notation xini +1 = x¯i , i ∈ I.

(2.5)

A representation of the forest in terms of the age distribution at time t is provided by the state x(t) = (x1 (t), . . . , xk (t)), where xi (t) = (xi1 (t), . . . , xini (t), xini +1 (t)), i ∈ I describes the areas occupied in year t by trees of species i with ages 1, 2, . . . , ni and over ni . The first and last components of each vector xi (t) are controlled by the sowing and harvesting policies. Note that we do not control x(0) which corresponds to the initial state reflecting the age class composition of the forest at time t = 0. m m  Let R+ = {x = (x1 , . . . , xm ) ∈ NR : xi ≥ 0, i = 1, . . . , m} and let N = (n + 1). Every vector x ∈ R is represented as x = (x1 , . . . , xk ), where i∈I i ni ni +1 1 n +1 i )∈R for all integers i = 1, . . . , k. xi = (xi , . . . , xi , xi N such that Denote by Δ the set of all x ∈ R+ ⎡ ⎤ i +1  n j ⎣ xi ⎦ = S. (2.6) i∈I

j =1

Now we give a formal description of the model. A sequence {x(t)}∞ t=0 ⊂ Δ is called a program if for all integers t ≥ 0 and all k. i ∈ I Eqs. (2.1)–(2.4) hold (see (2.5)) with some v(t) = (v1 (t), . . . , vk (t)) ∈ R+ T2 Let the integers T1 , T2 satisfy 0 ≤ T1 < T2 . A sequence {x(t)}t=T1 ⊂ Δ is called a program if Eqs. (2.1)–(2.4) hold for all i ∈ I and for all integers t = T1 , . . . , T2 −1 k. (see (2.5)) with some v(t) = (v1 (t), . . . , vk (t)) ∈ R+ An alternative equivalent definition of a program can be given with the help of the transition possibility. Put

j +1 j = xi for all i ∈ I and all j ∈ {1, . . . , ni } \ {ni } Ω = (x, y) ∈ Δ × Δ : yi  and for all i ∈ I, xini +1 + xini − yini +1 ≥ 0 . (2.7)

2.1 The Forest Management Problem

13

Clearly, if (x, y) ∈ Ω, then  i∈I

yi1 =

 n(i)+1 (xi + xini − yin(i)+1 ).

(2.8)

i∈I

It is easy to see that a sequence {x(t)}∞ t=0 ⊂ Δ is a program if and only if (x(t), x(t + 1)) ∈ Ω for all integers t ≥ 0. Let the integers T1 , T2 satisfy 0 ≤ T1 < T2 . It is easy to see that a sequence 2 {x(t)}Tt=T ⊂ Δ is a program if and only if (x(t), x(t + 1)) ∈ Ω for all t = 1 T1 , . . . , T2 − 1. For each (x, y) ∈ Ω set V (x, y) = (v1 (x, y), . . . , vk (x, y)), where for i = 1, . . . , k, vi (x, y) = xini +1 + xini − yini +1 . Define  Δ0 = v ∈

k R+

:

k 

 vi ≤ S .

i=1

In this book we assume that a benefit at moment t = 0, 1, . . . is represented by an upper semicontinuous function wt : Δ0 → R 1 and at a moment t = 0, 1, . . . , wt (V (x, y)) is the benefit obtained today if the forest today is x and the forest tomorrow is y, where (x, y) ∈ Ω. Note that usually in the literature it is assumed that for t = 0, 1, . . . , wt (V (x, y)) =

k 

W (i) (vi (x, y)), (x, y) ∈ Ω

i=1

where W i : [0, ∞) → R 1 , i = 1, . . . , k are strictly concave, smooth, and increasing functions. Clearly, Δ is a compact set in R N , Ω is a closed subset of Δ × Δ, and wt ◦ V : Ω → R 1 is an upper semicontinuous function for all integers t ≥ 0. Set n¯ = max{ni : i ∈ I }.

(2.9)

Our model has an important property established by the following result. Its proof is given in [80]. +n+1 ¯ Proposition 2.1 Let x, y ∈ Δ. Then there exists a program {x(t)}N such that t=0 x(0) = x and x(N + n¯ + 1) = y.

14

2 Infinite Horizon Optimal Control Problems

Proof Put x(0) = x. For all integers t = 0, . . . , N − 1 define j +1

xi

j

(t + 1) = xi (t), i ∈ I, j ∈ {1, . . . , ni } \ {ni },

xi1 (t + 1) = 0, i ∈ I, xini +1 (t + 1) = xini +1 (t) + xini (t), i ∈ I.

(2.10)

It is easy to see that x(t) ∈ Δ for all t = 0, . . . , N, {x(t)}N t=0 is a program and j

xi (N ) = 0, i ∈ I, j = 1, . . . , ni , 

xini +1 (N ) = S.

(2.11)

i∈I

For each s = 1, . . . , n¯ put Is = {i ∈ I : ni = s}.

(2.12)

(Note that for some integers s we can have Is = ∅.) We assume that sum over empty set is zero. Define x(N + 1) ∈ Δ as follows. Set xi1 (N + 1) = yini +1 , i ∈ In¯ ,

(2.13)

xi1 (N + 1) = 0, i ∈ I \ In¯ . For i ∈ I and all integers j satisfying 1 < j ≤ ni set j

xi (N + 1) = 0.

(2.14)

ui ∈ [0, xini +1 (N )], i ∈ I

(2.15)

Clearly, there exist

such that  i∈I

ui =



xi1 (N + 1).

(2.16)

i∈In¯

Put xini +1 (N + 1) = xini +1 (N ) − ui , i ∈ I.

(2.17)

2.1 The Forest Management Problem

15

By (2.13)–(2.17), x(N + 1) ∈ Δ and (x(N ), x(N + 1)) ∈ Ω. Assume that q is an integer, 1 ≤ q < n, ¯ and we have defined a program N +q {x(t)}t=0 such that the following properties hold: (P1)

If an integer i ∈ ∪{Is :

an integer s satisfies 1 ≤ s ≤ n¯ − q},

j

(P2)

then xi (N + q) = 0 for all integers j satisfying 1 ≤ j ≤ ni ; If an integer s satisfies n¯ ≥ s > n¯ − q and i ∈ Is , then n +1+p−(q+s−n) ¯

p

xi (N + q) = yi i

, p = 1, . . . , q + s − n, ¯

p

xi (N +q) = 0 for all integers p satisfying q+s−n¯ < p ≤ ni .

(2.18) (2.19)

(Note that for q = 1 our assumptions hold.) Define x(N +q +1) ∈ Δ as follows. Let i ∈ I . If i ∈ Is , where 1 ≤ s ≤ n−q ¯ −1, then set j

xi (N + q + 1) = 0 for all integers j satisfying 1 ≤ j ≤ ni .

(2.20)

, then set If i ∈ In−q ¯ xi1 (N + q + 1) = yini +1 , xi (N + q + 1) = 0 p

(2.21)

for all integers p satisfying 1 < p ≤ ni . If i ∈ Is , where n¯ ≥ s > n¯ − q, then set (see (2.18)) p+1

xi

p

n +1+p−(q+s−n) ¯

(N + q + 1) = xi (N + q) = yi i

n +1−(q+s−n) ¯

xi1 (N + q + 1) = yi i

, p = 1, . . . , q + s − n, ¯ (2.22)

,

p

xi (N + q + 1) = 0 for all integers p satisfying q + 1 + s − n¯ < p ≤ ni . It is not difficult to see that   n +1 xi1 (N + q + 1) ≤ xi i (N + q). i∈I

i∈I

(2.23) (2.24)

(2.25)

16

2 Infinite Horizon Optimal Control Problems

Therefore there exists   xini +1 (N + q + 1) ∈ 0, xini +1 (N + q) , i ∈ I, such that 

[xini +1 (N + q) − xini +1 (N + q + 1)] =

i∈I



xi1 (N + q + 1).

(2.26)

(2.27)

i∈I

Clearly, x(N + q + 1) ∈ Δ, (x(N + q), x(N + q + 1)) ∈ Ω and the assumption made for q holds also for q + 1. Thus by induction we have +n¯ ¯ constructed a program {x(t)}N t=0 such that (P1) and (P2) hold for all q = 1, . . . , n. Consider the state x(N + n). ¯ Let i ∈ Is where 1 ≤ s ≤ n. ¯ By (P1) and (P2), p

p+1

xi (N + n) ¯ = yi

, p = 1, . . . , ni .

(2.28)

Set x(N + n¯ + 1) = y.

(2.29)

By (2.28) and (2.29), (x(N + n), ¯ x(N + n¯ + 1)) ∈ Ω. Proposition 2.1 is proved.

2.2 Infinite Horizon Problems Without Discounting Let (K, ρ) be a compact metric space and Ω be a nonempty closed subset of K ×K. A sequence {xt }∞ t=0 ⊂ K is called an (Ω)-program (or a program if the set Ω is understood) if (xt , xt+1 ) ∈ Ω for all integers t ≥ 0. 2 Let the integers T1 , T2 satisfy T1 < T2 . A sequence {xt }Tt=T ⊂ K is called an 1 (Ω)-program (or a program if the set Ω is understood) if (xt , xt+1 ) ∈ Ω for all integers t satisfying T1 ≤ t < T2 . For each integer t ≥ 0, let ut : Ω → R 1 be a bounded upper semicontinuous function. For each pair of integers T1 , T2 satisfying 0 ≤ T1 < T2 and each y, z ∈ K we consider the optimization problems

2.2 Infinite Horizon Problems Without Discounting T 2 −1

17

ui (xi , xi+1 ) → max,

i=T1 2 {xi }Ti=T ⊂ K, xT1 = y, xT2 = z 1

and T 2 −1

ui (xi , xi+1 ) → max,

i=T1 2 {xi }Ti=T ⊂ K, xT1 = y. 1

The results of this section were obtained from [81]. For each integer t ≥ 0 set ut  = sup{|ut (z)| : z ∈ Ω}

(2.30)

sup{ut  : t = 0, 1, . . . } < ∞.

(2.31)

and assume that

In the sequel we assume that supremum of empty set is −∞. For each y ∈ K and each pair of integers T1 , T2 satisfying T1 < T2 set U (y, T1 , T2 ) = sup

⎧ 2 −1 ⎨T ⎩

2 ut (xt , xtt+1 ) : {xt }Tt=T is a program and xT1 = y 1

t=T1

⎫ ⎬ ⎭

.

(2.32) Let y, y˜ ∈ K and integers T1 , T2 satisfy T1 < T2 . Set U (y, y, ˜ T1 , T2 ) = sup

⎧ 2 −1 ⎨T ⎩

ut (xt , xtt+1 ) :

t=T1

2 is a program and xT1 = y, xT2 = y˜ {xt }Tt=T 1

⎫ ⎬ ⎭

(2.33)

.

Let the integers T1 , T2 satisfy T1 < T2 . Set (T1 , T2 ) = sup U

⎧ 2 −1 ⎨T ⎩

t=T1

⎫ ⎬

2 ut (xt , xtt+1 ) : {xt }Tt=T is a program . 1 ⎭

(2.34)

18

2 Infinite Horizon Optimal Control Problems

Upper semicontinuity of the functions ut , t = 0, 1, . . . implies the following three results. Proposition 2.2 For each z ∈ K and each pair of integers T1 < T2 such that 2 such that xT1 = z and U (z, T1 , T2 ) is finite there exists a program {xt }Tt=T 1 T 2 −1

ut (xt , xt+1 ) = U (z, T1 , T2 ).

t=T1

Proposition 2.3 Let y0 , y˜0 ∈ K, integers T1 , T2 satisfy T1 < T2 and let U (y0 , y˜0 , T1 , T2 ) 2 such that be finite. Then there exists a program {xt }Tt=T 1

xT1 = y0 , xT 2 = y˜0 , T 2 −1

ut (xt , xt+1 ) = U (y0 , y˜0 , T1 , T2 ).

t=T1

(T1 , T2 ) be finite. Proposition 2.4 Let the integers T1 , T2 satisfy T1 < T2 and let U T2 Then there exists a program {xt }t=T1 such that T 2 −1

(T1 , T2 ). ut (xt , xt+1 ) = U

t=T1

In this section we suppose that the following assumption holds. (A)

There exists a natural number L¯ such that for each y, z ∈ K there is a program ¯ {xt }L t=0 such that x0 = y and xL¯ = z. Note that in [72, 73, 75] the case where L¯ = 1 was studied. In view of Proposition 2.1, assumption (A) holds for the forest management problem.

Proposition 2.5 Let the integers T1 , T2 satisfy T1 ≤ T2 − L¯ and let y, z ∈ K. Then U (y, z, T1 , T2 ) is finite. ¯ then by Assumption (A), U (y, z, T1 , T2 ) is finite. Proof If T2 − T1 = L, ¯ By (A) there exists a program {xt }T2 −L¯ such that Assume that T2 − T1 > L. t=T1

2 xT1 = y and a program {xt }Tt=T ¯ such that xT2 = z. This completes the proof of 2 −L Proposition 2.5.

In Sect. 2.4 we prove the following two results.

2.2 Infinite Horizon Problems Without Discounting

19

Theorem 2.6 There exists M > 0 such that for each x0 ∈ K there exists a program {x¯t }∞ t=0 such that x¯ 0 = x0 and that for each pair of integers T1 , T2 ≥ 0 satisfying T1 < T2 ,    T2 −1     ut (x¯t , x¯t+1 ) − U (T1 , T2 ) ≤ M.    t=T1 Theorem 2.7 Assume that {xt }∞ t=0 is a program and there exists M0 > 0 such that for each integer T > 0, T −1

ut (xt , xt+1 ) ≥ U (x(0), x(T ), 0, T ) − M0 .

t=0

Then there exists M1 > 0 such that for each pair of integers T1 ≥ 0, T2 > T1 ,    T2 −1    (T1 , T2 ) ≤ M1 . ut (xt , xt+1 ) − U     t=T1 The program {x¯t }∞ t=0 whose existence is guaranteed by Theorem 2.6 in infinite horizon optimal control is considered as an (approximately) optimal program [76, 84, 86]. Proposition 2.8 Let x0 ∈ K and let a program {x¯t }∞ t=0 and M > 0 be as guaranteed by Theorem 2.6. Assume that {xt }∞ is a program. Then either the t=0 sequence T −1 

ut (xt , xt+1 ) −

t=0

T −1

∞ ut (x¯t , x¯t+1 ) T =1

t=0

is bounded or T −1

ut (xt , xt+1 ) −

t=0

T −1

ut (x¯t , x¯t+1 ) → −∞ as T → ∞.

t=0

Proof Assume that the sequence T −1  t=0

ut (xt , xt+1 ) −

T −1 t=0

∞ ut (x¯t , x¯t+1 ) T =1

(2.35)

20

2 Infinite Horizon Optimal Control Problems

is not bounded. Then by Theorem 2.6, lim inf T →∞

T −1 

ut (xt , xt+1 ) −

t=0

T −1

 ut (x¯t , x¯t+1 ) = −∞.

t=0

Let Q > 0. Then there exists an integer T0 > 0 such that T 0 −1

ut (xt , xt+1 ) −

t=0

T 0 −1

ut (x¯t , x¯t+1 ) < −Q − M.

(2.36)

t=0

By (2.36), the choice of {x¯t }∞ t=0 and Theorem 2.6 for each integer T > T0 , T −1

ut (xt , xt+1 ) −

t=0

−

T −1

ut (x¯t , x¯t+1 ) =

T 0 −1

t=0 T 0 −1 t=0

ut (x¯t , x¯t+1 ) +

ut (xt , xt+1 )

t=0 T −1

ut (xt , xt+1 ) −

t=T0

(T0 , T ) − < −Q − M + U

T −1

ut (x¯t , x¯t+1 )

t=T0 T −1

ut (x¯t , x¯t+1 ) ≤ −Q.

t=T0

Since Q is any positive number we conclude that (2.35) is true. Proposition 2.8 is proved. Now assume that ut = u0 , t = 0, 1, . . . . The following result will be proved in Sect. 2.5. Theorem 2.9 Let M > 0 be as guaranteed by Theorem 2.6. There exists μ = (0, p)/p and limp→∞ U (0, p) − μ| ≤ 2M/p for all natural numbers p. |p−1 U The following result will be proved in Sect. 2.6. Theorem 2.10 Let K be a compact subset of a normed space (Z,  · ), the set Ω be convex, ut = u0 , t = 0, 1, . . . , the function u0 be concave, and let μ be as guaranteed by Theorem 2.9. Then μ = sup{u0 (x, x) : x ∈ K such that (x, x) ∈ Ω}. In view of Proposition 2.1, the results stated in this section are applied to the forest management problem which is discussed in Sect. 2.1. These results allow us to use new optimality criterions for the forest management problem when the functions

2.3 Auxiliary Results

21

ut are not concave. In particular, the program {x¯t }∞ t=0 whose existence is guaranteed by Theorem 2.6 can be considered as an optimal solution of the corresponding forest management problem.

2.3 Auxiliary Results Set M = 2(2L¯ + 4) sup{ut  : t = 0, 1, . . . }.

(2.37)

¯ y, z ∈ K, and Lemma 2.11 Let the integers T1 , T2 ≥ 0 satisfy T2 ≥ T1 + L, T2 M1 ≥ 0 and let a program {xt }t=T1 satisfy x0 = y, xT = z, T 2 −1

ut (xt , xt+1 ) ≥ U (y, z, T1 , T2 ) − M1 .

(2.38)

(2.39)

t=T1

Then for each pair of integers τ1 , τ2 ≥ 0 satisfying T1 ≤ τ1 < τ2 ≤ T2 the following inequality holds: τ 2 −1

(τ1 , τ2 ) − M1 − M. ut (xt , xt+1 ) ≥ U

(2.40)

t=τ1

Proof Let the integers τ1 , τ2 ≥ 0 satisfy T1 ≤ τ1 < τ2 ≤ T2 . 2 such that By Proposition 2.4 and assumption (A), there is a program {xt }τt=τ 1

τ 2 −1

(τ1 , τ2 ). ut (xt , xt+1 )=U

(2.41)

t=τ1

If τ2 − τ1 ≤ 2L¯ + 2, then in view of (2.37) inequality (2.40) holds. Assume that τ2 − τ1 > 2L¯ + 2.

(2.42)

22

2 Infinite Horizon Optimal Control Problems ¯

1 +L 2 By assumption (A) there exist programs {yt }τt=τ , {yt }τt=τ 1

yτ

¯ 2 −L

= xτ

¯ 2 −L

¯ 2 −L

such that

, yτ 2 = xτ2 , yτ1 = xτ1 , yτ1 +L¯ = xτ

¯ 1 +L

.

(2.43)

Set ¯ zt = xt , t = T1 , . . . , τ1 , zt = yt , t = τ1 + 1, . . . , τ1 + L, ¯ zt = xt , t = τ1 + L¯ + 1, . . . , τ2 − L, zt = yt , t = τ2 − L¯ + 1, . . . , τ2 , zt = xt for all integers t satisfying τ2 < t ≤ T2 . 2 is a program. By (2.37), (2.39), (2.41), (2.43), and (2.44), Clearly, {zt }Tt=T 1

−M1 ≤

T 2 −1

ut (xt , xt+1 ) −

t=T1

=

≤

τ 2 −1

T 2 −1 t=T1

ut (xt , xt+1 ) −

τ 2 −1

t=τ1

t=τ1

τ 2 −1

τ 2 −1

t=τ1

ut (zt , zt+1 )

ut (xt , xt+1 ) −

ut (zt , zt+1 )

ut (xt , xt+1 )

t=τ1

+ 2 sup{ut  : t = 0, 1, . . . }2(L¯ + 1) ≤

τ 2 −1

(τ1 , τ2 ) + M ut (xt , xt+1 ) − U

t=τ1

and τ 2 −1

(τ1 , τ2 ) − M1 − M. ut (xt , xt+1 ) ≥ U

t=τ1

Lemma 2.11 is proved. Lemma 2.11 and (2.37) imply the following auxiliary result.

(2.44)

2.4 Proofs of Theorems 2.6 and 2.7

23

Lemma 2.12 Let the integers T1 , T2 ≥ 0 satisfy T2 > T1 , y ∈ K, M1 ≥ 0 and let a 2 satisfy program {xt }Tt=T 1 x0 = y, T 2 −1

ut (xt , xt+1 ) ≥ U (y, T1 , T2 ) − M1 .

t=T1

Then for each pair of integers τ1 , τ2 ≥ 0 satisfying T1 ≤ τ1 < τ2 ≤ T2 , τ 2 −1

(τ1 , τ2 ) − M1 − M. ut (xt , xt+1 ) ≥ U

t=τ1

2.4 Proofs of Theorems 2.6 and 2.7 Proof of Theorem 2.6 Let x0 ∈ K. By Proposition 2.2, for each natural number k there exists a program {xt(k) }kt=0 such that (k)

x0 = x0 ,

k−1 

(k)

(k)

ut (xt , xt+1 ) = U (x0 , 0, k).

(2.45)

t=0

By (2.45) and Lemma 2.12 the following property holds: (P3) For each integer k ≥ 0 and each pair of integers τ1 , τ2 ≥ 0 satisfying 0 ≤ τ1 < τ2 ≤ k, τ 2 −1

(k) (T1 , T2 ) − M. ut (xt(k) , xt+1 )≥U

t=τ1

Clearly, there exists a strictly increasing sequence of natural numbers {kj }∞ j =1 such that for each integer t ≥ 0 there exists (kj )

x¯t = lim xt j →∞

.

(2.46)

In view of (2.45) and (2.46), x¯0 = x0 .

(2.47)

24

2 Infinite Horizon Optimal Control Problems

It follows from (P3), (2.46), and upper semicontinuity of the functions ut , t = 0, 1, . . . that for each pair of integers τ1 , τ2 ≥ 0 satisfying τ1 < τ2 ,    τ   2 −1   ut (x¯t , x¯t+1 ) − U (T1 , T2 ) ≤ M.    t=τ1 Theorem 2.6 is proved. Proof of Theorem 2.7 By Lemma 2.11, for all integers τ1 , τ2 ≥ 0 satisfying 0 ≤ τ1 < τ2 , τ 2 −1

(τ1 , τ2 ) ≥ −M0 − M. ut (xt , xt+1 ) − U

t=τ1

This completes the proof of Theorem 2.7.

2.5 Proof of Theorem 2.9 Set u = u0 . Let x0 ∈ K and let {x¯t }∞ t=0 be as guaranteed by Theorem 2.6. Then for each pair of integers T1 , T2 ≥ 0 satisfying T1 < T2 ,    T2 −1     u(x¯t , x¯t+1 ) − U (T1 , T2 ) ≤ M.    t=T1

(2.48)

Choose Δ > 0 such that Δ > u0 .

(2.49)

Let p be a natural number. We show that for all sufficiently large natural numbers T,   T −1    −1   u(x¯t , x¯t+1 ) ≤ 2M/p. p U (0, p) − T −1  

(2.50)

t=0

Assume that T ≥ p is a natural number. Then there exist integers q, s such that q ≥ 1, 0 ≤ s < p, T = pq + s.

(2.51)

2.5 Proof of Theorem 2.9

25

It follows from (2.51) that T −1

T −1

(0, p) u(x¯t , x¯t+1 ) − p−1 U

t=0

=T

⎛

pq−1  −1 ⎝

u(x¯t , x¯t+1 )

t=0

⎞  (0, p) + {u(x¯t , x¯t+1 ) : t is an integer such that pq ≤ t ≤ T −1}⎠ −p −1 U = T −1 + (T

 {u(x¯t , x¯t+1 ) : t is an integer such that pq ≤ t ≤ T − 1} −1

pq)(pq)

−1

⎡ = (T −1 pq)(pq)−1 ⎣

q−1 (i+1)p−1   t=0 q−1  i=0

⎛ ⎝

(0, p) u(x¯t , x¯t+1 ) − p−1 U

t=ip (i+1)p−1 

⎞

⎤

(0, p)⎠ + q U (0, p)⎦ u(x¯t , x¯t+1 ) − U

t=ip

(0, p) − p−1 U

  u(x¯t , x¯t+1 ) : t is an integer such that pq ≤ t ≤ T − 1 . + T −1 (2.52) By (2.48), (2.49), (2.51), and (2.52),   T −1    −1   −1  u(x¯t , x¯t+1 ) − p U (0, p) T   t=0

(0, p)||q/T − 1/p| ≤ T −1 pΔ + (pq)−1 qM + |U (0, p)|s(pT )−1 → M/p as T → ∞. ≤ T −1 pΔ + M/p + |U Since p is any natural number we conclude that ∞  T −1 −1 u(x¯t , x¯t+1 ) T T =1

t=0

is a Cauchy sequence. Clearly, there exists lim T −1

T →∞

T −1 t=0

u(x¯t , x¯t+1 ),

(2.53)

26

2 Infinite Horizon Optimal Control Problems

and that in view of (2.53) for each natural number p, (0, p) − lim T −1 |p−1 U

T −1

T →∞

u(x¯t , x¯t+1 )| ≤ 2M/p.

(2.54)

t=0

Since (2.54) is true for any natural number p we obtain that lim T −1

T →∞

T −1

(0, p)/p. u(x¯t , x¯t+1 ) = lim U p→∞

t=0

(2.55)

Set (0, p)/p. μ = lim U

(2.56)

p→∞

By (2.54)–(2.56), for all natural numbers p, (0, p) − μ| ≤ 2M/p. |p−1 U Theorem 2.9 is proved.

2.6 Proof of Theorem 2.10 Put u = u0 . By Proposition 2.4 and assumption (A), for each integer T > 0 there is a program {xt(T ) }Tt=0 such that T −1

(T )

u(xt

(T ) (0, T ). , xt+1 ) = U

(2.57)

t=0

Set x (T ) = T −1

T −1 t=0

(T )

xt

, y (T ) = T −1

T −1

(T )

xt+1 .

t=0

Clearly, (0, T ), (x (T ) , y (T ) ) ∈ Ω, T u(x (T ) , y (T ) ) ≥ U (0, T ), u(x (T ) , y (T ) ) ≥ T −1 U x (T ) − y (T )  ≤ 2T −1 sup{z : z ∈ K}.

(2.58)

2.7 Infinite Horizon Problems with Discounting

27

There is a strictly increasing sequence of natural numbers {Tk }∞ k=1 such that there exist lim x (Tk ) = x, ¯ lim y (Tk ) = y. ¯

k→∞

k→∞

Clearly, x¯ = y, ¯ (x, ¯ x) ¯ ∈ Ω, (0, T ) = μ. u(x, ¯ x) ¯ ≥ lim T −1 U T →∞

On the other hand, it is easy to see that u(z, z) ≤ μ for all z ∈ K such that (z, z) ∈ Ω. Theorem 2.10 is proved.

2.7 Infinite Horizon Problems with Discounting We consider the optimal control problem of Sect. 2.2 and continue to use the notation, definitions, and assumptions introduced there. In this section we suppose that the following equality holds: lim sup{|ut (z)| : z ∈ Ω} = 0.

t→∞

(2.59)

By (A) and Proposition 2.2, for each y ∈ K and each natural number T there is (y,T ) T }t=0 such that a program {xt (y,T )

x0 T −1

(y,T )

ut (xt

= y,

(2.60)

(y,T )

, xt+1 ) = U (y, 0, T ).

(2.61)

t=0

In Sect. 2.9 we prove the following result. Theorem 2.13 For any y ∈ K there exists a program {xt }∞ t=0 such that x0 = y and the following property holds: For each  > 0 there exists a natural number τ such that for each y ∈ K and each integer T ≥ τ , (y)

(y)

28

2 Infinite Horizon Optimal Control Problems

T −1      (y) (y) ut (xt , xt+1 ) − U (y, 0, T ) ≤ .    t=0

The next corollary easily follows from Theorem 2.13. Corollary 2.14 Let y ∈ K. Then for any program {xt }∞ t=0 satisfying x0 = y, lim sup

T −1 

T →∞ t=0

 (y) (y) ut (xt , xt+1 ) − ut (xt , xt+1 ) ≤ 0.

Note that the program {xt }∞ t=0 which exists by Corollary 2.14 is called in the literature as an overtaking optimal program [76, 84, 86]. Example 2.15 Let w : Ω → [0, ∞) be a bounded upper semicontinuous function, {ρt }∞ t=0 ⊂ (0, 1) satisfy lim ρt = 0,

t→∞

(2.62)

and let ut = ρt w, t = 0, 1, . . . . Then (2.59) holds. In the literature it is also considered an optimality criterion with ρt = α t , t = 0, 1, . . . where α ∈ (0, 1). ∞ ∞ In this case for any program {xt }t=0 , t=0 α t w(xt , xt+1 ) < ∞. This convergence does not hold in the general case with {ρt }∞ t=0 ⊂ (0, 1) satisfying (2.62). Therefore in the general case the existence problem of an overtaking optimal program is more difficult and less understood. The results of this section were obtained from [80].

2.8 An Auxiliary Result for Theorem 2.13 Recall that for any integer t ≥ 0, ut  = sup{|ut (z)| : z ∈ Ω}.

(2.63)

Lemma 2.16 Let  > 0. Then there exists a natural number τ such that for each ¯ y ∈ K and each pair of integers T1 ≥ τ and T2 ≥ T1 + L, T 1 −1

  (y,T ) (y,T ) ut xt 2 , xt+1 2 ≥ U (y, 0, T1 ) − .

t=0

Proof By (2.59) and (2.63), there exists a natural number τ such that ¯ −1 for all integers t ≥ τ. ut  ≤ (4L)

(2.64)

2.8 An Auxiliary Result for Theorem 2.13

29

Assume that y ∈ K and that integers ¯ T1 ≥ τ, T2 ≥ T1 + L.

(2.65)

¯

1 +L such that In view of (A), there exists a program {xt }Tt=T 1

(y,T1 )

xT1 = xT1

(y,T2 ) ¯ . 1 +L

(2.66)

, t = 0, . . . , T1 − 1,

(2.67)

, xT1 +L¯ = xT

Set (y,T1 )

xt = xt (y,T2 )

xt = xt

for all integers t satisfying T1 + L¯ < t ≤ T2 .

2 Clearly, {xt }Tt=0 is a program and

x0 = y.

(2.68)

By (2.60), (2.61), (2.63)–(2.66), and (2.68), 0≤

T 2 −1

(y,T2 )

ut (xt

(y,T )

, xt+1 2 ) −

t=0

=

T 2 −1

¯ T1 +L−1

(y,T2 )

ut (xt

(y,T )

, xt+1 2 ) −

¯ T1 +L−1

t=0

≤

T 1 −1

T 1 −1

ut (xt , xt+1 )

t=0

(y,T ) (y,T ) ut (xt 2 , xt+1 2 ) −

t=0

≤

ut (xt , xt+1 )

t=0

T 1 −1

(y,T ) (y,T ) ut (xt 1 , xt+1 1 ) + 2

¯ T1 +L−1

t=0 (y,T2 )

ut (xt

(y,T ) ¯ ¯ −1 ) , xt+1 2 ) − U (y, 0, T1 ) + 2L((4 L)

t=0

and T 1 −1 t=0

Lemma 2.16 is proved.

(y,T2 )

ut (xt

(y,T )

, xt+1 2 ) ≥ U (y, 0, T1 ) − .

t=T1

||ut ||

30

2 Infinite Horizon Optimal Control Problems

2.9 Proof of Theorem 2.13 Let y ∈ K. Using the diagonalization process and the compactness of K we obtain a strictly increasing sequence of natural numbers {Tk }∞ k=1 such that for any integer t ≥ 0 there exists (y)

xt

(y,Tk )

= lim xt k→∞

(2.69)

.

Clearly, {xt }∞ t=0 is a program for all y ∈ K. Let  > 0 and let a natural number τ be as guaranteed by Lemma 2.16. Assume that an integer T ≥ τ and y ∈ K. Then for all sufficiently large natural numbers k, y

T −1

(y,Tk )

ut (xt

(y,T )

, xt+1 k ) ≥ U (y, 0, T ) − .

t=0

By the inequality above, (2.69) and upper semicontinuity of the functions ut , t = 0, 1, . . . , T −1

(y)

(y)

ut (xt , xt+1 ) ≥ U (y, 0, T ) − .

t=0

Theorem 2.13 is proved.

2.10 An Application to the Forest Management Problem We consider the forest management problem discussed in Sect. 2.1 using the notation, definitions, and assumptions introduced there. Recall that for each (x, y) ∈ Ω, V (x, y) = (v1 (x, y), . . . , vk (x, y)), where for i = 1, . . . , k, vi (x, y) = xini +1 + xini − yini +1 , and that  Δ0 = u ∈

k R+

:

k  i=1

 ui ≤ S .

2.10 An Application to the Forest Management Problem

31

In this section we assume that a benefit at moment t = 0, 1, . . . is represented by an upper semicontinuous function wt : Δ0 → R 1 and at a moment t = 0, 1, . . . , wt (V (x, y)) is the benefit obtained today if the forest today is x and the forest tomorrow is y, where (x, y) ∈ Ω. In view of Proposition 2.1, (A) holds. Assume that sup{|wt (z)| : z ∈ Δ0 , t = 0, 1, . . . } < ∞. It is easy now to see that all the results of Sect. 2.2 hold for our model with ut = wt ◦ V , t = 0, 1, . . . . If sup{|wt (z)| : z ∈ Δ0 } → 0 as t → ∞, then all the results of Sect. 2.7 hold for our model with ut = wt ◦ V , t = 0, 1, . . . .

Chapter 3

Turnpike Properties

In this chapter we consider a discrete-time optimal control problem. The forest management problem is its particular case. We establish turnpike results for approximate solutions and the stability of the turnpike phenomenon. We prove the existence of solutions of the corresponding infinite horizon problems and show the equivalence of different optimality criterions for these problems. Most results of this chapter were obtained from [83].

3.1 Preliminaries and Main Results We continued to study the class of optimal control problems introduced in Sect. 2.2 using the same notation, definitions, and assumptions. Let (K, ρ) be a compact metric space and let Ω be a nonempty closed subset of K × K. A sequence {xt }∞ t=0 ⊂ K is called an (Ω)-program (or a program if Ω is understood) if (xt , xt+1 ) ∈ Ω for all integers t ≥ 0. 2 Let the integers T1 , T2 satisfy T1 < T2 . A sequence {xt }Tt=T ⊂ K is called an 1 (Ω)-program (or a program if Ω is understood) if (xt , xt+1 ) ∈ Ω for all integers t satisfying T1 ≤ t < T2 . For each integer t ≥ 0, let ut : Ω → R 1 be a bounded upper semicontinuous function. For each pair of integers T1 , T2 satisfying 0 ≤ T1 < T2 and each y, z ∈ K we consider the optimization problems T 2 −1

ui (xi , xi+1 ) → max,

(P1 )

i=T1

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2019 A. J. Zaslavski, Optimal Control Problems Arising in Forest Management, SpringerBriefs in Optimization, https://doi.org/10.1007/978-3-030-23587-1_3

33

34

3 Turnpike Properties 2 {xi }Ti=T ⊂ K is a program, xT1 = y, xT2 = z, 1

T 2 −1

ui (xi , xi+1 ) → max,

(P2 )

i=T1 2 {xi }Ti=T ⊂ K is a program, xT1 = y 1

and T 2 −1

ui (xi , xi+1 ) → max,

(P3 )

i=T1 2 {xi }Ti=T ⊂ K is a program. 1

The interest in discrete-time optimal problems of these types stems from the study of various optimization problems which can be reduced to it, e.g., continuoustime control systems which are represented by ordinary differential equations whose cost integrand contains a discounting factor [16], tracking problems in engineering [44, 87], the study of Frenkel-Kontorova model related to dislocations in one-dimensional crystals [6, 71], the analysis of a long slender bar of a polymeric material under tension in [19, 45, 50, 51], and models of economic growth [84, 86]. See also [72, 73] where these problems were studied with Ω = K × K. For each integer t ≥ 0 set ut  = sup{|ut (z)| : z ∈ Ω}

(3.1)

sup{ut  : t = 0, 1, . . . } < ∞.

(3.2)

and assume that

In the sequel we assume that supremum over empty set is −∞. For every point y ∈ K and every pair of integers T1 , T2 satisfying T1 < T2 set U (y, T1 , T2 ) = sup

⎧ 2 −1 ⎨T ⎩

t=T1

2 ut (xt , xtt+1 ) : {xt }Tt=T is a program and xT1 = y 1

⎫ ⎬ ⎭

.

(3.3) Let a point y, y˜ ∈ K be given and let the integers T1 , T2 satisfy T1 < T2 . Set

3.1 Preliminaries and Main Results

U (y, y, ˜ T1 , T2 ) = sup

2 {xt }Tt=T 1

35

⎧ 2 −1 ⎨T ⎩

ut (xt , xtt+1 ) :

t=T1

is a program and xT1 = y, xT2 = y˜

⎫ ⎬ ⎭

.

Let the integers T1 , T2 satisfy T1 < T2 . Define ⎧ ⎫ 2 −1 ⎨T ⎬ 2 (T1 , T2 ) = sup U ut (xt , xtt+1 ) : {xt }Tt=T is a program . 1 ⎩ ⎭

(3.4)

(3.5)

t=T1

In this chapter we suppose that the following assumption, introduced in Sect. 2.2, holds. (A)

There exists a natural number L¯ such that for every pair of points y, z ∈ K ¯ there exists a program {xt }L t=0 which satisfies x0 = y and xL¯ = z.

This assumption holds for our forest management problems. Note that all the results stated in Sect. 2.2 (Theorems 2.6, 2.7, 2.9, and 2.10 and Propositions 2.5 and 2.8) are valid. We suppose that ut = u0 for all integers t ≥ 0. Set u = u0 and u = sup{|u(z)| : z ∈ Ω}.

(3.6)

Let us consider the constant μ defined by Theorem 2.9 and suppose that μ = μ(u) = sup{u(x, x) : x ∈ K and (x, x) ∈ Ω}.

(3.7)

By upper semicontinuity of u, there exists a point x¯ ∈ K such that (x, ¯ x) ¯ ∈ Ω and u(x, ¯ x) ¯ = μ.

(3.8)

Denote by Card(A) the cardinality of a set A. A program {xt }∞ t=0 is called (u, Ω)-good ((u)-good if the set Ω is understood or good if the pair (u, Ω) is understood) if the sequence T −1 ∞  {u(xt , xt+1 ) − μ t=0

T =1

is bounded. A program {xt }∞ ((u)-bad if the set Ω is understood or t=0 is called (u, Ω)-bad T −1 (u(xt , xt+1 ) − μ) → −∞ as T → ∞. bad if the pair (u, Ω) is understood) if t=0 Theorem 2.9, Proposition 2.8, and Theorem 2.6 imply the following result. Proposition 3.1 Let {xt }∞ t=0 be a program. Then it is either good or bad. For every point x0 ∈ K there exists a good program {xt }∞ t=0 .

36

3 Turnpike Properties

We study the structure of approximate solutions of problems (P1 )-(P3 ) and suppose that the following assumptions hold. (B1) For every good program {xt }∞ ¯ t=0 , limt→∞ xt = x. (B2) The function u is continuous at the point (x, ¯ x) ¯ with respect to Ω. (B3) There exists a natural number L∗ such that for every positive number  there exists a positive number δ such that for every pair of points y1 , y2 ∈ K which satisfies ¯ ρ(x, ¯ y2 ) ≤ δ ρ(y1 , x), ∗ ¯ ≤ there exists a program {xt }L t=0 such that x0 = y1 , xL∗ = y2 and ρ(xt , x) , t = 0, . . . , L∗ .

We will prove the following results which describe the structure of approximate solutions of problems (P1 )–(P3 ). Theorem 3.2 Let  be a positive number. Then there exist an integer τ > 0 and a positive number δ such that for every natural number T > 2τ and every program {xt }Tt=0 which satisfy T −1

u(xt , xt+1 ) ≥ U (x0 , xT , 0, T ) − δ

t=0

there exist integers τ1 ∈ [0, τ ], τ2 ∈ [T − τ, T ] such that ¯ ≤ , t = τ1 , . . . , τ2 . ρ(xt , x) ¯ ≤ δ and τ2 = T if ρ(x, x) ¯ ≤ δ. Moreover τ1 = 0, if ρ(x0 , x) Theorem 3.2 is proved in Sect. 3.3. Theorem 3.3 Let , M be positive numbers. Then there exist integers τ, M1 > 0 such that for every natural number T > τ and every program {xt }Tt=0 which satisfies T −1

u(xt , xt+1 ) ≥ U (x0 , xT , 0, T ) − M

t=0

the inequality ¯ > }) < M1 Card({t ∈ {0, . . . , T } : ρ(xt , x) holds. Theorem 3.3 is proved in Sect. 3.4. Theorem 3.4 Let z ∈ K. Then there exists a program {xt }∞ t=0 such that x0 = z and that for each program {yt }∞ satisfying y = z, 0 t=0

3.1 Preliminaries and Main Results T −1

lim sup

T →∞ t=0

37

[u(yt , yt+1 ) − u(xt , xt+1 )] ≤ 0.

(3.9)

Theorem 3.4 is proved in Sect. 3.5. A program {xt }∞ t=0 is called (u, Ω)-overtaking optimal ((u)-overtaking optimal if the set Ω is understood or overtaking optimal if the pair (u, Ω) is understood) if T −1

lim sup

T →∞ t=0

[u(yt , yt+1 ) − u(xt , xt+1 )] ≤ 0

for all programs {yt }∞ t=0 satisfying x0 = y0 [76, 84, 86]. In other words Theorem 3.4 establishes the existence of an overtaking optimal program for any initial state. We also use another optimality criterion which was introduced and applied in [6, 84, 86]. A program {xt }∞ t=0 is (u, Ω)-locally optimal ((u)-locally optimal if the set Ω is understood or locally optimal if the pair (u, Ω) is understood) if for any integer T >0 T −1 u(xt , xt+1 ) = U (x0 , xT , 0, T ). t=0

The following result will be proved in Sect. 3.6. Theorem 3.5 A program {xt }∞ t=0 is locally optimal if and only if it is overtaking optimal. The next result is a generalization of Theorem 3.2 which will be proved in Sect. 3.7. Theorem 3.6 Let  be a positive number. Then there exist an integer τ > 0 and a positive number δ such that for every natural number T > 2τ and every program {xt }Tt=0 satisfying S+τ −1

u(xt , xt+1 ) ≥ U (x(S), x(S + τ ), S, S + τ ) − δ

t=S

for all integers S ∈ {0, . . . , T − τ } there exist integers τ1 , τ2 such that τ1 ∈ [0, τ ], τ2 ∈ [T − τ, T ], ρ(xt , x) ¯ ≤ , t = τ1 , . . . τ2 . ¯ ≤ δ and τ2 = T if ρ(xT , x) ¯ ≤ δ. Moreover, τ1 = 0 if ρ(x0 , x) The results of this section were obtained from [83].

38

3 Turnpike Properties

3.2 Auxiliary Results ¯ Then Proposition 3.7 Let x, y ∈ K and an integer T > 2L. U (x, y, 0, T ) ≥ T μ − 4(L¯ + 1)u. Proof In view of (A) and (3.8), there exists a program {xt }Tt=0 such that ¯ . . . , T − L, ¯ xT = y. x0 = x, xt = x, ¯ t = L, Combined with (3.6)–(3.8) this implies that U (x, y, 0, T ) ≥

T −1

u(xt , xt+1 ) ≥ T μ − 4(L¯ + 1)u.

t=0

This completes the proof of Proposition 3.7. Lemma 3.8 Let , M be positive numbers. Then there exists an integer T > 2L¯ such that for every program {xt }Tt=0 which satisfies T −1

u(xt , xt+1 ) ≥ U (x0 , xT , 0, T ) − M,

t=0

there exists an integer j ∈ {1, . . . , T } for which ρ(xj , x) ¯ ≤ . Proof Assume the contrary. Then there exist a strictly increasing sequence of (k) Tk natural numbers {Tk }∞ k=1 and a sequence of programs {xt }t=0 , k = 1, 2, . . . such that for every natural number k, T k −1

    (k) (k) (k) (k) u xt , xt+1 ≥ U x0 , xTk , 0, Tk − M,

(3.10)

t=0

  (k) ρ xj , x¯ > , j = 1, . . . , Tk .

(3.11)

In view of (3.10), for every natural number k and every pair of integers S1 , S2 which satisfy 0 ≤ S1 < S2 ≤ Tk we have S 2 −1 t=S1

(k)

(k)

(k)

(k)

u(xt , xt+1 ) ≥ U (xS1 , xS2 , 0, S2 − S1 ) − M.

(3.12)

3.2 Auxiliary Results

39

Extracting a subsequence and re-indexing if necessary we may assume that for every nonnegative integer t ≥ 0 there is xt = lim xt(k) . k→∞

(3.13)

It follows from Proposition 3.7 and (3.12) that the following property holds: (P1)

For every natural number k and every pair of integers S1 , S2 which satisfy 0 ≤ S1 and S1 + 2L¯ < S2 ≤ Tk , we have S 2 −1

(k) (k) u(xt , xt+1 ) ≥ (S2 − S1 )μ − 4(L¯ + 1)u − M.

t=S1

Property (P1) and (3.13) imply that for every pair of integers S1 , S2 ≥ 0 which satisfy S2 − S1 > 2L¯ we have S 2 −1

! u(xt , xt+1 ) ≥ (S2 − S1 )μ − 4 L¯ + 1 u − M.

(3.14)

t=S1

Proposition 3.1, (3.14), and property (B1) imply that the program {xt }∞ t=0 is good and that lim ρ(xt , x) ¯ = 0.

t→∞

Thus there exists a natural number S0 such that ¯ < /4 for all integers t ≥ S0 . ρ(xt , x)

(3.15)

In view of (3.13), there exists a natural number k such that Tk > S0 + 4 and that (k)

ρ(xS0 , xS0 ) ≤ /4. (k)

Combined with (3.15) this implies that ρ(xS0 , x) ¯ ≤ /2. This contradicts (3.11). The contradiction we have reached proves Lemma 3.8. Lemma 3.9 Let  be a positive number. Then there exits a positive number δ such that for every natural number T > 2L∗ and every pair of points y, z ∈ K which satisfies ρ(y, x), ¯ ρ(z, x) ¯ ≤ δ, the inequality U (y, z, 0, T ) ≥ T μ −  holds.

(3.16)

40

3 Turnpike Properties

Proof Since the function u is continuous at the point (x, ¯ x) ¯ (see assumption (B2)) there exists a number 0 ∈ (0, ) such that for every point (y, z) ∈ Ω which satisfies ρ(y, x), ¯ ρ(z, x) ¯ ≤ 0 the inequality |u(y, z) − u(x, ¯ x)| ¯ ≤ (4L∗ )−1

(3.17)

holds. In view of assumption (B3), there exists a number δ ∈ (0, 0 ) such that for every pair of points y, z ∈ K which satisfies ρ(y, x), ¯ ρ(z, x) ¯ ≤δ ∗ there exists a program {ξt }L t=0 for which

ξ0 = y, ξL∗ = z, ρ(ξt , x) ¯ ≤ 0 , t = 0, . . . , L∗ .

(3.18)

Assume that a natural number T > 2L∗ and that a pair of points y, z ∈ K satisfies (3.16). It follows from (3.16) and the choice of the number δ (see (3.18)) that there exists a program {xt }Tt=0 such that x0 = y, ρ(xt , x) ¯ ≤ 0 , t = 0, . . . , L∗ , xt = x, ¯ t = L ∗ , . . . , T − L∗ , xT = z, ρ(xt , x) ¯ ≤  0 , t = T − L∗ , . . . , T .

(3.19)

In view of (3.19), (3.8), and the choice of 0 (see (3.17)), U (y, z, 0, T ) ≥

T −1

u(xt , xt+1 ) = T μ +

t=0

+

L ∗ −1

[u(xt , xt+1 ) − u(x, ¯ x)] ¯

t=0 T −1

[u(xt , xt+1 ) − u(x, ¯ x)] ¯ ≥ Tμ

t=T −L∗

− 2L∗ (4L∗ )−1 ≥ T μ − /2. This completes the proof of Lemma 3.9. Lemma 3.10 Let  be a positive number. Then there exists a number δ ∈ (0, ) such that for every natural number T > 2L∗ and every program {xt }Tt=0 which satisfies ¯ ρ(xT , x) ¯ ≤ δ, ρ(x0 , x),

T −1 t=0

u(xt , xt+1 ) ≥ U (x0 , xT , 0, T ) − δ,

3.2 Auxiliary Results

41

the inequality ¯ ≤ , t = 0, 1, . . . , T ρ(xt , x) holds. Proof Let k be a natural number. In view of assumption (B2), there exists a number k ∈ (0, 2−k )

(3.20)

such that for every point (y, z) ∈ Ω which satisfies ρ(y, x), ¯ ρ(z, x) ¯ ≤ k the inequality |u(y, z) − u(x, ¯ x)| ¯ ≤ 2−k

(3.21)

holds. Lemma 3.9 implies that there exists a number δk,1 ∈ (0, k ) such that the following property holds: (P2)

For every natural number T > 2L∗ and every pair of points y, z ∈ K which satisfies ρ(y, x), ¯ ρ(z, x) ¯ ≤ δk,1 , we have U (y, z, 0, T ) ≥ T μ − 2−k .

In view of assumption (B3), there exists a number δk,2 ∈ (0, k ) such that the following property holds: (P3)

For every pair of points y, z ∈ K which satisfy ρ(y, x), ¯ ρ(z, x) ¯ ≤ δk,2 , ∗ there exists a program {xt }L t=0 which satisfies

x0 = y, xL∗ = z and ¯ ≤ k , t = 0, . . . , L∗ . ρ(xt , x)

42

3 Turnpike Properties

Choose a positive number δk < min{δk,1 δk,2 }.

(3.22)

We may assume without loss of generality that the sequence {δi }∞ i=1 is monotone decreasing. Assume that the assertion of the lemma does not hold. Then for every integer k k ≥ 1 there exist an integer Tk > 2L∗ and a program {xt(k) }Tt=0 which satisfies (k)

T k −1

(k)

ρ(x0 , x), ¯ ρ(xTk , x) ¯ ≤ δk ,

(3.23)

(k)

(3.24)

(k)

(k)

(k)

u(xt , xt+1 ) ≥ U (x0 , xTk , 0, Tk ) − δk ,

t=0 (k)

¯ : t = 0, . . . , Tk } > . max{ρ(xt , x)

(3.25)

Set S0 = 0, Sk =

k  (Ti + L∗ )

(3.26)

i=1

for all natural numbers k. By induction define a program {xt∗ }∞ t=0 . Set xt∗ = xt , t = 0, 1, . . . , T1 . (1)

(3.27)

It follows from (3.22), (3.23), and property (P3) that there exists a program 1 +L∗ {xt∗ }Tt=T such that 1 ¯ ≤  1 , t = T 1 , . . . , T 1 + L∗ . xT∗1 +L∗ = x0 , ρ(xt∗ , x) (2)

(3.28)

1 It is clear that {xt∗ }St=0 is a program. k Assume that k ≥ 1 is an integer and that we have defined a program {xt∗ }St=0 such that

xS∗k = x0(k+1)

(3.29)

and that for each integer i ∈ {0, . . . , k − 1} xS∗i +t = xt

(i+1)

, t = 0, . . . , Ti+1 ,

ρ(xS∗i +Ti+1 +t , x) ¯ ≤ i+1 , t = 0, . . . , L∗ . (It is clear that this assumption holds for k = 1.)

(3.30)

3.2 Auxiliary Results

43

Set xS∗k +t = xt

(k+1)

, t = 1, . . . , Tk+1 .

(3.31)

In view of (3.31), (3.23), property (P3), (3.26), and (3.22), there exists a program Sk+1 {xt∗ }t=S such that k +Tk+1 xS∗k +1 = x0

(k+2)

, ρ(xt∗ , x) ¯ ≤ k+1 , t = Sk + Tk+1 , . . . , Sk+1 .

(3.32)

k+1 It is not difficult to see that {xt∗ }t=0 is a program and the assumption we made for k also holds for k + 1. Thus the program {xt∗ }∞ t=0 was defined by induction such that (3.30) holds for all integers i ≥ 0. For every nonnegative integer k it follows from (3.30), (3.26), and (3.24), the choice of k+1 (see (3.21)), property (P2), (3.23), (3.20), and (3.22) that

S

Sk+1 −1

Tk+1 −1    "  # (k+1) ∗ ∗ u(xt(k+1) , xt+1 u(xt , xt+1 ) − u(x, ¯ x) ¯ = ) − u(x, ¯ x) ¯

t=Sk

t=0 Sk+1 −1



+

" ∗ ∗ # u(xt , xt+1 ) − u(x, ¯ x) ¯

t=Sk +Tk+1

  (k+1) (k+1) ≥ U x0 , xTk+1 , 0, Tk+1 − δk+1 − Tk+1 μ − 2−k+1 L∗ ≥ −2−k+1 (L∗ + 2). This implies that for every nonnegative integer k we have S k +1

k+1  " ∗ ∗ # u(xt , xt+1 ) − μ ≥ −(L∗ + 2) 2−i+1

t=0

i=0

and by Proposition 3.1 and assumption (B1), the program {xt∗ }∞ t=0 is good and satisfies ¯ lim x ∗ = x.

t→∞

Combined with (3.30) this contradicts (3.25). The contradiction we have reached proves Lemma 3.10.

44

3 Turnpike Properties

3.3 Proof of Theorem 3.2 We may assume that  < 1/2. Lemma 3.10 implies that there exists a number δ ∈ (0, ) such that the following property holds: (P4)

For every natural number T > 2L∗ and every program {xt }Tt=0 satisfying ¯ ρ(xT , x) ¯ ≤δ ρ(x0 , x), and T −1

u(xt , xt+1 ) ≥ U (x0 , xT , 0, T ) − δ

t=0

the inequality ρ(xt , x) ¯ ≤  is true for all t = 0, . . . , T . Lemma 3.8 implies that there exists an integer T∗ > 2L¯ such that the following property holds: (P5)

∗ which satisfies For every program {xt }Tt=0

T ∗ −1

u(xt , xt+1 ) ≥ U (x0 , xT∗ , 0, T∗ ) − 4,

t=0

there exists an integer j ∈ {1, . . . , T∗ } for which ρ(xj , x) ¯ ≤ δ. Set τ = T∗ + L ∗ .

(3.33)

Assume that a natural number T > 2τ and that a program {xt }Tt=0 satisfies T −1

u(xt , xt+1 ) > U (x0 , xT , 0, T ) − δ.

(3.34)

u(xt , xt+1 ) ≥ U (x0 , xT∗ , 0, T∗ ) − δ,

(3.35)

u(xt , xt+1 ) ≥ U (xT −T∗ , xT , 0, T∗ ) − δ.

(3.36)

t=0

In view of (3.34), T ∗ −1 t=0 T −1 t=T −T∗

3.4 Proof of Theorem 3.3

45

Property (P5), (3.35), and (3.36) imply that there exist integers τ1 ∈ [0, T∗ ], τ2 ∈ [T − T∗ , T ] such that ρ(xτ1 , x) ¯ ≤ δ, ρ(xτ2 , x) ¯ ≤ δ.

(3.37)

¯ ≤ δ, then we may set τ1 = 0, and if ρ(xT , x) ¯ ≤ δ, then It is clear that if ρ(x0 , x) we may set τ2 = T . In view of (3.34), τ 2 −1

u(xt , xt+1 ) ≥ U (xτ1 , xτ2 , τ1 , τ2 ) − δ.

t=τ1

Combined with (3.37), property (P4), (3.33), and the inequality T > 2τ this implies that ¯ ≤ , t = τ1 , . . . , τ2 . ρ(xt , x) This completes the proof of Theorem 3.2.

3.4 Proof of Theorem 3.3 Theorem 3.2 implies that there exist a positive number δ and an integer τ0 > 0 such that the following property holds: (P6)

For every natural number T > 2τ0 and every program {xt }Tt=0 which satisfies T −1

u(xt , xt+1 ) ≥ U (x0 , xT , 0, T ) − δ,

t=0

there exist integers τ1 ∈ [0, τ0 ], τ2 ∈ [T − τ0 , T ] such that ¯ ≤ , t = τ1 , . . . , τ2 . ρ(xt , x) Choose an integer τ > 0 satisfying τ > 8τ0 (δ −1 M + 1).

(3.38)

Set M1 = τ . Assume that an integer T > τ and that a program {xt }Tt=0 satisfies T −1

u(xt , xt+1 ) ≥ U (x0 , xT , 0, T ) − M.

(3.39)

t=0

Set T0 = 0. By induction we define a strictly increasing sequence of natural numbers Ti , i = 0, 1, . . . as follows.

46

3 Turnpike Properties

Assume that T0 , . . . , Tk have been defined where an integer k ≥ 0. If Tk = T , then the construction of the sequence is completed. Assume that Tk < T . If T ≤ Tk + 2τ0 , then set Tk+1 = T and the construction is completed. Assume that Tk < T − 2τ0 . If T −1

u(xt , xt+1 ) ≥ U (xTk , xT , 0, T − Tk ) − δ,

t=Tk

then set Tk+1 = T and the construction is completed. Assume that T −1

u(xt , xt+1 ) < U (xTk , xT , 0, T − Tk ) − δ.

t=Tk

Then there exists an integer Tk+1 > Tk such that Tk+1 ≤ T and that Tk+1 −1



u(xt , xt+1 ) < U (xTk , xTk +1 , 0, Tk+1 − Tk ) − δ,

(3.40)

u(xt , xt+1 ) ≥ U (xTk , xTk+1 −1 , 0, Tk+1 − Tk − 1) − δ.

(3.41)

t=Tk Tk+1 −2



t=Tk

Therefore by induction we have constructed a finite sequence Ti , i = 0, . . . , q where q is a natural number and Tq = T is the last element of the sequence. It follows from q (3.39) and the construction of {Tk }k=0 (see (3.40), (3.41)) that M ≥ U (x0 , xT , 0, T ) −

T −1

u(xt , xt+1 )

t=0

≥

⎧ ⎨ ⎩

Tk+1 −1

U (xTk , xTk+1 , 0, Tk+1 − Tk ) −



u(xt , xt+1 ) :

t=Tk

⎫ ⎬ k is an integer such that 0 ≤ k < q − 1 ≥ δ|q − 1|, ⎭ |q − 1| ≤ δ −1 M and q ≤ δ −1 M + 1.

(3.42)

It follows from the construction of Ti , i = 0, . . . , q that if an integer k satisfies 0 ≤ k ≤ q − 1 and Tk+1 − Tk − 1 > 2τ0 , then (3.41) is valid. Combined with property (P6) this implies that ρ(xt , x) ¯ ≤ , t = Tk + τ0 , . . . , Tk+1 − 1 − τ0 .

3.5 Proof of Theorem 3.4

47

Together with (3.42) and (3.38) this implies that ¯ > }) Card({t ∈ {0, . . . , T } : ρ(xt , x) ≤ Card({0, . . . , T } \ ∪({[Ti + τ0 , . . . , Ti+1 − 1 − τ0 ] : i is an integer such that 0 ≤ i ≤ q − 1, Ti+1 − Ti − 1 > 2τ0 })) ≤ 2qτ0 + 4qτ0 < 8qτ0 ≤ 8τ0 (δ −1 M + 1) < τ = M1 . This completes the proof of Theorem 3.3.

3.5 Proof of Theorem 3.4 Proposition 2.2 implies that for every natural number T there exists a program (T ) {xt }Tt=0 which satisfies x0(T ) = z,

T −1

  (T ) = U (z, 0, T ). u xt(T ) , xt+1

(3.43)

t=0

It follows from Proposition 3.7 and (3.43) that for every integer T > 0 and every ¯ we have pair of integers S1 , S2 satisfying 0 ≤ S1 < S2 ≤ T and S2 − S1 > 2L, S 2 −1

  (T ) (T ) u xt , xt+1 ≥ (S2 − S1 )μ − 4(L¯ + 1)(u).

(3.44)

t=S1

Theorem 3.2 and (3.43) imply that the following property holds: (P7)

For every positive number  there exists an integer τ > 0 such that for every natural number T > 2τ , we have   (T ) ρ xt , x¯ ≤ , t = τ , . . . , T − τ .

It is clear that there exists a strictly increasing sequence of natural numbers {Tj }∞ j =1 such that for all integers t ≥ 0 there exists (Tj )

xt = lim xt j →∞

.

(3.45)

Evidently, {xt }∞ t=0 is a program satisfying x0 = z.

(3.46)

48

3 Turnpike Properties

In view of (3.44) and (3.45), for all pairs of integers S1 , S2 satisfying S1 < S2 and S2 − S1 > 2L¯ the inequality S 2 −1

! u(xt , xt+1 ) ≥ (S2 − S1 )μ − 4 L¯ + 1 (u)

(3.47)

t=S1

is valid. Proposition 3.1, assumption (B1), and (3.47) imply that {xt }∞ t=0 is a good program and that lim xt = x. ¯

t→∞

(3.48)

∞ We claim that the program {xt }∞ t=0 satisfies (3.9) for every program {yt }t=0 such that y0 = z. Assume the contrary. Then there exists a program {yt }∞ t=0 such that

y0 = z, γ := lim sup

T −1

T →∞ t=0

[u(yt , yt+1 ) − u(xt , xt+1 )] > 0.

(3.49)

In view of (3.49), Proposition 3.1, (3.47), and assumption (B1) {yt }∞ =0 is a good program and lim yt = x. ¯

t→∞

(3.50)

Assumption (B2) implies that there exists a number  ∈ (0, 1) such that for every point (y1 , y2 ) ∈ Ω which satisfy ¯ ρ(y2 , x) ¯ ≤ ρ(y1 , x), the inequality ¯ x)| ¯ ≤ γ (16L∗ )−1 |u(y1 , y2 ) − u(x, holds. Assumption (B3) implies that there exists a number δ ∈ (0, ) such that the following property holds: (P8)

¯ ≤ δ, i = 1, 2 For every pair of points y1 , y2 ∈ K which satisfies ρ(yi , x) ∗ such that there exists a program {zt }L t=0 ¯ ≤ , t = 0, . . . , L∗ . z0 = y1 , zL∗ = y2 , ρ(yt , x) It follows from (3.48) and (3.50) that there exists an integer S0 > 0 such that ¯ ρ(yt , x) ¯ ≤ δ/4 for all integers t ≥ S0 . ρ(xt , x),

(3.51)

3.5 Proof of Theorem 3.4

49

In view of (3.48), there exists an integer S1 > S0 + 4L∗ + 4 for which S 1 −1

[u(yt , yt+1 ) − u(xt , xt+1 )] > γ /2.

(3.52)

t=0

It follows from (3.45) and upper semicontinuity of the function u that there exists an integer number k > 0 such that Tk > S1 + 4,

(3.53)

  (T ) ρ xt k , xt ≤ δ/4, t = 0, . . . , S1 + 4,

(3.54)

  (Tk ) ≤ u(xt , xt+1 ) + γ (16(S1 + 4))−1 , t = 0, . . . , S1 + 4. u xt(Tk ) , xt+1

(3.55)

Relations (3.51), (3.53), and (3.54) imply that     (T ) (T ) ¯ ≤ δ/4, ρ xS1 k , x¯ ≤ ρ xS1 k , xS1 + ρ(xS1 , x) ¯ ≤ δ/4 + δ/4. ρ(yS1 −L∗ , x) (3.56) k By (3.56) and property (P8), there exists a program {x¯t }Tt=0 such that x¯t = yt , t ∈ {0, . . . , S1 − L∗ }, x¯t = xt(Tk ) , t = S1 , . . . , Tk ,

(3.57)

¯ ≤ , t = S1 − L∗ , . . . , S1 . ρ(x¯t , x) It follows from (3.57), (3.49), (3.43), (3.55), the choice of , (3.51), and (3.52) that 0≤

T k −1

(Tk )

u(xt

(T )

, xt+1k ) −

T k −1

t=0

=

S 1 −1

(T ) (T ) u(xt k , xt+1k ) −

S 1 −1

t=0

−

u(x¯t , x¯t+1 )

t=0

u(x¯t , x¯t+1 ) ≤

t=0

S1 −L ∗ −1

S 1 −1

u(xt , xt+1 ) + γ (16)−1

t=0

u(yt , yt+1 ) − (u(x, ¯ x) ¯ − γ (16L∗ )−1 )(2L∗ )

t=0

≤ γ (16)−1 + γ /8 +

S 1 −1

u(xt , xt+1 )

t=0

−

S 1 −1

u(yt , yt+1 ) + γ /8 ≤ γ /4 + γ 16−1 − γ /2 < 0.

t=0

The contradiction we have reached completes the proof of Theorem 3.4.

50

3 Turnpike Properties

3.6 Proof of Theorem 3.5 It is clear that if {xt }∞ t=0 is an overtaking optimal program, then it is locally optimal. Assume that the program {xt }∞ t=0 is locally optimal. Proposition 3.7 implies that the program {xt }∞ is good. By assumption (B1), t=0 lim xt = x. ¯

(3.58)

t→∞

Theorem 3.4 implies that there exists an overtaking optimal program {yt }∞ t=0 satisfying y0 = x0 .

(3.59)

It is clear that the program {yt }∞ t=0 is good. In view of assumption (B1), ¯ lim yt = x.

(3.60)

t→∞

We show that the program {xt }∞ t=0 is overtaking optimal. Assume the contrary. Then γ := lim sup T →∞

T −1 

 u(yt , yt+1 ) − u(xt , xt+1 ) > 0.

(3.61)

t=0

Assumption (B2) implies that there exists a number  ∈ (0, γ /2) such that for every point (z1 , z2 ) ∈ Ω which satisfies ρ(z1 , x), ¯ ρ(z2 , x) ¯ ≤ , the inequality ¯ x)| ¯ ≤ γ (16L∗ )−1 |u(z1 , z2 ) − u(x,

(3.62)

holds. In view of assumption (B3), there exists a number δ ∈ (0, ) such that the following property holds: (P9)

¯ ≤ δ, i = 1, 2 For every pair of points z1 , z2 ∈ K which satisfies ρ(zi , x) ∗ such that there exists a program {ξt }L t=0 ¯ ≤ , t = 0, . . . , L∗ . ξ0 = z1 , ξL∗ = z2 , ρ(ξt , x) It follows from (3.58) and (3.60) that there exists an integer S0 > 4L∗ + 4 such that ¯ ρ(yt , x) ¯ ≤ δ for all integers t ≥ S0 − L∗ − 2. ρ(xt , x),

(3.63)

3.7 Proof of Theorem 3.6

51

In view of (3.61), there exists a natural number S1 > S0 + 2 + 2L∗ such that S 1 −1

(u(yt , yt+1 ) − u(xt , xt+1 )) > γ /2.

(3.64)

t=0 1 such that Property (P9) and (3.63) imply that there exists a program {x˜t }St=0

x˜t = yt , t = 0, . . . , S1 − L∗ , ρ(xt , x) ¯ ≤ , t = S1 − L∗ , . . . , S1 ,

(3.65)

x˜S1 = xS1 . It follows from local optimality of {xt }∞ t=0 , (3.65), (3.59), (3.64), (3.63), and the choice of  (see (3.62)) that 0≤

S 1 −1

u(xt , xt+1 ) −

t=0

=

u(x˜t , x˜t+1 )

t=0

S 1 −1

u(xt , xt+1 ) −

t=0

+

S 1 −1

S 1 −1

u(yt , yt+1 )

t=0 S 1 −1

u(yt , yt+1 ) −

t=S1 −L∗

S 1 −1

u(x˜t , x˜t+1 )

t=S1 −L∗

≤ −γ /2 + γ 16−1 L−1 ∗ (2L∗ + 2) < 0, a contradiction. The contraction we have reached proves Theorem 3.5.

3.7 Proof of Theorem 3.6 In view of Theorem 3.2, there exist an integer τ0 > 0 and a positive number δ such that the following property holds: (P10)

For every natural number T > 2τ0 and every program {xt }Tt=0 which satisfies T −1 t=0

u(xt , xt+1 ) ≥ U (x0 , xT , 0, T ) − δ,

52

3 Turnpike Properties

there exist integers τ1 ∈ [0, τ0 ], τ2 ∈ [T − τ0 , T ] such that ¯ ≤ , t = τ1 , . . . , τ2 , ρ(xt , x) ¯ ≤ δ and τ2 = T if ρ(xT , x) ¯ ≤ δ. τ1 = 0 if ρ(x0 , x) Set τ = 3τ0 + 1.

(3.66)

Assume that an integer T > 2τ and that a program {xt }Tt=0 for every integer S ∈ {0, . . . , T − τ } satisfies S+τ −1

u(xt , xt+1 ) ≥ U (x(S), x(S + τ ), S, S + τ ) − δ.

(3.67)

t=S

Property (P10), (3.66), and (3.67) imply that ¯ ≤ , t = τ0 , . . . , τ − τ0 , ρ(xt , x) ¯ ≤ , t = T − τ + τ0 , . . . , T − τ0 , ρ(xt , x) ¯ ≤ δ, then ρ(xt , x) ¯ ≤ , t = 0, . . . , τ − τ0 and if ρ(xT , x) ¯ ≤ δ, then if ρ(x0 , x) ¯ ≤ , t = T − τ + τ0 , . . . , T . ρ(xt , x) In order to complete the proof it is sufficient to show that ¯ ≤ , t = τ − τ0 , . . . , T − τ0 . ρ(xt , x) Assume that an integer S ∈ [τ − τ0 , T − τ0 ].

(3.68)

It follows from (3.66) and (3.68) that there exist integers S1 , S2 ∈ [0, T ] such that S1 + τ0 ≤ S ≤ S2 − τ0 , S2 − S1 = τ. In view of (3.69), (3.67), and property (P10), ¯ ≤ . ρ(xS , x) Theorem 3.6 is proved.

(3.69)

3.8 Stability Results

53

3.8 Stability Results For every function φ : Ω → R 1 set φ = sup{|φ(z)| z ∈ Ω}. Let T1 , T2 be integers such that T1 < T2 and ut : Ω → R 1 , t = T1 , . . . T2 − 1 be bounded functions. For every pair of points z0 , z1 ∈ K define ⎧ 2 −1 ⎨T   T2 −1 2 U {ut }t=T = sup , z , z ut (xt , xt+1 ) : {xt }Tt=T is a program 0 1 1 1 ⎩ t=T1

such that xT1 = z0 , xT2 = z1

⎫ ⎬ ⎭

and ⎧ 2 −1 ⎨T   T2 −1 2 = sup U {ut }t=T , z ut (xt , xt+1 ) : {xt }Tt=T is a program 0 1 1 ⎩ t=T1

such that xT1 = z0

⎫ ⎬ ⎭

.

(Here we assume that supremum over an empty set is −∞.) It is easy to see that the following result holds. Proposition 3.11 Let y0 , y˜0 ∈ K, T1 < T2 be integers, ut , t = T1 , . . . , T2 − 1 be T2 −1 bounded and upper semicontinuous functions and let U ({ut }t=T , y0 , y˜0 ) be finite. 1 2 Then there exists a program {xt }Tt=T such that 1

T 2 −1

  T2 −1 , xT1 = y0 , xT2 = y˜0 . ut (xt , xt+1 ) = U {ut }t=T , y , y ˜ 0 0 1

t=T1

Theorem 3.12 Let  be a positive number. Then there exist an integer τ > 0 and a positive number γ such that for each natural number T > 2τ , each sequence of bounded functions ut : Ω → R 1 , t = 0, . . . , T − 1 satisfying ut − u ≤ γ , t = 0, 1, . . . , T − 1 and each program {xt }Tt=0 which satisfies S+τ −1 t=S

  S+τ −1 ut (xt , xt+1 ) ≥ U {ut }t=S xS , xS+τ − γ

54

3 Turnpike Properties

for every integer S ∈ {0, . . . , T − τ } there exist integers τ1 , τ2 such that τ1 ∈ [0, τ ], τ2 ∈ [T − τ, T ], ¯ ≤ , t = τ1 , . . . τ2 . ρ(xt , x) Moreover, τ1 = 0 if ρ(x0 , x) ¯ ≤ γ and τ2 = T if ρ(xT , x) ¯ ≤ γ. Proof Theorem 3.12 follows easily from Theorem 3.6. Namely let a natural number τ and δ > 0 be as guaranteed by Theorem 3.6. Put γ = δ(4(τ + 1))−1 . Now it is not difficult to see that the assertion of Theorem 3.12 holds. Theorem 3.12 implies the following result. Theorem 3.13 Let  > 0. Then there exist a natural number τ , λ > 1, and γ > 0 such that for each integer T > 2τ , each sequence of bounded functions ut : Ω → R 1 , t = 0, . . . , T − 1 satisfying ut − u ≤ γ , t = 0, 1, . . . , T − 1, each sequence T −1 {αi }i=0 ⊂ (0, 1] such that for each i, j ∈ {0, . . . , T − 1} satisfying |i − j | ≤ τ the inequality αi αj−1 ≤ λ holds and each program {xt }Tt=0 which satisfies S+τ −1

  S+τ −1 αt ut (xt , xt+1 ) ≥ U {αt ut }t=S , xS , xS+τ − γ αS

t=S

for each integer S ∈ {0, . . . , T − τ } there exist integers τ1 , τ2 such that τ1 ∈ [0, τ ], τ2 ∈ [T − τ, T ],

(3.70)

¯ ≤ , t = τ1 , . . . τ2 . ρ(xt , x) Moreover, ¯ ≤ γ, τ1 = 0 if ρ(x0 , x)

(3.71)

¯ ≤ γ. τ2 = T if ρ(xT , x)

(3.72)

and

Theorem 3.13 implies the following result. Theorem 3.14 Let  > 0. Then there exist a natural number τ , γ > 0, and λ > 1 such that for each integer T > 2τ , each sequence of bounded upper semicontinuous functions ut : Ω → R 1 , t = 0, . . . , T −1 satisfying ut −u ≤ γ , t = 0, 1, . . . , T −

3.8 Stability Results

55

T −1 1, each sequence {αi }i=0 ⊂ (0, 1] such that for each i, j ∈ {0, . . . , T −1} satisfying |i −j | ≤ τ the inequality αi αj−1 ≤ λ holds and each program {xt }Tt=0 which satisfies T −1

  T −1 αt ut (xt , xt+1 ) = U {αt ut }t=0 , x0 , xT

t=0

there exist integers τ1 , τ2 such that (3.70)–(3.72) hold. Theorems 3.12–3.14 were obtained from [83]. The next turnpike result is new. Theorem 3.15 Let , M be positive numbers. Then there exist a natural number Q and γ > 0 such that for each integer T > Q, each finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . , T −1 satisfying ut −u ≤ γ t = 0, 1, . . . , T − 1, and each program {xt }Tt=0 which satisfies T −1

T −1 ut (xt , xt+1 ) ≥ U ({ut }t=0 , x0 , xT ) − M

t=0

the following inequality holds: Card({t ∈ {0, . . . , T } : ρ(xt , x) ¯ > }) ≤ Q. Proof Theorem 3.12 implies that there exist an integer τ ≥ 1 and a positive number γ such that the following property holds: (i) for each integer S > 2τ , each finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . , S − 1 satisfying ut − u ≤ γ , t = 0, . . . , S − 1 and each program {xt }St=0 which satisfies S−1 

ut (xt , xt+1 ) ≥ U ({ut }S−1 t=0 , x0 , xS ) − γ

t=0

we have ¯ ≤ , t = τ, . . . , S − τ. ρ(xt , x) Choose a natural number Q > 9τ (γ −1 M + 1).

(3.73)

56

3 Turnpike Properties

Assume that an integer T > Q, a finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . , T − 1 satisfies ut − u ≤ γ , t = 0, . . . , T − 1

(3.74)

and that a program {xt }Tt=0 satisfies T −1

  ut (xt , xt+1 ) ≥ U {ut }S−1 t=0 , x0 , xT − M.

(3.75)

t=0

Set T0 = 0. By induction we define a strictly increasing sequence of integers T0 < · · · < Tk ≤ T . Assume that k ≥ 0 is an integer and T0 , . . . , Tk have been defined. If Tk = T , then the construction of the sequence is completed. Assume that Tk < T . In T ≤ Tk + 2τ , then we set Tk+1 = T and the construction is completed. Assume that Tk < T − 2τ. If T −1

  T −1 − γ, ut (xt , xt+1 ) ≥ U {ut }t=T , x , x T T k k

t=Tk

then set Tk+1 = T and the construction is completed. Assume that T −1

  T −1 ut (xt , xt+1 ) < U {ut }t=T , xTk , xT − γ . k

t=Tk

Then there exists an integer Tk+1 ≤ T such that Tk+1 ≥ Tk + 2, Tk+1 −1



  Tk+1 −1 −γ ut (xt , xt+1 ) < U {ut }t=T , x , x T T k k+1 k

(3.76)

  Tk+1 −2 ut (xt , xt+1 ) ≥ U {ut }t=T , xTk , xTk+1 −1 − γ . k

(3.77)

t=Tk

and Tk+1 −2



t=Tk

3.8 Stability Results

57

Thus by induction we have constructed a finite sequence of integers T 0 = 0 < · · · , Tq = T , q

where q > 0 is an integer. It follows from (3.75) and the construction of {Ti }i=0 (see (3.76) and (3.77)) that T −1 , x0 , xT ) − M ≥ U ({ut }t=0

T −1

ut (xt , xt+1 )

t=0

≥



T

−1

Tk+1 −1

k+1 , xTk , xTk+1 ) − {U ({ut }t=T k



ut (xt , xt+1 ) :

t=Tk

k is an integer satisfying 0 ≤ k < q − 1} ≥ γ (q − 1) and q ≤ γ −1 M + 1. It follows from property (i), (3.74), and the construction of Ti , i = 0, . . . , q that if an integer k satisfies 0 ≤ k ≤ q − 1, Tk+1 − Tk > 2τ + 1, then (3.77) holds and ρ(xt , x) ¯ ≤ , t = Tk + τ, . . . , Tk+1 − τ − 1. Combined with (3.73) and the inequality q ≤ γ −1 M + 1, this implies that ¯ > }) Card({t ∈ {0, . . . , T } : ρ(xt , x) ≤ Card({t ∈ {0, . . . , T } \ ∪({Ti + τ, . . . , Ti+1 − τ − 1} : i is an integer satisfying 0 ≤ i ≤ q − 1, Ti+1 − Ti > 2τ + 1)}) ≤ 3qτ + 6qτ ≤ 9τ (γ −1 M + 1) < Q. Theorem 3.15 is proved.

58

3 Turnpike Properties

3.9 Agreeable Programs A program {xt∗ }∞ t=0 is called agreeable if for any natural number T0 and any  > 0 there exists an integer T > T0 such that for any integer T ≥ T there exists a program {xt }Tt=0 which satisfies xt = xt∗ , t = 0, . . . , T0 and T −1

u(xt , xt+1 ) ≥ U (x0∗ , 0, T ) − .

t=0

The notion of agreeable programs is well known in the economic literature [34– 36]. In this section we employ a strong version of it. A program {xt∗ }∞ t=0 is called strongly agreeable if for any natural number T0 and any  > 0 there exist an integer T > T0 and δ > 0 such that for each integer T ≥ T and each finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . , T satisfying ut − u ≤ δ, t = 0 . . . , T − 1 there exists a program {xt }Tt=0 which satisfies xt = xt∗ , t = 0, . . . , T0 and T −1

  T −1 ∗ ut (xt , xt+1 ) ≥ U {ut }t=0 , x0 − .

t=0

Theorem 3.16 Let {xt }∞ t=0 be a program. Then the following properties are equivalent: (a) the program {xt }∞ t=0 is locally optimal; (b) the program {xt }∞ t=0 is agreeable; (c) the program {xt }∞ t=0 is strongly agreeable. Proof Clearly, property (c) implies property (b) and property (b) implies property (a). We show that (a) implies (c). Assume that the program {xt }∞ t=0 is locally optimal. We show that the program is strongly agreeable. {xt }∞ t=0 Let  ∈ (0, 1) and T0 be a natural number. By (B2) there is a positive number δ1 < /2 such that the following property holds:

3.9 Agreeable Programs

59

(i) for each (x, y) ∈ Ω satisfying ¯ ≤ δ1 ρ(x, x) ¯ ≤ δ1 , ρ(y, x) we have |u(x, y) − u(x, ¯ x)| ¯ ≤ (/8)(L∗ + 1)−1 . By (B3) there exists δ2 ∈ (0, δ1 ) such that the following property holds: (ii) for each z1 , z2 ∈ K satisfying ¯ ≤ δ2 , i = 1, 2 ρ(zi , x) ∗ there exists a program {ξt }L t=0 such that

¯ ≤ δ1 , t = 0, . . . , L∗ , ρ(ξt , x) ξ0 = z1 , ξL∗ = z2 . By Theorem 3.12, there exist a natural number L1 and δ3 ∈ (0, δ2 ) such that the following property holds: (iii) for each integer T > 2L1 , each finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . , T − 1 satisfying ut − u ≤ δ3 , t = 0 . . . , T − 1 and each program {zt }Tt=0 which satisfies T −1

T −1 ut (zt , zt+1 ) ≥ U ({ut }t=0 , z0 , zT ) − δ3

t=0

we have ¯ ≤ δ2 , t = L1 , . . . , T − L1 . ρ(zt , x) Properties (a) and (iii) imply that ¯ ≤ δ2 for all integers t ≥ L1 . ρ(xt , x)

(3.78)

T > T0 + 2L∗ + 2L1 + 8

(3.79)

Choose an integer

60

3 Turnpike Properties

and set δ = δ3 (2T + 2)−1 /16.

(3.80)

Assume that an integer T ≥ T , a finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . , T − 1 satisfies ut − u ≤ δ, t = 0 . . . , T − 1.

(3.81)

There exists a program {yt }Tt=0 such that y0 = x0 , T −1

  −1 ut (yt , yt+1 ) ≥ U {ut }Tt=0 , x0 − δ.

(3.82) (3.83)

t=0

It follows from property (iii) and (3.79)–(3.83) that ρ(yt , x) ¯ ≤ δ2 , t = L1 , . . . , T − L1 .

(3.84)

 Property (ii), (3.78), (3.79), and (3.84) imply that there exists a program {zt }Tt=0 such that

zt = yt , t = 0, . . . , T − L∗ − L1 ,

(3.85)

zT −L1 = xT −L1 ,

(3.86)

¯ ≤ δ1 , t = T − L1 − L∗ , . . . , T − L1 , ρ(zt , x)

(3.87)

zt = xt , t = T − L1 , . . . , T .

(3.88)

By properties (a) and (i) and (3.81), (3.85)–(3.87), T −L 1 −1

u(xt , xt+1 ) ≥

t=0

T −L 1 −1

u(zt , zt+1 )

t=0

≥

T −L ∗ −L1 −1

u(yt , yt+1 ) + L∗ (u(x, ¯ x) ¯

t=0

− (/8)(L∗ + 1)−1 ) ≥

T −L ∗ −L1 −1 t=0

u(yt , yt+1 ) + L∗ u(x, ¯ x) ¯ − /8.

(3.89)

3.9 Agreeable Programs

61

Property (i), (3.78), (3.79), and (3.89) imply that T −L ∗ −L1 −1

u(xt , xt+1 )

t=0

≥

T −L ∗ −L1 −1

u(yt , yt+1 ) + L∗ u(x, ¯ x) ¯ − /8

t=0 T −L 1 −1

−

u(xt , xt+1 )

t=T −L∗ −L1 −1

≥

T −L ∗ −L1 −1

u(yt , yt+1 ) + L∗ u(x, ¯ x) ¯ − /8 − L1 u(x, ¯ x) ¯ − /8

t=0

=

T −L ∗ −L1 −1

u(yt , yt+1 ) − /4.

(3.90)

t=0

It follows from (3.80), (3.81), and (3.90) that T −L ∗ −L1 −1

T −L ∗ −L1 −1

ut (xt , xt+1 ) ≥

t=0

ut (yt , yt+1 ) − /4 − /8.

(3.91)

t=0

Property (ii), (3.78), (3.79), and (3.84) imply that there exists a program {ξt }Tt=0 such that ξt = xt , t = 0, . . . , T − L∗ − L1 , ρ(ξt , x) ¯ ≤ δ1 , t = T − L1 − L∗ + 1, . . . , T − L1 , ξt = yt , t = T − L1 , . . . , T .

ut (ξt , ξt+1 ) −

T −1

t=0

≥

ut (yt , yt+1 )

t=0 T −L ∗ −L1 −1

ut (xt , xt+1 ) −

T −L ∗ −L1 −1

t=0

+

T −L 1 −1 t=T −L∗ −L1

ut (yt , yt+1 )

t=0

ut (ξt , ξt+1 ) −

(3.93) (3.94)

By property (i), (3.78), (3.79), (3.84), and (3.91)–(3.94), T −1

(3.92)

T −L 1 −1

ut (yt , yt+1 )

t=T −L∗ −L1

≥ −3/8 − 2L∗ (/8)(L∗ + 1)−1 ≥ −5/8.

62

3 Turnpike Properties

Combined with (3.80) and (3.83) this implies that T −1 t=0

ut (ξt , ξt+1 ) ≥

T −1

ut (yt , yt+1 ) − 5/8

t=0

T −1 ≥ U ({ut }t=0 , x0 ) − 5/8 − δ T −1 ≥ U ({ut }t=0 , x0 ) − .

Thus (c) holds. Theorem 3.16 is proved.

Chapter 4

Generic Turnpike Properties

In this chapter we consider a class of discrete-time optimal control problems identified with a complete metric space of objective functions. Using the Baire category approach, we show that for most problems the results of Chap. 3 are true. In particular, a typical (generic) problem possesses the turnpike property.

4.1 Preliminaries We continue to study the class of optimal control problems introduced in Sect. 2.2 using the same notation, definitions, and assumptions. Let (K, ρ) be a compact metric space and let Ω be a nonempty closed subset of K × K. We recall that a sequence {xt }∞ t=0 ⊂ K is called an (Ω)-program (or a program if Ω is understood) if (xt , xt+1 ) ∈ Ω for all integers t ≥ 0. 2 Let the integers T1 , T2 satisfy T1 < T2 . A sequence {xt }Tt=T ⊂ K is called an 1 (Ω)-program (or a program if Ω is understood) if (xt , xt+1 ) ∈ Ω for all integers t satisfying T1 ≤ t < T2 . Let u : Ω → R 1 be a bounded upper semicontinuous function. Set u = sup{|u(z)| : z ∈ Ω}. Recall that the supremum over empty set is −∞. For each pair of integers T1 , T2 satisfying 0 ≤ T1 < T2 and each y, y˜ ∈ K define U (u, y, T1 , T2 ) = sup

⎧ 2 −1 ⎨T ⎩

2 u(xt , xtt+1 ) : {xt }Tt=T is a program and xT1 = y 1

t=T1

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2019 A. J. Zaslavski, Optimal Control Problems Arising in Forest Management, SpringerBriefs in Optimization, https://doi.org/10.1007/978-3-030-23587-1_4

⎫ ⎬ ⎭

,

63

64

4 Generic Turnpike Properties

U (u, y, y, ˜ T1 , T2 ) = sup

⎧ 2 −1 ⎨T ⎩

u(xt , xtt+1 ) :

t=T1

2 {xt }Tt=T is a program and xT1 = y, xT2 = y˜ 1

(u, T1 , T2 ) = sup U

⎧ 2 −1 ⎨T ⎩

t=T1

⎫ ⎬ ⎭

, ⎫ ⎬

2 u(xt , xtt+1 ) : {xt }Tt=T is a program . 1 ⎭

In this chapter we suppose that the following assumption, introduced in Sect. 2.2, holds. (A)

There exists a natural number L¯ such that for each y, z ∈ K there is a program ¯ {xt }L t=0 such that x0 = y and xL¯ = z.

This assumption holds for the important class of forest management problems. Note that in Chaps. 2 and 3 the function u was fixed and for each pair of integers T1 , T2 satisfying 0 ≤ T1 < T2 and each y, y˜ ∈ K we used the notation ˜ T1 , T2 ), U (y, y, ˜ T1 , T2 ) = U (u, y, y, U (y, T1 , T2 ) = U (u, y, T1 , T2 ), (T1 , T2 ) = U (u, T1 , T2 ). U Note that all the results stated in Sect. 2.2 (Theorems 2.6, 2.7, 2.9, and 2.10 and Propositions 2.5 and 2.8) hold in our case with ut = u, t = 0, 1, . . . . The results of this chapter are new.

4.2 Equivalence of the Turnpike Properties We use the constant μ defined by Theorem 2.9 and suppose that μ = sup{u(x, x) : x ∈ K and (x, x) ∈ Ω}.

(4.1)

By the upper semicontinuity of u, there exists x¯ ∈ K such that (x, ¯ x) ¯ ∈ Ω and u(x, ¯ x) ¯ = μ.

(4.2)

Recall that a program {xt }∞ t=0 is called (u)-good (or good if the function u is understood) if the sequence

4.2 Equivalence of the Turnpike Properties

T −1 

65

∞ {u(xt , xt+1 ) − μ T =1

t=0

is bounded. A program {xt }∞ t=0 is called (u)-bad (or bad if the function u is understood) if T −1 t=0 (u(xt , xt+1 ) − μ) → −∞ as T → ∞. By Proposition 3.1, any program is either good or bad and for any x0 ∈ K there is a good program {xt }∞ t=0 . We suppose that assumptions (B2) and (B3) introduced in Sect. 3.1 hold. (B2) (B3)

u is continuous at (x, ¯ x) ¯ with respect to Ω. There exists a natural number L∗ such that for each  > 0 there exists δ > 0 such that for each y1 , y2 ∈ K satisfying ρ(y1 , x), ¯ ρ(x, ¯ y2 ) ≤ δ there is ∗ such that x = y , x = y and ρ(x ¯ ≤ , t = a program {xt }L 0 1 L 2 t , x) ∗ t=0 0, . . . , L∗ .

In Chap. 3 we also use the following assumption. (B1)

¯ For each good program {xt }∞ t=0 , limt→∞ xt = x.

Property (B1) is known in the literature and is called as the asymptotic turnpike property. Clearly, if (B1) holds, then all the results of Chap. 3 are true. In particular, Theorem 3.2, a turnpike result, is true. The next result shows that if the turnpike property established by Theorem 3.2 holds, then (B1) holds too. Theorem 4.1 Assume that for each  > 0 there exist a natural number τ and δ > 0 such that the following property holds: (i) for each integer T > 2τ and each program {zt }Tt=0 satisfying T −1

u(zt , zt+1 ) ≥ U (z0 , zT , 0, T ) − δ

t=0

the inequality ρ(zt , x) ¯ ≤ holds for all integers t = τ, . . . , T − τ. Then for each good program {xt }∞ t=0 , lim xt = x. ¯

t→∞

66

4 Generic Turnpike Properties

Proof Assume that {xt }∞ t=0 is a good program. Then there exists M > 0 such that for each natural number T , T −1      u(xt , xt+1 ) − T μ ≤ M.   

(4.3)

t=0

Let  > 0 and δ > 0 and a natural number τ be such that property (i) holds. We show that there exists an integer Tδ ≥ 0 such that for each integer T > Tδ , T −1

u(xt , xt+1 ) ≥ U (xTδ , xT , Tδ , T ) − δ.

(4.4)

t=Tδ

Assume the contrary. Then there exists a strictly increasing sequence of integers {Ti }∞ i=0 such that T0 = 0 and for each integer i ≥ 1, Ti+1 −1



u(xt , xt+1 ) < U (xTi , xTi+1 , Ti , Ti+1 ) − δ.

(4.5)

t=Ti

By (4.5), there exists a program {yt }∞ t=0 such that for each integer i ≥ 0, yTi = xTi , Ti+1 −1



Ti+1 −1

u(xt , xt+1 )
Tδ , (4.4) is true.

4.3 Generic Results

67

Properties (i) and (ii), (4.9), and the choice of δ, τ imply that for each integer T > Tδ + 2τ , we have ρ(xt , x) ¯ ≤ , t = τ + Tδ , . . . , T − τ. This implies that ¯ ≤  for all integers t ≥ τ + Tδ . ρ(xt , x) Theorem 4.1 is proved.

4.3 Generic Results We use the notation, definitions, and assumptions introduced in Sect. 4.1. Denote by C(Ω) the set of all continuous functions on Ω. Clearly, (C(Ω),  · ) is a Banach space. For each u1 , u2 ∈ C(Ω) set d(u1 , u2 ) = u1 − u2 . Let v ∈ C(Ω). By Theorem 2.9 there exist (v, 0, p)p−1 μ(v) = lim U

(4.6)

   (v, 0, p)p−1  ≤ Mv /p μ(v) − U

(4.7)

p→∞

and Mv > 0 such that

for all natural numbers p. Assume that there exists z ∈ K such that (z, z) ∈ Ω. Denote by M the set of all v ∈ C(Ω) such that μ(v) = sup{v(x, x) : x ∈ K and (x, x) ∈ Ω}.

(4.8)

It is not difficult to see that M is a closed subset of C(Ω). We consider the complete metric space (M, d). Suppose that the following assumption holds. (B4)

There exists a natural number L∗ such that for each  > 0 there exists δ > 0 such that for each y1 , y2 ∈ K satisfying ρ(y1 , y2 ) ≤ δ there is a program ∗ {xt }L t=0 such that x0 = y1 , xL∗ = y2

68

4 Generic Turnpike Properties

and ρ(xt , x) ¯ ≤ , t = 0, . . . , L∗ . Clearly, (B4) is a strong version of (B3). Let v ∈ M. Recall that a program {xt }∞ t=0 is called (v)-good if the sequence T −1 

∞ {v(xt , xt+1 ) − μ(v) T =1

t=0

is bounded. Theorem 4.2 There exists a set F ⊂ M which is a countable intersection of open everywhere dense subsets of M such that for each v ∈ F the following properties hold: 1. there exists a unique point xv ∈ K such that μ(v) = v(xv , xv ). 2. for each (v)-good program {xt }∞ t=0 , ¯ = 0. lim ρ(xt , x)

t→∞

It is clear that if u ∈ F, then all the results of Chap. 3 hold for u. Proof of Theorem 4.2 Let v ∈ M and r > 0. There exists xv ∈ K such that (xv , xv ) ∈ Ω, μ(v) = v(xv , xv ).

(4.9)

vr (x, y) = v(x, y) − rρ(x, xv ).

(4.10)

For (x, y) ∈ Ω define

It is not difficult to see that vr ∈ M and μ(vr ) = μ(v) = v(xv , xv ).

(4.11)

Clearly, the set {vr : v ∈ M, r ∈ (0, 1]} is an everywhere dense subset of M. Lemma 4.3 Let v ∈ M and r ∈ (0, 1]. Assume that {xt }∞ t=0 is a (vr )-good program. Then lim ρ(xt , xv ) = 0.

t→∞

4.3 Generic Results

69

Proof There exists M > 0 such that T −1      vr (xt , xt+1 ) − T v(xv , xv ) ≤ M for all natural numbers T.   

(4.12)

t=0

Proposition 3.1, (4.10), and (4.12) imply that for all natural numbers T , −M ≤

T −1

v(xt , xt+1 ) − T v(xv , xv ) − r

t=0

T −1

ρ(xt , xv ).

(4.13)

t=0

In view of (4.13), the program {xt }∞ t=0 is (v)-good and lim

T −1

T →∞

ρ(xt , xv ) < ∞.

t=0

This implies that lim ρ(xt , xv ).

t→∞

Lemma 4.3 is proved. In view of Lemma 4.3, all the results of Chap. 3 are true for u = vr . Completion of the Proof of Theorem 4.2 Let v ∈ M, r ∈ (0, 1] and n be a natural number. Theorem 3.12 implies that there exist an open neighborhood U (v, r, n) of vr in the metric space (M, d), a natural number τ (v, r, n), and γ (v, r, n) > 0 such that the following property holds: (i) for each integer T > 2τ (v, r, n), each ut ∈ U (v, r, n), t = 0, . . . , T − 1 and each program {xt }Tt=0 which satisfies T −1

  T −1 ut (xt , xt+1 ) ≥ U {ut }t=0 x0 , xT − γ (v, r, n)

t=0

we have ρ(xt , xv ) ≤ 1/n, t = τ (v, r, n), . . . , T − τ (v, r, n). Define F = ∩∞ p=1 ∪ {U (v, r, n) : v ∈ M, r ∈ (0, 1], n ≥ p is an integer}.

(4.14)

Clearly, F is a countable intersection of open everywhere dense sets in (M, d).

70

4 Generic Turnpike Properties

Let w ∈ F,  ∈ (0, 1).

(4.15)

Choose a natural number p such that p > 4 −1 .

(4.16)

It follows from (4.14) and (4.15) that there exist v ∈ M, r ∈ (0, 1], and an integer n ≥ p such that w ∈ U (v, r, n).

(4.17)

Assume that ξ1 , ξ2 ∈ K, w(ξi , ξi ) = μ(w), i = 1, 2.

(4.18)

It is not difficult to see that for each i = 1, 2 and each integer T > 0, U (w, ξi , ξi , 0, T ) = T w(ξi , ξi ). Combined with property (i) this implies that ρ(xv , ξi ) ≤ n−1 , i = 1, 2.

(4.19)

By (4.16) and (4.19), ρ(ξ1 , ξ2 ) ≤ 2/n ≤ 2/p < . Since  is an arbitrary positive number we conclude that ξ1 = ξ2 and {ξ ∈ K : (ξ, ξ ) ∈ Ω, w(ξ, , ξ ) = μ(w)} = {ξ1 }, a singleton. Assume that an integer T > 2τ (v, r, n) and that a program {xt }Tt=0 satisfies T −1 t=0

w(xt , xt+1 ) ≥ U (w, x0 , xT , 0, T ) − γ (v, r, n).

4.3 Generic Results

71

Together with property (i), (4.16), (4.17), and (4.19) this implies that for all integers t = τ (v, r, n), . . . , T − τ (v, r, n), ρ(xt , xv ) ≤ 1/n and ρ(xt , ξ1 ) ≤ ρ(xt , xv ) + ρ(xv , ξ1 ) ≤ 2/n ≤ 2/p < . Combined with Theorem 4.1 this implies that every (w)-good program converges to ξ1 . Theorem 4.2 is proved.

Chapter 5

Structure of Solutions in the Regions Close to the Endpoints

In this chapter we continue to study the structure of approximate solutions of the autonomous nonconcave discrete-time optimal control system with a compact metric space of states. This control system is described by a bounded upper semicontinuous objective function which determines an optimality criterion. In the turnpike theory, it is known that approximate solutions are determined mainly by the objective function, and are essentially independent of the choice of time intervals and data, except in regions close to the endpoints of the time intervals. In this chapter our main goal is to analyze the structure of approximate solutions in regions close to the endpoints of the time intervals.

5.1 Preliminaries We continued to study the class of optimal control problems studied in Chap. 3 (see Sects. 3.1 and 3.8) using the same notation, definitions, and assumptions. Let (K, ρ) be a compact metric space and let Ω be a nonempty closed subset of K × K. Let u : Ω → R 1 be a bounded upper semicontinuous function. In this chapter we suppose that the following assumption, introduced in Sect. 2.2, holds. (A)

There exists a natural number L¯ such that for each y, z ∈ K there is a (Ω)¯ program {xt }L t=0 such that x0 = y and xL¯ = z.

We also assume that there is x¯ ∈ K such that (x, ¯ x) ¯ ∈ Ω and u(x, ¯ x) ¯ = μ(u) = μ.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2019 A. J. Zaslavski, Optimal Control Problems Arising in Forest Management, SpringerBriefs in Optimization, https://doi.org/10.1007/978-3-030-23587-1_5

(5.1)

73

74

5 Structure of Solutions in the Regions Close to the Endpoints

We also assume that the following assumptions (B1)–(B3) introduced in Sect. 3.1 hold. (B1) (B2) (B3)

For each (u, Ω)-good program {xt }∞ ¯ t=0 , limt→∞ xt = x. u is continuous at (x, ¯ x) ¯ with respect to Ω. There exists a natural number L∗ such that for each  > 0 there exists δ > 0 such that for each y1 , y2 ∈ K satisfying ρ(y1 , x), ¯ ρ(x, ¯ y2 ) ≤ δ there is an L∗ ¯ ≤ , t = (Ω)-program {xt }t=0 such that x0 = y1 , xL∗ = y2 and ρ(xt , x) 0, . . . , L∗ .

We study the structure of approximate solutions of problems (P1 )–(P3 ) (see Sect. 3.1). We define a function π u (x), x ∈ K which plays an important role in our study. Let x ∈ K. Set  π (x) = sup lim sup u

T −1

T →∞ t=0

{xt }∞ t=0

(u(xt , xt+1 ) − v(x, ¯ x)) ¯ : 

is an (Ω) − program such that x0 = x .

(5.2)

In view of Theorems 2.6 and 2.9, there exists Mu > 0 such that − Mu ≤ π u (x) ≤ Mu for all x ∈ K.

(5.3)

Let x ∈ K. By (5.3),  π u (x) = sup lim sup

T −1

T →∞ t=0

{xt }∞ t=0

(u(xt , xt+1 ) − u(x, ¯ x)) ¯ : 

is an (u, Ω) − good program such that x0 = x .

(5.4)

Denote by P(u, x) the set of all (u, Ω)-overtaking optimal programs {xt }∞ t=0 satisfying x0 = x. By Theorem 3.4, the set P(u, x) is nonempty. Definition (5.2) implies the following result. Proposition 5.1 Let T ≥ 1 be an integer and {xt }Tt=0 be a (Ω)-program. Then for each integer t = 0, . . . , T − 1, π u (xt ) ≥ u(xt , xt+1 ) − u(x, ¯ x) ¯ + π u (xt+1 ). The next result follows from the definition of (u, Ω)-overtaking optimal programs.

5.1 Preliminaries

75

Proposition 5.2 Let {xt }∞ t=0 be a (u, Ω)-overtaking optimal program. Then π u (x0 ) = lim sup

T −1

T →∞ t=0

(u(xt , xt+1 ) − u(x, ¯ x)). ¯

Corollary 5.3 Let {xt }∞ t=0 be a (u, Ω)-overtaking optimal program. Then for any integer t ≥ 0, π u (xt ) = u(xt , xt+1 ) − u(x, ¯ x) ¯ + π u (xt+1 ).

(5.5)

sup(π u ) = sup{π u (z) : z ∈ K}.

(5.6)

Set

¯ = 0. Proposition 5.4 π u (x) Proof Set xt = x¯ for all integers t ≥ 0. By Theorem 3.5 and (5.1), the program u ¯ = 0. {xt }∞ t=0 is a (u, Ω)-overtaking optimal. In view of Proposition 5.2, π (x) Proposition 5.4 is proved. Proposition 5.5 The function π u is continuous at x. ¯ Proof Let  > 0. In view of (B2), there exists 1 ∈ (0, ) such that |u(x, y) − u(x, ¯ x)| ¯ ≤ (4L∗ )−1 for each (x, y) ∈ Ω satisfying ρ(x, x), ¯ ρ(y, x) ¯ ≤ 1 .

(5.7)

By (B3), there exists δ ∈ (0, 1 ) such that the following property holds: (i) for each z1 , z2 ∈ K satisfying ¯ ≤ δ, i = 1, 2 ρ(zi , x) ∗ there exists an (Ω)-program {xt }L t=0 such that

x0 = z1 , xL∗ = z2 and ¯ ≤ 1 , t = 0, . . . , L∗ . ρ(xt , x) Assume that ¯ ≤ δ, i = 1, 2. z1 , z2 ∈ K, ρ(zi , x)

(5.8)

76

5 Structure of Solutions in the Regions Close to the Endpoints

∗ Property (i) and (5.8) imply that there exists an (Ω)-program {xt }L t=0 such that

x0 = z1 , xL∗ = z2 ,

(5.9)

¯ ≤ 1 , t = 0, . . . , L∗ . ρ(xt , x)

(5.10)

and

By (5.7) and (5.10), for all t = 0, . . . , L∗ − 1, ¯ x)| ¯ ≤ (4L∗ )−1 . |u(xt , xt+1 ) − u(x, It follows from the relation above, (5.2), and (5.9) that π u (z1 ) = π u (x0 ) ≥

L ∗ −1

(u(xt , xt+1 ) − u(x, ¯ x)) ¯ + π u (xL∗ )

t=0

≥ −4−1 (/L∗ )L∗ + π u (z2 ) = π u (z2 ) − . Proposition 5.5 is proved. Proposition 5.6 Assume that x0 ∈ K and {xt }∞ t=0 ∈ P(u, x0 ). Then T −1

π (x0 ) = lim u

T →∞

(u(xt , xt+1 ) − u(x, ¯ x)). ¯

t=0

Proof It follows from (B1), Propositions 5.4, 5.5, and Corollary 5.3 that π u (x0 ) = lim (π u (x0 ) − π u (xT )) T →∞

= lim

T −1

T →∞

(u(xt , xt+1 ) − u(x, ¯ x)). ¯

t=0

Proposition 5.6 is proved. Proposition 5.7 The function π u : K → R 1 is upper semicontinuous. Proof Assume that {x (i) }∞ i=1 ⊂ K, x ∈ K and lim x (i) = x.

i→∞

(5.11)

5.1 Preliminaries

77

We show that   π u (x) ≥ lim sup π u x (i) . i→∞

We may assume without loss of generality that     lim sup π u x (i) = lim π u x (i) . i→∞

i→∞

(5.12)

By Theorem 3.4 and Proposition 5.6, for each integer i ≥ 1, there exists a (u, Ω)(i) overtaking optimal program {xt }∞ t=0 such that (i)

x0 = x (i) ,

(5.13)

T −1     (i) (i) u(xt , xt+1 ) − u(x, ¯ x) ¯ . π u x (i) = lim T →∞

(5.14)

t=0

Extracting a subsequence and re-indexing, if necessary, we may assume without loss of generality that for each integer t ≥ 0 there exists (i)

xt = lim xt .

(5.15)

i→∞

In view of (5.15), {xt }∞ t=0 is an (Ω)-program. Let  > 0. By Propositions 5.4 and 5.5, there exists δ ∈ (0, ) such that for each x ∈ K satisfying ρ(x, x) ¯ ≤ δ, |π u (x)| ≤ /2.

(5.16)

Since {xt(i) }∞ t=0 , i = 1, 2, . . . are (u, Ω)-overtaking optimal programs it follows from Theorem 3.2 that there exists a natural number L1 such that for all sufficiently large natural numbers i,   (i) ρ xt , x¯ ≤ δ for all integers t ≥ L1 .

(5.17)

Let T ≥ L1 be an integer. By Corollary 5.3, (5.16), and (5.17), for all integers i ≥ 1, T −1

(i)

(i)

(u(xt , xt+1 ) − u(x, ¯ x)) ¯

t=0 (i)

(i)

(i)

= π u (x0 ) − π u (xT ) ≥ π u (x0 ) − /2.

78

5 Structure of Solutions in the Regions Close to the Endpoints

In view of the relation above and the upper semicontinuity of u, for all integers T ≥ L1 , T −1

(u(xt , xt+1 ) − u(x, ¯ x)) ¯

t=0

≥ lim sup i→∞

T −1 

(i) u(xt(i) , xt+1 ) − u(x, ¯ x) ¯



t=0

  ≥ lim sup π u x0(i) − /2. i→∞

By (5.2), (5.11), (5.13), (5.15), and the relation above, π u (x) ≥ lim sup

T −1

T →∞ t=0

  (u(xt , xt+1 ) − u(x, ¯ x)) ¯ ≥ lim sup π u x (i) − /2. i→∞

Since  is any positive number we conclude that   π u (x) ≥ lim sup π u x (i) . i→∞

Proposition 5.7 is proved. Proposition 5.8 Let {xt }∞ t=0 be an (Ω)-program such that for all integers t ≥ 0, ¯ x) ¯ = π u (xt ) − π u (xt+1 ). u(xt , xt+1 ) − u(x,

(5.18)

Then {xt }∞ t=0 is a (u, Ω)-overtaking optimal program. Proof Proposition 5.1 and (5.18) imply that {xt }∞ t=0 is a (u, Ω)-locally optimal program. By Theorem 3.5, {xt }∞ is a (u, Ω)-overtaking optimal program. Propot=0 sition 5.8 is proved.

5.2 Lagrange Problems In order to study the structure of solutions of the problems in the regions close to the end points we introduce the following notation and definitions. Set Ω¯ = {(x, y) ∈ K × K : (y, x) ∈ Ω}. Clearly, Ω¯ is a nonempty closed subset of K × K.

(5.19)

5.2 Lagrange Problems

79

For each function w : Ω → R 1 define w¯ : Ω¯ → R 1 by ¯ w(x, ¯ y) = w(y, x), (x, y) ∈ Ω.

(5.20)

2 be an (Ω)-program. Define Let 0 ≤ T1 < T2 be integers and let {xt }Tt=T 1

2 {x¯t }Tt=T ⊂ K by 1

x¯t = xT2 −t+T1 , t = T1 , . . . , T2 .

(5.21)

2 ¯ Clearly, {x¯t }Tt=T is an (Ω)-program. 1 Assume that ut : Ω → R 1 , t = T1 , . . . , T2 − 1. It is easy to see that

T 2 −1

u¯ T2 −t+T1 −1 (x¯t , x¯t+1 )

t=T1

=

T 2 −1

uT2 −t+T1 −1 (xT2 −t+T1 −1 , xT2 −t+T1 )

t=T1

=

T 2 −1

ut (xt , xt+1 ).

(5.22)

t=T1

The next result easily follows from (5.22). Proposition 5.9 Let T ≥ 1 be an integer, M ≥ 0, ut : Ω → R 1 , t = 0, . . . , T (i) and {xt }Tt=0 , i = 1, 2 are (Ω)-programs. Then T −1

−1   T   (1) (1) (2) (2) ut xt , xt+1 ≥ ut xt , xt+1 − M

t=0

t=0

if and only if T −1

−1   T   (1) (1) (2) (2) u¯ T −t−1 x¯t , x¯t+1 ≥ u¯ T −t−1 xt , xt+1 − M.

t=0

t=0

Let T1 , T2 be integers such that T1 < T2 and ut : Ω → R 1 , t = T1 , . . . T2 − 1 be bounded functions. For each z0 , z1 ∈ K set ⎧ ⎫ 2 −1 ⎨T ⎬   T2 −1 T2 U {ut }t=T1 , Ω = sup ut (xt , xt+1 ) : {xt }t=T1 is an (Ω)-program , ⎩ ⎭ t=T1

(5.23)

80

5 Structure of Solutions in the Regions Close to the Endpoints





T2 −1 U {ut }t=T , Ω, z0 = sup 1

⎧ 2 −1 ⎨T ⎩

2 ut (xt , xt+1 ) : {xt }Tt=T is an (Ω)-program 1

t=T1

such that xT1 = z0





 {ut }T2 −1 , Ω, z1 = sup U t=T1

⎧ 2 −1 ⎨T ⎩

⎫ ⎬ ⎭

(5.24)

,

2 ut (xt , xt+1 ) : {xt }Tt=T is an (Ω)-program 1

t=T1

such that xT2 = z1

⎫ ⎬ ⎭

(5.25)

,

⎧ 2 −1 ⎨T   T2 −1 2 = sup , Ω, z , z ut (xt , xt+1 ) : {xt }Tt=T is an (Ω)-program U {ut }t=T 0 1 1 1 ⎩ t=T1

such that xT1 = z0 , xT2 = z1

⎫ ⎬ ⎭

.

(5.26)

Let T1 , T2 be integers such that T1 < T2 and wt : Ω¯ → R 1 , t = T1 , . . . T2 − 1 be bounded functions. For each z0 , z1 ∈ K set ⎧ 2 −1 ⎨T   T2 −1 ¯ 2 ¯ U {wt }t=T1 , Ω, z0 = sup wt (xt , xt+1 ) : {xt }Tt=T is an (Ω)-program 1 ⎩ t=T1

such that xT1 = z0

⎫ ⎬ ⎭

(5.27)

,

⎧ 2 −1 ⎨T   T2 −1 ¯ 2 ¯ wt (xt , xt+1 ) : {xt }Tt=T is an (Ω)-program U {wt }t=T1 , Ω, z0 , z1 = sup 1 ⎩ t=T1

such that xT1 = z0 , xT2 = z1

⎫ ⎬ ⎭

,

(5.28)

5.2 Lagrange Problems

81





 {wt }T2 −1 , Ω, ¯ z1 = sup U t=T1 $

⎧ 2 −1 ⎨T ⎩

2 ¯ wt (xt , xt+1 ) : {xt }Tt=T is an (Ω)-program 1

t=T1

such that xT2 = z1 ,

(5.29)

⎧ ⎫ 2 −1 ⎨T ⎬   T2 −1 ¯ T2 ¯ U {wt }t=T , Ω = sup w (x , x ) : {x } is an ( Ω)-program . t t t+1 t t=T1 1 ⎩ ⎭ t=T1

(5.30) Proposition 5.9 implies the following result. Proposition 5.10 Let T ≥ 1 be an integer, M ≥ 0, ut : Ω → R 1 , t = 0, . . . T − 1 ¯ be bounded functions and {xt }Tt=0 be an (Ω)-program. Then {x¯t }Tt=0 is an (Ω)program and the following assertions hold:   T −1 T −1 if t=0 ut (xt , xt+1 ) ≥ U {ut }t=0 , Ω − M, then T −1

  T −1 ¯ u¯ T −t−1 (x¯t , x¯t+1 ) ≥ U {u¯ T −t−1 }t=0 , Ω − M;

t=0

if

T −1 t=0

  T −1 ut (xt , xt+1 ) ≥ U {ut }t=0 , Ω, x0 , xT − M, then

T −1

  −1 ¯ u¯ T −t−1 (x¯t , x¯t+1 ) ≥ U {u¯ T −t−1 }Tt=0 , Ω, x¯0 , x¯T − M;

t=0

if

T −1 t=0

  T −1 ut (xt , xt+1 ) ≥ U {ut }t=0 , Ω, x0 − M, then T −1

   {u¯ T −t−1 }T −1 , Ω, ¯ x¯T − M; u¯ T −t−1 (x¯t , x¯t+1 ) ≥ U t=0

t=0

if

T −1 t=0

   {ut }T −1 , Ω, xT − M, then ut (xt , xt+1 ) ≥ U t=0 T −1

  T −1 ¯ u¯ T −t−1 (x¯t , x¯t+1 ) ≥ U {u¯ T −t−1 }t=0 , Ω, x¯0 − M.

t=0

¯ satisfies assumptions (A), (B2), and (B3), and that Note that the pair (u, ¯ Ω) μ(u) ¯ = μ(u) = μ.

82

5 Structure of Solutions in the Regions Close to the Endpoints

It follows from Theorems 3.2 and 4.1 and Proposition 5.10 that (B1) also holds for ¯ Therefore all the results presented above for the pair (u, Ω) are also the pair (u, ¯ Ω). ¯ true for the pair (u, ¯ Ω). We prove the following results which describe the structure of approximate solutions of our optimal control problems in the regions close to the endpoints. Theorem 5.11 Let τ0 ≥ 1 be an integer and  > 0. Then there exist δ > 0 and an integer T0 ≥ τ0 such that for each integer T ≥ T0 , each finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . T − 1 satisfying ut − u ≤ δ, t = 0 . . . , T − 1 and each (Ω)-program {xt }Tt=0 which satisfies T −1

  T −1 ut (xt , xt+1 ) ≥ U {ut }t=0 , Ω, x0 − δ

t=0

¯ there exists a (u, ¯ Ω)-overtaking optimal program {xt∗ }∞ t=0 π u¯ (x0∗ ) = sup(π u¯ ), ρ(xT −t , xt∗ ) ≤ , t = 0, . . . , τ0 . Theorem 5.12 Let τ0 ≥ 1 be an integer and  > 0. Then there exist δ > 0 and an integer T0 ≥ τ0 such that for each integer T ≥ T0 , each finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . T − 1 satisfying ut − u ≤ δ, t = 0 . . . , T − 1 and each (Ω)-program {xt }Tt=0 which satisfies T −1

  T −1 ut (xt , xt+1 ) ≥ U {ut }t=0 ,Ω − δ

t=0 (1) ¯ ¯ Ω)-overtaking there exist a (u, Ω)-overtaking optimal program {yt }∞ t=0 and a (u, (2) ∞ optimal program {yt }t=0 such that

    (1) (2) π u y0 = sup(π u ), π u¯ y0 = sup(π u¯ ) and that for all integers t = 0, . . . , τ0 ,     (2) (1) ≤ , ρ xt , yt ≤ . ρ xT −t , yt

5.3 An Auxiliary Result for Theorem 5.13

83

Theorem 5.13 Let τ0 ≥ 1 be an integer and  > 0. Then there exist δ > 0 and an integer T0 ≥ τ0 such that for each integer T ≥ T0 , each finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . T − 1 satisfying ut − u ≤ δ, t = 0 . . . , T − 1 and each (Ω)-program {xt }Tt=0 which satisfies T −1

   {ut }T −1 , Ω, xT − δ ut (xt , xt+1 ) ≥ U t=0

t=0

there exists a (u, Ω)-overtaking optimal program {xt∗ }∞ t=0 such that π u (x0∗ ) = sup(π u ), ρ(xt , xt∗ ) ≤ , t = 0, . . . , τ0 . Theorem 5.13 is proved in Sect. 5.4 while Sect. 5.3 contains an auxiliary result. ¯ and Proposition 5.10. Theorem 5.11 follows from Theorem 5.13 applied with (u, ¯ Ω) Theorem 5.12 easily follows from Theorems 5.11 and 5.13.

5.3 An Auxiliary Result for Theorem 5.13 Lemma 5.14 Let T0 ≥ 1 be an integer and  ∈ (0, 1). Then there exists δ ∈ (0, ) 0 such that for each (Ω)-program {xt }Tt=0 which satisfies π u (x0 ) ≥ sup(π u ) − δ, T 0 −1

(u(xt , xt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (xT ) ≥ −δ,

t=0

there exists a (u, Ω)-overtaking optimal program {zt }∞ t=0 such that π u (z0 ) = sup(π u ), ρ(zt , xt ) ≤ , t = 0, . . . , T0 .

(5.31)

Proof Assume that the lemma does not hold. Then there exist a sequence {δk }∞ k=1 ⊂ (k) T0 (0, 1) and a sequence of (Ω)-programs {xt }t=0 , k = 1, 2, . . . such that lim δk = 0

k→∞

(5.32)

84

5 Structure of Solutions in the Regions Close to the Endpoints

and that for each integer k ≥ 1 and each (u, Ω)-overtaking optimal program {zt }∞ t=0 satisfying (5.31),   (k) ≥ sup(π u ) − δk , π u x0

(5.33)

T 0 −1 

       (k) (k) (k) (k) u xt , xt+1 − u(x, ¯ x) ¯ − π u x0 + π u xT0 ≥ −δk

(5.34)

t=0

we have  

 (k) : t = 0, . . . , T0 > . max ρ zt , xt

(5.35)

Extracting a subsequence and re-indexing, if necessary, we may assume without loss of generality that for each integer t ∈ [0, T0 ] there exists (k)

xt = lim xt .

(5.36)

k→∞

Proposition 5.7, (5.32), and (5.33) imply that   (k) ≥ sup(π u ). sup(π u ) ≥ π u (x0 ) ≥ lim sup π u x0

(5.37)

k→∞

By upper semicontinuity of u, Proposition 5.7, (5.32)–(5.34), (5.36), and (5.37), T 0 −1

(u(xt , xt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (xT0 )

t=0

⎛

⎞

≥ lim sup ⎝

(u(xt , xt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (xT0 )⎠

T 0 −1

k→∞

(k)

(k)

(k)

(k)

t=0

≥ lim sup(−δk ) = 0. k→∞

Thus T 0 −1

(u(xt , xt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (xT0 ) ≥ 0.

t=0

Together with Proposition 5.1 this implies that for all integers t = 0, . . . , T0 − 1, ¯ x) ¯ = π u (xt ) − π u (xt+1 ). u(xt , xt+1 ) − u(x,

(5.38)

5.4 Proof of Theorem 5.13

85

Theorem 3.4 implies that there is a (u, Ω)-overtaking optimal program {x˜t }∞ t=0 satisfying x˜0 = xT0 .

(5.39)

xt = x˜t−T0 .

(5.40)

For all integers t > T0 set

∞ It is clear that {xt }∞ t=0 is an (Ω)-program. Since {x˜ t }t=0 is a (u, Ω)-overtaking optimal program, Corollary 5.3 implies that (5.38) holds for all integers t ≥ 0. It follows from (5.38) and Proposition 5.8 that {xt }∞ t=0 is a (u, Ω)-overtaking optimal program. In view of (5.37),

π u (x0 ) = sup(π u ). (k)

By (5.36), for all sufficiently large natural numbers k, ρ(xt , xt ) ≤ /4, t = 0, . . . , T0 . This contradicts (5.35). The contradiction we have reached proves Lemma 5.14.

5.4 Proof of Theorem 5.13 We may assume that  < 1. By Lemma 5.14, there exists δ1 ∈ (0, /4) such that the following property holds: 0 (i) for each (Ω)-program {yt }τt=0 which satisfies

π u (y0 ) ≥ sup(π u ) − δ1 , τ 0 −1

(u(yt , yt+1 ) − u(x, ¯ x)) ¯ − π u (y0 ) + π u (yτ0 ) ≥ −δ1

t=0

there exists a (u, Ω)-overtaking program {zt }∞ t=0 such that π u (z0 ) = sup(π u ), ρ(zt , yt ) ≤ , t = 0, . . . , τ0 . By Propositions 5.4 and 5.5 and (B2), there exists δ2 ∈ (0, δ1 ) such that the following properties hold: (ii) for each z ∈ K satisfying ρ(z, x) ¯ ≤ 2δ2 , we have ¯ ≤ δ1 /8; |π u (z)| = |π u (z) − π u (x)|

86

5 Structure of Solutions in the Regions Close to the Endpoints

(iii) for each (x, y) ∈ Ω satisfying ¯ ≤ 2δ2 ρ(x, x) ¯ ≤ 2δ2 , ρ(y, x) we have |u(x, y) − u(x, ¯ x)| ¯ ≤ (δ1 /8)(L∗ + 1)−1 . By (B3), there exists δ3 ∈ (0, δ2 ) such that the following property holds: (iv) for each x, y ∈ K satisfying ρ(x, x) ¯ ≤ δ3 , ρ(y, x) ¯ ≤ δ3 ∗ there exists an (Ω)-program {ξt }L t=0 such that

ξ0 = x, ξL∗ = y and ρ(ξt , x) ¯ ≤ δ2 , t = 0, . . . , L∗ . By Theorem 3.12, there exist an integer L0 ≥ 1 and a number δ4 > 0 such that the following property holds: (v) for each integer T > 2L0 , each finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . T − 1 satisfying ut − u ≤ δ4 , t = 0 . . . , T − 1 and each (Ω)-program {zt }Tt=0 which satisfies T −1

   {ut }T −1 , Ω, zT − δ4 ut (zt , zt+1 ) ≥ U t=0

t=0

we have ¯ ≤ δ3 , t = L0 , . . . , T − L0 . ρ(zt , x) By Theorem 3.4, there exists a (u, Ω)-overtaking optimal program {zt }∞ t=0 such that π u (z0 ) = sup(π u ).

(5.41)

(B1) implies that there exists a natural number τ1 such that ¯ ≤ δ3 for all integers t ≥ τ1 . ρ(zt , x)

(5.42)

5.4 Proof of Theorem 5.13

87

Choose a positive number δ and an integer T0 such that δ < (16(L0 + L∗ + τ1 + τ0 + 6))−1 min{δ1 , δ2 , δ3 , δ4 }, T0 > 2L∗ + 2L0 + 2τ0 + 2τ1 + 4.

(5.43) (5.44)

Assume that an integer T ≥ T0 , a finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . T − 1 satisfies ut − u ≤ δ, t = 0 . . . , T − 1

(5.45)

and {xt }Tt=0 is an (Ω)-program which satisfies T −1

  T −1 ut (xt , xt+1 ) ≥ U {ut }t=0 , Ω, xT − δ.

(5.46)

t=0

By (5.43)–(5.46) and the property (v), ρ(xt , x) ¯ ≤ δ3 , t = L0 , . . . , T − L0 .

(5.47)

Property (iv), (5.42), (5.44), and (5.47) imply that there exist an (Ω)-program {x˜t }Tt=0 such that x˜t = zt , t = 0, . . . , L0 + τ0 + τ1 + 3,

(5.48)

ρ(x˜t , x) ¯ ≤ δ2 , t = L0 + τ0 + τ1 + 4, . . . , L0 + L∗ + τ0 + τ1 + 3,

(5.49)

x˜t = xt , t = L0 + L∗ + τ0 + τ1 + 3, . . . , T .

(5.50)

It follows from (5.45), (5.46), and (5.50) that δ≥

T −1

(ut (x˜t , x˜t+1 ) − ut (xt , xt+1 ))

t=0

=

L0 +L∗ +τ0 +τ1 +2

(ut (x˜t , x˜t+1 ) − ut (xt , xt+1 ))

t=0

≥

L0 +L∗ +τ0 +τ1 +2

(u(x˜t , x˜t+1 ) − u(xt , xt+1 ))

t=0

− 2δ(L0 + L∗ + τ0 + τ1 + 3).

(5.51)

88

5 Structure of Solutions in the Regions Close to the Endpoints

Properties (ii) and (iii), (5.42), (5.48), and (5.49) imply that ¯ x)| ¯ ≤ (δ1 /8)(L∗ + 1)−1 , |u(x˜t , x˜t+1 ) − u(x,

(5.52)

t = L0 + τ0 + τ1 + 3, . . . , L0 + L∗ + τ0 + τ1 + 2. Corollary 5.3, property (ii), (5.42), (5.43), (5.44), (5.47), (5.48), (5.51), and (5.52) imply that δ≥

L0 +L∗ +τ0 +τ1 +2

u(zt , zt+1 ) + L∗ u(x, ¯ x) ¯ − L∗ (δ1 /8)(L∗ + 1)−1

t=0

−

L0 +L∗ +τ0 +τ1 +2

u(xt , xt+1 ) − 2δ(L0 + L∗ + τ0 + τ1 + 3)

t=0

= π u (z0 ) − π u (zL0 +τ0 +τ1 +3 ) + (L0 + L∗ + τ0 + τ1 + 3)u(x, ¯ x) ¯ − δ1 /4 ⎛

⎞

−⎝

(u(xt , xt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (xL0 +L∗ +τ1 +3 )⎠

L0 +L ∗ +τ1 +2  t=0

− (L0 + L∗ + τ1 + 3)u(x, ¯ x) ¯ − π u (x0 ) + π u (xL0 +L∗ +τ1 +3 ) ≥ sup(π u ) − δ1 /8 − δ1 /4 − π u (x0 ) − δ1 /8 ⎛

L0 +L ∗ +τ1 +2 

−⎝

⎞ (u(xt , xt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (xL0 +L∗ +τ1 +3 )⎠ .

t=0

(5.53) It follows from Proposition 5.1, (5.43), and (5.53) that π u (x0 ) +

τ 0 −1

(u(xt , xt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (xτ0 )

t=0

≥ sup(π u ) − δ1 , π u (x0 ) ≥ sup(π u ) − δ1 , τ 0 −1 t=0

(u(xt , xt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (xτ0 ) ≥ −δ1 .

(5.54)

(5.55)

5.5 The First Class of Bolza Problems

89

By (5.54), (5.55), and the property (i), there exists a (u, Ω)-overtaking optimal program {yt }∞ t=0 such that π u (y0 ) = sup(π u ), ρ(yt , xt ) ≤ , t = 0, . . . , τ0 . Theorem 5.13 is proved.

5.5 The First Class of Bolza Problems For each nonempty set Y and each function h : Y → R 1 put sup(h) = sup{h(y) : y ∈ Y }. Denote by M(K) the set of all bounded functions h : K → R 1 . For each h ∈ M(K) set h = sup{|h(x)| : x ∈ K}. Clearly, (M(K),  · ) is a Banach space. For each h1 , h2 ∈ M(K) set dK (h1 , h2 ) = h1 − h2 . For each x ∈ K, each pair of integers T1 , T2 satisfying 0 ≤ T1 < T2 , each finite sequence of bounded functions ut : Ω → R 1 , t = T1 , . . . , T2 − 1, and each h ∈ M(K) we consider the problem T 2 −1

T −1 ut (xt , xt+1 ) + h(xT2 ) → max, {(xt , xt+1 )}t=0 ⊂ Ω, xT1 = x

(PB,1 )

t=T1

and set  U

T2 −1 h, {ut }t=T , Ω, x 1



= sup

⎧ 2 −1 ⎨T ⎩

ut (xt , xt+1 ) + h(xT2 ) :

t=T1

2 is an (Ω) − program and xT1 = x {xt }Tt=T 1

⎫ ⎬ ⎭

.

For each x ∈ X, each pair of integers T1 , T2 satisfying 0 ≤ T1 < T2 , each bounded function u : Ω → R 1 , and each h ∈ M(K) set T2 −1 , Ω, x) where ut = u, t = T1 , . . . , T2 − 1. U (h, u, T1 , T2 , Ω, x) = U (h, {ut }t=T 1

90

5 Structure of Solutions in the Regions Close to the Endpoints

In Sect. 5.7 we prove the following result which describes the structure of approximate solutions of the problems of the type (PB,1 ) in the regions close to the right endpoints. Theorem 5.15 Let g ∈ M(K) be upper semicontinuous function, τ0 ≥ 1 be an integer, and  > 0. Then there exist δ > 0 and an integer T0 ≥ τ0 such that for each integer T ≥ T0 , each h ∈ M(K) satisfying h − g ≤ δ, each finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . , T −1 satisfying ut − u ≤ δ, t = 0 . . . , T − 1 and each (Ω)-program {xt }Tt=0 which satisfies τ +T 0 −1 

  τ +T0 −1 ut (xt , xt+1 ) ≥ U {ut }t=τ , Ω, xτ , xτ +T0 − δ

t=τ

for each τ ∈ {0, . . . , T − T0 }, h(xT ) +

T −1 t=T −T0

  T −1 ut (xt , xt+1 ) ≥ U h, {ut }t=T −T0 , Ω, xT −T0 − δ

¯ there exists a (u, ¯ Ω)-overtaking optimal program {xt∗ }∞ t=0 such that (π u¯ + g)(x0∗ ) = sup(π u¯ + g), ρ(xT −t , xt∗ ) ≤ , t = 0, . . . , τ0 . Let g ∈ M(K) be upper semicontinuous. Then π u¯ + g : K → R 1 is an upper semicontinuous, bounded function. Therefore there exists x ∈ K such that (π u¯ + g)(x) = sup(π u¯ + g).

5.6 An Auxiliary Result for Theorem 5.15 Lemma 5.16 Let g ∈ M(K) be an upper semicontinuous function, T0 ≥ 1 be an integer, and  ∈ (0, 1). Then there exists δ ∈ (0, ) such that for each (Ω)-program 0 {xt }Tt=0 which satisfies

5.6 An Auxiliary Result for Theorem 5.15

91

(π u + g)(x0 ) ≥ sup(π u + g) − δ, T 0 −1

(u(xt , xt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (xT0 ) ≥ −δ

t=0

there exists a (u, Ω)-overtaking optimal program {zt }∞ t=0 such that (π u + g)(z0 ) = sup(π u + g),

(5.56)

ρ(zt , xt ) ≤ , t = 0, . . . , T0 . Proof Assume that the lemma does not hold. Then there exist a sequence of real (k) T0 numbers {δk }∞ k=1 ⊂ (0, 1] and a sequence of (Ω)-programs {xt }t=0 , k = 1, 2, . . . such that lim δk = 0

(5.57)

k→∞

and that for each natural number k and each (u, Ω)-overtaking optimal program {zt }∞ t=0 satisfying (5.56),   (k) (π u + g) x0 ≥ sup(π u + g) − δk , T 0 −1 

       (k) u xt(k) , xt+1 − u(x, ¯ x) ¯ − π u x0(k) + π u xT(k) ≥ −δk , 0

(5.58)

(5.59)

t=0

we have 

max ρ(zt , xt(k) ) : t = 0, . . . , T0 > .

(5.60)

Extracting a subsequence and re-indexing, if necessary, we may assume without loss of generality that for each integer t ∈ [0, T0 ] there exists (k)

xt = lim xt .

(5.61)

k→∞

By (5.61) and upper semicontinuity of g and π u , (k)

π u (x0 ) ≥ lim sup π u (x0 ), k→∞

(k)

g(x0 ) ≥ lim sup g(x0 ). k→∞

(5.62)

92

5 Structure of Solutions in the Regions Close to the Endpoints

It follows from (5.57), (5.58), and (5.62) that sup(π u + g) ≥ (π u + g)(x0 ) ≥ lim sup(π u + g)(x0(k) ) ≥ sup(π u + g), k→∞

(k)

sup(π u + g) = (π u + g)(x0 ) = lim (π u + g)(x0 ).

(5.63)

k→∞

Relations (5.62) and (5.63) imply that π u (x0 ) = lim π u (x0(k) ), k→∞

(k)

g(x0 ) = lim g(x0 ).

(5.64)

k→∞

It follows from upper semicontinuity of u and π u , (5.57), (5.59), (5.61), and (5.64) that T 0 −1

(u(xt , xt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (xT0 )

t=0

⎛

⎞

≥ lim sup ⎝

(k) (k) (k) (k) (u(xt , xt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (xT0 )⎠

T 0 −1

k→∞

t=0

≥ lim sup(−δk ) = 0. k→∞

Combined with Proposition 5.1 this implies that for all integers t = 0, . . . , T0 − 1, ¯ x) ¯ = π u (xt ) − π u (xt+1 ). u(xt , xt+1 ) − u(x,

(5.65)

It follows from Theorem 3.4 that there exists a (u, Ω)-overtaking optimal program {x˜t }∞ t=0 satisfying x˜0 = xT0 .

(5.66)

xt = x˜t−T0 .

(5.67)

For all integers t > T0 set

Evidently, {xt }∞ t=0 is an (Ω)-program. It follows from Corollary 5.3 that (5.65) holds for all integers t ≥ 0. Proposition 5.8 and (5.65) imply that {xt }∞ t=0 is a (u, Ω)overtaking optimal program satisfying (5.63). By (5.61), for all sufficiently large (k) natural numbers k, ρ(xt , xt ) ≤ /4, t = 0, . . . , T0 . This contradicts (5.60). The contradiction we have reached proves Lemma 5.16.

5.7 Proof of Theorem 5.15

93

5.7 Proof of Theorem 5.15 ¯ there exists a real number By Lemma 5.16 applied to the pair (u, ¯ Ω) δ1 ∈ (0, /2) such that the following property holds: 0 ¯ which satisfies (i) for each (Ω)-program {yt }τt=0

(π u¯ + g)(y0 ) ≥ sup(π u¯ + g) − δ1 , τ 0 −1

(u(y ¯ t , yt+1 ) − u( ¯ x, ¯ x)) ¯ − π u¯ (y0 ) + π u¯ (yτ0 ) ≥ −δ1

t=0

¯ there exists an (u, ¯ Ω)-overtaking optimal program {zt }∞ t=0 such that (π u¯ + g)(z0 ) = sup(π u¯ + g), ρ(zt , yt ) ≤ , t = 0, . . . , τ0 . ¯ and assumption (B2) imply Propositions 5.4 and 5.5 applied to the pair (u, ¯ Ω) that there exists a real number δ2 ∈ (0, δ1 ) such that the following properties hold: (ii) for each point z ∈ K satisfying ρ(z, x) ¯ ≤ 2δ2 , ¯ ≤ δ1 /16; |π u¯ (z)| = |π u¯ (z) − π u¯ (x)| (iii) for each (x, y) ∈ Ω satisfying ¯ ≤ 2δ2 , ρ(x, x) ¯ ≤ 2δ2 , ρ(y, x) |u(x, y) − u(x, ¯ x)| ¯ ≤ (δ1 /8)(L∗ + 1)−1 . By (B3), there exists δ3 ∈ (0, δ2 ) such that the following property holds: (iv) for each x, y ∈ K satisfying ¯ ≤ δ3 ρ(x, x) ¯ ≤ δ3 , ρ(y, x) ∗ there exists an (Ω)-program {ξt }L t=0 such that

ξ0 = x, ξL∗ = y and ρ(ξt , x) ¯ ≤ δ2 , t = 0, . . . , L∗ .

94

5 Structure of Solutions in the Regions Close to the Endpoints

By Theorem 3.12, there exist an integer L1 ≥ 1 and a number δ4 ∈ (0, δ3 ) such that the following property holds: (v) for each integer T > 2L1 , each finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . T − 1 satisfying ut − u ≤ δ4 , t = 0 . . . , T − 1 and each (Ω)-program {zt }Tt=0 which satisfies τ +L 1 −1 

  τ +L1 −1 ut (zt , zt+1 ) ≥ U {ut }t=τ , Ω, zτ , zτ +L1 − δ4

t=τ

for each integer τ ∈ [0, T − L1 ], we have ¯ ≤ δ3 , t = L1 , . . . , T − L1 . ρ(zt , x) ¯ By Theorem 3.4, there exists a (u, ¯ Ω)-overtaking optimal program {zt }∞ t=0 such that (π u¯ + g)(z0 ) = sup(π u¯ + g).

(5.68)

(B1) implies that there exists a natural number τ1 such that ¯ ≤ δ3 for all integers t ≥ τ1 . ρ(zt , x)

(5.69)

Choose a positive number δ and an integer T0 such that δ < (16(L1 + L∗ + τ1 + τ0 + 8))−1 min{δ1 , δ2 , δ3 , δ4 }, T0 > 2L∗ + 2L1 + 2τ0 + 2τ1 + 8.

(5.70) (5.71)

Assume that an integer T ≥ T0 , h ∈ M(K) satisfies h − g ≤ δ,

(5.72)

a finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . T − 1 satisfies ut − u ≤ δ, t = 0 . . . , T − 1

(5.73)

and {xt }Tt=0 is an (Ω)-program which satisfies τ +T 0 −1  t=τ

  τ +T0 −1 ut (xt , xt+1 ) ≥ U {ut }t=τ , Ω, xτ , xτ +T0 − δ

(5.74)

5.7 Proof of Theorem 5.15

95

for each integer τ ∈ [0, T − T0 ], h(xT ) +

T −1 t=T −T0

  T −1 ut (xt , xt+1 ) ≥ U h, {ut }t=T −T0 , Ω, xT −T0 − δ.

(5.75)

By (5.70), (5.71), (5.73), (5.74), and the property (v), ρ(xt , x) ¯ ≤ δ3 , t = L1 , . . . , T − L1 .

(5.76)

Property (iv), (5.69), (5.71), and (5.76) imply that there exists an (Ω)-program {x˜t }Tt=0 such that x˜t = xt , t = 0, . . . , T − L1 − L∗ − τ0 − τ1 − 4,

(5.77)

x˜t = zT −t , t = T − L1 − τ0 − τ1 − 4, . . . , T ,

(5.78)

ρ(x˜t , x) ¯ ≤ δ2 , t = T − L1 − L∗ − τ0 − τ1 − 4, . . . , T − L1 − τ0 − τ1 − 4. (5.79) By property (iii), (5.71)–(5.73), (5.75), and (5.77), T −1   T −1 δ ≥ U h, {ut }t=T − h(x , Ω, x ) − ut (xt , xt+1 ) T −T0 T −T0 t=T −T0

≥ h(x˜T ) +

T −1

ut (x˜t , x˜t+1 ) − h(xT ) −

t=T −T0

T −1

ut (xt , xt+1 )

t=T −T0 T −1

= h(x˜T ) − h(xT ) +

(ut (x˜t , x˜t+1 ) − ut (xt , xt+1 ))

t=T −L1 −L∗ −τ0 −τ1 −4 T −1

≥ g(x˜T ) − g(xT ) +

u(x˜t , x˜t+1 )

t=T −L1 −L∗ −τ0 −τ1 −4

−

T −1

u(xt , xt+1 ) − 2δ(L1 + L∗ + τ0 + τ1 + 5)

t=T −L1 −L∗ −τ0 −τ1 −4

≥ g(x˜T ) − g(xT ) +

T −1

u(x˜t , x˜t+1 ) + L∗ u(x, ¯ x) ¯ − L∗ (δ1 /8)(L∗ + 1)−1

t=T −L1 −τ0 −τ1 −4

−

T −1 t=T −L1 −L∗ −τ0 −τ1 −4

u(xt , xt+1 ) − 2δ(L1 + L∗ + τ0 + τ1 + 5).

96

5 Structure of Solutions in the Regions Close to the Endpoints

Combined with (5.22), (5.70), and (5.78) this implies that T −1

g(xT ) +

u(xt , xt+1 )

t=T −L1 −L∗ −τ0 −τ1 −4 T −1

≥ g(x˜T ) +

u(x˜t , x˜t+1 ) + L∗ u(x, ¯ x) ¯ − δ1 /8 − δ1 /8

t=T −L1 −τ0 −τ1 −4

= g(z0 ) +

L1 +τ 0 +τ1 +2

u(z ¯ t , zt+1 ) + u(x˜T −L1 −τ0 −τ1 −4 , x˜T −L1 −τ0 −τ1 −3 )

t=0

¯ x) ¯ − δ1 /4. + L∗ u(x,

(5.80)

Property (iii), (5.69), and (5.78) imply that ¯ x)| ¯ ≤ δ1 /8. |u(x˜T −L1 −τ0 −τ1 −4 , x˜T −L1 −τ0 −τ1 −3 ) − u(x,

(5.81)

In view of (5.80) and (5.81), T −1

g(xT ) +

u(xt , xt+1 )

t=T −L1 −L∗ −τ0 −τ1 −4

≥ g(z0 ) +

L1 +τ 0 +τ1 +2

u(z ¯ t , zt+1 ) + (L∗ + 1)u(x, ¯ x) ¯ − 3δ1 /8.

(5.82)

t=0

Set yt = xT −t , t = 0, . . . , T .

(5.83)

By (5.22), (5.82), and (5.83), g(y0 ) +

L1 +L∗ +τ0 +τ1 +3

u(y ¯ t , yt+1 )

t=0 T −1

= g(xT ) +

u(xt , xt+1 )

T −L1 −L∗ −τ0 −τ1 −4

≥ g(z0 ) +

L1 +τ 0 +τ1 +2 t=0

u(z ¯ t , zt+1 ) + (L∗ + 1)u(x, ¯ x) ¯ − 3δ1 /8.

(5.84)

5.7 Proof of Theorem 5.15

97

By (5.68), (5.84), Proposition 5.1, and Corollary 5.3, (π u¯ + g)(y0 ) − sup(π u¯ + g) +

τ 0 −1

(u(y ¯ t , yt+1 ) − u( ¯ x, ¯ x)) ¯ − π u¯ (y0 ) + π u¯ (yτ0 )

t=0

≥ (π u¯ + g)(y0 ) − (π u¯ + g)(z0 ) +

L1 +L∗ +τ0 +τ1 +3

(u(y ¯ t , yt+1 ) − u(x, ¯ x)) ¯ − π u¯ (y0 ) + π u¯ (yL1 +L∗ +τ0 +τ1 +4 )

t=0

≥ π u¯ (y0 ) − π u¯ (z0 ) +

L1 +τ 0 +τ1 +2

(u(z ¯ t , zt+1 ) − u( ¯ x, ¯ x)) ¯

t=0

− 3δ1 /8 − π u¯ (y0 ) + π u¯ (yL1 +L∗ +τ0 +τ1 +4 ) = −π u¯ (zL1 +τ0 +τ1 +3 ) − π u¯ (yL1 +L∗ +τ0 +τ1 +4 ) − 3δ1 /8.

(5.85)

Property (ii), (5.69), (5.71), (5.76), and (5.83) imply that |π u¯ (zL1 +τ0 +τ1 +3 )| ≤ δ1 /8,

(5.86)

|π u¯ (yL1 +L∗ +τ0 +τ1 +4 )| ≤ δ1 /8.

(5.87)

It follows from (5.85)–(5.87) that (π u¯ + g)(y0 ) − sup(π u¯ + g) +

τ 0 −1

(u(y ¯ t , yt+1 ) − u( ¯ x, ¯ x)) ¯

t=0

− π u¯ (y0 ) + π u¯ (yτ0 ) ≥ −δ1 . Together with Proposition 5.1 this implies that (π u¯ + g)(y0 ) − sup(π u¯ + g) ≥ −δ1 , τ 0 −1

(u(y ¯ t , yt+1 ) − u( ¯ x, ¯ x)) ¯

t=0

−π u¯ (y0 ) + π u¯ (yτ0 ) ≥ −δ1 .

98

5 Structure of Solutions in the Regions Close to the Endpoints

It follows from the relations above, (5.83) and the property (i) that there exists an ¯ (u, ¯ Ω)-overtaking optimal program {ξt }∞ t=0 such that (π u¯ + g)(ξ0 ) = sup(π u¯ + g), ρ(ξt , xT −t ) = ρ(ξt , yt ) ≤ , t = 0, . . . , τ0 . Theorem 5.15 is proved.

5.8 The Second Class of Bolza Problems Denote by M(K × K) the collection of all bounded functions h : K × K → R 1 . For every function h ∈ M(K × K) put h = sup{|h(x, y)| : x, y ∈ K}. It is not difficult to see that (M(K × K),  · ) is a Banach space. For every pair of nonnegative integers T1 < T2 , every finite sequence of bounded functions ut : Ω → R 1 , t = T1 , . . . , T2 − 1, and every function h ∈ M(K × K) we consider the problem T 2 −1

T2 −1 ut (xt , xt+1 ) + h(xT1 , xT2 ) → max, {(xt , xt+1 )}t=T ⊂Ω 1

t=T1

and define ⎧ 2 −1 ⎨T   T2 −1 ut (xt , xt+1 ) + h(xT1 , xT2 ) : U h, {ut }t=T1 , Ω = sup ⎩ t=T1

⎫ ⎬

2 is an (Ω) − program . {xt }Tt=T 1 ⎭

For every pair of nonnegative integers T1 < T2 , every bounded function v : Ω → R 1 , and every function h ∈ M(K × K) define T2 −1 U (h, v, Ω, T1 , T2 ) = U (h, {vt }t=T , Ω) where vt = v, t = T1 , . . . , T2 − 1. 1

Let g ∈ M(K × K) be an upper semicontinuous function. Then the function ψg (ξ, η) = π u (ξ ) + π u¯ (η) + g(ξ, η), (ξ, η) ∈ Ω is upper semicontinuous bounded function which has a point of minimum.

(5.88)

5.9 Auxiliary Results for Theorem 5.17

99

Our next result describes the structure of approximate solutions in the regions close to the endpoints. Theorem 5.17 Suppose that g ∈ M(K × K) is an upper semicontinuous function, τ0 is a natural number, and  ∈ (0, 1). Then there exist a positive number δ and a natural number T0 ≥ τ0 such that for every natural number T ≥ T0 , every function h ∈ M(K × K) satisfying h − g ≤ δ, every finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . , T − 1 which satisfies ut − u ≤ δ, t = 0 . . . , T − 1, and every (Ω)-program {xt }Tt=0 which satisfies h(x0 , xT ) +

T −1

  T −1 ut (xt , xt+1 ) ≥ U h, {ut }t=0 , Ω − δ,

t=0

¯ there exist a (u, Ω)-overtaking optimal program {xt∗ }∞ ¯ Ω)-overtaking t=0 and a (u, ∞ ∗ optimal program {x¯t }t=0 such that π u (x0∗ ) + π u¯ (x¯0∗ ) + g(x0∗ , x¯0∗ ) ≥ π u (ξ ) + π u¯ (η) + g(ξ, η) for all ξ, η ∈ K and that for all integers t = 0, . . . , τ0 , ρ(xt , xt∗ ) ≤  and ρ(xT −t , x¯t∗ ) ≤ .

5.9 Auxiliary Results for Theorem 5.17 Lemma 5.18 Suppose that g ∈ M(K × K) is an upper semicontinuous function, T0 is a natural number, and  ∈ (0, 1). Then there exists δ ∈ (0, ) such that for 0 0 ¯ every (Ω)-program {xt }Tt=0 and every (Ω)-program {yt }Tt=0 which satisfy ψg (x0 , y0 ) + δ ≥ sup(ψg ), T 0 −1

(u(xt , xt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (xT0 ) ≥ −δ,

t=0 T 0 −1 t=0

(u(y ¯ t , yt+1 ) − u(x, ¯ x)) ¯ − π u¯ (y0 ) + π u¯ (yT0 ) ≥ −δ,

100

5 Structure of Solutions in the Regions Close to the Endpoints

¯ there exist a (u, Ω)-overtaking optimal program {xt∗ }∞ ¯ Ω)-overtaking t=0 and an (u, such that optimal program {yt∗ }∞ t=0 ψg (x0∗ , y0∗ ) = sup(ψg ) and that for all t = 0, . . . , T0 , ρ(xt , xt∗ ) ≤ , ρ(yt , yt∗ ) ≤ . Proof Assume that the lemma does not hold. Then there exist a sequence {δk }∞ k=1 ⊂ (k) T0 ¯ (0, 1], a sequence of (Ω)-programs {xt }t=0 , k = 1, 2, . . . and a sequence of (Ω)(k) T0 programs {yt }t=0 , k = 1, 2, . . . such that lim δk = 0,

(5.89)

  ψg x0(k) , y0(k) + δk ≥ sup(ψg ),

(5.90)

k→∞

for every integer k ≥ 1,

T 0 −1 

     (k) (k) (k) (k) u(xt , xt+1 ) − u(x, ¯ x) ¯ − π u x0 + π u xT0 ≥ −δk ,

(5.91)

t=0 T 0 −1 

     (k) (k) (k) (k) u(y ¯ t , yt+1 ) − u(x, ¯ x) ¯ − π u¯ y0 + π u¯ yT0 ≥ −δk

(5.92)

t=0

and that the following property holds: ¯ (i) for each (u, Ω)-overtaking optimal program {zt }∞ ¯ Ω)-overtaking t=0 and each (u, ∞ optimal program {ξt }t=0 satisfying ψg (z0 , ξ0 ) = sup(ψg ), we have    

 (k) (k) + ρ ξt , yt : t = 0, . . . , T0 > . max ρ zt , xt Extracting a subsequence and re-indexing, if necessary, we may assume without loss of generality that for every integer t ∈ [0, T0 ] there exist xt = lim xt(k) , yt = lim yt(k) . k→∞

k→∞

(5.93)

5.9 Auxiliary Results for Theorem 5.17

101

It follows from (5.88)–(5.90) and (5.93) and upper semicontinuity of the functions g, π u , and π u¯ that π u (x0 ) + π u¯ (y0 ) + g(x0 , y0 ) ≥ π u (ξ ) + π u¯ (η) + g(ξ, η) for all ξ, η ∈ K, (5.94)   (k) π u (x0 ) ≥ lim sup π u x0 , (5.95) k→∞

  π u¯ (y0 ) ≥ lim sup π u¯ y0(k) ,

(5.96)

  (k) (k) g(x0 , y0 ) ≥ lim sup g x0 , y0 .

(5.97)

k→∞

k→∞

Relations (5.89), (5.90), and (5.94) imply that π u (x0 ) + π u¯ (y0 ) + g(x0 , y0 )        (k) (k) (k) (k) . = lim π u x0 + π u¯ y0 + g x0 , y0

(5.98)

k→∞

By (5.95)–(5.98),   (k) π u (x0 ) = lim π u x0 ,

(5.99)

  (k) π u¯ (y0 ) = lim π u¯ y0 ,

(5.100)

  (k) (k) g(x0 , y0 ) = lim g x0 , y0 .

(5.101)

k→∞

k→∞

k→∞

By upper semicontinuity of the functions u, π u , and π u¯ , (5.80), (5.91)–(5.93), (5.99), and (5.100), we have T 0 −1

(u(xt , xt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (xT0 )

t=0

⎛

≥ lim sup ⎝ k→∞

T 0 −1 

(k) u(xt(k) , xt+1



− u(x, ¯ x)) ¯ − πu

t=0

≥ lim sup(−δk ) = 0, k→∞ T 0 −1 t=0

(u(y ¯ t , yt+1 ) − u(x, ¯ x)) ¯ − π u¯ (y0 ) + π u¯ (yT0 )



x0(k)



+ πu



xT(k) 0



⎞ ⎠

102

5 Structure of Solutions in the Regions Close to the Endpoints

⎛ ≥ lim sup ⎝ k→∞

T 0 −1 



(k)

(k)

u yt , yt+1



⎞      (k) (k) − u(x, ¯ x) ¯ − π u¯ y0 + π u¯ yT0 ⎠

t=0

≥ lim sup(−δk ) = 0. k→∞

Together with Proposition 5.1 this implies that for all integers t = 0, . . . , T0 − 1, we have ¯ x) ¯ = π u (xt ) − π u (xt+1 ), u(xt , xt+1 ) − u(x,

(5.102)

u(y ¯ t , yt+1 ) − u(x, ¯ x) ¯ = π u¯ (yt ) − π u¯ (yt+1 ).

(5.103)

It follows from Theorem 3.4 that there exist a (u, Ω)-overtaking optimal program {x˜t }∞ t=0 satisfying x˜0 = xT0

(5.104)

¯ and a (u, ¯ Ω)-overtaking optimal program {y˜t }∞ t=0 satisfying y˜0 = yT0 .

(5.105)

xt = x˜t−T0 , yt = y˜t−T0 .

(5.106)

For all integers t > T0 put

∞ ¯ Clearly, {xt }∞ t=0 is an (Ω)-program and {yt }t=0 is an (Ω)-program. Corollary 5.3 implies that (5.102) and (5.103) hold for all integers t ≥ 0. It follows from (5.102), (5.103), and Proposition 5.8 that {xt }∞ t=0 is a (u, Ω)-overtaking optimal ∞ ¯ ¯ Ω)-overtaking optimal program satisfying (5.98). program and {yt }t=0 is a (u, In view of (5.93), for all sufficiently large natural numbers k, (k)

(k)

ρ(xt , xt ) ≤ /4, ρ(yt , yt ) ≤ /4, t = 0, . . . , T0 . Together with (5.98) this contradicts property (i). The contradiction we have reached proves Lemma 5.18.

5.10 Proof of Theorem 5.17 Lemma 5.18 implies that there exists a number δ1 ∈ (0, /2) such that the following property holds:

5.10 Proof of Theorem 5.17

103

0 0 ¯ (i) for every (Ω)-program {ξt }τt=0 and every (Ω)-program {ηt }τt=0 which satisfy

π u (ξ0 )+π u¯ (η0 )+g(ξ0 , η0 )+2δ1 ≥ π u (ξ )+π u¯ (η)+g(ξ, η) for all ξ, η ∈ K, τ 0 −1

(u(ξt , ξt+1 ) − u(x, ¯ x)) ¯ − π u (ξ0 ) + π u (ξτ0 ) ≥ −2δ1 ,

t=0 τ 0 −1

(u(η ¯ t , ηt+1 ) − u(x, ¯ x)) ¯ − π u¯ (η0 ) + π u¯ (ητ0 ) ≥ −2δ1

t=0

¯ there exists a (u, Ω)-overtaking optimal program {xt∗ }∞ ¯ Ω)t=0 and a (u, such that overtaking optimal program {yt∗ }∞ t=0 ψg (x0∗ , y0∗ ) = sup(ψg ) and that for all t = 0, . . . , τ0 , ρ(ξt , xt∗ ) ≤ , ρ(ηt , yt∗ ) ≤ . Propositions 5.4, 5.5, and assumption (B2) imply that there exists a number δ2 ∈ (0, δ1 ) such that the following properties hold: (ii) for every point z ∈ X satisfying ρ(z, x) ¯ ≤ 2δ2 , we have ¯ ≤ δ1 /16, |π u (z)| = |π u (z) − π u (x)| ¯ ≤ δ1 /16; |π u¯ (z)| = |π u¯ (z) − π u¯ (x)| (iii) for every (x, y) ∈ Ω satisfying ρ(x, x) ¯ ≤ 2δ2 , ρ(y, x) ¯ ≤ 2δ2 , we have |u(x, y) − u(x, ¯ x)| ¯ ≤ (δ1 /16)(L∗ + 1)−1 By (B3), there exists δ3 ∈ (0, δ2 ) such that the following property holds: (iv) for each z1 , z2 ∈ K satisfying ¯ ≤ δ3 , i = 1, 2 ρ(zi , x) ∗ there exists an (Ω)-program {ξt }L t=0 such that

ξ0 = z1 , ξL∗ = z2 and ¯ ≤ δ2 , t = 0, . . . , L∗ . ρ(ξt , x)

104

5 Structure of Solutions in the Regions Close to the Endpoints

Theorem 3.12 implies that there exist a natural number L1 and a positive number δ4 ∈ (0, δ3 ) such that the following property holds: (v) for every natural number T > 2L1 , every finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . T − 1 satisfying ut − u ≤ δ3 , t = 0 . . . , T − 1 and every (Ω)-program {xt }Tt=0 which satisfies T −1

  T −1 ut (xt , xt+1 ) ≥ U {ut }t=0 , Ω, x0 , xT − δ4

t=0

the inequality ρ(xt , x) ¯ ≤ δ3 holds for all t = L1 , . . . , T − L1 . By Theorem 3.4, there exists a (u, Ω)-overtaking optimal program {ξt }∞ t=0 and a ∞ ¯ (u, ¯ Ω)-overtaking optimal program {ηt }t=0 such that ψg (ξ0 , η0 ) = sup(ψg ).

(5.107)

¯ ≤ δ3 for all integers t ≥ L1 , ρ(ξt , x)

(5.108)

ρ(ηt , x) ¯ ≤ δ3 for all integers t ≥ L1 .

(5.109)

In view of property (v),

Choose a positive number δ and an integer T0 ≥ 1 such that δ < (64(2L1 + L∗ + τ0 + 8))−1 min{δ1 , δ2 , δ3 , δ4 }, T0 > 4L∗ + 8L1 + 4τ0 + 16.

(5.110) (5.111)

Assume that an integer T ≥ T0 , a function h ∈ M(K × K) satisfies h − g ≤ δ,

(5.112)

a finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . T − 1 satisfies ut − u ≤ δ, t = 0 . . . , T − 1

(5.113)

and that {xt }Tt=0 is an (Ω)-program which satisfies h(x0 , xT ) +

T −1 t=0

  T −1 ut (xt , xt+1 ) ≥ U h, {ut }t=0 , Ω − δ.

(5.114)

5.10 Proof of Theorem 5.17

105

In view of (5.110), (5.111), (5.113), (5.114), and property (v), ¯ ≤ δ3 , t = L1 , . . . , T − L1 . ρ(xt , x)

(5.115)

Property (iv), (5.108), (5.109), (5.111), and (5.115) imply that there exists an (Ω)program {x˜t }Tt=0 such that x˜t = ξt , t = 0, . . . , 2L1 + τ0 + 3,

(5.116)

ρ(x˜t , x) ¯ ≤ δ2 , t = 2L1 + τ0 + 3, . . . , 2L1 + L∗ + τ0 + 3,

(5.117)

x˜t = xt , t = 2L1 + τ0 + 3 + L∗ , . . . , T − 2L1 − τ0 − 3 − L∗ ,

(5.118)

¯ ≤ δ2 , t = T − 2L1 − τ0 − 3 − L∗ , . . . , T − 2L1 − τ0 − 3, ρ(x˜t , x)

(5.119)

x˜t = ηT −t , t = T − 2L1 − τ0 − 3, . . . , T .

(5.120)

It follows from (5.110), (5.112)–(5.114), (5.116)–(5.120), and property (iii) that T −1 δ ≥ U (h, {ut }t=0 , Ω) − h(x0 , xT ) −

T −1

ut (xt , xt+1 )

t=0

≥ h(x˜0 , x˜T ) +

T −1

ut (x˜t , x˜t+1 ) − h(x0 , xT ) −

t=0

T −1

ut (xt , xt+1 )

t=0

= h(ξ0 , η0 ) − h(x0 , xT ) +

2L1 +τ 0 +2+L∗ 

(ut (x˜t , x˜t+1 ) − ut (xt , xt+1 ))

t=0 T −1

+

(ut (x˜t , x˜t+1 ) − ut (xt , xt+1 ))

t=T −2L1 −τ0 −3−L∗

≥ g(ξ0 , η0 ) − g(x0 , xT ) +

2L1 +τ 0 +2+L∗ 

(u(x˜t , x˜t+1 ) − u(xt , xt+1 ))

t=0 T −1

+

(u(x˜t , x˜t+1 ) − u(xt , xt+1 )) − 2δ(1 + 2L1 + τ0 + 4 + L∗ )

t=T −2L1 −τ0 −3−L∗

≥ g(ξ0 , η0 ) − g(x0 , xT ) +

2L1 +τ0 +2 t=0

u(ξt , ξt+1 ) + L∗ (u(x, ¯ x) ¯ − (δ1 /16)(L∗ + 1)−1 )

106

5 Structure of Solutions in the Regions Close to the Endpoints

−

2L1 +τ 0 +2+L∗ 

T −1

ut (xt , xt+1 ) +

u(ηT −t , ηT −t−1 )

t=T −2L1 −τ0 −3

t=0

+ L∗ (u(x, ¯ x) ¯ − (δ1 /16)(L∗ + 1)−1 ) T −1

−

ut (xt , xt+1 ) − δ1 /16

t=T −2L1 −τ0 −3−L∗

≥ g(ξ0 , η0 ) − g(x0 , xT ) +

2L1 +τ0 +2

u(ξt , ξt+1 ) +

2L1 +τ0 +2

t=0

−

u(η ¯ t , ηt+1 ) + 2L∗ u(x, ¯ x) ¯

t=0

2L1 +τ 0 +2+L∗ 

T −1

u(xt , xt+1 ) −

u(xt , xt+1 ) − 3δ1 /16.

t=T −2L1 −τ0 −3−L∗

t=0

(5.121) Define yt = xT −t , t = 0, . . . , T .

(5.122)

Relations (5.22), (5.121), and (5.122) imply that g(x0 , y0 ) +

2L1 +τ 0 +L∗ +2 

u(xt , xt+1 ) +

2L1 +τ 0 +L∗ +2 

t=0

≥ g(ξ0 , η0 ) +

u(y ¯ t , yt+1 )

t=0

2L1 +τ0 +2

u(ξt , ξt+1 ) +

t=0

2L1 +τ0 +2

u(η ¯ t , ηt+1 )

t=0

¯ x) ¯ − 3δ1 /16 − δ1 /16. + 2L∗ u(x,

(5.123)

It follows from (5.108), (5.109), (5.111), (5.112), (5.115), (5.123), Proposition 5.1, Corollary 5.3, and properties (ii) and (iii) that g(x0 , y0 ) − g(ξ0 , η0 ) +

τ 0 −1

(u(xt , xt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (xτ0 )

t=0

+

τ 0 −1 t=0

(u(y ¯ t , yt+1 ) − u(x, ¯ x)) ¯ − π u¯ (y0 ) + π u¯ (yτ0 )

5.10 Proof of Theorem 5.17

107

≥ g(x0 , y0 ) − g(ξ0 , η0 ) +

2L1 +τ 0 +2+L∗ 

(u(xt , xt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (x2L1 +τ0 +L∗ +3 )

t=0

+

2L1 +τ 0 +L∗ +2 

(u(y ¯ t , yt+1 ) − u( ¯ x, ¯ x)) ¯ − π u¯ (y0 ) + π u¯ (y2L1 +τ0 +3+L∗ )

t=0

≥

2L1 +τ0 +2

(u(ξt , ξt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (x2L1 +τ0 +3+L∗ )

t=0

+

2L1 +τ0 +2

(u(η ¯ t , ηt+1 ) − u( ¯ x, ¯ x)) ¯ − π u¯ (y0 ) + π u¯ (y2L1 +τ0 +3+L∗ ) − δ1 /4

t=0

≥ π u (ξ0 ) − π u (ξ2L1 +τ0 +3 ) − π u (x0 ) + π u (x2L1 +τ0 +3+L∗ ) + π u¯ (η0 ) − π u¯ (η2L1 +τ0 +3 ) − π u¯ (y0 ) − π u¯ (y2L1 +τ0 +3+L∗ ) − δ1 /4 ≥ π u (ξ0 ) − π u (x0 ) + π u¯ (η0 ) − π u¯ (y0 ) − δ1 /4 − δ1 /4.

(5.124)

In view of (5.124), we have g(x0 , y0 ) + π u (x0 ) + π u¯ (y0 ) − (g(ξ0 , η0 ) + π u (ξ0 ) + π u¯ (η0 )) +

τ 0 −1

(u(xt , xt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (xτ0 )

t=0

+

τ 0 −1

(u(y ¯ t , yt+1 ) − u(x, ¯ x)) ¯ − π u¯ (y0 ) + π u¯ (yτ0 ) ≥ −δ1 .

t=0

By (5.107), (5.125), and Proposition 5.1, g(x0 , y0 ) + π u (x0 ) + π u¯ (y0 ) ≥ sup(ψg ) − δ1 , τ 0 −1

(u(xt , xt+1 ) − u(x, ¯ x)) ¯ − π u (x0 ) + π u (xτ0 ) ≥ −δ1 ,

t=0 τ 0 −1 t=0

(u(y ¯ t , yt+1 ) − u(x, ¯ x)) ¯ − π u¯ (y0 ) + π u¯ (yτ0 ) ≥ −δ1 .

(5.125)

108

5 Structure of Solutions in the Regions Close to the Endpoints

It follows from the relations above and property (i) that there exists a (u, Ω)¯ overtaking optimal program {xt∗ }∞ ¯ Ω)-overtaking optimal program t=0 and a (u, ∞ ∗ {yt }t=0 such that ψg (x0∗ , y0∗ ) = sup(ψg ) and for all t = 0, . . . , τ0 , ρ(xt , xt∗ ) ≤ , Theorem 5.17 is proved.

ρ(xT −t , yt∗ ) = ρ(yt , yt∗ ) ≤ .

Chapter 6

Applications to the Forest Management Problem

In this chapter we continue the discussion of the forest management problem and show that for this problem the results of Chaps. 3 and 5 hold.

6.1 Preliminaries We consider a discrete time model for the optimal management of a forest of total area S occupied by k species I = {1, . . . , k} with maturity ages of n1 , . . . , nk years respectively. This model was introduced in Sect. 2.1. For the reader’s convenience we recall here the notation and definitions. j For each period t = 0, 1, . . . we denote xi (t) ≥ 0 the area covered by trees of species i that are j years old with j = 1, . . . , ni and x¯i (t) ≥ 0 the area occupied by over-mature trees (older than ni ). Assuming that only mature trees can be harvested we must have ui (t) ≤ x¯i (t) + xini (t),

(6.1)

and then the area not harvested in that period will comprise the over-mature trees at the next step, namely x¯i (t + 1) = x¯i (t) + xini (t) − ui (t).

(6.2)

The fact that immature trees cannot be harvested is represented by j +1

xi

j

(t + 1) = xi (t), j = 1, . . . , ni − 1.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2019 A. J. Zaslavski, Optimal Control Problems Arising in Forest Management, SpringerBriefs in Optimization, https://doi.org/10.1007/978-3-030-23587-1_6

(6.3)

109

110

6 Applications to the Forest Management Problem



The total harvested area expressed by the equation

i∈I



ui (t) is allocated to new seedlings which is

xi1 (t + 1) =

i∈I



ui (t).

(6.4)

i∈I

In the sequel we use the notation xini +1 = x¯i , i ∈ I.

(6.5)

A representation of the forest in terms of the age distribution at time t is provided by the state x(t) = (x1 (t), . . . , xk (t)), where xi (t) = (xi1 (t), . . . , xini (t), xini +1 (t)) describes the areas occupied in year t by trees of species i with ages 1, 2, . . . , ni and over ni . The first and last components of each vector xi (t) are controlled by the sowing and harvesting policies. m = {x = (x , . . . , x ) ∈ R m : x ≥ 0, i = 1, . . . , m}. Let R+ 1 m i  Let N = i∈I (ni +1). Every vector x ∈ R N is represented as x = (x1 , . . . , xk ), where xi = (xi1 , . . . , xini , xini +1 ) ∈ R ni +1 for all integers i = 1, . . . , k. N such that Denote by Δ the set of all x ∈ R+  i∈I

⎡ ⎣

n i +1

⎤ xi ⎦ = S. j

(6.6)

j =1

A sequence {x(t)}∞ t=0 ⊂ Δ is called a program if for all integers t ≥ 0 and all k. i ∈ I (6.1)–(6.4) hold (see (6.5)) with some u(t) = (u1 (t), . . . , uk (t)) ∈ R+ T2 Let the integers T1 , T2 satisfy 0 ≤ T1 < T2 . A sequence {x(t)}t=T1 ⊂ Δ is called a program if (6.1)–(6.4) hold for all i ∈ I and for all integers t = T1 , . . . , T2 − 1 k. (see (6.5)) with some u(t) = (u1 (t), . . . , uk (t)) ∈ R+ An alternative equivalent definition of a program is given with the help of the transition possibility. Define

j +1 j = xi for all i ∈ I and all j ∈ {1, . . . , ni } \ {ni } Ω = (x, y) ∈ Δ × Δ : yi  and for all i ∈ I, xini +1 + xini − yini +1 ≥ 0 . (6.7) Evidently, if (x, y) ∈ Ω, then  i∈I

yi1 =

   n(i)+1 xi + xini − yin(i)+1 . i∈I

(6.8)

6.1 Preliminaries

111

It is not difficult to see that a sequence {x(t)}∞ t=0 ⊂ Δ is a program if and only if (x(t), x(t + 1)) ∈ Ω for all integers t ≥ 0. Let the integers T1 , T2 satisfy 0 ≤ T1 < T2 . It is not difficult to see that a 2 sequence {x(t)}Tt=T ⊂ Δ is a program if and only if (x(t), x(t + 1)) ∈ Ω for all 1 t = T1 , . . . , T2 − 1. For each (x, y) ∈ Ω set V (x, y) = (v1 (x, y), . . . , vk (x, y)), where for i = 1, . . . , k, vi (x, y) = xini +1 + xini − yini +1 . Set  Δ0 = x ∈

k R+

:

k 

 xi ≤ S .

i=1

In this chapter we assume that a benefit at moment t = 0, 1, . . . is represented by an upper semicontinuous function wt : Δ0 → R 1 and at a moment t = 0, 1, . . . , wt (V (x, y)) is the benefit obtained today if the forest today is x and the forest tomorrow is y, where (x, y) ∈ Ω. We suppose that wt = w for all integers t ≥ 0. Evidently, Δ is a compact set in R N , Ω is a closed subset of Δ × Δ and w ◦ V : Ω → R 1 is an upper semicontinuous function. Put n¯ = max{ni : i ∈ I }.

(6.9)

This model is a particular case of the general optimal control system studied in Chaps. 2, 3, and 5. Proposition 2.1 implies that assumption (A) holds with L¯ = N + n¯ + 1. All the results of Chap. 2 hold for the particular case considered here with u = w◦V . We use the notation and definitions introduced and used in Chaps. 2 and 3. Let y, z ∈ Δ and let the integers T1 , T2 satisfy T1 < T2 . Set U (y, T1 , T2 ) = sup

⎧ 2 −1 ⎨T ⎩

w(V (xt , xtt+1 )) :

t=T1

2 is a program and xT1 = y {xt }Tt=T 1

⎫ ⎬ ⎭

,

112

6 Applications to the Forest Management Problem

U (y, y, ˜ T1 , T2 ) = sup

⎧ 2 −1 ⎨T ⎩

w(V (xt , xtt+1 )) :

t=T1

2 is a program and xT1 = y, xT2 = y˜ {xt }Tt=T 1

(T1 , T2 ) = sup U

⎧ 2 −1 ⎨T ⎩

t=T1

⎫ ⎬ ⎭

, ⎫ ⎬

2 w(V (xt , xtt+1 )) : {xt }Tt=T is a program . 1 ⎭

Let μ := μ(w ◦ V ) be defined by Theorem 2.9. For any x = (x1 , . . . , xm ) ∈ R m set x = max{|xi | : i = 1, . . . , m}.

6.2 Auxiliary Results Lemma 6.1 Assume that w(x) ≤ w(y)

(6.10)

for each x = (x1 , . . . , xk ), y = (y1 , . . . , yk ) ∈ Δ0 satisfying xi ≤ yi , i = 1, . . . , k and that μ(w ◦ V ) = sup{(w ◦ V )(x, x) : x ∈ Δ, (x.x) ∈ Ω}.

(6.11)

Then there exists xk ) ∈ Δ  x = ( x1 , . . . , 

(6.12)

6.2 Auxiliary Results

such that

113

   xi =  xi1 . . . ,  xini +1 , i = 1, . . . , k, ( x,  x ) ∈ Ω,

w(V ( x,  x )) ≥ w(V (x, x)) for all x ∈ Δ such that (x, x) ∈ Ω,

(6.13) (6.14)

and that for each i = 1, . . . , k, xi1 , j = 1, . . . , ni ,  xini +1 = 0.  xi =  j

Proof In view of (6.11), there exists  x = ( x1 , . . . ,  xk ) ∈ Δ satisfying (6.12)–(6.14). By (6.12), (6.13), and the definition of Ω, for each i = 1, . . . , k, j

 xi =  xi1 , j = 1, . . . , ni . If  xini +1 = 0, i = 1, . . . , k, then the assertion of the lemma holds. Assume that d :=

k 

 xini +1 > 0.

(6.15)

i=1

Then V ( x,  x ) = ( x11 , . . . ,  xk1 ) by definition. Define y = (y1 , . . . , yk ), for i = 1, . . . , k by ⎛ j yi

j = xi

k 

⎞−1 , j = 1, . . . , ni , yini +1 = 0.

(6.16)

  y ∈ Δ, (y, y) ∈ Ω, V (y, y) = y11 , . . . , yk1 .

(6.17)

+d⎝

np ⎠

p=1

It is not difficult to see that

It follows from (6.10), (6.11), and (6.17) that w(V (y, y)) ≥ w(V ( x,  x )) ≥ w(V (x, x)) for all x ∈ Δ satisfying (x, x) ∈ Ω. Lemma 6.1 is proved.

114

6 Applications to the Forest Management Problem

Let  x = ( x1 , . . . ,  xk ) ∈ Δ, ( x,  x ) ∈ Ω,

(6.18)

xi1 . . . ,  xini +1 ), i = 1, . . . , k,  xi = ( for i = 1, . . . , k,  xi =  xi1 , j = 1, . . . , ni ,  xini +1 = 0,

(6.19)

I˜ = {i ∈ I : xi1 > 0}.

(6.20)

 ∈ (0, min{ xi1 : i ∈ I˜}).

(6.21)

j

Proposition 6.2 Let

Then for each x, y ∈ Δ satisfying x − x, ¯ y − x ¯ ≤ (2N )−1 , +n+1 ¯ such that there is a program {x(t)}N t=0

x(0) = x, x(N + n¯ + 1) = y, x(t) −  x  ≤ , t = 0, . . . , N + n¯ + 1.

(6.22)

Proof We may assume without loss of generality that I˜ = {1, . . . , k0 }, where k0 ∈ [1, k] is an integer. Let x, y ∈ Δ, x −  x , y −  x  ≤ (2N )−1 .

(6.23)

+n+1 ¯ which satisfies (6.22). Set x(0) = x. We construct a program {x(t)}N t=0 Assume that t ∈ {0, . . . , N + n} ¯ is an integer and that a program {x(p)}tp=0 was k and defined. In order to define x(t + 1) we need to choose v(t) ∈ R+ k (x11 (t + 1), . . . , xk1 (t + 1)) ∈ R+

such that for all integers i = 1, . . . , k we have vi (t) ≤

xini (t) + xini +1 (t),

k  i=1

xi1 (t

+ 1) =

k  i=1

vi (t)

(6.24)

6.2 Auxiliary Results

115

and then the state x(t + 1) is defined as follows for all integers i = 1, . . . , k: j −1

j

xi (t + 1) = xi

(t), j is an integer satisfying 2 ≤ j ≤ ni ,

xini +1 (t + 1) = xini (t) + xini +1 (t) − vi (t).

(6.25) (6.26)

Thus the construction of the program is done as follows: if the states are defined till k and (x 1 (t +1), . . . , x 1 (t +1)) ∈ R k moment t ≤ N + n, ¯ then we choose V (t) ∈ R+ + k 1 such that (6.24) holds and define x(t + 1) by (6.25) and (6.26). For each t = 0, . . . , N − 1 set xi1 − (2N )−1 , i = 1, . . . , k0 , xi1 (t + 1) = vi (t) =  xi1 (t + 1) = vi (t) = 0 for all integers i satisfying k0 < i ≤ k

(6.27)

j

and define xi (t + 1), i = 1, . . . , k, j = 2, . . . , ni + 1 by (6.25) and (6.26). In order to show that {x(t)}N t=0 is a program it is sufficient to show that the first part of (6.24) is true for all integers i = 1, . . . , k0 . It follows from (6.23), (6.27), and (6.25) that for all integers t = 0, . . . , N, all integers i = 1, . . . , k0 , and all integers j = 1, . . . , ni we have  xi1 − (2N )−1 ≤ xi (t) ≤  xi1 + (2N )−1 . j

(6.28)

Equations (6.27) and (6.28) imply (6.24) for all integers i = 1, . . . , k0 . Then {x(t)}N t=0 is a program. In view of (6.23), (6.27), and (6.28), for all integers t = 0, . . . , N , 0≤

k 

xini +1 ≤ S −

i=1

k0 

ni  xi1 + N (2N )−1 ≤ 2−1 .

(6.29)

i=1

By (6.19), (6.28), and (6.29), we have x(t) −  x  ≤ 2−1 , t = 0, . . . , N.

(6.30)

Let us consider the state x(N ). In view of (6.25) and (6.27), xi1 − (2N )−1 , j = 1, . . . , n(i), xi (N ) =  j

j

i = 1, . . . , k0 , xi (N ) = 0, j = 1, . . . , ni , where an integer i satisfies k0 < i ≤ k. We continue to construct the program.

(6.31)

116

6 Applications to the Forest Management Problem

If at the moment t the state x(t) has already been defined, then we define V (t) ∈ k and (x 1 (t + 1), . . . , x 1 (t + 1)) ∈ R k such that (6.24) holds and then define R+ + k 1 x(t + 1) by (6.25) and (6.26). For every integer s = 1, . . . , n¯ set Is = {i ∈ {1, . . . , k} : ni = s}.

(6.32)

(Note that for some integers s we can have Is = ∅.) For all integers i = 1, . . . , k and all integers t = N, . . . , N + n¯ − 1 we define (1) k which satisfies (6.24) xi (t + 1) ≥ 0, show that there exists a vector V (t) ∈ R+ j and then define xi (t + 1), i = 1, . . . , k where an integer j satisfies 2 ≤ j ≤ ni + 1. We begin with the definition of xi1 (t + 1), i = 1, . . . , k, t = N, . . . , N + n¯ − 1. Let i ∈ {1, . . . , k}. If i ∈ I1 (ni = 1), then we set

 xi1 − (2N )−1 , 0 for all integers t satisfying 1 ≤ t < n, ¯ xi1 (t + N ) = max  (6.33) xi1 (n¯ + N ) = yi1 + yi2 . If i ∈ Is (ni = s) with s > 1, then we set

 xi1 (N + t) = max  xi1 − (2N )−1 , 0 for all integers t satisfying 1 ≤ t < n¯ + 1 − ni ,

(6.34)

xi1 (n¯ + 1 − s + N ) = yini +1 + yi1 ,

(6.35)

¯ xi1 (N + t) = yin+2−t for all integers t satisfying n¯ + 1 − s < t ≤ n. ¯

(6.36) j

¯ Then we define xi (t + 1) by We have defined xi1 (t + N ), i ∈ I, t = 1, . . . , n. (6.25) for all t = N, . . . , N + n¯ − 1, all integers j satisfying 2 ≤ j ≤ ni , and all j i ∈ I . Thus we have defined xi (t + N ), t = 1, . . . , n, ¯ i ∈ I, j = 1, . . . , n(i). Set φ(t) =

ni 

j

xi (N + t) −

i∈I j =1

+





xini (N + t)

i∈I

xi1 (N + t + 1), t = 0, . . . , n¯ − 1.

(6.37)

i∈I

In view of (6.37), (6.35), and (6.33), for all integers t = 0, . . . , n¯ − 1, we have φ(t) =

ni  i∈I j =1

j

xi (N + t + 1).

(6.38)

6.2 Auxiliary Results

117

By (6.33)–(6.38), (6.31), and (6.23), φ(t) ≤ φ(t + 1) for all integers t ∈ [0, n¯ − 1].

(6.39)

It follows from (6.38) and (6.33)–(6.36) that φ(n¯ − 1) =

ni 

j

xi (N + n) ¯ =

i∈I j =1

i +1  n

j

yi = S.

i∈I j =1

Hence φ(t) ≤ S, t = 0, . . . , n¯ − 1.

(6.40)

+n¯ In order to complete the construction of the program {x(t)}N t=0 we need to n +1 k t = N, . . . , N + n determine V (t) ∈ R+ ¯ − 1 and xi i (t + 1) ≥ 0, t = N, . . . , N + n¯ − 1] such that the second part of (6.24) is valid for all integers t = N, . . . , N + n¯ − 1 and

x(t) ∈ Δ, t = N + 1, . . . , N + n. ¯

(6.41)

We complete this construction by induction. Let t = 0. In view of (6.37) and (6.40), 

xi1 (N + 1) ≤

i∈I



 xini (N ) + xini +1 (N ) .

k such that Relation (6.42) implies that there exists V (N) ∈ R+   xi1 (N + 1) = vi (N ), vi (N ) ≤ xini (N ) + xini +1 (N ), i ∈ I. i∈I

(6.42)

i∈I

(6.43)

i∈I

Define xini +1 (N + 1), i ∈ I by (6.20) with t = N. k, Assume that an integer τ satisfies 1 ≤ τ < n, ¯ we have defined V (t) ∈ R+ t = N, . . . , N + τ − 1, xini +1 (t), i ∈ I , t = N + 1, . . . , N + τ , and x(t) ∈ Δ, t = N + 1 . . . , N + τ such that (6.24) is valid for all integers t = N, . . . , N + τ − 1. (Note that for τ = 1 this assumption holds.) By (6.37) and (6.40),     n +1 xi i (N + τ ) + xini (N + τ ) . xi1 (N + τ + 1) ≤ i∈I

i∈I

k such that This implies that there is V (N + τ ) ∈ R+   xi1 (N + τ + 1) = Vi (N + τ ), i∈I

i∈I

Vi (N + τ ) ≤ xini +1 (N + τ ) + xini (N + τ ), i ∈ I.

118

6 Applications to the Forest Management Problem

Define xini +1 (N + τ + 1) by (6.26) with t = N + τ . Clearly, the assumption made for τ also holds for τ + 1. Thus by induction we defined xini +1 (t + 1) ≥ 0, t = k , t = N, . . . , N + n ¯ − 1 such that (6.31) holds and N, . . . , N + n¯ − 1, V (t) ∈ R+ (6.24) holds for all t = N, . . . , N + n¯ − 1. +n¯ Thus we have constructed a program {x(t)}N t=0 . It follows from (6.33)–(6.36) that for all integers i ∈ I , xini (N + n) ¯ = yini +1 + yi1 , j

j +1

xi (N + n) ¯ = yi

for all integers j satisfying 1 ≤ j < n(i),

xi1 (N + n) ¯ = yi2 . Evidently, ni  i∈I j =1

j

xi =

i +1  n

j

yi = S.

i∈I j =1

+n+1 ¯ Set x(N + n¯ + 1) = y. It is clear that (x(N + n), ¯ y) ∈ Ω. Hence {x(t)}N is a t=0 program such that x(0) = x, x(N + n¯ + 1) = y. In view of (6.33)–(6.36),

x(t) − x ¯ ≤ , t = 0, . . . , N + n¯ + 1. This completes the proof of Proposition 6.2.

6.3 Turnpike Results In the sequel the notation x ≥ y, x > y, x >> y where x, y ∈ R q have their usual meaning. We assume that w : Δ0 → R 1 is a continuous and a strictly concave function such that w(x) ≤ w(y) for all x, y ∈ Δ0 satisfying x ≤ y.

(6.44)

Note that w(V (x, y)) is the benefit obtained today if the forest today is x and the forest tomorrow is y where (x, y) ∈ Ω. In the sequel we assume that supremum over empty set is −∞. Theorem 2.9 implies the following result. Theorem 6.3 There exists M > 0 and the limit (0, p)/p μ := μ(w ◦ V ) := lim U p→∞

(6.45)

6.3 Turnpike Results

119

such that (0, p) − μ| ≤ M/p for all natural numbers p. |p−1 U Clearly Ω is a convex set and w ◦V is a concave function. Let μ be as guaranteed by Theorem 6.3. Then Theorem 2.10 implies the following result. Theorem 6.4 μ = sup{w(V (x, x)) : x ∈ Δ and (x, x) ∈ Ω}. Lemma 6.1 implies that there exists  x = ( x1 , . . . ,  xk ) ∈ Δ such that ( x,  x ) ∈ Ω, for i = 1, . . . , k,   xi1 . . . ,  xini +1 ,  xi =   xi =  xi1 , j = 1, . . . , ni ,  xini +1 = 0, j

μ = w(V ( x,  x )) ≥ w(V (x, x)) for all x ∈ Δ satisfying (x, x) ∈ Ω. Proposition 6.2 implies that (B3) holds. It is clear that (B2) holds too. Theorem 6.5 Assume that a program {x(t)}∞ t=0 is good. Then x. lim x(t) = 

t→∞

Proof First we show that x,  x ). lim V (x(t), x(t + 1)) = V (

t→∞

(6.46)

For all nonnegative integers t set x ). z(t) = 2−1 (x(t) + 

(6.47)

Evidently, {z(t)}∞ t=0 is a program and x,  x ). V (z(t), z(t + 1)) = 2−1 V (x(t), x(t + 1)) + 2−1 V (

(6.48)

120

6 Applications to the Forest Management Problem

It follows from (6.48) and the concavity of the function w that {z(t)}∞ t=0 is a good program and the sequence ∞ T −1  w(V (z(t), z(t + 1))) − T μ is bounded. (6.49) T =1

t=0

Assume that (6.46) does not hold. There exist a positive number  and a strictly increasing sequence of natural numbers {tj }∞ j =1 such that V ( x,  x ) − V (x(tj ), x(tj + 1)) ≥ .

(6.50)

Since the function w is continuous and strictly concave there exists a positive number δ such that for every pair of points y1 , y2 ∈ Δ0 which satisfies y1 − y2  ≥ , we have   w 2−1 (y1 + y2 ) − 2−1 w(y1 ) − 2−1 w(y2 ) ≥ δ. (6.51) Since the program {xt }∞ t=0 is good there exists a positive number M0 such that for every integer T > 0, we have T −1      (w(V (x(t), x(t + 1))) − μ) ≤ M0 . (6.52)    t=0

In view of (6.48), the strict concavity of the function w, (6.50) and the choice of the number δ (see (6.51)), (6.52), for every integer k > 0, t k +1

w(V (z(t), z(t + 1))) − tk μ − 2μ

t=0

=

t k +1 

 w(2−1 V (x(t), x(t + 1)) + 2−1 V ( x,  x )) − tk μ − 2μ

t=0

=

t k +1

[w(2−1 V (x(t), x(t + 1))

t=0 −1

+2

V ( x,  x )) − 2−1 w(V (x(t), x(t + 1))) − 2−1 w(V ( x,  x ))]

− tk μ − 2μ +

t k +1

2−1 w(V (x(t), x(t + 1))) + 2−1 (tk + 2)w(V ( x,  x ))

t=0

≥ δk + 2−1

t k +1

[w(V (x(t), x(t + 1))) − μ]

t=0

≥ δk − M0 → ∞ as k → ∞, a contradiction. The contradiction we have reached proves that (6.46) is valid.

6.3 Turnpike Results

121

Set  = V ( x21 , . . . ,  xk1 ) V x,  x ) = ( x11 , 

(6.53)

and for all integers t ≥ 0 set V (t) = V (x(t), x(t + 1)),

(6.54)

V (t) = (v1 (t), . . . , vk (t)).

(6.55)

where

Let t ≥ 0 be an integer. In view of (6.2), (6.5), (6.26), and (6.54), for all integers i = 1, . . . , k, xini (t) = xini +1 (t + 1) − xini +1 (t) + vi (t), xini +1 (t + 1) = xini +1 (t) + xini (t) − vi (t), i = 1, . . . , k

(6.56)

and j +1

xi

j

(t + 1) = xi (t), j = 1, . . . , ni − 1

(6.57)

for all integers i = 1, . . . , k if ni ≥ 2. It follows from (6.56) and (6.57) that for all integers T ≥ 1 and for all integers i = 1, . . . , k, T +n i −1 

vi (t) =

T +n i −1  

t=T

xini +1 (t) + xini (t) − xini +1 (t + 1)



t=T

= xini +1 (T ) − xini +1 (T + ni ) +

T +n i −1 

xini (t)

t=T

= xini +1 (T ) − xini +1 (T + ni ) +

ni 

j

(6.58)

xi (T ).

j =1

By (6.58) ⎤ ⎤ ⎡ ⎡ ni i −1     T +n   n +1  n 1 j ⎣ ⎣ xi i (T ) − xi + (T + ni ) + vi (t)⎦ = xi (T )⎦ i∈I

t=T

i∈I

=



⎡ ⎣

i∈I

=S−

n i +1 j =1

 i∈I

⎤ xi (T )⎦ − j

i∈I



j =1

xini +1 (T + ni )

i∈I

xini +1 (T + ni ).

(6.59)

122

6 Applications to the Forest Management Problem

By (6.46), (6.53)–(6.55), and (6.59), we have  i∈I

ni  xi1 = lim

T →∞



⎡ ⎣

T +n i −1 

i∈I

⎤ vi (t)⎦ = S − lim

T →∞

t=T



xini +1 (T + ni ).

(6.60)

i∈I

By (6.53), (6.60), the choice of  x , we have lim x ni +1 (T T →∞ i

+ ni ) = 0, i = 1, . . . , k.

(6.61)

It follows from (6.56) and (6.61) that for all integers i = 1, . . . , k, lim (xini (t) − vi (t)) = 0.

t→∞

(6.62)

In view of (6.46), (6.59), and (6.62), for all integers i = 1, . . . , k, lim x ni (t) t→∞ i

= lim vi (t) = Vi ( x,  x) =  xi1 . t→∞

(6.63)

By (6.63), for all integers i = 1, . . . , k and every integer j which satisfies 1 ≤ j ≤ ni , j lim x (t) t→∞ i

= lim xini (t + ni − j ) =  xi1 . t→∞

Combined with (6.61) this implies that x. lim x(t) = 

t→∞

This completes the proof of Theorem 6.5. Thus (B1) holds for w ◦ V and all the results of Chaps. 3 and 5 are true in our particular case.

6.4 Generic Results Denote by C(Δ0 ) the set of all continuous functions on the set k Δ0 = {v ∈ R+ :

k 

vi ≤ S},

i=1

where S > 0. For each f ∈ C(Δ0 ) set f  = sup{|f (x)| : x ∈ Δ0 }.

6.4 Generic Results

123

We continue to study the forest management problem which is a particular case of the optimal control problem considered in Chaps. 3 and 5 with u = w ◦ V , where w ∈ C(Δ0 ). Recall that for each (x, y) ∈ Ω, V (x, y) = (v1 (x, y), . . . , vk (x, y)),

(6.64)

vi (x, y) = xini +1 + xini − yini +1 .

(6.65)

where for i = 1, . . . , k,

Denote by Cm (Δ0 ) the set of all w ∈ C(Δ0 ) such that w : Δ0 → [0, ∞), w(x) ≤ w(y) for all x, y ∈ Δ0 satisfying x ≤ y, and that μ(w ◦ V ) = sup{w(V (x, x)) : x ∈ Δ and (x, x) ∈ Ω}.

(6.66)

It is not difficult to see that Cm (Δ0 ) is a closed subset of C(Δ0 ). The space Cm (Δ0 ) is equipped with the metric dC (f1 , f2 ) = f1 − f2 , f1 , f2 ∈ Cm (Δ0 ). Let w ∈ Cm (Δ0 ). Lemma 6.1 implies that there exists  x = ( x1 , . . . ,  xk ) ∈ Δ such that ( x,  x ) ∈ Ω, w(V ( x,  x )) ≥ w(V (x, x)) for all x ∈ Δ satisfying (x, x) ∈ Ω,

(6.67) (6.68)

for i = 1, . . . , k, xi1 . . . ,  xini +1 ),  xi = ( xi1 , j = 1, . . . , ni ,  xini +1 = 0.  xi = 

(6.69)

μ(w ◦ V ) = w(V ( x,  x )) > 0.

(6.70)

xi1 > 0}. I0 = {i ∈ {1, . . . , k} : 

(6.71)

j

Assume that

Let γ ∈ (0, k −1 ) and set

124

6 Applications to the Forest Management Problem

For each i ∈ I0 define a function φi : [0, ∞) → [0, ∞) by x,  x )) for all x ∈ [ xi1 , ∞), φi (x) = w(V (

(6.72)

φi (x) = x( xi1 )−1 w(V ( x,  x )) for all x ∈ [0,  xi1 ].

(6.73)

For every x = (x1 , . . . , xk ) ∈ Δ0 define wγ (x) = w(x) + γ



φi (xi ).

(6.74)

i∈I0

Clearly, w is a continuous increasing function. Lemma 6.6 wγ ∈ Cm (Δ0 ) and for each (wγ ◦ V )-good program {x(t)}∞ t=0 , x. lim x(t) = 

t→∞

Proof Theorem 2.9 and (6.70) imply that there exists M0 > 0 such that for each (Ω)-program {x(t)}∞ t=0 and each integer T ≥ 1, T −1

w(V (x(t), x(t + 1))) ≤ T w(V ( x,  x )) + M0 .

(6.75)

t=0

Let {x(t)}∞ t=0 be a program. By (6.72)–(6.75), for each integer T > 0, T −1

wγ (V (x(t), x(t + 1)))

t=0

=

T −1

w(V (x(t), x(t + 1))) + γ

t=0

≤ T w(V ( x,  x )) + M0 + γ T



T −1 

φi (vi (x(t), x(t + 1)))

t=0 i∈I0

(w ◦ V )( x,  x)

i∈I0

= M0 + T w(V ( x,  x ))(1 + γ Card(I0 )) = M0 + (wγ ◦ V )( x,  x )T . This implies that x,  x ), μ(wγ ◦ V ) = (wγ ◦ V )( and that wγ ∈ Cm (Δ0 ). Let {x(t)}∞ t=0 be a (wγ ◦ V )-good program. We show that x. lim x(t) = 

t→∞

(6.76)

6.4 Generic Results

125

In view of (6.76), there exists M1 > 0 such that for each integer T ≥ 1, T −1      wγ (V (x(t), x(t + 1))) − T wγ (V ( x,  x )) ≤ M1 .   

(6.77)

t=0

It follows from (6.72)–(6.75) and (6.77) that for all integers T ≥ 1, x,  x ))(1 + γ Card(I0 )) − M1 + T w(V ( = −M1 + T (wγ ◦ V )( x,  x) ≤

T −1

wγ (V (x(t), x(t + 1)))

t=0

=

T −1

w(V (x(t), x(t + 1))) + γ

T −1 

φi (vi (x(t), x(t + 1)))

t=0 i∈I0

t=0

≤ T (w ◦ V )( x,  x ) + M0 + γ

T −1 

φi (vi (x(t), x(t + 1))).

(6.78)

t=0 i∈I0

By (6.78), for each integer T > 0, M0 + M1 ≥ γ T Card(I0 )(w ◦ V )( x,  x) − γ

T −1 

φi (vi (x(t), x(t + 1)))

t=0 i∈I0

=γ

T −1 

((w ◦ V )( x,  x ) − φi (vi (x(t), x(t + 1)))).

(6.79)

t=0 i∈I0

Let i ∈ I0 . It follows from (6.72), (6.73), and (6.79) that for every integer T > 0, γ −1 (M0 + M1 ) ≥

T −1

((w ◦ V )( x,  x ) − φi (vi (x(t), x(t + 1)))).

t=0

Let  > 0. In view of (6.80), Card({t ∈ {0, 1, . . . } : (w ◦ V )( x,  x ) − φi (vi (x(t), x(t + 1))) ≥ } ≤ (γ )−1 (M0 + M1 ) and there exists an integer t > 0 such that for all integers t ≥ t , (w ◦ V )( x,  x ) − φi (vi (x(t), x(t + 1))) < .

(6.80)

126

6 Applications to the Forest Management Problem

Since  an arbitrary positive number it follows from (6.72) and (6.73) that xi1 , i ∈ I0 . lim inf vi ((x(t), x(t + 1)) ≥ 

(6.81)

t→∞

For every nonnegative integer t ≥ 0, set V (t) = (v1 (t), . . . , vk (t)) = V (x(t), x(t + 1)).

(6.82)

Let t ≥ 0 be an integer. By (6.65) and (6.82) for all i = 1, . . . , k, xini (t) = xini +1 (t + 1) − xini +1 (t) + vi (t),

(6.83)

xini +1 (t + 1) = xini +1 (t) + xini (t) − vi (t), i = 1, . . . , k

(6.84)

j +1

xi

j

(t + 1) = xi (t), j = 1, . . . , ni − 1

(6.85)

for i = 1, . . . , k if ni ≥ 2. By (6.84) and (6.85) and all integers T ≥ 1 for all i = 1, . . . , k, T +n i −1 

vi (t) =

t=T

T +n i −1  

xini +1 (t) + xini (t) − xini +1 (t + 1)



t=T

=

xini +1 (T ) − xini +1 (T

+ ni ) +

T +n i −1 

xini (t)

t=T

= xini +1 (T ) − xini +1 (T + ni ) +

ni 

j

(6.86)

xi (T ).

j =1

By (6.86), for every integer T ≥ 1, k 

⎡ ⎣

i=1

=

T +n i −1 

⎡ ⎤ ni k  k     j n +1 n +1 ⎣ xi i (T ) − xi i (T + ni ) + vi (t)⎦ = xi (T )⎦ ⎤

t=T k 

⎡ ⎣

i=1

=S−

i=1 n i +1 j =1

k  i=1

⎤ xi (T )⎦ − j

i=1 k 

j =1

xini +1 (T + ni )

i=1

xini +1 (T + ni ).

(6.87)

6.4 Generic Results

127

In view of (6.67), (6.69), (6.71), (6.81), (6.82), and (6.87), S=

k  i=1

≤

k 

ni  xi1 ⎡ ⎣

T +n i −1  t=T

i=1

≤ lim inf T →∞

k 

⎤ lim inf vi (x(t), x(t + 1))⎦ T →∞

⎡ ⎣

T +n i −1 

⎤ vi (t)⎦

t=T

i=1

 = lim inf S − T →∞

k 

 xini +1 (T

+ ni )

i=1

= S − lim sup

k 

T →∞ i=1

xini +1 (T + ni ).

(6.88)

It follows from (6.88) that lim x ni +1 (T T →∞ i

+ ni ) = 0, i = 1, . . . , k.

(6.89)

By (6.83) and (6.89) for all i = 1, . . . , k lim (xini (t) − vi (t)) = 0.

t→∞

(6.90)

By (6.87) and (6.89), lim

k T +n i −1  

T →∞

vi (t) = S.

(6.91)

t=T

i=1

In view of (6.71) and (6.81), for every i = 1, . . . , k, xi1 , lim inf vi (t) ≥  t→∞

By (6.88) and (6.90)–(6.92), xi1 , i = 1, . . . , k, lim vi (t) = 

t→∞

lim x ni (t) t→∞ i j lim x t→∞ i

Lemma 6.6 is proved.

= xi1 , i = 1, . . . , k,

= xi1 , i = 1, . . . , k, j = 1, . . . , ni .

(6.92)

128

6 Applications to the Forest Management Problem

Theorem 6.7 There exists a set F ⊂ Cm (Δ0 ) which is a countable intersection of open everywhere dense subsets of Cm (Δ0 ) such that for each w ∈ F there exists x w = (x1w , . . . , xkw ) ∈ Δ such that (x w , x w ) ∈ Ω, for i = 1, . . . , k, xiw = (xiw,1 . . . , xiw,ni +1 ), w,j

xi

= xiw,1 , j = 1, . . . , ni , xiw,ni +1 = 0

and that for each (w ◦ V )-good program {x(t)}∞ t=0 , lim x(t) = x w .

t→∞

Proof Let w ∈ Cm (Δ0 ) be such that μ(w ◦ V ) > 0, γ ∈ (0, k −1 ) and p ≥ 1 be an integer. By Lemmas 6.1 and 6.6 and Theorem 3.12, there exist an open neighborhood U (w, γ , p) of wγ in Cm (Δ0 ), δ(w, γ , p) > 0, a natural number τ (w, γ , p) and x w = (x1w , . . . , xkw ) ∈ Δ such that the following properties hold: (a) (x w , x w ) ∈ Ω, for i = 1, . . . , k, xiw = (xiw,1 . . . , xiw,ni +1 ), w,j

xi

= xiw,1 , j = 1, . . . , ni , xiw,ni +1 = 0,

μ(w ◦ V ) = w(V (x w , x w )); (b) for each (wγ ◦ V )-good program {x(t)}∞ t=0 , lim x(t) = x w ;

t→∞

6.4 Generic Results

129

(c) for each integer T > 2τ (w, γ , p), each ut ∈ U (w, γ , p), t = 0, . . . , T − 1 and each program {x(t)}Tt=0 which satisfies T −1

  T −1 (ut ◦ V )(xt , xt+1 ) ≥ U {ut ◦ V }t=0 , x0 , xT − δ(w, γ , p)

t=0

we have x(t) − x w  ≤ 1/n, t = τ (w, γ , p), . . . , T − τ (w, γ , p). Define F = ∩∞ q=1 ∪ {U (w, γ , p) : w ∈ Cm (Δ0 ), μ(w ◦ V ) > 0, γ ∈ (0, 1/k), p ≥ q is an integer}.

(6.93)

Clearly, F is a countable intersection of open everywhere dense sets in Cm (Δ0 ). Let u ∈ F,  ∈ (0, 1).

(6.94)

Choose a natural number q such that q > 4 −1 .

(6.95)

It follows from (6.93) and (6.94) that there exist w ∈ Cm (Δ0 ), γ ∈ (0, k −1 ), and an integer p ≥ q such that μ(w ◦ V ) > 0, u ∈ U (w, γ , p). Assume that ξ1 , ξ2 ∈ Δ, (ξi , ξi ) ∈ Ω, i = 1, 2, (u ◦ V )(ξi , ξi ) = μ(u ◦ V ), i = 1, 2. It is not difficult to see that for each i = 1, 2 and each integer T > 0, U (u ◦ V , ξi , ξi , 0, T ) = T (u ◦ V )(ξi , ξi ).

(6.96)

130

6 Applications to the Forest Management Problem

Combined with property (c) and (6.96) this implies that x u − ξi  ≤ p−1 ≤ q −1 < /4, i = 1, 2,

(6.97)

ξ1 − ξ2  < . Since  is an arbitrary positive number we conclude that ξ1 = ξ2 and {ξ ∈ Δ : (ξ, ξ ) ∈ Ω, u(V (ξ, ξ )) = μ(u ◦ V )} is a singleton. Denote its unique element by x u . In view of (6.97), x u − x w  ≤ p−1 < .

(6.98)

Since  is an arbitrary positive number it follows from property (a) that x u = (x1u , . . . , xku ) ∈ Δ, (x u , x u ) ∈ Ω, for i = 1, . . . , k, xiu = (xiu,1 . . . , xiu,ni +1 ), u,j

xi

= xiu,1 , j = 1, . . . , ni , xiu,ni +1 = 0.

Assume that an integer T > 2τ (w, γ , p) and that a program {xt }Tt=0 satisfies T −1

(u ◦ V )(xt , xt+1 ) ≥ U (u ◦ V , x0 , xT , 0, T ) − δ(w, γ , p).

t=0

Together with property (c), (6.96), and (6.98) this implies that for all integers t = τ (w, γ , p), . . . , T − τ (w, γ , p), x(t) − x w  ≤ 1/p and x(t) − x u  ≤ 2/p < . Combined with Theorem 4.1 this implies that every (u◦V )-good program converges to x u . Theorem 6.7 is proved. It is clear that for every w ∈ F (B1), (B2), and (B3) hold for w ◦ V and all the results of Chaps. 3 and 5 hold.

References

1. Anderson BDO, Moore JB (1971) Linear optimal control. Prentice-Hall, Englewood Cliffs, NJ. 2. Arkin VI, Evstigneev IV (1987) Stochastic models of control and economic dynamics. Academic Press, London. 3. Aseev SM, Kryazhimskiy AV (2004) The Pontryagin Maximum principle and transversality conditions for a class of optimal control problems with infinite time horizons, SIAM J. Control Optim.: 43, 1094–1119. 4. Aseev SM, Veliov VM (2012) Maximum principle for infinite-horizon optimal control problems with dominating discount, Dynamics Contin. Discrete Impuls. Syst., Series B: 19, 43–63. 5. Atsumi H (1965) Neoclassical growth and the efficient program of capital accumulation, Review of Economic Studies: 32, 127–136. 6. Aubry S, Le Daeron PY (1983) The discrete Frenkel-Kontorova model and its extensions I, Physica D: 8, 381–422. 7. Bachir M, Blot J (2015) Infinite dimensional infinite-horizon Pontryagin principles for discretetime problems, Set-Valued and Variational Analysis: 23, 43–54. 8. Bachir M, Blot J (2017) Infinite dimensional multipliers and Pontryagin principles for discretetime problems, Pure and Applied Functional Analysis: 2, 411–426. 9. Baumeister J, Leitao A, Silva GN (2007) On the value function for nonautonomous optimal control problem with infinite horizon, Syst. Control Lett.: 56, 188–196. 10. Blot J (2009) Infinite-horizon Pontryagin principles without invertibility, J. Nonlinear Convex Anal.: 10, 177–189. 11. Blot J, Cartigny P (2000) Optimality in infinite-horizon variational problems under sign conditions, J. Optim. Theory Appl.: 106, 411–419. 12. Blot J, Hayek N (2000) Sufficient conditions for infinite-horizon calculus of variations problems, ESAIM Control Optim. Calc. Var.: 5, 279–292. 13. Blot J, Hayek N (2014) Infinite-horizon optimal control in the discrete-time framework. SpringerBriefs in Optimization, New York. 14. Brock WA (1970) On existence of weakly maximal programmes in a multi-sector economy, Review of Economic Studies: 37, 275–280. 15. Carlson DA (1990) The existence of catching-up optimal solutions for a class of infinite horizon optimal control problems with time delay, SIAM Journal on Control and Optimization: 28, 402–422. 16. Carlson DA, Haurie A, Leizarowitz A (1991) Infinite horizon optimal control. Berlin, SpringerVerlag.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2019 A. J. Zaslavski, Optimal Control Problems Arising in Forest Management, SpringerBriefs in Optimization, https://doi.org/10.1007/978-3-030-23587-1

131

132

References

17. Carlson DA, Jabrane A, Haurie A (1987) Existence of overtaking solutions to infinite dimensional control problems on unbounded time intervals, SIAM Journal on Control and Optimizaton: 25, 517–1541. 18. Cartigny P, Michel P (2003) On a sufficient transversality condition for infinite horizon optimal control problems, Automatica J. IFAC: 39, 1007–1010. 19. Coleman BD, Marcus M, Mizel VJ (1992) On the thermodynamics of periodic phases, Arch. Rational Mech. Anal.: 117, 321–347. 20. Cominetti R, Piazza A (2009) Asymptotic convergence of optimal harvesting policies for a multiple species forest, Mathematics of Operations Research: 34, 576–593. 21. Damm T, Grune L, Stieler M, Worthmann K (2014) An exponential turnpike theorem for dissipative discrete time optimal control problems, SIAM Journal on Control and Optimization: 52, 1935–1957. 22. De Oliveira VA, Silva GN (2009) Optimality conditions for infinite horizon control problems with state constraints, Nonlinear Analysis: 71, 1788–1795. 23. Evstigneev IV, Flam SD (1998) Rapid growth paths in multivalued dynamical systems generated by homogeneous convex stochastic operators, Set-Valued Anal.: 6, 61–81. 24. Gaitsgory V, Grune L, Thatcher N (2015) Stabilization with discounted optimal control, Systems and Control Letters: 82, 91–98. 25. Gaitsgory V, Mammadov M, Manic L (2017) On stability under perturbations of long-run average optimal control problems, Pure and Applied Functional Analysis: 2, 461–476. 26. Gaitsgory V, Parkinson, A, Shvartsman, I (2017) Linear programming formulations of deterministic infinite horizon optimal control problems in discrete time, Discrete Contin. Dyn. Syst. Ser. B: 22, 3821–3838. 27. Gaitsgory V, Rossomakhine S, Thatcher N (2012) Approximate solution of the HJB inequality related to the infinite horizon optimal control problem with discounting, Dynamics Contin. Discrete Impuls. Syst., Series B: 19, 65–92. 28. Gale D (1967) On optimal development in a multi-sector economy, Rev. Econ. Stud.: 34, 1–18. 29. Glizer VY, Kelis O (2017) Upper value of a singular infinite horizon zero-sum linear-quadratic differential game, Pure and Applied Functional Analysis: 2, 511–534. 30. Grune L, Guglielmi R (2018) Turnpike properties and strict dissipativity for discrete time linear quadratic optimal control problems, SIAM J. Control Optim.: 56, 1282–1302. 31. Gugat M, Trelat E, Zuazua, E (2016) Optimal Neumann control for the 1D wave equation: finite horizon, infinite horizon, boundary tracking terms and the turnpike property, Systems Control Lett. 90, 61–70. 32. Guo X, Hernandez-Lerma O (2005) Zero-sum continuous-time Markov games with unbounded transition and discounted payoff rates, Bernoulli: 11, 1009–1029. 33. Hayek (2011) Infinite horizon multiobjective optimal control problems in the discrete time case, Optimization: 60, 509–529. 34. Hammond PJ (1974) Consistent planning and intertemporal welfare economics. University of Cambridge, Cambridge. 35. Hammond PJ (1975) Agreeable plans with many capital goods, Rev. Econ. Stud.: 42, 1–14. 36. Hammond PJ, Mirrlees JA (1973) Agreeable plans, Models of economic growth. Wiley, New York, 283–299. 37. Jasso-Fuentes H, Hernandez-Lerma O (2008) Characterizations of overtaking optimality for controlled diffusion processes, Appl. Math. Optim.: 57, 349–369. 38. Jerry M, Rapaport A, Cartigny P (2010) Can protected areas potentially enlarge viability domains for harvesting management? Nonlinear Anal. Real World Appl.: 11 720–734. 39. Khan MA, Piazza A (2011) The economics of forestry and a set-valued turnpike of the classical type, Nonlinear Anal.: 74, 171–181. 40. Khan MA, Piazza A (2012) On the Mitra-Wan forestry model: a unified analysis, J. Econom. Theory: 147, 230–260. 41. Khlopin DV (2017) On Lipschitz continuity of value functions for infinite horizon problem, Pure and Applied Functional Analysis: 2, 535–552.

References

133

42. Kolokoltsov V, and Yang W (2012) The turnpike theorems for Markov games, Dynamic Games and Applications: 2, 294–312. 43. Leizarowitz A (1985) Infinite horizon autonomous systems with unbounded cost, Appl. Math. and Opt.: 13, 19–43. 44. Leizarowitz A (1986) Tracking nonperiodic trajectories with the overtaking criterion, Appl. Math. and Opt.: 14, 155–171. 45. Leizarowitz A, Mizel VJ (1989) One dimensional infinite horizon variational problems arising in continuum mechanics, Arch. Rational Mech. Anal.:106, 161–194. 46. Lykina V, Pickenhain S, Wagner M (2008) Different interpretations of the improper integral objective in an infinite horizon control problem, J. Math. Anal. Appl.: 340, 498–510. 47. Makarov VL, Rubinov AM (1977) Mathematical theory of economic dynamics and equilibria. Springer-Verlag, New York. 48. Malinowska AB, Martins N, Torres DFM (2011) Transversality conditions for infinite horizon variational problems on time scales, Optimization Lett.: 5, 41–53. 49. Mammadov M (2014) Turnpike theorem for an infinite horizon optimal control problem with time delay, SIAM Journal on Control and Optimization: 52, 420–438. 50. Marcus M, Zaslavski AJ (1999) On a class of second order variational problems with constraints, Israel J. Math.: 111, 1–28. 51. Marcus M, Zaslavski AJ (1999) The structure of extremals of a class of second order variational problems, Ann. Inst. H. Poincaré, Anal. non linéaire: 16, 593–629. 52. Maruyama T (1981) Optimal economic growth with infinite planning time horizon, Proc. Japan Acad. Ser. A Math. Sci.: 57, 469–472. 53. McKenzie LW (1976) Turnpike theory, Econometrica: 44, 841–866. 54. Mitra T, Wan HW (1985) Some theoretical results on the economics of forestry, Review of Economics Studies: 52, 263–282. 55. Mitra T, Wan HW (1986) On the Faustmann solution to the forest management problem, Journal of Economic Theory: 40, 229–249. 56. Mordukhovich BS (1990) Minimax design for a class of distributed parameter systems, Automat. Remote Control: 50, 1333–1340. 57. Mordukhovich BS (2011) Optimal control and feedback design of state-constrained parabolic systems in uncertainly conditions, Appl. Analysis: 90, 1075–1109. 58. Mordukhovich BS, Shvartsman I (2004). Optimization and feedback control of constrained parabolic systems under uncertain perturbations, Optimal control, stabilization and nonsmooth analysis. Lecture Notes Control Inform. Sci., Springer, 121–132. 59. Moser J (1986) Minimal solutions of variational problems on a torus, Ann. Inst. H. Poincaré, Anal. non linéaire: 3, 229–272. 60. Ocana Anaya E, Cartigny P, Loisel P (2009) Singular infinite horizon calculus of variations. Applications to fisheries management, J. Nonlinear Convex Anal.: 10, 157–176. 61. Piazza A (2009) The optimal harvesting problem with a land market: a characterization of the asymptotic convergence, Economic Theory: 40, 113–138. 62. Pickenhain S, Lykina V, Wagner M (2008) On the lower semicontinuity of functionals involving Lebesgue or improper Riemann integrals in infinite horizon optimal control problems, Control Cybernet.: 37, 451–468. 63. Rapaport A, Sraidi S, Terreaux JP (2003) Optimality of greedy and sustainable policies in the management of renewable resources, Optimal Control Application and Methods: 24, 23–44. 64. Rockafellar RT (1970) Convex analysis, Princeton University Press, Princeton, NJ. 65. Rubinov AM (1984) Economic dynamics, J. Soviet Math.: 26, 1975–2012. 66. Sagara N (2018) Recursive variational problems in nonreflexive Banach spaces with an infinite horizon: an existence result, Discrete Contin. Dyn. Syst. Ser. S: 11, 1219–1232. 67. Samuelson PA (1965) A catenary turnpike theorem involving consumption and the golden rule, Amer. Econom. Rev.: 55, 486–496. 68. Samuelson PA (1976) Economics of forestry in an evolving society, Economic Inquiry: XIV, 466–492.

134

References

69. Trelat E, Zhang C, Zuazua E (2018) Optimal shape design for 2D heat equations in large time, Pure and Applied Functional Analysis: 3, 255–269. 70. von Weizsacker CC (1965) Existence of optimal programs of accumulation for an infinite horizon, Rev. Econ. Studies: 32, 85–104. 71. Zaslavski AJ (1987) Ground states in Frenkel-Kontorova model, Math. USSR Izvestiya: 29, 323–354. 72. Zaslavski AJ (1995) Optimal programs on infinite horizon 1, SIAM J. Control Optim.: 33, 1643–1660. 73. Zaslavski AJ (1995) Optimal programs on infinite horizon 2, SIAM J. Control and Optim.: 33, 1661–1686. 74. Zaslavski AJ (1999) Turnpike property for dynamic discrete time zero-sum games, Abstract and Applied Analysis: 4, 21–48. 75. Zaslavski AJ (2005) The turnpike property of discrete-time control problems arising in economic dynamics, Discrete and Continuous Dynamical Systems, B: 5, 2005, 861–880. 76. Zaslavski AJ (2006) Turnpike properties in the calculus of variations and optimal control. Springer, New York. 77. Zaslavski AJ (2007) Turnpike results for a discrete-time optimal control systems arising in economic dynamics, Nonlinear Anal.: 67, 2024–2049. 78. Zaslavski AJ (2009) Two turnpike results for a discrete-time optimal control systems, Nonlinear Anal.: 71, 902–909. 79. Zaslavski AJ (2010) Stability of a turnpike phenomenon for a discrete-time optimal control systems, Journal of Optimization theory and Applications: 145, 597–612. 80. Zaslavski AJ (2010) On a class of infinite horizon optimal control problems, Communications in Mathematical Analysis: 9, 1–11. 81. Zaslavski AJ (2010) Good programs for a class of infinite horizon optimal control problems, Communications on Applied Nonlinear Analysis: 17, 1–14. 82. Zaslavski AJ (2011) The existence and structure of approximate solutions of dynamic discrete time zero-sum games, Journal of Nonlinear and Convex Analysis: 12, 49–68. 83. Zaslavski AJ (2011) Turnpike properties of solutions for a class of optimal control problems with applications to a forest management problem, Dynamics of Continuous, Discrete and Impulsive Systems, Ser. B Appl. Algorithms: 18, 399–434. 84. Zaslavski AJ (2014) Turnpike phenomenon and infinite horizon optimal control. Springer Optimization and Its Applications, New York. 85. Zaslavski AJ (2016) Structure of solutions of optimal control problems on large intervals: a survey of recent results. Pure and Applied Functional Analysis: 1, 123–158. 86. Zaslavski AJ (2017) Discrete-Time optimal control and games on large intervals. Springer Optimization and Its Applications, Springer, 2017. 87. Zaslavski AJ, Leizarowitz A (1997) Optimal solutions of linear control systems with nonperiodic integrands, Math. Op. Res.: 22, 726–746.

Index

A Agreeable program, 58–62 Approximate solution, 6, 8, 9, 33, 35, 36, 73, 74, 82, 90, 99 Asymptotic turnpike property, 65 Autonomous discrete-time control system, 8

B Bad program, 35, 65 Baire category approach, 63 Banach space, 67, 89, 98 Bolza problem, 89–90, 98–99

E Euclidean norm, 1 Euclidean space, 1 F Forest management problem, 9, 11–16, 18, 20, 21, 30–31, 33, 35, 64, 109–130 G Generic turnpike property, 2, 6, 8, 9, 63–71 Good program, 35, 36, 65, 66, 68, 71, 74, 124, 128, 130

C Cardinality of a set, 6, 35 Compact metric space, 8, 16, 33, 63, 73 Compact set, 13, 111 Complete metric space, 63, 67 Concave function, 13, 20, 118, 119 Constrained problem, 8 Continuous-time problem, 1 Control system, 2, 8, 34, 73, 111 Convex discrete-time problems, 1–7 Convex problems, 1 Convex set, 119

I Increasing function, 124 Infinite horizon, 9, 11–31, 33 Infinite horizon optimal control problem, 11–31 Inner product, 1 Interior point, 9

D Differentiable function, 2, 3, 8 Discrete-time problem, 1–8

N Neumann path, 2 Normed space, 20

L Lagrange problem, 78–83 Locally optimal program, 37, 50, 58, 78

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2019 A. J. Zaslavski, Optimal Control Problems Arising in Forest Management, SpringerBriefs in Optimization, https://doi.org/10.1007/978-3-030-23587-1

135

136 O Objective function, 2, 8, 63, 73 Optimal control problem, 1, 2, 8, 9, 11–31, 33, 63, 73, 82, 123 Optimality criterion, 8, 20, 28, 33, 37, 73 Overtaking optimal program, 28, 50, 74, 75, 77, 78, 82–86, 90–94, 98–100, 102–104, 108

P Program, 8, 12–19, 21–23, 26–30, 33–45, 47–56, 58–61, 63–71, 73–87, 89–95, 98–100, 102–105, 108, 110–112, 114, 115, 117–120, 124, 128–130

Index S Smooth function, 13 Strictly convex function, 2, 3, 13 Strongly agreeable program, 58 T Turnpike, 7–9, 73 Turnpike phenomenon, 8–9, 33 Turnpike property, 1, 2, 6, 8, 9, 33–71 Turnpike result, 2, 7, 8, 33, 55, 65, 118–122 U Unconstrained problem, 8 Upper semicontinuous function, 8, 13, 28, 31, 53, 63, 73, 90, 98, 99, 111