Universal Theory for Strong Limit Theorems of Probability 9811212821, 9789811212826

Table of contents :
Contents
Preface
Acronyms
1. Strong Laws and Large Deviations
1.1 Strong Limit Theorems of Probability Theory: Results, Problems and Methods
1.2 The Universal Strong Laws and the Large Deviations Method
2. Large Deviations for Sums of Independent Random Variables
2.1 Probabilities of Large Deviations
2.2 The Method of Conjugate Distributions
2.3 Completely Asymmetric Stable Laws with Exponent α >1
2.4 Functions of Large Deviations Theory and a Classification
of Probability Distributions
2.5 Large Deviations and a Non-Invariance
2.6 Methods of Conjugate Distributions and Truncations
2.7 Asymptotic Expansions of Functions of Large Deviations
Theory in Case of Finite Variations
2.8 Large Deviations in Case of Finite Variations
2.9 Asymptotic Expansions of Functions of Large Deviations
Theory for D(2)
2.10 Asymptotic Expansions of Functions of Large Deviations
Theory for DN(α) and D(α)
2.11 Large Deviations for D(2)
2.12 Large Deviations for DN(α) and D(α)
2.13 Large Deviations and the Classification of Distributions
2.14 Bibliographical Notes
3. Strong Limit Theorems for Sums of Independent Random Variables
3.1 Norming Sequences in Strong Limit Theorems
3.2 Universal Strong Laws in Case of Finite Exponential
Moments
3.3 Universal Strong Laws for Random Variables without
Exponential Moment
3.4 Corollaries of the Universal Strong Laws
3.4.1 The Erdos–Renyi and Shepp Laws
3.4.2 The Csorgo–Revesz Laws
3.4.3 The Law of the Iterated Logarithm
3.4.4 The Strong Law of Large Numbers
3.4.5 Results for Moduli of Increments of Sums of Independent
Random Variables
3.5 Optimality of Moment Assumptions
3.6 Necessary and Sufficient Conditions for the Csorgo–Revesz Laws
3.7 Bibliographical Notes
4. Strong Limit Theorems for Processes with Independent Increments
4.1 The Universal Strong Laws for Processes with Independent
Increments
4.2 Strong Laws for Increments of Wiener and Stable
Processes without Positive Jumps
4.3 Applications of the Universal Strong Laws
4.4 Compound Poisson Processes
4.5 Bibliographical Notes
5. Strong Limit Theorems for Renewal Processes
5.1 The universal Strong Laws for Renewal Processes
5.2 Corollaries of the Universal Strong Laws
5.3 Bibliographical Notes
6. Increments of Sums of Independent Random Variables over Head Runs and Monotone Blocks
6.1 Head Runs and Monotone Blocks
6.2 Increments of Sums over Head Runs and Monotone
Blocks
6.3 The Universal Strong Laws
6.4 Corollaries of the Universal Strong Laws
6.5 Bibliographical Notes
Bibliography
Author Index
General Index

Citation preview

UNIVERSAL THEORY FOR STRONG LIMIT THEOREMS OF PROBABILITY

b2530   International Strategic Relations and China’s National Security: World at the Crossroads

This page intentionally left blank

b2530_FM.indd 6

01-Sep-16 11:03:06 AM

UNIVERSAL THEORY FOR STRONG LIMIT THEOREMS OF PROBABILITY

A .N .Frolov St. Petersburg State Universit y, Russia

World Scientific NEW JERSEY

•

LONDON

•

SINGAPORE

•

BEIJING

•

SHANGHAI

•

HONG KONG

•

TAIPEI

•

CHENNAI

•

TOKYO

Published by World Scientific Publishing Co. Pte. Ltd. 5 Toh Tuck Link, Singapore 596224 USA office: 27 Warren Street, Suite 401-402, Hackensack, NJ 07601 UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE

Library of Congress Control Number: 2019044572 British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library.

UNIVERSAL  THEORY  FOR  STRONG  LIMIT  THEOREMS  OF  PROBABILITY Copyright © 2020 by World Scientific Publishing Co. Pte. Ltd. All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the publisher.

For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher.

ISBN 978-981-121-282-6

For any available supplementary material, please visit https://www.worldscientific.com/worldscibooks/10.1142/11625#t=suppl

Printed in Singapore

RokTing - 11625 - Universal Theory.indd 1

01-10-19 11:11:57 AM

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Dedicated to the memory of my parents

v

9789811212826

page v

b2530   International Strategic Relations and China’s National Security: World at the Crossroads

This page intentionally left blank

b2530_FM.indd 6

01-Sep-16 11:03:06 AM

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Preface

Limit theorems form an immanent part of probability and statistics. Strong limit theorems for sums of independent random variables are of essential interest in there. In particular, the strong law of large numbers provides the almost surely convergence of frequencies of events to their probabilities. Therefore, the abstract probability theory based on Kolmogorov’s axiomatic approach describes laws of nature. In statistics, it yields desired properties of estimators and tests. A significant attention of investigators was first paid to the strong law of large numbers and the law of the iterated logarithm. Later various results for increments of sums of independent random variables have been derived. The Kolmogorov strong law of large numbers, the Hartman–Wintner law of the iterated logarithm, the Erd˝os–R´enyi and Shepp laws and the Cs¨ org˝ o–R´ev´esz laws are famous achievements for the case of independent, identically distributed random variables. Numerous generalizations and augments of these results have further been obtained in hundreds papers published on this subject for now. In this book, we present a unified approach to strong limit theorems. It turns out that the results mentioned above are partial cases of general laws which we call universal strong laws. Besides sums of independent, identically distributed random variables, we derive such laws for processes with independent increments and renewal processes as well. We also concern with strong laws for some maxima from sums of independent random variables over head runs and monotone blocks. Note that an analysis of probabilities of large deviations is the best tool to prove strong limit theorems. Hence, the text includes corresponding methods and results of the large deviations theory which are of independent interest. Finally, mention that similar universal theories exist for independent, non-identically vii

page vii

September 13, 2019

viii

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

distributed random variables, random fields and some classes of stochastic processes. The book is organized as follows. In Chapter 1, we start with a survey of results, methods and problems related to strong limit theorems. Further, we discuss relationships between universal strong laws and large deviations. Chapter 2 is devoted to the large deviations theory for sums of independent, identically distributed random variables. We discuss methods of conjugate distributions and truncations and derive logarithmic asymptotics for large deviations probabilities. We first consider random variables with a finite variation. Further, we turn to random variables from domains of attraction of the normal law and completely asymmetric stable laws with exponent from (1, 2). In Chapter 3, we describe a universal theory of strong limit theorems for sums of independent, identically distributed random variables. We start with universal strong laws. From them, we derive the Erd˝os–R´enyi and Shepp laws, the Cs¨org˝ o–R´ev´esz laws, the law of the iterated logarithm and the strong law of large numbers. We consider random variables with finite variations and those from domains of attraction mentioned above. Finally, we establish an optimality of one-sided moment assumptions which we deal with. In Chapters 4 and 5, we describe universal theories for homogeneous processes with independent increments and renewal processes, correspondingly. In Chapter 6, the universal theory is discussed for increments of sums of independent random variables over head runs and monotone blocks. In there, we deal with functionals those are of interest in game settings. We follow the pattern of Chapter 3 in Chapters 4–6. Staring with the universal laws, we derive all spectrum of strong limit theorems from them. In fact, we show that there are universal laws which manage the behaviours of increments for a variety of random sequences and processes. I hope that this monograph will be useful for scientists, teachers and students being interested in probability theory. I thank very much all my colleagues from Chair of Probability Theory and Mathematical Statistic of Saint–Petersburg State University and, especially, Prof. V.V.Petrov, Prof. A.I.Martikainen and Prof. V.B.Nevzorov for support and cooperation in many years. St.Petersburg, September, 2018

Andrei Frolov Professor of Saint–Petersburg State University

page viii

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Acronyms

i.i.d. independent, identically distributed d.f. distribution function c.f. characteristic function m.g.f. moment generating function a.s. almost surely w.p. 1 with probability 1 i.o. infinitely often LDT large deviations theory SLLN strong law of large numbers LIL law of the iterated logarithm CLT central limit theorem end of a proof R the set of real numbers N the set of natural numbers an = o(bn ) means that an /bn → 0 an = O(bn ) means that lim sup |an |/bn < ∞ an ∼ bn means that an /bn → 1 IB (x) the indicator of a Borel set B IB the indicator of an event B DX the variation of a random variable X DN (α) domain of normal attraction of the asymmetric stable law with exponent α>1 D(α) domain of non-normal attraction of the asymmetric stable law with exponent α > 1 ix

page ix

September 13, 2019

x

15:48

Universal theory for strong limit theorems of probability – 11625

Universal theory for strong limit theorems of probability

SVa the set of slowly varying at a functions RVa the set of regularly varying at a functions f −1 (x) the inverse function to f (x) #B the number of elements of a finite set B [x] the integer part of x

9789811212826

page x

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Contents

Preface

vii

Acronyms

ix

1.

Strong Laws and Large Deviations 1.1 1.2

2.

1

Strong Limit Theorems of Probability Theory: Results, Problems and Methods . . . . . . . . . . . . . . . . . . . . 1 The Universal Strong Laws and the Large Deviations Method 15

Large Deviations for Sums of Independent Random Variables

27

2.1 2.2 2.3 2.4

27 28 31

2.5 2.6 2.7 2.8 2.9 2.10 2.11 2.12 2.13

Probabilities of Large Deviations . . . . . . . . . . . . . . The Method of Conjugate Distributions . . . . . . . . . . Completely Asymmetric Stable Laws with Exponent α > 1 Functions of Large Deviations Theory and a Classification of Probability Distributions . . . . . . . . . . . . . . . . . Large Deviations and a Non-Invariance . . . . . . . . . . . Methods of Conjugate Distributions and Truncations . . . Asymptotic Expansions of Functions of Large Deviations Theory in Case of Finite Variations . . . . . . . . . . . . Large Deviations in Case of Finite Variations . . . . . . . Asymptotic Expansions of Functions of Large Deviations Theory for D(2) . . . . . . . . . . . . . . . . . . . . . . . Asymptotic Expansions of Functions of Large Deviations Theory for DN (α) and D(α) . . . . . . . . . . . . . . . . Large Deviations for D(2) . . . . . . . . . . . . . . . . . . Large Deviations for DN (α) and D(α) . . . . . . . . . . Large Deviations and the Classification of Distributions . xi

33 37 38 41 46 52 56 61 68 71

page xi

September 13, 2019

15:48

xii

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

2.14 Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . 3.

Strong Limit Theorems for Sums of Independent Random Variables 77 3.1 3.2 3.3 3.4

3.5 3.6 3.7 4.

4.2 4.3 4.4 4.5

The Universal Strong Laws for Processes with Independent Increments . . . . . . . . . . . . . . . . . . . . . . . . . . . Strong Laws for Increments of Wiener and Stable Processes without Positive Jumps . . . . . . . . . . . . . . Applications of the Universal Strong Laws . . . . . . . . . Compound Poisson Processes . . . . . . . . . . . . . . . . Bibliographical Notes . . . . . . . . . . . . . . . . . . . . .

Strong Limit Theorems for Renewal Processes 5.1 5.2 5.3

6.

Norming Sequences in Strong Limit Theorems . . . . . . . Universal Strong Laws in Case of Finite Exponential Moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Universal Strong Laws for Random Variables without Exponential Moment . . . . . . . . . . . . . . . . . . . . . . Corollaries of the Universal Strong Laws . . . . . . . . . . 3.4.1 The Erd˝os–R´enyi and Shepp Laws . . . . . . . . . 3.4.2 The Cs¨org˝o–R´ev´esz Laws . . . . . . . . . . . . . . 3.4.3 The Law of the Iterated Logarithm . . . . . . . . 3.4.4 The Strong Law of Large Numbers . . . . . . . . . 3.4.5 Results for Moduli of Increments of Sums of Independent Random Variables . . . . . . . . . . . . . Optimality of Moment Assumptions . . . . . . . . . . . . Necessary and Sufficient Conditions for the Cs¨ org˝o–R´ev´esz Laws . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bibliographical Notes . . . . . . . . . . . . . . . . . . . . .

77 79 84 87 88 89 98 101 103 105 111 115

Strong Limit Theorems for Processes with Independent Increments119 4.1

5.

75

126 127 133 135 137

The universal Strong Laws for Renewal Processes . . . . . 137 Corollaries of the Universal Strong Laws . . . . . . . . . . 146 Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . 151

Increments of Sums of Independent Random Variables over Head Runs and Monotone Blocks 6.1 6.2

119

153

Head Runs and Monotone Blocks . . . . . . . . . . . . . . 153 Increments of Sums over Head Runs and Monotone Blocks 155

page xii

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Contents

6.3 6.4 6.5

9789811212826

xiii

The Universal Strong Laws . . . . . . . . . . . . . . . . . 157 Corollaries of the Universal Strong Laws . . . . . . . . . . 168 Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . 174

Bibliography

175

Author Index

185

General Index

187

page xiii

b2530   International Strategic Relations and China’s National Security: World at the Crossroads

This page intentionally left blank

b2530_FM.indd 6

01-Sep-16 11:03:06 AM

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Chapter 1

Strong Laws and Large Deviations

Abstract.We start with a survey of strong laws and theory of large deviations in probability and statistics. Corresponding methods are discussed as well. Further, we give a universal approach to strong limit theorems that includes the SLLN, the LIL, the Erd˝os–R´enyi law, the Shepp law and the Cs¨ org˝o–R´ev´esz laws. We prove universal strong laws without a specification of norming sequences by an application of a large deviations method.

1.1

Strong Limit Theorems of Probability Theory: Results, Problems and Methods

In probability, one deals with mathematical models of random experiments (or observations) those are defined by two properties. First, all possible results of such experiment are known, but the result of a single one can not be predicted. Second, the empirical law of stability of frequencies is observed in long series of these experiments. For example, one can not predict the result of a single coin tossing, but one can see that the numbers of occurrences of heads and tails are relatively close after ten thousands of a symmetric coin tossing. So, frequencies of head (ratios of head numbers to numbers of experiments) converge to 0.5 as numbers of coin tossing tend to infinity. This is the law of stability of frequencies in our example. The probability theory is a mathematical one and, consequently, it is based on axioms similarly to Euclidean geometry, for instance. Now Kolmogorov’s axiomatic approach is usually applied. Hence, mathematical models of random experiments are measurable spaces with probability measures. To be sure that our models are relevant, we need a theorem corresponding to the empirical law of stability of frequencies. This is the strong law of large numbers (SLLN). 1

page 1

September 13, 2019

2

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Let X, X1 , X2 , . . . be a sequence of independent, identically distributed (i.i.d.) random variables with a finite mean. Put S0 = 0 and Sn = X 1 + X 2 + · · · + X n . By the Kolmogorov SLLN, we have Sn → EX a.s. (1.1) n Here and in the sequel, all limits are taken as n → ∞ if it is not pointed otherwise and a.s. means almost surely. Note that Sn /n → c a.s. implies E|X| < ∞. E.Borel has first proved SLLN for Bernoulli trails when P (X = 1) = 1 − P (X = 0) = p ∈ (0, 1). If p = 0.5 and Xi is the number of heads in i-th repetition of a coin tossing, then (1.1) corresponds to the law of stability of frequencies in the above example. Moreover, since Sn /n is the frequency of head (or tail), sums Sn are basic objects of the theory. Applying the SLLN in the numbers theory, E.Borel has also proved the normality of almost all (with respect to the Lebesgue measure) real numbers (see [Feller (1971)] and [Lamperti (1996)] for details). In statistics, one checks correspondences of probabilistic models and collections of data obtained from random experiments. For example, one could check hypothesis of symmetry of the coin using results of ten thousands of coin tossing. Strong laws imply properties of estimators and statistics which statistical tests are based on. The Borel SLLN yields the following Glivenko–Cantelli theorem: sup |Fn (x) − F (x)| → 0

a.s.,

x∈R

where Fn (x) is the empirical distribution function (d.f.) for sample X1 , . . . , Xn and F (x) is the d.f. of X. (See [Lo`eve (1963)] for details.) Unbounded growths of chi square statistics under alternative hypotheses follow from the Borel SLLN and define the structure of the test. The Kolmogorov SLLN yields the latter for all tests based on statistics those are sums of i.i.d. random variables. Moreover, the Kolmogorov SLLN provides the consistency of empirical moments, for example, that of the empirical mean and variation. Hence, SLLN is a basis of statistics as well. The next famous a.s. result is the law of the iterated logarithm (LIL). By the Hartman–Wintner LIL, if EX = 0 and EX 2 = 1, then lim sup √

Sn =1 2n log log n

a.s.

(1.2)

page 2

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Strong Laws and Large Deviations

9789811212826

3

Note that one can replace Sn by |Sn | in relation (1.2). Later, it was independently established in [Martikainen (1980)], [Rosalsky (1980)] and [Pruitt (1981)] that EX = 0 and EX 2 = 1 are also necessary conditions for (1.2). For the Bernoulli case with P (X = −1) = 1 − P (X = 1) = 0.5, the SLLN yields Sn = o(n) a.s. Further a.s. bounds for Sn have been obtained consequently by F.Haussdorff, G.Hardy and J.Littlewood and, finally, by A.Khinchine who has proved LIL. In particular, they also concerned with Rademaher functions those are i.i.d Bernoulli random variables on [0, 1] with the Lebesgue measure. (See [Lamperti (1996)] for details.) The Hartman–Wintner LIL yields the following property of trajectories of random walks {Sn }: with probability 1 (w.p. 1), for every ε > 0 the √ inequalities |Sn | ≤ (1 + ε) 2n log log n hold for all sufficiently large n. The latter gives bounds for convergence rates of statistical estimators to parameters. For the sample mean X = S n /n, it follows that w.p. 1, for every ε > 0 we get |X − EX| ≤ (1 + ε) 2 log log n/n for all sufficiently large n. Detailed discussions on the SLLN and LIL, a history and references may be found in [Petrov (1975, 1987, 1995)]. Relation (1.2) fails when EX 2 = ∞. Nevertheless, similar results may hold true for X from domain of attraction of a completely asymmetric stable law with exponent α ∈ (1, 2]. For α = 2, it is the Gaussian distribution. Remember that stable laws are limit distributions for normalized sums Sn with respect to weak convergence. The simplest case is the L´evy central limit theorems (CLT). LIL (1.2) is related with the asymptotic normality √ of Sn / n. A similar situation holds for non-Gaussian case. Distributions of normalized Sn are close or coincide with a stable law. LIL for this case may be found in [Mijnheer (1974)] and references therein. For X from the domain of normal attraction of the completely asymmetric stable law with α ∈ (1, 2) and EX = 0, the LIL holds with the norming cn1/α (log log n)(α−1)/α which turns to that from (1.2) for α = 2. But take into account that in the completely asymmetric stable case with α < 2, the LIL does not hold for |Sn | while LIL (1.2) holds. Nevertheless, the completely asymmetric stable laws are very similar to the normal law. They are only stable laws having one-sided exponential moments that yields this similarity. Hence, we concern with such stable laws in what follows. Random variables with distributions from the domains of attractions of completely asymmetric stable laws appear in many theoretical and practical problems both. For example, X with the Pareto d.f. F (x) = (−x)−α , x < −1, belongs to the domain of attraction of the completely asymmet-

page 3

September 13, 2019

4

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

ric stable law. The Pareto distribution is widely used in economics. It describes amounts of claims in some risk models of actuarial and financial mathematics, for example. In here, X is negative since every satisfied claim is a loss for an insurance company. Further properties of trajectories of random walks are related with increments of sums Sn . Define functionals on these trajectories those we deal with. Let {an } be a non-decreasing sequence of natural numbers such that 1 ≤ an ≤ n for all n. Denote Un = Wn =

max

0≤k≤n−an

max

(Sk+an − Sk ), max (Sk+j − Sk ),

0≤k≤n−an 1≤j≤an

Rn = Sn − Sn−an ,

Tn = Sn+an − Sn .

Maxima Un and Wn have a clear sense in game settings when Xk is a gain of a player in k-th repetition of a game (negative gains are losses). Then Un and Wn are the maximal gains of the player over successive series of repetitions of the game with length an and less than or equal to an , correspondingly. If n is regarded as time, then Un and Wn are the maximal gains in time an and less than or equal to an . Increments Rn and Tn describe the evolution of a capital of the player as well. Thus, Un , Wn , Rn and Tn are natural objects of the probability theory. We see that Un = Rn = Sn

and Wn = max Sk , 1≤k≤n

when an = n for all n. If an = 1 for all n, then Un = Wn = max Xk . 1≤k≤n

Hence, the asymptotic behaviour of increments is close to that of sums Sn for large lengths an and that of maxima of i.i.d. random variables for small an . We built a universal theory (for all lengths an ) which include results for sums, their increments and maxima of i.i.d. random variables. Describe now the a.s. behaviour of increments of sums Sn . First results of this type (the Erd˝ os–R´enyi and Shepp laws) have been proved in [Shepp (1964)] and [Erd˝os and R´enyi (1970)]. They have established that for X with an exponential moment and an = [c log n], relation   Tn 1 Un = lim sup =γ lim a.s. (1.3) an an c

page 4

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

5

Strong Laws and Large Deviations

holds. Here γ(x) is the inverse function to the function of large deviations which we define in this section later. Now we postpone a discussion of properties of γ(x) to Chapter 2. We only mention that γ(x) depends on full distribution of X and determines this distribution sometimes. So, this result shows that trajectories of Sn can “remember” the distribution of X. This yields a variant of solution for the stochastic geyser problem: can Robinson Crusoe find a distribution of time between two successive eruptions of a geyser on his island? The answer is yes provided he knows org˝ o and R´ev´esz (1981)] for details). the trajectory of Sn (see, [Cs¨ Note that [Cs¨org˝ o (1979)] has proved a one-sided generalization of the Erd˝ os–R´enyi law. [Deheuvels and Devroye (1987)] have obtained (1.3) for c ∈ (0, c0 ) (see Chapter 2 for a definition of c0 ). [Steinebach (1978)] and [Lynch (1983)] have proved that exponential moment conditions are necessary for (1.3). [Mason (1989)] has obtained an extension of the Erd˝ os–R´enyi law for an / log n → 0. It turns out in this case that lim sup

Un = 1 a.s. an γ(log n/an )

(1.4)

and one can replace lim sup by lim for a class of distributions containing normal and exponential ones. One can also replace Un by Tn in (1.4). This implies that if an = m for all n and P (X = −1) = P (X = 1) = 0.5, then γ(∞) = 1 and lim sup

Xn+1 + · · · + Xn+m =1 m

a.s.

So, the upper limit for m-period moving averages (Xn+1 +· · ·+Xn+m )/m is not EX even for large, but fixed m. Note that moving averages are widely used in analysis of processes in economics on macro and micro levels both. In particular, trading terminals for Forex or stock markets allow to plot various moving averages over bars charts. The case an = 1 for all n in (1.4) turns us to the asymptotic theory of extreme order statistics. Norming sequences in strong laws for maxima may depend on the √distribution of X. If X has the standard normal distribution, then γ(x) = 2x and lim √

1 max Xk = 1 2 log n 1≤k≤n

a.s.

√ For standard exponential X, the latter holds with log n instead of 2 log n. From the other side, if F (x) = 1 − 1/x, x ≥ 1, then for every increasing

page 5

September 13, 2019

6

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

sequence {bn } of positive constants, one has lim sup

1 max Xk = 0 bn 1≤k≤n

or ∞ a.s.

and (1.4) fails. Further discussion on the asymptotic behaviour of extremes may be found in [Galambos (1978)], for example. M.Cs¨org˝o and P.R´ev´esz have proved that if EX = 0, EX 2 = 1, X has the two-sided exponential moment and an / log n → ∞, then Un lim sup  =1 2an (log(n/an ) + log log n)

a.s.

(1.5)

and one can replace lim sup by lim in (1.5) provided log(n/an )/ log log n → ∞ in addition (see [Cs¨ org˝ o and R´ev´esz (1981)]). For an = n, we have LIL (1.2). If log(n/an ) ∼ c log log n, then we get the LIL with a certain constant instead of 1. [Cs¨ org˝ o and R´ev´esz (1981)] have also discussed weaker twosided moment assumptions. Note that the moment assumption defines the minimal rate of the growth of an which relation (1.5) may hold for. One can find statistical applications in [Cs¨ org˝ o and R´ev´esz (1981)]. [Frolov (1990, 2000, 1998, 2002c, 2003b,d)] has obtained one-sided generalizations of the Cs¨ org˝ o–R´ev´esz laws. Note that the behaviour of Wn is the same as that of Un while for Rn and Tn , lim sup results hold only. Moreover, if increments of sums are replaced by moduli of increments in the definition of Un , Wn , Rn and Tn , then the results will be the same. The norming in (1.5) is the same for all distributions. Hence, large increments “forget” the distribution of X and “remember” its several moments only. This is a strong invariance while a strong non-invariance appears in (1.3). One of the main goals of every science is a systematization. We see a series of results for sums Sn and their increments. We show that they may be written in one scheme and proved simultaneously. More precisely, our main goal is to describe the a.s. asymptotic behaviour of Un , Wn , Rn and Tn as n → ∞. We will find sequences of positive numbers {bn } and sufficient and/or necessary conditions for either lim sup

Un = 1 a.s., bn

(1.6)

or lim

Un = 1 a.s. bn

(1.7)

page 6

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

7

Strong Laws and Large Deviations

Remember that all limits are taken as n → ∞ if it is not pointed otherwise. Of course, we will consider analogues of (1.6) and (1.7) for Wn , Rn and Tn as well. We see (1.6) and (1.7) in partial cases mentioned above. If an = n and bn = nEX for all n with EX > 0, then relation (1.7) turns to SLLN (1.1). (Note that one can prove (1.1) for EX > 0 only. The general case easily follows by a centering of summands at 2EX.) When an = n √ and bn = 2n log log n for all n, relation (1.6) turns to LIL (1.2). For an = [c log n] and bn = an γ(1/c), relation (1.7) is (1.3). If an = o(log n) and bn = an γ(log n/an ), relation (1.6) coincides with (1.4). The case an = 1 for all n is in the results for extreme order statistics mentioned above. One can prove strong laws by various methods. M.Cs¨org˝ o and P.R´ev´esz have used the strong approximation of sums by a Wiener process. This method is based on the following fact: on a certain probability space, one can define a sequence of sums {Sn } of i.i.d. random variables and a Wiener process such that trajectories of sums and the process are closed enough. Then one can derive results for sums from results for the Wiener process. If, for example, EX = 0, EX 2 = 1 and X has the two-sided exponential moment, then on a certain probability space, one can define {Sn } and the standard Wiener process W (t) such that Sn − W (n) = O(log n) a.s.

(1.8)

This approximation is also referred in literature as Koml´ os–Major– Tushn´ady strong approximation. Note that the Erd˝os–R´enyi law implies that O can not be replaced by o in the last relations. One can prove that if (1.8) holds with o instead of O, then X has the standard normal distribution. Assume that aT is a positive, non-decreasing function such that aT ≤ T and T /aT is non-decreasing. Suppose that w.p. 1, the process W (t) has continuous trajectories. For increments of W (t), M.Cs¨org˝ o and P.R´ev´esz have proved that sup

0≤t≤T −aT

(W (t + aT ) − W (t))

=1 lim sup  2aT (log(T /aT ) + log log T ) T →∞

a.s.

(1.9)

One can replace lim sup by lim in (1.9) provided log(T /aT )/ log log T → ∞ as T → ∞ (see [Cs¨org˝ o and R´ev´esz (1981)]). Relation (1.9) holds for sup

sup (W (t + s) − W (t)) ,

0≤t≤T −aT 0≤s≤aT

page 7

September 13, 2019

8

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

W (T ) − W (T − aT ) and W (T + aT ) − W (T ) instead of the supremum in the left-hand side. For the last supremum, one can replace lim sup by lim in (1.9) when log(T /aT )/ log log T → ∞ as T → ∞ Note that these results include the Erd˝ os–R´enyi law and the LIL for increments of the Wiener process. In particular, relation (1.9) for aT = T turns to the LIL W (T ) = 1 a.s. (1.10) lim sup √ 2T log log T T →∞ Moreover, for aT = 1, we get a continuous analogue of the above result for maxima of i.i.d. standard normal random variables from the theory of extreme order statistics. Since −W (t) is the standard Wiener process as well, the above results hold for moduli of increments of the Wiener process. Relations (1.9) and (1.8) imply (1.5) for {an } such that an / log n → ∞. Further results for increments of sums follow in the same way. For X without the exponential moment, the techniques is the same, but the accuracy in (1.8) decreases. This yields results for a more narrow range of {an } as mentioned after relation (1.5). The approximation like (1.8) does not work in general case. It can not be applied when an = O(log n) (the non-invariance) and EX 2 = ∞. For EX 2 < ∞, the accuracy of such approximation is not enough to provide optimal results under minimal assumptions on the left tail of the distribution of X. Hence, we use the method of an analysis of probabilities of large deviations or, briefly, the large deviations method. Note that the SLLN, the LIL, the Shepp law, the Erd˝ os–R´enyi law and its extension have been proved in this way. To prove relations (1.6) and (1.7), the large deviations method will be applied as follows. Since Rn ≤ Un ≤ Wn , we derive the upper bound for Wn and the lower bounds for either Rn (in case of lim sup), or Un (in case of lim). For the upper bounds, we will estimate the probabilities Pn = P (Wn ≥ (1+ε)bn ) from above by probabilities of large deviations P (San ≥ (1+ε)bn ) with some coefficients. Appropriate bounds for the last probability imply  that the series k Pnk converges for a special subsequence {nk } of natural numbers with nk → ∞ as k → ∞. The Borel–Cantelli lemma yields Wnk lim sup ≤ 1 + ε a.s. bnk k→∞ Monotonicity of Wn , regularity conditions on {bn } and limit passing as ε → 0 imply the upper bound.

page 8

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Strong Laws and Large Deviations

9789811212826

9

Turn to the lower bound in (1.6). If an /n → c < 1, then we choose a sequence of natural numbers {nk } such that nk → ∞ as k → ∞ and the events {Rnk ≥ (1 − ε)bnk } are independent. The divergence of the series from probabilities of these events is provided by appropriate asymptotics of probabilities of large deviations P (San ≥ (1 − ε)bn ). Finally, the Borel– Cantelli lemma and limit passing as ε → 0 yield the lower bound in (1.6). If an /n → 1, then we put nk = ck for all k ∈ N and prove that the series from probabilities of independent events {Snk − Snk+1 ≥ (1 − ε)bnk } converges. An application of a special analogue of the Borel–Cantelli lemma gives the result. Here, the asymptotics of large deviations of Sn is used as well. Lower bound in (1.7) is derived in a similar way. Making use of asymptotics for probabilities of large deviations of sums Sn , we prove that for  every ε > 0, the series n P (Un < (1 − ε)bn ) converges. This and the Borel–Cantelli lemma imply the result. Note that there are no specifications for {bn } in the above scheme. Hence, we first derive general results for arbitrary sequences {bn } and further find formulae for them from results on large deviations. Thus asymptotics of probabilities of large deviations of sums of i.i.d. random variables play a key role below. We discuss the large deviations theory (LDT) in Chapter 2 that is of essential independent interest. Probabilities of large deviations play an important role in statistics and various applications. Critical domains of statistical tests are often defined by large deviations of corresponding statistics. Many statistics coincide or may be approximated by sums of i.i.d. random variables. So, results for them easily follow. We deal with the asymptotic behaviour of ln P (Sn ≥ nxn ), where {xn } is a sequence of positive numbers such that xn = O(1). In LDT, a significant place takes the large deviations function which we introduce as follows. Assume that X has the one-sided exponential moment. By the Tchebyshev inequality, we get P (Sn ≥ nx) ≤ e−nζ(x) , where ζ(x) =

sup h≥0: EehX 0, we get bn ∼ an γ(0) = an EX √that is the norming for SLLN. If EX = 0 and EX 2 = 1, then γ(x) ∼ 2x as x → 0 and bn is equivalent to the norming from the Cs¨ org˝o–R´ev´esz laws. Similar results hold for domains of attractions of the normal law and the completely asymmetric stable laws with α ∈ (1, 2). For X without the exponential moment, γ(x) in the above definition of bn can be replaced by its variant for truncated from above random variables. Then we have the Cs¨ org˝ o–R´ev´esz laws. We also have their variants for the case of domains of attractions of the normal law and the completely asymmetric stable laws with α ∈ (1, 2). Note that that results will be derived under optimal moment conditions. Besides sums of independent random variables, various classes of stochastic processes are of essential interest in probability, statistics and their applications. Below we deal with renewal processes and processes with independent increments. For a sequence of positive i.i.d. random variables {Yn }, the renewal process N (t) is defined by N (t) = max{n ≥ 0 : Rn ≤ t}, where R0 = 0, Rn = Y1 + Y2 + · · · + Yn . One of the most important case arises when Yn has the exponential distribution and N (t) is the Poisson process. In particular, N (t) describes the number of identical devices (electric bulbs, for instance) broken (and renewed) up to time t. Then {Yn } are random times of lives of the devices. In queuing theory, N (t) may be the number of customers. In actuarial and financial mathematics, N (t) may be the number of claims. We choose renewal processes by two reasons. First, they are widely used in numerous applications. Second, a duality between Rn and N (t) provides relatively simple passages from results for sums to results for renewal processes. Indeed, we have (1.11) {N (t) ≥ n} = {Rn ≤ t} for all t > 0 and natural n. This and the Kolmogorov SLLN for Rn yield the following SLLN for renewal processes: 1 N (T ) = a.s. lim T →∞ T a

page 11

September 13, 2019

12

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

provided a = EY1 exists and it is positive. Note that EN (1) is not equal to 1/a in the general case, but the equality holds for the Poisson process. In the same way, duality (1.11) and the Hartman–Wintner LIL imply the next LIL for renewal processes: N (T ) − (T /a) σ = 3/2 lim sup √ a 2T log log T T →∞

a.s.,

where a = EY1 and σ 2 = DY1 . One can see that in the last relation, N (t) is  centered at EN (T ) for the Poisson process. The norming in this case is DN (T ) provided we divide that relation by σ/a3/2 . This holds since the Poisson process is the homogeneous process with independent increments. Hence, the Kolmogorov SLLN and the Hartman–Wintner LIL hold for {N (n)} which is the sequence of sums of i.i.d. random variables with the same distribution as N (1). Below, we use a more complicated techniques which gives results for increments of renewal processes. Besides an analogue of the supremum for the Wiener process from (1.9), we also investigate the following centered variants: s

sup N (t + s) − N (t) − , sup a 0≤t≤T −aT 0≤s≤aT aT . sup (N (t + aT ) − N (t)) − a 0≤t≤T −aT We construct a universal theory of strong laws for renewal processes. From the universal law, we derive the SLLN, the LIL, the Erd˝ os–R´enyi and the Cs¨ org˝ o–R´ev´esz laws for renewal processes. In there, the results depend on centering. We get the SLLN and the Erd˝ os–R´enyi law for non-centered processes and the LIL, the Erd˝os–R´enyi law and the Cs¨ org˝o–R´ev´esz laws for centered ones. Homogeneous processes with independent increments are also of essential interest in probability, statistics and various applications. The Wiener and Poisson processes are two first examples. The Wiener process or Brownian motion is a model for a motion of a particle randomly hitting by a large number of another ones. In probability, the Wiener process is often used as an approximation for random walks. Then results for functionals from random walks may be derived by applications of those for functionals from the Wiener process. We have above demonstrated this approach to sums of i.i.d. random variables. In statistics, this approach leads to results for empirical processes, for example.

page 12

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Strong Laws and Large Deviations

9789811212826

13

Important processes with independent increments may be constructed from sums Sn and a Poisson process N (t) (independent with sums) by ξ(t) = SN (t) . This is the Compound Poisson process. Such processes are widely used in actuarial and financial mathematics as models for aggregate claim amounts of insurance companies for portfolios of insurance polices of the same type. The behaviours of homogeneous processes with independent increments and sums of i.i.d. random variables are quite similar. For example, relations (1.5) and (1.9) turn each to other when we replace max by sup, sums Sn by the standard Wiener process W (t), n by t and via versa. For these processes, some technical problems are related with continuous time, but increments are infinitely divisible. This and the importance of this class of processes are reasons for considerations below. Note that we can easily define various suprema from the Poisson process and Compound Poisson process since their trajectories are step functions. The Wiener process has a modification which trajectories are continuous w.p. 1. For a general case, we consider stochastically continuous, homogeneous process ξ(t) with independent increments such that ξ(0) = 0 a.s. and Eξ(1) < ∞. Then ξ(t) has a modification which has trajectories from the space of c´ adl` ag (right-continuous and having left limit at every point) functions on [0, ∞) w.p. 1. We only concern with such modifications. Since ξ(t) is right continuous and has left limits, we have no problems with definitions of considered functionals from trajectories of ξ(t). Stable processes are important examples of homogeneous processes with independent increments. We restrict our attention to such stable processes that ξ(1) has a completely asymmetric stable distribution with the exponent α ∈ (1, 2). It turns out that these processes only have the one-sided exponential moment and finite mean. They are very similar to the Wiener process. Of course, they have modifications with trajectories from the space of c´ adl` ag function (w.p. 1) while trajectories of the Wiener process are continuous w.p. 1. Nevertheless, analogues of a.s. results for the Wiener process hold for such stable processes as well. For example, if Eξ(1) = 0 α and ξ(1) has the m.g.f. eh /α for h > 0, then the LIL is as follows: lim sup T →∞

ξ(T ) = 1 a.s., λ−λ T 1/α (log log T )λ

where λ = (α − 1)/α. Putting formally α = 2 in the last relation, we arrive at LIL (1.10) for the Wiener process. The nature of this relationship is clear because we can consider the standard normal distribution as “completely

page 13

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

14

9789811212826

Universal theory for strong limit theorems of probability

asymmetric” stable law with α = 2. For α = 2, the parameter of symmetry in the canonic representation of the stable c.f. is absent and we can choose it arbitrarily. Note that the LIL for |ξ(t)| fails while the LIL for |W (t)| holds. We construct a universal theory of strong laws for homogeneous processes with independent increments. The Wiener process, the Compound Poisson process and the stable processes mentioned above are separately presented as important partial cases. From the universal strong laws, we derive the SLLN, the LIL, the Erd˝ os–R´enyi and the Cs¨org˝ o–R´ev´esz laws for processes with independent increments. Finally, we deal with the a.s. asymptotic behaviour of special functionals from trajectories of random walks. We first state the problem in game settings. Suppose that the sequence of sums {Sn } describes fluctuations of the capital of a player. We are interested in the behaviour of an aggregate amount of the gain of the player in series of games without losses. Formally, we arrive at the maximum Mn =

max

0≤k≤n−an

(Sk+an − Sk )I{Xk+1 ≥0, Xk+2 ≥0, ..., Xk+an ≥0} ,

where IB is the indicator of the event B. If the player pay a fee c for the participation in every repetition of the game, then we replace all zeros by c in the indicator of the event. Another interesting maximum is Mn =

max

0≤k≤n−an

(Sk+an − Sk )I{0≤Xk+1 ≤Xk+2 ≤···≤Xk+an } .

It is the gain of a player in series with successively increasing gains. If there is a fee, then we replace 0 in the indicator by the fee c again. Of course, the first question is as follows. What is the maximal length of the series in the indicators? If the sequence {an } increases very fast, then it may happen that all indicators are zeros. Hence, the maxima will be zeros and the problem is trivial. It turns out that the maximal (random) length of considered series has the logarithmic order. Nevertheless, we will construct the corresponding universal theory and will show that all theorems mentioned above holds true for those maxima. In the sequel, we deal with more general settings than before. We assume that we have an accompanying sequence of random variables which forms blocks in the indicators. Let (X, Y ), (X1 , Y1 ), (X2 , Y2 ), . . . be a sequence of i.i.d. random vectors. Note that X and Y may be dependent and the case X = Y is available as well. The sums of X’s are denoted by Sn as before. Put Mn =

max

0≤k≤n−jn

(Sk+[jn ] − Sk )I{u≤Yk+1 ≤···≤Yk+[j

n ] ≤v

},

page 14

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Strong Laws and Large Deviations

15

where u, v are fixed with −∞ ≤ u ≤ v ≤ +∞ and {jn } such that 1 ≤ jn ≤ n for all n. Here [x] is the integer part of the number x. Formally, jn may also be a random variable, but we do not touch this case in there. If u = v = 1 and Y = I{X≥0} , then Mn = Mn . If u = 0, v = +∞ and Y = X, then Mn = Mn . When Y is a Bernoulli random variable with P (Y = 1) = p = 1−P (Y = 0), p ∈ [0, 1], the series of Y ’s with Yk+1 = Yk+2 · · · = Yk+s = 1 is called a head run of length s. In this case, we mention Mn as the maximum of increments of sums over head runs. If Y is continuous, the series Yk+1 ≤ Yk+2 ≤ · · · ≤ Yk+s is called a monotone block (or an increasing run) of length s. In this case, Mn is the maximum of increments over monotone blocks. (For sake of brevity, we do not write further that sums are sums of i.i.d random variables and head runs and monotone blocks are in the accompanying sequence.) Note that there are investigations of another blocks of random variables in literature. Unfortunately, the relationship of large deviations of sums over blocks and sums of i.i.d. random variables is known for head runs and monotone blocks. So, there is a number of open questions. It turns out that the length of the longest head run in Y1 , Y2 , . . . , Yn is (asymptotically) log n/ log(1/p) a.s. while the length of the longest monotone block is log n/ log log n a.s. Therefore, we are able to prove results for increments over head runs and monotone blocks simultaneously. At the same time, there is no sense to consider jn greater than log n/ log(1/p) for head runs and log n/ log log n for monotone blocks. Nevertheless, we will prove all the spectrum of the results as for sums of i.i.d. random variables. Below, we derive the SLLN, the LIL, the Erd˝ os–R´enyi and the Cs¨org˝ o– R´ev´esz laws from the corresponding universal laws. In the next section, we establish relationships between universal theorems and large deviations results for Sn those are basic for our theory below. 1.2

The Universal Strong Laws and the Large Deviations Method

We start this section with a natural choice for the length an of increments of sums Sn . Of course, we could consider any monotone sequences of natural numbers {an }, but this only yields more complicated statements of results. In what follows, an will be the integer part of the value of a continuous function at n.

page 15

September 13, 2019

16

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Let a(x) be a non-decreasing, continuous function such that 1 ≤ a(x) ≤ x for all x ∈ R and x/a(x) is non-decreasing. For all n ∈ N, put an = [a(n)] and n + log log(max(n, 3)), βn = log an where [·] is the integer part of the number in brackets. Our first result contains an a.s. upper bound for normalized maxima Wn . Theorem 1.1. Let {bn } be a sequence of positive constants. Assume that the following conditions hold: 1) the sequence {bn } is equivalent to some non-decreasing sequence and lim lim

θ1 k→∞

b[θk+1 ] = 1. b[θk ]

(1.12)

 2) n P (X ≥ cbn ) < ∞ for some c > 0. 3) For all small enough ε > 0, there exist δ > 0 and H ≥ 0 such that  P S[(1+ε)an ] ≥ (1 + ε)bn ≤ e−(1+δ)βn + Han P (X ≥ cbn ) (1.13) for all sufficiently large n. 4) For all ε > 0 there exists q ∈ (0, 1) such that  P S[(1+ε)an ] ≥ −εbn ≥ q

(1.14)

for all sufficiently large n. Then lim sup

Wn ≤1 bn

a.s.

(1.15)

One can replace Wn by Tn in the last relation. If condition 3) holds with H = 0, then one can omit condition 2). If inequality (1.13) holds with H = 0 for all Si , 1 ≤ i ≤ [(1 + ε)an ], and log an / log n → 0, then one can omit conditions 2) and 4). Proof. We need two lemmas. The first one is a variant of the L´evy inequality. Lemma 1.1. If r, s ≥ 0, q > 0 and P (Si ≥ −s) ≥ q for all i = 1, 2, . . . , n, then   P max (Sj − Si ) ≥ r ≤ q −2 P (Sn ≥ r − 2s). 0≤i 0. Then there exists q > 0 such that P (SIn ≥ −εbn ) ≥ q for all n ≥ N0 . Suppose that n ≥ N0 and i ≤ In . If i ≥ IN0 , then there exists ni such that ni ≥ N0 and Ini ≤ i ≤ Ini + m, where m = 2 + [2ε]. Hence, P (Si ≥ −εbn ) = P (SIni + Si − SIni ≥ −εbn ) ≥ P (SIni ≥ −εbn )P (Si − SIni ≥ 0) ≥ P (SIni ≥ −εbn )(P (X ≥ 0))m ≥ P (SIni ≥ −εbni )(P (X ≥ 0))m ≥ q(P (X ≥ 0))m . If i ≤ IN0 , then P (Si ≥ −εbn ) ≥ P (Si ≥ 0) ≥ (P (X ≥ 0))IN0 > 0. Putting q  = min{q(P (X ≥ 0))m , (P (X ≥ 0))IN0 }, we get the assertion of the lemma. Turn to the proof of Theorem 1.1. Without loss of generality, we suppose that {bn } is non-decreasing. We will check that (1.15) holds for Wn∗ = max max (Sk+j − Sk ) 0≤k≤n 1≤j≤an

instead of Wn . It is clear that Wn ≤ Wn∗ and Tn ≤ Wn∗ for all n. Assume first that conditions 1)–4) are satisfied and H > 0. Take ε > 0 and denote An (ε) = {Wn∗ ≥ (1 + ε)bn }. By Lemmas 1.1 and 1.2 and relation (1.14), we have Pn = P (An (3ε)) ⎛ ⎞  [n/(εan )]+1  max (Sm+k − Sm ) ≥ (1 + 3ε)bn ⎠ ≤P⎝ max (j−1)εan ≤m≤jεan 1≤k≤an

j=1



[n/(εan )]+1

≤

 j=1

P

 max

max (Sm+k − Sm ) ≥ (1 + 3ε)bn

(j−1)εan ≤m≤jεan 1≤k≤an

  (1 + ε)n P max max (Sm+k − Sm ) ≥ (1 + 3ε)bn ≤ 0≤m≤εan 1≤k≤an εan (1 + ε)n  ≤ P S[(1+ε)an ] ≥ (1 + ε)bn εan q 2

(1.16)

page 18

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

19

Strong Laws and Large Deviations

for all sufficiently large n. Take θ > 1. For k ∈ N, put  nk = min n : θk−1 < n ≤ θk , nP (X ≥ cbn ) = Then



min

θ k−1 0 such that P (San ≥ (1 − ε)bn ) ≥ e−(1−τ )βn

(1.23)

for all sufficiently large n. If an /n → 1, then assume in addition that conditions 1) and 4) of Theorem 1.1 are satisfied. Then Rn ≥ 1 a.s. (1.24) lim sup bn One can replace Rn by Tn in the last relation. Proof. By (1.23), we have Pn = P (Rn ≥ (1 − ε)bn ) ≥ e−(1−τ )βn ≥

an (log n)−(1−τ ) n

(1.25)

for all sufficiently large n. Assume first that an /n → ∈ [0, 1). Choose δ ∈ ( , 1) and a natural number N such that inequalities (an + 1)/n < δ and (1.25) hold for all n > N . Put n1 = N and nk+1 = min {n : n > nk , n − an ≥ nk } for k ∈ N. Making use of the inequality an ≥ (an + 1)/2, we get m  k=2

Pnk

m  ank + 1 1 −1+τ ≥ (log nm ) . 2 nk k=2

page 21

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

22

9789811212826

Universal theory for strong limit theorems of probability

It yields from the inequalities nk − ank − 1 ≤ nk − ank −1 − 1 < nk−1 and − log(1 − x) ≤ Cδ x for x ∈ (0, δ) that Cδ

m m m   ank + 1  nk nk nm ≥ ≥ log log = log . nk nk − ank − 1 nk−1 n1 k=2

k=2

k=2

It follows that m 

and the series

 k

Pnk ≥ C(log nm )τ ,

k=2

Pnk diverges.

Using the independence of the events

{Rnk ≥ (1 − ε)bnk }, we arrive at the conclusion of the theorem from the Borel–Cantelli lemma. Suppose now that an /n → 1. Put nk = [θk ] for k ∈ N, where θ > 1. We will check that relation Rnk ≤ a < 1 a.s. (1.26) lim sup k→∞ bnk does not hold. To this end we will use the next result. Lemma 1.3. Let {An } and {Dn } be two sequences of events such that for every n the following pairs of events are independent: An and Dn , An and Dn An−1 Dn−1 , An and Dn An−1 Dn−1 An−2 Dn−2 , . . .  P (An ) diverges, then If the series n

P (An Dn i.o.) ≥ lim inf P (Dn ). Proof. Put p = lim inf P (Dn ) and suppose that p > 0. For all n ≥ 1 and j ≥ 0, put Unj =

n+j  k=n

Ak Dk ,

Un∞ =

∞ 

Ak Dk ,

Inj = Unj =

k=n

n+j 

Ak Dk .

k=n

Assume that the conclusion of the lemma does not hold. Then q = P (An Dn i.o.) = lim P (Un∞ ) < p. n→∞

Take δ = (p − q)/3. Then there exists N such that for every n ≥ N , the inequalities P (Dn ) ≥ p − δ and P (Unj ) ≤ q + δ hold for all j . Suppose that n ≥ N . Then P (Dn+j Inj−1 ) = P (Dn+j ) − P (Dn+j Unj−1 ) ≥ δ and P (An+j Dn+j Inj−1 ) = P (An+j )P (Dn+j Inj−1 ) ≥ δP (An+j )

page 22

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

23

Strong Laws and Large Deviations

for all j ≥ 1. It follows that P (Un∞ ) = P (An Dn ) +

∞ 

P (An+j Dn+j Inj−1 ) ≥ δ

j=1

∞ 

P (Ak ) = ∞.

k=n+1

This contradiction shows that the above assumption is not true. Let us consider the events   Ak = Snk − Snk−1 > (1 + ε)abnk and

  Dk = Snk−1 − Snk −ank > −εabnk

for k = 2, 3, . . .. The relation nk−1 − nk + ank ∼ nk−1 as k → ∞ implies that nk−1 − nk + ank ≤ nk for all sufficiently large k. Condition 4) and Lemma 1.2 yield that P (Dk ) ≥ q for all sufficiently large k. Choose mk such that amk = nk − nk−1 . This is possible because the set of values of an (as a function of n) coincides with N. The relation ank ∼ nk ∼ (nk − nk−1 )θ/(θ − 1) as k → ∞ yields that nk ∼ mk θ/(θ − 1) as k → ∞. Condition 1) implies that for every δ > 0, there exists 0 > 1 such that b[ t] ≤ (1+δ)b[t] for all 1 < < 0 and all sufficiently large t. Choosing large θ with = θ/(θ − 1) < 0 , we conclude that bnk ≤ b[ 2 mk ] ≤ (1 + ε)2 bmk for all sufficiently large k. It follows that P (Ak ) = P (Snk −nk−1 > (1 + ε)abnk ) = P (Samk > (1 + ε)abnk ) ≥ P (Samk > (1 + ε)3 abmk )

for all sufficiently large k. Choose small ε such that (1 + ε)3 a < 1. By (1.23), we get P (Ak ) ≥ exp{−(1 − τ )βnk } ≥ k −(1−τ )  for all sufficiently large k. Then the series P (Ak ) diverges. k

By Lemma 1.3, we have P (Ak Dk i.o.) ≥ q > 0. From the other hand, relation (1.26) implies that P (Ak Dk i.o.) = 0.

page 23

September 13, 2019

24

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

This contradiction shows that (1.26) does not hold and the theorem is proved for Rn . The proof for Tn follows the same way. By (1.23), we have Qn = P (Tn ≥ (1 − ε)bn ) ≥ e−(1−τ )βn

(1.27)

for all sufficiently large n. Assume first that an /n → ∈ [0, 1). Choose δ ∈ ( , 1) and a natural N such that inequalities (an + 1)/n < δ and (1.27) hold for all n > N . Put n1 = N and nk+1 = min {n : n > nk , n ≥ nk + ank } for k ∈ N. In the same way as before, we obtain m  Qnk ≥ C(log nm )τ . Then the series

 k

k=2

Qnk diverges and the events {Tnk ≥ (1 − ε)bnk } are

independent. The desired assertion follows from the Borel–Cantelli lemma. Suppose now that an /n → 1. Denote nk = [θk ] for k ∈ N, where θ > 1. We will prove that relation Tn (1.28) lim sup k ≤ a < 1 a.s. k→∞ bnk does not hold. For k = 2, 3, . . . , consider the events   Ak = Snk +ank − Snk−1 +ank > (1 + ε)abnk and

  Dk = Snk−1 +ank − Snk > −εabnk .

Condition 4) and Lemma 1.2 imply that P (Dk ) ≥ q for all sufficiently large k. In the same way as before, it follows from relation (1.23) that the series  P (Ak ) diverges. k

Applying Lemma 1.3, we conclude again that relation (1.28) does not hold. The theorem is now proved for Tn as well. If the sequence {an } increases slow enough, then the behaviour of Un and Wn becomes more stable that follows from the next result. Theorem 1.3. If the conditions of Theorem 1.2 hold and log log n = o(log(n/an )), then Un lim inf ≥ 1 a.s. (1.29) bn

page 24

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

25

Strong Laws and Large Deviations

Proof. Take ε > 0 and put An = {Un ≤ (1 − ε)bn } . Making use of (1.23), n/an → ∞ and the inequality 1 − x ≤ e−x , x > 0 , we get ⎞ ⎛ [n/an ]−1    P (An ) ≤ P ⎝ S(m+1)an − Sman ≤ (1 − ε)bn ⎠ m=0

[n/an ]−1 [n/an ]−1 = (P (San ≤ (1 − ε)bn )) ≤ 1 − e−(1−τ )βn 

1−τ [n/an ]−1 

1−τ n/(2an ) a a n n = 1− ≤ 1− n log n n log n

  τ 1 n ≤ exp − (log n)−1+τ 2 an for all sufficiently large n. It follows from log log n = o(log(n/an )) that the inequality n/an ≥ (log n)2/τ holds for all sufficiently large n. Then

 1 P (An ) ≤ exp − (log n)1+τ ≤ n−2 2  for all sufficiently large n. This implies that the series n P (An ) converges. The Borel–Cantelli lemma yields (1.29). Theorems 1.1–1.3 and the inequalities Rn ≤ Un ≤ Wn imply the following result. Theorem 1.4. If the conditions of Theorems 1.1 and 1.2 hold, then lim sup

Un Rn Tn Wn = lim sup = lim sup = lim sup =1 bn bn bn bn

a.s.

If, in addition, log log n = o(log(n/an )), then lim

Un Wn = lim =1 bn bn

a.s.

Theorems 1.1–1.4 work for various sequences {an }. Therefore, they are the universal strong laws. Note that other authors may use this term in another senses. For example, results under minimal moment assumptions on X may be called universal as well. The assumptions of Theorems 1.1–1.4 are very general. They illustrate tools that we need to prove strong limit theorems for increments. Relations

page 25

September 13, 2019

26

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

(1.13) and (1.23) are bounds for probabilities of large deviations for sums Sn . Inequality (1.14) is a kind of the L´evy inequality. Condition (1.12) is a regularity condition for norming sequence {bn } that will be specified later. In the sequel, we will first study the asymptotic behaviour of probabilities of large deviations for Sn . This will allow us to find a formula for bn and to obtain further universal results under various one-sided moment assumptions. Then we derive corollaries of the universal strong laws that include the results mentioned above. We will also prove that the moment assumptions are optimal.

page 26

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Chapter 2

Large Deviations for Sums of Independent Random Variables

Abstract. We discuss the method of conjugate distributions, properties of related functions (the moment generating function and its logarithmic derivatives, the large deviations function and its inverse) and a classification of probability distributions with exponential moments. We derive asymptotics of these functions at zero for random variables from domains of attractions of the normal law and completely asymmetric stable laws with index from (1, 2). Then we find logarithmic asymptotics of large deviations for such random variables. For infinite exponential moments, we use truncations. Finally, we obtain large deviations results associated with the classification of distributions. 2.1

Probabilities of Large Deviations

Let {Xn } be a sequence of i.i.d. random variables and {xn } be a sequence of positive real numbers such that xn → ∞ as n → ∞. For all n ∈ N, put Sn = X 1 + X 2 + · · · + X n . Probabilities P (Sn ≥ xn ) are called probabilities of large deviations. We will assume that the sequences {xn } increase fast enough. For example, if EX1 = 0 and EX12 = 1, then by the central limit theorem, asymptotic √ √ behaviours of P (Sn ≥ xn ) and 1 − Φ(xn / n) coincide for xn = O( n), where Φ(x) is the standard normal d.f. Hence, in this case, we will only √ deal with the sequences {xn } such that xn / n → ∞. Investigations of the asymptotic behaviour of probabilities of large deviations are of essential interest in probability theory and its applications. We mentioned above that the method of analysis of large deviations probabilities allows to prove various strong limit theorems under optimal one-sided moment assumptions. We will use this method to derive the universal 27

page 27

September 13, 2019

28

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

laws for sums of independent random variables. To this end, we only need asymptotics of functions of the large deviations theory (LDT) that we will introduce below. At the same time, these asymptotics may be used to derive logarithmic asymptotics of probabilities of large deviations which is of independent interest. We will start this chapter with a method of investigations of the large deviations. Further, we will describe the behaviour of the functions of the LDT and their relationships with the distributions of summands. Moreover, we will find conditions under which logarithms of probabilities of large deviations have the same asymptotics as logarithms of tails of the standard normal distribution or completely asymmetric stable laws with index α ∈ (1, 2). 2.2

The Method of Conjugate Distributions

In this section, we derive a method of investigations of asymptotic behaviour for large deviation probabilities of sums of independent random variables. Start with a single random variable X. Let X be a non-degenerate random variable such that EX ≥ 0 and h0 = sup{h : EehX < ∞} > 0.

(2.1)

The function ϕ(h) = Ee , 0 ≤ h < h0 , is called the moment generating function (m.g.f.). Condition (2.1) provides the existence of the function ϕ(h) in some interval with left point at 0. It is called the one-sided Cram´er condition. Note that it imposes no assumptions on the left tail of the distribution of X and, consequently, one may consider the random variable X without the second moment as well. First and second logarithmic derivatives of ϕ(h) play an important role in what follows. For 0 ≤ h < h0 , put ϕ (h) , m(h) = (log ϕ(h)) = ϕ(h)   2 ϕ (h) ϕ (h) σ 2 (h) = (log ϕ(h)) = − . ϕ(h) ϕ(h) The method of conjugate distributions is based on a passing from the distribution of X to a distribution having all moments. Put F (x) = P (X < x) for x ∈ R. The distribution, corresponding to the d.f. x 1 ¯ ehu dF (u), F (x) = ϕ(h) hX

−∞

page 28

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Large Deviations for Sums

29

is called a conjugate distribution. Here h ∈ (0, h0 ) is a parameter which we can vary appropriately. ¯ be a random variable with the d.f. F¯ (x). We have Let X 1 m(h) = ϕ(h) σ 2 (h) =

∞

∞ ¯ udF¯ (u) = E X,

hu

ue dF (u) = −∞ ∞

1 ϕ(h)

−∞

¯ 2 − (E X) ¯ 2 = DX, ¯ u2 ehu dF (u) − m2 (h) = E X

−∞

for 0 ≤ h < h0 . One can easily find all moments of the conjugate random ¯ variable X. This yields the following properties of the functions ϕ(h), m(h) and σ 2 (h): 1) m(h) is a continuous, strictly increasing function with m(0) = EX; 2) σ 2 (h) is continuous, σ 2 (0) = DX (if the variation is finite) and 2 σ (h) > 0; 3) ϕ(0) = 1, ϕ(h) is a convex function and log ϕ(h) has derivatives of all orders in the interval 0 < h < h0 . Further, by the definition of the d.f. F¯ (x), we have ∞ P (X ≥ x) = ϕ(h)

e−hu dF¯ (u).

x

This relation allows us to investigate the behaviour of the tail of the distribution of X and, therefore, that of large deviation probabilities. The corresponding method is called the method of conjugate distributions. Lemma 2.1. Let X be a non-degenerate random variable such that EX ≥ 0 and h0 > 0. Then for all h ∈ (0, h0 ), the following inequalities hold P (X ≥ m(h)) ≤ e−f (h) , P (X ≥ m(h) − 2σ(h)) ≥

3 −f (h)−2hσ(h) e , 4

where f (h) = hm(h) − log ϕ(h). Proof. The first inequality follows from the Tchebyshev inequality. Prove

page 29

September 13, 2019

15:48

30

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

¯ − m(h))/σ(h), V (x) = P (Z < x). Then the second one. Put Z = (X ∞

e−hu dF¯ (u)

P (X ≥ m(h) − 2σ(h)) = ϕ(h) m(h)−2σ(h)

∞ = ϕ(h)

e−h(m(h)+yσ(h)) dV (y) = e−f (h)

−2

≥ e−f (h)

2

e−hyσ(h) dV (y)

−2

e−hyσ(h) dV (y) ≥ e−f (h)−2hσ(h) P (|Z| ≤ 2)

−2

≥e

∞

−f (h)−2hσ(h)



EZ 2 1− 4

 =

3 −f (h)−2hσ(h) e . 4

In the last inequality, we have used the Tchebyshev inequality again. It follows from the properties of the functions ϕ(h), m(h) and σ 2 (h) mentioned above that the function f (h) is continuous, strictly increasing and f (0) = 0. The second property follows from the relation f  (h) = hσ 2 (h). To apply Lemma 2.1 for X = Sn , where Sn is a sum of i.i.d. random variables, we only need to find the m.g.f. and its logarithmic derivatives for Sn . It is clear that EehSn = (ϕ(h))n . It follows that m(h), f (h) and σ 2 (h) have to be replaced in Lemma 2.1 by nm(h), nf (h) and nσ 2 (h) correspondingly. Then we get the following result. Lemma 2.2. Let X, X1 , X2 , . . . , Xn be i.i.d. non-degenerate random variables such that EX ≥ 0 and h0 > 0. Put Sn = X1 + X2 + · · · + Xn . Then the inequalities P (Sn ≥ nm(h)) ≤ e−nf (h) , √ √ 3 P (Sn ≥ nm(h) − 2 nσ(h)) ≥ e−nf (h)−2h nσ(h) 4 hold for all h ∈ (0, h0 ). We will see below that logarithmic asymptotics of left-hand and righthand sides of the inequalities in Lemma 2.2 coincide under certain conditions. This will allow us to find the logarithmic asymptotics of large deviation probabilities for sums of i.i.d. random variables.

page 30

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Large Deviations for Sums

2.3

31

Completely Asymmetric Stable Laws with Exponent α>1

We consider completely asymmetric stable laws Fα with the characteristic function (c.f.) 

 πα t ψ(t) = exp −c|t|α 1 + i tan , (2.2) |t| 2 where c = − cos(πα/2)/α and α ∈ (1, 2]. For α = 2, it is the standard normal c.f. Such stable laws have densities, zero means and finite exponential moments. Indeed, for all real t, we have  |t|α πα t πα  cos + i sin ψ(t) = exp α 2 |t| 2  (it)α   |t|α t πα  ei |t| 2 = exp . = exp α α An analytic extension of ψ(t) in the complex half-plane {z : Rez ≥ 0} is the function ψ(z) = exp{z α /α}. Hence, the distribution with the c.f. (2.2) has the m.g.f.

α h ϕ(h) = exp for h > 0. α It follows that α−1 α h m(h) = hα−1 , σ 2 (h) = (α − 1)hα−2 , f (h) = α for all h > 0. Here α = 2 corresponds to the standard normal law when one can check by direct calculations that ϕ(h) = h2 /2 for all real h. Of course, the standard normal law is the only completely asymmetric stable law with a finite m.g.f. on the hole real line. For α ∈ (1, 2), the m.g.f. exists for positive h only. Another (non-asymmetric) stable laws have not the exponential moment. So, we deal with the stable laws Fα . Note that the scale parameter c is such that ϕ(h) = exp {hα /α} for h > 0. The results of the previous sections give us the asymptotic for tails of Fα . Let Fα (x) denote the d.f. of the stable law Fα . By Lemma 2.1, we have log(1 − Fα (xn )) ∼ −λx1/λ n for every sequence {xn } such that xn → ∞, where λ = (α − 1)/α. For α = 2, the latter easily follows from the well known relation 2 1 e−xn /2 . 1 − Φ(xn ) ∼ √ 2πxn

page 31

September 13, 2019

32

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Remember some definitions and facts related to our topic. We will use the following notation: g(x) ∈ SVa (g(x) ∈ RVa ) if g(x) is a slowly (regularly) varying function at a. The distribution F (and its d.f.) belongs to the domain of attraction of Fα , if there exists a sequence {Bn } of positive constants such that the distributions of Sn /Bn converge weakly to Fα where Sn is a sum of n i.i.d. random variables with the distribution F . Note that we always deal with centering at zero. We write F ∈ DN (α) or F ∈ D(α) if the d.f. F (x) belongs to the domain of normal or non-normal attraction of the stable law Fα , correspondingly. The relation F ∈ DN (α) means that Bn = bn1/α for all n ∈ N. We always assume that b = 1. The relation F ∈ D(α) means that Bn = n1/α h(n) for all n ∈ N, where h(x) ∈ SV∞ and either h(x) → ∞, or h(x) → 0 as x → ∞. One usually uses the notation D(α) for the full domain of attraction (i.e. for DN (α) ∪ D(α) in our notation), but we prefer to split the domain into two disjoint parts. Moreover, one usually deals with normings bn1/α with b > 0. We always assume b = 1. We finish this section with necessary and sufficient conditions for F ∈ D(α) and F ∈ DN (α). Theorem 2.1. Assume that EX = 0 and α ∈ (1, 2). The following assertions hold: 1) F ∈ DN (2) iff EX 2 = 1. 0 2 2) Assume that E(X + )2 < ∞. Then F ∈ D(2) iff G(x) = u dF (u) ∈ −x

SV∞ and G(x) → ∞ as x → ∞. 3) F ∈ DN (α) iff xα F (−x) → (α − 1)/(αΓ(2 − α)) as x → ∞. 4) F ∈ D(α) iff G(x) = xα F (−x) ∈ SV∞ and G(x) → 0 or ∞ as x → ∞. For 2) and 4), the constants Bn may be found from nBn−2 G(Bn ) → 1. Theorem 2.1 follows from results in [Ibragimov and Linnik (1971)]. One has to take into account that we impose additional assumptions with respect to general results on domains of attraction of stable laws. We deal with the norming sequences n1/α for the normal attraction. We assume E(X + )2 < ∞ for F ∈ D(2). Moreover, the asymmetry of stable laws yields that right-hand tails are negligible in cases 3) and 4). This implies the difference.

page 32

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Large Deviations for Sums

2.4

33

Functions of Large Deviations Theory and a Classification of Probability Distributions

In this section, we describe the properties of the m.g.f., its logarithmic derivatives and the large deviation function. We also give a classifications of distributions based on these properties. Examples of distributions from each classes are given as well. Let X be a non-degenerate random variable such that EX ≥ 0 and h0 > 0. Denote F (x) = P (X < x). We established in previous sections that the m.g.f. ϕ(h) = EehX , 0 ≤ h < h0 , has the following properties: ϕ(0) = 1, ϕ(h) is increasing and convex, log ϕ(h) has derivatives of all orders in the interval 0 < h < h0 . It has also been shown that m(h) = (log ϕ(h)) is continuous and increasing, m(0) = EX, and σ 2 (h) = (log ϕ(h)) is continuous, σ 2 (0) = DX (if the variation exists), σ 2 (h) > 0. It follows that the function f (h) = hm(h) − log ϕ(h) is continuous and increasing and f (0) = 0. Put 1 . A = lim m(h), c0 = hh0 lim f (h) hh0

Here A ≤ ∞ and c0 ≥ 0. We also assume that 1/∞ = 0 and 1/0 = ∞. Denote (2.3) ζ(x) = f (m−1 (x)) for x ∈ [EX, A),   1 γ(x) = m(f −1 (x)) for x ∈ 0, , (2.4) c0 where m−1 (·), f −1 (·) are the inverse functions to m(h) and f (h), correspondingly. The function ζ(x) is called the large deviations function. It plays a key role in what follows. We see that ζ(x) is continuous and increasing and ζ(EX) = 0. Moreover, ζ  (x) = (xm−1 (x) − log ϕ(m−1 (x))) = m−1 (x), ζ  (x) = (m−1 (x)) > 0. It yields that ζ(x) is convex and γ(x) = ζ −1 (x) is continuous, increasing, concave and γ(0) = EX. In what follows, we assume that ζ(x) and γ(x) are defined by relations ζ(x) =

sup

{xh − log ϕ(h)},

h≥0: ϕ(h) 0. Suppose that ω = ∞. Then P (X ≥ 2A) > 0 and 1≥e

−Ah

ϕ(h) = e

∞ ∞ hu −Ah e dF (u) ≥ e ehu dF (u) ≥ eAh P (X ≥ 2A) → ∞

−Ah

−∞

2A

as h → ∞. This contradiction shows that ω < ∞. If ω < ∞, then evidently h0 = ∞. Further, by the definition of ω, we have m(h) =

1 EXehX ≤ ω. ϕ(h)

It follows that A ≤ ω < ∞. To finish the proof, we need only check that A ≥ ω. Assume that A < ω. Then there exists ε > 0 such that A(1 + ε) < ω. We have P (X ≥ (1 + ε)A) > 0 and 1 ≥ e−Ah ϕ(h) ≥ eεAh P (X ≥ (1 + ε)A) → ∞ as h → ∞. This contradiction yields that A ≥ ω. a) Suppose that P (X = ω) > 0. By the Lebesgue dominated convergence theorem, we get ω

ω e

ϕ(h) = −∞

hu

dF (u) = e

hω

eh(u−ω) dF (u) = ehω P (X = ω)(1 + o(1)) −∞

page 34

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Large Deviations for Sums

35

as h → ∞. Applying again the dominated convergence theorem, we obtain ω h ehω  (ϕ (h) − ωϕ(h)) = h(m(h) − ω) = h(u − ω)eh(u−ω) dF (u) → 0 ϕ(h) ϕ(h) −∞

as h → ∞. Therefore, f (h) = hm(h) − hω − log P (X = ω) + o(1) → − log P (X = ω) as h → ∞. Hence, c0 = −1/ log P (X = ω) > 0. We get additional properties of ζ(x) and γ(x) in this case. We have ζ(A) = 1/c0 and ζ(x) = ∞ for x > A. Indeed, x > A yields xh − log ϕ(h) ≥ xh − Ah → ∞ as h → ∞. Further, γ(x) = ω for x > 1/c0 . If P (X = ±1) = 1/2, then h0 = ∞, A = 1 and c0 = 1/ log 2 > 0. b) Assume that P (X = ω) = 0. Suppose without loss of generality that EX > 0. For ε ∈ (0, cω), put Xε = XI{X EX, where ζε (x) is the large deviation function for Xε . Then lim inf ζ(x) ≥ ζε (A) = − log P (X ≥ ω − ε). xA

Passing to the limit as ε → 0, we get ζ(x) → ∞ as x → A. This yields that c0 = 0. In this case, γ(x) → ω for x → ∞. If X has the uniform distribution on [−1, 1], then h0 = ∞, A = 1 and c0 = 0. We further assume that at least one of the conditions h0 = ∞ and A < ∞ fails. Then ω = ∞ by Lemma 2.3. 2) Suppose that h0 = ∞ and A = ∞. We have h

1 (m(h) − m(u))du ≥

f (h) = 0

(m(h) − m(u))du ≥ m(h) − m(1) 0

for all h ≥ 1. It follows that f (h) → ∞ as h → ∞ and c0 = 0. Hence ζ(x) → ∞ and γ(x) → ∞ as x → ∞.

page 35

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

36

9789811212826

Universal theory for strong limit theorems of probability

If X has the stable distribution Fα then h0 = ∞, A = ∞ and c0 = 0. Remember that α = 2 for the standard normal law. 3) Assume that h0 < ∞ and A = ∞. We prove in the same way as in 2) that    h0 h0 f (h) ≥ m(h) − m 2 2 for all h ≥ h0 /2. This implies that f (h) → ∞ as h → h0 and c0 = 0. Therefore, ζ(x) → ∞ and γ(x) → ∞ as x → ∞. If X has the exponential distribution with the density p(x) = e−x I(0,∞) (x), then h0 = 1, A = ∞ and c0 = 0. 4) Suppose that h0 < ∞ and A < ∞. We have h m(u)du ≤ lim Ah = Ah0 < ∞.

log ϕ(h0 ) = lim

hh0

0

hh0

Then f (h) → h0 A − log ϕ(h0 ) < ∞ as h → h0 and c0 = 1/(h0 A − log ϕ(h0 )) > 0. Note that the functions ζ(x) and γ(x) are defined for large x as well. Indeed, if x > A, then the function xh − log ϕ(h) is increasing in h. Hence ζ(x) = xh0 − log ϕ(h0 ) for x > A and x + log ϕ(h0 ) h0 for x > 1/c0 . In this case, one can define f (h) and m(h) for h > h0 by formulae 1 m(h) = A + h − h0 , f (h) = + h0 (h − h0 ). (2.5) c0 Then ζ(x) = f (m−1 (x)) for x ≥ EX and γ(x) = m(f −1 (x)) for x > 0. If X has the density p(x) = cx−3 e−x I(1,∞) (x), then h0 = 1, A = 2 and c0 = 1/(2 − log(c/2)). Thus, there exist five classes of distributions corresponding to various combinations of h0 , A and c0 . Denote these classes as follows: γ(x) =

K1 = {F (x) : h0 = ∞, A < ∞, c0 = 0}, K2 = {F (x) : h0 = ∞, A < ∞, c0 > 0}, K3 = {F (x) : h0 = ∞, A = ∞}, K4 = {F (x) : h0 < ∞, A = ∞}, K5 = {F (x) : h0 < ∞, A < ∞}.

page 36

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Large Deviations for Sums

37

Note that K1 and K2 have been considered in items 1,b) and 1,a), correspondingly. We have also proved that c0 = 0 for F ∈ K3 ∪ K4 and c0 = 1/(h0 A − log ϕ(h0 ) > 0 for F ∈ K5 . Moreover, X is bounded from above for F ∈ K1 ∪ K2 by ω = A and c0 = −1/ log P (X = A) for F ∈ K2 . 2.5

Large Deviations and a Non-Invariance

We consider the asymptotic behaviour of P (Sn ≥ bn) in this section under h0 > 0. It turns out that the logarithmic asymptotics of these probabilities depend on the large deviations function ζ(x). This function depends on the full distribution of X and, sometimes, determines this distribution uniquely. This property is called the non-invariance in the sequel. The main result is as follows. Theorem 2.2. Let X, X1 , X2 , . . . be a sequence of i.i.d. random variables such that EX ≥ 0 and h0 > 0. Put Sn = X1 + X2 + · · · + Xn . Then for all x ∈ [EX, A), the following relation holds log P (Sn ≥ xn) ∼ −ζ(x)n.

(2.6)

Proof. Let h be a solution of the equation m(h) = x. This solution exists and it is unique that follows from the properties of m(h) and the definition of A. By Lemma 2.2 and the definition of ζ(x), we have 1 1 log P (Sn ≥ xn) = lim sup log P (Sn ≥ nm(h)) n n ≤ −f (h) = −ζ(x).

lim sup

(2.7)

Take ε ∈ (0, 1) such that y = x/(1 − ε) < A. Let h be a solution of √ the equation m(h) = y. Since h is a fixed number, we have 2 nσ(h) = √ o(nm(h)) and 2h nσ(h) = o(nf (h)). By Lemma 2.2, we get √ P (Sn ≥ xn) = P (Sn ≥ (1 − ε)yn) ≥ P (Sn ≥ nm(h) − 2 nσ(h)) √ 3 3 3 ≥ e−nf (h)−2h nσ(h) ≥ e−nf (h)(1+ε) = e−nζ(y)(1+ε) 4 4 4 3 −nζ(x/(1−ε))(1+ε) = e 4 for all sufficiently large n. This yields that   x 1 lim inf log P (Sn ≥ xn) ≥ −ζ (1 + ε). n 1−ε

page 37

September 13, 2019

38

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Passing to the limit in this inequality as ε → 0, we obtain 1 lim inf log P (Sn ≥ xn) ≥ −ζ(x). n The last inequality and (2.7) imply (2.6). Note that the centering of sums Sn in Theorem 2.2 is automatic since the norming and centering (at mean) sequences have the same order n. If X has c.f. (2.2), then ζ(x) = λx1/λ

for

x ≥ 0,

where λ = (α − 1)/α. Calculations of ζ(x) for other distributions show that formulae are much more complicated. Note that ζ(x) uniquely determines the distribution of X provided ϕ(h) < ∞ for 0 ≤ |h| < h1 . Indeed, ζ  (x) = m−1 (x) and m(h) = (log ϕ(h)) . It turns out that ζ(x) defines the m.g.f. ϕ(h) in a neighbourhood of zero. At the same time, the c.f. of X is analytic and may be expanded to Taylor’s series. The coefficients of this series are uniquely defined by the coefficient of the expansion of ϕ(h). It follows that ζ(x) defines the distribution of X in this case. We will show below that for x = xn = o(1), the behaviour of the probabilities P (Sn ≥ xn) depends on the asymptotic of ζ(x) at 0 provided X is from the domain of attraction of the stable laws with c.f. (2.2). 2.6

Methods of Conjugate Distributions and Truncations

The method of conjugate distributions can not be directly used for random variables without exponential moments. A general way is to present the sum of independent random variables as the sum of truncated from above random variables and a remainder. The method of conjugate distributions may be applied to the sum of truncated random variables and the remainder will be negligible. We develop this techniques below. The aim of this section is to derive an analogue of Lemma 2.2. Let X, X1 , X2 , . . . , Xk be non-degenerate, i.i.d. random variables such that EX = 0. Fix y > 0. Let us introduce random variables ˆ = min{X, y} = XI{X 0, put ˆ

ˆ

  ˆ = (log ϕ(h)) ˆ , σ ˆ 2 (h) = (log ϕ(h)) ˆ , ϕ(h) ˆ = Eeh(X−E X) , m(h) ˆ ˆ ˆ f (h) = hm(h) ˆ − log ϕ(h), ˆ ζ(x) = sup{xh − log ϕ(h)}. h≥0

The next lemma contains an upper bound for P (Sk ≥ kx). Lemma 2.4. Assume that x > 0, ρ ∈ [0, 1), δ ∈ (0, 1) and kP (X ≥ y) ≤ log 2. Then ˆ

P (Sk ≥ kx) ≤ e−kζ(x) + 2kP (X ≥ y). If, in addition, the inequality ˆ

ˆ

kP (X ≥ y)e−kζ(δx) + (kP (X ≥ y))2 ≤ H1 e−kζ(x)(1−ρ)

(2.8)

holds, then ˆ

P (Sk ≥ kx) ≤ (1 + H1 )e−kζ(x)(1−ρ) + kP (X ≥ (1 − δ)kx). Proof. Assume first that (2.8) holds. Consider the events A = {Sk ≥ ˆ i } for i = 1, 2, . . . , k and Bi = {exactly k − kx}, Ai = {Xi = X i events from A1 , A2 , . . . , Ak occur} for i = 0, 1, 2, . . . , k. We have P (Sk ≥ kx) =

k 

P (ABi )

i=0

= P (AA1 . . . Ak ) +

k 

Cki P (AA1 . . . Ak−i Ak−i+1 . . . Ak )

i=1

≤ P (AA1 . . . Ak ) + kP (AA1 . . . Ak−1 Ak ) +

k 

Cki (P (A1 ))i

i=2

= a1 + ka2 + a3 . We derive bounds for a1 , a2 and a3 successively. By Lemma 2.2, we get  k   ˆ ˆ Xi ≥ kx ≤ P (Vk ≥ kx) ≤ e−kζ(x) . (2.9) a1 ≤ P i=1

For a2 , we have k−1   ˆ i + Xk ≥ kx, Xk ≥ y ≤ P (Vk−1 + Xk ≥ kx, Xk ≥ y) X a2 ≤ P i=1

≤ P (Vk−1 + Xk ≥ kx, Xk ≥ y, Xk < (1 − δ)kx) + P (X ≥ (1 − δ)kx) ≤ P (Vk−1 ≥ δkx)P (X ≥ y) + P (X ≥ (1 − δ)kx).

page 39

September 13, 2019

15:48

40

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

ˆ The last inequality in (2.9) and the convexity of ζ(x) yield ˆ

ˆ

P (Vk−1 ≥ δkx) ≤ e−(k−1)ζ(δkx/(k−1)) ≤ e−kζ(δx) . Hence, ˆ

a2 ≤ e−kζ(δx) P (X ≥ y) + P (X ≥ (1 − δ)kx). Further, making use of condition kP (X ≥ y) ≤ log 2, we get a3 ≤

k (kP (A1 ))2 kP (A1 ) (kP (A1 ))2  (kP (A1 ))i−2 ≤ e ≤ (kP (X ≥ y))2 . 2 (i − 2)! 2 i=2

Using inequality (2.8), we obtain ˆ

ˆ

P (Sk ≥ kx) ≤ e−kζ(x) + kP (X ≥ y)e−kζ(δx) + kP (X ≥ (1 − δ)kx) +(kP (X ≥ y))2 ˆ

≤ (1 + H1 )e−kζ(x)(1−ρ) + kP (X ≥ (1 − δ)kx). If we do not assume (2.8), then it is clear that a2 ≤ P (X ≥ y) and a3 ≤ kP (x ≥ y). This yields the result in this case. Turn to lower bounds. ˆ ≤ τ (1 − τ )x Lemma 2.5. Assume that x = m(h) ˆ and τ ∈ (0, 1). If −E X √ ˆ and 2hˆ σ(h) ≤ τ k f (h), then P (Sk ≥ (1 − τ )2 kx) ≥

3 −kζ(x)(1+τ ˆ ) e . 4

Proof. Since ϕ(h) ˆ ≥ 1 for all h > √ 0, we get fˆ(h) ≤ hm(h) ˆ by the definition ˆ of f (h). It follows that 2ˆ σ (h) ≤ τ k m(h). ˆ Making use of Lemma 2.2, we have  k   ˆ i ≥ (1 − τ )2 kx X P (Sk ≥ (1 − τ )2 kx) ≥ P i=1 2

ˆ ≥ P (Vk ≥ (1 − τ )kx) = P (Vk ≥ (1 − τ ) kx − kE X)) √ √ 3 ˆ ≥ P (Vk ≥ k m(h) ˆ − 2 kˆ σ (h)) ≥ e−kf (h)−2h kˆσ (h) 4 3 −kfˆ(h)(1+τ ) 3 −kζ(x)(1+τ ˆ ) ≥ e = e . 4 4 The proof is completed.

page 40

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

41

Large Deviations for Sums

Applying Lemma 2.5, we have bounds for P (Sk ≥ kx) provided x is from the set of values of m(h). ˆ We can use the next lemma otherwise. Lemma 2.6. Take k ≥ 1, δ > 0, x > 0. If P (Sk−1 ≥ −δkx) − (k − 1)P (X ≥ (1 + δ)kx) ≥ q, then P (Sk ≥ kx) ≥ qkP (X ≥ (1 + δ)kx). Proof. We only need to check that for all u and v, the inequality P (Sk ≥ u) ≥ kP (X ≥ u + v) (P (Sk−1 ≥ −v) − (k − 1)P (X ≥ u + v)) (2.10) holds. Put Aj = {Xj ≥ u + v}, Bj = {Sk − Xj ≥ −v}, j = 1, 2, . . . , k. We have k k

  P (Aj An Bj Bn ) P (Aj Bj ) − P (Sk ≥ u) ≥ P (∪kj=1 Aj Bj ) ≥ n=1 n =j

j=1

≥

k k k k



   P (Aj An ) = P (Aj ) P (Bj ) − P (An ) . P (Aj )P (Bj ) − n=1 n =j

j=1

n=1 n =j

j=1

The latter coincides with the right-hand side of (2.10). 2.7

Asymptotic Expansions of Functions of Large Deviations Theory in Case of Finite Variations

Asymptotic expansions for the functions of the LDT at zero play a key role in proofs below. In this section, we deal with the case of finite variations. Our first result is for random variables with exponential moments. Lemma 2.7. If EX = 0, EX 2 = 1 and h0 > 0, then ϕ(h) = 1 + f (h) =

h2 (1 + o(1)), 2

h2 (1 + o(1)) 2

as

m(h) = h(1 + o(1)),

σ 2 (h) = 1 + o(1),

h → 0.

Proof. The conditions EX = 0 and EX 2 = 1, the definitions of m(h) and σ 2 (h) and their properties imply that ϕ (0) = m(0) = 0 and ϕ (0) = σ 2 (0) = 1.

page 41

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

42

9789811212826

Universal theory for strong limit theorems of probability

Take h ∈ (0, h0 ). By Taylor’s formula, we have ϕ(h) = ϕ(0) + hϕ (0) +

h2  h2 ϕ (θ) = 1 + ϕ (θ), 2 2

where θ ∈ (0, h). Hence, ϕ (θ) → 1 and ϕ(h) = 1 +

h2 (1 + o(1)) 2

as h → 0. Further, m(h) =

ϕ (h) ϕ (0) + hϕ (θ1 ) h(1 + o(1)) = = = h(1 + o(1)) ϕ(h) 1 + o(1) 1 + o(1)

as h → 0. The continuity of σ 2 (h) yields σ 2 (h) = 1 + o(1) as h → 0. It remains to apply the definition of f (h). Note that the asymptotics in Lemma 2.7 coincide with those for the standard normal random variable. Lemmas 2.7 and 2.2 allow to obtain large deviations results for h0 > 0. We now turn to random variables without exponential moments. Using truncations, we derived absolute inequalities for P (Sk ≥ kx) in the last section. In the next section, they will be applied with k = n, x = xn and y = yn → ∞ to study large deviations. To this end, we need asymptotic exˆ − EX ˆ with y = yn . In this section, pansions for the functions of LDT for X we find conditions under which these functions have the same asymptotics as in Lemma 2.7. Let X be a random variable with a d.f. F (x). Let {yn } be a sequence of positive numbers such that yn → ∞. For n ∈ N, put Yn = min{X, yn }

Zn = Yn − EYn ,

Fn (x) = P (Zn < x).

For h ≥ 0 and x > 0, define functions ϕn (h) = EehZn ,

mn (h) = (log ϕn (h)) ,

fn (h) = hmn (h) − log ϕn (h),

σn2 (h) = (log ϕn (h)) ,

ζn (x) = sup{xh − log ϕn (h)}, h≥0

where n ∈ N. It is clear that for every fixed n, these functions coincide ˆ correspondingly with ϕ(h), ˆ m(h), ˆ fˆ(h) and ζ(x) when y = yn . So, they are ˆ − EX ˆ with y = yn . the functions of the LDT for the random variable X It is convenient to put yn = ∞ for all n when h0 > 0 and we need no truncations. Of course, we then assume that h < h0 and x < A in the above formulae.

page 42

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

43

Large Deviations for Sums

Our next result is as follows. Lemma 2.8. Assume that EX = 0, EX 2 = 1, h = hn → 0 and one of the following conditions holds: 1) h0 > 0 and y = yn = ∞ for all n, 2) y = yn → ∞ and ∞ (2.11) h x3 ehx dFn (x) = o(1). 0

Then

h2 (1 + o(1)), 2 mn (h) = h(1 + o(1)),

ϕn (h) = 1 +

(2.12) (2.13)

σn2 (h)

= 1 + o(1), (2.14) h2 (1 + o(1)). (2.15) fn (h) = 2 Proof. If condition 1) holds, then Lemma 2.7 yields (2.12)–(2.15). Assume that condition 2) holds. For k = 0, 1, 2, denote k  xj , ek (x) = ex − j! j=0 0 k

Lk,n =

x e2−k (xh)dFn (x), −∞

Then

0

∞ ehx dFn (x) = 1 +

ϕn (h) = −∞

Ik,n

+∞  = xk e2−k (xh)dFn (x).

∞

h2 EZn2 + L0,n + I0,n , 2

(2.16)

xehx dFn (x) = hEZn2 + L1,n + I1,n ,

(2.17)

∞ + (mn (h)) )ϕn (h) = x2 ehx dFn (x) = EZn2 + L2,n + I2,n .

(2.18)

ϕn (h)mn (h) = −∞

(σn2 (h)

2

−∞

We will use the following well known result. Lemma 2.9. For all x > 0, the inequalities xm+1 x xm+1 xm+1 e , C ≤ |em (−x)| ≤ D , em (x) ≤ (m + 1)! 1+x 1+x hold, where C and D are absolute positive constants.

m = 0, 1, 2,

page 43

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

44

9789811212826

Universal theory for strong limit theorems of probability

By Lemma 2.9 with m = 2 − k and (2.11), we have Ik,n

h3−k ≤ (3 − k)!

+∞  x3 ehx dFn (x) = o(h2−k ),

k = 0, 1, 2.

(2.19)

0

Applying again Lemma 2.9 with m = 2 − k, we obtain Ch2−k I(n) ≤ |Lk,n | ≤ Dh2−k I(n),

k = 0, 1, 2,

where 0 I(n) = −∞

Denote vn (u) =

u −∞

−1/h 



−∞ √ 1/  h

=h

0

−1/h

1/h

+ 0

x2 dFn (x). Then

x2 dFn (x) + h

I(n) ≤

  0 1 |x|3 dFn (x) = vn − |x|dvn (x) +h h

vn (−x)dx ≤

√ 1/ h

h|x|3 dFn (x). 1 + |hx|

−1/h

√

  √ 1 hvn (0) + (1 − h)vn − √ . h

Since u+EY  n

u+EY  n

2

x dF (x) − 2EYn

vn (u) = −∞

2

u+EY  n

xdF (x) + (EYn ) −∞

dF (x) −∞

√ and EYn → 0, we have vn (0) → E(X − )2 and vn (−1/ h) → 0. It follows that I(n) = o(1) and Lk,n = o(h2−k ), EYn2

k = 0, 1, 2. EZn2

EYn2

(2.20) 2

It is clear that EYn → 0, → 1 and = − (EYn ) → 1. Relations (2.16), (2.19) and (2.20) imply (2.12). Assertions (2.12), (2.17), (2.19) and (2.20) yield (2.13). From (2.12), (2.13), (2.18), (2.19) and (2.20), we obtain (2.14). The definition of the function fn (h), relations (2.12) and (2.13) imply (2.15). In the next lemma, we replace condition (2.11) on distributions of truncated random variables by those on the distributions of summands. Lemma 2.10. Assume that EX = 0, EX 2 = 1, y = yn → ∞, h = hn → 0 and one of the following conditions holds:

page 44

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

45

Large Deviations for Sums

∞,

1) there exists β ∈ (0, 1) such that lim sup hn yn1−β < 1 and Ee(X

+ β

)


0 and EX 2 g(X) < ∞. Hence, yn yn yn x 3 dF (x) ≤ C2 , x dF (x) = x2 g(x) g(x) g(yn ) 0

0

and P (X ≥ y) ≤ C2

1 y 2 g(y)

.

This yields that hJn ≤ C3

hyn hz 3 1 + C3 2 n ≤ C4 →0 g(yn ) y g(y) g(yn )

page 45

September 13, 2019

15:48

46

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

and relation (2.11) holds. Assume that condition 1) is satisfied. Let a be a positive real number such that hy 1−β ≤ a < 1 for all sufficiently large n. Then yn

3 hx

yn

x e dF (x) ≤ 0

3 axβ

x e 0

  yn β 3 (a−1)xβ dF (x) ≤ sup x e ex dF (x) ≤ C5 x≥0

0

for all sufficiently large n. Further, we have β

β

zn3 ehzn P (X ≥ y) ≤ C6 y 3 ehy P (X ≥ y) ≤ C7 y 3 ehy e−y ≤ C7 y 3 e(a−1)y → 0. It follows that hJn = o(1) and we get (2.11) again. In the next lemma, we deal with a special choice of h. Lemma 2.11. Assume that EX = 0, EX 2 = 1, y = yn → ∞, un → 0 and one of the following conditions holds: + β 1) there exists β ∈ (0, 1) such that lim sup un yn1−β < 1/2 and Ee(X ) < ∞, 2) un yn = O(1). For every fixed n, let hn be a solution of the equation mn (h) = un . Then relations (2.12)–(2.15) hold. Proof. For every fixed n, we have by Lemma 2.3 that mn (h) yn − EYn as h ∞. It follows that the equation mn (h) = un has a unique solution hn provided n is fixed and large enough. Lemma 2.10 yields that mn (2un ) = 2un (1 + o(1)). It follows that m(2un ) ≥ un = mn (hn ) for all large n. Since mn (h) is an increasing function of h, we arrive at hn ≤ 2un for all large n. Lemma 2.11 now follows from Lemma 2.10. 2.8

Large Deviations in Case of Finite Variations

In this section, we find the logarithmic asymptotics of large deviation probabilities for sums of i.i.d. random variables with a finite second moment. √ We first show that if h0 > 0 and xn = o( n), then this asymptotic coincides with that in the Gaussian case. Further, we deal with the case h0 = 0. We prove that the asymptotic is Gaussian for xn from regions depending on moment assumptions. The Linnik condition yields the result in a power region while E(X + )p < ∞ provides a logarithmic zone only.

page 46

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Large Deviations for Sums

9789811212826

47

Assume first that h0 > 0. Theorem 2.3. Let X, X1 , X2 , . . . be a sequence of i.i.d. random variables such that EX = 0, EX 2 = 1 and h0 > 0. Put Sn = X1 + X2 + · · · + Xn . √ Then for every sequence {xn } with xn → ∞ and xn = o( n), the following relation holds √ x2 (2.21) log P (Sn ≥ xn n) ∼ − n . 2 √ Proof. Let h = hn be a solution of the equation m(h) = xn / n. Since m(h) is increasing and m(0) = 0, this equation has a unique solution for all sufficiently large n. Moreover, h → 0. Making use of Lemma 2.7, we obtain h2 x2 xn h = √ (1 + o(1)), nf (h) = n (1 + o(1)) = n (1 + o(1)). n 2 2 Take ε ∈ (0, 1). By Lemma 2.2, we have √ x2 log P (Sn ≥ xn n) ≤ − n (1 − ε) 2 for all sufficiently large n. It follows that √ 1 1−ε lim sup 2 log P (Sn ≥ xn n) ≤ − . xn 2 Passing in this relation to the limit as ε → 0, we arrive at √ 1 1 lim sup 2 log P (Sn ≥ xn n) ≤ − . xn 2

(2.22)

Take ε ∈ (0, 1). Put yn = xn /(1 − ε). Let h = hn be a solution of the √ equation m(h) = yn / n. Using Lemma 2.7, we have √ √ √ nm(h) − 2 nσ(h) = yn n(1 + o(1)) + O( n) √ √ xn n(1 + o(1)) . = yn n(1 + o(1)) = 1−ε By Lemma 2.2, we get √ √ √ 3 P (Sn ≥ xn n) ≥ P (Sn ≥ nm(h) − 2 nσ(h)) ≥ e−nf (h)−2h nσ(h) 4 for all sufficiently large n. Applying of Lemma 2.7 yields √ x2 (1 + o(1)) y2 y2 . nf (h) − 2h nσ(h) = n (1 + o(1)) + O(yn ) = n (1 + o(1)) = n 2 2 2(1 − ε)2 It follows that √ x2n 3 log P (Sn ≥ xn n) ≥ log − 4 2(1 − ε)3

page 47

September 13, 2019

15:48

48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

for all sufficiently large n. Hence √ 1 1 . lim inf 2 log P (Sn ≥ xn n) ≥ − xn 2(1 − ε)3 Passing in this inequality to the limit as ε → 0, we get √ 1 1 lim inf 2 log P (Sn ≥ xn n) ≥ − . xn 2 This relation and (2.22) imply (2.21). We now derive analogues of Theorem 2.3 when the one-sided Cram´er condition is violated. Theorem 2.4. Let X, X1 , X2 , . . . be a sequence of i.i.d. random variables + β such that EX = 0, EX 2 = 1 and Ee(X ) < ∞ for some β ∈ (0, 1). Then relation (2.21) holds for every sequence {xn } with xn → ∞ and xn = o(nβ/(4−2β) ). β

Remember that Ee|X| < ∞ for some β ∈ (0, 1) is the Linnik condition. In Theorem 2.4, we assume the Linnik condition for X + instead of |X| which is called the one-sided Linnik condition. √ √ Proof. Put un = xn / n and y = yn = xn n for n ∈ N. We have n−β/2 = o(1) and condition 1) of Lemma 2.11 is fulfilled. un y 1−β = x2−β n Let h = hn be a solution of the equation mn (h) = un . By Lemma 2.11, relations (2.12)–(2.15) hold. xn = h(1 + o(1)) in view of Put k = n, x = mn (h). Note that x = √ n (2.13). Verify condition (2.8) of Lemma 2.4. By (2.15), we have x2 h2 ˆ k ζ(x) = k fˆ(h) = nfn (h) = n (1 + o(1)) = n (1 + o(1)). 2 2 Using the one-sided Linnik condition, we get

√ β β/2 kP (X ≥ y) = nP (X ≥ xn n) = o ne−xn n = o(1). Hence, ˆ

kP (X ≥ y)e−kζ(δx) + (kP (X ≥ y))2 ≤ 2kP (X ≥ y)



β β/2 ˆ = o ne−xn n = o e−kζ(x) , and condition (2.8) holds with H1 = 1 for all sufficiently large n. It follows by Lemma 2.4 that ˆ

P (Sk ≥ kx) ≤ 2e−kζ(x) + kP (X ≥ (1 − δ)kx)

page 48

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

49

Large Deviations for Sums

for all sufficiently large n. Taking into account (2.13) and the one-sided Linnik condition, we have √ kP (X ≥ (1 − δ)kx) ≤ nP (X ≥ (1 − δ)2 xn n)



2β β β/2 ˆ = o ne−(1−δ) xn n ) = o e−kζ(x) . This implies that ˆ

P (Sk ≥ kx) ≤ 3e−kζ(x) for all sufficiently large n. Hence, for every ε > 0, √ 2 P (Sn ≥ xn n) ≤ 3e−(1−ε)xn /2

(2.23)

for all sufficiently large n. It follows that √ 1 1−ε . lim sup 2 log P (Sn ≥ xn n) ≤ − xn 2 Passing to the limit as ε → 0, we conclude that √ 1 1 lim sup 2 log P (Sn ≥ xn n) ≤ − . xn 2

(2.24)

Test the conditions of Lemma 2.5. Take δ ∈ (0, 1). We have ∞



ˆ = − udF (u) + yP (X ≥ y) = o ye−yβ = o e−(1−δ)xβn nβ/2 . EX y

ˆ = o(x), hσn (h) = This and assertions (2.14) and (2.15) imply that −E X √ o( nfn (h)), and the conditions of Lemma 2.5 are satisfied. Hence, for every τ ∈ (0, 1), the inequalities √ P (Sn ≥ (1 − τ )3 xn n) ≥ P (Sk ≥ (1 − τ )2 kx) ˆ

2

≥ e−kζ(x)(1+τ ) ≥ e−xn (1+τ )

2

/2

(2.25)

hold for all sufficiently large n. Then lim inf

√ 1 (1 + τ )2 . log P (Sn ≥ (1 − τ )3 xn n) ≥ − 2 xn 2

Making the replacement xn = (1 − τ )3 xn in the last inequality, we have √ 1 (1 + τ ) lim inf 2 log P (Sn ≥ xn n) ≥ − . xn 2(1 − τ )6 Passing to the limit as τ → 0 in the latter inequality, we obtain √ 1 1 lim inf 2 log P (Sn ≥ xn n) ≥ − . xn 2 Relation (2.21) follows from (2.24) and (2.26).

(2.26)

page 49

September 13, 2019

15:48

50

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

We now replace the one-sided Linnik condition by a weaker power condition. Theorem 2.5. Let X, X1 , X2 , . . . be a sequence of i.i.d. random variables such that EX = 0, EX 2 = 1 and E(X + )p < ∞, p > 2. Then relation (2.21) holds for every sequence {xn } with xn → ∞ and lim sup x2n (log n)−1 ≤ p − 2. Proof. We need the next technical lemma. Lemma 2.12. If x2n ≤ (1 + τ )((p − 2) log n + p log log n) for τ > 0 and n ≥ n0 , then 2

(2−p)/2 x−p ≤ (p − 2)−p/2 e−xn /(2+2τ ) n n

for n ≥ n0 . 2

Proof. The function u−p eu (p − 2) log n, then

/2

strictly increases for u >

√ p. If x2n ≤

2

xn /2 x−p ≤ (p − 2)−p/2 (log n)−p/2 n(p−2)/2 n e

and, consequently, 2

(2−p)/2 x−p ≤ (p − 2)−p/2 (log n)−p/2 e−xn /2 . n n

If x2n ≥ (p − 2) log n, then 2

(2−p)/2 x−p ≤ (p − 2)−p/2 (log n)−p/2 n(2−p)/2 ≤ (p − 2)−p/2 e−xn /(2+2τ ) . n n

The lemma follows. √ √ Put un = xn / n and y = yn = n/xn for n ∈ N. Let h = hn be a solution of the equation mn (h) = un . Since un y = 1, relations (2.12)–(2.15) hold by Lemma 2.11. xn = h(1 + o(1)) in view of Put k = n, x = mn (h). Note that x = √ n (2.13). Check condition (2.8) of Lemma 2.4. By (2.15), we have ˆ k ζ(δx) = δ2

x2n (1 + o(1)), 2

where δ ∈ (0, 1]. Making use of the condition E(X + )p < ∞ and Lemma 2.12, we get  √ 

n kP (X ≥ y) = nP X ≥ = o xpn n(2−p)/2 = o(1). xn

page 50

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Large Deviations for Sums

51

This yields that

 ˆ −(1+δ 2 )x2n /2 2−p + o x2p kP (X ≥ y)e−kζ(δx) + (kP (X ≥ y))2 = o x2p n e n n





ˆ −(1+δ 2 )x2n /2 −x2n = o x2p + o x4p = o e−kζ(x) , n e n e and condition (2.8) is fulfilled with H1 = 1 for all sufficiently large n. It follows by Lemma 2.4 that ˆ

P (Sk ≥ kx) ≤ 2e−kζ(x) + kP (X ≥ (1 − δ)kx) for all sufficiently large n. E(X + )p < ∞, we have

Taking into account (2.13) and condition

√ (2−p)/2 . kP (X ≥ (1 − δ)kx) ≤ nP (X ≥ (1 − δ)2 xn n) = o x−p n n

Take ε > 0. Put τ = ε/(1 − ε). By Lemma 2.12, inequality (2.23) holds for all sufficiently large n. In the same way as in the proof of Theorem 2.4, we get (2.24). Check now the conditions of Lemma 2.5. We have ∞

 (1−p)/2 ˆ . n E X = − udF (u) + yP (X ≥ y) = o y 1−p = o xp−1 n y

ˆ = o(x), hσn (h) = This and relations (2.14) and (2.15) yield that −E X √ o( nfn (h)), and the conditions of Lemma 2.5 hold. The remainder repeats the end of the proof of the previous theorem. By Lemma 2.5, for every τ ∈ (0, 1) inequality (2.25) holds for all sufficiently large n. This yields (2.26). Relation (2.21) follows from (2.24) and (2.26). Theorems 2.3–2.5 imply that weaker moment assumptions give narrower zones in which large deviations have the Gaussian asymptotic. If the onesided Linnik condition is satisfied, then this zone is o(nβ/(4−2β) ). For β = 1, we arrive at the Cram´er zone which is widest possible by Theorem 2.2. If E(X + )p < ∞ for some p > 2, then the zone is logarithmic. Moreover, Theorems 2.3–2.5 yield that the behaviour of log P (Sn ≥ √ xn n) depends on the right-hand tail of the distribution of X more than on the left-hand one. The conditions of Theorems 2.3–2.5 are not symmetric in this sense. We only assume E(X − )2 < ∞. If we impose the same conditions for X − as for X + , then we get the Gaussian asymptotics for √ log P (Sn ≤ −xn n) as well.

page 51

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

52

9789811212826

Universal theory for strong limit theorems of probability

2.9

Asymptotic Expansions of Functions of Large Deviations Theory for D(2)

In this section, we derive asymptotic expansions of the functions of the LDT for random variables from the domains of non-normal attraction of the normal law. We first prove the following analogue of Lemma 2.8. The case of the finite exponential moment is included in there as well. Lemma 2.13. Assume that EX = 0, E(X + )2 < ∞ and F ∈ D(2). Assume that h = hn → 0 and one of the following conditions holds: 1) h0 > 0 and y = yn = ∞ for all n, 2) y = yn → ∞ and ∞ x2 ehx dFn (x) = O(1). (2.27) 0

Then

  1 h2 ϕn (h) = 1 + G (1 + o(1)), 2 h   1 (1 + o(1)), mn (h) = hG h   1 σn2 (h) = G (1 + o(1)), h   1 h2 G fn (h) = (1 + o(1)), 2 h

where G(x) =

0

(2.28) (2.29) (2.30) (2.31)

u2 dF (u), x > 0.

−x

Note that if condition 1) holds then ϕn (h) = ϕ(h), mn (h) = m(h), = σ 2 (h) and fn (h) = f (h) for h ∈ (0, h0 ) and all n.

σn2 (h)

Proof. Assume that condition 2) holds. Put e−1 (x) = ex . For k = 0, 1, 2, denote ek (x) = ex −

k  xj j=0

j!

,

0 k

Lk,n =

x e1−k (xh)dFn (x), −∞

Ik,n

+∞  = xk e1−k (xh)dFn (x). 0

page 52

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

53

Large Deviations for Sums

Then we have ∞ ehx dFn (x) = 1 + L0,n + I0,n ,

ϕn (h) = −∞

(2.32)

∞ xehx dFn (x) = L1,n + I1,n ,

(2.33)

∞ + (mn (h)) )ϕn (h) = x2 ehx dFn (x) = L2,n + I2,n .

(2.34)

ϕn (h)mn (h) = −∞

(σn2 (h)

2

−∞

By Lemma 2.9 and (2.27), we have in the same way as in Lemma 2.8 that Ik,n ≤ h

2−k

∞

x2 ehx dFn (x) = O(h2−k ),

k = 0, 1, 2.

(2.35)

0

To estimate Lk,n , we need the following result on properties of G(x). Lemma 2.14. If the conditions of Lemma 2.13 hold then G(x) ∈ SV∞ and  |u|dF (u) = o(G(x)), (2.36) x |u|>x

x2 P (|X| > x) = o(G(x)), x G(u)du ∼ xG(x),

(2.37) (2.38)

0

0

u3 dF (u) = o(xG(x))

(2.39)

−x

as x → ∞. Relations (2.36)–(2.38) may be found in [Feller (1971)]. Integrating by parts, one can easily derive relation (2.39) from (2.38). We write ⎞ ⎛ −1/h  0 ⎟ k ⎜ ¯ k,n + L ˆ k,n . + Lk,n = ⎝ ⎠ x e1−k (xh)dFn (x) = L −∞

−1/h

page 53

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

54

9789811212826

Universal theory for strong limit theorems of probability

Making use of Lemma 2.9, we get

−1/h 

¯ k,n | ≤ Dh−k |L

−∞

= Dh

1−k

−1/h+EY n 

|x − EYn |dF (x) ≤ Dh

|x|dFn (x) −∞ −1/h 

1−k

−∞

≤ Dh1−k

−1/h 

|xh|2 dFn (x) ≤ Dh1−k 1 + |xh|

|x − EYn |dF (x) −∞

−1/h 

|x|dF (x) + Dh1−k |EYn |P

−∞

  1 |X| ≥ , h

k = 0, 1, 2.

By (2.36) and (2.37), we arrive at    1 2−k ¯ G Lk,n = o h , h

k = 0, 1, 2.

(2.40)

Further, for k = 0, 1, 2, we write

0 ˆ k,n = L

h2−k x e2−k (xh)dFn (x) + (2 − k)!

0

k

−1/h

x2 dFn (x).

−1/h

We have 0

x2 dFn (x) =

−1/h

EY  n

−1/h+EYn

0 = −1/h+EYn

EY  n

(x − EYn )2 dF (x) =

x2 dF (x) + o(1) = G

−1/h+EYn

  1 (1 + o(1)). h

x2 dF (x) + o(1)

page 54

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

55

Large Deviations for Sums

Moreover, by Lemma 2.9, (2.37) and (2.39), we get 0 |x e2−k (xh)|dFn (x) ≤ Dh k

−1/h

3−k

0

|x|3 dFn (x)

−1/h EY  n

= Dh3−k

|x − EYn |3 dF (x)

−1/h+EYn

0

≤ Dh3−k

|x − EYn |3 dF (x)

−1/h+EYn

⎛

⎜ ≤ 8Dh3−k ⎝

0

−1/h+EYn

⎞   1 ⎟ |x|3 dF (x) + |EYn |3 P |X| ≥ + EYn ⎠ h

   1 = o h2−k G h for k = 0, 1, 2. It follows that

  1 (1 + o(1)), k = 0, 1, 2. h ¯ k,n yield that The latter and the above bounds for L   1 h2−k G Lk,n = (1 + o(1)), k = 0, 1, 2. (2 − k)! h ˆ k,n = L

h2−k G (2 − k)!

(2.41)

Relations (2.32), (2.35) and (2.41) imply (2.28). Assertions (2.28), (2.33), (2.35) and (2.41) yield (2.29). From (2.28), (2.29), (2.34), (2.35) and (2.41), we obtain (2.30). Relation (2.31) follows from (2.28) and (2.29). If condition 1) holds, the proof follows the same pattern. One has to put yn = ∞ for all n. The above calculations change evidently in this case. We omit details. The next result shows that one can replace (2.27) by simpler conditions. Lemma 2.15. Assume that EX = 0, E(X + )2 < ∞ and F ∈ D(2). Suppose that y = yn ∞, h = hn → 0 and one of the following conditions holds: + β 1) there exists β ∈ (0, 1) such that lim sup hn yn1−β < 1 and Ee(X ) < ∞. 2) hn yn = O(1). Then relations (2.28)–(2.31) hold.

page 55

September 13, 2019

56

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

The proof of Lemma 2.15 is similar to that of Lemma 2.10. We need only prove that (2.27) follows from conditions 1) or 2). To this end, we replace x3 by x2 in the definition of the integral Jn and note that bounds for the integral In require the condition E(X + )2 < ∞. Changes in the proof of Lemma 2.10 are obvious and we omit details. 2.10

Asymptotic Expansions of Functions of Large Deviations Theory for DN (α) and D(α)

In this section, we find asymptotic expansions for the LDT functions when F ∈ D(α) and F ∈ DN (α), α ∈ (1, 2). The cases of finite and infinite exponential moments are included both in the next result. Lemma 2.16. Put G(x) = xα F (−x) for x > 0. (Here G(x) ∈ SV∞ .) Assume that EX = 0, h = hn → 0 and one of the following conditions holds: 1) h0 > 0 and y = yn = ∞ for all n, 2) y = yn → ∞ and    ∞ 1 2−α 2 hx x e dFn (x) = o G h . (2.42) h 0

If F ∈ D(α), α ∈ (1, 2), then

  1 (1 + o(1)), (2.43) ϕn (h) = 1 + c1 h G h   1 mn (h) = c2 hα−1 G (1 + o(1)), (2.44) h   1 (1 + o(1)), (2.45) σn2 (h) = c3 hα−2 G h   1 fn (h) = c4 hα G (1 + o(1)), (2.46) h where c1 = Γ(2 − α)/(α − 1), c2 = αΓ(2 − α)/(α − 1), c3 = αΓ(2 − α) and c4 = Γ(2 − α). Here Γ(x) is the gamma-function. If F ∈ DN (α), α ∈ (1, 2), then hα ϕn (h) = 1 + (1 + o(1)), (2.47) α mn (h) = hα−1 (1 + o(1)), (2.48) α

σn2 (h) = (α − 1)hα−2 (1 + o(1)), α−1 α fn (h) = h (1 + o(1)). α

(2.49) (2.50)

page 56

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

57

Large Deviations for Sums

Note that if condition 1) holds then ϕn (h) = ϕ(h), mn (h) = m(h), σn2 (h) = σ 2 (h) and fn (h) = f (h) for h ∈ (0, h0 ) and all n. Proof. Assume that condition 2) holds. Suppose first that F ∈ D(α), α ∈ (1, 2). By Theorem 2.1, we get G(x) ∈ SV∞ . Let Lk,n and Ik,n be the integrals from the proof of Lemma 2.13. We write (2.32)–(2.34) and investigate the behaviour of Lk,n and Ik,n . By Lemma 2.9 and (2.42), we have Ik,n ≤ h2−k

∞ 0

   1 x2 ehx dFn (x) = o hα−k G , h

k = 0, 1, 2. (2.51)

Put rk (x) = xk e1−k (x), k = 0, 1, 2. Then an integration by parts yields Lk,n = h−k

= h−k

∞ 0

0

rk (hx)dFn (x) = h−k+1

−∞

0

Fn (x)rk (hx)dx

−∞

x

r (−x)dx. Fn − h k

We now check that ∞ 1

  ∞ x

1  α rk (−x)dx ∼ h G Fn − x−α rk (−x)dx, h h

k = 0, 1, 2. (2.52)

1

Since Fn (−x) = F (−x + EYn ) for x ≥ 0, EYn < 0 and EYn → 0, we have x

ρx

x hα x

ρx

hα − = F − ≤ F ≤ F − = αG G n (ρx)α h h h h x h for every ρ > 1, all x ≥ ε and all sufficiently large n. In view of r0 (−x) = e−x − 1 ≤ 0 and r1 (−x) = e−x − 1 − xe−x ≤ 0 for x ≥ 0, it follows by Theorem 2.6 on p. 64 in [Seneta (1976)] that ∞ 1

∞

x

x −α   α rk (−x)dx ≥ h x rk (−x)dx Fn − G h h 1

  ∞ 1 α ∼h G x−α rk (−x)dx h 1

page 57

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

58

9789811212826

Universal theory for strong limit theorems of probability

for k = 0, 1. By the same argument, we have ∞ ∞

x  hα ρx −α  r (−x)dx ≤ α x rk (−x)dx Fn − G h k ρ h 1

1

  ∞ 1 hα ∼ αG x−α rk (−x)dx ρ h 1

for k = 0, 1. Since ρ may be chosen arbitrary close to 1, we arrive at (2.52) for k = 0, 1. Further, r2 (−x) = x(x − 2)e−x . Hence r2 (−x) ≤ 0 for x ∈ (1, 2) and  r2 (−x) ≥ 0 for x ≥ 2. In the same way as before, one can check that   ∞ ∞ 1 x  α r (−x)dx ∼ h G Fn − x−α r2 (−x)dx h 2 h 2

2

and 2 1

  2 x

1  α r (−x)dx ∼ h G Fn − x−α r2 (−x)dx. h 2 h 1

The latter two relations imply (2.52) for k = 2. We now prove that 1 0

  1 x

1  α rk (−x)dx ∼ h G Fn − x−α rk (−x)dx, h h

k = 0, 1, 2. (2.53)

0

It is clear that for every ρ > 1 and all sufficiently large n the following inequalities 1

1 1 x

ρx

x    rk (−x)dx ≥ Fn − rk (−x)dx ≥ F − r (−x)dx F − h h h k 0

ρ−1

0

hold for k = 0, 1, 2. Theorem 2.7 on p. 66 in [Seneta (1976)] yields 1 0

  1 x

1  α r (−x)dx ∼ h G F − x−α rk (−x)dx, h k h

k = 0, 1, 2.

0

By relation (2.11) on p. 63 in [Seneta (1976)], we have 1 ρ−1

  1 x

1  α r F − (−x)dx ∼ h G x−α rk (−x)dx, h k h ρ−1

k = 0, 1, 2.

page 58

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

59

Large Deviations for Sums

Since ρ may be chosen arbitrarily close to 1, the last two relations yield (2.53). It follows from (2.52) and (2.53) that   1 α−k G Lk,n = ck+1 h (1 + o(1)), k = 0, 1, 2, (2.54) h where ∞ c1 = 0

∞ c2 = 0

∞ c3 =

x−α (1 − e−x )dx =

Γ(2 − α) , α−1

x−α (1 − e−x + xe−x )dx =

αΓ(2 − α) , α−1

x−α (2x − x2 )e−x dx = αΓ(2 − α).

0

Relations (2.32), (2.51) and (2.54) imply (2.43). Assertions (2.43), (2.33), (2.51) and (2.54) yield (2.44). From (2.44), (2.43), (2.34), (2.51) and (2.54), we get (2.45). Relation (2.46) follows from (2.43) and (2.44). Finally, we assume that F ∈ DN (α), α ∈ (1, 2). The proof is the same as that above. We need only mention that Theorem 2.6.7 and the last two formulae before the case (3) on p. 22 in [Ibragimov and Linnik (1971)] yield that α−1 (1 + o(1)) as x → ∞. F (−x) = x−α αΓ(2 − α) It follows that

  1 α−1 (1 + o(1)). G = h αΓ(2 − α)

Hence, we get (2.47)–(2.50) instead of (2.43)–(2.46). If condition 1) holds, the proof follows the same pattern. One has to put yn = ∞ for all n. The above calculations change evidently in this case. We omit details. The next result allows to replace condition (2.42) by simpler ones. Lemma 2.17. Assume that EX = 0, y = yn → ∞, h = hn → 0 and one of the following conditions holds: + β 1) there exists β ∈ (0, 1) such that lim sup hn yn1−β < 1 and Ee(X ) < ∞.

page 59

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

60

9789811212826

Universal theory for strong limit theorems of probability

2) hn yn = O(1). If F ∈ D(α), α ∈ (1, 2) then relations (2.43)–(2.46) hold. If F ∈ DN (α), α ∈ (1, 2) then relations (2.47)–(2.50) hold. Proof. We need the next result that is a consequence of the asymmetry of stable laws under consideration. Lemma 2.18. If F ∈ DN (α) or F ∈ D(α), α ∈ (1, 2), then 1 − F (x) = o(F (−x)) and x  u2 dF (u) = o x2−α G(x) as x → ∞. 0

Proof. From [Feller (1971)], we have the first relation and μ(x) ∼ x2−α G(x) as x → ∞, where μ(x) = μ+ (x) + μ− (x), −

0

μ (x) =

2

u dF (u),

+

x

μ (x) =

−x

u2 dF (u).

0

An integrating by parts gives x 2 u(1 − F (u))du = μ+ (x) + x2 (1 − F (x)), 0

x 2

uF (−u)du = μ− (x) + x2 F (−x).

0

Take ε > 0. Then 1 − F (x) ≤ εF (−x) for all x ≥ x0 = x0 (ε) and x x x u(1 − F (u))du ≤ x20 + ε uF (−u)du ≤ 2ε uF (−u)du 0

x0

0

for all sufficiently large x. Here we have used that the last integral tends to infinity. Hence, μ+ (x) = o(μ− (x) + x2 F (−x)) = o(μ(x) + x2 F (−x)) as x → ∞ and the second relation follows. The proof of Lemma 2.17 is similar to that of Lemma 2.10. Assume that condition 2) holds. Then ∞ zn 2 hx Jn = x e dFn (x) = x2 ehx dFn (x + EYn ) + zn2 ehzn P (X ≥ y). 0

0

page 60

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Large Deviations for Sums

9789811212826

61

In the same way as in Lemma 2.10, we obtain zn y 2 hx In = x e dFn (x + EYn ) ≤ C x2 ehx dF (x). 0

0

Taking into account that y ≤ C1 /h and Lemma 2.18, we have In = o(hα−2 G(1/h)). Applying again Lemma 2.18, we get   zn2 ehzn P (X ≥ y) = o y 2 F (−y) = o y 2−α G(y) . By [Seneta (1976)], there exists a non-decreasing function R(x) ∈ RV∞ such that R(y) ∼ y 2−α G(y). Since R(y) ≤ R(C1 /h) ∼ C12−α R(1/h), we have that    1 2 hzn α−2 G zn e P (X ≥ y) = o h . h Hence, condition (2.42) holds and the result follows from Lemma 2.16. If condition 1) is satisfied, then calculations are similar to those in Lemma 2.10. One has to replace x3 by x2 only. Then we get Jn = O(1) which yields (2.42) in view of G(x) ∈ SV∞ . 2.11

Large Deviations for D(2)

In this section, we discuss the asymptotic behaviour of the large deviation probabilities for random variables from the domain of non-normal attraction of the normal law. We will use the following notations. For g(x) ∈ RV∞ (g(x) ∈ RV0 ), let −1 g (x) be an asymptotically inverse function, i.e. g(g −1 (x)) ∼ g −1 (g(x)) ∼ x as x → ∞ (x → 0). The asymptotically inverse function is asymptotically unique. One can find details in [Seneta (1976)] for RV∞ . We consider the case RV0 below. Our first result is for random variables satisfying the one-sided Cram´er condition. Theorem 2.6. Assume that h0 > 0, EX = 0 and F ∈ D(2). For h > 0 and x > 0, put    

1 1 h2 G (2.55) , fˆ(h) = , γˆ (x) = m ˆ fˆ−1 (x) m(h) ˆ = hG h 2 h 0 2 u dF (u). Then for every sequence {xn } with xn → 0 and where G(x) = −x

nxn → ∞, the following relation holds γ (xn )) = −nxn (1 + o(1)). log P (Sn ≥ nˆ

(2.56)

page 61

September 13, 2019

62

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

The functions m(h), ˆ fˆ(h) and γˆ(x) are the main terms of asymptotic expansions for m(h), f (h) and γ(x) at zero, correspondingly. Moreover, ˆ ∈ RV0 and fˆ(h) ∈ RV0 . G(x) ∈ SV∞ , m(h) In view of an obvious replacement, the conclusion of Theorem 2.6 means that  2   xn x2 (2.57) γ ∼− n log P Sn ≥ nˆ 2n 2 √ for every sequence {xn } with xn → ∞ and xn = o( n). Note that F ∈ DN (2) formally corresponds to the case G(x) = 1 for all x for which γˆ (x) = √ 2x and (2.57) turns to (2.21) of Theorem 2.3. Proof. We first prove three lemmas. Lemma 2.19. For every sequence {xn } of positive numbers, one has γn (xn (1 + o(1))) = γn (xn )(1 + o(1)), ζn (xn (1 + o(1))) = ζn (xn )(1 + o(1)). Proof. Let {εn } be a sequence of real numbers such that εn → 0. The concavity and monotonicity of γn (x) and γn (0) = 0 yield that (1 − |εn |)γn (xn ) ≤ γn (xn (1 + εn )) ≤ (1 + |εn |)γn (xn ) for all n. This implies the first relation. The second one follows by the same way in view of the convexity and monotonicity of ζn (x) and ζn (0) = 0. Lemma 2.20. Assume that f (x) = xα G(1/x), where G(x) ∈ SV∞ . Then there exists a function L(x) ∈ SV∞ such that the function f −1 (x) = x1/α L(1/x) is the asymptotically inverse function to f (x). Moreover, if f−1 (x) is another asymptotically inverse function then f−1 (x) ∼ f −1 (x) as x → 0. Proof. By the representation of a slowly varying function (see Theorem 1.2 on p.2 [Seneta (1976)]), there exist functions η(t), η(t) → const as t → ∞, and (t), (t) → 0 as t → ∞, and a positive constant B such that 1/x 1/x   α − (t)   (t) − α  dt ∼ C exp − dt = Cr1 (x), f (x) = exp η(1/x) + t t B

B

as x → 0. We further assume that x ≤ 1/B and α − (t) > 0 for t ≥ B. The function r1 (x) is continuous, strictly increasing on [0, 1/B] and r1 (0) = 0. Let r2 (x) be the inverse function to r1 (x). Hence, r2 (x) is

page 62

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

63

Large Deviations for Sums

continuous, strictly increasing on [0, r1 (1/B)] and r2 (0) = 0. We have from r2 (r1 (x)) = r1 (r2 (x)) = x that r2 (r1 (x))r1 (x) = r1 (r2 (x))r2 (x) = 1. This yields that 1 r2 (r1 (x))r1 (x) = . x α − (1/x) Putting x = r2 (t) in the last relation, we have r2 (t)t 1 1 α−2 (1/r2 (t)) 1 = = + = + 1 (t) −1 r2 (t) α − (1/r2 (t)) α 1 − α (1/r2 (t)) α for all t ≤ B1 = r1 (1/B), where 1 (t) → 0 as t → 0. It follows that ⎫ ⎧ ⎫ ⎧ B 1/x ⎪ ⎬ ⎨  α−1 +  (1/u) ⎪ ⎨  1 α−1 +  (t) ⎬ 1 1 r2 (x) = exp − dt = exp − du ⎪ ⎭ ⎪ ⎩ t u ⎭ ⎩ = x1/α exp

⎧ ⎪ ⎨ ⎪ ⎩

x

1/x B2 − 1/B1

⎫ ⎪ ⎬

1/B1

2 (u) du = x1/α (1/x). ⎪ u ⎭

Here (t) ∈ SV∞ by the representation of a slowly varying function. Put f −1 (x) = x1/α L(1/x), where L(x) = C −1/α (x). By the uniform convergence theorem (Theorem 1.1 on p.2 in [Seneta (1976)]), we have f (f −1 (x)) ∼ Cr1 (C −1/α r2 (x)) = Cr1 (r2 (C −1 x(1 + o(1)))) ∼ x, and f −1 (f (x)) = C −1/α r2 (Cr1 (x)(1 + o(1))) = C −1/α r2 (r1 (C 1/α x(1 + o(1)))) ∼ x, as x → 0. Check that f −1 (x) is asymptotically unique. For another asymptotically inverse function f−1 (x), the uniform convergence theorem yields f−1 (x) ∼ f −1 (f (f−1 (x))) = f −1 (x(1 + o(1))) ∼ f −1 (x) as x → 0. Lemma 2.21. If m(h), ˆ fˆ(h) and γˆ (x) are defined by formulae (2.55), then there exists a function L(x) ∈ SV∞ such that   √ 1 fˆ−1 (x) = 2xL , (2.58) x √ 2x as x → 0. (2.59) γˆ(x) ∼ L (1/x)

page 63

September 13, 2019

15:48

64

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Proof. An application of Lemma 2.20 yields relation (2.58). This and (2.55) imply that     √ 1 1 G √ γˆ (x) = 2xL . x 2xL (1/x) In view of

  2   1 1 −1 ˆ ˆ G √ x ∼ f (f (x)) = x L x 2xL (1/x)

as x → 0,

we obtain (2.59). Note that L(x) → 0 as x → ∞ in Theorem 2.6 because G(x) → ∞ as x → ∞. Put yn = ∞ and hn = fˆ−1 (xn ) for all n. By Lemmas 2.13 and 2.19 and the definition of γ(x), we have γ(xn ) = γ(fˆ(hn )) = γ(f (hn )(1 + o(1))) ∼ γ(f (hn )) = m(hn ) ∼ m(h ˆ n ) ∼ γˆ (xn ). By Lemma 2.2, we get log P (Sn ≥ nγ(xn )) ≤ −nxn .

√ γ (xn ). Then Take ε > 0. We have by (2.59) that γˆ((1 + ε)xn ) ∼ 1 + εˆ γ(xn ) ≤ γˆ ((1 + ε)xn ) for all sufficiently large n. It yields that γ ((1 + ε)xn )) ≤ P (Sn ≥ nγ(xn )) P (Sn ≥ nˆ for all sufficiently large n. Hence, γ ((1 + ε)xn )) ≤ −nxn log P (Sn ≥ nˆ for all sufficiently large n. Replacing (1 + ε)xn by xn , we have xn log P (Sn ≥ nˆ γ (xn )) ≤ −n 1+ε for all sufficiently large n. The latter implies that lim sup

1 1 . log P (Sn ≥ nˆ γ (xn )) ≤ − nxn 1+ε

Passing to the limit as ε → 0 in the last inequality, we obtain 1 log P (Sn ≥ nˆ γ (xn )) ≤ −1. (2.60) nxn √ √ By Lemma 2.13, the relations nσ(hn ) = o(nm(hn )) and hn nσ(hn ) = o(nf (hn )) hold. lim sup

page 64

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Large Deviations for Sums

65

√ Take ε ∈ (0, 1). Since γˆ((1 − ε)xn ) ∼ 1 − εˆ γ (xn ), the inequality √ γ ((1 − ε)xn ) holds for all sufficiently large n. nm(hn ) − 2 nσ(hn ) ≥ nˆ Hence, √ γ ((1 − ε)xn )) P (Sn ≥ nm(hn ) − 2 nσ(hn )) ≤ P (Sn ≥ nˆ for all sufficiently large n. Lemma 2.2 implies that γ ((1 − ε)xn )) ≥ −nxn (1 + ε) log P (Sn ≥ nˆ for all sufficiently large n. Replacing (1 − ε)xn by xn , we get 1+ε γ (xn )) ≥ −nxn log P (Sn ≥ nˆ 1−ε for all sufficiently large n. It follows that 1 1+ε lim inf log P (Sn ≥ nˆ γ (xn )) ≥ − . nxn 1−ε Passing to the limit as ε → 0, we arrive at 1 log P (Sn ≥ nˆ γ (xn )) ≥ −1. lim inf nxn The last relation and (2.60) yield (2.56). We deal with the one-sided Linnik condition in the next result. + β

Theorem 2.7. Assume that Ee(X ) < ∞, EX = 0 and F ∈ D(2). Define m(h), ˆ fˆ(h) and γˆ (x) by formulae (2.55). Let L(x) be the function from (2.58). Then relation (2.56) holds for every sequence {xn } with nxn → ∞ (2−β)/2 (L(1/xn ))β → 0. and n1−β xn Writing (2.56) in form of (2.57), we see that Theorem 2.7 corresponds to Theorem 2.4 in which F ∈ DN (2) and G(x) = L(x) = 1 for all x. Proof. The second condition on xn yields that xn → 0 in Theorem 2.7. ˆ fˆ−1 (xn )) and h = hn = fˆ−1 (xn ). Relations (2.58) Put y = yn = nm( and (2.59) imply that y → ∞, h → 0 and hy 1−β = o(1). By Lemma 2.15, relations (2.28)–(2.31) hold. By Lemmas 2.15 and 2.19 and the definition of γn (x), we have γn (xn ) = γn (fˆ(hn )) = γn (fn (hn )(1 + o(1))) ˆ n ) ∼ γˆ (xn ). ∼ γn (fn (hn )) = mn (hn ) ∼ m(h We will apply Lemma 2.4 with k = n and x = γˆ (xn ). We have by (2.31) that ˆ ∼ fˆ(h) = xn . ζn (xn ) = ζ(x)

page 65

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

66

9789811212826

Universal theory for strong limit theorems of probability

By relation (2.59), we get (nˆ γ (xn ))β n−β/2 ∼

(2nxn )β/2 →∞ (L(1/xn ))β

and (β−2)/2

(nˆ γ (xn ))β nβ−1 xn ∼ 2β/2 → ∞. nxn (L(1/xn ))β These relations and the one-sided Linnik condition imply that



 β β kP (X ≥ y) = o ne−(nˆγ (xn )) = o e−(nˆγ (xn )) (1+o(1)) = o e−2nxn = o(1). Therefore ˆ

kP (X ≥ y)e−kζ(δx) + (kP (X ≥ y))2 ≤ 2kP (X ≥ y) = o(e−2nxn ), and condition (2.8) holds with H1 = 1 for all sufficiently large n. It follows by Lemma 2.4 that ˆ

P (Sk ≥ kx) ≤ 2e−kζ(x) + kP (X ≥ (1 − δ)kx) for all sufficiently large n. Taking again into account the asymptotic behaviour of (nˆ γ (xn ))β and the Linnik condition, we have

 β kP (X ≥ (1 − δ)kx) = o ne−((1−δ)nˆγ (xn )) = o e−2nxn . This implies that ˆ

P (Sk ≥ kx) ≤ 3e−kζ(x) for all sufficiently large n. Hence, for every ε ∈ (0, 1), the inequality γ (xn )) ≤ 3e−(1−ε)nxn P (Sn ≥ nˆ holds for all sufficiently large n. It follows that lim sup

1 log P (Sn ≥ nˆ γ (xn )) ≤ −(1 − ε). nxn

Passing to the limit as ε → 0, we conclude that lim sup

1 log P (Sn ≥ nˆ γ (xn )) ≤ −1. nxn

(2.61)

Now we check that the conditions of Lemma 2.5 hold. Take δ ∈ (0, 1). We have ∞



β β ˆ E X = − udF (u) + yP (X ≥ y) = o ye−y = o e−(1−δ)(nˆγ (xn )) . y

page 66

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

67

Large Deviations for Sums

ˆ = o(x), hσn (h) = This and assertions (2.30) and (2.31) imply that −E X √ o( nfn (h)), and the conditions of Lemma 2.5 are satisfied. γ (xn ) ≥ (1 − Take τ ∈ (0, 1). Since nˆ γ ((1 − τ )3 xn ) ∼ (1 − τ )3/2 nˆ 2 τ ) nˆ γ (xn ) for all sufficiently large n, by Lemma 2.5, the inequalities ˆ

γ ((1 − τ )3 xn )) ≥ P (Sk ≥ (1 − τ )2 kx) ≥ e−kζ(x)(1+τ ) P (Sn ≥ nˆ hold for all sufficiently large n. Then 1 log P (Sn ≥ nˆ γ ((1 − τ )3 xn )) ≥ −(1 + τ ). lim inf nxn Replacing (1 − τ )3 xn by xn in the last inequality, we have 1 1+τ lim inf log P (Sn ≥ nˆ γ (xn )) ≥ − . nxn (1 − τ )3 Passing to the limit as τ → 0 in the latter inequality and taking into account (2.61), we get (2.56). In the following result, we consider a power moment condition. Theorem 2.8. Assume that EX = 0, E(X + )p < ∞ for some p > 2 and F ∈ D(2). Define m(h), ˆ fˆ(h) and γˆ (x) by formulae (2.55). Let L(x) be the function from (2.58). Then relation (2.56) holds for every sequence {xn } with nxn → ∞ and lim sup(2nxn )/ log n ≤ p − 2. Writing relation (2.56) in form of (2.57), we conclude that Theorem 2.8 corresponds to Theorem 2.5 where F ∈ DN (2) and G(x) = L(x) = 1 for all x. Proof. Note that xn → 0 in the assumptions of Theorem 2.8. Put h = hn = fˆ−1 (xn ) and y = yn = 1/hn for all n. By (2.58), we have h → 0 and y → ∞. Lemma 2.15 implies that relations (2.28)–(2.31) hold. Put k = n and x = γˆ (xn ). We will check that the conditions of Lemma 2.4 hold. For δ ∈ (0, 1], we have ˆ γ (xn )) = ζn (ˆ γ (δ 2 xn )(1+o(1))) = ζn (γn (δ 2 xn (1+o(1))) ∼ δ 2 xn . ζ(δx) = ζn (δˆ It follows by Lemma 2.12 that for every ε ∈ (0, 1),

n1−p x−p/2 = O e−nxn (1−ε) . n The condition E(X + )p < ∞ yields that    p 

1 kP (X ≥ y) = o nxp/2 L = o (nxn )p e−nxn (1−ε) = o(1). n xn

page 67

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

68

9789811212826

Universal theory for strong limit theorems of probability

It follows that ˆ

kP (X ≥ y)e−kζ(δx) + (kP (X ≥ y))2 =



 2 = o (nxn )p e−(1−ε+δ )nxn + o (nxn )2p e−2nxn (1−ε) = o e−nxn , provided δ 2 > ε. Hence, the conditions of Lemma 2.4 hold for all sufficiently large n. Then P (Sk ≥ kx) ≤ 2e−nxn + kP (X ≥ (1 − δ)kx) for all sufficiently large n. Making use of E(X + )p < ∞ again, we have    p 

1 1−p −p/2 kP (X ≥ (1 − δ)kx) = o n xn L = o e−nxn (1−ε) . xn This implies that P (Sk ≥ kx) ≤ 3e−nxn (1−ε) for all sufficiently large n. The rest of the proof for the upper bound coincides with that of the previous theorem. We omit details and turn to the lower bound. To this end, we check the conditions of Lemma 2.5. We have    p−1  ∞ 1 1−p (p−1)/2 ˆ L ) = o xn . E X = − udF (u)+yP (X ≥ y) = o(y xn y

ˆ = o(x). By hσn (h) = o(√nfn (h)), we conThe latter implies that −E X clude that the conditions of Lemma 2.5 hold. The remainder of the proof for the lower bound is the same as that of Theorem 2.7. Details are omitted. 2.12

Large Deviations for DN (α) and D(α)

We turn now to large deviations for F ∈ DN (α) and F ∈ D(α) with α ∈ (1, 2). We again start with random variables satisfying the Cram´er condition. Theorem 2.9. Assume that h0 > 0, EX = 0 and α ∈ (1, 2). If F ∈ D(α), then for h > 0 and x > 0, put    

1 1 m(h) ˆ = c2 hα−1 G , fˆ(h) = c4 hα G , γˆ (x) = m ˆ fˆ−1 (x) , (2.62) h h

page 68

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Large Deviations for Sums

9789811212826

69

where G(x) = xα F (−x), c2 = Γ(2 − α)/(α − 1) and c4 = Γ(2 − α). If F ∈ DN (α), then put x λ α−1 α h , γˆ (x) = (2.63) m(h) ˆ = hα−1 , fˆ(h) = α λ for h > 0 and x > 0, where λ = (α − 1)/α. Then for every sequence {xn } with xn → 0 and nxn → ∞, the following relation holds γ (xn )) = −nxn (1 + o(1)). log P (Sn ≥ nˆ

(2.64)

Note that the functions m(h), ˆ fˆ(h) and γˆ(x) are the main terms of asymptotic expansions for m(h), f (h) and γ(x) at zero. Moreover, G(x) ∈ ˆ ∈ RV0 and fˆ(h) ∈ RV0 . SV∞ , m(h) For F ∈ DN (α), relation (2.64) imply that

(2.65) log P Sn ≥ xn n1/α ∼ −λx1/λ n for every sequence {xn } with xn → ∞ and xn = o(nλ ). For α = 2, this coincides with the result of Theorem 2.3. For F ∈ D(α), the cases α < 2 and α = 2 are quite different. This is the result of a difference of conditions necessary and sufficient for attraction to the normal law and the stable laws with index α < 2. The function G(x) is the truncated second moment for α = 2 and G(x) → ∞ as x → ∞. For α < 2, it is the slowly varying part of the left-hand tail of the distribution of X and G(x) → ∞ or 0 as x → ∞. The proof of Theorem 2.9 is similar to that of Theorem 2.6. We only need the next analogue of Lemma 2.21. Lemma 2.22. If m(h), ˆ fˆ(h) and γˆ (x) are defined by formulae (2.62), then there exists a function L(x) ∈ SV∞ such that  1/α   x 1 −1 ˆ f (x) = L , (2.66) c4 x γˆ (x) ∼

c2 xλ λ c4 L (1/x)

as

x → 0.

(2.67)

For F ∈ DN (α), we may directly use formulae (2.63) instead of Lemma 2.22 and calculations are simpler then. The proof of Lemma 2.22 repeats that of Lemma 2.21 and therefore we omit details. Note that L(x) → 0 or ∞ in Lemma 2.22 while it only tends to ∞ for an attraction to the normal law. Moreover, L(x) depends on the tail of

page 69

September 13, 2019

70

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

the distribution of X. For D(2), it is determined by the truncated second moment. Now the proof of Theorem 2.9 repeats that of Theorem 2.6. We only mention that γˆ (cxn ) ∼ cλ γˆ (xn ) for every fixed c. Changes of the calculations are obvious. Hence, we omit details. We now turn to a result under the Linnik condition. + β

Theorem 2.10. Assume that Ee(X ) < ∞ for some β ∈ (0, 1), EX = 0 and α ∈ (1, 2). For F ∈ D(α), define m(h), ˆ fˆ(h) and γˆ (x) by relaˆ tion (2.62). For F ∈ DN (α), define m(h), ˆ f (h) and γˆ (x) by (2.63). Then relation (2.64) holds for every sequence {xn } with nxn → ∞, −1/λ β → ∞ and n1−β x1−βλ (L (1/xn )) → 0. nxn (L (1/xn )) n Note that the second condition on nxn in Theorem 2.10 disappears provided L(x) → 0 or c as x → ∞. The latter holds for F ∈ DN (α). Remember that L(x) → 0 as x → ∞ for F ∈ D(2). This was used in the proof of Theorem 2.7. When F ∈ D(α), α < 2, we may have L(x) → ∞ as x → ∞ which gives a stronger condition on nxn . Theorem 2.10 implies that for F ∈ DN (α), relation (2.65) holds for  βλ(1−λ)/(1−λβ) . For every sequence {xn } with xn → ∞ and xn = o n α = 2, this corresponds to Theorem 2.4. The proof of Theorem 2.10 repeats that of Theorem 2.7. We need only apply Lemma 2.22 instead of Lemma 2.21 and γˆ(cxn ) ∼ cλ γˆ (xn ) for every fixed c. One can obviously change the calculations. Details are omitted. We finally discuss the case of power moment assumptions. Theorem 2.11. Assume that E(X + )p < ∞ for some p > α, EX = 0 and α ∈ (1, 2). For F ∈ D(α), define m(h), ˆ fˆ(h) and γˆ (x) by relation (2.62). ˆ For F ∈ DN (α), define m(h), ˆ f (h) and γˆ (x) by (2.63). Then relation (2.64) holds for every sequence {xn } with nxn (max{1, log L(1/xn )})−1 → ∞ and lim sup(αnxn )/ log n ≤ p − α. Note that the first condition on nxn in Theorem 2.11 turns to nxn → ∞ when L(x) → 0 or c as x → ∞. The latter holds provided F ∈ DN (α). Remember that L(x) → 0 as x → ∞ for F ∈ D(2). When F ∈ D(α), α < 2, we may also have L(x) → ∞ as x → ∞ which yields a stronger condition on the rate of the growth of nxn . If F ∈ DN (α), then by Theorem 2.11, relation (2.65) holds for for every 1/λ sequence {xn } with xn → ∞ and lim sup xn / log n ≤ (p − α)/(α − 1). For α = 2, this is the result of Theorem 2.5.

page 70

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

71

Large Deviations for Sums

The proof of Theorem 2.11 is the same as that of Theorem 2.8. We need only apply Lemma 2.22 instead of Lemma 2.21 and γˆ (cxn ) ∼ cλ γˆ (xn ) for every fixed c. Calculations may be changed obviously. We omit details. 2.13

Large Deviations and the Classification of Distributions

In this section, we give generalizations of the results of Section 2.2 related to the classification of distributions from Section 2.4. Lemma 2.23. Assume that X is non-degenerate, EX ≥ 0 and h0 > 0. Put h0 = ∞ for F ∈ K5 and h0 = h0 otherwise. Then P (Sn ≥ nm(h)) ≤ e−nf (h)

(2.68)

for all h ∈ (0, h0 ) and n. Lemma 2.23 is a slight generalization of Lemma 2.2 for F ∈ K5 . It follows from the Tchebyshev inequality and the definitions of m(h) and f (h). Lemma 2.24. Assume that X is non-degenerate, EX ≥ 0 and h0 > 0. Let {hn } be a sequence of positive numbers. Assume that one of the following condition holds: 1) hn < h0 , nf (hn ) → ∞ and hn = O(1); 2) F ∈ K1 ∪ K2 and hn → ∞; 3) F ∈ K3 ∪ K5 , hn → ∞ and P (X ≥ (1 − τ )m(h)) ≥ e−(1+δ)f (h)

(2.69)

for all τ > 0, δ > 0 and all sufficiently large h; 4) F ∈ K4 , hn h0 and inequality (2.69) holds for all τ > 0, δ > 0 and all h sufficiently close to h0 ; 5) F ∈ K5 and hn = h∗ > h0 . Then P (Sn ≥ (1 − ε)nm(hn )) ≥ e−nf (hn )(1+δ) for all ε > 0, δ > 0 and all sufficiently large n. Proof. We first prove an auxiliary result. √ Lemma 2.25. If condition 1) holds, then hn σ(hn ) = o( nf (hn )).

(2.70)

page 71

September 13, 2019

72

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Proof. If 0 < ε ≤ hn ≤ 1/ε for all large n, then 1 sup σ(h) = O(1) hn σ(hn ) ≤ ε ε≤h≤1/ε √ √ and nf (hn ) ≥ nf (ε) → ∞. Thus, we examine the case hn → 0. For EX 2 < ∞, the result follows by Lemma 2.7. Assume that EX 2 = ∞. For h > 0, put Yh = min{h2 X 2 , 1}. Then EYh = h

2

1/h

x2 dF (x) + P (|X| ≥ 1/h).

−1/h 2

It is clear that h = o(EYh ) as h → 0. Applying the inequality log x ≤ x − 1 for x ≥ 1, we have f (h) ≥ hm(h) − ϕ(h) + 1 = 1 − ϕ(h) + hϕ (h) − hm(h)(ϕ(h) − 1). Taking into account that 1 − ev + vev ≥ (1 − 2e−1 ) min{v 2 , 1} for all real v, we have 1 − ϕ(h) + hϕ (h) = E(1 − ehX + hXehX ) ≥ (1 − 2e−1 )EYh . Further, for h ≤ ε0 = min{h0 /2, 1}, we get ∞ ∞ ∞ hx hx ϕ(h) − 1 ≤ (e − 1)dF (x) ≤ hxe dF (x) ≤ h xeε0 x dF (x). 0

0

0

It follows that ϕ(h) − 1 = O(h) as h → 0. Moreover, ∞  hm(h)ϕ(h) = hϕ (h) ≤ h xehx dF (x) 0

which yields that hm(h) = O(h) in view of ϕ(h) → 1 as h → 0. Hence, hm(h)(ϕ(h) − 1) = O(h2 ) = o(EYh ) as h → 0. Finally, we get 1 f (h) ≥ (1 − 2e−1 )EYh 2 for all small h. From the other hand, in view of v 2 ev ≤ 4e2 min{v 2 , 1} for v ≤ 1, we have ∞ ∞ 2 2 2 2 hx 2 2 h x e dF (x) ≤ 4e EYh + h x2 eε0 x dF (x) h σ (h)ϕ(h) ≤ −∞

−∞

provided h ≤ ε0 . It follows that h2 σ 2 (h) ≤ 5e2 EYh ≤ 10e2 (1 − 2e−1 )−1 f (h) for all small h. This yields the result.

page 72

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

73

Large Deviations for Sums

If condition 1) holds and EX = 0 then inequality (2.70) follows from Lemmas 2.2 and 2.25. If EX > 0, then P (Sn ≥ (1 − ε)nm(hn )) ≥ P (Sn − nEX ≥ (1 − ε)n(m(hn ) − EX)). Note that a shift of X does not change f (h) while m(h) increases on the value of the shift. Hence, the result follows. Suppose that condition 2) is satisfied. Then P (Sn ≥ (1 − ε)nm(hn )) ≥ (P (X ≥ (1 − ε)ω))n = e−n log P (X≥(1−ε)ω) . If F ∈ K1 , then this bound and f (hn ) → ∞ yield inequality (2.70). If F ∈ K2 , then we apply the same bound with ε = 0 and f (hn ) = − log P (X = ω)(1 + o(1)). If either condition 3) or condition 4) holds, then (2.70) follows from (2.69) and the inequality P (Sn ≥ nx) ≥ (P (X ≥ x))n . Assume that condition 5) is satisfied. We first check that for all τ ∈ (0, 1), δ > 0 and H > 0 there exists h > H such that P (h ≥ X ≥ (1 − τ )h) ≥ e−(1+δ)h0 h .

(2.71)

Suppose that the latter fails. Then there exist τ ∈ (0, 1), δ > 0 and H > 0 such that for all h > H, the inequality opposite to (2.71) holds. Put tk = (1 − τ )−k , k ∈ N. For all sufficiently large k, the inequality tk

ehx dF (x) ≤ ehtk P ((1 − τ )tk ≤ X ≤ tk ) < e(h−(1+δ)h0 )tk

(1−τ )tk

holds. Hence, ϕ(h) < ∞ for 0 < h < (1 + δ)h0 which contradicts to the definition of h0 . Inequality (2.71) is proved. ¯ X ¯1, X ¯ 2 , . . . be a sequence of independent random variables with Let X, the d.f. 1 ϕ(h0 )

x eh0 t dF (t). −∞

¯1 + · · · + X ¯ n and F¯n (t) = P (S¯n < t). Denote S¯n = X Fix a positive number δ and put τ = δ/2, τ = 6ρ. Put z = h∗ − h0 , x = nm(h∗ ), y = x + τ nz, Qn = P (x ≤ S¯n ≤ y). By the definition of the conjugate distribution, we have ∞ n e−th0 dF¯n (t) ≥ (ϕ(h0 ))n e−h0 y Qn . (2.72) Pn = P (Sn ≥ x) = (ϕ(h0 )) x

page 73

September 13, 2019

15:48

74

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Let a > 1 be a fixed number which will be chosen later. Put Tn = ¯ Sn − E S¯n , u = m(h0 ) + (1 + 2ρ)za, v = m(h0 ) + (1 + 3ρ)za. Taking into account (2.5) and E S¯n = nm(h0 ), we have Qn = P (zn ≤ Tn ≤ (1 + 6ρ)zn) ≥ P ((1 + 2ρ)zn ≤ Tn/a ≤ (1 + 3ρ)zn)P (−2ρzn ≤ Tn − Tn/a ≤ 3ρzn) 1 1 ¯ ≤ v))n/a ≥ P ((1 + 2ρ)zn ≤ Tn/a ≤ (1 + 3ρ)zn) ≥ (P (u ≤ X 2 2 for all sufficiently large n by the weak law of large numbers. We get Qn ≥

1 1 2 ϕ(h0 )

v

n/a

n/a 1 eh0 u P (u ≤ X ≤ v) eh0 t dF (t) ≥ (2.73) 2 ϕ(h0 )

u

for all sufficiently large n. Note that lim u/v = ε ∈ (0, 1). So, if a is large a→∞ enough, then  1 + 4ρ  P (u ≤ X ≤ v) ≥ P (vε1/2 ≤ X ≤ v) ≥ exp − h0 v . (2.74) 1 + 3ρ In the last inequality, we have applied (2.71). Relations (2.72)–(2.74) imply that Pn ≥

1 np(a) e 2

(2.75)

for all sufficiently large n, where h0 y h0 u 1 (1 + 4 )h0 v + − log ϕ(h0 ) − n a a (1 + 3 )a h0 = log ϕ(h0 ) − h0 (m(h∗ ) + 6 z) + (m(h0 ) + (1 + 2 )za) a (1 + 4 )h0 1 (m(h0 ) + (1 + 3 )za). − log ϕ(h0 ) − a (1 + 3 )a p(a) = log ϕ(h0 ) −

In the last equality, we have used the definitions of y, u, v and τ = 6 . It yields that p(a) does not depend on n and lim p(a) = log ϕ(h0 ) − h0 m(h∗ ) − 6h0 z + (1 + 2 )h0 z − (1 + 4 )h0 z

a→∞

= −f (h∗ ) − 8ρh0 z ≥ −(1 + 8ρ)f (h∗ ). We have used (2.5) in the last equality. If a is large enough, then (2.75) implies (2.70).

page 74

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Large Deviations for Sums

2.14

9789811212826

75

Bibliographical Notes

An asymptotic behaviour of probabilities of large deviations has been first investigated in [Khintchine (1929a,b)]. General results for large deviations have been proved in [Cram´er (1938)]. Generalizations of these results was proved by [Feller (1943)] and [Petrov (1954)]. Under the Cram´er condition, limit behaviour of probabilities of large deviations for independent, non-identically distributed random variables has been studied by [Petrov (1954, 1961, 1968)], [Feller (1943)], [Statuleviˇcius (1966)] and many others. Results in this topic may be found in monographs [Petrov (1975, 1995)], for example. A generalization of results from [Petrov (1954)] for a scheme of series has been obtained in [Frolov et al. (1997)]. [Linnik (1961a,b,c, 1962)] has developed methods of investigations of asymptotic behaviour for large deviations probability when the Cram´er conditions is violated. Results and the techniques may be found in [Ibragimov and Linnik (1971)]. Linnik’s researches was extended by [Petrov (1963, 1964)], [Nagaev (1965)], [Wolf (1968, 1970)], [Osipov (1972)] and others. Various one-sided results on large deviations have been obtained by [Chernoff (1952)], [Daniels (1954)], [Bahadur and Ranga Rao (1960)], [Zolotarev (1962)], [Borovkov (1964)], [Petrov (1965)], [Feller (1969)], [Osipov (1972)], [Rozovskii (1999, 2001, 2003)], [Nagaev (1969)], [Frolov (1998, 2002c)] and other. Results for probabilities of moderate deviations have been proved by [Amosova (1972, 1979, 1980)], [Rozovskii (1981, 2004)], [Slastnikov (1978, 1984)], [Michel (1976)], [Rychlik (1983)], [Frolov (1998, 2002b, 2008)] and references therein. Results for large deviations in the case of attraction to stable laws may be found in [Lipschutz (1956b)], [Bahadur and Ranga Rao (1960)], [Kim and Nagaev (1975)], [H¨ oglund (1979)], [Nagaev (1981)], [Nagaev (1983)], [Amosova (1984)], [Rozovskii (1989a,b, 1993, 1997)] and references therein. Bibliography on related topics (large deviations on the hole line, local limit theorems in relation to large deviations etc.) may be found in the book [Petrov (1975)]. Large deviations of sums of independent random variables are also discussed in monograph [Borovkov and Borovkov (2008)]. Note that we do not discuss large deviation principles in this book. Properties of stable laws are considered in [Mijnheer (1974)] and [Samorodnitsky and Taqqu (1994)]. Lemma 2.2 is proved by [Feller (1969)]. Theorem 2.3 follows from the

page 75

September 13, 2019

76

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

main result of the last paper. Theorem 2.2 is proved by [Chernoff (1952)]. In the Chapter 2, we follows a pattern of the papers [Frolov (1998, 2002c, 2003b)]. Partially, results of Chapter 2 are discussed in [Frolov (2014)].

page 76

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Chapter 3

Strong Limit Theorems for Sums of Independent Random Variables

Abstract. A universal theory of strong limit theorems is discussed. Starting with a formula for norming sequences, we prove universal strong laws. Then we show that they imply the Erd˝ os–R´enyi law and its Mason’s extension, the Shepp law, the Cs¨ org˝ o–R´ev´esz laws, the SLLN and the LIL. Moment assumptions are either optimal, or close to optimal. 3.1

Norming Sequences in Strong Limit Theorems

Let X, X1 , X2 , . . . be a sequence of i.i.d. random variables with a nondegenerate d.f. F (x) = P (X < x) and EX ≥ 0. Put Sn = X 1 + X 2 + · · · + X n ,

S0 = 0.

Assume that a(x) is a non-decreasing, continuous function such that 1 ≤ a(x) ≤ x and x/a(x) is non-decreasing. Put an = [a(n)], where [·] is the integer part of the number in brackets. Denote Un = Wn =

max

0≤k≤n−an

max

(Sk+an − Sk ), max (Sk+j − Sk ),

0≤k≤n−an 1≤j≤an

Rn = Sn − Sn−an ,

Tn = Sn+an − Sn .

In this section, we give a formula for norming sequences in strong limit theorems for increments, i.e. a formula for {bn } such that Wn lim sup = 1 a.s. bn Put h0 = sup{h : EehX < ∞}. Assume first that h0 = 0. It follows that ω = ∞. Let {yn } be a sequence of positive numbers such that yn → ∞. For n ∈ N, put Yn = min{X, yn }, 77

(3.1)

page 77

September 13, 2019

78

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

and Zn =

 Y − EY , n n Yn ,

for for

EX = 0, EX > 0.

Random variables Zn are bounded from above. Hence, they have an exponential moment. For every Zn , n ∈ N, define the functions ϕn (h), mn (h), σn2 (h), fn (h), ζn (z) and γn (x) by ϕ (h) , ϕn (h) = EehZn , mn (h) = n ϕn (h) σn2 (h) = mn (h),

fn (h) = hmn (h) − log ϕn (h),

ζn (z) = sup{zh − log ϕn (h) : h ≥ 0, ϕn (h) < ∞}, γn (x) = sup{z : ζn (z) ≤ x}. Assume now that h0 > 0. For n ∈ N, put yn = ∞ in (3.1) and define ϕn (h), mn (h), σn2 (h), fn (h), ζn (z) and γn (x) by the above formulae. Of course, we assume that h < h0 and x and z are from corresponding regions in there. It is clear that for h0 > 0, we have Zn = Yn = X, ϕn (h) = ϕ(h), mn (h) = m(h), σn2 (h) = σ 2 (h), fn (h) = f (h), ζn (z) = ζ(z) and γn (x) = γ(x) for all n, where ϕ(h), m(h), σ 2 (h), f (h), ζ(z) and γ(x) are the functions of the LDT discussed in Chapter 2. We now write the formula for {bn }. For n ∈ N, put bn = an γn (dn ),

(3.2)

where βn n and βn = log + log log(max(n, 3)). (3.3) an an Formula (3.2) is one of main results of this book. We will show that relation (3.2) defines norming sequences in the SLLN, the Erd˝os–R´enyi law and its Mason’s extension, the Shepp law, the Cs¨ org˝ o–R´ev´esz laws and the LIL. Relation (3.2) allows to calculate {bn } for all {an } provided h0 > 0. If h0 = 0, then to find {bn } we need an asymptotic of γn (dn ) which crucially depends on a choice of {yn }. This can not be done without additional assumptions on the distribution of X. We will check in the same way as in Chapter 2 that for distributions from the domains of attraction of the normal law and the asymmetric stable laws with index α ∈ (1, 2), one can choose {yn } such that dn =

γn (dn ) = γˆ (dn )(1 + o(1)),

(3.4)

page 78

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Strong Limit Theorems for Sums

79

where γˆ (x) = Cα x(α−1)/α L(x) and L(x) ∈ SV0 . Note that relation (3.4) only holds for an / log n → ∞. Indeed, if an = c log n and (3.4) holds, then  bn ∼ c1 log n. Hence, the condition n P (X ≥ bn ) < ∞ implies h0 > 0. + β Assume that either Ee(X ) < ∞, β ∈ (0, 1), or E(X + )p < ∞, p > 2. We know from Chapter 2 that under some restriction on {yn } and {hn } with yn → ∞ and hn → 0, the relations mn (hn ) = m(h ˆ n )(1 + o(1)) and fn (hn ) = fˆ(hn )(1 + o(1)) hold, where m(h) ˆ and fˆ(h) are positive, strictly increasing functions. Put −1 ˆ hn = f (dn ) and γˆ (x) = m( ˆ fˆ−1 (x)). Take yn = an γˆ (dn ), if the Linnik condition is satisfied, and yn = 1/hn , otherwise. Then (3.4) holds. Moreover, relations between {yn } and {hn } yield restrictions on the minimal rate of the growth of an for which (3.4) holds. ˆ = h, fˆ(h) = h2 /2, For example, if EX = 0 and EX 2 = 1, then m(h) √ √ γˆ (x) = 2x. It follows that bn = 2an βn . When the Linnik condition is fulfilled, this formula holds for an ≥ Cβ (log n)2/β−1 , where Cβ is a certain positive constant. If E(X + )p < ∞, p > 2, then an ≥ Cn2/p / log n has to be satisfied, where C is an arbitrary positive constant. If h0 > 0 and an / log n → ∞, then we need an asymptotic of γ(x) at zero to find a simple formula for bn . This problem may be solved in the same way as a deduction of (3.4). In particular, for EX = 0 and EX 2 = 1, √ we have bn = 2an βn again. This corresponds to the limit case β = 1 in the Linnik condition. Hence, formula (3.2) is applicable for h0 > 0 and h0 = 0 both. We will show below that (3.2) determines norming sequences in the SLLN, the Erd˝ os–R´enyi law and its Mason’s extension, the Shepp law, the Cs¨ org˝ o– R´ev´esz laws and the LIL. 3.2

Universal Strong Laws in Case of Finite Exponential Moments

We assume in this section that the random variable X satisfies the Cram´er condition. Since h0 > 0, the functions ϕn (h), mn (h), σn2 (h), fn (h), ζn (z) and γn (x) coincide for all n with ϕ(h), m(h), σ 2 (h), f (h), ζ(z) and γ(x) which are discussed in Chapter 2. In this case, formula (3.2) turns to bn = an γ(dn )

page 79

September 13, 2019

80

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

for n ∈ N with dn from (3.3). The universal results of Section 1.1 contain conditions on distributions of sums Sn . Our goal is to derive results under restrictions on the distribution of X. It turns out that they crucially depend on classes of distributions from Section 2.4. We start with upper bounds. Theorem 3.1. Assume that one of the following conditions holds: 1) log an / log n → 0; 2) for every ε > 0 there exists q ∈ (0, 1) such that inequality (1.14) of Theorem 1.1 holds for all sufficiently large n. Then Wn ≤ 1 a.s. lim sup bn One can replace Wn by Tn in the last relation. We need the following lemma to check property (1.12) of {bn }. Lemma 3.1. Let g(x) be a continuous, positive, non-decreasing, concave function with g(0) ≥ 0. For all n ∈ N, put bn = an g(dn ). Then there exists a non-decreasing sequence {˜bn } such that bn ∼ ˜bn and (1.12) holds. Proof. Check first that for every τ ∈ (0, 1), there exists N such that βn+k ≥ (1 − τ )βn for all n ≥ N and k ∈ N. If the function a(x) is bounded, then an = const and βn+k ≥ βn for all sufficient large n. Assume that a(x) → ∞ as x → ∞. Since x/a(x) is non-decreasing, we have

n

n+k log n ≥ log log n ∼ βn βn+k ≥ log a(n + k) a(n) for all fixed k ∈ N. Using the concavity of g(x), we conclude that for every τ ∈ (0, 1) there exists N such that β g( n+k g((1 − τ )dn ) an+k g(dn+k ) bn+k an ) ≥ ≥ ≥1−τ (3.5) = bn an g(dn ) g(dn ) g(dn ) for all n ≥ N and k ∈ N. Put ˜bn = inf k≥0 bn+k . It is clear that ˜bn+1 ≥ ˜bn and ˜bn ≤ bn . By (3.5), we get ˜bn ∼ bn . For k ∈ N, denote nk = [θk ], θ > 1. The function a(x) is non-decreasing and x/y ≥ log x/ log y for x ≥ y ≥ e. It follows that log( nak+1 log nk+1 ) ank βnk+1 βnk+1 nk+1 log nk+1 nk ≤ ≤ ≤ ≤ θ2 rk = nk βnk ank+1 βnk log( an log nk ) nk log nk k

page 80

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Strong Limit Theorems for Sums

81

for all sufficiently large k. The function x/a(x) is non-decreasing and, therefore, ank+1 nk+1 a(nk+1 ) ≤θ ≤θ ≤ θ3 ank a(nk ) nk for all sufficiently large k provided a(x) → ∞ as x → ∞. If a(x) is bounded then the latter evidently holds. By the concavity of g(x), we have bnk+1 an g(rk dnk ) g(θ2 dnk ) ≤ θ3 ≤ θ5 = k+1 bnk ank g(dnk ) g(dnk ) for all sufficiently large k. Choosing θ with θ5 ≤ 1 + τ , we get (1.12) from the last relation and (3.5). We also need the next result. Lemma 3.2. For all x ≥ 0 and δ > 0, one has ζ((1 + δ)γ(x)) ≥ (1 + δ)x. Note that the left-hand side of the last inequality may be infinite. Proof. If (1 + δ)γ(x) < A (for A = ∞ and A < ∞ both), then we apply the inequality (1 + δ)γ(x) ≥ γ((1 + δ)x) which follows from the concavity of γ(x) and γ(0) = EX ≥ 0. We have ζ((1 + δ)γ(x)) ≥ ζ(γ((1 + δ)x)) = (1 + δ)x. For (1 + δ)γ(x) > A, we have to consider the classes K1 , K2 and K5 . If F ∈ K1 ∪ K2 , then (1 + δ)γ(x) > ω that yields ζ((1 + δ)γ(x)) = ∞. For F ∈ K5 , we get ζ((1 + δ)γ(x)) = (1 + δ)γ(x)h0 − log ϕ(h0 ) ≥ (1 + δ)x in view of γ(x) ≥ (x + log ϕ(h0 ))/h0 for all x ≥ 0. The latter follows from the concavity of γ(x) and γ  (1/c0 ) = 1/h0 . We now turn to the proof of Theorem 3.1. Proof. By Lemma 3.1, relation (1.12) holds. Assume that condition 2) holds. Making use of Lemma 3.2 and the definitions of γ(x) and ζ(z), we have P (S[(1+ε)an ] ≥ (1 + ε)bn ) ≤ P (S[(1+ε)an ] ≥ [(1 + ε)an ]γ(dn )) ≤ e−[(1+ε)an ]βn /an . This yields (1.13) with H = 0.

page 81

September 13, 2019

82

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Suppose that condition 1) holds. By the definition of ζ(z) and Lemma 3.2, we have P (Sk ≥ (1 + ε)bn ) ≤ e−kζ((1+ε)bn /k) = e−kζ((1+ε)an γ(dn)/k) ≤ e−(1+ε)an dn = e−(1+ε)βn for all k ≤ [(1 + ε)an ]. This implies (1.13) with H = 0 for Sk . Theorem 3.1 follows from Theorem 1.1. We turn now to lower bounds. Theorem 3.2. For every ε ∈ (0, 1), put h∗ = f −1 ((1 − ε)dn ). Assume that one of the following conditions holds: 1) h∗ < h0 and h∗ = O(1); 2) F ∈ K1 and h∗ → ∞; 3) F ∈ K2 and dn ≥ 1/c0 ; 4) F ∈ K3 ∪ K5 , h∗ → ∞ and inequality (2.69) holds for all τ > 0, δ > 0 and all sufficiently large h; 5) F ∈ K4 , h∗ h0 and inequality (2.69) holds for all τ > 0, δ > 0 and all h sufficiently close to h0 ; 6) F ∈ K5 , h∗ > h0 , h∗ does not depend on n. If an /n → 1, then assume in addition that condition 2) of Theorem 3.1 holds. Then Rn ≥ 1 a.s. lim sup bn One can replace Rn by Tn in the last relation. Note that h0 may be infinite and h∗ = O(1) is a restriction in this cases. Conditions 1)–6) of Theorem 3.2 allow to consider various sequences {an } for all five classes of distributions. Condition h∗ = O(1) is equivalent to lim inf an / log n > 0 while h∗ → ∞ is equivalent to an = o(log n). If F ∈ K1 , then one has to check condition 1) for lim inf an / log n > 0 and condition 2) for an = o(log n). If F ∈ K2 , then the function f (h) takes its values in [0, 1/c0 ). Hence, one can use condition 1) for an > c0 log n only. The case an ≤ c0 log n is included in 3). If F ∈ K3 ∪ K4 , then condition 1) may be applied for lim inf an / log n > 0. Conditions 4) and 5) may be simpler for an = o(log n). If F ∈ K5 , then one can use condition 1) for lim inf an / log n > 0, condition 6) for a fixed h∗ > h0 (an = c log n, 0 < c < c0 ) and condition 4) for an = o(log n).

page 82

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Strong Limit Theorems for Sums

83

Note that condition 1) and one of conditions 2)–6) may be satisfied simultaneously. Turn to the proof of Theorem 3.2. Proof. We first check that relation (1.23) holds. Denote Aεn = {San ≥ (1 − ε)bn }, where ε ∈ (0, 1). Suppose that one of conditions 1), 2), 4)–6) is satisfied. By the concavity of γ(x), the inequality (1−ε)2 bn ≤ (1−ε)an γ((1−ε)dn ) holds. Using (2.70) and the definition of γ(x), we have 2

) = P (San ≥ (1 − ε)2 bn ) P (A2ε−ε n ≥ P (San ≥ (1 − ε)an γ((1 − ε)dn )) ≥ e−(1+τ )(1−ε)βn for all sufficiently large n. Taking τ small enough, we arrive at (1.23). If condition 3) is fulfilled, then bn = an ω. We assume further that an → ∞ since (1.23) automatically holds otherwise. Let m be a natural number which will be chosen later. We have P (Aεn ) ≥ (P (Sm ≥ (1 − ε)ωm))an /m ≥ (P (Sm ≥ (1 − ε)ωm))c0 βn /m . Take x0 with F (x0 ) < F (ω). Chose m large enough to satisfy (1 − ε)mω ≤ x0 + (m − 1)ω. Then P (Sm ≥ (1 − ε)ωm) ≥ P (Sm ≥ x0 + (m − 1)ω) ≥ mP (ω > X1 ≥ x0 , X2 = · · · = Xm = ω) = mP (ω > X ≥ x0 )(P (X = ω))m−1 = m(F (ω) − F (x0 ))e−(m−1)/c0 . This yields (1.23). Theorem 3.2 now follows from Theorem 1.2 and Lemma 3.1. Our next result is a lower bound for Un when {an } increases slow enough. Theorem 3.3. If the conditions of Theorem 3.2 hold and log log n = o(log(n/an )), then lim inf

Un ≥1 bn

a.s.

Proof. We checked in the proof of Theorem 3.2 that relations (1.23) and (1.12) hold. Theorem 3.3 now follows from Theorem 1.3.

page 83

September 13, 2019

15:48

84

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Theorems 3.1–3.3 and the inequality Rn ≤ Un ≤ Wn imply the following result. Theorem 3.4. If the conditions of Theorems 3.1 and 3.2 hold, then lim sup

Un Rn Tn Wn = lim sup = lim sup = lim sup =1 bn bn bn bn

a.s.

If log log n = o(log(n/an )) in addition, then lim 3.3

Un Wn = lim =1 bn bn

a.s.

Universal Strong Laws for Random Variables without Exponential Moment

In this section, we investigate the asymptotic behaviour of increments of sums of i.i.d. random variables when the Cram´er condition is violated. Then h0 = 0 and we define the norming sequence {bn } by formula (3.2) in which γ(x) is replaced by a similar function constructed from truncated random variables. We first deal with upper bounds again. Theorem 3.5. Assume that the sequence {bn } is equivalent to a nondecreasing sequence and condition (1.12) of Theorem 1.1 holds. Assume that one of the following conditions holds:  1) n P (X ≥ bn ) < ∞ and yn = bn .  2) n P (X ≥ cbn ) < ∞ for every c > 0, and for all small enough ε > 0 there exists δ > 0 such that an P (X ≥ yn )e−((1+ε)an−1)ζn ((1−δ)γn (dn )) +(an P (X ≥ yn ))2 ≤ e−(1+ε)βn (3.6) for all sufficiently large n. Suppose that for every ε > 0 there exists q ∈ (0, 1) such that inequality (1.14) of Theorem 1.1 holds for all sufficiently large n. Then Wn ≤ 1 a.s. lim sup bn One can replace Wn by Tn in the last relation. Proof. Assume that condition 1) holds. Define nk by formula (1.17) with  c = 1 from the proof of Theorem 1.1. Then the series k nk P (X ≥ bnk ) converges. This implies that ank P (X ≥ bnk ) → 0 as k → ∞. By Lemma

page 84

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

85

Strong Limit Theorems for Sums

2.4 with k = [(1 + ε)ank ], y = ynk and x = γnk (dnk ), we get (1.22) for all sufficiently large k. We obtain the result by Remark 1.2. Assume that condition 2) holds. Since βn → ∞, relation (3.6) yields that an P (X ≥ yn ) → 0. By Lemma 2.4 with k = [(1 + ε)an ], y = yn and x = γn (dn ), we get (1.13) for all sufficiently large n. The result follows by Theorem 1.1. Turn now to lower bounds. Theorem 3.6. Assume that one of the following conditions holds: 1) lim inf γn (dn ) > 0, and for every ε > 0 there exists τ > 0 such that an P (X ≥ (1 − ε)bn ) ≥ e−(1−τ )βn for all sufficiently large n. 2) The inequality dn ≤ − log P (X ≥ yn )

(3.7)

holds for all sufficiently large n and

√ h∗ σn (h∗ ) = o( an fn (h∗ )),

(3.8)

where h∗ = fn−1 ((1 − ε)dn ), ε ∈ (0, 1), and for EX = 0, EYn = o(γn (dn )).

(3.9)

If an /n → 1, then suppose in addition that (1.14) and (1.12) hold. Then Rn ≥ 1 a.s. lim sup bn One can replace Rn by Tn in the last relation. Note that the distribution of Yn is from the class K2 and the function fn (h) is bounded from above by − log P (X ≥ yn ). Hence, h∗ is well defined for all n such that (3.7) is satisfied. Proof. Check that inequality (1.23) holds for all sufficiently large n. Suppose that condition 1) is satisfied. Assume first that an = C1 for all sufficiently large n and lim sup γn (dn ) < ∞. Then bn ≤ C2 and P (San ≥ (1 − ε)bn ) ≥ P (San ≥ (1 − ε)C2 ) = p for all sufficiently large n. If p = 0, then   C1 C2 0 = P (San ≥ (1 − ε)C2 ) ≥ P X ≥ (1 − ε) . C1

page 85

September 13, 2019

86

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

It follows that the random variable X is bounded from above. This contradicts to the equality h0 = 0. It yields that p > 0. Hence, the left-hand side of inequality (1.23) is separated from zero and the right-hand one tends to zero since βn → ∞. It implies that inequality (1.23) holds for all sufficiently large n. Assume now that an = C1 for all sufficiently large n and lim sup γn (dn ) = ∞. Then there are two sequences {nm } and {nm } such % that {nm } {nm } = N, bnm → ∞ as m → ∞ and bnm ≤ C3 for all sufficiently large m. For n ∈ {nm }, we prove in the same way as above that P (San ≥ (1 − ε)bn ) ≥ p > 0 and, therefore, (1.23) holds for all sufficiently large n ∈ {nm }. Consider now n ∈ {nm }. We have an − 1 EX + → 0. (an − 1)P (X ≥ (1 − ε2 )bn ) ≤ (1 − ε2 )bn Since EX ≥ 0 and X is non-degenerate, we have P (San ≥ −ε(1 − ε)bn ) ≥ P (San ≥ 0) ≥ (P (X ≥ 0))an = C4 > 0. Hence, C4 P (San ≥ −ε(1 − ε)bn ) − (an − 1)P (X ≥ (1 − ε2 )bn ) ≥ 2 for all sufficiently large n ∈ {nm }. By Lemma 2.6 with δ = ε, x = (1 − ε)γn (dn ) and k = an , inequality (1.23) holds for all sufficiently large n ∈ {nm } as well. Assume now that an → ∞. Since γn (dn ) > > 0 for all sufficiently large n, we have P (San ≥ −εbn ) ≥ P (San ≥ −ε an ) → 1 by the weak law of large numbers. Moreover, ∞ an − 1 2 udF (u) → 0. (an − 1)P (X ≥ (1 − ε )bn ) ≤ (1 − ε2 )bn (1−ε2 )bn

Applying again Lemma 2.6 with δ = ε, x = (1 − ε)γn (dn ) and k = an , we conclude that inequality (1.23) holds for all sufficiently large n. Suppose that condition 2) is satisfied. Relations (3.8) and (3.9) imply that the conditions of Lemma 2.5 are fulfilled. By Lemma 2.5 with y = yn , k = an , h = h∗ and x = mn (h∗ ), we have P (San ≥ (1 − ε)3 bn ) ≥ P (San ≥ (1 − ε)2 an γn ((1 − ε)dn )) 2 3 3 = P (San ≥ (1 − ε)2 an mn (h∗ )) ≥ e−(1−ε)(1+ε)βn = e−(1−ε )βn 4 4 for all sufficiently large n and (1.23) holds. Theorem 3.6 follows now from Theorem 1.2.

page 86

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

87

Strong Limit Theorems for Sums

Now we turn to lower bounds for sequences {an } which increase slowly enough. Theorem 3.7. If the conditions of Theorem 3.6 hold and log log n = o(log(n/an )), then lim inf

Un ≥1 bn

a.s.

We checked above that under the assumptions of Theorem 3.6, inequality (1.23) holds. Hence, Theorem 3.7 follows from Theorem 1.3. Theorems 3.5–3.7 and the inequalities Rn ≤ Un ≤ Wn yield the next result. Theorem 3.8. If the conditions of Theorems 3.5 and 3.6 hold, then lim sup

Wn Un Rn Tn = lim sup = lim sup = lim sup =1 bn bn bn bn

a.s.

If log log n = o(log(n/an )) in addition, then lim

3.4

Un Wn = lim =1 bn bn

a.s.

Corollaries of the Universal Strong Laws

We derive various corollaries of Theorems 3.4 and 3.8 in this section. The results crucially depend on the growth rate of an . We start with short increments when an = O(log n). In this case, the norming sequences depend on the full distribution of X and, sometimes, determine this distribution uniquely. Such results are called the Erd˝os–R´enyi and Shepp laws. Further, we turn to large increments when an / log n → ∞. In this case, the norming sequences depend on parameters of the completely asymmetric stable laws which domains of attraction distributions of summands belong to. For example, EX = 0 and EX 2 = 1 (i.e. F ∈ DN (2)) are necessary and √ sufficient conditions for the LIL with the norming sequence 2n log log n. Analogues of the LIL for F ∈ DN (α), α ∈ (1, 2), hold with the norming sequences C(α)n1/α (log log n)1−1/α . The non-normal attraction requires an additional slowly varying multiplier in the normalization. We show that the situation is similar for large increments. Results for them are called the Cs¨ org˝ o–R´ev´esz laws. They also imply some special cases of the LIL that are discussed as well. Finally, we discuss the SLLN.

page 87

September 13, 2019

88

3.4.1

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

The Erd˝ os–R´ enyi and Shepp Laws

Consider short increments including the case an = 1 for all n. Start with an = c log n. Theorem 3.9. If EX ≥ 0, h0 > 0 and an = [c log n], c > 0, then   Un Tn Rn 1 Wn = lim = lim sup = lim sup =γ lim a.s. an an an an c Theorem 3.9 follows from Theorem 3.4. It is the full form of the Erd˝ os– R´enyi law (for Un and Wn ) and the Shepp law (for Tn ). The Erd˝ os–R´enyi and Shepp laws are quite different from the SLLN and LIL. For example, the norming sequences depend in the last strong laws on first or second moments of X for F ∈ DN (2). To find γ(x), we have to know the full distribution of X. Moreover, γ(x) defines uniquely this distribution provided the two-sided Cram´er condition holds (see Section 2.5 for details). random variable,  √ In particular, if X is a standard normal then γ(x) = 2x and the limit in Theorem 3.9 is 2/c for all c > 0. The latter may hold for the normal case only. The Erd˝ os–R´enyi and Shepp laws yield that the best accuracy of approximation of sums Sn by the Wiener process in the strong invariance principle is O(log n). We now turn to the case an = o(log n). Theorem 3.10. Assume that EX ≥ 0, h0 > 0 and an = o(log n). For ω = ∞, assume in addition that either condition 4), or condition 5) of Theorem 3.2 holds. Then Un Tn Wn = lim = lim sup = 1 a.s. lim an γ(log n/an ) an γ(log n/an ) an γ(log n/an ) One can replace Tn by Rn in the last relation. Theorem 3.10 follows from Theorem 3.4. For Un , Theorem 3.10 is Mason’s extension of the Erd˝ os–R´enyi law. If an = 1 for all n, then Theorem 3.4 yields 1 max Xk = 1 a.s. lim γ(log n) 1≤k≤n √ Hence, the norming is 2 log n for the standard normal random variables. If X has the exponential distribution with the density p(x) = e−x for x > 0, then ζ(x) = x − 1 − log x for x ≥ 1. This implies that γ(x) ∼ x as x → ∞. It follows that the norming for maxima is log n in the exponential case. We mentioned these results of the theory of extremes in Section 1.1.

page 88

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Strong Limit Theorems for Sums

3.4.2

9789811212826

89

The Cs¨ org˝ o–R´ ev´ esz Laws

The case an / log n → ∞ is quite different. The results below do not follows immediately from Theorems 3.4 and 3.8 as above. Their proofs additionally require complicated technical calculations. We now present simple formulae for bn under three one-sided moment conditions: the Cram´er condition, the Linnik condition and E(X + )p < ∞. These conditions determine different minimal rates of the growth of an under consideration. First, we deal with the case of finite variations. Theorem 3.11. Assume that EX = 0, EX 2 = 1 and one of the following conditions hold: 1) h0 > 0 and an / log n → ∞; + β 2) Ee(X ) < ∞ for some β ∈ (0, 1) and an ≥ C(log n)2/β−1 , where C = C(β) is an absolute positive constant; 3) E(X + )p < ∞ for some p > 2 and an ≥ Cn2/p / log n, where C is an arbitrary positive constant. Then the relation Un Rn Tn Wn = lim sup = lim sup = lim sup = 1 a.s. (3.10) lim sup bn bn bn bn holds with &   n + log log n . bn = 2an log an If log log n = o(log(n/an )) in addition, then lim

Un Wn = lim =1 bn bn

a.s.

(3.11)

We postpone the proof of Theorem 3.11 and join it with that in the case DN (α). Theorem 3.11 generalizes the strong approximation laws obtained by M.Cs¨org˝ o and P.R´ev´esz (see [Cs¨org˝ o and R´ev´esz (1981)], Theorem 3.2.1). They have used the strong approximation of sums of independent random variables by the Wiener process. It is only possible under two-sided moment restrictions. Our assumptions are one-sided and we use an analysis of probabilities of large deviations in our proofs. This yields us to stronger results those are called the Cs¨org˝ o–R´ev´esz laws. √ If X has the standard normal distribution, then γ(x) = 2x for all x ≥ 0. Hence, the norming in Theorem 3.11 is the same as that in the

page 89

September 13, 2019

90

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Gaussian case. In this sense, the result is invariant from the distribution of X. This distinguishes the Cs¨ org˝o–R´ev´esz laws and the Erd˝os–R´enyi law discussed above. Turn to the case EX 2 = ∞. Start with F ∈ D(2). Note that F ∈ DN (2) in Theorem 3.11. Theorem 3.12. Assume that EX = 0 and F ∈ D(2). Put ˆ fˆ−1 (dn )), bn = an m(

(3.12)

where   1 m(h) ˆ = hG , h

h2 G fˆ(h) = 2

  1 , h

0 G(x) =

u2 dF (u), x > 0.

−x

(G(x) ∈ SV∞ .) Assume that one of the following conditions holds: 1) h0 > 0 and an / log n → ∞; + β 2) Ee(X ) < ∞ for some β ∈ (0, 1) and bn ≥ C(log n)1/β , where C = C(β) is an absolute positive constant; 3) E(X + )p < ∞ for some p > 2 and bn ≥ Cn1/p , where C is an arbitrary positive constant. Then relation (3.10) holds with bn from (3.12). If log log n = o(log(n/an )) in addition, then relation (3.11) holds with bn from (3.12). Below we prove Theorem 3.12 together√with its analogue for D(α), α ∈ (1, 2). In there, we check that fˆ−1 (x) = 2xL(1/x), where L(x) ∈ SV∞ . Hence, (3.12) is equivalent to √ 2an βn . (3.13) bn = L(1/dn ) Note that we give a way of calculation of the function L(x) from G(x). Comparing the last formula and the formula for bn from Theorem 3.11 yields that there is a slowly varying multiplier in (3.13) which depends on the truncated second moment. The inequalities for an from conditions 2) and 3) of Theorem 3.11 imply the inequalities bn ≥ C1 (log n)1/β and bn ≥ C2 n1/p , correspondingly. It means that conditions 2) and 3) of Theorems 3.11 and 3.12 are similar. Unfortunately, the slowly varying multiplier in (3.13) does not allow to transform inequalities for bn in simple inequalities for an in Theorem 3.12. Since L(x) → 0 as x → ∞, the conclusion of Theorem 3.12 holds if

page 90

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Strong Limit Theorems for Sums

9789811212826

91

conditions 2) and 3) are replaced by conditions 2) and 3) of Theorem 3.11, correpondingly. Turn to the case F ∈ DN (α), α ∈ (1, 2). Theorem 3.13. Assume that EX = 0, F ∈ DN (α), α ∈ (1, 2) and one of the following conditions hold: 1) h0 > 0 and an / log n → ∞; + β 2) Ee(X ) < ∞ for some β ∈ (0, 1) and an ≥ C(log n)α/β−α+1 , where C = C(α, β) is an absolute positive constant; 3) E(X + )p < ∞ for some p > α and an ≥ Cnα/p /(log n)α−1 , where C is an arbitrary positive constant. Then relation (3.10) holds with λ  n −λ 1/α + log log n , (3.14) b n = λ an log an where λ = (α − 1)/α. If log log n = o(log(n/an )) in addition, then relation (3.11) holds with bn from (3.14).  Note that bn = 2an (log(n/an ) + log log n) provided we formally put α = 2 in relation (3.14). We now prove Theorems 3.11 and 3.13 both. The cases α = 2 and α ∈ (1, 2) correspond to Theorems 3.11 and 3.13. Proof. Assume that condition 1) is satisfied. Put yn = ∞ and hn = (dn /λ)1/α . Applying Lemmas 2.8, 2.16 and 2.19 and relation (2.4), we get γn (dn ) ∼ γn (fn (hn )) ∼ mn (hn ) ∼ hα−1 = λ−λ dλn . n

(3.15)

It follows that one can replace (3.2) by (3.14). Check the conditions of Theorem 3.4. To this end, we need the next result. Lemma 3.3. Let V (x) be a d.f. such that V (x) is continuous at zero and V (0) < 1. Assume that P (X ≥ 0) > 0 and there exists a sequence of positive constants {Bn } such that the distributions of Sn /Bn converge weakly to the distribution V . Then there exists q > 0 such that P (Sn ≥ 0) ≥ q for all n ∈ N. Proof. Zero is a continuity point of V (x). By the definition of the weak convergence, we have   Sn ≥ 0 → 1 − V (0). P (Sn ≥ 0) = P Bn

page 91

October 2, 2019

92

10:39

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Hence, there exists N0 such that P (Sn ≥ 0) ≥

(1 − V (0)) 2

for all n ≥ N0 . For n < N0 , we get P (Sn ≥ 0) ≥ (P (X ≥ 0))n ≥ (P (X ≥ 0))N0 . Put q = min{(1 − V (0))/2, (P (X ≥ 0))N0 }. The assumptions yield q > 0 and the result follows. By Lemma 3.3, relation (1.14) holds. It is clear that condition 1) of Theorem 3.2 is satisfied. Hence, Theorem 3.4 yields the result. 1/α Suppose that condition 2) is fulfilled. Put yn = λ−λ an βnλ and hn = 1/α dn . If an ≥ C(log n)α/β−α+1 , then, taking into account that βn ≤ 2 log n, we have hn yn1−β = λβλ−1 βn1−βλ a−β/α ≤ λβλ−1 21−λβ C −β/α . n Choosing large enough C = C(α, β), we have lim sup hn yn1−β < 1. Making use of Lemmas 2.10, 2.17 and 2.19 and relation (2.4), we get (3.15). It follows that one can define bn by formula (3.14). Check now the conditions of Theorem 3.8. Condition (1.12) follows from Lemma 3.1 and (3.14). Taking into account that an ≥ C(log n)α/β−α+1 and the function x(log(n/x) + log log n) is increasing in x ≤ n, we have    α−1 1/α n −λ α/β−α+1 log yn = b n ≥ λ C(log n) C(log n)α/β−α ≥ (2λ)−λ C 1/α (log n)1/β ≥ (log n)1/β for all sufficiently large n provided C has been chosen large enough. Then P (X ≥ bn ) ≤ P (e(X

+ β

)

≥ n)

for all sufficiently large n. It follows that the series from condition 1) of Theorem 3.5 converges. Relation (1.14) follows from Lemma 3.3. Relation (3.7) hods in view of an / log n → ∞. By Lemmas 2.10 and 2.17, we get (h∗ )2 σn2 (h∗ ) = O(fn (h∗ )) = O(dn ). √ Then (3.8) is equivalent to 1 = o( an dn ) that holds by βn → ∞.

(3.16)

page 92

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

93

Strong Limit Theorems for Sums

Furthermore, yn ∞ ∞ β xdF (x) − yn P (X ≥ yn ) ≤ xdF (x) ≤ ex /2 dF (x) −EYn = − −∞ β

≤ e−yn /2

∞

β

yn

ex dF (x) ≤ 0

yn

+ β

Ee(X yn

)

for all sufficiently large n. Since yn = bn and b−1 n = o(γn (dn )), relation (3.9) holds. Then the conditions of Theorem 3.8 hold and we get the result. Assume finally that condition 3) is satisfied. Put yn = (λ/dn )1/α and hn = (dn /λ)1/α . By Lemmas 2.10, 2.17 and 2.19 and relation (2.4), we get (3.15). It follows that one can choose bn from (3.14). Check the conditions of Theorem 3.8. Condition (1.12) holds by Lemma 3.1 and (3.14). Taking into account that an ≥ Cnα/p /(log n)α−1 and x(log(n/x) + log log n) is increasing in x ≤ n, we have   α−1 1/α Cnα/p n(log n)α −λ bn ≥ λ ≥ C1 n1/p , log (log n)α−1 Cnα/p for all sufficiently large n. This yields that p   X ≥n P (X ≥ cbn ) ≤ P cC1 for all sufficiently large n. Hence, the series from condition 2) of Theorem 3.5 converges for all c > 0. Applying the inequalities an ≥ Cnα/p /(log n)α−1 and βn ≤ 2 log n, we get E(X + )p an P (X ≥ yn ) ≤ an ≤ C2 a1−p/α βnp/α ≤ C3 nα/p−1 (log n)1−α+p n ynp for all sufficiently large n. Further, −(1+ε) ≥ C6 n(1+ε)(α/p−1) (log n)−(1+ε)α . e−(1+ε)βn = a1+ε n (n log n)

It follows that

(an P (X ≥ yn ))2 = o e−(1+ε)βn .

(3.17)

Note that the sequence {δhn }, δ ∈ (0, 1), satisfies to the conditions of Lemmas 2.10 and 2.17 as well as {hn }. By Lemmas 2.10, 2.17 and 2.19 and relations (3.15) and (2.3), we have ) ∼ ζn (mn (δ 1/(α−1) hn )) ∼ fn (δ 1/(α−1) hn ) ∼ δ 1/λ dn . ζn (δγn (dn )) ∼ ζn (δhα−1 n

page 93

September 13, 2019

15:48

94

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Hence, for every μ ∈ (0, 1), we write e(1+ε)βn e−((1+ε)an −1)ζn ((1−δ)γn (dn )) ≤ eνβn ≤ C8 n−ν(α/p−1) (log n)C9 for all sufficiently large n, where ν = (1 + ε)(1 − (1 − δ)1/λ (1 − μ)). Take small enough δ and μ such that ν < 1. Then ν(α/p − 1) > α/p − 1. It follows that

(3.18) an P (X ≥ yn )e−((1+ε)an −1)ζn ((1−δ)γn (dn )) = o e−(1+ε)βn . Taking into account (3.17), we get (3.6). Relation (1.14) follows from Lemma 3.3. Relation (3.7) holds for all sufficiently large n in view of an / log n → ∞. By Lemmas 2.10 and 2.17, we obtain (3.16). Then (3.8) holds again. Since E(X + )2 < ∞, we have ∞ −EYn ≤

 xdF (x) = o yn−1 = o (γn (dn )) .

yn

This yields (3.9). The result follows from Theorem 3.8. We now turn to domains of non-normal attraction. Theorem 3.14. Assume that EX = 0 and F ∈ D(α), α ∈ (1, 2). Put bn = an m( ˆ fˆ−1 (dn )), where m(h) ˆ =

αΓ(2 − α) α−1 h G α−1

  1 , h

fˆ(h) = Γ(2 − α)hα G

(3.19)   1 , h

G(x) = xα F (−x), x > 0. (G(x) ∈ SV∞ .) Assume that one of the following conditions holds: 1) h0 > 0 and an / log n → ∞; + β 2) Ee(X ) < ∞ for some β ∈ (0, 1) and bn ≥ C(log n)1/β , where C = C(β) is an absolute positive constant; 3) E(X + )p < ∞ for some p > α and bn ≥ Cn1/p , where C is an arbitrary positive constant. Then relation (3.10) holds with bn from (3.19) . If log log n = o(log(n/an )) in addition, then relation (3.11) holds with bn from (3.19).

page 94

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

95

Strong Limit Theorems for Sums

We find in the proof of Theorem 3.14 that fˆ−1 (x) = λλ−1 x1/α L(1/x), where L(x) ∈ SV∞ . Hence, relation (3.19) may be written as 1/α

bn =

λ−λ an βnλ . L(1/dn )

(3.20)

Note that we give a way for calculation of L(x). Hence, for F ∈ D(α), the norming is distinguished from that for F ∈ DN (α) (see Theorem 3.13) by a multiplier which depends of the slowly varying part of the tail of the distribution F . Unlike the case F ∈ D(2), it turns out that L(x) may tend to either infinity, or zero as x → ∞. It depends on the behaviour of G(x). If G(x) → 0 as x → ∞, then L(x) → ∞ as x → ∞. If G(x) → ∞ as x → ∞, then L(x) → 0 as x → ∞. In the last case, conditions 2) and 3) of Theorem 3.13 imply conditions 2) and 3) of Theorem 3.14, correspondingly. We now prove Theorems 3.12 and 3.14. The cases α = 2 and α ∈ (1, 2) correspond to Theorems 3.12 and 3.14. Proof. For x > 0, put G(x) = xα F (−x) for α ∈ (1, 2) and G(x) = 0 2 u dF (u) for α = 2. For h > 0, denote −x

m(h) ˆ = c1 (α)h

α−1

  1 G , h

fˆ(h) = c2 (α)hα G

  1 , h

where c1 (2) = 1, c2 (2) = 1/2, c1 (α) = c2 and c2 (α) = c4 with the constants c2 and c4 from Lemma 2.16. Assume first that condition 1) holds. Put yn = ∞, hn = fˆ−1 (dn ). Applying Lemmas 2.13, 2.16 and 2.19 and relation (2.4), we get γn (dn ) ∼ γn (fˆ(hn )) ∼ γn (fn (hn )) ∼ mn (hn ) ˆ fˆ−1 (dn )). ∼ m(h ˆ n ) ∼ m( By Lemma 2.20, we have fˆ−1 (x) = c3 (α)x1/α L

(3.21)   1 , x

(3.22)

where c3 (α) = λ−1/α and L(x) ∈ SV∞ . Relation (3.21) and the definition of m(h) ˆ yield that    α−1  1 1 α−1 λ dn L G . γn (dn ) ∼ c1 (α)(c3 (α)) 1/α dn c3 (α)dn L(1/dn ) Since

  α   1 1 G x ∼ fˆ(fˆ−1 (x)) = c4 (α)x L x c3 (α)x1/α L(1/x)

page 95

September 13, 2019

15:48

96

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

as x → 0, where c4 (2) = 1 and c4 (α) = c4 /λ for α ∈ (1, 2), we get     −α 1 1 c4 (α)G ∼ L x c3 (α)x1/α L(1/x) as x → 0. Note that L(x) → ∞ as x → 0 for α = 2. Hence, we have γn (dn ) ∼

λ−λ dλn . L(1/dn )

(3.23)

It follows that one can put 1/α

λ−λ an βnλ . (3.24) L(1/dn ) Check the conditions of Theorem 3.4. Relation (1.14) follows from Lemma 3.3. It is clear that condition 1) of Theorem 3.2 is satisfied. By Theorem 3.4, we get the result under condition 1). ˆ fˆ−1 (dn )), hn = Suppose now that condition 2) holds. Put yn = an m( −1 fˆ (dn ). ˆ = λ−1 fˆ(h) and Check that lim sup hn yn1−β < 1. Note that hm(h) ˆ n ) = λ−1 an fˆ(hn ) ∼ λ−1 βn . hn yn = an hn m(h bn =

We have hn yn1−β ≤

2βn λynβ

=

2βn λbβn

≤

4 2βn ≤ λC β log n λC β

for all sufficiently large n. It follows that lim sup hn yn1−β < 1 provided C = C(α, β) is chosen large enough. By Lemmas 2.15, 2.17 and 2.19 and relation (2.4), we get (3.21) and (3.23). It follows that the norming bn may be taken from (3.24). Check the conditions of Theorem 3.8. Remember that one can replace condition 1) by condition (1.21) of Remark 1.1 in Theorem 1.1 and, consequently, in Theorem 3.5. Note that inequalities (1.14) for Si in Remark 1.1 follow from Lemma 3.3. 1/α Put bn = λ−λ an βnλ and nk = [θk ] for k ∈ N. We have bn bn bn bnk+1 L(1/dnk ) L(1/dnk ) = k+1 ∼ k+1 (3.25) ∼ k+1 bnk bnk L(1/dnk+1 ) bnk L((1/dnk )θ(1 + o(1))) bnk as k → ∞ by the uniform convergence theorem (Theorem 1.1, p. 2 from [Seneta (1976)]). Assume that an /n → 1. Then  1/α   n n bn ∼ λ−λ (log log n) L . log log n log log n

page 96

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

97

Strong Limit Theorems for Sums

The function x1/α L(1/x) is a regularly varying function of positive order and, hence, it is equivalent to a non-decreasing regularly varying function (see [Seneta (1976)], p. 20). Since the sequence {n/ log log n} is nondecreasing, {bn } is equivalent to a non-decreasing sequence. Relation (1.12) follows from (3.25) and Lemma 3.1. Then condition 1) of Theorem 1.1 holds. Suppose that an /n is separated from 1. Proving Theorems 3.11 and 3.13, we checked that {bn } is equivalent to a non-decreasing sequence and (1.12) holds for {bn }. Define nk by formula (1.17). Then θk−1 < nk−1 ≤ θk < nk ≤ θk+1 . It follows that (1.12) holds for {bn } with nk instead of [θk ] as well. Hence, 1≤

bnk L (1/dnk ) bnk ≤ θ2 ≤θ  mk bnk−1 min L (1/dn ) nk−1 ≤n≤nk

L (1/dnk ) min L (1/dn )

(3.26)

nk−1 ≤n≤nk

for all sufficiently large k, where L (x) = 1/L(x). Assume that nk−1 ≤ n ≤ nk . Taking into account that an and βn are non-decreasing and the inequality x/y ≥ log x/ log y holds for x ≥ y ≥ e, we have βn log(nk−1 log nk−1 /ank−1 ) nk−1 log nk−1 1 ank βn ≥ ≥ k−1 = ≥ 3 an βnk βnk log(nk log nk /ank ) nk log nk θ for all sufficiently large k. In the last inequality, we have used again that θk−1 < nk−1 ≤ θk < nk ≤ θk+1 by (1.17). From the other hand, since x/a(x) is non-decreasing, we get ank βn ank nk ≤ ≤ ≤ θ3 an βnk ank−1 nk−1 for all sufficiently large k. It follows that     1 θ˜k   L , min =L nk−1 ≤n≤nk dn dnk where θ−3 ≤ θ˜k ≤ θ3 for all sufficiently large k. Using again the uniform convergence theorem, we conclude that L (1/dnk ) →1 L (θ˜k /dnk ) as k → ∞. This and (3.26) imply (1.21). Hence, for an /n → 1, condition 1) of Theorem 1.1 holds and for an /n separated from 1, condition (1.21) from Remark 1.1 holds.

page 97

September 13, 2019

98

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Relation yn = bn ≥ (log n)1/β implies that the series in condition 1) of Theorem 3.5 converges. Relation (1.14) follows from Lemma 3.3. From Lemmas 2.15 and 2.17, we obtain (3.16) that implies (3.8). In the same way as in the proof of Theorems 3.11 and 3.13 for the case 2), we have −EYn = o(γn (dn )). Then we get (3.9). By Theorem 3.8, we obtain the result in the case 2). Assume now that condition 3) holds. Put yn = 1/fˆ−1 (dn ), hn = −1 ˆ f (dn ). Applying Lemmas 2.15, 2.17 and 2.19 and relation (2.4), we prove (3.21) and (3.23). The last relation yields (3.24). In the same way as for the case 2), we prove that if an /n → 1, then condition 1) of Theorem 1.1 holds and if an /n is separated from 1, then condition (1.21) from Remark 1.1 holds. The inequality bn ≥ Cn1/p implies the convergence of the series from condition 2) of Theorem 3.5. In the same way as for the case 3) of Theorems 3.11 and 3.13, we get (3.17) and (3.18). The last two relations yield (3.6). Relation (1.14) follows by Lemma 3.3. We check (3.8) in the same way as for the case 2). We get relation (3.9) in the same way as in the proof of the case 3) of Theorems 3.11 and 3.13. By Theorem 3.8, we have the result under condition 3). Theorems 3.12 and 3.14 are proved. 3.4.3

The Law of the Iterated Logarithm

Turn to the LIL for increments. We start with simple corollaries of Theorems 3.11–3.14. We consequently consider the cases of finite variations, a domain of attraction of the normal law and domains of normal and nonnormal attraction of the asymmetric stable laws. Corollary 3.1. Assume that EX = 0, EX 2 = 1, E(X + )p < ∞ for some p > 2 and log(n/an ) → q ≥ 0. log log n

(3.27)

Then  Wn Un lim sup √ = lim sup √ = 1+q 2an log log n 2an log log n

a.s.

(3.28)

page 98

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

99

Strong Limit Theorems for Sums

Corollary 3.1 follows from Theorem 3.11 and (3.27). For an = n, q = 0 and EX 2 = 1, Corollary 3.1 yields the LIL for Sn and max Sk with the 1≤k≤n √ standard norming bn = 2n log log n. Corollary 3.2. Assume that EX = 0, F ∈ D(2), E(X + )p < ∞ for some p > 2 and relation (3.27) holds. Then L(an / log log n) Wn √ 2an log log n  L(an / log log n) Un = lim sup √ = 1+q 2an log log n lim sup

a.s.,

(3.29)

where L(x) is a slowly varying function from (3.13). Corollary 3.2 follows from Theorem 3.12 and relation (3.13). Corollary 3.3. Assume that EX = 0, F ∈ DN (α), α ∈ (1, 2), and E(X + )p < ∞ for some p > α. Assume that (3.27) holds. Then lim sup

Wn 1/α λ−λ an (log log n)λ

= lim sup

Un 1/α λ−λ an (log log n)λ

= (1 + q)λ

a.s.,

(3.30)

where λ = (α − 1)/α. Corollary 3.3 follows from Theorem 3.13 and (3.27). The next result follows from Theorem 3.14. Corollary 3.4. Assume that EX = 0, F ∈ D(α), α ∈ (1, 2). Assume that condition 3) of Theorem 3.14 and relation (3.27) hold. Then lim sup

L(an / log log n)Wn 1/α

λ−λ an (log log n)λ L(an / log log n)Un = lim sup = (1 + q)λ 1/α λ−λ an (log log n)λ

a.s.,

(3.31)

where λ = (α − 1)/α and L(x) is a slowly varying function from (3.20). We dealt before with the moment conditions E(X + )p < ∞ for p > α. We now turn to the case p = α. Theorem 3.15. Assume that α ∈ (1, 2], EX = 0 and E(X + )α (log(1 + X + ))τ < ∞ for some τ > 1/2. Suppose that relation (3.27) holds for

page 99

September 13, 2019

100

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

some q ∈ [0, 2τ − 1) and an ≥ Cn(log n)−τ (log log n)−(α−1) , where C is an arbitrary positive constant. If EX 2 = 1, then (3.28) holds. If F ∈ D(2), then (3.29) holds. If F ∈ DN (α), α ∈ (1, 2), then (3.30) holds. If F ∈ D(α), α ∈ (1, 2), then (3.31) holds. For an = n, q = 0 and EX 2 = 1, we have the norming sequence √ bn = 2n log log n from the LIL while the last theorem yields the LIL for sums under E(X + )2 (log(1 + X + ))τ < ∞, τ > 1/2. This is a more restrictive than the Hartman–Wintner theorem. The above technique may be modified to the case of independent non-identically distributed random variables. Then one can derive a result for increments that implies the Hartman–Wintner theorem. See [Frolov (2004a)] for details. Proof. Assume first that F ∈ DN (α), α ∈ (1, 2]. Put hn = (dn /λ)1/α and yn = (λ/dn )1/α . Then Lemmas 2.10, 2.17 and 2.19 and relation (2.4) give γn (dn ) ∼ λ−λ dλn Hence, one can put λ bn = λ−λ (1 + q)λ a1/α n (log log n) .

(3.32)

Check the conditions of Theorem 3.8. Condition (1.12) holds by (3.32) and Lemma 3.1. It is not difficult to check that Put H(x) = xα (log x)τ . −1 H (x) ∼ ατ /α x1/α (log x)−τ /α as x → ∞. The inequality an ≥ Cn(log n)−τ (log log n)−α+1 yields that bn ≥ C1 H −1 (n) for all sufficiently large n. Hence,    X   P (X ≥ cbn ) ≤ P H ≥n 0. Further, we have an P (X ≥ yn ) ≤ C2 an yn−α (log yn )−τ ≤ C3 (log n)−τ log log n for all sufficiently large n. For every > 0, the inequality e−(1+ε)βn ≥ (log n)−(1+ε)(1+ )(1+q) holds for all sufficiently large n. It follows that for all sufficiently small ε > 0, we get the relation

(an P (X ≥ yn ))2 = o e−(1+ε)βn . For all positive μ < 1 and δ < 1, put ν = (1 − μ)(1 − δ)α/(α−1) . Then e−(1+ε)βn +((1+ε)an −1)ζn ((1−δ)γn (dn ) ≥ e−(1+ε)βn +(1+ε)νβn ≥ (log n)−(1+ε)(1−ν)(1+ )(1+q) ≥ C3 (log n)−τ log log n ≥ an P (X ≥ yn )

page 100

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Strong Limit Theorems for Sums

9789811212826

101

for all sufficiently large n provided μ, δ and are small enough. It follows that (3.6) holds. The remain conditions of Theorem 3.8 can be verify in the same way as in the proof of Theorems 3.11 and 3.13. We finally need to apply Theorem 3.8. Assume now that F ∈ D(α), α ∈ (1, 2]. Put hn = fˆ−1 (dn ) and yn = ˆ 1/f −1 (dn ). By Lemmas 2.15, 2.17 and 2.19 and relation (2.4), we get λ−λ dλn , γn (dn ) ∼ L(1/dn ) where L(x) is a slowly varying function. Hence, we put 1/α

λ−λ (1 + q)λ an (log log n)λ . L(1/dn ) We check all conditions of Theorem 3.8 besides (3.6) in the same way as in the proofs of Theorems 3.12 and 3.14. We only mention that   α 1 an P (X ≥ yn ) ≤ C4 (log n)−τ log log n L dn bn =

≤ C4 (log n)−τ + log log n for all sufficiently large n, where may be chosen arbitrary small. It yields that we keep the relations, obtained above in the proof of (3.6) for the case F ∈ DN (α). We finally apply Theorem 3.8 to finish the proof. 3.4.4

The Strong Law of Large Numbers

Assume now that EX > 0 and an / log n → ∞. Note that the case an = O(log n) is included in Theorems 3.9 and 3.10. If h0 > 0, then the definition of bn and the equality γ(0) = EX imply that bn ∼ an EX. Hence, Theorem 3.4 yields the SLLN for increments. Theorem 3.8 implies SLLN as well. These results are unified in the next theorem. Theorem 3.16. Assume that EX > 0 and X − EX satisfies to the conditions of one of Theorems 3.11–3.15. Then Un Rn Tn Wn = lim sup = lim sup = lim sup = EX a.s. lim sup an an an an If log log n = o(log(n/an )) or an = n, then Wn Un lim = lim = EX a.s. an an

page 101

September 13, 2019

15:48

102

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

We assume in Theorem 3.16 that EX > 0, but one can easily check that the theorem holds for Wn in the case EX = 0 as well. For other statistics, these relations holds for EX ≤ 0, too. Note that Rn = Sn for an = n and one can replace lim sup by lim in this case. One can prove the result for non-negative random variables only when Sn is non-decreasing. Hence, we get SLLN for sums. Further, Wn = max Sk for an = n and maxima of sums and sums have the same 1≤k≤n

behaviour. Of course, the moment conditions in Theorem 3.16 are not optimal. Remember that the proof of the Kolmogorov SLLN essentially uses the formula for the norming sequence. This is impossible in the proof of the universal theorems since the structure of the norming sequence is more complicated. Moreover, the objects under investigation are more complicated than sums of independent random variables. We emphasize that we do not prove the SLLN and LIL separately. We derive them from the universal strong laws. The price for that is such non-optimality of moment assumptions in the SLLN and LIL for sums. Nevertheless, we will check in the next section that our moment assumptions in the theorems for increments are optimal. Proof. Assume that EX 2 = 1. Put hn = (2dn )1/2 and yn = (1/2dn )1/2 . Denote Yn = min{X − EX, yn }, Zn = Yn − EYn , yn = yn − EX. It is clear that Yn = Zn + EYn . By Lemma 2.10, we have mn (hn ) = EYn +hn (1+o(1)),

σn2 (hn ) = 1+o(1),

fn (hn ) =

h2n (1+o(1)). 2

Taking into account that EYn → EX, we get mn (hn ) → EX. By (2.4), we can put bn = an EX. We check the conditions of Theorem 3.8 in the same way as in the proof of Theorem 3.11. Assume that X > 0 and an = n. By Theorem 3.8, we have lim sup

Sn = EX n

a.s.

(3.33)

The random variable −X + 2EX is bounded and, therefore, it satisfies to the Cram´er condition. Applying of Theorem 3.8 to −X +2EX yields (3.33) with lim inf instead of lim sup. It yields the SLLN for X > 0. Using the representation X = X + − X − , we obtain the SLLN for arbitrary X. The cases F ∈ DN (α), α ∈ (1, 2) and F ∈ D(α), α ∈ (1, 2], can be considered in the same way. We omit details.

page 102

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

103

Strong Limit Theorems for Sums

3.4.5

Results for Moduli of Increments of Sums of Independent Random Variables

All previous results of this section can not hold for moduli of increments of sums of i.i.d. random variables. For moduli of increments, the random variables X and −X have to satisfy the one-sided moment conditions used above. The latter is only possible when EX 2 < ∞. In this case, we are able to derive corollaries from the previous results for Un =

max

0≤k≤n−an

Wn =

max

|Sk+an − Sk |, max |Sk+j − Sk |,

0≤k≤n−an 1≤j≤an

Rn = |Sn − Sn−an |,

Tn = |Sn+an − Sn |.

It is clear from the equality |X| = max{X, −X} that for instance, the norming for Wn has to be a maximum from the normings for Wn and the analogue of Wn constructed from −X’s. So, for short increments, we only can write the Erd˝ os–R´enyi law for X with a symmetrical distribution. The result is the same as Theorem 3.9. In non-symmetrical case, we can not specify the limit. Hence, we below deal with large increments only. Our first result is the next theorem. Theorem 3.17. Assume that EX = 0, EX 2 = 1 and one of the following conditions hold:  1) Eeh |X| < ∞ for some h > 0 and an / log n → ∞; β 2) Ee|X| < ∞ for some β ∈ (0, 1) and an ≥ C(log n)2/β−1 , where C = C(β) is an absolute positive constant; 3) E|X|p < ∞ for some p > 2 and an ≥ Cn2/p / log n, where C is an arbitrary positive constant. Then the relation lim sup

U R T Wn = lim sup n = lim sup n = lim sup n = 1 bn bn bn bn

holds with

a.s. (3.34)

& bn =

  n 2an log + log log n . an

If log log n = o(log(n/an )) in addition, then

lim

Wn U = lim n = 1 bn bn

a.s.

(3.35)

page 103

September 13, 2019

104

Un

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Theorem 3.17 follows from Theorem 3.11. Indeed, the inequality Rn ≤ ≤ Wn and the inequality

 max (−Sk+j + Sk ) Wn = max Wn , max 0≤k≤n−an 1≤j≤an

yield the upper bounds in (3.34) and (3.35). The lower bounds follow from the inequalities Rn ≤ Un ≤ Wn , Rn ≤ Rn , Un ≤ Un and Tn ≤ Tn . To this end, the one-sided conditions of Theorem 3.11 is replaced in Theorem 3.17 by the two-sided ones. Hence, the Cram´er condition, the Linnik condition and the power moment condition appear in 1)–3) correspondingly. The LIL for moduli of increments is as follows. Corollary 3.5. Assume that EX = 0, EX 2 = 1, E|X|p < ∞ for some p > 2 and log(n/an ) → q ≥ 0. log log n

(3.36)

Then  Wn Un lim sup √ = lim sup √ = 1+q 2an log log n 2an log log n

a.s.

(3.37)

Corollary 3.5 follows from Theorem 3.17 and (3.36). For an = n, q = 0 and EX 2 = 1, Corollary 3.5 yields the LIL for |Sn | and max |Sk | with the 1≤k≤n √ standard norming bn = 2n log log n. Reducing the range of {an }, we can relax the condition E|X|p < ∞ for p > 2 in the LIL for moduli of increments of sums. In this case, we have the next better result. Theorem 3.18. Assume that EX = 0, EX 2 = 1 and EX 2 (log(1 + |X|))τ < ∞ for some τ > 1/2. Suppose that relation (3.36) holds for some q ∈ [0, 2τ − 1) and an ≥ Cn(log n)−τ (log log n)−1 , where C is an arbitrary positive constant. Then (3.37) holds. Theorem 3.18 follows from Theorem 3.15 and the inequalities mentioned after Theorem 3.17. We see that our techniques is sharp enough. We have the LIL under the “2 + log” moment condition instead of 2, but we keep a zone for the sequence {an }. Of course, the best condition is the existence of the second moment for an = n.

page 104

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

105

Strong Limit Theorems for Sums

For SLLN, we have the following result. Theorem 3.19. Assume that either the conditions of Theorem 3.17, or the conditions of Theorem 3.18 hold. Then lim

U R T Wn = lim n = lim n = lim n = 0 an an an an

a.s.

Since considered functionals are non-negative, Theorem 3.19 follows from Theorems 3.17 and 3.18 and the relation an / log n → ∞ which implies that bn = o(an ). 3.5

Optimality of Moment Assumptions

We first prove the necessity of the one-sided moment conditions for the Cs¨ org˝o–R´ev´esz laws. We start with the Cram´er condition. Theorem 3.20. Assume that lim sup

Wn =1 bn

a.s.

for all sequences {an } with an / log n → ∞, where   an λ β L bn = λ−λ a1/α , n n βn

(3.38)

(3.39)

L(x) ∈ SV∞ , α ∈ (1, 2], λ = (α − 1)/α. Then h0 > 0. Proof. We need the next result. Lemma 3.4. Let X be a random variable with P (X ≥ 0) = 1. Assume that EeX/g(X) < ∞ for every non-decreasing function g(x) with g(x) → ∞ as x → ∞. Then h0 > 0. Proof. Assume that Eehx = ∞ for all h > 0. Let {hn } be a strictly decreasing sequence of positive numbers such that hn → 0. Define a sequence of positive numbers {xn } as follows. Take x1 > 0 such that Eeh1 X I{X 2. If xk is chosen, then we take xk+1 such that Eehk+1 X I{X 2Eehk X I{X 0, put k(y) =

y2 y 0

.

E|X|I{|X|>u} du

Let K(x) be the inverse function to k(x). Put cn = K(n/ log log n) log log n for n  3. Then Sn  1 a.s. lim sup cn Turn to the proof of Theorem 3.23. Proof. Assume that an = n for all n. We checked in the proofs of Theorem 3.11–3.14 that {bn } is equivalent to a non-decreasing sequence when an /n → 1. Without loss of generality, we further assume that {bn } is non-decreasing. It is clear that there exists a sequence of random indices {kn } such that kn ≤ n and Wn = max Sk = Skn . 1≤k≤n

Since bn is non-decreasing, we get Sk Sk Sn Wn = lim sup n ≤ lim sup n ≤ lim sup lim sup bn bn bkn bn

a.s.

From the other hand, the inequality Wn ≥ Sn holds. It follows that Sn lim sup = 1 a.s. (3.41) bn √ If α = 2 and L(x) ≡ 1, then bn = 2n log log n. In this case, the relations EX = 0 and EX 2 = 1 follow from Theorem 3.24. Relation (3.41) implies that Sn = 0 a.s. (3.42) n Hence, Theorem 3.25 yields that EX + < ∞, or EX − < ∞, or these two relations hold both. If EX = 0, then (3.42) contradicts to the SLLN. It follows that EX = 0. By Theorem 3.26, we have Sn ≥ 1 a.s. lim sup cn lim sup

page 109

September 13, 2019

15:48

110

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

This and relation (3.41) imply that lim inf

bn ≥ 1. cn

Hence, there exists a sequence {nk } such that nk ∞ as k → ∞ and cnk ≤ 2bnk . Note that   n bn = λ−λ n1/α (log log n)λ L . log log n It follows that for every ε > 0, the inequalities    1/α   nk nk nk K L ≤ 2λ−λ log log nk log log nk log log nk  1/(α−ε) nk ≤ 2λ−λ log log nk hold for all sufficiently large k. Put yn = K(n/ log log n). Then k(ynk ) ≥

ynα−ε k (2λ−λ )α−ε

for all sufficiently large k. The definition of k(y) and the last inequality yield that ynk

E|X|I{|X|>u} du ≤ (2λ−λ )−(α−ε) yn2−α+ε k

0

for all sufficiently large k. Taking into account that y

y E|X|I{|X|>u} du =

0

2

2

∞

u dF (u) + y P (|X| > y) + y −y

P (|X| > y)du, y

we have P (|X| > ynk ) ≤ (2λ−λ )−(α−ε) yn−α+ε k for all sufficiently large k. + β Applying one of conditions h0 > 0, Ee(X ) < ∞ or E(X + )p < ∞, we obtain that P (X > y) = o(P (X < −y)) as y → ∞. Hence, P (|X| > y) is a regularly varying function. Let be the order of P (|X| > y). Then the last inequality implies that ≤ −α. If < −α, then (3.41) holds with another norming sequences {bn } by Theorems 3.11—3.14. It follows that = −α. Hence, F belongs to a domain of attraction of the stable law with the c.f. (2.2).

page 110

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

111

Strong Limit Theorems for Sums

Assume that α < 2. Assume that L(x) ≡ 1. If F belongs to the domain of non-normal attraction, then (3.41) contradicts to Theorem 3.14. It follows that F belongs to the domain of normal attraction and P (X < cx) ∈ DN (α) for some c > 0. By Theorem 3.13, relation (3.41) holds with bn = cλ−λ n1/α (log log n)λ . It yields that c = 1 and F ∈ DN (α). If L(x) → 0 or L(x) → ∞ as x → ∞, then F can not belong to the domain of normal attraction by Theorem 3.13. It follows that F ∈ D(α). Assume that α = 2 and L(x) → ∞ as x → ∞. If F is from the domain of normal attraction of the normal law, then by Theorem 3.11, relation √ (3.41) holds with bn = c 2n log log n for some c > 0. This contradiction yields that F ∈ D(2). We finish this section with the necessary condition for the Erd˝ os–R´enyi law. It turns out that it is the one-sided Cram´er condition. Theorem 3.27. Assume that an = [c log n] for some c > 0 and (3.38) holds with bn = Can for some C > 0. Then h0 > 0. Proof. We get (3.40) in the same way as in the proof of Theorem 3.20. Since bn ∼ cC log n, the latter yields h0 > 0. 3.6

Necessary and Sufficient Conditions for the Cs¨ org˝ o– R´ ev´ esz Laws

Results of the previous section allow us to write the results for large increments under necessary and sufficient conditions. We do that in this section. Statements of results are very simple for the case of finite variations. We start with the strongest moment restrictions and the largest range of lengths of increments. In this case, Theorems 3.11, 3.13, 3.20 and 3.23 yield the following result. Theorem 3.28. Assume that α ∈ (1, 2]. If EX = 0, F ∈ DN (α) and h0 > 0, then lim sup

Wn 1/α

λ−λ an βnλ

=1

a.s.,

(3.43)

for every sequence {an } with an / log n → ∞, where λ = (α − 1)/α. One can replace Wn by Un , Tn and Rn in the last relation. If log log n = o(log(n/an )) in addition, then one can replace lim sup by lim in (3.43). This remains true for Un as well.

page 111

September 13, 2019

112

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Conversely, if α ∈ (1, 2], relation (3.43) holds for every sequence {an } with an / log n → ∞ and F (−x) ∈ RV∞ , then EX = 0, F ∈ DN (α) and h0 > 0. One can omit the regularity condition for F (−x) in the case α = 2. For α = 2, Theorem 3.28 yields the next result. Corollary 3.6. For every sequence {an } with an / log n → ∞, the relation Wn lim sup √ =1 2an βn

a.s.,

holds if and only if EX = 0, EX 2 = 1 and h0 > 0. Now, we relax the moment assumptions and reduce the range of lengths of increments. Theorems 3.11, 3.13, 3.21 and 3.23 imply the next result. Theorem 3.29. Assume that α ∈ (1, 2] and β ∈ (0, 1). + β If EX = 0, F ∈ DN (α) and Ee(X ) < ∞, then relation (3.43) holds for every sequence {an } with an ≥ C(log n)α/β−α+1 , where C = C(α, β) is an absolute positive constant. One can replace Wn by Un , Tn and Rn in (3.43). If log log n = o(log(n/an )) in addition, then one can replace lim sup by lim in (3.43). This remains true for Un as well. Conversely, if relation (3.43) holds for every sequence {an } with an ≥ C(log n)α/β−α+1 and F (−x) ∈ RV∞ , then EX = 0, F ∈ DN (α) and + β Eet0 (X ) < ∞ for some t0 > 0. One can omit the regularity condition for F (−x) in the case α = 2. Note that we do not specify the constant C(α, β) from the first part of Theorem 3.29. This is possible, but we only mention for simplicity that this constant exists. Hence, t0 appears in the Linnik type condition in the converse part of Theorem 3.29. Of course, if C coincides with C(α, β) in the converse part, then one can put t0 = 1. For α = 2, Theorem 3.29 implies the next result. Corollary 3.7. Assume that β ∈ (0, 1). For every sequence {an } with an ≥ C(log n)2/β−1 , the relation Wn lim sup √ =1 2an βn

a.s.,

holds if and only if EX = 0, EX 2 = 1 and Eet0 (X

+ β

)

< ∞ for some t0 > 0.

page 112

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

113

Strong Limit Theorems for Sums

Turn to the weakest moment assumptions and narrowest range of lengths of increments. In this case Theorems 3.11, 3.13, 3.22 and 3.23 yield the following result. Theorem 3.30. Assume that α ∈ (1, 2] and p > α. If EX = 0, F ∈ DN (α) and E(X + )p < ∞, then relation (3.43) holds for every sequence {an } with an ≥ Cnα/p /(log n)α−1 , where C is an arbitrary positive constant. One can replace Wn by Un , Tn and Rn in (3.43). If log log n = o(log(n/an )) in addition, then one can replace lim sup by lim in (3.43). This remains true for Un as well. Conversely, if relation (3.43) holds for every sequence {an } with an ≥ Cnα/p /(log n)α−1 , C > 0, and F (−x) ∈ RV∞ , then EX = 0, F ∈ DN (α) and E(X + )p < ∞. One can omit the regularity condition for F (−x) in the case α = 2. For α = 2, Theorem 3.30 gives the following result. Corollary 3.8. Assume that p > 2. For every sequence {an } with an ≥ Cn2/p / log n, the relation Wn =1 lim sup √ 2an βn

a.s.,

holds if and only if EX = 0, EX 2 = 1 and E(X + )p < ∞. Turn to the case F ∈ D(α). It is more complicated since a slowly varying multiplier appears in the norming sequence. For the strongest moment conditions and widest range of {an }, we have the next result which follows from Theorems 3.12, 3.14, 3.20 and 3.23. Theorem 3.31. Assume that α ∈ (1, 2]. If EX = 0, F ∈ D(α) and h0 > 0, then lim sup

Wn 1/α λ−λ an βnλ L(an /βn )

=1

a.s.,

(3.44)

for every sequence {an } with an / log n → ∞, where L(x) ∈ SV∞ and λ = (α − 1)/α. One can replace Wn by Un , Tn and Rn in the last relation. If log log n = o(log(n/an )) in addition, then one can replace lim sup by lim in (3.44). This remains true for Un as well. Conversely, if for every sequence {an } with an / log n → ∞, relation (3.44) holds with some function L(x) ∈ SV∞ , L(x) → 0 or L(x) → ∞, and F (−x) ∈ RV∞ , then EX = 0, F ∈ D(α) and h0 > 0.

page 113

September 13, 2019

114

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Remember that in the first part of Theorem 3.31, the function L(x) appears in relation (3.44) from the left tails of distribution of X for α < 2 and the truncated second moment of X for α = 2. In particular, L(x) can not tend to zero for α = 2. Relaxing moment assumptions and narrowing the range of {an }, we get the following result from Theorems 3.12, 3.14, 3.21 and 3.23. Theorem 3.32. Assume that α ∈ (1, 2] and β ∈ (0, 1). + β If EX = 0, F ∈ D(α) and Ee(X ) < ∞, then (3.44) holds for every sequence {an } with bn ≥ C(log n)1/β , where bn is the norming from (3.44), L(x) ∈ SV∞ , λ = (α − 1)/α and C = C(α, β). One can replace Wn by Un , Tn and Rn in the last relation. If log log n = o(log(n/an )) in addition, then one can replace lim sup by lim in (3.44). This remains true for Un as well. Conversely, if for every sequence {an } with bn ≥ C(log n)1/β , relation (3.44) holds with some function L(x) ∈ SV∞ , L(x) → 0 or L(x) → ∞, and + β F (−x) ∈ RV∞ , then EX = 0, F ∈ D(α) and Eet0 (X ) < ∞ for some t0 > 0. One can check that the condition bn ≥ C(log n)1/β turns to the condition of Theorem 3.29 provided we put L(x) ≡ 1 in the formula for bn . In the case of arbitrary L(x), one can not write a simple conditions on an . Note that if C from the converse part coincides with C(α, β) from the first part, then t0 = 1. For the weakest moment assumptions and narrowest range of {an }, the result is as follows. Theorem 3.33. Assume that α ∈ (1, 2] and p > α. If EX = 0, F ∈ D(α) and E(X + )p < ∞, then (3.44) holds for every sequence {an } with bn ≥ Cn1/p , where bn is the norming from (3.44), L(x) ∈ SV∞ , λ = (α − 1)/α and C > 0. One can replace Wn by Un , Tn and Rn in the last relation. If log log n = o(log(n/an )) in addition, then one can replace lim sup by lim in (3.44). This remains true for Un as well. Conversely, if for every sequence {an } with bn ≥ Cn1/p , relation (3.44) holds with some function L(x) ∈ SV∞ , L(x) → 0 or L(x) → ∞, and + β F (−x) ∈ RV∞ , then EX = 0, F ∈ D(α) and Eet0 (X ) < ∞ for some t0 > 0. Theorem 3.33 follows from Theorems 3.12, 3.14, 3.22 and 3.23.

page 114

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Strong Limit Theorems for Sums

9789811212826

115

Note that the condition bn ≥ Cn1/p turns to the condition of Theorem 3.30 when we put L(x) ≡ 1 in the formula for bn . One can not write a simple conditions on an for arbitrary function L(x).

3.7

Bibliographical Notes

The first result for increments of sums of i.i.d. random variables has been obtained in [Shepp (1964)]. Nevertheless, the real interest to this topic started from the paper [Erd˝os and R´enyi (1970)]. In [Cs¨org˝o (1979)], the Erd˝ os–R´enyi law has been proved under one-sided exponential moments. Full forms of the Erd˝ os–R´enyi and Shepp laws have been derived in [Deheuvels and Devroye (1987)]. The necessity of the Cram´er condition for the Erd˝os–R´enyi and Shepp laws has been proved in [Steinebach (1978)] and [Lynch (1983)] correspondingly. Bounds for convergence rates in the Erd˝ os–R´enyi and Shepp laws have been obtained in [Cs¨ org˝ o and Steinebach (1981)], [Deheuvels and Devroye (1987)]. The case an = o(log n) has been investigated in [Mason (1989)] and [Bacro and Brito (1991)]. Results for large increments may be found in [Cs¨org˝ o and R´ev´esz (1981)]. Certain results of this type have been derived by [Book (1975b)] and [Frolov (1990)]. Necessity of two-sided moment conditions in the Cs¨ org˝ o–R´ev´esz results for moduli of increments of sums has been established in [Shao (1989)]. In [Lanzinger (2000)] and [Lanzinger and Stadtm¨ uller (2000)], the behaviour of increments for an = [c(log n)2/p−1 ] has been investigated under a Linnik type condition. This is a border case between the Erd˝ os–R´enyi law and the Cs¨ org˝ o–R´ev´esz laws. Similar result for random variables from DN (α) may be found in [Terterov (2011)]. Results for maxima similar to max max (Sk+j − Sk )/f (k, n) has 0≤j≤n 0≤k≤n−j

been proved in [Shao (1995)], [Steinebach (1998)] and [Lanzinger and Stadtm¨ uller (2000)]. The behaviour of increments has been studied for non-negative random variables and random variables from the Feller class in [Einmahl and Mason (1996)]. The results are universal since minimal moment conditions are assumed while we suggest universality in another sense. Remember that our universal theorems work for a wider range for an under ”fixed” moment assumptions. M.Cs¨org˝ o and P.R´ev´esz used the Koml´ os–Major–Tushn´ ady strong approximation of sums by the Wiener process (see [Koml´os et al. (1975,

page 115

September 13, 2019

116

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

1976)]) and own results for increments of the Wiener process. We mentioned that the analysis of probabilities of large deviations is the strongest method of proofs of limit theorems on a.s. convergence. This method has been used for investigations of the a.s. behaviour of increments of sums of i.i.d. random variables in [Frolov (1990, 2000, 1998, 2002c, 2003b,d)]. In there, one can find one-sided generalisations of the Cs¨org˝ o–R´ev´esz results for random variables from domains of attraction of completely asymmetric stable laws. So, the universal theory has been built. It includes the SLLN, the LIL, the Erd˝ os–R´enyi and Shepp laws, the Mason extensions of the last laws and the Cs¨org˝ o–R´ev´esz laws. Moment assumptions are either optimal, or close to optimal. The converses of the Cs¨ org˝ o–R´ev´esz laws have been derived in [Frolov (2005)]. For independent, non-identically distributed random variables, the Erd˝ os–R´enyi and Shepp laws are obtained in [Steinebach (1981)] and [Frolov (1993b)], correspondingly. Bounds for rates of convergence in these laws are derived in [Lin (1990)], [Frolov (1993b,a)] and [Frolov et al. (1997)]. For the Cs¨org˝ o–R´ev´esz laws, one can confer [Book (1975b,a)], [Hanson and Russo (1985)], [Lin (1990)], [Lin et al. (1991)], [Cai (1992)], [Frolov (1991, 1993b, 2002a, 2004b)]. The universal strong laws are obtained in [Frolov (2004b)]. We now mention related results on LIL. Note that in this case, the large deviations method is the best as well. For the i.i.d. random variables, see, for example, [Klass (1976, 1977)] and [Pruitt (1981)]. Converses of the Hartman–Wintner theorem were independently obtained by [Martikainen (1980)], [Rosalsky (1980)] and [Pruitt (1981)]. LIL for asymmetric stable i.i.d. random variables may be found in [Mijnheer (1974, 1972)]. LIL for random variables from domain of attractions one can find in [Lipschutz (1956a)], [Brieman (1968)] and [Kalinauska˘ıte (1971)]. One-sided LIL for increments of independent, non-identically distributed random variables is proved by [Frolov (2004a)]. For an = n, results from [Frolov (2004a)] imply those in [Martikainen (1985)] which, in turn, are generalizations of the Kolmogorov LIL [Kolmogorov (1929)] and the Hartman–Wintner LIL [Hartman and Wintner (1941)] both. In this book, we deal with the i.i.d. case only. The results for nonidentically distributed random variables allow to include the Kolmogorov LIL and the Hartman–Wintner LIL in the universal theory. Remember that the Hartman–Wintner theorem may be proved with an application of the Kolmogorov LIL for truncated from above random variables and the SLLN for sums of remainder parts of summands. The same holds true for

page 116

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Strong Limit Theorems for Sums

9789811212826

117

increments as well. So, we can first build the universal theory for sums of independent, non-identically distributed random variables. Then we can finish the universal theory for i.i.d. case. This is done in [Frolov (2004b)]. In particular, a result of [Strassen (1964)] is derived for increments of sums of i.i.d. random variables. Detailed bibliography on LIL and SLLN may be found in monographs [Petrov (1975, 1987, 1995)].

page 117

b2530   International Strategic Relations and China’s National Security: World at the Crossroads

This page intentionally left blank

b2530_FM.indd 6

01-Sep-16 11:03:06 AM

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Chapter 4

Strong Limit Theorems for Processes with Independent Increments

Abstract. Universal strong laws are derived for homogeneous processes with independent increments. It follows the SLLN, the LIL, the Erd˝ os– R´enyi–Shepp laws and the Cs¨org˝o–R´ev´esz laws for such processes. The Wiener process, the Poisson process, the compound Poisson processes and stable processes are examples. 4.1

The Universal Strong Laws for Processes with Independent Increments

Let ξ(t), t ≥ 0, be a stochastically continuous, homogeneous process with independent increments such that ξ(0) = 0 a.s. and μ = Eξ(1) ∈ [0, ∞). There exists a modification of ξ(t) with trajectories from the space of c´adl`ag functions on [0, ∞) which we only deal with. Hence, we assume that ξ(t) is right continuous and has left limits. Let aT be a non-decreasing function such that T /aT is non-decreasing and 0 < aT ≤ T . Put WT = UT =

sup

sup (ξ(t + s) − ξ(t)),

0≤t≤T −aT 0≤s≤aT

sup

0≤t≤T −aT

(ξ(t + aT ) − ξ(t)),

QT = ξ(T + aT ) − ξ(T ),

RT = ξ(T ) − ξ(T − aT ).

Note that WT = sup ξ(s) and UT = RT = ξ(T ) for aT = T . 0≤s≤T

For every continuous function cT , the sets {UT < cT }, {UT < cT i.o.} etc. are events since they are determined by the process at rational points only. The same holds true for WT , RT , QT , sup ξ(t) and inf ξ(t). t≤T

t≤T

In this section, we describe the set of norming (non-random) functions 119

page 119

September 13, 2019

120

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

bT for which either UT = 1 a.s., bT or the last relation holds with lim instead of lim sup when it is possible. We assume in what follows that lim sup, lim inf, lim, O, o, → are taken as T → ∞ if not pointed otherwise. We follow the same pattern as that for sums of i.i.d. random variables. The only difference is that T is a continuous parameter. Put X = ξ(1) and h0 = sup{h : ϕ(h) = EehX < ∞}. If h0 > 0, then we define the functions of the LDT m(h), σ 2 (h), f (h), ζ(x) and γ(x) in the same way as in Chapter 3. We also put lim sup

γT (x) = γ(x),

for all T ≥ 0.

(4.1)

For h0 = 0, take a continuous function yT with yT → ∞. For all T ≥ 0, we put  Y − EX, for μ = 0, T YT = min{X, yT }, ZT = for μ > 0. YT , Then we define the functions of the LDT mT (h), σT2 (h), fT (h), ζT (x) and γT (x) for the random variable ZT in the same way as before. Put βT T , βT = log + log log max(T, 3). bT = aT γT (dT ) , where dT = aT aT This formula is the main result of this chapter in fact. We will see below that it is the formula of norming sequences in strong limit theorems for processes with independent increments. The first result is as follows. Theorem 4.1. Assume that bT is equivalent to a continuous, nondecreasing function and lim sup lim sup θ1

T →∞

bθT = 1. bT

(4.2)

Assume that one of the following conditions holds: 1) h0 > 0.  2) n P (ξ(1) ≥ bn ) < ∞ and for every small enough ε > 0, there exist τ > 0 and H ≥ 0 such that P (ξ((1 + ε)aT ) ≥ (1 + ε)bT ) ≤ e−(1+τ )βT + HaT P (ξ(1) ≥ bT ) for all sufficiently large T .

(4.3)

page 120

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Strong Limit Theorems for Processes with Independent Increments

9789811212826

121

Suppose that for every ε > 0, there exists q ∈ (0, 1) such that P (ξ(t) ≥ −εbT ) ≥ q

(4.4)

for all t ≤ (1 + ε)aT and all sufficiently large T . Then WT ≤ 1 a.s. (4.5) lim sup bT For aT → ∞, one can replace (1 + ε)aT by [(1 + ε)aT ] in (4.3). Here, [·] is the integer part of the number in brackets. Proof. We need the next result. Lemma 4.1. Let r, c ≥ 0 and q > 0. If P (ξ(t) ≥ −c) ≥ q for all t ≤ T , then   P sup (ξ(t) − ξ(s)) ≥ r ≤ q −2 P (ξ(T ) ≥ r − 2c). 0≤s≤t≤T

Proof. Put tkn = k2−n T for 1 ≤ k ≤ 2n and every n ∈ N. The result follows from Lemma 1.1 and     P sup (ξ(t)−ξ(s)) ≥ r = lim P sup (ξ(tjn )−ξ(tkn )) ≥ r . n→∞

0≤s≤t≤T

0≤tkn ≤tjn ≤T

Assume that condition 2) holds. Take ε ∈ (0, 1). Put AT = {WT ≥ (1 + 3ε)bT } and uT = εaT . By Lemma 4.1, we have PT = P (AT ) ⎛ ' [T /uT ]+1  ⎝ ≤P j=1

sup

(⎞ sup (ξ(t + s) − ξ(t)) ≥ (1 + 3ε)bT ⎠

(j−1)uT ≤t≤juT 0≤s≤aT

  (1 + ε)T P sup sup (ξ(t + s) − ξ(t)) ≥ (1 + 3ε)bT uT 0≤t≤uT 0≤s≤aT 2T ≤ 2 P (ξ((1 + ε)aT ) ≥ (1 + ε)bT ). q uT

≤

for all sufficiently large T . By inequality (4.3), we get

2T −(1+τ )βT PT ≤ 2 e + HaT P (ξ(1) ≥ bT ) q uT ≤ C1 (log T )−(1+τ ) + C2 T P (ξ(1) ≥ bT )

(4.6)

page 121

September 13, 2019

122

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

for all sufficiently large T . Take θ > 1. Put

nk = min n : θk−1 < n ≤ θk , nP (ξ(1) ≥ bn ) = for k ∈ N. Then the series

 k

min

θ k−1 0. Put ϕt (h) = Eehξ(t) . It is not difficult to check that ϕt (h) = (ϕ(h))t for all t > 0 and h ∈ (0, h0 ). Indeed, for every naturals k and n, ϕk/n (h) = (Eehξ(1/n) )k = (Eehξ(1) )k/n = (ϕ(h))k/n . The first equality follows by the independence of increments and the homogeneity of ξ(t). The second equality is a result of an application of the first one with k = n. Then for rational t, we have ϕt (h) = (ϕ(h))t . We get the last relation for all positive t by a limit passing over rational {tn } with tn → t as n → ∞. It yields that the function of large deviations and its inverse function for the random variable ξ(t) are tζ(z/t) and tγ(x/t) correspondingly. Take ε ∈ (0, 2). Put uT = εaT . By (4.6) and Tchebyshev’s inequality, we get PT ≤

2T −(1+ε)aT ζ(bT /aT ) 2T e = 2 e−(1+ε)βT 2 q uT q uT

for all sufficiently large T . Put nk = [θk ], where θ > 1. Then the series  Pnk converges. The remainder of the proof coincides with that for the k

case 2). For aT → ∞, we take uT = εaT − {(1 + ε)aT }, where {·} is a fractional part of the number in brackets. The condition aT → ∞ yields that uT > 0 for all sufficiently large T . The remainder of the proof is the same as that for the case 2). Our next result is the following theorem. Theorem 4.2. Assume that for every ε > 0 there exists τ > 0 such that P (ξ(aT ) ≥ (1 − ε)bT ) ≥ e−(1−τ )βT

(4.7)

page 122

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Strong Limit Theorems for Processes with Independent Increments

123

for all sufficiently large T . For aT /T → 1, assume additionally that conditions (4.2) and (4.4) are satisfied. Then RT ≥ 1 a.s. (4.8) lim sup bT One can replace RT by QT in the last relation. Proof. Take ε ∈ (0, 1). By (4.7), we get PT = P (RT ≥ (1 − ε)bT ) = P (ξ(aT ) ≥ (1 − ε)bT ) ≥ e−(1−τ )βT for all sufficiently large T . Suppose that aT /T → ρ ∈ [0, 1). Put n1 = 1 and nk+1 = min{n : n > nk , n − an ≥ nk } for k ∈ N. We have in the same way as in the proof of Theorem 1.2 that  Pnk diverges. Since the events the series k

{Rnk ≥ (1 − ε)bnk } are independent, the Borel–Cantelli lemma yields (4.8). Assume now that aT /T → 1. Take θ > 1 and put nk = [θk ], Ak = {ξ(nk ) − ξ(nk−1 ) ≥ (1 − 2ε)bnk }, Dk = {ξ(nk−1 ) − ξ(nk − ank ) ≥ −εbnk } for k ∈ N. Note that ank ∼ (nk − nk−1 )θ/(θ − 1) as k → ∞. This and (4.4) imply that P (Dk ) ≥ q for all sufficiently large k. The following pairs of events are independent: Ak and Dk , Ak and Dk Ak−1 Dk−1 , Ak and Dk Ak−1 Dk−1 Ak−2 Dk−2 ,. . . Choose Tk such that aTk = nk − nk−1 , k ∈ N. Then Tk ∼ aTk ∼ nk (θ − 1)/θ as k → ∞. If θ is large enough, then (θ − 1)/θ is close to 1 and we conclude by (4.2) that for every ρ > 0, the inequality bnk ≤ (1 + ρ)bTk holds for all sufficiently large k. Take ρ such that (1 − 2ε)(1 + ρ) = 1 − ε. Taking into account (4.7), we have P (Ak ) ≥ P (ξ(nk − nk−1 ) ≥ (1 − 2ε)bnk ) ≥ P (ξ(aTk ) ≥ (1 − ε)bTk ) ≥ e−(1−τ )βTk  for all sufficiently large k. This yields that the series P (Ak ) diverges. k

Lemma 1.3 implies that P (Ak Dk i.o.) ≥ q > 0. Hence, P (Rnk ≥ (1 − 3ε)bnk i.o.) > 0. By “0 or 1” law, the last probability equals to 1 and Theorem 4.2 follows for RT . For QT , the proof follows a similar way.

page 123

September 13, 2019

124

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

The next result is useful to check conditions (4.4) and (4.7). Remark 4.1. 1) If aT > 1 and bT → ∞, then one can check inequality (4.4) for t = [(1 + ε)aT ] only. 2) If bT → ∞, then one can replace ξ(aT ) by ξ([aT ]) in inequality (4.7). Proof. Assume that for every ε > 0 there exists q ∈ (0, 1) such that inequality (4.4) holds with t = [(1 + ε)aT ] for all T ≥ T0 . By mT =

min

0≤k≤[(1+ε)aT0 ]

P (ξ(k) ≥ −εbT ) → 1,

we get mT ≥ q for all T ≥ T1 ≥ T0 . Suppose that T ≥ T1 . If [(1 + ε)aT0 ] ≤ k ≤ [(1 + ε)aT ], then k = [(1 + ε)as ] for some s ∈ [T1 , T ]. Using that bT is non-decreasing, we have P (ξ(k) ≥ −εbT ) ≥ P (ξ(k) ≥ −εbs ) ≥ q. Thus, (4.4) holds for t = k = 1, 2, . . . , [(1 + ε)aT ]. Assume now that k ≤ t ≤ k + 1 ≤ [(1 + ε)aT ] + 1. Taking into account the independence of increments, homogeneity of the process and bT → ∞, we get P (ξ(t) ≥ −2εbT ) ≥ P (ξ(k) ≥ −εbT )P (ξ(t) − ξ(k) ≥ −εbT )   ≥ qP (ξ(t − k) ≥ −εbT ) ≥ qP inf ξ(s) ≥ −εbT ≥ 0.5q 0≤s≤1

for all T ≥ T2 ≥ T1 . The first assertion follows. Turn to the next one. By the independence of increments and the homogeneity of ξ(t) and bT → ∞, we have P (ξ(aT ) ≥ (1 − 2ε)bT ) ≥ P (ξ([aT ]) ≥ (1 − ε)bT )P (ξ(aT ) − ξ([aT ]) ≥ −εbT )   ≥ P (ξ([aT ]) ≥ (1 − ε)bT )P inf ξ(s) ≥ −εbT 0≤s≤1

≥ 0.5P (ξ([aT ]) ≥ (1 − ε)bT ) for all sufficiently large T . The next result is as follows. Theorem 4.3. Assume that bT is equivalent to a non-decreasing function, bT → ∞ and condition (4.2) is satisfied. Suppose that the conditions of Theorem 4.2 hold and log log T = o(log(T /aT )). Then UT lim inf ≥ 1 a.s. bT

page 124

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

125

Strong Limit Theorems for Processes with Independent Increments

Proof. Assume that j ≤ T < j + 1, where j ≥ 2 and j/aj > 1. Put nj = [j/aj ] − 1, Ik =

inf

aj ≤s≤aj+1

(ξ(kaj + s) − ξ(kaj ))

for k ∈ K, where K = {0, 2, . . . , 2([nj /2] − 1)}. It is clear that UT ≥ max Ik = Mj . k∈K

By aj /j ≥ aj+1 /(j + 1), we have 0 ≤ aj+1 − aj ≤ aj /j ≤ 1 and kaj + aj+1 ≤ (k + 1 + 1/j)aj < (k + 2)aj . Hence, pj = P (Mj ≤ (1 − 2ε)bj ) = (P (I0 ≤ (1 − 2ε)bj ))[nj /2] ≤ e−[nj /2]P (I0 ≥(1−2ε)bj ) . We have used here the independence of increments and the homogeneity of ξ(t). Using these properties again, we get P (I0 ≥ (1 − 2ε)bj )





≥ P (ξ(aj ) ≥ (1 − ε)bj ) P inf (ξ(s) − ξ(aj )) ≥ −εbj aj ≤s≤aj+1   = P (ξ(aj ) ≥ (1 − ε)bj )P inf ξ(s) ≥ −εbj 0≤s≤aj+1 −aj   ≥ P (ξ(aj ) ≥ (1 − ε)bj )P inf ξ(s) ≥ −εbj 0≤s≤1

≥ 0.5P (ξ(aj ) ≥ (1 − ε)bj ) for all sufficiently large j. Taking into account (4.7), we obtain   )n * j pj ≤ exp −0.5 e−(1−τ )βj 2  for all sufficiently large j. By log log T = o(log(T /aT )), the series pj j

converges. The Borel–Cantelli lemma implies that with probability 1, the inequality Mj > (1 − 2ε)bj holds for all sufficiently large j. Since bj+1 ∼ bj as j → ∞, we conclude that with probability 1, the inequality UT > (1 − 3ε)bT holds for all sufficiently large T . Theorem 4.3 follows. Theorems 4.1–4.3 and the inequalities RT ≤ UT ≤ WT yield the next result. Theorem 4.4. If the conditions of Theorems 4.1 and 4.2 hold, then UT RT QT WT = lim sup = lim sup = lim sup = 1 a.s. (4.9) lim sup bT bT bT bT If the conditions of Theorems 4.1 and 4.3 hold, then WT UT lim = lim = 1 a.s. (4.10) bT bT

page 125

September 13, 2019

126

4.2

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Strong Laws for Increments of Wiener and Stable Processes without Positive Jumps

In this section, we state corollaries of Theorem 4.4 for important processes with independent increments: the Wiener process and the stable processes without positive jumps. In the last case, increments are assumed to have asymmetric stable distributions with α ∈ (1, 2). For these processes, norming functions can be calculated for a largest range of aT . For the Gaussian case, we assume in addition that ξ(t) is a modification of the Wiener process which has continuous trajectories w.p. 1. We concern with the processes having non-negative linear drifts, that is the c.f. of ξ(1) is  

t π α , (4.11) ψ(t) = exp iμt − c|t| 1 + i tg α |t| 2 where c = cos(π(2 − α)/2)/α, 1 < α ≤ 2 and μ ≥ 0. Since ξ(1) − μ has the c.f. (2.2), we have  (α−1)/α αx γ(x) = μ + α−1 for all x ≥ 0. Taking this into account, we arrive at the next result. Theorem 4.5. If ξ(1) has c.f. (4.11) with μ ≥ 0, then relation (4.9) holds with λ  T −λ 1/α + log log T , (4.12) log bT = μaT + λ aT aT where λ = (α − 1)/α. If, in addition, log log T = o(log(T /aT )) or aT = T and μ > 0, then (4.10) holds with bT from (4.12). Theorem 4.5 follows from Theorem 4.4 and the formula for bT . Condition (4.2) may be easily checked. Condition (4.4) for the stable laws holds obviously. One can check inequality (4.7) with an application of Remark 4.1 and Lemma 2.1. For aT = O(log T ) and μ > 0, we get Theorem 4.5 from the result for μ = 0 and the properties of γ(x). For aT → ∞ and μ > 0, Theorem 4.5 follows from Theorem 4.13 which will be proved below. Theorem 4.5 includes the SLLN, the LIL, the Erd˝ os–R´enyi law and the Cs¨ org˝ o–R´ev´esz laws for the Wiener process and the stable processes without positive jumps.

page 126

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

127

Strong Limit Theorems for Processes with Independent Increments

If α = 2 and μ = 0, then ξ(t) is the standard Wiener process and &   T bT = 2aT log + log log T . (4.13) aT √ For aT = T , we have the LIL with bT = 2T log log T . If α < 2 and μ = 0, then ξ(t) is a stable process without positive jumps and the LIL holds with bT = λ−λ T 1/α (log log T )λ . Note that for μ = 0, Theorem 4.5 also yields the Erd˝os–R´enyi law and the Mason’s extension of this law when aT = c log T and aT = o(log T ) correspondingly. Remember, that c0 = 0 in this case. If μ > 0 and aT = O(log T ), the situation is the same. When μ > 0 and aT / log T → ∞, then μaT is the dominating part of the norming in (4.12). We consider this case in Theorem 4.13 below. From there, the SLLN follows, in particular. We included the SLLN for the Wiener process and the stable processes without positive jumps (α ∈ (1, 2)) in Theorem 4.5 as the case aT = T for μ > 0. Note that the latter yields the SLLN for arbitrary μ. One realizes that the behaviours of the Wiener and the stable process without positive jumps are very similar. Nevertheless, this similarity of is not absolute. To see that, we have the result which holds for the Wiener process only. Theorem 4.6. If ξ(t) is the standard Wiener process, then the conclusion of Theorem 4.5 holds true provided the increments of the process are replaced by their moduli in Wn , Un , Rn and Tn . 4.3

Applications of the Universal Strong Laws

In this section, we discuss various corollaries of Theorem 4.4 for arbitrary processes with independent increments satisfying the conditions of Section 4.1. In general case, we can not write one simple formula of norming sequence for all aT as in the previous section. Hence, we further consider small and large increments separately. Assume first that aT = O(log T ) and start with the Erd˝os–R´enyi and Spepp laws for processes with independent increments. Theorem 4.7. If h0 > 0 and aT = c log T , c > c0 , then lim

WT UT RT QT = lim = lim sup = lim sup =γ aT aT aT aT

  1 c

a.s.

page 127

September 13, 2019

128

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Theorem 4.7 follows from Theorem 4.4 and the formula for bT . Condition (4.2) may be easily checked. Condition (4.4) for the stable laws holds obviously. In Theorem 4.7, condition (4.4) follows from Remark 4.1 and the weak law of large numbers for i.i.d. random variables. One can check inequality (4.7) with an application of Remark 4.1 and Lemma 2.1. Suppose now that aT = o(log T ). In this case, we have the following analogue of Mason’s extension for the Erd˝ os–R´enyi law. Theorem 4.8. Assume that h0 > 0 and aT = o(log T ). If the conditions of Theorems 4.1 and 4.2 are satisfied, then relation (4.9) holds with bT = aT γ(log T /aT ). If the conditions of Theorems 4.1 and 4.3 are satisfied, then (4.10) holds with the same bT . Theorem 4.8 follows from Theorem 4.4, the formula for bT and the concavity of the function γ(x). We stated the Erd˝os–R´enyi and Spepp laws for processes with independent increments for c > c0 only. For c ≤ c0 , results are similar to that for sums of i.i.d. random variables. We do not state them here. Assume now that aT / log T → ∞. The behaviour of large increments depends on moment conditions on the right-hand tail of ξ(1) in the same manner as for sums of i.i.d. random variables. It is clear that results for centered (μ = 0) and non-centered increments are quite different. For example, if aT = T , then we have the LIL for centered increments and the SLLN for non-centered ones. We deal with these cases separately in the sequel. Consider first centered processes. We have Cs¨ org˝ o–R´ev´esz laws in this case. Put ξ + (1) = max{ξ(1), 0}. Theorem 4.9. Assume that μ = 0, Eξ 2 (1) = 1 and one the following conditions holds: 1) h0 > 0 and aT / log T → ∞; + β 2) Ee(ξ (1)) < ∞ for some β ∈ (0, 1) and aT ≥ C(log T )2/β−1 , where C = C(β) is an absolute positive constant; 3) Eξ + (1)p < ∞ for some p > 2 and aT ≥ CT 2/p / log T where C is an arbitrary positive constant. Then (4.9) holds with bT from (4.13). If, in addition, log log T = o(log(T /aT )), then (4.10) holds with bT from (4.13). Theorem 4.9 yields the following result on the LIL for increments of

page 128

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

129

Strong Limit Theorems for Processes with Independent Increments

processes with independent increments. Corollary 4.1. Assume that μ = 0, Eξ 2 (1) = 1 and Eξ + (1)p < ∞ for some p > 2. If log(T /aT )/ log log T → c, then √ UT WT = lim sup √ = 1+c lim sup √ 2aT log log T 2aT log log T

a.s.

If aT = T , then from Corollary 4.1, we obtain the LIL for processes with independent increments. Turn to the case Eξ 2 (1) = ∞. Put F (x) = P (ξ(1) < x). We start with the case of non-normal attraction to the normal law. We then have the following result. Theorem 4.10. Assume that μ = 0, F ∈ D(2) and one of conditions 1) − 3) of Theorem 4.9 is satisfied. Then (4.9) holds with ˆ fˆ−1 (dT )), bT = aT m(

(4.14)

where   1 m(h) ˆ = hG , h

h2 G fˆ(h) = 2

  1 , h

0 G(x) =

u2 dF (u), x > 0.

−x

(G(x) ∈ SV∞ .) If, in addition, log log T = o(log(T /aT )), then (4.10) holds with bT from (4.14). If log(T /aT )/ log log T → c, then Theorem 4.10 implies the LIL for increments of processes under consideration. For aT = T , we arrive at the LIL. The result is similar to that of Corollary 4.1, but a slowly varying multiplier will appear in the normalizing function. Turn to the case α ∈ (1, 2). For domains of normal attraction of the stable laws, the result is as follows. Theorem 4.11. Assume that μ = 0, F ∈ DN (α) for some α ∈ (1, 2) and one of the following conditions holds: 1) h0 > 0 and aT / log T → ∞; + β 2) Ee(ξ (1)) < ∞ for some β ∈ (0, 1) and aT ≥ C(log T )α/β−α−1 , where C = C(α, β) is an absolute positive constant; 3) Eξ + (1)p < ∞ for some p > 2, aT ≥ CT α/p /(log T )α−1 where C is an arbitrary positive constant. Then (4.9) holds with bT from (4.12). If, in addition, log log T = o(log(T /aT )), then (4.10) holds with bT from (4.12).

page 129

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

130

9789811212826

Universal theory for strong limit theorems of probability

Theorem 4.11 implies the next result on the LIL for increments of processes with independent increments. Corollary 4.2. Assume that μ = 0, Eξ 2 (1) = 1 and Eξ + (1)p < ∞ for some p > 2. If log(T /aT )/ log log T → c, then lim sup

WT 1/α λ−λ aT (log log T )λ

= lim sup

UT 1/α λ−λ aT (log log T )λ

= (1 + c)λ

a.s.

For aT = T , then Corollary 4.2 turns to the LIL for processes with independent increments. For domains of non-normal attraction, we have the following result. Theorem 4.12. Assume that μ = 0 and F ∈ D(α) for some α ∈ (1, 2). Put ˆ fˆ−1 (dT )), bT = aT m( where m(h) ˆ =

αΓ(2 − α) α−1 h G α−1

  1 , h

fˆ(h) = Γ(2 − α)hα G

(4.15)   1 , h

and G(x) = xα F (−x), x > 0. (G(x) ∈ SV∞ .) Assume that one the following conditions holds: 1) h0 > 0 and aT / log T → ∞; + β 2) Ee(ξ (1)) < ∞ for some β ∈ (0, 1) and bT ≥ C(log T )1/β , where C = C(α, β) is an absolute positive constant; 3) Eξ + (1)p < ∞ for some p > α and bT ≥ CT 1/p , where C is an arbitrary positive constant. Then (4.9) holds with bT from (4.15). If, in addition, log log T = o(log(T /aT )), then (4.10) holds with bT from (4.15). For log(T /aT )/ log log T → c, Theorem 4.10 yields the LIL for increments and the LIL for processes with independent increments when aT = T . The result is similar to that of Corollary 4.2, but a slowly varying multiplier will appear in the normalizing function. Proofs of Theorems 4.9–4.12 consist of choices of yT (if h0 = 0), calculations of bT and verifications of corresponding conditions for bT . They are n  (ξ(k)− the same as for sums of i.i.d. random variables. Indeed, ξ(n) = k=1

ξ(k − 1)), where {ξ(k) − ξ(k − 1)} is a sequence of i.i.d. random variables. Remember that we may write the conditions of Theorem 4.4 for natural times. Thus, we omit details.

page 130

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Strong Limit Theorems for Processes with Independent Increments

131

We finally deal with the case μ > 0. Theorem 4.13. Assume that the conditions of one of Theorems 4.9–4.12 are satisfied and μ > 0 (instead of μ = 0). Then (4.9) holds with bT = μaT . If, in addition, log log T = o(log(T /aT )) or aT = T , then (4.10) holds with the same bT . Theorem 4.13 for aT = T implies the SLLN for processes with μ > 0. Applying of this result to the process ξ(t) − μt + t, we get the SLLN for μ ≤ 0 as well. Proof. We have bn ∼ μan as n → ∞ (see the proof of Theorem 3.16). If n ≤ T < n + 1, then bn ≤ bT ≤ (1 + ε)μan+1 ≤ (1 + ε)μaT (n + 1)/T for every ε > 0 and all sufficiently large n. By Theorem 4.4, we get the result besides the SLLN in the case aT = T for all T . Assume now that aT = T . Then (see the proof of Theorem 3.16) ξ(n)/n → μ a.s. Put

 sup |ξ(t) − ξ(n)| ≥ 2εn An = n−1≤t 0. For all n, we have   P (An ) ≤ P sup |ξ(t)| ≥ 2εn  =P

0≤t≤1

   sup ξ(t) ≥ 2εn + P sup (−ξ(t)) ≥ 2εn .

0≤t≤1

For all t ∈ [0, 1], we get P (ξ(t) ≥ −c) ≥ P

0≤t≤1



 sup ξ(t) ≥ −c

0≤t≤1

≥ 0.5

provided c is large enough. By Lemma 4.1, we obtain   P sup ξ(t) ≥ 2εn ≤ 4P (ξ(1) ≥ 2εn − 2c) ≤ 4P (ξ(1) ≥ εn) 0≤t≤1

for all sufficiently large n. Note that Lemma 4.1 holds for −ξ(t) as well. In the same way, we prove that   P sup (−ξ(t)) ≥ 2εn ≤ 4P (−ξ(1) ≥ εn) 0≤t≤1

for all sufficiently large n.

page 131

September 13, 2019

15:48

132

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

It follows that P (An ) ≤ 4P (|ξ(1)| ≥ εn) for all sufficiently large n. Since E|ξ(1)| < ∞, the series



P (An ) con-

n

verges. By the Borel–Cantelli lemma, we have P (An i.o.) = 0 and the result follows. In the proof of Theorem 4.13, we applied Theorem 3.16 that is the corollary of the universal strong laws. Previous results can not hold for moduli of increments of all considered processes. In this case, ξ(1) and −ξ(1) have to satisfy the same moment assumptions of the theorems discussed above. The latter is only possible for Eξ(1) = 0 and Eξ(1)2 < ∞. Turn now to results for moduli of increments. Put WT = UT =

sup

sup |ξ(t + s) − ξ(t)|,

0≤t≤T −aT 0≤s≤aT

sup

0≤t≤T −aT

|ξ(t + aT ) − ξ(t)|,

QT = |ξ(T + aT ) − ξ(T )|,

RT = |ξ(T ) − ξ(T − aT )|.

From Theorem 4.9 and the equality |X| = max{X, −X}, we obtain the following result. Theorem 4.14. Assume that μ = 0, Eξ 2 (1) = 1 and one the following conditions holds:  1) Eeh |ξ(1)| < ∞ for some h > 0 and aT / log T → ∞; β 2) Ee|ξ(1)| < ∞ for some β ∈ (0, 1) and aT ≥ C(log T )2/β−1 , where C = C(β) is an absolute positive constant; 3) E|ξ(1)|p < ∞ for some p > 2 and aT ≥ CT 2/p / log T where C is an arbitrary positive constant. Then the relation lim sup

WT U R Q = lim sup T = lim sup T = lim sup T = 1 bT bT bT bT

a.s. (4.16)

holds with bT from (4.13). If, in addition, log log T = o(log(T /aT )), then the relation lim holds with bT from (4.13).

U WT = lim T = 1 bT bT

a.s.

(4.17)

page 132

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

133

Strong Limit Theorems for Processes with Independent Increments

Theorem 4.14 is the Cs˝org¨ o–R´ev´esz law for moduli of increments of processes with independent increments. Theorem 4.14 implies the following result on the LIL for moduli of increments of processes with independent increments. Corollary 4.3. Assume that μ = 0, Eξ 2 (1) = 1 and E|ξ(1)|p < ∞ for some p > 2. If log(T /aT )/ log log T → c, then √ WT UT lim sup √ = lim sup √ = 1+c 2aT log log T 2aT log log T

a.s.

If aT = T , then from Corollary 4.3, we obtain the LIL for moduli of processes with independent increments. For SLLN, we get the next result. Theorem 4.15. If the conditions of Theorem 4.14 hold, then lim

U R Q WT = lim T = lim T = lim T = 0 aT aT aT aT

a.s.

In view of non-negativity of the considered functionals, Theorem 4.15 follows from Theorem 4.14 and the relation bn = o(an ) which is a result of an / log n → ∞. 4.4

Compound Poisson Processes

Now we describe an important class of processes with independent increments which the results of previous section hold for. Let {Xk } be a sequence of i.i.d. random variables and ν(t) be a Poisson process independent with X’s. Then the process ξ(t) =

ν(t) 

Xk ,

ξ(0) = 0,

k=1

is called the Compound Poisson process. Such processes play an important role in the probability theory and its applications. In financial and actuarial mathematics, they are models for claim processes. For example, if ν(t) is the number of claims up to time t and Xi is the amount of i-th claim, then ξ(t) is the aggregate amount of claims up to time t over a portfolio of insurance policies of the same type. It is not difficult to check that the c.f. of ξ(t) is Ψt (u) = Eeiuξ(t) = eνt(g(u)−1) ,

page 133

September 13, 2019

134

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

where g(u) = EeiuX1 and ν is a parameter of the Poisson process ν(t). It implies that ξ(t) is a process with independent increments. Moreover, the formula for Ψ1 (u) allows to calculate moments of ξ(1) provided they exist. In particular, calculating the derivatives of Ψ1 (u) at zero, we get μ = Eξ(1) = νEX1 ,

Eξ(1)2 = νEX12 + ν 2 (EX1 )2 ,

Dξ(1) = νEX12 .

The latter means that ξ(t) may be centered and non-centered. Remember that the results of previous section crucially depends on that. Further, it is not difficult to check that the m.g.f. of ξ(1) satisfies the relation hX1 −1

ϕ(h) = eν (Ee

)

for every h such that EehX1 < ∞. To apply the results of the previous section for Eξ(1)2 = ∞, we need conditions sufficient for F ∈ D(α), where F (x) is the d.f. of ξ(1). It turns out that the d.f. of ξ(1) belongs to the domain of attraction of considered stable laws provided the d.f. of X1 belongs to the same domain of attraction. To check this, we prove the next result. Lemma 4.2. If EX1 = 0 and P (X1 < x) ∈ DN (α) or P (X1 < x) ∈ DN (α) with α ∈ (1, 2], then F (x) ∈ DN (α) or F (x) ∈ DN (α) correspondingly. Proof. If EX1 = 0 and the d.f. of X1 is from the domain of attraction of the stable law with the c.f. ψ(t) from (2.2), then for every fixed u ∈ R, we have   u → ψ(u), gn Bn where {Bn } is an appropriate sequence of norming constant with Bn → ∞ as n → ∞. It turns out that        u u n n − 1 = n log g + o(1) = log ψ(u) + o(1) n g Bn Bn as n → ∞. It follows that  

    u u n n = exp n g −1 → ψ(u) Ψ1 Bn Bn as n → ∞. The latter means that ξ(1) is from the domain of attraction of the same stable law as X1 .

page 134

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Strong Limit Theorems for Processes with Independent Increments

9789811212826

135

It follows that all results of the previous section hold under appropriate assumptions on the distribution of X1 . If P (X1 = 1) = 1 in the definition of the Compound Poisson process, then ξ(t) tuns to the Poisson process. It is not centered and, therefore, we have the SLLN (Theorem 4.13) and the Erd˝os–R´enyi laws (Theorems 4.7 and 4.8). But for ξ(t) centered at mean, we will have the Cs¨org˝ o– R´ev´esz laws (Theorem 4.9) and LIL (Corollary 4.1). Results for moduli of increments holds as well. 4.5

Bibliographical Notes

Results for increments of Wiener process are proved by [Book and Shore (1978)], [Cs´aki and R´ev´esz (1979)], [Cs¨org˝o and R´ev´esz (1979)], [Hanson and Russo (1983)] and [Ortega and Wschebor (1984)]. Increments of stable processes with jumps of one sign have been studied by [Zinchenko (1987)]. LIL for asymmetric stable processes may be found in [Fristedt (1964)], [Brieman (1968)], [Millar (1972)] and [Mijnheer (1974)]. The universal strong laws for homogeneous processes with independent increments are obtained in [Frolov (2003a, 2004c)]. Many results on increments of various stochastic processes (processes related with the Wiener process, empirical processes, local times) and references may be found in monographs [Cs¨ org˝ o and R´ev´esz (1981)] and [R´ev´esz (1990)]. In this book, we do not deal with random fields, but the situation is similar to those for sums of i.i.d. random variables and processes with independent increments. The behaviour of increments of random fields is investigated in [Steinebach (1983)], [Deheuvels (1985)], [Pfuhl and Steinebach (1988)], [Scherbakova (2003, 2004)], [Frolov (2003c, 2002d)]. Asymptotics of increments of Gaussian and stable random fields may be derived from those of corresponding multiparameter processes. Results for multiparameter Gaussian processes are obtained in [Chan (1976)], [Cs¨ org˝ o and R´ev´esz (1978, 1981)], [Lin et al. (2001)], [Choi and Kˆono (1999)]. Results for stable processes are proved in [Zinchenko (1994)]. The universal strong laws for random fields may be found in [Frolov (2002d, 2003c)].

page 135

b2530   International Strategic Relations and China’s National Security: World at the Crossroads

This page intentionally left blank

b2530_FM.indd 6

01-Sep-16 11:03:06 AM

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Chapter 5

Strong Limit Theorems for Renewal Processes

Abstract. Universal strong laws are derived for renewal processes. Results include the SLLN, the LIL, the Erd˝os–R´enyi–Shepp laws and the Cs¨org˝ o– R´ev´esz laws. Poisson processes are partial cases. 5.1

The universal Strong Laws for Renewal Processes

Let Y, Y1 , Y2 , . . . be a sequence of i.i.d. non-degenerate random variables such that ess inf Y = 0 and μ = EY < ∞. Put R0 = 0, Rn = Y1 + Y2 + · · · + Yn for n ∈ N and define the renewal process by N (t) = max{n ≥ 0 : Rn ≤ t},

t ≥ 0.

Note that if Y has the exponential distribution with the density p(x) =

1 −x/μ e I[0,+∞) (x), μ

then N (t) is the Poisson process and EN (t) = t/μ. Let aT be a non-decreasing, positive function such that aT ≤ T , T /aT is a non-decreasing function and aT → ∞. Put s

, WT = sup sup N (t + s) − N (t) − μ 0≤t≤T −aT 0≤s≤aT aT uT = . sup (N (t + aT ) − N (t)), UT = uT − μ 0≤t≤T −aT Note that uT = N (T ) for aT = T . The maximum UT is a centered variant of uT . The centering function corresponds to that from the SLLN for renewal processes. A non-centered analogue for WT coincides with uT since N (T ) is non-decreasing. 137

page 137

September 13, 2019

138

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

We find necessary and/or sufficient conditions for lim sup

WT =1 bT

a.s.,

where bT is a non-decreasing positive function. Sometimes, one can replace lim sup by lim. The behaviour of UT and uT is discussed as well. Writing lim sup, lim inf, lim, O, o, →, we assume that T → ∞, if it is not pointed otherwise. We follow the pattern of the previous chapter. We first find a formula for the norming function bT . Further, we state the universal strong laws for increments of renewal processes. Finally, we derive corollaries which include the Erd˝os–R´enyi and Shepp laws, the Cs¨org˝o–R´ev´esz laws, the LIL and the SLLN. We consider renewal times from domains of attraction of completely asymmetric stable laws with the exponent α ∈ (1, 2]. Renewal times with finite variations are partial cases. Put X = μ − Y and F (x) = P (X < x). The random variable X is bounded from above and ω = μ. It is clear that either F ∈ K1 , or F ∈ K2 . We use the functions of the LDT ϕ(h) = EehX , m(h), σ 2 (h), f (h), ζ(z) and γ(x) from Chapter 2. Let cT be a non-decreasing function such that log(T /c ) + log log T

T cT μ − γ (5.1) = aT cT and T /cT is non-decreasing for all sufficiently large T . If F ∈ K2 , then we assume in addition that cT / log T > c0 . It is not difficult to check that by the properties of γ(x), relation (5.1) determines the function cT from aT and, moreover, cT > aT /μ for all T . Put  c − (a /μ), for UT and WT , T T (5.2) bT = for uT . cT , It is clear that bT > 0 for all T . Since aT → ∞, we have bT → ∞. For uT , this is obvious. Otherwise, it easily follows from the properties of γ(x) and log(T /c ) + log log T

T . (5.3) μbT = cT γ cT The latter is another form of relation (5.1). For F ∈ K2 and cT / log T → c0 , relations (5.1) and (5.3) yield that aT = o(cT ) and bT ∼ c0 log T . We assume in what follows that bT is equivalent to a non-decreasing function.

page 138

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Strong Limit Theorems for Renewal Processes

9789811212826

139

We start with the next result. Theorem 5.1. Assume that lim sup lim sup θ1

T →∞

bθT =1 bT

and for every ε > 0 there exists q ∈ (0, 1) such that P (S[(1+ε)(1+3ε)cT ] ≥ −εμbT ) ≥ q

(5.4)

for all sufficiently large T . Here and in the sequel, Sn = nμ − Rn for all n ∈ N and [·] is the integer part of the number in brackets. Then WT lim sup ≤ 1 a.s. bT One can replace WT by uT in the last relation. For cT = O(log T ), one can omit condition (5.4). Remember that replacing WT by uT , one changes bT in view of (5.2). We first establish a duality between increments of renewal processes and sums Sn . It is a basic tool of our proofs below. Lemma 5.1. The following relations hold:

 max max (Sk+j − Sk ) ≥ μx ,(5.5) {WT ≥ x + 1} ⊂ [x]≤j≤[x+μ−1 aT ]+1 0≤k≤N (T )−j

 max −1 (Sk+[x+μ−1 aT ] − Sk ) < μx ,(5.6) {UT < x − 2} ⊂ 0≤k≤N (T )−[x+μ

aT ]+1

{UT < x − 2} ⊂ {S[x+μ−1 aT ] < μx},

 min (Rk+[x] − Rk ) ≥ aT . {uT < x − 2} ⊂ 0≤k≤N (T )−[x]+1

(5.7) (5.8)

Proof. If WT ≥ x + 1, then N (t + s) − (x + 1 + sμ−1 ) ≥ N (t) for some s ∈ [0, aT ] and t ∈ [0, T − aT ]. Since Rn is non-decreasing, we have R[N (t+s)−(x+1+sμ−1 )]+1 ≥ RN (t)+1 > t ≥ RN (t+s) − s by the definition of N (t). The last inequality yields that SN (t+s) − S[N (t+s)−(x+sμ−1 )] ≥ xμ. Put k = [N (t+s)−(x+sμ−1 )] and j = (x+sμ−1 )+{N (t+s)−(x+sμ−1 )}, where {·} is a fractional part of the number in brackets. Then N (t + s) = k+j, [x] ≤ j ≤ [x+aT μ−1 ]+1 and 0 ≤ k ≤ N (T )−j. Hence, the inequality Sk+j − Sk ≥ xμ

page 139

September 13, 2019

140

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

holds. Relation (5.5) follows. If UT < x − 2, then N (t + aT ) − N (t) < [x + aT μ−1 ] − 1 for all t ∈ [0, T − aT ]. Since Rn is non-decreasing, we have RN (t+aT )−[x+aT μ−1 ]+1 < RN (t) ≤ t < RN (t+aT )+1 − aT by the definition of N (t). It follows from the last inequality that SN (t+aT )+1 − SN (t+aT )−[x+aT μ−1 ]+1 < μx for all t ∈ [0, T − aT ]. Putting k = N (t + aT ) − [x + aT μ−1 ] + 1 implies that for all k with 0 ≤ k ≤ N (T ) − [x + aT μ−1 ] + 1, the inequality Sk+[x+aT μ−1 ] − Sk < xμ holds. This yields relations (5.6) and (5.7). Relation (5.8) may be proved in the same way as (5.6). We omit details.

We turn to the proof of Theorem 5.1. Proof. Take ε > 0 and δ > 0. Put K = [(1 + δ)T μ−1 ] and J = [(1 + 3ε)bT + aT μ−1 ]. Since bT → ∞, an application of (5.5) with x = (1 + 3ε)bT yields P (WT ≥ (1 + 4ε)bT ) ≤ P (WT ≥ (1 + 3ε)(bT + 1))   ≤P max max (Sk+j − Sk ) ≥ (1 + 3ε)μbT 1≤j≤J+1 0≤k≤N (T )−j   ≤P max max (Sk+j − Sk ) ≥ (1 + 3ε)μbT + P (N (T ) ≥ K) 1≤j≤J+1 0≤k≤K

= PT + QT for all sufficiently large T . We first check that every ε ∈ (0, ε0 ) there exists τ > 0 such that PT ≤ (log T )−(1+τ ) for all sufficiently large T , where ε0 is an absolute constant. Put T dT = log + log log T. cT

(5.9)

(5.10)

Suppose that εcT ≤ log T . The concavity of γ(x) and (5.3) imply that   dT (1 + 3ε)μbT ≥ jγ (1 + 3ε) j

page 140

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

141

Strong Limit Theorems for Renewal Processes

for 1 ≤ j ≤ J1 = [(1 + 3ε)cT ]. Taking into account Lemma 2.2 and the definition of γ(x), we have    J1 J+1   dT PT ≤ K P (Sj ≥ (1 + 3ε)μbT ) ≤ K P Sj ≥ jγ (1 + 3ε) j j=1 j=1 ≤K

J1 

e−(1+3ε)dT ≤ KJ1 T −(1+3ε) c1−3ε (log T )−(1+3ε) T

j=1

for all sufficiently large T . This implies (5.9). Suppose now that εcT > log T . Put m = [εJ], L = [K/m] + 1 and J2 = J + m + 1. Note that J2 ∼ (1 + ε)(cT + 3εbT ). By (5.3), we get   dT ≤ μ−1 cT γ(2ε) bT = μ−1 cT γ cT for all sufficiently large T . Hence, for all ε < ε0 , the inequalities (1 + ε)cT ≥ (1 + ε){(1 + 0.1ε)(1 + ε)(1 + 3εμ−1 γ(2ε))}−1 J2 ≥ J2 hold for all sufficiently large T . By Lemma 1.1, condition (5.4) and (5.3), we have   L  PT ≤ P max max (Sk+j − Sk ) ≥ (1 + 3ε)μbT 1≤j≤J+1 (l−1)m≤k≤lm

l=1

 ≤ LP ≤



max (Sk+j − Sk ) ≥ (1 + 3ε)μbT    dT K +m ≥ (1 + ε)μbT ) ≤ P S J 2 ≥ J2 γ 2 mq cT

max

1≤j≤J+1 0≤k≤m

L P (SJ2 q2

for all sufficiently large T . This bound, Lemma 2.2 and the definition of γ(x) yield (5.9). We now examine the rate of decreasing for QT . We have Kμ

Kμδ

= P SK ≥ ≤ e−Kζ(δμ/(1+δ) = e−ρT (1+o(1)) . QT = P RK ≤ 1+δ 1+δ Take θ > 1. Put Tk = θk for all k. By (5.9), we get ∞ 

P (WTk ≥ (1 + 4ε)bTk ) ≤

k=1

∞ 

PTk +

k=1

∞ 

QTk < ∞.

k=1

The Borel–Cantelli lemma implies that lim sup k→∞

WTk ≤ 1 + 4ε bTk

a.s.

(5.11)

page 141

September 13, 2019

142

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Further, bTk ≤ bTk+1 ≤ bθ3 Tk ≤ (1 + τ )bTk for all τ > 0, θ closed to 1 and large k. Since WT is non-decreasing, we have WTk+1 WT ≤ (1 + τ ) bT bTk+1 for Tk ≤ T ≤ Tk+1 . The last inequalities and (5.11) yield lim sup

WT ≤ (1 + τ )(1 + 4ε) a.s. bT

Passing to the limit as τ → 0 and ε → 0, we arrive at the result for Wn . Taking into account that

  aT {uT ≥ (1 + ε)cT } ⊂ UT ≥ (1 + ε) cT − μ 

 aT ⊂ WT ≥ (1 + ε) cT − , μ we obtain the result for uT . Now we turn to lower bounds. Theorem 5.2. Assume that log T /cT < min{1/c0 − ε0 , 1/ε0 } for all sufficiently large T and some ε0 > 0. If aT /T → 1, then suppose additionally that condition (5.4) is satisfied. Then lim sup

UT ≥1 bT

a.s.

One can replace UT by uT in the last relation. Proof. Take ε > 0 and δ > 0. Put K = K(T ) = [(1 − δ)T μ−1 ], J = J(T ) = [(1 − ε)bT + aT μ−1 ]. Assume first that aT /T < 1 − ρ for all sufficiently large T . By (5.6), we have {UT ≥ (1−2ε)bT} ⊃ {UT ≥ (1−ε)bT −2}

 (Sk+J −Sk ) ≥ (1−ε)μbT ⊃ max 0≤k≤N (T )−J+1

 ⊃ max (Sk+J − Sk ) ≥ (1 − ε)μbT ), N (T ) ≥ K ⊃ AT BT , 0≤k≤K−J+1

(5.12)

for all sufficiently large T , where AT = {SK+1 − SK−J+1 ≥ (1 − ε)μbT }

and BT = {N (T ) ≥ K}.

page 142

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Strong Limit Theorems for Renewal Processes

143

Let {Tk } be a sequence of positive numbers such that Tk ∞ as k → ∞. By the SLLN for sums and

 Kδμ , BT = {RK ≤ T } = SK ≥ − 1−δ w.p. 1 events BTk occur for all sufficiently large k. Hence, we only need to check that P (ATk i.o.) = 1. Define dT by formula (5.10). Making use of (5.3), we have μbT = cT γ(dT /cT ) ≤ aμcT for all sufficiently large T , where a < 1. Hence, J = [cT − εbT ] ≥ [(1 − εa)cT ] and (1 − ε)cT ≤ (J + 1)(1 − ε)(1 − εa)−1 ≤ J(1 − ε1 )2 for all sufficiently large T , where ε1 > 0. Using (2.70) and the concavity of γ(x), we get    dT P (AT ) ≥ P (SJ ≥ (1 − ε)μbT ) ≥ P SJ ≥ (1 − ε1 )2 Jγ cT    dT ≥ P SJ ≥ (1 − ε1 )Jγ (1 − ε1 ) (5.13) ≥ e−(1−ε2 )dT cT for all sufficiently large T . Put T1 = 1, Tk+1 = min{T : T > Tk , K(Tk+1 ) − J(Tk+1 ) = K(Tk )} for k ∈ N. In the same way as in the proof of Theorem 1.2, we prove that  the series P (A(Tk )) diverges. Since the events {A(Tk )} are independent, k

the Borel–Cantelli lemma implies P (A(Tk ) i.o.) = 1. Assume now that aT /T → 1. Then dT /cT → 0 and (5.3) yields that bT = o(cT ). It follows that cT ∼ aT /μ ∼ T /μ. By (5.7), we have {UT ≥ (1 − 2ε)bT } ⊃ {SJ ≥ (1 − ε)μbT } for all sufficiently large T . Take θ > 1. Put Tk = θk and Jk = J(Tk ) for k ∈ N. Denote Ck = {SJk − SJk−1 ≥ (1 − 0.5ε)μbTk },

Dk = {SJk−1 ≥ −0.5εμbTk }.

The relation Jk−1 ∼ cTk−1 as k → ∞ and condition (5.4) yield that P (Dk ) ≥ q for all sufficiently large k.

page 143

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

144

9789811212826

Universal theory for strong limit theorems of probability

The following pairs of events are independent: Ck and Dk , Ck and Dk Ck−1 Dk−1 , Ck and Dk Ck−1 Dk−1 Ck−2 Dk−2 ,. . . Since cTk ∼ (Jk − Jk−1 )θ/(θ − 1) as k → ∞, we have    θ dTk (Jk − Jk−1 )γ P (Ck ) ≥ P SJk −Jk−1 > (1 + τ )(1 − 0.5ε) θ−1 cTk for all sufficiently large k. Choose large θ and small τ such that (1 + τ )(1 − 0.5ε)θ/(θ − 1) = (1 − ρ)2 , where ρ > 0. Using the concavity and the definition of γ(x) and relation (2.70) in the same way as in the proof of (5.9), we obtain

 dTk P (Ck ) ≥ exp −(1 − ρ)(1 + η)(Jk − Jk−1 ) ≥ k −(1−ρ)(1+η)(1+ξ) cTk for all sufficiently large k. Choosing of small enough η and ξ implies the  P (Ck ). divergence of the series k

By Lemma 1.3, we have P (Ck Dk i.o.) ≥ q > 0. It follows that P (SJk ≥ (1 − ε)μbTk a.s. ) > 0. By Kolmogorov’s 0 or 1 law, the last probability equals to 1. This yields that P (UTk ≥ (1 − 2ε)bTk a.s.) = 1. Taking into account that {uT ≥ (1 − ε)cT } ⊃

  aT UT ≥ (1 − ε) cT − , μ

we get the result for uT . Turn to lower bounds for sufficiently slowly increasing functions cT . Theorem 5.3. If the conditions of Theorem 5.2 hold and log log T = o(log(T /cT )), then lim inf

UT ≥1 bT

a.s.

One can replace UT by uT in the last relation.

page 144

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Strong Limit Theorems for Renewal Processes

9789811212826

145

Proof. Under the notations of the proof of Theorem 5.2, write (5.12) in {UT < (1 − 2ε)bT } ⊂ ET ∪ FT for all sufficiently large T , where ET = AT and FT = BT . By the SLLN for sums and

 Kδμ FT = {RK > T } = SK < − , 1−δ w.p. 1 the events FT do not occur for all sufficiently large T . Hence, it is sufficient to prove the same for the events ET . We have ⎞ ⎛ [K/J]  P (ET ) ≤ P ⎝ (S(m+1)J − SmJ ) < (1 − ε)μbT ⎠ m=0

≤ (1 − P (SJ ≥ (1 − ε)μbT ))[K/J] ≤ e−[K/J]P (SJ ≥(1−ε)μbT ) . (5.14) Applying (5.13), the definitions of K and J and the condition log log T = o(log T /cT ), we get P (ET ) ≤ T −3

(5.15)

for all sufficiently large T . Put Tn = μn/(1 − δ) for n ∈ N. For every n with J(Tn ) < J(Tn+1 ), put Tn (j) = inf{T ∈ [Tn , Tn+1 ) : J(T ) = j},

j ∈ In = (J(Tn ), J(Tn+1 )].

For n with J(Tn ) = J(Tn+1 ), put Tn (j) = Tn+1 ,

j ∈ In = {J(Tn+1 )}.

By the definition of Tn (j), we have ET ⊂ ETn (j+1) for T ∈ (Tn (j), Tn (j+1)]. Further, ∞   n=N j∈In ∞ 

P (ETn (j+1) ) ≤

∞  

(Tn (j + 1))−3

n=N j∈In

(J(Tn+1 ) − J(Tn ) + 1)Tn−3 .

≤

n=N

Here N is chosen large enough to satisfy (5.15). In the same way as in the proof of Theorem 5.2, we get cT ≥ J(T ) ≥ [(1 − aε)cT ]. Hence, J(Tn+1 ) − J(Tn ) ≤ cTn+1 − (1 − aε)cTn + 1.

page 145

September 13, 2019

146

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Since cTn /cTn+1 ≤ Tn+1 /Tn → 1 and cTn = O(Tn ) as n → ∞, we have J(Tn+1 ) − J(Tn ) = O(Tn ) ∞   as n → ∞. Then the series P (ETn (j+1) ) converges. By the Borel– n=N j∈In

Cantelli lemma, w.p. 1 a finite number of events ETn (j+1) occurs. It follows that w.p. 1 the events ET do not occur for all sufficiently large T . Making use of

  aT {uT ≥ (1 − ε)cT } ⊃ UT ≥ (1 − ε) cT − , μ we obtain the result for uT . Theorems 5.1–5.3 and the inequality Un ≤ Wn imply the next result. Theorem 5.4. If the conditions of Theorems 5.1 and 5.2 hold, then UT uT WT = lim sup = lim sup = 1 a.s. (5.16) lim sup bT bT bT If the conditions of Theorems 5.1 and 5.3 hold, then WT UT uT lim = lim = lim = 1 a.s. (5.17) bT bT bT 5.2

Corollaries of the Universal Strong Laws

We state a numbers of important corollaries of Theorem 5.4 in this section. We start with the case aT = O(log T ). It is not difficult to check that relation (5.1) is equivalent to   log(T /cT ) + log log T cT = a T β , (5.18) aT where β(x) is the inverse function to κ(u) = uζ(μ − u−1 ), u ≥ μ−1 . It follows from the properties of ζ(x) that β(0) = 1/μ, β(x) ∞ as x → ∞ and β(x) is concave. The first result is as follows. Theorem 5.5. If aT = c log T , c > c0 , then WT UT uT lim = lim = lim =1 (β(1/c) − 1/μ)aT (β(1/c) − 1/μ)aT β(1/c)aT

a.s.

Theorem 5.5 is the Erd˝ os–R´enyi law for renewal processes. Remember that the norming functions are calculated from different parts of formula (5.2) for UT and uT . Simple calculations show that the relations with UT and uT in Theorem 5.5 are equivalent.

page 146

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Strong Limit Theorems for Renewal Processes

9789811212826

147

Proof. Consider UT . Define dT by formula (5.10). By the conditions of theorem, γ(dT /cT ) is separated from zero and μ for all sufficiently large T . Hence, aT = O(log T ). cT = μ − γ(dT /cT ) Making use of (5.18) and the concavity of β(x), we get     1 log T cT = aT β (1 + o(1)) ∼ aT β . aT c It follows that bT ∼ aT (β(1/c) − 1/μ). The case of uT follows from the above provided cT = bT . For aT = o(log T ), we have the next result. Theorem 5.6. Assume that aT / log T  0 and one of the following conditions holds: 1) P (X = μ) > 0; 2) P (X = μ) = 0 and μ − γ(− log P (X > μ − x)) ∼ x as x  0. Then relation (5.17) holds with bT = aT β(log T /aT ). Our formula for norming sequences works for aT = o(log T ) as well. To check this, we only need to prove that log T

. (5.19) b T ∼ aT β aT Define dT by formula (5.10). Check that γ(dT /cT ) → μ. Assume that there exists a sequence {Tk }, Tk ∞ as k → ∞, such that μ−γ(dTk /cTk ) > ρ > 0 for some ρ. Then dTk /cTk < a < ∞ for some a. Hence, log Tk /cTk < 2a for all sufficiently large follows that  k. It  ρ dTk cTk > aTk = μ − γ log Tk cTk 2a for all sufficiently large k, which contradicts to aT = o(log T ). Relation (5.1) implies that aT = o(cT ) and cT ∼ bT . In particular, the latter means that in Theorem 5.6, the norming functions for UT and uT coincide since the centering aT /μ is negligible. If P (X = 0) = 0, then γ(dT /cT ) → μ implies that dT /cT → ∞. Hence, cT = o(log T ) and bT = o(log T ). By (5.18) and concavity of β(x), we have     log T log T cT = aT β (1 + o(1)) ∼ aT β . aT aT This gives (5.19). If P (X = 0) > 0, then γ(dT /cT ) → μ yields dT /cT → 1/c0 . Then cT ∼ c0 log T and bT ∼ c0 log T . But the left-hand side of (5.19) is equivalent to c0 log T as well.

page 147

September 13, 2019

15:48

148

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Proof. Suppose first that condition 1) holds. Put bT = c0 log T . Using the notations from the proof of Theorem 5.3, we get inequality (5.14) and J ∼ (1 − ε)bT . Taking into account that c0 = − log P (X = μ), we have P (SJ ≥ (1 − ε)μbT ) ≥ (P (X = μ))J = e−J/c0 ≥ T −1+ε/2 for all sufficiently large T . It follows that P (ET ) ≤ e−T

ε/4

for all sufficiently large T . The remainder coincides with that of the proof of Theorem 5.3. Suppose now that condition 2) holds. Define bT by bT = aT β(log T /aT ). Take ε, δ ∈ (0, 1). Put K = K(T ) = [(1 − δ)T /μ], J = J(T ) = [(1 − ε)bT ] and Tn = μn/(1 − δ) for n ∈ N. By (5.12), we have P (uT < (1 − 2ε)bT ) ≤ P (uT < (1 − ε)bT − 2)   (Rk+J − Rk ) ≥ aT ≤P min 0≤k≤N (T )−J   ≤P min (Rk+J − Rk ) ≥ aTn−1 0≤k≤N (T )−J

for Tn−1 ≤ T ≤ Tn . Put Jn = J(Tn ) and   pn = P min (Rk+Jn − Rk ) ≥ aTn−1 0≤k≤n−Jn

for n ∈ N. Note that K(Tn ) = n for all n. In the same way as in the proof of Theorem 5.3, the SSLN for sums yields that the result follows  pn < ∞. Check the latter. from n

Since aTn /aTn−1 ≤ log Tn / log Tn−1 → 1 as n → ∞, we have   3/2 min (Rk+Jn − Rk ) ≥ (1 − ε) aTn pn ≤ P 0≤k≤n−Jn

≤ (P (RJ ≥ (1 − ε)3/2 aTn ))[n/Jn ] ≤ e−[n/Jn ]P (RJn ≥(1−ε)

3/2

aT n )

for all sufficiently large n. Further, P (RJn ≥ (1 − ε)3/2 aTn ) ≥ ≥ e−Jn log P (X>μ−(1−ε)

3/2

  Jn aT P Y ≥ (1 − ε)3/2 n Jn

aTn /Jn )

for all sufficiently large n. This yields

 n

≥ Tn−1+ε pn < ∞.

page 148

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Strong Limit Theorems for Renewal Processes

9789811212826

149

Turn to the case aT / log T → ∞. Then the behaviour of uT is quite different from that of WT and UT . For example, if EX 2 < ∞ and aT = T , then the result for UT turns to the LIL for renewal processes while that for uT turns to the SLLN. Therefore, we deal with WT and UT separately from uT . We start with the case EX 2 < ∞. Our first result is as follows. Theorem 5.7. If σ 2 = EX 2 < ∞, then UT WT = lim sup = 1 a.s., lim sup bT bT where &   σ T bT = 3/2 2aT log + log log T . aT μ If additionally log log T = o(log(T /aT )), then WT UT lim = lim = 1 a.s. bT bT

(5.20)

(5.21)

Note that for Y with the exponential distribution, we have σ 2 = DY = μ . Hence, for the Poisson process, the constant in the above formula for bT is μ−1/2 . Theorem 5.7 is the Cs¨ org˝o–R´ev´esz laws for renewal processes. It yields the LIL for increments of renewal processes. 2

Corollary 5.1. If the conditions of Theorem 5.7 hold and log(T /aT )/ log log T → c ≥ 0, then √ σ UT WT a.s. = lim sup √ = 1 + c 3/2 lim sup √ μ 2aT log log T 2aT log log T For aT = T , the last relation is the LIL for N (T ). The condition EX 2 < ∞ is necessary in this case. Note that Theorem 5.7 may be derived by an application of the strong approximation of renewal processes by the Wiener process and the results for increments of the Wiener processes. Nevertheless, this method can not be used for small increments and for the case EX 2 = ∞. Assume now that EX 2 = ∞. For a domain of attraction of the normal law, the result is as follows. Theorem 5.8. If F ∈ D(2), then (5.20) holds with    log(T /aT ) + log log T ˆ fˆ−1 , b T = aT m aT

(5.22)

page 149

September 13, 2019

150

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

where

  1 m(h) ˆ = hG , h

h2 G fˆ(h) = 2

  1 , h

0 G(x) =

u2 dF (u), x > 0.

−x

(G(x) ∈ SV∞ .) If additionally log log T = o(log(T /aT )), then (5.21) holds with bT from (5.22). If log(T /aT )/ log log T → c ≥ 0 in Theorem 5.8, then we have an analogue of Corollary 5.1 with an additional slowly varying multiplier in the norming function. For aT = T , the LIL follows as well. We now deal with X from domains of attraction of completely asymmetric stable laws with α ∈ (1, 2). For domains of normal attraction, we have the following result. Theorem 5.9. If F ∈ DN (α), α ∈ (1, 2), then relation (5.20) holds with λ  T −1−1/α −λ 1/α λ aT + log log T , (5.23) log bT = μ aT where λ = (α − 1)/α. If additionally log log T = o(log(T /aT )), then (5.21) holds with bT from (5.23). Turn to the LIL for increments of renewal processes in this case. We have the following result. Corollary 5.2. If the conditions of Theorem 5.9 hold log(T /aT )/ log log T → c ≥ 0, then WT lim sup = 1/α −λ λ aT (log log T )λ UT = (1 + c)λ μ1+1/α a.s. lim sup 1/α −λ λ aT (log log T )λ

and

For aT = T , Corollary 5.2 yields the LIL for renewal processes in this case. Turning to domains of non-normal attraction, we get the next result. Theorem 5.10. If F ∈ D(α), α ∈ (1, 2), then relation (5.20) holds with    log(T /aT ) + log log T −1 ˆ ˆ f , (5.24) b T = aT m aT where     1 1 αΓ(2 − α) α−1 α ˆ h G m(h) ˆ = , f (h) = Γ(2 − α)h G , α−1 h h G(x) = xα F (−x), x > 0. (G(x)SV∞ .) If additionally log log T = o(log(T /aT )), then (5.21) holds with bT from (5.24).

page 150

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Strong Limit Theorems for Renewal Processes

9789811212826

151

Proof. Since aT / log T → ∞, we have dT /cT → 0. By (5.1), we get cT ∼ aT /μ and bT = o(aT ). Theorems 5.7–5.10 now follow from (5.3) and the results on asymptotics of γ(x) at zero from Chapter 2. If log(T /aT )/ log log T → c ≥ 0 in Theorem 5.10, then we obtain an analogue of Corollary 5.2. For this case, a slowly varying multiplier appears in the norming function. For aT = T , this yields the LIL. We end this section with a result for uT that includes the SLLN for renewal processes. Theorem 5.11. If aT / log T → ∞, then lim sup

1 uT = aT μ

a.s.

If log log T = o(log(T /aT )) or aT = T , then one can replace lim sup by lim in the last relation. Proof. The condition aT / log T → ∞ yields that dT /cT → 0. By (5.1), we have bT = cT ∼ aT /μ. By Theorem 5.2, the result follows besides the case aT = T . For aT = T , we have uT = N (T ). Then we get the result from the duality {N (T ) < k} = {Rk > T }, k ∈ N, and the SLLN for sums Rn . 5.3

Bibliographical Notes

Limit theorems for renewal processes may be found in [Feller (1971)] and [Gut (2009)]. Limit theorems for increments of renewal processes are derived in [Steinebach (1982, 1986, 1991)], [Deheuvels and Steinebach (1989)], [Bacro et al. (1987)], [Frolov (2003f,g)]. The universal strong laws for Compound renewal processes are obtained by [Frolov (2007)].

page 151

b2530   International Strategic Relations and China’s National Security: World at the Crossroads

This page intentionally left blank

b2530_FM.indd 6

01-Sep-16 11:03:06 AM

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Chapter 6

Increments of Sums of Independent Random Variables over Head Runs and Monotone Blocks Abstract. We derive universal strong laws for increments of sums of independent random variables over head runs and monotone blocks. They yield the SLLN, the LIL, the Erd˝ os–R´enyi law and the Cs¨ org˝ o–R´ev´esz laws. 6.1

Head Runs and Monotone Blocks

We start with notions of head runs and monotone blocks. Let {Yn } be a sequence of independent Bernoulli random variables. A series of ones is called a head run. A.s. asymptotic behaviour of the longest head run in Y1 , Y2 , . . . , Yn is well known. In the sequel, we permanently use this asymptotic and, therefore, we need the following result. Lemma 6.1. Put p = P (Y1 = 1) and Lhr n = max {k : Yi+1 = · · · = Yi+k = 1 for some i, 0 ≤ i ≤ n − k} . If p ∈ (0, 1), then lim

Lhr n =1 log n/log(1/p)

a.s.

In what follows, we assume that n → ∞ in ∼, →, lim sup, lim inf, o, O provided it is not pointed otherwise. Proof. Put ln = log n/ log(1/p). For ε > 0, we have Pn = P (Lhr n ≥ (1 + ε)ln ) ≤ nP (Y1 = · · · = Y[(1+ε)ln ] = 1) ≤ np

[(1+ε)ln ]

≤n

(6.1)

−ε/2

for all sufficiently large n. In the last inequality, we have used the definition  of ln . Put nk = 2k for all k ∈ N. Then the series k Pnk converges. By 153

page 153

September 13, 2019

154

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

the Borel–Cantelly lemma, we have lim sup k→∞

Lhr nk ≤1+ε lnk

a.s.

For nk ≤ n ≤ nk+1 , we have Lhr Lhr Lhr nk+1 lnk+1 nk+1 n ≤ ≤ (1 + ε) ln lnk+1 lnk lnk+1 for all sufficiently large k. Hence, lim sup

Lhr n ≤ (1 + ε)2 ln

a.s.

Passing to the limit as ε → 0 in this relation, we get the upper bound. Turn to the lower bound. For ε > 0, we have  [n/ln ]−1 (6.2) Qn = P (Lhr n ≤ (1 − ε)ln ) ≤ 1 − P (Y1 = · · · = Y[(1−ε)ln ] = 1)  +

   n − 1 p[(1−ε)ln ] ≤ exp −nε/2 ≤ exp − ln for all sufficiently large n. In the last inequality, we have used the definition  of ln . It follows that the series n Qn converges. By the Borel–Cantelli lemma, we get lim inf

Lhr n ≥1−ε ln

a.s.

Passing to the limit as ε → 0 in this relation, we get the lower bound. Turn to monotone blocks. Let {Yn } be a sequence of independent, continuous random variables. A series Yi+1 < Yi+2 < · · · < Yi+k with 0 ≤ i ≤ i + k ≤ n is called a monotone block (or an increasing run) of length k in Y1 , Y2 , . . . , Yn . The asymptotic of the length of the longest monotone block in Y1 , Y2 , . . . , Yn will also be used in the sequel. Hence, we need the next result. Lemma 6.2. Put Lmb n = max {k : Yi+1 < · · · < Yi+k for some i, 0 ≤ i ≤ n − k} . Then lim

Lmb n =1 log n/ log log n

a.s.

page 154

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Increments of Sums over Head Runs and Monotone Blocks

155

Proof. Note that P (Y1 < Y2 < · · · < Yk ) = 1/k! for all k. Put ln = log n/ log log n. For ε > 0, we have the following analogue of (6.1): P (Lmb n ≥ (1 + ε)ln ) ≤ nP (Y1 < · · · < Y[(1+ε)ln ] ). This yields P (Lmb n ≥ (1 + ε)ln ) ≤

n ≤ n−ε/2 [(1 + ε)ln ]!

(6.3)

for all sufficiently large n. The last inequality follows from the Stirling formula √ n! ∼ 2πn nn e−n and the definition of ln . We get instead of (6.2) that  [n/ln ]−1 P (Lmb . n ≤ (1 − ε)ln ) ≤ 1 − P (Y1 < · · · < Y[(1−ε)ln ] ) It implies that

 +    n 1 ε/2 P (Lmb ≤ (1−ε)l ) ≤ exp − − 1 (6.4) ≤ exp −n n n ln [(1 − ε)ln ]!

for all sufficiently large n. We have used the definition of ln and the Stirling formula in the last inequality. Inequalities (6.3) and (6.4) imply the result in the same way as in the proof of Lemma 6.1. In literature, one can find results for another interesting cases as well. Rejecting the assumption of continuity for Y , we arrive at a quite different problem with the lengths of the longest monotone blocks. Moreover, we can also investigated non-strictly monotone blocks which are defined with a replacement of signs < by ≤. Further generalizations are blocks for ddimensional Y ’s or random fields. (See, [Frolov and Martikainen (1999, 2001)] and references therein). Below we mention another generalizations of monotone blocks as well. 6.2

Increments of Sums over Head Runs and Monotone Blocks

We are going to investigate a.s. limit behaviour of increments of sums of i.i.d. random variables over head runs and monotone blocks in an accompanying sequence of i.i.d. random variables. To this end, we deal with the settings as follows.

page 155

September 13, 2019

156

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Let (X, Y ), (X1 , Y1 ), (X2 , Y2 ), . . . be a sequence of i.i.d. random vectors. Note that X and Y can be dependent and the case X = Y can be considered as well. Put Sn = X 1 + X 2 + · · · + X n ,

S0 = 0.

We will investigate the a.s. asymptotic behaviour for maxima Mn =

max

0≤k≤n−jn

(Sk+[jn ] − Sk )I{u≤Yk+1 ≤···≤Yk+[j

n ] ≤v

},

where u, v are fixed, −∞ ≤ u ≤ v ≤ +∞ and {jn } is such that 1 ≤ jn ≤ n for all n. Here IB is the indicator of the event B and [x] is the integer part of the number x. Note that jn may be a random variable. Our notation for the lengths of increments is different from that in the previous chapters. This is to emphasize that we will never have jn so large as an . Formally, we assume that jn ≤ n, but jn will always have a logarithmic or smaller order. Indeed, all indicators in Mn are zeros when jn is greater than the length of the longest block of Y ’s. By Lemmas 6.1 and 6.2, these lengths have a logarithmic order of growth w.p. 1 in the cases we concern with. Our aim is to find a sequence of positive numbers {bn } such that either lim sup n→∞

Mn =1 bn

a.s.,

or the latter holds with lim sup replaced by lim when it is possible. The maximum Mn has an interesting interpretation in game settings. Assume that Xn is a gain (the negative gain is a loss) of a player in n-th repetition of a game of chance and Yn is a characteristic of a success in n-th repetition. For example, if Yn = I{Xn >0} , then n-th repetition is successful provided the gain is positive. Note that sometimes we can only define a success for a successive series of repetitions. Indeed, try to tell that n-th repetition is successful if the gain in this repetition is greater than the gain in the previous one. Then Yn = I{Xn >Xn−1 } and the vectors (Xn , Yn ) are not independent. This difficulty can be eliminated as follows. Put Yn = Xn and say that a series of repetitions of the game is successful when the gain in every next repetition is greater than that in the previous one. In these settings, Mn is a maximal gain in successful series of lengths jn . Investigations of asymptotics for Mn is of interest in probability and actuarial and financial mathematics. In the last case, it allows to estimate possible losses on time subintervals (see, for example [Binswanger and Embrechts (1994)]). Moreover, the procedure of rarefying naturally appears

page 156

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Increments of Sums over Head Runs and Monotone Blocks

9789811212826

157

in there (see [Embrechts and Cl¨ uppelberg (1993)]). For example, insurance companies pay compensations after investigations and some claims are rejected. In our settings, we put Yn = 1, when n-th claim is accepted and Yn = 0 otherwise. Maxima Mn for u = v = 1 is a maximal aggregate amount of compensations in time jn over series of accepted claims. The above settings are very general. Interesting partial appear under various additional assumptions on distributions of the vector (X, Y ), the sequence {jn }, u and v. Assume first that u = v = 1, P (Y = 1) = 1 − P (Y = 0) = p ∈ (0, 1]. If X is a non-degenerate random variables and p = 1, then Mn coincides with Un which has been investigated in Chapter 3. If X = 1 a.s., p ∈ (0, 1) and jn is the length of the longest head run in Y1 , Y2 , . . . , Yn . Then Mn = jn = Lhr n which behaviour is described by Lemma 6.1. If X is a non-degenerate random variables and p < 1, then Mn is the maximal increments of sums Sn over head runs. This case will be considered below. Assume further that u = −∞, v = +∞ and Y has a continuous distribution. If X = 1 a.s. and jn is the length of the longest monotone block in Y1 , Y2 , . . . , Yn , then Mn = jn = Lmb n which the asymptotic is found in Lemma 6.2 for. If X is a non-degenerate random variables, then Mn is the maximal increments of sums Sn over monotone blocks and its limiting behaviour will be described below. Assume finally that −∞ < u < v < +∞ and Y has a continuous distribution. Then Mn is the maximal increments of sums Sn over monotone blocks in the interval (u, v). This case will be discussed below as well. In the next section, we derive the universal laws for Mn when X is a non-degenerate random variable and Y is either continuous, or P (Y = 1) = 1 − P (Y = 0) = p ∈ (0, 1). Further, we concern with various corollaries of the universal laws. 6.3

The Universal Strong Laws

Dealing with Mn , we only consider sequences {jn } with jn ≤ Ln a.s., where Ln is the length of the longest sequence of Y ’s taking its value in [u, v], i.e. Ln = max {k : u ≤ Yi+1 ≤ · · · ≤ Yi+k ≤ v for some i, 0 ≤ i ≤ n − k} . Consider two following cases. A) Assume that u = v = 1 and P (Y = 1) = p = 1 − P (Y = 0) ∈ (0, 1). Then Ln = Lhr n is the length of the longest head run in Y1 , Y2 , . . . , Yn and Ln ∼ log n/ log(1/p) a.s. by Lemma 6.1.

page 157

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

158

9789811212826

Universal theory for strong limit theorems of probability

B) Suppose that −∞ ≤ u < v ≤ +∞ and P (Y = y) = 0 for all y. is the length of the longest If u = −∞ and v = +∞, then Ln = Lmb n monotone block in Y1 , Y2 , . . . , Yn and Ln ∼ log n/ log log n a.s. by Lemma 6.2. For finite u and v, the asymptotic of Ln is the same provided p = P (u < Y < v) > 0. This is clear from the proof of Lemma 6.2 and the equality P (u < Y1 < Y2 < · · · < Yk < v) = pk /k! for all k. One can check that the additional multiplier in the last formula is negligible in analogues of bounds (6.3) and (6.4) for this case. The behaviour of Ln has been investigated for other blocks and distribution of Y ’s as well (see [Frolov and Martikainen (1999)]). But large deviations asymptotics are known for head runs and monotone blocks (see Lemmas 6.3–6.5 below). Hence, we restrict our attention by cases A) and B). For n ∈ N, put ln =

log n log(1/p)

in case A) and let ln be a solution of the equation √ 2πl ll e−l = n in case B). Note that the left-hand part of this equation is the asymptotic of l! from Stirling’s formula. It is not difficult to check that in this case, ln ∼

log n . log log n

¯ be a random variable with the d.f. Put p = P (u ≤ Y ≤ v) > 0. Let X F (x) = P (X < x|u ≤ Y ≤ v). Note that for u = −∞ and v = +∞, the conditional d.f. F (x) coincides with the d.f. of X. ¯ is nonIn the sequel, we will assume that the random variable X ¯ degenerate, E X ≥ 0 and ¯

h0 = sup{h : ϕ(h) = EehX < ∞} > 0. ¯ For 0 < h < h0 , put Put ω = esssupX. m(h) =

ϕ (h) , ϕ(h)

σ 2 (h) = m (h),

f (h) = hm(h) − log ϕ(h).

Denote ζ(z) =

sup

{zh − log ϕ(h)},

h≥0,ϕ(h) 0, σ 2 (0) = DX ¯ = 0, ζ(z) , ζ(z) is convex, ζ(z) = +∞ for z > A if A < ∞, ζ(E X) ¯ γ(x) for x < 1 , γ(x) is concave, γ(0) = E X, c0 1 if c0 > 0 and ω < ∞, γ(x) → ω as x → ∞. γ(x) = ω for x > c0 Moreover, we have ¯ A), ζ(z) = f (m−1 (z)) for z ∈ [E X,   1 γ(x) = m(f −1 (x)) for x ∈ 0, , (6.5) c0 where m−1 (·) and f −1 (·) are inverse functions to m(·) and f (·) correspondingly. Let {δn } be a sequence of real numbers with δn ∈ (0, 1). In the sequel, we only consider sequences {jn } as follows: jn = δn ln . For n ∈ N, put in = [δn ln ], 1 in case A), τn = (ln − in ) log p     1 1 1 in case B). τn = ln + log ln − ln − in + log in + in − in log 2 2 p The formula for τn is rather complicated for the case B). To realise the asymptotic behaviour of  τn , we write  1 1 τn = (ln − in ) log ln + in + (log ln − log in ) + (in − ln ) − in log . 2 p For in → ∞, the main part of the asymptotic of τn is (1 − δn )ln log ln for {δn } separated from 1 and (1 − δn )ln log ln − δn ln log(1/p) for {δn } with δn → 1. Hence, we will assume that (1 − δn ) log ln > log(1/p). For the case of monotone blocks, the last equality is trivial by p = 1. Our first result is as follows. ¯ is non-degenerate, E X ¯ ≥ 0, h0 > 0, and Theorem 6.1. Assume that X {in } is a non-decreasing sequence with in ≥ 1 for all n. Then Mn ≤ 1 a.s., (6.6) lim sup bn where bn = in γ((τn + log ln )/in ).

page 159

September 13, 2019

160

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

To prove Theorem 6.1, we need some auxiliary results. We start with bounds for probabilities of large deviations. Two first Lemmas are variants of Lemmas 2.2 and 2.24 from Chapter 2 which we rewrite in term of random variables with the (conditional) d.f. F (x). ¯i} Lemma 6.3. Assume that the conditions of Theorem 6.1 hold. Let {X ¯ be a sequence of i.i.d. random variables with the same distribution as X. ¯ ¯ ¯ Put Sn = X1 + · · · + Xn . Then P (S¯n ≥ nm(h)) ≤ e−nf (h) , √ √ 3 P (S¯n ≥ nm(h) − 2 nσ(h)) ≥ e−nf (h)−2 nhσ(h) 4 for all n and h ∈ (0, h0 ). For lower bounds, we have the following result. Lemma 6.4. Let {hn } be a sequence of real numbers with hn ∈ (0, h0 ). Assume that the conditions of Lemma 6.3 and one of two following conditions hold: √ 1) nf (hn ) → ∞ and hn σ(hn ) = o( nf (hn )). ˜ < h0 . 2) hn ≤ h Then 3  P S¯n ≥ (1 − ε)nm(h) ≥ e−nf (h)(1+δ) 4 for all ε ∈ (0, 1), δ > 0 and all sufficiently large n. The next lemma describes relationships between bounds for large deviations of sums over monotone blocks and head runs and those for sums of i.i.d. random variables with the conditional d.f. F (x). Lemma 6.5. If u < v and P (Y = y) = 0 for all y, then for all x > 0, the following relations hold: pn   P S¯n ≥ x , (6.7) P Sn I{u≤Y1 ≤···≤Yn ≤v} ≥ x = n! jn  np P S¯jn ≥ x , (6.8) P (Mn ≥ x) ≤ jn !

 + jn   n p P S¯jn ≥ x . P (Mn < x) ≤ exp − (6.9) jn jn ! If u = v = 1 and P (Y = 1) = p > 0, then for all x > 0, relation (6.7) holds with a replacement of n! by 1 and relations (6.8) and (6.9) hold with a replacement of jn ! by 1.

page 160

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

161

Increments of Sums over Head Runs and Monotone Blocks

Proof. Assume that u = v = 1 and P (Y = 1) = p > 0. It is not difficult to check that ¯ Eg(X)I{Y =1} = pEg(X) (6.10) for every bounded Borel function g(x). For sake of brevity, we prove relation (6.7) for n = 3. The proof for another n follows the same pattern. Put Ij = I{Yj =1} for j = 1, 2, 3. By the Fubini theorem, we get  P = P X1 + X2 + X3 < x, I{Y1 =Y2 =Y3 =1} = 1 = EI{X1 +X2 +X3 1. For i ≥ 2 and j ≥ 0, define Nij and nij by (6.12) again. Take ε > 0. Applying Lemmas 6.3 and 6.5 and relation (6.5), we have npin  ¯ P Sin ≥ (1 + ε)bn in ! ≤ C exp {−(1 + ε + ερn ) log in }

R(n) = P (Mn ≥ (1 + ε)bn ) ≤

page 163

September 13, 2019

164

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

for all sufficiently large n, where C is a positive constant. The remainder of the proof repeats that for head runs. We omit details.

Theorem 6.1 gives an upper bound for the rate of the growth of Mn . Turn to the lower bounds. The first result is as follows. ¯ is non-degenerate, E X ¯ ≥ 0 and h0 > 0. Theorem 6.2. Assume that X Suppose that in ≥ 1, in is non-decreasing and one of two following conditions holds: 1) hn = f −1 (τn /in ) < h0 and √ hn σ(hn ) = o( in f (hn )). (6.13) 2) τn /in → ∞. Then lim sup

Mn ≥1 cn

a.s.,

(6.14)

where cn = in γ(τn /in ). The next remark follows from Lemma 2.25. ˜ < h0 for some h ˜ < ∞ and τn → ∞, then (6.13) Remark 6.1. If hn ≤ h holds. Proof. Assume first that condition A) holds, i.e. u = v = 1 and P (Y = 1) = p ∈ (0, 1). Suppose that condition 1) holds. Put Rn = (Sn − Sn−in )I{Yn−in +1 =···=Yn =1} . Take ε ∈ (0, 1/2). By Lemmas 6.4 and 6.5 and relation (6.5), we have  P (n) = P (Rn ≥ (1 − 2ε)cn ) = pin P S¯in ≥ (1 − 2ε)cn 1 ≥ pin exp{−(1 − ε)τn } ≥ n for all sufficiently large n. Put nk = ak log k for k ≥ 3, where a is chosen such that nk+1 − ink+1 >  nk . Then the series k P (nk ) diverges and the events {Rnk ≥ (1 − 2ε)cnk } are independent. Applying the Borel–Cantelli lemma and the inequality Mn ≥ Rn , we obtain (6.14).

page 164

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

165

Increments of Sums over Head Runs and Monotone Blocks

Assume that condition 2) holds. Put c(n) = τn /in . For all ε > 0, we have  Q(n) = P Mn < (1 + ε)−3 bn

 in npin   ¯ ≤ exp − P X ≥ (1 + ε)−3 γ(c(n)) . 2in By the method used in Lemma 2.5 from [Mason (1989)], we construct the sequence of natural numbers {nr } such that     ¯ ≥ (1 + ε)−3 γ(c(nr )) log nr /c(nr ) ≥ exp −(1 + ε)−1 log nr (6.15) P X and nr ≥ r for all sufficiently large r. Then we get Q(nr ) ≤ exp{−Cnδr } for some δ > 0 and all sufficiently large r. Hence, the series converges. Applying of the Borel–Cantelli lemma yields (6.14). The proof is completed in case A). Turn to the case B). Assume that condition 1) holds. Put

(6.16)

 r

Q(nr )

Rn = (Sn − Sn−in )I{u≤Yn−in +1 ≤···≤Yn ≤v} . Take ε ∈ (0, 1/2). By Lemmas 6.4 and 6.5 and relation (6.5), we have pin  ¯ P Sin ≥ (1 − 2ε)cn P (n) = P (Rn ≥ (1 − 2ε)cn ) = in ! pin 1 ≥ exp {−(1 − ε)τn } ≥ in ! n for all sufficiently large n. In the same way as for the case A), we again arrive at (6.14). Assume that condition 2) holds. Put c(n) = τn /in . Remember that c(n) is quite different in this case. Take ε > 0. By (6.9), we have  Q(n) = P Mn < (1 + ε)−3 bn 

 + in  n p ≤ exp − P S¯in ≥ (1 + ε)−3 in γ(c(n)) in in !

 + in   n p ¯ ≥ (1 + ε)−3 γ(c(n)) in . P X ≤ exp − in in ! By the same method as used in Lemma 2.5 from [Mason (1989)], we construct the sequence of natural numbers {nr } such that (6.15) holds and nr ≥ r for all sufficiently large r. An application of the Stirling formula, the definition of ln and (6.15) yields the similar bound for Q(nr ) as before.  It follows that the series r Q(nr ) converges and by the Borel–Cantelli lemma, we obtain (6.14) again.

page 165

September 13, 2019

15:48

166

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

We now investigate a minimal rate of the growth of Mn . ¯ is non-degenerate, E X ¯ ≥ 0, h0 > 0, in ≥ 1, Theorem 6.3. Assume that X in is non-decreasing and τn > 1. (6.17) lim inf log ln Assume that one of three following condition hold: 1) hn = f −1 (τn /in ) < h0 and (6.13) holds. 2) τn /in → ∞ and ω < ∞. 3) τn /in → ∞ and γ(− log(1 − F (z))) =1. z→∞ z

(6.18)

Mn ≥1 dn

(6.19)

lim

Then lim inf

a.s.,

where dn = in γ((τn − log ln )/in ). According to [Mason (1989)], condition (6.18) holds for the normal, geometric, Poisson and Weibull distributions. Proof. Start with the case A). Assume that condition 1) holds. Suppose first that ρn = τn / log ln → ∞. Take ε ∈ (0, 1/2). By Lemmas 6.4 and 6.5 and relation (6.5), we have

 npin  ¯ P Sin ≥ (1 − 2ε)dn Q(n) = P (Mn < (1 − 2ε)dn ) ≤ exp − 2in 

  in 3np n −1) exp{−(1 − ε)(τn − log ln )} ≤ exp −iε(ρ ≤ exp − (6.20) n 8in  for all sufficiently large n. It implies that the series n Q(n) converges and an application of the Borel–Cantelli lemma yields (6.19). Suppose now that ρn ≤ a. Take θ > 1. For i ≥ 2 and j ≥ −1, put Ni,−1 = {n : δ < ρn − 1 < 1, in = i} ,   Nij = n : θj ≤ ρn − 1 < θj+1 , in = i , mij = min {n : n ∈ Nij } .   Hence, the series j k Q(mjk ) converges and by the Borel–Cantelli lemma, we have lim inf i→∞

Mnij ≥ 1 − 2ε a.s. dnij

page 166

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Increments of Sums over Head Runs and Monotone Blocks

9789811212826

167

uniformly over j. Making use of the concavity of the function γ(x), we conclude that dn ≤ θ3 dnij for n ∈ Nij , all sufficiently large i and all j. From the last inequality and the inequality Mn ≥ Mnij for n ∈ Nij , it follows that lim inf in (6.19) is greater than (1 − 2ε)θ−3. This yields (6.19). Relation (6.19) for oscillating ρn can be proved in the same way as before in Theorem 6.1. Assume that condition 2) holds. Take 0 < δ < 1. We have again that

 in npin  ¯ P (X ≥ δ1 ω) Q(n) = P (Mn < δωin ) ≤ exp − . 2in  In the same way as before, we prove that the series n Q(n) converges. An application of the Borel–Cantelli lemma completes the proof of (6.19) in this case. Assume that condition 3) is satisfied. Put c(n) = τn /in . For all ε > 0, we have Q(n) = P (Mn < (1 + ε)−2 bn ) 

in npin  ¯ −2 . ≤ exp − P (X ≥ (1 + ε) γ(c(n))) 2in Making use of the inequality    ¯ ≥ (1 + ε)−2 γ(c(n)) ≥ exp −(1 + ε)−1 c(n) P X  (see [Mason (1989)], p. 264) we prove that the series n Q(n) converges. Together with the Borel–Cantelli lemma, this completes the proof of Theorem 6.2 in case A). Turn to the case B). Assume that condition 1) holds. Define Q(n) as in the proof for the case A) under condition 1). In the same way as before, we prove bound (6.20). The remainder part of the proof is the same as before. Suppose that condition 2) holds. Since ω < ∞, we can assume that bn = in ω. Define Q(n) as in the proof for the case A) under condition 2).  In the same way as before, one can check that the series n Q(n) converges. Applying of the Borel–Cantelli lemma completes the proof in this case. Assume that condition 3) holds. Define Q(n) as in the proof for the case A) under condition 3). In the same way as before, one can check that  the series n Q(n) converges. Making use of the Borel–Cantelli lemma, we arrive at the desired conclusion in this case. The proof of Theorem 6.3 is completed.

page 167

September 13, 2019

168

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Theorems 6.1–6.3 imply the following result which gives universal strong laws for increments of sums of i.i.d. random variables over head runs and monotone blocks. Theorem 6.4. If the conditions of Theorem 6.2 hold and τn = ∞, lim log ln

(6.21)

then lim sup

Mn =1 bn

a.s.

(6.22)

If the conditions of Theorem 6.3 and relation (6.21) hold, then lim

Mn =1 bn

a.s.

(6.23)

Note that lim inf τn /in > 0 implies (6.21) and (6.17). It is not difficult to check that bn ∼ cn ∼ dn provided (6.21) holds. 6.4

Corollaries of the Universal Strong Laws

In this section, we discuss various corollaries from the results of the previous section. First of all, note that the norming sequences in the universal laws can easily be calculated for the completely asymmetric stable laws and the normal law. Remember that for the stable d.f. F with c.f. (2.2), we have αx (α−1)/α γ(x) = α−1 √ for all x > 0. If α = 2, then F is the standard normal d.f. and γ(x) = 2x. It implies that the norming cn for the stable laws is as follows:  (α−1)/α ατn 1/α . cn = i n α−1 It yields that cn = (2in τn )1/2 , in the Gaussian case. Formulae for bn and dn follow from the last formulae by a replacement of τn by τn ± log ln correspondingly. We now state strong limit theorems for increments of sums over head runs and monotone blocks. It turns out in these cases that the type of asymptotic behaviour of Mn depends on asymptotics of ratios τn and in . If this ratio is separated from zero, then we get the Erd˝os–R´enyi laws. If this ratio tends to zero, then we arrive at the Cs¨ org˝ o–R´ev´esz laws.

page 168

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Increments of Sums over Head Runs and Monotone Blocks

9789811212826

169

We start with the Erd˝os–R´enyi law for a general situation. Further, we deal with the cases of head runs and monotone blocks separately. ¯ is non-degenerate, E X ¯ ≥ 0 and h0 > 0. Theorem 6.5. Assume that X Assume that in ≥ 1, in is non-decreasing and lim inf

τn > 0. in

(6.24)

Then (6.22) holds with bn = in γ((1 − δn ) log n/in ). If additionally ω < ∞ or (6.18) holds, then (6.23) holds. The Erd˝ os–R´enyi law for increments of sums over head runs is as follows. ¯ is non-degenerate, E X ¯ ≥ 0 and h0 > 0, Theorem 6.6. Assume that X Assume that in ≥ 1, in is non-decreasing and condition A) holds. If δn → B ∈ [0, 1), then (6.24) is satisfied and, consequently, the result of Theorem 6.5 holds. If in → ∞, then bn ∼ in γ((1 − δn ) log(1/p)/δn ) and bn ∼ Bln γ((1 − B) log(1/p)/B) for B > 0. The next theorem is the corresponding result for increments of sums over monotone blocks. ¯ is non-degenerate, E X ¯ ≥ 0 and h0 > 0, Theorem 6.7. Assume that X Assume that in ≥ 1, in is non-decreasing and condition B) holds. If (1 − δn ) log log n → ∞, then (6.24) is satisfied and, consequently, the result of Theorem 6.5 holds. If in → ∞, then bn ∼ in γ((1−δn ) log log n/δn ). If (1 − δn ) log log n → B + log(1/p) with B ∈ (0, 1/c0 ), then (6.24) holds and bn ∼ ln γ(B). We see that the results of Theorems 6.6 and 6.7 are quite different. We have the Erd˝ os–R´enyi law in the case of head runs when δn is separated from one. In the case of monotone blocks, δn can tend to one with an appropriate rate of convergence. Turn to the Cs¨org˝ o–R´ev´esz laws. This case arises when the arguments of the function γ(x) in the definitions of bn , cn and dn tend to zero. Under ¯ the asymptotics various additional conditions on the distribution of X, of γ(x) at zero is known from above considerations. This yields simpler formulae for norming sequences. We separately consider the case of finite variations, the case of non-normal attraction to the normal law and the cases of normal and non-normal attraction to completely asymmetric stable laws.

page 169

September 13, 2019

170

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

¯ 2 < ∞. Then we have the next general result. Assume first that E X ¯ = 0, E X ¯ 2 = 1 and h0 > 0. Assume that Theorem 6.8. Assume that E X in is non-decreasing and lim

τn = 0. in

(6.25)

Then (6.6) hods with bn = (2ln (τn + log ln ))1/2 . If additionally τn → ∞, then (6.14) hold with cn = (2ln τn )1/2 . If additionally (6.17) is satisfied, then (6.19) holds with dn = (2ln (τn − log ln ))1/2 . If additionally (6.21) is satisfied, then (6.23) holds with bn = (2ln τn )1/2 . Note that condition (6.25) fails when {δn } is separated from 1. Hence, one can assume that δn → 1. It follows that in ∼ ln and one can replace in by ln in the normalizing sequences. Hence, one can assume without loss of generality that in ≥ 1 for all n. For increments over head runs, we obtain the following result. ¯ = 0, E X ¯ 2 = 1 and h0 > 0. Assume that Theorem 6.9. Assume that E X condition A) holds and in is non-decreasing. If δn → 1, then (6.25) is satisfied and, consequently, the result of Theorem 6.8 holds. For increments over monotone blocks, we arrive at the next result. ¯ = 0, E X ¯ 2 = 1 and h0 > 0. Assume Theorem 6.10. Assume that E X that condition B) holds and in is non-decreasing. If (1−δn ) log ln  log(1/p), then (6.25) is fulfilled, the result of Theorem 6.8 holds and τn ∼ (ln − in ) log ln − in log(1/p). For p < 1, we have τn ∼ ln (rn +(log(1/p))2 / log ln ), where rn  0, and, consequently, bn ∼ cn ∼ dn . The last relation can fail for p = 1. We see from Theorems 6.9 and 6.10 that the behaviour of increments of sums of i.i.d. random variables over monotone blocks is much more complicated. Two last theorems yield variants of the LIL. In the case of head runs, if τn ∼ c log ln , then relations (6.6) and (6.14) imply the LIL with the norming sequence (2ln log ln )1/2 which is equivalent to (2 log n log log n/ log(1/p))1/2 . In the case of monotone blocks with p = 1 and τn ∼ c log ln , we also get the LIL with the norming sequence is (2ln log ln )1/2 , but it is equivalent to (2 log n)1/2 . Unfortunately, the exact constant can not be specified since bn

page 170

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Increments of Sums over Head Runs and Monotone Blocks

9789811212826

171

and cn distinguish. Moreover, we have LIL for the case of monotone blocks provided p = 1. We then arrive at the following result for the case of head runs. Corollary 6.1. Assume that conditions of Theorem 6.9 hold. If τn ∼ c log ln , then Mn = L a.s., lim sup  2 log n log log n/ log(1/p) √ √ where c ≤ L ≤ c + 1. The LIL for the case of monotone blocks is as follows. Corollary 6.2. Assume that conditions of Theorem 6.10 hold. If p = 1 and τn ∼ c log ln , then Mn lim sup √ = L a.s. 2 log n √ √ where c ≤ L ≤ c + 1. ¯ 2 = ∞. Turn to the case E X We start with the domain of non-normal attractions of the normal law. We have the following result. ¯ = 0, h0 > 0 and F ∈ D(2). Put Theorem 6.11. Assume that E X     0 1 h2 1 ˆ m(h) ˆ = hG , f (h) = G , where G(x) = u2 dF (u). h 2 h −x

Suppose that in is non-decreasing and (6.25) holds. ˆ fˆ−1 ((τn + log ln )/in )). If additionThen (6.6) holds with bn = in m( ˆ fˆ−1 (τn /in )). If addially τn → ∞, then (6.14) holds with cn = in m( ˆ fˆ−1 ((τn − tionally (6.17) is satisfied, then (6.19) holds with dn = in m( log ln )/in )). If additionally (6.21) is satisfied, then (6.23) holds with ˆ fˆ−1 (τn /in )). bn = in m( ¯ from the domain of the normal attraction of Our next result is for X the completely asymmetric stable laws. ¯ = 0, h0 > 0 and F ∈ DN (α), α ∈ (1, 2). Theorem 6.12. Assume that E X Suppose that in is non-decreasing and (6.25) holds. Put λ = (α − 1)/α. 1/α Then (6.6) holds with bn = λ−λ ln (τn + log ln )λ . If additionally 1/α τn → ∞, then (6.14) holds with cn = λ−λ ln τnλ . If additionally (6.17) 1/α is satisfied, then (6.19) holds with dn = λ−λ ln (τn − log ln )λ . If addition1/α ally (6.21) is satisfied, then (6.23) holds with bn = λ−λ ln τnλ .

page 171

September 13, 2019

172

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

For increments over head runs, we obtain the following result. ¯ = 0, h0 > 0 and F ∈ DN (α), α ∈ (1, 2). Theorem 6.13. Assume that E X Suppose that condition A) holds, in is non-decreasing and (6.25) holds. Put λ = (α − 1)/α. If δn → 1, then (6.25) is satisfied and, consequently, the result of Theorem 6.12 holds. For increments over monotone blocks, we have the next result. ¯ = 0, h0 > 0 and F ∈ DN (α), α ∈ (1, 2). Theorem 6.14. Assume that E X Suppose that condition B) holds, in is non-decreasing and (6.25) holds. Put λ = (α − 1)/α. If (1−δn ) log ln  log(1/p), then (6.25) is fulfilled, the result of Theorem 6.12 holds and the behaviour of τn is the same as in Theorem 6.10. Turn to analogues of LIL for increments of sums over head runs and monotone blocks in the case of domains of normal attraction of completely asymmetric stable laws. In the case of head runs, if τn ∼ c log ln , then the norming sequence 1/α is λ−λ ln (log ln )λ ∼ λ−λ log(1/p)−1/α (log n)1/α (log log n)λ . In the case of monotone blocks with p = 1 and τn ∼ c log ln , the norming sequence is 1/α λ−λ ln (log ln )λ ∼ λ−λ (log n)1/α (log log n)λ−1/α . We then get the next result in the case of head runs. Corollary 6.3. Assume that conditions of Theorem 6.13 hold. If τn ∼ c log ln , then lim sup

λ−λ

Mn −1/α log(1/p) (log n)1/α (log log n)λ

=L

a.s.,

where cλ ≤ L ≤ (1 + c)λ . The LIL for the case of monotone blocks is as follows. Corollary 6.4. Assume that conditions of Theorem 6.14 hold. If p = 1 and τn ∼ c log ln , then lim sup

Mn =L λ−λ (log n)1/α (log log n)λ−1/α

where cλ ≤ L ≤ (1 + c)λ .

a.s.,

page 172

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Increments of Sums over Head Runs and Monotone Blocks

173

Turn to the case of non-normal attraction to completely asymmetric stable laws. ¯ = 0, h0 > 0 and F ∈ D(α), α ∈ (1, 2). Theorem 6.15. Assume that E X Put     1 1 Γ(2 − α) α−1 α ˆ G m(h) ˆ =α h , f (h) = Γ(2 − α)h G , α−1 h h where G(x) = xα F (−x), x > 0. Suppose that in is non-decreasing and (6.25) holds. ˆ fˆ−1 ((τn + log ln )/in )). If additionThen (6.6) holds with bn = in m( ˆ fˆ−1 (τn /in )). If addially τn → ∞, then (6.14) holds with cn = in m( ˆ fˆ−1 ((τn − tionally (6.17) is satisfied, then (6.19) holds with dn = in m( log ln )/in )). If additionally (6.21) is satisfied, then (6.23) holds with ˆ fˆ−1 (τn /in )). bn = in m( We finally concern with the SLLN which follows from Theorem 6.4. ¯ is non-degenerate, E X ¯ ≥ 0 and h0 > 0. Theorem 6.16. Assume that X If in is non-decreasing and relations (6.25) and (6.21) hold, then lim

Mn ¯ = EX ln

a.s.

¯ > 0, then the result immediately follows from Theorem 6.4. Proof. If E X ¯ and the equality γ(0) = E X. ¯ = 0. Take ε > 0. We have Assume that E X Mn − εin ≤ Mn ≤ Mn where Mn =

max

0≤k≤n−in

(Sk+in − Sk + εin )I{u≤Yk+1 ≤···≤Yk+in ≤v} .

By the first part of the proof, we get lim

Mn =ε ln

a.s.

Since ε is an arbitrary positive number, the result follows from the last relation and the above inequalities.

page 173

September 13, 2019

174

6.5

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Bibliographical Notes

The a.s. asymptotic behaviour of the longest head runs has been considered in [Feller (1971)], [Erd˝ os and R´ev´esz (1975)] and [Deheuvels (1985)]. Various generalizations of these results may be found in [Frolov and Martikainen (1999)] and references therein. The a.s. limiting behaviour of monotone blocks has been studied in [Pittel (1981)], [R´ev´esz (1983)], [Novak (1992)]. Generalizations of these results may be found in [Frolov and Martikainen (1999)] and references therein. Results on the asymptotic behaviour of increasing runs in Rd and random fields may be found in [Frolov and Martikainen (1999)] and [Frolov and Martikainen (2001)]) correspondingly. Results for increments of sums over head runs and monotone blocks have been proved in [Frolov et al. (1998)] , [Frolov (1999)], [Frolov et al. (2000a)] , [Frolov et al. (2000b)] and [Frolov (2001)]. Main results of this chapter are obtained in [Frolov (1999, 2001, 2003e)].

page 174

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Bibliography

Amosova, N. N. (1972). On limit theorems for probabilities of moderate deviations, Vestnik Leningrad Univ. , 13, pp. 5–14, english transl.: Vestnik Leningrad Univ. Math., 5, 197–210, (1978). Amosova, N. N. (1979). On probabilities of moderate deviations for sums of independent random variables, Theor. Probab. Appl. 24, 4, pp. 858–865, english transl.: Theor. Probab. Appl., 24, no. 4, 856–863, (1980). Amosova, N. N. (1980). On narrow zones of integral normal convergence, Zap. Nauchn. Semin. LOMI 97, pp. 6–14, english transl.: J. Sov. Math., 24, no. 5, 483–489, (1984). Amosova, N. N. (1984). Probabilities of large deviations in the case of stable limit distributions, Matem. Zametki 35, pp. 125–131, english transl.: Mathematical Notes, 35, no. 1, 68–71, (1984). Bacro, J.-N. and Brito, M. (1991). On Mason’s extension of the Erd˝ os–R´enyi law of large numbers, Statist. Probab. Lett. 11, pp. 43–47. Bacro, J.-N., Deheuvels, P. and Steinebach, J. (1987). Exact convergence rates in Erd˝ os–R´enyi type theorems for renewal processes, Ann. Inst. Henri Poincar´e 23, pp. 195–207. Bahadur, R. R. and Ranga Rao, R. (1960). On deviations of the sample mean, Ann. Math. Statist. 31, 4, pp. 1015–1027. Binswanger, K. and Embrechts, P. (1994). Longest runs in coin tossing, Insur. Math. Econom. 15, pp. 139–149. Book, S. A. (1975a). A version of the Erd˝ os–R´enyi law of large numbers for independent random variables, Bull. Inst. Math. Acad. Sinica 3, 2, pp. 199–211. Book, S. A. (1975b). An extension of the Erd˝ os–R´enyi new law of large numbers, Proc. Amer. Math. Soc. 48, 2, pp. 438–446. Book, S. A. and Shore, T. R. (1978). On large intervals in the Cs¨ org˝ o–R´ev´esz theorem on increments of a Wiener process, Z. Wahrsch. Verw. Geb. 46, pp. 1–11. Borovkov, A. A. (1964). Analisys of large deviations in boundary problems with arbitrary bounds. I, Siberian Math. J. 5, 2, pp. 253–289. Borovkov, A. A. and Borovkov, K. A. (2008). Asymptotic analysis of random 175

page 175

September 13, 2019

176

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

walks. Heavy-tailed distributions (Cambridge University Press, New York). Brieman, L. (1968). A delicate LIL for non-decreasing stable processes, Ann. Math. Statist. 39, pp. 1818–1824. Cai, Z. (1992). Strong approximation and improved Erd˝ os–R´enyi laws for sums of independent non-identically distributed random variables, J. Hangzhou Univ. 19, 3, pp. 240–246. Chan, A. H. (1976). Erd˝ os–R´enyi type modulus of continuity theorems for Brownian sheets, Studia Sci. Math. Hungar. 11, pp. 59–68. Chernoff, H. (1952). A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations, Ann. Math. Statist. 23, 4, pp. 493–507. Choi, Y. K. and Kˆ ono, N. (1999). How big are increments of a two-parameter Gaussian processes, J. Theoret. Probab. 12, pp. 105–129. Cram´er, H. (1938). Sur un nouveau th´eor`eme limite de la th´eorie des probabilit´es, Actual. Sci. Indust. , 736, pp. 5–23. Cs´ aki, E. and R´ev´esz, P. (1979). How big must be the increments of a Wiener process? Acta Math. Acad. Sci. Hungar. 33, pp. 37–49. Cs¨ org˝ o, M. and R´ev´esz, P. (1978). How big are the increments of a multiparameter Wiener process? Z. Wahrsch. verw. Geb. 42, pp. 1–12. Cs¨ org˝ o, M. and R´ev´esz, P. (1979). How big are the increments of a Wiener process? Ann. Probab. 7, pp. 731–737. Cs¨ org˝ o, M. and R´ev´esz, P. (1981). Strong approximations in probability and statistics (Akad´emiai Kiad´ o, Budapest). Cs¨ org˝ o, M. and Steinebach, J. (1981). Improved Erd˝ os–R´enyi and strong approximation laws for increments of partial sums, Ann. Probab. 9, pp. 988–996. Cs¨ org˝ o, S. (1979). Erd˝ os–R´enyi laws, Ann. Statist. 7, pp. 772–787. Daniels, H. E. (1954). Saddlepoint approximations in statistics, Ann. Math. Statist. 25, 4, pp. 631–650. Deheuvels, P. (1985). On the Erd˝ os–R´enyi theorem for random fields and sequences and its relationships with the theory of runs and spacings, Z. Wahrsch. verw. Geb. 70, pp. 91–115. Deheuvels, P. and Devroye, L. (1987). Limit laws of Erd˝ os–R´enyi–Shepp type, Ann. Probab. 15, pp. 1363–1386. Deheuvels, P. and Steinebach, J. (1989). Sharp rates for increments of renewal processes, Ann. Probab. 17, pp. 700–722. Einmahl, U. and Mason, D. M. (1996). Some universal results on the behaviour of increments of partial sums, Ann. Probab. 24, pp. 1388–1407. Embrechts, P. and Cl¨ uppelberg, C. (1993). Some aspects of insurance mathematics, Theor. Probab. Appl. 38, 2, pp. 374–416, english transl.: Theor. Probab. Appl., 38, no. 3, 262–295, (1993). Erd˝ os, P. and R´enyi, A. (1970). On a new law of large numbers, J. Analyse Math. 23, pp. 103–111. Erd˝ os, P. and R´ev´esz, P. (1975). On the length of the longest head-run, In: Topics in Information Theory, Csisz´ ar, I., Elias, P. (eds.). Amsterdam: North Holland,– Coll. Math. Soc. J. Bolyai 16, pp. 218–228. Feller, W. (1943). Generalization of a probability limit theorem of Cram´er, Trans. Amer. Math. Soc. 54,, 3, pp. 361–372.

page 176

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Bibliography

9789811212826

177

Feller, W. (1969). Limit theorems for probabilities of large deviations, Z. Wahrsch. verw. Geb. 14, pp. 1–20. Feller, W. (1971). An introduction to probability theory and its applications, Vol. 1,2 (2nd edn.) (Wiley, New York). Fristedt, B. (1964). The behavior of increasing stable processes for both small and large times, J. Math. Mech. 13, pp. 849–856. Frolov, A. N. (1990). On the Erd˝ os–R´enyi law of large numbers when Cram´er’s condition is not fulfilled, Vestnik Leningrad Univ., Ser. 1, i. 4 , 22, pp. 26–31, english transl.: Vestnik Leningrad Univ. Math., 23, no. 4, 30–35, (1990). Frolov, A. N. (1991). On the Erd˝ os–R´enyi law of large numbers for non-identically distributed random variables under violation of Cram´er’s condition, Vestnik Leningrad Univ., Ser. 1, i. 2 , 8, pp. 57–61, english transl.: Vestnik Leningrad Univ. Math., 24, no. 2, 66–70, (1991). Frolov, A. N. (1993a). Exact convergence rate in the Erd˝ os–R´enyi law for nonidentically distributed random variables, Vestnik St.Petersburg Univ., Ser. 1, i. 1 , 1, pp. 124–125, english transl.: Vestnik St.Petersburg Univ. Math., 26, no. 1, 60–61, (1993). Frolov, A. N. (1993b). On asymptotic behaviour of increments of sums of independent random variables, Vestnik St.Petersburg Univ., Ser. 1, i. 4 , 22, pp. 45–48, english transl.: Vestnik St.Petersburg Univ. Math., 26, no. 4, 52–56, (1993). Frolov, A. N. (1998). On one-sided strong laws for large increments of sums, Statist. Probab. Lett. 37, pp. 155–165. Frolov, A. N. (1999). On asymptotic behaviour of increments of sums over head runs, Zap. Nauchn. Semin. POMI 260, pp. 263–277, english transl.: J. Math. Sci., 109, no. 6, 2229–2240, (2002). Frolov, A. N. (2000). On the asymptotic behaviour of increments of sums of independent random variables, Doklady Akademii nauk 372, 5, pp. 596– 599, english transl.: Doklady Math., 61, no. 3, 409–412, (2000). Frolov, A. N. (2001). On asymptotic behaviour of increments of sums over increasing runs, Zap. Nauchn. Semin. POMI 278, pp. 248–260, english transl.: J. Math. Sci., 118, no. 6, 5650–5657, (2003). Frolov, A. N. (2002a). On asymptotic behaviour of large increments of sums of independent random variables, Theor. Probab. Appl. 47, 2, pp. 366–374, english transl.: Theor. Probab. Appl., 47, no. 2, 315–323, (2003). Frolov, A. N. (2002b). On probabilities of moderate deviations of sums of independent random variables, Zap. Nauchn. Semin. POMI 294, pp. 200–215, english transl.: J. Math. Sci., 127, no. 1, 1787–1796, (2005). Frolov, A. N. (2002c). One-sided strong laws for increments of sums of i.i.d. random variables, Studia Sci. Math. Hungar. 39, pp. 333–359. Frolov, A. N. (2002d). Strong limit theorems for increments of random fields, Theory Stoch. Processes 8 (24), Ns. 1–2, pp. 89–97. Frolov, A. N. (2003a). Limit theorems for increments of processes with independent increments, Doklady Akademii nauk 393, 2, pp. 165–169, english transl.: Doklady Math., 68, no. 3, 345–349, (2003).

page 177

September 13, 2019

178

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Frolov, A. N. (2003b). Limit theorems for increments of sums independent random variables, Theor. Probab. Appl. 48, 1, pp. 104–121, english transl.: Theor. Probab. Appl., 48, no. 1, 93–107, (2004). Frolov, A. N. (2003c). On asymptotic behaviour of increments of random fields, Zap. Nauchn. Semin. POMI 298, pp. 191–207, english transl.: J. Math. Sci., 128, no. 1, 2604–2613, (2005). Frolov, A. N. (2003d). On asymptotics of large increments of sums in non-i.i.d. case, Acta Applicandae Mathematicae 78, pp. 129–136. Frolov, A. N. (2003e). On asymptotics of the maximal gain without losses, Statist. Probab. Lett. 63, pp. 13–23. Frolov, A. N. (2003f). On the Erd˝ os–R´enyi law for renewal processes, Teor. veroyatn. matem. statist. 68, pp. 142–151, (In Russian). Frolov, A. N. (2003g). Strong limit theorems for increments of renewal processes, Zap. Nauchn. Semin. POMI 298, pp. 208–225, english transl.: J. Math. Sci., 128, no. 1, 2614–2624, (2005). Frolov, A. N. (2004a). On the law of the iterated logarithm for increments of sums of independent random variables, Zap. Nauchn. Semin. POMI 320, pp. 174–186, english transl.: J. Math. Sci., 137, no. 1, 4575–4582, (2006). Frolov, A. N. (2004b). Strong limit theorems for increments of sums of independent random variables, Zap. Nauchn. Semin. POMI 311, pp. 260–285, english transl.: J. Math. Sci., 133, no. 3, 1356–1370, (2006). Frolov, A. N. (2004c). Universal strong laws for increments of processes with independent increments, Theor. Probab. Appl. 49, 3, pp. 601–609, english transl.: Theor. Probab. Appl., 49, no. 3, 531–540, (2005). Frolov, A. N. (2005). Converses to the Cs¨ org˝ o–R´ev´esz laws, Statist. Probab. Lett. 72, pp. 113–123. Frolov, A. N. (2007). Limit theorems for increments of compound renewal processes, Zap. Nauchn. Semin. POMI 351, pp. 259–283, english transl.: J. Math. Sci., 152, no. 6, 944–957, (2008). Frolov, A. N. (2008). Asymptotic behaviour of probabilities of moderate deviations, Trudy St.Peterburgskogo Mat. Obschestva 14, pp. 197–211, english transl.: Amer. Math. Soc. Translations. Ser. 2, 228, Proc. St.Petersburg Math. Soc., vol. 14, 157–168, (2009). Frolov, A. N. (2014). Limit theorems of probability theory (Izd. St.Petersburg State Univ., St.Petersburg), (In Russian). Frolov, A. N. and Martikainen, A. I. (1999). On the length of the longest increasing run in Rd , Statist. Probab. Lett. 41, pp. 153–161. Frolov, A. N. and Martikainen, A. I. (2001). On the largest cave in a random field, Studia Sci. Math. Hungar. 37, pp. 213–223. Frolov, A. N., Martikainen, A. I. and Steinebach, J. (1997). Erd˝ os–R´enyi–Shepp type laws in non-i.i.d. case, Studia Sci. Math. Hungar. 33, pp. 127–151. Frolov, A. N., Martikainen, A. I. and Steinebach, J. (1998). On the maximal gain over head runs, Studia Sci. Math. Hungar. 34, pp. 165–181. Frolov, A. N., Martikainen, A. I. and Steinebach, J. (2000a). On the maximal excursion over increasing runs, In: Asymptotic methods in probability and statistics with applications. Birkh¨ auser, Boston, pp. 225–242.

page 178

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Bibliography

9789811212826

179

Frolov, A. N., Martikainen, A. I. and Steinebach, J. (2000b). Strong laws for the maximal gain over increasing runs, Statist. Probab. Lett. 50, pp. 305–312. Galambos, J. (1978). The asymptotic theory of extreme order statistics (John Wiley and sons, New York etc.). Gut, A. (2009). Stopped random walk, (2nd edn.) (Springer Science+Business Media, New York). Hanson, D. L. and Russo, R. P. (1983). Some results on increments of Wiener process with applications to lag sums of independent identically distributed random variables, Ann. Probab. 11, pp. 609–623. Hanson, D. L. and Russo, R. P. (1985). Some limit results for lag sums of independent, non-i.i.d. random variables, Z. Wahrsch. verw. Geb. 68, pp. 425–445. Hartman, P. and Wintner, A. (1941). On the law of the iterated logarithm, Amer. J. Math. 63, pp. 169–176. H¨ oglund, T. (1979). A unified formulation of the central limit theorem for small and large deviations from the mean, Z. Wahrsch. verw. Geb. 49, pp. 105– 117. Ibragimov, I. A. and Linnik, Y. V. (1971). Independent and stationary sequences of random variables (Volters-Noordhoff, Groningen). Kalinauska˘ıte, N. (1971). Upper and lower functions for sums of independent random variables with stable limit distributions, Select. Transl. Math. Statist. Probab. 10, pp. 305–314. Kesten, H. (1970). The limit points of a normalized random walk, Ann. Math. Statist. 41, pp. 1173–1205. ¨ Khintchine, A. Y. (1929a). Uber dei positiven und negativen abweichungen des arithmetischen mittels, Math. Ann. 101, pp. 381–385. ¨ Khintchine, A. Y. (1929b). Uber einen neuen grenzwertsatz der wahrscheinlichkeitsrechnung, Math. Ann. 101, pp. 745–752. Kim, L. V. and Nagaev, A. V. (1975). On the asymmetrical problem of large deviations, Theor. Probab. Appl. 28, pp. 58–68, english transl.: Theor. Probab. Appl., 20, no. 1, 57–68, (1975). Klass, M. J. (1976). Toward a universal law of the iterated logarithm. I, Z. Wahrsch. verw. Geb. 36, pp. 165–178. Klass, M. J. (1977). Toward a universal law of the iterated logarithm. II, Z. Wahrsch. verw. Geb. 39, pp. 151–165. ¨ Kolmogorov, A. N. (1929). Uber das gesetz des iterierten logarithmus, Math. Ann. 101, pp. 126–135. Koml´ os, J., Major, P. and Tusn´ ady, G. (1975). An approximation of partial sums of independent rv’s, and the sample df. I, Z. Wahrsch. Verw. Geb. 32, pp. 111–131. Koml´ os, J., Major, P. and Tusn´ ady, G. (1976). An approximation of partial sums of independent rv’s, and the sample df. II, Z. Wahrsch. Verw. Geb. 34, pp. 33–58. Lamperti, J. (1996). Probability: a survey of mathematical theory, (2nd edn.) (John Wiley and Sons, New York). Lanzinger, H. (2000). A law of the single logarithm for moving averages of random

page 179

September 13, 2019

180

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

variables under exponential moment conditions, Studia Sci. Math. Hungar. 36, pp. 65–91. Lanzinger, H. and Stadtm¨ uller, U. (2000). Maxima of increments of partial sums for certain subexponential distributions, Stoch. Processes Appl. 86, pp. 307– 322. Lin, Z. Y. (1990). The Erd˝ os–R´enyi laws of large numbers for non-identically distributed random variables, Chin. Ann. Math. 11B, pp. 376–383. Lin, Z. Y., Choi, Y. K. and Hwang, K. S. (2001). Some limit theorems on the increments of a multi-parameter fractional Brownian motion, Stoch. Anal. Appl. 19, pp. 499–517. Lin, Z. Y., Lu, C. R. and Shao, Q.-M. (1991). Contribution to the limit theorems, Contemporary Math. 118, pp. 221–237. Linnik, Y. V. (1961a). Limit theorems for sums of independent random variables taking into account large deviations. I, Theor. Probab. Appl. 6, pp. 145–163, english transl.: Theor. Probab. Appl., 6, no. 2, 131–148, (1961). Linnik, Y. V. (1961b). Limit theorems for sums of independent random variables taking into account large deviations. II, Theor. Probab. Appl. 6, pp. 377– 391, english transl.: Theor. Probab. Appl., 6, no. 4, 345–360, (1961). Linnik, Y. V. (1961c). On the probability of large deviations for the sums of independent variables, Proc. 4th Berkeley Symp. on Math. Statist. and Probab., 2, Univ. Calif. Press, Berkeley and Los Angeles, pp. 289–306. Linnik, Y. V. (1962). Limit theorems for sums of independent random variables taking into account large deviations. III, Theor. Probab. Appl. 7, pp. 121– 134, english transl.: Theor. Probab. Appl., 7, no. 2, 115–129, (1962). Lipschutz, M. (1956a). On strong bounds for sums of independent random variables which tend to a stable distribution, Trans. Amer. Math. Soc. 81, pp. 135–154. Lipschutz, M. (1956b). On the magnitude of the error in the approach to stable distributions, Indag. Math. 18, pp. 281–294. Lo`eve, M. (1963). Probability theory, (3rd edn.) (Van Nostrand, Princeton). Lynch, J. (1983). Some comments on the Erd˝ os–R´enyi law and a theorem of Shepp, Ann. Probab. 11, pp. 801–802. Martikainen, A. I. (1980). Converse of the law of the iterated logarithm for random walk, Theor. Probab. Appl. 25, 2, pp. 364–366, english transl.: Theor. Probab. Appl., 25, no. 2, 361–362, (1981). Martikainen, A. I. (1985). On one-sided law of the iterated logarithm, Theor. Probab. Appl. 30, 4, pp. 694–705, english transl.: Theor. Probab. Appl., 30, no. 4, 736–749, (1986). Mason, D. M. (1989). An extended version of the Erd˝ os–R´enyi strong law of large numbers, Ann. Probab. 17, pp. 257–265. Michel, R. (1976). Nonuniform central limit bounds with applications to probabilities of deviations, Ann. Probab. 4, 1, pp. 102–106. Mijnheer, J. L. (1972). A law of the iterated logarithm for the asymmetric stable law with characteristic exponent one, Ann. Math. Statist. 43, pp. 358–362. Mijnheer, J. L. (1974). Sample path properties of stable processes (Mathematish Centrum, Amsterdam).

page 180

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Bibliography

9789811212826

181

Millar, P. W. (1972). Some remarks on asymmetric processes, Ann. Math. Statist. 43, pp. 597–601. Nagaev, A. V. (1969). Limit theorems including the effect of large deviations when Cram´er’s condition is violated, Izv. AN UzSSR, seriya fiz.-matem. nauk , 6, pp. 17–22, (In Russian). Nagaev, A. V. (1983). On asymmetric problem of large deviations when the limit law is stable, Theor. Probab. Appl. 28, pp. 637–645, english transl.: Theor. Probab. Appl., 28, no. 4, 670–680, (1984). Nagaev, S. V. (1965). Some limit theorems for large deviations, Theor. Probab. Appl. 10, pp. 231–254, english transl.: Theor. Probab. Appl., 10, no. 2, 214–235, (1965). Nagaev, S. V. (1981). On the asymptotic behaviour of one-sided large deviation probabilities, Theor. Probab. Appl. 26, pp. 369–372, english transl.: Theor. Probab. Appl., 26, no. 2, 362–366, (1982). Novak, S. Y. (1992). Longest runs in a sequence of m-dependent random variables, Probab. Theory Relat. Fields 91, pp. 269–281. Ortega, J. and Wschebor, M. (1984). On the increments of the Wiener process, Z. Wahrsch. Verw. Geb. 65, pp. 329–339. Osipov, L. V. (1972). On probabilities of large deviations of sums of independent random variables, Theor. Probab. Appl. 17, 2, pp. 320–341, english transl.: Theor. Probab. Appl., 17, no. 2, 309–331, (1973). Petrov, V. V. (1954). Generalization of Cram´er’s limit theorem, Uspekhi mat. nauk 9, 4, pp. 195–202, english transl.: Selected Transl. Math. Statist. Probab., 6, 1–8, (1966), AMS, Providence, RI. Petrov, V. V. (1961). On large deviations of sums of random variables, Vestnik Leningrad Univ. , 1, pp. 25–37, (In Russian). Petrov, V. V. (1963). Limit theorems for large deviations under violation of Cram´er’s condition. I, Vestnik Leningrad Univ. , 19, pp. 49–68, english transl. in ”Selected Transl. on Math. and Probab.” 7, 235–280, (1968). Petrov, V. V. (1964). Limit theorems for large deviations under violation of Cram´er’s condition. II, Vestnik Leningrad Univ. , 1, pp. 58–75, english transl. in ”Selected Transl. on Math. and Probab.” 7, 235–280, (1968). Petrov, V. V. (1965). On probabilities of large deviations of sums of independent random variables, Theor. Probab. Appl. 10, 2, pp. 310–322, english transl.: Theor. Probab. Appl., 10, no. 2, 287–298, (1965). Petrov, V. V. (1968). Asymptotic behaviour of probabilities of large deviations, Theor. Probab. Appl. 13, 3, pp. 432–444, english transl.: Theor. Probab. Appl., 13, no. 3, 408–420, (1968). Petrov, V. V. (1975). Sums of independent random variables (Springer, New York etc.). Petrov, V. V. (1987). Limit theorems for sums of independent random variables (Nauka, Moscow), (In Russian). Petrov, V. V. (1995). Limit theorems of probability theory (Oxford University Press, New York). Pfuhl, W. and Steinebach, J. (1988). On precise asymtotics for the Erd˝ os–R´enyi increments of random fields, Pub. Inst. Stat. Univ. Paris 33, pp. 49–66.

page 181

September 13, 2019

182

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Pittel, B. G. (1981). Limiting behaviour of a process of runs, Ann. Probab. 9, pp. 119–129. Pruitt, E. W. (1981). General one-sided laws of the iterated logarithm, Ann. Probab. 9, pp. 1–48. R´ev´esz, P. (1983). Three problems on the lengths of increasing runs, Stoch. Process. Appl. 15, pp. 169–179. R´ev´esz, P. (1990). Random walk in random and non-random environment (World Scientific Publ., Singapore). Rosalsky, A. (1980). On the converse of the iterated logarithm law, Sankhya A 42., N. 1–2, pp. 103–108. Rozovskii, L. V. (1981). Limit theorems for large deviations in a narrow zone, Theor. Probab. Appl. 26, pp. 834–845, english transl.: Theor. Probab. Appl., 26, no. 4, 834–845, (1982). Rozovskii, L. V. (1989a). An estimate for probabilities of large deviations, Matem. Zametki 42, pp. 145–156, english transl.: Mathematical Notes, 42, no. 1, 590–597, (1987). Rozovskii, L. V. (1989b). Probabilities of large deviations of sums of independent random variables with common distribution function in the domain of attraction of the normal law, Theor. Probab. Appl. 38, pp. 686–705, english transl.: Theor. Probab. Appl., 34, no. 4, 625–644, (1989). Rozovskii, L. V. (1993). Probabilities of large deviations on the whole axis, Theor. Probab. Appl. 38, pp. 79–109, english transl.: Theor. Probab. Appl., 38, no. 1, 53–79, (1993). Rozovskii, L. V. (1997). Probabilities of large deviations for sums of independent random variables with a common distribution function from the domain of attraction of an asymmetric stable law, Theor. Probab. Appl. 42, pp. 496–530, english transl.: Theor. Probab. Appl., 42, no. 3, 454–482, (1998). Rozovskii, L. V. (1999). A lower bound for probabilities of large deviations of sums of independent random variables with finite variations, Zap. Nauchn. Semin. POMI 260, pp. 218–239, english transl.: J. Math. Sci., 109, no. 6, 2192–2209, (2002). Rozovskii, L. V. (2001). On lower bound for probabilities of large deviations of sample mean under Cram´er’s condition, Zap. Nauchn. Semin. POMI 278, pp. 208–224, english transl.: J. Math. Sci., 118, no. 6, 5624–5634, (2003). Rozovskii, L. V. (2003). Probabilities of large deviations for some classes of distributions satisfying Cram´er’s condition, Zap. Nauchn. Semin. POMI 298, pp. 161–185, english transl.: J. Math. Sci., 128, no. 1, 2585–2600, (2005). Rozovskii, L. V. (2004). Sums of independent random variables with finite variances – moderate deviations and bounds in CLT, Zap. Nauchn. Semin. POMI 311, pp. 242–259, english transl.: J. Math. Sci., 133, no. 3, 1345– 1355, (2006). Rychlik, Z. (1983). Nonuniform central limit bounds with applications to probabilities of deviations, Theor. Probab. Appl. 28, 4, pp. 646–652. Samorodnitsky, G. and Taqqu, M. (1994). Stable non-Gaussian processes (Chapman and Hall/CRC, New York). Scherbakova, O. E. (2003). Rate of convergence of increments for random fields,

page 182

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

Bibliography

9789811212826

183

Zap. Nauchn. Semin. POMI 298, pp. 304–315, english transl.: J. Math. Sci., 128, no. 1, 2669–2676, (2005). Scherbakova, O. E. (2004). Convergence rate for large increments of random fields, Zap. Nauchn. Semin. POMI 320, pp. 187–225, english transl.: J. Math. Sci., 137, no. 1, 4583–4608, (2006). Seneta, E. (1976). Regularly varying functions, Lecture Notes in Mathematics 508. Shao, Q.-M. (1989). On a problem of Cs¨ org˝ o and R´ev´esz, Ann. Probab. 17, pp. 809–812. Shao, Q.-M. (1995). On a conjecture of R´ev´esz, Proc. AMS 123, pp. 575–582. Shepp, L. A. (1964). A limit law concerning moving averages, Ann. Math. Statist. 35, pp. 424–428. Slastnikov, A. D. (1978). Limit theorems for moderate deviation probabilities, Theor. Probab. Appl. 23, pp. 322–340, english transl.: Theor. Probab. Appl., 23, no. 2, 322–340, (1979). Slastnikov, A. D. (1984). Narrow zones of normal convergence for sums of nonidentically distributed random variables, Theor. Probab. Appl. 29, pp. 570– 574, english transl.: Theor. Probab. Appl., 29, no. 3, 570–574, (1985). Statuleviˇcius, V. A. (1966). On large deviations, Z. Wahrsch. verw. Geb. 6, pp. 133–144. Steinebach, J. (1978). On a necessary condition for the Erd˝ os–R´enyi law of large numbers, Proc. Amer. Math. Soc. 68, pp. 97–100. Steinebach, J. (1981). On general versions of Erd˝ os–R´enyi laws, Z. Wahrsch. Verw. Geb. 56, pp. 549–554. Steinebach, J. (1982). Between invariance principles and Erd˝ os–R´enyi laws, Coll. Math. Soc. J. Bolyai 36, pp. 981–1005. Steinebach, J. (1983). On the increments of partial sum processes with multidimensional indices, Z. Wahrsch. verw. Geb. 63, pp. 59–70. Steinebach, J. (1986). Improved Erd˝ os–R´enyi and strong approximation laws for increments of renewal processes, Ann. Probab. 14, pp. 547–559. Steinebach, J. (1991). Strong laws for small increments of renewal processes, Ann. Probab. 19, pp. 1768–1776. Steinebach, J. (1998). On a conjecture of R´ev´esz and its analogue for renewal processes, In: Szyszkowicz B. (Ed.), Asymptotic methods in probability and statistics. ICAMPS 197. Amsterdam, North Holland/Elsevier, pp. 311–322. Strassen, V. (1964). An invariance principle for the law of the iterated logarithm, Z. Wahrsch. Verw. Geb. 3, pp. 211–226. Terterov, M. N. (2011). On limiting behaviour of increments of sums of independent random variables from domains of attractions of asymmetric stable laws, Vestnik St.Petersburg Univ., Ser. 1 , 2, pp. 95–103, english transl.: Vestnik St.Petersburg Univ. Math., 44, no. 2, 147–154, (2011). Wolf, V. (1968). Some limit theorems for large deviations, Doklady AN SSSR 178, 1, pp. 21–23, english transl.: Soviet. Math., Doklady 9, no. 1, 14–17, (1968). Wolf, V. (1970). Some limit theorems for large deviations of sums of independent random variables, Doklady AN SSSR 191, 6, pp. 1209–1211, english transl.:

page 183

September 13, 2019

184

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Soviet. Math., Doklady 11, no. 2, 509–512, (1970). Zinchenko, N. M. (1987). Asymptotic for increments of stable stochastic processes with jumps of one sign, Theor. Probab. Appl. 32, 4, pp. 793–796, english transl.: Theor. Probab. Appl., 32, no. 4, 724–727, (1987). Zinchenko, N. M. (1994). On the asymptotic behaviour of increments of certain classes of random fields, Theor. Probab. Math. Statst. 48, pp. 7–11. Zolotarev, V. M. (1962). On a new point of view on limit theorems taking into account large deviations, Trudy VI Vsesouyuzn. soveschaniya po teorii veroatn. i matem. statist. Gos. izd. polit. i nauchn. liter. LitSSR, Vilnius, pp. 43–47, (In Russian).

page 184

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Author Index

Amosova, N.N., 75

Feller, W., 2, 53, 60, 75, 151, 174 Fristedt, B., 135 Frolov, A.N., 6, 75, 76, 100, 115– 117, 135, 151, 155, 158, 174

Bacro, J.-N., 115, 151 Bahadur, R.R., 75 Binswanger, K., 156 Book, S.A., 115, 116, 135 Borovkov, A.A., 75 Borovkov, K.A., 75 Brieman, L., 116, 135 Brito, M., 115

Galambos, J., 6 Gut, A., 151 H¨ oglund, T., 75 Hanson, D.L., 116, 135 Hartman, P., 116 Hwang, K.S., 135

Cai, Z., 116 Chan, A.H., 135 Chernoff, H., 75, 76 Choi, Y.K., 135 Cl¨ uppelberg, C., 157 Cram´er, H., 75 Cs¨ org˝ o, M., 5–7, 89, 115, 135 Cs¨ org˝ o, S., 5, 115 Cs´ aki, E., 135

Ibragimov, I.A., 32, 59, 75 Kˆ ono, N., 135 Kalinauska˘ıte, N., 116 Kesten, H., 108 Khintchine, A.Ya., 75 Kim, L.V., 75 Klass, M.J., 109, 116 Kolmogorov, A.N., 116 Koml´ os, J., 116

Daniels, H.E., 75 Deheuvels, P., 5, 115, 135, 151, 174 Devroye, L., 5, 115

Lamperti, J., 2, 3 Lanzinger, H., 115 Lin, Z.Y., 116, 135 Linnik, Yu.V., 32, 59, 75 Lipschutz, M., 75, 116

Einmahl, U., 115 Embrechts, P., 156, 157 Erd˝ os, P., 4, 115, 174 185

page 185

September 13, 2019

186

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Lo`eve, M., 2 Lu, C.R., 116 Lynch, J., 5, 115

Rozovskii, L.V., 75 Russo, R.P., 116, 135 Rychlik, Z., 75

Major, P., 116 Martikainen, A.I., 3, 75, 108, 116, 155, 158, 174 Mason, D.M., 5, 115, 165–167 Michel, R., 75 Mijnheer, J.L., 3, 75, 116, 135 Millar, P.W., 135

Samorodnitsky, G., 75 Scherbakova, O.E., 135 Seneta, E., 57, 58, 61–63, 96, 97 Shao, Q.-M., 115, 116 Shepp, L.A., 4, 115 Shore, T.R., 135 Slastnikov, A.D., 75 Stadtm¨ uller, U., 115 Statuleviˇcius, V.A., 75 Steinebach, J., 5, 75, 115, 116, 135, 151, 174 Strassen, V., 117

Nagaev, A.V., 75 Nagaev, S.V., 75 Novak, S.Yu., 174 Ortega, J., 135 Osipov, L.V., 75 Petrov, V.V., 3, 75, 117 Pfuhl, W., 135 Pittel, B.G., 174 Pruitt, E.W., 3, 116 R´ev´esz, P., 5–7, 89, 115, 135, 174 R´enyi, A., 4, 115 Ranga Rao, R., 75 Rosalsky, A., 3, 108, 116

Taqqu, M., 75 Terterov, M.N., 115 Tusn´ ady, G., 116 Wintner, A., 116 Wolf, V., 75 Wschebor, M., 135 Zinchenko, N.M., 135 Zolotarev, V.M., 75

page 186

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

General Index

Chernoff theorem, 37 classification of distributions, 33, 37 CLT, 3 completely asymmetric stable laws, 3, 4, 31 asymptotic of tail of distribution, 31 characteristic function, 31 large deviations function, 38 moment generating function, 31 compound Poisson process, 13, 133 and domains of attraction, 134 characteristic function, 134 moments, 134 conjugate distribution, 10, 29 Cs¨ org˝ o–R´ev´esz laws, 1, 6, 89 for increments of sums over head runs, 169 for increments of sums over monotone blocks, 169 for moduli of increments of processes with independent increments, 133 for moduli of increments of sums, 104 for moduli of increments of Wiener process, 8 for processes with independent increments, 128

for renewal processes, 149 for stable processes without positive jumps, 126 for Wiener process, 7, 126 domain of attraction, 3, 32 of asymmetric stable laws, 32 duality between sums and renewal processes, 11, 139 Erd˝ os–R´enyi law, 1, 4, 88 for increments of sums over head runs, 169 for increments of sums over monotone blocks, 169 for processes with independent increments, 127 for renewal processes, 146 for the stable processes without positive jumps, 127 for Wiener process, 8, 127 extreme order statistics, 5 a.s. behaviour absence of non-trivial limit, 6 maximum from exponential random variables, 5 maximum from normal random variables, 5 187

page 187

September 13, 2019

188

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

Universal theory for strong limit theorems of probability

Glivenko–Cantelli theorem, 2

Linnik condition, 48

head run, 15, 153 length of the longest head run, 153, 157

m-period moving average, 5 Mason’s extension of Erd˝ os–R´enyi law, 5, 88 maximal gain over series of increasing gains, 14 without losses, 14 moment generating function, 28, 33 monotone block, 15, 154 length of the longest monotone block, 154, 158

increasing run, 15, 154 increments of sums over head runs, 14, 15, 156 increments of sums over monotone blocks, 14, 15, 156 large deviations function, 5, 9, 10, 33 large deviations method, 8 Legendre transform, 10 LIL, 1–3, 6, 98 for completely asymmetric stable random variables, 3 for increments of processes with independent increments, 129, 130 for increments of renewal processes, 149 for increments of sums, 6 for increments of sums over head runs, 171 for increments of sums over monotone blocks, 171 for increments of Wiener process, 8 for moduli of increments of processes with independent increments, 133 for moduli of increments of sums, 104 for moduli of processes with independent increments, 133 for modulus of Wiener process, 8 for processes with independent increments, 128–130 for renewal processes, 12, 149 for stable processes without positive jumps, 13, 126, 127 for Wiener process, 8, 126, 127

non-invariance, 37, 38 one-sided Cram´er condition, 28 one-sided Linnik condition, 48 Pareto distribution, 4 Poisson process, 12, 135 probabilities of large deviations, 8, 9, 27 process with independent increments, 12, 119 Rademaher functions, 3 renewal process, 11, 137 Shepp law, 1, 4, 88 for processes with independent increments, 127 SLLN, 1–3, 101 for increments of processes with independent increments, 131 for increments of renewal processes, 151 for increments of sums over head runs, 173 for increments of sums over monotone blocks, 173 for moduli of increments of processes with independent increments, 133 for moduli of increments of sums, 105

page 188

September 13, 2019

15:48

Universal theory for strong limit theorems of probability – 11625

9789811212826

General Index

for processes with independent increments, 128, 131 for renewal processes, 12, 151 for stable processes without positive jumps, 126 for Wiener process, 126 space of c´ adl` ag functions, 13, 119 stable process without positive jumps, 126 standard normal law large deviations function, 10 asymptotic of tail of distribution, 32 Stirling’s formula, 155 stochastic geyser problem, 5

189

strong approximation, 7, 89, 149 strong invariance, 6 strong non-invariance, 6 universal norming sequence, 11, 78 universal strong laws, 79, 84 for increments of sums over head runs, 168 for increments of sums over monotone blocks, 168 for processes with independent increments, 119 for renewal processes, 137 Wiener process, 7, 12, 13, 126

page 189