The Asymptotic Theory of Extreme Order Statistics, Second Edition 0898749573

Citation preview

The Asymptotic Theory }of Extreme Order Statistics _

Second Edition

The Asymptotic Theory of Extreme OrderStatistics JANOSGALAMBOS Temple University Philadelphia, Pennsylvania

&

ROBERTE. KRIEGER PUBLISHING COMPANY MALABAR, FLORIDA 1987

Original Edition 1978 Second Edition

1987

Printed and Published by ROBERTE. KRIEGER PUBLISHING COMPANY, INC. KRIEGER DRIVE MALABAR, FLORIDA 32950 Copyright © 1978 (original edition) by John Wiley & Sons, Inc. Transferred to author 1985 Copyright © 1987 (new material) by Robert E. Krieger Publishing Co.., Inc.

All rights reserved. No part of this book may be reproduced in any form or by any means, electronic or mechanical, including information storage and retrieval systems without permission in writing from the publisher. Noliability is assumed with respectto the use ofthe information containedherein. Printed in the United States of America.

Library of Congress Cataloging-in-Publication Data

Galambos, Janos, 1940-

The asymptotic theory of extreme order statistics.

Bibliography:p. Includes index. 1. Mathematical statistics—Asymptotic theory. 2. Stochastic processes. 3. Random variables. 4. Extreme value theory. 5. Distribution (Probability theory) I. Title. QA274.G34 1987 519.2 86-7207 ISBN 0-89874-957-3

he wtf

SYC.AR"P g9

Preface to the Second Edition

Sincethe first edition of this book there has been an active developmentof

extreme value theory. Thisis reflected both in the added material, and in the

increased numberof references. Substantial changes have been madeon the material related to the von Misesconditions, on the estimates of the speed of convergenceofdistribution, and in Chapter 5 on multivariate extremes. In particular, James PickandsIII’s proof of the representation theorem is included with the kind

permission of Pickands, for which I am grateful. I am also indebted to R.H. Berk, R. Mathar, R. Mucci, J. Tiago de Oliveira and I. Weissman, who pointedouterrors in the book, and to M. Falk, L.

de Haan, R.D. Reiss, E. Seneta and W. Vervaatfor their commentseither on the original book or on the new material. JANOS GALAMBOS Willow Grove, Pa.

Preface

The asymptotic theory of extreme order statistics provides in some cases exact but in most cases approximate probabilistic models for random quantities when the extremes govern the laws of interest (strength of materials, floods, droughts, air pollution, failure of equipment, effects of

food additives, etc.). Therefore, a complicated situation can be replaced by a comparatively simple asymptotic model if the basic conditions of the

actual situation are compatible with the assumptions of the model. In the present book I describe all known asymptotic models. In addition to finding the asymptotic distributions, both univariate and multivariate, I also include results on the almost sure behavior of the extremes. Finally, random sample sizes are treated and a special random size, the so-called record times, is discussed in more detail. A short section of the last chapter dealing with extremal processes is more mathematical than the rest of the book and intended for the specialist only. Let me stress a few points about the asymptotic theory of extremes. I have mentioned that an asymptotic model may sometimes lead to the exact stochastic description of a random phenomenon.Such cases occur when a random quantity can be expressed as the minimum or maximum of the quantities associated with an arbitrarily large subdivision (for example, the strength of a sheet of a metal is the minimum ofthe strengths of the pieces of the sheet). But whether a model is used as an exact solution or as an approximation, its basic assumptions decide whether it is applicable in a given problem. Therefore, if the conclusions for several models are the same, each model contributes to the theory by showing that those conclu-

sions are applicable under different circumstances. One of the central

problems of the theory is whether the use of a classical extreme value distribution is justified—that is, a distribution which can be obtained as

the limit distribution of a properly normalized extreme of independent and identically distributed random variables. Several of the models of the book give an affirmative answer to this question. In several other cases, however, Vii

Vili

PREFACE

limiting distributions are obtained that do not belong to the three classical types. This is what Bayesian statisticians and reliability scientists expected all along (and they actually used these distributions without appealing to extreme value theory). I sincerely hope that these distributions will be widely used in other fields as well. One more point that is not encountered in most cases of applied Statistics comes up in the theory of extremes. Even if one can accept that the basic random variables are independent and identically distributed, one cannot make a decision on the population distribution by standard statistical methods (goodness of fit tests). I give an example (Example 2.6.3) where, by usual statistical methods, both normality and lognormality are acceptable but the decision in terms of extremes is significantly different depending on the actual choice of one of the two mentioned distributions. It follows that this choice has to be subjective (this is the reason for two groups coming to opposite conclusions, even though they had the same information). The book is mathematically rigorous, but I have kept the applied scientist in mind both in the selection of the material and in the explanations of the mathematical conclusions through examples and remarks. These remarks and examples should also make the book moreattractive as a graduate text. I hope, further, that the book will promote the existing theory among applied scientists by giving them access to results that were scattered in the mathematical literature. An additional aim was to bring the theory together for the specialists in the field. The survey of the literature at the end of each chapter and the extensive bibliography are for these purposes. The prerequisites for reading the book are minimal; they do not go beyond basic calculus and basic probability theory. Some theorems of probability theory (including expectations or integrals), which I did not expect to have been covered in a basic course, are collected in Appendix I. The only exception is the last section in Chapter 6, which is intended mainly for the specialists. By the nature of the subject matter, some familiarity with statistics and distribution functions is an advantage, although I introduce all distributions used in the text. The book can be used as a graduate text in any department where probability theory or mathematical statistics are taught. It may also serve as a basis for a nonmathematical course on the subject, in which case most proofs could be dropped but their ideas presented through special cases (e.g., starting with a simple class of population distributions). In a course, or at a first reading, Chapter 4 can be skipped; Chapters 5 and 6 are not dependent on it. No books nowin print cover the materials of any of Chapters 1 or 3-6. The only overlap with existing monographs is Chapter 2, whichis partial-

PREFACE

iX

ly contained in the book by Gumbel (1958) and in the monograph of de Haan (1970) (see the references). It should be added, however, that Gumbel’s book has an applied rather than theoretical orientation. His methods are not applicable when the restrictive assumptions of Chapter 2 are not valid. Although many proofs are new here, the credit for the theory, as it is known at present, is due to those scientists whose contributions raised it to

its current level. It is easy to unify and simplify proofs when the whole material is collected at one place. I did not have time to thank the manyscientists individually who responded to my requests and questions. My heartiest thanks go to them all over the world. My particular thanks are due to those scientists who apply extreme value theory and who so patiently discussed the problems with meeither in person orin letters. I am indebted to Professor David G. Kendall, whom I proudly count among myfriends, for presenting my plan to John Wiley & Sons, Inc. I should also like to thank Mrs. Mittie Davis for her skill and care in typing the manuscript. JANOS GALAMBOS Willow Grove, Pennsyloania

March 1978

Contents

Notations and conventions

XV

INTRODUCTION: ESTIMATES IN THE UNIVARIATE CASE

1.

1.2 1.3 1.4 1.5 1.6

1.7 1.8

Problems leading to extreme values of random variables The mathematical model Preliminaries for the independent and identically distributed case Bounds onthe distribution of extremes Illustrations Special properties of the exponential distribution in the light of extremes Survey of the literature Exercises

WEAK CONVERGENCE FOR INDEPENDENT AND IDENTICALLY DISTRIBUTED VARIABLES

2.1 2.2 2.3 2.4 2.5 2.6 2.7

Limit distributions for maxima and minima: sufficient conditions Other possibilities for the normalizing constants in Theorems 2.1.1—2.1.6 The asymptotic distribution of the maximum and minimum for somespecial distributions Necessary conditions for weak convergence The proofof part (iii) of Theorem 2.4.3 Illustrations Further results on domains ofattraction xi

16 26 31 43 45

51

53 39 67 74 87 96 101

xii

CONTENTS

2.8 2.9 2.10 2.11 2.12

Weak convergence for the kth extremes The range and midrange Speed of convergence Survey of the literature Exercises

125 128 134 149 155

WEAK CONVERGENCE OF EXTREMES IN THE GENERAL CASE

160

3.1 3.2 3.3. 3.4 3.5 3.6

160 163 170 173 179

3.7 3.8 3.9 3.10 3.11 3.12

A new look at failure models The special role of exchangeable variables Preparations for limit theorems A limit theorem for mixtures A theoretical model Segments of infinite sequences of exchangeable variables Stationary sequences Stationary Gaussian sequences Limiting formsof the inequalities of Section 1.4 Minimum and maximum of independentvariables The asymptotic distribution of the kth extremes Some applied models 3.12.1.

Exact models via characterization theorems 3.12.2. Exact distributions via asymptotic theory (strength of materials) 3.12.3. Strength of bundles of threads 3.12.4. Approximation by i.i.d. variables 3.12.5 Timetofirst failure of a piece of equipment with a large number of components

184 192 199 210 216 220 224 224 225 227 228 231

3.13 Surveyof theliterature 3.14 Exercises

233 238

DEGENERATE LIMIT LAWS; ALMOST SURE RESULTS

243

4.1 4.2 43

243 248

Degenerate limit laws Borel-Cantelli lemmas Almost sure asymptotic properties of extremes of ii.d. variables 4.4 Limsup and liminf of normalized extremes 4.5 The role of extremes in the theory of sums of random variables

251 263

270

CONTENTS

4.6 4.7

5.

Survey of the literature Exercises

279 282

MULTIVARIATE EXTREME VALUE DISTRIBUTIONS

286

5.1 Basic properties of multivariate distributions 5.2 Weak convergence of extremes for iid. random

287

5.3

vectors: basic results Furthercriteria for the i.i.d. case

290 296

5.5

Concomitants of order statistics

314

5.7

Exercises

322

5.4 On the properties of H (x) 5.6

6.

Xlil

Survey of the literature

302

318

MISCELLANEOUS RESULTS

328

6.1 6.2

328 330

6.3 6.4

The maximum queuelength in a stable queue Extremes with random samplesize Record times Records

6.5 Extremal processes 6.6 Survey of the literature 6.7

Exercises

APPENDIXES I. SOME BASIC FORMULAS FOR PROBABILITIES AND EXPECTATIONS

II.

III.

338 347

352 356 360

362

THEOREMS FROM FUNCTIONAL ANALYSIS

370 .

SLOWLY VARYING FUNCTIONS

377

REFERENCES

379

INDEX

411

Notations and Conventions

X1,Xy.--5

Z,

W,

n

F(x)= P(X o(F) toa(F) H(x)

L(x)

basic random variables.

maximum of X,,X,,...,X,. minimum of X,,X,,...,X,.

distribution function of X. distribution function of Z,. distribution function of W,. inf{ x: F(x) >0}. sup{ x: F(x)o(F) means t > a(F) and t3a(F) limit of H,(a,+5,x) with some constants a, and b, >0. limit of L,(c,+4,x) with some constants c, and d, >0. the rth order statistic of X,,X,,...,X,. Thus X,., < X3.,,..., X,, and the behavior of either

Z,, =max(X,X>)...,X,) or

W, =min(X,,X>,...,X,) was ofinterest. As a matter of fact, in terms of floods, X; may denote the waterlevel of a given river on day j, “day 1” being, for example, the day of publication of this book. Since we do not know the water levels in advance, they are random to us. A question such as, “How likely is it that in this century the water level of our river remains below 230 cm?”is evidently asking the value P(Z, < 230), the probability of the event {Z, < 230}. Here is, of course, the number of days remaining in this century after the publication of this book. On the other hand, if we want to use the river as a source of

4

INTRODUCTION TO THE UNIVARIATE CASE

1.2

energy, then our interest is in how far the water level can fall. This

translates into P(W,, > a), where we specify the value of a. Similarly, if X; is the amountof rainfaJl, the highest temperature, and so on, during the jth time unit, then whether a new agricultural product can be produced under certain climates is again dependent on the extreme

rainfalls, temperatures, etc.—that is, Z, or W,. The present author lived

through this argumentin two different climates with two different products: the widespread production of rice in Hungary, for which a minimum amountof rainfall per week is needed (that is, decision is in terms of W, with time unit a week), and the production of potatoes in West Africa, where high daily temperatures were the problem (that is, decision is in terms of Z, with time unit one day). Thetranslation of the other examples to our mathematical modelis evident. In caseoffailure of a piece ofequipment, the fault-free operation of the equipmentlasts for W, time units, where X; is the time to failure of the jth component(or jth group of components) and n is the number of components (or of groups of components). On the other hand, the service time is . Z, where X; is the time needed forservicing thejth component andz is the number of components to be serviced. In the corrosion model, if the

surface has n pits and X, is the depth of thejth pit at a given time, then our interest is P(Z, ,...,X, are dependent. The only exception can be the theoretical concept ofstatistical samples when one assumes that the experimenteris free to choose his data and therefore he does so independently at each observation. We would like to coverall practical situations, however, hence the following discussion is not representative of our aim and method, but will serve as an approximation and guide in several situations. Let X,,X3,...,X, be independent andidentically distributed (i.i.d.) random variables. With the notations Z, = max(X,,X>,..-,X_)s

W,, =min(X,,X>,...,X,) and

F(x) = P(X;< x),

formulas (4)-(8) now reduce to P(Z,< x)= H, (x)= F"(x)

(9)

and P(W, > x)=1—L,(x)=(1— F(x)”.

(10)

Wegive below two simple approximations to H,(x) and L,(x). These

will serve as guides—one for obtaining limit theorems, the other for obtaining estimates for dependent systems. We start with two lemmas.

8

INTRODUCTION TO THE UNIVARIATE CASE

13

Lemma 1.3.1. For any 0 x1 < j0)=1. Thus +00

0< E(X,)= [O°xdF(x)= & [i xar(s)< ¥ KF) F(e-0)} which, in view of

F(k)— F(k—1)=[1-F(k-1)]-[1-F(k)],

1.6

SPECIAL PROPERTIES OF THE EXPONENTIAL DISTRIBUTION

37

can be rearranged as

0< E(X,)< 3 [1-F(k-D)]. k=l

But, by (10) and (67a),

1- F(k-1)=[1-F(1)]}*"'=G(1)*"', and thus +00

00, or, F(x)=0 for all x>0, which is not a proper distribution function. This completes the proof. A Theorem 1.6.2 is of theoretical value only, since it requires to check (72) or (67b) for infinitely many values of n. It is, however, significant because it shows that a very simple property of the minimum determines the distribution of the population.It will also serve as a major tool in proving another practical theorem in Section 3.12.1, which also leads to the exponential distribution. The condition expressed in (72) is a very sensitive tool for determining the population distribution. Indeed, if we take the condition that, for all nol,

(n+1E(W, )=E(X,),

(74)

where E(X,) is assumed finite, we can then conclude that F(x)=x/a for

0 1} uniquely determines the

distribution of the population. A similar statement is true with E(Z,) or with more general sequences (see Exercises 23 and 24). The simple property of lack of memory turned out to be equivalent to a distributional property (67b) of the minimum, which is unique for the exponential distribution. We shall now turn to the maximum andestablish another type of unique property of the exponential variates. Let X,,X>,...,X, be iid. random variables with common distribution function F(x). Assumethat F(x) is differentiable and let f(x) = F’(x). Let

38

INTRODUCTION TO THE UNIVARIATE CASE

1.6

Xin SXq,,5 °° SX,,, be the order statistics of the X, We can determine the joint density function of all orderstatistics X,.,, 10.

(79)

On the other hand, by the independence of U, and U, and bythe choice of w, for any e>0, P(U, > w—)P(U,>&)= P(U, >w—e, U,>e)

< P(X, ., 2 0—&,X2., >)

w

forsome /)

< > P(X; >w)=0. j=l

In view of (79), we thus have that, for any e>0, P(U,>0)= P(X>.,—X iin > €) =O.

Such relation is possible only for degenerate distributions, which are excluded by the assumption that F(x) is continuous. Hence, our claim of F(x)2. We can now easily arrive at the conclusion of the present theorem by looking at an arbitrary integral in terms of T7,, j>2. Let C be an (n—1)-dimensional set and

INTRODUCTION TO THE UNIVARIATE CASE

40

1.6

consider the probability that A={T>,T3,...,T,) EC}.

Since the 7), j > 2, are independentof U,,

P(A)=P(A|U,=1)= f aP(T,0 of constants such that, as n—> 00, lim H,(a, + b,x) = H (x)

(6)

lim L,(c, + d,x) = L(x)

(7)

and

exist for all continuity points of H (x) and L(x), respectively, where H (x) and L(x) are nondegenerate distribution functions. 51

52

WEAK CONVERGENCE FORI.1.D. VARIABLES

2.1

Such convergence will be called weak convergence of distribution func-

tions or of random variables. That is, a sequence U, of random variables,

or their distribution functions R,(x), are said to converge weakly if, as n— +o, lim R,(x)= R(x) exists for all continuity points x of the limit R(x). With this term, our aim in this chapter is therefore to find conditions on F(x) under which Z, (or W,) can be normalized by constants a, and 5, >0 so that (Z, — a,)/b, (or (W, —c,)/d,) converges weakly to a nondegenerate distribution. In view of (4) and (5), the expressions (6) and (7) are equivalent to

limF"(a, + b,x) = H(x)

(6a)

lim{1 — F(c, + d,x)f" = 1— L(x),

(7a)

and

where the limits are for n—»+ oo and in the sense of weak convergence.

_ Corollaries 1.3.1 and 1.3.2 contain an answer to our problem. In a set of

theorems, we now make these corollaries more specific. Namely, we shall

give rules for the construction of the sequences a,, b,, c,, and d, as well as find criterions for F(x) under which (6a) and (7a) hold. In addition, we shall make H (x) and L(x) explicit. Before starting.with this program,let us introduce two notations which will be used throughout this book. We say that a(F), defined as

a(F)=inf{ x: F(x)>0},

(8)

is the lower endpoint of the distribution function F(x). Similarly, the upper endpoint w(F) of F(x) is defined by

w(F)=sup{x: F(x) 0, a(F)=0 and w(F) = + 00. If F(x)is the distribution function of a random variable, taking the values 0 and | only, then a(F)=0 and w(F)=1. As a final example, we take F(%)=1/(1+e ~~), for which a(F)= —0o and w(F)= + 00. The notations (1)-(9) will be used repeatedly without any further reference to their location. The reader is advised to become familiar with them.

2.1

SUFFICIENT CONDITIONS

53

2.1. LIMIT DISTRIBUTIONS FOR MAXIMA AND MINIMA:SUFFICIENT CONDITIONS As in the previous chapter, we state separately theorems for maxima and minima. However, recall that theorems on maxima and minima are equivalent, as emphasized at the end of Section 1.2. Indeed, we shall obtain results on the minimum from those on the maximum by transforming the random variables X; into (— X,), by which maximum becomes minimum and vice versa. Wefirst state three theorems on the maximum and then give proofs. Their counterparts on the minimum follow afterward. Theorem 2.1.1.

Let w(F)= +00. Assume that there is a constant y>0

such that, for all x >0, as t+ 00,

F

1— F(tx)

ae

(10)

lim ————— i-F() mx,

Then there is a sequence b, >0 such that, as n— + 0,

lim P(Z, < b,x) = H,,, (x), where

Hee

ty)

exp(—x~’)

i

if x>0,

otherwise.

1

ay)

The normalizing constant b, can be chosen as

by mint x:1—-F(x)0, satisfies condition (10) of the preceding theorem. Then there are sequences a, and b, >0 such that, as n—»+ 0, lim P(Z, 0,

Hz,(x)= exp(—(-x)") ifx0. A Proof of Theorem 2.1.3. For the proof we choose a, and b, by (19) and (20), respectively. We shall follow the method of proof of Theorem 2.1.1, which in turn was reduced to Corollary 1.3.1. Observing that a, of (19) tends to w(F) as n—>» + 00, (17) implies that, as n> + 00,

1— F(a,+5,x)

um = F(a)

e*

(25)

for all x. Therefore, for arbitrary x, as n— + 00,

limn(1— F(a, +5,x))=limn(1— F(a,))

1—F(a,+5,x)

1— F(q,)

=e -*limn[ 1—F(a,)]. Consequently, if we show that, as n— + 00,

limn(1—F(a,))=1,

(26)

Corollary 1.3.1 immediately yields Theorem 2.1.3. For proving (26), we write the definition of a, in (19) in detail, and, by a trick, we appeal to (25). Moreprecisely, by (19)

1— F(a, +0) 0

such that, for all x >0, as t-»— ©,

lim

F(tx)

F(t)

mx,

(27)

Then there is a sequence d,>Q such that, as n—-»+ 0, lim P( W, < d,x) iz Ly,,(x),

where

Lim { 1—exp(—(—x)~’) if x0.

The normalizing constant d, can be chosen as

d,=sup{ x:F(x)