Probablistic Models in Operations Research [1 ed.] 9781608765331, 9781606922330

Citation preview

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

PROBABILISTIC MODELS IN OPERATIONS RESEARCH

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

No part of this digital document may be reproduced, stored in a retrieval system or transmitted in any form or by any means. The publisher has taken reasonable care in the preparation of this digital document, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained herein. This digital document is sold with the clear understanding that the publisher is not engaged in rendering legal, medical or any other professional services.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

PROBABILISTIC MODELS IN OPERATIONS RESEARCH

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

SILVIU GUIASU

Nova Science Publishers, Inc. New York

Copyright © 2009 by Nova Science Publishers, Inc.

All rights reserved. No part of this book may be reproduced, stored in a retrieval system or transmitted in any form or by any means: electronic, electrostatic, magnetic, tape, mechanical photocopying, recording or otherwise without the written permission of the Publisher. For permission to use material from this book please contact us: Telephone 631-231-7269; Fax 631-231-8175 Web Site: http://www.novapublishers.com NOTICE TO THE READER The Publisher has taken reasonable care in the preparation of this book, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained in this book. The Publisher shall not be liable for any special, consequential, or exemplary damages resulting, in whole or in part, from the readers’ use of, or reliance upon, this material. Any parts of this book based on government reports are so indicated and copyright is claimed for those parts to the extent applicable to compilations of such works. Independent verification should be sought for any data, advice or recommendations contained in this book. In addition, no responsibility is assumed by the publisher for any injury and/or damage to persons or property arising from any methods, products, instructions, ideas or otherwise contained in this publication. This publication is designed to provide accurate and authoritative information with regard to the subject matter covered herein. It is sold with the clear understanding that the Publisher is not engaged in rendering legal or any other professional services. If legal or any other expert assistance is required, the services of a competent person should be sought. FROM A DECLARATION OF PARTICIPANTS JOINTLY ADOPTED BY A COMMITTEE OF THE AMERICAN BAR ASSOCIATION AND A COMMITTEE OF PUBLISHERS.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

LIBRARY OF CONGRESS CATALOGING-IN-PUBLICATION DATA Available upon request

ISBN 978-1-60876-533-1 (E-Book)

Published by Nova Science Publishers, Inc.

New York

Contents

i

Contents Introduction

1

Chapter 1. Game theory 1. Generalities . . . . . . . . . 2. Pure strategies . . . . . . . 3. Mixed strategies . . . . . . . 4. Graphical method . . . . . . 5. Brown’s algorithm . . . . . 6. The linear programs induced 7. Computer tips . . . . . . . . 8. Prisoner’s dilemma . . . . . 9. n-person games . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . by a two-person zero-sum game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Chapter 2. Decision theory 1. Generalities . . . . . . . . . . . . 2. Oil-drilling example . . . . . . . . 3. Decision without experimentation 4. Decision with experimentation . . 5. Risk function . . . . . . . . . . . 6. Computer tips . . . . . . . . . . . 7. Multiple-stage decision . . . . . .

3 3 5 14 18 33 37 47 60 63

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

73 73 74 77 84 90 94 109

Chapter 3. Simulation 1. Generalities . . . . . . . . . . . . . . . . . . . . . . 2. Minitab . . . . . . . . . . . . . . . . . . . . . . . . 3. Random numbers . . . . . . . . . . . . . . . . . . . 4. Sampling from discrete probability distributions . . 5. Sampling from continuous probability distributions 6. Monte Carlo method . . . . . . . . . . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

117 117 119 125 127 142 159

Chapter 4. Reliability 1. Structure function . . 2. Exact reliability . . . 3. Bounds for reliability 4. Lifetime . . . . . . .

. . . .

. . . .

. . . .

. . . .

190 190 193 200 203

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . . . . .

. . . .

. . . . . . .

. . . .

. . . . . . .

. . . .

Chapter 5. Queueing systems 1. Generalities . . . . . . . . . . . . . . . 2. Steady state condition . . . . . . . . . 3. M/M/1 with constant mean arrival rate service rate . . . . . . . . . . . . . . . . .

. . . . . . .

. . . .

. . . . . . .

. . . .

. . . . . . .

. . . .

. . . . . . .

. . . .

. . . . . . .

. . . .

. . . . . . .

. . . .

. . . .

. . . . . . . . . . . . . . . . . . . . . . and constant mean . . . . . . . . . . .

216 216 220 225

Contents

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

4. M/M/s with constant mean arrival rate and constant mean service rate per busy server . . . . . . . . . . . . . . . . . . . 5. Waiting time . . . . . . . . . . . . . . . . . . . . . . . . . 6. Finite queue . . . . . . . . . . . . . . . . . . . . . . . . . . 7. Finite input source . . . . . . . . . . . . . . . . . . . . . . 8. Other queueing systems . . . . . . . . . . . . . . . . . . . 9. Number of arrivals during the time interval [0,t] . . . . . . 10. Computer tips . . . . . . . . . . . . . . . . . . . . . . . . 11. Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . .

ii 230 234 246 257 262 267 270 278

Appendix: The dual theorem

281

References

286

Index

289

Introduction

1

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

INTRODUCTION Operations Research, often abbreviated as OR, is a relatively new discipline. Its main aim is to optimize mathematical models of reality. As the real world is very complex, in building up mathematical models we generally make a lot of simplifying assumptions. Therefore, the results provided by Operations Research have only a limited validity and must be considered only as approximations of what really is going on. It is surprising, however, to see that often these approximations are accurate and reliable when they are applied to real life problems. Operations Research is a branch of Applied Mathematics based on optimization. Its objective is to choose the best solution out of a set of feasible solutions, where ‘the best’ means maximizing or minimizing an objective function like profit or cost, for instance. There are some very good textbooks on Operations Research available in bookstores and libraries. All of them are big books, containing more than 500 or even 1000 pages in large format, with a lot of details that can rather discourage a potential reader. It is obvious that there is a need for a user friendly book on Operations Research, using fewer pages, a simple approach, and focusing on the main topics and ideas in this field without compromising, however, the depth of the basic results or the necessary balance between the theoretical justification, the numerical examples, and the computer applications. The topics of Operations Research may be grouped in two large categories: the strictly deterministic models and the probabilistic models. The first category, including linear programming, integer programming, dynamic programming, transportation models, networks, and inventory problem, is well covered in many OR books existing on the market. These topics, basically requiring only linear algebra as a prerequisite, are generally taught in a third year undergraduate full academic course. The probabilistic models, however, are more complex and use previous knowledge from linear programming, multivariate calculus, probability theory, and statistical inference. The aim of this book is to cover in a condensed but rigorous way the main chapters of the OR probabilistic models based on optimization. The topics are taken from Game Theory, Decision Theory, Simulation, Reliability, and Queueing Theory. Game Theory deals with how to find the best winning strategies in a confrontation between two players with opposite interests and how to evaluate the value of different possible coalitions in a game involving more than two players. In Decision Theory the main objective is to find the optimum actions of the decision maker for minimizing his loss when

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Introduction

2

he has limited information available and faces uncertainty about the future possible events. Simulation is a powerful instrument for obtaining representative samples, of any length, from probability distributions of interest and for approximating multiple definite integrals and the solutions of linear and nonlinear equations and programs. In the chapter dealing with Reliability, the main problem is how to evaluate the probability that a system functions properly, or is still alive, during an arbitrary time interval. Finally, Queueing Theory deals with how to assess what may happen in a complex system where the customers arrive randomly, wait in line, and later are served. A main concept is the waiting time in the queueing system, waiting to be served and being served. In each chapter, some representative examples and the mathematical formalism are discussed first, eventually followed by the comments on ramifications, generalizations of the basic results, and assignment problems. The text is based on lectures delivered for more than twenty-five years in an Operations Research II fourth-year undergraduate full course, cross-listed with two graduate half courses, in the Department of Mathematics and Statistics at York University in Toronto. It is not really a textbook but rather a monograph on the probabilistic models in OR. The book claims to be self-contained, compact, and accessible. It contains also new original topics and proofs not covered in the standard textbooks. Elementary prerequisites from linear programming, multivariate calculus, probability theory, and statistics are kept to a minimum. Each chapter ends with hints about computation. Bibliographic references refer to books and papers used by the author or recommended to the reader for a further study. This book tries to preserve the direct, straightforward approach a teacher is using in class while lecturing. Its main aim is to cover the basic results from the main probabilistic chapters of Operations Research following a simple approach. It is, however, only an introduction into this field and the reader is encouraged to subsequently look into more specialized books, depending on his interest. Almost each chapter of what we call today Operations Research has very specialized monographs dealing only with that specific topic. The structure of each chapter is basically the same. We start with some representative examples that allow us to get familiar with the specific optimization problem, notation, and terminology. We switch to the general way of solving such problems and eventually we go back to the original examples in order to get and interpret the final answers. Computer tips and comments are inserted when needed, together with a discussion of necessary mathematical prerequisites. The chapters are kept to be as independent as possible. The author expresses his thanks to Nova Science Publishers, and especially to Maya Columbus, Lorna Loperfido, Michael Knespler, Susan Boriotti, Donna Dennis, and Ellen Stavro, for printing this book.

Game Theory

3

Chapter 1. GAME THEORY

ABSTRACT

This chapter deals with the analysis of a two-person zero-sum game, involving two players with opposite interests, competing against each other for getting a bigger payoff. The basic concepts here are: the maximin and minimax pure and mixed strategies, the equilibrium point of the game, and the expected payoffs. Solving a game means to find the optimum strategies of the players and the corresponding optimum expected payoffs. The graphical method is used mainly for visualizing the main concepts even if this method may be used only when at least one player has only two possible pure strategies. For an arbitrary two-person zero-sum game, Brown’s algorithm is used for approximating the optimum solution, and the linear programming is brought in for getting the exact optimum solution. Computer tips are given involving the computer packages: Lindo, Sas-or, Excel, and Mathematica. Prisoner’s dilemma shows what happens when the players have not only opposite interests but common ones as well. A final section deals with what happens in games with more than two players involved, where the concept of coalition plays an essential role.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

1. Generalities 1. A brief history. Game theory has been the first domain of applied mathematics in which human beings have played the central role, as entities capable of taking rational decisions. The games of chance have been the first models of the theory. Le Chevalier de M´er´e approached Blaise Pascal (1623-1662) with two questions about gambling. The first question asked for the minimum number of rollings of a pair of fair dice which would guarantee a probability larger than 1/2 of getting a 6 on each of the two dice. The second question was referred to betting in order to find out how much a stake in a game would be worth if the game were interrupted at a given stage in the proceedings. To be more specific, two players repeatedly play a game of chance. Each player

Game Theory

4

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

has equal chances of winning in each play of the game. At the beginning of the game, both players put down the same amount of money. The total sum is given to the player who first wins n plays of the game. Assume that the game stops before its end, with the two players winning a and b plays of the game, respectively, where a < n and b < n. How to fairly share the total amount between the two players? The correspondence between Pascal and Pierre de Fermat (1601-1665) on this problem and on related questions led to the birth of probability theory. It was Pascal who invented a pattern for structuring decision problems in which one has incomplete information. In the modern game theory the presence of a competitor, an intelligent opponent, becomes ´ essential. In 1921, Emile Borel (1871-1956) published a seminal paper in which game theory was presented using a mathematical language and two problems were clearly stated: For particular classes of games, does a winning (or nonlosing) strategy exist? If it exists, how does the player discover the strategy? He also suggested that game theory had certain analogies with problems of military strategy and economics, but warned that the strategies recommended by game theory did not take into account the psychology of the adversary. In 1928, in a paper on the theory of parlor games, John von Neumann provided answers to Borel’s questions and proved the famous ‘minimax theorem’. A collaboration between von Neumann and the economist Oskar Morgenstern led to the publication of the seminal book ‘Theory of Games and Economic Behavior’ in 1944, in which the theory of games was presented as the first serious step toward a comprehensive mathematical economics, providing a technique for making ‘good’ decisions from the point of view of one’s own self-interest when faced with other ‘players’ whose interests may conflict with one’s own. Since 1944, game theory has become a vigorous chapter of operations research. The only fairly complete part of the theory deals with two-person zero-sum games. The general theory for n-person noncooperative or cooperative games is far from complete although various approaches have been proposed. Game theory has influenced the development of mathematical theories concerning economics and other social sciences. 2. An example: Two-finger mora game. To give a simple example, let us consider the following, so called, two-finger mora game: Mary and John show simultaneously and independently one finger or two fingers. If the two numbers match, then Mary gets $1 from John; if the two numbers do not match, then John gets $1 from Mary. How are supposed the players to play the game, once or repeatedly, in order to maximize the own payoff? Unfortunately, in spite of its simplicity, we cannot solve yet the two-finger mora game given above. Its solution will be given later. This example, however, allows us to see the essential elements that define a two-person zero-sum game. First, we have two players involved. Mary is ‘Player I’ and John ‘Player II’. Each player has two

Game Theory

5

available ‘pure strategies’ to choose from, namely: ‘show one finger’ and ‘show two fingers’. The two players act independently and simultaneously. A pair of strategies chosen by the players is a ‘variant’ of the game. There are two pure strategies for each player and four variants of the game. The outcome of each variant of the game is a pair of numbers representing the ‘payoff’ for each player, respectively. This game is ‘zero-sum’ because the sum of the payoffs of the two players is equal to zero for each possible variant of the game. This simply means that the two players have opposite interests and the win for a player means the loss, by the same amount, for the other player. 2. Pure strategies 1. Terminology and notations. A game describes the interaction between rational entities, called players, having generally opposite interests and trying to choose their strategies in order to maximize their own payoff. A pure strategy of a player is an action, or a series of actions, chosen by that player. A variant of the game consists of a set of pure strategies, one for each player engaged in the game. The players choose their strategies independently and simultaneously. Each variant of the game results in a payoff for each player. The game is played once or several times. A two-person game is characterized by the set X = {x1 , . . . , xr } of ‘pure strategies of Player I’, the set Y = {y1 , . . . , ys } of ‘pure strategies of Player II’, and a ‘utility’ or ‘payoff’ function u(k) : X × Y −→ R1 , (k = 1, 2), for each player, where R1 is the set of real numbers. The number u(k) (xi , yj ) represents the ‘utility’ or ‘payoff’ for player k induced by the ‘variant of the game’ consisting of the pure strategies xi and yj chosen by the two players, (k) respectively. Often, we denote u(k) (xi , yj ) by uij . Such a game is abbreviated as {X, Y, u(1) , u(2) }. A two-person game is a ‘zero-sum game’ if: u(1) (xi , yj ) + u(2) (xi , yj ) = 0,

(i = 1, . . . , r; j = 1, . . . , s),

or, equivalently: (1)

(2)

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

uij + uij = 0,

(i = 1, . . . , r; j = 1, . . . , s).

In such a case, we denote by u the payoff function of the first player, i.e., u = u(1) , in which case u(2) = −u. In such a game the interests of the two players engaged in the game are opposite: the amount won by a player equals the amount lost by the opponent. A two-person zero-sum game is abbreviated as {X, Y, u}. Denoting uij = u(xi , yj ), the utility or payoff function u is defined by the utility or payoff matrix u = [uij ]. The objective of each player is to maximize his own payoff. Solving the game means finding the optimum strategies for each player, i.e., the strategies

Game Theory

6

that guarantee the maximum own payoff, and the value of the game, i.e., the payoff corresponding to the optimum strategies, for each player. Going back to the two-finger mora game defined in the previous section, we notice that this is a two-person zero-sum game with X = Y = {1, 2}, where 1 is the label of the pure strategy ‘show one finger’, and 2 is the label of the pure strategy ‘show two fingers’. The payoff matrix for Mary (Player I) is: II 1

2

1

−1

2 −1

1

1 I

and the payoff matrix for John (Player II) is: II 1

2

1 −1

1

I 2

−1

1

2. Dominated strategies. Let {X, Y, u} be a two-person zero-sum game. For Player I, a pure strategy xi1 ‘dominates’ the pure strategy xi2 , or xi2 ‘is dominated by’ xi1 , if ui1 j ≥ ui2 j for all j = 1, . . . , s. Similarly, for Player II, a pure strategy yj1 dominates the pure strategy yj2 , or yj2 is dominated by yj1 , if −uij1 ≥ −uij2 , or equivalently uij1 ≤ uij2 , for all i = 1, . . . , r. An intelligent player never plays a dominated strategy. Therefore, the dominated strategies have to be eliminated from the game. This helps to simplify the game. Sometimes the solution of the game may be obtained by simply eliminating the dominated strategies.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Problem 1: In a two-person zero-sum game, the payoff matrix of Player I, in thousands dollars, is given below. Solve the game.

I

1 1 0 2 1 3 1

II 2 1 0 2

3 −1 5 4

Solution: This game may be solved by simply eliminating the dominated strategies. For Player I ‘better’ means a larger entry in the payoff matrix. There is no domination between his pure strategies 1 and 2 because if Player

Game Theory

7

II plays the pure strategy 1, then, for Player I, his pure strategy 2 is better than his pure strategy 1, but if Player II plays the pure strategy 2, then, for Player I, his pure strategy 1 is better than his pure strategy 2. Comparing the pure strategies 1 and 3 of Player I, we can easily see that 3 dominates 1 because, whatever Player II does, Player I can do better by playing his pure strategy 3. We eliminate the first row of the payoff matrix. The simplified game becomes: II 1 2

3

2 1

0

5

3 1

2

4

I

For Player II ‘better’ means a smaller entry in the payoff matrix of Player I. There is no domination between the pure strategies 1 and 2 of Player II. Indeed, if Player I plays his pure strategy 2, then, for Player II, his pure strategy 2 is better than his pure strategy 1 but, if Player I plays 3, then, for Player II, his pure strategy 1 is better than his pure strategy 2. Whatever Player I does, however, the pure strategy 1 of Player II is better than his pure strategy 3. Thus, for Player II, 1 dominates 3. Player II’s dominated strategy 3 is eliminated, and the simplified game becomes: II 1

2

2

1

0

3

1

2

I

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Under the new circumstances, for Player I, the pure strategy 3 dominates 2. We eliminate the pure strategy 2 of Player I. The payoff matrix of the game becomes: II

I

3

1

2

1

2

Under the new circumstances, for Player II, the pure strategy 1 dominates 2. We eliminate the pure strategy 2 of Player II. Finally, the simplified game has the payoff matrix of Player I:

Game Theory

8 II 1 I

3

1

Obviously, the only solution left is the optimum solution of the game. Thus, the optimum pure strategies of the game are: 3 for Player I, and 1 for Player II. The optimum outcome of the game is a gain of $1,000 for the first player and a loss of $1,000 for the second player. At first sight it seems somewhat ridiculous to say that a loss of $1,000 could be considered an optimum payoff for Player II but if he chooses the pure strategy 2 or the pure strategy 3, he will do even worse, losing $2,000 or $4,000, respectively. 3. Equilibrium point. Let {X, Y, u} be a two-person zero-sum game. If Player I chooses the pure strategy xi , then he cannot prevent Player II from choosing his own pure strategy in order to minimize Player I’s payoff. Therefore, Player I can count only on getting minj uij . But, as far as he is concerned, he can choose his own pure strategy in order to maximize the minimum payoff allowed by Player II. Thus, Player I can force getting the payoff: u = max min uij , i

j

called the ‘lower value’ of the game. On the other side, if Player II chooses his own pure strategy yj , then he knows that he cannot prevent Player I from choosing a pure strategy in order to maximize his own payoff. Therefore, he knows that Player I can get a payoff equal to maxi uij . But, as far as he is concerned, he can choose his own pure strategy in order to minimize the maximum payoff Player I can get. Thus, Player II can impose the payoff of Player I to be: u = min max uij , j

i

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

which is called the ‘upper value’ of the game. Proposition 1: In a two-person zero-sum game {X, Y, u}, ‘maximin’ cannot exceed ‘minimax’, i.e.: max min uij ≤ min max uij . i

j

j

i

Proof: Obviously: A(i) = min uij ≤ uij ≤ max uij = B(j). j

i

Game Theory

9

As A(i) ≤ B(j) for all i, j, we have: A(i) ≤ min B(j) = u, j

for all i. Therefore: u = max A(i) ≤ u. 2 i

Remark: The inequality from the above proposition justifies the names ‘lower value’ and ‘upper value’ of the game given to u and u, respectively. 2 A pair (xi0 , yj0 ) of pure strategies of the two players is an ‘equilibrium point’ (or ‘saddle-point’) of the game if the double inequality: uij0 ≤ ui0 j 0 ≤ ui0 j is true for all values of i, j. Obviously, in the payoff matrix of Player I, the number ui0 j 0 is the smallest entry in its row and the largest entry in its column. Proposition 2: If the two-person zero-sum game {X, Y, u} has a saddlepoint (xi0 , yj0 ), then: u = u = ui0 j0 . Proof: Taking into account the definition of the saddle-point, we have: u = min max uij ≤ max uij0 = ui0 j0 = j

i

i

= min ui0 j ≤ max min uij = u. j

i

j

As Proposition 1 tells us that u ≤ u, we have proved that: u = u = ui0 j0 .

2

Proposition 3: If in a two-person zero-sum game {X, Y, u} the lower value of the game is equal to the upper value of the game, i.e., u = u, then the game has a saddle-point.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Proof: Let xi0 be a pure strategy of Player I for which: min ui0 j = max min uij = u. j

i

j

Let yj0 be a pure strategy of Player II for which: max uij0 = min max uij = u. i

j

i

Therefore: u = min ui0 j ≤ ui0 j0 ≤ max uij0 = u. j

i

Game Theory

10

But, as u = u, by hypothesis, from the above inequalities we get: max uij0 = ui0 j0 = min ui0 j , j

i

implying: uij0 ≤ ui0 j 0 ≤ ui0 j , for all values of i and j, which shows that (xi0 , yj0 ) is a saddle-point of the game. 2 Problem 2: In a two-person zero-sum game, the payoff matrix of Player I, in thousands dollars, is given below. Solve the game.

I

1 1 −3 2 5 3 2

II 2 3 −2 6 −2 −4 0 2

Solution: There are no dominated strategies. We are looking for an equilibrium point. In the payoff matrix of Player I we introduce a new column, containing the minimum entry in each row of the initial payoff matrix and a new row, containing the maximum entry in each column of the initial payoff matrix:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

I

1 1 −3 2 5 3 2 max 5

II 2 3 min −2 6 −3 −2 −4 −4 0 2 0 0 6

We can see that the lower value and the upper value of the game are equal, namely, u = u = 0. According to Proposition 3, the game has an equilibrium point, (3, 2). The optimum solution of the game is: the pure strategy (maximin strategy) 3, for Player I, and the pure strategy (minimax strategy) 2, for Player II. The optimum value of the game is u∗ = 0, for Player I, and −u∗ = 0, for Player II. Such a game, in which the value of the game is the same for both players, is called a ‘fair game’. 2 The rationale behind playing the maximin (for Player I) and minimax (for Player II) strategies in a two-person zero-sum game that has an equilibrium point (xi0 , yj0 ) is contained in the following comments: (a) According to the maximin strategy, Player I wants to maximize the minimum payoff imposed by Player II. According to the minimax strategy,

Game Theory

11

Player II wants to minimize the maximum payoff Player I can get. Player II cannot prevent Player I from getting the payoff ui0 j0 . Player I cannot prevent Player II from getting the payoff −ui0 j0 . From this cautious point of view, the outcome of a game that has a saddle-point in terms of pure strategies is predictable. If the two-person zero-sum game has a saddle-point, then the maximin strategy xi0 for Player I and the minimax strategy yj0 for Player II are called ‘optimum strategies’. The number u∗ = ui0 j0 is the ‘value of the game for Player I’. The ‘value of the game for Player II’ is −u∗ . (b) If only one player gives up his maximin/minimax strategy, then the outcome of the game for him can only be worse. Indeed, looking at the payoff matrix from Problem 2, if Player II does not play the minimax strategy 2 and chooses 1 instead while Player I sticks to his maximin strategy 3, then Player II will lose $2000 instead of losing nothing. Similarly, if Player I prefers to play 2, instead of his maximin strategy 3, but Player II sticks to his minimax strategy 2, then Player I will lose $2000 instead of losing nothing in the fair game. (c) If both players give up their maximin/minimax strategies, then the outcome of the game is unpredictable. Indeed, looking again at the payoff matrix from Problem 2, if both players give up their optimum strategies and Player I decides to play 1, for instance, then he cannot predict the outcome of the game because the pair of pure strategies (1, 1) will make Player II a winner whereas (1, 3) will make Player I a big winner.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Remark: Obviously, the maximin strategy of a player in terms of his own payoff matrix is equivalent to his minimax strategy in terms of the payoff matrix of his opponent. Equivalently, the maximin strategy of a player in terms of the payoff matrix of his opponent is equivalent to his minimax strategy in terms of his own payoff matrix. 2 In the above problems the payoff matrix of the corresponding game was given. Sometimes, however, we have to get it from the indirect available information about the respective game, like in the following problem. This problem is also important because it shows that the optimum solution of a two-person zero-sum game is not necessarily unique and that if a game may be solved by simply eliminating the dominated strategies, then we obtain some optimum solutions of the game but we can lose some other equivalent optimum solutions as well, whereas looking for equilibrium points we get all the optimum solutions. Therefore, we always have to start by looking for domination and eliminate the dominated strategies, simplifying the respective game, but if the game may be solved by simply eliminating the dominated strategies, then we have to look also for equilibrium points in order to be sure that all the optimum solutions of the game are obtained.

Game Theory

12

Problem 3: In a big factory, the negotiations about a wage increase between the union and management have reached an impasse. At this stage, the management has made an offer of $0.90 increase per hour while the union is asking for a $1.30 increase per hour. The management refuses to go up and the union does not want to go down from their last figures. An impartial arbitrator is brought in who asks each side to reconsider its last proposal and submit to him a new proposal for the wage increase per hour, rounded to the nearest dime. Both sides know that the arbitrator accepts the new proposal of the side that gives most (with the union going down and the management going up) from its previous figure. If neither side changes its previous figure, or if they both give in by the same amount, then the arbitrator compromises halfway between, which is ($0.90+$1.30)/2=$1.10 in this case. The management and the union have to determine what new wage increase to submit to the arbitrator for their own maximum advantage. Formulate the problem as a two-person zero-sum game and solve the game. Solution: Assume that the union is Player I and the management is Player II in this game. The pure strategies of Player I are the following possible decreases: $0.00 (no change of its previous offer of $1.30 wage increase), $0.10, $0.20, $0.30, $0.40 (agreement with the previous proposal of $0.90 wage increase made by the management). The pure strategies of Player II are the following possible increases: $0.00 (no change of its previous offer of $0.90 wage increase), $0.10, $0.20, $0.30, $0.40 (agreement with the previous proposal of $1.30 wage increase made by the union). The payoff matrix of Player I is given below. It contains the final wage increase per hour (in dollars) for the employees from the respective factory, as decided by the arbitrator.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

I

0.00 0.10 0.20 0.30 0.40

0.00 1.10 1.20 1.10 1.00 0.90

0.10 1.00 1.10 1.10 1.00 0.90

II 0.20 1.10 1.10 1.10 1.00 0.90

0.30 1.20 1.20 1.20 1.10 0.90

0.40 1.30 1.30 1.30 1.30 1.10

If we eliminate the dominated strategies of Player I, the reduced payoff matrix of Player I becomes:

I

II 0.00 0.10 0.20 0.30 0.40 0.10 1.20 1.10 1.10 1.20 1.30

If we eliminate now the dominated strategies of Player II, the reduced payoff matrix of Player I is:

Game Theory

13 II I

0.10 0.10 1.10

0.20 1.10

and, therefore, the optimum solutions of the game are (0.10,0.10) and (0.10,0.20), corresponding to the $1.10 wage increase per hour. If we are looking for equilibrium points of the game, we obtain:

I

0.00 0.00 1.10 0.10 1.20 0.20 1.10 0.30 1.00 0.40 0.90 max 1.20

II 0.20 1.10 1.10 1.10 1.00 0.90 1.10

0.10 1.00 1.10 1.10 1.00 0.90 1.10

0.30 1.20 1.20 1.20 1.10 0.90 1.20

0.40 1.30 1.30 1.30 1.30 1.10 1.30

min 1.00 1.10 1.10 1.00 0.90

and, therefore, the optimum solutions of the game are (0.10,0.10), (0.10,0.20), (0.20,0.10), and (0.20,0.20), corresponding to the $1.10 wage increase per hour. 2 Does any two-person zero-sum game have an equilibrium point in terms of pure strategies? The answer is no. Indeed , if we take the two-finger mora game, we can see that: II 1

1 1

2 min −1 −1

I

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

2 −1 max 1

1 1

−1

which shows that u = −1 < 1 = u. The lack of a saddle-point makes the outcome of the game unpredictable. Indeed, if John intended to show two fingers, then it would be stupid for Mary to show one finger; she could do better showing two fingers too. But if Mary showed two fingers, it would be stupid for John to show two fingers; he could do better showing one finger. But if John showed one finger, it would be stupid for Mary to show two fingers; she could do better showing one finger too. But if Mary showed one finger, it would be stupid for John to show one finger too; he could do better showing two fingers. The lack of an equilibrium point of the game generates such a vicious circle. Each player could take advantage of any knowledge about the opponent’s way of acting. Therefore, in order to counteract such a possibility the only solution left would be to play randomly. But how?

Game Theory

14

3. Mixed strategies Let {X, Y, u} be a two-person zero-sum game. A ‘mixed’ or ‘random strategy’ of Player I is a probability distribution ξ = (ξ1 , . . . , ξr ) on the set of pure strategies X = {x1 , . . . , xr }. We have: ξi ≥ 0,

r X

ξi = 1.

i=1

Such a mixed strategy ξ = (ξ1 , . . . , ξr ) tells us that Player I plays the pure strategy x1 with probability ξ1 ,. . . , and the pure strategy xr with probability ξr . Similarly, a mixed or random strategy of Player II is a probability distribution η = (η1 , . . . , ηs ) on the set of pure strategies Y = {y1 , . . . , ys }. We have: s ηj ≥ 0,

X

ηi = 1.

j=1

Such a mixed strategy η = (η1 , . . . , ηs ) tells us that Player II plays the pure strategy y1 with probability η1 ,. . . , and the pure strategy ys with probability ηs . A pure strategy of Player I, say x1 , is a particular case of mixed strategy, namely ξ = (ξ1 , . . . , ξr ) such that ξ1 = 1, ξ2 = . . . = ξr = 0, which says that Player I plays the pure strategy x1 with certainty and does not play any other pure strategy. Similarly, a pure strategy of Player II, say y1 , is a particular case of mixed strategy, namely η = (η1 , . . . , ηs ) such that η1 = 1, η2 = . . . = ηs = 0, which says that Player II plays the pure strategy y1 with certainty and does not play any other pure strategy. The ‘mean’ or ‘expected payoff’ of the game for Player I is: U (ξ, η) =

r X s X

uij ξi ηj .

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

i=1 j=1

The mean or expected payoff of the game for Player II is −U (ξ, η). In terms of mixed strategies, we can introduce the ‘lower value’, the ‘upper value’, and the ‘saddle-point’, or ‘equilibrium point’, of the game, as we did in dealing with pure strategies. This time, however, we are using the expected payoff instead of the components of the payoff matrix of the game. Propositions 1, 2, and 3, may be also proved the same way in terms of mixed strategies using the expected payoff. The ‘lower value’ of the game is the number: U (ξ, η), U = max min η ξ

and the ‘upper value’ of the game is the number: U = min max U (ξ, η), η

ξ

Game Theory

15

where the maximum is taken with respect to all probability distributions ξ on the set X of pure strategies of the first player and the minimum is taken with respect to all probability distributions η on the set Y of pure strategies of the second player. Although the sets of pure strategies X and Y of the two players are finite, there are infinitely many mixed strategies ξ and η on X and Y , respectively. Let us notice, however, that the expected payoff U (ξ, η) is a linear function of ξ, for each η, and a linear function of η, for each ξ, on the compact sets: {(ξ1 , . . . , ξr ); ξi ≥ 0, (i = 1, . . . , r),

r X

ξi = 1},

s X

ηj = 1},

i=1

{(η1 , . . . , ηs ); ηj ≥ 0, (j = 1, . . . , s),

j=1

respectively. Therefore, both maxξ (minη U (ξ, η)) and minη (maxξ U (ξ, η)) do exist. A pair of mixed strategies (ξ ∗ , η ∗ ) of the two players is a ‘saddle-point’ or ‘equilibrium point’ of the game if: U (ξ, η ∗ ) ≤ U (ξ ∗ , η ∗ ) ≤ U (ξ ∗ , η), for all mixed strategies ξ and η of the two players. Proposition 1’: In a two-person zero-sum game {X, Y, u} we have the inequality: max min U (ξ, η) ≤ min max U (ξ, η). ξ

η

η

ξ

Proof: For all mixed strategies ξ and η of the two players, we have: A(ξ) = min U (ξ, η) ≤ U (ξ, η) ≤ max U (ξ, η) = B(η). η

ξ

Given an arbitrary mixed strategy η of the second player, we have:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

A(ξ) ≤ B(η) for all mixed strategies ξ of the first player, which implies: U = max A(ξ) ≤ B(η), ξ

for all mixed strategies η of the second player. Thus: U ≤ min B(η) = U . η

2

Game Theory

16

Remark: The inequality from Proposition 1’ justifies the names ‘lower value’ and ‘upper value’ for U and U , respectively. 2 Proposition 2’: If in a two-person zero-sum game {X, Y, u} there is a saddle-point (ξ ∗ , η ∗ ), then: U = U = U (ξ ∗ , η ∗ ). Proof: As (ξ ∗ , η ∗ ) is a saddle-point, supposed to exist, we have the double inequality: U (ξ, η ∗ ) ≤ U (ξ ∗ , η ∗ ) ≤ U (ξ ∗ , η), for all mixed strategies ξ and η of the two players. Therefore: U = min max U (ξ, η) ≤ max U (ξ, η ∗ ) = η

ξ

ξ

U (ξ ∗ , η) ≤ max min U (ξ, η) = U . = U (ξ ∗ , η ∗ ) = min η η ξ

But, according to Proposition 1’, we always have: U ≤ U . Therefore, the two values are equal and, from the last double inequality, we get: U = U = U (ξ ∗ , η ∗ ).

2

Proposition 3’: If in a two-person zero-sum game {X, Y, u} we have U = U , then there is a saddle-point (ξ ∗ , η ∗ ) of the game. Proof: As the lower value of the game is: U = max min U (ξ, η) = max(min U (ξ, η)), ξ

η

ξ

η

let ξ ∗ be a mixed strategy of Player I for which minη U (ξ, η), as a function of ξ, attains its maximum value, i.e.:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

U = min U (ξ ∗ , η). η Such a mixed strategy ξ ∗ does exist but it is not necessarily unique. As the upper value of the game is: U = min max U (ξ, η) = min(max U (ξ, η)), η

η

ξ

ξ

let η ∗ be a mixed strategy of Player II for which maxξ U (ξ, η), as a function of η, attains its minimum value, i.e.: U = max U (ξ, η ∗ ). ξ

Game Theory

17

Such a mixed strategy η ∗ does exist but it is not necessarily unique. Let us show than the pair (ξ ∗ , η ∗ ) thus obtained is a saddle-point of the game. Indeed, as by hypothesis U = U , the two equalities defining ξ ∗ and η ∗ , respectively, imply: U (ξ ∗ , η) = max U (ξ, η ∗ ). min η ξ

But, we obviously have: max U (ξ, η ∗ ) ≥ U (ξ ∗ , η ∗ ), ξ

and also: min U (ξ ∗ , η) ≤ U (ξ ∗ , η ∗ ). η

From the last equality and two inequalities, we get: max U (ξ, η ∗ ) = U (ξ ∗ , η ∗ ) = min U (ξ ∗ , η). η

ξ

But, for every mixed strategy ξ of the first player, we have: U (ξ, η ∗ ) ≤ max U (ξ, η ∗ ), ξ

and, for every mixed strategy η of the second player, we also have: U (ξ ∗ , η) ≥ min U (ξ ∗ , η). η From the last equality and two inequalities, we finally get: U (ξ, η ∗ ) ≤ U (ξ ∗ , η ∗ ) ≤ U (ξ ∗ , η),

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

for every pair of mixed strategies (ξ, η) of the two players, which shows that (ξ ∗ , η ∗ ) is indeed a saddle-point of the game. 2 As discussed when we dealt with pure strategies, in terms of mixed strategies it is also true that if there is an equilibrium point (ξ ∗ , η ∗ ) of the game then the optimum strategy for Player I is his maximin strategy ξ ∗ , whereas the optimum strategy for Player II is his minimax strategy η ∗ . The optimum mean payoff is U (ξ ∗ , η ∗ ) for Player I and −U (ξ ∗ , η ∗ ) for Player II. But we have already seen that not every two-person zero-sum game has an equilibrium point in terms of pure strategies, as it happens in the two-finger mora game, for instance. The great difference between dealing with pure strategies and dealing with mixed strategies is that every two-person zero-sum game has a saddle-point or equilibrium point in terms of mixed strategies, as stated by the following theorem:

Game Theory

18

Minimax theorem: Every two-person zero-sum game {X, Y, u} has an equilibrium point (ξ ∗ , η ∗ ) of the game.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

The ‘minimax theorem’ shows that any two-person zero-sum game has an optimum solution in terms of mixed strategies. As the mixed strategies are probability distributions on the sets of pure strategies of the players, we can see that the concept of probability distribution had to be introduced at least for making any two-person zero-sum game solvable. We can see here an analogy with solving algebraic equations. The complex numbers had to be introduced there at least for assigning n roots to every polynomial of degree n with integer coefficients. The minimax theorem was first proved by John von Neumann (1928), and later by other mathematicians, using difficult techniques, mainly based on the existence of fixed points of some operators. We are not going to follow such an approach here and, in fact, we are postponing its proof until we are going to discuss the relationship between a two-person zero-sum game and the duality from linear programming. As, according to the minimax theorem, every two-person zero-sum game has a saddle-point in terms of mixed strategies, our objective now is to find some ways for getting the optimum mixed strategies of the two players and to see how to play effectively the game in agreement with these optimum mixed strategies. John (or Johann) von Neumann (1903-1957) was born in Budapest (Hungary), the son of a rich banker, and received his mathematical training from private instructors. He taught in Berlin (1927-1929), in Hamburg (1929-1930), and in the Institute for Advanced Study, in Princeton, from 1933. He took part in numerous scientific projects connected with the war effort, such as the construction of the first big computer and the making of the atomic bomb. He was one of the most profound analysts of the 20th century and was also one of the most skilled in the application of mathematics to other sciences, like game theory and automata theory. He used to say that “Truth is much too complicated to allow anything but approximations.” His friend and colleague Stan M. Ulam mentioned that “Johnny used to say that after the age of twenty-six a mathematician begins to go downhill. When I met him he was just past that age. As time went on he extended the limit, but kept it always a little below his age.”

4. Graphical method The graphical method may be used when at least one player has only two pure strategies available to use. The method is useful mainly because it allows us to prove analytically the minimax theorem in the case when both players

Game Theory

19

have only two pure strategies available, and to visualize the expected payoff, the lower and upper values of the game, the optimum strategies of the players, and the space of feasible solutions. 1. 2 × 2 two-person zero-sum game. Let {X, Y, u} be a two-person zerosum game where each player has only two possible pure strategies available, i.e.: r = 2, X = {x1 , x2 },

s = 2, Y = {y1 , y2 },

u = [uij ], (i = 1, 2; j = 1, 2).

Let: ξ = (ξ1 , ξ2 ) and η = (η1 , η2 ) be two arbitrary mixed strategies of the two players, respectively, i.e.: ξi ≥ 0, (i = 1, 2),

ηj ≥ 0, (j = 1, 2),

ξ1 + ξ2 = 1;

η1 + η2 = 1.

The payoff tableau for Player I is: II

ξ1

x1

η1 y1 u11

ξ2

x2

u21

η2 y2 u12 u22

I The expected payoff for Player I is: U (ξ, η) =

2 X 2 X

uij ξi ηj =

i=1 j=1

= u11 ξ1 η1 + u12 ξ1 η2 + u21 ξ2 η1 + u22 ξ2 η2 . (a) Replacing ξ2 = 1 − ξ1 , the expected payoff of Player I becomes: U (ξ, η) = u11 ξ1 η1 + u12 ξ1 η2 + u21 (1 − ξ1 )η1 + u22 (1 − ξ1 )η2 = = [(u11 − u21 )ξ1 + u21 ]η1 + [(u12 − u22 )ξ1 + u22 ]η2 = = L1 (ξ1 )η1 + L2 (ξ1 )η2 ,

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

where: L1 (ξ1 ) = (u11 − u21 )ξ1 + u21 ,

L2 (ξ1 ) = (u12 − u22 )ξ1 + u22

are linear functions on the interval 0 ≤ ξ1 ≤ 1. As: η1 ≥ 0,

η2 ≥ 0,

η1 + η2 = 1,

the above equalities show us that U (ξ, η) is a linear convex combination of the linear functions L1 (ξ1 ) and L2 (ξ1 ). For the pure strategies of Player II, the expected payoff of Player I is:

Game Theory

20 η1

η2

U (ξ, η)

1

0

(u11 − u21 )ξ1 + u21

L1 (ξ1 )

0

1

(u12 − u22 )ξ1 + u22

L2 (ξ1 )

If we plot the lines L1 (ξ1 ) and L2 (ξ1 ) on the unit interval 0 ≤ ξ1 ≤ 1, due to the linearity and convexity mentioned above, once the mixed strategy of Player I is fixed by choosing an arbitrary ξ˜1 on the horizontal axis, then Player ˜ η) could be located anywhere II can choose his mixed strategy η such that U (ξ, ˜ on the vertical line passing through ξ1 , in between the lines L1 (ξ1 ) and L2 (ξ1 ) but certainly not beyond these two lines, where: ξ˜ = (ξ˜1 , ξ˜2 ), ξ˜2 = 1 − ξ˜1 . U (ξ, η) L1r u11

6

u22 r L Z 2

   

Z  Z r Z  Z u21 r Z Z r r r - ξ1 Z ∗ (0, 0) (1, 0) Z ξ1 Zr

U r

u12

Figure 1. Solving a 2 × 2 game.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

As Player II wants to minimize the expected payoff U (ξ, η), Player I cannot prevent Player II from choosing his mixed strategy η in order to reach the bottom lines. As far as Player I is concerned, he will try to maximize the minimum expected payoff imposed by Player II, by choosing his own mixed strategy ξ ∗ = (ξ1∗ , 1 − ξ1∗ ) corresponding to the highest point on the bottom lines. This will be just Player I’s maximin mixed strategy, and the corresponding value of the expected payoff will be just the lower value of the game: U = max min U (ξ, η). ξ

η

The optimum value ξ1∗ is obtained by intersecting the lines L1 (ξ1 ) and L2 (ξ1 ). We obtain: (u11 − u21 )ξ1 + u21 = (u12 − u22 )ξ1 + u22 ,

Game Theory

21

from which we get: ξ1∗ =

u22 − u21 , u11 − u12 − u21 + u22

ξ2∗ = 1 − ξ1∗ ,

and the lower value of the game is obtained by replacing the value of ξ1∗ just got into L1 (ξ1 ) (or L2 (ξ1 )), i.e.: U = L1 (ξ1∗ ) = (u11 − u21 )ξ1∗ + u21 = u11 u22 − u12 u21 . u11 − u12 − u21 + u22 (b) Replacing η2 = 1 − η1 , the expected payoff of Player I becomes: =

U (ξ, η) = u11 ξ1 η1 + u12 ξ1 (1 − η1 ) + u21 ξ2 η1 + u22 ξ2 (1 − η1 ) = = [(u11 − u12 )η1 + u12 ]ξ1 + [(u21 − u22 )η1 + u22 ]ξ2 = = L01 (η1 )ξ1 + L02 (η1 )ξ2 , where: L01 (η1 ) = (u11 − u12 )η1 + u12 ,

L02 (η1 ) = (u21 − u22 )η1 + u22

are linear functions on 0 ≤ η1 ≤ 1. As: ξ1 ≥ 0,

ξ2 ≥ 0,

ξ1 + ξ2 = 1,

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

the above equalities show us that U (ξ, η) is a linear convex combination of the linear functions L01 (η1 ) and L02 (η1 ). For the pure strategies of Player I, the expected payoff of Player I is: ξ1

ξ2

U (ξ, η)

1

0

(u11 − u12 )η1 + u12

L01 (η1 )

0

1

(u21 − u22 )η1 + u22

L02 (η1 )

If we plot the lines L01 (η1 ) and L02 (η1 ) on the unit interval 0 ≤ η1 ≤ 1, due to linearity and convexity mentioned above, once the mixed strategy of Player I is fixed by choosing an arbitrary η˜1 on the horizontal axis, then Player I can choose his mixed strategy ξ such that U (ξ, η˜) could be located anywhere on the vertical line passing through η˜1 , in between the lines L01 (η1 ) and L02 (η1 ) but certainly not beyond these two lines, where: η˜ = (η˜1 , η˜2 ), η˜2 = 1 − η˜1 . As Player I wants to maximize the expected payoff U (ξ, η), Player II cannot prevent Player I from choosing his mixed strategy ξ in order to reach the top lines. As far as Player II is concerned, he will try to minimize the maximum

Game Theory

22

expected payoff imposed by Player I, by choosing his own mixed strategy η ∗ = (η1∗ , 1 − η1∗ ) corresponding to the lowest point on the top lines. This will be just Player II’s minimax mixed strategy, and the corresponding value of the expected payoff will be just the upper value of the game: U = min max U (ξ, η). η

ξ

U (ξ, η) 6

% u11

L01 % %

u22 a L0 a2

% aa % a %raa aa % a u % 21 % r - η1 r

U r

(0, 0)

%

r %

u12 %

η1∗

(1, 0)

Figure 2. Solving the dual 2 × 2 game. The optimum value η1∗ is obtained by intersecting the lines L01 (η1 ) and L02 (η1 ). We obtain: (u11 − u12 )η1 + u12 = (u21 − u22 )η1 + u22 , from which we get: η1∗ =

u22 − u12 , u11 − u12 − u21 + u22

η2∗ = 1 − η1∗ ,

and the upper value of the game is obtained by replacing the value of η1∗ just got into L01 (η1 ) (or L02 (η1 )), i.e.:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

U = L01 (η1∗ ) = (u11 − u12 )η1∗ + u12 = =

u11 u22 − u12 u21 . u11 − u12 − u21 + u22

Therefore, we have: U = U = U∗ =

u11 u22 − u12 u21 , u11 − u12 − u21 + u22

and according to Propositions 2’ and 3’, (ξ ∗ , η ∗ ) is a saddle-point of the game. The optimum payoffs are U ∗ for Player I and −U ∗ for Player II. We have

Game Theory

23

proved, analytically, the minimax theorem for a 2 × 2 two-person zero-sum game. Problem 4: In the two-finger mora game Mary is Player I and John is Player II. The payoff matrix for Player I is: II 1 1

2 −1

2 −1

1

1 I

Solve the game. Solution: There are no dominated strategies. Looking for an equilibrium point in terms of pure strategies, we have: II 1 1

2 −1

min −1

2 −1 max 1

1 1

−1

1 I

which shows that: u = −1 < 1 = u. We are looking now for a saddle-point in terms of mixed strategies. The payoff matrix for Player I is: II

ξ1

1

η1 1 1

η2 2 −1

ξ2

2 −1

1

I

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Being a 2 × 2 two-person zero-sum game, we apply the formulas obtained above and we get: ξ1∗ =

1 (1) − (−1) = , (1) − (−1) − (−1) + (1) 2

ξ2∗ = 1 −

1 1 = , 2 2

η1∗ =

1 (1) − (−1) = , (1) − (−1) − (−1) + (1) 2

η2∗ = 1 −

1 1 = , 2 2

U∗ =

(1)(1) − (−1)(−1) = 0, (1) − (−1) − (−1) + (1)

−U ∗ = 0,

Game Theory

24

and, therefore, the game is fair. How to play the game? Each player has to use the equally likely outcomes of a probabilistic experiment. Thus, both Mary and John have to toss their own fair coin, playing the pure strategy 1 if the outcome is ‘head’ and the pure strategy 2 if the outcome is ‘tail’. Problem 5: In a two-person zero-sum game, the payoff matrix of Player I, in thousands dollars, is given below. Solve the game. 1 1 0 2 5 3 2

I

II 2 3 −2 2 4 −3 3 −4

Solution: This game may be simplified by eliminating the dominated strategies 3 for Player I and 1 for Player II. The simplified game has no saddle-point in terms of pure strategies because:

I

1 1 0 2 5 3 2 max 5

II 2 3 −2 2 4 −3 3 −4 4 2

min −2 −3 −4

which shows that: u = −2 < 2 = u, and, therefore, we have to look for a saddle-point in terms of mixed strategies. The payoff matrix for Player I is: II

ξ1

η2 2 1 −2

ξ2

2

η3 3 2

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

I 4

−3

Being a 2 × 2 two-person zero-sum game, we apply the formulas obtained above and we get: ξ1∗ =

(−3) − (4) 7 = , (−2) − (2) − (4) + (−3) 11

η1∗ = 0,

η2∗ =

ξ2∗ = 1 −

5 (−3) − (2) = , (−2) − (2) − (4) + (−3) 11

4 7 = , 11 11

ξ3∗ = 0,

η3∗ = 1 −

5 6 = , 11 11

Game Theory

25

because a dominated strategy is never used, and: U∗ =

2 (−2)(−3) − (2)(4) = , (−2) − (2) − (4) + (−3) 11

−U ∗ = −

2 . 11

How to play the game? Player I puts seven red and four green identical balls into an urn, mixes well the balls and selects randomly a ball playing the pure strategy 1 if the selected ball is red and the pure strategy 2 if the selected ball is green. The selected ball is put back into the urn and the random sampling with replacement is repeated. Similarly, Player II puts five red and six green identical balls into an urn, mixes well the balls and selects randomly a ball playing the pure strategy 1 if the selected ball is red and the pure strategy 2 if the selected ball is green. The selected ball is put back into the urn and the random sampling with replacement is repeated. Remark: We have obtained the analytical formulas for the optimum strategies ξ ∗ and η ∗ assuming that the corresponding pairs of lines L1 (ξ1 ), L2 (ξ1 ), and L01 (η1 ), L02 (η1 ) do intersect, respectively. This is what happens in general. If, however, the lines do not intersect, then the optimum strategies are either pure strategies or arbitrary mixed strategies. 2 2. 2×s two-person zero-sum game. Let {X, Y, u} be a two-person zero-sum game where Player I has only two possible pure strategies available, i.e.: r = 2,

X = {x1 , x2 },

u = [uij ],

Y = {y1 , . . . , ys },

(i = 1, 2; j = 1, . . . , s).

Let ξ = (ξ1 , ξ2 ) and η = (η1 , . . . , ηs ) be two arbitrary mixed strategies of the two players, respectively, i.e.: ξi ≥ 0, (i = 1, 2),

ηj ≥ 0, (j = 1, . . . , s),

ξ1 + ξ2 = 1;

s X

j=1

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

The payoff tableau for Player I is:

ξ1

x1

η1 y1 u11

ξ2

x2

u21

II ... ... ...

ηs ys u1s

...

u2s

I The expected payoff for Player I is: U (ξ, η) =

s 2 X X

i=1 j=1

uij ξi ηj =

s X

j=1

u1j ξ1 ηj +

s X

j=1

u2j ξ2 ηj =

ηj = 1.

Game Theory

26 =

s X

u1j ξ1 ηj +

j=1

=

s X

s X

u2j (1 − ξ1 )ηj =

j=1

[(u1j − u2j )ξ1 + u2j ]ηj =

j=1

s X

Lj (ξ1 ) ηj ,

j=1

where: Lj (ξ1 ) = (u1j − u2j )ξ1 + u2j ,

(j = 1, . . . , s).

Therefore, the expected payoff of Player I is a linear convex combination of the linear functions Lj (ξ1 ), (j = 1, . . . , s), because: s X

ηj ≥ 0, (j = 1, . . . , s),

ηj = 1.

j=1

If we plot the lines Lj (ξ1 ), (j = 1, . . . , s), on the unit interval 0 ≤ ξ1 ≤ 1, once the value of ξ1 , and therefore of ξ2 = 1 − ξ1 , is fixed, say ξ˜1 , and therefore ˜ η), is a linear ξ˜2 = 1 − ξ˜1 , the corresponding expected payoff of Player I, U (ξ, function of the mixed strategy η of Player II. Due to the linearity and convexity ˜ η), located just discussed, whatever the mixed strategy η is, the value of U (ξ, on the vertical line segment joining the points Lj (ξ˜1 ), (j = 1, . . . , s), belongs to the interval: ˜ η) ≤ max Lj (ξ˜1 ). min Lj (ξ˜1 ) ≤ U (ξ, 1≤j≤s

1≤j≤s

Player II wants to minimize the expected payoff of his opponent. Whatever mixed strategy ξ is using, Player I cannot prevent Player II from choosing his own mixed strategy η such that U (ξ, η) could reach the bottom lines, more exactly such that: U (ξ, η) = min Lj (ξ1 ). 1≤j≤s

As far as he is concerned, Player I will choose his own mixed strategy ξ in order to maximize the minimum Player II can impose, i.e., the mixed strategy ξ ∗ corresponding to the highest point on the bottom lines. Let Lj1 (ξ1 ) and Lj2 (ξ1 ) be two lines that intersect at the highest point on the bottom lines, and take the 2 × 2 game whose payoff matrix for Player I is:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

II

ξ1

x1

ηj1 yj1 u1j1

ξ2

x2

u2j1

ηj2 yj2 u1j2 u2j2

I After solving this 2 × 2 game, let: (ξ1∗ , ξ2∗ ),

(ηj∗1 , ηj∗2 ),

U ∗,

−U ∗

Game Theory

27

be its optimal solution. The optimum mixed strategies of the initial game are: ξ ∗ = (ξ1∗ , ξ2∗ ),

η ∗ = (0, . . . , 0, ηj∗1 , 0, . . . , 0, ηj∗2 , 0, . . . , 0).

The following picture shows what happens in an 2×3 game where there are three bottom line segments and optimum strategy for Player I corresponds to the highest point on the bottom line segments. The optimum expected payoff U ∗ for Player I and the optimum strategies of the two players: ξ ∗ = (ξ1∗ , ξ2∗ ),

η ∗ = (0, η2∗ , η3∗ ),

are obtained by solving the 2 × 2 two-person zero-sum game corresponding to the lines L2 and L3 . U (ξ, η) 6 %

L1 % % %

aa % L2 aa % L3 aa % a r  ∗ r aa U %  a   %  - ξ1 r% r r (0, 0) % ∗ (1, 0) ξ1 %

Figure 3. Solving a 2 × 3 game. Remark: If there are more than two lines intersecting at the highest point on the bottom lines, we take all pairs of such lines and we solve each corresponding 2 × 2 two-person zero-sum game. We obtain in such a case several equivalent optimum solutions of the original game. 2

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Problem 6: Solve the 2×3 two-person zero-sum game whose payoff matrix for Player I is:

ξ1

x1

η1 y1 1

ξ2

x2

5

II η2 y2 6

η3 y3 14

3

2

I

Game Theory

28

Solution: We notice that this game has no dominated strategies and no saddle-point in terms of pure strategies because: u = max min uij = 2 < 5 = min max uij = u. i

j

j

i

Looking for the optimum mixed strategies, we notice that the expected payoff for Player I is: U (ξ, η) =

3 2 X X

uij ξi ηj =

i=1 j=1

= ξ1 η1 + 6ξ1 η2 + 14ξ1 η3 + 5ξ2 η1 + 3ξ2 η2 + 2ξ2 η3 = = ξ1 η1 + 6ξ1 η2 + 14ξ1 η3 + 5(1 − ξ1 )η1 + 3(1 − ξ1 )η2 + 2(1 − ξ1 )η3 = = (−4ξ1 + 5)η1 + (3ξ1 + 3)η2 + (12ξ1 + 2)η3 = = L1 (ξ1 )η1 + L2 (ξ1 )η2 + L3 (ξ1 )η3 , where: L1 (ξ1 ) = −4ξ1 + 5,

L2 (ξ1 ) = 3ξ1 + 3,

L3 (ξ1 ) = 12ξ1 + 2.

We plot the three lines Lj (ξ1 ), (j = 1, 2, 3), on the unit interval 0 ≤ ξ1 ≤ 1. The lines passing through the highest point on the bottom lines are L1 (ξ1 ) and L2 (ξ1 ). The corresponding 2 × 2 payoff matrix for Player I is: II

ξ1

x1

η1 y1 1

ξ2

x2

5

η2 y2 6 3

I whose optimum solution is:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

ξ1∗ =

3−5 2 = , 1−6−5+3 7

5 ξ2∗ = 1 − ξ1∗ = , 7

3−6 3 4 = , η2∗ = 1 − η1∗ = , 1−6−5+3 7 7 27 6 27 6 (1)(3) − (5)(6) = = 3 , −U ∗ = − = −3 . U∗ = 1−6−5+3 7 7 7 7 The optimum strategies of the two players for the original game are: η1∗ =

ξ∗ =



2 5 , , 7 7 

η∗ =



3 4 , ,0 . 2 7 7 

Game Theory

29

3. r×2 two-person zero-sum game. Let {X, Y, u} be a two-person zero-sum game where Player II has only two possible pure strategies available, i.e.: X = {x1 , . . . , xr }, u = [uij ],

Y = {y1 , y2 },

s = 2,

(i = 1, . . . , r; j = 1, 2).

Let ξ = (ξ1 , . . . , ξr ) and η = (η1 , η2 ) be two arbitrary mixed strategies of the two players, respectively, i.e., r X

ξi ≥ 0, (i = 1, . . . , r),

ηj ≥ 0, (j = 1, 2),

ξi = 1;

η1 + η2 = 1.

i=1

The payoff tableau for Player I is: II

ξ1

x1 .. .

η1 y1 u11 .. .

ξr

xr

ur1

I

η2 y2 u12 .. . ur2

The expected payoff for Player I is: U (ξ, η) =

r X 2 X

uij ξi ηj =

i=1 j=1

=

r X i=1

=

r X

ui1 ξi η1 +

r X

ui1 ξi η1 +

i=1

r X

r X

ui2 ξi η2 =

i=1

ui2 ξi (1 − η1 ) =

i=1

[(ui1 − ui2 )η1 + ui2 ]ξi =

i=1

r X

L0i (η1 ) ξi ,

i=1

where: L0i (η1 ) = (ui1 − ui2 )η1 + ui2 ,

(i = 1, . . . , r).

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Therefore, the expected payoff of Player I is a linear convex combination of the linear functions L0i (η1 ), (i = 1, . . . , r), because: ξi ≥ 0, (i = 1, . . . , r),

r X

ξi = 1.

i=1

If we plot the lines L0i (η1 ), (i = 1, . . . , r), on the unit interval 0 ≤ η1 ≤ 1, once the value of η1 , and therefore of η2 = 1 − η1 , is fixed, say η˜1 , and therefore η˜2 = 1 − η˜1 , the corresponding expected payoff of Player I, U (ξ, η˜), is a linear function of the mixed strategy ξ of Player I. Due to the linearity and convexity

Game Theory

30

just discussed, whatever the mixed strategy ξ is, the value of U (ξ, η˜), located on the vertical line segment joining the points L0i (˜ η1 ), (i = 1, . . . , r), belongs to the interval: min L0i (˜ η1 ) ≤ U (ξ, η˜) ≤ max L0i (˜ η1 ). 1≤i≤r

1≤i≤r

Player I wants to maximize his expected payoff. Whatever mixed strategy η is using, Player II cannot prevent Player I from choosing his own mixed strategy ξ such that U (ξ, η) could reach the top lines, more exactly such that: U (ξ, η) = max L0i (η1 ). 1≤i≤r

As far as he is concerned, Player II will choose his own mixed strategy η in order to minimize the maximum Player I can impose, i.e., the mixed strategy η ∗ corresponding to the lowest point on the top lines. Let L0i1 (η1 ) and L0i2 (η1 ) be two lines that intersect at the lowest point on the top lines, and take the 2 × 2 game whose payoff matrix for Player I is: II

ξi1

xi1

η1 y1 ui1 1

ξi2

xi2

ui2 1

η2 y2 ui1 2 ui2 2

I After solving this 2 × 2 game, let: (ξi∗1 , ξi∗2 ),

(η1∗ , η2∗ ),

U ∗,

−U ∗

be its optimal solution. The optimum mixed strategies of the two players for the initial game are: ξ ∗ = (0, . . . , 0, ξi∗1 , 0, . . . , 0, ξi∗2 , 0, . . . , 0),

η ∗ = (η1∗ , η2∗ ).

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Remark: If there are more than two lines intersecting at the lowest point on the top lines, we take all pairs of such lines and we solve each corresponding 2 × 2 two-person zero-sum game. We obtain in such a case several equivalent optimum solutions of the original game. 2 Remark: As convexity has played an important role in this section, let us remind from calculus that the linear convex combinations of some numbers from the real line describe all the points of the line segment generated by these numbers. 10 . Let a < b on the real line and x = λ1 a + λ2 b. If λ1 and λ2 are arbitrary real numbers then this linear combination generates the entire real line. If, however, we have: λ1 ≥ 0, λ2 ≥ 0, λ1 + λ2 = 1,

Game Theory

31

then the above linear combination generates only the line segment [a, b]. Indeed, λ1 = 1 and λ2 = 0 imply x = a. Also, λ1 = 0 and λ2 = 1 imply x = b. In general, we may write: x = λ1 a + λ2 b = λ1 a + (1 − λ1 )b = b − (b − a)λ1 . But, as (b − a)λ1 ≥ 0, the last equality implies that x ≤ b. On the other hand, as 0 ≤ λ1 ≤ 1, we have: x = b − (b − a)λ1 ≥ b − (b − a) = a. Therefore: a ≤ x ≤ b. 20 . Let a < b < c on the real line and x = λ1 a + λ2 b + λ3 c. If λ1 , λ2 , and λ3 are arbitrary real numbers, then this linear combination generates the entire real line. If, however, we have: λ1 ≥ 0,

λ2 ≥ 0,

λ3 ≥ 0,

λ1 + λ2 + λ3 = 1,

then the above linear combination generates only the line segment [a, c]. Indeed: λ1 = 1, λ2 = 0, and λ3 = 0 imply x = a; λ1 = 0, λ2 = 1, and λ3 = 0 imply x = b; λ1 = 0, λ2 = 0, and λ3 = 1 imply x = c. In general, we may write: x = λ1 a + λ2 b + λ3 c ≤ (λ1 + λ2 + λ3 ) c = c, and, also: x = λ1 a + λ2 b + λ3 c ≥ (λ1 + λ2 + λ3 ) a = a. 2 Remark: Convexity helps us understand better the existence of the lower and upper values of a two-person zero-sum game. Let {X, Y, u} be a twoperson zero-sum game and let: ξ = (ξ1 , . . . , ξr ),

η = (η1 , . . . , ηs )

be two arbitrary mixed strategies of the two players. Then, the expected payoff of Player I, namely:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

U (ξ, η) =

r X s X

i=1 j=1

uij ξi ηj =

s X

j=1

r X

!

uij ξi ηj =

i=1

s X

Lj (ξ) ηj

j=1

is a convex linear combination of the linear functions L1 (ξ), . . . Ls (ξ), where each Lj (ξ) is defined on the compact set: A = {ξ = (ξ1 , . . . , ξr ); ξi ≥ 0, (i = 1, . . . , r),

r X i=1

ξi = 1}.

Game Theory

32

Therefore, there exists the function: V (ξ) = min U (ξ, η) = min Lj (ξ) = min {L1 (ξ), . . . , Ls (ξ)}. η

j

If {ξ (n) } is a sequence from the set A which converges to ξ ∗ ∈ A, when n tends to infinity, then: V (ξ (n) ) = min {L1 (ξ (n) ), . . . , Ls (ξ (n) )} converges to: min {L1 (ξ ∗ ), . . . , Ls (ξ ∗ )} = V (ξ ∗ ), when n tends to infinity. Therefore, V (ξ) is a continuous function on the compact set A. Thus, there exists: max V (ξ) = max min U (ξ, η) = U . η ξ

ξ

Similarly, the expected payoff of Player I may be written as: U (ξ, η) =

s r X X

uij ξi ηj =

i=1 j=1

r X i=1

 

s X

j=1



uij ηj  ξi =

r X

˜ i (η) ξi L

i=1

which shows that it is a convex linear combination of the linear functions ˜ 1 (η),. . . L ˜ r (η), where each L ˜ i (η) is defined on the compact set: L B = {η = (η1 , . . . , ηs ); ηj ≥ 0, (j = 1, . . . , s),

s X

ηj = 1}.

j=1

Therefore, there exists the function: ˜ i (η) = max {L ˜ 1 (η), . . . , L ˜ r (η)}. V˜ (η) = max U (ξ, η) = max L i

ξ

If {η (n) } is a sequence from the set B which converges to η ∗ ∈ B, when n tends to infinity, then: Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

˜ 1 (η (n) ), . . . , L ˜ r (η (n) )} V˜ (η (n) ) = max {L converges to: ˜ 1 (η ∗ ), . . . , L ˜ r (η ∗ )} = V˜ (η ∗ ), max {L when n tends to infinity. Therefore, V˜ (η) is a continuous function on the compact set B. Thus, there exists: min V˜ (η) = min max U (ξ, η) = U . η

η

ξ

Game Theory

33

The fact that the maximin strategy of Player I and the minimax strategy of Player II do exist does not imply that they are unique. Also, there are two-person zero-sum games whose maximin and minimax strategies are degenerate; in such a case the game has an equilibrium point in terms of pure strategies. 2 5. Brown’s algorithm G.W. Brown, from RAND corporation proposed an algorithm in which the two players engaged in a two-person zero-sum game are using their maximin (Player I) and minimax (Player II) strategies during several plays of the game. The optimality of Brown’s algorithm was proved by Julia Robinson. The algorithm gives only an approximation of the optimum solution of the game. Problem 7: Use Brown’s algorithm, with 10 iterations, for approximating the optimum solution of the two-person zero-sum game whose payoff matrix for Player I is:

I

1 1 1 2 1 3 4

II 2 3 −2 3 3 −2 2 1

Solution: The following table contains the 10 iterations of Brown’s algorithm. The upper left hand corner of the table contains the payoff matrix of the game for Player I. Below it and to the right of it we have some numbers referring to the plays of the game, i.e., to the ten iterations of the algorithm. More exactly, the rows 5-14 and the columns 5-14 of the table, numbered 1-10, refer to the ten iterations of the algorithm. In order to simplify the writing, we denote by (k, `, m) a row vector and by (k, `, m)t the transposed vector, which is the column vector: 



Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

k  (k, `, m)t =   ` . m The algorithm starts with an arbitrary choice of a pure strategy of Player I, let it be the pure strategy 2. The corresponding row from the payoff matrix, i.e., (1,3,−2), is reproduced in the fifth row of the table, which becomes the row of iteration 1. Player II wants to minimize Player I’s payoff and will choose his pure strategy 3 corresponding to the minimum entry −2 from the fifth row. The column from the payoff matrix corresponding to the pure strategy 3 of Player II, i.e., (3, −2, 1)t , is reproduced in the fifth column of the table, which

Game Theory

34

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

becomes the column of iteration 1. Player I wants to maximize his payoff and will choose the pure strategy 1 corresponding to the maximum entry 3 from the fifth column of the table. The entries of the payoff matrix corresponding to the pure strategy 1 of Player I, i.e., (1,−2,3), are summed up with the corresponding entries from the fifth row, i.e., (1,3,−2), and the new values, i.e., (2,1,1), are put in the sixth row of the table, which becomes the row of iteration 2.

1

2

3

1

1

−2

3

2

1

3

3

4

2

1

1

3

2

2

3

6

3

2

4

10

5

3

5

11

3

6

6

15

5

7

7

19

7

8

8

23

9

9

9

27

11

10

10

31

13

11

η∗

0

4/10

1

1

2

3

3

1

4

−2

−2

1

−1

1

1

3

4

5

6

7

8

9

10

ξ∗

7

5

3

1

4

7

10

2/10

−3

0

3

6

4

2

0

0

5

7

9

11

12

13

14

8/10

4

−2 1

6/10

Again, Player II wants to minimize the payoff of Player I and chooses his pure strategy corresponding to the minimum entry 1 from the row of iteration 2. But, as we can see, there is a tie between the pure strategies 2 and 3 of

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Game Theory

35

Player II. We solve the tie by applying the rule ‘left first’, abbreviated as the rule ‘→’. This means that we choose the left strategy when the tie comes up the first time but, if another tie happens between the same pair of pure strategies of Player I, we select the second pure strategy of the corresponding pair. {Similarly, in an arbitrary r×s two-person zero-sum game, if there is a tie between three pure strategies of Player II, say the strategies (y2 , y5 , y9 ), we give preference to y2 , when the first tie occurs, to y5 , when the second tie occurs, to y9 , when the third tie occurs, and again to y2 , when the fourth tie occurs, etc.} Returning to our example, Player II plays his pure strategy 2 corresponding to the first from the left minimum entry 1 of the row of iteration 2. The entries of the corresponding column of the payoff matrix, i.e., (−2, 3, 2)t , are added to the corresponding entries (3, −2, 1)t of the column of iteration 1 and the new column of iteration 2 becomes (1, 1, 3)t . Player I chooses the pure strategy 3 corresponding to the largest entry 3 from the column of iteration 2. The entries (4,2,1) of the row of the payoff matrix corresponding to the pure strategy 3 of Player I are added to the entries (2,1,1) of the row of iteration 2 and we get the row (6,3,2) of iteration 3. Player II chooses the pure strategy 3 corresponding to the minimum value 2 from the row of iteration 3 and the entries of the corresponding column of the payoff matrix, i.e., (3, −2, 1)t , are added to the corresponding entries (1, 1, 3)t of the column of iteration 2 giving the new column (4, −1, 4)t of iteration 3. As before, Player I is supposed to play his pure strategy corresponding to the maximum entry from the column of iteration 3 but now there is a tie between the pure strategies 1 and 3. In such a case, we apply the rule ‘bottom first’, abbreviated as the rule ‘↑’, which means that now, at the first tie between the pure strategies 1 and 3, we give priority to the bottom strategy, i.e., the pure strategy 3, but, if a second tie comes up between the same pure strategies, then we will select the pure strategy 1. {Similarly, in an arbitrary r × s two-person zero-sum game, if there is a tie between three pure strategies of Player I, say the strategies (x1 , x3 , x7 )t , we give preference to x7 , when the first tie occurs, to x3 , when the second tie occurs, to x1 , when the third tie occurs, and again to x7 , when the fourth tie occurs, etc.} Returning to our example, Player I chooses the pure strategy 3 and the entries (4,2,1) from the payoff matrix, representing the row corresponding to the pure strategy 3 of Player I, are added to the respective entries (6,3,2) of the row of iteration 3, which gives the row (10,5,3) of iteration 4, etc. The last column of the table contains the relative frequencies of the pure strategies played by Player I thus far and, consequently, the approximation of his maximin strategy is: 8 2 , ξ2∗ ≈ 0, ξ3∗ ≈ . 10 10 The last row of the table contains the relative frequencies of the pure ξ1∗ ≈

Game Theory

36

strategies played by Player II thus far and, consequently, the approximation of his minimax strategy is: η1∗ ≈ 0,

η2∗ ≈

4 , 10

η3∗ ≈

6 . 10

During the ten plays of the game Player I has acted according to the maximin strategy, maximizing the minimum payoff Player II can impose. Similarly, Player II has acted according to the minimax strategy, minimizing the maximum payoff Player I can impose. After ten iterations of the algorithm, the row of iteration 10, i.e., the 14th row of the table, contains the sum of the entries of the rows of the payoff matrix played thus far by Player I, and the column of iteration 10, i.e., the 14th column of the table, contains the sum of the entries of the columns of the payoff matrix played thus far by Player II. The maximum value 14 from the column of iteration 10, or the 14th column of the table, divided by 10, the number of iterations performed, gives an approximation of the lower value of the game U . The minimum value 11 from the row of iteration 10, or the 14th row of the table, divided by 10, the number of iterations performed, gives an approximation of the upper value of the game U . Therefore, the optimum payoff of Player I belongs to the interval: 1.1 =

11 14 ≤ U∗ ≤ = 1.4 10 10

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

The ‘tolerance’ is given by the difference of the two bounds obtained thus far, i.e.:  = 1.4 − 1.1 = 0.3. 2 Generally, we stop the algorithm either when a certain number of iterations, initially chosen, have been performed, or when an initially prescribed value of the tolerance  has been reached. The second procedure of stopping is more laborious because we have to calculate the corresponding value of the tolerance  after each iteration of the algorithm. Let us notice that the algorithm converges because the two players are acting according to their maximin and minimax strategies, respectively, and, according to the minimax theorem, the game does have an equilibrium point in terms of mixed strategies. In fact, due to the minimax theorem, the lower and upper values of the game should be sooner or later equal and, consequently, the tolerance  should become smaller and smaller. As the true lower value U of the game cannot exceed the true upper value U of the game, according to Proposition 1’, but the approximation of U , i.e., 1.4 in our example, can be larger than the approximation of U , i.e., 1.1 in our example, by increasing the number of iterations of the algorithm, the approximations of U will move upwards and the approximations of U will move downwards, tending to become eventually equal.

Game Theory

37

If a large number of iterations of the algorithm are performed, the arbitrary choice of the pure strategy of Player I at the first iteration does not have any significant impact on the final approximation of the optimum solution of the game. 6. The linear programs induced by a two-person zero-sum game The objective of this section is to show that solving any two-person zerosum game may be reduced to solving a primal or dual linear program having a special form. Let {X, Y, u} be a two-person zero-sum game, completely defined by the sets of pure strategies X and Y of the two players and by the payoff matrix u of Player I. We have: X = {x1 , . . . , xr }, u = [uij ],

Y = {y1 , . . . , ys },

(i = 1, . . . , r; j = 1, . . . , s).

Let ξ = (ξ1 , . . . , ξr ) and η = (η1 , . . . , ηs ) be two arbitrary mixed strategies of the two players, respectively, where: ξi ≥ 0,

(i = 1, . . . , r),

r X

ξi = 1;

s X

ηj = 1.

i=1

ηj ≥ 0,

(j = 1, . . . , s),

j=1

The expected payoff of Player I corresponding to the mixed strategies ξ of Player I and η of Player II is: U (ξ, η) =

r X s X

uij ξi ηj .

i=1 j=1

(a) As the lower value of the game is: U = max min U (ξ, η), η Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

ξ

let ξ ∗ be a mixed strategy of Player I for which: min U (ξ ∗ , η) = U . η Therefore, for every mixed strategy η, we have: U (ξ ∗ , η) ≥ U ,

Game Theory

38

which is the same thing as: r X s X

uij ξi∗ ηj ≥ U .

i=1 j=1

In particular, replacing the arbitrary mixed strategy η by the pure strategies of Player II, i.e., η = (1, 0, . . . , 0), . . . , η = (0, 0, . . . , 1), we get: r X

uij ξi∗ ≥ U ,

(j = 1, . . . , s).

i=1

Conversely, if we assume that the last s inequalities hold, then, for every mixed strategy η = (η1 , . . . , ηs ) of Player II, we have: s r X X

uij ξi∗ ηj

=

s X

j=1

i=1 j=1

r X

uij ξi∗

!

ηj ≥ U

i=1

s X

ηj = U .

j=1

Thus, we have just proved that an infinite set of inequalities: r X s X

uij ξi∗ ηj ≥ U ,

ηj ≥ 0, (j = 1, . . . , s),

i=1 j=1

s X

ηj = 1,

j=1

is equivalent to a finite set of inequalities, namely: r X

uij ξi∗ ≥ U ,

(j = 1, . . . , s).

i=1

Explicitly, the last set of s inequalities may be written as: u11 ξ1∗ + . . . + ur1 ξr∗ ≥ U .. .. .. . . . ∗ ∗ u1s ξ1 + . . . + urs ξr ≥ U

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Therefore, the maximin strategy ξ ∗ = (ξ1∗ , . . . , ξr∗ ) of Player I satisfies the system of inequalities: u11 ξ1 + . . . + ur1 ξr ≥ U .. .. .. . . . u1s ξ1 + . . . + urs ξr ≥ U ξ1 + . . . + ξr = 1 ξ1 ≥ 0, . . . , ξr ≥ 0 This is a system of linear inequalities common in a linear program but, apparently, there is no objective function to optimize. But, in using his maximin strategy, the objective of Player I is to maximize the minimum expected payoff induced by Player II. Let us introduce: ξr+1 = min U (ξ, η) = min η

η

r X s X

i=1 j=1

uij ξi ηj .

Game Theory

39

Obviously, ξr+1 is a linear function of the variables ξ1 , . . . , ξr . Also: ∗ = max ξr+1 = max min U (ξ, η) = U , ξr+1 ≤ ξr+1 ξ

which shows that:

r X

ξ

uij ξi ≥ U ,

η

(j = 1, . . . , s),

i=1

imply the set of inequalities: r X

uij ξi ≥ ξr+1 ,

(j = 1, . . . , s),

i=1

which is equivalent to: r X

uij ξi − ξr+1 ≥ 0,

(j = 1, . . . , s),

i=1

Thus we have replaced the initial feasible space defined by the initial inequality constraints by a larger feasible space. At the optimum solution, (i.e., ∗ when ξr+1 is replaced by ξr+1 = U ), this larger feasible space is shrinking back to the initial feasible space defined by the initial inequality constraints. Therefore, the maximin strategy ξ ∗ = (ξ1∗ , . . . , ξr∗ ) of Player I is the solution of the linear program: max ξr+1 ξ

subject to:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

u11 ξ1 + . . . + ur1 ξr − ξr+1 ≥ 0 .. .. .. . . . u1s ξ1 + . . . + urs ξr − ξr+1 ≥ 0 ξ1 + . . . + ξr = 1 ξ1 ≥ 0, . . . , ξr ≥ 0 When the constraints are of the form ≥, like in our case, the linear program would be in a normal form if the objective involved a minimization, which could be done, however, because maxξ ξr+1 may be replaced by minξ (−ξr+1 ), taking into account that: max ξr+1 = − min (−ξr+1 ), ξ

ξ

and if all variables were nonnegative. But in our case only ξ1 ,. . . ,ξr are nonnegative whereas ξr+1 is unrestricted in sign because U (ξ, η) may be negative. This last inconvenience may be fixed as well in a very easy way. Let us notice that if all entries of the payoff matrix of Player I are nonnegative, i.e., if: uij ≥ 0,

(i = 1, . . . , r; j = 1, . . . , s),

Game Theory

40

then we have:

r X s X

U (ξ, η) =

uij ξi ηj ≥ 0,

i=1 j=1

which, obviously, implies that: ξr+1 = min U (ξ, η) ≥ 0. η If, however, there are negative entries in the payoff matrix of Player I, let −c be the most negative such entry, i.e.: −c = min uij < 0, i,j

and let us switch from the initial payoff matrix u = [uij ] to the new one, u˜ = [˜ uij ], whose elements are the nonnegative numbers: u˜ij = uij + c,

(i = 1, . . . , r; j = 1, . . . , s).

As a result of this translation of the payoff values, we have: U˜ (ξ, η) =

r X s X

u˜ij ξi ηj =

i=1 j=1

=

r X s X

uij ξi ηj + c

i=1 j=1

r X s X

(uij + c) ξi ηj =

i=1 j=1

r X i=1

 ! s X ξi  ηj  = U (ξ, η) + c, j=1

which implies: ξ˜r+1 = min U˜ (ξ, η) = min [U (ξ, η) + c] = η

η

U (ξ, η) + c = ξr+1 + c, = min η and, consequently: ∗ ξ˜r+1 = max ξ˜r+1 = max (ξr+1 + c) = max ξr+1 + c =

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

ξ

ξ

ξ

= max min U (ξ, η) + c = U + c. η ξ

Summarizing: In order to find the maximin strategy ξ ∗ = (ξ1∗ , . . . , ξr∗ ) of Player I and the lower value U of the game, let −c be the most negative entry in the payoff matrix u = [uij ] of Player I. We calculate: u˜ij = uij + c, and we solve the linear program:

(i = 1, . . . , r; j = 1, . . . , s),

Game Theory

41 min (−ξ˜r+1 ) ξ

subject to: u˜11 ξ1 + . . . + u˜r1 ξr − ξ˜r+1 ≥ 0 .. .. .. . . . ˜ u˜1s ξ1 + . . . + u˜rs ξr − ξr+1 ≥ 0 ξ1 + . . . + ξr = 1 ˜ ξ1 ≥ 0, . . . , ξr ≥ 0, ξr+1 ≥ 0 ∗ From the optimum solution ξ1∗ , . . . , ξr∗ , ξ˜r+1 of this linear program, the max∗ ∗ ∗ imin strategy of Player I is: ξ = (ξ1 , . . . , ξr ), and the lower value of the initial game is: ∗ − c, U = ξ˜r+1

because: ∗ = min (−ξ˜r+1 ), −ξ˜r+1 ξ

which implies: ∗ ξ˜r+1 = − min (−ξ˜r+1 ) = max ξ˜r+1 = max (ξr+1 + c) = ξ

ξ

ξ

= max ξr+1 + c = max min U (ξ, η) + c = U + c. η ξ

ξ

Remarks: 1) Generally: max [f (x) + g(x)] 6= max f (x) + max g(x), x x x min [f (x) + g(x)] 6= min f (x) + min g(x), x

x

x

but we have equality in both if either f is the constant function, i.e., f (x) ≡ c, or g is the constant function, i.e., g(x) ≡ c. 2) If the function f has a maximum at x∗ , then the function −f has a minimum at the same point x∗ , and: f (x∗ ) = max f (x) = − min [−f (x)] = −[−f (x∗ )]. Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

x

x

Similarly, if the function f has a minimum at x∗ , then the function −f has a maximum at the same point x∗ , and: f (x∗ ) = min f (x) = − max [−f (x)] = −[−f (x∗ )]. x x (b) As the upper value of the game is: U = min max U (ξ, η), η

ξ

2

Game Theory

42

let η ∗ be a mixed strategy of Player II for which: max U (ξ, η ∗ ) = U . ξ

Therefore, for every mixed strategy ξ, we have: U (ξ, η ∗ ) ≤ U , which is the same thing as: r X s X

uij ξi ηj∗ ≤ U .

i=1 j=1

In particular, replacing the arbitrary mixed strategy ξ by the pure strategies of Player I, i.e., ξ = (1, 0, . . . , 0), . . . , ξ = (0, 0, . . . , 1), we get: s X

uij ηj∗ ≤ U ,

(i = 1, . . . , r).

j=1

Conversely, if we assume that the last r inequalities hold, then, for every mixed strategy ξ = (ξ1 , . . . , ξr ) of Player I, we have: r X s X

uij ξi ηj∗ =

i=1 j=1

r X i=1

 

s X

j=1



uij ηj∗  ξi ≤ U

r X

ξi = U .

i=1

Thus, we have just proved that an infinite set of inequalities: r X s X

uij ξi ηj∗ ≤ U ,

ξi ≥ 0, (i = 1, . . . , r),

i=1 j=1

r X

ξi = 1,

i=1

is equivalent to a finite set of inequalities, namely: s X

uij ηj∗ ≤ U ,

(i = 1, . . . , r).

j=1

Explicitly, the last set of r inequalities may be written as:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

u11 η1∗ + . . . + u1s ηs∗ ≤ U .. .. .. . . . ∗ ∗ ur1 η1 + . . . + urs ηs ≤ U Therefore, the maximin strategy η ∗ = (η1∗ , . . . , ηs∗ ) of Player II satisfies the system of inequalities: u11 η1 + . . . + u1s ηs ≤ U .. .. .. . . . ur1 η1 + . . . + urs ηs ≤ U η1 + . . . + η s = 1 η1 ≥ 0, . . . , ηs ≥ 0

Game Theory

43

This is a system of linear inequalities common to a linear program but, apparently, there is no objective function to optimize. But, in using his minimax strategy, the objective of Player II is to minimize the maximum expected payoff induced by Player I. Let us introduce: ηs+1 = max U (ξ, η) = max ξ

ξ

r X s X

uij ξi ηj .

i=1 j=1

Obviously, ηs+1 is a linear function of the variables η1 , . . . , ηs . Also: ∗ ηs+1 ≥ ηs+1 = min ηs+1 = min max U (ξ, η) = U , η

which shows that:

s X

η

uij ηj ≤ U ,

ξ

(i = 1, . . . , r),

j=1

imply the set of inequalities: s X

uij ηj ≤ ηs+1 ,

(i = 1, . . . , r),

j=1

which is equivalent to: s X

uij ηj − ηs+1 ≤ 0,

(i = 1, . . . , r),

j=1

Thus we have replaced the initial feasible space defined by the initial inequality constraints by a larger feasible space. At the optimum solution, (i.e., ∗ when ηs+1 is replaced by ηs+1 = U ), this larger feasible space is shrinking back to the initial feasible space defined by the initial inequality constraints. Therefore, the minimax strategy η ∗ = (η1∗ , . . . , ηs∗ ) of Player II is the solution of the linear program: min ηs+1 η

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

subject to: u11 η1 + . . . + u1s ηs − ηs+1 ≤ 0 .. .. .. . . . ur1 η1 + . . . + urs ηs − ηs+1 ≤ 0 η 1 + . . . + ηs = 1 η1 ≥ 0, . . . , ηs ≥ 0 When the constraints are of the form ≤, like in our case, the linear program would be in a normal form if the objective involved a maximization, which

Game Theory

44

could be done, however, because minη ηs+1 may be replaced by maxη (−ηs+1 ), taking into account that: min ηs+1 = − max(−ηs+1 ), η

η

and if all variables were nonnegative. But in our case only η1 ,. . . ,ηs are nonnegative whereas ηs+1 is unrestricted in sign because U (ξ, η) may be negative. This last inconvenience may be fixed as well in a very easy way. Let us notice that if all entries of the payoff matrix of Player I are nonnegative, i.e., if: uij ≥ 0,

(i = 1, . . . , r; j = 1, . . . , s),

then we have: U (ξ, η) =

r X s X

uij ξi ηj ≥ 0,

i=1 j=1

which, obviously implies that: ηs+1 = max U (ξ, η) ≥ 0. ξ

If, however, there are negative entries in the payoff matrix of Player I, let −c be the most negative such entry, i.e.: −c = min uij < 0, i,j

and let us switch from the initial payoff matrix u = [uij ] to the new one, u˜ = [˜ uij ], whose elements are the nonnegative numbers: u˜ij = uij + c,

(i = 1, . . . , r; j = 1, . . . , s).

As a result of this translation of the payoff values, we have: U˜ (ξ, η) =

r X s X

u˜ij ξi ηj =

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

i=1 j=1

=

r X s X

uij ξi ηj + c

i=1 j=1

r X s X

(uij + c) ξi ηj =

i=1 j=1

r X i=1

 ! s X ξi  ηj  = U (ξ, η) + c, j=1

which implies: η˜s+1 = max U˜ (ξ, η) = max [U (ξ, η) + c] = ξ

ξ

= max U (ξ, η) + c = ηs+1 + c, ξ

Game Theory

45

and, consequently: ∗ = min η˜s+1 = min (ηs+1 + c) = min ηs+1 + c = η˜s+1 η

η

η

= min max U (ξ, η) + c = U + c. η

ξ

Summarizing: In order to find the minimax strategy η ∗ = (η1∗ , . . . , ηs∗ ) of Player II and the upper value U of the game, let −c be the most negative entry in the payoff matrix u = [uij ] of Player I. We calculate: u˜ij = uij + c,

(i = 1, . . . , r; j = 1, . . . , s),

and we solve the linear program: max (−˜ ηs+1 ) η subject to: u˜11 η1 + . . . + u˜1s ηs − η˜s+1 .. .

≤ .. .

0 .. .

u˜r1 η1 + . . . + u˜rs ηs − η˜s+1 ≤ 0 η 1 + . . . + ηs = 1 η1 ≥ 0, . . . , ηs ≥ 0, η˜s+1 ≥ 0 ∗ From the optimum solution η1∗ , . . . , ηs∗ , η˜s+1 of this linear program, the min∗ ∗ imax strategy of Player II is: η = (η1 , . . . , ηs∗ ), and the upper value of the initial game is: ∗ U = η˜s+1 − c,

because: ∗ = max (−˜ ηs+1 ), −˜ ηs+1 η

which implies: ∗ η˜s+1 = − max (−˜ ηs+1 ) = min η˜s+1 = min (ηs+1 + c) =

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

η

η

η

= min ηs+1 + c = min max U (ξ, η) + c = U + c. η

η

ξ

We notice that the linear program whose optimum solution is the minimax strategy of Player II and the upper value of the game is the dual of the linear program whose optimum solution is the maximin strategy of Player I and the lower value of the game. But, according to the dual theorem from linear programming, proved in the appendix of the book, the optimum objective value of the primal linear program is equal to the optimum objective value of

Game Theory

46

its dual linear program. Therefore, in our case, the corresponding optimum objective values are: ∗ ∗ −ξ˜r+1 = −˜ ηs+1 ,

∗ ∗ or ξ˜r+1 = η˜s+1 ,

which is equivalent to: ∗ ∗ U = ξ˜r+1 − c = η˜s+1 − c = U,

and, from Propositions 2’ and 3’, we conclude that the pair of maximin strategy of Player I and the minimax strategy of Player II, i.e., the pair of mixed strategies (ξ ∗ , η ∗ ), is a saddle-point, or equilibrium point, of the game, which is just the proof of the minimax theorem. The optimum expected payoff for Player I is U ∗ = U = U , and the expected optimum payoff for Player II is −U ∗ = −U = −U . Remarks: 1) Historically, the game theory, and the minimax theorem in particular, have been obtained before and independently of the linear programming. G. Dantzig, one of the main contributors in linear programming, acknowledged the seminal input obtained from John von Neumann who immediately had noticed that the duality from linear programming could be viewed as a zero-sum game between two players with opposite interests. Today, it is more logical to derive the crucial minimax theorem of the two-person zero-sum games from the dual theorem from linear programming and to solve such a game by taking advantage of the considerable and fast development of linear programming that has occurred in the last decades. 2) Practically, for getting the maximin and minimax strategies of the two players in a two-person zero-sum game, it is not necessary to solve both linear programs mentioned in this section. In fact, it is enough to solve one of them and the optimal solution of such a primal linear program is given together with its dual optimum solution. 2

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Problem 8: Write the primal and dual linear programs which give the maximin and minimax mixed strategies for the two players and the optimum payoff for Player I for a two-person zero-sum game whose payoff matrix for Player I is:

I

1 1 1 2 1 3 4

II 2 3 −2 3 3 −2 2 1

Solution: Using the notations discussed above, we have r = 3, s = 3, −c = −2, and taking: u˜ij = uij + c = uij + 2,

Game Theory

47

we obtain the new payoff matrix with all entries nonnegative:

I

1 1 3 2 3 3 6

II 2 0 5 4

3 5 0 3

The linear program whose optimum solution is the maximin strategy of Player I and the optimum mean payoff ξ˜4∗ − 2 of Player I is: min (−ξ˜4 ) ξ

subject to: 3ξ1 + 3ξ2 + 6ξ3 − ξ˜4 0ξ1 + 5ξ2 + 4ξ3 − ξ˜4 5ξ1 + 0ξ2 + 3ξ3 − ξ˜4 ξ1 + ξ2 + ξ3 ξi ≥ 0, (i = 1, 2, 3), ξ˜4 ≥ 0

≥ ≥ ≥ =

0 0 0 1

The linear program whose optimum solution is the minimax strategy of Player II and the optimum mean payoff η˜4∗ − 2 of Player I is: max (−˜ η4 ) η

subject to: 3η1 + 0η2 + 5η3 − η˜4 3η1 + 5η2 + 0η3 − η˜4 6η1 + 4η2 + 3η3 − η˜4 η1 + η 2 + η 3 ηi ≥ 0, (i = 1, 2, 3), η˜4 ≥ 0

≤ ≤ ≤ =

0 0 0 1 2

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

7. Computer tips 1. LINDO. The easiest way of solving a two-person zero-sum game is to use Lindo, a simple software specialized in solving linear programs. It is very similar to writing algebraic linear inequalities and it is widely available both on Dos and on Windows computer operating systems. Some useful hints about Lindo: The names for the variables of the linear program are limited to eight characters and cannot start with a digit. The strict inequalities > and < are interpreted by the computer as ≥ and ≤, respectively.

Game Theory

48

Parentheses are not accepted. Comments may be placed anywhere when we type the linear program and are preceded by an exclamation point ! Constraints and objective functions may be split over multiple lines. Lindo is not case-sensitive. The input is converted to upper-case internally by Lindo. Only constant values, not variables are permitted on the right-hand side of the inequality constraints. Only variables and their coefficients are permitted on the left-hand side of the inequality constraints. Problem 9: In a two-person zero-sum game, the payoff matrix for Player I is shown below:

I

1 1 3 2 3 3 6

II 2 0 5 4

3 5 0 3

Use the computer software Lindo for getting the maximin strategy of Player I and the optimum expected payoff of Player I. Solution: According to the analysis made in section 6, the corresponding linear program whose optimum solution is the maximin strategy of Player I and the optimum expected payoff of Player I has the form: min −x4 subject to:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

3x1 + 3x2 + 5x2 + 5x1 + x1 + x2 + x1 , x 2 ,

6x3 4x3 3x3 x3 x3 ,

− x4 ≥ 0 − x4 ≥ 0 − x4 ≥ 0 = 1 x4 ≥ 0

(a) Using Lindo for Windows. In what follows, an arrow → means ‘move the cursor to’ or ‘follow the path’. Left click the mouse on the first and on the last options of a sequence of options: Start → Programs → Lindo A window is available on the screen. Type the linear program and press the key ‘Enter’ after each line:

Game Theory

49 min −x4 st 3x1+3x2+6x3−x4>0 5x2+4x3−x4>0 5x1+3x3−x4>0 x1+x2+x3=1

Click on: File → Save As and, with a diskette inserted into drive ‘a:’, save the file as: ‘a:john.ltx’. Click ‘OK’. Once the linear program is saved in the file ‘john.ltx’, it may be recovered later by selecting this file after choosing: File → Open and typing: ‘a:john.ltx’. If you want to print the file, click on: File → Print Now, if you want to let the computer solve the linear program from the file ‘john.ltx’, choose from the top menu: Solve → Solve The message: ‘Do range (sensitivity) analysis?’ appears on the screen. Click on: ‘No’ and ‘Close’. A ‘Reports’ window appears in the background of the screen. Click anywhere in the ‘Reports’ window. The front window, containing the linear program, disappears and you get the solution of the linear program in the open ‘Reports’ window. If the solution is too long, move forward or backwards, by using the keys ↓ and ↑ located on the left-hand side of the screen, respectively. The output containing the solution of our problem looks like: OBJECTIVE FUNCTION VALUE

1) −3.333333

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

which gives the optimum value of the objective function of the linear program. VARIABLE X4 X1 X2 X3

VALUE

REDUCED COST

3.333333 .166667 .000000 .833333

.000000 .000000 1.666667 .000000

ROW

SLACK OR SURPLUS

DUAL PRICES

2) 3) 4) 5)

2.166667 .000000 .000000 .000000

.000000 −.333333 −.666667 3.333333

Game Theory

50

In order to read the solution of our game, just ignore the content of columns ‘Reduced Cost’ and ‘Slack or Surplus’. The optimum expected payoff of Player I and the maximin strategy of Player I are in column ‘Value’, namely: U ∗ = x∗4 = 3.333333, x∗1 = 0.166667, x∗2 = 0.000000, x∗3 = 0.833333. The optimum expected payoff of Player II and the minimax strategy of Player II are in column ‘Dual Prices’, with changed signs, namely: y1∗ = 0.000000, y2∗ = 0.333333, y3∗ = 0.666667, −U ∗ = y4∗ = −3.333333. The solution of the linear program may be saved on a diskette inserted into drive ‘a:’ by clicking on: File → Save As Type: a:soljohn.ltx and click ‘OK’. At any time during the Lindo session you can choose the option ‘Help’ from the top menu. If you want to print the file, choose: File → Print To end the Lindo session, click on:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

File → Exit (b) Using Lindo for Dos. We type the following lines, pressing the key Enter after each command: cd\ lindo lindo edit john.lnd min −x4 st 3x1+3x2+6x3−x4>0 5x2+4x3−x4>0 5x1+3x3−x4>0 x1+x2+x3=1 end To end the editing press the key ‘Esc’ and type: look all If there are mistakes in your linear program, type: edit john.lnd Make the necessary corrections; at the end press the ‘Esc’ key. If you want the results to be shown on the screen, type: go

Game Theory

51

After giving the same solution of our problem, as mentioned before, Lindo asks us: ‘Do you want sensitivity analysis?’ Type: no If you want the linear program and the solution to be saved and eventually printed, type: divert Insert a diskette in drive ‘a:’ and type: a:soljohn.lnd look all go Lindo is asking: ‘Do you want sensitivity analysis?’ Type: no If you want help, type: help commands For information on a specific command type ‘help’ followed by the command name. For instance, if help is needed about the command ‘divert’, type: help divert To end the Lindo session type: quit cd\ If you want to see the content of the file ‘soljohn.lnd’, use the command: type a:soljohn.lnd If you want to print the file ‘a:soljohn.lnd’, type: print a:soljohn.lnd When Lindo asks for the ‘Name of list device [PRN]:’ press the key Enter and, after the printing is done, switch off the computer. Obviously, instead of saving the files on a diskette in drive ‘a:’, we can save them on a CD in drive ‘d:’ or on the hard disk using the path: ‘c:\windows’.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

2. SAS-OR. The computer software Sas has been intensively used in the applications of mathematical statistics. Not so long ago, a package of procedures known as Sas-or has been added to it for solving linear programs. Assuming that Sas-or is available on a Unix machine and we use a text editor called ‘pico’ available on Unix, we want to solve the following problem: Problem 10: In a two-person zero-sum game, the payoff matrix for Player I is shown below:

I

1 1 3 2 3 3 6

II 2 0 5 4

3 5 0 3

Game Theory

52

Use the computer software Sas-or for getting the maximin strategy of Player I and the optimum expected payoff of Player I. Solution: According to the analysis made in section 6, the corresponding linear program whose optimum solution is the maximin strategy of Player I and the optimum expected payoff of Player I has the form: min −x4 subject to:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

3x1 + 3x2 + 5x2 + 5x1 + x1 + x2 + x1 , x 2 ,

6x3 4x3 3x3 x3 x3 ,

− x4 ≥ 0 − x4 ≥ 0 − x4 ≥ 0 = 1 x4 ≥ 0

Assuming that we have accessed the Unix machine, we use the text editor ‘pico’ and type our linear program according to the rules requested by Sas-or. The Sas-or file will get the name ‘john.sas’ (the name is arbitrary but the file type ‘sas’ is compulsory). Each sas command ends with a semicolon. After each command we press the key ‘Enter’. The command starting with ‘input’ lists all the numerical variables of the linear program, an alphanumerical variable ‘ − type− ’,followed by the dollar sign $, whose possible values are ‘max’, ‘min’, ‘ge’ (≥), ‘le’ (≤), or ‘eq’ (=), and a last numerical variable ‘ − rhs− ’, which is the right-hand side of the linear program. In the data set, starting with ‘cards;’ and ending by a semicolon ‘;’, the coefficients of the objective function and of the constraints of the respective linear program are specified. In each line, each variable listed in the ‘input’ line has to get a certain value for its coefficient. If such a value is missing, like the right-hand side in the objective of the linear program, we use the dot ‘.’ to indicate that the respective variable has no value there. However, if a numerical variable of the linear program is missing in a constraint or in the objective function, the respective coefficient ‘0’ should be explicitly mentioned. Sas-or assumes that the variables of the linear program are nonnegative if nothing is mentioned to the contrary in distinct constraints; therefore, in our case the nonnegativity constraints are not needed. For solving our problem, type the commands of the sas program ‘john.sas’ as follows: pico john.sas data; input x1 x2 x3 x4 − type− $ − rhs− ; cards; 0 0 0 −1 min . 3 3 6 −1 ge 0

Game Theory

53

0 5 4 −1 ge 0 5 0 3 −1 ge 0 1 1 1 0 eq 1 ; proc lp; The last command in the above program is asking for the ‘procedure lp’, containing the subroutines that solve a linear program. We close the editing by pressing the keys ‘Ctrl’ and ‘x’ together. We can read the data file by typing: more john.sas This command allows us to read the sas file one screen at a time and we advance by pressing the space-bar. If we want to make corrections to the file ‘john.sas’ we use the text editor ‘pico’ again and at the end we press the keys ‘Ctrl’ and ‘x’ together. We execute the sas program by typing: sas john.sas To see the results which are automatically stored by sas in a listing file called ‘john.lst’, type: more john.lst From the output we get both the optimum solution of the given linear program and the optimum solution of its dual. In the first part of the output, ‘Variable Summary’, we focus only on the entries from the column ‘Activity’, where we have the optimum numerical values of the variables x1 , x2 , x3 , x4 , in this order. In the second part of the output, ‘Constraint Summary’ we get the optimum expected payoff of Player I, as the value of OBS1, and the components of the minimax strategy y1 , y2 , y3 of Player II, in this order, as the values of OBS2, OBS3, and OBS4, respectively. In our case, the output is: VARIABLE SUMMARY ACTIVITY

x1 x2 x3 x4

0.16667 0.00000 0.83333 3.33333

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

CONSTRAINT SUMMARY OBS1 OBS2 OBS3 OBS4

ACTIVITY DUAL ACTIVITY 3.33333 0.00000 0.33333 0.66667

Therefore, we get: x∗1 = 0.16667,

x∗2 = 0.00000,

x∗3 = 0.83333,

x∗4 = 3.33333,

Game Theory y1∗

= 0.00000,

54 y2∗

= 0.33333,

y3∗

= 0.66667

y4∗

= 3.33333. 2

We end the Sas-or session by typing: exit Remark: If we want to use the version SAS 9.1 on Windows, we click on: Start → Programs → SAS 9.1. The Editor window is active and we type the same SAS-OR program as shown before, adding the command ‘run;’ at the end, namely: data; input x1 x2 x3 x4 − type− $ − rhs− ; cards; 0 0 0 −1 min . 3 3 6 −1 ge 0 0 5 4 −1 ge 0 5 0 3 −1 ge 0 1 1 1 0 eq 1 ; proc lp; run; From the top menu we click on: Run → Submit

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

and the results are shown in the Output window. We end the session by clicking on: File → Exit, followed by: Start → Shut Down 2 3. EXCEL. Microsoft Excel is another popular software for solving linear programs. Excel uses a worksheet. Numerical values, words, variables are put in the cells of the worksheet. We move inside the worksheet, from cell to cell, using the arrow keys. The columns of the worksheet are labelled by capital letters A,B,C,. . . . The rows of the worksheet are labelled by positive integer numbers 1,2,3,. . . . Each cell is identified by its column and row like B5, for instance, located in column B and row 5. In order to activate a cell we move the cursor into it and click the mouse twice. After we type a number, a text, or a formula, press the ‘Enter’ key. The operators are: +, −, *, and /, for addition, subtraction, multiplication, and division, respectively. We access the Microsoft Excel by clicking on: Start → Programs → Microsoft Excel Problem 11: In a two-person zero-sum game, the payoff matrix for Player I is shown below:

Game Theory

55

I

1 1 3 2 3 3 6

II 2 0 5 4

3 5 0 3

Use the computer software Excel for getting the maximin strategy of Player I and the optimum expected payoff of Player I. Solution: According to the analysis made in section 6, the corresponding linear program whose optimum solution is the maximin strategy of Player I and the optimum expected payoff of Player I has the form: min −x4 subject to: 3x1 + 3x2 + 5x2 + 5x1 + x1 + x2 + x1 , x 2 ,

6x3 4x3 3x3 x3 x3 ,

− x4 ≥ 0 − x4 ≥ 0 − x4 ≥ 0 = 1 x4 ≥ 0

In solving the above linear program, we reserve the cells B6, C6, D6, E6 for the values of the variables x1 , x2 , x3 , x4 , respectively. The coefficients of the objective function will be in the cells B7, C7, D7, E7, respectively. The value of the objective function will be in the cell F7. The matrix of the constraint coefficients of the constraints are in the cells:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

B8, C8, D8, E8 B9, C9, D9, E9 B10, C10, D10, E10 B11, C11, D11, E11, respectively. The right-hand side of the constraints are put in the cells G8, G9, G10, G11, respectively. We start with the initial value 0 for all variables. The algebraic expressions of the left-hand side of the linear constraints are typed, as formulas, in the cells F8, F9, F10, F11, respectively. We have to be very careful in typing correctly these algebraic expressions because, once typed, after pressing the key ‘Enter’, the corresponding cell gets the current value of the respective algebraic expression which, in our case, is the initial value 0. The algebraic expression itself is no longer shown in the respective cell and only its current value is displayed. Thus, for solving our linear program, we put: the numbers 0, 0, 0, 0 in the cells B6, C6, D6, E6, respectively; the numbers 0, 0, 0, −1 in the cells B7,

Game Theory

56

C7, D7, E7, respectively; the numbers 3, 3, 6, −1 in the cells B8, C8, D8, E8, respectively; the numbers 0, 5, 4, −1 in the cells B9, C9, D9, E9, respectively; the numbers 5, 0, 3, −1 in the cells B10, C10, D10, E10, respectively; the numbers 1, 1, 1, 0 in the cells B11, C11, D11, E11, respectively; the numbers 0, 0, 0, 1 in the cells G8, G9, G10, G11, respectively. In the cell F6 we list the location of our variables by typing B6:E6. In the cell F7 we type the formula for the objective function, namely: = B7*B6+C7*C6+D7*D6+E7*E6 or, equivalently: = SUM(B7*B6:E7*E6) After typing this formula, when we press the key ‘Enter’, Excel will memorize this formula from the cell F7, but in the cell F7 itself the current value of this formula, 0 in our case, will show up, due to the current values from the cells mentioned in the formula just typed. In the cell F8 we type the formula for the left-hand side of the first constraint, namely: = B8*B6+C8*C6+D8*D6+E8*E6 or, equivalently: = SUM(B8*B6:E8*E6) After typing this formula, when we press the key ‘Enter’, Excel will memorize this formula from the cell F8, but in the cell F8 itself the current value of this formula, 0 in our case, will show up, due to the current values from the cells mentioned in the formula just typed. In the cell F9 we type the formula for the left-hand side of the second constraint, namely: = B9*B6+C9*C6+D9*D6+E9*E6 or, equivalently:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

= SUM(B9*B6:E9*E6) After typing this formula, when we press the key ‘Enter’, Excel will memorize this formula from the cell F9, but in the cell F9 itself the current value of this formula, 0 in our case, will show up, due to the current values from the cells mentioned in the formula just typed. In the cell F10 we type the formula for the left-hand side of the third constraint, namely: = B10*B6+C10*C6+D10*D6+E10*E6 or, equivalently: = SUM(B10*B6:E10*E6)

Game Theory

57

After typing this formula, when we press the key ‘Enter’, Excel will memorize this formula from the cell F10, but in the cell F10 itself the current value of this formula, 0 in our case, will show up, due to the current values from the cells mentioned in the formula just typed. In the cell F11 we type the formula for the left-hand side of the fourth constraint, namely: = B11*B6+C11*C6+D11*D6+E11*E6 or, equivalently: = SUM(B11*B6:E11*E6) After typing this formula, when we press the key ‘Enter’, Excel will memorize this formula from the cell F11, but in the cell F11 itself the current value of this formula, 0 in our case, will show up, due to the current values from the cells mentioned in the formula just typed. At this stage the worksheet for our problem is filled in with the data of our linear program. Click ‘Tools’ from the top menu and choose ‘Solver’. Type in the ‘Set Target Cell’: $F$7 to indicate where the objective value is located. The dollar signs denote absolute cell references. Do not press the ‘Enter’ key but move the cursor to the line ‘Equal To’ and click on ‘Min’. Do not press the ‘Enter’ key but move the cursor to the line ‘By Changing Cells’ and type: $B$6:$E$6 to indicate where the values of the variables are located. Do not press the ‘Enter’ key but move the cursor to the line ‘Subject to the Constraints’. Click the ‘Add’ button and type the nonnegativity constraint:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

$B$6:$E$6 >= 0 The options >=, =, = $G$8:$G$10 Click the ‘Add’ button and type the fourth constraint: $F$11 = $G$11

Game Theory

58

We finally solve the linear program by clicking on the button ‘Solve’ and ‘OK’. We can choose the option ‘Analysis’ and ‘OK’ for keeping the ‘Solver solution’. Removing the ‘Solver tableau’, and going back to the initial worksheet, we find the optimum solution in the cells B6, C6, D6, E6, respectively: x∗1 = 0.1666667, x∗2 = 0, x∗3 = 0.833334, x∗4 = 3.333337. 2 The dual linear program may be solved similarly. To end the Excel session, click on ‘File’, from the top menu, and choose ‘Exit’. Remark: If we want to use the version Excel 2007, we follow the same procedure, except that the location of the powerful tool ‘Solver’ is different. For loading the Solver Add-in: 1. Click the ‘Microsoft Office Button’ (on the first top left-hand position on the screen), and then click ‘Excel Options’. 2. Click ‘Add-ins’ and then, in the ‘Manage’ box, select ‘Excel Add-ins’. 3. Click ‘Go’. 4. In the ‘Add-ins’ available box select the ‘Solver Add-in’ check box amd then click ‘OK’. If ‘Solver Add-in’ is not listed in the ‘Add-ins’ available box click ‘Browse’ to locate the add-in. If you get prompted that the ‘Solver Addin’ is not currently installed on your computer, click ‘Yes’ to install it from: c:\Program Files\Microsoft Office\Office12\Library\Solver 5. After you load the ‘Solver Add-in’, the ‘Solver’ command is available in the ‘Analysis’ group on the ‘Data’ tab on the right-hand last position of the top menu. 2

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

4. MATHEMATICA. Mathematica, created by Stephen Wolfram, is a wonderful software for doing computational and symbolic mathematics. Chapters 2 and 5 will make intensive use of it and many details will be given there. Here we simply mention that it offers the easiest way of solving the linear programs that give the optimum solution of a two-person zero-sum game. We will refer here to the Student Version of Mathematica for Windows. There are many books dealing with Mathematica and its applications. The basic textbook remains Wolfram (1991). Problem 12: In a two-person zero-sum game, the payoff matrix for Player I is shown below: II 1 2 3 1 3 0 5 I 2 3 5 0 3 6 4 3

Game Theory

59

Use the computer software Mathematica for getting the maximin strategy of Player I, the minimax strategy of Player II, and the optimum expected payoff of Player I. Solution: According to the analysis made in section 6, the corresponding primal linear program whose optimum solution is the maximin strategy of Player I and the optimum expected payoff of Player I has the form: min −x4 subject to: 3x1 + 3x2 + 5x2 + 5x1 + x1 + x2 + x1 , x 2 ,

6x3 4x3 3x3 x3 x3 ,

− x4 ≥ 0 − x4 ≥ 0 − x4 ≥ 0 = 1 x4 ≥ 0

whereas the dual linear program, whose optimum solution is the minimax strategy of Player II and the optimum expected payoff of Player I, has the form: max −y4 subject to: 3y1 + 5y3 3y1 + 5y2 6y1 + 4y2 + 3y3 y1 + y2 + y3 y1 , y2 , y3 ,

− y4 ≤ 0 − y4 ≤ 0 − y4 ≤ 0 = 1 y4 ≥ 0

Assuming that Mathematica is already installed on a PC equipped with a Windows operating system, we start the Mathematica session by moving the cursor and clicking the left-hand side of the mouse on the first and last option of the following sequence:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Start → Program → Mathematica → Mathematica #. For solving the primal linear program, inside the window created on the screen, we use the command ‘ConstrainedMin’ (Mathematica is case sensitive and the distinction between small and capital letters is essential) followed by the objective function, the list of inequality constraints (an equality constraint should be replaced by two opposite inequality constraints), between curly braces, and the list of the variables, also between curly braces. The operations are +, −, ∗, and /; for multiplication we can also leave an empty

Game Theory

60

space between factors, like 3 x1 when we want to multiply the variable x1 by 3, for instance. After each command we press the keys ‘Shift’ and ‘Enter’ together: ConstrainedMin[−x4,{3 x1+3 x2+6 x3−x4>0, 5 x2+4 x3−x4>0, 5 x1+3 x3−x4>0, x1+x2+x31, x1>0, x2>0, x3>0, x4>0}, {x1,x2,x3,x4}] After we press the keys ‘Shift’ and ‘Enter’ together, we get the output containing the solution: Out[1]= {−10/3, {x1− >1/6, x2− >0, x3− >5/6, x4− >10/3}} In order to replace the fractions by decimal approximations, we type the following command meaning ‘give numerical approximations in the previous list’, namely: N[%] Out[2]= {−3.333333,{x1− >0.166667, x2− >0, x3− >0.833333, x4− >3.333333}} Similarly, for solving the dual linear program, we type the command: ConstrainedMax[−y4,{3 y1+5 y3−y40, y4>0}, {y1,y2,y3,y4}] and, after pressing the keys ‘Shift’ and ‘Enter’ together, we get the solution: Out[3]= {−10/3, {y1− >0, y2− >1/3, y3− >2/3, y4− >10/3}} N[%] Out[4]= {−3.333333,{y1− >0, y2− >0.333333, y3− >0.666667, y4− >3.333333}} We end the Mathematica session by clicking on: File → Exit 2 8. Prisoner’s dilemma Let us notice first that a two-person constant-sum game {X, Y, u(1) , u(2) }, i.e., a game for which: (1)

(2)

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

uij + uij = c,

(i = 1, . . . , r; j = 1, . . . , s),

where c is a constant, may be reduced to a two-person zero-sum game. Indeed, it is enough to introduce the new payoff matrix of Player I: c (1) (1) u˜ij = uij − , (i = 1, . . . , r; j = 1, . . . , s), 2 and the new payoff matrix of Player II: c (2) (2) u˜ij = uij − , (i = 1, . . . , r; j = 1, . . . , s), 2

Game Theory

61

and we notice that: (1) u˜ij

+

(2) u˜ij

=

(1) uij

c c c c (2) = 0. − + uij − = c − + 2 2 2 2 



Therefore: U˜ (1) (ξ, η) =

r X s X

u˜ij ξi ηj =

r X s X

(2) u˜ij ξi ηj

(1)

i=1 j=1

U˜ (2) (ξ, η) =

r X s X

uij ξi ηj −

r X s X

uij ξi ηj −

(1)

i=1 j=1

i=1 j=1

=

(2)

i=1 j=1

c c = U (1) (ξ, η) − , 2 2 c c = U (2) (ξ, η) − , 2 2

which imply: c c c c U˜ (1) (ξ, η) + U˜ (2) (ξ, η) = U (1) (ξ, η) + U (2) (ξ, η) − − = c − + = 0. 2 2 2 2 



Thus, the new game {X, Y, u˜(1) , u˜(2) } is a two-person zero-sum game. The optimum strategy of each player is not affected by the translation (change of scale) of their payoff matrices, due to the following relations: ˜ (1) (ξ, η) = max min U (1) (ξ, η) − c , max min U η η ξ ξ 2 c max min U˜ (2) (ξ, η) = max min U (2) (ξ, η) − . η η ξ ξ 2 Also, the maximin strategy of Player II in terms of his own payoff matrix (2) [˜ u ] is equivalent to the minimax strategy of Player II in terms of the payoff matrix of Player I, namely [˜ u(1) ], because: max min U˜ (2) (ξ, η) = max min {−U˜ (1) (ξ, η)} = η η

ξ

ξ

= max {− max U˜ (1) (ξ, η)} = − min max U˜ (1) (ξ, η). η η ξ

ξ

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Thus, a two-person constant-sum game {X, Y, u(1) , u(2) } may be reduced to a two-person zero-sum game {X, Y, u˜(1) , u˜(2) }. However, the two-person nonconstant-sum games, where the interests of the players are not totally opposed, raise new problems, not met before, like the distinction between optimum individual and optimum collective way of acting. The following prisoner’s dilemma is a relevant example of what happens in such a game. In a dark park a murder has been committed. The victim died of one single bullet in his head. Two suspects are arrested on the spot. Each has a gun of the same caliber with a missing bullet. The two suspect are taken to the police station and put in two separate cells. They have one night to think about how to behave on the next day when they are going to meet separately the judge, knowing that the possible results of their way of acting are as shown in the following table:

Game Theory

62 II Talk

Do not

Talk

(−5, −5)

(0, −20)

Do not

(−20, 0)

(−1, −1)

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

I The pure strategy ‘talk’ is an abbreviation for pleading not guilty and accusing the other suspect, like ‘I am not guilty, he is guilty!’ On the other side, ‘do not’ means saying nothing or keeping your mouth shut. In the payoff matrix, (−5, −5) means that both prisoners get five years in jail if each of them pleads not guilty and accuses the other suspect. If one prisoner, say I, talks and the other one does not, then I gets free and II goes to jail for 20 years. Finally, if both prisoners do not talk, then the judge has no other witness to listen to and gives one year in jail to both suspects because they have no gun permit. Let us notice that this game is not a zero-sum game and the payoff matrix of each player is given. An equilibrium point of the game may be defined even in this new context. A pair of pure strategies (xi , yj ) of the two players is a ‘saddle-point’ or ‘equilibrium point’ of the game if neither player can do better by unilaterally abandoning his/her own strategy xi or yj , respectively. For our game, obviously, (Talk,Talk) is an equilibrium point of the game and the result is five years in jail for each suspect, i.e., (−5, −5). Such an equilibrium point is the optimum noncooperative solution of the game. As the game is not a zero-sum one, the interests of the two players are not totally opposed and, obviously, there is a better solution for both, namely (Do not, Do not), resulting in only one year in jail for each, i.e., (−1, −1). This pair of pure strategies (Do not, Do not) is the optimum cooperative solution of the game. It is based, however, on trust, fair play, and reliability. Put in the same cell, the two suspects could analyze together the game and decide to keep their mouth shut in front of the judge on the next day, as the best solution for them, as a team, but imagine what happens if on the next day one prisoner says nothing, keeping his promise, but the other one talks, breaking the agreement! There are many serious games like this in international affairs and, unfortunately, a better solution for both sides cannot be reached because of the lack of trust or unreliability of some players. We may generalize Prisoner’s dilemma in the following way. We suppose that in a two-person non-zero sum game, each player has two possible pure strategies namely, the cooperative pure strategy ‘C’ and the noncooperative pure strategy ‘NC’. In Prisoner’s dilemma analyzed before, ‘C’ corresponds to ‘Do not talk’ and ‘NC’ to ‘Talk’. The payoff matrices of the two players are:

Game Theory

63 II NC

C

NC

(p, p)

(t, s)

C

(s, t)

(r, r)

I where: p is the penalty for noncooperating, r is the reward for cooperating, t is the temptation of double-crossing, and s is the sufferance of the double-crossed player. Such a game is of prisoner’s dilemma type if t > r > p > s. Prisoner’s dilemma has proved very significant in political negotiation, in economic bargaining, and in animal behaviour. In the last field, we can find many examples where, during a long evolution, based on trial-and-error, different species have ‘learned’ that a cooperative strategy could be more profitable than a fighting, noncooperative one. This is why a hippopotamus, for instance, allows and does not hurt some little birds who enter his mouth and clean his teeth while they are eating the residues of food found there. 9. n-person games 1. Characteristic function. Let N = {1, 2, . . . , n} be the set of players, and P(N ) be the class of all subsets of N , i.e.: P(N ) = {S; S ⊆ N }. A subset S ∈ P(N ) is called a coalition. There are 2n possible coalitions, including the empty set ∅ and the set of all players N . The characteristic function of the game is a set function: ν : P(N ) −→ R, which is superadditive, i.e., satisfies the inequality: ν(S ∪ S 0 ) ≥ ν(S) + ν(S 0 ),

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

for all disjoint coalitions: S ∈ P(N ),

S 0 ∈ P(N ),

S ∩ S 0 = ∅.

The number ν(S) represents the payoff, or value, of coalition S whose members are acting together, as a team. The superadditivity of the characteristic function means that joining two distinct coalitions, S and S 0 , we obtain a bigger coalition, S ∪ S 0 , whose value ν(S ∪ S 0 ) cannot be smaller than the sum of the values ν(S) and ν(S 0 ) of this coalitions taken separately. The characteristic function shows the value of each possible coalition. To define a game

Game Theory

64

means to indicate the n players and to give the corresponding characteristic function. 2. Reward. A reward is any vector ω = {ω1 , . . . , ωn } whose components are real numbers, representing the payoff assigned to the players 1, 2, . . . , n, respectively. Obviously, a reward defined this way is a concept too vague to be of any value. It must be related somehow to the characteristic function of the game which measures the value of the possible coalitions of the players engaged in the game. This way, we arrive to a special class of rewards called imputations. 3. Imputation. An imputation is a reward ω = {ω1 , . . . , ωn } satisfying the following conditions: (a) Group rationality: n X

ωi = ν(N );

i=1

(b) Individual rationality: ωi ≥ ν({i}),

(i = 1, . . . , n).

Group rationality simply says that you cannot split among players more than the total payoff, or value, of the game, as measured by ν(N ), but the entire value ν(N ) available should be shared. Individual rationality, on the other hand, naturally requires that the reward ωi to player i cannot be smaller than the payoff ν({i}) that player i can get by playing alone. 4. Domination. An imputation ω ˜ = {˜ ω1 , . . . , ω˜n } dominates the imputation ω = {ω1 , . . . , ωn } with respect to the coalition S, and we write ω ˜ >S ω, if the following two conditions are satisfied: (a) Feasibility of ω ˜: X ω ˜ i ≤ ν(S); i∈S

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

(b) Improvement on S: ω ˜ i ≥ ωi ,

for every i ∈ S,

but there is i0 ∈ S such that: ω ˜ i0 > ωi0 . Feasibility means that, as far as the imputation ω ˜ is concerned, the members of coalition S cannot get more than the value ν(S) of coalition S. Improvement condition says that the imputation ω ˜ is better if it does not assign smaller rewards to the members of coalition S than the imputation ω and, at

Game Theory

65

the same time, at least one member of S gets more in ω ˜ than in ω. 5. Core. The set of nondominated imputations form the core of the game. Any imputation from the core is a von Neumann - Morgenstern solution of the game. The following theorem gives a necessary and sufficient condition for an imputation to belong to the core of the game: Proposition 4: An imputation ω = {ω1 , . . . , ωn } belongs to the core of the game if and only if: X ωi ≥ ν(S), i∈S

for each coalition S ⊆ N . Proof: (a) Assume that an imputation ω = {ω1 , . . . , ωn } satisfies the inequality from the theorem for each coalition S ⊆ N but that it does not belong to the core of the game. Therefore, there is an imputation ω ˜ = {˜ ω1 , . . . , ω ˜n} which dominates ω with respect to a certain coalition S, i.e., ω ˜ >S ω. Then, for this coalition S, we get: X ω ˜i, ν(S) ≥ i∈S

from the feasibility of ω ˜, X i∈S

ω ˜i >

X

ωi ,

i∈S

because ω ˜ dominates ω with respect to the coalition S, and: X

ωi ≥ ν(S),

i∈S

from the condition of the theorem supposed to be true. Summarizing, we may write:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

ν(S)

≥ ↑ feasibility

P

i∈S

ω ˜i

> ↑ domination

P

i∈S

ωi

≥ ν(S). ↑ condition

Therefore, we have obtained ν(S) > ν(S), which is absurd. Consequently, ω, satisfying the condition of the theorem, cannot be dominated by any imputation and, therefore, it does belong to the core of the game. (b) Conversely, let us take an imputation ω = {ω1 , . . . , ωn } that belongs to the core of the game and we want to show that it satisfies the condition of the theorem. Assume that it does not and, consequently, there is at least one coalition S such that: X ωi < ν(S). i∈S

Game Theory

66

As ω is an imputation, the group rationality implies: ν(N ) =

n X

ωi =

i=1

X

ωi +

X

ωj ,

j∈N −S

i∈S

where N − S is the set of those elements of N that do not belong to the coalition S, and, due to the superadditivity of the set function ν, we have: ν(N ) ≥ ν(S) + ν(N − S). Therefore, we get: X

ωj > ν(N − S),

j∈N −S

and using again the superadditivity of ν, we have: X

ν(N − S) ≥

ν({j}).

j∈N −S

From the last two inequalities, we conclude that there is at least one element j1 ∈ N − S, such that: ωj1 > ν({j1 }). Let us choose a real number a such that: 0 < a ≤ min{ν(S) −

X

ωi , ωj1 − ν({j1 })},

i∈S

and an arbitrary element i1 ∈ S. We finally define:

ω ˜i =

  ωi1 + a,             

if i = i1 ;

ωj1 − a,

if i = j1 ;

ωi ,

if i 6= i1 , i 6= j1 .

It is easy to see that ω ˜ = (˜ ω1 , . . . , ω ˜ n ) is an imputation. Indeed: n X

ω ˜i = ω ˜ i1 + ω ˜ j1 +

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

i=1

= (ωi1 + a) + (ωj1 − a) +

X

ω ˜i =

i6=i1 ,j1

X

i6=i1 ,j1

ωi =

n X

ωi = ν(N ),

i=1

because ω is an imputation. Also, the individual rationality of ω ˜ is implied by that of ω, because, taking into account the way in which the real number a has been introduced, we have: ω ˜ i ≥ ωi , ≥ ν({i}) (i = 1, . . . , n; i 6= i1 , i 6= j1 ),

Game Theory

67 ω ˜ i1 = ωi1 + a > ωi1 ≥ ν({i1 }), ω ˜ j1 = ωj1 − a ≥ ωj1 − [ωj1 − ν({j1 })] = ν({j1 }).

From the definition of ω ˜ , and of the real number a, we have: X

ω ˜i = ω ˜ i1 +

i∈S

X

ω ˜i =

i∈S,i6=i1

= ωi1 + a +

X

ωi =

i∈S,i6=i1

X

ωi + a ≤ ν(S),

i∈S

which shows that ω ˜ is indeed feasible with respect to the coalition S, and: ω ˜ i ≥ ωi ,

(i ∈ S);

ω ˜ i1 > ωi1 ,

which shows that indeed ω ˜ dominates ω with respect to the coalition S. This is absurd, because ω was assumed to be an imputation from the core of the game. Consequently, the inequality from the statement of the theorem is true for every coalition S ⊆ N . 2 6. Shapley value. Another solution of an n-person game is provided by the Shapley values assigned to the players, showing how powerful they are. More exactly, the Shapley value of the i-th player is: ωi =

X

[ν(S ∪ {i}) − ν(S)] pn (S),

S⊂N,i∈S /

where:

#(S)! (n − #(S) − 1)! , n! #(S) being the number of elements of the set S, and k! the factorial of the positive integer k, i.e.: pn (S) =

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

k! = (1)(2) . . . (k − 1)(k). Justification: If i is a player, i ∈ N , and S is a coalition which does not contain {i}, i.e., i ∈ / S, then player i’s contribution to joining S is just: ν(S ∪ {i}) − ν(S). But, let us imagine that coalition S, consisting of #(S) players, is formed first. There are: #(S)! = #(S)(#(S) − 1) . . . (2)(1) ways of forming the coalition S. Once the coalition S is formed, the new player, who wants to join in arrives. Eventually, the other n − #(S) − 1 players arrive, in: (n − #(S) − 1)! = (n − #(S) − 1)(n − #(S) − 2) . . . (2)(1)

Game Theory

68

ways.Therefore, the multiplication principle from statistics tells us that there are: #(S)! (n − #(S) − 1)!, ways of having the coalition S formed, followed by the arrival of a new player who wants to join in, and finally by the arrival of the other n − #(S) − 1 players. But there are n! permutation of the players from N . Therefore, the probability of forming a coalition S to which a new player could join in is: pn (S) =

#(S)! (n − #(S) − 1)! . n!

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Thus, the contribution Player i brings by joining in the coalition S, must be multiplied by the probability pn (S) that such a coalition is formed. Summing up with respect to all coalitions S that do not contain the Player i, we obtain the Shapley value ωi showing how powerful is this Player i. Problem 13: A chemist, assistant professor at a university, with an annual salary of $50,000, has just obtained the formula for a new drug. He cannot manufacture the drug by himself but two small companies, denoted by A and B, respectively, have the technology and manpower necessary to make the drug separately or working together. Currently, each company makes a profit of $200,000 per year, when they work separately, and $600,000 per year, when they work together. The chemist is willing to take a leave of absence from the university, for one year, and collaborate on a full time basis with either company A or company B, or with both companies, which are both willing to focus only on the manufacturing the new drug. The chemist and one of the companies, working together only on the new drug, could make an extra profit of $1,000,000 per year. Finally, if the chemist and both companies are working together only on the new drug project, the extra profit could be $1,500,000 per year. The main difficulty is to decide how to share the annual extra profit. (a) Find the characteristic function of this three-person game; (b) Determine the core of the game and find a von Neumann-Morgenstern solution of the game; (c) Calculate the Shapley value for each player. Remark: This is a variant of the ‘drug game’ analyzed by W.L. Winston ([37], p.854). The changes were needed to make it more realistic. 2 Solution: (a) We have n = 3, and N = {1, 2, 3}, where 1 = the chemist, 2 = company A, and 3 = company B. The possible 23 = 8 coalitions are: P(N ) = {∅, {1}, {2}, {3}, {1, 2}, {1, 3}, {2, 3}, {1, 2, 3}}, and the characteristic function, reflecting the value of each coalition, is:

Game Theory

69 ν(∅) = 0;

ν({1, 2}) = 1, 000, 000;

ν({1}) = 50, 000;

ν({1, 3}) = 1, 000, 000;

ν({2}) = 200, 000;

ν({2, 3}) = 600, 000;

ν({3}) = 200, 000;

ν({1, 2, 3}) = 1, 500, 000.

(b) Let ω = (ω1 , ω2 , ω3 ) be an imputation belonging to the core. Being an imputation, we have: Group rationality: ω1 + ω2 + ω3 = ν(N ) = ν({1, 2, 3}) = 1, 500, 000. Individual rationality: ω1 ≥ ν({1}) = 50, 000; ω2 ≥ ν({2}) = 200, 000; ω3 ≥ ν({3}) = 200, 000. As ω = (ω1 , ω2 , ω3 ) belongs to the core of the game then, according to Proposition 4, we must have: X

ωi ≥ ν(S),

i∈S

for every coalition S ⊆ N . Therefore, the following inequalities hold: ω1 ≥ ν({1}) = 50, 000, ω2 ≥ ν({2}) = 200, 000, ω3 ≥ ν({3}) = 200, 000, which are just the inequalities induced by the individual rationality, and:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

ω1 + ω2 ≥ ν({1, 2}) = 1, 000, 000, ω1 + ω3 ≥ ν({1, 3}) = 1, 000, 000, ω2 + ω3 ≥ ν({2, 3}) = 600, 000, ω1 + ω2 + ω3 ≥ ν({1, 2, 3}) = 1, 500, 000, where the last inequality has to become an equality constraint due to the group rationality requirement.

Game Theory

70

Let us analyze the implications of these linear constraints. We have: ω2 + ω3 = 1, 500, 000 − ω1 ≤ 1, 450, 000, ω1 + ω3 = 1, 500, 000 − ω2 ≤ 1, 300, 000, ω1 + ω2 = 1, 500, 000 − ω3 ≤ 1, 300, 000, and, therefore: 600, 000 ≤ ω2 + ω3 ≤ 1, 450, 000, 1, 000, 000 ≤ ω1 + ω3 ≤ 1, 300, 000, 1, 000, 000 ≤ ω1 + ω2 ≤ 1, 300, 000. Thus, we have: ω3 = 1, 500, 000 − (ω1 + ω2 ) ≤ 1, 500, 000 − 1, 000, 000 = 500, 000, ω2 = 1, 500, 000 − (ω1 + ω3 ) ≤ 1, 500, 000 − 1, 000, 000 = 500, 000, ω1 = 1, 500, 000 − (ω2 + ω3 ) ≤ 1, 500, 000 − 600, 000 = 900, 000, which, give: 200, 000 ≤ ω3 ≤ 500, 000, 200, 000 ≤ ω2 ≤ 500, 000, 50, 000 ≤ ω1 ≤ 900, 000. The above inequalities, taken two at a time, imply: 400, 000 ≤ ω2 + ω3 ≤ 1, 000, 000, 250, 000 ≤ ω1 + ω3 ≤ 1, 400, 000, 250, 000 ≤ ω1 + ω2 ≤ 1, 400, 000. From the above inequalities, we get the form of the imputations that belong to the core: 600, 000 ≤ ω2 + ω3 ≤ 1, 000, 000, Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

ω1 = 1, 500, 000 − (ω2 + ω3 ), or: 1, 000, 000 ≤ ω1 + ω3 ≤ 1, 300, 000, ω2 = 1, 500, 000 − (ω1 + ω3 ), or: 1, 000, 000 ≤ ω1 + ω2 ≤ 1, 300, 000, ω3 = 1, 500, 000 − (ω1 + ω2 ).

Game Theory

71

In order to get some von Neumann-Morgenstern solutions of the game, let us notice that: ω2 + ω3 = 600, 000 implies ω1 = 900, 000, whereas: ω2 + ω3 = 1, 000, 000 implies ω1 = 500, 000. In particular, if the two companies, which are symmetric in fact, have an equal share of the profit, i.e., ω2 = ω3 , then if ω2 (and therefore ω3 too) increases from the minimum amount of $300,000 to the maximum amount of $500,000, then ω1 , the payoff of the chemist, decreases from the maximum value of $900,000 to the minimum value of $500,000. We can see that the chemist is the most important player in of the game but an equal share of the profit among the three players is still an imputation of the core and, therefore a von Neumann-Morgenstern solution of the game. (c) In this context, the Shapley value of Player i is: ωi =

X

[ν(S ∪ {i}) − ν(S)] p3 (S),

S⊂N,i∈S /

where:

#(S)! (3 − #(S) − 1)! . 3! For instance, taking S = ∅ and S = {1}, we have: p3 (S) =

p3 (∅) =

2 0! (3 − 0 − 1)! = , 3! 6

p3 ({1}) =

1! (3 − 1 − 1)! 1 = . 3! 6

For S = {1, 2}, we have: p3 ({1, 2}) =

2 2! (3 − 2 − 1)! = . 3! 6

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

The first table below contains everything we need for calculating the Shapley value of Player 1. Using it, we get: 1 1 2 2 + 800, 000 + 800, 000 + 900, 000 = 583, 333 ω1 = 50, 000 6 6 6 6  

 

 

 

The second table below contains everything we need for calculating the Shapley value of Player 2. Using it, we get: ω2 = 200, 000

1 1 2 2 + 950, 000 + 400, 000 + 500, 000 = 458, 333 6 6 6 6

 

 

 

 

Because the two companies are symmetric, we get also ω3 = 458, 333.

Game Theory

72 S

S ∪ {1}

ν(S ∪ {1}) − ν(S) p3 (S)

∅

{1}

50,000

2/6

{2}

{1, 2}

800,000

1/6

{3}

{1, 3}

800,000

1/6

{2, 3}

{1, 2, 3}

900,000

2/6

S

S ∪ {2}

∅

{2}

200,000

2/6

{1}

{1, 2}

950,000

1/6

{3}

{2, 3}

400,000

1/6

{1, 3}

{1, 2, 3}

500,000

2/6

ν(S ∪ {2}) − ν(S) p3 (S)

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Therefore, according to the Shapley value, the most powerful player is the chemist who has done research and created the formula of the new drug, but each of the two companies, however, has a considerable power, which is understandable because without them the drug could not be manufactured anyway. 2

Decision Theory

73

Chapter 2. DECISION THEORY

ABSTRACT

The decision maker has to find the best action for minimizing his loss, facing Nature which could be in one of several possible states. If the only available information is provided by a decision tableau, assigning real numbers to the loss associated to each possible action of the decision maker and each possible state of Nature, and by a prior probability distribution on the set of possible states of Nature, several criteria for taking the optimum action (the minimax criterion, Savage’s minimax regret criterion, Hurwicz’s criterion, Laplace’s criterion, Bayes’s criterion without experimentation) are given. When an auxiliary experiment can be used, providing indirect, probabilistic information about the possible states of Nature, Bayes’s criterion with experimentation is analyzed. The risk function and the multiple-stage decision are also presented. 1. Generalities

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

A decision making process involves the following components: (a) ‘Nature’: the generic environment, that can be in one state from the set of possible states Θ = {θ1 , . . . , θn }; (b) ‘Decision maker’: a rational entity, supposed to select an action from the following set of possible actions A = {a1 , . . . , am }; (c) ‘Loss function’: a rule that assigns a real number `(ai , θj ) (representing the loss for the decision maker) corresponding to each pair (action ai , state of Nature θj ). The matrix [`(ai , θj )] is called decision matrix. A decision tableau is denoted by: θ1

...

θn

a1 .. .

`(a1 , θ1 ) .. .

...

`(a1 , θn ) .. .

am

`(am , θ1 ) . . .

`(am , θn )

Decision Theory

74

To give a simple example, below is a decision tableau about what to do in the morning before leaving: Fair Weather Rain Take umbrella 3 0 Do not take umbrella 0 6 A decision making process may be: (a) ‘Under certainty’, when there is only one possible state of Nature θ. Such a case is trivial and the optimum action a∗ is chosen to minimize the loss function corresponding to this unique state of Nature, i.e., `(a∗ , θ) = mini `(ai , θ); (b) ‘Under uncertainty’, when there are several possible states of Nature and the model is essentially probabilistic. This is the case which interests us in this chapter. The decision making process may also be classified in two broad categories: (a) ‘Decision without experimentation’, when all information available consists of the decision table and, maybe, a prior probability distribution on the set of possible states of Nature; (b) ‘Decision with experimentation’, when, except a decision table and a prior probability distribution on the set of possible states of Nature, we have the possibility to benefit from the outcome of an auxiliary experiment which could provide indirect probabilistic information about the states of Nature. In such a case, the decision maker has to evaluate the value of the auxiliary experiment first and, eventually, if this experiment proves to be worth doing, to use its outcome in order to take the optimum action.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

2. Oil-drilling example The oil-drilling example was analyzed by F.S. Hillier and G.J. Lieberman more than 30 years ago. It is a very representative type of problem from decision theory and this is the reason why we are referring to it here, but its analysis will be slightly different in what follows. Also, all the decision making criteria that are presented in this chapter will be tested on the oildrilling example, to see in what ways their solutions are identical, similar, or simply different. The data values involving money are kept as in the original example even though, due to inflation, all the figures should be multiplied by a considerable amount. But, again, what really matters is the way of solving such a complex decision problem and not the numbers themselves. Ewing family buys a new piece of land in a region rich in oil. Such a piece of land there could be classified as one of the following four categories (the states of Nature):

Decision Theory θ1 : θ2 : θ3 : θ4 :

75

500,000-barrel well (‘very rich well’); 250,000-barrel well (‘rich well’); 50,000-barrel well (‘poor well’); 0-barrel well (‘dry well’).

Ewing family has to decide which of the following three actions to take: a1 : ‘Drill for oil’, knowing that the profit is $1.5 per barrel and the cost of drilling is $100,000 for a producing well and $75,000 if the well proves to be dry; a2 : ‘Unconditionally lease the land’ to an independent driller for a fixed amount of $45,000; a3 : ‘Conditionally lease the land’ to an independent driller with a profit of $0.50 per barrel if the well proves to be very rich or rich and nothing if the well is poor or dry. The above information allows us to get the corresponding decision table. Indeed, we have (in dollars): −`(a1 , θ1 ) = (500, 000)(1.5) − 100, 000 = 650, 000; −`(a1 , θ2 ) = (200, 000)(1.5) − 100, 000 = 200, 000; −`(a1 , θ3 ) = (50, 000)(1.5) − 100, 000 = −25, 000; −`(a1 , θ4 ) = (0)(1.5) − 75, 000 = −75, 000; −`(a2 , θj ) = 45, 000 (j = 1, 2, 3, 4); −`(a3 , θ1 ) = (500, 000)(0.5) = 250, 000; −`(a3 , θ2 ) = (200, 000)(0.5) = 100, 000; −`(a3 , θ3 ) = −`(a3 , θ4 ) = 0.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

We obtain the decision tableau:

a1 a2 a3

θ1 θ2 θ3 θ4 −650, 000 −200, 000 25,000 75,000 −45, 000 −45, 000 −45, 000 −45, 000 −250, 000 −100, 000 0 0

Looking into what happened in the past years with the pieces of land from that region where the owners had decided to drill for oil, we learn that 10% of the wells proved to be very rich, 15% rich, 25% poor, and 50% dry. They are the components of a prior probability distribution on the set of possible states of Nature:

Decision Theory

76 θj θ1 θ2 θ3 θ4 P (θj ) 0.10 0.15 0.25 0.50

Before deciding whether to drill for oil or lease the land, Ewing family could ask for the services of a company specialized in taking seismic records on the land. This is something that could be done relatively easy. A microseism (an explosion) is performed at a certain point of the land, sending waves in the ground. Several geophones, planted about 25 meters apart, are registering the waves reflected by the different underground strata, of different geological compositions, and these waves are collected at a central station. The seismic traces are processed by the computer, filtering the ground noise and correlating the spikes of the reflected waves registered by the geophones. In the end, the geometry of the underground strata is obtained, down to a depth of about 8,000 meters. The seismic records do not tell us whether there is or not oil underground but they tell us what kind of geological structure we have underground. Specialists in geology distinguish four kinds of structures: very favourable, denoted by x1 , favourable, denoted by x2 , relatively unfavourable, denoted by x3 , and unfavourable, denoted by x4 , geological structures relative to hiding oil and gas underground. The company Prospect Ltd, asks $12,000 for taking seismic records on the land. The correspondence between the kind of geological structure (very favourable, favourable, relatively unfavourable, unfavourable) obtained before drilling for oil and the kind of well (very rich, rich, poor, dry) obtained after drilling is not a rigid one. This means that often a rich well has been associated with a very favourable geological structure, but not always, and there were cases when in a favourable geological structure the well, after drilling, proved to be poor or even dry. The past experience the company had in 100 cases where seismic records were taken and the drilling was subsequently done is shown in the tableau given below:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

θ1 θ2 θ3 θ4 x1 7 9 11 9 x2 4 3 6 13 x3 1 2 3 15 x4 0 2 4 11 Total 12 16 24 48 It is quite easy to read this tableau. The entry from the second row and third column, for instance, tells us that, out of 100 cases, in 6 cases the seismic records showed a favourable geological structure before drilling and the well proved to be poor after drilling, Dividing the entries of each column by the sum of all entries from the respective column, we obtain the so called prediction matrix shown below. It is a stochastic matrix because in each column we have a probability distribution. The entry from row k and column j, denoted by P (xk |θj ) represents the probability of outcome xk if (given that, conditioned

Decision Theory

77

by) the state of Nature is θj . Thus, P (x1 |θ2 ) = 9/16 which means that, based on the past experience of Prospect Co. Ltd, the probability of having a very favourable geological structure if the well is rich is 9/16=0.5625. x1 x2 x3 x4

θ1 7/12 4/12 1/12 0

θ2 θ3 θ4 9/16 11/24 9/48 3/16 6/24 13/48 2/16 3/24 15/48 2/16 4/24 11/48

Taking seismic records is an auxiliary experiment whose result does not tell us what state of Nature we really have. It gives us only an indirect information because it will allow us to calculate the so called retrodiction matrix whose general entry is the ‘inverse’ probability P (θj |xk ) of having the state of Nature θj if the seismic record proves to be xk . Generally, there is a certain global cost of the auxiliary experiment, like $12,000 asked by Prospect Co. Ltd. It is possible, however, for the cost to be different for different possible outcomes of the auxiliary experiment. This describes the maximum kinds of information available in decision making under uncertainty. If, however, the only information available consists of the loss matrix and, maybe, the prior probability distribution on the possible states of Nature, we have to deal with decision without experimentation. If we also know a prediction matrix and the cost of the auxiliary experiment, we deal with decision with experimentation. In such a case we have to decide first whether or not to perform the auxiliary experiment. If the answer is ‘no’, then we resign ourselves to making decision without experimentation. If the answer is ‘yes”, then we have to see how the outcome of the auxiliary experiment obtained after performing the experiment could be used for getting the best action of minimum loss.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

3. Decision without experimentation Let A = {a1 , . . . , am } be the set of possible actions of the decision maker and Θ = {θ1 , . . . , θn } be the set of possible states of Nature. Assume that we have a decision making problem when the only information available is the loss m × n matrix [`(ai , θj )]. How to choose the best action? 1. Minimax criterion. To somebody familiar with game theory, the first idea coming to his mind is that the situation here seems to be like in a twoperson zero-sum game: the decision maker is Player I, Nature is Player II, and the loss matrix, if each entry changes its sign, is the payoff matrix of Player I. But in such a case, the maximin strategy of Player I, in terms of his playoff matrix, becomes the minimax strategy, in terms of his loss matrix. Therefore,

Decision Theory

78

the two-person zero-sum game suggests the choosing of the action a∗ for which we get: min max `(ai , θj ). i

j

The minimax criterion is a conservative, pessimistic viewpoint in decision making. It assumes that Nature is acting against us, like the opponent in a two-person zero-sum game. Nature, however, is not a malevolent opponent and this makes us look for other decision criteria. In the oil-drilling example: A = {a1 , a2 , a3 },

(m = 3),

Θ = {θ1 , θ2 , θ3 , θ4 },

(n = 4),

and the loss 3 × 4 matrix [`(ai , θj )], in thousands of dollars, is: a1 a2 a3

θ1 θ2 θ3 θ4 −650 −200 25 75 −45 −45 −45 −45 −250 −100 0 0

We add a new column containing the maximum value in each row of the loss matrix and select the minimum value in that new column: a1 a2 a3

θ1 θ2 θ3 θ4 −650 −200 25 75 −45 −45 −45 −45 −250 −100 0 0

max 75 −45 0

Thus, according to the minimax criterion, borrowed from game theory, the optimum action is a∗ = a2 , which means ‘unconditionally lease the land to an independent driller for $45,000’. 2. Savage’s minimax regret criterion. We first calculate the regret matrix: r(ai , θj ) = `(ai , θj ) − min `(ai , θj ), i

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

and then we apply the minimax criterion to the regret matrix: min max r(ai , θj ). i

j

The number r(ai , θj ) reflects the regret of the decision maker for not taking the optimum action when the state of Nature is θj . Indeed, if ai is the optimum action when the the state of Nature is θj , then the respective loss is minimum, namely: `(ai , θj ) = min `(ai , θj ), i

Decision Theory

79

and the corresponding regret r(ai , θj ) is equal to zero, which means no regret. To see the difference between the minimax criterion and Savage’s minimax regret criterion, let us take a simple 2 × 2 decision tableau, whose entries are dollars, as given below: a1 a2

θ1 θ2 10,000 10,000 10,100 100

According to the minimax criterion, the optimal action is the minimax action a∗ = a1 , as can be seen from the tableau given below: a1 a2

θ1 θ2 10,000 10,000 10,100 100

max 10,000 10,100

Selecting the minimum entry in each column of the initial decision tableau, we have: θ1 θ2 a1 10,000 10,000 a2 10,100 100 min 10,000 100 and, subtracting the entry from the last row from each entry of the respective column, we get the regret matrix to which we apply the minimax criterion which gives the optimum action to be a∗ = a2 : a1 a2

θ1 θ2 max 0 9,900 9,900 100 0 100

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Playing a two-person zero-sum game against a hostile opponent would certainly make a1 the best action but, as Nature is not necessarily a malevolent opponent, a2 obviously appears to be a better option because taking it, even if the loss corresponding to θ1 is slightly bigger, the loss corresponding to θ2 is considerably smaller then what happens when we take the action a1 . In the oil-drilling example: A = {a1 , a2 , a3 },

(m = 3),

Θ = {θ1 , θ2 , θ3 , θ4 },

and the loss 3 × 4 matrix [`(ai , θj )], in thousands dollars, is: θ1 θ2 θ3 θ4 a1 −650 −200 25 75 a2 −45 −45 −45 −45 a3 −250 −100 0 0 min −650 −200 −45 −45

(n = 4),

Decision Theory

80

and, therefore, the minimax applied to the regret matrix given below tells us that, according to Savage’s criterion, the optimum action is a∗ = a1 , which means ‘drill for oil’: θ1 θ2 θ3 θ4 0 0 70 120 605 155 0 0 400 100 45 45

a1 a2 a3

max 120 605 400

3. Hurwicz’s criterion. As Nature could act in our favour or against us, a cautious attitude would be to balance these extreme situations. This is just what Hurwicz’s criterion does. If the decision maker takes the action ai and Nature is acting in his favour, we have: min `(ai , θj ). j

If the decision maker takes the action ai and Nature is acting against him, we have: max `(ai , θj ). j

A balance between these two cases is given by: α min `(ai , θj ) + (1 − α) max `(ai , θj ), j

j

where 0 ≤ α ≤ 1 is a real number called ‘index of optimism’. The decision maker will choose his action to minimize this linear convex combination between the two extreme cases. Thus, Hurwicz’s criterion is: min {α min `(ai , θj ) + (1 − α) max `(ai , θj )}. i

j

j

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

We notice that if the index of optimism α = 0, Hurwicz’s criterion becomes the minimax criterion. If nothing is said about the index of optimism it is assumed to be α = 0.5. In the oil-drilling example, taking α = 0.5, we have: a1 a2 a3

θ1 θ2 θ3 θ4 min max 0.5 min + 0.5 max −650 −200 25 75 −650 75 −287.5 −45 −45 −45 −45 −45 −45 −45 −250 −100 0 0 −250 0 −125

which shows that the optimum action according to Hurwicz’s criterion is a∗ = a1 , ‘drill for oil’. 4. Laplace’s criterion. Long ago, in his treaty on probability theory, Laplace mentioned the so called ‘Principle of insufficient reason’, according

Decision Theory

81

to which if we have no reasons to discriminate between several possible events, then the best attitude is to take them as being equally likely. This was perhaps the first attempt at dealing with subjective probabilities. We apply this principle, for instance, when tossing a coin we assign equal probabilities to the two possible outcomes, tail or face, without checking whether the coin is really fair. In decision making, Laplace’s criterion is applied when, having no reasons to discriminate between the possible states of Nature, we take them as being equally likely. Let A = {a1 , . . . , am } be the set of possible actions of the decision maker, Θ = {θ1 , . . . , θn } be the set of possible states of Nature, and [`(ai , θj )] be the m × n loss matrix. Assuming the states of Nature to be equally likely, we calculate the expected loss corresponding to each action, namely: n 1 X `(ai , θj ), `(ai ) = n j=1

(i = 1, . . . , m).

According to Laplace’s criterion, the optimum action a∗ is the one for which the minimum of the expected loss is obtained: `(a∗ ) = min `(ai ). i

In the oil-drilling example, m = 3, n = 4, and from the loss matrix: a1 a2 a3

θ1 θ2 θ3 θ4 −650 −200 25 75 −45 −45 −45 −45 −250 −100 0 0

we calculate the expected loss: `(ai ) =

4 1X `(ai , θj ), 4 j=1

(i = 1, 2, 3, 4)

Thus, we obtain the numerical values:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

`(a1 ) = [(−650) + (−200) + (25) + (75)]/4 = −187.5, etc. In dollars, we obtain: `(ai ) a1 −187, 500 a2 −45, 000 a3 −87, 500 min −187, 500

Decision Theory

82

which shows that the optimum action according to Laplace’s criterion is again a∗ = a1 , ‘drill for oil’, and the corresponding expected loss is `(a1 ) = −187, 500, which means a gain of $187,500. 5. Bayes’s criterion without experimentation. In decision without experimentation, Bayes’s name is used when a prior probability distribution on the set of possible states of Nature is given, namely: P (θj ) > 0, (j = 1, . . . , n),

n X

P (θj ) = 1,

j=1

along with the m × n loss matrix: [`(ai , θj )],

(i = 1, . . . , m; j = 1, . . . , n).

We calculate the expected loss corresponding to each action of the decision maker, namely: `(ai ) =

n X

`(ai , θj ) P (θj ),

(i = 1, . . . , m),

j=1

and the optimum action a∗ corresponds to the minimum expected loss: `(a∗ ) = min `(ai ). i

In the oil-drilling example, we use the given prior probability distribution on the set of possible states of Nature: θj θ1 θ2 θ3 θ4 P (θj ) 0.10 0.15 0.25 0.50 and the loss matrix, in thousands of dollars:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

a1 a2 a3

θ1 θ2 θ3 θ4 −650 −200 25 75 −45 −45 −45 −45 −250 −100 0 0

in order to calculate the expected loss corresponding to each action, namely: `(ai ) =

4 X

`(ai , θj ) P (θj ),

(i = 1, 2, 3, 4).

j=1

Thus we obtain the numerical values: `(a1 ) = (−650)(0.10) + (−200)(0.15) + (25)(0.25) + (75)(0.50) = −51.25,

Decision Theory

83

etc.,and the expected losses, in dollars, are shown in the table below. Selecting the minimum expected loss, we conclude that the optimum action according to Bayes’s criterion without experimentation is again a∗ = a1 , ‘drill for oil’, and the corresponding expected loss, in dollars, is: EBayes,prior = `(a1 ) = −$51,250, as can be seen from the tableau below: a1 a2 a3 min

`(ai ) −51, 250 −45, 000 −40, 000 −51, 250

We have a perfect decision when we choose the optimum action for every state of Nature. This would happen, for instance, if somebody would inform us in which state Nature is. In such a case, if the state of Nature is θj , we take the action a∗ corresponding to: min `(ai , θj ). i

Accordingly, the expected loss corresponding to perfect information is: Eperf ect,inf ormation =

n X

j=1

{min `(ai , θj )} P (θj ). i

Therefore, the value of perfect information is given by: VPI = EBayes,prior − Eperf ect,inf ormation .

We notice that VPI cannot be negative. In the oil-drilling example, we use the given prior probability distribution on the set of possible states of Nature:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

θ1 θ2 θ3 θ4 θj P (θj ) 0.10 0.15 0.25 0.50 and the loss matrix, in thousands of dollars, with an extra row containing the minimum loss in each column: θ1 θ2 θ3 θ4 a1 −650 −200 25 75 a2 −45 −45 −45 −45 a3 −250 −100 0 0 min −650 −200 −45 −45

Decision Theory

84

We calculate: Eperf ect,inf ormation =

4 X

j=1

{min `(ai , θj )} P (θj ) = i

= (−650)(0.10) + (−200)(0.15) + (−45)(0.25) + (−45)(0.50) = −128.75. Before, we have already evaluated the minimum expected loss: EBayes,prior = min i

4 X

`(ai , θj ) P (θj ) = −51.25.

j=1

Therefore, in dollars, the value of perfect information in the oil-drilling example is: V P I = EBayes,prior − Eperf ect,inf ormation = = (−51, 250) − (−128, 750) = $77, 500. 4. Decision with experimentation Assume that in a decision problem the following information is available: 1. The ‘decision maker’ has to choose one of the following possible actions: A = {a1 , . . . , am }. 2. ‘Nature’ could be in one of the following possible states: Θ = {θ1 , . . . , θn }. 3. The ‘loss matrix’ for the decision maker is: [`(ai , θj )],

(i = 1, . . . , m; j = 1, . . . , n),

where every `(ai , θj ) is a real number (a negative loss is a gain). 4. The ‘prior probability distribution’ on the possible states of Nature is: P (θj ) > 0, (j = 1, . . . , n);

n X

P (θj ) = 1.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

j=1

5. An ‘auxiliary experiment’, once it is performed, gives one of the possible outcomes: X = {x1 , . . . , xs }. 6. Based on past experience, the ‘prediction matrix’ is: [P (xk |θj )],

(k = 1, . . . , s; j = 1, . . . , n),

Decision Theory

85

where P (xk |θj ) is the probability of having the outcome xk of the auxiliary experiment if the state of Nature is θj , such that: P (xk |θj ) ≥ 0, s X

(k = 1, . . . , s; j = 1, . . . , n),

P (xk |θj ) = 1,

for every j, (j = 1, . . . , n).

k=1

7. The ‘cost’ of the auxiliary experiment corresponding to each possible outcome is: c(xk ) ≥ 0, (k = 1, . . . , s). This is the data set available. There are two steps in such a decision process. First, we have to decide whether or not to perform the auxiliary experiment. Second, if the answer is ‘no’, then we do decision without experimentation and try all the criteria mentioned in the corresponding section 3, whereas, if the answer is ‘yes’, we obtain a certain outcome of the auxiliary experiment and we have to determine how to use it for finding the corresponding optimum action in order to minimize the expected loss. The basic problem here is how to use the prior probability distribution [P (θj )] and the prediction stochastic matrix [P (xk |θj )] for getting the retrodiction stochastic matrix [P (θj |xk )], where P (θj |xk ) is the posterior probability of state of Nature θj if (given, conditioned by) the outcome xk of the auxiliary experiment occurs. We should obviously have: P (θj |xk ) ≥ 0, n X

(j = 1, . . . , n; k = 1, . . . , s),

P (θj |xk ) = 1,

for every k, (k = 1, . . . , s).

j=1

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

The stochastic matrix [P (xk |θj )] is a random correspondence from the set Θ to the set X, such that to every θj ∈ Θ it corresponds every xk ∈ X with probability P (xk |θj ). The inverse random correspondence [P (θj |xk )] from X to Θ would assign to every xk ∈ X every θj ∈ Θ with the probability P (θj |xk ). But how to define this inverse random correspondence from X to Θ? Long ago, Bayes suggested the formula: P (θj |xk ) =

P (xk |θj ) P (θj ) , P (xk )

where the probability of the outcome xk , taking all possible states of Nature into account, is: P (xk ) =

n X

j=1

P (xk |θj ) P (θj ).

Decision Theory

86

If we accept this particular random correspondence [P (θj |xk )] from the set of possible outcomes X to the set of possible states of Nature Θ to be the inverse of the random correspondence [P (xk |θj )] from the set of the possible states of Nature to the set of possible outcomes, given the prior probability [P (θj )] on the set Θ, this means that we adopt the Bayesian viewpoint in decision theory. P (x|θ) ?

P (θ)

Θ

X

P (x)

6

P (θ|x) Figure 4. Direct and inverse random correspondences. Thomas Bayes (1702-1761) was an English clergyman who proposed a formula, known today as Bayes’s formula, for revising the prior probability distribution in light of new evidence. This formula was discussed in his paper ‘An essay towards solving a problem in the doctrine of chances’ but, for unknown reasons, he did not publish his paper. Fortunately, his friend Richard Price found the manuscript among Bayes’s papers and sent it to the Royal Society in London. The paper was published in 1763, two years after Bayes’s death.

Once the retrodiction matrix [P (θj |xk )] is obtained, we calculate the expected loss if we take the action ai , given (conditioned by) the outcome xk of the auxiliary experiment, namely: `(ai |xk ) =

n X

`(ai , θj ) P (θj |xk ) + c(xk ),

j=1

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

for every i, (i = 1, . . . , m) and k, (k = 1, . . . , s). Notice that if xk is the outcome of the auxiliary experiment then the corresponding cost c(xk ) is an extra loss for the decision maker. Once the expected loss matrix [`(ai |xk )] is obtained, let us denote by a∗ (xk ) the optimum action given (conditioned by) the outcome xk of the auxiliary experiment, and denote by: `(a∗ (xk )) = `(a∗ (xk )|xk ) = min `(ai |xk ) i

the minimum expected loss if the outcome of the auxiliary experiment is xk . But the outcome xk occurs with probability P (xk ). The expected loss for the decision maker if the auxiliary experiment is performed is: EBayes,posterior =

s X

k=1

`(a∗ (xk )) P (xk ).

Decision Theory

87

In section 4 we have already defined the minimum expected loss if only the loss matrix [`(ai , θj )] and the prior probability distribution [P (θj )] are taken into account, namely: EBayes,prior = min i

n X

`(ai , θj ) P (θj ).

j=1

The value of the auxiliary experiment is defined by: VAE = EBayes,prior − EBayes,posterior .

(a) If VAE > 0, we do perform the auxiliary experiment and whatever outcome xk the auxiliary experiment gives, the optimum action is a∗ (xk ), and the corresponding expected loss is `(a∗ (xk )). (b) If VAE ≤ 0, we do not perform the auxiliary experiment and we apply the minimax, Savage’s, Hurwicz’s, Laplace’s, and Bayes’s criteria for decision making without experimentation. In the oil-drilling example: A = {a1 , a2 , a3 },

(m = 3),

Θ = {θ1 , θ2 , θ3 , θ4 },

X = {x1 , x2 , x3 , x4 },

(n = 4),

(s = 4).

1. The prior probability distribution [P (θj )] on the set of possible states of Nature is: θ1 θ2 θ3 θ4 θj P (θj ) 0.10 0.15 0.25 0.50 2. The 3 × 4 loss matrix [`(ai , θj )], in thousands of dollars, is:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

θ1 θ2 θ3 θ4 a1 −650 −200 25 75 a2 −45 −45 −45 −45 a3 −250 −100 0 0 min −650 −200 −45 −45 3. The prediction matrix [P (xk |θj )] is: x1 x2 x3 x4

θ1 7/12 4/12 1/12 0

θ2 θ3 θ4 9/16 11/24 9/48 3/16 6/24 13/48 2/16 3/24 15/48 2/16 4/24 11/48

4. The cost [c(xk )] of the auxiliary experiment, in thousands of dollars, is:

Decision Theory

88 xk x1 x2 x3 c(x) 12 12 12

x4 12

This is the data set available. We calculate first the probability distribution on the set of possible outcomes: P (xk ) =

4 X

P (xk |θj ) P (θj ),

(k = 1, 2, 3, 4).

j=1

We get the numerical values: P (x1 ) =

7 9 11 9 (0.10) + (0.15) + (0.25) + (0.50) = 0.351042, 12 16 24 48

for the first possible outcome. For the other three possible outcomes, we get: xk x1 x2 x3 x4 P (xk ) 0.351042 0.259375 0.214583 0.175 Applying Bayes’s formula: P (θj |xk ) =

P (xk |θj ) P (θj ) , P (xk )

(j, k = 1, 2, 3, 4),

we calculate: P (θ1 |x1 ) =

P (x1 |θ1 ) P (θ1 ) (7/12)(0.10) = = 0.166172, P (x1 ) 0.351042

etc., and we get the retrodiction matrix [P (θj |xk )], namely: θ1 θ2 θ3 θ4

x1 0.166172 0.240356 0.326409 0.267062

x2 x3 x4 0.128514 0.038835 0.000000 0.108434 0.0873786 0.107143 0.240964 0.145631 0.238095 0.522088 0.728155 0.654762

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

The loss matrix and the retrodiction matrix are used to calculate the expected loss corresponding to each possible action conditioned by each possible outcome of the auxiliary experiment according to the formula: `(ai |xk ) =

4 X

`(ai , θj ) P (θj |xk ) + c(xk ),

j=1

for (i = 1, 2, 3) and (k = 1, 2, 3, 4). Thus, in thousands of dollars: `(a1 |x1 ) =

4 X

j=1

`(a1 , θj ) P (θj |x1 ) + c(x1 ) =

Decision Theory

89

= (−650)(0.166172) + (−200)(0.240356) + (25)(0.326409)+ +(75)(0.267062) + 12 = −115.893, and so on for the other actions and outcomes. The matrix [`(ai |xk )] of the expected loss, in thousands of dollars, conditioned by the outcomes of the auxiliary experiment, is: x1 −115.893 −33 −53.5786

a1 a2 a3

x2 −48.0402 −33 −30.9719

x3 x4 27.534 45.631 −33 −33 -6.4466 1.2857

where the minimum entry in each column is put inside a box. Therefore, the minimum expected loss `(a∗ (xk )), in thousands of dollars, and the optimum action a∗ (xk ) corresponding to each possible outcome xk of the auxiliary experiment are given in the following tableau: x1 x2 x3 x4

a∗ (xk ) `(a∗ (xk )) a1 −115.893 a1 −48.0402 a2 −33 a2 −33

We obtain, in thousands of dollars: EBayes,posterior =

4 X

`(a∗ (xk )) P (xk ) = (−115.893)(0.351042)+

k=1

+(−48.0402)(0.259375) + (−33)(0.214583) + (−33)(0.175) = −66, which, in dollars, means: EBayes,posterior = −$66,000.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

On the other hand, in section 3, on decision without experimentation, subsection 5, dealing with Bayes’s criterion without experimentation, we obtained, in thousands of dollars: EBayes,prior = min i

4 X

`(ai , θj ) P (θj ) =

j=1

= min {−51.25, −45, −40} = −51.25, which, in dollars, is: EBayes,prior = −$51,250.

Decision Theory

90

Therefore, the value of the auxiliary experiment is: VAE = EBayes,prior − EBayes,posterior =

= (−51, 250) − (−66, 000) = $14,750 > 0. The conclusion is that, in this case, it is worth performing the auxiliary experiment. Now, from the previous analysis, we are well-prepared to choose the best action once the outcome of the auxiliary experiment is handed to us. Thus, if the outcome is x1 or x2 , then the optimum action is a1 , ‘drill for oil’, with an expected gain of $115,893 or $48,040.20, respectively; if the outcome is x3 or x4 , then the optimum action is a2 , ‘unconditionally lease the land’ to an independent driller and the gain is $33,000. 5. Risk function

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Assume that the decision maker has to choose one of the following possible actions: A = {a1 , . . . , am }. Nature can be in one of the possible states Θ = {θ1 , . . . , θn }. If an auxiliary experiment is performed, one of the following possible outcomes X = {x1 , . . . , xs } occurs. Let [`(ai , θj )] be the loss matrix for the decision maker, [P (θj )] be the prior probability distribution on the possible states of Nature, [P (xk |θj )] be the prediction matrix, and [c(xk )] be the cost of the auxiliary experiment, where i = 1, . . . , m, j = 1, . . . , n, and k = 1, . . . , s. A decision function d : X → A assigns an action to each outcome of the auxiliary experiment. Thus, d(xk ) is the action to be taken by the decision maker if the outcome of the auxiliary experiment is xk according to the decision function d. A decision function describes a strategy of acting, taking all possible outcomes of the auxiliary experiment into account. Generally, a set D = {d1 , . . . , dr } of decision functions of interest is given and the decision maker has to analyze which one is optimum when minimizing the loss is the main objective. The corresponding risk function R, defined on the Cartesian product D×Θ and taking real values, is: R(dt , θj ) =

s X

[`(dt (xk ), θj ) + c(xk )] P (xk |θj ),

k=1

where: t = 1, . . . , r and j = 1, . . . , n. The matrix [R(dt , θj )] is called the risk matrix. As can be seen from its definition, the risk increases if the loss increases, and/or the cost increases. As the states of Nature are not under control, we have to choose the decision function d from the given set D for which the risk

Decision Theory

91

R is minimum. A decision function d∗ ∈ D is a strong optimum solution if it minimizes the risk functions for every state of Nature, namely: R(d∗ , θj ) = min R(dt , θj ), t

for all j = 1, . . . , n.

Unfortunately, in many cases there are no strong solutions. Therefore, let us introduce the Bayes’s risk, defined on the given set D of decision functions of interest and taking real numbers as values: B(dt ) =

n X

R(dt , θj ) P (θj ),

t = 1, . . . , r.

j=1

Bayes’s risk is the mean risk with respect to the prior probability distribution on the possible states of Nature. A decision function d∗ ∈ D is a weak optimum solution if it minimizes Bayes’s risk: B(d∗ ) = min B(dt ). t

A weak optimum solution always exists. Remark: In this section, Bayes’s formula for calculating the retrodiction matrix [P (θj |xk )] has not been used. 2 In the oil-drilling example: A = {a1 , a2 , a3 },

(m = 3),

Θ = {θ1 , θ2 , θ3 , θ4 },

X = {x1 , x2 , x3 , x4 },

(n = 4),

(s = 4).

The following information is available: The prior probability distribution [P (θj )] on the set of possible states of Nature is: θ1 θ2 θ3 θ4 θj P (θj ) 0.10 0.15 0.25 0.50 Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

The 3 × 4 loss matrix [`(ai , θj )], in thousands of dollars, is: θ1 θ2 θ3 θ4 a1 −650 −200 25 75 a2 −45 −45 −45 −45 a3 −250 −100 0 0 min −650 −200 −45 −45 The prediction matrix [P (xk |θj )] is:

Decision Theory

92 θ1 7/12 4/12 1/12 0

x1 x2 x3 x4

θ2 θ3 θ4 9/16 11/24 9/48 3/16 6/24 13/48 2/16 3/24 15/48 2/16 4/24 11/48

The cost [c(xk )] of the auxiliary experiment, in thousands of dollars, is: xk x1 x2 x3 c(xk ) 12 12 12

x4 12

Assume that the decision maker wants to see which one of the decision functions given in the table below is the best in order to minimize his loss:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

d1 d2 d3 d4 d5 d6

x1 a1 a1 a1 a3 a1 a2

x2 a3 a3 a1 a3 a1 a2

x3 a2 a3 a2 a2 a1 a2

x4 a2 a2 a2 a2 a1 a2

We can deal with any decision function d : X → A possible, but here we focus on some decision functions we find interesting. Each such decision function describes a strategy of acting when the outcome of the auxiliary experiment is known. Thus, d1 tells us to drill for oil if the seismic record shows a very favourable geological structure, to conditionally lease the land to an independent driller if the seismic record shows a favourable geological structure, and to unconditionally lease the land to an independent driller for the fixed amount of $45,000 if the seismic record indicated a relatively unfavourable or unfavourable geological structure. The other decision functions listed in the table may be interpreted similarly. Let us notice that while d1 is a ‘common sense’ way of acting, the last two decision functions describe drastic acting strategies, with d5 telling to drill for oil and d6 to unconditionally lease the land whatever the outcome of the auxiliary experiment is. The fourth decision function, d4 , on the other hand, refers to a very cautious attitude, where we are not supposed to drill for oil even when the seismic record indicates a very favourable geological structure. We evaluate the risk matrix [R(dt , θj )], where: R(dt , θj ) =

4 X

[`(dt (xk ), θj ) + c(xk )] P (xk |θj ),

k=1

where: t = 1, . . . , 6 and j = 1, . . . , 4. Taking into account that: d1 (x1 ) = a1 ,

d1 (x2 ) = a3 ,

d1 (x3 ) = a2 ,

d1 (x4 ) = a2 ,

Decision Theory

93

the first entry of the risk matrix is: R(d1 , θ1 ) =

4 X

[`(d1 (xk ), θ1 ) + c(xk )] P (xk |θ1 ) =

k=1

= [`(a1 , θ1 ) + c(x1 )] P (x1 |θ1 ) + [`(a3 , θ1 ) + c(x2 )] P (x2 |θ1 )+ +[`(a2 , θ1 ) + c(x3 )] P (x3 |θ1 ) + [`(a2 , θ1 ) + c(x4 )] P (x4 |θ1 ) = = [(−650) + 12](7/12) + [(−250) + 12](4/12)+ +[(−45) + 12](1/12) + [(−45) + 12](0) = −454.25 Doing the same kind of computation for the other entries, we obtain the risk matrix, in thousands of dollars, shown in the tableau given below where, in each column, the minimum risk has been highlighted: d1 d2 d3 d4 d5 d6

θ1 θ2 θ3 θ4 −454.25 −130.5 10.3333 1.6875 −471.333 −137.375 15.9583 15.75 −587.583 −149.25 16.5833 22. −220.917 −74.25 −1.125 −12.375 −638. −188. 37. 87. −33. −33. −33. −33.

For a decision function to be the strong optimum, the risk entries in its row have to be the smallest in all the columns corresponding to the possible states of Nature. In our case, d5 is the best for θ1 and θ2 , but d6 is the best for θ3 and θ4 . Therefore, there is no strong optimum solution in our example. Let us calculate the Bayes’s risk: B(dt ) =

4 X

R(dt , θj ) P (θj ),

t = 1, . . . , 6.

j=1

For the first decision function d1 , we get:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

B(d1 ) =

4 X

R(d1 , θj ) P (θj ) =

j=1

= (−454.25)(0.10) + (−130.5)(0.15)+ +(10.3333)(0.25) + (1.6875)(0.50) = −61.5729 Calculating Bayes’s risk for the other decision functions, we obtain the following tableau, in dollars, and select the decision function corresponding to the minimum Bayes’s risk value:

Decision Theory

94 d1 d2 d3 d4 d5 d6

B(dt ) −61,573 −55,875 −66,000 −39,698 −39,250 −33,000

Having the minimum Bayes’s risk, the decision function d3 is the weak optimum solution in our case. Remark: Obviously, a strong optimum solution is also a weak optimum solution. The converse statement is not true, as shown by this example. 2 6. Computer tips

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

1 FORTRAN. The book written by B.E. Gillett (1976) contains many valuable Fortran programs for solving operations research problems. One of these programs (Gillett, 1976, pp. 394-397) uses Bayes’s criterion for decision with experimentation when the possible states of Nature are mean values of random variables normally distributed. This Fortran program has been modified here to apply Bayes’s criterion with experimentation when the possible states of Nature and the prior probability distribution of the states of Nature are arbitrary. C C C C C C C C C C C C C C C C C C C C

DECISION WITH EXPERIMENTATION. BAYES’S CRITERION. THE PROGRAM IS DESIGNED TO APPLY BAYES’S CRITERION WITH EXPERIMENTATION TO THE COMMON CASE WHEN THE PRIOR PROBABILITY DISTRIBUTION OF THE STATES OF NATURE AND THE PREDICTION MATRIX ARE AVAILABLE. TO CREATE THE DATA FILE: *CARD 1, COLUMNS 2-80: TITLE; *CARD 2: NUMBER OF ACTIONS M, NUMBER OF STATES OF NATURE N, NUMBER OF OUTCOMES S, (EACH IN FORMAT I5); *NEXT CARDS CONTAIN THE LOSS MATRIX, TYPED ROWWISE (M ROWS, N COLUMNS, USING THE FORMAT F10.1); *NEXT ROW: PRIOR DISTRIBUTION OF STATES OF NATURE (USING THE FORMAT F10.6); *NEXT ROWS: PREDICTION MATRIX, TYPED ROWWISE (S ROWS, N COLUMNS, USING THE FORMAT F10.6); *LAST ROW: COST OF THE AUXILIARY EXPERIMENT FOR EACH OUTCOME (USING THE FORMAT F10.2); AFTER TYPING THE LAST ROW PRESS THE KEYS: Ctrl-x

Decision Theory

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

C C C C C C C C C C C C C C C C C

ASSUMING THAT YOUR DATA FILE IS NAMED bayes1.data, CONTINUE THE UNIX SESSION AS FOLLOWS: more bayes1.data f77 bayes1.f ./a.outbayes1.output more bayes1.output exit REMARK: THE FOLLOWING PROGRAM IS WRITTEN FOR SOLVING A DECISION THEORY PROBLEM WITH EXPERIMENTATION INVOLVING UP TO: 40 POSSIBLE ACTIONS, 50 POSSIBLE STATES OF NATURE, AND 50 POSSIBLE OUTCOMES OF THE AUXILIARY EXPERIMENT. IF WE ARE DEALING WITH BIGGER DIMENSIONS, WE HAVE TO MAKE THE NECESSARY CHANGES IN THE FIRST STATEMENT OF THE PROGRAM.

DIMENSION A(40,50),P(50),EL(40),HTHETA(50,50), *ELPOST(40,50),PRED(50,50),PROB(50), *SMPOST(50),INPOST(50),COST(50) INTEGER M,N,S,I,J,K,INDEX,INPOST REAL*4 TITLE(20) 5 READ(5,10) TITLE 10 FORMAT (20A4) WRITE(6,6) TITLE 6 FORMAT (’1’,20A4,//) READ(5,15) M,N,S 15 FORMAT (3I5) DO 20 I=1,M 20 READ(5,25) (A(I,J),J=1,N) 25 FORMAT (8F10.1) READ(5,75) (P(J),J=1,N) 75 FORMAT (8F10.6) DO 31 K=1,S 31 READ(5,85) (PRED(J,K),J=1,N) 85 FORMAT (8F10.6) READ(5,95) (COST(K),K=1,S) 95 FORMAT (8F10.2) WRITE(6,567) 567 FORMAT (2X,’THE LOSS MATRIX’/) WRITE(6,100) 100 FORMAT (9X,’STATES OF NATURE’/’ACTIONS’/)

95

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Decision Theory DO 105 I=1,M 105 WRITE(6,110) (A(I,J),J=1,N) 110 FORMAT (8F10.1) WRITE(6,111) (P(J),J=1,N) 111 FORMAT (//’PRIOR DISTRIBUTION OF STATES OF *NATURE’,/8F10.4) WRITE(6,678) 678 FORMAT (//’THE PREDICTION MATRIX’/) WRITE(6,200) 200 FORMAT (9X,’STATES OF NATURE’/’OUTCOMES’/) DO 205 K=1,S 205 WRITE(6,210) (PRED(J,K),J=1,N) 210 FORMAT (8F10.4) WRITE(6,211) (COST(K),K=1,S) 211 FORMAT (//’COST OF THE AUXILIARY EXPERIMENT’,/8F10.1) SMALL=10.E10 DO 30 I=1,M EL(I)=0.0 DO 35 J=1,N 35 EL(I)=EL(I)+P(J)*A(I,J) IF(EL(I).GE.SMALL) GO TO 30 SMALL=EL(I) INDEX=I 30 CONTINUE DO 300 K=1,S PROB(K)=0.0 DO 400 J=1,N PROB(K)=PROB(K)+PRED(J,K)*P(J) 400 CONTINUE 300 CONTINUE DO 301 J=1,N DO 401 K=1,S HTHETA(J,K)=P(J)*PRED(J,K)/PROB(K) 401 CONTINUE 301 CONTINUE DO 302 I=1,M DO 402 K=1,S ELPOST(I,K)=0.0 DO 502 J=1,N ELPOST(I,K)=ELPOST(I,K)+(A(I,J)+ *COST(K))*HTHETA(J,K) 502 CONTINUE

96

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Decision Theory 402 CONTINUE 302 CONTINUE DO 303 K=1,S SMPOST(K)=10.E10 DO 403 I=1,M IF(ELPOST(I,K).GE.SMPOST(K)) GO TO 403 SMPOST(K)=ELPOST(I,K) INPOST(K)=I 403 CONTINUE 303 CONTINUE EBPOST=0.0 DO 304 K=1,S EBPOST=EBPOST+SMPOST(K)*PROB(K) 304 CONTINUE VAE=SMALL−EBPOST WRITE(6,608) 608 FORMAT(//’ RESULTS:’) WRITE(6,999) 999 FORMAT (//’*BAYES CRITERION WITHOUT *EXPERIMENTATION:’//) WRITE(6,609) (EL(I),I=1,M) 609 FORMAT (’EXPECTED LOSS FOR EACH ACTION BASED ON *PRIOR PROBABILITY DISTRIBUTION’,/8F10.1) WRITE(6,610) SMALL, INDEX 610 FORMAT (//4X,’EXPECTED LOSS BAYES-PRIOR IS *EQUAL TO ’,F12.2// *4X,’THE OPTIMAL ACTION IS A(’,I2,’)’) WRITE(6,888) 888 FORMAT (//’*BAYES CRITERION WITH EXPERIMENTATION:’) WRITE(6,599) 599 FORMAT (//’POSTERIOR PROBABILITIES’/) WRITE(6,600) 600 FORMAT (20X,’OUTCOMES’/’STATES OF NATURE’/) DO 601 J=1,N 601 WRITE(6,602) (HTHETA(J,K),K=1,S) 602 FORMAT (8F10.4) WRITE(6,603) (PROB(K),K=1,S) 603 FORMAT(//’PROBABILITIES OF POSSIBLE *OUTCOMES’,/8F10.4) WRITE(6,604) 604 FORMAT (//’CONDITIONAL EXPECTED LOSS’/) WRITE(6,605)

97

Decision Theory

98

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

605 FORMAT (15X,’OUTCOMES’/’ACTIONS’/) DO 606 I=1,M 606 WRITE(6,607) (ELPOST(I,K),K=1,S) 607 FORMAT (8F10.1) WRITE(6,611) SMALL 611 FORMAT (//4X,’EXPECTED LOSS BAYES-PRIOR IS *EQUAL TO ’,F12.2) WRITE(6,612) EBPOST 612 FORMAT (//4X,’EXPECTED LOSS BAYES-POSTERIOR IS *EQUAL TO ’,F12.2) WRITE(6,613) VAE 613 FORMAT (//4X,’VALUE OF THE AUXILIARY EXPERIMENT IS *EQUAL TO ’,F12.2) DO 614 K=1,S WRITE(6,615) K,INPOST(K),SMPOST(K) 615 FORMAT (//4X,’IF THE OUTCOME IS X(’,I2,’), THEN THE *OPTIMAL ACTION IS A(’,I2,’),’/’AND THE *CORRESPONDING OPTIMAL EXPECTED LOSS IS ’, F12.2) 614 CONTINUE STOP END Assume that we have access to a UNIX computer equipped with a Fortran

f77 compiler and the name of the text editor is ‘pico’. The above Fortran program has the name ‘bayes1.f’ and we want to use it for solving the oildrilling problem discussed above. We create first the data file, as explained, in plain English, at the beginning of ‘bayes1.f’. To open the data file ‘bayes1.data’ we type the following command and press the ‘Enter’ key: pico bayes1.data The data file has the following structure: *In line 1, columns 2-80, we type the title: Oil-drilling example. *In line 2, we put the values of m, n, and s, as integers, in columns 5, 10, and 15, respectively: 3 4 4 *In lines 3-5, we put the 3 × 4 loss matrix, rowwise, with each entry, taken as a real number with a decimal point, in a field of 10 consecutive columns for each entry: −650000. −200000. 25000. 75000. −45000. −45000. −45000. −45000. −250000. −100000. 0. 0. *In line 6, we put the prior probability distribution, with each entry, taken as a real number with a decimal point, in a field of 10 consecutive columns for

Decision Theory

99

each entry: 0.10 0.15 0.25 0.50 *In lines 7-10, we put the 4 × 4 prediction matrix, rowwise, with each entry, taken as a real number with a decimal point, in a field of 10 consecutive columns: 0.583333 0.333333 0.083334 0.000000

0.5625 0.1875 0.1250 0.1250

0.458333 0.250000 0.125000 0.166667

0.187500 0.270833 0.312500 0.229167

*In the last line 11, we put the cost of the auxiliary experiment corresponding to each outcome, as real numbers with a decimal point, in a field of 10 consecutive columns for each entry; after filling in this last line, do not press the ‘Enter’ key but press the key ‘Ctrl’, keep it down and press the key ‘x’ as well, to close the editing session with the text editor ‘pico’: 12000.

12000.

12000.

12000.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

To see the data file again, type the command: more bayes1.data If we want to make some changes, type: pico bayes1.data and when we finish reediting the data file press again the key ‘Ctrl’, keep it down and press the key ‘x’ as well. We now can compile the file ‘bayes1.f’: f77 bayes1.f UNIX generates the compiled file ‘a.out’ which is now to be run using the data file ‘bayes1.data’, with the output in file ‘bayes1.output’: ./a.outbayes1.output We read the output from the file ‘bayes1.output’ typing: more bayes1.output and for advancing, one screen at a time, we press the ‘Space-bar’. 2. MATHEMATICA. As mentioned in chapter 1, Mathematica, created by S. Wolfram (1991), is a powerful software for doing interactively computational and symbolic mathematics. We are using here the student version of Mathematica 2.1 on Windows. Mathematica is case sensitive and the built in commands start with a capital letter. The basic arithmetic operations are: x+y for addition, x−y for subtraction, x y (notice the space between factors!) or x*y for multiplication, x/y for division, and x∧ y for ‘x raised at power y’. Elementary functions are: Sqrt[x] for the square root function, Exp[x] for the exponential function, Log[x] for the natural logarithm, Log[b,x] for the logarithm in base b, Abs[x] for the absolute value, Round[x] for the closest integer to x, Max[x,y,. . . ] for the maximum, Min[x,y,. . . ] for the minimum.

Decision Theory

100

Important for us here is the command N[expr,n] whose output is the numerical value of expr with n-digit precision. We refer by % to the last result generated. The input sent to the computer by us and the corresponding output from the computer are numbered. We can refer by %n to the result on output line Out[n]=. If we define the value of a variable, like x=2, it remains subsequently unchanged and sending later the command x∧ 3, for instance, we get the output 8. To clear x of its given value we use the command Clear[x]. A vector is a list of its components, like v={−3, 2.4, 80}, for instance. Algebraic operations on vectors result in applying these operations on the corresponding components of the respective vectors. The symbol list[[i]] means the i-th element of the list and list[[{i,j,. . . }]] means a new list of the i-th, jth,. . . , elements of the initial list. Let us notice that i,j,. . . are not necessarily different. Thus, in our example, v[[3]] means 80, and v[[{2,1,3,1}]] is the list {2.4,−3,80,−3}. There are four kinds of bracketing in Mathematica: regular parentheses for grouping (term); square brackets for functions f[x]; curly braces for lists {a,b,c}; double brackets for indexing v[[i]]. A matrix is a list of lists. Thus, m={{2,−3},{4,7}}, for instance, is a 2 × 2 matrix whose first row has the components 2 and −3 and second row has the components 4 and 7. In such a case m[[1]] is {2,−3} and m[[1,2]] is −3. In general, if t is a list of lists, t[[i]] gives the i-th sublist in t. Also, t[[{i,j}]] gives a list of the i-th and j-ths parts of t, and t[[i,j]] gives the part of t corresponding to t[[i]][[j]]. Thus, if t={{a,b},{c,d}}, then t[[1]] is {a,b}, t[[1,2]] is b, t[[1]][[2]] is also b, and t[[{2,2}]] is {{c,d},{c,d}}. If m={{a,b},{c,d}} is a matrix and v={p,q} and w={r,s} are two vectors, then: (a) for multiplying m by v, we use m.v, which means the list: {ap + bq,cp + dq}; (b) For multiplying v by m, we use v.m, which means the list: {ap + cq,bp + dq};

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

(c) For multiplying m by m, we use m.m, which means the list: {{a2 + bc,ab + bd},{ac + cd,bc + d2 }}; (d) For the dot (scalar) product of two vectors v and w, we use v.w, which means the scalar: pr + qs; (e) If g and h are two scalars, then g v+h is the list: {h+gp,h+gq}.

Decision Theory

101

(f) To transpose the matrix m, we use Transpose[m], which means the matrix: {{a,c},{b,d}}. The following commands are also very useful in this subsection: Table[g,{j,n}] makes a vector with n components evaluating g, depending on the index j, for j=1,. . . ,n. Thus, Table[f[i],{i,3}] generates the vector {f[1],f[2],f[3]}. The same thing when more than one index are involved. Thus, Table[i−j,{i,2},{j,2}] generates the matrix: {{0,−1},{1,0}}, whereas Table[i−j,{i,2,4},{j,2}] generates {{1,0},{2,1},{3,2}}. Sum[g,{k,m,n}], where m 0, it is worth doing the auxiliary experiment. Whatever the outcome xk of the auxiliary experiment is, we know by now what optimum action a∗ (xk ) to take. Thus, we decide to ‘drill for oil’ (i.e., a1 ), if the outcome of the auxiliary experiment is x1 or x2 , and to ‘unconditionally lease the land’ (i.e., a2 ), if the outcome is x3 or x4 . We close the Mathematica session by clicking on ‘File’ and then on ‘Exit’ from the top menu. III. Risk function For simplifying the writing, the loss and the cost will be expressed again in thousands of dollars. We move the cursor, along the path shown below, clicking the left-hand mouse on the beginning and the end of this path:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Start → Program → Mathematica → Mathematica 2.1 In[1]:= l={{−650,−200, 25, 75},{−45,−45,−45,−45}, {−250,−100, 0, 0}} Out[1]= {{−650,−200, 25, 75},{−45,−45,−45,−45}, {−250,−100, 0 ,0}} The matrix l is the loss matrix [`(ai , θj )], in thousands of dollars. In[2]:= q={0.10, 0.15, 0.25, 0.50} Out[2]= {0.10, 0.15, 0.25, 0.50} The vector q is the prior probability distribution [P (θj )]. In[3]:= p={{7/12, 9/16, 11/24, 9/48},{4/12, 3/16, 6/24, 13/48}, {1/12, 2/16, 3/24, 15/48},{0, 2/16, 4/24, 11/48}} Out[3]= {{7/12,9/16,11/24,3/16}, {1/3,3/16,1/4,13/48}, {1/12,1/8,1/8,5/16},{0,1/8,1/6,11/48}} The matrix p is the prediction matrix: [P (xk |θj )]. In[4]:= c={12,12,12,12} Out[4]= {12,12,12,12} The list c contains the cost of the auxiliary experiment corresponding to each possible outcome: [c(xk )]. In[5]:= t=Transpose[l] Out[5]= {{−650,−45,−250},{−200,−45,−100}, {25,−45,0},{75,−45,0}} This is the transposed loss matrix. In[6]:= u=Transpose[p] Out[6]= {{7/12,1/3,1/12,0},{9/16,3/16,1/8,1/8}, {11/24,1/4,1/8,1/6},{3/16,13/48,5/16,11/48}} This is the transposed prediction matrix. In[7]:= d={{1,3,2,2},{1,3,3,2},{1,1,2,2},{3,3,2,2},

Decision Theory

108

{1,1,1,1},{2,2,2,2}} Out[7]= {{1,3,2,2},{1,3,3,2},{1,1,2,2}, {3,3,2,2},{1,1,1,1},{2,2,2,2}} The list d contains the subscripts of the actions assigned by the decision functions d1 , . . . , d6 to the outcomes of the auxiliary experiment. The first sublist {1,3,2,2}, for instance, are the subscripts of the actions assigned by the decision function d1 to the four possible outcomes of the auxiliary experiment, namely: .

d1 (x1 ) = a1 ,

d1 (x2 ) = a3 ,

d1 (x3 ) = a2 ,

d1 (x4 ) = a2 .

In[8]:= r=Table[(t[[j]][[d[[i]]]]+c).u[[j]],{i,6},{j,4}] Out[8]= {{−1817/4,−261/2,31/3,27/16}, {−1414/3,−1099/8,383/24,63/4}, {−7051/12,−597/4,199/12,22}, {−2651/12,−297/4,−9/8,−99/8}, {−638,−188,37,87}, {−33,−33,−33,−33}} The 6 × 4 matrix r is the risk matrix [R(dt , θj )], where: R(dt , θj ) =

4 X

[`(dt (xk ), θj ) + c(xk )] P (xk |θj ),

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

k=1

for t = 1, 2, 3, 4, 5, 6; j = 1, 2, 3, 4. In[9]:= e=N[%] Out[9]= {{−454.25,−130.5,10.3333,1.6875}, {−471.333,−137.375,15.9583,15.75} {−587.583,−149.25,16.5833,22.}, {−220.917,−74.25,−1.125,−12.375}, {−638.,−188.,37.,87.}, {−33.,−33.,−33.,−33.}} This is the same risk matrix [R(dt , θj )] with the entries written as decimal numbers. In[10]:= v=Transpose[%] Out[10]= {{−454.25,−471.333,−587.583,−220.917,−638.,−33.}, {−130.5,−137.375,−149.25,−74.25,−188., −33.}, {10.3333,15.9583,16.5833,−1.125,37.,−33.}, {1.6875,15.75,22.,−12.375,87.,−33.}} This is the transposed risk matrix. In[11]:= w=Table[Min[v[[k]]],{k,4}] Out[11]= {−638.,−188.,−33.,−33.} The list w contains the minimum risk corresponding to each state of Nature: min R(dt , θj ), t

(j = 1, 2, 3, 4).

Decision Theory

109

In[12]:= m=Table[Position[v[[k]],w[[k]]],{k,4}] Out[12]= {{{5}},{{5}},{{6}},{{6}}} This shows that the decision function d5 is the best if the state of Nature is θ1 or θ2 , whereas the decision function d6 is the best when the state of Nature is θ3 or θ4 . Therefore, none of the decision functions considered here is a strong optimum solution. In[13]:= b=e.q Out[13]= {−61.5729,−55.875,−66.,−39.6979,−39.25,−33.} The list b contains the Bayes’s risk: [B(dt )], where: B(dt ) =

4 X

R(dt , θj ) P (θj ),

(t = 1, 2, 3, 4, 5, 6).

j=1

In[14]:= f=Min[b] Out[14]= −66. This number is the minimum Bayes’s risk. In[15]:= Position[b,f] Out[15]= {{3}} This shows that the decision function d3 is the weak optimum solution. We close the Mathematica session by clicking on ‘File’ and then on ‘Exit’ from the top menu. 7. Multiple-stage decision In some cases there are several stages during a decision making process. To a multiple-stage decision problem we generally associate a decision tree. A decision tree consists of nodes and arrows. There are two kinds of nodes: (a) ‘Decision node’, represented by a little square; the branches coming out of it are possible actions the decision maker has to choose from at that stage. (b) ‘Chance node’, represented by a little circle; the branches coming out of it are possible states of Nature or possible outcomes of an auxiliary experiment. The decision nodes are under decision maker’s control. The chance nodes may not be controlled but must be taken into account by the decision maker.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Figure 5. Nodes. -

-

-

 

-

-

Decision node

Chance node

Decision Theory

110

In this section, unlike what happened in the other sections of this chapter, we are using the profit instead of the loss. Sometimes, time plays an important role. The total amount of time between the beginning of the multiple-stage decision and its end is called horizon. In dealing with a multiple-stage decision problem there are three steps to go through: 1. Draw the corresponding decision tree. 2. Evaluate the tree. This means to put on the tree all information available. 3. Solve the tree. This means to find the optimum subtree along which the expected profit is maximum. The trick in solving the decision tree is to go backwards, from the end of the decision process to its start, jumping from decision node to decision node, and cutting down the nonoptimum subtrees. Optimality here means maximum expected profit. Let us notice that the optimum solution in a multiple-stage decision is always a subtree and not a path because any such optimum subtree has to contain at least one chance node and once a chance node belongs to an optimum subtree all the branches coming out of it have to belong to the respective subtree. By contrast, at each decision node we choose only one branch emerging from it, corresponding to the action the decision maker is taking at that decision node. A decision tree could be very cumbersome and difficult to draw on a single book page. We prefer here to deal with a simple two-stage problem analyzed by H.A. Taha (1992) in his popular textbook published in many editions that started almost 30 years ago. Some changes are made in the data set, however, in order to get an optimum solution which is more subtle and less obvious.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Problem: Assume that we have the following two-stage decision problem: Now, we have to decide whether to build a large plant (the cost is $6,500,000) or a small plant (the cost is $1,000,000) for manufacturing a certain item for which it is estimated that there are 75% chances of a high demand and 25% of a small demand in the following 10 years. Two years later, if the demand is high, we have to decide whether to expand the small plant (the cost would be $3,000,000) or not. The profit per year is shown in the table below: lp ep sp

hd ld $1,000,000 $300,000 $900,000 $200,000 $250,000 $200,000

where: ‘lp’ means large plant; ‘ep’ means expanded plant; ‘sp’ means small plant; ‘hd’ means high demand; ‘ld’ means low demand. Draw, evaluate, and solve the corresponding decision tree.

Decision Theory

111

Solution: 1. Draw the decision tree. Let us notice that we have two decision nodes: the node #1, corresponding to the decision we are making now about building a large plant or a small plant, and the node #4, corresponding to the decision we will make, two years from now, about expanding or nor the small plant if the demand is high. The possible actions are: ‘blp’ (build a large plant) and ‘bsp’ (build a small plant) at node #1; ‘esp’ (expand the small plant) and ‘dne’ (do not expand) at node #4. For each possible action coming from a decision node there is a chance node about facing a high demand or a low demand for the respective item we are manufacturing. As the total number of actions is four, there are four chance nodes (#2, #3, #5, and #6) to take into account, with two branches (‘hd’ and ‘ld’) emerging from each such chance node, representing here the two possible states of Nature. The horizon of this two-stage decision problem is 10 years. The decision tree in our problem is given below: - 2i - 5i

1

-4 - 6i - 3i

-

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

 2 years - 

8 years

-

Figure 5. Drawing the tree. 2. Evaluate the tree. We put on our tree all the information available, consisting of the cost of building a small plant or a large plant or expanding the small plant. Where an action implies a certain cost we indicate this by a little triangle, as an obstacle for taking the respective action. Thus, 46.5 means that there is a cost of 6.5 millions if we take the action ‘blp’. Also, to action ‘bsp’ we assign 41, and to action esp we assign a cost 43. In order to

Decision Theory

112

use fewer digits, we deal with millions of dollars instead of dollars. We also put on the tree the profit per year, corresponding to the different types of plant and states of Nature. We also assign a probability for each possible state of Nature: 0.75 to hd and 0.25 to ld.

hd.75

Profit - 1.0/year

- 2i

ld.25

46.5

- .30/year

hd.75

blp

- .90/year

43 - i 5

1

esp hd.75 -

bsp

41

ld.25 4

dne

.25/year - 3i

 2 years - 

- .20/year

hd.75

- .25/year

ld.25

- .20/year

- 6i

ld.25

- .20/year

8 years

-

Figure 7. Evaluating the tree. 3. Solve the tree. Going backwards, we evaluate the expected profit at node #4 first and later at node #1. At node #4: E(Profit|esp) = (0.9)(0.75)(8) + (0.2)(0.25)(8) − 3 = 2.8

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

E(Profit|dne) = (0.25)(0.75)(8) + (0.2)(0.25)(8) = 1.9 In order to explain the first line above: If we expand and have high demand, which has a probability 0.75 to happen, the profit per year is 0.9 and, during 8 years, the corresponding part of the expected profit is (0.9)(0.75)(8). If we expand and have a low demand, which has a probability of 0.25 to happen, the profit per year is 0.2 and, during 8 years, the corresponding part of the expected profit is (0.2)(0.25)(8). The expected profit corresponding to the action esp is obtained by summing up these two quantities and subtracting the cost of expanding the small plant, which is 3 millions.

Decision Theory

113

Therefore, if we are at node #4, which means that after two years since deciding to build a small plant the demand for the item manufactured by us has proved to be high, taking the action ‘expand the small plant’ is more profitable. Thus, at this node, if we ever arrive at it (!), we have to follow the arrow ‘esp’ and cut out the arrow ‘dne’, which is indicated by the sign × on the decision tree. Thus we learn that if we reach node #4, then we have to take the action ‘esp’, selecting the corresponding subtree, and the expected profit for the second stage of the decision process is 2.8 millions.

hd.75

Profit - 1.0/year

- 2i

ld.25

46.5 blp

- .30/year

hd.75

- .90/year

ld.25

- .20/year

hd.75

- .25/year

ld.25

- .20/year

43 - i 5

× 1.975 1

esp hd.75 -

bsp

41

dne

4

×

.25/year - 3i

 2 years - 

- 6i

ld.25

- .20/year

8 years

-

Figure 8. Solving the decision tree. At node #1:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

E(Profit|blp) = (1)(0.75)(10) + (0.3)(0.25)(10) − 6.5 = 1.75 E(Profit|bsp) = [2.8 + (0.25)(2)](0.75) + (0.2)(0.25)(10) − 1 = 1.975 Let us explain the computation from the last line above: If at node #1 we build a small plant and the demand is high for the first two years, which happens with probability 0.75, we reach node #4 where we know by now that we can get an expected profit of 2.8 millions, and for the first two years a small plant will have a profit of 0.25 millions per year. Therefore, if we build a small plant and the demand is high, the corresponding part of the expected profit

Decision Theory

114

is: [2.8+(0.25)(2)](0.75). On the other hand, if we build a small plant and the demand is low, which happens with probability 0.25, the profit per year is 0.2 millions and, during 10 years, the corresponding part of the expected profit is (0.2)(0.25)(10). The expected profit corresponding to the action ‘bsp’ is obtained by summing up these two quantities and subtracting the cost of building a small plant, which is 1 million. Looking at the results of the last two computations, we can see that at node #1, it is more profitable to build a small plant than a large plant. As a final conclusion, the optimum solution is to build a small plant now and if the demand is high after two years, to expand the small plant. The corresponding expected profit is estimated to be 1.975 millions. In the decision tree we have highlighted the corresponding optimum subtree.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

A variant of the decision tree problem. In the previous two-stage decision problem we assumed that the probability 0.75 of a high demand (‘hd’) and the probability 0.25 of a low demand (‘ld’) referred to the entire period of time elapsed between consecutive decision nodes or between an ultimate decision node and the end of the horizon. Let us assume now that there are 75% chances of a high demand and 25% chances of a low demand for the resective item in every year of the horizon taken into account. Assume that in a new two-stage decision problem we have to decide now whether to build a large plant (‘BLP’), which would cost $1,000,000, or a small plant (‘BSP’), which would cost $300,000, for manufacturing a certain item for which the forecast is that there are 75% chances of a high demand (‘H’) and 25% of a low demand (‘L’) in each of the following three years. One year from now, if the demand is high, we have to decide whether to expand the small plant (‘ESP’), which would cost $800,000, or to not expand the small plant (‘DNE’). As in the previous problem: for a large plant the profit per year is $1,000,000 if the demand is high and $300,000 if the demand is low; for an expanded plant the profit per year is $900,000 if the demand is high and $200,000 if the demand is low; for a small plant the profit per year is $250,000 if the demand is high and $200,000 if the demand is low. The corresponding decision tree is shown below. The expected profit corresponding to the action ‘ESP’ emerging from the decision node 4 consists of: HH: (900, 000 × 2)(0.75)2 = $1,012,500; LH: (900, 000 + 200, 000)(0.75)(0.25) = $206,250; HL: (900, 000 + 200, 000)(0.75)(0.25) = $206,250; LL: (200, 000 × 2)(0.25)2 = $25,000. Summing up, the total expected profit corresponding to the action ‘ESP’ at node 4 is $1,450,000, without taking the cost of ‘ESP’ into account.

Decision Theory

115

The expected profit corresponding to the action ‘DNE’ emerging from the decision node 4 consists of: HH: (250, 000 × 2)(0.75)2 = $281,250; LH: (250, 000 + 200, 000)(0.75)(0.25) = $84,375; HL: (250, 000 + 200, 000)(0.75)(0.25) = $84,375; LL: (200, 000 × 2)(0.25)2 = $25,000. Summing up, the total expected profit corresponding to the action ‘DNE’ at node 4 is $475,000.

-

 - 2 

BLP

HHH LHH HLH HHL LLH LHL HLL LLL

 - HH ESP LH - 5  - HL LL

1 - 4

BSP H

 - 3 

 - 6 DNE 

-

HH LH HL LL

-

LHH LLH LHL LLL

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Figure 9. The case with annual high/low demand. If the cost of ‘ESP’ is taken into account, then the expected profit corresponding to the two possible actions emerging from the decision node 4 is: E(profit| ESP) = $1,450,000−$800,000= $650,000 ; E(profit| DNE) = $475,000. Consequently, the optimum action at the decision node 4 is ‘ESP’ and the corresponding optimum expected profit is $650,000. We cut out the subtree corresponding to the nonoptimum action ‘DNE’. The expected profit corresponding to the action ‘BLP’ emerging from the decision node 1 consists of:

Decision Theory HHH: (1, 000, 000 × 3)(0.75)3 = $1,265,625; LHH: (1, 000, 000 × 2 + 300, 000)(0.75)2 (0.25) = HLH: (1, 000, 000 × 2 + 300, 000)(0.75)2 (0.25) = HHL: (1, 000, 000 × 2 + 300, 000)(0.75)2 (0.25) = LLH: (1, 000, 000 + 300, 000 × 2)(0.75)(0.25)2 = LHL: (1, 000, 000 + 300, 000 × 2)(0.75)(0.25)2 = HLL: (1, 000, 000 + 300, 000 × 2)(0.75)(0.25)2 = LLL: (300, 000 × 3)(0.25)3 = $14,062.5.

116 $323,437.5; $323,437.5; $323,437.5; $75,000; $75,000; $75,000;

Summing up, the total expected profit corresponding to the action ‘BLP’ at node 1 is $2,475,000, without taking the cost of ‘BLP’ into account. The expected profit corresponding to the action small ‘BSP’ emerging from the decision node 1 if the demand in the first year is high ‘H’ is: H: (250,000+650,000)(0.75) = $675,000. The expected profit corresponding to the action ‘BSP’ emerging from the decision node 1, if the demand in the first year is low ‘L’, consists of: LHH: (250, 000 × 2 + 200, 000)(0.75)2 (0.25) = $98,435.5; LLH: (250, 000 + 200, 000 × 2)(0.75)(0.25)2 = $30,468.75; LHL: (250, 000 + 200, 000 × 2)(0.75)(0.25)2 = $30,468.75; LLL: (200, 000 × 3)(0.25)3 = $9,375. Summing up, the total expected profit corresponding to the action ‘BSP’ emerging from node 1, if the demand in the first year is ‘L’, is $168,750. If the costs of ‘BLP’ and ‘BSP’ are taken into account, the expected profit corresponding to the two possible actions emerging from the decision node 1 is: E(profit| BLP) = $2,475,000−$1,000,000= $1,475,000 ; E(profit| BSP) = (675,000+168,750)−300,000=$543,750.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Consequently, the optimum action at the decision node 1 is ‘BLP’ and the corresponding optimum expected profit is $1,475,000. We cut out the subtree corresponding to the nonoptimum action ‘BSP’.

Simulation

117

Chapter 3. SIMULATION

ABSTRACT

Simulation is a powerful tool for doing experimental mathematics. Using random numbers, it allows us to get representative samples, of any length, from any discrete or continuous probability distribution of interest. Using it, we can obtain a representative history of what actually happens in a system with random input, random inner behaviour, and random external fluctuations. Monte Carlo method gives us the possibility to approximate with great accuracy the value of multiple definite integrals, the optimum value of objective functions subject to constraints, and the solutions of equations which cannot be solved by using the standard strictly deterministic approaches.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

1. Generalities It seems that long ago, the French naturalist Georges-Louis Leclerc, Comte de Buffon (1707-1788), who encouraged the popular study of natural history and published a huge ‘Natural History’ in 36 volumes, dealt with the first problem involving geometrical probability. This was the so-called needle problem, which has appealed to the imagination of many people because it allows the ‘experimental’ determination of π by throwing a needle on a plane covered with parallel and equidistant lines and counting the number of times the needle hits a line. The idea of approximating strictly deterministic entities by performing a probabilistic experiment was new indeed but its effective applications had to wait for more than 160 years in order to be consistently implemented using large sets of computer generated random numbers. During the World War II, when the first powerful computer was created in the United States, new approximation methods using random numbers started to emerge. Stanislaw M. Ulam, John von Neumann, and Enrico Fermi, in this order, should take the credit for rediscovering the Monte Carlo method and for bringing up convincing examples of its efficiency. In an interview, Stan Ulam mentioned that he had been very interested in heuristic reasoning: “About the

Simulation

118

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

time when I left Los Alamos just when the war ended, I had the first thoughts about [what is called now as Monte Carlo method]. When I came back to Los Alamos I developed it some more and then, mainly in collaboration with [John] von Neumann, I established several regions of applications.” Being told that “Everyone seems to recognize [the Monte Carlo method] as [being] important,” Stan Ulam answered with modesty: “I know, but intellectually it wasn’t a great deal.” In his autobiography he described briefly this new approach: “One may choose a computation of a volume of a region of space defined by a number of equations or inequalities in spaces of a high number of dimensions. Instead of the classical method of approximating everything by a network of points or ‘cells,’ which would involve billions of individual elements, one may merely select a few thousand points at random and obtain by sampling an idea of the value one seeks.” The new technique, essentially based on randomness, was called Monte Carlo method after one of the three communes of Monaco, in Europe, famous for its gaming tables. Today, simulation is a new chapter of applied mathematics. It is more and more intensively used in that part of operations research which deals with probabilistic modelling. It has two distinct parts: sampling from probability distributions and the Monte Carlo method. Both of them are entirely based on random numbers and computers. Simulation, in general, and the Monte Carlo method, in particular, entirely rely on common sense and the use of a large number of random numbers. The implementation is very fast and the solution is generally approximated or belongs to a certain confidence interval. The technique may be easily learned and is applicable to a larger number of cases and under more general conditions than those investigated with the analytical tool provided by calculus or classic optimization methods. Mathematically speaking, a ‘random number’ is a random variable, uniformly distributed in the unit interval [0, 1]. Its possible values, called also random numbers, represent pure chaos. There are tables containing pure random numbers, obtained using physical devices that generate the so-called white noise. There are also computer programs which, being inevitably written according to some rules, generate only ‘pseudo-random numbers’, good enough, however, to qualify as chaos. In dealing with sampling, simulation is at the opposite end with respect to what statistical inference does. Thus, in statistics we generally do not know the probability distribution and we use a sample for estimating the unknown parameters of the unknown probability distribution or for testing hypotheses about it. Practically, the problem is delicate because we must be sure that the sample is a random one and its size sufficiently large in order to make our conclusions reliable. In simulation the probability distributions are known and

Simulation

119

we want to obtain representative samples from them. The random numbers are like a primordial, homogeneous, unorganized clay which, taken as input, is moulded according to the constraints induced by the probability distribution we are sampling from, obtaining as output a sequence of outcomes occurring in agreement with the respective probability distribution. On the other hand, as we have plenty of random numbers available, the sample size can be taken to be as large as we want it to be. To give an example, if in a queueing system the arrivals and service time are described by two random variables, assumed by us or obtained from what happened in the past, we can simulate a representative history of events (i.e., random arrivals and departures) occurring in the respective system during a week, a month, or several years. Monte Carlo method, on the other hand, uses random numbers in order to approximate definite simple and multiple integrals or to optimize objective functions with or without constraints. We have only to calculate the values of the functions involved in the respective problem at a large number of random points covering the corresponding domain of definition as uniformly as possible and select the optimum or express the approximation using confidence intervals The moral is simple: Back to common sense but using a lot of random numbers. The only golden rule is: do not use the same random number twice. In the next sections, we simulate the simplest possible probabilistic experiment, like tossing a fair coin, show how to sample from a discrete or continuous probability distribution, approximate a definite multiple integral, and optimize a linear program or a nonlinear function of several variables. As simulation cannot be done without using a computer, we have opted for the popular computer package Minitab explained in many textbooks of elementary statistics like [17] and [25], for instance. Minitab is widely available under Windows or Dos and its commands are simple, straightforward English statements. It is particularly useful in simulation.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

2. Minitab The computer software Minitab was originally designed for students in introductory statistics courses, at Pennsylvania State University, in 1972. Since that time, Minitab has expanded, through a large number of versions, into a powerful package which is widely used today in statistics and probabilistic operations research. It is also a user friendly software whose commands are very close to the everyday English language, allowing a conversational dialogue between the computer and the user. Minitab proves to be very useful in simulation, in general, and in the applications of the Monte Carlo method, in particular. The computer is a must in simulation. Minitab was originally designed at the Pennsylvania State University, in introductory statistics courses, for

Simulation

120

students with little computer experience. Its simple English-like commands, easy to learn and remember, made it the most popular and accepted statistical analysis package. Since its modest initial didactic objectives, Minitab’s features and applications have expanded considerably, keeping however its use simple and user-friendly. There are many versions of Minitab available for personal computers working either under Dos or under Windows. Throughout this book we refer to Minitab on Windows. In order to start a Minitab session, we click and move the mouse through the following options: Start → Programs → Minitab for Windows → Minitab (This means that we click on Start, move successively to Programs, Minitab for Windows, Minitab, and click on Minitab. Click means click the left-hand side of the mouse. Clicking the right-hand side of the mouse, which is rarely used, generally shows us several options to choose from at the respective stage.) At the beginning, the Data section (a worksheet containing 100000 cells for data) is active in the lower part of the screen. We click on the upper half of the screen, to activate the Minitab Session. There are two ways of using the Minitab for solving our problems: (a) Enabling the Command Language. From the top menu we choose and click on: Editor → Enable Command Language or, in some other versions of Minitab:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Editor → Enable Commands The prompt ‘MTB>’ appears on the screen which means that we can send our Minitab commands to the computer in an interactive way. After each command we press the ‘Enter’ key. If we make a mistake we are told about it by the computer and we can resend the correct command or we can click on ‘Help’ from the top menu for more information and examples. Throughout this chapter, each forthcoming Minitab interactive session will be shown as a sequence of commands starting after the prompt ‘MTB>’ is active on the screen. To end the Minitab session, from the top menu, we choose and click on: File → Exit (b) Using the Command Line Editor. From the top menu we choose and click on: Edit → Command Line Editor Now, we are not in an interactive mode, and we write all the commands needed for solving our problem without getting any feedback from the computer. This

Simulation

121

is particularly convenient when we have to type in the commands of a long program. When we finish, we click on ‘Submit Commands’ at the bottom of the window. Then we either get the answers to our commands or we are told by computer what kind of errors have been made by us. The advantage of this approach is that the commands of the typed long program are not lost and we can recover the entire program and subsequently make corrections by choosing from the top menu and clicking on: Edit → Edit Last Command Dialogue

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

In writing or correcting the commands of our Minitab program we can also use the ‘Help’ option from the top menu. Throughout this chapter, each forthcoming Minitab noninteractive session will be shown as a sequence of successive commands starting after the ‘Command Line Editor’ has been activated from the ‘Edit’ option. To end the Minitab session, from the top menu, we choose and click on: File → Exit In Minitab, the data sets are stored in columns, labelled c1, c2, . . . , and constants, labelled k1, k2, . . . . Each column is treated as a vector and the operations on columns are operations between the corresponding vectors. The operations are: + (addition), − (subtraction), * (multiplication), / (division), and ** (exponentiation). The elementary functions are: ‘exp’ (exponential), ‘sqrt’ (square root), ‘log’ (natural logarithm), ‘log10’ (logarithm in base 10), ‘sin’ (sine), ‘cos’ (cosine), ‘tan’ (tangent), ‘asin’ (arcsine), ‘acos’ (arccosine), ‘atan’ (arctangent), ‘max’ (maximum), ‘min’ (minimum), ‘sum’ (sum of all the components of a column). They are applied to columns or constants. The computer package is not case sensitive; therefore, we can use small or capital letters in our commands. Only regular parentheses, namely ‘(’ and ‘)’, are used but they may be iterated any number of times. If we want to put some numbers into a column, say c1, we simply type them, separated by a blank space, starting with the command ‘set’ and ending with ‘end’. After each command we press the ‘Enter’ key. If we want to introduce in column c1 the consecutive integers 2,3,4,5,6, for instance, we may use the commands: set c1 2:6 end Arbitrary values to be introduced in column c2, for instance, are separated by a blank space: set c2 2 3 3.6 −4.78 25 end The command ‘read’ allows us to introduce numerical values, row by row, simultaneously in several columns:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Simulation

122

read c3−c5 3 67 −5 −6 23 −8.6 8 35 −6.8 end in which case, c3 will contain the values 3, −6, 8, column c4 will contain the values 67, 23, 35, and column c5 will contain the values −5, −8.6, −6.8. We assign a value to a constant or the result of an operation on some columns to a new column, by using the command ‘let’. Thus, if we want to assign the value −2.3 to constant k1 and introduce the sum of columns c1 and c2 in column c3, we use the commands: let k1=−2.3 let c3=c1+c2 provided that the columns c1 and c2 contain numerical values in the same number of rows. If we want to introduce the new value −99 instead of the current second row of column c1, we use: let c1(2)=−99 If we want to insert the new values 5 and −7.2 between the lines 2 and 3 of column c3, we use: insert 2 3 c3 5 −7.2 end The command: print c1 k1 shows on the screen the values stored in column c1 and the numerical value of the constant k1. We delete (empty) the columns c2, c3, c4, c5 and the values stored in k1, k2, k3 by typing: erase c2−c5 k1−k3 but if we want to delete the first 10 rows and the 13th row in the columns c23, c24 and c25, we use the command: delete 1:10 13 c23−c25 For getting general information about our columns and constants we type: info If the columns c6 and c7 contain the same number of numerical components and we want to subtract the components of c7 from the corresponding components of c6 and store the results in column c8, we use the command: let c8 = c6 − c7 The command ‘let’ is used in dealing with the operators mentioned above. Thus, if column c12 contains positive numbers then, after using the commands: let c15 = log(c12) let c16 = sqrt(c12) let c17 = c12**2

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Simulation

123

we get the natural logarithm (in column c15), the square root (in column c16), and the squares (in column c17), of the corresponding values from the rows of column c12. Dealing directly with columns, which are in fact vectors, proves to be a major advantage in using Minitab. In simulation, more often than not, the main program of a Minitab session involves the execution, several times, of a subprogram. For instance, if a subprogram describes what happens in a shop during one day (random supply, random demand, inventory, daily cost, penalty, revenue, and profit), in order to see what happens during 90 consecutive days we have to execute this subprogram 90 times. Older versions of Minitab (the versions on Dos and the first 11 versions on Windows) allow to store a subprogram inside the main program before executing it. In such older versions of Minitab, a subprogram is typed and given a name starting with the command: store ’john’ where john is an arbitrary name given by us and, after typing all its commands, it is closed by the command: end If we want to execute the subprogram ‘john’ 25 times, for instance, and we do not want to see the lines of the subprogram printed on the screen 25 times during the execution of the subprogram, we use the commands: noecho execute ’john’ 25 The command ‘noecho’ is cancelled by a subsequent command ‘echo’. In more recent versions of Minitab on Windows the command ‘store’ is not supported. The respective subprogram is supposed to be written as a text file, separately, using any text editor, like ‘Notepad’, ‘WordPad’, ‘Word’, or ‘ChiWriter’, and saved on a diskette, on a CD, or on the hard disk. An easy way of doing this is by choosing and clicking on: Start → Programs → Accessories → Notepad After typing the commands of the subprogram ‘john’, we save it as: a:john.txt on a diskette in drive ‘a:’, or as: d:john.txt on a CD in drive ‘d:’, or as: c:\windows\john.txt on the hard disk, respectively. With the subprogram ‘john’ on a diskette in drive ‘a:’, or on a CD in drive ‘d:’, or on the hard disk ‘c:’, we start the Minitab session and we type the commands of the main Minitab program. If we want to execute the subprogram ‘john’ 25 times, for instance, we type: noecho execute ”a:john.txt” 25 or:

Simulation

124 noecho execute ”d:john.txt” 25

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

or, respectively: noecho execute ”c:\windows\john.txt” 25 In the next Minitab programs, we will assume that the subprograms are saved as text files on a diskette in the drive ‘a:’ and, therefore, we will use the first command given above for executing them inside the main program. If a certain command has subcommands, then the command itself and the subcommands have to end by a semicolon, except the last subcommand which has to end by a period. If we want to copy the contents of columns c1, c2, c3 and constants k1, k2, k3 into the new columns c4, c5, c6 and constants k4, k5, k6, respectively, we use the command: copy c1−c3 k1−k3 c4−c6 k4−k6 but if we want to copy into column c3 only those entries of column c1 that correspond to the rows in which in another column c2 there is the numerical value 5, for instance, we use the command and subcommand: copy c1 c3; use c2 = 5. provided that columns c1 and c2 have the same number of components. In the last subcommand above the blank spaces before and after ‘=’ are essential. Starting with the release 15 of Minitab, the subcommand ‘use’ may be replaced by the subcommand ‘include’: copy c1 c3; include; where ”c2 = 5”. It is anticipated that in a future release of Minitab the subcommand ‘use’ will be discarded and replaced by the subcommand ‘include’ just mentioned. Finally, assume that the column c1 contains some numerical values having the minimum value stored in it −132 and the maximum value 236, for instance. If we want the computer to put in a new column c2 the ‘code’ value 1 if the corresponding entry from c1 is nonpositive and the ‘code’ value 0 if the corresponding entry from c1 is positive, then we use the command: code (−132:0)1 (0:236)0 c1 c2 Let us notice that to the number 0, belonging to both coding intervals, it corresponds the ‘code’ value 1 because if the coding intervals overlap, as in this example, the first assignment counts. In such a case, we can easily count how many nonpositive values are stored in c1 by simply asking for: let k1=sum(c2) print k1 where ‘sum(c2)’ gives the sum of the values stored in column c2. Similarly, if we want to store in k2 and k3 the maximum value and the minimum value

Simulation

125

from column c12, respectively, we use the commands: let k2 = max(c12) let k3 = min(c12) print k2 k3

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

3. Random numbers Roughly speaking, random numbers represent pure chaos, taken as primal clay, from which we mould representative samples from probability distributions of interest. The approach is opposite to what happens in statistical inference, for instance. There, we assume that the population is characterized by a certain probability distribution (the normal distribution, for instance) depending on unknown parameters (the mean or the standard deviation, for instance) and a random sample from this population is available. The statistician has to use this given sample in order to estimate unknown parameters of the respective probability distribution or to test hypotheses about conjectured values of the unknown parameters of the assumed probability distribution of the population. The sampling process could be influenced by many perturbing factors that are generally out of control. The sample size (i.e. the number of observations in the sample, generally expected to be large) and the randomness of the sample (if the sample size is n then all possible sequences of n observations from the population should have the same chance to be selected) could pose serious problems from practical point of view. In simulation the probability distribution is known and we use random numbers in order to generate a random sample from a population whose values are distributed according to the given probability distribution. The random numbers, representing pure randomness, are combined and used according to the constraints induced by the given probability distribution and the outcome of this moulding process is a representative sample from this probability distribution. As there are plenty of random numbers available, getting samples of large sizes is no longer a problem. Just this freedom in generating many and large samples from probability distributions of interest makes simulation a powerful instrument of probabilistic modelling in operations research. Simulation does the transition from theoretical results expressed in terms of probability distributions to the effective way of implementing them in practice. As we have seen, in a twoperson zero-sum game, for instance, the optimum solution could be a pair of mixed strategies for the players. A mixed strategy is a probability distribution on the possible pure strategies of the respective player. But how is the player supposed to effectively play the game once or several times? Simulation comes up and generates a sequence of pure strategies according to the constraints induced by the given optimum mixed strategy. It is also useful to notice that in communication theory, a structured input

Simulation

126

(the message) enters a noisy communication channel and, as a consequence, arrives perturbed at the output (the destination). In simulation the approach is different: a noisy input (the sequence of random numbers) is structured according to the constraints induced by a certain probability distribution of interest and arrives at the output as a representative random sample from the corresponding probability distribution. Simulation may be viewed as a method of structuring chaos according to the constraints induced by given probability distributions. A ‘random number’ is a random variable R uniformly distributed in the interval [0, 1]. Its probability density is:

fR (x) =

   1,  

if 0 ≤ x ≤ 1,

0, elsewhere.

We have to make a clear distinction between a random number, as a random variable uniformly distributed in [0, 1], and a possible numerical value of it, from [0, 1], also called random number. There are tables with random numbers. They are genuine random numbers, normally obtained as discrete values of the output given by a physical device that generates a purely random series of values called white noise. Practically, any Statistics textbook has such a table that contains random digits arranged in columns like the following ones: 269850 482376 909974 875849 169316 772395

321439 932871 072743 593465 349253 686309

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

We use them by inserting a decimal point in front of each of them, i.e.: 0.269850 0.482376 0.909974 0.875849 0.169316 0.772395

0.321439 0.932871 0.072743 0.593465 0.349253 0.686309

A golden rule in simulation is: ‘never use the same random number twice in the same application’. There are also many computer programs for generating pseudo-random numbers that are good enough, however, to be taken as random numbers. For instance, the Minitab commands for getting 100 random numbers and store

Simulation

127

them in column c1 are: random 100 c1; uniform 0 1. print c1 In the next two sections we will see how to obtain samples from discrete or continuous probability distributions of interest and how to use random numbers for approximating the values of simple or multiple definite integrals and the roots of algebraic equations, for instance. Simulation will also be used in the last chapter dealing with queueing theory. 4. Sampling from discrete probability distributions 1. Tossing a fair coin. This is the simplest probabilistic experiment. There are two equally likely outcomes, denoted by H (head) and T (tail), respectively. The following constraints will obviously transform a sequence of random numbers into a succession of Hs and T s uniformly distributed: 0.0 ≤ R ≤ 0.5 −→ H 0.5 < R ≤ 1.0 −→ T For instance, the random numbers: 0.269850 0.482376 0.909974 0.875849 0.169316 0.772395

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

give the outcomes H, H, T, T, H, T , respectively. Thus we have simulated six tosses of a fair coin without actually tossing it. Obviously, the above set of constraints is not unique. Thus, if D1 and D2 are two disjoint unions of a finite number of disjoint subintervals of [0, 1], both having the length equal to 1/2, then the constraints: R ∈ D1 −→ H R ∈ D2 −→ T will also generate a sequence of Hs and T s uniformly distributed. 2. Rolling a fair die. The probabilistic experiment has six equally likely possible outcomes, labelled 1, 2, 3, 4, 5, and 6, respectively. The following set of constraints will transform a sequence of random numbers into a succession of outcomes of the probabilistic experiment induced by rolling a fair die:

Simulation

128 0.0 ≤ R ≤ 1/6 1/6 < R ≤ 2/6 2/6 < R ≤ 3/6 3/6 < R ≤ 4/6 4/6 < R ≤ 5/6 5/6 < R ≤ 1.0

−→ −→ −→ −→ −→ −→

1 2 3 4 5 6

Taking into account these constraints, the random numbers: 0.321439 0.932871 0.072743 0.593465 0.349253 0.686309 will give the following outcomes of six rollings of a fair die: 2, 6, 1, 4, 3, and 5, respectively. 3. Sampling from a finite discrete probability distribution. Let X be a discrete random variable with the possible values (in increasing order) and the corresponding probabilities given in the following tableau: x1 x P (x) p1

... ...

xk pk

Obviously: pi > 0,

k X

pi = 1.

i=1

The key concept in simulation is the cumulative probability distribution function which assigns to each real number x the probability that X does not exceed the value x, i.e.: FX (x) = P (X ≤ x).

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

F is a nondecreasing function and 0 ≤ F (x) ≤ 1. It is constant on each interval [xi , xi+1 ] and has jumps at the points x1 , . . . , xk . In order to sample from the probability distribution whose cumulative distribution function is F , we use the constraints: 0 ≤ R ≤ F (x1 ) −→ x1 F (xi−1 ) < R ≤ F (xi ) −→ xi , (i = 2, . . . , k). Indeed, we have: P (F (xi−1 ) < R ≤ F (xi )) =

Z

F (xi )

F (xi−1 )

fR (x) dx =

Simulation

129 = F (xi ) − F (xi−1 ) =

i X

pj −

j=1

i−1 X

pj = pi ,

j=1

which shows that according to the above mentioned constraints, the value xi is generated during simulation with the corresponding probability pi . Once the sample of size n is generated, let: x x1 Q(x) q1

. . . xk . . . qk

be the sample probability distribution, where qi is the relative frequency of the value xi in the sample. We can apply the χ2 goodness-of-fit test for checking the accuracy of the sample obtained. We calculate the number: χ2 = n

k X (qi − pi )2 i=1

pi

and the critical region is: χ2 > χ2 α,k−1 ,

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

where χ2 α,k−1 is the critical point corresponding to the chi-square probability distribution with k − 1 degrees of freedom and the level of significance 0 < α < 1. The standard value of α is 0.05. If the calculated value of χ2 falls into the critical region, then we reject the hypothesis that the sample probability distribution {q1 , . . . , qk } ’comes from’ the given probability distribution {p1 , . . . , pk } we have sampled from. If the sample value of χ2 , denoted by χ∗2 , does not fall into the critical region, then we do not reject the above hypothesis. The Minitab commands needed for getting a sample of size 100 from a finite discrete probability distribution are: random 100 c3; discrete c1 c2. print c3 where column c1 contains the possible values of the random variable, column c2 contains the corresponding probabilities, and the sample values are stored in column c3. Problem 1: Use the random numbers: 0.934123 0.178239 0.347270 0.564395 0.452943 0.213960

Simulation

130

in order to generate a sample of size 6 from the finite discrete probability distribution: x −2.5 P (x) 0.3

1.2 3 0.5 0.2

Use the χ2 goodness-of-fit test for checking the accuracy of the result obtained. Solution: The cumulative probability distribution function is: x −2.5 FX (x) 0.3

1.2 0.8

3 1.0

The constraints induced by this probability distribution are: 0 ≤ R ≤ 0.3 −→ −2.5 0.3 < R ≤ 0.8 −→ 1.2 0.8 < R ≤ 1 −→ 3 The requested sample is: 3, −2.5, 1.2, 1.2, 1.2, −2.5 and the sample probability distribution is: x −2.5 1.2 Q(x) 2/6 3/6

3 1/6

We have: n = 6, k = 3, χ∗2 = 0.0555556, χ2 0.05,2 = 5.99146. As χ∗2 = 0.0555556 < 5.99146, we do not reject the hypothesis that our sample ‘comes from’ the given probability distribution. 2

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Problem 2: Use Minitab for getting a sample of size 150 from the finite discrete probability distribution given in the previous problem. Solution: We store the values of the random variable in column c1 and the corresponding probabilities in column c2. The sample will be stored in column c3. The constant stored as k6 is the value of 1 − α, which is the confidence level. The constants k5 and k7, printed at the end, give the value of χ∗2 and the critical point χ20.05,2 , respectively. If the value of k5 does not exceed the value of k7, then we do not reject the hypothesis that the generated sample ’comes from’ the given probability distribution. First, we create two subprograms saved as text files on a diskette in drive ‘a:’, on a CD in drive ‘d:’, or on the hard disk ‘c:’, namely, the file ‘mary.txt’: let k2 = c1(k3) copy c3 c4; use c3 = k2.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Simulation

131

let k4 = count(c4) let c5(k3) = k4 let k3 = k3+1 and the file: ‘john.txt’: random k1 c3; discrete c1 c2. execute ”a:mary.txt” 3 {or: execute ”d:mary.txt” 3, or: execute ”c:\windows\mary.txt” 3, respectively} let c6 = c2*k1 let c7 = ((c5−c6)**2)/c6 let k5 = sum(c7) invcdf k6 k7; chisquare 2. histogram c3 print k5 k7 The new commands here are: ‘count(c4)’, which gives the number of components stored in column c4, and ‘histogram c3’, which gives the graphic of the absolute frequencies of the values stored in column c3. Also, the pair of commands: invcdf k6 k7; chisquare 2. gives in k7 the value x for which the value of the cumulative distribution function F (x) of the chi-square distribution with 2 degrees of freedom is chosen by us in k6. Afterwards, we open a Minitab session and we type the following commands: read c1 c2 −2.5 0.3 1.2 0.5 3 0.2 end let k1 = 150 let k3 = 1 let k6 = 0.95 noecho execute ”a:john.txt” {or: execute ”d:john.txt”, or: execute ”c:\windows\john.txt”, respectively} 2 Remarks: (a) If we need another sample of the same size or a different size from the same probability distribution, we repeat the last five commands, without typing again the subprograms ‘john’ and ‘mary’, which remain stored and can be used again.

Simulation

132

(b) The command ‘random’ refers to sampling with replacement. It is perhaps the most frequent command used in simulation. By contrast, the command ‘sample’ asks for a sampling without replacement and is not useful in simulation. As a simple example, here is a Minitab program which can help to win Lotto 6/49. The six winning numbers, out of 49, which obviously have to be different, are stored in column c2: set c1 1:49 end sample 6 c2 print c2 2 4. A modelling based on simulation. We want to simulate what happens in a shop, where the supply and demand are finite discrete random variables, in order to estimate the mean daily profit. Problem 3: Let us assume that we are the manager of a small shop selling cheese cakes. Taking into account what happened during the work days of the last year, we learn that the daily supply and daily demand are two finite discrete random variables: 0 1 2 3 4 5 0.08 0.25 0.20 0.17 0.17 0.13

!

0 1 2 3 4 5 0.07 0.09 0.14 0.18 0.22 0.30

!

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

and, respectively: .

This means, for instance, that in 8% of the days no cake was delivered to the shop and in 7% of the days no cheese cake was asked for, but in 13% of the days the supply consisted of five cakes whereas in 30% of the days five cakes were demanded. The cakes come from a bakery and one cheese cake costs $10. The selling price is $20 per cake. If a cake is not sold on the same day, then the holding cost for it is $0.50 per day. If a customer is asking for a cake which is not available, then a penalty of $2 is to be paid by the shop. We want to simulate what happens in such a shop during any number, say 30, of work days and estimate the mean daily profit. Solution: The corresponding Minitab session consists of the following steps: First, we use a text editor, like ‘Notepad’, ‘WordPad’, ‘Word’, or ‘ChiWriter’, to type the following subprogram called ‘john.txt’: let c7(k1) = c8(k2)+c3(k1)−c6(k1) code (−10000:0)0 c7 c8

Simulation

133 code (0:10000)0 c7 c9 let c9 = −c9 let k2 = k1 let k1 = k1+1

We save it as ‘a:john.txt’ on a diskette in drive ‘a:’, or as ‘d:john.txt’ on a

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

CD in drive ‘d:’, or as ‘c:\windows\john.txt’ on the hard disk. Then, we start

a Minitab session and we type the commands: set c1 0:5 end set c2 0.08 0.25 0.20 0.17 0.17 0.13 end random 30 c3; discrete c1 c2. set c4 0:5 end set c5 0.07 0.09 0.14 0.18 0.22 0.30 end random 30 c6; discrete c4 c5. let c20 = 10*c3 let k1 = 1 let k2 = 1 let c8(1) = 0 noecho execute ”a:john.txt” 30 {or: execute ”d:john.txt” 30, or: execute ”c:\windows\john.txt” 30, respectively} let c21 = 20*(c6−c9) let c22 = 2*c9 let c23 = 0.5*c8 let c24 = c21−c20−c22−c23 tinterval 95 c24 The last command ‘tinterval 95 c24’ gives a 95% confidence interval for the population mean based on the sample from this population, x1 ,. . . ,xN , stored in column c24. The output of it gives: N

Mean

StDev

SE Mean

95.0% CI

Simulation

134

where: ‘N ’ is the sample size, ‘Mean’ is the sample mean, namely: x¯ =

N 1 X xi , N i=1

‘StDev’ is the sample standard deviation, namely: s=

N 1 X (xi − x¯)2 N − 1 i=1

!1/2

,

‘SE Mean’ is the standard error of the mean, namely: s √ , N and ‘95.0% CI’ is the 95% confidence interval for the unknown population mean µ, namely:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

s s x¯ − t0.025,N −1 √ < µ < x¯ + t0.025,N −1 √ , N N where t0.025,N −1 is the critical point of the t-distribution with N − 1 degrees of freedom corresponding to the significance level 0.05. There are tables containing values of tα/2,N −1 for different values of the significance level α and N . We can also use Minitab for getting the value of the critical point t0.025,15 , for instance, stored as constant k1, using the commands: invcdf 0.975 k1; t 15. print k1 In the given program, column c3 contains the supply and column c6 the demand of cheese cakes in the shop during the 30 days of the simulation process. They are random samples from the supply and demand described by the two probability distributions mentioned above. Column c20 contains the daily cost of the cakes provided to the shop by the bakery. Column c8 contains the inventory, i.e., the cakes remained unsold at the end of each day. Column c7 contains the daily balance, i.e., inventory plus supply minus the demand for each of the 30 days. If an entry of c7 is positive, it is stored in c8 as unsold cakes. If an entry of c7 is negative, the corresponding absolute value, representing the missing cakes with respect to the demand on that day, is stored in column c9. Columns c21, c22, and c23 contain the daily selling revenue, penalty, and holding cost, respectively. Finally, the entries of column c24 represent the daily profit. The subprogram called ‘john’ refers to the daily computation of the revenue, penalty, and holding cost. It is executed 30 times because our simulation involves 30 days. In the subprogram, k1 and k2 are

Simulation

135

counters: k1 refers to the current day and k2 to the previous day. Before executing the program, we initialize the counters k1 and k2 to be equal to 1. We have also assumed that at the beginning of our simulation there is no cheese cake in the inventory (namely, the initial first component of c8 is equal to zero); obviously, it could be any positive integer. The command ‘noecho’ asks the computer to abstain from printing all the instructions of the subprogram ‘john’ 30 times during its execution. The last command asks Minitab to give a 95% confidence interval for the mean of the daily profit from column c24. We obtain: the number of runs N = 30, the mean of the daily profit Mean = 19.18, the standard deviation StDev = 22.14, and the 95% confidence interval for the mean of the daily profit: (10.92,27.45). Therefore, we are 95% confident that the mean daily profit in the shop is in between $10.92 and $27.45. If we do the simulation again, taking k1=1, k2=1, c8(1)=0, and executing ‘john.txt’ 30 more times, we will obtain a different confidence interval, but close enough if the number of simulation runs N is much larger than 30. 2 The simple solution just discussed is mathematically sound only if we assume that the daily profit is a random variable normally distributed. When the simulation is applied to approximate the mean daily profit in a shop as we just did, or the area of a circle and multiple definite integrals, as we are going to do, the results are generally expressed as confidence intervals for the entities that are approximated. But are we justified to use such confidence intervals when we have no reasons to assume that the population we are sampling from is normally distributed? This is an essential problem and, fortunately, it may be solved in simulation. Let X be a random variable describing a population with mean µ and variance σ 2 . Let X1 , . . . , XN be a random sample from this population. The random variables X1 , . . . , XN are independent and have the mean µ and the variance σ 2 . The sample mean: X=

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

has the mean:

N 1 X Xi N i=1

N 1 1 X E(Xi ) = N µ = µ, E(X) = N i=1 N

and the variance: V (X) =

N 1 σ2 1 X 2 V (X ) = N σ = . i N 2 i=1 N2 N

If X is normally distributed N (µ, σ 2 ), then the sample mean X is also normally distributed N (µ, σ 2 /N ). If X is not N (µ, σ 2 ) distributed, then the

Simulation

136

sample mean X is however approximately normally distributed N (µ, σ 2 /N ), if N ≥ 30, due to the Central Limit Theorem. Therefore, if X is normally distributed N (µ, σ 2 ) then the standardized sample mean: X −µ √ σ/ N is normally distributed N (0, 1). If X is not normally distributed N (µ, σ 2 ), then the standardized sample mean given above is approximately N (0, 1) distributed if N ≥ 30. If X is normally distributed N (µ, σ 2 ) but the population variance σ 2 is not known and it is replaced by the sample variance: N 1 X S = (Xi − X)2 , N − 1 i=1 2

then the random variable:

X −µ √ S/ N

is t-distributed, with N − 1 degrees of freedom, which allows us to get a 100(1 − α)% confidence interval for the population mean µ, namely: S S X − tα/2,N −1 √ < µ < X + tα/2,N −1 √ , N N where tα/2,N −1 is the critical point of the t-distribution with N − 1 degrees of freedom corresponding to the significance level α, which means:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

P

!

X −µ √ < tα/2,N −1 > 1 − α. S/ N

Practically, the t-distribution (also known as the Student distribution, or Gosset distribution) may be replaced by the standard normal distribution N (0, 1) if the number of degrees of freedom exceeds 20, in which case tα/2,N −1 may be replaced by the critical point Zα/2 of the standard normal distribution N (0, 1). There are tables that give critical points for the t-distribution and standard normal distribution for some values of the significance level α, or we can use Minitab for getting them. Thus, in order to get the value of the critical point t0.025,7 , for instance, we use the Minitab commands: invcdf 0.975 k1; t 7. print k1 and the result is k1 = t0.025,7 = 2.36463. Similarly, in order to get the value of the critical point Z0.025 , for instance, we use the Minitab commands:

Simulation

137

invcdf 0.975 k2; normal 0 1. print k2 and the result is k2 = Z0.025 = 1.95996. Therefore, in order to use a confidence interval for the population mean when the population variance is not known, the respective population X has to be normally distributed. If the original polulation X is not normally distributed, as the sample mean X is normally distributed when the sample size is large, we take now a random sample from the population of sample means X. A random sample of sample means can be used now in order to get confidence intervals for the mean of the original population because “the mean of the mean is the mean”. As we have plenty of random numbers and we know how to sample from practically any probability distribution of interest, in simulation we build up first the population of the sample means, and then we take a random sample of sample means for constructing a 100(1 − α)% confidence interval for the mean of the original population. Technically, whatever the original population X with the mean µ is, in one run we calculate the sample mean: n 1X Xj , X= n j=1

with n ≥ 30. Taking N such runs, with N ≥ 30, we get a sample of size N from the normal population of sample means X, namely X 1 , . . . , X N , and we calculate the sample mean of these sample means: X=

N 1 X X i, N i=1

which may be considered a normal random variable with the mean value µ if N ≥ 30. Using the sample variance: S2 =

N 1 X (X i − X)2 N − 1 i=1

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

as an estimate of the unknown variance of X, the random variable: X −µ √ S/ N is t-distributed with N − 1 degrees of freedom, which allows us to obtain a 100(1 − α)% confidence interval for the mean µ of the original population: S S X − tα/2,N −1 √ < µ < X + tα/2,N −1 √ . N N

Simulation

138

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

We interpret it by saying that we are 100(1−α)% sure that the mean µ belongs to this interval. 2 A better solution: Taking the above remark into account, we go back to the cheese cake shop problem to give a better modelling solution to it. We want to simulate what happens in such a shop during 100 months of 30 days each. We use Minitab on Windows and we start by editing the file ’john.txt’, which contains a subprogram showing what happens in the shop during one month, and the file ‘mary.txt’ which refers to one day of the month. For editing the files we may use the text editors ‘Notepad’, ‘Wordpad’, ‘Word’, or ‘ChiWriter’. In the main program, the subprogram ‘john.txt’ is going to be executed 100 times. We simulate what happens in the shop when the random supply and the random demand are generated according to the given probability distributions obtained from the past experience. The final objective is to get a 95% confidence interval for the expected daily profit. We move the cursor, clicking on the beginning and the end of following sequence of choices, followed by typing the respective commands: {Start → Programs → Accessories → Notepad} set c1 0:5 end set c2 0.08 0.25 0.20 0.17 0.17 0.13 end random 30 c3; discrete c1 c2. set c4 0:5 end set c5 0.07 0.09 0.14 0.18 0.22 0.30 end random 30 c6; discrete c4 c5. let k1 = 1 let k2 = 1 let c8(1) = 0 noecho execute ”a:mary.txt” 30 {or: execute ”d:mary.txt” 30, or: execute ”c:\windows\mary.txt” 30} let c20 = 10*c3 let c21 = 20*(c6−c9)

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Simulation

139

let c22 = 0.5*c8 let c23 = 2*c9 let c24 = c21−c20−c22−c23 let k3 = mean(c24) let c10(k4) = k3 let k4 = k4+1 {File → Save As} a:john.txt {or: ”d:john.txt”, or: ”c:\windows\john.txt”, respectively} {Save} {File → New} let c7(k1) = c8(k2)+c3(k1)−c6(k1) code (−10000:0)0 c7 c8 code (0:10000)0 c7 c9 let c9 = −c9 let k2 = k1 let k1 = k1+1 {File → Save As} a:mary.txt {or: ”d:mary.txt”, or: ”c:\windows\mary.txt”, respectively} {Save} {File → Exit} {Start → Programs → Minitab on Windows → Minitab} {Click anywhere in the upper half of the screen to activate the Minitab Session} {Editor → Enable Command Language} let k4 = 1 noecho execute ”a:john.txt” 100 {or: execute ”d:john.txt” 100, or: execute ”c:\windows\john.txt” 100, respectively} tinterval 95 c10 {File → Exit} {Start → Shut Down} 2 In about two minutes, Minitab will analyze what is going on in the shop during 30 × 100 = 3000 days. In the above program: c1 contains the list of possible cakes that could come daily to the shop; c2 contains the relative frequencies of these posible arrivals based on past experience; c3 contains the daily supply of cakes during the 30 days of a month; c4 contains the list of possible cakes that could be demanded daily by customers; c5 contains the relative frequencies of these possible demands based on past experience; c6 contains the daily demand for

Simulation

140

cakes during the 30 days of a month; k1 is a counter of the consecutive days of a month; k2 is a counter which refers to the previous day; c7 contains the number of the cakes at the end of each of the 30 days of a month; c8 contains the daily number of cakes remained unsold; c9 contains the daily number of missing cakes when the demand exceeds the number of cakes available in the shop; c20 contains the daily cost for the cakes delivered to the shop by a laboratory during a 30-day month; c21 contains the daily revenue during a 30-day month; c22 contains the daily holding cost during a 30-day month; c23 contains the daily penalty cost during a 30-day month; c24 contains the daily profit during a 30-day month; k3 contains the mean daily profit during a 30-day month; c10 contains the random sample of mean daily profits of 100 30-day months; k4 is a counter of the number of 30-day months taken into account. The random sample of mean daily profits from 100 30-day months is used for getting a 95% confidence interval for the expected daily profit in the shop. The output is: Variable N Mean StDev SE Mean 95.0% CI c10 100 21.142 2.943 0.294 (20.558, 21.726) The conclusion is that we are 95% confident that the mean daily profit is larger than $20.55 and smaller than $21.72. If we repeat this Minitab session we do not obtain exactly the same numerical values but when N is large, like in our example, the new confidence interval will be very close to the previous one. 5. Sampling from the Poisson distribution. Poisson distribution with parameter λ > 0 is a discrete countable probability distribution on the nonnegative integers, whose probability function is: λx −λ e , x = 0, 1, 2, . . . . x! Both its mean value and variance are equal to λ. Theoretically, any nonnegative integer is a possible value. Practically, however, if the value of λ is given and we take into account only a finite number of integers for the values of x, we are dealing with a finite discrete random variable and the approach discussed in the previous section may be applied again for sampling from the corresponding Poisson distribution. The Statistics textbooks contain tables that give either the values of f (x) or of the cumulative probability distribution function F (x) for different values of the parameter λ. On the other side, computer packages give either the values of f (x), or F (x), or allow us to obtain directly a random sample from the Poisson distribution when the mean is known. The Minitab commands, for instance, for getting in column c1 a random sample of size 200 from a Poisson distribution with the mean 5.6 are: random 200 c1; poisson 5.6.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

f (x) =

Simulation

141

Problem 4: (a) In a shop, customers arrive randomly. Assume that the number of arrivals during a unit time interval is Poisson distributed with the mean equal to 2.1. Simulate the number of arrivals during 10 unit time intervals. {Hint: Use the random numbers: 0.12651, 0.61646, 0.11769, 0.75109, 0.86996, 0.97669, 0.25757, 0.32535, 0.07122, 0.45409} (b) For the Poisson distribution mentioned above, write a Minitab program which gives a sample of size 100 from this probability distribution in column c1, the values of f (x) for x = 0, 1, 2, . . . , 11, in column c3, the values of F (x), for x = 0, 1, 2, . . . , 11, in column c4, and the values of x for which F (x) = 0.23, F (x) = 0.46, F (x) = 0.87, respectively, in column c6. Solution: (a) From statistical tables giving the values of Poisson distribution, for λ = 2.1, we get: x 0 1 2 3 4 5 6 7 8 9 10 11

f (x) 0.122456 0.257158 0.270016 0.189011 0.099231 0.041677 0.014587 0.004376 0.001149 0.000268 0.000056 0.000011

F (x) 0.12246 0.37961 0.64963 0.83864 0.93787 0.97955 0.99414 0.99851 0.99966 0.99993 0.99999 1.00000

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

The random sample is: 1, 2, 0, 3, 4, 5, 1, 1, 0, 2. (b) The Minitab program requested is: random 100 c1; poisson 2.1. set c2 0:11 end pdf c2 c3; poisson 2.1. cdf c2 c4; poisson 2.1. set c5 0.23 0.46 0.87 end

Simulation

142

invcdf c5 c6; poisson 2.1. print c1 c3 c4 c6 Here the commands ‘pdf’, ‘cdf’, and ‘invcdf’ correspond to the probability distribution function, the cumulative distribution function, and the inverse of the cumulative distribution function, respectively. In the Minitab program just given: the output of the first two commands is a sample of size 100 from the Poisson distribution with the mean 2.1; the sixth and seventh commands give in column c3 the corresponding probabilities of the values stored in column c2 for the Poisson distribution with the mean 2.1; the eighth and ninth commands give in column c4 the values of the cumulative probability distribution function F (x) of the Poisson distribution with the mean 2.1 corresponding to the values of x stored in column c2; and, finally, the thirteenth and fourteenth commands give in column c6 the values of x corresponding to the values of the cumulative probability distribution function F (x) of the Poisson distribution with the mean 2.1 stored in column c5. 2 5. Sampling from continuous probability distributions

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

In sampling from continuous probability distributions the cumulative distribution function remains the basic tool but we are going to apply three different methods: (a) The method of inversion is the best but it may be used only if the cumulative distribution function of the corresponding probability distribution has an analytical simple formula which has an inverse. (b) If the method of inversion cannot be applied, we can use the discrete approximation of a continuous probability distribution for which sampling may be easily done. (c) Sometimes we can use a relationship between the continuous probability distribution we want to take a sample from and another continuous probability distribution for which the method of inversion may be applied. 1. Sampling from the continuous uniform distribution. This section deals with how to get random numbers from an arbitrary interval [a, b] using standard random numbers from [0, 1]. The probability density function of the uniform distribution on [a, b] is:

f (x) =

   1/(b − a),  

0,

if a ≤ x ≤ b,

elsewhere.

The mean value is (a + b)/2, the variance is (b − a)2 /12, and the cumulative

Simulation

143

probability distribution function is a linear function given by:

F (x) =

Z

  0,      

x

−∞

if x < a,

f (t) dt =  (x − a)/(b − a),

if a ≤ x ≤ b,

      1,

if x > b.

Taking:

x−a , b−a we get the formula for sampling from the uniform distribution on [a, b]: R = F (x) =

x = (b − a)R + a. Remark: We have just applied the so called ‘method of inversion’ which may be used when the cumulative probability distribution has a simple analytical expression that has an inverse easy to calculate. 2 Problem 5: (a) Get six random numbers from the interval [−2, 7]; (b) Use Minitab for getting 100 random numbers from the same interval. Solution: (a) In this case a = −2 and b = 7. Using six random numbers from [0, 1], namely: 0.259431,

0.428887,

0.610825,

0.353139,

0.142315,

0.789435,

and the formula: x = [7 − (−2)]R + (−2) = 9 R − 2, we get: 0.33488,

1.85998,

3.49742,

1.17825,

−0.71917,

5.10492.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

(b) The Minitab commands are given below; the requested random numbers are stored in column c1: random 100 c1; uniform −2 7. print c1 2 2. Sampling from the exponential distribution. The method of inversion may be also used for sampling from the exponential distribution with parameter α. The probability density function is: f (x) =

 −αx  ,  αe  

0,

if x ≥ 0,

elsewhere.

Simulation

144

The mean value is 1/α, the variance is 1/α2 , and the cumulative probability distribution function is: F (x) =

Z

x

Z

f (t) dt =

−∞

if x ≥ 0. Taking:

0

x

h

α e−αu du = −e−αu

ix 0

= 1 − e−αx ,

R = F (x) = 1 − e−αx ,

we get the formula: 1 x = − log(1 − R). α But if R is a random number from [0, 1], then 1−R is also a random number from [0, 1]. Consequently, for sampling from the exponential distribution with the parameter α, we may use the simpler, but equally valid formula: 1 x = − log R. α The input R in the above formula represents pure chaos, whereas the output x gives positive real numbers exponentially distributed with the parameter α. The Minitab commands for getting in column c1 a random sample of size 200 from an exponential distribution with the mean 5.3 are: random 200 c1; exponential 5.3. Problem 6: In a shop, customers arrive randomly. The interarrival time, namely, the time elapsed between two consecutive arrivals, is exponentially distributed. On average, there are 30 arrivals per hour. (a) Simulate five arrivals in the shop. (b) Simulate 150 arrivals in the shop using Minitab. Solution: (a) The mean interarrival time is: 1 1 60 = hr = min = 2 min. α 30 30

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Using five random numbers from [0, 1], namely: 0.624714,

0.089243,

0.932998,

0.155351,

0.418833,

and the formula x = −2 log R, we get: 0.94092,

4.83278,

0.13870,

3.72414,

1.74057.

Therefore, the interarrival times in our sample are: 56.46 sec,

4 min 49.97 sec,

8.32 sec,

Simulation

145 3 min 43.45 sec,

1 min 44.43 sec.

(b) The 150 interarrival times requested are given in minutes with decimals in column c1, or, separately, in integer minutes (column c2) and the corresponding seconds with decimals (column c3): random 150 c1; exponential 2. let c2 = round(c1) let c3 = c1−c2 code (0:1)0 (−1:0)1 c3 c4 let c3 = c3+c4 let c2 = c1−c3 let c3 = c3∗60 name c2 ’minutes’ name c3 ’seconds’ print c2 c3 The new commands here are ‘round’ which assigns the nearest integer (which is 2.0 for 2.1, −2.0 for −2.1, 3.0 for 2.6, −9.0 for −8.9, and −4.0 for −3.5) and ‘name’ which assigns a name chosen by us to a given column. 2 Remark: The exponential probability distribution has been used in many applications of probability theory. It is less known that it has a variational property which explains why it has been so successful in its applications. Thus, if f (x) is a probability density function, the amount of uncertainty contained by it is measured by Shannon’s entropy; H(f ) = −

+∞

Z

f (x) log f (x) dx,

−∞

inspired by Boltzmann H-function from statistical mechanics. Generally, if we know the mean value µ of a nonnegative random variable then there are infinitely many probability densities f (x) compatible with this mean value: Z

+∞

x f (x) dx = µ.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

0

In this set of probability densities on [0, +∞) compatible with the mean value µ, the exponential distribution with the parameter α = 1/µ is the only probability density which maximizes the entropy: H(f ) = −

Z

+∞

f (x) log f (x) dx

0

subject to the given mean value µ. Therefore, the exponential distribution with the parameter 1/µ is the most unbiased probability density on [0, +∞), subject to the given mean value µ. This justifies, a posteriori, why the exponential distribution has been so successful in many applications where the

Simulation

146

only information available is represented by a given mean value.

2

3. Sampling from the standard normal distribution. The standard normal distribution, with mean 0 and variance 1, often abbreviated as N (0, 1), has the probability density function: 1 2 f (x) = √ e−x /2 , 2π

(−∞ < x < +∞),

and the cumulative probability distribution function: x

1 Z x −t2 /2 Φ(x) = f (t) dt = √ e dt. −∞ 2π −∞ Z

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

There is no simple, explicit expression for Φ and, therefore, the method of inversion cannot be used for getting a formula for sampling from N (0, 1). We can approximate, however, the continuous probability distribution N (0, 1) by a finite discrete probability distribution. The degree of accuracy of such an approximation depends on us. In this section, for brevity reasons, we are using a rough approximation, taking an approximation step equal to 0.5. It is well-known that the graph of f (x) is a bell-shaped curve, centered at origin, with very narrow tails tending to ±∞, with almost all probability mass concentrated between −3.5 and +3.5. There are statistics tables containing values of Φ(x). Taking an approximation step of 0.5, as mentioned before, we get the following table for the cumulative distribution function of N (0, 1): x −3.0 −2.5 −2.0 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 +∞

Φ(x) 0.001350 0.006210 0.022750 0.066807 0.158655 0.308538 0.500000 0.691462 0.841345 0.933193 0.977250 0.993790 0.998650 1.000000

The restrictions induced by this discrete approximation of N (0, 1) are:

Simulation

147 0.000000 ≤ R ≤ 0.001350 0.001350 < R ≤ 0.006210 0.006210 < R ≤ 0.022750 0.022750 < R ≤ 0.066807 0.066807 < R ≤ 0.158655 0.158655 < R ≤ 0.308538 0.308538 < R ≤ 0.500000 0.500000 < R ≤ 0.691462 0.691462 < R ≤ 0.841345 0.841345 < R ≤ 0.933193 0.933193 < R ≤ 0.977250 0.977250 < R ≤ 0.993790 0.993790 < R ≤ 0.998650 0.998650 < R ≤ 1.000000

−→ −→ −→ −→ −→ −→ −→ −→ −→ −→ −→ −→ −→ −→

−3.25 −2.75 −2.25 −1.75 −1.25 −0.75 −0.25 0.25 0.75 1.25 1.75 2.25 2.75 3.25

According to these constraints, in getting a random sample containing values of a standard normal random variable X, one possible value of X corresponds to one random number R. The Minitab commands for getting in column c1 a random sample of size 300 from the standard normal probability distribution are: random 300 c1; normal 0 1. print c1

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Problem 7: (a) Get a random sample of size 35 from N (0, 1). Apply the χ2 goodness-of-fit test with the level of significance α = 0.05. (b) Use Minitab for getting in column c1 a sample of size 200 from N (0, 1). Use the command ’describe’ in order to get the sample mean value, the sample standard deviation, the minimum and maximum values in the sample, and the first three sample quartiles. Draw the histogram of the values obtained in the sample. Get in column c3 the values of Φ(x) for x taking values from −3.0 to 3.0 with an increment equal to 0.5 in order to check the accuracy of the values of the cumulative probability distribution of the standard normal distribution given in the last two tables. Get in column c5 the first three quartiles of the standard normal probability distribution N (0, 1). Solution: (a) The χ2 goodness-of-fit test is applied here for testing the null hypothesis that the sample obtained is indeed representative for the probability distribution we are sampling from. Taking 35 random numbers from [0, 1], namely:

Simulation

148 0.117436 0.560851 0.045392 0.325651 0.113268 0.215217 0.128039

0.679505 0.206409 0.432451 0.070654 0.158530 0.371456 0.309212

0.938084 0.301073 0.674034 0.831753 0.816292 0.902106 0.048297

0.399022 0.634115 0.254210 0.969159 0.036494 0.763272 0.603586

0.260487 0.264363 0.776260 0.244906 0.561722 0.409024 0.473001

and using the restriction induced by N (0, 1) in the approximation mentioned above, we get the random sample: −1.25 0.25 −1.75 −0.25 −1.25 −0.75 −1.25

0.25 1.75 −0.25 −0.75 −0.75 −0.75 0.25 −0.75 −0.25 0.25 −0.75 0.75 −1.25 0.75 1.75 −0.75 −1.25 0.75 −1.75 0.25 −0.25 1.25 0.75 −0.25 −0.25 −1.75 0.25 −0.25

The observed absolute frequency distribution is: x Oi

−1.75 3

−1.25 5

−0.75 7

−0.25 7

0.25 6

0.75 4

1.25 1

1.75 2

The χ2 goodness-of-fit test requires a minimum number of five observations in each class. After joining together neighbouring classes in order to satisfy this requirement, we get the final observed absolute frequency distribution: x Oi

≤ −1.25 −0.75 −0.25 0.25 ≥ 0.75 8 7 7 6 7

Using the values of the cumulative probability distribution function Φ(x) given above, we get the probabilities: 0.158655,

0.149883,

0.191462,

0.191462,

0.308538,

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

of the corresponding intervals: (−∞, −1.0],

(−1.0, −0.5],

(−0.5, 0.0],

(0.0, 0.5],

(0.5, +∞),

respectively. For instance: P (−1.0 < X ≤ −0.5) = Φ(−0.5) − Φ(−1.0) = = 0.308538 − 0.158655 = 0.149883. Multiplying these probabilities by the sample size 35, we get the expected absolute frequency distribution:

Simulation

149 x Ei

≤ −1.25 −0.75 −0.25 0.25 ≥ 0.75 5.5529 5.2459 6.7012 6.7012 10.7988

We calculate the sample χ2 -value: χ∗2 =

5 X (Oi − Ei )2 i=1

Ei

= 3.08797.

The critical (rejection) region is:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

χ2 > χ20.05,5−1 = 9.48773. As the sample value χ∗2 does not fall into the critical region, we do not reject the hypothesis that our sample ‘comes from’ a standard normal distribution. (b) The Minitab program is: random 200 c1; normal 0 1. describe c1 histogram c1 set c2 −3:3/0.5 end cdf c2 c3; normal 0 1. print c2 c3 set c4 0.25 0.5 0.75 end invcdf c4 c5; normal 0 1. print c4 c5 The new commands here are ‘describe c1’ (which gives the number of components ‘N ’, the mean value ‘Mean’, the trimmed mean ‘Tr Mean’, which gives the average of the middle 90% of the values after ignoring the 5% smaller and the 5% larger values, the median Median, the standard deviation ‘StDev’, the standard error of the mean ‘SE Mean’ which is the ratio between the standard deviation and the square root of N , the minimum ‘Min’, the maximum ‘Max’, and the first and third quartiles ‘Q1’ and ‘Q3’, of the components stored in the respective column c1). The quartiles of the standard normal distribution are given in column c5, namely: −0.674490, 0.000000, 0.674490, respectively. Also, ‘−3 : 3/0.5’ means: from −3 to 3, using a step of 0.5. 2 4. Sampling from the normal distribution using the central limit theorem. Let {Xn , n = 1, 2, . . .} be a sequence of independent random variables which

Simulation

150

are identically distributed (i.e. all have the same probability function in the discrete case or the same density function in the continuous case) and have finite mean µ and variance σ 2 . Then, if: Sn =

n X

Xi ,

(n = 1, 2, . . .),

i=1

then, according to the central limit theorem, the standardized Sn is asymptotically distributed like the standard normal distribution N (0, 1), i.e.: 



Sn − E(Sn ) ≤ b = lim P a ≤ q V (Sn )

n→+∞

!

Sn − nµ √ ≤b = = lim P a ≤ n→+∞ σ n 1 Z b −x2 /2 dx. =√ e 2π a Let X be a random variable with a normal distribution N (µ, σ 2 ), with mean µ and variance σ 2 . The probability density function is: 1 2 2 f (x) = √ e−(x−µ) /(2σ ) , (−∞ < x < +∞). σ 2π The standardized X, i.e.: X −µ σ is the standard normal random variable, having the probability distribution N (0, 1). Let {Rn , n = 1, 2, . . .} be a sequence of random numbers from [0, 1], as random variables uniformly distributed in [0, 1]. They are independent, identically distributed random variables, with the mean 1/2 and variance 1/12. Therefore: ! ! n n X X n n Ri = , E V( Ri = , 2 12 i=1 i=1 Z=

and, according to the central limit theorem, asymptotically, we have: Pn

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

i=1

Ri −

q

n 2

n 12

∼Z=

X −µ , σ

where ∼ means that the left-hand random variable and the right-hand random variable have the same probability distribution when n is large. This justifies the use of the following formula for sampling from N (µ, σ 2 ): n X

!

n x=µ+ q Ri − . 2 n/12 i=1 σ

Simulation

151

As a rule of thumb, this formula is used taking n = 30, with very good results, and n = 12, with acceptable results. In the last case, we get the following formula for sampling from N (µ, σ 2 ): x=µ+σ

12 X

!

Ri − 6 .

i=1

We notice that 12 random numbers are used in this formula for getting one value of X. The Minitab commands for getting in column c1 a random sample of size m from N (µ, σ 2 ) are: random m c1; normal µ σ. Problem 8: (a) Get in column c1 a sample of size 100 from N (10, 4) using the Minitab commands for directly sampling from a normal distribution. (b) Get in column c2 a sample of size 100 from N (10, 4) using the sampling formula: ! n X n σ , Ri − x=µ+ q 2 n/12 i=1 with n = 30. (c) Get in column c3 a sample of size 100 from N (10, 4) using the sampling formula: ! x=µ+σ

12 X

Ri − 6 .

i=1

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

(d) Compare the three random samples obtained. Solution: In the following programs we have: k1 = µ, k2 = σ, k3 = n, k5 = x. First we type the subprogram ‘sampling.txt’ using a text editor like ‘Notepad’, ‘WordPad’, ‘Word’, or ‘ChiWriter’: random k3 c4; uniform 0 1. let k4 = k3/12 let k5 = k1+(k2*(sum(c4)−(k3/2)))/sqrt(k4) let c5(k6) = k5 let k6 = k6+1 erase c4 The new command here is ‘erase c4’ which does just what it says, erasing all the components of the column c4. We save this file as: ‘a:sampling.txt’ on a diskette in drive ‘a:’, or ‘d:sampling.txt’ on a CD in drive ‘d:’, or on the hard disk, ‘c:\windows\sampling.txt’. Starting a Minitab session, we use the commands:

Simulation

152

random 100 c1; normal 10 2. let k1 = 10 let k2 = 2 let k3 = 30 let k6 = 1 noecho execute ”a:sampling.txt” 100 {or: execute ”d:sampling.txt” 100, or: execute ”c:\windows\sampling.txt” 100, respectively} let c2 = c5 erase c5 let k3 = 12 let k6 = 1 execute ”a:sampling.txt” 100 {or: execute ”d:sampling.txt” 100, or: execute ”c:\windows\sampling.txt” 100, respectively} let c3=c5 histogram c1−c3 describe c1−c3 ttest 10 c1−c3 The new command here is ‘ttest 10 c1−c3’. It performs a separate t-test of the mean on the data in the columns c1, c2, and c3. It performs a test of the ‘null hypothesis’ H0 : µ = 10, versus the ‘alternative hypothesis’ Ha : µ 6= 10. The value of the test statistics:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

t=

x¯ − µ √ s/ N

is calculated, where x¯ is the mean of the data, or sample mean, s is the sample standard deviation, N is the sample size, and µ is the given population mean. Minitab calculates the probability of getting a value as large in magnitude as the value obtained for the test statistic t in the respective sample or even larger from a t-distribution with N − 1 degrees of freedom. This probability is called the p-value of the test. The output specifies the numerical values of the following entities, for the samples stored in c1, c2, and c3: T-Test of the Mean Test of µ = 10 versus µ 6= 10 Variable N Mean StDev SE Mean t p-Value C1 100 9.877 1.915 0.191 −0.64 0.52 C2 100 9.961 1.824 0.182 −0.22 0.83 C3 100 10.072 1.997 0.200 0.36 0.72

Simulation

153

Remark: The p-value is used more often than not in statistical inference when dealing with hypotheses testing. If α is the probability of rejecting the null hypothesis when it is true, we can use the p-value to do the test for any prespecified α-level we choose. When the p-value is less than our specified α-level, we reject the null hypothesis and conclude that the test is significant. The p-value is the value of α at which the hypothesis test procedure changes conclusions. It is the smallest value of α for which we can reject the null hypothesis, that is, at which the test is significant. The p-value is the point at which we switch from rejecting the null hypothesis to failing to reject the null hypothesis. In reporting the result of a test, rather than stating the level (α = 0.05, or α = 0.10, for instance) and stating whether or not the null hypothesis H0 is rejected or not by the data at that level, we can instead simply give the p-value. The p-value is the level of significance determined by the particular value of the test statistic obtained in the respective sample, such that, if 0 < α < p-value, we do not reject H0 and if α > p-value, we reject H0 , at the level α. If the p-value is 0.09, for instance, the data reject H0 for every α greater than 0.09 but not for any α less than 0.09. 2 Remark: The normal probability distribution has been used in many applications of probability theory. It is less known that it has a variational property which explains why it has been so successful in its applications. Generally, if we know the mean value µ and the variance σ 2 of a random variable then there are infinitely many probability densities f (x) on the real line (−∞, +∞) compatible with these two constraints: Z

+∞

x f (x) dx = µ.

−∞

Z

+∞

−∞

(x − µ)2 f (x) dx = σ 2 .

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

In this set of probability densities on (−∞, +∞) compatible with the mean value µ and the variance σ 2 , the normal distribution N (µ, σ 2 ) with the parameters µ and σ 2 is the only probability density which maximizes the Shannon’s entropy, measuring the amount of uncertainty contained by the probability distribution having the density f (x): H(f ) = −

Z

+∞

f (x) log f (x) dx,

−∞

subject to the given mean value µ and variance σ 2 . Therefore, the normal distribution N (µ, σ 2 ) is the most unbiased probability density on (−∞, +∞), subject to the given mean value µ and variance σ 2 . This justifies, a posteriori, why the normal distribution has been so successful in many applications where the only information available is represented by a given mean value and a given variance. We can say that this unique variational property shows why

Simulation

154

the normal distribution is ‘normal’ !

2

5. Sampling from the Erlang distribution. The Gamma distribution with parameters α > 0 and β > 0 has the probability density function:  β β−1 −αx  e ,  (α /Γ(β)) x

f (x) =  

0,

x ≥ 0,

elsewhere.

where the Gamma function Γ(β) is a special function defined by: Z

Γ(β) =

+∞

xβ−1 e−x dx.

0

The mean value and the variance of a random variable X Gamma distributed with the parameters α and β are: β , α

E(X) =

V (X) =

β . α2

Remark: As we have: d  β−1 −x  x e = (β − 1)xβ−2 e−x − xβ−1 e−x , dx integrating between 0 and +∞, we get: h

xβ−1 e−x

i+∞ 0

= (β − 1)

Z

0

+∞

xβ−2 e−x dx −

Z

+∞

xβ−1 e−x dx,

0

which gives: 0 = (β − 1)Γ(β − 1) − Γ(β), which shows that the Gamma function satisfies the recurrent property: Γ(β) = (β − 1)Γ(β − 1).

2

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

If β is a positive integer, namely β = n, then the corresponding Gamma distribution is called the Erlang distribution with parameters α and n. In such a case, the above equality may be applied several times and we get: Γ(n) = (n − 1)Γ(n − 1) = = (n − 1)(n − 2)Γ(n − 2) = (n − 1)(n − 2)(n − 3)Γ(n − 3) = . . . . . . = (n − 1)(n − 2)(n − 3) . . . (2)(1)Γ(1) = (n − 1)! because: Γ(1) =

Z

0

+∞

x

1−1 −x

e dx =

Z

0

+∞

e−x dx =

Simulation

155 +∞

Z

d 



h

−e−x dx = −e−x dx Taking now into account that: =

0

i+∞ 0

= 0 − (−1) = 1.

Γ(n) = (n − 1)! the probability density of the Erlang distribution is:  n n−1 −αx  e ,  (α /(n − 1)!) x

f (x) = 

0,



x ≥ 0,

elsewhere.

The mean value and the variance of a random variable X Erlang distributed with the parameters α and n are: E(X) =

n , α

V (X) =

n . α2

For any random variable X, its moment generating function, denoted by MX (t), is defined by: 

tX

MX (t) = E e



=

Z

+∞

etx f (x) dx,

−∞

where f (x) is the probability density of X. Its name is justified if we take the successive derivatives of MX (t) with respect to the real variable t and we calculate their values at t = 0, namely: 

MX0 (t) = E XetX 

M ”X (t) = E X 2 etX



⇒

MX0 (0) = E(X),



⇒

M ”X (0) = E(X 2 ),

and, inductively, for every positive integer k, we have: 

(k)

MX (t) = E X k etX



⇒

(k)

MX (0) = E(X k ).

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

We also have: MX (0) =

Z

+∞

−∞

0x

e f (x) dx =

Z

+∞

f (x) dx = 1.

−∞

Remark: If only a finite number of moments E(X k ), (k = 1, . . . , m) are given, then there are many probability densities of the random variable X compatible with these moments. A probability density is uniquely identified with the sequence of all its moments {E(X k ), (k = 1, 2, . . .)} and, as the derivatives of the moment generating function MX (t) at t = 0 give all the moments of X,

Simulation

156

we can say that the distribution of the random variable X is uniquely identified with its moment generating function. 2 Proposition 1: If X is a random variable Gamma distributed with parameters α > 0 and β > 0 then, for t < α, we have: MX (t) =



α α−t

β

.

Proof: We have: Z

MX (t) = =

Z

0

+∞

+∞

etx f (x) dx =

−∞

αβ Z +∞ β−1 −(α−t)x α x e dx. xβ−1 e−αx dx = Γ(β) Γ(β) 0 β

etx

Making the substitution: (α − t)x = y,

x=

y , α−t

dx =

dy , α−t

we get: αβ 1 1 Z +∞ β−1 −y y e dy = MX (t) = Γ(β) (α − t)β−1 α − t 0 α αβ 1 Γ(β) = = β Γ(β) (α − t) α−t 

β

. 2

Remark: Let us use the formula just obtained for the moment generating function in order to calculate the mean value and the variance of the Gamma distribution with the parameters α > 0 and β > 0. The first derivative of the moment generating function is: MX0 (t)

α =β α−t 

β α = α−t α−t As MX (0) = 1, we obtain: Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.



β

β−1

=

α = (α − t)2

β MX (t). α−t

E(X) = MX0 (0) =

β . α

The second derivative of the moment generating function is: M ”X (t) =

β β MX (t) + MX0 (t) = 2 (α − t) α−t

Simulation

157 !2

β β MX (t) + 2 (α − t) α−t

=

MX (t),

from which we get the second moment of X, namely: β β E(X ) = M ”X (0) = 2 + α α 2

!2

.

The variance of X is: β β V (X) = E(X 2 ) − (E(X))2 = 2 + α α

!2

β − α

!2

=

β . α2

Proposition 2: If X is a random variable exponentially distributed with parameter α > 0 then, for t < α, we have: MX (t) =

α . α−t

Proof: We have: MX (t) = =

Z

+∞

Z

+∞

etx f (x) dx =

−∞

etx αe−αx dx = α

0

Z

+∞

e−(α−t)x dx.

0

Making the substitution: (α − t)x = y,

x=

y , α−t

dx =

dy , α−t

we get: MX (t) = α

α h −y i+∞ 1 Z +∞ −y α e dy = −e . 2 = 0 α−t 0 α−t α−t

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Based on the last two propositions, we may prove an important third proposition which is to be used both for sampling from the Erlang distribution and in queueing theory later on. Proposition 3: If X1 , . . . , Xn are independent random variables exponentially distributed with the parameter α > 0 then their sum X = X1 + . . . + Xn is Erlang distributed with parameters α and n. Proof: As the random variables X1 , . . . , Xn are independent and exponentially distributed with the parameter α, the moment generating function of their sum X is: 



h

i

MX (t) = E etX = E et(X1 +...+Xn ) =

Simulation

158 

tX1

=E e

tXn

...e





tX1

=E e





tXn

...E e



=

n α α α = MX1 (t) . . . MXn (t) = ... = , α−t α−t α−t which is the moment generating function of the Gamma distribution with parameters α and n, which is the Erlang distribution with the parameters α and n. 2 As the sampling from an exponential distribution may be done using a very simple formula obtained in section 2, the last proposition allows us to sample from an Erlang distribution with the parameters α and n by using:





1 1 x = x1 + . . . + xn = − log R1 + . . . + − log Rn = α α 







1 1 = − (log R1 + . . . + log Rn ) = − log(R1 . . . Rn ). α α We notice that n random numbers R1 ,. . . Rn from the unit interval [0, 1] are used for getting one value of the random variable X Erlang distributed with the parameters α and n. Problem 9: Get a sample of size 2, by hand, and size 100, using Minitab, from the Erlang distribution with the parameters α = 0.1 and n = 3. Solution: (a) We use the formula: x = −10 log(R1 R2 R3 ), for two sets of three random numbers from [0, 1]. For: R1 = 0.673284,

R2 = 0.479909,

R3 = 0.948578,

we get the first value of X: x1 = 11.82538082, and for:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

R1 = 0.613960,

R2 = 0.593277,

R3 = 0.934123,

we get the second value of X: x2 = 10.7806653. (b) The Minitab commands are: random 100 c1; gamma 3 10. print c1 where Minitab requires to identify the Gamma distribution we are sampling from by listing, in the corresponding subcommands of the ‘random’ command, the values of n and 1/α, in this order, i.e., ‘gamma n 1/α.’ which in our case are: n = 3 and 1/α=1/0.1=10. 2

Simulation

159

6. Monte Carlo method 1. Area of a circle. Assume that we do not know the formula which gives the area of a circle and we want to approximate it by using a probabilistic experiment, namely, throwing a dart, randomly and uniformly, at a square containing the circle, and counting the proportion of random points thus obtained that fall inside or on the circle. Problem 10: Apply the Monte Carlo method for approximating the area of the circle centered at (1,2) with radius 5. Solution: The circle: (x − 1)2 + (y − 2)2 = 25 may be inscribed in the square of side 10 centered at (1,2). The square is a simple geometric figure whose area is 102 = 100, but we assume that we do not know the formula for calculating the area of a circle. Let [a, b] be an arbitrary interval on the real axis and X a random variable uniformly distributed on [a, b]. Its probability density is: f (x) =

1 , b−a

a ≤ x ≤ b,

and zero elsewhere. The cumulative probability distribution is: F (x) =

Z

x

f (t) dt =

a

x−a , b−a

a ≤ x ≤ b.

Using the method of inversion, if R is a random number from [0, 1], by taking R = F (x) we obtain the formula x = a + (b − a)R for getting random numbers from [a, b]. Returning to our problem, we want to generate random points (x, y) that cover uniformly the square [−4, 6]×[−3, 7]. Therefore, if R1 and R2 are random numbers from [0, 1], we calculate:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

x = −4 + 10 R1 ,

y = −3 + 10 R2 .

Let us take n such random points (x, y) and let m be the number of those random points falling inside or on the circle, i.e., satisfying the inequality: (x − 1)2 + (y − 2)2 − 25 ≤ 0. The area of the circle is approximated by the proportion of the area of the square covered by the points (x, y) falling inside or on the circle: A1 =

m m × (area of the square) = × 100. n n

Simulation

160

y (−4, 7)

(6, 7)

6

'$

.

. &%

(0, 0)

(−4, −3)

-x

(6, −3)

Figure 10. Area of a circle inscribed in a square. This is what happens in one run of the simulation process. Let us take N such runs and let A1 , . . . , AN be the respective approximations of the area A of the circle. Let: A=

N 1 X Ai , N i=1

s2 =

N 1 X (Ai − A)2 , N − 1 i=1

be the sample mean and the sample variance. A 100(1 − α)% confidence interval for the area A of the circle is:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

s s A − tα/2,N −1 √ < A < A + tα/2,N −1 √ , N N where tα/2,N −1 is the critical point of the t-distribution with N − 1 degrees of freedom corresponding to the level of significance α. The approximation of A is good to the extent to which we cover the square as well as possible with random points. Therefore, n has to be a large number and the Monte Carlo method just presented gives good results only when a computer is used for implementing the approach just explained. Using a text editor, like ‘Notepad’, ‘WordPad’, ‘Word’, or ‘ChiWriter, for instance, we write the subprogram ‘area.txt’, namely: random k1 c1; uniform −4 6. random k1 c2; uniform −3 7. let c3 = (c1−1)**2+(c2−2)**2−25 code (−25:0)1 (0:25)0 c3 c4 let k2 = sum(c4) let k3 = 100*k2/k1

Simulation

161

let c5(k4) = k3 let k4 = k4+1 We save this subprogram as ‘a:area.txt’ on a diskette in the drive ‘a:’, or as ‘d:area.txt’ on a CD in drive the ‘d:’, or as ‘c:\windows\area.txt’ on the hard disk. Then we start a Minitab session and we type the commands: let k1 = 10000 let k4 = 1 noecho execute ”a:area.txt” 25 {or: execute ”d:area.txt” 25, or: execute ”c:\windows\area.txt” 25} tinterval 95 c5 The subprogram ‘area.txt’ refers to what happens in one run of the simulation. There, k1 is the number n of random points, uniformly distributed in the square [−4, 6] × [−3, 7], used in one run; c1 and c2 contain the corresponding coordinates x and y of these random points; c3 contains the values of the expression (x − 1)2 + (y − 2)2 − 25; c4 contains 1 if the corresponding random point falls inside or on the circle and 0 if it falls outside the circle; k2 contains the total number m of random points that fall inside or on the circle; k3 is the approximation of the area of the circle in the corresponding run and it is stored as the k4-th component in column c5. The counter k4 is updated by increasing its value with one time unit. The subprogram ‘area.txt’ is executed N = 25 times after initializing the values of k1 and k4 to be 10,000 and 1, respectively. As column c5 contains the approximations of the area A of the circle in 25 runs, the command ‘tinterval 95 c5’ gives the 95% confidence interval for the true area of the circle: 78.23 < A < 78.90. The middle of the confidence interval is A¯ = 78.565. The above simulation, based on the Monte Carlo method, has not used the formula A = πr2 from geometry for the area of a circle of radius r. If we use it now, at the end of the simulation process, we get the approximation of the transcendental number π as being:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

π=

78.565 A¯ = = 3.1425, 2 r 25

instead of the ‘exact’ value π ≈ 3.1416, for which the area of the circle of radius 5 is A ≈ 78.5398. The above simulation used 2 × 10, 000 × 25 = 500, 000 random numbers. Obviously, if we execute again the simulation, we do not obtain exactly the same confidence interval but the variations are small if n and N are chosen to be reasonably large. 2 A better solution: The simple solution just given is mathematically sound only if we assume that the random sample obtained in the N runs comes from a normally distributed population. In order to get rid of such a supposition, which is quite often hard to defend, we are going to give now a better solution

Simulation

162

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

where a confidence interval for the true area of the circle is obtained not from a random sample of approximations of the area of the circle, as we did before in the simpler solution, but from taking a random sample from the population of mean approximations of this area. In one run we get the mean approximation of several approximations and we take N such runs for getting a confidence interval for the true area of the circle using the mean of the mean approximations and the corresponding sample variance. We want to use the Monte Carlo Method for getting a better approximation of the area of the circle of radius 5 centered at (1,2), inscribed in a square of side 10, i.e., the area of the domain: (x − 1)2 + (y − 2)2 − 25 ≤ 0, without using the well-known formula from geometry but knowing that the area of a square is the square of its side. We use a text editor to type the subprogram ‘mary.txt’, which describes what happens in one run, and the subprogram ‘john.txt’ which executes ‘mary.txt’ n = 30 times storing the mean value of the area of our circle at the end of these n iterations. Afterwards, the subprogram ‘john.txt’ is executed N = 30 times to get a random sample of 30 mean approximations which are finally used to construct a 95% confidence interval for the true area of the circle. We move the cursor, clicking on the beginning and the end of the following sequence of choices, followed by typing the respective commands: {Start → Programs → Accessories → Notepad} random k1 c1; uniform −4 6. random k1 c2; uniform −3 7. let c3 = (c1−1)**2+(c2−2)**2−25 code (−25:0)1 (0:25)0 c3 c4 let k2 = sum(c4) let k3 = 100*k2/k1 let c5(k4) = k3 let k4 = k4+1 {File → Save As} a:mary.txt {or: ‘d:mary.txt’, or: ‘c:\windows\mary.txt’, respectively} {Save} {File → New} let k1 = 1000 let k4 = 1 noecho execute ”a:mary.txt” 30 {or: execute ”d:mary.txt” 30, or: execute ”c:\windows\mary.txt” 30}

Simulation

163

let c6(k5) = mean(c5) let k5 = k5+1 {File → Save As} a:john.txt {or: ‘d:john.txt’, or: ‘c:\windows\john.txt’, respectively} {Save} {File → Exit} {Start → Programs → Minitab on Windows → Minitab} {Click anywhere in the upper half of the screen to activate the Minitab Session} {Editor → Enable Command Language} let k5 = 1 noecho execute ”a:john.txt” 30 {or: execute ”d:john.txt” 30, or: execute ”c:\windows\john.txt” 30} tinterval 95 c6 {File → Exit} {Start → Shut Down} In the above program: k1 is the number of random points from the square of side 10 used in one run; c1 contains the values of x in one run; c2 contains the values of y in one run; c3 contains the values of (x − 1)2 + (y − 2)2 − 25 in one run; c4 contains the value 1 if the corresponding random point is inside or on the circle and the value 0 if the corresponding random point is in the equare but outside the circle in one run; k2 gives the number of random points that are inside or on the circle in one run; k3 contains an approximation of the area of a circle in one run; k4 is a counter of the components of column c5; c6 contains the random sample of 30 mean approximations of the area of a circle; k5 is a counter of the components of column c6. The final command ‘tinterval 95 c6’ gives a 95% confidence interval for the true area of a circle based on the mean of the mean approximations. The output looks like:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Variable N Mean StDev SE Mean 95.0% CI c6 30 78.5374 0.2544 0.0464 (78.4425, 78.6324) Remark: Therefore, we are 95% confident that the true area of the circle is in between 78.4425 and 78.6324. Indeed, if we use the formula for the area of a circle from geometry, we get: A = πr2 ≈ 3.1416 × 52 = 78.5394, which is also only an approximation because π is a transcendental number, namely, a real number with an infinite sequence of decimals with no finite pattern of decimals that repeats itself. Let us notice that using the mean

Simulation

164

value of the area obtained above, and the formula for the area of a circle from geometry, we get the approximation of the value of π to be: π≈

78.5374 A = = 3.14150, 2 r 25

whereas its ‘true’ value (using six decimals) is: π = 3.141592. If we repeat this Minitab session we do not obtain exactly the same numerical values but when N is large, like in our example, the new confidence interval will be very close to the previous one. 2 2. Definite multiple integrals. In calculus, when we want to integrate a function f (x) on an interval [a, b], we take a partition a = x0 < x1 < . . . < xn−1 < xn = b of [a, b], with equidistant points, we calculate the Riemann’s sum of areas of rectangles erected on the intervals of our partition: Sn =

n X

f (ξi )(xi − xi−1 ),

i=1

where xi−1 < ξi < xi , and we take the limit of the sequence {Sn } for finer and finer partitions of the interval [a, b] in subintervals of equal length, making n tend to infinity. The Monte Carlo method completely changes the strategy, ˜ from the interval [a, b] and calculating the mean value taking random points R of the areas of the rectangles erected on the entire interval [a, b] with the ˜ namely: heights f (R), ˜ (b − a), f (R)

˜ ∈ [a, b]. R

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

˜ cover the interval [a, b] as The approximation is good if the random points R well as possible, in a uniform way. The result is expressed not as a unique number but by a confidence interval to which the true value of the integral belongs. Back to common sense but using a lot of random numbers! In order to simplify the writing and avoid cumbersome formulas, the Monte Carlo method is used here in order to approximate the value of a double definite integral. The generalization to an arbitrary multiple definite integral is quite obvious. Problem 11: Use the Monte Carlo method in order to approximate the definite double integral: θ=

Z

b1

a1

Z

b2

h(x, y)dxdy.

a2

Remark: As different textbooks of multivariate calculus use different interpretations of the multiple definite integrals, let us emphasize that in the above double integral the domain of integration is: {(x, y); a1 ≤ x ≤ b1 , a2 ≤ y ≤ b2 }.

2

Simulation

165

Solution: 1. Using the simple estimator. There is nothing random in this problem from calculus but we randomize it by replacing the variables x, y by ˜ 1 and R ˜ 2 , uniformly two random numbers, i.e., independent random variables R distributed in [a1 , b1 ] and [a2 , b2 ], respectively. Thus, we have: ˜ 1 = a1 + (b1 − a1 )R1 , R

˜ 2 = a2 + (b2 − a2 )R2 , R

where R1 and R2 are random numbers from [0, 1], i.e., two independent random variables uniformly distributed in the unit interval. The probability densities ˜ 1 and R ˜ 2 are: of R    1/(b1 − a1 ),

f1 (x) = 

0,



f2 (y) =

elsewhere.

   1/(b2 − a2 ),  

if a1 ≤ x ≤ b1 ,

0,

if a2 ≤ x ≤ b2 ,

elsewhere.

˜ 1 and R ˜ 2 are independent, the joint probability As the random variables R ˜1, R ˜ 2 ) is the product of the marginal probability density of the random vector (R densities, namely: f (x, y) = f1 (x)f2 (y) =

1 , (b1 − a1 )(b2 − a2 )

if a1 ≤ x ≤ b1 , a2 ≤ y ≤ b2 , and equal to 0 elsewhere. An unbiased estimator of a parameter θ is any random variable whose mean value is equal to θ. In our case, the random variable: ˜ R ˜1, R ˜ 2 ) = (b1 − a1 )(b2 − a2 ) h(R ˜1, R ˜2) h( is an unbiased estimator of θ. It is called the ‘simple estimator’. ˜ R ˜1, R ˜ 2 ) is: Indeed, the mean value of h(

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

=

b1

Z

a1

Z

b1

a1

Z

−∞

Z

= (b1 − a1 )((b2 − a2 )

+∞

Z

˜ R ˜1, R ˜ 2 )] = E[h(

˜ y) f (x, y)dxdy = h(x,

−∞

b2

h(x, y)

a2

Z

+∞

1 dxdy = (b1 − a1 )(b2 − a2 )

b2

h(x, y) dxdy = θ.

a2

Its variance is: ˜ R ˜ R ˜1, R ˜ 2 )] = E[(h( ˜1, R ˜ 2 ) − θ)2 ] = σ 2 = V [h(

Simulation

166 =

Z

+∞

−∞

Z

+∞

−∞

˜ y) − θ]2 f (x, y)dxdy = [h(x,

Z b1 Z b2 1 ˜ y) − θ]2 dxdy. = [h(x, (b1 − a1 )(b2 − a2 ) a1 a2 ˜1, R ˜ 2 ) from [a1 , b1 ] and [a2 , b2 ], In one run, take a pair of random numbers (R ˜ R ˜1, R ˜ 2 ). Take now N runs, respectively, and calculate the numerical value h( using N such pairs of random numbers:

˜ 1(1) , R ˜ 2(1) ), . . . , (R ˜ 1(N ) , R ˜ 2(N ) ), (R ˜ 1(i) is a random number from [a1 , b1 ], and R ˜ 2(i) is a random number from where R [a2 , b2 ], for every i, (1 ≤ i ≤ N ), and calculate the values: ˜ i = h( ˜ R ˜ 1(i) , R ˜ 2(i) ), h

(i = 1, . . . , N ).

Then, we calculate the sample mean: N X ˜= 1 ˜ i, h h N i=1

and the sample variance: s2 =

N 1 X ˜ 2. ˜ i − h] [h N − 1 i=1

A 100(1 − α)% confidence interval for θ is: ˜ − tα/2,N −1 √s < θ < h ˜ + tα/2,N −1 √s , h N N

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

where tα/2,N −1 is the critical point of the t-distribution with N − 1 degrees of freedom corresponding to the significance level α. 2. Using the better estimator. Let us use, in one run, n pairs of random numbers: ˜ 1,1 , R ˜ 2,1 ), . . . , (R ˜ 1,n , R ˜ 2,n ), (R ˜ 1,j is a random number from [a1 , b1 ], and R ˜ 2,j is a random number where R from [a2 , b2 ], for every j, (1 ≤ j ≤ n). We define: n X ˜ R ˜ 1,j , R ˜ 2,j ). ˜ 1,1 , R ˜ 2,1 ; . . . ; R ˜ 1,n , R ˜ 2,n ) = 1 H(R h( n j=1

˜ 1,1 , R ˜ 2,1 ; . . . ; R ˜ 1,n , R ˜ 2,n ) is also an unbiased estiThe random variable H(R mator of θ. Indeed, its mean value is: ˜ 1,1 , R ˜ 2,1 ; . . . ; R ˜ 1,n , R ˜ 2,n )] = E[H(R

Simulation

167 

=E =



n 1X ˜ R ˜ 1,j , R ˜ 2,j ) = h( n j=1

n 1X ˜ R ˜ 1,j , R ˜ 2,j )] = 1 nθ = θ. E[h( n j=1 n

˜ 1,1 , R ˜ 2,1 ; . . . ; R ˜ 1,n , R ˜ 2,n ) is called the ‘better estiThe random variable H(R ˜ 1,1 , . . . , R ˜ 2,n , mator’ because, due to the independence of all random variables R its variance, showing the spread of its values around the mean θ, is: 



n X ˜ R ˜ 1,j , R ˜ 2,j ) = ˜ 1,1 , R ˜ 2,1 ; . . . ; R ˜ 1,n , R ˜ 2,n )] = V  1 h( V [H(R n j=1





n X

n 1  1 X ˜ R ˜ ˜ 1,j , R ˜ 2,j )] = ˜ ˜  = 2V V [h( h(R1,j , R2,j ) = 2 n n j=1 j=1

σ 1 nσ = , 2 n n which tends to zero when n tends to +∞, which means that the accuracy of ˜ 1,1 , R ˜ 2,1 ; . . . ; R ˜ 1,n , R ˜ 2,n ) estimating the value of θ by using the estimator H(R ˜ 1,j , R ˜ 2,j ), (j = increases if a large number of pairs of random numbers (R 1, . . . , n), are used in one run. Take now N such runs, using 2nN random numbers: =

(i) ˜ (i) (i) ˜ (i) ˜ 1,1 ˜ 1,n (R , R2,1 ), . . . , (R , R2,n ),

(i = 1, . . . , N ),

(i) (i) ˜ 1,j ˜ 2,j where: R is a random number from [a1 , b1 ] and R is a random number from [a2 , b2 ], for every j, (j = 1, . . . n), and every i, (i = 1, . . . , N ). For each run i, (1 ≤ i ≤ N ), we calculate the numerical value: n 1X (i) ˜ (i) (i) ˜ (i) (i) ˜ (i) ˜ R ˜ 1,1 ˜ 1,n ˜ 1,j h( , R2,j ). Hi = H(R , R2,1 ; . . . ; R , R2,n ) = n j=1

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

We calculate the sample mean: N 1 X Hi , H= N i=1

and the sample variance: S2 =

N 1 X (Hi − H)2 . N − 1 i=1

Simulation

168

A 100(1 − α)% confidence interval for θ is: S S H − tα/2,N −1 √ < θ < H + tα/2,N −1 √ , N N where tα/2,N −1 is the critical point of the t-distribution with N − 1 degrees of freedom corresponding to the significance level α. 2 Problem 12: Use the Monte Carlo method in order to approximate the value of the definite simple integral: θ=

Z

0

1

ex − 1 dx. e−1

1. Solve the problem by using a simple pocket calculator, the simple estimator, and 16 runs. 2. Solve the problem by using Minitab and the better estimator. Solution: 1. Particularizing the general formalism presented above, we have: ex − 1 h(x) = , e−1 eR − 1 ˜ h(R) = (1 − 0)h(R) = , e−1 where R is a random number from [0, 1]. In each run we use one random number R. In N = 16 runs we need 16 random numbers from [0, 1], taken from a table with random numbers. Thus, taking for R(i) , (i = 1, . . . , 16), the values: 0.058962, 0.613960, 0.347270, 0.767638,

0.673284, 0.593277, 0.564395, 0.893129,

0.479909, 0.934123, 0.352943, 0.391962,

0.948578, 0.178239, 0.364609, 0.787674,

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

˜ (i) ), (i = 1, . . . , 16), and we obtain: we calculate the corresponding values of h(R 0.0353463, 0.4933614, 0.2416360, 0.6719916,

0.5590850, 0.4713486, 0.4413616, 0.8396534,

0.3584554, 0.8991427, 0.2463217, 0.2792798,

0.9207077, 0.1135500, 0.2560412, 0.6973695.

The sample mean: 16 X ˜ (i) ) = 7.5246518 = 0.4702907, ˜= 1 h(R h 16 i=1 16

Simulation

169

the sample variance: s2 =

16 1 X ˜ (i) ) − h] ˜ 2 = 1.1385018 = 0.0759001, [h(R 16 − 1 i=1 15

and the sample standard deviation: √ √ s = s2 = 0.0759001 = 0.2754998, allow us to get a 95% confidence interval for θ: ˜ − t0.025,15 √s < θ < h ˜ + t0.025,15 √s , h 16 16 where, from statistical tables, the critical point of the t-distribution with 15 degrees of freedom corresponding to the significance level α = 0.05 is: tα/2,N −1 = t0.025,15 = 2.131. As we have: ˜ = 0.4702907, h

s √ = 0.0688749, 16

s t0.025,15 √ = 0.1467725, 16

a 95% confidence interval for the value of θ is: 0.3235182 < θ < 0.6170632 We are therefore 95% confident that the value of the given integral belongs to the above interval. Obviously, the approximation is not very good because we used the simple estimator and only a sample of size 16, which is a ridiculously small number in simulation. However, let us notice that the ”exact” value of this integral is: θ=

Z

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

0

1

ex − 1 e−2 dx = , e−1 e−1

which, as e is a transcendental number, can nevertheless be only approximated, namely, θ ≈ 0.4180233, and it does belong to the confidence interval obtained. 2. Using any text editor, like ‘Notepad’, ‘WordPad’, ‘Word’, or ‘ChiWriter’, we type the subprogram ‘mary.txt’: random k1 c1; uniform 0 1. let c2 = (exp(c1)−1)/(exp(1)−1) let k2 = mean(c2) let c3(k3) = k2 let k3 = k3+1

Simulation

170

We save this subprogram as the textfile ‘a:mary.txt’ on a diskette in the drive ‘a:’, or as ‘d:mary.txt’ on a CD in the drive ‘d:’, or, on the hard disk, as ‘c:\windows\mary.txt’. We start a Minitab session and type the commands: let k1 = 1000 let k3 = 1 noecho execute ”a:mary.txt” 50 {or: execute ”d:mary.txt” 50, or: execute ”c:\windows\mary.txt” 50} tinterval 95 c3 In this program: k1 is the number n of random points Ri from the domain of integration [0, 1] taken in one run (subsequently, it was taken to be equal to ˜ i ); the constant k2 contains 1000); c2 contains the corresponding values of h(R ˜ in the respective run which is stored in the k3-th row of the the value of h column c3. The subprogram ‘mary.txt’ is executed 50 times and the last command of the main program asks for a 95% confidence interval for the value of the integral θ after N = 50 runs. The results obtained are: Variable N Mean StDev SE Mean 95.0% CI c3 50 0.41896 0.00906 0.00128 (0.41638,0.42153) which, obviously, is a much better approximation of the value of the integral θ. If we repeat the Minitab session we will obtain a slightly different confidence interval but the variations are small if the number of runs (i.e., the sample size) N is large. 2 Remark: Dealing with a simple definite integral allows us to see, intuitively, the difference between the strictly deterministic approach given in calculus and the random approach used by the Monte Carlo method in calculating the integral of a function of one variable on an interval [a, b]. Figure 11 shows what happens in integral calculus: f (x) 6

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

'

$

&

%

r r r r r - x a a1 a2 a3 b Figure 11. The deterministic approach: We take a partition ∆n of the domain of integration [a, b] consisting of n

Simulation

171

equal disjoint subintervals: a = a0 < a1 < . . . < ai−1 < ai < . . . < an−1 < an = b, and the value of the integral is bounded by: Z

s(∆n ) ≤

b

a

f (x) dx < S(∆n ),

where the lower bound is the sum of the areas of the rectangles erected on the subintervals of the partition ∆n situated under the given curve y = f (x), namely: s(∆n ) =

n X

min

ai−1 ≤x≤ai

i=1

f (x)(ai − ai−1 ),

and the upper bound is the sum of the areas of the rectangles erected on the subintervals of the partition ∆n and raising above the curve y = f (x), namely: S(∆n ) =

n X i=1

max

ai−1 ≤x≤ai

f (x)(ai − ai−1 ),

The approximation is better when n increases. Figure 12 shows what the Monte Carlo method does: f (x) 6 '

$

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

&

%

r r r r - x ˜ ˜ a b R1 R2 Figure 12. The probabilistic approach: ˜ j , (j = 1, . . . , n) from the domain of integration Here, random points R ˜ j , we calculate the area of [a, b] are taken, in one run, and for each point R ˜ j ). The the rectangle erected on the whole interval [a, b] with the height f (R better estimator in the corresponding run is the mean value of the areas of these rectangles, namely:

H=

n 1X ˜ j ). (b − a)f (R n j=1

Simulation

172

We repeat this for N such runs and we obtain the corresponding values H1 , . . . , HN of the better estimator. As the mean value of the better estimator is just the value of our integral, we use this sample of size N in order to get a 95% confidence interval whose limits give a lower bound and an upper bound for the given integral, namely: Z b S S f (x) dx < H + t0.025,N −1 √ , H − t0.025,N −1 √ < a N N

where H is the sample mean and S is the sample standard deviation, namely: H=

N 1 X Hi , N i=1

S2 =

N 1 X [Hi − H]2 , N − 1 i=1

and t0.025,N −1 is the critical point of the t-distribution with N − 1 degrees of freedom corresponding to the significance level α = 0.05. This critical point t0.025,N −1 may be replaced by the critical point Z0.025 of the standard normal distribution if the sample size N > 20, such that: P (−Z0.025 < Z < Z0.025 ) =

Z

Z0.025

−Z0.025

1 2 √ e−x /2 dx = 0.95. 2π

The approximation becomes better if we use more random points in one run, uniformly covering better the domain of integration [a, b], and if we perform more runs. 2 Problem 13: Write a Minitab program for approximating the double integral: Z 1Z 3 x dx dy 0 1 1 + xy by using the Monte Carlo method with the better estimator. Solution: The domain of integration is:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

{(x, y); 0 ≤ x ≤ 1, 1 ≤ y ≤ 3}, and therefore a1 = 0, b1 = 1, a2 = 1, b2 = 3, and the function which is to be integrated is: x h(x, y) = . 1 + xy Using any text editor, like ‘Notepad’, ‘WordPad’, ‘Word’, or ‘ChiWriter’, we type the subprogram ‘integral.txt’ which describes what happens in one arbitrary run: random k1 c1;

Simulation

173

uniform 0 1. random k1 c2; uniform 1 3. let c3 = (1−0)*(3−1)*c1/(1+c1*c2) let k2 = sum(c3)/k1 let c4(k3) = k2 let k3 = k3+1 We save this subprogram as ‘a:integral.txt’ on a diskette in the drive ‘a:’, or as ‘d:integral.txt’ on a CD in the drive ‘d:’, or as ‘c:\windows\integral.txt’ on the hard disk. Afterwards, we start a Minitab session and we run the following main program: let k1 = 1000 let k3 = 1 noecho execute ”a:integral.txt” 25 {or: execute ”d:integral.txt” 25, or: execute ”c:\windows\integral.txt” 25} tinterval 95 c4 In this program: k1 corresponds to n = 1000; k3 corresponds to i and takes (i) ˜ 1,j on values from 1 to N = 25; column c1 contains the random numbers R for (i) ˜ j = 1, . . . , 1000, in the i-th run; column c2 contains the random numbers R2,j (i) ˜ (i) ˜ R ˜ 1,j for j = 1, . . . , 1000, in the i-th run; column c3 contains the values h( , R2,j ), for j = 1, . . . , 1000, in the i-th run; k2 contains the value of Hi in the i-th run. The last command asks for a 95% confidence interval for θ. Running this program, the output obtained was: Variable N Mean StDev SE Mean 95.0% CI C4 25 0.46294 0.00657 0.00131 (0.46023,0.46566)

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

where: ‘N ’ is the number of runs, √ namely 25; ‘Mean’ is H = 0.46294, ‘StDev’ is S = 0.00657; ‘SE Mean’ is S/ N = 0.00131; and CI is the 95% confidence interval required, i.e., 0.46023 < θ < 0.46566. 2 Remark: Running this program again will give a slightly different confidence interval for the value θ of the double integral, but the accuracy of this approximation will increase if the small number of runs, namely 25 here, is replaced by a larger number, say 100. 2 3. Solving equations. (a) Fermat’s quintic equation. It is well-known that Pierre de Fermat long ago conjectured and Andrew Wiles recently proved that no distinct positive integers could be the roots of an equation of the form: xn + y n = z n ,

(n > 2).

Simulation could be used, however, to approximate the solution of such an

Simulation

174

equation. Problem 14: Use the Monte Carlo method to approximate the solution of the the quintic Fermat’s equation: x5 + y 5 = z 5 , where x, y, z are distinct positive integers larger than 1. Solution: Dividing by z 5 , we get the quintic equation in two variables: p5 + q 5 = 1,

p=

x , z

y q= , z

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

where: 0 < p < 1, 0 < q < 1. In this context, Fermat’s last theorem says that there are no rational numbers p, q from the unit interval [0, 1] satisfying this quintic equation. We are using the Monte Carlo method, however, to find rational numbers p and q which make the absolute value of the expression p5 + q 5 − 1 as small as possible. Therefore, we are using many rational values of p and q from the unit interval [0, 1] and we keep those values of p and q for which we get: min |p5 + q 5 − 1|. The common sense idea is again to uniformly cover, as well as possible, the square [0, 1] × [0, 1] with random points, selecting those values of p and q for which the above quintic equation is satisfied with a given approximation error. Again, Minitab on Windows proves to be very convenient for implementing this approximation. Using any text editor, for instance ‘Notepad’, ‘WordPad’, ‘Word’, or ‘ChiWriter’, we create a text file named ‘a:fermat.txt’, on a diskette in the drive ‘a:’, or ‘d:fermat.txt’, on a CD in the drive ‘d:’, or ‘c:\windows\fermat.txt’, on the hard disk. Its content is: random k1 c1; uniform 0 1. random k1 c2; uniform 0 1. let c3 = c1**5+c2**5−1 let c4 = abs(c3) let k2 = min(c4) code (k2:k2)1 (k2:1)0 c4 c5 copy c1−c3 c6−c8; use c5 = 1. let c9(k3) = c6(1) let c10(k3) = c7(1)

Simulation

175

let c11(k3) = c8(1) let k3 = k3+1 We access Minitab on Windows, successively clicking the mouse on: ‘Session’, ‘Editor’, and ‘Enable Command Language’, followed by typing: let k1 = 1000 let k3 = 1 noecho execute ”a:fermat.txt” 50 {or: execute ”d:fermat.txt” 50, or: execute ”c:\windows\fermat.txt” 50} let c12=abs(c11) let k4 = min(c12) code (k4:k4)1 (k4:1)0 c12 c13 copy c9−c11 c14−c16; use c13 = 1. print c14−c16 In the subprogram ‘fermat.txt’: the constant k1 shows the number of random points uniformly distributed in the unit square used in one simulation run; the columns c1 and c2 contain the random values of p and q, respectively; the column c3 contains the corresponding values of p5 + q 5 − 1; column c4 contains the absolute values |p5 + q 5 − 1|; the constant k2 contains the minimum value min |p5 + q 5 − 1| in the respective run. The components of a pair (p, q) for which |p5 + q 5 − 1| is minimum in the respective run are kept in a distinct row of columns c9 and c10, respectively. The subprogram ‘fermat.txt’ is executed 50 times and the columns c14 and c15 contain the first component p and the second component q, respectively, of a pair (p, q) for which the smallest value of |p5 + q 5 − 1| is obtained in all those 50 runs. This smallest value of |p5 + q 5 − 1| is in the corresponding row of column c16. Using 1000 random points from the unit square in one run and taking 50 runs into account, the following solution was obtained: p = 0.589644,

q = 0.985323,

in column c14 and column c15, respectively, giving the approximation error:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

|p5 + q 5 − 1| = 0.0000167 in column c16. 2 Remarks: 1. In the subcommand ‘use c13 = 1.’ the blank spaces before and after ‘=’ are essential. Use the ‘Space-bar’ for getting them. 2. Starting with the release 15 of Minitab on Windows, the commands: copy c9−c11 c14−c16; use c13 = 1. may be replaced by the commands:

Simulation

176

copy c9−c11 c14−c16; include; where ”c13 = 1”. It is anticipated that in a future release of Minitab the subcommand ‘use’ will be discarded and replaced by the subcommand ‘include’ just mentioned. 2 (b) Polynomial equations. The same random search methodology may be applied for approximating the real or complex solutions of any polynomial equation: n X

ak xn−k = 0,

k=0

which, by taking: x = r(cos θ + i sin θ), √ with r ≥ 0, 0 ≤ θ ≤ 2π, i = −1, becomes: n X

ak rn−k cos(n − k)θ = 0,

n X

ak rn−k sin(n − k)θ = 0.

k=0

k=0

In order to approximate the common solutions of the last two equations in a chosen domain: {x | r1 ≤ r ≤ r2 , θ1 ≤ θ ≤ θ2 } from the complex plane, we cover this domain as uniformly as possible with random points and we select those of them for which the relaxed constraints: −1 ≤

n X

ak rn−k cos(n − k)θ ≤ 2 ,

n X

ak rn−k sin(n − k)θ ≤ 2 ,

k=0

−1 ≤

k=0

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

of the two equations are simultaneously satisfied, where 1 and 2 are the approximation errors chosen by us. Problem 15: Use the Monte Carlo method to approximate the solutions of the equation: x3 + 1 = 0 on the unit circle in the complex plane. Solution: Taking: x = cos θ + i sin θ,

Simulation

177

where: 0 ≤ θ ≤ 2π ≈ (2)(3.14159) = 6.28316,

i=

√

−1,

the equation: x3 + 1 = 0 becomes the complex equation: cos 3θ + i sin 3θ + 1 = 0, which is equivalent to two real equations:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

cos 3θ + 1 = 0,

sin 3θ = 0,

(0 ≤ θ ≤ 6.28316).

Using any text editor, like ‘Notepad’, ‘WordPad’, ‘Word’, or ‘ChiWriter’, we type the subprogram: random k1 c1; uniform 0 6.28319. let c2 = cos(3*c1)+1 let c3 = sin(3*c1) code (k2:k3)1 (k4:k5)0 c2 c3 c4 c5 let c6 = c4*c5 copy c1 c7; use c6 = 1. let c8 = cos(c7) let c9 = sin(c7) print c7−c9 and we save it as ‘a:equation.txt’ on a diskette in the drive ‘a:’, or as ‘d:equation.txt’ on a CD in the drive ‘d:’, or as ‘c:\windows\equation.txt’ on the hard disk. Then we start a Minitab session and execute the main program: let k1 = 1500 let k2 = −0.01 let k3 = 0.01 let k4 = −1 let k5 = 2 noecho execute ”a:equation.txt” {or: execute ”d:equation.txt”, or: execute ”c:\windows\equation.txt”} Here: k1 is the number of random complex points uniformly distributed in the domain in one run of the program, k2 = −1 , k3 = 2 , where 1 = 2 = 0.01, whereas k4 = −1 is the lower bound and k5 = 2 is the upper bound for the possible values of the left-hand sides of the two real equations given above, taking into account that: 0 ≤ cos 3θ + 1 ≤ 2,

−1 ≤ sin 3θ ≤ 1.

Simulation

178

The final approximations of the solutions of our problem are in column c7, for θ, in column c8, for cos θ, and in column c9, for sin θ. After only one run the following values were obtained: c7 3.14255 3.14094 3.14241 1.04712 1.04718 5.23379 5.23331

c8 −1.00000 −1.00000 −1.00000 +0.50007 +0.50001 +0.49810 +0.49768

c9 +0.000955 +0.000653 −0.000819 +0.865987 +0.866017 −0.867122 −0.867359

Within an error of ±0.01, these values are good approximations of the complex solutions:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

x1 = cos π + i sin π = −1 + i 0, x2 = cos π3 + i sin π3 = √ = (1/2) + i ( 3/2) ≈ 0.5 + i 0.866025, x3 = cos 5π + i sin 5π = 3 3 √ = (1/2) − i ( 3/2) ≈ 0.5 − i 0.866025. Obviously, we may take several runs, by executing the subprogram ‘equation.txt’ several times, with different approximation errors. 2 Remarks: 1. In the subcommand ‘use c6 = 1.’ the blank spaces before and after ‘=’ are essential. Use the ‘Space-bar’ for getting them. 2. Starting with the release 15 of Minitab on Windows, the commands: copy c1 c7; use c6 = 1. may be replaced by the commands: copy c1 c7; include; where ”c6 = 1”. It is anticipated that in a future release of Minitab the subcommand ‘use’ will be discarded and replaced by the subcommand ‘include’ just mentioned. 2 4. Optimization. (a) Approximating the optimum solution of a linear program. Linear programming deals with the optimization of a linear function, called objective function, subject to linear inequalities. The points satisfying all the constraints make up the feasible space and are called feasible solutions. The optimum points, i.e., the feasible solutions at which the objective function is optimum (maximum or minimum), are located at corner points on

Simulation

179

the boundary of the polygonal feasible space and therefore they are solutions of some linear equations. Usually, the Simplex algorithm is used for solving linear programs. It is a search algorithm which starts from a corner point of the feasible space and subsequently jumps to neighbouring corner points until the optimum of the objective function is reached. Nobody has proved that this is the best strategy for solving linear programs and, in fact, some people have tried to locate the optimum corner points by following some well defined paths inside the feasible space. In order to approximate the optimum solution of a linear program we may adopt, however, the maximum uncertainty strategy used by the general Monte Carlo method in simulation. As the discrete uniform distribution is the most uncertain, or unbiased, probability distribution when no restrictions are imposed, we cover as uniformly as possible the feasible space generated by the constraints and select those solutions for which the objective function is most improved. The main advantage of the optimization based on heuristic simulation is that it may be applied even when the objective functions and/or the constraints are not necessarily linear. For performing such a heuristic simulation, the computer package Minitab on Windows proves to be efficient and user friendly. The Monte Carlo method breaks away again with the current routine and simply calculates the values of the objective function at points randomly chosen from the feasible space, selecting those solutions for which the best objective value is obtained, and the results are good to the extent to which a large number of such random points, uniformly covering the feasible space, are used. With the more and more powerful new computers, having huge memory and a tremendous speed, the efficiency of the Monte Carlo method has increased dramatically. Problem 16: Use the Monte Carlo method for approximating the optimum solution of the following normal linear program:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

max z subject to: 2x1 − x2 x1 + 2x2 x 1 , x2

=

3x1 + x2

≤ 2 ≤ 5 ≥ 0

Solution: From the second constraint we get: x1 ≤ 5 and x2 ≤ 2.5. From the first constraint we have 2 x1 ≤ x2 + 2 ≤ 4.5, which implies x1 ≤ 2.25. Therefore, the feasible space of the linear program is included into the rectangle: {(x1 , x2 ); 0 ≤ x1 ≤ 2.25, 0 ≤ x2 ≤ 2.5}. Using any text editor, like ‘Notepad’, ‘WordPad’, ‘Word’, or ‘ChiWriter’, for instance, we create a text file named ‘lp.txt’. Its content is: random k1 c1;

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Simulation

180

uniform 0 2.25. random k1 c2; uniform 0 2.5. let c3 = 3*c1+c2 let c4 = 2*c1−c2−2 let c5 = c1+2*c2−5 code (−10000:0)1 (0:10000)0 c4 c6 code (−10000:0)1 (0:10000)0 c5 c7 let c8 = c6*c7 copy c1−c3 c9−c11; use c8 = 1. let k2 = max(c11) code (k2:k2)1 (0:k2)0 c11 c12 copy c9−c11 c13−c15; use c12 = 1. let c16(k3) = c13(1) let c17(k3) = c14(1) let c18(k3) = c15(1) let k3 = k3+1 We save this subprogram as the textfile ‘a:lp.txt’ on a diskette in the drive ‘a:’, or as ‘d:lp.txt’ on a CD in the drive ‘d:’, or as ‘c:\windows\lp.txt’ on the hard disk. We access Minitab on Windows, successively clicking the mouse on: ‘Session’, ‘Editor’, and ‘Enable Command Language’, followed by typing: let k1 = 1000 let k3 = 1 noecho execute ”a:lp.txt” 50 {or: execute ”d:lp.txt” 50, or: execute ”c:\windows\lp.txt” 50, respectively} let k4 = max(c18) code (k4:k4)1 (0:k4)0 c18 c19 copy c16−c18 c20−c22; use c19 = 1. print k4 c20−c22 In the above subprogram: c1 and c2 contain the values of the variables x1 and x2 , uniformly distributed in [0, 2.25] and [0, 2.5], respectively; c3 contains the values of the objective function 3x1 +x2 ; c4 and c5 contain the corresponding values of the constraints 2x1 − x2 − 2 and x1 + 2x2 − 5, respectively; k1 is the number of solutions (x1 , x2 ) randomly selected in one run of the simulation; any solution (x1 , x2 ) randomly generated gets the value 1 in c8 if it is feasible and the value 0 if it is not; c9, c10, and c11 contain only those values of x1 , x2 , and z that correspond to the feasible solutions generated; k2 is the maximum objective value for the feasible solutions generated in one run; c13

Simulation

181

and c14 contain those feasible values of x1 and x2 for which the corresponding objective value in the respective run is maximum. One such best feasible solution in the corresponding run is put in a distinct row k3, which is updated in each run, in columns c16 and c17, respectively. The subprogram ‘lp.txt’, randomly generating 1000 solutions (x1 , x2 ) in one run, is executed 50 times. The constant k4 contains the maximum of the maximum objective values obtained in those 50 runs, whereas columns c20 and c21 contain the values of the feasible decision variables x1 and x2 , respectively, for which the corresponding objective value is equal to k4. The approximations of the optimum solution of the given linear program obtained at the end of the 50 executions of the subprogram ‘lp.txt’ are given in the rows of column c20 (the values of x1 ) and column c21 (the values of x2 ), corresponding to the optimum objective value k4. One such a Minitab session gave the results: z ∗ = 6.98702, in k4, x∗1 = 1.79644, in c20, and x∗2 = 1.59769, in c21. The exact optimum solution of the given linear program is: z ∗ = 7, x∗1 = 1.8, and x∗2 = 1.6. 2 Remarks: 1. In the subcommand ‘use c8 = 1.’ the blank spaces before and after ‘=’ are essential. Use the ‘Space-bar’ for getting them. 2. Starting with the release 15 of Minitab on Windows, the commands: copy c1−c3 c9−c11; use c8 = 1. may be replaced by the commands: copy c1−c3 c9−c11; include; where ”c8 = 1”. It is anticipated that in a future release of Minitab the subcommand ‘use’ will be discarded and replaced by the subcommand ‘include’ just mentioned. 2 It is a sound strategy to shrink the domain of possible values of the decision varaibles, by changing the corresponding scales of values. Even if, mathematically, the intervals [0, 1] and [0, 2], for instance, have the same cardinal, it is obvious that by using 1000 random numbers from each of them, we cover better the first interval.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Problem 17: Use the Monte Carlo method for solving the linear program: max z subject to: x1 + x2 2x1 + x2 x 1 , x2

=

4x1 + 3x2

≤ 40 ≤ 60 ≥ 0

Solution: The constraints imply that the two variables belong to the intervals: 0 ≤ x1 ≤ 30 and 0 ≤ x2 ≤ 40. Even if any two finite intervals have

Simulation

182

the same cardinal, when using a certain large but nevertheless finite set of random numbers, a shorter interval may be covered better than a longer one. In this case we shrink the intervals by changing the scale of possible values of the variables. Dividing the values of x1 and x2 by 20, in order to shrink the feasible space, we obtain the new linear program:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

max z subject to: x1 + x2 2x1 + x2 x 1 , x2

=

4x1 + 3x2

≤ 2 ≤ 3 ≥ 0

for which 0 ≤ x1 ≤ 1.5 and 0 ≤ x2 ≤ 2. Taking now random points (x1 , x2 ) covering uniformly the rectangle [0, 1.5] × [0, 2], we keep those points that satisfy the two constraints of the linear program, we calculate the value of the objective function at these points, and we select the points at which the objective value is maximum. Using any text editor, like ‘Notepad’, WordPad’, ‘Word’, or ‘ChiWriter’, for instance, we type the subprogram ‘maxlp.txt’ as a text file: random k1 c1; uniform 0 1.5. random k1 c2; uniform 0 2. let c3 = 4*c1+3*c2 let c4 = c1+c2−2 let c5 = 2*c1+c2−3 code (−10000:0)1 (0:10000)0 c4 c6 code (−10000:0)1 (0:10000)0 c5 c7 let c8 = c6*c7 copy c1−c3 c11−c13; use c8 = 1. let k2 = max(c13) copy c11 c12 c14 c15; use c13 = k2. let c20(k3) = k2 let c21(k3) = c14(1) let c22(k3) = c15(1) let k3 = k3+1 We save the subprogram as the text file ‘a:maxlp.txt’ on a diskette in the drive ‘a:’, or ‘d:maxlp.txt’ on a CD in the drive ‘d:’, or ‘c:\windows\maxlp.txt’ on the hard disk. We start a Minitab session and we type the commands: let k1 = 1000 let k3 = 1

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Simulation

183

noecho execute ”a:maxlp.txt” 100 {or: execute ”d:maxlp.txt” 100, or: execute ”c:\windows\maxlp.txt” 100} let k4 = max(c20) copy c21 c22 c31 c32; use c20 = k4. print k4 c31 c32 In this program: the constant k1 is the number of random points (x1 , x2 ) used during one run of the simulation; the columns c1 and c2 contain the random values of the variables x1 and x2 used in one run, respectively; the column c3 contains the corresponding values z of the objective function in one run; the columns c4 and c5 contain the difference between the left hand side and the right hand side of the two constraints, respectively, in one run; the column c6 contains 1 if the first constraint is satisfied by the corresponding pair (x1 , x2 ) and 0 if not; the column c7 contains 1 if the second constraint is satisfied by the corresponding pair (x1 , x2 ) and 0 if not; the column c8 contains 1 if both constraints are satisfied by the corresponding pair (x1 , x2 ) and 0 if not; the columns c11, c12, and c13 contain only those values of x1 , x2 , and z for which (x1 , x2 ) are the feasible solutions; the constant k2 is the maximum value of z obtained during one run of the simulation; the columns c14 and c15 contain the coordinates x1 and x2 of the feasible solutions (x1 , x2 ) for which the objective function z is maximum during the corresponding run of the simulation. All this refers to what happens in one run, which uses k1=1000 random points. The optimum objective value z and one optimum solution (x1 , x2 ) found thus far, in one run, are stored in the k3-th component of columns c20, c21, and c22, respectively. The counter k3 increases by one unit after each run. Executing the subprogram ‘maxlp.txt’ 100 times, which is equivalent to performing 100 runs, we take the best optimum objective value z, stored in k4, and the corresponding optimum feasible solutions (x1 , x2 ), stored in columns c31 and c32, respectively, from all 100 runs performed. The result obtained at the end of one of such a Minitab session was: z ∗ = 6.98664, x∗1 = 1.00070, x∗2 = 0.99462. Changing the scale, i.e., multiplying all the values by 20 for going back to the original range of values, we get: z ∗ = 139.733, x∗1 = 20.0140, x∗2 = 19.8923. Rounding the solution to the closest integers, we get z ∗ ≈ 140, x∗1 ≈ 20, x∗2 ≈ 20. The entire Minitab session lasted a couple of seconds on a laptop IBM Thinkpad computer. This is indeed the optimum solution which is normally obtained by applying the popular Simplex algorithm or, as we have only two variables here, the graphical method. 2 Remark: It is true that the Monte Carlo method only approximates the optimum solution but, on the other hand, it may be applied absolutely similarly to cases where the objective function or/and the constraints are not necessarily linear, whereas the Simplex algorithm cannot. This is a major change! 2

Simulation

184

Remarks: 1. In the subcommand ‘use c8 = 1.’ the blank spaces before and after ‘=’ are essential. Use the ‘Space-bar’ for getting them. 2. Starting with the release 15 of Minitab on Windows, the commands: copy c1−c3 c11−c13; use c8 = 1. may be replaced by the commands: copy c1−c3 c11−c13; include; where ”c8 = 1”. It is anticipated that in a future release of Minitab the subcommand ‘use’ will be discarded and replaced by the subcommand ‘include’ just mentioned. 2 We have already noticed that shrinking the domain of possible values of the variables improves the results of the Monte Carlo method. But, it is known ([37], pp.592, 597-599) that the feasible space of any linear program may be shrunk to become a subset of the unit cube of a multi-dimensional Euclidean space. Thus, each linear program with inequality constraints may be put in a normed form, which is a linear program with equality constraints whose feasible solutions are probability distributions, namely:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

min z subject to: ai1 x1 + . . . + ain xn x1 + . . . + xn xj

=

c1 x1 + . . . + cn xn

= 0, = 1 ≥ 0,

(i = 1, . . . , m) (j = 1, . . . , n)

for which the point (1/n, . . . , 1/n) is a feasible solution and the optimum zvalue equals 0. Surprisingly, any linear program may be put in this form and, as shown by the above constraints, the feasible solutions x = (x1 , . . . , xn ) are probability distributions with n components. In order to approximate the optimum solution of a normed linear program we may adapt the maximum uncertainty strategy used by the general Monte Carlo method in simulation. As the discrete uniform distribution is the most uncertain, or unbiased, probability distribution when no restrictions are imposed, we cover as uniformly as possible the feasible space generated by the constraints and select those solutions for which the objective function approaches the optimum value 0. In general, equality constraints are very rigid and, practically, we have to relax them by replacing them with close double inequalities. Using random numbers uniformly distributed in [0, 1], we generate probability distributions x = (x1 , . . . , xn ) that approximately satisfy the constraints and, subsequently, we select those for which the corresponding values of the objective function are close to zero. In the end, we obtain confidence intervals for the optimum values of the variables and the optimum

Simulation

185

objective value of the normed linear program. Problem 18: Use the Monte Carlo method for approximating the optimum solution of the following normed linear program: min z subject to: x2 − x3 x1 + x2 + x3 xj

=

x1 + 3x2 − 3x3

= 0 = 1 ≥ 0,

(j = 1, 2, 3)

Solution: From the second equality constraint we get: x1 = 1 − x2 − x3 . We relax the first equality constraint to be satisfied with an approximation of ±, namely: − ≤ x2 − x3 ≤ . Using any text editor, like:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Start −→ Programs −→ Accessories −→ Notepad, or ‘WordPad‘, also available in ‘Accessories’, or the popular text editors ‘Word’, or ‘ChiWriter’, for instance, we type the subprogram ’minlp.txt’: random k1 c2; uniform 0 1. random k1 c3; uniform 0 1. let c1=1−c2−c3 let c4 = c1+3*c2−3*c3 let c5 = c2−c3 code (k2:k3)1 (k6:k2)0 (k3:k7)0 c5 c6 copy c1−c4 c11−c14; use c6 = 1. let c15=abs(c14) let k4 = min(c15) copy c11−c14 c21−c24; use c15 = k4. let c31(k5) = c21(1) let c32(k5) = c22(1) let c33(k5) = c23(1) let c34(k5)=c24(1) let k5 = k5+1

Simulation

186

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

We save this subprogram as the text file ‘a:minlp.txt’ on a diskette in the drive ‘a:’, or as ‘d:minlp.txt’ on a CD in the drive ‘d:’, or, on the hard disk, as ‘c:\windows\minlp.txt’. We start a Minitab session and type the following commands: let k1 = 8000 let k2 = −0.005 let k3 = 0.005 let k5 = 1 let k6 = −10000 let k7 = 10000 noecho execute ”a:minlp.txt” 200 {or: execute ”d:minlp.txt” 200, or: execute ”c:\windows\minlp.txt” 200} code (−0.01:0.01)0 c34 c35 copy c31−c34 c41−c44; use c35 = 0. tinterval 95 c41−c44 The subprogram ‘minlp.txt’ refers to what happens in one arbitrary run of the simulation session. Inside it: c1, c2, and c3 contain the values of the variables x1 , x2 , and x3 , respectively, with x2 and x3 uniformly distributed in [0, 1]; c4 contains the corresponding values of the objective function; c5 contains the corresponding values of the first constraint; k1 is the number of solutions (x1 , x2 , x3 ) that satisfy the second constraint, randomly selected in one run of the simulation; k2 and k3 give the approximation levels − and +, for the relaxed first constraint; k5 is a counter; k6 and k7 are lower and upper bounds for the possible values of the first constraint; c41, c42, and c43 contain the values of x1 , x2 , x3 which satisfy the second constraint and the first constraint within the error of approximation 2 × 0.005 = 0.01 and give the optimum objective value z in c44 within the error of approximation 2 × 0.01 = 0.02. After 200 runs, the 95% confidence intervals for the three variables and the objective value of the normed linear program were: Variable C41 C42 C43 C44

N 200 200 200 200

Mean 0.00096 0.49943 0.49961 0.00040

StDev 95.0% CI 0.01010 (-0.00095,0.00287) 0.00636 (0.49823,0.50063) 0.00388 (0.49886,0.50039) 0.00551 (-0.00064,0.00144)

We notice that the components of the exact solution of this linear program, namely, x1 = 0.00, x2 = 0.50, x3 = 0.50, and z = 0.00, are contained in the corresponding confidence intervals and are well approximated by their mean values. 2 Remark: If the linear program has more than one optimum solution, then we do not use the command ‘tinterval’, because confidence intervals for the

Simulation

187

components of different alternative optimum solutions would be misleading, but we simply print at the end of the Minitab session the columns containing the final approximative values obtained for the optimum variables and optimum objective value of the corresponding linear program. In the program given above, this would mean to read the approximation of the optimum solutions from the computer’s answer to the command ‘print c41−c44’. 2 Remarks: 1. In the subcommand ‘use c8 = 1.’ the blank spaces before and after ‘=’ are essential. Use the ‘Space-bar’ for getting them. 2. Starting with the release 15 of Minitab on Windows, the commands: copy c11−c13 c21−c23; use c15 = k4. may be replaced by the commands: copy c11−c13 c21−c23; include; where ”c15 = k4”. It is anticipated that in a future release of Minitab the subcommand ‘use’ will be discarded and replaced by the subcommand ‘include’ just mentioned. 2 (b) Nonlinear optimization. Monte Carlo method may be also used for approximating the optimum solutions in nonlinear optimization. Here is an example which shows how this could be done. Problem 19: Approximate the maximum value of the function: u(x, y) = x − ex sin y in the unit circle centered at the origin. Solution: u(x, y) is a harmonic function. Indeed, taking the first two partial derivatives with respect to x and the first two partial derivatives with respect to y, we get: ux = 1 − ex sin y, uxx = −ex sin y, uy = −ex cos y,

uyy = ex sin y,

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

which shows that Laplace’s equation is satisfied, namely: uxx + uyy = −ex sin y + ex sin y = 0. A harmonic function reaches the maximum on the frontier of the domain: D = {(x, y); x2 + y 2 ≤ 1}, which is the circle x2 + y 2 = 1. Switching to the polar coordinates, we have: x = cos θ,

y = sin θ,

Simulation

188

and the function u(x, y) becomes a function of only one variable θ, on the unit circle centered at the origin, namely:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

u(θ) = u(cos θ, sin θ) = cos θ − ecos θ sin(sin θ),

0 ≤ θ ≤ 2π.

We apply the Monte Carlo method, covering uniformly the unit circle centered at the origin, calculating the values of the function u(θ) at all these points of the circle, and selecting the maximum of these values of the objective function. The Minitab program which implements this is given below. First, using a text editor, as the ‘Notepad’ or ‘WordPad’ from ‘Accessories’, or one of the popular text editors ‘Word’ or ‘ChiWriter’, for instance, we create a subprogram called ‘harmonic.txt’, namely: let k5 = 2*3.14159 random k1 c1; uniform 0 k5. let c2 = cos(c1)− sin(sin(c1))*(2.71828**cos(c1)) let k2 = max(c2) let k3 = min(c2) code (k2:k2)1 (k3:k2)0 c2 c3 copy c1 c2 c4 c5; use c3 = 1. let c6(k4)=c4(1) let c7(k4)=c5(1) let k4=k4+1 We have used the approximations π ≈ 3.14159, 2π ≈ 6.28318, and e ≈ 2.71828 for the transcendental numbers π, 2π, and e. We save the subprogram as ‘a:harmonic.txt’ on a diskette in the drive ‘a:’, or as ‘d:harmonic.txt’ on a CD in the drive ‘d:’, or on the hard disk as ‘c:\windows\harmonic.txt’. Then, we start a Minitab session and we type the following commands, taking 2000 random numbers uniformly distributed on the unit circle and executing the subprogram ‘harmonic.txt’ 50 times: let k1=2000 let k4=1 execute ”a:harmonic.txt” 50 {or: execute ”d:harmonic.txt” 50, or: execute ”c:\windows\harmonic.txt” 50} tinterval 95 c6 c7 where: c7 contains max u(θ), and c6 contains the corresponding value of θ. The results obtained were: Variable N Mean StDev 95.0% CI C6 50 5.61652 0.00210 (5.61592,5.61711) C7 50 2.05796 0.00001 (2.05795,2.05796) If we want to pick up the best solution obtained in 50 runs, we use the commands:

Simulation let k6 = max(c7) copy c6 c8; use c7 = k6. print c8 k6 We got: θ = 5.61649 in c8 and max u(θ) = 2.05797 in k6.

189

2

Remarks: 1. In the subcommand ‘use c3 = 1.’ the blank spaces before and after ‘=’ are essential. Use the ‘Space-bar’ for getting them. 2. Starting with the release 15 of Minitab on Windows, the commands: copy c1 c2 c4 c5; use c3 = 1. may be replaced by the commands: copy c1 c2 c4 c5; include; where ”c3 = 1”. It is anticipated that in a future release of Minitab the subcommand ‘use’ will be discarded and replaced by the subcommand ‘include’ just mentioned. 2

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Remark: Monte Carlo method relies on repeated random sampling and is used when simulating physical and mathematical systems. Because of their reliance on repeated computation and a lot of random numbers, the Monte Carlo method is most suited to calculation by a computer. It tends to be used when it is infeasible or impossible to compute an exact result with deterministic methods or algorithms. In general, the Monte Carlo method is used in mathematics to solve various problems by generating suitable random numbers and observing that fraction of the numbers obeying some properties. The method is useful for obtaining numerical approximate solutions to problems which are too complicated to be solved analytically. The most common mathematical applications of the Monte Carlo method are in integration, linear and mostly nonlinear optimization, and solving equations. More often than not, the results of the Monte Carlo method are expressed not as rigid, unique numbers, but as confidence intervals. 2

Reliability

190

Chapter 4. RELIABILITY

ABSTRACT

A system is defined by its elementary components and the connection between them. In the first part of the chapter, time is frozen and reliability is a number representing the probability that the system functions properly during the given, fixed time interval. Four methods are given for calculating the exact reliability of the system, using the minimal paths and the minimal cuts, the essential subsystems of any system. As all these methods are quite laborious, a lower bound and an upper bound for the reliability of the system, both easy to calculate, are also given. In the second part of the chapter, time t is a variable and the reliability of the system, representing the probability that the system functions properly, or is still alive, at time t, is a function of time. Different probability distributions for the lifetime of the system are analyzed, when the failure rate is constant or variable in time.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

1. Structure function A ‘system’ is a set of ’elementary components’ and a ‘connection’ between them. To each component we assign its ‘structure function’ X which is a binary 0-1 random variable taking on the value 1 if the component functions (or performs) properly and the value 0 if the component fails to function properly. A component is generally identified with its structure function. The ‘reliability’ of such a component is defined as being the probability of functioning properly and is denoted by p = P (X = 1). Let q = P (X = 0) = 1 − p denote the probability that the respective component fails to function properly. For a system consisting of n components X1 , . . . , Xn , the structure function Φ(X1 , . . . , Xn ) is a binary 0-1 random variable, equal to 1 if the system functions (or performs) properly and to 0 if the system does not perform properly (or fails to function properly). The reliability of the system is the probability that the system functions (or performs) properly, denoted by R = P (Φ(X1 , . . . , Xn ) = 1).

Reliability

191

A system is called a ‘series system’ when it functions (or performs) properly if and only if each of its components functions (or performs) properly. Such a system fails to function (or perform) properly if at least one of its components fails to function properly. The structure function of a series system is: Φ(X1 , . . . , Xn ) = min {X1 , . . . , Xn } = X1 . . . Xn . and, as the elementary components are independent, its reliability is equal to: R = P (Φ(X1 , . . . , Xn ) = 1) = P (X1 . . . Xn = 1) = = P (X1 = 1, . . . , Xn = 1) = P (X1 = 1) . . . P (Xn = 1) = p1 . . . pn . A system is called a ‘parallel system’ when it fails to function (or perform) properly if and only if each of its components fails to function (or perform) properly. Such a system functions (or performs) properly if at least one of its components functions (or performs) properly. The structure function of a parallel system is: Φ(X1 , . . . , Xn ) = max {X1 , . . . , Xn } = = 1 − (1 − X1 ) . . . (1 − Xn ), and its reliability is equal to: R = P (Φ(X1 , . . . , Xn ) = 1) = 1 − P (Φ(X1 , . . . , Xn ) = 0) = = 1 − P (1 − (1 − X1 ) . . . (1 − Xn ) = 0) = 1 − P ((1 − X1 ) . . . (1 − Xn ) = 1) = = 1 − P (1 − X1 = 1, . . . , 1 − Xn = 1) = 1 − P (X1 = 0, . . . , Xn = 0) = = 1 − P (X1 = 0) . . . P (Xn = 0) = 1 − q1 . . . qn = = 1 − (1 − p1 ) . . . (1 − pn ).

-

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

- X1

X

- Elementary component

- X2

- Series connection

- X1 - Parallel connection

- X2

Figure 13. Elementary components.

Reliability

192

The structure function of more complex systems may be obtained by decomposing the given system into series and parallel subsystems for which we know how to write the corresponding structure functions. Thus, for the seriesparallel system from the figure given below, the structure function is: Ψ(X1 , X2 , X3 ) = X1 [1 − (1 − X2 )(1 − X3 )], whereas the structure function of the parallel-series system from the same figure, the structure function is: Ψ(X1 , X2 , X3 ) = 1 − (1 − X1 X2 )(1 − X3 ). - X2 - X1

-

- X3

Figure 14. Parallel-series system - X1

- X2

-

- X3

Figure 15. Series-parallel system Remark: As shown above, the reliability of a series system or a parallel system may be formally obtained replacing Xi by pi in the expression of its structure function. This procedure, however, does not work in general, as it will be shown soon. 2 A system is called a ‘k-out-of-n system’ if it functions (or performs) properly if at least k of its components function (or perform) properly. The structure function of such a system is:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Φ(X1 , . . . , Xn ) =

 Pn   1 if i=1 Xi ≥ k,  

0 if

Pn

i=1

Xi < k.

Obviously, a parallel system is a 1-out-of-n system whereas a series system is an n-out-of-n system. In the particular case when all components have the same reliability, i.e. pi = p for every i, (i = 1, . . . , n), then: R=

X

s≥k

n s

!

ps (1 − p)n−s .

Reliability

193

Remark: The elementary components of a system are independent, and therefore the structure functions of the elementary components are independent binary 0-1 random variables. The subsystems of a system, however, may be independent, if they have no elementary components in common, or dependent, if they have at least one common elementary component. 2 Remark: The binary 0-1 random variables have unique properties. Thus, if X is a binary 0-1 random variable, then X 2 = X and any other positive integer power of X has the same property, namely X k = X. Performing algebraic operations on binary 0-1 random variables becomes much simpler due to the fact that there are no powers of such random variables. Also, the complementary random variable Y = 1 − X is also binary 0-1. We have: P (Y = 1) = P (X = 0) = 1 − p = q, P (Y = 0) = P (X = 1) = p = 1 − q. The elementary component whose structure function is Y functions properly with the same probability that the elementary component whose structure function is X fails to function properly. Similarly, the elementary component whose structure function is X functions properly with the same probability that the elementary component whose structure function is Y fails to function properly. If X1 and X2 are binary 0-1 random variables, taking the maximum or minimum of the pair {X1 , X2 } is reduced to performing simple algebraic operations on X1 and X2 , namely: min {X1 , X2 } = X1 X2 , max {X1 , X2 } = 1 − (1 − X1 )(1 − X2 ). This may be generalized to several binary 0-1 random variables: min {X1 , . . . , Xn } = X1 . . . Xn , max {X1 , . . . , Xn } = 1 − (1 − X1 ) . . . (1 − Xn ). 2

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

2. Exact reliability In order to calculate the exact reliability of a system, it is preferable to represent a system by a network. A graph consists of nodes and branches or arcs connecting some of the nodes. A network is a graph and a flow from a certain node called source to another node called sink. The components of the system are the branches of the corresponding network, whereas the nodes are used for connections among components. A component functions properly if the flow can go through the corresponding branch and fails to function

Reliability

194

properly if the flow cannot go through the corresponding branch. The system functions properly if the flow can go from the source to the sink, following any possible path allowed by the structure of the corresponding network, and fails to function properly if the flow emerging from the source cannot reach the sink. We assume here that ‘functioning properly’ and ‘failing to function properly’ refer to an arbitrary but fixed time interval. This is why reliability of a component or of a system is a number, representing the probability of functioning properly during this fixed time interval. Later, in section 4 of this chapter, time will become a variable and reliability, referring to the probability of functioning properly during a variable time interval [0, t], will be a function of time R(t). A system may be very complex. In any system, however, we can distinguish two kinds of subsystems, minimal paths and minimal cuts, that prove to be essential for the proper functioning or failure of the respective system. A ‘minimal path’ is a minimal subset of components such that if each of them functions properly the entire system functions properly. A ‘minimal cut’ is a minimal subset of components such that if each of them fails the entire system fails. A system may be viewed as being the parallel connection of its minimal paths. Inside a minimal path its components are joined in series. In a dual way, a system may also be viewed as being the series connection of its minimal cuts. Inside a minimal cut its components are joined in parallel. Remark: As a series connection is associated with minimization and a parallel connection is associated with maximization, the previous statements reveal the dual ‘maximin’ and ‘minimax’ structure of any system in terms of minimal paths and minimal cuts, respectively. 2 There are four methods for calculating the exact reliability of a system: Method 1, using minimal paths, and Method 2, using minimal cuts, are based on the probability of an alternative as a mathematical tool. For two arbitrary events A and B, we have: P (A or B) = P (A) + P (B) − P (A and B).

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

For three arbitrary events A, B, C, we have: P (A or B or C) = P (A) + P (B) + P (C)− −P (A and B) − P (A and C) − P (B and C) + P (A and B and C), and, similarly, for the probability of an alternative involving more than three events. In order to simplify the writing, we prefer to replace the logical conjunction ‘and’, which corresponds to the intersection of sets, by a comma, such

Reliability

195

that instead of P (A and B) we prefer to write P (A, B). With this convention, the last equality may be written as: P (A or B or C) = P (A) + P (B) + P (C)− −P (A, B) − P (A, C) − P (B, C) + P (A, B, C), Method 3, using minimal paths, and Method 4, using minimal cuts, are based on the fact that the expected value of the structure function of the system is just its reliability. This is also a remarkable property that only the binary 0-1 random variables have. Indeed, for the structure function X of an elementary component its mean (expected) value is: E(X) = 1 P (X = 1) + 0 P (X = 0) = P (X = 1) = p, and, similarly, for the structure function Φ(X1 , . . . , Xn ) of a system whose elementary components are X1 , . . . , Xn , its mean (expected) value is: E[Φ(X1 , . . . , Xn )] = 1 P (Φ(X1 , . . . , Xn ) = 1)+ +0 P (Φ(X1 , . . . , Xn ) = 0) = P (Φ(X1 , . . . , Xn ) = 1) = R Practically, if the number of minimal paths is smaller than the number of minimal cuts then Method 1 or Method 3 should be used. If the number of minimal cuts is smaller than the number of minimal paths then Method 2 or Method 4 should be used. For the system from the following problem, however, all four methods for calculating its reliability will be used. Problem 1: A system consists of five components whose structure functions are X1 , . . . , X5 , respectively. The network showing how these components are joined together is shown below. Calculate its reliability as a function of the reliabilities p1 , . . . , p5 of the five components of the system.   X4  ?   

- 2 X1 

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Source

1

X3

4

Sink

   ?6  - 3 X2  X5  

Figure 16. An example of network. Solution: The system whose network is shown above has three minimal paths and four minimal cuts:

Reliability

196 minimal paths minimal cuts {X1 , X4 } {X1 , X2 } {X2 , X5 } {X4 , X5 } {X1 , X3 , X5 } {X1 , X5 } {X2 , X3 , X4 }

Method 1 (using the minimal paths and the probability of an alternative): Here we take into account that the system may be viewed as being a parallel connection of its minimal paths and inside a minimal path its components are joined in series. R = P (System functions) = = P ((path 1 functions) or (path 2 functions) or (path 3 functions)) = = P ((X1 X4 = 1) or (X2 X5 = 1) or (X1 X3 X5 = 1)) = = P (X1 X4 = 1) + P (X2 X5 = 1) + P (X1 X3 X5 = 1)− −P (X1 X4 = 1, X2 X5 = 1) − P (X1 X4 = 1, X1 X3 X5 = 1)− −P (X2 X5 = 1, X1 X3 X5 = 1)+ +P (X1 X4 = 1, X2 X5 = 1, X1 X3 X5 = 1) = = p1 p4 + p2 p5 + p1 p3 p5 − p1 p2 p4 p5 − p1 p3 p4 p5 − −p1 p2 p3 p5 + p1 p2 p3 p4 p5 , because, using the independence of the random variables X1 , . . . , X5 , we have: P (X1 X4 = 1) = P (X1 = 1, X4 = 1) = P (X1 = 1) P (X4 = 1) = p1 p4 , and, if there are common factors: P (X1 X4 = 1, X1 X3 X5 = 1) = = P (X1 = 1, X4 = 1, X1 = 1, X3 = 1, X5 = 1) = = P (X1 = 1, X3 = 1, X4 = 1, X5 = 1) =

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

= P (X1 = 1)P (X3 = 1)P (X4 = 1)P (X5 = 1) = p1 p3 p4 p5 , etc., for the other terms. We have used here the fact that a product of binary 0-1 random variables is equal to 1 if and only if each factor is equal to 1, i.e.: X1 X4 = 1 if and only if X1 = 1 and X4 = 1, and that the intersection of events is commutative: (X4 = 1, X3 = 1) = (X3 = 1, X4 = 1),

Reliability

197

and the intersection of an event with itself leaves the respective event unchanged: (X1 = 1, X1 = 1) = (X1 = 1). In particular, if all five components are identical, which means that pi = p, (i = 1, . . . , 5), then the corresponding reliability of the system is: R = 2p2 + p3 − 3p4 + p5 . Method 2 (using the minimal cuts and the probability of an alternative): Here we take into account that the system may be viewed as being a series connection of its minimal cuts and inside a minimal cut its components are joined in parallel. In what follows, we use Yi = 1 − Xi , (i = 1, . . . , 5). R = P (System functions) = 1 − P (System fails) = = 1 − P ((cut 1 fails) or (cut 2 fails) or (cut 3 fails) or (cut 4 fails)) = = 1 − P ((1 − (1 − X1 )(1 − X2 ) = 0) or (1 − (1 − X4 )(1 − X5 ) = 0) or or (1 − (1 − X1 (1 − X5 ) = 0) or (1 − (1 − X2 )(1 − X3 )(1 − X4 ) = 0)) = = 1 − P ((Y1 Y2 = 1) or (Y4 Y5 = 1) or (Y1 Y5 = 1) or (Y2 Y3 Y4 = 1)) = = 1 − P (Y1 Y2 = 1) − P (Y4 Y5 = 1) − P (Y1 Y5 = 1)− −P (Y2 Y3 Y4 = 1) + P (Y1 Y2 = 1, Y4 Y5 = 1) + P (Y1 Y2 = 1, Y1 Y5 = 1)+ +P (Y1 Y2 = 1, Y2 Y3 Y4 = 1) + P (Y4 Y5 = 1, Y1 Y5 = 1)+ +P (Y4 Y5 = 1, Y2 Y3 Y4 = 1) + P (Y1 Y5 = 1, Y2 Y3 Y4 = 1)− −P (Y1 Y2 = 1, Y4 Y5 = 1, Y1 Y5 = 1)− −P (Y1 Y2 = 1, Y4 Y5 = 1, Y2 Y3 Y4 = 1)− −P (Y1 Y2 = 1, Y1 Y5 = 1, Y2 Y3 Y4 = 1)− −P (Y4 Y5 = 1, Y1 Y5 = 1, Y2 Y3 Y4 = 1)+ +P (Y1 Y2 = 1, Y4 Y5 = 1, Y1 Y5 = 1, Y2 Y3 Y4 = 1) = Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

= 1 − q1 q2 − q4 q 5 − q1 q5 − q 2 q3 q 4 + q1 q2 q4 q5 + q1 q2 q5 + +q1 q2 q3 q4 + q1 q4 q5 + q2 q3 q4 q5 + q1 q2 q3 q4 q5 − −q1 q2 q4 q5 − q1 q2 q3 q4 q5 − q1 q2 q3 q4 q5 − q1 q2 q3 q4 q5 + +q1 q2 q3 q4 q5 = 1 − q1 q2 − q4 q5 − q1 q5 − q2 q3 q4 + q1 q2 q5 + +q1 q2 q3 q4 + q1 q4 q5 + q2 q3 q4 q5 − q1 q2 q3 q4 q5

Reliability

198

because, the independence of X1 , . . . , X5 implies that Y1 , . . . , Y5 are independent too and: P (Yi = 1) = P (Xi = 0) = qi ,

(i = 1, . . . , 5),

P (Y1 Y2 = 1) = P (Y1 = 1, Y2 = 1) = P (Y1 = 1) P (Y2 = 1) = q1 q2 , P (Y1 Y2 = 1, Y1 Y5 = 1) = P (Y1 = 1, Y2 = 1, Y1 = 1, Y5 = 1) = = P (Y1 = 1, Y2 = 1, Y5 = 1) = P (Y1 = 1) P (Y2 = 1) P (Y5 = 1) = q1 q2 q5 , etc., for the other terms above. In particular, if all five components are identical, which means that qi = q, (i = 1, . . . , 5), then the corresponding reliability of the system is: R = 1 − 3q 2 + q 3 + 2q 4 − q 5 . Method 3 (using the minimal paths and the expected value of the structure function): Here we take again into account that the system may be viewed as being a parallel connection of its minimal paths and inside a minimal path its components are joined in series. Consequently, the structure function of the system is: Φ(X1 , X2 , X3 , X4 , X5 ) = 1 − (1 − X1 X4 )(1 − X2 X5 )(1 − X1 X3 X5 ) = = 1 − (1 − X1 X4 − X2 X5 + X1 X2 X4 X5 )(1 − X1 X3 X5 ) = = X1 X4 + X2 X5 − X1 X2 X4 X5 + X1 X3 X5 − X1 X3 X4 X5 − −X1 X2 X3 X5 + X1 X2 X3 X4 X5 , where we have taken into account that in the algebra of binary 0-1 random variables Xik = Xi for every positive integer k. As the expected (mean) value of the structure function is qual to the reliability of the system, we get: R = E[Φ(X1 , X2 , X3 , X4 , X5 )] = p1 p4 + p2 p5 + p1 p3 p5 −

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

−p1 p2 p4 p5 − p1 p3 p4 p5 − p1 p2 p3 p5 + p1 p2 p3 p4 p5 , because the expectation E is a linear function and the expected value of the product of independent random variables is equal to the product of the expected values of the factors of the product, as in: E(X1 X4 ) = E(X1 ) E(X4 ) = p1 p4 , etc., for the other terms of the structure function.

Reliability

199

In particular, if all five components are identical, which means that pi = p, (i = 1, . . . , 5), then the corresponding reliability of the system is: R = 2p2 + p3 − 3p4 + p5 . Method 4 (using the minimal cuts and the expected value of the structure function): Here we take again into account that the system may be viewed as being a series connection of its minimal cuts and inside a minimal cut its components are joined in parallel. In what follows, we use again Yi = 1 − Xi , (i = 1, . . . , 5). The structure function of the system is: Φ(X1 , X2 , X3 , X4 , X5 ) = [1 − (1 − X1 )(1 − X2 )] [1 − (1 − X4 )(1 − X5 )]× ×[1 − (1 − X1 )(1 − X5 )] [1 − (1 − X2 )(1 − X3 )(1 − X4 )] = = (1 − Y1 Y2 )(1 − Y4 Y5 )(1 − Y1 Y5 )(1 − Y2 Y3 Y4 ) = = (1 − Y1 Y2 − Y4 Y5 + Y1 Y2 Y4 Y5 )(1 − Y1 Y5 − Y2 Y3 Y4 + Y1 Y2 Y3 Y4 Y5 ) = = 1 − Y1 Y2 − Y4 Y5 − Y1 Y5 + Y1 Y2 Y5 + Y1 Y4 Y5 − Y2 Y3 Y4 + +Y1 Y2 Y3 Y4 + Y2 Y3 Y4 Y5 − Y1 Y2 Y3 Y4 Y5 , where we have taken into account that in the algebra of binary 0-1 random variables Yik = Yi for every positive integer k. As the expected (mean) value of the structure function is qual to the reliability of the system, we get: R = E[Φ(X1 , X2 , X3 , X4 , X5 )] = = 1 − q1 q2 − q 4 q5 − q1 q5 + q1 q2 q5 + q 1 q4 q 5 − −q2 q3 q4 + q1 q2 q3 q4 + q2 q3 q4 q5 − q1 q2 q3 q4 q5 , because the expectation E is a linear function and the expected value of the product of independent random variables is equal to the product of the expected values of the factors of the product, as in:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

E(Y1 Y2 ) = E(Y1 ) E(Y2 ) = q1 q2 , etc., for the other terms of the structure function. We have also used the fact that the constant 1 is a degenerate binary 0-1 random variable that takes on the value 1 with probability 1 and the value 0 with probability 0; consequently, its mean value is: E(1) = (1)(1) + (0)(0) = 1. In particular, if all five components are identical, which means that qi = q, (i = 1, . . . , 5), then the corresponding reliability of the system is: R = 1 − 3q 2 + q 3 + 2q 4 − q 5 .

Reliability

200

3. Bounds for reliability As seen in the previous section, the exact computation of system’s reliability is tedious. There is, however, an easy way of getting upper and lower bounds for it. Such bounds are obtained by using an inequality involving products of independent binary 0-1 random variables. Proposition 1: If Z1 , . . . , Zn are arbitrary products of independent binary 0-1 random variables, then the following inequality holds: P (Z1 = 0, . . . , Zn = 0) ≥ P (Z1 = 0) . . . P (Zn = 0). Proof: Let us notice that if Z1 , . . . , Zn have no common factors, then they are independent and the above statement is true with equality. Therefore, suppose that at least two of the products Z1 , . . . , Zn have common factors. We prove the inequality by mathematical induction: (a) Let us prove the above inequality for n = 2. For simplifying the writing, let us take Z1 = X1 X2 , and Z2 = X2 X3 . Doing this, we do not sacrifice generality because X1 may be thought of as being the product of all the initial factors of Z1 which are not initial factors of Z2 , X3 is the product of all the initial factors of Z2 which are not initial factors of Z1 , and X2 is the product of all common initial factors of Z1 and Z2 . Then, we have: P (Z1 = 0, Z2 = 0) = 1 − P ((Z1 = 1) or (Z2 = 1)) = 1 − P (Z1 = 1)− −P (Z2 = 1) + P (Z1 = 1, Z2 = 1) = 1 − P (X1 X2 = 1)− −P (X2 X3 = 1) + P (X1 X2 = 1, X2 X3 = 1) = 1 − P (X1 = 1, X2 = 1)− −P (X2 = 1, X3 = 1) + P (X1 = 1, X2 = 1, X3 = 1) = = 1 − p1 p2 − p2 p3 + p1 p2 p3 , where pi = P (Xi = 1). Also: P (Z1 = 0) P (Z2 = 0) = [1 − P (Z1 = 1)][1 − P (Z2 = 1)] = Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

= [1 − P (X1 X2 = 1)][1 − P (X2 X3 = 1)] = = [1 − P (X1 = 1) P (X2 = 1)][1 − P (X2 = 1) P (X3 = 1)] = = (1 − p1 p2 )(1 − p2 p3 ) = 1 − p1 p2 − p2 p3 + p1 p22 p3 . But, as p1 p2 p3 ≥ p1 p22 p3 , we have: P (Z1 = 0, Z2 = 0) ≥ P (Z1 = 0) P (Z2 = 0).

Reliability

201

(b) Assume that the inequality from Proposition 1 is true for n. We want to prove that it is true for n + 1 as well. Let us notice that Zi = 0 if at least one of its factors is equal to zero. If we define the events: A = {a factor of Zn+1 common with at least one of Z1 , . . . , Zn is zero}, B = {no factor of Zn+1 common with at least one of Z1 ,. . . ,Zn is zero}, C = {Z1 = 0, . . . , Zn = 0, A},

D = {Z1 = 0, . . . , Zn = 0, B}.

then we have the conditional probabilities: P (Zn+1 = 0|C) = 1, because the event C implies the event {Zn+1 = 0}, and: P (Zn+1 = 0|D) = P (Zn+1 = 0), because the events D and {Zn+1 = 0} are independent. Therefore: P (Z1 = 0, . . . , Zn = 0, Zn+1 = 0) = = P (C, Zn+1 = 0) + P (D, Zn+1 = 0) = = P (Zn+1 = 0|C) P (C) + P (Zn+1 = 0|D) P (D) = = P (C) + P (Zn+1 = 0) P (D) ≥ P (Zn+1 = 0)[P (C) + P (D)] = = P (Zn+1 = 0) P (Z1 = 0, . . . , Zn = 0), and based on the assumption that the inequality of Proposition 1 is true for n, we get: P (Z1 = 0, . . . , Zn = 0, Zn+1 = 0) ≥ ≥ P (Z1 = 0) . . . P (Zn = 0) P (Zn+1 = 0). 2

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Going back to the system whose corresponding network was given in the previous section, and applying the inequality from Proposition 1, we get an upper bound, using minimal paths, and a lower bound, using minimal cuts. (a) Upper bound. Here we take again into account that the system may be viewed as being a parallel connection of its minimal paths and inside a minimal path its components are joined in series. Using this and applying Proposition 1, we get: R = P (System functions) = 1 − P (System fails) = = 1 − P (path 1 fails, path 2fails, path3 fails) = = 1 − P (X1 X4 = 0, X2 X5 = 0, X1 X3 X5 = 0) ≤

Reliability

202 ≤ 1 − P (X1 X4 = 0) P (X2 X5 = 0) P (X1 X3 X5 = 0) =

= 1 − [1 − P (X1 X4 = 1)] [1 − P (X2 X5 = 1)] [1 − P (X1 X3 X5 = 1)] = = 1 − [1 − P (X1 = 1) P (X4 = 1)] [1 − P (X2 = 1) P (X5 = 1)]× ×[1 − P (X1 = 1) P (X3 = 1) P (X5 = 1)] = = 1 − (1 − p1 p4 )(1 − p2 p5 )(1 − p1 p3 p5 ). Remark: As the structure function of the system in terms of minimal paths is: Φ(X1 , X2 , X3 , X4 , X5 ) = = 1 − (1 − X1 X4 )(1 − X2 X5 )(1 − X1 X3 X5 ), we notice that, formally (!), we can obtain the upper bound for the reliability of the system by simply replacing Xi by pi in the expression of the structure function of the system written in terms of minimal paths. But we keep in mind that the structure function is a binary 0-1 random variable whereas the upper bound is a number. 2 (b) Lower bound. Here we take again into account that the system may be viewed as being a series connection of its minimal cuts and inside a minimal cut its components are joined in parallel. Using this, applying Proposition 1, and introducing Yi = 1 − Xi , we get: R = P (System functions) = = P (cut 1 functions, cut 2 functions, cut 3 functions, cut 4 functions) = = P (1 − (1 − X1 )(1 − X2 ) = 1, 1 − (1 − X4 )(1 − X5 ) = 1, 1 − (1 − X1 )(1 − X5 ) = 1, 1 − (1 − X2 )(1 − X3 )(1 − X4 ) = 1) = = P (Y1 Y2 = 0, Y4 Y5 = 0, Y1 Y5 = 0, Y2 Y3 Y4 = 0) ≥ ≥ P (Y1 Y2 = 0) P (Y4 Y5 = 0) P (Y1 Y5 = 0) P (Y2 Y3 Y4 = 0) = = [1 − P (Y1 Y2 = 1)] [1 − P (Y4 Y5 = 1)]× Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

×[1 − P (Y1 Y5 = 1)] [1 − P (Y2 Y3 Y4 = 1)] = = [1 − P (Y1 = 1) P (Y2 = 1)] [1 − P (Y4 = 1) P (Y5 = 1)]× ×[1 − P (Y1 = 1) P (Y5 = 1)] [1 − P (Y2 = 1) P (Y3 = 1) P (Y4 = 1)] = = (1 − q1 q2 )(1 − q4 q5 )(1 − q1 q5 )(1 − q2 q3 q4 ) = = [1 − (1 − p1 )(1 − p2 )] [1 − (1 − p4 )(1 − p5 )]× ×[1 − (1 − p1 )(1 − p5 )] [1 − (1 − p2 )(1 − p3 )(1 − p4 )].

Reliability

203

Remark: As the structure function of the system in terms of minimal cuts is: Φ(X1 , X2 , X3 , X4 , X5 ) = = [1 − (1 − X1 )(1 − X2 )] [1 − (1 − X4 )(1 − X5 )]× ×[1 − (1 − X1 )(1 − X5 )] [1 − (1 − X2 )(1 − X3 )(1 − X4 )], we notice that, formally (!), we can obtain the lower bound for the reliability of the system by simply replacing Xi by pi in the expression of the structure function of the system written in terms of minimal cuts. But we keep in mind that the structure function is a binary 0-1 random variable whereas the lower bound is a number. 2 4. Lifetime In the previous sections, time has been ignored. Reliability of a system, however, may depend on time. Let T > 0 be the lifetime of a certain system. It is a continuous random variable taking on nonnegative real values. Let f (t) be its probability density. Reliability of the system at time t is defined as being the probability that the system functions properly (or, using a more liberal way of talking, is still alive) at time t. Obviously: R(t) = P (T > t) =

Z

+∞

f (x) dx.

t

The cumulative probability distribution function of lifetime T is: F (t) = P (T ≤ t) =

Z

t

f (x) dx,

0

and may be interpreted as being the probability that the system fails (or dies) before or at the time instant t. Obviously: R(t) = P (T > t) = 1 − P (T ≤ t) = 1 − F (t). The function: Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Z(t) =

f (t) R(t)

is called the (instantaneous) failure rate of the system. Its name is motivated by the following interpretation: Justification: The probability that the system will fail before or at time t + ∆t given that the system functions properly at time t is: P (T ≤ t + ∆t | T > t) =

P (t < T ≤ t + ∆t) = P (T > t)

Reliability

204 = =

1 R(t)

Z

t+∆t

f (x) dx =

t

f (t) 1 f (ξ) ∆t ≈ ∆t = Z(t) ∆t, R(t) R(t)

where ξ ∈ [t, t + ∆t]. 2 The instantaneous failure rate plays an important role in dealing with the lifetime of a system. It shows the speed of failing, and it is practically available by simply counting how many similar systems have died (failed) per unit time. What is remarkable about it is that the failure rate Z(t) completely characterizes both the probability density of the life time and the reliability of the system. Indeed, we have: Proposition 2: If F (0) = 0, which is equivalent to saying that T > 0, then the probability density of the lifetime of the system is: f (t) = Z(t) e−

Rt 0

Z(s) ds

,

and the reliability of the system is: R(t) = e−

Rt 0

Z(s) ds

.

Proof: The general formula from Calculus about how to take the derivative with respective to t of an integral where both the function which is integrated and the two limits of integration depend on t is: Z b(t) ∂g(x, t) d Z b(t) g(x, t) dx = dx+ dt a(t) ∂t a(t)

da(t) db(t) − g(a(t), t) . dt dt As the cumulative probability distribution function of the lifetime is: +g(b(t), t)

F (t) =

Z

t

f (x) dx,

0

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

the above formula, with: g(x, t) = f (x),

a(t) = 0,

b(t) = t,

gives: F 0 (t) = f (t)

dt = f (t). dt

As: R(t) = 1 − F (t),

Reliability

205

we get: R0 (t) = −F 0 (t) = −f (t). Therefore, if ‘log’ is the natural logarithm, we have: R0 (t) f (t) d log R(t) = =− = −Z(t). dt R(t) R(t) Integrating on [0, t], and taking into account that the assumption F (0) = 0 is equivalent to R(0) = 1, we get: log R(t) = −

Z

t

Rt

Z(s) ds

Z(s) ds,

0

which is equivalent to: −

R(t) = e

0

,

and if we take into account that: Z(t) =

f (t) , R(t)

we get: f (t) = Z(t) R(t) = Z(t) e−

Rt 0

Z(s) ds

,

2

Remark: Once the probability density f (x) of the lifetime of a system has been obtained, or the respective cumulative distribution function F (t) is available, our main objective is to calculate the probability that the system is still alive, or functions properly, during any time interval [t1 , t2 ], i.e.: P (t1 < T ≤ t2 ) =

Z

t2

t1

f (x) dx = F (t2 ) − F (t1 ). 2

1. Constant failure rate. The lifetime of a system with constant failure rate is exponentially distributed, as shown by the next proposition: Proposition 3: The failure rate of a system is constant, namely, Z(t) = α, (α > 0), if and only if:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

f (t) = α e−αt ,

(t ≥ 0).

Proof: (a) Assume that the probability density of the lifetime of a system is exponential with parameter α: f (t) = α e−αt ,

(t ≥ 0).

Then, the reliability of the system is: R(t) = 1 − F (t) = 1 −

Z

0

t

α e−αx dx =

Reliability

206 t

d (−e−αx ) dx = 1 − −e−αx 0 dx Therefore, the instantaneous failure rate is: =1−

Z

h

Z(t) = f (t)/R(t) =

it

0

= e−αt .

α e−αt = α. e−αt

(b) Conversely, if Z(t) = α, then from the first formula of Proposition 2, we have: Rt Rt f (t) = Z(t) e− 0 Z(s) ds = α e−α 0 ds = α e−αt , for t ≥ 0, which is just the probability density of the exponential distribution with parameter α. 2 Remark: It is remarkable that the exponential probability distribution, well-known in statistics, proves to characterize the lifetime of a system whose rate of dying (failing) is constant, its parameter α being just the corresponding constant failure rate. There are systems, however, whose instantaneous failure rate is not constant, essentially depending on time. 2 Problem 1: A fine electronic device manufactured in a factory has a constant failure rate and the expected lifetime is 1000 weeks. Find the time at which 75% of similar devices will still function properly. Solution: According to Proposition 3, the lifetime T of such a system is exponentially distributed with parameter: α=

1 1 = = 0.001. E(T ) 1000

We need the time t at which: 0.75 = R(t) = e−α t .

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Taking the natural logarithm, we get: t=−

1 log 0.75 = −(1000)(−0.2877) = 287.7weeks. 2 α

Problem 2: A new brand of cars is tested at a certain auto plant. For the engineers working on it, the lifetime of such a car means the time (in years), elapsed until the first major repair is needed. Unfortunately, the failure rate proves to be constant and equal to 0.1. 1. Calculate the reliability of this new brand of cars at times 1, 2.5, and 3 units of time.

Reliability

207

2. Find the times at which 25%, 40%, and 80% of such cars are still without a major repair. 3. Use Minitab for answering these questions. Depending on the results obtained, what conclusion could the engineers draw? Solution: 1. The lifetime is exponentially distributed with the parameter α = 0.1. Using the formula: R(t) = e−α t = e−0.1 t , we obtain the results: t R(t) t1 = 1 0.904837 t2 = 2.5 0.778801 t3 = 3 0.740818 2. Using the formula: t=−

1 log R(t) = −(10) log R(t), α

we get the numerical results: R(t) t R(t1 ) = 0.25 13.8629 R(t2 ) = 0.40 9.1629 R(t3 ) = 0.80 2.2314 3. As Minitab gives us the values of the cumulative distribution function F (t), we use here the simple relationship between the reliability and the cumulative distribution function:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

R(t) = 1 − F (t). In the Minitab program, we put the values of t in column c1, and we get the corresponding values of F (t) and R(t) in columns c2 and c3, respectively. For the converse problem, we put the given values of R(t) in column c4 and we get the corresponding values of F (t) and t in columns c5 and c6, respectively. Asking for the cumulative distribution F (t) when t is given, we use the command ‘cdf’ and we mention in a subcommand that we are dealing with an exponential distribution with the mean equal to the value of 1/α. Asking for the value of t when F (t) is given, we use the command ‘invcdf’ and we mention in a subcommand that we are dealing with an exponential distribution with the mean equal to the value of 1/α.

Reliability

208 set c1 1 2.5 3 end cdf c1 c2; exponential 10. let c3 = 1−c2 print c3 set c4 0.25 0.40 0.80 end let c5 = 1−c4 invcdf c5 c6; exponential 10. print c6

Even if the mean life time, which here means the expected time elapsed until the first major repair, is E(T ) = 1/α = 1/0.1 = 10 years, which is not so bad, only about 90% of such cars are OK after one year and only about 74% after three years. Therefore, the engineers have still to work on improving the new brand of cars. 2 2. Failure rate depending on time. In order to get the probability density of the lifetime of a system and the corresponding reliability, Proposition 2 allows us to take into account any failure rate Z(t) provided that we can calculate the definite integral mentioned in the respective formulas. In particular, this can be easily done if the failure rate Z(t) is a polynomial of any degree with respect to time t and, with polynomials, we can practically describe the behaviour of any kind of failure rate as a function of time. In the current applications, however, the following failure rate depending on time has been used, namely: Z(t) = α β tβ−1 ,

(α > 0, β > 0).

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Such a failure rate is increasing if β > 1, constant if β = 1, and decreasing if 0 < β < 1. According to Proposition 2, the lifetime corresponding to such an instantaneous failure rate has the probability density: f (t) = Z(t) e−

Rt

Z(s) ds

Z

t

0

β

= α β tβ−1 e−α t ,

because: Z

0

t

Z(s) ds =

β−1

αβs

0

sβ ds = α β β

and the corresponding reliability is: R(t) =

f (t) = Z(t)

"

(t ≥ 0),

#t 0

= α tβ ,

Reliability

209 h

β−1 −α tβ

= αβt

e

i h

β−1

/ αβt

i

−α tβ

=e

(t ≥ 0).

,

We have obtained a new kind of probability distribution, called Weibull distribution with parameters α and β. Its probability density is: β

f (t) = α β tβ−1 e−α t ,

t ≥ 0.

The mean and variance of the Weibull distribution are obtained using the integral from the next proposition: Proposition 4: We have: Z

+∞

 (m+1)/n

1 α

1 n

n

xm e−αx dx =

0

Γ



m+1 , n 

where m, n are positive real numbers and the special Gamma function is: Γ(t) =

+∞

Z

y t−1 e−y dy.

0

Proof: Making the substitution of variable: α xn = y,

x=

 1/n

y α

,

 (1/n)−1

1 y n α

dx =

1 dy, α

in the integral we want to calculate, we get: Z

+∞

m −αxn

x e

dx =

0

Z

+∞

 (m/n)

y α

0

 (m+1)/n Z

1 1 = n α

+∞

−y

e

 (1/n)−1

1 y n α

1 dy = α

y ((m+1)/n)−1 e−y dy =

0

1 1 (m+1)/n m+1 Γ . 2 n α n Proposition 5: If the lifetime of a system has a Weibull probability distribution with the parameters α and β, then its expected (mean) value is: =



 

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

µ = E(T ) = α

−1/β



!

1 Γ 1+ . β

Proof: Using the formula from Proposition 4 with m = n = β, we get: µ = E(T ) =

Z

+∞

t f (t) dt = αβ

0

β+1 1 = α β α−(β+1)/β Γ β β

Z

+∞

β

tβ e−αt dt =

0

!

=α

−1/β

!

1 Γ 1+ . β

2

Reliability

210

Proposition 6: If the lifetime of a system has a Weibull probability distribution with parameters α and β, then its variance is: 2

σ = V (T ) = α

−2/β

"

2 Γ 1+ β

!

2

−Γ

1 1+ β

!#

.

Proof: Using the formula from Proposition 4 with m = β + 1, and n = β, we get: Z Z +∞

E(T 2 ) =

+∞

t2 f (t) dt = αβ

0

β

tβ+1 e−αt dt =

0

β+2 1 = α β α−(β+2)/β Γ β β

!

=α

−2/β

!

2 Γ 1+ . β

Using the mean value obtained in the previous proposition, we get: σ 2 = V (T ) = E(T 2 ) − [E(T )]2 =

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

= α−2/β

"

2 Γ 1+ β

!

− Γ2

1 1+ β

!#

.

2

Problem 3: Let the lifetime T of a system be Weibull distributed with parameters α = 0.1, and β = 0.5. Use Minitab to solve the direct problem (the time t is given, find the corresponding reliability R(t)) and the converse problem (the reliability R(t) is given, find the corresponding time t). We notice that when we ask Minitab for the cumulative distribution function (the ‘cdf’ command) or for the inverse of cumulative distribution function (the ‘invcdf’ command) of the Weibull distribution with parameters α and β, the subcommand has to be: ‘weibull β α−1/β ’, namely: let k1 = 0.5 let k2 = 0.1**(−1/k1) set c1 0.5 1 10 15.5 end cdf c1 c2; weibull k1 k2. let c3 = 1−c2 print c3 0.931731 0.904837 0.728893 0.674556 set c4 0.95 0.75 0.50 0.25 end let c5 = 1−c4

Reliability

211

invcdf c5 c6; weibull k1 k2. print c6 0.263 8.276 48.045 192.181 In this Minitab program: k1 contains the value of β; k2 contains the value of α−1/β ; c1 contains the values of t given by us (the input of the direct problem); c2 contains the values of F (t) calculated by Minitab; c3 contains the values of R(t) calculated by Minitab (the output of the direct problem); c4 contains the values of R(t) given by us (the input of the converse problem); c5 contains the values of F (t) calculated by Minitab; c6 contains the values of t calculated by Minitab (the output of the converse problem). We can look at the numerical results obtained in many ways. Thus, as R(10) = 0.728893, we can say either that the probability that the respective system is still alive (functions properly) at time t = 10 is 0.728893, or that 72.89% of similar systems will still function properly at time t = 10. The number F (10) = 1 − R(10) = 0.271107 is the probability that the respective system will die (fail to function) before or at time t = 10, or that 27.11% of similar systems will fail during the time interval [0, 10]. When we know the values of cumulative probability distribution function F (t) or of the reliability R(t) we can easily calculate the probability that the lifetime of the system belongs to any time interval (t1 , t2 ], because: P (t1 < T ≤ t2 ) =

Z

t2

f (t) dt =

t1

= F (t2 ) − F (t1 ) = R(t1 ) − R(t2 ), where f (t) is the probability density of the lifetime T . Thus, in our example: P (10 < T ≤ 15.5) = F (15.5) − F (10) =

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

= R(10) − R(15.5) = 0.325444 − 0.271107 = 0.0543373, showing that though the probability of dying (failing) in the time interval (0, 10] is 0.271107, the probability of dying (failing) during the next time interval (10, 15.5] is only 0.0543373. 2 3. Normal lifetime. There are applications in which the lifetime T is supposed to be normally distributed with mean µ and standard deviation σ. As T is nonnegative, we assume that the area below the left-hand side tail of the normal distribution that corresponds to the negative real numbers is negligible. Denoting by Φ(t) the cumulative distribution function of the standard normal distribution N (0, 1), we find that the reliability of a system

Reliability

212

with the lifetime normally distributed N (µ, σ 2 ) is: 1 Z t −(x−µ)2 /(2σ2 ) √ dx = e R(t) = 1 − P (T ≤ t) = 1 − σ 2π −∞   1 Z (t−µ)/σ −y2 /2 t−µ √ =1− dy = 1 − Φ e . σ 2π −∞ This formula:   t−µ R(t) = 1 − Φ , σ involving R(t), t, µ, and σ, is used for calculating one of them when the other three are given. As an example, let us solve one of the four possible types of problems.

Problem 5: The lifetime of a certain device is normally distributed with the standard deviation 10 hours. We know that R(100) = 0.99. Find the mean lifetime of this device. Solution: Let us notice that the information R(100) = 0.99 may be interpreted in the following way: In a set of similar devices, after using them 100 hours, 99% of them are still functioning properly (or are still alive). Thus, R(100) = 0.99, and σ = 10. We want to find µ. Therefore: 0.99 = R(100) = 1 − Φ which implies:



100 − µ , 10 

100 − µ = 0.01. Φ 10 



Denote:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

100 − µ = −t0 , 10 in which case, due to the symmetry of the standard normal distribution, we have: Φ(t0 ) = 0.99 and, from tables for the cumulative distribution function of the standard normal distribution, we get t0 = 2.3263. Therefore: 100 − µ = −2.3263, 10 which implies: µ = 100 + (2.3263)(10) = 123.263 units of time. 2 Remark: The value t0 for which Φ(t0 ) = 0.99 may be obtained from tables or asking Minitab for it, using the commands:

Reliability

213 invcdf 0.99 k1; normal 0 1. print k1 2.3263 2

Remark: In another kind of problem, if we know that the lifetime T is normally distributed N (123.263, 100) and we want to find R(100), then we calculate:   100 − 123.263 R(100) = 1 − Φ = 0.99, 10 or we can ask Minitab to solve this question using the program: cdf 100 k1; normal 123.263 10. let k2 = 1−k1 print k2 0.989999 where the subcommand asking for the normal distribution has to specify its mean and standard deviation, i.e., ‘normal µ σ.’ We can also use the equivalent program involving the standard normal distribution N (0, 1): let k1 = (100−123.263)/10 cdf k1 k2; normal 0 1. let k3 = 1−k2 print k3 0.989999 2 Back to series and parallel systems. When the time is taken into account, each elementary component of a system has its own lifetime T , as a continuous positive random variable. For two elementary components, the respective lifetimes T1 and T2 are independent.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

- T1

- T2

- Series system

- T1 -

-

Parallel system

- T2

Figure 17. Connections.

Reliability

214

(a) Series system. A series system functions properly (is alive) at time t if all its elementary components function properly (are alive) at time t. For the series system from our figure, whose lifetime is T , its reliability at time t is: R(t) = P (T > t) = P (T1 > t, T2 > t) = = P (T1 > t) P (T2 > t) = R1 (t) R2 (t). In particular, if the failure rate of each component is constant, i.e.: Z1 (t) = α1 ,

Z2 (t) = α2 ,

then the lifetimes T1 and T2 are exponentially distributed with the parameters α1 and α2 , respectively. Consequently: R(t) = R1 (t) R2 (t) = e−α1 t e−α2 t = e−(α1 +α2 ) t . The probability density of the lifetime T in this case is: f (t) = −R0 (t) = (α1 + α2 ) e−(α1 +α2 ) t ,

(t ≥ 0),

which shows that T is exponentially distributed with the parameter α1 + α2 . The corresponding failure rate is: Z(t) = α1 + α2 = Z1 (t) + Z2 (t). The probability that the lifetime T takes values in the interval [t1 , t2 ] is: P (t1 ≤ T ≤ t2 ) =

Z

t2

f (t) dt =

t1

= e−(α1 +α2 ) t1 − e−(α1 +α2 ) t2 , and the mean and variance of T are:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

E(T ) =

1 , α1 + α2

V (T ) =

1 . (α1 + α2 )2

(b) Parallel system. A parallel system functions properly (is alive) at time t if at least one of its elementary components functions properly (is alive) at time t. For the parallel system from our figure above, whose lifetime is T , its reliability at time t is: R(t) = P (T > t) = P (T1 > t or T2 > t) = = P (T1 > t) + P (T2 > t) − P (T1 > t, T2 > t) = = P (T1 > t) + P (T2 > t) − P (T1 > t) P (T2 > t) =

Reliability

215 = R1 (t) + R2 (t) − R1 (t) R2 (t).

In particular, if the failure rate of each component is constant, i.e.: Z1 (t) = α1 ,

Z2 (t) = α2 ,

then the lifetimes T1 and T2 are exponentially distributed with the parameters α1 and α2 , respectively. Consequently: R(t) = R1 (t) + R2 (t) − R1 (t) R2 (t) = = e−α1 t + e−α2 t − e−α1 t e−α2 t = = e−α1 t + e−α2 t − e−(α1 +α2 ) t . The probability density of the lifetime T is: f (t) = −R0 (t) = = α1 e−α1 t + α2 e−α2 t − (α1 + α2 ) e−(α1 +α2 ) t ,

(t ≥ 0),

which shows that T is not exponentially distributed but its probability density is a linear combination of exponential densities. The probability that the lifetime T takes on values in the interval [t1 , t2 ] is: P (t1 ≤ T ≤ t2 ) =

Z

t2

f (t) dt =

t1

= e−α1 t1 + e−α2 t1 − e−(α1 +α2 ) t1 − −e−α1 t2 − e−α2 t2 + e−(α1 +α2 ) t2 , and the mean lifetime is:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

E(T ) =

1 1 1 + − . α1 α2 α1 + α2

Queueing Systems

216

Chapter 5. QUEUEING SYSTEMS

ABSTRACT

There are queueing systems everywhere. What makes their study difficult is the fact that they contain two kinds of randomness: customers arrive randomly and the service time is random. The results obtained in this chapter refer to queueing systems in a steady state condition, which means that, in spite of random fluctuations, on average, the mean arrival rate is equal to the mean service rate. A special attention is given to the waiting time per individual customer and the number of arrivals during an arbitrary time interval. The queueing systems analyzed include the systems with infinite or finite capacity, with infinite or finite number of potential customers, the self-service queueing system, and the queueing systems with pressure on arrival or/and service. Simulation in queueing theory is also discussed.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

1. Generalities In a queueing system, customers arrive randomly, at a constant or variable rate. Eventually, they join or form a line, waiting to be served. After waiting in line, the customers are served at a constant or variable rate. Served customers leave the system. There is a double cost involved: (a) Operating cost, necessary for keeping the service facility going (salaries, energy, taxes, machines, equipment); (b) Waiting cost, induced by the loss of customers and deterioration of service quality in a crowded, busy system. The objective of the queueing theory is to describe the structure and evolution of a queueing system, to minimize the total cost, a combination of the opposite operating and waiting costs, and to predict the values of some characteristics of the queueing system like the waiting time in the system per individual customer, for instance. There are queueing systems everywhere. In the emergency room of a hospital the patients are customers and the doctors and nurses are servers. In a factory, the items to be manufactured are customers and the workers and machines are servers. An airplane is a server for passengers but a customer for

Queueing Systems

217

the airport. Other common queueing systems are: a cafeteria, a barber-shop, a bank, a typing pool, a gas station, a repair-shop, a self-service food mart, etc. As shown in the figure below, the basic general structure of a queueing system consists of an input source, randomly generating customers, a queue, the service facility, and the output. The proper queueing system includes the queue and the service facility:

Input Source

- Queue

Service -

Facility

- Output

Figure 18. The queueing system. The main features of these components are: (a) Input source: The ‘number of arrivals’ is a discrete random variable, often Poisson distributed. The ‘interarrival time’ is a continuous random variable, often exponentially distributed. The maximum number of potential customers, called the ‘size’ of the input source, may be finite or infinite.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

(b) Queue: A queueing system can have one or several ‘queues’ in the system. The maximum number of customers allowed to wait, the so called ‘maximum queue length’, may be finite or infinite. The ‘queue discipline’ may be FIFO (first-in-first-out, or first-come-first-served, the most common type of queue discipline), LIFO (last-in-first-out, as it happens in an elevator, for instance), random selection (arbitrary choosing customers for service), and preferential selection (service based on a priority list). (c) Service: The number of servers is generally finite but could be infinite too. The ‘servers’ are independent or interact in certain ways (in series, in parallel, etc.). The ‘service time’ is a continuous random variable, whose probability distribution is either exponential, or Erlang, or degenerate (constant service time). The four figures below show four types of queueing systems, namely: a single-queue-single-server queueing system, a single-queue-multiple-server-inparallel queueing system, a single-queue-multiple-server-in-series queueing system, and a multiple-queue-multiple-server queueing system, respectively.

Queueing Systems

218 - i - i - i -

-

Figure 19. Single-queue-single-server system. - i - i- i -

- i - i - i -

-

Figure 20. Single-queue-multiple-server-in-series system. -

-

- i- i - i - i -

-

-

-

Figure 21. Single-queue-multiple-server-in-parallel system. - i - i - i -

-

- i - i - i -

-

- i - i - i -

-

Figure 22. Multiple-queue-multiple-server system. The common way of identifying a queueing system is by specifying:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Input / Service / Number of Servers / Capacity / Discipline where the ‘capacity’ of the queueing system is the maximum number of customers allowed in the system (waiting to be served and being served). This capacity may be finite or infinite. The standard symbols are: M for exponentially distributed interarrival time or for exponentially distributed service time; GI for general input, i.e. an interarrival time arbitrarily distributed; G for a service time arbitrarily distributed; Ek for a service time Erlang distributed having as second parameter the integer k; D for a degenerate (i.e., constant) service time. By default, the capacity of the queueing system is supposed to be infinite and the queue discipline F IF O. Thus, M/M/s is equivalent to M/M/s/∞/F IF O and represents a queueing system with exponentially distributed interarrival and service times, s servers, infinite capacity, and obeying a first-in-first-out queue discipline. The following notation and terminology will be used throughout this chapter: N (t), the ‘state’ of the queueing system at time t, i.e. the number of customers in the system (waiting to be served or being served) at time t. This is a discrete random variable with nonnegative integer values; N , the state of the queueing system when time is irrelevant; pn (t), the probability of having n customers in the queueing system at time instant t;

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Queueing Systems

219

pn , the probability of having n customers in the queueing system when time is irrelevant; T , the ‘interarrival time’, a continuous nonnegative random variable; λn , the ‘mean arrival rate’ (i.e., the expected number of customers entering the queueing system per unit time) if there are already n customers in the queueing system; ¯ the ‘effective mean arrival rate’, which is the mean of the mean arrival λ, rates; λ, the mean arrival rate when it does not depend on how many customers are in the queueing system; 1/λn , the ‘expected interarrival time’ if there are already n customers in the queueing system; 1/λ, the expected interarrival time when it does not depend on how many customers are in the queueing system; S, the ‘service time’ per individual customer, a continuous, nonnegative random variable; µn , the ‘mean service rate’ (i.e., the expected number of customers served by the service facility per unit time) if there are n customers in the queueing system; µ, the mean service rate when it does not depend on how many customers are in the queueing system; 1/µn , the ‘expected service time’ per individual customer if there are n customers in the queueing system; 1/λ, the expected service time per individual customer when it does not depend on how many customers are in the queueing system; Lq , the ‘mean queue length’, i.e., the expected number of customers waiting to be served; L, the ‘mean number of customers’ in the queueing system, waiting to be served and being served; Wq , the ‘waiting time in the queue’ per individual customer. It is a continuous, nonnegative random variable; Wq , the ‘mean waiting time in the queue’ per individual customer; W, the ‘waiting time in the queueing system’, waiting to be served and being served, per individual customer. It is a continuous, nonnegative random variable; W , the ‘mean waiting time in the queueing system’, waiting to be served and being served, per individual customer. Denoting by P (·) the probability of an event and by E(·) the expected value of a random variable, according to the above terminology, we have: pn (t) = P (N (t) = n),

pn (t) ≥ 0,

+∞ X

n=0

pn (t) = 1,

Queueing Systems

220

pn = P (N = n),

pn ≥ 0, (n = 0, 1, . . .),

+∞ X

pn = 1,

n=0

L = E(N ) =

+∞ X

n pn =

n=0

Wq = E(Wq ), λ=

+∞ X

+∞ X

n pn ,

n=1

W = E(W), λn p n .

n=0

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

2. Steady state condition The present treatment of the main queueing systems is based on the birthand-death stochastic process whose basic assumptions are: Assumption 1: If there are already n customers in the queueing system, the probability distribution of the remaining time until a new birth (arrival) will occur is exponential with parameter λn ; Assumption 2: If there are already n customers in the queueing system, the probability distribution of the remaining time until a new death (departure, service completion) will occur is exponential with parameter µn ; Assumption 3: Only one birth (arrival) and one death (departure, service completion) can occur at a time. The first two assumptions are well justified. As discussed in chapter 3, if the state of the queueing system is N = n and the only information available is the mean arrival rate λn and the mean service rate µn , then the most unbiased and uncertain assumptions about the interarrival time and the service time, both being nonnegative random variables, is that they are exponentially distributed with parameter λn and parameter µn , respectively. Assumption 3, however, may be dropped and replaced by other rules concerning the instant arrivals and departures. In the evolution of any system, there are inevitable random fluctuations to be taken into account. A coherent system theory may be however developed only if there is a stability at the level of mean values. If during the evolution of a system this stability at the level of mean values holds, then we say that the respective system is in a steady state condition. A naive example will help better understand what a steady state condition really means. Let us consider a queueing system consisting of a tap, a bathtub, and a sink. The tap represents the input, the sink is the output of the system and the water coming in and going out the bathtub represents customers. If no water comes in and no water goes out, which means that there is no input and no output, the systems is in a static equilibrium state. If there is water in the bathtub, the tap is off, but the sink is open, there is output, for a while, but no input, and the system is in a transient state, tending to a static equilibrium

Queueing Systems

221

when all water goes out. If the tap is on but the sink is off, there is input but no output and the system is also in a transient state but it will blow up eventually. Finally, if both the tap and the sink are on, the system has both input and output and, generally, it is in a transient state, depending on how much water comes in and how much water goes out. If, however, the amount of water coming in and the amount of water going out are the same, the system is in a dynamic equilibrium and we say that the system is in a steady state condition. Our understanding of what a dynamic equilibrium means should be in fact much larger: The amount of water coming in and the amount of water going out can fluctuate but, on average, the input rate is equal to the output rate. A queueing system is in a ‘steady state condition’ if the following balance between input and output holds: The mean rate at which the entering incidents occur is equal to the mean rate at which the leaving incidents occur. By mean rate we understand the expected number of occurrences per unit time. This statistical equilibrium between input and output at the level of mean rates is called the ‘Rate In = Rate Out Principle’. Throughout this chapter we will assume that the queueing systems are in a steady state condition. For a system in a steady state condition the following equalities between mean values, known as Little’s formulas, are supposed to hold: λ W q = Lq ,

λ W = L.

If the mean arrival rate is constant, i.e., λn = λ, (n = 0, 1, . . .), then the ‘effective mean arrival rate’, which is the mean of the mean arrival rates, is: λ=

+∞ X

λn p n = λ

n=0

+∞ X

pn = λ,

n=0

and Little’s formulas become:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

λ W q = Lq ,

λ W = L.

The interpretation of Little’s formulas, is obvious: The average number of customers entering the queueing system per unit time multiplied by the average amount of time spent by a customer in line gives the average number of customers waiting to be served. Also, the average number of customers entering the queueing system per unit time multiplied by the average waiting time in the queueing system (waiting to be served and being served) gives the average number of customers in the system. If the mean service rate per busy server is constant, i.e.: µn = µ,

(n = 1, 2, . . .),

Queueing Systems

222

then, a third Little’s formula says that: 1 . µ It corresponds to the intuitive fact that, on average and per individual customer, the waiting time spent in the queueing system is equal to the waiting time spent in line, waiting to be served, plus the service time. There is a general relationship between the expected number of customers in line, Lq , and the expected number of customers in the queueing system (waiting to be served and being served) L. In getting this relation, we take into account the obvious fact that a customer has to wait only if all servers are busy when he arrives. Assuming that there are s servers in the queueing system, we have: W = Wq +

+∞ X

Lq =

(n − s) pn =

=

n pn − s

n=s

+∞ X

" s−1 X

pn =

n=s

− =

s−1 X

(n − s) pn =

n=s

n=s+1 +∞ X

+∞ X

n pn +

=L−s+

n pn −

s−1 X

!

pn =

n=0 s−1 X

n pn − s +

n=0

#

n=s

n=0

n pn − s 1 −

n=0 +∞ X

+∞ X

(s − n) pn =

n=0 s−1 X

(s − n) pn .

n=0

Therefore: L = Lq + s −

s−1 X

(s − n) pn .

n=0

The following rate diagram describes the transition between the possible states of the system when only one arrival and one departure (or service completion) can occur at a time:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

λ0

λ1 ?

0

?

1

6

3 . . . n−2 6

µ2

λn−2 ?

2 6

µ1

λ2

6

µ3

λn−1 ?

?

n−1

n

6

µn−1

λn

µn

Figure 23. The general rate diagram.

?

n+1 . . . 6

µn+1

Queueing Systems

223

Let us apply the ‘Rate In = Rate Out Principle’ to the above rate diagram: State

Balance equations

Formulas

0

µ1 p 1 = λ 0 p 0

p1 =

λ0 µ1

p0

1

λ 0 p 0 + µ2 p 2 = λ 1 p 1 + µ1 p 1

p2 =

λ1 µ2

p1

.. .

.. .

.. .

n − 2 λn−3 pn−3 + µn−1 pn−1 = λn−2 pn−2 + µn−2 pn−2 n−1

λn−2 pn−2 + µn pn = λn−1 pn−1 + µn−1 pn−1

n .. .

λn−1 pn−1 + µn+1 pn+1 = λn pn + µn pn .. .

pn−1 = pn =

λn−2 µn−1

λn−1 µn

pn+1 = .. .

pn−2

pn−1

λn µn+1

pn

From the last column of the above tableau we get: p1 =

λ0 p0 , µ1

p2 =

λ0 λ1 p0 , µ1 µ2

...,

pn =

λ0 λ1 . . . λn−1 p0 , µ1 µ1 , . . . µn

...

Denoting by: Cn =

λ0 λ1 . . . λn−1 , µ1 µ2 , . . . µn

we obtain: pn = Cn p0 ,

(n = 1, 2, . . .).

But, we also have: 1=

+∞ X

pn = p0 +

n=0

+∞ X

Cn p0 ,

n=1

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

from which, if the series is convergent, we find: "

p0 = 1 +

+∞ X

Cn

#−1

,

n=1

"

pn = Cn 1 +

+∞ X

Cn

#−1

,

(n = 1, 2, . . .).

n=1

The rate diagram describes what happens on average in a queueing system. If the system is in a steady state condition, the ‘Rate In = Rate Out Principle’

Queueing Systems

224

allows us to determine the probability distribution of the possible states of the system. Often, especially in small queueing systems, the input is influenced by the so called ‘balking effect’. Fewer customers will enter the system if it is crowded. If the mean arrival rate to the queueing system is constant and equal to λ, denoting by bn the probability of balking if there are n customers in the system, then the effective mean arrival rate in the queueing system is: λn = (1 − bn ) λ. Problem 1: A gas station is a queueing system in a steady state condition. There is one gas pump, serving on average µ cars per hour and the cars arrive for refueling at a mean rate of λ cars per hour. If there are n cars at the gas station, the probability of balking (i.e., giving up and going to another gas station) is estimated to be n/4. Calculate the probability distribution of the number of cars waiting to be served and being served at the respective gas station. Solution: Obviously, the system has only five possible states, namely n = 0, 1, 2, 3, 4. The mean arrival rates into the queueing system are:    [1 − (n/4)] λ,

λn =  

0,

n = 0, 1, 2, 3;

n = 4, 5, . . .

The corresponding rate diagram is shown below. The mean service rate is constant but the mean arrival rates depend on how many customers are in the queueing system, gradually diminishing due to the balking effect:

λ

0 6

λ/2

λ/4

?

?

?

?

1

2

3

4

6

µ Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

3λ/4

6

µ

6

µ

µ

Figure 24. The rate diagram induced by balking. Denoting by ρ = λ/µ, the ‘Rate In = Rate Out Principle’ applied to this rate diagram gives the following ‘balance equations’ from which the probability distribution of the possible states of the queueing system is obtained, namely:

Queueing Systems

225

State

Balance equation

Formula

0

µ p1 = λ p 0

p 1 = ρ p0

1

λ p0 + µ p2 = (3/4) λ p1 + µ p1

p2 = (3/4) ρ2 p0

2

(3/4) λ p1 + µ p3 = (1/2) λ p2 + µ p2

p3 = (3/8) ρ3 p0

3

(1/2) λ p2 + µ p4 = (1/4) λ p3 + µ p3

p4 = (3/32) ρ4 p0

4

(1/4) λ p3 = µ p4

p4 = (3/32) ρ4 p0

Therefore, p1 , p2 , p3 , and p4 are completely determined by λ, µ and p0 . But we also have: 4 X

pn = 1

n=0

which implies that p0 is also completely determined by the mean arrival rate λ, before balking, and the mean service rate µ, namely: 

p0 = 1 + ρ +

3 2 3 3 3 4 ρ + ρ + ρ 4 8 32

−1

. 2

3. M/M/1 with constant mean arrival rate and constant mean service rate Assume that in an M/M/1 queueing system we have: s = 1;

λn = λ,

(n = 0, 1, . . .);

The rate diagram is given below: λ λ λ

0 Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

6

?

?

1

2 6

µ

µ

λ

?

3 . . . n−2 6

µn = µ,

6

µ

(n = 1, 2, . . .).

λ

λ

?

?

n−1

n

6

µ

?

n+1 . . . 6

µ

µ

Figure 25. The rate diagram for M/M/1. The ‘utilization factor’ of the queueing system M/M/1 is the ratio: ρ=

λ . µ

Queueing Systems

226

Using the notations from the previous sections, we have: Cn =

λ0 λ1 . . . λn−1 = ρn , µ1 µ2 , . . . µn +∞ X

"

p0 = 1 +

Cn

#−1

(n = 1, 2, . . .),

"

= 1+

n=1

+∞ X

ρ

n

#−1

,

n=1

pn = Cn p0 = p0 ρn ,

(n = 1, 2, . . .).

The following remark about sums of numbers in geometric progression will be subsequently used several times. Remark: Let a0 be a real number and q > 0, (q 6= 1). Denote by: Sn = a0 + a0 q + a0 q 2 + . . . + a0 q n−2 + a0 q n−1 . Multiplying the previous equality by −q, we get: −q Sn = −a0 q − a0 q 2 − a0 q 3 − . . . − a0 q n−1 − a0 q n . Summing up the last two equalities, we have: 1 − qn . S n = a0 1−q If q < 1, then: lim q n = 0,

n→+∞

and, asymptotically, we obtain the sum of the geometric series: S∞ = lim Sn = n→+∞

a0 . 1−q

2

If ρ < 1, which is equivalent to λ < µ, meaning that on average the mean input rate does not exceed the mean service rate, then we have: +∞ X

Cn =

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

n=1

+∞ X

ρn =

n=1

ρ 1−ρ

and we get: pn = (1 − ρ) ρn ,

(n = 0, 1, . . .).

There is a very simple formula for calculating the probability of having at least k customers in the queueing system. Indeed: P (N ≥ k) =

+∞ X

n=k

pn =

+∞ X

n=k

(1 − ρ) ρn =

Queueing Systems

227 k

= (1 − ρ)

ρ = ρk . 1−ρ

The probability of having more than k customers in the queueing system is: P (N > k) = P (N ≥ k + 1) = ρk+1 . We have supposed that ρ < 1 and we will continue to assume this here. Obviously, if ρ > 1, or equivalently λ > µ, which means that the mean arrival rate exceeds the mean service rate, then the system blows up and the series: +∞ X

Cn =

n=1

+∞ X

ρn

n=1

is divergent. The expected number of customers in the system may be calculated in a straightforward way, namely: L=

+∞ X

n pn =

n=0

= (1 − ρ) ρ

+∞ X

+∞ X

n pn =

n=1

n ρn−1 = (1 − ρ) ρ

n=1

+∞ X

dρn = n=1 dρ

X d +∞ d = (1 − ρ) ρ ρn = (1 − ρ) ρ dρ n=1 dρ !

= (1 − ρ)ρ

ρ 1−ρ

!

=

1−ρ+ρ ρ = . 2 (1 − ρ) 1−ρ

Therefore, we have proved that if ρ < 1, then: L=

λ ρ = . 1−ρ µ−λ

The expected number of customers waiting to be served is:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Lq =

+∞ X

(n − 1) pn =

n=1

=

+∞ X

n=1

n pn −

+∞ X

pn = L − (1 − p0 ).

n=1

Thus, the relationship between L and Lq is given by: Lq = L − 1 + p 0 = L − ρ =

ρ2 = 1−ρ

Queueing Systems

228 2

λ . µ(µ − λ) As the effective mean arrival rate, which is the mean of the mean arrival rates, is in this case: = ρL =

λ=

+∞ X

λn p n = λ

+∞ X

pn = λ,

n=1

n=1

from Little’s formulas we get the expected waiting time in the system and in the queue, per individual customer, to be: W =

L 1 = , λ µ−λ

λ Lq = , λ µ(µ − λ) 1 W = Wq + . µ Problem 2: A barber-shop is an M/M/1 queueing system in a steady state condition, with constant mean arrival rate and constant mean service rate. There is one barber. The mean interarrival time is 20 minutes and the mean service time is 15 minutes. Wq =

1. Find the proportion of time when the barber is lazy. Solution: We have s = 1, and we take one hour as the unit of time. The mean arrival rate is: λ=

1 60 customers/min = = 3 customers/hr. 20 20

The mean service rate is: µ=

60 1 customers/min = = 4 customers/hr. 15 15

The utilization factor is:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

ρ=

3 λ = < 1, µ 4

and, therefore we can apply the mathematical formalism discussed before: p0 = 1 − ρ = 1 −

3 1 = = 0.25. 4 4

Therefore, 25% of the time there is nobody in the barber-shop and the barber is lazy. 2

Queueing Systems

229

2. Find the probability of having more than four customers in the barbershop. Solution: We have: P (N > 4) = ρ5 = (0.75)5 = 0.237304687. 2 3. Find the expected number of customers in the barber-shop. Solution: 0.75 ρ L= = = 3 customers. 2 1−ρ 1 − 0.75 4. Find the expected number of customers in the barber-shop. Solution: We have: Lq = L − 1 + p0 = 3 − 1 + 0.25 = 2.25 customers. 2 5. Find the expected amount of time spent by a customer in the barbershop (waiting to be served and being served). Solution: We have: W =

3 L = = 1 hr = 60 min. 2 λ 3

6. Find the expected amount of time spent by a customer in line, waiting to be served. Solution: We have: Wq = W −

1 1 3 = 1 − = hr = 45 min. µ 4 4

7. The management has decided to hire a second barber if the expected waiting time in line per individual customer exceeds one hour. Find the increase in the mean arrival rate that will result in the hiring of a second barber. ˜ > λ a larger mean arrival rate for which the expected Solution: Denote by λ waiting time in line per customer does not exceed the bound imposed by the management. We must have:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

˜ < µ = 4 customers, λ = 3 customers < λ and: ˜q = W

˜ ˜ λ λ = ≤ 1 hr. ˜ ˜ µ (µ − λ) 4 (4 − λ)

We have: ˜ ≤ 16 − 4 λ ˜⇒λ ˜ ≤ 16 = 3.2 customers/hr. λ 5

Queueing Systems

230

Therefore, if the mean arrival rate in the barber-shop increases by more than: (3.2−3)=0.2 customers/hr, a second barber has to be hired. 2 4. M/M/s with constant mean arrival rate and constant mean service rate per busy server There are s servers in the queueing system. Let λ be the constant mean arrival rate and µ the mean service rate per busy server. The rate diagram of the queueing system is given below: λ

λ ?

0

?

3 . . . s−2

2 6

µ

λ

?

1

6

λ

2µ

6

λ ?

?

s−1

s

6

3µ

λ

6

(s−1)µ

?

s+1 . . . 6

sµ

sµ

Figure 26. The rate diagram for M/M/s. We have: λn = λ,    n µ,

Therefore, as:

µn =  

we obtain: Cn = and:

(n = 1, 2, . . . , s − 1);

s µ,

(n = s, s + 1, . . .).

Cn =

λ0 λ1 . . . λn−1 , µ 1 µ 2 . . . µn

λn , n! µn

(n = 1, . . . , s − 1),

λn

, (n = s, s + 1, . . .). s! sn−s µn The utilization factor is again the ratio between the mean arrival rate and the mean service rate, which in this case becomes: Cn =

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

(n = 0, 1, . . .),

ρ=

λ . sµ

Using the utilization factor in the above expressions of Cn we get: sn λn Cn = = n! µn n!

λ sµ

!n

=

sn n ρ , n!

(n = 1, . . . , s − 1),

Queueing Systems and:

231

ss Cn = = s! sn−s µn s! λn

λ sµ

!n

ss n ρ , s!

=

(n = s, s + 1, . . .).

Taking into account that the probability of having n customers in the system is: pn = P (N = n) = Cn p0 , (n = 1, 2, . . .), we have: pn =

sn n ρ p0 , n!

(n = 1, . . . , s − 1),

and:

ss n ρ p0 , (n = s, s + 1, . . .). s! Proposition 1: Assuming ρ < 1, the probability of having no customer in the queueing system is equal to: pn =

p0 =

" s−1 X sn n=0

ss ρs ρ + n! s! 1 − ρ n

#−1

.

Proof: From the general formalism given in section 2, and using the sum of the convergent geometric series, we have: +∞ X

"

p0 = 1 +

Cn

#−1

"

= 1+

n=1

Cn +

= 1+

s−1 X

n=1

" s−1 X sn n=0

X sn n ss +∞ ρn ρ + n! s! n=s

s s ρs ρ + n! s! 1 − ρ n

+∞ X

Cn

#−1

=

n=s

n=1

"

=

s−1 X

#−1

#−1

=

. 2

Proposition 2: If ρ < 1, the probability of having at least s customers in the queueing system, which is also the probability that all servers are busy, is:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

P (N ≥ s) =

ss ρs p0 . s! 1 − ρ

Proof: Using the formula for the sum of a convergent geometric series, we have: P (N ≥ s) =

+∞ X

pn =

n=s

=

X ss +∞ ss ρs p0 ρn = p 0 . 2 s! n=s s! 1 − ρ

Queueing Systems

232

Proposition 3: If ρ < 1, the expected number of customers waiting to be served is: ss ρs+1 Lq = p 0 . s! (1 − ρ)2 Proof: Using the standard notations from the second section of this chapter, we have: +∞ +∞ X X ss Lq = (n − s) pn = (n − s) ρn p0 = s! n=s n=s =

+∞ +∞ X X d ss ss p0 ρs+1 (n − s)ρn−s−1 = p0 ρs+1 (ρn−s ) = s! s! dρ n=s n=s +∞ +∞ X X ss ss s+1 d n−s s+1 d ρn = = p0 ρ ρ = p0 ρ s! dρ n=s s! dρ n=0

!

ss d = p0 ρs+1 s! dρ

1 1−ρ

!

!

=

ss ρs+1 p0 . 2 s! (1 − ρ)2

For an M/M/s with constant mean arrival rate and constant mean service rate the relationship between the expected number L of customers in the queueing system and the expected number Lq of customers waiting to be served, proved in section 2, holds and is given by the equality: L = Lq + s −

s−1 X

(s − n) pn .

n=0

As the effective mean arrival rate, which is the mean of the mean arrival rates, is: λ=

+∞ X

λn p n = λ

n=0

+∞ X

pn = λ,

n=0

Little’s formulas have the form:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

W =

L , λ

Wq =

Lq , λ

W = Wq +

1 . µ

Problem 3: A typing pool of a company has three typists equally qualified. Letters to be typed arrive randomly with a mean rate of 15 letters per hour. Each typist can type on average 6 letters per hour. Assume that the typing pool is an M/M/3 queueing system in a steady state condition with constant mean arrival rate and constant mean service rate per busy server. 1. Find the proportion of time when all three typists are working. Solution: We have s = 3, λ = 15 letters/hr, µ = 6 letters/hr. The utilization factor is: 15 5 λ ρ= = = < 1. sµ (3) (6) 6

Queueing Systems

233

The probability of having no letter in the typing pool is: p0 =

" s−1 X sn n=0

=

"

  2 X 3n 5 n

n=0

n!

6

s s ρs ρ + n! s! 1 − ρ n

33 (5/6)3 + 3! 1 − (5/6)

#−1

#−1

=

= 0.04494382.

According to Proposition 2, the probability that the service facility is fully working is: 33 ρ3 P (N ≥ 3) = p0 = 3! 1 − ρ =

33 (5/6)3 (0.04494382) = 0.702247191. 2 3! 1 − (5/6)

2. Find the average number of letters waiting to be typed. Solution: According to Proposition 3, we have: Lq =

33 ρ3+1 = p0 3! (1 − ρ)2

(5/6)4 33 = (0.04494382) = 3.511235937 ≈ 3.5 letters. 2 3! [1 − (5/6)]2 3. Find the expected amount of time spent by a letter in the typing pool. Solution: Applying the third and second Little’s formulas, we get: W = Wq +

1 Lq 1 = + = µ λ µ

3.511235937 1 + = 0.400749062 hr ≈ 24.04 min = 24 min 2.4 sec. 2 15 6 4. The management would like to know which one of the following two arrangements of the typing pool would be more economical:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

=

-

-

Letters - i- i- i- i-

-

-

-

Figure 27. Arrangement 1. - i- i - i -

-

Letters - i- i- i-

-

- i- i - i -

-

Figure 28. Arrangement 2.

Queueing Systems

234

Solution: Arrangement 1, where the arriving letters form one line waiting to be typed by the first available typist, is an M/M/3 queueing system with constant mean arrival rate λ = 15 letters/hour and constant mean service rate per busy server µ = 6 letters/hour. In evaluating the efficiency of our arrangement, the expected waiting time in the typing pool per letter would be a good indicator. For Arrangement 1 we have already obtained that W ≈ 24.04 min. Arrangement 2, where each typist has her own line with waiting letters and the arriving letters are uniformly distributed in the three waiting lines, consists of three M/M/1 queueing systems, each with constant mean arriving ˜ = 5 letters/hour and constant mean service rate µ = 6 letters/hour. rate λ The expected waiting time in the typing pool per letter is: ˜ = W

1 1 = = 1 hr = 60 min. ˜ 6−5 µ−λ

Therefore, Arrangement 1 of the typing pool is much better.

2

5. Waiting time The expected waiting time in the queueing system per individual customer W and the expected waiting time in the queue per individual customer Wq give only a global indication about waiting time. More often than not we should like to know the answer to more specific questions like: “What is the probability of waiting in line or in the system less than t1 units of time, more than t2 units of time, or in between t1 and t2 units of time?” (a) The waiting time W per individual customer in the queueing system M/M/1 with constant mean arrival rate λ and constant mean service rate µ. Assume that a new customer arrives in the queueing system when there are already n customers there. For this new customer, the waiting time to spend in the queueing system, waiting to be served and being served, is:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Tn+1 = S1 + . . . + Sn + Sn+1 , where Si is the service time for the i-th customer already in the system, (i = 1, . . . , n), and Sn+1 is the service time for the newly arrived customer. As S1 , . . . , Sn+1 are independent random variables exponentially distributed with the same parameter µ, the waiting time Tn+1 is Erlang (Gamma) distributed with parameters µ and n + 1, as proved in chapter 3. Its density, called the probability density of the waiting time in the queueing system for a new customer provided that there are already n customers in the queueing system, is: µn+1 n −µt t e , (t ≥ 0). fw (t | n) = n!

Queueing Systems

235

The expected waiting time in the system for a new customer and the variance of the waiting time in the system for a new customer if there are already n customers in the system when the new customer arrives are: E(W | n) =

n+1 , µ

V (W | n) =

n+1 . µ2

Proposition 4: The probability density of the waiting time W in the queueing system M/M/1, with constant mean arrival rate λ and constant mean service rate µ, per individual customer is: fw (t) = (µ − λ)e−(µ−λ)t ,

(t ≥ 0),

if we have:

λ < 1. µ Proof: The function fw (t | n) gives the probability density of the waiting time in the queueing system for a new customer arriving when the state of the system is n. But the system is in state n with probability pn . Therefore, the probability density fw (t) of the waiting time in the queueing system, whatever the state of the system is when a new customer arrives is: ρ=

+∞ X

+∞ X

µn+1 n −µ t fw (t | n) pn = fw (t) = t e (1 − ρ) ρn = n=0 n=0 n! = µ(1 − ρ) e−µ t

+∞ X

(µ t ρ)n = µ (1 − ρ) e−µ t eµ t ρ = n! n=0

= µ (1 − ρ) e−µ (1−ρ) t = (µ − λ) e−(µ−λ) t . 2 Proposition 4 tells us that the waiting time in M/M/1 per individual customer is exponentially distributed with the parameter µ − λ. The expected waiting time is equal to: 1 , µ−λ

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

W = E(W) =

which we already know from section 3 of this chapter. But now, as the probability density of W is exponentially distributed with the parameter µ − λ, we also know that the variance of the waiting time in the system per individual customer is: 1 V (W) = , (µ − λ)2 and that the probability of waiting in the queueing system in between t1 and t2 units of time is: P (t1 < W ≤ t2 ) =

Z

t2

t1

fw (t) dt =

Queueing Systems

236 h

= −e−(µ−λ) t

it2 t1

= e−(µ−λ) t1 − e−(µ−λ) t2 .

Similarly, the probability that a customer has to spend more than t units of time in the queueing system M/M/1 is: h

P (W > t) = −e−(µ−λ)t

i+∞ t

= e−(µ−λ) t .

Problem 4: A barber-shop is an M/M/1 queueing system in a steady state condition, with constant mean arrival rate and constant mean service rate. There is one barber. The mean interarrival time is 20 minutes and the mean service time is 15 minutes. 1. Find the probability that a new customer has to wait more than 30 minutes in the barber-shop if there are four customers already there. Solution: For the barber-shop queueing system: s = 1,

λ = 3 customers/hr,

µ = 4 customers/hr.

and the utilization factor is: ρ=

λ 3 = < 1. µ 4

The probability of waiting more than 30 minutes in the barber-shop when there are four customers already there is: P (W > 30 min | 4) = P (W >

1 1 hr | 4) = 1 − P (W ≤ hr | 4). 2 2

But:

1 hr | 4) 2 is the cumulative probability distribution function of Gamma distribution, with the parameters α = µ = 4 and β = 4 + 1 = 5, at 1/2=0.5. We can use Minitab for getting its value. We remember that the Minitab subcommand for the Gamma distribution has to indicate the parameters β = 5 and 1/α = 1/4 = 0.25, in this order: cdf 0.5 k1; gamma 5 0.25. let k2=1−k1 print k1 k2 We obtain the values:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

P (W ≤

P (W ≤

1 hr | 4) = k1 = 0.052653, 2

Queueing Systems

237

and, the answer to our question: P (W > 30 min | 4) = k2 = 0.947347. 2 2. Find the probability that a customer has to wait more than 30 minutes in the barber-shop. Solution: As W is exponentially distributed with the parameter µ − λ = 4 − 3 = 1, we have: P (W > 30 min) = P (W > =

Z

+∞

0.5

h

fw (t) dt = −e−t

i+∞ 0.5

1 hr) = 2

= e−0.5 = 0.606531. 2

(b) The waiting time in the queue Wq per individual customer in the queueing system M/M/1 with constant mean arrival rate λ and constant mean service rate µ. Assume that a new customer arrives in the queueing system when there are already n customers there. For this new customer, the waiting time to spend in line, waiting to be served, is: Tn = S1 + . . . + Sn , where Si is the service time for the i-th customer already in the system, (i = 1, . . . , n). The independent random variables S1 , . . . , Sn are exponentially distributed with the same parameter µ. The waiting time Tn is Erlang (Gamma) distributed with parameters α = µ and β = n. Its density, called the probability density of the waiting time in the queue for a new customer, provided that there are already n customers in the system, is: fwq (t | n) =

µn tn−1 e−µt , (n − 1)!

(t ≥ 0).

The expected waiting time in the queue for a new customer when there are already n customers in the system and its variance are:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

E(Wq | n) =

n , µ

V (Wq | n) =

n . µ2

Also, the probability that a new customer is waiting in the queue between t1 and t2 units of time if there are n customers in the queueing system already when the new customer arrives is: P (t1 < Wq ≤ t2 ) = =

Z

t2

t1

fwq (t | n) dt = F (t2 ) − F (t1 ),

Queueing Systems

238

where F (t) is the cumulative probability distribution function of the Gamma distribution with parameters α = µ, β = n. Proposition 5: The probability density of the waiting time in the queue Wq per individual customer in a queueing system M/M/1 with constant mean arrival rate λ and constant mean service rate µ is: fwq (t) = 0,

fwq (0) = 1 − ρ,

(t < 0);

fwq (t) = ρ(µ − λ) e−(µ−λ)t , if we have: ρ=

(t > 0),

λ < 1. µ

Proof: The function fwq (t | n) gives the probability density of the waiting time in the queue for a new customer arriving when the state of the system is n. But the system is in state n with probability pn . The new customer will wait in line only if there is at least one customer in the system when the new customer arrives. Therefore, the probability density fwq (t) of the waiting time in the queue per individual customer taking all possible states n ≥ 1 into account is: fwq (t) =

+∞ X

fwq (t | n) pn =

n=1 +∞ X

+∞ X (µ t ρ)n−1 µn n−1 −µ t n −µ t = t = e (1 − ρ) ρ = ρ µ (1 − ρ) e n=1 (n − 1)! n=1 (n − 1)!

= ρ µ (1 − ρ) e−µ t

+∞ X

(µ t ρ)n = ρ µ (1 − ρ) e−µ t eµ t ρ = n! n=0

= ρ µ (1 − ρ) e−µ (1−ρ) t = ρ (µ − λ) e−(µ−λ) t , if t > 0. However, the new customer does not wait if there is no customer in the queueing system:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

P (Wq = 0) = p0 = 1 − ρ. 2 Remark: Proposition 5 tells us that the waiting time in the queue per individual customer in an M/M/1 queueing system has a mixed probability distribution, equal to zero on the interval (−∞, 0), with a continuous part of exponential type on the interval (0, +∞), and a discrete component at origin t = 0. The expected waiting time in the queue per individual customer is: Wq = E(Wq ) =

Z

+∞

−∞

t fwq (t) dt =

Queueing Systems

239

Z

=

0

−∞

Z

t fwq (t) dt + 0 fwq (0) +

= 0 p0 + ρ

Z

+∞

0

+∞

t fwq (t) dt =

0

t (µ − λ) e−(µ−λ) t dt

To evaluate the above integral, we notice that: d h −(µ−λ) t i te = e−(µ−λ) t − t (µ − λ) e−(µ−λ) t , dt and integrating the last equality between 0 and +∞, we obtain: 0=

h

i+∞ t e−(µ−λ) t 0

=

Z

+∞

−(µ−λ) t

e

0

dt −

Z

+∞

0

t (µ − λ) e−(µ−λ) t dt,

which implies: Z

+∞

0

=

Z

+∞

0

t (µ − λ) e−(µ−λ) t dt =

"

#

Z

+∞

e−(µ−λ) t dt =

0

"

d 1 1 e−(µ−λ) t dt = − e−(µ−λ) t − dt µ−λ µ−λ

#+∞ 0

=

1 . µ−λ

Using this result, we can go back to the expected value of the random variable Wq and we get: Wq = E(Wq ) =

ρ = ρ W. µ−λ

The probability that a new customer has to wait for service between t1 > 0 and t2 > t1 units of time is: P (t1 < Wq ≤ t2 ) = =ρ

Z

t2

t1

Z

t2

t1

fwq (t) dt = h

(µ − λ) e−(µ−λ) t dt = ρ −e−(µ−λ) t

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

= ρ e−(µ−λ) t1 − ρ e−(µ−λ) t2 .

it2 t1

=

2

Proposition 6: In an M/M/1 queueing system with constant mean arrival rate λ and constant mean service rate µ, the probability that a new customer has to spend in the queue, waiting to be served, more than t > 0 units of time is: P (Wq > t) = ρ e−(µ−λ) t = P (N ≥ 1) e−(µ−λ) t = = P (the new customer has to wait) e−(µ−λ) t , if we have: ρ=

λ < 1. µ

Queueing Systems

240

Solution: Taking into account the expression of the probability density fwq (t) of the random variable Wq , as given in Proposition 5, for t > 0 we have: P (Wq > t) = =ρ

Z

+∞

t

+∞

Z

fwq (x) dx =

t

h

(µ − λ) e−(µ−λ) x dx = ρ −e−(µ−λ) x

i+∞ t

= ρ e−(µ−λ) t .

On the other hand, in M/M/1 we have: p0 = 1 − ρ, and we obtain: ρ = 1 − p0 =

+∞ X

pn = P (N ≥ 1).

n=1

Therefore: P (Wq > t) = ρ e−(µ−λ) t = P (N ≥ 1) e−(µ−λ) t . 2 Problem 5: A barber-shop is an M/M/1 queueing system in a steady state condition, with constant mean arrival rate and constant mean service rate. There is one barber. The mean interarrival time is 20 minutes and the mean service time is 15 minutes. 1. Find the probability that a new customer has to wait more than 30 minutes in line, waiting to be served, if there are four customers already there when he arrives. Solution: For the barber-shop queueing system: s = 1,

λ = 3 customers/hr, ρ=

µ = 4 customers/hr.

3 λ = < 1. µ 4

We have:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

P (Wq > 30 min | 4) = P (Wq >

1 1 hr | 4) = 1 − P (Wq ≤ hr | 4). 2 2

But we notice that:

1 hr | 4) 2 is the cumulative probability distribution function of Gamma distribution, with the parameters α = µ = 4 and β = 4, at 1/2=0.5. We can use Minitab for getting its value. We remember that the Minitab subcommand for the Gamma distribution has to indicate the parameters β = 4 and 1/α = 1/4 = 0.25, in P (Wq ≤

Queueing Systems

241

this order: cdf 0.5 k1; gamma 4 0.25. let k2=1−k1 print k1 k2 We obtain the values: P (Wq ≤

1 hr | 4) = k1 = 0.142877, 2

and the answer to our question is: P (Wq > 30 min | 4) = k2 = 0.857123. 2 2. Find the probability that a customer has to wait more than 30 minutes in line, waiting to be served. Solution: We have: P (Wq > 30 min) = P (Wq > =

Z

+∞

0.5

= 0.75

Z

+∞

0.5

fwq (t) dt = ρ

+∞

Z

h

0.5

e−t dt = 0.75 −e−t

(µ − λ) e−(µ−λ) t dt =

i+∞ 0.5

1 hr) = 2

= 0.75 e−0.5 = 0.454898. 2

(c) The waiting time in the queue Wq per individual customer in the queueing system M/M/s with constant mean arrival rate λ and constant mean service rate µ per busy server. Proposition 7: In an M/M/s queueing system with constant mean arrival rate λ and constant mean service rate µ per busy server, the probability that a customer has to wait more than t units of time in line, waiting to be served is: P (Wq > t) = P (N ≥ s) e−(s µ−λ) t , if we have: Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

ρ=

λ < 1. µ

Proof: Let us notice that, if all s servers are busy, a queueing system M/M/s may be viewed as being an M/M/1 system with the service rate equal to s µ. Obviously, a new customer arriving in an M/M/s system has to wait before being served only if all s servers are busy. Therefore, taking this into account and applying the formula from Proposition 6, we get: P (Wq > t) =

Queueing Systems

242 −(s µ−λ) t

= P (the new customer has to wait) e

=

= P (all servers are busy) e−(s µ−λ) t = = P (N ≥ s) e−(s µ−λ) t , where, as proved in Proposition 2, we have: ss ρs P (N ≥ s) = p0 . s! 1 − ρ Problem 6: A typing pool of a company has three typists equally qualified. Letters to be typed arrive randomly with a mean rate of 15 letters per hour. Each typist can type on average 6 letters per hour. Assume that the typing pool is an M/M/3 queueing system in a steady state condition with constant mean arrival rate and constant mean service rate per busy server. Calculate the probability that a letter has to spend more than 30 minutes waiting to be typed. Solution: We have an M/M/3 queueing system with: s = 3,

λ = 15 letters/hr,

µ = 6 letters/hr.

In Problem 2, answering the first question, we have already calculated the probability that all typists are busy as being: P (N ≥ 3) = 0.702247191. Using the formula proved in Proposition 7, namely: P (Wq > t) = P (N ≥ s) e−(s µ−λ) t , we obtain in our case: P (Wq > 30 min) = P (Wq >

1 hr) = 2

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

= P (N ≥ 3) e−[(3)(6)−15] (0.5) = (0.702247191) e−(3)(0.5) = 0.157. 2 (d) The waiting time W per individual customer in the queueing system M/M/s with constant mean arrival rate λ and constant mean service rate µ per busy server. The waiting time W spent by a new customer in the queueing system is the sum of two independent continuous random variables, namely: the waiting time Wq spent in the queue, waiting to be served while other customers are served, and his own service time denoted by S. The next proposition shows how to calculate the probability density of the sum of two independent random variables.

Queueing Systems

243

Proposition 8: Let X and Y be two independent random variables with the probability distributions f and g, respectively. Then, the probability density h of their sum Z = X + Y is the convolution of f and g, denoted by h = f ⊗ g and defined by: h(z) =

Z

+∞

−∞

f (w) g(z − w) dw.

Proof: The linear transformation z = x + y, w = x, whose inverse transformation is x = w, y = z − w, has the absolute value of the Jacobian | J | equal to 1, because: ∂(x, y) = J= ∂(w, z)

∂x ∂w

∂x ∂z

∂y ∂w

∂y ∂z

1 0 = = 1. −1 1

As the joint probability density φ(x, y) of the random vector (X, Y ) is φ(x, y) = f (x) g(y), the probability density of the random vector (W, Z), where W = X and Z = X + Y , is: ψ(w, z) = φ(w, z − w) | J |= f (w) g(z − w). The marginal probability density h of Z is: h(z) =

Z

+∞

ψ(w, z) dw =

−∞

Z

+∞

−∞

f (w) g(z − w) dw. 2

Proposition 9: In an M/M/s queueing system with constant mean arrival rate λ and constant mean service rate µ per busy server, the probability that the waiting time spent by a customer in the queueing system exceeds the time t > 0 is: 



1 − e−µ (s−1−λ/µ) t  , P (W > t) = e−µ t 1 + P (N ≥ s) s − 1 − µλ if we have: Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

ρ=

λ < 1. sµ

Proof: Obviously: W = S + Wq , where: W is the waiting time in the queueing system per customer, S is the service time per customer, and Wq is the waiting time in the queue per customer. Let fw , fs , and fwq be the probability density functions of the random variables W, S, and Wq , respectively. We notice that S is always positive, because service requires time to be performed, but Wq could be both equal

Queueing Systems

244

to zero, when the new customer is served immediately upon arrival, or positive, when the new customer has to wait because all servers are busy when he arrives. If Wq > 0, then, the probability density of the sum S + Wq is the convolution: Z +∞ (fs ⊗ fwq )(z) = fs (w) fwq (z − w) dw. −∞

The remaining proof is split in three parts: (a), (b), (c). (a) P (W > t) = P (W > t, less than s servers are busy)+ +P (W > t, all s servers are busy). (b) P (W > t, less than s servers are busy) = = P (S > t) =

Z

+∞

fs (x) dx =

t

+∞

Z

=

t

h

µ e−µ x dx = −e−µ x

i+∞ t

= e−µ t .

(c) P (W > t, all s servers are busy) = = P (S + Wq > t, Wq > 0) = =

Z

+∞

Z

+∞

Z

=



fs (w) fwq (z − w) dw dz = Z

fs (w)

−∞

t

+∞



fwq (z − w) dz dw =

Change the variable: z − w = u,

=

Z

+∞

−∞

fs (w)

Z

+∞

t−w

(fs ⊗ fwq )(z) dz =

t

+∞

−∞

t

+∞

Z



fwq (u) du dw =

z = u + w,

Z

dz = du,

+∞

−∞

fs (w) P (Wq > t − w) dw =

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

(S > 0 ⇒ w > 0; Wq > 0 ⇒ t − w > 0) ⇒ 0 < w < t.

=

Z

0

=

Z

0

t

t

fs (w) P (Wq > t − w) dw =

µ e−µ w P (N ≥ s) e−(s µ−λ) (t−w) dw = −(s µ−λ) t

= µ P (N ≥ s) e

Z

0

t

e−µ w e(s µ−λ) w dw =

Queueing Systems

245 

We have: −µ + s µ − λ = µ s − 1 −

−(s µ−λ) t

= µ P (N ≥ s) e e

= µ P (N ≥ s) e−µ t e−µ (s−1−λ/µ) t

t



eµ (s−1−λ/µ) w dw =

0

−µ t −µ(s−1−λ/µ) t

= µ P (N ≥ s) e

Z

λ µ

"

1 eµ (s−1−λ/µ) w µ (s − 1 − λ/µ)

#t

=

0

h i 1 eµ (s−1−λ/µ) t − 1 = µ (s − 1 − λ/µ)

= e−µ t P (N ≥ s)

1 − e−µ (s−1−λ/µ) t . s − 1 − µλ

The last result from part (b) and the last result from part (c), introduced into (a), give: 

P (W > t) = e−µ t 1 + P (N ≥ s)

−µ (s−1−λ/µ) t

1−e s−1−

λ µ



.

2

Problem 7: A typing pool of a company has three typists equally qualified. Letters to be typed arrive randomly with a mean rate of 15 letters per hour. Each typist can type on average 6 letters per hour. Assume that the typing pool is an M/M/3 queueing system in a steady state condition with constant mean arrival rate and constant mean service rate per busy server. Calculate the probability that a letter has to spend more than 30 minutes in the typing pool, waiting to be typed and being typed. Solution: We have an M/M/3 queueing system with: s = 3,

λ = 15 letters/hr,

µ = 6 letters/hr.

In Problem 2, answering the first question, we have already calculated the probability that all typists are busy as being:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

P (N ≥ 3) = 0.702247191. Using the formula proved in Proposition 9, namely: −µ t

P (W > t) = e





1 − e−µ (s−1−λ/µ) t  1 + P (N ≥ s) , s − 1 − µλ

in our case we have: P (W > 30 min) = P (W >

1 hr) = 2

Queueing Systems

246

"

= e−(6)(0.5) 1 + P (N ≥ 3)

−(6)(3−1−15/6)(0.5)

1−e 3 − 1 − 15/6

#

= 0.293246. 2

6. Finite queue Let the number M be the capacity of the queueing system, representing the maximum number of customers allowed in the queueing system. If more than M customers arrive, where M ≥ s and s is the number of servers, then they are turned away. Many queueing systems in real life are of this type; they do not allow more than M − s customers to wait for service, often due to insufficient space. Let us analyze an M/M/s/M queueing system in a steady state condition with constant mean arrival rate λ and constant mean service rate µ per busy server. The rate diagram is shown below: λ

λ ?

0

?

2 . . . s−2

1

6

λ

6

µ

λ ?

?

s−1

s

6

2µ

λ

6

(s−1)µ

λ ?

6

sµ

?

s+1 . . . M -1

M

6

sµ

sµ

Figure 29. The rate diagram for M/M/s/M . Therefore:

   λ,

λn =  

(n = 0, 1, . . . , M − 1),

0, (n = M, M + 1, . . .),

   n µ,

where M ≥ s. Let:

µn =  

(n = 1, . . . , s − 1),

s µ, (n = s, s + 1, . . .),

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

ρ=

λ sµ

be the utilization factor, without imposing any restrictions on it, except ρ 6= 1. Then, as: λ0 λ1 . . . λn−1 Cn = , µ1 µ2 . . . µn we have: Cn =

sn n λn = ρ , n! µn n!

(n = 1, 2, . . . , s − 1);

Queueing Systems

247 λ

Cn =

n

s! sn−s

s

µn

=

Cn = 0,

s n ρ , (n = s, s + 1, . . . , M ); s! (n = M + 1, M + 2, . . .),

Proposition 10: The probability of having an empty queueing system is: p0 =

" s−1 X sn n=0

ss 1 − ρM −s+1 ρn + ρs n! s! 1−ρ

#−1

.

Proof: According to the formalism from the second section, and using the formula for the sum of a finite set of numbers in geometric progression proved in section 3, we have: p0 = 1 +

+∞ X

Cn

!−1

"

= 1+

s−1 X

Cn +

s−1 X

M sn n ss X = 1+ ρ + ρn s! n=s n=1 n!

"

=

" s−1 X sn n=0

Cn

#−1

=

n=s

n=1

n=1

M X

#−1

ss 1 − ρM −s+1 ρn + ρs n! s! 1−ρ

=

#−1

. 2

Once we know the value of p0 , and the expressions of Cn for different values of n, the probability distribution of the possible states of the queueing system M/M/s/M is obtained from: pn = Cn p0 , namely:

(n = 1, 2, . . .),

sn p0 ρn , (n = 1, 2, . . . , s − 1); n! ss pn = p0 ρn , (n = s, s + 1, . . . , M ); s! (n = M + 1, M + 2, . . .). pn = 0, pn =

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

In particular, the probability of turning away new customers is the probability that the queueing system has reached its capacity: pM =

ss p0 ρM . s!

Proposition 11: In an M/M/s/M queueing system in a steady state condition with constant mean arrival rate λ and constant mean service rate µ per busy server, the probability that the service facility is fully working is: P (N ≥ s) =

ss 1 − ρM −s+1 p 0 ρs , s! 1−ρ

Queueing Systems

248

where: ρ=

λ . sµ

Proof: As pn = 0, for n > M , and: pn =

ss p0 ρn , s!

(n = s, s + 1, . . . , M ),

we have: P (N ≥ s) =

+∞ X

pn =

n=s

M X

pn =

n=s

M M M −s X X X ss ss ss n s n−s s ρn = = p0 ρ = p0 ρ ρ = p0 ρ s! s! s! n=s n=s n=0

=

ss 1 − ρM −s+1 p0 ρs . 2 s! 1−ρ

Proposition 12: The expected number of customers waiting to be served in an M/M/s/M queueing system in a steady state condition with constant mean arrival rate λ and constant mean service rate µ per busy server is: Lq =

ss ρs+1 p0 {1 − [(M − s) (1 − ρ) + 1] ρM −s }, s! (1 − ρ)2

where the utilization factor is: ρ=

λ . sµ

Proof: We have: Lq =

+∞ X

(n − s) pn =

n=s

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

=

M X ss p0 (n − s) ρn = s! n=s

M M −s X X ss ss p0 ρs+1 (n − s) ρn−s−1 = p0 ρs+1 n ρn−1 = s! s! n=s n=0

M −s X ss d n ss d = p0 ρs+1 (ρ ) = p0 ρs+1 s! s! dρ n=0 dρ

ss d = p0 ρs+1 s! dρ =

1 − ρM −s+1 1−ρ

!

M −s X

n

ρ

!

=

n=0

=

i ss ρs+1 h M −s+1 M −s p0 1 − ρ − (M − s + 1) ρ (1 − ρ) = s! (1 − ρ)2

Queueing Systems ss ρs+1 {1 − [(M − s) (1 − ρ) + 1] ρM −s }. 2 = p0 s! (1 − ρ)2

249

Remark: We remember that for an M/M/s queueing system with constant mean arrival rate λ and constant mean service rate µ per busy server we have: Lq =

ss ρs+1 p0 . s! (1 − ρ)2

Therefore, the quantity: [(M − s) (1 − ρ) + 1] ρM −s is the correction factor, or the decreasing factor, of Lq , due to turning away customers. 2 The expected number L of customers in the queueing system, waiting to be served and being served, may be obtained from the general expression: L = Lq + s −

s−1 X

(s − n) pn .

n=0

Proposition 13: The effective mean arrival rate in an M/M/s/M queueing system in a steady state condition with constant mean arrival rate λ and constant mean service rate µ per busy server is: λ = λ (1 − pM ). Proof: As the mean arrival rates are: λn =

   λ,  

(n = 0, 1, . . . , M − 1),

0, (n = M, M + 1, . . .),

we get: λ=

+∞ X

λn p n = λ

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

n=0

M −1 X

pn = λ (1 − pM ) ≤ λ. 2

n=0

Remark: The number 1 − pM is called the decreasing factor of the mean arrival rate λ, due to turning away new customers when the finite capacity of the queueing system is reached. 2 The expected amount of time spent by a customer in the queueing system, W , and the expected amount of time spent by a customer in line, waiting to be served, Wq , may be obtained from Little’s formulas: W =

L , λ

Wq =

Lq , λ

Queueing Systems

250

1 . µ Proposition 14: The relationship between the effective mean arrival rate and the mean service rate µ in an M/M/s/M queueing system in a steady state condition with constant mean arrival rate λ and constant mean service rate µ per busy server is: W = Wq +

"

λ=µ s−

s−1 X

#

(s − n) pn .

n=0

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Proof: Applying the ‘Rate In = Rate Out Principle’ to the rate diagram of an M/M/s/M queueing system in a steady state condition with constant mean service rate λ and constant mean service rate µ per busy server given in Figure 29, we obtain the following balance equations: State

Balance equations

Formulas

0

µ p1 = λ p0

λ p 0 = µ p1

1

λ p0 + 2µ p2 = µ p1 + λ p1

λ p1 = 2µ p2

2 .. .

λ p1 + 3µ p3 = 2µ p2 + λ p2 .. .

λ p2 = 3µ p3 .. .

s−2

λps−3 + (s − 1)µps−1 = (s − 2)µps−2 + λps−2

λps−2 = (s − 1)µps−1

s−1

λ ps−2 + sµ ps = (s − 1) µ ps−1 + λ ps−1

λ ps−1 = sµ ps

s

λ ps−1 + s µ ps+1 = s µ ps + λ ps

λ ps = s µ ps+1

.. .

.. .

.. .

M −1

λ pM −2 + s µ pM = s µ pM −1 + λ pM −1

λ pM −1 = s µ pM

Summing up the formulas from the last column of the above tableau we get: " s−1 # M −1 M X X X λ pn = µ n pn + s pn , n=0

n=1

n=s

which is equivalent to: λ (1 − pM ) = µ

" s−1 X

n=0

n pn + s 1 −

s−1 X

n=0

pn

!#

,

Queueing Systems

251

which finally may be written as: "

λ=µ s−

s−1 X

#

(s − n) pn . 2

n=0

Remarks: The above formulas have a much simpler form in the case of an M/M/1/M queueing system in a steady state condition with constant mean arrival rate λ and constant mean service rate µ.Thus, as s = 1, the probability of being served immediately upon arrival is equal to: p0 =

1−ρ , 1 − ρM +1

because for such a queueing system we have: λn = λ,

(n = 0, 1, . . . , M − 1);

Cn = and, therefore:

 n   ρ ,

λ0 λ1 . . . λn−1 =  µ1 µ2 . . . µn  +∞ X

"

p0 = 1 +

0,

Cn

#−1

µn = µ,

=

M X

ρn

#−1

λ , µ

(n = M + 1, M + 1, . . .)

"

= 1+

M X

n

ρ

#−1

=

n=1

1 − ρM +1 = 1−ρ "

n=0

ρ=

(n = 1, 2, . . . , M );

n=1

"

(n = 1, 2, . . .),

#−1

=

1−ρ . 1 − ρM +1

The probability distribution of the possible states of the queueing system is:

 1−ρ n    1 − ρM +1 ρ , pn = Cn p0 =   

0,

(n = 1, 2, . . . , M );

(n = M + 1, M + 2, . . .).

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

In particular, the probability of turning away new customers is: pM =

1−ρ ρM . M +1 1−ρ

The expected number of customers in the queueing system is: L=

ρ (M + 1)ρM +1 . − 1−ρ 1 − ρM +1

Queueing Systems

252

Indeed, a direct approach shows that: +∞ X

L=

n pn =

n=0

M ρ (1 − ρ) X n ρn−1 = 1 − ρM +1 n=0

M ρ (1 − ρ) d X ρ (1 − ρ) d = ρn = M +1 1−ρ dρ n=0 1 − ρM +1 dρ

!

=

1 − ρM +1 1−ρ

!

=

ρ (M + 1)ρM +1 ρ (1 − ρ) 1 − ρM +1 − (M + 1)ρM (1 − ρ) = . − 1 − ρM +1 (1 − ρ)2 1−ρ 1 − ρM +1

The relationship between the expected number L of customers in the queueing system and the expected number Lq of customers waiting to be served is given by the following equality: L = Lq + 1 − p 0 . because, according to the definition of L, we have: L=

+∞ X

M X

n pn =

n=0

=

M X

(n − 1) pn +

n pn =

n=1 M X

p n = Lq + 1 − p 0 .

n=1

n=1

The effective mean arrival rate in the queueing system is: λ = λ (1 − pM ) = µ (1 − p0 ). Indeed, on one hand we have: λ=

+∞ X

λn p n = λ

n=0

M −1 X

pn = λ (1 − pM ).

n=0

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

On the other hand, we get: 1−ρ ρM λ (1 − pM ) = λ 1 − 1 − ρM +1 =λ

!

=

1 − ρM λ ρ − ρM +1 = = 1 − ρM +1 ρ 1 − ρM +1

1−ρ =µ 1− 1 − ρM +1

!

= µ (1 − p0 ).

Queueing Systems

253

Little’s formulas are used to calculate the mean waiting time in the queueing system and the mean waiting time in the queue per individual customer, namely: Lq L Wq = . 2 W = , λ λ Problem 8: A barber-shop is a queueing system in a steady state condition, with constant mean arrival rate and constant mean service rate. There is one barber. The mean interarrival time is 20 minutes and the mean service time is 15 minutes. Assume that there are six chairs to accommodate waiting customers. If all chairs are taken, new customers arriving are turned away. 1. Find the probability of being served immediately upon arrival. Solution: The number of servers, the capacity, and the utilization factor of this queueing system are: s = 1,

M = 6 + 1 = 7,

ρ=

λ 3 = = 0.75, µ 4

respectively, because λ = 3 customers/hr, and µ = 4 customers/hr. The barber-shop is an M/M/1/7 queueing system. A new customer is served immediately upon arrival when there is no customer in the system. The corresponding probability is: p0 =

1 − 0.75 1−ρ = = 0.2778. 2 M +1 1−ρ 1 − (0.75)8

2. Find the effective mean arrival rate. Solution: Due to turning away customers, the average number of customers arriving in the barber-shop per unit time (one hour) is: λ = µ (1 − p0 ) = 4 (1 − 0.2778) = 2.89 customers/hr. 2 3. Find the expected number of customers waiting to be served. Solution: The expected number of customers in the barber-shop is:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

L= =

(M + 1) ρM +1 ρ − = 1−ρ 1 − ρM +1

0.75 (7 + 1) (0.75)7+1 − = 2.11 customers, 1 − 0.75 1 − (0.75)7+1

and the expected number of customers waiting to be served is: Lq = L − 1 + p0 = 2.11 − 1 + 0.2778 = 1.39 customers. 2 4. Find the expected time spent by one customer in the barber-shop.

Queueing Systems

254

Solution: We have: W =

2.11 L = = 0.73 hr. = 43.8 min. = 43 min 48 sec. 2 2.89 λ

5. Calculate the probability of turning away new customers when the barber-shop is full at its capacity. Solution: The probability of turning away new customers is: λ pM = 1 − . λ As the capacity is M = 7, we get: p7 = 1 −

2.89 = 0.037. 2 3

6. By what amount should the mean arrival rate increase in order to have the probability of turning away customers equal to 0.1? ˜ be the new mean arrival rate, and: Solution: Let λ ρ˜ =

˜ ˜ λ λ = µ 4

be the new utilization factor such that the new probability of turning away customers is: 1 − ρ˜ 7 p˜7 = ρ˜ = 0.1. 1 − ρ˜8 We have to solve the equation of eighth degree: 0.9 ρ˜8 − ρ˜7 + 0.1 = 0. We notice that one of the eight roots is ρ˜ = 1, but we need the smallest root and we do not know whether 1 is indeed the needed one. We can use the computer software ‘Mathematica’ for getting an approximation of the solutions of this equation. We use the command:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

NSolve[lhs == rhs, x] where ‘lhs’ means the left-hand side of the equation, ‘rhs’ means the righthand side, and x is the variable with respect to which we want to solve the respective equation; notice the double ‘==’ sign needed. After writing the command, we send it by pressing together the keys: ‘Shift’ and ‘Enter’. The input from us and the output from the computer for solving our equation are: In[1]:= NSolve[0.9 rho∧ 8 − rho∧ 7 + 0.1 == 0, rho] Out[1]= {{rho − > −0.614319 − 0.278712 I}, {rho − > −0.614319 + 0.278712 I},

Queueing Systems

255

{rho − > −0.198632 − 0.66249 I}, {rho − > −0.198632 + 0.66249 I}, {rho − > 0.398082 − 0.61972 I}, {rho − > 0.398082 + 0.61972 I}, {rho − > 0.940849}, {rho − > 1.}} √ Therefore, the equation has six complex roots (I means −1), and two real roots. Taking the smallest real root, we get: ˜ = µ ρ˜ = 4 ρ˜ = (4) (0.940849) ≈ 3.7634 customers. λ The increase in the mean arrival rate should be: ˜ − λ = 3.7634 − 3 = 0.7634 customers. 2 λ 7. Calculate the average annual loss in revenue due to turning away customers, knowing that the average revenue per customer is $20, and the shop is open 10 hours a day, 24 days a month. Solution: The average loss of customers per hour, due to turning away customers, is λ − λ. Thus, the average annual loss in revenue is: (revenue/customer) × [(λ−λ) customers/hour] × (number of hours/day) × (number of days/month) × (number of months/year): (20)(3−2.89)(10)(24)(12) = $6336.

2

Problem 9: A barber-shop is a queueing system in a steady state condition, with constant mean arrival rate of 3.7634 customers per hour and constant mean service rate of 4 customers per hour per busy server. There are two barbers. Assume that there are five chairs to accommodate waiting customers. If all chairs are taken, new customers arriving are turned away. 1. Calculate the probability of being served immediately upon arrival. Solution: The barber-shop is an M/M/2/7 queueing system with:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

s = 2,

λ = 3.7634 customers/hr,

µ = 4 customers/hr,

and capacity: 5 (maximum number of customers allowed to wait) + 2 (maximum number of customers that can be served) = 7 customers. The utilization factor is: ρ=

3.7634 λ = = 0.470425. sµ (2) (4)

The probability of having no customer in the barber-shop is: p0 =

" s−1 X sn n=0

ss 1 − ρM −s+1 ρ + ρs n! s! 1−ρ n

#−1

=

Queueing Systems "

= 1+

256

2 22 1 − (0.470425)7−2+1 (0.470425) + (0.470425)2 1! 2! 1 − 0.470425

#−1

≈ 0.36133.

The probability of having only one customer in the barber-shop is: p1 =

s1 21 p0 ρ1 = (0.36133) (0.470425) ≈ 0.339957. 1! 1!

Therefore, the probability of being served immediately upon arrival is: p0 + p1 ≈ 0.36133 + 0.339957 ≈ 0.7013. 2 2. Calculate the average number of customers waiting to be served. Solution: As proved in Proposition 12, we have: Lq =

ss ρs+1 p0 [1 − ρM −s − (M − s) ρM −s (1 − ρ)] = s! (1 − ρ)2 =

(0.470425)2+1 22 (0.36133) × 2! (1 − 0.470425)2

×[1 − (0.470425)7−2 − (7 − 2)(0.470425)7−2 (1 − 0.470425)] ≈ ≈ 0.2457 customers. 2 3. Calculate the effective mean arrival rate. Solution: As proved in Proposition 14, an equivalent formula for the effective mean arrival rate is: "

λ=µ s−

s−1 X

#

(s − n) pn = µ (2 − 2 p0 − p1 ) =

n=0

= (4)[2 − (2)(0.36133) − 0.339957] ≈ 3.7495 customers/hr. 2 4. Calculate the expected waiting time in the barber-shop per customer. Solution: From the second Little’s formula we calculate the expected waiting time in line per customer:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Wq =

0.2457 Lq = ≈ 0.065528737 hr ≈ 4 min. 3.7495 λ

From the third Little’s formula we get the expected waiting time in the barber-shop per customer: W = Wq +

1 1 = 0.065528737 + = µ 4

= 0.315528737 hr ≈ 19 min. 2 5. Calculate the probability of turning away customers.

Queueing Systems

257

Solution: As the capacity of the barbershop is M = 7, the probability of having a full barber-shop is: p7 = 1 −

λ 3.7495 =1− ≈ 0.004. 2 λ 3.7634

7. Finite input source Until now we assumed that there was no upper bound for the number of customers that could come to the queueing system. Let now M be a positive finite integer representing the size of the input source, i.e., the maximum number of potential customers. In such a case, let λ be the expected number of visits paid by a potential customer to the queueing system during a unit time interval. Then, 1/λ is the mean amount of time elapsed between two consecutive visits paid by a potential customer to the queueing system. As far as µ and 1/µ are concerned, they continue to be the mean service rate per busy server and the mean service time per individual customer, as before. Obviously, when n customers are in the queueing system then M −n customers are out. Therefore, the rate diagram of such a queueing system is: Mλ

(M-1)λ

?

0

?

2 . . . s−2

1

6

(M-s+2)λ (M-s+1)λ (M-s)λ

6

µ

?

?

s−1

s

6

2µ

6

(s−1)µ

λ

?

6

sµ

sµ

Thus:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

λn =

 

0

µn =

   nµ  

if

sµ if

if

n = 0, 1, . . . , M ,

if

n = M + 1, M + 2, . . ., n = 1, . . . , s − 1; n = s, s + 1, . . .

According to the general formalism from section 2, we get: Cn =

M

6

Figure 30. The rate diagram for the finite input source.

   (M − n)λ

?

s+1 . . . M -1

λ0 λ1 . . . λn−1 = µ 1 µ 2 . . . µn

sµ

Queueing Systems  M (M − 1) . . . (M − n + 1)λn   ,   n!µn   =

             

258 (n = 1, 2, . . . , s − 1);

M (M − 1) . . . (M − n + 1)λn , s!sn−s µn

(n = s, s + 1, . . . , M );

0,

(n = M + 1, M + 2, . . .)

or, equivalently:          

!  n

M

λ µ

n

M

Cn =  n! n−s   s!s      

n

(n = 1, 2, . . . , s − 1);

,

!  n

λ µ

,

0,

(n = s, s + 1, . . . , M ); (n = M + 1, M + 2, . . .)

The probability of having no customer in the queueing system is: +∞ X

"

p0 = 1 +

Cn

#−1

"

= 1+

n=1

=

" s−1 X

n=0

M

n

!

s−1 X

Cn +

!n

Cn

#−1

=

n=s

n=1

λ µ

M X

M X

n! + n−s n=s s!s

!

M

n

λ µ

!n #−1

.

The probability of having an arbitrary number n of customers in the queueing system is:          

M

n

!

pn = Cn p0 =  n! n−s   s!s      

 n

λ p0 µ M

n

!

if (n = 1, 2, . . . , s − 1);

,  n

λ p0 µ

0,

,

if (n = s, s + 1, . . . , M ); if (n = M + 1, M + 2 . . .)

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

The expected number of customers waiting to be served is: +∞ X

M X

n! Lq = (n − s)pn = (n − s) n−s s!s n=s n=s

M

n

!

p0

λ µ

!n

.

The expected number of customers in the queueing system is obtained from the general formula connecting Lq and L, namely: L = Lq + s −

s−1 X

n=0

(s − n) pn ,

Queueing Systems

259

or, equivalently: Lq = L − s +

s−1 X

(s − n) pn .

n=0

The effective mean arrival rate has a simple formula: λ=

+∞ X

λn p n =

n=0

=λ M

M X

n=0

M X

(M − n)λ pn =

n=0

pn −

M X

!

npn = λ (M − L).

n=0

Therefore, the effective mean arrival rate induced by the finite size of the input source is: λ = λ (M − L), which allows us to get an equivalent formula for the expected number of customers in the queueing system, namely: λ L=M− . λ Little’s formulas are used for getting the expected waiting time in line, i.e., Wq , and the expected waiting time in the queueing system, i.e., W , per individual customer, namely: Lq Wq = ¯ , λ

L W = ¯, λ

W = Wq +

1 . µ

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

In order to get another formula for the effective mean arrival rate, which will also show the relationship between the effective mean arrival rate and the mean service rate per busy server, we use the balance equations implied by the ‘Rate In = Rate Out Principle’ applied to the rate diagram of the finite input source queueing system given in Figure 30, at the beginning of this section. For an arbitrary state n, the balance equation induced by the ‘Rate In = Rate Out Principle’ is: (M − n + 1) λ pn−1 + (n + 1) µ pn+1 = (M − n) λ pn + n µ pn , if 1 ≤ n ≤ s − 1, and: (M − n + 1) λ pn−1 + s µ pn+1 = (M − n) λ pn + s µ pn , if s ≤ n ≤ M − 1. We get the tableau:

Queueing Systems State

260 Balance Equations

0

µ p1 = M λ p0

1

M λ p0 + 2µ p2 = (M -1)λ p1 + µ p1

2

(M -1)λ p1 + 3µ p3 = (M -2)λ p2 + 2µ p2

.. .

.. .

s−2

(M -s+3)λps−3 +(s-1)µps−1 =(M -s+2)λ ps−2 +(s-2)µ ps−2

s−1

(M -s+2)λ ps−2 +sµ ps =(M -s+1)λ ps−1 +(s-1)µ ps−1

s

(M -s+1)λ ps−1 +sµ ps+1 =(M -s)λ ps +sµ ps

.. .

.. .

M −1

2λ pM −2 +sµ pM =λ pM −1 +sµ pM −1

State

Consequences

0 1 2 .. .

M λ p0 = µ p1 (M − 1)λ p1 = 2µ p2 (M − 2)λ p2 = 3µ p3 .. .

s−2 s−1 s .. .

(M − s + 2)λ ps−2 = (s − 1)µ ps−1 (M − s + 1)λ ps−1 = sµ ps (M − s)λ ps = sµ ps+1 .. .

M −1

λ pM −1 = sµ pM

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Summing up the M equalities from the second column of the last table above, we obtain: λ

M −1 X

(M − n) pn = µ

n=0

λ

M X

n=0

(M − n) pn = µ

s−1 X

n pn + s

n=1

" s−1 X

λ(M − L) = µ s −

pn ,

s−1 X

n=0 s−1 X

n=0

!

n=s

n pn + s 1 −

n=0

"

M X

#

(s − n) pn .

pn

!#

,

Queueing Systems

261

Thus, we get the second formula for the effective mean arrival rate: "

λ=µ s−

s−1 X

#

(s − n) pn .

n=0

Remark: We have ignored the state M in the above computation because the corresponding balance equation, namely: λ pM −1 = sµ pM , has already been taken into account, as a consequence of the balance equation corresponding to the state M − 1. 2 Problem 10: A taxicab company has 25 cars. There is a maintenance crew inspecting the cars daily. When there is a serious problem, the car is sent to the repair shop of the company. On average, a car is sent to the repair shop after two years. The team in the repair shop deals with a car for a third of a month on average. Calculate the effective mean arrival rate of the cars to the repair shop. Solution: We want not only to solve the problem but also to show that, in dealing with a finite input source, the finite sums involved contain a lot of terms but, practically, only relatively few terms are essential, the rest of them being negligible. At the same time, some partial numerical results from some formulas may be subsequently used in other formulas. In what follows, we are going to use four decimals in our computation. The 25 cars represent the potential customers to the repair shop. Therefore, we are dealing here with a finite input source queueing system in a steady state condition. The team represents the server. Thus, s = 1. The size of the input source is M = 25. Also, taking one year as the time unit, the mean arrival rate of the cars to the repair shop and the mean service time are: λ = 0.5 visits (payed by a car to the repair shop)/year;

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

1 1 1 1 = months = years = years, µ 3 (3)(12) 36 respectively, which implies that the mean service rate is µ = 36 cars/year. The probability of having no car in the repair shop is: p0 =

"

M X

n=0

"

n!

M

n

25! 25! 1 = 1+ + 24! 72 23!

!



1 72

λ µ

!n #−1

2

25! + 22!

"

M X

M! = n=0 (M − n)! 

1 72

3

25! + ... + 0!

λ µ

!n #−1



1 72

=

25 #−1

=

Queueing Systems 262  −1 24 23 1 25 ≈ = 1+ + (previous #) + (previous #) + . . . + (previous #) 72 72 72 72 ≈ (1+0.3472+0.1157+0.0370+0.0113+0.0033+0.0009+0.0002+0.0001)−1 = = (1.5157)−1 = 0.6598 Let us mention that the last term in the formula of p0 is: 

25!

1 72

25

= 5.719173 × 10−22 .

The expected number of cars in the repair shop is: +∞ X

M X

M! n L= n p n = p0 n=1 n=1 (M − n)!

λ µ

!n

=

= (0.6598)(0.3472 + 0.2314 + 0.1110 + 0.0452 + +0.0165 + 0.0054 + 0.0014+ +0.0008 + 0.0002) = (0.6598)(0.7591) = 0.50085418 cars. Let us notice that the numbers from the last parenthesis are just the terms from the formula of p0 multiplied by n, (n = 0, 1, 2, . . .), respectively. Finally, the effective mean arrival rate of the cars to the repair shop is: λ = λ (M − L) = 0.5(25 − 0.50085418) = 12.24957291 cars/year. 2 8. Other queueing systems (a) Self-service queueing system. In a self-service system every customer is his own server. If the mean arrival rate λ and the mean self-service rate µ per customer are constant, then the rate diagram is given below: λn = λ,

n = 0, 1, . . . ;

µn = nµ,

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

λ

λ ?

0 6

2µ

λ ?

3 . . . n-2

2 6

µ

λ ?

1

n = 1, 2, . . . .

6

λ ?

?

n-1

n

6

3µ

λ

(n − 1)µ

6

nµ

Figure 31. The self-service queueing system.

?

n+1 . . . 6

(n + 1)µ

Queueing Systems

263

In such a case, we get: Cn =

λn λ0 λ1 . . . λn−1 = , µ1 µ2 . . . µn n!µn

n = 1, 2, . . . .

The probability of having no customer in the system is: "

p0 = 1 +

+∞ X

Cn

#−1

=

n=1

" +∞ #−1 X (λ/µ)n n=0

n!

= e−λ/µ .

Thus, the probability distribution of the states of the system is: p0 = e−λ/µ ,

pn = Cn p0 =

(λ/µ)n −λ/µ e , n!

(n = 1, 2, . . .),

which is just the Poisson probability distribution with the parameter λ/µ. The mean value of the number of customers in the self- service system is: L = E(N ) = and the variance: V (N ) =

λ , µ

λ . µ

The effective mean arrival rate is: λ=

+∞ X

λn pn = λ,

n=0

and, according to the first Little’s formula: W =

L 1 L = = . λ µ λ

The third and second Little’s formulas tell us that:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Wq = W −

1 = 0, µ

Lq = λ Wq = λ Wq = 0,

which is not surprising because in such a system nobody waits in line in order to be served by somebody else. Remark: A self-service queueing system is denoted by M/M/∞. Except its intrinsic value as a special system where each customer is his own server, an M/M/∞ system is sometimes used to approximate what happens in an M/M/s queueing system when the number of servers s is very large. 2

Queueing Systems

264

(b) Queueing systems with pressure coefficients. Quite often in real life, fewer customers are tempted to enter a crowded queueing system. On the other hand, the servers tend to speed up the service when more customers come to the queueing system and service tends to deteriorate when there are too many customers. In order to describe what happens in such cases, we have to analyze queueing systems with pressure on arrival or/and pressure on service coefficients. In a queueing system with pressure coefficients, the mean arrival rates and the mean service rates have the forms: λn =

  λ,      

(n = 0, 1, . . . , s − 1);

s n+1

b

λ, (n = s, s + 1, . . .),

    n µ,

(n = 1, 2, . . . , s);

µn =    c  n  s s µ, (n = s + 1, s + 2, . . .), where s is the number of servers. The number b, (0 ≤ b ≤ 1), is called the pressure-on-arrival coefficient. If b = 0, we have a constant mean arrival rate. The number c, (0 ≤ c ≤ 1), is called the pressure-on-service coefficient. If c = 0, we have no pressure on service. Following the general approach outlined in section 2, we have: Cn =

=

λ0 λ1 . . . λn−1 = µ 1 µ 2 . . . µn

 n  1 λ ,      n! µ

(n = 1, 2, . . . , s − 1);

 n   (n−s)(b+c−1)  s λ 1   b+c µ , (n = s, s + 1, . . .).

s! (n!/s!) The probability of having no customer in the queueing system is: "

p0 = 1 +

+∞ X

Cn

#−1

=

n=1

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

=

" s−1 X 1 n=0

n!

λ µ

!n

X s(n−s)(b+c−1) 1 +∞ + s! n=s (n!/s!)b+c

λ µ

!n #−1

.

The other components of the probability distribution of the possible states of the queueing system:  n  1 λ p,   0    n! µ

pn = Cn p0 =  

(n−s)(b+c−1)    1s b+c

s! (n!/s!)

(n = 1, 2, . . . , s − 1);  n

λ µ

p0 , (n = s, s + 1, . . .).

Queueing Systems

265

The expected number of customers in line, waiting to be served is: Lq =

+∞ X

(n − s) pn =

n=s

+∞ X

(n − s) pn =

n=s+1

+∞ X (n − s) s(n−s)(b+c−1) 1 = p0 s! n=s+1 (n!/s!)b+c

λ µ

!n

.

Using the general formula proved in section 2, the expected number of customers in the queueing system is: L = Lq + s −

s−1 X

(s − n) pn .

n=0

The effective mean arrival rate is: λ=

+∞ X

λn p n =

n=0

= λ p0

" s−1 X 1 n=0

n!

λ µ

!n

X s(n−s)(b+c−1) sb +∞ 1 + b+c s! n=s (n!/s!) (n + 1)b

λ µ

!n #

.

The expected waiting time in the system and in line, per individual customer, are given again by Little’s formulas: W =

L , λ

Wq =

Lq . λ

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Remark: In the above formulas there are some infinite series with positive terms which decrease, however, very rapidly, due to the pressure coefficients. 2 Problem 11: A certain barber-shop is a queueing system with pressure coefficients. There are two barbers and four chairs to accommodate waiting customers. When all chairs are taken, new customers are turned away. We know that the mean interarrival time is 15 minutes if both barbers are free and 20 minutes if both barbers are busy but nobody is waiting. The mean service time is 10 minutes when there are two customers in the barber-shop and 8 minutes when there is one customer waiting. 1. Calculate the probability of being served immediately upon arrival. Solution: We have to determine the pressure coefficients for this queueing system. We know that: s = 2,

Capacity = M = 4 + 2 = 6,

Queueing Systems

266

60 60 = 4 customers/hr, λ2 = = 3 customers/hr, 15 20 60 60 = 6 customers/hr,; µ3 = = 7.5 customers/hr. µ2 = 10 8 But, taking the pressure on arrival into account, we have: λ = λ0 =

λn =



s n+1

b

(n ≥ s).

λ,

Therefore, in our case we have: 3 = λ2 =



2 2+1

b

(4),

which implies that the pressure-on-arrival coefficient is: b=

log (3/4) = 0.7095. log (2/3)

Also, due to the pressure on service, we have:     n µ,

(n = 1, 2, . . . , s);

µn =    c   n s s µ, (n = s + 1, s + 2, . . .), according to which, in our case we have: 6 = µ2 = 2µ,

⇒

µ = 3 customers/hr,

 c

3 c 3 2µ = (6), 7.5 = µ3 = 2 2 which implies that the pressure-on-service coefficient is: c=

 

log (7.5/6) = 0.5503. log (3/2)

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Therefore: b + c = 0.7095 + 0.5503 = 1.2598. The probability of having no customer in the barber-shop is: p0 =

" s−1 X 1 n=0

"

n!

λ µ

!n

 1

1 4 = 1+ 1! 3

6 1 X s(n−s)(b+c−1) + s! n=s (n!/s!)b+c

 2

1 4 + 2! 3

1 20.2598 + 2! 31.2598

λ µ

!n #−1

 3

4 3

+

=

Queueing Systems +

1 20.5196 2! 121.2598

267  4

4 3

+

 5

1 20.7794 2! 601.2598

4 3

+

 6 #−1

1 21.0392 2! 3601.2598

4 3

=

= (1 + 1.33 + 0.89 + 0.36 + 0.10 + 0.02 + 0.00)−1 = (3.70)−1 ≈ 0.27. The probability of having only one customer in the barber-shop is: 1 p1 = 1!

λ µ

!1

p0 =

4 3

 

(0.27) ≈ 0.36.

Therefore, the probability that a new customer is served immediately upon arrival is: p0 + p1 ≈ 0.27 + 0.36 ≈ 0.63. 2 2. Calculate the effective mean arrival rate. Solution: Using intermediary numerical results already obtained above, we have: λ = λ p0

" s−1 X 1 n=0

n!

λ µ

!n

X s(n−s)(b+c−1) 1 sb +∞ + b+c s! n=s (n!/s!) (n + 1)b

"

= (4)(0.27) 1 + 1.33 + (0.89) +(0.10)

 0.7095

2 5

 0.7095

2 3

+ (0.02)

+ (0.36)

 0.7095 #

2 6

λ µ

!n #

 0.7095

2 4

=

+

=

= (1.08)(1 + 1.33 + 0.67 + 0.22 + 0.05 + 0.01) = 3.54 customers/hr. 2 3. Calculate the probability of turning away customers. Solution: As the capacity of the barber-shop is 6 customers, the probability of turning away new customers due to the lack of available waiting chairs is: 1 s(6−s) (b+c−1) p6 = C6 p0 = s! [(s + 1)(s + 2)(s + 3)(s + 4)]b+c

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

=

2(4) (0.2598) 1 2! [(3)(4)(5)(6)]1.2598

 6

4 3

λ µ

!6

p0 =

(0.27) = 0.000938 ≈ 0.001. 2

9. Number of arrivals during the time-interval [0, t] In this section we focus only on the arrivals occurring in a queueing system, ignoring service. The interarrival time, which is the time elapsed between two consecutive arrivals, is a positive continuous random variable. The number of arrivals during a given time interval is a discrete random variable taking

Queueing Systems

268

on positive integer values. There is, however, a nice relationship between the probability distributions of these two very different random variables. Proposition 15: In a queueing system with interarrival time exponentially distributed with the constant mean arrival rate λ, the number of arrivals during the time interval [0, t] is Poisson distributed with the parameter λ t. Proof: Let N ([0, t]) be the number of customers arriving in a queueing system during the arbitrary but fixed time interval [0, t]. The interarrival time is assumed to be exponentially distributed with the constant mean arrival rate λ. Obviously, N ([0, t]) is a discrete random variable and its possible values are the nonnegative integers 0, 1, 2, . . .. The amount of time elapsed from the opening of the queueing system (time 0) until the n-th customer arrives is: Sn = T1 + T2 + . . . + Tn , where: T1 is the amount of time elapsed until the first customer arrives, T2 is the interarrival time between the arrival of the first customer and the arrival of the second customer, . . ., Tn is the interarrival time elapsed between the arrival of the (n − 1)-th customer and the arrival of the n-th customer. The independent random variables T1 , T2 , . . . , Tn are exponentially distributed with the parameter λ and, according to Proposition 3 from the fifth section of the third chapter, their sum Sn is Gamma (Erlang) distributed with the parameters α = λ and β = n. Its probability density function is: gn (x) =

λn xn−1 e−λx , (n − 1)!

(x ≥ 0),

and zero if x < 0. Similarly, the amount of time elapsed from the opening of the queueing system until the (n + 1)-th customer arrives is:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Sn+1 = T1 + T2 + . . . + Tn + Tn+1 , where Tn+1 is the intearrival time elapsed between the arrival of the n-th customer and the arrival of the (n+1)-th customer. The random variable Sn+1 is Gamma (Erlang) distributed with the parameters α = λ and β = n + 1. Its probability density function is: λn+1 n −λx x e , gn+1 (x) = n! and zero if x < 0.

(x ≥ 0),

Queueing Systems

269

In order to have exactly n arrivals during the time interval [0, t], the amount of time Sn needed for the n-th customer to arrive has to be smaller than or equal to t, but the amount of time Sn+1 needed for the (n + 1)-th customer to arrive has to exceed t. Thus: P (N ([0, t]) = n) = P (Sn ≤ t, Sn+1 > t). On the other hand: P (Sn ≤ t) = P (Sn ≤ t, Sn+1 ≤ t) + P (Sn ≤ t, Sn+1 > t) = = P (Sn+1 ≤ t) + P (Sn ≤ t, Sn+1 > t), because, as Sn+1 = Sn + Tn+1 and Tn+1 is nonnegative, the event Sn+1 ≤ t implies Sn ≤ t and, therefore, the intersection between the two events Sn ≤ t and Sn+1 ≤ t is equal to the event Sn+1 ≤ t. Therefore, we have: P (Sn ≤ t, Sn+1 > t) = = P (Sn ≤ t) − P (Sn+1 ≤ t) = Gn (t) − Gn+1 (t), where Gn and Gn+1 are the cumulative probability distribution functions of Sn and Sn+1 , respectively. For t > 0, we have: Gn+1 (t) = P (Sn+1 ≤ t) =

Z

t

−∞

gn+1 (x) dx =

λn+1 Z t n −λ x = x e dx. n! 0 Integrating from 0 to t both sides of the equality: d  n −λ x  x e = nxn−1 e−λ x − λ xn e−λ x , dx we get:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

h

it xn e−λx 0

=n

Z

0

t

x

n−1 −λ x

e

dx − λ

Z

t

xn e−λ x dx,

0

from which we have: Z

0

t

n −λ x

x e

tn n Z t n−1 −λ x dx = x e dx − e−λ t . λ 0 λ

Therefore: Z t (λ t)n −λ t λn xn−1 e−λ x dx − e Gn+1 (t) = = (n − 1)! 0 n!

Queueing Systems =

Z

270 n

t

−∞

gn (x) dx −

n

(λt) −λ t (λ t) −λ t e e , = Gn (t) − n! n!

and we obtain:

(λ t)n −λ t e . n! Going back to the probability of having n arrivals during the time interval [0, t], we get: Gn (t) − Gn+1 (t) =

P (N ([0, t]) = n) = P (Sn ≤ t, Sn+1 > t) = Gn (t) − Gn+1 (t) = (λ t)n −λ t e , (n = 0, 1, 2, . . .), n! which is just the Poisson probability distribution with the parameter λ t. =

2

Consequently, the mean and variance of the number of arrivals during the time interval [0, t] are: E[N ([0, t])] = λ t,

V [N ([0, t])] = λ t.

Problem 12: A barber-shop is an M/M/1 queueing system with constant mean arrival rate. The mean interarrival time is 20 minutes. Calculate the probability of having three arrivals in the first 45 minutes after its opening. Solution: We have: 20 1 1 = 20 min = = hr, λ 60 3 which implies λ = 3 customers/hr. According to Proposition 15: P (N ([0, 45 min]) = 3) = P (N ([0,

3 hr]) = 3) = 4

[(3)(3/4)]3 −(3)(3/4) e = 0.20. 2 3! 10. Computer tips Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

=

S. Wolfram’s computer software Mathematica proves to be a powerful and convenient tool in queueing theory. In this section we illustrate how Mathematica may be used for solving the typing pool problem and how to find the optimum number of server in an M/M/s queueing system when operating and waiting costs and aspiration levels are taken into account. Except the Mathematica commands discussed and applied in the fist two chapters, we need here some other commands explained below.

Queueing Systems

271

To define the product function xy, for instance, we give it a name, say f, and we use the command: f[x ,y ]:=x*y If we want to calculate its numerical value at the point (2,3), for instance, we can use either the command: f[2,3] or the command: f[x,y] /. {x− >2,y− >3} obtaining the output: 6. A very useful command is: Sum[g[n],{n,nmin,nmax}] which calculates the sum:

nmax X

g[n]

n=nmin The similar command: NSum[g[n],{n,nmin,Infinity}] gives the numerical approximation of the series: +∞ X

g[n]

n=nmin The command: Table[expression,{n,nmax}] makes a list of the values of the expression, depending on n, with n running from 1 to nmax. Let us mention that the factorial n! has the same notation in Mathematica as in standard number theory, meaning: Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

n!=(1)(2). . . (n−1)(n) Also, Exp[x] is the exponential function ex and N[expression] means the decimal approximation of the given numerical expression. Problem 13: A typing pool of a company has three typists equally qualified (s = 3). Letters to be typed arrive randomly with a mean rate of λ = 15 letters per hour. Each typist can type on average µ = 6 letters per hour. Assume that the typing pool is an M/M/3 queueing system in a steady state

Queueing Systems

272

condition with constant mean arrival rate and constant mean service rate per busy server. Use the computer software Mathematica for calculating: 1. Probability of having no customer in the queueing system: p0 =

" s−1 X sn n=0

#−1

ss ρs ρ + n! s! 1 − ρ n

,

where: ρ =

λ ; sµ

2. Probability that there are n, (1 ≤ n < s), customers in the queueing system: sn pn = p0 ρn , (n = 1, 2, . . . , s − 1); n! 3. Probability that there are n, (n ≥ s), customers in the queueing system: pn =

ss p0 ρn , s!

(n = s, s + 1, . . .);

4. Probability that all servers are busy: P (N ≥ s); 5. Expected number of customers in the line, waiting to be served: Lq =

ss ρs+1 p0 ; s! (1 − ρ)2

6. Expected number of customers in the queueing system, waiting to be served and being served: L = Lq + s −

s−1 X

(s − n) pn ;

n=0

7. Expected waiting time in the queueing system per customer: W =

L ; λ

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

8. Expected waiting time in line per customer: Wq =

Lq ; λ

9. Percentage of idle time per server: I=

X 100 s−1 (s − n) pn ; s n=0

Queueing Systems

273

10. Probability of waiting more than time t in the queueing system per customer: 



Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

1 − e−µ (s−1−λ/µ) t  . P (W > t) = e−µ t 1 + P (N ≥ s) s − 1 − µλ Solution: We use the following names for the functions involved: ‘po’ for p0 ; ‘pnls’ for pn with 1 ≤ n < s; ‘pngs’ for pn with n ≥ s; ‘lq’ for Lq ; ‘l’ for L;‘w’ for W ; ‘wq’ for Wq ; ‘i’ for I; ‘wt’ for W. We start the Mathematica session defining these functions for the arbitrary variables s, r, n, la, mu (for s, ρ, n, λ, and µ, respectively) before calculating their values for the typing pool where s = 3, λ = 15 letters per hour, µ = 6 letters per hour, ρ = (15)/((3)(6))=5/6. After each command we press the key ‘Shift’, keep it down and press the key ‘Enter’ as well. When a new function is defined, there is no output from the computer. The Mathematica session is: In[1]:= po[s ,r ]:=(Sum[(s∧ n)*(r∧ n)/(n!), {n,0,s−1}]+ (s∧ s)*(r∧ s)/((s!)*(1−r)))∧ (−1) In[2]:= pnls[s ,r ,n ]:=(s∧ n)*po[s,r]* (r∧ n)/(n!) In[3]:= pngs[s ,r ,n ]:=(s∧ s)*po[s,r]* (r∧ n)/(s!) In[4]:= lq[s ,r ]:=(s∧ s)*po[s,r]*(r∧ (s+1))/ ((s!)*((1−r)∧ 2)) In[5]:= l[s ,r ]:=lq[s,r]+s−s*po[s,r]− Sum[(s−n)*pnls[s,r,n],{n,1,s−1}] In[6]:= w[s ,r ,la ]:=l[s,r]/la In[7]:= wq[s ,r ,la ]:=lq[s,r]/la In[8]:= i[s ,r ]:= 100*(s*po[s,r]+ Sum[(s−n)*pnls[s,r,n],{n,1,s−1}])/s In[9]:= N[po[3,5/6]] Out[9]= 0.0449438 In[10]:= N[Sum[pngs[3,5/6,n],{n,3,Infinity}]] Out[10]= 0.702247 In[11]:= N[lq[3,5/6]] Out[11]= 3.51124 In[12]:= N[l[3,5/6]] Out[12]= 6.01124 In[13]:= N[w[3,5/6,15]] Out[13]= 0.400749 In[14]:= N[wq[3,5/6,15]] Out[14]= 0.234082 In[15]:= N[i[3,5/6]] Out[15]= 16.6667 In[16]:= wt[s ,r ,la ,mu ,t ]:= Exp[−mu*t]*(1+((s∧ s)/(s!))*po[s,r]* ((r∧ s)/(1−r)) (1−Exp[−mu*(s−1−(la/mu))*t])/ (s−1−(la/mu)))

Queueing Systems

274

In[17]:= N[wt[3,5/6,15,6,0.5]] Out[17]= 0.293246 From the top menu we select ‘File’ and ‘Exit’. The numerical results for the typing pool queueing system are: p0 = 0.0449438, L = 6.01124 letters,

P (N ≥ 3) = 0.702247,

Lq = 3.51124 letters,

W = 0.400749 hours = 24 minutes 2.69 seconds,

Wq = 0.234082 hours = 14 minutes 2.69 seconds,

I = 16.6667%.

P (W > 0.5 hr) = 0.293246. As the nine functions defined in this Mathematica session contain the generic variables ‘s’, ‘r’, and ‘la’ (which mean s, ρ, and λ, respectively), we can use them for any s, any M/M/s queueing system with any constant mean arrival rate λ and any constant mean service rate µ per busy server. Problem 14: In the typing pool of a company letters arrive randomly with a constant mean arrival rate of 15 letters per hour. Taking into account the kind of letters the company sends out to its clients, a typist is qualified for the job if she can type on average 6 letters per hour. The operating cost is $9 per hour per typist. Delays in sending out letters result in losing customers and the waiting cost is estimated to be $30 per hour per letter. If there are s typists, the average total cost per hour is:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

C(s) = C1 s + C2 L(s), where C1 = $9, C2 = $30, L(s) is the expected number of letters in the typing pool, C1 s is the operating cost per hour, and C2 L(s) is the average waiting cost per hour. The operating cost increases and the waiting cost decreases when the number of typists increases. The operating cost decreases and the waiting cost increases when the number of typists decreases. The management would like to know the optimum number of typists needed in the typing pool in order to minimize the total cost C(s). Solution: For an M/M/s queueing system in a steady state condition with constant mean arrival rate λ and constant mean service rate µ, the formulas for the probability of having no customer in the queueing system p0 , the probability pn that there are n, (1 ≤ n < s), customers in the queueing system, the expected number Lq of customers waiting to be served, and the expected number L of customers in the queueing system are listed in Problem 13. When we deal with an M/M/s queueing systems in a steady state condition with constant mean arrival rate λ and constant mean service rate µ but the number of server s is a positive integer variable, then the expected number L

Queueing Systems

275

of customers in the queueing system becomes a function of the variable s and we denote it by L(s). We want to minimize the total cost: C(s) = C1 s + C2 L(s), where C1 and C2 are positive given numbers. If s∗ is the optimum number of servers, then: C(s∗ − 1) ≥ C(s∗ ),

and C(s∗ + 1) ≥ C(s∗ ),

or, equivalently: C1 (s∗ − 1) + C2 L(s∗ − 1) ≥ C1 s∗ + C2 L(s∗ ), and: C1 (s∗ + 1) + C2 L(s∗ + 1) ≥ C1 s∗ + C2 L(s∗ ), These inequalities imply the optimality condition for s∗ , namely: L(s∗ ) − L(s∗ + 1) ≤

C1 ≤ L(s∗ − 1) − L(s∗ ) C2

For solving our typing pool optimization problem, we use Mathematica in order to evaluate L(s) with:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

ρ=

λ 15 = , sµ 6s

for different values of the number s of typists and select the number s∗ for which the above optimality condition is satisfied. We give the following names for the function involved: ‘po’ for p0 ; ‘pnls’ for pn with n = 1, . . . , s − 1; ‘lq’ for Lq ; ‘l’ for L. The Mathematica session is: In[1]:= po[s ,r ]:=(Sum[(s∧ n)*(r∧ n)/(n!), {n,0,s−1}]+ (s∧ s)*(r∧ s)/((s!)*(1−r)))∧ (−1) In[2]:= pnls[s ,r ,n ]:=(s∧ n)*po[s,r]* (r∧ n)/(n!) In[3]:= lq[s ,r ]:=(s∧ s)*po[s,r]*(r∧ (s+1))/ ((s!)*((1−r)∧ 2)) In[4]:= l[s ,r ]:=lq[s,r]+s−s*po[s,r]− Sum[(s−n)*pnls[s,r,n],{n,1,s−1}] In[5]:= Table[N[l[s,15/(s*6)]],{s,7}] Out[5]= {−1.66667, −4.44444, 6.01124, 3.03309, 3.63037, 2.53389, 2.50858} The last output gives the list of the values of L(s) for s = 1, . . . , 7. As L(s) should be nonnegative, the values for s = 1 and s = 2 have to be discarded.

Queueing Systems

276

The table below gives the values of L(s − 1) − L(s): s 1 2 3 4 5 6 7

L(s) L(s − 1) − L(s) − − − − 6.01124 − 3.03309 2.97815 2.63037 0.40272 2.53389 0.09648 2.50858 0.02531

Therefore, we have: L(5) − L(6) = 0.09648
0,

and by taking one or several current nonbasic variables to be positive, instead of having the current value equal to zero, would increase the objective function value z, contradicting the fact that the current solution of the standard primal linear program was supposed to be optimum. Thus: cB B −1 N − cN ≥ 0 and, consequently:

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

y N = cB B −1 N ≥ cN . Finally, we notice that in the standard primal linear program the column of the constraint coefficients corresponding to the i-th slack variable contains 1 in row i and all the other entries are equal to 0. Also, if a slack variable is a nonbasic variable then its component in cN is equal to zero. Therefore, the objective coefficient of the i-th slack variable is just the i-th component of −cB B −1 , therefore the value of −yi . Since the current solution of the standard primal linear program is optimum, each slack variable has a nonpositive objective function coefficient. Thus, for i = 1, . . . , m, yi ≥ 0, which means that y = cB B −1 ≥ 0. The objective function value of the dual linear program is: w = y b = cB B −1 b, and the optimum objective function value of the primal linear program is: z ∗ = cB B −1 b

Appendix

285

because xN = 0. Using the consequences of the weak duality listed in the above remark, we see that the optimum objective function of the dual linear program is: w∗ = cB B −1 b.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Therefore, we have proved the dual theorem, z ∗ = w∗ , showing that the optimum objective function value in the primal linear program is equal to the optimum objective function value in its dual linear program.

References

286

References [1] Bayes, T.R. Phil. Trans. Royal Soc. London 1763, 53, 370-418. [2] Borel, E. C. R. Acad. Sci. Paris 1921, 173, 1304-1308. [3] Bronson, R. Theory and Problems of Operations Research; Schaum’s Outline Series, McGraw-Hill: New York, NY, 1982. [4] Clemen, R.T.; Reilly, T. Making Hard Decisions with Decision Tools; Duxbury Press: Pacific Grove, CA, 2001. [5] Daellenbach, H.G.; George, J.A. Introduction to Operations Research Techniques; Allyn and Bacon: Boston, MA, 1978. [6] Dantzig, G.B. Linear Programming and Extensions; Princeton University Press: Princeton, NJ, 1963. [7] Davis, M. The Art of Decision-Making; Springer-Verlag: New York, NY, 1986. [8] Dresher, M. Games of Strategy. Theory and Applications; Prentice-Hall: Englewood, NJ, 1961. [9] Gillett, B.E. Introduction to Operations Research; A Computer-Oriented Algorithmic Approach; McGraw-Hill, New York, NY, 1976. [10] Guiasu, S. Information Theory with Applications. McGraw-Hill: New York, NY, 1977. [11] Guiasu, S.; Malitza, M. Coalition and Connection in Games; Pergamon Press: Oxford, England, 1980. [12] Guiasu, S.; Shenitzer, A. Math. Intell. 1985, 7, 42-48.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

[13] Hillier, F.S. and Lieberman, G.J. Introduction to Operations Research; Holden-Day: Oakland, CA, 1980. [14] Jaynes, E.T. In The Maximum Entropy Formalism; Levin, R.; Tribus, M; MIT Press: Cambridge, MA, 1979, pp.15-118. [15] Jeffrey, R.C. The Logic of Decision; McGraw-Hill: New York, NY, 1965. [16] Jensen, P.A.; Bard, J.F. Operations Research. Models and Methods; Wiley: Hoboken, NJ, 2003. [17] Kvanli, A.H. Statistics. A Computer Integrated Approach. West Publishing Company: St. Paul, MN, 1988.

References

287

[18] Laplace, P.S. Th´eorie analytique des probabilit´es; Courcier: Paris, France, 1812. [19] Laplace, P.S. Essai philosophique sur les probabilit´es; Courcier: Paris, France, 1814. [20] Luce, R.D.; Raifa, H. Games and Decisions; Wiley: New York, NY, 1957. [21] McKinsey, J.C.C. Introduction to the Theory of Games; McGraw-Hill: New York, NY, 1952. [22] Pearson, K. Phil. Mag. (5th Series), 1900, 50, 157-175. [23] Rapoport, A. Two-Person Game Theory; University of Michigan Press: Ann Arbor, MI, 1966. [24] Rapoport, A. N-Person Game Theory. Concepts and Applications; University of Michigan Press: Ann Arbor, MI, 1970. [25] Ryan, B.F.; Joiner, B.L.; Ryan Jr., T.A. Minitab; Duxbury Press: Boston, MA, 1985. [26] Shafer, G. A Mathematical Theory of Evidence; Princeton University Press: Princeton, NJ, 1976. [27] Shannon, C.E. Bell Syst. Tech. J. 1948, 27, 379-423; 623-656. [28] Shapley, I.S. In Contributions to the Theory of Games II; Kuhn, H.W.; Tucker, A.W.; Annals of Mathematics Studies 28; Princeton University Press: Princeton, NJ, 1953; pp.307-317. [29] Taha, H.A. Operations Research. An Introduction; Macmillan: New York, NY, 1992. [30] Tankard Jr., J.W. The Statistical Pioneers; Schenkman: Cambridge, MA, 1984.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

[31] Taylor III, B.W. Introduction to Management Science; Prentice Hall: Upper Saddle River, NJ, 2002. [32] Ullman, J.E. Theory and Problems of Quantitative Methods in Management; Schaum’s Outline Series; McGraw-Hill: New York, NY, 1976. [33] von Neumann, J. Math. Ann. 1928, 100, 295-320. [34] von Neumann, J.; Morgenstern, O. Theory of Games and Economic Behaviour; Wiley: New York, NY, 1944.

References

288

[35] Wagner, H.M. Principles of Operations Research; Prentice-Hall: Englewood Cliffs, NJ, 1975. [36] Wilkes, M. Operational Research: Analysis and Applications; McGrawHill: Maidenhead, Berkshire, England, 1989. [37] Winston, W.L. Operations Research. Applications and Algorithms; Duxbury Press: Belmont, CA, 1994.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

[38] Wolfram, S. Mathematica. A System of Doing Mathematics by Computer; Addison-Wesley Publishing: Redwood City, CA, 1991.

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Index Approximating the optimum solution of a linear program 178 Area of a circle 159 Assumptions in queueing theory 220 Auxiliary experiment 77, 84 Balance equations 223 Balking 224 Barber-shop problem 228 Bard J.F. 286 Bayes T.R. 86, 286 Bayes’s criterion with experimentation 94 Bayes’s criterion without experimentation 82, 103 Bayes’s formula 85 Bayes’s risk 91, 109 Better estimator 166 Binary 0-1 random variables 193 Birth-and-death stochastic process 220 Boltzmann H-function 145 Borel E. 4, 286 Bounds for reliability 200 Bronson R. 286 Brown G.W. 33 Brown’s algorithm 33 Capacity 219 Central limit theorem 136, 150 Chance node 109 Characteristic function 63 Chevalier de M´er´e 3 Chi-square goodness of fit test 129, 149 Clemen R.T. 286 Coalition 63 Comte de Buffon G.-L. Leclerc 117 Confidence interval 135, 136, 137, 160, 166 Constant failure rate 205

Convexity 30 Cooperative solution of the game 63 Core 65 Cost of the auxiliary experiment 85 Daellenbach H.G. 286 Dantzig G.B. 46, 286 Davis M. 286 Decision function 90 Decision maker 73 Decision matrix 73 Decision node 109 Decision tableau 73 Decision tree 110 Decision under certainty 74 Decision under uncertainty 74 Decision with experimentation 74, 84, 104 Decision without experimentation 74, 77, 82, 101 Definite multiple integrals 164 Definite simple integral 168 Dominated strategy 6 Domination 64 Dresher M. 286 Drug game 69 Dual linear program 281 Dual theorem 46, 281 Equilibrium point 9, 15, 60 Erlang distribution 155 Exact reliability 193 Excel 54 Expected loss 82, 105 Expected loss conditioned by an outcome 86, 106 Expected loss corresponding to perfect information 83, 104 Expected number of customers in line 222 Expected number of customers in 289

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Index the queueing system 222 Expected payoff 14 Expected posterior loss 89, 106 Expected prior loss 87, 104 Failure rate depending on time 208 Feasibility 65 Fermat P. de 4 Fermat’s last theorem 173 Fermat’s quintic equation 174 Fermi E. 117 FIFO 217 Finite input source 257 Finite queue 246 Fortran 94 Gamma distribution 154 Gamma function 154 George J.A. 286 Gillett B.E. 94, 286 Gosset W.S. 136 Graphical method 18 Group rationality 64 Guiasu S. 286 Hillier F.S. 72, 286 Hurwicz’s criterion 80, 103 Improvement 65 Imputation 64 Individual rationality 64 Input source 217 Instantaneous failure rate 203 Jaynes E.T. 286 Jeffrey R.C. 286 Jensen P.A. 286 Joiner B.L. 287 k-out-of-n system 192 Kvanli A.H. 286 Laplace P.S. 80, 287 Laplace’s criterion 73, 81, 103 Lieberman G.J. 74, 286 Lifetime 203 Lindo 47 Linear programs induced by a twoperson zero-sum game 37

290 Little’s formulas 221, 222 Loss function 73 Loss matrix 73, 77 Lower bound for reliability 202 Lower value of a game 8, 14, 31 Luce R.D. 287 Malitza M. 286 Mathematica 58, 99, 270, 275, 277 Maximin strategy 10 McKinsey J.C.C. 287 Method 1 for calculating the exact reliability 194, 196 Method 2 for calculating the exact reliability 194, 197 Method 3 for calculating the exact reliability 195, 198 Method 4 for calculating the exact reliability 195, 199 Minimal cut 194, 196 Minimal path 194, 196 Minimax strategy 10 Minimax criterion 77, 102 Minimax theorem 18, 46 Minitab 119 Mixed strategies 14 M/M/1 queueing system 225 M/M/s queueing system 230 Modeling based on simulation 132 Moment generating function 155 Monte Carlo method 159 Morgenstern O. 4, 287 Multiple-stage decision 109 n-person games 63 Nature 73 Network 193 Noncooperative solution of the game 62 Nonlinear optimization 187 Normal lifetime 211 Normed linear program 184 Number of arrivals during the time interval [0,t] 267

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Index Oil-drilling example 74 Operating cost 216, 274 Optimum number of servers 274, 276 p-value 153 Parallel system 191, 214 Pascal B. 3, 4 Payoff 5 Pearson K. 287 Polynomial equations 176 Prediction matrix 84 Pressure coefficients 264 Price R. 86 Primal linear program 281 Principle of insufficient reason 80 Prior probability distribution 84 Prisoner’s dilemma 60 Pure strategy 5 Queue 217 Queue discipline 217 Queueing system 217 r × 2 two-person zero-sum game 29 Raiffa H. 287 Random correspondence 86 Random numbers 125 Rapoport A. 287 Rate diagram 222, 225, 230, 246 Rate in = rate out principle 223 Reilly T. 286 Regret matrix 78 Reliability 190 Repair shop problem 261 Retrodiction matrix 85 Reward 64 Riemann B. 164 Risk function 90, 107 Risk matrix 90, 108 Robinson J. 33 Rolling a fair die 127 Ryan B.F. 287 Ryan Jr. T.A. 287 Saddle-point 9, 15, 60 Sampling from a finite discrete prob-

291 ability distributions 128 Sampling from continuous probability distributions 142 Sampling from the continuous uniform distribution 142 Sampling from the Erlang distribution 154 Sampling from the exponential distribution 143 Sampling from the normal distribution 149 Sampling from the Poisson distribution 140 Sampling from the standard normal distribution 146 Sas-or 51 Savage’s minimax regret criterion 73, 78, 102 Self-service queueing system 262 Series system 191, 214 Service 217 Shannon C.E. 287 Shannon’s entropy 145, 153 Shapley I.S. 287 Shapley value 67 Shafer G. 287 Shenitzer A. 286 Simple estimator 165 Simulation in queueing theory 278 Solution of a game 5 Solving equations 173 State of the queueing system 218 Steady state condition 220 Stochastic matrix 85 Structure function 190 t-distribution 136 Taha H.A. 110, 287 Tankard Jr. J.W. 287 Taylor III B.W. 287 Tolerance 35 Tossing a fair coin 127 Two-finger mora game 4, 6, 13, 23

Index

Copyright © 2009. Nova Science Publishers, Incorporated. All rights reserved.

Two-person constant-sum game 61 Two-person zero-sum game 5 Typing pool problem 232 Ulam S. 18, 117 Ullmann J.E. 287 Union versus management problem 12 Upper bound for reliability 201 Upper value of a game 8, 14, 31 Utility 5 Value of the auxiliary experiment 87 Value of perfect information 83 von Neumann J. 4, 18, 46, 117, 287 von Neumann-Morgenstern solution of the game 65 Wagner H.M. 288 Waiting cost 216, 274 Waiting time in the queue in M/M/1 237 Waiting time in the queue in M/M/s 241 Waiting time in the system in M/M/1 234 Waiting time in the system in M/M/s 242 Weak duality 282 Weibull distribution 209 Wilkes M. 288 Winston W.L. 69, 288 Wolfram S. 58, 99, 270, 288 2×2 two-person zero-sum game 19 2×s two-person zero-sum game 25

292