Linear Inequalities and Related Systems. (AM-38), Volume 38 9781400881987

Table of contents :
CONTENTS
Preface
1. Dual Systems of Homogeneous Linear Relations
2. Polyhedral Convex Cones
3. Resolution and Separation Theorems for Polyhedral Convex Sets
4. Theory of Linear Programming
5. On Systems of Linear Inequalities
6. Infinite Programs
7. A Primal-Dual Algorithm for Linear Programs
8. Marginal Values of Matrix Games and Linear Programs
9. Determinateness of Polyhedral Games
10. On Systems of Distinct Representatives
11. Dilworth’s Theorem on Partially Ordered Sets
12. On the Max-Flow Min-Cut Theorem of Networks
13. Integral Boundary Points of Convex Polyhedra
14. An Extension of a Theorem of Dantzig
15. Neighboring Vertices on a Convex Polyhedron
16. On a Theorem of Wald
17. On the Solution of a Game-Theoretic Problem
18. The Closed Linear Model of Production
Bibliography

Citation preview

Annals of Mathematics Studies Number 38

ANNALS OF MATHEMATICS STUDIES Edited by Emil Artin and Marston Morse 1. Algebraic Theory of Numbers, by H e r m a n n W e y l 3. Consistency of the Continuum Hypothesis, by K u r t G o d e l 6. The Calculi of Lambda-Conversion, by A l o n z o C h u r c h 7 . Finite Dimensional Vector Spaces, by P a u l R . H a l m o s 10. Topics in Topology, by S o l o m o n L e f s c h e t z 11. Introduction to Nonlinear Mechanics, by N. K r y l o f f and N. B o g o l i u b o f f 15. Topological Methods in the Theory of Functions of a Complex Variable, by M a r s t o n M o r s e 16. Transcendental Numbers, by C a r l L u d w i g S i e g e l 17. Probleme General de la Stabilite du Mouvement, by M. A. L i a p o u n o f f 19. Fourier Transforms, by S. B o c h n e r and K. C h a n d r a s e k h a r a n 20. Contributions to the Theory of Nonlinear Oscillations, Vol. I, edited by S. L e f s c h e t z 21. Functional Operators, Vol. I, by J o h n v o n N e u m a n n 2 2 . Functional Operators, Vol. II, by J o h n v o n N e u m a n n 2 3 . Existence Theorems in Partial Differential Equations, by D o r o t h y L. B e r n s t e in 24.

25. 26. 27. 28.

29. 30. 31. 32.

33. 34. 35. 36. 37. 38.

Contributions to the Theory of Games, Vol. I, edited by H. W . K u h n and A. W . T u c k e r Contributions to Fourier Analysis, edited by A. Z y g m u n d , W . T r a n s u e , M. M o r s e , A. P. C a l d e r o n , and S. B o c h n e r A Theory of Cross-Spaces, by R o b e r t S c h a t t e n Isoperimetric Inequalities in Mathematical Physics, by G. P o l y a and G. S z e g o Contributions to the Theory of Games, Vol. II, edited by H. K u h n and A. W . T u c k e r Contributions to the Theory of Nonlinear Oscillations, Vol. II, edited by S. L e f s c h e t z Contributions to the Theory of Riemann Surfaces, edited by L. A h l f o r s et al. Order-Preserving Maps and Integration Processes, by E d w a r d J. M c S h a n e Curvature and Betti Numbers, by K. Y a n o and S. B o c h n e r Contributions to the Theory of Partial Differential Equations, edited by L. B e r s , S. B o c h n e r , and F. J o h n Automata Studies, edited by C. E. S h a n n o n and J. M c C a r t h y Surface Area, by L a m b e r t o C e s a r i Contributions to the Theory of Nonlinear Oscillations, Vol. Ill, edited by S. L e f s c h e t z Lectures on the Theory of Games, by H a r o l d W . K u h n . In press Linear Inequalities and Related Systems, edited by H. W . K u h n and A. W. T ucker

39. Contributions to the Theory of Games, Vol. I ll, edited by M. and A. W . T u c k e r . In p r e s s

D

r e sh e r

LINEAR INEQUALITIES AND RELATED SYSTEMS G.

B . D A N T Z IG I.

R. J . D U F F IN

H ELLER

J . B. K R U S K A L

K . FAN

H. W . KUHN

L . R . FO R D , J R .

H . D. M I L L S

D. R . F U L K E R S O N

G. L . T H O M P S O N

D. G A L E

C. B. T O M P K IN S

A. J . GOLDM AN

A. W . TU CKER

P. W O LFE

Edited by H. W. Kuhn and A. W. Tucker

Princeton, New Jersey Princeton U niversity Press 1956

Copyright © 195 6 , by Princeton University Press London: Geoffrey Cumberlege, Oxford University Press All Rights Reserved L. C. Card 56-8385

This research was supported in part by the Office of Naval Researbh. Reproduc­ tion, translation, publication, use and disposal in whole or in part by or for the United States Government is permitted. Papers 1, 11, and 12 are published by per­ mission of The RAND Corporation.

Printed in the United States of America

PREFACE The eighteen papers collected, here explore various aspects of one mathematical theme, the theory of linear inequalities. Although they are related, by this fact, the papers are bound together more closely by the areas of intended application. Without exception, the direction or tech­ nique of each is determined by recent developments in the subjects of linear programming, matrix games, and related or derivative economic models. In their rapid growth during the last decade, these disciplines have not only posed questions of direct economic importance, but have also suggested peripheral mathematical problems of interest in themselves and have stimu­ lated a thorough reconsideration of several mathematical topics. Follow­ ing these lines of development, the papers of this volume fall roughly into three categories. Papers of the first type present a detailed exposition of the fundamental mathematical results which form a basis for the models. Such an inventory is peculiarly necessary in fields such as these, where the pressure for practical results is so great that there is little time for reflective appraisal of past accomplishment. The second type of paper answers purely mathematical questions that have appeared in the elaboration of the economic theory, or else exploits results such as the duality theo­ rem of linear programming for independent,mathematical purposes. The third type considers problems that bear on the economic applications of the models. The first five papers of this study are in the first category, and offer expository treatments of those facets of the theory of linear in­ equalities that have been highlighted by the needs of linear programming and the theory of matrix games. PAPER 1 studies pairs of finite systems of homogeneous linear inequalities (written uniformly > o) and/or equations involving variables that are nonnegative and/or unrestricted. To each in­ equality or equation in one system there corresponds a nonnegative or un­ restricted variable in the other, and conversely; the array of coefficients in one system is the negative transpose ofthe array In the other. This formal tabular duality furnishes the foundation for the duality of matrix games anc. linear programming. Various pairs of dual systems give rise to v

PREFACE sharpened "transposition theorems" of classical type and to a new property of "complementary slackness." In particular, it is shown that a self-dual system KW ^ o, W ^ o, where K is skew-symmetric, possesses a solution * * * W such that KW + W > 0 (in each component). Since I > i kijw j = J for all W, it follows that either both, for each i.

* w^ > o

0

or

*#■

> °>

nQ^

PAPER 2 deals with the set A* of solutions of a finite system AX b. The value of the program is M = inf (a, x) for such x. The dual to a program (A, b, a) consists of the triple (- A*, - a, - b), where A* is the transformation adjoint to A. The value of the dual is denoted by M 1. vii

PREFACE For a finite program there are essentiallythree situations to be distinguished; a program, can only be inconsistent, consistent with in­ finite value, or consistent with a finite value which is necessarily achieved. Of these three exhaustive and mutually exclusive possibilities, the first is dual to itself or to the second, while the third is self-dual. To discuss the analogous situation for infinite programs, a program is de­ fined to be sub-consistent if there exist sequences xR > o and qn > o such that lim(Axn - qn ) = b; the totality of such sequences (x ) de­ fines a sub-value for the program., m ! = inf lim. (a, xn ). Then a program, is consistent and has a finite value M if and only if the dual program is sub-consistent and has the finite sub-value m.1; under these conditions, M + m f = o. A program (A, b, a) is said to be super-consistent if there is an x such that x ^ 0 and Ax > b. Then a program is consistent with a feasible vector x achieving the finite value (a, x) = M if the dual program, is super-consistent with finite value; under these conditions, m = M = - M 1 = - m 1. Methods designed for special classes of linear programs have dominated recent work on computational routines. One phase of this develop­ ment has been the construction of a special algorithm for the transportation problem, by Ford and Fulkerson; partial evidence indicates that their method is quite efficient. PAPER 7 generalizes their process to solve the general linear programming problem,. The distinctive feature of the routine is the use of a feasible solution to the dual program to define an alteration of the original program (called the "restricted primal") in which several of the variables are dropped, and the objective is changed to that of finding a solution as nearly feasible as possible. Optimal solutions of the "restricted primal" are then used to improve the feasible solution to the dual program. After a finite number of repetitions, the optimal solutions of the "restricted primal" coincide with those of the unaltered primal program. The particular efficacy of this algorithm, for the transportation problem is due to the fact that the optimization of the "restricted primal" program can be performed without recourse to the Simplex Method. Other­ wise, the procedure can be considered as a promising variant of the Simplex Method. PAPER 8 investigates the rate of change of the value (i.e., the "marginal" value) of a matrix game, or the rate of change of the optimum value (i.e., the "marginal" optimum value) of the objective function of a linear program, as the payoffs of the game or the coefficients of the pro­ gram are varied. This study promises practical application whenever these parameters can be controlled or altered since it indicates which changes will have a beneficial effect on the value. The determination of the marginal value of a matrix game is straightforward and uncomplicated by viii

PREFACE existence questions. Let v(G) denote the value of the matrix game G, and let H he a matrix of the same size as G. The marginal value of G with respect to H is defined as the one-sided derivative dv(G) = m

lim v(G+aH) - v(G) ► o+ a

a —

It is shown that this marginal value is equal to the value of the matrix game H when the players are constrained to play in H from among the optimal strategies for G. The search for the marginal value of a linear program is handled simultaneously for a pair of dual programs by combining them into a single saddlepoint problem. Consider the function m 0 (X ,

n

y, A) = aoQ + l X a l0 + 1=1 j=i

defined by the (m+1 ) by

+

I xiaif j i,j

(n+1 ) matrix

where i = 1, ..., m and j = 1, ..., n, x = (x^) > 0 and y = (y.) > 0. Any pair (x°, y ° ) such that 0 (x, y°, A) g 0 (x°, y°, A) g 0 (x°, y, A) for all x > 0 and y > 0 is called a saddlepoint for 0; such pairs are exactly the solutions to the dual linear programs: (1) maximize aQ0 + Ej_x i_aj_0 subject to aQj + >0, xj_ ^ °> all i and j, and (2) minimize a00 + z^a^y^ subject to aiQ + 2ja1 .y., y . > o, for all i and j. The number 0(A) = 0 (x°, y°, A) is the common value of the programs if either program has a solution. Let H be a matrix of the same size as A; then the marginal value of the dual programs is de­ fined as the one-sided derivative d0 (A)

_ 5 lr " " „ ^

0 (A+aH) - 0(A) o* ------ ^------ •

if this limit exists. It is shown that, if 0 (A + aH) exists for some interval o and v2 = + °o. If v 1 and v2 are finite (and hence equal) the common value is attained for suitable x € X and y e Y. The origins of linear programming, in the work of G. B. Dantzig and Marshall Wood on program planning for the U. S. Air Forces and in the research of Hitchcock and Koopmans on static models of transportation, are innately practical in character. Supported by the simultaneous develop­ ment of efficient computational techniques, linear programming has since found many areas of application as a concrete approach to problems of allocating limited resources, and has never lost its orientation toward practice. The practical nature of the subject is underlined by the fact that, almost by definition, the theoretical aspects are somewhat restricted. It is not entirely inaccurate to assert that the mathematical foundations of the subject are no more extensive than the theory of finite systems of linear inequalities. Thus, in view of its strictly utilitarian origins on the one hand, and its modest mathematical extent on the other, the next group of papers presents a development of the subject that is both unex­ pected and gratifying. The authors investigate various instances of a new method, that employs the results and techniques of linear programming to treat certain combinatorial problems.1 The problems dealt with in these papers have one feature in common: they ask either for an extreme value of an integral valued func­ tion defined on a finite set or for a member of a finite setthat can be characterized by such an extremal property. In discussing the problems, it is suggestive to call the elements of the set "trials" and the func­ tion an "objective function." The first step of the method is the ex­ pression of the trials as points of an Euclidean space in such a manner that the objective function can be presented as a linear form. Clearly, this can always be done in many ways; the mode of expressionin each case is suggested by the original setting of the problem and by conditions that

The use of linear inequalities (precisely, the theorem of Farkas cited in the discussion of Paper 2 above) in graph theory seems to have been introduced by R. Rado (Ann. Math. (2) (1 9 4 3 ), 268-270).

x

PREFACE must be met in the later stages of the method. Once the problem has been given its geometric formulation, it is not changed essentially if the ob­ jective function is examined on the convex hull of the set of trial points, since the extreme values are achieved at extreme points of this polyhedron and these are a subset of the trial points. In some cases, the later stages of the method are simplified by using an even larger convex polyhedron, con­ structed so that every face on which the form assumes its extreme value con­ tains a trial point. The problem is now a linear program since it asks for an extreme value of a linear form on a convex polyhedron. This may be true only in a non-effective sense if the polyhedron is presented in terms of its extreme points rather than by linear inequalities. To express the program explicit­ ly, the equations of the faces of the convex polyhedron must be found. This is a difficult step for most polyhedra; it has been possible for the cases considered here because they have been recognized as programs of the transportation type. Once the program has been stated explicitly, it is a routine matter to write out the dual program. The dual program presents itself explicitly as the problem of finding an extreme value of a linear form subject to linear equations and inequalities. The aim of the original formulation of the trials and objective function as points and as a linear form., respectively, can now be explained. It is to make the candidates for a solution (whether they are the extreme points of the constraint set or merely points in the faces on which the form assumes its extreme value) have all integral coordinates, and thus admit a combinatorial interpreta­ tion. If this is possible, then the dual program yields a combinatorial problem, that is equivalent, according to the duality theorem of linear programming, to the original combinatorial problem.. The steps in the method just outlined can be illustrated by the solution of the classical problem of choosing a system of distinct repre­ sentatives from a family of sets. To pose the problem., let {S^ ..., Sn ) be a family of subsets of the set S = {a^ ..., am ), where m > n. A set of n distinct elements a. , ..., a. is said to be a system of dis1 n tinct representatives (abbreviated S.D.R.) if a. e S. for j = 1, ..., n. j J Thus, a trial for this problem is quite naturally an assignment of n dis­ tinct elements to the sets ^ j ^ ’ ^he objective function is defined to count the number of elements in a trial that are in the sets to which they are assigned, the existence of an S.D.R. is reduced to the question of whether the maximum of this function is n. Each trial can be expressed as a point of an mn-dimensional space by forming the m by n matrix X = (x^.) with a 1 in row i and column xi

PREFACE j if is assigned to Sj and a zero, otherwise. The trials are thus identified with the m by n matrices formed of zeros with the exception of a single 1 in each column, and no more than a single 1 in each row. If the membership of the elements in the sets is also expressed by the m by n matrix

otherwise then the objective function is given by the linear form

The convex hull of the trial points is easily calculated to be the set of all X = (x^j ) such that: (1)

(i = 1,

.,m; j = 1, .•.,n)

(2 )

(j = 1,••.,n)

(3 )

(i =

1 ,...,m).

(This assertion is easily reduced to the theorem of D. Konig that the con­ vex hull of the n by n permutation matrices is exactly the doubly stochastic matrices; this is a somewhat uncomfortable fact since the same theorem will also solve the problem of choosing an S.D.R. However, this does not vitiate the problem as an illustration of the method.) To simplify the treatment of the dual program, it is convenient to enlarge this poly­ hedron, replacing (2 ) by

(2 ')

(j = 1, .. .,n).

The dual program then asks for the minimum of + zjvj where u^ > o and vj > 0 are chosen so that u^ + v . > a ^ . for all i and j. Trivially, all of the u^ and v . can be chosen to be 0 or 1 , and the dual combinatorial problem is: Choose the smallest total number of elements a^ and sets S- such that, if a. e S., then a. or S. J J J is chosen.

PREFACE By the duality theorem of linear programming, this minimum is equal to the maximum number of distinct representatives in a trial. If there is no S.D.R and hence both maximum and minimum are less than n, then the k sets not chosen in the solution to the dual program must contain fewer than k distinct elements among them. Since the converse is trivial, this proves the theorem of P. Hall: There exists an S.D.R if and only if every k of the sets of the family contain at least k distinct elements. In PAPER 1o, this method is applied to prove the following gen­ eralization of Hall’s Theorem: Let {S.j, ..., Sn ) be a family of subsets of a given set S. Let there be given a finite partition of S into disjoint subsets Tk with associated integers 0 ^ ck ^ ^or k = 1> •••> P* The cardinality of a set A is denoted by A. In order that there ex­ ist a subset R of S that is an S.D.R. with ck = ^ ^ = ^k ^or and sufficient that

^ = 1 > •••> P>

it is necessary

and

for all subsets

AC

{1, ..., n]

and

B C {1

P) •

The methods used in this paper parallel the treatment of Hall’s Theorem given above. The problem of PAPER 11 asks for the smallest number of disjoint chains contained in a finite partially ordered set P such that every element of the set belongs to one of the chains. Thus, a trial is a de­ composition of P into disjoint chains and theobjective function (to be minimized) counts the number of chains In a trial. Suppose P contains n elements, denoted by 1,2, ..., n. A trial can be realized as an (n+1 ) by (n+1 ) matrix X = (x^ .) of zeros andones as follows: x 0j = 1 x^Q = 1 xQ 0 = n x^j - 1

if j is the initial element of a chain; if i is the terminal element of a chain; k if k is the number of chains; if i immediately precedes j In a chain; xiil

PREFACE where i, j = o, 1, 2, ..., n and all other x ^ - = 0. The objective func­ tion (to be maximized) can then be taken to be the linear form C . X = Xcij^j

where C = (c^) is defined by c00 * 1 and c = 0 otherwise. If does not immediately precede j in P, the definition of c^. is immaterial since x^ . = 0 for such pairs in all trials.

i

To convert this problem into one of transportation type, the authors use the convex polyhedron composed of all X = (x^ •) which are nonnegative and satisfy n

n

X x oj = Z x±o ■ n j=o 1=0 n Z xij = 1 j=o n

(i = 1,..•,n)

X xij = i=o

(j = 1,••-,n)

1

and redefine c^. = - 00 for those i not preceding j. The extreme points of the polyhedron which have x^j = 0 for c^ . = - «> have an obvious in­ terpretation as trials. The dual to this linear program asks in a natural way for the largest number of mutually incomparable elements of P. There­ fore, the duality theorem asserts the equality of this number with the smallest number of chains in a trial decomposition and proves the finite case of a theorem, of Dilworth. The problem discussed in PAPER 12 arose in the study of trans­ portation networks and, although it is not as clearly combinatorial as the situations of Papers 10 and 11, it yields to a similar attack. Consider a network connecting two nodes by way of a number of intermediate arcs and nodes, and suppose that these can handle certain limited amounts of traffic per ‘unit time. Assuming a steady state condition, find a maximal flow of traffic from one given node (the source) to the other (the sink). When this problem is formulated as a linear program, the dual program asks for a "cut" in the network (i.e., a collection of nodes and arcs that meets every chain from source to sink) with minimum total capacity. A simple device permits the derivation of the following theorem of Menger as a corollary: The maximum number of pairwise node-disjoint chains joining xiv

PREFACE two given disjoint sets of nodes of a linear graph is equal to the minimum number of nodes necessary to separate the sets. In applying the method of Papers 10 , 1 1 , and 12 to combinatorial problems, the most difficult step is the formulation of a linear program equivalent to the original situation. Although almost any choice of vari­ ables suggests natural linear constraints, the difficulty consists in showing that these insure solutions among the original integral trials. For exsunple, in the problem of systems of distinct representatives discuss­ ed above, the natural constraints are given by (1), (2), and (3). It is clear _:hat the trials are among the extreme points of the polyhedron de­ fined by these relations, but this does not exclude the existence of ex­ treme points other than the trials. However, if it were known that every extreme solution of this system had all integral coordinates, then every extreme solution must be a trial. This follows directly from the fact that the only integral values for the are 0 or 1, and thus (1 ), (2), and (3), when applied to integral values, parallel the definition of a trial exactly. In varying degrees, the same statement is true of each of the problems in Papers 10 , 11, and 12; namely, if it were known that the only extreme solutions of the relations used in the linear program formulations had all integral coordinates, then the appropriateness of those formula­ tions could be established directly. PAPER 13 was written to fill this need; it seeks to characterize a wide class of linear programs which have integral solutions irrespective of their objective functions. In geometric terms, let P be a convex polyhedron defined as the set of all n-vectors x, such that b Ax Gordan [5] and Stiemke [9], the Transposition Theorem of Motzkin [7], and the Theorem of the Alternative for Matrices of von Neumann and Morgenstern [8 ] (numbers in square brackets refer to the bibliography at the end of the paper). But, in part, it is quite new information about a formal tabular property of "complementary slackness" which pertains to dual systems of wide generality (see Theorem 6 ) and to self-dual systems (see Theorem 7 )* The systems of homogeneous linear relations to be studied con­ sist of homogeneous linear inequalities, written uniformly ^ 0, and possibly also linear equations. Specifically, we deal with the following systems, four pairs of "dual" systems and one "self-dual" system: 1.

A^U ^

0,

2.

ATU ^

0,

3. 4

.

5.

V > 0,

BTU =

0,

and

AX >

and

AX+BX =

CTV > 0,

and

V ^ 0, ATU+CTV > 0, BTU+DTV = 0 KW

> 0,

and W

0, 0,

-CX > 0 ,

X >

0.

X >

0.

X > 0.

-AX-BX = 0, -CX-BX > 0, X > 0.

> 0,

(KT = -K).

The letters A, B, C, D, K denote matrices (rectangular in general, ex­ cept K which is square) and U, V, W, X, Y denote vectors (treated as one-column*matrices)• The superscript T indicates transposition; each This paper was written for the Office of Naval Research Logistics Project in the Department of Mathematics at Princeton University. 3

TUCKER

k

vector inequality (> 0) holds for every component individually. It will be noticed that the first three pairs of dual systems can be regarded as special cases of the fourth: take B, C, D vacuous and change -AX = 0 into AX = 0 to get the first pair; take C, D vacuous and change -AX - BY = 0 into AX + BY = 0 to get the second pair; and take A , B, D vacuous to get the third pair. The self-dual system (No. 5), which has special importance in the Theory of Linear Programming (see Paper k in this Study), arises from the third pair of dual systems by taking C skewT symmetric and setting K = C . To explain the sense in which the term dual is here being used, we exhibit the left and right systems of the general case (No. h, above) in the following greater detail:

(uh

unrestricted)

(h = 1,...,p)

(j =

(yk

a

xj £ 0 unrestricted)

•

_2jcijxj " ^ i k ^ k = 0

** •

> 0 V n j uh + + 2idikvi = 0 ^hk^h

= 0

II

v± > °

"zjahjxj ' V W k

...,n)

(k = 1,...,q)

It will be noticed (1) that there is a one-to-one correspondence between un­ restricted variables in one system and equations in the other, indexed by h and k, and betweeji nonnegative variables in one system and "fully coeffic iented" inequalities in the other, indexed by i and j, and (2) that the arrays of coefficients at lower left and upper right are such that the array in one system is the negative transpose of the array in the other. This pre­ scription enables one to pass by a well-defined procedure from a given system of homogeneous linear inequalities and/or equations involving nonnegative and/or unrestricted variables to a second such system. The procedure is re­ versible, as duality should be: if system a has system p as dual, then p has a as dual. The following small-scale example serves to illustrate a general pair of dual systems (No. 4 , above).

DUAL SYSTEMS OF RELATIONS

5

REMARK. The starred inequalities in this example exhibit the propertyof "complementary slackness" that will be established in Theorem 6 for a.general pair of dual systems. For reference at that time, we ob­ serve that

(u.| j v ^ V g , ^ ) = ( - 1; 1 , 0 , 1 )

(x 1,xz, X y X ^ ;

and

= ( 2 , 0 , 0 , 1; - 2 , 1 )

are solutions of the left and right systems which satisfy the starred In­ equalities as strict inequalities (> o) and the remaining relations as equations (= o). *

ATU > c

*

*

*

*

*

*

We begin with a lemma concerning the pair of dual systems and AX - o, X > 0, where A = [A1, •••, Aq]

is an n-columned matrix with arbitrary real entries (but we could work equally well in the rational field or in any ordered field). We use an argument adapted from an unpublished proof by David Gale of the Fundamental Theorem of Hermann Weyl [1 3 ] that the convex hull of finitely many halflines is the intersection of finitely many halfspaces. LEMMA.

The dual systems ATU > 0

possess solutions

and AX = 0, X > 0

U

and

X

such that

A^U + x1 >0. PROOF. We proceed by induction on n,the numberof columns in A. The initial case n = 1is trivial: if A 1 = 0, take U = 0 and x 1 = 1; if A 1 4 0 take U = A^ and x 1 = 0. We now assume that the Lemma holds for amatrix umns and proceed to prove it for a matrix

A

of n col­

A = [A, An+1) = [A 1, ..., An , An+1 ] of

n + 1

columns-

A,

we get

A^U +

X 1 > 0.

Applying the Lemma to

ATU >

0 , AX

=

0, X

>

0,

U

and

X

such that

6 If

TUCKER A^+1U > 0,

we take

X = (X, o).

Then

ATU > 0, AX = 0, X > 0, A^U + x, >0, which extends the Lemma to the matrix A = [A, AQ+1] T However, if A^+(JU .nAn+1]J

where Xj - - AjU/An+^U

(j-l>

> 0

so that btu

This second use of the Lemma yields

= o.

V

and

Y

such that

BTV > 0, BY = 0, Y > 0, B^V + y 1 > 0 . We take

Y = (Y, EX.y.).

Then

J J

AY = 0 because

Y ^

0,

zx .y. > o J J

and

Y > 0

and because

AY = [A, An+1J? = BY = 0. We now take

W = V + nU,

where

>* =

-

so that ftT A^+.W i+1 =

0.

Then ATW > 0 because

and

A^W + y 1 > 0

•••>!!

DUAL SYSTEMS OF RELATIONS

7

ATW = BTW

(since a £+1W = 0)

= BTV

(since BTU = 0)

> 0

(by choice of

V

above)

and because A^W + y 1 = B^V + y 1 > 0 Thus, by means of A = [A, An+1].

elusive tending matrix and the

W

and

Y,

(by above).

we have extended the Lemma to the matrix

In the last two paragraphs, by considering the two mutually exm m possibilities A^+1U > o and An+1U < 0, we have succeeded in ex­ the Lemma from the n-columned matrix A to the (n+1 )-columned A = [A, An+1J. Therefore the induction on n is fully established Lemma must hold for all n. This completes the proof of the Lemma. COROLLARY. If the Inequality A^U ^ 0 holds for all solutions U of the system A^U > 0 , then AQ = AX for some X ^ o. PROOF.

We apply the Lemma to the dual systems

[-A0, A]TU > 0 in which - AQ and in theLemma itself.

and

[-AQ, A]

[*°]

= 0,

[*°] > 0,

xQ assume the leading positions held by A 1 and x 1 We are thus assured of some U and xQ, X such that

-AqU J 0, ATU > 0, -Aqx0 + AX = 0, xQ > 0, X and

£ 0,

+ xo > 0#

T T But, by the hypothesis of the Corollary, AQU > o since A U > o. Hence xQ > A^U > 0. Therefore AQ = AX° for X° = X/xQ ^ 0. This completes the proof. This Corollary is a classical theorem of J. Farkas (see [3], page 5 )* It will recur in geometric form in Paper 2, "Polyhedral Convex Cones," as the theorem (Theorem 3) that a polyhedral convex cone is the polar of its polar, and in Paper 3, "Resolution and Separation Theorems for Polyhedral Convex Sets," as the separation theorem. (Lemma 3) for a polyhedral convex cone and an individual vector.

8

TUCKER THEOREM 1. The dual systems A^U > 0 possess solutions

and

U*

AX = 0, X > 0

and

X*

such that

A U* + X* > 0. T [Note: A U* + X* > 0 means that all components of A^U* + X* are positive.] PROOF. In the Lemma the column A 1 played a special role. But, by renumbering, any column Aj could equally well play that special role. Consequently, for j = 1, ..., n, there exist pairs U^, X^ such that ATuJ > 0, AXJ' = 0, X-j > o, aTuj’ + x i > 0. J

Take

U* = Z,UJ’, X* = Z .xt J

J

Then

J

ATU* = Z .ATUj’ > 0, AX* = ZjAxj = 0, X* = Z^xj > 0. Moreover, for

j = 1, ..., n,

A^U* + xt =

+ xj j ^ A ju^ + xj > 0

because aV

+ A

> o

(since

ATUk >0, Xk k o)

and

A^W + xi > o J

J

(as constructed above).

Therefore A U* + X* > 0, which completes the proof. COROLLARY 1A. The system of equations AX = (i) a fully positive solution X (> o) if is no U such that A U > o and 4 °> a^d a nonnegative non-trivial solution X o

0 has there (ii) and

DUAL SYSTEMS OF RELATIONS 7/0)

if there is no

PROOF.

By Theorem

ATU* >

0,

U

there exist

1

AX* =

0,

such that

X* ^

0,

9

A^U > 0. U*

and

and

X*

such that

ATU* + X* >

0.

Either ATU* = 0 or ATU* 4 0 ; in the former case X* > 0. This estab­ lishes (i). Also, either X* 4 0 or X* = 0; in the latter case A^U* > This establishes (ii) and completes the proof. Parts (i) and (ii) of Corollary 1A are Theorems I and II of E. Stiemke [9]. Part (ii) was published still earlier by P. G-ordan [5]; it seems to be the earliest known instance of a ’’transposition theorem. ” The dual systems

THEOREM 2.

ATU >

0,

BTU

and

= 0

possess solutions

U*

and

AX + BY = X*, Y*

0,

X >

0

such that

ATU* + X* > o. PROOF.

We apply Theorem

"x

"X “ 0

and

[A, B, -B]

=

0.

X

X to get

U*

and

X*, Y*, Y*

ATU* > 0, BTU* > 0, -BTU* >

o Ail

m A, B, -B] U >

1 to the dual systems

such that 0,

AX* + BY* - BY* =

0,

X* > 0, Y* J

and ATU* + X* > Take

Y* = Y* - Y*.

0,

BTU* + Y* > 0, -BTU* + Y* >

0.

Then

ATU* > o, BTU* = o, AX* + BY* = and ATU* + X* > This completes the proof.

0.

0,

X* J o

0,

Y* >

0

0.

TUCKER COROLLARY

2 A.

ATU >

Let the dual systems

0,

BTU =

and.

0

A X + B Y = 0, X

>0

have the partitioned presentation A 1TU >

0,

A2TU >

0,

btu =

0

and A 1X, + A 2 X 2 + BY =

0,

X1 >

0,

X2 >

0,

where A 1 is any nonvacuous set of columns of A and A is the set (possibly vacuous) of remaining columns of A. Then (i) either the left system has a 1T / solution U such that A U f o or the right system has a solution X such that X 1 > o. Also (ii) either the left system has a solution U such that 1T A U > o or the right system has a solution X such that X ] / 0 . PROOF. A 1TU*

2- 0 ,

By Theorem

A2 TU* >

0,

BTU* =

1

there exist 0,

U*

and

A 1X* + A 2 X* + BY* =

X*, X* 0,

such that

X* > 0 , X* 2 0 ,

and A 1TU* + X* >

0. ,

A2 TU* + X* > o.

Either A 1^U* 4 0 or A 1TU* = 0 ; in the latter case X* > 0 , since A 1TU* + X* > 0 . This establishes (i). Also, either X* = 0 or X* 4 in the former case A 1TU* > 0 , since A 1TU* + X* > 0 . This establishes (ii) and completes the proof. The alternatives in parts (i) and (ii) of Corollary 2 A are m mutually exclusive because U (AX + BY) = o for any solutions of the dual systems (this is an aspect of the general "complementary slackness" to be established later in Theorem 6 ). Part (ii) is the Transposition Theorem of T. S. Motzkin (see [7], page 51 )• When the matrix B is vacuous, (i) and (ii) become "theorems of alternatives" for the pair of matrices A 1, A 2 (see [10], [1 ] ); these two-matrix transposition theorems have various geometric interpretations (see [10], [1 1 ]), one of which appears as the Separation Theorem (Theorem 2) of Paper 3 in this Study. From a stand­ point of logic, transposition theorems may be viewed as asserting the

DUAL SYSTEMS OP RELATIONS disjoint alternatives of solvability or contradiction via linear combina­ tion (see [6] ). THEORM

3*

The dual systems

V ^ o , CTV > 0

possess solutions

V*

and - CX ;> o , X J o and

v* - CX* > 0 PROOF.

I

such that

and CTV* + X* > 0.

We apply Theorem 1 to the dual systems

[I, C]TV > 0 where that

X*

and

[I, C]

is an identity matrix.

[i, c r v * > o, [I, C]

W* X*

= o,

Thus there exist

W* X* ]

0,

>

0,

j > 0, V*

and

[I, C]TV* •+

W*, X*

w* X*

such

> 0.

That is, V* £ 0, CTV* > 0, - CX* = W* > 0, x* ^ 0, v* - CX* > 0, CTV* + X* > 0. This completes the proof. By applying Theorem 3 to the payoff matrix C of a "fair" zerosum two-person game, it is possible to obtain not only the Main Theorem of von Neumann and Morgenstern (see [8], page 1 5 4 ) but also the characteriza­ tion of the "essential" or "active" pure strategies (first indicated by Bohnenblust, Karlin and Shapley in [2], page 5 4 ). [To make any given matrix game "fair" we subtract from all the given payoffs the least constant k which will produce a matrix C with the property that - CX > 0 for some X > o with Zxj = 1. This constant k, uniquely determined by a Dedekind cut, is the "value" of the given game; it can be imagined as the least side-payment from the first player to the second that would induce the second player to play the given game.] COROLLARY 3A. V > o,

The dual systems C1V j 0

and

possess solutions V* and ing alternatives hold:

X*

- CX £ 0, X > 0 for which the follow­

12

TUCKER (

i)

e i t h e r C^V* 4

o

or

X* > o,

or

X* 4 °>

( ii)

e i t h e r CTV* > 0

(iii)

e ith e r

V* >

o

or

-

CX* 4 °>

( iv)

e ith e r

V* /

o

or

-

CX* > o.

This Corollary is an immediate consequence of Theorem. 3• The m alternatives in (i) - (iv) are mutually exclusive because V CX = 0 for all solutions of the dual systems (this is an aspect of the general "comple­ mentary slackness" to be established in Theorem 6). Parts (ii) and (iv) yield dual forms of the Theorem of the Alternative for Matrices of J. von Neumann and 0. Morgenstern (see [8], pp. 140-1^1), as well as a similar earlier theorem of J. Ville (see [12.], page 1 0 5 ). THEOREM 4. (U

The general dual systems

unrestricted)

- AX - BY = 0

V > 0

- CX - BY > 0

atu

+ CTV > 0

X > 0

b 1 + D^V = 0 possess solutions

(Y

U*, V*

and

V* - CX* -BY* a tu *

> 0,

X* > 0.

ET r " V

""-A

-B

A

B

-B

_ C

D

-IL _v _

U2

~A

> 0

and

-B

B“

A

B

-B

_C

D

-B _

X ” Yi

“X ~ All

_

such that

0

All

_v

0

U2

X*, Y*

We apply Theorem 3 to the dual systems

PROOF.

ui

+ CTV* +

unrestricted)

Ly2_

Thus there exist U* > 0, U* > 0, V* > 0 such that

and

X* g- 0, Y* £ 0, Y* > 0

Yi

Ly2J

DUAL SYSTEMS OP RELATIONS

13

atu *

T + a tu * + C V* > 0

AX* + BY* - BY* > 0

btu *

T + b t u * + D V* > 0

- AX* - BY* + BY* > 0

T * - BTU* - D V* > 0

- CX* - BY* + BY* > 0

bY

atu *

T + a t u * + C V* + X* > o

■ = u* - u*

V* - CX* - BY* + EY* > 0 .

Y* = Y*1 - Y*. 2

and

Then

atu *

+ CTV* J 0

- AX* - BY* = 0

b tu *

+ DTV* = 0

- CX* - BY* £ 0

atu *

+ CTV* + X*

>

V* - CX* - BY* > 0

0

This completes the proof. THEOREM

5.

The self-dual system KW > o,

has a solution

W*

w > o

(KT = - K)

such that KW"* +

> o•

PROOF. We apply Theorem. 3 (which depends on Theorem 1 and its preceding Lemma) to the skew-symmetric matrix C = KT = - K. Then there exist V* and X* such that V* > 0 , KV* :> 0 , KX* > 0 , X* > 0

,

V* + KX* > 0 , KV* + X* > 0 .

Hence

K(V* + X*) > Take

W* = V* + X*.

0,

V* + X* > 0, K(V* + X*) + (V* + X*) > 0.

This completes the proof.

This theorem will he used in Paper 4 , "Theory of Linear Pro­ gramming, n as an omnibus means of proving the basic duality and existence theorems of Linear Programming. It can also be used to establish the Main ("Minimax11) Theorem for symmetric zero-sum two-person games and to char­ acterize at the same time the "essential" or "active" pure strategies of such a game. [The solution of any zero-sum two-person game in normalized form can be obtained through the solution of an associated symmetric game (see [4 ] ).]

TUCKER

The final section of this paper deals withthe property of "comple­ mentary slackness" in pairs of dual systems and in aself-dual system. DEFINITION. A slack inequality in a system is an in­ equality (> o) which is satisfied as a strict in­ equality (> o) by some solution of the system. m

m

Given a (mixed) system A U > o, B U = o, let J denote the set of inT dices j such that A.U. > o for some solution U. of the system; then J J J z U ., summed for all j in J, is a solution of the system which makes A^tzU.) > o for all j in J. This shows that the slack inequalities in J J a system can be characterized collectively as the maximum set of inequali­ ties of the system which are satisfied as strict inequalities by some so­ lution ofthe system.. The remaining inequalities in a system arethose which are satisfied as equations by all solutions of the system. THEOREM 6.

In the general dual systems

(U unrestricted)

- AX - BY = 0

V^O atu +

- CX - DY > 0

cTv ^ o

x ;> o

BTU + DTV = 0 each of the

m+n v± > °

(Y

unrestricted)

pairs of corresponding inequalities -

- EicdikYk > 0

zhahjuh + 2icijvi = ^

^

(i = 1, ..., m) (j =

..., n)

contains exactly one inequality that is slack (rela­ tive to its system). PROOF. Let U, V and X, Y be any solutions of the dual sys­ tems. Then, by multiplying together corresponding items in the dual sys­ tems and summing, we have (1)

- u t a x - u t b y = UT (- AX - BY) = 0

(2 ) - VTCX - vtdy =

vT(- CX - BY) > 0

vTcx = (atu + CTV)TX > 0

(3 )

u tax

+

(4 )

u tby

+ v t b y = (b t u + d t v )ty = 0.

15

DUAL SYSTEMS OP RELATIONS Adding (1 ) and (3) and also (2) and (A), we have - UTBY + VTCX > So

U^EGT - V^CX = o.

and

0

UTBT - VTCX >

0.

Combining this with (1 ) and (4 ), we get - u t a x = u t b x = vTcx = - v Tm .

When use is made of these equalities in (2) and (3), we see that 0 = vT ( - CX - nr) = z ± V l ( 0

= (ATU + CTV)TX

= Zj (Zjj&hjUft + Eicijvi^xj*

These two equations show that in each pair of dual inequalities vi > 0

and

- 2jcijx j - Ekdikyk = 0

(1=1,•••,m)

+ Zicijvi = 0

and

xj = 0

(J= 1^ •••*n )

at least one sign of equality (=) must hold for all solutions; otherwise we would have V^(- CX - D E ) > 0 or (ATU + C^V)^X > 0 for some solu­ tions. Therefore each pair of dual inequalities contains at most one in­ equality that is slack. But, by Theorem, k, there exist solutions of the dual systems such that V*

+

( -

Z jC ^ x *

-

S fc d ^ s )

( V h ^ h + 2icijvi ) + xj

>

0

>0

U*, V*

and

X*, Y*

(i-1,...,m) (J=1,

T h e r e f o r e each pair of corresponding inequalities contains at least one in­

equality that is slack. Taken together, the last two paragraphs assert that each pair of corresponding inequalities in the dual systems contains exactly one ine­ quality that is slack (relative to its system). This completes the proof. The property that Theorem 6 attributes to the individual inequali­ ties in a pair of dual systems can be described collectively as comple­ mentary slackness: the set of slack inequalities in the one system is ex­ actly complementary to the set of slack inequalities in the other system. This is illustrated by the small-scale example of a general pair of dual systems given in the introductory part of this paper. The slack inequali­ ties in each system of the example are starred to make it quite evident

16

TUCKER

that opposite each slack inequality (at left or right) there is a non-slack inequality (at right or left), and conversely. The fact that the starred inequalities are slack (relative to their systems) is verified in a remark following the example; the fact that the remaining inequalities are not slack is a consequence of the first paragraph of the proof of Theorem 6 . The property of complementary slackness applies, of course, to the pairs of dualsystems consideredin Theorems 1 -3 , sinceTheorem he specialized to these pairs of dual systems by letting certain of the matrices A, B, C, D become vacuous. In particular, this shows that the alternatives in the various parts of Corollaries 1A, 2k and 3A are mutually exclusive. THEOREM

7-

In the self-dual system KW ^ 0, W J 0

each of the

n

pairs of corresponding inequalities

z.k. .w. > 0 and J

(KT = - K)

J J

w. > 0

(i = 1,

..., n)

contains exactly one inequality that is slack (rela­ tive to the full self-dual system). PROOF. WTKW =

For any

W, W^KU =

zzw.k. .w• = 0 J J

because

= 2ZW1 (- kj.pWj = - zzw.k.jW.^ = - WTKW.

Hence, in each pair of corresponding inequalities Zjk^jWj >

0 and

w^ > 0

(i = 1,

..., n)

at least one sign of equality (=) must hold for all solutions W; otherT wise we would have W KW > 0 for some solution W. Therefore each pair of corresponding inequalities contains at most one inequality that is slack. But, by Theorem 5, there exist-s a solution ZjkijWj + Wi > °'

W*

such that

(i = 1, ..., n).

Therefore each pair of corresponding inequalities contains at least one in­ equality that is slack. Taken together, the last two paragraphs assert that each pair of corresponding inequalities contains exactly one inequality that is slack

6

can

DUAL SYSTEMS OP RELATIONS (relative to the full self-dual system).

17

This completes the proof.

The following small-scale example Illustrates the complementary slackness within a self-dual system.

-v2 W1 -V 1 W1

+V3

-v4 >

-w3

+uk > 0

+W2

-v2

+W3

0

>

0

>

0*

V1 V2 V3

>

0*

>

0*

>

0*

W4 >

0

The starred inequalities are slack (relative to the full system), as can easily he verified: W = (1 , 1 , 1 , o) is a solution of the system, which satisfies all four starred inequalities as strict inequalities. The fact that the remaining four ("complementary11) inequalities are satisfied as equations by all solutions ¥ of the self-dual system is a consequence of the proof above. BIBLIOGRAPHY [1 ]

ANTOSIE¥ICZ, H. A., MA theorem on alternatives for pairs of matrices Pacific J. Math. 5 (1 9 5 5 ), 6 4 1 -6 4 2 .

[2 ]

BOHNENBLUST, H. P., KARLIN, S., and SHAPLEY, L. S., "Solutions of discrete, two-person games," Contributions to the Theory of Games, Vol. I, pp. 5 1 - 7 2 , Annals of Mathematics Study No. 2 4 , Princeton, 19 5 0 .

[5]

PARKAS, J., "Uber die theorie der einfachen Ungleichungen, " J. Reine Angew. Math. 124 (1902), pp. 1-2 4 .

[4 ]

GALE, D., KUHN, H. ¥., and TUCKER, A. W., "On symmetric games," Contributions to the Theory of Games, Vol. I, pp. 81-87, Annals of Mathematics Study No. 2 4 , Princeton, 1950.

[5]

GORDAN, P., "Uber die Auflosungen linearer Gleichungen mit reelen Coefficienten," Math. Ann. 6 (1875), 23-28.

[6]

KUHN, H. ¥., "Solvability and consistency for linear equations and inequalities," Amer. Math. Monthly 63 (1956), 217-232.

[7]

MOTZKIN, T. S., Beitrage zur Theorie der Linearen Ungleichungen (Dissertation, Basel, 1933 ) Jerusalem, 1936.

[8]

von NEUMANN, J., and MORGENSTERN, 0 . , Theory of Games and Economic Behavior, Princeton, 1944 (3rd edition 1 9 5 3 )-

[9] [10]

STJEMKE, E., "Uber positive Losungen homogener linearer Gleichungen, Math. Ann. 76 (1 9 1 5 ), 3 4 0 -3 4 2 . TUCKER, A. ¥., "Theorems of alternatives for pairs of matrices,"

18

TUCKER Symposium on Linear Inequalities and Programming, A. Orden and L. Goldstein, eds., pp. 180-181, Comptroller, Hq. USAF, Washington, D. C., 1952. Also Abstract No. 7 6, Bull. Amer. Math. Soc. 56 (1 9 5 0 ), 5 7 .

[11]

TUCKER, A. W., Game Theory and Programming, pp. 3 2 -4 4 , College Book­ store, Oklahoma A. and M. C., Stillwater, Okla. 1 9 5 5 * Also Abstract No. 2 1 4 , Bull. Amer. Math. Soc. 61 (1 9 5 5 ), 1 3 5 *

[12]

VTT.TE, J., "Sur la theorie generate des jeux ou intervient l'habilet

0)

= (X | X =

+ ... + B ^ ;

v. >

0 }.

These definitions carry over to Y-space; thus, the polar of a set X-space is B* = (Y | YX g and

0)

= {Y I YX |

the convex cone spanned by a finite set

0;

A**

= AZ,

2

A = {A1, ..., A^}

i.e.,

we show (Theorem 3) that, for

CY | YX g 0

whenever

in

X e B)

AZ = {Y |Y = UA; U > 0 ) = (Y I Y = u,A, + ... + u ^ ;

In Part

B

A

is

u± >

0 }.

finite,

AX g o) = {Y | Y = UA;

U>

0 },

which expresses a classical result of J. Parkas (see [2], page 5 ). We also show (Theorems 2 and 4 ) that, given any finite A there is some finite B such that A* = BZ ,

i.e.

{X | AX g

0}

= {X | X = BV; V ^

0 ),

and, at the same time, AZ = B*,

i.e.

{Y | Y = UA; U >

0}

= {Y | YB g

0 ).

The relation A* = BZ saysthat any intersectionof finitely manyhalfspaces is the convex hull of finitely many halflines; as a "finite basis theorem" for the solutions of a finite system of homogeneous linear in­ equalities, this was stated by H. Minkowski (see [8 ], page A3 ) and proved by Parkas (see [2], page 9 )* The relation = B* says that any convex

21

POLYHEDRAL CONVEX CONES

hull of finitely many halflines is an intersection of finitely many half­ spaces, a result established by H. Weyl for A of rank n (see [12], Theorem 1). In Part 3 we show that, corresponding to any two polyhedral con­ vex cones in X-space c 1 = ( A 1 )* = ( B 1 )Z

and

C2 = (A2 ) * = (B2 )Z,

there is a "greatest" polyhedral convex cone c 1 n c2 = {X I A 1X J 0, A2 X j o ) contained in both

C1

and

C2 ,

( n is "cap")

and a "least" polyhedral convex cone

C 1 U C2 = {X | X = V 1b ’ + V2B2j V 1 > 0, V 2 > 0}

1

(U

i3 "cup")

2

containing both C and C . Hence the system of polyhedral convex cones in X-space is a lattice with respect to set-inclusion C (see [1] for lattice concepts). The correspondence A* = BZ = C

- — ► C* = B* = AZ

is a dual isomorphism (or involutory anti-isomorphism) between the lattice of polyhedral convex cones in X-space and that in Y-space. If we identify Y-space with X-space to form, an n-dimensional Euclidean vector space with inner product X • Y = YX, we find that for any polyhedral convex cone C C D C* =o

and

C U C* = the entire n-space.

Hence, under this identification, the lattice of polyhedral convex cones is orthocomplemented by the dual automorphism C

-----► C* .

These lattice properties of polyhedral convex cones under set-inclusion were formulated by David Gale (see [ b] and [5 1 ). In Part 1, to lay a groundwork for proving that any intersection of finitely many halfspaces is a convex hull of finitely many halflines (A* = B^), we examine the face structure of a polyhedral convex cone A* = (X

determined

|AX ^ 0} = (X | A.jX g o ,

by a set A = {A1, ..., A_]

of rank

...,

Ap X g 0}

n - d.

We show (in Lemma

22

GOLDMAN AND TUCKER

1 )that a vector XQ interior to a face of dimension greater than d + 1 can beexpressed as a sum XQ = X-j + X2 of vectors X 1, X2 interior to boundary faces ofdimensions greater than d. Through this reduction we establish that the polyhedral convex cone A* is either just itsd-face, which is the d-dimensional subspace AX = o, or the convex hull of this d-face and its finitely many (d+1 )-faces. The d-face and (d+1)-faces are shown in Part b to be precisely the extreme faces of the polyhedral convex cone A*, and this fact is used in characterizing the minimal sets B such that A* = B^. PART l.

Pace Structure of the

Convex Cone

A*

We turn first to the face-structure of the polyhedral convex cone A* = {X | A.jX ^ 0 ,

...,

ApX g 0 } .

To each subset H (which may be the null set 0 ) of the indices there corresponds a subset F^ of A* defined by the conditions A^X < o

for each

h in

AhX = o

for each

h not

1, . .., p

H, in

H.

PH is the (open) face of A* corresponding to H. Since H can be chosen in 2^ possible ways, and since nonvacuous faces corresponding to distinct subsets of indices are clearly disjoint, we see that A* is par­ titioned into 2P faces, some of which may be vacuous. If F^ is not vacuous, then it is the intersection PH = ° H n %

of the (open) set

°H

of all vectors A^X < o

X

for each

satisfying h

in

H \

and the linear subspace A^X = 0 The dimension

djj of

L^

consisting of all vectors X for each

h

not in

satisfying

H.

is given by

dH - n ' rH' where

rjj is the maximal number of linearly independent equations in the

POLYHEDRAL CONVEX CONES

23

system of equations determining L^ (i-e., the maximal number of linearly independent vectors in the set (A^ | h not in H}). We say that F^ is a faces of dimension

dH .

Let r = r0 = rank of A and d = d0 = n - r. It is not hard to see that A* has no faces of dimension o is an edge of A*: if X satisfies AX ^ o then the open halfline t(- X) for t > 0 is an edge of A*. The notion of "boundary face" mustnow be introduced. Suppose that H and G are subsets of the index set, that G is a proper sub­ set of H, and that % and fg are both nonvacuous. Then we say that % is a boundary face of F n o t e that (because we work with open

2b

GOLDMAN AND TUCKER

faces) Fq is not a subset of Fjj, and is in fact disjoint from F^. set of all boundary faces of F-^- constitutes the "boundary" of F

The

If Fq is a boundary face of FH' then cIq < d^. To prove this, we note that the system of equations defining L jj is a subset of the sys­ tem of equations defining L^, so that d^ g dH . Choose an index g in H - G (the part of H not in G); if d^ = djj, then the equation AgX = 0 would be linearly dependent on the equations A^X = 0 for all h not in H, so that any vector X satisfying these last equations would also satisfy A^X =0. It would follow that F^r must be vacuous (con­ trary to hypothesis). Thus the assumption = djj is untenable, and ^G < dH

must hold.

Before proceeding further, we give two simple examples to illus­ trate the concepts just considered and the results soon to be proved. EXAMPLE 1. -1

A =

E 1

X 1

P 2

X 1

F 0

X 1

>

>

0,

>

0,

x 2

=

0

=

0,

x 2

>

0

=

0,

x 2

=

0

X2

0

d 12 d 1

d 2 d

=

2

=

1

=

1 II

X 1

(See Figure

o

F 12

x 1 > o, x2 > o.

n

Here the cone A* is the "first quadrant" 1). We list the nonvacuous faces:

F 1, F2, and F0 are boundary faces of E 12* F 1 and F2 have F0 as boundary face. The edges of A* are found by the process described above to be F 1 : tE1

all t > 0

F2 : sE2

all s > 0

and

where E 1 and E2 are the unit vectors along the spectively. We point out that any nonzero vector in as a convex combination

X1

^

.

X2

A*

X- +x„ (xi + X 2 ) E 1 + 3T«c 7 (xi + x 2 ) E 2

and x2-axes re­ can be expressed

POLYHEDRAL CONVEX CONES

25

Figure 2

Figure 1

C^ = shaded quadrant Figure k

Figure 5

of vectors (x1 + x2 )E] and (x1 + x 2 )E2 in the edges, so that the convex hull of its edges and the vector o.

A*

is

EXAMPLE 2.

1

1

A = L

-1

-1 J

A* here consists of the line x,1 + x,2 = 0 (see Figure 2). The only nonvacuous face is F0 = A*, and d = 1. The edges of A* are found to be tEJ

all

t > 0

26

GOLDMAN AND TUCKER

and sE^ where

Ej

has components

all

s > 0,

(1, - 1)

and

E^

has components

(- 1, 1 ).

In Example 1 , we found that A* was the convex hull of its d-dimensional face and its faces of dimension d + 1, while in Example 2 , A* coincided with its(unique) face F0 of dimension d. We will show (Theorem 1 ) that these are the only possible cases. First we prove a "reduction lemma." LEMMA 1. A vector XQ lying in a face FR of dimension d^ > d + 1 can he expressed as a sum

X0

=

X1

+

X2

of vectors X 1, X2 belonging to boundary faces of Fjj of dimension > d (and necessarily < d^). PROOF. The linear subspace L^ which "carries" F^ is of di­ mension dH > d + 1. Therefore we can find in L^ a vector X 1 which is not in the linear subspace generated by XQ and the vectors of the d-di­ mensional space F0 . XQ and X 1togetherspan a 2-dimensional linear subspace L° of Ljj consisting of all vectors (1 )

X = tX

+ sX1.

L° intersects the cone A in a subset whose parameters t ands satisfy (2)

M° consisting

of the vectors

X

AhX = t(AhX0 ) + s(AhX l) ^ 0

for each index

h.

A vector X of the form (1 ) is in matically satisfies (2) for each h not in only if it satisfies (2)for each h in H. dH > d = d0 . We now draw (see Figure

5

t(AhX0 ) + s(AhX*) = 0

L jj and therefore auto­ H; thus X is in M° if and H is not vacuoussince

) the graphs of the equations each

h

in

H

in the (s,t)-plane, obtaining a set of straight lines through the origin with slopes

POLYHEDRAL CONVEX CONES

27

t

Figure

5

fh = - AhX'/AhX0 For

h in

H we have

Aj1X0 < o

(3)

each h

in

H.

and socan rewrite (2) as t > fhs .

The set of straight lines in the (s,t)-plane does not consist of just a single straight line with some slope f. Otherwise we would have Ah (fXQ + X !) = 0 and

the

sameequation

each h

wpuld holdfor hnot in H

in

H

because

XQ

and

X1

28

GOLDMAN AND TUCKER

lie in L^. This would mean A(fXQ + X 1 ) = 0 , so that fXQ + X ’ would be in F0, contradicting the way in which X f was chosen.

slope f^, H. We set

Therefore the set of lines contains distinct lines of maximal and minimal slope f^"> corresponding to certain h 1, h M in

f

—U* , -h 1r ^h"

t- = ■*— — 1 fhi'T h 1 h ,f

to get

X 1 = t-X. + 3 . X ' 1 1 0 1

and f I! 32

= rh'~-fh ,?

These vectors

X

and

t2

X2

= fh»-^h,v

t0

set

X2

= taX° + S2X'‘

are such that Xi ♦ X2 = X0 .

For

X1

and

X2, (3 ) reduces to

fh - * fh

and

respectively. We see that X 1 satisfies (3) (and thus (2)) as an equation for h = h* and as a strict inequality for h = h M, while X 2 satisfies (3) (and thus (2)) as an equation for h = h" and as a strict inequality for h = h !. Thus each of X 1, X 2 satisfies (2) as a strict inequality for some but not all h in H; X 1 and X 2 lie in and therefore satisfy (2) as an equation for each h not in H. It follows that each of X^ and X 2 lies in a boundary face of F^ of dimension > d; this completes the proof of the lemma. THEORM 1 . A* is either just its unique d-face or the convex hull of this d-face and its (d+1 )-dimensional faces. PROOF. We have already pointed out that A* has no faces of di­ mension < d. Suppose A* does not consist only of its d-dimensional face. Consider first any vector X lying in a face of dimension > d + 1 . We may apply Lemma 1 repeatedly to show that X Is a finite sum of vectors each lying in a (d+1 )-dimensional face of A*, say X = x1 + ...

+

xm.

POLYHEDRAL CONVEX CONES The vectors

mX1, .

m^

29

each lie in a (d+1 )-dimensional face of

A*,

and by -writing X =

1

(mX, ) + ... +

1

(mXj

we see that X is a convex combination of the vectors mX1, ..., m\ 1* As for the other vectors of A*, which lie in faces of dimension d or d + 1 , they certainly lie in the convex hull of these faces• The inclusion in the; other direction follows immediately from the convexity of A*, and so A* is the convex hull of its d-face and its faces of dimension d + 1 . The following corollaries are direct consequences of Theorem. and our earlier comments on the nature of (d+1 )-faces and edges.

1

COROLLARY 1A. The intersection of finitely many halfspaces is either a linear subspace (of dimension d) or the convex hull of finitely many half-subspaces (of dimension d + 1 ) bounded by a common subspace (of di­ mension d). COROLLARY 1B. If the rank: of A is n, either just o or the convex hull of o of A*. PART

2.

In Part

then A* is and the edges

The Theorems of Minkowski, Farkas, and Weyl. 2

we consider, in addition to the convex cone A* = {X | A,X g o ,

ApX g

0}

a convex cone

BZ = (X | X = BV; V >

0}

= {X | X =

+ ... + VqB ; v - >

0)

where B = {B-, ..., B_} is a finite set of vectorsin X-space and is Z ^ Z said to span B . In other words, the set B consists of all finite linear combinationswith nonnegative coefficients of vectors in B. It is easy to see that B^ is the convex hull of the halflines v- > 0 , generated by the vectors B. of B. Conversely, the convex hull J J z of any finite family of halflines can be written in the form B ; to form B, one chooses a nonzero vector from each of the nondegenerate halflines (and chooses B = {0 } if the only halfline in the family is the degenerate one consisting of the single vector o).

30

GOLDMAN AND TUCKER THEOREM 2 . (Minkowski [8 ]). Given of vectors in Y-space, there exists of vectors in X-space such that A*

a finite set A a finite set B = B^.

PROOF. We wish to prove thatA* has a spanning set B. By Theorem 1 , it suffices to show that thed-face F0 and the (d+1 )-faces FH each have (finite) spanning sets; B can then be taken as the union of these spanning sets. If d = o, then F 0 consists of o alone, and so (0 ) is a spanning set for F0 . If d > 0 , let B 1, ..., Bd be a basis for F0, so that any vector X in F0 can be written in the form X — c^B^ + ... + Set

u^ = max (c^, o) > o

and

*

v^ = max (- c^, o) ^

0;

since

X = u ]B 1 + ... + udBd + v 1 (- B 1 ) + ... + vd (- Bd ), it follows that

{B1, ...,

Bd, - B 1, ...,

- Bd )spans

F0 .

Now letPH be any (d+1 )-face. The (d+1 )-dimensionallinear subspace which "carries" F-^- contains the d-dimensional subspace F0; thus Ljj is linearly generated by the vectors of F 0 and any vector Bjj which lies in % (and so lies in L^. but notin F0 ). Therefore any vector X 1 in PH can be written in the form X ! = X + cBjp

X € F0 .

H is not vacuous; if we multiply the last equation on the left by A^ for some h in H and recall that A^X 1 < 0 , A^X = 0 , and find that c > o. It follows that a spanning set for F-^- is obtained by adjoining B^ to any spanning set B of F0 . This completes the proof. COROLLARY 2 A. An intersection of finitely many halfspaces is a convex hull of finitely many halflines. We recall that the polar E* of a set is by definition the (closed) convex cone E* = (Y | YX £ 0 ; X in Y-space; in particular vectors in X-space.

A** = (A*)*

g

E

of vectors- from X-space

E)

is defined whenever

A

is a set of

POLYHEDRAL CONVEX CONES

31

THEOREM 3 (Parkas [2]). If A is a finite set of vectors in Y-space, then A** = A^. PROOF. The fact that A** is a subset of A^ follows immediate ly from the Corollary to the Lemma of the preceding paper, "Dual Systems of Homogeneous Linear Relations", which we rewrite as follows: If the inequality AX

0,

•••, Up >

0.

T In this rewriting we have replaced the A of the Corollary by A, the column vector AQ by a vector Y in Y-space (the space of "rowvectors"), the variable column vector U by a vector (- X) in X-space (the space of "column vectors"), and the column vector X by a row vector (u1, ..., Up).

if

Y

To prove the inclusion in the other direction, we observe that is in A^, so that Y = u 1A 1 + ... + UpApj

u 1>

0,

..., Up >

0,

then YX ^ 0 certainly holds for allX such that AX ^ 0 (i.e., A^X 0,

Eu^ = 1}

and a polyhedral convex cone o f = {X I X = v 1Q1 + ... + vqQq;

v. > 0 } .

Thistwofold resolution of a polyhedral convex set is essentially equivalent to the threefold representation (or basis) theorem of T. S. Motzkin [7 ]; taken with its converse, this result constitutes our Resolution Theorem (Theorem 1 ). The significance of the extreme vectors of S when A has rank nis discussed in two corollaries (1 A and 1B). We remark that the symbols A and z in PA and are to be pictorially suggestive: a triangle A (interior and boundary) typical two-dimensional "bounded convex polyhedron" and an angle z terior and boundary) is a typical two-dimensional "polyhedral convex

chosen is a (in­ cone".

Our other main result (Theorem 2 ) asserts that a bounded convex polyhedron and a polyhedral convex cone can be separated by a hyperplane if they do not intersect. This Separation Theorem is a geometric variant of the Motzkin Transposition Theorem [7 ] (see remark following Corollary 2 A of Paper 1 in this Study). Our first aim, then, is the characterization of all sets of the form S = {X | AX g B] , where

Arepresents

the m by n matrix llaj_jll or the set of row vectors B represents the column vector formed by b 1# ..., bm . Such a set may, of course, be vacuous. If nonvacuous, it is convex, that is, whenever X 1 and X2 are in S, and t1 > 0 and t2 > 0 are numbers such that t 1 +t2 = 1, the vector t 1X 1 + t2X2 (a typical vector "with endpoint on the line segment joining the endpoints of X^ and X2 ") is also in S. To emphasize the fact that S is the intersection of finitely many halfspaces we call it a polyhedral convexset. If the system of in­ equalities is homogeneous, i.e., B = 0, then S is a polyhedral convex cone: if X is in S, then the vectors tX for all t > 0 ("the ray or halfline generated by X") are also in S. Theorem 2 of Paper 2 shows that such a polyhedral convex cone {X | AX § 0} can also be expressed as the convex-cone hull Qf~ of a finite set Q = (Q^ ..., Q^) : Q, = {X | X = v lQ 1 + ... + v Q : v. > 0} = {X | X = QV; * J Concretely,

Q

is the

n

by

q matrix with

Q 1, •••, Qq

V £ 0). as columns.

RESOLUTION AND SEPARATION One more type of geometric object will play a role in our dis­ cussion. This is the bounded convex polyhedron P^, which is the convex hull of a finite nonvacuous set P = {P^ •.., P^) of vectors: P4 = IX I X = u lP 1 + ... + UpPp; = {X I X = PU;

u± > 0 ,

Su± = 1)

U > o, Zu± = 1}.

Concretely, Pis the n by p matrix with P ^ ..., Pp as columns. Our results will imply that any bounded polyhedral convex set is a bounded convex polyhedron. The key step in the method used below is the transition from poly­ hedral convex sets S in the n-dimensional X-space to polyhedral convex cones Cn+1 in the (closed) half space t > o of the (n+1 )-dimensional spaceof vectors X = (X, t). The results of Paper 2 are then applied to these cones in X-space. [We write (X, t) as a row instead of a column for typographical convenience.] LEMMA 1.

The rule

{X | AX 0,

all

Vj > 0.

Clearly PA + = (X | (X, 1 ) € Dn+1}. Drl+1 lies in the half space t > 0 and meets the hyperplane t = 1 (since p > o); thus, by Lemma 1, (X | (X, 1) 6 Dn+1} = S(Dn+1) is a nonvacuous polyhedral convex set. (c)

We must still prove that if S = (X I AX g B) =

+qf-

then o f = {X | AX g 0).

The cone

Dn+1

constructed S = /

so

S = S(Dn+1);

t

in (b) has the property ^

= (X I (X, 1 ) e Dn+1 },

by Lemma 1 it follows that

Dn+1 = Cn+1(S) = {X | AX - Bt g 0 , - t g o ) . Thus

= {X | (X, 0) e Dn+1) = (X | (X, 0) e Cn+1(S)} = {X | AX 0 ), or there exists a hyperplane YX = 0 which separates XQ from QZ in the sense that XQ lies in the open halfspace YX > 0 but QZ lies in the closed halfspace YX g 0 . PROOF.

By Theorem 3 of Paper 2

QZ - (X | X = QV, V ^ 0 } = (X | YX g 0 ,

all

Y

In

Q*} = Q**

50

GOLDVLAN

Thus XQ fails to lie in QZ if and only if there is some vector Y in Q* = (Y | YQ, ^ 0} such that YXQ > 0; clearly YX ^ 0 for all X in QZ = Q**. THEOREM 2. Let PA = {X | X = PU, U > 0, Zu± = 1 ) be a given bounded convex polyhedron and QZ = (X | X = QV, V > 0} a given polyhedral convex cone. Either PA and QZ intersect, or there exists a hyperplane YX = o which separates PA and QZ in the sense that PA lies in the open halfspace YX > 0 but QZ lies in the closed halfspace YX § 0. PROOF. It is clear that the alternatives are mutually exclusive. Let P = {P.J, ..., Pp); if p = 1 then the desired result follows from Lemma 3, so we assume p > 1. We shall suppose that P^ and do not intersect (i.e., that the first alternative does not hold) and prove that the second alternative holds. We first observe that for {- P 1,

"■

“

i § i § p,

P^

Pi+1' •••> ” Fp, Q-j > •••>

P± = X 5k (- PY k^l

+

does not lie in * For if

^k = °’

= 0

then

1 V k + pi = k^i

'

and if we set

a = 1 +

Y

^k 51 °> uk = ^k^8" for

^ J 1,

k^i u. = 1/a,

and

v. = v./a J

J

we obtain X k

ukPk = 2vjQj; uk = °' Euk = 1' vj = °'

which contradicts the disjointness of By Lemma 3, for each

i

PA

and

(1 § i % p)

QZ . there is a vector

Y^

RESOLUTION AND SEPARATION such that

Y^P^

> 0 and

Y^X g 0

for each vector

51 X

in

{- P ^

...,

“ ^j_+i > ***> ” Q-| > *•*} • Thus Y^P^ > 0, Y^P-^. > 0 for k 4 i and Y^X g 0 forall X in Q I t follows easily that the second alternative of the theorem holds with Y * Y 1 + ... + Y^. References [1 ]

DAVIS, C., "The intersection of a linear subspace with the positive orthant," Michigan Mathematical Journal 1 (1 9 5 2 ), pp. 1 6 3 - 1 6 8 .

[2 ]

DAVIS, C ., "Remarks on a previous paper, " Michigan Mathematical Journal 2 (1953-4), pp. 23-25*

[3 ]

DAVIS, C., "The theory of positive linear dependence," American' Journal of Math. 76 (1954), pp. 733-746.

[4]>

PARKAS, J., "Uber die theorie der einfachen Ungleichungen," Journal fur reine und angewandte Mathematik 124 (1 9 0 2 ), pp. 1-27*

[5 ]

GOLDMAN, A. J., and TUCKER, A. W., "Polyhedral convex cones," Paper 2 in this Study.

[6]

MINKOWSKI, H., Geometrie der Zahlen, Teubner, Leipzig (1 9 1 0 ).

[7 ]

MOTZKIN, T. S., "Beitrage zur theorie der linearen Ungleichungen," (Dissertation, Basel, 1933) Jerusalem, 1 9 3 6 .

[8]

MOTZKIN, T. S., RAIPFA, H., THOMPSON, G. L., THRALL, R. M., "The double description method, f Annals of Mathematics Study 2 8 ,pp. 5 1 -7 4 . See [12].

[9 ]

TUCKER, A. W., "Dual systems of homogeneous linear relations," Paper 1 in this Study.

[1 0 ]

WEYL, H., "The elementary theory of convex polyhedra," Annals of Mathematics Study 24, Princeton (1 9 5 0 , pp. 3 - 1 8 . See [1 1 ].

[11]

Contributions to the Theory of Games (Vol. 1 ), ed. by H. W. Kuhn and A. W. Tucker, Annals of Mathematics Study 24, Princeton, 1 9 5 0 .

[12]

Contributions to the Theory of Games (Vol. 2 ), ed. by H. W. Kuhn and A. W. Tucker, Annals of Mathematics Study 2 8 , Princeton, 1953•

Princeton University

A. J. Goldman

THEORY OP LINEAR PROGRAMMING* A. J. Goldman and A. W. Tucker Foreword The aim of this paper is a systematic presentation of theoretical properties of dual linear programs. The main results in Parts 1 , 2 , and k are already well established, but most of the remaining material has not been published previously. At the end of the paper there Is a brief biblio­ graphy, to which references are made in the text by means of numbers in square brackets.

Contents

PART 1

Dual Linear Programs....................................

Page 5^

2

Duality and Existence Theorems.....................

58

3

Dual Problems with Mixed Constraints.....................

62

k

Linear Programming and Matrix Games......................

70

5

Lagrange Multipliers....................................

76

6

Systems of Equated Constraints...........................

78

7

Optimal Rays............................................

87

8

Analysis and Synthesis Theorems..........................

90

This paper was written for the Office of Naval Research Logistics Project in the Department of Mathematics at Princeton University. 53

GOLEMM AND TUCKER

5k

PART 1: Dual Linear Programs The following constitute a pair of dual linear programs (or dual linear programming problems ): I . To maximize c ix i + ••• + V S l ’ subject to the

m+n

constraints

a 11x ! +

+ a1nxn £ b 1

(1) a + ... + awrx < b ml 1 mn n = m

(2)

X 1> 0,

II.

X2 > 0,

To minimize

u 1b 1 + •” subject to the

n + m

. • u 1a m

(*0

+ V m

constraints

u ian (3)

Xn > 0.

u 1> o,

+ ••• + V m i . + ••• + W

. •

? ci

• £ cn

u2 > o, ..., Um > °-

Here the aj_j> 10±> and- cj are given real numbers. (But the results of this paper hold equally for numbers drawn from any ordered field. The inequalities of (1) are called row constraints (since they involve the rows of the matrix of aqj,s^ those of (3 ) are column constraints. These problems may be represented jointly by the following con­ venient scheme

55

THEORY OF LINEAR PROGRAMMING

X1

X2

U1

a T1

a i2

U2

a 21

a22

am1

ata2

(> 0 )

...

•••

• • •

• • •

xn aln a2n

amn

An interpretation may be given in terms of Activity Analysis. (See [1 9 ].) Let there be n "activities"; i.e., ways of making a single desired commodity from available stocks of m primary commodities ("ma­ be the amount of the i-th material used in one unit terials"). Let T -j of the j-th activity, b^ the available stock of the i-th material, c . the quantity of the desired commodity made by one unit of the j-th activity, and x. the number of units of the j-th activity to be undertaken. The maximization problem is then a search for an "activity vector" ..., x ) which will yield the greatest possible total output Scjx j ° ? the desired commodity, subject to the constraints (1 ) set by the available stocks of the m materials and by the natural impossibility (2 ) of-negative "ac­ tivity levels". The dual problem pertains to accounting (fictitious or "shadow") prices attached to the m. materials, on a scale whose unit is the price of the desired commodity. One seeks a "price vector" (u^ ..., u^) that minirrlzes the total accounting value zu^b^ of the available stocks of materials, subject to the "one-can1t-get-something-for-nothing" require­ ment (3 ) that the accounting value of the quantity of the desired commodity made by one unit of an activity can never exceed the total accounting value of the materials used in that unit, and to the natural requirement (t) that all accounting prices be non-negative. These may be summarized schematically as follows: activity levels (> 0)

accounting prices of materials

X1

xn

U! .. • . •.

ai1 . materials materials used used . per activity unit unit •.

a ln .. . •.

h . . .

%

aml

atan

bm

(>)

1 n outputs per activity unit

available stocks of materials

56

GOLDMAN AND TUCKER

Returning to the mathematical discussion, we give a more compact notation for the problems. If we form the row vectors u = (u,,

V

C = (c.,,

■' cn ^

and also the column vectors

xr •

X =

•

B =

. • x_ _ n_

• • __ bm _

then we may use matrix symbolism to write the problems as I

To maximize subject to (1 ) AX C

(k)

(2 ) X > 0

UB,

U > 0.

Joint scheme: (> 0 )

XT

(g)

UT

A

B

C m-T ■tT and Xx Ux (a transposes of U and X. by component, just as the Strict inequality between Y 1 < Y 2 to indicate that

column and a row vector, respectively) are the The vector inequalities written here hold component equality of two vectors holds componentwise. vectors is also defined componentwise. We write Y 1 ^ Y 2 but Y 1 4 Y 2 •

We say that the m + n constraints (1 ) and (2 ) are feasible if there exists some vector X satisfying them; we then call the maximization problem feasible, and call such an X a feasible vector for it. A feasible X° which provides the desired maximum for CX is called an optimal vector for the maximization problem. The terms "feasible" and "optimal" are de­ fined analogously for the minimization problem. At this point we prove the following lemmas. LE2VQVLA 1 .

If

X

and

U

are feasible, then

CX ^ UB.

THEORY OF LINEAR PROGRAMMING PROOF.

57

CX g (UA)X = U(AX) £ UB.

LEMMA 2 . If X° and U° are feasible, and CX° = U°B (or merely CX° > U°B), then X° and U° are optimal.

feasible optimal.

PROOF. If CX° = UB°, then by Lemma 1 U° B = CX° g UB for all U. Thus U° isoptimal, and a similar argument shows X° is The parenthetical insertion is justified by Lemma 1 .

According to Lemma 2 , the assertion that X° and U° are a pair of optimal vectors can be checked directly and simply in the schematic form of the dual programs: 1 . One checks the feasibility of X° by making sure that all its components are non-negative and that the inner product of each row of the scheme with X° is not greater than the corresponding b^. 2 . One checks the feasibility of U° by making sure that all its components are non-negative and that its inner product with each column of the scheme is not less than the corresponding c.. J 3 * Finally, one checks the equality of the inner product of C and X° with that of U° and B.

LEMMA 3 . If all b^ are non-negative, then the maxi­ mization problem has feasible constraints. If all a^. are non-negative and some a^j is positive for each j, then the minimization problem has feasible constraints. PROOF. The first statement is immediately proved by noting that the zero vector is feasible for the maximization problem. The second state­ ment is proved by noting that under our hypotheses any U will be feasible if all its components are sufficiently large. The following two examples illustrate the possibility that the constraints in one or both problems may be infeasible: EXAMPLE 1. (> 0 ) u

Here

X

y

- 1 - 2

V

1

3

(>)

0

0

U = (0 , 0 ) is feasible, but the requirements

GOLDMAN AND TUCKER

58

- x

x

- 2y g - 3

+ 3y s J

>

2 O

clearly cannot be met, so the maximization problem has no feasible vector. EXAMPLE 2 .

0)

X

y

u

0

1

- 1

V

o

b

3

(> )

1

2

Here neither the requirements Ox + y ^ - 1 y > o nor the requirement Ou + Ov > 1 can be met, so neither problem has a feasible vector. It will be shown in the "Existence Theorem" of Part 2 that both problems possess optimal vectors if they both possess feasible vectors. PART 2:

Duality and Existence Theorems

In this part we derive the two basic theorems of linear programming theory, together with several corollaries. The essential step in the analy­ sis is the application of Theorem 5 of the preceding paper ("Dual Systems of Homogeneous Linear Relations") to the skew-symmetric matrix

-

K = to obtain a column vector (1)

Bt0 > AXq

(11)

U0A > CtQ

(111)

CXQ > UqB

0

A

B ~

at

0

CT

bt

- C

0 _

V

to ) ^ 0

such that

THEORY OP LINEAR PROGRAMMING (iv)

CX0+ tQ > U0B

(v)

U* + Bt0> AXq

(vi)

UqA

+

59

>CtQ .

Throughout Part 2 the symbols MX0 M, Mu 0 "j "to" w111 pe“ served for these particular XQ, UQ, tQ, and the numbering (i) through (vi) will be reserved for the inequalities just listed. The cases tQ > o and t = o will be investigated separately. LEMMA b . Suppose tQ > o. Then there are optimal vectors X° and U° for the dual programs such that CX° = U°B,

(U°)T + B > AX°,

U°A + (X° )T > C.

PROOF. Since tQ > 0, the non-negative vector (U^, XQ,t ) can be "normalized" so that tQ = 1 without affecting the validity of the homogeneous inequalities (i) through (vi). Then (i) and (ii) (with tQ = 1 show that XQ andU0 are feasible; (iii) and Lemma 2 show that XQ and U0 are optimal, and Lemma 1 shows that CXQ = UQB. The relations + B > AJt0 and UQA + X^ > C follow from (v) and (vi) (with tQ = l ). Thus we can choose the normalized XQ and U0 to be the desired X° and LEMMA 5* Suppose tQ = 0. (a) At least one of the dual programs has no feasible vector. (b) If the maximization problem has a feasible vector, then the set of its feasible vectors is unbounded and CX is unbounded on this set. Dually for the minimization problem. (c) Neither problem has an optimal vector. PROOF. Suppose X is a feasible vector for the maximization problem. Using (ii) with tQ = 0 and the non-negativity of X, we obtain UqAX > 0. This last inequality, together with (iv) (with tQ = o) and the feasibility of X, yields (*)

0 ^ UqAX g UqB < CXQ .

To prove (a), we note that if the minimization problem had a _ feasible vector U, then we could obtain the "duals" of the first two in­ equalities in (*):

6o

GOLEMM AND TUCKER 0 > UAXq > CXQ .

This clearly contradicts (*), and so (a) is proved. To prove (b), we examine the infinite ray consisting of the vectors X + XXQ

(all

X > 0 ).

Clearly X + XXQ > 0; using (i) with tQ = 0, we see that A(X + XXQ ) ^ AX < B. Thus the entire infinite ray consists of feasible vectors, which proves the first assertion of (b). Furthermore, since CXQ > o (by (*)), we see that C(X + XXQ ) = CX + \CX Q can be made arbitrarily large by choosing x large enough* So the second assertion of (b) is proved. Finally, (c) is an immediate consequence of (b). This completes the proof of the lemma. COROLLARY 1A. Either both the maximization and mini­ mization problems have optimal vectors, or neither does. In the first case the (attained) maximum and minimum are equal; their common magnitudeiscalled the optimal value of the dual programs. PROOF. If one problem has optimal vectors then (c) of Lemma 5 shows that tQ > 0 and Lemma k shows that both problems have optimal vectors X°, U°, such that the nmaximumM CX° is equal to the "minimum1’ U°B. COROLLARY 1B. A necessary and sufficient condition that either (and thus both) of the dual programs have optimal vectors is that either CX or UB be bounded on the corresponding nonvacuous set of feasible vectors. PROOF. The necessity of the condition is clear. To prove its sufficiency, suppose that the maximization problem has a nonvacuous set offeasible vectors and that CX is bounded on this set. By (b) of Lemma 5 we must have tQ > 0, and by Lemma 4 both problems have optimalvectors. THEOREM 1 (Duality Theorem). A feasible X° is opti­ mal if and only if there is a feasible U° with U°B = CX°. A feasible U° is optimal if and only if there is a feasible X° with CX° = U°B.

61

THEORY OF LINEAR PROGRAMMING

PROOF. We shall demonstrate only the first statement, since the second clearly has an analogous proof. Lemma 2 immediately yields the sufficiency of the condition. To prove its necessity, suppose that X° is optimal. By (c) of Lemma 5 we must have t > 0, by Lemma h the minimiza­ tion problem also has an optimal vector U°, and by Corollary 1A the attain­ ed maximum CX° and minimum U°B are equal. THEOREM 2 (Existence Theorem). A necessary and sufficient condition that one (and thus both) of the dual problems have optimal vectors is that both have feasible vectors. PROOF. The necessity of the condition is clear. To prove its sufficiency, suppose that both problems have feasible vectors. By (a) of Lemma 5 we have t > 0 , and by Lemma k both problemshave optimal vectors. This completes the proof. The Duality and Existence Theorems were first proved by Gale, Kuhn, and Tucker in [7 ].

X°, U°

The next pair of corollaries deal with the way in which a pair of optimal vectors satisfy the 2m + 2n feasibility constraints X > 0,

AX g B,

U > 0,

UA

> C.

Corollary 2B will imply that any such pair X°, U°satisfies at least m+n of these constraints as equations, while Corollary 2A will imply the existence of a particular pair which satisfies at most m + n (and thus exactly m + n ) of the constraints as equations. There is a convenient way to pair offthe constraints associated with the two problems. The i-th relations summarized in the matrix state­ ments AX g B, U ^ o, are called a pair of dual constraints, as are the j-th relations given by UA > C, X ^ 0. COROLLARY 2A. If both problems have feasible then they have optimal vectors X°, U°, such (1) If X° satisfies a row constraint as EQUATION, then U° satisfies the dual straint as a STRICT INEQUALITY. (2 ) If U° satisfies a column constraint EQUATION, then X° satisfies the dual straint as a STRICT INEQUALITY. PROOF.

vectors, that: an con­ as an con­

By Theorem 2 both problems have optimal vectors, so (c)

GOLDMAN AND TUCKER

62 of Lemma 5 yields such that

tQ > o.

By Lemma b there are optimal vectors

(U°)T + B > AX°, Thi8

X°

and

U°

X°, U°

U°A + (X°)T > C.

clearly have the desired properties.

In the statement and proof of the next corollary, it is convenient to use the notation for the k-th component of a vector Y. COROLLARY 2B. If both problems are feasible, then for any i either (AX° < b^ for some optimal X° and u^ = 0 for every optimal U, or (AX)^ = b^ for every optimal X and u^ > 0 for some optimal U°. (Of course the dual statement holds for (XJ°A) . > c ., etc.) J

J

PROOF. By Theorem 2 both problems have optimal vectors, so the alternatives are meaningful. Corollary 2A implies that if (AX)^ = b^ for every optimal X, then u° > o for some optimal U°; thusit suffices to prove that if (AX0 )^ 0 and

together Imply

811(1 so' to­ UB, we have (using

CX° ^ UAX° < UB. Since X° and U are optimal, Corollary 1A asserts that thus a contradiction is reached.

CX° = UB,

and

A constraint is said to be tight if every optimal vector satisfies it as an equation; otherwise the constraint is loose. We restate Corollary 2B in this language: The dual of a tight constraint is loose, and vice versa. PART 3:

Dual Problems with Mixed Constraints

Let N be the set of indices [1, 2, ..., n], while M is the corresponding set [1, 2, ..., m]. Suppose N 1 and N2 are complementary subsets of N with n 1 and n2 elements respectively, while M 1 and M2 are complementary subsets of M with m 1 and m2 elements respectively. We are concerned here with problem-pairs of the following sort:

THEORY OP LINEAR PROGRAMMING

63

I . To maximize c lXl + ... + cnxn subject to the constraints b^

for each

i

in

M1

b^

for each

i

in

M2

non-negative for each

j

in

N1

unrestricted for each

j

in

N2

aiix i + ••• +

xj

II.

{

To minimize u 1b l +

+ V m

subject to the constraints > c . for

each j in N 1

u 1a 1j + ••• + V t a j j c.

J

for

non-negative for ui

{

each j in Np ^

each i

unrestricted for each

i

in M 1 in

M2

Note that the constraints which are equations correspond to the variables which are unrestricted. We may adopt the previous terminology, and speak of X or U as feasible if it satisfies the constraints and as optimal if it is feasible and achieves the desired maximization or minimization. If we decompose the matrix A and the vectors B, C, X and U into blocks corresponding to the decompositions of M and N into M 1 + M2 and N 1 + N2 respectively, then the problems take the following forms: I.

To maximize c1x l + c2x2

subject to the constraints A-1 1X 1 + A 12X2 g B 1

A21X 1 + A22X 2 = B2 X 1 > 0,

(x2

unrestricted).

GOLDMAN AND TUCKER

6k II.

To minimize

U ! B 1 + U2B2 subject to the constraints U 1A 11 + U2A21 =

U 1A 12 + U2A 22 " C2 (U2

unrestricted).

Joint Scheme: An

o

*U~

X2

Ail

S B1

(M, )

= B2

(Mg)

o An

o and - f ;> o. To effect this formally, set max (o, U2 )

= max (o, X2 ) X ” = max (0, - X2 )

TIM

max (0, - UQ

1 2 where max (Y , Y ) is the vector Z whose components are The new problems, which are of canonical type, are: I 1.

To maximize

o1x 1 + c2x^ + (- c2 )x” subject to the constraints

1 z^ = max (y^,

THEORY OF LINEAR PROGRAMMING

An X, + W l

65

A 12X^ + ( - A 12) X |' § B 1

+

+ ("

( - Ag1 )X, +( - A22x p XT

- B2

+ A22X ^'

g - Ba

^ °> x2 ? °>x2' ? °*

II1. To minimize

U 1B 1 + U^B2 + U p ( - B2 ) subject to the constraints

u i Ai 1 + ^ 1

+ o ' * ( - a2 1 ) > C ,

U1A 12 + ^ A22 + ^ ' ( - A2 2 ^ C2 u ^ - a 1 2 ) + u - ( - a ^ ) + u - ' a 22 > - c 2 u, > 0,

> 0, u p

> 0.

~ X1

K

“

1

1

The new problem-pair is equivalent to the old in the sense that (U1, U^, 1) is feasible (optimal) if and only if (U1, U2 ) is feasible (optimal), and similarly for

and

xl

X2

2_

_ X2

(Because of this equivalence, the Duality and Existence Theorems of Part 2, which are certainly valid for the "canonical" problems I 1 and II1, also hold for the problems I and II with mixed constraints.) Furthermore, the old problems may be regained from the new (i.e., the process can be reversed) by setting

x2 = X^

xf up

Unfortunately the matrix for the new problems, which is given by

A 11

A 12

- A 12~

*21

A22

-

*21

-

A22

*2 2 *

22_

66

GOLDMAN AND TUCKER

is larger than the original

A 11

A 12

A = 22

We shall next describe two methods for bringing a pair of prob­ lems with mixed constraints into canonical form, which are free from this disadvantage. METHOD 1: Here we use the fact that a constraint equation can be employed to eliminate any variable appearing in it with non-zero coefficient after this is done the equation of course appears as a defining relation rather than as a constraint, Suppose, for instance, that we had as a "row constraintM the equation a ,x, + ... + awvx = b , mi 1 mn n m with

a ^ 4 °*

(1 )

Then

x^ n

xn

can be eliminated by

= -J— (h a m

- a -x. - ... - a „ .xM . ). ml 1 m, n-1 n-1

Let 5 - a aij " aij

r

5 j

able

(2)

um

If, on the one hand, may be eliminated by

^

= i

-u i

i “

v

bmaln a mn

for and

i =1, j =1,

..., n - 1 ..., n - 1

=o - ^ 2 J atan n

is in

(°n ' U,am

N2,

then the unrestricted vari­

V l V ^ n ’1

The dual problems given by the matrix of the a ^ .1s and the vectors of the b^'s and c-'s are then equivalent to the previous ones, in that (x.j, ..., xn ; or (u1, •••>■%) feasible (optimal) for the old prob­ lem if and only if (x-j, ..., or (u1, ..., u&-1 ) is feasible (opti­ mal) for the new. Note that we have eliminated the variables xn and um , the m-th row constraint, and the n-th column constraint. In reversing the

67

THEORY OF LINEAR PROGRAMMING operation (i.e., in regaining the original problems from the transformed ones), (1) and (2 ) are used to define xn and u^. If, on the other hand,

n for

replace the unrestricted variable

u^

is in

N 1,

then we also set

j = 1,

>

by the non-negative variable

(3) and form as new m-th row constraint

a,ml1x 1 „ -x„ < 5m 1 + ... + a.m, n-1 n-11 = Note that we have eliminated the variable xn and the n-th column con­ straint, and have changed the m-th row constraint from an equation to an inequality. Again we obtain an equivalent pair of problems, with (x.,, ..., xR ) and (u1, ..., t^) corresponding to (x^ ..., xn-1 ) and (u1, ..., Ufa) respectively, and (1 ) and (3 ) are used to define xn and in reversing the operation. In either case (n in N 1 or in N2 ) the method changes the forms CX and UB only by the constant ^)mcn/amn# Here we have dealt with a "row constraint" equation; of course a similar operation applies to a "column constraint" equation. After a finite sequence of such operations (applied to both rows and columns), we clearly reach a situation in which all constraint equations (if any exist) have only zero coefficients. If all the constant terms (b^!s or cj ,s) in these equations are zero, then we may delete the associated rows and columns of zeros (cf. the inverses of Types 5 and 10 in Method 2 below), yielding a problem-pair of canonical form (m2 = n2 = 0); if any of the constant terms is non-zero, then the corresponding member of the problempair (and thus the corresponding member of the original problem-pair) is exhibited as unfeasible. METHOD 2: We begin by listing eleven types ofelementary transformations on dual linear programs. Type 1. Multiply a row constraint equation through by a non­ zero quantity. Suppose, for instance, that the equation is V i

+ ••• + V n

= V

68

GOLDMAN AND TUCKER

Then for any k 4 o, we can replace unrestricted variable u^, by

V

a^

(j = 1 , ..., n), bm ,

and the

= kamj

5m = kba %

= >

•

(x-j, ..., xn ) or (u^ ..., u^) is feasible (optimal) in the old situa­ tion if and only if (x^ ..., xR ) or (u1, ..., u^^, \ i ) is feasible (optimal) in the new one. The operation is clearly reversible, and leaves the values of CX and UB unchanged. Type 2 . Add a multiple of a row constraint equation to another row constraint.Suppose, for instance,that theequation is (4), and that the otherconstraint isassociated with the first row. Then, for any X, we can replace a 1 • (j = 1 , ..., n), b ^ and the unrestricted variable

v

by

V =V +XV b, = b ! + Xbm %

= ■% - V

•

The new problems are equivalent to the old in the same sense as for Type 1 , and the operation is reversible. Again the values of CX and UB are unchanged. Type 3 . Add to C a multiple of the i-th row vector of A, where i is in M2 * Suppose, for instance, that i= m, so that the con­ straint equation is ( k ) ; then for any p.we can replace Cj (j = 1 , ..., n) and the unrestricted variable um by

5j=cj+"V %

=

+ " *

The new problems are equivalent to the old in the same sense as for Type 1 , and the operation is reversible. The values of CX and UBare both changed by pbm . Type 4. If m is in M 1, then adjoin to the matrix A an (n+1 )-st column (0 , ..., 0 , 1 ), to the vector C an (n+1 )-st component cn +1 = 0 , and to the vector X an (n+1 )-st component xn + 1 (a new

THEORY OP LINEAR PROGRAMMING

69

non-negative variable) defined by

(5)

^1*1 + ••• + atan*n + xn+ l =

V

The result is a new problem-pair in which (5 ) appears as the m-th row con­ straint. Note that was a non-negative variable in the old problems; in the new ones 1^ is (formally) unrestricted but is subjected to the new (n+1 )-st column constraint um > 0. Thus and n 1 are increased by one and m^ is reduced by one. ..., xn ) or (u^ ..., u^) is feasible (optimal) in the old situation if and only if (x^ ..., xn, xn+1 ) or (u.j, ...> v ^ ) is feasible (optimal) in the new. The operation is re­ versible; the values of CX and UB are unchanged. *Type

5 .Adjoin an

(m+1 )-st row ofzero entries to

A,

an

(m+1 )-stcomponent bm+1 = 0 to B, and. an (m+1 )-st component (a new unrestricted variable) to U, together with the (m+1 )-st row constraint 0x1 + ... + oxn = 0 . (Xj, •••, xn ) or (u^ ..., u^) is feasible (optimal) in the old situation if and only if (x^ •••, xn ) or (Uj, ..., u^, uJn+1 ) is feasible (opti­ mal) in the new one. The operation is reversible, and does not change the values of CX and UB. Types 6, 7 , 8, 9, 10 are the respective analogues of Types1,2 , rows. In Type 9, the (m+1 )-st row to be to A is (0 , ..., 0 , - 1 ).

3 , 4, 5 for columns instead of

adjoined

Type 1 1 . Border the matrix A with an (m+1 )-st row (0 , ..., 0 , + and an (n+ 1 )-st column (0 , ...,0 , + 1 ), intersecting in the entry (+ 1 ). Also adjoin components bm+1 = 0 and cn + 1 = 0 to B and C, and adjoin components um+1 and xn + 1 (new unrestricted variables) to U and X, together with the (m+1 )-st row constraint 0*1

+ ••• + 0xn ± Xn + 1 " 0

and the (n+1 )-st column constraint

A feasible vector for either new problem has zero as its last component; (x.j, ..., xn ) or (u1, ..., 1^) is feasible (optimal) in the old situa­ tion if and only if (x^ ..., xn, 0 ) or (u^ ..., 0 ) is feasible (optimal) in the new one. The operation is reversible, and does notchange the values of CX and UB.

TO

GOLDMAN AND TUCKER

To see how these elementary transformations are used, suppose we have a row constraint equation with a non-zero coefficient. By renumbering, we may suppose the equation is aml 1 + ... + s lmn x n- b m= o with 4 o. Applying Type 1 with k = 1 /a^, we can "make a ^ = 1." Using Type 2 to multiply the new m-th row constraint by - a ^ and add the result to the i-th row constraint (for i = 1 , 2 , ..., m - t ) , we reach a situation in which the last column of A is (0 , ..., 0 , 1 ). By applying Type 3 with i = m and ii = - cn, we can also make the last component of C equal to 0 . So far we have not changed any of m 1, m2, n 1, n2 * If n is in N 1 (so that the last component of X is a non-negative variable), then we can apply the inverse of Type k; this reduces m 2 and n 1 by one, increases m 1 by one, and leaves n 2 unchanged. If n is in Ng, we first apply Types 7 and 8 to make the last row of A takethe form (o, ..., 0 , 1 ) and the last component of B equal to 0 ;then we can apply the inverse of Type 1 1 , thus reducing m 2 and n2 by onewithout changing m 1 or n 1 .In either case (n in N 1 or N2 ) we have eliminated the row constraint equation with which we began, without increasing n 2 or changing m^ + n 1 . It is clear that the elementary transformations can be used similar­ ly to eliminate a column constraint equation with a non-zero coefficient. Thus, after finitely many applications of these transformations, we reach dual problems for which the constraint equations have only zero coefficients. If the constant terms (b^’s and cj ,s) "these equations are all zero, then we may apply the inverses of Types 5 and 1 0 to "eliminate" the associ­ ated rows and columns of zeros, thus reaching a problem-pair of canonical form. If any of the constant terms is non-zero, then it is clear that the corresponding problem is unfeasible. The results of this Part can be summarized as follows:

In

finitely many steps one can either display the unfeasibility of one or both components of a problem-pair with associated m x n matrix, or else can re­ duce the problems to an "equivalent" pair in canonical form with an m 1 x n 1 matrix, where m ! g m, n 1 g n, and m 1 + n 1 = m 1 + n 1 . PART 4:

Linear Programming and Matrix Games

In this Part we discuss some relationships between the theory of linear programming and that of matrix games. First we describe briefly the fundamental concepts of the latter field, beginning with the notion of matrix game.

THEORY OF LINEAR PROGRAMMING

71

Let there he two players, I and II, with m and n possible courses of action respectively. Each player chooses his course of action without knowing the other player’s choice. If I chooses his i-th alterna­ tive and II chooses his j-th, then II must pay I an amount p ^ . (which may be positive, negative or zero) determined by the miles of the game; equivalently we could say that I must pay II an amount - ^ij* assumed that the players are concerned solely with maximizing their expected payoffs; thus the payoff matrix P, whose (i,j)-th entry is Pj_j> contains all the information relevant to a rational analysis of the game, and so we may speak simply of "the game P M. Furthermore, any matrix P can be in­ terpreted as the payoff matrix.of the game in which I acts by choosing a row of P, II acts by choosing a column of P, and p^j is the payoff to I if the i-th row and j-th column are chosen. We point out in passing that our "matrix games" are precisely the "rectangular games" of McKinsey and the "zero-sum two-person games in normalized form" of von Neumann and Morgens tern. A probability vector is a vector whose components are non-negative and sum to 1 (i.e., it is a discrete probability "distribution"). By a mixed strategy for I we mean a (row) probability vector W = (w1, ..., w ), while a mixed strategy for I I is a (column) probability vector Z = (z^ ..., zn ). Here w^ is interpreted as the probability (or relative frequency) with which I employs his i-th course of action, and z . is interpreted similarly relative to I I !s j-th alternatives The symbol "ifl, which has already appeared as an index, may also be used without fear of ambiguity to designate the i-th unit vectorof m-space; similarly "j" denotes the j-th u n i t v e c t o r o f n - s p a c e . In v ie w o f th e f o r e g o i n g i n t e r p r e ­ t a t i o n o f m ixed s t r a t e g i e s ,

( 1

)

e ( w ,

it

z )

i s n a tu r a l to d e fin e

=

y

w i p i

j

z

j

=

w

z

j as the expected payoff to course - E(W, Z).

Z° (2 )

I

if

W

and

Suppose there existed a number such that E(W, Z°) j v s E(W°, Z)

Z are used; that to

v

II

and mixed strategies

for all

is of

W° and

W, Z.

By employing W°, I could ensure that his expected payoff is at least v (and that II’s is at most - v) no matter what mixed strategy II em­ ploys, while by choosing Z , II could ensure that his expected payoff is at least - v (and that I ’s is at most v) no matter what mixed strategy I employs. Thus I and II might as well settle for expected

GOLDMAN AND TUCKER

72

payoffs of v and - v respectively, and might as well assure these by employing mixed strategies W° and Z satisfying (2 ). The number v is called a value of the game P, and W° and Z° are optimal strategies (called ,rgood strategies" by von Neumann and Morgenstem). It is clear that (2 ) implies the statement (3)

E(i, Z°)

E(W°, j)

for all

1, j.

Conversely (3 ) implies (2 ), for by (3 ) we have E(W, Z°) =

ZWjEd, Z°) vn,

(5 )

PZ°

s V 31,

the two conditions

where V31 and V111 are n-dimensional and m-dimenslonal vectors with all components equal to v. LEMMA 6.

A matrix game has at most one value.

PROOF. Suppose the triples satisfy (2 ). Then we have

(v, W°, Z°)

and

(v1, W 1, Z1 ) both

v g E(W°, Z1 ) g v 1 5 E(w\ Z°) $v, and so

v =

•

This completes the proof.

Two matrix games are here called strategically equivalent if they have the same sets of optimal strategies for both players. Using this terminology, we may prove the following result: LEMMA 7 . If matrix Q is obtained by adding a constant increment k to every entry of matrix P, then the games P and Q are strategically equiva­ lent. If game P has value v, then game Q has value v + k. PROOF.

Let

E(W, Z)

be the expected payoff function for game

P;

73

THEORY OF LINEAR PROGRAMMING then that for game

X

Q

is given by

wi ( p i j + k K * = Z j

wiPijzj +

( Z wi ) k

( Z zj)

= E(w> z) + k -

j

From this it is clear that v, W°, and Z° obey (2 ) for game P if and only if v + k, W°, and Z° obey (2 ) for game Q. This completes the proof. The following theorem of von Neumann is the basic result in the theory of matrix games: THEOREM 3 . Every matrix game has a value and optimal strategies. PROOF. By Lemma 7, we may arrange that the matrix P of the game has all its entries positive. By applying Lemma 3 and Theorem 2 to the pairof dual programs with A = P, B = (1 , ..., 1 ), and C = (1 ,..., we obtain vectors X° > 0 and U° > 0 such that (i)

U°P > (1,

1)

(ii)

PX° s (1,

1)T

(ill)

1

Z*j = Z u £ •

(Here (iii) is just the statement and so we may define

"CX° = U°Bn. ) From (i) we know

»- ' / Y A ■ '/

U° 4 0 ,

2> 5

wi = ^ i Zj = TOj * Then (i) gies and

and (ii) imply (4) and (5 ), and so W° and Z° are optimal strate v is the value of the game P. This completes the proof. COROLLARY 3 A. Any matrix game with value v has optimal strategies W° and Z° such that w^ > 0 for all i with E(i, Z°) = v, and z9 > 0 for all j with E(W°, j) = v. COROLLARY 3B. Let P be a matrix gamewith value v. Then for any 1, either E(i, Z°) < v for some optimal strategy Z° and w1 = 0 for all

GOLIKAN AND TUCKER optimal strategies W, or E(i, Z) = v for all opti­ mal strategies Z and w ? > o for some optimal strategy W ° . (Of course the dual statement also holds for E(W°, j) > v, etc.) PROOFS. From the statement and proof of Lemma 7, we see that add­ ing any k to all elements of P leads to a strategically equivalent game and adds k to both the expected payoff function and the value. Thus we may assume that all the entries of P are positive. By passing to the (feasible) dual programs considered in the proof of Theorem 3 and applying Corollaries 2A and 2B to these programs, we obtain the desired results. We note, in connection with the device employed in the last few proofs, that the process converts any matrix game P with positive value into "equivalent” dual programs with A = P, B = (1, ..., 1), and C = (1, ..., 1). Conversely, consider any pair of feasible dual programs with A > 0 , B > 0 , and C > o. The optimal value 5 = U°B of the pro­ grams must be positive, for this could fail to hold only if U° = 0, and then constraint U°A > C would be violated. The dual programs can be transformed by the substitution P ij = ai j /bi cj into a matrix game P whose value is 1/5 and whose optimal strategies W° and Z° are related to the optimal vectors X°, U° of the programs by

w i = uibi/8 *

zS = cj xj /8 • The following theorem, published in [4 ], is due to Dantzig and Brown. THEOREM 4. The optimal vectors of the dual programs with matrix A and constraint vectors B and C, are precisely the vectors x° = x/t,

u° = u/t,

where (u^ ..., ..., xn, t) is an optimal strategy with positive last component t for the (symmetric) game /

o

- A

at

o

\ - bt

C

B

\

- ct 0J

THEORY OF LINEAR PROGRAMMING

75

PROOF. We first note that the matrix is skew-symmetric, so that an optimal strategy for one player is also optimal for the other, and the value of the game must he 0. The mixed strategy (u1, ..., x ^ ..., x , t) is optimal if and only if

If t > o, then by Lemma 2.

X/t

and

Conversely, let dual programs, so that

(i)

AX g Bt

(ii)

UA > tC

(iii)

UB § CX.

U/t

are optimal vectors for the dual programs

XQ > o

andU° > 0

be optimal vectors for the

(iv) AX° < B (v) U°A > C (vi) U°B > CX°. Set t = 1/(1 + zu? + zxj) > 0; then (iv), (v), and (vi) imply that (tu°, ..., tu^, tx°, ..., tx°, t) satisfies (i), (ii), and (iii), and is thus an optimal mixed strategy. This completes the proof. The earliesttransformation of a pairof dual linear programs into a matrix game was given byGale, Kuhn, and Tucker [7 ],and ispresented in the next theorem. THEOREM 5- The necessary and sufficient condition that X° and U° be optimal vectors for the dual program is that the matrix game

has value zero and optimal strategies (for the first and second players) proportional to (U°, 1) and (X°, 1 ) respectively. PROOF. The matrix game has value zero and optimal strategies proportional to (U°, 1 ) and (X°, 1 ) if and only if (i)

AX° - B ^ 0

(ii) - CX° + U°AX° g 0

76

GOLEMAN AND TUCKER (ill) (iv) (v)

U°A -C > 0 U°B +U°AX° X°>

> 0

0, U°> 0.

(i), (ill), and (v) are the necessary and sufficient conditions that and U° be feasible. (ii) and (iv) together are equivalent to

X°

(vi) U°B g U°AX° ^ CX°, which (by Lemma 2, Theorem 2 , and the feasibility of X° to be a necessary and sufficient condition that X° and This completes the proof.

and U ° ) is seen U° be optimal.

Some of the results to be established later in this report are closely connected with theorems on matrix games, and we shall point out. such correspondences as they arise. PART 5:

Lagrange Multipliers

We begin this Part by recalling to the reader the method of Lagrange multipliers traditionally used in the calculus for finding con­ strained extrema. If G(x1, . . x ) = G(X) is to be maximized or mini­ mized subject to the constraints F,(X) =0,

Fm (X) = 0 ,

then we form the Lagrangian function m H(X, u,,

u^) = H(X, U) = G(X) + Y, uipi(x ) i= 1

using the Lagrange multipliers u^. The necessary conditions that H(X, U) have an extreme value (viz., the vanishing of the first derivatives of H) are also necessary conditions that G(X) have its constrained maximum or minimum, so that instead of seeking a constrained extremum for G we may seek an unconstrained extremum for the more complicated function H. A similar situation arises in linear programming, where the object is the maximization or minimization of a linear function subject to con­ straints which are linear inequalities. We introduce the Lagrangian Function L(X, U) (1)

=

CX+UB - UAX

= CX + u, (b1 - Sa1jXj) + ... + = UB

+ X-| (c, - 2u ia11 ) + . . . + xn (cn - su^a^).

77

THEORY OF LINEAR PROGRAMMING

If, keeping the maximization problem in mind, we write G(X) = CX, F^ (X) =(bi - za^ .x .), and regard the u^ as "multipliersM, then the formal analogy with the technique described above is exhibited by the second form for L(X, U); if the roles of X and U are interchanged thenthe third form for L(X, U) exhibits the analogyfor the minimization problem. In the next theorem we shall show that the analogy has more than formal validity. THEOREM 6. The necessary and sufficient condition that X° > o and U° > o be optimal vectors for the dual programs is that (X°, U ° ) be a saddlepoint for L(X, U) in the sense that L(X, U°) g L(X°, U°) g L(X°, U) for all X > o, U > 0. If X° and U° are optimal, then L(X°, U ° ) is the optimal value of the dual programs.

have

PROOF. CX° = U°B.

(a) Suppose X° and U°are optimal. Since CX° g U°AX° < U°B, we have

By Theorem 2 we

L(X°, U°) = CX° = U°B, as desired.

For any

X > 0, U ^ o,

we have

L(X, U°) = U°B + (C - U°A)X ^ U°B = L(X°, U°), L(X°, U) = CX° + U(B - AX°) > CX° = L(X°, U°), so

(X°, U ° ) is a saddle-point.

(b) Now suppose (X°, U ° ) is a saddle-point, with U° > o. For any X > 0,we have U°B - (U°A - C)X =

L(X, U°) g L(X°, U ° ) = U°B - (U°A

X° > o

and

- C)X°,

or (U°A - C)(X - X°) > 0. If we choose X = X° + E-?, where E. is the J last relation yields (U°A - C). > 0. Taking U°A > C, and so U° is feasible. Using the statement similarly, we see that X° is also L(o,

U°) g L(X°,

we have U°B g CX°, and so by completes the proof.

Lemma 2

j-th unit vector, then the j = 1, ..., n, we have other half of the saddle-point feasible. From

U°) < L(X°, 0 ) X° and

U°

areoptimal.

This

GOLDMAN AND TUCKER

78

For comparison with relation (2 ) of Part k (which defines "optimal strategy") we restate Theorem 5 in the following form: X° > 0

and

U° > 0

are optimal if and only if

L(X, U°) £ U°AX° 0 , U > 0.

We note that the "Lagrange multiplier" situation for dual linear programs differs from that in the calculus in that (a) the fact that we are dealing simultaneously with a maximum problem and a minimum problem leads to consideration of a saddle-point (rather than an extremum) for L(X, U), (b) the conditions involved are not only necessary but also sufficient, and (c) after passing to the Lagrangian Function, we are still left with the (simple) constraints X ^ 0 , U > 0 . Lagrange multipliers also play a similar role in Non-Linear Programming (see [1 2 ]). The method of Lagrange multipliers can also be applied to the "problems with mixed constraints" considered in Part 3« In this case the multipliers which are dual to constraint equations are not requiredto be non-negative; we omit further details. PART 6:

Systems of Equated Constraints

Many of the results to be presented in this section stem from the work of Shapley and Snow [1 5 ] on matrix games. We will reverse the his­ torical development by proving the linear-programming versions of the theo­ rems first, and then deriving the (original) game-theoretic results as consequences. We begin by sketching informally the geometric ideas behind the next few definitions and theorems. The set of feasible vectors for the maximization problem is defined by a finite system of linear inequalities and is therefore a (possibly unbounded) convex polyhedral set; the system (I) of equated constraints defined below determines a linear subspace (of the space of vectors X) whose intersection with this polyhedral set, if nonvacuous, is a (closed) face of the latter. Similarly the system (II) (see below) defines a face of the convex polyhedral set of feasible vectors for the minimization problem (again if the corresponding "intersection" is nonvacuous). If systems (I) and (II) are dual (in the sense described be­ low), we will say they define dual faces of the two polyhedral sets in question. Theorem 7 , when translated into this geometric language, asserts that feasible X, U are optimal if and only if they lie in dual faces. Theorem 8 says that (a) the optimal vectors for each problem form a face of the corresponding polyhedral set of feasible vectors, and (b) these two

THEORY OP LINEAR PROGRAMMING

79

"optimal faces" are dual, and are the only pair of dual faces which consist entirely of optimal vectors. A nonsingular square system of equated con­ straints leads to zero-dimensional faces or "vertices", and Theorem 9 shows that the notion of "vertex" is essentially the same as that of "extreme vector" for the sets of feasible vectors. Theorem 10 studies the extreme vectors of the sets of optimal vectors; feasible nonzero X and U are of this type if and only if they are dual vertices of the sets of feasible vectors. Now we proceed to the definitions and theorems. Let M 1 and M2 be arbitrary subsets of the set M of indices [1, ..., m], while Nu1 and N2 are arbitrary subsets of the set N of indices [1, ..., n]. Then each of (AX)± = b±

all

1

In

M1

all

j

in

N2

(i)

*3 = ° and (UA). = c. (i d

J

J

u. = o

all

j

in N,

all

i in

is a system of equated constraints. Systems (I) and (II) are dual if M2 = M - M 1 and N2 = N - N 1; in this case we renumber so that M 1 = [1, •• •, p] and N 1 = [1, •••, q], and the systems may be written as r a 11x l +

+ alqxq = b i

u’ a!l +

+

U la1q +

+

P 1

V

=

C 1

(II)

(I) ap1x 1 + Xq+1 = °>

+ apqxq = \

Vi =°'

xn = 0 ‘

v = 0•

If a system of equated constraints is written in one of these last forms after a renumbering, and if the matrix

a 11 •” a lq Ai = ap1

apq

is square and nonsingular, then the system is a nonsingular square system. THEOREM 7* Feasible vectors X and U are optimal if and only if they satisfy dual systems of equated constraints.

80

GOLEMAN AND TUCKER

PROOF. To prove the necessity of the condition, suppose that X and U are optimal; let M 1 be the set ofi ’s such that (AX)^ = b^, and let M2 = M - M 1. Then Corollary 2B ensures that u^ = 0 for all i in M2 - We define N 1 and N2 analogously, and find that x^ = o for all j in Ng. Thus X and U satisfy the dual systems of equated con­ straints associated with M 1, Mg, N 1 and N2 « To prove the sufficiency, suppose that dualsystems (I) and (II). Then

p

P

to= x Vi

/

q

\

q/

X

and

P

U

\

- 2X \Zaijxj/ = X( Zuiaij )xj -

and so by Lemma 2

X

and

U

are optimal.

satisfy the

q

Zcjxj =cx>

This completes the proof.

THEOREM 8. There is a unique pair of dual systems of equated constraints such that each is the maxi­ mal system of equated constraints satisfied by all optimal vectors. PROOF. Consider the i !s such that (AX)^ = b^ for all optimal X, and the j's such that (UA)j = Cj for all optimal U. By Corollary 2B, therow and columnconstraints corresponding to these indices determine dual systems of equated constraints which clearly have the desired property; uniqueness is also clear. We* now turn to the study of extreme feasible or optimal vectors; 11 2 these are feasible or optimal vectors which are not the mean ^-(X + X ) 1 1 t2 or 7j-(U + IT) of two other feasible or optimal vectors. THEOREM 9- A feasible non-zero X (or U) Is an extreme feasible vector if and only if it satisfies a nonsingular square system of equated constraints. PROOF, (a) Suppose feasible X satisfies the nonsingular square system with associated matrix A 1. For any n-dimensional vector Y, set Y = (y-, ..., y ). Suppose X = -^-(X1 + X2 ), where X 1 and X2 are 1 1 +2x .) = 0, and so, since feasible. For j > q we would have x . = ^-(x. 1 2 i J £ 3 J x. >0 and x. > 0, we would have x. = x^. = o. It would also be true 3 33 J that A,X1 S B,,

A,X2 j B , ,

1 A , ( X 1 + X2 ) = B , ,

where B 1 = (b^ ..., bp). From these relations it would follow that A.jX1 = A^X2 = B 1, and so (since A 1 is nonsingular) X 1 = X2 . Thus

81

THEORY OF LINEAR PROGRAMMING X

1 = X 2 . Prom this, we may conclude that X

is extreme.

(b) Suppose non-zero X is an extreme feasible vector. be the set of i fs for which (AX)., = b .. We prove first that M 1 1 _L I P void. Since X 4 0, some x^ is > o. The vectors X and X ed from X by replacing x • with x • +e and x . - e respectively X as their mean; if M r were void (i.e., if (AX). < b. for all 1 2 x x then X and X would be feasible for sufficiently small € > o, dicting the fact that X is extreme.

Let M 1 is not obtain­ have i), contra­

Now let N2 be the set of j!s for which x . = o. Delete from A the rows which are not "in M f" and the columns which are "in N2 ". Since M f is not void and N 1 = N - N2 is not void (because X 4 o), we actually have a submatrix A left, and it is clear that AX = B, where the components of X are the positive components of components of B are the appropriate components of B.

X

and the

We next assert that the columns of A are linearly independent. If this were not so then there would exist X 1 4 0 such that A X 1 = 0, and if we first set X 1 = X + e X 1, X2 = X' - e X f, and then adjoined appropriate zero components to X 1 and X2 to obtain n-dimensional vectors 1 2 1 2 X and X , we would find that X and X have X as their mean and are feasible for sufficiently smalle > 0,contradicting the fact that X is extreme. Since the columns of A are linearly independent, we can delete suitable rows of A to obtain a nonsingular square submatrix A2, and X clearly satisfies the system of equated constraints associated with A2 « This completes the proof. The restriction to non-zero vectors in the last theorem is un­ important, since clearly the zero vector is an extreme feasible vector if it is feasible. THEOREM 1 0 . Feasible non-zero X and U are extreme optimal vectors if and only if they satisfy dual non­ singular square systems of equated constraints. PROOF. The sufficiency of the condition followsimmediately from Theorem rj and Theorem 9* To prove its necessity, suppose that X and U areextreme optimal vectors. Renumber so that (AX)^ = b^ for 1 ^ i g p, (AX)± < b1 for i > p, (UA)j = for 1 ^ j ^ q, (UA)j > c . for j > q. By Corollary 2B, x . = 0 for j > q and u^ = 0 for i > p. Since some x . with j ^ q or some u, with i g p can also vanish, we further J

GOLDMAN AND TUCKER

82

renumber using p o for 1 £ j g q, x . = o for j > q, u^ > 0 for 1 g i g p, and u^ = 0 for i > p. The hypotheses X 4 0, U 4 0 ensure that p > 0 and q > 0; therefore p > 0 and q > 0. Let A be the matrix obtained as the intersection of the first p rows and first q columns of A. The first q columns of A are linearly independent; the proof of this fact parallels the argument used in (b) of the proof of Theorem 9 to show that the matrix A considered there had linearly independent col­ umns, except that here (in order to obtain a contradiction to the hypothesis that X is an extreme optimal vector) we must also show that if X 1 and X2 are feasible vectors with X = -^(X1 + X2 ), then X 1 and X2 are opti­ mal. This follows since the relations CX1 S UB CX2 £ UB |u(X1 + X2 ) = UB (which hold since CX1 = CX2 = UB,

1 g X , X

so that

are feasible and X 1 and

X2

X, U

are optimal) imply

are optimal by Lemma 2.

Similarly, the first p rows of A are linearly independent. We now renumber in the intervals p g i p) are a maximal linearly independent subset of the rows of A and the first q 1 columns of A (q* > q) are a maximal linear­ ly independent subset of the columns of A. The intersection of the first p 1 rows and the first q* columns of A is a matrix A 1. Because the last q - q 1 columns of A are de­ pendent on the first q ’ columns, any linear relationship between the p* rows of A 1 can be extended to a relationship between the first p 1 rows of A. The latter, however, are independent, and so the rows of A 1 are linearly independent; similarly, the columns of A 1 are linearly independ­ ent, and so A 1 is a nonsingular square matrix. Clearly X and U satisfy the dual nonsingular square systems of equated constraints associ­ ated with A 1; this completes the proof. COROLLARY 10A. The sets of feasible and optimal vectors of linear programs have finitely many ex­ treme vectors. PROOF.

This is an immediate consequence of Theorems 9 and 10.

Theorem 1o provides a systematic method for finding extreme

THEORY OF LINEAR PROGRAMMING

83

optimal vectors; one examines all square systems of equated constraints, dis­ cards those with singular associated matrices, and lists those solutions of the remaining systems which are feasible vectors. This process, however, usually requires a prohibitively great amount of work. A system of equated constraints definition) one of the forms ] ? i 1 zz.1

+

for a matrix game P

.. + p. z - t = 0 *un n

(I!)

Zj = °

has (by

all

i

in M 1

all

j

in N2

all

j in N 1

all

i

z, + ..• + zn = 1

W 1P1J+ ••• + W j

( I I 1) -s

w^ = 0

W1

+

+

w m

=

in M2

1

Dual systems and square systems are defined as for linear programs; a square system is nonsingular if It has a Unique solution (z^ ..., z , t) or (w1, w , s). An extreme optimal strategy for P is an optimal strat­ egy which is not the mean of two other optimal strategies. Before deriving the matrix-game analogues of Theorems 9 and 10 , two preliminary remarks are in order. We note first that if matrix Q, is obtained from P by adding a constant k to each entry of P, then (z.j, ..., zn, t) or (w-j, wm , s) satisfies a system (I1) or (II1) for P If and only If (z.,, ..., z , t + k) or (w^ ..., w , s + k) satisfies the corresponding system for Q. Second, we recall from Part k that with any matrix game P with positive value v we may associate a feasible pair of dual linear programs with A = P, B = (1 , ..., 1 ), and C = (1, 1 ); the optimal strategies of P are put into one-one corre­ spondence with the optimal vectors of the dual programs by the relations (R)

W° = vU°,

Z° = vX°.

It Is clear that (R) also gives a one-one correspondence between the ex­ treme optimal strategies of P and the extreme optimal vectors of the dual programs. We can set up a one-one correspondence between the systems of equated constraints; (I) and (I*) correspond if and only If they have the same M 1 and N2, while (II) and (II!) correspond if and only If they have the same N 1 and M2 . LEMMA 8. Let P be a matrix game with positive value v. If W° is an optimal strategy for P and (W°, v)

8^

GOLDMAN AND TUCKER Is the (■unique) solution of a nonsingular square sys­ tem (II1), then the corresponding square system (II) for the associated dual programs is also nonsingular. Conversely, if optimal vector U° is the (unique) solution of a nonsingular square system (II), then the corresponding square system (II1) is also non­ singular. (Of course the dual statement for systems (I1), (I) also holds.)

PROOF. (a) Suppose W° is an optimal strategy and (W°, v) is the solution of nonsingular square system (II1). The U° associated with W° by (R) is a solution of (II), and zu°4 0 (since U° 4 o, in view of the constraint UA > (1, ..., 1 )). If U is any solution of (II) with zu^ 4 °> then (U/zu.^, 1/zu^) is a solution of (II1); thus U/zu.^ = W° and l/£Uj_ = v, so U = vW° = U°. If U 1 is any solution of (II) with zu| = 0, then U2 = ^-(U° + U 1 ) is a solution of II withzu^4 °> and so U2 = U°; this implies U 1 = U°, contradicting Zu^ 4 0 = 2u|. So (II) has U° as unique solution, and is therefore nonsingular. (b) Now suppose optimal vector U° is the solution of nonsingular square system (II). If (W, s ) Is any solution of (II1) with s 4°, then W/s is a solution of (II); clearly different choices of(W, s) yield different solutions of (II) in this way. Since (II) is nonsingular, only one such (W, s) can exist, and it must be (W°, v) where W° corresponds to U° by (R). If (W1, s1) is any solution of (II1) other than (W°, v), then s1 = 0; (W2, s2 ) = (^-(W° + W 1 ), -l(v + s1)) is a (W, s ) of the type just mentioned, and so (W2, s2 ) = (W°, v). This implies (W1, s1 ) = (W°, v), a contradiction. So (II1) has (W°, v) as unique solution, and Is thus nonsingular. THEOREM 11 (Kuhn [11 ] ). Let P be a matrix game with value v. Optimal strategy W° is an extreme optimal strategy if and only If (W°, v) is the solution of a nonsingular square system of equated constraints for P. (Of course the dual statement for optimal strategy Z° also holds. ) PROOF. In view of Lemma 7 and the first preliminary remark above, we may arrange that v > o. Note that the relations (R) effect a one-one correspondence between extreme optimal strategies and extreme optimal vec­ tors for the associated dual programs.

ing to

If W° is an extreme optimal strategy, then the U° correspond­ W° by (R) Is an extreme optimal vector for the associated

THEORY OF LINEAR PROGRAMMING

85

minimization problem.. U° is also an extreme feasible vector, for if U 1 and U2 are feasible and U° = ^-(U1 + U2 ), then U 1B g U°B, U2B £ U°B and ~(U1B + I^B) = U°B, so that U 1B = U2B = U°B and U 1, U2 are opti­ mal, a contradiction unless U 1 = U2 = U°. By Theorem. 9, U° satisfies a nonsingular square system (II) of equated constraints for the minimiza­ tion problem; Lemma 8 ensures that the corresponding system (II1) is also nonsingular, and it is clear that (W°, v) satisfies (II!). Conversely, suppose (W°, v) is the solution of a nonsingular square system (II?). By Lemma 8, the corresponding system. (II) is also non­ singular. The U° corresponding to W° by (R) is clearly the solution of (II); by Theorem 9 U° is an extreme optimal vector, and so W° is an ex­ treme optimal strategy. This completes the proof. THEOREM 12. Let P be a matrix game with value v different from zero. Optimal strategies W° and Z are extreme optimal strategies if and only if (W°, v) and (Z°, v) satisfy dual nonsingular square systems of equated constraints for P. PROOF. If v > o, then the necessity and sufficiency of the con­ dition follows immediately from Theorem 10, Lemma 8, and the fact that the relations (R) make extreme optimal strategies of P correspond to extreme optimal vectors of the associated dual programs. The case v < o can be reduced to the previous one by noting that Players I and II (for P ) can be considered to be the second and first T players} respectively, in the game - P withvalue - v > o. This com­ pletes the proof. In the statement and proof of the next result, we shall employ the symbol J_ as an abbreviation for the row vector (1, ..., 1 ) of appropri­ ate dimension; adj Q, will denote the adjoint of matrix Q. If Q is a submatrjx of P and W, Z are mixed strategies for P, then W, Z will denote the vectors obtained by deleting from W and Z the components corresponding to the rows and columns of P which must be deleted to ob­ tain Q. THEOREM 13 (Shapley-Snow [1 5 ] )• Let P be a matrix game with value v. Optimal strategies W and Z for P are extreme optimal strategies if and only if there is a square submatrix Q of P with 1* adj (Q) iT 4 0 and V = ______ & I

_

i (adj Q) Tt

86

GOLEMAN AND TUCKER f t

=

_

L

i !

4

J . . f l ________

i (adj Q) iT

z . J s i L a U l 1 (adj Q) I1

zero.

PROOF. (a) Suppose the condition holds and We have the identity

(Adj)

v

is different from

Q(adj Q) = |Q| I,

where I is the identity matrix; since v 4 °> we have |Q| ^ 0, and so Q is nonsingular and Q”1 exists and is given by Q“1 = (adj Q)/|Q|. Thus the hypotheses may be rewritten as v = 1/1Q“11T W = viqf1 • _ 1.T* Z = vQ V . Therefore WQ, = vl and QZ = vl^; furthermore Wl^ = 1Z = 1, so that W and Z are probability vectors and the components of W and Z not in W and Z must all vanish. We have shown that W and Z satisfy the dual square systems of equated constraints with associated nonsingular matrix Q,; by Theorem 12, W and Z are extreme optimal strategies. (b) Suppose W and Z are extreme optimal strategies and v 4 By Theorem 12, W and Z satisfy dual nonsingular square systems of equated constraints, with some associated square matrix Q. Thus we have ¥Q = vi,

QZ = viT,

• _i

• -1 •T Z = vQ 1I1,

or W =

V1Q

iz = wiT = 1 *• ••T 1Z = W11 = 1,

which implies • _ 1.T1 •• V1Q 1 = 1Z = 1; so that v = 1/1*Q”1iT . The relations imply that W

P,

W = vlQ“1, Z = vQ”1iT, v = l/lQ“M T, and Z satisfy the condition.

together with (Adj),

(c) Suppose v = 0; add a positive quantity k to each entry of obtaining a strategically equivalent game P(k) with positive value k.

THEORY OP LINEAR PROGRAMMING

87

By (a) and (b), W and Z are extreme optimal strategies for P(k) (and thus for P) if and onlyif there is a square submatrix Q(k) of P(k) with i (adj Q(k)) 4 0 and k =......1 W 1 .. . 1 (adj Q(k)) iT

With each submatrix Q(k) obtained by subtracting k identities

W =

i adj Q(k) i (adj Q(k)) iT

£ =

(ad.j Q,(k)) i (adj Q(k)) f t

of P(k) is associated a submatrix Q of from each entry of Q(k). In view of the

P

1 adj Q(k) = i adj Q, |Q(k)| = |Q| + ki (adj Q) 1T, it Is clear that Q(k) has the properties just stated if and only if Q, has the properties stated in the theorem (with v = 0). This completes the proof. PART 7: An optimal ray [X°; X] optimal vectors of the form

Optimal Rays for the maximization problem is a set of

X° + XX, where x runs through all non-negative numbers, X° is a fixed optimal vector, and X is a fixed vector whose components Siam to 1. (This last condition serves only as a normalization.) Optimal rays for the minimiza­ tion problem are defined analogously. In the next lemma we characterize the directions X of the optimal rays. LEMMA 9 . Suppose the components of X sum to 1. Then [X°; X] is an optimal ray (for optimal X ° ) if and only if (1 ) X is a probability vector. (2 ) AX g 0 . (3 ) CX = 0.

GOLDMAN M D TUCKER

88

The dual statement holds for the minimization problem; (2 ) becomes UA > 0 .

analogous.

PROOF. We give the proof only for X, since that for [X°; X] is an optimal ray if and only if (i) (ii) (iii)

U

is

AX° + XAX g B CX° + XCX = CX° X° + XX > 0

for all A. > 0 .Clearly these conditions are satisfied if and only if (1 ), (2 ), and (3 ) hold. This completes the proof. In studying the existence of optimal rays, it is useful to intro­ duce the notion of an admissible number \±, which is a number such that the matrix game

has value zero. If the dual linear programs are feasible, then Theorem 5 asserts that their optimal value is admissible. THEOREM 14. For a feasible pair of dual linear programs, statements (1 ) and (2 ) are equivalent and statements (3 ) and (A) are equivalent: (1 ) The maximization problem has an optimal ray. (2 ) (3 ) (4)

Some (i > n° is admissible. The minimization problem has an optimal ray. Some 11 < \±° is admissible.

Furthermore, if (1 ) or (2 ) holds then every \i > |i° is admissible, while if (3 )or( k ) holds then every n < ji° is admissible. PROOF. It clearly suffices to deal onlywith (1 ) and (2 ). note first that m> Is admissible if and only if (I)

AX -

(ii)

UA -

(iii)

nt -

(iv)

sn -

Bt^0 sC>0 CX^ 0 UB>0

We

THEORY OF LINEAR PROGRAMMING for some probability vectors

(X, t ) and

89

(U, s ).

If (1 ) holds, then by Lemma 9 there is a probability vector X with AX g o and CX = o. Using this X and taking t = 0 , we see that (i) and (iii) are satisfied. If U° is any optimal vector for the mini­ mization problem (here Theorem 2 is used), then the probability vector (U, s) proportional to (U°, l) clearly satisfies (iv’) S(i° - UB = o. Using the same any jl > *i°.

(U, s ),

we see from (iv1) that (iv) will be satisfied for

Now suppose (2 ) holds; then there is a probability vector (X, t) and a number £ > \±° such that AX g tB and CX > tji. If t > 0 , then X/t is feasible and C(X/t) > fl > |i°, contradicting the fact that n° is the optimal valueof the dual programs. Thus t = 0 , and so X is a probability vector obeying AX § 0 and CX ^ 0 . Thus CX = 0 , for if CX > 0 then for any optimal X° we would have X° + XX feasible, while C(X° + XX) would exceed the optimal value of the dual programs for x > 0 . So AX £ 0 and CX = 0 ; by Lemma 9 the maximization problem has an opti­ mal ray. This completes the proof. LEMMA 1 0 . The set of directions of optimal rays of a linear programming problem has finitely many ex­ treme vectors. PROOF. According to Lemma 9 , X > 0 is the direction of an opti­ mal ray for the maximization problem if and only if (i)

Exj g 1

(il) z( - xj ) §1 (iii)

AX g 0

(iv)

CX g 0

(v)

(- C)X § 0.

(i) - (v) can be considered the row constraints of a suitable new maximiza­ tion problem; the desired conclusion follows by applying Corollary 10A. Of course an analogous argument holds for the minimization problem. This com­ pletes the proof. THEOREM 15. Let (Xr ) be the (finite) set of extreme optimal vectors for the maximization problem and (Xs) be the (finite) set of extreme directions

GOLDMAN AND TUCKER

90

of optimal rays for the maximization problem. Then the set of all optimal vectors X is the set of all vectors of the form

x =

+

2 > s xS '

with all \ T > 0, all \±3 > o, and = 1. (Of course the dual statement holds for the minimization problem.) PROOF. Let X° be a fixed optimal vector. The set of all opti­ mal vectors X is the solution-set of the system (S): -

IX go AX

g

B

CX

g

cx°

(- c)x g

-

cx°

Here I is the Identity matrix; the presence of - I as a "block" in the coefficient matrix of (S) ensures thatthis matrix has linearly independ­ ent columns. Let (S1)be the system obtained by making (S)homogeneous Lemma 9 shows that the directions of optimal rays are precisely the solu­ tions of (Sr) which obey the extra normalization condition EXj = 1. We now apply Corollary 1A of [9 ] (in this Study) to obtain the desired result; the normalization condition serves to fix the vectors "Qj" the "basis" which that corollary had determined only to within a positive scalar factor

PART 8:

Analysis and Synthesis Theorems

The scheme (> 0)

x?

4

4

A 1!

A 12

A i3

B!

A21

A 22

A2 3

B2

A31

A32 .

A33

B3

C1

°2

°3

represents (for our first theorem) a feasible pair of dual linear programs partitioned (after a possible renumbering of rows and columns) so that (l ) U° > 0,

> 0,

> C3

^or some optimal

U°;

THEORY OF LINEAR PROGRAMMING

(2 ) X° > 0, (3 ) A^ 2

91

X° > 0, X A3jXj < B3 for some optimal X°;

is a maximal nonsingular square block in

A 11 L A21

A 12 A22 J

Corollary 2Aasserts that at least one such partitionexists (with some blocks possiblyvacuous), but no claim is made thatfinding one is prac­ ticable . In this Part we shall use the scheme above to give an explicit description of the sets of optimal vectors for the dual programs, and shall then show how to construct dual programs with preassigned sets of optimal vectors.. These two steps together will implicitly characterize the sets which appear as sets of all optimal vectors for some linear programming problem.. Analogous investigations for matrix games have been made by Gale and Sherman [8] and by Bohnenblust, Karlin, and Shapley [1]; we shall not derive their results. LHVMA 11.

There are matrices

R

and

A 12 - RA22

B 1 = BB2

A21 = A 22 _S

C, = C0S w2

A 11

S

such that

RA22S *

PROOF. By condition (3 ) above, each row of ( A ^ A ^ ) is a linear combination of the rows of (A21A22); clearly the coefficients of these linear combinations form a matrix R such that A 12 = RA22 and A 11 The maximization problem has an optimal vector, and so, by condition (1 ) and Corollary 2B, the system of equations A 11X 1 + A 12X2 = R (A21X 1 + A22X2 ) = B !

A 2 1 X 1 + A 2 2 X 2 = B2 has a solution. Thus B 1 = RB2 . Similarly we can find a matrix that A2 -J ~ A22S, C1 = C2S, and A^ ^ — RA2 ^ =: RA22S. THEOREM 16 (Analysis Theorem). The optimal vectors for the dual programs are precisely the vectors X, U, such that

S

such

GOLDMAN AND TUCKER

92

x1 > ° ^

(I)

(

A

2 1 X 1

-

B 2

)

S

0

(A31 ~ A 3 2 A 2 2 A2 l) X 1 “ (B3 " A32A22B2) = ° X2 = A22(B2 " W l ) X3 “ °

u

1

0

>

(c2 - U lAl2) ^ (II)

> o

Ul ( A 13 ” k \2k 2.2k 2 l )

~ ( C3 “ B2 A2 2 A2 3 )

> 0

U2 = (°2 - U 1A 12)A22 U3 = o The optimal value of the programs is given by (III)

Z u 1Bi = C2A2’B2 = X CjX j

PROOF. (a) Suppose X is optimal. Corollary 2B we see that = 0 and that

By condition (1) above and

A 11X 1 * A 12X2 = B 1’ “^21X 1 + A22X2 = B2’ A31X 1 + A32X2 ^ B3 * Thesecond of

these equations yields a value for

X2,

X2 = A22(B2 ~ A21X l)' which maybesubstituted into

( A 31

Since

X2 > 0,

the last inequalityabove,

yielding

~ A 3 2 A2 2 A 2 l) X 1 “ (B3 “ A 3 2 A 2 2 B2 ) = °*

we must also have A21X 1 - B2 ) S 0*

X is optimal, and so X 1 > 0; we have derived all the conditions (I). Similarly we can show that the conditions (II) are necessary. (b)

Suppose

X

satisfies (I).

By reversing the steps in (a),

t h eo ry : o f l i n e a r p r o g r a m m i n g

we see that

X > 0

93

and that A^X, + A22X2 = B2, A 3 1 X 1 + A 3 2 X 2 = B3 *

Furthermore we have

A 11X 1 + A 12X2

(A 1 1

A 12A2 2 A2 l ) X 1 + A 12A2 2 B2

9

using L€>mma 11, we see that the coefficient of X 1 in the last expression vanishes, while the remaining term is just B 1. Thus A 11X 1 + A 12X2 = B 1 9 and so we have proved that X is feasible. Let U° be any optimal vector for the minimization problem; by (a) it must satisfy (II).Then C 1X ! + °2X2 ■

(C 1 " °2Ai A2l)X 1 + °2A22B2 '

D?B-| -

U?(B1 - A 12A ^ B 2 ) +

=

C2A-^B2 .

Applying Lemma 11 to these equations yields = C2A-^B2 =

X CjX j

‘

The last equation, together with Lemma 2, shows that X is optimal and that (III) holds. Thus the conditions (I) are sufficient conditions that X be optimal; similarly we can prove that the conditions (II) are sufficient. This completes the proof. We note that B 1 and C 1 do not appear in (I) or (II); thus these terms play no part in determining the sets of optimal vectors. A22 is the only term to appear in both (I) and (II); this double appearance thus expresses the sole connection between the sets of optimal vectors for the two problems. In the next theorem we let (1*)

A21X^ < B*,

and letA ^ , A ^ , C2, C* (2*)

A21,

A 3 1 X