Network Games, Control, and Optimization: Proceedings of NETGCOOP 2018, New York, NY [1st ed.] 978-3-030-10879-3, 978-3-030-10880-9

This contributed volume offers a collection of papers presented at the 2018 Network Games, Control, and Optimization con

347 40 8MB

English Pages XII, 287 [293] Year 2019

Report DMCA / Copyright

DOWNLOAD FILE

Polecaj historie

Network Games, Control, and Optimization: Proceedings of NETGCOOP 2018, New York, NY [1st ed.]
 978-3-030-10879-3, 978-3-030-10880-9

Table of contents :
Front Matter ....Pages i-xii
Pricing of Coexisting Cellular and Community Networks (Patrick Maillé, Bruno Tuffin, Joshua Peignier, Estelle Varloot)....Pages 1-16
Achieving Arbitrary Throughput–Fairness Trade-offs in the Inter-cell Interference Coordination with Fixed Transmit Power Problem (Vaibhav Kumar Gupta, Gaurav S. Kasbekar)....Pages 17-35
Coexistence of LTE-Unlicensed and WiFi with Optimal Channel Aggregation (Naveen Kolar Purushothama)....Pages 37-54
Analysis of Sponsored Data Practices in the Case of Competing Wireless Service Providers (Patrick Maillé, Bruno Tuffin)....Pages 55-70
Media Delivery Competition with Edge Cloud, Remote Cloud and Networking (Xinyi Hu, George Kesidis, Behdad Heidarpour, Zbigniew Dziong)....Pages 71-87
An Algorithmic Framework for Geo-Distributed Analytics (Srikanth Kandula, Ishai Menache, Joseph (Seffi) Naor, Erez Timnat)....Pages 89-105
The Stackelberg Equilibria of the Kelly Mechanism (Francesco De Pellegrini, Antonio Massaro, Tamer Başar)....Pages 107-123
To Participate or Not in a Coalition in Adversarial Games (Ranbir Dhounchak, Veeraruna Kavitha, Yezekael Hayel)....Pages 125-144
On the Asymptotic Content Routing Stretch in Network of Caches: Impact of Popularity Learning (Boram Jin, Jiin Woo, Yung Yi)....Pages 145-163
Tiered Spectrum Measurement Markets for Licensed Secondary Spectrum (Arnob Ghosh, Randall Berry, Vaneet Aggarwal)....Pages 165-182
On Incremental Passivity in Network Games (Lacra Pavel)....Pages 183-200
Impact of Social Connectivity on Herding Behavior (Deepanshu Vasal)....Pages 201-217
A Truthful Auction Mechanism for Dynamic Allocation of LSA Spectrum Blocks for 5G (Ayman Chouayakh, Aurélien Bechler, Isabel Amigo, Loutfi Nuaymi, Patrick Maillé)....Pages 219-232
Routing Game with Nonseparable Costs for EV Driving and Charging Incentive Design (Benoît Sohet, Olivier Beaude, Yezekael Hayel)....Pages 233-248
The Social Medium Selection Game (Fabrice Lebeau, Corinne Touati, Eitan Altman, Nof Abuzainab)....Pages 249-269
Public Good Provision Games on Networks with Resource Pooling (Mohammad Mahdi Khalili, Xueru Zhang, Mingyan Liu)....Pages 271-287

Citation preview

Static & Dynamic Game Theory: Foundations & Applications

Jean Walrand Quanyan Zhu Yezekael Hayel Tania Jimenez Editors

Network Games, Control, and Optimization Proceedings of NETGCOOP 2018, New York, NY

Static & Dynamic Game Theory: Foundations & Applications Series Editor Tamer Başar, University of Illinois, Urbana-Champaign, IL, USA Editorial Advisory Board Daron Acemoglu, MIT, Cambridge, MA, USA Pierre Bernhard, INRIA, Sophia-Antipolis, France Maurizio Falcone, Università degli Studi di Roma “La Sapienza,” Italy Alexander Kurzhanski, University of California, Berkeley, CA, USA Ariel Rubinstein, Tel Aviv University, Ramat Aviv, Israel; New York University, NY, USA William H. Sandholm, University of Wisconsin, Madison, WI, USA Yoav Shoham, Stanford University, CA, USA Georges Zaccour, GERAD, HEC Montréal, Canada

More information about this series at http://www.springer.com/series/10200

Jean Walrand Quanyan Zhu Yezekael Hayel Tania Jimenez •





Editors

Network Games, Control, and Optimization Proceedings of NETGCOOP 2018, New York, NY

Editors Jean Walrand University of California, Berkeley Berkeley, CA, USA Yezekael Hayel LIA/CERI Université d’Avignon Avignon, France

Quanyan Zhu Tandon School of Engineering New York University Brooklyn, NY, USA Tania Jimenez Laboratoire Informatique d’Avignon Université d’Avignon Avignon, France

ISSN 2363-8516 ISSN 2363-8524 (electronic) Static & Dynamic Game Theory: Foundations & Applications ISBN 978-3-030-10879-3 ISBN 978-3-030-10880-9 (eBook) https://doi.org/10.1007/978-3-030-10880-9 Library of Congress Control Number: 2018966117 © Springer Nature Switzerland AG 2019 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This book is published under the imprint Birkhäuser, www.birkhauser-science.com by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

This volume of Static & Dynamic Game Theory is a collection of the papers published in the International Conference on NETwork Games, COntrol and OPtimization (NetGCooP) 2018 conference. The event took place at the New York University Tandon School of Engineering, New York City, NY, USA, during November 14–16, 2018. Networks form the backbone of many complex systems, ranging from the Internet to social interactions. The proper design and control of networks have been a long-standing issue in various engineering and science disciplines. The vision of the conference is to provide a platform for researchers to share novel basic research ideas as well as network applications in control and optimization. Network control and optimization have been of increasing importance in many networking application domains, such as mobile and fixed access networks, computer networks, social networks, transportation networks, and more recently electricity grids and biological networks. Both conceptual and algorithmic tools are needed for efficient and robust control operation, for performance optimization, and for better understanding the relationships between entities that may be cooperative or act selfishly, in uncertain and possibly adversarial environments. The goal of NetGCooP is to bring together researchers from different areas with theoretical expertise in game theory, control, and optimization and with applications in the diverse domains listed above. During the conference, three plenary talks were given by three world-leading researchers: Leandros Tassiulas from Yale University on Optimizing the Network Edge for Flexible Service Provisioning; R. Srikant from University of Illinois at Urbana–Champaign on Network Algorithms and Delay Performance in Data Centers; and Gil Zussman from Columbia University on Power Grid State Estimation Following a Joint Cyber and Physical Attack. NetGCooP 2018 had 26 paper presentations, 11 selected from papers received through open calls and 15 via invitation. We thank the authors, the speakers, and the participants in the conference and hope that everyone enjoyed the conference. The success of the conference would not have been possible without the huge effort of several key people and organizations, especially the local organization committee in New York, the TPC members, and many students and volunteers who v

vi

Preface

contributed. Finally, we thank the Springer team, Benjamin Levitt and Christopher Tominich, for their confidence, help, and kindness in the publication process. General Co-Chairs: Jean Walrand (UC Berkeley); General Vice Co-Chair: Yezekael Hayel (University of Avignon) and Quanyan Zhu (New York University). TPC Co-Chairs: Rachid El-Azouzi (University of Avignon), Jianwei Huang (Chinese University of Hong Kong), and Ishai Menache (Microsoft Research). New York, USA November 2018

Rachid Elazouzi Jiamwei Huang Ishai Menache Tania Jimenez TPC Chairs

Contents

Pricing of Coexisting Cellular and Community Networks . . . . . . . . . . . Patrick Maillé, Bruno Tuffin, Joshua Peignier and Estelle Varloot Achieving Arbitrary Throughput–Fairness Trade-offs in the Inter-cell Interference Coordination with Fixed Transmit Power Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Vaibhav Kumar Gupta and Gaurav S. Kasbekar

1

17

Coexistence of LTE-Unlicensed and WiFi with Optimal Channel Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Naveen Kolar Purushothama

37

Analysis of Sponsored Data Practices in the Case of Competing Wireless Service Providers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Patrick Maillé and Bruno Tuffin

55

Media Delivery Competition with Edge Cloud, Remote Cloud and Networking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xinyi Hu, George Kesidis, Behdad Heidarpour and Zbigniew Dziong

71

An Algorithmic Framework for Geo-Distributed Analytics . . . . . . . . . . Srikanth Kandula, Ishai Menache, Joseph (Seffi) Naor and Erez Timnat

89

The Stackelberg Equilibria of the Kelly Mechanism . . . . . . . . . . . . . . . 107 Francesco De Pellegrini, Antonio Massaro and Tamer Başar To Participate or Not in a Coalition in Adversarial Games . . . . . . . . . . 125 Ranbir Dhounchak, Veeraruna Kavitha and Yezekael Hayel On the Asymptotic Content Routing Stretch in Network of Caches: Impact of Popularity Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 Boram Jin, Jiin Woo and Yung Yi

vii

viii

Contents

Tiered Spectrum Measurement Markets for Licensed Secondary Spectrum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 Arnob Ghosh, Randall Berry and Vaneet Aggarwal On Incremental Passivity in Network Games . . . . . . . . . . . . . . . . . . . . . 183 Lacra Pavel Impact of Social Connectivity on Herding Behavior . . . . . . . . . . . . . . . . 201 Deepanshu Vasal A Truthful Auction Mechanism for Dynamic Allocation of LSA Spectrum Blocks for 5G . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 Ayman Chouayakh, Aurélien Bechler, Isabel Amigo, Loutfi Nuaymi and Patrick Maillé Routing Game with Nonseparable Costs for EV Driving and Charging Incentive Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 Benoît Sohet, Olivier Beaude and Yezekael Hayel The Social Medium Selection Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 Fabrice Lebeau, Corinne Touati, Eitan Altman and Nof Abuzainab Public Good Provision Games on Networks with Resource Pooling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 Mohammad Mahdi Khalili, Xueru Zhang and Mingyan Liu

Program Committee

Tansu Alpcan, University of Melbourne Konstantin Avrachenkov, Inria Sophia Antipolis Randall Berry, Northwestern University Vivek Borkar, Indian Institute of Technology Bombay Peter Caines, McGill University Xu Chen, Sun Yat-sen University Kobi Cohen, Ben-Gurion University of the Negev Marceau Coupechoux, Telecom ParisTech Francesco De Pellegrini, Fondazione Bruno Kessler (FBK) Jelena Diakonikolas, Boston University Jocelyne Elias, Paris Descartes University Anthony Ephremides, University of Maryland Dieter Fiems, Ghent University Bruno Gaujal, Inria Majed Haddad, University of Avignon Longbo Huang, Tsinghua University Carlee Joe-Wong, Carnegie Mellon University Vijay Kamble, University of Illinois at Chicago Vasileios Karyotis, Institute of Communication and Computer Systems Veeraruna Kavitha, IIT Bombay George Kesidis, Pennsylvania State University Seong-Lyun Kim, Yonsei University Iordanis Koutsopoulos, Athens University of Economics and Business Samson Lasaulce, CNRS, Supelec Lasse Leskel, Aalto University Patrick Loiseau, University Grenoble Alpes, LIG Yuan Luo, Imperial College London Lorenzo Maggi, Huawei Technologies, France Research Center D. Manjunath, IIT Bombay Daniel Menasch, Federal University of Rio de Janeiro Panayotis Mertikopoulos, French National Center for Scientific Research (CNRS) ix

x

Dusit Niyato, Nanyang Technological University Lacra Pavel, University of Toronto H. Vincent Poor, Princeton University Balakrishna Prabhu, LAAS-CNRS Walid Saad, Virginia Tech Essaid Sabir, ENSEM, Hassan II University of Casablanca Gabriel Scalosub, Ben-Gurion University of the Negev Soumya Sen, University of Minnesota Alonso Silva, Nokia Bell Labs David Starobinski, Boston University Nicolas Stier, Facebook Core Data Science Yutaka Takahashi, Kyoto University Corinne Touati, Inria Mikal Touati, Orange Labs Bruno Tuffin, Inria Rennes—Bretagne Atlantique Ermin Wei, Northwestern University Sabine Wittevrongel, Ghent University Dejun Yang, Colorado School of Mines Haoran Yu, Northwestern University

Program Committee

Author Index

A Alpcan, Tansu 54 Altman, Eitan 1, 12, 23, 140 B Bachmann, Ivana 24 Belhadj Amor, Selma 35 Belmega, Elena Veronica 149 Berri, Sara 45 Bustos-Jiménez, Javier 24 C Chen, Yaojun 179 Chorppath, Anil Kumar 54 Courcoubetis, Costas 63 D De Pellegrini, Francesco 72 Debbah, Mérouane 102 Dimakis, Antonis 63 Douros, Vaggelis 82 F Fijalkow, Inbar 149 G Goratti, Leonardo 72 Grammatico, Sergio 93 Gubar, Elena 170

H Hamidouche, Kenza 102 Hasan, Cengis 111 Hayel, Yezekael 1, 140 J Jain, Atulya 1, 12 Jorswieck, Eduard 54 Ju, Min 121 K Kanakakis, Michalis 63 Krishna Chaitanya, A 131 L Lasaulce, Samson 45, 190 Legenvre, François-Xavier 140 M Marcastel, Alexandre 149 Marina, Mahesh K. 111 Massaro, Antonio 72 Mertikopoulos, Panayotis 149 Morales, Fernando 24 Mukherji, Utpal 131 P Perlaza, Samir M. 35 Polyzos, George 82

xi

xii

R Rachid, El-Azouzi 72 Radjef, Mohammed Said 45 S Saad, Walid 102 Sharma, Vinod 131 Shimkin, Nahum 12 Silva, Alonso 24, 158 T Taynitskiy, Vladislav 170 Torres-Moreno, Juan-Manuel 121 Touati, Corinne 12, 23 Toumpis, Stavros 82

Author Index

V Varma, Vineeth 45, 190 W Wu, Haitao 179 X Xiao, Shilin 121 Z Zappone, Alessio 54 Zhang, Chao 190 Zhou, Fen 121, 179 Zhu, Quanyan 170 Zhu, Zuqing 179

Pricing of Coexisting Cellular and Community Networks Patrick Maillé, Bruno Tuffin, Joshua Peignier and Estelle Varloot

Abstract Community networks have emerged as an alternative to licensed band systems (WiMAX, 4G, etc.), providing an access to the Internet with Wi-fi technology while covering large areas. A community network is easy and cheap to deploy, as the network is using members’ access points in order to cover the area. We study the competition between a community operator and a traditional operator (using a licensed band system) through a game-theoretic model, while considering the mobility of each user in the area.

1 Introduction Wireless technologies are becoming ubiquitous in Internet usage. Operators try to provide a whole wireless coverage on urban areas, in order to offer an Internet access to everyone, with a guaranteed quality. However, this system requires huge investment costs in terms of infrastructure and spectrum licenses. This has repercussions on the subscription fees, which can be large enough for users to prefer other options. Due to this, community networks [6] have been imagined as an alternative. The principle is simple: When a user subscribes to a community network, he sets an access point where he lives (and is responsible for its maintenance), which can be used by all members of the community network. As a counterpart, he gains access to the P. Maillé IMT Atlantique / IRISA, Rennes, France e-mail: [email protected] B. Tuffin (B) Inria, Rennes, France e-mail: [email protected] J. Peignier · E. Varloot ENS Rennes, Rennes, France e-mail: [email protected] E. Varloot e-mail: [email protected] © Springer Nature Switzerland AG 2019 J. Walrand et al. (eds.), Network Games, Control, and Optimization, Static & Dynamic Game Theory: Foundations & Applications, https://doi.org/10.1007/978-3-030-10880-9_1

1

2

P. Maillé et al.

Internet through every access point belonging to the community network. This approach presents the advantage that the infrastructure is cheaper and easier to maintain, from a provider perspective. However, the quality of service cannot be guaranteed, since it depends on the size of the community. Currently, the largest community operator is FON.1 From the user point of view, a community network has the particularity of having both positive and negative externalities; i.e., having more subscribers is both beneficial (larger coverage when roaming) and a nuisance (more traffic to serve from one’s access point). An analysis of those effects and on the impact of prices, with users being heterogeneous in terms of their propension to roam, is carried out in [1]. In the present paper, we add another dimension, that is, how–i.e., where–users roam. Also, we consider that users can choose between two competing providers, a “classical” one and one operating a community network, that compete over prices. Community networks have already been studied under a game-theoretic framework, with operators as players. In [3], the authors first study how a community network evolves, depending on its initial price and coverage, and then investigate using a game-theoretic framework [7] the repartition of users having the choice between a community network and an operator on a licensed band. The competition is first studied when each player decides its price once and the size of the community network changes over time. Then, a discrete-time dynamic model is studied, where operators can change their price at each time step, taking into account the preferences of the users concerning price and coverage. The authors show the existence of one or several Nash equilibria under specific conditions. An extension in [4] investigates whether it is profitable for a licensed band operator to complement the service it provides with a community network service. It is shown that this is generally not the case, as users will more likely choose the (less profitable) community network. In [5], the same authors study an optimal pricing strategy for a community network operator alone in both static and semi-dynamic models, while considering a mobility factor for each user (e.g., each user makes requests, but not all in the same spot). They also allow the operator to set different prices for each user. In the following, we will refer to the traditional operator as the classical Internet service provider (ISP). In this article, we study a model similar to both [3, 5]. In [3], the users all present the same characteristics, while in [5] there is a mobility factor but the paper considers a community network alone. We consider here a more general and realistic framework: Users are considered located in places heterogeneous in terms of attractiveness for connections (an urban area is more likely to see connections than the countryside). Moreover, their mobility behavior is also heterogeneous: They do not all plan to access the Internet from the same places. Instead of a mobility parameter, we rather consider a density function, which represents the probability that a user makes a request while being near the access point of another user. But in our paper, all users will have the same sensitivity toward quality; indeed, our goal is rather to focus on the impact of geographical locations of spots and connections on users’ subscription and on the competition between the operator and the community network. The model 1 https://fon.com/.

Pricing of Coexisting Cellular and Community Networks

3

is analyzed using noncooperative game theory [2, 7]. The decisions are taken at different timescales: First, the networks fix their price, and then users choose which network to subscribe to. We illustrate on different scenarios that for fixed subscription prices to the ISP and the community network, several equilibria on the repartition of users can exist; the one we can expect depends on the initial mass of the subscribers to the community network. The pricing competition between operators is played anticipating the choice of users. The paper is organized as follows. Section 2 presents the model; the basic notions are taken from the literature, but we extend it with the modeling of mobility via a continuous distribution. In Sect. 3, we describe how, for fixed subscription prices, the repartition of users is determined. In Sect. 4, we introduce the pricing game between the operator and the community network, as well as our method to compute a Nash equilibrium. Section 5 presents two scenarios as examples of application of our method.

2 Model Definition We present here the basic elements of the model taken from the literature, mainly [3], which we complement with more heterogeneity among users related to location and mobility, as well as the possible nuisance from providing service to other community members.

2.1 Actors and Strategies To study the competition between a community operator and a classical ISP, we need to define a model for the profit of each operator, but also for users, in order to explain what will make them choose a service rather than the other one. The decisions of these actors are taken at different timescales, defining a multilevel game: 1. First, the classical ISP and the community network play noncooperatively on the subscription prices, in order to maximize their own revenue (expressed as the product of price and mass of users). 2. Given the prices and qualities of service, users choose their network based on price (we assume a flat-rate pricing is applied by each operator) and quality of service. We will describe how, depending on an initial repartition between operators, users can switch operators up to a situation when nobody has an interest to move. The results of the game for the operators are given once all users have settled on an operator (if any). Even though the operators play first (subscription is impossible until the price of subscription is set), they are assumed to make their decision strategically, anticipating the subsequent decisions of users. Hence, the game is analyzed

4

P. Maillé et al.

by backward induction [7]: We determine the user choices for any fixed prices and consider that operators are able to compute those choices when selecting their prices.

2.2 Modeling of Users, Quality, and Mobility We consider a continuum of users characterized by their type u; this type typically represents a home location. In the following, we will not distinguish between a location u and the user  u living there. Let Ω be the space of users and f their density over space Ω (with Ω f (v) dv = 1). We also assume that Ω is the support of f , i.e., f (u) > 0 for all u ∈ Ω. Let D be the subset of Ω of users subscribing to the community network; we call it the domain, since it also represents the domain of coverage of the community network. Each user makes requests when using the Internet. Let m(v) be the average number of communication requests of user v per time unit. Depending on their location, users may also present different mobility patterns. To express this heterogeneity, define for each user u the density function g(v|u) that a request from u occurs at v. Note that users may move to uninhabited regions: We aggregate those regions into one item, denoted by ⊥, and we define the set of mobility ¯ locations as Ω¯ := Ω ∪ ⊥. Then, over a location area  A ⊂ Ω, the probability that a ¯ type u user’s request is in A (rather than Ω \ A) is A g(v|u) dv. If we define n(u) as the density (number per space unit) of requests at u from users of the community network, it can be computed as  n(u) =

g(u|v)m(v) f (v) dv. D

The quality of a given service is defined as the probability that a request is fulfilled. For the ISP assumed to have a full coverage, it is therefore 1, in line with the literature. For a user u, the quality of the community network will depend on whether the requests by u are generated when in the coverage domain; hence, it can be computed as  g(v|u)dv. (1) qu = D

2.3 User Preferences How will user u decide whether to subscribe to the classical ISP, to the community network, or to none of them? Following [3, 4], define U I (u), UC (u), and U∅ (u) as the respective utility functions for choosing the classical ISP, the community network, or none. These functions depend on the price the user has to pay, the quality of the

Pricing of Coexisting Cellular and Community Networks

5

service he is provided, and his sensitivity toward quality. As in [3, 4], we consider a simple quasi-linear form for utilities: A user u, whose sensitivity toward quality is denoted by a and who benefits from service quality q˜ (assumed in the interval [0, 1]) at price p, perceives a utility au q˜ − p. Note that in [3, 4] the sensitivity parameter au depends on the user type, u, but we limit ourselves in this paper to a constant value a for all users since our goal is to focus on the geographical heterogeneity of users. In addition, as in [1] we consider a disturbance factor for the community network: Satisfying requests for other members can indeed become an annoyance, which we model through a negative term -cn(u) in the utility function, with c a unit cost per request at u. Here, we assume that the nuisance is due to Wi-fi spectrum usage; hence, it depends on the total density of requests in u and is independent of the density of users at u. Let p I and pC be the respective flat-rate subscription fees to the ISP and to the community network, respectively. We assume users are rational: A type u user will choose the network providing the largest utility (or no network), where the utilities at the community network, the ISP, or for not subscribing to any are, respectively (recall that those functions depend on the set D of users in the community network), UC (u) = aqu − pC − cn(u) U I (u) = a − p I U∅ (u) = 0, where U∅ (u) is used to say that users with negative utilities at the operators do not subscribe to any of them; we also assume that users with null utilities at the operators do not subscribe. In the following, we will therefore always assume that p I < a, because otherwise the classical operator would get no subscriber (as for all u, we would have U I (u) ≤ 0 and all users prefer the no-subscription option over the classical operator). With the same argument, we assume that pC < a. We now have, for all u, U I (u) > 0, which implies that each user will necessarily subscribe to one operator, since they strictly prefer the classical operator over the no-subscription option. However, the repartition between the classical ISP and the community network is not trivial, since the utilities expressed above, which determine user choices, also depend on user choices through the set D. Hence the notion of equilibrium (or fixed point), which we define and analyze in Sect. 3.

2.4 Operators’ Model The utilities for the classical ISP and the community operator are simply defined as their profits. For each operator, the profit depends on the price it chose and on the number of users subscribing to its service, which depends on both prices.

6

P. Maillé et al.

Let d I and dC be the number (or mass) of users subscribing, respectively, to the classical ISP and to the community operator. For a set D ⊂ Ω of users subscribing to the community network (which depends on prices as we see later on), those masses can be written as  dI = f (v)dv Ω\D  f (v)dv. dC = D

The utilities are then expressed by VC = dC pC VI = d I p I − χ I , where χ I is the infrastructure cost for the ISP. Each operator chooses (plays with) its price to maximize its revenue, but that revenue also depends on the decision of the competing operator which can attract some customers, hence the use of noncooperative game theory to solve the problem.

3 User Equilibrium With the characterization of user behavior above, we aim in this section at determining if, for fixed subscription prices, there is an equilibrium user repartition among operators and also if it is unique. We first define what such an equilibrium is. We consider here that prices p I and pC have already been decided.

3.1 Definition and Characterization Definition 1 A user equilibrium domain is a domain D ⊂ Ω such that no user, in D or in Ω\D, has an interest to change his choice of network. Mathematically, this means that ∀u ∈ D UC (u) ≥ U I (u) ∀u ∈ Ω\D UC (u) ≤ U I (u). Consider a user u. For a given domain D, he will prefer the community network if UC (u) ≥ U I (u), that is, if a(qu − 1) + ( p I − pC ) − cn(u) ≥ 0. Let us define the domain-dependent function Φ D : Ω → R as the difference UC (u) − U I (u), that is,

Pricing of Coexisting Cellular and Community Networks



7

  g(v|u)dv − 1 + ( p I − pC ) − c g(u|v)m(v) f (v) dv.

Φ D (u) := a D

(2)

D

 Then, D is a user equilibrium domain if and only if

Φ D (u) ≥ 0 ∀u ∈ D Φ D (u) ≤ 0 ∀u ∈ Ω \ D.

Example 1 Consider the case of users with homogeneous mobility behavior, that is, where g(v|u) does not depend on u so we only denote it by g(v). From (1), we network users also get that the quality qu does not depend on u: All the community  g(v)dv). Moreover, experience the same quality, which we denote by q and equals D   n(u) = g(u) D m(v) f (v)dv = Mg(u) with M := D m(v) f (v)dv the total request mass from community network users. At a user equilibrium D, user u prefers the community network if and only if a(q − 1) + ( p I − pC ) − cMg(u) ≥ 0. The domain D is then made of all users u with attractiveness g(u) below a threshold.

3.2 Existence and Uniqueness Proposition 1 A user equilibrium is not unique in general. Proof An example of nonuniqueness is shown in Sect. 5.1 when users present a homogeneous mobility pattern g(v|u) = g(v) ∀u.  Proposition 2 D = ∅, that is no user subscribes to the community network, is a user equilibrium (but not necessarily the only one) if and only if p I ≤ pC + a. In other words, if the price difference between the community network and the ISP is not large enough (it has to be larger than a), nobody subscribing to the community network is a user equilibrium, even if not necessarily the unique possibility. Proof D = ∅ is a user equilibrium if and only if, when there are no community network users, UC (u) ≤ U I (u) ∀u ∈ Ω; that is, −a + ( p I − pC ) ≤ 0, i.e., p I ≤  pC + a. We can also consider the other case of “degenerate” equilibrium, that is, when all users in Ω subscribe to the community network. Proposition 3 D = Ω is a user equilibrium if and only if ΦΩ (u) ≥ 0 for all u ∈ Ω. Corollary 1 Under our assumption p I < a, there always exists at least one user equilibrium. Proof Since we have assumed p I < a, Proposition 2 holds and nobody subscribing to the community network is an equilibrium. 

8

P. Maillé et al.

3.3 A Dynamic View Given that several user equilibria might exist, which one would be observed in practice? This may depend on a dynamic evolution of subscriptions: We can study how users make their choice and how the repartition evolves, depending on an initial situation. If a user u is associated with the ISP (resp. community network) but UC (u) > U I (u) (resp. UC (u) < U I (u)), then it will switch to the other operator. Without loss of generality, we can first partition users by assuming that those with the largest UC (u) − U I (u) subscribe to the community network and the others to the ISP (a natural move to that situation will occur otherwise). We can relate this to the function / D (resp. u ∈ D) with the largest value Φ D (x) defined in (2). For a given D, users u ∈ Φ D (u) > 0 (resp. lowest value Φ D (u) < 0) will have an incentive to switch operator and join (resp. leave) D. Hence, D will change up to a moment when no user has an interest to move, that is, up to reaching a user equilibrium as defined above. All this will be made more specific and clearer in Sect. 5 on the analysis of two scenarios. Depending on the initial situation (that is, the initial mass of users subscribing to the community network), we may end up in different user equilibria. We can assume that the community network will offer free subscriptions or make offers to users so that an initial point will allow to lead to different equilibria.

3.4 Stability Among user equilibrium domains, some are more likely to be observed. They are the so-called stable user equilibrium domains, which can basically be defined as domains that are stable to small perturbations in the following sense. Definition 2 A user equilibrium domain D is said to be stable if there exists ε > 0 such that ∀D with (D∪D )\(D∩D ) f (v) dv ≤ ε (that is, any D with “measure” close enough to D); then starting from D , the user repartition will converge to D. The following straightforward result establishes that there always exists at least one stable equilibrium. Proposition 4 If for all u ∈ Ω, the ratio of the densities g(·|u)/ f (·) is upperbounded on Ω, then for any price profile ( p I , pC ) with p I < a, the situation D = ∅ is a stable equilibrium. Similarly, the other degenerate equilibrium D = Ω is stable if ΦΩ (u) > 0 for all u ∈ Ω. Proof From Corollary 1, D = ∅ is a user equilibrium domain. Since p I < a, (2) yields Φ D (u) = p I − a − pC < 0 ∀u ∈ Ω. We also have from (2) that for any domain D and any u,

Pricing of Coexisting Cellular and Community Networks

9

 Φ D (u) ≤ p I − a − pC + a

g(v|u) dv. D

But by some value L, then the integral   when the ratio g(·|u)/ f (·) is upper-bounded I than a L f (v) dv. Therefore, with ε < pC +a−P , for a a D g(v|u) dv is smaller D aL  domain D such that D f (v) dv ≤ ε all users in D would be better off switching back to the ISP; hence, D = ∅ is a stable equilibrium domain.  Further characterizations are provided in Sect. 5 for specific scenarios.

4 Pricing Game We study here the prices that are set by both network operators. To do so, we assume that they are able to anticipate the consequences of their pricing decisions, so that we can apply the backward induction approach to analyze the pricing game [7]. For any price pair ( p I , pC ), being able to characterize all stable user equilibria, we can reasonably assume that the community network will set up things (again, by initial offers/bargains) such that the largest (defined in terms of demand dC ) stable user equilibrium domain is reached in the end. Therefore, for given prices, we will be able to compute the corresponding values of the utility functions VC and VI of each operator. Hence, providers can noncooperatively play the pricing game where the community network chooses pC and the ISP chooses p I , each operator trying to maximize its utility function. The solution concept is the classical Nash equilibrium [7], a pair ( p ∗I , pC∗ ) from which no provider can improve its revenue from a unilateral price change. Note that to anticipate the impact of one’s decisions, the operators have to have some knowledge about user preferences and (mobility) behaviors, an assumption we make here. In practice, that behavior can be learnt by the operator to which the user subscribes (following which access point the user is connected), and/or from more general survey that are out of the scope of this paper.

5 Analysis and Discussion of Two Scenarios 5.1 Users with a Homogeneous Mobility Pattern We first consider the simplest situation where the mobility pattern is the same for all users, which means that g(v|u) does not depend on u, that is g(v|u) = g(v) ∀u as treated in Example 1. From this assumption, qu = q does not depend on u (but still depends on D). We also get a much simpler expression for Φ D , which is now: ∀u ∈ Ω, (3) Φ D (u) = a(q − 1) + ( p I − pC ) − cMg(u),

10

P. Maillé et al.

 which depends on u only through the term g(u), with q = D g(v) dv and M =  D m(v) f (v) dv. From such an expression, we can show that with an homogeneous mobility pattern, user equilibria have a specific form.

5.1.1

Characterization of User Equilibria

Proposition 5 Assume that location attractiveness values are distributed regularly over Ω: i.e., mathematically, that for all y ∈ R+ , the mass of users with the specific value g(u) = y is null. Then, a nondegenerate user equilibrium domain has the form Dx := {u ∈ Ω | g(u) ≤ x} for a given x ≥ 0, with x solution of  a 

Dx

  g(v) dv − 1 + p I − pC − cx m(v) f (v) dv = 0. Dx 

:=Ψ (x)

In the above characterization, x is a threshold such that all users u with mobility attractiveness density g(u) below x subscribe to the community network. Remark that Ψ (x) corresponds to Φ Dx (u) for a user u such that g(u) = x; since Dx is continuous in x under our assumption, Ψ is also a continuous function of x. Proof See Appendix.



We can characterize, among all domains Dx , which ones will actually be user equilibrium domains, and the corresponding dynamics. Assume that the set of subscribers to the community network is of the form Dx = {u ∈ Ω : g(u) ≤ x} for some x ∈ R+ . • If Ψ (x) > 0 and Dx = Ω, it means that users u with g(u) just above x are associated with the ISP and are those with the largest utility difference and incentive to switch to the community network (indeed, from (3) that utility difference Φ Dx (u) is continuous and strictly decreasing in g(u)); hence, they switch such that x and Dx increase. • If Ψ (x) < 0 and Dx = ∅, it is the opposite situation: Users u with value g(u) just below x are with the community network but have the largest incentive to switch to the ISP; hence, x and Dx decrease. • If Ψ (x) = 0, all users u ∈ Ω are such that Φ Dx (u) ≥ 0 and hence have no interest to switch; we are then in an equilibrium situation. We thus end up with the following characterization of user equilibrium domains. Let y := supu∈Ω {g(u)} (possibly ∞), such that D y = Ω. • If Ψ (y) ≥ 0, then D = Ω (all users subscribe to the community network) is an equilibrium. • Since Ψ (0) = −a + p I − pC ≤ 0 by assumption, ∅ is always a user equilibrium domain (no users associated with the community network). • If Ψ (x) = 0, Dx is a user equilibrium domain.

Pricing of Coexisting Cellular and Community Networks

5.1.2

11

Stable Equilibria

Among all user equilibrium domains, we can characterize the stable ones. Proposition 6 As suggested by the dynamics described in Sect. 3.3, we consider that the community network subscriber set D is always of the form D y for some y. Then, if Ψ (x) = 0 and Ψ (x) < 0, Dx is a stable equilibrium. Proof Assume a small variation, from x to x = x ± ε, in D (hence, from Dx to Dx ). If Ψ (x) < 0, for ε small enough, Ψ (x ) > 0 (resp. < 0) if x < x (resp. x > x); hence, users u with g(u) between x and x are incentivized to switch back to their  initial choice, driving back to the (then stable) equilibrium domain Dx .

5.1.3

Nash Equilibria for the Pricing Game Between Operators

For any pair ( pC , p I ), we consider that the largest equilibrium domain is selected. Operators then play a noncooperative game to determine their optimal strategy [2, 7]. The output concept is that of a Nash equilibrium, a point ( pC∗ , p ∗I ) such that no operator has an interest to unilaterally move from, because it would decrease its utility (revenue). Due to analytical intractability, we are going to study the existence and characterize Nash equilibria numerically, for specific parameter values; the procedure can be repeated for any other set of parameters.

5.1.4

Examples

We first show situations where there are several user equilibria and even several stable user equilibria. We then discuss the solution of the pricing game between operators. Example 2 Consider Ω¯ = Ω = R+ ; i.e., users are placed over the positive line (negative values could be described as the sea). We assume: • m(u) = 1, i.e., all users generate the same amount of requests. • f (u) = α/(1 + u)1+α with α > 0. In other words, users are located according to a Pareto distribution, potentially with infinite expected value. The closer the 0 value (which can be thought of as the town center), the more users you can find. • g(u) = λe−λu , meaning that connections are exponentially distributed with rate λ, with more connections close to 0; even faraway users are more likely to require connections there. With these functions, noting that g is strictly decreasing, the set Dx is simply the interval [ln(λ/x)/λ, +∞) when x ∈ (0, λ], Dx = Ω when x > λ, and Dx = ∅ when x = 0. It gives  Ψ (x) = −a(1 − x/λ) + p I − pC − cx

1 1 + ln(λ/x)/λ



12

P. Maillé et al.

Ψ (x)

0 0 0

0

0.2 0.4 0.6 0.8 x

1

0

0.2

0

0.4

0.5

x

1 x

2

1.5

Fig. 1 Ψ (x) when λ = 1.0 (left), when λ = 0.5 (center), and when λ = 2.0 (right) Fig. 2 Best responses to the pricing game when λ = 0.25, α = 1.2, a = 1, c = 1, and χ I = 0 pC

1

BRCom (pI ) BRISP (pC )

0.5

0 0

0.2

0.4

0.6

0.8

1

pI

for x ∈ [0, λ], Ψ (x) = p I − pC − cx for x > λ, and Ψ (0) = p I − a − pC . Note that the assumption made in Proposition 5 holds here; hence, Ψ is continuous over R+. Three outcomes are illustrated in the next three cases for λ when α = 1.2, a = 1, c = 1, p I = 0.95, and pC = 0.1. In Fig. 1 (left), there are two solutions for Ψ (x) = 0, with only the second one leading to a stable equilibrium domain (in addition to D0 = ∅, which is stable too from Proposition 4, for which the assumptions also hold). In Fig. 1 (center), Ψ (x) = 0 has only one (unstable) solution, but D0 = ∅ and D0.5 = Ω (because Ψ (0.5) > 0) are stable equilibrium domains. In Fig. 1 (right), there is no solution to Ψ (x) = 0, and ∅ is the only equilibrium domain. Remark that Ω is a (stable) equilibrium domain if Ψ (λ) = p I − pC − cλ ≥ 0.2 Assuming now that the community network plays such that the largest equilibrium domain is selected (thanks to discounts for example), we can draw the best responses to the pricing game between operators. It is, for example, displayed in Fig. 2 for specific parameter values, here when the infrastructure cost for the ISP is χ I = 0. 2 Note

that with other functions, depending on the variations of f and g, an arbitrary number of solutions of Ψ (x) = 0 can be obtained.

Pricing of Coexisting Cellular and Community Networks

13

With these parameters, the community network is able to get a positive demand only if p I ≥ 0.76. dC is then jumping from 0 to 0.574 and is readily and slightly increasing to 0.678 when p I = 0.99. We actually have here a price war where each operator has an interest to give a price just below that of the opponent, and we end up with a Nash equilibrium ( p I = 0.23985, pC = 0) where one operator stops with the zero price. Due to price war, only one operator survives. With an infrastructure cost χ I = 0, it is the ISP, but we note that the threat of the development of a community network significantly decreases the price set by the ISP (whose monopoly price was 1 as can be seen in Fig. 2 when pC is prohibitively high). Also, there is a threshold on χ I over which it is the community network that survives. Indeed, remark that the value of χ I does not change the best-response values since it appears as a constant in the expression of VI , but the game stops as soon as one of the two providers gets a zero revenue. The revenue of the ISP will go to zero before that of the community network if χ I > 0.23.

5.2 Several Populations We slightly modify the model such that Ω = R+ with a mass of users in 0, seen as a town (probability/mass π0 ), while others with u > 0 are regularly distributed over the “countryside” with conditional density f . In terms of mobility, we assume that users at u = 0 do not move, while those at u > 0 have a probability π1 to call from 0 and a conditional density (when connecting from another place) g(v) to make a connection from v > 0. With those assumptions, remark that q0 = 1 (resp. is chosen (resp., not chosen) at 0, and for u > 0, q0 = 0) if the community network  qu = π1 1l{0∈D} + (1 − π1 ) D1 g(v)dv where D 1 = D ∩ (0, ∞) is D excluding 0 and 1l{·} is 1 if the condition is satisfied and 0 otherwise. The number of connections to a member of the community network at 0 (assuming 0 ∈ D) is then n(0) = π0 m(0) + (1 − π0 )π1 D1 m(v) f (v)dv and n(u) = g(u)(1 − π0 ) D1 m(v) f (v)dv for u > 0. The level of annoyance (interferences, etc.) is again assumed linear in n(·), leading to Φ D (0) = ( p I − pC )    m(v) f (v)dv −c π0 m(0) + (1 − π0 )π1 D1    g(v)dv − 1 Φ D (u) = a π1 1l{0∈D} + (1 − π1 ) D1  m(v) f (v)dv). +( p I − pC ) − c(g(u)(1 − π0 ) D1

Assuming that 0 ∈ D or not, the above last equation tells us as in the previous subsection that D 1 is of the form Dx1 = {u : g(u) ≤ x} for some value x. Exactly as in the previous homogeneous case, there might be several solutions, and we will

14

P. Maillé et al.

assume that the selected one will be a stable one leading to the largest market share (revenue) for the community network. The more the subscribers of the community network, the less likely users in 0 will subscribe because since they do not move, they experience only losses from an increased number of subscribers. But there might be a risk of oscillations on the user equilibrium. Indeed, if x is small enough, then users in 0 subscribe. This increases the interest for others to subscribe, leading to a larger value of x, which might deter users in 0, and so on. Example 3 We again consider m(u) = 1, f (u) = α/(1 + u)1+α with α > 0, and g(u) = λe−λu . Then, users in 0 join iff for D 1 = Dx1 = [ln(λ/x)/λ, +∞), they prefer the community network over the ISP, knowing that there is coverage in 0, i.e., iff  α   1 ( p I − pC ) − c π0 + (1 − π0 )π1 ≥ 0, 1 + ln(λ/x)/λ  

:=Φ D (0)

or x ≤ λe

1/α (1−π )π1 λ 1− ( p − p 0)/c−π I

C

0

.

Note that Φ D is a slight abuse of notation, since it corresponds to D = {0} ∪ Dx1 or just Dx1 whether users at 0 join. To get a nondegenerate equilibrium, x ∈ [0, λ] must be a solution of 

Ψ (x) = a π1 1l{0∈D} − (1 − π1 )(1 − x/λ) + p I − pC α  1 −cx(1 − π0 ) =0 1 + ln(λ/x)/λ leading to curves of ψ(·) similar to what we got in Example 2. Again, several solutions are possible, so we assume that the largest stable domain D1 is reached, thanks to a valid choice of initial conditions. Let D10 (resp. D1∅ ) denote D1 when 0 is (resp. is not) choosing the community network. There are three possibilities depending on Φ D (0) and D1 : • If Φ D (0) ≥ 0 when D = {0} ∪ D10 , it means that users 0 are satisfied with the gain in price to be in the community network, and D = {0} ∪ D10 is the considered user equilibrium domain. • If Φ D (0) < 0 when D = D1∅ , users in 0 have no interest in joining the community network and D = D1∅ is an equilibrium. • If Φ D (0) < 0 when D = {0} ∪ D10 and Φ D (0) > 0 when D = D1∅ , there are oscillations in the user equilibrium domain, which does not exist with the chosen strategy from the community network. In that case, we are going to consider that when computing best responses to the pricing game, operators will make use of the “worst” scenario for them in the pricing game, in terms of market share: D = D1∅ for the community network, and D = {0} ∪ D10 for the ISP.

Pricing of Coexisting Cellular and Community Networks Fig. 3 Best responses to the pricing game with heterogeneous users when λ = 0.25, α = 1.2, a = 1, c = 1, π0 = 0.7, π1 = 0.1, and χ I = 0

pC

1

15

BRCom (pI ) BRISP (pC )

0.5

0 0

0.2

0.4

0.6

0.8

1

pI

Using the above user equilibrium domain, we can draw the best responses to the pricing game between operators. It is, for example, displayed in Fig. 3 for specific parameter values when the infrastructure cost for the ISP is χ I = 0. With these numerical values, the best response of the ISP to the community network price pC is such that d I = 1 for pC ≤ 0.63, and d I = 0.7 (just users in 0) when pC is above that value. We again have a price war (each provider setting its price just below its opponent) in the high-price region. Predicting the outcome of the competition is not trivial, since following a best-response dynamics leads to a cycle (appearing in the top right corner in the figure): Prices slide downward until reaching ( p I , pC ) ≈ (0.71, 0.63), at which point the ISP sets p I = 1, reinitiating the cycle.

Appendix All Domains on Equilibrium Situations Have the Form Dx for Homogeneous and Regular Mobility Patterns Proof At an equilibrium situation, the community domain D must be such that ∀u ∈ D, ∀u ∈ / D,

Φ D (u) ≥ 0 Φ D (u) ≤ 0.

Let p := p I − pC . Under the assumption that g(v|u) does not depend on u and with the expressions of q and n we had before, we now have ∀u ∈ Ω, 

  g(v)dv − 1 + p − cg(u) m(v) f (v)dv.

Φ D (u) = a D

D

16

P. Maillé et al.

So, an equilibrium domain D should be made of u such that  g(v) dv − 1 , g(u) < a D c D m(v) f (v) dv plus possibly some of the users for which there is equality above, but which are of measure 0 under our assumption. Hence, the general form of the solution Dx = {u ∈ Ω | g(u) ≤ x}. Using that form Dx for candidate domains, one can write Φ Dx (u) as a function of x and g(u), and an equilibrium is reached when the users wanting to subscribe to the community network (currently  made of Dx ) are exactlyDx , i.e., when x is a root of the function Ψ (x) := a Dx g(v)dv − 1 + p − cx Dx m(v) f (v) dv. 

References 1. M. H. Afrasiabi and R. Guérin. Exploring user-provided connectivity. IEEE/ACM Trans. on Networking, 24(1), 2016. 2. P. Maillé and B. Tuffin. Telecommunication Network Economics: From Theory to Applications. Cambridge University Press, 2014. 3. M. H. Manshaei, J. Freudiger, M. Felegyhazi, P. Marbach, and J.-P. Hubaux. On wireless social community networks. In Proc. of IEEE INFOCOM, Phoenix, AZ, USA, 2008. 4. M. H. Manshaei, P. Marbach, and J.-P. Hubaux. Evolution and market share of wireless community networks. In Proc. of IEEE GameNets, Istanbul, Turkey, 2009. 5. A. Mazloumian, M. H. Manshaei, M. Felegyhazi, and J.-P. Hubaux. Optimal pricing strategy for wireless social community networks. In Proc. of ACM NetEcon, Seattle, WA, USA, 2008. 6. P. Micholia, M. Karaliopoulos, I. Koutsopoulos, L. Navarro, R. Baig, D. Boucas, M. Michalis, and P. Antoniadis. Community networks and sustainability: a survey of perceptions, practices, and proposed solutions. IEEE Communications Surveys & Tutorials, 2018. 7. M. J. Osborne and A. Rubinstein. A course in game theory. MIT Press, 1994.

Achieving Arbitrary Throughput–Fairness Trade-offs in the Inter-cell Interference Coordination with Fixed Transmit Power Problem Vaibhav Kumar Gupta and Gaurav S. Kasbekar Abstract We study the problem of inter-cell interference coordination (ICIC) with fixed transmit power in OFDMA-based cellular networks, in which each base station (BS) needs to decide as to which subchannel, if any, to allocate to each of its associated mobile stations (MS) for data transmission. In general, there exists a trade-off between the total throughput (sum of throughputs of all the MSs) and fairness under the allocations found by resource allocation schemes. We introduce the concept of τ − α−fairness by modifying the concept of α−fairness, which was earlier proposed in the context of designing fair end-to-end window-based congestion control protocols for packet-switched networks. The concept of τ − α−fairness allows us to achieve arbitrary trade-offs between the total throughput and degree of fairness by selecting an appropriate value of α in [0, ∞). We show that for every α ∈ [0, ∞) and every τ > 0, the problem of finding a τ − α−fair allocation is NP-complete. Also, we propose a simple, distributed subchannel allocation algorithm for the ICIC problem, which is flexible, requires a small amount of time to operate, and requires information exchange among only neighboring BSs. We investigate via simulations as to how the algorithm parameters should be selected so as to achieve any desired trade-off between the total throughput and fairness. Keywords Cellular networks · Inter-cell interference coordination · Complexity Algorithms · Fairness

1 Introduction The long-term evolution (LTE)-advanced cellular system, which is a 4G technology that is being extensively deployed throughout the world, relies on orthogonal frequency-division multiple access (OFDMA) technology [7]. Often, an OFDMAV. K. Gupta (B) · G. S. Kasbekar Indian Institute of Technology Bombay, Mumbai, India e-mail: [email protected] G. S. Kasbekar e-mail: [email protected] © Springer Nature Switzerland AG 2019 J. Walrand et al. (eds.), Network Games, Control, and Optimization, Static & Dynamic Game Theory: Foundations & Applications, https://doi.org/10.1007/978-3-030-10880-9_2

17

18

V. K. Gupta and G. S. Kasbekar

based cellular network is deployed with frequency reuse factor one; i.e., the entire available frequency band can be potentially used in all the cells. Also, the dense deployment of small-sized cells in 4G systems to increase the system capacity results in non-negligible inter-cell interference [15, 17]. 4G can also support a large number of mobile devices simultaneously, which generate high data traffic in each cell, and this results in heavy inter-cell interference [15, 17]. Therefore, how to combat inter-cell interference in these systems is an important question. Moreover, although it is expected that in 5G cellular networks, mmWave spectrum will be used, on which communication will take place using highly directional antennas, which reduces the amount of inter-cell interference, it is likely that lower-frequency bands will continue to be used in the future (e.g., to achieve wide coverage, support high mobility users, etc.), on which a large amount of inter-cell interference can potentially take place [2]. Static and dynamic schemes are the two broad categories of interference avoidance techniques. Inter-cell interference coordination (ICIC) is a prime class of dynamic interference avoidance schemes which can be further categorized into the schemes using variable transmit power and fixed transmit power allocations on subchannels [15]. Although the ICIC with variable transmit power model allows a more flexible allocation than the ICIC with fixed transmit power model, the latter is simpler and easier to implement, and its performance loss can be negligible relative to the former especially for dense deployments of BSs [15]. Hence, we focus on the ICIC with fixed transmit power problem in this paper. In this problem, each base station (BS), if a given subchannel is assigned to a mobile station (MS) within its cell, transmits with fixed power on the assigned subchannel and does not transmit on subchannels that are not assigned to any MS. Therefore, the problem translates into a problem of deciding as to which MS, if any, to allocate each available subchannel to in each cell. Note that typically in each cell, some of the subchannels are not assigned to any MS in order to limit the inter-cell interference. Most of the proposed resource allocation schemes to address the ICIC problem consider maximizing the total throughput, i.e., the sum of throughputs of all the MSs in the system, while completely neglecting the aspect of fairness [5, 8, 14, 17, 20, 25]. In the context of cellular systems, fairness means that each MS, irrespective of its channel gain (which is a measure of the quality of the channel from the BS to the MS), has an equal chance of being allocated each of the available subchannels; i.e., no MS is preferred over the other MSs while allocating a subchannel in the system. Maximization of the total throughput results in high throughput of the MSs with good channel gain values; however, this is at the cost of low throughput of the MSs with poor channel gain values such as MSs at the cell boundaries [6]. However, one of the objectives of 4G systems is to offer good data rates to the MSs at the cell boundaries [20]. On the other hand, if lower (respectively, higher) throughputs were assigned to MSs with good (respectively, poor) channel gains, then it would lead to better fairness, but at the expense of the total throughput. So, there exists a tradeoff between the total throughput and fairness of resource allocation schemes [4]. Motivated by this fact, our objective in this paper is to formulate the problem of achieving different trade-offs between the total throughput and fairness, study its complexity, and design a distributed resource allocation algorithm to solve it.

Achieving Arbitrary Throughput–Fairness Trade-offs in the Inter-cell Interference …

19

We use Jain’s fairness index, which was proposed in [9] and has been extensively used in the networking literature, e.g., in [4, 6, 22], as a fairness metric. One way to optimize the total throughput–fairness trade-off in cellular systems is to allocate resources such that the total throughput is maximized subject to the constraint that the throughput of each MS must exceed some predefined lower bound [6]. The tradeoff can be optimized by varying the values of these lower bounds over the set of achievable rates. Another way is the α−fair scheme, which was originally proposed in the context of designing fair end-to-end window-based congestion control protocols for packet-switched networks [19]. Subsequently, this scheme has been used in several other contexts, e.g., resource allocation in wireless networks [1]. In the α−fair scheme, a parametric objective function, which is a function of the throughputs of the users and a parameter α, is maximized. The parameter α is varied to achieve the required trade-off between the total throughput and fairness. For instance, the maximum total throughput (and minimum degree of fairness) is obtained when α = 0. Similarly, proportional fairness [10] and max–min fairness [3] correspond to α = 1 and α = ∞, respectively. In general, the degree of fairness (respectively, total throughput) increases (respectively, decreases) as α increases [16]. In this paper, we adapt the concept of α−fairness to the problem of ICIC with fixed transmit power and show via simulations in Sect. 5 that when the adapted α-fair scheme is used to find a subchannel allocation, the Jain’s fairness index increases and the total throughput decreases with α. Thus, the adapted α−fair scheme provides any degree of fairness by choosing an appropriate value of α in [0, ∞), which is not possible in the scheme using predefined lower bounds [6] (see the previous paragraph). In addition, there is no clear procedure to select the lower bounds on the throughput of each MS in the latter scheme. In contrast, no lower bounds on the throughputs of MSs need to be selected in the adapted α−fair scheme, which makes its implementation simpler than that of the scheme that uses predefined lower bounds. However, the concept of α−fairness in [19] has to be modified by introducing a new parameter τ > 0 in the original parametric objective function. If the original parametric objective function were directly used in our context without change, the following problem would arise. If no subchannel is allocated to a MS (e.g., when the number of available subchannels is small relative to the number of MSs), its throughput is 0; this makes the value of the originally defined parametric objective function of the system −∞ for α > 1. Therefore, we introduce the concept of τ − α−fairness, which is a modification of the aforementioned α−fairness, and we define a new parametric objective function in Sect. 2, which is a function of both α and τ . We prove that the problem of finding a τ − α−fair allocation in the ICIC with fixed transmit power problem is NP-complete for all values of α in the range [0, ∞) and for all τ > 0 (see Sect. 3). Also, we propose a simple distributed subchannel allocation algorithm for the ICIC with fixed transmit power problem (see Sect. 4) and investigate as to how the algorithm parameters should be selected so as to achieve a desired trade-off between the total throughput and fairness, via simulations. The proposed algorithm is flexible, requires a small amount of time to operate, and requires information exchange among only neighboring BSs.

20

V. K. Gupta and G. S. Kasbekar

Resource allocation algorithms for the ICIC problem were proposed in [5, 8, 14, 17, 20, 25], but the aspect of fairness was not considered. We now briefly discuss the existing literature on resource allocation with fairness for cellular systems. The authors in [4] proposed two multiuser resource allocation schemes to achieve an optimal system efficiency–fairness trade-off. For these schemes to apply, the user’s benefit set must satisfy the monotonic trade-off property in which the Jain’s fairness index decreases with the increase in the system efficiency beyond a threshold value. In contrast, our proposed scheme does not require such a monotonic trade-off condition to be satisfied. A two-stage resource allocation algorithm was proposed in [11]. They have only considered the cell edge MSs and the interference caused only by the dominant BS. In contrast, we have considered all the MSs and the interference caused by all the other BSs which are transmitting over the same subchannel. A waterfilling cumulative distribution function-based scheduling scheme for uplink transmissions in cellular networks was proposed in [12]. A single-cell system model was considered. In contrast, we have considered a multicell system model. A joint user association and ICIC problem was formulated as a utility maximization problem, and an iterative algorithm was proposed to solve the formulated problem in [18]. A logarithmic utility function was used to obtain a proportional fair solution, which is similar to the case α = 1 in our work. However, no results were provided to study the trade-off between the total throughput and fairness achieved by the proposed resource allocation scheme. In contrast, in this paper, we provide a resource allocation scheme that can be used to achieve arbitrary trade-offs between the total throughput and level of fairness. The authors in [23] formulated the resource allocation problem as a mixed integer problem and proposed two suboptimal algorithms, for chunk allocation and power allocation, respectively. In [23], only a single-cell scenario was considered. In contrast, we have considered a multicell network. Also, in [23], a chunk allocation algorithm was proposed to provide max–min fairness only, which corresponds to the case α = ∞ of the scheme proposed in our work. Note that we consider all the values of α in [0, ∞), which correspond to different trade-offs between the total throughput and fairness. To the best of our knowledge, our work is the first to formulate the ICIC with fixed transmit power problem with the goal of achieving arbitrary trade-offs between the total throughput and fairness; in addition, we characterize the complexity of this problem, propose a distributed algorithm to solve it, and evaluate its performance via simulations.

2 System Model, Problem Definition, and Background We consider an OFDMA-based cellular system in which there are multiple cells; in each cell, a base station (BS) serves the mobile stations (MSs) in the cell. The available frequency band (channel) is divided into multiple subchannels; each subchannel has equal bandwidth. Let the set of all BSs and the set of all available subchannels be denoted by B = {1, . . . , K } and N = {1, . . . , N }, respectively. The cardinality of a set A is denoted by |A|. Suppose frequency reuse factor one is used, which implies

Achieving Arbitrary Throughput–Fairness Trade-offs in the Inter-cell Interference … 1

5

8

2

4

a

7

3

c

21

b

6

Base Station (BS) Mobile Station associated with BS a Mobile Station associated with BS b

10

9

d

Mobile Station associated with BS c Mobile Station associated with BS d

Fig. 1 In the example in the figure, there are two subchannels; let {1, 2} be the two subchannels in N . Subchannel 1 is allocated to the 1st, 4th and 8th MSs, and subchannel 2 is allocated to the 2nd, 6th and 9th MSs as shown by the different arrows in the figure

that any subset of the BSs in B may use the same subchannel in N simultaneously. Let Ma represent the set of all the MSs associated with BS a ∈ B and let |Ma | = Ma . Similarly, the set of all MSs in the system is represented by M= ∪a∈B Ma . Therefore, the total number of MSs in the system is given by M = a∈B Ma . Whenever two or more BSs simultaneously allocate a given subchannel to one of their associated MSs, it results in inter-cell interference. Note that typically in each cell, some of the subchannels are not assigned to any MS in order to limit the inter-cell interference. The example in Fig. 1 illustrates the model. We consider the problem of subchannel allocation to MSs for downlink transmissions (i.e., transmissions from BSs to MSs) in a given time slot. Let  n z a, j =

1, if MS j ∈ Ma is assigned subchannel n, 0, otherwise.

(1)

n The complete allocation is denoted by Z = {z a, j : a ∈ B, j ∈ Ma , n ∈ N }. Let

yan =



n z a, j.

(2)

j∈Ma

Intra-cell interference can be avoided by introducing the constraint that any subchannel n cannot be allocated to more than one MS within a cell; thus, we obtain the constraint: (3) yan ∈ {0, 1}, ∀a ∈ B, n ∈ N . Also, yan equals 1 if subchannel n is assigned to one of the MSs in Ma , else 0. Any n given BS a ∈ B transmits on a subchannel n ∈ N with fixed power P if z a, j = 1 for some j ∈ Ma ; else transmits with power 0. Assume that the noise power spectral density is N0 . Let each subchannel n ∈ N be an approximately flat fading channel; that is, the coherence bandwidth is larger than the subchannel bandwidth [21]. Let

22

V. K. Gupta and G. S. Kasbekar

Ha,n j denote the channel gain (which is a measure of the channel quality) from BS a to MS j on subchannel n; we assume that the channel gain values {Ha,n j : a ∈ B, j ∈ M, n ∈ N } remain unchanged during the considered time slot. Orthogonal cell-specific reference signals can be used to estimate the channel gain values {Ha,n j : a ∈ B, j ∈ M, n ∈ N } [15]. Hence, we assume that the channel gain values {Ha,n j : j ∈ M, n ∈ N } are known to BS a. n Consider an allocation Z = {z a, j : a ∈ B, j ∈ Ma , n ∈ N }. If Z satisfies (2) and (3), it is called a feasible allocation. Given a feasible allocation Z, the total throughput of all the MSs in the network is given by: ⎛ U (Z) =



⎜ ⎟ P Ha,n j ⎜ ⎟ n 1 + z a, log  ⎜ ⎟. j n n ⎝ P Hi, j yi + N0 ⎠ j∈Ma n∈N

   a∈B

(4)

i∈B,i=a

In (4), the throughput of the channel from BS a to MS j is calculated using the Shannon capacity formula for each a ∈ B and j ∈ Ma [24]; in particular, the second term inside the log(·) is the signal-to-interference-and-noise ratio on subchannel n from BS a to MS j. As a normalization, we assume that each subchannel has unit bandwidth. For future use, suppose the throughput of MS j ∈ Ma is denoted as follows: ⎛ U j (Z) =



⎜ ⎟ P Ha,n j ⎜ ⎟ n 1 + z a, log  ⎜ ⎟. j n n ⎝ P Hi, j yi + N0 ⎠ n∈N 

(5)

i∈B,i=a

Note that: U (Z) =

  a∈B j∈Ma

U j (Z) =



U j (Z).

(6)

j∈M

The notion of α−fair allocation was introduced in the context of multiple flows over a network having multiple nodes and links [19] as illustrated by the example in Fig. 2. The capacity of each link is finite and fixed. Each flow traverses a path that consists of multiple links and transmits at some flow rate. The concept of α−fair allocation was introduced to address the problem of how the bandwidths of the links in the network can be shared in a fair manner among the different flows [19]. Suppose S, L, and F are the sets of all the nodes, links, and flows, respectively, in a network (see Fig. 2). Let xr ≥ 0 be the flow rate of flow r ∈ F and let X = {xr : r ∈ F} represent the flow rate vector. For α > 0, the utility of a flow r is defined as Urα (xr ) = log(xr ) xr1−α if α = 1. The flow rate vector X which maximizes if α = 1 and Urα (xr ) = 1−α  α U (x ), i.e., the total utility of all the flows, such that the sum of the flow rates r r ∈F r through any link does not exceed its capacity, is known as the α−fair allocation.

Achieving Arbitrary Throughput–Fairness Trade-offs in the Inter-cell Interference …

23

s3

Fig. 2 A network with multiple flows over it. There are four nodes, S0 , . . . , S3 , five links, L 1 , . . . , L 5 , and three flows, f 0 , f 1 , f 2 , in the network

f2

f1 L5

L2

L1

s0

f0

L3

L4

s2

s1

In general, the degree of fairness (respectively, total throughput) under the α-fair allocation increases (respectively, decreases) as α increases [16]. In the model in this paper, the users of the network are MSs, in contrast to the above model where the users are the various flows. If the above definition of α−fairness were directly used in our context without change, i.e., if we defined the α-fair allocation to be the feasible allocation Z that maximizes j∈M log(U j (Z))  U j (Z)1−α if α = 1 and if α = 1, the following problem would arise. If no j∈M 1−α subchannel is allocated to a MS j, its throughput, U j (Z), is 0 (see (5)); this makes  U (Z)1−α Ul (Z)1−α = −∞ for α > 1 since j1−α = −∞ for U j (Z) = 0 and α > 1. l∈M 1−α Note that this is a potentially commonly arising situation in practice; e.g., some of the MSs would not be assigned any subchannels when the number of subchannels is small relative to the number of MSs in the network. To avoid this situation, we define a modified α−fair utility function by incorporating a positive number τ . Specifically, for a given α ∈ [0, ∞), τ > 0 and a feasible allocation Z, we define the τ − α−fair utility function of the system as follows:  Uα,τ (Z) =



j∈M j∈M

log(τ + U j (Z)), if α = 1, (τ +U j (Z))1−α , 1−α

if α = 1.

(7)

Suppose the set of all possible feasible allocations is denoted by Z. We define a τ − α−fair allocation to be a feasible allocation Z ∈ Z that maximizes the function Uα,τ (Z) in (7). Our goal is to find a τ − α−fair allocation: Problem 1 Find a τ − α−fair allocation. Our simulations (see Sect. 5) show that by solving Problem 1 with a fixed value τ > 0 and different values of α ∈ [0, ∞), allocations that achieve various tradeoffs between the total throughput and degree of fairness can be obtained. Also, the question of how the value of τ in Problem 1 should be selected is addressed in Sect. 5.

24

V. K. Gupta and G. S. Kasbekar

3 Complexity Analysis In this section, we show that for each α ∈ [0, ∞) and τ > 0, Problem 1 is NPcomplete. The decision version associated with Problem 1 is: For a given number T , can we find a feasible allocation Z which satisfies the condition Uα,τ (Z) ≥ T ? The following result shows that (the decision version of) Problem 1 is NP-complete. Theorem 1 For each α ∈ [0, ∞) and τ > 0, Problem 1 is NP-complete. Proof For any allocation Z, it is possible to verify in polynomial time whether Z is feasible or not using (2) and (3). Also, we can calculate Uα,τ (Z) using (7) and verify whether Uα,τ (Z) ≥ T in polynomial time. Hence, Problem 1 lies in class NP [13]. Next, we show NP-completeness of Problem 1 by reducing the maximum independent set (MIS) problem, which is known to be NP-complete [13], to Problem 1 in polynomial time; i.e., we show that MIS < p Problem 1. Consider the following instance of the MIS problem: We are given an undirected graph G = (V, E), in which V and E are vertex set and edge set, respectively, and a positive integer k. Does there exist an independent set of size at least k in G? From the above instance, a particular instance of Problem 1 is generated as follows: Suppose that only one subchannel1 is available (i.e., N = 1). Let B = V , i.e., corresponding to each node a ∈ V , there is a BS a ∈ B. Also, there is 1 MS associated with each BS (i.e., Ma = 1 for all a ∈ B). Let ja denote the MS associated with BS a and the edge connecting the two distinct nodes u and v is denoted by (u, v). Consider the above generated instance of Problem 1, and let No = P. Suppose the channel gains are modeled as follows: Hu, ju = 2, ∀u ∈ V  Hu, jv =

∞, if (u, v) ∈ E, u = v, 0, else.

(8) (9)

Consider an allocation Z = {z u, ju ∈ {0, 1} : u ∈ V } in the generated instance of Problem 1. Since Mu = 1 for all u ∈ V , it implies that yu ∈ {0, 1} for all u ∈ V , so constraints (2) and (3) are satisfied. Thus, every allocation Z = {z u, ju ∈ {0, 1} : u ∈ V } is feasible in the generated instance. Now, we divide the proof into three cases depending on the value of α. Case A: α > 1: The utility of the system under an allocation Z is calculated by (7) for α > 1. In the generated instance of Problem 1, we want to verify whether there exists a (fea|−k 1−α k τ ? sible) allocation Z which satisfies Uα,τ (Z) ≥ (1−α) (τ + log (3))1−α + |V1−α Our claim is that the answer is yes if and only if an independent set of size at least k exists in G. To show sufficiency, suppose an independent set, I , of size k  ≥ k exists in G. Then by (7), (8), and (9), the following allocation: 1 For

simplicity, we discard the superscript n (subchannel number) in the remaining proof.

Achieving Arbitrary Throughput–Fairness Trade-offs in the Inter-cell Interference …

 z u, ju =

1, if u ∈ I, 0, else,

25

(10)





|−k 1−α |−k k k τ ≥ (1−α) has utility (1−α) (τ + log (3))1−α + |V1−α (τ + log (3))1−α + |V1−α τ 1−α since k  ≥ k, which shows sufficiency. To show necessity, suppose that an allocation Z = {z u, ju ∈ {0, 1} : u ∈ V } exists such that:

Uα,τ (Z) ≥

k |V | − k 1−α τ (τ + log (3))1−α + (1 − α) 1−α

(11)

and let I = {u ∈ V : z u, ju = 1}. If two nodes u, v ∈ I are connected by an edge, then 1−α 1−α τ 1−α u (Z)) v (Z)) = (τ +U1−α = (1−α) , which are the by (7), (8), and (9), it follows that (τ +U1−α same as when both u and v are not allocated a subchannel. By this fact and by (7), it follows that: (12) Uα,τ (Z ) = Uα,τ (Z), where allocation Z is given as follows:  z u, ju =



1, if u ∈ I  , 0, else,

(13)

and I  is the independent set derived from I by excluding all node pairs having an edge between them. Let |I  | = k  . Then: Uα,τ (Z ) =

k |V | − k  1−α τ (τ + log (3))1−α + (1 − α) 1−α

(14)

by (7), (8), and (9). By (11), (12), and (14), we get: k |V | − k  1−α k |V | − k 1−α τ τ ≥ (τ + log (3))1−α + (τ + log (3))1−α + (1 − α) 1−α (1 − α) 1−α

(15)



=⇒ (k  − k) (τ + log (3))1−α − τ 1−α ≤ 0

(as α > 1)

(16)

So k  ≥ k. Hence, necessity holds as an independent set of size at least k exists in G. The result follows. Case B: α = 1: The utility of the system under an allocation Z is calculated by (7) for α = 1. In the generated instance of Problem 1, we want to verify whether there exists a (feasible) allocation Z which satisfies Uα,τ (Z) ≥ k log (τ + log(3)) + (|V | − k) log(τ )? Our claim is that the answer is yes if and only if an independent set of size at least k exists in G. To show sufficiency, suppose an independent set, I , of size k  ≥ k exists in G. Then, by (7), (8), and (9), the following allocation:

26

V. K. Gupta and G. S. Kasbekar

 z u, ju =

1, if u ∈ I, 0, else,

(17)

has utility k  log (τ + log(3)) + (|V | − k  ) log(τ ) ≥ k log (τ + log(3)) + (|V | − k) log(τ ) since k  ≥ k, which shows sufficiency. To show necessity, suppose that an allocation Z = {z u, ju ∈ {0, 1} : u ∈ V } exists such that: Uα,τ (Z) ≥ k log (τ + log(3)) + (|V | − k) log(τ ),

(18)

and let I = {u ∈ V : z u, ju = 1}. If two nodes u, v ∈ I are connected by an edge, then by (7), (8) and (9) it follows that log(τ + Uv (Z)) = log(τ + Uu (Z)) = log(τ ), which are the same as when both u and v are not allocated a subchannel. By this fact and (7), it follows that: (19) Uα,τ (Z ) = Uα,τ (Z), where allocation Z is given as follows:  z u, ju =



1, if u ∈ I  , 0, else,

(20)

and I  is the independent set derived from I by excluding all node pairs having an edge between them. Let |I  | = k  . Then: Uα,τ (Z ) = k  log (τ + log(3)) + (|V | − k  ) log(τ )

(21)

by (7), (8), and (9). By (18), (19), and (21) we get: k  log (τ + log(3)) + (|V | − k  ) log(τ ) ≥ k log (τ + log(3)) + (|V | − k) log(τ ) (22)   log(3) ≥0 (23) =⇒ (k  − k) log 1 + τ So k  ≥ k. Hence, necessity holds as an independent set of size at least k exists in G. Case C: α < 1: The proof in this case is similar to that in Case A and is omitted for brevity.

4 τ − α−Fair Distributed Subchannel Allocation Algorithm To approximately solve the NP-complete Problem 1 defined in Sect. 2, we propose a simple, distributed subchannel allocation algorithm in this section. This algorithm is a generalization of an algorithm proposed in our prior work [8] to solve the ICIC

Achieving Arbitrary Throughput–Fairness Trade-offs in the Inter-cell Interference …

27

with fixed transmit power problem with the objective of maximizing the sum of throughputs of all the MSs in the network. Let Ba ⊆ B be the set of neighboring BSs of BS a. Every BS a is directly connected to each of its neighboring BSs via high-speed links; these links are used to exchange information during the algorithm execution. For example, in LTE systems, X2 interfaces [7] are used to connect neighboring BSs. The proposed algorithm proceeds as explained below: During the initialization phase, the channel gain values are estimated as discussed in Sect. 2. Each BS a ∈ B obtains channel gain information {Hb,n j : j ∈ Ma , b ∈ Ba , n ∈ N } from its neighboring BSs in Ba . In practice, each BS has a small number of neighboring BSs; therefore, the amount of information exchanged would be small. After the initialization phase, the algorithm executes in iterations and each BS n ˆan and yˆbn , b ∈ Ba after each a ∈ B updates the variables {ˆz a, j : j ∈ Ma , n ∈ N }, y n iteration. Note that the temporary values of z a, j and yan , specified in Sect. 2, are n ˆan , respectively, after each iteration. Each BS a contained in the variables zˆ a, j and y n n n initializes zˆ a, j = 0, yˆa = 0 and yˆb = 0 for all j ∈ Ma , n ∈ N , b ∈ Ba at the beginning of the first iteration, and in subsequent iterations, if MS j ∈ Ma is allocated n subchanneln, then BS a assigns zˆ a, j = 1 and correspondingly calculates the varin n able yˆa = j∈Ma zˆ a, j . The following operations are executed during each iteration r = 1, 2, 3, . . .: (1) At the beginning of an iteration r , each BS a ∈ B computes and conveys pa to all the BSs in Ba . For a BS a, pa is defined as: ⎞⎫ ⎪ ⎪ ⎜ ⎟⎬ P Ha,n j ⎜ ⎟ log 1 +  max maxn ⎠ . m ⎪ ⎝ j∈Ma :ˆz a, j =0 ∀m∈N n∈N : yˆa =0 ⎪ ⎪ P Hb,n j yˆbn + N0 ⎪ ⎭ ⎩ ⎧ ⎪ ⎪ ⎨



(24)

b∈Ba

(2) Let j and n be the maximizers in (24). If pa ≥ pb ∀b ∈ Ba , then the MS j ∈ Ma n is assigned the subchannel n, and BS a updates both the variables zˆ a, ˆan to 1. j and y Note that it is possible that multiple BSs allocate subchannels to their associated MS simultaneously in an iteration. (3) Each BS a ∈ B conveys the information of the subchannel, if any, allocated to one of its associated MSs in Step 2, say n, to all the BSs in Ba and updates the values of yˆbn ∀b ∈ Ba , n ∈ N . Each BS a executes the above steps until at least one of the following conditions is fulfilled: (i) All the MSs in Ma are allocated subchannels. (ii) All the subchannels in N have been allocated to the MSs in Ma . (iii) pa < p0 , where p0 is given by (26) and (27) in Sect. 5. As soon as the algorithm terminates at BS a, its allocation is obtained using n n z a, j = zˆ a, j , ∀ j ∈ Ma and n ∈ N .

28

V. K. Gupta and G. S. Kasbekar

During each iteration, the distributed algorithm adopts a greedy approach in (24) and step 2 to choose (MS, subchannel) pairs with high throughputs. From (24) and the rule to update the variables ( yˆin : i ∈ B, n ∈ N ), pa either decreases or remains unchanged for each BS a ∈ B during each iteration. Also from (26) and (27), the higher the value of α, the lower the value of p0 . By condition (iii) above for termination, when α is high, the distributed algorithm operates for a longer duration and hence subchannels are allocated to more MSs, which leads to high interference. Due to the increased interference, the total throughput is lower (see Sects. 5.2 and 5.3), but the allocation is fairer since resources (subchannels) are allocated to more MSs. In summary, condition (iii) above for termination ensures that higher the value of α, greater the degree of fairness, and lower the total throughput of the allocation found by the above algorithm. This is confirmed by the simulation results in Sect. 5.

5 Simulations In this section, we provide simulation results to investigate the trade-off between the total throughput and fairness achieved using the exhaustive search algorithm and the proposed τ − α−fair distributed subchannel allocation algorithm in Sect. 4. We consider the following scenario throughout our simulations. Suppose that K BSs and M MSs are placed uniformly at random in a square area of dimension 1×1 unit2 . However, any two BSs must be at least dmin distance apart from each other, where dmin is a parameter. Let dmin = 0.1 units, and suppose all the BSs which are within a radius of 0.4 units from BS a are considered as the neighboring BSs of a (i.e., in the set Ba ). Further, suppose the MS-BS association is distance-dependent; i.e., each MS associates with the BS that is nearest to it. To account for the effects of fast fading, shadow fading, and the path loss phek Si j X n nomenon, we consider that the channel gains are given by Hi,n j = d γ i j , where di j ij denotes the distance between BS i and MS j, γ denotes the path loss exponent which can take values in the range (2, 4), and k is a constant [21]. To model the effect of shadow fading, a log-normal random variable Si j is considered. For distinct pairs (i, j), Si j are independent and identically distributed (iid) random variables. Similarly, Rayleigh distributed iid random variables X inj are considered to model the effect of fast fading. Next, we consider Jain’s fairness index as a fairness metric which is defined as follows [9]:  2 ( M j=1 U j (Z)) FI = , (25) M M( j=1 U 2j (Z)) where U j (Z) is given by (5). The value of FI lies between 0 and 1. Also, it increases with the degree of fairness of the distribution of throughput; if all MSs get exactly equal throughput, it takes value 1 and it equals mn when exactly n out of m MSs have

72

τ = 8,

τ = 13,

τ = 20

Total Throughput

Total Throughput

Achieving Arbitrary Throughput–Fairness Trade-offs in the Inter-cell Interference …

70 68 66

0

2

4

6

α

8

10

12

Fairness Index (FI)

Fairness Index (FI)

70 68

τ = 8, 2

τ = 13, 4

τ = 20 6

α

8

10

12

0.58

0.37

0.36 0

72

66 0

0.38

29

τ = 8, 2

4

τ = 13, 6

α

8

τ = 20 10

(a) For K = 3, N = 3, M = 11

12

0.56 0.54 0

τ = 8, 2

4

τ = 13, 6

8

τ = 20 10

12

α (b) For K = 2, N = 4, M = 7

Fig. 3 Total throughput and fairness index (FI) values under the allocation obtained by exhaustive search over all possible subchannel allocations versus α for different K , N and M

equal throughput and the remaining (m − n) MSs have 0 throughput [9]. See [9] for further properties of the fairness index.

5.1 Trade-Off Between the Total Throughput and Fairness Index Under the Exhaustive Search Algorithm and Selection of τ First, for different values of α we found the allocation that maximizes the system utility function in (7) by exhaustive search over all possible combinations of subchannel allocation to all the MSs of the system. Then, the total throughput and fairness index FI were calculated for the obtained allocation using (4) and (25), respectively. Figures 3 and 4a plot2 the variation of the total throughput and fairness index FI with α for different values3 of the parameters K , N , and M and for three different values of τ . In Figs. 3 and 4a, the total throughput decreases and fairness index FI increases with α. Next, we address the question of how the value of τ should be selected. Figures 4b and 5 show the variation of the total throughput and fairness index FI with τ , for different values of the parameters K , N and M and for three different values of α. In Figs. 4b and 5, the total throughput first increases and then approximately saturates 2 For

all the plots in Figs. 3, 4, 5, 6, 7, 8, and 9, each data point was obtained by averaging across 50 runs with different random seeds. 3 Note that for all the plots in Figs. 3, 4, and 5, only small values of the parameters K , N , and M were used since it is computationally prohibitive to execute the exhaustive search algorithm with large values of K , N , and M.

30

V. K. Gupta and G. S. Kasbekar τ = 8,

τ = 13,

τ = 20

85

80 0

90

Total Throughput

Total Throughput

90

85 80 75

6

4

2

α

12

10

8

α = 0.5, 0

Fairness Index (FI)

Fairness Index (FI)

0.47

0.45 0.44 0

τ = 8, 6

4

2

τ = 13,

τ = 20 10

8

α (a) For K = 3, N = 4, M = 10

α = 1.5,

τ

α = 0.5,

α=5 20

15

10

α = 1.5,

α=5

0.46 0.45 0.44 0

12

5

5

10

20

15

τ (b) For K = 3, N = 4, M = 10

72 70 68 66

α = 1.5,

α = 0.5,

0 0.58

5

10

τ

15

α = 1.5,

α = 0.5,

α=5 20

α=5

0

5

10

15

τ (a) For K = 2, N = 4, M = 7

α = 0.5,

α = 1.5,

α=5

70

5

10

τ

15

20

0.36

0.56

0.54

75

65 0

Fairness Index (FI)

Fairness Index (FI)

Total Throughput

Total Throughput

Fig. 4 Figure a (respectively, b) total throughput and fairness index (FI) values under the allocation obtained by exhaustive search over all possible subchannel allocations versus α (respectively, τ )

20

0.35 0.34 0

α = 0.5, 5

α = 1.5, 10

τ

α=5 15

20

(b) For K = 3, N = 3, M = 11

Fig. 5 Total throughput and fairness index (FI) values under the allocation obtained by exhaustive search over all possible subchannel allocations versus τ for different K , N and M

as τ increases. Also, in most cases in Figs. 4b and 5, the fairness index FI slightly decreases as τ increases. Similar to the trends in Figs. 3 and 4a, for fixed τ , the total throughput decreases and fairness index FI increases with α. Figures 4b and 5 show that the values of total throughput and fairness index FI are not very sensitive to the value of τ . Nevertheless, the figure shows that the choice τ ∈ [8, 9] results in large total throughput and large FI. From Figs. 3, 4, and 5, it can be concluded that by solving Problem 1 with a fixed value τ > 0 and different values of α ∈ [0, ∞), allocations that achieve various trade-offs between the total throughput and degree of fairness can be obtained.

Achieving Arbitrary Throughput–Fairness Trade-offs in the Inter-cell Interference …

31

5.2 To Obtain the Value of p0 that Maximizes the Total Throughput For the distributed τ − α−fair subchannel allocation algorithm, we want to first find the value of the parameter p0 (see the condition (iii) for termination of the algorithm in Sect. 4), say p0∗ , that results in the maximum total throughput under the allocation found by the algorithm. The value p0∗ will later be used in Sect. 5.3 to investigate as to how p0 should be selected as a function of α such that the higher the value of α, the lower the total throughput, and higher the degree of fairness under the allocation found by the algorithm. The variation of the total throughput with the parameter p0 is depicted in Figs. 6a, b and 7a for different values of K , N , and M, respectively. In Figs. 6 and 7a, the total throughput is maximized for medium values of p0 . Intuitively, this is because for too low values of p0 , the proposed algorithm allocates subchannels to a large number of MSs (see condition (iii) for termination of the algorithm in Sect. 4), which results in high interference and low total throughput. Similarly, for too high values of p0 , the algorithm does not allocate subchannels to enough of MSs, which results in low total throughput. Therefore, the total throughput first increases and then decreases as p0 increases. Figures 7b, 8a, b present the variation of the FI with p0 for different values of M, K , and N , respectively. Figures 7b and 8 show that the FI decreases as p0 increases. Intuitively, this is because as p0 decreases, the algorithm runs for a longer duration and allocates subchannels to more number of MSs which increases fairness. After extensive simulations, we empirically found that the value of the parameter p0 (say p0∗ ) which gives close to maximum total throughput in terms of the parameters K , M, and N is given by the following expression: p0∗

=

M , 2(N K )

if M ≤ K × N ,

1+

log(N K ) , 2 log M

otherwise.

530 520 510 500 490 480 470

440 420 400

0.5

1

1.5

2

2.5

p0

(a) For N = 20, M = 300

3

N = 12 N = 16 N = 20

380 360 340 320

0

(26)

460

K = 13 K = 17 K = 20

540

Total Throughput

Total Throughput

550

1+

0

0.5

1

1.5

2

2.5

3

p0

(b) For K = 15, M = 240

Fig. 6 Figure a (respectively, b) total throughput under the allocation found using the distributed τ − α−fair algorithm versus p0 for different K (respectively, N )

32

V. K. Gupta and G. S. Kasbekar

5.3 Selection of the Value of p0 as a Function of α From Figs. 6, 7, and 8, it can be concluded that there is a trade-off between the total throughput and degree of fairness when the parameter p0 is in the range [0, p0∗ ]. In particular, within the range p0 ∈ [0, p0∗ ], the total throughput (respectively, fairness) is maximized at p0 = p0∗ (respectively, p0 = 0). However, recall that α = 0 (respectively, α = ∞) corresponds to maximum total throughput (respectively, fairness) and minimum fairness (respectively, total throughput). This motivates us to set p0 , in terms of α, as: 1 . (27) p0 = 1 +α p∗ 0

0.3

350

Fairness Index (FI)

Total Throughput

In summary, the choice of p0 in (27) ensures that as α increases from 0 to ∞, the total throughput (respectively, degree of fairness) of the allocation found using the algorithm described in Sect. 4 decreases (respectively, increases).

300

250

M = 130 M = 185

200

0.5

0.26 0.24 0.22 0.2 0.18 0.16

M = 200

0

M = 130 M = 185 M = 200

0.28

1

1.5

2

2.5

3

0

0.5

1

1.5

2

2.5

p

p

(a) For K = 12, N = 16

(b) For K = 12, N = 16

0

3

0

Fig. 7 Figure a (respectively, b) total throughput (respectively, fairness index (FI)) under the allocation found using the distributed τ − α−fair algorithm versus p0 for different M 0.26

K = 13

Fairness Index (FI)

Fairness Index (FI)

0.24

K = 17

0.22

K = 20

0.2 0.18 0.16

0

0.5

1

1.5

2

2.5

p

0

(a) For N = 20, M = 300

3

N = 12 N = 16 N = 20

0.24 0.22 0.2 0.18 0.16 0.14 0.12

0

0.5

1

1.5

2

2.5

3

p

0

(b) For K = 15, M = 240

Fig. 8 Figure a (respectively, b) fairness index (FI) under the allocation found using the distributed τ − α−fair algorithm versus p0 for different K (respectively, N )

550 K = 9,

K = 15,

K = 20

500 450

20

10

0

Total Throughput

Total Throughput

Achieving Arbitrary Throughput–Fairness Trade-offs in the Inter-cell Interference …

33

600 500 400

N =17,

N =24,

N =30

20

10

0

α

α Fairness Index (FI)

K = 9,

K = 15,

K = 20

Fairness Index (FI)

0.3

0.25

0.2

0.2 0.1 0

10

20

Total Throughput

α (a) For N = 20 and M = 300 600

N =24,

N =30

10

α

20

(b) For K = 12 and M = 300 M = 300,

M = 350

500

400 0

Fairness Index (FI)

M = 240,

N =17, 0

5

10

α

15

20

0.25 0.2 M = 240,

0.15 0

5

M = 300,

10

α

M = 350

15

20

(c) For K = 15 and N = 20 Fig. 9 Figure a (respectively, b and c) total throughput and fairness index (FI) under the allocation found using the distributed τ − α−fair algorithm versus α for different values of K (respectively, N and M)

5.4 Performance Evaluation of the Proposed Distributed Algorithm For different sets of values of K , M, and N and for different values of α, p0 was computed using (26) and (27). Using the calculated value of p0 , the proposed distributed τ − α−fair subchannel allocation algorithm was run and a subchannel allocation was obtained. The total throughput and FI under the obtained allocation were calculated using (4) and (25), respectively. Figure 9a, b, c depicts the variation of the total throughput and FI with α for different values of K , N , and M, respectively. In Fig. 9, the total throughput decreases and the fairness index FI increases as α increases. Therefore, it can be verified from Fig. 9 that the distributed τ − α−fair subchannel allocation algorithm proposed in Sect. 4 and the expressions for p0∗ and p0 in (26) and (27) provide the required trade-off between the total throughput and degree of fairness.

34

V. K. Gupta and G. S. Kasbekar

6 Conclusions In this paper, we introduced the concept of τ − α−fairness in the context of the ICIC with fixed transmit power problem by modifying the concept of α−fairness. The concept of τ − α−fairness allows us to achieve arbitrary trade-offs between the total throughput and degree of fairness by selecting an appropriate value of α in [0, ∞). We showed that for every α ∈ [0, ∞) and every τ > 0, the problem of finding a τ − α−fair allocation is NP-complete. Also, we proposed a simple, distributed subchannel allocation algorithm for the ICIC problem, which is flexible, requires a small amount of time to operate, and requires information exchange among only neighboring BSs. We investigated via simulations as to how the algorithm parameters should be selected so as to achieve any desired trade-off between the total throughput and fairness.

References 1. E. Altman, and K. Avrachenkov, and A. Garnaev, “Generalized α-fair resource allocation in wireless networks”, Proc. of the IEEE Conference on Decision and Control, pp. 2414–2419, Dec. 2008. 2. J. G. Andrews, S. Buzzi, W. Choi, S. V. Hanly, A. Lozano, A. C. K. Soong, J. C. Zhang, “What will 5G be?”, IEEE Journal on Selected Areas in Communications, Vol. 32, No. 6, pp. 1065–1082, Nov. 2014. 3. D. Bertsekas, and R. Gallager, “Data Networks”, Prentice-Hall, Inc. 1992. 4. A. Bin Sediq, R. H. Gohary, H. Yanikomeroglu, “Optimal tradeoff between efficiency and Jains fairness index in resource allocation”, Proc. 2012 IEEE PIMRC, pp. 577583, Nov. 2012. 5. A. Bin Sediq, R. Schoenen, H. Yanikomeroglu, G. Senarath, “Optimized Distributed Inter-Cell Interference Coordination (ICIC) Scheme Using Projected Subgradient and Network Flow Optimization”, IEEE Transactions on Communications, Vol. 63, No. 1, pp. 107–124, 2015. 6. H. T. Cheng and W. Zhuang, “An optimization framework for balancing throughput and fairness in wireless networks with QoS support”, IEEE Trans. of Wireless Commun. Vol. 7, No. 2, pp. 584–593, Feb. 2008. 7. A. Ghosh, J. Zhang, J. Andrews, R. Muhamed, “Fundamentals of LTE”, Pearson Education, 2011. 8. V. K. Gupta, A. Nambiar and G. S. Kasbekar, “Complexity Analysis, Potential Game Characterization and Algorithms for the Inter Cell Interference Coordination with Fixed Transmit Power Problem”, IEEE Transactions on Vehicular Technology, Vol. 67, No. 4, pp. 3054–3068, Nov. 2017. 9. R. Jain, D. Chiu, and W. Hawe “A quantitative measure of fairness and discrimination for resource allocation in shared systems”, Digital Equipment Corporation, Tech. Rep. DEC-TR301, Sep. 1984. 10. F. P. Kelly, A. K. Maulloo, and D. K. H. Tan, “Rate control for communication networks: shadow prices, proportional fairness and stability”, J. Oper. Res. Soc. Vol. 49, No. 3, pp. 237–252, 1998. 11. S. Kim, H. K. Jwa, J. Moon, Jee-Hyeon Na, “Achieving fair cell-edge performance: Lowcomplexity interference coordination in OFDMA networks”, Proc. 2018 IEEE ICACT, pp. 6–11, Feb. 2018. 12. S. Kim, H. K. Jwa, J. Moon, Jee-Hyeon Na, “Joint opportunistic user scheduling and power allocation: throughput optimisation and fair resource sharing”, IET Communications, Vol. 12, No. 5, pp. 634–640, March 2018.

Achieving Arbitrary Throughput–Fairness Trade-offs in the Inter-cell Interference …

35

13. J. Kleinberg, E. Tardos, “Algorithm Design”, Addison Wesley, 2005. 14. C. Kosta, B. Hunt, A. U. Quddus, R. Tafazolli, “A Low-Complexity Distributed Inter-Cell Interference Coordination (ICIC) Scheme for Emerging Multi-Cell HetNets”, Proc. of IEEE VTC, 2012. 15. C. Kosta, B. Hunt, A. U. Quddus, R. Tafazolli, “On Interference Avoidance Through Inter-Cell Interference Coordination (ICIC) Based on OFDMA Mobile Systems”, IEEE Communications Surveys & Tutorials, Vol. 73-99515, No. 3, pp. 9, 2013. 16. T. Lan, D. Kao, M. Chiang, and A. Sabharwal, “An axiomatic theory of fairness in network resource allocation”, Proc. IEEE Int. Conf. Comput. Commun. 2010. 17. D. Lopez-Perez, I. Guvenc, G. De la Roche, M. Kountouris, T. Q. S. Quek, J. Zhang, “Enhanced Intercell Interference Coordination Challenges in Heterogeneous Networks”, IEEE Wireless Communications, Vol. 18, No. 3, pp. 22–30, 2011. 18. N. Miki, Y. Kanehira, and H. Tokoshima, “Investigation on joint optimization for user association and inter-cell interference coordination based on proportional fair criteria”, Proc. of ICSPCS, Dec. 2017. 19. J. Mo and J. Walrand, “Fair end-to-end window-based congestion control”, IEEE/ACM Trans. Networking, Vol. 8, No. 5, pp. 556–567, Oct. 2000. 20. M. Rahman, H. Yanikomeroglu, “Enhancing Cell-edge Performance: A Downlink Dynamic Interference Avoidance Scheme with Inter-cell Coordination”, IEEE Transactions on Wireless Communications, Vol. 9, No. 4, pp. 1414–1425, 2010. 21. T. S. Rappaport, “Wireless Communications: Principles and Practice”, Pearson Education, Second Edition, 2009. 22. S. Sheikh, R. Wolhuter, and H. A. Engelbrecht, “An Adaptive Congestion Control and Fairness Scheduling Strategy for Wireless Mesh Networks”, Proc. IEEE SSCI, pp. 1174–1181, Dec. 2015. 23. Y. Shen, X. Huang, BoYang, S. Gong, S. Wang, “Fair Resource Allocation Algorithm for Chunk Based OFDMA Multi-User Networks”, Proc. 2017 IEEE VTC-Fall, Sept. 2017. 24. D. Tse, and V. Pramod, “Fundamentals of wireless communication”, Cambridge University Press, 2005. 25. M. Yassin, “Inter-Cell Interference Coordination in Wireless Networks”, Thesis May 2016 [Online]. Available: https://tel.archives-ouvertes.fr/tel-01316543.

Coexistence of LTE-Unlicensed and WiFi with Optimal Channel Aggregation Naveen Kolar Purushothama

Abstract In this work, we investigate the problem of channel and rate allocation for LTE-Unlicensed (LTE-U) to efficiently coexist with WiFi access points (APs). Specifically, we formulate an auction mechanism where LTE-U first announces an aggregation number (number of WiFi channels that LTE-U wishes to aggregate) and a reserve rate (maximum rate that LTE-U is willing to allocate to an AP), following which the APs decide their mode of operation—cooperation or competition. In cooperation mode, an AP allows LTE-U to exclusively occupy its channel (in return for a rate) while in the competition mode both LTE-U and AP simultaneously contend for channel access. We characterize the solution to the auction problem in terms of Symmetric Bayesian Nash Equilibrium (SBNE) and prove results illustrating its structure. We then optimize the auction mechanism by evaluating the optimal aggregation number and reserve rate that maximizes the total rate achieved by LTEU subject to a constraint on APs’ rates. Finally, through simulation experiments we demonstrate the efficacy of our algorithm in comparison with (1) the strategy of aggregating a fixed number of channels and (2) a heuristic algorithm where a random number of channels are aggregated.

1 Introduction The ever-increasing demand for mobile data [1] is motivating the cellular operators to consider extending LTE services onto the unlicensed spectrum, particularly the WiFi band. This development is popularly referred to as the LTE-Unlicensed [2, 3]. Since LTE uses a schedule-based MAC (e.g., TDMA) while WiFi is contentionbased (CSMA/CA), coexistence of LTE with WiFi is not natural. Hence, practical

This work was supported by an INSPIRE Faculty Award (No. DST/INSPIRE/04/2015/000928) of the Department of Science and Technology, Government of India. N. Kolar Purushothama (B) Indian Institute of Technology Tirupati, Tirupati, India e-mail: [email protected] © Springer Nature Switzerland AG 2019 J. Walrand et al. (eds.), Network Games, Control, and Optimization, Static & Dynamic Game Theory: Foundations & Applications, https://doi.org/10.1007/978-3-030-10880-9_3

37

38

N. Kolar Purushothama

deployment of LTE-Unlicensed requires new protocols for enabling efficient coexistence of these two technologies. Toward this direction, one line of work considers modifying the existing MAC of LTE so as to enable LTE to contend with WiFi for channel access [4, 5]. We refer to this approach as coexistence-by-competition whereby the throughput of both LTE and WiFi reduces due to contention. Other work considers the approach of coexistence-by-cooperation where LTE and WiFi cooperate by sharing resources (channel time) and/or by traffic offloading [6]; however, cooperation may not always yield a rational solution in the sense that either LTE-U or WiFi could benefit by defecting from cooperation. In this context, the motivation for our work comes from the recent work of Yu et al. [7] where the authors introduce an unified approach of coopetition (cooperation + competition) using auction theory. Specifically, a reverse auction mechanism has been set up in [7] to enable WiFi APs to choose their mode of operation (i.e., cooperation or competition). The LTE base station on the other hand optimizes the auction mechanism to efficiently select a WiFi band for its transmissions. The work in [7] is, however, limited to the case where LTE-Unlicensed is restricted to transmit on a single channel. In the current work, we generalize the setting in [7] by allowing LTE-Unlicensed to aggregate multiple channels. In the process, we also propose a novel optimization framework for maximizing the rate achieved by LTE-Unlicensed subject to a constraint on the QoS guarantee for WiFi. The outline of the paper is as follows. In Sect. 2, we briefly discuss the system model, while in Sect. 3 we set up the auction mechanism in detail. In Sect. 4 we derive the solution to the auction problem. Computation of the expected rates is presented in Sect. 5. In Sect. 6, we report the results from our numerical and simulation work. For the ease of exposition we have moved the proofs of our main results to Sect. 7. We finally draw our conclusions in Sect. 8.

2 System Model We consider a wireless system comprising an LTE-unlicensed base station (referred to as LTE-U hereafter) that intends to share the unlicensed spectrum occupied by the WiFi access points (APs) in its vicinity. Specifically, let K := {1, 2, . . . , K} denote the set of APs that are co-located with LTE-U. For simplicity, we assume that each AP is operating in a different WiFi band (channel); the channel occupied by AP-k is referred to as channel k. Thus, the respective transmissions from the APs do not interfere with each other. Let Rk (k ∈ K ) be the rate that AP-k can achieve if it exclusively occupies its channel. We refer to Rk as the contention-free transmission rate of AP-k. The knowledge of rate Rk is private in the sense that Rk is known only to AP-k. However, the p.d.f f (·) of Rk , and equivalently its c.d.f F(·), are publicly known to all players in the system (i.e., other APs and LTE-U). We assume that the support of f (·) is

Coexistence of LTE-Unlicensed and WiFi with Optimal Channel Aggregation

39

[0, rmax ], i.e., f (r) > 0 if and only if r ∈ [0, rmax ]. Finally, let RLTE denote the (channel independent) contention-free transmission rate of LTE-U. The task of the APs is to decide how they choose to coexist with LTE-U. The available coexistence mechanisms are cooperation and competition. In cooperation mode, LTE-U agrees to allocate a negotiated rate for the AP’s traffic. The AP then creates a contention-free channel for LTE-U by switching off its transmissions. On the other hand, in competition mode both LTE-U and the AP transmit simultaneously, in which case, due to contention, the rates of both LTE-U and AP get discounted by factors δL and δA (both taking values in (0, 1)), respectively. Our objective is to design an auction mechanism that will enable APs’ to negotiate their mode of coexistence with LTE-U in a rational fashion. Specifically, we recognize LTE-U as the buyer and the APs as the sellers. The commodity for sale is the APs’ right to on-load its traffic onto LTE-U. The auction mechanism consists of two stages (details are discussed in Sect. 3): 1. In the first stage (Stage-I), LTE-U announces an aggregation number N which is the number of WiFi channels that LTE-U wishes to aggregate (i.e., buy). LTE-U also announces a reserve rate C which is the maximum rate that LTE-U is willing to pay per channel. 2. In the second stage (Stage-II), APs place their bids (bk : k ∈ K ) where bk indicates the rate that AP-k demands from LTE-U. After collecting all the bids, LTE-U employs uniform-price auction mechanism to determine the winning APs.1 The auction mechanism also determines the amount of rate that LTE-U should allocate to the winning APs. The overall solution to the coopetition problem thus involves first determining the value of (N , C) that LTE-U should announce in Stage-I. For this purpose, we propose an optimization framework that maximizes the system performance by taking into account the expected rates achieved by both LTE-U and the APs. We then determine the APs’ equilibrium bidding strategies (bk : k ∈ K ) that constitute a solution to the auction problem of Stage-II. The solution is derived via backward induction where we first solve Stage-II for a given (N , C); we then solve the optimization framework to determine the optimal value of (N , C) that is to be announced in Stage-I. The details are presented in the subsequent sections. Remarks: The rates Rk (respectively RLTE ) should be thought as the aggregate rate at which AP-k (respectively LTE-U) can serve all users associated with it, provided there is no contention from other entities (WiFi or LTE-U). Since the users’ associations are bound to change over time (due to mobility, channel conditions, etc.), the rate values are subject to change as well. However, we assume a quasi-static system where the user associations, and hence the rates, do not vary (significantly) over a 1 Uniform-price

auction is a generalization of the second-price auction (or the Vickrey auction) to the scenario where N ≥ 1 identical commodities are auctioned for sale. The bidders (APs in our case) with the first-N least bid values are chosen as winners and are made a payment (rate allocated in our case) equal to the value of the (N + 1)th least bid. For uniform-price auctions, it is known that truthful bidding is a weakly dominant strategy.

40

N. Kolar Purushothama

time period of interest. The coexistence mechanism that we design is applicable for a generic time period (the mechanism, however, has to be reimplemented whenever the associations and the corresponding rates change). Finally, for simplicity we assume that the rate RLTE achieved by LTE-U is independent of the channel. This assumption can be partly justified by noting that the cellular technology (owing to TDMA, coded modulation, etc.) can guarantee a stable rate irrespective of the channel conditions. We, however, acknowledge that relaxing this assumption is an interesting scope for future work.

3 Auction Mechanism Suppose LTE-U has announced an (N , C) in Stage-I where, recall that, N is the number of WiFi channels that LTE-U wishes to aggregate, and C is the reserve rate per channel. Let b = (b1 , b2 , . . . , bk ) denote the bid vector, where the bid bk of AP-k is allowed to take a value from the set [0, C] ∪ {X } with the understanding that • bk ∈ [0, C] indicates the data rate that AP-k demands from LTE-U, and • bk = X is used to represent the scenario where AP-k is unwilling to on-load its traffic onto LTE-U. In the following for the ease of exposition, we extend the definition of min operator by including X into operations:  min{bk , X } =

bk if bk ∈ [0, C] X if bk = X .

Further, we let min(φ) = X where φ is empty. Given a bid vector b, we denote the set of APs whose bids are the minimum as I1 = arg min{bk : k ∈ K }. Similarly, the set of APs whose bids are the secondminimum is given by I2 = arg min{bk : k ∈ K \I1 }. In general, denoting I0 = φ, we define (for k ∈ K )   j−1 Ik = arg min bj : j ∈ K \ ∪ Ii . i=0

(1)

Let Ik = |Ik |. The value of kth minimum bid is denoted b(k) : b(k)

  j−1 = min bj : j ∈ K \ ∪ Ii . i=0

(2)

For convenience we define, for k = 0, 1, . . . , K, Mk = I0 ∪ I 1 ∪ · · · ∪ Ik and Mk = |Mk |.

(3)

Coexistence of LTE-Unlicensed and WiFi with Optimal Channel Aggregation

41

3.1 LTE-U and APs’ Pay-off Functions We are interested in computing the total number of APs whose bid values are in [0, C]. These are the APs that intend to cooperate with LTE-U. For this, we first define (with the understanding that max(φ) = 0)   L = max k ∈ K : b(k) ∈ [0, C] .

(4)

Then, the total number of APs that intend to cooperate is simply given by ML . Depending on the value of ML , the following two cases are possible: (C1)

(C2)

(C2a)

(C2b)

ML ≤ N : In this scenario, LTE-U operates in cooperation mode with all the ML APs in ML by allocating a rate of C to each AP. LTE-U accrues a total rate (pay-off ) of ML (RLTE − C) by transmitting on these channels. The remaining (N − ML ) APs (i.e., channels) are randomly selected from IL+1 .2 LTE-U competes with these APs, and hence accrues a discounted rate of (N − ML )δL RLTE on these channels, while the corresponding APs achieve their discounted rates of δA Rki (i = 1, 2, . . . , N − ML ), where we have used {k1 , k2 , . . . , kN −ML } ⊂ IL+1 to denote the randomly chosen set of APs. ML > N : The LTE-U is now required to choose N out of the ML channels for its transmissions. The selection of the channels is based on the uniformprice auction mechanism according to which the channels corresponding to the N “best” bidders (i.e., APs demanding the N least rates) are chosen as winners. The rate allocated to the winning APs is the rate that is bid by the (N + 1)th best  AP. In order to formalize the above discussion, we first define  = max k ∈ K : Mk ≤ N . Then, the following two sub-cases are possible: M < N , in which case all APs in M are chosen, while the remaining (N − M ) APs are randomly selected from I+1 . The rate allocated to the winning APs is b(+1) since the (N + 1)th best bidder is in I+1 . The aggregate rate achieved by LTE-U on these channels is N (RLTE − b(+1) ). M = N , in which case M constitutes the set of winning APs with the rate allocated being b(+1) . The pay-off to LTE-U is again N (RLTE − b(+1) ).

We summarize the above discussion in expressions (5) and (6) below, where the pay-off functions of LTE-U and AP-k (k ∈ K ) are denoted Π LTE (·) and ΠkAP (·), respectively.

∈ / ML , we must have bk = X . Thus, X being the next-least bid value, all the remaining APs are in IL+1 .

2 Expression (4) implies that for any AP-k

42

N. Kolar Purushothama

⎧   ⎨ ML (RLTE − C) + (N − ML )δL RLTE if ML ≤ N Π LTE b; (N , C) = ⎩ if ML > N . N (RLTE − b(+1) ) ⎧ C ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ N −ML K−N ⎪ ⎪ δ R + K−M Rk ⎪ K−ML A k L ⎪ ⎪ ⎪   ⎨ ΠkAP b; (N , C) = b(+1) ⎪ ⎪ ⎪ ⎪ ⎪ M+1 −N N −M ⎪ ⎪ ⎪ I+1 b(+1) + I+1 Rk ⎪ ⎪ ⎪ ⎪ ⎩ Rk

(5)

if ML ≤ N , k ∈ ML if ML ≤ N , k ∈ / ML if ML > N , k ∈ M

(6)

if ML > N , k ∈ / M , k ∈ M+1 if ML > N , k ∈ / M+1

Note that these expressions are also applicable when N = K as well in which case all cooperating APs (i.e., those whose bids are in [0, C]) are allocated a rate of C while the rate of the competing APs (i.e., those bidding X ) is discounted by the factor δA due to contention from LTE-U.

3.2 Bidding Strategies A bidding strategy bN ,C (·) is a mapping that decides the value of the bid for an AP given its true rate value, i.e., if AP-k uses a bidding strategy bN ,C (·), then bN ,C (Rk ) ∈ [0, C] ∪ {X } is its bid. We are interested in symmetric scenarios where all APs use the same bidding strategy. Then, the solution to the auction problem is characterized by Symmetric Bayesian Nash Equilibrium (SBNE) which is defined as follows: Definition 1 A bidding strategy b∗N ,C (·) is said to constitute a SBNE if, for all k ∈ K and all sk ∈ [0, C] ∪ {X }, we have   

ER−k ΠkAP b(1) ; (N , C) Rk   

≥ ER−k ΠkAP b(2) ; (N , C) Rk   (i) (i) where b(i) = b(i) 1 , b2 , . . . , bK , i = 1, 2, are bidding vectors satisfying ∗ b(1) j = bN ,C (Rj ) for all j  ∗ bN ,C (Rj ) if j = k b(2) j = sk if j = k.

Coexistence of LTE-Unlicensed and WiFi with Optimal Channel Aggregation

43

The expectation in the above expression is with respect to the joint distribution of R−k = (R1 , . . . , Rk−1 , Rk+1 , . . . , RK ). Thus, deviating from the recommendation of b∗N ,C (·) is not beneficial for any AP when all other APs adhere to b∗N ,C (·).

3.3 Pay-off Optimization Given a SBNE b∗N ,C (·), the expected pay-off to LTE-U can be written as   LTE (N , C) = ER Π LTE b∗N ,C ; (N , C) Π

(7)

where b∗N ,C = (b∗N ,C (Rk ) : k ∈ K ) denotes the vector of equilibrium bids of all APs, and the expectation is over the joint distribution of R = (R1 , R2 , . . . , RK ). Similarly, the expected pay-off to AP-k can be written as   kAP (N , C) = ER ΠkAP b∗N ,C ; (N , C) . Π

(8)

Since the APs are assumed to be statistically identical, it follows that the payoff achieved by the different APs are equal. We denote this common pay-off as AP (N , C). Π Finally, we propose the pay-off maximization problem: LTE (N , C) Maximize: Π AP (N , C) ≥ R Subject to: Π Over: N ∈ {1, 2, . . . , K}

(9)

C ∈ [0, RLTE ] AP (N , C) is used to guarantee a minimum rate (i.e., QoS) where the constraint R on Π to the APs. The constraints on the optimization variables N and C are natural as LTEU is allowed to aggregate at most K channels while ensuring a reserve rate of not more than RLTE (which is the rate that is possible under contention-free conditions). We use (N ∗ , C ∗ ) to denote the optimal points of the above problem.

4 Equilibrium Bidding Strategy In this section, we derive the structure of a SBNE. We consider two cases that are possible depending on the value of the reserve rate C: (1) C ∈ [0, rmax ) and (2) C ∈ [rmax , ∞). For notational simplicity, for a tagged AP, say AP-k, we use Rk to also denote a realization of the random variable Rk . Thus, functions such as pn,m (Rk ),

44

N. Kolar Purushothama

H (Rk ), etc., defined in this section should be understood as deterministic functions of the variable Rk ∈ [0, rmax ] (unless expectations are involved).

4.1 Case-1: C ∈ [0, rmax ) In this case, we have a variant of the traditional uniform-price auction where APk (k ∈ K ) is restricted from bidding its true rate Rk whenever Rk > C. Hence, complete truthful bidding cannot constitute a solution. In Theorem 1, we propose a SBNE involving partial truthful bidding where AP-k bids truthfully whenever Rk ∈ [0, C), while optimally choosing between C and X if Rk ∈ [C, rmax ]. We first introduce further notation and an auxiliary result (Lemma 1) before proceeding to the details of Theorem 1. For any k ∈ K and Rk ∈ [C, rmax ], let pn,m (Rk ) denote the probability that exactly n and m APs’ bids (among the APs in K \{k}) are in [0, C) and [C, Rk ], respectively (while the remaining (K − 1 − n − m) APs’ bids are in (Rk , rmax ]). The expression of pn,m (·) follows a multinomial distribution and is given by pn,m (Rk ) =

m  K−1−n−m  (K − 1)! 1 − F(Rk ) F(C)n F(Rk ) − F(C) n! m! (K − 1 − n − m)!

(10)

where, recall that, F(·) denotes the common c.d.f of Rj , j ∈ K . Note that qn := K−1−n pn,m (Rk ) is simply the probability that the bids of exactly n APs take values m=0 in [0, C) (note that qn is not dependent on Rk ). Finally, using pn,m (Rk ) we define the function H (·) as H (Rk ) =

N −1  n=0

N −1−n  m=0

  (N − n − m)δA + (K − N ) pn,m (Rk ) C − Rk K −n−m +

K−1−n  m=N −n

(N − n)(C − Rk ) pn,m (Rk ) m+1



(11)

The following lemma then generalizes the result in Yu et al. [7, Lemma 1]. Lemma 1 For N < K there exists a TN ,C ∈ (C, rmax ) satisfying H (TN ,C ) = 0 (the subscripts in the notation of T are used to represent its dependence on (N , C)). Proof (Outline) We first show that H (Rk ) is continuous in Rk . Then, we prove that the extreme points satisfy H (C) > 0 and H (rmax ) < 0. Finally, employing intermediate value theorem we conclude that H (TN ,C ) = 0 for some TN ,C ∈ (C, rmax ). The details are available in Sect. 7.1. Remark For the case N = K, each AP has to invariably decide whether to cooperate or compete. Cooperation yields a pay-off of C while competition results in a discounted rate of δA Rk . Thus, the threshold TK,C can be obtained by simply comparing both the pay-offs: TK,C = C/δA . We now present our main result.

Coexistence of LTE-Unlicensed and WiFi with Optimal Channel Aggregation

45

Theorem 1 Given a TN ,C satisfying H (TN ,C ) = 0, the following bidding strategy is a SBNE: ⎧ ⎨ Rk if Rk ∈ [0, C) b∗N ,C (Rk ) = C if Rk ∈ [C, TN ,C ] (12) ⎩ X if Rk ∈ (TN ,C , rmax ]. Discussion: First, we see that truthful bidding constitutes a Nash equilibrium strategy for AP-k (k ∈ K ) whenever its rate Rk ∈ [0, C). Next, in the regime Rk ∈ [C, TN ,C ] bidding a truncated value of C constitutes a Nash equilibrium strategy; thus, AP-k is better-off participating in the auction (by bidding a lower value of C) rather than compete with the LTE-U by not participating. Finally, when Rk > TN ,C the rate value is large enough for AP-k to accrue a better pay-off by not participating in the auction (by bidding X ). Proof (Outline) The theorem has three parts. The proof of Part-I (i.e., Rk ∈ [0, C)) is along the lines of the solution to the standard uniform-price auction, for which it is known that at equilibrium the bidders have no incentive to deviate from their true values. Our proof, in fact, generalizes the standard setting by extending the bid values to encompass X which denotes the option to not participate in the auction. For Part-II (i.e., Rk ∈ [C, TN ,C ]), we first compare expected pay-off achieved by AP-k for bidding C (denoted ΠC (Rk ) for simplicity) against the pay-off received for bidding X (denoted ΠX (Rk )) and show that ΠC (Rk ) ≥ ΠX (Rk ); the expression of these pay-off terms is as given below. ΠC (Rk ) =

N −1

⎡ ⎣

N −1−n 

n=0

ΠX (Rk ) =

N −1 n=0

pn,m (TN ,C ) C +

m=0

⎡ ⎣

N −1−n  m=0

pn,m (TN ,C )

K−1−n  m=N −n

 pn,m (TN ,C )

⎤  K−1  (N − n)(C − Rk ) qn Rk + Rk ⎦ + m+1

(N − n − m)δA + (K − N ) Rk + K −n−m

(13)

n=N

K−1−n  m=N −n

⎤ pn,m (TN ,C ) Rk ⎦ +

K−1 

qn Rk

n=N

(14) Next, we show that ΠC (Rk ) is also greater than the pay-off received for bidding any sk ∈ [0, C), thus concluding that bidding C is optimal whenever Rk ∈ [C, TN ,C ]. Part-III is similarly completed by showing that ΠX (Rk ) > ΠC (Rk ) whenever Rk ∈ (TN ,C , rmax ] so that bidding X is optimal. Details of the proof are available in Sect. 7.2.

4.2 Case-2: C ∈ [rmax , ∞) Here the APs are not restricted from bidding their true rate values. The scenario hence reduces to the tradition uniform-price auction setting. As a result, it follows that truthful bidding constitutes a solution. We formally state this result:

46

N. Kolar Purushothama

Theorem 2 When C ∈ [rmax , ∞), the following strategy is a SBNE: b∗N ,C (Rk ) = Rk for all Rk ∈ [0, rmax ]. Thus, truthful bidding constitutes a solution in this case. Proof As in the proof of Theorem 1 in Sect. 7.2, let e denote the vector of bids of all APs other than k. Note that here e = R−k = (R1 , . . . , Rk−1 , Rk+1 , . . . , RK ) as all other APs are assumed to be adhering to truthful bidding. Let e(1) ≤ e(2) ≤ · · · ≤ e(N −1) denote the ordered statistics of the values in e. Then the remainder of the proof is identical to the proof of Part-I, Case-(a) (see Sect. 7.2); we do not repeat the details here for brevity.

5 Computation of Expected Pay-offs We first make the following simplifying assumption: Assumption 1 Threshold TN ,C in Lemma 1 is unique. The above assumption implies that the equilibrium strategy b∗N ,C (·) derived in the previous section (in particular recall (12)) is defined uniquely for a given (N , C). In this section, imposing the above assumption, we derive explicit expressions for the expected pay-offs defined in (7) and (8). The validity of the assumption will be confirmed through numerical experiments in Sect. 6. We proceed by again considering the cases C ∈ [0, rmax ) and C ∈ [rmax , ∞) separately.

5.1 Case-1: C ∈ [0, rmax ) Recall from (4) that ML denotes the number of APs whose bids are in the interval [0, C]. From the structure of b∗N ,C (·) in (12), it follows that ML is given by the number of APs whose true rates are in the interval [0, TN ,C ]. Thus, we have P(ML = m) =

  K F(TN ,C )m (1 − F(TN ,C ))K−m . m

For simplicity, we use λm to denote P(ML = m). Now, using the pay-off function in LTE (N , C) can be written as (5) the expression for Π LTE (N , C) = Π

N  m=0

  λm m(RLTE − C) + (N − m)δL RLTE +

K  m=N +1

 

λm N RLTE − E b(+1) ML = m

(15) where b(+1) denotes the value of the (N + 1)th minimum bid (which represents the payment made to all the winning APs; recall the discussion in Sect. 3.1). The LTE (N , C) can be expression for the expectation of b(+1) required to compute Π obtained as follows.

Coexistence of LTE-Unlicensed and WiFi with Optimal Channel Aggregation

47

First, for simplicity, let us denote the rate values of the ML = m APs (whose rates are in [0, TN ,C ]) as R1 , R2 , . . . , Rm . Conditioned on the event that these rates are in [0, TN ,C ], their common (conditional) c.d.f and p.d.f, respectively, are given by f (r) F(r)  and  f (r) := F(T for r ∈ [0, TN ,C ]. Let R(1) ≤ R(2) ≤ · · · ≤ R(m) F(r) := F(T N ,C ) N ,C ) denote the order statistics of the above random variables [8]. Then, the expression for b(+1) can be computed using the (N + 1)th-order statistics R(N +1) . Specifically, we have

 

E b(+1) ML = m =

C 0

  r fR(N +1) (r) dr + C 1 −  FR(N +1) (C)

(16)

   where  fR(N +1) (r) = m m−1 F(r))m−1−N  f (r), and  FR(N +1) (·) is the correF(r)N (1 −  N sponding c.d.f. AP (N , C), we first fix an AP-k ∈ K . Then, recalling the expressions To compute Π in (13) and (14), we recognize that the pay-off to AP-k is given by ΠC (Rk ) whenever Rk ∈ [C, TN ,C ] and ΠX (Rk ) for Rk ∈ (TN ,C , rmax ]. To similarly obtain the pay-off when Rk ∈ [0, C), we first define μn,m (Rk ) to be the probability that exactly n and m APs’ rates (out of the (K − 1) APs in K /{k}) take values in [0, Rk ] and (Rk , C) respectively. The expression for μn,m (Rk ) can be written as μn,m (Rk ) =

 K−1−n−m m  (K − 1)! 1 − F(C) (17) F(Rk )n F(C) − F(Rk ) n! m! (K − 1 − n − m)!

Then, the pay-off (denoted ΠR (Rk )) when Rk ∈ [0, C) is given by ΠR (Rk ) =

N −1 

N −1−n 

n=0

m=0

μn,m (Rk )C +

 K−1 K−1−n    μn,m (Rk )E  μn,m (Rk )Rk R(N −n) +

K−1−n  m=N −n

(18)

n=N m=0

where  R(N −n) is the (N − n)th-order statistics of i.i.d random variables  R1 ,  R2 , . . . , f (r) 3  f (r) = F(C)−F(R for r ∈ (R , C). Finally, Rm whose common p.d.f is given by  k k) AP  using the above conditional pay-offs, the total pay-off Π (N , C) achieved by AP-k can be expressed as AP (N , C) = Π



C

 f (rk )ΠR (rk )drk +

0

TN ,C C

 f (rk )ΠC (rk )drk +

rmax

f (rk )ΠX (rk )drk .

TN ,C

(19)

5.2 Case-2: C ∈ [rmax , ∞) Since all APs participate in the auction by bidding their true rate values (recall Theorem 2), the expected pay-off to LTE-U in this case is given by variables  R1 ,  R2 , . . . ,  Rm denote the rates (of exactly m APs) that lie in the interval (Rk , C); hence, their p.d.f  f (·) is derived by truncating the unconditional p.d.f f (·).

3 Random

48

N. Kolar Purushothama

  LTE (N , C) = N RLTE − E[R(N +1) ] Π

(20)

where R(N +1) denotes the (N + 1)th-order statistics of R1 , R2 , . . . , RK . Finally, to determine APs’ pay-off, we define νn (Rk ) as the probability that exactly n (out of K − 1) APs’ rates lie in the interval [0, Rk ], i.e., νn (Rk ) =

 (K−1−n) (K − 1)! F(Rk )n 1 − F(Rk ) n!(K − 1 − n)!

Then, the expected pay-off can be written as AP (N , C) = Π



rmax 0

f (rk )

N −1 

νn (rk )E[R(N −n) ] +

n=0

K−1 

 νn (rk )rk drk

n=N

where R(N −n) is the (N − n)th-order statistics of i.i.d random variables R1 , R2 , . . . , f (r) RK−1−n whose common p.d.f is given by f (r) = 1−F(R for r ∈ [Rk , rmax ]. k)

6 Numerical and Simulation Results We begin by presenting the details of the considered empirical setting. The total number of APs (i.e., channels) is chosen to be K = 5. The rate random variables, Rk , are assumed to follow truncated Gaussian distribution of mean 125 MBPS, variance 2500 MBPS 2 , and truncated to take values in the interval [0, 200 MBPS]. The contention-free rate of LTE-U is RLTE = 100 MBPS. Finally, the values of the discount factors are chosen to be δL = 0.4 and δA = 0.3. The motivation for considering the above setting comes from the recent work of Yu et al. [7]. We first demonstrate the validity of Assumption 1 about the uniqueness of TN ,C . In Fig. 1a, we plot H (Rk ) in (11) as a function Rk for different values of N ; the value of C is fixed at 50 MBPS. Similarly, in Fig. 1b we plot H (Rk ) for different values of C by fixing N = 3. From both the plots, we see that each curve intersects the x-axis (shown as horizontal dashed line) only once, thus implying that TN ,C is unique (recall that TN ,C is the solution to H (Rk ) = 0). Further observations about the behavior of TN ,C can be made from these plots. For instance, from Fig. 1a we see that TN ,C increases with N for a fixed C. This is because, a larger value of N implies an higher probability that a tagged AP, say AP-k, may be chosen for coopetition. Thus, it is profitable for AP-k to cooperate than compete, unless its rate Rk is already high. Similarly, from Fig. 1b we see that TN ,C increases with C for a fixed N . Again the reason is because a higher value of C yields an higher pay-off for cooperating; hence, the threshold TN ,C where the transition from cooperation to competition occurs increases with C. We now proceed to the performance evaluation part. In Fig. 1c, we plot the aggregation number N ∗ and the reserve rate C ∗ as functions of the constraint R on the APs’ rate in the pay-off maximization problem in (9); recall that (N ∗ , C ∗ ) is the optimal

Coexistence of LTE-Unlicensed and WiFi with Optimal Channel Aggregation

49

(b)

(a)

(c)

Fig. 1 Plot of the function H (Rk ) in (11): a for C = 50 MBPS and different values of N , b for N = 3 and different values of C (in MBPS), and c optimal channel aggregation number N ∗ and reserve rate C ∗ as functions of rate constraint R

point of the problem in (9). Figure 1c depicts an interesting interaction that occurs between N ∗ and C ∗ , which can be understood as follows. When the constraint R is low, LTE-U’s rate is maximized by simply aggregating all the K = 5 channels. As R increases, the rate constraint is satisfied by increasing the reserve rate C ∗ that is offered to the cooperating APs. However, we see that there is a threshold (on R) beyond which LTE-U’s pay-off is maximized by aggregating one-less channel, rather than by increasing C ∗ further. This phenomenon continues, and eventually for large values of R we have N ∗ = 1 so that the constraint R is met by reducing the probability of contention with the APs. In Fig. 2a, we depict the performance of our algorithm (labeled Optimal Agg) where optimal number of channels are aggregated by LTE-U. The performance is evaluated in terms of the rate achieved by LTE-U as the APs’ rate constraint R

50

N. Kolar Purushothama

(a)

(b)

Fig. 2 a LTE-U rate versus R for RLTE = 100 MBPS. b LTE-U rate versus RLTE for R = 80 MBPS

is varied. We observe that a natural trade-off exists between LTE-U’s rate and the minimum rate that can be guaranteed to the APs. At lower values of R, LTE-U achieves an higher rate by aggregating all the channels (recall Fig. 1c). On the other hand, as R increases, the rate constraint is guaranteed by either aggregating fewer channels or by offering a higher reserve rate; in the process, the rate achieved by LTEU decreases. We have also compared the performance of our Optimal Agg algorithm against the strategy of aggregating a fixed number of channels (labeled Fixed Agg N = n for n = 1, 2, 3). The optimization framework for implementing Fixed Agg N = n is similar to the setting in (9) except that here we fix N = n, so that the optimization is only over C ∈ [0, RLTE ]. We note that the case N = 1 corresponds to the recent work of Yu et al. [7]; the other cases can thus be considered as generalization of the work in [7]. Although the performance of a Fixed Agg strategy coincides with that of Optimal Agg strategy over some range of R (for instance, for R ∈ [115, 120] in the case of Fixed Agg N = 2), the performance difference is large for values of R away from the range (again for Fixed Agg N = 2, the performance difference is approximately 175 MBPS when R is set to 40 MBPS). Further, aggregating a fixed number of channels may impose an upper bound on the rate that the APs’ can achieve. For instance, when N = 3 channels are aggregated, it is not possible for the APs to achieve a rate of more that 115 MBPS; hence, the curve corresponding Fixed Agg N = 3 strategy abruptly drops to 0 MBPS at R = 115 MBPS indicating that the optimization constraint in (9) is not feasible when N = 3. Also shown in Fig. 2a is the performance of a heuristic algorithm (labeled Heuristic) where LTE-U randomly chooses a subset of channels for aggregation. An AP whose channel is chosen will decide to cooperate with LTE-U with probability p, in which case LTE-U allocates rate C to the AP; with the remaining probability, the AP decides to compete, in which case the rates get discounted due to contention. The performance of the heuristic algorithm is obtained by optimizing, over (p, C), the rate achieved by LTE-U subject to a constraint on the APs’ rate as before. We see that our algorithm outperforms the heuristic for larger values of R. However, there is a

Coexistence of LTE-Unlicensed and WiFi with Optimal Channel Aggregation

51

range of R where the heuristic achieves a better performance. At this point, we would like to emphasize that the heuristic is implemented only for the sake of comparison; it does not constitute a rational solution since it may be possible for an AP to benefit by unilaterally deviating to a different strategy. Finally, in Fig. 2b we compare the performances of the above algorithms by varying RLTE ; we fix R = 80 MBPS. We find that our algorithm, employing optimal channel aggregation, achieves a better rate for LTE-U. Further, note that the performance improvement is higher for larger values of RLTE . This is because, large RLTE allows LTE-U to offer a high reserve rate C (so that the rate constraint is met) while maximizing its pay-off by aggregating all K = 5 channels. In summary, through our simulation experiments we find that channel aggregation results in significant performance gains for LTE-Unlicensed, while still ensuring a minimum rate guarantee for the WiFi APs.

7 Proof of Main Results 7.1 Proof of Lemma 1 Proof Since Rj are continuous random variables, it follows that F(·) is continuous. Hence, pn,m (Rk ), being a product of continuous functions, is continuous. Similarly, H (Rk ) in (11) which is expressed as a sum of product of continuous functions is continuous as well. We now proceed to compute H (C). For this, first note that pn,m (C) = 0 for m = 0, while for m = 0 we have 

  K−1−n K −1 pn,0 (C) = F(C)n 1 − F(C) n for n = 0, 1, . . . , N − 1. Note that f (r) > 0, ∀r ∈ [0, rmax ], implies that 0 < F(C) < 1, so that we have pn,0 (C) > 0. Thus, recalling that δA ∈ (0, 1), we obtain 

N −1 

(N − n)δA + (K − N ) C pn,0 (C) C − H (C) = K −n n=0 =

N −1  n=0

pn,0 (C)



(N − n)(1 − δA ) C > 0. K −n

Similarly, to compute H (rmax ) we first note that pn,m = 0 for m = (K − 1 − n), while for m = K − 1 − n    m K −1 F(C)n F(rmax ) − F(C) . pn,m (rmax ) = n

52

N. Kolar Purushothama

Since 0 < F(C) < F(rmax ) < 1 (again because f (r) > 0 ∀r ∈ [0, rmax ]), we have pn,K−1−n > 0. Thus, we have H (rmax ) =

N −1 

pn,K−1−n (rmax )

n=0

(N − n)(C − rmax ) < 0. K −n

Finally, the proof is completed by invoking the intermediate value theorem.

7.2 Proof of Theorem 1 Proof Fix an AP k ∈ K . Let e denote the vector of bids of all APs other than k, i.e., e = (ej : j ∈ K \{k}). The bid of AP-k is denoted sk . Assuming that all APs in K \{k} bid according to the strategy b∗N ,C (·) in (12) (i.e., ej = b∗N ,C (Rj ) ∀ j ∈ K \{k}), we show that AP-k has no incentive to deviate from b∗N ,C (·). We consider different parts in the RHS of (12) separately. Part-I (Rk ∈ [0, C)): Let e(1) ≤ e(2) ≤ · · · ≤ e(K−1) denote the ordered statistics [8] of the bid values in e. There are two cases that are possible: (a) e(N ) ∈ [0, C], and (b) e(N ) = X . Under case-(a), the scenario reduces to an uniform-price auction setting (recall Footnote-1) where it is known that truthful bidding is a weakly dominant strategy; we derive the result here for completeness. Suppose that AP-k makes an under-bid of sk < Rk . We then have the following: • If e(N ) < sk , AP-k will lose the auction irrespective of whether it bids sk or Rk , thus fetching an identical pay-off of Rk . • On the other hand, if e(N ) > Rk , AP-k will win the auction bidding either sk or Rk , fetching an identical pay-off of e(n) . • If sk < e(N ) < Rk , bidding sk a pay-off of e(N ) is accrued by AP-k (by winning the auction), while bidding Rk yields an higher pay-off of Rk (which corresponds to the pay-off for losing the auction). • If e(N ) = sk , then bidding sk yields a pay-off in the interval (e(N ) , Rk ) (since, because of the tie, AP-k wins the auction with some non-zero probability), while Rk yields a higher pay-off of Rk (by losing the auction). Finally, if e(N ) = Rk the pay-off (equal to Rk ) fetched by bidding sk or Rk is identical. Similarly, when AP-k makes an over-bid sk > Rk (including X ), it can be shown that AP-k cannot benefit by deviating from its true value Rk . Now, consider case-(b) where e(N ) = X . For any value of the bid sk ∈ [0, C] AP-k wins the auction, achieving a pay-off of C. If sk = X , the achieved pay-off lies in the interval (Rk , C). Thus, AP-k has no benefit of deviating from Rk . Part-II (Rk ∈ [C, TN ,C ]): Recall the expressions for ΠC (Rk ) and ΠX (Rk ) from (13) and (14), respectively. It is easy to see that ΠC (Rk ) − ΠX (Rk ), as a function of Rk , is strictly decreasing. Further, the value of ΠC (Rk ) − ΠX (Rk ) evaluated at Rk = T is H (T ) = 0 (recall (11) and Lemma 1). Hence, we have ΠC (Rk ) ≥ ΠX (Rk )

Coexistence of LTE-Unlicensed and WiFi with Optimal Channel Aggregation

53

for all Rk ∈ [C, T ] with the inequality being strict for Rk ∈ [C, T ). Thus, the pay-off received by AP-k for bidding C is greater than that received for bidding X . We now compare bid C against any bid sk ∈ [0, C). We consider different cases depending on the value of e(N ) . Suppose e(N ) ∈ [0, sk ), then bidding C or sk will both fetch an identical pay-off of Rk to AP-k (by losing the auction). Similarly, if e(N ) = X , then a pay-off of C is accrued by bidding C or sk . A difference in pay-off is incurred only when e(N ) ∈ [sk , C]. We consider the following sub-cases: • If e(N ) = sk , then bidding sk yields a pay-off in (sk , Rk ) while bidding C yields an higher pay-off of Rk . • If e(N ) ∈ (sk , C), then sk yields a pay-off of e(N ) while C yields a pay-off of Rk > e(N ) . • Finally, if e(N ) = C then sk fetches a pay-off of C while bidding C yields a pay-off in (C, Rk ). From the above discussion, we see that whenever Rk ∈ (C, T ] the strategy of bidding C yields an higher pay-off to AP-k. Part-III (Rk ∈ (TN ,C , rmax ]): The proof of this part is analogous to the proof of Part-II. Indeed, recalling that the difference ΠC (Rk ) − ΠX (Rk ) is decreasing in Rk , it follows that ΠC (Rk ) < ΠX (Rk ) whenever Rk > T (since the difference is 0 at Rk = T ). The argument in Part-II can be similarly extended to show that bidding X is better than bidding any sk ∈ [0, C]. Thus, the strategy of bidding X is optimal for this part.

8 Conclusion We designed an auction mechanism for WiFi APs to decide their mode of coexistence with LTE-U. While the solution to the auction problem is characterized by equilibrium bidding strategy (Theorems 1 and 2), efficient coexistence is obtained by computing optimal aggregation number and reserve rate that maximizes LTE-U’s rate subject to a constraint on the APs’ rate. Through simulation work, we demonstrated the benefit of optimal channel aggregation in LTE-Unlicensed.

References 1. “Cisco Visual Networking Index: Forecast and Methodology, 2016–2021,” Cisco White Paper, June 2017. 2. C. Cano, D. Lopez-Perez, H. Claussen, and D. J. Leith, “Using LTE in Unlicensed Bands: Potential Benefits and Coexistence Issues,” IEEE Communications Magazine, vol. 54, no. 12, pp. 116–123, December 2016. 3. R. Zhang, M. Wang, L. X. Cai, Z. Zheng, X. Shen, and L. L. Xie, “LTE-Unlicensed: The Future of Spectrum Aggregation for Cellular Networks,” IEEE Wireless Communications, vol. 22, no. 3, pp. 150–159, June 2015.

54

N. Kolar Purushothama

4. Z. Guan and T. Melodia, “CU-LTE: Spectrally-Efficient and Fair Coexistence between LTE and Wi-Fi in Unlicensed Bands,” in IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications, April 2016. 5. R. Zhang, M. Wang, L. X. Cai, X. Shen, L. L. Xie, and Y. Cheng, “Modeling and Analysis of MAC Protocol for LTE-U Co-Existing with Wi-Fi,” in 2015 IEEE Global Communications Conference (GLOBECOM), Dec 2015. 6. Q. Chen, G. Yu, H. Shan, A. Maaref, G. Y. Li, and A. Huang, “Cellular Meets WiFi: Traffic Offloading or Resource Sharing?” IEEE Transactions on Wireless Communications, vol. 15, no. 5, pp. 3354–3367, May 2016. 7. H. Yu, G. Iosifidis, J. Huang, and L. Tassiulas, “Auction-Based Coopetition Between LTE Unlicensed and Wi-Fi,” IEEE Journal on Selected Areas in Communications, vol. 35, no. 1, pp. 79–90, Jan 2017. 8. H. A. David and H. N. Nagaraja, Order Statistics (Wiley Series in Probability and Statistics). Wiley-Interscience, August 2003.

Analysis of Sponsored Data Practices in the Case of Competing Wireless Service Providers Patrick Maillé and Bruno Tuffin

Abstract With wireless sponsored data, a third party, content or service provider, can pay for some of your data traffic so that it is not counted in your plan’s monthly cap. This type of behavior is currently under scrutiny, with telecommunication regulators wondering if it could be applied to prevent competitors from entering the market and what the impact on all telecommunication actors can be. To answer those questions, we design and analyze in this paper a model where a content provider (CP) can choose the proportion of data to sponsor and a level of advertisement to get a return on investment, with several Internet service providers (ISPs) in competition. We distinguish three scenarios: no sponsoring, the same sponsoring to all users, and a different sponsoring depending on the ISP you have subscribed to. This last possibility may particularly be considered an infringement of the network neutrality principle. We see that sponsoring can be beneficial to users and ISPs depending on the chosen advertisement level. We also discuss the impact of zero-rating where an ISP offers free data to a CP to attract more customers, and vertical integration where a CP and an ISP are the same company.

1 Introduction With the improving capacity of smartphones, wireless data consumption is exponentially increasing. According to previsions,1 mobile data consumption will be sevenfold larger by 2021 than by 2017. Wireless communication subscription offers are often made of unlimited telephony but cap on data over which a volume-based 1 See

http://www.businessinsider.fr/us/mobile-data-will-skyrocket-700-by-2021-2017-2/ among others.

P. Maillé IMT Atlantique/IRISA, Rennes, France e-mail: [email protected] B. Tuffin (B) Inria, Rennes, France e-mail: [email protected] © Springer Nature Switzerland AG 2019 J. Walrand et al. (eds.), Network Games, Control, and Optimization, Static & Dynamic Game Theory: Foundations & Applications, https://doi.org/10.1007/978-3-030-10880-9_4

55

56

P. Maillé and B. Tuffin

fee is applied. Users may therefore be forced to monitor and even limit their data usage. Content providers (CPs) on the other hand are economically dependent on the amount of data consumed by users, typically via displayed advertisements: The more the volume consumed, the more the advertisement and then revenue. For this reason, there might be an incentive for content providers to sponsor data, that is, to at least partially pay the Internet service provider (ISP) for the user’s consumption of its own services. This way, users may expand their usage, CPs may get more revenue due to advertisement, and ISPs are paid for the traffic increase. Examples in the USA are: Netflix or Binge On with T-Mobile, DIRECTV and U-verse Data-Free TV with AT&T, etc. ISPs are even offering services such that CPs can, if they wish, apply sponsored data: Examples are Verizon Wireless, AT&T Mobility and T-Mobile US, or Orange with DataMI in France, among others. But sponsored data bring concerns from user associations and small content providers. It is claimed to give an unfair advantage to some content/service providers and to eventually prevent some actors from entering the market, due to lower visibility and high entrance costs to the sponsored data system if they want to get the same service as incumbent and big providers already present. The culmination of the sponsored data principle is the so-called zero-rating where ISPs (freely) remove some content providers from their data caps, hoping more customers will subscribe due to potentially unlimited usage. Those providers could then hardly be challenged. The French ISP SFR did it with YouTube in its offer RED a few years ago. Similarly, Facebook, Google, and Wikipedia have built special programs in developing countries, with the claimed goal to increase connectivity and Internet access. For those reasons, sponsored data and zero-rating are currently under investigation by regulators to determine if rules should be imposed to prevent or limit their use. In Chile, for example, the national telecom regulator has stated that it was violating net neutrality laws and that it should be forbidden. Net neutrality means that all packets/flows are treated the same, independently of their origin, destination, and type of service [3, 4]. The purpose of this paper is to design and analyze a mathematical model of sponsored data to understand if this type of behavior should be banned or encouraged. To our knowledge, very few similar works exist that we now summarize and from which we differentiate ourselves. The notable works are first [2], where the authors consider a model with a discrete set of users, a single ISP, and several (complement) CPs able to choose a different level of sponsoring per user. They show that sponsoring can benefit more to users than to CPs. Our model presents similarities with that one but strongly differs on many aspects: We use a different model for CPs’ advertisement level and determination of the best strategy; our model also represents the negative externality of advertisements on users, something often forgotten, and a different model of volume consumption highlighting the correlation between willingness-topay for connectivity and consumed volume of data; finally and more importantly, we model competition between ISPs to better link the sponsored data practice to the network neutrality debate. We can also mention [7], dealing with several substitutable CPs in competition, or [8, 9] including network externalities, but again a single ISP, or

Analysis of Sponsored Data Practices …

57

[6] which nicely combines sponsored data and caching strategies, but not considering any ISP in the equation. Our contributions are the following: (i) the use of a model different from the literature to represent user behavior. This model separates the willingness-to-pay for connectivity and for CP content. Remark that using a new model is important to better understand the assumptions that may change the results, or see the robustness of conclusions. (ii) We focus on CPs’ strategical decisions and their impact on all other actors (ISPs and users). The decision variables are the amount (proportion) of data to be sponsored and the amount of advertisement CPs will set; our study allows us to see if sponsoring data also means more advertisement for a return on investment for CPs. (iii) We investigate the best sponsoring and advertisement strategies in the case of competition between ISPs; three situations are compared: (a) no sponsored data, (b) the same level of sponsoring at all ISPs, and (c) the possibility to sponsor differently according to the ISP; these situations also correspond to three different levels of neutrality in the net neutrality debate. (iv) We investigate and compare with all other situations the case of zero-rating, when the CP content is not counted in the user data cap; ISPs could indeed be in favor of such a strategy if it brings additional subscriptions. (v) Finally, we look at the case when the CP is managed by one ISP, that is the so-called vertical integration; does it lead to an unfair advantage if data are sponsored with respect to a non-sponsored situation? This corresponds to practical situations within an increasingly vertically integrated ecosystem. The remainder of the paper is organized as follows. Section 2 presents the model, the decision variables, and their hierarchy. Section 3 discusses the last level and how users distributes themselves among ISPs depending on prices and strategies of CPs. Then, Sect. 4 describes the three sponsoring strategies and compares their impact on all actors in the case of independent CPs and ISPs. The particular situation of zero-rating, with the CP content not included in one ISP data plan but included in the other, is investigated in Sect. 5. Section 6 presents the case of a vertically integrated CP–ISP. Finally, from all the described scenarios, we conclude in Sect. 7 on the need to regulate sponsored data.

2 Model We consider three types of actors: users, CPs, and ISPs. We describe here the model with full generality and will later restrict the number of actors to focus on specific–and to our knowledge un-investigated–aspects of the sponsored data debate. Consider M CPs indexed by j and N ISPs indexed by i.

58

P. Maillé and B. Tuffin

2.1 Users We index users by θ ∈ R+ , which characterizes their type. We consider a continuum of users of total mass 1 and denote by F the cumulative distribution function of θ . Each user θ has several variables of decision: its choice of ISP that we will denote by i(θ ), but also the volume of CP j data (without advertisements) he will consume if using ISP i, which we will denote by vi, j (θ ). The dependence on i comes from the fact that prices for data may differ between ISPs, and the consumed volume too as a consequence. To simplify, let θ be the willingness-to-pay for connectivity and pi the subscription price at ISP i, which would be giving a “utility for connectivity” θ − pi . But we additionally weigh the difference using an ISP reputation parameter ai ≥ 0, meaning that users value more the gain obtained with incumbent providers than with newcomers, resulting in a “weighted utility for connectivity” ai (θ − pi ). In addition, User θ gains satisfaction from using each CP j. The cost per unit of volume he pays in his data plan is denoted by ci, j , which depends on j because that usage can be sponsored by CP j as we will see in the next subsection. Moreover,  there is a valuation for each unit of volume of CP j. Let rθ, j be the marginal valuation that we will consider to be  2 + rθ, j (x) = [θ − (α j s j )x]

for the xth unit of useful volume, i.e., without advertising, where [y]+ := max(y, 0), α j is a fixed parameter, and s j ≥ 1 corresponds to the relative increase of volume due to advertisement displayed by CP j, expressed as the total downloaded volume divided by the volume of “useful” data, excluding ads. The larger this value of s j , the smaller the service valuation for users. We consider a square value of s j to assure the reasonable assumption that at one point, for the CPs, the loss due to users unpleased by advertisements exceeds the gain from those ads (see later (2) and (3) with s j tending to infinity); any other choice can be used without difficulty though. Then, the willingness-to-pay rθ, j (x) for consuming a volume x of CP j data over a month is ⎧ ⎨ θ x − α j s 2j x 2 if x ≤ θ 2 2 αjsj rθ, j (x) = 2 ⎩ 2αθ s 2 otherwise. j j

This form is commonly adopted in [1, 5] and presents the advantage of positively correlating willingness-to-pay for data and willingness-to-pay for connectivity. We have added in the model the negative correlation due to advertisement (some kind of “pollution” which can advert users). Note that a positive effect (only) from advertisement was considered in [2] due to relevant ads which can be clicked, but ads seem to us rather negatively perceived.

Analysis of Sponsored Data Practices …

59

Our assumptions give a utility for user θ at ISP i M    Ui (θ ) := ai (θ − pi ) + rθ, j (vi, j (θ )) − ci, j vi, j (θ )s j ,

(1)

j=1

using the fact that the total downloaded volume is actually vi, j (θ )s j from the definition of s j . Users indeed also download advertisements, which are not differentiated from “real” content by ISPs. The additive expression indicates that CPs are assumed to be independent in terms of content. Our modeling, unlike those of the literature ([2] for example), chooses to represent user preferences for ISPs, in terms of connectivity and a correlation between willingness-to-pay for connectivity and consumed volume of data; it separates what comes from ISPs from what comes from CPs. If subscribing to ISP i, the volume vi, j (θ ) User θ chooses is the one maximizing rθ, j (vi, j (θ )) − ci, j s j vi, j (θ ) and can easily be computed [1] to be  vi, j (θ ) = leading to rθ, j (vi, j (θ )) − ci, j s j vi, j (θ ) =

θ − ci, j s j α j s 2j

+ ,

(2)

(θ−ci, j s j )2 1l{θ>ci, j s j } . 2(α j s 2j )

Summarizing, User θ chooses ISP i(θ ) maximizing his overall utility: i(θ ) = argmaxi Ui (θ ) = argmaxi ai (θ − pi ) +

M  (θ − ci, j s j )2 1l{θ>ci, j s j } 2(α j s 2j ) j=1

if the max is nonnegative, otherwise i(θ ) = 0, meaning no subscription at all.

2.2 CPs CP j is assumed to have a utility (a revenue) linearly increasing with the volume of displayed advertisement, with CP-dependent linear parameter β j . The advertisement volume is the total volume s j vi(θ), j (θ ) minus the “real” data volume vi(θ), j (θ ), hence (s j − 1)vi(θ), j (θ ). As a consequence, the gain for CP j is

θ

β j (s j − 1)vi(θ), j (θ )dF(θ ).

Each CP j can decide to sponsor a fraction γi, j of the data usage cost of ISP i’s users. This could be an incentive to consume more CP j content and therefore generate more revenue from advertisement. Of course, there is a trade-off, which

60

P. Maillé and B. Tuffin

is one purpose of this paper, between sponsoring cost and generated volume of consumed data. We will investigate three different sponsoring policies: 1. No sponsored data: γi, j = 0 ∀i, j; 2. The same data sponsoring level for all ISPs: γi, j = γ j ∀i, j; 3. A possible differentiation between ISPs, with γi, j = γi  , j for i = i  . In terms of the network neutrality debate, those three scenarios correspond to three different levels of neutrality. Option 1 corresponds to full neutrality. Option 2 seems neutral with respect to ISPs, even if non-neutral between applications because different applications can have different levels of sponsoring (note on the other hand that advocates of sponsored data also claim that packets are treated the same within the network, but it is not the issue we want to address here). Option 3 is the fully non-neutral one. Let qi be the unit price ISP i is (officially) charging users for data. After data sponsoring, the unit cost seen by users when consuming data of CP j is then ci, j = qi (1 − γi, j ). If sponsoring data, CP j has to pay to each ISP for the proportion of volume it has chosen to sponsor, hence a total cost: θ γi(θ), j qi(θ) s j vi(θ), j (θ )dF(θ ), where the proportion of volume paid is the proportion γ j of the total volume s j vi(θ), j (θ ) (which includes ads). Compiling all the elements, the revenue of CP j is

Gj =

θ

(β j (s j − 1) − γ j qi(θ) s j )vi(θ), j (θ )dF(θ ).

(3)

CP j has several decision variables: the sponsoring levels γi, j ∀i and the advertisement volume increased level s j ≥ 1. Note that we could also consider advertising levels that would depend on the user ISP (i.e., si, j instead of s j ): We rather impose an equal advertising level, since our focus is on the sponsoring strategies, and CPs may not know the ISP of the user upon a request for content.

2.3 ISPs Each ISP i tries to maximize its revenue. Revenue comes from subscriptions and consumed data: ⎞ ⎛

 Ri = ⎝ pi + qi s j vi, j (θ )⎠ 1l{i(θ)=i} dF(θ ). θ

j

Analysis of Sponsored Data Practices …

61

2.4 What we Analyze Our purpose in this paper is to analyze the impact of the decisions of CPs on all actors, in the context of competing ISPs. In particular, we would like to see the impact of a “neutrally” sponsoring CP (that is, γi, j = γi  , j for any ISPs i, i  ) with respect to sponsoring differently depending on ISPs and with respect to no sponsoring. Three scenarios are analyzed in the next sections: 1. In the case of independent ISPs and CPs, what are the best advertising and sponsoring levels of CPs and the impact on all actors? We can compare the outputs with optimal solutions for the three sponsoring possibilities. 2. Does zero-rating, where an ISP leaves a CP out of its data, plan to try to attract customers and harm users and the other ISPs? Should it be forbidden? 3. Does vertical integration of a CP and an ISP have a positive or negative impact on the other ISPs and on users? This, with a dominant or a non-dominant ISP. Since CPs are complements (due to the additive form of users’ utility), we will consider without much loss of generality in the next sections the case of a single CP. This in order to focus on our purpose, that is the impact of advertisement and sponsoring strategies on competition between ISPs, and on users. Hence, we can simplify the writing by removing the CP-relative indices in the notations defined before. We will also limit ourselves to two ISPs, labeled 1 and 2.

3 User Subscription Decisions User decisions depend on prices set by ISPs, but also on decisions of the CP. Even if the CP “plays” first, its optimal strategy should anticipate users’ choices. For this reason, we discuss here the repartition of users depending on fixed CP decisions. The optimal CP decisions are analyzed in the next sections, anticipating this subsequent choice (this implicitly assumes that the CP knows the distribution F). In other works using the same type of models for user preferences [1, 5], we had depending on the value of θ an interval over which users do not subscribe, then an interval (possibly empty) over which one ISP is chosen, and then another one for the other ISP (that is, users with small valuations do no subscribe, those with “intermediate” valuation go with one ISP and those with the largest go with the other). Here, things can be more complicated, with interestingly more intricate intervals. To illustrate this, consider the values a1 = 1.3, a2 = 1, s = 1, α = 1, c1 = 2, c2 = 1, p1 = 1, and p2 = 0.97; i.e., ISP 1 has a better reputation but higher subscription and usage prices. Figure 1 displays the difference max(U1 (θ ), 0) − max(U2 (θ ), 0) in terms of θ . We consider the max with 0 to show only the situations where a user θ is willing to subscribe to an ISP. We can see (when the curve gives zero) that users with small valuations θ prefer not to subscribe to any ISP; then, there is an interval where ISP 2 is preferred, then an

0

1

0.5

Users prefer ISP 2

Users do not subscribe

Users prefer ISP 1

0 Users prefer ISP 2

Fig. 1 [U1 (θ)]+ − [U2 (θ)]+ in terms of θ. In the left zone (small θ values), users do not subscribe to any provider

P. Maillé and B. Tuffin

[U1 (θ )]+ − [U2 (θ )]+

62

1.5

θ

interval where it is ISP 1, and then again ISP 2. An interpretation is as follows: Users with small valuation (but large enough to subscribe) are first interested in connectivity with a smaller connectivity price at ISP 2. At one point, due to its larger reputation, connectivity attractiveness of ISP 1 becomes larger, but it is counterbalanced by the data cost of ISP 2 (the parabola). In general, in terms of the combination of prices, we can express how many such successive intervals above the “no subscription” possibility can be found, between one and three. We do not give such details here since it is a list of cases which is not very instructive. The situation becomes even more intricate if we increase the number of ISPs (and CPs). Solving it numerically in the next section does not pose any problem.

4 Analysis with Independent CP and ISPs In all our numerical investigations, θ follows an exponential distribution with rate 1 truncated at 2, i.e., F(θ ) = (1 − e−θ )/(1 − e−2 ) for θ ∈ [0, 2]. Unless specified otherwise (i.e., when a parameter explicitly varies), the parameter values we consider throughout all the paper are summarized in Table 1. ISP 1 has the position of an incumbent: With a better reputation than its competitor, it can charge higher subscription and per-usage prices. We discuss in this section the case where the ISP and CP are independent. We display in Fig. 2 the CP revenue as s varies for fixed values of the other parameters, including fixed values of the sponsoring coefficient γi on ISP i = 1, 2, but using the subsequent user subscription decision. It can be checked that there is an optimal level Table 1 Parameter values for the numerical investigations

a1

a2

p1

p2

q1

q2

β

α

1.5

1

1

0.6

0.6

0.4

1

1

Analysis of Sponsored Data Practices … Fig. 2 CP revenue, with γ1 = 0.7, γ2 = 0.4

63

CP revenue

0

1

2

3

4

Advertisement overhead s =

5

6

total volume volume without advertisement

of advertising s ∗ around 2.7 above which increasing the displayed ads diminishes revenue. In Fig. 3, we display the CP-revenue-maximizing sponsoring levels γi as s varies and the corresponding CP revenue. We can see that the optimal γi is increasing with the level of advertisement s: The more ads you put, the more you can sponsor data because you earn more, and the more you have to do to compensate the negative externality for users. Typically also, applying the same level of sponsoring (γ1 = γ2 ) leads to a sponsoring level in between the differentiated levels of sponsoring (γ1 = γ2 ). Similar to Fig. 2 where sponsoring was fixed, in the present case of optimal sponsoring for each s, there is an optimal advertising level for each sponsoring scenario in Fig. 3 (right). As expected, the CP revenue with differentiated sponsoring is larger than the one with identical sponsoring, itself larger than the one with no sponsoring, because the optimization is on a larger set of sponsoring parameters. When the advertisement level is low, there is no significative difference between ·10−2 4

0.5 γ1 = γ2 γ1 γ2

0 2

4 s

CP revenue G

γ

1

γ1 = γ2 = 0 Opt. γ1 = γ2 Opt. (γ1 , γ2 )

2

0 2

4 s

Fig. 3 CP-revenue-maximizing sponsoring coefficient γ (left), and corresponding CP revenue (right) in each scenario

64

P. Maillé and B. Tuffin γ1 = γ2 = 0 γ1 = γ2 Opt. (γ1, γ2)

0.45

ISP 1

0.3

γ1 = γ2 = 0 γ1 = γ2 opt. (γ1, γ2)

0.2 ISP 2

Consumer Surplus

Revenue ISPs

0.4

0.1

0.4

0.35 2 4 Advertisement level s

2 4 Advertisement level s

Fig. 4 ISP revenues (left) and consumer surplus (right) with CP-revenue-maximizing γ values

the three options; it is significant when s is large. The optimal advertising levels s ∗ are 1.34 with no sponsoring, 1.34 too with identical sponsoring, and 1.44 with differentiated sponsoring, a larger value. “No sponsoring” or “identical sponsoring” yields a similar optimal revenue, while it is significantly better for differentiated sponsoring which then seems the appropriate option for the CP. Figure 4 displays the impact of sponsoring on the revenues of ISPs and on consumer surplus. In terms of ISP revenue first, we can see that ISP 1, the one with the highest reputation, makes more money than ISP 2. For very low values of s, the three policies give the same revenues. ISP 2 always prefers sponsoring to no sponsoring, while for ISP 1 an identical sponsoring is always the best option, something not so obvious at first sight. As a second choice for ISP 1, depending on the advertisement level, preference varies between no sponsoring and a differentiated one. Sponsoring is also always preferred by users, since giving a higher consumer surplus. On the other hand, the consumer surplus decreases as the advertisement level increases: The increased sponsoring level does not compensate enough the negative externality of ads. Interestingly, there is no significant difference for users between the two sponsoring options, and no dominance either. From the figures, sponsoring seems a relevant option for all actors, with some differences in preferences between the two sponsoring options depending on the actors. Table 2 compares the outputs at the optimal advertising levels selected by the CP. Note that in the case of identical sponsoring policy, the optimal advertising level is Table 2 Comparison of the three sponsoring scenarios with CP-revenue-maximizing advertisement and sponsoring levels, for the parameter values of Table 1 s∗ γi G R1 R2 CS No sponsoring Id. sponsoring Dif. sponsoring

1.34 1.34 1.45

(0,0) (0,0) (0,0.41)

0.0378 0.0378 0.0411

0.4124 0.4124 0.3317

0.0931 0.0931 0.1559

0.3813 0.3813 0.3770

Analysis of Sponsored Data Practices …

65

such that there is no sponsoring at all. Hence, no sponsoring and identical sponsoring lead to the same output with our set of parameters. Differentiated sponsoring leads to slightly more ads. With this optimal choice, only customers of ISP 2 get sponsored data, 41% of data being sponsored. It leads to a larger CP revenue than the two other options. The revenue of ISP 1 decreases because attracting less customers, to the benefit of ISP 2. Consumer surplus is then slightly decreased. In conclusion, with this set of parameters and optimal advertising, identical sponsoring is of no help since leading to no sponsoring at all, and differentiated sponsoring just slightly changes consumer surplus but really modifies ISP revenues to the advantage of the one with the lowest reputation; this could be beneficial for competition with incumbent providers having usually a better reputation than newcomers.

5 Zero-Rating Zero-rating consists in leaving a CP out of the data plan, making it free of access for users. The potential interest for an ISP is to attract more customers and therefore compensate the volume-based revenue loss by a subscription revenue increase. We study here whether this kind of strategy makes sense and what is the impact on all actors. Implementing zero-rating translates into ci, j = 0. In the case of a single CP, it is equivalent to considering qi = 0. Figures 5, 6, 7, and 8 alternatively display the outputs in the cases where ISP 1 or ISP 2 implements zero-rating. Looking at the two scenarios helps to see if zero-rating is more relevant and has more impact depending on the market position of the ISP. Indeed, by keeping all values of parameters the same as in the previous section, ISP 1 is more established than ISP 2. We see in Fig. 5 (left) that in the case of zero-rating for ISP 1, the optimal sponsoring strategy for ISP 2 does not impact much the CP revenue, whatever the level of advertising. With respect to no zero-rating in Fig. 3, the CP gain is substantial.

0.1 CP revenue

0.1 CP revenue

γ1 = 0 γ1 > 0

0.05 γ2 = 0 γ2 > 0

0 1

2 3 Advertisement level s

4

0.05

0 1

2 3 Advertisement level s

Fig. 5 CP revenue with zero-rating for ISP 1 (left) or for ISP 2 (right)

4

66

P. Maillé and B. Tuffin

γ

1

0.5

Opt. γ 2 (zero-rating for ISP 1) Opt. γ 1 (zero-rating for ISP 2)

0 1

1.5

2 3 2.5 3.5 Advertisement level s

4

Fig. 6 Optimal γ for an ISP, with zero-rating for the other ISP

0.4 ISP revenues

ISP Revenues

0.4

ISP 1, γ 2 = 0 ISP 2, γ 2 = 0 ISP 1, γ 2 > 0 ISP 2, γ 2 > 0

0.2

0

0.2

0 1

2 3 Advertisement level s

4

1

2 3 Advertisement level s

4

Fig. 7 ISP revenues with zero-rating for ISP 1 (left) and for ISP 2 (right) Fig. 8 Consumer Surplus with zero-rating for one ISP Consumer Surplus

0.7

γ 2 = 0, zero-rating for ISP 1 γ 2 > 0, zero-rating for ISP 1 γ 1 = 0, zero-rating for ISP 2 γ 1 > 0, zero-rating for ISP 2

0.6 0.5 0.4 1

1.5

2 3 2.5 3.5 Advertisement level s

4

Similar results can be observed when it is ISP 2 which applies zero-rating: substantial gain for the CP, but at a smaller advertising level than when it is applied by ISP 1, while at high advertising levels the difference between no sponsoring or sponsoring for ISP 1 is significant. At the optimal advertising level, there is no difference

Analysis of Sponsored Data Practices …

67

though. In Fig. 6, we can see that, again, the sponsoring level increases with the level of advertisement. But it can be checked that for smaller advertising levels, there is no sponsoring (which is understandable due to the low(er) revenue of the CP based on visits) and that sponsoring (for ISP 2 users) starts at a smaller advertising level for zero-rating with ISP 1 than in the opposite situation. In terms of ISP revenue in Fig. 7, a zero-rating at ISP i induces for small advertising levels no revenue at the other ISP, which can be seen as against competition and could lead to action from a regulator, especially given that the optimal advertising levels are actually quite low from Figs. 5, 6, 7 and 8. Sponsoring tends to reduce the gap in revenue with respect to no sponsoring when zero-rating is applied to ISP 1 only, while it is the opposite when applied to ISP 2. Interestingly in Fig. 8, consumer surplus does not change much with the policy if zero-rating is applied to ISP 1, the largest ISP. It changes a bit more when it is applied to ISP 2, with large levels of advertising only. In both cases anyway, note that sponsoring is beneficial to users, but at the advertising level chosen optimally by the CP, there is no real difference (because the selected value is γ1 = 0 in case of zero-rating at ISP 2).

6 Vertical Integration We now look at the case of vertical integration: when the CP and one of the ISPs are the same firm. This situation becomes increasingly frequent in practice, with ISPs offering video-on-demand services, among others. In that case, the optimization of parameters (sponsoring and advertising) is based on the combined revenues of the CP and the ISP, by just adding them. Figure 9 (left) displays the revenue of the integrated CP–ISP in terms of s, in the cases where it is either ISP 1 or ISP 2 which is integrated, and with fixed sponsoring levels. ISP 1 being more established, it is not surprising that integrating it yields a larger revenue. But the optimal advertising level is larger in the case of ISP 2 with those parameters. Figure 9 (right) shows the revenue of the integrated entity (CP– ISP 1 or CP–ISP 2) with the corresponding optimal γ . Recall that in the case of non-integration, the optimal advertisement level was 1.35 for no or identical sponsoring, and 1.45 for differentiated sponsoring. We still have the dominance such that differentiated sponsoring is better than identical one, itself better than no sponsoring. It can be seen that for an integrated CP–ISP 1, identical sponsoring leads to the most ads (a level around 1.6 and 1.0—i.e., no ads—for no sponsoring and 1.4 for a differentiated one), while it is differentiated sponsoring in the case of an integrated CP–ISP 2 (a level around 1.65 and 1.05 both for no sponsoring and for an identical one). So, the optimal advertising level may be less or more for the same sponsoring policy if the CP is integrated or not. But if choosing the optimal policy, advertising is larger if vertical integration is applied. Figure 10 gives the consumer surplus in terms of s for the two integration cases and compares it with the optimal non-integrated strategy obtained in Sect. 4. It allows

P. Maillé and B. Tuffin

Revenue

0.8 Integrated CP-ISP 1 Integrated CP-ISP 2

0.6

Revenue CP-ISP

68

0.6

0.4

 1 = 2 = 0  1 = 2 Opt. (1 , 2 )

CP-ISP 1

CP-ISP 2

0.4 2 4 Advertisement level s

6

2 4 Advertisement level s

1 = 2 = 0 1 = 2 Opt. i No integ., opt. i

0.5 0.45 0.4

1 = 2 = 0 1 = 2 With opt. i No integ., opt. i

0.45 User Welfare

Consumer Surplus

Fig. 9 CP+ISP revenue in the two integration scenarios, when γ1 = 0.7, γ2 = 0.4 (left), and with optimized values of (γ1 , γ2 ) (right)

0.4

0.35

0.35 2

4 sA

2

4 sA

Fig. 10 Consumer surplus with integrated CP–ISP 1 (left) or integrated CP–ISP 2 (right), for β A = 1.00, β B = 1.50, r1 = 1.50, r2 = 1.00, p1 = 1.00, p2 = 0.60, q1 = 0.60, q2 = 0.40, α A = 1.00, α B = 1.00, s B = 5.00

to see if integration has a positive or negative impact on users, something of interest for the regulator. We observe that vertical integration is good for users, especially if sponsoring data is allowed. An identical sponsoring is the best option with this regard. The reason is that it induces much more data sponsoring resulting in lower user cost. Finally, Fig. 11 displays the revenue of the non-integrated ISP for all strategies, in order to illustrate whether integration brings down competition. We also again plot the revenues in cases of no integration for comparison sake. An integrated ISP 1 reduces drastically ISP 2 revenue, which was expected, but at a lesser extent for the identical sponsoring strategy because of less freedom. Integrating the incumbent ISP is typically against competition. Focusing on identical sponsoring, when it is ISP 2 who is integrated, ISP 1 can surprisingly benefit from the integration of the competitor. Note it is not the case for the other sponsoring strategies. Table 3 compares the outputs at the optimal advertising levels selected by the CP in the two integrated scenarios. They can be compared with the non-integrated

Analysis of Sponsored Data Practices … γ1 = γ2 = 0 γ1 = γ2 Opt. γ i No integ., γ i = 0 No integ., γ 1 = γ 2 No integ., opt. γ i

0.2 0.15

0.5

ISP 1 revenue

ISP 2 revenue

0.25

69

0.4 0.3 0.2

0.1

0.1

2 4 Advertisement level s

2 4 Advertisement level s

Fig. 11 Revenue of the non-integrated ISP if ISP 1 (left) or ISP 2 (right) is integrated Table 3 Output at optimal s when the CP is integrated with an ISP CP integrated with ISP 1 s∗

CP integrated with ISP 2

(γ1 , γ2 )

G + R1

R2

CS

s∗

(γ1 , γ2 )

G + R2

R1

No spons. 1.0

(0,0)

0.6445

0.1653

0.4600

1.05

(0,0)

0.4370

0.4062

0.4420

Id. spons. 1.6

(0.998, 0.998)

0.6614

0.1171

0.4833

1.05

(0,0)

0.4370

0.4062

0.4420

Dif. spons.

(0.783, 0.995)

0.6697

0.1925

0.4882

1.65

(0.243, 0.997)

0.4644

0.2021

0.3978

1.4

CS

case of Table 2. Vertical integration, except partially for the last line of Table 3, always significantly benefits to users and competitors, something not so intuitive. We see in Table 3 that vertical integration leads to more sponsoring than in Table 2, so that consumer surplus is increased. With an integrated CP–ISP 2, the increased satisfaction in the first two lines is due to the smaller optimal advertising level; for the last line, the advertising level is larger but only partially counterbalanced by the increased sponsoring.

7 Conclusions The purpose of this paper was to study the impact of sponsored data, differentiated or not, on all telecommunication actors, in the presence of competing ISPs. Our main results are the following: • Sponsoring can be beneficial to users and ISPs, and the impact depends on the chosen advertisement level. • Zero-rating at one ISP can lead to no revenue at the other ISP, which can be seen as harming competition. • Vertical integration can be beneficial for all actors in this context if the CP is integrated into the incumbent ISP (the one with the largest reputation).

70

P. Maillé and B. Tuffin

Conclusions of course depend on the arbitrarily chosen set of parameters, but the paper provides insight into the potential effects of sponsored data, and a model to perform such an analysis. We plan to extend the promising first results of this work in one main direction: We aim at inserting another round of decisions, on ISP prices (for subscription and for data usage). Our purpose here was to see the impact of having several ISPs on the CP decisions leading to important conclusions. Inserting a pricing game will complete the analysis but is computationally demanding. Other interesting extensions include considering a larger set of ISPs to show that the obtained results are without loss of generality in this respect, as well as considering CPs which can be substitutes.

References 1. M. Cho and M. Choi. Pricing for mobile data services considering service evolution and change of user heterogeneity. IEICE Transactions on Communications, E96-B(2):543–552, 2013. 2. C. Joe-Wong, S. Ha, and M. Chiang. Sponsoring mobile data: An economic analysis of the impact on users and content providers. In Proc. of INFOCOM, Hong-Kong, China, 2015. 3. P. Maillé, P. Reichl, and B. Tuffin. Internet governance and economics of network neutrality. In A. Hadjiantonis and B. Stiller, editors, Telecommunications Economics - Selected Results of the COST Action IS0605 EconTel, pages 108–116. Lecture Notes in Computer Science 7216, Springer Verlag, 2012. 4. P. Maillé, G. Simon, and B. Tuffin. Toward a net neutrality debate that conforms to the 2010s. IEEE Communications Magazine, 54(3):94–99, 2016. 5. P. Maillé and B. Tuffin. Users facing volume-based and flat-rate-based charging schemes at the same time. In 8th Latin American Network Operations and Management Symposium (LANOMS), pages 23–26, Joao Pessoa, Brazil, October 2015. 6. H. Pang, L. Gao, Q. Ding, and L. Sun. When data sponsoring meets edge caching: A gametheoretic analysis. CoRR, arXiv:abs/1709.00273, 2017. 7. R. Somogyi. The economics of zero-rating and net neutrality. CORE Discussion Papers 2016047, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE), 2016. 8. Z. Xiong, S. Feng, D. Niyato, P. Wang, and Y. Zhang. Competition and Cooperation Analysis for Data Sponsored Market: A Network Effects Model. ArXiv e-prints, November 2017. 9. L. Zhang, W. Wu, and D. Wang. Sponsored data plan: A two-class service model in wireless data networks. SIGMETRICS Perform. Eval. Rev., 43(1):85–96, June 2015.

Media Delivery Competition with Edge Cloud, Remote Cloud and Networking Xinyi Hu, George Kesidis, Behdad Heidarpour and Zbigniew Dziong

Abstract We describe a marketplace for content distribution, specifically stored-video streaming, involving both edge cloud (fog) and remote cloud computing and storage resources. Three different types of participants are considered: providers that are affiliated with the remote cloud, those that are affiliated with the ISP/edge, and those affiliated with neither. For a simple model, we explore the existence of a Nash equilibrium. Furthermore, we formulate a leader-follower game involving a market regulator maximizing a social welfare and study its Stackelberg equilibrium. For a market regulator seeking to limit prices charged by an edge-cloud entrant, we show an interesting trade-off between “moderate” edge-cloud prices and existence of follower (Nash) equilibrium.

1 Introduction A primary objective of network neutrality regulations is to address antitrust concerns, promote fair competition and innovation, and reduce costs for consumers [7, 12, 30]. Complicating the role of eyeball ISPs [21] with respect to the content it handles is the fact that many are themselves also content providers (over “managed services” they provide to their end-users) and content creators, and thus in competition with This research was supported by NSF CNS grant 1526133. X. Hu (B) · G. Kesidis Pennsylvania State University, University Park, PA, USA e-mail: [email protected] G. Kesidis e-mail: [email protected] B. Heidarpour · Z. Dziong École de technologie supérieure (ÉTS), Montreal, Canada e-mail: [email protected] Z. Dziong e-mail: [email protected] © Springer Nature Switzerland AG 2019 J. Walrand et al. (eds.), Network Games, Control, and Optimization, Static & Dynamic Game Theory: Foundations & Applications, https://doi.org/10.1007/978-3-030-10880-9_5

71

72

X. Hu et al.

some content providers (CPs) that they enable over their commodity Internet service. Though this is a growing trend, considering AT&T’s recent attempt to acquire TimeWarner, the growth of Google Fi in America, and Facebook’s Express WiFi e.g., in India, some content providers/producers, e.g., Netflix, are not (yet) themselves a subsidiary of a network provider. Since the onset of the neutrality debate,1 researchers have extensively studied parsimonious models of the Internet marketplace to gain insight into the economic forces in play, especially to study the effects on competition (including barriers to market entry) and costs-to-end-users of non-neutral actions by ISPs. Performance is often assessed based on the Bertrand-Nash equilibria (NE) of noncooperative, decentralized games, and in terms of dynamical convergence to these equilibria, often considering limited resources (particularly bandwidth) as in classical Cournot games [5, 11]. The role of the regulator can be considered using a Stackelberg (leaderfollowers) game framework (discussed further below). For example, games involving end-users and content providers on an ISP platform were studied in, e.g., [13, 24]. Network neutrality and side payments. Shapley values, indicating fair division of revenue within a coalition (or cooperative game), are used to argue for sidepayments between ISPs and CPs in, e.g., [21, 22] and the references therein. Pricing congestible commodities has been extensively studied [9, 29], e.g., in [15] a demand model is based on a “cost” that is the sum of a price and latency term. Extensive prior work has also used simple models to study the impact of: direct CP-to-ISP side payments for access or differential service to end-users, advertising revenue and/or direct subscription payments from end-users to CPs,2 network caching (e.g., [20, 27] and the references therein), and competition among ISPs and/or among CPs including CPs affiliated/integrated with ISPs. When network-neutrality regulations are/were in play, ISPs were prevented from demanding direct side payments from content providers for access to the ISP’s endusers/subscribers (origin neutrality). However, neutrality rules do not preclude asymmetric Service-Level Agreements (SLAs) at network-to-network interfaces (NNIs, or peering points) that are (neutrally) based on traffic aggregates. In this way, an “eyeball” ISP3 can effectively demand additional payment from the transit ISP of a large (e.g., video) content provider. The transit ISP will naturally pass on these costs to the CP or be squeezed out of business forcing the CP to directly engage with the ISP - either way, a side-payment from the CP to the eyeball ISP effectively ensues, e.g., [17, 20]. In previous related work, different scenarios for transit networks and content distribution networks (CDNs) [1, 23] were considered in [2], including those in which the CDN (or individual CP) is incentivized to compensate the transit network (ISP) to cache its content. 1 The

recent actions of the American FCC to revoke network neutrality regulations, which some states are litigating against [14], is just the recent salvo of the American debate. Europe generally remains strongly in favor of network neutrality. 2 ISPs have pruned ads from delivered content arguing that they were not requested by the end-users [16], while CPs argued that (otherwise free) content or services are monetized through embedded advertisements the receipt of which end-users implicitly request. 3 Here, ISPs that service end-users.

Media Delivery Competition with Edge Cloud, Remote Cloud and Networking

73

Edge-cloud services. Reference [18] considered a neutral cloud and raised issues of fair competition in cases where public cloud providers compete with their tenants, e.g., both Amazon Prime and Netflix use (Amazon’s) AWS.4 In this paper, we consider a future marketplace also involving edge-cloud facilities run by the ISPs. One expects that service offerings of the edge cloud will largely involve VNFs and their service chains/graphs for the ISP, its subscribers and their content and service providers, but there may be other application-layer and general-purpose computing services that may be implemented there. One can also envision fog-computing customers employing a combination of (relatively cheaper) public cloud-computing services and fog-computing services.5 For example, a commercial video streaming provider may want to establish caches in the fog. One obvious benefit is the reduction in egress networking costs of its (remote) public cloud operations. Unlike the mid-2000s, when the majority of Internet traffic was unencrypted and pirated, today the majority of Internet traffic is encrypted media.6 That is, media is encrypted for specific end-users. One could encrypt for the ISP/edge-cloud in the public cloud, and then use the edge cloud to encrypt for individual end-users (i.e., both edge caching and computing). Considering heightened costs, objects of only the highest popularity would be cached in the fog. That is, a content provider’s edge-caches may not have a fixed amount of capacity and operate under, e.g., the least-recently used (LRU) eviction policy (e.g., [10]); instead the popularity of each active object would be directly estimated (e.g., [8, 28]), and only the objects whose estimated popularity exceeds a threshold would be cached in the edge.7 A principal practical use of the following work is to help inform the actions of a market regulator. A regulator would base its decisions on a model of the complex system considered herein. One can introduce many parameters in an attempt to, e.g., model the dynamic traffic characteristics of individual sessions/users. The parameters would need to be fit to different real-world datasets provided by the different parties. The validity of one party’s model and dataset would be contested by opposing parties. Thus, we propose to use a “lumped” noncooperative game model of the complex system that could be agreed upon by the different parties, understood by the regulator, and used to determine how to regulate side payments (and yield other insights). We will see that, though the model is simple, it has interesting behavior. Such models are typically evaluated in terms of their Nash equilibria (a stalemate among the competitors) if one exists and how the dynamics converge to it. A market 4 Netflix

now uses AWS primarily for client login and content search/selection. To stream selected video, Netflix now employs its own CDN. 5 Note that CDNs like Akamai and Amazon CloudFront provide datacenter presence nearer to the network edge. 6 This transformation may be due to the ease through which media from legitimate sources can be purchased online, new and very popular content only available as streamed online, and the presence of trojans in pirated media. 7 Note that LRU approximates as the least popular object the current LRU one residing in the cache. Evicting the least popular object is an element of the optimal noncausal (offline) caching policy, e.g., [19].

74

X. Hu et al.

Fig. 1 Three types of content providers, where CP3 is affiliated with ISP/edge-cloud, CP2 is affiliated with the remote cloud, and CP1 is affiliated with neither

regulator may wish to “steer” a Nash equilibrium to maximize a social welfare objective and avoid volatile cases where no Nash equilibrium exists, cf., Sect. 7. In this paper, we consider three types of content providers, see Fig. 1: 1. Those not affiliated with either (remote) cloud or edge-cloud.8 2. Those affiliated with remote-cloud providers, (e.g., Amazon Prime). 3. Those affiliated with edge-cloud providers (ISP, e.g., AT&T U-verse). This paper is organized as follows. We consider a marketplace for a single popular data object, e.g., a newly released commercial movie (arguably priced per download as assumed in the following for video-on-demand/pay-per-view service, instead of a flat rate subscription covering multiple videos). We describe our model in Sect. 2 and make a preliminary observation regarding the utilities in Sect. 3. In Sects. 4 and 5, we explore existence and uniqueness of the noncooperative game among the CPs for two different parametric cases; related numerical results are given in Sect. 6. In Sect. 7, we describe a leader-follower game involving a market regulator managing a social welfare for the marketplace. We conclude with a summary in Sect. 8.

2 Problem Set-Up Consider the scenario of Fig. 1 for a single cell and a single “hot” media object with three CPs. Let: • • • • •

X be the total demand (number of subscribers) for the object. xi ≥ 1 be the share of this demand for CPi, X = i xi . ρi > 0 be the price charged to a subscriber by CPi. δi ∈ {0, 1} indicate whether CPi decides to cache the object in the edge cloud. ν ≥ 0 be the networking price of the media object (regarding the remote cloud).

8 Again,

Netflix has developed its own CDN and currently only uses public cloud facilities for its user interface. So, compared to this type of provider, it would have reduced remote cloud and networking costs, and may not need edge-cloud facilities depending on how extensive its CDN is, but would have significant operating and amortized capital expenditures associated with its CDN.

Media Delivery Competition with Edge Cloud, Remote Cloud and Networking

75

Fig. 2 CPi employs edge cloud or not

• σ ≥ 0 be the computing and storage price of the media object in the remote cloud. • κ ≥ 0 be the computing (related to encryption for authorized end-user) and storage price of the media object in the edge cloud. • σ˜ < σ , ν˜ < ν, κ˜ < κ represent the base operational expenditures associated with remote cloud, networking and edge cloud charges (both remote clouds in Fig. 1 have the same basic costs σ˜ ). • Ui be the net revenue of CPi. Assuming that ISP-affiliated CP3 always employs the edge cloud, we get the following utilities for the different CPs (Fig. 2): U1 = max{U1+ , U1− }, where

U1+ = x1 (ρ1 − κ) − (ν + σ ) and U1− = x1 (ρ1 − ν − σ );

U2 = max{U2+ , U2− }, where U2+ = x2 (ρ2 − κ) − (ν + σ˜ ) and U2− = x2 (ρ2 − (ν + σ˜ )); U3 = x3 ρ3 − x3 κ˜ − (˜ν + σ˜ ).

Assume fixed total demand, X = x1 + x2 + x3 . To determine demand from prices, we can use a simple competition model, xi ∝ ρi−α with parameter α > 0, i.e., xi = X

ρ1−α

ρi−α . + ρ2−α + ρ3−α

(1)

Thus, we can eliminate the x terms and instead express utilities Ui in terms of prices (CP/player control actions) ρ. Note that the competition results in lowest-pricetakes-all as α → ∞, and equal share xi ≈ X /3 when α ↓ 0. Lowest-price-takes-all does not capture consumer “inertia” when changing providers9 or other factors not 9 More

complex models of inertia involve hysteresis in dynamic response to competitive prices wherein demand as a function of price is larger when prices increase compared to when prices decrease.

76

X. Hu et al.

considered in herein (e.g., regional variations in service quality including reliability, better user interface) affecting provider preferences. Since under (1) all demand is divided among providers irrespective of prices, to prevent prices from growing very large (even for non-cooperative CPs, i.e., they do not form an oligopoly), a regulator can set certain elements of the game toward maximizing a social welfare trading off CP utilities/revenues and average price to consumers, cf., Sect. 7.

3 Initial Observation Regarding the Utilities For the following, define ρk,max so that prices are the range ρk ∈ (0, ρk,max ].

(2)

Note that limρk ↓0 xk = X . Also, in our competition model, the parameter α will be chosen larger, i.e., closer to lowest-price-takes-all than equal share (α = 0), so we assume α > 1 in the following. Proposition 1 If α > 1 then U1− is either unimodal or increasing as a function of ρ1 ∈ (0, ρ1,max ]. Remark: This proposition implies that there is a unique ρ1∗ > 0 such that U1− has a global maximum at min{ρ1∗ , ρ1,max }. In the following, we will typically take ρk,max finite but very large. Proof ∂U1− x2 = 1 f (ρ1 ), where ∂ρ1 X f (ρ1 ) = ρ1α−1 (ρ2−α + ρ3−α )((1 − α)ρ1 + α(ν + σ )) + 1. Note that f also depends on ρ2 , ρ3 > 0. Thus, the first order necessary condition (FONC) to maximize U1− , ∂U1− /∂ρ1 = 0, holds if and only if f (ρ1 ) = 0. Since f (ρ1 ) = (ρ2−α + ρ3−α )(α − 1)ρ1α−2 (ν + σ − ρ1 ), we find that f is unimodal with maximum at ν + σ , where f (0+) = 1 < f (ν + σ ). ∗ >ν+σ Also, limρ1 →∞ f (ρ1 ) = −∞. So by continuity of f , there is a unique ρ1− − ∗ ∗ such that f (ρ1− ) = 0. Thus, if ρ1− < ρ1,max then U1 is unimodal with maximum at ∗ , else U1− is increasing with maximum at ρ1,max . ρ1− A similar proposition holds for U1+ , U2± , U3 .

Media Delivery Competition with Edge Cloud, Remote Cloud and Networking

77

4 Case κ > ν + σ : NE Existence and Uniqueness The assumption κ > ν + σ (⇒ κ > ν + σ˜ ), is reasonable for a “mature” market wherein the costs associated with using edge clouds are larger than those for remote cloud. If κ > ν + σ , then U1− > U1+ for all x1 ≥ 0 (i.e., ρ1 ≥ 0); so, U1 = U1− is quasiconcave by the above proposition. Similarly if κ > ν + σ˜ then U2 = U2− is quasiconcave. If we define all the play-action sets as closed, [ε, ρk,max ] for small ε > 0 and ρk,max < ∞ (i.e., compact strategy sets), then all utilities Uk are continuous and quasiconcave in ρk when κ > ν + σ (recall σ ≥ σ˜ ), and so a Nash equilibrium of the three-player game exists. Proposition 2 If α > 1, κ > ν + σ , and the price ranges are such that ρ1 >

α α α (ν + σ ), ρ2 > (ν + σ˜ ), ρ3 > κ, ˜ α−1 α−1 α−1

then there is a unique Nash equilibrium. Proof We can directly show from the FONCs that all best-response prices βi of all CPs are “standard” functions, i.e., they are positive, monotonic in ρ −i (where, e.g., ρ −2 = (ρ1 , ρ3 )), and “scalable” (i.e., ∀μ > 1, μβi (ρ −i ) > βi (μρ −i )). Thus, the Nash equilibrium is unique, see e.g., [26]. For example, for α = 2 and sufficiently large ρ1,max , we find from FONCs that the best-response prices are:  β1 (ρ2 , ρ3 ) = (ν + σ ) +

(ν + σ )2 + 

β2 (ρ1 , ρ3 ) = (ν + σ˜ ) +

(ν + σ˜ )2 +

 β3 (ρ1 , ρ2 ) = κ˜ +

κ˜ 2 +

1 ρ2−2 + ρ3−2 1 ρ1−2

+ ρ3−2

1 ρ1−2

+ ρ2−2

∗ i.e., β1 = ρ1− . Recall that the utilities of different market competitors of an economic model are typically assessed at its NE when it exists.

78

X. Hu et al.

5 Case κ < ν + σ˜ : NE Existence Assume now that κ < ν + σ˜ ⇒ κ < ν + σ. The motivation for this assumption is that when the fog/edge-cloud-affiliated CP3 is a marketplace entrant, it could artificially (initially) reduce its associated costs to grow its customer base. Since κ < ν + σ , we can define positive ε=

κ < 1. ν+σ

So, if the total demand X > then we can define

 R1 =

1 ν+σ = ν+σ −κ 1−ε X (1 − ε) − 1 ρ2−α + ρ3−α

(3)

1/α > 0,

where U1− = U1+ when ρ1 = R1 . Thus, if κ < ν + σ and (3) then U1 = U1+ δ1 + U1− (1 − δ1 ) with δ1 := 1{ρ1 ≤ R1 } = {U1 = U1+ } ∈ {0, 1}. A similar statement holds for U2 with δ2 := 1{ρ2 ≤ R2 } = 1{U2 = U2+ }. If (3), R1 < ρ1,max , and ρk ≤ ρk,max for k = 2, 3, then there are valid prices ρ2 , ρ3 ∗ where U1 may be bimodal, e.g., when R1 is between ρ1− and (similarly defined) ± ∗ ρ1+ , the local maximizers of U1 , see the example of Fig. 3. Again, there are similar statements for U2 for κ < ν + σ˜ . For κ < ν + σ˜ , there are existence results for (pure) Nash equilibria for nonquasiconcave utilities; see e.g. [6, 25] for conditions that can be interpreted in terms of our model parameters. However, we have found cases where the best-response functions for CP1 (i.e., ρ1 maximizing U1 as a function of ρ2 , ρ3 ) or CP2 are discontinuous and the Nash equilibria fail to exist. Such cases involve continual price oscillations (limit cycles) which a market regulator may try to avoid.

Media Delivery Competition with Edge Cloud, Remote Cloud and Networking

79

6 Numerical Study 6.1 Selected Parameters for the Following Numerical Study For realistic model parameters, we considered current pricing used by AWS. Network data transfer (network) costs are about $.10/GB [3]. So to transfer a 6 GigaByte (GB) video would cost $.60 = ν. Additionally, we assume video needs to be encrypted to particular requesting end-users. The “duration” cost of AWS Lambda service is $1.667 × 10−5 /(GB · s) [4]. So, transmitting the 6GB video at say 5 Megabits/s requires 9600s and costs $0.96 = σ . We take base operating costs to be 75% of corresponding prices (e.g., σ˜ = 0.75σ ): σ = .96, ν = .60, σ˜ = .72, ν˜ = .45.

(4)

6.2 Two-Player Game Between CP1, CP3 with κ < ν + σ For example, if we take edge-cloud parameters to be 25% more than those of the remote cloud (κ = 1.25σ ), κ = 1.20, κ˜ = 0.90,

(5)

and take total demand X = 20, then (3) is satisfied.10 Again, see Fig. 3. Smaller κ − κ˜ may be assumed if the edge-cloud is lowering its prices κ to attract customers, possibly even operating at a loss. Fig. 3 U1 is bimodal when ρ2 = 2, ρ3 = 4.5, α = 2 and all other parameter are given in Sects. 6.1 and 6.2 below: The red vertical line (ρ1 = 3.4750) indicates the inflection point where U1+ = U1− ; the blue vertical line (ρ1 = 3.3864) indicating the maximum of U1+ , and green (ρ1 = 3.9629) indicating the maximum of U1− are on opposite sides of the red line

10 Numerical

results are qualitatively the same for much larger values of X , e.g., X = 106 .

80

X. Hu et al.

Fig. 4 Best responses of CP1 and CP3: a unique NE exist when ρ2 = 2.8, no NE when ρ2 = 2.55

With ρ2 fixed, let β1 (ρ3 ) and β3 (ρ1 ) be the best response for CP1 and CP3 respectively, and (ρ1∗ , ρ3∗ ) be the Nash equilibrium. We illustrate two cases where ρ2 = 2.80 in case (a) and ρ2 = 2.55 in case (b) of Figs. 4, 5, 6 and 7. Figure 4 shows the best response curves β1 (ρ3 ) and β3 (ρ1 ). There exists a unique Nash equilibrium (ρ1∗ , ρ3∗ ) which is located at the intersection point of two curves in case (a), while no NE exists in (b) where β1 (ρ3 ) is discontinuous due to δ1 . When ρ2 continues to decrease from 2.55, e.g., ρ2 = 2.40, β3 (ρ1 ) will intersect the right part of β1 (ρ3 ), e.g., a NE exists when ρ2 = 2.40. According to the two cases in Fig. 4, we suppose CP1 and CP3 engage in a discrete step iterated game in which each player is assumed to estimate the opponent’s strategy and then play the best response to it. Players’ strategies can be updated at (discrete) time t as ρ1t+1 = β1 (ρ3t ), ρ3t+1 = β3 (ρ1t ) In one typical set of experiments, initial prices ρ1 = 2 and ρ3 = 2, and the total number of iterations is 50. Experimental results show the system converges to the NE over time when a unique NE exists in case (a) of Figs. 5, 6 and 7. However, oscillatory behavior (i.e., δ1 oscillates between 0 and 1 in Fig. 5b) occurs for the case without a NE. The prices and utilities of two players show oscillations in Figs. 6b and 7b as well. We also observed that ρ1 is bigger than ρ3 in Fig. 6, but this is not the case for the corresponding utilities in Fig. 7. Here, CP1 needs to charge higher prices than CP3 owing to higher side-payments κ < ν + σ and lower market share. Again note that our competition model is not lowest-price-take-all. Also, note that the NE is affected by side payments ν, σ , κ, quantities that could be set by the regulator to optimize the social welfare (and ensure that a NE exists).

Media Delivery Competition with Edge Cloud, Remote Cloud and Networking

Fig. 5 δ1 is converged in a and oscillated in b over time

Fig. 6 Players’ strategies (prices ρi set) are converged in a and oscillated in b over time

Fig. 7 Players’ utilities Ui converge in a and oscillated in b over time

81

82

X. Hu et al.

6.3 The NE of Three Players with κ < ν + σ Numerical results above show NE (ρ1∗ , ρ3∗ ) does not exist for every ρ2 . When ρ2 varies in the range [0.05,6], the line (ρ1∗ , ρ3∗ )(ρ2 ) is discontinuous. In the following, we present a typical set of numerical results to illustrate when a unique NE ρˆ = (ρˆ1 , ρˆ2 , ρˆ3 ) of the players exists. Here, we also illustrate two cases: when κ = 1.20 in Fig. 8 and κ = 1.11 in Fig. 9. For both cases, κ˜ = 0.9 and the remaining parameters are (4) as above. In these two figures, the red line is (ρ1∗ , ρ3∗ )(ρ2 ), and the surface is the best response β2 (ρ1 , ρ3 ). Note that a unique ρˆ exists if the line intersects the surface, otherwise there is no global NE. Numerical results show that though the line (ρ1∗ , ρ3∗ )(ρ2 ) is discontinuous, it may have a unique intersection point ρˆ with the surface β2 in Fig. 8. That is, there is unique interior NE for κ = 1.20. In Fig. 9, the surface β2 has a jump discontinuity (caused by δ2 ) where the line (ρ1∗ , ρ3∗ )(ρ2 ) passes through. Therefore, no NE exist for three players when κ = 1.11. In other parameter settings, it possible that the discontinuity of the line (ρ1∗ , ρ3∗ )(ρ2 ) occurs at the surface β2 (ρ1 , ρ3 ), in which case there is no NE as well. For the two cases above, we also did the 3-player discrete-time iterated game with initial ρ1 = 2, ρ2 = 2 and ρ3 = 2. Similar oscillatory or convergence behavior was observed when the NE does not exist or uniquely exists, respectively.

Fig. 8 There exists a unique interior NE (ρˆ1 , ρˆ2 , ρˆ3 ) with a discontinuity in (ρ1∗ , ρ3∗ )(ρ2 ), when κ = 1.2, κ˜ = 0.9, ν = 0.6, ν˜ = 0.45, σ = 0.96, σ˜ = 0.72

Media Delivery Competition with Edge Cloud, Remote Cloud and Networking

83

Fig. 9 There exists no NE caused by the discontinuity of β2 (ρ1 , ρ3 ) when κ = 1.11, κ˜ = 0.9, ν = 0.6, ν˜ = 0.45, σ = 0.96, σ˜ = 0.72

7 Leader-Follower Game Consider a Stackelberg game with the regulator (the leader) and three CPs (followers). To optimize a social welfare objective, the regulator may act to determine the side payments from CPi to remote/edge cloud φ = (φν , φσ , φκ ), where ˜ φν = ν − ν˜ , φσ = σ − σ˜ , and φκ = κ − κ. Iteratively, we assume that the leader will act and the followers will respond and reach NE (if it exists), i.e., Subgame Perfect NE (SPNE). Both leader and followers are informed by each others’ actions. Again, suppose the edge cloud is an emerging industry entering the market. An effective side-payment of the edge cloud may need to be regulated to preserve fair competition between all the content providers. In the following, we consider the case that the regulator determines φκ only and the other two side payments are fixed as (4).

7.1 Preliminaries on 3-Player Follower Game and φκ Again, it is possible that the edge-cloud marketplace entrant may operate at a loss initially to build a customer base. So, we consider an extended range for φκ to see how this side-payment affects the CPs’ (followers’) NE of the Stackelberg game: φκ ∈ [−0.10, 0.50], equivalently κ ∈ [0.80, 1.40], with step-size is 0.01. Figure 10 shows the CPs’ NE as a function of φκ . We find there is no NE when φκ ∈ [0.19, 0.21] (κ ∈ [1.09, 1.11]), which is consistent with Sect. 6.3.

84

X. Hu et al.

Fig. 10 CPs’ NE as a function of φκ , where the indicated vertical line is φκ = ν + σ˜ − κ˜ i.e. κ = ν + σ˜

When φκ ∈ [−0.1, 0.18], δ1 = δ2 = 1, i.e., all CPs deploy the edge cloud, see Fig. 10a. In this range, as φκ increases, CP1 and CP2 will naturally pass on these higher side-payments to their subscribers, i.e., ρˆ1 and ρˆ2 are higher as indicated in Fig. 10b. Also, their subscriber market share xˆ 1 /X and xˆ 2 /X becomes smaller in Fig. 10d. When φκ ∈ [0.22, 0.4], δ1 = 1 and δ2 = 0, i.e., only CP1 pays the side-payment to the edge cloud. As φκ increases, CP1 continues to increase its price to compensate for the higher side-payment, and its subscriber market share keeps decreasing. When φκ ≥ 0.41, δ1 = δ2 = 0, i.e., no one pays the side-payment to the edge cloud and the system does not change as φκ increases further.

7.2 Leader’s Socioeconomic Cost For this leader-followers game, the socioeconomic cost function (negative social welfare) can be modeled as

Media Delivery Competition with Edge Cloud, Remote Cloud and Networking

85

Fig. 11 Ω(φκ ) with different ξ

Ω(φκ ) =

3  i=1

ρˆi (φκ )

xˆ i − ξ [Uˆ 1 (φκ ) + γ2 Uˆ 2 (φκ ) + γ3 Uˆ 3 (φκ )], X

where • ρ(φ ˆ κ ) = (ρˆ1 (φκ ), ρˆ2 (φκ ), ρˆ3 (φκ )) are the NE prices of the CPs as a function of φκ , ˆ κ )) is the NE net utility of CPi, i.e., SPNE, • Uˆ i (φκ ) = Ui (ρ(φ • ξ weights CPs’ net revenues against the average subscribers’ payments, and • γi indicates a regulator’s preference among the three CPs.11 We took γ2 = γ3 = 0.5 to benefit CP1, the weakest CP who has to pay both the remote cloud and the edge cloud. Figure 11 shows Ω(φκ ) with ξ ∈ [0.01, 0.15]. To deal with the cases that do not have a NE, we take the average of oscillated values in one cycle (red dots in Fig. 11). The leader should determine an effective side-payment to minimize Ω and also guarantee a unique NE exists. 11 Note

that if γ2 = 1 = γ3 , the side payments among all CPs cancel out in the expression for Ω.

86

X. Hu et al.

7.3 Numerical Results and Their Interpretation When ξ is small, to benefit subscribers, the leader will choose (the lowest) φκ = −0.1 in Fig. 11a, b. When ξ = 0.095 (a critical value where the the regulator balances the subscribers and the CPs), minimal Ω(−0.1) = Ω(φκ ) for φκ ≥ 0.41, see Fig. 11c. The leader may choose a side-payment from these two minima to benefit subscribers or companies. Alternatively, the regulator could choose an intermediate value of φκ = 0.22 where Ω is close to its minimum, but this operating point is also near the range in which a NE does not exist. This is an interesting trade-off for the regulator. For larger ξ , e.g., ξ = 0.11, to benefit the CPs, the leader will choose φκ ≥ 0.41, see Fig. 11d. In this case, if the leader encourages CPs to use the edge cloud, it will choose φκ = 0.18 (all CPs use the edge cloud) or φκ = 0.22 (CP1 and CP3 use the edge cloud).

8 Summary We described a marketplace for content distribution involving both fog/edge and remote cloud computing and storage resources. The existence of a Nash equilibrium was explored for a model involving one popular media object sold on-demand (as in pay-per-view video) and three market participants: content providers that are affiliated with the remote cloud, those that are affiliated with the ISP (edge), and those affiliated with neither. Finally, we formulated a social cost balancing consumer and provider interests and used it to formulate a leader-follower game involving a market regulator. The Stackelberg equilibrium of this game was studied to maximize social welfare (minimize social cost) evaluated at Nash equilibrium when it exists. In particular, we numerically evaluated a critical value for the parameter trading off subscriber and provider revenues at which the regulator can select the edge cloudcosts to favor the subscribers, providers, or an interesting compromise where the last is near an undesirable parametric region where Nash equilibria do not exist.

References 1. M. Adler, R.K. Sitaraman, and H. Venkataraman. Algorithms for optimizing the bandwidth cost of content delivery. Computer Networks, Dec. 2011. 2. P. Agyapong and M. Sirbu. Economic incentives in content-centric networking: Implications for protocol design and public policy. In Proc. 39th Telecommunications Policy Research Conference, Arlington, VA, 2011. 3. Amazon CloudFront Pricing. https://aws.amazon.com/cloudfront/pricing/?p=ps. 4. AWS Lambda Pricing. https://aws.amazon.com/lambda/pricing. 5. T. Basar and G. J. Olsder. Dynamic noncooperative game theory, 2nd Ed. Academic Press, 1995.

Media Delivery Competition with Edge Cloud, Remote Cloud and Networking

87

6. M.R. Baye, G. Tian, and J. Zhou. Characterizations of the existence of equilibria in games with discontinuous and non-quasiconcave payoffs. Review of Economic Studies, 60(4):935–948, 1993. 7. T. Berners-Lee. Net Neutrality: This is serious. TimBL’s blog, http://dig.csail.mit.edu/ breadcrumbs/node/144, June 2006. 8. L.C. Drazek. Intensity estimation for Poisson processes. MS Thesis, School of Mathematics, University of Leeds, https://www1.maths.leeds.ac.uk/~voss/projects/2012-Poisson/Drazek. pdf, Sept. 2013. 9. N. Economides and B. Hermalin. The strategic use of download limits by a monopoly platform. Working Paper 13-26, www.NETinst.org, Dec. 2013. 10. R. Fagin. Asymptotic approximation of the move-to-front search cost distribution and leastrecently-used caching fault probabilities, 1977. 11. D. Fudenberg and J. Tirole. Game Theory. MIT Press, 1991. 12. R. Hahn and S. Wallsten. The economics of net neutrality. Economists’ Voice, The Berkeley Economic Press, 3(6):1–7, 2006. 13. P. Hande, M. Chiang, R. Calderbank, and S. Rangan. Network pricing and rate allocation with content provider participation. In Proc. IEEE INFOCOM, 2009. 14. C. Herreria. 22 States Sue FCC For Axing Net Neutrality. https://www.huffingtonpost.com/ entry/attorneys-general-sue-fcc-net-neutrality_us_5a5e84c3e4b00a7f171b815a, Jan. 16, 2018. 15. R. Johari, G.Y. Weintraub, and B. Van Roy. Investment and market structure in industries with congestion. Operations Research, 58(5), 2010. 16. D. Kerr. France orders Internet provider to stop blocking Google ads. http://news.cnet.com/ 8301-1023_3-57562638-93/france-orders-internet-provider-to-stop-blocking-google-ads, Jan. 7, 2013. 17. G. Kesidis. A simple two-sided market model with side-payments and ISP service classes. In Proc. IEEE INFOCOM Workshop on Smart Data Pricing, Toronto, May 2014. 18. G. Kesidis, N. Nasiriani, B. Urgaonkar, and C. Wang. Neutrality in Future Public Clouds: Implications and Challenges. In Proc. USENIX HotCloud, 2016. 19. D.E. Knuth. An analysis of optimal caching. J. Algorithms, 6:181–199, 1985. 20. F. Kocak, G. Kesidis, and S. Fdida. Network neutrality with content caching and its effect on access pricing. In Smart Data Pricing. S. Sen and M. Chiang (Eds.), Wiley, 2013. 21. R.T.B. Ma, D.-M. Chiu, J.C.S. Lui, V. Misra, and D. Rubenstein. Interconnecting eyeballs to content: A Shapley value perspective on ISP peering and settlement. In Proc. Int’l Workshop on Economics of Networked Systems, pages 61–66, 2008. 22. R.T.B. Ma, D.-M. Chiu, J.C.S. Lui, V. Misra, and D. Rubenstein. On cooperative settlement between content, transit and eyeball Internet service providers. In Proc. ACM CoNEXT, 2008. 23. B. Maggs. Presentation on CDNs (Akamai) at RESCOM, Porquerolles, France: http://blog. lrem.net/2013/05/23/rescom-2013-bruce-maggs/, May 2013. 24. J. Musacchio, G. Schwartz, and J. Walrand. A two-sided market analysis of provider investment incentives with an application to the net-neutrality issue. Review of Network Economics, 8(1), 2009. 25. P.J. Reny. Nash equilibrium in discontinuous games. Econ. Theory, 61:553–569, 2016. 26. C.U. Saraydar, N.B. Mandayam, and D.J. Goodman. Efficient power control via pricing in wireless data networks. IEEE Transactions on Communications, 50(2):291–303, Feb. 2002. 27. D. Sohn. Neutrality and Caching. https://cdt.org/blog/neutrality-and-caching/, Dec. 16, 2008. 28. G.S. Watson. Estimating the intensity of a Poisson process, May 1976. 29. D. Weller and B. Woodcock. Bandwidth bottleneck: The hardware at the heart of the Internet is not fast enough. IEEE Spectrum, Jan. 2013. 30. T. Wu. Network neutrality, broadband discrimination. Journal of Telecommunications and High Technology Law, 2:141, 2003.

An Algorithmic Framework for Geo-Distributed Analytics Srikanth Kandula, Ishai Menache, Joseph (Seffi) Naor and Erez Timnat

Abstract Large-scale cloud enterprises operate tens to hundreds of datacenters, running a variety of services that produce enormous amounts of data, such as search clicks and infrastructure operation logs. A recent research direction in both academia and industry is to attempt to process the “big data” in multiple datacenters, as the alternative of centralized processing might be too slow and costly (e.g., due to transferring all the data to a single location). Running such geo-distributed analytics jobs at scale gives rise to key resource management decisions: Where should each of the computations take place? Accordingly, which data should be moved to which location, and when? Which network paths should be used for moving the data, etc. These decisions are complicated not only because they involve the scheduling of multiple types of resources (e.g., compute and network), but also due to the complicated internal data flow of the jobs—typically structured as a DAG of tens of stages, each of which with up to thousands of tasks. Recent work [17, 22, 25] has dealt with the resource management problem by abstracting away certain aspects of the problem, such as the physical network connecting the datacenters, the DAG structure of the jobs, and/or the compute capacity constraints at the (possibly heterogeneous) datacenters. In this paper, we provide the first analytical model that includes all aspects of the problem, with the objective of minimizing the makespan of multiple geodistributed jobs. We provide exact and approximate algorithms for certain practical scenarios and suggest principled heuristics for other scenarios of interest.

S. Kandula · I. Menache (B) Microsoft Research, Redmond, WA, USA e-mail: [email protected] S. Kandula e-mail: [email protected] J. (Seffi) Naor Technion – Israel Institute of Technology, Haifa, Israel e-mail: [email protected] E. Timnat Google, Tel Aviv, Israel e-mail: [email protected] © Springer Nature Switzerland AG 2019 J. Walrand et al. (eds.), Network Games, Control, and Optimization, Static & Dynamic Game Theory: Foundations & Applications, https://doi.org/10.1007/978-3-030-10880-9_6

89

90

S. Kandula et al.

1 Introduction Many enterprises have data and computation clusters spread across the world. Due to efficiency or privacy considerations, the data may only be available in distributed locations. For example, datacenter server logs [1, 19, 23] and surveillance videos [2, 18] are massive datasets that are accessed infrequently. It is efficient to store such datasets in-place, that is at or close to where the data is generated. As another example, due to privacy considerations, the EU and China now require user and enterprise data to be stored within their borders. Our goal in this paper is to consider algorithmic questions that arise in executing data-parallel queries [10, 24, 26] on such geographically distributed datasets. We identify three key aspects of this problem. First, distributed sites where data is generated have limited amount of compute and storage capacity (e.g., a Starbucks store recording video). Hence, it may not be efficient to run all of the computation at the site having the data. Further, capacities can vary across sites by several orders of magnitude; an organization may have rented hundreds of VMs at Azure or EC2 locations and have thousands or more servers at various on-premise locations. Second, when large amounts of data have to be analyzed, data-parallel queries can exhaust the network bandwidth available to-and-from the various sites. Hence, task scheduling has to more carefully account for network usage. However, the topology of the physical network that interconnects the various sites is complex: (a) when data is present at off-net locations such as Starbucks stores or airports, the network paths cross many ISPs leading to capacity bottlenecks within the network; and (b) even when data is distributed among on-net sites, such as datacenters connected with a private wide-area network [15, 20], there can be bottlenecks within the network. Furthermore, the round trip delays in the wide-area network can be several orders of magnitude larger than those within a cluster; the delays and the available capacity on the network paths can also vary over time. Third, the queries that need to be supported are not necessarily a single-map stage followed by a single reduce stage. Rather, decision support queries tend to be quite complex. For example, the benchmarks in TPC-H [3] and TPC-DS [4] lead to directed acyclic graphs (DAGs) containing many stages, in various popular frameworks such as Hive [24], Spark-SQL [8], or SCOPE [10]. Furthermore, most production systems have a cadence that offers a predictable set of recurring queries. Such queries can, for example, digest raw logs and video streams into structured datasets so as to speed up the processing of subsequent user queries. We are unaware of any prior work that takes into account these three aspects. Many systems ignore all three issues. For example, Iridium [22] supports mapreduce queries (i.e., DAG of depth 2), assumes that the only network bottlenecks are at the in- and out-links of sites (“congestion-free core”), and does not account for limits on compute capacity. Many other works such as SWAG [17], Tetrium [16], Carbyne [14] make identical assumptions. Geode [25] supports more general queries, but only considers minimizing the total amount of data crossing sites; a measure that may or may not translate to fast query execution.

An Algorithmic Framework for Geo-Distributed Analytics

91

The reason behind these rather inadequate solutions is that the underlying algorithmic problem—how to schedule a DAG of stages (each consisting of many tasks) across a network of sites—is difficult. Scheduling a dependent set of tasks on a pool of processors is itself challenging [13] and has received attention in the scheduling literature, e.g., [9, 12, 21] and references therein. In addition, the problem at hand has to also route the network traffic resulting from task placement. That is, the demands on the network depend on task placement. Furthermore, the way that the network routes these demands affects the finish time of the tasks. Many prior works model less general versions of the problem described above [14, 16, 17, 22, 25] and motivate their heuristics by arguing that the general problem is impractical to solve.

1.1 Our Results The primary contribution of our paper is a rigorous algorithmic model for studying the geo-distributed analytics scheduling problem, which captures the aspects highlighted above: (1) a general directed acyclic graph (DAG) of dependencies between stages, (2) compute capacity and other limits per site, and (3) a general topology for the physical network. In our model, we are given a DAG representing the jobs and their inter-stage dependencies. Our goal is to minimize the time to complete all of the jobs, i.e., the makespan. We consider different types of dependencies between stages (of a job), both of which arise in practice: soft precedence and strict precedence. With strict precedences, each stage has to wait for all stages preceding it in the dependency DAG to complete their work entirely, before it can start its processing. With soft precedences, dependent stages can overlap in execution, though the advancement of a stage is bounded by the advancement of the stages on which it depends. We also model different dependency types between any pair of stages u and v: all-to-all, where all tasks in u communicate with all tasks in v, and scatter-gather, a much sparser and targeted communication pattern (see Sect. 2.2). We choose these precedences and dependency types because they effectively capture practice (see Sect. 2). We make several important modeling decisions allowing us to formulate the problem in a tractable way. In particular, we operate at a stage granularity, which makes the problem-size orders of magnitude smaller and allows us to bypass excess rounding of variables. After defining the general model (Sect. 2), we derive exact and approximate algorithms for several scenarios of interest. In particular, we present an optimal solution for soft precedence with scatter-gather dependency (Sect. 3). This solution is achieved using a linear programming formulation. Appealingly, the linear program for soft precedence directly dictates a feasible schedule. For the strictprecedence case, we formulate a different linear program that unfortunately does not directly imply a schedule. However, it does indicate where to execute every stage, but not when to do that. Thus, we apply an algorithm on top of the LP solution to construct a practical schedule. We show an ω-approximation for strict precedence with all-to-all dependency, under a bus network topology, where ω is the width of the logical DAG (Sect. 4). We note that this result implies, as an important special

92

S. Kandula et al.

case, an optimal algorithm for a single geo-distributed map-reduce [11] job. For the remaining practical scenarios, we have designed principled heuristics (Sect. 5)—allto-all dependencies with both soft- and strict precedences, under a general network topology. Notably, we show how to address the multiplicative constraints that arise when modeling all-to-all dependencies by heuristically linearizing them using Taylor expansion. The linear programs are solved iteratively, using values from previous iterations as the points around which we expand the Taylor series. Our simulations show that this heuristic is nearly optimal, offering a practical solution for the remaining scenarios, although we do not have formal guarantees for it.

2 The Model In this section, we present the geo-distributed analytics model. In Sect. 2.1, we describe the basic properties and assumptions of the model. In Sects. 2.2 and 2.3, we define the dependency types and the precedence models across stages, respectively. Finally, in Sect. 2.4, we describe the different network topologies that we consider.

2.1 Preliminaries We are given a set of jobs. Each job is modeled as a DAG whose nodes correspond to stages; a stage consists of tasks that perform similar computation on different subsets of the data in parallel. An edge (u, v) in the DAG indicates that stage v depends on stage u, in the sense that it needs some data from u to complete its own computation. We will often use the terminology “logical” edge for (u, v), to distinguish such edges from the “physical” links that connect datacenters. Stages and tasks. Each stage v contains n v tasks. We assume that n v is large which lets us use the fractional output of a linear program (LP) and incur small error. For example, a fraction 0.1 of a stage translates to 10% of its tasks; we convert this to an integer and handle the (small) rounding errors with heuristics. In an examined data-parallel cluster at Microsoft consisting of tens of thousands of servers that ran SCOPE [10] jobs, the median stage had 50 tasks and 25% of the stages had over 250 tasks. Data flow. We assume that each task reads (writes) some fraction of the input (output) of the stage. We make a simplifying assumption that these fractions are equal for tasks within a stage. For every stage v, we denote the ratio between the size of the output and the size of the input by sv (its selectivity). Most stages output less data than their input; however, about 10% generate up to 10× more output [6]. Also given is cv , the progress rate of a stage per core, per unit time. We assume that the progress rate is linearly proportional to the number of cores that the stage is given, up to a limit.

An Algorithmic Framework for Geo-Distributed Analytics

93

For every edge e = (u, v), we denote by De the total amount of data to be transferred between the stages u and v. For every stage v, we denote by DIN,v the total amount of input  data for that stage; its total  output data DOUT,v equals sv × DIN,v . Further, DIN,v = uv∈E Duv , and DOUT,u = uv∈E Duv . Compute datacenters. We assume that there are n datacenters numbered 1, 2, 3..., n. For datacenter (DC) i, we denote the compute capacity by Ci , i.e., the number of cores. The DCs are connected via a physical network for which we will consider different possible topologies and link capacities in Sect. 2.4. Since the number of machines in a DC is large, we ignore potential machine fragmentation issues. Dataset locations. For every source stage v ∈ V , its input data can be distributed across different DCs. The amount of input data for the stage v at DC i is denoted by Ii,v . For every sink node v ∈ V , it may be required that its output data be distributed across DCs. The amount of output data of node v that should reside in DC i is denoted by Oi,v and is part of the input to the problem.

2.2 Stage Dependency Types Consider two stages u and v connected via a logical edge e = (u, v). Each stage is composed of many tasks. The modeling question is which tasks of stage v receive input data from which tasks of stage u. We define two types of dependencies: all-to-all and scatter-gather. The same DAG can have both types of dependencies. All-to-all dependency. A logical edge e = (u, v) means that every task of stage u sends data to every task of stage v (see left part of Fig. 1 for an illustration). Moreover, we assume that the same amount of data is sent between each pair of tasks (i.e., De /n u n v per pair of tasks). This dependency typically arises during a shuffle; for example, in map-reduce, each task in the reduce stage is responsible for a partition and receives data corresponding to keys in that partition from every map task. In the examined production cluster, roughly 50% of the edges are all-to-all.

Fig. 1 We illustrate here two DAG jobs. The left job has four stages with an all-to-all dependency type between u 1 and v1 . The right job is a chain of three stages with a scatter-gather dependency type between stages u 2 and v2

94

S. Kandula et al.

Scatter-gather dependency. A logical edge e = (u, v) means that each task in stage u sends input to exactly one task in stage v or vice versa (see right part of Fig. 1). The former happens when n u ≤ n v , and the latter happens when n u > n v . Note that this generalizes the one-to-one dependency. This dependency occurs when aggregations or joins are performed over partitioned data. In the examined production cluster, 50% of the edges are scatter-gather; about 16% are one-to-one.

2.3 Precedence Models Another important modeling aspect relates to the precedence between tasks in stages connected by an edge. We offer two models: soft precedence and strict precedence. Soft precedence. Here, we assume that each task can make fractional progress (say 1%) as long as every input-generating parent of that task has made at least an equivalent amount (1%) of progress. Such dependent tasks can execute simultaneously. Strict precedence. Here, we assume that a task can make no progress until all of its input-generating parent tasks in the DAG have finished completely. We consider the above two models since they are extreme points of the design space. Note that a solution under the strict-precedence model is also valid under the soft-precedence model but the converse case does not hold. A typical setting in Hadoop launches reduce tasks after 80% of the map tasks have finished. In general, data-parallel frameworks allow overlapping to pipeline network transfers with task execution; however, overlapping adds to cost since both tasks simultaneously hold resources, such as memory.

2.4 Network Topologies In this work, we consider two types of network topologies. General network topology. The most general topology can be modeled as follows. The n DCs are connected via a physical network of m nodes, where m ≥ n. Nodes numbered n + 1 . . . m are relay nodes. Edges in the physical network (k, ) have a corresponding maximum data transfer rate denoted by Bk, . We assume that intra-DC transfer rate is unlimited; thus, Bk,k = ∞. Bus (or star) topology. Here, all of the DCs are connected to a bus (or to one hypothetical relay node). The bus has unbounded capacity and the bottlenecks are only in uploading and downloading data from a DC to the bus. For DC i, we denote the maximum upload and download data transfer rates by u i and di respectively. With the advent of full-bisection datacenter backplanes [5, 7], this topology matches the network within a DC; the only bottlenecks are at the servers or in and out of racks

An Algorithmic Framework for Geo-Distributed Analytics

95

of servers while the core is congestion-free. On the wide-area network, it remains a useful simplification; for example, an actual physical topology can be approximated by choosing appropriate values for {u i , di }.

3 Algorithms for Soft-Precedence Constraints In this section, we construct an LP representing the soft-precedence model. Then, we obtain an optimal solution for soft precedence with scatter-gather stage dependency. Finally, we add multiplicative constraints to incorporate all-to-all stage dependency. In Sect. 5, we show a heuristic approach for dealing with these constraints. Given a value of time T as input, the LP examines whether the problem can complete within time T . The optimal value of T can be found, e.g., via binary search. The number of variables in the LP depends on T . To have data flow from input locations to the DCs where the first computation stages execute, we pad the graph with dummy stages for the input and for the output; we omit the specific details due to lack of space.

3.1 Computation and Logical Flow Constraints We define variables xi,u,t to denote the number of cores given to stage u on DC i, during time frame [t, t + 1).Since DC i has Ci cores, we have the following constraint: ∀i ∈ [n], t ∈ [T ] : u∈V xi,u,t ≤ Ci . Note that the tasks in a stage may be constrained if needed by the maximum number of cores that they can use: ∀i ∈ [n], t ∈ [T ] : xi,u,t ≤ Cu . We define variables ri, j,e,t to represent the rate of data transfer from DC i to DC j, on edge e = (u, v) during time frame [t, t + 1). Note that this definition refers to the logical traffic demand on the network due to a logical edge between stages; the data can be transferred using any routes on the physical network. We define variables IN j,v,t to be the total amount of input data for stage v that reached DC j by time t. This amount is equal to the sum of all input data for v that was transferred into DC j by time t, including all logical edges entering v, which are denoted by I n(v). Data transfers should also include data transfers from the DC j to itself—if stage u finished on DC j, it can transfer its data to itself for processing stage v on the same datacenter j. These data transfers are unlimited in rate, B j, j = ∞, but they should still be accounted for. As for the data that originates at j, it is denoted by I j,v . So the total input data IN j,v,t is given by: ∀ j ∈ [n], v ∈ V, t ∈ [T ] : IN j,v,t =

 i∈[n],e∈I n(v),t  ≤t

ri, j,e,t  .

96

S. Kandula et al.

We define variables COMPi,v,t to be the total amount of output data for stage v that was computed in DC i by time t. This value is obtained by summing over all cores in i given to stage v over time: ∀i ∈ [n], v ∈ V, t ∈ [T ] : COMPi,v,t =



xi,v,t  × cv .

t  ≤t

We define variables OUTi,v,t to be the total amount of output data for stage v that was transferred away from DC i to the required destinations by time t, including all logical edges outgoing from v, which are denoted by Out (v). This constraint is: ∀i ∈ [n], v ∈ V, t ∈ [T ] : OUTi,v,t =



ri, j,e,t  .

j∈[n],e∈Out (v),t  ≤t

As mentioned earlier, for every stage v, DC i, time t—the amount of data computed and output COMP cannot exceed the amount of input data that has become available IN. For instance, if only half of the input of a task has arrived by that time, then no more than half of the data could have been computed. More generally, the amount of data computed and output by v is bounded by the amount of input data that is available by time t times the selectivity of that stage (ratio of output to input), which is sv . The constraint is therefore ∀i ∈ [n], v ∈ V, t ∈ [T ] : COMPi,v,t ≤ INi,v,t × sv . Similarly, the amount of output data sent from the stage is bounded by the amount of data computed, yielding the constraint ∀i ∈ [n], v ∈ V, t ∈ [T ] : OUTi,v,t ≤ COMPi,v,t . We also demand that for every logical edge e = (u, v), all necessary data be transferred. That is, the total amount  of data transfers over time will be equal to De , yielding the constraint: ∀e ∈ E : i, j∈[n],t∈[T ] ri, j,e,t = De .

3.2 Physical Flow Constraints for General Topology We define variables f i, j,e,k,l,t to represent the physical rate of data transfer from physical node k to physical node  during time frame [t, t + 1), to fulfill the demand of the logical flow ri, j,e,t . That is, the physical flow is a way to transfer the logical traffic demands on the physical network, given network capacity constraints. Obviously, the two plans have to match. Every logical demand ri, j,e,t should equal the sum of the physical flows going out of i that implement it and to the sum of physical flows going into j, leading to the following constraints: ∀i, j ∈ [n], e ∈ E, t ∈ [T ] : ri, j,e,t =

 l∈[m]

f i, j,e,i,l,t =



f i, j,e,k, j,t .

k∈[m]

Additionally, corresponding to a logical flow from i to j, there should be no physical flow leaving j or entering i. The corresponding constraints are:

An Algorithmic Framework for Geo-Distributed Analytics

97

∀i, j ∈ [n], k ∈ [m], e ∈ E, t ∈ [T ] : f i, j,e,k,i,t = f i, j,e, j,k,t = 0. The last constraints do not apply for i = j = k, in which case the only physical flow is from the DC to itself which we do allow. For any physical node k, other than i, j, flow conservation dictates that the incoming flow to node k is equal to the outgoing flow from it. This is true for every high-level flow ri, j,e,t separately. This leads us to the following constraints: ∀i, j ∈ [n], k ∈ [m]/{i, j}, e ∈ E, t ∈ [T ] :

 ∈[m]

f i, j,e,k,,t =



f i, j,e,,k,t .

∈[m]

Data transfer is also required to meet the maximum data transfer rate for every physical link Bk, . This requirement implies the following constraint: ∀k,  ∈ [m], t ∈ [T ] :



f i, j,e,k,,t ≤ Bk, .

i, j∈[n],e∈E

3.3 Optimal Solution for Scatter-Gather Dependency We now show that a solution to the LP implies an optimal schedule. Theorem 1 Consider the LP described in Sects. 3.1–3.2 with the objective to minimize the total time. A solution to this LP is a near-optimal solution for the scattergather dependency model. Proof We observe that any solution to the problem also defines a feasible solution to the LP, and thus, the value achieved by the LP is a lower bound on the optimal value. Conversely, a solution to the LP can be translated into a solution for the problem; the only concern with this solution is the rounding errors; the impact of rounding errors can be minimized through a simple heuristic (we omit the details for brevity), and it is typically small since the number of tasks is very large. Hence, this solution is nearly optimal.

3.4 All-to-All Dependency We now show the multiplicative constraints needed for all-to-all stage dependency. We later present a heuristic approach for solving the program with these non-convex constraints. Consider a logical edge e = (u, v). Say half of stage u is computed on DC i and the second half on j. Additionally, assume half of stage v is computed on DC k and half on . In this case, we need exactly 25% of the data to flow from each DC in {i, j} to a DC in {k, l}. In general, the flow from i, u to k, v is proportional to the product OU Ti,u,T · I N j,v,T . Naively, the constraint should be:

98

S. Kandula et al.

∀i, j ∈ [n], e = (u, v) ∈ E :



ri, j,e,t = De

t≤T

OUTi,u,T IN j,v,T · . DOUT,u DIN,v

is a constant, but the product OUTi,u,T · IN j,v,T contains two Note that D De D OUT,u IN,v variables and is thus non-convex. We use a first-order Taylor expansion to approximate this product. While this approach falls short of guaranteeing performance bounds, it obtains good results in practice. See Sect. 5 for details and a simulation study. All-to-all dependency leads to one more complication: The progress of a stage may be limited by its slowest parent. Consider a logical edge e = (u, v). Assume that stage u was scheduled half each on DC i, and DC j; it has completed 50% of its work at i but only 30% of its work at j. Then, stage v can complete no more than 30% of its work at any DC. This leads us to the following constraint: 

ri, j,e,t  COMP j,v,t ≥ .  r COMP  i, j,e,t j,v,T t ≤T 

t ≤t ∀i, j ∈ [n], e = (u, v) ∈ E, t ∈ [T ] : 

That is, the fraction of the data sent is an upper bound for the fraction of the data computed of v, so that if only 30% was sent by time t, no more than 30% of v will be  computed by time t. Multiplying both sides of the inequality by COMP j,v,T × t  ≤T ri, j,e,t  , we obtain yet another multiplication of variables. This product can also be approximated similarly with first-order Taylor expansion; see Sec. 5 for details.

4 Strict Precedence We now proceed to study the case of strict-precedence constraints, which requires a different LP formulation. The solution to the new LP does not induce a schedule as in previous section, rather we need to construct a feasible schedule (satisfying strict precedences) from this solution. We present an approximation algorithm for all-to-all stage dependency, under a bus network topology. We conclude this section by highlighting the additional constraints needed for general network topology. These constraints contain multiplicative constraints that require a heuristic approach for solving them, which we describe in Sec. 5. We now present the LP for strict-precedence constraints. In this LP, the total makespan T is a variable, and the objective function is to minimize T .

4.1 Physical Flow Constraints for Bus Topology For simplicity of exposition, we present the physical flow constraints for a bus topology. The constraints required for a general network topology are highlighted in

An Algorithmic Framework for Geo-Distributed Analytics

99

Sect. 4.4. We define variables di, j,e for e = (u, v) to denote the total amount of data which we transfer from stage u on datacenter i to stage v on DC j. We define variables ti, j,e to denote the total amount of time it takes us to transfer that data. We define variables ri, j,e,c to denote the total amount of time in which we dedicate exactly c Mbps for the data transfer  di, j,e . The total amount of data transferred is: ∀i = j ∈ [n], e ∈ E : di, j,e = c∈[min{u i ,d j }] c ∗ ri, j,e,c . will be the sum of times at different transfer As for the total transfer time ti, j,e —it speeds: ∀i = j ∈ [n], e ∈ E : ti, j,e = c∈[min{u i ,d j }] ri, j,e,c . We denote the time it takes to complete every logical edge e by te . We know that the time to complete every logical edge is lower bounded by the time it takes to transfer its data for any pair of DCs (i, j): ∀i, j ∈ [n], e ∈ E : te ≥ ti, j,e . If edge e takes time te , then the total amount of data sent within that time frame  from DC i cannot exceed u i te . This leads us to the constraint: ∀i ∈ [n], e ∈ E : j=i,c∈[min{u i ,d j }] c ∗ ri, j,e,c ≤ u i te . Similarly, the total amount of data sent to DC j within te time cannot exceed d j te .  This leads us to the constraint: ∀ j ∈ [n], e ∈ E : i= j,c∈[min{u i ,d j }] cri, j,e,c ≤ d j te .

4.2 Computation and Logical Flow Constraints We define variables xi,u,c to denote the amount of time for which we dedicate exactly c cores of DC i to the computation of stage u. We define variables di,u to denote the amount of output data of stage u to be found on DC i. We further denote by cu the output size of stage u, per core per unit time. The total amount of output data for stage u on DC i, di,u , is the sum of matching computations: ∀i ∈ [n], u ∈ V : di,u =  c∈[Ci ] ccu x i,u,c . This amount is equal to the total amount of outgoing data, plus the amount of data there  to be used as output. The corresponding constraint is: ∀i ∈ [n], u ∈ V : di,u = j∈[n],e∈Out (u) di, j,e + Oi,u . We used Out (u) to denote the outgoing edges of u. Similarly, the amount of data there, over the output to input ratio su , is equal to the amount of incoming data, plus the amount of data that was part of the job input there. The corresponding constraint is: ∀ j ∈ [n], v ∈ V : d j,v /sv = i∈[n],e∈I n(v) di, j,e + I j,v . We used I n(v) to denote the incoming edges of v. We define variables ti,u to denote the time it takes to compute stage u on DC i, and takes to compute the stage u over all DCs. It variables tu to denote the total time it  follows that: ∀i ∈ [n], u ∈ V : ti,u = c∈[Ci ] xi,u,c , and ∀i ∈ [n], u ∈ V : ti,u ≤ tu . With these constraints, tu is the maximum time between all values ti,u . We also require that all computations are fully completed. Assume stage u requires  a total of D OU T,u /cu cores×time, then: ∀u ∈ V : i∈[n] di,u = Du /cu . Similarly, we require that all data is indeed  transferred. Assume edge e requires De data to be transferred, then ∀e ∈ E : i, j∈[n] di, j,e = De . For every logical chain, i.e., sequence of edges C ∈ G—we demand that the total computation and  time of the chain does not exceed T . This leads us to: data transfer s.t. ∀C ∈ G : u∈C tu + e∈C te ≤ T .

100

S. Kandula et al.

Note that for a general DAG, the number of chains can be exponential. However, these constraints can be rewritten in a way that their number becomes polynomial. In a nutshell, we can do so by introducing new variables ST A RTu , E N Du for the start and end times of every stage u; we omit the details for brevity. We also demand that the total cores×hours  on every DC would not exceed its computation limits, that is:  ∀i ∈ [n] : u∈V c∈[Ci ] c ∗ xi,u,c ≤ T Ci . Additionally, we demand that if Ii,u input data  arrives at DC i for stage u, it will all be transferred out of it: ∀i ∈ [n], u ∈ V : j∈[n],e∈Out (u) di, j,e = Ii,u su , where we used Out (u) to denote the outgoing edges of u. Similarly,  if O j,v output data is demanded for stage v on DC j, then: ∀ j ∈ [n], v ∈ V : i∈[n],e∈I n(v) di, j,e = O j,v , where we used I n(v) to denote the incoming edges of v.

4.3 The Algorithm We now show how to turn the solution of the LP into an actual feasible schedule. We emphasize that the algorithm holds for general topology, although we can guarantee an approximation ratio only for bus topology, as we show later. We say a stage u is available if all incoming data transfers for stage u have already completed. We say a logical edge e = (u, v) is available if stage u has already completed. We use the LP variables di,u to dictate the amount of data of stage u to be calculated on DC i. At every point in time, we denote by kout,i the number of logical edges currently available to be transferred from DC i, and by kin, j the number of logical edges currently available to be transferred into DC j. The algorithm is as follows: 1. While some stage has not completed: 2. For every DC i, denote by k the number of different stages currently available, having di,u > 0. Start running each of these stages on Cki of the cores of DC i. 3. For every pair of DCs i, j, logical edge e = (u, v), denote di, j,e = di,u · d j,v . u i  di, j,e . 4. For every pair of DCs i, j, logical edge e, denote cout,i, j,e = kout,i  d  j

d

i, j ,e

d

5. For every pair of DCs i, j, logical edge e, denote cin,i, j,e = kin,j j   i,dj,e . i i , j,e 6. Start using min(cout,i, j,e , cin,i, j,e ) Mbps for sending data from i to j associated with the logical edge e. 7. Continue running until some stage or data transfer completes, and then go back to step 1. It follows directly from the definition of the algorithm that the solution produced is feasible; we omit a formal proof due to lack of space. Analysis for bus topology. We now analyze the approximation factor of the algorithm with all-to-all stage dependency, under a bus network topology. For a general DAG, we define the width ω as the maximum between the maximum number of stages that can run in parallel (i.e., independent of each other) and the maximum number of edges that can run in parallel. We will show that our algorithm is within ω

An Algorithmic Framework for Geo-Distributed Analytics

101

times of optimal. For the special case of a single chain DAG, ω = 1; this includes the case of all map-reduce job DAGs; hence, our algorithm is optimal for map-reduce jobs on a bus network—the predominant case considered by prior work (e.g., [22]). Theorem 2 The algorithm is an ω-approximation for the strict-precedence model, with all-to-all stage dependencies, under a bus topology. Proof Since every feasible solution to the problem is also a feasible solution for the LP, we know that T is a lower bound on the optimal execution time. We show that ωT is an upper bound for the execution time of our solution, and from this obtain the desired approximation factor. In a general DAG of width ω, we might have up to ω stages executing in parallel, in the worst case. This means that ω stages are competing over the same DC i. Thus, each will get at least Cωi cores and will complete within at most ωti,u time. Therefore, every stage u will complete within at most ωtu .  We know from the LP that ∀i ∈ [n], e ∈ E : j=i,c∈[min{u i ,d j }] c ∗ ri, j,e,c ≤ u i te ,  and that ∀ j ∈ [n], e ∈ E : i= j,c∈[min{u i ,d j }] c ∗ ri, j,e,c ≤ d j te . For every logical edge e, we assume at least one of these constraints is tight, otherwise we can lower the value of te . Assume w.l.o.g. that this constraint is for an upload link, and some DC i. The total data that needs to be transferred from DC i regarding e is: u i te . We know there are at most ω logical edges that run in parallel with e, and thus kout,i ≤ ω. ui ≥ uωi to the edge e, we know it will complete within at Since we give a total of kout,i most ωte time.   tu + e∈C te ≤ T . The total execuWe know from the LP that ∀C ∈ G : u∈C tion time for every chain C ∈ G will not exceed u∈C ωtu + e∈C ωte ≤ ωT . Since every chain completes within ωT time, the entire DAG finishes in ωT time. Hence, for a general DAG, we guarantee an approximation factor of ω.

4.4 Physical Flow Constraints for a General Network Topology For a general network topology, we need to apply different physical constraints. In an all-to-all scenario, these constraints also include multiplicative constraints. In a bus topology, we were able to avoid this issue, but in a general topology, the flow has different paths, and we need the LP to know exactly how much flow goes between every pair of DCs i, j. In the full version of the paper, we define the full set of constraints. These constraints, together with the previous ones, lead to an LP formulation. This LP is then solved iteratively as described in Sect. 5. The LP solution allows us to use the algorithm presented in Sect. 4.3 as a heuristic solution to the general network model.

102

S. Kandula et al.

(a) Convergence of values of two variables.

(b) Makespan with(out) mult. constraints.

1 OUT1,1,T OUT1,2,T

Makespan

Value

0.8

0.6

0.4

0.2

0

0

5

10

15

20

20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0

Without multiplicative constraints With multiplicative constraints

0

1

Iteration

2

3

4

5

6

7

8

9

10

Bandwidth for (i, j), where i > j.

Fig. 2 Performance of the iterative approximation heuristic

5 Heuristics In this section, we first describe how we deal with the non-convex constraints which we obtain for the all-to-all dependency model. We then perform basic simulations to demonstrate that “linearizing" these constraints works well in practice. Linearizing multiplicative constraints. Recall that our multiplicative constraints  IN i,u,T · D j,v,T . are of the form: ∀i, j ∈ [n], e = (u, v) ∈ E : t≤T ri, j,e,t = De OUT DOUT,u IN,v The Taylor series is expanded around estimated values for OUTi,u,T and IN j,v,T ,  j,v,T respectively. We solve the LP iteratively and use the  i,u,T and IN denoted OUT values that the LP found for the variables in the previous iteration as the estimated values for the next iteration. The stopping condition for this procedure is when the difference in consecutive values is smaller than a configurable parameter (or when exceeding a maximal number of iterations). This iterative procedure is described in detail in Appendix 6. We next show empirically that the estimations converge quickly, and further that the corresponding values lead to adequate performance. Simulations. Our goal is twofold: (i) Demonstrate that the iterative approach for approximating the multiplication constraints indeed converges quickly to a feasible solution. (ii) Show that the obtained solution is “good.” In particular, albeit using approximations, we would like to demonstrate convergence to near-optimal values. Setting. In our simulations, we use a chain DAG, consisting of ten stages with different initial data distribution and computation requirements. We assume soft precedence between stages, and all-to-all dependency. The network topology is a clique, i.e., there is a link between every two datacenters. In each run, we vary the bandwidths of the links, as we elaborate below. Convergence. Figure 2a shows the convergence of the variables OUTi,u,T for i = 1 and u = 1, 2. As can be seen, we obtain rather stable values after 15 iterations. We

An Algorithmic Framework for Geo-Distributed Analytics

103

note that we obtain similar convergence behavior for other variables, and also for other DAGs that we have tested. The significance of convergence is that our scheme stabilizes, and we can rely on the obtained values for the multiplication of two variables. This does not imply that we converge to “good” values of the variables. We next address the quality of the approximation. Quality of the approximation. We check the obtained makespan with and without the multiplicative constraints. Any feasible solution to the actual problem must satisfy all the constraints, including the multiplicative constraints. Accordingly, the optimal solution that satisfies the constraints excluding the multiplicative ones is obviously a lower bound for the optimum. We next show empirically that the value of the execution time with the multiplicative constraints is indeed very close to the value without them, thus close to the optimal value. We again use a chain DAG with ten stages. The physical network is a clique. For each link (i, j), we assign a bandwidth of one if i < j, and for each run, choose a different value out of the set {1, 2, 3, 4, 5, 10} for the remaining links (i.e., (i, j) s.t. i > j). Figure 2b shows that the execution time without the multiplicative constraints is always 12 time units. Adding the multiplicative constraints increased the execution time to 13–14 units in all of the runs (i.e., an 8–17% increase in the makespan). This indicates that the potential loss due to the approximation of multiplications is not substantial. Note that the makespan of our algorithm is non-necessarily monotone in the bandwidth assigned to the i > j links—indicating that our approximation is suboptimal. However, what matters most here is that the obtained makespan is close to optimal.

6

Appendix: Linearizing Multiplicative Constraints

Recall that our multiplicative constraints are of the form ∀i, j ∈ [n], e = (u, v) ∈ E :

 t≤T

ri, j,e,t = De

OU Ti,u,T I N j,v,T · D OU T,u D I N ,v

The Taylor series is expanded around the estimated values  OU T i,u,T , I N j,v,T . We solve the LP iteratively and use the values the LP found for the variables in the previous iteration as the estimated values. After several iterations, the values converge to their true value, and thus, the multiplications become more accurate. Recall that the first-order Taylor expansion for the multiplication x · y expanded around the point (x, ˆ yˆ ) is: x · y xˆ · yˆ + (x − x) ˆ · yˆ + (y − yˆ ) · x. ˆ Dividing both De and approximating the multiplication using firstsides by the constant D OU T,u D I N ,v order Taylor, we obtain ∀i, j ∈ [n], e = (u, v) ∈ E:

104

S. Kandula et al.

D OU T,u D I N ,v  ri, j,e,t ≥  OU T i,u,T · I N j,v,T De t≤T + (OU Ti,u,T −  OU T i,u,T ) · I N j,v,T + (I N j,v,T − I N j,v,T ) ·  OU T i,u,T . Note that we have turned this equality constraint into a non-equality, since we have other constraints for the total flow from previous sections. Our multiplication evaluating the flow might be slightly more or less than the true multiplication value. In case it is more than the true value, we simply send less flow, and no constraints are violated. In case it is less than the true value, we need to send more data than planned—which might violate capacity constraints. In this case, we simply use a little more time for the entire flow to be sent. As long as the approximation is reasonable, this extra-time will be small. N j,v,T , we solve the LP iterTo obtain relatively accurate values for  OU T i,u,T , I atively and use the previous values as our approximation. For the first iteration only we use  OU T i,u,T = 0, I N j,v,T = 0. We use the same approach for our second setof multiplicative constraints (see Sect. 3.4) ∀i, j ∈ [n], e = (u, v) ∈ E, t ∈ r   C O M P j,v,t [T ] :  t ≤t ri, j,e,t  ≥ C O M P j,v,T ; details omitted for brevity. The resulting LP is then t ≤T i, j,e,t solved iteratively, as described, to obtain a nearly feasible solution. The small infeasibility translates into some extra-time required for completing flows that are larger than anticipated by the LP.

References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.

13.

Hadoop YARN Project. http://bit.ly/1iS8xvP. Seattle department of transportation live traffic videos. http://web6.seattle.gov/travelers/. TPC-H Benchmark. http://bit.ly/1KRK5gl. TPC-DS Benchmark. http://bit.ly/1J6uDap, 2012. A. Greenberg, N. Jain, S. Kandula, C. Kim, P. Lahiri, D. A. Maltz, P. Patel, and S. Sengupta. VL2: A Scalable and Flexible Data Center Network. In SIGCOMM, 2009. Sameer Agarwal, Srikanth Kandula, Nico Burno, Ming-Chuan Wu, Ion Stoica, and Jingren Zhou. Re-optimizing data parallel computing. In NSDI, 2012. Mohammad Al-Fares, Alexander Loukissas, and Amin Vahdat. A scalable, commodity data center network architecture. In SIGCOMM, 2008. Michael Armbrust et al. Spark sql: Relational data processing in spark. In SIGMOD, 2015. Peter Bodík, Ishai Menache, Joseph Seffi Naor, and Jonathan Yaniv. Brief announcement: deadline-aware scheduling of big-data processing jobs. In SPAA, pages 211–213, 2014. Ronnie Chaiken et al. SCOPE: Easy and Efficient Parallel Processing of Massive Datasets. In VLDB, 2008. Jeffrey Dean and Sanjay Ghemawat. Mapreduce: simplified data processing on large clusters. In OSDI, 2004. Pierre-François Dutot, Grégory Mounié, and Denis Trystram. Scheduling parallel tasks approximation algorithms. In Handbook of Scheduling - Algorithms, Models, and Performance Analysis. 2004. Ronald L. Graham. Bounds on multiprocessing timing anomalies. SIAM Journal on Applied Mathematics, 1969.

An Algorithmic Framework for Geo-Distributed Analytics

105

14. Robert Grandl, Mosharaf Chowdhury, Aditya Akella, and Ganesh Ananthanarayanan. Altruistic scheduling in multi-resource clusters. In OSDI, 2016. 15. Chi-Yao Hong, Srikanth Kandula, Ratul Mahajan, Ming Zhang, Vijay Gill, Mohan Nanduri, and Roger Wattenhofer. Achieving high utilization with software-driven wan. In SIGCOMM, 2013. 16. Chien-Chun Hung, Ganesh Ananthanarayanan, Leana Golubchik, Minlan Yu, and Mingyang Zhang. Wide-area analytics with multiple resources. In EuroSys, 2018. 17. Chien-Chun Hung, Leana Golubchik, and Minlan Yu. Scheduling jobs across geo-distributed datacenters. In SOCC, 2015. 18. IDC. Network video surveillance: Addressing storage challenges. http://bit.ly/1OGOtzA, 2012. 19. Michael Isard. Autopilot: Automatic Data Center Management. OSR, 41(2), 2007. 20. Sushant Jain, Alok Kumar, Subhasree Mandal, Joon Ong, Leon Poutievski, Arjun Singh, Subbaiah Venkata, Jim Wanderer, Junlan Zhou, Min Zhu, et al. B4: Experience with a globallydeployed software defined wan. In SIGCOMM, 2013. 21. Klaus Jansen and Hu Zhang. Scheduling malleable tasks with precedence constraints. J. Comput. Syst. Sci., 78(1):245–259, 2012. 22. Qifan Pu, Ganesh Ananthanarayanan, Peter Bodik, Srikanth Kandula, Aditya Akella, Paramvir Bahl, and Ion Stoica. Low latency geo-distributed analytics. In SIGCOMM, 2015. 23. Malte Schwarzkopf, Andy Konwinski, Michael Abd-El-Malek, and John Wilkes. Omega: Flexible, scalable schedulers for large compute clusters. In EuroSys, 2013. 24. Ashish Thusoo et al. Hive- a warehousing solution over a map-reduce framework. In VLDB, 2009. 25. Ashish Vulimiri, Carlo Curino, P. Brighten Godfrey, Thomas Jungblut, Jitu Padhye, and George Varghese. Global analytics in the face of bandwidth and regulatory constraints. In NSDI, 2015. 26. M. Zaharia et al. Spark: Cluster computing with working sets. Technical Report UCB/EECS2010-53, EECS Department, University of California, Berkeley, 2010.

The Stackelberg Equilibria of the Kelly Mechanism Francesco De Pellegrini, Antonio Massaro and Tamer Ba¸sar

Abstract The Kelly mechanism dictates that players share a resource proportionally to their bids. The corresponding game is known to have a unique Nash equilibrium. A related question arises, which is the nature of the behavior of the players for different prices imposed by the resource owner, who may be viewed as the leader in a Stackelberg game where the other players are followers. In this work, we describe the dynamics of the Nash equilibrium as a function of the price. Toward that goal, we characterize analytical properties of the Nash equilibrium by means of the implicit function theorem. With regard to the revenue generated by the resource owner, we provide a counterexample which shows that the Stackelberg equilibrium of the Kelly mechanism may not be unique. We obtain sufficient conditions which guarantee the set of Stackelberg equilibria to be finite and unique in the symmetric case. Finally, we describe the dependency between the resource’s signal and the maximum revenue that the resource owner can generate. Keywords Kelly mechanism · Nash equilibrium · Stackelberg equilibrium

1 Introduction The Kelly mechanism provides a method to share a resource among N players. The resource can be split in a continuous manner, and the resource owner charges the players in proportion to their bids z n , n = 1 . . . , N , assigning to the nth player the F. De Pellegrini (B) CERI/LIA, University of Avignon, chemin des Meinajaries 339, 84911 Avignon, France e-mail: [email protected] F. De Pellegrini · A. Massaro Fondazione Bruno Kessler, via Sommarive, 18, 38123 Trento, Italy e-mail: [email protected] T. Ba¸sar University of Illinois at Urbana Champaign, Urbana, IL 61801, USA e-mail: [email protected] © Springer Nature Switzerland AG 2019 J. Walrand et al. (eds.), Network Games, Control, and Optimization, Static & Dynamic Game Theory: Foundations & Applications, https://doi.org/10.1007/978-3-030-10880-9_7

107

108

F. De Pellegrini et al.

 fraction z n /( z n + δ). Here, the quantity δ ≥ 0, i.e., the resource’s signal [6], can be viewed as a reservation price at the resource owner side. Under the linear bidding model [6], the mechanism maps to a competitive N players game, where the nth player has strategy z n ∈ [0, Z n ]. N [0, Z n ] → R is taken to be The nth players’ objective function Cn : n=1  Cn (z n , z−n ) = Vn

N



zn

m=1 z m



+ λ zn

(1)

where the utility Vn : [0, 1] →R is a convex decreasing C 2 function. When convenient, we shall write z −n = m=n z m since the remaining players’ multistrategy z−n appears in aggregated form in (1). In this formulation of the game, each player n minimizes Cn using strategy z n , under perfect information on the other players’ strategy. We are interested in the revenue that can be generated by the Kelly mechanism in the corresponding equilibria of the system. In order to do so, we first have to provide a full analytical characterization of the Nash equilibrium, i.e., its differentiability and continuity, as a function of the price λ. Hence, we will have to study the total revenue of the resource owner and its maxima in order to determine the set of the Stackelberg equilibria where the resource owner can maximize her revenue. In the process of characterizing the Stackelberg equilibria, we also describe the structure of the Nash equilibria of the Kelly mechanism as a function of the price. The paper is organized as follows. In the next section, we introduce known results on uniqueness of the Nash equilibrium for the Kelly mechanism when the players’ strategy sets are bounded. Then, we proceed to the characterization of the properties of the Nash equilibrium as a function of the price in Sect. 3, proving that it is a.e. C 1 , except at most 2N points. In Sect. 4, we describe the set of the Stackelberg equilibria for δ = 0, where we show that, in this case, the prices corresponding to the Stackelberg equilibria form a half line. In Sect. 5, we show through a counterexample that also for the case δ > 0 the prices set where the maximal revenue is attained may not be a singleton. In Sect. 6, we provide sufficient conditions under which such a set is finite and a singleton in the symmetric case. In Sect. 7, we discuss the role of the resource’s signal: In the case of unbounded strategy set, the maximal revenue attained by the content provider is actually invariant with the resource’s signal. Finally, in Sect. 8, we provide a numerical example illustrating the results of the paper. A concluding section summarizes the results provided in this work.

2 Nash Equilibrium Since Vn is monotone and convex, Cn (z n , z−n ) in (1) is strictly convex given z −n [6]. Hence, the best reply z n∗ is unique and determined according to the following result:

The Stackelberg Equilibria of the Kelly Mechanism

109

Lemma 1 Under strategy profile z−n : i. z n∗ = 0 if and only if V˙n (0, z −n ) > −λ ii. If z n∗ > 0, then z n∗ = min{z n , Z n }, where V˙n (z n , z −n ) = −λ. In the original formulation [5, 7], the Kelly mechanism sets δ = 0. In this case, the null strategy vector z∗ = 0 is not a Nash equilibrium [5]. Overall, the cases of z∗ = 0 and the saturated equilibrium z∗ = Z, where Z := (Z 1 , . . . , Z N ) are covered by: Proposition 1 i. z∗ = 0 is the unique Nash equilibrium if and only if δ > 0 and V˙n (0, 0) ≥ −λδ for n = 1, . . . , N  ii. z∗ = Z is the unique Nash equilibrium if and only if V˙n (Z n , v=n Z v + δ) ≤   2 Zv −λ  v Z v for each player n. v =n

From a continuity argument, it is immediate to observe that there exists a value of λ above which z∗ = 0; further, there exists a value of λ below which it holds z∗ = Z. It follows from Rosen [2] that the game admits a Nash equilibrium, in view of the fact that the Kelly game is such that: i. the multistrategy set is a convex compact subset of R N and ii. for each n, Vn (z n , z −n ) is convex conditioned on the remaining players’ strategies. With respect to the uniqueness, z∗ = 0 and z∗ = Z are always unique from Proposition 1. Once we excluded those trivial cases, uniqueness can be derived by extending the argument in [6] to the case of bounded strategy sets as showed in [4]. Theorem 1 The Kelly game has a Nash equilibrium, and it is unique. We recall that uniqueness holds for general increasing convex costs λn (z n ). In this work, we focus on the case of linear uniform costs.1 The proof of Theorem 1 is actually  constructive and provides insight into the relationship between the demand p := z n + δ and the player n allocation as a function of p. The simultaneous condition on the best responses of all players at the Nash equilibrium ⎧ ⎪ ⎨ ≥ 0 if z n = 0 (2) ∃ z ∈ R+N : ∂zn Cn (z n , z−n ) = 0 if 0 < z n < Z n ⎪ ⎩ ≤ 0 if z n = Z n is equivalent to the condition

∃ (x, p) ∈ [0, 1] N × R+ :

1 In

⎧ ˙ ⎪ ⎨Vn (xn )(1 − xn ) + λ p = 0, ∀n N Z

δ n ⎪ =1− max 0, min xi , ⎩ p p i=1

the case of linear costs a direct proof from generalized diagonal convexity is given in [1].

(3)

110

F. De Pellegrini et al.

N where the mapping from n=1 [0, Z n ] to [0, 1] N × R+ is given by xn =  zzmn +δ  and p = z m + δ; the reverse mapping is z n = xn · p. Finally, the equivalence guarantees that there exists a unique (x1∗ , ..., x N∗ , p ∗ ) such that the Nash condition holds where z n∗ = xn∗ · p ∗ .

3 Smoothness of the Nash Equilibrium In this section, we establish the differentiability properties of the Nash equilibrium as a function of λ. For the sake of notation, in the rest of the paper, we define Fn (y) := −V˙n (y)(1 − y), which is positive decreasing for y ∈ (0, 1). In what follows, we shall make use of the implicit function theorem. First, we need to describe the set of critical prices with respect to (3). Price λ is critical for player n if at the corresponding Nash equilibrium the best response for player n is 0 or Z n and equality Fn (xn (λ)) = λ p(λ) holds. Hence, the set of critical prices can be given as D := ∪Dn , where    Zn  , and Fn (xn (λ)) = λ p(λ) Dn := λ  xn (λ) = 0 or xn (λ) = p(λ) We shall next prove that, due to monotonicity properties, D is a finite set with at most 2N points. First, we need the following smoothness result: Theorem 2 i. xn∗ (λ) is C 1 for λ ∈ R \ D; ii. xn∗ is continuous at λ ∈ D. We proceed first by describing the behavior of the Nash equilibrium in the neighborhood of regular points, i.e., for λ ∈ / D. Proof (Proof of Part i.) In the first step, we apply the standard implicit function theorem to prove that all the best responses at the Nash equilibrium xn∗ (λ) are actually C 1 if (x1∗ (λ), ..., xn∗ (λ), p ∗ (λ)) belongs to the interior of the domain, i.e., 0 < xn (λ)∗ < Z n / p ∗ (λ). Let consider the following function: H : [0, 1) N × R+ × R+ → R N +1 ⎛ ⎞ ⎞ ⎛ x1 V˙1 (x1 )(1 − x1 ) + λ · p ⎜ ... ⎟ ⎜ ⎟ ⎟ ⎜ ... ⎜xn ⎟ → ⎜ ⎟ ⎜ ⎟ ⎝V˙ N (x N )(1 − x N ) + λ · p ⎠ . ⎝ p⎠  xi − 1 + δ/ p λ The fact that, for a given λ there exists a Nash equilibrium in the interior of the domain, translates into the fact that, for such λ, there exists (x1∗ , ..., x N∗ , p ∗ ) such that H (x1∗ , ..., x N∗ , p ∗ , λ) = (0, ..., 0) ∈ R N +1 . Applying the implicit function theorem,

The Stackelberg Equilibria of the Kelly Mechanism

111

we conclude (as explained below) that there exists a neighborhood of λ, namely Iλ , and a C 1 function of λ such that H (x1 (λ), ..., x N (λ), p(λ)) = (0, ..., 0), ∀λ ∈ Iλ . From this, it follows that z n∗ (λ) = xn∗ (λ) · p ∗ (λ) depends on λ in a C 1 fashion. In order to apply the implicit function theorem, we need to show that the differential at (x1∗ , ..., x N∗ , p ∗ , λ∗ ) with respect to the first N + 1 variables is invertible. This requires determination of the Jacobian matrix ∂R N +1 H . For notation’s sake, let us 2 define G n (xn ) := ddx 2 Vn (xn )(1 − xn ) − ddxn Vn (xn ). A straightforward computation n yields ⎛ ⎞ G 1 (x1 ) 0 ... 0 λ ⎜ 0 G 2 (x2 ) ... 0 λ ⎟ ⎜ ⎟ ⎜ ... ... ... ... ⎟ ∂R N +1 H = ⎜ ... ⎟ ⎝ ... ... ... G N (x N ) λ ⎠ 1 ... ... 1 − pδ2 2

Given our assumption that ddxn Vn (xn ) < 0, ddx 2 Vn (xn ) ≥ 0 and by the fact that xn < 1, n it follows that G n (xn ) > 0. By this observation, it is trivial to show that the matrix above has maximal rank if and only if −

1 δ , = λ 2 p G n (xn )

which holds true; hence, the implicit function theorem can be applied. Let us now tackle the case when (x1∗ , ..., x N∗ , p ∗ ) does not belong to the interior of the domain. Without loss of generality, let us consider x1∗ = 0, 0 < xn∗ < Z n / p ∗ for n > 1. Since (0, x2∗ , ..., x N∗ , p ∗ ) is a Nash point, we must have V˙1 (0) + λ p ∗ ≤ 0, where V˙1 (0) represents the right derivative of V1 at 0. But, since we are excluding critical prices, it holds V˙1 (0) + λ p ∗ < 0 Restricting the argument above to the subspace [0, 1) N −1 × R+ × R+ , we can claim the existence of an implicit function (x2 (λ), ..., x N (λ), p(λ)) such that V˙n (xn (λ))(1 − xn (λ)) + λ p(λ) = 0, n > 1. Letting x1 (λ) = 0 and restricting the domain of the implicit function which defines p to a sufficiently small set, by continuity we can state that also V˙1 (x1 (λ)) + λ p(λ) < 0, so that x1∗ = 0 in the same neighborhood. Hence, the dependency is still C 1 also for this case.  Before proceeding to prove the second part, we characterize the behavior of players’ best responses at the Nash equilibrium. We restrict to the set  where i. the Nash / strategy is differentiable and ii. it is not saturated,  = {λ ≥ 0 | λ ∈ / D and z∗ (λ) ∈ {0, Z}}. We consider the local behavior, i.e., the dynamics with the price in any open neighborhood of prices in :

112

F. De Pellegrini et al.

Lemma 2 p ∗ and



xn are locally decreasing in .

We can now characterize the set D more precisely. Corollary 1 Let Dn0 = {λ ∈ Dn |xn (λ) = 0} and DnZ = {λ ∈ Dn |z n (λ) = Z n }, so that Dn = Dn0 ∪ DnZ . Then i. DnZ is either empty or a singleton ii. If δ > 0, Dn0 is either empty or it is a singleton. If δ = 0, Dn0 is either empty or Dn = [λ0n , +∞). From the above result, for all players n = 1, . . . , N , there exists at most a unique λnZ ∈ Dn such that xn∗ (λnZ ) = Z n / p ∗ (λnZ ) and at most a unique λn0 ∈ Dn such that xn∗ (λn0 ) = 0, where λn0 > λnZ . Using this notation, we shall now prove the second part of Theorem 2. Proof (Proof of Part ii.) We restrict to the case λ = λk0 for some k ∈ {1, . . . , N }, the proof being similar for points of the kind λkZ . In this case, we have to prove that the function λ → (x1∗ (λ), ..., x N∗ (λ), p ∗ (λ)) is left-continuous. Let {λr }r ∈N be a nonnegative sequence such that λr ≤ λ, and limr λr = λ. We need to prove that limr (x1∗ (λr ), ..., xn∗ (λr ), p ∗ (λr )) = (x1∗ (λ), ..., x N∗ (λ), p ∗ (λ)). We can restrict to a left neighborhood A of λ. We can prove that in A, the limit exists for each component of (x1∗ (λr ), ..., x N∗ (λr ), p ∗ (λr )). In fact, p ∗ (λr ) indeed converges because it is nonnegative and monotone decreasing. From the result in Theorem 3, we can choose A such that xn∗ (λr ) is either identically 0, or decreasing if 0 < xn∗ (λr ) < 1 or increasing if xn∗ (λr ) = Z n / p ∗ (λr ). Hence, the same monotonicity argument implies that the limit exists component-wise; let (y1 , ..., y N , q) be such limit. Finally, since Fn ((x1∗ (λr ), ..., x N∗ (λr ), p ∗ (λr )) = 0 and Fn is continuous, Fn ((y1 , ..., y N , q, λr )) = Fn (lim(x1∗ (λr ), ..., x N∗ (λr ), p ∗ (λr ), λr )) = n

= lim Fn (x1∗ (λr ), ..., x N∗ (λr ), p ∗ (λr ), λr ) = 0. r

 The same argument holds for the normalization condition; i.e., it holds yi = 1 − δ/q, so that (y1 , ..., y N , q) is a Nash equilibrium. From the uniqueness, it holds  yn = xn (λ), and q = p(λ) which concludes the proof. Thus, the Nash equilibrium of the game is continuously differentiable almost everywhere, with the exception of a finite set of at most 2N points where simple continuity is ensured. Actually, the points in D are the only ones where the Nash equilibrium is not continuously differentiable.

4 The Stackelberg Equilibria This section studies how to maximize the revenue obtained from the Kelly mechanism by the resource owner.

The Stackelberg Equilibria of the Kelly Mechanism

113

For any multistrategy z, and for a given price λ, the resource owner revenue is  N z n . At this point, we consider Z n < +∞ for all players. We shall comment λ n=1 on the case Z n = +∞ at the end of this section. We are interested in the existence and multiplicity of Stackelberg equilibria induced by the Kelly game. A Stackelberg equilibrium [3] is defined as a market equilibrium where one leader, i.e., the resource owner in our case, moves first. She decides the price λ of the Kelly mechanism. Afterward, the followers, i.e., the players, engage in the corresponding resulting Kelly mechanism, and the game settles on the Nash equilibrium corresponding to λ. First, we observe that the problem is well posed for the Kelly mechanism: For each price λ ≥ 0, it is possible to consider the revenue generated by the unique Nash equilibrium x∗ (λ). Correspondingly, one defines the revenue function W : R+ → R+ W (λ) = λ

N

z n∗ (λ)

(4)

n=1

Here, λ z n∗ (λ) is the revenue generated by player n at the unique Nash equilibrium induced by price λ. The Stackelberg equilibria of the game belong to the set of prices λ∗ ≥ 0 such that (5) λ∗ ∈ arg max W (λ) λ

In this section, we discuss the existence and multiplicity of equilibria solving (5). For a given price λ, we denote the total value of the Kelly game as R(λ) = λ p(λ) = W (λ) + δλ In order to proceed further, we need to establish monotonicity properties of the total value of the game. Lemma 3 For every value of λ ≥ 0, there exists a neighborhood of λ where: i. R(λ) is constant if and only if δ = 0 and 0 ≤ z n∗ < Z n for each player; ii. R(λ) is strictly increasing otherwise. Proof Without loss of generality, we assume that, for λ = λ , the players’ indexes are sorted such that for n = 1, . . . , k 0 ≤ z n∗ < Z n ∗ zn = Z n for n = k + 1, . . . , N

(6)

Using a continuity argument, we denote A the largest neighborhood of λ where (6) holds. We can rewrite condition (3) for λ ∈ A , to express the Nash equilibrium. The relation V˙n (xn )(1 − xn ) + λ p = 0 identifies xn for each player according to (3). Thus, we have

114

F. De Pellegrini et al.

R = Fn (xn )

(7)

We can express xn = Fn−1 (R), so that xn is also decreasing in R. The second condition in (3) thus becomes N

 min

 0, max

n=1

  Zn , Fn−1 R(λ) p(λ)

 =1−

δ p(λ)

(8)

We now study the monotonicity of R by writing (8) as:   λDk min 0, Fn−1 (R(λ)) = 1 + R(λ) n=1 k

(9)

N where Dk = δ + n=k+1 Zn. The unique solution in R of (9) is attained, for each λ ∈ A , when the left-hand term of (9)—is continuous and decreasing in R—equates the right-hand term, which is a constant. If D = 0, due to the monotonicity of Fn−1 , a unique value of R(λ) is possible for all λs in A , which proves case i. Conversely, for D > 0, i.e., in case ii, it is immediate to observe that the only possible case is for R(λ) to be strictly increasing  in A for (9) to hold true. It is interesting to characterize the reaction of individual players to the increase of price λ. We define Cn∗ (λ) := Cn (z n , z −n , λ), i.e., the value of the player n’s objective function at the Nash equilibrium corresponding to price λ. Theorem 3 Let A be an open set where 0 < z n∗ (λ) < Z n ; then: i. z n∗ (λ) is decreasing; xn∗ (λ) is either decreasing or a constant, where the latter holds if and only if δ = 0 and z k < Z k for each player k. ii. The optimal cost at the Nash equilibrium for the nth player is Cn∗ (λ) = Vn (xn∗ ) − xn∗ V˙ (xn∗ )(1 − xn∗ ) and it is an increasing function of λ. Monotonicity properties of the Kelly game in the price λ have been so far provided in a local sense. The global form of such properties is captured in the next result. Corollary 2 For each player n of the Kelly game, we have ⎧ ⎪ 0 ≤ λ ≤ λnZ ⎨Zn ∗ z n (λ) = p ∗ (λ) xn∗ (λ) λnZ < λ < λn0 ⎪ ⎩ 0 λn0 ≤ λ

(10)

The Stackelberg Equilibria of the Kelly Mechanism

115

Fig. 1 The pricing intervals k = [λk+1 , λk ] and the corresponding types of Nash equilibria; the lower the price, the higher the number of saturated best responses of the type z n∗ = Z n

We can hence classify the price intervals depending on the set of players such that z n∗ < Z n . We denote λk = sup{λ | z n∗ (λ) = Z n }. Without loss of generality, we can assume 0 = λ N +1 ≤ λ N ≤ . . . ≤ λ2 ≤ λ1 ≤ λ0 , where λ0 ≤ +∞, for notations’ sake. Hence, in the interval k := [λk−1 , λk ], where 0 < λk−1 < λk , the multistrategy set is of the kind z∗ = (z 1∗ , . . . , z k∗ , Z k+1 , . . . , Z N −1 , Z N ) (Fig. 1). Theorem 4 For λ ∈ k , k = 1, . . . , N , the total value R of the Kelly game is increasing. For λ ∈ 0 , R is affine increasing when δ > 0 and constant when δ = 0. Proof The proof is established using an argument similar to that in Lemma 3. In this case, we need to study the monotonicity of R in each interval k = [λk , λk−1 ], k = 1, . . . , N , where players n = N − 1, . . . , N − k + 1 play Z n at the Nash equilibrium. Hence, (8) becomes    λDk + min 0, Fn−1 R(λ) = 1 R(λ) n=1 k

(11)

N where Dk = δ + n=N −k+1 Z n . We note that the left-hand term is continuous and decreasing in R, and thus, it has always a unique solution in R. However, since the right-hand term of (11) is a constant, the solution R(λ) has to be nondecreasing in λ and strictly increasing if Dk > 0 for k = 1, . . . , N . Let denote Rk∗ = maxλ∈k R(λ), which is hence attained for λ = λk . Finally, for λ ≥ λ0 := max λn0 , it is immediate to see that R(λ) = R(λ0 ) + δλ, which concludes the proof.  In the case δ = 0, we have R = W , so that we can obtain the following Corollary 3 For δ = 0, the prices at which the Stackelberg equilibria of the Kelly game are attained is the half line {λ∗ ≥ λ1 }. Proof From Theorem 3 we already know that the maximum is attained for λ ∈ 0 , where the following equation solves for the revenue N n=1

max{0, Fn−1 (W (λ))} = 1, ∀λ ≥ λ1

(12)

116

F. De Pellegrini et al.

Because the left-hand term of (12)  decreases in W , a unique solution exists W (λ) = W0∗ for all λ ≥ λ1 . It follows that λ c bc∗ (λ) is constant, and thus, a unique maximal revenue exists in 0 . The corresponding z n∗ is determined as z n∗ = xn∗

k=1 N

z k∗ = z c∗

1 W0∗ 1 = W0∗ max{0, Fn−1 (W0∗ )} = z n0 λ λ λ

where we see that the Nash equilibrium is proportional to z 0 = (z 10 , . . . , z 0N ).



We note that for δ = 0, the resource owner is indifferent to prices λ ∈ 0 . In fact, a price increase only causes a corresponding rescaling of the players’ best responses at the Nash equilibrium. Corollary 4 Let δ = 0 and z∗ be a Nash equilibrium for the Kelly mechanism for λ ≥ 0. Then, z = λλ z∗ is the Nash equilibrium for all λ ≥ λ, and further also p(λ) = λ λ

p ∗ (λ).

Proof Of course, x ∗ (λ) = x(λ), so that a direct calculation provides

V˙n xn∗ (λ)) (1 − xn∗ (λ)) + λ p ∗ (λ) = V˙n (xn (λ))) (1 − xn∗ (λ)) + λ p(λ) so that z∗ satisfies (2) if and only if z does for λ ≥ λ.



From the previous result, we also obtain that if 0 < z n∗ (λ1 ), then λ0n = +∞; i.e., the player will play a vanishing best response at the Nash equilibrium for increasing prices. When Z n = +∞, there exists basically a unique Nash equilibrium which is rescaled as 1/λ and provides the same revenue to the resource owner. In the case δ > 0, one may think that the Stackelberg equilibrium point is unique. However, this is not the case, as shown in the next section. Nevertheless, under specific sufficient conditions imposed onto players’ utility functions, we shall show that the number of Stackelberg equilibria is in fact at most finite, and it shall be unique in the symmetric case. Remark 1 In the case Z n = +∞, for all n = 1, . . . , N , it is not immediate to interpret the behavior of the revenue when the price λ → 0. Actually, by continuity and monotonicity of R(λ), it is well possible to define the value 0 ≤ R(0) = W (0) < +∞. However, this may imply that, in the limit for vanishing prices, player n can have an infinite bid, while attaining a finite share xn∗ (0). To avoid this situation, we should consider Z n < +∞, or, as an alternative solution, we should impose λ >  for some reservation price  > 0.

The Stackelberg Equilibria of the Kelly Mechanism

117

5 Counterexample for δ > 0 For the case when δ > 0, we introduce a counterexample where the Kelly game does not have a unique Stackelberg equilibrium. Let Vi (xi ) = ai (1 − xi ),

i = 1, 2.

(13)

Then, the best response has closed form   λ p(λ) Z i , , xi∗ (λ) = max 0, min 1 − ai p(λ)

i = 1, 2

(14)

Hence, the parameters of the game, namely Z 1 , Z 2 , a1 , a2 , and δ, can be chosen such in a way that two distinct Stackelberg equilibria exist. For the sake of calculation, let us assume λ Z := λ1Z = λ2Z and λ01 < λ02 . Case λ Z < λ < λ01 . The best response of the first player vanishes when p(λ) = a1 /λ. The Nash equilibrium at λ is determined by λ p(λ) δ 1− =1− , ai p(λ) i from which p(λ) =

1+

 2 1 + 4δλ aa11+a a2 2 2λ aa11+a a2

.

The best response of the first player vanishes when λ1 p(λ01 )/a1 = 1, which corre2 sponds to the critical price λ01 = aa21 δ . It is immediate to observe that λ01 < λ02 if and only if a1 < a2 . From the expression 1 − δ, W˙ (λ) = δ  1 +a2 ) 1 + 4 δ λ (a a1 a2 we see that the revenue is decreasing. Case λ01 ≤ λ ≤ λ02 . We have p(λ) =

a2 δ , λ

W˙ (λ1 ) = δ



so that  a2 −1 . 2a1

Hence, whenever a2 > 2a1 , the revenue W (λ) decreases up to λ02 and increases in a right neighborhood of λ02 . This is the key observation for the counterexample: The Stackelberg equilibrium may not be unique because we could choose the parameters of the system such in a way that there exist two optimal prices which provide the

118

F. De Pellegrini et al.

same revenue. We describe precisely such a condition below. We remark here that the δa 2 local maximum in λ01 ≤ λ ≤ λ02 is attained at λmax = a22 /4, where W (λmax ) = 42 . Condition λ Z = λ1Z = λ2Z . We impose continuity in λiZ for each player λ Z p(λiZ ) Zi , i = 1, 2 =1− i Z ai p(λi ) and assume that p(λiZ ) = Z 1 + Z 2 + δ. It follows λiZ =

ai (Z −i + δ) , i = 1, 2 (Z 1 + Z 2 + δ)2

Simultaneous saturation occurs if and only if a1 (Z 2 + δ) = a2 (Z 1 + δ). Now, we can solve the following equation to equalize the local maxima, i.e., W (λmax ) = W (λ Z ), providing the following solution for √ 2 3 − α − (3 − α)α , Z1 = δ (α + 1)(3 − α)

Z 2 = αZ 1 − (α − 1)δ

where 2 < α = a2 /a1 < 3. We have depicted in Fig. 2a a graphic representation of the dynamics of the resource owner revenue as a function of λ for a choice of the parameters determined according to the above construction. In the example, the values are a2 = 2.7, a1 = 1, δ = 1/2. By solving the corresponding equations, we obtain the values Z 1  0.13 and Z 2  1.2. Note that λ1Z = λ2Z = λ Z = 0.51. The owner’s revenue attains the same value W (λmax ) = W (λ Z ) = 0.675.

6 Finite Sets of Stackelberg Equilibria In this section, we provide sufficient conditions in order to guarantee that the set of Stackelberg equilibria is finite. Theorem 5 i. Let either Vn be affine or gn (y) = −V˙n (y)/V¨n (y) be nonincreasing for 0 ≤ y ≤ 1/2. Then, there are at most N 2 /2 Stackelberg equilibria. ii. Under the same assumptions as in (i), in the symmetric case, i.e., Vn (y) = V (y) for n = 1, . . . , N , the Stackelberg equilibrium is unique. Proof i. We start defining the players’ contribution to the provider’s revenue, i.e., Wn (λ) := λ z n∗ (λ). We recall the expression ⎧ ⎪ ⎨λ Z n , Wn (λ) = −xn∗ V˙ (xn∗ )(1 − x ∗ ), ⎪ ⎩ 0,

0 ≤ λ ≤ λnZ λnZ < λ ≤ λn0 λ > λn0

The Stackelberg Equilibria of the Kelly Mechanism

119

Let us consider the nth player over interval λnZ ≤ λ ≤ λn0 . We consider first the case when Vn is not affine. We can write   −V˙n (xn∗ ) 1 − 2xn∗ ∗ ∗ ¨ ∗ ∗ ˙ Wn (λ) = −x˙n xn Vn (xn )(1 − xn ) 1 − V¨n (xn∗ ) xn∗ (1 − xn∗ ) 1−2x ∗

Let f (y) := x ∗ (1−xn∗ ) : f (y) is increasing in [0, 1/2] and f (0) = 0. Furthermore, n n g(0) > 0 and g(y) is decreasing by assumption. Hence, the equation g(y) = f (y) has one solution 0 < y < 1/2: Since xn∗ is decreasing in the interval considered, the sign of the term in brackets can switch from positive to negative at most once at λn such that y = xn∗ (λn ). The same reasoning holds in the case Vn is affine, because W˙ n (λ) = −x˙n∗ V˙n (xn∗ )(1 − 2xn∗ ) the same interval. and xn∗ is decreasing in Finally, since W = Wn , there can be at most N /2 local maxima per interval, from which the statement follows. ii. It is a straightforward consequence of the fact that Wn and xn do not depend on the player’s index. 

6.1 Example: A Caching Game Here, we apply the results obtained above to a caching game. The shared resource is the aggregate memory of small cells which occupy a given area in a 5G network, where mobile terminals have radio range rn > 0. Small cell is dispersed in the area, according to a Poisson process of intensity ζ. Players are content providers: They populate the caches with their contents. As a result of the Kelly mechanism, the nth content provider would be able to occupy a fraction 0 ≤ xn ≤ 1 of the caches. Then, the probability that a customer of player n would find a cache in radio range—and the nth content provider’s contents therein— is ex p(−ζπrn2 xn ). We will now take the cost function of player n to be in the form Vn (xn ) = an exp(−bn xn ) with bn := ζπrn2 and an nonnegative constants. For the sake of simplicity, we assume δ > 0 and Z n = +∞ for all players. In this case, gn (xn ) = 1/bn so that Theorem 5 holds true, and we should expect at most N 2 /2 equilibria. It is immediate to derive the relation xn∗ (λ)

   ebn 1 = max 0, 1 − L λ p(λ) bn an

120

F. De Pellegrini et al.

where L is the Lambert function, which is positive increasing (we are considering bn the principal branch). By letting rn := ean , we see that λ0n is defined by the following equation, which has a unique solution:     L rn λ0n p(λ0n ) = L rn R(λ0n ) It is possible to write the behavior of the caches’ owner revenue provided by each player as Wn (xn ) = xn∗ (λ)R(λ). It is hence seen that   L(rn R(λ)) 1 + 2L(rn R(λ)) ˙ ˙ Wn (λ) = R(λ) 1 − bn 1 + L(rn R(λ)) As expected, the term in brackets is seen to be decreasing, since R(λ) is increasing. It √ bn −1+

b2 +6bn +1

n + + is seen to vanish at λmax such that L(rn R(λmax . n n )) = αn where αn := 2 We can consider each interval of the kind Ik = {λ0k ≤ λ ≤ λ0k+1 }, for k = 1, . . . , N − 1, with λ0 = 0. For each such interval Ik , we can pick λk the solution of the following equation

N L(rn R(λ)) 1 + 2L(rn R(λ)) = N −k bn 1 + L(rn R(λ)) n=k+1

(15)

The above equation has always a solution since the left-hand term is increasing, as R is strictly increasing for δ > 0. In fact, (15) determines the local extrema of W (λ) as a function of λ: The point of maximum in Ik is determined by the following pair ⎧ 0 0 ⎪ ⎨(W (λk−1 ), λk−1 ) max (Wkmax , λmax (W (λmax k )= k ), λk ) ⎪ ⎩ (W (λ0k ), λ0k )

if λk < λ0k−1 if λ0k−1 ≤ λk ≤ λ0k if λk > λ0k

(16)

Clearly, the candidate Stackelberg equilibria are found in the set {(Wkmax , λmax k ), k = 1, . . . , N }, so that the following Proposition 1 The caching game has at most N − 1 Stackelberg equilibria.

7 The Choice of δ The choice of δ > 0 affects the utilization of the resource. However, it turns out that there exists a very tight relation between the Kelly games obtained for different values of δ. For the sake of simplicity, we deal with the unbounded case in this section. In the numerical section, we shall describe related results for the bounded case.

The Stackelberg Equilibria of the Kelly Mechanism

121

We can denote by W and R, respectively, the revenue and the total value of the game obtained for a certain value of δ. Also, W and R are the revenue and the total value of the game for δ = γδ for γ > 0. Theorem 6 Let Z n = +∞ for n = 1, . . . , N . Then, W (λ) = W (λ/γ) and R (λ) = R(λ/γ). Thus, the effect of δ > 0 is a simple compression of the price axis, so that the players switch off sooner for larger values of δ; in particular, it holds λ 01 = γλ01 . However, the revenue owner should be indifferent to the choice of δ > 0 because the maximal revenue is the same. Actually, the transformation of the game through γ described in Theorem 6 is an isomorphism between the set of the Nash equilibria Nδ = {z∗ (λ), λ ≥ 0}. More precisely ϕ : Nδ → Nδ , ϕ : z∗ (λ) →

1 ∗ λ z γ γ

(17)

Corollary 5 For Z i = +∞, a Kelly mechanism is either isomorphic to the Kelly mechanism with δ = 1 through (17) or it is the Kelly mechanisms with δ = 0. We recall that the Kelly mechanism with δ = 0 does not admit the null Nash equilibrium for any value of λ. However, from Proposition 1, we know that for δ > 0 there always exists a value λ01 above which the null Nash is the only possible equilibrium. The above result shows that the two cases in the above corollary actually correspond to two different games.

8 Numerical Results We have depicted in Fig. 2 numerical outcomes to describe in a graphic manner the results from the previous analysis. For the sake of simplicity, in the example we have considered a three-player caching game, where Z i = ai = 1 for all players, whereas b1 = 5, b2 = 5 · 2/3 and b3 = 5 · 1/3 for δ = 0.1. As it can be seen in Fig. 2b, the share of the resource requested by the players decreases as the price increases, as expected. The zoom on the dynamics of the share which players attain at the Nash equilibrium is shown in Fig. 2c. There, we can see that for small prices, i.e., 0 ≤ λ ≤ 0.21, at first all the three players saturate their best responses at the Nash equilibrium, i.e., z i∗ = Z i = 1 for i = 1, 2, 3. Afterward, around λ = 0.7, players 1 and 2 stay saturated, z 1∗ = z 2∗ = 1, which is denoted by the fact that xi , i = 1, 2 increases. This happens since the third player plays a bid decreasing with the price. However, at λ = 0.22, player 1 starts playing smaller bids. The interpretation of this fact is that for player 1, it is not convenient to bid to obtain a large share, because the marginal cost leads to price increase. Conversely, for the third player, the cost marginal performs the price increase. Actually, the third player switches off first, since we have λ03 < λ02 < λ01 .

122

F. De Pellegrini et al.

(a)

(b)1

(c)

0.6

0.34

0.4

0.5 0.32

0.2 0 0

2

4

0 0

6

20

0.3 0.2

40

0.22

0.24

Fig. 2 a The counterexample for n = 2 and a choice of parameters equalizing the local maxima W (λmax ) = W (λ Z ); b Share of the three players at the Nash equilibrium as a function of the price and c Share detail for small prices

(a)

(b)

(c)

0.6

0.6

4

0.4

0.4

2

0.2

0.2

6

0 0

20

40

60

0 0

20

40

60

0 0

20

40

Fig. 3 a Revenue versus total value of the Kelly game; b Breakdown of the revenue contributions of different players; c Revenue of the resource owner for various values of δ = 0.1, 0.4, 0.6, 1

It is interesting to note that there is an inversion in the shares attained by player 2 and player 1. In fact, after the second player exits from the saturation condition, we observe that x1 < x2 up to λ  4.2. However, at larger prices, the fact that b1 > b2 prevails, since for player 2 soon the price is not worth buying a larger share than x1 . For player 1, the cost reduction prevails, so that in the right part of the curve, we have instead x2 < x1 . In Fig. 3a, we have shown a comparison of the revenue of the game W and the total value of the game R. As seen there, R is strictly increasing as expected, whereas W , for the considered set of parameters, is unimodal and vanishes for λ ≥ λ01 . Also, in Fig. 3b, we have depicted W and superimposed the revenue generated by each player. In particular, we can see that the game has a unique Stackelberg equilibrium, attained for λmax = 2.8350, where x1∗ = 0.2618, x2∗ = 0.2684, and x3∗ = 0.1854. Thus, the resource owner, for this choice of parameters, has an incentive to keep the price relatively small and yet attain a relatively high utilization of the resource. Finally, in Fig. 3c, we have depicted the effect of the choice of δ on the revenue, for same choice of the parameters. In this case, the rescaling effect of the price axis appears clearly also in the bounded case, i.e., Z n < +∞. Choosing δ < 1, in fact, causes a stretch of the revenue dynamics toward the left side of the price axis with respect to the case δ = 1. In the bounded case, we see that perfect rescaling as described by Theorem 6 is not possible. In fact, there exist border effects due to

The Stackelberg Equilibria of the Kelly Mechanism

123

saturation, which appear for small prices. We conjecture, though, that the perturbation effect compared to the unbounded case is small. As such, we expect that the impact on the maximum revenue of the resource owner is negligible, but we leave more precise insight into future works.

9 Conclusion In this paper, we have considered a Stackelberg game with one leader (resource owner, who sets the price for the resource) and N followers (who receive a fraction of the resource according to their bids, under the Kelly mechanism). We have shown that the game may not possess a unique Stackelberg equilibrium, and we have detailed a counterexample where two optimal prices exist for the resource owner. We have shown that the set of Stackelberg equilibria is uncountable for the classical Kelly mechanism with null resource signal (δ = 0). Moreover, we have shown that under mild regularity assumptions on the players’ cost functions, use of nonzero resource signal can guarantee that the number of optimal prices is at most N /2, where N is the number of followers, and unique in the symmetric case. Finally, with respect to the revenue of the resource owner, in the unbounded case (Z n = +∞), there exist only two Kelly games. The former has null resource signal (δ = 0). The latter is equivalent to the one under unitary resource signal (δ = 1): Equivalence is given by an isomorphism preserving the interval of revenues available to the resource owner. In future works, we shall investigate the effect of the bound the strategy sets (Z n < +∞) on the resource’s signal.

References 1. Altman, E., Hanawal, M.K., Sundaresan, R.: Generalising diagonal strict concavity property for uniqueness of Nash equilibrium. Indian Journal of Pure and Applied Mathematics 47(2), 213–228 (2016) 2. Rosen, J.B.: Existence and uniqueness of equilibrium points for concave N-person games. Econometrica 33(3) (July 1965) 3. Ba¸sar, T., Olsder, G.: Dynamic Noncooperative Game Theory. Society for Industrial and Applied Mathematics, 2nd edn. (1998) 4. De Pellegrini, F., Massaro, A., Goratti, L., El-Azouzi, R.: Bounded generalized Kelly mechanism for multi-tenant caching in mobile edge clouds. In: Network Games, Control, and Optimization. pp. 89–99. Springer International Publishing, Cham (2017) 5. Johari, R.: Efficiency Loss in Market Mechanisms for Resource Allocation. Ph.D. thesis, Department of Electrical Engineering and Computer Science (2004) 6. Maheswaran, R., Baçar, T.: Efficient signal proportional allocation (ESPA) mechanisms: Decentralized social welfare maximization for divisible resources. IEEE J. Sel. A. Commun. 24(5), 1000–1009 (2006) 7. Yang, Y., Ma, R.T.B., Lui, J.C.S.: Price differentiation and control in the Kelly mechanism. Elsevier Perf. Evaluation 70(10), 792–805 (2013)

To Participate or Not in a Coalition in Adversarial Games Ranbir Dhounchak, Veeraruna Kavitha and Yezekael Hayel

Abstract Cooperative game theory aims to study complex systems in which players have an interest to play together instead of selfishly in an interactive context. This interest may not always be true in an adversarial setting. We consider in this paper that several players have a choice to participate or not in a coalition in order to maximize their utility against an adversarial player. We observe that participating in a coalition is not always the best decision; indeed selfishness can lead to better individual utility. However, this is true under rare yet interesting scenarios. This result is quite surprising as in standard cooperative games; coalitions are formed if and only if it is profitable for players. We illustrate our results with two important resource-sharing problems: resource allocation in communication networks and visibility maximization in online social networks. We also discuss fair sharing using Shapley values, when cooperation is beneficial. Keywords Cooperative game theory · Shapely value · Social networks

1 Introduction Resource allocation problem is a well-known generic problem that involves complex optimal solutions. This type of problem is well known in networking context. One of the most studied models is the proportional framework [11]. Several users share a common resource and each one gets a part of it proportionally to his/her action. In fact, this mechanism induced a utility function which is linear with respect to user’s R. Dhounchak (B) · V. Kavitha IIT Bombay, Mumbai, India e-mail: [email protected] V. Kavitha e-mail: [email protected] Y. Hayel University of Avignon, Avignon, France e-mail: [email protected] © Springer Nature Switzerland AG 2019 J. Walrand et al. (eds.), Network Games, Control, and Optimization, Static & Dynamic Game Theory: Foundations & Applications, https://doi.org/10.1007/978-3-030-10880-9_8

125

126

R. Dhounchak et al.

action. This model has lead to the well-known proportional fair solution concept which has been applied with success in numerous resource allocation problems in networking and in security [7]. In another communication domain, social networking, such linear utility function is useful to model user preferences [13]. Indeed, such type of function has been proved to efficiently model mean number of messages on a timeline that belong to a particular source. This mean number is proportional to the ratio of the sending rate of that message over the total rate of all messages. When one is interested in relative visibility, which is defined as the ratio of expected copies of a message currently alive in a social network to the total number of expected copies of all such competing messages, it results in an exponential cost (e.g., [6]). Such message propagation processes are modelled predominantly using branching processes. We consider linear as well as exponential cost functions. A problem closely related to our first linear model problem is considered in [5]. However, the utility of the adversary is different from that considered in [5]: in our case the adversary can be optimizing its own utility like any other player, but this will have adversarial influence on the rest. Assuming a mechanism to identify the adversary by the rest of the other players, this type of game is well adapted to formulate a security game with several defenders as in [8]. Further, this player is not interested in participating in cooperation with any other player. The main theme of this paper is to study the possible improvement obtained by the rest of the players, in presence of such an ‘adversarial’ player. In regular coalition formation games, like the ones of [9], there is no adversarial context. In a wireless communication context, a coalition formation game against multiple attackers has been studied in [10] and the authors have shown an increase of the average secrecy rate per user up to 25%. We also consider coalition game against (one) adversary, our study is theoretical and also deals with the fairness of the Shapley value mechanism. Two main features are considered in our study which builds our contributions to resource-sharing games: • Adversarial context: We consider that players are in an adversarial environment. Particularly, one specific player’s objective is to minimize the total utility of all the other players, the social welfare of other players. • Cooperation is possible: To improve the utility at equilibrium, the players can participate into a coalition and merge their efforts to defeat the adversary. When players decide to form a coalition, it leads to a non-cooperative game between a coalition (group of players) and one adversary. We show that to form a coalition is not always the best choice in an adversarial resource-sharing games. But this is the case only for some rare scenarios. Another interesting result is, when coalition is beneficial for players, all but one of them should remain silent (their actions at equilibrium are to be inactive/zeros). It is this silencing which not only helps the coalition, but also the opponents (or non-participants), and eventually leads to situations where grand coalition may not be the best. In such cases, the Shapley values fail to divide the gains fairly. All these results are deeply investigated in two resource-sharing games: a linear model which describe Kelly’s fairness mechanism and visibility competition; and an exponential model that describes the visibility via propagation of messages in social networks.

To Participate or Not in a Coalition in Adversarial Games

127

The paper is organized as follows. In Sect. 2, we study an adversarial resourcesharing game in which individual’s utility is linear with its action. We first describe the fully non-cooperative scenario in which all individuals play selfishly. Second, we consider the cooperative setting in which players form coalitions in order to enhance, if possible, their individual utilities. Section 5 is devoted to another scenario inspired by social networks. Players aim is to maximize the visibility of their contents. This visibility is measured in terms of expected number of timelines reached through the process of re-posting or forwarding. This utility function is exponential in the player’s preference parameters.

2 Linear Model Consider a system with many players competing against each other, for shared resources. In such scenarios, it might be beneficial for the players to cooperate with each other. Further, assume there exists an adversary whose aim is to harm the rest of the players. Alternatively the adversary might be aiming for its own benefit, but its actions adversely influence the utilities of the rest of the players. Possibly the adversary is the player that is not interested in participating in any sort of cooperation. We refer the rest of the players, willing to explore cooperation benefits, as C-players. We study the gain of the C-players, with and without participating into cooperation, in an interaction with an adversary. We deal with two possible scenarios: (a) Non-cooperative scenario (NCS): This is a complete non-cooperative game between all the players (C-players and the adversary); (b) Cooperative scenario (CS): C-players participate into a common coalition and we are faced with a twoplayer non-cooperative game, one of them being the aggregate player formed by the C-players and the second player is the adversary. We also derive a transferable utility (TU) game, that defines worth for each subcoalition of C-players (in the presence of adversary), to share fairly the benefits derived by cooperation (when beneficial) using well-known cooperative solution concepts, e.g., Shapley value.

2.1 Fully Non-cooperative Scenario The resource-sharing game involves C-players and the adversary in a standard noncooperative game. Let n (with n ≥ 2) represent the number of C-players. The utility of C-players is given by (with {λi } the influence factors, γ the cost factor): λi ai − γai , a := (a0 , a1 , · · · , an ) for any 1 ≤ i ≤ n, Ui (a) = n j=0 λ j a j

(1)

128

R. Dhounchak et al.

where the action of player i, ai ∈ [0, a], ¯ for some a¯ < ∞. The utility function of the ¯ adversarial player, denoted by U0 , is given by (with a0 ∈ [0, a]): n λi ai λ0 a 0 U0 (a) = − ni=1 − γa0 , or equivalently, U0 (a) = n − γa0 . j=0 λ j a j j=0 λ j a j

Let U := (U0 , U1 , · · · , Un ), represent the utility functions of all players. Thus we  non-cooperative strategic form game given by tuple  have an (n + 1)-player {0, 1, · · · , n}, [0, a] ¯ n+1 , U . We analyse this game using the well-known solution concept, the Nash Equilibrium (NE). This game setting is an adversarial extension of the well-known Kelly’s problem about optimal resource allocation, particularly studied in communication networks [11]. This utility function represents a compromise between proportional share of a global resource for each user and a cost which depends on the action taken. We first derive the NE when it lies in the interior of the strategy space, for each player. Throughout we consider the actions in a bounded domain with a¯ > n/γ. A generalization could be of future interest.  Lemma 1 (Positive NE) Define s := nj=0 1/λ j . Assume s > n/λ j for all j. Then there exists unique NE a∗ = (a0∗ , · · · , an∗ ) which lies in the interior, i.e., a∗ ∈ (0, a) ¯ n+1 . The NE and the corresponding utilities are given by (for any j): a ∗j

=

 n s− γλ j

n λj s2



 and U ∗j =

s− s

n λj

2 .

(2) 

Proof The proof is given in Appendix A of [14].

We now consider the situation for which the conditions of the above Lemma are not satisfied, i.e., when s ≤ n/λ j for some j. In this case, one can expect that at least one of the players would be silent in the NE, i.e., we meant a ∗j = 0 for at least one j. Towards investigating this, we first  claim that there exists a unique subset of players J ⊂ {0, 1, · · · , n} with sJ := i∈J 1/λi such that sJ >

|J | − 1 |J | − 1 for all j ∈ J and sJ ≤ for all j ∈ J c . λj λj

(3)

When the above happens, one can expect a NE with nonzero components for only players in J (as in Lemma 1). We will show that such a J indeed exists, and then show that the game has unique NE with the said zero/nonzero components. Towards this, consider the permutation π on the set of players {0, 1, · · · , n} such that λπ(0) ≥ λπ(1) ≥ · · · ≥ λπ(n) . Let λi := λπ(i) and λn+1 = any value. Theorem 1 (i) There exists a unique 1 ≤ k ∗ ≤ n such that (1 is indicator): ⎛ ⎞−1 ⎞−1 ∗ +1 k k∗ ∗+1 ∗ k k 1 1  ⎠ ≤ λ0 < ⎝ ⎠ . 1{k ∗ n/λ j }. (ii) There exists unique NE, (a0∗ , · · · , an∗ ), with nonzero components only in J ∗ : a ∗j =

 (|J ∗ | − 1) sJ ∗ −

|J ∗ |−1 λj



2 γλ j sJ ∗

⎛ 1{ j∈J ∗ } and U ∗j = ⎝

sJ ∗ −

|J ∗ |−1 λj

sJ ∗

is the optimal utility for any 0 ≤ j ≤ n at NE, where sJ :=

 j∈J

⎞2 ⎠ 1{ j∈J ∗ } ,

1/λ j . 

Proof is in Appendix A of [14].

If hypotheses of Lemma 1 are satisfied, it is clear that k ∗ = n and all the players have nonzero utility at NE. It is beneficial for the weaker agents (given by (J ∗ )c ) to remain silent and the players forced to remain silent is determined by relative values of the inverses {1/λi }i .

2.2 Cooperative Scenario (CS) In cooperative scenario, C-players explore the cooperative opportunities, if any, whereas recall that the adversary does not take part in any coalition. The adversary remains a particular player. Each player among the C-players seeks to form appropriate coalition with the other C-players, such that they have the best share in presence of the adversary. We first study the grand coalition of all C-players. Towards this consider a two-player non-cooperative game: the adversary is one player and the C-players join together to form one aggregate player. The utility of the  aggregate player equals the sum of the utilities of all C- players, i.e., Uag = j≥1 U j , while that of the adversary equals Uad = U0 . The strategy set of the aggregate player equals the product strategy set [0, a] ¯ n again with a¯ > n/γ. We study the NE of this two-player game, with the aim to study the maximum improvement possible by ‘grand coalition’ of C-players. We call the corresponding NE as Cooperative NE (CNE) to distinguish it from the NE of the previous subsection. Thus the strategic form game to study the cooperative scenario is described as ¯ n+1 , {Uad (.), Uag (.)} with ag := (a1 , · · · , an ) and ad := a0 and the {ad , ag }, [0, a] utilities of aggregate and adversary are n

j=1 λ j a j

Uag (ag , ad ) = n

j=0 λ j a j

−γ

i≥1

λ0 a 0 ai , Uad (ad , ag ) = n − γa0 . j=0 λ j a j

Without loss of generality, we assume λ1 ≥ λ2 ≥ · · · ≥ λn , throughout the paper. The CNE is given by (proof is in [14]):

130

R. Dhounchak et al.

Lemma 2 (i) When λ1 > max j≥2 λ j , the CNE is λ1 λ0 ∗ a1∗ = a0∗ =  2 , and a j = 0 for all j > 1. γ λ1 + λ0

∗ The utilities are Uag =



λ1 λ1 + λ0

2

∗ and Uad =



λ0 λ1 + λ0

2 .

(4)

(ii) When |Jm | > 1, with Jm := arg max j≥1 λ j , we have infinitely many CNE:   λ1 λ0 (a0∗ , a1∗ , · · · , an∗ ) : a ∗j = a0∗ , a ∗j = 0 ∀ j ∈ / Jm and a0∗ =  2 . γ λ1 + λ0 j∈Jm But the utilities at any CNE remains the same and equals that given in (4).



Remarks: (1) A close look at the CNE reveals that (all) the weaker C-players are silenced, i.e., a ∗j = 0 for j ≥ 2. Thus the benefit of cooperation (if any) is obtained by the weaker players agreeing to remain silent. Observe that (all) these players may not remain silent in non-cooperative scenario. (2) The proof (provided in Appendix A of [14]) basically shows that the utility (and thus the best response) of the aggregate player, at any action profile and against any a0 , is dominated by the utility (respectively the best response) at an appropriate action profile with nonzero value only for player 1. The rest of the proof is obtained by solving the extremely simplified reduced game (the aggregate player uses only one component action). This result is readily applicable for any sub-coalition (with more than one players), even when it includes adversary (as would be required for completely defining the TU game). (3) When some C-players are of equal influence, say if λ1 = λ2 , then one can have multiple CNE, but the aggregate utility at any CNE remains the same.

3 Benefit of Cooperation In the previous sections, we studied the utilities derived by C-players in noncooperative and cooperative frameworks. The natural question then arises, “whether forming a grand coalition among C-players ameliorates the total utility driven by the  said players as obtained in non-cooperative framework.” Towards this, let UT∗ := nj=1 U ∗j be the total/aggregate utility in non-cooperative scenario, where {U ∗j } j are the utilities of C-players under the Nash Equilibrium (Theorem 1), and ∗ is the utility of the aggregate player under the CNE (given by Lemma 2). recall Uag We define an appropriate indicator, which we refer as ‘Benefit of Cooperation’ (BoC) Ψ for measuring the normalized advantage (if any) obtained by forming coalition:

To Participate or Not in a Coalition in Adversarial Games Ψ =

∗ − U∗ Uag T ∗ + U∗ Uag T

131

× 200.

If the C-players are better without cooperation, i.e., if their utilities under the NE are better than those under CNE, Ψ will be negative. Otherwise we have positive BoC, Ψ . We study the influence of adversary player on BoC. In particular, we analyse the variations in Ψ , as λ0 increases from zero to a large value, much larger than λ1 . From Theorem 1, the total utility of C-players in non-cooperative scenario is UT∗ =



 U ∗j = |J ∗ | − 2(|J ∗ | − 1)

j≥1, j∈J ∗

sJ ∗

j≥1

Further using Lemma 2, we have:

1/λ j 

Ψ = 200 

 + (|J ∗ | − 1)2

λ1 λ1 +λ0 λ1 λ1 +λ0

2 2

− UT∗ + UT∗

j≥1, j∈J ∗ s2 J∗

1/λ2j

.

.

(5)

3.1 Equal C-Players We begin with the study of the special case, when λ j = λ1 for all j ≥ 1. In this case, s = n/λ1 + 1/λ0 and all the C-players satisfy the hypothesis of Lemma 1 (clearly s > n/λ1 ). However, the adversary may or may not satisfy the same and we study the different sub-cases in the following (proof in Appendix A of [14]). Lemma 3 (1) If the adversary is weak, i.e., when s ≤ n/λ0 , the BoC Ψ is decreasing in λ0 as below:  Ψ = 200

λ1 λ1 + λ0

2

1 − n

  

λ1 λ1 + λ0

2

 1 + , with J ∗ = {1, · · · , n}. n

Further, we have the smallest BoC at λ∗0 =

n−1 n3 − (2n − 1)2 λ1 and the minimum BoC equals Ψ (λ∗0 ) = 200 3 . n n + (2n − 1)2

(2) If the adversary is strong i.e., when s > n/λ0 (i.e., if λ0 > λ1 (n − 1)/n), the BoC is increasing with λ0 and reaches 200(n − 1)/(n + 1) as λ0 → ∞:  Ψ = 200 

λ1 λ1 +λ0 λ1 λ1 +λ0

2 2

−n +n

 

λ1 nλ0 +λ1 λ1 nλ0 +λ1

2 ∗ 2 with J = {0, 1, · · · , n}.



132

R. Dhounchak et al.

Fig. 1 λ1 = λ2 , n = 2, BoC increases, rate of convergence faster with small λ1

To Participate or Not: Thus the BoC decreases monotonically with λ0 until λ∗0 = (n − 1)λ1 /n and increases monotonically afterwards. The minimum possible BoC Ψ (λ∗0 ) provided by the above Lemma can easily be computed. It is easy to verify that the minimum BoC is positive for all n > 2. When n = 2, it equals -200/17 approximately and by continuity (which can easily be verified) we have a range of adversary strength λ0 (around λ∗0 ) for which the BoC is negative. Thus it is always beneficial to participate in cooperation when there are more than two equal strength C-players. However when there are only two equal strength C-players, it may not always be beneficial to cooperate. These results are reinforced in the numerical example of Fig. 1. As the number of C-players increases, irrespective of weak or strong adversary, UT∗ always decreases in n and limn→∞ UT∗ = 0. This is kind of obvious and is mainly because the interference is increasing. However, the more interesting fact is that the cooperative scenario is lot more beneficial. From Lemma 3, the BoC  Ψ = 200 

λ1 λ0 +λ1 λ1 λ0 +λ1

2 2



1 n

+

1 n

 → 200 and Ψ = 200 

1 λ0 +λ1 1 λ0 +λ1

2 2

−n +n

 

λ1 nλ0 +λ1 λ1 nλ0 +λ1

2 2 → 200,

as n → ∞, respectively, with weak and strong adversary. In all, irrespective of the strength of the adversary, a large number of equally strong C-players benefit to the maximum extent by participating in coalition. One may have very different results with unequal C-players, we next study the same.

Fig. 2 λ1 = λ2 , n = 2, BoC decreases, increases and finally converges to 0

To Participate or Not in a Coalition in Adversarial Games Table 1 Example scenarios in which Cooperation Fails Sr. No. λ0 λ1 λ2 λ3 λ4 J 1 2 3 4

2.1 1.36 1.3 1.34

3 2.6 2.7 2.7

1.2 2.4 2.4 2.4

1 1.3 1.32 1.32

0.5 1.3 1.1 1.31

{0, 1} {0, 1, 2} {0, 1, 2, 3} {0, 1, 2, 3, 4}

133

Ψ

{λ0 : Ψ ≤ 0}

0 −6.17 −5.63 −5.44

(2,∞) [1.18, 1.6] [1.2, 1.55] [1.22, 1.62]

Unequal C-Players We numerically compute the BoC (5) for some examples in Figs. 1, 2. We plot Ψ as a function of λ0 , ranging from 0 to 3 and with γ = 0.4. In Fig. 1 with equal players, the BoC Ψ decreases initially with λ0 till λ1 /2, takes minimum value −200/17 at λ0 = λ1 /2, and further on increases towards 200/3. This is true for all the examples of the Figure and is exactly as depicted by Lemma 3. On the other hand, when λ1 = λ2 as in Fig. 2, we have a similar trend for initial values of λ0 . BoC decreases as λ0 increases, reaches a minimum value and starts raising again. However, in contrast to the equal C-player case, the BoC eventually decreases to zero. It is clearly the case for λ2 ≤ .95 and one can observe a similar trend even for the case with λ2 = 1.2. One can compute the strength of the adversary for which BoC is zero, using Theorem 1. For example, when λ1 = 1.5 and λ2 = 0.5 the BoC is zero at λ0 = 0.3974, by using: Ψ/200 =



λ21 + λ22 λ1 2 2.25 2.25 + 0.25 − = − = 0. λ0 + λ1 (λ2 + λ1 )2 (λ0 + 1.5)2 4

We notice again the cases with negative BoC surrounding such zero BoC cases. When to participate: For equal C-player case, it is beneficial to cooperate once the number of C-players is greater than two. However, this may not be true when the players are of uneven strengths. In Table 1, we tabulate the cases with negative BoC for the case with n = 4. Thus we have a second contrast with respect to the equal player case: cooperation may not be beneficial even when there are more number of C-players. It rather depends upon the relative strengths of the C-players and that of the adversary. For example, one can have huge number of C-players, however most of them are not sufficiently strong, and in effect we have only two players with nonzero components in NE (can be verified easily): Lemma 4 When λ0 ≥ λ1 λ2 /(λ1 − λ2 ) with λ1 > λ2 , J ∗ = {0, 1} irrespective of the number of C-players, and so UT∗ =



λ1 λ1 + λ0

2 and hence Ψ = 0 ∀ n ≥ 2.



Observe that λ1 , slightly greater than λ2 , is sufficient for this zero BoC case. In Fig. 3, we considered two examples with n = 4 to further illustrate this. The BoC for the case with equal agents converges towards 120 (as given by Lemma 3), while that with

134

R. Dhounchak et al.

Fig. 3 Drastic difference between equal and unequal agents

unequal agents converges to zero, as λ0 increases. The differences (0.01, 0.02, etc.) in the strengths of the respective agents, in the two examples is negligible, however the outcome is drastically different. Lemma 4 also illustrates a third contrast: there would be scenarios when BoC converges to zero (or is zero) as the number of C-players increases.

4 Fair Sharing In the previous section, we studied the BoC derived by the grand coalition of C-players. In the scenarios where BoC is positive, it is important to quantify the sharing of the worth among C-players. Towards this, we form a relevant Transferable utility (TU) game (e.g., [1]), which has many cooperative solutions (e.g., core, Shapley value, etc). We consider only Shapley value in the current paper, while other solution concepts could be of future interest.

4.1 Transferable Utilities Let S := {C : C ⊂ {1, · · · , n}}, represent the collection of all subsets of the players, basically the collection of all possible coalitions. The characteristic function v : S → R, maps every coalition to a value on real line R, which represents the worth of the coalition. We first discuss a commonly used practice that defines a TU game from any given strategic form game. Consider any coalition C ⊂ {1, · · · , n}. The worth of this coalition v(C) is defined as the value of the two-player zero-sum game, where the first player is the aggregate of all the players from C and the second player is aggregate of the rest of the players C c := {1, · · · , n} − C. The first aggregate player

To Participate or Not in a Coalition in Adversarial Games

135

maximizes the sum of utilities (utilities as given in strategic form game) of all the players from C, while the second aggregate player minimizes the same sum utility. We need some changes to this procedure to suit the problem underhand: (a) each player pays some cost for using an action; (b) we have an adversary. To incorporate the second factor, we include the adversary as a part of the second aggregate player. To incorporate the first factor, we define the reward of the second aggregate player as negative of the reward of the first aggregate player and then include the cost for choosing the particular action(s). Thus we define the worth of the coalition C as the utility obtained by the first aggregate player using the NE of the following two-player strategic game:  j∈C

Uag,1 =  j∈C

λjaj +



λjaj λ j a j + λ0 a 0

j∈C c

 j∈C

Uag,2 = 1 −  j∈C

λjaj +



−γ

λjaj

j∈C c

λ j a j + λ0 a 0

j∈C

a j and ⎛

−γ⎝



⎞ a j + a0 ⎠ .

j∈C c

The analysis of this game is exactly as in Lemma 2. By Lemma 2 and remarks thereafter, this aggregate game may have multiple NE but the utility of the two aggregate players remain the same irrespective of the CNE. The worth of coalition C is defined as the utility under a CNE, i.e., from Lemma 2:  v(C) :=

2 λC c with λC := max λ j , λC := max{λ0 , λC c }, λc{1,··· ,n} = λ0 . c j∈C λC + λC

(6)

  Thus we have a TU game {1, · · · , n}, v(·) . There are many fair solutions for TU games, and we consider the well-known Shapley value. The Shapley value of player k, φk , is given by the following (e.g., [1]): φk =

C∈S ,k ∈C /

|C|!(n − |C| − 1)! [v(C ∪ {k}) − v(C)] for any player k. n!

(7)

4.2 Shapley Value We compute the Shapley value for this TU game and the details are in [14]. Towards that, we first compute the improvement in worth of a coalition C when player k joins it, we group coalitions into sub-classes based on the improvement they provide and finally derive the Shapley value. The Shapley value for the case with λ0 ≥ λ1 is given by (details in [14]): 2 2   n λk λm 1 1 − . (8) φk = k λk + λ0 m(m − 1) λm + λ0 m=k+1

136

R. Dhounchak et al.

For the case with λ1 ≥ λ0 the Shapley value equals (details in [14]): 1 φ1 = n



λ1

2

n−1

1 (1 + m)m



λ1

2



λ1+m λ1+m + λ1

2 

+ − λ1 +  λ2 λ1 +  λ2 m=2   2  2  2  2  n−1 λ2 λ2 λ1 λ1 1 1 + − − + , (m + 1)m λ2 + λ1 n λ1 + λ0 λ2 + λ1 λ1 +  λ1+m m=2

and for any k ≥ 2:  2 2  2  n−k λk λk 1 λk+m + − λk + λ1 (k + m)(k + m − 1) λk + λ1 λk+m + λ1 m=1       2    n 2 2 2 1 1 λ1 λ1 λ1 λ1 + − − + . m(m − 1) n λ1 + λ0 λ1 +  λm λ1 +  λk λ1 +  λk φk =

1 n



m=k+1

Numerical observations: Figures 4, 5 show the Shapley values and the utilities under NE for two examples with strong adversary (λ0 ≥ λ1 ) and n = 4. Both the examples have positive BoC. In Fig. 4, the sharing as given by Shapley values (lines with markers) is fair: all players derive more than their respective utilities at NE (non-cooperative scenario). In Fig. 5, the allocation is not fair: the stronger agents derive lesser utilities than those under NE while the weaker ones improve. Thus the conclusion is that the Shapley values are not always fair for this system. The game is not super-additive (and hence not convex) and this could be the reason. Further the grand coalition may not be the best in this case: one can easily gather examples for which v({1}) = v({1, · · · , N }). The above discussion shows that one needs an alternate way of sharing the utilities among C-players. We intend to work towards this in future. We close this discussion by suggesting a heuristic, which could be a part of our future work: ψi := Ui∗ + 

Fig. 4 Shapley values are always fair

λi j≥1 λ j

  ∗ Uag − UT∗ .

(9)

To Participate or Not in a Coalition in Adversarial Games

137

Fig. 5 Shapley values are not always fair

Few initial remarks: Basically share the overall gain (above the non-cooperative utility), among the C-players, proportional to their influence factors. Note that (9) need not be individually rational, as v({1}) = v({1, · · · , N }) and v({1})  U1∗ for some examples (e.g., Fig. 5). These observations are mainly due to the following: “when players cooperate they not only aid themselves, they also aid the opponents. And this is because the weaker ones remain silent is the best strategy for cooperative scenario, which reduces the interference to all, including the opponents”.

5 Game with Exponential Utilities Online social networks (OSNs) play an instrumental role in marketing of products or services of the organizations. Due to immense activities of the users, OSNs become predominant vehicles to proliferate contents therein. The marketing/advertising companies make use of these platforms to publicize their ‘content of interest’. We use the model and results of [6] (studies content propagation properties using branching processes), for analysing the importance of cooperation when ‘relative visibility’ of a content is important to its content provider (CP). We first summarize their model and results. Each user of an OSN possesses timeline (TL) structure, basically an inverse stack of certain length, where different contents appear on different levels based on their newness. The old content is pushed down by the arrival of new content to the TL. On the other hand, the content gets ‘replicated’, if a user forwards the post to its friends. After a series of such events a particular content, depending upon the interest it generates, either gets extinct (gets deep down the TLs, to attract any further attention) or gets viral; i.e., the copies of the post grow exponentially fast due to rigorous sharing (see [6]), after considerable time t. A marketing content is posted to some users initially by a content provider (CP). The extinction/virality of a post depends upon network parameters as well as the quality of the post represented by η. It is shown that the number of copies of a content grows in accordance with branching process under certain assumptions. Using the theory of branching processes, the growth rate of the content is shown to be proportional to η (see [6]). And growth rate characterizes the visibility of the content: the more the growth rate, the more the visibility.

138

R. Dhounchak et al.

The authors in [6] obtained the expected number of TLs (E[X (t)]) containing the post at time t, after inception with one copy. We reproduce the expression for this expected number, for one interesting example scenario in which the shared post always sits on the top of the recipient TL immediately after sharing (see [6, Theorem 1 and Lemma 1] with ρi = 1i=1 ): ¯ αt = kecwη , where E[X (t)] ≈ ke

(10)

  α := (1 − θ)md1 d2 wη − 1 + θd2 (λ + ν), is the virality coefficient, ri = d1 d2i , is the probability that user reads the post at level i of TL, λ, ν − rates (of appropriate Poisson processes) at which users visit the OSN, w − the influence factor of CP,  m − the  mean number of friends −1+θd2 (λ+ν)t

(1−θ)md2 wη 1 ¯ , k = ke k¯ = 1−d 2 (1−θ)md2 wη+θd2 −θ c := (1 − θ)md2 (λ + ν)t and θ =

,

λ . λ+ν

The above is the expected number of TLs, with the post under consideration, at some level of the TL. Here wη represents the probability that a typical user shares the post. Further, we consider the case with d2 close to 1 (users read post from a good number of levels), and we, hence, approximate k¯ ≈ 1/(1 − d2 ), a constant independent of η. Thus in the expression (10) for the expected number of posts, k and c are constants independent of the action/strategy η. Further the probability of virality (i.e., the probability that a post gets viral) is positive if and only if the virality coefficient α > 0 ([6, Lemma 2]). Thus E[X (t)] > 0 if and only if α > 0. Hence E[X (t)] = 0 for any η ≤ η¯ where: η=

1 − θd2 1 × , (1 − θ)md2 w

(11)

and if η > η, then E[X (t)] is given by (10). Thus summarizing, the expected number of TLs with the content of a CP depends on its quality η:  kecwη η < η ≤ y(η; w) = 0 else,

1 w

 ke x x < x ≤ c or equivalently y(x; w) := (12) 0 else,

after the change of variable to x = cwη and where x := cwη. It is important to note that the constants c, x depend only upon the network parameters, and are not altered by w the CP related parameter (see Eqs. (10)–(11)). Thus these boundary points would be the same for any CP using the given OSN. Content of competing CPs, when propagate through the same OSN and at the same time, creates interference to each other by reducing the visibility of each other’s post. The visibility of the content of a particular CP is proportional to the number of TLs with its own content and inversely proportional to the number of TLs with content of the competing CPs. We consider (n + 1) number of competing CPs, one among

To Participate or Not in a Coalition in Adversarial Games

139

them being an adversary. As before, adversary could be a player that might not be interested in participating in any coalition or the aim of the adversary could be to jam the visibility of the content of other CPs. We study (as before) if it is good to participate in cooperative strategies.

5.1 Non-cooperative N Players Let x j represent the quality of CP- j content. Its content gets viral only if x j > x. If the content gets viral, the expected number of shares is given by (12). The CP incurs a cost proportional to its (actual or non-transformed) quality η j = x j /w j and its aim is to get best relative visibility of its content over OSN. Thus the utility of CP j when it creates a content of quality x j and when others create content with respective qualities (x0 , · · · , x j−1 , x j+1 , · · · , xn ) equals (see (12)):  U j (x0 , x1 , · · · , xn ) =

0−

γx j , wjc

xj

 ex i i e 1x j >x



xj ≤ x

γx j , wjc

x < x j ≤ c.

(13)

This type of utility again induces a (n + 1)-player non-cooperative strategic form game. We begin with non-cooperative scenario where each CP chooses its quality factor to maximize its own utility function. The objective functions corresponding to this game have discontinuities. Further it is clear from the above utility that the effective domain of optimization is {0} ∪ (x, c], which is not connected. Thus as expected, the analysis is far more complicated. We observe (through numerical computations) that in many cases pure strategy NE does not even exist. We performed the best response analysis of utility of any typical player (details in [14]) and found that the best response is one among {0, x, c}, in most of the cases. Further since x is just at the border of virality (the post gets extinct for any x ≤ x and gets viral for any x > x), it may not be a right choice for practical purposes. Because of all these reasons, we continue further analysis with binary actions, i.e., the players choose a ∈ {0, c}. That is, the CPs either prepare the best quality post or do not even participate. NE with binary actions We first obtain the NE for this strategic form game. Lemma 5 This game can have multiple NE. The set of NE, N ∗ , is given by: ⎧ ⎫ n ⎨  ⎬ ∗ ∗ ∗  N := x 1{x j =c} = k , x j = 0 if w j < k γ and x j = c if w j ≥ (k + 1)γ ⎩ ⎭ ∗

j=0

⎧ ⎫ n ⎨ ⎬ with k ∗ := max 0 ≤ k ≤ n : 1{w j ≥kγ} ≥ k , k¯ + := 1{w j ≥(k ∗ +1)γ} and ⎩ ⎭ j=0

k¯ ∗ :=

j

j

 ¯∗ ¯+   k −k 1{w j ≥k ∗ γ} . Also N ∗  = N ∗ := ∗ ¯ + . k −k

(14)

140

R. Dhounchak et al.

In any NE the number of players with nonzero action equals k ∗ , the action of first k¯ + strong players is c and there are N ∗ number of NE for this game.  Thus we have multiple NE for this scenario. Our aim  is to compute the BoC (benefit of cooperation) and towards this we need UT∗ = j≥1 U ∗j , the total utility of C-players at an ‘appropriate’ NE. We consider the minimum total utility of C-players among all possible NE as the utility derived in the non-cooperative scenario. If the adversary is weak, i.e., if w0 < k ∗ γ, then at any NE x∗ ∈ N ∗ UT∗ (x∗ ) = 1 −

γx ∗j wjc

j

, and min∗ UT∗ (x) = 1 − x∈N

k¯ k¯ γ − wj ∗ ∗ +



j=1

j=k¯ −k +k¯ + +1

γ . wj

(15)

If the adversary is strong, i.e., if w0 ≥ k ∗ γ, the minimum total utility equals: min∗ UT∗ (x) =

x∈N

k¯ +



j=1

j=k¯ ∗ −k ∗ +k¯ + +2

k∗ − 1 γ − − k∗ wj



γ . wj

(16)

5.2 Cooperative N Players When the players decide to participate in a coalition, they make a combined post having content of all the participating players. The combined post gets liked by an user of OSN, if user likes the content of any one of them. Thus the probability that a combined post (of grand coalition) is liked by any user equals: "      ηag = 1 − 1 − w1 η1 1 − w2 η2 · · · 1 − wn−1 ηn−1 = 1 − (1 − xi /c).

(17)

1≤i≤n

In non-cooperative scenario, content of each CP starts with one copy (at start one user has content stored on its TL). Equivalently for any coalition C, we consider that the process starts with |C|-copies of the combined post. Thus the aggregated utility of C-players for this exponential-utility game equals (see (12), (13) and with x = (x1 , · · · , xn ) ∈ [0, c]n+1 ): Uag (x0 , x) =

1{xag ≤x} ne xag ne xag + e xad

⎤ ⎡ n "  x i ⎦ γxi ⎣ 1− − , with xag = c 1 − . wi c c i=1

1≤i≤n

and that of adversary is: Uad (x0 , x) =

1{xad ≤x} e xad xad −γ , with xad = cw0 ηad = x0 . ne xag + e xad w0 c

To Participate or Not in a Coalition in Adversarial Games

141

As before we consider two-player non-cooperative game with the above two as the players, and compute the NE, which we again refer as CNE. This two-player game is to study the benefit of cooperation when all C-players form a (grand) coalition. We begin with the following basic result which could also be used while studying the Shapley value. When more than one players participate in a coalition, optimizing the aggregate utility by only the strongest player’s strategy is better than (or as good as) that obtained by optimizing using the actions of all/some of the coalescing players. Basically, it states that the weaker players should remain silent (as in Lemma 2) and this is the best way to cooperate. Assume throughout w1 ≥ w2 · · · ≥ wn . Lemma 6 Say w1 = max j≥1 w j . For any x0 , x1n := (x1 , · · · , xn ) ∈ (0 ∪ (x, c])n  Uag (x0 , x1n ) ≤ Uag (x0 , x  , 0, 0, · · · , 0) with x  (x1n ) = min c, w1

n xi wi

' .

(18)

i=1

Hence for any given x0 , the best response of the aggregate player is obtained by: sup

(x 1 ,x 2 ,x 3 ··· ,x n )∈[x,c]n

=

sup

Uag (x0 , x1 , x2 · · · , xn−1 , xn )

Uag (x0 , x, 0, 0, · · · , 0) =

x∈{0}∪(x,c]

 sup x∈{0}∪(x,c]

ne x 1x>x γx − x x ne + 1{x0 >x} cw1 ¯ e 0

 .



Remarks: (1) As in (linear case) Lemma 2, the best response is dominated by best response when the weaker C-players remain silent. (2) However, the reduced game can’t be analysed as easily as in Lemma 2. Nevertheless, as in the linear case, a NE (x0∗ , x ∗ ) of following the reduced game (action profile of aggregate player has only one component) with utilities of the two players as  Uag (x0 , x) =

ne x

ne x 1x>x e x0 1x0 >x γx γx0 − , Uad (x0 , x) = x − , x x 0 + 1{x0 >x} cw1 ne + 1{x0 >x} cw0 ¯ e ¯ e 0

(19)

gives a CNE to the original game, (x0∗ , x ∗ , 0, · · · , 0). Once again, the original game can have more CNEs than those derived from the reduced game. For example, when w1 = w2 , if CNE derived from reduced game is (x0∗ , x ∗ , 0, · · · , 0) then (x0∗ , 0, x ∗ , 0, · · · , 0) is also a CNE (as in Lemma 2ii). However by virtue of the above Lemma, the utility under any CNE of the original game equals that at a corresponding CNE derived using the reduced game. (3) Once again the result is readily true for any sub-coalition, even when it includes adversary, as would be required for defining the TU game (which we might consider in future). Binary actions We again consider the case with binary actions. We have situations with multiple CNE and unlike in the linear case, we have (drastically) different utilities at different CNE, at (0, c) utility of aggregate player is 1 − γ/w1 and at (c, 0) it has 0 utility. Thus as in the concept of ‘security level’, we define the utility in cooperative scenario as the minimum possible utility at any CNE. With this definition (details are in [14]):

142

R. Dhounchak et al.

Fig. 6 OSN: Boc for exponential utilities

∗min Uag =

 1{w1 >γ} 1{w0



1 n +1

w1 >



1 n +1

 1 γ {w0 (n+1)γ}





 n γ . − n+1 w1

1−

γ w1



(20)

5.3 Benefit of Cooperation As in linear case, one can define the BoC Ψ = 200

∗min − U ∗min ) (Uag T ∗min + U ∗min ) (Uag T

which can be computed using (15) and (20). When w1 < γ, the utility in cooperative as well as non-cooperative scenario is ∗ zero and so is the BoC. When γ ≤ w1 < (1/n + 1)γ and w0 > γ then Uag = 0, ∗min ∗ ∗ = 0 as (c, 0, · · · 0) is one of the NE which while k = 1 and k¯ ≥ 2. Thus UT gives the minimum total utility for C-players, and hence BoC is again 0. Thus with k ∗ < 2, we have zero BoC cases, and hence, cooperation may not be beneficial. But otherwise, it is always beneficial to cooperate (proof in [14]): Lemma 7 If k ∗ ≥ 2, then BoC Ψ > 0, i.e., cooperation is always beneficial.



We consider three further sub-cases, with k ∗ ≥ 2, to study the exact extent of the benefit in [14] and using the expressions so derived we plot the BoC for some examples in Fig. 6. Important conclusions with exponential utilities are (more details in [14]): (a) BoC is never negative; and (b) BoC is positive once k ∗ > 1.

To Participate or Not in a Coalition in Adversarial Games

143

Conclusions We considered adversarial resource-sharing games and investigated the improvement possible with cooperative strategies. We considered linear as well as exponential user utilities. The message propagation process in a social network is well modelled by branching processes, and the related visibility performance results in exponential cost. The following are our findings: (a) there are (rare) scenarios in which cooperation is not beneficial, sometimes even with sufficient number of cooperating partners; (b) even when the adversary is weak, it may not always be beneficial to cooperate; (c) there are scenarios in which players benefit well, even when the adversary is strong; and (d) we have simple expressions which identify the favourable conditions for cooperation. The best cooperative strategy is that all but one (strongest) partner should remain silent (irrespective of the strengths of the rest). However this also reduces interference to the opponents, we might have scenarios in which grand coalition may not be the best. As a result, the allocation provided by Shapley values is not always fair. Most of the conclusions are similar for exponential utilities, the major differences are: (a) we do not have scenarios where selfish strategies are strictly better than cooperative strategies; (b) with more than two significant partners (even including the adversary), it is always beneficial to cooperate. This research has opened up many more questions, which require further investigation.

References 1. Narahari, Yadati. Game theory and mechanism design. World Scientific, 2014. 2. Gast, Nicolas, and Bruno Gaujal. Computing absorbing times via fluid approximations. Advances in Applied Probability 49.3 (2017): 768–790. 3. Rosen, J. Ben. Existence and uniqueness of equilibrium points for concave n-person games. Econometrica: Journal of the Econometric Society (1965): 520–534. 4. Cheng, Shih-Fen, et al. Notes on equilibria in symmetric games (2004): 71. 5. Xu, Yuedong, et al. Efficiency of Adversarial Timeline Competition in Online Social Networks. arXiv preprint arXiv:1602.01930 (2016). APA 6. Dhounchak, Ranbir, Veeraruna Kavitha, and Eitan Altman. A Viral Timeline Branching Process to Study a Social Network. Teletraffic Congress (ITC 29), 2017 29th International. Vol. 3. IEEE, 2017. 7. A. Vulimiri, G. Agha, P. Godfrey, and K. Lakshminarayanan, How well can congestion pricing neutralize denial of service attacks? in Proc. of ACM Sigmetrics 2012. 8. J. Lou, A. Smith, Y. Vorobeychik, Multidefender Security Games, in IEEE Intelligent Systems, vol. 32, no. 1, pp. 1541–1672, 2017. 9. W. Saad, Z. Han, M. Debbah, A. Hjorungnes, T. Basar, Coalitional game theory for communication networks, in IEEE Signal Processing Magazine, vol. 26, no. 5, 2009. 10. W. Saad, Z. Han, T. Basar, M. Debbah, A. Hjorungnes, Distributed Coalition Formation Games for Secure Wireless Transmission, in Mobile Networks and Applications, vol. 16, no. 2, 2011. 11. F. Kelly, A. Maulloo, D. Tan, Rate control for communication networks: shadow prices, proportional fairness and stability, The Journal of the Operational Research Society, Vol. 49, No. 3, 1998.

144

R. Dhounchak et al.

12. E. Altman, P. Kumar, and A. Kumar, Competition over timeline in social networks, in Proc. of Social Network Analysis in Applications Workshop (in conjunction with COMSNETS), 2013. 13. A. Reiffers, Y. Hayel, E. Altman, Game theory approach for modeling competition over visibility on social networks, in proceedings of IEEE COMSNETS, workshop on Social Networking, 2014. 14. Manuscript under preparation and the Technical report downloadable at http://www.ieor.iitb. ac.in/files/faculty/kavitha/ParticipateOrNot_TR.pdf

On the Asymptotic Content Routing Stretch in Network of Caches: Impact of Popularity Learning Boram Jin, Jiin Woo and Yung Yi

Abstract In this paper, we study the asymptotic average routing stretch for random content requests in a general network of caches. The key factor considered in our study is the need of learning content popularity in an online manner to consider time-varying changes of content popularity, where there exists a complex inter-play among (a) how long we should learn popularity, (b) how often we should change cached contents, and (c) how we use learnt popularity in caching contents over the network. We model this inter-play in a broad class of caching policies, called Repeated Learning and Placement (RLP), and aim at quantifying the asymptotic routing stretch of content requests under various external conditions. Our derivation of this scaling law in the routing stretch is made under different dependence of the speed of popularity change, average routing stretch in the network of caches, the shape of the popularity distribution, and heterogeneous cache budget allocation based on nodes’ geometric importance. We believe that our analytical results, even if they are asymptotic, provide additional ways and implications on understanding the delay performance of large-scale content distribution network (CDN) and informationcentric network (ICN). Keywords Cache networks · Popularity · Learning

1 Introduction Internet has increasingly become content-oriented and experienced the exponential traffic growth, where applications’ QoS requirement becomes more and more stringent and diverse. People constantly seek for ways of adapting the Internet to such B. Jin · J. Woo · Y. Yi (B) School of Electrical Engineering, KAIST, Daejeon, South Korea e-mail: [email protected] B. Jin e-mail: [email protected] J. Woo e-mail: [email protected] © Springer Nature Switzerland AG 2019 J. Walrand et al. (eds.), Network Games, Control, and Optimization, Static & Dynamic Game Theory: Foundations & Applications, https://doi.org/10.1007/978-3-030-10880-9_9

145

146

B. Jin et al.

a trend. In addition to the simple effort of providing wider network pipes, it is of significant importance to build a network more content-friendly, e.g., enhancing existing content distribution network (CDN) technologies as an evolutionary approach, or proposing revolutionary architectures such as information-centric networking (ICN) and content-centric networking (CCN); see, e.g., [13, 16]. In a content-oriented architecture (whether it is evolutionary or revolutionary), content caching seems to be a crucial component to reduce delay of fetching contents (from users’ perspective) and/or to cut down overall traffic transport cost (from providers’ perspective), often forming a group of large-scale caches (e.g., servers, access points, or devices), namely a cache network. In this paper, we aim at analytically understanding a sort of fundamental limit on how long it takes for a content requester to fetch the content when caches are connected as a network. Analyzing a network of caches is known to be a daunting task in general, since there exists a complex inter-play among the underlying (i) time-varying content popularity, (ii) content request routing, and (iii) dynamic content replacement policy. In particular, a dynamic content replacement policy is the key mechanism that plays a role of learning the popularity of contents and adaptively reconfiguring the contents in caches to the changes of active contents, where popular examples include LFU, LRU, and their variants (e.g., k-LRU and LRFU). However, analytically studying such policies even just for a single cache is known to be challenging [9, 10, 14], and thus, a network of caches with the replacement policies is significantly challenging to analyze. To achieve our goal, despite a large degree of theoretical challenges of a network of caches, we model a network of caches under the random dynamics of content arrivals and asymptotically study the routing stretch that refers to the number of hops until a requested content is served to its requester when the size of networked caches and the number of contents scales. The metric of routing stretch in the network of caches can be used as a good approximation of the average delay in accessing the contents, provided the network load is stable so that queueing delay at a cache is regarded as an averaged constant. Contributions. First, to provide a wide spectrum of caching strategies to the centralized planner, we first propose a highly broad class of policies, called RLP (Repeated Learning and Content Placement), that can contain a variety of options on popularity learning and content placement strategies. Then, we compute a lower bound on the routing stretch by studying an ideal policy—Oracle—that is assumed to (i) magically obtain the knowledge on the content popularity distribution for free and (ii) (unrealistically) place contents from a requester to its original server in the decreasing order of popularity. We prove that Oracle is always better than any other policy P in the RLP class in the sense of the routing stretch, and characterize the asymptotic routing stretch performance of Oracle for different types of popularity distributions. Second, we develop a policy in the RLP class, called RLP-TC (RLP with Tilting and Cutting), that is near optimal; i.e., its routing stretch is very close to that of Oracle, where by “very close” we mean that RLP-TC achieves the same scaling of routing stretch as Oracle except for a very restricted case. The smartness of RLP-TC lies in the idea of tilting and cutting, where tilting refers to

On the Asymptotic Content Routing Stretch in Network of Caches …

147

the mechanism that modifies the learnt empirical popularity distribution into a less biased, tilted form and use this modified distribution to place contents in the caches, but we render unpopular contents uncached, i.e., cutting. We asymptotically analyze the routing stretch of RLP-TC. Related work. Analyzing cache performance started from a single-cache case in the area of computer architecture and operating system [7, 14], where the main focus was on deriving asymptotic or approximate closed form of cache hit probabilities for well-known cache policies such as LRU, LFU, and FIFO, often on the assumption of Independence Reference Model (IRM). Recently, a technique called Che’s approximation [5, 9] has been applied to a simple setup, being extended to a network of caches [10, 19]. Due to the analytic hardness of general topology for a network of caches, there exists work with topological restriction. Examples include the cascade [3, 18] and tree topologies [3, 20]. A few recent works started to consider general topologies [23, 24], where in [23] an algorithm called a-NET is proposed to approximate the behavior of multi-cache networks, and in [24], when a steadystate characterization of cache networks is possible. There exists related work on asymptotic analysis of cache networks with the emphasis on throughput [2, 17] and capacity [21, 22]. In [2, 17], a dynamic content change at caches was modeled by abstracting the cache dynamics with a limited lifetime of cached content. Our work is partially inspired by [17], but we consider a more general network of caches with popularity learning.

2 Model and Problem Statement Network. We consider a sequence of graph G(n) = (V(n), E(n)), where V(n) is the set of nodes or caches with |V(n)| = n, where n is the system scale size in our asymptotic study, and E(n) ⊂ V(n) × V(n) describes direct connectivity between caches. Let dmax (n) be the maximum distance between two nodes. We let C(n) be the set of entire contents, and for each content c ∈ C(n), there exists its associated repository (or simply called server) that originally and permanently contains c and thus are finally accessed if a content is not fetched from an intermediate cache. Denote sc be the content c’s repository. We assume that contents are of equal size and each content c ∈ C is stored in a single server,1 say sc , and each server sc ∈ S is attached to a node vc := vsc ∈ V. Let S(n) be the set of such original content servers. Each content server is located uniformly at random in V. Each node v ∈ V can cache a set of contents (thus, node and cache can be interchangeably used throughout this paper), having the cache size bv (n) ≥ 0 with the network-wide cache budget B(n). We assume that each cache size is equivalent across nodes; i.e., b(n) = bv (n) = B(n) n for all v ∈ V. 1 This assumption does not restrict our results, because even when per-content multiple repositories

exist, our asymptotic results hold as long as the number of repositories is Θ(1).

148

B. Jin et al.

Content requests, routing, and popularity. Exogenous content requests are generated at each node, which are homogeneous and independent across nodes, following a simple counting process that satisfies: The average request rate of content ci ∈ C = {c1 , c2 , . . .} is proportional to a Zipf-like distribution with parameter α > 0: pi = K /i α ,

(1)

|C| where the normalizing constant K is such that K1 = i=1 1/i α ; i.e., large values of α imply higher content popularity bias. We use the notation rank(c) to refer to the popularity ranking of content c. We abuse the notation pc := prank(c) to refer to the popularity distribution of content c. When a request for a content c ∈ C is generated at node v, it is forwarded along the path given by some routing algorithm [11, 12], e.g., the shortest path routing in G, from v to the server vc . Let dA be the average distance to from each node generating a request of content c and c’s repository, which we call average to-server-distance, where the average is taken over all pairs of request-generating node and the request’s repository. In the nodes consisting of the routing path from v to vc , the request generates HIT at w in the path if c is cached at w, and w is the first cache containing c in the path, and the content c is fetched to v via the reverse path from w to v. If no cache in the routing path has the requested content, it is MISS, in which case the content c is fetched from the original server vc to v. Popularity: Distribution and change. In terms of time-varying population changes, we consider the so-called block change model, as in [17]. In this model, during each time block, content popularity remains constant over a given T (n) content requests in the entire network of size n and then changes to some other arbitrary distribution still following Zipf-like distribution in (1), but with possibly different α, so a new time block is assumed to start. Due to our intention of asymptotic approaches, our interest lies in the order of T (n). In this block change model, it suffices to study our target performance metric only within the time window [0, T (n)] in a single block, where the performance will be determined by a caching policy, i.e., how and how long we learn content popularity and where we place the cached contents. Cache versus content size. We focus on the large content with small cache size regime that for any given content request, the number of contents are significantly larger than the entire cache size budget B(n), formally b(n) × dmax (n) = o(|C(n)|) and b(n) = Θ(1), where b(n) × dmax (n) corresponds to an upper bound of the total possible amount of cache storage for an individual content in the entire network. Studying this regime seems quite valuable, considering the recent trend of a highly growing number of contents with the aim of reducing the content access delay at small cost of operating caches. For notational simplicity, we will drop the subscript n for all quantities that depend on n, unless confusion arises. Performance metric: Average routing stretch. Our primary performance metric is the response delay till a content request is fetched and served. As a useful approximation of the delay, we use the (content) routing stretch, defined as the number of (expected)

On the Asymptotic Content Routing Stretch in Network of Caches …

149

hops until it finds the desired content; i.e., HIT occurs. Formally, let random variable X i be the routing stretch of the i-th content request (in the entire system). Then, the average routing stretch Δ of the cache network is defined as follows: T (n)

Δ  E [D] ,

1  Xi D := T (n) i=1

(2)

which depends on the given system setups G, α, and a routing policy, as well as our controlled caching policy. Our interest is on the asymptotic characterization of Δ for large n.

3 Centralized Popularity Learning and Content Replacement Policies 3.1 Oracle Policy This policy is the one that is assumed to obtain the true popularity statistics [ pc : c ∈ C] for free, and its algorithm is described in what follows: Oracle policy S1. Whenever any new request for a content c arrives at a node, say v, a routing path from v to c’s original server vc is ready from a given routing algorithm. Let such a routing path be a sequence of cache nodes Pv,c = (v1 , v2 , . . . , vc ). S2. Then, the contents are magically placed in the sequence of nodes from v1 to vc , with the decreasing order of content popularity (which is given for free), where each node w in the routing path can cache the b = B/n number of contents for the corresponding request. To illustrate, we consider the example in Fig. 1, where suppose that b = 2; i.e., each node can cache 2 contents. Assume that we have a new content request generated at v1 , the content’s original server is at v4 , and its priori given routing path is (v1 , v2 , v3 , v4 ). Then, we cache the contents from v1 to v4 in the decreasing order of popularity, two contents at each node in the routing path, thus total 8 contents in the path. Note that Oracle is unrealistic due to the following reasons: In addition to magically given knowledge on the true popularity statistics, there may be the case when a cache should store the contents beyond its given cache size. For example, as illustrated in Fig. 1, for a request generated at v5 whose routing path to the original server is (v5 , v3 , v6 ), v3 should store the contents c3 , c4 , whereas v3 is supposed to store the contents c5 , c6 for the request by v1 . We allow this violation of cache size limit in Oracle, because we plan to use Oracle as a policy providing a lower bound of the routing stretch.

150

B. Jin et al.

Fig. 1 Example of content placement in Oracle: ci is the i-th popular content in its ranking

Request

Request

3.2 RLP Class and RLP-TC (Tilting and Cutting) We now consider a class of policies, called Repeated Learning and Placement (RLP), where a policy in the RLP class has repeated steps for popularity learning and content placement. We claim that the RLP class is highly general so as to include any possible policies that mix popularity learning and configuring cache contents in a centralized manner. We now elaborate on the class of RLP policies. An RLP policy, RLP(a, m, P), is parameterized by (i) the number of repetition steps m, (ii) m-dimensional vector a(m) = [ai : i = 0, 1, . . . , m], where [ai : i = 0, 1, . . . , m] defines the m + 1 sequential temporal partitions from the first to T (n) requests, and (iii) content placement a pictorial description of an RLP(a, m, P). Note that k−1 m strategy P. See Fig. 2 for i=0 ai = T (n). Let L k = i=0 ai , i.e., the aggregate number of requests until the partition ak−1 . Then, the partition ak turns out to be the number of requests from when the system receives (L k + 1)-th request to L k+1 -th request. The basic idea of the RLP(a, m, P) is that at each partition ak , we first learn and estimate the content popularity distribution using the L k requests, and use the learnt popularity in the content placement strategy P. Examples of content placement strategy P include a random strategy of simply placing contents uniformly at random and a popularityproportional strategy where the probability that a content is placed in a cache is proportional to the (learnt) popularity. RLP(a, m, P) is formally described by the following recursive procedures that specify what have to be done at the start of each partition ak , k = 0, 1, . . .:

Fig. 2 Framework of online caching policy RLP(a, m, P )

On the Asymptotic Content Routing Stretch in Network of Caches …

151

RLP(a, m, P) At partition a0 : Contents are placed uniformly at random at each cache of size b = B/n. At partition ak : • Popularity learning phase: The system learns the popularity distribution [lc (k) : c ∈ C] by computing the following empirical distribution: L k lc (k) =

j=1

Lk

j

Yc

,

j

where Yc = 1 if j th request is for content c, and 0 otherwise. • Content placement phase: Then, the content placement strategy P is applied at all caches based on the learnt popularity distribution [lc (k) : c ∈ C], and new requests over the partition ak are served.

Note that at partition a0 , it is natural to employ a uniformly random policy because there is no knowledge about popularity obtained in the past. One of the trivial policies belonging to the RLP class is RLP(a, 0, RANDOM) corresponding to the policy that caches the contents uniformly at random in the network without any learning of content popularity. Our goal is to achieve short routing stretch by intelligently choosing m, a(m), and P. We now propose a policy in the RLP class, called RLP-TC (RLP with tilted popularity distribution with cutting. Since RLP-TC is a policy in the RLP class, its unique feature is characterized by (i) content placement strategy P and (ii) a construction of a(m) (see Sect. 3.2). As a content placement strategy P, we remanufacture the learnt empirical distribution into a “tilted distribution” and use it for content placement, which we call TC (Tilted learnt popularity with Cutting). Regimes: Speed of popularity change. Prior to describing RLP-TC, we first describe three different regimes with respect to how fast content popularity changes: Fast, Normal, and Slow, which, in practice, may differ depending on the content categories [25]. This classification is used to describe the caching policies and present our analytical results and their interpretations. We define three regimes of T (n) as follows: ⎧ ⎪ if Θ(log dA ) ≤ T (n) ≤ Θ(dA log dA ), ⎨FAST (3) NORMAL if Θ(dA log dA ) < T (n) < Θ(dA2 log dA · M 2 ), ⎪ ⎩ SLOW if Θ(dA2 log dA · M 2 ) ≤ T (n), where recall dA is the average to-server-distance, and M = M(α) is such that:

152

B. Jin et al.

⎧ ⎪ if α > 2, dA ⎨Θ(1)  1 1 = Θ(log dA ) if α = 2, = ⎪ M i α/2 ⎩ 1−α/2 i=1 ) if 2 > α > 1. Θ(dA

(4)

Note that our classification differs in NORMAL and SLOW regimes depending on M, which also relies on the popularity bias parameter α. Policy description. We first describe and explain RLP-TC policy, followed by its rationale. RLP-TC = RLP(a, m, TC) policy INPUT: T (n) and α. • Construction of m and a: We first choose a0 with its dependence on T (n) as follows:  T (n) if FAST, a0 = (5) Θ(log dA ) if NORMAL and SLOW, Then, for Θ(dA log dA ) < T (n), choose a1 = max{o(a0 ), Θ(1)}, and select m:

(1 − r )(T (n) − a0 ) , m = logr 1 − a1

(6)

where r =1−

a1 (< 1). T (n)

(7)

Then, the remaining sequence (a2 , a3 , . . . , am ) is constructed by the geometric series, starting from a1 , with the common ratio r ; i.e., ak = a1r k−1 , k = 1, . . . , m. • TC strategy: In the general RLP policy description, at each step ak , we apply the following procedures to the content placement phase: S1. Construction of tilted popularity distribution. Using the empirical distribution [lc (k) : c ∈ C], we compute the following tilted distribution [lˆc (k) : c ∈ C]:  lˆc =

√ lc √ c=1 lc

iˆ

ˆ if rank(c) ≤ i,

0

otherwise,

(8)

where iˆ = b · dA and recall that rank(c) is the popularity ranking of content c. S2. At each cache, b (= B/n) contents are randomly selected according to the distribution [lˆc (k) : c ∈ C] without duplication of contents.

On the Asymptotic Content Routing Stretch in Network of Caches …

153

Policy explanation. We present the rationale of RLP-TC. First, the constructed a and m depend on a given T (n) and the popularity bias α, where a0 ’s dependence is of significant importance. As observed in (5), the length of a0 decreases in NORMAL and SLOW rather than FAST, because for larger T (n), more chances to learn the popularity are allowed, whereas for smaller T (n), the initial learning becomes more crucial. In NORMAL and SLOW, the remaining steps (a1 , a2 , . . . , am ) as well as the total number of iterations m are chosen as a geometric series such that their sum equals to T (n), as seen in (6) and (7). Second, as a content placement strategy, the proposed TC strategy first re-manufactures the learnt popularity distribution into a tilted form with cutting, as in (8) at each step k. It means that we nullify the distribution of unpopular contents, where we maintain the popularity only up to the ranking index iˆ = b · d√A , and re-normalize the distribution (more specifically taking the square root, i.e., lc , as seen in (8)). Then, we randomly place the contents according to the computed tilted distribution (S2). To intuitively understand, take an example of three contents, where the originally learnt distribution in step k is (lc (k)) = (0.7, 0.2, 0.1) with iˆ = 2. Then, in our tilted distribution with cutting, we have (lˆc (k)) = (0.65, 0.35, 0). We will explain how this helps in achieving short routing stretch next. Rationale of tilting with cutting. A natural way is to directly use the empirically learnt popularity distribution with which contents are randomly placed at ˆ each cache. √ However, what we do is to use a tilted distribution [lc (k)], such that [lˆc (k) ∝ lc (k) : c ∈ C]. The key effects of tilting with cutting are summarized in what follows: (i) tilting: making the popularity distribution less biased and (ii) cutting: making unpopular contents uncached. If we consider only a single, stand-alone cache, just the empirical distribution-based placement might be enough. However, in the network of caches, the performance of our interest, which is a routing stretch, is a non-trivial complex function of coupled behaviors among the caches in a given routing path. This non-trivial relationship requires us to re-consider the obtained empirical popularity distribution and also effectively use the available rooms for caching whose size is strictly smaller than the number of contents. This motivation leads us to tilt the empirical distribution with unpopular contents excluded from caches. Why √ square root in tilting? The remaining question in tilting, is why the choice of lc (k) is made for the obtained empirical distribution lc (k)? Just for simplicity of exposition, assume b = 1; i.e., each node can cache only one content, and also the to-server-distance dA is large. We now consider a cache placement strategy under which content ci (whose popularity distribution is pi ) is cached in each cache with probability qi . For large T (n), the expected routing stretch Δ for dA roughly becomes: Δ=

|C|  i=1

|C|

pi ·

|C|

|C|

 1   1 2 1 = pi qi ≥ pi 2 , qi qi i=1 i=1 i=1

154

B. Jin et al.

where 1/qi is the average stretch from a requester to the cached node of content ci , and the last inequality comes from the Cauchy–Schwarz inequality. In Cauchy– Schwarz inequality, it is widely known that the equality holds if and only if there is some constant k such that pi q1i = k · qi for all i. Therefore, Δ is minimized when √ |C| α 1 2 2 qi ∝ i − 2 , and the minimum value is . This is why l i is selected for i=1 pi TC where li goes to pi for sufficiently large T (n). Note that a special case when qi = pi corresponds to the case utilizing content popularity distribution directly for the cache placement, and Δ = |C|.

4 Analysis: Routing Stretch We now present our main results on the average routing stretch Δ, as addressed in (2), for Oracle and RLP-TC, where all the proofs are presented in our technical report [15]. To make our analysis of Δ tractable, we first rewrite Δ as the average over a random to-server-distance dtsd from a content requester to the corresponding content server (where the randomness comes from the location of content requesters and the corresponding content servers) as follows: Δ = E[Δ(dtsd )] =



f tsd (d)Δ(d)

(9)

d

where, to abuse the notation, Δ(d) be the “expected” routing stretch when to-serverdistance is d, where the expectation is taken with respect to the randomness in the contents and content caching policy, and f tsd (d) is the distribution of d.2 Note that having a closed form f tsd (d) is challenging and thus makes the routing-stretch analysis hard, because f tsd (d) depends on the given topology G and the underlying routing algorithm. For example, even for a Erdös–Rényi (ER) random graph, which is one of the simplest random graphs, when the shortest path routing algorithm is used, f tsd (d) is still unknown. Thus, to purely focus on our interest, we use Jensen’s inequality and obtain:   

Δ = E Δ(dtsd ) ≤ Δ E[dtsd ] = Δ(dA ),

(10)

where recall that dA is the average to-server-distance, and we now consider Δ(dA ) as our major metric to analyze. To differentiate from Δ(d) for any given d, we often call Δ(dA ) average stretch upper bound, or simply stretch upper bound.

2d

is also a random variable since a chosen content is also random.

On the Asymptotic Content Routing Stretch in Network of Caches …

155

4.1 Oracle This is formally presented in Theorem 1 which states that it is optimal in the sense that Oracle has shorter routing stretch than any other policy in the RLP class, and in Theorem 2 which presents the asymptotic average routing stretch of Oracle. Theorem 1 Let D O and D A be the random routing stretches (as defined in (2)) of Oracle and an arbitrary policy A in the RLP class, respectively. Then, the following  stochastic dominance of Oracle holds: D O ≤st D A , which means P D O > x ≤

 P D A > x , f orany x ≥ 0. Theorem 2 For a given to-server-distance d between a pair of a content requester and its server, the average routing stretch Δ(d) and the stretch upper bound Δ(dA ) of Oracle scale as those in Table 1. Thanks to Theorem 1, the result of asymptotic routing stretches in Table 1 provides lower bounds of Δ(d) of any policy in the RLP class. As expected, as α decreases, we lose the power of caching, thus leading to the increase of routing stretch. When α > 2, the stretch is order-wise optimal (i.e., a constant order), and only up to α = 2, the stretch is sub-polynomial. Note that when 0 < α ≤ 1, the caching gain vanishes, so requiring to reach the corresponding original server. We now seek to find a policy in the RLP class whose performance is close to that of Oracle, if any.

4.2 RLP-TC We now present the result on the stretch bound Δ(dA ) of RLP-TC in Theorem 3. We will focus only on the case when α > 1, because even the lower bound provided by Oracle proves that there is no caching gain for α ≤ 1, and see Theorem 1. Theorem 3 For α > 1, the upper bound of routing stretch Δ(dA ) for RLP-TC scales as follows: With high probability, ⎧ ⎪ ⎪ ⎨Θ(d  if FAST,  A) 2 log dA Δ(dA ) = Θ dA T (n) if NORMAL, ⎪ ⎪ ⎩Θ( 1 ) if SLOW. M2

(11)

Note that the performance of RLP-TC depends on T (n), as expected. In FAST, Δ(dA ) = Θ(dA ); i.e., there is no gain due to the lack of time to learn and apply such a learning result to the content placement. In NORMAL, Δ(dA ) decreases as T (n) increases, because the repeated learning process helps where we have enough time to learn the popularity and use such knowledge in placing contents, until Δ(dA ) reaches Θ(1/M 2 ) in (11) at the threshold T (n) = Θ(dA2 log dA · M 2 ) in (3). After this

156

B. Jin et al.

Table 1 Routing stretch: Oracle

Popularity

Δ(d)

Δ(dA )

20} c j ,

(1)

where n j is the number of SAs acquiring information from the SA and 1{n j >0} is the indicator function of the event {n j > 0}.5 In the first stage, each ESC selects the price to the SAs.

4 Our

analysis can easily be extended to the case where instead ESC’s A and B make independent errors. 5 Note this models a situation where the cost c is incurred primarily for operating an ESC’s sensor j network and there are no additional costs when this is used to inform multiple SAs. The model and analysis can easily be extended to the case where there is an additional marginal cost per SA.

Tiered Spectrum Measurement Markets for Licensed Secondary Spectrum

169

2.2 SAs Decisions Each SA must make two decisions. First, it must decide whether to acquire information from ESC A, ESC B, or to not acquire any information at all. Second, if SA i acquires information, then it must decide on a price pi that it will charge users for its service. We assume that these decisions are made in stages. Each SA i seeks to maximize its profit given by (2) πi = pi λi − p˜ k where λi indicates the number of users SA i serves and p˜ k is the price it pays to acquire information from ESC k. If SA i decides not to acquire information in the first stage, then we set p˜ k = 0 and λi = 0 so that the overall profit is also zero; i.e., this models a case where SA i decides not to enter the market. The above may occur when the revenue the SA i would generate is not sufficient enough to recover the cost of acquiring information from one of the ESCs.

2.3 User’s Subscription Model We consider a mass Λ of non-atomic users, so that we have λ1 + λ2 ≤ Λ. We assume that each user obtains a value v for getting service from either SA. However, users also incur a cost for using the service, which as in [6–9] is given by the sum of the price charged to them by the SA and a congestion cost they incur when using this service. The congestion cost models the degradation in service due to congestion of network resources. Since each SA has a licensed band, the users of an SA will only face congestion from the other subscribers of that SA. We model the congestion cost by g(x/W ), where x is the total mass of users using that band and g is a convex, increasing function. Hence, the ex-post payoff of a user receiving service from SA i is given by v − pi − g(x/W ).

(3)

The dependence of g on the bandwidth W models the fact that a larger band of spectrum is able to support more users. The mass of users, x, using the band depends in turn on the licensing policy and the information available to the SAs. The SAs knowledge of spectrum availability in turn depends on the information they acquire from the ESCs. In particular, if SA i obtains information from ESC k and has λi users, these users are only able to use the spectrum when the ESC k reports the spectrum is available (which occurs with probability qk ). When users cannot use the spectrum, we assume their payoff is zero. When users can use the spectrum, they receive a payoff as in (3). Hence, the payoff obtained will be a random variable.

170

A. Ghosh et al.

We assume that users seek to maximize the expected value of this quantity.6 Thus, if SA i obtains information from ESC j ∈ {A, B}, the expected payoff of any subscriber of SA i is q j v − q j g((λi )/W ) − pi .

(4)

Furthermore, users can choose not to purchase service from either SA, giving them a payoff of zero. To facilitate our analysis, we make the following assumption regarding the congestion costs: Assumption 1 Assume that g(·) is linear, i.e., g(x/W ) = x/W . Throughout the rest of this paper, Assumption 1 will be enforced.

2.4 Multistage Market Equilibrium We model the overall setting as a game with the ESCs, the SAs, and the users as the players. Each ESC’s payoff is its profit (cf. (1)). Each SA’s payoff in this game is its profit (cf. (2)), while each user’s objective is the expected payoff described in (4). This game consists of the following stages: 1. In the first stage, ESC j ∈ {A, B} selects the price p˜ j . 2. In the second stage, each SA selects one of the ESCs and pays p˜ j j = A, B or selects to stay out of the market. 3. In the third stage, SA i selects its price pi knowing the decisions made in stage 1. 4. In the last stage, given the first two stages’ decisions, the subscribers will choose one of the SAs from which to receive serve or choose not to receive service. We refer to the sub-game perfect Nash equilibrium of this game as a market equilibrium.

3 Market Equilibrium Characterization We now turn to our main results which are a characterization of the market equilibrium. We proceed via backward induction, starting next with characterizing the user equilibrium in the last stage. Subsequently, we characterize the equilibrium prices of the SAs in the third stage. We, then, characterize the SAs’ ESC selection strategy. Finally, we characterize the equilibrium strategy of ESCs in the first stage.

6 For

example, this is reasonable when users are purchasing service contracts with a long enough duration so that they see many realizations of the ESC reports.

Tiered Spectrum Measurement Markets for Licensed Secondary Spectrum

171

3.1 User Equilibrium Given the prices selected by the SAs and their ESC choices, the user equilibrium can be characterized as a Wardrop equilibrium as in [4]. Namely in such an equilibrium, the expected payoffs of subscribers in any band that is utilized must be equal and the expected payoff in any unused band must be lower than that in any used band. We next use this characterization to specify the user equilibrium under different ESC selection choices. First, we consider the case where both SAs obtain information from the same ESC. Theorem 1 Assume that both SAs obtain information from ESC j ( j ∈ {1, 2}) and that for each SA i, pi ≤ q j v, 1. If p1 = p2 and q j v − q j Λ/(2W ) − p1 ≥ 0, then λ1 = λ2 = Λ/2. 2. If p1 = p2 and q j v − q j α/W − p1 = 0 for some α < Λ/2, then λ1 = λ2 = α. 3. If pi = pk (for i = k), then the unique equilibrium is q j v − q j λ1 /W − pk = q j v − q j λ2 /W − pi , where 0 ≤ λ1 ≤ Λ, and 0 ≤ λ2 < λ1 . If the condition pi ≤ q j v is not satisfied, then it can be seen that in equilibrium SA i will never attract any customers and so the resulting user equilibrium is the same as if that SA did not enter the market. When this condition is strictly satisfied, that SA will always be able to attract some customers in equilibrium. Note also that if p1 = p2 , then unlike with unlicensed spectrum in [4], the user equilibrium is unique and equally divided. If one of the SAs sets a higher price, i.e., pi > p j , unlike the unlicensed case, both the SAs may have a nonzero user base. If the total number of customers served is less than Λ, then fixing the prices, the market coverage (given by the parameter α) will be higher if the SAs obtain information from ESC A rather from ESC B. Next, consider the monopolistic scenario in which one SA obtains information from an ESC and the other does not. Theorem 2 If the SA i ∈ {1, 2} obtains information from ESC j ∈ {A, B}, and the other SAs do not obtain information from the ESC, then the unique user equilibrium is λi = max{0, min{W (v − pi /q j ), Λ}}

(5)

If the bandwidth is higher, the number of subscribers is higher. Finally, we consider the scenario when SAs 1 and 2 obtain information from different ESCs. Without loss of generality, we assume that SA 1 (2) obtains information from ESC A (B). Theorem 3 Assume SA 1 (2) obtains information from ESC A (B), the unique user equilibrium (λ1 , λ2 ) satisfies: 1. λ1 = λ2 = 0, if q A v − p1 < 0, and q B v − p2 < 0. 2. λ1 = min{α, Λ}, and λ2 = 0 if q A v − q A λ1 /W − p1 ≥ q B v − p2 where α satisfies q A v − q A α/W − p1 = 0.

172

A. Ghosh et al.

3. λ2 = min{α, Λ}, and λ1 = 0, if q B v − q B λ2 /W − p2 ≥ q A v − p1 where α satisfies q A v − q A 2α/W − p1 = 0. 4. λ1 > 0, λ2 > 0 such that q A v − q A 2λ1 /W − p1 = q B v − q B 2λ2 /W − p2 ; and q A v − q A 2λ1 /W − p1 ≥ 0, and λ1 + λ2 ≤ Λ. In the first case in Theorem 3, users do not subscribe to any of the SAs. Similar to the discussion following Theorem 1, this is because their prices are too high to attract any customers. In the second case, the subscribers only subscribe to SA 1 as the expected payoff attained by the users is positive for SA 1 and strictly greater than that of SA 2 even when SA 2 has no congestion. Case 3 is the corresponding result when only SA 2 serves customers. In Case 4, both SAs serve the market. However, they may or may not serve the entire market. This will depend on the prices p1 , p2 and the probabilities q A , q B and the valuation v. The split of the market is unique. In this unique split, as the quality of information from ESC A (i.e., q A ) increases, the market share of SA 2 will decrease. On the other hand, if pi increases, the market share of SA i will decrease.

3.2 Price Equilibrium Next, we turn to the third stage. Recall, in this stage, given the ESC choices, each SA i selects its service price pi to maximize the revenue pi λi . When doing this, λi will be specified by the corresponding user equilibrium determined in the previous section, which in turn depends on if the SAs obtain information from the same ESC or a different ESC, or if one SA does not obtain information. We treat each of these cases separately. Both SAs obtain information from the same ESC. First, we describe the equilibrium pricing strategy when both the SAs obtain information from the same ESC. We further divide this into three cases depending on the relationship of v, Λ, and W . Theorem 4 If both SAs obtain information from ESC j and if v ≥ 3Λ/(2W ), the third-stage pricing strategy is pi∗ = q j Λ/W

(6)

for i = 1, 2. The last-stage user equilibrium is λ1 = λ2 = Λ/2.

(7)

q j Λ2 /(2W ) − p˜ j .

(8)

The profits of the SAs are

Note that the traffic is equally split among the SAs as the prices selected by the SAs are the same. Also, note that the price is higher if the SAs obtain information from

Tiered Spectrum Measurement Markets for Licensed Secondary Spectrum

173

the ESC A rather than B. Even though q A Λ2 /2 > q B Λ2 /2, p˜ A may be higher than p˜ B . Thus, it is not clear whether the SAs will attain a higher payoff if they obtain information from ESC A in the first stage. The user’s surplus is strictly positive in this scenario. Note that the condition v ≥ 3Λ/(2W ) is more likely to be satisfied when W is large. However, from (8) the profits of the SAs decrease as W increases. Thus, the profits may be negative for large enough W . Later, we will show that in the equilibrium path, this is not sustainable when W is very large. Thus, as in the unlicensed case [4], there may not be any competition when W is large. The profits decrease as W increases due to the fact that the prices also decrease with the increase in W . Intuitively, as W increases, the market becomes more competitive which drives down the prices, eventually leading to an SA leaving the market. Theorem 5 If both SAs obtain information from ESC j and Λ/W ≤ v ≤ 3Λ/(2W ), the third-stage pricing strategy is pi∗ = q j (v − Λ/(2W ))

(9)

for i = 1, 2. The last-stage user equilibrium is λ1 = λ2 = Λ/2.

(10)

q j (v − Λ/(2W ))Λ/2 − p˜ j .

(11)

The payoffs of the SAs are

Similar to Theorem 4, in Theorem 5, each SA covers the half of the user base. However, unlike Theorem 4, the user’s surplus is 0. The price is higher if the SAs obtain information from ESC A rather than B. However, the payoff will again depend on p˜ j . Note that when W is large, 3Λ/W is very small. Thus, the upper bound on v stated in this theorem is less likely to be satisfied. Similarly, when W is small, 2Λ/W is large, making the lower bound on v less likely to hold. Also note that the prices and profits of the SAs increase as W increases unlike Theorem 4. Finally, we show the equilibrium when v ≤ Λ/W . Theorem 6 If both SAs obtain information from ESC j and v ≤ Λ/W , the secondstage pricing strategy is pi∗ = q j v/2

(12)

for i = 1, 2. The last-stage user equilibrium is λ1 = λ2 = W v/2.

(13)

174

A. Ghosh et al.

The profits of the SAs are W q j v 2 /4 − p˜ j .

(14)

Similar to Theorems 4 and 5, in Theorem 6, the traffic is equally split. However, it does not cover the whole user base unlike Theorems 4 and 5. The user surplus is again 0 similar to Theorem 5 but unlike Theorem 4. The price is higher if the SAs obtain information from the ESC A rather than B but the payoff will again depend on p˜ j . As W increases, the profits of SAs increase since the demand increases with W . However, the prices are independent of W . Note that when W is small, Λ/W is large, making the bound on v hold for a larger range of v. However, when W is small, the profits may be negative, which will not be sustainable on the equilibrium path. Monopoly scenario. Next, we consider the scenario where only one of the SAs obtains information from an ESC and so is essentially a monopolist when making its pricing decision. Theorem 7 If SA i obtains information from the ESC j, while SA k = i does not obtain information from either ESC, the unique equilibrium price for SA i is pi∗ = max{q j v − q j Λ/W,

qjv }. 2

(15)

The last-stage user equilibrium is λi∗ = W min{v −

pi∗ , Λ/W }. qj

(16)

The monopolistic profit of the SAS i is πi = pi∗ λi∗ − p˜ j .

(17)

The monopolistic profit in (17) can also be written as r M, j − p˜ j , where  r M, j =

W q j v 2 /4, if v/2 ≤ Λ/W, q j (v − Λ/W )Λ, otherwise.

Note that though the first term in the expression of πi is higher for j = A as q A > q B , this does not necessarily mean that SA i will get a higher profit if it attains information from ESC A. This is because the price paid by the SA to obtain information from ESC A may be higher than that paid by the other SAs, i.e., p˜ A > p˜ B . Clearly, if p˜ A ≤ p˜ B , the profit attained by the SA will be higher if it selects ESC A. From (15), the price selected by SA i will be higher if it obtains information from ESC A. However, from (16), the market share (λi ) is independent of the ESC selected by the SA. Thus, surprisingly, in a monopolistic scenario the number of users which receive service is independent of the choice of ESC made by the SA.

Tiered Spectrum Measurement Markets for Licensed Secondary Spectrum

175

Next, consider how the bandwidth W impacts the results in Theorem 7. As W increases, the profit of the SA increases. However, the rate of increase decreases when v > 2Λ/W . This is because when v > 2Λ/W , the monopoly SA serves the whole market. Thus, the demand cannot increase beyond that point, though a larger W can still enable the SA to increase its price. If v < 2Λ/W , the SA does not serve the whole market due to the high congestion cost. SAs obtain information from different ESCs. Finally, we consider the price equilibrium when the SAs obtain information from different ESCs. Later, we will show that with licensed sharing such an equilibrium is not sustainable on the equilibrium path. Again, we divide this into two cases depending in part on the user valuation v. Theorem 8 Assume that SA 1 (2) obtains information from ESC A (B). If v≥

5q A q B + 2q B2 + 2q 2A Λ/W ) q 2A + 4q A q B + q B2

(18)

then the unique price equilibrium ( p1∗ , p2∗ ) is given by 2q B Λ/W + q A Λ/W + (q A − q B )v , 3 2q A Λ/W + q B Λ/W − (q A − q B )v p2∗ = . 3

p1∗ =

(19)

The corresponding user equilibrium is given by 2q B Λ + q A Λ + (q A − q B )vW , 3(q A + q B ) 2q A Λ + q B Λ − (q A − q B )vW λ∗2 = . . 3(q A + q B ) λ∗1 =

(20)

The profits of the SAs are, respectively, qA + qB ∗ 2 (λ1 ) − p˜ A , W qA + qB ∗ 2 (λ2 ) − p˜ B . π2 = W W π1 =

(21)

The condition in (18) implies that the market share of SA 1 is higher than that of SA 2. The first term in the profit of SA 1 is also strictly larger compared to SA 2. However, SA 1’s profit may be lower compared to that of SA 2 due to the payment p˜ A . Also, note that as the difference between q A and q B decreases, the profit of the SA 2 becomes closer to SA 1’s profit. When q B = q A , note that the condition in (18) becomes the same as that in Theorem 4 and prices and quantities are also equal to those in that theorem. Different from the unlicensed case [4], here equalizing the quantities does not necessarily lead to a negative profit for the SAs. The sum of λ1

176

A. Ghosh et al.

and λ2 in the equilibrium is equal to the total number of subscribers. Hence, when the SAs obtain information from different ESCs and the condition in (18) is satisfied, the SAs select prices such that they cover the entire subscription base. The condition in (18) is clearly satisfied when W is very large. Though the profit of SA 1 increases with W , the profit of SA 2 decreases with W . Intuitively, when W is large, SA 1 can select lower prices and serve a large number of users. Hence, SA 2 suffers because of the inferior quality of information. Next, we characterize a price equilibrium when condition (18) is not satisfied. Theorem 9 Assume that SA 1 (2) obtains information from ESC A (B). If 3Λ/(2W ) < v
3Λ/(2W ). Theorem 11 In the second stage, only one of the following five equilibria is possible: 1. Both the SAs obtain information from ESC A if r A − p˜ A ≥ 0 and r A − p˜ A ≥ r B − p˜ B . 2. Both the SAs obtain information from the ESC B if r B − p˜ B ≥ 0 and r B − p˜ B ≥ r A − p˜ A . 3. One of the SAs obtains information from the ESC A if r A − p˜ A < 0, r B − p˜ B < 0, and q A v2 , q A (v − Λ/W )Λ/W } − p˜ A ≥ 0. W max{ 4

178

A. Ghosh et al.

4. One of the SAs obtains information from the ESC B if r A − p˜ A < 0, r B − p˜ B < 0, and q B v2 , q B (v − Λ/W )Λ/W } − p˜ B ≥ 0. W max{ 4 5. Neither SA obtains information from any of the ESCs if max{

W qB v min{v/2, Λ}, q B (v − Λ/W )Λ} − p˜ B < 0 2

max{

W qAv min{v/2, Λ}, q A (v − Λ/W )Λ} − p˜ A < 0. 2

and

Thus, in contrast to the model in [4], with licensed spectrum, there can be an equilibrium in which both the SAs obtain information from the same ESC. Thus, only one ESC can exist in the market if the spectrum is licensed. Hence, the licensed spectrum does not lead to a competition among the ESCs unlike the unlicensed case. However, multiple SAs with the same quality can coexist with licensed access but not with unlicensed [4]. Similar to the unlicensed case, there can be scenarios in which only one SA exists in the market (cases 3 and 4) or in which no SA will find it profitable to enter (Case 5). When only one SA exists in the market, the corresponding price equilibrium is as in Theorem 7. When both SAs are in the market (cases 1 and 2), the corresponding price equilibrium depends on how v compares to Λ/W as in Theorems 8–10. In each of these cases, both SAs serve the same number of users at the same price. The competition between the SA can generate positive consumer surplus, but only when the user valuation is sufficiently high, namely when v > 3Λ/(2W ). Note that when both the ESCs obtain information from the same ESC (say ESC j), their payoffs are r j − p˜ j , which we refer to as their competitive profit. On the other hand, if only one of the SAs obtains information from ESC j and the other SAs do not obtain any information from any of the ESCs, we denote the SA’s monopoly profit by r M, j − p˜ j (cf. (17)).

3.4 First-Stage Equilibrium: ESC’s Price Selection We, now, describe the first-stage equilibrium. Recall that in this stage, ESC j selects the price p˜ j . Also recall from Theorem 11 that if both the SAs obtain information from the same ESC j, the payoffs of both the SAs are r j − p˜ j . On the other hand, if only one of the SAs obtains information (say, from ESC j), that SA’s payoff is r M, j p˜ j (cf.(17)). Note that ESC j incurs a cost c j for obtaining information regarding the presence of the incumbent. Recall that when both the SAs obtain information from ESC j, the payoff of the ESC j is

Tiered Spectrum Measurement Markets for Licensed Secondary Spectrum

2 p˜ j − c j .

179

(29)

Hence, it is apparent that the ESC j will not select a price lower than c j /2. On the other hand, if only one of them obtains information from ESC j, its payoff is p˜ j − c j . So, only one of the SAs obtains information from ESC j, it will not select any price lower than c j . In the following, we characterize the first-stage pricing strategy of the ESCs. Using this, we then have the following theorem. Theorem 12 1. If c j ≤ 2r j for each ESC j, and c j /2 < ck /2 + (r j − rk ) for some j, k ∈ {A, B}, k = j, there does not exist any pure strategy Nash equilibrium. There exists an -Nash equilibrium7 where p˜ j = ck /2 + (r j − rk ) − , and p˜ k = ck /2. In the second stage, both the SAs obtain information from ESC j. 2. If c j ≤ 2r j for each ESC j, and c j = ck + (r j − rk ) for some j, k ∈ {A, B}, k = j, the unique Nash equilibrium is p˜ j = c j /2, for j ∈ {A, B}. In the second stage, both the SAs obtain information from either ESC A or ESC B. 3. If c j ≤ 2r j , ck > 2rk , for ESC j = k, and 2r j ≥ r M, j − max{r M,k − ck , 0}, the unique first-stage pricing strategy is p˜ j = r j and p˜ k = ck . In the second stage, both the SAs obtain information from the ESC j. 4. If c j ≤ 2r j , ck > 2rk , for ESC j = k, and r M, j > 2r j + max{r M,k − ck , 0}, the unique first-stage pricing strategy is p˜ j = r M, j − max{r M,k − ck , 0}. Only one of the SAs obtains information from ESC j. 5. If r M, j ≥ c j > 2r j , for both ESCs j, and c j < ck + (r M, j − r M,k ) for ESC k = j, there does not exist any Nash equilibrium. There exists an -Nash equilibrium where p˜ j = ck + (r M, j − r M,k ) − , p˜ k = ck . In the second-stage equilibrium, only one of the SAs obtains information from ESC j. 6. If r M, j ≥ c j > r j for both ESCs j, and c j = ck + (r M, j − r M,k ) for ESC j = k, the unique Nash equilibrium is p˜ j = c j , for j ∈ {A, B}. In the second stage, only one of the SAs obtain information from ESC j. 7. If 2r j < c j ≤ r M, j , ck > r M,k , for ESC j = k, the unique first-stage pricing strategy is p˜ j = r M, j , and p˜ k = ck . In the second-stage equilibrium strategy, only one of the SAs obtains information from ESC j. 8. If c j > r M, j for both ESCs j, the unique first-stage pricing strategy is p˜ j = c j . In the second stage, neither of the SAs obtain information from the ESCs. In cases 1 and 2, if both ESCs’ costs are small, both the SAs would obtain information from the same ESC. Since the SAs will obtain information from only one of the ESCs, it leads to a price war between the ESCs. Thus, the competition is similar to the Bertrand–Edgeworth competition [15]. However, ESC k cannot decrease its price p˜ k below ck /2. The other ESC j can still undercut the ESC by selecting the price ck /2 + (r j − rk ) −  for any small  > 0 so that the resulting price is no smaller than c j /2. Thus, SA j will select the ESC j since the SA can achieve  savings. The profit attained by the SA in this scenario is at least rk − ck /2. an −Nash equilibrium, no player can gain more than  by unilaterally deviating from its own strategy.

7 In

180

A. Ghosh et al.

Case 2 shows that if ck /2 + (r j − rk ) = c j /2, each of the ESCs will select a price of c j /2. The SAs profit is the same r j − c j /2 = rk − ck /2. Thus, the SAs are indifferent between the ESCs. Case 3 shows that if 2rk < ck and 2r j ≥ c j for ESCs k = j, then ESC j may extract all the revenues from the SAs even when the ESC j sells information to both the SAs if r M, j < 2r j . Intuitively, since 2rk < ck , if both the SAs obtain information from ESC k, their payoff will be negative as the lowest possible price that can be set by ESC k is ck /2. Thus, the ESC j enjoys a monopoly power and can extract the maximum possible profit from the SAs. Note that when 2rk < ck , ESC k can sell information to at most one SA. Thus, the price which ESC k can set is ck . The SA will have a monopoly power; hence, the maximum profit achieved by the SA is r M,k − ck . Thus, if the price is set at min{ck + (r M, j − r M,k ) − , r M, j }, the SA will have a greater incentive to obtain information from ESC j. Thus, the maximum price set by the ESC j is ck + (r M, j − r M,k ) − . Hence, if this price is higher than 2r j , the revenue achieved in the monopoly scenario will be higher for ESC j compared to the scenario where both the SAs obtain information from the ESC j. Case 3 and Case 4 entail the above pricing strategy. Thus, if the monopoly profit from the SA is high enough, and the cost of one of the ESCs is high, the other ESCs (say A) set the price such that only one of the ESCs can remain in the market even though ESC A has a low cost. Cases 5 and 6 correspond to the scenarios where c j > 2r j for both the ESCs. Thus, the equilibrium where both the SAs will obtain information from the same ESC is not sustainable. Only a monopoly scenario can exist in the downstream market. The competition again becomes similar to the Bertrand–Edgeworth model as in cases 1 and 2. The ESC j for which j = arg maxk (r M,k − ck ) will win this competition. Case 6 indicates that if there is a tie, the SA is indifferent between the ESCs. Moreover, in the downstream market since only one of the SAs obtains information from the ESCs, the user’s surplus will be zero in cases 5 and 6. The SAs profit will still be positive if r M, j > c j for each ESC j. Case 7 indicates that if r M,k < ck for an ESC k, and r M, j > c j for the other ESC j, ESC j will set the price p˜ j at r M, j . Hence, the ESC j extracts all the profit from the monopoly SA. Intuitively, since the ESC has to set a price at least equal to the cost of obtaining the spectrum measurement data, thus, the SA never obtains any information from ESC k in an equilibrium. Hence, ESC j enjoys a monopoly power and sets the price such that only a monopoly can exist in the downstream market, and only one of the SAs obtains information from ESC j. Case 8 indicates that if the costs of both the ESCs are high enough, both the SAs will opt not to obtain information from the ESCs. Note that in the second stage both the SAs will obtain information from at most one of the ESCs; thus, one of the ESCs may prefer not to enter the market. It is similar to the scenario where one of the ESC’s (say B) cost of spectrum measurement is ∞. However, Theorem 12 shows that in such a scenario ESC A can select the monopoly price and extracts all the revenue from the SAs. It may select a price such that in the downstream there will not be any competition between the SAs. Hence, regulation may be needed to avoid such a scenario.

Tiered Spectrum Measurement Markets for Licensed Secondary Spectrum

181

3.5 Impact of Bandwidth In the following, we further characterize the impact of W on the second stage. 4c j , neither SA obtains information from an Corollary 1 When W ≤ min j∈{A,B} q j v2 ESC. When W is small, the profits of the SAs are very low even in the monopolistic scenario. Thus, neither of the SAs obtains information from an ESC. q j Λ2 , there is no equilibrium where both the SAs cj obtain information from an ESC. Corollary 2 When W ≥ max j

When W is large, the competitive equilibrium where both the SAs will serve users is not sustainable similar to the unlicensed case in [4]. However, when W is large enough, the monopolistic scenario may arise where only one of the SAs will obtain information from one of the ESCs.

4 Conclusion We have considered a simple model of markets for spectrum measurements motivated by the CBRS system. A key feature of our model is that firms offering wireless service must acquire information about spectrum availability from an ESC, where different ESCs may offer different qualities of information. Our results show that the different information qualities cannot be supported in equilibrium when SAs have licensed secondary spectrum. Our analysis also shows that the ESC’s cost of obtaining spectrum measurement data and the competitiveness between ESCs play an important role in the SAs’ competition. Only if the cost of obtaining the spectrum measurement data is low, can both SAs can exist in the market. Moreover, if only one ESC has a high cost, then the low-cost ESC may price its service so that only a single SA will enter the market. Our analysis also shows such a monopoly scenario can arise if the amount of spectrum is sufficiently large. There are many directions this work could be extended including considering more SAs or ESCs in the market. We have considered models where the spectrum is either entirely licensed or unlicensed. A hybrid model in which portions of the spectrum are licensed and unlicensed is another possible extension.

References 1. Federal Communications Commission, “Amendment of the commission’s rules with regard to commercial operations in the 3550-3650 MHz band,” FCC 15-47 Report and order and second further notice of proposed rulemaking, April 2015.

182

A. Ghosh et al.

2. Federal Communications Commission, “Wireless telecommunications bureau and office of engineering and technology conditionally approve seven spectrum access system administrators for the 3.5 GHZ band,” FCC Public Notice, Dec. 2016. 3. Federal Communications Commission, “Wireless telecommunications bureau and office of engineering and technology conditionally approve four environmental sensing capability operators for the 3.5 GHZ band,” FCC Public Notice, February 2018. 4. A. Ghosh, R. Berry, and V. Aggarwal, “Spectrum measurement markets for tiered spectrum access,” in 2018 IEEE International Conference on Communications (ICC), May 2018, pp. 1–6. 5. F. Zhang and W. Zhang, “Competition between wireless service providers: Pricing, equilibrium and efficiency,” in 2013 11th International Symposium on Modeling & Optimization in Mobile, Ad Hoc & Wireless Networks (WiOpt), IEEE, 2013, pp. 208–215. 6. P. Maillé, B. Tuffin, and J.-M. Vigne, “Competition between wireless service providers sharing a radio resource,” in International Conference on Research in Networking. Springer, 2012, pp. 355–365. 7. T. Nguyen, H. Zhou, R. Berry, M. Honig, and R. Vohra, “The cost of free spectrum,” Operations Research, vol. 64, no. 6, pp. 1217–1229, 2016. 8. T. Nguyen, H. Zhou, R. A. Berry, M. L. Honig, and R. Vohra, “The impact of additional unlicensed spectrum on wireless services competition,” in 2011 IEEE Symposium on New Frontiers in Dynamic Spectrum Access Networks (DySPAN). IEEE, 2011, pp. 146–155. 9. C. Liu and R. A. Berry, “Competition with shared spectrum,” in 2014 IEEE International Symposium on Dynamic Spectrum Access Networks (DYSPAN). IEEE, 2014, pp. 498–509. 10. D. Acemoglu and A. Ozdaglar, “Competition and efficiency in congested markets,” Mathematics of Operations Research, vol. 32, no. 1, pp. 1–31, 2007. 11. A. Hayrapetyan, E. Tardos, and T. Wexler, “A network pricing game for selfish traffic,” in Proc. of SIGACT-SIGOPS Symposium on Principles of Distributed Computing (PODC), 2005. 12. A. Ghosh, S. Sarkar, and R. Berry, “The value of side-information in secondary spectrum markets,” IEEE Journal on Selected Areas in Communications, vol. 35, no. 1, pp. 6–19, 2017. 13. K. Bimpikis, D. Crapis, and A. Tahbaz-Salehi, “Information sale and competition,” under submission. 14. X. Vives, “Duopoly information equilibrium: Cournot and bertrand,” Journal of economic theory, vol. 34, no. 1, pp. 71–94, 1984. 15. R. J. Deneckere and D. Kovenock, “Bertrand-edgeworth duopoly with unit cost asymmetry,” Economic Theory, vol. 8, no. 1, pp. 1–25, Feb 1996. [Online]. Available: https://doi.org/10. 1007/BF01212009

On Incremental Passivity in Network Games Lacra Pavel

Abstract In this paper, we show how control principles and passivity properties can be used in analysing and designing learning rules/dynamics for agents playing a network game. We focus on two instances: (1) agents learning about the others’ actions and (2) agents learning about the game (reinforcement-learning). In both cases, we show the trade-off between game properties and agent learning dynamics properties, underpinned by passivity/monotonicity and the balancing principle.

1 Introduction Classical game theory, pioneered by John von Neumann and John F. Nash, studies multi-player strategic interactions, their possible strategies/decisions and their outcomes. Many examples of decision-making problems in multi-agent networks can be modelled as games on networks, or network games, such as resource allocation games in cloud computation, search and rescue using autonomous vehicles (AV) or robots; demand response management in smart grids; power allocation games over cognitive radio networks or optical networks, e.g. [1, 3, 23, 27]. The agents can be users in a communication network or robots in a coverage control problem, each with their own goal to pursue (e.g. maximum transmission bandwidth or minimum interference). They are pursuing their individual goals (non-cooperative), and the overall goal is that agents achieve an optimal game solution, a Nash equilibrium (NE), in an online and autonomous manner. In a networked setting of game theory, there are many challenges, such as complexity of interaction/communication; imperfect or partial information, possibly delayed and asynchronous, hence the need for learning; simultaneous agents’ learning, hence the need for adaptation to a moving target; large scale and heterogeneity of agents in This work was supported by a NSERC Discovery Grant. L. Pavel (B) University of Toronto, Toronto, ON, Canada e-mail: [email protected] © Springer Nature Switzerland AG 2019 J. Walrand et al. (eds.), Network Games, Control, and Optimization, Static & Dynamic Game Theory: Foundations & Applications, https://doi.org/10.1007/978-3-030-10880-9_11

183

184

L. Pavel

a changing (dynamic) environment and/or agents setting, hence the need for robustness; strategic interaction and possible deceiving/cheating/misinformation, hence the need for coping mechanisms. A general theory does not exist yet. In this paper, we argue for a unified perspective on game theory on networks, using control theory (for learning) and graph theory (for networks). We show how control principles and properties such as feedback and passivity can be used in analysing and designing learning rules/dynamics for agents playing a network game. We consider that agents self-optimize their decisions even though they have limited or partial information (over a network). This means the agents need to learn about the game and about the other agents. By learning, we mean dynamic processing of information, or in system control terms, dynamic estimation. The design of such learning rules (a.k.a. algorithms or dynamics) involves the use of feedback; the interconnected feedback loops between multiple agents’ learning rules introduce the control aspect. Learning can be abstracted as a set of interconnected systems of agents trying to optimize their cost/payoff. The networked multi-agent interaction and communication can be modelled using graph theory. We show that in such network game settings, the lack of full information can be compensated by agents/players using feedback either from: (1) their neighbours, thus potentially collaborating in sharing information, while still pursuing their own selfish goals, or (2) from the game environment itself, and aggregating such information in a dynamic manner. Literature review. In the classical approach to Nash equilibrium seeking, players observe all the other players’ actions, e.g. [31, 32] or have knowledge of their realized payoff, e.g. [10]. Observing all others’ actions/decisions as considered in the classical approach can be impractical in distributed networks, and hence, some information exchange is required. As pointed out in [22], in large-scale multi-agent systems, a player is inherently limited to being observable to and communicate with a few other players and usually has relatively weak computational capabilities. In an economic setting, [5] draws attention to the problem of “who interacts with whom” in a network and to the importance of communication with neighbouring players. The effect of local peers on increasing the usage level of consumers is addressed in [7]. Motivated by the above, in recent years there has a sustained effort to generalize the classical setting of Nash equilibrium seeking and consider the issue of incomplete information on the opponents’ decisions, and how to deal with it via networked information exchange, e.g. [13, 20, 22, 30, 34, 35]. The information exchange could be in form of each agent communication with a central node, which then can broadcast that information to all others (as in [15, 26]), or in the form of local communication between agents over some communication graph (as in [20, 30, 34, 35]). Contributions. In this paper, we provide a feedback control perspective on learning in network games. We focus on two instances: (1) agents learning about the others’ actions via projected gradient (PG) and consensus learning (primal space dynamics) in a continuous-action (CA) game over a network; and (2) agents learning about the game environment via exponential reinforcement-learning (RL) (dualspace dynamics) in a finite-action (FA) game. In both cases, agent learning dynamics converge in the class of (strictly) monotone games, based on their overall interconnected dynamics being passive. We show the trade-off between game properties and

On Incremental Passivity in Network Games

185

agent learning dynamics properties, underpinned by passivity/monotonicity and the balancing principle. Specifically, whenever there is shortage of passivity, this should be compensated by other terms in the learning dynamics that have excess passivity. Furthermore, we show how based on the principle of feedback and passivity, one can design higher-order generalizations of RL or PG/consensus with the same guarantee of convergence. The use of passivity to investigate game dynamics was first proposed in [9] for population games, based on the notion of δ-passivity. Here we use incremental and equilibrium-independent passivity, [16], drawing on results from [11, 12]. The paper is organized as follows. Background is given in Sect. 2 and the problem statement in Sect. 3. Learning about the others is presented in Sect. 4, while learning about the game via reinforcement-learning is discussed in Sect. 5. Discussions and conclusions are presented in Sect. 6.

2 Background 2.1 Monotone Operators Let z ∈ Rn , z = [z 1 , ..., z n ] , also denoted as z = (z 1 , ..., z n ) or z = (z i )i={1,...,n} . We n   product z, z  := z assume that Rn is equipped with the standard inner i=1 i z i = √ z  z  and the induced Euclidean (2-norm) z := z, z. An operator (or mapping) F : D ⊆ Rn → Rn is said to be monotone on D if (F(z) − F(z  )) (z − z  ) ≥ 0, ∀z, z  ∈ D. It is strictly monotone if strict inequality holds ∀z, z  ∈ D, z = z  . F is μ-strongly monotone if for some μ > 0, (F(z) − F(z  )) (z − z  ) ≥ μz −  2 z  , ∀z, z  ∈ D. F : D ⊆ Rn → Rn is θ -Lipschitz if there exists a θ > 0 such that F(z) − F(z  ) ≤ θ z − z  , ∀z, z  ∈ D. F is non-expansive if θ = 1, and contractive if θ ∈ (0, 1). F : D ⊆ Rn → Rn is β-cocoercive if there exists a β > 0 such that (F(z) − F(z  ) (z − z  ) ≥ βF(z) − F(z) 2 , ∀z, z  ∈ D. F is referred to as firmly nonexpansive for β = 1. For f : Rn → R differentiable, ∇ f (x) = ∂∂ xf (x) ∈ Rn denotes its gradient. We note that a C 1 function f is convex if and only if (∇ f (z) − ∇ f (z  )) (z − z  ) ≥ 0, ∀z, z  ∈ dom f , and strictly convex if and only if (∇ f (z) − ∇ f (z  )) (z − z  ) > 0, ∀z, z  ∈ dom f, z = z  . Thus, f is convex, strictly (strongly) convex if and only if its gradient ∇ f is monotone, strictly (strongly) monotone, respectively (Theorem 4.1.4 in [17]). Monotonicity plays in variational inequalities the same role as convexity plays in optimization.

186

L. Pavel

2.2 Incremental Passivity and Equilibrium-Independent Passivity Passivity is a dynamical system property that can be used in analysing stability of interconnected systems, [19]. Consider Σ  Σ:

x˙ = f (x, u), y = h(x, u),

(1)

with x ∈ Rn , u ∈ Rq , y ∈ Rq , f locally Lipschitz, h continuous. Let u, x, y be an equilibrium condition, such that 0 = f (x, u), y = h(x, u). Assume there exists Γ ⊂ Rq and a continuous function k x (u) such that for any constant u ∈ Γ , f (k x (u), u) = 0 (basic assumption). The following are from [6, 16]. Definition 1 System Σ (1) is equilibrium-independent (EIP) passive if for every u ∈ Γ there exists a differentiable, positive semi-definite storage function Vx : Rn → R such that Vx (x) = 0 and, for all u ∈ Rq , x ∈ Rn , V˙x (x) ≤ (y − y) (u − u).

(2)

where V˙x (x):=∇Vx (x) f (x, u) is the time derivative of Vx along solutions of (1). Σ is output strictly EIP (OSEIP) if there is aβ > 0 such that V˙x (x) ≤ (y − y) (u − u) − βy − y2 .

(3)

Σ is input strictly EIP (ISEIP) if there exists μ > 0 such that V˙x (x) ≤ (y − y) (u − u) − μu − u2 .

(4)

Note that EIP passivity requires that (2) holds for every u ∈ Γ (Σ to be passive independent of the equilibrium point), while traditional passivity, [19] requires that it holds only for a particular u (usually associated with the origin as equilibrium). When passivity holds in comparing any two trajectories of Σ, a stronger property called (IP) incremental passivity holds, [28]. Definition 2 System Σ (1) is incrementally (IP) passive if there exists a C 1 , positive semi-definite storage function V : Rn × Rn → R such that for any two inputs u, u  and any two corresponding solutions x, x  , the outputs y, y  satisfy V˙ (x, x  ) ≤ (y − y  )T (u − u  ) OSIP and ISIP properties can be defined similarly to OSEIP, ISEIP. When u  , x  , y  are constant this recovers EIP definition. For a linear time invariant system Σ, all these passivity concepts, i.e. EIP passivity, incremental passivity and passivity, are equivalent. Furthermore, for static

On Incremental Passivity in Network Games

187

Fig. 1 Feedback(Σ, −Π ) interconnection and parallel Σ||Π interconnection

maps there is an interesting connection between convexity, monotonicity and passivity. When system Σ is just a static map Φ, i.e. y = Φ(u), EIP/incremental passivity is equivalent to monotonicity of the map. A static map y = Φ(u) which is (OSEIP/ISEIP/strictly) EIP/incrementally passive is equivalently (β-cocoercive/ μstrongly/strictly) monotone, respectively. Monotonicity plays an important role in optimization and variational inequalities, while passivity plays a critical a role in analysing interconnections of dynamical systems. Feedback and parallel interconnection of two EIP passive systems Σ and Π are an EIP passive system, (see (Σ, −Π ) and Σ||Π in Fig. 1, cf. Properties 2 and 3 in [16]). Given a matrix M and an EIP passive system Σ, then M T Σ M is also EIP passive. Moreover, the principle of balancing shortage of passivity in some parts of the system by excess passivity in other parts can be used to ensure overall system passivity. EIP passivity properties help in deriving stability and convergence properties for feedback systems without requiring knowledge of an equilibrium point, but rather only required to know that such an equilibrium point exists. Furthermore, for an interconnected feedback system of EIP passive systems, the sum of the storage functions can be used as a Lyapunov function to prove stability of an equilibrium point of the system. Considering that in games the Nash equilibrium (NE) is unknown a-priori, this property is a useful one to seek and exploit.

2.3 Game Formulation Consider a game G between a set N = {1, . . . , N } of players (agents). Each player i ∈ N has a real-valued cost Ji , or a utility Ui = −Ji function, and aims to minimize (maximize) its (expected) cost Ji (utility Ui ), which depends on the others’ actions/strategies. A player has either a finite set of actions (pure strategies), Ai , with |Ai | = n i , or a continuous set, Ωi ⊂ Rni . Continuos-Action (CA) Games. In a continuous-action (CA) game, each player takes its action, denoted by xi , from Ωi ⊂ Rni . Let x = (xi , x−i ) ∈ Ω denote all 1)-tuple of all agents’ agents’ action profile (N -tuple), where  x−i is the (N − actions except agent i’s and Ω = i∈N Ωi ⊆ Rn , n = i∈N n i . Alternatively, x = (x1 , . . . , x N ) or as a stacked vector x = (xi )i∈N ∈ Ω ⊆ Rn . Each player i aims

188

L. Pavel

to minimize (maximize) its own cost/utility Ji (xi , x−i ) (Ui (xi , x−i )), Ji /Ui : Ω → R. Denote this game by G(N , Ω, Ji /Ui ). ∗ Definition 3 Given a game G(N , Ω, Ji /Ui ), an action profile x ∗ = (xi∗ , x−i )∈Ω is a Nash equilibrium (NE) of G if ∗ ∗ ) ≤ Ji (xi , x−i ), ∀xi ∈ Ωi , ∀i ∈ N Ji (xi∗ , x−i ∗ ∗ ) ≥ Ui (xi , x−i ), ∀xi ∈ Ωi , ∀i ∈ N . or Ui (xi∗ , x−i

Assumption 1 For all i ∈ N , Ωi ⊂ Rni is non-empty, convex and compact and Ji /Ui : Ω → R is C 1 and (strictly) convex /concave in xi , for every x−i ∈ Ω−i . Under Assumption 1, it follows that a (pure) NE exists, based on Kakutani’s fixed point theorem, (cf. Theorem 4.4 in [2]). A NE x ∗ ∈ Ω can be characterized as solution of the variational inequality VI(Ω, F) (cf. Proposition 1.4.2, [8]), (x − x ∗ )T F(x ∗ ) ≥ 0 ∀x ∈ Ω

(5)

where F(x) := (∇xi Ji (x))i∈N , ∇xi Ji (x) = ∂∂ xJii (xi , x−i ), or F(x):= (−∇xi Ui (x))i∈N , i ∇xi Ui (x) = ∂U (x , x−i ), is the pseudo-gradient game map, F : Ω → Rn . ∂ xi i Finite-Action (FA) Games. In a finite-action (FA) game, each player takes its action, denoted by ai , from Ai , with |Ai | = n i . Without loss of generality, we identify Ai as the corresponding index set and a generic action ai ∈ Ai as an index, j ∈ Ai = {1, . . . , n i }. A mixed strategy of player i, xi = (xi j ) j∈A i , is a probability distribution over its set of actions Ai , xi ∈ Δi , Δi := {xi ∈ Rn≥0i | j∈Ai xi j = 1}. A mixed-strategy profile is x = (x1 , . . . , x N ), or x = (x −i is the i , x−i ) ∈ Δ, where x strategy profile of the other players except i and Δ := i∈NΔi ⊂ Rn , n = i∈N n i . Each player has a cost/utility Ji /Ui : A → R, where A = i∈N Ai . Typically, a utility (payoff) function is employed. Player i’s expected payoff in the mixed-strategy profile x ∈ Δ is   ··· Ui ( j1 , . . . , j N )x1 j1 . . . x N jN , (6) Ui (x) = j1 ∈A1

j N ∈A N

Player i’s expected payoff to using pure strategy j ∈ Ai in the mixed profile x ∈ Δ, Ui ( j, x−i ), is denoted by Ui j (x). Hence Ui ( j, x−i ) ≡ Ui j (x), and (6) is   Ui ( j, x−i ) xi j = xi j Ui j (x) = xi  Ui (x) (7) Ui (x) = j∈Ai

j∈Ai

where Ui (x) := (Ui j (x)) j∈Ai ≡ (Ui ( j, x−i )) j∈Ai ∈ Rni . Ui (x) is called the payoff vector of player i at x ∈ Δ, (7) indicating the duality between xi and Ui , [24]. Let the mixed extension of G be denoted by G(N , Δ, Ui ). Similar to Definition 3, a ∗ ) ∈ Δ is NE of G(N , Δ, Ui ) if (mixed-strategy) profile x ∗ = (xi∗ , x−i

On Incremental Passivity in Network Games ∗ Ui (xi∗ , x−i ) ≥ Ui (xi , x ∗ −i ), ∀xi ∈ Δi , ∀i ∈ N .

189

(8)

Assumption 1 is satisfied for Δ = Ω, so a (mixed) strategy NE exists in any FA game. Such a NE x ∗ satisfies xi∗ ∈ B Ri (x ∗ ), ∀i ∈ N , or, x ∗ ∈ B R(x ∗ ), where B R = (B Ri )i∈N , B Ri : x → argmax xi ∈Δi Ui (x) is i’th mixed best-response map. Alternatively, by (7), (8), xi∗  Ui (x ∗ ) ≥ xi  Ui (x ∗ ), ∀xi ∈ Δi , ∀i ∈ N , or, − (x − x ∗ ) U (x ∗ ) ≥ 0, ∀x ∈ Δ

(9)

where U : Δ → Rn , U (x) = (Ui (x))i∈N , is the payoff game map. Based on (7), since i (xi , x−i ) =Ui (x) ∈ Rni and (∇xi Ui (x))i∈N Ui (x) is independent of xi , ∇xi Ui (x) := ∂U ∂ xi =U (x). Recall (5) F(x) = −(∇xi Ui (x))i∈N = −U (x), and hence in FA games the pseudo-gradient map is the (negative) payoff game map. Thus, a NE x ∗ is characterized as a solution of VI(Δ, −U), (9). The CA and FA game setups are related by identifying Ω with Δ and F with −U , with xi denoting a (pure) action in a CA game and a mixed strategy in a FA game.

3 Problem Statement Consider a repeated CA/FA game, where players use previous game iterations to gather information about other agents or the game and adjust their decisions. The learning process, or dynamics, of agent i, denoted by Σi describes the update in time of agents’ decisions (actions/strategies). Typical learning rules/dynamics are bestresponse (fictitious-play), projected gradient (better-response) play (model-based learning), and reinforcement-learning (payoff-based learning). The inherent coupling between the agent’s actions/strategies introduces challenges: when optimizing for its own reward, an agent needs to know the others’ decisions. The goal is for agents to come up with a solution of the game, hence find an unknown NE x ∗ , by themselves. They have to do that in an online, but distributed fashion, relying on minimal information on the others’ decisions or on the game (cost/payoff functions). In a networked setting, the agents’ strategic interaction can be abstracted by an interference graph G I (which could be complete, in a fully coupled game), while their communication is represented by a communication graph Gc , see Fig. 2. In such a game, agents should self-optimize their decisions even though they have limited local or partial information. This means they need to learn about the other agents in the network and/or about the game. An agents lack of information can be compensated by using feedback from its communication neighbours and/or from the game (see Fig. 3). Learning is based on aggregating the received information in a dynamic manner. Thus, an agents learning rule (algorithm/dynamics) must involve feedback; the interconnected feedback loops between multiple agents lead to an overall interconnected dynamical system. Our aim in this paper is to show that feedback control principles and passivity properties play an instrumental role in analysis/design. We consider two scenarios:

190

L. Pavel 1

2

N

3

Interference Graph GI 1

2

N

3

Communication Graph

GC

Fig. 2 Multi-agent games on networks: graph model abstraction Fig. 3 Multi-agent games on networks: block diagram model

sj

xi

Σi Gc

2

si

Σ−i

Game Environment (Cost/Utility)

1

2

GI

x−i

first, learning about other players (Sect. 4) when an agent knows its own cost/payoff model, and secondly, learning about the game (reinforcement-learning) (Sect. 5). In the first case, we consider agents with projected gradient (PG) dynamics playing a continuous-action (CA) game, while in the second case, we consider exponential Q-learning type of learning playing a finite-action (FA) game. In these two representative instances, we show that one can exploit geometric features of different classes of games together with passivity properties of interconnected systems to guarantee convergence.

4 Learning About Other Players In this section, we consider that each player i knows its individual utility/cost model (Ui /Ji ), but not all others’ decisions and hence a game with incomplete/partial information. To compensate for the lack of global information, we assume that players agree to share some information and use local feedback from their neighbours over Gc . Since they know their individual cost model, they only need to learn about the others. Thus, learning refers to agent i estimating the strategies, decisions x−i or the

On Incremental Passivity in Network Games

191

Fig. 4 Learning about others: block diagram model

sj

xi

Σi Gc

Game Environment (Cost/Utility)

1

si

Σ−i

GI x−i

states of other agents by using only local information/feedback from its neighbours (see Fig. 4). The feedback signal s j from neighbour j can be its decision, i.e. x j (in a continuous-action set/CA game), or a j (in a finite-action set/FA game), and/or its signalling/state variables. In this section, we consider a CA game G(N , Ω, Ji ) and assume that players use with a projected gradient (PG) update rule/dynamics. We review first the case of perfect information, where each agent has knowledge of the actions of all other players x−i (see Fig. 5, left). Typical assumptions on the pseudo-gradient F(x) = (∇xi Ji (x))i∈N are: Assumption 2 The pseudo-gradient F : Ω → Rn is (i) strictly monotone (x − x  )T (F(x) − F(x  )) > 0, ∀x = x  . (ii) strongly monotone (x − x  )T (F(x) − F(x  )) ≥ μx − x  2 , ∀x, x  ∈ Ω, μ > 0, and Lipschitz continuous, F(x) − F(x  ) ≤ θ x − x  , ∀x, x  ∈ Ω, θ > 0. Assumption 2(i), (ii) is equivalent to F strictly EIP and and input strictly EIP (incrementally passive), respectively. Under either, the game has a unique NE, (cf. Theorem 3 in [31]). With perfect information of x−i , at iteration k, player i updates its action according to a typical projected gradient (PG) rule,

xi

Σi

+

xi x−i

Σ−i

u

In ⊗

1 s

ΠΩ (x, −F (x)) x−i

Fig. 5 Perfect information case

Block diagram of Σ = (Σi , Σ−i )

x

192

L. Pavel

Σi : xi (k + 1) = PΩi (xi (k) − αi ∇xi Ji (xi (k), x−i (k))), ∀i ∈ N where αi > 0, PΩi (xi ) is projection of xi onto Ωi . Under Assumption 2(i) or (ii), PG converges to the NE [8]. Note that with PΩi = (I d + NΩi )−1 , where NΩi is the normal cone of Ωi at xi , Σi : xi (k) − xi (k + 1) − αi ∇xi Ji (xi (k), x−i (k)) ∈ NΩi (xi (k + 1)). In continuous time, this is represented by a differential inclusion −x˙i ∈ ∇xi Ji (xi , x−i ) + NΩi (xi ), or by an individual PG dynamics, Σi : x˙i = ΠΩi (xi , −∇xi Ji (xi , x−i )), ∀i ∈ N where ΠΩi (xi , v) is projection of v onto the tangent cone of Ωi at xi . The overall PG dynamics of all agents Σ = (Σi , Σ−i ) is Σ : x˙ = ΠΩ (x, −F(x)). can be represented as the feedback interconnected system in Fig. 5 right, between a bank of integrators and a static map involving the pseudo-gradient map F. One can start from this to design learning rules for the partial information case (see Fig. 4 or 6 left). When no central node exists to disseminate information, players can build an estimate of actions of all others, x−i by communicating with neighbours over a partial communication graph Gc , [11]. Each player i is endowed with a state i i , and hence xi = (xi , x−i ), encompasses its that estimates the others’ actions, x−i decision and its estimate. The idea is to add communication to PG dynamics so the equilibrium is the NE, and exploit passivity (monotonicity) of F and L to show that all states reach consensus and converge to the NE. i is given as: In this case, each individual player dynamics Σ

⎤ ⎡  i j i i  i (xi − xi ) ⎥ ⎢ΠΩi xi , −∇xi Ji (xi , x−i ) − x˙ i ⎥ ⎢ i : j∈Ni Σ (10) = i ⎦ ⎣  i x˙ −i j − (x−i − x−i ) j∈Ni

This is an augmented projected gradient (PG) with consensus correction based on local feedback; players perform simultaneous consensus of estimates and player= by-player optimization. With x = (xi )i∈V , the overall interconnected dynamics Σ   (Σi , Σ−i ) are written in stacked form as : Σ

sj

x˙ = RT ΠΩ (Rx, −F(x) − RLx) − S T SLx

xi

Σi Gc

Game Environment (Cost/Utility)

1

si

Σ−i

+

GI x−i

Fig. 6 Partial information over Gc

IN n ⊗

1 s

x

RT ΠΩ (Rx, −F(x)) −L ⊗ In  = (Σ, −L), Σ = (Σi , Σ−i ), L = L ⊗ In Block diagram of Σ

On Incremental Passivity in Network Games

193

where F(x) := (∇xi Ji (xi ))i∈V : Ω N → Rn is the extended pseudo-gradient of the game, L = L ⊗ In , L is the Laplacian of Gc , and R, S are matrices used to select  actions and estimates from x, e.g. x = Rx. The overall set of learning dynamics Σ is represented as the feedback interconnected system in Fig. 6 right. Note that the  = (Σ, −L), correction appears as an extra feedback loop via the Laplacian L of Gc , Σ instrumental to proving convergence on a single timescale. Under strict monotonicity of F and a monotonicity (incremental passivity) assumption of the extended pseudo-gradient F, single timescale convergence can be proved based on passivity properties for any connected Gc , (Theorems 1 and 4 in [11]). When monotonicity of F does not hold, convergence can be achieved by balancing shortage of passivity (monotonicity) of F on the augmented space with excess passivity of L, (in terms of its connectivity of Gc as described by λ2 (L)) (Theorems 2 and 5 in [11]). This is summarized in the next result. Assumption 3 The extended pseudo-gradient F : Ω N → Rn is (i) monotone, (x − x )T (F(x) − F(x )) ≥ 0, ∀x, x ∈ Ω N . (ii) θ -Lipschitz continuous, F(x) − F(x ) ≤ θ x − x , ∀x, x ∈ Ω N , θ > 0. Theorem 1 Consider a game G(V, Ωi , Ji ) over an undirected communication graph Gc . Under Assumptions 1, 2(i) and 3(i), any solution of (10) will converge asymptotically to 1 N ⊗ x ∗ , and the action components converge to the NE of the game, x ∗ , for any connected Gc . Under Assumptions 2(ii) and 3(ii), the same con2 clusion holds if λ2 (L) > θμ + θ . Under Lipschitz continuity of F (holding in any quadratic game automatically), the idea is to decompose the augmented space R N n into the consensus subspace C Nn = {1 N ⊗ x |x ∈ Rn } and its orthogonal complement (C Nn )⊥ . Then F(x) − F(x) = F(x⊥ + x|| ) − F(x|| ) + F(x|| ) − F(x)       off consensus

on consensus

where x = x|| + x⊥ , x|| ∈ C Nn , x⊥ ∈ (C Nn )⊥ , x ∈ C Nn . One can leverage passivity (monotonicity) of F on C Nn , due to F(1 N ⊗ x) = F(x) (original pseudo-gradient), while off C Nn , the shortage of passivity of F can be balanced by Lipschitz and excess passivity (strong monotonicity) of L on (C Nn )⊥ . Thus, a passivity-based approach highlights the trade-off between properties of the game and those of the communication graph. Note that one can use again to relax the connectivity bound on the  with separated stacked actions x = Rx Laplacian L. Moreover, a block diagram of Σ and stacked estimates z = Sx is shown in Fig. 7, where H (s) = I d. Based on passivity properties, different higher-order learning dynamics could be designed by simply replacing the identity block Id with any other passive system H (s), which potentially can have better properties.

194

L. Pavel

u

−R

x˙ = ΠΩ (x,−F(RTx+S Tz)+u)

x

RT

z −S



I(N −1)n ⊗

Q ⊗ In

1 s

z

QT ⊗ In

H(s)

+

ST

x

 = (Σ, −L) with separated stacked actions x and stacked estimates z Fig. 7 Block diagram of Σ dynamics, x = Rx, z = S x, L = L ⊗ In , L = Q Q T

5 Learning About the Game In this section, we consider a setting where agent i does not know its individual payoff/cost function Ui /Ji . In this case, the game/environment is unknown; the most an agent can know is its realized payoff πi as a result of some action. Learning refers to agent i estimating the payoff functional dependency, based on πi , hence based on feedback from the game being played. This is shown in Fig. 8, left, equivalent to using only feedback labelled 2 in the block diagram in Fig. 3, in the form of realized payoff, hence called reinforcement-learning. Even though there is no explicit interconnection between Σi s, indirect coupling exists due to feedback from the game. We consider an instance of reinforcement-learning (RL), namely exponential (discounted) Q-learning (EXP-D-RL) in a FA game G(N , Δ, Ui ). We model the EXP-D-RL scheme in continuous time as in [24, 25]. We assume that each player keeps a score z i of all its actions, based on its received payoff Ui (x) ≡ Ui ( j, x−i ), maps this score (Q vector) into a strategy xi ∈ Δi and selects an action according to the strategy xi . This process is repeated indefinitely, with an infinitesimal time step between each stage and hence can be modelled in continuous time. Thus, the EXP-D-RL scheme for player i ∈ N is written as,

Fig. 8 Reinforcement-learning Σ = (Σi )i∈N , (13)

Block

diagram

of

EXP-D-RL

as

(Σ, U ),

On Incremental Passivity in Network Games

z˙ i = γ (Ui (x) − z i ), z i (0) ∈ Rni xi = σi (z i ), where σi (z i ) := 

j∈Ai

1 exp( 1ε z i j )

195

(11)

  exp( 1ε z i1 ). . .exp( 1ε z ini ) and ε > 0 a regularization/

temperature parameter, [24]. For ε = 1, σi is known as the standard softmax function. As ε → ∞, actions are selected with uniform probability (“exploration”), and as ε → 0, the softmax function selects the action associated with the highest score (best-response/“exploitation”). In fact, σi is generated as the smooth best  response, σi : z i → argmax xi ∈Δi {x i z i − ψi (x i )}., where ψ is the (negative) Gibbs  entropy. ψi (xi ) := ε j∈Ai xi j log(xi j ) used as a regularizer. Remark 1 Equation (11) is the mean field of the discrete-time stochastic process, [4],   z i (k + 1) = z i (k) + α(k)γ uˆ i (k) − z i (k) xi (k + 1) = σi (z i (k + 1)) where xi (k) = (xi j (k)) j∈Ai , xi j (k) is the probability of playing j ∈ Ai at the k-th instance of play, uˆ i (k) = (uˆ i j (k)) j∈Ai is an unbiased estimator of Ui (x(k)), i.e. such that E(uˆ i (k)) = Ui (x(k)), and {α(k)} is a diminishing sequence of step sizes, e.g. 1 . If player i can only observe the payoff πi (k) := Ui ( j (k); a−i (k)) of its chosen k+1 action j (k), then a typical choice for uˆ i j (k) is uˆ i j (k) = πi (k)/xi j (k), if j (k) = j, [21, 24, 25]. Results from the theory of stochastic approximation [4] can be used to tie convergence of such discrete-time algorithms to the asymptotic behaviour of (11). Equation (11) corresponds to the individual Q-learning algorithm, which has been shown to converge in 2-player zero-sum games and 2-player partnership games (Proposition 4.2, [21]). In this paper, we restrict our focus to the continuoustime learning scheme to (11), as in [24, 32, 33]. Similar forms of “exponentially discounted” score dynamics have been investigated in [25, 29]. Structurally, EXPD-RL is similar to online mirror descent (OMD) in convex optimization, recently studied for CA games in [36]. In particular, the score z i is the dual variable to the primal variable xi . Therefore, (11) describes the evolution of learning in the dual space Rni , whereas the induced strategy trajectory describes the evolution in the primal space Δi . This type of duality is discussed in [24]. In the following, we discuss how a passivity approach used in the dual space can be used to guarantee convergence for a class of N -player games, (cf. [12]). To analyse the convergence of EXP-D-RL, let z = (z i )i∈N ∈ Rn , x = (xi )i∈N ∈ Δ and U (x) = (Ui (x))i∈N denote the stacked scores, mixed strategies and payoff game map, respectively. The overall EXP-D-RL learning of all agents (11) is 

z˙ = γ (U (x) − z), z(0) ∈ Rn x = σ (z),

(12)

196

L. Pavel

where σ (z) := (σi (z i ))i∈N . With u = U (x), EXP-D-RL (12) can be represented as the feedback interconnected system in Fig. 8, right, (Σ, U ), between decoupled dynamics Σ = (Σi )i∈N , indirectly coupled via the payoff game map U , where  Σ:

z˙ = γ (u − z) x = σ (z),

(13)

Any x  = σ (z  ) corresponding to an equilibrium (fixed point) σ (z  ) = U (σ (z  )) is a Nash equilibrium of a perturbed game by ε, i.e. a Nash distribution or a logit equilibrium. For small ε, it approximates the Nash equilibria of G, [14, 29]. Once this representation of EXP-D-RL as (Σ, U ) in Fig. 8 is identified, one can use passivity properties to prove convergence to a Nash distribution. The softmax function σ (·) is monotone (incrementally passive), and moreover it is ε-cocoercive (hence output strictly EIP) and is the gradient of the lse function (cf. Proposition 2 in [12]). Based on this, it can be shown that system Σ (13) is output strictly EIP (OSEIP) (cf. Proposition 3 in [12]). Then under monotonicity (incremental passivity) of the (negative) payoff game map −U , convergence can be proved by exploiting these passivity properties and Fig. 8. Furthermore, relaxation to only hypo-monotonicity of −U can be dealt with by balancing shortage of passivity with excess of passivity of σ and Σ (due to OSEIP). This is summarized in the following result (Theorem 1 in [12]), under one of the following assumptions on the payoff map U (x) := (Ui (x))i∈N = (Ui ( j; x−i ) j∈Ai )i∈N . Assumption 4 The payoff game map U : Δ → Rn is such that: (i) −U (·) is monotone, i.e. −(x − x  ) (U (x) − U (x  )) ≥ 0, ∀x, x  ∈ Δ. (ii) −U (·) is μ-hypo-monotone, −(x − x  ) (U (x) − U (x  )) ≥ μx − x  2 , ∀x,  x ∈ Δ., for some μ < 0. Theorem 2 Let G(N , Δ, Ui ) be a finite game with players’ learning schemes as given by EXP-D-RL, (11). Assume there are a finite number of isolated fixed points z  of U ◦ σ . Then, under Assumption 4(i), for any ε > 0, players’ scores z(t) = (z i (t))i∈N converge to a rest point z  , while players’ strategies x(t) = (xi (t))i∈N , x(t) = σ (z(t)) converge to a Nash distribution x  = σ (z  ) of G. Under Assumption 4(ii), the same conclusions hold for any ε > −μ > 0. Remark 2 Assumption 4(ii) corresponds to an unstable game and is equivalent to shortage of monotonicity (passivity) of −U (·) as described by μ < 0. Note that we only consider the weaker case, μ < 0 in Assumption 4(ii). This is because for μ > 0, −U (·) is strongly monotone (or input strictly EIP), and hence monotone is covered by Assumption 4(i). The monotonicity of the negative payoff map in Assumption 4(i) (μ = 0) is equivalent to −U (·) being EIP (incrementally passive). In population games, Assumption 4(i) corresponds to G being a stable game cf. [18], while in games with continuous-action set (CA) it corresponds to games with monotone pseudo-gradient map F. Recall Assumption 2(i), (ii), i.e. F strictly or strongly monotone (strictly or input strictly EIP/incrementally passive), and compare

On Incremental Passivity in Network Games

197

Fig. 8 to Fig. 5 in Sect. 4. Note that here Assumption 4(i), (ii) is weaker than those, i.e. −U is monotone or hypo-monotone (EIP or hypo EIP). This is possible because here, due to the regularization with ε > 0, Σ itself is strictly EIP (forward path of Fig. 8), and one can relax the assumption on the game to being just monotone, not necessarily strictl/strong, cf. Assumption 4(i). On the other hand, in Fig. 5 the bank of integrators is EIP, not strictly EIP, and convergence is guaranteed in strictly (strongly) monotone games, corresponding to the feedback path being strictly EIP. Remark 3 As in stable games, [18], Assumption 4(i) can be characterized via y  DU (x)y ≤ 0, for all x ∈ Δ, y ∈ T Δ, and Assumption 4(ii) via y  DU (x)y ≤ where DU (x) is the Jacobian of U (x) and T Δ = μy2 , for all x ∈ Δ, y ∈ T Δ,   ni T Δ , T Δ := {y ∈ R | i i i i∈N j∈Ai yi j = 0} is the tangent space of Δi . For 2player games, U is linear and Assumption 4 can be checked based on the payoff matrices of the two players, A and B. Since      0 A x1 U1 (x) := Φx, =  U (x) = U2 (x) B 0 x2 hence −(x − x  ) Φ(x − x  ) ≥ 0, for all x, x  ∈ Δ is equivalent to y  (Φ + Φ  )y ≤ 0 for all y ∈ T Δ, or Φ + Φ  is negative semi-definite with respect to T Δ. This is met, for example, in zero-sum games, where B = −A, hence Φ is skew-symmetric and x  Φx = 0, for all x ∈ Δ. Another class is the class of concave potential games where the payoff vector U (x) can be expressed as the gradient of a C 1 , concave potential function P, as in congestion games, [29]. Higher-order learning dynamics with guaranteed convergence properties in the same class of games can be designed by exploiting passivity properties (cf. Theorem 2 in [12]). One can leverage that feedback interconnection of Σ with another EIP system Ha will be preserve passivity properties (Fig. 9).

 Σ va



u −



Ha (s)

In ⊗

γ s+γ

z

σ(·)

−U (·)

 U ), Σ  = (Σ, −Ha ) Fig. 9 H-EXP-D-RL as feedback interconnection (Σ,

x

x

198

L. Pavel

6 Discussions and Conclusions We have shown how incremental/EIP passivity properties play an important role in analysing and designing agent learning rules in network games. Unlike multi-agent agreement and distributed optimization, agents do not have individually (incrementally) passive dynamics, but rather, each learning dynamics is individually only partially (incrementally) passive. Hence, agents have partially active dynamics, due to the coupling to the others’ simultaneous active learning. However, if the overall interconnected system is incrementally passive (which translates in monotonicity on an augmented space), convergence can be achieved on a single timescale in the class of strictly monotone games by using a Laplacian consensus (correction) component which is itself strongly passive off the consensus space. Hence, this is according to the principle of balancing passivity but on the augmented space. In a more general case, monotonicity (incremental passivity) on the augmented space does not hold. In this case, convergence can be achieved in the class of strongly monotone games, based on decomposing the augmented space in a consensus subspace (where the game is strongly monotone/passive) and its orthogonal complement (where the Laplacian is strongly passive). Then, the shortage of passivity off the consensus space, due to active learners with limited authority, can be balanced by a combination of excess game passivity on the consensus subspace and excess Laplacian passivity on its orthogonal complement, which can be seen again as an instance of the passivity balancing principle. Thus, the “easier” the game is in terms of passivity (monotonicity), the more aggressive the individual agent’s learning can be on its own. The “harder” the game is, i.e. less passive/monotone or even active/“unstable” game, the less aggressive the learning should be on its own, i.e. either balanced by a stronger passive consensus term (to compensate for the others’ being active) (PG/consensus), or by allowing more exploration (less exploitation) (RL). On the other hand, in the second part we showed that by employing a regularization, RL allows to relax the monotonicity property of the game to only hypo-monotonicity. Open problems are how to generalize learning rules, relax game and graph properties, deal with delayed/asynchronous information and improve agents’ privacy, or design incentives against cheating so that agents truthfully share some information.

References 1. Alpcan, T., Ba¸sar, T.: Distributed Algorithms for Nash Equilibria of Flow Control Games, pp. 473–498. Birkhäuser Boston (2005) 2. Ba¸sar, T., Olsder, G.: Dynamic Noncooperative Game Theory: Second Edition. Classics in Applied Mathematics, SIAM (1999) 3. Bauso, D.: Game Theory with Engineering Applications. SIAM, ser. Advances in Design and Control Series (2016) 4. Benaïm, M.: Dynamics of stochastic approximation algorithms. in Le Seminaire de Probabilites, Lecture Notes, Graduate Texts in Mathematics 1709, 1–68 (1999)

On Incremental Passivity in Network Games

199

5. Bramoullé, Y., Kranton, R., D’Amours, M.: Strategic interaction and networks. The American Economic Review 104(3), 898–930 (2014) 6. Bürger, M., Zelazo, D., Allgöwer, F.: Duality and network theory in passivity-based cooperative control. Automatica 50(8), 2051–2061 (2014) 7. Candogan, O., Bimpikis, K., Ozdaglar, A.: Optimal pricing in networks with externalities. Operations Research 60(4), 883–905 (2012) 8. Facchinei, F., Pang, J.: Finite-Dimensional Variational Inequalities and Complementarity Problems. Springer New York (2007) 9. Fox, M.J., Shamma, J.S.: Population Games, Stable Games, and Passivity. In: 51st IEEE Conf. on Decision and Control. pp. 7445–7450 (Dec 2012) 10. Frihauf, P., Krstic, M., Ba¸sar, T.: Nash Equilibrium Seeking in Noncooperative Games. IEEE Trans. on Automatic Control 57(5), 1192–1207 (2012) 11. Gadjov, D., Pavel, L.: A Passivity-Based Approach to Nash Equilibrium Seeking over Networks. IEEE Trans. on Automatic Control https://doi.org/10.1109/TAC.2018.2833140 12. Gao, B., Pavel, L.: On Passivity and Reinforcement Learning in Finite Games. In: 51st IEEE Conf. on Decision and Control (Dec 2018) 13. Gharesifard, B., Cortes, J.: Distributed convergence to Nash equilibria in two-network zero-sum games. Automatica 49(6), 1683–1692 (2013) 14. Govindan, S., Reny, P.J., Robson, A.J.: A short proof of Harsanyi’s purification theorem. Games Econ. Behav. 45, 369–374 (2003) 15. Grammatico, S., Parise, F., Colombino, M., Lygeros, J.: Decentralized Convergence to Nash Equilibria in Constrained Deterministic Mean Field Control. IEEE Trans. on Automatic Control 61(11), 3315–3329 (2016) 16. Hines, G., Arcak, M., Packard, A.: Equilibrium-independent passivity: A new definition and numerical certification. Automatica 47(9), 1949–1956 (2011) 17. Hiriart-Urruty, J.B., Lemaréchal, C.: Fundamentals of Convex Analysis (2001) 18. Hofbauer, J., Sandholm, W.H.: Stable games and their dynamics. J. Econ. Theory 144(4), 1665–1693 (2009) 19. Khalil, H.: Nonlinear Systems. Prentice Hall (2002) 20. Koshal, J., Nedic, A., Shanbhag, U.V.: Distributed algorithms for aggregative games on graphs. Operations Research 64(3), 680–704 (2016) 21. Leslie, D., Collins, E.: Individual Q-Learning in Normal Form Games. SIAM J. Control and Optimiz. 44(2), 495–514 (2005) 22. Li, N., Marden, J.R.: Designing games for distributed optimization. IEEE Journal of Selected Topics in Signal Processing 7(2), 230–242 (2013) 23. Marden, J.R., Shamma, J.S.: Game theory and distributed control. Handbook of Game Theory 4, 2818–2833 (2013) 24. Mertikopoulos, P., Sandholm, W.: Learning in Games via Reinforcement and Regularization. Mathematics of Operations Research 41(4), 1297–1324 (2016) 25. P. Coucheney, B. Gaujal, Mertikopoulos, P.: Penalty-Regulated Dynamics and Robust Learning Procedures in Games. Mathematics of Operat. Research 40(3), 611–633 (2015) 26. Parise, F., Gentile, B., Grammatico, S., Lygeros, J.: Network aggregative games: Distributed convergence to Nash equilibria. In: Proc. of the 54th IEEE CDC. pp. 2295–2300 (2015) 27. Pavel, L.: Game theory for control of optical networks. Birkhäuser-Springer (2012) 28. Pavlov, A., Marconi, L.: Incremental passivity and output regulation. System & Control Letters 57, 400– 409 (2008) 29. R. Cominetti, E. Melo, Sorin, S.: A payoff-based learning procedure and its application to traffic games. Games and Economic behaviour 70(1), 71–83 (2010) 30. Salehisadaghiani, F., Pavel, L.: Distributed Nash equilibrium seeking: A gossip-based algorithm. Automatica 72, 209–216 (2016) 31. Scutari, G., Facchinei, F., Pang, J.S., Palomar, D.P.: Real and Complex Monotone Communication Games. IEEE Trans. on Inform. Theory 60(7), 4197–4231 (2014) 32. Shamma, J.S., Arslan, G.: Dynamic fictitious play, dynamic gradient play, and distributed convergence to Nash equilibria. IEEE Trans. on Automatic Control 50(3), 312–327 (2005)

200

L. Pavel

33. Sorin, S.: Exponential weight algorithm in continuous time. Mathematical Programming 116(1), 513–528 (2009) 34. Swenson, B., Kar, S., Xavier, J.: Empirical centroid fictitious play: An approach for distributed learning in multi-agent games. IEEE Trans. on Signal Processing 63(15), 3888–3901 (2015) 35. Wang, X., Xiao, N., Wongpiromsarn, T., Xie, L., Frazzoli, E., Rus, D.: Distributed consensus in noncooperative congestion games: an application to road pricing. In: IEEE 10th Int. Conf. on Control and Automation (ICCA). pp. 1668–1673 (2013) 36. Zhou, Z., Mertikopoulos, P., Moustakas, A.L., Bambos, N., Glynn, P.: Mirror descent learning in continuous games. In: 56th IEEE Conf. on Decision and Control. pp. 5776–5783 (Dec 2017)

Impact of Social Connectivity on Herding Behavior Deepanshu Vasal

Abstract Information cascades have been studied in the literature where myopic selfish users sequentially appear and make a decision to buy a product based on their private observation about the value of the product and actions of their predecessors. Bikhchandani et. al (1992) and Banerjee (1992) introduced such a model and showed that after a finite time, almost surely, users discard their private information and herd on an action asymptotically. In this paper, we study a generalization of that model where we assume users are connected through a random tree, which locally acts as an approximation for Erdös–Rényi random graph when the degree distribution of each vertex of the tree is binomial and as the number of nodes grows large. We show that informational cascades on such tree-structured networks may be analyzed by studying the extinction probability of a certain branching process. We use the theory of multi-type Galton–Watson branching process and calculate the probability of the tree network falling into a cascade. More specifically, we find conditions when this probability is strictly smaller than 1 that are in terms of the degree distributions of the vertices in the tree. Our results indicate that groups that are less tightly knit, i.e., have lesser connection probability (and as a result have lesser diversity of thought), tend to herd more than the groups that have more social connections. Keywords Information cascades · Multi-type Galton–Watson process · Social learning

Supported in part by Department of Defense grant W911NF1510225 and Simon’s Foundation grant 26-7523-99. D. Vasal (B) Department of Electrical and Computer Engineering, University of Texas, Austin, TX 78751, USA e-mail: [email protected] © Springer Nature Switzerland AG 2019 J. Walrand et al. (eds.), Network Games, Control, and Optimization, Static & Dynamic Game Theory: Foundations & Applications, https://doi.org/10.1007/978-3-030-10880-9_12

201

202

D. Vasal

1 Introduction People’s decisions in the real world are influenced not only by their own opinions or predilections, but also by the decisions and opinions of their peers. For example, decisions such as buying a product, voting for a candidate, or choosing a restaurant among many others are clearly a combined product of people’s individual preferences and that of their peers. It is commonly observed that people tend to “herd”; i.e., follow the majority opinion. For instance, if there is a product on an online retail Web site with many positive reviews, it is highly probable that it will be chosen by a new buyer. Similarly, it is more likely that a person will choose to go to a restaurant that is recommended by many of their friends. With the ever-increasing presence of Internet platforms and social media apps, and the very real consequences of people making decisions based on dubious information that spreads via such apps, it is even more important to understand how people’s actions are influenced by their peers and these actions further affect the spread of information in the network. In the seminal papers [3, 4], the authors considered a simple model for decision making which demonstrated a herding behavior among users: Users flocked to the same action regardless of their private opinions, and furthermore, with a nonzero probability this action may be wrong despite all the users being perfectly rational. The model considered a binary state of the world which represents value of a product, and infinite sequence of users that appear in some fixed order. Each user (or agent) makes an independent noisy observation of the state of the world. In addition to this observation, agents have access to the entire history of decisions by past agents. The current agent then makes a binary decision to either buy or not buy the product based on their private observation and the historical information about past actions. An agent’s decision, once made, is fixed and does not change with time. In this model, the authors of [3, 4] showed that a surprising phenomenon of herding (or cascading) occurs: With probability one, there is a finite time such that all agents that appear after this time choose to ignore their private observations and simply copy the action of the previous agent. This model was later generalized in [1, 20], and similar cascading phenomena were shown to exist in these general models as well. Agents herding to the wrong action may be highly undesirable for a society, and there has been a considerable amount in work in developing strategies that avoid herding. Peres et al. in [18] consider this question for the model in [3, 4]. They propose a stochastic strategy such that each user independently and randomly discards the information of actions of previous players and plays an action solely based on its private information. It is shown that this strategy avoids cascades and that an asymptotic rate for the probability of learning by the users as a team may be calculated. Authors in [22] study an ergodic version of this model where users also have the ability to post a review about the product at some cost. They design incentives to be given by social planner that aims to align the goals of strategic players with the team objective and incentivizes users to reveal their private information through the reviews. Acemoglu et al. in [2] study learning dynamics under two classes of rating systems: full history, where customers see the full history of reviews, and summary

Impact of Social Connectivity on Herding Behavior

203

statistics, where the platform reports some summary statistics of past reviews. They provide conditions for asymptotic learning under both cases. Harel et al. in [10] study the speed of learning with myopically selfish agents acting repeatedly and show that only a fraction of players’ private information is transmitted through their actions, where this fraction goes to zero as the number of players goes to infinity, demonstrating groupthink behavior. Dhounchak et al. in [7] explore the use of multi-type branching processes to analyze viral properties of content in a social network, with and without competition from other sources. Vasal and Anastasopoulos in [21] consider a dynamic model where users with independent Markovian types make noisy observations of their types privately and publicly observe actions of everybody else. The users are not myopic anymore and participate throughout the game through an endogenous or exogenous process. Based on a sequential decomposition methodology to compute perfect Bayesian equilibria, they characterize set of informational cascades for their dynamic model. In this paper, we study the problem of social learning and specifically herding behavior when the social network corresponding to the agents is a tree. Herding or cascading on such a graph may be interpreted as spreading of a rumor, or infection on the graph. Infection spread processes have been studied extensively in network science [12, 17], with problems of interest being maximizing (or minimizing) the spread of an infection [13], or identifying (or hiding) sources of infections [5, 8, 19]. In our model, we consider a random rooted tree where each node corresponds to an agent. We call the root of this tree the source (or the ancestor). Agents make private observations of the state of the world, and these observations are independent and identically distributed conditioned on the state of the world. Furthermore, an agent can observe the (possibly noisy) history of actions of all its ancestors, i.e., of all the agents that lie on the path joining the source node to the agent’s node. Based on the agent’s private observation and observation past actions, the agent performs an action, such as to buy or not buy a product. As in the previously described models, herding occurs if an agent performs the same action as their parent node, without considering their private observation. We show that probability of users herding reduces to studying extinction probability in a branching process, which we characterize using Galton–Watson branching process theory. In the setting we consider, users may not herd to the same action: Different branches of the tree may herd to different actions. In the infection spread interpretation, this analysis yields which of two competing infections (or products) becomes eventually dominant. It is known that random trees are good local approximation for random graphs, for instance a graph locally behaves like a random tree whose degree distribution is binomial (n, np ) if the probability of connection in an Erdös–Rényi graph1 is p [6]. Our model can be considered as a subset of the random observation model considered in [1]. Authors in [1] consider a sequential model where every user observes a random subset of the actions of the previous users. They provide sufficient conditions of expansive observations for probability of asymptotic learning (where users 1 Erdös–Rényi random graph with parameter p is defined as an undirected random graph where each pair of vertices is connected independently with probability p.

204

D. Vasal

Fig. 1 Galton–Watson branching process

correctly learn the state of the system) to be 1. In our model, we first consider a general observation model which we later specialize to a binary symmetric channel and provide conditions when users fall into a cascade (which may not correspond to the right action) is 1 and when it is strictly smaller than 1. Thus, our focus is not on asymptotic learning, rather on information cascades. The paper is structured as follows. In Sect. 4, we define equilibrium strategies of the players and other preliminary notation used later in the paper. In Sect. 3, we use the theory of multi-type Galton–Watson process to study this general model. In Sect. 5, we specialize our results to a binary symmetric channel with erroneous actions where players observe actions of their ancestors through a binary symmetric channel. We conclude in Sect. 6. All proofs are presented in Appendices (Fig. 1).

2 Model A Galton–Watson tree is defined as a branching process which starts with one node and each node of the tree gives birth to D children, where D is a discrete random variable with known probability generating function (PGF) φ D , i.e., φ D (s) = E[s D ]. Let there be a product whose value is V ∈ {0, 1} which is random such that P(V = 0) = P(V = 1) = 1/2. There exists infinite selfish players who appear sequentially in a predefined order such that each player acts on one of the node of a tree with a random degree whose probability generating function (PGF) is given by φ D , where P(D = 0) = 0.2 We denote kth player at stage t of the tree by ptk . Player ptk privately receives an observation xtk ∈ X about the value of the product through a general channel Q(xtk |V = v).3 Equivalently, we can assume that players observe qtk := P(V = 1|xtk ) such that qtk conditioned on value v is distributed according to CDF F v , v ∈ {0, 1}, (i.e., F v (q) := P(qtk ≤ q|V = v) ) where ¯ ⊆ [0, 1], and 0 ≤ b < 1/2 < b¯ ≤ 1. We also assume that F 0 , F 1 supp(F v ) = [b, b] are not identical and are absolutely continuous with respect to each other so that no signal is fully revealing. Each player appears exactly once in the game and shares its actions with all its descendants on the tree. Thus, each player observes actions of its k ancestors, denoted by a P( pt ) , and its own private observation, xtk or qtk . Based on this 2 This assumption excludes the possibility the tree goes extinct due to lack of child and thus implies that probability of extinction is the same as probability of a cascade (as justified later). 3 We

assume that the Q kernels have monotonicity property, as in [4], which implies is increasing in x. This fact is used in the proof of Lemma 1.

Q(X tk =x|V =1) Q(X tk =x|V =0)

Impact of Social Connectivity on Herding Behavior

205

information, it takes an action atk ∈ {0, 1} to either buy the product (atk = 1) or not the system, implying that it is myopic in buy the product (atk = 0) and then it leaves ⎧ ⎨ 1 if atk = 1, v = 1, nature. It gets a reward Rtk (atk , v) = −1 if atk = 1, v = 0, ⎩ 0 if atk = 0.

3 Preliminaries: Multi-type Galton–Watson Process In this section, we will review the theory of multi-type Galton Watson population process [16] which will be used in Sect. 4 to study the probability of falling into cascades. A Galton–Watson tree is defined as a branching process which starts with one node and each node of the tree gives birth to D children, where D is random with known probability generating function (PGF) φ D . Galton and Watson in [14] used this branching process to study extinction rate of family names. Since then it has been used to study nuclear reactions, population growth, and cosmic rays [11]. In a single type Galton–Watson process, all nodes are of the same type. It is known that the tree goes extinct almost surely if E[D] < 1 and it goes extinct with a probability strictly less than 1 if E[D] > 1, where this probability is given by the smallest solution of the fixed-point equation y = φ D (y). A multi-type Galton–Watson process is defined as a branching process where each node of the tree could be of multiple type (finite, countable, or uncountable) and gives birth to children of any of the types with certain probability. Moyal in [16] studied this process for arbitrary type in [0, 1]. Suppose every node has a type x ∈ [0, 1], and it gives birth to n children of types (y1 , . . . , yn ) with probability P1(n) (dy n |x), where yi ∈ [0, 1]. Here the subscript (1) in P1(n) (dy n |x) defines the depth of the tree, and superscript (n) defines the number of children. Then for any function ξ : [0, 1] → [0, 1], the probability generating functional G 1 : ([0, 1] → [0, 1]) × [0, 1] → [0, 1] of this process at depth 1 with 1 ancestor be defined as follows, G 1 (ξ|x) :=

∞   n=0

Xn

ξ(y1 ) . . . ξ(yn )P1(n) (dy n |x)

(1)

and the probability generating functional of this process at depth k with r ancestors, where x r = (x1 , x2 , . . . , xr ), is defined as G k (ξ|x r ) :=

∞   n=0

Xn

ξ(y1 ) . . . ξ(yn )Pk(n) (dy n |x r )

(2)

Then it is shown in [16] that for every stage, the probability generating functional is multiplicative in the number of ancestors, i.e.,

206

D. Vasal

G 1 (ξ|x r ) =

r 

G 1 (ξ|xi )

(3)

i=1

and for each ancestor, it is compositional in the depth of the tree, i.e., G k (ξ|x) = G j {G k− j [ξ|·]|x}

(4)

Moreover, the asymptotic extinction probability, q(x), is given by the minimal nonnegative solution of the functional equation ξ(x) = G 1 (ξ|x)

(5)

4 Analysis : Informational Cascades as Extinction Probability In this section, we show that the problem of occurrence of informational cascades on random trees is equivalent to extinction probability of an appropriately defined random tree. Then using the results from multi-type Galton–Watson process, we find conditions when the random tree falls into an informational cascade. In our model description, we define two equivalent observation models, one through observation kernel Q(xtk |V = v) and other through the CDF F v of qtk conditioned on state v, where qtk ∈ [0, 1], qtk := P(V = 1|xtk ). In the following lemma, we first show how CDF of observations xtk is related to F v . Lemma 1 v

F (q) =

FXv



Q(·|V = 1) Q(·|V = 0)

−1

q 1−q

(6)

Proof Please see Appendix A.

4.1 Perfect Bayesian Equilibrium In our model, user ptk observes actions of its ancestors a P( pt ) , and its own private observation, xtk and therefore takes decision atk through a policy gtk of the form k k atk = gtk (a P( pt ) , xtk ). Its objective is to maximize its expected reward Egt {Rtk }. Since users are (myopically) strategic and there is asymmetry of information, this results in a dynamic game of asymmetric information. An appropriate notion of equilibrium for such games is perfect Bayesian equilibrium [9], which consists of an equilibrium belief profile and an equilibrium strategy profile. In general, such equilibria are difficult to compute because of interdependence of equilibrium strategies and equilibrium k

Impact of Social Connectivity on Herding Behavior

207

beliefs, which cannot be sequentially decomposed, rendering computing equilibria for such problems difficult (see [9, chap. 8]). However, for this game (and other such games in the literature where users are myopic), computing such equilibria is easier as equilibrium strategies and beliefs can be easily decoupled shown as follows. User k ptk ’s objective to maximize Egt {Rtk } over strategies gtk is equivalent to maximizing k k E{Rtk |a P( pt ) , xtk , atk } over actions atk for every history (a P( pt ) , xtk ). Thus, E{Rtk |a P( pt ) , xtk , atk } k k 1.P(V = 1|a P( pt ) , xtk ) − 1.P(V = 0|a P( pt ) , xtk ) if atk = 1 = k 0 if at = 0 k 2P(V = 1|a P( pt ) , xtk ) − 1 if atk = 1 = k 0 if at = 0. k

(7a) (7b)

Equilibrium decision strategy: Equation (17) implies that user ptk takes action atk = 1 k if P(V = 1|a P( pt ) , xtk ) > 1/2. We assume that in case of ties, i.e., when P(V = k 1|a P( pt ) , xtk ) = 1/2, it chooses an action based only on its private information (as shown below in (8)). k We next define public belief πtk ∈ [0, 1] as πtk := P(V = 1|a P( pt ) ). Similar to k [1, Theorem 1], it can be shown that the decision rule P(V = 1|a P( pt ) , xtk ) > 1/2 k is equivalent to P(V = 1|a P( pt ) ) + P(V = 1|xtk ) > 1, i.e., πtk + qtk > 1. A proof is provided for convenience in Appendix C. Thus, player ptk takes action according to the following equilibrium decision rule g, where atk =

1 if qtk > 1 − πtk or qtk = 1 − πtk and qtk ≥ 1/2 0 if qtk < 1 − πtk or qtk = 1 − πtk and qtk < 1/2.

(8)

In the following lemma, we find the update of the belief πtk under above strategy. Lemma 2 Under the equilibrium policy g defined above and for a ∈ {0, 1}, there exist functions ψa : [0, 1] → [0, 1] such that for any action atk at time t, the common belief πtk is updated as  = ψatk (πtk ). πt+1

(9)

Proof Please see Appendix B. The update of πtk satisfies the following property that when (1 − πtk ) is outside the ¯ then the update of πtk for both actions leads / [b, b], support of F v , i.e., for (1 − πtk ) ∈  k to same belief, i.e., πt+1 = ψatk (πt ) = πtk . Furthermore, ψatk (0) = 0, ψatk (1) = 1∀atk . Thus, in this process, the probability of actions of the users is determined using the equilibrium strategies as follows. For v ∈ {0, 1}, probability that a user takes action 0 when the value of the product is v is given by

208

D. Vasal

P g (atk = 0|V = v, a P( pt ) ) k

= P(qtk < 1 − πtk |V = v) + P(qtk = 1 − πtk , qtk ≤ 0.5|V = v)

(10)

=

(11)

=

P(qtk v

b, ¯ as “extinct” from the point / [b, b] ψ0 (m) with probability 1. We define types 1 − m ∈ of view of the branching process. Lemma 3 Informational cascades are equivalent to extinction probability in the branching process defined above. ¯ is in cascade, it Proof Since player of any other type m such that 1 − m ∈ / [b, b] will only have one type of children with the same type. This is because ψ0 (m) = ¯ as noted earlier. Thus, types in 1 − m ∈ ¯ are / [b, b] / [b, b] ψ1 (m) = m for 1 − m ∈ ¯ absorbing types, whereas for 1 − m ∈ [b, b], ψ0 (m) < m < ψ1 (m). This implies the statement of the lemma. Thus, using multi-type Galton–Watson branching process theory described in Sect. 3, starting from type x, the probability of falling into a cascade, q(·), is given by the minimal non-negative solution of the functional equation ξ(x) = G 1 (ξ|x)

(13)

ξ(y1 ) . . . ξ(yn )P1(n) (dy n |x)

(14a)

where G 1 (ξ|x) =

∞   n=0

4 Note

Xn

that in the case of more than two actions, there will be corresponding number of types of children.

Impact of Social Connectivity on Herding Behavior

=

∞ 

P(D = d)[F 1 (1 − x)ξ(ψ0 (x))d + (1 − F 1 (1 − x))ξ(ψ1 (x))d ]

d=0 1

= F (1 − x)φ D (ξ(ψ0 (x))) + (1 − F 1 (1 − x))φ D (ξ(ψ1 (x))).

209

(14b) (14c)

Since G 1 (·) is a probability generating functional, ξ(x) = 1 is always a solution of the fixed-point equation (5). Furthermore, since we assume that P(V = 1) = 1/2, the probability of cascades is given by q(1/2), where q(·) is given by the minimal non-negative solution of the functional equation (5).

4.3 Finite Types Definition 1 Let Q be the class of observation channels such that for every given ¯ there exists a finite set Z f (Q) ⊂ (b, b) ¯ such that 0.5 ∈ Q ∈ Q with support [b, b], Z f (Q) and ∀a ∈ {0, 1} and ∀x ∈ Z f (Q), ψa (x) ∈ Z f (Q). We conjecture that such channels can be characterized as ψ1k (x) = ψ0−1 (x) for ¯ or ψ0k (x) = ψ1−1 (x) for some k such that ψ0k (0.5) > b. some k such that ψ1k (0.5) < b, Note that one example of such a channel is the case of the binary symmetric channel (BSC), as described below. In general, it appears hard to construct a channel with finite types. Note, however, that the general theory from subsection B always holds, except that the probability of extinction is hard to compute when the channel is not finite type. Analyzing the BSC: Consider the binary symmetric channel with cross over probability p ∈ [0.5, 1]. It is easy to show that this channel belongs to the set Q 2 2 ¯ 0.5, p, p2 p+ p¯ 2 }. Let ri , i ∈ {1, . . . , 5} be the elements of with Z f (Q) = { p2 p+¯ p¯ 2 , p, Z f (Q). Then ⎧ ⎧ ⎨ 0 if x < p¯ ⎨ 0 if x < p¯ F 0 (x) = p if p¯ ≤ x < p , F 1 (x) = p¯ if p¯ ≤ x < p ⎩ ⎩ 1 if p < x 1 if p < x

(15)

(Note that F 1 stochastically dominates F 0 as expected). Then ⎧ ⎨x

⎧ 0 ≤ x < p¯ ⎨ x 0 ≤ x < p¯ xp x p¯ p¯ ≤ x < p p ¯ ≤ x ¯ < p ψ0 (x) = x¯ p+x , ψ (x) = 1 ¯ p p¯ ⎩ ⎩ x¯ p+x x p≤x 0. ¯ D ( p) ¯ > 0 and f (y) is monotonically increasing Since f (0) = p(φ D ( p)) + pφ and strictly convex, there exists another fixed point of f (y) = y in the range (0, 1) if d f (y) | y=1 > 1. The smallest fixed point is stable since the derivative of and only if dy f (y) is less than 1 at that point, and thus by Theorem 1, it represents the probability of cascading. The derivative of f(y) at y = 1 is given by, d d f (y) = p(φ D ( p + pφ ¯ D (y))) + pφ ¯ D ( pφ D (y) + p) ¯ (29) dy dy d f (y) = p(φD ( p + pφ ¯ D (y))) pφ ¯ D (y) + pφ ¯ D ( pφ D (y) + p) ¯ pφD (y) (30) dy d f (y) | y=1 = 2 p pφ ¯ D (1)φD (1) dy

(31) 

Thus, the tree cascades with probability 1 if and only if 2 p p(φ ¯ D (1))2 ≤ 1, i.e., φ D (1) = E[D] ≤ √21p p¯ . Otherwise the tree cascades with a nonzero probability which is the smallest fixed point of f (y) = y. 

Appendix E Lemma 4 f (y) = pφ D ( p + pφ ¯ D (y)) + pφ ¯ D ( pφ D (y) + p) ¯ increasing and strictly convex for y > 0.

is

monotonically

Proof We will show that pφ D ( p + pφ ¯ D (y)) is strictly increasing and convex, and the proof of the other part is identical. d pφ D ( p + pφ ¯ D (y)) (32a) dy 



= pφ D ( p + pφ ¯ D (y)) pφ ¯ D (y) >0

(32b) (32c)

216

D. Vasal

where the last inequality is true since φ D (y) = E[y D ] > 0 and E[Dy D−1 ] > 0 since y > 0 and D ≥ 1 a.s..

d φ (y) dy D

=

d2 pφ D ( p + pφ ¯ D (y)) dy 2 d   = pφ D ( p + pφ ¯ D (y)) pφ ¯ D (y) dy 

    = p p¯ φ D ( p + pφ ¯ D (y))φ D (y) + φ D ( p + pφ ¯ D (y)) p(φ ¯ D (y))2

(34)

>0

(36)

(33)

(35)

where the last inequality is true by similar arguments as before.

References 1. Acemoglu, D., Dahleh, M.A., Lobel, I., Ozdaglar, A.: Bayesian learning in social networks. The Review of Economic Studies 78(4), 1201–1236 (2011) 2. Acemoglu, D., Makhdoumi, A., Malekian, A., Ozdaglar, A.: Fast and slow learning from reviews. Tech. rep., National Bureau of Economic Research (2017) 3. Banerjee, A.V.: A simple model of herd behavior. The Quarterly Journal of Economics pp. 797–817 (1992) 4. Bikhchandani, S., Hirshleifer, D., Welch, I.: A theory of fads, fashion, custom, and cultural change as informational cascades. Journal of Political Economy 100(5), pp. 992–1026 (1992), http://www.jstor.org/stable/2138632 5. Bubeck, S., Devroye, L., Lugosi, G.: Finding adam in random growing trees. Random Structures & Algorithms 50(2), 158–172 (2017) 6. Clauset, A., Newman, M.E., Moore, C.: Finding community structure in very large networks. Physical review E 70(6), 066111 (2004) 7. Dhounchak, R., Kavitha, V., Altman, E.: A viral timeline branching process to study a social network. arXiv preprint arXiv:1705.09828 (2017) 8. Fanti, G., Kairouz, P., Oh, S., Ramchandran, K., Viswanath, P.: Hiding the rumor source. IEEE Transactions on Information Theory (2017) 9. Fudenberg, D., Tirole, J.: Perfect bayesian equilibrium and sequential equilibrium. journal of Economic Theory 53(2), 236–260 (1991) 10. Harel, M., Mossel, E., Strack, P., Tamuz, O.: The speed of social learning. arXiv preprint arXiv:1412.7172 (2014) 11. Harris, T.E.: The theory of branching processes. Courier Corporation (2002) 12. Jackson, M.O.: Social and economic networks. Princeton university press (2010) 13. Kempe, D., Kleinberg, J., Tardos, É.: Maximizing the spread of influence through a social network. In: Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining. pp. 137–146. ACM (2003) 14. Kendall, D.G.: Branching processes since 1873. Journal of the London Mathematical Society 1(1), 385–406 (1966) 15. Le, T.N., Subramanian, V., Berry, R.: The impact of observation and action errors on informational cascades. In: Decision and Control (CDC), 2014 IEEE 53rd Annual Conference on. pp. 1917–1922 (Dec 2014). https://doi.org/10.1109/CDC.2014.7039678 16. Moyal, J.: Multiplicative population chains. In: Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences. vol. 266, pp. 518–526. The Royal Society (1962)

Impact of Social Connectivity on Herding Behavior

217

17. Newman, M.: Networks: an introduction. Oxford university press (2010) 18. Peres, Y., Racz, M.Z., Sly, A., Stuhl, I.: How fragile are information cascades? arXiv preprint arXiv:1711.04024 (2017) 19. Shah, D., Zaman, T.: Rumors in a network: Who’s the culprit? IEEE Transactions on information theory 57(8), 5163–5181 (2011) 20. Smith, L., Sörensen, P.: Pathological outcomes of observational learning. Econometrica 68(2), 371–398 (2000). https://doi.org/10.1111/1468-0262.00113 21. Vasal, D., Anastasopoulos, A.: Decentralized Bayesian learning in dynamic games. In: Allerton Conference on Communication, Control, and Computing (2016), https://arxiv.org/abs/1607. 06847 22. Vasal, D., Subramanian, V., Anastasopoulos, A.: Incentive design for learning in userrecommendation systems with time-varying states. In: 2015 49th Asilomar Conference on Signals, Systems and Computers. pp. 1080–1084 (Nov 2015). https://doi.org/10.1109/ACSSC. 2015.7421305

A Truthful Auction Mechanism for Dynamic Allocation of LSA Spectrum Blocks for 5G Ayman Chouayakh, Aurélien Bechler, Isabel Amigo, Loutfi Nuaymi and Patrick Maillé

Abstract Licensed shared access is a new frequency sharing concept that allows mobile network operators (MNOs) to use some of the spectrum initially allocated to other incumbents, after obtaining a temporary license from the regulator. The allocation is made among groups such that two base stations in the same group can use the same spectrum simultaneously. In this context, different auction schemes were proposed; however, they consider the scenario in which the regulator has one and only one block of LSA frequency to allocate. In this paper, we remove this hypothesis: We suppose that the regulator has K identical blocks of spectrum to allocate, and we propose a truthful auction mechanism based on the Vickrey–ClarkeGroves mechanism (VCG). We evaluate the efficiency of our mechanism in terms of social welfare, which depends on the allocation rule of the mechanism. Simulations show that the efficiency of the proposed mechanism is at least 60% of that of VCG, which is known to be optimal.

1 Introduction Mobile data traffic continues to grow exponentially; by 2020, there will be nearly eight times more mobile Internet traffic than in 2016 [1]. Hence, more radio spectrum is needed to carry this traffic. In 2011, the Radio Spectrum Policy Group (REPG) A. Chouayakh (B) · A. Bechler Orange Labs, Chatillon, France e-mail: [email protected] A. Bechler e-mail: [email protected] I. Amigo · L. Nuaymi · P. Maillé IMT Atlantique, Brest, France e-mail: [email protected] L. Nuaymi e-mail: [email protected] P. Maillé e-mail: [email protected] © Springer Nature Switzerland AG 2019 J. Walrand et al. (eds.), Network Games, Control, and Optimization, Static & Dynamic Game Theory: Foundations & Applications, https://doi.org/10.1007/978-3-030-10880-9_13

219

220

A. Chouayakh et al.

[2] has proposed a new sharing concept called Licensed Shared Access (LSA) [3]. Under this concept, an incumbent can temporarily share his spectrum with mobile network operators (MNOs). Before accessing the incumbent’s spectrum, MNOs have to obtain a license from the regulator. This license includes the duration of sharing, the geographical area, and the conditions of sharing. The LSA concept guarantees to the incumbent and the LSA licensee a certain level of QoS according to the LSA license. This differs from the traditional definition of sharing [4] in which the second user (MNOs in our case) has the right to access other spectrum without a licensing process and without any guarantees; i.e., the duration of sharing is not defined and the access to spectrum is not guaranteed which is not desirable from the point of view of MNOs. In this context, auctioning the LSA spectrum is a natural approach to allocate it to the MNOs. A well-designed auction mechanism should be truthful; i.e., each player should not be able to play the system by bidding strategically. Also it has to take into account the spacial reusability of spectrum; i.e., two base stations could use the same spectrum if they do not cause interference to each other. In [5–7], the authors designed mechanisms to allocate LSA spectrum, that can be applied in the case where there is one and only one block to allocate. On the other hand, in [8] we have designed and analyzed a truthful scheme for the case where the spectrum is infinitely divisible. But reality lies between those two extremes: Spectrum can be split in several sub-bands or blocks, that have a predetermined size. Therefore, in this paper, we suppose that the regulator has K identical blocks to allocate. Identical meaning that there is no preference over blocks from the point of view of base stations [9, 10]. The rest of this paper is organized as follows: Sect. 2 presents the system model. In Sect. 3, we present the proposed mechanism and discuss its properties. Simulation results are shown in Sect. 4, and we conclude and provide some perspectives in Sect. 5.

2 System Model 2.1 Grouping Players Before the Auction We consider N base stations (BS) of different operators in competition to obtain altogether K identical blocks of spectrum under the LSA scheme. (In this paper, we will use player and base station interchangeably.) When designing a spectrum allocation mechanism, we have to take into account spectrum space reusability: If some base stations interfere with each other, then they cannot use the bandwidth simultaneously. This can be easily captured in a model by using an interference graph, of which an example is shown in Fig. 1; base stations are represented by vertices, and an edge between two vertices means that those two base stations interfere. In our example, since base stations {1, 3, 5} do not interfere with each other, then they could

A Truthful Auction Mechanism for Dynamic Allocation …

221

Fig. 1 Some base stations and their coverage areas (left), and the corresponding interference graph

represent a “group” and can use the same blocks simultaneously. Hence, the allocation could be made for groups. We denote by M the number of groups. An example of groups construction for Fig. 1 is g1 = {1, 3, 5}, g2 = {2, 5}, and g3 = {1, 4}. Note that there exists more than one method for group construction or grouping. In Sect. 3.1, we present our algorithm for grouping.

2.2 Preferences of Operators We consider K identical blocks of spectrum to allocate under the LSA scheme, and a quasi-linear utility model: The utility of a bidder is the difference between his valuation for the resource he gets, and the price he is charged. Each base station i has a private vector-valuation vi composed by K elements: The first element vi (1) represents the value of one block. The n th element vi (n) (n > 1) represents the value of the base station i for a n th block given that it has already n − 1 blocks. We suppose that the value of a block, for a base station, decreases with the number of blocks already obtained. This corresponds to a discretization of concave valuation functions for spectrum [11], as illustrated in Fig. 2. Finally, we adopt a quasi-linear utility model; if a base station i obtains n i blocks and pays pi , its utility is then: ui =

ni 

vi (n) − pi ,

n=1

with pi =

ni 

pi,n and pi,n is the payment of the base station i for its nth block. In

n=1

particular, an operator obtaining no block gets a utility equal to zero.

2.3 Steps of the Auction In our proposed mechanism, LSA frequency blocks are numbered by the regulator from 1 to K . The steps of the auction can be summarized as follows; only notations

222

Block size

vi (4)

vi (5)

vi (6)

vi (3)

Valuation

Fig. 2 An example of a concave valuation function of obtained spectrum, and the corresponding per-block valuations vi (n) for a player i

A. Chouayakh et al.

vi (2) vi (1) Obtained spectrum (MHz)

are given in the following and not how each step is implemented. Each step is detailed in Sect. 3. 1. Group construction: From the interference graph, the regulator constructs groups with respect to the constraint “each two base stations in the same group do not interfere with each other.” 2. Bid collecting: Bidders are asked to declare their valuation for blocks in a bid vector bi . The nth element bi (n) (n > 1) represents the bid of the base station i for a nth block given that it has already n − 1 blocks. 3. Block allocation: We denote by n h the number of blocks allocated to group h, gh , by Y the allocation matrix for players, yi,n = 1 means that block number n is allocated to player i and by X the allocation matrix for groups, x h,n = 1 means that block number n is allocated to group h, so that yi,n = x h,n 1i∈gh . After that allocation phase, each player i is allocated a number n i of blocks. 4. Payment: Each player i is charged a price pi , computed so as to make the mechanism truthful, as detailed later. All used notations are summarized in Table 1.

2.4 Objectives of the Regulator We propose to focus on two objectives for the regulator: • Maximizing the efficiency the social welfare W , that is the sum of all individual utilities the operator’s (the sum of players’ payments), or mathematically  N including ni N K n−1 i=1 i=1 n=1 vi (1 + t=1 yi,t )yi,n . n=1 vi (n) which can be written as • Preserve truth-telling: For each player i, proposing a bid bi = vi maximizes his utility.

A Truthful Auction Mechanism for Dynamic Allocation …

223

Table 1 Notations used in the paper M N K bi

Number of groups Number of base stations Number of blocks Bid vector of base station i, bi (n), is the bid for a nth block given that he has obtained n − 1 blocks The hth group Allocation-player matrix (N × K ) yi,n = 1 means that player i has obtained the nth block Allocation-group matrix (M × K ) x h,n = 1 means that group h has obtained the n th bloc Number of blocks allocated to player i Number of blocks allocated to group h Groupbid vector of gh ,

gh Y X ni nh Bh

3 Proposed Mechanism In this section, we develop some theoretical results and we present the proposed mechanism.

3.1 NP-Hardness of Optimal Allocation with General Groups The first step of the auction (grouping or setting of BS groups) is very important because it has a direct impact on the outcome in terms of allocation and payments. Let us introduce the following proposition: Proposition 1 For a given configuration of groups and with the hypothesis that a player can belong to more than one group, allocating resources in an efficient manner (choosing X which maximizes W ) is an NP-hard problem. Proof We start by the following equality: N  K 

vi (1 +

i=1 n=1

n−1 

yi,t )yi,n =

t=1

M  K 

Vh (n)x h,n ,

(1)

h=1 n=1

In fact, N  K  i=1 n=1

vi (1 +

n−1  t=1

yi,t )yi,n =

N  K  i=1 n=1

vi (1 +

n−1  t=1

yi,t ) ×

M  h=1

x h,n 1i∈h =

K  M  n=1 h=1

x h,n Vh (n)

224

A. Chouayakh et al.

N  where Vh (n) = i=1 vi (1 + n−1 t=1 yi,t )1i∈gh , and the right-hand side of (1) is called the group welfare of the allocation. As a consequence, maximizing social welfare is equivalent to maximizing group welfare. However, maximizing the group welfare is an NP-hard problem. We show this by the reduction of the maximum coverage problem [12]. The maximum coverage problem is known to be NP-hard. It can be described as follows: Given a collection of sets S = S1, S2, . . . , Sm which may have common elements, select at most k of these sets such that the maximum number of elements are covered; i.e., the union of the selected sets has maximal size. We consider an instance of the previous problem and reduce it to our problem. The reduction may be made as follows: • • • •

The number of sets is the number of groups. An element of a set is a player. The number of sets to select is the number of blocks to allocate. Each player wants exactly one block, and his valuation equals 1.

Clearly, solving our problem leads to solving the previous problem.

3.2 Proposed Grouping Method As shown in Sect. 3.1, with allowing players to belong to more than one group, allocation in an efficient manner is an NP-hard problem. Hence, to avoid this problem we propose to add the constraint that a player could belong to one and only one group. For the grouping method: We sort in a ascending order all the base stations by the degree of interference, so base stations with low degree are ranked first and grouped together as much as possible. The detailed algorithm for grouping is as follows: Algorithm 1 Grouping algorithm implemented in this paper Ω: sorted base stations by increasing degree in the interference graph (ties broken randomly) Let us denote by h the current number of groups and set h = 1 for i = 1,i ≤ N ,i + + do for j = 1, j ≤ h, j + + do if vertex Ω(i) has no interference to any node in group g j then Put the vertex (i) into g j break else if j == h then h++ create one more group gh , put Ω(i) into gh end if end if end for end for

A Truthful Auction Mechanism for Dynamic Allocation …

225

3.3 Allocation When groups satisfy the one-group-per-player constraint, maximizing efficiency is simple: After collecting bids, and for each group h, the regulator  Nconstructs the bi (n)1i∈gh ). groupbid vector Bh which is the sum of bids of players (Bh (n) = i=1 Here, Bh (n) represents the bid of gh for his nth block. Intuitively, since groups are independent (there is no player in common), to maximize the efficiency, the regulator has to allocate blocks to the highest K bids among the M × K bids.

3.4 Payments Let us denote by C h the vector of competing bids facing group h, i.e., C h composed by the highest K bids of all others groups. C h is sorted in an ascending order. The number of blocks that a group wins is the number of competing bids he defeats. Since the allocation rule is efficient (once groups are fixed), to preserve truthfulness, we have to apply Vickrey–Clarke–Groves (VCG) [13] payment rule: Players should pay the “damage” in terms of efficiency they impose; i.e., each player pays his “social cost” (how much his existence hurts the others). In our context, the payment rule is simple and given by the following proposition. Proposition 2 With the proposed grouping method and according to VCG, the payment of a player i who belongs to group h is given by: pi =

nh  [C h (n) − (Bh (n) − bi (n))]+ n=1

Proof piV C G =

K N   j=i n=1

=

K  N  n=1 j=i

=

K  M 

b j (n)1n≤niaj − b j (n)1n≤niaj − (Bne )−i 1n≤Neia

n=1 e=1

=

K N  

b j (n)1n≤ni p j

j=i n=1 K  N 

b j (n)1n≤ni p

n=1 j=i



K  M  n=1 e=1

K 

K  E ia (n) − (E i p (n))−i

n=1

n=1

j

(Bne )−i 1n≤Nei p

226

A. Chouayakh et al.

where • n iaj represents the number of blocks obtained by player j when i is absent. ip

• n j represents the number of blocks obtained by player j when i is present. • E ia (n) is the sum of the declared valuations of players obtaining block n when i is absent. As an example, if block n is the jth block allocated to group h, then E ia (n) = Bh ( j). • E i p (n) is the sum of the declared valuations of players obtaining block n when i is present. Without loss of generality, we suppose that group h obtains the first n h blocks when i is absent and the first n h blocks when i is present. piV C G

=

n h 

Bh (n) − bi (n) +

n=1

K 

   nh K  C h (n) − Bh (n) − bi (n) + C h (n)

n=n h +1

n=1

n=n h +1

  nh  C h (n) − (Bh (n) − bi (n) = n=n h +1

=

+ nh   C h (n) − (Bh (n) − bi (n)) n=1

3.5 Example and Illustration Suppose we have five blocks to allocate to three groups which are composed by one, two, and three players, respectively. We want to compute and illustrate graphically the payments of all players. Graphical illustration was done for the first player of the third group only (it is similar for others). Numerical values are assumed as follows: • bids of the first group composed by player 1: {(25, 19, 10, 8, 2)} • bids of the second group (which is composed by players 2 and 3 : {(10, 9, 4, 3, 2), (11, 8, 3, 2, 1)} • bids of the third group composed by players 4, 5, and 6: {(13, 10, 9, 8, 5), (11, 8, 6, 5, 2), (9, 8, 5, 3, 2)} • B1 = {25, 19, 10, 8, 2} • B2 = {(21, 17, 7, 5, 3)} • B3 = {33, 26, 20, 16, 9} • C1 = {17, 20, 21, 26, 33} • C2 = {19, 20, 25, 26, 33} • C3 = {10, 17, 19, 21, 25}. Allocation: The fifth highest groupbid are {33, 26, 25, 21, 20}. Hence, for the allocation, we obtain one block for the first group, one for the second, and three for the third. The payments are given as follows:

A Truthful Auction Mechanism for Dynamic Allocation …

Group bid vector of g3 Group bid vector of g3 without player 4 Competing bids facing g3

30

Bids (see legend)

227

20

10

0

1

2

3

4

5

LSA frequency blocks

Fig. 3 Graphical illustration of different bids of the first player (player 4) of the third group, noted g3

• Players of the first group p1 = p1,3 = [17 − 0]+ = 17 • Players of the second group p2 = p2,4 = [19 − (21 − 10)]+ = 8 p3 = p3,4 = [19 − (21 − 11)]+ = 9 • Players of the third group: p4 = p4,1 + p4,2 + p4,5 = 0 + 1 + 8 = 9 p5 = p5,1 + p5,2 + p5,5 = 0 + 0 + 5 = 5 3 = 0 + 0 + 4 = 4. p6 = p6,1 + p6,2 + p6,5

The gray surface in Fig. 3 illustrates how much player one of the third group has to pay after obtaining three blocks. As this figure shows, for his first block he pays nothing because his group obtains this block whether he is present or not. However, for the second and the third blocks, he pays because his presence has changed the outcome.

3.6 Why Not Allocate LSA Blocks Separately? A question which may arise is: Why do not allocate blocks separately? Meaning collect allocating blocks separately will lead to lose truthfulness. Consider the following

228

A. Chouayakh et al.

examples, with two blocks and two players with valuations respectively (10, 2) and (9, 5). By allocating blocks separately and by bidding truthfully, player one obtains the first block and pays 9 and player two obtains the second block and pays 2. In this situation, the utility of player one is 1 and the utility of player two is 7. Clearly, for player one, any 5 < b1 (1) < 9 leads to higher utility, because he will obtain the second block and he pays 5 leading to a higher utility (5).

3.7 Discussion: Mechanism Properties Since our method consists on applying VCG to fixed groups, VCG properties are preserved.

3.7.1

Truthfulness

Proposition 3 The proposed mechanism is truthful; i.e., for each bidder i = 1, . . . , N , bidding one’s true valuation vector bi = vi is a dominant strategy.

3.7.2

Individual Rationality

A mechanism is individually rational if each player has an incentive to participate in the auction; i.e., he has a strategy guaranteeing him a nonnegative utility. Here, since players have a dominant strategy, this translates into u i (vi ) ≥ 0 ∀ vi ≥ 0. Proposition 4 The proposed auction mechanism is individually rational.

4 Performance Evaluation In this section, we evaluate the performance of the proposed method of grouping and compare it to the optimal one in terms of efficiency. We define the normalized efficiency E Nor as follows: E prop , (2) E Nor := E opt where E prop is the efficiency generated with the proposed method of grouping and E opt is the optimal efficiency which is obtained in two steps: 1. Extracting all maximal independents sets of the interference graph. This can be done using a software like Julia. Notice that this step depends on the density of the graph, i.e., the number of edges divided by all possible edges which is equal to N (N − 1). We can distinguish two special cases:

A Truthful Auction Mechanism for Dynamic Allocation …

229

• The graph is complete; i.e., each base station interferes with all the other base stations: In this situation, each base station represents a group. • There is no interference between base stations; then, in this situation, we have only one group composed by all base stations. In these two special cases, the proposed method of grouping coincides certainly with the optimal one. 2. Computing an optimal allocation for that set of groups. We denote by Bhu (n) the updated groupbid of group h for the bloc number n. To obtain the optimal allocation, the regulator has to solve the following problem: M K  

x h,n Bhu (n)

(3)

x h,n = 1, ∀n ∈ 1; K 

(4)

maximize X

n=1 h=1

subject to M  h=1

Bhu (1) =

N 

γi,h bi (1), ∀h ∈ 1; M

(5)

γi,h bi (n i ), ∀n ∈ 2; K , ∀h ∈ 1; M

(6)

γi,h x h,c , ∀n ∈ 2; K , ∀i ∈ 1; N 

(7)

i=1

Bhu (n) =

N  i=1

ni =

n−1  M  c=1 h=1

where γi,h = 1 if player i belongs to the group h. The first constraint ensures that each block is allocated to one and only one group. The second constraint means that for each player, his bids for the first block are exactly the first element of his bid vector. The third constraint is for updating bids; i.e., bid of a player for the current block depends on the number of blocks that he has obtained before. Finally, Eq. (7) represents the number of blocks obtained by a player till block n. This problem is a combinatorial and nonlinear problem because of the objective function and constraint (6). Having a linear formulation is an important task to solve effectively the problem with classic solvers. Transforming the problem into a integer linear problem (ILP) problem consists in developing an equivalent expression where objective function and all constraints are linear. In order to obtain a linear objective function, and we have introduced constraints (14), (15), and (16). Constraints (12) and (13) are introduced to linearize constraint (6): Constraint (6) is nonlinear because we have indexed by a variable (n i ), we have introduced a new variable an,i,e

230

A. Chouayakh et al.

which represents the number of blocks e obtained by player i before block n. Thus, the linear problem is represented as follows: maximize X

subject to

K  M 

wh,n

(8)

n=1 h=1 M 

x h,n = 1, ∀n ∈ 1; K 

(9)

h=1

Bhu (1) =

N 

γi,h bi (1), ∀h ∈ 1; M

(10)

i=1

Bhu (n) =

−1 N K  

γi,h bi (e + 1)an,i,e , ∀n ∈ 2; K 

(11)

i=1 e=0 n−1 

ean,i,e =

e=0 n−1 

n−1  M 

γi,h x h,c , ∀n ∈ 2; K ,

(12)

c=1 h=1

an,i,e = 1, ∀n ∈ 2; K , ∀i ∈ 1; N 

(13)

wh,n ≤ Mmax x h,n , ∀n ∈ 1; K , ∀h ∈ 1; M

(14)

e=0

wh,n ≤ wh,n ≥

Bhu (n), ∀n ∈ 1; K , ∀h ∈ 1; M Bhu (n) − Mmax (1 − x h,n ), ∀n ∈ 1;

K

(15) (16)

where Mmax is a constant such that Mmax > Bhu (n) ∀n ∈ 1; K , ∀h ∈ 1; M.

4.1 Simulation Settings As for the test settings, the computations have been made on a server of 16 processors Intel Xeon of CPU 5110 and clocked at 1.6 GHz each. The code has been written in Julia 0.5.0, and the solver used is Cplex 12.6 (default branch-and-cut algorithm [14]). Steps of simulation could be summarized as follows: • Fix d, N and K . • Generate an interference graph randomly with respect to N and d. • Create groups in two manner: by the proposed method and by extracting all maximum independents sets; the generation of the interference graph and the extraction of all maximum independents sets are made by Julia. • Generate bids. • Allocate blocks with respect to the two methods of grouping method. (using Cplex for the maximal independent sets).

A Truthful Auction Mechanism for Dynamic Allocation …

231

Simulations was made over 100 independent draws. A draw means that we generate a graph with respect to d and N , and for each player i, we generate K random variable drawn from the uniform distribution over the interval [0, 100] and we sorted those variables in a decreasing order to construct bi .

4.2 Simulation Results As we see from Fig. 4 and Tables 2 and 3, with the proposed method of grouping, efficiency is at least 60% of the optimal one. Table 2 shows the average resolution time of the optimal allocation which varies exponentially as a function of the number of blocks K . This illustrates the NP-hardness of the problem. Table 3 shows the impact of the density of the graph on the resolution time. To summarize, the resolution time of the optimal allocation depends not only on the number of blocks but also on the density of the graph. 1

d = 0.2 d = 0.4 d = 0.6 d = 0.8

0.5 0 0.6

0.7

0.8

0.9

1

Fig. 4 Cumulative density function of E Nor as a function of the density of the graph for K = 2 and N = 20 Table 2 Average normalized efficiency and resolution time as a function of the number of blocks for N = 30 and d = 0.4 K 1 2 3 5 Average E Nor Average resolution time (s)

0.76 0.07

0.80 12.7

0.773 87

0.81 1380

Table 3 Average normalized efficiency and resolution time as a function of the density of the graph for N = 20 and K = 3 d 0.2 0.4 0.6 0.8 Average E Nor Average resolution time (s)

0.829 1013.2

0.81 113.5

0.83 57.8

0.872 24.6

232

A. Chouayakh et al.

5 Conclusion In this paper, we have studied the case when a regulator has several identical blocks to allocate in the context of LSA. We have adapted to the case—the VCG mechanism— that is known to have good incentive and efficiency properties. Since the initial problems (group construction and allocation) are NP-hard, we have proposed a method of grouping which leads to a linear-complexity allocation. We have studied the impact of that method on the efficiency (social welfare): Simulation results show that with our method, efficiency is at least 60% of the optimal one. As directions for future works, we would like to relax some of the assumptions made. In particular, we want to treat the cases when blocks are non-identical, and when one player (operator) controls several base stations, which complicates the auction analysis since that player could coordinate several bids.

References 1. “Commission européenne - communiqué de presse.” http://europa.eu/rapid/press-release_IP16-207_en.htm, Feb 2016. 2. “ECC Report 205.” http://www.erodocdb.dk/Docs/doc98/official/pdf/ECCREP205.PDF, 2014. 3. M. Matinmikko, H. Okkonen, M. Malola, S. Yrjola, P. Ahokangas, and M. Mustonen, “Spectrum sharing using licensed shared access: the concept and its workflow for LTE-advanced networks,” IEEE Wireless Communications, vol. 21, pp. 72–79, May, 2014. 4. K. W. Sung, S.-L. Kim, and J. Zander, “Temporal spectrum sharing based on primary user activity prediction,” IEEE Transactions on Wireless Communications, vol. 9, no. 12, pp. 3848– 3855, 2010. 5. Y. Chen, J. Zhang, K. Wu, and Q. Zhang, “Tames: A truthful auction mechanism for heterogeneous spectrum allocation,” in IEEE INFOCOM, pp. 180–184, May 2013. 6. X. Zhou and H. Zheng, “Trust: A general framework for truthful double spectrum access,” in IEEE INFOCOM, 2009. 7. H. Wang, E. Dutkiewicz, G. Fang, and M. Dominik Mueck, “Spectrum Sharing Based on Truthful Auction in Licensed Shared Access Systems,” in Vehicular Technology Conference, Jul 2015. 8. A. Chouayakh, A. Bechler, I. Amigo, L. Nuaymi, and P. Maillé, “PAM: A Fair and Truthful Mechanism for 5G Dynamic Spectrum Allocation,” in IEEE PIMRC, 2018. 9. X. Zhou, S. Gandhi, S. Suri, and H. Zheng, “ebay in the sky: Strategy-proof wireless spectrum auctions,” in Proceedings of the 14th ACM international conference on Mobile computing and networking, pp. 2–13, ACM, 2008. 10. W. Wang, B. Liang, and B. Li, “Designing truthful spectrum double auctions with local markets,” IEEE Transactions on Mobile Computing, vol. 13, no. 1, pp. 75–88, 2014. 11. N. Enderle and X. Lagrange, “User satisfaction models and scheduling algorithms for packetswitched services in umts,” in Vehicular Technology Conference, 2003. VTC 2003-Spring. The 57th IEEE Semiannual, vol. 3, pp. 1704–1709, IEEE, 2003. 12. C. Chekuri and A. Kumar, “Maximum coverage problem with group budget constraints and applications,” in Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (K. Jansen, S. Khanna, J. D. P. Rolim, and D. Ron, eds.), (Berlin, Heidelberg), pp. 72–83, Springer Berlin Heidelberg, 2004. 13. V. Krishna, Auction Theory. Academic Press, 2009. 14. J. E. Mitchell, “Branch-and-cut algorithms for combinatorial optimization problems,” Handbook of applied optimization, pp. 65–77, 2002.

Routing Game with Nonseparable Costs for EV Driving and Charging Incentive Design Benoît Sohet, Olivier Beaude and Yezekael Hayel

Abstract Designing optimal incentive mechanisms for electric vehicles is an important challenge nowadays. In fact, this new type of vehicle influences several parts of society, at the transport level through congestion/pollution and at the energy level. In this paper, we consider the design of driving and charging optimal incentive through a routing game approach with multiple types of vehicles: gasoline and electric. We show that the game is not standard and needs a particular framework. We are able to prove the existence of a Wardrop equilibrium of this routing game with nonseparable costs, due to interaction through the energy cost. Our analysis is applied to a particular transportation network in which two paths are possible for vehicles, mainly one through the city center and another one outside. A fully characterization of Wardrop equilibrium is proposed, and optimal tolls are computed in order to minimize an environmental cost. Numerical results are provided on real data of electricity consumptions in France and in Texas, USA. Keywords Congestion game · Electric vehicle · Nonseparable costs · Wardrop equilibrium

1 Introduction Electric vehicles (EV) will have more and more impact on urban systems, both concerning issues related to mobility—when driving—as well as for the ones related to the grid—when charging. For mobility, EV seems a very promising solution thanks B. Sohet (B) · O. Beaude EDF Lab’, EDF R&D, OSIRIS Department, University of Paris-Saclay, 91120 Palaiseau, France e-mail: [email protected] O. Beaude e-mail: [email protected] Y. Hayel LIA/CERI, University of Avignon, 84000 Avignon, France e-mail: [email protected] © Springer Nature Switzerland AG 2019 J. Walrand et al. (eds.), Network Games, Control, and Optimization, Static & Dynamic Game Theory: Foundations & Applications, https://doi.org/10.1007/978-3-030-10880-9_14

233

234

B. Sohet et al.

to the limited negative externalities, first and foremost the absence of noise and local air pollution. For the grid, the flexibility of EV charging—in time of operation and level of power—makes it a great actor in “Demand Response” mechanisms (thanks to charging scheduling) which is an emerging field in “smart grids.” It consists in controlling consumption profile by, e.g., postponing usages in time, or reducing the level of power consumed, and with different objectives for the system: local management of production–consumption balance, mitigate impact on the electricity network, constitute a “Virtual Power Plant” by aggregating flexible usages, etc. It is revolutionizing the traditional paradigm of this system, where almost only production units were flexible to ensure the effective operation of this system [8]. Currently, even if the penetration rate of EV is not really significant at the scale of a country, it can be already substantial locally, e.g., at the scale of a region.1 In this case, the mass of already driving and charging EV has an influence on both the mobility and grid metrics. In this context, taking into account charging strategies into everyday EV routing decisions will become an important issue in smart cities [2]. A first problem is the design of charging incentives (e.g., under the form of a pricing) to share—in space and time—EV charging infrastructure [19]. Directly related to this day-by-day setting is the problem of charging infrastructure sizing, in terms of 1. number of charging points; 2. location in space; and 3. available levels of power [12]. In a more futuristic vision, the EV charging and driving coupling can be transposed into a charging-bydriving one (with an inductive charging system under the road) as suggested in [21]. In this case, the charging operation is directly (physically, only parts of the roads being equipped with induction charging systems) dependent on the route choice. Following the same line of research, this work proposes a coupled analysis of driving and charging decisions, but with a different methodological setting. This coupled system is modeled by a bilevel framework. At the upper level, a traffic manager decides tolls that induce a financial cost to drivers, thus influencing their driving choices.2 At the lower level, a large number of EV drivers—approximated as infinite—make two decisions: travel path and energy scheduling. Finally, these EV drivers share the road with gasoline ones, which corresponds to a game-theoretic equilibrium in a multi-class setting at the lower level. The main novelty of our problem is to couple both routing and charging decisions for EV users. By doing so, the problem can be classified as a nonatomic congestion game with nonseparable and nonlinear cost functions. The nonlinearity comes from the driving congestion cost , but also from a charging cost, which is introduced here to represent the cost of the energy needed to drive. This charging cost corresponds to the one faced by an operator which would be responsible for the charging operation of all the EV; this includes the case of a pool of shared EV, or when the charging flexibility is managed 1 See,

e.g., the case of “Ile-de-France,” compared to France where the current number of EV in circulation is around 150 000 http://www.automobile-propre.com/dossiers/voitures-electriques/ chiffres-vente-immatriculations-france/ when the total number of French vehicles of around 39 millions. 2 Such incentive mechanism is already applied in practice, in metropolis like London (https://tfl. gov.uk/modes/driving/congestion-charge).

Routing Game with Nonseparable Costs for EV Driving and Charging Incentive Design

235

by an aggregator (see, e.g., [7] for a presentation of such stakeholder in the case of EV aggregation). Solving a centralized optimization problem, this charging operator schedules EV charging to obtain the quantity of energy needed by the EV (to drive) at a minimal cost. This leads to the nonseparability property3 of our problem, with the coupling between routing and charging decisions: The routing problem gives an energy charging need to the charging problem, which, “in response,” provides a cost for a unit of energy used to drive. From a methodological standpoint, this coupling makes the use of standard techniques like Beckmann formula and Frank–Wolfe algorithm not trivial for determining the Wardrop equilibrium [15]. The paper is organized as follows. After describing this multi-class model and particularly the cost functions in Sect. 3.1, a preliminary analysis gives technical properties on the nonseparable charging cost function for a realistic representation of charging operation, in Sect. 3.2. These properties are then used in order to highlight the link between charging and driving problems: In Sect. 4, the optimal routing problem is considered where EV determine selfishly their route. Properties of the Wardrop equilibrium are illustrated in Sect. 5 on real datasets and also on the problem of minimizing an environmental objective function. Finally, we conclude the paper in Sect. 6 and give some perspectives.

2 State of the Art Basic traffic assignment problem (TAP) with single-class customers is defined and studied in [18]. They showed that when all customers are equally affected by the congestion (symmetric setting) and when the cost functions are increasing, the equilibrium is unique. In recent years, there has been an increasing interest for mixed TAP (MTAP) where more than two classes of vehicles are considered [10]. Uniqueness of Wardrop equilibrium in mixed TAP is proved in [1] when the cost functions are the same for every customer, up to an additive constant. The complexity induced by mixing different types of traffic in a routing game comes from the difficulty to use standard approaches like Beckmann in order to determine the Wardrop equilibrium. In fact, different impacts on travel costs cause an asymmetry in the Jacobi matrix [5]. Therefore, different techniques and approaches like nonlinear complementarity, variational inequality and fixed-point problems for characterizing equilibrium are reviewed in [6]. Studies of nonatomic games with nonseparable costs are highly complicated and very few papers deal with this framework. In [3], the authors generalize the bound obtained by Roughgarden and Tardos [17] on the Price of Anarchy for nonseparable, symmetric and affine costs functions. In [16], the author considers a similar framework but with asymmetric and nonlinear costs. The bounds obtained are tight and are based on a semidefinite optimization problem. In [4], the authors propose a new proof for the Price of Anarchy in nonatomic congestion games, particularly with 3 Costs

functions are separable if c(x) = ca (xa ) for all arc a, xa being the flow on a.

236

B. Sohet et al.

nonseparable and nonlinear cost functions. Their geometric approach leads to obtain in a simple manner the bounds found in [3, 16]. All these works focus on performance or cost evaluation at the equilibrium situation, compared to the one at social optimum (Price of Anarchy metric). In our work, we focus on the algorithmic part and techniques to obtain the value of the equilibrium, based on potential functions and Beckmann’s techniques, for nonatomic routing games with nonseparable cost functions. In fact, our framework induces particular costs functions which enable us to characterize the Wardrop equilibrium as the minimum of a global function. Regarding the applicative setting, a few references address the coupling of driving and charging decisions, but with different methodologies. Among the papers mentioned in the introduction, let us detail the contribution of [19], the closest to our work in terms of methodology. The authors of this paper propose a bilevel model (Stackelberg), where EV charging station operators compete at the upper level, offering charging prices to attract EV to their station. At the lower level, EV takes charging decisions based on a traffic flow model (driving endogenous congestion effect) and a charging model (with queuing, i.e., charging congestion effect). A game-theoretic evolutionary dynamics between EV is considered, for them to update their charging strategy. The main differences with our setting are that: 1. the traffic network consists only on a unique path (with congestion only in time) and 2. the charging cost congestion effect corresponds to queuing and does not impact the electricity price for charging. While this seems to be very appropriate for long-distance travels (e.g., on highways), the setting proposed here could be more suitable for the urban case, where charging can be scheduled in a centralized manner (e.g., by an EV carsharing operator) to avoid impacts on the electricity system. Finally, note that the charging problem considered here is not innovative in itself; it is directly related to the seminal paper [13], where “valley-filling” type solutions are obtained. However, to the best of our knowledge, the integration of such charging problem into a model coupling driving and charging decisions constitutes a novelty.

3 Model Two types of vehicles are considered: gasoline vehicles (GV) and EV with resp. indices e and g. They determine optimally their travel path, knowing perfectly the costs. The novelty is that the cost depends on congestion of the path but also on the energy used to travel along it. Moreover, the energy cost for EV depends on the global demand of electricity, which depends itself on the routing strategies of all EV in a way described later.

3.1 Driving Problem We consider here a simple graph made of only two nodes, O (origin) and D (destination). Two paths a and b link O to D, with lengths la < lb expressed in distance unit

Routing Game with Nonseparable Costs for EV Driving and Charging Incentive Design

237

(km typically). In addition to allowing a tractable theoretical analysis, this simple setting represents a city where drivers can either directly cross the city (path a) or take the ring road (path b) in order to join the opposite side of the city. An important practical problem in this situation is to reduce as much as possible the pollution level in the city center, or equivalently (as expressed mathematically a little further) the flow of GV going through path a. The total flow X of vehicles is normalized: X = 1. The proportion of EV in the population of vehicles is X e ; the one of GV is obtained by X g = X − X e . Note that the choice of using either an EV or a GV is not part of the decisions modeled here for the users, and only taken as a (fixed) parameter. Analyzing this “first stage” choice of the user could constitute a natural extension of this work. For each arc r , xr,g (resp. xr,e ) is the flow—in number of vehicles per unit of time—of GV (resp. EV) on arc r . The flow of vehicles on this arc r is directly obtained by xr = xr,g + xr,e . These flows must verify the following constraints: 

= Xs , ≥ 0 ∀r .

r xr,s

∀s ∈ {e, g},

xr,s

(1)

The travel duration on arc r is given by the following BPR function [14]:  dr (xr ) =

dr0



xr 1+α Cr

β  ,

(2)

where Cr is the capacity of link r in vehicles per unit of time and dr0 is the free flow reference time: dr0 = vlrr where vr is the maximum speed limit on arc r . Travelers are subject to tolls depending on whether they use GV or EV. Let tr,g and tr,e be the toll of arc r for, respectively, a GV  and an EV. The vector of tolls for all arcs and types of vehicles is denoted by t = ta,e , ta,g , tb,e , tb,g , where bold is used for vectors. Finally, vehicles are energy consuming and the driving costs should also depend on this feature. We denote by λs the price of energy unit (for example, λe is typically expressed in e/kWh). Then, using a distance-based energy consumption model, the driving total cost cr,s for a vehicle of type s ∈ {g, e} traveling on route r is given by: cr,s (x, t) = tr,s + τ dr (xr ) + λs m s lr ,

(3)

where m s is the energy (or fuel) consumption per unit of distance and τ is the cost per unit of delay. The next section describes how to determine the price per energy (kWh) unit λe for EV depending on the flows xr,e of EV on each arc r . Notice here a fundamental difference between λg and λe : While the former is exogenous (independent of the drivers’ flows), the latter is endogenous and results from the resolution of an optimization problem solved by an EV aggregator, as explained in the following section.

238

B. Sohet et al.

3.2 Charging Problem In this section, we consider the energy scheduling problem itself, separately from the routing problem. As the EV flows on the network are given, the global charging need L e for all the EV is

xr,e lr m e . (4) Le = r

Note that this value depends on the flows on all paths into the network, which makes the total cost for EV nonseparable. The charging need L e is expressed in energy per unit of time. The electricity unit cost λe depends on the charging profile of all EV. This profile is determined optimally based on an optimization problem defined thereafter, and which corresponds to the one faced by a centralized controller who determines optimally the charging profile of all the EV. This scheduling problem is made in discrete time, with a discretization from 1 to T . Typically, this discretization represents a full day, in which each EV has to be charged once by the charging operator. The scheduling problem takes into consideration a nonflexible (fixed)  consumption, which is denoted 0,t for each time slot t. We denote by 0 = 0,t t the vector of nonflexible consumption. This nonflexible profile includes electrical usages that are present in locations where EV are charged (e.g., household appliances when charging at home, tertiary ones when at professional sites). As modeled in [13] (and in many other papers about EV smart charging), EV consumption has indeed to be scheduled depending on other electrical usages, the impact on the grid being dependent on the aggregate consumption. Note that the implicit simple choice made here is that all EV share the same (local) electricity network. In an extended setting, the aggregate consumption profile could be calculated at different charging locations, with possibly different electricity network sizing, and nonflexible profiles. The EV charging operator has to determine the optimal aggregated charging profile denoted by the vector e := (e,1 , . . . , e,T ) for all EV taking into account the global charging need L e , solving the following problem: min

e =(e,t )t

T

 f t 0,t + e,t , s.t.

t=1

 e,t ≥ 0 ∀t , T t=1 e,t = L e .

(5)

The cost function f t is a proxy4 to represent time-dependent cost/impact on the electricity network, or to provide electricity to EV (generation, transmission, distribution). A typical choice for f t is a quadratic function [13]: Assumption 1 For all t, f t (t ) = f (t ) = (t )2 . Remark 1 f is strictly convex. Suppose now without loss of generality (because there are no dynamical effect taken into account here) that the nonflexible part of the load is ordered such that 4 It

does not include dynamic or locational effects.

Routing Game with Nonseparable Costs for EV Driving and Charging Incentive Design

239

0,1 ≤ · · · ≤ 0,T ≤ 0,T +1 = +∞, where time slot T + 1 is added to unify notations. With this assumption, the optimal value of problem (5), denoted by val(L e ), can be explicitly obtained from the nonflexible consumption profile 0 . Proposition 1 Given a global charging need L e and a nonflexible vector 0 , the solution of the energy scheduling problem described in (5) yields the value:  

T  L e − Δt¯−1 val(L e ) = t¯ × f 0,t¯ + f 0,t , + ¯t ¯

(6)

t=t +1

with the energy thresholds defined recursively by

Δ0 = 0 ,  Δt = Δt−1 + t 0,t+1 − 0,t ,

(7)

and t¯ such that L e ∈] Δt¯−1 , Δt¯]. Proof Because f is convex, problem (5) is equivalent to its Karush–Kuhn–Tucker  conditions. Further, calculations lead to the explicit expressions given in (6).5 Remark 2 The valley-filling structure of the optimal solution is directly observable in (6): while the first t time slots contribute equally to the total cost at optimum (they share the same water level, sum of nonflexible plus EV consumption), the next ones (t ≥ t + 1) provide different contributions corresponding only to the nonflexible part of consumption, as they are not used for charging. Finally, the average electricity unit cost depends on val as follows: 

x l m r r,e r e . λe (L e ) =  T r xr,e lr m e + t=1 0,t val

(8)

The meaning of the modeling choice made here is that total impact on (local) electricity network, val(L e ), is shared among EV and other electricity appliances. Follows, general properties of the function λe which will be useful for describing our method to find Wardrop equilibrium considering energy costs. Even though val was piecewise defined in Proposition 1, the first property is: Lemma 1 Function L e −→ val(L e ) is C 1 on R+ := [0, ∞ [ . (For a proof of Lemma 1, see footnote 5.) Let us define the two following values: αt =

t

s=1

5 For

0,s and βt =

T

f (0,s ).

(9)

s=t+1

greater details of the proofs, please visit https://sites.google.com/site/olivierbeaudeshomepage/olivier-beaude-publications.

240

B. Sohet et al.

The following proposition characterizes the monotony of λe : Proposition 2 Under Assumption 1, the electricity cost function L e −→ λe (L e ) has a unique minimum at x ∗ such that:  x∗ =

xt i f ∃ t ∈ {1, · · · , T − 1} s.t. xt = 0 other wise.



(αT − αt )2 + tβt − αT ∈ [Δt−1 , Δt ],

Proof The quadratic form of f taken in Assumption 1 leads to the strict convexity  of λe up to ΔT −1 , after what it is linearly increasing (see footnote 5). We will see that the extent of the region where λe is increasing is important for WE uniqueness matters. The next proposition gives a sufficient condition on the nonflexible load 0 to locate the minimum x ∗ , from which λe starts increasing. Proposition 3 The set T = {t s.t. 0,t+1 ≥ 0,T /2} is not empty and its minimum t0 is such that x ∗ ≤ Δt0 , meaning that from Δt0 , λe is increasing. Proof The result is obtained in two steps: First, if 0,t+1 is greater than a quantity rt,± (defined in footnote 5), then xt ≤ Δt . Then, we showed that 0,T /2 ≥  rt,± ∀t. Thanks to this proposition, if 0,1 > 0,T /2 (e.g., peak and off-peak electricity consumptions are close), then λe is increasing for any global charging need L e . This result will be particularly useful to get theoretical results about the routing game including a charging cost, as explained in the following part.

4 Optimal Routing with Energy Cost Given the energy cost obtained in the previous section, we now go back to study the routing game in which each player’s cost is defined in (3). Let us remind that our routing game is particularly complex as cost functions are nonseparable, i.e., depend on flows over all links and types of vehicles. Even with this complexity, the cost functions have good properties that enable us to build a Beckmann’s like formula. Before determining this, we define the Wardrop equilibrium.

4.1 Wardrop Equilibrium In his seminal paper [20], Wardrop stated two principles that formalize the notion of equilibrium that minimizes the total travel costs. His first principle says that the journey times on all the routes actually used are equal, and less than those which would be experienced by any other player on any unused route, which is expressed mathematically in the following definition.

Routing Game with Nonseparable Costs for EV Driving and Charging Incentive Design

241

Definition 1 A flow x is a Wardrop equilibrium (WE) if and only if: ∀s ∈ {e, g}, cr,s (x) ≤ cr ,s , (x) f or all ar cs r, r such that xr,s > 0 .

(10)

For the study of WE of the problem described in the previous section, the focus must then be on the signs of costs differences ca,s − cb,s for s ∈ {e, g}. In this special setting with only two arcs, the constraints given by (1) can be replaced by the variables substitutions xb,i = X i − xa,i , with 0 ≤ xa,s ≤ X s . Then, the exogenous term λe depends only on xa,e , while λg is constant. Note that both classes share the same congestion cost term da (xa ) and db (1 − xa ), respectively, on arcs a and b.

4.2 Beckmann’s Method Even if our routing game is nonseparable and with multiple types of flows, the structure of electricity cost function λe enables us to apply a similar approach to Beckmann’s model, as described in the following proposition. Proposition 4 The local minima of the following function B are WE:  B(x) =τ

xa



1−xa





L e (xa,e )

da (x) dx + db (x) dx + λe (x) dx 0 0 0

 + m g λg xa,g la + X g − xa,g lb + ta,s xa,s ,

(11)

a,s

defined for (xa,e , xa,g ) ∈ I = [0, X e ] × [0, X g ]. Proof The sign of the partial derivatives of B at a minimum x∗ gives relations corresponding to the Definition 1 of a WE (see footnote 5).  This result gives a simple method to find a WE and leads to some WE properties, given in the following corollary. Corollary 1 There exists a WE, and a sufficient condition for uniqueness is that the electricity unit cost λe be increasing. Proof The uniqueness comes by studying the Hessian of B (see footnote 5). In practice, the uniqueness of WE allows authorities to predict drivers’ behavior.



242

B. Sohet et al.

5 Numerical Illustrations First numerical experiments will complement the theoretical analysis in Sect. 3.2 of the charging problem, in particular by testing on real data the sufficient condition given in Proposition 3 for an increasing electricity unit cost λe , leading to a unique WE of the coupled problem of driving–charging. Then, the sensitivity of this WE with respect to realistic parameters will be highlighted, like the proportion X e of EV and incentive tolls to reduce the environmental cost.

5.1 Electricity Cost This section focuses on two different sets of real data about hourly residential electricity consumption of households throughout a year, which corresponds to the non flexible load 0 = 0,t t=1,T introduced in our theoretical study. The first dataset is named “Recoflux” from Enedis—the French main Distribution Network Operator—and is a statistical representation of a typical French household consumption profile, taking into account heating, water heating and all the other consumptions.6 We consider that this profile represents the consumption of a neighborhood, inside which takes place our routing problem. For instance, the EV may be part of a carsharing service where the company in charge makes sure that charging costs are minimized. At any moment and at any location, we assume that every EV may find an available charging station. These stations are linked to the neighborhood network, so that the price of charging depends each day d on the nonflexible profile 0 (d) given by Enedis. The cost function f is supposed to be quadratic, as in Assumption 1. The unit cost of electricity for a EV is then described by the function λe studied in Sect. 3.2. According to Proposition 3, 0,1 (d) > 0,T (d)/2 is sufficient to guarantee an increasing λe , and consequently a unique WE. In the raw data, daily consumption is split in T = 24 time slots. In Fig. 1, these 24 time slots have been merged in T = 2. To do that, for each day d the hourly values 0,t (d) are increasingly sorted and the first half is summed to get 0,1 (d), while the other half provides 0,2 (d). Note that consecutive hourly consumptions might not be in the same new time slot, as it is the case for the off-peak hours in France, corresponding to night hours and some of the afternoon hours. In Fig. 1, each peak corresponds to weekend consumptions, and the change of month in the computed data can easily be seen, especially in summer. Consumptions are expressed in an arbitrary energy unit here (Recoflux data are normalized, which is without loss of generality regarding their usage here). The hatched region denotes a consumption less than 0,T (d)/2. For a given day d, Proposition 3 is verified if and only if 0,1 (d) (the thinnest line) is above this region this day. This appears to be the case everyday of the year when T = 2, according to Fig. 1. 6 Data

are available at https://www.enedis.fr/coefficients-des-profils.

Routing Game with Nonseparable Costs for EV Driving and Charging Incentive Design

243

Fig. 1 Illustration of Proposition 3 with real data Recoflux from Enedis. The proposition is verified all year when there are T = 2 time slots (thin line over colored region), thus ensuring a unique WE (see Corollary 1)

Fig. 2 Illustration of Proposition 3 with real data in Texas from Pecan Street. The sufficient condition of Proposition 3 is not verified for 87% of the days (see red colored region)

The second dataset is the hourly electric consumption throughout a year of one household located in Texas, given by the company Pecan Street.7 The same method as for the previous dataset is applied, and the consumption is split again in T = 2 time slots. In Fig. 2, there is much more noise in the data compared to the statistical 7 Data

are available at http://www.pecanstreet.org/.

244

B. Sohet et al.

profile Recoflux presented earlier, and consumption is higher during summer, when it is really hot in Texas. When 0,1 (d) (thin line) is inside the hatched region, representing consumptions lower than 1/2 0,2 (d), Proposition 3 is not verified and the red color shows how much 0,1 (d) is lower than 1/2 0,2 (d). It appears that Proposition 3 is verified only 13% of the year (proportion of days where there is no red region in Fig. 2). The reason is that nights consumption is very low compared to days consumption, especially in hot periods of the year, which is typical in regions like Texas where electricity consumption is strongly sensible to the use of air-conditioning. Yet, λe is increasing 34% of the days, which highlights the fact that Prop. 3 is simply a sufficient condition for monotonicity. Note that mean profiles like Recoflux data might smooth extreme consumptions for which Proposition 3 is not verified and λe is not increasing, explaining part of the differences between the two dataset in Figs. 1 and 2.

5.2 Minimizing Environmental Cost In this section, we are interested in optimizing a global function that depends on the overall WE into the network. One particular function could be the level of pollution, considering that one path is inside the city and the other one outside. Therefore, an urban authority may want to control the toll prices on both paths such that the global level of pollution is minimum. Only GV release polluting substances into the air, therefore we choose to measure the level of pollution depending on the average number of GV on each arc. Moreover, the impact of pollution is more critical on the arc which goes across the city, i.e., arc a, compared to the other arc, as mathematically expressed just after.  The parameters of the problem are set as follows: The nonflexible load 0,t t consists in T = 2 time slots such that 0,T = 1.6 0,1 , with 1.6 being the mean ratio 0,T /0,1 in Fig. 1 (Recoflux data). This ensures the uniqueness of the WE, assuming that we use an electricity unit cost function f verifying Assumption 1. Actually, we choose f (x) = ηx 2 , which does not change the results obtained in Sect. 3.2 since λe, f (x) = ηλe,x 2 . The coefficient η makes it possible to scale the electricity unit cost λe, f (x) to around 0.20 e, in line with Eurostat data for France in 2017. The length la = 30 (km) of arc a is a good approximation of the daily mean individual driving distance in France (ENTD8 2008). The ratio between lb and la is set to 2, which is approximately the ratio between the ring road linking two opposite places at the border of a city and the distance travelled to cross the city. The BPR functions dr defined in (2) are such that for both arcs r , vr = 50 (km/h), the speed limit in French cities, and α = 0.15 and β = 4 empirically [9]. The capacity of both arcs is set to Cr = X/2, so that there is a congestion phenomenon when all vehicles are on the same arc. The value of time τ is set to 10 (e/h) according to a French government report.9 8 ENTD

= Enquête Nationale Transports et Déplacements (in French).

9 http://www.strategie.gouv.fr/sites/strategie.gouv.fr/files/archives/Valeur-du-temps.pdf.

Routing Game with Nonseparable Costs for EV Driving and Charging Incentive Design

245

Fig. 3 Repartition of EV and GV among the two arcs at the WE which minimizes pollution, as the proportion X e of EV grows. The optimal toll forces all EV on arc a, and a decreasing proportion of GV

For GV, we approximate parameters by λg = 1.5 (e/L) and m g = 0.06 (L/km) and for EV, m e = 0.2 (kWh/km) [13]. Finally, the nonflexible load 0 is scaled by a multiplicative factor so that the maximum need L e of one EV is roughly the fourth of a household consumption, which corresponds to Pecan Street data. The authority chooses the toll ta,g on the shortest arc, which crosses the city, only for GV. For all the other traffics, there is no toll. The purpose of that is to limit the number of GV contributing to the city’s pollution cenv , defined as cenv (x) =



εr xr,g dr (xr,e + xr,g ),

(12)

r

where εa = 2εb are the importance given to the level of pollution of each arc on the global level of pollution of the entire area/city. Note that by applying Little’s law [11], the term xr,g dr (xr ) determines the average number of GV on arc r . Observe also that the environmental cost function cenv depends implicitly on the toll ta,g through the WE flows xr,s . The authority can thus control this pollution level, solving an optimization problem written as a mathematical programming with equilibrium constraints (MPEC) problem:  ∗ = min cenv x∗ (ta,g ) , cenv ta,g ≥0

with x∗ the equilibrium.

(13)

As there is no explicit formulation of the WE, it is difficult to determine explicit solutions for this constrained optimization problem, or even to integrate optimality conditions of the lower-level (WE) problem into the upper one. In turn, numerical analysis is performed in the sequel section. More precisely, the goal here is to show the impact of a growing proportion X e of EV on the potential gain δ(X e ) on pollution by taxing optimally (relatively to the worst pollution case), defined as:

246

B. Sohet et al.

Fig. 4 Evolution of the ∗ and the optimal toll ta,g potential gain δ(X e ), with respect to the proportion X e of EV. As X e grows, the traffic manager will induce a bigger impact on pollution if choosing the optimal toll, which decreases

δ(X e ) =

∗ maxx cenv (x) − cenv . maxx cenv (x)

(14)

For the traffic manager, it is interesting to know how to fix the tolls in order to ∗ minimize pollution. The optimal toll value ta,g depends on the proportion of EV X e . At first, when X e is small, there must be a large enough toll ta,g (see Fig. 4). Otherwise, the majority of GV would go on arc a to minimize the duration of their travel, which would be a more polluting situation. The best situation is a right balance of GV flow between arcs a and b, so that there is not too much congestion (see Fig. 3). ∗ decreases slightly. The reason is As the number of EV grows, the optimal toll ta,g that when there are fewer GV, the impact on congestion if all GV switch from arc a to arc b (for example) is less significant. Thus, there is no need to set high taxes to prevent GV from using arc a. Above an EV proportion of X e = 60%, the best solution for pollution is to have all GV on arc b, because their number is too small to create a significant congestion. Note that in this case, the manager may set a toll ta,g as large as it wants. Approaching X e = 1, some EV start using arc b because arc a gets too congested. The potential gain δ(X e ) on environmental cost by taxing optimally increases with X e : As there will be more and more EV, optimizing tolls will have more and more impact on pollution. When the proportion of GV X g is almost zero, the potential gain stagnates.

6 Conclusions With the growing number of electric vehicles (EV), new challenges related to driving and charging matters arise. In this article, the coupling between driving and charging is modeled by a nonseparable, nonlinear electricity unit cost function λe , which depends not only on the EV flow on the road used by a given user, but also on the EV

Routing Game with Nonseparable Costs for EV Driving and Charging Incentive Design

247

flows on the other roads. For this type of cost function in nonatomic congestion games, no result on the properties of Wardrop equilibria (WE) exists. In this article, the existence of such WE is proved for a simple two-road traffic network, by generalizing Beckmann’s method to this new type of cost function. Moreover, it is shown that the uniqueness of WE depends on the monotonicity of λe . A detailed analysis of this function allows to show that its monotonicity is strongly dependent on the profile of the nonflexible load—electricity needs other than the EV one—a key parameter of the charging problem considered here. This uniqueness condition has been tested numerically on two sets of real data for nonflexible loads, one in France and the other in Texas (USA). While there is always a unique WE in France when there exist off-peak and on-peak electricity prices, the uniqueness for any charging need L e is not guaranteed in Texas. Another numerical experiment illustrates one kind of incentives and the effect they have on the WE: If a toll system can distinguish the GV from the EV, it can be used to induce an expected ratio of GV on arc a for example. This way, environmental costs may be minimized, choosing optimal tolls. This incentive is useful only if there are enough GV and can bring great environmental improvements when there will be roughly as many EV as GV. In a future work, the theoretical results obtained here—on the existence and uniqueness of a WE—will be generalized to more general cost functions and to any number of arcs. Regarding the uniqueness property, an equivalent condition on the nonflexible load—or failing this, a more precise sufficient condition—will be investigated. Finally, a theoretical framework needs to be set to study more thoroughly the incentives problem, making it a bilevel game.

References 1. E. Altman and H. Kameda. Equilibria for multiclass routing problems in multi-agent networks. Dynamic Games and Applications, 7:343–367, 2005. 2. M. Brenna, M. Falvo, F. Foiadelli, L. Martirano, F. Massaro, D. Poli, and A. Vaccaro. Challenges in energy systems for the smart-cities of the future. In Energy Conference and Exhibition, 2012 IEEE International, pages 755–762, 2012. 3. C. Chau and K. Sim. The price of anarchy for non-atomic congestion games with symmetric cost maps and elastic demands. Operations Research Letters, 31, 2003. 4. J. Correa, A. Schulz, and N. Stier-Moses. A geometric approach to the price of anarchy in nonatomic congestion games. Games and Economic Behavior, 64:457–469, 2008. 5. S. Dafermos. The traffic assignment problem for multiclass-user transportation networks. Transportation Science, 6(1):73–87, 1972. 6. M. Florian and D. Hearn. Network equilibrium models and algorithms. Network routing, pages 485–550, 1995. 7. S. Han, S. Han, and K. Sezaki. Development of an optimal vehicle-to-grid aggregator for frequency regulation. IEEE Transactions on smart grid, 1(1):65–72, 2010. 8. A. Ipakchi and F. Albuyeh. Grid of the future. IEEE power and energy magazine, 7(2):52–62, 2009. 9. M. Jeihani, S. Lawe, and J. Connolly. Improving traffic assignment model using intersection delay function. 47th Annual Transportation Research Forum, New York, New York, March 23–25, 2006 208046, Mar. 2006.

248

B. Sohet et al.

10. N. Jiang and C. Xie. Computing and analysing mixed equilibrium network flows with gasoline and electric vehicles. Computer-aided civil and infrastructure engineering, 29(8):626–641, 2014. 11. L. Kleinrock. Queueing systems. Wiley Interscience, 1975. 12. A. Y. Lam, Y.-W. Leung, and X. Chu. Electric vehicle charging station placement: Formulation, complexity, and solutions. IEEE Transactions on Smart Grid, 5(6):2846–2856, 2014. 13. A.-H. Mohsenian-Rad, V. W. Wong, J. Jatskevich, R. Schober, and A. Leon-Garcia. Autonomous demand-side management based on game-theoretic energy consumption scheduling for the future smart grid. IEEE transactions on Smart Grid, 1(3):320–331, 2010. 14. B. of Public Roads. Traffic assignment manual. Technical report, U.S. Department of Commerce, Urban Planning Division; 1964. 15. M. Patriksson. The Traffic Assignment Problem: Models and Methods. Dover Publications, 2015. 16. G. Perakis. The price of anarchy when costs are non-separable and asymmetric. proccedings of IPCO, 2004. 17. T. Roughgarden and E. Tardos. How bad is selfish routing. in proceedings of FOCS, 2000. 18. Y. Sheffi. Urban transportation networks: Equilibrium analysis with mathematical programming methods. Prentice-Hall, Inc., 1985. 19. J. Tan and L. Wang. Real-time charging navigation of electric vehicles to fast charging stations: A hierarchical game approach. IEEE Transactions on Smart Grid, 8(2):846–856, 2017. 20. J. Wardrop. Some theoretical aspects of road traffic research. Proc Inst Civ Eng Part II, 1:325– 278, 1952. 21. W. Wei, S. Mei, L. Wu, M. Shahidehpour, and Y. Fang. Optimal traffic-power flow in urban electrified transportation networks. IEEE Transactions on Smart Grid, 8(1):84–95, 2017.

The Social Medium Selection Game Fabrice Lebeau, Corinne Touati, Eitan Altman and Nof Abuzainab

Abstract We consider in this paper competition of content creators in routing their content through various media. The routing decisions may correspond to the selection of a social network (e.g., Twitter versus Facebook or Linkedin) or of a group within a given social network. The utility for a player to send its content to some medium is given as the difference between the dissemination utility at this medium and some transmission cost. We model this game as a congestion game and compute the pure potential of the game. In contrast to the continuous case, we show that there may be various equilibria. We show that the potential is M-concave which allows us to characterize the equilibria and to propose an algorithm for computing it. We then give a learning mechanism which allow us to give an efficient algorithm to determine an equilibrium. We finally determine the asymptotic form of the equilibrium and discuss the implications on the social medium selection problem.

1 Introduction Social networks involve many actors who compete over many resources. This gives rise to competitions at different levels which need to be taken into account in order to explain and predict the system behavior. In this paper, we focus on competition of individual content creators over media. A content creator has to decide which one of several media to use. The media choice may correspond to a social network that will be used for sending (and disseminating) some content. For instance, the decision

F. Lebeau · C. Touati (B) · E. Altman Inria, Le Chesnay, France e-mail: [email protected] E. Altman e-mail: [email protected] N. Abuzainab Department of Electrical and Computer Engineering, Virginia Tech, Blacksburg, VA, USA e-mail: [email protected] © Springer Nature Switzerland AG 2019 J. Walrand et al. (eds.), Network Games, Control, and Optimization, Static & Dynamic Game Theory: Foundations & Applications, https://doi.org/10.1007/978-3-030-10880-9_15

249

250

F. Lebeau et al.

can consist in choosing between Twitter and Facebook, or in deciding to which of several Facebook groups to send the content. The game we study in this paper is atomic and non-splittable. We consider a decision maker (or a player) to be a single content instead. This regime can well approximate decision making where a content creator, say a blogger, occasionally sends content. Here, occasionally implies that the time intervals between generation of consecutive contents by the blogger are large enough so that the states of the system at the different times of creation of content are independent one from another. This regime is interesting not only because it is characteristic of systems with many sources of contents, but also and foremost, it turns out that it precisely characterizes bloggers that have more popularity and influence. This was established experimentally in [1] which analyses the role of intermediate actors in the dissemination of content. A similar game as the one in this paper was already studied in [2, 3], but there the players control rates of creation of contents and/or decide how to split the rates. The resulting games are simpler than ours as they possess a single equilibrium. The model studied in this paper brings many novelties both in the system behavior and in the tools used to study it. The difficulty in studying the game in our atomic nonsplittable game framework is due to the integrity constraint on the players as they cannot split their content between several media. This implies that the action space is discrete and thus non-convex which may result in problems in the existence and/or uniqueness of the equilibrium. Related work Game-theoretic models for competition have been proposed in a growing number of references. The authors of [4] focus on the competition over budget of attention of content consumers and the impact of this competition on the dynamic popularity of the content. A game model related to intermediate actors that participate in the spreading of news is considered in [1]. The authors study how to choose the type and amount of content to send so as to be influential. In [5], the authors study competition over space among content creators. The space may represent a slot (say the top one) in a timeline, and a content that arrives occupies the space pushing out the one that is already there. It then stays visible there till the next arrival of content that pushes it away. The authors study the timing game: when should a content be sent to the timeline so as to maximize the expected time it remains visible. The authors of [6] study a dynamic competition model over visibility in which the rate at which creator of contents sends their traffic is controlled. Game theory has been used not only to model competition in social networks but also to design algorithms for the analysis of social networks [7]: This includes community detection [8], discovery of influential nodes [9], and more. Contributions of our paper Our first contribution is to make the observation that the game complies with the definition of congestion game introduced by Rosenthal [10]. This allows us to show that our game is a potential game, for which Nash equilibria exist. This further implies that algorithms based on best response converge to an equilibrium. We show that surprisingly, although the potential approximates a strictly concave function (in the continuous space), there may exist many equilibria. This is quite a new phenomenon in networking games, and it is due to the non-

The Social Medium Selection Game

251

convexity of the action space (due to the non-splittable assumption). In order to have uniqueness, a new concept of integer concavity has to be used. We rely on the theory of M-concavity [11] which allows us to establish the structure of the set of equilibria for this problem. We propose a learning algorithm that converges to an equilibrium and is more efficient than the best response algorithm. We finally study the asymptotic behavior of the system as the number of players grows, both in terms of characterization of the equilibria and in the price of anarchy.

2 Model and Notations We consider a set K = {1, ..., K } of seeds (content producers, bloggers, etc.) that aim to publish their content in social media. We focus on the problem where each seed needs to publish in some social medium j ∈ J = {1, ..., J }. The strategy sk of seed k is the social medium it selects for disseminating its content. Define the load of competing seeds that send their content to  j on social medium j as the number  medium j. It can be written as  j = k δ(sk , j) (δ is the Kronecker symbol, i.e., δ(a, b) equals 1 if a = b and 0 otherwise). Assume that social medium j has N j > 0 subscribers who are interested in content shipped to that medium. The utility of a player (seed) is given as the difference between a dissemination utility and a dissemination cost. The former, that is, the value of disseminating a content at the jth social network is (i) proportional to N j and (ii) inversely proportional to  j . Further, each seed pays a constant dissemination cost γ j for publishing on social medium j. This structure of utility is very common. In the networking community, we find it in resource sharing of link capacity for flow control problems. It is also associated with the so-called Kelly mechanism (see [12] for a similar utility in cloud computing), and models tracing back to the Tullock rent-seeking problem [13]. In the social medium context, this utility naturally arises if different seeds create similar content (say news) and thus a subscriber is not interested in receiving more than one content. This implies the structure of point (ii). The use of this type of utility in competition for resources in social networks can be found in [2]. Hence, the utility of seed k is given by u k (s) =

Nsk − γsk , sk

(1)

where s is the vector of strategies of the seeds s = (sk )k∈K . For notational convenience, in the following we will denote by S = JK the set of strategy profiles, and by Γ = (K , (N j , γ j ) j∈J ) the game setting with K players (the seeds), the set J of social media of parameters ((N j , γ j ) j∈J ) and the utility (u k )k∈K defined in Eq. (1). Since the utility of each player (the seeds) only depends on the number of users choosing the same action (i.e., the same social media), the game is equivalent to a

252

F. Lebeau et al.

congestion game in the sense of Rosenthal [10], where the resources are the social media. It therefore is a potential game [14], that is to say that there exists a function Pot such that: ∀k ∈ K, ∀s ∈ S, ∀sk , u k (s1 , ..., sk−1 , sk , sk+1 , ..., s K ) − u k (s) = Pot(s1 , ..., sk−1 , sk , sk+1 , ..., s K ) − Pot(s).

 Let us introduce the Harmonic number: Hn = nj=1 1/j if n ≥ 1 and Hn = 0 if n ≤ 0. Then, one can readily check that a suitable potential of the game is: Pot(s) =



 N j H j − γ j  j .

j

Let  be the vector of loads induced by s and D the set of possible vector loads: D=

⎧ ⎨ ⎩

(1 , ...,  J ) ∈ N J |

J  j=1

j = K

⎫ ⎬ ⎭

.

By using the equivalence with the congestion game of [10], we get the potential as a function of the loads  ∈ D of the social media:   N j H j − γ j  j . Pot() = j

Let us finally introduce the following notations that will come in handy in the rest of the paper: • • • •

For x ∈ Rn , supp+ (x) = { j | x j > 0} ,   . | .  is the euclidean scalar product over R J :  x | y  = j x j y j , . ∞ is the uniform norm: x ∞ = max j |x j |, (e j ) j is the Euclidean base of R J .

3 Discrete Potential Analysis 3.1 Nash Equilibria Potential games have received a lot of attention in the past years as they draw a natural bridge between the theory of games and optimization. Indeed, the definition of a potential implies the following. A strategy profile s is a Nash equilibrium iff it is a maximizer of the potential, that is, ∀s, ∀k ∈ K ∀sk ∈ J,

Pot(s1 , ..., sk−1 , sk , sk+1 , ..., s K ) ≤ Pot(s).

(2)

The Social Medium Selection Game

253

Note that a change of strategy of a seeds from social media i to social media j amount in reducing the load i of network i and increase that of j by one unit. Hence, in the space of load vectors, Eq. (2) becomes: ∀i, j :

 j > 0 ⇒ Pot( + ei − e j ) ≤ Pot().

(3)

For a given load vector , let V() be the set of possible load vectors obtained after

the deviation of a single player: V() =  + ei − e j | i, j ∈ J st  j > 0 . Then, the Nash equilibria are all the strategy profiles s for which the vector of loads is a local (in the sense of V) maximum of Pot : D → R.

3.2 M-Concavity The potential is defined over a discrete set, and therefore classical convexity properties do not hold. In order to understand the structural and uniqueness properties of the Nash equilibrium, we study the properties of the potential function in terms of M-concavity.1 Definition 1 A function f : Z J → R is M-concave if for all x, y in D and for all u ∈ supp+ (x − y): ∃v ∈ supp+ ( y − x),

f (x) + f ( y) ≤ f (x − eu + ev ) + f ( y − ev + eu ).

We have the fundamental property: Theorem 1 The function f : Z J → R ∪ {−∞} defined by:  f (x) =

Pot(x) if x ∈ D, −∞ otherwise

is M-concave. Proof Let x, y ∈ D. First, if supp+ (x − y) = ∅, then the property is trivially true. Otherwise, assume that there is some i in supp+ (x − y). If we had supp+ ( y − x) = ∅, then we would also have     y j = yi + y j < xi + xj = x j = K. j

j=i

j=i

This is absurd since y ∈ D. Hence, supp+ ( y − x) = ∅. 1 For

more information about M-convexity, see [11, Sect. 4.2].

j

254

F. Lebeau et al.

Then, let u ∈ supp+ (x − y) and v ∈ supp+ ( y − x). Since xu > yu ≥ 0 and yv > xv ≥ 0 we have that x − eu + ev and y − ev + eu are in D. Then: 

f (x − eu + ev ) + f ( y − ev + eu ) − f (x) − f ( y) = Nv

   1 1 1 1 − − +Nu . xv + 1 yv yu + 1 xu       ≥0

≥0



Note that since we did not choose a particular v in the preceding proof, we have actually shown a much stronger property, that is, that the inequality holds for any v, which is decisive for the rest of the analysis: ∀x, y ∈ D, ∀u ∈ supp+ (x − y), ∀ v ∈ supp+ ( y − x), f (x) + f ( y) ≤ f (x − eu + ev ) + f ( y − ev + eu ).

(4)

3.3 Properties of the Nash Equilibria In this section, we show properties of the Nash equilibria of this game using the M-concavity of f . Theorem 2 Let s ∈ S and let  be the vector of loads of s. Then: s is a Nash equilibrium for the game ⇔  maximizes globally the potential over D. Proof The sufficient condition is a direct consequence of Eq. (3). Conversely, assume that s is a Nash equilibrium. Then, by (3), we know that  is a local maximum of Pot (over V()). Let u, v ∈ J: If  − eu + ev ∈ / D, then f () > f ( − eu + ev ). Otherwise, we have  − eu + ev ∈ V(). Hence,  satisfies the property: ∀u, v ∈ J, f () ≥ f ( − eu + ev ). We then apply [11, Thm 4.6] with the M-concave function f on , given that  is a global maximum of f . Therefore,  is a global maximum of Pot on D. We show next that the set of loads corresponding to the different Nash equilibria of the game are all neighbors of each other: Theorem 3 Let EΓ be the set of the loads of the Nash equilibria of the game. Then: ∀x, y ∈ EΓ , x − y =

 u∈supp+ (x− y)

eu −



ev .

v∈supp+ ( y−x)

In other words, all Nash equilibria x, y ∈ EΓ satisfy x − y ∞ ≤ 1. Proof First, if EΓ = {x}, then the theorem is trivially true. Otherwise, assume that there exists x, y ∈ EΓ such that x − y ∞ > 0. Since x and y are in D, we can write x − y as x − y = eu − ev + z for some u, v ∈ J and z ∈ Z J satisfying

The Social Medium Selection Game

255

u = v,  z | eu  ≥ 0 and  z | ev  ≤ 0. Therefore, we have u ∈ supp+ (x − y) and v ∈ supp+ ( y − x). Let a = x − eu + ev and b = y − ev + eu . By Eq. (4), we have: f (x) + f ( y) = 2 f (x) ≤ f (a) + f (b). Hence, we get f (a) = f (b) = f (x) by global maximality of f (x). Then f (x) − f (a) + f ( y) − f (b) = 0, which in turns implies that Nv Nv Nu Nu + . + = xu yv yu + 1 xv + 1 Since xu ≥ yu + 1 and yv ≥ xv + 1, the that xu = yu + 1 and last equation implies  xv = yv − 1. Therefore, x − y = eu − ev .  u∈supp+ (x− y)

v∈supp+ ( y−x)

Using this result, we can find a bound over the number of Nash equilibria: Proposition 1 For any setting Γ , the number of Nash equilibria is upper bounded. More precisely, let EΓ be the set of the loads of the Nash equilibria of Γ . Then:   J |EΓ | ≤  J  . 2 ∗ + Further, this bound  define the  J  is tight. Indeed, let J ≥ 2, m ∈ N and γ ∈ R J. We game Γ by K = 2 and ∀ j ∈ J, N j = m, γ j = γ. Then |EΓ | =  J  . 2

The proof of Proposition 1is given  in Appendix 1. 2J √ Note that the bound is in O J and that it is independent of the number of seeds K . As the number of social media is typically small, then there is a limited number of equilibria.

4 Algorithmic Determination of an Equilibrium In this section, we see how to compute a Nash equilibrium. Note that,from  Theorem 1, J 2 computing all the loads of the Nash equilibria would require Ω √ J operations. Further, for each load vector  maximizing Pot, computing all the corresponding Nash equilibria s ∈ S would require up to O(K !) operations because of the symmetry of the game.

4.1 Maximization of the Potential Consider the following optimization mechanism: step (1) Start with some  ∈ D step (2) Find ∗ the argmax of Pot on V()

256

F. Lebeau et al.

step (3) If ∗ =  then stop step (4) Let  = ∗ and repeat from step (2). Thanks to Theorem 2, we know that this mechanism converges to a vector of loads of a Nash equilibrium. From the point of view of the seeds, it is similar to a guided best response mechanism where at each step the seed which could increase the most the potential by changing its strategy is selected. The problem is that, in the worst cases, this visits all the load vectors   algorithm of the domain D, which leads to O K J steps to find a maximum. However, we can exploit the M-concavity of function f to compute a Nash equilibrium in a far more efficient way. To that end, we adapt the algorithm MODIFIED_STEEPEST_DESCENT given in [15, p. 8] to our problem, which is presented in Algorithm 1 below.

Algorithm 1: SD_MAX Input: Γ = (K , (N j , γ j ) j∈J ) Output: A vector in EΓ 1 Let  = K e1 and b = 0 ∈ Z J 2 while ∃u, u − 1 ≥ bu do  3

Compute v ∈ arg maxt∈J

4 5

bv ← v + 1  ←  − eu + ev

6

Nt t +1

− γt



return 

Proposition2 Algorithm 1 terminates and returns a vector in EΓ with a time com plexity in O K J 2 . Proof We implemented the active domain B of the algorithm used in [15, p. 8] by a vector b satisfying: B=

⎧ ⎨ ⎩

x|



xu = K and ∀u ∈ J, xu ≥ bu

u∈J

⎫ ⎬ ⎭

.

We then remarked that we do not need to compute the potential since v = arg maxt∈J f ( − eu + et ) is equivalent to: ∀t ∈ J, f ( − eu + ev ) − f ( − eu + et ) ≥ 0

Nv Nt − γv − + γt ≥ 0 v + 1 t + 1   Nt ⇔ v = arg maxt∈J − γt . t + 1

⇔ ∀t ∈ J,

We can then apply the same analysis as in [15] for the correctness of the algorithm.

The Social Medium Selection Game

257

 The quantity 0 ≤ u∈J (K − bu ) ≤ K J decreases by at least one at each step of the algorithm. Therefore, the algorithm terminates with at most K J iterations. Moreover, finding a u satisfying the loop condition and computing the  of v can  value be done in O(J ). Hence, this algorithm has a time complexity in O K J 2 .   An algorithm in O J 3 log K /J is further discussed in Appendix 2.

4.2 An Efficient Learning Mechanism Note that the previous algorithm starts with some arbitrary load vector in D and then iteratively finds the best improvement until reaching a global maximum. Instead, we propose a novel approach in which the seeds arrive one by one. We then show that at each arrival of seed k, the strategy sk can be computed in such way that after all arrivals, the resulting vector s is a Nash equilibrium. This approach relies on the following theorem: Theorem 4 Let Γ = (K , (N j , γ j ) j∈J ) be a setting of the game and Γ  = (K + 1, (N j , γ j ) j∈J ) be the setting obtained by adding an extra seed on Γ . Let s be a Nash equilibrium for Γ and σ the strategy profile of Γ  in which the K first seeds choose the same strategy as in s (i.e., sk = σk for all k ≤ K ) and the additional seed chooses one of the social media which maximizes its payoff. Then σ is a Nash equilibrium for Γ  . Formally, let  ∈ EΓ be the load of some Nash equilibrium of Γ and w ∈ J. Then   + ew ∈ EΓ  ⇔ w ∈ arg maxt∈J

 Nt − γt . t + 1

  t Proof Let w ∈ arg maxt∈J tN+1 − γt . We proceed to show that  + ew is in EΓ  . Let u, v ∈ J such that u + δ(u, w) > 0 and u = v. We need to show that Pot( + ew ) ≥ Pot( + ew − eu + ev ). There are three cases detailed below. First, consider that u = w and v = w. Then Pot( + ew ) − Pot( + ew − eu + ev ) = Pot() − Pot( − eu + ev ) so it is proven in this case. Second, consider that v = w. We have Pot( + ew ) − Pot( + ew − eu + ev ) = Pot( + ew ) − Pot( + 2ew − eu ) Nw Nu + γw − γu − = u w + 2 Nu Nw + γw − γu ≥ 0 ≥ − u w + 1 since  ∈ EΓ .

258

F. Lebeau et al.

Third, consider that u = w. We have Pot( + ew ) − Pot( + ew − eu + ev ) = Pot( + ew ) − Pot( + ev ) Nv Nw − + γw − γv ≥ 0 = w + 1 v + 1 by definition of w. w The reciprocal follows from the last formula: If wN+1 − γw was not maximal, then there would be some v such that Pot( + ew ) < Pot( + ev ). Hence,  + ew would not be in EΓ  from Theorem 2. We use Theorem 4 to build an efficient algorithm finding a vector of loads of a Nash equilibrium (Algorithm 2): It begins with no seed and simulates K times the arrival of a seed maximizing its payoff.

Algorithm 2: ORDER_LEARNING Input: Γ = (K , (N j , γ j ) j∈J ) Output: A vector in EΓ 1 Let  = 0 ∈ Z J and k = 1 2 while k ≤ K do  3

Compute w ∈ arg maxt∈J

4 5

 ←  + ew k ←k+1

6

Nt t +1

− γt



return 

Proposition 3 Algorithm 2 terminates and returns a vector in EΓ with a time complexity in O(K J ). Proof Let Γk = (k, (N j , γ j ) j∈J ) for k ∈ {0, ..., K }. Since 0 ∈ Z J is a vector of loads of a Nash equilibrium of Γ0 , then, from Theorem 4, at the end of the kth iteration of the loop,  is a vector of loads of a Nash equilibrium of Γk , hence the correctness of Algorithm 2. Since we can compute w in O(J ) and there are K iterations of the loop, then the time complexity of Algorithm 2 is in O(K J ).

5 Asymptotic Behavior In this section, we discuss the form of the Nash equilibria when we have a lot more seeds than social media, so K  J . We are interested in this case in practice as the activity in the Internet tend to be concentrated in a restricted number of famous Web sites.

The Social Medium Selection Game

259

5.1 Intuition First, we can make a hypothesis about the asymptotic behavior of this game when K → ∞ according to the form of the potential. Recall that Hn ∼ ln(n) + μ, where n→∞ μ is the Euler–Mascheroni constant. Then in order to find approximate Nash equilibria, we can study the function  

P() =

 N j ln( j ) − γ j  j .

j,  j >0

We can see that, for large values of  j , the linear part in γ j  j is determinant compared to the logarithm  part in N j ln( j ). Therefore, we can make the hypothesis that when the quantity j  j = K is large enough, then the only  j that continue to increase are the ones with minimal cost. Then, it seems natural that all social media with minimal cost would behave as if they were in a subgame where new seeds would only choose them.

5.2 Asymptotic Analysis Following our hypothesis, we define γm the minimal cost and G the set of social media with minimal costs: γm = min γ j and G = arg min j γ j . j

We know, thanks to Theorem 4, that when K increases, the coordinates of the loads of the Nash equilibrium we consider can only increase. We proceed to show our intuition. In the following, we note Γ K = (K , (N j , γ j ) j∈J ) and E K = EΓ K . We study the vectors in E K obtained with the mechanism implemented in Algorithm 2. Let (K ) be the vector in E K obtained after the K th iteration of the loop in the algorithm. Theorem 5 When K goes to infinity, at the Nash equilibria, the social media are divided into two groups: • The loads of the social media with non-minimal cost stop increasing when they reach a constant. Formally: ) −→ ∀ j ∈ J \ G, (K j

K →∞



Nj γ j − γm

 − 1.

• The loads of the social media with minimal cost goes to infinity, and the proportion of seeds a social medium get among the one with minimal cost is equal to its market share. Formally:

260

F. Lebeau et al.

Fig. 1 Convergence to the asymptotic behavior. Case with: (N1 , γ1 ) = (100, 2), (N2 , γ2 ) = (25, 1) and (N3 , γ3 ) = (20, 1)

200

SM 1 SM 2 SM 3

Load j

150

100

50

0

0

100

200

300

400

500

Number of seeds K

∀w ∈ G, 

(K ) w

−→ 

(K ) K →∞ t∈G t

Nw t∈G

Nt

.

The proof of Theorem 5 is given in Appendix 3.

6 Numerical Results Figure 1 shows the convergence of the equilibrium when the number of seeds K grows large. Note that there may be up to 3 equilibria and that the plots of the figure correspond to the outputs of Algorithm 2. The asymptotes obtained in Theorem 5 are represented in dashed lines with colors matching those of the loads of their associated social media (SM). The SM 2 and 3, which have minimal cost, have loads growing to infinity with the number of seeds. The asymptote of SM 2 has a higher slope than that of 3 because it has a higher number of subscribers (N2 > N3 ). Finally, while for large values of seeds the cost of the social media is predominant, in contrast, for low values of seeds, the number of customers M plays the larger role in determining the loads of the different social media. Figure  2 shows the evolution of the social welfare, that is, the sum of total utilities, k u k (s), at the Nash equilibrium and at the social optimum. The asymptotic behavior at the Nash equilibria is given by −γm K from Theorem 5. Further, let L = { j, N j ≥ γ j }. Recall that the social optimum is the strategy vector maximizing the social welfare. Then,  for K large enough, the social optimum satisfies max j, j >0 (N j − γ j  j ) = j∈L (N j − γ j ) − γm (K − |L|) ∼ −γm K . Hence, as the number of seeds grows to infinity, the price of anarchy converges to 1. Finally, Figs. 3 and 4 show the sensitivity of the equilibria with respect to N and γ for a case with J = 2 social media.

The Social Medium Selection Game Social Optimum Nash Equilibrium

100

uk (s)

0

k

-100 -200

Sum of utilities



Fig. 2 Sum of utilities at equilibrium compared with the social optimum (same setting that Fig. 1)

261

-300 -400 -500 -600 -700 100

200

300

400

500

600

700

800

Number of seeds K 100

γ1 = 15 γ1 = 10 γ1 = 5

80

Load 2

Fig. 3 Influence of the number of users. Case with 2 social media: N1 = 250, γ2 = 10 and K = 100

60

40

20

0

0

200

400

600

800

1000

Number of users of SM 2: N2

Fig. 4 Influence of the dissemination cost. Case with 2 social media: γ1 = 30, N2 = 250 and K = 100

100

N1 = 100 N1 = 250 N1 = 1000

Load 2

80

60

40

20

0

0

10

20

30

40

50

Cost of SM 2: γ2

60

70

80

262

F. Lebeau et al.

We observe that the load of a social medium is increasing with its number of customers, as expected (Fig. 3). Further, if the dissemination cost of SM 2 is higher or equal to that of SM 1 and if it has no customer, then its load is zero, as exhibited in the red and blue plots. Otherwise, even though it has no customer, if its cost is minimal, it will receive some seeds (green plot). Finally, note that as the cost of SM 1 decreases, the number of customers in the SM 2 has lower effect of the evolution of the load 2 . We also observe that the load of a social medium is decreasing with its dissemination cost, as expected (Fig. 4). Further, numerical results show that the load decreases more abruptly for lower number of users N1 , but that the drop occurs for larger values of γ2 .

7 Conclusion In this paper, we have studied competition for popularity of seeds among several social media. We have shown that the game is equivalent to a congestion game and hence has a potential. We then studied the properties of the potential in terms of M-concavity. We have shown that there may exist several Nash equilibria, all belonging to a single neighborhood and provided examples where the number of equilibria is maximal. We have provided a novel efficient learning algorithm based on a remarkable property of the Nash equilibria in some subgames. We also investigated the asymptotic behavior of the equilibria of the game and the price of anarchy. As future work, we will study the underlying competition among the social media in the Stackelberg setting for a discrete number of seeds: according to their number of subscribers (who consume content), how could they appropriately set up their prices? We further plan to extend our model to the case where seeds have different dissemination utilities for sending to the various media. The game is no more equivalent to a congestion game but turns out to be equivalent to crowding games [16]. This allows to show existence of (pure) equilibria but best response policies need not converge, as there need not be a potential anymore. Thus, designing learning algorithms for this extension is yet an open problem.

Appendix A. Proof of Proposition 1  A.1. Proof of the Upper Bound |EΓ | ≤

J



J 2

Lemma 1 Let x be a load vector at a Nash equilibrium, x ∈ EΓ , and u a social medium, u ∈ J. Then (∃ y ∈ EΓ , xu > yu ) ⇒ (∀z ∈ EΓ , xu ≥ z u ) .

The Social Medium Selection Game

263

Proof Assume that there exists y and z in EΓ such that xu > yu and xu < z u . Then, by Theorem 3, yu = xu − 1 and z u = xu + 1. Hence, z u − yu = 2 which contradicts Theorem 3. Lemma 2 Let α ∈ J. Then min(α,J −α)  k=0

α k



   J −α J = . k α

Proof   We  show this result using a combinatorial argument. First, note that since J J = , then one can restrict the analysis to the case where α ≤ J − α. α J −α We want to select α elements in J. To do that, we partition the set J into two subsets A and B such that |A| = α (so |B| = J − α). Selecting α elements in J amounts to choosing k the number of elements we select in B, then select these k elements and finally select α − k elements in A. Therefore,         min(α,J | A|  α  −α) α J − α J |B| |A| α J −α = = = . α k α−k α−k k k k k=0

k=0



k=0

We can now proceed to the proof of Theorem 1: Proof Let x ∈ EΓ and:  supp+ (x − y) and U = {u | ∃ y ∈ EΓ , xu > yu } = y∈EΓ  supp+ ( y − x). V= y∈EΓ

By Lemma 1, we have U ∩ V = ∅ and |U| + |V| ≤ J . We then define the set A = ⎫  ⎬  C ⊂ U,  . eu + ev  D ⊂ V, x−  ⎭ ⎩ |C| = |D| = k u∈C v∈D ⎧

min(|U |,|V|)⎨

{x} ∪

k=1





We know by Proposition 1 and Lemma 1 that all vectors in EΓ are of the form given in the previous expression; hence EΓ ⊂ A. Let α = |U|. We have |V| ≤ J − α. Then   |U| |V| k k k=1 min(α,J −α) α J − α . ≤1+ k k k=1

|EΓ | ≤ |A| = 1 +

min(|U |,|V|) 

264

F. Lebeau et al.

We conclude the proof by applying Lemma 2, using   theincreasing property of   function J. over {0, ..., J/2} and the fact that Jp = J −J p for all p: |EΓ | ≤

    J J ≤ J . α 2



A.2. A Tight Class of Settings Let J ≥ 2, m ∈ N∗ and γ ∈ R+ . We define the game Γ by K = J, N j = m, γ j = γ.

J 2

and ∀ j ∈

Lemma 3 The Nash equilibria of game Γ satisfy the property:    J  ∈ EΓ ⇒ ∃A ⊂ J, |A| = and  = eu . 2 u∈A Proof Assume that there exists x ∈ EΓ and u ∈ J such that xu > 1. Since K < J , there exists v ∈ J such that xv = 0. Consider the vector y = x − xu eu + xu ev . Since all the N j and γ j are equal, the potential of y is equal to the potential of x. Therefore, y ∈ EΓ . But we have yv − xv = xu > 1 which contradicts Theorem 3 and concludes the proof.  Since EΓ = ∅, let x ∈ EΓ . By Lemma 3, we can note x = u∈A eu for some J A ⊂ J. Let B ⊂ J verifying |B| = 2 and y = v∈B ev . Then, we have     J (m − γ) = Pot(x) = (m − γ) = (m − γ) = Pot( y). 2 u∈A v∈B   !    J  J  = J . Therefore, |EΓ | =  A ⊂ J | |A| = 2  2

Appendix B. A Polynomial Algorithm to Find a Maximum of f A polynomial algorithm to determine the minimum of an M-convex function is given in [15, Sect. 4.2]. We can adapt it for maximizing our M-concave function f . In fact, this algorithm does not have a polynomial complexity in general. We proceed to show the property required which is that the M-concavity of f is respected by a scaling operation.

The Social Medium Selection Game

265

Proposition 4 Let α ∈ N∗ and x ∈ D. We define the function " f as  " f ( y) =

f (x + α y) if x + α y ∈ D −∞ otherwise.

Then " f is an M-concave function. Proof Let y, z ∈ Z J such that x + α y ∈ D and x + αz ∈ D. Let u ∈ supp+ ( y − z). Since α > 0, the same argument as in the proof of the M-concavity of f gives us that there exists some v ∈ supp+ (z − y). Then we calculate " f ( y) = f (x + α( y − eu + ev )) − f (x + α y) f ( y − eu + ev ) − " xv +α(yv +1)



=

i=xv +αyv

=

α  i=1

xu +αyu  1 1 − i i +1 i=x +α(y −1)+1 u

1 − xv + αyv + i

u

α  i=1

1 . xu + α(yu − 1) + i

We have a similar expression for " f (z − ev + eu ) − " f (z). Then " f ( y) + " f (z − ev + eu ) − " f (z) = f ( y − eu + ev ) − "

α  i=1

+

α  i=1

α

 1 1 − xv + αyv + i xv + α(z v − 1) + i i=1 α

 1 1 − . xu + αz u + i yu + α(yu − 1) + i i=1

Since yv ≤ z v − 1 and z u ≤ yu − 1, then the last quantity is positive. Hence, " f is M-concave. In order to implement Algorithm SCALING_MODIFIED_STEEPEST_DESCENT given in [15], we represent the active domain of the search of a maximizer B using two vectors m and M such that ⎧ ⎫ ⎨ ⎬  x j = K and ∀ j, m j ≤ x j ≤ M j . B= x| ⎩ ⎭ j

We choose the origin point to be x = K e1 . Then the only difficulty that remains is to compute some y such that x + α y ∈ B in order to search for a maximum of the scaled auxiliary function. In fact, a solution is also given in [15, Sect. 5.1] with Algorithm FIND_VECTOR_IN_N B which find a vector  y whose components are M−x and and for which within the constraints x−m j y j = 0. This algorithm has α α a time complexity in O(J ).

266

F. Lebeau et al.

  Hence, we have an algorithm to compute some  ∈ EΓ in O J 3 log(K /J ) , which is better than O(K J ) if we study the case when there are a lot more seeds than social media.

Appendix C. Proof of the Asymptotic Behavior Note that, by definition of the learning mechanism implemented in Algorithm 2, for all K ∈ N and j ∈ J we have +1) ) (K = (K + 1 ⇒ j ∈ arg maxt∈J j j



 Nt − γt . t + 1

(5)

C.1. Social Media with Non-minimal Cost We want to prove that ∀ j ∈ J \ G,

) (K j

 −→

K →∞

Nj γ j − γm

 − 1.

(6)

We begin by proving the following two lemmas. Lemma 4 The quantity # M

(K )

= max t∈J

Nt t(K ) + 1

$ − γt

is arbitrarily close to γm for K large enough.2 Proof First, this quantity is decreasing. Moreover, by definition of the (K ) , we have  ) that for all K , (K = K −→ ∞. Therefore, there exists some u ∈ J such that j j

u(K )

K →∞

n +1) n) −→ ∞. It means that there exists (K n )n∈N such that ∀n, (K = (K + 1, u u

K →∞

which implies by (5) that ∀n,

Nu n) (K +1 u

− γu = M (K n ) .

Hence, M (K n ) is arbitrarily close to −γu for n large enough. We conclude by noticing that −γu ≤ −γm . Nu  Lemma 5 Let K > 0 and u ∈ J. Then (K ) − γu > −γm ⇔ ∃ K  > K , u(K ) > u + 1 u(K ) . 2 We denote by

K “large enough” the fact that there exists some K 0 such that the property is verified for all K > K 0 .

The Social Medium Selection Game

Proof First, assume that K we have

Nu ) (K u +1

267

− γu ≤ −γm . Then for some w ∈ G and for all K  ≥

Nu u(K ) + 1

since γw = γm . This implies that 

− γu
K , (5)

) leads to u(K ) = (K u .  Then assume that (KN)u − γu > −γm . According to Lemma 5, M (K ) is arbitraru +1 ily close to −γm for K  large enough. Therefore, there exists K  > K such that Nu   ) M (K ) < (K ) − γu . Hence, u(K ) > (K which concludes the proof.  u u + 1

We can now proceed with the proof of (6). Let j ∈ J \ G. We know by Lemma 5 Nj ) that (K − γ j > −γm . Therefore, for K large increases with K as long as (K ) j j + 1 enough we have ) (K j

! Nj − γ j > −γm . = 1 + max p ∈ N | p+1

Then, let p ∈ N. Since j ∈ J \ G, we have γ j >  γm . We solve Nj Nj Nj (K ) which − γ j > −γm ⇔ p + 1 < . Hence,  j + 1 = p+1 γ j − γm γ j − γm concludes the proof.

C.2. Social Media with Minimal Cost We can directly conclude from Lemma 5 that the load of any social medium having a minimal cost goes to infinity as K increases. Formally: ) −→ ∞. ∀w ∈ G, (K w

(7)

K →∞

(K ) Now we proceed to find the values of w for the social media with minimal ) cost. Let K be large enough so that (6) is verified. Let K G = K − (K be the j j∈J\G

number of seeds sharing the social media in G and  DG = (xt )t∈G |



% xt = K G and ∀t ∈ G, xt > 0 .

t∈G

Consider the game ΓG = (K G , (Nt , γm )t∈G ). From (7), the loads of the social media in G can be arbitrarily high with K large enough, so we determine an approx-

268

F. Lebeau et al.

imation of a load of a Nash equilibrium for the social media in G by solving max P(x) =

x∈RG



(Nt ln(xt ) − γm xt ) s.t. x ∈ DG .

t∈G

Since P is concave, we apply a Lagrangian maximization method. Let L be the Lagrangian for this problem: L(x, λ) = P(x) − λ

# 

$ xt − K G ,

t∈G

where λ and the xt are nonnegative. Since P is concave, the unique maximum x ∗ verifies ∀t ∈ G,

∂L ∗ (x ) = 0. ∂xt

Nt Nt . − γm − λ = 0 ⇔ xt∗ = xt∗ γm + λ Now we determine the value of λ: # $   Nt 1  ∗ xt = K G ⇒ Nt − γm . = KG ⇒ λ = γ +λ K G t∈G t∈G t∈G m

Therefore, we get that for any t:

Hence, ∀w ∈ G, xw∗ = K G  Nw Nt . t∈G Thanks to (7) and since Hn − μ ∼ ln n, we finally get that ∀w ∈ G, 

) (K w

t∈G

t(K )

n→∞

−→ 

K →∞

Nw

t∈G

Nt

.

References 1. A. May, A. Chaintreau, N. Korula, and S. Lattanzi, “Game in the newsroom: Greedy bloggers for picky audience,” in Proc. of the 20th International Conference Companion on World Wide Web, February 2013, pp. 16–20. 2. E. Altman, “A semi-dynamic model for competition over popularity and over advertisement space in social networks,” in 6th International Conference on Performance Evaluation Methodologies and Tools, Oct. 2012, pp. 273–279. 3. A. Reiffers Masson, E. Altman, and Y. Hayel, “A time and space routing game model applied to visibility competition on online social networks,” in Proc. of the International Conference on Network Games, Control and Optimization, 2014. 4. N. Hegde, L. Massoulié, and L. Viennot, “Self-organizing flows in social networks,” in Structural Information and Communication Complexity, ser. Lecture Notes in Computer Science. Springer International Publishing, 2013, vol. 8179, pp. 116–128. 5. Z. Lotker, B. Patt-Shamir, and M. R. Tuttle, “A game of timing and visibility,” Games and Economic Behavior, vol. 62, no. 2, pp. 643 – 660, 2008.

The Social Medium Selection Game

269

6. L. Maggi and F. De Pellegrini, “Cooperative online native advertisement: A game theoretical scheme leveraging on popularity dynamics,” in Proc. of IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), 2014, pp. 334–339. 7. N. Yadati and R. Narayanam, “Game theoretic models for social network analysis,” in Proc. of the 20th International Conference Companion on World Wide Web, 2011, pp. 291–292. 8. R. Narayanam and Y. Narahari, “A game theory inspired, decentralized, local information based algorithm for community detection in social graphs,” in Pattern Recognition (ICPR), 2012 21st International Conference on, 2012, pp. 1072–1075. 9. ——, “A Shapley value-based approach to discover influential nodes in social networks,” Automation Science and Engineering, IEEE Transactions on, vol. 8, no. 1, pp. 130–147, 2011. 10. R. W. Rosenthal, “A class of games possessing pure-strategy Nash equilibria,” International Journal of Game Theory, vol. 2, no. 1, 1973. 11. K. Murota, “Discrete convex analysis,” Mathematical Programming, vol. 83, pp. 313–371, 1998. 12. R. Ma, D. Chiu, J. Lui, and V. Misra, “On resource management for cloud users: A generalized Kelly mechanism approach,” Technical Report, CS, Columnia Univ, NY, 2010. 13. G. Tullock, “Efficient rent-seeking,” in Efficient Rent Seeking, 2001, pp. 3–16. 14. D. Monderer and L. S. Shapley, “Potential Games,” Games and Economic Behavior, vol. 14, 1996. 15. S. Muriguchi, K. Murota, and A. Shioura, “Scaling Algorithms for M-convex Function Minimization,” IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, vol. E85-A, pp. 922–929, 2002. 16. I. Milchtaich, “Congestion games with player-specic payoff functions,” Games and Economic Behavior, vol. 13, pp. 111–124, 1996.

Public Good Provision Games on Networks with Resource Pooling Mohammad Mahdi Khalili, Xueru Zhang and Mingyan Liu

Abstract We consider the interaction of strategic agents and their decision-making process toward the provision of a public good. In this interaction, each user exerts a certain level of effort to improve his own utility. At the same time, the agents are interdependent and the utility of each agent depends not only on his own effort but also on the other agents’ effort level. As the agents have a limited budget and can exert limited effort, question arises as to whether there is advantage to agents pooling their resources. In this study, we show that resource pooling may or may not improve the agents’ utility when they are driven by self-interest. We identify some scenarios where resource pooling does lead to social welfare improvement as compared to without resource pooling. We also propose a taxation–subsidy mechanism that can effectively incentivize the agents to exert socially optimal effort under resource pooling. Keywords Public good · Resource pooling · Social welfare

1 Introduction The interactions among strategic agents form a network game [5, 10], and the provision of public goods is one particular type of such games when their decision-making processes concern the provision of a public good [3, 6, 15]. The interdependent security (IDS) game [9, 12, 16] is one such example. Other examples of a network game include networked Cournot competition [4, 7]. The goal in studying these games This work is supported by the NSF under grant CNS-1422211, CNS-1616575 and CNS-1739517. M. M. Khalili (B) · X. Zhang · M. Liu University of Michigan, Ann Arbor, MI, USA e-mail: [email protected] X. Zhang e-mail: [email protected] M. Liu e-mail: [email protected] © Springer Nature Switzerland AG 2019 J. Walrand et al. (eds.), Network Games, Control, and Optimization, Static & Dynamic Game Theory: Foundations & Applications, https://doi.org/10.1007/978-3-030-10880-9_16

271

272

M. M. Khalili et al.

is often to characterize their equilibrium and the effect of the underlying network structure on such equilibrium. Such findings can help policy and network designers make better decisions in inducing desirable outcomes. The conditions for existence and uniqueness of equilibrium in a network game with linear best-response functions have been studied in [11, 15]. Miura-Ko et al. [11] consider a security decision-making problem and find a sufficient condition for the uniqueness of the equilibrium, while [15] considers the provision of a public good and introduces a necessary and sufficient condition for the uniqueness of the equilibrium. Network games with nonlinear best-response functions have also been studied in the literature [2, 14]. Acemoglu et al. [2] show that if the best-response function is a contraction or non-expansive mapping, then the equilibrium is unique. Naghizadeh et al. [14] provide a condition on the smallest eigenvalue of a matrix composed of interdependency factor and derivative of the best-response function to guarantee the uniqueness of equilibrium. Similarly, a potential game for modeling the Cournot competition is introduced in [1], where it is shown that if the cost function is strictly convex, then the equilibrium is unique, while [4] provides a sufficient condition for uniqueness of equilibrium and studies the effect of the competition structure on the firms’ profit. Of equal interest in the context of network games is the question of designing mechanisms that induce network games with desirable equilibrium properties. For instance, in the literature of IDS games where the public good being provisioned is agents’ investment in security, incentive mechanisms have been proposed to induce higher levels of effort by agents. In [9], Grossklags et al. suggest bonus and penalty based on agents’ security outcome, while Parameswaran et al. [16] propose a mechanism to overcome the free-riding/underinvestment issue, where an authority monitors user investment. In this paper, we study a generic problem of public good provision on networks, where each agent/player exerts a certain effort (the provision of a public good) that impacts themselves as well as their neighbors on a connectivity graph. The difference between this and most prior work (including those cited above) is that we assume each agent has a budget constraint but may be allowed to pool their resources. Specifically, we consider three different scenarios: (i) Agents are not allowed to pool their resources. (ii) Agents are free to pool their resources but will do so selfishly. (iii) Agents are free to pool their resources and are incentivized to do so optimally (in terms of social welfare) and voluntarily. Case (i) is a baseline scenario. The agents’ utility in this case is also considered their outside option when presented with a mechanism. Case (ii) allows us to investigate the behavior of strategic agents when resource pooling is allowed but not regulated. As we will show the Nash equilibrium in this case may lead to improved as well as worsening social welfare as compared to Case (i). In Case (iii), we design a mechanism to incentivize agents to choose socially optimal actions in resource pooling; this mechanism is budget-balanced, incentive-compatible, and individually rational.

Public Good Provision Games on Networks with Resource Pooling

273

The remainder of the paper is organized as follows. We present the model, preliminaries, and Case (i) in Sect. 2. We then analyze Case (ii) in Sect. 3. We propose a mechanism that attains the socially optimal solution at the equilibrium of its induced game in Sect. 4. We present numerical examples in Sect. 5 and conclude the paper in Sect. 6.

2 Model: A Scenario Without Resource Pooling We study the interaction of N agents in a directed network G = (V, E), where V = {1, 2, · · · , N } is a set of N agents and E = {(i, j)|i, j ∈ V } is a set weighted edges between them. An agent i ∈ V has limited budget Bi and chooses to exert effort xi ∈ [0, Bi ] toward improving his utility. Agent i’s payoff depends on his own effort, as well as the effort exerted by others with nonzero influence on i. An edge (i, j) ∈ E indicates that agent j depends on agent i (or that agent i influences j) with edge weight gi j ∈ [0, 1]. The dependence need not be symmetrical, i.e., gi j = g ji in general. We shall assume gii = 1, i = 1, 2, · · · , N and gi j < 1, ∀i = j to reflect the notion that an agent is his own biggest influence. Let x = [x1 , x2 , · · · , x N ] be the profile of exerted efforts by all N agents. Then, the utility of agent i is given by, u i (xx ) =



g ji μ j (x j ) ,

(1)

j∈V

where μi (a) is a function determining the benefit to agent i from effort a. We will assume that μi (.) is differentiable and strictly increasing for all i and that μi is strictly concave. This implies that while the initial effort leads to a considerable increase in utility, the marginal benefit decreases as effort increases. Since μi (.) is strictly increasing and agents are strategic, the best strategy for each agent is to use all of his budget. Therefore, vio =



g ji μ j (B j )

(2)

j∈V

is the highest utility that agent i achieves without resource pooling. We shall take vio to be the participation constraint of agent i in deciding whether there is incentive to participate in any mechanism.

3 Public Good Provision with Resource Pooling Consider now the scenario where the agents can pool their resources. Let x i = [xi1 , xi2 , · · · , xi N ]T be the action of agent i where xi j ≥ 0 denotes the effort exerted by agent i on behalf of agent j, e.g., by transferring part of its budget to j.

274

M. M. Khalili et al.

Let X = [xx 1 , x 2 , · · · , x N ]T be the profile of the exerted efforts, an N × N matrix. Due to the increasing nature of μ j , we have Nj=1 xi j = Bi . As a result, the agent’s utility given action profile X is: vi (X = [xx 1 , x 2 , · · · , x N ]T ) =



g ji · μ j (

j∈V

N 

xk j )

(3)

k=1

Notice that vi (X ) is concave in X , but it is not necessary strictly concave. Moreover, vi (X ) is strictly concave in x i . This game is denoted as G P and given by the tuple (V, {xi j |i, j ∈ V }, {vi (.)|i ∈ V }). Let Bi (xx −i ) denote the best-response function of agent i: Bi (xx −i ) = arg max xi

s.t.

N 

 j∈V

g ji · μ j (

N 

xk j )

(4)

k=1

xi j = Bi , xi j ≥ 0, j = 1, 2, · · · , N

j=1

Since vi (X ) is strictly concave in x i , this maximization has a unique solution and can be solved using KKT conditions, for j = 1, 2, · · · , N : N −g ji · μj ( k=1 xk j ) − λ j + ν = 0 , N λ j · xi j = 0, xi j ≥ 0, j=1 x i j = Bi , λ j ≥ 0 ,

(5)

where λ j and ν are dual variables of the jth inequality and the equality constraints, respectively. We can simplify above KKT conditions and write the best response of agent i given action profile x −i as follows: ⎡

⎤+  ν xi j = ⎣(μj )−1 ( ) − xk j ⎦ , g ji k∈V −{i}

(6)

where [a]+ = max{0, a}. A pure-strategy Nash equilibrium (NE) of the public good provision game is a matrix of efforts X = [xx i , x −i ]T , for which vi (xx i , x −i ) ≥ vi (xx i , x −i ), ∀xx i ∈ Si , ∀i,

(7)

where Si = {(s1 , s2 , · · · , s N )|s1 + s2 + · · · + s N = Bi , s j ≥ 0 ∀ j ∈ V }. We have the following result on the existence of NE in the game G P . Theorem 1 The game G P (public good provision with resource pooling) has at least one NE.

Public Good Provision Games on Networks with Resource Pooling

275

Proof The strategy spaces Si are nonempty, compact, and convex. Moreover, payoff functions vi (.) are continuous and concave in x i for all i ∈ V . Using [8], there exists at least one pure-strategy Nash equilibrium.  Let vi∗ be the utility of agent i at the NE in game G P . Then, in general, we may or may not have vi∗ ≥ vio ; i.e., agents do not necessarily gain from resource pooling. In Sect. 5, we give some examples. There are however special cases where all agents obtain higher utility in G P as we show below. We begin by characterizing the NE. Lemma 1 Let X be the action profile of agents in the NE of G P . Then, x i j · x ji = 0, ∀i = j. N N x ki ) ≤ g ji μj ( k=1 x k j ) since Proof Let us assume that x i j = 0. Note that μi ( k=1 otherwise agent i can improve its utility by increasing x ii and decreasing x i j . Therefore, we have μi (

N k=1

x ki ) ≤ g ji μj (

N k=1

x k j ) → gi j μi (

N k=1

x ki ) < μj (

N k=1

xkj ) ,

(8)

which implies that x ji = 0 since otherwise agent j can improve its utility by decreasing x ji and increasing x j j . As a result, x i j · x ji = 0.  Lemma 1 says that if agent i improves agent j’s utility by offering nonzero xi j , then agent j will necessarily set x ji = 0. Next, we provide some examples where the agents’ utility improves in the NE of game G P as compared to that without resource pooling (vio ). Theorem 2 1. Let N = 2. Then, vi∗ ≥ vio , i = 1, 2. In other words, in a network consisting of two agents, both achieve equal or higher utility at the NE under game G P . 2. Consider a network consisting of one parent (agent 1) and N − 1 children: gi j = 0 if i = j and i, j > 1. Moreover, assume that μi (x) = μ j (x) and Bi = B j and gi1 = g j1 and g1i = g1 j for all i, j > 1. Let vi∗ be the utility of the agent i at the symmetric Nash equilibrium where x i1 = x j1 and x 1i = x 1 j for all i, j > 1. Then vi∗ ≥ vio , ∀i ∈ V . Proof 1. Let X be the action profile of the agents in the NE. Then, by Lemma 1, we know that x 12 = 0 or x 21 = 0. Without loss of generality, let us assume that x 12 = 0. Therefore, x 11 = B1 . By the definition of NE, we have v2∗ = μ2 (x 22 ) + g12 μ1 (B1 + x 21 ) ≥ μ2 (B2 ) + g12 μ1 (B1 ) = v2o .

(9)

Next, we show that v1∗ ≥ v1o . We proceed by contradiction. Let us assume μ1 (B1 + x 21 ) + g21 μ2 (x 22 ) = v1∗ < v1o = μ1 (B1 ) + g21 μ2 (B2 ). This implies that decrease in x 21 and increase in x 22 improve the utility of agent 1 or equivalently μ1 (B1 + x 21 ) < g21 μ2 (x 22 ). This is a contradiction since agent 1 can improve its utility by decreasing x 11 and increasing x 12 . Therefore, v1∗ ≥ v1o .

276

M. M. Khalili et al.

2. Consider a symmetric equilibrium where x i1 = x j1 and x 1i = x 1 j for all i, j > 1. By Lemma 1, we know that x i1 = x j1 = 0 or x 1i = x 1 j = 0. Therefore, we consider two different cases: Case 1: x i1 = x j1 = 0. Let x 1i = x 1 j = x. We have v1∗ = μ1 (x 11 ) + (N − 1)g21 μ2 (B2 + x) μ1 (B1 ) + (N − 1)g21 μ2 (B2 ) = v1o

≥  Definition of NE

(10)

To show that v2∗ ≥ v2o , we proceed by contradiction. Let us assume that v2∗ < v2o . Then, we have μ2 (B2 + x) + g12 μ1 (x 11 ) < μ2 (B2 ) + g12 μ1 (B1 ) → μ2 (B2 + x) ≤ g12 μ1 (x 11 ) → g21 μ2 (B2 + x) < μ1 (x 11 )

(11)

The last equation implies that agent 1 can improve its utility in NE by decreasing x and increasing x 11 . This is a contradiction and v2∗ ≥ v2o . Case 2: x 1i = x 1 j = 0. The proof is similar to case 1.  Theorem 2 provides two examples where game G P can improve social welfare with resource pooling. While this cannot be guaranteed in general, we next design a mechanism guaranteed to induce a public good provision game with resource pooling where agents exert the socially optimal efforts at its NE.

4 A Mechanism with Socially Optimal Outcome In this section, we present a taxation mechanism that implements the socially optimal solution at the NE of the game it induces in a decentralized setting. We begin by defining socially optimal strategies. that solves the A socially optimal strategy is a strategy profile X ∗ = xi∗j N ×N

following optimization problem (total utility): X ∗ ∈ arg

max T X =[x i ;xx −i ] ,xx i ∈Si

N 

vi (X ) .

(12)

i=1

Notice that vi (X ) is concave in X , but it is not necessarily strictly concave. Therefore, socially optimal effort profile X ∗ is not necessarily unique. Generally, agents’ actions at the NE of a game are not socially optimal; this is the case in the game G P as we showed in the previous section. To induce socially optimal behavior, the approach of mechanism design is often used, whereby incentives are

Public Good Provision Games on Networks with Resource Pooling

277

provided to induce a different game whose equilibrium coincides with the socially optimal solution. We next describe such a mechanism based on taxation. In designing such a mechanism, we will assume that μi (.) and Bi are the private information of agent i unknown to the mechanism designer. Let ti be the tax (punishment) levied on agent i by the mechanism designer; ti < 0 is also referred to as a subsidy (reward). Agent i’s utility after ti is given by: ri (X, ti ) = vi (X ) − ti . (13) A taxation mechanism is budget-balanced if the taxes at equilibrium are such N ti = 0; i.e., the mechanism designer neither seeks to make money from that i=1 the agentsnor injects money  N into the system. Under a budget-balanced mechanism, N vi (X ) = i=1 ri (X, ti ). Our goal is to design a taxation mechanism we have i=1 which satisfies the following conditions: 1. The game induced by the mechanism has a NE. 2. The mechanism is socially optimal; i.e., it implements the socially optimal efforts at all NEs of the game it induces. 3. The mechanism is budget-balanced. 4. The mechanism is individual rational; i.e., the utility of agent i at the NE of the induced game is at least vio for all i.1 Our mechanism is inspired by [17] and satisfies all of the above conditions.  NA decentralized mechanism consists of a game form (M, h) where M := i=1 Mi and Mi is the set of all possible messages of agent/player i. Moreover, h : M → A is the outcome function and determines the effort profile and tax profile. Note that A is the space of all effort and tax profiles. The game form (M, h) together with utility functions ri (.) defines a game given by (M, h, {ri (.)}). We refer to this game as the game induced by the mechanism. A message profile of the decentralized mechanism m = [m 1 , m 2 , · · · , m N ] is a Nash equilibrium of this game if ri (h(m i , m −i )) ≥ ri (h(m i , m −i )), ∀m i , ∀i .

(14)

The components of our decentralized mechanism are as follows. The Message Space: Each agent i reports message m i = (xx (i) , π (i) ), where x (i) is a vector with N · (N − 1) elements and is agent i’s suggestion of all agents’ effort profiles. In other words,

(i) (i) (i) (i) (i) , x13 , · · · , x1N , x21 , x23 , · · · , x N(i)(N −1) , x (i) x (i) = x12 jk ∈ R, j  = k,

1 This

(15)

is a weaker condition than voluntary participation, which requires that an agent’s utility in the mechanism with everyone else is no less than his utility when unilaterally opting out. It has been shown in [13] that it is generally impossible to simultaneously achieve social optimality, weak budget balance, and voluntary participation.

278

M. M. Khalili et al.

where x (i) jk is agent i’s suggestion of agent j’s effort to improve agent k’s utility. Note (i) (i) that x has only N · (N − 1) elements because x (i) j j , j = 1, 2, · · · , N are not in x (i) but are completely determined by x . π (i) is a price vector of real positive numbers used by the designer to determine the tax of each agent. Similar to x (i) , π (i) has N · (N − 1) elements:

(i) (i) (i) (i) (i) (i) , π13 , · · · , π1N , π21 , π23 , · · · , π (i) π (i) = π12 N (N −1) , π jk ∈ R+ , j  = k .

(16)

The Outcome Function: The outcome function determines the tax profile for each m ). The investment profile xˆ (m m ) is calculated agent as well as investment profile xˆ (m as follows, N (17) x (k) . xˆ (m m ) = N1 k=1 The amount of tax tˆi agent i pays is given by m ) = (π π (i+1) − π (i+2) )T xˆ (m m) tˆi (m π (i) )(xx (i) − x (i+1) ) +(xx (i) − x (i+1) )T diag(π x (i+1)

−(xx

x (i+2) T

−x

π (i+1)

) diag(π π

x (i+1)

)(xx

(18) x (i+2)

−x

).

Note that N + 1 and N + 2 are treated as 1 and  2 in (18). For the notational convem ) = Bi − k∈V −{i} xˆik , ∀i ∈ V. nience and future use, we define xˆii (m Note that  NEq. (18) implies that the proposed mechanism is always budget-balanced m ) = 0, ∀m m . Moreover, we have the following lemma on the tax because i=1 tˆi (m term of the proposed mechanism at the NE of the induced game. Lemma 2 Let m be a Nash equilibrium of the proposed mechanism, and m i = m ). Then, (xx (i) , π (i) ) and x = xˆ (m m ) = (π π (i+1) − π (i+2) )T · x ∀i tˆi (m Proof The proof is provided in appendix.

(19) 

π (i) )(xx (i) − x (i+1) ) and Lemma 2 implies that both terms (xx (i) − x (i+1) )T diag(π (i+2) T (i+1) (i+1) (i+2) π −x ) diag(π )(xx −x ) in (18) vanish at the equilibrium of (xx the proposed mechanism. The inclusion of these two terms is necessary to make sure that the mechanism implements the socially optimal effort profile at each NE. We next show that the proposed mechanism is individually rational. x (i+1)

Theorem 3 Let m be a NE of the proposed mechanism. Then, the agents achieve higher utility at the NE than their outside option if they all opt out. That is, m ), tˆi (m m )) ≥ vio , ∀i ∈ V , where Xˆ (m m ) = [xˆi j ] N ×N is the matrix of agents’ ri ( Xˆ (m effort. Proof Proof is provided in appendix.



Public Good Provision Games on Networks with Resource Pooling

279

We are now ready to prove the main theorem about socially optimal mechanism m ) is the outcome of with resource pooling. The next theorem shows that if Xˆ (m m ) is a solution to the the proposed mechanism at a Nash equilibrium, then Xˆ (m optimization problem (12). In other words, the NEs of the game induced by proposed mechanism implement a socially optimal effort profile. Note that the game does not necessarily have a unique NE in terms of the messages, and the outcome is one of the socially optimal solutions to optimization problem (12). Theorem 4 Let m be a NE of game (M, h, {ri (.)}) induced by the proposed mechm ) is a optimal solution to optimization problem (12). anism. Then, Xˆ (m Proof Let m = [(xx (1) , π (1) ), · · · , (xx (N ) , π (N ) )] be a NE of the proposed mechanism. By definition of the Nash equilibrium, we can write, m ), tˆi (m m )) ri ( Xˆ ((xx (i) , 0 ), m −i ), ti ((xx (i) , 0 ), m −i )) ≤ ri ( Xˆ (m

(20)

π (i+1) − π (i+2) )T xˆ ((xx (i) , 0 ), By Lemma 2 and Eq. (18), ti ((xx (i) , 0 ), m −i ) = (π m −i )). Therefore, (20) can be written as follows, π (i+1) − π (i+2) )T xˆ ((xx (i) , 0 ), m −i )) ≤ vi (A(xˆ ((xx (i) , 0 ), m −i ))) − (π (i+1) (i+2) T m ))) − (π π vi (A(xˆ (m −π ) x ∀xx (i)

(21)

where ⎡ ⎢ ⎢ A(xx ) = ⎢ ⎣

B1 −



k∈V −{1} x 1k .. .

xN1 Substituting x =

1 (xx (i) N

+



k∈V −{i} x

⎤ x12 · · · x1N ⎥ ⎥ .. . . .. ⎥ . . . ⎦

 x N 2 · · · B N − k∈V −{N } x N k (k)

(22)

) and using (21), we have

π (i+1) − π (i+2) )T x , x = arg max{xx ∈R N (N −1) ,A(xx )∈S} vi (A(xx )) − (π

(23)

   where S is the feasible set of effort profiles: S = {X = xi j ∈ R N ×N | Nj=1 xi j = Bi , xi j ≥ 0, ∀i, j ∈ V }. Because the optimization in (23) is convex, KKT conditions are necessary and sufficient for the optimality of x . The KKT conditions for (23) are given by: π (i+1) − π (i+2) ) − x vi (A(xx )) − λ i + θ i = 0, (π λi )T x = 0, (λ  θij (−B j + k∈V −{ j} x jk ) = 0, λi

λ ≥ 0, θi ≥ 0 ,

(24)

280

M. M. Khalili et al.

T 2 λ i = λi12 , λi13 , · · · , λi1N , λi21 , λi23 , · · · , λi(N −1)N ∈ R+N −N and θ i = ⎤T

where ⎡

⎢ i N 2 −N i i i i i ⎥ . Here, λijk is the dual vari⎣θ1 , · · · , θ1 , θ2 , · · · , θ2 , · · · , θ N , · · · , θ N ⎦ ∈ R+    N −1 times

N −1 times

N −1 times

able corresponding to constraint x i j ≥ 0 and θij is the dual variable corresponding  to constraint k∈V −{ j} x jk ≤ B j . Summing (24) over all i in V we get   x )) − λ + θ = 0 − i∈V x vi (A(x λ T x = 0,  θ j (−B j + k∈V −{ j} , x jk ) = 0, λ ≥ 0, θ ≥0.

whereθθ =

 i∈V



(25)

⎤T

θ i = ⎣θ 1 , · · · , θ 1 , θ 2 , · · · , θ 2 , · · · , θ N , · · · , θ N ⎦ , θ j =   

 i∈V

θij

N −1 times N −1 times N −1 times  and λ = i∈V λ i . Note that (25) is the KKT conditions for following convex optimization problem:  max{xx ∈R N (N −1) } i∈V vi (A(xx )) s.t., (26) xi j ≥ 0 ∀i = j  k∈V −{ j} x jk ≤ B j ∀ j ∈ V .

Because (26) is a convex problem, then KKT conditions (25) are necessary and sufficient for the optimal solution of (26). As a result, x is a socially optimal effort profile.  m) As we can see from the proof of Theorem 4, socially optimal effort profile Xˆ (m is individually optimal at the NE of the induced game. That is, m ) = A(x), where, Xˆ (m π (i+1) − π (i+2) )T x . x = arg max{xx ∈R N (N −1) ,A(xx )∈S} vi (A(xx )) − (π

(27)

Theorem 4 implies that the NEs of the game induced by the proposed mechanism implement a socially optimal effort profile. We next show the converse of Theorem 4, i.e., given any socially optimal effort profile X ∗ , the induced game has at least one NE which implements effort profile X ∗ . Theorem 5 Let X ∗ be a socially optimal effort profile. Then, there is a Nash equilibrium of game (M, h(.), {ri (.)}) induced by the proposed mechanism which implements socially optimal effort profile X ∗ . Proof The proof is provided in appendix.



Public Good Provision Games on Networks with Resource Pooling

281

Theorem 4 and 5 together imply that the game induced by the proposed mechanism always has at least a Nash equilibrium and each Nash equilibrium implements a socially optimal effort profile.

5 Numerical Example 5.1 An Example of Three Interdependent Agents In this section, we provide an example of three interdependent agents. Consider the following parameters. N = 3, gi j = e−1 , ∀i = j, μ1 (y) = μ2 (y) = μ3 (y) = −e−y B1 = 5, B2 = B3 = 1 . The utility of the agents without resource pooling is given by v1o = −e−5 − 2e−2 v2o = −e−1 − e−6 − e−2 v3o = −e−1 − e−6 − e−2 .

(28)

If we consider G P , then the best response of agent i is given by ⎡





⎤ x11 ⎥ ⎢ Br1 (xx −1 ) = ⎣[− ln ν1 − 1 − x22 − x32 ]+ ⎦ = ⎣x12 ⎦ , x13 [− ln ν1 − 1 − x33 − x23 ]+ [− ln ν1 − x21 − x31 ]+

(29)

where ν1 is a nonnegative number and is determined by the budget constraint x11 + x12 + x13 = B1 . Similarly, we can write the best-response function of the other agents as follows. ⎡



⎡ ⎤ x21 ⎥ ⎢ Br2 (xx −2 ) = ⎣ [− ln ν2 − x12 − x32 ]+ ⎦ = ⎣x22 ⎦ , x23 [− ln ν2 − 1 − x33 − x13 ]+ [− ln ν2 − 1 − x11 − x31 ]+





⎡ ⎤ x31 ⎥ ⎢ Br3 (xx −3 ) = ⎣[− ln ν3 − 1 − x22 − x12 ]+ ⎦ = ⎣x32 ⎦ . x33 [− ln ν3 − x13 − x23 ]+ [− ln ν3 − 1 − x21 − x11 ]+

(30)

The fixed point of these three best-response mappings is given by

(31)

282

M. M. Khalili et al.



⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 3 0 0 x 11 x 21 x 31 ⎣x 12 ⎦ = ⎣1⎦ , ⎣x 22 ⎦ = ⎣1⎦ , ⎣x 32 ⎦ = ⎣0⎦ , 1 0 1 x 13 x 23 x 33

(32)

and the agents’ utility at the NE of game G P is given by v1 (X ) = −3e−3 v2 (X ) = −e−2 − e−4 − e−3 −2 v3 (X ) = −e − e−4 − e−3  X = x i, j 3×3 .

(33)

It is easy to check that vi (X ) ≥ vio . Therefore, in this example the utility of agents at the NE of the public good game with resource pooling is higher than that without resource pooling. It is also easy to calculate the socially optimal efforts: X ∗ ∈ arg max X ∈S v1 (X ) + v2 (X ) + v3 (X ) = −(1 + 2e−1 ) · (exp{−x11 − x21 − x31 }+ exp{−x12 − x22 − x32 } + exp{−x13 − x23 − x33 }) ⎡5⎤ ⎡1⎤ ⎡1⎤ x ∗1 =

3 ⎢5⎥ ⎢ ⎥ ⎣3⎦ 5 3

, x ∗2 =

3 ⎢1⎥ ⎢ ⎥ ⎣3⎦ 1 3

, x ∗3 =

3 ⎢1⎥ ⎢ ⎥ ⎣3⎦ 1 3

(34)

.

This is one of the socially optimal strategies, and there exits a NE of the induced game which implements this socially optimal effort profile.

5.2 An Example where Resource Pooling Worsens Social Welfare Consider an example of N = 12 interdependent agent, where g21 = e−1 and g1 j = e−1 , ∀ j > 2, gii = 1, ∀i, and all other edge weights are zero. Moreover, consider the following parameters: μi (y) = −e−y , ∀i B1 = 3, Bi = 0, ∀i ≥ 2 . The utility of the agents when the first agent does not pool his resource is: v1o = −e−3 − e−1 v2o = −1 vio = −1 − e−4 , ∀i ≥ 3 .

(35)

Public Good Provision Games on Networks with Resource Pooling

283

At the same time, it is easy to see that if the first agent chooses x 11 = 2 and x 12 = 1, then his utility is maximized. We have the following utilities at the NE after this resource pooling: v1 (X ) = −2e−2 v2 (X ) = −e−1 vi (X ) = −e−3 − 1, ∀i ≥ 3

(36)

x 11 = 2, x 12 = 1, x i j = 0, ∀i, j and (i, j) = (1, 1) or (1, 2) . 12 12 o In this example, i=1 vi (X ) < i=1 vi . That is, resource pooling worsens the ∗ = 3 and xi∗j = social welfare without the proposed mechanism. Incidentally, x11 0, ∀(i, j) ∈ V × V − {(1, 1)} constitute the socially optimal effort profile.

6 Conclusion We studied a public good provision game with resource pooling. We showed that resource pooling does not necessarily improve social welfare when agents act selfishly. We then presented a tax-based mechanism which incentivizes agents to pool their resources in a desirably manner. This mechanism is budget-balanced and implements the socially optimal solution at the Nash equilibrium of the game it induces.

Appendix Proof (Lemma 2) Let m be a Nash equilibrium of the game induced by proposed mechanism, and m i = (xx (i) , π (i) ). We need to show that the following term is equal to zero at NE: π (i) )(xx (i) − x (i+1) ) (xx (i) − x (i+1) )T diag(π π (i+1) )(xx (i+1) − x (i+2) ) . −(xx (i+1) − x (i+2) )T diag(π

(37)

Because m is the Nash equilibrium, we have m ), tˆi (m m )), ∀m i ∈ Mi . ri ( Xˆ (m i , m −i ), tˆi (m i , m −i )) ≤ ri ( Xˆ (m

(38)

We substitute m i = (xx (i) , π (i) ) in (38). Using (17) and (18), we have, ri ( Xˆ ((xx (i) , π (i) ), m −i ), tˆi ((xx (i) , π (i) ), m −i )) = m ), tˆi ((xx (i) , π (i) ), m −i )) ≤ ri ( Xˆ (m m ), tˆi (m m )), ∀π π (i) ∈ R+N (N −1) . ri ( Xˆ (m

(39)

284

M. M. Khalili et al.

Because ri (., .) is a decreasing function in ti , (39) implies that, m ), ∀π π (i) ∈ R+N (N −1) . tˆi ((xx (i) , π (i) ), m −i ) ≥ tˆi (m

(40)

In other words, π (i+1) − π (i+2) )T x (π π (i) )(xx (i) − x (i+1) ) +(xx (i) − x (i+1) )T diag(π π (i+1) )(xx (i+1) − x (i+2) ) ≤ −(xx (i+1) − x (i+2) )T diag(π π (i+1) − π (i+2) )T x (π

(41)

π (i) )(xx (i) − x (i+1) ) +(xx (i) − x (i+1) )T diag(π π (i+1) )(xx (i+1) − x (i+2) ) , −(xx (i+1) − x (i+2) )T diag(π π (i) ∈ R+N ·(N −1) . ∀π By simplifying the above equation, we have π (i) − π (i) )(xx (i) − x (i+1) ) ≥ 0 , ∀π π (i) ∈ R+N ·(N −1) . (xx (i) − x (i+1) )T diag(π

(42)

Because the above equation is valid for all π (i) ∈ R+N (N −1) , it implies π (i) )(xx (i) − x (i+1) ) = 0, ∀i ∈ V . (xx (i) − x (i+1) )T diag(π

(43)

Therefore, at the NE we have m ) = (π π (i+1) − π (i+2) )T x , ∀i ∈ V . tˆi (m

(44) 

Proof (Theorem 3) Let m be a NE of the proposed mechanism, and m i = (xx (i) , π (i) ) m ). By definition, we have and x = xˆ (m m ), tˆi (m m )), ri ( Xˆ ((xx (i) , π (i) ), m −i ), tˆi ((xx (i) , π (i) ), m −i )) ≤ ri ( Xˆ (m ∀m i = (xx (i) , π (i) ) ∈ Mi Let x˜ (i) be a vector such that By Lemma 2, we have

1 (x˜ (i) N

+



k∈V −{i} x

(k)

(45)

) = 0. Moreover, we set π (i) = 0.

xˆ ((x˜ (i) , 0 ), m −i ) = 0 Xˆ ((x˜ (i) , 0 ), m −i ) = diag(B1 , B2 , · · · , B N ) tˆi ((x˜ (i) , 0 ), m −i ) = 0 ri ( Xˆ ((x˜ (i) , 0), m −i ), tˆi ((x˜ (i) , 0), m −i )) = vio .

(46)

Public Good Provision Games on Networks with Resource Pooling

285

m ), tˆi (m m )) ≥ vio . Equations (45) and (46) together imply that ri ( Xˆ (m



Proof (Theorem 5) Let us assume X ∗ is a socially optimal effort profile. Let x = ∗ ∗ ∗ ∗ ∗ , x13 , · · · , x1N , x21 , x23 · · · , x N∗ (N −1) ]. First we show that there is vector l i such [x12 that T max −ll i x + vi (A(xx )) . (47) x ∈ arg x ∈R N (N −1) ,A(xx )∈S

As A(xx ) is the socially optimal effort profile, we have N x = arg max{xx ∈R N (N −1) ,A(xx )∈S} i=1 vi (A(xx )) → KKT Conditions:   x )) − λ + θ = 0 − i∈V x vi (A(x λ T x = 0,  θ j (−B j + k∈V −{ j} x jk ) = 0 λ≥0 ⎡

(48)

⎤T

θ = ⎣θ 1 , · · · , θ 1 , θ 2 , · · · , θ 2 , · · · , θ N , · · · , θ N ⎦ ≥ 0 .    N −1 times

N −1 times

N −1 times

We can define l i = x vi (A(xx )) + λ /N − θ /N . Then we have l i − x vi (A(xx )) − λ /N + θ /N = 0 ,

(49)

which implies that x , λ /N , θ /N satisfies the KKT conditions for the following optimization problem: T

arg max{xx ∈R N (N −1) ,A(xx )∈S} − l i x + vi (A(xx )) .

(50)

As the above optimization problem is convex and the KKT conditions are necessary and sufficient for optimality, x is the solution to (50). Now let us assume that we have already found l i , ∀i ∈ V . Consider following system of equations, (i) 1 N = i=1 x N (i+1) (i+2)

π

(xx

−π

(i)

= l i , i = 1, · · · , N

x (i+1) T

−x

(51.a)

x

π (i) )(xx ) diag(π

π (i) ≥ 0 , i = 1, · · · , N

(i)

−x

(i+1)

(51.b) ) = 0, i = 1, · · · , N (51.c)

(51)

(51.d)

First, we show that the above system of equations has at least one solution. If we set x (i) = x , then Eqs. (51.a), (51.c) are satisfied. Moreover, the summation of left-hand side and right-hand side of (51.b) is zero which implies that one of the equations of type (51.b) is redundant. Therefore, if we choose an arbitrary value for π (1) , then π (2) , π (3) , · · · , π (N ) can be determined accordingly based on (51.b).

286

M. M. Khalili et al.

Moreover, notice that if we add all π (i) by a constant vector c , then they still satisfy (51.a), (51.b), (51.c). Therefore, we can select an appropriate constant vector c to satisfy (51.d). Now, we show the solution introduced above is a Nash equilibrium of the proposed mechanism. We chose l i such that it satisfies the following: T

x (i) = x ∈ arg maxx ∈R N ·(N −1) − l i x + vi (A(xx )) .

(52)

We use the following change of variable for the above optimization problem: Nxx −  ( j) = x (i) . We have j∈V −{i} x T 1 (xx (i) N

x (i) ∈ x ∈ arg maxx (i) ∈R N ·(N −1) − l i

+vi (A( N1 (xx (i) +

+ 



j∈V −{i} x

j∈V −{i} x

( j)

( j)

)

(53)

))) .

By (51.c) and the fact that the users’ utility function is decreasing in tax, we have T 1 (xx (i) N

(xx (i) , π (i) ) ∈ arg max{xx (i) ∈R N ·(N −1) ,ππ (i) } − l i (xx

(i+1)

x (i+2) T

−x

π ) diag(π

(i+1)

+

)(xx



j∈V −{i} x

(i+1)

( j)

)

x (i+2) T

−x

)

π (i) )(xx (i) − x (i+1) )T −(xx (i) − x (i+1) )T diag(π  +vi (A( N1 (xx (i) + j∈V −{i} x ( j) )))

(54)

The last equation implies that the solution to (51) is the fixed point of the best-response mapping. Therefore, the solution to (51) is a NE of the proposed mechanism. 

References 1. Abolhassani, M., Bateni, M.H., Hajiaghayi, M., Mahini, H., Sawant, A.: Network cournot competition. In: Liu, T.Y., Qi, Q., Ye, Y. (eds.) Web and Internet Economics. pp. 15–29. Springer International Publishing, Cham (2014) 2. Acemoglu, D., Ozdaglar, A., Tahbaz-Salehi, A.: Networks, shocks, and systemic risk. Working Paper 20931, National Bureau of Economic Research (February 2015). https://doi.org/10.3386/ w20931, http://www.nber.org/papers/w20931 3. Allouch, N.: On the private provision of public goods on networks. Journal of Economic Theory 157, 527–552 (2015) 4. Bimpikis, K., Ehsani, S., Ilkiliç, R.: Cournot competition in networked markets. In: Proceedings of the Fifteenth ACM Conference on Economics and Computation. pp. 733–733. EC ’14, ACM, New York, NY, USA (2014). https://doi.org/10.1145/2600057.2602882, http://doi.acm.org/10. 1145/2600057.2602882 5. Bramoullé, Y., Kranton, R.: Games played on networks (2015) 6. Bramoullé, Y., Kranton, R., D’amours, M.: Strategic interaction and networks. American Economic Review 104(3), 898–930 (2014) 7. Cai, D., Bose, S., Wierman, A.: On the role of a market maker in networked cournot competition. arXiv preprint arXiv:1701.08896 (2017)

Public Good Provision Games on Networks with Resource Pooling

287

8. Debreu, G.: A social equilibrium existence theorem. Proceedings of the National Academy of Sciences 38(10), 886–893 (1952) 9. Grossklags, J., Radosavac, S., Cárdenas, A.A., Chuang, J.: Nudge: Intermediaries role in interdependent network security. In: International Conference on Trust and Trustworthy Computing. pp. 323–336. Springer (2010) 10. Jackson, M.O., Zenou, Y.: Games on networks. In: Handbook of game theory with economic applications, vol. 4, pp. 95–163. Elsevier (2015) 11. Miura-Ko, R.A., Yolken, B., Mitchell, J., Bambos, N.: Security decision-making among interdependent organizations. In: 2008 21st IEEE Computer Security Foundations Symposium. pp. 66–80 (June 2008). https://doi.org/10.1109/CSF.2008.25 12. Naghizadeh, P., Liu, M.: Closing the price of anarchy gap in the interdependent security game. In: 2014 Information Theory and Applications Workshop (ITA). pp. 1–8 (Feb 2014). https:// doi.org/10.1109/ITA.2014.6804216 13. Naghizadeh, P., Liu, M.: Opting out of incentive mechanisms: A study of security as a nonexcludable public good. IEEE Transactions on Information Forensics and Security 11(12), 2790–2803 (2016) 14. Naghizadeh, P., Liu, M.: On the uniqueness and stability of equilibria of network games. In: Communication, Control, and Computing (Allerton), 2017 55th Annual Allerton Conference on. pp. 280–286. IEEE (2017) 15. Naghizadeh, P., Liu, M.: Provision of public goods on networks: on existence, uniqueness, and centralities. IEEE Transactions on Network Science and Engineering (2017) 16. Parameswaran, M., Zhao, X., Whinston, A.B., Fang, F.: Reengineering the internet for better security. Computer 40(1) (2007) 17. Sharma, S., Teneketzis, D.: A game-theoretic approach to decentralized optimal power allocation for cellular networks. Telecommunication Systems 47(1), 65–80 (Jun 2011). https://doi. org/10.1007/s11235-010-9302-6, https://doi.org/10.1007/s11235-010-9302-6