Brain Storm Optimization Algorithms: Concepts, Principles and Applications [1st ed.] 978-3-030-15069-3;978-3-030-15070-9

Table of contents :
Front Matter ....Pages i-xv
Front Matter ....Pages 1-1
Brain Storm Optimization Algorithms: More Questions than Answers (Shi Cheng, Hui Lu, Xiujuan Lei, Yuhui Shi)....Pages 3-32
Front Matter ....Pages 33-33
Brain Storm Optimization for Test Task Scheduling Problem (Hui Lu, Rongrong Zhou, Shi Cheng, Yuhui Shi)....Pages 35-59
Oppositional Brain Storm Optimization for Fault Section Location in Distribution Networks (Guojiang Xiong, Jing Zhang, Dongyuan Shi, Yu He)....Pages 61-77
Multi-objective Differential-Based Brain Storm Optimization for Environmental Economic Dispatch Problem (Yali Wu, Xinrui Wang, Yingruo Xu, Yulong Fu)....Pages 79-104
Enhancing the Local Search Ability of the Brain Storm Optimization Algorithm by Covariance Matrix Adaptation (Seham Elsayed, Mohammed El-Abd, Karam Sallam)....Pages 105-122
Brain Storm Algorithm Combined with Covariance Matrix Adaptation Evolution Strategy for Optimization (Yang Yu, Lin Yang, Yirui Wang, Shangce Gao)....Pages 123-154
Front Matter ....Pages 155-155
A Feature Extraction Method Based on BSO Algorithm for Flight Data (Hui Lu, Chongchong Guan, Shi Cheng, Yuhui Shi)....Pages 157-188
Brain Storm Optimization Algorithms for Solving Equations Systems (Liviu Mafteiu-Scai, Emanuela Mafteiu, Roxana Mafteiu-Scai)....Pages 189-220
StormOptimus: A Single Objective Constrained Optimizer Based on Brainstorming Process for VLSI Circuits (Satyabrata Dash, Deepak Joshi, Sukanta Dey, Meenali Janveja, Gaurav Trivedi)....Pages 221-243
Brain Storm Optimization Algorithms for Flexible Job Shop Scheduling Problem (Xiuli Wu)....Pages 245-271
Enhancement of Voltage Stability Using FACTS Devices in Electrical Transmission System with Optimal Rescheduling of Generators by Brain Storm Optimization Algorithm (G. V. Nagesh Kumar, B. Sravana Kumar, B. Venkateswara Rao, D. Deepak Chowdary)....Pages 273-297
Back Matter ....Pages 299-299

Citation preview

Adaptation, Learning, and Optimization 23

Shi Cheng Yuhui Shi Editors

Brain Storm Optimization Algorithms Concepts, Principles and Applications

Adaptation, Learning, and Optimization Volume 23

Series Editors Yew Soon Ong, Nanyang Technological University, Singapore, Singapore Meng-Hiot Lim, Singapore, Singapore

The role of adaptation, learning and optimization are becoming increasingly essential and intertwined. The capability of a system to adapt either through modification of its physiological structure or via some revalidation process of internal mechanisms that directly dictate the response or behavior is crucial in many real world applications. Optimization lies at the heart of most machine learning approaches while learning and optimization are two primary means to effect adaptation in various forms. They usually involve computational processes incorporated within the system that trigger parametric updating and knowledge or model enhancement, giving rise to progressive improvement. This book series serves as a channel to consolidate work related to topics linked to adaptation, learning and optimization in systems and structures. Topics covered under this series include: • complex adaptive systems including evolutionary computation, memetic computing, swarm intelligence, neural networks, fuzzy systems, tabu search, simulated annealing, etc. • machine learning, data mining & mathematical programming • hybridization of techniques that span across artificial intelligence and computational intelligence for synergistic alliance of strategies for problem-solving. • aspects of adaptation in robotics • agent-based computing • autonomic/pervasive computing • dynamic optimization/learning in noisy and uncertain environment • systemic alliance of stochastic and conventional search techniques • all aspects of adaptations in man-machine systems. This book series bridges the dichotomy of modern and conventional mathematical and heuristic/meta-heuristics approaches to bring about effective adaptation, learning and optimization. It propels the maxim that the old and the new can come together and be combined synergistically to scale new heights in problem-solving. To reach such a level, numerous research issues will emerge and researchers will find the book series a convenient medium to track the progresses made. *Indexing: The books of this series are submitted to DBLP, SCOPUS, Google Scholar and Springerlink.*

More information about this series at http://www.springer.com/series/8335

Shi Cheng Yuhui Shi •

Editors

Brain Storm Optimization Algorithms Concepts, Principles and Applications

123

Editors Shi Cheng School of Computer Science Shaanxi Normal University Xi’an, China

Yuhui Shi Shenzhen Key Laboratory of Computational Intelligence, Department of Computer Science and Engineering Southern University of Science and Technology Shenzhen, China

ISSN 1867-4534 ISSN 1867-4542 (electronic) Adaptation, Learning, and Optimization ISBN 978-3-030-15069-3 ISBN 978-3-030-15070-9 (eBook) https://doi.org/10.1007/978-3-030-15070-9 © Springer Nature Switzerland AG 2019 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

The brain storm optimization (BSO) algorithm is a new kind of swarm intelligence method, which is based on the collective behavior of human being, that is, the brainstorming process. Since the introduction of BSO algorithm in 2011, many studies have been conducted from different aspects. It not only is an optimization method but also could be viewed as a framework of optimization technique. The process of BSO algorithm could be simplified as a framework with two basic operations: the converging operation and the diverging operation. A “good enough” optimum could be obtained through recursive solution divergence and convergence in the search space. The designed optimization algorithm will naturally have the capability of both convergence and divergence. This book aims to present a collection of recent advances in brain storm optimization algorithms. Based on a peer review process, eleven chapters were accepted to be included in the book, covering various topics ranging from variants of BSO algorithms, theoretical analysis, and applications. The book is structured into three parts. In the first part, i.e., Chapter “Brain Storm Optimization Algorithms: More Questions than Answers”, a comprehensive introduction to the basic concepts, developments, and future research of BSO algorithms are described. Many works have been conducted on the BSO algorithms; however, there are still massive questions on this algorithm need to be studied. The second part, i.e., the methodology part, containing Chapters “Brain Storm Optimization for Test Task Scheduling Problem”–“Brain Storm Algorithm Combined with Covariance Matrix Adaptation Evolution Strategy for Optimization”, introduces several variants of BSO algorithms. Chapter “Brain Storm Optimization for Test Task Scheduling Problem” introduces a modified BSO algorithm for solving the single-objective and the multi-objective test task scheduling problem (TTSP), respectively. Chapter “Oppositional Brain Storm Optimization for Fault Section Location in Distribution Networks” describes an oppositional brain storm optimization (OBSO) algorithm on solving the fault section location (FSL) problem. Chapter “Multi-objective Differential-Based Brain Storm Optimization for Environmental Economic Dispatch Problem” defines a multi-objective differential brain storm optimization (MDBSO) algorithm to solve v

vi

Preface

environmental–economic dispatch (EED) problem. Chapter “Enhancing the Local Search Ability of the Brain Storm Optimization Algorithm by Covariance Matrix Adaptation” designs a hybrid algorithm that combines the exploration ability of the global-best BSO (GBSO) algorithm and the local ability of the covariance matrix adaptive evolutionary strategy (CMA-ES) algorithm. Chapter “Brain Storm Algorithm Combined with Covariance Matrix Adaptation Evolution Strategy for Optimization” proposes a hybrid algorithm of the BSO algorithm and the covariance matrix adaptive evolutionary strategy (CMA-ES) algorithm to solve optimization problems. Finally, the third part, i.e., the applications part, contains Chapters “A Feature Extraction Method Based on BSO Algorithm for Flight Data”–“Enhancement of Voltage Stability Using FACTS Devices in Electrical Transmission System with Optimal Rescheduling of Generators by Brain Storm Optimization Algorithm,” which gives a set of examples of utilizing BSO algorithms on solving real-world optimization problems. With applications in complex engineering or design problems, the strength and limitation of various brain storm optimization algorithms could be revealed and interpreted. Chapter “A Feature Extraction Method Based on BSO Algorithm for Flight Data” introduces BSO algorithms on solving the feature extraction problem for flight data. Chapter “Brain Storm Optimization Algorithms for Solving Equations Systems” describes BSO algorithms in solving equation systems (ES). Chapter “StormOptimus: A Single Objective Constrained Optimizer Based on Brainstorming Process for VLSI Circuits” presents the main aspects and implications of design optimization of electronic circuits using the StormOptimus algorithm based on the brainstorming process. Chapter “Brain Storm Optimization Algorithms for Flexible Job Shop Scheduling Problem” describes the BSO algorithms for solving flexible job shop scheduling problems (FJSP). Chapter “Enhancement of Voltage Stability Using FACTS Devices in Electrical Transmission System with Optimal Rescheduling of Generators by Brain Storm Optimization Algorithm” introduces the BSO-algorithm-based voltage stability maintenance by using flexible alternating current transmission system (FACTS) devices. This book volume is primarily intended for researchers, engineers, and graduate students with interests in BSO algorithms and their applications. The chapters cover different aspects of BSO algorithms, and as a whole should provide broad perceptive insights on what BSO algorithms have to offer. It is very suitable as a graduate-level textbook whereby students may be tasked with the study of the rich variants of BSO algorithms coupled with a project-type assignment that involves a hands-on implementation to demonstrate the utility and applicability of BSO algorithms in solving optimization problems. We wish to express our sincere gratitude to all the authors of this book who has put in so much effort to prepare their manuscripts, sharing their research findings as a chapter in this volume. Based on a compilation of chapters in this book, it is clear that the authors have done a stupendous job, each chapter directly or indirectly hitting on the spot certain pertinent aspect of swarm intelligence. As editors of this volume, we have been very fortunate to have a team of editorial board members

Preface

vii

who besides offering useful suggestions and comments, helped to review manuscripts submitted for consideration. Finally, we would like to put in no definitive order our acknowledgments of the various parties who played a part in the successful production of this volume: • Authors who contributed insightful and technically engaging chapters in this volume; • The panel of experts in brain storm optimization algorithms that form editorial board members. • The book series editor, Prof. Meng-Hiot Lim, who has made great efforts and support on this book. • The publisher, who has been very helpful and accommodating, making our tasks a little easier. Last but not least, we thank our families and friends for the constant support during the production of this volume. This work was supported in part by the National Natural Science Foundation of China under grants 61806119, 61761136008, 61773119, 61703256, and 61771297; in part by the Shenzhen Science and Technology Innovation Committee under grant number ZDSYS201703031748284, and in part by the Fundamental Research Funds for the Central Universities under grant number GK201901010. Xi’an, China Shenzhen, China December 2018

Shi Cheng Yuhui Shi

Contents

Part I

Foundations

Brain Storm Optimization Algorithms: More Questions than Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shi Cheng, Hui Lu, Xiujuan Lei and Yuhui Shi Part II

3

Methodology

Brain Storm Optimization for Test Task Scheduling Problem . . . . . . . . Hui Lu, Rongrong Zhou, Shi Cheng and Yuhui Shi

35

Oppositional Brain Storm Optimization for Fault Section Location in Distribution Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Guojiang Xiong, Jing Zhang, Dongyuan Shi and Yu He

61

Multi-objective Differential-Based Brain Storm Optimization for Environmental Economic Dispatch Problem . . . . . . . . . . . . . . . . . . . Yali Wu, Xinrui Wang, Yingruo Xu and Yulong Fu

79

Enhancing the Local Search Ability of the Brain Storm Optimization Algorithm by Covariance Matrix Adaptation . . . . . . . . . . 105 Seham Elsayed, Mohammed El-Abd and Karam Sallam Brain Storm Algorithm Combined with Covariance Matrix Adaptation Evolution Strategy for Optimization . . . . . . . . . . . . . . . . . . 123 Yang Yu, Lin Yang, Yirui Wang and Shangce Gao Part III

Applications

A Feature Extraction Method Based on BSO Algorithm for Flight Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 Hui Lu, Chongchong Guan, Shi Cheng and Yuhui Shi

ix

x

Contents

Brain Storm Optimization Algorithms for Solving Equations Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 Liviu Mafteiu-Scai, Emanuela Mafteiu and Roxana Mafteiu-Scai StormOptimus: A Single Objective Constrained Optimizer Based on Brainstorming Process for VLSI Circuits . . . . . . . . . . . . . . . . . . . . . 221 Satyabrata Dash, Deepak Joshi, Sukanta Dey, Meenali Janveja and Gaurav Trivedi Brain Storm Optimization Algorithms for Flexible Job Shop Scheduling Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 Xiuli Wu Enhancement of Voltage Stability Using FACTS Devices in Electrical Transmission System with Optimal Rescheduling of Generators by Brain Storm Optimization Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 273 G. V. Nagesh Kumar, B. Sravana Kumar, B. Venkateswara Rao and D. Deepak Chowdary Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299

Contributors

Shi Cheng School of Computer Science, Shaanxi Normal University, Xi’an, China D. Deepak Chowdary Dr. L. Bullayya College of Engineering (for Women), Visakhapatnam, India Satyabrata Dash Indian Institute of Technology Guwahati, Guwahati, Assam, India Sukanta Dey Indian Institute of Technology Guwahati, Guwahati, Assam, India Mohammed El-Abd American University of Kuwait, Salmiya, Kuwait Seham Elsayed Faculty of Computer and Informatics, Zagazig University, Zagazig, Egypt Yulong Fu Shaanxi Key Laboratory of Complex System Control and Intelligent Information Processing, Xi’an, China; Faculty of Automation and Information Engineering, Xi’an University of Technology, Xi’an, China Shangce Gao Faculty of Engineering, University of Toyama, Toyama-shi, Japan Chongchong Guan School of Electronic and Information Engineering, Beihang University, Beijing, China Yu He Guizhou Key Laboratory of Intelligent Technology in Power System, College of Electrical Engineering, Guizhou University, Guiyang, China Meenali Janveja Indian Institute of Technology Guwahati, Guwahati, Assam, India Deepak Joshi Indian Institute of Technology Guwahati, Guwahati, Assam, India

xi

xii

Contributors

B. Sravana Kumar GITAM University, Visakhapatnam, India G. V. Nagesh Kumar JNTUA College of Engineering, Pulivendula, India Xiujuan Lei School of Computer Science, Shaanxi Normal University, Xi’an, China Hui Lu School of Electronic and Information Engineering, Beihang University, Beijing, China Liviu Mafteiu-Scai Computer Science Department, West University of Timisoara, Timisoara, Romania Roxana Mafteiu-Scai Computer Science Department, West University of Timisoara, Timisoara, Romania Emanuela Mafteiu Computer Science Department, West University of Timisoara, Timisoara, Romania B. Venkateswara Rao V R Siddhartha Engineering College (Autonomous), Vijayawada, India Karam Sallam UNSW, Canberra, Australia; Zagazig University, Zagazig, Egypt Dongyuan Shi State Key Laboratory of Advanced Electromagnetic Engineering and Technology, Huazhong University of Science and Technology, Wuhan, China Yuhui Shi Shenzhen Key Lab of Computational Intelligence, Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China Gaurav Trivedi Indian Institute of Technology Guwahati, Guwahati, Assam, India Xinrui Wang Shaanxi Key Laboratory of Complex System Control and Intelligent Information Processing, Xi’an, China; Faculty of Automation and Information Engineering, Xi’an University of Technology, Xi’an, China Yirui Wang Faculty of Engineering, University of Toyama, Toyama-shi, Japan Xiuli Wu Department of Logistic Engineering, School of Mechanical Engineering, University of Science and Technology Beijing, Beijing, China Yali Wu Shaanxi Key Laboratory of Complex System Control and Intelligent Information Processing, Xi’an, China; Faculty of Automation and Information Engineering, Xi’an University of Technology, Xi’an, China

Contributors

xiii

Guojiang Xiong Guizhou Key Laboratory of Intelligent Technology in Power System, College of Electrical Engineering, Guizhou University, Guiyang, China Yingruo Xu Shaanxi Key Laboratory of Complex System Control and Intelligent Information Processing, Xi’an, China; Faculty of Automation and Information Engineering, Xi’an University of Technology, Xi’an, China Lin Yang Faculty of Engineering, University of Toyama, Toyama-shi, Japan Yang Yu Faculty of Engineering, University of Toyama, Toyama-shi, Japan Jing Zhang Guizhou Key Laboratory of Intelligent Technology in Power System, College of Electrical Engineering, Guizhou University, Guiyang, China Rongrong Zhou School of Electronic and Information Engineering, Beihang University, Beijing, China

Acronyms

Lists of abbreviations are as follows: ABC ACO BSO BSO-OS CI CMA-ES EA EC EMO EP ES GA GBSO KNN MOO OBSO PSO SI TTSP

Artificial bee colony Ant colony optimization Brain storm optimization Brain storm optimization in objective space Computational intelligence Covariance matrix adaptive evolutionary strategy Evolutionary algorithm Evolutionary computation Evolutionary multi-objective optimization Evolutionary programming Evolution strategies Genetic algorithm Global-best brain storm optimization k Nearest neighbor Multi-objective optimization Oppositional brain storm optimization Particle swarm optimizer/optimization Swarm intelligence Test task scheduling problem

xv

Part I

Foundations

Brain Storm Optimization Algorithms: More Questions than Answers Shi Cheng, Hui Lu, Xiujuan Lei and Yuhui Shi

Abstract Brain storm optimization (BSO) algorithms is a framework that indicates algorithms using converging operation and diverging operation to locate the optima of optimization problems. Hundreds of articles on the BSO algorithms have been published in different journals and conference proceedings, even though there are more questions than answers. In this chapter, BSO algorithms are comprehensively surveyed and the future research directions are discussed from the perspective of model-driven and data-driven approaches. For swarm intelligence algorithms, each individual in the swarm represents a solution in the search space, and it also can be seen as a data sample from the search space. Based on the analyses of these data, more effective algorithms and search strategies could be proposed. Through the convergent operation and divergent operation, individuals in BSO are grouped and diverged in the search space/objective space. Every individual in the BSO algorithm is not only a solution to the problem to be optimized, but also a data point to reveal the landscape of the problem. Many works have been conducted on the BSO algorithms, there are still massive questions on this algorithm need to be answered. These questions, which include the properties of algorithms, the connection between optimization algorithms and problems, and the real-world application should be studied for the BSO algorithms or, more broadly, for the swarm intelligence algorithms.

S. Cheng (B) · X. Lei School of Computer Science, Shaanxi Normal University, Xi’an 710119, China e-mail: [email protected] X. Lei e-mail: [email protected] H. Lu School of Electronic and Information Engineering, Beihang University, Beijing 100191, China e-mail: [email protected] Y. Shi Shenzhen Key Lab of Computational Intelligence, Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen 518055, China e-mail: [email protected] © Springer Nature Switzerland AG 2019 S. Cheng and Y. Shi (eds.), Brain Storm Optimization Algorithms, Adaptation, Learning, and Optimization 23, https://doi.org/10.1007/978-3-030-15070-9_1

3

4

S. Cheng et al.

Keywords Brain storm optimization · Data-driven based algorithm · Developmental brain storm algorithm · Developmental swarm intelligence

1 Introduction Optimization, in general, is concerned with finding “best available” solution(s) for a given problem. An optimization problem in Rn , or simply an optimization problem, is a mapping f : Rn → Rm , where Rn is termed as decision space [1] (or parameter space [39], problem space), and Rm is termed as objective space [72]. Each solution in decision space is associated with a fitness value or a vector of fitness values in objective space. Without loss of generality, a minimum optimization problem is illustrated as follows: min f(x) = [f1 (x), f2 (x), . . . , fm (x)] subject to gi (x) ≤ 0, i = 1, 2, . . . , k hj (x) = 0, j = 1, 2, . . . , p.

(1)

where f(·) is the objective function, x = (x1 , x2 , . . . , xn ) is the variable vector to be optimized, gi (·) and hj (·) are the constraints. The evaluation function in optimization, f (x), maps decision variables to objective vectors. Optimization problems can be divided into two categories according to the value of m. When m = 1, this kind of problem is called single objective problem, when m = 2 or m = 3, this is called multi-objective problem, and when m ≥ 4, this is called many objective optimization problem [2, 22, 56]. For single objective optimization problems, it can be simply divided into unimodal problem and multimodal problem. As the name indicated, unimodal problem has only one optimum solution, on the contrary, multi-modal functions have several or numerous optimum solutions, of which many are local optimal solutions. It is difficult for optimization algorithms to find the global optimum solution. Figure 1 give an example of optimization problem. The aim of solving this problem is to find the minimum of f (x), where f (x) = x sin 1x , x ∈ [0, 0.4]. There are three local optima in this problem, and only one global optimum. The search is performed on the x-axis (solution space), while the evaluation is based on the f (x) axis (objective space). Thus, to obtain the best fitness value on the f (x) axis, an algorithm should have an ability to find the optimum, i.e., the best solution in the x axis. Multiple conflicted objectives need to be optimized simultaneously in multiobjective optimization. An example of multiobjective problem is represented in Fig. 2. The illustrated example has three decision variables and two objectives, i.e., n = 3 and m = 2. Many real-world problems could be modeled as optimization problems and solved by using optimization algorithms. These problems often have several conflicting objectives to be optimized. For example, different factors, which include economic benefits, production efficiency, energy consumption, and environmental protection,

Fig. 1 An example of optimization problem: min f (x) = x sin 1x , x ∈ (0, 0.4]

5

f (x)

Brain Storm Optimization Algorithms: More Questions than Answers

x sin x1

0

0.1

0.2

0.3

x

f (x)

(a) Problem x sin x1

0

0.1

0.2

0.3

x

(b) Optimization

need to be optimized simultaneously in steel production industry [76, 77]. Having many factors makes an optimization problem difficult to solve. For example, several characteristics of difficult problems are as follows: • The non-linear, non-quadratic, or non-convex functions are much harder to solve than linear or quadratic functions; • The functions have the non-smooth, discontinuous, multimodal, and/or noisy environment properties; • The dimensionality of optimization problems is extremely large, for example, the dimensionality of problems are large than 1000 for large scale optimization. In addition, the non-separability dependencies or partially non-separability dependencies between the variables for problems with huge number of dimensions; • Several conflicted objectives are needed to be optimized for optimization problems to be solved simultaneously. Evolutionary optimization (EC) algorithms and swarm intelligence (SI) algorithms are generally difficult to find the global optimum solutions for multimodal problems due to the possible occurrence of the premature convergence [21, 31, 51].

6

S. Cheng et al.

(a)

x2

(b)

Ω = {x ∈ Rn }

f2

x1

Λ = {y ∈ Rm }

x3

Solution Space

Objective Space

f1

Fig. 2 The mapping from solution space to objective space

Premature converge means that the population converged before the satisfying solution is reached. Avoiding premature converge is important in multimodal problem optimization, i.e., an algorithm should have a balance between fast converge speed and the ability of “jumping out” of local optima. Swarm intelligence, which is based on a population of individuals, is a collection of nature-inspired searching techniques [44]. In swarm intelligence, an algorithm maintains and successively improves a collection of potential solutions until some stopping condition is met. The solutions are initialized randomly in the search space, and are guided toward the better and better areas through the interaction among solutions. In evolutionary optimization/swarm intelligence algorithms, individuals, which are the basic building blocks of population or groups, have different structures to form various functions. Individuals are competed and cooperated with others during the search process. The position of each individual, i.e., the numerical value of solution is updated iteration over iteration. Mathematically, the updating process of population of individuals over iterations can be looked as a mapping process from one population of individuals to another population of individuals from one iteration to the next iteration, which can be represented as Pt+1 = f (Pt ), where Pt is the population of individuals at the iteration t, f (·) is the mapping function. As a general principle, the expected fitness of a solution returned should improve as the search method is given more computational resources in time and/or space. More desirable, in any single run, the quality of the solution returned by the method should improve monotonically—that is, the quality of the solution at time t + 1 should be no worse than the quality at time t, i.e., fitness(t + 1) ≤ fitness(t) for minimum problems [28]. In swarm intelligence algorithms, there are several solutions exist at the same time. The premature convergence may happen due to the solution getting clustered together too fast. However, the solution clustering is not always harmful for optimization. In a brain storm optimization algorithm, the clustering information is utilized to reveal the landscapes of problems and to guide the individuals to move toward the better and better areas [62, 63]. Every individual in the brain storm optimization algorithm

Brain Storm Optimization Algorithms: More Questions than Answers

7

is not only a solution to the problem to be optimized, but also a data point to reveal the landscapes of the problem. The swarm intelligence and data analysis techniques can be combined to produce benefits above and beyond what either method could achieve alone [10, 14, 18]. There are two approaches, the model-driven and the data-driven, in solving an optimization problem [33, 69]. A brief comparison is as follows. • Model-Driven approach: formulate the problem as a parametric model and the objective is to search for the optimal parameters that best fit the evidence and prior. • Data-Driven approach: map the features extracted from the solved problem to the landscape by learning on the data set. The model-driven approach are used in most traditional swarm intelligence algorithms. Two or three basic equations are contained in these algorithms. The aim is to find the optima more effective and efficiency by tuning the parameters in equations. The data-driven approach could be used to learn the landscape of the problem being solved. The model-driven approach and data-driven approach are combined into the designing of brain storm optimization (BSO) algorithm. The solutions found are analyzed as data points in the search or the objective space. The information learned could be used to update the search model, i.e., the equations in the BSO algorithms. The aim of this chapter is to provide a comprehensive review of the BSO algorithms from the perspective of model driven and data-driven approaches. Compared with other swarm intelligence algorithms, BSO algorithms have two properties: • The data mining techniques are embedded into the algorithm design. All solutions are grouped (clustered or classified) during each iteration. Based on this properties, algorithm could has an ability to learn the landscape of the problem being solved. • The convergence speed is not the only criterion in algorithm design. Based on grouping strategy, the solutions are separated on different local optima. Thus, the BSO algorithm is focused on the multimodal attribute of problems being solved. The remaining of the chapter is organized as follows. The basic concepts of evolutionary computation and swarm intelligence are reviewed in Sect. 2. The developmental history of brain storm optimization algorithm is described in Sect. 3. In Sect. 4, different variants of BSO algorithms are introduced. The applications on solving various problems are described in Sect. 5. Finally, Sects. 6 and 7 concludes with future research directions and some remarks, respectively.

2 Nature-Inspired Intelligent Algorithms The evolutionary computation (EC) and swarm intelligence algorithm are two kind of nature-inspired intelligent algorithms. There are several common features in two kinds of algorithms: • Both EC and SI algorithms are a kind of stochastic algorithms. There is no guarantee to find the global optima in limited iterations.

8

S. Cheng et al.

• There are many iterations or generations in two kinds of algorithms. In general, the term ‘generation’ is used in EC algorithms while the term ‘iteration’ is often used in SI algorithms. • Two kinds of algorithms are population-based algorithms, i.e., many solutions competed and cooperated with others in the search. • Two kinds of algorithms belong to black-box optimization techniques. Normally, the information of problems is not required during in the search. The differences of two kinds of algorithms are as follows: • EC algorithms are inspired by the biological phenomenon, especially the nature selection process. Thus, the metaphors from evolutionary theory, such as ‘generation’, ‘parent’, and ‘child’ are often used in the algorithm description. • SI algorithm mimics the social behavior of individuals in a biological or physical population. The metaphors from biology or physics, such as ‘ant’ (ant colony optimization), ‘particle’ (particle swarm optimization), ‘idea’ (brain storm optimization) are used in algorithm description.

2.1 Evolutionary Computation There are a lot of evolutionary algorithms out there in the literature. The most popular evolutionary algorithms, which were inspired by biological evolution, are as follows: • • • •

genetic algorithm [36, 52]; genetic programming [46]; evolutionary strategy [4]; evolutionary programming [29, 30].

In evolutionary algorithms, population of individuals survives into the next generation. Which individual has higher probability to survive is proportional to its fitness value according to some evaluation function. The survived individuals are then updated by utilizing evolutionary operators such as crossover operator and mutation operator, etc. In evolutionary programming and evolution strategy, only the mutation operation is employed, while in genetic algorithms and genetic programming, both the mutation operation and crossover operation are employed. The optimization problems to be optimized by evolutionary algorithms do not need to be mathematically represented as continuous and differentiable functions; they can be represented in any form. Only requirement for representing optimization problems is that each individual can be evaluated as a value called fitness value. Therefore, evolutionary algorithms can be applied to solve more general optimization problems, especially those that are very difficult, if not impossible, for traditional hill-climbing algorithms to solve. Taking genetic algorithm as an example, there are three operators, mutation operator, crossover operator, and selection operator, in the genetic algorithm [36]. Figure 3 shows the basic process of genetic algorithm.

Brain Storm Optimization Algorithms: More Questions than Answers

9

Fig. 3 The basic process of genetic algorithm

Start Initialization Mutation

Evaluation

Crossover

Selection

No

End? Yes Stop

2.2 Swarm Intelligence The swarm intelligence algorithms are inspired by the interaction among individuals in a group. The interactive behaviors of individuals in a group, such as a flock of birds or a cluster of ants, are modeled as an optimization process. The most used swarm intelligence algorithms are as follows: • particle swarm optimization (PSO) algorithm [26, 43, 67]; • ant colony optimization (ACO) algorithm [24]. Recently, many swarm intelligence algorithms, which based on different phenomena, have been proposed. The following list gives several examples: • fireworks algorithm (FWA) [73–75]; • brain storm optimization (BSO) algorithm [62, 63]; • pigeon-inspired optimization (PIO) algorithm [25]. Taking particle swarm optimization algorithm as an example to illustrate the swarm intelligence algorithm, the velocity and position update equations in canonical PSO algorithm are as follow [27, 44]: vi ← wvi + c1 rand1 ()(pi − xi ) + c2 rand2 ()(pn − xi ) xi ← xi + vi

(2) (3)

where w denotes the inertia weight [67], c1 and c2 are two positive acceleration constants, rand 1 () and rand2 () are two random functions to generate uniformly distributed random numbers in the range [0, 1) and are different for each dimension and each particle, xi represents the ith particle’s position, vi represents the ith particle’s velocity, pi is termed as personal best, which refers to the best position found by the

10

S. Cheng et al.

Fig. 4 The flowchart of particle swarm optimization algorithm

Start Initialization Update Position

Evaluation

Update Velocity

Update pbest, nbest

No

End? Yes Stop

ith particle, and pg is termed as local best, which refers to the position found by the members in the ith particle’s neighborhood that has the best fitness evaluation value so far. Random variables are frequently utilized in swarm optimization algorithms. The length of search step is not determined in the optimization. This approach belongs to an interesting class of algorithms that are known as stochastic algorithms. A randomized algorithm does not guarantee an exact result but instead provides a high probability guarantee that it will return the correct answer or one close to it. The result(s) of optimization may be different in each run, but the algorithm has a high probability to find a “good enough” solution(s). The flowchart of particle swarm optimization algorithm is shown in Fig. 4 and the procedure of particle swarm optimization algorithm is given in Algorithm 1.

Algorithm 1: The procedure of particle swarm optimization algorithm 1 2 3 4

5 6 7 8

Initialization: Initialize velocity and position randomly for each particle in every dimension; while not find the “good” solution or not reach the maximum iteration do Calculate each particle’s fitness value; Compare fitness value between current value and best position in history (personal best, termed as pbest). For each particle, if fitness value of current position is better than pbest, then update pbest as current position; Selection a particle which has the best fitness value from current particle’s neighborhood, this particle is called the neighborhood best (termed as nbest); for each particle do Update particle’s velocity according equation (2); Update particle’s position according equation (3);

Brain Storm Optimization Algorithms: More Questions than Answers

11

Generate A Group Of Ideas (Creative) Create Possible Solution (Creative)

Clarify & Condense Ideas (Creative)

Evaluate & Edit Ideas (Critical) Fig. 5 The steps of brainstorming process

3 Brain Storm Optimization 3.1 Brainstorming Process Model The brain storm optimization (BSO) algorithm was proposed in 2011 [62, 63], which mimics the brainstorm processing in problem solving. Figure 5 shows the steps of brainstorming process.1

3.2 Brain Storm Optimization Algorithms Brain storm optimization (BSO) algorithm is a new and promising swarm intelligence algorithm, which simulates the human brainstorming process. Since the introduction of BSO algorithm in 2011, many studies have been conducted from different aspects, such as the theoretical analysis of the BSO algorithm, the improvement of search efficiency and its applications for real-world problems [17]. It is simple in concept and easy in implementation for the original BSO algorithm [62, 63]. Figure 6 shows the framework of brain storm optimization algorithm. The procedure of BSO algorithm is given in Algorithm 2 There are three strategies in this algorithm: the solution clustering/classification, new individual generation, and selection [15]. The solution clustering and solution classification strategies are used in the original BSO algorithm and BSO in objective space algorithm, respectively [65]. BSO, a new swarm intelligence algorithm, is inspired by collective behavior of human beings. It has attracted many researchers and has good performance in its applications for complex problems. The comprehensive review about BSO can be refer to [13] and [17]. 1 If the current particle’s neighborhood includes all particles then this neighborhood best is the global

best (termed as gbest), otherwise, it is the local best (termed as lbest).

12

S. Cheng et al.

Fig. 6 The process of brain storm optimization algorithms

Start Initialization Solution Evaluation Solution Clustering/Classification Solution Generation

No

Solution Selection

End? Yes Stop

In a brain storm optimization algorithm, the solutions are separated into several clusters. The best solution of the population will be kept if the new generated solution at the same index is not better. New individual can be generated based on one or two individuals in clusters. The exploitation ability is enhanced when the new individual is close to the best solution found so far. While the exploration ability is enhanced when the new individual is randomly generated, or generated by individuals in two clusters. Algorithm 2: The basic procedure of the brain storm optimization algorithms 1 2 3 4 5

6

Initialization: Randomly generate n individuals (potential solutions), and evaluate the n individuals; while not find “good enough” solution or not reach the pre-determined maximum number of iterations do Solution clustering/classification operation: Diverge n individuals into m groups by a clustering/classification algorithm; New solution generation operation: Select solution(s) from one or two group(s) randomly to generate new individual (solution); Solution selection operation: Compare the newly generated individual (solution) and the existing individual (solution) with the same individual index; the better one is kept and recorded as the new individual; Evaluate the n individuals (solutions);

Brain Storm Optimization Algorithms: More Questions than Answers

3.2.1

13

Solution Clustering

The aim of solution clustering is to converge the solutions into small regions. Different clustering algorithms can be utilized in the brain storm optimization algorithm. In the original BSO algorithm, the basic k-means clustering algorithm is utilized. The clustering strategy has been replaced by other convergence methods, such as simple grouping method (SGM) [86], affinity propagation clustering [7]. Clustering is the process of grouping similar objects together. From the perspective of machine learning, the clustering analysis is sometimes termed as unsupervised learning. There are N points in the given input, D = {xi }Ni=1 , the useful and functional patterns can be obtained through the similarity calculation among points. Every solution in the brain storm optimization algorithm spreads in the search space. The distribution of solutions can be utilized to reveal the landscapes of a problem. The procedure of solution clustering is given in Algorithm 3. The clustering strategy divides individuals into several clusters. This strategy could refine a search area. After many iterations, all solutions may be clustered into a small region. A probability value pclustering is utilized to control the probability of replacing a cluster center by a randomly generated solution. This could avoid the premature convergence, and help individuals “jump out” of the local optima.

Algorithm 3: The solution clustering strategy 1 2 3 4 5 6

Clustering: Cluster n individuals into m clusters by k-means clustering algorithm; Rank individuals in each cluster and record the best individual as its cluster center in each cluster; Randomly generate a value rclustering in the range [0, 1); if the value rclustering is smaller than a pre-determined probability pclustering then Randomly select a cluster center; Randomly generate an individual to replace the selected cluster center;

3.2.2

New Individual Generation

The procedure of new individual generation is given in Algorithm 4. A new individual can be generated based on one or several individuals or clusters. In the original brain storm optimization algorithm, a probability value pgeneration is utilized to determine a new individual being generated by one or two “old” individuals. Generating an individual from one cluster could refine a search region, and it enhances the exploitation ability. On the contrast, an individual, which is generated from two or more clusters, may be far from these clusters. The exploration ability is enhanced under this scenario. The probability poneCluster and probability ptwoCluster are utilized to determine the cluster center or another normal (or non-cluster center) individual will be chosen in

14

S. Cheng et al.

one cluster or two clusters generation case, respectively. In one cluster generation case, the new individual from center or normal individual can control the exploitation region. While in several clusters generation case, the normal individuals could increase the population diversity of swarm. The new individuals are generated according to the functions (4) and (5). i i = xold + ξ(t) × N (μ, σ 2 ) xnew   0.5 × T − t × rand() ξ(t) = logsig k

(4) (5)

where the value xold is a copy of one individual or the combination of two individuals, i i and xold are the ith dimension of the value xnew is a new generated individual, xnew xnew and xold , respectively; and rand() is a random function to generate uniformly distributed random numbers in the range [0, 1). The N (μ, σ 2 ) is a random value that generated with a Gaussian distribution. The parameter T is the maximum number of iterations, t is the current iteration number, k is a coefficient to change logsig() function’s slope of the step size function ξ(t), which can be utilized to balance the convergence speed of the algorithm. A modified step size and individual generation was proposed in [88].

Algorithm 4: The new individual generation strategy 1 2 3 4 5 6 7 8 9

New individual generation: randomly select one or two cluster(s) to generate new individual; Randomly generate a value rgeneration in the range [0, 1); if the value rgeneration is less than a probability pgeneration then Randomly select a cluster, and generate a random value roneCluster in the range [0, 1); if the value roneCluster is smaller than a pre-determined probability poneCluster ; then Select the cluster center and add random values to it to generate new individual; else Randomly select a normal individual from this cluster and add random value to the individual to generate new individual;

else randomly select two clusters to generate new individual; Generate a random value rtwoCluster in the range [0, 1); if the value rtwoCluster is less than a pre-determined probability ptwoCluster then the two cluster centers are combined and then added with random values to generate new individual; 15 else 16 two normal individuals from each selected cluster are randomly selected to be combined and added with random values to generate new individual;

10 11 12 13 14

17

The newly generated individual is compared with the existing individual with the same individual index; the better one is kept and recorded as the new individual;

Brain Storm Optimization Algorithms: More Questions than Answers

3.2.3

15

Selection

Selection strategy plays an important role in an optimization algorithm. The aim of selection strategy is to keep good or more representative solutions in all individuals. The better solution is kept by the selection strategy after each new individual generation, while clustering strategy and generation strategy add new solutions into the swarm to keep the diversity for the whole population. Different selection strategies in genetic algorithms, such as ranking selection, tournament selection, and other selection schemes, are analyzed and compared in [32]. The selection strategy has determined the lifecycle of individuals. Individuals can only do one of the three things in selection strategy: be chosen, be kept, and be replaced. In other words, individuals are born, live, and die in the optimization process by the selection strategy [32]. A general equation of the individuals’ lifecycle is as follows: (6) Pt+1 = Pt + Pt,b − Pt,d , where P is the population, t is the index of iteration, b means individuals being born, and d means dying individuals. Different selection strategies have different selection pressure. Currently, there are only few research on the selection strategy in the BSO algorithm. To obtain an effective BSO algorithms, more selection strategies should be discussed for BSO algorithms.

4 Variants of BSO Algorithms 4.1 Theory Analysis The brain storm optimization algorithm is not only an optimization method but also could be viewed as a framework of optimization technique. The process of BSO algorithm could be simplified as a framework with two basic operations: the converging operation and the diverging operation. Figure 7 shown the two basic operations in BSO algorithms. To analyze the functions of different components, some research has been conducted on the theoretical analysis of BSO algorithms, such as population interaction with a power law distribution [82], search strategies investigation [81], solution clustering strategy [15], and population diversity [16], etc. In the original BSO, the k-means clustering algorithm was utilized to implement the convergent operation in which the population of individuals are clustered into several clusters with the best idea in each cluster being used to represent one better idea picked up [65]. In addition, the original BSO is based on the decision space. However, the real word problems are complex and the relationship between the decision space and objective space is not clear. The clustering operation in the original BSO is time consuming because of the distance calculation.

16

S. Cheng et al.

Fig. 7 The two basic operations in brain storm optimization algorithms

Start Initialization Converging Operation Diverging Operation

No

End? Yes Stop

Some variants of BSO algorithms have been proposed to improve the search ability of the original BSO algorithm. The solutions get clustered after a few iterations and the population diversity decreases quickly during the search, which are common phenomena in swarm intelligence. A definition of population diversity in BSO algorithm is introduced to measure the change of solutions’ distribution. According to the analysis, different kinds of partial re-initialization strategies are utilized to improve the population diversity in BSO algorithm [16]. Under similar consideration, chaotic predator-prey brain storm optimization was proposed to improve its ability for continuous optimization problems. A chaotic operation is further added to increase the diversity of population [57]. BSO performs better often due to its communication mechanisms. Different types of topology structures are incorporated into brain storm optimization algorithm to utilize different information exchange mechanisms [49]. Another challenging task is multimodal optimization, which needs to find multiple global and local optima at the same time. To obtain potential multiple global and local optima, a self-adaptive brain storm optimization (SBSO) algorithm has been proposed. A max-fitness grouping cluster method is used to divide the ideas into different sub-groups and these sub-groups can help to find the different optima [34]. The BSO in objective space algorithm could be used as an illustration to show the modification of BSO algorithms [65]. In the original BSO algorithm, a lot of computational resources are spent on the clustering strategy at each iteration. To reduce the computational burden, the brain storm optimization in objective space (BSO-OS) algorithm was proposed, and the clustering strategy was replaced by a simple elitist strategy based on the fitness values [65]. The procedure of the BSO in objective space algorithm is given in Algorithm 5.

Brain Storm Optimization Algorithms: More Questions than Answers

17

Algorithm 5: The basic procedure of the BSO in objective space algorithm 1 2 3 4 5 6

7

Initialization: Generate n individuals (potential solutions) randomly, and evaluate them; while have not found “good enough” solution or not reached the pre-determined maximum number of iterations do Elitist strategy: Classify all solutions into two categories: the solutions with better fitness values as elitists and the others as normals; New individual generation operation: randomly select one or two individuals from elitists or normal to generate new individual; Solution disruption operation: re-initialize one dimension of a randomly selected individual and update its fitness value accordingly; Selection operation: The newly generated individual is compared with the existing individual with the same individual index; the better one is kept and recorded as the new individual; Evaluate all individuals;

4.2 New Individual Generation Operation The new individual generation strategy is the major difference between the original BSO and the BSO-OS algorithm. Individuals are clustered into several groups in the original BSO algorithm. While for the BSO-OS algorithm, individuals are classified into two categories, the elitists and the normals, according to their fitness values. The procedure of new individual generation strategy is given in Algorithm 6. Two parameters, probability pelitist and probability pone , are used in this strategy. Algorithm 6: The basic procedure of New individual generation Operation New individual generation: Select one or two individual(s) randomly to generate new individual; 2 if random value rand < a probability pelitist then /* generate a new individual based on elitists */ 3 if random value rand < a pre-determined probability pone then 4 generate a new individual based on one randomly selected elitist; 1

5 6 7 8 9 10 11 12

else two individuals from elitists are randomly selected to generate new individual; else /* generate a new individual based on normal */ if random value rand < a pre-determined probability pone then generate a new individual based on one randomly selected normal individual; else two individuals from normal are randomly selected to generate new individual; The newly generated individual is compared with the existing individual with the same individual index, the better one is kept and recorded as the new individual;

18

S. Cheng et al.

4.3 Distribution Function An individual is generated by adding a Gaussian random value in Eq. (4). The distribution of this random number could be changed to Cauchy distribution. The BSOOS algorithm with the Gaussian random values is termed as the BSO-OS-Gaussian, while the BSO-OS algorithm with the Cauchy random values is termed as the BSOOS-Cauchy algorithm. The new individual generation equation for BSO-OS-Cauchy algorithm is in Eq. (7). i i = xold + ξ(t) × C(μ, σ 2 ) xnew

(7)

4.4 Transfer Function A transfer function logsig(a), which is given in Eq. (8), has been deployed in step size Eq. (5). The parameter k is used to adjust the function’s slope. In the original BSO-OS algorithm, the maximum number of iterations T is set as 2000, and the slope to k is 25 [65]. Thus, the value 0.5 ×kT − t in Eq. (5) is linearly decreased from 1000−0 25 1000−2000 , i.e., it is mapped into a range [−40, 40]. Figure 8 gives the function logsig() 25 with different variable ranges. It shows that, for function logsig() with variables in range [−40, 40], nearly the half values are close to one, and the other half values are close to zero. Normally, to have a smooth curve of step size, it’s better to map the value 0.5 ×kT − t into the range [−10, 10]. logsig(a) =

1 1 + exp(−a)

(8)

The original transfer function has several weaknesses, such as the setting of the parameter k and the choosing of the range of logsig(a) during optimization. Some modified transfer functions have been designed to overcome these obstacles [48]. An example of a modified transfer function δ() is given as follows. δ() = 0.5 × 2exp [1−(T /(T +1−t))] The parameter k is omitted in the modified transfer function. Figure 9 gives the function δ() with different variable ranges. The modified transfer function has nearly the same curve with different variable ranges, which may be benefited for solving large-scale optimization problems.

Brain Storm Optimization Algorithms: More Questions than Answers 1

1

0.9

0.9 a

0.8

[−40,40]

a

0.8

0.7 0.6 0.5 0.4 0.3

0.6 0.5 0.4 0.3

0.2

0.2

0.1

0.1

0 −40 −30 −20 −10

0

10

20

30

0 −10

40

−5

0

a

(a) a

5

10

a

(b) a

40, 40

1

10, 10

0.75

0.9

0.7 a

0.8

[−5,5]

a

0.65

0.7

0.6

0.6

0.55

logsig(a)

logsig(a)

[−10,10]

0.7

logsig(a)

logsig(a)

19

0.5 0.4

0.5 0.45

0.3

0.4

0.2

0.35

0.1

[−1,1]

0.3

0 −5

0

0.25 −1

5

−0.5

0

a

(c) a

0.5

1

a

5, 5

(d) a

1, 1

1 0.95

t

0.9

[1−(T/(T+1−t))]

1 0.95 [1,100]

0.85 0.8

(t) = 0.5 * 2e

(t) = 0.5 * 2e

[1−(T/(T+1−t))]

Fig. 8 The function logsig(−a) with different variable ranges

0.75 0.7 0.65 0.6 0.55 0.5

t

0.9

[1,10000]

0.85 0.8 0.75 0.7 0.65 0.6 0.55

0

20

60

40

80

100

0.5

0

2000

6000

4000

t

t

(a) T = 100

(b) T = 10000

Fig. 9 An example of a modified transfer function

8000

10000

20

S. Cheng et al.

4.5 Hybrid Algorithms The aim of hybrid algorithm is to combine the strength of two or more algorithms, while simultaneously trying to minimize any substantial disadvantage [79]. The BSO algorithm has been combined with several methods to solve various problems, such as the chaotic local search [85], whale optimization algorithm [78], moth flame optimization [37], Nelder-Mead Simplex (NMS) method [8], just to name a few.

5 Problems Solving 5.1 Single Objective Problems The single objective problems, especially problems only with the boundary constraints, are the basic problems in optimization. Normally, new proposed swarm intelligence algorithms are firstly tested on the simple objective problems, and the BSO algorithm is no exception. The original BSO algorithm was tested on ten benchmark functions [62, 63].

5.2 Multi-objective Optimization A general multiobjective optimization problem (MOP) can be described as a vector function f that maps a tuple of n parameters (decision variables) to a tuple of k objectives. Without loss of generality, a minimization problem is assumed throughout this chapter. minimize f(x) = (f1 (x), f2 (x), . . . , fk (x)) subject to x = (x1 , x2 , . . . , xn ) ∈ X y = (y1 , y2 , . . . , yk ) ∈ Y where x is called the decision vector, X is the decision space, y is the objective vector, and Y is the objective space, and f : X → Y consists of k real-valued objective functions (Fig. 10). A decision vector, x1 dominates a decision vector, x2 (denoted by x1 ≺ x2 ), if and only if • x1 is not worse than x2 in all objectives, i.e. fk (x1 ) ≤ fk (x2 ), ∀k = 1, . . . , nk , and • x1 is strictly better than x2 in at least one objective, i.e. ∃k = 1, . . . , nk : fk (x1 ) < fk (x2 ). A point x∗ ∈ X is called Pareto optimal if there is no x ∈ X such that x dominates x . The set of all the Pareto optimal points is called the Pareto set (denoted as PS). ∗

Brain Storm Optimization Algorithms: More Questions than Answers Fig. 10 The example of domination: set A dominates the set B, and the points x1 and x2 dominate the point x3 and x4

21

f2 B

x3

x2

x4 A

x1 A

B

f1

The set of all the Pareto objective vectors, PF = {f (x) ∈ X |x ∈ PS}, is called the Pareto front (denoted as PF). In a multiobjective optimization problem, the aim is finding the set of optimal trade-off solutions known as the Pareto optimal set. Pareto optimality is defined with respect to the concept of nondominated points in the objective space. Swarm intelligence is particularly suitable to solve multiobjective optimization problems because they deal simultaneously with a set of possible solutions. This allows us to find an entire set of Pareto optimal set in a single run of the algorithm, instead of having to perform a series of separate runs as in the case of the traditional mathematical programming techniques [20, 45]. Additionally, swarm intelligence is less susceptible to the shape or continuity of the Pareto front. Several new techniques are combined to solve multiobjective problems with more than ten objectives, in which almost every solution is Pareto nondominated in the problems [38]. These techniques include objective decomposition [87], objective reduction [6, 61], and clustering in the objective space [62, 63]. The brain storm optimization algorithm also can be extended to solve multiobjective optimization problems [35, 68, 84]. Unlike the traditional multiobjective optimization methods, the brain storm optimization algorithm utilized the objective space information directly. Clusters are generated in the objective space; and for each objective, individuals are clustered in each iteration [68] or clustered in the objective space [84]. The individual, which performs better in most of objectives are kept to the next iteration, and other individuals are randomly selected to keep the diversity of solutions.

5.3 Multimodal Optimization Many optimization algorithms are designed for locating a single global solution. Nevertheless, many real-world problems may have multiple satisfactory solutions exist. The multimodal optimization problem is a function with multiple global/local optimal values [9]. The equal maxima function, given in Eq. (9), is an example of the multimodal optimization problem,

22

S. Cheng et al.

(a)

(b)

1

0.5 0.4 0.3 0.2 0.1 0

0

0.2

0.4

0.6

0.8

0.8

−0.05))

0.6

3/4

0.7

6

y = sin6(5 π x)

0.8

0.9 0.7 0.6

sin (5 π (x

2 y=exp(−2log(2)((x−0.08)/0.854) )

1

0.9

1

0.5 0.4 0.3 0.2 0.1 0

0

0.2

0.4

0.6

0.8

1

x ∈ [0, 1]

x ∈ [0, 1]

Fig. 11 Example of function with the equal or uneven decreasing maxima

f (x) = sin6 (5 × π × x)

(9)

where x ∈ [0, 1]. From Fig. 11a, it can be seen that there are five equal optima for Eq. (9). The uneven decreasing maxima function in Eq. (10) is another example of the multimodal optimization problem. From Fig. 11b, it can be seen that there are one global optimum and four local optima for Eq. (10). 



x − 0.08 f (x) = exp −2 log(2) 0.854

2 

3

sin6 (5π(x 4 − 0.05))

(10)

where x ∈ [0, 1]. For multimodal optimization, the objective is to locate multiple peaks/optima in a single run [58, 59], and to keep these found optima until the end of a run [50, 55, 60]. An algorithm on solving multimodal optimization problems should have two kinds of abilities: find global/local optima as many as possible and preserve these found solutions until the end of the search. Niching method is able to locate multiple solutions in multimodal search areas. The BSO algorithm could be combined with other local search strategies or niching techniques to solve multimodal optimization problems or dynamic multimodal optimization problems [12].

5.4 Real-World Applications Many real-world applications could be modeled as optimization problems. As an outstanding swarm intelligence algorithm, the brain storm optimization has been widely used to solve enormous real-word problems. Different scheduling problems

Brain Storm Optimization Algorithms: More Questions than Answers

23

have been solved by the brain storm optimization algorithms, such as grid scheduling [3], heating dispatch scheduling of thermal power plant problem [40], and cumulative capacitated vehicle routing problem [41]. Other applications include sales territory design problem [54], multimodal vein-based recognition system [80], just to name a few.

6 Future Research Domingos has stated that “The brain can learn anything, but it can’t evolve a brain” [23]. This statement is correct for an individual but may not be correct for a group in a long time. The function of the brain is evolving over time, and individuals may have the different ability on different tasks. In order to solve complex problems, intelligent algorithms should be able to learn and evolve like a group of brains. The evolution and learning are two basic characteristics of BSO algorithms, or more general, swarm intelligence algorithms. There are many unsolved issues for swarm intelligence algorithms, such as the exploration versus exploitation, the population diversity, and the parameter setting.

6.1 Unified Swarm Intelligence Various frameworks have been proposed for swarm intelligence algorithms, such as developmental swarm intelligence (DSI) [64], unified swarm intelligence algorithm [66], learning-interaction-diversification framework [19], and so on. The developmental swarm intelligence (DSI) is a new framework of swarm intelligence [64]. DSI algorithm has two kinds of functionalities: capability learning and capacity developing. The capacity developing is a top-level learning or macro-level learning methodology. The capacity developing describes the learning ability of an algorithm to adaptively change its parameters, structures, and/or its learning potential according to the search states of the problem to be solved. In other words, the capacity developing is the search potential possessed by an algorithm. The capability learning is a bottom-level learning or micro-level learning. The capability learning describes the ability for an algorithm to find better solution(s) from current solution(s) with the learning capacity it possesses. Figure 12 shows the flowchart of developmental swarm intelligence algorithm. The swarm intelligence algorithm, such as the particle swarm optimization algorithms and the brain storm optimization could be analyzed by using the developmental swarm intelligence frameworks. The framework of developmental particle swarm optimization algorithms and developmental brain storm optimization algorithms are shown in Fig. 13 and Fig. 14, respectively. More details of developmental swarm intelligence algorithms are discussed in [64].

24

S. Cheng et al.

Fig. 12 The framework of developmental swarm intelligence algorithms

Start Initialization Capacity developing Capability learning

No

End? Yes Stop

Even there are many kinds of swarm intelligence algorithms, the way of new individual generation could be categorized into several classes. Three means of evolutionary change, which include genetically, ecologically, and developmentally are compared in biological theory [5]. These kinds of changes may be modeled as strategies into swarm intelligence algorithms. The framework of unified swarm intelligence algorithm is introduced to give an illustration on different swarm intelligence algorithms [66]. Figure 15 shows the flowchart of unified swarm intelligence algorithm. The capability learning could be categorized into three classes: learning, emulating, and exploring.

Particle Swarm Optimization

Developmental Swarm Intelligence

Parameters and/or neighborhood topology setting

Capacity Developing

Velocity and position updating

Capability Learning

Fig. 13 The framework of developmental particle swarm optimization algorithms Brain Storm Optimization

Developmental Swarm Intelligence

Convergent Operation

Capacity Developing

Divergent Operation

Capability Learning

Fig. 14 The framework of developmental brain storm optimization algorithms

Brain Storm Optimization Algorithms: More Questions than Answers Fig. 15 The framework of unified swarm intelligence algorithms

25

Start Initialization Capacity developing

Learning

Capability learning

Emulating Exploring

No

End? Yes Stop

6.2 Evolution and Learning The evolution theory and learning theory are two foundational issues in the evolutionary computation/swarm intelligence. Generally, the evolution theory indicates random variation and selection to iterative adaptation while the learning theory indicates obtain knowledge from previous experience and utilize this knowledge to guide future behaviors [83]. Evolution is an important concept in evolutionary computation and swarm intelligence. In biology, the “evolution” is defined as “a change in the heritable characteristics of biological populations over successive generations.” In swarm intelligence algorithms, the evolution indicates that the solutions are generated iteration by iteration, and move toward better and better search areas over the course of the search process. For BSO algorithms, the role of a solution, i.e., at different groups or as a group center or not, is selected in this self-evolution or self-improvement process. The obtained fitness values are improved based on the self-evolution process. Normally, there are multiple groups in the swarm intelligence algorithms, and information is propagated in all individuals. The data-driven algorithm indicates that the algorithm could extract the features from the solved problem and obtain the landscape by learning on the data set [11]. Figure 16 gives a framework of data-driven swarm intelligence algorithms. Each candidate solution is a data sample from the search space. The model could be designed or adjusted via the data analysis on the previous solutions. The landscape or the difficulty of a problem could be obtained during the search, i.e., the problem could be understood better. With the learning process, more suitable algorithms could be designed to solve different problems, thus, the performance of optimization could be improved. Learning has two aspects in swarm intelligence algorithms. One is learning from the problems, which means the algorithm is able to adjust its search strategies or

26

S. Cheng et al.

Random initialization Initial solutions Data analysis (clustering/classification) Solutions with fitness values Learning/ Optimization

Design/ Adjusting Optimization model Brain storm optimization algorithms

Optimal solution(s) Fig. 16 A framework for data-driven swarm intelligence algorithms

parameters dynamically during the search. The other one is learning from solutions themselves, which indicates the approach of search information propagated among all individuals. The capability learning indicates that the algorithm has an ability that could learning from problem. The BSO could dynamically adjust its search strategy during the search. The objects of learning could be different for various BSO algorithms. Several brain storm optimization algorithms with different capability learning strategies have been proposed, such as periodic quantum learning strategy [70] and vector grouping learning strategy [47]. An individual in the swarm intelligence algorithm updates its numerical value in the search space at each iteration. The solution update indicates that it is learning from other individuals or learning from the problems. The BSO algorithm can be considered as a combination of swarm intelligence algorithms and data mining techniques. The data mining techniques can also be applied to design swarm intelligence algorithms. Massive information exists during the search process. For swarm intelligence algorithms, there are several individuals existed at the same time, and each individual has a corresponding fitness value. The individuals are created iteration over iteration. There is also massive volume of information on the “origin” of an individual, such as that an individual was created by applying which strategy and parameters to which former individual(s). The datadriven evolutionary computation/swarm intelligence is a new approach to analyze and guide the search in evolutionary algorithms/swarm intelligence. These strategies could be divided into off-line methods and online methods. More researches should be conducted on the combination of data mining and swarm intelligence, especially for the online analysis. Not only the parameter could be adjusted, but also the algorithm, i.e. the whole swarm could be adaptively adjusted during the search. The above can be considered from the framework of the developmental swarm intelligence (DSI) algorithm. The BSO algorithm is a good example

Brain Storm Optimization Algorithms: More Questions than Answers

27

of DSI algorithms, including the same operators: divergence operator and convergence operator. The “good enough” optimum could be obtained through the solutions divergence and convergence in the search space. The BSO algorithm could be analyzed by the convergence and divergence framework. In BSO algorithm, the random initialized solutions are convergent to different areas. This is a convergence strategy, and the new solutions are generated to diverge in the search space. The solutions with good fitness values are selected, which indicates that the solutions are converged to small areas. The convergence and divergence strategies process iteration over iteration. Based on the iterations of convergence and divergence, the solutions could be clustered to small regions finally.

6.3 Real-World Applications Different optimization problems could be modeled in many areas in our everyday life. Various algorithms have been proposed with different metaphors [71]. However, an important thing is utilized evolutionary algorithms and/or swarm intelligence algorithms to solve real-world problems [53]. The applications of genetic programming and hyper-heuristics, especially for automated software development, are summarized [42]. With the brain storm optimization algorithms, or more generally swarm intelligence, more effective applications or systems can be designed to solve real-world problems. The brain storm optimization algorithm not only could be used in problem with explicit model, but also in problem with implicit model. With applications in complex engineering or design problems, the strength and limitation of various brain storm optimization algorithms could be revealed and interpreted.

7 Conclusion In this chapter, brain storm optimization algorithms have been comprehensively surveyed from the perspective of model-driven and data-driven approaches. Brain storm optimization (BSO) algorithms is a framework that indicates algorithms using converging operation and diverging operation to locate the optima of optimization problems. For swarm intelligence algorithms, each individual in the swarm represents a solution in the search space, and it also can be seen as a data sample from the search space. Based on the analyses of these data, more effective algorithms and search strategies could be proposed. Brain storm optimization (BSO) algorithm is a new and promising swarm intelligence algorithm, which simulates the human brainstorming process. Through the convergent operation and divergent operation, individuals in BSO are grouped and diverged in the search space/objective space. In this Chapter, the history development, and the state-of-the-art of the BSO algorithm have been reviewed. In addition, the convergent operation and divergent operation in the BSO algorithm have also been discussed from the data analysis perspective.

28

S. Cheng et al.

Every individual in the BSO algorithm is not only a solution to the problem to be optimized, but also a data point to reveal the landscape of the problem. Many works have been conducted on the BSO algorithms, there are still massive questions on this algorithm need to be answered. These questions, which include the properties of BSO algorithms, the connection between optimization algorithms and problems, and the real-world applications should be studied for the BSO algorithms or, more broadly, for the swarm intelligence algorithms. Acknowledgements This research is supported by the National Natural Science Foundation of China under Grant No. 61806119, 61761136008, 61773119, 61703256, and 61771297; in part by the Shenzhen Science and Technology Innovation Committee under grant number ZDSYS201703031748284; and in part by the Fundamental Research Funds for the Central Universities under Grant GK201703062.

References 1. Adra, S.F., Dodd, T.J., Griffin, I.A., Fleming, P.J.: Convergence acceleration operator for multiobjective optimization. IEEE Trans. Evol. Comput. 12(4), 825–847 (2009) 2. Adra, S.F., Fleming, P.J.: Diversity management in evolutionary many-objective optimization. IEEE Trans. Evol. Comput. 15(2), 183–195 (2011) 3. Arsuaga-Ríos, M., Vega-Rodríguez, M.A.: Cost optimization based on brain storming for grid scheduling. In: Proceedings of the 2014 Fourth International Conference on Innovative Computing Technology (INTECH), pp. 31–36. Luton, UK (2014) 4. Beyer, H.G., Schwefel, H.P.: Evolution strategies—a comprehensive introduction. Nat. Comput. 1(1), 3–52 (2002) 5. Blute, M.: Three modes of evolution by natural selection and drift: a new or an extended evolutionary synthesis? Biol. Theory 12(2), 67–71 (2017) 6. Brockhoff, D., Zitzler, E.: Objective reduction in evolutionary multiobjective optimization: theory and applications. Evol. Comput. 17(2), 135–166 (2009) 7. Chen, J., Cheng, S., Chen, Y., Xie, Y., Shi, Y.: Enhanced brain storm optimization algorithm for wireless sensor networks deployment. In: Proceedings of 6th International Conference on Swarm Intelligence (ICSI 2015), pp. 373–381. Springer International Publishing, Beijing, China (2015) 8. Chen, W., Cao, Y., Cheng, S., Sun, Y., Liu, Q., Li, Y.: Simplex search based brain storm optimization. IEEE Access (2018, in press) 9. Cheng, S., Chen, J., Lei, X., Shi, Y.: Locating multiple optima via developmental swarm intelligence. IEEE Access 6, 17039–17049 (2018) 10. Cheng, S., Liu, B., Shi, Y., Jin, Y., Li, B.: Evolutionary computation and big data: key challenges and future directions. In: Tan, Y., Shi, Y. (eds.) Data Mining and Big Data (DMBD 2016). Lecture Notes in Computer Science, vol. 9714, pp. 3–14. Springer International Publishing Switzerland (2016) 11. Cheng, S., Lu, H., Lei, X., Shi, Y.: A quarter century of particle swarm optimization. Complex & Intell. Syst. 4(3), 227–239 (2018) 12. Cheng, S., Lu, H., Song, W., Chen, J., Shi, Y.: Dynamic multimodal optimization using brain storm optimization algorithms. In: Qiao, J., Zhao, X., Pan, L., Zuo, X., Zhang, X., Zhang, Q., Huang, S. (eds.) Bio-Inspired Computing: Theories and Applications (BIC-TA 2018), pp. 1–10. Springer Nature Singapore (2018) 13. Cheng, S., Qin, Q., Chen, J., Shi, Y.: Brain storm optimization algorithm: a review. Artif. Intell. Rev. 46(4), 445–458 (2016)

Brain Storm Optimization Algorithms: More Questions than Answers

29

14. Cheng, S., Shi, Y.: Thematic issue on “brain storm optimization algorithms". Memetic Comput. 10(4), 351–352 (2018) 15. Cheng, S., Shi, Y., Qin, Q., Gao, S.: Solution clustering analysis in brain storm optimization algorithm. In: Proceedings of the 2013 IEEE Symposium on Swarm Intelligence, (SIS 2013), pp. 111–118. IEEE, Singapore (2013) 16. Cheng, S., Shi, Y., Qin, Q., Ting, T.O., Bai, R.: Maintaining population diversity in brain storm optimization algorithm. In: Proceedings of 2014 IEEE Congress on Evolutionary Computation (CEC 2014), pp. 3230–3237. IEEE, Beijing, China (2014) 17. Cheng, S., Sun, Y., Chen, J., Qin, Q., Chu, X., Lei, X., Shi, Y.: A comprehensive survey of brain storm optimization algorithms. In: Proceedings of 2017 IEEE Congress on Evolutionary Computation (CEC 2017), pp. 1637–1644. IEEE, Donostia, San Sebastián, Spain (2017) 18. Cheng, S., Zhang, Q., Qin, Q.: Big data analytics with swarm intelligence. Ind. Manag. & Data Syst. 116(4), 646–666 (2016) 19. Chu, X., Wu, T., Weir, J.D., Shi, Y., Niu, B., Li, L.: Learning-interaction-diversification framework for swarm intelligence optimizers: a unified perspective. Neural Comput. Appl. (2018) 20. Coello Coello, C.A., Lamont, G.B., Van Veldhuizen, D.A.: Evolutionary Algorithms for Solving Multi-objective Problems. Genetic and Evolutionary Computation Series, 2nd edn. Springer (2007) 21. De Jong, K.A.: An analysis of the behavior of a class of genetic adaptive systems. Ph.D. thesis, Department of Computer and Communication Sciences, University of Michigan (1975) 22. Deb, K., Jain, H.: An evolutionary many-objective optimization algorithm using referencepoint-based nondominated sorting approach, part I: solving problems with box constraints. IEEE Trans. Evol. Comput. 18(4), 577–601 (2014) 23. Domingos, P.: The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World. Basic Books, New York (2015) 24. Dorigo, M., Gambardella, L.M.: Ant colony system: a cooperative learning approach to the traveling salesman problem. IEEE Trans. Evol. Comput. 1(1), 53–66 (1997) 25. Duan, H., Qiao, P.: Pigeon-inspired optimization: a new swarm intelligence optimizer for air robot path planning. J. Intell. Comput. Cybern. 7(1), 24–37 (2014) 26. Eberhart, R., Kennedy, J.: A new optimizer using particle swarm theory. In: Proceedings of the Sixth International Symposium on Micro Machine and Human Science, pp. 39–43 (1995) 27. Eberhart, R., Shi, Y.: Computational Intelligence: Concepts to Implementations. Morgan Kaufmann Publishers, San Francisco (2007) 28. Ficici, S.G.: Monotonic solution concepts in coevolution. In: Genetic and Evolutionary Computation Conference (GECCO 2005), pp. 499–506 (2005) 29. Fogel, L.J.: Evolutionary programming in perspective: the top-down view. In: Zurada, J.M., Marks, R.I., Robinson, C.J. (eds.) Computational Intelligence: Imitating Life, pp. 135–146. IEEE Press, Piscataway, NJ (1994) 30. Fogel, L.J., Owens, A.J., Walsh, M.J.: Artificial Intelligence Through Simulated Evolution. Wiley, New York, NY (1966) 31. Goldberg, D.E.: Genetic Algorithms in Search. Optimization and Machine Learning. AddisonWesley Longman Publishing Co., Inc, Boston, MA, USA (1989) 32. Goldberg, D.E., Deb, K.: A comparative analysis of selection schemes used in genetic algorithms. In: Rawlins, G.J. (ed.) Foundations of Genetic Algorithms, vol. 1, pp. 69–93. Elsevier (1991) 33. Gopalan, R.: Model-driven and data-driven approaches for some object recognition problems. Ph.D. thesis, Department of Electrical and Computer Engineering, University of Maryland (2011) 34. Guo, X., Wu, Y., Xie, L.: Modified brain storm optimization algorithm for multimodal optimization. In: Tan, Y., Shi, Y., Coello, C.A.C. (eds.) Advances in Swarm Intelligence. Lecture Notes in Computer Science, vol. 8795, pp. 340–351. Springer International Publishing (2014) 35. Guo, X., Wu, Y., Xie, L., Cheng, S., Xin, J.: An adaptive brain storm optimization algorithm for multiobjective optimization problems. In: Proceedings of 6th International Conference on Swarm Intelligence (ICSI 2015), pp. 365–372. Springer International Publishing, Beijing, China (2015)

30

S. Cheng et al.

36. Holland, J.H.: Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology. The University of Michigan Press, Control and Artificial Intelligence (1975) 37. Ibrahim, R.A., Elaziz, M.A., Ewees, A.A., Selim, I.M., Lu, S.: Galaxy images classification using hybrid brain storm optimization with moth flame optimization. J. Astron. Telesc. Instrum. Syst. 4(3), 1–18 (2018) 38. Ishibuchi, H., Tsukamoto, N., Nojima, Y.: Evolutionary many-objective optimization: a short review. In: Proceedings of 2008 IEEE Congress on Evolutionary Computation (CEC2008), pp. 2424–2431. Hong Kong (2008) 39. Jin, Y., Sendhoff, B.: A systems approach to evolutionary multiobjective structural optimization and beyond. IEEE Comput. Intell. Mag. 4(3), 62–76 (2009) 40. Kang, L., Wu, Y., Wang, X., Feng, X.: Brain storming optimization algorithm for heating dispatch scheduling of thermal power plant. In: Proceedings of 2017 29th Chinese Control and Decision Conference (CCDC 2017), pp. 4704–4709 (2017) 41. Ke, L.: A brain storm optimization approach for the cumulative capacitated vehicle routing problem. Memetic Comput. 10, 411–421 (2018) 42. Kendall, G.: Is evolutionary computation evolving fast enough? IEEE Computational Intelligence Magazine 13(2), 42–51 (2018) 43. Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of IEEE International Conference on Neural Networks, pp. 1942–1948 (1995) 44. Kennedy, J., Eberhart, R., Shi, Y.: Swarm Intelligence. Morgan Kaufmann Publishers, San Francisco (2001) 45. Kim, Y.S.: Multi-objective clustering with data- and human-driven metrics. J. Comput. Inf. Syst. 51(4), 64–73 (2011) 46. Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. A Bradford Book (1992) 47. Li, C., Hu, D., Song, Z., Yang, F., Luo, Z., Fan, J., Liu, P.X.: A vector grouping learning brain storm optimization algorithm for global optimization problems. IEEE Access (2018, in press) 48. Li, C., Song, Z., Fan, J., Cheng, Q., Liu, P.X.: A brain storm optimization with multi-information interactions for global optimization problems. IEEE Access 6, 19304–19323 (2018) 49. Li, L., Zhang, F.F., Chu, X., Niu, B.: Modified brain storm optimization algorithms based on topology structures. In: Proceedings of 7th International Conference on Swarm Intelligence (ICSI 2016), pp. 408–415. Springer International Publishing, Bali, Indonesia (2016) 50. Li, X.: Niching without niching parameters: particle swarm optimization using a ring topology. IEEE Trans. Evol. Comput. 14(1), 150–169 (2010) 51. Mauldin, K.A.: Maintaining diversity in genetic search. In: Proceedings of the National Conference on Artificial Intelligence (AAAI 1984), pp. 247–250 (1984) 52. Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs, 3rd, revised and extended edn. Springer (1996) 53. Michalewicz, Z.: Ubiquity symposium: evolutionary computation and the processes of life: the emperor is naked: evolutionary algorithms for real-world applications. Ubiquity 3(1–3), 13 (2012) 54. Nucamendi-Guillén, S., Dávila, D., Camacho-Vallejo, J.F., González-Ramírez, R.G.: A discrete bilevel brain storm algorithm for solving a sales territory design problem: a case study. Memetic Comput. 10(4), 441–458 (2018) 55. Parrott, D., Li, X.: Locating and tracking multiple dynamic optima by a particle swarm model using speciation. IEEE Trans. Evol. Comput. 10(4), 440–458 (2006) 56. Purshouse, R.C., Fleming, P.J.: On the evolutionary optimization of many conflicting objectives. IEEE Trans. Evol. Comput. 11(6), 770–784 (2007) 57. Qiu, H., Duan, H., Zhou, Z., Hu, X., Shi, Y.: Chaotic predator-prey brain storm optimization for continuous optimization problems. In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–7. Honolulu, HI, USA (2017) 58. Qu, B.Y., Liang, J., Suganthan, P.: Niching particle swarm optimization with local search for multi-modal optimization. Inf. Sci. 197, 131–143 (2012)

Brain Storm Optimization Algorithms: More Questions than Answers

31

59. Qu, B.Y., Suganthan, P., Liang, J.: Differential evolution with neighborhood mutation for multimodal optimization. IEEE Trans. Evol. Comput. 16(5), 601–614 (2012) 60. Rönkkönen, J.: Continuous multimodal global optimization with differential evolution-based methods. Ph.D. thesis, Department of information technology, Lappeenranta University of Technology (2009) 61. Saxena, D.K., Duro, J.A., Tiwari, A., Deb, K., Zhang, Q.: Objective reduction in many-objective optimization: linear and nonlinear algorithms. IEEE Trans. Evol. Comput. 17(1), 77–99 (2013) 62. Shi, Y.: Brain storm optimization algorithm. In: Tan, Y., Shi, Y., Chai, Y., Wang, G. (eds.) Advances in Swarm Intelligence. Lecture Notes in Computer Science, vol. 6728, pp. 303–309. Springer, Berlin/Heidelberg (2011) 63. Shi, Y.: An optimization algorithm based on brainstorming process. Int. J. Swarm Intell. Res. (IJSIR) 2(4), 35–62 (2011) 64. Shi, Y.: Developmental swarm intelligence: developmental learning perspective of swarm intelligence algorithms. Int. J. Swarm Intell. Res. (IJSIR) 5(1), 36–54 (2014) 65. Shi, Y.: Brain storm optimization algorithm in objective space. In: Proceedings of 2015 IEEE Congress on Evolutionary Computation (CEC 2015), pp. 1227–1234. IEEE, Sendai, Japan (2015) 66. Shi, Y.: Unified swarm intelligence algorithms. In: Shi, Y. (ed.) Critical Developments and Applications of Swarm Intelligence, pp. 1–26. IGI Global (2018) 67. Shi, Y., Eberhart, R.: A modified particle swarm optimizer. In: Proceedings of the 1998 Congress on Evolutionary Computation (CEC1998), pp. 69–73 (1998) 68. Shi, Y., Xue, J., Wu, Y.: Multi-objective optimization based on brain storm optimization algorithm. Int. J. Swarm Intell. Res. (IJSIR) 4(3), 1–21 (2013) 69. Solomatine, D., See, L., Abrahart, R.: Data-driven modelling: concepts, approaches and experiences. In: Abrahart, R.J., See, L.M., Solomatine, D.P. (eds.) Practical Hydroinformatics: Computational Intelligence and Technological Developments in Water Applications, pp. 17– 30. Springer, Berlin Heidelberg, Berlin, Heidelberg (2008) 70. Song, Z., Peng, J., Li, C., Liu, P.X.: A simple brain storm optimization algorithm with a periodic quantum learning strategy. IEEE Access 6, 19968–19983 (2018) 71. Sörensen, K.: Metaheuristics—the metaphor exposed. Int. Trans. Oper. Res. 22(1), 3–18 (2015) 72. Sundaram, R.K.: A First Course in Optimization Theory. Cambridge University Press, Cambridge, United Kingdom (1996) 73. Tan, Y.: Fireworks Algorithm: A Novel Swarm Intelligence Optimization Method. SpringerVerlag, Berlin, Heidelberg (2015) 74. Tan, Y., Yu, C., Zheng, S., Ding, K.: Introduction to fireworks algorithm. Int. J. Swarm Intell. Res. (IJSIR) 4(4), 39–70 (2013) 75. Tan, Y., Zhu, Y.: Fireworks algorithm for optimization. In: Tan, Y., Shi, Y., Tan, K.C. (eds.) Advances in Swarm Intelligence. Lecture Notes in Computer Science, vol. 6145, pp. 355–364. Springer, Berlin, Heidelberg (2010) 76. Tang, L., Liu, J., Rong, A., Yang, Z.: A multiple traveling salesman problem model for hot rolling scheduling in shanghai baoshan iron & steel complex. Eur. J. Oper. Res. 124(2), 267–282 (2000) 77. Tang, L., Zhao, Y., Liu, J.: An improved differential evolution algorithm for practical dynamic scheduling in steelmaking-continuous casting production. IEEE Trans. Evol. Comput. 18(2), 209–225 (2014) 78. Thanga Revathi, S., Ramaraj, N., Chithra, S.: Brain storm-based whale optimization algorithm for privacy-protected data publishing in cloud computing. Clust. Comput., pp. 1–10 (2018) 79. Ting, T.O., Yang, X.S., Cheng, S., Huang, K.: Hybrid metaheuristic algorithms: past, present, and future. In: Yang, X.S. (ed.) Recent Advances in Swarm Intelligence and Evolutionary Computation, Studies in Computational Intelligence (SCI), vol. 585, pp. 71–83. Springer International Publishing (2015) 80. Verma, D., Dubey, S.: Fuzzy least brain storm optimization and entropy-based euclidean distance for multimodal vein-based recognition system. J. Cent. South Univ. 24(10), 2360–2371 (2017)

32

S. Cheng et al.

81. Wang, J., Chen, J., Xue, X., Du, J.: Search strategies investigation in brain storm optimization. Memetic Comput. 10(4), 397–409 (2018) 82. Wang, Y., Gao, S., Yu, Y., Xu, Z.: The discovery of population interaction with a power law distribution in brain storm optimization. Memetic Comput. (2018, in press) 83. Watson, R.A., Szathmáry, E.: How can evolution learn? Trends Ecol. & Evol. 31(2), 147–157 (2016) 84. Xue, J., Wu, Y., Shi, Y., Cheng, S.: Brain storm optimization algorithm for multi-objective optimization problems. In: Tan, Y., Shi, Y., Ji, Z. (eds.) Advances in Swarm Intelligence. Lecture Notes in Computer Science, vol. 7331, pp. 513–519. Springer, Berlin/Heidelberg (2012) 85. Yu, Y., Gao, S., Cheng, S., Wang, Y., Song, S., Yuan, F.: CBSO: a memetic brain storm optimization with chaotic local search. Memetic Comput. 10(4), 353–367 (2018) 86. Zhan, Z.H., Zhang, J., Shi, Y.H., Liu, H.l.: A modified brain storm optimization. In: Proceedings of the 2012 IEEE Congress on Evolutionary Computation (CEC 2012), pp. 1–8. Brisbane, QLD, Australia (2012) 87. Zhang, Q., Liu, W., Tsang, E., Virginas, B.: Expensive multiobjective optimization by MOEA/D with Gaussian process model. IEEE Trans. Evol. Comput. 3(14), 456–474 (2010) 88. Zhou, D., Shi, Y., Cheng, S.: Brain storm optimization algorithm with modified step-size and individual generation. In: Tan, Y., Shi, Y., Ji, Z. (eds.) Advances in Swarm Intelligence. Lecture Notes in Computer Science, vol. 7331, pp. 243–252. Springer, Berlin/Heidelberg (2012)

Part II

Methodology

Brain Storm Optimization for Test Task Scheduling Problem Hui Lu, Rongrong Zhou, Shi Cheng and Yuhui Shi

Abstract The test task scheduling problem (TTSP) has attracted increasing attention due to the wide range of automatic test systems applications. It is one kind of combinatorial optimization problem with the property of multimodal. There are a lot of global optima and local optima in its huge solution space. Brain storm optimization algorithm (BSO) is a newly proposed swarm intelligence algorithm, which can utilize the evolutionary information of solution space. BSO has two main operations including convergent operation and divergent operation to balance the exploration and exploitation ability in the search process. In addition, the clustering strategy in BSO can explore different regions simultaneously regarding the multimodal property. The generation of new solutions depends on several ways, which greatly improve the possibility of finding better solutions. According to the characteristic of TTSP, BSO is more suitable for solving TTSP comparing to other metaheuristic algorithms. In this chapter, BSO is first applied to solving TTSP cooperating with a real number coding strategy. For single-objective TTSP, BSO divides the solution space into several clusters and produces the new solution based on various methods. Moreover, some modifications are proposed on BSO for solving multi-objective TTSP. It mainly includes the determination of centers and the reservation of better solutions. From the point of view of single objective and multi-objective problem, BSO has better performance comparing to recent scheduling algorithms for solving TTSP. H. Lu (B) · R. Zhou School of Electronic and Information Engineering, Beihang University, Beijing 100191, China e-mail: [email protected] R. Zhou e-mail: [email protected] S. Cheng School of Computer Science, Shaanxi Normal University, Xi’an 710119, China e-mail: [email protected] Y. Shi Shenzhen Key Lab of Computational Intelligence, Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen 518055, China e-mail: [email protected] © Springer Nature Switzerland AG 2019 S. Cheng and Y. Shi (eds.), Brain Storm Optimization Algorithms, Adaptation, Learning, and Optimization 23, https://doi.org/10.1007/978-3-030-15070-9_2

35

36

H. Lu et al.

Keywords Brain storm optimization · Test task scheduling problem · Multi-objective optimization

1 Introduction Test task scheduling problem (TTSP) derives from automatic test system (ATE) or automatic test equipment (ATS). Automatic test technology offers much more efficiency, repeatability, and reliability than manual operations in modern manufacturing processes and is thus widely used in many applications. In mobile phone terminal manufacture, it is used to measure the radio frequency (RF) or intermediate frequency (IF) performance. In spacecraft, missile, airplane and other aeronautics or astronautics systems, it represents one of the important technologies for ensuring safety of these systems. Automatic test technology covers the whole life cycle of the unit under test (UUT) or system for verifying the performance and monitoring running state of device. Test task scheduling problem is one key challenge in automatic test technology. This scheduling problem addresses allocating tasks to the available instruments to improve throughput, reduce test time, and optimize resource allocation [27]. Considering TTSP owing a finite set of tasks with multiple schemes to be processed on several unrelated resources, the TTSP can be decomposed into the task sequence problem and the scheme choice problem. The scheme choice of the task sequence increases exponentially by the number of tasks while the total number of the full array of task orders is factorial of the number of tasks. Hence from the view of time complexity, the TTSP is a NP-complete problem [19]. In detail, each scheme has the corresponding resource distribution. Only the scheme of a task is decided can the resources which are eligible to process the particular task as well as the setup time be assigned. These schemes are independent to each other, but tasks with different schemes may occupy the same resource. Resource conflicts exist among tasks, which may result in idle and waiting time. The task orders together with their scheme choices affect the finishing time of the test. Hence, the task sequence and the scheme choice have the close interactions. As a result, the solving methods can be categorized into hierarchical approaches and integrated approaches. The hierarchical approaches decompose the problem into two sub-problems: the assignment of instrument resources and the sequence of test tasks. Integrated approaches solve the test task sequence problem and scheme choice problem in the same process, while hierarchical approaches solve the two problems separately. The first issue related to test task scheduling problem is parallel test, which was proposed by National Instruments, a famous American equipment company, in their plan named ‘Next Generation Test System Framework’. In the company’s software, TestStand and TestBase, we can find some simple application of scheduling test sequence. Also, Teradyne adopted the concept of parallel test based on the multichannel hardware. Definitely, scheduling method is one key technology here. Inspired by these products, some researchers have been investigating the theoretical study. In

Brain Storm Optimization for Test Task Scheduling Problem

37

addition, we take a large-scale instance with 20 tasks for illustrating the property of TTSP. If one TTSP has 20 test tasks and every task has three alternative schemes on average, the total number of task sequence is 20! ≈ 1018 and the total number of scheme combination is 320 ≈ 109 . Therefore, the solution space is nearly 1027 . If we can obtain one solution by one nanosecond, we can access every solution by 1011 years, which is longer than the existence time of earth. The larger the instance is, the huger the solution space is. It is too difficult to find the optimal. Therefore, meta-heuristics algorithms outcome the classical optimization methods and become the mainstream of solving method. With the development of complex systems and equipment, the requirements are becoming more and more. In the automatic test system, it often needs to meet the requirements of multiple scheduling targets at the same time and these objectives usually conflict with each other. Example objectives include the maximal test completion time (makespan), the mean workload of the instruments and so on. Multi-objective TTSP is a research hot spot at present. A large number of meta-heuristics algorithms have been proposed to solve complex optimization problems. Among them, brain storm optimization (BSO) algorithm has attracted the attention of many researchers since it was proposed in 2011 [21]. It is a new swarm intelligence method that arising from the process of human beings problem-solving [8]. It was inspired from the brainstorming process [21, 22]. There are usually three kinds of persons in the brainstorming process, including a facilitator, a group of brainstorming persons, and problem owners. The facilitator is responsible for the main process of the brainstorming and ensure the process to be run smoothly without being biased. The brainstorming persons are mainly for generating different ideas and the problem owners are responsible for collecting better ideas in each run. There are two major components which are convergent operation and divergent operation in BSO. Although the components are similar to other swarm intelligence algorithms, the specific operations are different. The convergent operation focuses on picking up better ideas. The divergent operation adopts the method of adding Gaussian random values to a selected idea. The algorithm balances the process of exploration and exploitation through the convergent and diversity operations. Our main work and contribution is first applying the BSO to solve TTSP from two aspects, including single objective and multi-objective problem. In the aspect of single objective optimization, TTSP has a lot of local optima and global optima, which is not fully considered in other algorithms. In this way, BSO is more suitable for solving TTSP because it first divides the solution space into several clusters and improves the possibility of deep exploration. A good algorithm need to be tested on different complex real word problems and this just can promote the development of the algorithm from the aspect of its application. From another aspect of multiobjective TTSP, we propose a hybrid algorithm MOBSO, which combines BSO and a fast non-dominated sorting genetic algorithm II (NSGA-II) [6]. BSO can improve the convergence and the framework of NSGA-II can maintain the diversity of Pareto front. In the remainder of the chapter, we briefly mention the researches of TTSP, BSO and multi-objective algorithms in Sect. 2. Thereafter, in Sect. 3, we give an introduc-

38

H. Lu et al.

tion to problem and the characteristics for TTSP. We describe the proposed BSO and MOBSO for TTSP in detail in Sect. 4. The experimental results and performance comparisons are presented and discussed in Sect. 5. Finally, in Sect. 6, we outline the conclusions of this chapter and give future research directions.

2 Related Work 2.1 Test Task Scheduling Problem (TTSP) Test task scheduling problem (TTSP), as a typical representative of combinatorial optimization problem, has been researched from the point of view of single objective and multi-objective. Different intelligent optimization algorithms have been applied to solve this problem. A genetic simulated annealing algorithm (GASA) is proposed to optimize the multi-UUT parallel test task scheduling problem [28], and matrix-encoding scheme is adopted to link the task and instrument resource. An Object-Oriented meta-model of the parallel test is proposed by [27] to achieve high levels of reusability and flexibility. A novel optimizing algorithm [32] is proposed for parallel test resources allocation. In addition, a hybrid particle swarm optimization (PSO) and taboo search strategies are proposed for the single objective TTSP [13]. PSO is used for determining the optimal scheduling sequence and taboo is to get a resource optimal matrix for making choice of scheme. An encoding of particle with constraints and a new inertia weight are proposed according to the characteristics of TTSP. The constraint-guided methods with evolutionary algorithm are also considered for TTSP. Topological sorting approach was proposed to solve the network constraints problem. For multi-objective TTSP, a chaotic non-dominated sorting genetic algorithm is proposed to solve multi-objective TTSP and an integrated encoding scheme is used, which transforms a combinatorial optimization problem into a continuous optimization problem [14]. Two chaotic maps, the logistic map and Arnold’s cat map are combined with NSGA-II in the part of initial population, crossover operator and mutation operator to obtain the better Pareto Front. Therefore, different variants of NSGA-II are obtained and have difference performance. A variable neighborhood MOEA/D for multi-objective test task scheduling problem is proposed by [16]. Three mathematical models for TTSP are investigated. They are the Petri net model, the Graph model and the integer programming model. In addition, three controlling curves for the neighborhood size in MOEA/D have been discussed for finding the suitable parameter of the evolution process. Finally, the convergence analysis is carried out. Based on the above research, chaotic multiobjective evolutionary algorithm based on decomposition for TTSP is proposed [15]. Here, ten kinds of chaotic map are combined with MOEA/D to find the suitable one. They are Baker’s map, Arnold’s map, Circle map, Cubic map, Gauss map, ICMIC map, Logistic map, Sinusoidal map, Tent map and Zaslavskii map. The chaotic map has the nature to avoid

Brain Storm Optimization for Test Task Scheduling Problem

39

becoming trapped in local optima. TTSP has many local optima. All the experiments illustrate the fact that using chaotic maps embedded with the evolutionary algorithm can help TTSP to obtain good solutions. In addition, the distribution of the chaotic map has influence on the effect. Some chaotic maps, like Circle map, Cubic Map and ICMIC map, are relatively nonuniformly distributed. It is very similar to the distribution of the optimal solution of TTSP. Therefore, the three chaotic maps got the better performance through the experiment. However, the similarity degree of the distribution between the chaotic maps and the problem is a very essential factor for the application of chaotic maps. As mentioned above, TTSP is one kind of typical practical problem, which has an apparent representative for real-world problems. Therefore, the researches focused on TTSP also have significant meaning for other scheduling problem. In addition, it can provide sophisticated support for other industrial applications.

2.2 Brain Storm Optimization Since Brain Storm Optimization (BSO) algorithm was proposed in 2011, many researches have been done from different aspects, such as the theoretical analysis of the BSO algorithm, the improvement of search efficiency and its applications for real-world problems [5]. In the original BSO, the k-means clustering algorithm is utilized to implement the convergent operation in which the population of individuals are clustered into several clusters with the best idea in each cluster being used to represent one better idea picked up [23]. In addition, the original BSO is based on the decision space. However, the real word problems are complex and the relationship between the decision space and objective space is not clear. The clustering operation in the original BSO is time consuming because of the distance calculation. Therefore, another form of BSO in objective space is proposed [23]. There are also some variants of BSO proposed to improve the comprehensive performance. The solutions get clustered after a few iterations and the population diversity decreases quickly during the search, which are common phenomena in swarm intelligence. A definition of population diversity in BSO algorithm is introduced to measure the change of solutions’ distribution. According to the analysis, different kinds of partial reinitialization strategies are utilized to improve the population diversity in BSO algorithm [4]. Under similar consideration, chaotic predator-prey brain storm optimization is proposed to improve its ability for continuous optimization problems. A chaotic operation is further added to increase the diversity of population [18]. BSO performs better often due to its communication mechanisms. Different types of topology structures are incorporated into brain storm optimization algorithm to utilize different information exchange mechanisms [12]. Another challenging task is multimodal optimization, which needs to find both multiple global and local optima at the same time. To obtain potential multiple global and local optima, a self-adaptive brain storm optimization (SBSO) algorithm is proposed. A max-fitness

40

H. Lu et al.

grouping cluster method is used to divide the ideas into different sub-groups and these sub-groups can help to find the different optima [7]. BSO also shows good performance in different applications. BSO is used to get the optimal distribution strategy in an optimal heat load dispatching model. The comparison with the original scheme shows the correctness of the model and the optimization algorithm [10]. Moreover, brain storm optimization algorithm is used to solve an economic load dispatch problem (ED) for the system consisting of both thermal and wind generators effectively [9]. A new hybrid self-evolving brain-storming inclusive teaching-learning-based algorithm is presented with its application to an optimal power flow problem in electrical engineering. The hybrid algorithm is a combination of the learning principles from BSO and Teaching-Learning-Based optimization algorithm, along with a self-evolving principle applied to the control parameter. The strategies used in the proposed algorithm makes it self-adaptive in performing the search over the multi-dimensional problem domain [11]. BSO, a new swarm intelligence algorithm, is inspired by collective behavior of human beings. It has attracted many researches and has good performance in its applications for complex problems. More reviews about BSO can refer to [3, 5].

2.3 Multi-objective Optimization Algorithms Many real world problems are represented as multiple objectives which usually conflict with each other. These problems are called multi-objective problems. To solve these problems, more and more researches are focusing on the studies of multiobjective optimization algorithms. During the last decades, there have been a number of evolutionary algorithms and population-based methods which are successfully used to solve multi-objective optimization problems. For example, there are multiobjective genetic algorithm (MOGA) [2], non-dominated sorting genetic algorithm (NSGA, NSGA-II) [6, 24], multi-objective evolutionary algorithm based on decomposition (MOEA/D) [30], multi-objective particle swarm optimization (MOPSO) [31] and so on. Most of the above algorithms are to improve the convergence and distribution of the Pareto-front. Among them, NSGA-II is a well-known and representative multi-objective optimization algorithm and has been widely studied and used. The framework of NSGA-II can increase the diversity of population and ensure the distribution of Pareto front. In addition, the fast sorting non-dominated sorting method can improve efficiency greatly. BSO also has development in the field of multi-objective optimization recently. A multi-objective brain storm algorithm (MOBSA) is proposed to optimize the job scheduling problem in Grid environments. Different from single objective BSO, there are some clients as the owners of the problem to pick up several ideas as better ideas for solving the problem. If there are clusters, ideas would be selected from the first Pareto fronts and they are going to be considered as centers or better ideas picked up by problem owners [1]. Moreover, a modified Self-adaptive Multi-objective Brain Storm Optimization (SMOBSO) algorithm with clustering strategy and mutation is

Brain Storm Optimization for Test Task Scheduling Problem

41

proposed to solve multi-objective optimization problems. It adopts a simple grouping method in the cluster operator to reduce the algorithm computational burden and a parameter named open probability is introduced in mutation, which is dynamically changed with the increasing of iterations. The simulation results showed that SMOBSO would be a promising algorithm in solving multi-objective optimization problems [8]. In addition, a Modified Multiobjective Brain Storm Optimization algorithm (MMBSO) is proposed to improve the usefulness and effectiveness of algorithm. In the algorithm, the clustering strategy acts directly in the objective space instead of in the solution space and potential Pareto-dominance areas in the next iteration are suggested [29].

3 Test Task Scheduling Problem Test task scheduling problem (TTSP) is one typical Job-based scheduling problem. This kind of scheduling problems represent one important branch of scheduling, including flexible job shop scheduling problem (FJSP) [20, 25], test task scheduling problem (TTSP), parallel machine scheduling problem (PMSP) [26] and so on. This type of scheduling problems consists of a series of sequential or parallel execution tasks (considering the appellation is different in different areas, the scheduling objects are collectively referred to as tasks). Tasks are mutually independent or have local priorities. The goal of scheduling is to assign all tasks to a number of independent resources in a reasonable sequence and manner. It is similar to FJSP and PMSP. However, TTSP has some special characteristics. In TTSP, each task may have multiple options to choose, and each task may be tested on multiple instruments at the same time. The mathematical programming model of TTSP is as follows.

3.1 The Mathematical Programming Model for TTSP 3.1.1

Basic Concept

The TTSP is to organize the execution of n tasks on m instruments. In this problem, m . The notifithere are a set of tasks T = {t j }nj=1 and a set of instruments R = {ri }i=1 i i i cations P j , S j , and C j present the test time, the test start time, and the test completion time of task t j tested on ri , respectively. Thereupon, an inequality C ij ≥ S ij + P ji is obtained. For TTSP, one task must be tested on one or more instruments. In other words, some instruments collaborate on one test task. A variable O ij is defined to express whether the task t j occupies the instrument ri . The definition is as follows [14]:  1 if t j occupies ri i (1) Oj = 0 others

42

H. Lu et al.

In typical cases, some instruments could have redundancies. For example, there could be two instruments with the same basic functionality but different accuracies. If one task is tested on the instrument with lower accuracy, it can also be tested on the instrument with higher accuracy. Thus, task t j could have several alternative schemes kj is used to denote the alternative schemes of task to complete the test. W j = {wkj }k=1 t j , where k j is the number of schemes of t j . K = {k j }nj=1 is the set containing the numbers of schemes that correspond to every task. Each wkj is a subset of R and u jk can be represented as wkj = {r ujk }u=1 , and u jk is the number of instruments for wkj .  k Obviously, 1≤k≤k j w j = R. The notification P jk = max P ji is used to express the test time of

3.1.2

ri ∈wkj

1≤ j≤n t j for wkj .

Constraint Relationships

The TTSP has two types of constraints: the restriction of resources and the precedence constraint between the tasks. The restriction of resources can be expressed as follows:  X kk∗ j j∗

=

1 if wkj ∩ wk∗ j∗  = ∅ 0 others

(2)

The precedence constraint between the tasks can be represented as follows: ⎧ ⎨ 0 if t j and t j∗ have equal priorities Y j j∗ = +d if t j needs to be tested before t j∗ with at least d unit time,i.e., t j t j∗ ⎩ −d if t j∗ t j with at least d unit time

(3)

where d ∈ R + . In this chapter, d equals the test time of the task with high priority. In addition, hypotheses considered in this chapter are the following: 1. At a given time, an instrument can execute only one task. 2. Preemption is not allowed. In other words, each task must be completed without interruption once it starts. 3. All of the tasks and instruments are available at time 0. 4. The setup time of the instruments and the move time between the tasks can be negligible. In other words, C ij = S ij + P ji . 5. Assume P ji = P jk , to simplify the problem. 3.1.3

Fitness Function

There are two objectives to be minimized: the maximal test completion time (makespan) f 1 (x) and the mean workload of the instruments f 2 (x):

Brain Storm Optimization for Test Task Scheduling Problem Table 1 A TTSP with four tasks and four instruments T Wj wkj P jk T t1 t2

w11 w12 w21 w22

r1 r2 r2 r4 r1 r3

5 3 4 1

t3 t4

43

Wj

wkj

P jk

w31 w41 w42 w43

r4 r1 r3 r2 r4 r2 r3

2 4 3 7

1. The maximal test completion time f 1 (x) The notification C kj = maxri ∈wkj C ij is the test completion time of t j for wkj . Thus, the maximal test completion time of all of the tasks can be defined as follows: f 1 (x) = max C kj 1≤k≤k j 1≤ j≤n

(4)

2. The mean workload of the instruments f 2 (x) First, a new notification Q is introduced to denote the parallel steps. The initial value of Q is 1. Assign the instruments for all of the tasks, if X kk∗ j j∗ = 1, Q = Q + 1. Therefore, the mean workload of the instruments can be defined as follows: f 2 (x) =

n m 1  i i P O Q j=1 i=1 j j

(5)

A TTSP with four tasks and four instruments is illustrated in Table 1.

3.2 Characteristics Analysis In order to solve a problem better, we should discuss the problem characteristics into consideration fully. A small scale TTSP instance with six tasks and eight instruments is shown in Table 2. This small example is easy to solve because its decision space can be traversed. Fig. 1 presents the statistical results of the two objectives of the instance in Table 2. In Fig. 1, the x-axis of left sub-figure is makespan, and the x-axis of right sub-figure is the mean workload of the instruments respectively. The y-axis of each sub-figure represents the number of solutions corresponding to x-axis in all the exhaustive solutions. From Fig. 1, it can be seen that solutions in the single object space distribute more in the middle part of fitness values. The proportion of the optimal solutions is particularly small and successfully searching optimal solution in huge space is extremely hard as a consequence. Once the scale becomes larger, the solution space will increase sharply. The search space experiences exponential growth as the number

44

H. Lu et al.

Table 2 A TTSP instance with six tasks and eight instruments T Wj wkj P jk T Wj t1

t2

w11 w12 w13 w21 w22 w23 w24

r1 r4 r3 r5 r6 r8 r2 r4 r6 r7

2 5 3 3 4 3 4

t3 t4 t5 t6

w31 w32 w41 w42 w51 w61 w62 w63

wkj

P jk

r3 r5 r4 r8 r7 r1 r4 r3 r7 r6 r8

5 5 22 20 23 7 8 8

Fig. 1 The statistical analysis of single objective for TTSP

of tasks and options increases. Therefore, it is hard to find optima from the huge solution space. Multi-objective TTSP also has its own characteristics. For the small scale instance in Table 2, it has 103,680 solutions in the decision space, but only hundreds of points in the objective space. Furthermore, there are only four points in the true Pareto front through the enumeration, as shown in Fig. 2. There are [(23, 70/3); (28, 19.5); (31, 18); (36, 14.6)] [14]. Therefore, TTSP has many local optima. The distribution of multi-objective TTSP in the objective space also demonstrates a certain pattern, similar to the single objective case. The closer the current Pareto front is to the true Pareto front, the fewer solutions are in the objective space. As mentioned above, multiple solutions in the decision space correspond to a point in the objective space as shown in Fig. 3. This characteristic is represented by uppercase characters MTO, which means that multiple solutions correspond to one solution. Therefore, the size of true Pareto set in the objective space is very small.

Brain Storm Optimization for Test Task Scheduling Problem

45

Fig. 2 The solution space for two-objective TTSP

Fig. 3 The MTO characteristic of TTSP

3.3 Encoding Strategy 3.3.1

Encoding Background

Utilising an appropriate representation in a chromosome is very important for solving search problems. TTSP is a combination of instrument assignment and task scheduling decisions. Thus, a chromosome should include these two types of information. Figure 4 illustrates the commonly used encoding schemes for TTSP and other scheduling problems [14]. The machine/instrument and the operation/task are expressed as a and t, respectively.

46

H. Lu et al.

Fig. 4 Some commonly used encoding schemes

The TSL represents the schedule in a list of operations, and each operation holds one cell. Every cell contains the operation number, the job number, and the machine number. For example, the cell (2, 1, 4) means operation 2 of job 1 is performed on machine 4. This encoding scheme is directly perceived, but it contains too much information in each cell, which causes complex genetic manipulations and a lot of unfeasible solutions. The two-vector representation cuts a chromosome into two parts: the machine assignment vector and the operation sequence vector. It can avoid unfeasible solutions. However, because the two vectors are divided and each of them contains different types of information, the crossover operators and the mutation operators of these two vectors are usually different and should be considered separately. Thus, the genetic manipulations are inconvenient. The matrix encoding scheme was proposed by [27] to solve the TTSP. The column vectors of the matrix chromosome present the instruments, while the row vectors express the tasks. For every instrument, if one task uses it, the task number is filled in the row vector of the instrument. Obviously, it could not solve the TTSP with multiple alternative schemes, because it contains no information about instruments (with no information above a). Meanwhile, it will generate unfeasible solutions during the genetic manipulations.

3.3.2

Integrated Encoding Scheme (IES)

As mentioned above, a more suitable encoding approach for the TTSP is necessary. For this reason, a new encoding scheme, called the integrated encoding scheme (IES) [14], is proposed, which is inspired by random key (RK) encoding. This encoding scheme can use only one chromosome to contain the information about both the processing sequence of the tasks and the occupancy of the instruments for each task. Thus, the encoding efficiency is improved and the subsequent genetic manipulations

Brain Storm Optimization for Test Task Scheduling Problem Table 3 Example of the integrated encoding scheme Decision 0.8147 0.9058 variables xi Task sequence t j 3 4 k 1 1 wkj {r4 } {r1 , r3 } p kj

2

4

47

0.1270

0.6324

1 2 {r2 , r4 }

2 1 {r1 }

3

4

need not be performed separately. Meanwhile, this encoding scheme transforms a combinatorial optimisation problem to a continuous optimisation problem, which can make the search algorithm suitable for the TTSP. Assume that the TTSP to be encoded is as shown in Table 3. The main concept of the IES is to use relationships between the decision variables to express the sequence of tasks, and the values of the variables to represent the occupancy of the instruments for each task. This concept is illustrated in Table 3. The entries in the first row of Table 3 are the decision variables, which range between 0 and 1. They are sorted in ascending order, and the rank of every variable denotes a test task index in sequence. Thus, the second row, the task sequence, is obtained. More specifically, the smallest variable is 0.1270, which represents t1 . The second smallest variable is 0.6324, which represents t2 , and so on. On the other hand, the instrument assignment can also be obtained from the decision variables. If we want to know which instruments will be occupied by the task t j , wkj should be ascertained. In other words, we should know the value of k, which can be calculated by the decision variable corresponding to t j , as follows: k = [xi j × 10]

mod k j + 1

(6)

where xi j represents the decision variable that corresponds to t j and k j is the number of schemes of t j . Square brackets are used to obtain the nearest integer, which is not more than xi j × 10. For example, for the task t4 , its corresponding decision variable is 0.9058 and its number of schemes is k4 = 3. Then the value of k can be calculated as follows according to the formula above: k = [0.9058 × 10] mod k4 + 1 = 9 mod 3 + 1 = 1. Thus, w41 = {r1 , r3 } is applied. Obviously, n decision variables should be used to encode n tasks. In other words, the decision space has n dimensions. From the truth above, some advantages of the IES can be summarised: 1. It never generates a duplication of a certain task. 2. It makes full use of decision variables by using only one chromosome to merge the information about both the instrument assignment and the task scheduling decisions and improves the encoding efficiency. 3. It reduces the complexity of the genetic manipulations. 4. It never generates unfeasible solutions that do not satisfy the resource constraint. 5. It can solve the TTSP with multiple alternative schemes to complete the test.

48

H. Lu et al.

It can transform a combinatorial optimisation problem to a continuous optimisation problem, which makes the search algorithms like BSO suitable for the TTSP.

4 BSO Based Test Task Scheduling Algorithm 4.1 BSO Based Single Objective Test Task Scheduling Algorithm The brain storm optimization (BSO) algorithm is proposed in 2011 [21, 22], which is a young and promising algorithm in swarm intelligence. The original BSO algorithm is simple in concept and easy in implementation. The main procedure is given in Algorithm 1 [3]. There are three strategies in this algorithm: the solution clustering, new individual generation, and selection.

Algorithm 1: Brain Storm Optimization (BSO) 1 2 3 4 5 6

Initialization: Randomly generate n potential solutions (individuals), and evaluate the n individuals; while not find “good enough” solution or not reach the pre-determined maximum number of iterations do Clustering: Cluster n individuals into m clusters by clustering algorithm; New individual generation: randomly select one or two cluster(s) to generate new individual; Selection: The newly generated individual is compared with the existing individual with the same individual index; the better one is kept and recorded as the new individual; Evaluate the n individuals;

In a BSO algorithm, an individual represents an idea, which in turn represents a potential solution to the problem to be solved. Similar to other swarm intelligence algorithms, uniformly randomized initialization is preferred especially when no prior knowledge is known about the problem to be solved. In the main loop of the algorithm, the population of individuals will be evaluated similar to what is done in other swarm intelligence algorithms according to a known evaluation function. The population of individuals will then be clustered into several clusters with the best individual in each cluster mimicking a better idea picked up by a problem owner. The clustering algorithm utilized in the original BSO is the k-means clustering algorithm, which is a popular and simple clustering algorithm. The operation “disrupting a cluster center” is utilized to increase the population diversity which is executed usually with a small probability and in which a randomly selected cluster center is replaced by a randomly generated individual. The operation “updating individuals” is the operation that actually generates new individuals which are expected to move towards better and better search areas iteration over iteration.

Brain Storm Optimization for Test Task Scheduling Problem

49

In the original BSO, a new individual is generated by adding Gaussian random values to a selected individual according to the following formula, i i (t + 1) = xselected (t) + ξ(t) × random() xnew

(7)

i represents the ith dimension of the newly generated individual, and where xnew i xselected represents the ith dimension of the selected individual based on the current population of individuals, which can be obtained according to the following formula,

 i xselected (t)

=

rand() ×

i xold1 (t)

i (t) xold1 i + (1 − rand()) × xold2 (t)

(8)

where xold1 and xold2 represent two individuals selected from the current population of individuals. rand() returns a random value within the range (0, 1). The first part in Eq. (8) is used to obtain the selected individual based on one individual from the current population of individuals, and second part in Eq. (8) is used to obtain the selected individual based on two individuals from the current population of individuals. In Eq. (7), random() is a random function which is the Gaussian random function in the original BSO. The coefficient ξ(t) is the step size which weights the contribution of the Gaussian random value to the new generated value. In the original BSO, the ξ(t) is obtained according to the following formula. ξ(t) = log sig(

T 2

−t ) × rand() k

(9)

where log sig() is a logarithmic sigmoid transfer function, T is the maximum number of iterations, and t is the current iteration number, k is for changing log sig() function’s slope, and rand() returns a random value within (0, 1). In BSO, three probability parameters are designed to select an individual under different cases. They are Pgeneration , PoneCluster and PtwoCluster . Pgeneration is used to determine a new individual being generated by one or two “old” individuals. PoneCluster is used to determine the cluster center or another normal individual will be chosen in one cluster generation case and PtwoCluster is used to determine the cluster center or another normal individual will be chosen in two clusters generation case. The process of selecting individuals to generate a new individual is given in Algorithm 2 as follows. The brain storm optimization algorithm is a kind of search space reduction algorithm. All solutions will get into several clusters eventually. These clusters indicate a problem’s local optima. The information of an area contains solutions with good fitness values are propagated from one cluster to another. This algorithm will explore in decision space at first, and the exploration and exploitation will get into a state of equilibrium after iterations.

50

H. Lu et al.

Algorithm 2: New individual generation 1 2 3 4 5 6 7

8 9 10 11 12

13 14

Randomly generate a value rgeneration in the range [0, 1); if the value rgeneration is less than a probability Pgeneration then Randomly select a cluster, and generate a random value roneCluster in the range [0, 1); if the value roneCluster is smaller than a pre-determined probability PoneCluster then Select the cluster center and add random values to it to generate new individual; else Randomly select a normal individual from this cluster and add random value to the individual to generate new individual; else Randomly select two clusters to generate a new individual; Generate a random value rtwoCluster in the range [0, 1); if the value rtwoCluster is less than a pre-determined probability PtwoCluster then The two cluster centers are combined and then added with random values to generate a new individual; else Two normal individuals from each selected cluster are randomly selected to be combined and added with random values to generate a new individual;

4.2 BSO Based Multi-objective Test Task Scheduling Algorithm The brain storm optimization algorithm can also be extended to solve multi-objective optimization problems. Unlike the traditional multi-objective optimization methods, the brain storm optimization algorithm utilized the solution space information directly [8]. Clusters are generated in the solution space, and for each objective, individuals are clustered in each iteration or clustered in the solution space. The individual, which performs better in most of objectives are kept to the next iteration, and other individuals are randomly selected to keep the diversity of solutions. The primary process of multi-objective BSO (MOBSO) is shown in Algorithm 3. In the framework of fast non-dominated sorting genetic algorithm II (NSGA-II) [6], the generation of new individuals is based on BSO. Comparing to BSO mentioned above, there are two main differences. First, the best individual in each cluster is evaluated according to the fitness for single objective BSO. However, it does not apply to multi-objective problems. Therefore, the non-dominated solutions in each cluster are selected as the best set. If the centers need to be used, the random individual in the best set is selected. Then, the individuals update is not as same as that in single objective BSO. The newly generated individual and the existing individual with the same individual index possibly dominate each other. Therefore, the remaining population are selected from the newly generated population and the current population together. The rules of preserving individuals are as same as that of NSGA-II, including the rank and crowded distance.

Brain Storm Optimization for Test Task Scheduling Problem

51

Algorithm 3: The basic process of multi-objective brain storm optimization (MOBSO)

6 7 8

Initialization: Randomly generate n potential solutions (individuals), and evaluate the n individuals; while have not found “good enough” solution or not reached the pre-determined maximum number of iterations do Clustering: Cluster n individuals into m clusters by clustering algorithm; Rank: rank individuals in each cluster; New individuals’ generation: randomly select one or two cluster(s) to generate a new individual; Make a set Pt of all new individuals;; Rank and selection: together the set Pt and its parent set Q t−1 ; Rank all individuals in the set and select remaining population Q t ;

9

Output: the non-dominated individuals ;

1 2 3 4 5

There are several differences between multi-objective BSO and single objective BSO. Because the different criteria for the evaluation of solutions, the center of a cluster is the first rank individuals instead of the best solution. Therefore, a problem owner picks up more than one idea a time. In addition, the selection strategy of comparing to corresponding index is no longer applicable to multi-objective optimization. From the perspective of population, selecting the higher rank individuals helps to improve the speed of coverage and maintain the diversity of population. The proposed MOBSO inherits the advantages of BSO and NSGA-II.

5 Experimental Study The experiments for studying the effect of BSO on TTSP consist of two parts. In the first part, we compare BSO with traditional genetic algorithm (GA) and particle swarm algorithm (PSO) for single objective TTSP. Subsequently, multi-objective BSO for solving TTSP is compared with the fast non-dominated sorting genetic algorithm (NSGA-II) [6] and the multi-objective evolutionary algorithm based on decomposition (MOEA/D) [30]. In all cases, the best performances are denoted in bold. There are four experimental instances, three of which are based on a real-world TTSP taken from a missile system adopted in this section. A small-scale instance with 20 tasks and 8 instruments [14], a middle-scale instance with 30 tasks and 12 instruments, and a large-scale instance with 40 tasks and 12 instruments [15] are adopted to measure the performance. Another larger scale TTSP instance with 50 tasks and 15 instruments is randomly generated in which the processing time of each task on an instrument is uniformly distributed between 0 and 20. To make the expression more convenient and clear, the abbreviation m × n instance is used to represent the TTSP instance with m tasks and n resources.

52

H. Lu et al.

Table 4 Parameters setting for single objective algorithms Instance Population Generations Pc , Pm (GA) 20 × 8 30 × 12 40 × 12 50 × 15

70 100 100 100

120 250 250 250

0.9, 0.1 0.9, 0.1 0.9, 0.1 0.9, 0.1

Table 5 Parameters setting for multi-objective algorithms Instance Population Generations Pc , Pm (NSGA-II) 20 × 8 30 × 12 40 × 12 50 × 15

70 100 100 100

120 250 250 250

0.9, 0.1 0.9, 0.1 0.9, 0.1 0.9, 0.1

c1 , c2 (PSO)

Nc (BSO)

2, 2 2, 2 2, 2 2, 2

3 5 5 5

Nc (MOBSO) N , T (MOEA/D) 3 5 5 5

70, 30 100, 30 100, 30 100, 30

All the experiments are simulated on a computer running Windows 7 with an Intel (R) Core (TM) i5-6500, 3.20 GHz, and 8.00 GB RAM.

5.1 Parameters Setting The basic parameters setting for all experiments are shown in Tables 4 and 5. For single objective TTSP, the GA used here adopts the tournament selection and simulated binary crossover (SBX). Among Table 4, Pc is the abbreviation of crossover probability and Pm is the abbreviation of mutation probability. For PSO algorithm, c1 and c2 are the learning factors. Among Table 5, Pc and Pm in NSGA-II are as the same as that in GA. For MOEA/D algorithm, N represents the number of the subproblems and T means the number of the weight vectors in the neighborhood of each weight vector. The abbreviation Nc is used to represent the number of clusters.

5.2 Experiment for Single Objective TTSP According to the basic parameters setting in Table 4, each algorithm runs 50 times to compare their performance on solving TTSP. The objective is makespan. The results are shown in Table 6. Mean represents the average makespan of 50 runs and var is the abbreviation of variance. From the Table 6, it can be seen that BSO is better than GA and PSO from the perspective of the average makeapan. In addition, this superiority is more evident when the scale of problems is larger. The degree of dispersion of

Brain Storm Optimization for Test Task Scheduling Problem Table 6 Comparison of BSO with other algorithms for TTSP Instance Makespan BSO GA 20 × 8 30 × 12 40 × 12 50 × 15

Mean Var Mean Var Mean Var Mean Var

32.6 0.6939 35.5 1.6020 44.8400 2.2188 61.3600 7.1331

33 0.6531 39.6 1.1020 48.5800 1.0241 70.9200 3.7894

53

PSO 33.6 1.0861 39.4 1.1106 50.1800 2.1914 69.8000 5.5918

BSO is slightly greater than that of GA. The performance of BSO is pretty good comparing to GA and PSO from the comprehensive consideration. Each run of an algorithm can produce a convergence curve, which means the best fitness found with the increasing of iteration. Figure 5 shows the average convergence curve of 50 runs for the three algorithms. Although the performance of the three algorithms is similar at the early stage of search, BSO shows better search ability later in the search process, as shown in Fig. 5. It indicates that BSO fully considers the information in the search process. When solutions in the objective space trend to convergence, more diverse solutions are generated by disturbance and information from the different clusters. Furthermore, the probability parameters keep the balance of exploration and exploitation of the search process. In general, a search algorithm focuses on exploration in the early stage and focuses on exploitation in the later period. However, BSO keeps the balance in the two aspects. In the later stage, the algorithm still has the ability to explore solution space. Furthermore, the competitive performance of BSO in the later stage of search process has a great relationship with the step size. The step size controls the generation of new solutions. It has a trend of declining overall along with randomness. It is more suitable for solving TTSP because there are key paths in combinatorial problem generally. If the better structure of a solution is reserved, the possibility of finding better solutions will be increased. Therefore, the convergence curves present the similar characteristic of converging in the late of search process.

5.3 Experiment for Multi-objective TTSP Multi-objective optimization problems (MOPs) arise numerously in the real world including the engineering fields and have become a hot spot in the research. To solve various problems, a lot of multi-objective evolutionary algorithms (MOEAs) have been proposed. Among them, a fast non-dominated sorting genetic algorithm II (NSGA-II) and a multi-objective evolutionary algorithm based on decomposition

54

H. Lu et al.

Fig. 5 The convergence curve of BSO, GA and PSO on solving different scale TTSP instances

(MOEA/D) are well-known and representative MOEAs and they have been widely studied and used. We compare the performance of the MOBSO with the two MOEAs for solving TTSP.

5.3.1

Performance Metrics

There have been many performance metrics proposed to measure the performance of multi-objective optimization algorithms. Among them, the comprehensive indicator of hypervolume (HV) [17] is an effective tool for evaluating the performance of MOEAs. It reflects the convergence and the distribution of Pareto front comprehensively. In addition, the greatest advantage of it is that it doesn’t need the information of the real Pareto front. It is noted that the results of HV metric in the following

Brain Storm Optimization for Test Task Scheduling Problem

55

Table 7 Comparison of MOBSO with other algorithms using HV metric for TTSP Instance HV MOBSO NSGA-II MOEA/D 20 × 8 30 × 12 40 × 12 50 × 15

Mean Var Mean Var Mean Var Mean Var

2.8363 4.0820 3.1837 7.5535 3.6401 7.9259 4.6140 13.9618

2.8135 4.1715 2.9621 11.1292 3.3072 25.6215 4.2870 29.1412

2.5790 24.0341 2.6348 42.3995 3.1174 38.6654 3.7250 74.9535

tables are the one thousandth of the original values because the magnitude of the HV values is large.

5.3.2

The Comparison of MOBSO with NSGA-II and MOEA/D

In order to observe the performance of BSO more intuitively, MOBSO is compared to NSGA-II and MOEA/D. Likewise, each experiment runs 50 times and var is the abbreviation of variance. The results can be seen in Table 7 and Fig. 6. Table 7 shows the average HV metrics and the variance of results and Fig. 6 presents the boxplot of the HV results of three algorithms on different TTSP instance. As seen from Table 7 and Fig. 6, the performance of MOBSO is superior to NSGA-II and MOEA/D from the point of view of the average makespan and the dispersion degree of results. It means that the Pareto front obtained from MOBSO is more convergent than that from NSGA-II and MOEA/D. In addition, MOBSO also performs better in larger scale TTSP instance. Furthermore, the Pareto fronts of the three multi-objective optimization algorithms are shown in Fig. 7. From Fig. 7, MOBSO has better performance than the other two algorithms in solving TTSP. It shows the brain storm optimization algorithm utilizes the solution space information more effectively. According to HV indicator, MOBSO has better performance in convergence and the spacing of Pareto front. Good performance may be determined by a combination of the following reasons. First, BSO ensures the exploration and exploitation of the search area, especially in the late stage of the algorithm. The generation of new solutions depends on various methods to increase the probability of finding the optimal solution. In particular, the way in which a new solution is generated based only on a center or a solution in a cluster is more suitable for solving TTSP. If a solution is generated through more than one solution, the structure of the solution would be completely destroyed. Second, the cluster strategy in BSO is fit for the multi-objective optimization. Multiple centers could speed up the convergence of the algorithm. Third, the framework of NSGA-II maintains the diversity of Pareto front. Therefore, the hybrid MOBSO not only fully

56

H. Lu et al.

Fig. 6 The boxplot of the comparison of MOBSO with other algorithms using HV metric for TTSP

utilizes the advantages of each algorithm, but also takes into account the features of TTSP. From the above experiments, it reveals that BSO has competitive performance on solving TTSP comparing to other classic algorithms from the point of view of single objective and multi-objective algorithms. All experiments tell us that BSO performs better than other mentioned algorithms and better in larger scale instances. It indicates that BSO utilizes the information of solution space more effectively and promote the further exploration and exploitation of the solution space.

6 Conclusion In this chapter, a brain storm optimization algorithm is applied to solve the automatic test task scheduling problem (TTSP). TTSP is a complex practical problem and it is hard to solve. The main characteristics of it include its huge solution space and relatively few optima. The brain storm optimization algorithm has its advantage in using the information of solution space through the clustering method. In addition, the con-

Brain Storm Optimization for Test Task Scheduling Problem

Fig. 7 The comparison of Pareto front for MOBSO with other algorithms

57

58

H. Lu et al.

vergent operation and divergent operation in the brain storm optimization algorithm balance the search process of exploration and exploitation. From the experiments of solving single objective and multi-objective TTSP, the brain storm optimization algorithm performs better than the other algorithms mentioned above. In the further work, the brain storm optimization algorithm would be further researched from the parameter tuning and the close relation with problems to better solve TTSP. Acknowledgements This research is supported by the National Natural Science Foundation of China under Grant No. 61671041, 61101153, 61806119, 61773119, 61703256, and 61771297; in part by the Shenzhen Science and Technology Innovation Committee under grant number ZDSYS201703031748284; and in part by the Fundamental Research Funds for the Central Universities under Grant GK201703062.

References 1. Arsuaga-Ríos, M., Vega-Rodríguez, M.A.: Cost optimization based on brain storming for grid scheduling. In: Proceedings of the 2014 Fourth International Conference on Innovative Computing Technology (INTECH), pp. 31–36. Luton, UK (Aug 2014) 2. Chen, W.H., Wu, P.H., Lin, Y.L.: Performance optimization of thermoelectric generators designed by multi-objective genetic algorithm. Appl. Energy 209, 211–223 (2018) 3. Cheng, S., Qin, Q., Chen, J., Shi, Y.: Brain storm optimization algorithm: a review. Artif. Intell. Rev. 46(4), 445–458 (2016) 4. Cheng, S., Shi, Y., Qin, Q., Zhang, Q., Bai, R.: Population diversity maintenance in brain storm optimization algorithm. J. Artif. Intell. Soft Comput. Res. (JAISCR) 4(2), 83–97 (2014) 5. Cheng, S., Sun, Y., Chen, J., Qin, Q., Chu, X., Lei, X., Shi, Y.: A comprehensive survey of brain storm optimization algorithms. In: Proceedings of 2017 IEEE Congress on Evolutionary Computation (CEC 2017), pp. 1637–1644. IEEE, Donostia, San Sebastián, Spain (2017) 6. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evolut. Comput. 6(2), 182–197 (April 2002) 7. Guo, X., Wu, Y., Xie, L.: Modified brain storm optimization algorithm for multimodal optimization. In: Tan, Y., Shi, Y., Coello, C.A.C. (eds.) Advances in Swarm Intelligence, Lecture Notes in Computer Science, vol. 8795, pp. 340–351. Springer International Publishing (2014) 8. Guo, X., Wu, Y., Xie, L., Cheng, S., Xin, J.: An adaptive brain storm optimization algorithm for multiobjective optimization problems. In: Tan, Y., Shi, Y., Buarque, F., Gelbukh, A., Das, S., Engelbrecht, A. (eds.) Advances in Swarm and Computational Intelligence, (International Conference on Swarm Intelligence, ICSI 2015), Lecture Notes in Computer Science, vol. 9140, pp. 365–372. Springer International Publishing (2015) 9. Jadhav, H., Sharma, U., Patel, J., Roy, R.: Brain storm optimization algorithm based economic dispatch considering wind power. In: Proceedings of the 2012 IEEE International Conference on Power and Energy (PECon 2012), pp. 588–593. Kota Kinabalu, Malaysia (Dec 2012) 10. Kang, L., Wu, Y., Wang, X., Feng, X.: Brain storming optimization algorithm for heating dispatch scheduling of thermal power plant. In: Proceedings of 2017 29th Chinese Control And Decision Conference (CCDC 2017), pp. 4704–4709 (May 2017) 11. Krishnanand, K., Hasani, S.M.F., Panigrahi, B.K., Panda, S.K.: Optimal power flow solution using self-evolving brain-storming inclusive teaching-learning-based algorithm. In: Tan, Y., Shi, Y., Mo, H. (eds.) Advances in Swarm Intelligence. Lecture Notes in Computer Science, vol. 7928, pp. 338–345. Springer, Berlin/Heidelberg (2013) 12. Li, L., Zhang, F.F., Chu, X., Niu, B.: Modified brain storm optimization algorithms based on topology structures. In: Proceedings of 7th International Conference on Swarm Intelligence (ICSI 2016), pp. 408–415. Springer International Publishing, Bali, Indonesia (2016)

Brain Storm Optimization for Test Task Scheduling Problem

59

13. Lu, H., Chen, X., Liu, J.: Parallel test task scheduling with constraints based on hybrid particle swarm optimization and Tabu search. Chin. J. Electron. 21(4), 615–618 (Oct 2012) 14. Lu, H., Niu, R., Liu, J., Zhu, Z.: A chaotic non-dominated sorting genetic algorithm for the multi-objective automatic test task scheduling problem. Appl. Soft Comput. 13(5), 2790–2802 (2013) 15. Lu, H., Yin, L., Wang, X., Zhang, M., Mao, K.: Chaotic multiobjective evolutionary algorithm based on decomposition for test task scheduling problem. Math. Probl. Eng. 1–25 (2014) 16. Lu, H., Zhu, Z., Wang, X., Yin, L.: A variable neighborhood MOEA/D for multiobjective test task scheduling problem. Math. Probl. Eng. 1–14 (2014) 17. Luo, J., Yang, Y., Li, X., Liu, Q., Chen, M., Gao, K.: A decomposition-based multi-objective evolutionary algorithm with quality indicator. Swarm Evolut. Comput. (2017) 18. Qiu, H., Duan, H., Zhou, Z., Hu, X., Shi, Y.: Chaotic predator-prey brain storm optimization for continuous optimization problems. In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–7. Honolulu, HI, USA (Nov 2017) 19. R˘adulescu, A., Nicolescu, C., van Gemund, A.J., Jonker, P.P.: CPR: mixed task and data parallel scheduling for distributed systems. In: Proceedings 15th International Parallel and Distributed Processing Symposium. IPDPS 2001, pp. 1–9 (April 2001) 20. Shen, L., Dauzère-Pérès, S., Neufeld, J.S.: Solving the flexible job shop scheduling problem with sequence-dependent setup times. Euro. J. Op. Res. 265(2), 503–516 (2018) 21. Shi, Y.: Brain storm optimization algorithm. In: Tan, Y., Shi, Y., Chai, Y., Wang, G. (eds.) Advances in Swarm Intelligence. Lecture Notes in Computer Science, vol. 6728, pp. 303–309. Springer, Berlin/Heidelberg (2011) 22. Shi, Y.: An optimization algorithm based on brainstorming process. Int. J. Swarm Intell. Res. (IJSIR) 2(4), 35–62 (2011) 23. Shi, Y.: Brain storm optimization algorithm in objective space. In: Proceedings of 2015 IEEE Congress on Evolutionary Computation, (CEC 2015), pp. 1227–1234. IEEE, Sendai, Japan (2015) 24. Srinivas, N., Deb, K.: Muiltiobjective optimization using nondominated sorting in genetic algorithms. Evolut. Comput. 2(3), 221–248 (Sept 1994) 25. Teekeng, W., Thammano, A.: Modified genetic algorithm for flexible job-shop scheduling problems. Proc. Comput. Sci. 12, 122–128 (2012) 26. Vallada, E., Ruiz, R.: A genetic algorithm for the unrelated parallel machine scheduling problem with sequence dependent setup times. Euro. J. Op. Res. 211(3), 612–622 (2011) 27. Xia, R., Xiao, M.Q., Cheng, J.J.: Parallel TPS design and application based on software architecture, components and patterns. In: 2007 IEEE Autotestcon, pp. 234–240 (Sept 2007) 28. Xia, R., Xiao, M., Cheng, J., Xinhua, F.: Optimizing the multi-UUT parallel test task scheduling based on multi-objective GASA. In: 2007 8th International Conference on Electronic Measurement and Instruments, pp. 839–844 (Aug 2007) 29. Xie, L., Wu, Y.: A modified multi-objective optimization based on brain storm optimization algorithm. In: Proceedings of 5th International Conference on Swarm Intelligence (ICSI 2014),. pp. 328–339. Springer International Publishing, Hefei, China (2014) 30. Zhang, Q., Li, H.: MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans. Evolut. Comput. 11(6), 712–731 (Dec 2007) 31. Zhang, R., Chang, P.C., Song, S., Wu, C.: Local search enhanced multi-objective pso algorithm for scheduling textile production processes with environmental considerations. Appl. Soft Comput. 61, 447–467 (2017) 32. Zhou, D., Qi, P., Liu, T.: An optimizing algorithm for resources allocation in parallel test. In: 2009 IEEE International Conference on Control and Automation, pp. 1997–2002 (Dec 2009)

Oppositional Brain Storm Optimization for Fault Section Location in Distribution Networks Guojiang Xiong, Jing Zhang, Dongyuan Shi and Yu He

Abstract Fault section location (FSL) is an important role in facilitating quick repair and restoration of distribution networks. In this chapter, an oppositional brain storm optimization referred to as OBSO is proposed to effectively solve the FSL problem. The FSL problem is transformed into a 0–1 integer programming problem. The difference between the reported overcurrent and expected overcurrent states of the feeder terminal units (FTUs) is used as the objective function. BSO has been shown to be competitive to other population-based algorithms. But its convergence speed is relatively slow. In OBSO, opposition-based learning method is utilized for population initialization and also for generation jumping to accelerate the convergence rate. The effectiveness of OBSO is comprehensively evaluated on different fault scenarios including single and multiple faults with lost and/or distorted fault signals. The experimental results show that OBSO is able to achieve more promising performance. Keywords Brain storm optimization · Fault section location · Distribution network · Opposition-based learning

1 Introduction Fault section location (FSL) is an important application among intelligent monitoring and outage management tasks used for realization of self-healing networks [1]. During the operation of a distribution network, fault is inevitable. When a fault event occurs, the well-installed feeder terminal units (FTUs) will acquire overcur-

G. Xiong (B) · J. Zhang · Y. He Guizhou Key Laboratory of Intelligent Technology in Power System, College of Electrical Engineering, Guizhou University, Guiyang 550025, China e-mail: [email protected] D. Shi State Key Laboratory of Advanced Electromagnetic Engineering and Technology, Huazhong University of Science and Technology, Wuhan 430074, China © Springer Nature Switzerland AG 2019 S. Cheng and Y. Shi (eds.), Brain Storm Optimization Algorithms, Adaptation, Learning, and Optimization 23, https://doi.org/10.1007/978-3-030-15070-9_3

61

62

G. Xiong et al.

rent information and then send them to the energy management systems (EMS). The overcurrent information can be used to identify the faulted section or sections. In order to solve the FSL problem effectively, many location approaches have been proposed. These approaches can be roughly divided into two categories: matrixbased approach [2–5] and analytic location approach. The matrix-based approach is also called direct approach which employs a matrix to map the network topology and then identifies the faulted section or sections by using matrix operation. The merit of this approach is simple and easy to implementation. However, its location accuracy is highly dependent on the correctness of fault information acquired by the FTUs. In practice, the fault information has the uncertainties of loss and distortion, which thereby makes this approach present poor fault tolerance. The analytic location approach is methodologically different from the matrix-based approach. It firstly creatively formulates the FSL as a 0–1 integer programming problem and then utilizes optimization methods to solve the problem. This approach possesses some technological traits such as rigorous mathematical logic, strong theoretical foundation, and good fault tolerance. Among the proposed optimization methods, metaheuristic methods have attracted comparatively increasing interest recently due to that they have no strict requirements on the problem formulation and can avoid the influences of the initial condition sensitivity and gradient information [6]. Up to now, the successfully proposed optimization methods include genetic algorithm [7, 8], immune algorithm [9], particle swarm optimization [10], bat algorithm [11], firefly algorithm [12], sine cosine algorithm [13], electromagnetism-like mechanism algorithm [14], hybrid methods [15–17], etc. These methods have verified their own merits in solving the FSL, yet much remains to be explored with regard to consistently improving the location results with higher performance. Brain storm optimization (BSO), proposed by Shi [18, 19], is an efficient and versatile swarm intelligence algorithm. BSO is inspired by the human brainstorming process, in which a diverse group of people with different backgrounds and expertise gather together to come up with new ideas to solve a problem. In BSO, each population individual is represented as an idea and all ideas are clustered into several groups by k-means clustering algorithm in each iteration. Then the ideas are updated based on one or two ideas in clusters by neighborhood searching and combination. Two functionalities, i.e. capability learning and capacity developing [20] make BSO have a good equilibrium between the exploration and exploitation. This fascinating feature soon makes BSO attract many attentions and be applied in various research fields [21–30]. However, to the best of our knowledge, BSO has not been used to solve the FSL problem. This motivates the authors to apply BSO to FSL problem. The original BSO utilizes the k-means clustering algorithm to cluster the population. It is implemented recursively in each iteration and thus is time-consuming. To reduce the computational burden, an improved variant of BSO named BSO in objective space (BSO-OS) algorithm [31], is proposed. BSO-OS replaces the k-means clustering algorithm with a simple grouping method. This grouping method is just based on the individuals’ fitness values and can make the algorithm more efficient and easier to implement. However, similar to other population-based methods, BSO-OS also faces up to some challenges. One typical issue in point is that, its convergence

Oppositional Brain Storm Optimization for Fault Section …

63

speed is relatively slow, which makes it not able to achieve the target value within a given number of fitness evaluations. Opposition-based learning (OBL) [32] is a concept in computational intelligence used to accelerate the convergence rate of different optimization techniques. It has been proven to be an effective concept to enhance various optimization methods [33–39]. Although OBL has been widely used in other population-based methods, it has not been employed to enhance the performance of BSO as per authors’ knowledge. In such contexts, the OBL is integrated into BSOOS to accelerate the convergence rate in this chapter. The proposed method, referred to as OBSO, is comprehensively evaluated on six different fault scenarios including single and multiple faults with lost and/or distorted fault signals. The experimental results demonstrate that OBSO is able to achieve more promising performance compared with PSO and BSO. The remainder of this chapter is organized as follows. Section 2 briefly introduces the formulation of FSL. Section 3 succinctly reviews BSO-OS. The proposed method, OBSO, is fully elaborated in Sect. 4. In Sect. 5, the details of application of OBSO for FSL are presented. The experimental results and comparisons are given in Sect. 6. Finally, Sect. 7 is devoted to conclusions and future work.

2 Problem Formulation 2.1 Objective Function Mathematically, the FSL problem can be expressed as the following 0–1 integer programming problem: min F(S)  S = [s1 , s2 , . . . , sD ] ∈ Z D s.t. si = 0 or 1

(1)

where S = [s1 , s2 , . . . , sD ] is the decision variable vector of the problem and D is the number of candidate faulted sections. si denotes the fault state of the ith candidate faulted section (fault = 1, sound = 0). When a distribution network suffers from a fault event, the FTUs that installed at the circuit breakers and sectionalizing switches will monitor the fault currents. If the fault currents exceed the corresponding preset value, the FTUs will acquire them and then send them to the EMS. In this context, the basic principle of FSL is to take up a backward reasoning method to build a fault hypothesis, in which the overcurrent information can be explained as logically as possible. In other words, under the fault hypothesis, the actual state of fault current associated with the fault event should be highly in coincidence with the expected state. The objective function [13] is derived as follows:

64

G. Xiong et al.

F(S) =

m 

|Ii − Ii∗ (S)| + ω

i=1

D 

|si |

(2)

i=1

where m is the number of FTUs. Ii and Ii∗ (S) denote the actual and expected current states of the ith FTU, respectively. If the ith FTU detects the fault overcurrent, then Ii gets the value of 1, otherwise it is 0. w ∈ (0, 1) is a weight and it is set to be 0.8 in this chapter. The last item of (2) is used to avoid misdiagnose [40].

2.2 Switching Function It can be seen from (2) that the expected current state Ii∗ (S) which is also called a switching function is the key for construction of the objective function. For a radial distribution network, when a fault occurs in one section, those downstream sections that between the faulted section and the terminal of the corresponding feeder line will suffer from power failure and thus there is no current flowing through them. While for those upstream sections that between the faulted section and power source, they flow overcurrents which can be monitored by the corresponding FTUs. Therefore, it can be concluded that the expected current state Ii∗ (S) is equivalent to the “CONTINUOUS XOR” operation of the fault state of the downstream sections and is formulated as follows: Ii∗ (S)=



si,d

(3)

⊕

 where ⊕ represents the “CONTINUOUS XOR” operator meaning that if the elements contained in ⊕ are not all zero, the summation result is 1, otherwise the result is 0. si,d is the fault state of the dth downstream section of the ith section.

3 Brain Storm Optimization in Objective Space Brain storm optimization (BSO) is a simple yet promising swarm intelligence algorithm inspired by human being’s behavior of brainstorming. In the original BSO [18, 19], the k-means clustering algorithm is utilized to cluster the population into several groups. It is implemented recursively in each iteration and thus is time-consuming. To simplify the grouping process and improve the implementation efficiency, the BSO in objective space algorithm [31] was proposed. In which, the k-means clustering algorithm is replaced by a simple grouping method which is just based on the individuals’ fitness values. It makes the BSO in objective space algorithm more efficient and easier to implement. Therefore, the BSO in objective space algorithm

Oppositional Brain Storm Optimization for Fault Section …

65

is employed in this chapter. Conveniently, the BSO in objective space algorithm is referred to as BSO later in this chapter unless otherwise specified. In BSO, each population individual is called as an idea denoted as Xi = [xi1 , xi2 , . . . , xiD ], where i = 1, 2, . . . , N , N is the population size and D is the problem dimension. Each idea is a real-coded vector and is initialized as follows: Xi,d = ld + rand(0, 1) × (ud − ld )

(4)

where rand(0, 1) is a uniformly distributed random real number in (0, 1). ld and ud are the lower bound and upper bound of the dth dimension, respectively. BSO mainly consists of three operators: grouping operator, replacing operator, and creating operator. In the grouping operator, a simple grouping method just based on the fitness values is implemented to cluster the N ideas into two groups: the top perce percentage as elitists and the remaining as normals. In the replacing operator, one randomly selected dimension of each idea is occasionally replaced by a newly generated random value with a probability of prep . This operator, on one hand, can effectively improve the population diversity and thus enhance the exploration ability. On the other hand, it is able to control the randomness by properly setting the value of prep to keep the exploitation ability. In the creating operator, BSO firstly makes use of a probability pe to determine whether to generate a candidate idea Vi (i = 1, 2, . . . , N ) based on “elitists” or “normals”, and then employs a probability pone to determine whether based on one or two selected ideas. The generating strategy is as follows: Vi = Hi + ξi N (μ, σ )  Hi =

one idea Xi , wi Xi + (1 − wi )Xj , two ideas

(5) (6)

where i = 1, 2, . . . , N , j = 1, 2, . . . , N . N (μ, σ ) is the Gaussian random value with mean μ and variance σ . w is a weight factor within the range (0, 1). ξ is an adjusting factor which can be calculated as follows: ξ = logsig((gmax /2 − g)/K) × rand(0, 1)

(7)

where logsig(·) is a logarithmic sigmoid transfer function. g and gmax are the current number of iteration and the maximum number of iteration, respectively. K is for adjusting the logsig(·) function’s slope. After generating the candidate idea Vi , the parent idea Xi will be replaced by Vi if Vi has a better fitness value.

66

G. Xiong et al.

4 Oppositional Brain Storm Optimization 4.1 Opposition-Based Learning The main idea of OBL [32] is the simultaneous consideration of an estimate and its corresponding opposite estimate to achieve a better approximation for the current  candidate solution. Assume that x ∈ [a, b] is a real number, its opposite number x is defined as the mirror point of the solution from the center of the search space and can be expressed as follows: 

x =a+b−x

(8)

The above definition can be easily extended to higher dimensions. Let X = [x1 , x2 , . . . , xD ] be a candidate solution in D-dimensional search space and xi ∈ 

[ai , bi ], i = 1, 2, . . . , D. The opposite candidate solution of X is defined by X =    [x1 , x2 , . . . , xD ] as follows: 

xi = ai + bi − xi

(9)

4.2 Oppositional Brain Storm Optimization 4.2.1

Opposition-Based Population Initialization

In general, random values are generated to initialize the population for the lack of valuable a prior knowledge. Even so, fitter initial candidate solutions namely opposite population may be obtained by utilizing the OBL. In this work, we utilize the OBL to initialize the population. The implementation is as shown in Algorithm 1.

Oppositional Brain Storm Optimization for Fault Section …

4.2.2

67

Opposition-Based Generation Jumping

Similarly, we also apply the OBL to the current population. A jumping rate Jr is used to probabilistically force the evolutionary process to jump to an opposite solution candidate, which can lead to a higher chance to find a more promising offspring. The concrete procedure is as shown in Algorithm 2.

4.2.3

Oppositional Brain Storm Optimization

In this work, we adopt the opposition-based population initialization and oppositionbased generation jumping to accelerate the convergence speed of OBSO. The proposed OBSO method is as shown in Algorithm 3. It can be seen that compared with BSO, OBSO keeps the basic structure of BSO except adding two opposition-based operators in the initialization and the body of the loop, which makes it easy to implement. Besides, these two opposition-based operators offer more opportunities for OBSO to search fitter solutions and thereby to accelerate the convergence rate.

68

G. Xiong et al.

5 Application of OBSO for FSL It should be noted that in the original BSO and BSO in objective space algorithm, each idea is directly encoded by floating point for the global continuous optimization problems. Because the FSL problem is a 0–1 integer programming problem, so each idea needs to be transformed into a binary string before calculating its fitness. In this work, the sigmoid function [41] is employed to transform the population. The flowchart of application of OBSO for FSL is depicted in Fig. 1. After an occurrence of a fault event, we firstly construct the switching function for each involved switch by

Oppositional Brain Storm Optimization for Fault Section …

69

Fig. 1 Flowchart of FSL using OBSO

using (3). The objective function shown in (2) then can be generated by the acquired fault signals accordingly. Finally, the proposed OBSO method is applied to solve the objective function to obtain the faulted section or sections.

6 Case Studies 6.1 Test System An IEEE 33-bus distribution network shown in Fig. 2 is utilized to validate the performance of the proposed FSL approach. This distribution network consists of 33 sections (se1 , se2 , …, se33 ) and 33 FTUs (M 1 , M 2 , …, M 33 ). In actual operating process, multiple sections may suffer from fault simultaneously and the fault signals have the possibility of loss and/or distortion, which makes the FSL more difficult to be solved. In order to fully validate the effectiveness of OBSO on different fault scenarios, six cases listed in Table 1 are employed for demonstration.

70

G. Xiong et al.

Fig. 2 IEEE 33 bus distribution network

Table 1 Typical fault scenarios

Scenario

Overcurrent monitoring points

Wrong information

Faulted section

1

M1, M2, M3, M4, M5

None

se5

2

M1, M2, M3, M 4 , M 5 , M 13

M 13 distorted

se5

3

M1, M3, M4, M5

M 2 lost

se5

4

M1, M2, M3, M 4 , M 5 , M 25 , M 26 , M 27

None

se5 , se27

5

M1, M2, M3, M 4 , M 5 , M 13 , M 25 , M 26 , M 27

M 13 distorted

se5 , se27

6

M1, M3, M4, M 5 , M 13 , M 25 , M 26 , M 27

M 2 lost, M 13 distorted

se5 , se27

6.2 Simulation Settings In this work, the PSO and BSO are employed to compare the performance of OBSO. The population size N is set to be 50 for all methods and experiments. The maximum number of fitness function evaluations (Max_NFFEs) is used as the stopping condition and its value is set to be 50,000 for all fault scenarios. A number of 100 independent runs are conducted to evaluate the optimization performance. The parameter settings for PSO, BSO and OBSO are listed in Table 2.

Oppositional Brain Storm Optimization for Fault Section …

71

Table 2 Parameter settings of different methods Algorithm

Parameters settings

PSO

w = 0.9, c1 = c2 = 2.0

BSO

perce = 30, prep = 0.2, pe = 0.2, pone = 0.7, K = 20

OBSO

perce = 30, prep = 0.2, pe = 0.2, pone = 0.7, K = 20

6.3 Performance Criteria The following performance criteria are used to measure the performance of each optimization method. (1) Successful rate: A successful run of a method means that the method can locate the faulted section or sections correctly within the Max_NFFEs. It is equal to the number of successful runs divided by the total number of runs. (2) Location error: The location error calculated as F(X) −F(X ∗ ) is recorded when the Max_NFFEs is reached, where X and X ∗ are the location result and practical faulted section or sections, respectively. The mean and standard deviation of the location error values of 100 runs are recorded. (3) Statistic by Wilcoxon test: The Wilcoxon’s rank sum test at 0.05 confidence level is conducted to identify the significant difference of two methods on the same fault scenario. (4) Convergence graphs: The convergence graphs show the mean location error performance of 100 runs.

6.4 Simulation Results and Comparison 6.4.1

Influence of Jumping Rate Jr

Compared to BSO, OBSO contains an additional parameter jumping rate Jr. To investigate the influence of Jr on the performance of OBSO, an experiment under different Jr values (0 to 1 with interval 0.1) is conducted. The successful rate and location error results are tabulated in Tables 3 and 4, respectively. The best results are highlighted in boldface. It can be seen that a larger value of parameter Jr is not conducive to the performance of OBSO. Specifically, Jr = 0.1 can achieve the highest successful rates on fault scenarios 1, 2, 5 and 6. Jr = 0.2 and 0.4 can yield the best performance on fault scenarios 3 and 4, respectively, but the gaps between the results achieved with Jr = 0.1 and those achieved with Jr = 0.2 and 0.4 on both fault scenarios are relatively small. In addition, the result in terms of location error is also consistent agreement with the statement above. Overall, OBSO with Jr = 0.1 can achieve the best performance on all fault scenarios. Therefore, it is used in the following comparisons.

72

G. Xiong et al.

Table 3 Influence of parameter Jr on successful rate Scenario

Jr 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

1

83

85

84

80

78

83

76

74

73

56

51

2

81

86

83

84

80

76

77

65

65

51

57

3

88

86

92

89

79

77

83

67

64

67

56

4

73

83

73

67

86

67

65

55

54

46

48

5

76

80

78

79

70

69

63

64

58

51

41

6

75

81

76

73

68

74

62

61

48

48

40

6.4.2

Comparison with PSO and BSO

The successful rates of different methods for all fault scenarios are presented in Table 5. It is clear that OBSO achieves the highest successful rates on all fault scenarios and all the rates exceed 80%. BSO is slightly better than PSO, but the rates of both methods are below 60% especially on fault scenario 6. The fault situation of this scenario is very complex. It has two faulted sections coupling with lost and distorted information. PSO and BSO achieve a very low successful rate (34% and 37%, respectively) on this scenario while the value of OBSO is still higher than 80% (81%), indicating that OBSO is more robust in solving different fault scenarios with varying complexities. The comparison indicates that OBSO can considerably enhance the performance of BSO due to the acceleration ability of opposition-based operators. The location errors summarized in Table 6 obviously show that OBSO can obtain significantly smaller errors than both PSO and BSO on all six fault scenarios, which is further demonstrated by the Wilcoxon’s rank sum test result. The result consistently verifies again that OBSO indeed can improve the performance of BSO. The convergence curves plotted in Fig. 3 clearly present that although PSO can get the fastest convergence speed on all fault scenarios at the beginning stage, it quickly stagnates and suffers from prematurity. OBSO consistently converges faster than BSO throughout the whole evolutionary process. The result intuitively illustrates that the use of opposition-based population initialization and opposition-based generation jumping is indeed able to accelerate the convergence rate of BSO. In addition, it also shows that OBSO can obtain better results than both PSO and BSO on all six fault scenarios from another perspective.

7 Conclusions and Future Work In this chapter, an improved variant of BSO called OBSO is proposed to solve the FSL problem in distribution networks. The experimental results of six different

1.50E−01±4.80E−01

1.28E−01±3.08E−01

1.50E−01±3.70E−01

1.78E−01±3.67E−01

2.50E−01±6.45E−01

2.12E−01±6.24E−01

2.48E−01±4.82E−01

3.16E−01±6.53E−01

3.36E−01±6.44E−01

7.06E−01±1.20E+00

1.50E−01±1.28E+00

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

1

Scenario

0

Jr

2.42E−01±9.89E−01

7.50E−01±1.11E+00

5.18E−01±8.72E−01

4.26E−01±7.42E−01

3.14E−01±7.18E−01

2.64E−01±5.41E−01

2.12E−01±4.65E−01

1.76E−01±4.52E−01

1.96E−01±4.87E−01

1.30E−01±3.44E−01

2.42E−01±5.99E−01

2

Table 4 Influence of parameter Jr on location error

1.18E−01±9.47E−01

4.44E−01±7.58E−01

4.08E−01±6.57E−01

3.80E−01±6.43E−01

1.80E−01±4.54E−01

2.62E−01±5.55E−01

2.60E−01±6.92E−01

1.18E−01±3.53E−01

9.00E−02±3.28E−01

1.80E−01±5.51E−01

1.18E−01±3.36E−01

3

2.60E−01±8.52E−01

6.16E−01±6.76E−01

5.40E−01±7.68E−01

4.70E−01±5.81E−01

4.52E−01±7.95E−01

4.06E−01±8.07E−01

1.46E−01±3.90E−01

3.60E−01±5.94E−01

2.54E−01±4.37E−01

1.74E−01±4.02E−01

2.60E−01±4.74E−01

4

2.42E−01±8.92E−01

5.98E−01±7.77E−01

5.48E−01±7.45E−01

4.00E−01±5.76E−01

3.88E−01±5.55E−01

3.40E−01±5.72E−01

3.38E−01±6.13E−01

1.98E−01±3.98E−01

1.94E−01±3.77E−01

1.86E−01±3.83E−01

2.42E−01±4.66E−01

5

2.62E−01±7.85E−01

6.12E−01±7.27E−01

5.82E−01±6.32E−01

5.20E−01±7.88E−01

4.72E−01±7.55E−01

2.56E−01±4.63E−01

3.80E−01±6.47E−01

3.00E−01±5.57E−01

2.24E−01±4.24E−01

1.92E−01±4.38E−01

2.62E−01±4.88E−01

6

Oppositional Brain Storm Optimization for Fault Section … 73

74

G. Xiong et al.

Table 5 Successful rates of different methods for all fault scenarios Scenario

PSO (%)

BSO (%)

OBSO (%)

1

48

58

85

2

45

52

86

3

50

54

86

4

36

47

83

5

43

45

80

6

34

37

81

Table 6 Mean and standard deviation of the location error values for all fault scenarios Scenario

PSO Mean ± Std Dev

BSO Mean ± Std Dev

OBSO Mean ± Std Dev

1

1.41E+00±2.34E+00a

8.00E−01±1.30E+00a

1.28E−01±3.08E−01

2

2.07E+00±2.73E+00a

9.60E−01±1.42E+00a

1.30E−01±3.44E−01

3

1.65E+00±2.57E+00a

9.30E−01±1.54E+00a

1.80E−01±5.51E−01

4

1.50E+00±1.99E+00a

9.44E−01±1.37E+00a

1.74E−01±4.02E−01

5

1.19E+00±1.45E+00a

7.48E−01±9.34E−01a

1.86E−01±3.83E−01

6

1.45E+00±1.75E+00a

8.48E−01±9.86E−01a

1.92E−01±4.38E−01

a Symbolizes that OBSO is statistically better than its competitor according to the Wilcoxon’s rank-

sum test

Fig. 3 Convergence graphs of different methods for all fault scenarios. a Scenario 1. b Scenario 2. c Scenario 3. d Scenario 4. e Scenario 5. f Scenario 6

Oppositional Brain Storm Optimization for Fault Section …

75

fault scenarios in terms of successful rate, location error, statistic and convergence speed comprehensively demonstrate that the OBL is indeed able to accelerate the convergence rate of OBSO. OBSO can be used as an efficient and reliable alternative method for the FSL problem. The successful rate results show that although OBSO can achieve relatively high rates (>80%) on all fault scenarios, there is still room for improvement. In future work, we will adopt parameter adaptive methods to further improve its performance. Acknowledgements The authors would like to thank the editor and the reviewers for their constructive comments. This work was supported by the National Natural Science Foundation of China under Grant No. 51867005, the Scientific Research Foundation for the Introduction of Talent of Guizhou University (Grant No [2017] 16), and the Guizhou Province Science and Technology Innovation Talent Team Project (Grant No [2018] 5615), the Science and Technology Foundation of Guizhou Province (Grant No. [2016]1036), and the Guizhou Province Reform Foundation for Postgraduate Education (Grant No. [2016]02).

References 1. Kezunovic, M.: Smart fault location for smart grids. IEEE Trans. Smart Grid 2(1), 11–22 (2011) 2. Chen, Z.X., Zhang, X.B., Chen, X.Y.: Study of fault location algorithm in distribution network based on fault matrix engineering and technology. In: Proceedings of Spring Congress on Engineering and Technology (S-CET), pp. 1–3, Xian, China (2012) 3. Ma, Y., Mao, Y., Li, H.: An improved matrix algorithm for fault location in distribution network. In: Proceedings of Third International Conference on Intelligent System Design and Engineering Applications (ISDEA), pp. 289–293, Hong Kong, China (2013) 4. Hu, F., Sun, S.: Fault location of distribution network by applying matrix algorithm based on graph theory. Electr. Power 49(3), 94–98 (2016) 5. Mei, N., Shi, D., Yang, Z., Duan, X.: A matrix-based fault section estimation algorithm for complex distribution systems. In: Proceedings of 42nd International Universities Power Engineering Conference (UPEC), pp. 283–289, Brighton, UK (2007) 6. Xiong, G., Shi, D., Zhang, J., Zhang, Y.: A binary coded brain storm optimization for fault section diagnosis of power systems. Electr. Pow. Syst. Res. 163, 441–451 (2018) 7. Yu, L., Sun, Y., Li, K.J., Xu, M.: An improved genetic algorithm based on fuzzy inference theory and its application in distribution network fault location. In: Proceedings of 11th Conference on Industrial Electronics and Applications (ICIEA), pp. 1411–1415, Hefei, China (2016) 8. Yue, Y., Zhao, Y., Zhao, H., Wang, H.: A study of distribution network fault location including Distributed Generator based on improved genetic algorithm. In: Proceedings of 3rd International Conference on System Science, Engineering Design and Manufacturing Informatization (ICSEM), pp. 103–106 (2012) 9. Zheng, T., Pan, Y., Zheng, K., Wang, Z., Sun, J.: Fault location of distribution network based on immune algorithm. Power Syst. Prot. Control. 42(1), 77–83 (2014) 10. Li, Y., Ye, H., Chen, Z.: Binary particle swarm optimization algorithm with gene translocation for distribution network fault location. In: Proceedings of Spring Congress on Engineering and Technology (S-CET), pp. 1–4, Xian, China (2012) 11. Fu, J., Lu, Q.: Fault sections location of distribution network based on bat algorithm. Power Syst. Prot. Control. 43(16), 100–105 (2015) 12. Wang, Q., Huang, W.: A fault location method in distribution network based on firefly algorithm. Adv. Mater. Res. 971–973, 1463–1466 (2014) 13. Guo, Y., Xiong, G.: Fault section location in distribution network by means of sine cosine algorithm. Power Syst. Prot. Control. 45(13), 97–101 (2017)

76

G. Xiong et al.

14. Guo, Z., Wu, J.: Electromagnetism-like mechanism based fault section diagnosis for distribution network. Proc. CSEE 30(13), 34–40 (2010) 15. Zhou, Q., Zheng, B., Wang, C., Zhao, J., Wang, Y.: Fault location for distribution networks with distributed generation sources using a hybrid DE/PSO algorithm. In: Proceedings of IEEE Power and Energy Society General Meeting (PES), pp. 1–4, Vancouver, Canada (2013) 16. Shi, H.T., Tan, Q.D., Li, C.G.: Fault section location for distribution network with DG based on binary hybrid algorithm. In: Proceedings of International Conference on Mechanical and Mechatronics Engineering (ICMME), pp. 109–114, Bangkok, Thailand (2017) 17. Shen, M., Peng, M., Liu, T., Zhu, L., Che, H., Liu, Z., et al.: Distribution network fault location based on electromagnetism-like mechanism combined with genetic algorithm. Lect. Notes Inf. Theory 2(2), 146–150 (2014) 18. Shi, Y.: Brain storm optimization algorithm. In: Proceedings of International Conference in Swarm Intelligence, pp. 303–309, Chongqing, China (2011) 19. Shi, Y.: An optimization algorithm based on brainstorming process. Int. J. Swarm Intell. Res. 2(4), 35–62 (2011) 20. Cheng, S., Qin, Q., Chen, J., Shi, Y.: Brain storm optimization algorithm: a review. Artif. Intell. Rev. 46(4), 445–458 (2016) 21. Duan, H., Li, C.: Quantum-behaved brain storm optimization approach to solving loney’s solenoid problem. IEEE Trans. Magn. 51(1), 1–7 (2015) 22. Wang, J., Hou, R., Wang, C., Shen, L.: Improved v-Support vector regression model based on variable selection and brain storm optimization for stock price forecasting. Appl. Soft Comput. 49, 164–178 (2016) 23. Sun, C., Duan, H., Shi, Y.: Optimal satellite formation reconfiguration based on closed-loop brain storm optimization. IEEE Comput. Intell. M 8(4), 39–51 (2013) 24. Ma, X., Jin, Y., Dong, Q.: A generalized dynamic fuzzy neural network based on singular spectrum analysis optimized by brain storm optimization for short-term wind speed forecasting. Appl. Soft Comput. 54, 296–312 (2017) 25. Zhang, G., Zhan, Z., Du, K., Chen, W.: Normalization group brain storm optimization for power electronic circuit optimization. In: Proceedings of the Annual Conference on Genetic and Evolutionary Computation, pp. 183–184, Vancouver, Canada (2014) 26. Chen, J., Cheng, S., Chen, Y., Xie, Y., Shi, Y.: Enhanced brain storm optimization algorithm for wireless sensor networks deployment. Lect. Notes Comput. Sci. 9140, 373–381 (2015) 27. El-Abd, M.: Global-best brain storm optimization algorithm. Swarm Evol. Comput. 37, 27–44 (2017) 28. Cheng, S., Chen, J., Lei, X., Shi, Y.: Locating multiple optima via brain storm optimization algorithms. IEEE Access 6, 17039–17049 (2018) 29. Khedr, A.Y.: Brain storm optimization for sensors’ duty cycle in wireless sensor network. Int. J. Adv. Res. Comput. Commun. Eng. 6(4), 510–514 (2017) 30. Xiong, G., Shi, D.: Hybrid biogeography-based optimization with brain storm optimization for non-convex dynamic economic dispatch with valve-point effects. Energy 157, 424–435 (2018) 31. Shi, Y.: Brain storm optimization algorithm in objective space. In: Proceedings of IEEE Congress on Evolutionary Computation, pp. 1227–1234, Sendai, Japan (2015) 32. Tizhoosh, H.R.: Opposition-based learning: a new scheme for machine intelligence. In: Proceedings of International Conference on Computational Intelligence for Modelling, Control and Automation, pp. 695–701 Vienna, Austria (2005) 33. Wang, H., Wu, Z., Rahnamayan, S., Liu, Y., Ventresca, M.: Enhancing particle swarm optimization using generalized opposition-based learning. Inf. Sci. 181(20), 4699–4714 (2011) 34. Rahnamayan, S., Tizhoosh, H.R., Salama, M.M.A.: Opposition-based differential evolution. IEEE Trans. Evolut. Comput. 12(1), 64–79 (2008) 35. Barisal, A.K., Prusty, R.C.: Large scale economic dispatch of power systems using oppositional invasive weed optimization. Appl. Soft Comput. 29, 122–137 (2015) 36. Lin, J.: Oppositional backtracking search optimization algorithm for parameter identification of hyperchaotic systems. Nonlinear Dynam. 80(1–2), 209–219 (2015)

Oppositional Brain Storm Optimization for Fault Section …

77

37. Ergezer, M., Dan, S., Du, D.: Oppositional biogeography-based optimization. In: Proceedings of IEEE International Conference on Systems, Man and Cybernetics (SMC), pp. 1009–1014, San Antonio, USA (2009) 38. Roy, P.K., Paul, C., Sultana, S.: Oppositional teaching learning based optimization approach for combined heat and power dispatch. Int. J. Elec. Power 57(5), 392–403 (2014) 39. Chen, X., Yu, K., Du, W., Zhao, W., Liu, G.: Parameters identification of solar cell models using generalized oppositional teaching learning based optimization. Energy 99, 170–180 (2016) 40. Wei, Z., He, H., Zheng, Y.: A refined genetic algorithm for the fault sections location. Proc. CSEE 22(4), 127–130 (2002) 41. Mirjalili, S., Lewis, A.: S-shaped versus V-shaped transfer functions for binary particle swarm optimization. Swarm Evol. Comput. 9, 1–14 (2013)

Multi-objective Differential-Based Brain Storm Optimization for Environmental Economic Dispatch Problem Yali Wu, Xinrui Wang, Yingruo Xu and Yulong Fu

Abstract Environmental Economic Dispatch (EED) problem has been paid more attention in recent years as it can save the cost of the fuel while reducing the environmental pollution. A novel Multi-objective Differential Brain Storm Optimization (MDBSO) algorithm is proposed to solve EED problem in this chapter. Different from the classical BSO, the clustering operation is designed in the objective space instead of solution space to improve the computing efficiency. The difference mutation operation is also adopted in the proposed algorithm to replace the Gaussian mutation in the original BSO algorithm for increasing the diversity of the population and improving the speed of convergence. The performance of the proposed algorithm is verified by two test systems with 6 units and 40 units in the literature. The simulation results show that comparing with other intelligent optimization method, MDBSO can maintain the diversity of Pareto optimal solutions and show better convergence at the same time. Keywords Multi-objective optimization · Environmental economic dispatch · Brain storm optimization · Differential equation

1 Introduction In recent years, environmental pollution in the power industry has been paid more attention. Many countries have enacted regulations to limit the emission of harmful gases from the thermal power plant. How to ensure the normal operation of the power system with the low fuel cost and emission under the premise of ensuring reliable Y. Wu (B) · X. Wang · Y. Xu · Y. Fu Faculty of Automation and Information Engineering, Xi’an University of Technology, Xi’an 710048, China e-mail: [email protected] Y. Wu · X. Wang · Y. Xu · Y. Fu Shaanxi Key Laboratory of Complex System Control and Intelligent Information Processing, Xi’an 710048, China © Springer Nature Switzerland AG 2019 S. Cheng and Y. Shi (eds.), Brain Storm Optimization Algorithms, Adaptation, Learning, and Optimization 23, https://doi.org/10.1007/978-3-030-15070-9_4

79

80

Y. Wu et al.

power supply, which is named Environment Economic Dispatch (EED) problem, becomes more and more desirable for minimizing cost and emission issues simultaneously [1–5]. From the mathematical model description, EED problem can be recognized as a complicated multi-objective constrained problem with two conflicting objectives, which is the minimum fuel cost and the minimum emission effects. Many different techniques have been reported in the literatures to solve EED problem in the past decades. Abido [6] worked on EED problem with the Non-dominated Sorting Genetic Algorithm (NSGA), which employed a diversity-preserving technique to overcome the premature convergence and search bias problems, producing a well-distributed Pareto-optimal set of non-dominated solutions. A hierarchical clustering algorithm was also imposed to provide the decision maker with a representative and manageable Pareto-optimal set. Moreover, fuzzy set theory is employed to extract the best compromise solution over the trade-off curve. Rose et al. [7] improved genetic algorithm with Gauss Newton for improving the speed of optimization. El-Sehiemy et al. [8] proposed a novel Multi-Objective improved Real-Coded Genetic Algorithm (MO-RCGA) for solving EED problems. The MO-RCGA technique evolves a multi-objective version of GA by proposing redefinition of global best and local best individuals in multi-objective optimization domain. MO-RCGA algorithm obtains feasible set of effective well-distributed solutions. Lu et al. [9] proposed an Enhanced Multi-Objective Differential Evolution (EMODE) algorithm to solve EED problem, in which a Local Random Search (LRS) operator was integrated with the method to improve the convergence performance and an elitist archive technique was adopted to retain the non-dominated solutions obtained during the evolutionary process, and the operators of DE were modified. Basu [10] adopted Differential Evolution (DE) algorithm to solve this highly non-linear EED problem. Pandit et al. [11] improved differential evolution method where the selection operation was modified to reduce the complexity of multi-attribute decision making with the help of a fuzzy framework. Solutions were assigned to a fuzzy rank on the basis of their level of satisfaction for different objectives before the population selection and then the fuzzy rank was used to select and pass on better solutions to the next generation. A well distributed Pareto-front was obtained which presented a large number of alternate trade-off solutions for the power system operator. A momentum operation was also included to prevent stagnation and create Pareto diversity. PSO was also one of the computational techniques to obtain Pareto solutions in solving EED problems. Agrawal et al. [12] proposed a Fuzzy Clustering-based Particle Swarm Optimization (FCPSO) algorithm to solve EED problem and the simulation results revealed that the approach was able to provide a satisfactory compromise solution in almost all the trials. Rose et al. [13] proposed a hybrid technique to optimize the economic and emission dispatch problem in power system which employs particle swarm optimization and Neural Network (NN). Prior to performing this, the neural network training method was used to train all the generating power with respect to the load demand. Zou et al. [14] proposed a Global Particle Swarm

Multi-objective Differential-Based Brain Storm Optimization …

81

Optimization (GPSO) algorithm which used a new position updating equation relied on the global best particle to guide the searching activities of all the particles. The randomization based on the uniform distribution was adopted to slightly disturb the flight trajectories of particles during the late evolutionary process. The algorithm increased the chance of exploring promising solution space and reduced the probabilities of getting trapped into local optima for all the particles. The simulation results showed that the proposed algorithm solves the EED problem effectively. In recent years, all kinds of novel swarm intelligent optimization were also proposed to solve EED problem. Roy and Bhui [15] proposed a Quasi-Opposite Teaching Learning Based Optimization (QOTLBO), which provided a relatively better solution to the EED problem with less computation time. Zhao [5] proved a summary Stochastic Fractal Search (SFS) to solve the highly nonlinear EED problem in power systems. Lu et al. [16] proposed a Multi-Objective optimization approach based on Bacterial Colony Chemotaxis (MOBCC) algorithm, which employed a Lamarckian constraint handling method to update the bacterial colony and the external archive. To sum up, swarm intelligence optimization algorithms have been widely used in solving the EED problem. Brain Storm Optimization (BSO) was proposed in 2011 and was inspired by the idea of being able to efficiently generate better ideas at the human brainstorming conference [17]. The algorithm was guided by the cluster center and learned from other individuals with probability, which can get the better balance between the convergence and the diversity. On this basis, a variety of MultiObjective Brain Storm Optimization (MOBSO) algorithms have been derived from the operations of clustering, mutation, and updating of archive sets. Nearly all of the algorithms have achieved the good results. Xue et al. [18] proposed multi-objective BSO algorithm with the non-dominated solutions to fit the true Pareto-front. Wu et al. [19] introduced Multi-Objective Brain Storm Optimization based on Estimating in Knee region and Clustering in Objective space (MOBSO-EKCO) algorithm to get the knee point of Pareto-optimal front. Guo et al. [20] proposed a modified Self-adaptive Multi-Objective Brain Storm Optimization (SMOBSO) algorithm which adopted the simple clustering operation to increase the searching speed. At the same time, the open probability was introduced to avoid the algorithm trapping into local optimum. The adaptive mutation method was adopted to give uneven distribution solutions. The differential evolution concept was integrated with original BSO to accelerate the convergence speed. And the clustering operation was designed in the objective space to save the computing time. In this chapter, a novel Multi-objective Differential Brain Storm Optimization(MDBSO)algorithm is proposed to solve EED problem. Different from the classical BSO, the clustering operation is designed in the objective space instead of solution space to improve the computing efficiency. The difference mutation operation is also adopted in the proposed algorithm to replace the Gaussian mutation in the original BSO algorithm for increasing the diversity of the population and improving the speed of convergence. The effectiveness and application of the proposed method are demonstrated by implementing it in standard 6-unit and 40-unit test systems.

82

Y. Wu et al.

The structure of the chapter is organized as follows. EED problem formulation is introduced and analyzed in Sect. 2. Section 3 presents the multi-objective optimization problem. The proposed differential BSO is revealed in Sect. 4. The detail of MDBSO is described in Sect. 5. The process of solving EED problem with MDBSO is discussed in Sect. 6. The numerical simulation cases are implemented to verify the effectiveness of the proposed method in Sect. 7. The conclusions is outlined in Sect. 8 followed by acknowledgements.

2 The Formulation of Environmental Economic Dispatch Problem 2.1 The Objective Function of EED Problem The purpose of EED problem is to minimize the fuel cost and emission effects simultaneously while satisfying various physical and operational constraints. Hence, there are two objective functions for EED problem as follows: min

 N  i=1

Fi (Pi ),

N 

 Ei (Pi )

(1)

i=1

  where Ni=1 Fi (Pi ) is the total fuel cost function; Ni=1 Ei (Pi ) is the total emission of pollutants and N is the number of generating units. (1) The fuel cost function There are usually two versions of the fuel cost function. The one is a quadratic function [10], and the other is a combination function of quadratic and sinusoidal (valve point effect) [10, 21]. The fuel cost function can be represented as follows: Fi (Pi ) = ai + bi Pi + ci Pi2     Fi (Pi ) = ai + bi Pi + ci Pi2 + ei sin(fi (Pimin − Pi ))

(2) (3)

where Pi is the real power output of the i-th generating unit; Fi (Pi ) is the cost function of the i-th generating unit; ai , bi , ci , ei , fi are cost coefficients of the i-th generating unit. (2) Pollutants emission function The environmental pollutants such as sulphur oxides SOx and nitrogen oxides NOx caused by fossil-fueled generating units can be described as follows [22, 23]:   Ei (Pi ) = αi + βi Pi + γi Pi2 /100 + ξi exp(λi Pi )

(4)

Multi-objective Differential-Based Brain Storm Optimization …

83

where Ei (Pi ) is the emission function of the i-th generating unit; αi , βi , γi , ξi , λi are emission coefficients of the i-th generating unit.

2.2 The Constraints of EED Problem The constraints of the EED problem have both equality constraints and inequality constraints. The equality constraints are power balance constraint: the total power generated must cover the total demand and the total transmission losses, which is expressed as: N 

Pi = PM + PLOSS

(5)

i=1

where PM is the total demand; PLOSS is the total transmission losses, which is usually calculated by using the B-coefficient method which can be described as follows: PLOSS =

N  N  i=1 j=1

Pi Bij Pj +

N 

Boi Pi + Boo

(6)

i=1

where Bij , Boi and Boo are the loss coefficient. The generation capacity constraint is the inequality constraint. The real power output of each generator is restricted by the minimum output power and the maximum output power as follows: Pimin ≤ Pi ≤ Pimax

(7)

where Pimin , Pimax are the minimum and maximum output limit of the i-th generating unit respectively. From the discussion above, we can get that the EED problem is a multi-objective optimization problem with both equality and inequality constraints. This makes the solution very difficult and complex. We’ll give the solution method of multi-objective optimization problem in the next section.

3 Multi-objective Optimization Problem (MOP) Without loss of generality, all of the multi-objective optimization problems can be formulated as minimization optimization problems, which is described as the follows: min f (x) = (f1 (x), f2 (x), . . . , fm (x))T

84

Y. Wu et al.

s.t.

gi (x) = 0, i = 1, 2, . . . , p hj (x) ≤ 0, j = 1, 2, . . . , q

(8)

where x = (x1 , · · · , xD ) ∈ D is called the decision vector in the D-dimensional search space which represents a solution of the problem, fn (x) ∈ Rm is the n-th objective function in the m-dimensional objective space. p and q are the numbers of equality and inequality constraints, respectively. To solve multi-objective optimization problems, the following concepts are essential. Feasible solution set: If x satisfies all constraints, x is called a feasible solution and the set of all feasible solutions are defined as the feasible solution set. Domination: Given two feasible solutions x1 and x2 , we say that x1 dominates x2 (denoted as x1 ≺ x2 ), if ∀n ∈ {1, 2, . . . , m}, fn (x1 ) ≤ fn (x2 ), ∃n ∈ {1, 2, . . . , m}, fn (x1 ) < fn (x2 ). Pareto-optimal set: If there does not exist another feasible solution x satisfying x ≺ x, we say that x is non-dominated, and this feasible solution x is defined as a Pareto-optimal solution as x∗ . The set of all Pareto-optimal solutions is defined as the Pareto-optimal set denoted as PS. Pareto-optimal front: The Pareto-optimal front is defined as PF, where PF = {f (x∗ ) = ( f1 (x∗ ), f2 (x∗ ), . . . , fm (x∗ ))T |x∗ ∈ PS}. Based on the above concepts, the key task of multi-objective optimization algorithms is to obtain the Pareto-optimal set of the problem so as to provide more choice for the decision maker.

4 Brain Storm Optimization Algorithm Based on Differential Evolution 4.1 Brain Storm Optimization Algorithm The BSO algorithm is designed based on the brainstorming process [17]. In the brainstorming process, the generation of the idea obeys the Osborn’s original four rules. The people in the brainstorming group will need to be open-minded as much as possible and therefore generate more diverse ideas. Any judgment or criticism must be held back until at least the end of one round of the brainstorming process, which means no idea will be ignored. The algorithm is described as follows. In the initialization, N potential individuals were randomly generated. During the evolutionary process, BSO generally uses the clustering technique, mutation operator and selection operator to create new ideas based on the current ideas, so as to improve the ideas generation by generation to approach the problem solution. Clustering is the process of grouping similar objects together. From the perspective of machine learning, the clustering analysis is sometimes termed as unsupervised learning [24]. In the mutation operator, BSO creates N new individuals one by one based on the

Multi-objective Differential-Based Brain Storm Optimization …

85

Table 1 Procedure of BSO algorithm Procedure of BSO Algorithm 1. Randomly generate n potential solutions(individuals); 2. Cluster n individuals into m clusters; 3. Evaluate the n individuals; 4. Rank individuals in each cluster and record the best individual as cluster center in each cluster; 5. Randomly generate a value between 0 and 1; a) If the value is smaller than a pre-determined probability P5 a , i. Randomly select a cluster center; ii. Randomly generate an individual to replace the selected cluster center; 6. Generate new individuals a) Randomly generate a value between 0 and 1;

P

b) If the value is less than a probability 6b , i. Randomly select a cluster with a probability p6bi; ii. Generate a random value between 0 and 1;

P

iii. If the value is smaller than a pre-determined probability 6biii , 1) Select the cluster center and add random values to it to generate new individual. iv. Otherwise randomly select an individual from this cluster and add random value to the individual to generate new individual. c) Otherwise randomly select two clusters to generate new individual i. Generate a random value; ii. If it is less than a pre-determined probability

P6 c , the two cluster centers are combined and then

added with random values to generate new individual; iii. Otherwise, two individuals from each selected cluster are randomly selected to be combined and added with random values to generate new individual. d) The newly generated individual is compared with the existing individual with the same individual index, the better one is kept and recorded as the new individual; 7. If n new individuals have been generated, go to step 8; otherwise go to step 6; 8. Terminate if pre-determined maximum number of iterations has been reached; otherwise go to step 2.

current ideas. To create a new individual, BSO first determines whether to create the new individual based on one selected cluster or based on two selected clusters. After the cluster(s) have been selected, BSO then determines whether create the new idea based on the cluster center(s) or random idea(s) of the cluster(s). No matter to use the cluster center or to use random idea of the cluster, the selected based idea 1 2 d , xselected , . . . , xselected ). Then a as Xselected can be expressed as Xselected = (xselected mutation of the Xselected is applied to get the new idea Xnew which can be expressed 1 2 d , xnew , · · · , xnew ). After the new idea Xnew has been created, BSO as Xnew = (xnew evaluates Xnew and replaces Xselected if Xnew has a better fitness than Xselected . The procedure of the BSO algorithm is listed in Table 1, which is shown in [17].

4.2 Brain Storm Optimization Algorithm Based on Differential Evolution BSO is a type of evolutionary algorithm for optimization problems over a continuous domain. The fittest of an offspring competes one-to-one with that of corresponding

86

Y. Wu et al.

parent in BSO. This one-to-one competition and the clustering technique give rise to faster convergence rate. The mutation operation in the original BSO is Gaussian mutation which is consistent as the range of variation is basically fixed. This always leads the individuals cannot make full use of information in the current population. Moreover, the computation complicity of the Gaussian mutation operation is also very high. As we all know, the perturbations of DE operation are large at the beginning of the evolution since parent populations are far away from each other. As the evolutionary process matures, the population converges to a small region and the perturbations adaptively become small. As a result, the DE performs a global exploratory search during the early stages of the evolutionary process and local exploitation during the mature stage of the search. So the differential evolution mutation is served in this chapter to replace of the Gaussian mutation.

5 Multi-objective Differential-Based Brain Storm Optimization Algorithm (MDBSO) Taking the advantage of the differential evolution and BSO, Multi-objective differential-based brain storm optimization based on clustering in objective-space is proposed in this chapter to solve multi-objective optimization problems. The framework of MDBSO can be described as Table 2. from the Table 2, we can get that compared with the original BSO, MDBSO introduces the strategy of archive set, that is, non-dominated sorting of populations and non-dominated solutions is put into the archive set in each iteration, finally getting a uniform group which close enough Pareto front. The main steps of MDBSO are detailed as follow: (1) Clustering Technique

Table 2 The framework of MDBSO

Input: NP (population size), Nmax (the maximum iteration number), NQ (the size of archive set), m (the number of clusters),

p1 , p5 a , p6b , p6biii , p6 c (the parameters of

the MDBSO algorithm) Output: Q (archive set) 1 Q ← Φ; 2P Initialization of the population (NP); Clustering Technique(P); 3P Archive updating(NQ, P, Q); 4Q 5 while termination criterion not fulfilled do Mutation Operator(P); 6 P Archive updating(NQ, P, Q); 7 Q Clustering Technique(P); 8 P

Multi-objective Differential-Based Brain Storm Optimization …

x2

f2

Decision Space

87

Objective Space

S2

C2 mapping

S1

x3

C1

x1 f1

Fig. 1 The mapping relationship between target space and decision space

In the original BSO algorithm, the clustering operation is performed for solutions in the solution space. As the dimension of the problem increases, the complex of computation increases. But in the multi-objective optimization problem, the numbers of optimization objectives are often smaller than the dimension of the decision variables. Therefore, clustering operations in the objective space will greatly reduce the amount of computation. The relationship between the solution space and the objective space is shown in Fig. 1. The objective vectors are clustered into k (k = 2) clusters. The points with diamond shape are non-dominated solutions of the Pareto front. The cluster C1 with non-dominated solutions is defined as the elite cluster, and the non-dominated solution with cluster C2 is the normal cluster. From the non-inferior relations in the cluster, it can be seen that the individuals in the cluster C1 dominate all the individuals in the cluster C2. Therefore, in the iterative update process, the existence of non-dominating classes greatly reduces the evaluation process of the single dominant relationship, which greatly reduce updates and maintenance processes of the dominating set. In this chapter, clustering fitness value replaces clustering individuals into clusters; the k-means method is adopted for clustering. The steps are the same in the solution space, the standard of Euclidean distance to judge the similarity between two individuals in the objective space. (2) Mutation Operator The Gaussian random values will be used as random values which are added to generate new individuals in the original BSO algorithm. The new individual generation can be represented as the Eq. (9): Pjnew = Pjselected + ξ ∗ n(μ, σ )

(9)

where Pjselected is the j-th dimension of the individual selected to generate new individual. Pjnew is the j-th dimension of the individual newly generated. n(μ, σ ) is the

88

Y. Wu et al.

Fig. 2 Gaussian distribution

Gaussian random function with mean μ and variance σ . The parameter ξ is a coefficient that weights the contribution of the Gaussian random value. For the purpose of fine tuning, the parameter ξ can be calculated as: ξ = log sig((0.5 ∗ Nmax − i)/K) ∗ rand ()

(10)

where log sig() is a logarithmic sigmoid transfer function. Nmax is the maximum number of iterations. i is the current iteration number. K is for changing function’s slope. rand () is a random value within (0, 1). As a common mutation method, the variance of Gaussian mutation is obtained by multiplying ξ and the Gaussian random function. The value of log sig() is between 0 and 1. As shown in Fig. 2, the Gaussian distribution follows the principle of 3σ . When μ and σ are fixed, the Gaussian function produces most values between (μ−3σ, μ + 3σ ), which is shown in Fig. 3. When μ = 0, σ = 1, the range of variation is (−3, 3). During the process of searching the optimal solution, all the solutions can be uniformly generated in the decision space at first. The global search should be carried out at the beginning. At the latter stage, the local variation should be optimized due to the overall rise of the solution quality; the variance should be smaller with the stage process lasts. However, Gaussian variation is consistent. The individuals cannot make full use of information in the current population because of the range of variation is basically fixed.

Multi-objective Differential-Based Brain Storm Optimization …

89

Fig. 3 The range of variation when μ = 0, σ = 1

Differential evolution (DE) mutation can be served as a useful tool to achieve the purpose. The mutation operation can be described as follows in Eq. (11). 

rand Pjmin , Pjmax if rand () < Pr new (11) Pj =   selected Pj + rand (0, 1) × P1,j − P2,j else where Pjselected is the j-th dimension of the individual selected to generate new individual. Pjnew is the j-th dimension of the individual newly generated. The upper and lower bounds of the dimension are P min ,P max , and P , P are the j th dimension j

j

1,j

2,j

of two different individuals in the contemporary global situation. Compared with Gaussian variation, only the random function and the four mixed operations are considered in the differential evolution. So the computation complexity will be reduced significantly. Whenever the generation of random values is based on other individuals within the contemporary population, the information of the other individuals within the population can be obtained immediately. This makes the searching efficiency higher than ever. Moreover, the differential mutation can balance the local search and global search in the search process greatly which can improve the algorithm performance effectively. (3) Archive updating External population (called as archive set) was firstly introduced by Zitzler and Thiele [25] to retain the non-dominated individuals found along with the evolutionary process. Pareto dominance is used to determine which solution should be selected to enter in archive set. Considering of the limitation of computation source, the size of archive set Q is usually a constant (named as NQ). All non-dominated individuals (named as NDSet(g)) found in current population Popg are added to archive set Q by using the following strategy:

90

Y. Wu et al.

For each individual ni,g in NDSet(g), (i) If archive set Q is empty, add individual ni,g to archive set Q. (ii) If archive set Q is not full and individual ni,g is not dominated by any individual in archive set Q, then add individual ni,g to archive set Q and delete the individuals which are dominated by individual ni,g . (iii) If the size of archive set Q is larger than the constant NQ, a truncation operation is needed to maintain the size of archive set Q by eliminating the redundant individuals. In this chapter, the crowding distance which was introduced in [26] is used to pick out the individual who has the minimum crowding distance.

6 MDBSO Algorithm for Environmental/Economic Dispatch Due to the actual problem have a lot of constraints, the framework of MDBSO solves the actual problem can be different with the Table 1. In this section, the structure of MDBSO for EED problem is listed in Table 2. And the difference of them are also described in the following (Table 3). When swarm intelligent optimization algorithms are used to solve the practical problem, one of the most important points is the mapping problem between the actual problem description and the algorithm model. For the environmental economic dispatch problem, the load distribution of each unit is determined under the premise of satisfying constraints. Therefore, individuals in the brain storm optimization algorithm are corresponding to a set of schedules of load distribution. However, the difficulty in obtaining feasible solution increases dramatically due to the mul-

Table 3 The framework of MDBSO solves the actual problem

Input: NP (population size), Nmax (the maximum iteration number), NQ (the size of archive set), m (the number of clusters),

p1 , p5 a , p6b , p6biii , p6 c (the parameters of

the MDBSO algorithm) Output: Q (archive set) 1 Q ← Φ; Initialization of the population (NP); 2P Constraint handling(P); 3P Clustering Technique(P); 4P Archive updating(NQ, P, Q); 5Q 6 while termination criterion not fulfilled do 7 P Mutation Operator(P); Constraint handling(P); 8 P Archive updating(NQ, P, Q); 9 Q Clustering Technique(P); 10 P

Multi-objective Differential-Based Brain Storm Optimization …

91

tiple constraints of units. It is worth noting that there are several differences in the application, which is detailed as follows: (1) Initialization of the population For coding convenience, the real power output of each generator in environmental economic dispatch problem is taken as the decision variable. The coding of the individual in the population can be written as follows: {P1 , P2 , . . . , PN }

(12)

The initial population is initialized by creating NP solutions randomly. For each individual, all elements are randomly generated within the feasible real power output range. An element of a solution is initialized as follows:  min + rand ∗ P max − P min (13) Pi,j = Pi,j i,j i,j max and P min are minimum and where Pi,j is the j-th element of i-th individual. Pi,j i,j maximum limit of the corresponding elements respectively. (2) Constraint handling A. Constraint handling method for the generation capacity The generation capacity constraint is considered firstly when we apply MDBSO method to solve EED problem. Most of the new generated solution which is created randomly may not meet the capacity constraint. Therefore, the value of the element which is violated the feasible regions should be modified as follows:

Pjmin if Pi,j < Pjmin Pi,j = i = 1, 2, . . . , NP; j = 1, 2, . . . , N (14) Pjmax if Pi,j < Pjmax B. Constraint handling method for the power balance constraint It is very difficult to meet the equation constraints (the power balance constraint) in classical BSO algorithm. According to the Eq. (5), two different kinds of constraints condition is considered in handling power balance. The transmission losses are ignored or considered, which is discussed in the following respectively. a. The total transmission losses ignored When the total transmission losses is ignored, which is means that PLOSS = 0, the feasible method to handle equation constraints is shown as follows. (1) Creating the first N −1 dimensions of each solution satisfy inequality constraints using the Eq. (14) firstly. (2) Then calculating the last dimension of each solution according to the equation PN = PM − P1 − P2 − P3 − P4 − · · · − PN −1

92

Y. Wu et al.

(3) If the last dimension of the solution are not satisfy the inequality constraint, adjusting one or two dimensions in the first N − 1 dimension so that the solution satisfy inequality constraints b. The total transmission losses considered When the total transmission losses are considered, the handling procedure is listed as follows. (1) Firstly, PLOSS is extended into the following formula:

PLOSS =

N  N  i=1 j=1

Pi Bij Pj +

N 

Boi Pi + Boo

i=1

= B11 P12 + B12 P1 P2 + B13 P1 P3 + · · · + B1N P1 PN + B21 P1 P2 + B22 P22 + B23 P2 P3 + · · · + B2N P2 PN + ··· + BN 1 P1 PN + BN 2 P2 PN + BN 3 P3 PN + · · · + BNN PN2 + Bo1 P1 + Bo2 P2 + Bo3 P3 + · · · BoN PN + Boo = Cbe + Cin PN + Cin2 PN2

(15)

where Cbe = B11 P12 + B12 P1 P2 + · · · B1(N −1) P1 PN −1 + B21 P1 P2 + · · · + B2(N −1) P2 PN −1 + · · · + B(N −1)1 P1 PN −1 + · · · B(N −1)(N −1) PN2 −1 + Boo Cin = B1N P1 + B2N P2 + · · · + B(N −1)N PN −1 + BN 1 P1 + BN 2 P2 + · · · + BN (N −1) PN −1   = (B1N + BN 1 )P1 + (B2N + BN 2 )P2 + · · · + B(N −1)N + BN (N −1) PN −1

Cin2 = BNN (2) Then PLOSS = Cbe + Cin PN + Cin2 PN2 from Eq. (15) replace PLOSS of power balance constraint (equality constraints): P1 + P2 + P3 + P4 + P5 + · · · + PN = PM + Cbe + Cin PN + Cin2 PN2 C1 PN2 + C2 PN + C3 = 0

(16) (17)

where C 1 = C in2, C 2 = C in – 1, C3 = PM + Cbe − P1 − P2 − P3 − P4 − · · · − PN −1 . (3) Initializing P1 , P2 , P3 , P4 , …, PN −1 , PN is calculated by Eq. (15).

Multi-objective Differential-Based Brain Storm Optimization …

93

7 Experiments and Discussions Two test systems are investigated and the computational results have been used to compare the performance of the proposed method with other evolutionary methods. The original multi-objective brainstorming optimization algorithm (MOBSO) and MDBSO are implemented for two typical power system environmental economic dispatch problems. One is the small-scale power system environmental economic dispatch with 6 generating units, and the other is the larger-scale environmental economic dispatch with 40 generating units.

7.1 Test System 1 This test power system is the standard IEEE 30-bus power system which consists of 6 generating units. The system connection diagram is shown in Fig. 4. In the figure, the nodes 1, 2, 5, 8, 11, 13 are generator nodes. The load demand in this system is PM = 2.834 MW. The coefficients of fuel cost, emission and the capacities of the generating units are shown in Table 4, and the transmission losses coefficients are specified in (18). Here, two cases are considered. Case 1 When the total transmission losses of the power system haven’t been considered, the population size (NP) and the maximum iteration number (Nmax) have been selected as 100 and 1000 respectively for this test system. The size of archive set is 50. The

Fig. 4 IEEE 30-bus power system

94

Y. Wu et al.

Table 4 The fuel cost, emission and the capacities coefficients of the generating units of test system 1 Unit

Capacities max min PGi PGi

Cost a

b

c

Emission α

β

γ

ξ

λ

1

0.05

0.5

10

200

100

4.091

−5.554

6.490

2.0e−4

2.857

2

0.05

0.6

10

150

120

2.543

−6.047

5.638

5.0e−4

3.333

3

0.05

1.0

20

180

40

4.258

−5.094

4.586

1.0e−6

8.000

4

0.05

1.2

10

100

60

5.526

−3.550

3.380

2.0e−3

2.000

5

0.05

1.0

20

180

40

4.258

−5.094

4.586

1.0e−6

8.000

6

0.05

0.6

10

150

100

6.131

−5.555

5.151

1.0e−5

6.667

other parameters of the proposed algorithm are set as follows: p1 = 0.2, p5a = 0.1, p6b = 0.8, p6biii = 0.4, p6c = 0.5, the number of clusters is m = 4. Table 5 shows the minimum cost and minimum emissions for case 1. From the Table 5, we can see that the minimum fuel cost by the MDBSO is 600.1128 ($/h). Compared with MOBSO method, the proposed MDBSO increase the fuel cost about 0.0014 ($/h). The minimum fuel cost by MOBSO is 600.1114 ($/h). The minimum fuel cost by MDBSO which correspond emissions is 0.2245 (t/h), compared with MOBSO method, the emissions by MDBSO is increased about 0.0004 (t/h). The minimum emission with MDBSO is 0.1962 (t/h), which is the same as MOBSO. But compared with other algorithms, the minimum emissions by MDBSO and MOBSO is increased about 0.002 (t/h). In order to more clearly show the detail of the solution such as quality and distribution, the Pareto fronts of MOBSO and MDBSO for EED problem are shown in Fig. 5 respectively. ⎡ ⎤ 0.0218 0.0107 −0.00036 −0.0011 0.00055 0.0033 ⎢ 0.0107 0.01704 −0.0001 −0.00179 0.00026 0.0028 ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ −0.0004 −0.0002 0.02459 −0.01328 −0.0118 −0.0079 ⎥ Bij = ⎢ ⎥ ⎢ −0.0011 −0.00179 −0.01328 0.0265 0.0098 0.0045 ⎥ ⎢ ⎥ ⎣ 0.0055 0.00026 0.0118 0.0098 0.0216 −0.0001 ⎦ 0.0033 0.0028 −0.00792 0.0045 −0.00012 0.02978   Boi = 0.010731 1.7704 −4.0645 3.8453 1.3832 5.5503 × e − 03 Boo = 0.0014

(18)

Case 2 The total transmission losses of the power system have been considered in this case. The problem is solved by using both the proposed MOBSO and MDBSO. The parameters of the case 2 are the same as the case 1.

0.5289

1.0025

0.5402

0.3664

3

4

5

6

E

0.2215

600.15

0.2897

2

C

0.1602

0.22188

600.155

0.3583

0.5159

1.0146

0.5216

0.3177

0.1059

0.2223

600.1315

0.3673

0.5300

1.0150

0.5250

0.2897

0.1070

0.2241

600.1114

0.3597

0.5243

1.0162

0.5243

0.2998

0.1097

0.2245

600.1128

0.3581

0.5238

1.0197

0.5245

0.3001

0.1079

MDBSO

0.1942

638.51

0.5148

0.5383

0.3832

0.5329

0.4532

0.4116

0.1942

638.269

0.5110

0.5352

0.3837

0.5389

0.4577

0.4074

NSGA-II [12]

SPEA [12]

MOBSO

Minimum emissions

FCPSO [12]

SPEA [12]

NSGA-II [12]

Minimum cost

1

Unit

Table 5 The simulation result of fuel cost and emission effects for MOBSO and MDBSO for case 1 of test system 1

0.1942

638.3577

0.5140

0.5348

0.3842

0.5363

0.4550

0.4097

FCPSO [12]

0.1962

638.2736

0.5100

0.5379

0.3830

0.5379

0.4591

0.4061

MOBSO

0.1962

638.1724

0.5106

0.5386

0.3834

0.5380

0.4579

0.4054

MDBSO

Multi-objective Differential-Based Brain Storm Optimization … 95

96

Y. Wu et al. 0.225 MDBSO MOBSO 0.22

0.215

0.21

0.205

0.2

0.195 600

605

610

615

620

625

630

635

640

Fig. 5 The Pareto front by MOBSO and MDBSO for case 1 of test system 1

As seen in Table 6, The minimum fuel cost by the MDBSO is 608.0953 ($/h). Compared with MOBSO method, the proposed MDBSO increase the fuel cost with about 0.0004 ($/h). But compared with other algorithms, the minimum fuel cost by FCPSO is 607.7862 ($/h), which reduce the fuel cost by 0.3091 ($/h). The minimum fuel cost by MDBSO which correspond emissions is 0.2203 (t/h), while the emissions by MDBSO is increased by about 0.0002 (t/h) compared with FCPSO method. The minimum emission by MDBSO is 0.1962 (t/h), which is the same as MOBSO. As for other algorithms, the minimum emission by NSGA-II is 0.19419 (t/h), which reduces the emission by about 0.00201 (t/h). The minimum emission by MDBSO which correspond fuel cost is 643.8548 ($/h), the fuel cost by MDBSO is reduced by about 0.2782 ($/h) compared with NSGA-II method. The Pareto fronts of MOBSO and MDBSO for EED problem are shown in Fig. 6. As it can be seen from Figs. 5 and 6, both the Pareto front of MOBSO and the MDBSO are smooth and uniform. So both MOBSO and MDBSO have good performance for small-scale EED problems.

7.2 Test System 2 40 generating units [14] are considered in this test power system. The load demand is PM = 10,500 MW. The coefficients of fuel cost, emission and the capacities of the generating units are shown in Table 7. The total transmission losses of the power system haven’t been considered. The population size and the maximum iteration number have been selected as 100 and 2000, respectively. The size of archive set is 50. The parameters of the proposed method are the same as the case 1.

0.5818

0.9846

0.5288

0.3584

607.807

3

4

5

6

C

0.22015

0.3065

2

E

0.1086

0.21891

607.801

0.3548

0.5172

0.9710

0.5910

0.3148

0.1182

0.2201

607.7862

0.3450

0.5264

0.9860

0.5826

0.3145

0.1130

0.2203

608.0949

0.3715

0.4978

0.9560

0.6216

0.3039

0.1162

0.2203

608.0953

0.3715

0.4993

0.9560

0.6192

0.3043

0.1168

MDBSO

0.19422

642.603

0.5005

0.5468

0.4079

0.5525

0.4525

0.4043

0.19419

644.133

0.5045

0.5422

0.4011

0.5429

0.4602

0.4141

NSGA-II [12]

SPEA [12]

MOBSO

Minimum emissions

FCPSO [12]

SPEA [12]

NSGA-II [12]

Minimum cost

1

Unit

Table 6 Min fuel cost and emission effects for MOBSO and MDBSO for case 2 of test system 1

0.1942

642.8964

0.4974

0.5432

0.4084

0.5510

0.4586

0.4063

FCPSO [12]

0.1962

643.8882

0.5138

0.5426

0.3883

0.5425

0.4624

0.4095

MOBSO

0.1962

643.8548

0.5130

0.5415

0.3889

0.5427

0.4642

0.4089

MDBSO

Multi-objective Differential-Based Brain Storm Optimization … 97

98 Fig. 6 The Pareto front by MOBSO and MDBSO for case 2 of test system 1

Y. Wu et al. 0.225 MDBSO MOBSO

0.22

0.215

0.21

0.205

0.2

0.195 605

610

615

620

625

630

635

640

645

As seen in Table 8, The minimum fuel cost by MDBSO is 121,468 ($/h), compared with MOBSO method, the proposed MDBSO can reduced the fuel cost by about 4525.8 ($/h). But compared with other algorithms, the minimum fuel cost by QOTLBO is 121,428 ($/h), which reduced the fuel cost by about 40 ($/h). The minimum fuel cost by MDBSO which correspond emissions is 358,006 (t/h), compared with QOTLBO method, the emissions by MDBSO is increased by about 4588.2 (t/h). The minimum emissions by MDBSO is 176,682.6 (t/h), which can reduced the emissions with about 140.5 (t/h) compared with MOBSO method. As for the other algorithms, the minimum emission by DE is 176,680 (t/h), which reduced the emission about 2.6 (t/h). The minimum emission by MDBSO which correspond fuel cost is 129,941.6 ($/h). The fuel cost by MDBSO is reduced by about 18.4 ($/h) than that of DE method. In order to more clearly show the solution quality and distribution, the Pareto fronts of the MOBSO and the MDBSO for EED problem are shown in Fig. 7. As it can be seen from Fig. 7, the Pareto-optimal solutions obtained by the proposed MDBSO are well distributed on the front with good diversity. Though the Pareto-optimal solutions obtained by MOBSO are also well distributed, the diversification of them is not as well as the ones obtained by the proposed MDBSO method. When the scale of the problem becomes larger, the MDBSO method has more significant advantages than MOBSO.

8 Conclusions MDBSO is proposed to solve the EED problem of power systems in this chapter. Differential mutation operation is adopted in the proposed algorithm to replace the Gaussian mutation in the MOBSO algorithm for increasing the diversity of the population and the speed of convergence. And the clustering operation is changed into

Multi-objective Differential-Based Brain Storm Optimization …

99

Table 7 The fuel cost, emission and the capacities coefficients of the generating units of test system 2 Unit Capacities P min P max

Cost

Emission β

γ

60

2.22

0.0480 1.3100 0.05690

0.084

60

2.22

0.0480 1.3100 0.05690

0.084

100

2.36

0.0762 1.3100 0.05690

150

0.063

120

3.14

0.0540 0.9142 0.04540

120

0.077

50

1.89

0.0850 0.9936 0.04060

0.01142

100

0.084

80

3.08

0.0854 1.3100 0.05690

8.03

0.00357

200

0.042

100

3.06

0.0242 0.6550 0.02846

391.98

6.99

0.00492

200

0.042

130

2.32

0.0310 0.6550 0.02846

455.76

6.60

0.00573

200

0.042

150

2.11

0.0335 0.6550 0.02846

300

722.82

12.90

0.00605

200

0.042

280

4.34

0.4250 0.6550 0.02846

94

375

635.20

12.90

0.00515

200

0.042

220

4.34

0.0322 0.6550 0.02846

94

375

654.69

12.80

0.00569

200

0.042

225

4.28

0.0338 0.6550 0.02846

13

125

500

913.40

12.50

0.00421

300

0.035

300

4.18

0.0296 0.5035 0.02075

14

125

500

1760.40

8.84

0.00752

300

0.035

520

3.34

0.0512 0.5035 0.02075

15

125

500

1760.40

8.84

0.00752

300

0.035

510

3.55

0.0496 0.5035 0.02075

16

125

500

1760.40

8.84

0.00752

300

0.035

510

3.55

0.0496 0.5035 0.02075

17

220

500

647.85

7.97

0.00313

300

0.035

220

2.68

0.0151 0.5035 0.02075

18

220

500

649.69

7.95

0.00313

300

0.035

222

2.66

0.0151 0.5035 0.02075

19

242

550

647.83

7.97

0.00313

300

0.035

220

2.68

0.0151 0.5035 0.02075

20

242

550

647.81

7.97

0.00313

300

0.035

220

2.68

0.0151 0.5035 0.02075

21

254

550

785.96

6.63

0.00298

300

0.035

290

2.22

0.0145 0.5035 0.02075

22

254

550

785.96

6.63

0.00298

300

0.035

285

2.22

0.0145 0.5035 0.02075

23

254

550

794.53

6.66

0.00284

300

0.035

295

2.26

0.0138 0.5035 0.02075

24

254

550

794.53

6.66

0.00284

300

0.035

295

2.26

0.0138 0.5035 0.02075

25

254

550

801.32

7.10

0.00277

300

0.035

310

2.42

0.0132 0.5035 0.02075

26

254

550

801.32

7.10

0.00277

300

0.035

310

2.42

0.0132 0.5035 0.02075

27

10

150

1055.10

3.33

0.52124

120

0.077

360

1.11

1.8420 0.9936 0.04060

28

10

150

1055.10

3.33

0.52124

120

0.077

360

1.11

1.8420 0.9936 0.04060

29

10

150

1055.10

3.33

0.52124

120

0.077

360

1.11

1.8420 0.9936 0.04060

30

47

97

148.89

5.35

0.01140

120

0.077

50

1.89

0.0850 0.9936 0.04060

31

60

190

222.92

6.43

0.00160

150

0.063

80

2.08

0.0121 0.9142 0.04540

32

60

190

222.92

6.43

0.00160

150

0.063

80

2.08

0.0121 0.9142 0.04540

33

60

190

222.92

6.43

0.00160

150

0.063

80

2.08

0.0121 0.9142 0.04540

34

90

200

107.87

8.95

0.00010

200

0.042

65

3.48

0.0012 0.6550 0.02846

35

90

200

116.58

8.62

0.00010

200

0.042

70

3.24

0.0012 0.6550 0.02846

36

90

200

116.58

8.62

0.00010

200

0.042

70

3.24

0.0012 0.6550 0.02846

37

25

110

307.45

5.88

0.01610

80

0.098

100

1.98

0.0950 1.4200 0.06770

38

25

110

307.45

5.88

0.01610

80

0.098

100

1.98

0.0950 1.4200 0.06770

39

25

110

307.45

5.88

0.01610

80

0.098

100

1.98

0.0950 1.4200 0.06770

40

242

550

647.83

7.97

0.00313

300

0.035

220

2.68

0.0151 0.5035 0.02075

a

b

Gi

Gi

1

36

114

2

36

114

3

60

120

309.54

4

80

190

5

47

97

6

68

7

c

e

f

94.705

6.73

0.00690

100

0.084

94.705

6.73

0.00690

100

7.07

0.02028

100

369.03

8.18

0.00942

148.89

5.35

0.01140

140

222.33

8.05

110

300

287.71

8

135

300

9

135

300

10

130

11 12

α

ξ

λ

110.9515

113.2997

98.6155

184.1487

86.4013

140

300

285.4556

297.511

130

168.7482

95.695

125

394.3545

305.5234

394.7147

489.7972

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

489.2933

304.5265

394.2724

304.5209

304.5206

168.7849

168.7987

130

284.5977

284.5983

259.6596

139.9918

88.2196

179.7438

97.4031

111.2043

111.6943

489.4934

394.2790

394.2920

304.6332

125.0005

168.8099

94

130.0001

285.1639

286.6903

300

140

95.6378

179.7520

97.5481

113.9998

113.9002

488.5362

394.5999

394.0693

393.4638

393.0476

317.5586

311.8760

194.1866

287.1207

287.3968

298.8238

139.4156

96.2819

181.3020

119.8613

111.1557

113.4300

MOBSO

489.4694

394.2754

394.2716

304.5233

214.9194

94.0836

94.0151

130.0202

285.2176

286.8098

259.4159

140

97

180.7563

119.9118

110.8839

114

MDBSO 114

439.2581

422.9508

422.628

421.9529

433.7471

297.7226

298.598

130

297.1332

297.8554

299.4564

124.2828

97

169.2933

120

114

439.4144

422.7808

422.7783

421.7308

433.56

298.0278

298.4145

130

297.2581

297.914

299.7114

124.2561

97

169.3712

119.9998

113.9992

113.9986

QOTLBO [15]

DE [10]

NGPSO [14]

DE [10]

QOTLBO [15]

Minimum emissions

Minimum cost

1

Unit

Table 8 Min fuel cost and emission effects of different methods for case 1 of test system 2

439.5065

422.7987

422.6132

421.6320

433.5537

298.0135

298.4433

130

297.2380

297.9758

299.8181

124.1729

97

169.3484

120

114

114

NGPSO [14]

440.0666

418.6954

420.3047

416.5340

435.7219

298.8741

301.1006

131.0585

298.8705

298.3015

299.5249

122.9590

96.9889

169.1480

119.9056

113.9454

113.9621

MOBSO

(continued)

439.2877

422.6175

422.6007

421.7847

433.5862

298.04713

298.37048

130.0055

297.2228

298.0681

300

124.2352

97

169.4033

120

114

114

MDBSO

100 Y. Wu et al.

MOBSO

MDBSO

489.362

520.9024

510.6407

524.5336

526.6981

530.7467

526.327

525.6537

522.9497

10

11.5522

10

89.9076

190

190

190

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

489.2806

189.9852

189.9994

189.9818

89.744

10.0016

10.0023

10.0046

523.2994

523.2799

523.3315

523.296

523.4356

523.3225

511.2812

511.2893

190

189.9997

189.9992

97

10

10.0161

10.0229

523.3058

523.3953

523.3203

523.3027

523.4957

523.4588

511.2889

511.2770

489.3192

189.9560

180.9190

178.4069

94.1716

10.1185

10.1659

10.1241

434.9752

433.5080

434.8747

435.3166

437.7366

433.5193

428.7140

424.3365

489.1771

190

190

190

96.9998

10.0075

10.0075

10.0075

523.5242

523.4178

523.3393

523.2960

523.3924

523.3618

511.3028

511.2888

489.3266

439.4411

172.3144

172.3956

172.4036

97

28.9047

29.0747

28.7758

439.8408

440.4401

439.8937

439.8757

439.6821

439.225

439.6189

439.4908

172.3304

172.3324

172.3331

97

28.9943

28.9931

28.9934

440.111

440.1155

439.7708

439.768

439.4469

439.446

439.4082

439.4128

439.4038

QOTLBO [15]

DE [10]

NGPSO [14]

DE [10]

QOTLBO [15]

Minimum emissions

Minimum cost

18

Unit

Table 8 (continued)

172.3681

172.3120

172.2440

97

28.9438

29.0956

29.0583

440.2365

440.1966

439.8784

439.8234

439.3784

439.3667

439.4306

439.1695

439.4425

NGPSO [14]

173.5104

171.5497

172.3499

96.9907

27.2928

27.3359

29.1849

443.2574

443.0610

440.6656

436.6943

439.9878

440.9359

438.8644

441.6826

441.6892

MOBSO

(continued)

172.2560

172.2953

172.3751

97

29.0411

28.8954

28.9407

440.2755

440.1077

439.6581

439.8185

439.4588

439.3812

439.5437

439.3835

439.4545

MDBSO

Multi-objective Differential-Based Brain Storm Optimization … 101

MOBSO

MDBSO

198.8403

174.1783

197.1598

110

109.3565

110

510.9752

121,840

374,790

35

36

37

38

39

40

C

E

353,417.8

121,428

511.2796

109.9984

109.9988

109.9968

164.9701

165.0289

165.3627

426.5930

109.8778

109.6526

109.7760

199.02169

199.9010

197.0321

–

196,319.9

121,513.48 125,993.8

511.3150

110

110

110

200

199.9995

196.2840

358,006

121,468

511.3356

110

110

110

199.8487

199.9704

200

200

176,680

129,960

439.1894

100.7784

100.9

100.8765

200

200

176,682.5

129,955

439.4138

100.8378

100.8385

100.8369

199.9998

199.9989

199.9996

QOTLBO [15]

DE [10]

NGPSO [14]

DE [10]

QOTLBO [15]

Minimum emissions

Minimum cost

34

Unit

Table 8 (continued)

129,836

437.6020

99.3682

100.4542

101.5760

199.9990

199.9898

199.9965

MOBSO

176,682.52 176,823.1

–

439.3895

100.8230

100.9019

100.8271

200

200

200

NGPSO [14]

176,682.6

129,941.6

439.4471

100.8105

100.8061

100.8209

200

200

200

MDBSO

102 Y. Wu et al.

Multi-objective Differential-Based Brain Storm Optimization … Fig. 7 The Pareto front by MOBSO and MDBSO for case 1 of test system 2

5

x 10

103

5

MDBSO MOBSO

4.5 4 3.5 3 2.5 2 1.5 1 0.5 1.2

1.25

1.3

1.35

1.4

1.45

1.5

1.55

1.6

1.65 x 10

5

objective space instead of the decision space to reduce the complexity of computation. The different constraints handling mechanism are designed according to the constraints of the EED model. Two cases of the different size of EED problems are verified the performance of the MOBSO and MDBSO. The simulation results shows that compared with MOBSO, MDBSO can provide better dispatch results although both of them can get good result for small size EED problem. MDBSO also shows the better performance compared with other optimization algorithm in solving EED problem. The further work may focus on the better algorithm for more complicit constraints such as ramp rate limits, prohibited operating zone and reliability, etc. Acknowledgements This chapter is supported by National Youth Foundation of China with Grant Number 61503299.

References 1. Abido, M.A.: Multi objective particle swarm optimization for environmental/economic dispatch problem. Electr. Power Syst. Res. 79(7), 1105–1113 (2009) 2. Yalcinoz, T., Köksoy, O.: A multi objective optimization method to environmental economic dispatch. Electr. Power Syst. Res. 29(1), 42–50 (2007) 3. Qu, B.Y., Zhu, Y.S., Jiao, Y.C., et al.: A Survey on multi-objective evolutionary algorithms for the solution of the environmental/economic dispatch problems. Swarm Evol. Comput. 38, 1–11 (2018) 4. Alomoush, M.I., Oweis, Z.B.: Environmental-economic dispatch using stochastic fractal search algorithm. Int. Trans. Electr. Energ. Syst. 3, e2530 (2018) 5. Zhao, B.: Research on Swarm Intelligence Computation and Multi-agent Technique and their Applications to Optimal Operation of Electrical Power System. Zhejiang University, Zhejiang (2005)

104

Y. Wu et al.

6. Abido, A.A.: A new multiobjective evolutionary algorithm for environmental/economic power dispatch. Electr. Power Syst. Res. 65(1), 71–81 (2003) 7. Rose, L.R., Selvi, B.D.A., Singh, R.L.R.: A novel approach to solve environmental economic dispatch problem using Gauss Newton based genetic algorithm 6(6), 1561 (2016) 8. El-Sehiemy, R.A., El-Hosseini, M.A., Hassanien, A.E.: Multiobjective real-coded genetic algorithm for economic/environmental dispatch problem. Stud. Inf. Control 22(2), 113–122 (2013) 9. Lu, Y., Zhou, J., Hui, Q., et al.: Environmental/economic dispatch problem of power system by using an enhanced multi-objective differential evolution algorithm. Energy Convers. Manag. 52(2), 1175–1183 (2011) 10. Basu, M.: Economic environmental dispatch using multi-objective differential evolution. Appl. Soft Comput. J. 11(2), 2845–2853 (2011) 11. Pandit, M., Srivastava, L., Sharma, M.: Environmental economic dispatch in multi-area power system employing improved differential evolution with fuzzy selection. Appl. Soft Comput. 28, 498–510 (2015) 12. Agrawal, S., Panigrahi, B.K., Tiwari, M.K.: Multiobjective particle swarm algorithm with fuzzy clustering for electrical power dispatch. IEEE Trans. Evol. Comput. 12(5), 529–541 (2008) 13. Rose, R.L., Selvi, B.D.A., Singh, R.L.R.: Development of hybrid algorithm based on PSO and NN to solve economic emission dispatch problem. Circuits Syst. 07(9), 2323–2331 (2016) 14. Zou, D., Li, S., Li, Z., et al.: A new global particle swarm optimization for the economic emission dispatch with or without transmission losses. Energy Convers. Manag. 139, 45–70 (2017) 15. Roy, P.K., Bhui, S.: Multi-objective quasi-oppositional teaching learning based optimization for economic emission load dispatch problem. Int. J. Electr. Power Energy Syst. 53(4), 937–948 (2013) 16. Lu, Z.G., Feng, T., Li, X.P.: Low-carbon emission/economic power dispatch using the multiobjective bacterial colony chemotaxis optimization algorithm considering carbon capture power plant. Int. J. Electr. Power Energy Syst. 53(1), 106–112 (2013) 17. Shi, Y.: Brain storm optimization algorithm. In: Advances in Swarm Intelligence Second International Conference, ICSI 2011, pp. 1–3. Chongqing, China, 12–15 June 2011 18. Xue, J., Wu, Y., Shi, Y., et al.: Brain storm optimization algorithm for multi-objective optimization problems. Lect. Notes Comput. Sci. 7331(4), 513–519 (2012) 19. Wu, Y, Xie, L, Liu, Q.: Multi-objective brain storm optimization based on estimating in knee region and clustering in objective-space. In: International Conference in Swarm Intelligence, pp. 479–490. Springer, Cham (2016) 20. Guo, X., Wu, Y., Xie, L., et al.: An adaptive brain storm optimization algorithm for multiobjective optimization problems. Lect. Notes Comput. Sci. 9140(4), 365–372 (2015) 21. Walters, D.C., Sheblé, G.B.: Genetic algorithm solution of economic dispatch with valve point loading. IEEE Trans. Power Syst. 8(3), 1325–1332 (1993) 22. Farag, A., Al-Baiyat, S., Cheng, T.C.: Economic load dispatch multiobjective optimization procedures using linear programming techniques. IEEE Trans. Power Syst. 10(2), 731–738 (1995) 23. Yokoyama, R., Bae, S.H., Morita, T., Sasaki, H.: Multiobjective generation dispatch based on probability security criteria. IEEE Trans. Power Syst. 3(1), 317–324 (1988) 24. Cheng, S, Shi, Y, Qin, Q, et al.: Solution clustering analysis in brain storm optimization algorithm. In: 2013 IEEE Symposium on Swarm Intelligence (SIS), pp. 1391–1399 (2013) 25. Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach. IEEE Trans. Evol. Comput. 3(4), 257–271 (1999) 26. Deb, K., Pratap, A., Agarwal, S., et al.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)

Enhancing the Local Search Ability of the Brain Storm Optimization Algorithm by Covariance Matrix Adaptation Seham Elsayed, Mohammed El-Abd and Karam Sallam

Abstract Recently, the Brain Storm Optimization (BSO) algorithm has attracted many researchers and practitioners attention from the evolutionary computation community. However, like many other population based algorithms, BSO shows good performance at global exploration but not good enough at local exploitation. To alleviate this issue, in this chapter, the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is utilized in the Global-best BSO (GBSO), with the aim to combine the exploration ability of BSO and local ability of CMA-ES and to design an improved version of BSO. The performance of the proposed algorithm is tested by solving 28 classical optimization problems and the proposed algorithm is shown to perform better than GBSO. Keywords Brain storming optimization · Covariance matrix adaptation · Optimization problems

1 Introduction Optimization is an important decision making tool in many fields, including, but not limited to, engineering design, operations research, and data mining. Without loss of generality, a global unconstrained single objective optimization problem, as considered in this chapter, can be stated as finding the values of a decision vector S. Elsayed (B) Faculty of Computer and Informatics, Zagazig University, Zagazig, Egypt e-mail: [email protected] M. El-Abd American University of Kuwait, Salmiya, Kuwait e-mail: [email protected] K. Sallam UNSW, Canberra, Australia e-mail: [email protected] Zagazig University, Zagazig, Egypt © Springer Nature Switzerland AG 2019 S. Cheng and Y. Shi (eds.), Brain Storm Optimization Algorithms, Adaptation, Learning, and Optimization 23, https://doi.org/10.1007/978-3-030-15070-9_5

105

106

S. Elsayed et al.

x = (x1 , x2 , . . . , xD ) ∈ RD , which satisfies the variable bounds, xmin < x < xmax and minimizes or maximizes an objective function f (x), where xmin and xmax are the lower and upper boundaries, respectively. In these problems, the decision variables may be integer, real, discrete, or mixed [20, 21] and the objective function can be linear or nonlinear, convex or non-convex, continuous or not continuous, and uni-modal or multi-modal [19, 22]. In the literature, many population based algorithms have been developed to solve the above mentioned problem. Among them, the Brain Storm Optimization (BSO) algorithm [23, 24], which imitates the brainstorming process performed by humans. In general, brainstorming sessions involve having a group of individuals with diverse expertise and abilities in order to solve a problem at hand. The initial proposed version of the BSO algorithm had a number of shortcomings which included the need to set the number of clusters, the computational complexity of the clustering process, the lack of an efficient mechanism to escape local-minima, and the fixed schedule for updating the step size. Some of these disadvantages are tackled in previous works by improving the clustering process [4, 6, 28], providing an enhanced update method [3, 28, 29], introducing a re-initialization step [9, 10, 12], and proposing a global-best version [13]. In the literature, the BSO algorithm is good at exploration but not good enough at exploitation process [7]. This motivates us to propose a new algorithm that combines CMA-ES’s exploitation ability [14, 22] and GBSO’s exploration ability, with the aim to develop an enhanced version of BSO algorithm. The rest of the chapter is organized as follows: Sect. 2 gives details about the BSO algorithm, covers different improvements proposed in the literature for BSO, and explains CMA-ES. The proposed GBSO is fully detailed in Sect. 3. Section 4 presents the experimental study and the reached results. Finally, the chapter is concluded in Sect. 5.

2 Literature Review and Related Work 2.1 Brain Storm Optimization In BSO, a population is a collection of ideas. Each idea represents a possible solution to the problem at hand. At the beginning, ideas are randomly initialized in the search space using uniform distribution. In every iteration, each idea ideai is updated as follows: • k-means clustering is run to group similar ideas together with the best idea in each cluster is identified as the cluster center, • BSO generates a new idea nideai by making it equal to one of the following four candidates: – A probabilistically selected cluster center, – A randomly selected idea from a probabilistically selected cluster,

Enhancing the Local Search Ability …

107

– A random combination of two probabilistically selected cluster centers, or – A random combination of two randomly selected ideas from two probabilistically selected clusters. One of these four operations is adopted randomly based on three parameters pone−cluster , pone−center , and ptwo−centers . Furthermore, a cluster is probabilistically selected according to its size (i.e. the number of ideas in the cluster), • The generated nideai is perturbed using a step-size parameter ξ and Gaussian distribution, • The current ideai is replaced with the generated nideai if nideai has a better fitness. Otherwise, nideai is discarded. The BSO algorithm is shown in Algorithm 1, where NP is the population size, m is the number of clusters, N (0, 1) represents a Gaussian distribution with mean 0 and a standard deviation of 1. rand is a uniformly distributed random number between 0 and 1. Finally, ξ is a dynamically updated step-size and k is for changing the slope of the logsig() function. For more on BSO, interested readers can refer to [8].

2.2 Previous BSO Improvements A number of improvements have been proposed in the literature to improve the performance of BSO by addressing some of its disadvantages. To remove the clustering computational overhead, the work in [28] adopted a Simple Grouping Strategy (SGM) instead of k-means clustering. In SGM, m seeds were randomly chosen at each iteration, and then each idea in the current population was assigned to the group of the nearest seed. Moreover, generated ideas were perturbed by an idea difference method. Although SGM reduced the computational cost of k-means, it required a large number of distance calculations in the search space. In addition, using SGM has resulted in a slight decline in performance on multi-modal functions. Overall, the algorithm produced better results when compared against PSO, DE, and BSO on a small number of classical functions. Another attempt to eliminate the use of k-means was presented in [4]. The authors proposed the random grouping BSO (RGBSO), which is achieved by randomly dividing the population into m clusters. The best idea in each group was selected as its center. The approach was adopted to simulate random discussions among human. However, it did not incorporate any similarity measures for clustering. The authors also modified the ξ update schedule as follows: Max_Iterations

ξ = rand × e1− Max_Iterations−Current_Iteration+1

(1)

This modification was proposed because the change of ξ in the old formula takes effect for a very small number of generations. The authors in [5] proposed to dynamically set the number of clusters using Affinity Propagation (AP) clustering. AP continuously adapts the number of groups

108

S. Elsayed et al.

Algorithm 1 The BSO algorithm Require: Max_Iterations, NP , m, pone−cluster ,pone−center , and ptwo−centers 1: Randomly initialize n ideas 2: Evaluate the n ideas 3: iter = 1 4: while iter ≤ Max_Iterations do 5: Cluster n ideas into m clusters using k -means 6: Rank ideas in each cluster and select cluster centers 7: for each idea i do 8: if rand < Pone−cluster then 9: Probabilistically select a cluster cr 10: if rand < Pone−center then 11: nideai = centercr 12: else 13: Randomly select an idea j in cluster cr j 14: nideai = ideacr 15: end if 16: else 17: Randomly select two clusters cr1 and cr2 j k 18: Randomly select two ideas cr1 and cr2 19: r = rand 20: if rand < Ptwo−centers then 21: nideai = r × centercr1 + (1 − r) × centercr2 22: else j 23: nideai = r × ideacr1 + (1 − r) × ideack r2 24: end if 25: end if 26: ξ = rand × logsig( 0.5×Max_Iterations−Current_Iteration ) k 27: nideai = nideai + ξ × N (0, 1) 28: if fitness(nideai ) > fitness(ideai ) then 29: ideai = nideai 30: end if 31: end for 32: end while 33: RETURNbest idea

according to their structure information. Comparisons regarding the computational cost of AP against k-means were not reported. Experiments were presented to show how the algorithm produces good deployment of a Wireless Sensor Network. Finally, the authors in [6] used Agglomerative Hierarchical Clustering (AHC) because it does not require pre-specifying the number of clusters. Furthermore, the authors used the quality of new solutions to adjust the probability of creating new ideas using a single or multiple clusters. The proposed algorithm provided better results than BSO for a number of classical functions. For adapting the fixed schedule of updating ξ, the work in [29] proposed an update process using the dynamic range of ideas in each dimension. The algorithm used two parameters ξicenter and ξiindividual as one was used when creating an idea based on cluster center(s) while the second was used when a new individual is created based on a randomly selected individual. In addition, new parameters k1 and k2 were introduced to control the step size but no guidelines were given on how to set these parameters. Experiments showed how step sizes ξicenter and ξiindividual oscillate during the search in comparison to the fast decrease of the step size in the original BSO. In addition to dynamically updating the step size, the authors modified the update equations. The

Enhancing the Local Search Ability …

109

new equations relied on generated ideas using the different ξ parameters proposed, randomly grouping the ideas, and finally copying the best idea in each group to the next population. Evaluating and replacing old population was implemented in batch mode. The developed algorithm produced better results than BSO for a number of classical functions. The width of the search range was taken into account in [12] by incorporating a parameter α into Eq. (1) as shown, where alpha was set as 20% of the search space width. Max_Iterations

ξ = rand × e1− Max_Iterations−Current_Iteration+1 × α

(2)

In addition, the work adopted re-initialization mechanism borrowed from the Artificial Bee Colony (ABC) algorithm [2]. A re-initialization process is initiated when any idea does not improve for a pre-set number of iterations. Moreover, the idea could be either randomly re-initialized or using a differential approach as shown (where F is defined as above while r1 , r2 and r3 are mutually exclusive integers randomly chosen between 1 and n): nideai = idear1 + F × (idear2 − idear3 )

(3)

The proposed algorithm (IRGBSO) provided better results over the works in [3, 4] on the CEC2015 benchmarks [18]. In [3], the authors used differential evolution equations for generating new ideas while the same step-size update in [4]. The algorithm was named BSODE. For the intra-cluster operator, the idea was generated as shown (where F is the mutation scaling factor while r1 and r2 are mutually exclusive integers randomly chosen in the selected cluster): r1 r2 − ideacr ) nideai = centercr + F × (ideacr

(4)

For the inter-cluster operator, the idea was generated as shown (where F is defined as above while Global_Idea is the best idea in the population): j

k ) nideai = Global_Idea + F × (ideacr1 − ideacr2

(5)

Their algorithm improved the performance over BSO with a faster speed of convergence. The performance was also competitive with PSO, CoDE [27], and SaDE [17] on the CEC05 benchmarks [26]. The work in [9, 10] improved the population diversity of BSO by re-initializing a small part of the population every a pre-specified number of iterations. Experiments were conducted to study the effect of changing the percentage of re-initialized ideas. The approached resulted in improving the BSO performance and enhancing the population diversity. The authors carried experiments showing how the diversity of the population in their algorithm is higher than the diversity of the population in the

110

S. Elsayed et al.

original BSO. In addition, their algorithm improved the performance of BSO on a number of classical functions. In [13], the author improved the performance of BSO by adopting a fitnessbased grouping mechanism to make sure good ideas are grouped in different groups, using the global-best idea information for updating the population, and applying the update scheme on every problem variable separately. The global-best tracking was implemented after generating the new idea nideai as follows: nideai = nideai + rand (1, DimSize) × C × (GlobalBest − nideai )

(6)

while the parameter C was gradually decreased during the search as follows: C = Cmin +

CurrentIteration × (Cmax − Cmin ) MaxIterations

(7)

The algorithm was referred to as Global-best BSO (GBSO). Experimental results have shown that GBSO has a better performance than BSODE, RGBSO, and IRGBSO on a suite of 20 well-known benchmark functions. Moreover, GBSO was proven better than the 2011 version of Standard PSO (SPSO) [25], Global-best guided ABC (GABC) [30] and the Improved Global-best Harmony Search (IGHS) [11] on the CEC05 [26] and the CEC14 [16] benchmarks.

2.3 Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES) CMA-ES, developed by Hansen [14], is an EAs which has been shown to efficiently solve diverse types of optimization problems. It is considered as state-of-the-art in evolutionary algorithms (EAs) and has been adopted as one of the standard tools for continuous optimization in many (probably hundreds of) research labs and industrial environments around the world that is proved to converge at optimal solution in very few generations. It was derived from the concept of self-adaptation in ES, which adapts the co-variance matrix of a multivariate normal search distribution. In CMAES, the new individuals are generated by sampling from a Gaussian distribution, and instead of using a single mutation step, it considers the path that the population takes over generations [15]. Below is a list of some of the key features of CMA-ES: • CMA-ES shows good performance on badly conditioned and/or non-separable problems as it is an approach estimating a positive definite matrix within an iterative procedure. • It seems to be a very reliable and extremely competitive EA for local optimization and, surprising at first sight, also for global one. • CMA-ES is a gradient free algorithm, that means it does not utilize or approximate gradients and does not even presume or need their existence. Thus, CMA-ES on

Enhancing the Local Search Ability …

111

non-smooth and even non-continuous problems, as well as on multi-modal and/or noisy problems. • The CMA-ES doesn’t need tedious parameter tuning for its application. In fact, the selection of strategy internal parameters is totally automatic.

Algorithm 2 CMA-ES algorithm 1: 2: 3: 4: 5: 6: 7: 8:

Generate an initial solution xm and evaluate the fitness function; Sample new individuals, such that xz,t+1 = xm + σN (0, Ct ) ∀z = 1 : NP; Evaluate the new offspring and sort them based on their fitness values. the best μ individuals are selected a parental vector, and their center is calculated according μ as μ to xm,t+1 = k=1 wk xk,t where k=1 wk = 1 and w1 ≥ w2 ≥ · · · ≥ wμ . s σ . Update the evolution path pt+1 and pt+1 Adapt the covariance matrix (Ct+1 ). Update global step size (σt+1 ). Repeat steps 2 to 7 until a stopping criterion is met.

Algorithm 2 shows the main steps of CMA-ES. CMA-ES consists of the following main steps (Generate Sample Population, selection and recombination, Covariance matrix adaptation and variance adaptation).

2.3.1

Generate Sample Population

The population vectors of a new search point at generation (g + 1) are produced in the CMA-ES by sampling a multi-variate of the normal distribution, which takes the help of the following equation: xz,t+1 = xm + σN (0, Ct ) ∀z = 1 : NP;

(8)

where the mean vector xm represents the favorite solution, the so-called step-size σ controls the step length and the covariance matrix C determines the shape of the distribution.

2.3.2

Selection and Recombination

The next distribution mean xm,t+1 is calculated as the weighted mean of the best μ individuals, by using the following equation: xm,t+1 =

μ 

wk xk,t

k=1

where

μ k=1

wk = 1 and w1 ≥ w2 ≥ · · · ≥ wμ .

(9)

112

2.3.3

S. Elsayed et al.

Evolution Path

The evolution path records the sum of consecutive update steps, i.e, μt+1 − μt . In case of consecutive update steps are towards the same direction, their updates will sum up. on the other hand, if they are towards different directions, the update directions cancel each other out. Using the information from the evolution path leads to significant improvements in terms of convergence speed, as it enables the algorithm to exploit correlations between consecutive steps. 2.3.4

Covariance Matrix (Ct+1 ) Adaptation

The covariance matrix update in step 6 consists of three parts: old information, rank-1 update in which the change of the mean over time as encoded in the evolution path Pc is computed, and rank-μ update which considers the good variations in the last population. The covariance update is defined as: Ct+1 = (1 − c1 − cμ ) × Ct + c1 Pt PtT + cμ

μ 

T wi yi:λ yi:λ

(10)

1

where Pc is the evolution path and calculated by the following equation: s Pt+1 = (1 − cc ) × Pts +

2.3.5



cc (2 − cc )μw y¯

(11)

Step Size Update

To update the step size (σ), the evolution path is used to find a new step size, such that the difference between the distributions of the actual evolution path and an evolution path under random selection is minimized. 

σt+1

cσ = σt exp dσ



 ||Pσ || −1 ||N (0, I )||

(12)

where Pσ is calculated by the following equation: Pσ = (1 − cσ ) × Pσ +



cσ (2 − cσ )μw C − 2 y¯ 1

(13)

Enhancing the Local Search Ability …

113

3 Proposed Algorithm In this section the proposed algorithm will be discussed. Algorithm 3 presents the steps of the proposed GBSO-CMA-ES algorithm. In the proposed algorithm the GBSO is used to evolve a population of individuals (number of ideas) as described in Sect. 2.2 for the first 85% of the maximum number of fitness evaluations. This provides a good exploration step in which GBSO identifies promising regions in the search space. Algorithm 3 The GBSO-CMA-ES algorithm Require: Max_Iterations, MAXFES , n, m, pone−cluster ,pone−center , and ptwo−centers , Step, Cmin , Cmax , FES ← 0, NP 1: Randomly initialize n ideas 2: Evaluate the n ideas 3: FES ← FES + NP 4: iter = 1 5: while FES ≤ MAXFES do 6: Perform fitness-based grouping 7: for each idea i do 8: for each problem variable j do 9: if rand < Pone−cluster then 10: Probabilistically select a cluster cr 11: if rand < Pone−center then j 12: nideaij = centercr 13: else 14: Randomly select an idea k in cluster cr kj 15: nideaij = ideacr 16: end if 17: else 18: Randomly select two clusters cr1 and cr2

19: 20: 21: 22: 23: 24: 25: 26: 27: 28: 29: 30: 31: 32: 33: 34: 35: 36: 37: 38: 39: 40:

k

k

Randomly select two ideas cr11 and cr22 r = rand if rand < Ptwo−centers then j j nideaij = r × centercr1 + (1 − r) × centercr2 else k1 j k2 j nideaij = r × ideacr1 + (1 − r) × ideacr2 end if end if end for C = Cmin + Current_Iteration−1 × (Cmax − Cmin ) Max_Iterations if C < rand then nideai = nideai + rand (1, DimSize) × C × (Global _Best − nideai ) end if 1−

Max_Iterations

ξ = rand × e Max_Iterations−Current_Iteration+1 × Step nideai = nideai + ξ × N (μ, 0) end for Update population and FES ← FES + NP if FES >= 0.85 × MAXFES then Apply CMA-ES every generation for 20 random Individual (See Algorithm 2) end if end while RETURNbest idea

114

S. Elsayed et al.

Afterwards, CMA-ES is applied at every generation with the GBSO to evolve the entire population until the stopping condition is met. This is to provide strong local search ability in the regions already identified by GBSO. CMA-ES is used to evolve to 20 randomly selected individuals to avoid strong bias towards local optima.

4 Experimental Results and Analysis In this section, the performances of the proposed algorithm for solving a benchmark test set of 28 classical optimization problems is discussed. The mathematical proprieties of these functions are presented in Appendix 1. The proposed algorithm was executed for 25 runs for each problem with up to a maximum number of fitness evaluations (FESmax ) = 10,000 ×D. It was coded using Matlab R2017a and run on a PC with a 3.4 GHz Core I7, 16 GB RAM, and Windows 7.

4.1 Algorithm Parameters For both algorithms (GBSO and GBSO-CMA-ES), the same parameters settings have been used. The population size PopSize = 25, the number of clusters m = 5, the probability of one cluster pone−cluster = 0.8, the probability of one center pone−center = 0.4, the probability of two centers ptwo−centers = 0.5, the Threshold = 10, F = 0.5, α = 0.05 × (U B − LB) where LB and U B are the lower and upper bounds of the search space, Cmin = 0.2, and Cmax = 0.8. For the local search, we apply it for the last 15% of the total number of fitness evaluations for 20 random individuals.

4.2 Detailed Results from GBSO-CMA-ES The proposed algorithm is compared against GBSO, which already outperformed BSODE, RGBSO, and IRGBSOd in [13]. The computational results of the from GBSO-CMA-ES for 10D, 20D and 30D problems are shown in Tables 1, 2 and 3. From the results obtained, it is clear that GBSO-CMA-ES performs better than GBSO in most of the problems. Regarding the best results obtained, GBSO-CMA-ES is superior to GBSO in 18, 18, and 17 functions for 10D, 20D, and 30D, respectively, GBSO-CMA-ES is inferior to GBSO in 0, 1, and 2 functions for 10D, 20D, and 30D, respectively, and both algorithms obtained similar results in 10, 9, and 9 functions for 10D, 20D, and 30D, respectively. Regarding the average results obtained, GBSO-CMA-ES is superior to GBSO in 17, 19, and 19 functions for 10D, 20D, and 30D, respectively, GBSO-CMA-ES is inferior to GBSO in 2, 3, and 3 functions for 10D, 20D, and 30D, respectively, and

Enhancing the Local Search Ability …

115

Table 1 The function values obtained by GBSO and GBSO-CMA-ES over 25 runs for 10D Algorithms

GBSO

GBSO-CMA-ES

Best

Mean

Std.

Best

Mean

Std.

F01

9.311E−23

1.389E−21

2.699E−21

2.620E−40

2.685E−36

1.061E−35

F02

5.639E−12

1.142E−11

6.097E−12

4.441E−15

4.441E−15

0.000E+00

F03

2.618E−01

2.153E+00

6.966E−01

3.017E−24

2.609E−08

1.290E−07

F04

1.315E+00

7.489E+01

8.645E+01

5.209E−08

8.136E+01

9.286E+01

F05

2.910E−11

2.621E−10

6.288E−10

5.334E−19

1.581E−17

2.402E−17

F06

3.769E−11

1.573E−09

2.497E−09

4.215E−18

1.908E−16

1.826E−16

F07

5.163E−11

3.006E−10

7.535E−10

4.088E−19

6.872E−17

1.928E−16

F08

2.188E−121 1.222E−97

6.110E−97

2.118E−178 8.351E−158 4.140E−157

F09

3.092E−06

3.245E−05

2.797E−05

2.126E−15

1.098E−13

1.718E−13

F10

1.520E−16

5.577E−13

1.182E−12

1.121E−16

3.231E−15

4.038E−15

F11

6.667E−01

6.667E−01

6.246E−11

6.667E−01

6.667E−01

2.266E−16

F12

3.953E−17

2.178E−14

6.011E−14

2.699E−35

3.428E−33

7.191E−33

F13

1.240E−22

1.092E−16

3.835E−16

8.364E−30

1.241E−29

2.950E−30

F14

3.410E−13

1.263E−12

1.280E−12

7.820E−18

2.760E−13

1.380E−12

F15

2.370E+04

2.370E+04

1.015E−11

2.370E+04

2.370E+04

6.765E−12

F16

1.226E−25

5.551E−24

1.231E−23

2.081E−39

4.613E−37

1.254E−36

F17

2.527E−02

2.527E−02

8.469E−18

2.527E−02

2.527E−02

6.661E−12

F18

1.000E+00

1.000E+00

2.266E−17

1.000E+00

1.000E+00

0.000E+00

F19

2.557E−01

5.761E−01

1.451E−01

5.708E−02

4.101E−01

2.879E−01

F20

1.000E+00

1.000E+00

2.266E−17

1.000E+00

1.000E+00

0.000E+00

F21

9.987E−02

1.119E−01

3.317E−02

9.987E−02

1.119E−01

3.317E−02

F22

1.256E−44

3.566E−38

1.716E−37

3.361E−78

5.757E−70

2.788E−69

F23

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

F24

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

F25

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

F26

2.034E−22

2.685E−18

5.302E−18

4.349E−39

2.439E−35

4.754E−35

F27

6.725E−01

2.280E+00

8.230E−01

5.190E−01

2.175E+00

8.961E−01

F28

5.661E−04

8.525E−04

3.832E−04

5.661E−04

1.150E−03

3.865E−04

both algorithms obtained similar results in 9, 6, and 6 functions for 10D, 20D, and 30D, respectively. Considering the median result obtained, GBSO-CMA-ES is superior to GBSO in 17, 17, and 16 functions for 10D, 20D, and 30D, respectively, GBSO-CMA-ES is inferior to GBSO in 2, 3, and 4 functions for 10D, 20D, and 30D, respectively, and both algorithms obtained similar results in 9, 9, and 8 functions for 10D, 20D, and 30D, respectively. To sum up GBSO-CMA-ES performs very well for the unimodal problems, the algorithm obtained the optimal solution and near-optimal solutions in regard to the

116

S. Elsayed et al.

Table 2 The function values obtained by GBSO and GBSO-CMA-ES over 25 runs for 20D Algorithms

GBSO

GBSO-CMA-ES

Best

Mean

Std.

Best

Mean

Std.

F01

9.035E−24

1.625E−22

2.986E−22

4.668E−50

2.247E−46

5.237E−46

F02

9.885E−13

4.801E−12

6.965E−12

4.441E−15

5.009E−15

1.329E−15

F03

9.110E+00

1.329E+01

9.530E−01

9.938E−02

1.165E+00

8.688E−01

F04

5.330E+01

4.284E+02

6.343E+02

1.666E−01

4.342E+02

6.547E+02

F05

1.606E−11

3.801E−10

8.259E−10

1.481E−22

7.946E−21

1.356E−20

F06

1.622E−09

2.539E−07

4.026E−07

3.817E−18

5.696E−16

1.277E−15

F07

9.713E−11

1.025E−08

4.827E−08

8.534E−23

8.715E−21

2.457E−20

F08

6.526E−108 1.014E−87

4.962E−87

4.423E−195 2.504E−172 0.000E+00

F09

2.193E−04

1.470E−03

8.446E−04

6.160E−09

8.794E−08

6.106E−08

F10

7.195E−17

1.939E−14

4.427E−14

2.307E−13

6.233E−12

4.855E−12

F11

6.667E−01

6.667E−01

1.618E−07

6.667E−01

6.667E−01

2.266E−16

F12

9.771E−11

1.387E−09

1.403E−09

2.112E−34

1.669E−32

3.160E−32

F13

6.146E−25

3.525E−15

8.128E−15

3.212E−29

6.440E−29

1.446E−29

F14

2.286E−13

5.996E−06

2.998E−05

2.175E−17

9.949E−05

4.969E−04

F15

2.638E+08

4.888E+08

1.628E+08

2.638E+08

4.524E+08

1.557E+08

F16

1.521E−26

1.304E−24

4.121E−24

1.051E−48

1.857E−46

3.348E−46

F17

5.054E−02

5.054E−02

1.780E−17

5.054E−02

5.054E−02

3.246E−11

F18

1.000E+00

1.000E+00

0.000E+00

1.000E+00

1.000E+00

0.000E+00

F19

1.290E+00

1.717E+00

1.231E−01

6.946E−02

1.028E+00

7.293E−01

F20

1.000E+00

1.000E+00

0.000E+00

1.000E+00

1.000E+00

0.000E+00

F21

1.999E−01

2.719E−01

5.416E−02

1.999E−01

2.639E−01

5.686E−02

F22

6.110E−45

2.511E−38

1.235E−37

7.845E−93

2.456E−84

9.109E−84

F23

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

F24

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

F25

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

F26

5.841E−16

2.308E−12

4.005E−12

6.860E−45

2.513E−42

6.096E−42

F27

6.222E+00

6.585E+00

1.723E−01

8.827E−01

5.174E+00

1.959E+00

F28

2.018E−07

2.285E−07

1.966E−08

1.602E−07

2.043E−07

2.000E−08

best and mean solutions. For the multi-modal problems, it was able to obtain near optimal solutions for most of the functions. Considering the average ranking results calculated by the Friedman test results (Table 4), GBSO-CMA-ES was the best algorithm. Statistically, a Wilcoxon test was carried out, with the results showing that GBSOCMA-ES was significantly better than GBSO for 10D, 20D and 30D for best, median and mean results. Table 5 shows a comparison summary of the results obtained from GBSO-CMAES and GBSO for 10D, 20D, and 30D problems. In Table 5 last column (+ means there

Enhancing the Local Search Ability …

117

Table 3 The function values obtained by GBSO and GBSO-CMA-ES over 25 runs for 30D Algorithms

GBSO

GBSO-CMA-ES

Best

Mean

Std.

Best

Mean

Std.

F01

3.768E−24

4.381E−22

1.923E−21

4.122E−51

2.595E−48

9.526E−48

F02

3.135E−13

1.691E−12

1.830E−12

4.441E−15

6.715E−15

1.740E−15

F03

2.264E+01

2.424E+01

5.883E−01

8.916E+00

1.109E+01

9.269E−01

F04

1.612E+02

1.499E+03

1.232E+03

1.435E+00

1.213E+03

1.167E+03

F05

9.806E−11

2.789E−07

1.220E−06

1.403E−22

2.213E−20

6.155E−20

F06

2.400E−06

3.918E−05

8.338E−05

1.709E−14

1.319E−12

3.389E−12

F07

1.476E−10

1.042E−07

4.571E−07

9.612E−23

1.727E−20

7.436E−20

F08

5.412E−97

1.320E−76

6.588E−76

3.685E−179 1.323E−155 6.530E−155

F09

8.392E−04

3.829E−03

1.491E−03

1.193E−07

6.430E−07

4.016E−07

F10

1.907E−17

5.703E−15

8.454E−15

4.234E−12

1.209E−11

6.747E−12

F11

6.667E−01

6.668E−01

7.496E−04

6.667E−01

6.667E−01

2.322E−16

F12

1.589E−08

3.458E−07

2.490E−07

4.946E−28

2.904E−26

3.668E−26

F13

2.716E−18

3.210E−14

6.506E−14

1.272E−28

1.983E−28

3.124E−29

F14

5.308E−13

1.809E−05

6.261E−05

8.714E−17

6.855E−04

1.079E−03

F15

1.774E+12

5.840E+12

2.828E+12

1.774E+12

5.130E+12

2.748E+12

F16

7.673E−27

4.463E−25

5.951E−25

8.889E−52

3.431E−48

9.516E−48

F17

7.582E−02

7.582E−02

2.023E−17

7.582E−02

7.582E−02

5.392E−11

F18

1.000E+00

1.000E+00

9.065E−17

1.000E+00

1.000E+00

8.171E−17

F19

2.703E+00

2.814E+00

6.685E−02

5.542E−01

1.756E+00

9.324E−01

F20

1.000E+00

1.000E+00

9.065E−17

1.000E+00

1.000E+00

8.171E−17

F21

2.999E−01

4.199E−01

6.455E−02

2.999E−01

4.079E−01

6.400E−02

F22

2.071E−46

6.985E−39

3.090E−38

1.084E−92

2.908E−85

1.291E−84

F23

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

F24

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

F25

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

0.000E+00

F26

1.776E−13

1.810E−09

4.501E−09

1.071E−43

1.700E−41

2.951E−41

F27

9.631E+00

1.029E+01

2.041E−01

1.369E+00

6.996E+00

3.346E+00

F28

1.474E−11

1.629E−11

9.448E−13

1.534E−11

1.759E−11

1.338E−12

Table 4 Ranks of proposed algorithm and GBSO, based on Friedman rank test Dimension Algorithm Ranking for best Ranking for Ranking for mean median 10D 20D 30D

GBSO-CMA-ES GBSO GBSO-CMA-ES GBSO GBSO-CMA-ES GBSO

1.18 1.82 1.20 1.80 1.23 1.77

1.23 1.77 1.23 1.77 1.29 1.71

1.23 1.77 1.21 1.79 1.21 1.79

118

S. Elsayed et al.

Table 5 Summary of comparisons of proposed algorithm and GBSA based on best, median and mean results, where ‘Dec.’ statistical decision based on Wilcoxon signed-rank test results Criteria Algorithms Better Equal Worse Prob. Dec. 10D

20D

30D

Best Mean Median Best Mean Median Best Mean Median

GBSO-CMA-ES versus GBSO GBSO-CMA-ES versus GBSO GBSO-CMA-ES versus GBSO GBSO-CMA-ES versus GBSO GBSO-CMA-ES versus GBSO GBSO-CMA-ES versus GBSO GBSO-CMA-ES versus GBSO GBSO-CMA-ES versus GBSO GBSO-CMA-ES versus GBSO

18 17 17 18 19 17 17 19 16

10 9 9 9 6 9 9 6 8

0 2 2 1 3 2 2 3 4

0.000 0.014 0.014 0.000 0.007 0.006 0.002 0.002 0.030

+ + + + + + + + +

Fig. 1 Comparison of performance profiles of proposed algorithm and GBSO. a Best 10D, b median 10D, and c mean 10D

Enhancing the Local Search Ability …

119

is a significant difference between the proposed algorithm and the other algorithm, and = means there is no significant difference). From the results, it is clear that, the proposed algorithm is superior to the other state-of-the-art algorithms. Furthermore, to graphically compare performances of both algorithms, the performance profiles method [1] was used. This method is a tool to compare the performances, of a number of algorithms (S) over a set of problems (p), and a comparison goal (such as the computational time and average number of fitness evaluations) to obtain a certain level, of a performance indicator (such as optimal fitness). The performance profile Rhos is calculated by Rhos (τ ) =

1 × |p ∈ P : rp,s ≤ τ | np

(14)

where Rhos (τ ) is the probability for an algorithm s ∈ S that a performance ratio rp,s is within a factor τ ∈ R of the best possible ratio. The function Rhos is the (cumulative) distribution function for the performance ratio.

Fig. 2 Comparison of performance profiles of proposed algorithm and GBSO. a Best 20D, b median 20D, and c mean 20D

120

S. Elsayed et al.

Fig. 3 Comparison of performance profiles of proposed algorithm and GBSO. a Best 30D, b median 30D, and c mean 30D

Based on the above-mentioned measure, the performance profiles of best, mean and median for 10D, 20D, and 30D are depicted in Figs. 1, 2, and 3, respectively. The proposed GBSO-CMA-ES is better for best, median, and mean for 10D, 20D and 30D. As for example at Fig. 1a GBSO-CMA-ES are able to reach probability of 1 first with τ of 1.

5 Conclusion and Future Work The Covariance matrix adaptation evolution strategy (CMA-ES) is used as a local search at the end of the evolutionary process to improve the search ability of GBSO. The proposed GBSO-CMA-ES algorithm is able to obtain better results than GBSO, because of the better balance between these two algorithms. In the future, we will investigate the performance of the proposed algorithm by solving more benchmark problems.

Enhancing the Local Search Ability …

121

Appendix The benchmark problems solved in this chapter can be found in the following link: Classical optimization problems.

References 1. Barbosa, H.J., Bernardino, H.S., Barreto, A.M.: Using performance profiles for the analysis and design of benchmark experiments. In: Advances in Metaheuristics, pp. 21–36. Springer (2013) 2. Basturk, B., Karaboga, D.: An artificial bee colony (ABC) algorithm for numeric function optimization. In: Proceedings of the IEEE Swarm Intelligence Symposium (2006) 3. Cao, Z., Hei, X., Wang, L., Shi, Y., Rong, X.: An improved brain storm optimization with differential evolution strategy for applications of ANNs. Math. Probl. Eng. 2015, 1–18 (2015) 4. Cao, Z., Shi, Y., Rong, X., Liu, B., Du, Z., Yang, B.: Random grouping brain storm optimization algorithm with a new dynamically changing step size. In: Proceedings of the International Conference on Swarm Intelligence, pp. 387–364 (2015) 5. Chen, J., Cheng, S., Chen, Y., Xie, Y., Shi, Y.: Enhanced brain storm optimization algorithm for wireless sensor networks deployment. In: Proceedings of the International Conference on Swarm Intelligence, pp. 373–381 (2015) 6. Chen, J., Wang, J., Cheng, S., Shi, Y.: Brain storm optimization with agglomerative hierarchical clustering analysis. In: Proceedings of the 7th International Conference on Swarm Intelligence, ICSI, pp. 115–122 (2016) 7. Chen, W., Cao, Y., Sun, Y., Liu, Q., Li, Y.: Improving brain storm optimization algorithm via simplex search (2017). arXiv:1712.03166 8. Cheng, S., Qin, Q., Chen, J., Shi, Y.: Brain storm optimization algorithm: a review. Artif. Intell. Rev. 1–14 (2016) 9. Cheng, S., Shi, Y., Qin, Q., Ting, T.O., Bai, R.: Maintaining population diversity in brain storm optimization algorithm. In: Proceedings of IEEE Congress on Evolutionary Computation, pp. 3230–3237 (2014) 10. Cheng, S., Shi, Y., Qin, Q., Zhang, Q., Bai, R.: Population diversity maintenance in brain storm optimization algorithm. J. Artif. Intell. Soft Comput. Res. 4(2), 83–97 (2014) 11. El-Abd, M.: An improved global-best harmony search algorithm. Appl. Math. Comput. 222, 94–106 (2013) 12. El-Abd, M.: Brain storm optimization algorithm with re-initialized ideas and modified step size. In: Proceedings of IEEE Congress on Evolutionary Computation, pp. 2682–2686 (2016) 13. El-Abd, M.: Global-best brain storm optimization algorithm. Swarm and Evolutionary Computation 37, 27–44 (2017) 14. Hansen, N.: Benchmarking a bi-population CMA-ES on the BBOB-2009 function testbed. In: Proceedings of the 11th Annual Conference Companion on Genetic and Evolutionary Computation Conference: Late Breaking Papers, pp. 2389–2396. ACM (2009) 15. Hansen, N., Müller, S.D., Koumoutsakos, P.: Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES). Evol. Comput. 11(1), 1–18 (2003) 16. Liang, J.J., Qu, B.Y., Suganthan, P.N.: Problem definitions and evaluation criteria for the CEC 2014 special session and competition on single objective real-parameter numerical optimization. Technical Report, Computational Intelligence Laboratory, Zhengzhou University, Zhengzhou, China and Nanyang Technological University, Singapore (2014). http://www.ntu. edu.sg/home/epnsugan/

122

S. Elsayed et al.

17. Qin, A.K., Huang, V.L., Suganthan, P.N.: Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Trans. Evol. Comput. 13(2), 398–417 (2009) 18. Qu, B.Y., Liang, J.J., Wang, Z.Y., Chen, Q., Suganthan, P.N.: Novel benchmark functions for continuous multimodal optimization with comparative results. Swarm Evol. Comput. 26, 23–34 (2016) 19. Sallam, K.M., Elsayed, S.M., Sarker, R.A., Essam, D.L.: Two-phase differential evolution framework for solving optimization problems. In: 2016 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–8. IEEE (2016) 20. Sallam, K.M., Elsayed, S.M., Sarker, R.A., Essam, D.L.: Differential evolution with landscapebased operator selection for solving numerical optimization problems. In: Intelligent and Evolutionary Systems, pp. 371–387. Springer (2017) 21. Sallam, K.M., Elsayed, S.M., Sarker, R.A., Essam, D.L.: Landscape-based adaptive operator selection mechanism for differential evolution. Inf. Sci. 418, 383–404 (2017) 22. Sallam, K.M., Elsayed, S.M., Sarker, R.A., Essam, D.L.: Multi-method based orthogonal experimental design algorithm for solving CEC2017 competition problems. In: 2017 IEEE Congress on Evolutionary Computation (CEC), pp. 1350–1357. IEEE (2017) 23. Shi, Y.: Brain storm optimization algorithm. In: Proceedings of the International Conference on Swarm Intelligence, pp. 303–309 (2011) 24. Shi, Y.: An optimization algorithm based on brainstorming process. Int. J. Swarm Intell. Res. 2(4), 35–62 (2011) 25. Standard PSO 2011 (2011). http://www.particleswarm.info 26. Suganthan, P.N., Hansen, N., Liang, J.J., Deb, K., Chen, Y.P., Auger, A., Tiwari, S.: Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. Technical Report No. 2005005, ITT Kanpur, India (2005) 27. Wang, Y., Cai, Z., Zhang, Q.: Differential evolution with composite trial vector generation strategies and control parameters. IEEE Trans. Evol. Comput. 15(1), 55–66 (2011) 28. Zhan, Z., Zhang, J., Shi, Y., Liu, H.: A modified brain storm optimization algorithm. In: Proceedings of IEEE Congress on Evolutionary Computation, pp. 1969–1976 (2012) 29. Zhou, D., Shi, Y., Cheng, S.: Brain storm optimization algorithm with modified step-size and individual generation. In: Proceedings of the International Conference on Swarm Intelligence, pp. 243–252 (2012) 30. Zhu, G., Kwong, S.: Gbest-guided artificial bee colony algorithm for numerical function optimization. Appl. Math. Comput. 217(7), 3166–3173 (2010)

Brain Storm Algorithm Combined with Covariance Matrix Adaptation Evolution Strategy for Optimization Yang Yu, Lin Yang, Yirui Wang and Shangce Gao

Abstract With the development of computational intelligence, many intelligence algorithms have attracted the attention of the scientific community, and a great deal of work on optimizing these algorithms is in full swing. One of the optimization techniques that we focus on is the hybridization of algorithms. Brain storm optimization algorithm (BSO), belonging to the swarm intelligence algorithms, is proposed by taking inspiration of human brain storming behavior. Meanwhile, the covariance matrix adaptive evolutionary strategy algorithm (CMA-ES) which belongs to the field of evolutionary strategy is also concerned. The purpose of this paper is to combine the search capability of BSO with the search efficiency of CMA-ES to achieve a relatively balanced and effective solution. Keywords Brain storm optimization · Evolutionary strategy · Covariance matrix adaptive evolution strategy · Hybrid algorithms · Parallel computation

1 Introduction Over the past half century, there is an increasing interest in computational intelligence which is inspired by the biological wisdom and the laws of nature. A large number of algorithms such as genetic algorithm (GA) [17], artificial neural network (ANN) [16], simulated annealing (SA), ant colony algorithm (ACO) [13], particle swarm optimization (PSO) [55], and brain storm optimization algorithm (BSO) are proposed during this period [52, 67]. However, no matter how different the mechanisms are, all these algorithms have the same goal, which is to find the optimal solution. For the purpose of finding the optimal solution, each algorithm shows its special prowess. ANN simulates the process of decision-making in animals’ nerves, while GA attempts to find good individuals by the evolutionary theory and genetics. PSO believes the wisdom of swarm behavior of organisms helps to find the optimal Y. Yu · L. Yang · Y. Wang · S. Gao (B) Faculty of Engineering, University of Toyama, Toyama-shi 930-8555, Japan e-mail: [email protected] © Springer Nature Switzerland AG 2019 S. Cheng and Y. Shi (eds.), Brain Storm Optimization Algorithms, Adaptation, Learning, and Optimization 23, https://doi.org/10.1007/978-3-030-15070-9_6

123

124

Y. Yu et al.

solution. They have obtained considerable successes in the field of computational intelligence. Nevertheless, in many previous studies, one thing seems no need to prove that is the original algorithms still have a large space to improve. A problem that all algorithms can not avoid is the balance between exploration and exploitation. Under this situation, it has been decades for people to work on algorithm optimization in order to improve their performances. One quite effective and widely accepted method is hybridization. A hybrid method helps to balance the performance of algorithm between exploration and exploitation. On the other hand, it can also make the universality and specialization become controllable. Table 1 is intended to give some examples of hybrid algorithms, but not to give a complete survey. Several examples in Table 1 are briefly described as follows: Li et al. propose a robust algorithm that combines local search in PSO and global search in ABC algorithm [34]. Sun et al. propose a gray neural network and initialize it with BSO. This hybrid approach effectively helps GNN solve the problem of falling into local optimum and low prediction accuracy. Another hybrid algorithm, BSODE which is proposed by Cao et al. combines a differential evolution strategy and a new step size scheme, which enables BSO to increase the exploration capability while maintaining the population diversity [2]. In the novel scheme suggested by Tran et al., the differential evolution operator is introduced into the artificial bee colony algorithm (ABC) [24], which simultaneously improves the efficiency and accuracy of the algorithm [60]. Zheng et al. enhance the information exchange between fireworks and sparks in the fireworks optimization and make the search process more diversified by introducing the differential evolution operator [68]. Lin et al. propose a scheme combining an evolutionary algorithm and an immune algorithm, which improves the capabilities and robustness of the algorithm and enables it to handle many types of multiobjective optimization problems [37]. Whether or not be mentioned above, examples listed in Table 1 prove one thing, that is the hybrid approach offers the potential to improve the performances of algorithms. In life, we often encounter problems that cannot be solved by a single person. To solve the problem, talents from various fields gathered to provide new solutions continuously. Such a process is called brainstorming. Inspired by this human group activity, to solve various problems in the field of computational intelligence, the brain storm optimization algorithm simulates the brainstorming process to explore more acceptable solutions. On the other hand, an evolutionary strategy algorithm, which uses covariance matrix adaptation, namely covariance matrix adaptive evolutionary strategy (CMA-ES), is also being applied to many optimization problems [22]. In this paper, we combine BSO and CMA-ES in a parallel way to get a better balance and improve the search performance of the algorithm. The new algorithm is abbreviated as BSO-CMAES. To testify the performance of BSO-CMAES, 57 benchmark functions were used. The nonparametric statistical tests with BSO and CMA-ES verify the effectiveness of the hybrid scheme. Extensive comparison experiments with five other algorithms verify the competitiveness of BSO-CMAES. In addition, in the discussion section we also found that BSO-CMAES is more able to ensure the diversity of the population compared to BSO.

Brain Storm Algorithm Combined with Covariance Matrix Adaptation …

125

Table 1 Summary of different hybrid algorithms Researchers

Hybrid meta-heuristic

Jiao and Wang [25]

GA + IA

Application

Lee et al. [32]

ACO + IA

Weapon target assignment

Juang [26]

GA + PSO

Recurrent network design

Shelokar et al. [51]

ACO + PSO

Continuous optimization

Lee et al. [33]

GA + ACO

Kao and Zahara [28]

GA + PSO

Multimodal functions

Kaveh and Talatahari [30]

ACO + PSO

Optimization of truss structures

Kampf and Robinson [27]

CMA-ES + DE

Application to solar energy potential

Niknam and Amiri [46]

FAPSO + ACO + K-means

Cluster analysis

Marinakis and Marinaki [43]

GRASP + ENS

Probabilistic traveling salesman Problem

Guvenc et al. [21]

GA + PSO

Economic dispatch problem

Shuang et al. [53]

ACO + PSO

Boussaid et al. [1]

BBO + DE

Deng et al. [10]

PSO + BPNN

Highway passenger volume Prediction

Deng et al. [8]

GA + PSO

Function approximation problem

Deng et al. [9]

GA + PSO + ACO

Xin et al. [64]

DE + PSO

Lien and Cheng [35]

BA + PSO

Construction site layout optimization

Chang et al. [3]

DE + ACO

Antenna design

Nwankwor et al. [47]

DE + PSO

Optimal well placement

Nguyen et al. [45]

HP + CRO

Gao et al. [12]

GSA + Chaos

Sun [59]

BSO + GNN

Stock index forecasting

Cao et al. [2]

BSO + DE

Applications of ANNs

Li et al. [34]

PSO + ABC

High-dimensional optimization

Trivedi et al. [61]

GA + DE

Unit commitment problem

Kaveh and Ghazaan [29]

ALC + PSO ; HALC-PSO

Tran et al. [60]

MOABC + DE

Panda and Yegireddy [50]

DE + PS

Panda et al. [49]

DE + PS

Soleimani and Kannan [54]

GA + PSO

Zheng et al. [68]

DE + FA

Lin et al. [37]

EA + IA

Multiobjective optimization problem

Xu et al. [65]

IA + EDA

Traveling salesman problem

Garg [20]

GA + PSO

Constrained optimization problem

Gao et al. [14]

ACO + K-means

Dynamic location routing problem

Pan et al. [48]

GA + IA + DO

Traveling salesman problem

Lazzus et al. [31]

ACO + PSO

Ji et al. [23]

GSA + CLS

Lieu et al. [36]

DE + FA

Optimal power allocation

Unconstrained numerical optimization

Time cost quality tradeoff problem

Network design

126

Y. Yu et al.

The outline of the remaining chapters is as follows: In Sect. 2, we give detailed running step of the original BSO, and introduce its parameters. In Sect. 3, the running process of CMA-ES is introduced, and the difference between it and the original ES is analyzed. Section 4 analyzes the characteristics of BSO and CMA-ES, as well as hybrid scheme. Section 5 is the experiments and analysis. Section 6 is the discussion. Finally, we conclude this paper in Sect. 7.

2 Brain Storm Optimization Algorithm BSO, a new algorithm inspired by human brainstorming process, has attracted a lot of attention in the swarm intelligence research community since introduced by Shi in 2011 [7, 66]. In brainstorming, new solutions continue to emerge from the associative reaction, and BSO models this process with the goal of obtaining “good enough” solutions. Figure 1 shows the process of original BSO [52, 63]. Just as most population-based algorithms do, the first step is to initialize. Step 2 Divide n individuals into m clusters. The condition for solution clustering is whether the solutions are similar or not. Similar solutions fall into one cluster, and after many iterations, all the solutions will converge into a small space. For clustering, the original BSO uses K-means clustering algorithm. However, there are some other clustering algorithms that can replace K-means, such as K-medians clustering algorithm and random grouping strategy. Step 3 Calculate and rank the fitness of each individual in each cluster. The fitness of each solution is calculated and based on the fitness values, the solutions are ranked in their respective clusters. Step 4 Randomly generate an individual to replace a randomly selected cluster center at a pre-setting probability. The purpose of this step is to preserve the diversity of solutions. According to a predetermined probability value, to determine whether to operate. When the condition is satisfied, a cluster center is randomly selected, and then be replaced by a randomly generated new individual in order to achieve the purpose of creating new ideas different from the existing ones. Step 5 Generate new individuals by single or double cluster centers or individuals. As proposed by Shi, a swarm intelligence algorithm should have two capabilities: the ability to learn and the ability to develop. The task of ability learning falls on the convergent operator, and the ability development is responsible for the divergence operator, and these two operators are the main operators of BSO. The new solution is born, and its inspiration may come from one, two or more individuals. If these inspirational solutions come from the same cluster, it means that

Brain Storm Algorithm Combined with Covariance Matrix Adaptation … Fig. 1 BSO flow chart

127

Start Randomly generate n potential individuals Divide n individuals into m clusters Calculate the fitness and rank individuals in each cluster Randomly generate an individual to replace a randomly selected cluster center at a presetting probability value preplace Generate new individuals by single or double cluster center or individual

N

N new individuals have been generated Y Meet the pre-setting maximum number of iterations

N

Y End

the algorithm starts to converge in a certain area and the development capability is strengthened. If these inspirational solutions come from different clusters, the newly generated solution are bound to stay away from these clusters, meaning that the algorithm begins to diverge and the exploration is strengthened. Considering the simplicity of the algorithm, BSO does not simulate the situation where new ideas are inspired by more than two ideas. The new individual is generated by one or two individuals, depending on the pre-setting probability value pone−two . Corresponding to these two cases, the probability values pone and ptwo are respectively set to decide whether the new individual is generated by the cluster center or the ordinary individual. The generation of new individuals follows Eq. (1). d d xnew = xchosen + ξ × n (μ, σ )

(1)

128

Y. Yu et al.

d where xchosen represents the d th dimension of the chosen individual, n (μ, σ ) represents the Gaussian distribution, where its mean is μ, and the variance is σ . Gaussian d to generate the random value multiplied by a weight coefficient ξ assigned to xchosen d d th dimension of a new individual xnew . Equation (2) is the calculation method of ξ .

 ξ = logsig

1 2

× itermax − itercur R

 × rand ()

(2)

where itermax is used to represent the maximum number of iterations and itercur is the current number of iterations. In order to make the convergence speed of the algorithm become controllable, the paramete R is added to change the slope of the logsig () function.

3 Covariance Matrix Adaptation Evolution Strategy 3.1 Evolution Strategy Evolutionary strategy is a method of solving parameter optimization problems by simulating the biological evolution [11], which is a black box optimization algorithm. The iteration of ES follows the process of generating new individuals from a normal distribution and then preserves good individuals. Good individuals will guide the next iteration, making the new individual’s fitness value closer to the goal of optimization. The initial evolutionary strategy optimization process can be shown in Fig. 2. In Fig. 2, the curve shows the contour of the function, and the dots represent the new individuals generated by sampling in each iteration. The cyan dots are the best fitness points in this iteration, and the red dots represent the average value of this iteration. It can be seen that, due to its greediness, the original ES is effective when facing simple problems. However, we are still worried about whether the original ES can fall into the local optimum when the problem is complicated. On one hand, we hope to explore more areas to ensure the diversity of samples. On the other hand, we feel that the best value is already near, and we need to turn more exploration energy to exploitation. Covariance-matrix adaptation evolution strategy (CMA-ES) provides us with this possibility.

3.2 Covariance Matrix Adaptation Evolution Strategy CMA-ES is a new type of global optimization algorithm based on ES algorithm proposed by Hansen [22]. It can make use of the results of each generation, and adaptively increase or decrease the search scope of the next The core   generation. theory of CMA-ES is to use the covariance matrix C in N mt , σt2 Ct to handle the

Brain Storm Algorithm Combined with Covariance Matrix Adaptation …

Step 1

Step 2

Step 3

Step 4

Step 5

Step 6

129

Fig. 2 ES optimization process

relationship and scaling of variables. mt represents the mean, which determines the center of the distribution, mt determines the search area. σt determines the scale of the search range and the intensity during the convergence. Ct is the covariance matrix, which determines the shape of the distribution. The main loop of CMA-ES consists of four key steps: Step 1. Sample in the normal distribution to generate a population. Step 2. Calculate and rank the fitness of individuals. Step 3. Pick out outstanding individuals. Step 4. Update distribution parameters mt , Ct and σt . Step 1 Sample in the Gaussian distribution to generate a population.  In CMA-ES, new individuals are generated by sampling in N mt , σt2 Ct . The new individual can be expressed as xi = mt + σt yi , yi ∼ N (0, Ct )

130

Y. Yu et al.

where each yi ∈ Rn represents a search direction. Step 2 Calculate and rank the fitness of individuals. Each newly generated individual is evaluated by the evaluation function for its fitness value and sorted by the fitness value. Step 3 Pick out outstanding individuals. From the rankings obtained in Step 2, the top μ individuals are selected. Empirically, μ is half the value of λ but there are other alternatives. Step 4 Update distribution parameters mt , Ct and σt . Using the individuals selected in Step 3, the distribution parameters mt , Ct and σt are updated separately. The distribution of selected front row individuals provides a general direction for the next iteration to seek more superior individuals. The main parameter updates of CMA-ES are listed as follows: (1) Mean update The mean of the distribution is the weighted maximum likelihood estimation of the selected μ individuals. mt+1 =

μ 

ωi xi:λ = mt +

i=1

μ 

ωi (xi:λ − mt )

(3)

i=1

As can be seen from the above equation, the mean is moved along the search direction. If the weight coefficient ωi = μ1 takes the same value, the function used is the maximum likelihood estimate of the selected individuals. In addition, in order to emphasize the influence of the most front row individuals, the actual algorithmic process usually takes ω1 ≥ ω2 ≥ · · · ωμ > 0. (2) Evolutionary path update In CMA-ES, the update of covariance matrix C contains two items of rank − 1 and rank − μ, of which item rank − 1 uses historical search information—evolutionary path, which is constructed by the following equation. pt+1 = (1 − c) pt +

 √ mt+1 − mt c (2 − c) μω σt

(4)

The above equation describes the moving state of the distribution mean. t The moving direction mt+1σ−m in each iteration is weighted average, which t makes the opposite direction components cancel each other and the same components are superposed. In addition, factor μω = μ1 ω2 , and c is the i=1 i learning rate which is inversely proportional to the degree of freedom of the adjusted variables. (3) Covariance matrix update The updating principle of covariance matrix C is to increase the variance of the successful search direction, which increases the probability of sampling in these directions. The covariance matrix update equation is shown below.

Brain Storm Algorithm Combined with Covariance Matrix Adaptation …

131

  T Ct+1 = 1 − c1 − cμ Ct + c1 pt+1 pt+1



μ  xi:λ − mt xi:λ − mt T +cμ ωi σt σt i+1

(5)

Rank − 1 update can be thought of as an evolutionary path taken directly as a successful search direction. The original CMA includes only rank − 1 update, which means that Cμ = 0. CMA-ES in this form, has a good performance when dealing with small-scale populations. Rank − μ update uses the weighted maximum likelihood estimation of the selected μ individuals, and can also be regarded as a natural gradient to the C. The principle of learning rate c1 and cμ is the same as the c in the evolutionary path update, which is inversely proportional to the degree of freedom of the variable. (4) Step size update By default, CMA-ES uses a cumulative step size adjustment (CSA). CSA is currently the most successful, most used step adjustment method. The principle of CSA can be summarized in two aspects: (1) If the successive search directions are positively correlated, it indicates that the step size is too small and should be increased. (2) If the successive search directions are negatively correlated, it indicates that the step size is too large and should be reduced. In a word, the direction of successive search should be conjugated. For easy and intuitive understanding, Fig. 3 gives a simplified flowchart of CMAES.

Start Calculate or directly give the initial Gaussian distribution of the sample space randomly select λ individuals from the sample space as a paternal population

Update step and CMA and form a new Gaussian distribution sample space

Calculate and rank the fitness of each individual in the population

Calculate a new expectation and variance with the distribution of offspring population

Reach the algorithm termination condition Y End

Fig. 3 CMA-ES flow chart

N

Select the top μ individuals as a offspring individual

132

Y. Yu et al.

Step 1

Step 2

Step 3

Step 4

Step 5

Step 6

Fig. 4 CMA-ES optimization process

3.3 Improvement of CMA-ES Compared to ES As we mentioned earlier, the pattern CMA-ES using seems to be what we are asking for. Overall, compared to the original ES, CMA-ES achieves two major breakthroughs: (1) The covariance matrix is used to guide the generation of new individuals and improve the efficiency of exploration. (2) The concept of evolutionary path is introduced, and the historical information of population optimization is used to avoid the degradation of population and enhance the accuracy of search. In short, CMAES not only focuses on the diversity of the population, but also improves the search efficiency of the algorithm. Figure 4 shows the optimization process of CMA-ES. The recorded points of each of these iterations form a convergent trajectory.

Brain Storm Algorithm Combined with Covariance Matrix Adaptation …

133

4 Hybrid Scheme of BSO and CMA-ES 4.1 Characteristics of BSO Looking back at the BSO process, there are a few key words that deserve our attention: clustering, ordinary individuals, and clustering centers. From these keywords, we can summarize the advantages of BSO: (1) The use of clustering, the search space is divided into several regions, making the distribution of individuals in space more organized. (2) The origin of new individuals is diverse, including single individual, single cluster center, double common individuals and double cluster centers. The process of generating new individuals by individuals in the same cluster symbolizes the ability of the algorithm to develop. The process of generating new individuals by individuals in the two clusters reflects the ability of the algorithm to explore. It can be said that taking into account the development and exploration is a clear feature of BSO. Although the BSO has the obvious advantages mentioned above, it seems that there is still some space for us to work hard to improve on. The following two points may be worth attention: (1) The algorithm contains a lot of pre-setting probability values, the change of the values may change the performance of the algorithm. (2) The introduction of different clustering algorithms makes the computation of the algorithm increase and decrease, and the calculation efficiency of BSO is worth considering. By the way, the K-means clustering algorithm used by the original BSO, due to its own nature, is inevitably in the face of high-dimensional problems.

4.2 Characteristics of CMA-ES The speed of function convergence has always been the focus of the ES field. Some ES studies are devoted to initializing a given algorithm and quickly converging to local optima. CMA-ES makes two improvements to the original ES, including covariance matrix adaptation and evolutionary path to improve the convergence speed. However, the balance between search performance and time cost of the algorithm deserves more attention. With the rapid convergence of CMA-ES, we hope it will take into account the diversity of some populations to avoid a premature convergence and fall into the local optimum.

4.3 Hybrid Scheme: BSO-CMAES Figure 5 gives a brief flowchart of our hybrid scheme. We expect that the new hybrid algorithm will retain the powerful global search capabilities of BSO, but we still hope that the search speed can be improved. Therefore, we apply CMA-ES which has a

134 Fig. 5 BSO-CMAES flowchart

Y. Yu et al. Start

Initialisation

Perform BSO to get a optimal individual

Perform CMA-ES to get a optimal individual

Compare and record the better individual

Reach the algorithm termination condition

N

Y End

high-speed search capability to run a parallel solution while the BSO is running. After each iteration, BSO and CMA-ES provide their best individuals for comparison. The fitness value of the better individual is recorded.

5 Experiments and Analysis In order to demonstrate that our hybrid is actually effective, we compared BSOCMAES with seven algorithms, which are BSO, CMA-ES, differential evolution algorithm (DE) [15, 58], gravitational search algorithm (GSA) [23, 56], grey wolf optimizer (GWO) [44], artificial bee colony algorithm (ABC) and whale optimization algorithm (WOA) [18]. In detail, the seven algorithm used for comparison are divided into two types. The one is comparing BSO-CMAES with BSO and CMA-ES. The interesting point of this comparison method is that the comparison between the hybrid algorithm and the two source algorithms can be used to illustrate that the hybrid scheme is effective and improves the computational performance of the algorithm. The second is comparing BSO-CMAES with algorithms proposed in recent years, such as DE. This comparison method is used to illustrate the competitiveness of BSO-CMAES. All 28 test functions of CEC’13 and 29 test functions in CEC’17 are used in this experiment. CEC’13 contains 5 unimodal functions, 15 basic multimodal functions and 8 composition functions. In CEC’17, 2 unimodal functions, 7 simple multimodal

Brain Storm Algorithm Combined with Covariance Matrix Adaptation … Table 2 Experimental parameters Population number Dimension (D) (N) 100

30

135

Running times

Maximum iterations

30

3000

functions, 10 hybrid functions and 10 composition functions are adopted. To avoid the possibility that the algorithm get into trouble in a particular function that it can not reflect the real performance. In the experiment, as many as 57 functions are used to test the performance of the algorithm. Moreover, we not only focus on the number of test functions, but also hope that test functions can try to detect the limit of algorithm ability as much as possible. Therefore, our test function also includes multimodal functions, hybrid functions and so on. All experiments are performed using a matlab software running with 2.40 GHz Intel(R) Core(TM) i7-5500U CPU and 8 GB RAM. We use the Wilcoxon nonparametric statistical test to clarify whether there are significant differences between two algorithms. Table 2 shows our experimental parameter settings.

5.1 Analysis for Experiment Results of BSO-CMAES, BSO and CMA-ES This part of the experiment is divided into two segments. The first segment, specific experimental data and several relative convergence graphs will be offered for detailed description and analysis. In the second segment, we will perform a statistical test to confirm that BSO-CMAES is really better than BSO and CMA-ES. Table 3 shows the test results of BSO-CMAES, BSO and CMA-ES on CEC’13 and CEC’17. The best value for each test function has been bolded. From Table 3, it can be found on 57 test functions, BSO-CMAES gets the best values on 48 test problems. In addition, on a test problem (F1) it gets the same first place with CMAES. On the remaining 8 test problems, BSO-CMAES still achieves better results than BSO or CMA-ES on 7 of the problems. Only the result obtained on F36 is not ideal. Based on the above data, there are two points which should be affirmed: the optimization effect of BSO-CMAES is indeed better than that of BSO and CMA-ES, and BSO-CMAES has a satisfactory stability. In addition to the intuitive experimental data, we also perform a nonparametric statistical test on the experimental results. Compared with parametric statistics, nonparametric statistics can use the sample data to infer the overall distribution pattern without knowing the overall distribution type, and the inference process does not involve the parameters related to the overall distribution, so it tends to have a better robustness. In this experiment, the non-parametric statistical test we use is the Wilcoxon signed rank test. The Wilcoxon signed rank test is a statistical method used

136

Y. Yu et al.

Table 3 Experimental results of BSO-CMAES, BSO and CMA-ES on CEC’13 and CEC’17 Algorithms F1 F2 F3 F4 BSO-CMAES BSO CMA-ES

BSO-CMAES BSO CMA-ES

BSO-CMAES BSO CMA-ES

BSO-CMAES BSO CMA-ES

BSO-CMAES BSO CMA-ES

BSO-CMAES BSO CMA-ES

−1.40E+03 ± 0.00E+00 −1.40E+03 ± 4.22E−14 −1.40E+03 ± 0.00E+00 F5 −1.00E+03 ± 9.95E−05 −1.00E+03 ± 1.52E−03 −1.00E+03 ± 1.06E−04 F9 −5.89E+02 ± 1.04E+01 −5.68E+02 ± 2.90E+00 −5.59E+02 ± 1.57E+00 F13 −1.81E+02 ± 1.17E+01 3.55E+02 ± 8.77E+01 −4.25E+01 ± 1.13E+01 F17 3.37E+02 ± 1.65E+00 7.31E+02 ± 8.13E+01 4.85E+02 ± 8.70E+00 F21 9.68E+02 ± 5.88E+01 1.03E+03 ± 8.91E+01 9.70E+02 ± 4.66E+01

2.26E+04 ± 1.93E+04 1.54E+06 ± 4.79E+05 2.41E+07 ± 1.02E+07 F6 −9.00E+02 ± 9.89E−01 −8.66E+02 ± 2.47E+01 −8.96E+02 ± 2.90E+00 F10 −5.00E+02 ± 5.09E−03 −5.00E+02 ± 1.93E−01 −4.98E+02 ± 4.83E−01 F14 1.67E+03 ± 6.46E+02 3.88E+03 ± 5.17E+02 7.24E+03 ± 3.25E+02 F18 4.37E+02 ± 1.86E+00 7.41E+02 ± 5.30E+01 5.86E+02 ± 9.57E+00 F22 1.35E+03 ± 2.97E+02 6.04E+03 ± 7.04E+02 8.26E+03 ± 2.93E+02

2.17E+03 ± 7.67E+03 1.11E+08 ± 1.74E+08 −1.20E+03 ± 0.00E+00 F7 −5.77E+02 ± 2.61E+02 −6.71E+02 ± 7.57E+01 −8.00E+02 ± 0.00E+00 F11 −3.90E+02 ± 3.35E+00 6.40E+01 ± 7.12E+01 −2.99E+02 ± 6.81E+01 F15 1.89E+03 ± 8.65E+02 4.25E+03 ± 5.57E+02 7.52E+03 ± 2.71E+02 F19 5.03E+02 ± 5.01E−01 5.09E+02 ± 2.04E+00 5.12E+02 ± 2.38E+00 F23 3.06E+03 ± 1.70E+03 5.98E+03 ± 7.32E+02 8.52E+03 ± 2.82E+02

−8.63E+02 ± 1.38E+02 2.17E+04 ± 5.65E+03 1.94E+05 ± 3.89E+04 F8 −6.79E+02 ± 6.15E−02 −6.79E+02 ± 7.60E−02 −6.79E+02 ± 5.73E−02 F12 −2.91E+02 ± 2.00E+00 2.06E+02 ± 8.43E+01 −1.40E+02 ± 9.72E+00 F16 2.00E+02 ± 3.24E−03 2.00E+02 ± 4.14E−02 2.00E+02 ± 0.00E+00 F20 6.15E+02 ± 9.04E−02 6.14E+02 ± 1.77E−01 6.15E+02 ± 2.20E−01 F24 1.25E+03 ± 3.59E+01 1.33E+03 ± 2.25E+01 1.30E+03 ± 2.92E+00 (continued)

Brain Storm Algorithm Combined with Covariance Matrix Adaptation … Table 3 (continued) F25 BSO-CMAES 1.39E+03 ± 1.63E+01 BSO 1.46E+03 ± 2.50E+01 CMA-ES 1.40E+03 ± 2.83E+00 F29 BSO-CMAES 1.00E+02 ± 0.00E+00 BSO 2.47E+03 ± 1.95E+03 CMA-ES 1.74E+04 ± 2.32E+04 F33 BSO-CMAES 6.00E+02 ± 1.21E−07 BSO 6.52E+02 ± 7.01E+00 CMA-ES 6.00E+02 ± 1.03E−07 F37 BSO-CMAES 3.06E+03 ± 8.35E+02 BSO 5.30E+03 ± 5.16E+02 CMA-ES 8.34E+03 ± 3.49E+02 F41 BSO-CMAES 1.81E+03 ± 2.38E+02 BSO 6.40E+03 ± 4.66E+03 CMA-ES 1.88E+05 ± 1.02E+05 F45 BSO-CMAES 1.29E+04 ± 7.32E+03 BSO 1.21E+05 ± 1.03E+05 CMA-ES 3.29E+06 ± 2.15E+06

F26 1.46E+03 ± 6.14E+01 1.50E+03 ± 8.60E+01 1.58E+03 ± 4.82E+01 F30 3.00E+02 ± 1.84E−02 5.34E+02 ± 2.66E+02 3.10E+05 ± 7.67E+04 F34 7.39E+02 ± 2.08E+00 1.15E+03 ± 9.65E+01 8.83E+02 ± 7.56E+00 F38 1.22E+03 ± 4.44E+01 1.23E+03 ± 4.05E+01 3.26E+03 ± 9.70E+02 F42 4.76E+03 ± 2.56E+03 2.95E+04 ± 1.61E+04 2.53E+06 ± 1.71E+06 F46 2.76E+03 ± 6.93E+02 1.52E+05 ± 6.43E+04 2.21E+06 ± 1.45E+06

F27 2.16E+03 ± 2.85E+02 2.49E+03 ± 9.32E+01 2.60E+03 ± 1.96E+01 F31 4.01E+02 ± 6.71E−01 4.67E+02 ± 2.19E+01 4.11E+02 ± 1.98E+00 F35 8.10E+02 ± 2.47E+00 9.47E+02 ± 2.82E+01 9.56E+02 ± 8.45E+00 F39 4.11E+03 ± 1.23E+03 1.77E+06 ± 1.25E+06 1.31E+07 ± 6.07E+06 F43 2.03E+03 ± 1.78E+02 3.20E+03 ± 4.24E+02 2.95E+03 ± 1.71E+02 F47 2.28E+03 ± 2.41E+02 2.67E+03 ± 1.74E+02 2.55E+03 ± 1.55E+02

137

F28 1.70E+03 ± 7.12E−13 5.73E+03 ± 5.38E+02 1.70E+03 ± 0.00E+00 F32 5.09E+02 ± 2.41E+00 6.87E+02 ± 4.05E+01 6.48E+02 ± 2.89E+01 F36 4.85E+03 ± 1.08E+03 3.98E+03 ± 6.95E+02 9.00E+02 ± 0.00E+00 F40 1.52E+04 ± 7.10E+03 5.36E+04 ± 2.85E+04 5.09E+06 ± 2.67E+06 F44 2.13E+03 ± 1.74E+02 2.48E+03 ± 2.56E+02 2.28E+03 ± 1.59E+02 F48 2.31E+03 ± 3.79E+00 2.50E+03 ± 4.48E+01 2.44E+03 ± 2.65E+01 (continued)

138 Table 3 (continued) F49 BSO-CMAES 2.78E+03 ± 3.50E+02 BSO 6.03E+03 ± 1.77E+03 CMA-ES 9.41E+03 ± 3.94E+02 F53 BSO-CMAES 3.74E+03 ± 1.97E+02 BSO 8.16E+03 ± 1.57E+03 CMA-ES 4.82E+03 ± 1.26E+02 F57 BSO-CMAES 2.25E+04 ± 2.26E+04 BSO 5.74E+05 ± 3.50E+05 CMA-ES 2.20E+06 ± 1.16E+06

Y. Yu et al.

F50 2.66E+03 ± 9.62E+00 3.29E+03 ± 1.27E+02 2.79E+03 ± 4.13E+01 F54 3.20E+03 ± 2.42E−04 3.82E+03 ± 2.91E+02 3.20E+03 ± 4.94E−05

F51 2.83E+03 ± 3.58E+00 3.50E+03 ± 1.13E+02 2.91E+03 ± 7.22E+01 F55 3.30E+03 ± 3.22E−04 3.21E+03 ± 2.57E+01 3.30E+03 ± 6.39E−05

F52 2.88E+03 ± 2.21E−01 2.89E+03 ± 1.46E+01 2.88E+03 ± 1.14E−01 F56 3.63E+03 ± 1.80E+02 4.38E+03 ± 2.79E+02 4.34E+03 ± 1.88E+02

to detect whether there are significant differences between two samples. The following is a simple example to test the significant difference between BSO-CMAES and BSO. 1. Establish the null hypothesis H0 : BSO-CMAES is not significantly better than BSO. 2. Calculate the difference Qi between the optimization results of BSO-CMAES and the BSO under 57 test functions. 3. Rank Qi with absolute value. 4. (1) If Qi > 0, put its rank into R+ ; (2) If Qi = 0, put its rank half into R+ , half into R− ; (3) If Qi < 0, put its rank into R− . 5. R = Min(R+ , R− ) 6. Calculate the value of p based on the value of R and determine whether the null hypothesis H0 can be overridden based on whether the value of p is less than 0.05 or 0.01. Table 4 shows the p-values obtained by BSO-CMAES versus BSO and CMA-ES, respectively. The sample data are derived from the averages of the three algorithms over the 57 test functions. In Table 4, the p-values among BSO-CMAES, BSO and CMAES are less than 0.01, which means that the BSO-CMAES is significantly better

Brain Storm Algorithm Combined with Covariance Matrix Adaptation …

139

Table 4 Wilcoxon signed rank test results of BSO-CMAES versus BSO and CMAES BSO-CMAES versus R+ R− p-value BSO CMA-ES

1515.0 1514.5

81.0 138.5

0 0

than BSO and CMAES. In other words, it is worthwhile to believe that our parallel hybrid scheme really improves the performance of BSO and CMA-ES. Another thing that can be proved is that the optimization effect of BSO-CMAES does not only come from either BSO or CMA-ES. In short, the hybrid algorithm contribute to a better optimization.

5.2 Comparison Between BSO-CMAES and Other Algorithms To demonstrate the competitiveness of BSO-CMAES, we compare it with five wellknown algorithms proposed in recent years. DE is a heuristic algorithm that is suitable for solving the problem of minimizing nonlinear and non-differentiable continuous space functions [57]. It has few numbers of control variables and has good robustness. GSA is an algorithm based on Newtonian gravity and the laws of motion which can achieve outstanding results in solving nonlinear function problems [23]. GWO inspired by the gray wolf wolves leadership mechanism and hunting mode, is ideal for solving challenging issues with unknown search space [44]. ABC is a swarm intelligence algorithm inspired by bee’s intelligent foraging behavior [18, 34]. It has few control parameters and is suitable for optimizing numerical test problems with large amounts of data. WOA is a natural heuristic optimization algorithm inspired by humpback whale hunting behavior that once stood out in the heuristic algorithm. The setting of experimental parameters still uses the data shown in Table 2. Tables 5 and 6 show the results of BSO-CMAES and the other five algorithms. Among these 57 test functions, BSO-CMAES obtains the best values on 37 functions, and one of the test functions (F1) is tied to the first with GSA. Among the remaining 20 test functions, BSO-CMAES is better than at least one algorithm; that is, it has never obtained the worst value. DE, GSA and GWO obtain the best value on some functions, respectively. However, ABC and WOA show inferior performances. According to the above data description, since BSO-CMAES has achieved the best values on a number of functions; we conclude from this that BSO-CMAES is highly competitive and has good stability. From the above intuitive experimental results, it can be seen that the BSO-CMAES has a satisfactory performance. We implement a Wilcoxon signed rank test between BSO and other algorithms in order to analyze their significant difference. Table 7 shows the Wilcoxon signed rank test results for BSO-CMAES versus other algo-

140

Y. Yu et al.

Table 5 Experimental results of BSO-CMAES and other algorithms in 57 benchmark functions (F1-F36) from CEC’13 and CEC’17 Algorithms F1 F2 F3 F4 BSO-CMAES DE GSA GWO ABC WOA

BSO-CMAES DE GSA GWO ABC WOA

BSO-CMAES DE GSA GWO ABC WOA

−1.40E+03 ± 0.00E+00 −1.40E+03 ± 4.97E−03 −1.40E+03 ± 0.00E+00 −7.16E+02 ± 6.54E+02 −1.40E+03 ± 1.03E−13 −1.40E+03 ± 1.60E−01 F5 −1.00E+03 ± 9.95E−05 −1.00E+03 ± 1.71E−02 −1.00E+03 ± 6.13E−13 −4.45E+02 ± 2.31E+02 −4.83E+01 ± 1.43E+02 −9.20E+02 ± 1.77E+01 F9 −5.89E+02 ± 1.04E+01 −5.61E+02 ± 9.81E−01 −5.67E+02 ± 2.46E+00 −5.82E+02 ± 2.23E+00 −5.61E+02 ± 1.44E+00 −5.64E+02 ± 2.78E+00

2.26E+04 ± 1.93E+04 1.87E+07 ± 6.55E+06 7.20E+06 ± 1.20E+06 1.58E+07 ± 9.24E+06 2.19E+08 ± 3.21E+07 3.47E+07 ± 1.64E+07 F6 −9.00E+02 ± 9.89E−01 −8.83E+02 ± 4.44E+00 −8.29E+02 ± 1.77E+01 −7.74E+02 ± 4.24E+01 −8.57E+02 ± 1.25E+01 −7.99E+02 ± 3.72E+01 F10 −5.00E+02 ± 5.09E−03 −4.95E+02 ± 1.57E+00 −5.00E+02 ± 6.24E−02 −3.06E+02 ± 1.42E+02 −2.37E+02 ± 3.51E+01 −4.45E+02 ± 2.21E+01

2.17E+03 ± 7.67E+03 3.44E+08 ± 1.66E+08 5.62E+09 ± 2.48E+09 2.80E+09 ± 2.86E+09 6.66E+10 ± 1.59E+10 1.35E+10 ± 7.98E+09 F7 −5.77E+02 ± 2.61E+02 −7.55E+02 ± 8.52E+00 −7.31E+02 ± 1.48E+01 −7.54E+02 ± 1.82E+01 −5.76E+02 ± 2.82E+01 -9.41E+01 ± 2.06E+03 F11 −3.90E+02 ± 3.35E+00 −2.06E+02 ± 1.85E+01 −1.16E+02 ± 1.77E+01 −3.14E+02 ± 3.19E+01 −1.84E+02 ± 1.78E+01 7.52E+01 ± 9.86E+01

-8.63E+02 ± 1.38E+02 3.69E+04 ± 7.77E+03 6.77E+04 ± 2.64E+03 2.69E+04 ± 7.43E+03 7.03E+04 ± 9.91E+03 5.69E+04 ± 2.12E+04 F8 −6.79E+02 ± 6.15E−02 −6.79E+02 ± 4.95E−02 −6.79E+02 ± 6.91E−02 −6.79E+02 ± 4.24E−02 −6.79E+02 ± 4.36E−02 −6.79E+02 ± 4.45E−02 F12 −2.91E+02 ± 2.00E+00 −7.87E+01 ± 1.24E+01 3.77E+01 ± 2.84E+01 −1.82E+02 ± 4.85E+01 −6.84E+01 ± 1.41E+01 1.46E+02 ± 1.03E+02 (continued)

Brain Storm Algorithm Combined with Covariance Matrix Adaptation … Table 5 (continued) F13 BSO-CMAES −1.81E+02 ± 1.17E+01 DE 2.56E+01 ± 8.98E+00 GSA 2.68E+02 ± 3.30E+01 GWO −2.68E+01 ± 3.66E+01 ABC 2.24E+01 ± 9.10E+00 WOA 2.99E+02 ± 8.96E+01 F17 BSO-CMAES 3.37E+02 ± 1.65E+00 DE 5.40E+02 ± 1.40E+01 GSA 3.66E+02 ± 8.76E+00 GWO 4.67E+02 ± 4.51E+01 ABC 5.38E+02 ± 1.08E+01 WOA 8.89E+02 ± 1.11E+02 F21 BSO-CMAES 9.68E+02 ± 5.88E+01 DE 9.98E+02 ± 7.17E+01 GSA 1.03E+03 ± 6.18E+01 GWO 1.53E+03 ± 2.64E+02 ABC 1.00E+03 ± 1.33E−02 WOA 1.03E+03 ± 6.87E+01

F14 1.67E+03 ± 6.46E+02 6.38E+03 ± 6.26E+02 3.97E+03 ± 5.28E+02 2.78E+03 ± 9.75E+02 7.15E+03 ± 2.18E+02 4.88E+03 ± 7.84E+02 F18 4.37E+02 ± 1.86E+00 6.60E+02 ± 1.50E+01 4.55E+02 ± 6.80E+00 6.41E+02 ± 2.87E+01 6.44E+02 ± 8.95E+00 1.01E+03 ± 1.20E+02 F22 1.35E+03 ± 2.97E+02 7.50E+03 ± 4.38E+02 7.62E+03 ± 3.91E+02 3.57E+03 ± 1.01E+03 8.74E+03 ± 2.01E+02 6.76E+03 ± 1.08E+03

F15 1.89E+03 ± 8.65E+02 7.48E+03 ± 1.97E+02 3.75E+03 ± 4.17E+02 3.41E+03 ± 1.09E+03 7.46E+03 ± 2.37E+02 5.49E+03 ± 1.02E+03 F19 5.03E+02 ± 5.01E−01 5.19E+02 ± 1.29E+00 5.10E+02 ± 2.41E+00 5.69E+02 ± 2.05E+02 1.74E+03 ± 5.62E+02 5.58E+02 ± 1.90E+01 F23 3.06E+03 ± 1.70E+03 8.38E+03 ± 2.41E+02 6.85E+03 ± 3.56E+02 4.82E+03 ± 1.51E+03 8.82E+03 ± 3.04E+02 7.61E+03 ± 8.55E+02

141

F16 2.00E+02 ± 3.24E−03 2.02E+02 ± 2.88E−01 2.00E+02 ± 3.60E−03 2.02E+02 ± 3.05E−01 2.02E+02 ± 2.93E−01 2.02E+02 ± 4.34E−01 F20 6.15E+02 ± 9.04E−02 6.13E+02 ± 1.99E−01 6.15E+02 ± 2.03E−01 6.12E+02 ± 1.63E+00 6.15E+02 ± 1.37E−01 6.15E+02 ± 3.05E−01 F24 1.25E+03 ± 3.59E+01 1.26E+03 ± 1.06E+01 1.34E+03 ± 6.91E+01 1.25E+03 ± 9.95E+00 1.29E+03 ± 5.63E+00 1.31E+03 ± 1.00E+01 (continued)

142 Table 5 (continued) F25 BSO-CMAES 1.39E+03 ± 1.63E+01 DE 1.38E+03 ± 1.95E+01 GSA 1.49E+03 ± 9.76E+00 GWO 1.37E+03 ± 9.54E+00 ABC 1.44E+03 ± 5.31E+00 WOA 1.42E+03 ± 9.66E+00 F29 BSO-CMAES 1.00E+02 ± 0.00E+00 DE 2.87E+05 ± 1.29E+05 GSA 2.15E+03 ± 9.62E+02 GWO 1.02E+09 ± 8.91E+08 ABC 4.72E+04 ± 7.74E+04 WOA 2.48E+06 ± 1.73E+06 F33 BSO-CMAES 6.00E+02 ± 1.21E−07 DE 6.02E+02 ± 3.38E−01 GSA 6.50E+02 ± 3.21E+00 GWO 6.04E+02 ± 2.33E+00 ABC 6.00E+02 ± 6.69E−03 WOA 6.67E+02 ± 1.12E+01

Y. Yu et al.

F26 1.46E+03 ± 6.14E+01 1.40E+03 ± 3.90E−01 1.56E+03 ± 1.18E+01 1.49E+03 ± 7.02E+01 1.42E+03 ± 6.02E+00 1.53E+03 ± 9.64E+01 F30 3.00E+02 ± 1.84E−02 4.21E+04 ± 8.90E+03 8.44E+04 ± 6.14E+03 2.85E+04 ± 9.70E+03 1.03E+05 ± 1.09E+04 1.60E+05 ± 8.12E+04 F34 7.39E+02 ± 2.08E+00 9.60E+02 ± 1.33E+01 7.84E+02 ± 1.05E+01 8.35E+02 ± 4.95E+01 9.43E+02 ± 9.71E+00 1.21E+03 ± 9.33E+01

F27 2.16E+03 ± 2.85E+02 2.55E+03 ± 6.18E+01 2.23E+03 ± 1.10E+02 2.10E+03 ± 5.93E+01 2.67E+03 ± 4.60E+01 2.61E+03 ± 6.95E+01 F31 4.01E+02 ± 6.71E−01 4.85E+02 ± 6.99E−01 5.43E+02 ± 2.03E+01 5.70E+02 ± 4.98E+01 5.19E+02 ± 2.79E+00 5.45E+02 ± 3.70E+01 F35 8.10E+02 ± 2.47E+00 1.02E+03 ± 9.39E+00 9.55E+02 ± 1.22E+01 8.81E+02 ± 1.25E+01 1.02E+03 ± 1.16E+01 9.99E+02 ± 3.20E+01

F28 1.70E+03 ± 7.12E−13 1.71E+03 ± 1.26E+00 4.97E+03 ± 2.23E+02 2.43E+03 ± 2.78E+02 1.70E+03 ± 1.47E+00 5.36E+03 ± 7.53E+02 F32 5.09E+02 ± 2.41E+00 7.17E+02 ± 1.06E+01 7.27E+02 ± 1.72E+01 5.92E+02 ± 2.63E+01 7.18E+02 ± 9.44E+00 7.64E+02 ± 6.16E+01 F36 4.85E+03 ± 1.08E+03 9.14E+02 ± 7.27E+00 2.79E+03 ± 2.30E+02 1.18E+03 ± 1.39E+02 1.90E+03 ± 4.45E+02 6.54E+03 ± 2.35E+03

Brain Storm Algorithm Combined with Covariance Matrix Adaptation …

143

Table 6 Experimental results of BSO-CMAES and other algorithms in 57 benchmark functions (F37-F57) from CEC’13 and CEC’17 Algorithms F37 F38 F39 F40 BSO-CMAES DE GSA GWO ABC WOA

BSO-CMAES DE GSA GWO ABC WOA

BSO-CMAES DE GSA GWO ABC WOA

3.06E+03 ± 8.35E+02 8.16E+03 ± 2.33E+02 4.86E+03 ± 4.39E+02 3.73E+03 ± 5.49E+02 8.10E+03 ± 3.19E+02 6.06E+03 ± 9.74E+02 F41 1.81E+03 ± 2.38E+02 1.48E+03 ± 6.19E+00 4.27E+05 ± 1.15E+05 8.10E+04 ± 1.76E+05 3.04E+05 ± 1.25E+05 7.27E+05 ± 6.79E+05 F45 1.29E+04 ± 7.32E+03 1.87E+03 ± 6.36E+00 3.22E+05 ± 1.13E+05 7.75E+05 ± 1.40E+06 6.34E+06 ± 3.20E+06 3.02E+06 ± 2.58E+06

1.22E+03 ± 4.44E+01 1.20E+03 ± 2.01E+01 1.46E+03 ± 1.04E+02 1.51E+03 ± 4.42E+02 4.37E+03 ± 7.31E+02 1.45E+03 ± 1.15E+02 F42 4.76E+03 ± 2.56E+03 1.57E+03 ± 7.02E+00 1.09E+04 ± 2.04E+03 2.44E+05 ± 5.82E+05 2.08E+07 ± 8.65E+06 6.97E+04 ± 4.48E+04 F46 2.76E+03 ± 6.93E+02 1.94E+03 ± 2.88E+00 1.34E+04 ± 4.34E+03 2.06E+05 ± 3.88E+05 2.39E+07 ± 1.03E+07 2.45E+06 ± 2.03E+06

4.11E+03 ± 1.23E+03 4.43E+05 ± 1.92E+05 1.57E+07 ± 2.63E+07 3.31E+07 ± 3.81E+07 1.17E+08 ± 2.66E+07 4.47E+07 ± 3.11E+07 F43 2.03E+03 ± 1.78E+02 2.93E+03 ± 2.36E+02 3.15E+03 ± 2.72E+02 2.32E+03 ± 2.39E+02 3.76E+03 ± 1.87E+02 3.47E+03 ± 5.17E+02 F47 2.28E+03 ± 2.41E+02 2.11E+03 ± 1.02E+02 3.03E+03 ± 1.97E+02 2.33E+03 ± 1.66E+02 2.74E+03 ± 8.15E+01 2.78E+03 ± 1.76E+02

1.52E+04 ± 7.10E+03 1.46E+03 ± 1.58E+01 3.03E+04 ± 5.05E+03 6.63E+06 ± 2.33E+07 8.02E+07 ± 3.32E+07 1.35E+05 ± 1.44E+05 F44 2.13E+03 ± 1.74E+02 1.96E+03 ± 1.71E+02 2.91E+03 ± 2.07E+02 1.93E+03 ± 1.16E+02 2.49E+03 ± 1.19E+02 2.53E+03 ± 2.16E+02 F48 2.31E+03 ± 3.79E+00 2.51E+03 ± 7.59E+00 2.56E+03 ± 2.29E+01 2.37E+03 ± 1.85E+01 2.52E+03 ± 1.18E+01 2.56E+03 ± 6.24E+01 (continued)

144

Y. Yu et al.

Table 6 (continued) F49 BSO-CMAES 2.78E+03 ± 3.50E+02 DE 2.30E+03 ± 5.07E−01 GSA 6.40E+03 ± 1.54E+03 GWO 4.47E+03 ± 1.45E+03 ABC 2.64E+03 ± 2.08E+02 WOA 6.65E+03 ± 1.87E+03 F53 BSO-CMAES 3.74E+03 ± 1.97E+02 DE 5.70E+03 ± 1.16E+02 GSA 6.66E+03 ± 1.09E+03 GWO 4.43E+03 ± 2.45E+02 ABC 5.74E+03 ± 1.28E+02 WOA 7.24E+03 ± 1.00E+03 BSO-CMAES DE GSA GWO ABC WOA

F57 2.25E+04 ± 2.26E+04 8.48E+03 ± 9.18E+02 1.79E+05 ± 8.84E+04 3.90E+06 ± 3.10E+06 2.67E+07 ± 1.04E+07 1.04E+07 ± 6.42E+06

F50 2.66E+03 ± 9.62E+00 2.85E+03 ± 1.13E+01 3.59E+03 ± 1.03E+02 2.73E+03 ± 3.04E+01 2.89E+03 ± 1.60E+01 3.05E+03 ± 8.54E+01 F54 3.20E+03 ± 2.42E−04 3.19E+03 ± 7.33E+00 4.61E+03 ± 2.91E+02 3.23E+03 ± 1.78E+01 3.46E+03 ± 3.87E+01 3.36E+03 ± 8.88E+01

F51 2.83E+03 ± 3.58E+00 3.02E+03 ± 1.00E+01 3.30E+03 ± 6.80E+01 2.90E+03 ± 4.70E+01 3.04E+03 ± 1.17E+01 3.16E+03 ± 9.15E+01 F55 3.30E+03 ± 3.22E−04 3.20E+03 ± 1.50E+01 3.33E+03 ± 4.94E+01 3.33E+03 ± 4.83E+01 3.26E+03 ± 2.59E+01 3.31E+03 ± 3.96E+01

F52 2.88E+03 ± 2.21E−01 2.89E+03 ± 6.65E−02 2.93E+03 ± 1.09E+01 2.96E+03 ± 2.69E+01 2.89E+03 ± 1.73E−01 2.94E+03 ± 2.73E+01 F56 3.63E+03 ± 1.80E+02 4.06E+03 ± 1.53E+02 4.74E+03 ± 2.10E+02 3.71E+03 ± 1.26E+02 4.93E+03 ± 1.31E+02 5.00E+03 ± 4.58E+02

Brain Storm Algorithm Combined with Covariance Matrix Adaptation …

145

Table 7 Wilcoxon signed rank test results of BSO-CMAES versus other algorithms BSO-CMAES versus R+ R− p-value DE GSA GWO ABC WOA

1120.5 1579.5 1525.0 1515.0 1594.5

475.5 73.5 128.0 81.0 1.5

0.0068 0 0 0 0

rithms. BSO-CMAES achieves obvious advantages in the tests comparing GSA, GWO, ABC and WOA. The value of R+ is significantly better than that of R− . The DE algorithm is the best among the five algorithms used for comparison with BSOCMAES. According to Table 7, all p-values are less than 0.01, which demonstrates that BSO-CMAES significantly outperforms the other five algorithms.

5.3 Comprehensive Explanation The above experimental section has shown the outstanding performance of BSOCMAES, but we still pick out some diagrams for a detailed description. These descriptions include the convergence performance and stability of the algorithm and another nonparametric analysis.

5.3.1

Convergence Performance Analysis

To visually show the convergence performance of BSO-CMAES, we plot six convergence graphs shown in Fig. 6. In F2 and F4, the other algorithms converge early and can not explore significantly better values. However, BSO-CMAES has maintained a decreasing speed and continuously explored lower values, so that, even though the number of iterations has reached the termination condition, the convergence curve is still not flattening. In F13, the convergence curve of BSO-CMAES also shows a large distance from those of other algorithms, and which is closer to the minimum value of the test function. It must be explained that, as the minimum value of F4 and F13 is negative, the strong convergence property of BSO-CMAES makes the convergence result go into the negative value early, so that it can not show the whole in the graph. We add the error values of results to all the algorithm. In F29, the convergence performance of each algorithm is significantly different. The BSO-CMAES achieves the best convergence effect around the 1000th iteration and is maintained until termination. In F30, BSO-CMAES appears to be lost to BSO in the early stage of the algorithm, but eventually it finds a lower value. F53 is a composition function, before the 500th iteration, BSO-CMAES gets the better value than the other algorithms and

146

Y. Yu et al. BSO−CMAES

BSO

CMA−ES

DE

GSA

GWO

ABC

WOA

(a) Convergence Graph: F2

10

Average best−so−far

Average best−so−far

10

8

10

6

10

4

10

(d) Convergence Graph: F29

12

10

10

10

8

10

6

10

4

10

2

0

500

1000

1500

2000

2500

10

3000

0

500

1000

Iterations

(b) Convergence Graph: F4

7

Average best−so−far

Average best−so−far +1100

6

5

10

4

10

3

10

3000

8

10

6

10

4

10

2

2

10 0

500

1000

1500

2000

2500

3000

0

500

1000

(c) Convergence Graph: F13

4

1500

2000

2500

3000

Iterations

Iterations

(f) Convergence Graph: F53

5

10

10

Average best−so−far

Average best−so−far +200

2500

(e) Convergence Graph: F30

10

10

3

10

2

10

1

10

2000

10

10

10

1500

Iterations

4

10

3

0

500

1000

1500

Iterations

2000

2500

3000

10

0

500

1000

1500

2000

2500

3000

Iterations

Fig. 6 Convergence graphs

maintains the best to the end. From the above description, we can conclude that the convergence speed of BSO-CMAES is satisfactory and the optimization effect is obvious.

Brain Storm Algorithm Combined with Covariance Matrix Adaptation …

5.3.2

147

Stability Analysis

Considering that the stability of the algorithm is essential for the computational performance, the box and whisker diagrams for the eight algorithms on several test functions are shown in Fig. 7. To make the explanation about stability more credible as much as possible, we take the comparison results of three test functions from CEC’13 and CEC’17 respectively, i.e., F2, F4, F13, F29, F30 and F53 in Fig. 7. In these six diagrams, it can be found that the median, maximum, and minimum values of BSO-CMAES are smaller than other algorithms. In F2, F4 and F13, the maximums of BSO-CMAES is even smaller than the minimums of other algorithms. In F29 and F30, BSO-CMAES has achieved a far better result than the other algorithms, so that the interquartile range (IQR) is surprisingly short. In short, the low values prove the power of BSO-CMAES’s search capability; and the short IQR indicates its stability is satisfactory.

5.3.3

Friedman Statistical Test

As a summary of the experimental part, another nonparametric statistical test, the Friedman test, is done to reinforce the affirmation of the BSO-CMAES performance. Unlike the Wilcoxon test, Friedman test is a multiple comparison test. Through it, a ranking of multiple programs can be obtained. We use the Friedman test for two purposes. Firstly, emphasize the difference between BSO-CMAES and other algorithms above. Secondly, the rankings of the algorithms are given to facilitate the confirmation of the competitive position of BSOCMAES. The following is the general procedure to verify the difference between the detection algorithms by the Friedman test. 1. Take the default null hypothesis H0 that the algorithms have no significant difference. 2. Assume that the number of algorithms is K and the number of problems is n. 3. Rank the results on each problem. From the best to the worst, the serial number is from 1 to K. The ranks of algorithm j on the ith problem are expressed as j ranki , j ∈ [1, K], j ∈ N . 4. Calculate the final rank of algorithm j using its average of all ranks on all problems.  j Expressed as Rankj = 1n i ri .  K(K+1)2 12n 2 . 5. Calculate the statistical value Fj = k(k+1) Rank − j j 4 The p-value in the Friedman test is calculated from the F. Similarly, when p < 0.05, we can doubt the authenticity of the recipient. However, when it comes to the multiple comparison test, it is best to explain the avoidances to the Type I error [19, 42]. Assume that in a multiple comparison test, the number of algorithms is N and the significance level is α. Then, the probability that this test will not trigger Type I error will become (1 − α)N −1 . Since 1 − α < 1, one thing can be inferred that is the larger N becomes, the lower the credibility of

148

Y. Yu et al.

(d) Solution Distribution: F29

(a) Solution Distribution: F2 8

10

8

Fitness

Fitness

10

6

10

6

10

4

10 4

10

2

W O A

AB C

G W O

E

G SA

C

BS O −C

M

D

A− ES

AE S

BS O

M

W O A

AB C

G W O

E

G SA

D

A− ES

BS O C

BS O −C

M

M

AE S

10

(b) Solution Distribution: F4

(e) Solution Distribution: F30

5

10

5

10

Fitness

Fitness

4

10

3

4

10

10

3

10 2

BS

A W

O

C AB

O

SA

W G

G

E D

ES

O

A−

BS

O

C

−C

M

M

BS

S AE

A O W

C AB

O

E

SA

W G

G

A−

D

ES

O BS

O

C

−C

M

M

AE

S

10

(c) Solution Distribution: F13

(f) Solution Distribution: F53 4

10

Fitness

Fitness

2

10

1

Fig. 7 Box and whisker diagrams

A O W

C AB

O

SA

W G

G

E D

ES

O

A− M C

M AE −C O BS

BS

S

A O W

C AB

O

E

SA

W G

G

A−

D

ES

O BS

M C

BS

O

−C

M

AE

S

10

Brain Storm Algorithm Combined with Covariance Matrix Adaptation … Table 8 Adjusted Friedman test p-value Algorithm Ranking Unadjusted p BSO-CMAES versus WOA ABC GSA BSO CMA-ES GWO DE

149

pBonf

pHolm

pHochberg

1.9298

–

–

–

–

6.4123 5.9123 5.0351 4.5614 4.4649 4.0175 3.6667

0 0 0 0 0 0.000005 0.000153

0 0 0 0 0 0.000038 0.001074

0 0 0 0 0 0.000011 0.000153

0 0 0 0 0 0.000011 0.000153

non-triggered Type I error becomes. In our test, to avoid Type I error, the BonferroniDunn procedure, the Holm procedure and the Hochberg procedure are used to adjust the p-value. Three adjusted p-values are shown in Table 8. Even if the adjustment scheme is conservative, the p-value is higher than the actual. All the adjusted p-values are less than 0.05 in Table 8, which means BSO-CMAES outperforms the other seven algorithms.

6 Discussion Whether an algorithm can easily fall into a local optimum is largely influenced by the population diversity. In general, the smaller the diversity is, the easier it is to fall into the local optimal. Therefore, in order to confirm the capability of BSO-CMAES in detail, we also calculate its population diversity and compare it with the original BSO. The following is the function we use to calculate the population diversity. xi − x¯  1

n i=1 max1≤i,j≤n xi − xj n

d (x) =

(6)

where x¯ is the average which is defined as 1 xi n i=1 n

x¯ =

(7)

In the above equation, n is the total number of individuals, xi is the ith individual, and xj is the jth individual. Figure 8 shows the population diversity curves of BSO-CMAES and BSO. The horizontal axis shows the number of iterations, and the vertical axis shows the popula-

150

Y. Yu et al.

(a)

(d)

F2

0.25

F29

0.25

BSO−CMAES BSO

Population Diversity

Popolation Diversity

BSO−CMAES BSO

0.2

0.15

0.1

0.05

0

0

0.5

1

1.5

2

0

0.5

1

1.5

2

(e)

BSO−CMAES BSO

Population Diversity

0.14

0.1 0.08 0.06 0.04

3 5 x 10

F29

0.25

0.12

2.5

Iterations

BSO−CMAES BSO

0.16

Population Diversity

0.05

x 10

F4

0.18

0.1

5

Iterations

(b)

0.15

0

3

2.5

0.2

0.2

0.15

0.1

0.05

0.02 0

0 0

0.5

1

1.5

2

2.5

Iterations

(c)

1

(f)

0.14 0.12 0.1 0.08 0.06

2

2.5

3 5 x 10

BSO−CMAES BSO

0.3

Population Diversity

0.18 0.16

1.5

F53

0.35 BSO−CMAES BSO

0.2

Population Diversity

0.5

Iterations

F13

0.22

0.25 0.2 0.15 0.1 0.05

0.04 0.02

0

3 5 x 10

0

0.5

1

1.5

Iterations

2

2.5

3 5 x 10

0

0

0.5

1

1.5

Iterations

2

2.5

3 5 x 10

Fig. 8 Population diversity

tion diversity. The diversity of BSO-CMAES outperforms BSO according to Fig. 8. Of the remaining issues, it is true that the peak of the BSO’s diversity curve has exceeded the diversity of BSO-CMAES at the current iteration. However, in terms of the number of iterations of the overall function, BSO-CMAES is in the leading

Brain Storm Algorithm Combined with Covariance Matrix Adaptation …

151

position most of the time. In addition, at the end of the iteration process, the population diversity of BSO-CMAES is also more dominant. Shortly, The parallel strategy between BSO and CMA-ES is indeed more able to ensure the population diversity.

7 Conclusions This paper is inspired by the current popular hybrid algorithm optimization scheme and proposes a hybrid strategy of BSO parallel CMA-ES. It is intended to combine the search strength of BSO and the search speed of CMA-ES. In order to prove the effectiveness of the hybrid scheme, we conduct two types of experiments. In the first type of experiment, we compare BSO-CMAES and its two source algorithms, including BSO and CMA-ES. The results show that the parallel scheme of BSOCMAES is indeed effective and improves the performance of the algorithm. In the second type of experiment, we compare BSO-CMAES with seven other algorithms. This experiment shows that BSO-CMAES still maintains its competitiveness in the face of multiple algorithms. After the experiments, the ability of the algorithm to ensure population diversity is included in the discussion. Comparison of the population diversity of BSO-CMAES and BSO is made. The ability of BSO-CMAES to ensure population diversity is superior to that of BSO. It also suggests that BSO-CMAES is more capable of ensuring that it does not fall into local optima. In our future work, the following points may be considered: (1) whether there are any other algorithms more suitable for paralleling with BSO; (2) whether there are other hybrid schemes to optimize the performance of BSO; (3) the effect of other clustering methods on BSO performance; and (4) apply BSO to other practical problems [4–6, 38–41, 62].

References 1. Boussaid, I., Chatterjee, A., Siarry, P., Ahmed-Nacer, M.: Hybridizing biogeography-based optimization with differential evolution for optimal power allocation in wireless sensor networks. IEEE Trans. Veh. Technol. 60(5), 2347–2353 (2011) 2. Cao, Z., Hei, X., Wang, L., Shi, Y., Rong, X.: An improved brain storm optimization with differential evolution strategy for applications of ANNs. Math. Probl. Eng. (2015) 3. Chang, L., Liao, C., Lin, W., Chen, L.L., Zheng, X.: A hybrid method based on differential evolution and continuous ant colony optimization and its application on wideband antenna design. Prog. Electromagnet. Res. 122, 105–118 (2012) 4. Cheng, J., Cheng, J., Zhou, M., Liu, F., Gao, S., Liu, C.: Routing in internet of vehicles: a review. IEEE Trans. Intell. Transp. Syst. 16(5), 2339–2352 (2015) 5. Cheng, J., Mi, H., Huang, Z., Gao, S., Zang, D., Liu, C.: Connectivity modeling and analysis for internet of vehicles in urban road scene. IEEE Access 6, 2692–2702 (2018)

152

Y. Yu et al.

6. Cheng, J., Wu, X., Zhou, M., Gao, S., Huang, Z., Liu, C.: A novel method for detecting new overlapping community in complex evolving networks. IEEE Trans. Syst. Man Cybern. Syst. (2018). https://doi.org/10.1109/TSMC.2017.2779138 7. Cheng, S., Qin, Q., Chen, J., Shi, Y.: Brain storm optimization algorithm: a review. Artif. Intell. Rev. 46(4), 445–458 (2016) 8. Deng, W., Chen, R., Gao, J., Song, Y., Xu, J.: A novel parallel hybrid intelligence optimization algorithm for a function approximation problem. Computers Math. Appl. 63(1), 325–336 (2012) 9. Deng, W., Chen, R., He, B., Liu, Y., Yin, L., Guo, J.: A novel two-stage hybrid swarm intelligence optimization algorithm and application. Soft Comput. 16(10), 1707–1722 (2012) 10. Deng, W., Li, W., Yang, X.H.: A novel hybrid optimization algorithm of computational intelligence techniques for highway passenger volume prediction. Expert Syst. Appl. 38(4), 4198– 4205 (2011) 11. Gao, S., Song, S., Cheng, J., Todo, Y., Zhou, M.: Incorporation of solvent effect into multiobjective evolutionary algorithm for improved protein structure prediction. IEEE/ACM Trans. Comput. Biol. Bioinf. 15(4), 1365–1378 (2018) 12. Gao, S., Vairappan, C., Wang, Y., Cao, Q., Tang, Z.: Gravitational search algorithm combined with chaos for unconstrained numerical optimization. Appl. Math. Comput. 231, 48–62 (2014) 13. Gao, S., Wang, W., Dai, H., Li, F., Tang, Z.: Improved clonal selection algorithm combined with ant colony optimization. IEICE Trans. Inf. Syst. 91(6), 1813–1823 (2008) 14. Gao, S., Wang, Y., Cheng, J., Inazumi, Y., Tang, Z.: Ant colony optimization with clustering for solving the dynamic location routing problem. Appl. Math. Comput. 285, 149–173 (2016) 15. Gao, S., Wang, Y., Wang, J., Cheng, J.: Understanding differential evolution: a Poisson law derived from population interaction network. J. Comput. Sci. 21, 140–149 (2017) 16. Gao, S., Zhang, J., Wang, X., Tang, Z.: Multi-layer neural network learning algorithm based on random pattern search method. Int. J. Innov. Comput. Inf. Control 5(2), 489–502 (2009) 17. Gao, S., Zhou, M., Wang, Y., Cheng, J., Yachi, H., Wang, J.: Dendritic neural model with effective learning algorithms for classification, approximation, and prediction. IEEE Trans. Neural Network. Learn. Syst. (2018). https://doi.org/10.1109/TNNLS.2018.2846646 18. Gao, W., Liu, S., Huang, L.: Enhancing artificial bee colony algorithm using more informationbased search equations. Inform. Sci. 270, 112–133 (2014) 19. García, S., Fernández, A., Luengo, J., Herrera, F.: Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power. Inform. Sci. 180(10), 2044–2064 (2010) 20. Garg, H.: A hybrid PSO-GA algorithm for constrained optimization problems. Appl. Math. Comput. 274, 292–305 (2016) 21. Guvenc, U., Duman, S., Saracoglu, B., Ozturk, A.: A hybrid GA-PSO approach based on similarity for various types of economic dispatch problems. Elektronika ir Elektrotechnika 108(2), 109–114 (2011) 22. Hansen, N.: The CMA evolution strategy: a comparing review. In: Towards a New Evolutionary Computation, pp. 75–102. Springer (2006) 23. Ji, J., Gao, S., Wang, S., Tang, Y., Yu, H., Todo, Y.: Self-adaptive gravitational search algorithm with a modified chaotic local search. IEEE Access 5, 17881–17895 (2017) 24. Ji, J., Song, S., Tang, C., Gao, S., Tang, Z., Todo, Y.: An artificial bee colony algorithm search guided by scale-free networks. Inform. Sci. 473, 142–165 (2019) 25. Jiao, L., Wang, L.: A novel genetic algorithm based on immunity. IEEE Trans. Syst. Man Cybern.-Part A: Syst. Hum. 30(5), 552–561 (2000) 26. Juang, C.F.: A hybrid of genetic algorithm and particle swarm optimization for recurrent network design. IEEE Trans. Syst. Man Cybern. Part B (Cybernetics) 34(2), 997–1006 (2004) 27. Kämpf, J.H., Robinson, D.: A hybrid CMA-ES and HDE optimisation algorithm with application to solar energy potential. Appl. Soft Comput. 9(2), 738–745 (2009) 28. Kao, Y.T., Zahara, E.: A hybrid genetic algorithm and particle swarm optimization for multimodal functions. Appl. Soft Comput. 8(2), 849–857 (2008)

Brain Storm Algorithm Combined with Covariance Matrix Adaptation …

153

29. Kaveh, A., Ghazaan, M.I.: Hybridized optimization algorithms for design of trusses with multiple natural frequency constraints. Adv. Eng. Softw. 79, 137–147 (2015) 30. Kaveh, A., Talatahari, S.: Particle swarm optimizer, ant colony strategy and harmony search scheme hybridized for optimization of truss structures. Comput. Struct. 87(5–6), 267–283 (2009) 31. Lazzús, J.A., Rivera, M., López-Caraballo, C.H.: Parameter estimation of lorenz chaotic system using a hybrid swarm intelligence algorithm. Phys. Lett. A 380(11–12), 1164–1171 (2016) 32. Lee, Z.J., Lee, C.Y., Su, S.F.: An immunity-based ant colony optimization algorithm for solving weapon-target assignment problem. Appl. Soft Comput. 2(1), 39–47 (2002) 33. Lee, Z.J., Su, S.F., Chuang, C.C., Liu, K.H.: Genetic algorithm with ant colony optimization (GA-ACO) for multiple sequence alignment. Appl. Soft Comput. 8(1), 55–78 (2008) 34. Li, Z., Wang, W., Yan, Y., Li, Z.: PS-ABC: a hybrid algorithm based on particle swarm and artificial bee colony for high-dimensional optimization problems. Expert Syst. Appl. 42(22), 8881–8895 (2015) 35. Lien, L.C., Cheng, M.Y.: A hybrid swarm intelligence based particle-bee algorithm for construction site layout optimization. Expert Syst. Appl. 39(10), 9642–9650 (2012) 36. Lieu, Q.X., Do, D.T., Lee, J.: An adaptive hybrid evolutionary firefly algorithm for shape and size optimization of truss structures with frequency constraints. Comput. Struct. 195, 99–112 (2018) 37. Lin, Q., Chen, J., Zhan, Z.H., Chen, W.N., Coello, C.A.C., Yin, Y., Lin, C.M., Zhang, J.: A hybrid evolutionary immune algorithm for multiobjective optimization problems. IEEE Trans. Evol. Comput. 20(5), 711–729 (2016) 38. Liu, C., Duan, H., Qingtian, Z., Zhou, M., Lu, F., Cheng, J.: Towards comprehensive support for privacy preservation cross-organization business process mining. IEEE Trans. Serv. Comput. (2016). https://doi.org/10.1109/TSC.2016.2617331 39. Liu, C., Zeng, Q., Duan, H., Zhou, M., Lu, F., Cheng, J.: E-net modeling and analysis of emergency response processes constrained by resources and uncertain durations. IEEE Trans.Syst. Man Cybern. Syst. 45(1), 84–96 (2015) 40. Liu, C., Zhang, J., Li, G., Gao, S., Zeng, Q.: A two-layered framework for the discovery of software behavior: a case study. IEICE Trans. Inform. Syst. 101(8), 2005–2014 (2018) 41. Liu, Y., Cheng, D., Wang, Y., Cheng, J., Gao, S.: A novel method for predicting vehicle state in internet of vehicles. Mob. Inform. Syst. (2018) 42. Luengo, J., García, S., Herrera, F.: A study on the use of statistical tests for experimentation with neural networks: analysis of parametric test conditions and non-parametric tests. Expert Syst. Appl. 36(4), 7798–7808 (2009) 43. Marinakis, Y., Marinaki, M.: A hybrid multi-swarm particle swarm optimization algorithm for the probabilistic traveling salesman problem. Comput. Oper. Res. 37(3), 432–442 (2010) 44. Mirjalili, S., Mirjalili, S.M., Lewis, A.: Grey wolf optimizer. Adv. Eng. Softw. 69, 46–61 (2014) 45. Nguyen, T.T., Li, Z., Zhang, S., Truong, T.K.: A hybrid algorithm based on particle swarm and chemical reaction optimization. Expert Syst. Appl. 41(5), 2134–2143 (2014) 46. Niknam, T., Amiri, B.: An efficient hybrid approach based on PSO, ACO and k-means for cluster analysis. Appl. Soft Comput. 10(1), 183–197 (2010) 47. Nwankwor, E., Nagar, A.K., Reid, D.: Hybrid differential evolution and particle swarm optimization for optimal well placement. Comput. Geosci. 17(2), 249–268 (2013) 48. Pan, G., Li, K., Ouyang, A., Li, K.: Hybrid immune algorithm based on greedy algorithm and delete-cross operator for solving tsp. Soft Comput. 20(2), 555–566 (2016) 49. Panda, S., Kiran, S.H., Dash, S.S., Subramani, C.: A PD-type multi input single output sssc damping controller design employing hybrid improved differential evolution-pattern search approach. Appl. Soft Comput. 32, 532–543 (2015) 50. Panda, S., Yegireddy, N.K.: Multi-input single output SSSC based damping controller design by a hybrid improved differential evolution-pattern search approach. ISA Trans. 58, 173–185 (2015) 51. Shelokar, P., Siarry, P., Jayaraman, V.K., Kulkarni, B.D.: Particle swarm and ant colony algorithms hybridized for improved continuous optimization. Appl. Math. Comput. 188(1), 129– 142 (2007)

154

Y. Yu et al.

52. Shi, Y.: Brain storm optimization algorithm. In: International Conference in Swarm Intelligence, pp. 303–309. Springer (2011) 53. Shuang, B., Chen, J., Li, Z.: Study on hybrid PS-ACO algorithm. Appl. Intell. 34(1), 64–73 (2011) 54. Soleimani, H., Kannan, G.: A hybrid particle swarm optimization and genetic algorithm for closed-loop supply chain network design in large-scale networks. Appl. Math. Modell. 39(14), 3990–4012 (2015) 55. Song, S., Ji, J., Chen, X., Gao, S., Tang, Z., Todo, Y.: Adoption of an improved PSO to explore a compound multi-objective energy function in protein structure prediction. Appl. Soft Comput. (2018). https://doi.org/10.1016/j.asoc.2018.07.042 56. Song, Z., Gao, S., Yu, Y., Sun, J., Todo, Y.: Multiple chaos embedded gravitational search algorithm. IEICE Trans. Inform. Syst. 100(4), 888–900 (2017) 57. Storn, R., Price, K.: Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J. Glob. Optimization 11(4), 341–359 (1997) 58. Sun, J., Gao, S., Dai, H., Cheng, J., Zhou, M., Wang, J.: Bi-objective elite differential evolution for multivalued logic networks. IEEE Trans. Cybern. (2018). https://doi.org/10.1109/TCYB. 2018.2868493 59. Sun, Y.: A hybrid approach by integrating brain storm optimization algorithm with grey neural network for stock index forecasting. Abstract Appl. Anal. 2014. Hindawi (2014) 60. Tran, D.H., Cheng, M.Y., Cao, M.T.: Hybrid multiple objective artificial bee colony with differential evolution for the time-cost-quality tradeoff problem. Knowl. Based Syst. 74, 176– 186 (2015) 61. Trivedi, A., Srinivasan, D., Biswas, S., Reindl, T.: Hybridizing genetic algorithm with differential evolution for solving the unit commitment scheduling problem. Swarm Evol. Comput. 23, 50–64 (2015) 62. Wang, J., Cen, B., Gao, S., Zhang, Z., Zhou, Y.: Cooperative evolutionary framework with focused search for many-objective optimization. IEEE Trans. Emerg. Top. Comput. Intell. (2018). https://doi.org/10.1109/TETCI.2018.2849380 63. Wang, Y., Gao, S., Yu, Y., Xu, Z.: The discovery of population interaction with a power law distribution in brain storm optimization. Memetic Comput. (2017). https://doi.org/10.1007/ s12293-017-0248-z 64. Xin, B., Chen, J., Zhang, J., Fang, H., Peng, Z.H.: Hybridizing differential evolution and particle swarm optimization to design powerful optimizers: a review and taxonomy. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 42(5), 744–767 (2012) 65. Xu, Z., Wang, Y., Li, S., Liu, Y., Todo, Y., Gao, S.: Immune algorithm combined with estimation of distribution for traveling salesman problem. IEEJ Trans. Electr. Electron. Eng. 11, S142– S154 (2016) 66. Yu, Y., Gao, S., Cheng, S., Wang, Y., Song, S., Yuan, F.: CBSO: a memetic brain storm optimization with chaotic local search. Memetic Comput. (2017). https://doi.org/10.1007/s12293017-0247-0 67. Yu, Y., Gao, S., Wang, Y., Cheng, J., Todo, Y.: ASBSO: an improved brain storm optimization with flexible search length and memory-based selection. IEEE Access 6, 36977–36994 (2018) 68. Zheng, Y.J., Xu, X.L., Ling, H.F., Chen, S.Y.: A hybrid fireworks optimization method with differential evolution operators. Neurocomputing 148, 75–82 (2015)

Part III

Applications

A Feature Extraction Method Based on BSO Algorithm for Flight Data Hui Lu, Chongchong Guan, Shi Cheng and Yuhui Shi

Abstract The feature extraction problem for flight data has aroused increasing attention in the practical and the academic aspects. It can reveal the inherent correlation relation among different parameters for the conditional maintenance of the aircraft. However, the high-dimensional and continuous features in the real number field bring challenges to the extraction algorithms for flight data. Brain Storm Optimization (BSO) algorithm can acquire the optimal solutions by continuously converging and diverging the solution set. In this chapter, a feature extraction method based on BSO algorithm is proposed to mine the associate rules from flight data. By using the designed real-number encoding strategy, the intervals and rule template can be handled directly without data discretization and rule template preset processes. Meanwhile, as the frequent item generation process is unnecessary in our proposed algorithm, the time and space complexity will be reduced simultaneously. In addition, we design the fitness function using support, confidence and length of the rules for the purpose of extracting more practical and intelligible rules without predetermining the parameter thresholds. Besides, high-dimensional problems can also be solved using our algorithm. The experiments using substantial flight data are conducted to illustrate the excellent performance of the proposed BSO algorithm comparing to the Apriori algorithm and Genetic algorithm (GA). Furthermore, the classification problems with two datasets from UCI database are also used to verify the practicability and universality of the proposed method based on BSO algorithm. H. Lu (B) · C. Guan School of Electronic and Information Engineering, Beihang University, Beijing 100191, China e-mail: [email protected] C. Guan e-mail: [email protected] S. Cheng School of Computer Science, Shaanxi Normal University, Xi’an 710119, China e-mail: [email protected] Y. Shi Shenzhen Key Lab of Computational Intelligence, Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen 518055, China e-mail: [email protected] © Springer Nature Switzerland AG 2019 S. Cheng and Y. Shi (eds.), Brain Storm Optimization Algorithms, Adaptation, Learning, and Optimization 23, https://doi.org/10.1007/978-3-030-15070-9_7

157

158

H. Lu et al.

Keywords Flight data · Feature extraction · BSO algorithm · Associate rule mining · Apriori algorithm

1 Introduction Flight data feature extraction is one kind of critical technology in the field of prognostic health management (PHM) for aircraft [15]. It can support the flight status monitoring, maintenance decisions making and accident investigation. Recently, with the occurrence of civil aviation accidents around the world, PHM attracts more and more attention. As a result, the flight data feature extraction problem has also drawn great public concern. The objective of the feature extraction problem is to mine the potential links among different parameters based on the flight data. The obtained associated rules can help us to analyze the flight status and predict the fault information. Actually, feature extraction is a very popular problem in different fields, like market trend analysis and fault diagnosis. For flight data feature extraction, there are many parameters for an aircraft to represent the quality and flight status. In addition, the flight data is from the flight parameter recorder, and the data sampling rate for each parameter is one kilohertz on average. Therefore, flight data feature extraction problem has large solution space. The target of the feature extraction is to mine the associate rules from the large scale database. As a popular and classical algorithm, the Apriori algorithm can be used to solve the feature extraction problem. For example, Harikumar and Dilipkumar [10] proposed a variant of Apriori algorithm to mine association rules from high-dimensional data. Besides, a novel Apriori algorithm with various techniques is proposed by Yadav et al. [29] to improve the efficiency and accuracy of the algorithm in solving feature extraction algorithm. In general, Apriori algorithm consists of two steps, the frequent items generation and association rules mining. It is a very convenient and easy-understanding algorithm. However, the algorithm has high time complexity and space complexity during the frequent items generation process. Moreover, it needs the priori knowledge for predetermining the minimum support and minimum confidence. Furthermore, the continuous values in the database need to be discretized for the adaptability of the algorithm. In order to simplify the frequent items generation process and solve the large scale problems, evolutionary algorithms are applied to solving the feature extraction problem, such as genetic algorithm (GA), Particle Swarm Optimization (PSO) and so on. For instance, Seki and Nagao [18] proposed an efficient java implementation of a GA-based miner for feature extraction problem. A hybrid Apriori-genetic algorithm is used to generate rules by Chadokar et al. [3] to reduce the computational time and simplify the frequent sets generation. In addition, the PSO algorithm is also adapted to solve feature extraction problem. For example, Indira and Kanmani [12] proposed an adaptive parameter control PSO algorithm to extract features. Shang and Wu [19] implemented an improved PSO algorithm to mine association rules. It is worth nothing that the evolutionary algorithms could reduce the computational

A Feature Extraction Method Based on BSO Algorithm for Flight Data

159

time or generate less frequent items. Nevertheless, the frequent items generation, the preset thresholds for support and confidence, and data discretization still have a great influence on the algorithm efficiency. Besides, the rule templates also need to be preset in the process of the evolutionary algorithm, which will surely reduce the diversity of the extracted rules. Therefore, extracting features from a large and numeric database without data discretization, frequent items generation, rule template predetermined and thresholds preset is the key issue for enhancing performance of the algorithm. In this chapter, an improved Brain Storm Optimization (BSO) [5] is used to solve the feature extraction problem for flight data. The BSO algorithm is a typical swarm intelligence optimization algorithm with excellent evolutionary performance. As shown in the experimental results of flight data, the proposed algorithms can get superior solutions comparing to Apriori algorithm and GA. In addition, the universality of the improved BSO algorithm is verified by solving the classification problems in UCI database. Our main contribution focus on three aspects. 1. We propose a feature extraction algorithm based BSO algorithm to find the association rules in the flight data which is high-dimensional. Comparing to the Apriori algorithm and GA, our proposed algorithm can extract features without complex data discretization, thresholds predetermined and rule template preset. As a result, the time complexity and space complexity are also decreased without frequent items generation process. 2. In addition, the rules we extracted are also more practical and nonredundant, comparing to the Apriori algorithm and GA. In addition, the proposed algorithm has better performance in support, confidence and rule length. 3. Moreover, the algorithm has the practicability and universality. We also apply our proposed algorithm to the classification feature extraction problem with dataset in UCI database and attain reasonable rules. The remainder of the chapter is constructed as follows. Section 2 investigates the related work in feature extraction problems and relevant algorithms. Section 3 introduces the framework and details of the improved BSO algorithm for the feature extraction problem. Section 4 illustrates the experiment design and analyzes the results. Finally, Sect. 5 summarizes the chapter.

2 Related Work 2.1 Feature Extraction Algorithm In general, the methods for feature extraction problems are divided into traditional and intelligent algorithms. For conventional algorithm, Dongre et al. [7] focused on the Apriori algorithm to find frequent item sets from a discrete transaction dataset and derive association rules.

160

H. Lu et al.

Many feature extraction algorithms are based on Apriori algorithm. As the Apriori algorithm faces challenges in time efficiency and space complexity, researchers explored out new extraction methods. For example, He et al. [11] used FP-tree and Mapreduce to extract features, which is a useful application to data mining. Solanki and Patel [22] investigated the improved algorithms for Boolean and Fuzzy feature extraction, like Apriori, FP-tree, Fuzzy FP-tree. However, most of these algorithms can only extract features from Boolean discrete data. Moreover, the extracted processes are still divided into the frequent items generation step and the features extraction step. During the two steps, the rule template, minimum support and minimum confidence need to be predetermined with priori knowledge. However, the priori knowledge is not available, and the databases we conduct are usually numeric or categorical. Therefore, many researchers concentrate on proposing new algorithms to extract features from the numeric databases directly. Besides, many intelligent optimization algorithms with global optimization ability have been used to solve the feature extraction problem. For instance, Jain et al. [13] used the improved GA algorithm to extract positive and negative rules and attained better results comparing with the previous algorithm. Dhanore and Chaturvedi [6] proposed an optimization algorithm of feature extraction for large database using GA and K-map. In addition, Nandhini et al. [16] applied the BSO algorithm and domain ontology to extract features. The data discretization process is non-ignorable in all the above algorithms. In fact, extracting features from a numeric database is a tough problem comparing to extracting that from the discrete database. In order to solve the feature extraction problem in numeric or category database, Wang et al. [27] used a hybrid GA to extract features from the numeric database. Furthermore, Alikhademi and Zainudin [1] applied the PSO algorithm to extract the fuzzy rules. In the above two proposed algorithms, the data discretization and the high-quality rules extraction are executed at the same time. For the purpose of extracting meaningful parameter features directly from various databases, containing the numeric and category database, researchers pay more attention to exploring new evolutionary computation techniques based on swarm intelligence. The BSO algorithm is an example. As a new and promising swarm intelligence algorithm, the BSO algorithm simulates the human brain storming process [20]. Each individual in the BSO algorithm is not only a solution to the optimal problem, but also a data point to reveal the landscape of the problem [5]. In the search space, the optimal solution can be obtained by continuously converging and diverging the solution set. Therefore, the characteristics of BSO algorithm decide that it can be applied in optimization problems. The advantage of BSO algorithm is also proved by a modified self-adaptive BSO algorithm [9].

A Feature Extraction Method Based on BSO Algorithm for Flight Data

161

2.2 Practical Application The feature extraction method is a practical technique in the real world. It is applied to solve many complex problems in various domains, such as medical, recommendation system, fault diagnosis, fraud detection and so on. In the medical and health care field, the clinical records of patients are applied to extract meaningful features for medical diagnosis. The information in the clinical records are complex and underlying. Therefore, it is necessary to simplify diagnostic process and improve accuracy simultaneously using the feature extraction method. For instance, Sonet et al. [23] used the Apriori algorithm to identify the hidden rules of the most commonly occurring heart diseases. Song and Ma [24] established an intelligent diagnosis model of lung cancer based on the Apriori algorithm using the testing data from clinical diagnoses. Rameshkumar et al. [17] used the partition based technique to reduce large number of irrelevant rules and produce new set of rules with high confidence. Moreover, most of the applications focus on Boolean type or discrete data. In addition, feature extraction is also crucial in the recommendation system. With the rise of big data in the network and other domains, various recommendation systems which can help us make decisions faster according to the big data. The feature extraction technology can increase the efficiency of the recommendation system. Tewari and Barman [25] proposed a book recommendation system that used association rule mining to capture current reading trend of users in the network. An intelligence recommendation scheme of scenic spots based on the Apriori algorithm was put forward by Xi and Yuan [28]. Besides, Zhang et al. [31] extracted the TaskOriented features with a multi-objective evolutionary algorithm. In the fault diagnosis domain, it is vital to detect fault in time as some unplanned failures occurring in equipment will cause huge economic losses and great society influence. Since the fault diagnosis is similar to the feature extraction problem, many researchers applied the feature extraction method to solving fault detection problem. For example, an approach based on association rule and knowledge base was proposed and implemented in the fault diagnosis of a transformer system [30]. Furthermore, a novel diagnosis approach combining SVM with association rule was used to achieve fault diagnosis for ship diesel engine [2]. Gao et al. [8] also presented a new method of the distribution network fault diagnosis based on the feature extraction algorithm, the Apriori algorithm. Moreover, the feature extraction techniques also play a central role in fraud detection domain. As a very challenging task, the fraud detection problem needs efficient feature extraction method to deal with the given dataset. Kareem et al. [14] proposed a feature extraction approach based on Apriori algorithm to detect fraudulent health insurance claims. Verma et al. [26] also used the feature extraction method to untangle the fraud identification frequent patterns underlying in the insurance claim data.

162

H. Lu et al.

There is no much study about BSO algorithm for feature extraction. In order to explore the excellent global optimization ability of BSO algorithm, we investigate the BSO in the feature extraction with flight data.

3 The Improved BSO Algorithm 3.1 Algorithm Framework The Brain Storm Optimization (BSO) algorithm is an emerging swarm intelligence optimization algorithm drawing on brain-storming process which simulates the collective behavior of a human brainstorming solution. The BSO algorithm is firstly introduced by Shi in 2011 [20, 21]. As an efficient global optimization algorithm, every individual in a group stands for a potential solution for the problem. New individuals are generated based on the existing individuals by several ways. The processes of the BSO algorithm consist of five parts: initialization, clustering, updating, selecting and evaluating. The initial BSO algorithm [4] is shown in Algorithm 1. • n: the number of initial individuals; • m: the number of the classes that the individuals are divided to

Algorithm 1: Brain storm optimization (BSO) algorithm process Step 1. Initialization: generate n initial individuals, and evaluate every individual; while the optimization solution does not satisfy conditions or the predetermined maximum number of iterations is not reached do Step 2. Clustering: divide these n individuals into m classes by clustering algorithm; Step 3. Updating: generate a new individual with one or two clusters randomly; Step 4. Selecting: select the central individual of a class randomly; Step 5. Evaluating: evaluate every individual;

In step 1, each individual is evaluated using the fitness function predetermined. In step 2, the k-means algorithm is used to cluster the n individuals, and the best individual in each category is selected as the center of the class. In addition, the operation of updating means generating new individuals using the existed ones. Furthermore, in the selecting process, after comparing the selected individual with a new generated individual, the better one is kept and becomes the new individual. At last, the n individuals need to be evaluated again. Step 2, 3, 4 and 5 will be repeated until the optimization solution satisfies conditions or the predetermined maximum number of iterations is reached.

A Feature Extraction Method Based on BSO Algorithm for Flight Data

163

3.2 Improved BSO Algorithm for Feature Extraction 3.2.1

Data Pre-processing

For flight data, the value ranges of attributes have a significant difference. For example, the value range of pressure altitude is from 268 to 27,668, the air temperature ranges from −1.79 to 30.5, and the values of engine pressure radio even vary from 0.819 to 1.58. Considering this situation, it is indispensable to normalize the parameters in advance. The data range comes to [−1, 1] after the normalization. The value of the database is normalized using the following equation. y=

(ymax − ymin ) × (x − xmin ) + ymin xmax − xmin

(1)

In the above normalized equation, x stands for the value which is going to be normalized and y is the normalized value. In addition, ymax is the upper limit, and ymin is the lower limit. After the normalization process, the numeric operation to the flight data will be unified.

3.2.2

Encoding Strategy

For solving the feature extraction problem, encoding strategy is an essential step for the design of the rule mining algorithm. Here, we encode one chromosome as one rule. In addition, the encoding strategy is one kind of real-number encoding. Therefore, it can deal with the intervals and rule templates directly without discretizing the database in advance. The structure of one encoded individual is shown in Table 1. ⎧ antecedent ⎨ 0 ≤ ACi ≤ 0.33, ACi = 0.33 < ACi ≤ 0.66, consequence (2) ⎩ out of rule. 0.66 < ACi ≤ 1, Every individual consists of p attributes. Moreover, each attribute consists of three parts. The first part of each attribute represents the position of the value, such as the antecedent or consequence of one rule. It ranges from 0 to 1, and the meaning of different values are described in Eq. (2). If ACi is in the interval [0, 0.33], the attribute will be the antecedent of a rule. At the same time, the attribute will be the

Table 1 Individual structure Attribute 1 Attribute 2 AC1 LB1 U B1

AC2 LB2 U B2

…

Attribute p

…

ACp LBp U Bp

164

H. Lu et al.

Fig. 1 Example of an individual

consequence of a rule if the value of ACi is in the second interval. Besides, the third interval means that the attribute is out of the rule and we can ignore the later part of the attribute. It can be find that all the attributes of an individual whose first part is in the antecedent interval will form the antecedent of a rule while all the attributes in consequence interval will constitute the whole consequence. For a valid attribute, the second part of it stands for the lower bound which is defined as LBi , and the third part U Bi is the upper bound. Here, we use an example of an individual in Fig. 1 to illustrate the encoding strategy. The rule in Fig. 1 is A [5, 10], B [6, 8] ⇒ D [12, 15]. It is obvious that attributes contained in the antecedent or consequence of a rule are uncertain, which means the consequence may have more than one attribute. In addition, the number of antecedent and consequence in a rule need to be greater than 0, because a rule which does not have antecedent or consequence does not exist in the real world. Considering the randomness of the individual initialization, the production of invalid solutions is possible. Therefore, we generate more generic individuals by reducing the number of attributes contained in the initial individuals. Specifically, we increase the possibility of the attributes out of the rule. After that, one rule tends to cover more records. For example, when the value of the first part AC is greater than 0.56 or lower than 0.1, it will be changed to 0.9, which means than the attribute is out of the rule. Moreover, in the classification feature extraction problem, because of the fixed category of each rule, the consequence of each individual is limited to one attribute.

3.2.3

Fitness Function

The support and confidence are two standard metrics for evaluation the quality of association rule in feature extraction problems. They are defined as follows. (1) Support Support is defined as the ratio of records that contain antecedent A and consequence B to the whole number of records in the database. It shows the occurrence frequency of the rule. The function can be calculated using the following equation. support(A ⇒ B) = P(A, B)

(3)

A Feature Extraction Method Based on BSO Algorithm for Flight Data

165

(2) Confidence Confidence is the ratio of the number of records that contain both antecedent A and consequence B to the whole number of records that contains antecedent A. The confidence is defined as a conditional probability as follows. confidence(A ⇒ B) = P(B|A)

(4)

Furthermore, It is apparent that the extracted rules, which have better support, higher confidence and fewer attributes are more practical. Hence, we prefer to extract shorter length rules. Specifically, we design the fitness function with support, confidence and length of the rules for the purpose of extracting more representative and concise rules. The definition of the fitness function is as following. fitness = α1 × support(Ant ⇒ Cons) + α2 × confidence(Ant ⇒ Cons) − α3 × p(∗)

(5)

The fitness function consists of three parts. For the first and second part, Ant and Cons stand for the antecedent and consequence of a rule, respectively. Moreover, support(Ant ⇒ Cons) and confidence(Ant ⇒ Cons) in the fitness function are defined by the two equations below. support(Ant ⇒ Cons) = P(Ant, Cons)

(6)

confidence(Ant ⇒ Cons) = P(Cons|Ant)

(7)

In addition, the third part of the fitness function is the length of the extracted rule. In this part, p(∗) is the number of attributes in the extracted rule. In the fitness function, α1 , α2 and α3 are the parameters predefined according to actual demand and they can be adjusted by users.

3.2.4

New Individual Generation

In the individual generation process, a new individual can be generated from one or several existed individuals. Specifically, it can be constructed with one or two cluster center or non-center. After comparing the new generated individual to the individual waiting to be updated, the better one will be retained in the new generation. In addition, the n individuals will be updated in turn until the predetermined maximum number of iterations is reached. Moreover, when the new individual is relatively close to the known optimal individual, the BSO algorithm tends to the exploitation ability. And the exploration ability of the algorithm will be enhanced when the new individual is randomly generated or based on two clusters.

166

H. Lu et al.

Fig. 2 New individual generation with one cluster or two clusters

The procedure of new individual generation is shown in Fig. 2. The left sub-figure shows the process of generating a new individual from one cluster, and the right one represents the process of two clusters. The functions used in the generation procedure is defined by Eqs. (8) and (9). In the BSO algorithm, the new individual is generated by adding a random value to an existed individual as shown in Eq. (8). In addition, the random value is limited to the step size, which is defined with Eq. (9). i i = xold + ξ(t) × N (μ, σ 2 ) xnew 0.5 × T − t ξ(t) = log sig( ) × rand() k

(8) (9)

In Eq. (8), xnew represents the new generated individual and xold is the existed individi i and xold stand for the ith dimension of xnew and xold , respectively. ual. Besides, xnew Furthermore, Eq. (9) defines the step size function. In this equation, t stands for current times of iteration. T is the maximum number of iterations. c is a coefficient which is used to change the slope of function log sig(·). The transfer function log sig(a) is defined by Eq. (10). 1 (10) log sig(a) = 1 + exp(−a) After the iteration process, the new individual will be compared with the existed individual which has the same individual index and the better one will be recorded as the new one.

A Feature Extraction Method Based on BSO Algorithm for Flight Data

167

Fig. 3 Moving infeasible individual back from one cluster or two clusters

3.2.5

Strategy for Infeasible Solution

In the process of new individuals generation, there will be infeasible solutions inevitably. For instance, when two individuals combine to produce a new individual, the values of some attributes may get out of their range. Whenever the attribute values escape from the defined value range, we should apply maintenance operations. For the purpose of moving back the infeasible solutions, we use the method of generating values randomly. For example, when the value of an attribute is out of bound, a random value in the bounds will be generated to replace the infeasible values. One thing we should pay attention to is that the lower bounds of an attribute in a rule must be less than the upper bounds. By this way, the invalid solution is moved back, and the diversity of individual still exists. The moving back process of an infeasible solution is shown in Fig. 3. The left subfigure shows the situation of one cluster, and the process of two clusters is represented with the right sub-figure.

3.2.6

Details of the Improved BSO Algorithm

We summary the details of the improved BSO algorithm, and the pseudo code of the BSO algorithm is shown in Algorithm 2. The parameters of the algorithm are as follows. • n is the number of initial individuals; • m is the number of the classes that the individuals are divided to; • Pgeneration is used to determine a new individual being generated by one or two existed individuals;

168

H. Lu et al.

• PoneCluster is used to determine the cluster center or another normal individual will be chosen in one cluster generation case; • PtwoCluster is used to determine the cluster center or another normal individual will be chosen in two clusters generation case; • Pelite is used to determine the individual will be chosen as the elite.

4 Experiments and Analysis The experiments consist of two parts. For the first part, we conduct the flight parameter feature extraction experiment to verify the performance of algorithm comparing with Apriori algorithm and GA. In GA, the tournament selection, single-point crossover and single-point mutation are used. In order to get better initial population and avoid the algorithm into local optimal solution, half of the initial population is selected from the existed individuals and the other half is generated randomly. Specifically, a initial population of 500 individuals consists of 250 existed individuals and 250 randomly generated individuals. In addition, we have adopted a general operation to the existing population to improve the initial individuals, which means reducing the number of attributes in each individual randomly. For the second part, the BSO algorithm is applied to the classification feature extraction problem with two datasets, the Iris and the Wine datasets, in UCI database. In the flight data feature extraction experiment, three algorithms are evaluated with a flight database which consists of 7984 instances, and each sample has 9 numeric attributes. They are pressure altitude, air temperature, radio altitude, accelerator, engine pressure radio, engine exhaust temperature, high-pressure rotor speed, fuel oil speed and air velocity. For the classification feature extraction problem, we use UCI database to evaluate the performance of BSO algorithm. The characteristics of the Iris dataset and Wine dataset we used are shown in the Table 2. For the Iris data with a balanced number of samples, each class in the dataset has the same number of records. Therefore, the occurrence probability of each classification sample is the same. For Wine data with unbalanced sample size, each classification sample does not have the same occurrence probability in global space. In all the experiments, the best results are denoted in bold. In addition, all of the experiments are carried out on personal computers running Windows 7 with an Intel (R) Core (TM) i5-6500 3.20 GHz and 16.0 GB RAM. The proposed BSO algorithm is coded on the MATLAB R2014a platform.

A Feature Extraction Method Based on BSO Algorithm for Flight Data

169

Algorithm 2: The improved Brain storm optimization (BSO) algorithm Step 1. Data pre-processing: Normalize the data in the database; Step 2. Initialization; 2.1 Randomly generate n initial individuals; 2.2 Optimize the n original individuals by general operation; 2.3 Evaluate the n optimized individuals using fitness function; while not reach the predetermined maximum number of iterations do Step 3. Clustering: Cluster n individuals into m clusters by k-means clustering algorithm; Step 4. Generate new individual: Select one cluster or two clusters randomly to generate a new individual; for i = 1 to n − 1 do 4.1 Produce a random value rgeneration within the range [0, 1); 4.2 if rgeneration < Pgeneration then Select a cluster randomly and produce a random value roneCluster within the range [0, 1); 4.2.1 if roneCluster < PoneCluster then Add random values to the selected cluster center to generate a new individual; 4.2.2 else Select a normal individual randomly from the selected cluster and add random values to it to generate a new individual; 4.3 else Select two clusters randomly and produce a random value rtwoCluster within the range [0, 1); 4.3.1 if rtwoCluster < PtwoCluster then Combine the two selected cluster centers and add random values to the combination to generate a new individual; 4.3.2 else Select two normal individual randomly from the two selected clusters and combine them, and then add random values to the combination to generate a new individual; Step 5. Selection: Pull back the infeasible solution and compare the new individual with the existed individual with the same individual index and keep the better one to be the new individual; Step 6. Elites selection: Evaluate the n individuals using fitness function and select the individuals with fitness value greater than Pelite as elites;

Table 2 Iris and Wine datasets Data set No. of instances Iris Wine

150 178

No. of attributes

No. of categories No. of per category

4 13

3 3

50, 50, 50 59, 71, 48

170

H. Lu et al.

Table 3 The parameters for BSO algorithm n Max_iteration m Pgeneration 500

50

5

0.8

Table 4 The parameters for GA algorithm n Max_iteration Pcrossover Pmutation 500

50

0.8

Table 5 The parameters for fitness function Parameter α1 Value

0.8

0.2

PoneCluster

PtwoCluster

Pelite

0.4

0.5

0.8

Min_support

Min_confidence

0.4

0.5

α2

α3

0.8

0.1

4.1 Parameters Setting For the two parts of experiments, the algorithm parameters are predetermined. Here, Table 3 and Table 4 show the parameters of BSO algorithm and GA for flight database experiment, respectively. In addition, the parameters for the Apriori algorithm and GA both contain Min_support and Min_confidence. Min_support stands for the minimum support and Min_confidence is the minimum confidence. Moreover, Pcrossover in Table 4 represents the crossover probability and mutation probability is Pmutation . The meaning of other parameters are shown in above part of this chapter. Furthermore, the parameters setting for the BSO algorithm in the classification feature extraction experiment is the same as the flight data experiment, which is shown in Table 3. In addition, the parameters setting of the fitness function for the BSO algorithm in the two part experiments are the same, which are shown in Table 5. The three parameters represent the weight of support, confidence and rule length in the fitness function, respectively.

4.2 The Experiment for Flight Data Extraction Problem First, we carried out the feature extraction experiment to verify the feasibility of the improved BSO algorithm using the normalized flight data. The fitness function of BSO algorithm is defined as Eq. (11). fitness = 0.8 × Indi_sup + 0.8 × Indi_sup/Ant_sup − 0.1 × length(Indi)

(11)

In order to measure the performance of the BSO algorithm, we adopt the support, confidence and length of the extracted rules as the metrics.

A Feature Extraction Method Based on BSO Algorithm for Flight Data

4.2.1

171

The Comparison of Different Algorithms

(1) Rule representation of different algorithms In order to illustrate the performance of the algorithm, we sample three rules extracted from flight data using the improved BSO algorithm. They are shown in Table 6. The first rule is explained as follows: Accelerator [11.49, 31.14] fuel oil speed [597.8, 9863.69] ⇒ engine exhaust temperature [240.44, 593.72] That means if the accelerator is in range [11.49, 31.14] and fuel oil speed is in range [597.8, 9863.69], then the engine exhaust temperature is in range of [240.44, 593.72]. In addition, the support of this rule is 85% and its confidence is 100%. In addition, we use the Apriori algorithm to extract the features with the flight data. The rules extracted by Apriori algorithm are shown in Table 7. We know that the data needs to be discretized in advance for Apriori algorithm. For instance, the data of pressure altitude is divided into seven levels from 0 to 30000. And A1 stands for [0, 1000], A2 stands for [1000, 5000] and so on. A7 ⇒ C3, D4, H1 The above rule means that if the pressure altitude is in range [25000, 30000], then the engine exhaust temperature is in range [400, 450], the engine pressure ratio is in

Table 6 The rules extracted by the improved BSO algorithm Rules Support (100%) Confidence (100%) Accelerator [11.49, 31.14] fuel oil speed [597.8, 9863.69] ⇒ engine exhaust temperature [240.44, 593.72] Accelerator [12.18, 25.57] ⇒ fuel oil speed [1374.94, 9445.24] Accelerator [19.84, 28.88] ⇒ engine exhaust temperature [339.8, 557.83]

100

3

76

90

2

85

99

2

Table 7 The rules extracted by Apriori algorithm Rules Support (100%) Confidence (100%) A7 ⇒ C3, D4, H1 A7 ⇒ C3, G3 G3 ⇒ H1

Rule length

85

41.11 42.55 49.66

74.19 76.78 99.15

Rule length 4 3 2

172

H. Lu et al.

Table 8 The statistical values of four indexes for three algorithms 500 * 50 No. of rules Support BSO GA Apriori BSO Mean Var Max Min

Mean Var Max Min

69.5 98.94 83 49 Confidence BSO 0.9108 1.9624e−04 0.9356 0.8884

273 0 273 273

273 0 273 273

GA 0.8224 0 0.8224 0.8224

Apriori 0.8224 0 0.8224 0.8224

0.6863 0.0006 0.7184 0.6431 Rule length BSO 2.4900 0.0156 2.7100 2.3500

GA

Apriori

0.4337 0 0.4337 0.4337

0.4337 0 0.4337 0.4337

GA 3.2600 0 3.2600 3.2600

Apriori 3.2600 0 3.2600 3.2600

[1.20, 1.30], and the air temperature is in range of [0.00, 2.50]. Besides, the support of this rule is 41.11% and its confidence is 74.19%. We can find that the extracted rules using the improved BSO algorithm have higher support and confidence. Besides, the rules consist of different antecedents and consequence, and the values of the attributes are continuous numerical intervals. Compared to the improved BSO algorithm, the rules extracted using the Apriori algorithm are not visual, as the data is discretized in advance. Furthermore, we use GA to search frequent items and extract rules from these frequent items, and the results of it are similar to the Apriori algorithm. (2) The statistical results of different algorithms Considering the randomness of the BSO algorithm, the proposed algorithm will be executed 10 times and the average values will be recorded. The statistical results of the three algorithms are shown in Table 8. The proposed BSO algorithm extracts rules with higher support and confidence. Moreover, the rules extracted by the BSO algorithm are shorter, which means the rules are more practical. In addition, the idea of GA and Apriori algorithm for feature extraction is similar to method of enumeration. As a result, the amount of extracted rule is too large. Meanwhile, considering the quality of the rules attained by the BSO algorithm, the number of rules for it is more reasonable. The boxplot of the above statistical results is shown in Fig. 4. It is apparent that the BSO algorithm has better performance in the support, confidence and rules length comparing to the other two algorithm. Besides, as for the support, the Apriori algorithm and GA generate greater outliers, and the results of BSO algorithm are more concentrated, which means the BSO algorithm is stable. The red points in Fig. 5 stand for the results for the BSO algorithm. Meanwhile, the blue points stands for the results for the Apriori algorithm and GA. It is obvious that the blue points all focus to one point, and their support, confidence and rule

A Feature Extraction Method Based on BSO Algorithm for Flight Data

173

Fig. 4 The boxplot of support, confidence and rule length for three algorithms

Fig. 5 The scatterplot of support, confidence and rule length for three algorithms

length are not as good as the red points, which means the BSO algorithm attains better results than the other two algorithms.

4.2.2

The Experiments Under Different Population Scale

We adjust the population scales and iterations to verify the quality of the Apriori algorithm, GA and the proposed BSO algorithm. The maximum iterations is 50. The number of population is 100, 200, 300 and 500, respectively. The other parameters settings remain unchanged. The results of the experiments are shown in Tables 9, 10, 11 and 12.

174

H. Lu et al.

Table 9 The statistical values of No. of rules with different population scales No. of rules 100 200 BSO GA Apriori BSO GA Mean Var Max Min

Mean Var Max Min

7.8 31.07 18 2 300 BSO 34.3 201.34 66 19

257.3 500.01 273 200

273 0 273 273

GA 273 0 273 273

Apriori 273 0 273 273

17.5 22.289 24 11 500 BSO 69.5 98.94 83 49

273 0 273 273

273 0 273 273

GA 273 0 273 273

Apriori 273 0 273 273

Table 10 The statistical value of support with different population scales Support 100 200 BSO GA Apriori BSO GA Mean Var Max Min

Mean Var Max Min

0.6367 0.0046 0.77 0.5256 300 BSO 0.6452 0.0015 0.7274 0.5986

0.4344 1.4916e−06 0.4361 0.4320

0.4337 0 0.4337 0.4337

GA 0.4337 0 0.4337 0.4337

Apriori 0.4337 0 0.4337 0.4337

0.6411 0.0013 0.7026 0.5762 500 BSO 0.6863 0.0006 0.7184 0.6431

Apriori

Apriori

0.4337 0 0.4337 0.4337

0.4337 0 0.4337 0.4337

GA 0.4337 0 0.4337 0.4337

Apriori 0.4337 0 0.4337 0.4337

Figure 6 is the boxplot of No. of rules, support, confidence and rule length for the proposed BSO algorithm and the other two algorithms. Here, ‘B’, ‘G’ and ‘A’ represent BSO, GA and Apriori algorithm, respectively. ‘1’, ‘2’, ‘3’ and ‘5’ stands for 100, 200, 300 and 500 individuals, respectively. It is obvious that the support, confidence and rule length for BSO algorithm are all better than the results of the other algorithms with various population scales. In addition, the number of rules extracted by the BSO algorithm is not as many as the other algorithms. However, considering the greater quality of the rules extracted by the BSO algorithm, the number of rules is acceptable. Furthermore, as the population scale increases, the number of rules for the BSO algorithm is increasing simultaneously. In addition, the confidence and rule length for the BSO algorithm are also

A Feature Extraction Method Based on BSO Algorithm for Flight Data Table 11 The statistical values of confidence with different population scales Confidence 100 200 BSO GA Apriori BSO GA Mean Var Max Min

Mean Var Max Min

0.9229 0.0027 0.9900 0.8400 300 BSO 0.9206 2.2046e−04 0.9397 0.8879

0.8205 1.0724e−05 0.8260 0.8141

0.8224 0 0.8224 0.8224

GA 0.8224 0 0.8224 0.8224

Apriori 0.8224 0 0.8224 0.8224

0.9157 5.0709e−04 0.9561 0.8833 500 BSO 0.9108 1.9624e−04 0.9356 0.8884

Mean Var Max Min

2.6170 0.1348 3.1700 2 300 BSO 2.7000 0.0403 3.0900 2.4600

3.2620 6.2222e−05 3.2700 3.2500

3.2600 0 3.2600 3.2600

GA 3.2600 0 3.2600 3.2600

Apriori 3.2600 0 3.2600 3.2600

2.7260 0.0691 3.0900 2.3800 500 BSO 2.4900 0.0156 2.7100 2.3500

Apriori

0.8224 0 0.8224 0.8224

0.8224 0 0.8224 0.8224

GA 0.8224 0 0.8224 0.8224

Apriori 0.8224 0 0.8224 0.8224

Table 12 The statistical values of rule length with different population scales Rule length 100 200 BSO GA Apriori BSO GA Mean Var Max Min

175

Apriori

3.2600 0 3.2600 3.2600

3.2600 0 3.2600 3.2600

GA 3.2600 0 3.2600 3.2600

Apriori 3.2600 0 3.2600 3.2600

more concentrated. At the same time, for the population size of 100, 200, 300 and 500, there is little change in the results of Apriori algorithm and GA.

4.2.3

The Experiments Under Different Maximum Iteration

Moreover, the effect of number of iterations on the algorithm performance is also investigated. We set the population to 500. The maximum iterations are set to 10, 20, 30 and 50 to represent different terminal conditions of the feature extraction problem. The other parameters remain the same. The statistical results of the number of rules, support, confidence and rule length for Apriori algorithm, GA and BSO algorithm with different maximum iterations are shown in Tables 13, 14, 15 and 16,

176

H. Lu et al.

Fig. 6 The boxplot of four indexes for BSO, GA and Apriori algorithm Table 13 The statistical values of No. of rules with different maximum iterations No. of rules 10 20 BSO GA Apriori BSO GA Mean Var Max Min

Mean Var Max Min

6.3000 5.3444 12 4 30 BSO 24 70.2222 43 14

0.2000 0.4000 2 0

273 0 273 273

GA 273 0 273 273

Apriori 273 0 273 273

14.7000 10.9000 19 9 50 BSO 69.5 98.94 83 49

Apriori

235.8000 2.2042e+03 273 116

273 0 273 273

GA 273 0 273 273

Apriori 273 0 273 273

respectively. The boxplot of the above results is in Fig. 7. Here, ‘B’, ‘G’ and ‘A’ represent BSO, GA and Apriori algorithm, respectively. ‘1’, ‘2’, ‘3’ and ‘5’ stands for the maximum iterations of 10, 20, 30 and 50, respectively. For the number of rules and support, with the increasing of iterations, the two indicators of the BSO algorithm increase simultaneously. Moreover, the value of

A Feature Extraction Method Based on BSO Algorithm for Flight Data Table 14 The statistical values of support with different maximum iterations Support 10 20 BSO GA Apriori BSO GA Mean Var Max Min

Mean Var Max Min

0.5952 0.0051 0.6825 0.4733 30 BSO 0.6326 0.0015 0.7133 0.5721

0.0502 0.0252 0.5018 0

0.4337 0 0.4337 0.4337

GA 0.4337 0 0.4337 0.4337

Apriori 0.4337 0 0.4337 0.4337

0.6477 0.0012 0.7100 0.5944 50 BSO 0.6863 0.0006 0.7184 0.6431

Mean Var Max Min

0.9244 0.0012 0.9750 0.8743 30 BSO 0.9105 1.9273e−04 0.9286 0.8855

0.0842 0.0708 0.8417 0

0.8224 0 0.8224 0.8224

GA 0.8224 0 0.8224 0.8224

Apriori 0.8224 0 0.8224 0.8224

0.8983 0.0012 0.9445 0.8393 50 BSO 0.9108 1.9624e−04 0.9356 0.8884

Apriori

0.4360 2.0290e−05 0.4476 0.4312

0.4337 0 0.4337 0.4337

GA 0.4337 0 0.4337 0.4337

Apriori 0.4337 0 0.4337 0.4337

Table 15 The statistical values of confidence with different maximum iterations Confidence 10 20 BSO GA Apriori BSO GA Mean Var Max min

177

Apriori

0.8220 5.2127e−05 0.8309 0.8041

0.8224 0 0.8224 0.8224

GA 0.8224 0 0.8224 0.8224

Apriori 0.8224 0 0.8224 0.8224

support, confidence and rule length are more concentrated. In addition, GA algorithm almost lost the ability to extract rules when the maximum iteration is 10. However, the BSO algorithm can still extract features with good performance. It means the BSO algorithm has good search ability even in the early period of evolution. Furthermore, for the maximum iterations of 20, 30 and 50, there is little change in the results of Apriori algorithm and GA. Moreover, the computational time of the Apriori algorithm and GA is longer than that of the BSO algorithm. The reason is that the Apriori algorithm and GA need to generate frequent items in advance, and the BSO algorithm can extract the features directly.

178

H. Lu et al.

Table 16 The statistical values of rule length with different maximum iterations Rule length 10 20 BSO GA Apriori BSO GA Mean Var Max Min

Mean Var Max Min

2.6060 0.0351 3 2.4300 30 BSO 2.6350 0.0138 2.7900 2.4500

0.2000 0.4000 2 0

3.2600 0 3.2600 3.2600

GA 3.2600 0 3.2600 3.2600

Apriori 3.2600 0 3.2600 3.2600

2.5120 0.0484 2.9200 2.2200 50 BSO 2.4900 0.0156 2.7100 2.3500

Apriori

3.2420 4.4000e−04 3.2700 3.2100

3.2600 0 3.2600 3.2600

GA 3.2600 0 3.2600 3.2600

Apriori 3.2600 0 3.2600 3.2600

Fig. 7 The boxplot of four indexes for BSO, GA and Apriori algorithm

A Feature Extraction Method Based on BSO Algorithm for Flight Data

179

4.3 The Experiment for Classification Feature Extraction Problem In order to verify the general performance of BSO algorithm for feature extraction problems. We use the classification feature extraction problem for further research. As a result, we conduct the experiments of proposed BSO algorithm with the dataset in UCI benchmarks. They are the Iris data and Wine data. The evaluation indexes of coverage, precision and recall have been used for the measure of the extracted rules.

4.3.1

Evaluation Index

For feature extraction problem, the confusion matrix is the basic to evaluate the performance of classification algorithm. As shown in Table 17, the rows of the matrix are real categories and the columns are test categories. The class A means the positive class, and the class B means the negative class. TP (True positive) is the condition that class A is classified as class A. In the same way, FN (False Negative) stands for the condition that class A is classified as the negative class, class B. FP (False Positive) and TN (True Negative) are conditions that class B is classified as class A and class B, respectively. Besides, NA is the number of class A which is equal to TP + FN. Similarly, NB = FP + TN, MA = TP + FP, MB = FN + TN and the total number of the database L is TP + FN + FP + TN . Moreover, for problem that consists of more than two classes, we can classify the specified class as a positive class and the remaining classes are considered as a negative class. (1) Coverage For a given instance, if the antecedents in a rule are all satisfied by the instance, the instance is covered by this rule. Coverage to a rule means that the number of the covered instances to the number of the whole test instances. Generally, the process of features extraction is divided into training and test stages. The researchers will use the training database to extract the features and another test database to examine the quality of the features. The training database and the test database are the identical one in this chapter. Moreover, considering the rules may not cover all the instances,

Table 17 Confusion matrix Confusion matrix Test categories Real categories

Class A Class B

Class A

Class B

TP (True Positive) FP (False Positive) MA

FN (False Negative) TN (True Negative) MB

NA NB L

180

H. Lu et al.

we set the Ntest as the whole number of the instances in the database. The coverage function is defined according to Eq. (12). coverage =

TP + FP Ntest

(12)

(2) Recall Recall means the ratio of the set classified correctly to the number of the instances of the specified class in the database. The recall rate of single class rules is defined as follows. TP TP = (13) recall = TP + FN NA (3) Precision Precision is the ratio of the set classified correctly to the total number of individuals identified as the specified class. For the single class rules, the precision is as follows. precision =

4.3.2

TP TP = TP + FP MA

(14)

The Classification Experiments with Iris Data

For the classification feature extraction problem, the rules extracted have a fixed consequence, the category which means that the pattern of the rule is predefined. Therefore, we only need to consider the antecedents when extracting the classification features. Besides, in order to get better initial individuals, the value of the first part of each attribute stands for different meanings in the initiation process of the population. For instance, if the first part is in the interval [0, 0.5], the attribute will be the antecedent of the rule. Meanwhile, if the interval is [0.5, 1] class, the attribute will be out of the rule and we can ignore the following of the attribute. According the parameters setting in Tables 3 and 5. The statistical results of the classification features extraction with Iris data set are shown in Tables 18 and 19. The performance of BSO algorithm is evaluated with several metrics, such as coverage, precision, recall, run time and number of rules. As the Iris data is divided into three classes, we calculate the measure indexes with rules belonging to each class and the total rules. In Table 18, we can find that rules belonging to each category and the total rules all have high recall values. Moreover, the run time of the BSO algorithm is steady, like the performance of the BSO algorithm. In terms of the coverage and precision, the total rules have greater values than the single category rules. As the precision values of rules belonging to each class is related to the coverage, the rules with higher coverage tend to have lower precision. The boxplot of above results is shown in Fig. 8. It is visual that the BSO algorithm performs well in the metric recall, especially for the rules belonging to class B. As

A Feature Extraction Method Based on BSO Algorithm for Flight Data

181

Table 18 The statistical values of coverage, precision and recall with Iris data BSO algorithm

Coverage

Precision

Class A

Class B

Class C

Total

Class A

Class B

Class C

Total

Mean

0.6460

0.7750

0.5950

0.9980

0.5380

0.4410

0.5770

0.9780

Var

0.0475

0.0172

0.0115

1.7778e−05 0.0283

0.0068

0.0192

0.0028

Max

0.9600

0.9700

0.6600

1

0.9200

0.6200

0.9400

1

Min

0.1700

0.5400

0.3300

0.9900

0.3500

0.3400

0.4900

0.8300

Recall Class A

Class B

Class C

Total

Mean

0.9460

1

0.9880

0.9770

Var

0.0268

0

6.4000e−04

0.0028

Max

1

1

1

1

Min

0.4800

1

0.9200

0.8300

Table 19 The statistical values of No. of rules and times with Iris data BSO algorithm (500 individuals × 50 generations) No. of rules

14

Times/s 25.10

12

20

9

13

18

12

18

13

14

24.17

24.07

24.28

24.04

24.24

24.56

24.50

24.47

24.30

Fig. 8 Boxplot of coverage, precision and recall for BSO algorithm using Iris dataset

for the metrics of coverage and precision, although the results of the single class rules are not good enough, the total rules have gotten better values, which means the BSO has better performance in the whole rules. Besides, the coverage and the precision of the rules belonging to class C are more centralized than the rules belonging to class A and class B. Moreover, the single class rules which have higher coverage tends to get lower precision, because the number of instances belonging to each class accounts for a third of the number of the distances in the total dataset.

182

H. Lu et al.

Table 20 The statistical values of three indexes with Wine data, threshold = 0.1 BSO Coverage Precision algorithm Class A Class B Class C Total Class A Class B Class C Mean Var Max Min

Mean Var Max Min

4.3.3

0.2318 0.0165 0.4800 0.0400 Recall Class A 0.4855 0.0515 0.9700 0.1400

0.1882 0.0285 0.4700 0.0100

0.1536 0.0431 0.6700 0.0100

0.4836 0.0421 0.7600 0.1400

Class B 0.2736 0.0492 0.6200 0.0100

Class C 0.2891 0.0721 0.8300 0.0200

Total 0.3482 0.0208 0.5800 0.1200

0.7473 0.0168 1 0.5900

0.7636 0.0505 1 0.4300

0.7682 0.0503 1 0.3400

Total 0.7409 0.0134 0.9000 0.4400

The Classification Experiments with Wine Data

For the Wine dataset, we know that the number of the attributes is larger than the Iris dataset, which means the data has higher dimension. As a result, the initial population in this experiment are not good enough even with the initial solution optimization operation. In order to solve the poor initial solutions, we decrease the fitness value threshold to extract more rules. The threshold is adjusted among 0.1, 0.01 and 0.001. In addition, the BSO algorithm are executed 10 times and the average values are recorded. From the Tables 20, 21 and 22, the results of Wine is not good enough as the results of Iris data, which is due to the higher dimensions and unbalanced category distribution of the Wine data. The both reasons led to less-than-ideal results of the wine data. In the Figs. 9, 10 and 11, we find that the precision and recall value of the total rules are more centralized than the single class rules. As for the metric coverage, the total rules also have higher values than the single category rules. Besides, the precision have better values than the other two metrics. In order to compare the performance of the three thresholds, we show the average values of the coverage, the precision and the recall with different fitness thresholds in Fig. 12. The horizontal coordinate stands for class A, class B, class C and total class of the rules extracted, respectively. Meanwhile, the vertical coordinate represents the average values of the three metrics. We can find that the smaller the threshold, the higher the coverage, which means a smaller threshold could help us to extract more universal rules. Furthermore, the total rules have better performance in the coverage than the single category rules as the database is formed by three categories. As for the average precision, each class rules and the total rules have similar values, and the

A Feature Extraction Method Based on BSO Algorithm for Flight Data

183

Table 21 The statistical values of three indexes with Wine data, threshold = 0.01 BSO Coverage Precision algorithm Class A Class B Class C Total Class A Class B Class C Mean Var Max Min

Mean Var Max Min

0.2773 0.0263 0.6200 0.0900 Recall Class A 0.5300 0.0500 0.9700 0.2200

0.2055 0.0300 0.4700 0.0100

0.1882 0.0492 0.7300 0.0100

0.5191 0.0255 0.7900 0.1400

Class B 0.3082 0.0550 0.6500 0.0100

Class C 0.2800 0.0278 0.5800 0.0200

Total 0.3627 0.0136 0.6200 0.1200

0.6982 0.0162 0.9200 0.4300

0.7618 0.0479 1 0.4300

0.7327 0.0525 1 0.3300

Table 22 The statistical values of three indexes with Wine data, threshold = 0.001 BSO Coverage Precision algorithm Class A Class B Class C Total Class A Class B Class C Mean Var Max Min

Mean Var Max Min

0.3864 0.0751 0.8700 0.0100 Recall Class A 0.5200 0.0764 0.9800 0.0300

0.3255 0.0683 0.7100 0.0100

0.1973 0.0719 0.9200 0.0100

0.6555 0.0623 0.9900 0.1300

Class B 0.3900 0.0791 0.8200 0.0300

Class C 0.2764 0.0612 0.7700 0.0200

Total 0.4027 0.0244 0.6900 0.1200

0.6136 0.0607 1 0.2300

0.6564 0.0608 1 0.3000

0.6127 0.0554 1 0.2300

Total 0.7173 0.0136 0.9000 0.4900

Total 0.6436 0.0189 0.8800 0.3900

algorithm with the larger threshold of 0.1 has better average values. Besides, there is no obvious correlation between the average recall and the fitness threshold values. The statistical results of the number of rules and run time are shown in Table 23, and the boxplot is Fig. 13. It is apparent that the greater is the threshold, the more concentrated are the rules. Furthermore, there is no distinct connection between the threshold and run time, and the threshold of 0.1 has the best performance for attaining better solutions with shorter time.

184

H. Lu et al.

Fig. 9 The boxplot of three indexes for BSO algorithm with Wine dataset, threshold = 0.1

Fig. 10 The boxplot of three indexes for BSO algorithm with Wine dataset, threshold = 0.01

Fig. 11 The boxplot of three indexes for BSO algorithm with Wine dataset, threshold = 0.001

A Feature Extraction Method Based on BSO Algorithm for Flight Data

185

Fig. 12 The bar of coverage, precision and recall with different thresholds Table 23 The statistical values of No. of rules and Times with different thresholds BSO 0.1 0.01 0.001 algorithm No. of rules Times/s No. of rules Times/s No. of rules Times/s Mean Var Max Min

7.9000 0.8900 10 5

74.5827 1.0967 77.4200 73.4700

8.1000 3.6900 11 5

76.4136 0.6357 78.3400 75.4000

8.2000 5.3600 13 5

Fig. 13 The boxplot of No. of rules and run time with different thresholds

74.7009 0.2264 75.5000 73.9500

186

H. Lu et al.

4.4 Summary of the Experiments The experiment results show that the extraction process using the proposed BSO algorithm does not relay on the Min_support, Min_confidence and rule template which are hard to be predetermined. In addition, with the direct extraction process, the data does not need to be discretized in advance. Hence, we can attain quantitative rules which are understandable. Beyond that, there is no need to generate frequent items like Apriori algorithm and GA in our proposed algorithm. Therefore, the time complexity and space complexity will be decreased simultaneously. Besides, the Apriori algorithm and GA have attained many rules comparing to the proposed algorithm. Considering that not all the rules are meaningful to us, the rules generated by BSO algorithm with better performance in support, confidence and rule length are more effective and reliable. Furthermore, the BSO algorithm also show its practicability and universality in the classification problem, especially for the Iris data. In addition, although the extracted rules with Wine dataset are not good enough, the results still demonstrate the usability of the proposed algorithm in dealing with high-dimensional dataset with unbalanced distribution. In summary, our supposed BSO algorithm not only has better performance in features extracted comparing with the Apriori algorithm and GA, but also has better performance in time complexity and space complexity. In addition, the extracted rules are also more effective and intelligible.

5 Conclusion The flight parameter feature extraction is a significant problem in the aerospace field. In addition, the feature extraction is also a promising direction in data mining domain. In this chapter, a proposed algorithm based on BSO algorithm has been applied to the flight data feature extraction problem. The performance of the proposed algorithm is better than that of Apriori algorithm and GA. In addition, the proposed algorithm has also been adopted to the classification problem, and experimental results also verified the practicability and universality of the proposed algorithm. Our future work focuses on applying the proposed algorithm to solve other problems. The proposed BSO algorithm not only has the inherent performance of the original BSO, but also has the some distinct features, such as encoding strategy and handling mechanism for infeasible solution. With these characteristics, it can solve the feature extraction problem effectively. However, considering the valid constraints and specific features of different area, there is still some challenges for BSO algorithm.

A Feature Extraction Method Based on BSO Algorithm for Flight Data

187

Acknowledgements This research was supported by the National Natural Science Foundation of China under Grant No. 61671041, 61101153, 61806119, 61773119, 61703256, and 61771297; in part by the Shenzhen Science and Technology Innovation Committee under grant number ZDSYS201703031748284; and in part by the Fundamental Research Funds for the Central Universities under Grant GK201703062.

References 1. Alikhademi, F., Zainudin, S.: Generating of derivative membership functions for fuzzy association rule mining by particle swarm optimization. In: 2014 International Conference on Computational Science and Technology (ICCST), pp. 1–6 (2014) 2. Cai, C., Zhang, C., Liu, G.: A novel fault diagnosis approach combining SVM with association rule mining for ship diesel engine. In: 2016 IEEE International Conference on Information and Automation (ICIA), pp. 130–135 (2016) 3. Chadokar, S.K., Singh, D., Singh, A.: Optimizing network traffic by generating association rules using hybrid apriori-genetic algorithm. In: 2013 Tenth International Conference on Wireless and Optical Communications Networks (WOCN), pp. 1–5 (2013) 4. Cheng, S., Qin, Q., Chen, J., Shi, Y.: Brain storm optimization algorithm: a review. Artif. Intell. Rev. 46(4), 445–458 (2016) 5. Cheng, S., Sun, Y., Chen, J., Qin, Q., Chu, X., Lei, X., Shi, Y.: A comprehensive survey of brain storm optimization algorithms. In: Proceedings of 2017 IEEE Congress on Evolutionary Computation (CEC 2017), pp. 1637–1644. IEEE, Donostia, San Sebastián, Spain (2017) 6. Dhanore, G., Chaturvedi, S.K.: An optimization of association rule mining for large database using k-map and genetic algorithm: a review. In: 2014 International Conference on Electronics and Communication Systems (ICECS), pp. 1–5 (2014) 7. Dongre, J., Prajapati, G.L., Tokekar, S.V.: The role of apriori algorithm for finding the association rules in data mining. In: 2014 International Conference on Issues and Challenges in Intelligent Computing Techniques (ICICT), pp. 657–660 (2014) 8. Gao, Z., Peng, Z., Gao, N., Chen, B.: A distribution network fault data analysis method based on association rule mining. In: 2014 IEEE PES Asia-Pacific Power and Energy Engineering Conference (APPEEC), pp. 1–5 (2014) 9. Guo, X., Wu, Y., Xie, L., Cheng, S., Xin, J.: An adaptive brain storm optimization algorithm for multiobjective optimization problems. In: Tan, Y., Shi, Y., Buarque, F., Gelbukh, A., Das, S., Engelbrecht, A. (eds.) Advances in Swarm and Computational Intelligence (International Conference on Swarm Intelligence, ICSI 2015). Lecture Notes in Computer Science, vol. 9140, pp. 365–372. Springer International Publishing (2015) 10. Harikumar, S., Dilipkumar, D.U.: Apriori algorithm for association rule mining in high dimensional data. In: 2016 International Conference on Data Science and Engineering (ICDSE), pp. 1–6 (2016) 11. He, B., Pei, J., Zhang, H.: The mining algorithm of frequent itemsets based on mapreduce and FP-tree. In: 2017 International Conference on Computer Network, Electronic and Automation (ICCNEA), pp. 108–111 (2017) 12. Indira, K., Kanmani, S.: Association rule mining through adaptive parameter control in particle swarm optimization. Comput. Stat. 30(1), 251–277 (2015) 13. Jain, J.K., Tiwari, N., Ramaiya, M.: Mining positive and negative association rules from frequent and infrequent pattern using improved genetic algorithm. In: 2013 5th International Conference and Computational Intelligence and Communication Networks, pp. 516–521 (2013) 14. Kareem, S., Ahmad, R.B., Sarlan, A.B.: Framework for the identification of fraudulent health insurance claims using association rule mining. In: 2017 IEEE Conference on Big Data and Analytics (ICBDA), pp. 99–104 (2017)

188

H. Lu et al.

15. Liu, Z., Jia, Z., Vong, C.M., Han, J., Yan, C., Pecht, M.: A patent analysis of prognostics and health management (PHM) innovations for electrical systems. IEEE Access 6, 18088–18107 (2018) 16. Nandhini, M., Janani, M., Sivanandham, S.N.: Association rule mining using swarm intelligence and domain ontology. In: 2012 International Conference on Recent Trends in Information Technology, pp. 537–541 (2012) 17. Rameshkumar, K., Sambath, M., Ravi, S.: Relevant association rule mining from medical dataset using new irrelevant rule elimination technique. In: 2013 International Conference on Information Communication and Embedded Systems (ICICES), pp. 300–304 (2013) 18. Seki, H., Nagao, M.: An efficient java implementation of a GA-based miner for relational association rules with numerical attributes. In: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 2028–2033 (2017) 19. Shang, Q., Wu, P.: Association rules mining based on improved PSO algorithm. In: 2017 2nd IEEE International Conference on Computational Intelligence and Applications (ICCIA), pp. 145–149 (2017) 20. Shi, Y.: Brain storm optimization algorithm. In: Tan, Y., Shi, Y., Chai, Y., Wang, G. (eds.) Advances in Swarm Intelligence. Lecture Notes in Computer Science, vol. 6728, pp. 303–309. Springer, Berlin, Heidelberg (2011) 21. Shi, Y.: An optimization algorithm based on brainstorming process. Int. J. Swarm Intell. Res. (IJSIR) 2(4), 35–62 (2011) 22. Solanki, S.K., Patel, J.T.: A survey on association rule mining. In: 2015 Fifth International Conference on Advanced Computing Communication Technologies (ACCT), pp. 212–216 (2015) 23. Sonet, K.M.H., Rahman, M.M., Mazumder, P., Reza, A., Rahman, R.M.: Analyzing patterns of numerously occurring heart diseases using association rule mining. In: 2017 Twelfth International Conference on Digital Information Management (ICDIM), pp. 38–45 (2017) 24. Song, S., Ma, Y.: The research and application of technology in the diagnosis of lung cancer warning association rule mining. In: 2016 8th International Conference on Information Technology in Medicine and Education (ITME), pp. 343–346 (2016) 25. Tewari, A.S., Barman, A.G.: Collaborative book recommendation system using trust based social network and association rule mining. In: 2016 2nd International Conference on Contemporary Computing and Informatics (IC3I), pp. 85–88 (2016) 26. Verma, A., Taneja, A., Arora, A.: Fraud detection and frequent pattern matching in insurance claims using data mining techniques. In: 2017 Tenth International Conference on Contemporary Computing (IC3), pp. 1–7 (2017) 27. Wang, J., Yang, F., Dong, X., Xu, B., Cui, B.: Improving association rules by optimizing discretization based on a hybrid GA: a case study of data from forest ecology stations in china. In: 2013 Fourth International Conference on Emerging Intelligent Data and Web Technologies, pp. 627–632 (2013) 28. Xi, Y., Yuan, Q.: Intelligent recommendation scheme of scenic spots based on association rule mining algorithm. In: 2017 International Conference on Robots Intelligent System (ICRIS), pp. 186–189 (2017) 29. Yadav, C., Wang, S., Kumar, M.: An approach to improve apriori algorithm based on association rule mining. In: 2013 Fourth International Conference on Computing, Communications and Networking Technologies (ICCCNT), pp. 1–9 (2013) 30. Zhang, T., Lu, J., Zhang, G., Ding, Q.: Fault diagnosis of transformer using association rule mining and knowledge base. In: 2010 10th International Conference on Intelligent Systems Design and Applications, pp. 737–742 ( 2010) 31. Zhang, X., Duan, F., Zhang, L., Cheng, F., Jin, Y., Tang, K.: Pattern recommendation in taskoriented applications: a multi-objective perspective [application notes]. IEEE Comput. Intell. Mag. 12(3), 43–53 (2017)

Brain Storm Optimization Algorithms for Solving Equations Systems Liviu Mafteiu-Scai, Emanuela Mafteiu and Roxana Mafteiu-Scai

Abstract This chapter refers to the use of Brain Storm Optimization (BSO) algorithms in solving equations systems (ES). BSO algorithm is a swarm intelligence algorithm, which simulates the human brainstorming process, a form of human collective creativity. Mainly, in this chapter, two algorithms are proposed: the first for ES preconditioning and second for solving ES. First, is proposed a BSO method aiming the bandwidth reduction of sparse matrices, a process that can improve a lot of computing processes, such as solving large systems of linear equations. The other one proposes a method for solving equations systems that uses BSO. For the second problem, a new crossover strategy as well as a hybridization of BSO with graph theory elements are proposed. Serial and parallel variants of both algorithms are presented. Experimental results obtained illustrate the fact that the proposed algorithms lead to good results, with respect to other methods. Keywords Bandwidth · Brain storm optimization · Equations systems · Parallel computing · Sparse matrices

1 Theoretical Considerations In this section, the theoretical elements used in proposed methods are briefly presented. A new crossover operator, elements of graph theory used and the BSO parallelizations are proposed and described here.

L. Mafteiu-Scai (B) · E. Mafteiu · R. Mafteiu-Scai Computer Science Department, West University of Timisoara, Timisoara, Romania e-mail: [email protected] E. Mafteiu e-mail: [email protected] R. Mafteiu-Scai e-mail: [email protected] © Springer Nature Switzerland AG 2019 S. Cheng and Y. Shi (eds.), Brain Storm Optimization Algorithms, Adaptation, Learning, and Optimization 23, https://doi.org/10.1007/978-3-030-15070-9_8

189

190

L. Mafteiu-Scai et al.

1.1 Solving Equations Systems A frequent problem in numerical analysis is the equations systems (ES) solving. This problem has generated in time a great interest among mathematicians and computer scientists, as evidenced by the large number of numerical methods developed. Besides the classical numerical methods, methods inspired by techniques from artificial intelligence (AI) have been proposed in recent years. Hybrid methods have been also proposed along the time. The classical methods are usually divided into exact/direct methods and iterative/indirect methods. In exact methods, the solution is obtained after a fixed number of operations, a number that is directly proportional to the size of the system. In these cases, the solutions are affected by rounding errors and as a consequence these methods are used only when the size of the systems is small. The most known direct methods are: Cramer, Gaussian elimination, Gauss-Jordan elimination, LU factorization, QR decomposition etc. In iterative methods the iterative process will be stopped after a preset number of steps or after fulfilling certain conditions. The process involves truncation errors, the main disadvantage of these methods. However, iterative methods are preferred especially for large systems of equations because the rounding errors are not cumulated and a smaller number of operations is required. A disadvantage of iterative methods occurs when the system size is smaller than the number of unknowns (under-determined system) when these methods are not applicable in general. The most popular iterative methods are Gauss-Seidel, Jacobi, Newton, Conjugate Gradient and Broyden. One of the most used is Conjugate gradient (CG) method that is recommended for large and sparse linear systems. It has the property of uniform convergence when the associated matrix of the equations system is symmetric and positive definite. More, CG method is good for parallel implementation, because its parallelism derives from parallel matrix-vector product. With all the advantages offered, the classical methods have specific restrictions (only for systems with positive defined matrices), truncation and rounding errors for large systems, or weak convergence if other particular conditions are not met, etc. The Artificial Intelligence (AI) and its results in recent decades have led to a new stage in solving equations systems. Different from traditional mathematical approach, this stage is not easily accepted by mathematicians, mainly due to the fact that there are no mathematical demonstrations on the convergence of these methods. There are a lot of AI approaches used to solve equations systems, such as Monte Carlo (MC) methods [1], Artificial Neural Networks (ANN) [2], Genetic Algorithms (GA) [3], Particle Swarm Optimization algorithms (PSO) [4], Ant Colony Optimization (ACO) [5], Memetic Algorithms (MA) [6], Cuckoo Search algorithms (CS) [7], Greedy Randomized Adaptive Search Procedure (GRASP) [8] and many more. We mention that in most cases solving equations systems are viewed as optimization tasks. In last years, a lot of hybrid methods are proposed for solving equations systems. These methods combine metaheuristics with performant iterative methods of systems solving, the results being methods with good spatial and temporal complexity and accurate solutions. A hybridization between a genetic algorithm and

Brain Storm Optimization Algorithms for Solving …

191

Fig. 1 History of solving ES using techniques from AI

an augmented Lagrangian function to solve equations systems is proposed in [9], where the original constrained problem is replaced by a sequence of unconstrained subproblems. In paper [10] a hybrid method for solving ill-conditioned non-linear equations is proposed, i.e. equations system for which the associated matrix has condition number greater than 1. In this method, there is a hybridization between an iterative method (CG conjugate gradient) and Flower Pollination Algorithm (FPA), a combination between a faster convergence and a faster global search. The main steps of the proposed algorithm are: convert the equations system into a fitness function, use the FPA part to obtain a good initial guess for CG and finally use the CG part to solve the system. The last two steps are repeated until an accurate solution is determined. Figure 1 shows researchers’ concern in solving equation systems using artificial intelligence techniques, reflected in the number of published scientific papers. An increase in this interest and a dominance of metaheuristic methods can be seen especially in the last decade.

1.2 Preconditioning Equations Systems and Minimum Bandwidth Problem (MBP) A proper reorganization of the associated matrix of an equations system can have a significant influence on the efficiency of systems equations solving, especially in parallel implementations of iterative methods [11–13]. Reducing the bandwidth (bw) or the Minimum Bandwidth Problem (MBP) of sparse matrices is one of a pre-processing technique in solving large systems of linear equations. Even in case of Gaussian elimination method, reducing the bandwidth can reduce the time complexity method.

192

L. Mafteiu-Scai et al.

Other real life applications of bandwidth reduction algorithms are found in chemistry [14], tomographic problems [15], simulation and design of circuits [16–19], network structure problems [20], symbolic model checking [21] etc. Related to bandwidth optimization problem, the Cuthill-Mckee algorithm [22] represents the starting point of this subject. From the matrix point of view, the bandwidth measure is given by relation: bw(A) = max|i − j|; aij = 0, i = 1, n, j = 1, n

(1)

where aij are elements of a square matrix A of size n. From the graph theory perspective, the value of the bandwidth is given by relation:   bw(G) = minσ maxu,v∈E (|σ (u) − σ (v)|)

(2)

where σ is a bijection between E and {1, 2, … |E|} in a undirected graph G(V, E). In time, many methods have been proposed for MBP. Mainly, there are two types of algorithms for this problem: exact and heuristic. Exact algorithms guarantee always to discover the optimal even if it takes a long time for this and there are useful only for small matrices. Mainly, exact methods proposed along the time are based on analysis of all n! possible permutations of lines/columns of n × n matrices, or, on branch and bound methods. The heuristic methods can discover good solutions in a short time, but do not guarantee the optimality, i.e. even these methods have a running time which is polynomial in the matrix size, they do not guarantee always to find the optimum. Most of the proposed heuristics are based on breadth-first search method from graph theory, node-centroid methodology, hill-climbing technique, spectral algorithms, direct matrix processing or combine algorithmic ideas from advanced math optimization method with techniques from artificial intelligence (genetic algorithms, genetic programming, ant colony optimization, particle swarm optimization and many more. Papers [23, 24] cover the state of the art in bandwidth reduction algorithms, including exact and heuristics methods. In work [25], a complex analysis of the parallel methods for this issue is made.

1.3 Creativity and Brainstorming Creativity is one of the most remarkable attributes of the human being, which essentially distinguishes it from other creatures on our planet (so we like to believe). A definition of the concept can be found in the paper [26], and there are many works that describe and analyze human creativity in terms of psychology [27, 28]. In the computer era, it was a natural occurrence of a study domain aiming imitating human creativity called computational creativity [28, 29]. Computational creativity is a subfield of artificial intelligence which studies how to build computational models of creative thought in science and arts. Computational creativity has been applied in

Brain Storm Optimization Algorithms for Solving …

193

many directions as well: design [30], games [31], culinary recipes [32], music [33] etc. At the same time, human creativity has always played an essential role in solving new problems or in finding new and more effective solutions to solve old ones. There are two forms of creativity in terms of the number of persons involved: singular and collective creativity. One collective creativity technique is the human brainstorming [34], which mainly aims to the generation of a large number of creative ideas through intensive and freewheeling group discussion aiming to solve a problem. The term brainstorming was introduced in 1963 by Alex Faickney Osborn [35]. There are four basic rules of brainstorming method: – – – –

focus on quantity; without criticism; welcome unusual ideas; combining and improving the ideas.

The human brainstorming technique has generated much controversy in time, being many researchers who criticize it [36] and even some who entirely reject it. Anyway, over time, the technique proposed by Osborn has undergone improvements. One of the greatest innovations of the verbal brainstorming proposed by Osborne was the electronic brainstorming (EBS) [37].

1.4 Modelling the Human Brainstorming in Our Proposed Optimization Problems There are some key points to design an optimization method based on human brainstorming model: (a) The need for experts After each brainstorming process phase it is expected to result a better idea than ideas proposed by each participant in part in the current phase. In our opinion if all participants are unskilled in the problem domain to be solved, it is unlikely to reach a good result. At the same time, the presence of experts would allow to remove those ideas that are not suitable. In our proposed methods on this chapter, the expert role is done by the fitness function of the evolutionary process. The chosen fitness function tells the algorithm how good a particular solution is. For example, in case of solving an equations system, the fitness value is given by the difference between left and right side of each equation in part after replacing the candidate solution vector in all equations. If this difference is equal to zero we have an exact solution. If this difference is smaller than a preset value, we say that we have an approximate solution, which could be good/acceptable on some situations.

194

L. Mafteiu-Scai et al.

(b) The diversity of participants In human brainstorming, it is necessary to have a great diversity of individuals from which they arise, because the same origin can lead to a premature convergence of the ideas issued. The ideas’ diversity in our methods is given by individuals’ diversity, where each individual—a solution vector—represents a possible solution for equations system. This diversity is ensured in our algorithms by a controlled random number generator and by evolutionary operators (crossover and mutations). (c) Starting point The starting point in our proposed algorithms, i.e. the initial population: in case of solving equations systems, there are generated random real values in vectors and each vector is assigned to an individual. A limited search space is used in first stage but in next stages the search space is expanded, the expanding having two purposes: to avoid the process of population degeneration and to try to find new solutions in a larger search space. More exactly, at first, the algorithm start with a given range that represents the initial search space. In next iterations, the initial search space is expanded to its right and left borders with a preset value r. (d) Ideas combination It is known from human brainstorming that combining similar or redundant ideas can lead to improved solutions. In proposed algorithms, an iteration represents a phase in the brainstorming process. Except for the first iteration, in the rest of all, only a part of the population will be generated randomly, the other part being obtained by crossover and mutation operators applied on the best individuals of the previous population, respecting the principle of genetic elitism. (e) Ideas classification The clustering of ideas is necessary to determine which should take priority in next brainstorming iteration. Thus, in proposed algorithms, the candidate solutions are clustering according to their associated fitness values. (f) Validation of ideas Brainstorming can be viewed as a method for generating ideas without worrying about solving or not solving the problem. After ideas generation, their implementation and validation as solutions for problem to be solved can begin. So, at the beginning, ideas have to be random generated and after that they must be validated to see if these can solve the problem. As mentioned in [38], generating new ideas in a software program is not difficult with artificial intelligence techniques, but the hard problem is their automating validation. In the proposed methods on this chapter related to solving equations systems, this validation is done by replacing the candidate solution vector in equations system and the difference between left and right side of each equation in part shows how good a solution is.

Brain Storm Optimization Algorithms for Solving …

195

(g) Creativity like a search space In work [38] the creative process is seen as a search process in a given space, space which can be extended at each iteration. On each iteration in search space is a set of objects and relationships between some of these objects. Linking a group of objects through relationships will be a new idea. The validations of these new ideas represent new candidate solutions for a given problem. At the same time, there are objects which can be linked by relationships and objects that cannot linked. The last are called “bad ideas” and these must be eliminated. In our approach, modeling the creativity like a search space is made by using the k-partite graph concept (see Sect. 1.6). (h) Process termination A human brainstorming process can be closed when a solution for the problem was found or a preset number of stages was reached. In our proposed methods, the termination criteria are: finding an exact solution (fitness function satisfying), finding an approximate solution, or reaching a preset number of iterations.

1.5 Brain Storm Optimization Algorithm In last years, a lot of population-based optimization algorithms have been applied to solve optimization problems. One of these is the Brain Storm Optimization (BSO) algorithm, inspired by the human brainstorming process and proposed by Yuhui Shi in 2011 [39]. Even if the method can be implemented in various ways by setting up algorithm’s parameters differently, the main steps of a BSO algorithm are [39]: 1. Randomly generate initial population (individuals that could be potential solutions); 2. Cluster individuals into clusters and put the best individuals as cluster center in each cluster; 3. Replace some cluster centers with randomly generated individuals; 4. Depending on a pre-determined probability, replace the cluster center with a selected individual from this cluster or generate new individuals. 5. Depending on a pre-determined probability, from the two cluster centers or from other two individuals of each selected cluster, generate new individuals; 6. The process is stopped after a predefined number of iterations. Mainly, there are two basic operations in BSO algorithms: clustering and adding some noise i.e. idea generating. A lot of subsequent works with many variants of BSO algorithm have demonstrated the effectiveness and robustness of this swarm intelligence optimization method. In paper [39], where the BSO was introduced for the first time, the author tests the proposed algorithm for two known functions: Sphere and Rastrigin. The preliminary results obtained validate the effectiveness and usefulness of the algorithm. A multi-objective optimization algorithm based on the brainstorming process

196

L. Mafteiu-Scai et al.

that used Gaussian mutation and Cauchy mutation is proposed in paper [40]. The replacement of the classical clustering method with a simple grouping method aiming to reduce the algorithm computational burden is proposed in paper [41]. A BSO algorithm for a real problem—an optimization problem modeled for a DC brushless motor, was proposed in [42], where the algorithm is highlighted by the possibility to jump out of local optima. In paper [43], a BSO algorithm is proposed to solve an optimization problem modeled for Loney’s solenoid problem. Another real problem solving with a BSO algorithms is proposed in paper [44], for the optimal formation reconfiguration of multiple satellites using closed-loop strategies. A modified BSO algorithm is proposed in [45], aiming to solve an optimization problem based on the nonlinear receding horizon control mode of unmanned aerial vehicles. A method of optimizing the control parameters in the automatic carrier landing system of F/A18A based on a simplified BSO algorithm is proposed in [46]. A BSO approach proposed in paper [47], is applied to optimize the dynamic deployments of two different wireless sensor networks. In paper [48], a metaheuristic inspired from human brainstorming for solving system of equations is proposed. In [49] a hybridization between BSO and simulated annealing (SA) algorithm for continuous optimization problems are described. The role of SA in proposed hybridization is to help BSO to jump out from local optimum. More, SA method can validate the solution obtained by BSO. BSO is applied to solve the optimal scheduling model of active distribution system for clean energy resources [50]. A technical approach for GAN transistor production in 5G of BSO algorithm is presented in [51]. In paper [52] an improving of brain storm optimization algorithm using Simplex Search method is proposed. For more information about BSO algorithms, papers [53, 54] present the current state-of-the-art.

1.6 Using Graph Theory in Modeling the Creativity In this subsection, elements from graph theory used for modeling the creativity like a search space are proposed. The basic element is k-partite graph concept. It is known that a k-partite graph is a graph whose vertices can be partitioned into k disjoint sets such that no two vertices within the same set are adjacent. If k = 2 we have a bi-partite graph and so on. The terms and definitions from bi-partite graphs can be easily extended to general case. In case of a graph G(S, A), where S is the set of the vertices and A is the set of the edges, we have the following definitions: – biclique C = (S , A ) is a subgraph of G induced by a pair of two disjoint subsets S ⊆ S, A ⊆ A, such that ∀ s ∈ S , a ∈ A , (s, a) ∈ E, meaning that a biclique in a bipartite graph is a complete bipartite subgraph that contains all permissible edges.

Brain Storm Optimization Algorithms for Solving …

197

– maximum biclique is a largest biclique in a bipartite graph. From this point of view there are two distinct variants of this problem: the vertex maximum biclique problem and the edge maximum biclique problem. – a biclique within another bipartite graph is called a maximal biclique if it is not contained in a larger biclique. The problem can be regarded distinctly in terms of the vertices respectively edges of graphs. The edge maximum biclique is often used in biological applications, web community discovery, and text mining because it models more balanced connectivity between the two vertex classes [55] and therefore over time many algorithms have been developed in this respect [56]. – a quasi-biclique is similar to biclique but contains almost of its edges. Maximal quasi-bicliques are proposed in [57] and are motivated by real-world applications where errors and missing data/information (edges in graph) are present. Also it was shown the versatility and effectiveness of maximal quasi-bicliques to discover correlated data sets. Regarding to quasi-biclique, in our opinion it is sufficient to consider the particular case in which the cliques relate to a particular partition. This approach was used on our BSO algorithm proposed for solving equations systems (see Sect. 3.1).

1.7 A New Crossover Strategy In evolutionary algorithms, the crossover explores the search space around the already found good or approximate solutions, while the mutation explores the search space for new solutions. There are a many crossover operators, each of them with its advantages and disadvantages: single-point, double-point, three-point, arithmetic, heuristic, intermediate, etc. It is known that if multi-point crossover operators are used, the problem space may be searched more thoroughly. In this subsection, a new crossover strategy, hereinafter referred to as circular crossover (CC) is proposed. This was tested on our algorithms for solving equations systems and preliminary results recommend it (see Sect. 3.1). A description of the proposed crossover operator is presented below. The main idea is that more than two individuals can pe be parents for a new individual, i.e. a child can have more than two parents: two biological and any persons who stand in loco parentis to the child. This is usual in human social life, where other persons or institutions (like primary and secondary education) could be loco parentis. Each of them have their own influence on child evolution and in our approach, equal influences from all parents are considered. We consider that at some point we have a population with n individuals. At first step, for each individual, the fitness value is computing. In second step, the individuals are sorted depending on their fitness values and after, they are arranged imaginary on the circumference of a circle, like in example from Fig. 2. In case of k parents for each child, the distance between two parents, direct neighbors, is n/k, as can be seen in Fig. 2. This distance could have any other values, depending on the problem to be solved. In our study, the

198

L. Mafteiu-Scai et al.

Fig. 2 Circular crossover strategy—an example

Fig. 3 Example of circular crossover process

parents’ contributions to a new individual are equal and fixed from position on circle point of view, as it can be seen in Fig. 3. These criteria could be changed depending on problem to be solved. For each group of k parent, a multi-points crossover, more exactly (k – 1)-points crossover was used. A particular case is presented for an easy understanding, i.e. the case on which one child has three parents. Each individual is recombined with the other two at a distance of n/3 on his right and left. In case of n/3 distance, as it can be seen in Fig. 2, where nine individuals are represented, the circular strategy mean that individuals 1, 4 and 7 will recombine to form six new individual. Same for groups 2-5-8 and 3-6-9. For each group in part, a multi-points crossover operator is used. Each old individual has its contribution on new individual, equal to one third. Depending on which part are used, for each group of three existing individuals, there are six recombination possibilities, i.e. six new individuals, as it can be seen in Fig. 3.

1.8 Parallelization Schemes Proposed for the BSO Algorithms Parallel computing is an efficient type of computation which uses parallel computers, in which large problems can be divided into smaller ones, which can then be solved at the same time on different processors. Mention that sometimes the parallelization

Brain Storm Optimization Algorithms for Solving …

199

of a process can lead to new algorithms, often very different from their serial variant [58], especially for heuristics [59]. For parallel performance measurement, one of the most used indicators in parallel computation is SpeedUp (S p ) with a particular case Relative Speedup. This indicator, a metric used to compare different parallel algorithms, shows how many times a parallel algorithm is faster than the serial algorithm i.e. it is the ratio between execution time achieved by a sequential algorithm (T 1 ) and execution time obtained with a parallel algorithm with p processors (T p ). Sp =

T1 Tp

(3)

Mainly, we have two modes of parallelization: synchronous and asynchronous, each with advantages and disadvantages. In last years, a lot of optimization algorithms were parallelized. In most cases, optimizations based on evolutionary algorithms are synchronous in nature, i.e. at the end of each iteration a synchronization point is required before continuing to the next iteration. The evolutionary algorithms can be implemented on parallel in various ways: parallelization of the evolutionary process itself or to parallelize only some specific operations. In first case, all population function evaluations must be done before start a new iteration with a new population. Because in these types of algorithms very large population is needed and the available processors in parallel computers are not large enough to load balancing the processors, the synchronous parallel algorithms are not very efficient due to the idle time, a factor that will significantly degrade the performance of the synchronous parallel algorithms. The problem of load balancing can be solved if the evolutionary algorithm can dynamically adjusts the workload assigned to each processor, as a result making more efficient the parallel process. From another point of view, an asynchronous approach is a solution to the load balancing of processors. In such a case, the process does not need synchronization points between two iterations, a processor that finished its work can proceed to the next iteration without waiting for the other processor. This approach is a little improper for a BSO algorithm, because a synchronization point is required in most cases. Further, a synchronous and an asynchronous approaches are proposed to minimize the idle times of processors and the communication costs in case of BSO parallelization. Let consider the basic BSO schema proposed in [39]: 1. Randomly generate initial population (individuals that could be potential solutions); 2. Cluster individuals into clusters and put the best individuals as cluster center in each cluster; 3. Replace some cluster centers with randomly generated individuals;

200

L. Mafteiu-Scai et al.

4. Depending on a pre-determined probability, replace the cluster center with a selected individual from this cluster or generate new individuals. 5. Depending on a pre-determined probability, from the two cluster centers or from other two individuals of each selected cluster, generate new individuals; 6. The process is stopped after a predefined number of iterations. In Fig. 4—using filled boxes for master activities and unfilled for slaves activities-, a synchronous master-slave model for BSO is proposed. We consider BSO a fully synchronous application, and in this case, all the processes are synchronized at regular points. To implement this, a basic mechanism for synchronizing processes—called barrier—was used at the point in each process where it must wait. In this case, all processes can continue from this point when all the processes have reached it.

Fig. 4 Synchronous model of BSO

Brain Storm Optimization Algorithms for Solving …

201

Fig. 5 Asynchronous model of BSO

In Fig. 5 an asynchronous master-slave model for BSO is proposed. This model enables the update sequence of the clusters to change dynamically, even if a cluster is updated more than once on an iteration. This approach can prevent the algorithm to fall in local optima. Anyway, the main advantage of this approach remains the economy of resources needed to synchronization. We mention that the number of fitness evaluation of individuals is equal on both approaches: synchronous and asynchronous. Even it is not easy to make a choice, the synchronous model is recommended when the computing processes are dominant in slaves’ works and the number of process communications is fewer. In such a case, the main disadvantage of idle times is offset.

2 The Proposed BSO Method for BMP 2.1 Serial Implementation The basic structure of the BSO algorithm proposed for bandwidth reduction in sparse matrices is described below: 1. Population initialization i.e. randomly generate n individuals (potential solutions)

202

L. Mafteiu-Scai et al.

2. Individuals evaluation 3. Clustering individuals into clusters and record the best individual as its cluster center in each cluster; 4. New individual’s generation: randomly select two clusters to generate new individual 5. Selection: the newly generated individual is compared with the existing individual with the same individual line indices; the better one is kept and recorded as the new individual 6. Individuals evaluation, i.e. calculating new bw values in the case of two lines/columns interchange 7. If stopping criteria are not met, goto 3. Mainly, our proposed algorithm is based on clustering. To improve the population diversity, diverging operations i.e. adding Gaussian noise to generate new idea, are used. To improve the algorithm performances, the basic BSO algorithm has been combined with a technique of the one proposed in [60], which allows the determination of the opportunity to interchange two lines/columns in a sparse matrix. More, the elitism strategy was used, i.e. the predominant use of individuals/solutions with better fitness values. The use of these two ingredients has allowed an increase in computational efficiency, especially by reducing operations in the objective search space. A brief description of the main objects and operations of the proposed algorithm is made further. – individuals: an individual represents, in our bandwidth optimization BSO method, an object/structure composed by a pair of lines/columns to be interchanged, the value of the bandwidth after the interchange will be made and the corresponding matrix of this interchange: struct individual { integer pair (i, j) integer bw integer matrix[][] – clustering: our proposed method is a converging BSO, based on clustering, that uses a self-determining number of clusters. Thus, for each line of the matrix is created a cluster, i.e. all the individuals that have the same first label to be interchanged are clustered together and record the best individual as its cluster center in each cluster; – elitism strategy: the best individuals from each cluster (those who have the lowest fitness value, i.e. the smallest values of the bandwidth respectively of the average bandwidth) are kept to be used in a new individuals generating process. These individuals which form the elite-set have more chances to generate new promising individuals. – stopping criteria: there are three stopping criteria. The first one is represented by a predefined number of iterations. The second criterion is to reach a predefined

Brain Storm Optimization Algorithms for Solving …

203

value for the value of the minimized indicator, called good-enough. The third stop criterion is extremely rarely achieved and is represented by reaching the ideal value for bw [61], i.e. a compact arrangement of all nonzero elements around the main diagonal, without “holes” (compact groups of zeroes around the main diagonal). This ideal value can be determined by a trivial algorithm. – diverging operations: the process is similar to a mutation process in genetic algorithms and it consists of replacing a cluster center by a randomly generated solution. This operation can avoid a premature algorithm’s convergence and help to “jump out” of the local optimum. Thus, after selecting an individual (i.e. an idea), a new one is generated using the relation: Inew = Iselected + δ · normrnd (0, 1)

(4)

where normrnd is the Gaussian random with mean 0 and variance 1, and δ is an adjusting factor which decreases the convergence speed down as the evolution goes [62]. Thus, this factor favors the global search at the beginning of the optimization process and improves the local search at the end of the process. The new individual will be accepted only if its fitness value is better than the old one. More, as shown in [62], time complexity of the BSO algorithms is reduced. This criterion was added at basic algorithm at Step 3.

2.2 Parallel Implementation First, we mention that an asynchronous model of parallelization was used, this mode of processing having the advantage of more efficient use of central memory. To implement in parallel the proposed algorithm, it was used IBM XL C compiler version 9.0 under Linux, and MPI (Message-Passing Interface), a standard in parallel computing which enables overlapping communications and computations among processors and avoid unnecessary messages in point to point synchronization. In the case of a n × n matrix and a parallel computing system with one master and p slaves, for each slave a number of lines from matrix, clusters (C) are allocated in terms of our BSO method. If p divides n (n MOD p = 0) the number of clusters allocated to each slave is C =(n DIV p). Otherwise (n MOD p = 0), first n MOD p slaves receive C = (n DIV p + 1) clusters and the other slaves are receiving C = (n DIV p) clusters, the difference of a unit being negligible in terms of processor balancing in the case of our problem. The proposed BSO parallel algorithm for BMP is briefly described below and its parallelization scheme can be seen in Fig. 6, where p represents the number of processors used (excluding master), the arrows represent the data flows and the numbers represent the order of the algorithm steps. 1. Master reads matrix from input 2. Master computes the bw value

204

L. Mafteiu-Scai et al.

Fig. 6 The parallelization scheme

3. Master sends matrix and bw value to slaves. 3 . Each slave receives matrix and bw values While termination conditions are not fullfiled: 4. Each Slave generates C random children (where child i is based on the current bw value in cluster i) 4 Each Slave selects the cluster with the worst quality (in terms of bw value) and split it in two sub-clusters (like in Bisecting K-means technique). Compute the centers of new sub-clusters and the best sub-cluster will be a new cluster for that slave. This step is repeated until they have been obtained C new clusters. In the end, only the best C clusters will be retained for that slave. 5. Each Slave sends the children to Master. 5 . Master receives children. 6. Master, for each child received, checks the child tag (index of first line swapped). 7. Master sends the child to the appropriate Slave. Once there are no more children to receive, Master sends to each slave the number of children that they should receive. 7 . Each Slave receives children until the limit has been reached (no more children incoming). Since it is random, some slaves can receive many children, some 0. 8. For each child, the Slave computes the bw, and if the child is better than the current cluster value, update the cluster. 9. Each Slave sends the best local bw value from its clusters to Master. 9 . Master receives the best local bw values from slaves 10. Master compares the the local bw values, and requests the matrix from the slave with the best value. 11. Each Slave receives the master’s answer, and only the chosen one sends the matrix to the Master 12. Master receives the matrix End While 13. Master send the final matrix to output. Related to Step 4 , it has the advantage of improving the quality of the solution, as expected from a genetic elitism strategy. In terms of time efficiency, there is a decrease

Brain Storm Optimization Algorithms for Solving …

205

in global performance. The strategy used (Bisecting K-means technique) was not included in the serial implementation because the method became prohibitive in terms of execution time in this case. In parallel implementation, the strategy bisecting k-means was included at the slave level, so, it does not significantly increase the total processing time because the number of master-slave communication processes does not increase.

2.3 Experimental Results In our experiments, we used matrices from the CAN, DWT, JAGMESH, LSHP and other sets included in Harwell-Boeing collection [63]. The results obtained with our algorithm—in different versions—for bw optimization are compared with results reported by others authors, and these results show that BSO is a good choice for optimizing the bw indicator.

2.3.1

Experimental Results—Serial Implementation

In Table 1, experimental results obtained using a classical BSO approach (algorithm version 1), without diverging operations described by relation (7), are compared with results reported by other authors related to bandwidth optimization. A second set of experiments was conducted by diverging operations criterion. In Table 1 (algorithm version 2), there are some experimental values obtained for bw values using matrices from Matrix Market [63], clustered by their size orders. Similar, it can be observed that using the criterion given by relation (7) in many cases, but not on all, can improve the quality of solutions, i.e. lower bw values. This behavior may be due to the increase of the convergence speed, i.e. getting better value in a predefined number of steps/iterations. Experimentally, we noticed that sometimes the same result can be obtained without using this criterion if the number of steps/iterations is considerably increased. In other situations, non-use of the criterion leads to blocking the application in local optima, and increasing the maximum number of steps/iterations no longer helps to improve the quality of the solution. It is noticed that the quality of the solutions—i.e. lower values—increases with the size of the matrix for both indicators. Such a phenomenon is expected in an evolutionary process, based on population, because the increasing the matrix size also increases the search space and the population diversity. We mention that the values obtained for bw in our experiments represent the average values of 10 tests for each matrix in part, with ± 25% deviation. The differences between results obtained with versions 1 and version 2 on serial implementation may be due to the fact that an increased speed of convergence (version 2) based on a finite/ limited iterations mean better value obtained in some cases for the indicator bw, as can be seen in Table 1.

206

L. Mafteiu-Scai et al.

Table 1 Bandwidth minimization experimental results Matrix

Initial bw value

CuthillHACS Mckee [65] [Cuthill969]

Hybrid heuristic [61]

MafteiuProposed BSO Scai and Version Version Cornigeanu 1 2 [66]

Parallel

can_24

21

8

11

8

19

9

11

9

can_61

50

26

42

17

18

28

28

28

can_62

48

9

12

33

9

21

16

21

can_73

39

27

22

27

31

28

28

19

can_96

31

23

17

41

23

31

26

23

can_187

63

23

33

63

23

63

26

33

can_229

172

49

120

89

95

131

70

102

can_256

251

116

148

139

116

128

128

127

can_268

263

134

165

98

133

133

132

133

dwt_59

25

8

–

20

8

16

11

9

dwt_66

44

3

–

4

3

16

28

6

dwt_72

12

8

–

9

8

12

16

11

dwt_87

63

18

–

31

18

63

28

44

dwt_162

156

20

–

24

20

72

34

62

bcspwr07

1488

–

–

–

439

440

445

440

bcspwr08

1495

–

–

–

428

430

429

430

bcspwr10

5190

–

–

–

2085

2080

2070

2081

dwt_1005

852

–

–

–

363

370

342

370

dwt_1242

937

–

–

–

326

330

331

331

dwt_2680

2500

–

–

–

745

750

731

740

jagmesh1

779

–

–

–

213

210

202

202

jagmesh3

1059

–

–

–

362

370

369

360

jagmesh4

1409

–

–

–

578

580

591

591

jagmesh5

758

–

–

–

326

325

321

320

jagmesh8

226

–

–

–

71

80

70

70

jagmesh9

1025

–

–

–

590

595

573

570

lshp1270

1251

–

–

–

624

628

628

625

lshp1561

1540

–

–

–

772

780

751

770

lshp3025

2996

–

–

–

1564

1560

1512

1510

2.3.2

Experimental Results—Parallel Implementation

Experimental results were obtained on IBM Blue Gene/P supercomputer using IBM C++ compiler and MPI model for processors communications. We mainly used real world instances, i.e. matrices from the CAN, DWT, JAGMESH, LSHP and other sets included in Harwell-Boeing collection (http://math.nist.gov/67/data/HarwellBoeing). We use Coordinate Format for matrix representation, a format that is suitable for representing sparse matrices. In this format, only nonzero elements are

Brain Storm Optimization Algorithms for Solving …

207

provided and the coordinates of each nonzero element are given explicitly. Thus, besides the great economy of computer memory, we obtained a more efficient process in terms of execution time compared to the classic format of representation of matrices in memory, especially in the case of medium and large sized matrices. We mention that only first version of serial algorithm was parallelized, i.e. the version without diverging operations criterion described by relation (4). In our parallelization, the matrices were divided into compact blocks, equal or approximately equal, depending on the number of processors allocated. Thus, we can say that we have a good load balancing for all slaves, fact that is very important in parallel processing. This is due to the fact that all slave processors perform the same operation on equal or approximately equal volumes of data. The maximum number of iterations was set to 1000. Two main issues were tracked in our experiments: the value of bw and the parallel algorithm performance. The first experiments did not include Step 4 and with regard to the first objective, bw values, it was observed that the values obtained are identical or very close to the values obtained with the serial variant, version 1 of the algorithm. Including the Step 4 in algorithm led to an improvement in the quality of the solution, i.e. lower values for the bw indicator, especially for larger matrices, as it can be seen in Table 1, last column. The inclusion of this step at slave level did not significantly increase the total processing time because the number of master-slave communication processes does not increase. With regard to the second objective, Relative Speedup was our main choice in performance evaluation of parallel algorithm implementation, i.e. the results of the parallel program running on a single processor were compared with results obtained with multiple processors. In Fig. 7, the average of experimental values for Relative Speedup indicator is plotted for a number of 50 different matrices with dimensions between 300 and 500, for different numbers of allocated processors. Figure 7 shows good values for Relative Speedup, especially for a small number of processors. The scalability of the algorithm decreases with increasing the number of processors, due to the increasing number of master-slave communication processes.

3 Equations Systems Solving Using BSO A first method for solving equations systems that uses brainstorming optimization was proposed in work [48]. An improved variant that uses circular crossover operator proposed in this work and is described in this section. At the base of this algorithm a few elements from human brainstorming are described in the next subsection.

208

L. Mafteiu-Scai et al.

Fig. 7 The Relative Speedup values example

3.1 The Proposed Algorithm In proposed algorithm designed to solve systems of equations, the problem is viewed as an multi-objective optimization problem [64], derived from single objective multimodal optimization problem. This approach has been chosen because some equation systems have multiple solutions. Thus, a given objective function, called fitness function, must be minimized. With this function, we know how good a particular solution is. In case of an equations system described by a function f (X) with X = {x 1 , x 2 , …, x n } the vector of unknowns, the solving process involves finding the values for each x i that verify all the equations of the system described by relation (5) ⎧ ⎪ f1 (x1 , x2 , . . . , xn ) = 0 ⎪ ⎪ ⎪ ⎪ f ⎪ ⎪ 2 (x1 , x2 , . . . , xn ) = 0 ⎨ . (5) ⎪ . ⎪ ⎪ ⎪ ⎪ . ⎪ ⎪ ⎩ fn (x1 , x2 , . . . , xn ) = 0 which has the fitness function abs(f i (X)). A solution is an assignment of values to the variables x 1 , x 2 , …, x n such that each equation is satisfied. The solution set is the set of all possible solution for equations system and there are four possible situations: unique, several, infinitely or no solution for system. The number of ideas generated from human brainstorming is represented by the size of population i.e. all X vectors in brain storm optimization process that exist on a stage/iteration. A round of idea generation from human brainstorming model is represented in our approach by an iteration in BSO process. The basic structure of the proposed algorithm is presented in Fig. 8.

Brain Storm Optimization Algorithms for Solving …

209

Fig. 8 Proposed BSO algorithm for solving equations systems

Crossover operator used in our approach was the circular crossover (see Sect. 1.7) with two cutting points, i.e. a new individual are obtained from other three individuals, each of them contributing one third of its part to the new individual, as it was exemplified in Sect. 1.4. Six new individuals are obtained from three current individuals. This means that from three candidate solution vectors other six candidate solution vectors are obtained, each of them containing one section of the three currents. We mention that the position of a section from a current individual is preserved in new individual, which leads to six possible combinations. Two types of mutation were used in the proposed algorithm: a pure-random mutation and a non-random mutation. First type ensures a high diversity of population in search space and avoid population degeneration. The number of individual generated in this mode was in our approach equal with n. The second type has small random variations for each of the three parts of an individual, thus resulting in three new individuals from each existing individual.

210

L. Mafteiu-Scai et al.

So, in case of a population with n individuals, after each iteration, there will be 6n new individuals generated by circular crossover and 4n individuals generated by mutation operators, that is, 11n individuals in total (one is the old). For all of these, the fitness values will be computed and only the first n individuals will be selected and form the new population. We mention that in first population generating stage, a selective initialization is used, meaning that a large number of random solutions are randomly generated and then the initial population is selected from these, according to genetic elitism. Three termination conditions were used in the proposed algorithm: finding an exact solution (fitness function satisfying), finding an approximate solution (the difference between right and left side of each equations must be less than a preset given value), or reaching a preset number of iterations. After experiments it was observed that the proposed method always finds a solution to a given system of equations (linear and nonlinear)—an exact or an approximate solution—even in situations when conventional methods fail (determinant null, system dependent, convergence conditions not satisfied, etc.). In the second approach the elements from graph theory presented in Sect. 1.6 were used to improve the proposed method. So, for a n dimensional system of equations a (n + 1)-partite graph is generated. The first partition, hereinafter called main partition is fixed from number of vertices point of view, the dimension being equal to n. In this partition the vertices are represented by equations. After that, new n partitions are generated, hereinafter called secondary partitions. In these secondary partitions, the vertices are represented by numerical values of possible solution vectors, which are obtained by the evolutionary process described before. Each partition corresponds in fact to all possible solutions for each unknown of system in part. The edges of the graph are determined by the fitness function and suggest that a numerical value from a particular secondary partition is a possible solution to some unknown and some equation of the system. A solution vector for equations system from this graph representation of the problem is in fact a (n + 1)-clique. Multiple solution are represented by multiple and distinct (n + 1)-cliques. The quasi-cliques represent a starting point for a new crossover process which aims to achieve more quickly the systems solution. These quasi-cliques are sorting descending by number of edges and a first part of them are subject to a new evolutionary process, i.e. crossover and mutation. A sample example of this new approach can be seen in Fig. 9. The graph representation is the final form of the associated graph, after the evolutionary process for a 3-dim system of linear equations: ⎧ ⎨ x + 2y + 3z = 14 x+y+z =6 ⎩ 3x + 2y + z = 10

(6)

Brain Storm Optimization Algorithms for Solving …

211

The system equation composed of by e1, e2 and e3 equations has multiple solutions, two of them are represented as cliques (edges in red and blue) and one is represented as quasi-clique (edges in yellow) in associated graph (see Fig. 9). The main task of proposed method is to determine the cliques and if they do not exist, the task continues to determining the quasi-cliques. In last case, the best quasi-cliques are selected, after a greedy selection. The selected quasi-cliques will be subjects of a new evolutionary process. So, the first variant of the algorithm will be completed with the sequence: … If cliques found then print solution else

Fig. 9 k-partite graph of equations system (6)

212

L. Mafteiu-Scai et al.

search for quasi-cliques sort quasi-cliques Apply circular crossover to elite of quasi-cliques … This sequence will be inserted after the crossover sequence in initial algorithm.

3.2 Experimental Results The values of the BSO algorithm parameters used in our experiments were: – population size = n, where n is the size of the equations system – maximum number of iteration = 1000 – value for maximum acceptable deviations (difference between left and right side) = 101 . The main advantage of using circular crossover and quasi-qliques consists in increasing the convergence speed of the proposed algorithm. There is not a constant value for this speedup, but an approximate value obtained from the experiments is around 2×, from runtime point of view. This value is low, because the computation process consumes resources with search and sort quasi-cliques. And more, the new crossover operator involves additional computational effort. A comparison of system (6) solutions obtained with our proposed method and other known methods is given in Table 2. Note that system (6) has an infinite number of solutions and Wolfram Mathematica software can produce a set of linearly independent vectors from which system solutions can be constructed. This test confirms the validity of the multiple solutions provided by our algorithm. Another example is given below. We consider three nonlinear Eq. (7) that can form seven different equations systems, as can be seen in Table 3. (3x − 2)1/4 + (15x + 1)1/4 − 3 = 0 (a) 2x + 3x + 6x − 3x2 − 5x − 3 = 0 (b) 23x+1 − 13 · 22x + 11 · 2x+1 − 8 = 0 (c)

(7)

In Table 3, for all possible systems of equations formed by these three equations, the exact and the approximate solutions obtained with the proposed method are given. Through “deviation” we mean the difference between left and right side of each equation in part after the approximate solution replaces the unknowns and shows how good the solution is. As can be observed in Table 3 this deviation is zero or very close. We mention that only integer solutions were searched for system (7) using our BSO approach, i.e. the search space is the set of integers.

Brain Storm Optimization Algorithms for Solving …

213

Table 2 Comparative solutions for system (6) Gausselimination

GaussJordan

LU Genetic factorization algorithm [67]

Conjugate gradient

fail

{4,− 4, 6}

fail

{4,−16,14} x = -2 + z fail y = 8-2z

{1, 2, 3} {0, 4, 2} {2, 0, 4}

Table 3 Solutions of non-linear system (7)

The equations of the system

Wolfram mathematica

Exact solution

Proposed BSO algorithm {1, 2, 3}, {0, 4, 2}, {2, 0, 4}, {−2,8, 0}, {4, −4, 6}, {1.45, 1.08, 3.45}, {−4.13, 12.26, − 2.13}, {75.2, − 146.3, 77.2}

Approximate solution with proposed BSO algorithm

1

(a)

1

2

2

(b)

1

−2, −1, 0, 1, 2

5 0 −1 3

(c)

2

0, 1, 2 −2, −1,

1 4



5



6



−1 (a)

1

1, 2

1

1, 2

(b) (a) (c)

7

(b)

(c) ⎧ ⎪ ⎪ ⎨ (a) (b) ⎪ ⎪ ⎩ (c)

1

−2, −1, 1, 2

−1 1

1, 2

214

L. Mafteiu-Scai et al.

3.3 Parallel Implementation Because the sizes of the equations systems used in our experiments were small, the experiments were performed using a personal computer with 16 cores (AMD RYZEN Threadripper 1950X 16-Core/32 Threads 3.4 GHz with 32 Gb DDR4 RAM). In a first variant of parallelization, loop parallelism was used for serial proposed algorithm. OpenMP has an easy mechanism for it, called OpenMP parallel loops. OpenMP has been designed as a programming model for taking advantages of multicore systems. From model of parallelism point of view, multi-core processors are MIMD Multiple Instructions Multiple Data), i.e. different cores execute different threads on different parts of memory, where the same memory are shared by cores. The sequential algorithm was adapted for paralelization (#pragma omp parallel for in C language) and a parallel region was created around the loop marked with “*” in Fig. 8, where the same independent operations are performed repeatedly on different data. The modified algorithm, written in C language, was given to a multiprocessor system that runs OpenMP and the work is distributed over the available threads in a parallel region. With OpenMP loop strategy parallelism we obtain a maximum speedup of 8×, but the average speedup was only 3×, due to imbalance of corres’ works. More parallel variants have been tried in our experiments but the Master-Slave Model (MSM) with a synchronous parallelization seems to be the most effective in case of our problem. Distribution of crossover and mutation operations, and fitness calculation, was done on the slaves. This allows utilization of several processors (or distributed computer systems) to solve the problem, the global model being a simple way to reduce very long computation times as long as the evaluation time of the objective functions is higher than the communication time between master and slaves. Parallelisation of these operations is done by assigning a fraction of the population to each of the processors available. The ideal case is one individual per processing element, but is rarely possible to accomplish. Communication processes occurs only as each slave receives a subset of individuals to evaluate and when the slaves return the fitness values, after crossover and mutation have been applied. On single systems such problems might not be possible to solve within a reasonable time when the equations systems sizes are greater than 50. Synchronization problems arise due to unbalanced processor load, this unbalance mainly due to fitness evaluations (candidate solution vector is replaced on a equation), these processes being different for each equation in part because the equations are different (more or less null coefficients). This model, synchronous master-slave, was relatively easy to implement and a significant speedup was achieved. This is mainly due to the fact that the communications cost does not dominate the computation cost. The solutions were not affected by parallelization, because the model used does not modify the algorithm, but only gives it extra processing speed. The parallel algorithm used is presented below, where (M) and (S) show who execute the operations: master or slaves.

Brain Storm Optimization Algorithms for Solving …

215

Begin (M) Read system and other working parameters (M) Search Space Initialization (M) Generating Random Initial Population (M) Repeat (M) Evaluate individuals by their fitness (M) Put and Sort individuals in clusters (M) Send clusters to Slaves (S) Select for each cluster in part the elite group (S) Crossover and Mutation for each elite group (S) Add new individuals to population (S) Send results to Master Repeat (M) Random generating new individuals If Degenerate (M) Expand Search Space Else (M) Complete Population with new individuals Until Not Degenerated (M and S)Compute Average deviation of solutions for each equation in part and for entire system Until Termination Conditions are fulfilled End

As a remark, can be observed that the iterations are controlled by master processor, which ensures increased stability of the algorithm. The best experimental speedup obtained is close to ideal speedup, as can be seen in Fig. 10 and was obtained for about 10% of the experiments. In the other side, there are cases with the lowest speedup—15%—with values between 1× and 2×. For 200 experiments, the average speedup is about 4×, as can be seen in Fig. 11. Mention that “Number of cores” is the number of cores used as slaves in parallelization model master-slave.

4 Conclusions and Future Work In this chapter, two BSO algorithms inspired from the human brainstorming process were proposed. First algorithm is for bandwidth reduction of sparse matrices, optimizations with many real applications, as some were listed. The results obtained with proposed BSO method for bandwidth optimization, compared with results reported by other authors, are promising and encouraging to improve the proposed methods in the future. Using diverging operations criterion and the Bisecting K-means technique conduct in many cases to lower values for bw and reduce the time complexity of the proposed algorithm. From these points of view, future use of the new theoretical results on BSO research, could improve the quality of the solutions and decrease the

216

L. Mafteiu-Scai et al.

Fig. 10 The best speedup obtained

Fig. 11 The average speedup obtained

runtime. Using the Coordinate Format, for representation the matrices in computer memory, improves the algorithm from the efficiency of using the memory space point of view. The parallel algorithm must be improved, especially regarding to its scalability from number of processors point of view. The second BSO algorithm proposed combined with concepts from graph theory can be very helpful in finding solution for equations and system of equations, linear and nonlinear. The proposed method can find solutions of a given system of equations, even in cases when traditional methods fail, as was shown in examples. If no exact solution found, an approximate solution is good and it can be obtained by the proposed method, solutions whose fitness is very close to zero or less than a preset value. These approximate solutions are good/acceptable in many real life application. Also, systems with multiple solutions can be solved, the proposed method has been able to find all the solutions inside of a given interval preset by user. The major disadvantage of the proposed algorithm is its running time that is prohibitive in many cases for systems with dimension greater than 50. Even for smaller systems the execution time

Brain Storm Optimization Algorithms for Solving …

217

is greater if the values of the solution vector are distributed in a large interval. So, the proposed method should be refined, and especially made more effective through parallelization. The proposed crossover operator, called circular crossover, must be tested in large variety of evolutionary problems and a complete theoretically approach must be done if the answer is positive.

References 1. Jablonski, A.: A Monte Carlo algorithm for solving systems of non-linear equations. J. Comput. Appl. Math. 6(3), 171–175 (1980). Elsevier 2. Al-Shakarchy, N.D.K., Abd, E.H.: Application of neural network for solving linear algebraic equations. J. Kerbala Univ. 10(4) (2012). Scientific 3. Ren, H., Wu. L., Bi, W., Ioannis, Argyros, K.: Solving nonlinear equations system via an efficient genetic algorithm with symmetric and harmonious individuals. Appl. Math. Comput. 219(23), 10967–10973 (2013). Elsevier 4. Zhou, Y., Huang, H., Zhang, J.: Hybrid artificial fish swarm algorithm for solving ill-conditioned linear systems of equations. In: Chen, R. (ed.) Intelligent Computing and Information Science. Communications in Computer and Information Science, vol. 134. Springer (2011) 5. Xia, Y.H., Li, Y.G.: An improved quantum ant colony algorithm of solving nonlinear equation groups. Adv. Mater. Res. 1049–1050, 1363–1366 (2014) 6. Mafteiu-Scai, L.O., Mafteiu-Scai, E.J.: Solving liniar systems of equations using a memetic algorithm. IJCA (0975–8887) 58(13) (2012). ISBN 973-93-80870-43-5 7. Abdollahi, M., Bouyer, A., Abdollahi, D.: Improved cuckoo optimization algorithm for solving systems of nonlinear equations. J. Supercomput. 72, 1246–1269 (2016). https://doi.org/10. 1007/s11227-016-1660-8 8. Hirsch, M.J., Pardalos, P.M., Resende, Mauricio, G.C.: Solving systems of nonlinear equations with continuous GRASP. Nonlinear Anal.: Real World Appl. 10(4), 2000–2006 (2009). Elsevier 9. Pourrajabian, A., Ebrahimi, R., Mirzaei, M., Shams, M.: Applying genetic algorithms for solving nonlinear algebraic equations. Appl. Math. Comput. 219(24), 11483–11494 (2013). Elsevier 10. Abdel-Baset, M., Hezam, I.M.: A hybrid flower pollination algorithm for solving illconditioned set of equations. Int. J. Bio-Inspired Comput. 8(4) (2016) 11. Arbenz, P., Cleary, A., Dongarra, J., Hegland, M.: Parallel Numerical Linear Algebra, Chapter A Comparison of Parallel Solvers for Diagonally Dominant and General Narrow Banded Linear Systems, pp. 35–56. Nova Science Publishers Inc, Commack, NY, USA (2001) 12. Mafteiu-Scai, L.O.: Average bandwidth relevance in parallel solving systems of linear equations. IJERA 3(1), 1898–1907 (2013). ISSN 2248-9622 13. Maruster, S., Negru, V., Mafteiu-Scai, L.O.: Experimental Study on Parallel Methods for Solving Systems of Equations. IEEE (2013). https://doi.org/10.1109/synasc.2012.7. ISBN 978-14673-5026-6 14. Chan, G.K., Head-Gordon, M.: Highly correlated calculations with a polynomial cost algorithm: a study of the density matrix renormalization group. J. Chem. Phys. 116(11) (2002). https://doi.org/10.1063/1.1449459 15. Huang, H., Dennis, J.M., Wang, L., Chen, P.: A scalable parallel LSQR algorithm for solving large-scale linear system for tomographic problems: a case study in seismic tomography, ICCS 2013, Proc. Comput. Sci. 18, 581–590 (2013) 16. Ababei, C., Feng, Y., Goplen, B., Mogal, H., Zhang, T., Bazargan, K., Sapatnekar, S.: Placement and Routing in 3D Integrated Circuits, Design and Test of Computers, pp. 520–531. IEEE (2005). ISSN 0740-7475

218

L. Mafteiu-Scai et al.

17. Bhatt, S.N., Leighton, F.T.: A Framework for Solving VLSI Graph Layout Problems, Computer and System Sciences, vol. 28. Elsevier (1984) 18. Caproni, A., Cervelli, F., Mongiardo, M., Tarricone, L., Malucelli, F.: Bandwidth reduced full-wave simulation of lossless and thin planar microstrip circuits. ACES J. 13(2), 197–204 (1998) 19. Ullman, J.D.: Computational Aspects of VLSI. Computer Science Press, Rockville, MD (1983) 20. Behrisch, M., Bach, B., Riche, N.H., Schreck, T., Fekete, J.D.: Matrix reordering methods for table and network visualization. Comput. Graph. Forum J. 35. ISSN 1467-8659 21. Meijer, J., van de Pol, J.: Bandwidth and Wavefront Reduction for Static Variable Ordering in Symbolic Reachability Analysis, NASA Formal Methods, vol. 9690, pp. 255–271. LNCS, Springer (2016) 22. Cuthill, E., McKee, J.: Reducing the bandwidth of sparse symmetric matrices. In: Proceeding of the 24th National Conference ACM, pp. 157–172 (1969) 23. Chinn, P.Z., Chvátalová, J., Dewdney, A.K., Gibbs, N.E.: The bandwidth problem for graphs and matrices—a survey. J. Graph Theo. (2006). https://doi.org/10.1002/jgt.3190060302 24. Mafteiu-Scai, L.O.: The bandwidths of a matrix. A survey of algorithms. Ann. West Univ. Timisoara-Math. 52(2), 183–223 (2014) 25. Ribeiro, J.A., Gonzaga de Oliveira, S.L.: Uma Revisao Sistematica Sobre Algoritmos Paralelos´ Para os Problemas de Reducoes de Largura de Banda e de Profile de Matrizes. In: XLIX Simpósio Brasileiro de Pesquisa Operacional Blumenau-SC, 27 a 30 de Agosto de 2017 (2017) 26. Runco, M.A., Jaeger, G.J.: The standard definition of creativity. Creativity Res. J. 24(1), 92–96 (2012). ISSN 1040-0419 27. Lytton, H.: Creativity and Education. Routlegde (2012). ISBN 978-0-415-67549-9 28. Sawyer, R.K.: Explaining creativity. In: The Science of Human Innovation, 2nd edn (2012). ISBN-10 0199737576 29. Colton, S., Wiggins, G.A.: Computational creativity: the final frontieer? In: De Raedt, L., Bessiere, C., Dubois, D. (eds.) ECAI 2012: 20th European Conference on Artificial Intelligence (2012) 30. Gero, J.S., Maher, M.L.: Modeling Creativity and Knowledge-Based Creative Design. Lawrence Publisher (1993). ISBN 0-8058-1153-2 31. Merrick, K.E., Isaacs, A., Barlow, M., Gu, N.: A shape grammar approach to computational creativity and procedural content generation in massively multiplayer online role playing games. Entertain. Comput. 4(2), 115–130 (2013) 32. Pinel, F., Varshney, L.R.: Computational creativity for culinary recipes. ACM Proc. CHI EA 14, 439–442 (2014). ISBN 978-1-4503-2474-8 33. McDermott, J.: Functional representations of music. In: Proceedings of the Third International Conference on Computational Creativity (2012). ISBN 978-1-905254668 34. Fen, L.H.: A review on the pragmatic approaches in educating and learning creativity. Int. J. Res. Stud. Educ. Technol. 1(1), 13–24 (2012). ISSN 2243-7738 35. Osborn, A.F.: Applied Imagination. Principles and Procedures of Creative Problem Solving. Charles Scribner’s Sons, New York, NY (1963) 36. Furnham, A.: The Brainstorming Myth. Wiley (2003). https://doi.org/10.1111/1467-8616. 00154 37. Dennis, A.R., Williams, M.L.: Electronic brainstorming: theory, research and future directions. In: Arlington, B. (eds.) Group Creativity: Innovation through Collaboration. Oxford University Press (2003) 38. Boden, M.A.: Creativity and artificial intelligence. Artif. Intell. 103, 347–356 (1998). Elsevier 39. Shi, Y.: Brain storm optimization algorithm. Adv. Swarm Intell. LNCS 6728, 303–309 (2011). Springer 40. Xue, J., Wu, Y., Shi, Y., Cheng, S.: Brain storm optimization algorithm for multi-objective optimization problems. ICSI 2012. LNCS 7331 (2012). Springer 41. Zhan, Z., Zhang, J., Shi, Y., Liu, H.: A modified brain storm optimization. In: 2012 IEEE Congress on Evolutionary Computation (2012). https://doi.org/10.1109/cec.2012.6256594. IEEE

Brain Storm Optimization Algorithms for Solving …

219

42. Duan, H., Li, S., Shi, Y.: Predator–prey brain storm optimization for DC brushless motor. IEEE Trans. Magn. 49(10) (2013). https://doi.org/10.1109/tmag.2013.2262296, IEEE 43. Duan, H., Li, C.: Quantum-behaved brain storm optimization approach to solving Loney’s solenoid problem. IEEE Trans. Magn. 51(1) (2015) 44. Sun, C., Duan, H., Shi, Y.: Optimal satellite formation reconfiguration based on closed-loop brain storm optimization. IEEE Comput. Intell. Mag. 8(4) (2013). https://doi.org/10.1109/mci. 2013.2279560. IEEE 45. Qiu, H., Duan, H.: Receding horizon control for multiple UAV formation flight based on modified brain storm optimization. Nonlinear Dyn. 78, 1973 (2014). https://doi.org/10.1007/ s11071-014-1579-7. Springer 46. Li, J., Duan, H.: Simplified brain storm optimization approach to control parameter optimization in F/A-18 automatic carrier landing system. Aerosp. Sci. Technol. 42 (2015) 47. Chen, J., Cheng, S., Chen, Y., Xie, Y., Shi, Y.: Enhanced brain storm optimization algorithm for wireless sensor networks deployment. In: ICSI 2015, vol. 9140. Springer (2015) 48. Mafteiu-Scai, L.O.: A new approach for solving equations systems inspired from brainstorming. IJNCAA 5(1), 10–18 (2015). ISSN 2412-3587 49. Jia, Z., Duan, H., Shi, Y.: Hybrid brain storm optimisation and simulated annealing algorithm for continuous optimization problems. Int. J. Bio-Inspired Comput. 8(2) (2016) 50. Jiandong, D., Lupeng, C., Qian, S., Jing, W., Teng, M., Fuping, Y.: Optimal scheduling strategy of active distribution system using brain storm optimisation algorithm. In: The 6th International Conference on Renewable Power Generation (RPG) (2017) 51. Xia, Y., Huang, J.S., Tang, W., Wu, D.: Quantum brain storm optimization of GaN power amplifier design. In: 2017 International Conference on Computer Science and Application Engineering (CSAE 2017) (2017). ISBN 978-1-60595-505-6 52. Chen, W., Cao, Y.Y., Sun, Y., Liu, Q., Li, Y.: Improving Brain Storm Optimization Algorithm via Simplex Search (2017). arXiv preprint arXiv:1712.03166 53. Cheng, S., Qin, Q., Chen, J., et al.: Brain storm optimization algorithm: a review. Artif. Intell. Rev. 46, 445 (2016). https://doi.org/10.1007/s10462-016-9471-0 54. Cheng, S., Sun, Y., Chen, J., Qin, Q., Chu, X., Lei, X., Shi, Y.: A comprehensive survey of brain storm optimization algorithms. In: Evolutionary Computation (CEC). IEEE, San Sebastian, Spain (2017). https://doi.org/10.1109/cec.2017.7969498 55. Gilliss, N., Glineur, F.: A continuous characterization of the maximum-edge biclique problem, ACM DL. J Global Optim. Arch. 58(3), 439–464 (2014) 56. Alexe, G., Alexe, S., Crama, Y., Foldes, S., Hammer, P., Simeone, B.: Consensus algorithms for the generation of all maximal bicliques. Discrete Appl. Math. 145, 11–21 (2004) 57. Sim, K., Li, J., Gopalkrishnan, V., Liu, G.: Mining maximal quasi-bicliques: novel algorithm and applications in the stock market and protein networks. Stat. Anal. Data Min. 2(4), 255–273 (2009) 58. Miklosko, J., Kotov, V.J.: Algorithms, Software and Hardware of Parallel Computers. Springer Science & Business Media (2013). ISBN 9783662111062 59. Malek, M., Guruswamy, M., Pandya, M.: Serial and parallel simulated annealing and tabu search algorithms for the traveling salesman problem. Ann. Oper. Res. 21, 59–84 (1989) 60. Mafteiu-Scai, L.O.: Interchange opportunity in average bandwidth reduction in sparse matrix. West Univ. Timisoara Ann. (2012). ISSN 1841-3293 61. Mafteiu-Scai, L.O., Negru, V., Zaharie, D., Aritoni, O.: Average bandwidth reduction in sparse matrices using hybrid heuristics-extended version. In: Proc. KEPT, 379–389 (2011). ISSN 2067-1180 62. Cao, Z., Rong, X., Du, Z.: An improved brain storm optimization with dynamic clustering strategy. In: ICMME 2016 (2017) https://doi.org/10.1051/matecconf/20179519002 63. http://math.nist.gov/67/data/Harwell-Boeings 64. Deb, K., Saha, A.: Finding Multiple Solutions for Multimodal Optimization Problems Using a Multi-Objective Evolutionary Approach (2010) (GECCO 2010, In press) 65. Pintea, C.M., Crisan, G.C., Chira, C.: A hybrid ACO approach to the matrix bandwidth minimization problem. In: M. Graa Romay et al. (eds.) HAIS 2010, Part I, LNAI 6076, pp. 407–414. Springer (2010)

220

L. Mafteiu-Scai et al.

66. Mafteiu-Scai, L.O., Cornigeanu, C.A.: A parallel heuristic for bandwidth reduction based on matrix geometry. In: SYNASC Timisoara, 2016. IEEE (2017). https://doi.org/10.1109/synasc. 2016.058. ISSN 2470-881X 67. Ikotun Abiodun, M., Lawal Olawale, N., Adelokun, Adebowale, P.: The effectiveness of genetic algorithm in solving simultaneous equations. Int. J. Comput. Appl. 14(8), 0975–8887 (2011)

StormOptimus: A Single Objective Constrained Optimizer Based on Brainstorming Process for VLSI Circuits Satyabrata Dash, Deepak Joshi, Sukanta Dey, Meenali Janveja and Gaurav Trivedi

Abstract This chapter presents the main aspects and implications of design optimization of electronic circuits using a general purpose single objective optimization approach based on the brainstorming process, which is referred as StormOptimus. The single objective optimization framework is utilized for sizing of four amplifiers, and one VLSI power grid circuit. During optimization, the problem specific information required for each circuit is kept to minimal, which consists of input specifications, design parameter ranges and a fitness function that represents the circuit’s desired behavior. Several experiments are performed on these circuits to demonstrate the effectiveness of the proposed approach. It is observed that a satisfactory design is achieved for each of the five circuits by using the proposed single objective optimization framework. Keywords Brain storm optimization · StormOptimus · VLSI circuits

1 Introduction With the advent of deep-submicron technologies, device scaling attempts to keep up with Moore’s law [19]. Further, with the increase in overall power dissipation of a chip, it is necessary to come up with new challenging and technical solutions, which can account for the tradeoff among silicon area and on-chip power dissipation. The explosive growth of portable and handheld devices, mobiles, PDAs, tablets, etc., has announced the advent of multi-core systems, and the design effort is shifted to the reduction of power dissipation in high-performance chips without affecting reliability. In response to the increase in the design of ultra-low power high-performance chips, it is impossible to handle the increased integration complexity, featuring multimillion transistor ICs. These higher levels of integration have unmasked the need to optimize dozens of performance parameters (e.g., gain, power consumption, area, S. Dash (B) · D. Joshi · S. Dey · M. Janveja · G. Trivedi Indian Institute of Technology Guwahati, Guwahati 781039, Assam, India e-mail: [email protected] © Springer Nature Switzerland AG 2019 S. Cheng and Y. Shi (eds.), Brain Storm Optimization Algorithms, Adaptation, Learning, and Optimization 23, https://doi.org/10.1007/978-3-030-15070-9_9

221

222

S. Dash et al.

etc.) that represent the behavior of underlying circuits. Estimating such performance parameters with respect to various design constraints is quite challenging in terms of the number of resources and time consumed during the design process. Therefore, a number of electronic design automation (EDA) tools are presented in the literature to design and analyze the various circuits so that the desired production goals can be met. Nowadays, these EDA tools are employed in conjunction with optimization techniques to reform the design process. These optimization techniques evaluate the performance parameters for a given set of design constraints. The optimization is carried out until the convergence is achieved. Finally, an effective circuit performance representing an optimal cost function is obtained satisfying all performance specifications subjected to design constraints. Prior to the development of function optimization techniques [7, 14, 27], several efforts [3, 18] have been made to find solutions to the circuit design problem by generating multiple samples (Monte-Carlo samples) of each circuit model to capture prominent functionalities of circuits. However, the cost of estimating performance parameters of the circuit increases with increase in the number of samples utilized during evaluation. On the other hand, employing less number of samples affects the accuracy of the solution. Therefore, new optimization techniques are presented to mitigate the computational cost while evaluating the performance parameters of circuits. These techniques tend to translate the design optimization problem into a function (cost function or fitness function) maximization or minimization problem before solving it. The cost function represents an analytical formulation of the circuit design parameter that needs to be optimized during the design process. Although these optimization techniques follow a deterministic approach to perform circuit design, it requires prior knowledge or additional design equation formulation for assessment of initial design point. Further, the degree of uncertainty increases with more complex circuits and in such case, analyzing the cost function leads to faults and inconsistent solutions. However, with the advent of evolutionary algorithms, swarm intelligence based algorithms and their variants [4, 25, 26], it is possible to efficiently analyze the circuit design problem by envisaging the uncertainties and by exploiting the design space in each evaluation to maintain design accuracy. Recently, the collective behavior of both nature-inspired pattern and data mining technique are applied together to explore and exploit the solution space naturally. This collective behavior is presented as a new optimization algorithm, called brain storm optimization (BSO) algorithm. This BSO algorithm employs a natural selection process to choose the optimum solution from a cluster of solutions, where the individuals are grouped using Osborn’s four rules. Various modifications are presented in literature to improve the performance of BSO algorithm [24, 34, 35]. Further, BSO algorithm is employed to solve various real-life problems to showcase the applicability [13, 22, 28, 30–32]. In view of this, we present a constrained-aware single objective optimization framework, called StormOptimus, based on brainstorming process [23] to analyze the circuit performance parameters and find an optimal solution for circuit performance. StormOptimus is capable of generating a near optimal solution for a given circuit parameter by employing exploitation and exploration of feasible design space

StormOptimus: A Single Objective Constrained Optimizer …

223

concurrently. The framework makes use of a clustering strategy to group distinct individuals into multiple clusters. Then an elite individual is chosen through a series of probabilistic decision-making strategy. A new individual is generated from the elite individual using a mutation strategy. Finally, both elite and new individuals are compared to select the best individual in each generation. The proposed single objective framework has following key features. • An efficient k-means++ algorithm is incorporated in the framework as an initial seeding technique for k-means clustering algorithm during clustering step. • New individuals are generated using an adaptive Lévy mutation operator to maintain diversity across the population. • Two different constraint handling strategies are introduced inside framework for handling boundary constraints using random procedure [16] and for handling functional constraints using constrained-domination principle [12]. Further, from point of view of electronic circuit design, StormOptimus is presented to explore the design space spanned by circuit design parameters to estimate the near optimal solution to the cost function subject to design constraints. The design constraints are handled using the constraint handling strategies introduced inside the proposed framework. A total of six different circuit design examples are considered for optimization using StormOptimus. To demonstrate the effectiveness of StormOptimus, the solutions are compared with other peer algorithms. It is observed that an effective design is obtained for each circuit by using StormOptimus. The rest of the chapter is organized as follows. An overview of problem definition is presented in Sect. 2. Brief details on proposed single objective framework are presented in Sect. 3. In Sect. 4, circuit design problems are analyzed and optimized using the proposed single objective optimization framework. Finally, Sect. 5 summarizes the chapter with future insights.

2 Problem Formulation for Circuit Optimization Any circuit design problem can naturally be expressed as a single objective constrained optimization problem with the circuit response being formulated as the objective, e.g., power consumption, area, DC gain, etc. to be maximized or minimized subject to few design constraints. Mathematically, the design problem can be expressed as follows, maximize f (x) subject to g(x) ≤ 0, x ∈ [xl , xu ],

(1)

where the objective function f (x) is the performance function to be maximized; g(x) represents the inequality constraints, x denotes the vector of design variables, and xl

224

S. Dash et al.

and xu are their lower and upper bounds, respectively. As most of the optimization techniques are developed as minimization techniques, the objective that is to be maximized is converted to a minimization problem by inverting its sign [12].

2.1 Properties of Design Problems In this chapter, several design problems (four analog/RF circuits and one power grid network) are considered for evaluation to demonstrate the applicability of proposed StormOptimus framework. The circuit equations with MOS transistors are abstracted by the square-law yielded posynomial expressions in earlier studies [3, 18]. Although, this yields quick optimization for circuits, the square-law is inadequate to model the transistors with shrinking technology and complicated BSIM models are adopted to model transistors. However, with further decrease in transistor size, even a single BSIM model is insufficient to capture the behaviour of transistor accurately. Therefore, in this chapter, the design problems are formulated in a different manner as described below. As mentioned in (1), the performance function denotes one of the circuit specifications, which is expressed as a function of transistor parameters. The transistor parameters are expressed as function of transistor design variables, such as width, length, gate-source voltage, drain-source voltage, drain current, etc. In such cases, the transistor parameters represent both small-signal and large signal parameters. As a result, the performance function is expressed as a function of both small-signal and large signal parameters. As an example, a single objective circuit design formulation is given in (7) for a two-stage operational amplifier circuit (as shown in Fig. 4). where PM, GBW, SR, and ICMR denote phase margin, gain-bandwidth product, slew rate and input common mode range of the amplifier circuit, respectively; gm2 and gm6 denote the transconductances of transistors M2 and M6, respectively, and gdsi = 1/roi , i = 2, 4, 6, 7, represents the reciprocal of output impedance. Here, gm2 , gm6 and gdsi represent the small-signal parameters and drain current I3 represents large signal parameter. Further, an IBM power grid network (ibmpg2) is considered as a design problem for evaluation. The power grid network is a resistive array arranged in horizontal and vertical cross-sections as shown in Fig. 7a. With increase in the size/area of power grid network, several issues arise, such as IR drop, electromigration, etc, which affect the functionality of the power grid design. Therefore, it is necessary to carefully model each cell (as shown in Fig. 7b) of power grid network to have minimum area. To have a compact and robust power grid design, optimization is needed to minimize the area of each cell subject to various issues. These issues are considered as design constraints during the design of power grid network. The sizes of each cell (width of metal wires) are considered as design variables and the performance function

StormOptimus: A Single Objective Constrained Optimizer …

225

(i.e., area of power grid network) is expressed as a function of widths and lengths of metal wires as shown in (13). A detailed description on design constraints and design variables considered during power grid area optimization is presented in Sect. 4.5.

3 Proposed Single Objective Framework The traditional BSO algorithm is proposed as a swarm intelligence by Shi [23] to solve global optimization problems. Due to the collective behavior of swarm intelligence and data mining, BSO has an inherent capability of employing two contradicting strategies, i.e., efficient exploration of search space by preserving diversity (exploration) and exploitation of solutions found during a number of experiments while solving the same problem instance (exploitation). The working principle of BSO algorithm is based on two basic steps, i.e., clustering and population generation. BSO makes use of k-means cluster algorithm as a clustering technique to group individuals into k clusters and the individual having the best fitness value is regarded as a centroid of the cluster. Depending upon a series of random probabilistic decision-making strategies, an individual is selected from a cluster or two clusters. A new individual is created from the selected individual using mutation operation (either Cauchy or Gaussian mutation). Finally, both the new and selected individuals are compared and the individual having the best fitness value survives to the next generation. The process is continued until a predefined number of generations is reached and the individual having the best fitness value is chosen as the optimal solution. Although a number of efforts are made to solve various real-world problems using BSO algorithm and its variants [13, 28], the clustering and mutation strategies of BSO can be improved by introducing new strategies, like k-means++ and Lévy mutation operator, respectively. Further, to promote broader applications of BSO, we present a constrained single objective framework for optimization of various electronic circuits. In this section, the single objective framework based on brainstorming process, i.e., StormOptimus, is presented while the different steps in the framework are discussed in detail.

3.1 Clustering Clustering is a standard procedure to group similar ideas (individuals) in the same cluster and place dissimilar ideas in different clusters. Clustering differs from classification as each idea in a cluster is closer (more similar) to an object of that cluster. This object is referred as centroid or medoid depending upon the clustering strategy. The traditional BSO algorithm makes use of k-means clustering technique to group individuals into different clusters. However, because of initial random assignments of cluster centroids, k-means clustering technique limits the generation of well-separated centroids and does not ensure a global minimum of variance [15].

226

S. Dash et al.

As the number of clusters is kept fixed during each iteration of k-means clustering, it is possible that distinct individuals (ideas) may be grouped together and the characteristics of these individuals get lost during analysis. Therefore, k-means++ clustering is utilized as an augmentation to k-means algorithm for optimal clustering in the StormOptimus framework. The k-means++ seeding technique avoids additional degrees of freedom by limiting introduction of any new parameters (such as the number of runs for k-means++) in the clustering algorithm. By clustering along distinct ideas (individuals), the technique introduces a measure of distance between cluster centroids before employing the k-means clustering technique for evaluating the minimum distance between cluster centroid and cluster individuals. The following steps are executed while selecting initial centers of clusters using k-means++ [6]. 1. Before starting the process of clustering, one individual is selected as the centroid uniformly at random among the individuals in each cluster. 2. The distance of each individual from the nearest selected center is evaluated using Minkowski metric (Lq-norm, where q = 1, 2) [11]. 3. Another individual is chosen at random as the new centroid using a weighted probability distribution, where each individual is chosen with probability proportional to squared distance. 4. Steps 2 and 3 are repeated until k centroids are selected. 5. Once initial centroids are chosen, standard k-means clustering algorithm can be applied to group the individuals. Considering k-means++ as seeding technique for k-means clustering technique, a number of clusters (two, five, ten and twenty) are generated for 500 random generated individuals as shown in Fig. 1. It can be observed that separate centroids are generated successfully using k-means++ technique, which are considered as initial centroids for standard k-means clustering technique. However, with increase in number of clusters, the convergence of StormOptimus is affected. Details on convergence properties are described in Sect. 3.5.

(a)

(b)

(c)

(d)

Fig. 1 Initial selection of cluster centroids using k-means++ clustering technique after dividing individuals into a two clusters, b five clusters, c ten clusters, d twenty clusters

StormOptimus: A Single Objective Constrained Optimizer …

227

3.2 Population Generation In StormOptimus, the new individual (y) is generated from a selected individual (c or x) of a single cluster or multiple clusters similar to BSO algorithm. The selected individual is chosen based on three probabilities (P2 , P3 and P4 ). Once the individual is selected from one cluster or multiple clusters, new individual (y) is generated using an adaptive Lévy mutation operator as follows, y = x + ξ L α (s),   T −t × rand(), ξ = logsig 2K

(2)

where L α (s) ∼ s −1−α , denotes a random number generated by following Lévy distribution; α denotes a parameter, such that 0 < α < 2 and s denotes the distance parameter [17]. α manages the shape of the distribution, i.e., for α = 1, the Lévy distribution functions as Cauchy distribution and for α = 2, the distribution behaves as Gaussian distribution. Different values of α can be employed to generate different random numbers by dynamically updating (2), which changes the step size frequently and directs the movement of the swarm (of individuals) towards the generation of distinct individuals (preservation of diversity by coming out of local optima). The details of the StormOptimus framework is described in Algorithm 1. Once the new individual is generated, the corresponding fitness value is compared with fitness value of the selected individual and the best fitness value is preserved for next generation. The process is continued until a maximum number of generations are reached or any other stopping criteria are satisfied.

3.3 Handling Constraints For handling both boundary and functional design constraints, two different constraint handling techniques are incorporated into StormOptimus framework. The details on these techniques are given below.

3.3.1

Handling Boundary Constraints

Boundary constraints are utilized during constrained optimization to ensure that the variables (design parameter values) lie inside the bounded region. As StormOptimus makes use of Lévy mutation operator to generate new individual (variable or design parameter), it is likely for variables to be generated outside the bounded region during evaluation. Therefore, to keep the variables within the bounded region, a random value is generated within the boundaries and is replaced with the variable. The random value within the boundary is evaluated as follows [16],

228

S. Dash et al.

xdi = xdl + rand(0, 1) × (xdu − xdl ),

(3)

where xdi denotes the ith variable having dimension d, rand(0, 1) returns a real value between 0 and 1 with uniform distribution, xdl and xdu denote the lower and upper bounds of dth dimension, respectively.

3.3.2

Handling Constraint Functions

Constraint functions, i.e., both inequality and equality constraints represent the design constraints, which are functions of several design parameters. Here, we consider inequality constraints during evaluation of different design examples. For handling inequality constraints, constrained-domination principle [12] is employed to identify and rank feasible solutions according to their nondomination level. The constraineddominance principle is based on the following definition. Definition 1 A solution x is said to constrained-dominate a solution y, if any of the following conditions is true [12]. 1. Solution x is feasible and solution y is not feasible. 2. Both solutions x and y are infeasible, but solution x has a smaller overall constraint violation. 3. Both solutions x and y are feasible and solution x dominates solution y. Usually, the feasible solution has a better nondomination rank than infeasible solution while employing constrained-domination principle. Among feasible solutions, the solution having the best fitness value is regarded as a potential optimal solution, while among infeasible solutions, the solution having fewer constraint violations is marked with a better rank. It is necessary to consider few infeasible solutions during evaluation so that the diversity among swarm population can be increased. Further, as no additional control parameters are involved in the handling of constraint functions, the selection of a potential optimal solution from a pool of feasible solutions becomes elementary and unequivocal.

3.4 Complexity Analysis The framework of StormOptimus starts by generating a random population of N individuals. The individuals are grouped into k clusters with dimension p using kmeans clustering technique as described in Algorithm 1. This requires O(N pk+1 ) number of computations to cluster N individuals into different clusters with an addition of O(logk) number of computations for selecting the initial cluster centroids using k-means++ seeding technique. Once the individuals are clustered into different clusters, the framework requires a minimum of O(N ) number of computations to generate a new offspring, which is followed by selection procedure. Once the

StormOptimus: A Single Objective Constrained Optimizer …

229

Algorithm 1: Procedure for StormOptimus 1: Generate N random ideas (random population individual); 2: F ← f (x);  evaluate fitness for all individuals 3: while stopping criteria do 4: Initialize cluster centroids, c1 , c2 , . . . , ck ∈ R of clusters C1 , C2 , . . . Ck , respectively using k-means++;

5: Cluster variable vector x into k (> 1) clusters;  Clustering analysis starts using k − means 6: p ← 1; 7: while p < N − 1 do 8: for i ← 1 to N do 9: for j ← 1 to k do 10: liq ← arg min ||xi − c j ||2 , q = j, q ∈ k; j

11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: 24: 25: 26: 27: 28: 29: 30: 31: 32: 33: 34: 35: 36: 37: 38: 39: 40: 41: 42: 43: 44: 45:

end for Cq ← Cq ∪ xi ; end for for j ← 1 to k do c j ← mean(C j ) ; end for p ← p+1 end while  Clustering ends using k − means if random(0,1) < P1 then Randomly select a cluster C j , j ∈ [1, k]; c j ← Mutation(c j ); end if for i → 1 to N do if random(0,1) < P2 then Randomly select a cluster C j , j ∈ [1, k]; if random(0,1) < P3 then yi ← Mutation(c j ); else yi ← Mutation(xrandom ), xrandom ∈ C j ; end if else Randomly select two clusters C j and Cl , j, l ∈ [1, k]; if random(0,1) < P4 then cnew ← c j × random(0,1) + (1 − random(0,1)) × cl , where c j ∈ C j and cl ∈ Cl ; yi ← Mutation(cnew ); else xrandom ← x j × random(0,1) + (1 − random(0,1)) × xl , where x j ∈ C j and xl ∈ Cl ; yi ← Mutation(xrandom ); end if end if if f (yi ) < f (c j ) or f (yi ) < f (cnew ) or f (yi ) < f (xrandom ) then c j ← yi or cnew ← yi or xrandom ← yi ; end if end for end while

230

S. Dash et al.

selection is made, the fitness of the best individual is evaluated and the process is continued till stopping criteria is satisfied. In this case, the number of function evaluations (N F E ) is considered as a stopping criterion for the framework. In view of this, the framework of StormOptimus requires a total of O(N F E (N + N pk+1 ) number of computations to evaluate an optimal solution for any given problem. Therefore, the runtime of StormOptimus increases with increase in size of the initial population, however, the framework achieves convergence at minimum function evaluations. Further, with increase in the number of clusters, the runtime is affected with an advantage of providing better solution at fewer function evaluations. Details on the convergence properties of StormOptimus is presented in Sect. 3.5.

3.5 Convergence Properties In this section, the convergence of StormOptimus is analyzed with respect to the number of clusters. In order to evaluate the convergence performance of StormOptimus, 100 random individuals are generated while solving Spher e and Rastrigin functions [1] and the individuals are grouped into different number of clusters, such as, k = 2, 5, 10 and 20. The evolution curves of mean fitness values (over 30 independent runs) of Spher e and Rastrigin functions are shown in Fig. 2. Further, it can be observed from Fig. 2a that StormOptimus can quickly converge toward optima (zero fitness value) within a relatively small number of function evaluations while the individuals are grouped into five, ten and twenty clusters (i.e., k = 5, 10 and 20). Although the start point of fitness value for Spher e function is less while k is set to 2, it is interesting to note that the convergence is extremely slow and the correct optimum is found at 1000 function evaluations. On the other hand, it is observed from Fig. 2b that there appears to be a bigger jump to low fitness value on the evolution curve of Rastrigin function while the individuals are grouped into five clusters (i.e., k = 5). One of the reasons, for this scenario may be due to the property of the Rastrigin function, which is a nonconvex multimodal function having a large search space. Another reason may be due to the incorporation of Lévy mutation operator in StormOptimus framework. This mutation allows the search operator to come out of any local optima by following a large jump.

4 Design Examples In this section, brief details on different circuits are described. To showcase the performance and applicability of StormOptimus framework, three analog circuits (basic cascode amplifier, two-stage operational amplifier and folded cascode amplifier), an RF circuit (low noise amplifier) and a power grid circuit (ibmpg2) are considered as design examples. Various design parameters are optimized in design examples sub-

StormOptimus: A Single Objective Constrained Optimizer … Fig. 2 Evolution of global fitness values (mean) at different number of clusters (k) of a Sphere, b Rastrigin functions at different function evaluations (FEs). Here FE is considered in log scale

231

(a) 0.5

k=2 k=5 k=10 k=20

Fitness value

0.4 0.3 0.2 0.1 0

1

10

100

1000

Number of FEs

(b)

0.2

k=2 k=5 k=10 k=20

Fitness value

0.15

0.1

0.05

0

1

10

100

1000

Number of FEs

ject to design constraints, which are handled using two constraint handling methods (random constraint handling method and constrained-dominance based method) as mentioned in Sect. 3.3.

4.1 Cascode Amplifier A basic cascode amplifier circuit (as shown in Fig. 3) is considered as a design example to maximize DC gain ( Av ). The amplifier is designed in 180 nm CMOS process technology using Cadence Virtuoso. The supply voltage (VDD ) is kept at 1.8 V, and various design constraints, e.g., GBW (≤3.2 MHz), power consumption (≤200 µW), etc. are employed during optimization. The circuit design problem is transformed into a single objective maximization problem as follows, maximize |Av | ≈ gm1 gm2 ro1ro2 ,

(4)

232

S. Dash et al.

Fig. 3 Cascode amplifier circuit

vDD RL vout vbias

M2

vin

M1

where gm1 and gm2 denote the transconductances of transistors M1 and M2, respectively, and ro1 and ro2 represent the output impedances of M1 and M2, respectively.  subject to

Pmin ≤ 200 µW, G BW ≤ 3.2 MHz,

(5)

4.2 Two-Stage Operational Amplifier As operation amplifiers are regarded as the fundamental analysis of analog circuit design, a case study of a two-stage operational amplifier is considered as shown in Fig. 4 to maximize the DC gain (Av ). The two-stage operational amplifier is designed in 0.18 µm CMOS process technology using Cadence Virtuoso. The closed form expressions of Av and different design constraints are approximated as described in [2]. The design problem is transformed into a single objective maximization problem as follows, gm2 gm6 , maximize Av = (6) (gds2 + gds4 )(gds6 + gds7 ) where gm2 and gm6 denote the transconductances of transistors M2 and M6, respectively, and gdsi = 1/roi , i = 2, 4, 6, 7, represents the reciprocal of output impedance,

StormOptimus: A Single Objective Constrained Optimizer …

233

VDD M4

M3

M6 Cc

I2

I1

V_out CL

M1

I3

-

M2

V_in + + V_BIAS

M7

M5

-

VSS

Fig. 4 Two-stage operational amplifier circuit [2]

⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ subject to

⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩

0.48 µm ≤ W2 ≤ 0.9 µm, i ∈ [1, 2], 30.1 µm ≤ W6 ≤ 38.88 µm, 16 µA ≤ I 3 ≤ 20 µA,

(7)

S R = CI3c ≥ 20 A/µF,

G BW = gCm1c ≤ 20 MHz,

where W denotes the width of transistors; S R and G BW represent slew rate, gainbandwidth product, respectively. The details on other design constraints considered during evaluation of two-stage operational amplifier are provided in Table 3. During design, the load capacitance C L is set to 10 pF. A relation between Miller capacitance (Cc ) and load capacitance (Cc > 2.2(gm2 /gm6 )C L ) is maintained in order to ensure the stable operation of the amplifier. The supply voltage (VDD ) is kept fixed at 1.8 V. The width (W ) and length (L) are considered in the range of [0.24, 1000 µm] and [0.18, 10 µm], respectively. The acceptable range of bias current (I3 = I1 + I2 ) is considered between 1 µA and 2.5 mA. For differential operation, the transistor pairs are matched accordingly, e.g., M1 ≡M2, M3 ≡ M4, and M5 ≡ M7.

4.3 Folded Cascode Amplifier As a third design example, a folded cascode amplifier is adopted for design optimization as shown in Fig. 5 to improve the DC gain (Av ) of the amplifier at the expense of several design constraints, e.g., power (PDC ), GBW, etc. The amplifier is frequently

234

S. Dash et al.

Vdd M4

M3 Mbp

M10

I1

M11

vcm M1

vi-

M2

vOUT

I4

vi+

vcm

cL M8

I3

I2

M9

Mbn

M5

M7

M6

Vss Fig. 5 Folded cascode amplifier [2]

used for high gain, better input common-mode range (ICMR) and self compensation. The folded cascode amplifier is designed in 0.18 µm CMOS process technology while setting the supply voltage (VDD ) to 1.8 V. A load capacitance (C L ) of 5 pF is chosen while designing the amplifier. The biasing current (Ibias ), width (W ) and length (L) are considered in the range of [1 µA, 2.5 mA], [0.24 µm, 1000 µm] and [0.18 µm, 10 µm], respectively. In Fig. 5, it can be observed that proper differential operation and current biasing is ensured by the matching relations between transistors, i.e., M1 ≡ M2, M3 ≡ M4, and M5 ≡ M7. The analytical problem formulation of the amplifier is described in (8) as follows, maximize

Av = gm2 [gm8rds8 rds6  (gm10 rds10 (ro2  ro10 ))],

(8)

where gm2 , gm8 and gm10 denote the transconductances of transistors M2, M8 and M10, respectively, and ro2 , ro6 , ro8 , ro10 represent the output impedances of transistors M2, M6, M8 and M10, respectively, ⎧ ⎨ subject to

⎩



S R = ICbiasL ≥ 1 V/µS,

G BW = gCm2L ≥ 2 MHz,

(9)

where S R, G BW , C L represent slew rate, gain bandwidth product and load capacitance, respectively. The details on other design constraints and their ranges considered during evaluation of folded cascode amplifier are provided in Table 4.

StormOptimus: A Single Objective Constrained Optimizer …

235

Fig. 6 Two-stage operational amplifier circuit [2]

4.4 Low Noise Amplifier As a fourth design example, a source degenerate cascode low noise amplifier (LNA) circuit [5] is considered (as shown in Fig. 6) to demonstrate the applicability of StormOptimus in minimizing noise figure of LNA. It can be observed from Fig. 6 that a common source amplifier having inductive source degeneration is connected with a common gate load, while the operating frequency of the circuit is 2.4 GHz. The quality factor (Q) of LNA is considered between 3 and 5 at an expense of 8mW power consumption (Pmax ). As the gain is crucial to the design process in LNA, the noise figure (NF) is maintained at a satisfactory level of 2.5 dB. Further minimization of NF can be achieved by sacrificing gain during the design process. However, it may result in transmission of noise to other blocks of the receiver system. Therefore, the LNA design problem is transformed into a single objective minimization problem with an objective to minimize NF subject to design constraints as follows, minimize N F = 1 +

(A + B + C + D) , E

where, A = (1/4)γ gdo , B = gm2 (C gs /Ctot )2 (Q 2 + 1/4)β/5gdo ,  C = gm c(C gs /Ctot ) (γ · β)/20, D = 1/Rout , E = gm2 Rs Q 2 ,

(10)

236

S. Dash et al.

where γ denotes the white noise factor and is approximately equal to 1.05 for 180 nm CMOS process technology; β represents the gate noise parameter and is approximately equal to 3.8; c expresses the correlation coefficient; C gs denotes intrinsic gate capacitance; Ctot represents the sum of C gs , Cd (=0.08 pF) and parasitic capacitance, and Q denotes the quality factor of the input circuit. Approximated expressions of gm and gdo are generated by curve-fitting [7], and they are expressed as follows, gm = A0 L A1 W A2 IdsA3 , B3 , gdo = B0 L B1 W B2 Ids

(11)

where A0 , A1 , A2 and A3 denote constants, whose values are equal to 0.0463, −0.4489, 0.5311 and 0.4689, respectively; B0 , B1 , B2 and B3 also represent constants, whose values are equal to 0.0096, −0.5595, 0.5194 and 0.4806, respectively. ⎧ ⎪ ⎪ ⎪ ⎨ subject to

⎪ ⎪ ⎪ ⎩

1 µm ≤ Wi ≤ 100 µm, i ∈ [1, 2], 0.1 mA ≤ I1 ≤ 4.5 mA, N Fmin ≤ 2.5 dB, Pmax ≤ 8 mW,

(12)

where N Fmin , Pmax , W represent minimum NF, maximum power consumption and width of transistors, respectively, and I1 denotes current across transistor M1 of LNA. The details on other design constraints and their ranges considered during evaluation of LNA are provided in Table 5.

4.5 Power Grid Network Power grid networks are metallic grids that help in distributing power to different functional blocks of an integrated circuit. Generally, designers over-design a power grid network by extending their margins and routability. Due to this over-designing of power grid network, the overall power distribution area increases and it becomes too congested for placement of other functional blocks inside the integrated circuit and internal routing. This requires manual efforts to slice the metal area (wire area) of power grid network to increase the design space for placement of additional system components. Therefore, it is necessary to minimize the metal area (wire area) of the entire power grid network by transforming the problem into a single objective constrained minimization problem subject to various constraints, such as voltage drop (IR drop), electromigration, etc. As a design example, an IBM power grid circuit (ibmpg2) [20] is considered to demonstrate the applicability and performance assessment of StormOptimus. The IBM power grid circuits are considered as standard benchmarks in literature. The problem of minimization of wire area on a power grid network (as shown in Fig. 7a)

StormOptimus: A Single Objective Constrained Optimizer …

(a)

237

(b)

2

Vdd

g2 1

g1

x

g4

g3

3

Ix

Vdd

4

Fig. 7 a A resistive model of power grid network [33], b a single cell of power grid network

can be formulated as follows [29], minimize Area =



l x y wx y ,

(x,y)∈E x y

=



ρl x2y gx y , gx y = wx y /ρl x y ,

(x,y)∈E x y

⎧ ρl x y gx y ≥ wmin , ⎪ ⎪ ⎪ ix y ⎨ (x,y) wx y ≤ Jmax , subject to

⎪ ⎪ (x,y) (v y − vx )gx y ≥ i x , ⎪ ⎩ Vdd − vx ≤ Vth , ∀x ∈ N ,

(13)

where ρ represents sheet resistance; l x y , wx y , gx y and i x y denote length, width, conductance of and current in any branch (x, y), respectively; E x y denotes the set of indexes of all branches in PDN; wmin denotes the minimum metal width of any branch; Jmax denotes the maximum current density; vx denotes the voltage at node x; Vth denotes the threshold voltage and N is the number of nodes in power grid network.

4.6 Performance Evaluation To demonstrate the applicability, StormOptimus is employed to optimize five different electronic circuits. Further, the performance of StormOptimus is compared with standard particle swarm optimization (SPSO) [8], differential evolution (DE) [21], and river formation dynamics (RFD) [10] methods while optimizing the circuits. Authors’ implementations of all algorithms are considered during evaluation and comparison of solutions. A total of 25 independent runs are carried out to reduce

238

S. Dash et al.

Table 1 Control parameters of DE, SPSO, RFD, StormOptimus Parameters DE SPSO Population size (N ) 100 FEmax /Max. iterations 2e+05 Mutation, mutation rate D E/curr ent-tobest/1, – Crossover, crossover rand/1/ex p, 0.9 probability c1 , c2 , vmax , vmin – w max , w min /w – Cr , F 0.9, 0.8 Erosion parameter () – Probability – (P1 , P2 , P3 , P4 ) k, µ, σ, α –

RFD

StormOptimus

100 2e+05 –

100 2e+05 –

100 2e+05 –

–

–

–

1.4, 1.4, −5, 5 1.0, 0.4/– – – –

– – – 0.1 –

– – – – 0.2, 0.8, 0.4, 0.5

–

–

5, 0, 1, (0, 2)

the amount of statistical error and the mean results are considered for comparison. The details on different control parameters and corresponding values are listed in Table 1. The problem of minimizing the DC gain of basic cascode amplifier is based on modifying the transconductances (gm1 , gm2 ) of transistors subject to power consumption and GBW. Both the transconductances are considered as design variables during optimization. Several experiments are performed on the basic cascode amplifier to maximize DC gain. A maximum DC gain of 18.3 dB has been achieved while using StormOptimus subject to power consumption of 200 µW and GBW of 2.9 MHz (refer Table 2). Further, it can be observed from (6) that DC gain of a two-stage operational amplifier is function of transconductances (gm2 and gm6 ) and output resistances. During optimization, both gm2 and gm6 are considered as design variables and their limits are imposed by evaluating all design variables from design constraints. Several experiments are performed on the two-stage operational amplifier to maximize DC gain and the results of different specifications are listed in Table 3. It can be observed from Table 3 that all algorithms are able to find feasible solutions and a satisfactory gain of 68.03 dB has been achieved using StormOptimus method. During optimization of folded cascode amplifier, gm2 , gm8 and gm10 are adopted as design variables to maximize DC gain at the expense of a maximum power consumption (Pmin ) of 250 µW and a minimum GBW of 2 MHz. The bias current (Ibias ) is assumed as, I1 = I2 = Ibias and I3 = I4 = 0.7Ibias , while the operating conditions of transistors are ensured to satisfy the matching relations during evaluation. It can be observed from Table 4 that a DC gain of 58.19 dB has been achieved using StormOptimus method while satisfying other design constraints. It can also be observed that other peer algorithms are able to generate a feasible solution. However, StormOptimus has shown better performance in terms of achieving DC gain at an expense of minimum power consumption of 200 µW and a GBW of 2.3 MHz.

StormOptimus: A Single Objective Constrained Optimizer …

239

Table 2 Design specifications and results of DE, SPSO, RFD and StormOptimus on a basic cascode amplifier Specifications Constraints DE SPSO RFD StormOptimus DC gain, Av (dB) Pmin (µW) GBW (MHz) M2 M1

maximize ≤200 ≤3.2 Saturation Saturation

18.00 210 3 Met Met

17.80 200 3 Met Met

17.10 200 2.8 Met Met

18.30 200 2.9 Met Met

Table 3 Design specifications and results of DE, SPSO, RFD and StormOptimus on two-stage operational amplifier Specifications Constraints DE SPSO RFD StormOptimus DC gain, Av (dB) Phase margin, φ (degree) Pmin (µW) Slew rate, S R (A/µF) ICMR (V) GBW (MHz)  Vinmin − Vss − Iβ5 − VTmax 1 (mV) M2 M4 M6

Maximize ≥60 ≤300 ≥20 [0.8, 1.6] ≤30

67.00 62 300 21 0.9 30

69.80 62 300 21 0.9 30

67.21 63 280 20 0.95 29

68.03 63 280 21 1.0 29

≥100

100

100

102

105

Saturation Saturation Saturation

Met Met Met

Met Met Met

Met Met Met

Met Met Met

Table 4 Design specifications and results of DE, SPSO, RFD and StormOptimus on folded cascode amplifier Specifications Constraints DE SPSO RFD StormOptimus DC gain, Av (dB) GBW (MHz) Pmin (µW) Phase margin, φ (degree) Output swing (V)

V Slew rate, S R µS

maximize ≥2 ≤250 ≥60 ≥1.2

55.23 2.02 220 60 1.2

56.42 2.15 200 65 2.2

56.02 2.02 220 65 1.2

58.19 2.30 200 65 2.4

≥1

1.4

1.1

1.3

1.1

M2 M6 M8 M10

Saturation Saturation Saturation Saturation

Met Met Met Met

Met Met Met Met

Met Met Met Met

Met Met Met Met

240

S. Dash et al.

Table 5 Design specifications and results of DE, SPSO, RFD and StormOptimus on LNA Specifications

Constraints

DE

SPSO

RFD

StormOptimus

Noise figure, N F (dB)

Minimize

0.87

0.80

0.71

0.68

Power gain, Av or S21 (dB)

≥10

15.20

13.80

14.81

13.02

Input reflection coefficient, S11 (dB) ≤−20

−20.02

−20.03

−20.02

−20.11

≤−50

−53.22

−54.03

−52.01

−54.10

Reverse isolation, S12 (dB)

Output reflection coefficient, S22 (dB) ≤−0.9

−2.01

−1.90

−2.00

−1.88

Min. power consumption, Pmin (mW) ≤8

8

7.80

7.80

8

Center frequency, f c f (GHz)

[2.1, 2.7]

2.4

2.4

2.4

2.4

Capacitance ratioa , C gs /Ctot

≤1

0.92

1

0.92

1

Quality factor, Q

[3, 5]

3.5

3.5

3.5

3.5

min ( ) Z out

≥50

500

500

500

500

M2

Saturation

Met

Met

Met

Met

M6

Saturation

Met

Met

Met

Met

M8

Saturation

Met

Met

Met

Met

M10

Saturation

Met

Met

Met

Met

a Ratio

of intrinsic gate capacitance of transistor M1 to total capacitance (Ctot = C gs + Cd + C p )

Further, three different design variables, such as W M1 , I1 and Cd are considered for evaluation of minimum NF in LNA. Since the objective of this experiment is to minimize NF, the gain is maintained above 10 dB for satisfactory performance and reduce the amount of any transmission of noise to the receiver system. It can be observed from Table 5 that NFs of 0.87 dB, 0.80 dB, 0.71 dB, 0.68 dB are achieved while maintaining satisfactory gains of 15.20 dB, 13.80 dB, 14.81 dB and 13.02 dB using DE, SPSO, RFD and StormOptimus algorithms, respectively. It can also be observed from Table 5 that all algorithms are able to satisfy the design constraints, such as S-parameters, Q, Pmin , capacitance ratio and NF, etc. at 2.4 GHz center frequency. Finally, StormOptimus is employed to minimize the wire area of ibmpg2 power grid benchmark. Before the optimization process, the entire power grid network is analyzed to determine IR drop across the network using RFD method [9]. Later, the areas of a single cell are considered for minimization using StormOptimus considering the widths and currents as design parameters. As each node in a power grid network is connected to a number of nodes, modifying the metal width of any branch (e.g., wx y , where y = 1, 2, 3, 4 as shown in Fig. 7b) affects the IR drop and electromigration constraints. Therefore, the entire power grid network is analyzed by dividing it into several cells as shown in Fig. 7b and the wire widths of a cell are altered to minimize the cell area. Each cell is optimized for minimum area and the overall area is evaluated by calculating the minimum areas of all cells. Table 6 lists the results of the percentage of reduction in area and the percentage of nodes below the threshold (Vth ) for ibmpg2 after minimization. The wire area is minimized by evaluating (13) while assigning metal sheet resistance (ρ) of

StormOptimus: A Single Objective Constrained Optimizer …

241

Table 6 Design specifications and results of DE, SPSO, RFD and StormOptimus on ibmpg2 power grid network Specifications Constraints DE SPSO RFD StormOptimus Area (% decrease) Nodes below threshold (% decrease) Avg. wmin (µm)

Minimize ≤5 ≤10

1.62 5.21 9.8

1.87 5.00 9.1

2.02 4.38 9.0

2.15 4.00 8.2

0.01 / and wire length (l) of 0.01 µm for all branches of the power grid network. It can be observed from Table 6 that a reduction of 1.62%, 1.87%, 2.02% and 2.15% wire area is achieved, and the percentage of affected nodes below threshold is reduced to 5.21%, 5.00%, 4.38% and 4.00% for ibmpg2 after optimization using DE, SPSO, RFD and StormOptimus algorithms, respectively. After optimization of five design examples, it can be seen that StormOptimus shows competitive performance in terms of achieving better solutions than other peer algorithms. This may be due to the inherent collective behavior of swarm intelligence and data mining process, which helps in achieving better solutions by avoiding premature convergence. As the optimization of StormOptimus is iterative, it demands extensive simulations to achieve diversity. Further, the incorporation of Lévy mutation operator ensures the generation of random distinct individuals within the feasible space, which preserves the diversity in each generation.

5 Summary This chapter presents a constrained optimization framework based on brainstorming process, i.e., StormOptimus and its application in the optimization of electronic circuits. Various improvements are incorporated both in clustering step (i.e., kmeans++) and population generation step (adaptive Lévy mutation operator) of BSO algorithm to improve the performance and applicability. Five different design examples are considered for optimization of various design specifications subject to design constraints using StormOptimus method. Several experiments are performed on these design examples and it is observed that StormOptimus shows competitive results as compared to other peer algorithms considered in this chapter. In view of this, brainstorming based optimization techniques can become one of the preferred choices of algorithms in practice. However, several variations can be incorporated to improve the overall performance of the StormOptimus method. The effectiveness and stability of the algorithm need to be observed while solving more complex problems, such as multimodal, multi-objective and many objective problems. Although with more number of clusters, the StormOptimus finds the optima at a reasonable number of function evaluations, the performance of StormOptimus is needed to be seen while solving a complex large scale variable function. Further, the random probabilistic

242

S. Dash et al.

decision-making strategy of StormOptimus can be replaced by introducing efficient Bayesian decision theory or a statistical measure that determines the selection of best individuals from a single or multiple clusters.

References 1. Ali, M.M., Khompatraporn, C., Zabinsky, Z.B.: A numerical evaluation of several stochastic algorithms on selected continuous global optimization test problems. J. Glob. Optim. 31(4), 635–672 (2005) 2. Allen, P.E., Holberg, D.R.: CMOS Analog Circuit Design. Oxford University Press (2002) 3. Alpaydin, G., Balkir, S., Dundar, G.: An evolutionary approach to automatic synthesis of highperformance analog integrated circuits. IEEE Trans. Evol. Comput. 7(3), 240–252 (2003) 4. Altay, E.V., Alatas, B.: Performance comparisons of socially inspired metaheuristic algorithms on unconstrained global optimization. In: Advances in Computer Communication and Computational Sciences, pp. 163–175. Springer (2019) 5. Andreani, P., Sjoland, H.: Noise optimization of an inductively degenerated cmos low noise amplifier. IEEE Trans. Circuits Syst. II: Analog. Digit. Signal Process. 48(9), 835–841 (2001) 6. Arthur, D., Vassilvitskii, S.: k-means++: the advantages of careful seeding. In: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1027–1035. Society for Industrial and Applied Mathematics (2007) 7. Boyd, S.P., Lee, T.H., et al.: Optimal design of a cmos op-amp via geometric programming. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst. 20(1), 1–21 (2001) 8. Bratton, D., Kennedy, J.: Defining a standard for particle swarm optimization. In: 2007 IEEE Swarm Intelligence Symposium (SIS 2007), pp. 120–127. IEEE (2007) 9. Dash, S., Baishnab, K.L., Trivedi, G.: Applying river formation dynamics to analyze vlsi power grid networks. In: 2016 29th International Conference on VLSI Design and 2016 15th International Conference on Embedded Systems (VLSID), pp. 258–263. IEEE (2016) 10. Dash, S., Joshi, D., Trivedi, G.: Cmos analog circuit optimization via river formation dynamics. In: 2016 26th International Conference Radioelektronika (Radioelektronika), pp. 51–55. IEEE (2016) 11. De Amorim, R.C., Mirkin, B.: Minkowski metric, feature weighting and anomalous cluster initializing in k-means clustering. Pattern Recognit. 45(3), 1061–1075 (2012) 12. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002) 13. Duan, H., Li, S., Shi, Y.: Predator-prey brain storm optimization for dc brushless motor. IEEE Trans. Magn. 49(10), 5336–5340 (2013) 14. Harjani, R., Rutenbar, R.A., Carley, L.R.: Oasys: a framework for analog circuit synthesis. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst. 8(12), 1247–1266 (1989) 15. Huang, Z.: Extensions to the k-means algorithm for clustering large data sets with categorical values. Data Min. Knowl. Discov. 2(3), 283–304 (1998) 16. Lampinen, J.: A constraint handling approach for the differential evolution algorithm. In: Proceedings of the 2002 Congress on Evolutionary Computation, 2002, CEC’02, vol. 2, pp. 1468–1473. IEEE (2002) 17. Lee, C.Y., Yao, X.: Evolutionary programming using mutations based on the lévy probability distribution. IEEE Trans. Evol. Comput. 8(1), 1–13 (2004) 18. Liu, H., Singhee, A., Rutenbar, R.A., Carley, L.R.: Remembrance of circuits past: macromodeling by data mining in large analog design spaces. In: Proceedings of the 39th Annual Design Automation Conference, pp. 437–442. ACM (2002) 19. Moore, G.E.: Cramming more components onto integrated circuits, reprinted from Electronics 38(8), April 19, 1965, pp. 114 ff. IEEE Solid-State Circuits Soc. Newsl. 20(3), 33–35 (2006)

StormOptimus: A Single Objective Constrained Optimizer …

243

20. Nassif, S.: Power grid analysis benchmarks. In: Asia and South Pacific Design Automation Conference (ASPDAC 2008), pp. 376–381, March 2008 21. Price, K., Storn, R.M., Lampinen, J.A.: Differential Evolution: A Practical Approach to Global Optimization. Springer Science & Business Media (2006) 22. Qiu, H., Duan, H., Shi, Y.: A decoupling receding horizon search approach to agent routing and optical sensor tasking based on brain storm optimization. Opt.-Int. J. Light. Electron Opt. 126(7), 690–696 (2015) 23. Shi, Y.: Brain storm optimization algorithm. In: International Conference in Swarm Intelligence, pp. 303–309. Springer (2011) 24. Shi, Y.: Brain storm optimization algorithm in objective space. In: 2015 IEEE Congress on Evolutionary Computation (CEC), pp. 1227–1234. IEEE (2015) 25. Shi, Y.: Unified swarm intelligence algorithms. In: Critical Developments and Applications of Swarm Intelligence, pp. 1–26. IGI Global (2018) 26. Slowik, A., Kwasnicka, H.: Nature inspired methods and their industry applications-swarm intelligence algorithms. IEEE Trans. Ind. Inform. 14(3), 1004–1015 (2018) 27. Sörensen, K., Sevaux, M., Glover, F.: A history of metaheuristics. In: Handbook of Heuristics, pp. 1–18 (2018) 28. Sun, C., Duan, H., Shi, Y.: Optimal satellite formation reconfiguration based on closed-loop brain storm optimization. IEEE Comput. Intell. Mag. 8(4), 39–51 (2013) 29. Tan, S.X.D., Shi, C.J.R., Lee, J.C.: Reliability-constrained area optimization of vlsi power/ground networks via sequence of linear programmings. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst. 22(12), 1678–1684 (2003) 30. Tuba, E., Strumberger, I., Zivkovic, D., Bacanin, N., Tuba, M.: Mobile robot path planning by improved brain storm optimization algorithm. In: 2018 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8. IEEE (2018) 31. Xiong, G., Shi, D.: Hybrid biogeography-based optimization with brain storm optimization for non-convex dynamic economic dispatch with valve-point effects. Energy (2018) 32. Xiong, G., Shi, D., Zhang, J., Zhang, Y.: A binary coded brain storm optimization for fault section diagnosis of power systems. Electr. Power Syst. Res. 163, 441–451 (2018) 33. Yu, T., Wong, M.: Pgt solver: an efficient solver for power grid transient analysis. In: ICCAD (2012) 34. Zhan, Z.H., Zhang, J., Shi, Y.H., Liu, H.l.: A modified brain storm optimization. In: 2012 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8. IEEE (2012) 35. Zhou, D., Shi, Y., Cheng, S.: Brain storm optimization algorithm with modified step-size and individual generation. In: Advances in Swarm Intelligence, pp. 243–252 (2012)

Brain Storm Optimization Algorithms for Flexible Job Shop Scheduling Problem Xiuli Wu

Abstract Scheduling is to determine when and who to process the task in order to minimize the makespan, the cost, and the tardiness/earliness of jobs, etc. Evolutionary algorithms (EAs) are very promising approaches for the scheduling problems due to its dynamic characteristics, multiple contradicting objectives and highly nonlinear constraints. Brain storm optimization (BSO) algorithm and its variations are new emerging evolutionary algorithms. In this chapter, our focus will be on BSOs for the flexible job shop scheduling problems (FJSP). First, FJSP will be formulated as a combinatorial optimization problem. Second, approaches for FJSP will be summarized briefly. Third, BSOs will be introduced and the key issues in the application of BSOs for FJSP will be emphasized. Fourth, four BSOs will be developed to solve FJSP and the details designed for FJSP will be illustrated. Their strength and weakness will be discussed as well. Finally, a group of experiments will be conducted to compare the four BSO algorithms. The results will be analyzed statistically. Finally, a conclusion will be summarized. Keywords Brain storm optimization · Flexible job shop scheduling problems · Scheduling

1 Flexible Job Shop Scheduling Efficiently production scheduling is vital for manufacturers, especially in today’s constantly changing environment. It has therefore attracted considerable research (see [10, 23, 27, 29], for example). According to the production features, the production scheduling can be modeled as single machine scheduling problem, parallel machine scheduling problem, flow shop scheduling problem (FSP), job shop scheduling problem (JSP) and open shop scheduling problem [15]. In the recent decades, X. Wu (B) Department of Logistic Engineering, School of Mechanical Engineering, University of Science and Technology Beijing, Beijing 100083, China e-mail: [email protected] © Springer Nature Switzerland AG 2019 S. Cheng and Y. Shi (eds.), Brain Storm Optimization Algorithms, Adaptation, Learning, and Optimization 23, https://doi.org/10.1007/978-3-030-15070-9_10

245

246

X. Wu

many factories have transferred to flexible manufacturing system (FMS) to respond to the changing market. Bruker et al. modeled the scheduling problem in FMS as the flexible job shop scheduling problem (FJSP). Compared with the classical JSP, FJSP has two challenges: to assign each operation to an appropriate machine and further to schedule the assigned operations on each machine. FJSP is an extension of JSP because it is closer to the production practice. FJSP is also a typical combinational optimization problem and has been proved to be a strongly NP-hard problem [1]. The FJSP can be described as follows. n jobs indexed by J = (J 1 , J 2 , …, J n ) need to be processed on m machines. Job J j is composed of nj operations which are to be processed in a pre-specified process sequence. More than one machine (defined as Sij ) is available for each operation Oij , which indicates the available machines for the ith operation of job J j . If machine k is selected to process operation Oij , pijk units time will be occupied on machine k. The task for the FJSP is to assign operations to machines in an appropriate way first and then to schedule the operations to optimize some objectives. Makespan C max , the maximum completion time of all the operations C ij , is a usually optimization objective. The maximal machine load and the total machine load are also the optimization objectives. In general, some assumptions are given as following. (1) (2) (3) (4) (5) (6) (7)

All jobs and machines are available at time 0. Jobs have no associated priorities. At a time point, a machine cannot perform more than one operation. The value pijk is given in advance. Setup time for each operation is negligible. An operation must be completed without interruption once started. Jobs are available for processing on a next machine immediately after completing processing on the previous machine.

Before we give the formulation of the FJSP, the notations for the variables are listed in Table 1 for clearness. FJSP can be formulated as following. The objective:    minCmax = min max Cij

(1)

s.t. Cij − Ci(j−1) ≥ pijk Xijk ,

j = 2, 3, . . . , ni

(2)

     Yhgij  Yhgij Cij − Chg − pijk Xhgk Xijk Yhgij − 1 + Chg − Cij − phgk Xhgk Xijk 2 2   (3) Yhgij + 1 ≥ 0, ∀(h, g), (i, j), k 

Xijk = 1, k ∈ Sij , ∀i, j

(4)

Brain Storm Optimization Algorithms … Table 1 Notations

247

Variables

Descriptions

j, g

The index of a job, where j, g = 1, 2, …, n

i, h

The index of an operation, where i, h = 1, 2, …, J j

k

The index of a machine, k = 1, 2, …, m

l

The index of a PM activity on machine k, l = 1, 2, …, l k

n

The total number of jobs

m

The total number of machines

Mk

The Machine index, k = 1, 2, …, m

Jj

The total number of operations of job j

Oij

The ith operation of job j

Oijk

The event that Oij is processed on machine k

pijk

The processing time of Oijk

C max

The makespan of a scheduling solution

Z

An arbitrary large positive number

X ijk

X ijk = 1 if machine k is selected for operation Oij , otherwise X ijk = 0

C ij

The completion time of operation Oij

yhgij

yhgij = 1 if Ohg is scheduled before Oij , otherwise yhgij = 0

S ij

The set of available machines for the operation Oij

Xijk ∈ {0, 1}

(5)

Yhgij ∈ {−1, 0, 1}

(6)

Equation (1) is the objective function. Inequality (2) is the precedence constraint, which ensures that the predefined operation sequence of all jobs would be considered, where X ijk is a binary variable. X ijk = 1 if machine k is selected for operation Oij , otherwise X ijk = 0. Inequality (3) is the operation constraints, which ensure that there are no overlaps between operations on one machine. Equations (4)–(6) are constraints on the values of the decision variables. Table 2 shows an instance of the FJSP. The number in each entry is pijk . If pijk = 0, it means the machine is not available for Oij .

2 Approaches for FJSP Compared with the study on JSP, the studies on FJSP have received less attention. Brucker and Schlie [2] developed a polynomial algorithm for solving a FJSP with two jobs, which is the first study on FJSP. Since then, the interest on the FJSP has

248

X. Wu

Table 2 An instance of FJSP J1

J2

J3

M1

M2

M3

O11

2

3

0

O21

0

5

2

O31

3

6

4

O12

0

4

5

O22

5

5

6

O32

2

4

8

O13

4

0

6

O23

4

4

4

O33

5

6

7

increased greatly. From the viewpoint of the optimization objective, the studies can be categorized into two groups: one is to aim at minimizing makespan only and the other is to aim at minimizing the makespan, the total workload, and the maximal workload. Below we will review the approaches in these two groups.

2.1 Studies on FJSP for Minimizing Makespan Only Studies on FJSP in this group focused on a single objective, i.e. makespan. Due to its strong NP-hard feature [1], it is natural to look for heuristics and meta-heuristics to search for an approximately optimal solution. The existing heuristic and metaheuristics for solving FJSP can be further classified into two main categories according to the sequence of solving the two sub-problems. The first is the hierarchical approaches, which assign operations to machines first and then optimize the sequence of the assigned operations on the machines separately. The second is the integrated approaches, which combine the two steps together and solve the two sub-problems simultaneously. The hierarchical approach could alleviate the difficulty of FJSP by decomposing a FJSP into a sequence of two sub-problems. Hence, in the early years, researchers usually employed this hierarchical strategy. For example, Brandimarte [1] was the first to apply a hierarchical tabu search algorithm to solve the FJSP. Their emphasis was on minimizing makespan, and the adaptability of the proposed approach to minimize the weighted tardiness was discussed as well. Ho and Tay [10] employed the dispatching rules and genetic algorithm to solve the two sub-problems, respectively. Although the hierarchical approach is easy to develop, the results are not always optimal especially for those large scale instances. Therefore, researchers considered to develop some integrated approaches. The integrated approaches are much more difficult to design than the hierarchical approach, but they generally achieved better results [5]. Hence, recent studies on FJSP mostly focused on the integrated approach. For example, Mati et al. [16] proposed a greedy algorithm to solve the FJSP. Pezzella

Brain Storm Optimization Algorithms …

249

et al. [19] proposed a genetic algorithm for the FJSP. Hmida et al. [9] presented a variant of the climbing discrepancy search approach. Xing et al. [30] developed a knowledge-based ant colony optimization algorithm. Yazdani et al. [32] proposed a parallel variable neighborhood search algorithm with six neighborhood structures. Wang et al. [25] presented an effective bi-population based estimation-of-distribution algorithm to solve the FJSP to minimize the makespan. Yuan et al. [33] proposed a hybrid differential evolution algorithm for solving the FJSP. Singh et al. [22] proposed a quantum-behaved particle swarm optimization to address the inherent drawback of getting trapped at the local optimum of the particle swarm optimization. Li et al. [13] proposed an effective hybrid algorithm which hybridized the genetic algorithm and tabu search (TS) for the FJSP with the objective to minimize the makespan. Wu et al. [27] presented an elitist quantum-inspired evolutionary algorithm for solving the FSJP to minimize the makespan objective.

2.2 Studies on FJSP for Minimizing Multiple Objectives Considering that not only one machine is available for one operation, the maximal load and the total machine load have been taken as the optimization objectives as well as makspan. Following a hierarchical approach and aiming to optimize the three objectives, Kacem et al. [11] adopted a local search to assign machines and employed the genetic algorithm to sequence the operations on the assigned machines. Similarly, Xia and Wu [29] employed particle swarm optimization algorithm to solve machine assignment problem and adopted the simulated annealing algorithm to solve operation scheduling problem. Chen et al. [4] developed a scheduling algorithm based on grouping GA. Their algorithm consisted of two modules: machine selection module and operation scheduling module, the first of which helped an operation to select one of the parallel machines to process it and the second of which was then used to arrange the sequences of all operations assigned to each machine. As to the integrated approach, Loukil et al. [14] utilized the simulated annealing algorithm to solve the FJSP. Gao et al. [6] proposed a genetic algorithm hybridizing with the variable neighborhood search. Li et al. [12] proposed an effective discrete chemicalreaction optimization algorithm for solving the FJSP subject to maintenance activity constraints. Recently, with the increasing concern on the energy efficiency of industries, the energy consumption, carbon emission, and other related objectives have also been considered to be one of the objectives when solving the FJSP. For example, Mokhtari et al. [17] developed a multi-objective optimization model with three objectives: (i) minimizing the makespan, (ii) maximizing the total availability of the system, and (iii) minimizing the total energy cost of both production and maintenance activities in the FJSP. They developed an enhanced evolutionary algorithm incorporated with the global criterion. Wu et al. [26] considered when to turn-on/off machines and which speed level to choose in order to save energy in solving FJSP. They employed a fast and elitist non-dominated sorted genetic algorithm to balance the makespan, the

250

X. Wu

energy consumption and the turning-on/off times of machines. Piroozfard et al. [20] presented a multi-objective FJSP to minimize the total carbon footprint and the total late work criterion. They proposed an improved multi-objective genetic algorithm to solve the presented bi-objective optimization problem. Gong et al. [7] studied a double flexible job-shop scheduling problem, in which both workers and machines are flexible and they proposed new hybrid genetic algorithm. In conclusion, due to its vast application prospects and the strong NP-hard complexity, more and more researchers have come to focus on FJSP with more and more constraints and objectives under consideration.

3 Introduction to Brain Storm Optimization Algorithms Brainstorm meeting is an effective way to help us to interactively collaborate to generate solutions for difficult problems. When we face a difficult problem, it is usually difficult for a single person to solve it by himself or herself. If we could organize a group of experts with different backgrounds and knowledge together, a brainstorming meeting could be organized. In the meeting, each expert proposes a solution for the problem first and then they discuss with the others deeply in order to make a commonly agreed solution. Usually, this interactive collaboration of human beings could bring us great and amazing intelligence and further generate a satisfied solution. Usually there are three roles in a brainstorming meeting: a meeting organizer, a brainstorming experts group, and the problem owners. The organizer is to organize the meeting and enforce the brainstorming group to obey the Osborn’s original four rules (Table 3) for generating solutions in a brainstorming process. Note that the organizer is neutral and should not be involved in the process of generating solutions. The brainstorming experts are in charge of generating new solutions according to their backgrounds, knowledge, experience and the way of thinking and the rules, which ensures that the experts generate varied solutions. The role of the problem owners is to choose the better solutions in the existing solutions and encourage the experts to generate a better solution based on the existing better solutions. The rules in Table 3 are to ensure that the brainstorm experts remain sufficiently open and divergent to generate as many as possible diverse solutions, so as not to fall into the existing solutions and cannot jump out of the mindset. Using the rules in Table 3 and the brainstorming process, the experts could effectively carry out divergent thinking to generate a greater variety of solutions. After several iterations, the best solution will be obtained.

Brain Storm Optimization Algorithms …

251

Table 3 Osborn’s original four rules for generating solutions Rule no.

Rule name

Descriptions

Rule 1

Suspend judgment

All solutions are good solutions

Rule 2

Anything goes

All solutions should be recorded and shared

Rule 3

Crossfertilize

All solutions should be based on the existing solutions, each of which should generate more solutions

Rule 4

Go for quantity

Try to generate more new solutions

4 Four BSOs for FJSP Based on the brainstorming process, the brain storm optimization algorithm and its varieties have been proposed. Up to now, in the BSO family, there are the basic BSO algorithm [21], the modified BSO (MBSO) algorithm [35], the discussion mechanism based BSO (DMBSO) algorithm [31] and the population diversity and discussion mechanism based BSO (PD-DMBSO) algorithm [28]. The details of each algorithm are described as following.

4.1 Basic BSO Algorithm The basic BSO algorithm is the original BSO algorithm proposed by Shi [21]. It consists of the initialization, clustering, generating new individuals and the terminal condition. First, a group of initial solutions are generated in the initialization process. Then the solutions are classified by clustering, based on which some new solutions are generated. This process iterates until the terminal condition is satisfied. Its pseudo code is listed in Table 4. In BSO algorithm, clustering and generating new individuals are two key steps. In the clustering process, individuals are classified into a cluster according to their fitness, and the best one is considered as the cluster center. The parameter Pa is the disturbance intensity and determines the probability of changing the cluster centers. If Pa is greater, the probability of directly changing the cluster center is greater, the disturbance to the population is stronger. The parameter Pb determines whether a cluster or two clusters is chosen to generate new individuals. If Pb is greater, the probability to select a cluster is bigger, otherwise, the probability to select two clusters is bigger. In the former case, a bigger Pb is beneficial to local search, while in the latter case, a smaller Pb is beneficial to global search. The parameter Pb1 determines whether the cluster center or the randomly selected individuals in a cluster is chosen to generate new individuals, Pb2 determines whether the two cluster centers or two individuals in two clusters are chosen to generate new individuals. Generating new

252

X. Wu

Table 4 Procedure of the basic BSO Algorithm 1. Initialization 2. Clustering 2.1. Cluster individuals into p clusters, and evaluate the n individuals. The best individual in each cluster is considered as the cluster center; 2.2. Generate randomly a variable Ra ∈ (0, 1), if Ra < Pa , proceed to step 2.3, otherwise to Step 3; 2.3. Select randomly a cluster center, and generate randomly a new individual to replace it. 3. Generating new solutions 3.1. Select randomly an individual as the current individual; 3.2. Generate randomly a variable Rb ∈ (0, 1), if Rb < Pb , proceed to step 3.3, otherwise to Step 3.4; 3.3. Select randomly a cluster, and generate a variable Rb1 ∈ (0, 1), if Rb1 < Pb1 , then randomly disturb the cluster center to generate new individuals, otherwise randomly disturb a randomly selected individual from the cluster to generate a new individual; 3.4. Select randomly two clusters, and generate a variable Rb2 ∈ (0, 1), if Rb2 < Pb2 then combine the two cluster centers and randomly disturb to generate new individuals, otherwise combine two randomly selected individuals from each cluster and randomly disturb the combiner to generate a new individual; 3.5. Compare the new individual with the current individual, store the better one; 3.6. If all individuals have been generated, proceed to step 4, otherwise return to step 3.1. 4. Iteration If the terminal condition is satisfied, stop; otherwise return to Step 2.

individuals by optimizing the cluster center is to ensure the convergence of the algorithm. Optimizing randomly selected individuals is to increase the diversity of the population and to avoid the premature.

4.2 MBSO Algorithm The MBSO algorithm is a modified version of BSO which was presented by Zhou et al. in 2011 [35]. It consists of initialization, clustering, generating new individuals and selection. A new method is designed to generate new individuals. A selection process is proposed to select the better solution from the new and the old solutions, which simulates the process in brainstorming meeting where the problem owners select a better solution from the new and the old solutions. The probability parameter Pa in the clustering process determines the perturbation extent. Three new generation mechanisms are designed to generate new individuals. The selection process provides a special individual surviving mechanism. Its pseudo code is listed in Table 5. The clustering process in MBSO is similar to that in BSO, individuals with similar fitness are classified into a cluster, and the best individual is considered as the cluster center. The parametere Pa plays the similar role of that in the basic BSO algorithm. The new individuals are generated by: (1) perturbing the cluster centers to generate new individuals, or (2) combining two individuals to generate new individuals, or (3) perturbing one individual to generate new individual. The first one improves the

Brain Storm Optimization Algorithms …

253

Table 5 Pseudo code of MBSO Algorithm 1. Initialization 2. Clustering 3. Generating new individuals 3.1. Select a cluster center, and disturb randomly the cluster center to generate new individuals, repeat until n new individuals are generated; 3.2. Combine two cluster centers, and then disturb the combiner to generate a new individual, repeat until n new individuals are generated; 3.3. Select randomly an individual and disturb to generate a new individual, repeat until n new individuals are generated. 4. Selecting 4.1. Evaluate the 3*n new individuals; 4.2. Randomly group the 4*n individuals (3*n new individuals plus n original individuals) into n groups, and thus there are 4 individuals in each group. Select the best individual of each group into the next generation. 5. Iterating If the terminal condition is satisfied, stop and output the results, otherwise return to step 2.

local search, while the latter two increase the diversity of population and avoid the premature.

4.3 DMBSO Algorithm The DMBSO algorithm is based on a discussion mechanism proposed by Yang et al. in 2015 [31]. It consists of initialization, clustering, and generating new individuals. Different from that in the basic BSO algorithm, new individual is generated by intergroup discussion and intra-group discussion. In DMBSO algorithm, clustering and generating new individuals are two core steps, the parameters Pa plays the same role as that in BSO. The parameters Pb1 , Pb2 and Pc1 , Pc2 determine the convergence and diversity of the inter-group and the intra-group, respectively. Its pseudo code is listed in Table 6. New individuals are generated through the inter-group discussion or the intragroup discussion. The inter-group discussion is carried out in a group, and the intra-group discussion is carried out between two groups. The two processes are sequentially executed. In the inter-group discussion, the parameters Pb1 and Pb2 determine to choose one among the three steps: (a) to perturb a cluster center, (b) to perturb a random chosen individual or (c) to combine two randomly selected individuals to generate new individuals. Such inter-group discussion is beneficial to the local search. The step (a) facilitates the local exploitation, the steps (b) and (c) help to keep the diversity of the population in local search. Similarly, in the intra-group discussion, the parameters Pc1 and Pc2 determine which step is to be carried out, i.e. (a) to combine the two cluster centers, (b) to combine two random individuals, and (c) to generate randomly a new individual.

254

X. Wu

Table 6 Pseudo code of DMBSO algorithm 1. Initialization 2. Clustering 3. Discussing in the inter-group For each individual in each cluster, carry out the following steps. 3.1. Generate randomly a variable Rb1 ∈ (0, 1), if Rb1 < Pb1 , disturb randomly the cluster center to generate a new individual; otherwise proceed to step 3.2; 3.2. Generate randomly a variable Rb2 ∈ (0, 1), if Rb2 < Pb2 , disturb randomly a random individual of the cluster to generate a new individual; otherwise combine two random individual of the cluster and then disturb the combiner to generate a new individual; 3.3. Compare the new individual with the current individual and save the better one; 3.4. Stop until all the individuals are discussed and proceed to step 4. 4. Discussing in the intra-group 4.1. Select randomly two clusters, and select randomly an individual as the current individual; 4.2. Generate randomly a variable Rc1 ∈ (0, 1), if Rc1 < Pc1 , Combine two cluster centers and then randomly disturb the combiner to generate a new individual; otherwise to step 4.3; 4.3. Generate randomly a variable Rc2 ∈ (0, 1), if Rc2 < Pc2 , Combine two random individual in the two clusters and then randomly disturb it to generate a new individual; otherwise randomly generate a new individual; 4.7. Compare the new individual with the current one and save the better one; Repeat the step 4.1–4.7 until the iteration times of the intra-group discussion is reached. 5. Iteration Terminate if the maximal iteration is reached; otherwise return to step 2.

Such intra-group discussion can increase the diversity of population and avoid the premature. The step (a) takes full advantage of the two cluster centers, and step (b) and step (c) increase the diversity and can speed up the convergence. The inter-group discussion and intra-group discussion can greatly improve the search capability of the algorithm.

4.4 PD-DMBSO Algorithm During the searching of the BSO algorithm, one can find that the searching is easily trapped into a premature. Through studying the population distribution, Wu et al. found that there are two cases: (1) the majority of the population individuals are the same; (2) the most individuals in population are different, but their fitness are same [28]. To deal with the premature, Wu et al. [28] proposed an improved algorithm on the base of the DMBSO algorithm. For convenience, Wu et al. name it population diversity and discussion mechanism based BSO (PD-DMBSO) algorithm. A trigger is integrated into to the algorithm. When the optimum fitness and average fitness values are nearly equal, trigger the diversity strategy. The two diversity strategies are designed as following:

Brain Storm Optimization Algorithms …

255

Table 7 Procedure of PD-DMBSO algorithm 1. Initialization 2. The Main Processes in DMBSO 2.1. Carry out the clustering process of the basic BSO algorithm; 2.2. Carry out the new individual generation process of DMBSO (i.e. the inter-group discussion and the intra-group discussion). 3. Population Diversity Strategy 3.1. If the best fitness is equal to the average fitness, proceed to step 3.2, otherwise proceed to step 4; 3.2. Randomly generate some, i.e.population size × α, new individuals, then randomly replace the equal amount of individuals in the population; 3.3. Local search the best solution to generate a set of neighbor solutions, i.e. population size × β, and randomly replace the same amount of individuals in the population. 4. Iteration Return to step 2 and repeat until the maximum iteration is reached; otherwise terminate the iteration.

Strategy 1. Randomly initialize some individuals; Strategy 2. Local search the best individual to generate a set of neighbor solutions and replace the individuals in the original population. The procedure of the PD-DMBSO is listed in Table 7. The parameters α and β determine the degree of keeping the diversity of the population, the greater the coefficients, the greater impact on the diversity of the population. The population is more likely to jump out of the local optima, but also probably destroy the searching effort of the original population. The parameter α determines the random initialization scale and it damages α percent of the individuals in the population. Such damaging can increase the diversity in a larger extent. Comparing the larger diversity from the parameter α, the parameter β determines the local search on the best individual to generate new individuals, and it makes a smaller influence on the population.

4.5 BSO Algorithm and Its Varieties for FJSP 4.5.1

Representation

Before using the BSO algorithm and its varieties to solve the FJSP, one must represent the FJSP firstly. In the existing literature, there are five types of chromosome representations for FJSP as following. Chromosome A, proposed by Chen et al. [3], consists of two integer strings (denoted as A1 and A2). The length of each string is equal to the total number of operations. String A1 assigns a machine to each operation. The value of the j-th position of the string A1 indicates the machine performing the j-th operation. String A2 encodes the order of operations on each machine.

256 Fig. 1 An example of a chromosome

X. Wu

Chromosome 2 Operations

1

3

3

2

2

1

1

3

O12 O11 O13 O23 O22 O32 O21 O31 O33

Chromosome B, proposed by Paredis [18], also consists of two strings (denoted as B1 and B2). String B1 is identical to A1. String B2 is a bit string that gives the order of any pair of operations. A bit value of “0” indicates that the first operation in the paired-combination must be performed before the second operation. Chromosome C, proposed by Ho and Tay [10], also consists of two strings (denoted as C1 and C2). This chromosome represents an instance of the FJSP, where the operations in each job have sequential precedence constraints. String C1 encodes the order of the operations with respect to other jobs. It does not specify the order of operations within the same job as this is already implied by its index value. String C2 represents the machine assignment to operations (as in A1 and B1) but with a twist. To ensure solution feasibility, the machine index is manipulated so that the string will always be valid. Chromosome D, proposed by Tay and Wibowo [24], is composed from chromosome representations B and C. The chromosome consists of three strings (denoted as D1, D2 and D3). D1 and D2 are equivalent to C1 and C2, respectively, while D3 is similar to B2. Chromosome E, proposed by Wu et al. [27], modified the operation-based encoding developed by Gen et al. Each chromosome consists of n × max{nj } numbers and each number corresponding a job occurs max{nj } times. A surplus part represents those jobs whose operation number are less than max{nj }. The surplus part is referred to be virtual operations that don’t take any machine time. For the 3 × 3 problem illustrated in Table 2, a feasible chromosome encoded with job number is [2 1 3 3 2 2 1 1 3]. The matchup between the chromosome and the sequence of the operations is described in Fig. 1. Following the definition of Tay and Wibowo [24], we denote T as the total number of job operations in an FJSP. In the best case when the numbers of operations for each job are the same, the length of chromosome E is T. However, in the worst case when there is a special job whose operation number is far more than the others (assuming there is only one operation for these jobs), the length of the chromosome is n * (T − n + 1). The worst case occurs with a very low probability because there are always similar jobs to be processed in a practical production workshop. So one can confidently conclude that the length of the chromosome E is normally shorter than that of the others. In order to make a comparison, the space complexity of the five chromosome representations is listed in Table 8. The variable d denotes the length of the string D3 [24]. It is obvious that the space complexity of the chromosome E is less than the others. Therefore, the chromosome E is adopted in the part of the experiment. Before evaluation, each chromosome should be decoded to be a scheduling solution. Thus, a scheduling algorithm is needed. First two variables are given.

Brain Storm Optimization Algorithms …

257

Table 8 Space complexity of the representations Chromosome

Length

Chromosome A (Chen et al. 1999) [3]

2T

Chromosome B (Paredis 1992) [18]

T + 0.5T (T – 1)

Chromosome C (Ho and Tay 2004) [10]

T + 0.5T (T – 1)

Chromosome D (Tay and Wibowo 2004) [24]

2T + d

Chromosome E (Wu et al. 2017) [27]

n × max{nj } ∈ (T, n(T – n + 1))

Machine sequence matrix J M. Its element J M (i, j) denotes the available machine for the (j/m + 1)-th operation of the job J i . The symbol   is to get the integral part of a float number. Each row, i.e. J M (i, :), denotes all the available machines for the operations of job J i . The length of J M (i, :) is max{ni } × m. In each line, from the beginning, every m numbers form a fragment which denotes all the available machines for an operation. Among the machines, if machine k is available for an operation, we denote it k; otherwise we denote it 0. In the totally FJSP, in which each operation can be processed on any of the machines, the fragment equals to ‘1 2 … m’. As for the partially FJSP, in which all machines are not always available, one can use ‘0’ to ensure the fixed length of the fragments. For example, in the FJSP in Table 2, the operation O11 can be processed on machine M 1 and machine M 2 only, so the first fragment of J M (1, :) should be ‘1 2 0’; the operation O21 can be processed on machine M 2 and machine M 3 only, so the second fragment of J M (1, :) should be ‘2 3 0’; the operation O31 can be processed on all the machines, so the third fragment of J M (1, :) should be ‘1 2 3’. Thus, J M (1, :) = [1 2 0 2 3 0 1 2 3]. The number of fragments is determined by the maximum operation number among the jobs, i.e. max{ni }. For those jobs whose operation number is less than max{ni }, it is designed to arrange the available machines following the above rule firstly and then set ‘0’ (max{ni } − ni ) × m times to achieve the length max{ni } × m. Processing time matrix T. Its element T(i, j) denotes the processing time on machine J M (i, j) for the operation j/m + 1 of the job J i . If J M (i, j) = 0, it means the machine mod (j/m) (where the symbol “mod” is a function to obtain the reminder of j/m) is not available for the operation j/m + 1, its processing time is, therefore, set to be “0” as well. For example, in the FJSP in Table 2, the operation O11 can be processed on machine M 1 and machine M 2 only and occupies 2 and 3 units of time respectively, so the first fragment of T(1, :) should be ‘2 3 0’. As such, the machine sequence matrix J M and the processing time matrix T for the FJSP in Table 2 can be defined as follows: ⎡

JM

⎤ ⎡ ⎤ 120230123 230520364 = ⎣ 2 3 0 1 2 3 1 2 3 ⎦, T = ⎣ 4 5 0 5 5 6 2 4 8 ⎦. 130123123 460444567

258

X. Wu Initialize chrom x=1

Get chrom(x)

Set r=Count(chrom(x))

Get Or, chrom(x) J M, T

Min(en(chrom(x),r))

Idletime(k)>pr, chrom(x),k? N

Y

Set st(chrom(x),r )=en(mach(k))

Insert Or, chrom(x)

Update st(chrom(x),r ), en(chrom(x),r ), mach(k) x=x+1

x>len(chrom)?

N

Y Output a scheduling solution

Fig. 2 The decoding algorithm

Once J M and T are determined, an individual can be decoded to a scheduling solution. The decoding algorithm is illustrated in Fig. 2. The process is shown as following. Step 1. Obtain an individual chrom and set x = 1; Step 2. Repeat when x is less than the length of the individual (=len(chrom)) Step 2.1. Obtain the x-th gene chrom(x), which is a job number; Step 2.2. Determine the operation order r (=count(chrom(x))) according to the appearing times of chrom(x). Search the available machines for the current operation Or,chrom(x) , and select the one on which the operation can be finished at the earliest time en(chrom(x), r). Note, en is a matrix recording the completing time of the operation Or,chrom(x) . If there is more than one machine available,

Brain Storm Optimization Algorithms …

Step 2.3.

Step 2.4.

Step 2.5.

Step 2.6.

259

select the one on which the processing time pchrom(x),r,k is the shortest. Compare the idle time of machine M k , if the processing time pchrom(x),r,k is shorter than the idle time idletime(k), go to step 2.4; otherwise go to step 2.5; Insert the operation Or,chrom(x) and determine its beginning time st(chrom(x), r) and completing time en(chrom(x), r). Update the available time and the idle time of machine M k . Go to step 2.6. Append the operation Or,chrom(x) at the end of machine M k and set its beginning time st(chrom(x), r) to be the completing time of the machine (mach(k)). Update its the completing time en(chrom(x), r) and the completing time mach(k) of machine M k by adding pchrom(x),r,k to st(chrom(x), r). Go to step 2.6. Obtain the next gene, set x = x + 1, and return to the step 2.1.

Step 3. Output the scheduling solution. An operation can be inserted into a machine’s idle interval [t 1 , t 2 ] if and only if its current operation processing time is shorter than t 2 − t 1 . The algorithm solves the machine assignment problem and the operation sequence problem simultaneously. The algorithm outputs a scheduling solution from which one can evaluate the fitness of the targeting chromosome according to the following fitness function. fitness =

1   max Cij

(7)

The proposed decoding algorithm has similar computational complexity to that of the existing methods. The major computational complexity lies in the loop of step 2. This loop needs repeating n × max{nj } times. The running time of step 2.1 is O(1). All the available machines for the current operation should be considered. In the worst case, the running time of the step 2.2 is O(m × n × max{nj }). The running time of step 2.3 is O(1). The running time of step 2.4 or step 2.5 is O(1). But only one of them is selected to be executed. The running time of step 2.6 is O(1). Therefore, the time complexity of the decoding algorithm is computed as following.   

 Ta = O n × max{nj } × 1 + m × n × max nj + 1 + 1 + 1  2   = O mn2 max nj + 4 n × max{nj }  2 = O mn2 max nj   = O T2 . All the time complexity of converting the five types of chromosome representations to scheduling solutions is listed in Table 9 in which variable c denotes the number of precedence constraints.

260

X. Wu

Table 9 Conversion complexity of the representations

4.5.2

Chromosome representation

Conversion complexity

Chromosome A (Chen et al. 1999) [3]

O(T + c)

Chromosome B (Paredis 1992) [18]

O(T 2 + c)

Chromosome C (Ho and Tay 2004) [10]

O(T + c)

Chromosome D (Tay and Wibowo 2004) [24]

O(T + c + d)

Chromosome E (Wu et al. 2017) [27]

O(T 2 )

Clustering Algorithm

Whether the basic BSO algorithm or its varieties, clustering is always needed. The frequently employed cluster method, such as K-means algorithm [8], can be employed in the clustering step to find the K cluster centers. The details of the K-means clustering algorithm are as follows: Step (1) Initialize. Randomly select K individuals as the initial cluster centers; Step (2) Classify. Compute the Euclidean distance of all the individuals to each cluster center, group those with a nearest distance to the cluster center; Step (3) Update. Select the individual whose fitness is equal to the median value in each class as the new cluster center; Step (4) Iterate. If the cluster centers are updated in step (3), return to the step (2), otherwise, terminate the clustering process.

4.5.3

The Disturbing Operators

When disturbing an individual in BSO algorithm and its varieties, the usual mutation operators can be employed to disturb an individual. For FJSP, several operators have been designed up to now. (1) Swap operator The swap operator randomly selects two positions and swaps the genes in the two positions. Figure 3 gives an example of how swap operator disturbs genes. Two chosen positions are highlighted and their values “3” and “1” are swapped. (2) Shift operator The shift operator randomly selects a position and then shifts it left or right s times where s is a random integer. Figure 4 gives an example of how shift operator disturbs genes. The highlighted position is chosen and it is shift right 4 times (s = 4).

Brain Storm Optimization Algorithms …

261

Before disturbing

2

3

3

1

3

2

1

2

1

After disturbing

2

3

1

1

3

2

3

2

1

Fig. 3 An example of how swap operator disturbs genes s Before disturbing

2

3

3

1

3

2

1

2

1

After disturbing

2

3

1

3

2

1

3

2

1

Fig. 4 An example of how shift operator disturbs genes

Before disturbing

2

3

3

1

3

2

1

2

1

After disturbing

2

3

2

3

1

3

1

2

1

Fig. 5 An example of how the inversion operator disturbs genes

(3) Inversion operator The inversion operator selects two positions in an individual and changes the order of the genes. The genes before and after the positions are directly copied from the original individual to the new individual. Figure 5 gives an example of how inversion operator disturbs genes. The highlighted “3 1 3 2” are genes between two randomly selected positions, their order is inverted to “2 3 1 3”. Other genes remain unchanged. (4) Insertion operator The insertion operator randomly selects two positions, say p1 and p2. The genes between p1 + 1 and p2 shifted to their previous positions and the genes in position p1 in inserted into the position p2. Figure 6 gives an example of how the insertion operator works. The genes in the two highlighted positions are “3” and “1”, the “1 3 2 1” are shifted to their previous positions and the highlighted “3” is assigned to the original highlighted positions “1”.

4.5.4

The Combining Operators

In the process of combining two individuals, the crossover in genetic algorithm can be employed to generate a new individual. There are several crossover operators up to

262

X. Wu s

Before disturbing

2

3

3

1

3

2

1

2

1

After disturbing

2

3

1

3

2

1

3

2

1

Fig. 6 An example of how the insertion operator disturbs genes Parent 1

2

3

3

1

3

2

1

2

1

Parent 2

1

2

2

3

1

2

3

3

1

Offspring 1

X

X

X

3

1

2

X

X

X

Offspring 2

X

X

X

1

3

2

X

X

X

Offspring 1

1

2

3

3

1

2

3

2

1

Offspring 2

3

2

2

1

3

2

1

3

1

Fig. 7 An example of how the OX crossover operator combines two individuals

now. The following is some frequently employed crossover operators for scheduling problems. It is noted that all the operators are designed for the operation-based chromosome. (1) Combining with order crossover operator Select two positions randomly in both parent individuals. Exchange the genes between the two positions to generate offspring. In each offspring, determine the positions of the genes which are the same with the ones taken by exchange and then assign the genes into the determined positions, starting from the second positions. Figure 7 shows an example of how the order crossover operator combines two individuals. The highlighted genes in the parent 1 and the highlighted genes in the parent 2 are exchanged first. Then, delete genes which are the same as the exchanged genes and label the positions with X to ensure the legality. For the X positions in the first offspring, the genes exchanged to the second offspring (1 3 2) are placed as in the order in the second parent. This procedure is repeated for the second procedure too. (2) Combining with linear order crossover operator Linear order crossover (LOX) works similarly to the order crossover operator. Select two positions randomly. Exchange the substring between the positions. Delete the

Brain Storm Optimization Algorithms …

263

Parent 1

2

3

3

1

3

2

1

2

1

Parent 2

1

2

2

3

1

2

3

3

1

offspring1

X

X

X

3

1

2

X

X

X

offspring2

X

X

X

1

3

2

X

X

X

offspring1

3

1

3

3

1

2

2

2

1

offspring2

2

3

1

1

3

2

2

3

1

Fig. 8 An example of how the LOX crossover operator combines two individuals

genes beyond the positions which are inherited from the other parent. Insert the rest into an offspring from left to right skipping the inherited part. Figure 8 shows an example of how the LOX operator combines two individuals. The highlighted genes in the first parent and the highlighted genes in the second parent are exchanged first. Then, delete genes beyond the inherited positions and label the positions with X. For the X positions in the first offspring, insert the genes as in the order in the parent 1 from left to right skipping the inherited part. (3) Combining with position-based crossover operator Position-based crossover (PBX) works like a uniform crossover operator and applies a procedure to repair illegal offspring. Generate a “0–1” string with the same length as the chromosomes and then execute the multi-point crossover on these positions marked with “1”. The other positions marked with “0” are labeled X. Then delete the exchanged genes and insert the rest genes into positions marked with “0” in the original order. Figure 9 shows an example of how the PBX operator combines two individuals. A 0–1 string “1 0 0 1 1 0 1 1 0” is generated. The positions marked “1” is assigned genes from the same positions in the parent 2. Then delete genes which are the same in the positions marked with “1” and insert the rest to the positions marked with “0”. This procedure is repeated for the second offspring too. (4) Combining with precedence operation crossover operator Precedence operation crossover (POX) is proposed by Zhang et al. [34]. Choose randomly one job numbers which will have the highest precedence to be kept. Delete the genes in the second parent which has the highest precedence and then insert them into the rest positions with the same order. Figure 10 shows an example of how the POX operator combines two individuals. The job 3 is chosen to have the highest precedence to be kept. Then all “3” genes

264

X. Wu Parent 1

2

3

3

1

3

2

1

2

1

Parent 2

1

2

2

3

1

2

3

3

1

0-1 string

1

0

0

1

1

0

1

1

0

Offspring 1

1

X

X

3

1

X

3

3

X

Offspring 2

2

X

X

1

3

X

1

2

X

Offspring 1

1

2

2

3

1

2

3

3

1

Offspring 2

2

2

3

1

3

3

1

2

1

Fig. 9 An example of how the PBX crossover operator combines two individuals Parent 1

2

3

3

1

3

2

1

2

1

Parent 2

1

2

2

3

1

2

3

3

1

Offspring 1

X

3

3

X

3

X

X

X

X

Offspring 2

X

X

X

3

X

X

3

3

X

Offspring 1

1

3

3

2

3

2

1

2

1

Offspring 2

2

1

2

3

1

2

3

3

1

Fig. 10 An example of how the POX crossover operator combines two individuals

in parent 1 and parent 2 are kept first and the other positions are marked with “X”. Next, delete all “3” genes in parent 2 and insert the rest to the positions marked with “X” in the offspring 1 with the same order. The procedure is repeated for the second offspring.

Brain Storm Optimization Algorithms …

265

5 Experimental Results 5.1 Experimental Design The BSO algorithm and its varieties are compiled on a PC with Intel Core i3-3240 CPU, CPU frequency 3.40 GHz, RAM 4.00 GB, Win7 64-bit operating systems and coded with MATLAB 2013b software. To begin with the experiment, the values of the parameters are set in Table 10 following the suggestion by Shi [21], Zhou et al. [35], and Yang et al. [31]. The experiment data come from Kacem benchmarks [11] and Brandimarte benchmarks [1]. The aim of the experiment is to compare the performance of the four algorithms for solving FJSP to optimize the makespan.

5.2 Experimental Results The results of the Kacem and Brandimarte benchmark instances are listed in Table 11 and Table 12, respectively. Among Brandimarte benchmark instances, the optimal solution for the MK01, MK03, MK08 and MK09 can easily be found with any of the four BSO algorithms. It is difficult to compare the difference of their performance in term of the solution quality. Hence, it is more suitable to test the other six instances in order to compare

Table 10 Parameters settings Algorithms

K

Pa

Pb

Pb1

Pb2

Pc1

Pc2

α

β

BSO

5

0.2

0.2

0.5

0.5

–

–

–

–

MBSO

5

0.2

–

–

–

–

–

–

–

DMBSO

5

0.2

–

0.6

0.5

0.5

0.7

–

–

PD-DMBSO

5

0.2

–

0.6

0.5

0.5

0.7

0.2

0.2

Table 11 Results of Kacem benchmark instances Benchmarks

Job × Machine

Best1

BSO

MBSO

DMBSO

PD-DMBSO

Kacem 1

4×5

11

11

11

11

11

Kacem 2

8×8

14

14

14

14

14

Kacem 3

10 × 7

11

11

11

11

11

Kacem 4

10 × 10

7

7

7

7

7

Kacem 5

15 × 10

11

11

11

11

11

Note Best1 is the best solution obtained in Kacem et al. [11]. The bold values are the best results up to now for the benchmark instance

266

X. Wu

Table 12 Results of Brandimarte benchmark instances Benchmarks

Job × Machine

MK01

10 × 6

MK02

10 × 6

MK03

15 × 8

MK04

15 × 8

(48, 81)

60

66

63

63

60

MK05

15 × 4

(168, 186)

172

176

174

176

173

MK06

10 × 15

(33, 86)

58

63

62

63

60

MK07

20 × 5

(133, 157)

139

144

141

144

141

MK08

20 × 10

(523, 523)

523

523

523

523

523

MK09

20 × 10

(299, 369)

307

307

307

307

307

MK10

20 × 15

(165, 296)

197

223

223

221

218

Best2

BSO

(36, 42)

40

40

40

40

(24, 32)

26

27

27

27

26

(204, 211)

204

204

204

204

204

(LB, UB)

MBSO

DMBSO PDDMBSO 40

Note Best2 is the best solution obtained in [9]. The bold values are the best results up to now for the benchmark instance

their performance. To show the difference clearly, we carry out 30 runs for those six instances and plot their boxplots (Fig. 11). Although one can get the visual difference among the four algorithms from the boxplots, some statistical analysis is still needed to make a more common conclusion. Therefore, Wilcoxon paired test is carried out. The mean value of the 30 runs results for each instance (Table 13) are used to do the Wilcoxon paired test. The test result (Table 14) reports the significant level among different algorithms. One can conclude: (1) the performance of the MBSO, DMBSO and PD-DMBSO algorithms is significantly better than the BSO algorithm; (2) there is no significant difference between DMBSO and PD-DMBSO; and (3) PD-DMBSO algorithm is significantly better than the DMBSO algorithm. Therefore, considering the quality of the best solution, the boxplots and the statistical analysis result, the performance ranking for the four algorithms is: BSO < DMBSO < MBSO < PD-DMBSO. (BSO: the basic BSO algorithm [21], DMBSO: the discussion mechanism based BSO algorithm [31], MBSO: the modified BSO algorithm [35] and PD-DMBSO: the population diversity and discussion mechanism based BSO algorithm [28].) The performance difference of the four algorithms is due to their mechanism. The common step for the four algorithms are they all cluster the individuals into several centers. The difference lies in that their different searching process. MBSO, DMBSO and PD-DMBSO search a new individual by not only one method as that in BSO. Such designing could further explore the solution space. From the boxplots, one can also see the difference of the four algorithms. PD-DMBSO algorithm performs best for solving the six instances. BSO algorithm performs worst. The MBSO algorithm performs equivalently as the DMBSO algorithm. But the MBSO performs unsteadily. For instance MK05 and instance MK07, MBSO performs similarly as PD-DMBSO algorithm. But for other instances, especially for instance MK02 and MK10, it per-

Brain Storm Optimization Algorithms …

267

Fig. 11 Boxplots of the four BSO algorithms

(a) The Boxplot for instance MK02

(b) The Boxplot for instance MK04

(c) The Boxplot for instance MK05

268

X. Wu

Fig. 11 (continued)

(d) The Boxplot for instance MK06

(e) The Boxplot for instance MK07

(f) The Boxplot for instance MK10

Brain Storm Optimization Algorithms … Table 13 The mean value of the experiments

269

Benchmarks

BSO

MBSO

DMBSO

PD-DMBSO

Kacem 1

11

11

11

11

Kacem 2

14

14

14

14

Kacem 3

11

11

11

11

Kacem 4

7

7

7

7

Kacem 5

11

11

11

11

MK01

40

40

40

40

MK02

27.7

27.3

27.1

26.7

MK03

204

204

204

204

MK04

67.2

65.2

65.6

63.1

MK05

177.4

175.9

177

175.9

MK06

64.5

63.7

64.4

62.9

MK07

145.8

142.7

145.1

143.1

MK08

523

523

523

523

MK09

307

307

307

307

MK10

226.4

225

224.5

223.6

Table 14 The result of the Wilcoxon paired test BSO

MBSO

DMBSO

PD-DMBSO

1

1

1

0

0

MBSO DMBSO

1

Note The confidence level is 0.05

forms similarly as BSO. The reason is its random searching and too much iteration also takes too much computing effort. The BSO, DMBSO and MBSO are easier to be trapped into a premature area. On the contrary, the PD-DMBSO escaped from the premature by integrating diversity strategy to increase the population diversity. When the individuals are highly similar, some individuals will be replaced by the new generated randomly individuals. Besides, the best individual will be deeply local searched to find a better neighbor individual.

6 Conclusions This chapter studies how to employ the brain storm optimization algorithm and its varieties to solve the flexible job shop scheduling problem. Firstly, an optimization model of the flexible job shop scheduling problem is formulated. The literatures on flexible job shop scheduling problem are reviewed and analyzed. Four brain storm optimization algorithms are designed to solve the flexible job shop scheduling prob-

270

X. Wu

lem. The existed five representations for flexible job shop scheduling problem are compared. The K-means based clustering algorithm is developed. Four disturbing operators and four combining operators are designed for the flexible job shop scheduling problem. In the experiment section, benchmark instances are tested to compare the performance of the four algorithms for solving the flexible job shop scheduling problem. From the results, one can conclude that the population diversity and discussion mechanism based brain storm optimization algorithm performs best because it is integrated with the population diversity mechanism and discussion mechanism which can balance the exploitation and exploration very well. Acknowledgements This work is supported by the National Natural Science Foundation of China Grant (Grant No. 50135024). The author also thanks the three anonymous reviewers for their constructive suggestions and comments to improve the quality of the work.

References 1. Brandimarte, P.: Routing and scheduling in a flexible job shop by tabu search. Ann. Oper. Res. 41(3), 157–183 (1993) 2. Brucker, P., Schlie, R.: Job-shop scheduling with multi-purpose machines. Computing 45(4), 369–375 (1990) 3. Chen, H., Ihlow, J., Lehmann, C.: A genetic algorithm for flexible job-shop scheduling. Proc. IEEE Int. Conf. Robot. Autom. 2, 1120–1125 (1999) 4. Chen, J.C., Wu, C., Chen, C., Chen, K.: Flexible job shop scheduling with parallel machines using genetic algorithm and grouping genetic algorithm. Expert Syst. Appl. 39, 10016–10021 (2012) 5. Dauzère-Pérès, S., Paulli, J.: An integrated approach for modeling and solving the general multiprocessor job-shop scheduling problem using tabu search. Ann. Oper. Res. 70, 281–306 (1997) 6. Gao, J., Sun, L., Gen, M.: A hybrid genetic and variable neighborhood descent algorithm for flexible job shop scheduling problems. Comput. Oper. Res. 35, 2892–2907 (2008) 7. Gong, G., Deng, Q., Gong, X., et al.: A new double flexible job-shop scheduling problem integrating processing time, green production, and human factor indicators. J. Clean. Prod. 174, 560–576 (2018) 8. Hartigan, J.A., Wong, M.A.: A K-means clustering algorithm. Appl. Stat. 28(1), 100–108 (1979) 9. Hmida, A.B., Haouari, M., Huguet, M.J., et al.: Discrepancy search for the flexible job shop scheduling problem. Comput. Oper. Res. 37(12), 2192–2201 (2010) 10. Ho, N.B., Tay, J.C.: GENACE: an efficient cultural algorithm for solving the flexible job-shop problem. In: Proceedings of the 2004 Congress on Evolutionary Computation, CEC2004, vol. 2, pp. 1759–1766 (2004) 11. Kacem, I., Hammadi, S., Borne, P.: Approach by localization and multi-objective evolutionary optimization for flexible job-shop scheduling problems. IEEE Trans. Syst. Man Cybern. C 32, 408–419 (2002) 12. Li, J., Pan, Q.: Chemical-reaction optimization for flexible job-shop scheduling problems with maintenance activity. Appl. Soft Comput. 12, 2896–2912 (2012) 13. Li, X., Gao, L.: An effective hybrid genetic algorithm and tabu search for flexible job shop scheduling problem. Int. J. Prod. Econ. 174, 93–110 (2016) 14. Loukil, T., Teghem, J., Tuyttens, D.: Solving multi-objective production scheduling problems using metaheuristics. Eur. J. Oper. Res. 161, 42–61 (2005)

Brain Storm Optimization Algorithms …

271

15. Ma, Y., Chu, C., Zuo, C.: A survey of scheduling with deterministic machine availability constraints. Comput. Ind. Eng. 58(2), 199–211 (2010) 16. Mati, Y., Rezg, N., Xie, X.: An integrated greedy heuristic for a flexible job shop scheduling problem. Proc. IEEE Int. Conf. Syst. Man Cybern. 4, 2534–2539 (2001) 17. Mokhtari, H., Hasani, A.: An energy-efficient multi-objective optimization for flexible job-shop scheduling problem. Comput. Chem. Eng. 104, 339–352 (2017) 18. Paredis, J.: Exploiting constraints as background knowledge for genetic algorithms: a casestudy for scheduling. In: Parallel Problem Solving from Nature 2, PPSN-II, Brussels, Belgium, September. DBLP, pp. 231–240 (1992) 19. Pezzella, F., Morganti, G., Ciaschetti, G.: A genetic algorithm for the flexible job-shop scheduling problem. Comput. Oper. Res. 35, 3202–3212 (2008) 20. Piroozfard, H., Wong, K.Y., Wong, W.P.: Minimizing total carbon footprint and total late work criterion in flexible job shop scheduling by using an improved multi-objective genetic algorithm. Resour. Conserv. Recycl. 128, 267–283 (2016) 21. Shi, Y.: Brain Storm Optimization Algorithm. Lecture Notes in Computer Science, vol. 4, no. 3, pp. 303–309 (2011) 22. Singh, M.R., Mahapatra, S.S.: A quantum behaved particle swarm optimization for flexible job shop scheduling. Pergamon Press Inc. (2016) 23. Syswerda, G.: Scheduling optimization using genetic algorithms. In: Handbook of Genetic Algorithms. Van Nostrand Reinhold, New York (1991) 24. Tay, J.C., Wibowo, D.: An effective chromosome representation for evolving flexible job shop schedules. In: Genetic and Evolutionary Computation—GECCO 2004. DBLP, 2004, pp. 210–221 (2004) 25. Wang, L., Wang, S., Xu, Y., Zhou, G., Liu, M.: A bi-population based estimation of distribution algorithm for the flexible job-shop scheduling problem. Comput. Ind. Eng. 62, 917–926 (2012) 26. Wu, X., Sun, Y.: A green scheduling algorithm for flexible job shop with energy-saving measures. J. Clean. Prod. 172, 3249–3264 (2018) 27. Wu, X., Wu, S.: An elitist quantum-inspired evolutionary algorithm for the flexible job-shop scheduling problem. J. Intell. Manuf. 28(6), 1441–1457 (2017) 28. Wu, X., Zhang, Z., Li, J.: A brain storm optimization algorithm integrating diversity and discussion mechanism for solving discrete production scheduling problem. Control Decis. 32(9), 1583–1590 (in Chinese) (2017) 29. Xia, W., Wu, Z.: An effective hybrid optimization approach for multi-objective flexible job-shop scheduling problems. Comput. Ind. Eng. 48, 409–425 (2005) 30. Xing, L.N., Chen, Y.W., Wang, P., Zhao, Q.S., Xiong, J.: A knowledge-based ant colony optimization for flexible job shop scheduling problems. Appl. Soft Comput. 10, 888–896 (2010) 31. Yang, Y., Shi, Y., Xia, S.: Advanced discussion mechanism-based brain storm optimization algorithm. Soft. Comput. 19(10), 2997–3007 (2015) 32. Yazdani, M., Amiri, M., Zandieh, M.: Flexible job-shop scheduling with parallel variable neighborhood search algorithm. Expert Syst. Appl. 37(1), 678–687 (2010) 33. Yuan, Y., Xu, H.: Flexible job shop scheduling using hybrid differential evolution algorithms. Comput. Ind. Eng. 65(2), 246–260 (2013) 34. Zhang, C.-Y., Li, P., Rao, Y., Li, S.: A new hybrid GA/SA algorithm for the job shop scheduling problem. In: Raidl, G.R., Gottlieb, J. (eds.) EvoCOP 2005. Lecture Notes in Computer Science, vol. 3448, pp. 246–259. Springer, Heidelberg (2005) 35. Zhou, D., Shi, Y., Cheng, S.: Brain storm optimization algorithm with modified step-size and individual generation. In: Advances in Swarm Intelligence, 243–252. Springer, Heidelberg

Enhancement of Voltage Stability Using FACTS Devices in Electrical Transmission System with Optimal Rescheduling of Generators by Brain Storm Optimization Algorithm G. V. Nagesh Kumar, B. Sravana Kumar, B. Venkateswara Rao and D. Deepak Chowdary Abstract Voltage stability is the capability of the power system to preserve the system under stable condition even exposed to small disturbances under normal or slightly over loaded conditions. Maintaining voltage stability is the one of the major factor for power system networks. In this chapter voltage stability has been examined by using Flexible Alternating Current Transmission System (FACTS) devices. These devices enhance the stability of the system. With proper generation reallocation, further enhance the stability of the system. It can be achieved by using latest swarm intelligence algorithm technique called Brain Storm Optimization (BSO) algorithm. Results attained by BSO are related to that attained by Genetic Algorithm (GA) in both without and with FACTS conditions. These results show that BSO produce better results compared to GA for solving optimal rescheduling of generators. The BSO algorithm based generation reallocation has been examined and tested on IEEE 30 bus system without and with FACTS devices. Keywords BSO algorithm · FACTS device · Generation reallocation · Voltage stability

G. V. N. Kumar (B) JNTUA College of Engineering, Pulivendula, India e-mail: [email protected] B. S. Kumar GITAM University, Visakhapatnam, India e-mail: [email protected] B. V. Rao V R Siddhartha Engineering College (Autonomous), Vijayawada, India e-mail: [email protected] D. D. Chowdary Dr. L.Bullayya College of Engineering (for Women), Visakhapatnam, India e-mail: [email protected] © Springer Nature Switzerland AG 2019 S. Cheng and Y. Shi (eds.), Brain Storm Optimization Algorithms, Adaptation, Learning, and Optimization 23, https://doi.org/10.1007/978-3-030-15070-9_11

273

274

G. V. N. Kumar et al.

1 Introduction A nation is dependent on electric energy for its overall growth and development. The prime reason for its essentiality and popularity is attributed to its massive scale of generation at a single place. It can also be transmitted to long distances at low cost. Most importantly, the adaptability and efficiency of the electric energy makes it easier to use it at different levels of applications. It can be utilized in jobs related to lighting and mechanical drives. Normally, electric force is acquired through transmutation from fossil fuels (like coal, oil, natural gas, etc.), nuclear and water-related resources. Electricity is derived when the combusting fossil fuels release thermal energy. Actually, in the process of fission in nuclear materials, there is a conversion of thermal force into mechanical energy via thermo-cyclic process. This energy in mechanical form is later converted into electric energy via generators. Fossil fuels and nuclear materials form the non-renewable resources of earth that cannot be replenished. The imbalance in natural distribution of such resources has left some countries with excess availability and others with inadequate supply. Even though hydro energy is renewable and easily available, it has other limitations in terms of power generation [1, 2]. Currently, the ever-growing complexities of the huge interconnected networks in most of the developing nations have lowered the dependability on the power supply. Hence, the power system engineers at present are bound to face a lot of issues like system unsteadiness, absence of appropriate control on the flow of power and most importantly, there are many system security problems [3, 4]. Consequently, these issues eventually make way to a multitude of blackouts at different parts of the globe. A blackout is a power outage, which means that there is a loss of electric supply to a part or the whole of the power system. This blackout can cause power system equipment damage, heavy economic losses, risk of poor economy functioning and life paralysis in remote parts of the country. It is necessary to notice that everyone is dependent on reliable and quality power supply [5]. Several power system blackouts have been happening in the world till date. These disturbances are caused due to various natural causes like lightning, rain, snow, wind storm, or technical failures such as transformer faults, short circuits, or human errors like error of judgment, communication errors between operators, or terrorism acts like bombing and cyberattacks [6]. These blackouts occur due to defective planning and execution, weak linkage in the power system, insufficient care and overburden in the grid. To ensure that these sorts of barriers are effectively handled and to make for a satisfactory supply of reliable power, installing fresh supply lines is by far one of the best solutions [7, 8]. However, for such an installation with such largely interconnected power system, there are some constraints in terms of economic viability and environmental issues. All such complexities in the process of installation of new transmission lines in a power system throw a challenge to the power engineers who explore paths to increase free-flowing power and to ensure reliability and security of the system.

Enhancement of Voltage Stability Using FACTS Devices …

275

Free-flowing power depends on transmission line resistance capacity and magnitude of absorbing end voltages, by taking the phase angle between voltages into consideration. By employing various approaches, we can secure the heavy-loaded branches. Also, optimum rearrangement of flow of power generation is the most appropriate methodology to enhance employment in the present system. To optimize is to acquire maximum output under the current circumstances. In spite of optimization of the system of power by general reallocation, there exists insufficiency and it needs to be properly refined. It’s a general routine in power systems to have installed shunt capacitors that support system voltages at correct levels. Transmission line reactance is lowered by using series capacitors and thus, enhancing the power transfer capability of the lines. Power flows in transmission lines are controlled through the application of phase-shifting transformers, which is done by getting in an extra phase shift in the middle of dispensing and absorbing point voltages. In the earlier days, similar equipment’s were managed manually and hence, they were extremely slow. The problem of security for a power system shall be appropriately addressed once mechanically-controlled systems are built for faster response. This further increases the system capability and maintains sufficient steadiness. With the advancement of power electronics, one of the innovative ways is to use Flexible AC Transmission Systems or simply FACTS. They were introduced by Electric Power Research Institute (EPRI) in the year 1980. They help in better productive use of existing resources of power systems by ensuring proper maintenance and security [9–11]. Two fundamental aims comes in FACTS system: the movement capacity of system of transfer is enhanced and to limit voltage constraints. Due to economic constraints of facts controllers, their utilization in power system is limited. In order to achieve results at optimum and productive levels of current power system network the positioning and fine tuning of facts controllers of power systems need optimization. Incorporating facts devices, in this chapter swam optimization algorithm called BSO algorithm has been used for solving multi objective OPF problem. This chapter aim is to choose the best values for all generator buses and the impact of FACTS device on generator reallocation. Multi objective function is considered, to minimize real power generation cost and active power losses in the system, and it is minimized subject to power balance constraint, active and reactive power generation limits voltage limits and transmission line limits and FACTS parameter limits. For IEEE 30 bus system, computer simulation using mat lab is done. The obtained results are compared with Genetic Algorithm (GA) The remaining organization of the chapter is as follows. FACTS Devices in Sect. 2; Multi objective OPF problem formulation for general reallocation in Sect. 3; Sect. 4 presents the implementation of swarm optimization algorithm to the problem; Numerical results and discussion are followed in Sect. 5. Section 6 presents’ further research directions finally, the conclusion is given in Sect. 7.

276

G. V. N. Kumar et al.

2 Facts Devices Modelling 2.1 Modelling of Static VAR Compensator Static VAR Compensator is a famous shunt-connected FACTS controller. It usually comes in dual designs: Fixed-capacitor–Thyristor controlled reactor (FC–TCR) and Thyristor-switched-capacitor–Thyristor controlled reactor (TSC–TCR). One very prominent configuration is a blend of both the fixed capacitor and thyristor-controlled reactor [12–14]. SVC has been designed in the form of variable susceptance equipment, and its value is optimized to accomplish a determined voltage scale which matches with the limitations. The SVC device has the ability to control reactive power (capacitive) as well as to draw in the power (inductive), given the system requisites. The SVC-positioned at a bus is adjusted till the voltage at the bus turns 1.0 p.u. The variable susceptance method is displayed in Fig. 1. Current taken in by the SVC is Isvc = j Bsvc Vk

(1)

SVC taken in or controlled reactive power at bus k is Q svc = −Vk2 Bsvc

(2)

The real as well as reactive power changes at kth bus connected to the susceptance of SVC have been furnished with the Eq. 3. The total susceptance Bsvc is accepted as state variable. 

Pk Q k

Fig. 1 Variable shunt susceptance

i

 =

0 0 0 Qk

i 

θk Bsvc /Bsvc

i (3)

Enhancement of Voltage Stability Using FACTS Devices …

277

Susceptance value from SVC gets altered in accordance with the optimization rules after every iteration. The SVC-linked power equations in Eq. 3 get mixed along with power system equations and are shown in Eq. 4 [F(x x)] = [J ] [X ]

(4)

T  [F(x x)] = Pk Pm Q k Q m

(5)

where,

P and Q are the power mismatch equations, and superscript ‘T ’ indicates transposition. T  [X ] = θk θm Vk Vm

(6)

[X] is the solution vector, [J] is the Jacobian matrix, Here, V k is kept constant at 1 p.u. The altered Jacobian matrix is given as ⎡ ∂ Pk ⎢ ⎢ [J ] = ⎢ ⎣

∂θk ∂ Pm ∂θk ∂ Qk ∂θk ∂ Qm ∂θk

∂ Pk ∂θm ∂ Pm ∂θm ∂ Qk ∂θm ∂ Qm ∂θm

∂ Pk ∂ Vk ∂ Pm ∂ Vk ∂ Qk ∂ Vk ∂ Qm ∂ Vk

∂ Pk ∂ Vm ∂ Pm ∂ Vm ∂ Qk ∂ Vm ∂ Qm ∂ Vm

⎤ ⎥ ⎥ ⎥ ⎦

(7)

The initial values of the SVC susceptance is considered to be B = 0.02 p.u, Bmin = −1.0 p.u, Bmax = 1.0 p.u [15, 16].

2.2 Modelling of TCSC The primary Thyristor-controlled series-connected capacitor system put forth by Vithaythil and the rest in 1986 was a technique related to “rapid adjustment of network impedance”. In addition to controlling the line power moving capacity, the TCSC goes on to enhance the system firmness. The primary unit of TCSC is displayed through Fig. 2. It makes up a series recompensing capacitor that is shunted via the thyristor controlled reactor. Thyristor incorporation in TCSC module facilitates smooth and steadier control of reactance against system criteria differences. In case of a massive power system, TCSC execution needs numerous similar primary compensators in order to get linked in series for acquiring required voltage rating as well as operating traits [17–19]. It has been modelled in the form of a controllable reactance, gets included in series with the transmission line to fine-tune line impedance and thus controls the power flow.

278

G. V. N. Kumar et al.

Fig. 2 Basic TCSC model

Fig. 3 Block diagram of TCSC

Here, there is a straight fine-tuning of the reactance of the transmission line with the use of TCSC [20–22]. The TCSC is modelled as varied impedance and its rating relies on the reactance of the transmission line where the TCSC is situated. From Fig. 3, the impedance equations are written as in Eqs. 8–10. Z i j = Ri j + X i j

(8)

Z line = Rline + X line

(9)

X i j = X line + X TCSC

(10)

where X TCSC is reactance of TCSC, without over compensation, the working range of the TCSC is selected to lie within −0.8X line and 0.6X line. The transfer admittance matrix of the TCSC is given by 

Ii Ij



 =

j Bii j Bi j j B ji j B j j



Vi Vj

 (11)

For capacitive operation, equations are given in (12) and (13) Bii = B j j =

1

X TCSC 1 Bi j = B ji = − X TCSC

(12) (13)

For inductive operation, the signs are inverted or reversed. The active and reactive power equations at bus k are: Pi = Vi V j Bi j sin(θi − θ j )

(14)

Enhancement of Voltage Stability Using FACTS Devices …

Q i = −Vi2 Bii − Vi V j Bi j cos(θi − θ j )

279

(15)

When the series reactance controls the amount of active power flowing from bus i to bus j the change in reactance of TCSC is: (i−1) i − X TCSC X TCSC = X TCSC

(16)

Based on optimization rules, the state variable XTCSC of the series controller is updated.

2.3 Modelling of Unified Power Flow Controller The unified power flow controller (UPFC) is proposed by Gyugyi in 1991 [23]. UPFC device is used for the real time control and dynamic compensation of ac transmission system, providing multifunctional flexibility required solving many of the problems facing the delivery industry. To solve reactive power compensation and voltage stability enhancement problems in power system multi-functional control is provided by UPFC multiple accepts of power system is controlled by UPFC, viz., voltage magnitude, line impedance and phase angle either individually or in combination thus improving the performance of the systems. Hence the term ‘unified’ has been designated. The UPFC not only performs the functions of STATCOM, SSSC, and the phase angle regulator but also provides additional flexibility by combining some of the functions of these controllers [24]. Two voltage source inverters (VSIs) are the main components of UPFC that shares a common dc storage capacitor, and connected to the power system through coupling transformers. one is connected in series through a series transformer and it can be used to control the real and reactive line power flow inserting required voltage with controllable magnitude and phase in series with the transmission line, while the other one VSI is connected in shunt to the transmission system via a shunt transformer and it is used for voltage regulation at the point of connection injecting required reactive power flow into the line and to balance the real power flow the exchanged between the series inverter and the transmission line. Thereby, the UPFC can fulfill functions of reactive shunt compensation, series compensation and phase shifting. This arrangement functions as an ideal ac to ac power converter in which the real power can freely flow in either direction between the ac terminals of two converters and each converter can independently generate or absorb reactive power at its own ac output terminals (Fig. 4). UPFC voltage sources are written in Eqs. 17 and 18 Vv R (cos δv R + j sin δv R )

(17)

Vc R (cos δc R + j sin δc R )

(18)

280

G. V. N. Kumar et al.

Fig. 4 Basic circuit arrangement of unified power flow controller Fig. 5 Equivalent circuit of the unified power flow controller

where V vR and δv R are the controllable voltage magnitude and phase angle of the voltage source representing the shunt converter. Similarly, V cR and δc R are the controllable voltage magnitude and phase angle of the voltage source representing the series converter. The source impedance is considered to be resistance less (Fig. 5). The active and reactive power equations are At bus k Pk = [Vk Vm Bkm sin(θk − θm )] + [Vk Vc R Bkm sin(θk − δc R )] + [Vk Vv R Bv R sin(θk − δv R )]

(19)

Q k = −Vk2 Bkk − [Vk Vm Bkm cos(θk − θm )] − [Vk Vc R Bkm cos(θk − δc R )] − [Vk Vv R Bv R cos(θk − δv R )] (20) The starting values of the UPFC voltage sources are taken to be V cr = 0.06 p.u, δ cr = 90°, V vr = 1 p.u. and δ vr = 0°. The source impedances are taken as Z cr = Z vr = 0.1 p.u.

Enhancement of Voltage Stability Using FACTS Devices …

281

3 Problem Formulation The prime motto of the present work is to find out the production reallocation and optimum values for parameter settings of the FACTS devices in the power system for regular condition. This can be composed as below: Minimize f (x) Subject to g(x) = 0, and h(x) < 0, xl ≤ x ≤ xu Here, function f (x) is a scalar, pointing at the sum total of real power generation cost, total voltage deviation and real power losses, g(x) is an equality constraint (which are nothing but the power flow equations), h(x) is an inequality constraint (which refer to the limitations of the control variables) and ‘x’ is the state variable vector which includes both the controllable and dependent variables (real power generation, reactive power generation, generator bus voltages, shunt susceptance, line reactance, shunt voltage and series converter voltages). The lower value and upper value limits are represented by xl and xu. The process of finding the solution encompasses optimizing the objective function and satisfying the constraints. The generation re-allocation and optimal sizing and placement of the FACTS devices, in the present work, are addressed by minimizing the multi-objective function subjected to network constraints. Objective function: Min F = Min(W1 ∗ F1 + W2 ∗ F2 + W3 ∗ F3 )

(21)

where, F 1 is the Real power loss, F 2 is the real power generation cost and F 3 is the Voltage deviation and W 1 , W 2 and W 3 are the weighting factors. W1 + W2 + W3 = 1

(22)

W1 = 0.7W2 = 0.15W3 = 0.15

(23)

Real Power Loss: F 1 is the Real power losses in transmission lines. This can be expressed as  ntl    i i F1 = min real S jk + Sk j

(24)

i=1

where ntl is number of transmission lines, S jk is the total complex power flows from bus j to bus k in line i.

282

G. V. N. Kumar et al.

Real power generation cost: F 2 represents the real power generation expenses and it can be presented using the below mentioned quadratic equation:   ng   2 F2 = min ai + bi PG i + ci Pcii (25) i=1

ng is the number of generators in the power system and a, b, c are the fuel-cost coefficients. Voltage Deviation: F 3 is the Voltage deviation, to acquire appropriate voltage performance; the voltage deviation at each bus is needed to be as minimal as possible. This Voltage Deviation (VD) can be presented as below:  N bus     2 2 F3 = min(V D) = min Vk − Vkne 

(26)

k=1

V k is the actual value of voltage magnitude value at bus k and V kref points to the reference value for voltage magnitude at the bus. Equality constraints: Power Balance Constraint N 

PGi =

i=1 N  i=1

N 

PDi + PL

(27)

Q Di + Q L

(28)

i=1

Q Gi =

N  i=1

where i = 1, 2, 3, …, N and N = no. of. Bus PL QL PGi QGi PDi QDi N

pinpoints to the active power loss of the system, refers to the total reactive power losses, indicates the active power generated at bus i, is the reactive power generated at bus i, is the power demand at bus i, is the power demand at bus i, is the number of buses.

Inequality constraints: Voltage balance constraint min ≤ V ≤ V max VGi Gi Gi

(29)

Enhancement of Voltage Stability Using FACTS Devices …

283

where Gi = 1, 2, 3, …, ng and ng is number of Generator buses. Generation limit real power: min ≤ P ≤ P max PGi Gi Gi

(30)

where, Gi = 1, 2, 3, …, ng and ng is number of generators buses. The voltage limits of the generator buses are taken between 0.9 p.u and 1.1 pu. Reactive Power generation limits: min Q Gi ≤ Q Gi ≤ Q max Gi

(31)

The limitations of FACTS controllers SVC, TCSC and UPFC attach to the aforementioned disadvantages in deciding generation reallocation and optimum sizing of the device with methods to optimize. SVC Limit min ≤ B ≤ B max Bsvc svc svc

(32)

min ≤ X ≤ X max X tcsc tcsc tcsc

(33)

TCSC Limits

UPFC Limits V vr is the shunt converter voltage magnitude (p.u) Vvrmin ≤ Vvr ≤ Vvrmax

(34)

V cr is the series converter voltage magnitude (p.u) Vcrmin ≤ Vcr ≤ Vcrmax

(35)

4 Swarm Optimization Algorithm (BSO Algorithm) A new kind of swarm intelligence algorithm is the Brain Strom Optimization (BSO) algorithm, which is based on collective behavior of human being that is, brainstorming process. Two major operations are involved in BSO algorithm i.e., convergent operation and divergent operation. A “good enough” optimum could be obtained through recursive solution divergence and convergence in the search space. The capability of both convergence and divergence will be seen in designed optimization algorithm. BSO algorithm possesses two kinds of functionalities: capability learning and capacity developing. The capability learning corresponds to the divergent operation while capability developing corresponds to the convergent operation.

284

G. V. N. Kumar et al.

The area(s) where higher potential solutions may exist due to capacity developing focuses on moving the algorithm’s search while the current solution for single point based Swarm intelligent algorithms and from the current population of solutions for population-based Swarm intelligence algorithm. Due to the capability learning focuses on its actual search towards new solution(s). The capability learning and capacity developing recycle to move individuals towards better and better solutions. The BSO algorithm, therefore, can also be called as a developmental brain storm optimization algorithm. The top level learning or macro-level learning methodology is capacity developing. The capacity developing describes the learning ability of an algorithm to adaptively change its parameters, structures, and/or its learning potential according to the search states of the problem to be solved. In other words, search potential possessed by an algorithm is the capacity developing. The capability learning is a bottom-level learning or micro-level learning. The capability learning describes the ability for an algorithm to find better solution(s) from current solution(s) with the learning capacity it possesses. A combination of Swarm intelligence and data mining techniques are seen in BSO algorithm. Every individual in the brain storm optimization algorithm is not only a solution to the problem to be optimized, but also a data point to reveal the landscapes of the problem. The data mining techniques can be combined to produce benefits above and beyond what either method could achieve alone. BSO algorithm: 1. 2. 3. 4.

‘n’ potential solutions (individuals) are randomly generated Cluster n individuals into m clusters; The n individuals are evaluate The best individual as cluster centre in each cluster by ranking individuals in each cluster and Record; 5. A value between 0 and 1 are randomly generated; (a) If the value is smaller than a pre-determined probability p5a, i. Randomly select a cluster centre; ii. Randomly generate an individual to replace the selected cluster centre; 6. New individuals are generated (a) A value between 0 and 1 are randomly generated. (b) If the value is less than a probability p6b, i. a cluster with probability p6bi is randomly selected; ii. a random value between 0 and 1 is generated; iii. If the value is smaller than a pre-determined probability p6biii, (1) Select the cluster center and add random values to it to generate new individual; iv. Otherwise randomly select an individual from this cluster and add random value to the individual to generate new individual. (c) Otherwise select two clusters to generate new individuals randomly individual i. a random value is generated; ii. If it is less than a pre-determined probability p6c, the two cluster centres are combined and then added with random values to generate new individual; iii. Otherwise, two individuals

Enhancement of Voltage Stability Using FACTS Devices …

285

from each selected cluster are randomly selected to be combined and added with random values to generate new individual. (d) The existing individual is compared with the newly generated individual with the same individual index, the better one is kept and recorded as the new individual; 7. If n new individuals have been generated, go to step 8; otherwise go to step 6; 8. If pre-determined maximum number of iterations has been reached then terminate; otherwise go to step 2.

5 Results and Discussion In the IEEE 30 bus system, bus 1 is deemed to be slack bus and buses 2, 5, 8, 11, 13 are the generator buses and further buses could be load buses and 41 transmission lines. Considering the lowering of the objective function utilizing BSO and GA Algorithms. The results are examined employing MATLAB software. Tables 1 and 2 specify the criteria of the presented algorithms. Table 3 displays generator characteristics related to IEEE 30 bus system that showcases cost coefficients and, minimum and maximum limitations of generation of each generator buses. Objective Function Value with the Variation of BSO Algorithm Parameters Various combinations of m and p6b are used and the obtained values of the objective function are furnished in Fig. 6. It’s identified that at values of m = 5 and p6b = 0.8, the lowest value of the objective function is acquired. A varied combination of weights of the objective function is employed and, thus obtained values of objective function are identified and organized in Table 4. It is

Table 1 Input criteria of BSO algorithm S. no

Criteria

Quantity

1

No. of individuals n

20

2

No. of clusters m

5

3

Probability p6b

0.8

4

Pre-determined probability p5a

0.2

5

Number of iterations

50

Table 2 Input criteria of genetic algorithm S. no

Criteria

Quantity

1

Population size

20

2

Number of generations

50

3

Stall generation limit

100

4

Stall time limit

300

286

G. V. N. Kumar et al.

Table 3 Generator characteristics of IEEE 30 bus system Generator bus no

A ($/MW2/h)

B ($/MW/h)

C ($/h)

PGmin (MW)

PGmax (MW)

1

0.005

2.45

105

10

200

2

0.005

3.51

44.1

10

50

5

0.005

3.89

40.6

10

50

8

0.005

3.25

0

10

50

11

0.005

3

0

10

100

13

0.005

2.45

105

0

10

190.6 ObjecƟve funcƟon

190.58 190.56 190.54 190.52 190.5 190.48 190.46 190.44

Best m=10 P6b=0.1

m=10 P6b=0.8

m=5 P6b=0.1

m=5 P6b=0.8

Fig. 6 Objective function value with the variation of BSO algorithm parameters Table 4 Weights versus objective function analysis Solution number

Weight W1

W2

W3

F 1 (objective function)

1

0.7

0.15

0.15

190.5

2

0.55

0.3

0.15

379.52

3

0.4

0.45

0.15

567

4

0.25

0.6

0.15

773.56

5

0.1

0.75

0.15

958.9

6

0.3

0.4

0.3

509.2

identified that at W 1 = 0.7, W 2 = 0.15 and W 3 = 0.15, the lowest value of objective function is generated. Hence, it is chosen for the study.

Enhancement of Voltage Stability Using FACTS Devices …

287

Generation Reallocation Employing BSO and GA Algorithms Considering Multiple Objective Function on IEEE 30 Bus System, Excluding and Including SVC The charges for whole real power generation, voltage deviation and real loss of power within the system are deliberated as the objective function. Power Flow at optimum level is performed through acquiring the objective function with limitations as given in Eq. 21. In this, SVC is positioned at Bus number 30 in IEEE 30 Bus system. By employing BSO and GA algorithms, sizing and generation reallocation is acquired. Firstly, the objective function is advised and examined for IEEE 30 bus system excluding SVC device and later, extended by putting SVC. The outcomes with reallocation of power generation, that is, including both the real and reactive power along with the active power losses, voltage deviation and production price value of the total lines are shown in Table 5 with respect to BSO and GA algorithms, while excluding as well as including SVC. In Table 5, voltage deviation, real and reactive supply losses, real power production in the system as well as at individual generators, and the real power production outlay with BSO-OPF excluding SVC, GA-OPF excluding SVC, BSO-OPF including SVC as well as GA-OPF including SVC are compared. It’s pointed that the BSO Algorithm serves a better purpose towards multi-objective optimization problem selected, unlike the other method GA. Additionally, it is identified that OPF with SVC is more apt and

Table 5 Contrast of objective function criteria employing BSO-OPF excluding and including SVC Parameter

BSO-OPF without SVC

GA-OPF without SVC

BSO-OPF with SVC

GA-OPF with SVC

PG1

106.63

126.65

108.53

165.94

PG2

19.660

27.337

21.288

45.707

PG5

35.021

27.334

23.33

28.198

PG8

50

21.327

49.99

10.194

PG11

68.314

84.822

75.51

34.510

PG13

10

3.9926

10

6.6718

Total real power generation (MW)

289.63

291.47

288.64

291.22

Total real power loss (MW)

6.2326

8.0713

5.2663

7.8267

Total reactive power loss (MVAR)

19.81

35.35

12.962

15.55

Voltage deviation

1.8229

2.5

0.2836

0.2853

Total real power generation cost ($/h)

1354.4

1366.9

1254

1283

Rating of SVC (p.u)

–

–

0.0678

0.0670

Real power generation (MW)

288 Fig. 7 Comparison of objective function value for BSO and GA algorithms

G. V. N. Kumar et al.

MulƟ ObjecƟve funcƟon value (p.u.) 215 210 205 200 195 190 185 180

211

207.79 197.9 192.4

GA-OPF without SVC

BSO-OPF without SVC

GA-OPF with SVC

BSO-OPF with SVC

efficient in contrast to the one without SVC. Hence, the device has more efficiency with respect to the optimization of generators. Comparison of Objective Function Value for BSO with GA Algorithm Without and With SVC Figure 7 compares the values of objective function excluding as well as alongside SVC positioning and its sizing, with BSO and GA Algorithms. Performance of OPF in the presence of optimally positioned and tuned SVC with BSO algorithm decreases the value of objective function significantly. Different values of the voltages of OPF, excluding and including SVC are compared within Table 6 and they are graphically depicted within Fig. 8. It is found that the OPF with SVC increases the voltage on the buses. Comparative Analysis for GA and BSO Optimization Algorithms Without and With TCSC A comparative analysis has been executed for GA and BSO algorithm for Optimal Power Flow problem. This analysis demonstrated that BSO algorithm showcased much better performance than the GA algorithm (Table 7). Generation Reallocation Using GA and BSO Algorithms with Multi Objective Functions Without and With UPFC In the IEEE 30 bus system considered for the analysis, bus 1 is the slack bus and buses 2, 5, 8, 11, 13 are generator buses. The other buses are load buses and there are a total of 41 transmission lines. By employing GA and BSO algorithms, sizing and generation reallocation can be acquired. Firstly, the objective function is taken and then, examined for IEEE 30 bus system excluding UPFC device, later stretched by including UPFC. Outcomes in connection with reallocation of power generation, that is, including the two real and reactive power with active power losses, voltage deviation and production price value for total lines are organized in Table 8 regards GA and BSO algorithms, excluding as well as including UPFC.

Enhancement of Voltage Stability Using FACTS Devices …

289

Table 6 Comparison of voltage profile using BSO algorithm without and with SVC Bus no

BSO without SVC Voltage magnitude (p.u)

BSO-OPF with SVC Voltage angle (°) 0

Voltage magnitude (p.u) 1.06

Voltage angle (°)

1

1.06

2

1.0252

−1.8211

1.043

−2.19241

0

3

0.9999

−2.2179

1.032464

−2.58981

4

0.9858

−2.6673

1.02583

−3.14166

5

0.9492

−6.1107

1.01

−7.3294

6

0.9732

−2.9144

1.018491

−3.37556

7

0.9486

−4.6756

1.001091

−5.39859

8

0.9637

−2.3591

1.01

−2.87524

9

0.9724

−1.2716

1.026796

−1.27151

10

0.9425

−5.0755

1.009956

−4.74134

11

1.0426

6.786

12

0.938

−5.7211

1.028673

13

0.9379

−4.8093

1.071

−5.16438

14

0.924

−6.5367

1.011767

−6.51736

15

0.9236

−6.4066

1.007671

−6.25997

16

0.9322

−5.807

1.013744

−5.70103

17

0.9336

−5.5568

1.005728

−5.2517

18

0.9167

−6.8072

0.995965

−6.46723

19

0.916

−6.8027

0.992253

−6.38438

20

0.9218

−6.4404

0.995903

−6.03266

21

0.9229

−5.9967

0.995362

−5.63329

22

0.9306

−5.6992

1.000639

−5.34698

23

0.9216

−6.0974

0.995342

−5.75186

24

0.9141

−6.4015

0.987346

−6.05069

25

0.9181

−7.0152

0.993477

−6.78607

26

0.8984

−7.5326

0.975358

−7.22652

27

0.9304

−7.0605

1.006513

−6.94978

28

0.9664

−3.1812

1.015317

−3.65719

29

0.9084

−8.5545

0.998359

−8.56787

30

0.8956

−9.6344

1

−9.84495

1.082

6.856342 −5.89248

290

G. V. N. Kumar et al.

Table 7 Comparison of objective function parameters using GA and BSO-OPF considering without and with TCSC for without contingency—IEEE 30 bus system Parameter

BSO-OPF without UPFC

GA-OPF without UPFC

BSO-OPF with UPFC

GA-OPF with UPFC

PG1

106.636

126.656

102.33

130.398

PG2

19.6605

27.3374

27.546

11.9481

PG5

35.0217

27.3348

24.96

20.945

PG8

50

21.3279

50.89

23.1834

PG11

68.3142

84.8224

70.75

90.9945

PG13

10

3.9926

8.9

11.0686

Total real power generation (MW)

289.632

291.471

288.476

289.538

Total real power loss (MW)

6.2326

8.0713

5.39

7.9138

Total reactive power loss (MVAR)

19.81

35.35

11.45

25.14

Voltage deviation (p.u.)

1.8229

2.5013

0.3851

0.3859

Total real power generation cost ($/h)

1354.4

1366.9

1264.3

1285

Rating of TCSC (p.u)

–

–

Ptcsc = 0.4314 Qtcsc = 0.1786

Ptcsc = 0.5663 Qtcsc = 0.1487

Objective function

211

207.79

198

194

Real power generation (MW)

Table 8 Contrast of objective function parameters of multi objective optimization using BSO-OPF without and with UPFC Parameter

BSO-OPF without UPFC

GA-OPF without UPFC

BSO-OPF with UPFC

GA-OPF with UPFC

PG1

106.636

126.656

106.33

131.398

PG2

19.6605

27.3374

23.546

12.9481

PG5

35.0217

27.3348

26.96

23.945

PG8

50

21.3279

48.89

19.1834

PG11

68.3142

84.8224

72.75

96.9945

PG13

10

3.9926

10

5.0686

Total real power generation (MW)

289.632

291.471

288.476

289.538

Total real power loss (MW)

6.2326

8.0713

5.09

6.138

Total reactive power loss (MVAR)

19.81

35.35

11.45

25.14

Voltage deviation (p.u.)

1.8229

2.5013

0.2651

0.273

Total real power generation cost ($/h)

1354.4

1366.9

1246.3

1260

Real power generation (MW)

Enhancement of Voltage Stability Using FACTS Devices … Fig. 8 Comparison of voltage magnitude of reallocation of generators without and with SVC

29

291

30 1.1

1

2

3

1

28

4

0.9

27

5

0.8

26

6

0.7 25

7

0.6 0.5

24

8

0.4 23

9

22

10 21

11 20

12 19

13 18

17

15 16 Voltage BSO in absence of SVC (p.u.)

14

Voltage BSO-OPF in presence of SVC (p.u.)

All the values presented in the above Table 8 in terms of parameters like real power generation of the system and at each single generator, real and reactive power deprivation, voltage deviation and real power production cost for BSO-OPF excluding UPFC, GA-OPF excluding UPFC, BSO-OPF along with UPFC and GA-OPF along with UPFC are compared. The following identifications can be made from the above table, BSO Algorithm better suited for the multi-objective optimization problem and not the GA. OPF with UPFC gives better results in contrast to the one without UPFC ensuring that the device is extremely efficient in optimization. Various values of objective function with and without UPFC positioning and sizing along with BSO and GA Algorithms are compared and analysed in the Fig. 9. It also conveys that the effective functioning of OPF in the availability of optimally positioned and tuned UPFC with BSO algorithm drops the value of objective function significantly. Comparison of Objective Function Value for BSO with GA Algorithm Without and With UPFC Contrast of Voltage Profile Using BSO Algorithm Without and With UPFC Voltage profile is shown and compared in Table 9 for OPF excluding as well as including UPFC. The above table clarifies that the OPF in the availability of UPFC along with BSO algorithm shoots up the voltage profile.

292

G. V. N. Kumar et al.

Table 9 Comparison of voltage profile using BSO algorithm without and with UPFC Bus no

BSO-OPF without UPFC

BSO-OPF with UPFC

Voltage magnitude (VM) (V)

Voltage magnitude (V)

Voltage angle (°) 0

Voltage angle (°)

1

1.06

1.06

0

2

1.0252

−1.8211

1.043

−2.1186

3

0.9999

−2.2179

1.0323

−2.593

4

0.9858

−2.6673

1.0257

−3.1456

5

0.9492

−6.1107

1.01

−7.0744

6

0.9732

−2.9144

1.0183

−3.3862

7

0.9486

−4.6756

1.001

−5.3013

8

0.9637

−2.3591

1.01

−2.9132

9

0.9724

−1.2716

1.0269

−1.4788

10

0.9425

−5.0755

1.0097

−4.8861

11

1.0426

6.786

1.082

6.3491

12

0.938

−5.7211

1.0287

−5.9745

13

0.9379

−4.8093

1.071

−5.2463

14

0.924

−6.5367

1.0117

−6.6099

15

0.9236

−6.4066

1.0075

−6.362

16

0.9322

−5.807

1.0136

−5.8092

17

0.9336

−5.5568

1.0055

−5.3855

18

0.9167

−6.8072

0.9957

−6.5846

19

0.916

−6.8027

0.992

−6.5109

20

0.9218

−6.4404

0.9956

−6.1638

21

0.9229

−5.9967

0.995

−5.7658

22

0.9306

−5.6992

1.0001

−5.4804

23

0.9216

−6.0974

0.995

−5.8802

24

0.9141

−6.4015

0.9865

−6.1674

25

0.9181

−7.0152

0.9916

−6.8573

26

0.8984

−7.5326

0.9734

−7.2995

27

0.9304

−7.0605

1.0039

−6.9943

28

0.9664

−3.1812

1.0149

−3.6723

29

0.9084

−8.5545

0.9966

−9.2958

30

0.8956

−9.6344

1

−11.3162

Enhancement of Voltage Stability Using FACTS Devices … 215

211

MulƟ ObjecƟve funcƟon value (p.u.)

207.79

210

293

205 200 193

195

190.5

190 185 180 GA-OPF without UPFC

BSO-OPF without UPFC

GA-OPF with UPFC

BSO-OPF with UPFC

Fig. 9 Contrast of objective function value for BSO and GA algorithms Table 10 BSO algorithm without and with SVC, TCSC and UPFC Algorithm

Parameter

SVC

TCSC

BSO algorithm

Power loss (MW)

5.266325

5.39

Voltage deviation (p.u.)

0.283655

0.3851

Generation cost ($/h)

1254

1264.3

Objective function

192.4

194

UPFC 5.09 0.2651 1246.3 190.5

In Table 9, different voltage values or the voltage profile for OPF excluding as well as including UPFC are compared and examined. It’s understood that the OPF in the presence of UPFC along with BSO algorithm kicks up the voltage profile. Comparison of the Proposed Method for SVC, TCSC and UPFC See Table 10. Convergence Characteristics of the Proposed Method for SVC, TCSC and UPFC See Figs. 10, 11 and 12.

294

G. V. N. Kumar et al.

ObjecƟve funcƟon value

BSO with SVC 192.8

BSO with SVC

192.6 192.4 192.2 192

No of IteraƟons 1 3 5 7 9 1113151719212325272931333537394143454749

ObjecƟve funcƟon value

Fig. 10 Convergence characteristics of IEEE-30 bus system with BSO-OPF with SVC 194.6

BSO with TCSC BSO with TCSC

194.4 194.2 194 193.8

No of IteraƟons 193.6 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

ObjecƟve FuncƟon value

Fig. 11 Convergence characteristics of IEEE-30 bus system with BSO-OPF with TCSC

BSO with UPFC BSO with UPFC

191 190.8 190.6 190.4

No of IteraƟons

190.2 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

Fig. 12 Convergence characteristics of IEEE-30 bus system with BSO-OPF with UPFC

6 Future Research Directions The viable suggestions to extend the work introduced and studied in this chapter are as below: • In this research, all loads are taken to be constant power type. More practical load models, taking into consideration their voltage and frequency dependency might be contemplated in future study. • Validation of proposed positioning strategy should be executed on large power systems and validation of placement strategy for dynamic system analysis has to be undertaken employing dynamic stability simulation packages.

Enhancement of Voltage Stability Using FACTS Devices …

295

• Multiple FACTS devices can be positioned in the power system to enhance the system performance with the identification of new indices. • Other methodical metaheuristic techniques could be attempted as well. • In this chapter, cost of the FACTS devices is not included in the OPF formulation. The cost may also be considered while examining its influence over the system performance.

7 Conclusion One of the most vital requisites for efficient utilization of the resources of power systems is the optimum level of power flow. These days, most power systems from emergent nations with hugely interconnected networks divide generation reserves to increase the dependability of the power systems. Even then, the escalating complexities within hugely interconnected networks and the fluctuations in the dependability of supply of power, system instability, problems in controlling the flow of power and issue relating to security are the status quo which manifest as blackouts in various nook and corners of the world. Systematic errors of planning and operation, feeble interconnection of the power system, meagre standards of maintenance or overload of networks can be identified as some of the causes leading to the aforementioned faults. Installations of new transmission lines are the prime requisites to reduce the above mentioned faults and to ensure provision of desired flow of power by ensuring stability and dependability of the system. But, some aspects such as economical viability, issues related to environment etc., pose a constraint for the installation of new lines of transmission with the hugely interconnected power system. Power engineers are facing challenges due to such complexities in the process of installation of new transmission lines in the power system and to continue their research on the ways of increasing flow of power with the available lines of transmission without lowering the stability of system and without any security issues as well. One of the best solutions to conquer the power system issues is by optimization of power generated in that system. Optimization can be described as acquiring the maximum output from the existing resources. A wide range of methods are described all throughout the literature. In order to achieve optimization of the IEEE 30 Bus system, technique such as swarm algorithm that includes BSO is employed. This algorithm, with a motive of subsiding the price of real power generation, active power losses and deviation of voltage values, are employed to acquire optimal power generation reallocation. Results derived from BSO are found to be superior to the outcomes received from the GA. In enhancing the stability of the system, series and shunt compensators in the power system contribute a lot. But, traditional compensators were not able to dispense the purpose in vision of the intensifying complexity and dynamic level of change in the system. A significant impact was found in FACTS controllers on the stability of power system. Employing the swarm technique BSO algorithm, the FACTS devices get employed in the power system in concurrence with optimal power generation reallocation. At the start, the shunt controller is positioned in the IEEE

296

G. V. N. Kumar et al.

30 bus system, as the optimal power flow solution via BSO method is inspected. The availability of the FACTS controller SVC certainly has its impact on optimal power generation reallocation in an improved mode. BSO method along with SVC, TCSC and UPFC facilitates to acquire a better power system voltage within limitations.

References 1. Acha, E., Fuerte-Esquivel, C., Ambriz-Perez, H., Angeles, C.: FACTS: Modelling and Simulation in Power Networks, pp. 105–120. Wiley (2004). https://doi.org/10.1002/0470020164 2. Jirapong, P.: FACTS devices allocation for power transfer capability enhancement and power system losses reduction. Int. J. Energy Optim. Eng. 2(2), 1–14 (2013). https://doi.org/10.4018/ ijeoe.2013040101 3. Hingorani, N.G., Gyugyi, L.: Understanding FACTS: Concepts and Technology of Flexible AC Transmission System, pp. 36–55. IEEE Press (2000) 4. Edris, A.: FACTS technology development: an update. IEEE Power Eng. Rev. 25–46 (2000) 5. Hsmmns, T.J., Lim, S.K.: Flexible AC transmission system (FACTS). Electr. Mach. Power Syst. 25, 73–85 (1997) 6. Tinney, W.F., Hart, C.E.: Power flow-solution by Newton’s method. IEEE Trans. (PAS) 86, 1449 (1967) 7. Stagg, G.W., El-Abid, A.H.: Computer Methods in Power System Analysis, pp. 110–115. McGraw-Hill Book Co. (1968) 8. Carpentier: Optimal power flows. Electr. Power Energy Syst. 1, 959–972 (1979) 9. Sun, D.I., Ashley, B., Brewer, B., Hughes, A., Tinney, W.F.: Optimal power flow by Newton approach. IEEE Trans. Power Appar. Syst. 103(10), 2864–2880 (1984) 10. Kumar Roy, P.: Hybridization of biogeography based optimization with differential evolution for solving optimal power flow problems. Int. J. Energy Optim. Eng. 2(3), 86–101 (2013). https://doi.org/10.4018/ijeoe.2013070106 11. Kundur, P.: Power System Stability and Control. McGraw-Hill Inc., New York (1993) 12. Chung, T.S., Li, Y.Z.: A hybrid CA approach for OPF with consideration of FACTS devices. IEEE Power Eng. Rev. 21(2), 47–50 (2001) 13. Gyugi, L.: A unified power control concept for flexible AC transmission systems. IEE Proc. C 139(4), 323–332 (1992) 14. LashkarAra, A., Kazemi, A., NabaviNiaki, S.A.: Multi objective optimal location of FACTS shunt-series controllers for power system operation planning. IEEE Trans. Power Deliv. 27(2), 481–490 (2012) 15. Tiwari, P.K., Sood, Y.R.: Efficient and optimal approach for location and parameter setting of multiple unified power flow controllers for a deregulated power sector. IET Gener. Transm. Distrib. 6(10), 958–967 (2012) 16. Gyugyi, L., Schauder, C.D., Williams, S.I., Rictman, T.R., Torgerson, D.R., Edris, A.: The unified power flow controller; a new approach to power transmission control. IEEE Trans. Power Deliv. 10(2), 1085–1097 (1995) 17. Padiyar, K.R., Kulakarni, A.M.: Control design and simulation of unified power flow controller. IEEE Trans. Power Deliv. 13(4), 1348–1354 (1998) 18. Padiyar, K.R., Uma Rao, K.: Modeling and control of unified power flow controller for transient stability. Electr. Power Energy Syst. 21 (1999) 19. Zheng, S., Tan, Y.L.: Dynamic character study of UPFC based on detailed simulation model. In: IEEE Power Conference 2000 (2000) 20. Yang, X.-S.: Nature-Inspired Metaheuristic Algorithms. Luniver Press (2008) 21. Rao, B.V., Nagesh Kumar, G.V.: Optimal location of thyristor controlled series capacitor for reduction of transmission line losses using bat search algorithm. WSEAS Trans. Power Syst. 9, 459–470 (2014)

Enhancement of Voltage Stability Using FACTS Devices …

297

22. Rao, B.V., Nagesh Kumar, G.V.: A comparative study of BAT and firefly algorithms for optimal placement and sizing of static VAR compensator for enhancement of voltage stability. Int. J. Energy Optim. Eng. (IJEOE) 4(1), 68–84 (2015) 23. Singh, J.G., Singh, S.N., Srivastava, S.C.: Enhancement of power system security through optimal placement of TCSC and UPFC. In: Power Engineering Society General Meeting, Tampa, pp. 1–6 (2007) 24. Rao, B.V., Nagesh Kumar, G.V.: Optimal location of thyristor controlled series capacitor to enhance power transfer capability using firefly algorithm. Electr. Power Compon. Syst. 42(14), 1541–1552 (2014)

Author Index

C Cheng, Shi, 3, 35, 157

N Nagesh Kumar, G. V., 2′73

D Dash, Satyabrata, 221 Deepak Chowdary, D., 273 Dey, Sukanta, 221

S Sallam, Karam, 105 Shi, Dongyuan, 61 Shi, Yuhui, 3, 35, 157 Sravana Kumar, B., 273

E El-Abd, Mohammed, 105 Elsayed, Seham, 105

T Trivedi, Gaurav, 221

F Fu, Yulong, 79

V Venkateswara Rao, B., 273

G Gao, Shangce, 123 Guan, Chongchong, 157

W Wang, Xinrui, 79 Wang, Yirui, 123 Wu, Xiuli, 245 Wu, Yali, 79

H He, Yu, 61 J Janveja, Meenali, 221 Joshi, Deepak, 221 L Lei, Xiujuan, 3 Lu, Hui, 3, 35, 157 M Mafteiu, Emanuela, 189 Mafteiu-Scai, Liviu, 189 Mafteiu-Scai, Roxana, 189

X Xiong, Guojiang, 61 Xu, Yingruo, 79 Y Yang, Lin, 123 Yu, Yang, 123 Z Zhang, Jing, 61 Zhou, Rongrong, 35

© Springer Nature Switzerland AG 2019 S. Cheng and Y. Shi (eds.), Brain Storm Optimization Algorithms, Adaptation, Learning, and Optimization 23, https://doi.org/10.1007/978-3-030-15070-9

299