Autonomous Agents and Multi-Agent Systems: Explorations in Learning, Self-Organization and Adaptive Computation [Hardcover ed.] 9810242824, 9789810242824

Table of contents :
Behavioural modelling, planning, and learning
synthetic autonomy
dynamics of distributed computation
self-organized autonomy in multi-agent systems
autonomy-oriented computation
dynamics and complexity of autonomy-oriented computation.

Citation preview

AUTONOMOUS AGENTS AND MULTI-AGENT CVCTEMC

v£ Explorations in Learning, Self-Organization and Adaptive Computation

AUTONOMOUS AGENTS AND MULTI-AGENT SYSTEMS

AUTONOMOUS AGENTS AND MULTI-AGENT SYSTEMS Explorations in Learning, Self-Organization and Adaptive Computation

Jiming Liu Hong Kong Baptist University

By request of the author, the royalty for this book goes to United Nations Children's Fund ( JNICEF) and a youth public welfare fund.

10 World Scientific ll

Singapore • New Jersey • London • Hong Kong

Published by World Scientific Publishing Co. Pte. Ltd. P O Box 128, Farrer Road, Singapore 912805 USA office: Suite IB, 1060 Main Street, River Edge, NJ 07661 UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE

British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library.

AUTONOMOUS AGENTS AND MULTI-AGENT SYSTEMS Explorations in Learning, Self-Organization and Adaptive Computation Copyright © 2001 by World Scientific Publishing Co. Pte. Ltd. All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the Publisher.

For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher.

ISBN 981-02-4282-4

Printed in Singapore by World Scientific Printers

This book is dedicated to Meilee, Isabella, Bernice, and my parents. Jiming Liu

Preface

Discovery consists in seeing what everyone else has seen and thinking what no one else has thought. Albert Szent-Gyorgi, 1937 Nobel Prize in Physiology and Medicine.

Autonomous agents are computational entities that have their own autonomy. Autonomy entails the unique capability of surviving (they will not die) and living (they will furthermore enjoy their existence). It is something that human beings, human societies, healthy economies, successful organizations, and biological organisms obviously have and may take for granted. As a companion to Multi-Agent Robotic Systems (by Jiming Liu and Jianbing Wu, CRC Press, 2001), this book is aimed at (1) providing a comprehensive overview of related research work in the field of autonomous agents and multi-agent systems with an emphasis on its theoretical and computational foundations, and (2) providing in-depth discussions on the useful techniques for developing various embodiments of agent-based systems, such as autonomous robots, collective vision and motion, autonomous

Vlll

Preface

animation, and search and segmentation agents. The core of those techniques is the notion of synthetic or emergent autonomy based on behavioral self-organization. Our discussions will address a number of important issues involved in autonomy-oriented computation, which include architecture, learning, adaptation, dynamics, and complexity. This book is organized into seven chapters: Chapter 1 examines the technical issues as well as computational models central to the development of autonomous agents, including agent action selection, adaptation, evolution, self-organization, learning, and architectures. While highlighting a number of significant results from recent studies, it also indicates the areas for further continued research efforts. Chapter 2 presents an example of agent behavioral modeling, planning, and learning in the applications of automatically programming robot manipulation strategies. It focuses on the underlying issues such as how behavioral plans can be computationally modeled and synthesized. Chapter 3 examines the role of behavioral self-organization in creating synthetic autonomy. It focuses on the underlying algorithms as well as the specific constructs for behavioral acquisition, and describes an implemented agent that interacts with its virtual environment and at the same time learns a sequence of coherent parameterized motion behaviors. Chapter 4 is concerned with the dynamics of distributed computation. It discusses (1) how certain tasks can be handled by breeds of distributed agents in response to their local environment and (2) how the behavioral repository of the agents can be constructed based on dynamic systems models. Chapter 5 introduces the notion of self-organized autonomy in a system of multiple agents, i.e., a multi-agent system. Specifically, it examines the cases of accomplishing collective behavior for finding some predefined target locations in a two-dimensional search space. Chapter 6 focuses on the topics of adaptation and self-organized autonomy-oriented computation. While presenting the formulation and algorithmic details, it also demonstrates the new computational paradigm with several case studies in image segmentation. In the case studies, the autonomous agents, being distributed computational entities, operate on the individual pixels of the given image, and execute a number of reactive behavioral responses. As part of the behavioral adaptation, the directions in which the agents self-reproduce and diffuse are inherited from the directions of their high-fitness parents and siblings.

Preface

IX

Chapter 7 describes the dynamics and complexity of autonomy-oriented computation. Examples of dynamics models are given for the cases of autonomous agents that attempt to extract certain image features, such as edges, borders, and complex curves. This self-contained book is intended for several categories of audience. It can be used by students of computer science and engineering disciplines in learning how to apply agent models and autonomy-oriented computation techniques to solve real-world problems. The book is also suited to computer scientists, engineers, researchers, and practitioners who are interested in finding solutions to their problems that arise in developing intelligent and autonomous systems. The methodologies, algorithmic details, and case studies presented in this book can serve as a convenient reference. Jiming Liu Kowloon Tong, Hong Kong Summer 2001

Acknowledgements

I would like to thank pioneers as well as fellow researchers in the fields of biology, ecology, psychology, artificial intelligence, autonomous agents and multi-agent systems, robotics, and complex dynamic systems for providing me with inspirations. In particular, I wish to acknowledge two enlightening experiences that have had a profound impact on my studies and research. The first was a visit to the Department of Psychology at Carnegie Mellon University in the winter of 1987, during which I was honored to have a meeting with Nobel Laureate Professor Herbert A. Simon (1916-2001) in his office to discuss the topics related to human cognition and models of learners. That discussion made me start to appreciate the roles of computer modeling and simulation, such as information-processing models and stochastic models, in unveiling and understanding human problem-solving activities. The second experience was a visit to the MIT Media Lab to meet with Professor Seymour Papert in the spring of 1987. That visit reinforced my belief that the goal of machine intelligence is to better discover and develop human intelligence. I wish to express my gratitude to all the people who have, over the years, participated and made important contributions in my previous and present research projects. In particular, I would like to thank the following individuals for their enthusiasm, diligence, and willingness to explore new areas with me: • Hong Qin who programmed a software system for conducting the experiments reported in Chapter 3; • Yi Zhao who made a contribution to the work described in Chapter 4; xi

Acknowledgements

XII

• Yunqi Lei who implemented part of the modeling and simulation work reported in Chapter 5; • Yichuan Cao and Huijian Zhou who programmed a software environment for conducting feature searching related experiments; • J u n Le who performed some of the experiments and analysis reported in Chapter 7; and • Jianbing Wu who participated in many helpful discussions on related projects and commented on the earlier versions of this book. I wish to offer my thanks to Hong Kong Baptist University (HKBU) and the Faculty of Science for providing me with a pleasant working environment t h a t enables me to pursue my research. W i t h o u t this environment, it might take longer for me to come u p with this new synthesis. I a m also grateful to the colleagues and students of C o m p u t e r Science D e p a r t m e n t at H K B U whom I have worked with over the years. T h e y have m a d e my tenure at H K B U a rewarding experience. P a r t of this book was written during my sabbatical leave in the Computer Science D e p a r t m e n t of Stanford University. Here I would like to thank Professor O u s s a m a K h a t i b for his kind invitation and also thank H K B U for granting m e this opportunity. I wish to acknowledge the research grants support provided by Hong Kong Baptist University under the scheme of Faculty Research G r a n t s and by Hong Kong Research Grants Council under the scheme of E a r m a r k e d Research G r a n t s . W i t h o u t this support, the results reported in this book would not be possible. T h a n k s also go to Ms. Lakshmi Narayanan and other staff of World Scientific Publishing for managing and handling this book project. Finally, to my wife Meilee, my daughters Isabella and Bernice, and my parents, my heartfelt t h a n k s for their love, inspiration, and encouragement. Other

Credits

I would like to acknowledge Kluwer Academic Publishers, Springer-Verlag, IEEE Press, and ACM Press for permission to reuse some portions of copyrighted materials from my other publications in this book, in particular the materials from the following sources. Figures 1.3-1.6, 3.3, 3.4, and 3.6-3.17 are reproduced from Liu, J. and Qin, H., Behavioral self-organization in lifelike synthetic agents, in Autonomous Agents and Multi-Agent Systems, Kluwer Academic Publishers, 2002. Figures 2.7, 2.8, and 2.10 are reproduced from Liu, J., Tang, Y. Y., and Khatib, O., Modeling and learning robot manipulation strategies, in Alicia Casals and Anibal de Almeida (Eds.), Experimental Robotics V (Lecture Notes in Control

Acknowledgements

xiii

and Information Sciences), Springer-Verlag, 1998. Figures 5.2, 5.4-5.6, 6.4, 6.21, 6.22, 6.27, and 7.12 are reproduced from Liu, J., Tang, Y. Y., and Cao, Y. C , An evolutionary autonomous agents approach to image feature extraction, in IEEE Transactions on Evolutionary Computation, Vol. 1, No. 2, 1997, pp. 141158. Figures 6.1, 6.2, 6.13, and 6.16-6.20 are reproduced from Liu, J. and Tang, Y. Y., Adaptive segmentation with distributed behavior-based agents, in IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 21, No. 6, 1999, pp. 544-551. Omissions of credit acknowledgement in this book, if any, will be corrected in future editions.

Jiming Liu

Contents

Preface

vii

Acknowledgements

xi

Chapter 1 Introduction 1.1 What is an Agent? 1.2 Basic Questions and Fundamental Issues 1.3 Learning 1.3.1 Learning in Natural and Artificial Systems 1.3.1.1 Skinner's Pigeons in a Pelican 1.3.1.2 Biologically Inspired Legged Robots 1.3.2 Agent Learning Techniques 1.3.2.1 Agent Belief Development and Updating . . . 1.3.2.2 Performance-Based Learning 1.3.2.3 Reinforcement Learning 1.4 Neural Agents 1.4.1 Self-Organizing Maps (SOM) 1.4.2 SOM Applications 1.5 Evolutionary Agents 1.6 Learning in Cooperative Agents 1.7 Computational Architectures 1.7.1 Subsumption Architecture 1.7.2 Action Selection 1.7.3 Motif Architecture 1.7.3.1 Overview

1 2 3 4 4 4 5 6 6 9 11 12 12 13 15 16 17 18 18 19 19

XV

xvi

1.8

Contents

1.7.3.2 Dimensions of Motif Interaction 1.7.3.3 Self-Similar Structures 1.7.3.4 Motifs 1.7.3.5 Computation in Two Interacting Motifs . . . 1.7.3.6 An Agent Behavioral Organization Approach Agent Behavioral Learning 1.8.1 What is the Behavior of a Learning Agent? 1.8.2 What is Behavioral Learning?

Chapter 2 Behavioral Modeling, Planning, and Learning 2.1 Manipulation Behaviors 2.2 Modeling and Planning Manipulation Behaviors 2.2.1 State-Oriented Representation 2.2.2 State-Transition Function (