An introduction to formal languages and machine computation 9789810221676, 9810221673, 9810234228

560 120 64MB

English Pages 400 [412] Year 1998

Report DMCA / Copyright

DOWNLOAD FILE

Polecaj historie

An introduction to formal languages and machine computation
 9789810221676, 9810221673, 9810234228

Citation preview

An Introduction to

pORMAI

LANGUAGES AND MACHINE COMPUTATION

This page is intentionally left blank

An Introduction to

FORMAL

LANGUAGES AND MACHINE COMPUTATION

Song Y Yan University of York, UK

World Scientific

Singapore • New Jersey • London • Hong Kong

Published by World Scientific Publishing Co. Pte. Ltd. P O Box 128, Farrer Road, Singapore 912805 USA office: Suite IB, 1060 Main Street, River Edge, NJ 07661 UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE

Library of Congress Cataloging-in-Publication Data Yan, Song Y. An introduction to formal languages and machine computation / S. Y. Yan. p. cm. Includes bibliographical references and index. ISBN 9810221673 -- ISBN 9810234228 (pbk) 1. Formal languages. 2. Machine theory. I. Title. QA267.3.Y36 1998 004'.0r51--dc21 97-44021 CIP British Library Cataloguing-in-Pu blication Data A catalogue record for this book is available from the British Library.

Copyright © 1998 by World Scientific Publishing Co. Pte. Ltd. All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the Publisher.

For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher.

This book is printed on acid-free paper.

Printed in Singapore by UtoPrint

Preface Computer Science: The study of the use and sometimes construction of digital com­ puters. It is a fashionable, interesting, difficult, and perhaps useful activity. -

C H R I S T O P H E R STRACHEY

(1916-1975)

This book provides a concise and a modern introduction to Formal Languages and Machine Computation, a group of disparate topics in the Theory of Com­ putation, including formal languages, automata theory, Turing machines, computability, complexity, number-theoretic computation and cryptography. The theory of computation is essentially a mathematical subject. Indeed, many theoretical computer scientists are mathematicians, and some of them are even pure mathematicians. On reviewing the history of computer science, one can easily see that the key figures in the development of modern computer science, such as John von Neumann, Alan Turing, Alonzo Church, Stephen Kleene, Donald Knuth, Stephen Cook, to name a few, are mathematicians. However, as pointed out by Prof. Richard Hamlet at the University of Mary­ land in his book Introduction to Computation Theory, 1974: "The disappoint­ ing fact is that while computer science welcomes some mathematical experts, and recommends that students of the subject have a dose of theory as 'preventative mathematics', not very many practical computer scientists attempt to understand and apply the deep results of theory". The reason for this, we think, is partly because mathematics, particularly certain branches of mathe­ matics such as number theory and topology, are very hard to understand for most practical computer scientists. But the problem is that in many areas of computer science, mathematics is a very useful and sometimes even indispens­ able tool. In fact, although computer science is a very practical subject, it has its root in mathematics; for example, logic programming has its root in first-order logic, compilers in automata theory, programming languages in for­ mal languages, computer architecture in Boolean algebra, algorithm analysis in combinatorial mathematics, and so on. In this book we shall try to bridge v

vi

Preface

the gap between mathematics and computer science by presenting some basic mathematical concepts and results which are useful in computer science, but are not too mathematically involved. In fact, the book was written essentially for students in "practical" computer science, rather than "theoretical" computer science, or mathematics. To achieve such a goal, we do not follow the tradi­ tional mathematics textbook style: "Definition-Theorem-Proof", rather, we use the more practical style: "Definition-Theorem/Algorithm-Example/Exercise", because we believe that most practical computer scientists, systems designers or computer programmers are generally more interested in the applicability and usefulness of the theorems in computer science/mathematics, rather than how to prove the theorems. However, for those who really want to know the nature of the theorems, it is a good exercise for them to try to give all the missing proofs. The book is organized as follows. Chapter 1 provides an introduction to topics in computation-related mathematics, including mathematical logic, set theory, functions and relations, graph theory, number theory, and abstract alge­ bra, which provide a necessary mathematical foundation for the rest of the book. Although this chapter is rather lengthy in contrast to other similar books on the market, it proves to be an invaluable source of information because practical computer scientists usually lack sufficient mathematical background to under­ stand the results in the theory of computation. Chapter 2 is about automata and formal languages. Finite-state automata (FA) are the simplest possible model for computation. In fact, the human brain is a type of finite-state automaton with some 1025 states. We shall first study the basic concepts and results of finite automata and their correspond­ ing regular languages LREG- There are actually two different types of finite automata: deterministic finite automata (DFA) and non-deterministic finite automata (NFA). We shall show that these two types of automata are equiva­ lent, because they can simulate each other, and more importantly, they accept exactly the same languages. Then we shall study push-down automata (PDA) and their corresponding languages, context-free languages LCFL- Unlike DFA and NFA, non-deterministic push-down automata (NPDA) can accept a larger class of languages than deterministic push-down automata (DPDA), i.e., DPDA accept only deterministic context-free languages LDCFL, whereas NPDA accept non-deterministic context-free languages LNCFL- Since LDCFL C -^NCFL, NPDA have more power than DPDA. Finally, we shall study Turing machines and their corresponding languages, called recursively enumerable languages L R E . We shall show that LRE is acceptable but not decidable by Turing machines.

Preface

vii

We shall also show that there are languages, called recursive languages Z/REC, that are recursive, but not enumerable, i.e., LREC C LRE- This class of recursive languages LREC is decidable by Turing machines. The machines we study in Chapter 2 act as language accepters. This does not mean that automata, or more generally Turing machines, can only be used to accept languages. Automata, particularly Turing machines, the highest-level automata, are general-purpose computing machines for general computation. So in Chapter 3, we shall study the computability/decidability and computa­ tional complexity within the theoretical framework of Turing machines. More specifically, we shall study what Turing machines (or general computers) can do and what Turing machines cannot do, what problems are feasible by a Turing machine (or a general computer) and what problems are infeasible by a Turing machine. We shall also provide a brief account of the asymptotic behaviours of some common complexity functions and the analysis of algorithms. Number theory has played an important role in the development of modern computer science and engineering, noting the titles of the seminal papers by Alan Turing "On Computable Numbers, with an Application to the Entscheidungsproblem" (1936) and by Alonzo Church "An Unsolved Problem of Elemen­ tary Number Theory" (1936). So in Chapter 4, we shall study some modern mathematical methods for large number-theoretic computations, particularly those methods for primality testing and integer factorization of large numbers. We shall also discuss some applications of number-theoretic computations in computer science, including public-key cryptography, database security, ran­ dom number generation, and high-speed computation. Turing machines were proposed by Alan Turing in 1936. Since then, many new computation models, particularly parallel computation models (e.g., neu­ ral networks, cellular automata, quantum computation and molecular biolog­ ical computation) have been proposed. In the last chapter of this book, we shall present a brief introduction to the two recently developed new models of computation, namely quantum computation and molecular biological compu­ tation. We shall show that from a computability point of view, both quantum and molecular biological models will have no more computational power than Turing machines. However, from a complexity point of view, both quantum and molecular biological computers do offer some more computational power. For example, the traveling salesman problem (TSP) and the directed Hamiltonian path problem (HPP) have been proven to be Af'P-complete on Turing machines, but they can be solved in polynomial biological steps by biological computers; whereas integer factorization and discrete logarithms are believed

Preface

viii

to be intractable by Turing machines, but they can be solved in polynomial time on a quantum computer. Of course, the key issue concerning quantum and molecular biological models is how to actually build a truly quantum or biological computer! At present, no quantum and molecular biological comput­ ers are available on the market, but there is no reason why we cannot build such new generations of computers. Compared with many other books in the theory of computation on the market, this book contains substantially more materials in number-theoretic computations and their applications in computer science and secure communications. As pointed out by Prof. Ian Stewart (Na­ ture, Vol. 325, 15 January 1987, page 199): "computer scientists working on algorithms for factorization would be well advised to brush up on their num­ ber theory", we hope these materials will not only help computer scientists to brush up on their number theory, but also encourage them to apply results in number theory to computer science in such diverse areas as computational com­ plexity, public-key cryptography, network and information security, computer arithmetic and architectures, graphic design, image and signal processing, and digital communications. A c k n o w l e d g e m e n t s : This book grew out from my lecture notes for the fol­ lowing three courses: Mathematics for Computing, Formal Languages and Ma­ chines, and Algorithms and Complexity, taught at La Trobe University, Aus­ tralia (1990-1992), and The Buckinghamshire College of Brunei University, England (1993-1995). It was completely rewritten while I was in the Depart­ ments of Mathematics and Computer Science at the University of York, Eng­ land, over the period of 1993 and 1997. No book is created in a vacuum, and this one is no exception; I could not have written this book without the help of many friends, colleagues, students and my family. I would like to take this opportunity to thank James Austin, Thram Dillon, William Freeman, Terence Jackson, Milton Munroe, Ian Robinson and John Zeleznikow for their encour­ agement and help during the writing of this book. I would also like to thank the editors at World Scientific; Tony Moore at the London Office and partic­ ularly Stephen Patt and Joy Marie Tan at the Singapore Office for their help and encouragement. Special thanks must be given to William Bloodworth, Gavin Doherty, William Freeman, Milton Munroe, and Robert Pargeter, Peter Pleasants who read the various earlier versions of the book and suggested many improvements. Finally, I would like to thank all the authors cited in this book; they are the giants on whose shoulders I can stand on. YORK,

ENGLAND

DECEMBER

1997

Contents Preface

v

Notations

xiii

1 Computation-Related Mathematics 1.1 Logics and Proofs 1.1.1 Propositional Logic 1.1.2 Predicate Logic 1.1.3 Programming in Logic 1.1.4 Proof Techniques 1.2 Sets, Functions and Graphs 1.2.1 Sets and Operations on Sets 1.2.2 Functions and Relations 1.2.3 Directed and Undirected Graphs 1.3 Divisibility, Continued Fractions and Congruences 1.3.1 Divisibility 1.3.2 Continued Fractions 1.3.3 Diophantine Equations 1.3.4 Congruences 1.4 Groups, Rings and Fields 1.4.1 Binary Operations 1.4.2 Groups 1.4.3 Rings and Fields 1.4.4 Elliptic Curves over a Ring/Field 2 Formal Languages and Automata 2.1 Languages, Grammars and Automata 2.2 Finite Automata and Regular Languages ix

1 1 2 7 11 20 27 27 33 42 45 46 54 59 64 79 79 81 85 89 95 95 100

Contents

xX

2.3

2.4

3

2.2.1 Deterministic Finite Automata (DFA) 2.2.2 Nondeterministic Finite Automata (NFA) 2.2.3 Regular Languages (Z/REG) 2.2.4 Applications of Finite Automata 2.2.5 Variants of Finite Automata Push-Down Automata and Context-Free Languages 2.3.1 Push-Down Automata (PDA) 2.3.2 Context-Free Languages (LCF) 2.3.3 Applications of Context-Free Languages Turing Machines and Recursively Enumerable Languages . . . . 2.4.1 Standard Turing Machines 2.4.2 Turing Machines as Language Accepters 2.4.3 Variants of Turing Machines 2.4.4 Recursively Enumerable Languages LRE 2.4.5 Context-Sensitive Languages Lcs 2.4.6 Hierarchy of Machines and Grammars/Languages . . . .

Turing Computability and Complexity 3.1 Computability and Noncomputability 3.1.1 Computation with Turing Machines 3.1.2 Computable Functions 3.1.3 Noncomputable Functions 3.1.4 Horn Clause Computability 3.2 Decidability and Undecidability 3.2.1 Decision Problems 3.2.2 Decidable Problems 3.2.3 Decidable Problems in Formal Languages 3.2.4 Undecidable Problems 3.2.5 Semi-decidable and Hilbert's Tenth Problem 3.2.6 Undecidability in Formal Languages 3.3 Computational Complexity 3.3.1 Time and Space Complexity Classes 3.3.2 Complexity Classification of Formal Languages 3.3.3 Polynomial-Bounded Complexity Classes 3.3.4 Parallel and Probabilistic Complexity Classes 3.3.5 JvT-Complete Problems 3.4 Design and Analysis of Algorithms 3.4.1 Asymptotic Complexity

101 106 108 115 118 128 129 130 134 135 136 137 146 152 153 155 163 163 164 169 175 179 182 182 185 187 188 193 197 198 199 201 203 208 215 218 219

Contents

xi

3.4.2 3.4.3 3.4.4 4

Analysis of Algorithms Design of Efficient Algorithms Relations between Arithmetic and Bit Operations . . . .

Number-Theoretic Computations and Applications 4.1 Primality Testing 4.1.1 Basic Algorithm 4.1.2 Strong Pseudoprimality Test 4.1.3 Lucas Pseudoprimality Test 4.1.4 Elliptic Curve Test 4.1.5 Complexity of Primality Testing 4.2 Integer Factorization 4.2.1 Complexity of Integer Factorization 4.2.2 Trial Division and Fermat Method 4.2.3 Legendre's Congruence 4.2.4 Continued FRACtion Method (CFRAC) 4.2.5 Quadratic and Number Field Sieves (QS/NFS) 4.2.6 urho" and "p - 1" Methods 4.2.7 Elliptic Curve Method (ECM) 4.3 Discrete Logarithms 4.3.1 Shanks'Baby-Step Giant-Step Algorithm 4.3.2 Silver-Pohlig-Hellman Algorithm 4.3.3 Subexponential Algorithms 4.3.4 Factoring versus Discrete Logarithms 4.4 Cryptology and Systems Security 4.4.1 Private/Public-Key Cryptosystems 4.4.2 Discrete Logarithm Based Cryptosystems 4.4.3 RSA Public-Key Cryptosystems 4.4.4 Elliptic Curve Public-Key Cryptosystems 4.4.5 Digital Signatures 4.4.6 Database Security 4.4.7 Random Number Generation 4.5 High-Speed Computation 4.5.1 Residue Number Systems (RNS) 4.5.2 Fast Computation in RNS 4.5.3 Residue Computers 4.6 Three More Applications in Computing

224 230 238 245 246 246 248 254 261 265 268 268 272 275 278 281 285 290 293 294 298 303 304 305 305 309 313 324 328 334 339 345 345 349 353 356

xii

Contents

5 New Models of Computation 5.1 Quantum Computation 5.1.1 Physics and the New Computation 5.1.2 Algorithms for Quantum Computation 5.2 Biological Computation 5.2.1 Biological Algorithms for HPP/SAT 5.2.2 Formal Models of DNA Biological Computation 5.3 Comparison of Quantum and DNA Biological Models 5.4 Comparison of Connectionist and DNA Biological Models . . . .

361 362 362 364 368 369 372 375 377

Bibliography

381

Index

393

Notations Notation

Explanation

N

set of natural numbers: N = {0,1, 2,3, • • •}

Z Z

set of integers: Z = {0, ± 1 , ±2, ± 3 , • • • } +

set of positive integers: Z + = {1,2,3, • • •}

Q

set of rational numbers: Q = I T- : a, b € Z, and b ^ 0 \

R

set of real numbers: R = {n + 0.cfid2^3 ■ ■ ■ : n £ Z, Vdi 6 [0,1, ■ • ■ ,9] and no infinite sequence of 9' appears}

C

set of complex numbers: C = {a + bi : a, 6 G R and i — \f—\)

Z/nZ Z/nZ (Z/nZ)' (Z/nZ)* #(Z/n2 #(Z/nZ)*

residue classes (ring of integers mod n; it's a field if n is prime) order of the multiplicative group (Z/nZ)*

Fp

finite

ite field vwith p elements, where p is a prime number field

¥q

finite

ite field vwith q = pk a prime power field

multiplicative group of Z / n Z

K

field

II

end of proof

—* -•

negation

A

conjunction

V

disjunction

=$>

implication