Systems and Computer Science: Proceedings of a Conference held at the University of Western Ontario September 10-11, 1965 9781487592769

Table of contents :
Preface
Acknowledgment
Contents
On the Structure of Finite Automata
Synthesis of Sequential Machines
Techniques for Manipulating Regular Expressions
Some Comments on Self-Reproducing Automata
Multiple Control Computer Models
Explicit Definitions and Linguistic Dominoes
Heuristic and Complete Processes in The Mechanization of Theorem Proving
An Approach to Heuristic Problem Solving and Theorem Proving in the Prepositional Calculus
New Directions in General Theory of Systems
Concerning an Algebraic Theory of Systems
REFERENCES

Citation preview

SYSTEMS AND COMPUTER SCIENCE This book presents the papers delivered at the Conference on Systems and Computer Science held at the University of Western Ontario in September of 1965. The primary purposes of the Conference were the promotion of research and the development of the teaching of computer science in Canadian universities. The papers focus attention on some of the concepts of Computer Science as a new field of study and at the same time provide a background for scientists looking at the subject for the first time. The chief developments in computer science have been concerned with the "applied" rather than the "pure" areas of the field: numerical analysis, applied statistics and operations research, and data processing. But there is something more to computers than the physical components and this book represents an attempt to correct the imbalance between "applied" and "pure" by drawing attention to certain theoretical aspects of computer and information science. Among the topics discussed are the theory of finite and infinite automata, aspects of formal language theory, heuristic and non-heuristic approaches to theorem proving and the mathematical formulation of the theory of general systems. There are also references to the problems of machine design, to software systems including higher-level languages, to multiple control computer models and to applied systems. This collection of papers will appeal first to graduate students and professors in Computer Science. It will also be of interest to computer scientists in industry and in government and university research groups and to the scientific public interested in discovering some of the principal ingredients and directions of the computer and information sciences. JOHN F. HART received his Ph.D. in Applied Mathematics at the University of Toronto. From 1953 to 1959 he was Assistant Research Officer, National Research Council, stationed at the University of Toronto as N.R.C. Representative where he worked on FERUT, Canada's first computer. In 1959 he became Director of the Computing Centre at the University of Western Ontario, and is now Head of its Department of Computer Science. SATORU TAKASU received his D.S. in Mathematics in Tokyo and after teaching for a year at Musashi College of Technology he became Senior Engineer at the Nippon Telegraph and Telephone Public Corporation, Tokyo. From 1963 to 1966 he was Assistant Professor at the University of Western Ontario and is at present with the Research Institute of Mathematical Sciences at the University of Kyoto, Japan.

This page intentionally left blank

PROCEEDINGS OF A CONFERENCE HELD AT THE UNIVERSITY OF WESTERN ONTARIO SEPTEMBER 10-11, 1965

Systems and Computer Science EDITORS

John F. Hart Computer Science Department University of Western Ontario

Satoru Takasu Research Institute for Mathematical Sciences University of Kyoto, Kyoto, Japan

Published in Association with The University of Western Ontario by University of Toronto Press

Copyright Canada 1967 by University of Toronto Press

Preface John F. Hart and Satoru Takasu

The ten chapters of this book are the revised and somewhat extended versions of papers given at the Conference on Systems and Computer Science, which was held at the University of Western Ontario, London, Ontario, Canada, September 10 and 11, 1965. The primary purposes of the Conference were the promotion of research and the advancement of the teaching of computer science in Canadian universities. In more specific terms, the objective was to draw attention to certain theoretical aspects of computer and information science, such as automata, linguistics, theorem proving and general systems. In line with these objectives, the invited lecturers were asked to bring out the nature of the different fields, the current problems, and an opinion concerning the hope for the future. In the interests of research, the speakers and topics were chosen in the expectation that there might be, within the fields themselves and globally in their interaction, a point of departure for new developments. The situation of computer science in Canada at the time of the Conference was one in which the emphasis of scientific computing was in numerical analysis applied statistics, and operations research, while in data processing, there was extensive experience in industrial applications. It might perhaps be possible to summarize these developments by noting that Canadian development was concerned with the "applied" rather than the "pure" side of computer science. There were, however, extensions into computer science per se. Thus, for example, a number of universities had intensive efforts in compiler design, particularly of ALGOL, or ALGOL-like compilers. These projects had the effect of inviting researchers to look into the more abstract aspects of formal languages and liguistics, with the consequence that there was beginning to be a demand for more abstract treatment. Another element in the Canadian scene, as elsewhere, had to do with the question of what subjects were to be included in a computer science curriculum. Many people were asking what were the subjects, after programming and

vi

JOHN F. HART

numerical analysis, which made up the offering of the computer science unit. Here, also, there was a question of determining to what extent the teaching of computer science should be vocational in nature and to what extent it should be purely educational. In considering this problem of the pure and applied in computer science the editors do not wish to come down in favour of one side or the other but to achieve a kind of balance. We wish to be able to advance the one without disparaging the other or, in other words, to train systems analysts as well as information scientists. This is perhaps an instance of the kind of parallelism with which the computer scientist is very familiar, since he must often arrange things so as to have certain processes occurring simultaneously rather than consecutively. Nevertheless, we were concerned in this Conference with the pure side rather than the applied and the subjects in that category which we thought should be included in the computer science curriculum. We wish to emphasize that our choice of topics was neither exhaustive nor restrictive, there being certain fields which were omitted for which a real case for inclusion can be made. That our choice was a reasonably good one was evidenced by the many favourable comments we received. In preparing this work, the editors have kept uppermost in mind that its usefulness would consist principally in pointing the way to advanced study and research. We have tried to promote the needs of the graduate student and the researcher by emphasizing general references as well as research papers. We hope that the book will serve as a stimulus to look further into the field and that it will act, to some extent, as an overall study guide. The Theory of Finite Automata, which first appeared in 1954, is a rather new branch of science. It has been extensively studied by electrical engineers, logicians, mathematicians, and linguists and has been taught in various university departments. However, it seems to us that this theory is most appropriately considered as a branch of computer science because it affords a theoretical description of a computer, and, in more specific terms, because it allows us to define information processing very elegantly in terms of "internal states." The concept of a finite automaton (sequential machine) was introduced in the early stages by Moore [11], and Mealy and Huffman [12] as a model of relay networks or simple digital computers. At about the same time, Kleene [10] invented the concept of regular expression to study the behaviour of nerve nets. He established the fact that a finite automaton can be considered as a mathematical model of a nerve net and he gave an explicit description in terms of regular expression of the sets of sequences which can be accepted by finite automata. However, the algorithms developed for this purpose did not provide a method of overall optimization for the design of a computer, since an "internal state" means the content of all memory elements at certain moments of time; consequently, the number of states will be enormous even if the circuit is fairly small. The main reason for theoretical difficulties of this type was that the theory was not equipped with concepts corresponding to subcircuits of a given digital circuit.

PREFACE

vii

Considerations relative to optimization have taken on a different emphasis due to the fact that the recent developments of electronics engineering have diminished the difference of prices of gate elements and memory elements, and the cost of wiring is becoming an important factor. The lack of concepts corresponding to subcircuits and the change of objective optimization functions give us the following problems. A. Is it possible to introduce concepts corresponding to the decomposition of a circuit into its subcircuit? B. Is there a method of synthesis for design of a circuit (finite automaton) starting from a given set of building blocks (not as small as flip-flops) whose functions and wiring, i.e., their structure, are specified? The answer to those questions has been studied by Hartmanis, Krohn-Rhodes, Brzozowski, Zeiger, and others. This type of study is called the structural theory of finite automata. In this Conference, Hartmanis talked about his results and about the recent study of Zeiger concerning question A, and Brzozowski reported on the recent development of the study corresponding to question B. On the other hand, the relationship between regular expressions and finite automata has been studied by Kleene, Myhill [5], Nerode [6], Brzozowski [6] and others, and also, from the viewpoint of formal language theory, by Chomsky [3], Bar-Hillel et al. [2]. As a result, the regular expression can be considered to be a nice specification language for digital circuits, and there is a method of deriving the minimal state automaton from a given regular expression. However, the main problem of this study lies in the difficulty of manipulating regular expressions. During the Conference, McNaughton reported on the recent development of the study of manipulation and structure of regular expressions. The study of infinite automata was begun by Turing in 1936, earlier than the study of finite automata [4, 16, 17]. Turing introduced an automaton (Turing machine), which has a finite automaton portion with a read-write head and a tape of infinite length, for the purpose of formalizing the concepts of algorithm and computability. A Turing machine utilizes a finite portion of the infinite tape during its computation and there is no bound to the length of this finite portion. This fact is sometimes called "potentially infinite memory." Therefore, an infinite automaton means a finite automaton with some kind of potentially infinite memory, or an infinite aggregate of finite automata. Because of this unlimited availability of work space, Turing machines are considered to be most powerful in computational capability among automata; any computation which can be done by a known (formalized) infinite automaton can be done (can be simulated) by a Turing machine. Furthermore, Turing demonstrated that a universal Turing machine exists which can simulate any computation of any Turing machine (and hence any infinite automaton). Besides the intensive study of algorithm and Turing machines in mathematical logic, there are three main streams of the recent study of infinite automata.

viii

JOHN F. HART

A. The study of complexity of computation. B. The study of infinite automata with restricted capability, such as linear bounded automata and push down storage automata. C. The study of some specific types of infinite automata. We shall discuss B later together with formal language theory. The topic A is one of the most interesting fields of study, but unfortunately it was not covered by our Conference; we only mention that it was begun by Yamada [18] and generalized by Hartmanis and Stearns [7, 8] and also by Ritchie [14]. At the Conference, two topics related to C were presented, one by Arbib and the other by Elgot et al. In his article, Arbib studied the problem of self-reproduction of automata, which was begun by Von Neumann. Von Neumann proposed two models to describe the process of self-reproduction: one was a 29 state cellular machine and the other was the incomplete thermodynamical model. Arbib discussed mainly the self-reproduction in cellular machines from the abstract point of view. It is quite interesting to observe that the study of self-reproduction in cellular machines, together with the indications of Von Neumann's unfinished thermodynamical model, give rise to the expectation that these theories may provide a logical framework for understanding embryological processes. Perhaps the development of the study of cellular machines will give us a basis for the logical analysis of the cancer explosion in a living organ. The other paper related to division C is by Elgot, Robinson, and Rutledge, and it discusses multiple control computer models. There has long been a need for an appropriate model of a computer, which could explain the relation between "programme" and "computer." Because of the arbitrariness of the machine and the tape-content structure, Turing machine theory does not give us a satisfactory explanation of "programme and computer." The above authors intend to specify on a trial basis the machine structure within a model more explicitly so that these trials will ultimately bridge the problem of programming and the problem of machine design—both very challenging problems. In mathematics, the interest in formal languages is associated with logic and linguistics. However, the interest of computer scientists in this field arises out of the desirability of having a method of describing higher level languages such as ALGOL. This interest is also a natural follow-up to the study and design of compilers which has occupied many computer scientists. It is hoped that the study of formal languages will lead eventually to new theories of programming. A good idea of the scope of this field may be obtained from the monograph of Bar-Hillel [2], from Chomsky's article in the Handbook of Mathematical Psychology [3], and from chapter xv of Harrison's [6] book. Professor Corn's paper is devoted to a study of the effect of extending languages by the introduction of explicitly defined new characters into their alphabets. He relates his work to problems in natural linguistics, logic, programming, and automata theory. Over the years, the algorithmic non-heuristic study of theorem proving has

PREFACE

ix

interested many researchers in mathematical logic. However, as a result of the impact of the development of computers, new problems of finding computationally efficient algorithms for proof procedure have arisen. During the Conference, Robinson gave us the history of theorem proving and an exposition of the essential part of his new efficient algorithm as a combinatorial problem. By contrast, Amarel's paper is principally concerned with the heuristic approach to theorem proving. The initiation of this work was no doubt due to Samuel with his contribution to checker playing. Important milestones in this development are the invention of the languages IPL V and LISP. In presenting his review of this subject, Amarel has paid particular attention to problem specification and to considering simplest proofs to theorems of the prepositional calculus by the method of natural deduction. The reader is advised to consult the work of Newell, Shaw, and Simon [13]. The subject of systems theory which is the concern of the last two chapters is very difficult to circumscribe because of its wide range of application. This wide extension is due in part to the generality expressed by the word "system." The word may be used in the context of real structures ranging through great differences in degrees of complexity and differences in the idealized form of abstract models. In addition, general systems may refer to an insight proper to the philosophy of science or to mathematical theories which are strongly dependent on the theory of relations. In some countries (Russia, for example), cybernetics refers to both the philosophical and the scientific aspects of general systems theory. Elsewhere cybernetics is reserved for those aspects of systems in which the emphasis is upon control. The editors, without considering themselves to be adept in these matters, are aware of a need for the development of various aspects of systems theory. We realize that the computer scientist is inevitably drawn to the deeper systems concepts. Without an appreciation of systems theory, he cannot appreciate the effect of introducing into the environment a machine such as a computer, which may become a "system breaking device." Indeed, it is obvious now that many failures in the early application of computers to accounting were due to the lack of adequate systems concepts. In their papers, Mesarovic and Windeknecht introduced some new directions in the study of abstract systems. Two aspects of this theory which were not dealt with explicitly are those having to do with control and internal states. These topics are treated to some extent by Zadeh and Desoer [19] in connection with engineering systems. We have found the Yearbook of the Society for General Systems Research to be a good source for general background and valuable insights. As a basis for introductory courses in general systems suitable for students in computer science, we recommend Ashby's book, An Introduction to Cybernetics [I]. Looking to the future, we anticipate with interest the advancement of the theory in the articles to be published in the recently announced Mathematical Systems Theory (SpringerVerlag, 1967).

X

JOHN F. HART

REFERENCES [1] Ashby, W. R., An Introduction to Cybernetics (New York: John Wiley & Son, 1964). [2] Bar-Hillel, Y., Languages and Information (New York: John Wiley & Son, 1964). [3] Chomsky, N., Formal Properties of Grammars: Handbook of Mathematical Psychology, Vol. 2 (New York: John Wiley & Son, 1963), 323-418. [4] Davis, M., Computability & Unsolvability (New York: McGraw-Hill, 1958). [5] Ginsburg, S., An Introduction to Mathematical Machine Theory (Reading, Mass. : Addison-Wesley, 1962). [6] Harrison, M. A., Introduction to Switching and Automata Theory (New York: McGrawHill, 1965). [7] Hartmanis, R., Lewis, P. M., and Stearns, R. E., "Classifications of Computations by Time and Memory Requirements," Proc. of IFID Congress (Washington, D.C. : Spartan Books, 1965). [8] Hartmanis, J., and Stearns, R. E., "On the Computational Complexity of Algorithms," Trans. American Mathematical Society, 117(5) (May, 1965), 285-306. [9] Hermes, H., Aufzàhlbarkeit, Entscheidbarkeit, Berechenbarkeit (Berlin: SpringerVerlag, 1961). [10] Kleene, S. C., "Representation of Events in Nerve Nets and Finite Automata," in Automata Studies (Princeton, N.J. : Princeton University Press, 1956). [11] Moore, E. F., "Gedanken Experiments on Sequential Machines," in Automata Studies (Princeton, N.J. : Princeton University Press, 1956). [12] éd., Sequential Machines: Selected Papers (Reading, Mass. : Addison-Wesley, 1964). [13] Newell, A., Shaw, J. C., and Simon, H. A., "Report on a General Problem-Solving Program," in Information Processing, Proc. International Conference on Information Processing, UNESCO (Paris: June, 1959), 256-64. [14] Ritchie, R. W., "Classes of Predictably Computable Functions," Trans. American Mathematical Society, 106 (1963), 139-73. [15] Smullyan, R. M., Theory of Formal Systems (Princeton, N.J.: Princeton University Press, 1961). [16] Trakhtenbrot, B. A., Algorithms and Automatic Computing Machines (Boston: D. C. Heath and Company, 1963). [17] Turing, A. M., "On Computable Numbers, with an Application to the Entscheidungsproblem," Proc. of the London Mathematical Society, 2, 42 (1936-7), 230-65. [18] Yamada, H., "Counting by a Class of Growing Automata" (Philadelphia: Doctoral Thesis, University of Pennsylvania, 1960). [19] Zadeh, L. A., and Desoer, C. A., Linear System Theory: The State Space Approach (New York: McGraw-Hill, 1963).

Acknowled gment

THE EDITORS are pleased to acknowledge financial support for the Conference from the National Research Council of Canada. They would also like to thank the University of Western Ontario for making available a grant to assist with the publication. The Information Services and Publications Department of the University has been particularly helpful in assisting with publishing details. Among those who made contributions to the book, we would like to acknowledge a special debt to Professor R. B. Banerji of Case Institute for his overall support and criticism. Thanks are also due to Professor R. R. Korfhage of Purdue University for his criticism of Professor Corn's paper and to Professor G. Ernst of Case Institute for his assistance in connection with Dr. Amarel's contribution. Finally, we thank the Programme Committee of the Computer Science Association, consisting of Dr. S. D. Baxter, National Research Council, Professor W. Graham, University of Waterloo, Professor L. Robichaud, l'université Laval, and Professor N. Shklov, University of Saskatchewan.

This page intentionally left blank

Contents

PREFACE

v

ACKNOWLEDGMENT

xi

On the Structure of Finite Automata, J. HARTMANIS

3

Synthesis of Sequential Machines, J. A. BRZOZOWSKI

14

Techniques for Manipulating Regular Expressions, ROBERT McNAUGHTON

27

Some Comments on Self-Reproducing Automata, MICHAEL A. ARBIB

42

Multiple Control Computer Models, C. C. ELGOT, A. ROBINSON, and J. D. RUTLEDGE

60

Explicit Definitions and Linguistic Dominoes, SAUL GORN

77

Heuristic and Complete Processes in the Mechanization of Theorem Proving, J. A. ROBINSON

116

An Approach to Heuristic Problem Solving and Theorem Proving in the Propositional Calculus, SAUL AMAREL

125

New Directions in General Theory of Systems, MIHAJLO D. MESAROVI

221

Concerning an Algebraic Theory of Systems, THOMAS G. WINDEKNECHT

232

This page intentionally left blank

SYSTEMS AND COMPUTER SCIENCE

This page intentionally left blank

On the Structure of Finite Automata J. Hartmanis*

INTRODUCTION

The purpose of this paper is to give an elementary and unified exposition of some of the recent results about the structure of finite automata. The principal problem is how a finite automaton can be realized by the interconnection of several smaller automata. Or, more generally, how can a finite automaton be realized by several simpler automata, how these simpler automata have to be interconnected, and how are they related to the automaton being realized? These problems are quite similar to ones pursued in modern algebra where the problem is to replace a given algebraic system by several simpler systems of the same type. A good example is provided by groups and their rich structure theory. Because of this similarity between the basic problems in the structure theories of automata and algebra, automata theory has borrowed heavily from algebra and has utilized directly several techniques and results. On the other hand, the structure theory for automata shows many interesting differences with previously studied algebraic structures which gives this work its own characteristic flavour. Though in this paper we are concerned entirely with the problems of realizing a given automaton from several simpler automata, there are other very interesting problems in the structure theory of automata which are not of this type. These are the problems related to how the basic memory and logical units can be interconnected to realize an automaton. In these problems we are not just breaking down an automaton into smaller automata, but we are interested further in how the automaton or the component automata can be broken down into more elementary building-blocks from which physical realizations of these devices can be constructed. We want to know how the topology of the interconnections between these atomic building-blocks affect the computational characteristics of the automaton and vice versa. An interesting example of work in this area is provided by the recent result of A. Friedman, which shows that every finite automaton can be realized by an automaton with only one binary feedback loop. Problems of this type do not have direct counterparts in algebra but form a very characteristic *Department of Computer Science, Cornell University. 3

4

J. HARTMANIS

part of automata theory. Possibly because they do not have algebraic counterparts the research in this area has not progressed as rapidly as in the area of breaking down an automaton into simpler component automata. At the same time, I believe that there is a unifying concept which is helpful in the study and exposition of both types of structure problems. This is the concept of "ignorance" or "information flow" in automata. This concept has been given precise algebraic formulation which preserves much of its intuitive appeal, and it has already found a number of interesting applications in automata theory [1]. ELEMENTARY DECOMPOSITIONS

We now turn to the problem of realizing an automaton from simpler component automata, and we use the concept of "ignorance" to yield a unified approach to these problems. This approach also shows how the generality of our concept of ignorance is related to the generality of our structural results. A finite automaton is a quintuplet where 5, /, and 0 are the finite non-empty sets of states, inputs, and outputs, respectively ; is the next state function, and is the output function. For sake of brevity we omit in most of our discussion the output function and concentrate on the state behaviour of the machine. An automaton A' is a homomorphic image of A if and only if there exist three mappings

such that and

If the three mappings are one-to-one, then they define an isomorphism between A and A'. Again for the sake of simplicity we omit the output function and assume that 1 = 1' and hi = identity function.

ON THE STRUCTURE OF FINITE AUTOMATA

5

To gain some insight in the decomposition problem we start with a simple concept of "A realizes A'," and then derive a corresponding ignorance concept and decomposition results. Both these concepts are generalized later in this paper to obtain far more general decomposition results. DEFINITION 1. An automaton A realizes A' if and only if A' is isomorphic to (a sub-automaton of) A. DEFINITION 2. The parallel connection of the automata

is the automaton with the next state function

We say that an automaton A has a parallel decomposition into A\ and AÍ, if and only if A is realized by the parallel connection of A\ and A^. To illustrate the realization and decomposition concepts consider automaton A of Fig. 1 and the parallel connection of automata A\ and AI of Fig. 2. It is easily seen that the two automata are isomorphic and the mapping hi is given by

FIGURE 1

FIGURE 2

6

J. HARTMANIS

Thus, in the parallel connection of A i and A 2, the state pair uniquely determines a state of A. On the other hand, each of the component automata separately can only have partial information about the state of A. For example, the state of AÏ can only determine whether the present state of A is 1, 3, 5 or whether it is one of the states 2, 4, 6. Thus A-¡ just keeps track of whether A is in a state designated by an odd or even number. Therefore, the ignorance which automaton A i has about the state of A can be described by the partition

Similarly, the ignorance of A\ about the state of A is described by the partition

Furthermore, we see that this ignorance is such that it does not increase when the automaton A operates. Equivalently, we can say that the blocks of -K\ and 1T2 are again mapped into blocks of these partitions as the automaton A operates. It will be seen, after we make these concepts precise, that they are sufficient to describe all parallel and serial realizations of finite automata in the sense of Definition 1. DEFINITION 3. A partition TT on the set of states of A has the substitution property (S.P.) if and only if for every input x and block B of ir(B £ IT) there is a block B' in IT such that It is easily verified that whenever an automaton A is realized by a parallel connection of the automata AI and A2 then, just as in the previous example, the information contained in A i and A i about the state of A is characterized by two partitions with S.P., irt and ir¡. To see this we merely observe that if we have a realization then we have an isomorphism and we define the blocks of TTI to be and, similarly, the blocks of v^ are given by The equations defining a homomorphism can now be used to verify that TI and 7T2 have the substitution property. Finally, it can be seen that wi and ir2 are such that they do not both identify the same pair of states. We write

ON THE STRUCTURE OF FINITE AUTOMATA

7

From these observations we can derive necessary and sufficient conditions that an automaton can be realized (in the sense of Definition 1) from two smaller component automata. THEOREM 1. An automaton A can be realized by a parallel connection of two smaller automata if and only if there exist two non-trivial partitions TTI and 7r2 on the set of states of A with the substitution property such that If we make the natural definition of a serial connection of two automata A\ and A2, then we can again, utilizing a simple argument about ignorance defined in terms of partitions, derive necessary and sufficient conditions that A has a serial decomposition. THEOREM 2. An automaton A can be realized by the serial connection of two smaller automata if and only if there exists a non-trivial S.P. partition IT on the set of states of A. These structure results are very similar to the results which were previously obtained in modern and universal algebra where homomorphisms and congruence relations (S.P. partitions) play a dominant role. Though these concepts are sufficient for most of the structure theory of algebra they are not sufficient for automata theory. The reason for this, once seen, is quite simple. Algebra uses basically the same realization concept as we have done so far. This is the realization concept formulated in terms of isomorphic subsystems. This works well in algebra where the main interest is concentrated on the elements of the algebra and relations between these elements. In automata theory these roughly correspond to the states of the automaton. It turns out though that for automata the states are only auxiliary concepts which were used to define the computation performed by the automaton. We could use many other automata with more states to perform the same computation. Some of these equivalent automata (which have the same input-output behaviour) may be decomposable where others may not. For example, automaton B of Fig. 3 has only trivial S.P. partitions and thus does not have a decomposition. On the other hand, the equivalent automaton B' of Fig. 4 has the S.P. partition and thus it has a realization by a serial connection of a two-state and a three-state automaton. Since both of these component automata have fewer states than B, this looks like a reasonable realization of B, but it is not given or admitted by our previous results. It is actually not very hard to construct examples of automata for which a larger equivalent automaton is easier to build (same number of binary memory elements and simpler combinational logic). Thus, even from a very practical point of view, our concept of realization and the corresponding concept of ignorance are too narrow [1].

8

J. HARTMANIS

FIGURE 3

SET SYSTEM

FIGURE 4

DECOMPOSITIONS

We now proceed to generalize our concept of "A realizes A'" and to derive the corresponding ignorance concept to overcome the previously encountered difficulties and to get more general decomposition results. DEFINITION 4. An automaton A realizes the automaton A' if and only if A' is a homomorphic image of (a sub-automaton of) A. This definition is sufficiently general for the purposes of this paper, and it includes almost all the realizations which we intuitively feel should be included. Still, it omits a few cases and, at the expense of making most theorems more cumbersome, it can be generalized (see [1]). To understand what ignorance definition should go with this definition of realization we look at our previous example. When we reduce automaton B' to B, by identifying states 3 and 3', then the blocks of the S.P. partition are mapped onto the sets We can say that w is mapped onto the "overlapping partition"

This $ is such that the blocks of it are again mapped into its blocks by B, and we can build a two-state automaton which keeps track of whether B is in one of the states 1, 2, 3 or one of the states 3, 4. Thus we are led to a new concept of ignorance which we now make precise. DEFINITIONS. A set system = {S t} on 5 is a collection of subsets of 5 such that and

ON THE STRUCTURE OF FINITE AUTOMATA

9

Thus a set system on 5 is a collection of subsets of 5 whose union is S (i.e., the St cover S) and among which there do not exist proper containments. DEFINITION 6. A set system has the substitution property with respect to an automaton A if and only if for every input x and B in there is a B' in such that Thus the S.P. set systems for an automaton describe this newly defined ignorance which does not spread as the automaton operates. It should be observed that if w is an S.P. partition on the automaton A, then the image automaton (whose states are the blocks of IT and which computes the block of TT which contains the state of A) is uniquely determined by A and IT. This, as the next example illustrates, is not the case for set systems with S.P. Consider automaton C of Fig. 5. The set system has S.P. In the first column of C each Bf of $ is mapped onto a unique and different Bt of . In the second column all Bt of # are mapped into and in the third column all Bt of 3. According to Theorem 3, therefore, this set system gives a realization of an w-state automaton by a serial connection of an w-state and an (n — l)-state automata. This may not be useful for practical purposes but we will make use of this particular S.P. set system decomposition to get a very general decomposition result later. The next result describes a necessary and sufficient condition that there exists a realization of an automaton by a serial connection of two smaller automata. The proof of this result is not hard and follows roughly the same reasoning as the proof of Theorem 2. THEOREM 5. An n-state automaton A can be realized by a serial connection of a k-state and l-state automata, k, I < n, if and only if there exists an S.P. set system with no more than k blocks and with no more than I elements in the largest block of . From these results we see that with the new concept of realization (Definition 4) goes the new set system concept of ignorance. The set systems and other concepts derived from them have a wide range of applications. In particular, the idea of set system pairs, which describe how information or ignorance spreads in an automaton when it operates, is a useful concept and has found interesting applications. These ideas are discussed in more detail in [2]. A CONONICAL DECOMPOSITION

We now start the derivation of a general existence result which describes a cononical set of automata from which any automaton can be realized. We saw before that for the set system the automaton C had an image automaton C« which had only permutation and reset inputs. We now define a permutation-reset automaton and show that for

ON THE STRUCTURE OF FINITE AUTOMATA

11

any automaton there exists an image automaton which is a permutation-reset automaton. DEFINITION 7. An automaton is said to be a permutation-reset, P-R, automaton if every input either permutes the set of states or maps all states onto a single state. LEMMA 1. Every automaton can be realized by a serial connection of P-R automata. Proof. If A has one or two states, then it is a P-R automaton. If A has more than two states let We know that has S.P. since every /-state block of an automaton is always mapped into some /-state block by every input. Furthermore, just like in the case of automaton C, there is an image automaton A^ which is a P-R automaton. This is so because permutation inputs are again permutation inputs under and if some input is not a permutation input then a state does not appear in the input column and there is some B in