Introduction to the Theory of Computation, Second Edition pdf

440  26 

Loading....

Loading....

Loading....

Loading....

Loading....

Full text

(1)
(2)

INTRODUCTION TO THE

THEORY OF COMPUTATION,

SECOND EDITION

MICHAEL SIPSER

Massachusetts Institute of Technology

THOMSON COURSE TECHNOLOGY

(3)

COURSE TECHNOLOGY

Introduction to the Theory of Computation, Second Edition

by Michael Sipser

Senior Product Manager: Alyssa Pratt

Executive Editor: Mac Mendelsohn

Associate Production Manager: Aimee Poirier

Senior Marketing Manager: Karen Seitz

COPYRIGHT

©

2006 Thomson Course Technology, a division of Thomson Learning, Inc. Thomson LearningTM is a trademark used herein under license.

Printed in the United States of America 1 2 3 45 67 89 QWT 0908070605 For more information, contact Thomson Course Technology 25 Thomson Place

Boston, Massachusetts, 02210. Or find us on the World Wide Web at: www.course.com

ALL RIGHTS RESERVED. No part of this work covered by the copyright hereon may be reproduced or used in any form or by any means graphic, electronic, or mechanical, including photocopying, recording, taping, Web distribution, or information storage and retrieval systems without the written permission of the publisher.

Associate Product Manager: Mirella Misiaszek

Editorial Assistant: Jennifer Smith

Senior Manufacturing Coordinator: Trevor Kallop

Cover Designer: Steve Deschesne

For permission to use material from this text or product, submit a request online at http://www.thomsonrights.com Any additional questions about

permissions can be submitted by e-mail to thomsonrights~thomson.com

Disclaimer

Thomson Course Technology reserves the right to revise this publication and make changes from time to time in its content without notice.

(4)
(5)

Preface to the First Edition xi

To the student .... ... xi

To the educator ... xii

The first edition ... ... xiii

Feedback to the author ... xiii

Acknowledgments ... ... xiv

Preface to the Second Edition xvii 0 Introduction 1 0.1 Automata, Computability, and Complexity . . . 1

Complexity theory ... 2

Computability theory ... 2

Automata theory ... 3

0.2 Mathematical Notions and Terminology ... 3

Sets ... ... 3

Sequences and tuples ... ... 6

Functions and relations ... ... 7

G raphs . . . 10

Strings and languages . . . 13

Boolean logic ... ... 14

Summary of mathematical terms . . . 16

0.3 Definitions, Theorems, and Proofs .. ... 17

Finding proofs . . . 17

0.4 Types of Proof . . . 21

Proof by construction ... 221

Proof by contradiction . . . 21

Proof by induction ... 22

Exercises, Problems, and Solutions ... .. 25

(6)

Vi CONTENTS

Part One: Automata and Languages

1 Regular Languages

1.1 Finite Automata...

Formal definition of a finite automaton ... Examples of finite automata ...

Formal definition of computation ... Designing finite automata ... The regular operations ...

1.2 Nondeterminism ...

Formal definition of a nondeterministic finite aut Equivalence of NFAs and DFAs ...

Closure under the regular operations ...

1.3 Regular Expressions ...

Formal definition of a regular expression ....

Equivalence with finite automata ... 1.4 Nonregular Languages ...

The pumping lemma for regular languages . . .

Exercises, Problems, and Solutions ...

2 Context-Free Languages

2.1 Context-free Grammars ...

Formal definition of a context-free grammar . . Examples of context-free grammars ... Designing context-free grammars ...

Ambiguity . . . . Chomsky normal form ...

2.2 Pushdown Automata ...

Formal definition of a pushdown automaton . . . Examples of pushdown automata ...

Equivalence with context-free grammars ... 2.3 Non-context-free Languages ...

The pumping lemma for context-free languages.

Exercises, Problems, and Solutions ...

Part Two: Computability Theory

. . . .

. . . .

. . . .

. . . .

tomaton .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

29

31 31 35 37 40 41 44 47 53 54 58 63 64 66 77 77 82 99 100 102 103 104 105 106 109 111 112 115 123 123 128

135

3 The Church-Turing Thesis 137

3.1 Turing Machines ... 137

Formal definition of a Turing machine ... 139

Examples of Turing machines ... .142

3.2 Variants of Turing Machines . . . 148

Multitape Turing machines ... .148

Nondeterministic Turing machines ... . 150

(7)

Equivalence with other models ... ... 153

3.3 The Definition of Algorithm ... ... 154

Hilbert's problems ... 154

Terminology for describing Turing machines . . . 156

Exercises, Problems, and Solutions ... 159

4 Decidability 165 4.1 Decidable Languages ... 166

Decidable problems concerning regular languages ... 166

Decidable problems concerning context-free languages . . . 170

4.2 The Halting Problem .... ... 173

The diagonalization method . . . 174

The halting problem is undecidable . . . 179

A Turing-unrecognizable language . . . 181

Exercises, Problems, and Solutions ... 182

5 Reducibility 187 5.1 Undecidable Problems from Language Theory ... 188

Reductions via computation histories ... 192

5.2 A Simple Undecidable Problem ... 199

5.3 Mapping Reducibility . . . 206

Computable functions . . . 206

Formal definition of mapping reducibility . ... 207

Exercises, Problems, and Solutions ... 211

6 Advanced Topics in Computability Theory 217 6.1 The Recursion Theorem . . . 217

Self-reference ... .218

Terminology for the recursion theorem . . . 221

Applications ... . 222

6.2 Decidability of logical theories ... ... 224

A decidable theory ... 227

An undecidable theory ... 229

6.3 Turing Reducibility . . . 232

6.4 A Definition of Information . . . 233

Minimal length descriptions ... ... 234

Optimality of the definition . . . 238

Incompressible strings and randomness ... ... 239

Exercises, Problems, and Solutions ... 242

Part Three: Complexity Theory

245

7 Time Complexity 247 7.1 Measuring Complexity ... . 247

(8)

Viii CONTENTS

Analyzing algorithms ... .. 251

Complexity relationships among models ... ... 254

7.2 The Class P . . . 256

Polynomial time . . . 256

Examples of problems in P ... .258

7.3 The Class NP . . . 264

Examples of problems in NP ... 267

The P versus NP question ... .269

7.4 NP-completeness . . . 271

Polynomial time reducibility ... 272

Definition of NP-completeness ... . 276

The Cook-Levin Theorem . . . 276

7.5 Additional NP-complete Problems ... 283

The vertex cover problem . . . 284

The Hamiltonian path problem ... ... 286

The subset sum problem ... 291

Exercises, Problems, and Solutions ... 294

8 Space Complexity 303 8.1 Savitch's Theorem ... .305

8.2 The Class PSPACE ... .308

8.3 PSPACE-completeness . . . 309

The TQBF problem . . . 310

Winning strategies for games . . . 313

Generalized geography .. ... 315

8.4 The Classes L and NL ... 320

8.5 NL-completeness . . . 323

Searching in graphs ... 325

8.6 NL equals coNl ... .. 326

Exercises, Problems, and Solutions ... 328

9 Intractability 335 9.1 Hierarchy Theorems ... 336

Exponential space completeness ... ... 343

9.2 Relativization ... 348

Limits of the diagonalization method ... ... 349

9.3 Circuit Complexity ... 351

Exercises, Problems, and Solutions ... 360

10 Advanced topics in complexity theory 365 10.1 ApproximationAlgorithms ... 365

10.2 Probabilistic Algorithms ... 368

The class BPP ... .. 368

Prim ality . . . 371

Read-once branching programs ... . 376

(9)

Alternating time and space . . . The Polynomial time hierarchy . 10.4 Interactive Proof Systems ...

Graph nonisomorphism ... Definition of the model ... IP = PSPACE ... 10.5 Parallel Computation...

Uniform Boolean circuits .... The class NC ...

P-completeness . . . .

10.6 Cryptographyh.y... Secret keys ...

Public-key cryptosystems . ...

One-way functions ... Trapdoor functions ...

Exercises, Problems, and Solutions . .

Selected Bibliography

Index

381 386 387 387 388 390 399 400 402 404 405 405 407 407 409 411

415

(10)

PREFACE TO THE

FIRST EDITION

TO THE STUDENT

Welcome!

You are about to embark on the study of a fascinating and important subject: the theory of computation. It comprises the fundamental mathematical proper-ties of computer hardware, software, and certain applications thereof. In

study-ing this subject we seek to determine what can and cannot be computed, how

quickly, with how much memory, and on which type of computational model. The subject has obvious connections with engineering practice, and, as in many sciences, it also has purely philosophical aspects.

I know that many of you are looking forward to studying this material but some may not be here out of choice. You may want to obtain a degree in com-puter science or engineering, and a course in theory is required-God knows why. After all, isn't theory arcane, boring, and worst of all, irrelevant?

To see that theory is neither arcane nor boring, but instead quite understand-able and even interesting, read on. Theoretical computer science does have many fascinating big ideas, but it also has many small and sometimes dull details that can be tiresome. Learning any new subject is hard work, but it becomes easier and more enjoyable if the subject is properly presented. My primary ob-jective in writing this book is to expose you to the genuinely exciting aspects of computer theory, without getting bogged down in the drudgery. Of course, the only way to determine whether theory interests you is to try learning it.

(11)

Theory is relevant to practice. It provides conceptual tools that practition-ers use in computer engineering. Designing a new programming language for a specialized application? What you learned about grammars in this course comes in handy. Dealing with string searching and pattern matching? Rememberfinite

automata and regular expressions. Confronted with a problem that seems to

re-quire more computer time than you can afford? Think back to what you learned about NP-completeness. Various application areas, such as modern cryptographic protocols, rely on theoretical principles that you will learn here.

Theory also is relevant to you because it shows you a new, simpler, and more elegant side of computers, which we normally consider to be complicated ma-chines. The best computer designs and applications are conceived with elegance in mind. A theoretical course can heighten your aesthetic sense and help you build more beautiful systems.

Finally, theory is good for you because studying it expands your mind. Com-puter technology changes quickly. Specific technical knowledge, though useful today, becomes outdated in just a few years. Consider instead the abilities to think, to express yourself clearly and precisely, to solve problems, and to know when you haven't solved a problem. These abilities have lasting value. Studying theory trains you in these areas.

Practical considerations aside, nearly everyone working with computers is cu-rious about these amazing creations, their capabilities, and their limitations. A whole new branch of mathematics has grown up in the past 30 years to answer certain basic questions. Here's a big one that remains unsolved: If I give you a large number, say, with 500 digits, can you find its factors (the numbers that di-vide it evenly), in a reasonable amount of time? Even using a supercomputer, no one presently knows how to do that in all cases within the lifetime of the universe! The factoring problem is connected to certain secret codes in modern

cryptosys-tems. Find a fast way to factor and fame is yours!

TO THE EDUCATOR

This book is intended as an upper-level undergraduate or introductory gradu-ate text in computer science theory. It contains a mathematical treatment of the subject, designed around theorems and proofs. I have made some effort to accommodate students with little prior experience in proving theorems, though more experienced students will have an easier time.

My primary goal in presenting the material has been to make it clear and interesting. In so doing, I have emphasized intuition and "the big picture" in the subject over some lower level details.

(12)

PREFACE TO THE FIRST EDITION Xiii

an induction risks teaching students that mathematical proof is a formal manip-ulation instead of teaching them what is and what is not a cogent argument.

A second example occurs in Parts Two and Three, where I describe algorithms in prose instead of pseudocode. I don't spend much time programming Turing machines (or any other formal model). Students today come with a program-ming background and find the Church-Turing thesis to be self-evident. Hence I don't present lengthy simulations of one model by another to establish their equivalence.

Besides giving extra intuition and suppressing some details, I give what might be called a classical presentation of the subject material. Most theorists will find the choice of material, terminology, and order of presentation consistent with that of other widely used textbooks. I have introduced original terminology in only a few places, when I found the standard terminology particularly obscure or confusing. For example I introduce the term mapping reducibility instead of

many-one reducibility.

Practice through solving problems is essential to learning any mathemati-cal subject. In this book, the problems are organized into two main categories called Exercises and Problems. The Exercises review definitions and concepts. The Problems require some ingenuity. Problems marked with a star are more difficult. I have tried to make both the Exercises and Problems interesting chal-lenges.

THE FIRST EDITION

Introduction to the Theory of Computation first appeared as a Preliminary Edition

in paperback. The first edition differs from the Preliminary Edition in several substantial ways. The final three chapters are new: Chapter 8 on space complex-ity; Chapter 9 on provable intractabilcomplex-ity; and Chapter 10 on advanced topics in complexity theory. Chapter 6 was expanded to include several advanced topics in computability theory. Other chapters were improved through the inclusion

of additional examples and exercises.

Comments from instructors and students who used the Preliminary Edition were helpful in polishing Chapters 0-7. Of course, the errors they reported have been corrected in this edition.

Chapters 6 and 10 give a survey of several more advanced topics in com-putability and complexity theories. They are not intended to comprise a cohesive unit in the way that the remaining chapters are. These chapters are included to allow the instructor to select optional topics that may be of interest to the serious student. The topics themselves range widely. Some, such as Turing reducibility and alternation, are direct extensions of other concepts in the book. Others, such as decidable logical theories and cryptography, are brief introductions to large fields.

FEEDBACK TO THE AUTHOR

(13)

and reporting errors for the Preliminary Edition. Please continue to correspond! I try to respond to each message personally, as time permits. The e-mail address for correspondence related to this book is

sipserbookfmath.mit.edu.

A web site that contains a list of errata is maintained. Other material may be added to that site to assist instructors and students. Let me know what you would like to see there. The location for that site is

http://www-math.mit.edu/-sipser/book.html.

ACKNOWLEDGM ENTS

I could not have written this book without the help of many friends, colleagues,

and my family.

I wish to thank the teachers who helped shape my scientific viewpoint and educational style. Five of them stand out. My thesis advisor, Manuel Blum, is due a special note for his unique way of inspiring students through clarity of thought, enthusiasm, and caring. He is a model for me and for many others. I am grateful to Richard Karp for introducing me to complexity theory, to John Addison for teaching me logic and assigning those wonderful homework sets, to Juris Hartmanis for introducing me to the theory of computation, and to my father for introducing me to mathematics, computers, and the art of teaching.

This book grew out of notes from a course that I have taught at MIT for the past 15 years. Students in my classes took these notes from my lectures. I hope they will forgive me for not listing them all. My teaching assistants over the years, Avrim Blum, Thang Bui, Andrew Chou, Benny Chor, Stavros Cos-madakis, Aditi Dhagat, Wayne Goddard, Parry Husbands, Dina Kravets, Jakov Kucan, Brian O'Neill, loana Popescu, and Alex Russell, helped me to edit and expand these notes and provided some of the homework problems.

Nearly three years ago, Tom Leighton persuaded me to write a textbook on the theory of computation. I had been thinking of doing so for some time, but it took Tom's persuasion to turn theory into practice. I appreciate his generous advice on book writing and on many other things.

I wish to thank Eric Bach, Peter Beebee, Cris Calude, Marek Chrobak, Anna Chefter, Guang-len Cheng, Elias Dahlhaus, Michael Fischer, Steve Fisk, Lance Fortnow, Henry J. Friedman, Jack Fu, Seymour Ginsburg, Oded Goldreich, Brian Grossman, David Harel, Micha Hofri, Dung T. Huynh, Neil Jones, H. Chad Lane, Kevin Lin, Michael Loui, Silvio Micali, Tadao Murata, Chris-tos Papadimitriou, Vaughan Pratt, Daniel Rosenband, Brian Scassellati, Ashish Sharma, Nir Shavit, Alexander Shen, Ilya Shlyakhter, Matt Stallmann, Perry Susskind, Y. C. Tay, Joseph Traub, Osamu Watanabe, Peter Widmayer, David Williamson, Derick Wood, and Charles Yang for comments, suggestions, and assistance as the writing progressed.

(14)

Ather-PREFACE TO THE FIRST EDITION XV

ton, Rolfe Blodgett, Al Briggs, Brian E. Brooks, Jonathan Buss, Jin Yi Cai, Steve Chapel, David Chow, Michael Ehrlich, Yaakov Eisenberg, Farzan Fallah, Shaun Flisakowski, Hjalmtyr Hafsteinsson, C. R. Hale, Maurice Herlihy, Vegard Holmedahl, Sandy Irani, Kevin Jiang, Rhys Price Jones, James M. Jowdy, David M. Martin Jr., Manrique Mata-Montero, Ryota Matsuura, Thomas Minka, Farooq Mohammed, Tadao Murata, Jason Murray, Hideo Nagahashi, Kazuo Ohta, Constantine Papageorgiou, Joseph Raj, Rick Regan, Rhonda A. Reumann, Michael Rintzler, Arnold L. Rosenberg, Larry Roske, Max Rozenoer, Walter L. Ruzzo, Sanatan Sahgal, Leonard Schulman, Steve Seiden, Joel Seiferas, Ambuj Singh, David J. Stucki, Jayram S. Thathachar, H. Venkateswaran, Tom Whaley, Christopher Van Wyk, Kyle Young, and Kyoung Hwan Yun.

Robert Sloan used an early version of the manuscript for this book in a class that he taught and provided me with invaluable commentary and ideas from his experience with it. Mark Herschberg, Kazuo Ohta, and Latanya Sweeney read over parts of the manuscript and suggested extensive improvements. Shafi Goldwasser helped me with material in Chapter 10.

I received expert technical support from William Baxter at Superscript, who wrote the LATEX macro package implementing the interior design, and from Larry Nolan at the MIT mathematics department, who keeps everything run-ning.

It has been a pleasure to work with the folks at PWS Publishing in

creat-ing the final product. I mention Michael Sugarman, David Dietz, Elise Kaiser,

Monique Calello, Susan Garland and Tanja Brull because I have had the most contact with them, but I know that many others have been involved, too. Thanks to Jerry Moore for the copy editing, to Diane Levy for the cover design, and to Catherine Hawkes for the interior design.

I am grateful to the National Science Foundation for support provided under grant CCR-9503322.

My father, Kenneth Sipser, and sister, Laura Sipser, converted the book di-agrams into electronic form. My other sister, Karen Fisch, saved us in various computer emergencies, and my mother, Justine Sipser, helped out with motherly advice. I thank them for contributing under difficult circumstances, including insane deadlines and recalcitrant software.

Finally, my love goes to my wife, Ina, and my daughter, Rachel. Thanks for putting up with all of this.

Cambridge, Massachusetts Michael Sipser

(15)

SECOND EDITION

Judging from the email communications that I've received from so many of you, the biggest deficiency of the first edition is that it provides no sample solutions to any of the problems. So here they are. Every chapter now contains a new

Selected Solutions section that gives answers to a representative cross-section of

that chapter's exercises and problems. To make up for the loss of the solved problems as interesting homework challenges, I've also added a variety of new problems. Instructors may request an Instructor's Manual that contains addi-tional solutions by contacting the sales representative for their region designated at www. course. comn.

A number of readers would have liked more coverage of certain "standard" topics, particularly the Myhill-Nerode Theorem and Rice's Theorem. I've par-tially accommodated these readers by developing these topics in the solved prob-lems. I did not include the Myhill-Nerode Theorem in the main body of the text because I believe that this course should provide only an introduction to finite automata and not a deep investigation. In my view, the role of finite automata here is for students to explore a simple formal model of computation as a prelude to more powerful models, and to provide convenient examples for subsequent topics. Of course, some people would prefer a more thorough treatment, while others feel that I ought to omit all reference to (or at least dependence on) finite automata. I did not include Rice's Theorem in the main body of the text be-cause, though it can be a useful "tool" for proving undecidability, some students might use it mechanically without really understanding what is going on. Using

(16)

XViii PREFACE TO THE SECOND EDITION

reductions instead, for proving undecidability, gives more valuable preparation for the reductions that appear in complexity theory.

I am indebted to my teaching assistants, Ilya Baran, Sergi Elizalde, Rui Fan, Jonathan Feldman, Venkatesan Guruswami, Prahladh Harsha, Christos Kapout-sis, Julia Khodor, Adam Klivans, Kevin Matulef, loana Popescu, April Rasala, Sofya Raskhodnikova, and Iuliu Vasilescu who helped me to craft some of the new problems and solutions. Ching Law, Edmond Kayi Lee, and Zulfikar Ramzan also contributed to the solutions. I thank Victor Shoup for coming up with a simple way to repair the gap in the analysis of the probabilistic primality algorithm that appears in the first edition.

I appreciate the efforts of the people at Course Technology in pushing me and the other parts of this project along, especially Alyssa Pratt and Aimee Poirier. Many thanks to Gerald Eisman, Weizhen Mao, Rupak Majumdar, Chris Umans, and Christopher Wilson for their reviews. I'm indebted to Jerry Moore for his superb job copy editing and to Laura Segel of ByteGraphics

(lauras~bytegraphics. com) for her beautifully precise rendition of the

fig-ures.

The volume of email I've received has been more than I expected. Hearing from so many of you from so many places has been absolutely delightful, and I've tried to respond to all eventually-my apologies for those I missed. I've listed here the people who made suggestions that specifically affected this edition, but I thank everyone for their correspondence.

(17)

Ger-man Muller, Donald Nelson, Gabriel Nivasch, Mary Obelnicki, Kazuo Ohta, Thomas M. Oleson, Jr., Curtis Oliver, Owen Ozier, Rene Peralta, Alexander Perlis, Holger Petersen, Detlef Plump, Robert Prince, David Pritchard, Bina Reed, Nicholas Riley, Ronald Rivest, Robert Robinson, Christi Rockwell, Phil Rogaway, Max Rozenoer, John Rupf, Teodor Rus, Larry Ruzzo, Brian Sanders, Cem Say, Kim Schioett, Joel Seiferas, Joao Carlos Setubal, Geoff Lee Seyon, Mark Skandera, Bob Sloan, Geoff Smith, Marc L. Smith, Stephen Smith, Alex C. Snoeren, Guy St-Denis, Larry Stockmeyer, Radu Stoleru, David Stucki, Hlisham M. Sueyllam, Kenneth Tam, Elizabeth Thompson, Michel Toulouse, Fric Tria, Chittaranjan Tripathy, Dan Trubow, Hiroki Ueda, Giora Unger, Kurt L. Van Etten, Jesir Vargas, Bienvenido Velez-Rivera, Kobus Vos, Alex Vrenios, Sven Waibel, Marc Waldman, Tom Whaley, Anthony Widjaja, Sean Williams, Joseph N. Wilson, Chris Van Wyk, Guangming Xing, Vee Voon Yee, Cheng

Yongxi, Neal Young, Timothy Yuen, Kyle Yung, Jinghua Zhang, Lilla Zollei.

Most of all I thank my family-lna, Rachel, and Aaron-for their patience, understanding, and love as I sat for endless hours here in front of my computer screen.

Cambridge, 1lassachusetts Michael Sipser

(18)

INTRODUCTION

We begin with an overview of those areas in the theory of computation that we present in this course. Following that, you'll have a chance to learn and/or review some mathematical concepts that you will need later.

0.1

AUTOMATA, COMPUTABILITY, AND COMPLEXITY

This book focuses on three traditionally central areas of the theory of computa-tion: automata, computability, and complexity. They are linked by the quescomputa-tion:

What are the fundamental capabilities and limitations of computers?

This question goes back to the 1930s when mathematical logicians first began to explore the meaning of computation. Technological advances since that time have greatly increased our ability to compute and have brought this question out of the realm of theory into the world of practical concern.

In each of the three areas-automata, computability, and complexity-this question is interpreted differently, and the answers vary according to the inter-pretation. Following this introductory chapter, we explore each area in a sepa-rate part of this book. Here, we introduce these parts in reverse order because starting from the end you can better understand the reason for the beginning.

(19)

COMPLEXITY THEORY

Computer problems come in different varieties; some are easy, and some are hard. For example, the sorting problem is an easy one. Say that you need to arrange a list of numbers in ascending order. Even a small computer can sort a million numbers rather quickly. Compare that to a scheduling problem. Say that you must find a schedule of classes for the entire university to satisfy some reasonable constraints, such as that no two classes take place in the same room at the same time. The scheduling problem seems to be much harder than the sorting problem. If you have just a thousand classes, finding the best schedule may require centuries, even with a supercomputer.

What makes some problems computationally hard and others easy?

This is the central question of complexity theory. Remarkably, we don't know the answer to it, though it has been intensively researched for the past 3 5 years. Later, we explore this fascinating question and some of its ramifications.

In one of the important achievements of complexity theory thus far, re-searchers have discovered an elegant scheme for classifying problems according to their computational difficulty. It is analogous to the periodic table for clas-sifying elements according to their chemical properties. Using this scheme, we can demonstrate a method for giving evidence that certain problems are compu-tationally hard, even if we are unable to prove that they are.

You have several options when you confront a problem that appears to be computationally hard. First, by understanding which aspect of the problem is at the root of the difficulty, you may be able to alter it so that the problem is more easily solvable. Second, you may be able to settle for less than a perfect solution to the problem. In certain cases finding solutions that only approximate the perfect one is relatively easy. Third, some problems are hard only in the worst case situation, but easy most of the time. Depending on the application, you may be satisfied with a procedure that occasionally is slow but usually runs quickly. Finally, you may consider alternative types of computation, such as randomized computation, that can speed up certain tasks.

One applied area that has been affected directly by complexity theory is the ancient field of cryptography. In most fields, an easy computational problem is preferable to a hard one because easy ones are cheaper to solve. Cryptogra-phy is unusual because it specifically requires computational problems that are hard, rather than easy, because secret codes should be hard to break without the secret key or password. Complexity theory has pointed cryptographers in the direction of computationally hard problems around which they have designed revolutionary new codes.

COMPUTABILITY THEORY

(20)

prob-0.2 MATHEMATICAL NOTIONS AND TERMINOLOGY 3

lem of determining whether a mathematical statement is true or false. This task is the bread and butter of mathematicians. It seems like a natural for solution by computer because it lies strictly within the realm of mathematics. But no computer algorithm can perform this task.

Among the consequences of this profound result was the development of ideas concerning theoretical models of computers that eventually would help lead to the construction of actual computers.

The theories of computability and complexity are closely related. In

com-plexity theory, the objective is to classify problems as easy ones and hard ones, whereas in computability theory the classification of problems is by those that are solvable and those that are not. Computability theory introduces several of the concepts used in complexity theory.

AUTOMATA THEORY

Automata theory deals with the definitions and properties of mathematical mod-els of computation. These modmod-els play a role in several applied areas of computer science. One model, called the finite automaton, is used in text processing, com-pilers, and hardware design. Another model, called the context-free grammar, is used in programming languages and artificial intelligence.

Automata theory is an excellent place to begin the study of the theory of computation. The theories of computability and complexity require a precise definition of a computer. Automata theory allows practice with formal definitions of computation as it introduces concepts relevant to other nontheoretical areas of computer science.

0.2

MATHEMATICAL NOTIONS AND TERMINOLOGY

As in any mathematical subject, we begin with a discussion of the basic mathe-matical objects, tools, and notation that we expect to use.

SETS

A set is a group of objects represented as a unit. Sets may contain any type of

object, including numbers, symbols, and even other sets. The objects in a set are called its elements or members. Sets may be described formally in several ways. One way is by listing a set's elements inside braces. Thus the set

{7, 21, 57}

contains the elements 7, 21, and 57. The symbols e and f denote set

member-ship and nonmembermember-ship. We write 7 E {7, 21, 57} and 8 X {7, 21, 57}. For two

(21)

A also is a member of B. We say that A is a proper subset of B, written A c B,

if A is a subset of B and not equal to B.

The order of describing a set doesn't matter, nor does repetition of its mem-bers. We get the same set by writing {57, 7, 7, 7, 211. If we do want to take the number of occurrences of members into account we call the group a multiset

in-stead of a set. Thus {7} and {7, 7} are different as multisets but identical as sets.

An infinite set contains infinitely many elements. We cannot write a list of all

the elements of an infinite set, so we sometimes use the ". ." notation to mean,

"continue the sequence forever." Thus we write the set of natural numbers X

as

{1,2,3,...}.

The set of integers Z is written

{ . .. , -2, -1,0, 1,2,. .. }.

The set with 0 members is called the empty set and is written 0.

When we want to describe a set containing elements according to some rule,

we write {nj rule about n}. Thus {nj n = m2

for some m E A} means the set of

perfect squares.

If we have two sets A and B, the union of A and B, written AUB, is the set we get by combining all the elements in A and B into a single set. The intersection

of A and B, written A n B, is the set of elements that are in both A and B. The

complement of A, written A, is the set of all elements under consideration that are not in A.

As is often the case in mathematics, a picture helps clarify a concept. For sets, we use a type of picture called a Venn diagram. It represents sets as regions enclosed by circular lines. Let the set START-t be the set of all English words that

start with the letter "t. " For example, in the following figure the circle represents

the set START-t. Several members of this set are represented as points inside the circle.

terrific

tundra theory

FIGURE 0.1

Venn diagram for the set of English words starting with "t"

Similarly, we represent the set END-z of English words that end with "z" in

the following figure.

(22)

0.2 MATHEMATICAL NOTIONS AND TERMINOLOGY 5

END-Z

quartz jazz razzmatazz

FIGURE 0.2

Venn diagram for the set of English words ending with "z"

To represent both sets in the same Venn diagram we must draw them so that they overlap, indicating that they share some elements, as shown in the following figure. For example, the word topaz is in both sets. The figure also contains a

circle for the set START-j. It doesn't overlap the circle for START-t because no

word lies in both sets.

START-t END-Z START-j

FIGURE 0.3

Overlapping circles indicate common elements

The next two Venn diagrams depict the union and intersection of sets A and B.

A B

(a) (b)

FIGURE 0.4

(23)

SEQUENCES AND TUPLES

A sequence of objects is a list of these objects in some order. We usually designate

a sequence by writing the list within parentheses. For example, the sequence 7, 21, 57 would be written

(7, 21, 57).

In a set the order doesn't matter, but in a sequence it does. Hence (7,21, 57) is not the same as (57, 7, 21). Similarly, repetition does matter in a sequence, but it doesn't matter in a set. Thus (7, 7, 21, 57) is different from both of the other sequences, whereas the set {7, 21, 57} is identical to the set {7, 7, 21, 57}.

As with sets, sequences may be finite or infinite. Finite sequences often are called tuples. A sequence with k elements is a k-tuple. Thus (7,21, 57) is a 3 -tuple. A 2-tuple is also called a pair.

Sets and sequences may appear as elements of other sets and sequences. For example, the power set of A is the set of all subsets of A. If A is the set {0, 1}, the power set of A is the set { 0, {0}, {1}, {0, 1} }. The set of all pairs whose elements are Os and is is { (0, 0), (0,1), (1, 0), (1,1) }.

If A and B are two sets, the Cartesian product or cross product of A and B, written A x B, is the set of all pairs wherein the first element is a member of A and the second element is a member of B.

EXAM PLE 0.5 ...

If A = {1, 2} and B {x, y, z},

A x B { (1, x), (1, y), (1, z), (2, x), (2, y), (2, z) }.

We can also take the Cartesian product of k sets, Al, A2, ... , Ak, written

Al x A2 x ... x Ak. It is the set consisting of all k-tuples (a,, a2, .. , ak) where

ai C Ai.

EXAMPLE

0.6

...

If A and B are as in Example 0.5,

A x B x A = (1, x, 1), (1, x, 2), (1, y, 1), (1, y, 2), (1, z, 1), (1, z, 2), (2, x, 1), (2, x, 2), (2, y, 1), (2, y, 2), (2, z, 1), (2, z, 2) }.

If we have the Cartesian product of a set with itself, we use the shorthand

k

(24)

0.2 MATHEMATICAL NOTIONS AND TERMINOLOGY

EXAM PLE 0.7 - . . . ..

...

The set Ar' equals Ar x .'V. It consists of all pairs of natural numbers. We also

maywrite it as {(i, j)I i, j > 1}.

FUNCTIONS AND RELATIONS

Functions are central to mathematics. A

function

is an object that sets up an

input-output relationship. A function takes an input and produces an output.

In every function, the same input always produces the same output. If f is a

function whose output value is b when the input value is a, we write

f(a) - b.

A function also is called a mapping, and, if f (a) = b, we say that f maps a to b.

For example, the absolute value function abs takes a number x as input and

returns x if x is positive and -x if x is negative. Thus abs(2) = abs(-2) = 2.

Addition is another example of a function, written add. The input to the addi-tion funcaddi-tion is a pair of numbers, and the output is the sum of those numbers.

The set of possible inputs to the function is called its domain. The outputs

of a function come from a set called its range. The notation for saying that f is

a function with domain D and range R is

f: D-OR.

In the case of the function abs, if we are working with integers, the domain and the range are Z, so we write abs: ZD- Z. In the case of the addition function for integers, the domain is the set of pairs of integers Z x Z and the range is Z,

so we write add: Z x Z -Z. Note that a function may not necessarily use all

the elements of the specified range. The function abs never takes on the value -1 even though -1 E Z. A function that does use all the elements of the range is said to be onto the range.

We may describe a specific function in several ways. One way is with a pro-cedure for computing an output from a specified input. Another way is with a table that lists all possible inputs and gives the output for each input.

EXAMPLE 0.8 ...

Consider the functions: {0,1, 2, 3, 4} {O. 1, 2, 3, 4}.

a f (n)

0 1

1 2

2 3

3 4

4 0

(25)

This function adds 1 to its input and then outputs the result modulo 5. A number modulo m is the remainder after division by m. For example, the minute hand on a clock face counts modulo 60. When we do modular arithmetic we define

Zm {0, 1, 2, . .., m - 1}. With this notation, the aforementioned function

f

has the form f: Z5 - A5.

EXAM PLE 0.9 . . . . . . . . . . . . . . . . . . . ...

Sometimes a two-dimensional table is used if the domain of the function is the

Cartesian product of two sets. Here is another function, g: Z4 x Z4 - Z4. The

entry at the row labeled i and the column labeled j in the table is the value of g(i,j).

g 0 1 2 3

0 0 1 2 3

1 1 2 3 0

2 2 3 0 1

3 3 0 1 2

The function g is the addition function modulo 4.

When the domain of a function

f

is Al x ... x Ak for some sets Al, ... Ak,

the input to

f

is a k-tuple (a,, a2, ... , ak) and we call the ai the arguments to f.

A function with k arguments is called a k-aryfunction, and k is called the arity

of the function. If k is 1,

f

has a single argument and f is called a unary.function.

If k is 2, f is a binaryfunction. Certain familiar binary functions are written in a special infix notation, with the symbol for the function placed between its two arguments, rather than in prefix notation, with the symbol preceding. For example, the addition function add usually is written in infix notation with the + symbol between its two arguments as in a + b instead of in prefix notation

add(a, b).

A predicate or property is a function whose range is {TRUE, FALSE}. For

example, let even be a property that is TRUE if its input is an even number and

FALSE if its input is an odd number. Thus even(4) = TRUE and even(5)

FALSE.

A property whose domain is a set of k-tuples A x x A is called a relation,

a k-ary relation, or a k-ary relation on A. A common case is a 2-ary relation,

called a binary relation. When writing an expression involving a binary rela-tion, we customarily use infix notation. For example, "less than" is a relation usually written with the infix operation symbol <. "Equality," written with the = symbol is another familiar relation. If R is a binary relation, the statement

aRb means that aRb = TRUE. Similarly if R is a k-ary relation, the statement

(26)

0.2 MATHEMATICAL NOTIONS AND TERMINOLOGY 9

EXAM PLE 0.10 ...-...

In a children's game called Scissors-Paper-Stone, the two players simultaneously select a member of the set {SCISSORS, PAPER, STONE} and indicate their selec-tions with hand signals. If the two selecselec-tions are the same, the game starts over. If the selections differ, one player wins, according to the relation beats.

beats SCISSORS PAPER STONE

SCISSORS FALSE TRUE FALSE

PAPER FALSE FALSE TRUE

STONE TRUE FALSE FALSE

From this table we determine that SCISSORS beats PAPER is TRUE and that

PAPER beats SCISSORS is FALSE.

Sometimes describing predicates with sets instead of functions is more

con-venient. The predicate P: D-o {TRUE, FALSE} may be written (D, S), where

S = {a e DI P(a) = TRUE}, or simply S if the domain D is obvious from the context. Hence the relation beats may be written

{(SCISSORS, PAPER), (PAPER, STONE), (STONE, SCISSORS)}.

A special type of binary relation, called an equivalence relation, captures the notion of two objects being equal in some feature. A binary relation R is an equivalence relation if R satisfies three conditions:

1. R is reflexive if for every x, xRx;

2. R is symmetric if for every x and y, xRy implies yRx; and

3. R is transitive if for every x, y, and z, xRy and yRz implies xRz.

EXAM PLE 0.1 1 *---.-..----...

Define an equivalence relation on the natural numbers, written -7. For i, j E X

say that i - 7 j, if i - j is a multiple of 7. This is an equivalence relation because it

satisfies the three conditions. First, it is reflexive, as i - i = 0, which is a multiple of 7. Second, it is symmetric, as i - j is a multiple of 7 if j - i is a multiple of 7.

Third, it is transitive, as whenever i -j is a multiple of 7 and j -k is a multiple

of 7, then i - k = (i - j) + (j - k) is the sum of two multiples of 7 and hence a

(27)

GRAPHS

An undirected graph, or simply a graph, is a set of points with lines connecting some of the points. The points are called nodes or vertices, and the lines are called edges, as shown in the following figure.

(a) (b)

FIGURE 0.12

Examples of graphs

The number of edges at a particular node is the degree of that node. In Figure 0.12(a) all the nodes have degree 2. In Figure 0.12(b) all the nodes have degree 3. No more than one edge is allowed between any two nodes.

In a graph G that contains nodes i and j, the pair (i, j) represents the edge that connects i and j. The order of i and j doesn't matter in an undirected graph, so the pairs (i, j) and (j, i) represent the same edge. Sometimes we describe edges with sets, as in {i, j}, instead of pairs if the order of the nodes is unimportant. If V is the set of nodes of G and E is the set of edges, we say G = (V, E). We can describe a graph with a diagram or more formally by specifying V and E. For example, a formal description of the graph in Figure 0.12(a) is

({1, 2, 3, 4, 5}, {(1, 2), (2, 3), (3, 4), (4, 5), (5, 1)}),

and a formal description of the graph in Figure 0.12(b) is

({1, 2,3,4}, {(1, 2), (1, 3), (1,4), (2, 3), (2,4), (3,4)}).

(28)

0.2 MATHEMATICAL NOTIONS AND TERMINOLOGY

FIGURE 0.13

Cheapest nonstop air fares between various cities

We say that graph G is a subgraph of graph H if the nodes of G are a subset of the nodes of H, and the edges of G are the edges of H on the corresponding

nodes. The following figure shows a graph H and a subgraph G.

Grap]

Subgraph G shown darker

FIGURE 0.14

Graph G (shown darker) is a subgraph of H

A path in a graph is a sequence of nodes connected by edges. A simple path

is a path that doesn't repeat any nodes. A graph is connected if every two nodes have a path between them. A path is a cycle if it starts and ends in the same node.

A simple cycle is one that contains at least three nodes and repeats only the first

and last nodes. A graph is a tree if it is connected and has no simple cycles, as shown in Figure 0. 15. A tree may contain a specially designated node called the

root. The nodes of degree I in a tree, other than the root, are called the leaves

of the tree.

(29)

(a) (b) (c)

FIGURE 0.15

(a) A path in a graph, (b) a cycle in a graph, and (c) a tree

If it has arrows instead of lines, the graph is a directed graph, as shown in the following figure. The number of arrows pointing from a particular node is the

outdegree of that node, and the number of arrows pointing to a particular node

is the indegree.

FIGURE 0.16

A directed graph

In a directed graph we represent an edge from i to j as a pair (i, j). The formal description of a directed graph G is (V, E) where V is the set of nodes and E is the set of edges. The formal description of the graph in Figure 0.16 is

({1,2,3,4,5,6}, {(1,2), (1,5), (2,1), (2,4), (5,4), (5,6), (6,1), (6,3)}).

A path in which all the arrows point in the same direction as its steps is called a

directed path. A directed graph is strongly connected if a directed path connects

(30)

0.2 MATHEMATICAL NOTIONS AND TERMINOLOGY

13

EXAM PLE 0.17 ...

The directed graph shown here represents the relation given in Example 0.10.

FIGURE 0.18

The graph of the relation beats

Directed graphs are a handy way of depicting binary relations. If R is a binary relation whose domain is D x D, a labeled graph G = (D, E) represents R, where E = {(x, y)I xRy}. Figure 0.18 illustrates this representation.

If V is the set of nodes and E is the set of edges, the notation for a graph G consisting of these nodes and edges is G = (V, E).

STRINGS AND LANGUAGES

Strings of characters are fundamental building blocks in computer science. The alphabet over which the strings are defined may vary with the application. For our purposes, we define an alphabet to be any nonempty finite set. The members of the alphabet are the symbols of the alphabet. We generally use capital Greek letters Z and F to designate alphabets and a typewriter font for symbols from an alphabet. The following are a few examples of alphabets.

Y = {0,1};

E 2 = {a,b,c,d,e,f,g.hij,k,lm,n,o,p,q,r,s,t,u,v,w,x,y,z};

F {=,1,x,y,z}.

A string over an alphabet is a finite sequence of symbols from that alphabet,

usually written next to one another and not separated by commas. If E1 = {0,1},

then 01001 is a string over El. If 2 = {a, b, c, . . , z}, then abracadabra is a

string over E2. If w is a string over E, the length of w, written Iwl, is the number

(31)

has length n, we can write w = w1W2 ... w, where each wi e E. The reverse

of w, written wv', is the string obtained by writing w in the opposite order (i.e., Wn -i ... wi). String z is a substring of w if z appears consecutively within w.

For example, cad is a substring of abracadabra

If we have string 2 of length rn and string y of length n, the concatenation

of x and y, written x2, is the string obtained by appending y to the end of x, as

in x1 .xyl ... yE. To concatenate a string with itself many times we use the

superscript notation

k

-- 2X = ok

The lexicographic ordering of strings is the same as the familiar dictionary ordering, except that shorter strings precede longer strings. Thus the lexico-graphic ordering of all strings over the alphabet {0,1} is

(E, O. A, 00, 0, 10, 11,00,. ..).

A language is a set of strings.

BOOLEAN LOGIC

Boolean logic is a mathematical system built around the two values TRUE and FALSE. Though originally conceived of as pure mathematics, this system is now considered to be the foundation of digital electronics and computer design. The

values TRUE and FALSE are called the Boolean values and are often represented

by the values 1 and 0. We use Boolean values in situations with two possibilities, such as a wire that may have a high or a low voltage, a proposition that may be true or false, or a question that may be answered yes or no.

We can manipulate Boolean values with specially designed operations, called

the Boolean operations. The simplest such operation is the negation or NOT

operation, designated with the symbol -. The negation of a Boolean value is the

opposite value. Thus -0 = 1 and -11 = 0. The conjunction, or AND, operation

is designated with the symbol A. The conjunction of two Boolean values is 1 if

both of those values are 1. The disjunction, or OR, operation is designated with

the symbol V. The disjunction of two Boolean values is 1 if either of those values is 1. We summarize this information as follows.

0A0 0 0 V 0 0 -0 1

0 A 1 0 O V I 1 -1 0

1 A 0 0 1 V 0 1

lAl 1 lVi 1

(32)

0.2 MATHEMATICAL NOTIONS AND TERMINOLOGY

representing the truth of the statement "the sun is shining" and Q represents

the truth of the statement "today is Monday", we may write P A Q to represent

the truth value of the statement "the sun is shining and today is Monday" and similarly for P V Q with and replaced by or. The values P and Q are called the

operands of the operation.

Several other Boolean operations occasionally appear. The exclusive or, or

XOR, operation is designated by the

e

symbol and is 1 if either but not both of

its two operands are 1. The equality operation, written with the symbol *-*, is

1 if both of its operands have the same value. Finally, the implication operation

is designated by the symbol - and is 0 if its first operand is 1 and its second

operand is 0; otherwise - is 1. We summarize this information as follows.

o o 0 0-0 1 0 0 1

O1 =1 I1 I 0 =1 01

I130

1

1I 0

0

1-0 0

ieiPI 0 11=1 11 1

We can establish various relationships among these operations. In fact, we

can express all Boolean operations in terms of the AND and NOT operations, as

the following identities show. The two expressions in each row are equivalent. Each row expresses the operation in the left-hand column in terms of operations above it and AND and NOT.

PVQ -({-PA -Q)

P- Q -PVQ

P-Q (P -Q) A (Q -P)

P Q

-(P

Q)

The distributive law for AND and OR comes in handy in manipulating

Boolean expressions. It is similar to the distributive law for addition and multi-plication, which states that a x (b + c) = (a x b) + (a x c). The Boolean version comes in two forms:

* P A (Q V R) equals (P A Q) V (P A R), and its dual

* P V (Q A R) equals (P V Q) A (P V R).

Note that the dual of the distributive law for addition and multiplication does not hold in general.

(33)

SUMMARY OF MATHEMATICAL TERMS Alphabet

Argument Binary relation Boolean operation Boolean value Cartesian product Complement Concatenation Conjunction Connected graph Cycle

Directed graph Disjunction Domain Edge Element Empty set Empty string Equivalence relation Function Graph Intersection k-tuple Language Member Node Pair Path Predicate Property Range Relation Sequence Set

Simple path String Symbol Tree Union Vertex

A finite set of objects called symbols An input to a function

A relation whose domain is a set of pairs An operation on Boolean values

The values TRUE or FALSE, often represented by 1 or 0

An operation on sets forming a set of all tuples of elements from respective sets

An operation on a set, forming the set of all elements not present An operation that sticks strings from one set together with strings

from another set Boolean AND operation

A graph with paths connecting every two nodes A path that starts and ends in the same node

A collection of points and arrows connecting some pairs of points Boolean OR operation

The set of possible inputs to a function A line in a graph

An object in a set The set with no members The string of length zero

A binary relation that is reflexive, symmetric, and transitive An operation that translates inputs into outputs

A collection of points and lines connecting some pairs of points An operation on sets forming the set of common elements A list of k objects

A set of strings An object in a set A point in a graph

A list of two elements, also called a 2-tuple A sequence of nodes in a graph connected by edges

A function whose range is {TRUE, FALSE}

A predicate

The set from which outputs of a function are drawn

A predicate, most typically when the domain is a set of k-tuples A list of objects

A group of objects A path without repetition

A finite list of symbols from an alphabet A member of an alphabet

A connected graph without simple cycles

(34)

0.3 DEFINITIONS, THEOREMS, AND PROOFS

0.3

DEFINITIONS, THEOREMS, AND PROOFS

Theorems and proofs are the heart and soul of mathematics and definitions are its spirit. These three entities are central to every mathematical subject, includ-ing ours.

Definitions describe the objects and notions that we use. A definition may be

simple, as in the definition of set given earlier in this chapter, or complex as in the definition of security in a cryptographic system. Precision is essential to any mathematical definition. When defining some object we must make clear what constitutes that object and what does not.

After we have defined various objects and notions, we usually make

mathe-matical statements about them. Typically a statement expresses that some object

has a certain property. The statement may or may not be true, but like a defini-tion, it must be precise. There must not be any ambiguity about its meaning.

A proof is a convincing logical argument that a statement is true. In mathe-matics an argument must be airtight, that is, convincing in an absolute sense. In everyday life or in the law, the standard of proof is lower. A murder trial demands proof "beyond any reasonable doubt." The weight of evidence may compel the jury to accept the innocence or guilt of the suspect. However, evidence plays no role in a mathematical proof. A mathematician demands proof beyond any doubt.

A theorem is a mathematical statement proved true. Generally we reserve the use of that word for statements of special interest. Occasionally we prove state-ments that are interesting only because they assist in the proof of another, more significant statement. Such statements are called lemmas. Occasionally a theo-rem or its proof may allow us to conclude easily that other, related statements are true. These statements are called corollaries of the theorem.

FINDING PROOFS

The only way to determine the truth or falsity of a mathematical statement is with a mathematical proof. Unfortunately, finding proofs isn't always easy. It can't be reduced to a simple set of rules or processes. During this course, you will be asked to present proofs of various statements. Don't despair at the prospect! Even though no one has a recipe for producing proofs, some helpful general strategies are available.

First, carefully read the statement you want to prove. Do you understand all the notation? Rewrite the statement in your own words. Break it down and consider each part separately.

(35)

only if Q", often written "P iff Q", where both P and Q are mathematical state-ments. This notation is shorthand for a two-part statement. The first part is "P only if Q," which means: If P is true, then Q is true, written P =# Q. The second is "P if Q," which means: If Q is true, then P is true, written P <= Q. The first of these parts is theforward direction of the original statement and the second is the reverse direction. We write "P if and only if Q" as P A> Q. To prove a statement of this form you must prove each of the two directions. Often, one of these directions is easier to prove than the other.

Another type of multipart statement states that two sets A and B are equal. The first part states that A is a subset of B, and the second part states that B is a subset of A. Thus one common way to prove that A = B is to prove that every member of A also is a member of B and that every member of B also is a member of A.

Next, when you want to prove a statement or part thereof, try to get an in-tuitive, "gut" feeling of why it should be true. Experimenting with examples is especially helpful. Thus, if the statement says that all objects of a certain type have a particular property, pick a few objects of that type and observe that they actually do have that property. After doing so, try to find an object that fails to have the property, called a counterexample. If the statement actually is true, you will not be able to find a counterexample. Seeing where you run into difficulty when you attempt to find a counterexample can help you understand why the statement is true.

EXAM PLE 0.19

...-.----Suppose that you want to prove the statement for every graph G, the sum of the

degrees of all the nodes in G is an even number.

First, pick a few graphs and observe this statement in action. Here are two examples.

sum = 2+2+2 sum = 2+3+4+3+2

= 6 = 14

(36)

0.3 DEFINITIONS, THEOREMS, AND PROOFS

Every time an edge is added, the sum increases by 2.

Can you now begin to see why the statement is true and how to prove it?

If you are still stuck trying to prove a statement, try something easier. Attempt to prove a special case of the statement. For example, if you are trying to prove that some property is true for every k > 0, first try to prove it for k = 1. If you succeed, try it for k = 2, and so on until you can understand the more general case. If a special case is hard to prove, try a different special case or perhaps a special case of the special case.

Finally, when you believe that you have found the proof, you must write it up properly. A well-written proof is a sequence of statements, wherein each one follows by simple reasoning from previous statements in the sequence. Carefully writing a proof is important, both to enable a reader to understand it and for you to be sure that it is free from errors.

The following are a few tips for producing a proof.

* Be patient. Finding proofs takes time. If you don't see how to do it right

away, don't worry. Researchers sometimes work for weeks or even years to find a single proof.

* Come back to it. Look over the statement you want to prove, think about it a bit, leave it, and then return a few minutes or hours later. Let the unconscious, intuitive part of your mind have a chance to work.

* Be neat. When you are building your intuition for the statement you are

trying to prove, use simple, clear pictures and/or text. You are trying to develop your insight into the statement, and sloppiness gets in the way of insight. Furthermore, when you are writing a solution for another person to read, neatness will help that person understand it.

* Be concise. Brevity helps you express high-level ideas without getting lost in

details. Good mathematical notation is useful for expressing ideas concisely. But be sure to include enough of your reasoning when writing up a proof so that the reader can easily understand what you are trying to say.

(37)

For practice, let's prove one of DeMorgan's laws.

THEOREM 0.20 ... ...

ForanytwosetsAandB,AUB= AnB.

First, is the meaning of this theorem clear? If you don't understand the mean-ing of the symbols u or n or the overbar, review the discussion on page 4.

To prove this theorem we must show that the two sets A U B and A n B are

equal. Recall that we may prove that two sets are equal by showing that every member of one set also is a member of the other and vice versa. Before looking at the following proof, consider a few examples and then try to prove it yourself.

PROOF This theorem states that two sets, A U B and A n B. are equal. We

prove this assertion by showing that every element of one also is an element of the other and vice versa.

Suppose that x is an element of A U B. Then x is not in A U B from the definition of the complement of a set. Therefore x is not in A and x is not in B, from the definition of the union of two sets. In other words, x is in A and x is in

B. Hence the definition of the intersection of two sets shows that x is in A n B.

For the other direction, suppose that x is in AnB. Then x is in both A and B. Therefore x is not in A and x is not in B, and thus not in the union of these two sets. Hence x is in the complement of the union of these sets; in other words, x is in A U B which completes the proof of the theorem.

...

Let's now prove the statement in Example 0. 19.

THEOREM 0.21 ... ... ...

For every graph G, the sum of the degrees of all the nodes in G is an even number.

PROOF Every edge in G is connected to two nodes. Each edge contributes 1 to the degree of each node to which it is connected. Therefore each edge con-tributes 2 to the sum of the degrees of all the nodes. Hence, if G contains e edges, then the sum of the degrees of all the nodes of G is 2e, which is an even number.

(38)

0.4 TYPES OF PROOF 21

0.4

TYPES OF PROOF

Several types of arguments arise frequently in mathematical proofs. Here, we describe a few that often occur in the theory of computation. Note that a proof may contain more than one type of argument because the proof may contain within it several different subproofs.

PROOF BY CONSTRUCTION

Many theorems state that a particular type of object exists. One way to prove such a theorem is by demonstrating how to construct the object. This technique is a proof by construction.

Let's use a proof by construction to prove the following theorem. We define a graph to be k-regular if every node in the graph has degree k.

THEOREM

0.22

...

...

For each even number n greater than 2, there exists a 3-regular graph with n nodes.

PROOF Let n be an even number greater than 2. Construct graph G =(V, E)

with n nodes as follows. The set of nodes of G is V {0, 1, ... n - 1}, and the

set of edges of G is the set

E {I{i, i + 1} I for 0 < i < n -2} U { n-1, O} } U {f{i, i + n/2} |for 0 < i < n/2 -1}.

Picture the nodes of this graph written consecutively around the circumference of a circle. In that case the edges described in the top line of E go between adjacent pairs around the circle. The edges described in the bottom line of E go between nodes on opposite sides of the circle. This mental picture clearly shows that every node in G has degree 3.

...

PROOF BY CONTRADICTION

Figure

FIGURE 0.1Venn diagram for the set of English words starting with "t"
FIGURE 0 1Venn diagram for the set of English words starting with t . View in document p.21
FIGURE Overlapping
FIGURE Overlapping . View in document p.22
FIGURE Graph G (shown
FIGURE Graph G shown . View in document p.28
FIGURE (a) A path
FIGURE a A path . View in document p.29
FIGURE State diagram
FIGURE State diagram . View in document p.48
FIGURE Finite automaton
FIGURE Finite automaton . View in document p.54
FIGURE 1.29The computation of N1 on input 010110
FIGURE 1 29The computation of N1 on input 010110. View in document p.65
FIGURE Deterministic
FIGURE Deterministic . View in document p.65
FIGURE 1.34The NFA N3
FIGURE 1 34The NFA N3. View in document p.68
FIGURE Construction
FIGURE Construction . View in document p.77
FIGURE Construction
FIGURE Construction . View in document p.78
FIGURE (converting
FIGURE converting . View in document p.91
FIGURE 1.69Converting a three-state
FIGURE 1 69Converting a three state . View in document p.92
FIGURE State diagram
FIGURE State diagram . View in document p.129
FIGURE State diagram
FIGURE State diagram . View in document p.130
FIGURE State diagram
FIGURE State diagram . View in document p.130
FIGURE P representing
FIGURE P representing . View in document p.132
FIGURE 2.23Implementing the shorthand (r, xyz) C 6(q, a, s)
FIGURE 2 23Implementing the shorthand r xyz C 6 q a s . View in document p.133
FIGURE 2.24State diagram of
FIGURE 2 24State diagram of . View in document p.134
FIGURE 2.26State diagram of
FIGURE 2 26State diagram of . View in document p.134
FIGURE PDA computation corresponding
FIGURE PDA computation corresponding . View in document p.136
FIGURE PDA computation corresponding
FIGURE PDA computation corresponding . View in document p.136
FIGURE Relationship
FIGURE Relationship . View in document p.139
FIGURE 2.35Surgery on parse
FIGURE 2 35Surgery on parse . View in document p.140
FIGURE 3.2Snapshots of Turing
FIGURE 3 2Snapshots of Turing . View in document p.154
FIGURE State diagram
FIGURE State diagram . View in document p.159
FIGURE 3.10State diagram for
FIGURE 3 10State diagram for . View in document p.160
FIGURE Representing
FIGURE Representing . View in document p.164
FIGURE Deterministic
FIGURE Deterministic . View in document p.166
FIGURE 3.20Schematic of an enumerator
FIGURE 3 20Schematic of an enumerator. View in document p.167

References

Updating...