Probability and Information Theory

Section 3.1. Introduction

Section 3.2. Basic Concept of Probability

Section 3.3. Properties

Section 3.4. Basic Calculation

Section 3.5. Random Variables and their Probability Distributions

Section 3.6. Birthday Paradox

Section 3.7. Information Theory

Section 3.8. Redundancy in Natural Languages

Section 3.9. Chapter Summary

• Table of Contents

Modern Cryptography: Theory and Practice By Wenbo Mao Hewlett-Packard Company

Publisher: Prentice Hall PTR Pub Date: July 25, 2003

ISBN: 0-13-066943-1 Pages: 648

Many cryptographic schemes and protocols, especially those based on public-keycryptography, have basic or so-called "textbook crypto" versions, as these versionsare usually the subjects for many textbooks on cryptography. This book takes adifferent approach to introducing

cryptography: it pays much more attention tofit-for-application aspects of cryptography. It explains why "textbook crypto" isonly good in an ideal world where data are random and bad guys behave nicely.It reveals the general unfitness of "textbook crypto" for the real world by demonstratingnumerous attacks on such schemes, protocols and systems under variousreal- world application scenarios. This book chooses to introduce a set of practicalcryptographic schemes, protocols and systems, many of them standards or de factoones, studies them closely, explains their working principles, discusses their practicalusages, and examines their strong (i.e., fit-for-application) security properties, oftenwith security evidence formally established. The book also includes self-containedtheoretical background material that is the foundation for modern cryptography.

3.1 Introduction

Probability and information theory are essential tools for the development of modern cryptographic techniques.

Probability is a basic tool for the analysis of security. We often need to estimate how probable it is that an insecure event may occur under certain conditions. For example, considering Protocol "Coin Flipping Over Telephone" in Chapter 1, we need to estimate the probability for Alice to succeed in finding a collision for a given one-way function f (which should desirably be bounded by a very small quantity), and that for Bob to succeed in finding the parity of x when given f(x) (which should desirably be very close to ).

Information theory is closely related to probability. An important aspect of security for an encryption algorithm can be referred to as "uncertainty of ciphers:" an encryption algorithm should desirably output ciphertext which has a random distribution in the entire space of its ciphertext message space. Shannon quantifies the uncertainty of information by a notion which he names entropy. Historically, the desire for achieving a high entropy in ciphers comes from the need for thwarting a cryptanalysis technique which makes use of the fact that natural languages contain redundancy, which is related to frequent appearance of some known patterns in natural languages.

Recently, the need for modern cryptographic systems, in particular public-key cryptosystems, to have probabilistic behavior has reached a rather stringent degree: semantic security. This can be described as the following property: if Alice encrypts either 0 or 1 with equal probability under a semantically secure encryption algorithm, sends the resultant ciphertext c to Bob and asks him to answer which is the case, then Bob, without the correct decryption key, should not have an algorithmic strategy to enable him to discern between the two cases with any "advantage" better than a random guessing. We notice that many "textbook" versions of encryption algorithms do not have this desirable property.

3.1.1 Chapter Outline

The basic notions of probability which are sufficient for our use in this book will be introduced in §3.2—§3.6. Information theory will be introduced in §3.7—§3.8.

• Table of Contents

Modern Cryptography: Theory and Practice By Wenbo Mao Hewlett-Packard Company

Publisher: Prentice Hall PTR Pub Date: July 25, 2003

ISBN: 0-13-066943-1 Pages: 648

Many cryptographic schemes and protocols, especially those based on public-keycryptography, have basic or so-called "textbook crypto" versions, as these versionsare usually the subjects for many textbooks on cryptography. This book takes adifferent approach to introducing

cryptography: it pays much more attention tofit-for-application aspects of cryptography. It explains why "textbook crypto" isonly good in an ideal world where data are random and bad guys behave nicely.It reveals the general unfitness of "textbook crypto" for the real world by demonstratingnumerous attacks on such schemes, protocols and systems under variousreal- world application scenarios. This book chooses to introduce a set of practicalcryptographic schemes, protocols and systems, many of them standards or de factoones, studies them closely, explains their working principles, discusses their practicalusages, and examines their strong (i.e., fit-for-application) security properties, oftenwith security evidence formally established. The book also includes self-containedtheoretical background material that is the foundation for modern cryptography.

3.2 Basic Concept of Probability

Let be an arbitrary, but fixed, set of points called probability space (or sample space). Any element x is called a sample point (also called outcome, simple event or

indecomposable event; we shall just use point for short). An event (also called compound event or decomposable event) is a subset of and is usually denoted by a capital letter (e.g., E). An experiment or observation is an action of yielding (taking) a point from . An

occurrence of an event E is when an experiment yields x E for some point x .

Example 3.1.

Consider an experiment of drawing one playing card from a fair deck (here "fair" means drawing a card at random). Here are some examples of probability spaces, points, events and

occurrences of events.

1: The space consists of 52 points, 1 for each card in the deck. Let event E1 be "aces" (i.e., E 1 = {A , A , A , A }). It occurs if the card drawn is an ace of any suit.

1.

2 = {red, black}. Let event E2 = {red}. It occurs if the card drawn is of red color.

2.

3: This space consists of 13 points, namely, 2, 3, 4, …, 10, J, Q, K, A. Let event E3 be "numbers." It occurs if the card drawn is 2, or 3, or …, or 10.

3.

Definition 3.1: Classical Definition of ProbabilitySuppose that an experiment can yield one