• No results found

Polarization codes and the rate of polarization

N/A
N/A
Protected

Academic year: 2021

Share "Polarization codes and the rate of polarization"

Copied!
96
0
0

Loading.... (view fulltext now)

Full text

(1)

Polarization codes and the rate of polarization

Erdal Arıkan, Emre Telatar

Bilkent U., EPFL

Sept 10, 2008

(2)

Channel Polarization

Given a binary input DMC W , i.i.d. uniformly distributed inputs (X1, . . . , Xn) ∈ {0, 1}n, in one-to-one correspondence with binary ‘data’

(U1, . . . , Un) ∈ {0, 1}n. Observe that Ui are i.i.d., uniform on {0, 1}.

W Yn Xn∈ Fn2

(3)

Channel Polarization

Given a binary input DMC W , i.i.d. uniformly distributed inputs (X1, . . . , Xn) ∈ {0, 1}n, in one-to-one correspondence with binary ‘data’

(U1, . . . , Un) ∈ {0, 1}n. Observe that Ui are i.i.d., uniform on {0, 1}.

W Yn Xn∈ Fn2

(4)

Channel Polarization

Given a binary input DMC W , i.i.d. uniformly distributed inputs (X1, . . . , Xn) ∈ {0, 1}n, in one-to-one correspondence with binary ‘data’

(U1, . . . , Un) ∈ {0, 1}n. Observe that Ui are i.i.d., uniform on {0, 1}.

W Gn: Fn2↔ Fn2

Yn Xn∈ Fn2

Un∈ Fn2

(5)

Channel Polarization

Given a binary input DMC W , i.i.d. uniformly distributed inputs (X1, . . . , Xn) ∈ {0, 1}n, in one-to-one correspondence with binary ‘data’

(U1, . . . , Un) ∈ {0, 1}n. Observe that Ui are i.i.d., uniform on {0, 1}.

W Gn: Fn2↔ Fn2

Yn Xn∈ Fn2

Un∈ Fn2

(6)

Channel Polarization

I (Un; Yn) =I (Xn; Yn)

=X

i

I (Xi; Yi)

=X

i

I (W )

I (Un; Yn) =X

i

I (Ui; Yn|Ui −1)

=X

i

I (Ui; YnUi −1)

Notation: I (P)denotes the mutual information between the input and output of a channel P when input is uniformly distributed.

(7)

Channel Polarization

I (Un; Yn) = I (Xn; Yn)

=X

i

I (Xi; Yi)

=X

i

I (W )

I (Un; Yn) =X

i

I (Ui; Yn|Ui −1)

=X

i

I (Ui; YnUi −1)

Notation: I (P)denotes the mutual information between the input and output of a channel P when input is uniformly distributed.

(8)

Channel Polarization

I (Un; Yn) = I (Xn; Yn)

=X

i

I (Xi; Yi)

=X

i

I (W )

I (Un; Yn) =X

i

I (Ui; Yn|Ui −1)

=X

i

I (Ui; YnUi −1)

Notation: I (P)denotes the mutual information between the input and output of a channel P when input is uniformly distributed.

(9)

Channel Polarization

I (Un; Yn) = I (Xn; Yn)

=X

i

I (Xi; Yi)

=X

i

I (W )

I (Un; Yn) =X

i

I (Ui; Yn|Ui −1)

=X

i

I (Ui; YnUi −1)

Notation: I (P)denotes the mutual information between the input and output of a channel P when input is uniformly distributed.

(10)

Channel Polarization

I (Un; Yn) = I (Xn; Yn)

=X

i

I (Xi; Yi)

=X

i

I (W )

I (Un; Yn) =X

i

I (Ui; Yn|Ui −1)

=X

i

I (Ui; YnUi −1)

#{i : I (Xi; Yi) ≤ ζ}/n

ζ

0 I (W ) 1

0 1

Notation: I (P)denotes the mutual information between the input and output of a channel P when input is uniformly distributed.

(11)

Channel Polarization

I (Un; Yn) = I (Xn; Yn)

=X

i

I (Xi; Yi)

=X

i

I (W )

I (Un; Yn) =X

i

I (Ui; Yn|Ui −1)

=X

i

I (Ui; YnUi −1)

#{i : I (Xi; Yi) ≤ ζ}/n

ζ

0 I (W ) 1

0 1

Notation: I (P)denotes the mutual information between the input and output of a channel P when input is uniformly distributed.

(12)

Channel Polarization

I (Un; Yn) = I (Xn; Yn)

=X

i

I (Xi; Yi)

=X

i

I (W )

I (Un; Yn) =X

i

I (Ui; Yn|Ui −1)

=X

i

I (Ui; YnUi −1)

#{i : I (Xi; Yi) ≤ ζ}/n

ζ

0 I (W ) 1

0 1

Notation: I (P)denotes the mutual information between the input and output of a channel P when input is uniformly distributed.

(13)

Channel Polarization

I (Un; Yn) = I (Xn; Yn)

=X

i

I (Xi; Yi)

=X

i

I (W )

I (Un; Yn) =X

i

I (Ui; Yn|Ui −1)

=X

i

I (Ui; YnUi −1)

#{i : I (Ui; YnUi −1) ≤ ζ}/n

ζ

0 1

0 1

Notation: I (P)denotes the mutual information between the input and output of a channel P when input is uniformly distributed.

(14)

Channel Polarization

I (Un; Yn) = I (Xn; Yn)

=X

i

I (Xi; Yi)

=X

i

I (W )

I (Un; Yn) =X

i

I (Ui; Yn|Ui −1)

=X

i

I (Ui; YnUi −1)

#{i : I (Ui; YnUi −1) ≤ ζ}/n

ζ

0 1

0 1

n = 1

Notation: I (P)denotes the mutual information between the input and output of a channel P when input is uniformly distributed.

(15)

Channel Polarization

I (Un; Yn) = I (Xn; Yn)

=X

i

I (Xi; Yi)

=X

i

I (W )

I (Un; Yn) =X

i

I (Ui; Yn|Ui −1)

=X

i

I (Ui; YnUi −1)

#{i : I (Ui; YnUi −1) ≤ ζ}/n

ζ

0 1

0 1

n = 2

Notation: I (P)denotes the mutual information between the input and output of a channel P when input is uniformly distributed.

(16)

Channel Polarization

I (Un; Yn) = I (Xn; Yn)

=X

i

I (Xi; Yi)

=X

i

I (W )

I (Un; Yn) =X

i

I (Ui; Yn|Ui −1)

=X

i

I (Ui; YnUi −1)

#{i : I (Ui; YnUi −1) ≤ ζ}/n

ζ

0 1

0 1

n = 24

Notation: I (P)denotes the mutual information between the input and output of a channel P when input is uniformly distributed.

(17)

Channel Polarization

I (Un; Yn) = I (Xn; Yn)

=X

i

I (Xi; Yi)

=X

i

I (W )

I (Un; Yn) =X

i

I (Ui; Yn|Ui −1)

=X

i

I (Ui; YnUi −1)

#{i : I (Ui; YnUi −1) ≤ ζ}/n

ζ

0 1

0 1

n = 28

Notation: I (P)denotes the mutual information between the input and output of a channel P when input is uniformly distributed.

(18)

Channel Polarization

I (Un; Yn) = I (Xn; Yn)

=X

i

I (Xi; Yi)

=X

i

I (W )

I (Un; Yn) =X

i

I (Ui; Yn|Ui −1)

=X

i

I (Ui; YnUi −1)

#{i : I (Ui; YnUi −1) ≤ ζ}/n

ζ

0 1

0 1

n = 212

Notation: I (P)denotes the mutual information between the input and output of a channel P when input is uniformly distributed.

(19)

Channel Polarization

I (Un; Yn) = I (Xn; Yn)

=X

i

I (Xi; Yi)

=X

i

I (W )

I (Un; Yn) =X

i

I (Ui; Yn|Ui −1)

=X

i

I (Ui; YnUi −1)

#{i : I (Ui; YnUi −1) ≤ ζ}/n

ζ

0 1

0 1

n = 220

Notation: I (P)denotes the mutual information between the input and output of a channel P when input is uniformly distributed.

(20)

Channel Polarization

I (Un; Yn) = I (Xn; Yn)

=X

i

I (Xi; Yi)

=X

i

I (W )

I (Un; Yn) =X

i

I (Ui; Yn|Ui −1)

=X

i

I (Ui; YnUi −1)

#{i : I (Ui; YnUi −1) ≤ ζ}/n

ζ

0 1

0 1

Notation: I (P)denotes the mutual information between the input and output of a channel P when input is uniformly distributed.

(21)

Channel Polarization

We saychannel polarization takes place if it is the case that almost all of the numbers I (Ui; YnUi −1) are near the extremal values,i.e.,

if 1

n#i : I (Ui; YnUi −1) ≈ 1 → I (W )

and 1

n#i : I (Ui; YnUi −1) ≈ 0 → 1 − I (W ).

(22)

Channel Polarization

We saychannel polarization takes place if it is the case that almost all of the numbers I (Ui; YnUi −1) are near the extremal values, i.e.,

if 1

n#i : I (Ui; YnUi −1) ≈ 1 → I (W )

and 1

n#i : I (Ui; YnUi −1) ≈ 0 → 1 − I (W ).

(23)

Channel Polarization

If polarization takes place and we wish to communicate at rate R:

Pick n, and k = nR good indices i such that I (Ui; YnUi −1) is high,

let the transmitter set Ui to beuncodedbinary data for good indices, and set Ui to random but publicly known values for the rest,

let the receiver decode the Ui successively: U1 from Yn; Ui from Yni −1.

One would expect this scheme to do do well as long as R < I (W ).

(24)

Channel Polarization

If polarization takes place and we wish to communicate at rate R:

Pick n, and k = nR good indices i such that I (Ui; YnUi −1) is high,

let the transmitter set Ui to beuncodedbinary data for good indices, and set Ui to random but publicly known values for the rest,

let the receiver decode the Ui successively: U1 from Yn; Ui from Yni −1.

One would expect this scheme to do do well as long as R < I (W ).

(25)

Channel Polarization

If polarization takes place and we wish to communicate at rate R:

Pick n, and k = nR good indices i such that I (Ui; YnUi −1) is high,

let the transmitter set Ui to beuncodedbinary data for good indices, and set Ui to random but publicly known values for the rest,

let the receiver decode the Ui successively: U1 from Yn; Ui from Yni −1.

One would expect this scheme to do do well as long as R < I (W ).

(26)

Channel Polarization

If polarization takes place and we wish to communicate at rate R:

Pick n, and k = nR good indices i such that I (Ui; YnUi −1) is high,

let the transmitter set Ui to beuncodedbinary data for good indices, and set Ui to random but publicly known values for the rest,

let the receiver decode the Ui successively: U1 from Yn; Ui from Yni −1.

One would expect this scheme to do do well as long as R < I (W ).

(27)

Channel Polarization

If polarization takes place and we wish to communicate at rate R:

Pick n, and k = nR good indices i such that I (Ui; YnUi −1) is high,

let the transmitter set Ui to beuncodedbinary data for good indices, and set Ui to random but publicly known values for the rest,

let the receiver decode the Ui successively: U1 from Yn; Ui from Yni −1.

One would expect this scheme to do do well as long as R < I (W ).

(28)

Polarization – HowTo

Given two copies of a binary input channel P : F2→ Y

P X1 P

X2

Y1

Y2

consider the transformation above to generate two channels P : F2 → Y2 and P+ : F2 → Y2× F2 with

P(y1y2|u1) =X

u2

1

2P(y1|u1+ u2)P(y2|u2) P+(y1y2u1|u2) = 21P(y1|u1+ u2)P(y2|u2).

(29)

Polarization – HowTo

Given two copies of a binary input channel P : F2→ Y

P U1 P

U2

+ Y1

Y2

consider the transformation above to generate two channels P : F2 → Y2 and P+ : F2 → Y2× F2 with

P(y1y2|u1) =X

u2

1

2P(y1|u1+ u2)P(y2|u2) P+(y1y2u1|u2) = 21P(y1|u1+ u2)P(y2|u2).

(30)

Polarization – HowTo

Given two copies of a binary input channel P : F2→ Y

P U1 P

U2

+ Y1

Y2

consider the transformation above to generate two channels P : F2 → Y2 and P+ : F2 → Y2× F2 with

P(y1y2|u1) =X

u2

1

2P(y1|u1+ u2)P(y2|u2) P+(y1y2u1|u2) = 21P(y1|u1+ u2)P(y2|u2).

(31)

Polarization – HowTo

Given two copies of a binary input channel P : F2→ Y

P U1 P

U2

+ Y1

Y2

consider the transformation above to generate two channels P : F2 → Y2 and P+ : F2 → Y2× F2 with

P(y1y2|u1) =X

u2

1

2P(y1|u1+ u2)P(y2|u2) P+(y1y2u1|u2) = 21P(y1|u1+ u2)P(y2|u2).

(32)

Polarization – HowTo

Given two copies of a binary input channel P : F2→ Y

P U1 P

U2

+ Y1

Y2

consider the transformation above to generate two channels P : F2 → Y2 and P+ : F2 → Y2× F2 with

P(y1y2|u1) =X

u2

1

2P(y1|u1+ u2)P(y2|u2) P+(y1y2u1|u2) = 21P(y1|u1+ u2)P(y2|u2).

(33)

Polarization – HowTo

Given two copies of a binary input channel P : F2→ Y

P U1 P

U2

+ Y1

Y2

consider the transformation above to generate two channels P : F2 → Y2 and P+ : F2 → Y2× F2 with

P(y1y2|u1) =X

u2

1

2P(y1|u1+ u2)P(y2|u2) P+(y1y2u1|u2) = 21P(y1|u1+ u2)P(y2|u2).

(34)

Polarization – HowTo

Observe that

U1

U2



=1 1 0 1

 X1

X2

 . With independent, uniform U1, U2,

I (P) = I (U1; Y1Y2), I (P+) = I (U2; Y1Y2U1).

Thus,

I (P) + I (P+) = I (U1U2; Y1Y2)

= I (X1; Y1) + I (X2; Y2)

= 2I (P), and I (P) ≤ I (P) ≤ I (P+).

(35)

Polarization – HowTo

Observe that

U1

U2



=1 1 0 1

 X1

X2

 . With independent, uniform U1, U2,

I (P) = I (U1; Y1Y2), I (P+) = I (U2; Y1Y2U1).

Thus,

I (P) + I (P+) = I (U1U2; Y1Y2)

= I (X1; Y1) + I (X2; Y2)

= 2I (P), and I (P) ≤ I (P) ≤ I (P+).

(36)

Polarization – HowTo

Observe that

U1

U2



=1 1 0 1

 X1

X2

 . With independent, uniform U1, U2,

I (P) = I (U1; Y1Y2), I (P+) = I (U2; Y1Y2U1).

Thus,

I (P) + I (P+) = I (U1U2; Y1Y2)

= I (X1; Y1) + I (X2; Y2)

= 2I (P), and I (P) ≤ I (P) ≤ I (P+).

(37)

Polarization – HowTo

Observe that

U1

U2



=1 1 0 1

 X1

X2

 . With independent, uniform U1, U2,

I (P) = I (U1; Y1Y2), I (P+) = I (U2; Y1Y2U1).

Thus,

I (P) + I (P+) = I (U1U2; Y1Y2)

= I (X1; Y1) + I (X2; Y2)

= 2I (P), and I (P) ≤ I (P) ≤ I (P+).

(38)

Polarization – HowTo

What we can do once, we can do many times: Given W , Duplicate W and obtain

W and W+.

Duplicate W (W+), and obtain W−− and W−+ (W+− and W++).

Duplicate W−− (W−+, W+−, W++) and obtain W−−− and W−−+

(W−+−, W−++,

W+−−, W+−+, W++−, W+++).

. . .

(39)

Polarization – HowTo

What we can do once, we can do many times: Given W , Duplicate W and obtain

W and W+.

Duplicate W (W+), and obtain W−− and W−+ (W+− and W++).

Duplicate W−− (W−+, W+−, W++) and obtain W−−− and W−−+

(W−+−, W−++,

W+−−, W+−+, W++−, W+++).

. . .

(40)

Polarization – HowTo

What we can do once, we can do many times: Given W , Duplicate W and obtain

W and W+.

Duplicate W (W+), and obtain W−− and W−+ (W+− and W++).

Duplicate W−− (W−+, W+−, W++) and obtain W−−− and W−−+

(W−+−, W−++,

W+−−, W+−+, W++−, W+++).

. . .

(41)

Polarization – HowTo

What we can do once, we can do many times: Given W , Duplicate W and obtain

W and W+.

Duplicate W (W+), and obtain W−− and W−+ (W+− and W++).

Duplicate W−− (W−+, W+−, W++) and obtain W−−− and W−−+

(W−+−, W−++,

W+−−, W+−+, W++−, W+++).

. . .

(42)

Polarization – HowTo

What we can do once, we can do many times: Given W , Duplicate W and obtain

W and W+.

Duplicate W (W+), and obtain W−− and W−+ (W+− and W++).

Duplicate W−− (W−+, W+−, W++) and obtain W−−− and W−−+

(W−+−, W−++,

W+−−, W+−+, W++−, W+++).

. . .

(43)

Polarization – HowTo

What we can do once, we can do many times: Given W , Duplicate W and obtain

W and W+.

Duplicate W (W+), and obtain W−− and W−+ (W+− and W++).

Duplicate W−− (W−+, W+−, W++) and obtain W−−− and W−−+

(W−+−, W−++,

W+−−, W+−+, W++−, W+++).

. . .

(44)

Polarization – HowTo

We now have a one-to-one transformation Gn between U1, . . . , Un and X1, . . . , Xn for n = 2`.

Furthermore, the n = 2` quantities

I (Wb1...,b`), bj ∈ {+, −}

are exactly the n quantities

I (Ui; YnUi −1), i = 1, . . . , n whose empirical distribution we seek. This suggests:

Set B1, B2, . . . , to be i.i.d., {+, −} valued, uniformly distributed random variables,

Define the random variable I` = I (WB1,...,B`). Study the distribution of I`.

(45)

Polarization – HowTo

We now have a one-to-one transformation Gn between U1, . . . , Un and X1, . . . , Xn for n = 2`.

Furthermore, the n = 2` quantities

I (Wb1...,b`), bj ∈ {+, −}

are exactly the n quantities

I (Ui; YnUi −1), i = 1, . . . , n whose empirical distribution we seek. This suggests:

Set B1, B2, . . . , to be i.i.d., {+, −} valued, uniformly distributed random variables,

Define the random variable I` = I (WB1,...,B`). Study the distribution of I`.

(46)

Polarization – HowTo

We now have a one-to-one transformation Gn between U1, . . . , Un and X1, . . . , Xn for n = 2`.

Furthermore, the n = 2` quantities

I (Wb1...,b`), bj ∈ {+, −}

are exactly the n quantities

I (Ui; YnUi −1), i = 1, . . . , n whose empirical distribution we seek. This suggests:

Set B1, B2, . . . , to be i.i.d., {+, −} valued, uniformly distributed random variables,

Define the random variable I` = I (WB1,...,B`). Study the distribution of I`.

(47)

Polarization – HowTo

We now have a one-to-one transformation Gn between U1, . . . , Un and X1, . . . , Xn for n = 2`.

Furthermore, the n = 2` quantities

I (Wb1...,b`), bj ∈ {+, −}

are exactly the n quantities

I (Ui; YnUi −1), i = 1, . . . , n whose empirical distribution we seek. This suggests:

Set B1, B2, . . . , to be i.i.d., {+, −} valued, uniformly distributed random variables,

Define the random variable I` = I (WB1,...,B`). Study the distribution of I`.

(48)

Polarization – HowTo

We now have a one-to-one transformation Gn between U1, . . . , Un and X1, . . . , Xn for n = 2`.

Furthermore, the n = 2` quantities

I (Wb1...,b`), bj ∈ {+, −}

are exactly the n quantities

I (Ui; YnUi −1), i = 1, . . . , n whose empirical distribution we seek. This suggests:

Set B1, B2, . . . , to be i.i.d., {+, −} valued, uniformly distributed random variables,

Define the random variable I` = I (WB1,...,B`). Study the distribution of I`.

(49)

Polarization – HowTo

Properties of the process I`: I0= I (W ) is a constant.

I` is lies in [0, 1], so is bounded.

Conditional on B1, . . . , B`, and with P := WB1,...,B`, I`+1 can take only two values I (P) and I (P+),

so,

E [I`+1|B1, . . . , B`] = 12[I (P) + I (P+)] = I (P) = I`. and we see that {I`} is a martingale.

(50)

Polarization – HowTo

Properties of the process I`: I0= I (W ) is a constant.

I` is lies in [0, 1], so is bounded.

Conditional on B1, . . . , B`, and with P := WB1,...,B`, I`+1 can take only two values I (P) and I (P+),

so,

E [I`+1|B1, . . . , B`] = 12[I (P) + I (P+)] = I (P) = I`. and we see that {I`} is a martingale.

(51)

Polarization – HowTo

Properties of the process I`: I0= I (W ) is a constant.

I` is lies in [0, 1], so is bounded.

Conditional on B1, . . . , B`, and with P := WB1,...,B`, I`+1 can take only two values I (P) and I (P+),

so,

E [I`+1|B1, . . . , B`] = 12[I (P) + I (P+)] = I (P) = I`. and we see that {I`} is a martingale.

(52)

Polarization – HowTo

Properties of the process I`: I0= I (W ) is a constant.

I` is lies in [0, 1], so is bounded.

Conditional on B1, . . . , B`, and with P := WB1,...,B`, I`+1 can take only two values I (P) and I (P+),

so,

E [I`+1|B1, . . . , B`] = 12[I (P) + I (P+)] = I (P) = I`. and we see that {I`} is a martingale.

(53)

Polarization – HowTo

Bounded martingales converge almost surely. Thus

I:= lim`→∞I` exists with probability one. So we know that the empirical distribution we seek exists — it is the

distribution of I — and E [I] = I0= I (W ).

Almost sure convergence of {I`} implies |I`+1− I`| → 0 a.s.

|I`+1− I`| = 12[I (P+) − I (P)].

But I (P+) − I (P) <  implies I (P) 6∈ (δ, 1 − δ). (The BEC and BSC turn out to be

extremal among all binary input channels).

Thus I is {0, 1} valued!

(54)

Polarization – HowTo

Bounded martingales converge almost surely. Thus

I:= lim`→∞I` exists with probability one. So we know that the empirical distribution we seek exists — it is the

distribution of I — and E [I] = I0= I (W ).

Almost sure convergence of {I`} implies |I`+1− I`| → 0 a.s.

|I`+1− I`| = 12[I (P+) − I (P)].

But I (P+) − I (P) <  implies I (P) 6∈ (δ, 1 − δ). (The BEC and BSC turn out to be

extremal among all binary input channels).

Thus I is {0, 1} valued!

(55)

Polarization – HowTo

Bounded martingales converge almost surely. Thus

I:= lim`→∞I` exists with probability one. So we know that the empirical distribution we seek exists — it is the

distribution of I — and E [I] = I0= I (W ).

Almost sure convergence of {I`} implies |I`+1− I`| → 0 a.s.

|I`+1− I`| = 12[I (P+) − I (P)].

But I (P+) − I (P) <  implies I (P) 6∈ (δ, 1 − δ). (The BEC and BSC turn out to be

extremal among all binary input channels).

Thus I is {0, 1} valued!

(56)

Polarization – HowTo

Bounded martingales converge almost surely. Thus

I:= lim`→∞I` exists with probability one. So we know that the empirical distribution we seek exists — it is the

distribution of I — and E [I] = I0= I (W ).

Almost sure convergence of {I`} implies |I`+1− I`| → 0 a.s.

|I`+1− I`| = 12[I (P+) − I (P)].

But I (P+) − I (P) <  implies I (P) 6∈ (δ, 1 − δ). (The BEC and BSC turn out to be

extremal among all binary input channels).

Thus I is {0, 1} valued!

I (P+) − I (P)

I (P)

0 1

1 2

(57)

Polarization – HowTo

Bounded martingales converge almost surely. Thus

I:= lim`→∞I` exists with probability one. So we know that the empirical distribution we seek exists — it is the

distribution of I — and E [I] = I0= I (W ).

Almost sure convergence of {I`} implies |I`+1− I`| → 0 a.s.

|I`+1− I`| = 12[I (P+) − I (P)].

But I (P+) − I (P) <  implies I (P) 6∈ (δ, 1 − δ). (The BEC and BSC turn out to be

extremal among all binary input channels).

Thus I is {0, 1} valued!

I (P+) − I (P)

I (P)

0 1

1 2

(58)

Polarization Rate

We have seen that polarization takes place. How fast?

Idea 0: We have seen in the I (P+) − I (P) vs I (P) graph that BSC is least polarizing. So we should be able to obtain a lower bound on the speed of polarization by studying BSC.

However, if P is a BSC, P+ and P are not. Thus the estimate based on this idea is turns out to be too pessimistic.

Instead, we introduce an auxiliary quantity Z (P) =X

y

pP(y |0)P(y |1)

as a companion to I (P).

(59)

Polarization Rate

We have seen that polarization takes place. How fast?

Idea 0: We have seen in the I (P+) − I (P) vs I (P) graph that BSC is least polarizing. So we should be able to obtain a lower bound on the speed of polarization by studying BSC.

However, if P is a BSC, P+ and P are not. Thus the estimate based on this idea is turns out to be too pessimistic.

Instead, we introduce an auxiliary quantity Z (P) =X

y

pP(y |0)P(y |1)

as a companion to I (P).

(60)

Polarization Rate

We have seen that polarization takes place. How fast?

Idea 0: We have seen in the I (P+) − I (P) vs I (P) graph that BSC is least polarizing. So we should be able to obtain a lower bound on the speed of polarization by studying BSC.

However, if P is a BSC, P+ and P are not. Thus the estimate based on this idea is turns out to be too pessimistic.

Instead, we introduce an auxiliary quantity Z (P) =X

y

pP(y |0)P(y |1)

as a companion to I (P).

(61)

Polarization Rate

We have seen that polarization takes place. How fast?

Idea 0: We have seen in the I (P+) − I (P) vs I (P) graph that BSC is least polarizing. So we should be able to obtain a lower bound on the speed of polarization by studying BSC.

However, if P is a BSC, P+ and P are not. Thus the estimate based on this idea is turns out to be too pessimistic.

Instead, we introduce an auxiliary quantity Z (P) =X

y

pP(y |0)P(y |1)

as a companion to I (P).

(62)

Polarization Rate

Properties of Z (P):

Z (P) ∈ [0, 1].

Z (P) ≈ 0 iff I (P) ≈ 1.

Z (P) ≈ 1 iff I (P) ≈ 0.

Z (P+) = Z (P)2. Z (P) ≤ 2Z (P).

Note that Z (P) is the Bhattacharyya upper bound on probability of error for uncoded transmission over P. Thus, we can choose the good indiceson the basis of Z (P); the sum of the Z ’s of the chosen channels will upper bound the block error probability. This suggests studying the polarization rate of Z .

(63)

Polarization Rate

Properties of Z (P):

Z (P) ∈ [0, 1].

Z (P) ≈ 0 iff I (P) ≈ 1.

Z (P) ≈ 1 iff I (P) ≈ 0.

Z (P+) = Z (P)2. Z (P) ≤ 2Z (P).

Z (P)

I (P) 1

1 0

0

Note that Z (P) is the Bhattacharyya upper bound on probability of error for uncoded transmission over P. Thus, we can choose the good indiceson the basis of Z (P); the sum of the Z ’s of the chosen channels will upper bound the block error probability. This suggests studying the polarization rate of Z .

(64)

Polarization Rate

Properties of Z (P):

Z (P) ∈ [0, 1].

Z (P) ≈ 0 iff I (P) ≈ 1.

Z (P) ≈ 1 iff I (P) ≈ 0.

Z (P+) = Z (P)2. Z (P) ≤ 2Z (P).

Z (P)

I (P) 1

1 0

0

Note that Z (P) is the Bhattacharyya upper bound on probability of error for uncoded transmission over P. Thus, we can choose the good indiceson the basis of Z (P); the sum of the Z ’s of the chosen channels will upper bound the block error probability. This suggests studying the polarization rate of Z .

(65)

Polarization Rate

Properties of Z (P):

Z (P) ∈ [0, 1].

Z (P) ≈ 0 iff I (P) ≈ 1.

Z (P) ≈ 1 iff I (P) ≈ 0.

Z (P+) = Z (P)2. Z (P) ≤ 2Z (P).

Z (P)

I (P) 1

1 0

0

Note that Z (P) is the Bhattacharyya upper bound on probability of error for uncoded transmission over P. Thus, we can choose the good indiceson the basis of Z (P); the sum of the Z ’s of the chosen channels will upper bound the block error probability. This suggests studying the polarization rate of Z .

(66)

Polarization Rate

Properties of Z (P):

Z (P) ∈ [0, 1].

Z (P) ≈ 0 iff I (P) ≈ 1.

Z (P) ≈ 1 iff I (P) ≈ 0.

Z (P+) = Z (P)2. Z (P) ≤ 2Z (P).

Z (P)

I (P) 1

1 0

0

Note that Z (P) is the Bhattacharyya upper bound on probability of error for uncoded transmission over P. Thus, we can choose the good indiceson the basis of Z (P); the sum of the Z ’s of the chosen channels will upper bound the block error probability. This suggests studying the polarization rate of Z .

(67)

Polarization Rate

Properties of Z (P):

Z (P) ∈ [0, 1].

Z (P) ≈ 0 iff I (P) ≈ 1.

Z (P) ≈ 1 iff I (P) ≈ 0.

Z (P+) = Z (P)2. Z (P) ≤ 2Z (P).

Z (P)

I (P) 1

1 0

0

Note that Z (P) is the Bhattacharyya upper bound on probability of error for uncoded transmission over P. Thus, we can choose the good indiceson the basis of Z (P); the sum of the Z ’s of the chosen channels will upper bound the block error probability. This suggests studying the polarization rate of Z .

(68)

Polarization Rate

Given a binary input channel W ,

just as for I`, define Z` = Z (WB1,...,B`).

We know that Pr(Z` → 0) = I (W ).

It turns out that when Z`→ 0 it does so fast:

Theorem

For any β < 1/2, lim

`→∞Pr(Z`< 2−2β`) = I (W ).

This means that for any β < 1/2, as long as R < I (W ) the error probability of polarization codes decays to 0 faster than 2−nβ.

(69)

Polarization Rate

Given a binary input channel W ,

just as for I`, define Z` = Z (WB1,...,B`).

We know that Pr(Z` → 0) = I (W ).

It turns out that when Z`→ 0 it does so fast:

Theorem

For any β < 1/2, lim

`→∞Pr(Z`< 2−2β`) = I (W ).

This means that for any β < 1/2, as long as R < I (W ) the error probability of polarization codes decays to 0 faster than 2−nβ.

(70)

Polarization Rate

Given a binary input channel W ,

just as for I`, define Z` = Z (WB1,...,B`).

We know that Pr(Z` → 0) = I (W ).

It turns out that when Z`→ 0 it does so fast:

Theorem

For any β < 1/2, lim

`→∞Pr(Z`< 2−2β`) = I (W ).

This means that for any β < 1/2, as long as R < I (W ) the error probability of polarization codes decays to 0 faster than 2−nβ.

(71)

Polarization Rate

Given a binary input channel W ,

just as for I`, define Z` = Z (WB1,...,B`).

We know that Pr(Z` → 0) = I (W ).

It turns out that when Z`→ 0 it does so fast:

Theorem

For any β < 1/2, lim

`→∞Pr(Z`< 2−2β`) = I (W ).

This means that for any β < 1/2, as long as R < I (W ) the error probability of polarization codes decays to 0 faster than 2−nβ.

(72)

Polarization Rate

Given a binary input channel W ,

just as for I`, define Z` = Z (WB1,...,B`).

We know that Pr(Z` → 0) = I (W ).

It turns out that when Z`→ 0 it does so fast:

Theorem

For any β < 1/2, lim

`→∞Pr(Z`< 2−2β`) = I (W ).

This means that for any β < 1/2, as long as R < I (W ) the error probability of polarization codes decays to 0 faster than 2−nβ.

(73)

Polarization Rate

Proof idea

The subset of the probability space where Z`(ω) → 0 has probability I (W ).

For any  > 0 there is an m and a subset of this set with probability at least I (W ) −  with Z`(ω) < 1/8 whenever

` > m.

Intersect this set with high probability event: that ‘Bi = +’

and ‘Bi = −’ occur with almost equal frequency for i ≤ `. It is then easy to see that lim`→∞P(log2Z`≤ −`) = I (W ).

(74)

Polarization Rate

Proof idea

The subset of the probability space where Z`(ω) → 0 has probability I (W ).

For any  > 0 there is an m and a subset of this set with probability at least I (W ) −  with Z`(ω) < 1/8 whenever

` > m.

Intersect this set with high probability event: that ‘Bi = +’

and ‘Bi = −’ occur with almost equal frequency for i ≤ `. It is then easy to see that lim`→∞P(log2Z`≤ −`) = I (W ).

(75)

Polarization Rate

Proof idea

The subset of the probability space where Z`(ω) → 0 has probability I (W ).

For any  > 0 there is an m and a subset of this set with probability at least I (W ) −  with Z`(ω) < 1/8 whenever

` > m.

Intersect this set with high probability event: that ‘Bi = +’

and ‘Bi = −’ occur with almost equal frequency for i ≤ `. It is then easy to see that lim`→∞P(log2Z`≤ −`) = I (W ).

(76)

Polarization Rate

Proof idea – cont.

`0

−`0

` log2Z`

(77)

Polarization Rate

Proof idea – cont.

`0`0+`1

−`0

` log2Z`

`1= `0

(78)

Polarization Rate

Proof idea – cont.

`0`0+`1

−`0

−212`1

` log2Z`

`1= `0

(79)

Polarization Rate

Proof idea – cont.

`0`0+`1 `0+`1+`2

−`0

−212`1

` log2Z`

`1= `0 `2= 212`1

(80)

Polarization Rate

Proof idea – cont.

`0`0+`1 `0+`1+`2

−`0

−212`1

−212`2

` log2Z`

`1= `0 `2= 212`1

References

Related documents

4.1 The Select Committee is asked to consider the proposed development of the Customer Service Function, the recommended service delivery option and the investment required8. It

The positive and signi…cant coe¢ cient on the post shipment dummy in the fourth column implies that prices charged in post shipment term transactions are higher than those charged

• Follow up with your employer each reporting period to ensure your hours are reported on a regular basis?. • Discuss your progress with

Based on this understanding, this work focuses on the effect of the inclusion of shallow donor dopant such as gallium into the normal cadmium chloride post-growth treatment as

The corona radiata consists of one or more layers of follicular cells that surround the zona pellucida, the polar body, and the secondary oocyte.. The corona radiata is dispersed

Proprietary Schools are referred to as those classified nonpublic, which sell or offer for sale mostly post- secondary instruction which leads to an occupation..

When transfecting HEK 293T cells with either “mIFP-P2A-mNG2(full)” or “mIFP- P2A-mNG211SpyCatcher and mNGX1-10 (X represent 2, 3A or 3K)”, we observed both mNG31-10

Currently, National Instruments leads the 5G Test &amp; Measurement market, being “responsible for making the hardware and software for testing and measuring … 5G, … carrier