On Some Probabilistic Decoding Algorithms

(1)

S

-

i

**? * COORDINATED SCIEN CE LABORATORY**

m b

ON SOME PROBABILISTIC

DECODING ALGORITHMS

A.H. HADDAD

U N IVERSITY OF ILLIN O IS - URBANA, ILLIN O IS

(2)

ON SOME PROBABILISTIC DECODING ALGORITHMS By

A. H. Haddad

T h is work was s u p p o r te d by th e Rome A i r Development C en ter under C o n t r a c t F 3 0 6 0 2 -7 0 -C -0 0 1 4 ; a u x i l i a r y s u p p o r t was p r o v i d e d by the J o i n t S e r v i c e s E l e c t r o n i c s Program (U. S. Army, U. S. Navy and U. S. A i r F o r c e ) under C o n t r a c t D A A B -07-67-C -0199.

R e p r o d u c t io n i n whole p u rp o se o f th e U n ited S t a t e s

o r i n p a r t i s p e r m i t t e d f o r any Government.

T h is document has been approved f o r p u b l i c r e l e a s e and s a l e i t s d i s t r i b u t i o n i s u n l i m i t e d .

(3)

A. H. Haddad

C o o r d in a t e d S c i e n c e Laboratory- U n i v e r s i t y o f I l l i n o i s Urbana, I l l i n o i s 61801

A b s t r a c t

S e v e r a l methods f o r the f o r m u la t i o n o f the p r o b a b i l i s t i c d e c o d in g problem f o r b in a r y group co d e s in a m athem atical programming form are d i s c u s s e d . In p a r t i c u l a r the f o r m u l a t i o n as a p se u d o -B o o le a n programming and a z e r o - o n e i n t e g e r programming are c o n s i d e r e d in d e t a i l . The d i s c u s s i o n i s co n c e rn e d p r i m a r i l y w it h the in d ep en d en t e r r o r s c a s e . A sim p le ca s e o f f i r s t - o r d e r Markov e r r o r model i s a l s o i n v e s t i g a t e d , and the r e s u l t i n g a l g o r i t h m is

d e r i v e d . The ca s e o f dependent e r r o r s may r e s u l t in f a s t e r d e c o d in g a l g o r i t h m .

T h is work was su p p o r te d by Rome A ir Development C en ter under C o n t r a c t F 3 0 6 0 2 -7 0 -C -0 0 1 4 ; a u x i l i a r y s u p p o r t was p r o v i d e d by th e J o i n t S e r v i c e s E l e c t r o n i c s Program (U. S. Army, U. S. Navy, and U. S. A i r F o r c e ) under C o n t r a c t DAAB-07-67-C-0199.

(4)

I . INTRODUCTION 1 . 1 , G en era l

The purpose o f t h i s work i s t o e x p l o r e v a r i o u s methods f o r the p r o b a b i l i s t i c d e c o d in g f o r b in a r y channels w it h l i n e a r b l o c k c o d e s . S in c e the

p r o b a b i l i s t i c d e c o d in g problem i s one o f s e l e c t i n g the most l i k e l y e r r o r p a t t e r n g iv e n the syndrome, i t a f f o r d s a m athem atical programming f o r m u l a t i o n . The f o r m u l a t i o n o f the problem f o r the b in a r y symmetric ch a n n el ( i . e . , independent e r r o r s ) and the c o r r e s p o n d i n g d e c o d in g a l g o r i t h m s are the s u b j e c t o f the f i r s t p a rt o f t h i s w ork. The se co n d p a r t d i s c u s s e s o t h e r p o s s i b l e e r r o r m o d e ls, in p a r t i c u l a r , a f i r s t - o r d e r Markov m odel, which may r e s u l t in a s i m p l i f i e d s e a r c h method f o r d e c o d i n g . The o b j e c t is t o red u ce the number o f s t e p s r e q u i r e d in

the s e a r c h a l g o r i t h m , w hich i s the p r i n c i p a l d is a d v a n t a g e o f the v a r io u s p r o gramming t e c h n i q u e s .

1 . 2 . Problem f o r m u la t i o n

C o n s id e r an ( n , k ) b in a r y group code w i t h ch eck m atrix H, which i s a b in a r y (nXr) m atrix s a t i s f y i n g

uH = 0 (mod. 2) (1 )

where r = n-k and u = [ u ^ , . . . , u ] i s any t r a n s m i t t e d v e c t o r . L et the r e c e i v e d v e c t o r be v = u + e where e = [ e - , . . . , e ] i s the e r r o r v e c t o r . Then a s e t o f e q u a t i o n s in GF(2) may be d e r i v e d f o r

e

and e x p r e s s e d as f o l l o w s

£H =

s_

(mod. 2) (2 )

s^ = vH = [ s ^ , . . . , s ] i s the syndrome. where

(5)

The v a r i a b l e s and the m a trix elem en ts are a l l b in a r y ( 0 , 1 ) , and the a d d i t i o n s in (2 ) are in G F (2 ) , i « e 0 , "sum modulo 2 " , w hich w i l l be d e n o te d by ©.

E x p l i c i t l y e q u a t io n (2 ) may be w r i t t e n as

R. = [ e - h . . © e . h 0 . ©• • •© e h . ] © s . = 0 ;

i 1 li 2 2i n ni l

i = 1 , 2 , . . . , r (3)

The problem i s t o f i n d e f o r a g iv e n s , However, t h e re i s more than one s o l u t i o n f o r

e

s a t i s f y i n g (2 ) o r ( 3 ) . I f , however, the ch a n n el i s a b in a r y symmetric ch a n n e l, then the e r r o r s in each symbol are independent., The p r o b a b i l i t y o f a g iv e n p a t t e r n o f m e r r o r s w i l l then be g iv e n by pm( l - p ) n m where p i s the p r o b a b i l i t y o f an e r r o r in one sym bol. T h e r e f o r e , the s o l u t i o n t o the problem i s s p e c i f i e d as the most p r o b a b le v e c t o r

e

s a t i s f y i n g (2 ) or ( 3 ) . In t h i s c a s e , i t i s e a s i l y seen t h a t the most p r o b a b le e i s the one w ith the l e a s t number o f e r r o r s , i . e , , the v e c t o r

e

w it h the l e a s t number o f l ' s „ Such a s o l u t i o n i s c a l l e d a minimum-weight s o l u t i o n , and may be o b t a i n e d by m in im izin g the f u n c t i o n f ( e )

n

f ( e ) = S e (4 )

i = l

s u b j e c t t o the c o n s t r a i n t s o f (2 ) or (3 ) I t s h o u ld be n oted t h a t in (4 ) the a d d i t i o n i s in the o r d i n a r y s e n s e , w h i l e the c o n s t r a i n t s i n v o l v e o p e r a t i o n s o f "sum modulo 2 " . The problem may be tra n sfo rm e d i n t o a m athem atical programming form i f the c o n s t r a i n t s are r e w r i t t e n as

(6)

where the f u n c t i o n s f ^ ( e ) i n v o l v e o n ly o r d i n a r y o p e r a t i o n s . In the f o l l o w i n g s e c t i o n s two methods are c o n s i d e r e d f o r the f o r m u la t i o n o f the problem : ( i ) The p s e u d o -B o o le a n programming C1 - 2 ] , ( i i ) The z e r o - o n e i n t e g e r programming [3] . A t h i r d a l g o r i t h m p rop osed by Omura [4] and i n v o l v e s random s e a r c h w i l l a l s o be summarized h e r e .

(7)

I I . PSEUDO-BOOLEAN PROGRAMMING FORMULATION 2 . 1 . P seu d o-B oolea n f u n c t i o n s

In t h i s s e c t i o n a s h o r t i n t r o d u c t i o n t o p s e u d o -B o o le a n programming [ l - 2 ] w i l l be g i v e n . A p se u d o -B o o le a n f u n c t i o n f ( x ^ , . . . , x n) i s an i n t e g e r - v a l u e d

f u n c t i o n o f the b in a r y ( 0 , 1 ) v a r i a b l e s x ^ , . . . , x n . Any p s e u d o -B o o le a n f u n c t i o n may be e x p r e s s e d as a p o ly n o m ia l l i n e a r in each v a r i a b l e and w it h i n t e g e r c o e f f i c i e n t s . The o p e r a t i o n s and are the o r d i n a r y o p e r a t i o n s o f a d d i t i o n , s u b t r a c t i o n , and m u l t i p l i c a t i o n , r e s p e c t i v e l y . The B oolean o p e r a t i o n s " o r " (U) , " a n d " (fl) , and "com plem ent" ( x ) may be e x p r e s s e d e q u i v a l e n t l y by

xUy = x + y - x y

xfly = x* y (6 )

x = 1-x

S i m i l a r l y , the "sum modulo 2" may be e x p r e s s e d by the o r d i n a r y o p e r a t i o n s as

x © y = x + y - 2 x y (7 )

An a l t e r n a t i v e e x p r e s s i o n may a l s o be used

1 - 2 (x©y) = ( l - 2 x ) ( l - 2 y ) (8 )

which r e s u l t s in the more g e n e r a l r e l a t i o n

n

l - 2 [ x 1©x ® . . ,©x ] = II ( l - 2 x . )

1 2 n j = i j

(8)

The m in im iz a tio n o f a p se u d o -B o o le a n f u n c t i o n f ( x ^ , „ . . ,x^) s u b j e c t t o c o n s t r a i n t s

f i ( x 1 , . . . , x n) = 0 ; i = 1 , 2 , . . . , r (1 0 )

may be fo r m u la t e d in two d i f f e r e n t ways.

( i ) Let the g e n e r a l s o l u t i o n o f the c o n s t r a i n t s (1 0 ) be g iv e n by

X j = x ( p x , . . . , p n) , j = 1 , 2 , . . . ,n (1 1 )

w it h p ^ , . . . , p n a r b i t r a r y b in a r y p a ra m e te rs . The m in im iz a tio n o f f ( x ) s u b j e c t t o (1 0 ) i s e q u i v a l e n t t o the m in im iz a tio n o f the u n c o n s t r a i n e d f u n c t i o n

F ( p ^ »•••»Pn) — f [ x 1 ( p 1 , . . . ,Pn) , . . . , x n ( p 1 , . - • ,Pn) l (12)

w it h r e s p e c t t o the parameters p ^ , . . . , P n . The d e s i r e d x ^ , . . . , x may then be o b t a i n e d from ( 1 1 ) .

( i i ) An a l t e r n a t i v e method f o r the m in im iz a t io n may be o b t a in e d i f the c o n s t r a i n t s s a t i s f y

f . ( x . , , , . . , x ) ^ 0 a l l ( x . , . . . , x ) (13)

1 n x 1 n

In t h i s c a s e , l e t D and E den ote an upper bound and a low er bound o f f ( x ) r e s p e c t i v e l y . The problem i s e q u i v a l e n t t o m in im izin g the u n c o n s t r a in e d f u n c t i o n

r

f ( x 1 , . . . , x n) = f ( x 1 , . . . , x n) + (D -E + l) £ f i ( x 1 , . . . , x n) i = l

(9)

I f the minimum v a lu e o f F i s l a r g e r than D then the c o n s t r a i n t s (1 0 ) are i n c o n s i s t e n t , o t h e r w i s e , the s o l u t i o n i s the d e s i r e d on e.

The d e c o d in g problem w i l l be f o r m u la t e d as a p s e u d o -B o o le a n p r o gramming problem , and the r e s u l t i n g a l g o r i t h m w i l l then be i n d i c a t e d .

2 . 2 . The d e c o d in g problem in a p se u d o -B o o le a n form

As s t a t e d in the i n t r o d u c t i o n the d e c o d in g problem may be s t a t e d as

An e q u i v a l e n t f o r m u la t i o n o f (1 6 ) may be o b t a i n e d from (9 ) which i s e x p r e s s e d in the p s e u d o -B o o le a n form n min f ( e ) = mm Z e . • i i e e i = l (15) s u b j e c t t o the c o n s t r a i n t s , . • . , r (1 6 ) n f . ( e ) = l - ( l - 2 s . ) n ( l - 2 e .h . . ) = 0 1 “ 1 . _ n J J i (17a) or n f . ( e ) = 1 - ( s . - s . ) II (e .h .. - e . h . . ) = 0 1 i 1 j =1 J J i J J i ' (1 7 b) 1 = 1,2, • . . , r

I t sh o u ld be n oted t h a t w h i l e the R. are a z e r o - o n e v a lu e d f u n c t i o n s , the f . ( e ) take v a lu e s ( 0 , 2 ) in o r d e r t o f u l f i l l the r e s t r i c t i o n o f i n t e g e r c o e f f i c i e n t s . A l t e r n a t i v e l y , the r c o n s t r a i n t s may be combined i n t o a s i n g l e e x p r e s s i o n

(10)

r R ( e ) = 1 - II [ l - R . ( e ) ] = 0 (1 8 a) i = l o r r R ( e ) = 2r - n [ 2 - f ( e ) ] = 0 (1 8 b) i = l

In t h i s c a s e t o o R ta k es the v a lu e s ( 0 , 1 ) w h i l e R takes the v a lu e s ( 0 , 2 r ) o The problem re d u c e s t h e r e f o r e t o m in im iz in g (1 5 ) s u b j e c t t o the c o n s t r a i n t s

(1 7 ) o r ( 1 8 ) .

The f i n a l f o r m u la t i o n as a p s e u d o -B o o le a n programming problem depends on the ap p roa ch u s e d .

( i ) I n d i r e c t approach

The m in im iz a tio n problem i s e q u i v a l e n t t o m in im izin g the s i n g l e p se u d o - B oolea n f u n c t i o n n r F ( e ) = S e + (D+l) S f , ( e ) (1 9 ) i = l i = l 1 o r n F ( e ) = £ e + ( D + l ) R ( e ) (2 0 ) i = l

where D i s an upper bound f o r f ( £ ) . A low er bound on f ( e ) i s o b v i o u s l y z e r o . S in c e a s o l u t i o n f o r

e

may be found such th a t i t has w e ig h t e q u a l t o

r LL = 2 S .

(11)

the upper bound f o r f ( e ) may be taken as jjl , o r in the w o r s t c a s e , as r . The problem i s t h e r e f o r e t o minimize the f u n c t i o n

n r n F ( e ) =

Z

e + 0i+1) Z [ (l - 2 s ) II ( l - 2 e h )] (2 2 ) i = l j = l J i = l J w it h r e s p e c t t o the b in a r y v a r i a b l e s e ^ , . . . , e . ( b ) D i r e c t approach I t i s not d i f f i c u l t t o o b t a i n a p a r t i c u l a r s o l u t i o n o r ° o o-i e — )6 , . > . ,e J — 1 2 n (23)

w hich s a t i s f i e s the c o n s t r a i n t s ( 1 8 ) . The g e n e r a l s o l u t i o n f o r e s a t i s f y i n g the c o n s t r a i n t s may then be e x p r e s s e d as

e ± = [ (e°©xi )R(x)]©xi , i = 1 , 2 , . . . ,n (24)

where x = [ x ^ , . . . , x n] is any arbitrary binary vector. The substitution of (24) in (15) resu lts in the following expression to be minimized with respect to x°

n

F(x) = 2 [ (e?®x . ) ® R (x )]© x . (25)

i = l 1 1 L

w hich may a l s o be w r i t t e n in a p s e u d o -B o o le a n form . The r e s u l t i n g x which m in im izes (25) can then be s u b s t i t u t e d in (24) t o o b t a i n the s o l u t i o n f o r e ,

( c ) The n o rm a lize d case

(12)

such t h a t

(2 6 )

where P i s a kXr m a t r i x , and 1^ i s the rXr i d e n t i t y m a t r i x . In t h i s c a s e the e r r o r v e c t o r

e

may be p a r t i t i o n e d i n t o

e = [_y 1 x j (2 7 )

where and x are r and k d im e n s io n a l b in a r y v e c t o r s r e s p e c t i v e l y . The c o n s t r a i n t s (1 6 ) can then be s o l v e d e x p l i c i t l y f o r _y, s i n c e

%

+ xP = s; (m od.2) (2 8 )

The e x p r e s s i o n f o r i s t h e r e f o r e g iv e n by

y i = s j0 [ x i p l i ® . . •®xkPk i ] > i - l , 2 , . . . , r (2 9 )

which in a p s e u d o -B o o le a n form becomes

k

2 y . = l - ( l - 2 s . ) n ( l - 2 x . p . . ) (3 0 )

i l . - i i i

J=1

Ths s u b s t i t u t i o n o f (3 0 ) in (1 5 ) r e s u l t s in the f i n a l form o f the f u n c t i o n t o be minimized

k r k

F ( x ) = 2 2 x . - 2 t ( l - 2 s . ) n ( l - 2 x . p . . ) ] j - 1 J i = l 1 j = l J J1

(13)

The advantage o f t h i s form i s t h a t the number o f v a r i a b l e s i s re d u ce d from n t o the s m a lle r number k.

The m in im iz a tio n a l g o r i t h m and i t s l i m i t a t i o n w i l l be c o n s i d e r e d in the n ext s e c t i o n .

2 . 3 . The m in im iz a tio n a l g o r i t h m

The a l g o r i t h m f o r m in im izin g a p s e u d o -B o o le a n f u n c t i o n i s v e ry complex and not e a s i l y c o m p u t e r iz e d . I t i s e f f i c i e n t o n l y f o r s m a ll number o f v a r i a b l e s , To i l l u s t r a t e the m in im iz a tio n a l g o r i t h m , the f i r s t s t e p s in m in im izin g F ( x ) in

(3 1 ) w i l l be g i v e n . The f u n c t i o n t o be m inim ized i s e x p r e s s e d as

F ( x ) F ( x ^ . ,x ^ ) x 1g 1( x 2 , . . . , x k) + h ^ ( x 2 , . »« >x k) (3 2 )

where f o r (3 1 ) the f u n c t i o n s

g-^

and h^ are g iv e n by

L

IX S , ( x , ...x ) = 2 { 1 + £ [ ( l - 2 s ) p n ( l - 2 x p ) ] } 1 2 K i=1 1 L1 j - 2 3 J1 k r k h (x , . . , , x ) = 2 2 x - E [ ( l - 2 s ) II ( l - 2 x p ) ] }

L Z

k

j=2 J i = l 1 j - 2 J J (3 3 a) (3 3 b)

Then the v a r i a b l e x^ i s d eterm in ed by the f u n c t i o n cp^ which sh o u ld be e x p r e s s e d in a p se u d o -B o o le a n fo rm , and i s d e f i n e d by x l = CP 1 ( X 2 ’ \ , f o r

gl

< 0 , f o r

gl

^ 0 (3 4 )

(14)

o r e q u i v a l e n t l y <Pi == 1 , for £ [ ( l - 2 s . ) P i . II ( l - 2 x . p . . ) ] < -1 i - l 1 l x j=2 3 31 0 , otherwise (3 5 )

The s e co n d s t e p w i l l be t o d e f i n e the f u n c t i o n F2 which i s then decomposed in the same manner as F^, i . e „ ,

F 2 ( x 2 »• • • , x k> = F 1(cP1 »x2 , . . . , x k) =

= ^ 2g2 ( x 3 , . . . , x k) + h2 ( x 3 , . . . 9x k) (3 6 )

In t h i s ca s e the e x p r e s s i o n s f o r g 2 and h2 are g iv e n by

M x3 ’ . , x . ) = 2 ( G { l + 2 _{i = l}C ( l - 2 s . ) p, ( l - 2 p 0 . ) n ( l - 2 x p i l i 2 i . q ) ] ) J=3 J J1 - G{1

+ 2

[ ( l-2 s

. ) p. . n

(l-2 x

P ) ] }

i - l j=3 J 3 + 1 + 2 [ ( l - 2 s . ) p II ( l - 2 x p ) ] ) 1=1 j=3 (3 7 a) h2 ( x 3 , . x . ) = 2 2 x - 2 [ ( l - 2 s . ) II ( l - 2 x p ) ] K j=3 3 i - l j=3 3 J1 + 2G{1 + 2 [ < l - 2 s ) p n ( l - 2 x p ) ] ] i - l 11j=3 3 3 (3 7 b)

(15)

g o o =

l <

0

i

^ 0

(3 8 )

The v a r i a b l e x^ can now be d eterm in ed as the f u n c t i o n cp^ w hich i s d e f i n e d by /

x 2 =<P2 ( x 3 ’ - - - ’ Xk) = '

g2 < 0

o t h e r w is e

(3 9 )

In t h i s manner the p r o c e s s c o n t i n u e s , and in g e n e r a l i s d e f i n e d and then decomposed as f o l l o w s F . (x . ,x - , l v i l + l * ’ XP F i - l ^ i - l , x i + l : ,x k] = = x , g , ( x i + 1 , . 1 1 . ,x, ) + h . (xi + 1 ’ •’ x k> (4 0 ) The c o r r e s p o n d i n g e x p r e s s i o n f o r x^ w i l l be g iv e n by x . = cp . (x . 1 T i v i-t-1

t

1

₉

g i < 0 0

9

_{S i 2 0} (4 1 )

The p r o c e s s c o n t in u e s u n t i l x^ i s o b t a i n e d , and then by s u c c e s s i v e s u b s t i t u t i o n in (4 1 ) a l l the x^ w i l l be d e r i v e d . I t i s e v i d e n t t h a t the m in im iz a tio n p r o ced u re in t h i s ca se i s in g e n e r a l i n e f f i c i e n t and i s e q u i v a l e n t t o e x h a u s t iv e

1c

s e a r c h . The number o f s t e p s r e q u i r e d w i l l t h e r e f o r e be 2 , e x c e p t where s i m p l i f i c a t i o n s are p o s s i b l e . The s t r u c t u r e o f the c o d i n g problem makes c e r t a i n s i m p l i f i c a t i o n s in the a l g o r i t h m s , however the improvement in the number o f s t e p s i s not

(16)

s i g n i f i c a n t . One b a s i c d i f f i c u l t y in the p se u d o -B o o le a n programming f o r m u la t i o n i s t h a t the c o n s t r a i n t s are n o n l i n e a r , which c o m p l i c a t e s the a l g o r i t h m . A

l i n e a r programming f o r m u l a t i o n i s p o s s i b l e by the i n t r o d u c t i o n o f a d d i t i o n a l v a r i a b l e s . Such a f o r m u l a t i o n i s the z e r o - o n e i n t e g e r programming which i s c o n s i d e r e d in the next s e c t i o n .

(17)

I I I . ZERO-ONE INTEGER PROGRAMMING FORMULATION 3 . 1 . Problem f o r m u la t i o n

The z e r o - o n e i n t e g e r programming method [ 3 , 5 ] u ses an i m p l i c i t enumer a t i o n t e ch n iq u e f o r the m in im iz a tio n o f the l i n e a r s c a l a r p ro d u ct

_c * x (4 2 )

s u b j e c t t o the l i n e a r c o n s t r a i n t s

x aA + b ^ 0 (4 3 )

The o p e r a t i o n s in ( 4 2 ) , (4 3 ) are the u s u a l a d d i t i o n and m u l t i p l i c a t i o n , the v e c t o r x takes o n l y b i n a r y ( 0 , 1 ) v a lu e s w h i l e the elem en ts o f the v e c t o r s _b, c and the m a trix A are i n t e g e r s .

S in ce the d e c o d in g problem i n v o l v e s the m in im iz a tio n o f a l i n e a r e x p r e s s i o n s u b j e c t t o n o n l i n e a r c o n s t r a i n t s , i t may be t ra n s fo r m e d i n t o the z e r o - o n e i n t e g e r programming i f the c o n s t r a i n t s are l i n e a r i z e d . One approach o f a c h i e v i n g t h i s o b j e c t i v e i s t o i n t r o d u c e a d d i t i o n a l b in a r y v a r i a b l e s {nu .} such t h a t the c o n s t r a i n t (3 ) may be w r i t t e n as

a j - 1 E e . h . . = s. + 2 E m. . 2J , i = 1 , 2 , . . . , 4 j j i i . . l i ’ ’ ’ j = l J J=1 J (44)

The number o f new v a r i a b l e s i s

vi

and

i

i s d eterm in ed by the r e l a t i o n

(18)

where |i i s an upper bound on the number o f e r r o r s . In the n o rm a liz e d ca s e

r |i = E s .

i = i 1

h ow ever, i t may be r e p l a c e d by a b e t t e r bound as the m in im iz a tio n p r o c e s s i s p e r f o r m e d . I f an rX4 m a trix M i s formed o f the nu , the c o n s t r a i n t s (3 ) may be w r i t t e n as

e*H - d*M' - s £ 0

- e*H + d*M' + s ^ 0

(4 6 )

r 2 Xn

where d = L2, 2 , . . . , 2 J . In o r d e r t o o b t a i n the f i n a l f o r m u l a t i o n , the v e c t o r s x , c and the m a trix A are d e f i n e d as f o l l o w s

— — o A = o H' o . o -d 1 0 | - ₁ _-d _« • lo * lo • • 0 ₁

_{o }*

_!

l

and where a i s t h e r e f o r e e c t t o the 2r ] , b = 0 -d (4 7 ) (4 8 )

(19)

T h is f o r m u la t i o n a llo w s the use o f the z e r o - o n e i n t e g e r programming method f o r the s o l u t i o n o f the d e c o d in g problem« Some a s p e c t s o f the methods are d i s cu s s e d in the next s e c t i o n .

3 . 2 « The z e r o - o n e programming a l g o r i t h m

An i n i t i a l s o l u t i o n which a l s o p r o v i d e s an upper bound on the number o f e r r o r s i s d e r i v e d and s t o r e d . The a l g o r i t h m i s based on s e l e c t i n g a p a r t i a l s o l u t i o n S w hich i s a p a r t i c u l a r valu e assign m en t t o some o f the v a r i a b l e s in x , Then com p lete s o l u t i o n s based on the p a r t i a l s o l u t i o n are c h e c k e d , and i f one such s o l u t i o n r e s u l t s in a lower valu e o f c* x i t r e p l a c e s the e a r l i e r s o l u t i o n , A f t e r a l l co m p le t io n s t o a p a r t i c u l a r p a r t i a l s o l u t i o n have been ch ecked the a l g o r i t h m b a c k t r a c k s and c h o o s e s a new p a r t i a l s o l u t i o n , which has not been a p a r t o f p r e v i o u s s o l u t i o n s . Due to the s i m p l i c i t y o f the c o n s t r a i n t s , the p r o c e s s o f c h e c k in g a p a r t i a l s o l u t i o n and i t s c o m p le t io n s can be s i m p l i f i e d . However, the b a c k t r a c k i n g r e q u i r e s a l a r g e number o f i t e r a t i o n s , and th e r e i s no g e n e r a l way o f r e d u c in g t h i s number e x c e p t in s p e c i a l c a s e s .

To i l l u s t r a t e how the s t r u c t u r e o f the i n e q u a l i t i e s a re h e l p f u l in s i m p l i f y i n g the check o f a p a r t i a l s o l u t i o n a few o f the c o n d i t i o n s f o r the g e n e r a l program w i l l be found f o r t h i s c a s e . L et W be the s e t o f i n d i c e s o f a s s ig n e d v a r i a b l e s o f a p a r t i a l s o l u t i o n S. The f o l l o w i n g q u a n t i t i e s are d e f i n e d y . ( S ) = b . + E x .a . „ = c u r r e n t value o f i - t h i n e q u a l i t y , 1 1 jew J 31 4 . ( S ) = y . ( S ) + £ a . . = min. value o f i - t h i n e q u a l i t y , 1 1 jew J1 a . .<0 J i u . ( S ) = T . ( S ) + Z a .. 1 1 J1 = max, value o f i - t h i n e q u a l i t y .

(20)

I t i s e a s y t o see t h a t in the d e c o d in g problem

y i + r (S) = - y i (S) i i + r (S ) = ' u i (S) u i + r (S) = - i ± (S )

i = 1 , 2 , . . . , r (4 9 )

Some o f the c o n d i t i o n s w hich may be based on the above d e f i n i t i o n s a r e : ( 1 ) I f u . < 0 o r 1 . < 0 f o r some i = l , 2 , . . . , r , then the p a r t i a l s o l u t i o n S

i s n o t f e a s i b l e ,

( 2 ) For r ^ i ^ 1, i f ¡ a ^ | > ^ 1 f o r some j$W, then x^. = 0 , and i f a ..1 >

-&

. ^ 1 f o r some then x . = 1

1 J i 1 i J

(3 ) I f u. = 0 , then x . = 0 f o r i > n and i#W; and i f a l s o h . . = 1, then

l j J J j i

0^ = 1 f o r j?W.

(4 ) I f

-

0 , then x^ = 1 f o r j > n and j$W; and i f a l s o h ^ = 1, then e . = 0 f o r i#W.

J

A d d i t i o n a l s i m p l i f y i n g c o n d i t i o n s f o r c o m p le t in g the p a r t i a l s o l u t i o n S may be s i m i l a r l y o b t a i n e d . Another s i m p l i f y i n g f e a t u r e i s the f a c t th a t as a new bound (jl^ f o r c ° x i s o b t a i n e d , the c o r r e s p o n d i n g

i

^ from (45) may then be u s e d , and the v a r i a b l e s m . . f o r j >

i,

w i l l be s e t t o z e r o . In t h i s manner the number

i j 1

o f f r e e v a r i a b l e s w i l l d e c r e a s e as the p ro c e d u re i s c o n t i n u e d . However, s i n c e the number o f c o n s t r a i n t s i s r e l a t i v e l y s m a ll ( 2 r ) compared t o the number o f v a r i a b l e s (n+rj£) the a l g o r i t h m i s e x p e c t e d t o be i n e f f i c i e n t due t o the b a c k t r a c k i n g . An e s t im a t e o f the number o f i t e r a t i o n s r e q u i r e d f o r the d e c o d in g may be o b t a i n e d o n l y through s i m u l a t i o n . I t i s e x p e c t e d t h a t even though t h i s

number may be p r o h i b i t i v e l y l a r g e , i t s t i l l may be low er than the number r e q u i r e d by d i r e c t e x h a u s t i v e s e a r c h .

(21)

IV . OMURA'S ALGORITHM 4 . 1 , B a s ic r e l a t i o n s

In t h i s s e c t i o n a randomized d e c o d in g a l g o r i t h m p ro p o se d by Omura [4] w i l l be summarized f o r c o m p l e t e n e s s . The a l g o r i t h m i s based on comparing s o l u

t i o n s o b t a i n e d a f t e r i n t e r c h a n g i n g two rows in the H m a trix and then n o r m a l i z i n g . The m a trix H and the v e c t o r

e

are p a r t i t i o n e d as f o l l o w s

(5 0 )

where B i s a n o n s in g u la r rXr m a t r i x . Let B c o n t a i n the rows { i , , i _ , . . . , i } = I .

1 2 r

The b a s i c e q u a t i o n (2 ) may now be w r i t t e n as

xB +

%R

=

s_

(mod. 2 ) (51) S in ce B i s n o n s in g u la r (2 ) may be n o rm a lize d as f o l l o w s x + jRB "*" = sB ^ (m o d .2) jg or by p r o p e r l y d e f i n i n g Z and s we have x + Z = £ (52) (5 3 ) A s o l u t i o n t o the d e c o d in g problem s a t i s f y i n g (5 3 ) i s g iv e n by x

2

=

0

(5 4 )

(22)

F u rth e rm o re , th e re e x i s t s a B such th a t the s o l u t i o n g iv e n by (5 4 ) i s the d e s i r e d minimum-weight s o l u t i o n .,

I f now a row h. in B i s in t e r c h a n g e d w it h a row h in R where z . = 1,

—j6 —m nu

l

then the new m atrix B' i s a l s o n o n s i n g u l a r . The s o l u t i o n e ' o f (5 4 ) c o r r e s p o n d i n g t o B 1 may then be compared t o the s o l u t i o n

e

c o r r e s p o n d i n g t o B. The d i f f e r e n c e in w e ig h t A^ o f the two s o l u t i o n s i s g iv e n by / A m n = 2 (e ! - e . ) = < . , v l i y i= l 0 r R 1 + 2 z . ( l - 2 s . ) . , mi l i = l , i f = 0 . . B _ 1 s - 1 (5 5 )

T h is l a s t r e l a t i o n forms the b a s i s o f the a l g o r i t h m w hich i s d e s c r i b e d n e x t .

4 . 2 o The randomized a l g o r i t h m

The a l g o r i t h m p erform s a random s e a r c h as f o l l o w s :

B

( a ) A b a s i c m a trix B i s c h o s e n , the e q u a t i o n s are n o rm a lize d t o o b t a i n s and Z, and the c o r r e s p o n d i n g s o l u t i o n f o r

e

i s fo u n d ,

( b ) The d i f f e r e n c e in w e ig h t A^ in (5 5 ) i s c a l c u l a t e d f o r a l l m ^ I .

( c ) I f th e r e are more than one A < 0 , the m w ith s m a l l e s t A i s c h o s e n , and an

m m

g a r b i t r a r y

£

€ I i s ch o se n such th a t z ^ = 1 and s^ = 1.

(d ) I f a l l A^ ^ 0 , an m and

£

are p ic k e d a t random such t h a t z ^ = 1 and = 0 . ( e ) With m and

£

from ( c ) o r (d ) a new b a s i s i s formed by in t e r c h a n g i n g h^ and

h^, and the p ro c e d u re i s r e p e a t e d as in ( a ) .

The l i m i t a t i o n s o f the a l g o r i t h m i s t h a t i t i s n ot p o s s i b l e t o guarantee c o n v e r g e n c e , o r th a t a l l p o s s i b i l i t i e s have been e x h a u s t e d , T h e r e f o r e , an

adequate s t o p p i n g r u l e i s r e q u i r e d , in ca s e o f no co n v e r g e n c e a f t e r a c e r t a i n number o f s t e p s . Omura [ 4 ] s im u la t e d the a l g o r i t h m f o r two c o d e s : one BCH code

(23)

and the o t h e r a randomly g e n e ra te d c o d e . The a vera ge number o f i t e r a t i o n s was s m a lle r f o r the random c o d e , T h e r e f o r e , i t seems th a t i f the s t r u c t u r e o f c e r t a i n known co d e s i s u t i l i z e d in p r o p o s i n g an a l g o r i t h m , i t may be p o s s i b l e t o reduce the number o f i t e r a t i o n s r e q u i r e d f o r d e c o d i n g .

F i n a l l y , in g e n e r a l m in im iz a tio n m ethods, i f a s y s t e m a t i c s e a r c h i s k]

t o be p e rfo rm e d , then the number o f i t e r a t i o n s r e q u i r e d i s g iv e n by —7-77 , , M< • M

)

where p. i s the number o f e r r o r s ( i . e . , the w e ig h t o f the s o l u t i o n ) . The number f o r l a r g e k i n c r e a s e s e x p o n e n t i a l l y w it h k . I t p r o v i d e s a m eaningful method o f s o l v i n g the d e c o d in g problem o n ly i f p, i s v e ry s m a l l . I t s h o u ld be n o t e d , t h a t a l l the above methods are r e s t r i c t e d t o the b in a r y symmetric c h a n n e l. I t i s p o s s i b l e t o c o n s i d e r o t h e r e r r o r models which may r e s u l t in a s m a lle r number o f s t e p s f o r d e c o d in g l a r g e r number o f e r r o r s . In the n e x t s e c t i o n , such a sim p le model i s i n v e s t i g a t e d .

(24)

V. MARKOV ERROR MODEL 5 . 1 . I n t r o d u c t i o n

The problem o f d e c o d in g i s t o f i n d a v e c t o r £ s a t i s f y i n g

eH = s (mod«2) (56)

w hich has the l a r g e s t p r o b a b i l i t y « In the BSC ca s e such a s o l u t i o n sh o u ld have the s m a l l e s t number o f ones« T h e r e f o r e , the s y s t e m a t i c a l g o r i t h m t o a c h ie v e t h i s would be t o t r y f i r s t a l l p o s s i b l e v e c t o r s £ w it h o n ly one e r r o r , then a l l p o s s i b l e v e c t o r s w i t h o n ly two e r r o r s , and so on u n t i l a v e c t o r s a t i s f y i n g (56) i s fo u n d . I f the H i s n o r m a l i z e d , then the number o f s t e p s t o d ecod e |i e r r o r s w i l l be

k

:

^ ( k - i ) * ! ' (5 7 )

I t i s e v i d e n t t h a t a s y s t e m a t i c a l g o r i t h m s e a r c h e s f o r the more p r o b a b le e r r o r p a t t e r n s f i r s t . I f the ch a n n el e r r o r s are n ot in d e p e n d e n t, the. number o f e r r o r s w i l l no lo n g e r determ ine t h e i r p r o b a b i l i t y , and the p a t t e r n w i l l a l s o be

im p o rta n t. T h e r e f o r e , i f the s e a r c h i s s t i l l perform ed in such a way t h a t the more p ro b a b le p a t t e r n s are ch e ck e d f i r s t , i t may be p o s s i b l e t o redu ce the

number o f s t e p s . As an exam ple, c o n s i d e r the ca s e th a t b u r s t s are more p r o b a b le than two independent e r r o r s , then the s e a r c h f o r b u r s t s w i l l be perform ed f i r s t , and s i n c e i t r e q u i r e s l e s s s t e p s than the s e a r c h f o r two e r r o r s , the number o f

i t e r a t i o n s w i l l be r e d u c e d . A model f o r the e r r o r s in the ch a n n el i s p r o p o s e d , and the r e s u l t i n g d e c o d in g a l g o r i t h m based on a s y s t e m a t i c s e a r c h i s c o n s i d e r e d .

(25)

5 . 2 . E r r o r Model

The model t o be c o n s i d e r e d i s a f i r s t - o r d e r Markov ch a in f o r the e r r o r s in the c h a n n e l. As l i m i t i n g ca s e s the BSC c a s e o r the b u r s t e r r o r s ca s e may be o b t a i n e d . I t can be shown by the f o l l o w i n g s t a t e diagram

l-(3

l-Ck'

where O', |3 , 1 - a , 1-3 are the t r a n s i t i o n p r o b a b i l i t i e s P (e j J e ^

])

f ° r two a d j a c e n t e r r o r b i t s . In the l i m i t i n g ca s e 3 = (1-cO i t re d u c e s t o the BSC ch a n n e l. The more i n t e r e s t i n g ca s e o f n o n -in d e p e n d e n t e r r o r s w i l l be c o n s i d e r e d w it h the

m

assum ptions 3 > ( l - a ) The s t e a d y s t a t e e r r o r p r o b a b i l i t y p w i l l be g iv e n by

1+ £f>

(5 8 ) which f o r a r e a l i s t i c ch a n n el r e q u i r e s t h a t

(if) » 1

s i n c e u s u a l l y p i s very s m a l l .

(26)

The s t a t i s t i c a l p r o p e r t i e s o f the e r r o r p a t t e r n s can be d eterm in ed by the f o l l o w i n g p r o b a b i l i t y

f^(l-L) = P rob, { a p a r t i c u l a r sequence o f

i

n o n - a d j a c e n t b u r s t s h a vin g t o t a l e r r o r s } , |i ^

i

I t can be shown th at the f u n c t i o n f^(M.) i s g iv e n a p p r o x im a t e ly by

f ,( p O = c (— ) ( " J ” ) <f) (59)

where the b l o c k le n g th i s assumed l a r g e w i t h r e s p e c t t o the number o f e r r o r s . S in c e the a l g o r it h m i s based on s e a r c h i n g f o r more p r o b a b le p a t t e r n s f i r s t , i t i s h e l p f u l t o o b t a i n i n e q u a l i t y r e l a t i o n s f o r the f^ ((j,), I t can be e a s i l y seen t h a t i f where

V 1 = V "-1

, 0 ^ i £ ( k - i ) m = r l o g ( l - 0 ) q - c O - 2 1 o l o g ^ - l o g 3 (6 0 ) (61) In p a r t i c u l a r , f o r

JL

= 1, k = 2 , i t i s seen t h a t a s i n g l e b u r s t o f up t o m o e r r o r s i s more l i e k l y than two independent e r r o r s . As an example f o r the valu e o f mQ, f o r a = 0 , 3 , p = 10 then m^ = 10, The d e c o d in g a l g o r i t h m w i l l be a s y s t e m a t i c s e a r c h based on the r e l a t i o n ( 6 0 ) ,

5 , 3 , The s e a r c h a l g o r i t h m

The a l g o r i t h m w i l l s e a r c h f o r the e r r o r s a c c o r d i n g t o the d e c r e a s i n g o r d e r o f the p r o b a b i l i t y o f the p a t t e r n . I t may be shown in the f o l l o w i n g f l o w diagram:

(27)

(28)

The r e s u l t i n g number o f s t e p s can be shown t o be s m a ll e r than the number o f s t e p s r e q u i r e d f o r the BSC c a s e « F u rt h e r a n a l y s i s o f the average number o f s t e p s is r e q u i r e d « S i m i l a r l y , e x p r e s s i o n s f o r the p r o b a b i l i t y o f e r r o r a f t e r d e c o d i n g , and the c a p a b i l i t y o f a g iv e n code have t o be i n v e s t i g a t e d . As an example o f the r e d u c t i o n in the number o f s t e p s , c o n s i d e r the ca se where e i t h e r one b u r s t o f le n g t h up t o m^ or 2 s i n g l e e r r o r s are more p ro b a b le than 3 s i n g l e e r r o r s » The number o f s t e p s in t h i s ca s e w i l l be k ( k - l )

2

m o E i = l ( k - i + 1 ) w h i l e the number o f s t e p s f o r 3 s i n g l e e r r o r s i s k + k ( k - l ) k ( k - l ) ( k - 2 ) 2 6

w hich i s c l e a r l y much l a r g e r than the form er e x p r e s s i o n . However, f o r a more g e n e r a l co m p a ris o n , and f o r the number o f e r r o r p a t t e r n s which can be c o r r e c t e d by a g i v e n code f u r t h e r i n v e s t i g a t i o n i s r e q u i r e d . More complex models may a l s o be more h e l p f u l in d e v e l o p i n g f a s t e r a l g o r i t h m s .

(29)

V I „ CONCLUSION

The p r o b a b i l i s t i c d e c o d in g problem has been fo r m u la te d as a mathe m a t i c a l programming prob lem . The p s e u d o -B o o le a n programming f o r m u la t i o n is n o n l i n e a r , and r e q u i r e s an e x h a u s t i v e s e a r c h in g e n e r a l . The z e r o - o n e i n t e g e r programming method i s l i n e a r , h ow ever, the number o f v a r i a b l e s i s l a r g e . The a l g o r i t h m r e q u i r e s a l a r g e number o f i t e r a t i o n s u n le s s some f u r t h e r p r o p e r t i e s o f p a r t i c u l a r ca s e s are u t i l i z e d . A randomized a l g o r i t h m p ro p o se d by Omura has a l s o been summarized, and the r e s u l t s i n d i c a t e a s lo w a l g o r i t h m w ith no gu aran tee o f c o n v e r g e n c e . F u rth e r s i m p l i f i c a t i o n s in the th re e a lg o r it h m s may be p o s s i b l e i f one c o n s i d e r s the s t r u c t u r e o f s p e c i a l co d e s i n s t e a d o f the

g e n e r a l ca s e o f an a r b i t r a r y ch e c k m a t r i x . A p ro m is in g problem may be the study o f e r r o r models which are not in d e p e n d e n t. A lg o rith m s f o r such models may

r e q u i r e a s m a lle r number o f i t e r a t i o n s . A sim p le example o f a f i r s t - o r d e r Markov c a s e has been i n v e s t i g a t e d b r i e f l y and the r e s u l t i n g a l g o r i t h m has been

c o n s i d e r e d . F u rth e r s tu d y i s r e q u i r e d o f the average number o f s t e p s needed f o r d e c o d i n g , the p r o b a b i l i t y o f e r r o r a f t e r d e c o d i n g , and the c a p a b i l i t i e s o f g iv e n cod es f o r such m o d e ls .

(30)

REFERENCES

lo P, L. I v a n e s c u , " P s e u d o -B o o le a n Programming and A p p l i c a t i o n s , L e ctu re N otes in M a t h . , No. 9, S p r i n g e r - V e r l a g , B e r li n - H e i d e l b e r g - N e w Y o rk , 1965« 2« Po L. Iv a n e s c u , and S . Rudeanu, "P s e u d o -B o o le a n Methods f o r B i v a l e n t

Programming," L e c t u r e Notes in Math, No. 2 3 , S p r i n g e r - V e r l a g , B e r l i n - Heide lberg-N ew Y o rk , 1966.

3» T« K„ L i u , "A Code f o r Zero-On e I n t e g e r L in e a r Programming by I m p l i c i t E n u m e r a tio n ," R e p o r t No, 3 02, D e p t, o f Computer S c i e n c e , U n i v e r s i t y

o f I l l i n o i s , Urbana, I l l i n o i s , D ec, 15, 1968,

4 , J . K. Omura, "A P r o b a b i l i s t i c D ecodin g A lg o r it h m f o r B in ary Group C o d e s , " S t a n f o r d R e s e a rch I n s t i t u t e , Menlo Park, C a l i f o r n i a , March 1969, 5 , T . I b a r a k i , T . K, L i u , C, R. Baugh, and S , Muroga, "An I m p l i c i t Enumeration

Program f o r Zero-On e I n t e g e r Programming," R e p o rt No, 3 05 , D e p t, o f Computer S c i e n c e , U n i v e r s i t y o f I l l i n o i s , Urbana, I l l i n o i s , January 6,

(31)

L. G. Hanscom Field

Bedford, Mass. 01731 2 copies

Naval Air Systems Command AIR 03

Washington, D. C. 20360 2 copies

Commanding General

U.S. Army Electronics Command Fort Monmouth, New Jersey 07703 Attn: AMSEL-RD-PB (2 copies)

(Miss F. Morris)

Naval Electronic Systems Command ELEX 03, Room 2046 Munitions Building Department of the Navy

Washington, D. C. 20360 2 copies

Director

Naval Research Laboratory Washington, D. C. 20390 Washinton, D. C. 2027 6 copies

Commander

U.S. Naval Ordnance Laboratory Attn: Librarian

White Oak, Md. 20910 2 copies

Dr. L. A. Wood, Director

Electronic and Solid State Sciences Air Force Office of Scientific Research 1400 Wilson Boulevard

Arlington, Virginia 22209 5 copies

Director, Electronic Programs Attn: Code 427

Office of Naval Research 800 North Quincy Street

Arlington, Virginia 22217 3 copies

Defense Documentation Center Attn: DDC-TCA

Cameron Station

Alexandria, Virginia 222314 50 copies

Attn: STEWS-RE-L, Technical Library White Sands Missile Range

New Mexico 88002 (2 copies)

Commander

Naval Electronics Laboratory Center Attn: Library

San Diego, Calif. 92152 2 copies

Dr. L. M. Hollingsworth AFCRL (CRN)

L. G. Hanscom Field Bedford, Massachusetts 01731

Division of Engineering and Applied Physics

210 Pierce Hall Harvard University

Cambridge, Massachusetts 02138

Director

Research Laboratory of Electronics Massachusetts Institute of Technology Cambridge, Massachusetts 02139

Miss R. Joyce Harman Project MAC, Room 810 545 Technology Square Cambridge, Mass. 02139

Professor R. H. Rediker Electrical Engineering, Prof. Mass. Institute of Technology Building 13-3050

Cambridge, Mass. 02139

Research Division Library 28 Seyon Street

Waltham, Massachusetts 02154

Sylvania Electronic Systems Applied Research Laboratory Attn: Documents Librarian 40 Sylvan Road

Waltham, Mass. 02154

Commanding Officer

Army Materials & Mechanics Res. Center Attn: Dr. H. Priest

Watertown Arsenal

Watertown, Massachusetts 02172

MIT Lincoln Laboratory Attn: Library A-082 P.0. Box 73

Lexington, Mass. 02173

Office of Naval Research Branch Office 495 Summer Street

Boston, Massachusetts 02210

Commanding Officer (Code 2064) U.S. Naval Underwater Sound Laboratory Fort Trumbull

New London, Connecticut 06320

Dept, of Eng. & Applied Science Yale University

New Haven, Conn. 06520

U.S. Army Electronics Command Attn: AMSEL-CT-A

Fort Monmouth, New Jersey 07703

U.S. Army Electronics Command Attn: AMSEL-CT-D

U.S. Army Electronics Command Attn: AMSEL-CT-I

U.S. Army Electronics Command Attn: AMSEL-CT-L (Dr. W. S. McAfee) Fort Monmouth, New Jersey 07703

U.S. Army Electronics Command Attn: AMSEL-CT-0

U.S. Army Electronics Command Attn: AMSEL-CT-R

U.S. Army Electronics Command Fort Monmouth, New Jersey 07703 Attn: AMSEL-CT-S

U.S. Army Electronics Command Attn: AMSEL-DL

U.S. Army Electronics Command Attn: AMSEL-CC-DD

Fort Monmouth, N. J. 07703

U.S. Army Electronics Command Attn: AMSEL-KL-D

U.S. Army Electronics Command Attn: AMSEL-KL-E

Fort Monmouth, New Jersey 07703

U.S. Army Electronics Command Attn: AMSEL-KL-I

U.S. Army Electronics Command Attn: AMSEL-KL-SM

U.S. Army Electronics Command Attn: AMSEL-KL-S

U.S. Army Electronics Command Attn: AMSEL-KL-T

U.S. Army Electronics Command Attn: AMSEL-NL-A

U.S. Army Electronics Command Attn: AMSEL-NL-C

U.S. Army Electronics Command Attn: AMSEL-NL-D (Dr. H. Bennett) Fort Monmouth, New Jersey 07703

U.S. Army Electronics Command Attn: AMSEL-NL-P

U.S. Army Electronics Command Attn: AMSEL-SC

U.S. Army Electronics Command Attn: AMSEL-VL-D

U.S. Army Electronics Command Attn: AMSEL-VL-F

U.S. Army Electronics Command Attn: AMSEL-WL-D

U.S. Army Electronics Command Attn: AMSEL-XL-DT

U.S. Army Electronics Command Attn: AMSEL-XL-D

(32)

Project Manager

Common Positioning & Navigation Systems Attn: Harold H. Bahr (AMCPM-NS-TM),

Bldg. 439

U.S. Army Electronics Command Fort Monmouth, New Jersey 07703

Dr. H. K. Ziegler, Chief Scientist Army Member TAC/JSEP (AMSEL-SC) U.S. Army Electronics Command Fort Monmouth, New Jersey 07703

Mr. X. A. Balton, AMSEL-XL-D Executive Secretary, TAC/JSEP U.S. Army Electronics Command Fort Monmouth, New Jersey 07703

U.S. Army Electronics Command Attn: AMSEL-XL-G (Dr. S. Kronenberg) Fort Monmouth, New Jersey 07703

U.S. Army Electronics Command Attn: AMSEL -XL-H (Dr. R. G. Buser) Fort Monmouth, New Jersey 07703

U.S. Army Munitions Command Attn: Science & Technology Info. B r ., Bldg. 59

Picatinny Arsenal, SMUPA-RT-S Dover, New Jersey 07801

European Office of Aerospace Research Technical Information Office Box 14, FPO New York 09510

Director

Columbia Radiation Laboratory Columbia University

538 West 120th St. New York, New York 10027

New York University Engineering Library Bronx, New York 10453

Mr. Jerome Fox, Research Coordinator Polytechnic Institute of Brooklyn 333 Jay St.

Brooklyn, New York 11201

Airborne Instruments Labroratory Deerpark, New York 11729

Dr. W. R. Lepage, Chairman Syracuse University

Dept. Of Electrical Engineering Syracuse, New York 13210

Rome Air Development Center Attn: Documents Library (EMTLD) Griffiss Air Force Base, New York 13440

Mr. H. E. Webb (EMBIS) Rome Air Development Center

Griffiss Air Force Base, New York 13440

Professor James A Cadzow

Department of Electrical Engineering State University of New York at Buffalo Buffalo, New York 14214

Dr. A. G. Jordan

Head of Dept, of Electrical Engineering Carnegie-Mellon University

Pittsburgh, Penn. 15213

Lehigh University

Department of Electrical Engineering Bethelehem, Penn. 18015

Commander (ADL)

Naval Air Development Center Attn: NADC Library

Johnsville, Warminister, Penn. 18974

Technical Director (SMUFA-A2000-107-1) Frankford Arsenal

Philadelphia, Penn. 19137

Mr. M. Zane Thornton, Chief, Network Engineering, Communications and

Operations Branch, Lister Hill National Center/Biomedical Communications 8600 Rockville Pike

Bethesda, Maryland 20014

U.S. Post Office Department Library - Room 6012

12th & Pennsylvania Ave. N.W. Washington, D.C. 20260

Technical Library DDR&E

Room 3C-122, The Pentagon Washington, D.C. 20301

Director for Materials Sciences Advanced Research Projects Agency Department of Defense

Washington, D.C. 20301

Assistant Director, (Research) Office of Director of Defense Research

& Engineering Pentagon, Rm. 3C128 Washington, D.C. 20301

Chief, R & D Division (340) Defense Communications Agency Washington, D.C. 20305

Commanding General U. S. Army Material Command Attn: AMCRD-TP Washington, D.C. 20315 Hq. USAF (AFRDD) The Pentagon Washington, D.C. 20330 Hq. USAF (AFRDDG) The Pentagon Washington, D.C. 20330 Hq. USAF (AFRDSD) The Pentagon Washington, D.C. 20330 AFSC (SCTSE)

Andrews Air Force Base, Maryland 20331

Dr. I. R. Mirman Hq. AFSC (SGGP)

Andrews AFB, Maryland 20331

Naval Ship Systems Command Ship 031

Naval Ship Systems Command Ship 035

U.S. Naval Oceanographic Office Attn: M. Rogofsky, Librarian (Code 640) Washington, D.C. 20390

Director

Naval Research Laboratory Washington, D.C. 20390

Attn: Dr. A. Brodizinsky, Sup. Elec. Div.

Director

Naval Research Laboratory Washington, D.C. 20390

Attn: Maury Center Library (Code 8050)

Director

Naval Research Laboratory Washington, D.C. 20390 Attn: Dr. W. C. Hall, Code 7000

Director

Naval Research Laboratory Attn: Library, Code 2029 (ONRL) Washington, D.C. 20390

Dr. G. M. R. Winkler

Director, Time Service Division U.S. Naval Observatory Washington, D.C. 20390 Colonel E. P. Gaines, Jr. ACDA/FO 1901 Pennsylvania Ave. N.W. Washington, D.C. 20451 Commanding Officer Harry Diamond Laboratories

Attn: Mr. Berthold Altman (AMXDO-TI) Connecticut Ave. & Van Ness St., N.W. Washington, D.C. 20438

Central Intelligence Agency Attn: CRS/ADD Publications Washington, D.C. 20505

Dr. H. Harrison, Code RRE Chief, Electrophysics Branch National Aeronautics & Space Admin. Washington, D.C. 20546

The John Hopkins University Applied Physics Laboratory Attn: Document Librarian 8621 Georgia Avenue

Silver Spring, Maryland 20910

Commanding Officer (AMXRD-BAT) U.S. Army Ballistics Research

Laboratory

Aberdeen Proving Ground Aberdeen, Maryland 21005

Technical Director

U.S. Army Land Warfare Laboratory Aberdeen Proving Ground

Aberdeen, Maryland 21005

Electromagnetic Compatibility Analysis Center (ECAC) A t t n : ACOAT

North Severn

Annapolis, Maryland 21402

U.S. Army Engineer Topographic Labs Attn: STINFLO Center

Fort Belvoir, Virginia 22060

Director (NV-D)

Night Vision Laboratory, USAECOM Fort Belvoir, Va. 22060

(33)

Fort Belvoir, Virginia 22060

Dr. Alvin D. Schnitzler Institute for Defense Analyses Science and Technology Division 400 Army-Navy Drive

Arlington, Va. 22202

Director

Physical & Engineering Sciences Division 3045 Columbia Pike

Arlington, Va. 22204

Commanding General U.S. Army Security Agency Attn: IARD-T

Arlington Hall Station Arlington, Virginia 22212

Dr. Joel Trimble, Code 437 Information Systems Branch Office of Naval Research 800 North Quincy Street Arlington, Virginia 22217

USACDC Institute of Land Combat Attn: Technical Library, Rm. 636 2461 Eisenhower Avenue

Alexandria, Virginia 22314

Director

USA Advanced Material Concepts Agency 2461 Eisenhower Avenue

Alexandria, Virginia 22314

VELA Seismological Center 300 North Washington St. Alexandria, Virginia 22314

U.S. Naval Weapons Laboratory Dahlgren, Virginia 22448

Research Laboratories for the Eng. Sciences, School of Engineering and Applied Science

University of Virginia Charlottesville, Virginia 22903

Dr. HermannRobl Deputy Chief Scientist

U.S. Army Research Office (Durham) Box CM, Duke Station

Durham, North Carolina 27706

Richard 0. Ulsh (CRDARD-IP) U.S. Army Research Office (Durham) Box CM, Duke Station

Durham, North Carolina 27706

ADTC (ADBPS-12)

Eglin AFB, Florida 32542

Naval Training Device Center Orlando, Florida 32813

Technical Library, AFETR (ETV.MU-135)

Patrick AFB, Florida 32925

Commanding General U.S. Army Missile Command Attn: AMSMI-RR

Redstone Arsenal, Alabama 35809

Redstone Scientific Information Center Attn: Chief, Document Section U.S. Army Missile Command Redstone Arsenal, Alabama 35809

Hq. AEDC (AETS) Attn: Library/Documents Arnold AFB, Tennessee 37389

Case Institute of Technology Engineering Division University Circle Cleveland, Ohio 44106

NASA Lewis Research Center Attn: Library

21000 Brookpark Road Cleveland, Ohio 44135

Director

Air Force Avionics Laboratory Wright-Patterson AFB, Ohio 45433

AFAL (AVT) Dr. H. V. Noble, Chief Electronics Technology Division Air Force Avionics Laboratory Wright-Patterson AFB, Ohio 45433

Dr. Robert E. Fontana

Head, Dept. of Electrical Engineering Air Force Institute of Technology Wright-Patterson AFB, Ohio 45433

AF A L /TEA

Wright-Patterson Air Force Base, Ohio 45433

Department of Electrical Engineering Clippinger Laboratory

Ohio University Athens, Ohio 45701

Commanding Officer Naval Avionics Facility Indianapolis, Indiana 46241

Dr. John C. Hancock, Head School of Electrical Engineering Purdue University

Lafayette, Indiana 47907

Professor Joseph E. Rowe

Chairman, Department of Electrical Engineering

The University of Michigan Ann Arbor, Michigan 48104

Dr. G. J. Murphy

The Technological Institute Northwestern University Evanston, 111. 60201

Office of Naval Research Branch Office 219 South Dearborn St.

Chicago, Illinois 60604

Illinois Institute of Technology Dept, of Electrical Engineering Chicago, Illinois 60616

Deputy for Res. and Eng. (AMSE-DRE) U.S. Army Weapons Command

Rock Island Arsenal Rock Island, Illinois 61201

Commandant

U.S. Army Command & General Staff College

Attn: Acquisitions, Library Division Fort Leavenworth, Kansas 66027

Department of Electrical Engineering Rice University

Houston, Texas 77001

HQ AMD (AMR)

Brooks AFB, Texas 78235

USAFSAM (SMKOR) Brooks AFB, Texas 78235

Mr. B. R. Locke

Technical Adviser, Requirements USAF Security Service

Kelly Air Force Base, Texas 78241

Director

Electronics Research Center The University of Texas at Austin Eng.-Science Bldg. 110

Austin, Texas 78712

Department of Electrical Engineering Texas Technological University Lubbock, Texas 79409

Commandant

U.S. Army Air Defense School

Attn: Missile Sciences D i v . , C&S Dept. P.0. Box 9390

Fort Bliss, Texas 79916

Director

Aerospace Mechanics Sciences

Frank J. Seiler Research Laboratory (OAR) USAF Academy

Colorado Springs, Colorado 80840

Director of Faculty Research Department of the Air Force U.S. Air Force Academy

Major Richard J. Gowen Tenure Associate Professor Dept, of Electrical Engineering U.S. Air Force Academy

Colorado Springs, Colorado 80840

Academy Library (DFSLB) U.S. Air Force Academy

M. A. Rothenberg (STEPD-SC(S)) Scientific Director

Desert Test Center Bldg. 100, Soldiers' Circle Fort Douglas, Utah 84113

Utah State University

Dept, of Electrical Engineering Logan, Utah 84321

School of Engineering Sciences Arizona State University Tempe, Arizona 85281

U.S. Army Strategic Communications Command

Attn: SCC-CG-SAE

Fort Huachuca, Arizona 85613

The University of Arizona Dept, of Electrical Engineering Tucson, Arizona 85721

Capt. C. E. Baum AFWL (WLRE)

Kirkland AFB, New Mexico 87117

Los Alamos Scientific Laboratory Attn: Report Library

P.0. Box 1663

(34)

Commanding Officer (AMSEL-BL-WS-R)

Atmospheric Sciences Laboratory White Sands Missile Range, New Mexico 88002

Chief, Missile Electronic Warfare Tech. Area

(AMSEL-WL-M)

Electronic Warfare Laboratory, USAFCOM White Sands Missile Range, N.M. 88002

Director

Electronic Sciences Lab. University of Southern California Los Angeles, Calif. 90007

Engineering & Mathematical Sciences Library

University of California at Los Angeles 405 Hilgred Avenue

Los Angeles, Calif. 90024

Aerospace Corporation P.O. Box 95085

Los Angeles, California 90045 Attn: Library Acquisitions Group

HQ SAMSO (SMTTA/Lt. Belate) AF Unit Post Office Los Angeles, Calif. 90045

Dr. Sheldon J. Wells

Electronic Properties Information Center Mail Station E-175

Hughes Aircraft Company Culver City, California 90230

Director, USAF PROJECT RAND Via: Air Force Liaison Office The RAND Corporation

Attn: Library D 1700 Main Street

Santa Monica, Calif. 90406

Deputy Director and Chief Scientist Office of Naval Research Branch Office 1030 East Green Street

Pasadena, California 91101

Point Mugu, California 93041

Sciences-Engineering Library University of California Santa Barbara, California 93106

Commander (Code 753) Naval Weapons Center Attn: Technical Library China Lake, California 93555

Library (Code 2124) Technical Report Section Naval Postgraduate School Monterey, California 93940

Glen A. Myers (Code 52Mv)

Assoc. Professor of Elec. Engineering Naval Postgraduate School

Monterey, California 93940

Dr. Leo Young

Stanford Research Institute Menlo Park, Calif. 94025

Lenkurt Electric Co., Inc. 1105 County Road

San Carlos, California 94070 Attn: Mr. E. K. Peterson Director Microwave Laboratory Stanford University Stanford, California 94305 Director

Stanford Electronics Laboratories Stanford University

Stanford, California 94305

Director, Electronics Research Laboratory

University of California Berkeley, California 94720

Aeronautics Library

Graduate Aeronautical Laboratories California Institute of Technology 1201 E. California Blvd.

Pasadena, California 91109

Professor Nicholas George California Institute of Technology Pasadena, California 91109

Commanding Officer Naval Weapons Center Corona Laboratories Attn: Library

Corona, California 91720

Dr. F. R. Charvat Union Carbide Corporation

Materials Systems Div., Crystal Products Dept., 8888 Balboa Avenue

P.O. Box 23017 San Diego, Calif. 92123

Hollander Associates P.O. Box 2276

Fullerton, California 92633

Commander, U.S. Naval Missile Center (56322) Point Mugu, California 93041

On Some Probabilistic Decoding Algorithms

S

-

? * COORDINATED SCIEN CE LABORATORY

ON SOME PROBABILISTIC

DECODING ALGORITHMS

A.H. HADDAD

U N IVERSITY OF ILLIN O IS - URBANA, ILLIN O IS

e

s_

e

e

e

e

Z

e

%

g-^

L

L Z

k

gl

gl

+ 2

. ) p. . n

P ) ] }

l <

i

t

9

9

vi

i

o *

l

-&

-

i

i,

e

%R

s_

2

0

l

e

e

£

£

£

*) *

:

])

m

1+ £f>

(if) » 1

i

i

V 1 = V "-1

JL

2

**? * COORDINATED SCIEN CE LABORATORY**

₉

_{o }*

)