Analog Neural Circuit by AC Operation and the Design of Deep Learning Model

) 7 1 0 2 E I I A ( g n ir e e n i g n E l a ir t s u d n I d n a e c n e g il l e t n I l a i c if it r A n o e c n e r e f n o C l a n o it a n r e t n I d r 3 7 1 0 2

8 7 9 : N B S

I -1-60595- 05 -9 2

l

a

n

A

o

g

N

e

u

r

a

l

C

ri

c

u

i

t

b

y

A

C

O

p

e

r

a

it

o

n

a

n

d

t

h

e

D

e

s

i

g

n

o

f

l

e

d

o

M

g

n

i

n

r

a

e

L

p

e

e

D

K

i

h

s

a

s

a

M

A

W

A

G

U

C

H

I

,

M

a

s

a

y

o

s

h

i

U

M

E

N

O

a

n

d

N

a

o

h

ri

o

I

S

H

I

I

1_{DepartmentofElectrica l&ElectronicEngineeringNaitona lInsttiuteofTechnology,}

0 1 5 e i M a k u z u S , o k o ri h S e g e ll o C a k u z u

S -0294Japan

2_{DepartmentofElectronicEngineeringChubuUniverstiy1200Matsumoto-cho,}

7 8 4 i h c i A ,i a g u s a

K -8501Japan

3_{DepartmentofInformaitonScienceAich iInsttiuteofTechnologyYachigusa,}

a s u g a

Y -cho,Toyota,470-0392Japan

: s d r o w y e

K Electroniccricuti, Neura lnetwork,Deepl earning.

.t c a r t s b

A Int heneura lnetworkfield ,manyapplicationmodelshavebeenproposed .Previousanalog t l u c i f f i d s i t I . e c n a t s i s e r d e x i f d n a r e i f i l p m a l a n o i t a r e p o e h t f o d e s o p m o c e r e w s l e d o m k r o w t e n l a r u e n

i t c e n n o c e h t e g n a h c o

t ngweigh tofnetwork .Int hisstudy ,weusedanalogelectronicA Ccircuits .The e

h t e b i r c s e d s t h g i e w g n i t c e n n o

c voltageandi npu tsigna ldescribet hefrequency .Iti seasyt ochange r

i c c i n o r t c e l e g o l a n a n o y l n o s k r o w l e d o m s i h T . t n e i c i f f e o c n o i t c e n n o c e h

t cuits .I tcan finish the

e h t , r e v e w o H . g n i n r a e l e l b i x e l f e r o m e l b a n e l l i w l e d o m s i h t d n a e m i t t r o h s y r e v a n i s s e c o r p g n i n r a e l

d n a t i n u f o r e b m u n e h t d e v o r p m i e W . k r o w t e n t u p t u o e n o d n a t u p n i e n o y l n o s i l e d o m s i h t f o e r u t c u r t s

e w , r e v o e r o M . r e y a l k r o w t e

n suggestt hepossibilityofr ealizationaboutt hehardwarei mplementation .l

e d o m g n i n r a e l p e e d e h t f o

n o it c u d o r t n I

y b k r o w t e n l a r u e n e h t f o g n i n r a e l c i m a n y d e h t e s o p o r p e

W ACoperationanalogelectroniccircuits . e

c i v e d l a n g i s w e n a p o l e v e d l l i w l e d o m s i h

T with the analog neura lelectronic circuit .Oneofthe , k r o w t e n l a r u e n f o d l e i f e h t n I . n o i t c n u f l a r u e n l a c i d e m o i b f o g n i l l e d o m e h t s i h c r a e s e r s i h t f o s t e g r a t

n e e b e v a h t a h t s l e d o m e r a w d r a h y n a m e r a e r e h t d n A . d e s o p o r p n e e b e v a h s l e d o m n o i t a c i l p p a y n a m

a e

r lized .Theseanalogneura lnetworkmodelswerecomposedoft heoperationa lamplifierandfixed .t

n e i c i f f e o c n o i t c e n n o c e h t e g n a h c o t t l u c i f f i d s i t I . e c n a t s i s e r

k r o w t e N l a r u e N g o l a n A

c a y b e g r a h c r o t n e r r u c , e g a t l o v e h t s e s s e r p x e k r o w t e n l a r u e n g o l a n a e h

T ontinuousquantity .The

k c o l c e h t y b m e t s y s e m i t e t e r c s i d a s a l l e w s a m e t s y s e m i t s u o u n i t n o c a t c u r t s n o c n a c t i s i t i r e m n i a m

n a e s u o t e l b a t i u s s i t I . g o l a n a s e z i l i t u l l e c n o r u e n l a u t c a e h t f o n o i t a r e p o e h t , y l s u o i v b O . n o i t a r e p o

i t a t i m i r o f d o h t e m g o l a n

a ngtheoperation ofan actua lneuron cell .ManyArtificia lneura lnetworks e l g n i s a n o d e l l a t s n i e b n a c s t i n u g n i s s e c o r p y n a M . d o h t e m g o l a n a e h t y b d e n g i s e d e r e w I S

L -chip ,

l p i t l u m , n o i t i d d a , s t n e m e l e f o r e b m u n l l a m s a h t i w d e v e i h c a e b n a c t i n u h c a e e s u a c e

b ication ,andt he

, t l u s e r a s A . n o i t a l u c l a c l e l l a r a p r e p u s e h t g n i s u e t a r e p o o t e l b i s s o p s i t i d n A . n o i t a m r o f s n a r t r a e n i l n o n

h g i h e h

t -speedoffersanadvantagecomparedt ot hedigitalneura lnetworkmethod[1][2] .

w e i v r e v O

k r o w t e n l a r u e n e h t f o s t l u s e r e h

i n u d n a t h g i e w g n i t c e n n o c e h t f o d e s o p m o c s i r e y a l h c a

E t .Aneura lnetwork iscomposed ofthose n i b m o c y b s r e y a l e e r h

t ingt heneuronstructures.

n e e b e v a h s l e d o m e r a w d r a h d n a s d o h t e m n o i t a c i l p p a y n a m , k r o w t e n l a r u e n f o d l e i f e h t n I e s i r p m o c o t d e p o l e v e d e r a p i h c a n i t e r l a i c i f i t r a n a d n a p i h c o r u e n A . d e s o p o r

p the neura lnetwork

e h t f o t i u c r i c e h t g n i d d a e r a e w , h c r a e s e r s i h t n I . m e t s y s n o i s i v l a c i d e m o i b e h t e t a l u m i s d n a l e d o m e h t n I . s t i u c r i c g n i d d a f o e g a t l o v t u p n i e h t s w o h s t h g i e w g n i t c e n n o c e h T . r e i f i l p m a l a n o i t a r e p o t e n l a r u e n f o s l e d o m e r a w d r a h s u o i v e r

p - ow rk ,changing connected weights was difficult ,because . s t h g i e w g n i t c e n n o c e h t s a s t n e m e l e e c n a t s i s e r e h t d e s u s l e d o m e s e h t , r e v e w o H . d e s o p o r p s a w s t h g i e w g n i t c e n n o c e h t s a r o t i c a p a c e h t d e s u h c i h w l e d o m e h t , r e v o e r o M n i t c e n n o c e h t t s u j d a o t t l u c i f f i d s i t

i g weights .In thepresen tstudy ,weproposed aneura lnetwork . s t i u c r i c e l p i t l u m f o e g a t l o v a s a n w o h s e r a s t h g i e w g n i t c e n n o c e h T . s t i u c r i c e l p i t l u m g o l a n a g n i s u w t s r i f t A . r e k c i u q e b l l i w s s e c o r p g n i n r a e l e h T . y l i s a e d e g n a h c e b n a c s t h g i e w g n i t c e n n o c e h

T emade

e h t s n a e m E C I P S . n o i t a l u m i s E C I P S y b t i u c r i c l a r u e n d n a m a r g o r p r e t u p m o c y b k r o w t e n l a r u e n a n o i t a m r i f n o c r o i v a h e b e h t d e r u s a e m e w t x e N . r e t p a h c t x e n e h t n i n w o h s s a r o t a l u m i s t i u c r i c c i r t c e l E m o c e W . n o i t a l u m i s E C I P S d n a n o i t a l u c l a c r e t u p m o c e h t f

o paredbothoutpu tresultsandconfirmed X E f o t n e t x e e m o

s -ORbehavior.

SPICE

e h t s i ) E C I P S ( r o t a l u m i s t i u c r i c c i r t c e l E . E C I P S r o t a l u m i s t i u c r i c c i r t c e l e e h t d e s u e w , h c r a e s e r s i h t n I a c t I . s i s a h p m E e l c r i C d e t a r g e t n I h t i w m a r g o r P n o i t a l u m i S f o n o i t a i v e r b b

a n reproducethe analog

e h t t e s , D A C y b n w a r d t i u c r i c e h t , s i h t r e t f A . t i u c r i c c i r t c e l e e h t d n a t i u c r i c l a c i r t c e l e n a f o n o i t a r e p o e h t e d a m e w , t s r i f t A . s i s y l a n a t n e i s n a r t d n a C D , C A f o n o i t c n u f e h t s a h E C I P S . e g a t l o v t u p n i s t i u c r i c r e i f i l p m a l a i t n e r e f f i

d andGilber tmultiplierscircuits .Andweconfirmedt herangeofvoltage l a n o i t a r e p o n a y b s t i u c r i c e l p i t l u m f o d e s o p m o c s a w e r u t c u r t s n o r u e n e h T . y l t n e l l e c x e d e t a r e p o n o n e v e i h c a o t s t i u c r i c r o r r i m t n e r r u c , t n e m e v e i h c a n o i t c n u f n o i t a c i l p i t l u m r o f r e i f i l p m

a linear

. s t i u c r i c r e i f i l p m a l a i t n e r e f f i d d n a n o i t c n u f g n i t c e n n o c a s a t n e m e l e e c n a t s i s e r e h t d e s u e w , k r o w t e n l a r u e n f o l e d o m e r a w d r a h s u o i v e r p e h t n I s e t a l u c l a c t i , n o i t c e n n o c l a r u e n e h t n I . e u l a v e c n a t s i s e r e h t e g n a h c o t t l u c i f f i d s i t i , r e v e w o H . t h g i e w g n i t c e n n o c e h t s a t i u c r i c e l p i t l u m e h t d e s u e W . t h g i e w g n i t c e n n o c d n a e u l a v t u p n i e h t t c u d o r p e h t e h T . t h g i e w g n i t c e n n o c d n a e u l a v t u p n i n a s n a e m s t i u c r i c e l p i t l u m f o s t u p n i o w t h c a E . t h g i e w t e g n a h c o t y s a e s i t I . e u l a v e g a t l o v e h t s w o h s t h g i e w g n i t c e n n o

c hevaluein thelearning stage of

. k r o w t e n l a r u e n f o c i t s i r e t c a r a h c e h t s e c u d o r p e r h c i h w t u p t u o e n o d n a s t u p n i o w t f o t i u c r i c l a r u e n e h t s i 1 e r u g i F d n a l a n g i s t u p n i e h t f o t c u d o r p e h t , s t i u c r i c r o r r i m t n e r r u c y b n o i t i d d a t n e r r u c g n i s u , n o r u e n e n o i t c e n n o

1 1 V

v 5 . 0

+

3

-2

+

V

7

-V

4

T U

O 6

1 S

O 1

2 S

O 5

3 U 1 4 7 A u

0 1 R

0 0 1

0 0

0 0

2 1 V

v 2 1

3 1 V

V 2 1

0 B 1 J

7 6 5 5 N D

4 1 V

c d V 1 . 0

5 1 V

v 5 . 0

-+

3

-2

+

V

7

-V

4

T U

O 6

1 S

O 1

2 S

O 5

4 U 1 4 7 A u

2 1 R

0 0 1

0 0

0

0 6 1 V

v 2 1

7 1 V

V 2 1

0 B 2 J

7 6 5 5 N D

8 1 V

c d V 1 . 0

+

3

-2

+

V

7

-V

4

T U

O 6

1 S

O 1

2 S

O 5

1 U 1 4 7 A u

0 0 2 V

v 2 1

3 V

V 2 1

4 R

0 0 1 0

5 R

k 1 . 0

6 R

k 1 . 0 0

1 1 R

0 0 1

0 3 1 R

0 0 1

0 5 1 R

0 0 1

3 R

0 0 1

V V

V

4 1 R

k 1 . 0 9

V

c d V 0

+

3

-2

+

V

7

-V

4

T U

O 6

1 S

O 1

2 S

O 5

2 U 1 4 7 A u

8 R

0 0 1

0 0

0 0

4 V

v 2 1

5 V

V 2 1

0 A 1 J

7 6 5 5 N D

0 1 V

c d V 1 . 0

0

g i

F u 1 .re Analogneura lnetworkbymultiplecircuit.

g i

F u 2 .re ACOperationNeura lCircuit.

i C l a r u e N n o it a r e p O C

A r tic u

a d e s o p o r p e

W nalogneura lnetworki nt hepreviousresearch ][ .3 However ,theworkingrangei svery .

s c i t s i r e t c a r a h c r o t c u d n o c i m e s e h t f o e s u a c e b l l a m

s We tried to use the alternative curren tin the o

l a n

a gneura lnetwork inFigure2 .Thealternativecurren thas twoelements ,voltageandf requency. In . t h g i e w g n i t c e n n o c d n a l a n g i s t u p n i e h t e t a l u c l a c o t t i u c r i c e l p i t l u m e h t e s u e w , l e d o m s u o i v e r p e h t

, r e v e w o

H thisACcircuitscancalculatet heproductsvoltageandf requencybyCRcircuits.Figure3 is z H k 3 m o r f e g n a r y c n e u q e r f e h t n i y l i r o t c a f s i t a s s e t a r e p o t I . t i u c r i c l a r u e n n o i t a r e p o C A f o t u p t u o e h t

l e d o M g n i n r a e L l a c i m a n y D

m a n y d e h t e s o p o r p e

W ica llearning mode lusing apureanalog electroniccircuit .Weused analog k c a b d e e f g o l a n a d e s u e w , e g a t s g n i n r a e l e h t n I . r e t p a h c s u o i v e r p a n i d e n i a l p x e , k r o w t e n l a r u e n

n i n r a e l e m i t l a e R . l a n g i s g n i h c a e t h c a e f o k r o w t e n l a r u e n e t a r a p e s a e s u e W . s t i u c r i

c gi spossible .We

g n i k r o w e h t n I . s t h g i e w n o i t c e n n o c e h t d l o h n a c t I . e g a t s g n i k r o w e h t n i t i u c r i c d l o h e l p m a s e h t d e s u

a e l , k r o w l a c i d o i r e p m r o f r e p n a c t i u c r i c s i h T . g n i k r o w s i k r o w t e n l a r u e n s i h t , e g a t

s rning modeand

e d o m g n i k r o

w .

e h t , d n a h r e h t o e h t n

O pulsedneura lNetworkhasanadvantage .Particularly,t hisnetworkcanalso g n i n r a e l e h t r o f e m i t g n o l a s e k a t t i , r e v e w o H . s s e c o r p g n i n r a e l e h t r e t f a s t h g i e w g n i t c e n n o c e h t p e e k

b a , l e d o m n o r u e n d e s l u p l a c i p y t e h t s A . d e r i u q e r e r a s e s l u p y n a m n e h w s s e c o r

p ou t1000pulseswere

l a c i r t c e l e p a e h c a h t i w d e t c u r t s n o c s i l e d o m d e s o p o r p r u o , r e v e w o H . s s e c o r p g n i n r a e l e h t r o f d e r i u q e r

e r o m d e v o r p m i e b l l i w d e e p s g n i n r a e l e h t , e c i v e d l a c i r t c e l e g o l a n a y t i l a u q h g i h e h t e s u e w f I . e c i v e d

h t n I . l e d o m n o r u e n d e s l u p n a h

t eresul tofthisexperimen ttheperformanceislow becauseofusing l

a r e n e

g -purpose , inexpensive parts . The operating speed wil l be improved by using a h

g i

h -performanceelemen twhichhasagoodslewrate .However,t hissystemi sasimplecircuit .The e

b m u

n rofpartsi sfew .Thecos twil lno trisemucheveni fgoodperformancepartsareused.

l e d o M g n i n r a e L p e e D

g n i n r a e l n i s m h t i r o g l a f o d n i k a s i g n i n r a e l p e e D . d e s o p o r p n e e b s a h l e d o m g n i n r a e l p e e d a , y l t n e c e R

h g i h e h t s t p m e t t a t I . l e d o

m -leve lcategorizingofdatausing multiplenon-lineartransformationsand p e e d e h t , n o i t i n g o c e r h c e e p s d n a n o i t i n g o c e r e g a m i f o d l e i f e h t n I . g n i n r a e l e n i h c a m f o d o h t e m e n o

n o i t n e t t a e h t d e t c a r t t a s a h d o h t e m g n i n r a e

l ][ . 4

T h Se tackedAu Eto ncoder

o t u a d e k c a t s e h

T -encoderisonemethod ofdeep learning .Thisisthepre-learning method oflarge .

s w o l l o f s a s i k r o w t e n r e y a l p e e d e h t t c u r t s n o c o t w o H . k r o w t e n r e y a l r e b m u n

o t u a d e k c a t s f o s s e c o r p g n i n r a e l e h t r e t f

A -encoderi scompleted ,removet hedecodingpar t(outpu t s

f o ) r e y a

l tacked auto-encoderandkeepthecodedportion(from theinpu tlayertotheintermediate )

r e y a

l showni nFigure4 .Thusweobtaint henetworkwhichconvertsf romi npu tsignalt ocompressed n

o i t a t n e s e r p e r n o i t a m r o f n

i showni nFigure5 . n

i a t b o e w , r e v o e r o

M morecompressed interna lrepresentation ,asthecompressed representation o

t u a e h t y l p p a o t l a n g i s t u p n

i -encoderl earning .Thus ,weobtainamulti-layeredhierarchica lnetwork , o

t u a d e t a e p e r y l e v i s r u c e

r -encoder learning ,and stacked the encoding par tof the network .This o

t u a d e k c a t s d e l l a c s i k r o w t e n r e y a l i t l u m d e t c u r t s n o

c -encoder .In this way , after building a i

t l u

m -layernetwork,t oaddt hei dentifiednetworkusingt heoutpu toft hef inall ayer ,anewsupervised .

d e s o p o r p s i d o h t e m g n i n r a e l

g i

g i

F u 5 .re TheCompressedInterna lRepresentation.

o t u a d e k c a t

S -encoderhasbeenappliedt ot hevarioussubjec taswel last heDNNwhichi sstacked e

h

t RBM .Recently ,i tbecame famous the learning experimen tof feature extractor from a large . e g a m i f o t n u o m a s i h t , r e v e w o H . l e d o m g n i n r a e l k r o w t e n l a r u e n l a c i m a n y d e h t d e b i r c s e d e w , h c r a e s e r s u o i v e r p e h t n I .t i n u t u p t u o e n o d n a t i n u t u p n i e n o y l n o s a h l e d o

m Torealizet hehardwaredeepl earningmodel ,we 2 d n a t u p t u o 1 , t u p n i 2 a d e t c u r t s n o c e w , t x e N . r e y a l h c a e n i s t i n u f o r e b m u n e h t e s a e r c n i o t e v a h l e d o m l a r u e n s n r e t t a p . n o is u l c n o C e e r h t a d e t c u r t s n o c e

W -layer neura lnetwork ,two-inpu tlayers ,two-middle layers and oneoutpu t g n i y l p i t l u m e h t h t i w k r o w t e n l a r u e n g o l a n a r e y a l e e r h t e h t f o n o i t a r e p o e h t d e m r i f n o c e W . r e y a l . n o i t a l u m i s E C I P S y b t i u c r i

c Theconnectionweigh tcanbechangedbycontrollingt hei npu tvoltage . l i b i x e l f h g i h y l e m e r t x e s a h l e d o m s i h

T ity characteristics .When the analog neura lnetwork is l a r u e n s i h t o t t h g i e w e s p a n y s e h t e v i g o t w o h s i t I .t n a t r o p m i y l l a i c e p s e s i t h g i e w e s p a n y s e h t , d e t a r e p o l u r n o i t a g a p o r p k c a b e h t f o d o h t e m e h t y l p p a o t y r a s s e c e n s i t i , m e l b o r p s i h t e v l o s o T . k r o w t e

n et hati s

e h t r o f e l u r g n i n r a e l l a r e n e g

a neura lnetworks .Thisneura lcircui tmodeli spossiblet hel earning .The e h t g n i t a l u c l a c s i d o h t e m e h T . d e z i l a e r e b l l i w g n i n r a e l c i m a n y d d n A . d i p a r e b l l i w d e e p s g n i n r a e l d n a e g a t l o v t u p t u o e h t n e e w t e b e c n e r e f f i

d thet eachingsigna loft hedifferen tcircuitsandt hef eedback h g i h y r e v s i l e d o m s i h t f o d e e p s g n i n r a e l e h T . s t h g i e w g n i t c e n n o c g n i g n a h c r o f e u l a v e c n e r e f f i d e h t f o . s t n e m e l e t s o c w o l g n i s u t i u c r i c e l p m i s y r e v a f o e t i p s n i e d o m s i h t f o e m i t g n i n r a e l e h

T li sveryshor tandt heworkingt imeoft hismodeli salmostr eal-time . t n e s e r p e r o T . s e r i f n o r u e n f o y t i l i b a b o r p e h t y b e u l a v t u p t u o e h t s t n e s e r p e r l e d o m n o r u e n d e s l u p e h T d w e f a t s a e l t a r o f e m i t h g u o n e , l e d o M n o r u e N d e s l u P e h t g n i s u y t i t n a u q g o l a n a e h

t ozen pulsesis

t r e v n o c o t d e e n t ’ n o d e W . t i u c r i c s i h t f o e g a t l o v t u p t u o e h t s i l e d o m s i h t f o e u l a v t u p t u o e h T . d e d e e n e d o m g n i k r o w e h t g n i h c t i w s r o f s w o l l a l e d o m s i h T . l e d o m s i h t m o r f a t a d w a r e h t e s u n a c e w ; a t a d e h t s s e c e n s y a w l a s i t I . e d o m g n i n r a e l d n

a ary to inpu tthe teaching signal . However ,the connecting y l i s a e o s l a n a c l e d o m s i h T . l a n g i s g n i h c a e t e h t f o g n i g n a h c e h t o t g n i d r o c c a s e g n a h c t h g i e w e v o r p m i l l i w t I . e l b i s s o p s i g n i n r a e l l a m i t p o , e n e c s h c a e n I .t n e m n o r i v n e e h t n i s e g n a h c e t a d o m m o c c a e h

t artificial intelligence elemen twith self-dynamica llearning .The realization of an integration t c e p s e r h t i w t s u b o r s i l e d o m d e s o p o r p e h T . d e c u d e r e b o t s t n e m e l e f o r e b m u n e h t e l b a n e l l i w e c i v e d n o i t c u r t s n o c m e t s y s e d u l c n i s k s a t e r u t u F . e c n a r e l o t t l u a f o

t andmountingal arge-scalei ntegration. p e e d e h t d r a w o t d e v o r p m i m e t s y s s i h t f I . y l t n e c e r d e s o p o r p s i d o h t e m g n i n r a e l p e e d , r e v o e r o M t I . l e d o m g n i n r a e l n i s m h t i r o g l a f o d n i k a s i t I . d e z i l a e r e b l l i w s n o i t a c i l p p a y n a m , l e d o m g n i n r a e l h g i h o t s t p m e t t

a -levelcategorizingdatausingmultiplenon-lineart ransformationsandonemethodof d o h t e m g n i n r a e l p e e d e h t , n o i t i n g o c e r h c e e p s d n a n o i t i n g o c e r e g a m i f o d l e i f e h t n I . g n i n r a e l e n i h c a m o b a n o i t a z i l a e r f o y t i l i b i s s o p e h t d e t s e g g u s e W . n o i t n e t t a e h t d e t c a r t t a s a

h u t the hardware

h t i w t n e m e l e e c n e g i l l e t n i l a i c i f i t r a e h t e v o r p m i l l i w t I . l e d o m g n i n r a e l p e e d e h t f o n o i t a t n e m e l p m i f l e

s e c n e r e f e R

] 1

[ C .Mead ,AnalogVLSIandNeura lSystems ,AddisonWesleyPublishingCompany ,Inc. ,1989. ]

2

[ W .StrunkJr. ,E.B .White ,TheElementsofStyle,t hirded. ,Macmillan ,NewYork ,1979. ]

3

[ M .Kawaguchi ,N .Ishii ,andM .Umeno ,Analogneura lcircui twithswitchedcapacitoranddesign n o i t a m r o f n I d n a g n i t u p m o C d e i l p p A n o e c n e r e f n o C l a n o i t a n r e t n I d r 3 , l e d o m g n i n r a e l p e e d f o

d n a y g o l o n h c e

T 2ndI nternationa lConferenceonComputationa lScienceandIntelligence ,ACIT-CSI, p

p . 23 -2 327 ,2015.

[4] Yoshua Bengio ,Aaron C .Courville ,Pasca lVincent :Representation Learning :A Review and 8

( 5 3 . l l e t n I . h c a M . l a n A n r e t t a P . s n a r T E E E I . s e v i t c e p s r e P w e