The Neuroeconomics of Learning and Information Processing; Applying Markov Decision Process

The Neuroeconomics of Learning and

Information Processing; Applying

Markov Decision Process

Chatterjee, Sidharta

Andhra University

14 February 2011

! ! ! ! """"

# $

% & ' (

& ) *+,- .

/ 0 *1 ' +2**

0000

3 0 40

. 5 '

3 6 7

. '

4 0

7 0 0

7 ' 4

4 0 .

'

6 6 0 6 3

3 3 ' 3

.

8 3 )

8 38 3 ))

8 3 ) ' $ '

' 9 ' ' '

:& ) :& :& ))

:& ) *+' ;*' <,

( 0 )

( 0 )

( 0 )

( 0 ) (

" 6 . . 8 3 $ ' / &

.' & ' ( ' ' 3 ' 3 = 0

0 ' ' ' 0 . #. . 8

0 ' ( ' .

0 >

' 0! .

) *+,- .

*.

*.

*.

*.

?

0

0 3

3

0

'

0 3

3

0 ' 3

' 3

=4

4444 . . $

@*<,1A

5 &$'

0

3 7 3

3 . 6 3

3 3 ' 6 7

3 6

. @ A 0 4

7 0 '

3

0 3 $

3 0 0

0 ' 0

. 3 ' 3 3 '

' $ 77 @+22,A' . . @+2**A' $ 77 @+22*A'

B $ @+2**A @*;;CA.

6 0

3 3 0

3 3 3 0 ' 3

6 7 3 . ' 0

' 6 3

6 ' 0 3 0 0 '

' 0 . ' 04

> 0 . 3

' > 6 7 =

0 0 .

0 4 '

3 6

0 0 . D

' 3 ' 0

6 40

3 ' 4

0 @B ' *;C1A 3 3 .

3 ' ' 7 6 7 0

0 40

3 ' 3

@% ' +22<A 40 > 0 .

3 ' 0 .

0

Δ ∑ E

D ' Δ > 0 '

' 3 ' ?

=

0 ? =.

0 0 3 0 .

> 0 0

3 3 ' 0

0 > > 0 .

4 3

7 ' 6 0 3

0 0 ! .

0 3

3 3 3

0 0 0 ' 0 0

' 0 0 0 . 5 ' 6 7

0 . ' 7

0 ' 3 '

3 ' 0 0

0 0 ' 0 !

3

0 0 . F ' 0

3 6 3 3

' 0 ' ' 0

. 0 0

. '

' 6 0 ?

>

0

=

> .

0 6 ' 3 '

' 3

6 . 0

6 7

0 6 3

' ' '

4 3 ' 3

4 4 .

0! 3 ' 0

0 ' 3 '

3 0

' 0 6 . 9

0 3 ' ' 0 3 ' >

' 3 ' > '

'

' 0 0

6 7 . '

3 7 0 0 0!

6 3 0 3

0 6 .

5 3 ' 0 0 3

3 ! ' 0 0 ' 4

4 . '

3 3

@ 4 A. ' 3

0

0 3 . B ' '

4 0

0 6 @

6 A 3 3 ' 0

5 0 0 *;CG 3

7 3 0 @ A 7

0 3 0 7 .

0 > ' 0 > '

3 ' 3 3 ' 3 0 H

5 ' 0 ! . 5 3 '

3 3 0

. $ > ' '

0 0

7 . 5 3 ' 6

6 . ' '

0 0 . ( ' 0

3 0

3 > 0 .

6 6 .

' '

0!

3 0 3 0

' 3 6 3 '

3 0

> 0 = 0 3

0 0 .

3

6 3 0 0 4 0

6

@ A. 6 6

6

3 0 3 3

. 0 ' 6

3

3 . $ 6

?3 0 = 3

6 3 3 6 0

@ 3 A. &

' 3 '

3 3 .

0 3 3 0 0

6 E . 3

> ' 3

?

=

0 3 0 7 ? 3 =

0 0

3

. 5 '

3 0

3

@ ' *;;C' $ 77 ' +221A' '

' 0 6 ' 3 '

@9

' *;<GA. ' '

' !

0 0

3 3 0 3

0

6 4 3

6 . 5 3 '

0

. 0 3

0 4 3

' ' 4

. 6 0 '

0

0 6 0

0 3 3 '

7 0

. 6 ' 3

0 3 3 ' 3

'

' 3 04 '

3 3 3

. B

. B

. B

. B

$

$

$

$

3

3

3

3

6

@ ' '

' .A 0 .

& 0 0

3 0 0! .

3 0

0 ' > 0 >

0 3 ' 3 ' >

0 . '

= 0 > 0 3

3 ' > 0 .

@/ ' *;I,A 4

0 0 3 3

' 0 0 @ ' *;C<A.

' 3

0 =

6

3 ' 3 ' 0 '

. ' 0

4

6

' . '

0 3 0 '

6 . 0 3 0

' 3 '

7

.

3

3

0 0 .

6 3

'

3

3 ' 6 6 3 '

3 . B '

3 3

. 5

0 6

6 0 3 ' '

'

6 0

.

0 3 3

7 > 0 3

6 > 0 ' 3

7 7 @ ' *<GC4GC'

*<;2A. ( ' 3

6

0 6 @ '

*;;CA. > ' 3 3

0 0 J &( 3 0

7 4 0 0 @*;I,A

4 0 3 6.

3 = 6

J 0 0

= J D 0

' .

3 = 6 3 6 .

' ' > 0 0 @ =

0 0 A 3 '

0 0 6 = 0 @ '

*;;CA. ' 3 3 4

0! &6 ( @ &( A '

' 4

@# A. 5 3 ' 3 >

@ A 3 =

3 ' =

3 =

. ? &> 0 ='

3

. 3 3 3 . .' 3

= 3 0 3 = 3 .

0

3 4 0 ! @

0 3 0 6'

0 A. ' 3 '

!

7 @ 7 A

0 ! 0 0 0 0

' 3 6 ' .

5 3 ' 3 ! 0 ' 0

' ' = 3 3

3 ' ' 3

0 .

0 8 @*;C;A

. ' 0

0 0 3 3 .

'

0 ' 3 ' 0 '

. '

6 ' ' 0

3 0 3 3

0 0 0 0 .

' 3

3

4

@*;I+A' ' B @*;IGA' @*;I,A' 8 '

' ' 5 0 ' 3

> 0

0 .

> 0 > 0 ' ' 3 > 0 ' 3

J

7 3 0

7 ' >

.

0

0 .

0

0 7

0 . 5 3 ' 7

3 3 3

3 @ . .

0 4 3

' 7 6 7 3

3 '

3 . 5 '

7 0 0 3

3 6 0

3 0 ' 6 0

. 6 3

= 0 4

. @B

$ ' +2**' B B 3 ' +2*2' 5 . . +221A'

0 0 3 6 0

6 0 ' 0 3 '

0 . 0 3

0 '

3 0 0 @8 '

*;C;A' 0 ' '

?

B

$

=4

5 0 = 0 . B

0 0 ' . .'

3 ! 3 0 7

3 6 3 . 3 3 0

0 3

= 0 ' =

.

0 3 3 '

@*;I2A 0

3 3 . 5 3 '

0 ' > 6

0 0! '

' ' '

. 5 = @*;I+A

0!

0 7 0!

3 ' 3 .

0 3 *

@$ 77 +22,A 3 0

0 0 . '

> 3 > 0 0

.

0 .

.... %

%

%

%

$

$

$

$

% 6 ' 0 3

0 6 . F '

0 6 3

7 4 .

. '

0

6 0 0

0 !

3 . 5 ' 6

*_{$ & & 3 0 $ 77 ' & @+22,' +221' +22GA}

0 ! . '

3

. 3 ' 0

0 3 6

6 ' ' 0 0

04 6 0 3

0 . $ ' 0 0

6 3 0

3 . 6

0 CEIK*+ ' CE K*+ 0 3 . D '

E0K 3 ' KC' 0KI' K*+. 5 '

0

' 0 3 > 7

3 ? =. 5 '

0

> 0 0 3 . & ' ?

=

?

0=

C ' 3 I

0

0

6

' 3 0 6 3

0 0

2

To be noted, is and nothing else as well as for and the gives nothing but as stated product of the effects of . From this representation, it seems that the law of this equation is bounded which is rational for which, there may be a fixed number of contexts to define either ‘ ’ or ‘ ’ while, the ‘ ’ remains unchanged. Herein, if we assign the concept of cause as unknown= , a requirement 7 12 to discover outside the concept of ‘ ’ a predicate ‘ ’ foreign to this concept (related to the concept as cause of ‘ ’) is sought for, and solving for x gives 5. Now, a different representation of the context as given by 7+ (

1 12 or 7 4 12 must be as a subproduct of the representation b=5 and nothing else so as 7 5 12. However, similarly, the cause may by represented as well by ( 1 5 12 !" 1 5 12 wherein, the product of

1 !" 1 must be, 7. While, a causality of 7 # 12 gives # 5. Now by substituting 7 # 5 12

yields the value for – # 0 and the rationality of 7 12 is established. Similarly, 7 & '

() 12 would require more

3 . 0 3 3 3

0 3 0 > .

5 ' 6 6 7

3 3 6 @B ' +22<A

?

!

3

!

0

=.

? =' 3 ' '

.

#

#

#

#....

.... $$$$

% >

0

' . .' 0 ' ' '

' ' 0 ' .

' 0

3 0! ! 4

?

=

3

0

0

3

6

0

0

M.

0 > 0

3 6 >

. B 3 3

' 3 !

' 0 3 .

*.

4 3 0

0

6 @

A. D 6 3 ?

9

=

0 .

0

3 3 3 ' 3 0

.

4 0 3 6 0 3

3 '

3 0 6

0 .

Human Cognition

Human Learning Behavioral Neuroscience

Computational/ANN/ Robotics/ AI

Cognitive Ethology

Natural Intelligence

Animal Behavior

Animal Learning Animal Intelligence

Cognitive Neuroscience Cognitive

Economics

Neuroeconomics

.... $$$$ 44440000

$

6 7 3

3 ' = 3

. 0 .

B 3 0 . 5 3 '

' ' 3

3 '

$. . ' ' 0

. 7

. B '

. '

0 . 5 3 ' : 5 = ?9

= 0 3 ' 5 =

@*;<GA

. 6 0

@*;**A 0 4 4 3 & 3

. $ 5 @*;G2A

@ A' B 3

0 .

$. . ' 3 '

. 0

. / 3 ' 0 3 7 3

6 3 .

6 $. . 0

= @*;;IA , ₃

0

* *

* *

+ 4I22 +2

/ ' ' 3 0

*. ?

=

K

+. ?

=

,

*. 2 →* →,→I +. 2 →* →1→I ,. 2 →+ →1→I

3' 0 ' 3 0 0

*. 2 →* →,→I K * E * E * K , +. 2 →* →1→I K * E * E +2 K ++ ,. 2 →+ →1→I K + QI22 E +2 K 41C<

6 0 3

3 6 3 @ 6

0 A. 0 3 4 3 3

3 ' 0 4 4 . /

6 ' 3

3 3 ' 0 6 6 ,C 3

.

1 3

0

2

5 4

A

A

B

A A

B

A

,_{$ $ 0 B ' +22I )}

/ .* 9 $ 3 ' ' $ 3 /

0 4 3 '

0 0 . >

! 0 3 4 4

0 0

6 3 . 6 $. .'

?2= K*S1 S2. . . Sn0.

3 3 ' 3 0

@0 4 A. 0 6 0

3 0 3 3 0 ' 0

6 ' 6 . 0 @

6A 0 0

3 0 . 0

0 3 ! .

*1 ++ 0 @ 0 A.

& 3 ' 0 > ' 0

3 4 4 '

0 7 10 4 1 9 2 5 14 11 12 8 6 3 13 8 3 1 5 1

5 2

1

6 1

1

12 >100

0 0

6 7 3 ' 3

. 0 3 0 '

@ A

3 ' 0 3

. 3

0 ++ 6

3 0 . .' ' 0'

' ' 0 ' . 3

5 ∑ 56, ,, … ,/

<I' 3 '

T

6 7 3 ' *2,' **+

**1 . @ A 0

3

3 . 0 7 )

*. @ *E +AK*+EC1K<G

+. @0*E0+E0,AK4,GEIEGCK,G

,. @ *E +E ,AKCI41E*,K<1

1. @ *E +E ,AK4*+E,EGIKIG

I. @ *E +E ,AK4,E*,ECGK<G'

G. @ *E +E ,AK**+E*2,E**1K,+;

$ 3 # 0 @ A) <;+ @ ;G.<IUA 6

Ρ K 8_, 8 … _/ @*A

' '

@+A

3' 0 0

0 0 3

; cos sin @ KA @,A

D ' Ρ 8_, 8 … _/

@,A G.2CGI' G.22<+' G.1C<1' C.C,,+' G.2CGI C.CG . 3 >

7 . >

'

sin ∑ ∑_{6, ,6} 5 @1A

D '5 g1 g2 g3' 0 .

B 0 @+A @1A' 3 ' 8_, 8 … _/ '

9_,: 8, 8 8D 8E 8F 8G sin ∑ ∑6, ,6 5

@IA

6 3 ' 6

0 0 . ( 4

' 0 0 ' '

= 6 .

> 3 ' 0 0

V 0 '

0 0 .

6 ' 3

0 0 .

' ? = 40 3

3 ' 0 ? = 0

. 0

0 / @*;C2A

. (

' ?

=' @VA.

0

ƒ N HL, _{*O + 0}PQRST

U N HL,_{*V + 0}

> 0 '

WX_{* |Z|0} W* ;Z 1@

Z 1

> @*1A'

> 0 V 9_[\,[RS .

0 0 '

6 0 7 ' 0

40

3 0 0

' 9_[\,[RS ' 9[R

RS] S ^_

7 0 0 0 0 . 7 > '

5 sin `ab c_{/ c\c} @GA

D ' 9_,: 8_, 8 … _/ ' ' ∑ ∑_{6, ,6} 5

. B 0 0 @GA'

> '

d 8, 8 … /

:

,

sin e e 5

,6 6,

sin ∑ ∑_{6, ,6} 5

Psin ∑ ∑_{6, ,6} 5 sin ∑ ∑_{6, ,6} 5T 7

D ' 9_,: 8_,

8 … / > @+A. 3' 0 @GA ' 0

5

0 0 @<A' '

2 g sinP∑ ∑_{6, ,6} 5T 2 h9_,: 8, 8 … / h

2 1

2 g sin ie e 5

,6 6,

j 2 kd 8, 8 … /

:

,

k 1

@;.+A

/ > @;A' 3 3

0 11.GG '

1*. 9 0 > @;.+A'

0 0 3 0 0 0 0 3

3 4 0

0 ' 3 <<.,,U

' *+U 4 . 0 '

3 @ 0 *A' 3 @

UA 0 3 3 3 ;G.<IU.

D ' 7

0 ' 0 6 0

> @GA

5 sin 2 1 @*2A

D '

+lm ie e 5

,6 6,

j 2 kd 8, 8 … /

:

,

k 1

@**A B > @**A' 3 C<.+<U C+*.+,' 3

5 +lm ie e 5

,6 6,

j kd 8, 8 … /

:

,

k

kd 8, 8 … /

:

,

k 1

@*+A

6 3 7

6 0

@3 0 3 A *,I,.,; @*+A' 3 @**A' 3 C+*.,, 3 . 3' 3

++ ;+*. ' 6 0

3 3 3 0 0 3 0

' #KC+*E;+*K*G1+' 3 ' 6 '

6 7 3 ! .

0

> 0 0 0 '

3 ' 0 0 3 0 7

. 0 0

' 3 ' 6 0

3

' '

0 .

> 0 0 . '

0 @;.+A '

oK

p,\q∏t_suSr_s\∏ vs^w∏ vs

t suS x t

suS

∏t_suSvs yz

{

∏t_suSr_s

@*,A

D ' o 7

7 0 @ _/ '

7 > )

2 g +lmP∑ 5_6, ∑ 5_6, T ∑ 5_6, g & 2 g h9[ 8, 8 … /

RS

[\, h)

q1 |∏ 5_6, ∏ 5 P∏ 56, 6, T

∏ 5_6, }y

∏ 5_6, ~/

@*1A

D ' & 2 g h9_[\,[RS 8_, 8 … _/ h) 0

• 2 g k9[RRS]S ^_

[LRS]_S \[ k 8, 8 … / €.

% '

2 g +lm ie 5

6,

e 5

6,

j e 5

6,

g

• 2 g k9[RRS]S ^_

[LL,:_{, \[} k 8, 8 … / €

@*IA +< 3 _/ 87 ' _/ 50' 3

7 *G' 3 ' 3 6

. 7 0 3 ' 9 _[[RS

^S . >

@*1A 3 > @<A' 3 7

> @9%#&A 3

0 0 0 3

' ' 3 ' '

0 . 5 ' ?~_/=

7 0 = >

. / > @*IA' 0

7 . > 3

3 6

J

0 0 0

3 3 J

0

' 0 3 0 =

$ J

> ) $. . 0

0 '

3

J

' 0

0 0 7

0 ' 0 3 3

3 0 3

3 . ' 3 >

5 ' > 0

3 '

' '

. '

3 3

' 3 0 3 3 0

7 3 0 3

' 3 0 ' 7 '

0

@ A . 3 0 ' 0 3

@ A @ A 3

3 7 .

!

0 @ ' +222A. 3 0

0

3 0 3 @ A 3 3 '

0 0 '

! ' 3 '

3 0

0 @/ ' *;;C' ' *;;;A'

3 3 ' 3 .

B 3 > 3 3 3

' 0

3' 0 0 ' . .' 3

' 6 '

3 0 0 0

3 6 7 3 .

3 ' 0 3

0 . '

0 6 7 3 3

3 3 3 .

0 3

$ @ A.

6 7 ' 3 '

6 7 3 0 .

5 3 ' !

0 0 3 '

' 0 3

. 0 0 3

3 0

3 3 ' '

6 ' 3 . 5 '

6

' @ '

/ ' *;I1A. 0

6 6

@ . . +22CA 3 ' 0

3 4 4

3 . 04

.

' 0 0 0

3 @ A. 5 3

0 @ A 3

! 3 6 0 . 5 3 '

?

=

3 3 > 6 0 3 .

#.

#.

#.

#.

0 0

' ' 3

0 ++ . 5 3 ' *I

0 ,C @

A. > = 0 0 0

4 0 0 6 = 0 3

0 3 ' 3 ' 4 ' 6 ' 0

6 ' 0 . D 0 6

0 ' 0

0 0 3

3 0 . 7 0

0

0 0 7 0

3 ' 0

3 6 0 . 7

0 3 ' '

3 6

. 3

0 6 0

6 .

0 3 3 @*;<*A 3

0

0 . '

=

> . 6 ' >

6 . 4

6 3

. ' > 0

. 0 0 '

. 4 ' 3

0 > .

'

4 4 3 ' .

6 > 6 0

0

> .

E

%

/ .+ 0 4& 4

$ @*<,1A'

0 3 0

3 > 4

?

0

0 3

Agents Given Tasks

Ability Effort

Success Failure

3

0

'

0 3

3

0 ' 3

' 3

=

0 3 3 0

3

0 3 0 '

6 ' 6 . 3

0 0 3 3

0 3 0 '

3 3 ' 3

0 3 4 '

0 . 0 ' 0

0 0

0

$ > 0 3 4 .

$ 3 0

6 '

6 @ ' +22;A 3 4 '

6 . 0 =

' 4 4 4 0

$. . 0 = 3 7 '

4 . / '

0 0 0 3 3 >

0 0 3 > .

/ 6

0 7 3 0

0 3 3 0

0 . 0 6

3 0 @B . .' +22<A

0 = 7 =

0 4 0 0

4 0

0 3 .

0000 ))))

8 3 0 3 0 3 6

0 3

' 0 . 5 ' 3

0! '

3 3 ' 0 ' 3 ! 3

. 0 6

*22 0 3 *22 . % '

0 I2'222 0 3 +24CI'

7 0 0 *2U 0 CI.

6 3

.

0 3

$

0 .

0 3 ' '

' 3 .

3 3 6 '

3 3 ' 0

6 0 3 . 6

0 0 . %

0 '

3

> 0 .

5 0 6

0 3

0 7

. 5

3 0 ' 0 '

3 0 6 .

3 0 6'

' > ' '

3 0 '

3 0 .

4 0

> ' 6' @9 ' *;;;A'

3

' 7

@/ . .' +221A' = 9 ' '

0

. 0 0

3

' 0 3

. . 0 @*;;+A 3 4

6 0 4 > 4B

@ 40 A 6

3 3

0 0 3

7 = . = '

0 4 0

0 3

= @ 7 ' *;;<A. (

3 0 ' 7 0 0

0 0 .

5 0

0 0 3

0 6

0 . 0

3 0 .

0 ' 0 0 0

;)*. 5 3 ' ' 0

‚ G„ e …ab S‡ bb O` Š ‹

/6, , @*IA

3 0 .

0 3 6 0 B 8 @*;C<A

0 3 .

' 0

' 0 3

' ' . 5 3 '

0

' 3 ' 0

3 4 . >

3 ' 0

J D 0 0

0 J

B. B. B.

% > ' 3

VΠ0 J 0

.

0 4 .

0

0 . 9 3

3 .

0 0

. 0 ' 0

0 ;)*. ' 3 0

3 0 0

' 0 0 . B

0 > 3

3 ' 0 . &

@ '

' ' I45 .A . & 3 6

3

@ *;;2' 8 *;;GA. 4

6 . 5 3 ' @ A

. 9

0 3

3 .

0 3

'

3 0 . 3 0

0

0 3 0 3

' 4

> . 0 0 0 @

. . +22+A @ 0 3 A

3 3

. ' 0 0 7

0 3 0 3

. ' 0

0 0

. ' 0

> 0 3 @ 3 A >

. 0

7 3

3 0 >

3 @ 4 6 A 3 6

0 0 ! .

' 6' ' ' * '

' 0 3

. ' 3 3 0

J 3

3 0 0 ' ' . .

0

. ' 4

0 0

3 3 0 3 0 .

' 0

3 W B 5 @8 ! ' +2*2A

@

JA 3 ' 0

.

L M

3 0

4 .

#

#

#

# .

.

.

.

0 3

' 3 40

0 0 .

4 0

4 3 .

0 0 0

3 0 3

0 3 '

. 0 6

3

3 0 . %

' '

' 0 ' L M 0

$

)

$

)

$

)

$

)

' . @*;I2A ?( ' & & =' :

& ' . I<) +**4++*.

' . @*;I+A' L X > ) >

XY Y M' & ' +*' . I2,41G.

' D. B . @+22IA. 5 0 & '

# . +) 4B & ' 8. : . ' '

& &# &$R 45 .

' D. B .' ' ' ' . @*;;CA. &

& 6 . .

' :.D. @*;ICA. ? 4 0 =.

$ 3' G1 ,I;4,C+.

B ' 4 $ ' $ ' . @+2**A. 0 0

4 .( ' ' D

. **R*G.

B .5.' .' ' . ' :. @+22<A. B

= ' % B

: ' +22<' +' *4*+.

B ' .:.' ' .@*;C1A % .

$ 3' <*' I2G4I+2.

B ' .' 8 ' 7 .@*;C<A. 0 . . $ .

.*),I@I;A.

B ' ' B 3 :. @+2*2A )

% B ' / . $ . B B D . *24*G.

B . +22<. ) '

' ' 0 ( .

5. : ' / &. ' 0 . ' :$) *;;,.

) ) . . . . ( # .

;2' . I;I;4I;G,' : *;;,' B .

.:. / . @*;C2A. B =.

0 ' . ,' *4+G.

. @*;;CA 4 W 5 & ) %

B WD&$ F. &

' 9.' / ' .' ' .' ' . @+2**A.

7 . 4 4 . 0

' = ' ' D '

+2**R21.

& @+22+A' ZB % 7 B Z' :.:. @ A'

& ' % 7 ) & $ .

' ' & 3 & .

/ ' . *;;C. # ) 3 ? = .

# ) ' / . ) .

/ ' $. .' 49 ' B. +22+. 3 4 '

)# . +;<' IIG4IG+.

/ ' . ' ! 3 ' W. : ' ' 4F # ' +221.

) 3 0 4 .

' # . *2' *2II4*2G,.

/ . @*;I,A' & & ' ( ' .

5 ' . .' $.&. / @*;IGA' L M' . B

. $ @ .A' 9 ) # . 0 ) (

.

5 ' /. . @*;I+A. % . > /

' ) $ [ 8 .

5 ' /. @*;11A. . $ 3'

I*) ,I<4,C,.

5 :.5.' 8. 5 . @*;<GA' $. 0 ' . ' ' .

5 ' ' 8 ' ' 0 8 @+221A B

0 ) 6 0

. & ) # . C+' C<*4<++.

8 ' ' . @*;C;A. )

. & ' # .1C' . +' +G,4+;+.

8 ! \ %. $ )@+2*2A ( & B '

W B 5 ' 8

> 7 X . / . +2*2 *) *I. )

.&.' 7 ' . . *;;<. = ' * +. 3 & :

' # .,,;' *2114*2I, **,24**1,.

. . . . D ' *;;1.

' : . +222. .

+;)I2C4I1<

' .' 9. ' . 7 . > @+222A' L 0 4

8 3 % 7 M'

' ;' CIC4C<<.

' . @*<GC4<A' F ' $ @*;;1A.

' . @*<;2A' & ' ) .

' &6 ( 5 0 & @:.

' D. 5 ' (. ' . ' & 3 & ' *;;C' . ,1+4,I2A.

' ' . +G +22C. )

' 0 6 & ' ( ]

) $ .,21;.

' ' $ 77 ' . @+221A B 3

& &6 & ) / 3 $ 5 '

. . & L . '

=& Z Z @ & & AD

. 2G.

' . 0 ' 0 J &&$ $

*+R+22;

% ff ' )@+22<A 0 8 4

0 40 . .

$ 77 ' ' & ' . @+22,A. & ) /

5 & . & L . '

=& Z Z @ & & A

D . 21.

$ 77 ' . @+22,A. 3 .

& L . '

=& Z Z @ & & A D . 2,.

$ 77 ' . +221. 8 3 4 ' :

$ ' . . @*<,1A4L 0 3

'M + .' ' .

. 5 0 . @*;C<A. $ 4 0 7 . 0

' < 0 ' *;C<. 4 ( ' 0 '

' ( .

. 5 0 @+222A' B $ )

3' [ ' # *' . *' +I412

' . 5 0 @*;CGA. L/ 0 $ M' ) .

@ .A & . 0 ) 0 ( .

.*+;4*1<.

' . 5 0 . @*;;;A. 3 . $ XY

. # . <<. + *;;;. . +,4,;.

' B.' .' 5 ' . .' 9 #.' / ' $. . +22+. ) 4

. '

# . ,G' .I' <II4<G<.

: 4F ' % 7 9 ' 9 . $ ' 5 9. . +221.

. 9 B ' # .*' ,4**.

' & 3 @*;**A' ) &6 ' 3 F '

6666

$ 3 / )

*. 2→*→C→;→**→*1 +. 2→G→<→1,→*+→*2→*1 ,. 2→1→I→*1

1. 2→*→1→I→*1 I. 2→G→1→I→*1

G. 2→G→<→+→,→*+→*2→*1 C. 2→G→<→+→I→*1

8. 2→G→<→*,→,→*1 ;. 2→G→1→C→;→**→*1 *2. 2→1→C→;→**→*1 **. 2→G→<→+→,→*1 *+. 2→*→1→C→;→**→*1 *,. 2→G→<→*,→,→*+→*2→*1 *1. 2→1→+→,→*1

*I. 2→1→+→I→*1

*G. 2→1→+→,→*+→*2→*1 *C. 2→*→1→+→I→*1 *<. 2→*→1→+→,→*1

*;. 2→*→1→+→,→*+→*2→*1 +2. 2→G→1→+→I→*1

+*. 2→G→1→+→,→*1

++. 2→G→1→+→,→*+→*2→*1

0 *. # / 0

! 3 0 )

*. 2→*→C→;→**→*1K*E,E*EIE*K**

+. 2→G→<→1,→*+→*2→*1K*E*E*+4*22E12E*2K4,G@0*A

,. 2→1→I→*1KIE+E<K*I 1. 2→*→1→I→*1 K*4IE+E<KG@ A I. 2→G→1→I→*1K*EGE+E<K*C

G. 2→G→<→+→,→*+→*2→*1K*E*EGE*IE+E12E*2KCI@ *A

C. 2→G→<→+→I→*1K*E*EG4+2E<K41 @ +A

8. 2→G→<→*,→,→*1K*E*E*+E*4*2KI@0+A

;. 2→G→1→C→;→**→*1K*EGE*22E*EIE*K**1

*2. 2→1→C→;→**→*1KIE*22E*EIE*K**+

**. 2→G→<→+→,→*1K*E*EGE*I4*2K*,@ ,A

*+. 2→*→1→C→;→**→*1K*4IE*22E*EIE*K*2,@ *A

*,. 2→G→<→*,→,→*+→*2→*1K*E*E*+E*E+E12E*2KGC@0,A

*1. 2→1→+→,→*1KIE+E*I4*2K *+@ *A

*I. 2→1→+→I→*1KIE+4+2E<K4I

*G. 2→1→+→,→*+→*2→*1K IE+E*IE+E12E*2KC1@ +A

*C. 2→*→1→+→I→*1K*4IE+4+2E<K4*+@*A

*<. 2→*→1→+→,→*1K*4IE+E*I4*2K,@ +A

*;. 2→*→1→+→,→*+→*2→*1K*4IE+E*IE+E12E*2KGI@,A

+2. 2→G→1→+→I→*1K*EGE+4+2E<K4,@*A

+*. 2→G→1→+→,→*1K*EGE+E*I4*2K*,@+A

6 B 6.* % &

Total Initial Value of the system before the program: 85 Total value unlocked or rewards derived by the agents: 921 Total No. of Policies: 22 Mean (avg.) Reward per agent: 41.86 Efficiency Rate: 49% Reward gained by following PP’s: (excluding. Top 3) 563

Reward Value of Top three Policies: 35.72 (329) Reward Value of Top three Policies as a %: 35.72%

Reward Value gained by top 14 patterns per agent: 40.21 Efficiency of patterned policy choices: 61.12% Mean reward gained by following random policies: 6.25 Total rewards gained following non-patterned policies as a percentage: 4.3%

6 0 6 +. 3

3 04 $ .)

*. $ $ )RR3334 3. . . R

+. ( 0 $

) )RR3 0 . . 0 . RS R0 R 0 R *.

FIELD N MEAN STD SEM MIN MAX SUM (Policy Groups)

A (a1+a2) 2 43.00 43.84 31.00 12 74 86

B (b1+b2+b3) 3 12.00 51.86 29.94 -36 67 36

D (d1+d2+d3) 3 28.00 41.58 24.01 -4 75 84

E (e1+e2+e3) 3 18.67 40.82 23.57 -12 65 56

F (f1+f2+f3) 3 28.67 41.77 24.11 -3 76 86