Expectation and Estimation

(1)

Expectation and Estimation

(2)

Multiple Random Variables

q 2차원 랜덤변수 사상

S

_J

x

y

1

s

2

s

2 1

( ( ), ( ))

X s

Y s

1 2

( ( ), ( ))

X s Y s

Y

X

(3)

Joint Cumulative Distribution Function

q Single Cumulative Distribution Function

q Joint Cumulative Distribution Function

(4)

Joint Cumulative Distribution Function

q Properties of Joint Cumulative Distribution Function

(5)

Joint Probability Density Function

q Joint Probability Density Function

(6)

(7)

(8)

(9)

Conditional Probability Density Function

q Statistical Independence

,

(10)

Conditional Probability Density Function

q Example , , 1 2 , _{0 0} 2 1 ₂ 0 0

Let the joint density of two random variables and be given by 1 ( 4 ), 0 2, 0 1 ( , ) ₆ 0, otherwise 1) ( , ) 0 and 1 ( , ) ( 4 ) 6 1 1 1 ( 4 ) 6 2 6 X Y X Y X Y x x X Y x y x y f x y f x y f x y dxdy x y dxdy x xy dy ¥ ¥ -¥ -¥ = = ì ₊ _{< <} _{< <} ï = í ïî ³ = + é ù = _ê + _ú = ë û

ò ò

ò

01 1 2 0 (2 8 ) 1 (2 4 ) 1 6

Probability Density function

(11)

(12)

Expected Value

q Expected value or Mean of a discrete random variable X

- The expected value (or expectation) refers, intuitively, to the value of a random variable one would "expect" to find if one could repeat the random variable process an infinite number of times and take the

average of the values obtained.

- The expected value is a weighted average of all possible values.

(13)

Expected Value

q Expected value a random variable X(General meaning)

(14)

Expected Value

q Ex) Mean of normal pdf

(15)

Moments

q Expected value functions of a random variable X

q Moments about the origin

q Central Moments

[

( )

]

( ) _X ( ) E g X ¥ g x f x dx -¥ =

ò

Let ( ) , 0,1,2,...

then the n-th moment of is given by [ ] ( )

n n n n X g X X n X m E X ¥ x f x dx -¥ = = = =

ò

0 0 Let ( ) ( ) , 0,1,2,...

where is the mean of , then the n-th central moment of

(16)

(17)

Moments

q Ex) 주사위를 1회 던졌을 때 나타나는 눈의 모멘트

( )

2 2 2 2 2 2 2 {1, 2, 3, 4, 5, 6}, ( ) 1/ 6, 1,2,...,6 1 1 21 7

Mean (the 1 moment) : [ ] 1 ... 6

6 6 6 2

1 1 1 91

The 2 moment : [ ] 1 2 ... 6

6 6 6 6

(18)

Moments

q Ex) Gaussian Random Variable

2 2 ( ) 2 2 2 2 1 ( ) 2

Mean (the 1 moment) : [ ]

Variance(the 2 central moment) : [( ) ]

The pdf of the gaussian RV is represented by its mean and variance.

(19)

Moments

q Joint moments about the origin

q Correlation

,

For two random variables and ,

the joint moment is given by ( , ) ,

where is the order of the joint moment.

n k n k nk X Y X Y m E X Y x y f x y dxdy n k ¥ -¥ é ù = _ë _û = +

ò

[ ]

11 , ( , )

If [ ] [ ] is satisfied, we say that there is

between and . Or, it is said that is statistically indpendent on no cor . relation XY X Y XY R m E XY x y f x y dxdy R E X E Y X Y X Y ¥ -¥ = = = =

ò

(20)

Moments

q Joint central moments

,

joint central mom

For two random variables and ,

(21)

-Moments

q Covariance 11 , , , , [ ] [ ] [ ] [ ] [ ] , 1 ( )( ) ( )( ) ( , ) ( , ) ( , ) ( , ) ( , ) XY X Y X Y X Y X Y E XY E XY XE Y XY E XY YE X XY X Y C E X X Y Y x X y Y f x y dxdy

xyf x y dxdy X y f x y dxdy Y x f x y dxdy XY f x y dxdy

µ

¥ -¥ ¥ ¥ ¥ -¥ -¥ -¥ = = = = = = = ¥ -¥ = é ù = = _ë - - _û = - -= - -+

ò

!"""#"""$ !"""#"""$ !"""#"""$ !""# no correlation or independent If 0( [ ] [ ]), there is

each other between and .

If and are orthogonal( 0), [ ] [ ] .

(22)

(23)

Multivariate random variables

q Multivariate random variables (Random vectors)

• Vectors with random variables

(24)

Multivariate random variables

q Mean and covariance under linear transformation

• Suppose the mean and the covariance of 𝑥 is given by

Mean: Covariance:

• A new variable via linear transformation 𝑦 = 𝐴𝑥. Then,

(25)

Multivariate Gaussian Random Variables

q Multivariate Gaussian Probability Density Function

(26)

Multivariate Gaussian Random Variables

q Conditioning

• Given a joint pdf 𝑓_%_'_%_$ 𝑥_!, 𝑥_" on R" • Measure 𝑥_";

we would like to find the conditional pdf of𝑥_!,

(27)

Multivariate Gaussian Random Variables

q Conditioning • Suppose 𝑥 ~ 𝑁 0, Σ with • Suppose we measure 𝑥_" = 𝑦. • Conditional pdf 𝑥_! given 𝑥_" = 𝑦: 𝑥 = _𝑥𝑥! " Σ = Σ_!! Σ_!" Σ_!"$ Σ_"" 𝑓_%_' 𝑥_! 𝑥_" = 𝑦 = ! "- ) _,_''_&,_'$_, $$ &'_, '$ % 𝑒

&'_$ (&,_'$,_$$&'. % ,_''&,_'$,_$$&',_'$% &' (&,_'$,_$$&'.

⟺ 𝑓_%_' 𝑥_! 𝑥_" = 𝑦 ~ 𝑁(Σ_!"Σ_""&!𝑦, Σ_!! − Σ_!"Σ_""&!Σ_!"$ )

(28)

(29)

Multivariate Gaussian Random Variables

q Estimation

• Suppose 𝑥 ~ 𝑁 0, Σ₍ , 𝑤 ~ 𝑁 0, Σ_/ uncorrelated, and a measurement 𝑦 = 𝐴𝑥 + 𝑤 is given:

• We want the best estimate on 𝑥 given 𝑦, i.e., 𝐸[𝑥|𝑦] • Let 𝑧 = 𝑥_{𝑦 =} 𝐼 0

𝐴 𝐼

𝑥 𝑤

• Then

• If Σ_/ = 𝐼 and Σ₍ ⟶ ∞, this approaches to the least squares