Lecture_05-06 Random Variables

(1)

Random Variables

A

random variable

is a numerical description of the outcome of a random experiment

The cumulative distribution function (cdf) for a random variable

X

is denoted by

F

_X (

x

) and is defined as the

probability

that the random variable is less than or equal to

x

.

(2)

Properties of cdf

Because F(x) is a probability, it must have the same

(3)

(4)

Probability Density

Function (pdf)

The probability density function (pdf) for a

continuous

random variable X

is denoted by

f

_X (

x

) and is defined as the derivative of

F

_X (

x

)

The area under the pdf curve is the probability

p

(

x

₁

< X

≤

(5)

(6)

Probability Mass

Function (pmf)

The pdf in the discrete case is called the

probability mass

function

(pmf). The pmf is defined as the probability that the random variable

X

has the value

x

and is

denoted by

p

_X (

x

).

(7)

Expected Value

For the continuous case, the

expected value

is given by

For the discrete case, the expected value is given by the weighted sum

(8)

The expectation is sometimes referred to as the

first

moment

of the random variable. Sometimes

μ

is used as another symbol for the expected value.

μ

=

E

[

X

]

The

variance

, or second central moment, of the random variable

(9)

The variance describes how much of the mass of the distribution is close to the expected value: a large

variance means that the pdf is large for values of

X

far away from

μ

.

The

standard deviation σ

is simply the square root of the variance.

(10)

The

joint cdf

of RVs

X

and

Y i

s

F

_XY (

x, y

) =

p

(

X

≤

x, Y

≤

y

)

For

independent random variables

the joint cdf is simply the product of the individual cdf’s.

F

_XY (

x, y

) =

F

_X (

x

)

F

_Y (

y

)

(11)

The joint pdf satisfies the following relation.

The two random variables are

independent

when the joint pdf can be expressed as the product of the individual pdf’s.

(12)

For discrete RVs, the

joint pmf

is defined as the probability that

X

=

x

and

Y

=

y

p

_XY (

x, y

) =

p

(

X

=

x, Y

=

y

)

The joint pmf satisfies the following relation.

Two random variables are

independent

when the joint pmf can be expressed as the product of the individual pmf’s.

(13)

The

correlation

between two random variables is defined as

We say that the two random variables

X

and

Y

are

(14)

The

covariance

between two random variables is defined as

The following equation can be easily proven.

We say that the two random variables

X

and

Y

are

(15)

The

correlation coefficient ρ

_XY is defined as

When we are dealing with two random variables obtained from the same random process, the correlation

coefficient would be written as

The correlation coefficient would decrease as the value of

n

becomes large to indicate that the random process “forgets” its past values.

(16)

Random (stochastic)

processes

A

random (stochastic) process

assigns a random

function of

time

as the outcome of a random experiment.

A stochastic process, formally denoted as {X(t),tT}, is a sequence of random variables X(t), where the parameter t — most often the time — runs over an index set T.

The

state space

of the stochastic process is the set of all

(17)

Random (stochastic)

Processes

A random process

X

(

t

) is described by

•

the

sample space S

which includes all possible outcomes

s

of a random experiment

•

the

sample function x

(

t

) which is the time function

the statistical dependencies among the random processes

(18)

Four different types

of random processes

We could have four different types of random processes: 1. Discrete time, discrete value

2. Discrete time, continuous value 3. Continuous time, discrete value

4. Continuous time, continuous value

We use the notation

X

(

t

) to denote a continuous-time

random process and also to denote the random variable measured at time

t

. When

X

)

We use the notation

X

(

n

) to denote a discrete-time random process and also to denote the random variable

measured at time

n

. That random variable is statistically described by a pdf

f

_X (

x

) when it is continuous, or it is described by a pmf

p

_X (

x

) when it is discrete.

(20)

Poisson Process

A Poisson process is a stochastic process in which the number of events occurring in a given period of time depends only on the length of the time period.

This number of events

k

is represented as a random variable

K

that has a Poisson distribution given by

(21)

Exponential Process

The exponential process is related to the Poisson process. The exponential process is used to model the

interarrival time between occurrence of random events.

The random variable

T

could be used to describe the

interarrival time. The probability that the interarrival time lies in the range

t

≤

T

≤

t

+

dt

is given by

(22)

Deterministic and

Nondeterministic Processes

A

deterministic

process is one where future values of the sample function are known if the present value is

known.

(23)

The random variable

X

(

n

₁) represents all the possible

values

x

obtained when time is frozen at the value

n

₁. In a sense, we are sampling the ensemble of random

functions at this time value.

The expected value of

X

(

n

₁) is called the

[

X

(

n

)] for discrete-time process

The ensemble average could itself be another random

variable since its value could change at random with our choice of the time value

t

or

n

.

(24)

The time averageis obtained by finding the average value for

one

sample function. The time average is expressed as

for continuous-time process

for discrete-time process

In either case we assumed we sampled the function for a period

T

or we observed

N

samples.

The time average could itself be a random variable since its value could change with our choice of the random function under consideration.

(25)

n

₂ respectively.

The

autocorrelation function

for these two random variables is defined by the following equation

r

_XX(

n

₁

, n

₂) =

E

[

X

₁

X

₂]

_XX(

τ

)

Such a process has a constant expected value and the autocorrelation function depends on the time difference between the two random variables.

(27)

For a discrete time random process, the equations for a wide-sense stationary random process become

E

[

X

(

n

)] =

μ

= constant

(28)

The autocorrelation function for a wide-sense stationary random process exhibits the following properties:

r

_XX(0) =

E

[

X

2₍

n

_{)] ≥ 0}

|

r

_XX(

n

)| ≤

r

_XX(0)

(29)

A stationary random process is

ergodic

n

₁ and

n

₂, respectively.

The

cross-correlation function

is defined by the following equation.

r

_XY (

n

₁

, n

₂) =

E

[

X

₁

Y

₂]

If the cross-correlation function is zero, i.e.

r

_XY = 0, then we say that the two processes are

orthogonal

.

If the two processes are

statistically independent

, then we have

₁

and

n

Correlation Matrix

We can express R_X in terms of the individual correlation functions

For a wide-sense stationary process, the correlation functions depend only on the difference in times

(35)

Covariance Matrix

The

covariance matrix

for many random variables obtained from the same random process

is the vector whose components are the expected values of our random variables.

When the process has zero mean, the covariance matrix equals the correlation matrix:

(36)

Covariance Matrix

The covariance matrix can be written explicitly in the form

(37)