analog communication Lecture 14

Review of Random Processes

Lecture 14

EEE 352 Analog Communication Systems Mansoor Khan

Electrical Engineering Dept. CIIT Islamabad Campus

Random Variable



A random variable is a mapping function which assigns

outcomes of a random experiment to real numbers.

Occurrence of the outcome follows certain probability

distribution. Therefore, a random variable is completely

characterized by its probability density function (PDF).

Random Process



_{A random variable that is a function of}

time is called a random or stochastic

process. Thus a random process is a

collection of infinite number of random

variables.



_{Examples of random processes}



Daily stream flow



Hourly rainfall of storm events

Stochastic process

Example: Let X(t) be the number of people in a particular railway

station from 8Am to t (>=8). Clearly, for each given t, X(t) is a random variable. Table below lists some possible values that X(t) could take:

date X(9) X(10) X(11) X(12) X(13) X(14) … Oct. 12 1360 1412 1750 1603 1598 1821 … Oct. 13 1362 1490 1713 1641 1601 1845 … Oct. 14 1289 1472 1739 1593 1614 1864 … Oct. 15 1313 1453 1721 1631 1622 1871 … Oct. 16 1368 1481 ? ? ? ? ? Oct. 17 ? ? ? ? ? ? ?

Stochastic Process

• Stochastic processes are processes that proceed randomly in time. • Rather than consider fixed random variables X, Y, etc. or even

sequences of random variables, we consider sequences X₀, X₁,

X₂, …. Where X_t represent some random quantity at time t.

• In general, the value X_t might depend on the quantity X_t-1 at time t-1, or even the value X_s for other times s < t.

Continuous and Discrete Time Stochastic

Process

• A stochastic process is a family of time indexed random variables X_t where t belongs to an index set. Formal notation, where I is an index set that is a subset of R.

• Examples of index sets:

1) I = (-∞ to ∞ or I = [0, ∞. In this case X_t is a continuous time stochastic process.

2) I = {0, ±1, ±2, ….} or I = {0, 1, 2, …}. In this case X_t is a discrete time stochastic process.

• We use uppercase letter {X_t } to describe the process. A time series, {x_t } is a realization or sample function from a certain process.

• We use information from a time series to estimate parameters and properties of process {X_t }.

Probability Distribution of a Process

• For any stochastic process with index set I, its probability distribution function is uniquely determined by its finite dimensional distributions.

• The k dimensional distribution function of a process is defined by

for any and any real numbers x₁, …, x_k .

• The distribution function tells us everything we need to know about the process {X_t }.









Statistical Averages Summary

• The first moment of a

probability distribution of a random variable X is called mean value m_X, or expected value of a random variable X

• The second moment of a

probability distribution is the mean-square value of X

• Central moments are the moments of the difference between X and m_X and the second central moment is the

variance of X

• Variance is equal to the difference between the mean-square value and the square of the mean

m_X = E{X} = ∫ x p_X(x) dx -∞ ∞ E{X2_{} =} ∫ x2 _p X(x) dx -∞ ∞ Var(X) = E{(X – m_X)2 _} = ∫ (x – m_X)2 p_x(x) dx -∞ ∞

Stationary Processes

• A process is said to be strictly stationary if has the same joint distribution as . That is, if

• If {X_t } is a strictly stationary process and then, the mean function is a constant and the variance function is also a constant.

• Moreover, for a strictly stationary process with first two moments finite, the covariance function, and the correlation function depend only on the time difference s.

















 

Weak Stationarity

• Strict stationary is too strong of a condition in practice. It is often

difficult assumption to assess based on an observed time series x₁,…,x_k. • In time series analysis we often use a weaker sense of stationary in

terms of the moments of the process.

• A process is said to be nth-order weakly stationary if all its joint

moments up to order n exists and are time invariant, i.e., independent of time origin.

• For example, a second-order weakly stationary process will have constant mean and variance, with the covariance and the correlation being functions of the time difference along.

• A strictly stationary process with the first two moments finite is also a second-ordered weakly stationary. Also called Wide sense Stationary

First order stationary process

The probability density function of a first

order stationary process satisfies



If x(t) is a first order stationary process

then













(

,

)

,

)

,

(

x

t

p

x

t

for

all

t

p

constant

)}

(

{

)}

(

{

x

t



E

x

t







x



E

Second order stationary process

The Second order density function of a

second order stationary process satisfies



If x(t) is a second order stationary

process then

















,

,

)

,

,

,

(

)

,

,

,

(

t

t

all

for

t

t

x

x

p

t

t

x

x

p

)

(

)}

(

)

(

{

}

,

(







t

t

E

x

t

x

t

R

R









Wide Sense Stationary

 A process is Wide Sense Stationary process if

 Second order stationary Wide Sense Stationary

process. •

)

(

)}

(

)

(

{

constant

}

)

(

{

x

t



R



t

x

E

x

t

x

E









Estimation of the mean

• Given a single realization {x_t} of a stationary process {X_t}, a natural estimator of the mean is the sample mean

which is the time average of n observations.

 



Sample Autocovariance Function

• Given a single realization {x_t} of a stationary process {X_t}, the sample autocovariance function given by

is an estimate of the autocavariance function.

 

_









Correlation Functions

)}

(

)

(

{

)

,

(

)

(

),

(

2

2

1

1

2

1

2

1

t

x

t

x

E

t

t

R

t

x

and

t

x

of

n

correlatio

The

xx



Time Average



The time average of x(t)











dt

t

x

t

x

T

dt

t

x

T

x

of

A

T

)

(

)

(

2

1

lim

)

(

R

is

function

ation

autocorrel

time

The

)

(

2

1

lim

.

.





Ergodic Process



If the time average and time autocorrelation are

equal to the statistical average and statistical

autocorrelation then the process is ergodic.



Ergodity is very restrictive. The assumption of

ergodity is used to simplify the problem

















dt

t

x

t

x

T

dx

x

p

t

x

t

x

dt

t

x

T

dx

x

p

t

x

)

(

)

(

2

1

lim

)

(

)

)

(

)

(

(

)

(

2

1

lim

)

(

)

(





Properties of PSD



Some properties of PSD are:

•

P

(f ) is always real

•

P

(f ) > 0

•

When x(t) is real, P

(-f )= P

(f )

•

If x(t) is WSS,

•

PSD at zero frequency is:

 

 

 

Total Normalized Power

0

P

f df

P

P

f df

P

x

R

 

 







 

0

 

P

R

 

d





Linear Systems

• Recall that for LTI systems:

• This is

still valid if x and y are random processes

, x

might be signal plus noise or just noise

• What is the autocorrelation and PSD for y(t) when x(t)

is known?

     

 

   

y t



h t



x t



Y f



H f X f

Linear Network

h(t)

H(f )

 

   

 

   

   

Output of an LTI System

• Theorem:

If a WSS random process x(t) is applied to a LTI

system with impulse response h(t), the output

autocorrelation is:

• And the output PSD is:

• The power transfer function is:

 

  



  



    



   

 

( ) (

)

R

y t y t

h

x t

d

h

x t

d

h

h

R

d d

h

h

R







 



  





  



 













 









 







 

 

 

 

P

f



H f

P

f

 

 

_{ }

 

P

f

G

f

H f

P

f



