Power
Spectral
Density
INTRODUCTION
Understandinghowthestrengthofasignalisdistributedinthefrequencydomain, relative to the strengths of other ambient signals, is central to the design of any LTIfilterintendedtoextractorsuppressthesignal. Weknowthiswellinthecase of deterministicsignals, and it turnsoutto be justas truein the caseofrandom signals. For instance, if a measured waveform is an audio signal (modeled as a random process sincethe specific audio signal isn’t known) with additive distur bance signals, you might want to build a lowpass LTI filter to extract the audio andsuppressthedisturbancesignals. Wewouldneedtodecidewheretoplace the cutofffrequencyofthefilter.
There are two immediate challenges we confront in trying to find an appropriate frequency-domaindescriptionforaWSSrandomprocess. First,individualsample functionstypicallydon’thavetransformsthatareordinary,well-behavedfunctions of frequency; rather, their transformsare onlydefinedin thesense of generalized functions. Second,since theparticular samplefunction isdetermined as the out comeofaprobabilisticexperiment,itsfeatureswillactuallyberandom,sowehave to search for features of thetransformsthat arerepresentative of thewhole class ofsamplefunctions,i.e.,oftherandomprocessasa whole.
It turnsoutthat thekeyistofocus ontheexpected powerinthesignal. This isa measure ofsignalstrengththat meshesnicelywiththesecond-momentcharacteri zationswehaveforWSSprocesses,asweshowin thischapter. Fora processthat issecond-order ergodic,thiswillalsocorrespondto thetimeaveragepowerinany realization. Weintroducethediscussion usingthecaseofCTWSSprocesses,but theDTcasefollowsverysimilarly.
10.1 EXPECTED INSTANTANEOUS POWER AND POWER SPECTRAL
DENSITY
Motivated bysituations in whichx(t) isthevoltage across(or currentthrough) a unitresistor,werefertox2(t)astheinstantaneouspowerinthesignalx(t). When x(t)isWSS,theexpectedinstantaneouspowerisgivenby
1 Z ∞
E[x2(t)]=Rxx(0)= Sxx(jω)dω, (10.1)
whereSxx(jω)istheCTFToftheautocorrelation functionRxx(τ). Furthermore,
when x(t) is ergodic in correlation, so that time averages and ensembleaverages areequalincorrelationcomputations,then(10.1)alsorepresentsthetime-average power in any ensemble member. Note that since Rxx(τ) = Rxx(−τ), we know
Sxx(jω) is always real and even in ω; a simpler notation such as Pxx(ω) might
thereforehavebeenmoreappropriateforit,butweshallsticktoSxx(jω)to avoid
a proliferation ofnotational conventions, and to keep apparent thefact that this quantityistheFouriertransformofRxx(τ).
The integral above suggests that we might be able to consider the expected (in stantaneous)power (or, assumingthe processis ergodic, thetime-average power) in a frequency band of width dω to be given by (1/2π)Sxx(jω)dω. To examine
this thought further,consider extractinga band of frequencycomponents ofx(t) bypassingx(t)throughanidealbandpassfilter,showninFigure10.1.
x(t) � H(jω) � y(t) � � H(jω) 1 �Δ� � Δ � ω0 ω −ω0
FIGURE10.1 Idealbandpassfiltertoextractabandoffrequenciesfrominput,x(t). Because of the way we are obtaining y(t) from x(t), the expected power in the outputy(t)canbe interpretedastheexpectedpowerthat x(t)hasintheselected passband. Usingthefactthat
Syy(jω) =|H(jω)|2Sxx(jω), (10.2)
weseethat thisexpectedpowercanbecomputedas
1 Z +∞ 1 Z E{y2(t)}=Ryy(0)= Syy(jω)dω= Sxx(jω)dω. (10.3) 2π −∞ 2π passband Thus 1 Z Sxx(jω)dω (10.4) 2π passband
isindeed theexpected powerofx(t)in thepassband. Itis thereforereasonable to callSxx(jω)thepowerspectraldensity(PSD)ofx(t). Notethattheinstanta
neouspowerofy(t),andhencetheexpectedinstantaneouspowerE[y2(t)],isalways nonnegative, no matter how narrow the passband, It follows that, in addition to being real and even in ω, thePSD is always nonnegative, Sxx(jω) ≥0 forall ω.
isusefultohave anamefortheLaplacetransformoftheautocorrelation function; weshallrefertoSxx(s)asthecomplexPSD.
Exactly parallel results apply for the DT case, leading to the conclusion that Sxx(ejΩ)isthepowerspectraldensityofx[n].
10.2 EINSTEIN-WIENER-KHINCHIN THEOREM ON EXPECTED TIME
AVERAGED POWER
TheprevioussectiondefinedthePSDasthetransformoftheautocorrelationfunc tion, and providedaninterpretation of this transform. We nowdevelop analter native route to the PSD. Consider a random realization x(t) of a WSS process. We have already mentionedthe difficulties with trying to take the CTFT of x(t) directly, so we proceedindirectly. LetxT (t)be thesignal obtained bywindowing
x(t),soitequals x(t)in theinterval(−T , T)butis0 outsidethisinterval. Thus xT (t) =wT (t)x(t), (10.5)
wherewedefinethewindowfunctionwT (t)tobe1for| |t < T and0otherwise. Let
XT (jω)denotetheFouriertransformofxT (t);notethatbecausethesignalxT (t)is
nonzeroonlyoverthefiniteinterval(−T, T),itsFouriertransformistypicallywell defined. Weknow thattheenergyspectral density (ESD)Sxx(jω)ofxT (t)is
givenby
Sxx(jω) =|XT (jω)|2 (10.6)
andthatthisESDisactuallytheFouriertransformofxT (τ)∗x←T (τ),wherex←T(t) =
xT (−t). Wethushave theCTFTpair
Z ∞
xT (τ)∗x←T(τ) = wT (α)wT (α−τ)x(α)x(α−τ)dα⇔ |XT (jω)|2 , (10.7)
−∞
or, dividingboth sidesby 2T (whichisvalid, since scalinga signal bya constant scalesitsFouriertransformbythesameamount),
1 Z ∞ 1 2
2T −∞ wT (α)wT (α−τ)x(α)x(α−τ)dα⇔ 2T |XT (jω)| . (10.8) Thequantityontherightiswhatwedefined(fortheDTcase)astheperiodogram ofthefinite-lengthsignalxT (t).
Because the Fourier transform operation is linear, the Fourier transform of the expected value of a signal is the expected value of the Fourier transform. We may therefore take expectations of both sides in the preceding equation. Since E[x(α)x(α−τ)]=Rxx(τ),weconclude that
1 Rxx(τ)Λ(τ)⇔
2T E[|XT (jω)|
2], (10.9)
where Λ(τ) is a triangular pulse of height 1 at the origin and decaying to 0 at |τ|= 2T, the resultof carryingout theconvolution wT ∗wT←(τ) and dividing by
6
2T. NowtakingthelimitasT goesto∞,wearriveat theresult 1
Rxx ⇔Sxx
T→∞ 2T E[|XT (jω)|
2]. (10.10) (τ) (jω)= lim
This is the Einstein-Wiener-Khinchin theorem (proved by Wiener, and inde pendently byKhinchin,in the early1930’s, but —as onlyrecently recognized— statedbyEinsteinin 1914).
The resultis importantto us because it underlies a basic method for estimating Sxx(jω): witha given T, computetheperiodogram forseveral realizations ofthe
randomprocess(i.e.,inseveralindependentexperiments),andaveragetheresults. Increasingthenumber ofrealizationsoverwhichtheaveragingisdonewill reduce the noise in the estimate, while repeating the entire procedure for larger T will improvethefrequencyresolutionoftheestimate.
10.2.1 SystemIdentificationUsingRandomProcessesasInput
Consider the problem of determining or “identifying” the impulse response h[n] of a stable LTI system from measurements of the input x[n] and output y[n], as indicatedinFigure10.2.
x[n] � h[n] � y[n]
FIGURE10.2 Systemwithimpulseresponse h[n]to bedetermined.
The most straightforward approach is to choose the input to be a unit impulse x[n] =δ[n], andto measure thecorresponding outputy[n], which bydefinition is theimpulseresponse. Itisoftenthecaseinpractice,however,thatwedonotwish to—or areunable to—pickthis simpleinput.
Forinstance,toobtainareliableestimateoftheimpulseresponseinthepresenceof measurementerrors,wemaywishtouseamore“energetic”input,onethatexcites thesystemmore strongly. There aregenerallylimitstotheamplitudewecan use ontheinput signal,so toget more energywehave to causethe inputto actover a longer time. We could then compute h[n] by evaluating the inversetransform of H(ejΩ), which in turn could be determined as the ratioY(ejΩ)/X(ejΩ). Care has to be taken, however, to ensure that X(ejΩ) = 0 for any Ω; in other words, the input has to be sufficiently “rich”. In particular, the input cannot be just a finitelinearcombinationofsinusoids(unlesstheLTIsystemissuchthatknowledge ofitsfrequencyresponse ata finitenumber offrequenciesserves todeterminethe frequency response at all frequencies — which wouldbe the case with a lumped system, i.e., a finite-order system, except that one would need to know anupper bound on the order of the system so as to have a sufficient number of sinusoids combinedintheinput).
Theabove constraintsmightsuggestusing arandomly generatedinputsignal. For instance,supposewelettheinputbeaBernoulliprocess,withx[n]foreachntaking the value +1 or −1 with equal probability, independently of the values taken at other times. This process is (strict- and) wide-sense stationary, with mean value 0 and autocorrelationfunctionRxx[m] =δ[m]. The correspondingpowerspectral
densitySxx(ejΩ)isflatatthevalue1overtheentirefrequencyrangeΩ∈[−π, π];
evidently the expected powerof x[n] isdistributed evenlyover all frequencies. A process with flatpowerspectrum is referred to asa white process (a termthat ismotivatedbytheroughnotionthatwhitelightcontainsallvisiblefrequenciesin equal amounts);aprocess thatisnotwhiteistermedcolored.
NowconsiderwhattheDTFTX(ejΩ)mightlooklikeforatypicalsamplefunction of a Bernoulli process. A typical sample function is not absolutely summable or square summable, and so does not fall into either of the categories for which we know that there are nicely behaved DTFTs. We might expect that the DTFT exists insomegeneralized-functionsense (sincethesamplefunctionsarebounded, and thereforedonotgrowfaster thanpolynomially withnforlarge| |n), andthis isindeedthecase,butitisnotasimplegeneralizedfunction;notevenas“nice”as theimpulsesorimpulsetrainsor doubletsthatwearefamiliarwith.
When the input x[n] is a Bernoulli process, the output y[n] will also be a WSS random process, and Y(ejΩ) will again notbe a pleasanttransform to dealwith. However, recallthat
Ryx[m] =h[m]∗Rxx[m], (10.11)
soifwecanestimatethecross-correlationoftheinputandoutput,wecandetermine the impulseresponse (forthis casewhere Rxx[m] =δ[m]) as h[m] =Ryx[m]. For
a more generalrandomprocess attheinput, witha more generalRxx[m],we can
solveforH(ejΩ)bytakingtheFouriertransformof(10.11),obtaining H(ejΩ) =Syx(e
jΩ)
. (10.12)
Sxx(ejΩ)
Iftheinputisnotaccessible,andonlyitsautocorrelation(orequivalentlyitsPSD) is known,then wecanstill determinethemagnitudeofthefrequencyresponse,as long as wecan estimatethe autocorrelation(or PSD)of theoutput. Inthis case, wehave
2 Syy(ejΩ)
|H(ejΩ)| =
Sxx(ejΩ)
. (10.13)
Given additionalconstraints orknowledge aboutthesystem,one can oftendeter minealotmore(oreveneverything)aboutH(ejω)fromknowledgeofitsmagnitude.
10.2.2 InvokingErgodicity
HowdoesoneestimateRyx[m]and/orRxx[m]inanexamplesuchastheoneabove?
Theusualprocedureistoassume(orprove)thatthesignalsxandy areergodic. What ergodicity permits— as we have noted earlier —is the replacement of an expectationorensemble average bya timeaverage, whencomputingtheexpected
valueofvariousfunctionsofrandomvariablesassociatedwithastationaryrandom process. ThusaWSSprocessx[n]wouldbecalledmean-ergodicif
N lim 1 X x[k]. (10.14) 2N+ 1 E{x[n]}= N→∞ k=−N
(Theconvergenceontheright hand sideinvolves a sequenceof randomvariables, sotherearesubtletiesinvolvedindefiningitprecisely,butwebypasstheseissuesin 6.011.) Similarly,forapairofjointly-correlation-ergodicprocesses,wecouldreplace thecross-correlationE{y[n+m]x[n]}bythetimeaverageofy[n+m]x[n]. Whatergodicitygenerallyrequiresisthatvaluestakenbyatypicalsamplefunction over time be representative of the values taken across the ensemble. Intuitively, whatthis requiresisthat thecorrelationbetweensamplestakenatdifferenttimes falls off fast enough. For instance, a sufficient condition for a WSS process x[n] with finite variance to be mean-ergodic turns out to be that its autocovariance functionCxx[m] tendsto 0 as|m|tendsto ∞, whichisthecasewith mostofthe
examples we deal with in these notes. A more precise (necessary and sufficient) condition for mean-ergodicity is that the time-averaged autocovariance function Cxx[m], weightedbyatriangularwindow,be 0:
L lim 1 X µ1− |m| ¶ Cxx[m] = 0. (10.15) L→∞2L+ 1 m=−L L+ 1
A similar statement holds in the CT case. More stringent conditions (involving fourth moments rather than just second moments) are needed to ensure that a processissecond-orderergodic;however,theseconditionsaretypicallysatisfiedfor theprocessesweconsider, wherethecorrelationsdecayexponentiallywithlag.
10.2.3 ModelingFiltersandWhiteningFilters
There are various detection and estimation problems that are relatively easy to formulate, solve, and analyzewhen some random process that is involved in the problem—forinstance,thesetofmeasurements—iswhite,i.e.,hasaflatspectral density. Whentheprocessiscoloredratherthanwhite,theeasierresultsfromthe whitecasecan stilloftenbeinvokedin someappropriatewayif:
(a) thecoloredprocessistheresultofpassing awhiteprocessthroughsome LTI modeling or shaping filter,which shapes thewhite processat theinput into one that hasthe spectral characteristics of thegiven colored process at the output;or
(b) thecoloredprocessistransformableintoawhiteprocessbypassingitthrough anLTI whiteningfilter, whichflattens outthespectral characteristicsofthe coloredprocess presentedattheinputintothoseofthewhitenoise obtained attheoutput.
6
Thus,amodelingorshapingfilterisonethatconvertsawhiteprocesstosomecol oredprocess,whileawhiteningfilterconvertsacoloredprocesstoawhiteprocess. Animportant resultthat followsfrom thinking in termsof modeling filtersis the following (statedand justified ratherinformally here — a more carefultreatment isbeyondourscope):
Key Fact: ArealfunctionRxx[m]istheautocorrelationfunctionofareal-valued
WSS random process if and onlyif its transformSxx(ejΩ) is real, even and non
negative. Thetransformin thiscaseisthePSDoftheprocess.
Thenecessityoftheseconditionsonthetransformofthecandidateautocorrelation function follows from properties we have already established for autocorrelation functions andPSDs.
Toarguethattheseconditionsarealsosufficient,supposeSxx(ejΩ)hastheseprop
erties, and assume for simplicity that it has no impulsive part. Then it has a real andeven squareroot, whichwe maydenote bypSxx(ejΩ). Now constructa
(possibly noncausal)modeling filter whose frequency response H(ejΩ) equals this squareroot; theunit-sample reponseofthis filterisfoundbyinverse-transforming H(ejΩ) =pS
xx(ejΩ). If we then apply to the input of this filter a (zero-mean)
unit-variancewhitenoiseprocess,e.g.,aBernoulliprocessthathasequalprobabil itiesoftaking+1and−1ateachDTinstantindependentlyofeveryotherinstant, then theoutput willbe a WSSprocess with PSDgivenby |H(ejΩ)
|2 =S
xx(ejΩ),
andhencewiththespecifiedautocorrelationfunction.
If the transformSxx(ejΩ) hadan impulse at theorigin, wecould capture this by
addinganappropriateconstant(determinedbytheimpulsestrength)totheoutput of amodeling filter,constructedas above byusingonly thenon-impulsivepartof thetransform. ForapairofimpulsesatfrequenciesΩ=±Ωo=0inthetransform,
wecouldsimilarlyaddatermoftheformAcos(Ωon+Θ),whereAisdeterministic
(anddetermined bytheimpulsestrength)andΘisindependentofallotherother variables,and uniformin[0,2π].
Similarstatementscan bemadeintheCTcase.
Weillustratebelowthelogicinvolvedindesigningawhiteningfilterforaparticular example;thelogicfora modelingfilterissimilar(actually,inverse)tothis. Consider thefollowing discrete-timesystemshownin Figure10.3.
x[n] � h[n] � w[n]
FIGURE10.3 Adiscrete-timewhiteningfilter.
Suppose that x[n] is a process with autocorrelation function Rxx[m] and PSD
Sxx(ejΩ), i.e., Sxx(ejΩ) = F {Rxx[m]}. We would like w[n] to be a white noise
outputwithvarianceσ2 .
Weknowthat Sww(ejΩ) =|H(ejΩ)|2 Sxx(ejΩ) (10.16) or, σ2 |H(ejΩ)|2 = Sxx( w ejΩ). (10.17) This then tells us what the squared magnitude of the frequency response of the LTI systemmustbe to obtaina white noise outputwith varianceσ2 . If we have
w
Sxx(ejΩ)availableasarationalfunctionofejΩ (orcanmodelitthatway),thenwe
canobtainH(ejΩ)byappropriatefactorizationof
|H(ejΩ) |2 .
EXAMPLE10.1 Whiteningfilter
Supposethat
Sxx(ejΩ) =
5
4 −cos(Ω). (10.18) Then,towhitenx(t),werequirea stableLTIfilterforwhich
|H(ejΩ)|2 = (1− 1 , (10.19) 1ejΩ)(1−1 e−jΩ) 2 2 orequivalently, 1 H(z)H(1/z) = (1−1z)(1−1 z−1) . (10.20) 2 2
ThefilterisconstrainedtobestableinordertoproduceaWSSoutput. Onechoice ofH(z)thatresultsina causalfilteris
1
H(z) = 1 , (10.21)
1−2 z−1
withregionofconvergence(ROC)givenby|z|>1 2 . Thissystemfunctioncouldbe multipliedbythesystemfunctionA(z)ofanyallpasssystem,i.e.,asystemfunction satisfyingA(z)A(z−1)=1, and still produce thesame whitening action, because |A(ejΩ)|2 =1.
10.3 SAMPLING OFBANDLIMITED RANDOMPROCESSES
A WSS random process is termed bandlimited if its PSD is bandlimited, i.e., is zero for frequencies outside some finite band. For deterministic signals that are bandlimited, we can sample at or above the Nyquist rate and recover the signal exactly. Weexaminehere whether wecan do thesame with bandlimitedrandom processes.
InthediscussionofsamplingandDTprocessingofCTsignalsinyourpriorcourses, thederivationsanddiscussionrelyheavilyonpicturingtheeffectin thefrequency
domain, i.e.,tracking theFouriertransformofthecontinuous-timesignal through the C/D(sampling)and D/C(reconstruction)process. Whilethe argumentscan alternatively be carried out directly in the time domain, for deterministic finite-energysignalsthefrequencydomaindevelopmentseemsmore conceptuallyclear. As you might expect, results similar to the deterministic case hold for the re constructionofbandlimitedrandomprocessesfrom samples. However,sincethese stochasticsignalsdonotpossessFouriertransformsexceptinthegeneralizedsense, we carry out thedevelopment for random processesdirectly in the time domain. Anessentiallyparallel argumentcouldhave been usedin thetime domainforde terministicsignals(byexaminingthetotalenergyinthereconstructionerrorrather than theexpectedinstantaneous powerin thereconstruction error,which iswhat wefocus onbelow).
Thebasicsamplingandbandlimitedreconstructionprocessshouldbefamiliarfrom your prior studies in signals and systems, and is depicted in Figure 10.4 below. In this figure we have explicitly used bold upper-case symbols for the signals to underscorethat theyarerandomprocesses.
� � C/D Xc(t) X[n] =Xc(nT) � T X[n] � D/C �Yc(t) =Pn+=∞−∞X[n] sinc(t−TnT) �
wheresincx= sinπx
T πx
FIGURE10.4 C/DandD/Cforrandomprocesses.
Forthedeterministiccase,weknowthatifxc(t)isbandlimitedtolessthanTπ,then
withtheD/Creconstructiondefinedas
yc(t) =
X
x[n] sinc(t−nT) (10.22) T
n
itfollowsthatyc(t) =xc(t). Inthecaseofrandomprocesses,whatweshowbelow
isthat,undertheconditionthatSxcxc(jω),thepowerspectraldensityofXc(t),is
bandlimitedto lessthat Tπ, themeansquare valueof theerrorbetweenXc(t)and
Yc(t)iszero;i.e.,if
π Sxcxc(jω) = 0 |w| ≥
then
=E{[Xc(t)−Yc(t)]2}= 0. (10.24)
E △
This, in effect,saysthat thereis “zeropower”in theerror. (An alternativeproof totheonebelow isoutlinedinProblem13at theend ofthischapter.)
Todeveloptheaboveresult,weexpandtheerrorandusethedefinitionsoftheC/D (or sampling) and D/C(or idealbandlimited interpolation)operations in Figure 10.4toobtain
(t)Xc (10.25)
E=E{X2 c(t)}+E{Yc2(t)} −2E{Yc (t)}.
Wefirstconsiderthelastterm,E{Yc(t)Xc(t)}:
+∞ t−nT E{Yc(t)Xc(t)}=E{ X Xc(nT)sinc( )Xc(t)} T n=−∞ +∞ nT −t = X Rxcxc(nT −t)sinc( T ) (10.26) n=−∞ (10.27) where, in thelast expression, wehave invoked thesymmetry of sinc(.)to change thesignofitsargumentfrom theexpressionthat precedesit.
Equation(10.26)canbeevaluatedusingParseval’srelationindiscretetime,which statesthat
+∞ 1 Z π
X
v[n]w[n] = V(ejΩ)W∗(ejΩ)dΩ (10.28)
n=∞ 2π −π
Toapply Parseval’s relation, notethat Rxcxc(nT−t)can be viewedas theresult
of the C/D or sampling process depicted in Figure 10.5, in which the input is consideredtobe afunctionofthevariableτ:
Rxcxc(τ−t) � C/D �Rx cxc(nT−t) � T FIGURE10.5 C/Dappliedto Rxcxc(τ−t).
TheCTFT (inthe variableτ) ofRxcxc(τ−t)is e−
jωtS
xcxc(jω), andsincethis is
bandlimitedto| |ω < Tπ,theDTFTofitssampledversionRxcxc(nT−t)is
−jΩt
1 Ω
e T Sx
cxc(j ) (10.29)
intheinterval|Ω|< π. Similarly,theDTFTofsinc(nT−t)is πe −j T Ωt .Consequently, T
under theconditionthatSxcxc(jω)isbandlimitedto| |ω < T,
1 Z π jΩ E{Yc(t)Xc(t)}= 2πT Sxcxc(T )dΩ −π 1 Z (π/T) = Sxcxc(jω)dω 2π −(π/T ) =Rxcxc(0)=E{Xc2(t)} (10.30)
Next,weexpandthemiddletermin equation(10.25): E{Yc2(t)}=E{ X X Xc(nT)Xc(mT) sinc(t−nT) sinc(t−mT)} T T n m =X XRxcxc(nT −mT) sinc( t−mT ) sinc(t−mT). (10.31) T T n m
Withthesubstitutionn−m=r,wecan express10.31as
E{Yc2(t)}= X Rxcxc(rT) X sinc(t−mT) sinc(t−mT −rT). (10.32) T T r m
Usingtheidentity
X
sinc(n−θ1)sinc(n−θ2)=sinc(θ2 −θ1), (10.33)
n
which again comes from Parseval’s theorem (see Problem 12 at the end of this chapter),wehave (rT) sinc(r) E{Yc2(t)}= X Rxcxc r =Rxcxc(0)=E{X 2 c } (10.34)
since sinc(r) =1 if r =0 and zerootherwise. Substituting 10.31 and 10.34 into 10.25,weobtaintheresultthat E=0,as desired.
http://ocw.mit.edu
6.011 Introduction to Communication, Control, and Signal Processing
Spring 2010
