3.3 Characteristic Functions
3.3.1 Definition, Inversion Formula
IfX is a random variable we define its characteristic function (ch.f.) by
ϕ(t) =EeitX=EcostX+iEsintX
The last formula requires taking the expected value of a complex valued random variable but as the second equality may suggest no new theory is required. If Z is complex valued we define EZ =E( ReZ) +iE( ImZ) where Re (a+bi) =a is the real partand Im (a+bi) =bis theimaginary part. Some other definitions we will need are: themodulusof the complex numberz=a+biis|a+bi|= (a2+b2)1/2,
and thecomplex conjugateofz=a+bi, ¯z=a−bi.
Theorem 3.3.1. All characteristic functions have the following properties: (a)ϕ(0) = 1,
(b)ϕ(−t) =ϕ(t),
(c)|ϕ(t)|=|EeitX| ≤E|eitX|= 1
(d)|ϕ(t+h)−ϕ(t)| ≤E|eihX−1|, soϕ(t)is uniformly continuous on (−∞,∞).
(e)Eeit(aX+b)=eitbϕ(at)
Proof. (a) is obvious. For (b) we note that
ϕ(−t) =E(cos(−tX) +isin(−tX)) =E(cos(tX)−isin(tX)) (c) follows from Exercise 1.6.2 sinceϕ(x, y) = (x2+y2)1/2 is convex.
|ϕ(t+h)−ϕ(t)|=|E(ei(t+h)X−eitX)|
≤E|ei(t+h)X−eitX|=E|eihX−1|
so uniform convergence follows from the bounded convergence theorem. For (e) we noteEeit(aX+b)=eitbEei(ta)X =eitbϕ(at).
The main reason for introducing characteristic functions is the following:
Theorem 3.3.2. If X1 and X2 are independent and have ch.f.’s ϕ1 and ϕ2 then X1+X2 has ch.f.ϕ1(t)ϕ2(t).
Proof.
Eeit(X1+X2)=E(eitX1eitX2) =EeitX1EeitX2
The next order of business is to give some examples.
Example 3.3.1. Coin flips. IfP(X = 1) =P(X =−1) = 1/2 then
EeitX= (eit+e−it)/2 = cost
Example 3.3.2. Poisson distribution. IfP(X =k) =e−λλk/k! fork= 0,1,2, . . .
then EeitX= ∞ X k=0 e−λλ keitk k! = exp(λ(e it −1)) Example 3.3.3. Normal distribution
Density (2π)−1/2exp(−x2/2)
Ch.f. exp(−t2/2)
Combining this result with (e) of Theorem 3.3.1, we see that a normal distribution with mean µ and variance σ2 has ch.f. exp(iµt−σ2t2/2). Similar scalings can be
applied to other examples so we will often just give the ch.f. for one member of the family. Physics Proof Z eitx(2π)−1/2e−x2/2dx=e−t2/2 Z (2π)−1/2e−(x−it)2/2dx
The integral is 1 since the integrand is the normal density with meanitand variance 1.
Math Proof. Now that we have cheated and figured out the answer we can verify it by a formal calculation that gives very little insight into why it is true. Let
ϕ(t) =
Z
eitx(2π)−1/2e−x2/2dx=
Z
costx(2π)−1/2e−x2/2dx
sinceisintxis an odd function. Differentiating with respect tot(referring to Theorem A.5.1 for the justification) and then integrating by parts gives
ϕ0(t) = Z −xsintx(2π)−1/2e−x2/2dx =− Z tcostx(2π)−1/2e−x2/2dx=−tϕ(t) This implies d dt{ϕ(t) exp(t 2/2)}= 0 soϕ(t) exp(t2/2) =ϕ(0) = 1.
In the next three examples, the density is 0 outside the indicated range. Example 3.3.4. Uniform distribution on (a, b)
Density 1/(b−a) x∈(a, b) Ch.f. (eitb−eita)/ it(b−a)
In the special casea=−c,b=c the ch.f. is (eitc−e−itc)/2cit= (sinct)/ct.
Proof. Once you recall that Rabeλxdx = (eλb−eλa)/λ holds for complex λ, this is immediate.
3.3. CHARACTERISTIC FUNCTIONS 93 Example 3.3.5. Triangular distribution
Density 1− |x| x∈(−1,1) Ch.f. 2(1−cost)/t2
Proof. To see this, notice that ifXandY are independent and uniform on (−1/2,1/2) then X +Y has a triangular distribution. Using Example 3.3.4 now and Theorem 3.3.2 it follows that the desired ch.f. is
{(eit/2−e−it/2)/it}2={2 sin(t/2)/t}2
Using the trig identity cos 2θ= 1−2 sin2θwithθ=t/2 converts the answer into the form given above.
Example 3.3.6. Exponential distribution
Density e−x x∈(0,∞)
Ch.f. 1/(1−it) Proof. Integrating gives
Z ∞ 0 eitxe−xdx= e (it−1)x it−1 ∞ 0 = 1 1−it since exp((it−1)x)→0 asx→ ∞.
For the next result we need the following fact which follows from the fact that
R
f d(µ+ν) =R
f dµ+R
f dν.
Lemma 3.3.3. IfF1, . . . , Fn have ch.f.ϕ1, . . . , ϕn andλi≥0haveλ1+. . .+λn= 1
thenPn
i=1λiFi has ch.f. P n i=1λiϕi.
Example 3.3.7. Bilateral exponential
Density 12e−|x| x∈(−∞,∞) Ch.f. 1/(1 +t2)
ProofThis follows from Lemma 3.3.3 withF1 the distribution of an exponential ran- dom variable X, F2 the distribution of −X, and λ1 = λ2 = 1/2 then using (b) of Theorem 3.3.1 we see the desired ch.f. is
1 2(1−it)+ 1 2(1 +it) = (1 +it) + (1−it) 2(1 +t2) = 1 (1 +t2)
Exercise 3.3.1. Show that ifϕis a ch.f. then Reϕand|ϕ|2 are also.
The first issue to be settled is that the characteristic function uniquely determines the distribution. This and more is provided by
Theorem 3.3.4. The inversion formula. Let ϕ(t) = R
eitxµ(dx) where µ is a
probability measure. If a < bthen
lim T→∞(2π) −1 Z T −T e−ita−e−itb it ϕ(t)dt=µ(a, b) + 1 2µ({a, b})
Remark. The existence of the limit is part of the conclusion. Ifµ=δ0, a point mass at 0, ϕ(t)≡1. In this case, ifa=−1 andb= 1, the integrand is (2 sint)/tand the integral does not converge absolutely.
Proof. Let IT = Z T −T e−ita−e−itb it ϕ(t)dt= Z T −T Z e−ita−e−itb it e itxµ(dx)dt
The integrand may look bad neart= 0 but if we observe that
e−ita−e−itb
it =
Z b
a
e−itydy
we see that the modulus of the integrand is bounded byb−a. Sinceµis a probability measure and [−T, T] is a finite interval it follows from Fubini’s theorem, cos(−x) = cosx, and sin(−x) =−sinxthat
IT = Z Z T −T e−ita−e−itb it e itxdt µ(dx) = Z (Z T −T sin(t(x−a)) t dt− Z T −T sin(t(x−b)) t dt ) µ(dx) IntroducingR(θ, T) =RT
−T(sinθt)/t dt, we can write the last result as
(∗) IT =
Z
{R(x−a, T)−R(x−b, T)}µ(dx) If we letS(T) =RT
0 (sinx)/x dxthen forθ >0 changing variablest=x/θshows that R(θ, T) = 2
Z T θ
0
sinx
x dx= 2S(T θ)
while forθ < 0,R(θ, T) =−R(|θ|, T). Introducing the function sgnx, which is 1 if
x >0,−1 ifx <0, and 0 ifx= 0, we can write the last two formulas together as
R(θ, T) = 2( sgnθ)S(T|θ|)
AsT → ∞,S(T)→π/2 (see Exercise 1.7.5), so we haveR(θ, T)→πsgnθand
R(x−a, T)−R(x−b, T)→ 2π a < x < b π x=aorx=b 0 x < a orx > b
|R(θ, T)| ≤ 2 supyS(y) < ∞, so using the bounded convergence theorem with (∗) implies
(2π)−1IT →µ(a, b) +
1
2µ({a, b}) proving the desired result.
3.3. CHARACTERISTIC FUNCTIONS 95 Exercise 3.3.2. (i) Imitate the proof of Theorem 3.3.4 to show that
µ({a}) = lim T→∞ 1 2T Z T −T e−itaϕ(t)dt
(ii) IfP(X ∈hZ) = 1 where h >0 then its ch.f. has ϕ(2π/h+t) =ϕ(t) so
P(X =x) = h 2π
Z π/h
−π/h
e−itxϕ(t)dt forx∈hZ
(iii) IfX =Y +b thenEexp(itX) =eitbEexp(itY). So if P(X ∈b+hZ) = 1, the
inversion formula in (ii) is valid forx∈b+hZ.
Two trivial consequences of the inversion formula are:
Exercise 3.3.3. Ifϕis real then X and−X have the same distribution.
Exercise 3.3.4. IfXi,i= 1,2 are independent and have normal distributions with mean 0 and variance σi2, thenX1+X2 has a normal distribution with mean 0 and varianceσ12+σ22.
The inversion formula is simpler whenϕis integrable, but as the next result shows this only happens when the underlying measure is nice.
Theorem 3.3.5. IfR
|ϕ(t)|dt <∞ thenµhas bounded continuous density
f(y) = 1 2π
Z
e−ityϕ(t)dt
Proof. As we observed in the proof of Theorem 3.3.4
e−ita−e−itb it = Z b a e−itydy ≤ |b−a|
so the integral in Theorem 3.3.4 converges absolutely in this case and
µ(a, b) +1 2µ({a, b}) = 1 2π Z ∞ −∞ e−ita−e−itb it ϕ(t)dt≤ (b−a) 2π Z ∞ −∞ |ϕ(t)|dt
The last result impliesµhas no point masses and
µ(x, x+h) = 1 2π Z e−itx−e−it(x+h) it ϕ(t)dt = 1 2π Z Z x+h x e−itydy ! ϕ(t)dt = Z x+h x 1 2π Z e−ityϕ(t)dt dy
by Fubini’s theorem, so the distributionµhas density function
f(y) = 1 2π
Z
e−ityϕ(t)dt
Exercise 3.3.5. Give an example of a measure µ with a density but for which
R
|ϕ(t)|dt=∞.Hint: Two of the examples above have this property.
Exercise 3.3.6. Show that ifX1, . . . , Xn are independent and uniformly distributed on (−1,1), then for n≥2,X1+· · ·+Xn has density
f(x) = 1
π
Z ∞
0
(sint/t)ncostx dt
Although it is not obvious from the formula,f is a polynomial in each interval (k, k+ 1),k∈Zand vanishes on [−n, n]c.
Theorem 3.3.5 and the next result show that the behavior ofϕat infinity is related to the smoothness of the underlying measure.
Exercise 3.3.7. SupposeXandY are independent and have ch.f.ϕand distribution
µ. Apply Exercise 3.3.2 toX−Y and use Exercise 2.1.8 to get lim T→∞ 1 2T Z T −T |ϕ(t)|2dt=P(X−Y = 0) =X x µ({x})2
Remark. The last result implies that if ϕ(t)→0 ast→ ∞,µhas no point masses. Exercise 3.3.13 gives an example to show that the converse is false. The Riemann- Lebesgue Lemma (Exercise 1.4.4) shows that ifµhas a density,ϕ(t)→0 ast→ ∞.
Applying the inversion formula Theorem 3.3.5 to the ch.f. in Examples 3.3.5 and 3.3.7 gives us two more examples of ch.f. The first one does not have an official name so we gave it one to honor its role in the proof of Polya’s criterion, see Theorem 3.3.10. Example 3.3.8. Polya’s distribution
Density (1−cosx)/πx2
Ch.f. (1− |t|)+ Proof. Theorem 3.3.5 implies
1 2π Z 2(1−coss) s2 e −isyds= (1− |y|)+ Now lets=x,y=−t.
Example 3.3.9. The Cauchy distribution Density 1/π(1 +x2)
Ch.f. exp(−|t|) Proof. Theorem 3.3.5 implies
1 2π Z 1 1 +s2e −isyds= 1 2e −|y|
Now lets=x,y=−tand multiply each side by 2.
Exercise 3.3.8. Use the last result to conclude that ifX1, X2, . . . are independent
and have the Cauchy distribution, then (X1+· · ·+Xn)/nhas the same distribution
3.3. CHARACTERISTIC FUNCTIONS 97
3.3.2
Weak Convergence
Our next step toward the central limit theorem is to relate convergence of character- istic functions to weak convergence.
Theorem 3.3.6. Continuity theorem. Letµn,1≤n≤ ∞be probability measures
with ch.f. ϕn. (i) Ifµn ⇒µ∞ thenϕn(t)→ϕ∞(t) for all t. (ii) If ϕn(t)converges
pointwise to a limit ϕ(t) that is continuous at 0, then the associated sequence of distributions µn is tight and converges weakly to the measure µ with characteristic function ϕ.
Remark. To see why continuity of the limit at 0 is needed in (ii), letµnhave a normal distribution with mean 0 and variancen. In this case ϕn(t) = exp(−nt2/2)→0 for t 6= 0, and ϕn(0) = 1 for all n, but the measures do not converge weakly since
µn((−∞, x])→1/2 for allx.
Proof. (i) is easy. eitxis bounded and continuous so ifµn⇒µ∞ then Theorem 3.2.3
impliesϕn(t)→ϕ∞(t). To prove (ii), our first goal is to prove tightness. We begin
with some calculations that may look mysterious but will prove to be very useful.
Z u
−u
1−eitxdt= 2u−
Z u
−u
(costx+isintx)dt= 2u−2 sinux
x
Dividing both sides by u, integrating µn(dx), and using Fubini’s theorem on the left-hand side gives
u−1 Z u −u (1−ϕn(t))dt= 2 Z 1−sinux ux µn(dx)
To bound the right-hand side, we note that
|sinx|= Z x 0 cos(y)dy ≤ |x| for allx
so we have 1−(sinux/ux)≥0. Discarding the integral over (−2/u,2/u) and using
|sinux| ≤1 on the rest, the right-hand side is
≥2 Z |x|≥2/u 1− 1 |ux| µn(dx)≥µn({x:|x|>2/u}) Sinceϕ(t)→1 ast→0, u−1 Z u −u (1−ϕ(t))dt→0 asu→0
Pick uso that the integral is< . Sinceϕn(t)→ϕ(t) for eacht, it follows from the bounded convergence theorem that forn≥N
2≥u−1
Z u
−u
(1−ϕn(t))dt≥µn{x:|x|>2/u}
Sinceis arbitrary, the sequenceµn is tight.
To complete the proof now we observe that if µn(k) ⇒ µ, then it follows from
the first sentence of the proof thatµhas ch.f. ϕ. The last observation and tightness imply that every subsequence has a further subsequence that converges toµ. I claim
that this implies the whole sequence converges to µ. To see this, observe that we have shown that if f is bounded and continuous then every subsequence of R
f dµn
has a further subsequence that converges toR
f dµ, so Theorem 2.3.3 implies that the whole sequence converges to that limit. This showsR
f dµn→
R
f dµfor all bounded continuous functionsf so the desired result follows from Theorem 3.2.3.
Exercise 3.3.9. Suppose thatXn⇒X andXn has a normal distribution with mean 0 and varianceσ2n. Prove thatσ2n→σ2∈[0,∞).
Exercise 3.3.10. Show that ifXnandYnare independent for 1≤n≤ ∞,Xn⇒X∞,
andYn⇒Y∞, then Xn+Yn⇒X∞+Y∞.
Exercise 3.3.11. Let X1, X2, . . . be independent and let Sn =X1+· · ·+Xn. Let
ϕj be the ch.f. ofXj and suppose thatSn→S∞ a.s. ThenS∞has ch.f.Q∞j=1ϕj(t).
Exercise 3.3.12. Using the identity sint = 2 sin(t/2) cos(t/2) repeatedly leads to (sint)/t = Q∞
m=1cos(t/2
m). Prove the last identity by interpreting each side as a
characteristic function.
Exercise 3.3.13. LetX1, X2, . . . be independent taking values 0 and 1 with proba-
bility 1/2 each. X = 2P
j≥1Xj/3j has the Cantor distribution. Compute the ch.f.ϕ
ofX and notice thatϕhas the same value att= 3kπfork= 0,1,2, . . .