The Moment Generating Function and Factorial Moment Generating Function of a Random Variable

1

Then

ii) Show that φYc(t) = φX1, . . . , Xk(c₁t, . . . , c_kt), t ∈ , and φX1, . . . , Xk(c₁, . . . , c_k)

=φYc(1);

ii) Conclude that the distribution of the X’s determines the distribution of Y_c for every c_j ∈ , j = 1, . . . , k. Conversely, the distribution of the X’s is determined by the distribution of Y_c for every c_j∈, j = 1, . . . , k.

6.5 The Moment Generating Function and Factorial Moment Generating Function of a Random Variable

The ch.f. of an r.v. or an r. vector is a function defined on the entire real line and taking values in the complex plane. Those readers who are not well versed in matters related to complex-valued functions may feel uncomfortable in dealing with ch.f.’s. There is a partial remedy to this potential problem, and that is to replace a ch.f. by an entity which is called moment generating function. However, there is a price to be paid for this: namely, a moment generating function may exist (in the sense of being finite) only for t= 0. There are cases where it exists for t’s lying in a proper subset of  (containing 0), and yet other cases, where the moment generating function exists for all real t. All three cases will be illustrated by examples below.

First, consider the case of an r.v. X. Then the moment generating function (m.g.f.) M_X (or just M when no confusion is possible) of a random variable X, which is also called the Laplace transform of f, is defined by M_X(t)= E(e^tX), t ∈ , if this expectation exists. For t = 0, MX(0) always exists and equals 1. However, it may fail to exist for t ≠ 0. If MX(t) exists, then formally φX(t)= MX(it) and therefore the m.g.f. satisfies most of the properties analo-gous to properties (i)–(vii) cited above in connection with the ch.f., under suitable conditions. In particular, property (vii) in Theorem 1 yields

d

dt X t

n n

nM t

( )

=0 ⁼E X

( )

, provided Lemma D applies. In fact, d

dt M t d

dt Ee E d

dt e

E X e E X

n

n X

t

n n

tX

t

n n

tX

t n tX

t

n

( )

⁼

( )

^{= ⎛}_⎝⎜ ^⎞_⎠⎟

=

( )

⁼

( )

= = =

=

0 0 0

0

. This is the property from which the m.g.f. derives its name.

Here are some examples of m.g.f.’s. It is instructive to derive them in order to see how conditions are imposed on t in order for the m.g.f. to be finite. It so happens that part (vii) of Theorem 1, as it would be formulated for an m.g.f., is applicable in all these examples, although no justification will be supplied.

6.5.1 The M.G.F.’s of Some R.V.’s

1. If X∼ B(n, p), then MX(t)= (pe^t+ q)ⁿ, t∈. Indeed,

6.5 The Moment Generating Function 155 expression is equal to

1

and distribution, and its mean and variance; namely

M t

t t EX X

X

( )

⁼_λ^λ₋ ^{, <λ, =} _λ^{1, σ2}

( )

⁼ _λ^{12 .}

5. Let X have the Cauchy distribution with parameters μ and σ, and without loss of generality, let μ = 0, σ = 1. Then the MX(t) exists only for t= 0. in the integral, we again reach the conclusion that M_X(t) = ∞ (see Exercise 6.5.9).

REMARK 4 The examples just discussed exhibit all three cases regarding the existence or nonexistence of an m.g.f. In Examples 1 and 3, the m.g.f.’s exist for all t∈; in Examples 2 and 4, the m.g.f.’s exist for proper subsets of ; and in Example 5, the m.g.f. exists only for t= 0.

For an r.v. X, we also define what is known as its factorial moment generating function. More precisely, the factorial m.g.f. ηX (or just η when no confusion is possible) of an r.v. X is defined by:

ηX

X X

t E t t E t

( )

⁼

( )

^{, ∈, if}

( )

^exists.

This function is sometimes referred to as the Mellin or Mellin–Stieltjes trans-form of f. Clearly, ηX(t)= MX(log t) for t> 0.

Formally, the nth factorial moment of an r.v. X is taken from its factorial m.g.f. by differentiation as follows:

6.5 The Moment Generating Function 157

provided Lemma D applies, so that the interchange of the order of differen-tiation and expectation is valid. Hence

d

REMARK 5 The factorial m.g.f. derives its name from the property just estab-lished. As has already been seen in the first two examples in Section 2 of Chapter 5, factorial moments are especially valuable in calculating the vari-ance of discrete r.v.’s. Indeed, since

σ²

( )

X ⁼E X

( )

^{2 −}

^{( )}

EX ^{2, and} E X

( )

^{2 =}E X X

[ ⁽

⁻1

⁾ ]

⁺E X

^{( )}

^, we get

σ²

( )

X ⁼E X X

[ (

⁻1

) ]

⁺E X

^{( )}

⁻

^{( )}

EX ^2;

that is, an expression of the variance of X in terms of derivatives of its factorial m.g.f. up to order two.

Below we derive some factorial m.g.f.’s. Property (9) (for n= 2) is valid in all these examples, although no specific justification will be provided.

6.5.2 The Factorial M.G.F.’s of some R.V.’s

1. If X∼ B(n, p), then ηX(t)= (pt + q)ⁿ, t∈. In fact,

Hence analo-gous to (i′)–(vii′), (viii) in Theorem 1′ hold true under suitable conditions. In particular,

Below, we present two examples of m.g.f.’s of r. vectors.

6.5.3 The M.G.F.’s of Some R. Vectors

1. If the r.v.’s X1, . . . , Xk have jointly the Multinomial distribution with

2. If the r.v.’s X1 and X2 have the Bivariate Normal distribution with parametersμ1,μ2,σ²1,σ²2 and ρ, then their joint m.g.f. is An analytical derivation of this formula is possible, but we prefer to use the matrix approach, which is more elegant and compact. Recall that the joint p.d.f. of X1 and X2 is given by

6.5 The Moment Generating Function 159

Therefore the p.d.f. is written as follows in matrix notation:

f x

( )

^{= ⎡−}

( )

⁻

( )

⁻

The exponent may be written as follows:

μμ′t+ t′ t μμ′t t′ t t x′ x μμ′ ¹ x μμ

Focus on the quantity in the bracket, carry out the multiplication, and observe thatΣΣΣΣΣ′ = ΣΣΣΣΣ, (ΣΣΣΣΣ⁻⁻⁻⁻⁻¹)′ = ΣΣΣΣΣ⁻⁻⁻⁻⁻¹, x′′′′′t = t′′′′′x,μμμμμ′t = t′μμμμμ, and x′′′′′ΣΣΣΣΣ⁻⁻⁻⁻⁻¹μμμμμ = μμμμμ′ΣΣΣΣΣ⁻⁻⁻⁻⁻¹x, to obtain

2μμ′t+t^′′∑∑t−2t x^′′ + −

( )

x μμ′^{∑∑−−1}

( )

x⁻μμ ^{= −}

[

x

(

μμ^{+∑∑ ∑}t

)

′^∑−−1

(

x⁻

(

μμ^+∑∑t

) ) ]

^{. (14)}

By means of (13) and (14), the m.g.f. in (12) becomes

However, the second factor above is equal to 1, since it is the integral of a Bivariate Normal distribution with mean vector μμμμμ + ΣΣΣΣΣt and covariance matrix ΣΣΣΣΣ. Thus

it follows that the m.g.f. is, indeed, given by (11).

Exercises

6.5.1 Derive the m.g.f. of the r.v. X which denotes the number of spots that turn up when a balanced die is rolled.

6.5.2 Let X be an r.v. with p.d.f. f given in Exercise 3.2.13 of Chapter 3.

Derive its m.g.f. and factorial m.g.f., M(t) and η(t), respectively, for those t’s for which they exist. Then calculate EX, E[X(X− 1)] and σ²(X), provided they are finite.

6.5.3 Let X be an r.v. with p.d.f. f given in Exercise 3.2.14 of Chapter 3.

Derive its m.g.f. and factorial m.g.f., M(t) and η(t), respectively, for those t’s for which they exist. Then calculate EX, E[X(X− 1)] and σ²(X), provided they are finite.

6.5 The Moment Generating Function 161

6.5.4 Let X be an r.v. with p.d.f. f given by f(x) = λe^{−λ(x −α)}I_(α,∞)(x). Find its m.g.f. M(t) for those t’s for which it exists. Then calculate EX and σ²(X), provided they are finite.

6.5.5 Let X be an r.v. distributed as B(n, p). Use its factorial m.g.f. in order to calculate its kth factorial moment. Compare with Exercise 5.2.1 in Chapter 5.

6.5.6 Let X be an r.v. distributed as P(λ). Use its factorial m.g.f. in order to calculate its kth factorial moment. Compare with Exercise 5.2.4 in Chapter 5.

6.5.7 Let X be an r.v. distributed as Negative Binomial with parameters r and p.

i) Show that its m.g.f and factorial m.g.f., M(t) and η(t), respectively, are given by

M t p

qe

t q t p

qt

t q

X

r

t r X

r

( )

⁼ r

(

¹−

)

^{< −}

^{( )}

⁼

⁽

¹⁻

⁾

^<

, log , η , 1;

ii) By differentiation, show that EX= rq/p and σ²(X)= rq/p²;

iii) Find the quantities mentioned in parts (i) and (ii) for the Geometric distribution.

6.5.8 Let X be an r.v. distributed as U(α, β).

ii) Show that its m.g.f., M, is given by M t e e

t

t t

( )

⁼

(

⁻₋

)

β α

β α ;

ii) By differentiation, show that EX=^{α β+}₂ and σ²(X)=( )^{α β− 2}

12 .

6.5.9 Refer to Example 3 in the Continuous case and show that MX(t)= ∞ for t< 0 as asserted there.

6.5.10 Let X be an r.v. with m.g.f. M given by M(t)= e^{αt +βt2}, t∈ (α ∈ , β > 0). Find the ch.f. of X and identify its p.d.f. Also use the ch.f. of X in order to calculate EX⁴.

6.5.11 For an r.v. X, define the function γ by γ(t) = E(1 + t)^Xfor those t’s for which E(1+ t)^X is finite. Then, if the nth factorial moment of X is finite, show that

dⁿ dtⁿ t E X X X n

( )

γ

^{( )}

t= ⁼

[ ⁽

⁻

⁾

^{⋅ ⋅ ⋅}

⁽

^{− +}

⁾ ]

0

1 1 .

6.5.12 Refer to the previous exercise and let X be P(λ). Derive γ(t) and use it in order to show that the nth factorial moment of X is λⁿ.

Exercises 161

6.5.13 Let X be an r.v. with m.g.f. M and set K(t)= logM(t) for those t’s for which M(t) exists. Furthermore, suppose that EX=μ and σ²(X)=σ² are both finite. Then show that

d

dtK t d

dt K t

t t

( )

⁼

( )

⁼

=0 =

2 2

0

μ and σ2.

(The function K just defined is called the cumulant generating function of X.) 6.5.14 Let X be an r.v. such that EXⁿ is finite for all n= 1, 2, . . . . Use the expansion

e x

n

x

n

n

=

=

∑

∞ _! 0

in order to show that, under appropriate conditions, one has that the m.g.f. of X is given by

M t EX t

n

n n

( )

⁼n

∑

^∞₌

( )

_!^.

0

6.5.15 If X is an r.v. such that EXⁿ= n!, then use the previous exercise in order to find the m.g.f. M(t) of X for those t’s for which it exists. Also find the ch.f. of X and from this, deduce the distribution of X.

6.5.16 Let X be an r.v. such that

EX k

k EX

k k

k

2 2 2 1

2 0

=

( )

⋅ ! ⁺ =

!, ,

k= 0, 1, . . . . Find the m.g.f. of X and also its ch.f. Then deduce the distribution of X. (Use Exercise 6.5.14)

6.5.17 Let X1, X2 be two r.v.’s with m.f.g. given by

M t₁ t₂ e^{t t} e^t e^t t t

2

1 2

1

3 1 1

6

1 2 1 2

( )

, ^=⎡_⎣^⎢

(

^{+ +}

)

⁺

(

⁺

)

^⎤_⎦^{⎥ , , ∈.}

Calculate EX₁,σ²(X₁) and Cov(X₁, X₂), provided they are finite.

6.5.18 Refer to Exercise 4.2.5. in Chapter 4 and find the joint m.g.f.

M(t₁, t₂, t₃) of the r.v.’s X₁, X₂, X₃ for those t₁, t₂, t₃ for which it exists. Also find their joint ch.f. and use it in order to calculate E(X₁X₂X₃), provided the assumptions of Theorem 1′ (vii′) are met.

6.5.19 Refer to the previous exercise and derive the m.g.f. M(t) of the r.v.

g(X₁, X₂, X₃)= X1+ X2+ X3 for those t’s for which it exists. From this, deduce the distribution of g.

6.5 The Moment Generating FunctionExercises 163163

6.5.20 Let X₁, X₂ be two r.v.’s with m.g.f. M and set K(t₁, t₂)= logM(t1, t₂) for those t₁, t₂ for which M(t₁, t₂) exists. Furthermore, suppose that expectations, variances, and covariances of these r.v.’s are all finite. Then show that for j= 1, 2, ii) Write out the joint m.g.f. of Xi, Xj, X, and by differentiation, determine the

E(XiXj);

ii) Calculate the covariance of X_i, X_j, Cov(X_i, X_j), and show that it is negative.

6.5.22 If the r.v.’s X₁ and X₂ have the Bivariate Normal distribution with parameters μ1, μ2, σ²1, σ²2 and ρ, show that Cov(X1, X₂) ≥ 0 if ρ ≥ 0, and all X₁, X₂ in , are said to be positively quadrant dependent or negatively quadrant dependent, respectively. In particular, if X₁ and X₂ have the Bivariate Normal distribution, it can be seen that they are positively quadrant depend-ent or negatively quadrant dependdepend-ent according to whether ρ ≥ 0 or ρ < 0.

6.5.23 Verify the validity of relation (13).

6.5.24

ii) If the r.v.’s X₁ and X₂ have the Bivariate Normal distribution with param-etersμ1,μ2,σ²1,σ²2 and ρ, use their joint m.g.f. given by (11) and property (10) in order to determine E(X₁X₂);

ii) Show that ρ is, indeed, the correlation coefficient of X1 and X₂.

6.5.25 Both parts of Exercise 6.4.1 hold true if the ch.f.’s involved are re-placed by m.g.f.’s, provided, of course, that these m.g.f.’s exist.

ii) Use Exercise 6.4.1 for k= 2 and formulated in terms of m.g.f.’s in order to

show that the r.v.’s X₁ and X₂ have a Bivariate Normal distribution if and only if for every c₁, c₂∈, Yc= c1X₁+ c2X₂ is normally distributed;

ii) In either case, show that c₁X₁+ c2X₂+ c3 is also normally distributed for any c₃∈.

164

In document A Course in Mathematical Statistics 0125993153 (Page 174-185)