Moment Generating Function

The Moment Generating Function (mgf) is a function that on being differentiated gives us the raw moments of a probability distribution

Given a random variable X, the r^th raw moment is defined as E[X^r] that is the expectation of the random variable raised to the r^th power. We use the symbol \mu_r' to denote the r^th raw moment.

Definition of Moment Generating Function:

For a random variable X the moment generating function(mgf) is defined as,

M_X(t)=E[e^{tx}] where, E denotes the expected value.

We see that the moment generating function is a function of ‘t’.

How to calculate the moment generating function?

For a discrete random variable X, with probability mass function p(x) we calculate the moment generating function as,

M_X(t)=E[e^{tx}]= \sum e^{tx}p(x) where, the sum runs over all values taken by X.

For a continuous random variable X, with probability density function f(x) we calculate the moment generating function as,

M_X(t)=E[e^{tx}]= \int e^{tx}f(x)dx where, the integral is taken over the range of values taken by X.

How do to find the raw moments using the Moment Generating Function?

We can find the r^th raw moment by differentiating the moment generating function M_X(t) ‘r’ times and then setting t=0.

Thus we see that,

\mu_r' = r^th raw moment = E[X^r] = \frac{d^r}{dt^r} M_X(t)|_{t=o}

Proof:

Let us see the proof for why differentiating the moment generating function gives us the raw moments.

The moment generating function mgf is defined as, M_X(t)=E[e^{tx}]

By using the Taylor expansion for the exponential function we get that, M_X(t)=E[e^{tx}] = E[1 + \frac{tx}{1!} + \frac{(tx)^2}{2!} \ldots + \frac{(tx)^r}{r!} + \ldots]

We know that expectation can be exchanged with sums therefore we get, M_X(t) = 1 + \frac{t}{1!} E[X] + \frac{t^2}{2!}E[X^2] \ldots + \frac{t^r}{r!}E[X^r] + \ldots

Differentiating this expression ‘r’ times we obtain the following equality, \frac{d^r}{dt^r} M_X(t) = E[X^r] + \frac{t}{r+1}E[X^{r+1}] + \ldots

Substituing t=0 we see that all the higher terms vanish and we get that, \frac{d^r}{dt^r} M_X(t)|_{t=o} = E[X^r] which completes our proof.

Examples of moment generating function:

1. The moment generating function of the normal distribution with mean \mu and variance \sigma is given as, M_X(t)= e^{t\mu +{\frac {1}{2}}\sigma ^{2}t^{2}}

2. The moment generating function of the exponential distribution with parameter \lambda is given as, M_X(t)= \frac{1}{1-t/\lambda}

3. The moment generating function of the Poisson distribution with parameter \lambda is given as, M_X(t)= e^{\lambda (e^{t}-1)}

4. The moment generating function of the bernoulli distribution with parameter p is given as, M_X(t)= q + pe^t

5. The moment generating function of the binomial distribution with parameters n and p is given as, M_X(t)= (q + pe^t)^n

6. The moment generating function of the uniform distribution with parameters a and b is given as, M_X(t)= \frac {e^{tb}-e^{ta}}{t(b-a)}

7. The moment generating function of the geometric distribution with parameter p is given as, M_X(t)= \frac {p}{1-qe^{t}}

8. The moment generating function of the gamma distribution with parameters ‘k’ and \lambda is given as, M_X(t)= \frac{1}{(1-kt)^\lambda}

9. The moment generating function of the chi squared distribution with ‘n’ degrees of freedom is given as, M_X(t)= \frac{1}{(1-t)^{n/2}}

Joint Moment Generating Function:

The joint moment generating function refers to the moment generating function when we are given the joint probability density of two or more random variables.

Let X and Y be two continuous random variables with joint probability density f(x,y). The joint moment generating function is defined as,

So we see that,