Moments & Moment Generating Functions (MGFs)

1. What are Moments? (The Physical Intuition)

The term "Moment" comes from Physics. If you think of a probability distribution as a physical object made of mass:

1. The 1st Moment (Mean): This is the Center of Mass. It’s the balance point of the object.
2. The 2nd Moment (Variance): This is the Moment of Inertia. It describes how far the mass is spread out from the center.
3. The 3rd Moment (Skewness): This describes the Asymmetry. Is the mass leaning more to the left or right?
4. The 4th Moment (Kurtosis): This describes the Tails. Is the mass concentrated in the center, or is there a lot of "heavy" mass far away in the outliers?

Raw vs. Central Moments

• Raw Moment ($E[X^n]$): Calculated relative to zero. The 1st raw moment is the Mean.
• Central Moment ($E[(X-\mu)^n]$): Calculated relative to the Mean ($\mu$). The 2nd central moment is the Variance.

2. The Moment Generating Function (MGF)

The MGF is a mathematical "magic trick." It allows us to pack every single moment (1st, 2nd, 3rd... to infinity) into a single function.

$M_X(t) = E[e^{tX}]$

Why $e^{tX}$? (The Taylor Expansion)

This is the part that usually confuses students. Why this specific formula? It comes down to the Taylor Series for $e^{z}$: $e^{tX} = 1 + tX + \frac{(tX)^2}{2!} + \frac{(tX)^3}{3!} + \dots$

When we take the Expected Value $E[\dots]$ of this expansion, we get: $E[e^{tX}] = 1 + t \cdot \mathbf{E[X]} + \frac{t^2}{2!} \cdot \mathbf{E[X^2]} + \frac{t^3}{3!} \cdot \mathbf{E[X^3]} + \dots$

Every raw moment of $X$ is sitting there as a coefficient!

3. How to "Extract" Moments

To get the $n$-th moment out of the MGF, we simply:

1. Differentiate the MGF $n$ times with respect to $t$.
2. Set $t = 0$.

$E[X^n] = M_X^{(n)}(0)$

Why do we need this?

Calculating $E[X^k]$ directly usually involves a very difficult integral ($\int x^k f(x) dx$). The MGF turns a hard calculus problem into a much easier differentiation problem.

4. The Uniqueness Property

This is how we prove things like the Central Limit Theorem. We show that as we add variables, their combined MGF approaches the MGF of a Normal distribution.

Test Your Knowledge