Variance
The expected value is one of several statistics sometimes referred to as a "measure of central tendency".
These are statistics that try to capture the "typical" value of the random variable.
Our next statistic tries to measure how spread out a distribution is.
Let X be a random variable.
The variance of X is:
\Var(X) = \E\left[ \left(X - \E(X)\right)^2 \right].
This formula tries to capture some measure of how spread out the distribution is.
One way we might hope to measure spread is to calculate how far away X is from its expected value on average.
Squaring the term X - \E(X) ensures that any difference between the value of the random variable and its expected value contributes positively.
("Distance away from" should be positive.) So, the formula in is conceptually clear.
In practice, there's another way to structure the formula that makes for cleaner calculation:
Let X be a random variable.
Then:
\Var(X) = \E\left(X^2\right) - \left(\E(X)\right)^2.
\Var(X) \amp = \E\left[ \left(X - \E(X)\right)^2 \right]
\amp = \E\left[ X^2 - 2 \E(X) X + \left(\E(X)\right)^2\right]
\amp = \E\left(X^2\right) - 2 \E(X) \E(X) + \left(\E(X)\right)^2
\amp = \E\left(X^2\right) - \left(\E(X)\right)^2.
Let R be the value of a fair 6-sided die roll.
We've seen in that \E(R) = \frac{7}{2}.
To find \E\left(R^2\right):
\E\left(R^2\right) \amp = 1^2 \left(\frac{1}{6}\right) + 2^2 \left(\frac{1}{6}\right) + 3^2 \left(\frac{1}{6}\right) + 4^2 \left(\frac{1}{6}\right) + 5^2 \left(\frac{1}{6}\right) + 6^2 \left(\frac{1}{6}\right)
\amp = 1 \left(\frac{1}{6}\right) + 4 \left(\frac{1}{6}\right) + 9 \left(\frac{1}{6}\right) + 16 \left(\frac{1}{6}\right) + 25 \left(\frac{1}{6}\right) + 36 \left(\frac{1}{6}\right)
\amp = \frac{91}{6}.
Then
\Var(R) = \E\left(R^2\right) - \left(\E(R)\right)^2 = \frac{91}{6} - \frac{49}{4} = \frac{35}{12}.
Notice that in the \E\left(R^2\right) calculation, we're just taking the basic structure of the \E(R) calculation and squaring the R values.
The analogous calculation for a continuous random variable looks similar:
Let X be uniform on the interval [a, b].
So X has the pdf f(x) = \frac{1}{b - a}, and we saw in that \E(X) = \frac{a + b}{2}.
Now:
\E\left(X^2\right) \amp = \int_a^b x^2 f(x)\ dx
\amp = \int_a^b \frac{x^2}{b - a} \ dx
\amp = \frac{1}{b-a} \left(\frac{x^3}{3}\right)\bigg|_a^b
\amp = \frac{1}{b-a} \left( \frac{b^3}{3} - \frac{a^3}{3} \right)
\amp = \frac{1}{b - a}\left( \frac{(b - a)(a^2 + ab + b^2)}{3}\right)
\amp = \frac{a^2 + ab + b^2}{3}.
Then:
\Var(X) \amp = \E\left(X^2\right) - \left(\E(X)\right)^2
\amp = \frac{a^2 + ab + b^2}{3} - \frac{(a+b)^2}{4}
\amp = \frac{a^2 + ab + b^2}{3} - \frac{a^2 + 2ab + b^2}{4}
\amp = \frac{4a^2 + 4ab + 4b^2}{12} - \frac{3a^2 + 6ab + 3b^2}{12}
\amp = \frac{a^2 - 2ab + b^2}{12}
\amp = \frac{(a - b)^2}{12}.
Variance is not linear.
In general, you can't split up variance across plus or minus signs, and you can't pull multiplied constants out of the variance like you can with expected value.
However, there are some properties we can use to do similar manipulations to the variance.
Let X and Y be random variables, and let \alpha \in \R.
Then:
-
\Var(X + \alpha) = \Var(X).
-
\Var(\alpha X) = \alpha^2 \Var(X).
-
If X and Y are independent, then \Var(X + Y) = \Var(X) + \Var(Y).
We'll omit the proofs and just casually observe that these are reasonable properties.
X + \alpha is a shift of X, and shifting shouldn't change how spread out the distribution is.
\alpha X scaled X by a constant, so it should scale the spread of the distribution by the same constant.
But recall that the variance calculation involves squaring; it's an indirect measure of spread.
So it's reasonable that scaling the distribution by a constant should scale the variance by its square.
Let's put these properties to use in a pair of related examples:
Let X indicate an event A which has probability p.
In , we saw \E(X) = p.
Notice that X takes the values 0 and 1 (with probabilities 1 - p and p, respectively).
But 0^2 = 0, and 1^1 = 1, so the expected value calculation for X^2 is precisely the same as for X!
\E\left(X^2\right) \amp = 0^2 \cdot (1-p) + 1^2 \cdot p = p.
So:
\Var(X) \amp \E\left(X^2\right) - \left(\E(X)\right)^2 = p - p^2 = p(1 - p).
Recall the setup from : a binomial random variable S is the sum H_1 + H_2 + \dotsb + H_n of n indicator random variables, each with parameter p.
The results of different coin flips are independent from each other, so the random variables H_1, \dotsc, H_n are independent from each other.
From , we know that \Var(H_i) = p(1-p) for each i.
Therefore:
\Var(S) \amp = \Var(H_1 + H_2 + \dotsb + H_n)
\amp = \Var(H_1) + \Var(H_2) + \dotsb + \Var(H_n)
\amp = \underbrace{p(1-p) + p(1-p) + \dotsb + p(1-p)}_{n \text{ times}}
\amp = np(1-p).
Remember this formula! We'll make frequent use of it.
Consider a random variable X with probability distribution below.
Find \Var(X).
| x |
\Pr(X = x) |
| 1 |
0.1 |
| 2 |
0.05 |
| 3 |
0.2 |
| 4 |
0.15 |
| 5 |
0.15 |
| 6 |
0.1 |
| 7 |
0.05 |
| 8 |
0.1 |
| 9 |
0.1 |
Let X be a random variable taking the values 1, 2, 3, 4, 5.
Write a distribution table for X, then use your table to write a distribution for X^2.
Then, find \Var(X).
Suppose we flip a coin n = 100 times, and let N count the number of heads.
If the coin comes up heads on a flip with probability p = 0.4, what is \Var(N)? What if n = 80 and p = 0.6? What if n = 200 and p = 0.5?
If \E(X) = 3, \Var(X) = 2, what is \E(X^2)?
A continuous random variable X taking values in [0, 1] has p.d.f.
f(x) = 2x.
What is \Var(X)?
A continuous random variable X taking values in [-1, 1] has p.d.f.
f(x) = \frac{3x^2}{2}.
What is \Var(X)?
A continuous random variable X taking values in [1, 4] has p.d.f.
f(x) = \frac{4}{3x^2}.
What is \Var(X)?
A continuous random variable X taking values in [1, 4] has p.d.f.
f(x) = k(x - \sqrt{x}) for some constant k.
In a previous problem, you found the value of k.
Now, find \Var(X).
A continuous random variable X taking values in [1, 2] has p.d.f.
\displaystyle{f(x) = \frac{1}{2}\left(\frac{1}{x^2} + x\right)}.
Find \Var(X).