Variance
Question: How spread out are X values? One answer we might try is to measure the average distance from the average value:
\E\left[|X - \E(X)|\right]
To simplify notation, we'll write \mu = \E(X).
Also, it's often usefull to square a term rather than take absolute value when we want to ensure a positive output, so we'll define...
Let X be a random variable with \E(X) = \mu.
The variance of X is:
\sigma^2 = \Var(x) = \E\left[ (X - \mu)^2\right]
\sigma = \sqrt{\Var(X)} is called the standard deviation.
We can write an alternative formula here:
\Var(X) \amp = \E\left[ (X - \mu)^2 \right]
\amp = \E\left[ X^2 - 2\mu X + \mu^2 \right]
\amp = \E(X^2) - 2\mu \E(X) + \E(\mu^2)
\amp = \E(X^2) - 2\mu^2 + \mu^2
\amp = \E(X^2) - \mu^2.
This formula is generally more useful for performing computations.
Let X indicate event A with \Pr(A) = p.
Distribution for <m>X</m>
| k |
\Pr(X = k) |
| 0 |
1 - p |
| 1 |
p |
Distribution for <m>X^2</m>
| k |
\Pr(X^2 = k) |
| 0^2 |
1 - p |
| 1^2 |
p |
Then \E(X) = p and \E(X^2) = p, so:
\Var(X) = \E(X^2) - \left(\E(X)\right)^2 = p - p^2 = p(1 - p).
Let R be the roll of a fair D6.
Distribution for <m>R</m>
| k |
\Pr(R = k) |
| 1 |
1/6 |
| 2 |
1/6 |
| 3 |
1/6 |
| 4 |
1/6 |
| 5 |
1/6 |
| 6 |
1/6 |
Distribution for <m>R^2</m>
| k |
\Pr(R^2 = k) |
| 1^2 |
1/6 |
| 2^2 |
1/6 |
| 3^2 |
1/6 |
| 4^2 |
1/6 |
| 5^2 |
1/6 |
| 6^2 |
1/6 |
Then \E(R) = \frac{7}{2} and \E(R^2) = \frac{91}{6}, so:
\Var(R) = \E(R^2) - \left(\E(R)\right)^2 = \frac{91}{6} - \frac{49}{4} = \frac{35}{12}.
Let X \in [0, 1] with pdf f(x) = 2x.
\E(X) \amp = \int_0^1 x \cdot 2x\ dx
\amp = \int_0^1 2x^2\ dx
\amp = \frac{2x^3}{3}\bigg|_0^1
\amp = \frac{2}{3} - 0
\amp = \frac{2}{3}
\E(X^2) \amp = \int_0^1 x^2\cdot 2x\ dx
\amp = \int_0^1 2x^3\ dx
\amp = \frac{2x^4}{4}\bigg|_0^1
\amp = \frac{1}{2} - 0
\amp = \frac{1}{2}
\Rightarrow \quad \Var(X) \amp = \E(X^2) - \left(\E(X)\right)^2
\amp = \frac{1}{2} - \frac{4}{9}
\amp = \frac{1}{18}.
WARNING!!! In general, variance is not linear! That is,
\Var(X + Y) \amp \neq \Var(X) + \Var(Y)
\Var(kX) \amp \neq k \Var(X)
But there are still relevant properties we can state here:
Let X be a random variable and k\in \R.
Then:
\Var(X + k) \amp = \Var(X) \amp \Var(kX) \amp = k^2\Var(X).
Moreover, if Y is another random variable and X, Y are independent, then:
\Var(X + Y) = \Var(X) + \Var(Y).
Let H_1, H_2, \dotsc, H_n be indicators with parameter p.
Let S = H_1 + \dotsb + H_n, so S \sim \Bin(n, p).
We know \Var(H_i) = p(1-p), and H_1, \dotsc, H_n are independent.
So:
\Var(S) \amp = \Var(H_1 + \dotsb + H_n)
\amp = \Var(H_1) + \dotsb + \Var(H_n)
\amp = p(1-p) + \dotsb + p(1-p)
\amp = np(1-p)