Skip to main content

Section 3.2 Variance

The expected value is one of several statistics sometimes referred to as a "measure of central tendency". These are statistics that try to capture the "typical" value of the random variable. Our next statistic tries to measure how spread out a distribution is.

Definition 3.2.1.

Let \(X\) be a random variable. The variance of \(X\) is:
\begin{gather*} \Var(X) = \E\left[ \left(X - \E(X)\right)^2 \right]. \end{gather*}
This formula tries to capture some measure of how spread out the distribution is. One way we might hope to measure spread is to calculate how far away \(X\) is from its expected value on average. Squaring the term \(X - \E(X)\) ensures that any difference between the value of the random variable and its expected value contributes positively. ("Distance away from" should be positive.) So, the formula in Definition 3.2.1 is conceptually clear. In practice, there’s another way to structure the formula that makes for cleaner calculation:

Proof.
\begin{align*} \Var(X) \amp = \E\left[ \left(X - \E(X)\right)^2 \right] \\ \amp = \E\left[ X^2 - 2 \E(X) X + \left(\E(X)\right)^2\right] \\ \amp = \E\left(X^2\right) - 2 \E(X) \E(X) + \left(\E(X)\right)^2\\ \amp = \E\left(X^2\right) - \left(\E(X)\right)^2. \end{align*}

Example 3.2.3.

Let \(R\) be the value of a fair 6-sided die roll. We’ve seen in Example 3.1.2 that \(\E(R) = \frac{7}{2}\text{.}\) To find \(\E\left(R^2\right)\text{:}\)
\begin{align*} \E\left(R^2\right) \amp = 1^2 \left(\frac{1}{6}\right) + 2^2 \left(\frac{1}{6}\right) + 3^2 \left(\frac{1}{6}\right) + 4^2 \left(\frac{1}{6}\right) + 5^2 \left(\frac{1}{6}\right) + 6^2 \left(\frac{1}{6}\right) \\ \amp = 1 \left(\frac{1}{6}\right) + 4 \left(\frac{1}{6}\right) + 9 \left(\frac{1}{6}\right) + 16 \left(\frac{1}{6}\right) + 25 \left(\frac{1}{6}\right) + 36 \left(\frac{1}{6}\right) \\ \amp = \frac{91}{6}. \end{align*}
Then
\begin{gather*} \Var(R) = \E\left(R^2\right) - \left(\E(R)\right)^2 = \frac{91}{6} - \frac{49}{4} = \frac{35}{12}. \end{gather*}
Notice that in the \(\E\left(R^2\right)\) calculation, we’re just taking the basic structure of the \(\E(R)\) calculation and squaring the \(R\) values. The analogous calculation for a continuous random variable looks similar:

Example 3.2.4.

Let \(X\) be uniform on the interval \([a, b]\text{.}\) So \(X\) has the pdf \(f(x) = \frac{1}{b - a}\text{,}\) and we saw in Example 3.1.7 that \(\E(X) = \frac{a + b}{2}\text{.}\) Now:
\begin{align*} \E\left(X^2\right) \amp = \int_a^b x^2 f(x)\ dx \\ \amp = \int_a^b \frac{x^2}{b - a} \ dx \\ \amp = \frac{1}{b-a} \left(\frac{x^3}{3}\right)\bigg|_a^b \\ \amp = \frac{1}{b-a} \left( \frac{b^3}{3} - \frac{a^3}{3} \right) \\ \amp = \frac{1}{b - a}\left( \frac{(b - a)(a^2 + ab + b^2)}{3}\right) \\ \amp = \frac{a^2 + ab + b^2}{3}. \end{align*}
Then:
\begin{align*} \Var(X) \amp = \E\left(X^2\right) - \left(\E(X)\right)^2 \\ \amp = \frac{a^2 + ab + b^2}{3} - \frac{(a+b)^2}{4} \\ \amp = \frac{a^2 + ab + b^2}{3} - \frac{a^2 + 2ab + b^2}{4} \\ \amp = \frac{4a^2 + 4ab + 4b^2}{12} - \frac{3a^2 + 6ab + 3b^2}{12} \\ \amp = \frac{a^2 - 2ab + b^2}{12} \\ \amp = \frac{(a - b)^2}{12}. \end{align*}
Variance is not linear. In general, you can’t split up variance across plus or minus signs, and you can’t pull multiplied constants out of the variance like you can with expected value. However, there are some properties we can use to do similar manipulations to the variance.
We’ll omit the proofs and just casually observe that these are reasonable properties. \(X + \alpha\) is a shift of \(X\text{,}\) and shifting shouldn’t change how spread out the distribution is. \(\alpha X\) scaled \(X\) by a constant, so it should scale the spread of the distribution by the same constant. But recall that the variance calculation involves squaring; it’s an indirect measure of spread. So it’s reasonable that scaling the distribution by a constant should scale the variance by its square.
Let’s put these properties to use in a pair of related examples:

Example 3.2.6.

Let \(X\) indicate an event \(A\) which has probability \(p\text{.}\) In Example 3.1.5, we saw \(\E(X) = p\text{.}\) Notice that \(X\) takes the values 0 and 1 (with probabilities \(1 - p\) and \(p\text{,}\) respectively). But \(0^2 = 0\text{,}\) and \(1^1 = 1\text{,}\) so the expected value calculation for \(X^2\) is precisely the same as for \(X\text{!}\)
\begin{align*} \E\left(X^2\right) \amp = 0^2 \cdot (1-p) + 1^2 \cdot p = p. \end{align*}
So:
\begin{align*} \Var(X) \amp \E\left(X^2\right) - \left(\E(X)\right)^2 = p - p^2 = p(1 - p). \end{align*}

Example 3.2.7.

Recall the setup from Example 3.1.11: a binomial random variable \(S\) is the sum \(H_1 + H_2 + \dotsb + H_n\) of \(n\) indicator random variables, each with parameter \(p\text{.}\) The results of different coin flips are independent from each other, so the random variables \(H_1, \dotsc, H_n\) are independent from each other. From Example 3.2.6, we know that \(\Var(H_i) = p(1-p)\) for each \(i\text{.}\) Therefore:
\begin{align*} \Var(S) \amp = \Var(H_1 + H_2 + \dotsb + H_n) \\ \amp = \Var(H_1) + \Var(H_2) + \dotsb + \Var(H_n) \\ \amp = \underbrace{p(1-p) + p(1-p) + \dotsb + p(1-p)}_{n \text{ times}} \\ \amp = np(1-p). \end{align*}
Remember this formula! We’ll make frequent use of it.

Exercises Exercises

1.

Consider a random variable \(X\) with probability distribution below. Find \(\Var(X)\text{.}\)
Table 3.2.8.
\(x\) \(\Pr(X = x)\)
1 0.1
2 0.05
3 0.2
4 0.15
5 0.15
6 0.1
7 0.05
8 0.1
9 0.1

2.

Let \(X\) be a random variable taking the values 1, 2, 3, 4, 5. Write a distribution table for \(X\text{,}\) then use your table to write a distribution for \(X^2\text{.}\) Then, find \(\Var(X)\text{.}\)

3.

Suppose we flip a coin \(n = 100\) times, and let \(N\) count the number of heads. If the coin comes up heads on a flip with probability \(p = 0.4\text{,}\) what is \(\Var(N)\text{?}\) What if \(n = 80\) and \(p = 0.6\text{?}\) What if \(n = 200\) and \(p = 0.5\text{?}\)

4.

If \(\E(X) = 3\text{,}\) \(\Var(X) = 2\text{,}\) what is \(\E(X^2)\text{?}\)

5.

A continuous random variable \(X\) taking values in \([0, 1]\) has p.d.f. \(f(x) = 2x\text{.}\) What is \(\Var(X)\text{?}\)

6.

A continuous random variable \(X\) taking values in \([-1, 1]\) has p.d.f. \(f(x) = \frac{3x^2}{2}\text{.}\) What is \(\Var(X)\text{?}\)

7.

A continuous random variable \(X\) taking values in \([1, 4]\) has p.d.f. \(f(x) = \frac{4}{3x^2}\text{.}\) What is \(\Var(X)\text{?}\)

8.

A continuous random variable \(X\) taking values in \([1, 4]\) has p.d.f. \(f(x) = k(x - \sqrt{x})\) for some constant \(k\text{.}\) In a previous problem, you found the value of \(k\text{.}\) Now, find \(\Var(X)\text{.}\)

9.

A continuous random variable \(X\) taking values in \([1, 2]\) has p.d.f. \(\displaystyle{f(x) = \frac{1}{2}\left(\frac{1}{x^2} + x\right)}\text{.}\) Find \(\Var(X)\text{.}\)
PrevTopNext