Section 4.2 Central Limit Theorem
Although we defined the normal distribution earlier (
Definition 2.2.8), we haven’t done much in the way of probability calculations with it. The sense in which this distribution is "normal" is that it turns out other distributions, when added together or averaged, tend to be well-approximated by a normal distribution.
Theorem 4.2.1.
Suppose \(X_1, X_2, \dotsc, X_n\) are independent and identically distributed (or iid) random variables with finite expected value \(\mu\) and finite variance \(\sigma^2\text{.}\) Let:
\begin{align*}
S_n \amp = X_1 + X_2 + \dotsb + X_n \\
A_n \amp = \frac{X_1 + X_2 + \dotsb + X_n}{n}
\end{align*}
Then, for sufficiently large \(n\text{:}\)
\begin{gather*}
S_n \approx \Norm(n \mu, n \sigma^2) \\
A_n \approx \Norm\left(\mu, \frac{\sigma^2}{n}\right)
\end{gather*}
When a normal distribution gets translated and/or scaled, the result is another normal distribution. It will often be useful to standardize a normal distribution \(X = \Norm(\mu, \sigma^2)\) as follows:
\begin{gather*}
Z = \frac{X - \mu}{\sigma} = \Norm(0, 1).
\end{gather*}
Given an \(X\)-value \(a\text{,}\) the corresponding value of \(Z\) is called a \(z\)-score, and is sometimes denoted \(a^*\)
Example 4.2.3.
Suppose we flip a fair coin 100 times and let \(S_{100}\) be the number of heads. In order to calculate, for example, \(\Pr(40 \leq S_{100} \leq 60)\) using the binomial distribution probability formula, we would need to calculate:
\begin{gather*}
\Pr(40 \leq S_{100} \leq 60) = \sum_{k = 40}^{60} {100 \choose k} (0.5)^k (1 - 0.5)^{100 - k}.
\end{gather*}
This could be done using a good calculator. If we increased the number of flips to 1000, or 10000, etc., even a powerful calculator might struggle with the computation. However, if we’re willing to accept an approximation, then
Theorem 4.2.1 gives a (mostly) straightforward path:
\begin{align*}
\E(S_{100}) \amp = np = (100)(0.5) = 50 \\
\Var(S_{100}) \amp = np(1-p) = (100)(0.5)(0.5) = 25 \\
\Rightarrow \quad S_{100} \amp\approx \Norm(50, 25) \\
Z \amp = \frac{S_{100} - 50}{\sqrt{25}} \\
\Pr(40 \leq S_{100} \leq 60) \amp\approx \Pr\left(\frac{40 - 50}{5} \leq Z \leq \frac{60 - 50}{5}\right) \\
\amp= \Pr\left(-2 \leq Z \leq 2\right)
\end{align*}
We now reach a potential impass.
The random variable \(Z\) is the standard normal distribution, with pdf:
\begin{gather*}
\phi(x) = \frac{1}{\sqrt{2\pi}} e^{-x^2/2}
\end{gather*}
To calculate probabilities like \(\Pr(-2 \leq Z \leq 2)\text{,}\) we should evaluate the integral:
\begin{gather*}
\Pr(-2 \leq Z \leq 2) = \int_{-2}^2 \phi(x)\ dx = \int_{-2}^2 \frac{1}{\sqrt{2\pi}} e^{-x^2/2}\ dx
\end{gather*}
Alternatively, we could use the cdf:
\begin{gather*}
\Phi(x) = \int_{-\infty}^x \phi(t)\ dt = \int_{-\infty}^x \frac{1}{\sqrt{2\pi}} e^{-t^2/2}\ dt
\end{gather*}
If we had a nice formula for \(\Phi(x)\text{,}\) then we could complete the probability calculation:
\begin{gather*}
\Pr(-2\leq Z \leq 2) = \Phi(2) - \Phi(-2).
\end{gather*}
Unfortunately, we don’t have a nice formula for \(\Phi(x)\text{.}\) In fact, there is provably no "nice" formula that could be written for \(\Phi(x)\) (where "nice" has some technical mathematical meaning that we won’t discuss here).
The true power of
Theorem 4.2.1 comes to the rescue: it doesn’t matter what type of distribution the individual
\(X_i\)’s have, and it doesn’t matter what type of distributions
\(S_n\) and
\(A_n\) have. The random variable
\(Z\) is
\(\Norm(0, 1)\text{.}\) Since it’s the same particular distribution all the time, and since we’re only aiming at an approximation anyway, we can calculate many estimations of probabilities like
\(\Pr(a \leq Z \leq b)\) ahead of time and just save the results to look up later.
A table of precalculated
\(\Phi(z)\) values can be found in
Section A.1.
Example 4.2.4.
Continuing the previous example, and having looked up the values \(\Phi(2) \approx 0.9772\) and \(\Phi(-2) \approx 0.0228\text{,}\) we complete our estimation:
\begin{align*}
\Pr(40 \leq S_{100} \leq 60) \amp\approx \Pr\left(\frac{40 - 50}{5} \leq Z \leq \frac{60 - 50}{5}\right) \\
\amp= \Pr\left(-2 \leq Z \leq 2\right) \\
\amp \approx \Phi(2) - \Phi(-2) \\
\amp \approx 0.9772 - 0.0228 \\
\amp = 0.9544.
\end{align*}
The true value is approximately \(0.965\text{,}\) so we underestimated the probability a bit. It turns out this will consistently be the case when using a normal approximation to a discrete distribution, such as the binomial distribution. The range of values \(40 \leq S_{100} \leq 60\) is a list of \(21\) discrete values. However, the continuous interval \([40, 60]\) has a length of only 20. So calculating the \(z\)-scores from the values \(40\) and \(60\) has the effect of shortening the interval length by 1. To mitigate this, we can make a continuity correction by extending the values out by \(0.5\) in each direction:
\begin{align*}
\Pr(40 \leq S_{100} \leq 60) \amp\approx \Pr\left(\frac{39.5 - 50}{5} \leq Z \leq \frac{60.5 - 50}{5}\right) \\
\amp= \Pr\left(-2.1 \leq Z \leq 2.1\right) \\
\amp \approx \Phi(2.1) - \Phi(-2.1) \\
\amp \approx 0.9821 - 0.0179 \\
\amp = 0.9642.
\end{align*}
This is a much more accurate estimate.
Exercises Exercises
1.
Suppose a coin has probability 0.4 of coming up heads, and we flip the coin 100 times. Let
\(S\) be the number of heads. Estimate the probability that
\(34 \leq S \leq 44\text{.}\)
Answer.
\(\Pr(34 \leq S \leq 44) \approx 0.8212 - 0.0918 = 0.7294.\)
2.
Suppose a fair die is rolled 100 times, and let
\(m\) be the average value of the rolls. Estimate the probability that
\(3.45 \leq m \leq 3.55\text{.}\)
Answer.
\(\Pr(3.45 \leq m \leq 3.55) \approx 0.6255 - 0.3745 = 0.2510.\)
3.
The heights of men in the US have a mean of 69 in and a variance of about 9 in
\(^2\text{,}\) and the heights of women in the US have a mean of 63.5 in with a variance of 6.25 in
\(^2\text{.}\)
(a)
Suppose the heights of 30 men are sampled, and a sample mean
\(m\) is taken. Find the expected value and variance of
\(m\text{.}\)
Answer.
\(\E(m) = 69\) and
\(\Var(m) = 0.3\text{.}\)
(b)
Estimate the probability that
\(m \geq 69.5\text{.}\)
Answer.
\(\Pr(m \geq 69.5) \approx 1 - 0.8186 = 0.1814\text{.}\)
(c)
If the sampled group was women, estimate the probability that
\(m \geq 64\text{.}\)
Answer.
\(\E(m) = 63.5\) and
\(\Var(m) \approx 0.208\text{.}\) \(\Pr(m \geq 64) \approx 1 - 0.8643 = 0.1357\text{.}\)
