Skip to main content

Section Thursday, Feb 19

This is an outline of the topics we covered in class. These notes are not a substitute for your own note-taking. I highly recommend that you take your own notes during class. If you ever miss a class for any reason, reach out to another student in class to get a copy of their notes.

Subsection Central Limit Theorem

Definition 116.

A continuous random variable \(X\) has the normal distribution with mean \(\mu\) and variance \(\sigma^2\) if it has pdf:
\begin{gather*} f(x; \mu, \sigma^2) = \frac{1}{\sqrt{2\pi \sigma^2}} e^{\frac{-(x - \mu)^2}{2\sigma^2}} \end{gather*}
We’ll write \(X \sim \Norm(\mu, \sigma^2)\text{.}\)
The specific distribution \(\Norm(0, 1)\) is called the standard normal distribution, and its pdf has the notation:
\begin{gather*} f(x; 0, 1) = \phi(x). \end{gather*}
If \(X \sim \Norm(\mu, \sigma^2)\text{,}\) then:
\begin{gather*} \Pr(a \leq X \leq b) = \int_a^b f(x; \mu, \sigma^2)\ dx. \end{gather*}
Unfortunately, \(e^{-x^2}\) has no elementary antiderivative. But we know \(\int_a^b f(x)\ dx\) represents the area under the graph \(y = f(x; \mu, \sigma^2)\text{,}\) and this area can be approximated to arbitrary precision.
We’ll use two general rules of thumb for determining whether the number of measurements \(n\) is large enough for the approximation to be a good one:
  1. \(\displaystyle n \geq 30\)
  2. If \(S\sim \Bin(n, p)\text{,}\) then \(n\) should be large enough so that there are at least 5 heads and 5 tails.
We’ll omit a proof of TheoremΒ 117, but we can at least check that the expected values and variances of \(S_n, A_n\) are correct. Given \(\E(X_i) = \mu, \Var(X_i) = \sigma^2\text{:}\)
\begin{align*} \E(S_n) \amp = \E(X_1 + \dotsb + X_n) \\ \amp = \E(X_1) + \dotsb + \E(X_n) \\ \amp = \mu + \dotsb + \mu \\ \amp = n \mu. \checkmark \\ \Var(S_n) \amp = \Var(X_1 + \dotsb + X_n) \\ \amp = \Var(X_1) + \dotsb + \Var(X_n) \\ \amp = \sigma^2 + \dotsb + \sigma^2 \\ \amp = n \sigma^2. \checkmark \\ \E(A_n) \amp = \E\left(\frac{S_n}{n}\right) \\ \amp = \frac{1}{n} \E(S_n) \\ \amp = \frac{1}{n} n \mu \\ \amp = \frac{1}{n} n \mu \\ \amp = \mu. \checkmark \\ \Var(A_n) \amp = \Var\left(\frac{S_n}{n}\right) \\ \amp = \frac{1}{n^2} \Var(S_n) \\ \amp = \frac{1}{n^2} n \sigma^2 \\ \amp = \frac{\sigma^2}{n}. \checkmark \end{align*}

Example 118.

Suppose we flip a fair coin 100 times, and let \(S\) be the number of heads. Estimate \(\Pr(40 \leq S \leq 60)\text{.}\)
Since \(S \sim \Bin(n, p)\text{,}\) we know:
\begin{align*} \E(S) \amp = np = (100)(0.5) = 50 \\ \Var(S) \amp = np(1-p) = (100)(0.5)(1 - 0.5) = 25 \end{align*}
So \(S \approx \Norm(50, 25)\text{.}\) Then:
\begin{gather*} \Pr(40 \leq S \leq 60) \approx \int_{40}^{60} f(x; 50, 25)\ dx, \end{gather*}
but we can’t compute this integral.
Instead, we can standardize by shifting and scaling:
\begin{gather*} Z = \frac{S - 50}{\sqrt{25}} = \frac{S - 50}{5}. \end{gather*}
Then \(Z \sim \Norm(0, 1)\text{.}\) Values of \(Z\) are called \(z\)-scores, sometimes indicated by an asterisk. (I.e., if \(a\) is a value of \(S\text{,}\) then \(a^*\) is the corresponding \(z\)-score.) Now:
\begin{align*} \Pr(40 \leq S \leq 60) \amp \approx \Pr\left(\frac{40 - 50}{5} \leq Z \leq \frac{60 - 50}{5}\right) \\ \amp = \Pr(-2 \leq Z \leq 2) \\ \amp = \int_{-2}^2 \phi(x)\ dx. \end{align*}
Now, we still can’t compute the value of the integral. However, there’s a huge benefit to translating to the standard normal distribution, regardless of which normal distribution we used to approximate \(S\text{.}\) We can approximate integrals like \(\int_a^b \phi(x)\ dx\) to abritrary precision and record the results in a table. Then, we can look up the values when needed. We don’t need to redo our approximations for different normal distributions, we just standardize whatever normal distribution we come across.
So:
\begin{align*} \Pr(40 \leq S \leq 60) \amp \approx \Pr(-2 \leq Z \leq 2) \\ \amp = \Phi(2) - \Phi(-2) \\ \amp \approx 0.9772 - 0.0228 \\ \amp \approx 0.9544. \end{align*}
TODO: complex image introducing the idea of the continuity correction.
We can make the approximation more accurate by extending the range of \(S\)-values by half a unit in each direction:
\begin{align*} \Pr(40 \leq S \leq 60) \amp \approx \Pr\left(\frac{39.5 - 50}{5} \leq Z \leq \frac{60.5 - 50}{5}\right) \\ \amp = \Pr(-2.1 \leq Z \leq 2.1) \\ \amp = \Phi(2.1) - \Phi(-2.1) \\ \amp \approx 0.9821 - 0.0179 \\ \amp \approx 0.9642. \end{align*}