Thursday, Feb 19

This is an outline of the topics we covered in class. These notes are not a substitute for your own note-taking. I highly recommend that you take your own notes during class. If you ever miss a class for any reason, reach out to another student in class to get a copy of their notes.

Central Limit Theorem

A continuous random variable X has the normal distribution with mean \mu and variance \sigma^2 if it has pdf: f(x; \mu, \sigma^2) = \frac{1}{\sqrt{2\pi \sigma^2}} e^{\frac{-(x - \mu)^2}{2\sigma^2}} We'll write X \sim \Norm(\mu, \sigma^2).

The specific distribution \Norm(0, 1) is called the standard normal distribution, and its pdf has the notation: f(x; 0, 1) = \phi(x).

If X \sim \Norm(\mu, \sigma^2), then: \Pr(a \leq X \leq b) = \int_a^b f(x; \mu, \sigma^2)\ dx. Unfortunately, e^{-x^2} has no elementary antiderivative. But we know \int_a^b f(x)\ dx represents the area under the graph y = f(x; \mu, \sigma^2), and this area can be approximated to arbitrary precision.

Suppose X_1, \dotsc, X_n are independent and identically distributed (or i.i.d.) with finite expected value \mu and finite variance \sigma^2. Define: S_n \amp = X_1 + X_2 + \dotsb + X_n A_n \amp = \frac{X_1 + X_2 + \dotsb + X_n}{n} = \frac{S_n}{n} Then, for large enough n: S_n \amp \approx \Norm(n\mu, n\sigma^2) A_n \amp \approx \Norm\left(\mu, \frac{\sigma^2}{n}\right)

We'll use two general rules of thumb for determining whether the number of measurements n is large enough for the approximation to be a good one:

n \geq 30
If S\sim \Bin(n, p), then n should be large enough so that there are at least 5 heads and 5 tails.

We'll omit a proof of , but we can at least check that the expected values and variances of S_n, A_n are correct. Given \E(X_i) = \mu, \Var(X_i) = \sigma^2: \E(S_n) \amp = \E(X_1 + \dotsb + X_n) \amp = \E(X_1) + \dotsb + \E(X_n) \amp = \mu + \dotsb + \mu \amp = n \mu. \checkmark \Var(S_n) \amp = \Var(X_1 + \dotsb + X_n) \amp = \Var(X_1) + \dotsb + \Var(X_n) \amp = \sigma^2 + \dotsb + \sigma^2 \amp = n \sigma^2. \checkmark \E(A_n) \amp = \E\left(\frac{S_n}{n}\right) \amp = \frac{1}{n} \E(S_n) \amp = \frac{1}{n} n \mu \amp = \frac{1}{n} n \mu \amp = \mu. \checkmark \Var(A_n) \amp = \Var\left(\frac{S_n}{n}\right) \amp = \frac{1}{n^2} \Var(S_n) \amp = \frac{1}{n^2} n \sigma^2 \amp = \frac{\sigma^2}{n}. \checkmark

Suppose we flip a fair coin 100 times, and let S be the number of heads. Estimate \Pr(40 \leq S \leq 60).

Since S \sim \Bin(n, p), we know: \E(S) \amp = np = (100)(0.5) = 50 \Var(S) \amp = np(1-p) = (100)(0.5)(1 - 0.5) = 25 So S \approx \Norm(50, 25). Then: \Pr(40 \leq S \leq 60) \approx \int_{40}^{60} f(x; 50, 25)\ dx, but we can't compute this integral.

Instead, we can standardize by shifting and scaling: Z = \frac{S - 50}{\sqrt{25}} = \frac{S - 50}{5}. Then Z \sim \Norm(0, 1). Values of Z are called z-scores, sometimes indicated by an asterisk. (I.e., if a is a value of S, then a^* is the corresponding z-score.) Now: \Pr(40 \leq S \leq 60) \amp \approx \Pr\left(\frac{40 - 50}{5} \leq Z \leq \frac{60 - 50}{5}\right) \amp = \Pr(-2 \leq Z \leq 2) \amp = \int_{-2}^2 \phi(x)\ dx. Now, we still can't compute the value of the integral. However, there's a huge benefit to translating to the standard normal distribution, regardless of which normal distribution we used to approximate S. We can approximate integrals like \int_a^b \phi(x)\ dx to abritrary precision and record the results in a table. Then, we can look up the values when needed. We don't need to redo our approximations for different normal distributions, we just standardize whatever normal distribution we come across.

So: \Pr(40 \leq S \leq 60) \amp \approx \Pr(-2 \leq Z \leq 2) \amp = \Phi(2) - \Phi(-2) \amp \approx 0.9772 - 0.0228 \amp \approx 0.9544.

TODO: complex image introducing the idea of the continuity correction.

We can make the approximation more accurate by extending the range of S-values by half a unit in each direction: \Pr(40 \leq S \leq 60) \amp \approx \Pr\left(\frac{39.5 - 50}{5} \leq Z \leq \frac{60.5 - 50}{5}\right) \amp = \Pr(-2.1 \leq Z \leq 2.1) \amp = \Phi(2.1) - \Phi(-2.1) \amp \approx 0.9821 - 0.0179 \amp \approx 0.9642.