+ Confidence Intervals
+
+
+ Idea: suppose we collected (i.i.d.) measurements X_1, \dotsc, X_n which have some population mean \mu and variance \sigma^2.
+ We calculate the average:
+
+ A_n = \frac{A_1 + \dotsb + A_n}{n}
+
+ and we want to estimate the value of \mu from the collected data.
+ Previously, we've discussed the MLE, the single most likely estimate of the parameter.
+ Now, we'd like to give a range [\mu_{\ell}, \mu_h] where we can say something like: "we are 95% confident that \mu \in [\mu_{\ell}, \mu_h]."
+
+
+
+ From the CLT, if n is large enough, we approximate A_n \approx \Norm\left(\mu, \frac{\sigma^2}{n}\right).
+ We can think of an interval centered at \mu as a collection of numbers within a certain distance d of \mu.
+ We'd like to choose a distance so that our measured A_n is within d of \mu.
+ We can shift our point of view here: if A_n is within d of \mu, then \mu is within d of A_n.
+
+
+
+ TODO: image
+
+
+
+ To pick the distance d, we want to make our interval large enough so that there's only a small amount of area under the normal curve outside of the interval.
+ So:
+
+ -
+
+ Pick some value \alpha \in (0, 1) (representing the area under the curve outside the interval)
+
+
+
+ -
+
+ Let c_{\alpha} = \Phi^{-1}\left(1 - \frac{\alpha}{2}\right) (this is the standardized z-score of the right-hand side of the interval)
+
+
+
+ -
+
+ Then, the interval will be:
+
+ \left[A_n - c_{\alpha} \sqrt{\frac{\sigma^2}{n}}, A_n + c_{\alpha} \sqrt{\frac{\sigma^2}{n}}\right].
+
+
+
+
+
+
+
+ For \alpha = 0.05, we'll have c_{\alpha} = \Phi^{-1}\left(1 - \frac{0.05}{2}\right) = \Phi^{-1}(0.975) = 1.96.
+ So:
+
+
+
+
+
+ The 95% confidence limits are given by:
+
+ \mu_{\ell} \amp = A_n - 1.96 \sqrt{\frac{\sigma^2}{n}}
+ \mu_{h} \amp = A_n + 1.96 \sqrt{\frac{\sigma^2}{n}}
+
+
+
+
+
+
+
+
+ The term \sqrt{\frac{\sigma^2}{n}} is called the standard error of the mean.
+ (It's the standard deviation of A_n.)
+
+
+
+
+
+ We don't know the true value of \sigma^2, just like we don't know the true value of \mu.
+ So we'll have to use our sample of measurements to estimate \sigma^2, and then use that estimate to give us our range of values for \mu.
+
+
+
+
+
+ Let X_1, \dotsc, X_n be (i.i.d.) measurements.
+ Then the sample mean is:
+
+ A_n = \frac{X_1 + \dotsb + X_n}{n}
+
+ and the sample variance is:
+
+ s^2 = \frac{\sum (X_i - A_n)^2}{n - 1} = \frac{\left(\sum X_i^2\right) - n(A_n^2)}{n - 1}
+
+
+
+
+
+
+ Why n - 1 here? It turns out that, as defined above, s^2 is an unbiased estimator for \sigma^2, whereas it wouldn't be if we divided by n instead.
+ (We'll ommit both the calculation justifying this and a more conceptual explanation for now.)
+
+
+
+ So, we can adjust the confidence limits from the theorem:
+
+ \mu_{\ell} \amp = A_n - 1.96 \sqrt{\frac{s^2}{n}}
+ \mu_{h} \amp = A_n + 1.96 \sqrt{\frac{s^2}{n}}
+
+
+
+
+
+
+ Suppose we're given measurements X_1, X_2, X_3, X_4, X_5 below.
+
+
+
+
+ | i |
+ X_i |
+
+
+
+ | 1 |
+ 19.2 |
+
+
+
+ | 2 |
+ 20.1 |
+
+
+
+ | 3 |
+ 21.3 |
+
+
+
+ | 4 |
+ 20.7 |
+
+
+
+ | 5 |
+ 19.8 |
+
+
+
+
+ We're just practicing the computation here, so we'll pretend that 5 measurements is large enough for the CLT to apply.
+
+ A_n \amp = \frac{19.2 + \dotsb + 19.8}{5} = 20.22
+ \sum X_i^2 \amp = 19.2^2 + \dotsb + 19.8^2 = 2046.87
+ s^2 \amp = \frac{2046.87 - 5(20.22^2)}{4} = 0.657
+ \sqrt{\frac{s^2}{n}} \amp = \sqrt{\frac{0.657}{5}} \approx 0.362
+ \mu_{\ell} \amp = 20.22 - 1.96(0.362) \approx 19.51
+ \mu_{h} \amp = 20.22 + 1.96(0.362) \approx 20.93
+
+
+
+
+
+
+ This way of computing confidence limits only applies if we can think of the parameter we're estimating as a mean.
+
+
+
+
+
+ Suppose we flip a coin 100 times and see 40 heads.
+ Find a 98% confidence interval for the bias p.
+
+
+
+ Let H_1, \dotsc, H_{100} indicate heads on each flip.
+ Let S_{100} = H_1 + \dotsb + H_{100} and A_n = \frac{S_n}{n}.
+ Then:
+
+ \E(A_n) \amp = \E\left(\frac{S_n}{n}\right) = \frac{1}{n}\E(S_n) = \frac{1}{n} np = p
+
+ So a 98% confidence interval for p is also a 98% confidence interval for p.
+
+
+
+ In this setting, we can simplify the computation of s^2 and \sqrt{\frac{s^2}{n}}, using our knowledge of the binomial distribution S_n.
+
+ \Var(S_n) \amp = np(1 - p)
+ \Var(A_n) \amp = \Var\left(\frac{S_n}{n}\right) = \frac{1}{n^2} \Var(S_n) = \frac{1}{n^2} np(1-p) = \frac{p(1-p)}{n}
+
+ So s^2 = p(1 - p) and \sqrt{\frac{s^2}{n}} = \sqrt{\frac{p(1-p)}{n}}.
+ We don't know the value of p, so we'll use the MLE \widehat{p} = \frac{40}{100} = 0.4.
+ So:
+
+ \sqrt{\frac{s^2}{n}} = \sqrt{\frac{(0.4)(0.6)}{100}} \approx 0.049.
+
+
+
+
+ For a 98% confidence interval, we have \alpha = 1 - 0.98 = 0.02, so:
+
+ c_{\alpha} = \Phi^{-1}\left(1 - \frac{\alpha}{2}\right) = \Phi^{-1}(0.99) \approx 2.33.
+
+ Then, the confidence limits are:
+
+ p_{\ell} \amp = 0.4 - 2.33(0.049) \approx 0.286
+ p_{h} \amp = 0.4 + 2.33(0.049) \approx 0.514
+
+ For comparison, the 95% confidence limits are:
+
+ p_{\ell} \amp = 0.4 - 1.96(0.049) \approx 0.304
+ p_{h} \amp = 0.4 + 1.96(0.049) \approx 0.496
+
+
+
+
+
+