Skip to main content

Section 4.3 Confidence Intervals

When estimating the value of some parameter, the maximum likelihood estimation provides the single estimate with highest possible likelihood. But we might prefer to have a range of values, with some quantitative measure of how certain we are that the true parameter value could be in the range.
Suppose some measurement has a mean \(\mu\) and variance \(\sigma^2\) across the entire population. Independent measurements \(X_1, X_2, \dotsc, X_n\) and calculate the sample mean
\begin{gather*} A_n = \frac{X_1 + \dotsb + X_n}{n}. \end{gather*}
Then, according to Theorem 4.2.1, \(A_n \approx \Norm\left(\mu, \frac{\sigma^2}{n}\right)\) as long as we take a sufficient number of measurements. However, we don’t have access to the true values of \(\mu, \sigma^2\text{.}\)
We can make an estimate \(s^2\) of \(\sigma^2\) from the collected data, so we know how spread out the normal distribution should be.

Definition 4.3.1.

Let \(X_1, X_2, \dotsc, X_n\) be independent samples taken from a population. Let \(A_n = \frac{X_1 + \dotsb + X_n}{n}\) be the sample mean. The sample variance is:
\begin{gather*} s^2 = \frac{\sum \left(X_i - A_n\right)^2}{n - 1} = \frac{\left(\sum X_i^2\right) - n \left(A_n\right)^2}{n - 1}. \end{gather*}
The expression:
\begin{gather*} \sqrt{\frac{s^2}{n}} = \frac{s}{\sqrt{n}} \end{gather*}
is called the standard error of the mean.
Note that \(s^2\) is an estimate of the variance of each individual measurement \(X_i\) taken from the population. Therefore, \(\frac{s^2}{n}\) is an estimate of the variance of the mean \(A_n\text{.}\) Hence "standard error of the mean" is an estimate of the standard deviation of the mean.
Next, we want to pick a distance \(c\) so that our measured sample mean \(A_n\) is within \(c\) of the true population mean \(\mu\) with a high probability. We can pick some value \(\alpha \in (0, 1)\text{,}\) and let \(c_{\alpha} = \Phi^{-1}\left(1 - \frac{\alpha}{2}\right)\text{.}\) Then the interval:
\begin{gather*} \left[A_n - c_{\alpha} \left(\frac{s}{\sqrt{n}}\right), A_n + c_{\alpha} \left(\frac{s}{\sqrt{n}}\right)\right] \end{gather*}
is called a confidence interval with confidence level \(1 - \alpha\text{.}\)

Example 4.3.3.

Suppose measurements are taken of the heights of several plants in a garden plot. The measurements are given below. (Note: for the sake of a managable calculation, only five measurements are given. However, for Theorem 4.2.1 to apply, the number of measurements would need to be greater.)
Table 4.3.4. Plant Height Measurements
Plant \(i\) Height \(H_i\) (in)
\(1\) \(9.2\)
\(2\) \(9.8\)
\(3\) \(10.1\)
\(4\) \(9.5\)
\(5\) \(10.4\)
Then the sample mean is:
\begin{align*} A_5 = \frac{H_1 + \dotsb + H_5}{5} \amp = \frac{9.2 + \dotsb + 10.4}{5} = 9.8. \end{align*}
The sum of the squares is:
\begin{align*} \sum H_i^2 \amp = H_1^2 + \dotsb + H_5^2 = 9.2^2 + \dotsb + 10.4^2 = 481.1 \end{align*}
So the sample variance is:
\begin{gather*} s^2 = \frac{\left(\sum H_i\right)^2 - 5 (A_5)^2}{5 - 1} = \frac{481.1 - 5(9.8)^2}{4} \approx 0.225 \end{gather*}
Now, the standard error of the mean is:
\begin{gather*} \frac{s}{\sqrt{n}} = \sqrt{\frac{0.225}{5}} \approx 0.212 \end{gather*}
Finally, we can give 95% confidence limits:
\begin{align*} \mu_{\ell} \amp = 9.8 - 1.96(0.212) \approx 9.38 \\ \mu_{h} \amp = 9.8 + 1.96(0.212) \approx 10.22 \end{align*}
If the individual measurements \(X_1, \dotsc, X_n\) are indicator random variables, then the mean \(A_n\) represents the proportion which take on the value 1. For example, if the \(X_i\)’s indicate heads on coin flips, then \(A_n\) represents the proportion which come up heads. In this setting, we have a nice shortcut formula (Example 3.2.7) for the variance of \(S_n = X_1 + \dotsc + X_n\text{.}\) We can make use of this formula to get the variance of \(A_n\) without going through the sample variance calculation above:
\begin{align*} \Var(S_n) \amp = np(1 - p) \\ \Var(A_n) \amp = \Var\left(\frac{S_n}{n}\right) \\ \amp = \frac{1}{n^2} \Var\left(S_n\right) \\ \amp = \frac{1}{n^2} (np(1 - p)) \\ \amp = \frac{p(1 - p)}{n}. \end{align*}
Since we won’t have access to the true value of \(p\text{,}\) we can use the maximum likelihood estimate \(\widehat{p}\text{.}\)

Example 4.3.5.

Suppose in a sample of 100 people, 40 have a particular trait. The maximum likelihood estimate for the probability \(p\) of a random person having the trait is \(\widehat{p} = \frac{40}{100} = 0.4\text{.}\) So the standard error will be:
\begin{gather*} \sqrt{\frac{(0.4)(1 - 0.4)}{100}} = \sqrt{\frac{0.24}{100}} \approx 0.049. \end{gather*}
Therefore, 95% confidence limits for \(p\) are:
\begin{align*} p_{\ell} \amp = 0.4 - 1.96(0.049) \approx 0.304 \\ p_{h} \amp = 0.4 + 1.96(0.049) \approx 0.496 \end{align*}

Exercises Exercises

1.

Suppose we flip a coin 100 times and count 60 heads. Let \(p\) be the (unknown) probability that the coin comes up heads on a flip. Give an approximate 95% confidence interval for the value of \(p\text{.}\)
Answer.
\([0.504, 0.696]\text{.}\)

2.

Suppose in a sample of 100 people, 12 are left-handed.

(a)

Give a 95% confidence interval for the proportion \(p\) of left-handed people.
Answer.
\([0.056, 0.184]\text{.}\)

(b)

Give a 90% confidence interval.
Answer.
\([0.066, 0.174]\text{.}\)

3.

The weights of five mice are measured and recorded below. Give a 95% confidence interval for the sample mean weight of mice. (Pretend 5 measurements is large enough for the CLT to apply.)
Table 4.3.6.
mouse \(i\) weight \(W_i\) (g)
1 26
2 32
3 33
4 20
5 29
Answer.
\([23.39, 32.61]\text{.}\)

4.

The heights of five plants are measured and recorded below. Give a 95% confidence interval around the sample mean for the heights of the plants. (Pretend 5 measurements is large enough for the CLT to apply.)
Table 4.3.7.
plant \(i\) height \(H_i\) (in)
1 15
2 14
3 18
4 21
5 17
Answer.
\([14.61, 19.39]\text{.}\)
PrevTopNext