# Power of a Test
Keywords: Authored in PreTeXt

Skip to main content
\(\newcommand{\N}{\mathbb N} \newcommand{\Z}{\mathbb Z} \newcommand{\Q}{\mathbb Q} \newcommand{\R}{\mathbb R} \DeclareMathOperator{\Bin}{Bin} \DeclareMathOperator{\Geom}{Geom} \DeclareMathOperator{\Poiss}{Poiss} \DeclareMathOperator{\Exp}{Exp} \DeclareMathOperator{\Norm}{N} \DeclareMathOperator{\E}{E} \DeclareMathOperator{\Var}{Var} \DeclareMathOperator{\Cov}{Cov} \renewcommand{\L}{\mathcal{L}} \newcommand{\est}{\widehat} \newcommand{\lt}{<} \newcommand{\gt}{>} \newcommand{\amp}{&} \definecolor{fillinmathshade}{gray}{0.9} \newcommand{\fillinmath}[1]{\mathchoice{\colorbox{fillinmathshade}{$\displaystyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\textstyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\scriptstyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\scriptscriptstyle\phantom{\,#1\,}$}}} \)

# Section 5.3 Power of a Test

Recall that the power of a test is the probability of rejecting a false null hypothesis. To understand the power, we must know:

1. What data would lead us to reject the null hypothesis?
   🔗
   🔗
2. What is the probability of seeing such data?
   🔗
   🔗

🔗

## Example 5.3.1.

We find a coin and wonder if it’s fair. Let \(\theta\) be the probability of the coin coming up heads on a flip. Let \(S\) count the number of heads in 100 flips. Under the null hypothesis that \(\theta = 0.5\text{,}\) \(S\) is binomial, with:
\begin{align*} \E(S) \amp = (n)(\theta) = (100)(0.5) = 50 \\ \Var(S) \amp = (n)(\theta)(1 - \theta) = (100)(0.5)(0.5) = 25 \end{align*}
Using a normal approximation, \(S \approx \Norm(50, 25)\text{.}\) In the calculation of the \(p\)-value for a 2-tailed test, we will eventually reach the expression:
\begin{gather*} \Pr(Z \geq z) + \Pr(Z \leq -z) \end{gather*}
for some \(z\)-score. To address the first question above, we can calculate, for a specified significance level \(\alpha\text{,}\) the \(z\)-score that represents data at the cutoff between accepting and rejecting the null hypothesis:
\begin{align*} \alpha \amp = \Pr(Z \geq z) + \Pr(Z \leq -z) \\ \amp = \Phi(-z) + (1 - \Phi(z)) \\ \amp = 2 \Phi(-z) \\ \Rightarrow \Phi(-z) \amp = \frac{\alpha}{2} \\ -z \amp = \Phi^{-1}\left(\frac{\alpha}{2}\right) \end{align*}
For the significance level \(\alpha = 0.05\text{,}\) we’ll have \(z = 1.96\text{.}\) Now, we convert back to values of \(S\text{,}\) keeping in mind that a continuity correction would be involved:
\begin{align*} \frac{S - 0.5 - 50}{\sqrt{25}} \amp = 1.96 \\ S \amp = 50.5 + 1.96 \sqrt{25} = 60.3. \end{align*}
We can’t flip a fractional number of heads, so it would take a result of 61 or more heads (or, equally extreme in the other direction, 39 heads or fewer) to reject the null hypothesis.
🔗
🔗
To address the second question, we find a obstacle. How can we calculate the probability of seeing data extreme enough data to reject \(H_0\) without knowing the true value of \(\theta\text{?}\) In fact, the power is not a single number, but a function of \(\theta\text{.}\) For each hypothetical value of \(\theta\text{,}\) we’ll calculate the probability of seeing extreme enough data to reject \(H_0\text{.}\)
🔗

## Example 5.3.2.

Continuing the previous example, we know that, if \(S \geq 61\) or \(S \leq 39\text{,}\) we would reject \(H_0\text{.}\) For a given value of \(\theta\text{:}\)
\begin{align*} \E(S) \amp = n\theta = 100\theta \\ \Var(S) \amp = n\theta(1-\theta) = 100\theta(1-\theta) \\ \Pr(S \geq 61) + \Pr(S \leq 39) \amp \approx \Pr\left(Z \geq \frac{60.5 - 100\theta}{\sqrt{100\theta(1 - \theta)}}\right) + \Pr\left(Z \leq \frac{39.5 - 100\theta}{\sqrt{100\theta(1 - \theta)}}\right) \end{align*}
For example, if the true value of the parameter is \(\theta = 0.5\text{:}\)
\begin{align*} \Pr(S \geq 61) + \Pr(S \leq 39) \amp \approx \Pr(Z \geq 1.11) + \Pr(Z \leq -3.12) \\ \amp \approx (1 - 0.8665) + 0.0009 \\ \amp = 0.1344 \end{align*}
If, instead, \(\theta = 0.6\text{:}\)
\begin{align*} \Pr(S \geq 61) + \Pr(S \leq 39) \amp \approx \Pr(Z \geq 0.10) + \Pr(Z \leq -4.18) \\ \amp \approx (1 - 0.5398) + 0 \\ \amp = 0.4602 \end{align*}
(Note: for the \(z\)-score \(-4.18\text{,}\) which is below the range covered by Section A.1, we will treat the \(\Phi\) value as close enough to 0. For a \(z\)-score outside the range on the other side, we would treat the \(\Phi\) value as 1. If we needed more precision than that, we would consult an expanded table.)
🔗
🔗
We can see in the example that the power of the test was greater when \(\theta = 0.6\) than when \(\theta = 0.55\text{.}\) The null hypothesis was \(\theta = 0.5\text{.}\) It is reasonable that, if the coin is more different from the null hypothesis, then it will have a higher probabiilty of producing data extreme enough to reject the null hypothesis.
🔗

## Exercises Exercises

### 1.

Suppose we find a coin and wonder whether it’s fair. As a first test, we decide to flip the coin 200 times and count the number of heads, \(S\text{.}\) What values of \(S\) would be extreme enough to reject the null hypothesis of a fair coin? If the coin actually has a 0.6 probability of coming up heads, what is the power of this test?
🔗
🔗

### 2.

Suppose we have a coin which we suspect comes up heads more often than a fair coin would. As a first test, we decide to flip the coin 200 times and count the number of heads, \(S\text{.}\) What values of \(S\) would be extreme enough to reject the null hypothesis of a fair coin? If the coin actually has a 0.6 probability of coming up heads, what is the power of this test?
🔗
🔗

### 3.

Suppose we find a six-sided die and wonder whether it’s fair. As a first test, we decide to roll the die 100 times and count the number of times it comes up 1. The expected number of 1’s is 50/3, with a variance of 125/9.
🔗

#### (a)

Using a normal approximation, what is the smallest number of 1’s greater than 50/3 that would be extreme enough to reject the null hypothesis of a fair die?
🔗
🔗

#### (b)

Using a normal approximation, what is the greatest number of 1’s less than 50/3 that would be extreme enough to reject the null hypothesis of a fair die?
🔗
🔗

#### (c)

Suppose that this die is weighted so that it rolls a 1 with probability 0.2. What would be the power of our test?
🔗
🔗

#### (d)

Suppose we roll the die 100 times and see 23 1’s. Use the maximum likelihood value for the probability of rolling a 1 to calculate the power of the test.
🔗
🔗
🔗

### 4.

Suppose a particular plant when grown outdoors has an average height of 39 in with a variance of 20 in\(^2\text{.}\) We suspect that growing this plant in a greenhouse will increase its height. We take the average height of a sample of 50 plants grown in a greenhouse. What is the minimum average height of this sample that would be extreme enough to reject the null hypothesis of equal means at the \(p = 0.05\) significance level? If the plants, when grown in a greenhouse, would truly have an average height of 41 in, what is the power of our test?
🔗
🔗
🔗
🔗
Prev Top Next