179 lines
7.6 KiB
Plaintext
179 lines
7.6 KiB
Plaintext
<section xml:id="sec-Power" xmlns:xi="http://www.w3.org/2001/XInclude">
|
|
<title>Power of a Test</title>
|
|
|
|
<p>
|
|
Recall that the power of a test is the probability of rejecting a false null hypothesis.
|
|
To understand the power, we must know:
|
|
<ol>
|
|
<li>
|
|
<p>
|
|
What data would lead us to reject the null hypothesis?
|
|
</p>
|
|
</li>
|
|
|
|
<li>
|
|
<p>
|
|
What is the probability of seeing such data?
|
|
</p>
|
|
</li>
|
|
</ol>
|
|
</p>
|
|
|
|
<example>
|
|
<statement>
|
|
<p>
|
|
We find a coin and wonder if it's fair.
|
|
Let <m>\theta</m> be the probability of the coin coming up heads on a flip.
|
|
Let <m>S</m> count the number of heads in 100 flips.
|
|
Under the null hypothesis that <m>\theta = 0.5</m>, <m>S</m> is binomial, with:
|
|
<md>
|
|
<mrow> \E(S) \amp = (n)(\theta) = (100)(0.5) = 50 </mrow>
|
|
<mrow> \Var(S) \amp = (n)(\theta)(1 - \theta) = (100)(0.5)(0.5) = 25 </mrow>
|
|
</md>
|
|
Using a normal approximation, <m>S \approx \Norm(50, 25)</m>.
|
|
In the calculation of the <m>p</m>-value for a 2-tailed test, we will eventually reach the expression:
|
|
<md>
|
|
<mrow> \Pr(Z \geq z) + \Pr(Z \leq -z) </mrow>
|
|
</md>
|
|
for some <m>z</m>-score.
|
|
To address the first question above, we can calculate, for a specified significance level <m>\alpha</m>, the <m>z</m>-score that represents data at the cutoff between accepting and rejecting the null hypothesis:
|
|
<md>
|
|
<mrow> \alpha \amp = \Pr(Z \geq z) + \Pr(Z \leq -z) </mrow>
|
|
<mrow> \amp = \Phi(-z) + (1 - \Phi(z)) </mrow>
|
|
<mrow> \amp = 2 \Phi(-z) </mrow>
|
|
<mrow> \Rightarrow \Phi(-z) \amp = \frac{\alpha}{2} </mrow>
|
|
<mrow> -z \amp = \Phi^{-1}\left(\frac{\alpha}{2}\right) </mrow>
|
|
</md>
|
|
For the significance level <m>\alpha = 0.05</m>, we'll have <m>z = 1.96</m>.
|
|
Now, we convert back to values of <m>S</m>, keeping in mind that a continuity correction would be involved:
|
|
<md>
|
|
<mrow> \frac{S - 0.5 - 50}{\sqrt{25}} \amp = 1.96 </mrow>
|
|
<mrow> S \amp = 50.5 + 1.96 \sqrt{25} = 60.3. </mrow>
|
|
</md>
|
|
We can't flip a fractional number of heads, so it would take a result of 61 or more heads (or, equally extreme in the other direction, 39 heads or fewer) to reject the null hypothesis.
|
|
</p>
|
|
</statement>
|
|
</example>
|
|
|
|
<p>
|
|
To address the second question, we find a obstacle.
|
|
How can we calculate the probability of seeing data extreme enough data to reject <m>H_0</m> without knowing the true value of <m>\theta</m>? In fact, the power is not a single number, but a function of <m>\theta</m>.
|
|
For each hypothetical value of <m>\theta</m>, we'll calculate the probability of seeing extreme enough data to reject <m>H_0</m>.
|
|
</p>
|
|
|
|
<example>
|
|
<statement>
|
|
<p>
|
|
Continuing the previous example, we know that, if <m>S \geq 61</m> or <m>S \leq 39</m>, we would reject <m>H_0</m>.
|
|
For a given value of <m>\theta</m>:
|
|
<md>
|
|
<mrow> \E(S) \amp = n\theta = 100\theta </mrow>
|
|
<mrow> \Var(S) \amp = n\theta(1-\theta) = 100\theta(1-\theta) </mrow>
|
|
<mrow> \Pr(S \geq 61) + \Pr(S \leq 39) \amp \approx \Pr\left(Z \geq \frac{60.5 - 100\theta}{\sqrt{100\theta(1 - \theta)}}\right) + \Pr\left(Z \leq \frac{39.5 - 100\theta}{\sqrt{100\theta(1 - \theta)}}\right) </mrow>
|
|
</md>
|
|
For example, if the true value of the parameter is <m>\theta = 0.5</m>:
|
|
<md>
|
|
<mrow> \Pr(S \geq 61) + \Pr(S \leq 39) \amp \approx \Pr(Z \geq 1.11) + \Pr(Z \leq -3.12) </mrow>
|
|
<mrow> \amp \approx (1 - 0.8665) + 0.0009 </mrow>
|
|
<mrow> \amp = 0.1344 </mrow>
|
|
</md>
|
|
If, instead, <m>\theta = 0.6</m>:
|
|
<md>
|
|
<mrow> \Pr(S \geq 61) + \Pr(S \leq 39) \amp \approx \Pr(Z \geq 0.10) + \Pr(Z \leq -4.18) </mrow>
|
|
<mrow> \amp \approx (1 - 0.5398) + 0 </mrow>
|
|
<mrow> \amp = 0.4602 </mrow>
|
|
</md>
|
|
(Note: for the <m>z</m>-score <m>-4.18</m>, which is below the range covered by <xref ref="app-Phi-table"/>, we will treat the <m>\Phi</m> value as close enough to 0.
|
|
For a <m>z</m>-score outside the range on the other side, we would treat the <m>\Phi</m> value as 1.
|
|
If we needed more precision than that, we would consult an expanded table.)
|
|
</p>
|
|
</statement>
|
|
</example>
|
|
|
|
<p>
|
|
We can see in the example that the power of the test was greater when <m>\theta = 0.6</m> than when <m>\theta = 0.55</m>.
|
|
The null hypothesis was <m>\theta = 0.5</m>.
|
|
It is reasonable that, if the coin is <em>more different</em> from the null hypothesis, then it will have a higher probabiilty of producing data extreme enough to reject the null hypothesis.
|
|
</p>
|
|
|
|
<exercises xml:id="exercises-Power">
|
|
<exercise>
|
|
<statement>
|
|
<p>
|
|
Suppose we find a coin and wonder whether it's fair.
|
|
As a first test, we decide to flip the coin 200 times and count the number of heads, <m>S</m>.
|
|
What values of <m>S</m> would be extreme enough to reject the null hypothesis of a fair coin? If the coin actually has a 0.6 probability of coming up heads, what is the power of this test?
|
|
</p>
|
|
</statement>
|
|
</exercise>
|
|
|
|
<exercise>
|
|
<statement>
|
|
<p>
|
|
Suppose we have a coin which we suspect comes up heads more often than a fair coin would.
|
|
As a first test, we decide to flip the coin 200 times and count the number of heads, <m>S</m>.
|
|
What values of <m>S</m> would be extreme enough to reject the null hypothesis of a fair coin? If the coin actually has a 0.6 probability of coming up heads, what is the power of this test?
|
|
</p>
|
|
</statement>
|
|
</exercise>
|
|
|
|
<exercise>
|
|
<introduction>
|
|
<p>
|
|
Suppose we find a six-sided die and wonder whether it's fair.
|
|
As a first test, we decide to roll the die 100 times and count the number of times it comes up 1.
|
|
The expected number of 1's is 50/3, with a variance of 125/9.
|
|
</p>
|
|
</introduction>
|
|
|
|
|
|
<task>
|
|
<statement>
|
|
<p>
|
|
Using a normal approximation, what is the smallest number of 1's greater than 50/3 that would be extreme enough to reject the null hypothesis of a fair die?
|
|
</p>
|
|
</statement>
|
|
</task>
|
|
|
|
|
|
<task>
|
|
<statement>
|
|
<p>
|
|
Using a normal approximation, what is the greatest number of 1's less than 50/3 that would be extreme enough to reject the null hypothesis of a fair die?
|
|
</p>
|
|
</statement>
|
|
</task>
|
|
|
|
|
|
<task>
|
|
<statement>
|
|
<p>
|
|
Suppose that this die is weighted so that it rolls a 1 with probability 0.2.
|
|
What would be the power of our test?
|
|
</p>
|
|
</statement>
|
|
</task>
|
|
|
|
|
|
<task>
|
|
<statement>
|
|
<p>
|
|
Suppose we roll the die 100 times and see 23 1's.
|
|
Use the maximum likelihood value for the probability of rolling a 1 to calculate the power of the test.
|
|
</p>
|
|
</statement>
|
|
</task>
|
|
</exercise>
|
|
|
|
<exercise>
|
|
<statement>
|
|
<p>
|
|
Suppose a particular plant when grown outdoors has an average height of 39 in with a variance of 20 in<m>^2</m>.
|
|
We suspect that growing this plant in a greenhouse will increase its height.
|
|
We take the average height of a sample of 50 plants grown in a greenhouse.
|
|
What is the minimum average height of this sample that would be extreme enough to reject the null hypothesis of equal means at the <m>p = 0.05</m> significance level? If the plants, when grown in a greenhouse, would truly have an average height of 41 in, what is the power of our test?
|
|
</p>
|
|
</statement>
|
|
</exercise>
|
|
</exercises>
|
|
</section> |