From e3347701e7afd09852efb4540b917705433fcf55 Mon Sep 17 00:00:00 2001 From: andyeisenberg Date: Thu, 19 Mar 2026 08:38:49 -0400 Subject: [PATCH] More Ch 5 work --- source/sec-Power.ptx | 93 ++++++++++++++++++++++++++- source/sec-Two-Sample-Tests.ptx | 107 +++++++++++++++++++++++++++++++- 2 files changed, 198 insertions(+), 2 deletions(-) diff --git a/source/sec-Power.ptx b/source/sec-Power.ptx index ab0af33..9210ca6 100644 --- a/source/sec-Power.ptx +++ b/source/sec-Power.ptx @@ -2,7 +2,98 @@ Power of a Test

- Text of section. + Recall that the power of a test is the probability of rejecting a false null hypothesis. + To understand the power, we must know: +

    +
  1. +

    + What data would lead us to reject the null hypothesis? +

    +
    2. + +
  2. +

    + What is the probability of seeing such data? +

    +
    3. +
+

+ + + +

+ We find a coin and wonder if it's fair. + Let \theta be the probability of the coin coming up heads on a flip. + Let S count the number of heads in 100 flips. + Under the null hypothesis that \theta = 0.5, S is binomial, with: + + \E(S) \amp = (n)(\theta) = (100)(0.5) = 50 + \Var(S) \amp = (n)(\theta)(1 - \theta) = (100)(0.5)(0.5) = 25 + + Using a normal approximation, S \approx \Norm(50, 25). + In the calculation of the p-value for a 2-tailed test, we will eventually reach the expression: + + \Pr(Z \geq z) + \Pr(Z \leq -z) + + for some z-score. + To address the first question above, we can calculate, for a specified significance level \alpha, the z-score that represents data at the cutoff between accepting and rejecting the null hypothesis: + + \alpha \amp = \Pr(Z \geq z) + \Pr(Z \leq -z) + \amp = \Phi(-z) + (1 - \Phi(z)) + \amp = 2 \Phi(-z) + \Rightarrow \Phi(-z) \amp = \frac{\alpha}{2} + -z \amp = \Phi^{-1}\left(\frac{\alpha}{2}\right) + + For the significance level \alpha = 0.05, we'll have z = 1.96. + Now, we convert back to values of S, keeping in mind that a continuity correction would be involved: + + \frac{S - 0.5 - 50}{\sqrt{25}} \amp = 1.96 + S \amp = 50.5 + 1.96 \sqrt{25} = 60.3. + + We can't flip a fractional number of heads, so it would take a result of 61 or more heads (or, equally extreme in the other direction, 39 heads or fewer) to reject the null hypothesis. +

+ + + +

+ To address the second question, we find a obstacle. + How can we calculate the probability of seeing data extreme enough data to reject H_0 without knowing the true value of \theta? In fact, the power is not a single number, but a function of \theta. + For each hypothetical value of \theta, we'll calculate the probability of seeing extreme enough data to reject H_0. +

+ + + +

+ Continuing the previous example, we know that, if S \geq 61 or S \leq 39, we would reject H_0. + For a given value of \theta: + + \E(S) \amp = n\theta = 100\theta + \Var(S) \amp = n\theta(1-\theta) = 100\theta(1-\theta) + \Pr(S \geq 61) + \Pr(S \leq 39) \amp \approx \Pr\left(Z \geq \frac{60.5 - 100\theta}{\sqrt{100\theta(1 - \theta)}}\right) + \Pr\left(Z \leq \frac{39.5 - 100\theta}{\sqrt{100\theta(1 - \theta)}}\right) + + For example, if the true value of the parameter is \theta = 0.5: + + \Pr(S \geq 61) + \Pr(S \leq 39) \amp \approx \Pr(Z \geq 1.11) + \Pr(Z \leq -3.12) + \amp \approx (1 - 0.8665) + 0.0009 + \amp = 0.1344 + + If, instead, \theta = 0.6: + + \Pr(S \geq 61) + \Pr(S \leq 39) \amp \approx \Pr(Z \geq 0.10) + \Pr(Z \leq -4.18) + \amp \approx (1 - 0.5398) + 0 + \amp = 0.4602 + + (Note: for the z-score -4.18, which is below the range covered by , we will treat the \Phi value as close enough to 0. + For a z-score outside the range on the other side, we would treat the \Phi value as 1. + If we needed more precision than that, we would consult an expanded table.) +

+ + + +

+ We can see in the example that the power of the test was greater when \theta = 0.6 than when \theta = 0.55. + The null hypothesis was \theta = 0.5. + It is reasonable that, if the coin is more different from the null hypothesis, then it will have a higher probabiilty of producing data extreme enough to reject the null hypothesis.

diff --git a/source/sec-Two-Sample-Tests.ptx b/source/sec-Two-Sample-Tests.ptx index badb7cc..0bc4ae4 100644 --- a/source/sec-Two-Sample-Tests.ptx +++ b/source/sec-Two-Sample-Tests.ptx @@ -2,9 +2,114 @@ Two Sample Tests

- Text of section. + In the previous section, we compared data collected from a sample against a known or supposed baseline. + But the statistics for that baseline would also generally be approximated by collecting data from a sample. + We'll now consider hypothesis tests that directly involve two samples of data, one called a control group and the other an experimental group.

+ + +

+ Suppose a sample of 50 plants have a mean height of 35 cm. + A separate sample of 30 plants are taken from another plot which has been using a new fertilizer intended to increase plant growth, and they have a mean height of 36.4 cm. + If the height of a plant has a variance of \sigma^2 = 12 \text{cm}^2, do we have strong enough evidence to reject the null hypothesis that the new fertilizer has no effect on plant height? +

+ +

+ Let A_1 be the random variable representing average height of the control group of plants. + This sample has size n_1 = 50, and we measured the average height \mu_1 = 35. + The variance of the average height would be: + + \Var(A_1) \amp = \frac{\sigma^2}{n_1} = \frac{12}{50} = 0.24. + + Similarly, let A_2 be the random variable representing average height of the experimental group of plants. + This sample has size n_2 = 30, and we measured the average height \mu_2 = 36.4. + The variance of the average height would be: + + \Var(A_2) \amp = \frac{\sigma^2}{n_2} = \frac{12}{30} = 0.4. + + Let D = A_2 - A_1 be the difference between the average heights. +

+ +

+ According to the null hypothesis, the two plots of plants should behave the same way, so \E(D) = 0. + We measured the value: + + D = A_2 - A_1 = 36.4 - 35 = 1.4 + + The variance would be: + + \Var(D) = \Var(A_2 - A_1) = \Var(A_2) + \Var(A_1) = 0.24 + 0.4 = 0.64. + + So, under the null hypothesis, we will treat D as \Norm(0, 0.64). + We then perform a 1-tailed test: + + \Pr(D \geq 1.4) \amp \approx \Pr\left(Z \geq \frac{1.4 - 0}{\sqrt{0.64}}\right) + \amp = 1 - \Phi(1.75) + \amp = 1 - 0.9599 + \amp = 0.0401 \lt 0.05, + + so we can reject the null hypothesis. +

+ + + +

+ However, we don't know \sigma^2, so we should replace it with the sample variance. +

+ + + +

+ Suppose a sample of 100 people from the general population contains 10 left-handed people, and a sample of 100 patients from a carpal tunnel syndrome clinic contains 16 left-handed people. + If \theta_1 is the proportion of left-handed people in the general population, then the maximum likelihood estimation based on our control group sample would be: + + \widehat{\theta}_1 = \frac{10}{100} = 0.1. + + If \theta_2 is the proportion of left-handed people in the patient population, then the maximum likelihood estimate would be: + + \widehat{\theta}_2 = \frac{16}{100} = 0.16. + + Write A_1 for the proportion of the control group sample which is left-handed and A_2 for the proportion of the experimental group sample. + Then we can estimate the variances: + + n_1 \amp = 100 \amp n_2 \amp = 100 + \Var(A_1) \amp = \frac{n_1 \widehat{\theta}_1(1 - \widehat{\theta}_1)}{n_1^2} \amp \Var(A_2) \amp = \frac{n_2 \widehat{\theta}_2(1 - \widehat{\theta}_2)}{n_2^2} + \amp = \frac{(0.1)(0.9)}{100} \amp \amp = \frac{(0.16)(0.84)}{100} + \amp = 0.0009 \amp \amp = 0.001344 + + Now, if D = A_2 - A_1, then: + + \Var(D) = \Var(A_2) + \Var(A_1), + + however, we should pause a moment here. + The null hypothesis would be that the two populations have the same statistics. + That should include the same variance for an individual measurement. + (The variances of A_1 and A_2 could still be different if, e.g., they were averaging over different sized samples.) So we should abandon our two distinct maximum likelihood estimates \widehat{\theta}_1, \widehat{\theta}_2 and instead find one estimate, \widehat{\theta}_0, estimated from all collected data: + + \widehat{\theta}_0 = \frac{10 + 16}{100 + 100} = \frac{26}{200} = 0.13. + + Then, we should recalculate the variances of A_1, A_2: + + \Var(A_1) \amp = \frac{(0.13)(0.87)}{100} = 0.001131 + \Var(A_2) \amp = \frac{(0.13)(0.87)}{100} = 0.001131 + + Finally: + + \Var(D) = \Var(A_1) + \Var(A_2) = 0.002262. + + Now we can apply a 1-tailed test: + + \Pr(D \geq 0.06) \approx \Pr\left( Z \geq \frac{0.06 - 0}{\sqrt{0.002262}}\right) + \amp \approx 1 - \Phi(1.27) + \amp \approx 1 - 0.8962 + \amp = 0.1038 \gt 0.05, + + so we do not reject the null hypothesis. +

+ + +