Two Sample Tests

In the previous section, we compared data collected from a sample against a known or supposed baseline. But the statistics for that baseline would also generally be approximated by collecting data from a sample. We'll now consider hypothesis tests that directly involve two samples of data, one called a control group and the other an experimental group.

Suppose a sample of 50 plants have a mean height of 35 cm. A separate sample of 30 plants are taken from another plot which has been using a new fertilizer intended to increase plant growth, and they have a mean height of 36.4 cm. If the height of a plant has a variance of \sigma^2 = 12 \text{cm}^2, do we have strong enough evidence to reject the null hypothesis that the new fertilizer has no effect on plant height?

Let A_1 be the random variable representing average height of the control group of plants. This sample has size n_1 = 50, and we measured the average height \mu_1 = 35. The variance of the average height would be: \Var(A_1) \amp = \frac{\sigma^2}{n_1} = \frac{12}{50} = 0.24. Similarly, let A_2 be the random variable representing average height of the experimental group of plants. This sample has size n_2 = 30, and we measured the average height \mu_2 = 36.4. The variance of the average height would be: \Var(A_2) \amp = \frac{\sigma^2}{n_2} = \frac{12}{30} = 0.4. Let D = A_2 - A_1 be the difference between the average heights.

According to the null hypothesis, the two plots of plants should behave the same way, so \E(D) = 0. We measured the value: D = A_2 - A_1 = 36.4 - 35 = 1.4 The variance would be: \Var(D) = \Var(A_2 - A_1) = \Var(A_2) + \Var(A_1) = 0.24 + 0.4 = 0.64. So, under the null hypothesis, we will treat D as \Norm(0, 0.64). We then perform a 1-tailed test: \Pr(D \geq 1.4) \amp \approx \Pr\left(Z \geq \frac{1.4 - 0}{\sqrt{0.64}}\right) \amp = 1 - \Phi(1.75) \amp = 1 - 0.9599 \amp = 0.0401 \lt 0.05, so we can reject the null hypothesis.

However, we don't know \sigma^2, so we should replace it with the sample variance.

Suppose a sample of 100 people from the general population contains 10 left-handed people, and a sample of 100 patients from a carpal tunnel syndrome clinic contains 16 left-handed people. If \theta_1 is the proportion of left-handed people in the general population, then the maximum likelihood estimation based on our control group sample would be: \widehat{\theta}_1 = \frac{10}{100} = 0.1. If \theta_2 is the proportion of left-handed people in the patient population, then the maximum likelihood estimate would be: \widehat{\theta}_2 = \frac{16}{100} = 0.16. Write A_1 for the proportion of the control group sample which is left-handed and A_2 for the proportion of the experimental group sample. Then we can estimate the variances: n_1 \amp = 100 \amp n_2 \amp = 100 \Var(A_1) \amp = \frac{n_1 \widehat{\theta}_1(1 - \widehat{\theta}_1)}{n_1^2} \amp \Var(A_2) \amp = \frac{n_2 \widehat{\theta}_2(1 - \widehat{\theta}_2)}{n_2^2} \amp = \frac{(0.1)(0.9)}{100} \amp \amp = \frac{(0.16)(0.84)}{100} \amp = 0.0009 \amp \amp = 0.001344 Now, if D = A_2 - A_1, then: \Var(D) = \Var(A_2) + \Var(A_1), however, we should pause a moment here. The null hypothesis would be that the two populations have the same statistics. That should include the same variance for an individual measurement. (The variances of A_1 and A_2 could still be different if, e.g., they were averaging over different sized samples.) So we should abandon our two distinct maximum likelihood estimates \widehat{\theta}_1, \widehat{\theta}_2 and instead find one estimate, \widehat{\theta}_0, estimated from all collected data: \widehat{\theta}_0 = \frac{10 + 16}{100 + 100} = \frac{26}{200} = 0.13. Then, we should recalculate the variances of A_1, A_2: \Var(A_1) \amp = \frac{(0.13)(0.87)}{100} = 0.001131 \Var(A_2) \amp = \frac{(0.13)(0.87)}{100} = 0.001131 Finally: \Var(D) = \Var(A_1) + \Var(A_2) = 0.002262. Now we can apply a 1-tailed test: \Pr(D \geq 0.06) \amp \approx \Pr\left( Z \geq \frac{0.06 - 0}{\sqrt{0.002262}}\right) \amp \approx 1 - \Phi(1.27) \amp \approx 1 - 0.8962 \amp = 0.1038 \gt 0.05, so we do not reject the null hypothesis.

A farmer is testing an experimental new plant fertilizer that is supposed to increase the weight of a particular apple variety. A control sample of 25 apples grown using the usual fertilizer have a mean weight of 75 grams and a sample variance of 90 grams^2 (for an individual apple). An experimental sample of 25 apples grown using the new fertilizer have a mean weight of 79 grams and a sample variance of 90 grams^2.

We have an established factory which produces coins that are close to fair. We're opening up a second factory, and we'd like to ensure the machines are calibrated to produce coins which behave similarly to the ones produced in the established factory. We pick one sample coin from each factory, and flip each sample coin 100 times. The coin from the established factory flips 52 heads in 100 flips. The coin from the new factory flips 62 heads in 100 flips. Is this strong enough evidence to reject the null hypothesis that the two factories produce similar coins at a 0.05 significance level?