Thursday, Feb 5

This is an outline of the topics we covered in class. These notes are not a substitute for your own note-taking. I highly recommend that you take your own notes during class. If you ever miss a class for any reason, reach out to another student in class to get a copy of their notes.

Summary

Here are the expected value and variance formulas for common distributions. Some of these, we've shown justification for. Others requires techniques beyond the scope of the class to justify.

Expected Value and Variance Formulas Distribution Parameters Expected Value Variance Indicator p p p(1-p) Binomial n, p np np(1-p) Geometric p \frac{1}{p} \frac{1-p}{p^2} Poisson \lambda \lambda \lambda Exponential \lambda \frac{1}{\lambda} \frac{1}{\lambda^2}
Covariance

If X, Y are random variables with expected values of \mu_X, \mu_Y, then the covariance of X and Y is: \Cov(X, Y) \amp = \E\left[ (X - \mu_X)(Y - \mu_Y)\right].

Observe that: \Cov(X, X) \amp = \E\left[(X - \mu_X)(X - \mu_X)\right] \amp = \E\left[(X - \mu_X)^2\right] \amp = \Var(X), so covariance generalizes the variance formula to two variables. As with variance, there's an alternative formula more suited to doing computations: \Var(X) \amp = \E\left[(X - \mu_X)(X - \mu_X)\right] = \E(X^2) - \mu_X^2 \Cov(X, Y) \amp = \E\left[(X - \mu_X)(Y - \mu_Y)\right] = \E(XY) - \mu_X\mu_Y

Consider the joint distribution table:

Joint Distribution X = 0 X = 1 Y = 0 0.2 0.1 Y = 1 0.05 0.65

From the table, we can calculate the marginal distributions: \Pr(X = 0) \amp = 0.25 \amp \Pr(Y = 0) \amp = 0.3 \Pr(X = 1) \amp = 0.75 \amp \Pr(Y = 1) \amp = 0.7 So \E(X) = \mu_X = 0.75 and \E(Y) = \mu_Y = 0.7. Then: \E(XY) \amp = (0)(0)(0.2) + (1)(0)(0.1) + (0)(1)(0.05) + (1)(1)(0.65) \amp = 0.65 \Cov(X, Y) \amp = \E(XY) - \mu_X \mu_Y \amp = 0.65 - (0.75)(0.7) \amp = 0.125.

Question: the formula \Var(X) = \E\left[(X - \mu_X)^2\right] makes it clear that variance cannot be negative, since squares are nonnegative. What about \Cov(X, Y)?

In the previous example, since X, Y were both indicator random variables, the variances for each were simply equal to the sum of the second row/column. Similarly, \E(XY) was equal to the X = 1, Y = 1 entry in the table. Using two indicator random variables, significantly simplifies the covariance calculation, so we can vary the table and recalculate covariance quickly. Consider the following joint distribution:

Joint Distribution X = 0 X = 1 Y = 0 0.1 0.4 Y = 1 0.3 0.2

Then: \Cov(X, Y) = 0.2 - (0.6)(0.5) = -0.1 \lt 0.

Question: how do we interpret \Cov(X, Y)? X - \mu_X is positive when X \gt \mu_X and negative when X \lt \mu_X. Y - \mu_Y is positive when Y \gt \mu_Y and negative when Y \lt \mu_Y. So the product (X - \mu_X)(Y - \mu_Y) is positive when X, Y are both larger or both smaller than their expected values, and negative when one is larger and one is smaller. That is, covariance tries to quantify the tendency of X, Y to get big/small at the same time.