Covariance
+
+
+
+ Let X and Y be random variables.
+ The covariance of X and Y is:
+
+ \Cov(X, Y) = \E\left[\left(X - \E(X)\right) \left(Y - \E(Y)\right)\right].
+
+
+
+
+
- Text of section.
+ Let's write \mu_X = \E(X) and \mu_Y = \E(Y) to make this is a bit cleaner to look at:
+
+ \Cov(X, Y) = \E\left[\left(X - \mu_X \right) \left(Y - \mu_Y\right)\right].
+
+ This looks like a pretty natural generalization of the variance.
+ In fact:
+
+ \Cov(X, X) \amp = \E\left[ (X - \mu_X)(X - \mu_X)\right] = \E\left[\left(X - \mu_X\right)^2\right] = \Var(X).
+
+ One difference worth noting is that \Var(X), as the average of squared values, must always be nonnegative.
+ The expression (X - \mu_X)(X - \mu_X) can be negative for combinations of X and Y values where one is greater than its average and the other is smaller.
+ If this is the average behavior, then \Cov(X, Y) can be negative overall.
+ This is useful information: a negative \Cov(X, Y) would imply that (on average, not necessarily always) when X is larger than typical, Y is smaller than typical, and vice versa.
+
+ Also as with variance, the covariance definition formula gives us some useful insight, but it can be unwieldy to work with.
+ There is an alternative formula, analogous to :
+
+
+
+
+
+ Let X and Y be random variables.
+ Then:
+
+ \Cov(X, Y) = \E\left(XY\right) - \E(X)\E(Y).
+
+
+
+
+
+
+ To make use of this formula, it's helpful to have both a joint distribution and marginal distributions.
+ The marginal distributions let you find \E(X) and \E(Y).
+ The joint distribution lets you find \E(XY).
+
+
+
+
+
+ Let X and Y be indicator random variables with joint and marginal distributions below.
+
+
+
+
+
+
+
+
+ |
+ X = 0 |
+ X = 1 |
+
+
+
+ | Y = 0 |
+ 0.2 |
+ 0.1 |
+
+
+
+ | Y = 1 |
+ 0.2 |
+ 0.5 |
+
+
+
+
+
+
+
+ | x | \Pr(X = x) |
+
+
+ | 0 | 0.4 |
+
+
+ | 1 | 0.6 |
+
+
+
+
+
+
+
+ | y | \Pr(Y = y) |
+
+
+ | 0 | 0.3 |
+
+
+ | 1 | 0.7 |
+
+
+
+
+
+ Since they're indicator random variables, we can immediately see that \E(X) = 0.6 and \E(Y) = 0.7 (see ).
+ For \E(XY), we'll traverse cell by cell through the joint distribution table, constructing products of the form X-value times Y-value times probability:
+
+ \E(XY) =\, \amp (0)(0)(0.2) + (1)(0)(0.1)
+ \amp (0)(1)(0.2) + (1)(1)(0.5)
+ =\, \amp 0.5.
+
+ Finally:
+
+ \Cov(X, Y) = \E(XY) - \E(X)\E(Y) = 0.5 - (0.6)(0.7) = 0.08.
+
+
+
+
+
+
diff --git a/source/docinfo.ptx b/source/docinfo.ptx
index ccf7d92..1a35ef9 100644
--- a/source/docinfo.ptx
+++ b/source/docinfo.ptx
@@ -35,6 +35,7 @@
\DeclareMathOperator{\E}{E}
\DeclareMathOperator{\Var}{Var}
+ \DeclareMathOperator{\Cov}{Cov}