Section 3.3 Covariance

Definition 3.3.1.

Let \(X\) and \(Y\) be random variables. The covariance of \(X\) and \(Y\) is:

\begin{gather*} \Cov(X, Y) = \E\left[\left(X - \E(X)\right) \left(Y - \E(Y)\right)\right]. \end{gather*}

Let’s write \(\mu_X = \E(X)\) and \(\mu_Y = \E(Y)\) to make this is a bit cleaner to look at:

\begin{gather*} \Cov(X, Y) = \E\left[\left(X - \mu_X \right) \left(Y - \mu_Y\right)\right]. \end{gather*}

This looks like a pretty natural generalization of the variance. In fact:

\begin{align*} \Cov(X, X) \amp = \E\left[ (X - \mu_X)(X - \mu_X)\right] = \E\left[\left(X - \mu_X\right)^2\right] = \Var(X). \end{align*}

One difference worth noting is that \(\Var(X)\text{,}\) as the average of squared values, must always be nonnegative. The expression \((X - \mu_X)(X - \mu_X)\) can be negative for combinations of \(X\) and \(Y\) values where one is greater than its average and the other is smaller. If this is the average behavior, then \(\Cov(X, Y)\) can be negative overall. This is useful information: a negative \(\Cov(X, Y)\) would imply that (on average, not necessarily always) when \(X\) is larger than typical, \(Y\) is smaller than typical, and vice versa.

Also as with variance, the covariance definition formula gives us some useful insight, but it can be unwieldy to work with. There is an alternative formula, analogous to Theorem 3.2.2:

Theorem 3.3.2.

Let \(X\) and \(Y\) be random variables. Then:

\begin{gather*} \Cov(X, Y) = \E\left(XY\right) - \E(X)\E(Y). \end{gather*}

To make use of this formula, it’s helpful to have both a joint distribution and marginal distributions. The marginal distributions let you find \(\E(X)\) and \(\E(Y)\text{.}\) The joint distribution lets you find \(\E(XY)\text{.}\)

Example 3.3.3.

Let \(X\) and \(Y\) be indicator random variables with joint and marginal distributions below.

	\(X = 0\)	\(X = 1\)
\(Y = 0\)	0.2	0.1
\(Y = 1\)	0.2	0.5

Table 3.3.4.

\(x\)	\(\Pr(X = x)\)
0	0.4
1	0.6

Table 3.3.5.

\(y\)	\(\Pr(Y = y)\)
0	0.3
1	0.7

Table 3.3.6.

Since they’re indicator random variables, we can immediately see that \(\E(X) = 0.6\) and \(\E(Y) = 0.7\) (see Example 3.1.5). For \(\E(XY)\text{,}\) we’ll traverse cell by cell through the joint distribution table, constructing products of the form \(X\)-value times \(Y\)-value times probability:

\begin{align*} \E(XY) =\, \amp (0)(0)(0.2) + (1)(0)(0.1) \\ \amp (0)(1)(0.2) + (1)(1)(0.5) \\ =\, \amp 0.5. \end{align*}

Finally:

\begin{gather*} \Cov(X, Y) = \E(XY) - \E(X)\E(Y) = 0.5 - (0.6)(0.7) = 0.08. \end{gather*}

Exercises Exercises

1.

Calculate the covariance of \(X\) and \(Y\text{:}\)

Table 3.3.7.

	\(X = 1\)	\(X = 2\)
\(Y = 0\)	0.12	0.24
\(Y = 1\)	0.3	34

2.

Calculate the covariance of \(X\) and \(Y\text{:}\)

Table 3.3.8.

	\(X = 0\)	\(X = 1\)	\(X = 2\)
\(Y = 0\)	0.08	0.16	0.1
\(Y = 1\)	0.14	0.2	0.32

3.

Suppose we roll a fair, 4-sided die two times. Let \(X\) be the sum of the rolls, and let \(Y\) be the product of the rolls. Find the covariance of \(X\) and \(Y\text{.}\)

[Note: Somewhat tedious.]

Prev Top Next