Sometimes multiple measurements are taken simultaneously and data is naturally grouped by combinations of measurement values. Consider a poll that asks respondents two yes or no questions. It may be natural to report counts of respondents who answered yes/yes, yes/no, no/yes, no/no.
Let \(X\) be a random variable taking values \(x_1, x_2, \dotsc, x_m\text{.}\) Let \(Y\) be a random variable taking values \(y_1, y_2, \dotsc, y_n\text{.}\) The joint distribution for \(X\) and \(Y\) is the collection of probabilities \(\Pr(X = x_i, Y = y_j)\) for all \(1 \leq i \leq m, 1 \leq j \leq n\text{.}\) The individual distributions for \(X\) and \(Y\) separately are called the marginal distributions.
In the probability \(\Pr(X = x_i, Y = y_j)\text{,}\) the comma should be read as "and". Think of the conditions \(X = x_i\) and \(Y = y_j\) as events, and \(\Pr(X = x_i, Y = y_j)\) is the probability of the intersection of these two events.
We’re restricting our attention here to joint distributions of two random variables, each taking only finitely many values. It shouldn’t be hard to imagine how to generalize this definition for a joint distribution of three or more discrete random variables. A bit more adjustment would need to be made for a joint distribution of continuous random variables, or a joint distribution between a discrete and continuous random variable.
A poll asks two yes/no questions. Let \(X\) indicate a yes on Question 1 and \(Y\) indicate a yes on Question 2. When the data is collected, the following joint distribution for \(X\) and \(Y\) is created:
Note that the sum of all values in the table is 1; this is a probability distribution, and total probability must be 1. This table doesn’t show the distributions for \(X\) or \(Y\) individually. The data for the two distributions is mixed together. We can, if we wish, take this information and determine separate distributions for \(X\) and \(Y\text{.}\) For the distribution for \(X\text{,}\) we need to know \(\Pr(X = 0)\) and \(\Pr(X = 1)\) (with no reference to \(Y\)). We can get these probabilities by summing along the columns of the table:
If we start with the joint distribution, we can get the marginal distributions. If we start with the marginal distributions, we can’t necessarily find the joint distribution, because we won’t know how the random variables interact.
Random variables \(X\) and \(Y\) are called independent if the events \(X = x\) and \(Y = y\) are independent for every possible combination of \(x\) and \(y\text{.}\)
Using the joint distribution and marginal distributions in Example 2.3.2, we can check independence of \(X\) and \(Y\) by going cell by cell through the joint distribution table. The top-left cell shows \(\Pr(X = 0, Y = 0) = 0.1\text{.}\) The marginal distributions have \(\Pr(X = 0) = 0.4\) and \(\Pr(Y = 0) = 0.3\text{.}\) If \(X\) and \(Y\) were independent, we would have
Note that we can stop checking now. As soon as we find one cell in the joint distribution table where the independence condition fails, the two random variables are not independent. In order to establish independence of random variables, all cells in the joint distribution table would have to pass our check.
