Definition 2.1.1.
Let \(\Omega\) be a sample space. A random variable is a function \(X \colon \Omega \to \R\) which assigns a number to each outcome.
Section 2.1 Discrete Random Variables
Subsection 2.1.1 Discrete Random Variables
Example 2.1.2.
Suppose an experiment consists of flipping a fair coin 4 times. Let \(X\) be the number of heads. The actual outcomes in the experiment are heads/tails sequences of length 4, such as \(HTHT\) and \(HHHT\text{.}\) The random variable \(X\) assigns a numerical measurement to the outcomes, such as \(X(HTHT) = 2\) and \(X(HHHT) = 3\text{.}\)
For any real number value \(a\text{,}\) we can consider the event consisting of outcomes such that \(X = a\text{.}\) For example, the flip sequences \(HHHT, HHTH, HTHH\text{,}\) and \(THHH\) all have 3 heads, so \(\Pr(X = 3) = \frac{4}{16} = \frac{1}{4}\text{.}\)
Example 2.1.3. Indicator Random Variable.
Let \(A\) be an event in the sample space \(\Omega\text{.}\) An indicator random variable for \(A\) is the random variable \(X\) such that
\begin{align*}
X(\omega) = \begin{cases} 1 \amp \omega \in A \\ 0 \amp \omega \notin A \end{cases}
\end{align*}
Indicator random variables are also called Bernoulli random variables, although we’ll prefer the former term since it more clearly states the purpose of these random variables: to indicate whether or not a particular event has occurred. We’ll use the language "\(X\) indicates \(A\)" to mean that \(X\) is an indicator random variable for the event \(A\text{.}\) In this case, \(\Pr(X = 1) = \Pr(A)\text{,}\) and \(\Pr(X = 0) = 1 - \Pr(A)\text{.}\)
Definition 2.1.4.
Consider the set \(S\) of all real numbers which are actually output by a random variable \(X\text{.}\) If \(S\) does not contain any interval of values, then the random variable \(X\) is called discrete. Otherwise, it’s called continuous.
Example 2.1.5.
Any random variable \(X\) defined on a finite sample space \(\Omega\) is discrete—the set of outputs of \(X\) cannot contain an interval if it only has finitely many values.
Suppose an experiment consists of growing a plant in a new fertilizer, and a random variable \(Y\) measures the height of the plant after a set growing period. Now \(Y\) could conceivably take on an interval’s worth of values (such as, e.g., any real number between 10 inches and 20 inches), so this random variable would be continuous.
The distinction between discrete and continuous random variables is not simply the distinction of whether the sample space is finite or infinite. Consider the experiment in which we flip a coin repeatedly until we first see a coin come up heads. Let \(Z\) be the number of times the coin is flipped. Then there are infinitely many possible values of \(Z\) (1, 2, 3, and so on), but there’s no interval of real numbers which are all possible outputs of \(Z\text{.}\) The possible outputs of \(Z\) are discrete (in the non-technical, English sense of the word: separated).
There are several particular types of distributions that appear repeatedly, since they model common behaviors. Indicator random variables, for example, will appear any time a random process generates a yes/no or success/failure type of answer to a question. Below are some other common distributions.
Subsection 2.1.2 The Binomial Distribution
Definition 2.1.6.
Suppose an event occurs with probability \(p\text{.}\) If we perform \(n\) independent trials, let \(S\) be the random variable which counts the number of trials in which the event occurred. Then \(S\) has the binomial distribution with parameters \(n\) and \(p\text{.}\) We’ll write \(S \sim \Bin(n, p)\) to denote this. For each value \(0 \leq k \leq n\text{,}\) we’ll write \(b(k)\) for \(\Pr(S = k)\text{.}\) (If we want to keep track of the parameter values, we may write \(b(k; n, p)\text{.}\))
Example 2.1.7.
Suppose we have a coin which comes up heads with probability \(p = 0.4\text{.}\) If we flip the coin three times and let \(S\) be the number of heads, then \(S\) is binomially distributed with parameters \(n = 3\) and \(p = 0.4\text{.}\) Then \(S\) has the distribution below.
| \(k\) | \(\Pr(S = k)\) |
|---|---|
| 0 | 0.216 |
| 1 | 0.432 |
| 2 | 0.288 |
| 3 | 0.064 |
To understand how these probabilities are calculated, consider a particular heads/tails sequence, such as \(HHHT\text{.}\) If the probability of heads is 0.4, then the probability of tails is 0.6. Since the results of different coin flips are independent from each other, the probability of seeing precisely the sequence \(HHT\) would be the product \((0.4)(0.4)(0.6) = 0.096\text{.}\) The probability of seeing precisely \(HTH\) would be \((0.4)(0.6)(0.4) = 0.096\text{,}\) the same product of terms in a different order. So, if we knew exactly the number of flip sequences of length 3 with exactly 2 heads, we could calculate the probability of seeing 2 heads as:
\begin{gather*}
\Pr(S = 2) = (\text{number of sequences})(\text{probability of each sequence})
\end{gather*}
The probability for each sequence is straightforward to generalize: if there are \(n\) flips and \(k\) of them are heads, then \(n - k\) of them are tails. So the product we should calculate will have \(k\) copies of the parameter \(p\) and \(n - k\) copies of the complementary probability \(1 - p\text{:}\)
\begin{gather*}
\text{probability of each sequence } = p^k(1-p)^k
\end{gather*}
Definition 2.1.9.
We’ll write the symbol \({n \choose k}\) to mean the number of ways to choose \(k\) things out of \(n\) things. The number \({n \choose k}\) is read "\(n\) choose \(k\)" and called a binomial coefficient.
The term "binomial coefficient" is used here because these numbers form coefficients of a polynomial in an important theorem about binomials called the Binomial Theorem. The binomial coefficients will precisely tell us the number of flip sequences of length \(n\) with precisely \(k\) heads (to build such a sequence, we must choose \(k\) out of \(n\) flips in the sequence to be heads; the rest must be tails). So, now we need a way to find the numbers \({n \choose k}\text{.}\)
Definition 2.1.10.
Let \(n\) be a positive integer. Then the factorial of \(n\text{,}\) written \(n!\text{,}\) is the product of the positive integers up to \(n\text{,}\) i.e.:
\begin{gather*}
n! = 1 \times 2 \times 3 \times \dotsb \times (n-1) \times n.
\end{gather*}
Fact 2.1.11. Binomial Coefficient Formula.
For any \(0 \leq k \leq n\text{,}\)
\begin{gather*}
{n \choose k} = \frac{n!}{k!(n-k)!}
\end{gather*}
We won’t provide a justification of this formula here, but we’ll put it to use to calculate probabilities in the binomial distribution. For example, the number of flip sequences of length 3 with exactly 2 heads is:
\begin{gather*}
{3 \choose 2} = \frac{3!}{2!1!} = \frac{6}{2} = 3.
\end{gather*}
Since we previously said the probability of such a sequence is 0.096, we can now find the probability of seeing exactly 2 heads in 3 flips of the coin:
\begin{align*}
\Pr(S = 2) \amp = (\text{number of sequences})(\text{probability of each sequence}) \\
\amp = (3)(0.096) \\
\amp = 0.288,
\end{align*}
which matches the value in Table 2.1.8. More generally:
Fact 2.1.12. Binomial Probability Formula.
\begin{gather*}
b(k; n, p) = {n \choose k} p^k (1-p)^{n-k}
\end{gather*}
Subsection 2.1.3 Geometric Distribution
Definition 2.1.13.
Suppose an event occurs with probability \(p\text{.}\) If we perform independent trials repeatedly, let \(N\) be the random variable which counts the number of trials performed until we see the event occur for the first time. (\(N\) counts the final trial in which the event occurs.) Then \(N\) has the geometric distribution with parameter \(p\text{.}\) We’ll write \(N \sim \Geom(p)\) to denote this. For each value \(k \geq 1\text{,}\) we’ll write \(g(k)\) for \(\Pr(N = k)\text{.}\) (If we want to keep track of the parameter value, we may write \(g(k; p)\text{.}\))
Suppose we flip a coin repeatedly until we first see heads. The number \(N\) of flips is geometrically distributed with parameter \(p\text{,}\) which gives the probability of a flip coming up heads. Then the probability of tails is \(1 - p\text{.}\) For any particular \(k \geq 1\text{,}\) in order to have \(N = k\text{,}\) we must see exactly the flip sequence \(TT\dotsm TH\text{,}\) where the number of tails is exactly \(k - 1\text{.}\) The probability of seeing this particular sequence is then:
Fact 2.1.14. Geometric Probability Formula.
\begin{gather*}
g(k; p) = (1-p)^{k-1} p
\end{gather*}
Subsection 2.1.4 Poisson Distribution
Definition 2.1.15.
A Poisson process is one in which some event occurs randomly at a constant probabilistic rate \(\lambda\) over time. Suppose we observe a Poisson process for a fixed amount of time \(t\text{,}\) and let \(N\) count the number of occurrences of the event. Then \(N\) has the Poisson distribution with parameters \(\lambda\) and \(t\text{.}\) We’ll write \(N \sim \Poiss(\lambda, t)\) to denote this. For each value \(k \geq 0\text{,}\) we’ll write \(p(k)\) for \(\Pr(N = k)\text{.}\) (If we want to keep track of the parameter values, we may write \(p(k; \lambda, t)\text{.}\) See Example 2.1.17 for an important detail about units.)
Fact 2.1.16. Poisson Probability Formula.
\begin{gather*}
p(k; \lambda, t) = \frac{(\lambda t)^k}{k!} e^{-\lambda t}.
\end{gather*}
The term \(\lambda t\) is potentially misleading here depending on the units given for each value. The idea is that the rate information might be given for a different period of time than the observation period, so the rate should be scaled before calculating a probability. But the rate and observation time must be using the same time unit before multiplying. See the following example.
Example 2.1.17.
Suppose we observe traffic along a particular stretch of highway which typically has 200 cars pass per hour. Let \(N\) be the number of cars seen in a 2-hour observation period. Since the parameter \(\lambda = 200\) cars per hour is given for a 1-hour period, we should scale it to find the rate for a 2-hour period: \(\lambda t = (200)(2) = 400\) cars per 2-hours. Then, for example:
\begin{gather*}
\Pr(N = 375) = p(375; 200, 2) = \frac{400^{375}}{375!} e^{-400} \approx 0.01.
\end{gather*}
Now consider the random variable \(M\) counting the number of cars seen in a 30-minute observation period. We should not calculate \(\lambda t = (200)(30) = 6000\text{,}\) since \(\lambda\) measures time in hours and \(t\) measures time in minutes. We should first convert \(t = 0.5\) hours, then we can find our appropriately scaled rate information: \((200)(0.5) = 100\) cars per half-hour. Then, for example:
\begin{gather*}
\Pr(M = 110) = p(110; 200, 0.5) = \frac{100^{110}}{110!}e^{-100} \approx 0.02.
\end{gather*}
Note that when we write the notation \(p(k; \lambda, t)\text{,}\) we assume that \(\lambda\) and \(t\) are expressed with the same time units already.
Exercises 2.1.5 Exercises
1.
An experiment consists of flipping a biased coin 20 times. If the coin comes up heads with probability \(p = 0.3\text{,}\) find the probability of seeing 5 heads. Find the probability of seeing up to (and including) 3 heads.
Solution.
Let \(S\) be the number of heads. Then \(S \sim \Bin(20, 0.3)\text{,}\) so:
\begin{align*}
\Pr(S = 5) \amp = {20 \choose 5} (0.3)^5 (0.7)^{20 - 5} \\
\amp = \frac{20!}{(5!)(15!)} (0.3)^5 (0.7)^{15} \\
\amp = \frac{20 \times 19 \times 18 \times 17 \times 16}{5 \times 4 \times 3 \times 2 \times 1} (0.3)^5 (0.7)^{15} \\
\amp = (19 \times 3 \times 17 \times 16) (0.3)^5 (0.7)^{15} \\
\amp \approx 0.179
\end{align*}
2.
An experiment consists of flipping a coin repeatedly until we first see heads.
(a)
If the coin comes up heads with probability 0.4, what is the probability we’ll see our first heads within three flips? What about precisely on the third flip?
Solution.
Let \(T\) be the number of flips until we see heads. Then \(T\) is geometric with parameter \(p = 0.4\text{,}\) so:
\begin{align*}
\Pr(T = k) \amp = (1-0.4)^{k-1}(0.4) = 0.6^{k-1} \cdot 0.4
\end{align*}
(b)
Which flip has the highest chance of being the first flip to come up heads?
Solution.
\(\Pr(T = k) = 0.6^{k-1} \cdot 0.4\text{,}\) so every additional flip multiplies the probability by 0.6. Therefore, the highest value for \(\Pr(T = k)\) occurs when \(k = 1\text{,}\) in which case \(\Pr(T = 1) = 0.4.\)
3.
A particular store has an average of 20 customers each hour. During a 4-hour afternoon shift, what is the probability of serving 80 customers.
Solution.
Let \(N\) be the number of customers seen during the afternoon shift. Since the store averages 20 customers per hour, it will average 80 per 4-hours. So \(N \sim \Poiss(20, 4)\) will have distribution \(\Pr(N = k) = \frac{80^{k}}{k!} e^{-80}\text{.}\) Therefore:
\begin{gather*}
\Pr(N = 80) = \frac{80^{80}}{80!} e^{-80} \approx 0.045.
\end{gather*}
