Skip to main content

Section 3.1 Expected Value

Subsection 3.1.1 Discrete Expected Value

A full probability distribution (for a discrete random variable) or a density function (for a continuous random variable) carry all of the probability information. Often, we seek statistics, i.e., numbers that summarize a distribution in some way.

Definition 3.1.1.

Let \(X\) be a discrete random variable taking the values \(x_1, x_2, \dotsc, x_n\text{.}\) The expected value (also called mean, or expectation) of \(X\) is the weighted average of the values of \(X\text{,}\) where the weights are the probabilities of \(X\) taking each value:
\begin{gather*} \E(X) = \sum_{i=1}^n x_i \Pr(X = x_i). \end{gather*}

Example 3.1.2.

If \(X\) takes on the values \(x_1, x_2, \dotsc, x_n\) uniformly (i.e., each value having probability \(\frac{1}{n}\)), then the expected value is the usual average:
\begin{align*} \E(X) \amp = x_1 \cdot \frac{1}{n} + x_2\cdot \frac{1}{n} + \dotsb + x_n \cdot \frac{1}{n} \\ \amp = \frac{x_1 + x_2 + \dotsb + x_n}{n} \end{align*}
For example, the expected value of a fair 6-sided die roll \(R\) is:
\begin{align*} \E(R) \amp = 1 \left(\frac{1}{6}\right) + 2 \left(\frac{1}{6}\right) + 3 \left(\frac{1}{6}\right) + 4 \left(\frac{1}{6}\right) + 5 \left(\frac{1}{6}\right) + 6 \left(\frac{1}{6}\right) \\ \amp = \frac{1 + 2 + \dotsb + 6}{6} = \frac{21}{6} = \frac{7}{2}. \end{align*}

Example 3.1.3.

Suppose we roll a fair 6-sided die two times, and let \(S\) be the sum of the rolls. It’s straightforward to check the distribution for \(S\) shown below. We’ve also included a column containing the products \(k \Pr(S = k)\text{,}\) which must be summed up to find \(\E(S)\text{.}\)
Table 3.1.4.
\(k\) \(\Pr(S = k)\) \(k \cdot \Pr(S = k)\)
2 \(1/36\) \(2/36\)
3 \(2/36\) \(6/36\)
4 \(3/36\) \(12/36\)
5 \(4/36\) \(20/36\)
6 \(5/36\) \(30/36\)
7 \(6/36\) \(42/36\)
8 \(5/36\) \(40/36\)
9 \(4/36\) \(36/36\)
10 \(3/36\) \(30/36\)
11 \(2/36\) \(22/36\)
12 \(1/36\) \(12/36\)
The sum of the third column gives \(\E(S) = \frac{252}{36} = 7\text{.}\)

Example 3.1.5.

Let \(X\) indicate an event \(A\) which has probability \(p\text{.}\) Then:
\begin{gather*} \E(X) = 0 \cdot (1-p) + 1 \cdot p = p. \end{gather*}
That is, the expected value of an indicator random variable is the probability of the event that it indicates.
Having found the expected value for one named distribution, we may be tempted to run through our other discrete distributions (binomial, geometric, and Poisson) and find nice formulas for their expected values. Unfortunately, both the geometric distribution and the Poisson distribution have infinitely many possible values, and the summation of infinitely many discrete terms is beyond the scope of this course. (A Calculus 2 course covering series would give the right tools to find expected value formulas with full justification for these distributions.)
The binomial distribution takes only finitely many values, and we have Fact 2.1.12 that lets us generate a distribution table reasonably quickly, given the parameters \(n\) and \(p\text{.}\) Feel free to pick some parameter values (keep \(n\) fairly small!) to try this calculation by hand, but we’ll use a powerful theorem a bit later to give a very nice general formula depending on \(n\) and \(p\text{.}\)

Subsection 3.1.2 Continuous Expected Value

Expected value for a discrete random variable is a weighted average of the values taken by the variable with weights provided by the probability distribution. We’d like to use the same idea for a continuous random variable, but now we’re averaging over an interval’s worth of values. Also, individual values don’t have probabilities, they have probability densities.

Definition 3.1.6.

Let \(X\) be a continuous random variable taking values in the interval \([a, b]\) with pdf \(f(x)\text{.}\) The expected value of \(X\) is given by:
\begin{gather*} \E(X) = \int_a^b x f(x)\ dx. \end{gather*}
It’s worth viewing this definition side-by-side with Definition 3.1.1 to understand that they have analogous structure. The pdf values \(f(x)\) are densities, not probabilities. But the product \(f(x)\ dx\) is probability. So the expression \(x f(x)\ dx\) is analogous to \(x_i \Pr(X = x_i)\text{:}\) value times probability. Finally, the integral symbol \(\int\) is meant to look like a stretched out letter "S" because it is a form of infinite summation. So, both \(\sum x_i \Pr(X = x_i)\) and \(\int_a^b x f(x)\ dx\) should be understood as taking the products of values with their probabilities and then summing up those products.

Example 3.1.7.

Let \(X\) be uniform on the interval \([a, b]\text{.}\) So \(X\) has the pdf \(f(x) = \frac{1}{b - a}\text{.}\) We might reasonably expect that the average value of \(X\) would be the midpoint of the interval. Let’s check that:
\begin{align*} \E(X) \amp = \int_a^b x f(x)\ dx \\ \amp = \int_a^b \frac{x}{b - a} \ dx \\ \amp = \frac{1}{b-a} \left(\frac{x^2}{2}\right)\bigg|_a^b \\ \amp = \frac{1}{b-a} \left( \frac{b^2}{2} - \frac{a^2}{2} \right) \\ \amp = \frac{1}{b - a}\left( \frac{(b - a)(b + a)}{2}\right) \\ \amp = \frac{a + b}{2}. \end{align*}

Example 3.1.8.

Let \(X\) take values in \([0, 4]\) with pdf \(f(x) = \frac{3\sqrt{x}}{16}\text{.}\)
\begin{align*} \E(X) \amp = \int_0^4 x f(x)\ dx \\ \amp = \int_0^4 \frac{3x^{3/2}}{16}\ dx \\ \amp = \frac{3}{16}\left(\frac{x^{5/2}}{5/2}\right)\bigg|_0^4\\ \amp = \frac{3}{16}\left(\frac{2x^{5/2}}{5}\right)\bigg|_0^4\\ \amp = \frac{3}{40}\left(x^{5/2}\right)\bigg|_0^4\\ \amp = \frac{3}{40}(32 - 0) \\ \amp = \frac{12}{5} \\ \amp = 2.4. \end{align*}

Subsection 3.1.3 Linearity of Expected Value

In the context of many areas of mathematics, the word "linearity" doesn’t refer to some graph being a straight line. Rather, it refers to a situation in which some mathematical object or operation behaves in the nicest possible way under the operations of addition and scalar multiplication (i.e., multiplication by a constant). You’ve already encountered linearity several times in Calculus 1, whether or not it was phrased that way.
  • Limits are linear:
    \begin{align*} \lim f(x) + g(x) \amp = \lim f(x) + \lim g(x)\\ \lim c f(x) \amp = c \lim f(x) \end{align*}
  • Derivatives are linear:
    \begin{align*} \frac{d}{dx}(f(x) + g(x)) \amp = \frac{d}{dx}(f(x)) + \frac{d}{dx}(g(x))\\ \frac{d}{dx}(c f(x)) \amp = c \frac{d}{dx}(f(x)) \end{align*}
  • Integrals (both definite and indefinite) are linear:
    \begin{align*} \int f(x) + g(x) \amp = \int f(x) + \int g(x)\\ \int c f(x) \amp = c \int f(x) \end{align*}
Now, we see that expected value is also linear:
Some application of the linearity of expectation will seem entirely reasonable:

Example 3.1.10.

In Example 3.1.2, we saw that the expected value of a fair 6-sided die roll is 3.5. In Example 3.1.3, we saw that the expected value of the sum of two die rolls is 7, which is precisely 2 times 3.5. The calculation in Example 3.1.3 would be very tedious to recreate for three die rolls, let alone generalizing for \(n\) rolls. However, suppose we define separate random variables \(R_1, R_2, \dotsc, R_n\) for the result of each individual roll. Let \(S\) be the sum of the \(n\) rolls. Then \(S = R_1 + R_2 + \dotsb + R_n.\) By linearity:
\begin{align*} \E(S) \amp = \E(R_1 + R_2 + \dotsb + R_n) \\ \amp = \E(R_1) + \E(R_2) + \dotsb + \E(R_n) \\ \amp = \underbrace{3.5 + 3.5 + \dotsb + 3.5}_{n \text{ times}} \\ \amp = 3.5n. \end{align*}
For two rolls, we recover our previous result: \(3.5(2) = 7.\) But now we have a general formula that tells the expected sum for any number of rolls. It also gives a perfectly reasonable answer: the expected sum of \(n\) rolls is \(n\) times the expected value of a single roll.

Example 3.1.11.

Suppose \(S \sim \Bin(n, p)\text{.}\) Definition 2.1.6 describes a binomially distributed random variable as counting the number of occurrences of some event, which either happens or not in each of \(n\) independent trials. This leads us to a very natural idea: identify each individual possible occurrence of the event, and assign it an indicator random variable. In this case, let \(H_i\) indicate that flip \(i\) comes up heads. Then
\begin{gather*} S = H_1 + H_2 + \dotsb + H_n. \end{gather*}
We already know (Example 3.1.5) that the expected value of each \(H_i\) is the probability of the indicated event—that flip \(i\) is heads. This is precisely the parameter \(p\text{.}\) Therefore:
\begin{align*} \E(S) \amp = \E(H_1 + H_2 + \dotsb + H_n) \\ \amp = \E(H_1) + \E(H_2) + \dotsb + \E(H_n) \\ \amp = \underbrace{p + p + \dotsb + p}_{n \text{ times}} \\ \amp = np. \end{align*}
Remember this formula! We’ll make frequent use of it.
The previous examples are so reasonable it may seem unimpressive. But consider the following use of linearity:

Example 3.1.12.

We say a flip sequence has a run of 4 heads starting at \(k\) if flips \(k, k+1, k+2, k+3\) are all heads. We don’t care about overlaps: the flip sequence \(HHHHH\) has a run of 4 heads starting at 1 and another run starting at 2.
Suppose we flip a fair coin 100 times. How many runs of 4 heads should we expect?
Hint.
Can you mimic the structure of Example 3.1.11?
Solution.
Let \(R_k\) indicate a run of 4 heads starting at \(k\text{.}\) Since this requires 4 precise flips to come up heads, the probability of the indicated event should be
\begin{gather*} \left(\frac{1}{2}\right)^4 = \frac{1}{16} \end{gather*}
The very last starting flip for a run of 4 heads is flip number 97 (so that flips 97, 98, 99, and 100 would all be heads). Then \(S = R_1 + R_2 + \dotsb + R_{97}\) precisely counts the number of runs of 4 heads. Using linearity:
\begin{align*} \E(S) \amp = \E(R_1 + R_2 + \dotsb + R_{97}) \\ \amp = \E(R_1) + \E(R_2) + \dotsb + \E(R_{97}) \\ \amp = \underbrace{\frac{1}{16} + \frac{1}{16} + \dotsb + \frac{1}{16}}_{97 \text{ times}} \\ \amp = \frac{97}{16} \\ \amp \approx 6. \end{align*}
Notice that, unlike Example 3.1.11, the events being indicated here are not independent from each other. Linearity is unaffected by the fact that that the runs of heads can overlap.

Exercises 3.1.4 Exercises

1.

Consider a random variable \(X\) with probability distribution below. Find \(\E(X)\text{.}\)
Table 3.1.13.
\(x\) \(\Pr(X = x)\)
1 0.1
2 0.05
3 0.2
4 0.15
5 0.15
6 0.1
7 0.05
8 0.1
9 0.1
Answer.
4.8.

2.

Let \(X\) and \(Y\) be random variables each taking the values 1, 2, 3, 4, 5. Write different distribution tables for \(X\) and \(Y\) so that they have the same expected value.
Answer.
Table 3.1.14. Example Distributions
\(a\) \(\Pr(X = a)\) \(\Pr(Y = a)\)
1 0.2 0.1
2 0.2 0.1
3 0.2 0.6
4 0.2 0.1
5 0.2 0.1

3.

Suppose we flip a coin \(n = 100\) times, and let \(N\) count the number of heads.

(a)

If the coin comes up heads on a flip with probability \(p = 0.4\text{,}\) what is \(\E(N)\text{?}\)
Answer.
40.

4.

If \(\E(X) = 3\text{,}\) \(\E(Y) = -2\text{,}\) and \(\E(Z) = 1\text{,}\) what is \(\E(4X + 5Y - Z + 3)\text{?}\)
Answer.
4.

5.

A continuous random variable \(X\) taking values in \([0, 1]\) has p.d.f. \(f(x) = 2x\text{.}\) What is \(\E(X)\text{?}\)
Answer.
2/3.

6.

A continuous random variable \(X\) taking values in \([-1, 1]\) has p.d.f. \(f(x) = \frac{3x^2}{2}\text{.}\) What is \(\E(X)\text{?}\)
Answer.
0.

7.

A continuous random variable \(X\) taking values in \([1, 4]\) has p.d.f. \(f(x) = \frac{4}{3x^2}\text{.}\) What is \(\E(X)\text{?}\)
Answer.
\(\frac{4}{3}\ln(4) \approx 1.85.\)

8.

A continuous random variable \(X\) taking values in \([1, 4]\) has p.d.f. \(f(x) = k(x - \sqrt{x})\) for some constant \(k\text{.}\) In a previous problem, you found the value of \(k\text{.}\) Now, find \(\E(X)\text{.}\)
Answer.
\(E(X) \approx 3.04.\)

9.

A continuous random variable \(X\) taking values in \([1, 2]\) has p.d.f. \(\displaystyle{f(x) = \frac{1}{2}\left(\frac{1}{x^2} + x\right)}\text{.}\) Find \(\E(X)\text{.}\)
Answer.
\(\frac{1}{2}\left( \ln(2) + \frac{7}{3}\right) \approx 1.513.\)
PrevTopNext