Let X be a discrete random variable taking the values x_1, x_2, \dotsc, x_n.
@@ -170,7 +170,68 @@
Continuous Expected Value
+ Expected value for a discrete random variable is a weighted average of the values taken by the variable with weights provided by the probability distribution.
+ We'd like to use the same idea for a continuous random variable, but now we're averaging over an interval's worth of values.
+ Also, individual values don't have probabilities, they have probability densities.
+
+
+
+
+ Let X be a continuous random variable taking values in the interval [a, b] with pdf f(x).
+ The expected value of X is given by:
+
+ \E(X) = \int_a^b x f(x)\ dx.
+
+
+
+
+
+
+ It's worth viewing this definition side-by-side with to understand that they have analogous structure.
+ The pdf values f(x) are densities, not probabilities.
+ But the product f(x)\ dx is probability.
+ So the expression x f(x)\ dx is analogous to x_i \Pr(X = x_i): value times probability.
+ Finally, the integral symbol \int is meant to look like a stretched out letter "S" because it is a form of infinite summation.
+ So, both \sum x_i \Pr(X = x_i) and \int_a^b x f(x)\ dx should be understood as taking the products of values with their probabilities and then summing up those products.
+
+
+
+
+
+ Let X be uniform on the interval [a, b].
+ So X has the pdf f(x) = \frac{1}{b - a}.
+ We might reasonably expect that the average value of X would be the midpoint of the interval.
+ Let's check that:
+
+ \E(X) \amp = \int_a^b x f(x)\ dx
+ \amp = \int_a^b \frac{x}{b - a} \ dx
+ \amp = \frac{1}{b-a} \left(\frac{x^2}{2}\right)\bigg|_a^b
+ \amp = \frac{1}{b-a} \left( \frac{b^2}{2} - \frac{a^2}{2} \right)
+ \amp = \frac{1}{b - a}\left( \frac{(b - a)(b + a)}{2}\right)
+ \amp = \frac{a + b}{2}.
+
+
+
+
+
+
+
+
+ Let X take values in [0, 4] with pdf f(x) = \frac{3\sqrt{x}}{16}.
+
+ \E(X) \amp = \int_0^4 x f(x)\ dx
+ \amp = \int_0^4 \frac{3x^{3/2}}{16}\ dx
+ \amp = \frac{3}{16}\left(\frac{x^{5/2}}{5/2}\right)\bigg|_0^4
+ \amp = \frac{3}{16}\left(\frac{2x^{5/2}}{5}\right)\bigg|_0^4
+ \amp = \frac{3}{40}\left(x^{5/2}\right)\bigg|_0^4
+ \amp = \frac{3}{40}(32 - 0)
+ \amp = \frac{12}{5}
+ \amp = 2.4.
+
+
+
+
@@ -178,7 +239,154 @@
Linearity of Expected Value
+ In the context of many areas of mathematics, the word "linearity" doesn't refer to some graph being a straight line.
+ Rather, it refers to a situation in which some mathematical object or operation behaves in the nicest possible way under the operations of addition and scalar multiplication (i.e., multiplication by a constant).
+ You've already encountered linearity several times in Calculus 1, whether or not it was phrased that way.
+
+ -
+
+ Limits are linear:
+
+ \lim f(x) + g(x) \amp = \lim f(x) + \lim g(x)
+ \lim c f(x) \amp = c \lim f(x)
+
+
+
+
+ -
+
+ Derivatives are linear:
+
+ \frac{d}{dx}(f(x) + g(x)) \amp = \frac{d}{dx}(f(x)) + \frac{d}{dx}(g(x))
+ \frac{d}{dx}(c f(x)) \amp = c \frac{d}{dx}(f(x))
+
+
+
+
+ -
+
+ Integrals (both definite and indefinite) are linear:
+
+ \int f(x) + g(x) \amp = \int f(x) + \int g(x)
+ \int c f(x) \amp = c \int f(x)
+
+
+
+
+ Now, we see that expected value is also linear:
+
+
+ Linearity of Expectation
+
+
+
+ Let X and Y be random variables and \alpha \in \R.
+ Then:
+
+ \E(X + Y) \amp = \E(X) + \E(Y)
+ \E(\alpha X) \amp = \alpha \E(X)
+
+
+
+
+
+
+ Some application of the linearity of expectation will seem entirely reasonable:
+
+
+
+
+
+ In , we saw that the expected value of a fair 6-sided die roll is 3.5.
+ In , we saw that the expected value of the sum of two die rolls is 7, which is precisely 2 times 3.5.
+ The calculation in would be very tedious to recreate for three die rolls, let alone generalizing for n rolls.
+ However, suppose we define separate random variables R_1, R_2, \dotsc, R_n for the result of each individual roll.
+ Let S be the sum of the n rolls.
+ Then S = R_1 + R_2 + \dotsb + R_n. By linearity:
+
+ \E(S) \amp = \E(R_1 + R_2 + \dotsb + R_n)
+ \amp = \E(R_1) + \E(R_2) + \dotsb + \E(R_n)
+ \amp = \underbrace{3.5 + 3.5 + \dotsb + 3.5}_{n \text{ times}}
+ \amp = 3.5n.
+
+ For two rolls, we recover our previous result: 3.5(2) = 7. But now we have a general formula that tells the expected sum for any number of rolls.
+ It also gives a perfectly reasonable answer: the expected sum of n rolls is n times the expected value of a single roll.
+
+
+
+
+
+
+
+ Suppose S \sim \Bin(n, p).
+ describes a binomially distributed random variable as counting the number of occurrences of some event, which either happens or not in each of n independent trials.
+ This leads us to a very natural idea: identify each individual possible occurrence of the event, and assign it an indicator random variable.
+ In this case, let H_i indicate that flip i comes up heads.
+ Then
+
+ S = H_1 + H_2 + \dotsb + H_n.
+
+ We already know () that the expected value of each H_i is the probability of the indicated eventthat flip i is heads.
+ This is precisely the parameter p.
+ Therefore:
+
+ \E(S) \amp = \E(H_1 + H_2 + \dotsb + H_n)
+ \amp = \E(H_1) + \E(H_2) + \dotsb + \E(H_n)
+ \amp = \underbrace{p + p + \dotsb + p}_{n \text{ times}}
+ \amp = np.
+
+ Remember this formula! We'll make frequent use of it.
+
+
+
+
+
+ The previous examples are so reasonable it may seem unimpressive.
+ But consider the following use of linearity:
+
+
+
+
+
+ We say a flip sequence has a run of 4 heads starting at k if flips k, k+1, k+2, k+3 are all heads.
+ We don't care about overlaps: the flip sequence HHHHH has a run of 4 heads starting at 1 and another run starting at 2.
+
+
+
+ Suppose we flip a fair coin 100 times.
+ How many runs of 4 heads should we expect?
+
+
+
+
+
+ Can you mimic the structure of ?
+
+
+
+
+
+ Let R_k indicate a run of 4 heads starting at k.
+ Since this requires 4 precise flips to come up heads, the probability of the indicated event should be
+
+ \left(\frac{1}{2}\right)^4 = \frac{1}{16}
+
+ The very last starting flip for a run of 4 heads is flip number 97 (so that flips 97, 98, 99, and 100 would all be heads).
+ Then S = R_1 + R_2 + \dotsb + R_{97} precisely counts the number of runs of 4 heads.
+ Using linearity:
+
+ \E(S) \amp = \E(R_1 + R_2 + \dotsb + R_{97})
+ \amp = \E(R_1) + \E(R_2) + \dotsb + \E(R_{97})
+ \amp = \underbrace{\frac{1}{16} + \frac{1}{16} + \dotsb + \frac{1}{16}}_{97 \text{ times}}
+ \amp = \frac{97}{16}
+ \amp \approx 6.
+
+ Notice that, unlike , the events being indicated here are not independent from each other.
+ Linearity is unaffected by the fact that that the runs of heads can overlap.
+
+
+