diff --git a/source/ch-Probability.ptx b/source/ch-Probability.ptx index ee7ec30..c05e756 100644 --- a/source/ch-Probability.ptx +++ b/source/ch-Probability.ptx @@ -306,7 +306,7 @@ - +

Let \Omega = \{x_1, x_2, \dotsc, x_n\} be a sample space and A\subset \Omega an event. diff --git a/source/ch-Random-Variables.ptx b/source/ch-Random-Variables.ptx index 3245914..faae2f6 100644 --- a/source/ch-Random-Variables.ptx +++ b/source/ch-Random-Variables.ptx @@ -12,80 +12,335 @@

Discrete Random Variables - - -

- Let \Omega be a sample space. - A random variable is a function X \colon \Omega \to \mathbb{R} which assigns a number to each outcome. -

- - + + Discrete Random Variables - - -

- Suppose an experiment consists of flipping a fair coin 4 times. - Let X be the number of heads. - The actual outcomes in the experiment are heads/tails sequences of length 4, such as HTHT and HHHT. - The random variable X assigns a numerical measurement to the outcomes, such as X(HTHT) = 2 and X(HHHT) = 3. -

+ + +

+ Let \Omega be a sample space. + A random variable is a function X \colon \Omega \to \R which assigns a number to each outcome. +

+ + -

- For any real number value a, we can consider the event consisting of outcomes such that X = a. - For example, the flip sequences HHHT, HHTH, HTHH, and THHH all have 3 heads, so \Pr(X = 3) = \frac{4}{16} = \frac{1}{4}. -

- - + + +

+ Suppose an experiment consists of flipping a fair coin 4 times. + Let X be the number of heads. + The actual outcomes in the experiment are heads/tails sequences of length 4, such as HTHT and HHHT. + The random variable X assigns a numerical measurement to the outcomes, such as X(HTHT) = 2 and X(HHHT) = 3. +

- - Indicator Random Variable +

+ For any real number value a, we can consider the event consisting of outcomes such that X = a. + For example, the flip sequences HHHT, HHTH, HTHH, and THHH all have 3 heads, so \Pr(X = 3) = \frac{4}{16} = \frac{1}{4}. +

+ + - -

- Let A be an event in the sample space \Omega. - An indicator random variable for A is the random variable X such that - - X(\omega) = \begin{cases} 1 \amp \omega \in A \\ 0 \amp \omega \notin A \end{cases} - -

- - + + Indicator Random Variable -

- Indicator random variables are also called Bernoulli random variables, although we'll prefer the former term since it more clearly states the purpose of these random variables: to indicate whether or not a particular event has occurred. - We'll use the language "X indicates A" to mean that X is an indicator random variable for the event A. - In this case, \Pr(X = 1) = \Pr(A), and \Pr(X = 0) = 1 - \Pr(A). -

+ +

+ Let A be an event in the sample space \Omega. + An indicator random variable for A is the random variable X such that + + X(\omega) = \begin{cases} 1 \amp \omega \in A \\ 0 \amp \omega \notin A \end{cases} + +

+ + - - -

- Consider the set S of all real numbers which are actually output by a random variable X. - If S does not contain any interval of values, then the random variable X is called discrete. - Otherwise, it's called continuous. -

- - +

+ Indicator random variables are also called Bernoulli random variables, although we'll prefer the former term since it more clearly states the purpose of these random variables: to indicate whether or not a particular event has occurred. + We'll use the language "X indicates A" to mean that X is an indicator random variable for the event A. + In this case, \Pr(X = 1) = \Pr(A), and \Pr(X = 0) = 1 - \Pr(A). +

- - -

- Any random variable X defined on a finite sample space \Omega is discretethe set of outputs of X cannot contain an interval if it only has finitely many values. -

+ + +

+ Consider the set S of all real numbers which are actually output by a random variable X. + If S does not contain any interval of values, then the random variable X is called discrete. + Otherwise, it's called continuous. +

+ + -

- Suppose an experiment consists of growing a plant in a new fertilizer, and a random variable Y measures the height of the plant after a set growing period. - Now Y could conceivably take on an interval's worth of values (such as, e.g., any real number between 10 inches and 20 inches), so this random variable would be continuous. -

+ + +

+ Any random variable X defined on a finite sample space \Omega is discretethe set of outputs of X cannot contain an interval if it only has finitely many values. +

-

- The distinction between discrete and continuous random variables is not simply the distinction of whether the sample space is finite or infinite. - Consider the experiment in which we flip a coin repeatedly until we first see a coin come up heads. - Let Z be the number of times the coin is flipped. Then there are infinitely many possible values of Z (1, 2, 3, and so on), but there's no interval of real numbers which are all possible outputs of Z. - The possible outputs of Z are discrete (in the non-technical, English sense of the word: separated). -

- - +

+ Suppose an experiment consists of growing a plant in a new fertilizer, and a random variable Y measures the height of the plant after a set growing period. + Now Y could conceivably take on an interval's worth of values (such as, e.g., any real number between 10 inches and 20 inches), so this random variable would be continuous. +

+ +

+ The distinction between discrete and continuous random variables is not simply the distinction of whether the sample space is finite or infinite. + Consider the experiment in which we flip a coin repeatedly until we first see a coin come up heads. + Let Z be the number of times the coin is flipped. + Then there are infinitely many possible values of Z (1, 2, 3, and so on), but there's no interval of real numbers which are all possible outputs of Z. + The possible outputs of Z are discrete (in the non-technical, English sense of the word: separated). +

+ + + +

+ There are several particular types of distributions that appear repeatedly, since they model common behaviors. + Indicator random variables, for example, will appear any time a random process generates a yes/no or success/failure type of answer to a question. + Below are some other common distributions. +

+ + + + + The Binomial Distribution + + + +

+ Suppose an event occurs with probability p. + If we perform n independent trials, let S be the random variable which counts the number of trials in which the event occurred. + Then S has the binomial distribution with parameters n and p. + We'll write S \sim \Bin(n, p) to denote this. + For each value 0 \leq k \leq n, we'll write b(k) for \Pr(S = k). + (If we want to keep track of the parameter values, we may write b(k; n, p).) +

+ + + + + +

+ Suppose we have a coin which comes up heads with probability p = 0.4. + If we flip the coin three times and let S be the number of heads, then S is binomially distributed with parameters n = 3 and p = 0.4. + Then S has the distribution below. +

+ + + <m>\Bin(3, 0.4)</m> + + + + k + \Pr(S = k) + + + + 0 + 0.216 + + + + 1 + 0.432 + + + + 2 + 0.288 + + + + 3 + 0.064 + + +
+ + + +

+ To understand how these probabilities are calculated, consider a particular heads/tails sequence, such as HHHT. + If the probability of heads is 0.4, then the probability of tails is 0.6. + Since the results of different coin flips are independent from each other, the probability of seeing precisely the sequence HHT would be the product (0.4)(0.4)(0.6) = 0.096. + The probability of seeing precisely HTH would be (0.4)(0.6)(0.4) = 0.096, the same product of terms in a different order. + So, if we knew exactly the number of flip sequences of length 3 with exactly 2 heads, we could calculate the probability of seeing 2 heads as: + + \Pr(S = 2) = (\text{number of sequences})(\text{probability of each sequence}) + + The probability for each sequence is straightforward to generalize: if there are n flips and k of them are heads, then n - k of them are tails. + So the product we should calculate will have k copies of the parameter p and n - k copies of the complementary probability 1 - p: + + \text{probability of each sequence } = p^k(1-p)^k + +

+ + + +

+ We'll write the symbol {n \choose k} to mean the number of ways to choose k things out of n things. + The number {n \choose k} is read "n choose k" and called a binomial coefficient. +

+ + + +

+ The term "binomial coefficient" is used here because these numbers form coefficients of a polynomial in an important theorem about binomials called the Binomial Theorem. + The binomial coefficients will precisely tell us the number of flip sequences of length n with precisely k heads (to build such a sequence, we must choose k out of n flips in the sequence to be heads; the rest must be tails). + So, now we need a way to find the numbers {n \choose k}. +

+ + + +

+ Let n be a positive integer. + Then the factorial of n, written n!, is the product of the positive integers up to n, i.e.: + + n! = 1 \times 2 \times 3 \times \dotsb \times (n-1) \times n. + +

+ + + + + Binomial Coefficient Formula + + +

+ For any 0 \leq k \leq n, + + {n \choose k} = \frac{n!}{k!(n-k)!} + +

+ + + +

+ We won't provide a justification of this formula here, but we'll put it to use to calculate probabilities in the binomial distribution. + For example, the number of flip sequences of length 3 with exactly 2 heads is: + + {3 \choose 2} = \frac{3!}{2!1!} = \frac{6}{2} = 3. + + Since we previously said the probability of such a sequence is 0.096, we can now find the probability of seeing exactly 2 heads in 3 flips of the coin: + + \Pr(S = 2) \amp = (\text{number of sequences})(\text{probability of each sequence}) + \amp = (3)(0.096) + \amp = 0.288, + + which matches the value in . + More generally: +

+ + + Binomial Probability Formula + + +

+ + b(k; n, p) = {n \choose k} p^k (1-p)^{n-k} + +

+ + + + + + + Geometric Distribution + + + +

+ Suppose an event occurs with probability p. + If we perform independent trials repeatedly, let N be the random variable which counts the number of trials performed until we see the event occur for the first time. + (N counts the final trial in which the event occurs.) Then N has the geometric distribution with parameter p. + We'll write N \sim \Geom(p) to denote this. + For each value k \geq 1, we'll write g(k) for \Pr(N = k). + (If we want to keep track of the parameter value, we may write g(k; p).) +

+ + + +

+ Suppose we flip a coin repeatedly until we first see heads. + The number N of flips is geometrically distributed with parameter p, which gives the probability of a flip coming up heads. + Then the probability of tails is 1 - p. + For any particular k \geq 1, in order to have N = k, we must see exactly the flip sequence TT\dotsm TH, where the number of tails is exactly k - 1. + The probability of seeing this particular sequence is then: +

+ + + Geometric Probability Formula + + +

+ + g(k; p) = (1-p)^{k-1} p + +

+ + + + + + + Poisson Distribution + + + +

+ A Poisson process is one in which some event occurs randomly at a constant probabilistic rate \lambda over time. + Suppose we observe a Poisson process for a fixed amount of time t, and let N count the number of occurrences of the event. + Then N has the Poisson distribution with parameters \lambda and t. + We'll write N \sim \Poiss(\lambda, t) to denote this. + For each value k \geq 0, we'll write p(k) for \Pr(N = k). + (If we want to keep track of the parameter values, we may write p(k; \lambda, t). + See for an important detail about units.) +

+ + + + + Poisson Probability Formula + + +

+ + p(k; \lambda, t) = \frac{(\lambda t)^k}{k!} e^{-\lambda t}. + +

+ + + +

+ The term \lambda t is potentially misleading here depending on the units given for each value. + The idea is that the rate information might be given for a different period of time than the observation period, so the rate should be scaled before calculating a probability. + But the rate and observation time must be using the same time unit before multiplying. + See the following example. +

+ + + +

+ Suppose we observe traffic along a particular stretch of highway which typically has 200 cars pass per hour. + Let N be the number of cars seen in a 2-hour observation period. + Since the parameter \lambda = 200 cars per hour is given for a 1-hour period, we should scale it to find the rate for a 2-hour period: \lambda t = (200)(2) = 400 cars per 2-hours. + Then, for example: + + \Pr(N = 375) = p(375; 200, 2) = \frac{400^{375}}{375!} e^{-400} \approx 0.01. + +

+ +

+ Now consider the random variable M counting the number of cars seen in a 30-minute observation period. + We should not calculate \lambda t = (200)(30) = 6000, since \lambda measures time in hours and t measures time in minutes. + We should first convert t = 0.5 hours, then we can find our appropriately scaled rate information: (200)(0.5) = 100 cars per half-hour. + Then, for example: + + \Pr(M = 110) = p(110; 200, 0.5) = \frac{100^{110}}{110!}e^{-100} \approx 0.02. + + Note that when we write the notation p(k; \lambda, t), we assume that \lambda and t are expressed with the same time units already. +

+ + + @@ -101,7 +356,7 @@

An experiment consists of flipping a biased coin 20 times. - If the coin comes up heads with probability \theta = 0.3, find the probability of seeing 5 heads. + If the coin comes up heads with probability p = 0.3, find the probability of seeing 5 heads. Find the probability of seeing up to (and including) 3 heads.

@@ -158,7 +413,78 @@ Continuous Random Variables

- Text of section. + We have to think about probability differently for continuous random variables. + If X is a continuous random variable taking values in an interval [a, b], it doesn't quite make sense to ask, as we have for discrete random variables, to find the probabilities of each individual outcome. +

+ +

+ When we roll a fair 6-sided die, there are 6 equally likely outcomes, so they each get assigned the probability 1/6. + If we try to select a random number in the interval [0, 1] in a similarly "fair" way (i.e., with each outcome equally likely), then what probability \alpha should we assign to each outcome? If \alpha > 0, then we have a problem: if you add up the probabilities of all the (infinitely many) outcomes, you get an infinite total, whereas the total probability in any sample space should be 1. + If \alpha = 0, then we also have a problem: if you add up the probabilities of all the outcomes, you get total of 0, whereas the total probability in any sample space should be 1. +

+ +

+ We need a new way to model and calculate probabilities for continuous random variables. +

+ + + +

+ Let X be a continuous random variable taking values in the interval [a, b]. + The probability density function (or pdf) for X is a function f(x) defined on [a, b] satisfying the properties: +

    +
  1. +

    + f(x) \geq 0 for all x \in [a, b], and +

    +
    2. + +
  2. +

    + \displaystyle{\int_a^b f(x)\ dx = 1}. +

    +
    3. +
+

+ + + +

+ It's important to keep in mind that the output values of a pdf f(x) are probability densities, not probabilities. + You can (somewhat casually) think of the units of f(x) values as "probability per <units of the sample space>". + You may not have discussed the units of the differential dx very much in your Calculus 1 course, but dx has the same units as x and is being multiplied by the integrand when you take an integral. + So, although f(x) is not a probability, the result of computing \int_c^d f(x)\ dx is a probability. +

+ +

+ A pdf's purpose in life is to be integrated, which tells us probabilities for a continuous random variable: + + \Pr(c \leq X \leq d) = \int_c^d f(x)\ dx. + +

+ + + +

+ Let X take values in the interval [a, b]. + We say that X has the uniform distribution if, for any subinterval [c, d] \subset [a, b], the probability that X lies within [c, d] is equal to the proportion of the total sample space taken by [c, d]. + That is: + + \Pr(c \leq X \leq d) = \frac{d - c}{b - a}. + + In this case, X has the pdf f(x) = \frac{1}{b - a}. +

+ + + +

+ We'll ignore for the moment how to find this pdf and instead we'll quickly check that it behaves the way we want it to: + + \Pr(c \leq X \leq d) \amp = \int_c^d \frac{1}{b-a}\ dx + \amp = \frac{x}{b - a}\bigg|_c^d + \amp = \frac{d - c}{b - a}, + + as we wanted.

diff --git a/source/docinfo.ptx b/source/docinfo.ptx index f5a7698..3fe5343 100644 --- a/source/docinfo.ptx +++ b/source/docinfo.ptx @@ -19,7 +19,18 @@ - \newcommand{\N}{\mathbb N} \newcommand{\Z}{\mathbb Z} \newcommand{\Q}{\mathbb Q} \newcommand{\R}{\mathbb R} + + + \newcommand{\N}{\mathbb N} + \newcommand{\Z}{\mathbb Z} + \newcommand{\Q}{\mathbb Q} + \newcommand{\R}{\mathbb R} + + + \DeclareMathOperator{\Bin}{Bin} + \DeclareMathOperator{\Geom}{Geom} + \DeclareMathOperator{\Poiss}{Poiss} +