431 lines
18 KiB
Plaintext
431 lines
18 KiB
Plaintext
<section xml:id="sec-Discrete-RVs" xmlns:xi="http://www.w3.org/2001/XInclude">
|
|
<title>Discrete Random Variables</title>
|
|
|
|
<subsection xml:id="subsec-Discrete-RVs">
|
|
<title>Discrete Random Variables</title>
|
|
|
|
<definition xml:id="def-RV">
|
|
<statement>
|
|
<p>
|
|
Let <m>\Omega</m> be a sample space.
|
|
A <term>random variable</term> is a function <m>X \colon \Omega \to \R</m> which assigns a number to each outcome.
|
|
</p>
|
|
</statement>
|
|
</definition>
|
|
|
|
<example>
|
|
<statement>
|
|
<p>
|
|
Suppose an experiment consists of flipping a fair coin 4 times.
|
|
Let <m>X</m> be the number of heads.
|
|
The actual outcomes in the experiment are heads/tails sequences of length 4, such as <m>HTHT</m> and <m>HHHT</m>.
|
|
The random variable <m>X</m> assigns a numerical measurement to the outcomes, such as <m>X(HTHT) = 2</m> and <m>X(HHHT) = 3</m>.
|
|
</p>
|
|
|
|
<p>
|
|
For any real number value <m>a</m>, we can consider the event consisting of outcomes such that <m>X = a</m>.
|
|
For example, the flip sequences <m>HHHT, HHTH, HTHH</m>, and <m>THHH</m> all have 3 heads, so <m>\Pr(X = 3) = \frac{4}{16} = \frac{1}{4}</m>.
|
|
</p>
|
|
</statement>
|
|
</example>
|
|
|
|
<example xml:id="example-indicator-distribution">
|
|
<title>Indicator Random Variable</title>
|
|
|
|
<statement>
|
|
<p>
|
|
Let <m>A</m> be an event in the sample space <m>\Omega</m>.
|
|
An <term>indicator random variable</term> for <m>A</m> is the random variable <m>X</m> such that
|
|
<md>
|
|
<mrow> X(\omega) = \begin{cases} 1 \amp \omega \in A \\ 0 \amp \omega \notin A \end{cases} </mrow>
|
|
</md>
|
|
</p>
|
|
</statement>
|
|
</example>
|
|
|
|
<p>
|
|
Indicator random variables are also called <term>Bernoulli random variables</term>, although we'll prefer the former term since it more clearly states the purpose of these random variables: to indicate whether or not a particular event has occurred.
|
|
We'll use the language "<m>X</m> indicates <m>A</m>" to mean that <m>X</m> is an indicator random variable for the event <m>A</m>.
|
|
In this case, <m>\Pr(X = 1) = \Pr(A)</m>, and <m>\Pr(X = 0) = 1 - \Pr(A)</m>.
|
|
</p>
|
|
|
|
<definition xml:id="def-discrete-and-continuous">
|
|
<statement>
|
|
<p>
|
|
Consider the set <m>S</m> of all real numbers which are actually output by a random variable <m>X</m>.
|
|
If <m>S</m> does not contain any interval of values, then the random variable <m>X</m> is called <term>discrete</term>.
|
|
Otherwise, it's called <term>continuous</term>.
|
|
</p>
|
|
</statement>
|
|
</definition>
|
|
|
|
<example>
|
|
<statement>
|
|
<p>
|
|
Any random variable <m>X</m> defined on a finite sample space <m>\Omega</m> is discrete<mdash/>the set of outputs of <m>X</m> cannot contain an interval if it only has finitely many values.
|
|
</p>
|
|
|
|
<p>
|
|
Suppose an experiment consists of growing a plant in a new fertilizer, and a random variable <m>Y</m> measures the height of the plant after a set growing period.
|
|
Now <m>Y</m> could conceivably take on an interval's worth of values (such as, e.g., any real number between 10 inches and 20 inches), so this random variable would be continuous.
|
|
</p>
|
|
|
|
<p>
|
|
The distinction between discrete and continuous random variables is not simply the distinction of whether the sample space is finite or infinite.
|
|
Consider the experiment in which we flip a coin repeatedly until we first see a coin come up heads.
|
|
Let <m>Z</m> be the number of times the coin is flipped.
|
|
Then there are infinitely many possible values of <m>Z</m> (1, 2, 3, and so on), but there's no interval of real numbers which are all possible outputs of <m>Z</m>.
|
|
The possible outputs of <m>Z</m> are discrete (in the non-technical, English sense of the word: separated).
|
|
</p>
|
|
</statement>
|
|
</example>
|
|
|
|
<p>
|
|
There are several particular types of distributions that appear repeatedly, since they model common behaviors.
|
|
Indicator random variables, for example, will appear any time a random process generates a yes/no or success/failure type of answer to a question.
|
|
Below are some other common distributions.
|
|
</p>
|
|
</subsection>
|
|
|
|
|
|
<subsection xml:id="subsec-binomial-distribution">
|
|
<title>The Binomial Distribution</title>
|
|
|
|
<definition xml:id="def-binomial-distribution">
|
|
<statement>
|
|
<p>
|
|
Suppose an event occurs with probability <m>p</m>.
|
|
If we perform <m>n</m> independent trials, let <m>S</m> be the random variable which counts the number of trials in which the event occurred.
|
|
Then <m>S</m> has the <term>binomial distribution</term> with parameters <m>n</m> and <m>p</m>.
|
|
We'll write <m>S \sim \Bin(n, p)</m> to denote this.
|
|
For each value <m>0 \leq k \leq n</m>, we'll write <m>b(k)</m> for <m>\Pr(S = k)</m>.
|
|
(If we want to keep track of the parameter values, we may write <m>b(k; n, p)</m>.)
|
|
</p>
|
|
</statement>
|
|
</definition>
|
|
|
|
<example>
|
|
<statement>
|
|
<p>
|
|
Suppose we have a coin which comes up heads with probability <m>p = 0.4</m>.
|
|
If we flip the coin three times and let <m>S</m> be the number of heads, then <m>S</m> is binomially distributed with parameters <m>n = 3</m> and <m>p = 0.4</m>.
|
|
Then <m>S</m> has the distribution below.
|
|
</p>
|
|
|
|
<table xml:id="table-binomial-example">
|
|
<title><m>\Bin(3, 0.4)</m></title>
|
|
|
|
<tabular halign="center">
|
|
<row header="yes" bottom="minor">
|
|
<cell><m>k</m></cell>
|
|
<cell><m>\Pr(S = k)</m></cell>
|
|
</row>
|
|
|
|
<row>
|
|
<cell>0</cell>
|
|
<cell>0.216</cell>
|
|
</row>
|
|
|
|
<row>
|
|
<cell>1</cell>
|
|
<cell>0.432</cell>
|
|
</row>
|
|
|
|
<row>
|
|
<cell>2</cell>
|
|
<cell>0.288</cell>
|
|
</row>
|
|
|
|
<row>
|
|
<cell>3</cell>
|
|
<cell>0.064</cell>
|
|
</row>
|
|
</tabular>
|
|
</table>
|
|
</statement>
|
|
</example>
|
|
|
|
<p>
|
|
To understand how these probabilities are calculated, consider a particular heads/tails sequence, such as <m>HHHT</m>.
|
|
If the probability of heads is 0.4, then the probability of tails is 0.6.
|
|
Since the results of different coin flips are independent from each other, the probability of seeing precisely the sequence <m>HHT</m> would be the product <m>(0.4)(0.4)(0.6) = 0.096</m>.
|
|
The probability of seeing precisely <m>HTH</m> would be <m>(0.4)(0.6)(0.4) = 0.096</m>, the same product of terms in a different order.
|
|
So, if we knew exactly the number of flip sequences of length 3 with exactly 2 heads, we could calculate the probability of seeing 2 heads as:
|
|
<md>
|
|
<mrow> \Pr(S = 2) = (\text{number of sequences})(\text{probability of each sequence}) </mrow>
|
|
</md>
|
|
The probability for each sequence is straightforward to generalize: if there are <m>n</m> flips and <m>k</m> of them are heads, then <m>n - k</m> of them are tails.
|
|
So the product we should calculate will have <m>k</m> copies of the parameter <m>p</m> and <m>n - k</m> copies of the complementary probability <m>1 - p</m>:
|
|
<md>
|
|
<mrow>\text{probability of each sequence } = p^k(1-p)^k</mrow>
|
|
</md>
|
|
</p>
|
|
|
|
<definition xml:id="def-binomial-coefficient">
|
|
<statement>
|
|
<p>
|
|
We'll write the symbol <m>{n \choose k}</m> to mean the number of ways to choose <m>k</m> things out of <m>n</m> things.
|
|
The number <m>{n \choose k}</m> is read "<m>n</m> choose <m>k</m>" and called a <term>binomial coefficient</term>.
|
|
</p>
|
|
</statement>
|
|
</definition>
|
|
|
|
<p>
|
|
The term "binomial coefficient" is used here because these numbers form coefficients of a polynomial in an important theorem about binomials called the <url href="https://en.wikipedia.org/wiki/Binomial_theorem">Binomial Theorem</url>.
|
|
The binomial coefficients will precisely tell us the number of flip sequences of length <m>n</m> with precisely <m>k</m> heads (to build such a sequence, we must choose <m>k</m> out of <m>n</m> flips in the sequence to be heads; the rest must be tails).
|
|
So, now we need a way to find the numbers <m>{n \choose k}</m>.
|
|
</p>
|
|
|
|
<definition xml:id="def-factorial">
|
|
<statement>
|
|
<p>
|
|
Let <m>n</m> be a positive integer.
|
|
Then the <term>factorial</term> of <m>n</m>, written <m>n!</m>, is the product of the positive integers up to <m>n</m>, i.e.:
|
|
<md>
|
|
<mrow> n! = 1 \times 2 \times 3 \times \dotsb \times (n-1) \times n. </mrow>
|
|
</md>
|
|
</p>
|
|
</statement>
|
|
</definition>
|
|
|
|
<fact xml:id="fact-binomial-coefficient-formula">
|
|
<title>Binomial Coefficient Formula</title>
|
|
|
|
<statement>
|
|
<p>
|
|
For any <m>0 \leq k \leq n</m>,
|
|
<md>
|
|
<mrow>{n \choose k} = \frac{n!}{k!(n-k)!}</mrow>
|
|
</md>
|
|
</p>
|
|
</statement>
|
|
</fact>
|
|
|
|
<p>
|
|
We won't provide a justification of this formula here, but we'll put it to use to calculate probabilities in the binomial distribution.
|
|
For example, the number of flip sequences of length 3 with exactly 2 heads is:
|
|
<md>
|
|
<mrow> {3 \choose 2} = \frac{3!}{2!1!} = \frac{6}{2} = 3. </mrow>
|
|
</md>
|
|
Since we previously said the probability of such a sequence is 0.096, we can now find the probability of seeing exactly 2 heads in 3 flips of the coin:
|
|
<md>
|
|
<mrow> \Pr(S = 2) \amp = (\text{number of sequences})(\text{probability of each sequence}) </mrow>
|
|
<mrow> \amp = (3)(0.096) </mrow>
|
|
<mrow> \amp = 0.288, </mrow>
|
|
</md>
|
|
which matches the value in <xref ref="table-binomial-example"/>.
|
|
More generally:
|
|
</p>
|
|
|
|
<fact xml:id="fact-binomial-probability-formula">
|
|
<title>Binomial Probability Formula</title>
|
|
|
|
<statement>
|
|
<p>
|
|
<md>
|
|
<mrow>b(k; n, p) = {n \choose k} p^k (1-p)^{n-k}</mrow>
|
|
</md>
|
|
</p>
|
|
</statement>
|
|
</fact>
|
|
</subsection>
|
|
|
|
|
|
<subsection xml:id="subsec-geometric-distribution">
|
|
<title>Geometric Distribution</title>
|
|
|
|
<definition xml:id="def-geometric-distribution">
|
|
<statement>
|
|
<p>
|
|
Suppose an event occurs with probability <m>p</m>.
|
|
If we perform independent trials repeatedly, let <m>N</m> be the random variable which counts the number of trials performed until we see the event occur for the first time.
|
|
(<m>N</m> counts the final trial in which the event occurs.) Then <m>N</m> has the <term>geometric distribution</term> with parameter <m>p</m>.
|
|
We'll write <m>N \sim \Geom(p)</m> to denote this.
|
|
For each value <m>k \geq 1</m>, we'll write <m>g(k)</m> for <m>\Pr(N = k)</m>.
|
|
(If we want to keep track of the parameter value, we may write <m>g(k; p)</m>.)
|
|
</p>
|
|
</statement>
|
|
</definition>
|
|
|
|
<p>
|
|
Suppose we flip a coin repeatedly until we first see heads.
|
|
The number <m>N</m> of flips is geometrically distributed with parameter <m>p</m>, which gives the probability of a flip coming up heads.
|
|
Then the probability of tails is <m>1 - p</m>.
|
|
For any particular <m>k \geq 1</m>, in order to have <m>N = k</m>, we must see exactly the flip sequence <m>TT\dotsm TH</m>, where the number of tails is exactly <m>k - 1</m>.
|
|
The probability of seeing this particular sequence is then:
|
|
</p>
|
|
|
|
<fact xml:id="fact-geometric-probability-formula">
|
|
<title>Geometric Probability Formula</title>
|
|
|
|
<statement>
|
|
<p>
|
|
<md>
|
|
<mrow> g(k; p) = (1-p)^{k-1} p </mrow>
|
|
</md>
|
|
</p>
|
|
</statement>
|
|
</fact>
|
|
</subsection>
|
|
|
|
|
|
<subsection xml:id="subsec-poisson-distribution">
|
|
<title>Poisson Distribution</title>
|
|
|
|
<definition xml:id="def-poisson-distribution">
|
|
<statement>
|
|
<p>
|
|
A <term>Poisson process</term> is one in which some event occurs randomly at a constant probabilistic rate <m>\lambda</m> over time.
|
|
Suppose we observe a Poisson process for a fixed amount of time <m>t</m>, and let <m>N</m> count the number of occurrences of the event.
|
|
Then <m>N</m> has the <term>Poisson distribution</term> with parameters <m>\lambda</m> and <m>t</m>.
|
|
We'll write <m>N \sim \Poiss(\lambda, t)</m> to denote this.
|
|
For each value <m>k \geq 0</m>, we'll write <m>p(k)</m> for <m>\Pr(N = k)</m>.
|
|
(If we want to keep track of the parameter values, we may write <m>p(k; \lambda, t)</m>.
|
|
See <xref ref="example-poisson-rate"/> for an important detail about units.)
|
|
</p>
|
|
</statement>
|
|
</definition>
|
|
|
|
<fact xml:id="fact-poisson-probability-formula">
|
|
<title>Poisson Probability Formula</title>
|
|
|
|
<statement>
|
|
<p>
|
|
<md>
|
|
<mrow> p(k; \lambda, t) = \frac{(\lambda t)^k}{k!} e^{-\lambda t}. </mrow>
|
|
</md>
|
|
</p>
|
|
</statement>
|
|
</fact>
|
|
|
|
<p>
|
|
The term <m>\lambda t</m> is potentially misleading here depending on the units given for each value.
|
|
The idea is that the rate information might be given for a different period of time than the observation period, so the rate should be scaled before calculating a probability.
|
|
But the rate and observation time must be using the same time unit before multiplying.
|
|
See the following example.
|
|
</p>
|
|
|
|
<example xml:id="example-poisson-rate">
|
|
<statement>
|
|
<p>
|
|
Suppose we observe traffic along a particular stretch of highway which typically has 200 cars pass per hour.
|
|
Let <m>N</m> be the number of cars seen in a 2-hour observation period.
|
|
Since the parameter <m>\lambda = 200</m> cars per hour is given for a 1-hour period, we should scale it to find the rate for a 2-hour period: <m>\lambda t = (200)(2) = 400</m> cars per 2-hours.
|
|
Then, for example:
|
|
<md>
|
|
<mrow> \Pr(N = 375) = p(375; 200, 2) = \frac{400^{375}}{375!} e^{-400} \approx 0.01. </mrow>
|
|
</md>
|
|
</p>
|
|
|
|
<p>
|
|
Now consider the random variable <m>M</m> counting the number of cars seen in a 30-minute observation period.
|
|
We should not calculate <m>\lambda t = (200)(30) = 6000</m>, since <m>\lambda</m> measures time in hours and <m>t</m> measures time in minutes.
|
|
We should first convert <m>t = 0.5</m> hours, then we can find our appropriately scaled rate information: <m>(200)(0.5) = 100</m> cars per half-hour.
|
|
Then, for example:
|
|
<md>
|
|
<mrow> \Pr(M = 110) = p(110; 200, 0.5) = \frac{100^{110}}{110!}e^{-100} \approx 0.02. </mrow>
|
|
</md>
|
|
Note that when we write the notation <m>p(k; \lambda, t)</m>, we assume that <m>\lambda</m> and <m>t</m> are expressed with the same time units already.
|
|
</p>
|
|
</statement>
|
|
</example>
|
|
</subsection>
|
|
|
|
<exercises xml:id="exercises-Discrete-RVs"> <!--
|
|
<exercise>
|
|
<statement>
|
|
<p>
|
|
We flip a coin with unknown parameter <m>p</m> 100 times and observe 43 heads.
|
|
What is the maximum likelihood estimation for the probability of the coin coming up heads?
|
|
</p>
|
|
</statement>
|
|
</exercise>
|
|
-->
|
|
<exercise>
|
|
<statement>
|
|
<p>
|
|
An experiment consists of flipping a biased coin 20 times.
|
|
If the coin comes up heads with probability <m>p = 0.3</m>, find the probability of seeing 5 heads.
|
|
Find the probability of seeing up to (and including) 3 heads.
|
|
</p>
|
|
</statement>
|
|
|
|
<solution>
|
|
<p>
|
|
Let <m>S</m> be the number of heads.
|
|
Then <m>S \sim \Bin(20, 0.3)</m>, so:
|
|
<md>
|
|
<mrow> \Pr(S = 5) \amp = {20 \choose 5} (0.3)^5 (0.7)^{20 - 5} </mrow>
|
|
<mrow> \amp = \frac{20!}{(5!)(15!)} (0.3)^5 (0.7)^{15} </mrow>
|
|
<mrow> \amp = \frac{20 \times 19 \times 18 \times 17 \times 16}{5 \times 4 \times 3 \times 2 \times 1} (0.3)^5 (0.7)^{15} </mrow>
|
|
<mrow> \amp = (19 \times 3 \times 17 \times 16) (0.3)^5 (0.7)^{15} </mrow>
|
|
<mrow> \amp \approx 0.179 </mrow>
|
|
</md>
|
|
</p>
|
|
</solution>
|
|
</exercise>
|
|
|
|
<exercise>
|
|
<introduction>
|
|
<p>
|
|
An experiment consists of flipping a coin repeatedly until we first see heads.
|
|
</p>
|
|
</introduction>
|
|
|
|
|
|
<task>
|
|
<statement>
|
|
<p>
|
|
If the coin comes up heads with probability 0.4, what is the probability we'll see our first heads within three flips? What about precisely on the third flip?
|
|
</p>
|
|
</statement>
|
|
|
|
<solution>
|
|
<p>
|
|
Let <m>T</m> be the number of flips until we see heads.
|
|
Then <m>T</m> is geometric with parameter <m>p = 0.4</m>, so:
|
|
<md>
|
|
<mrow> \Pr(T = k) \amp = (1-0.4)^{k-1}(0.4) = 0.6^{k-1} \cdot 0.4 </mrow>
|
|
</md>
|
|
</p>
|
|
</solution>
|
|
</task>
|
|
|
|
|
|
<task>
|
|
<statement>
|
|
<p>
|
|
Which flip has the highest chance of being the first flip to come up heads?
|
|
</p>
|
|
</statement>
|
|
|
|
<solution>
|
|
<p>
|
|
<m>\Pr(T = k) = 0.6^{k-1} \cdot 0.4</m>, so every additional flip multiplies the probability by 0.6.
|
|
Therefore, the highest value for <m>\Pr(T = k)</m> occurs when <m>k = 1</m>, in which case <m>\Pr(T = 1) = 0.4.</m>
|
|
</p>
|
|
</solution>
|
|
</task>
|
|
</exercise>
|
|
<exercise>
|
|
<statement>
|
|
<p>
|
|
A particular store has an average of 20 customers each hour.
|
|
During a 4-hour afternoon shift, what is the probability of serving 80 customers.
|
|
</p>
|
|
</statement>
|
|
|
|
<solution>
|
|
<p>
|
|
Let <m>N</m> be the number of customers seen during the afternoon shift.
|
|
Since the store averages 20 customers per hour, it will average 80 per 4-hours.
|
|
So <m>N \sim \Poiss(20, 4)</m> will have distribution <m>\Pr(N = k) = \frac{80^{k}}{k!} e^{-80}</m>.
|
|
Therefore:
|
|
<md>
|
|
<mrow> \Pr(N = 80) = \frac{80^{80}}{80!} e^{-80} \approx 0.045. </mrow>
|
|
</md>
|
|
</p>
|
|
</solution>
|
|
</exercise>
|
|
</exercises>
|
|
</section> |