andy/Probability-and-Statistics-Notes
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

First pass at Chapter 2

Browse Source
This commit is contained in:
andyeisenberg
2025-12-31 20:43:24 +00:00
parent 8cfd805aeb
commit 2114c9c90e
2 changed files with 289 additions and 68 deletions
Download Patch File Download Diff File Expand all files Collapse all files
source/ch-Random-Variables.ptx
+286 -66
View File
@@ -412,80 +412,300 @@
<section xml:id="sec-Continuous-RVs" xmlns:xi="http://www.w3.org/2001/XInclude">
<title>Continuous Random Variables</title>
<p>
We have to think about probability differently for continuous random variables.
If <m>X</m> is a continuous random variable taking values in an interval <m>[a, b]</m>, it doesn't quite make sense to ask, as we have for discrete random variables, to find the probabilities of each individual outcome.
</p>
<subsection xml:id="subsec-pdfs">
<title>Probability Density Functions</title>
<p>
When we roll a fair 6-sided die, there are 6 equally likely outcomes, so they each get assigned the probability <m>1/6</m>.
If we try to select a random number in the interval <m>[0, 1]</m> in a similarly "fair" way (i.e., with each outcome equally likely), then what probability <m>\alpha</m> should we assign to each outcome? If <m>\alpha > 0</m>, then we have a problem: if you add up the probabilities of all the (infinitely many) outcomes, you get an infinite total, whereas the total probability in any sample space should be 1.
If <m>\alpha = 0</m>, then we also have a problem: if you add up the probabilities of all the outcomes, you get total of 0, whereas the total probability in any sample space should be 1.
</p>
<p>
We have to think about probability differently for continuous random variables.
If <m>X</m> is a continuous random variable taking values in an interval <m>[a, b]</m>, it doesn't quite make sense to ask, as we have for discrete random variables, to find the probabilities of each individual outcome.
</p>
<p>
We need a new way to model and calculate probabilities for continuous random variables.
</p>
<p>
When we roll a fair 6-sided die, there are 6 equally likely outcomes, so they each get assigned the probability <m>1/6</m>.
If we try to select a random number in the interval <m>[0, 1]</m> in a similarly "fair" way (i.e., with each outcome equally likely), then what probability <m>\alpha</m> should we assign to each outcome? If <m>\alpha > 0</m>, then we have a problem: if you add up the probabilities of all the (infinitely many) outcomes, you get an infinite total, whereas the total probability in any sample space should be 1.
If <m>\alpha = 0</m>, then we also have a problem: if you add up the probabilities of all the outcomes, you get total of 0, whereas the total probability in any sample space should be 1.
</p>
<definition xml:id="def-pdf">
<statement>
<p>
Let <m>X</m> be a continuous random variable taking values in the interval <m>[a, b]</m>.
The <term>probability density function</term> (or <term>pdf</term>) for <m>X</m> is a function <m>f(x)</m> defined on <m>[a, b]</m> satisfying the properties:
<ol>
<li>
<p>
<m>f(x) \geq 0</m> for all <m>x \in [a, b]</m>, and
</p>
</li>
<p>
We need a new way to model and calculate probabilities for continuous random variables.
</p>
<li>
<p>
<m>\displaystyle{\int_a^b f(x)\ dx = 1}</m>.
</p>
</li>
</ol>
</p>
</statement>
</definition>
<definition xml:id="def-pdf">
<statement>
<p>
Let <m>X</m> be a continuous random variable taking values in the interval <m>[a, b]</m>.
The <term>probability density function</term> (or <term>pdf</term>) for <m>X</m> is a function <m>f(x)</m> defined on <m>[a, b]</m> satisfying the properties:
<ol>
<li>
<p>
<m>f(x) \geq 0</m> for all <m>x \in [a, b]</m>, and
</p>
</li>
<p>
It's important to keep in mind that the output values of a pdf <m>f(x)</m> are <em>probability densities</em>, not probabilities.
You can (somewhat casually) think of the units of <m>f(x)</m> values as "probability per &lt;units of the sample space&gt;".
You may not have discussed the units of the differential <m>dx</m> very much in your Calculus 1 course, but <m>dx</m> has the same units as <m>x</m> and is being multiplied by the integrand when you take an integral.
So, although <m>f(x)</m> is not a probability, the result of computing <m>\int_c^d f(x)\ dx</m> <em>is</em> a probability.
</p>
<li>
<p>
<m>\displaystyle{\int_a^b f(x)\ dx = 1}</m>.
</p>
</li>
</ol>
</p>
</statement>
</definition>
<p>
A pdf's purpose in life is to be integrated, which tells us probabilities for a continuous random variable:
<md>
<mrow> \Pr(c \leq X \leq d) = \int_c^d f(x)\ dx. </mrow>
</md>
</p>
<p>
It's important to keep in mind that the output values of a pdf <m>f(x)</m> are <em>probability densities</em>, not probabilities.
You can (somewhat casually) think of the units of <m>f(x)</m> values as "probability per &lt;units of the sample space&gt;".
You may not have discussed the units of the differential <m>dx</m> very much in your Calculus 1 course, but <m>dx</m> has the same units as <m>x</m> and is being multiplied by the integrand when you take an integral.
So, although <m>f(x)</m> is not a probability, the result of computing <m>\int_c^d f(x)\ dx</m> <em>is</em> a probability.
</p>
<definition xml:id="def-continuous-uniform-distribution">
<statement>
<p>
Let <m>X</m> take values in the interval <m>[a, b]</m>.
We say that <m>X</m> has the <term>uniform distribution</term> if, for any subinterval <m>[c, d] \subset [a, b]</m>, the probability that <m>X</m> lies within <m>[c, d]</m> is equal to the proportion of the total sample space taken by <m>[c, d]</m>.
That is:
<md>
<mrow> \Pr(c \leq X \leq d) = \frac{d - c}{b - a}. </mrow>
</md>
In this case, <m>X</m> has the pdf <m>f(x) = \frac{1}{b - a}</m>.
</p>
</statement>
</definition>
<p>
A pdf's purpose in life is to be integrated, which tells us probabilities for a continuous random variable:
<md>
<mrow> \Pr(c \leq X \leq d) = \int_c^d f(x)\ dx. </mrow>
</md>
</p>
<p>
We'll ignore for the moment how to find this pdf and instead we'll quickly check that it behaves the way we want it to:
<md>
<mrow> \Pr(c \leq X \leq d) \amp = \int_c^d \frac{1}{b-a}\ dx </mrow>
<mrow> \amp = \frac{x}{b - a}\bigg|_c^d </mrow>
<mrow> \amp = \frac{d - c}{b - a}, </mrow>
</md>
as we wanted.
</p>
<definition xml:id="def-continuous-uniform-distribution">
<statement>
<p>
Let <m>X</m> take values in the interval <m>[a, b]</m>.
We say that <m>X</m> has the <term>uniform distribution</term> if, for any subinterval <m>[c, d] \subset [a, b]</m>, the probability that <m>X</m> lies within <m>[c, d]</m> is equal to the proportion of the total sample space taken by <m>[c, d]</m>.
That is:
<md>
<mrow> \Pr(c \leq X \leq d) = \frac{d - c}{b - a}. </mrow>
</md>
In this case, <m>X</m> has the pdf <m>f(x) = \frac{1}{b - a}</m>.
</p>
</statement>
</definition>
<p>
We'll ignore for the moment how to find this pdf and instead we'll quickly check that it behaves the way we want it to:
<md>
<mrow> \Pr(c \leq X \leq d) \amp = \int_c^d \frac{1}{b-a}\ dx </mrow>
<mrow> \amp = \frac{x}{b - a}\bigg|_c^d </mrow>
<mrow> \amp = \frac{d - c}{b - a}, </mrow>
</md>
as we wanted.
</p>
<example>
<statement>
<p>
Let <m>X</m> be a continuous random variable taking values in the interval <m>[0, 4]</m> with pdf <m>f(x) = k \sqrt{x}</m> for some constant <m>k</m>.
Because of the restriction that the total probability must be 1, we can determine the value of <m>k</m>:
<md>
<mrow> 1 = \Pr(0\leq X \leq 4) \amp = \int_0^4 k\sqrt{x}\ dx </mrow>
<mrow> \amp = \frac{kx^{3/2}}{3/2}\bigg|_0^4 </mrow>
<mrow> \amp = \frac{2kx^{3/2}}{3}\bigg|_0^4 </mrow>
<mrow> \amp = \left(\frac{2k(4^{3/2})}{3}\right) - 0 </mrow>
<mrow> \amp = \frac{16k}{3} </mrow>
</md>
Therefore <m> k = \frac{3}{16}</m>.
Now we can calculate probabilities, e.g.:
<md>
<mrow> \Pr(1 \leq X \leq 2) \amp = \int_1^2 \frac{3\sqrt{x}}{16}\ dx </mrow>
<mrow> \amp = \frac{3}{16}\left(\frac{2x^{3/2}}{3}\right)\bigg|_1^2 </mrow>
<mrow> \amp = \frac{1}{8}\left(x^{3/2}\right)\bigg|_1^2 </mrow>
<mrow> \amp = \frac{1}{8}\left( 2^{3/2} - 1 \right) </mrow>
<mrow> \amp \approx 0.23. </mrow>
</md>
</p>
</statement>
</example>
</subsection>
<subsection xml:id="subsec-cdf">
<title>Cumulative Distribution Functions</title>
<p>
When a random variable takes values in an interval <m>[a, b]</m>, it's often useful to consider how much probability has been accumulated up to some specified value <m>x \in [a, b]</m>.
For each value of <m>x</m>, we can calculate this accumulated probability.
Since there's an association between inputs (the <m>x</m> values) and output (the accumulated probability), we can naturally build a function.
</p>
<definition xml:id="def-cdf">
<statement>
<p>
Let <m>X</m> be a continuous random variable taking values in an interval <m>[a, b]</m>.
The <term>cumulative distribution function</term> (or <term>cdf</term>) is the function <m>F(x)</m> defined on <m>[a, b]</m> such that
<md>
<mrow> F(x) = \Pr(a \leq X \leq x). </mrow>
</md>
</p>
</statement>
</definition>
<p>
How do we actually calculate these cumulative probabilities? Luckily, these are precisely the sort of probabilities we can use the pdf to find:
<md>
<mrow> F(x) = \Pr(a \leq X \leq x) = \int_a^x f(t)\ dt. </mrow>
</md>
(Note that on the far left side of the equation, we used the letter <m>x</m> to refer to some point within <m>[a, b]</m> up to which we're trying to accumulate probability.
So that value <m>x</m> must also appear at the top of the integral sign.
We can't then use the same letter for the integrand and differential inside the integral; we can't use the same letter to refer to two different variables at the same time.
So, we have changed the letter inside the integral to <m>t</m>.)
</p>
<p>
This equation may look familiar from Calculus 1.
If you review <url href="https://en.wikipedia.org/wiki/Fundamental_theorem_of_calculus#First_part">the First Fundamental Theorem of Calculus</url>, you'll see precisely the relationship between the cdf <m>F(x)</m> and the pdf <m>f(x)</m>:
<md>
<mrow> F'(x) = f(x). </mrow>
</md>
Let's see how we can put this relationship to use.
</p>
<example>
<statement>
<p>
Suppose <m>X</m> is uniformly distributed in the interval <m>[a, b]</m>.
We previously said in <xref ref="def-continuous-uniform-distribution"/> that, for any subinterval <m>[c, d] \subset [a, b]</m>, we should have
<md>
<mrow> \Pr(c\leq X \leq d) = \frac{d - c}{b - a}. </mrow>
</md>
In particular, for any <m>x\in [a, b]</m>, we would have:
<md>
<mrow> F(x) = \Pr(a \leq X \leq x) = \frac{x - a}{b - a}. </mrow>
</md>
Therefore, taking the derivative:
<md>
<mrow> f(x) = F'(x) = \frac{1}{b - a}. </mrow>
</md>
</p>
</statement>
</example>
<p>
So, we can see that the idea we wanted to capture in probability led to a formula that gave accumulated probability, which then led to the pdf.
Both the cdf and the pdf contain essentially all of the information we need to answer probability questions about <m>X</m>.
However, keep in mind that these functions output different mathematical objects.
<m>F(x)</m> outputs probabilities already; <m>f(x)</m> must be integrated over a subinterval to find probabilities.
</p>
</subsection>
<subsection xml:id="subsec-exponential-distribution">
<title>The Exponential Distribution</title>
<definition xml:id="def-exponential-distribution">
<statement>
<p>
Suppose we observe a Poisson process in which an event occurs with rate parameter <m>\lambda</m>.
Let <m>T</m> be the time until the event is first observed.
Then <m>T</m> has the <term>exponential distribution</term> with parameter <m>\lambda</m>.
We'll write <m>T \sim \Exp(\lambda)</m> to denote this.
</p>
</statement>
</definition>
<fact xml:id="fact-exponential-pdf">
<title>Exponential Density Function</title>
<statement>
<p>
<md>
<mrow> f(t) = \lambda e^{-\lambda t} </mrow>
</md>
</p>
</statement>
</fact>
<p>
The binomial, geometric, Poisson, and exponential distributions are conceptually grouped together.
Consider the following two dichotomies:
<ol>
<li>
<p>
Is time being measured discretely or continuously?
</p>
</li>
<li>
<p>
Does the random variable count occurrences of an event, or does it measure time until an event?
</p>
</li>
</ol>
There are four combinations of answers to these two questions.
<dl>
<li>
<title>Binomial</title>
<p>
"Time" is discrete; the random variables counts occurrences.
</p>
</li>
<li>
<title>Geometric</title>
<p>
"Time" is discrete; the random variable measures "time" until an event.
</p>
</li>
<li>
<title>Poisson</title>
<p>
Time is continuous; the random variable counts occurrences.
</p>
</li>
<li>
<title>Exponential</title>
<p>
Time is continuous; the random variable measures time until an event.
</p>
</li>
</dl> When we talk about discrete "time", we don't always mean something measured in actual units of time. We might; for example, an experiment may involve a bacterial colony growing in a petri dish over time, but we only check in and measure something about the bacterial colony once each day. In this case, since our measurements of time must occur in whole numbers of days, time is being measured in a discrete way (that is, not involving an interval of real numbers).
</p>
<p>
Consider, instead, an experiment in which we flip a coin repeatedly until we first see heads appear.
The number of flips doesn't have units of time, but counting flips is analogous to measuring time until an event occurs.
The flip count is a discrete stand-in for something analogous to time.
We can consider this experiment to be a discrete analog of a Poisson process.
</p>
<p>
It's also worth including a warning here about the Poisson distribution, specifically.
The dichotomy mentioned above about whether time is being measured discretely or continuously is <em>not</em> the same as the consideration of whether the random variable itself is discrete or continuous.
In a Poisson process, time is being measured continuously.
But the Poisson random variable counts the number of occurrences of the event, and this number must be a whole number.
So a Poisson random variable is a discrete random variable.
</p>
</subsection>
<subsection xml:id="subsec-normal-distribution">
<title>Normal Distribution</title>
<definition xml:id="def-normal-distribution">
<statement>
<p>
Consider the parametrized family of functions:
<md>
<mrow> f(x; \mu, \sigma^2) = \frac{1}{\sqrt{2\pi \sigma^2}} e^{\frac{-(x-\mu)^2}{2\sigma^2}}. </mrow>
</md>
The function <m>f(x; \mu, \sigma^2)</m> is called a <term>normal density function</term> with parameters <m>\mu</m> and <m>\sigma^2</m>.
A continuous random variable <m>X</m> with pdf <m>f(x; \mu, \sigma^2)</m> is said to have a <term>normal distribution</term>.
In the specific case that <m>\mu = 0</m> and <m>\sigma^2 = 1</m>, we call the resulting distribution the <term>standard normal distribution</term>.
</p>
</statement>
</definition>
<p>
It's an unfortunate reality that the normal density functions can't easily be integrated.
(In fact, it is provably impossible to write nice formulas for their antiderivatives).
So we won't worry much about probability computations like <m>\Pr(c \leq X \leq d)</m> for normal distributions.
The normal distributions will play a <em>central</em> role in later chapters (after which you can return here and appreciate this pun).
</p>
</subsection>
<exercises xml:id="exercises-Continuous-RVs">
<exercise>
source/docinfo.ptx
+3 -2
View File
@@ -5,9 +5,9 @@
<!-- settings. -->
<docinfo xmlns:xi="http://www.w3.org/2001/XInclude">
<!-- A unique name for the book in a catalog; for example, see https://pretextbook.org/catalog.html and https://runestone.academy/ns/books/index. It contains a simple lowercase string with no dashes or other special characters. See https://pretextbook.org/doc/guide/html/sec-publishing-to-runestone-academy.html for more information. -->
<document-id>changeme</document-id>
<!-- <document-id>changeme</document-id> -->
<!-- A brief description for the book in a catalog; see the links above. The @shelf is used only when publishing to Runestone; see https://runestone.academy/ns/books/index for a list of valid values. You must copy these @shelf names exactly, including capitalization. -->
<blurb shelf="AP Computer Science"> A simple string (no markup is allowed) to describe your book. </blurb>
<!-- <blurb shelf="AP Computer Science"> A simple string (no markup is allowed) to describe your book. </blurb> -->
<!-- The next three lines are likely for mature projects only: -->
@@ -30,6 +30,7 @@
\DeclareMathOperator{\Bin}{Bin}
\DeclareMathOperator{\Geom}{Geom}
\DeclareMathOperator{\Poiss}{Poiss}
\DeclareMathOperator{\Exp}{Exp}
</macros>
<!-- If you put any latex-image elements you can include preambles -->