STAT 542: Theory of Probability and Statistics
a Probability is a branch of mathematics concerned with the study of random
phenomenon (e.g., experiments, models of populations)
9 We are primarily interested in probability as it relates to
statistical infe
Functions of a random variable
Continuous r.v.s: the monotone case
Recall the support 26' = {CC E R : fX(a:) > 0}. Consider Y = g(X).
Additionally, suppose has a strictly positive derivative. Then,
0‘ g is strictly (monotone) increasing (u < in X iff g(u)
Functions of a random variable
Continuous r.v.s: the non—monotone case
The previous ideas on obtaining the pdf of Y = g(X) can be extended as follows.
If 9 isn’t monotone for all 1;, but there’s a way to break up the support X :2
{a3 : fX(cc) > 0} into se
Moment generating functions
Deﬁnition
, I ‘
Chwﬁfah‘sifc fmﬂwciuom [(3 We ch(.9/w “fhwszéwmdgau
430M 2 5 MM) ﬁx). 01X
: fkéQ‘f)
Deﬁnition: The moment generating function (mgf) of a random variable X
is _ X‘l‘
Mxm =EietX] [(60+) 0 a '
,—
assuming that E
Expected values
Variance
An important instance of this Eg(X) notion comes using g(X) = [X —- EX ]2
Deﬁnition: The variance of a random variable X, denoted Var(X) or 0%, is
Var(X) = 0% = — 1333?,
the expected squared distance between X and its mean EX
/7‘
Expected values
Examples
Examples:
1. Random seating of ten people around a table: X = # seats between A 85 B.
2. Toss a/ coin With P (‘T’ on toss = p. Supposing coin ﬂips are independent,
let Y =toss on which 1st ‘T’ is observed so that
(1—p)y“1p 2/: 1
Moment generating functions
More applications (cont’d)
3. Convergence (more on this later)
‘/ {C(00) ﬂl‘ ‘/\J «
0 Suppose a r.v. X has mgf MX(t) and suppose X1, X2, . .
of random variables, Where Xn has mgf M Xn (t) for each n 2 1.
If
leXn (1)
holds for
Inequalities
Jensen’s inequality
0 Deﬁnition of a convex function: g($) is convex if
Add»).
9W: +(1— My) S MM) + (1 - AMy)
2; M MM)sz
for all 50,3; and all 0 < A < 1.
c Also, 9(33) is convex if g”(;z:) Z 0 for all :1:
o 9(x) is concave if —g(ac) is conv
Common univariate distributions
Discrete distributions: Hypergeometric and Binomial
1. Hypergeometric: sampling without replacement
i.e., choose a: special objects in a size K sample from a collection Where M
objects are “special” 85 N -— M are not
2. Bin
Conditional Probability and independence
Independence example
/
The assumption of independence of events allows the computation of joint occur—
rences of events through simple calculations
Example: “Parallel System Reliability”
Suppose one can send a mess
Functions of a random variable
Introduction
9 Consider a random variable X N and a function g : R ——> R
' 6 Then, Y : g(X) is also a r.v., having its own cdf
o Formally, there is also an inverse mapping 9‘1 deﬁned by
9-1(14) = {:13 E R : 9(30) E A} for
Random variables
More cdf examples
Example 1: Let p =probability of a head on one toss of a coin 85
X =# of coin ﬂips required to get a ‘head’
0 It turns out (for later): P(X =32 = p(1 —~;p)"”1 for a: = 1,2, 3, . . .,
X is said to have a geometric distrib
Convergence concepts
Convergence in distribution
Deﬁnition: Yn converges in distribution to Y, denoted as Yn 3+ Y as n ~—+ 00,
if
FY71?) = FYW)
for any y E R at which is continuous (i.e., not all y)
M». “pm”
a Concerns the limiting distribution of a sequ
Introduction to Probability
Thinking about probability
a Want to assign probabilities to events A C_ S
.e Interpretations of probability
- limiting relative frequency nevi“? k 00"“ a (Mje nméey' 9f 2‘in
— subjective belief
o For now, we’ll ignore interpre
Introduction to Probability
Set Theory: Deﬁnitions
9 set: A is a collection of elements
(in our case A is a collection of outcomes)
Am evm‘t 1‘3. 54 set ,
a membership: 30 E A or a: E A
(:2: is in A or as is not in A)
a complement: AC 2 {51: : 33 E A}
(:1
Introduction to Probability
‘ Example: the equally likely outcome case
a lotto games often require a player to pick 7" from among the ﬁrst n integers
e e.g., Minnesota Lottery “Gopher 5”: pick 7" = 5 numbers from n z 47
9 Number of possibilities
1. if the
Conditional Probability and independence
Conditional probability for computing the probability of intersections
It follows from our deﬁnition of conditional probabilitythat
P(A n B) = P(B1A)P(A) = P(A|B)P(B)
Example: Urn 1 has 3 white 85 1 red balls; Urn
Random variables
PrOperties of probability density or mass functions
A function f is a pdf (or pmf) for some random variableif and only if
1. f(:r) ZOfor anyxelR
zigﬂmw=1 oitﬂa=n
Any nonnegative function having a ﬁnite integral (or sum) can be turncfd int
Common univariate distributions
Continuous distributions: Gamma
/9/44‘19g mom/1e"? :
dole ﬂav‘m/Wfo/f‘ .
X ~Gamma(oz,8f oz >0, > 0
6 pdf given by
1
fX(33) = ma“1e‘m/ﬁ, 0 < a: < 00
WOW“
o Motivation: ﬂexible family for positive quantities
0 or > 0 is sha
Common univariate distributions
Discrete distributions: Binomial distribution
Binomial distribution, X ~Binom(n, p), 0 < p < 1
o prnf given by
n _
fX<£I3> : : 2 2 m, [L‘ : 0,1, . . . ,n
FM
0 Motivation: distribution for the number of successes in n indepe
Common univariate distributions
Continuous distributions: Beta
X ~Beta(oz,ﬁ) oz > 0,8 > 0
0 pdf given by
fX($) : B(a I8)xaﬂl(1 Haﬁﬁglv
O<m<1
e Motivation: ﬂexible family, often for modeling quantities as proportions
0 oz, ,8 > O are both shape parameter
Conditional distributions
Conditional expectations turned into random variables
Note again that E[g(X,Y)|X : w] = E[g(X, can be thought of simply as
some function of :3, say
This means that we can invent a random variable that is a function of X by
deﬁnin
Independence
Introduction
Recall: Two events A, B are independent if P(A F] B) = P(A)P(B)
We next want to extend the concept of independence to random variables
Deﬁnition: Random variables X and Y are independent if
P(X e A,Y e B) = P(X Xe A)P(Y e B) for
Independence
Some important results 85 facts
4. Corollary: If X , Y are independent and EX 2, EY2 exist, then
Var(aX + bY) = Var(aX) + Var(bY) = a2Var(X) + bZVar(Y)
w (m w) am + mmﬂlﬂm,
I /
O (A “5/6 1‘ 1:4 M€f.&l’tj€n€€
5. Corollary: If X, are independen
Multivariate transformations
Bivariate discrete example (ﬁnding pmf directly)
o Suppose X ~Binomial(n1,p) and Y ~Bin0mia1(n2, p) are independent
gm pow
W
W
oDeﬁneUngfEdng
. _ ’ MSWMZ
V“ V‘ uswm
e SupportisB={(u,v):v=0,.,n2; u=v,v+1,.,v+n1}
o joint pmf of
a *9 #1
Convergence concepts )61‘: N / W; A
Central limit theorem (CLT) I: 1
MV‘Hh/Mi‘aie CLT >EGk lat
CL’I‘: If X1, X2, . . . , are iid random vectors in R“ with mean EXi = u E R1“ and
Var(Xi) = 2, then E X“ v id] - - .
M v «a (a — u) 3» Mme, 2) I l '
as
Probabilistic Simulation
General Method for Direct Simulation
General Method for Discrete Distributions
1. Suppose F is a discrete cdf with jumps at :31 < :52 < :63 < ~ - - .
mfi fag/3:132)“- og anemic.—
t
S”
2. Then for N U(O, 1), deﬁne
X={m
532'
3. Then
Convergence concepts
Tools of convergence in probability (cont’d)
3. Theorem (continuous mapping theorem): Suppose X, are RIC-valued
random vectors such that Yn A c 65 RI“. Suppose also that g 2 Rh -> R is
continuous at 0. Then,
9(Yn) 33> 9(0) as n v 00-