cs336f1018 - 11/8/10 What
we’ll
cover 
 Probability...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 11/8/10 What
we’ll
cover 
 Probability Basics Lecture 19 •  Tying
up
loose
ends
 •  Bayes’
Theorem
 •  Random
Variables
 •  Expectation
 •  Bounds:

Bonferonni,
Markov,
Chebyshev’s
 Inequalities
 Inclusion
Exclusion
Principle 
 •  The
inclusion–exclusion
principle
states
that
if
 A
and
B
are
two
(finite)
sets,
then
 •  Similarly,
for
three
sets
A,
B
and
C,
 •  For
the
general
case
of
the
principle,
let
A1,
...,
 An
be
finite
sets.
Then
 Example:
Modeling
Energy
Saving
 Transmissions
in
Wireless
Networks 
 
 One
node
transmits
on
a
random
k
out
of
N
 timeslots,
the
second
node
listens
on
a
 random
k
out
of
N
timeslots.
They
are
idle
in
 the
remaining
N‐k
timeslots.
What
is
the
 probability
that
2nd
node
hears
the
first?
 Michael
J.
McGlynn,
Steven
A.
Borbash.
Birthday
Protocols
 for
Low
Energy
Deployment
and
Flexible
Neighbor
 Discovery
in
Ad
Hoc
Wireless
Networks
(MobiHoc,
2001)
 The Key to Bayesian Methods P(A ∩ B) P(A|B) P(B) P(B|A) = ----------- = --------------- P(A) P(A) This is Bayes Rule General Forms of Bayes Rule Bayes, Thomas (1763) An essay towards solving a problem in the doctrine of chances. Philosophical Transactions of the Royal Society of London, 53:370-418 1 11/8/10 An Example General Forms of Bayes Rule Based on Andrew L. Moore tutorials Definition of Independent Events Events E and F are called independent iff P(E∩F)= P(E) P(F). Note that this is equivalent to P(E|F)=P(E) and P(F|E)=P(F). Random
Variables
 •  A
random
variable
is
a
function
that
maps
a
set
of
 outcomes
to
real
numbers
(or
subsets
of
real
numbers).

 We
use
capital
letters
to
represent
random
variables
(Eg.
X)
 and
lower
case
letters
to
represent
particular
outcomes.
 •  A
random
variable
can
be
discrete
or
continuous.

We
make
 a
distinction
between
the
two.
Today
we
are
going
to
 concentrate
on
discrete
random
variables.
 Random
Variables
 •  Since
a
random
variable
is
a
function
that
maps
a
 set
of
outcomes
of
an
experiment
(sample
space)
 to
real
numbers,
a

random
variable
is
a
quantity
 that
depends
on
chance.

 •  p(x)=P(X=x)
is
the
probability
mass
function
(pf)
 for
the
discrete
random
variable.

 
 
 For
example,
let
X
be
the
number
of
heads
in
 two
tosses
of
a
fair
coin.
 
 P(X=
1)
=
P
(HT
or
TH)
=
1/2
 Random
Variables
 What
is
the
expected
number
of
0’s
in
a
ternary
string
of
 length
4
if
all
such
strings
are
equally
likely.
 2 11/8/10 Random
Variables
 What
is
the
expected
number
of
0’s
in
a
ternary
string
of
 length
4
if
all
such
strings
are
equally
likely.
 Binomial
Random
Variables
 What
is
the
expected
number
of
0’s
in
a
ternary
string
of
 length
4
if
all
such
strings
are
equally
likely.
 Binomial
Random
Variables 
 •  A
binomial
experiment
is
defined
by
the
following
 conditions:
 1.  There
are
n
“trials”
where
n
is
determined
in
advance
and
 is
not
a
random
value.
 2.  There
are
two
possible
outcomes
on
each
trial,
called
 “success”
and
“failure”
and
denoted
S
and
F.
 3.  The
outcomes
are
independent
from
one
trial
to
the
next.
 4.  The
probability
of
a
“success”
remains
the
same
from
one
 trial
to
the
next,
and
this
probability
is
denoted
by
p.
The
 probability
of
a
“failure”
is
1

p
for
every
trial.
 Binomial
Theorem 
 •  Watch
a
YOUTUBE
video
at
http:// www.youtube.com/watch?v=Cv4YhIMfbeM
 The
Expectation
 
 The
expected
value
of
a
discrete
random
variable
is
the
sum
of
 values
of
the
random
variable
weighted
by
the
probabilities.

This
 is
also
referred
to
as
the
expectation,
the
average
or
the
mean
of
 the
random
variable.
 Expected
Value
 •  E[X]=(Σx:X=x:
xP(X=x))
 Example…the expected number of heads in two tosses E[X]= 0*(16/81) +1*(32/81)+ 2*(24/81)+3(8/81)+4(1/81) =(0+32+48+24+4)/81=108/81=4/3 Note that X is a random variable. It takes on different values with probability p(x). E[X] is constant, nonrandom. 3 11/8/10 Freeze
 •  Freeze is a game of chance where the players try to get to 100 first. After each roll of two number cubes, the player may have to make the decision to stop and freeze the score he or she has accumulated or risk everything. At each turn, if the player rolls doubles, he or she must take a zero for the score for this turn. Otherwise, the score for the turn is the sum of all of the rolls the player has made before he or she freezes. 
 Freeze
 •  For example, the player begins by rolling a 4 and 5 on the number cubes getting a total of 9 then decides to continue to play. This time the roll is a 3 and 2 for a sum of 5 so now the player ’s score is 14. Again the player decides continue to play. The roll is two fours (doubles). Now the player ’s score for the turn is 0 and another player begins.
 Freeze
 •  If the current score is X, the new score after a roll is X + sum on roll or 0 (call it S). A player might want to continue to play as long as the expected value of your new score is greater than the current score (E(S)>X). Freeze
 •  Use the table to find a strategy to decide when to freeze by looking at the expected score (S) if you continued play. Freeze
 S X+3 X+4 X+5 X+6 X+7 X+8 X+9 X+10 X+11 0 S X+3 X+4 X+5 Freeze
 X+6 X+7 X+8 X+9 X+10 X+11 0 P(S) P(S) 2/36 2/36 4/36 4/36 6/36 4/36 4/36 2/36 2/36 6/36 •  You will continue to play as long as the expected value of your new score is greater than your current score. •  E(S)= 30/36 X + (6+8+20+24+42+32+36+20+22)/36 =30/36 X + 210/36 •  When is E(S) < X, 30/36 X + 210/36 < X. So if X > 35, that is if your current score is over 35, freeze, otherwise go for it. 4 11/8/10 The
Expectation
is
linear

 •  E[aX+b]=aE[X]+b
 •  If
you
have
several
variables
X1,X2,…,Xn
then

 
 E[X1+X2+…+Xn]=E[X1]+E[X2]+…+E[Xn]

 Question:

 •  At
each
time
step,
I
flip
a
fair
coin.
If
it
comes
 up
Heads,
I
walk
one
step
to
the
right;
if
it
 comes
up
Tails,
I
walk
one
step
to
the
left.
 How
far
do
I
expect
to
have
traveled
from
my
 starting
point
after
n
moves?
 
 X
=
X1+X2+…+Xn,
 
 where
Xi
=
 +1
if
ith
toss
is
Heads;
 −1
otherwise.
 Variance
 •  For
independent
random
variables
X,Y,
we
 have
E[XY]
=
E[X]E[Y].
You
are
strongly
 encouraged
to
prove
this
as
an
exercise.
 •  Definition
Variance):
For
a
r.v.
X
with
 expectation
E[X]
=
m,
the
variance
of
X
is
 defined
to
be
 •  Var(X)
=
E[(X
−m)2].
 Variance
 •  Var(X)
=
E[(X
−m)2]=E[X2]
−m2.
 •  The
square
root
of
Var(X)
is
called
the
 standard
deviation
of
X.
 •  The
variance
(or,
the
standard
deviation)
is
a
 measure
of
“spread”,
or
deviation
from
the
 mean.

 Bonferoni’s inequality 
 •  P(E
∩
F)
≥
P(E)
+P(F)
‐
1
 Markov’s
Inequality
 5 11/8/10 Markov’s
Inequality
(a>0
and
non‐negative
rv
X)
 Chebyshev’s
Inequality
 •  Theorem:
[Chebyshev’s
Inequality]
For
a
 random
variable
X
with
expectation
E[X]
=
m,
 and
for
any
a
>
0,
 
 Pr[|X
−m|
≥a]
≤
Var(X)/(a2)
 Proof:
 Chebyshev’s
Inequality
 Theorem:
For
a
random
variable
X
with
expectation

 E[X]
=
m,
and
for
any
a
>
0,
Pr[|X
−m|
≥a]
≤
Var(X)/(a2)
 Proof:
 Pr[|X
−m|
≥a]
 =<squaring>
 
Pr[(X
−m)2
≥a2]
 ≤<Markov
Inequality>
 E[(X‐m)2]/a2
 =<Def.
of
Var>
 Var(X)/(a2)
 6 ...
View Full Document

This note was uploaded on 11/30/2010 for the course CS 336 taught by Professor Myers during the Fall '08 term at University of Texas at Austin.

Ask a homework question - tutors are online