STA 3007 Applied Probability
Tutorial 2
2005
1. Introduction to Markov Chain
(a) Exercise
i. Consider the problem of sending a binary message, 0 or 1, through
a signal channel consisting of several stages, where transmission
through each stage is subject
STA 3007 Applied Probability
Tutorial 7 Suggested Solution
2005
1. Revision
i. Yes.
ii.
0
00
P = 1+
2+
1
+
0
+
2
+
+
0
0
0+
P2 = 1 +
2+
1
+
+
+
2
+
+
+
and row sums equal 1.
Therefore, P is a regular Markov matrix.
iii. Since row sum of row 2 is not 1, it
STA 3007 Applied Probability
Tutorial 8
2005
1. Poisson Process
(a) Poisson Process
Denition A Poisson process of intensity, or rate, > 0 is an integer-valued stochastic process cfw_X (t); t 0 for which
(1) for any time points t0 = 0 < t1 < t2 < < tn , th
STA 3007 Applied Probability
Tutorial 8 Suggested Solution
2005
1. Poisson Process
i. (a) P rcfw_X (1) = 2
= P rcfw_X (1) X (0) = 2
2 (2)(1)
e
= [(2)(1)]2!
= 0.2706
(b) P rcfw_X (1) = 2 and X (3) = 6
= P rcfw_X (1) X (0) = 2 and X (3) X (1) = 4
= P rcfw_X
STA 3007 Applied Probability
Tutorial 9
2005
1. Poisson Process
(a) Distribution Associated with the Poisson Process
i. Example
1 Let X1 (t) and X2 (t) be independent Poisson processes having parameters 1 and 2 ,
respectively. What is the probability that
STA 3007 Applied Probability
Tutorial 10
2005
A The Uniform Distribution and Poisson Processes
(a) Theorem 3.6 Let W1 , W2 , . . . be the occurrence times in a Poisson process of rate > 0.
Conditioned on X (t) = n, the random variables W1 , W2 , . . . , W
STA 3007 Applied Probability
Tutorial 10 Suggested Solution
2005
1. The Uniform Distribution and Poisson Processes
X (t)
i. M = E [ k=1 2000eWk ]
X (t)
= 2000E [ k=1 eWk ]
X (t)
= 2000 n=1 E [ k=1 eWk |X (t) = n]P rcfw_X (t) = n
X (t)
E [ k=1 eWk |X (t) =
STA 3007 Applied Probability
Tutorial 11
2005
A. Marked Poisson Processes
A random variable Yk is associated with the k th event in a Poisson process of rate . We stipulate
that Y1 , Y2 , are independent and share the common distribution function
G(y ) =
STA 3007 Applied Probability
Tutorial 11 Suggested Solution
2005
1. Marked Poisson Processes
i. No. of English emigrating to area A Poisson()
1
= pt = 10( 12 )(4) = 10
3
10
Prob. that no English in Feb = e 3
ii. Let X0 (t) and X1 (t) be the Poisson proce
Suggested Solution to STA 3007 Assignment 1
1. Whether the next step shows a good item or a defective one only depends on the current step. So
this process is a Markov Chain.We can dene
0, if the nth item is a good item
1, if the nth item is a def ective
STA 3007 Assigment 2
Due date: 26th October, 2005
1. You have ve fair coins. You toss them all so that they randomly fall
heads or tails. Those that fall tails in rst you pick up and toss again.
You toss again those which show tails after the second toss,
STA 3007
Assignment 5: Self-correction problems
Students are not required to submit.
Solutions for the rst three questions will be provided shortly.
1. Customers demanding service at a central processing facility arrive according to a Poisson process a in
STA 3007 Applied Probability
Tutorial 7
2005
1. Revision
(a) Review on Probability
(b) Introduction to Markov Chain
(c) First Step Analysis
i. Probability of absorption in k, given an initial state i
Uik = P cfw_XT = k |X0 = i
ii. Expected absorption time
STA 3007 Applied Probability
Tutorial 6 Suggested Solution
2005
1. The Long Run Behavior of Markov Chains
i. Since 01 and 12, so 02.
All states can communicate with each other, the Markov chain is irreducible.
ii. 45 and 52, so 42
23 and 34, so 24
Therefo
STA 3007 Applied Probability
Tutorial 6
2005
1. The Long Run Behavior of Markov Chains
(a) Regular Markov Matrices
A Markov Matrix P is said to be regular if P k has all of its elements strictly postive for some
power k.
Theorem 2.1
Let P be a Regular Mar
What is probability?
Probability is a kind of mathematics.
It deals with problems with uncertainty.
Applied Probability
We need probability because
We are ignorance.
The underlying physical rule is unclear.
We have to do something that we dont know.
Finding a cheese in the maze
mouse
Markov Chain
Black
hole
Chapter 1
Cheese
2
1
Assumptions
Questions
The mouse does not know where the
cheese is.
The mouse will get nothing and leaves the
game if it enters the black hole.
A biologist may want to know the
Long Run Behavior of
Markov Chains
Regular Markov matrices
Chapter 2
Chapter 2.1
1
Definition
2
Mathematical Description
A Markov matrix P is said to be regular if Pk
has all of its elements strictly positive for some
power k.
The corresponding Markov cha
Poisson Processes
Poisson Distribution and
the Poisson Process
Chapter 3
Chapter 3.1
1
Poisson Distribution
Theorem 3.1
The Poisson distribution with parameter > 0 is
given by
k
Prcfw_X = k = pk =
2
e
k!
Let X be Poisson r.v. Then, E[X]= .
Moreover, E[X
STA 3007: Applied Probability
Textbook: Taylor and Karlin, An Introduction to Stochastic Modeling (latest edition),
Academic Press.
Instructor: Dr. Wong, Hoi-Ying
Website: http:/www.sta.cuhk.edu.hk/hywong/
Email: hywong@cuhk.edu.hk
Office: Room G20, La
STA 3007 Applied Probability
Tutorial 1
2005
1. Review on Probability
(a) Discrete Probability Distributions
pX (x) = P rcfw_X = x
i. pX (x) 0
ii. x pX (x) = 1
(b) Joint Probability Mass Function (Discrete)
pXY (x, y ) = P rcfw_X = x, Y = y
i. pXY (x, y
STA 3007 Applied Probability
Tutorial 3
2005
1. First Step Analysis
(a) Notation
i. Absorbing time T
ii. Probability of absorption in k, given an initial state i
Uik = P cfw_XT = k |X0 = i
iii. Expected absorption time given an inital transient state i
vi