1
ISyE 6761 Fall 2012
Homework #3 Solutions (revised 10/24/12)
1. Three white and three black balls are distributed in two urns in such a way that
each urn contains 3 balls. The system is in state i, i = 0, 1, 2, 3, if the rst urn
contains i white balls.
CME308: Assignment 3
Due: 5:00 pm, May 9
Problem 1: In Monte Carlo method, we generate X1 , . . . , Xn with density p(x) and use
an estimate for the quantity Ep f (X) = R f (x)p(x)dx.
1
n
n
i=1
f (Xi ) as
To reduce variance we can use the importance sampl
CME308: Assignment 3
Due: 5:00 pm, May 9
Problem 1: In Monte Carlo method, we generate X1 , . . . , Xn with density p(x) and use
an estimate for the quantity Ep f (X) = R f (x)p(x)dx.
1
n
n
i=1
f (Xi ) as
To reduce variance we can use the importance sampl
CME308-2014: Assignment 4
Due: Friday May 30, 5:00pm
Problem 1 (10 pts): First passage probabilities. Consider a nite state Markov chain with transition
probabilities P (x, y), x, y S and let F n (x, y) be the probability that y is reached for the rst tim
CME308-2014: Assignment 4
Due: Friday May 30, 5:00pm
Problem 1 (10 pts): Consider a nite state Markov chain with transition probabilities P (x, y), x, y S
and let Fn (x, y) be the probability that y is reached for the rst time at n, starting from x
Fn (x,
Math 285, Spring 2014
Homework 3 Solutions
2.3. Clearly this Markov chain is irreducible. It is also recurrent: For T0 = mincfw_n 1 :
Xn = 0 we have P0 [T > n] = (2/3)n , so P0 [T0 = ] = 0, and the state 0 is recurrent.
Let us nd the invariant measure and
Math 285, Spring 2014
Homework 2 Solutions
1.9. (a) Yes. In fact, by the discussion in part (b), it is clear that for each pair of states
(i, j) we have pn (i, j) > 0 for some n cfw_1, 2, 3.
(b) The period of the chain is 3: Starting in state 1 the chain
Math 285
Spring 2014
Homework 1 Solutions
1.1. It is reasonable to model this by a Markov chain with state space S = cfw_0, 1, 2, 3, 4.
The transition matrix is
1/3
1/3
P = 1/3
1/3
1
2/3
0
0
0
0
0
2/3
0
0
0
0
0
0
0
2/3
0 .
0
2/3
0
0
1.4. This transition
CME308: Assignment 2- Solution
Problem 1: Suppose that X1 , . . . , Xn are i.i.d. uniformly distributed in the interval [0, ]. Here is the
unknown parameter to be estimated from the sample.
1. Find the MLE of , denoted by .
2. Prove that converges to in p
CME308: Assignment 1
Due: Apr 19, 2013
Problem 1: Some useful results.
This problem will cover some useful convergence results.
If cfw_Xn are a series of random variables, we have the following denition.
i=1
p
We say Xn converges to X in probability, de
CME308: Assignment 2
Due: Apr 25, 2014
You can give the homeworks to me in class, you can put them in my mailbox in the math department
(by the elevator, rst oor), or slip them under my door 380-383V. You can also email me the pdf of your
solutions if you
1
ISyE 6761 Fall 2012
Homework #4 Solutions
1. Given a nite aperiodic irreducible MC, prove that for some n, all terms of Pn are
positive.
Solution: Since the MC is aperiodic, for every state i, there exists N (i) such that
(n)
Pii > 0 whenever n N (i). S
1
ISyE 6761 Fall 2012
Homework #5 Solutions (Revised 12/9/12)
1. For a renewal process, let A(t) be the age at time t. Prove that if the interarrival
mean < , then w.p.1, A(t)/t 0 as t .
Solution: Note that
t SN (t)
SN (t)
SN (t) N (t)
A(t)
=
= 1
= 1
.
t
1
ISyE 6761 Fall 2012
Homework #6 Solutions
1. The time T required to repair a machine is Exp(2).
(a) Whats the probability that a repair time exceeds 1/2 hour?
Solution: Since T Exp(2), we have
2e2t dt = e1 .
P(T > 1/2) =
1/2
(b) Whats the probability th
1
ISyE 6761 Fall 2012
Homework #1 Solutions (revised 10/6/12)
1. The probability of winning on a single toss of the dice is p. Player A starts,
and if he fails, he passes the dice to B, who then attempts to win on her toss.
They continue tossing back and
1
ISyE 6761 Fall 2012
Homework #2 Solutions
1. The joint p.m.f. of X and Y is
f (x, y)
y=1
y=2
y=3
x=1 x=2 x=3
1/9
1/3
1/9
1/9
0
1/18
0
1/6
1/9
(a) Find E[X|Y = y] for y = 1, 2, 3.
(b) Find E[E[X|Y ].
(c) Are X and Y independent?
Solution: (a) By denition
CME308: Assignment 1
Due: Apr 18, 2014
Problem 1: Some useful results.
This problem will cover some useful convergence results.
If cfw_Xn are a series of random variables, we have the following denition.
i=1
p
We say Xn converges to X in probability, de
CME308: Assignment 1
Due: Apr 18, 2014
You can give the homeworks to me in class, you can put them in my mailbox in the math department
(by the elevator, rst oor), or slip them under my door 380-383V. You can also email me the pdf of your
solutions if you
CME308: Assignment 1
Due: Apr 19, 2013
Problem 1: Some useful results.
This problem will cover some useful convergence results.
If cfw_Xn are a series of random variables, we have the following denition.
i=1
p
We say Xn converges to X in probability, de
Math 285, Spring 2014
Homework 4 Solutions
2.12. For the Galton-Watson branching process with ospring generating function (s) = (1+s+
s2 )/3 we are asked to nd 1 qn for n = 20, 100, 200, 1000, 1500, 2000, 5000. Here qn = P[Xn = 0]
is the probability of ex
Math 285, Spring 2011
Homework 5 Solutions
1. (Exercise 4.9 from the text)
(a) Assuming that Xn > 0,
2
E[f (Xn+1 )|Xn ] = E[Xn+1 |Xn ] =
(Xn + 1)2 + (Xn 1)2
2
2
= Xn + 1 > Xn = f (Xn ).
2
Thus, if the random walker is in a strictly positive state, there i
Problem #8, Chapter 4 of HPS
Peng pointed out to me that the assigned Problem #9 relies heavily on the
unassigned problem #8 and that the wording of #8 is a little ambiguous.
The comments below will try to unravel the ambiguity while giving a
complete sol
Review of Elementary Probability Theory
Math 495, Spring 2011
0.1 Probability spaces
0.1.1 Definition. A probability space is a triple (H,T,P) where
(3 H is a non-empty set
33 T is a collection of subsets of H for which H T, the
complement A- HA =cfw_= H:
Math 495, Spring 2011
Statement of Theorem from Homework#2
Hypothesis: S is a nite state space and P is the S X S transition
matrix for an Suvaiued Markov chain. We denote by V1 the +1 eigenspace
of the iinear transformation R? :RSarlRS dened by
(RPfliy)
MATH 495, Spring, 2011
Hints on Problem 1.15 (Lawler)
As mentioned in class, Lawler follows the practice common in probability for
using P as the universal symbol for a probability measure with E(Y) (or lE(Y) as the
universal symbol for the expected. (or
MATH 495, Spring, 2011
Homework Assignment # (
DUE DATE: Friday, April 1 (turn in during class or put under Prof. Wilson's office
door before 5:00 p.m. on Friday)
Read Chapter 4 of HPS (I think you'll find it fairly straightforward), then do Problems 2-6