EE604- Stochastic Processes # 1
Fall 2013
This problem set should serve as a diagnostic problem set. It covers the background in
probability and random variables that is required.
Problem 1: A random variable X() takes non-negative values and has the prob

Random Variables
79
i nor i + 1 has yet been visited, it follows that i will be the last node visited if and
only if i + 1 is visited before i. This is so because in order to visit i + 1 before i
the particle will have to visit all the nodes on the counte

398
Introduction to Probability Models
Thus, if we let n be a power of 2, say, n = 2k , then we can approximate P(t) by raising
the matrix M = I + Rt/n to the nth power, which can be accomplished by k matrix
multiplications (by rst multiplying M by itself

'Fi.
TM 15'? draw :5 rim. armed acre-u. W h Parsi-
ptmo a monk, am a. 5rd: emu A. is serwut
NI Lupin mt- mam, away +.-.1 a, M,
(We? F: safeties! 3+ La LL cfw_~me In) M W
+LL ramwg Mk bat? :5 WM a M! 61 019+
M- Tm mrmmdrj em, gunk swam
NM'uj L1 awful, wX a

Random Variables
79
i nor i + 1 has yet been visited, it follows that i will be the last node visited if and
only if i + 1 is visited before i. This is so because in order to visit i + 1 before i
the particle will have to visit all the nodes on the counte

Markov Chains
261
Exercises
*1. Three white and three black balls are distributed in two urns in such a way that
each contains three balls. We say that the system is in state i,i = 0, 1, 2, 3, if the
rst urn contains i white balls. At each step, we draw o

Markov Chains
261
Exercises
*1. Three white and three black balls are distributed in two urns in such a way that
each contains three balls. We say that the system is in state i,i = 0, 1, 2, 3, if the
rst urn contains i white balls. At each step, we draw o

468
Introduction to Probability Models
Dividing through by h gives
G(x + h) G(x)
1
=
h
h
x+h
t(x + h y)k(y) dy
x
x
+
0
t(x + h y) t(x y)
k(y) dy
h
Letting h 0 gives the result
x
G (x) = t(0) k(x) +
t (x y) k(y) dy
0
Exercises
1. Is it true that
(a) N (t)

338
Introduction to Probability Models
Exercises
1. The time T required to repair a machine is an exponentially distributed random
variable with mean 21 (hours).
(a) What is the probability that a repair time exceeds 21 hour?
(b) What is the probability t

14
Introduction to Probability Models
Exercises
1. A box contains three marbles: one red, one green, and one blue. Consider an
experiment that consists of taking one marble from the box then replacing it in
the box and drawing a second marble from the box

Conditional Probability and Conditional Expectation
163
Regard p as xed, and call N an NB(r ) random variable. The random variable M 1
has probability mass function
(n + 1)Pcfw_N = n + 1
E[N ]
(n + 1) p n + r
=
pr (1 p)n+1
r (1 p) r 1
(n + r )! r +1
p (1

Conditional Probability and Conditional Expectation
163
Regard p as xed, and call N an NB(r ) random variable. The random variable M 1
has probability mass function
(n + 1)Pcfw_N = n + 1
E[N ]
(n + 1) p n + r
=
pr (1 p)n+1
r (1 p) r 1
(n + r )! r +1
p (1

398
Introduction to Probability Models
Thus, if we let n be a power of 2, say, n = 2k , then we can approximate P(t) by raising
the matrix M = I + Rt/n to the nth power, which can be accomplished by k matrix
multiplications (by rst multiplying M by itself

468
Introduction to Probability Models
Dividing through by h gives
G(x + h) G(x)
1
=
h
h
x+h
t(x + h y)k(y) dy
x
x
+
0
t(x + h y) t(x y)
k(y) dy
h
Letting h 0 gives the result
x
G (x) = t(0) k(x) +
t (x y) k(y) dy
0
Exercises
1. Is it true that
(a) N (t)

University of Waterloo
Department of Electrical and Computer Engineering
ECE 604 Stochastic Processes
Midterm Examination
11:30am to 1:00pm, October 29, 2012
Time allowed: 90 minutes.
Instructor: W. Zhuang
Closed book exam. A formula sheet (11 in x 8.5 in

ECE604- Stochastic Processes # 2
Fall 2013
Do all problems. Some problems extend the theoretical results we have developed in class. I have
indicated the implications of the result.
Problem 1: Let X be a non-negative integer valued r.v. with moment genera

ECE 604- PSET 2 Solution
1.
1) By denition g(z) = pn z n where pn = P (X = n). Therefore for z [0, 1] we have:
n=0
g (z) = npn z n1 0 and is therefore non-decreasing and
n=1
n(n 1)z n2 pn 0
g (z) =
n=2
and is therefore convex.
2) For g (z) to be 0 for som

ECE604- Stochastic Processes Problem Set # 3
Fall 2013
Due Monday Oct. 27, 2013
Do all problems. This problem set is essentially computational. You should get used to perform standard computations using Jacobians, conditional probabilities, expectations,

EE604- Stochastic Processes Problem Set # 5
Fall 2013
These are some problems on SLLN, and second-order processes.
1.
a) Show
W t is a standard Brownian motion process.
b) Let cfw_Xt ; < t < be a 0 mean Gaussian process with E[Xt Xs ] = e|ts| . Express
X

ECE 604- PSET 3 Solution
1. First note that E[ Sn ] = 1 = n
Sn
n
Xi
i=1 E[ Sn ]
Therefore
E[
Sm
]=
Sn
Xi
1
which that E[ Sn ] = n .
m
E[
i=1
Xi
Xi
m
] = mE[ ] =
Sm
Sn
n
1
Also (from the previous problem set) E[Xi |Sn ] = n Sn and therefore E[Sm |Sn ] =
No

ECE 604- PSET 4 Solution
Answers to selected problems only given in detail. Routine problems are not worked out in detail
1. This problem basically shows that any Gauss-Markov process can be viewed as a time-changed
Brownian motion.
a)To show that Xt = W

ECE604- Stochastic Processes
Problem set # 6 Fall 2013
This is the nal problem set. These problems deal with various aspects of Markov chains
1. Let cfw_Xn be an irreducible, homogeneous Markov chain with transition probabilities
pi,j ; i, j E E.
a) Show

ECE 604- PSET 5 Solution
1.
a) Dene Yn = XN n for some N xed. It can be taken to be 0 wlog.
We need to show IPcfw_Yn+1 A|Y0 , Y1 , , Yn = IPcfw_Yn+1 A|Yn .
By abuse of notation:
IP(Yn+1 A, Y0 , Y1 , , Yn )
IP(Y0 , , Yn )
IP(Yn+1 A|Y0 , , Yn ) =
IP(X(n+1)

Assigment Problems on Convergence
1. Let the sequence of random variables Xn (s) consist of independent equiprobable Bernoulli random variables, that,
P [Xn (s) = 0] = 0.5 = P [Xn (s) = 1].
Does this random sequence converge?
2. An urn contains 2 black ba

14
Introduction to Probability Models
Exercises
1. A box contains three marbles: one red, one green, and one blue. Consider an
experiment that consists of taking one marble from the box then replacing it in
the box and drawing a second marble from the box

338
Introduction to Probability Models
Exercises
1. The time T required to repair a machine is an exponentially distributed random
variable with mean 21 (hours).
(a) What is the probability that a repair time exceeds 21 hour?
(b) What is the probability t