This preview shows page 1. Sign up to view the full content.
Unformatted text preview: ve to pick one of the N i balls in the other
urn. The number can also decrease by 1 with probability i/N . In symbols, we
have computed that the transition probability is given by
p(i, i + 1) = (N i)/N, p(i, i 1) = i/N for 0 i N with p(i, j ) = 0 otherwise. When N = 4, for example, the matrix is
0
0
1
2
3
4 0
1/4
0
0
0 1 2 3 4 1
0
0
0
0
3/4
0
0
2/4
0
2 /4
0
0
3/4
0
1 /4
0
0
1
0 3 1.1. DEFINITIONS AND EXAMPLES In the ﬁrst two examples we began with a verbal description and then wrote
down the transition probabilities. However, one more commonly describes a
Markov chain by writing down a transition probability p(i, j ) with
(i) p(i, j ) 0, since they are probabilities.
P
(ii) j p(i, j ) = 1, since when Xn = i, Xn+1 will be in some state j . The equation in (ii) is read “sum p(i, j ) over all possible values of j .” In words
the last two conditions say: the entries of the matrix are nonnegative and each
ROW of the matrix sums to 1.
Any matrix with properties (i) and (ii) gives rise to a Markov chain,...
View
Full
Document
This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).
 Spring '10
 DURRETT
 The Land

Click to edit the document details