# Stochastic

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ve to pick one of the N i balls in the other urn. The number can also decrease by 1 with probability i/N . In symbols, we have computed that the transition probability is given by p(i, i + 1) = (N i)/N, p(i, i 1) = i/N for 0 i N with p(i, j ) = 0 otherwise. When N = 4, for example, the matrix is 0 0 1 2 3 4 0 1/4 0 0 0 1 2 3 4 1 0 0 0 0 3/4 0 0 2/4 0 2 /4 0 0 3/4 0 1 /4 0 0 1 0 3 1.1. DEFINITIONS AND EXAMPLES In the ﬁrst two examples we began with a verbal description and then wrote down the transition probabilities. However, one more commonly describes a Markov chain by writing down a transition probability p(i, j ) with (i) p(i, j ) 0, since they are probabilities. P (ii) j p(i, j ) = 1, since when Xn = i, Xn+1 will be in some state j . The equation in (ii) is read “sum p(i, j ) over all possible values of j .” In words the last two conditions say: the entries of the matrix are nonnegative and each ROW of the matrix sums to 1. Any matrix with properties (i) and (ii) gives rise to a Markov chain,...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online