Unformatted text preview: such a way that each contains three balls. We say that the system is in state i , if the ±rst urn contains i white balls. At each step, we draw one ball from each urn and place the ball drawn from the ±rst urn into the second, and conversely with the ball drawn from the second urn. Let X n denote the state of the system after the n th step. Explain why X n is a Markov chain and calculate its transition probability. Classify the states, determine transient and recurrent states, and decide whether the chain is irreducible or aperiod. Problem 5: Let X n be a sequence of random variables with values in Z , such that X n = X 2 n1 . Is X n a Markov chain? If so, determine the transition matrix and ±nd all recurrent states....
View
Full Document
 Spring '08
 DENKER
 Probability, Stochastic Modeling, Probability theory, Stochastic process, Markov chain, $2

Click to edit the document details