This preview shows page 1. Sign up to view the full content.
Unformatted text preview: such a way that each contains three balls. We say that the system is in state i , if the rst urn contains i white balls. At each step, we draw one ball from each urn and place the ball drawn from the rst urn into the second, and conversely with the ball drawn from the second urn. Let X n denote the state of the system after the n th step. Explain why X n is a Markov chain and calculate its transition probability. Classify the states, determine transient and recurrent states, and decide whether the chain is irreducible or aperiod. Problem 5: Let X n be a sequence of random variables with values in Z , such that X n = X 2 n-1 . Is X n a Markov chain? If so, determine the transition matrix and nd all recurrent states....
View Full Document
- Spring '08
- Stochastic Modeling