416-5 - such a way that each contains three balls. We say...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
Denker Spring 2010 416 Stochastic Modeling - Assignment 5 Due Date: Monday, February 22, 2010 NOTE: All problems from assignemnts 1–5 may appear in the Frst exam on ±eb 26!!! Problem 1: (Example 4.6) A gambler beds at each play one third of his present fortune rounded to the next full $-amount. For example, if his fortune is $4, he beds $2. At each play he wins twice the amount of his bed or he loses it. If he is broken he stops playing and if he owns a fortune of $ N , he stops as well. Model the gambler’s fortune by a Markov chain. Classify the states and ±nd absorbing states. Problem 2: (Example 4.13) Consider the matrix 0 0 1 / 3 0 1 / 3 1 / 3 0 3 / 4 0 0 0 1 / 4 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 3 / 4 1 / 4 0 1 / 6 0 0 0 5 / 6 Which states are transient and which are recurrent? Problem 3: (Problem 6) Let P = p p 1 - p 1 - p p P Find (by mathematical induction) P n . Problem 4: (Problem 1) Three white and three black balls are distributed in two urns in
Background image of page 1
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: such a way that each contains three balls. We say that the system is in state i , if the rst urn contains i white balls. At each step, we draw one ball from each urn and place the ball drawn from the rst urn into the second, and conversely with the ball drawn from the second urn. Let X n denote the state of the system after the n th step. Explain why X n is a Markov chain and calculate its transition probability. Classify the states, determine transient and recurrent states, and decide whether the chain is irreducible or aperiod. Problem 5: Let X n be a sequence of random variables with values in Z , such that X n = X 2 n-1 . Is X n a Markov chain? If so, determine the transition matrix and nd all recurrent states....
View Full Document

Ask a homework question - tutors are online