02faex1 - 632 Introduction to Stochastic Processes Fall...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
632 Introduction to Stochastic Processes Fall 2002 Midterm Exam I Instructions: Hand in problem no. 1 for 60 points, and one other prob- lem for 40 points. Show calculations and justify non-obvious statements for full credit. General notation: P x ( A ) is the probability of the event A when the chain starts in state x , P μ ( A ) the probability when the initial state is random with distribution μ . T y = min { n 1 : X n = y } is the first time after 0 that the chain visits y , or if no visit to y ever happens. ρ x,y = P x ( T y < ) is the probability that the chain visits y some time after time 0, given that it started at x . N ( y ) is the number of visits to state y , not counting a possible visit at time 0. 1. Fix a constant α (0 , 1), and consider the Markov chain on the state space S = { 1 , 2 , 3 , 4 , 5 , 6 } with transition matrix P = 0 0 1 - α α 0 0 0 0 0 0 1 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 1 - α 0 α 0 (a) Draw the arrow diagram for the Markov chain. Classify the states
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 08/11/2008 for the course MATH 632 taught by Professor Seppalainen during the Spring '07 term at Wisconsin.

Page1 / 2

02faex1 - 632 Introduction to Stochastic Processes Fall...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online