445Lecture6.pdf - Lecture 6 Mariana Olvera-Cravioto UNC...

This preview shows 1 out of 8 pages.

Lecture 6 Mariana Olvera-Cravioto UNC Chapel Hill [email protected] January 29th, 2019 STOR 445, Introduction to Stochastic Modeling Lecture 6 1/16
Image of page 1

Subscribe to view the full document.

Example: A random walk on a graph I Consider the following undirected graph. I All edges can be travelled in any direction. 1 2 6 7 3 8 5 4 STOR 445, Introduction to Stochastic Modeling Lecture 6 2/16
Image of page 2
A random walk on a graph... continued I Suppose we want to travel through the states in the graph starting from some initial vertex. I Now suppose that on each time step we choose randomly, with the same probability, to move to one of the neighbors of the current vertex. I Let X n denote the position after n steps. I Then X n is a Markov chain. STOR 445, Introduction to Stochastic Modeling Lecture 6 3/16
Image of page 3

Subscribe to view the full document.

A random walk on a graph... continued I Suppose we want to travel through the states in the graph starting from some initial vertex. I Now suppose that on each time step we choose randomly, with the same probability, to move to one of the neighbors of the current vertex. I Let X n denote the position after n steps. I Then X n is a Markov chain. I Its one step transition matrix is given by: 1 2 3 4 5 6 7 8 P = 1 2 3 4 5 6 7 8 1 / 4 1 / 4 0 0 0 1 / 4 1 / 4 0 1 / 3 0 0 1 / 3 1 / 3 0 0 0 0 0 0 0 0 0 0 1 0 1 / 3 0 0 0 1 / 3 1 / 3 0 0 1 / 4 0 0 1 / 4 1 / 4 0 1 / 4 1 / 4 0 0 1 / 4 1 / 4 0 1 / 4 0 1 / 3 0 0 1 / 3 0 1 / 3 0 0 0 0 1 / 2 0 1 / 2 0 0 0 STOR 445, Introduction to Stochastic Modeling Lecture 6 3/16
Image of page 4
A random walk on a graph... continued I If we start at vertex 1, what is the probability that we will be in state 8 after five steps? STOR 445, Introduction to Stochastic Modeling Lecture 6 4/16
Image of page 5

Subscribe to view the full document.

A random walk on a graph... continued I If we start at vertex 1, what is the probability that we will be in state 8 after five steps? I To answer the question we compute P 5 : P 5 = 10 - 2 17 . 36 15 . 63 1 . 17 11 . 77 12 . 53 20 . 05 15 . 65 5 . 83 20 . 83 8 . 34 4 . 89 17 . 52 19 . 62 13 . 81 11 . 26 3 . 72 4 . 69 14 . 67 1 . 56 2 . 86 13 . 80 15 . 71 5 . 03 41 . 67 15 . 70 17 . 52 0 . 95 9 . 67 11 . 36 21 . 36 16 . 44 7 . 00 12 . 53 14 . 71 3 . 45 8 . 52 15 . 11 17 . 71 11 . 42 16 . 55 20 . 05 10 . 36 3 . 93 16 . 02 17 . 71 15 . 45 12 . 85 3 . 63 20 . 86 11 . 26 1 . 68 16 . 44 15 . 23 17 . 14 13 . 35 4 . 04 11 . 67 5 . 58 20 . 83 10 . 50 33 . 10 7 . 26 6 . 05 5 . 01 I And we look at its (1 , 8) component, i.e., p (5) 1 , 8 = ( P 5 ) 1 , 8 = 0 . 0583 STOR 445, Introduction to Stochastic Modeling Lecture 6 4/16
Image of page 6
The hitting time of a state I For any set A ⊂ S , we will write P x ( A ) = P ( A | X 0 = x ) , i.e., the probability that event A happens given that the Markov chain { X n : n 0 } started at X 0 = x . I In particular, we will be interested in the time of the first return to state y , i.e., T y = min { n 1 : X n = y } , and consider the probabilities ρ y,y = P y ( T y < ) . I It sounds reasonable that if ρ y,y > 0 , then the probability that we return to state y , say k times, would also be positive, since on each return is as if we start all over again.
Image of page 7

Subscribe to view the full document.

Image of page 8
You've reached the end of this preview.
  • Spring '07
  • AndrewB.Nobel
  • Markov chain

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern