SPLecture7 - Lecture 7 Mariana Olvera-Cravioto Columbia...

Info icon This preview shows pages 1–8. Sign up to view the full content.

View Full Document Right Arrow Icon
Lecture 7 Mariana Olvera-Cravioto Columbia University [email protected] February 11th, 2015 IEOR 4106, Intro to OR: Stochastic Models Lecture 7 1/18
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
In the long run I Consider once again the example about a random walk on a graph. 1 2 6 7 3 8 5 4 P = 1 / 4 1 / 4 0 0 0 1 / 4 1 / 4 0 1 / 3 0 0 1 / 3 1 / 3 0 0 0 0 0 0 0 0 0 0 1 0 1 / 3 0 0 0 1 / 3 1 / 3 0 0 1 / 4 0 0 1 / 4 1 / 4 0 1 / 4 1 / 4 0 0 1 / 4 1 / 4 0 1 / 4 0 1 / 3 0 0 1 / 3 0 1 / 3 0 0 0 0 1 / 2 0 1 / 2 0 0 0 I Run the program MarkovChain.m in Matlab, which computes the first n powers of P . I Notice anything special? IEOR 4106, Intro to OR: Stochastic Models Lecture 7 2/18
Image of page 2
In the long run I Consider once again the example about a random walk on a graph. 1 2 6 7 3 8 5 4 P = 1 / 4 1 / 4 0 0 0 1 / 4 1 / 4 0 1 / 3 0 0 1 / 3 1 / 3 0 0 0 0 0 0 0 0 0 0 1 0 1 / 3 0 0 0 1 / 3 1 / 3 0 0 1 / 4 0 0 1 / 4 1 / 4 0 1 / 4 1 / 4 0 0 1 / 4 1 / 4 0 1 / 4 0 1 / 3 0 0 1 / 3 0 1 / 3 0 0 0 0 1 / 2 0 1 / 2 0 0 0 I Run the program MarkovChain.m in Matlab, which computes the first n powers of P . I Notice anything special? I P n seems to be converging as n → ∞ ! I Moreover, all the rows of P n look the same! IEOR 4106, Intro to OR: Stochastic Models Lecture 7 2/18
Image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
In the long run... continued I Let X n be the current vertex in the random walk after n steps. I Run the program MarkovChainSim.m in Matlab, which generates { X 0 , X 1 , X 2 , . . . , X n } for a pre-specified initial value X 0 and a given number of steps n . I In addition to the sequence, the program returns a vector of the form: NumberOfVisits = ( N 1 , N 2 , N 3 , N 4 , N 5 , N 6 , N 7 , N 8 ) , where N i is the number of times { X k : 0 k n } is in state i , and RelativeFrequency = 1 n NumberOfVisits . I Notice anything special? IEOR 4106, Intro to OR: Stochastic Models Lecture 7 3/18
Image of page 4
In the long run... continued I Let X n be the current vertex in the random walk after n steps. I Run the program MarkovChainSim.m in Matlab, which generates { X 0 , X 1 , X 2 , . . . , X n } for a pre-specified initial value X 0 and a given number of steps n . I In addition to the sequence, the program returns a vector of the form: NumberOfVisits = ( N 1 , N 2 , N 3 , N 4 , N 5 , N 6 , N 7 , N 8 ) , where N i is the number of times { X k : 0 k n } is in state i , and RelativeFrequency = 1 n NumberOfVisits . I Notice anything special? I The vector RelativeFrequency is very close to the rows of P n ! IEOR 4106, Intro to OR: Stochastic Models Lecture 7 3/18
Image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Aperiodicity and positive recurrence I We say that state i has period d if P n ii = 0 whenever n is not divisible by d , and d is the largest integer with this property. I A state with period 1 is said to be aperiodic . I Aperiodicity is a class property. IEOR 4106, Intro to OR: Stochastic Models Lecture 7 4/18
Image of page 6
Aperiodicity and positive recurrence I We say that state i has period d if P n ii = 0 whenever n is not divisible by d , and d is the largest integer with this property.
Image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 8
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern