Lecture_1_II-2 - STOCHASTIC MODELS LECTURE 1 PART II MARKOV...

Info icon This preview shows pages 1–13. Sign up to view the full content.

View Full Document Right Arrow Icon
STOCHASTIC MODELS LECTURE 1 PART II MARKOV CHAINS Nan Chen MSc Program in Financial Engineering The Chinese University of Hong Kong (ShenZhen) Sept. 16, 2015
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Outline 1. Long-Term Behavior of Markov Chains 2. Mean-Time Spent in Transient States 3. Branching Processes
Image of page 2
1.5 LONG-RUN PROPORTIONS AND LIMITING PROBABILITIES
Image of page 3

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Long-Run Proportions of MC In this section, we will study the long-term behavior of Markov chains. Consider a Markov chain Let denote the long-run proportion of time that the Markov chain is in state , i.e., j j j lim n  # 1 i n : X i j n .
Image of page 4
Long-Run Proportions of MC (Continued) A simple fact is that, if a state is transient, the corresponding For recurrent states, we have the following result: Let , the number of transitions until the Markov chain makes a transition into the state Denote to be its expectation, i.e. . j 0. j N j min k 0 : X k j j . m j m j E [ N j | X 0 j ]
Image of page 5

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Long-Run Proportions of MC (Continued) Theorem: If the Markov chain is irreducible and recurrent, then for any initial state j 1 m j .
Image of page 6
Positive Recurrent and Null Recurrent Definition: we say a state is positive recurrent if and say that it is null recurrent if It is obvious from the previous theorem, if state is positive recurrent, we have m j   ; m j   . j j 0.
Image of page 7

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
How to Determine ? Theorem: Consider an irreducible Markov chain. If the chain is also positive recurrent, then the long-run proportions of each state are the unique solution of the equations: If there is no solution of the preceding linear equations, then the chain is either transient or null recurrent and all j j i p ij i , j j j 1. j 0.
Image of page 8
Example I: Rainy Days in Shenzhen Assume that in Shenzhen, if it rains today, then it will rain tomorrow with prob. 60%; and if it does not rain today, then it will rain tomorrow with prob. 40%. What is the average proportion of rainy days in Shenzhen?
Image of page 9

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Example I: Rainy Days in Shenzhen (Solution) Modeling the problem as a Markov chain: Let and be the long-run proportions of rainy and no-rain days. We have P 0.6 0.4 0.4 0.6 . 0 1 0 0.6 0 0.4 1 ; 1 0.4 0 0.6 1 ; 0 1 1. 0 1 1/ 2.
Image of page 10
Example II: A Model of Class Mobility A problem of interest to sociologists is to determine the proportion of a society that has an upper-, middle-, and lower-class occupations. Let us consider the transitions between social classes of the successive generations in a family. Assume that the occupation of a child depends only on his or her parent’s occupation.
Image of page 11

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Example II: A Model of Class Mobility (Continued) The transition matrix of this social mobility is given by That is, for instance, the child of a middle- class worker will attain an upper-class occupation with prob. 5%, will move down to a lower-class occupation with prob. 25%.
Image of page 12
Image of page 13
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern