Lecture_1_II-2 - STOCHASTIC MODELS LECTURE 1 PART II MARKOV...

Info icon This preview shows pages 1–13. Sign up to view the full content.

STOCHASTIC MODELS LECTURE 1 PART II MARKOV CHAINS Nan Chen MSc Program in Financial Engineering The Chinese University of Hong Kong (ShenZhen) Sept. 16, 2015
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

Outline 1. Long-Term Behavior of Markov Chains 2. Mean-Time Spent in Transient States 3. Branching Processes
Image of page 2
1.5 LONG-RUN PROPORTIONS AND LIMITING PROBABILITIES
Image of page 3

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

Long-Run Proportions of MC In this section, we will study the long-term behavior of Markov chains. Consider a Markov chain Let denote the long-run proportion of time that the Markov chain is in state , i.e., j j j lim n  # 1 i n : X i j n .
Image of page 4
Long-Run Proportions of MC (Continued) A simple fact is that, if a state is transient, the corresponding For recurrent states, we have the following result: Let , the number of transitions until the Markov chain makes a transition into the state Denote to be its expectation, i.e. . j 0. j N j min k 0 : X k j j . m j m j E [ N j | X 0 j ]
Image of page 5

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

Long-Run Proportions of MC (Continued) Theorem: If the Markov chain is irreducible and recurrent, then for any initial state j 1 m j .
Image of page 6
Positive Recurrent and Null Recurrent Definition: we say a state is positive recurrent if and say that it is null recurrent if It is obvious from the previous theorem, if state is positive recurrent, we have m j   ; m j   . j j 0.
Image of page 7

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

How to Determine ? Theorem: Consider an irreducible Markov chain. If the chain is also positive recurrent, then the long-run proportions of each state are the unique solution of the equations: If there is no solution of the preceding linear equations, then the chain is either transient or null recurrent and all j j i p ij i , j j j 1. j 0.
Image of page 8
Example I: Rainy Days in Shenzhen Assume that in Shenzhen, if it rains today, then it will rain tomorrow with prob. 60%; and if it does not rain today, then it will rain tomorrow with prob. 40%. What is the average proportion of rainy days in Shenzhen?
Image of page 9

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

Example I: Rainy Days in Shenzhen (Solution) Modeling the problem as a Markov chain: Let and be the long-run proportions of rainy and no-rain days. We have P 0.6 0.4 0.4 0.6 . 0 1 0 0.6 0 0.4 1 ; 1 0.4 0 0.6 1 ; 0 1 1. 0 1 1/ 2.
Image of page 10
Example II: A Model of Class Mobility A problem of interest to sociologists is to determine the proportion of a society that has an upper-, middle-, and lower-class occupations. Let us consider the transitions between social classes of the successive generations in a family. Assume that the occupation of a child depends only on his or her parent’s occupation.
Image of page 11

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

Example II: A Model of Class Mobility (Continued) The transition matrix of this social mobility is given by That is, for instance, the child of a middle- class worker will attain an upper-class occupation with prob. 5%, will move down to a lower-class occupation with prob. 25%.
Image of page 12
Image of page 13
This is the end of the preview. Sign up to access the rest of the document.
  • Spring '14
  • Markov chain, Shenzhen, Nan Chen

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern