IBBA
Lecture_1_II-2

# Lecture_1_II-2 - STOCHASTIC MODELS LECTURE 1 PART II MARKOV...

• Notes
• 47

This preview shows pages 1–13. Sign up to view the full content.

STOCHASTIC MODELS LECTURE 1 PART II MARKOV CHAINS Nan Chen MSc Program in Financial Engineering The Chinese University of Hong Kong (ShenZhen) Sept. 16, 2015

This preview has intentionally blurred sections. Sign up to view the full version.

Outline 1. Long-Term Behavior of Markov Chains 2. Mean-Time Spent in Transient States 3. Branching Processes
1.5 LONG-RUN PROPORTIONS AND LIMITING PROBABILITIES

This preview has intentionally blurred sections. Sign up to view the full version.

Long-Run Proportions of MC In this section, we will study the long-term behavior of Markov chains. Consider a Markov chain Let denote the long-run proportion of time that the Markov chain is in state , i.e., j j j lim n  # 1 i n : X i j n .
Long-Run Proportions of MC (Continued) A simple fact is that, if a state is transient, the corresponding For recurrent states, we have the following result: Let , the number of transitions until the Markov chain makes a transition into the state Denote to be its expectation, i.e. . j 0. j N j min k 0 : X k j j . m j m j E [ N j | X 0 j ]

This preview has intentionally blurred sections. Sign up to view the full version.

Long-Run Proportions of MC (Continued) Theorem: If the Markov chain is irreducible and recurrent, then for any initial state j 1 m j .
Positive Recurrent and Null Recurrent Definition: we say a state is positive recurrent if and say that it is null recurrent if It is obvious from the previous theorem, if state is positive recurrent, we have m j   ; m j   . j j 0.

This preview has intentionally blurred sections. Sign up to view the full version.

How to Determine ? Theorem: Consider an irreducible Markov chain. If the chain is also positive recurrent, then the long-run proportions of each state are the unique solution of the equations: If there is no solution of the preceding linear equations, then the chain is either transient or null recurrent and all j j i p ij i , j j j 1. j 0.
Example I: Rainy Days in Shenzhen Assume that in Shenzhen, if it rains today, then it will rain tomorrow with prob. 60%; and if it does not rain today, then it will rain tomorrow with prob. 40%. What is the average proportion of rainy days in Shenzhen?

This preview has intentionally blurred sections. Sign up to view the full version.

Example I: Rainy Days in Shenzhen (Solution) Modeling the problem as a Markov chain: Let and be the long-run proportions of rainy and no-rain days. We have P 0.6 0.4 0.4 0.6 . 0 1 0 0.6 0 0.4 1 ; 1 0.4 0 0.6 1 ; 0 1 1. 0 1 1/ 2.
Example II: A Model of Class Mobility A problem of interest to sociologists is to determine the proportion of a society that has an upper-, middle-, and lower-class occupations. Let us consider the transitions between social classes of the successive generations in a family. Assume that the occupation of a child depends only on his or her parent’s occupation.

This preview has intentionally blurred sections. Sign up to view the full version.

Example II: A Model of Class Mobility (Continued) The transition matrix of this social mobility is given by That is, for instance, the child of a middle- class worker will attain an upper-class occupation with prob. 5%, will move down to a lower-class occupation with prob. 25%.
This is the end of the preview. Sign up to access the rest of the document.
• Spring '14
• Markov chain, Shenzhen, Nan Chen

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern