This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: IEOR 4106: Professor Whitt Lecture Notes, Tuesday, February 10 More Markov chains 1. Cars and Trucks . We discussed Exercise 4.30 on cars and trucks , which was in the homework. The problem wording does not actually imply a Markov chain, but if you use a Markov chain, it turns out that you do get the right answer (whether or not it is a Markov chain). It is easy to set up the matrix P . Then it is easy to solve π = πP ; remember that one equation is always redundant. So that leaves only a single equation. You get an extra equation by noting that the probabilities (the entries of the vector π ) must sum to 1. But actually the problem formulation does not imply that the system evolves as a Markov chain. Here is a counterexample showing that it is not so: The counterexample is a determin- istic periodic sequence with the period containing 19 vehicles: 4 trucks and 15 cars. A single period of the deterministic sequence looks like: TTCTCTCCCCCCCCCCCCC The entire sequence repeats that pattern over and over again, with no randomness at all: | TTCTCTCCCCCCCCCCCCC | TTCTCTCCCCCCCCCCCCC | TTCTCTCC ......
View Full Document
- Spring '08
- Probability theory, Markov chain, πk vk