MarkovChains

# MarkovChains - ECE 496 SUPPLEMENTARY HANDOUT Spring 2007...

This preview shows pages 1–2. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ECE 496 SUPPLEMENTARY HANDOUT Spring 2007 COUNTABLE-STATE DISCRETE-TIME HOMOGENEOUS MARKOV CHAINS 1. Basic definitions, transition probabilities, etc. Picture a huge panel of colored lights. The panel might feature finitely many lights or countably infinitely many of them. You are looking at the panel and can’t see what’s going on behind it. What you observe is that at each integer time k ≥ 0, exactly one of the lights flashes. What light flashes at what time appears random. Sometimes one light will flash several times in a row. Some lights flash only rarely. Now for the part you can’t observe. Behind each light is an operator. A light flashes if and only if its operator has been told to flash it. At time k = 0, a supervisor tells one of the operators to flash his light. The supervisor might decide which operator goes first by (figuratively) rolling a die or flipping a coin, i.e., by some kind of random choice mechanism. The lucky operator who flashes his light at time k = 0 (let’s call him Adam) then has the job of deciding which operator to direct to flash his light at time k = 1. He makes that decision by rolling his own figurative die. If it’s a strain to visualize many-sided dice or coins, think of each operator as possessing a spinner such as you might use in playing a game of Twister, say. Adam’s random-choice device has a certain probability of fingering each light operator, including Adam himself. So Adam spins his spinner and sees who the k = 1-operator will be, at which point directs her to flash her light at k = 1. The k = 1-operator, let’s call her Brittany, having flashed her light, spins her own spinner to decide who goes at time k = 2. Brittany’s spinner might be quite different from Adam’s. The most important thing about the two spinners is that they are completely independent of each other. The k = 2-operator to whom Brittany gives the “Flash” order has his own independent spinner, which he will use to decide who goes at k = 3. And so on. What you see from your side of the panel is some random sequence of lights flashing. The special way the randomness arises is what makes the light-board a Markov chain. If, for example, the red light flashes at some specific time n , the probability that the blue light flashes at time n + 1 depends only on the structure of the spinner belonging to the operator behind the red light. What light actually flashes at time n + 1 depends only on the outcome of that spinner’s spin. All past history, i.e., the record of what lights have flashed at times up through n- 1, is irrelevant to determining the probability that a given light will flash at time n + 1 given that the red light has flashed at time n . Furthermore, all the operators’ spinners are independent and don’t change over time....
View Full Document

## This note was uploaded on 02/01/2010 for the course ECE 496 taught by Professor Delchamps during the Spring '07 term at Cornell.

### Page1 / 20

MarkovChains - ECE 496 SUPPLEMENTARY HANDOUT Spring 2007...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online