Lecture 12

# The possible combinations are time c d to c c eg

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: The possible combinations are: Time C D to C C e.g. brownian motion - not covered D C Waiting time of n-th arrival in a queue D f0 state-space D to discuss discuss In discrete state space, the stochastic process is called a chain with values denoted, e.g., 1 ::: mg. S = 2. Discrete-time Markov chain A stochastic process fAn n  0g is called a Markov chain if for every xi 2 S , we have PrfA n = x jA ,1 = x ,1 :::A0 = x0 g = PrfA n n n n = x jA , 1 = x ,1 g: n n n (In this deﬁnition, we use time to be discrete.) What this means is that for a Markov chain, the probability at time n depends only on the previous state and nothing before that. This is known as the memoryless property of a Markov chain. Now, what is the probability of the process being in state j given that it was in state i in the preceding time? This is the transition probability from state i to j . It is written as: CS 522, v 0.94, d.medhi, W’99 5 pij = PrfAn = j jAn,1 = ig Given, you are at state i at time n , 1, the probabilities of moving to all states (in the next time slot) must add up to 1, i.e. 1 X pij = 1 j =0 for ea...
View Full Document

## This document was uploaded on 03/19/2014 for the course CS 6030 at Western Michigan.

Ask a homework question - tutors are online