This preview shows page 1. Sign up to view the full content.
Unformatted text preview: The possible combinations
are:
Time
C D to C C e.g. brownian motion  not covered D C Waiting time of nth arrival in a queue D f0 statespace D to discuss discuss In discrete state space, the stochastic process is called a chain with values denoted, e.g.,
1 ::: mg. S = 2. Discretetime Markov chain
A stochastic process fAn n 0g is called a Markov chain if for every xi 2 S , we have PrfA n = x jA ,1 = x ,1 :::A0 = x0 g = PrfA
n n n n = x jA , 1 = x ,1 g:
n n n (In this deﬁnition, we use time to be discrete.) What this means is that for a Markov chain, the
probability at time n depends only on the previous state and nothing before that. This is known as the
memoryless property of a Markov chain.
Now, what is the probability of the process being in state j given that it was in state i in the preceding
time? This is the transition probability from state i to j . It is written as: CS 522, v 0.94, d.medhi, W’99 5 pij = PrfAn = j jAn,1 = ig
Given, you are at state i at time n , 1, the probabilities of moving to all states (in the next time slot)
must add up to 1, i.e. 1
X pij = 1
j =0 for ea...
View
Full
Document
This document was uploaded on 03/19/2014 for the course CS 6030 at Western Michigan.
 Fall '08
 Staff
 Computer Networks

Click to edit the document details