Lecture 12

# Lecture 12

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ch i: A Markov chain is called temporally homogeneous if : PrfAn = j jAn,1 = ig = PrfAn+m = j jAn+m,1 = ig: The transition probability is then denoted by pij . For all possible values of i j , one can denote he the transition probability as a matrix with elements pij . The transition in n-step is given by pij n = PrfAn = j jA0 = ig: Since a Markov chain has stationary transition probabilities, we have pij n = PrfAm+n = j jAm = ig for all m  0 and n 0: 3. Continuous-time, Markov Chain Let fX t 0  t 1g be a Markov process with countable state space S = f0 1 2 :::g over continuous time-space t. For, example, X t can be the number of customers in the system at time t. For continuous time, discrete space (Markov chains) the transition probability is denoted by, pij t = Prfxt + u = j jxu = ig t 0 i j 2 S X pij t = 1 Note, j Now, we can consider all the possible cases for i in matrix notation, P t := pij t := CS 522, v 0.94, d.medhi, W’99 for each i: j at time t giving us a matrix of information. Thus, transition...
View Full Document

## This document was uploaded on 03/19/2014 for the course CS 6030 at Western Michigan.

Ask a homework question - tutors are online