handoutmpSP06

# handoutmpSP06 - A Primer on Markov Processes Markov...

This preview shows pages 1–2. Sign up to view the full content.

APr imeronMarkovProcesses Markov processes will prove useful to represent uncertainty in dynamic pro- gramming problems. See Ch. 7,8,11 of S-L. Def. Let S be a set and let S be a collection of subsets of S . S is called a σ algebra if: (1) ,S S ;(2) A S implies A c = S \ A S (closed under complementation); and (3) A n S implies n =1 A n S (closed under countable union) By DeMorgan’s law it is also closed under intersection. Ex. S = { Heads,Tails } . Then the power set of S is { , { H } , { T } ,S } . Notice this satis f es all the properties of an algebra. Def. ( S, S ) is called a measurable space if S is a σ -algebra of S . Def. Let ( S, S ) be a measurable space. A measure is a real valued function µ : S R such that: (1) µ ( )=0 ;(2) µ ( A ) 0 for all A S ; and (3) if { A n } is a countable, disjoint sequence of subsets in S ,th en µ ( A n )= P µ ( A n ) . Def. If µ ( S )=1 ,then µ is a probability measure. Ex. µ ( H )= 1 2 . Def. Given a measurable space ( S, S ), a real-valued function f : S R is ( S ) measurable if { s S : f ( s ) a } S for all

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 2

handoutmpSP06 - A Primer on Markov Processes Markov...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online