SP10 cs188 lecture 10 -- MDPs II (2PP)

SP10 cs188 lecture 10 -- MDPs II (2PP) - CS 188 Artificial...

Info iconThis preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
1 CS 188: Artificial Intelligence Spring 2010 Lecture 10: MDPs 2/18/2010 Pieter Abbeel – UC Berkeley Many slides over the course adapted from either Dan Klein, Stuart Russell or Andrew Moore 1 Announcements s P2: Due tonight s W3: Expectimax, utilities and MDPs---out tonight, due next Thursday. s Online book: Sutton and Barto http://www.cs.ualberta.ca/~sutton/book/ebook/the-book.html 2
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 Recap: MDPs s Markov decision processes: s States S s Actions A s Transitions P(s’|s,a) (or T(s,a,s’)) s Rewards R(s,a,s’) (and discount γ ) s Start state s 0 s Quantities: s Policy = map of states to actions s Utility = sum of discounted rewards s Values = expected future utility from a state s Q-Values = expected future utility from a q-state a s s, a s,a,s’ s’ 4 Recap MPD Example: Grid World s The agent lives in a grid s Walls block the agent’s path s The agent’s actions do not always go as planned: s 80% of the time, the action North takes the agent North (if there is no wall there) s 10% of the time, North takes the agent West; 10% East s If there is a wall in the direction the agent would have been taken, the agent stays put s Small “living” reward each step s Big rewards come at the end s Goal: maximize sum of rewards
Background image of page 2
3 Why Not Search Trees? s
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 03/01/2010 for the course COMPUTER S 188 taught by Professor Abbel during the Spring '10 term at Berkeley.

Page1 / 10

SP10 cs188 lecture 10 -- MDPs II (2PP) - CS 188 Artificial...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online