This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: CS221 Problem Set #3 Programming Part 1 CS 221, Autumn 2007 Problem Set #3 Programming Part MDPs Due: 11:59pm, Thursday November 15. 1 Overview In this programming assignment, youll use the value iteration algorithm to find a policy for driving a car on a loose-surface road. The car will begin facing down the road in one direction, travelling at a fixed speed. Your policy will need to learn to spin the car around and then drive off in the opposite direction as quickly as possible. You are provided with a simple simulator of the car that, given a (real-valued) state vector, will simulate forward in time using actions chosen by your policy. Since the simulator operates on continuous-valued states and actions, weve discretized the state-space for you. In this discretized space, youll use the simulator to collect data, building up a probabilistic model of the cars dynamics. Once you have such a model, youll compute the value function (using value iteration) for the discrete MDP, and finally compute the optimal policy from the value function. You will then save the policy to a file. These files can be loaded and viewed in the provided graphical simulator so that you can see how your car performs. You can copy the code from the cs221 directory on AFS: cp -r /afs/ir/class/cs221/code/pa3 . Or download the tar-zipped file from the course website. The graphical simulator requires GLUT. We recommend that you work on the myth workstations, since these are known to have GLUT correctly installed. Please keep all of your code confined to main.cpp. The code necessary to complete this assignment is not very long you shouldnt need any other files. 2 The Continuous State Model The road on which the car is driving defines the X axis of the coordinate system (which you can think of as East). The Y-axis points to the left side of the road (North) if the car is facing in the positive X direction. Angles are measured counter-clockwise from the X axis, so if the cars heading is 0, then it is facing directly down the road, and if its heading is , then it is facing CS221 Problem Set #3 Programming Part 2 down the road in the opposite direction. (1) The state of the car, s , can be described by an array of 4 numbers: s = [ y, v x , v y , ]. y is the Y position of the car relative to the road. v x and v y are the cars velocity along the X and Y axes, and is the cars heading. In each state, one may choose an action for the car to execute. The action is an array of 2 numbers: a = [ , w ]. is the steering angle (i.e., the angle of the tires relative to the cars centerline positive angles cause a left turn, negative angles cause a right turn), and w is the velocity of the cars wheels. As an example, if a = [0 , 10], then the car will drive straight, with the wheels spinning at a rate such that they propel the car at 10m/s (i.e., they spin at a rate of = 10 /r radians/sec, where r is the radius of the wheel). The car has an infinite amountis the radius of the wheel)....
View Full Document
This note was uploaded on 11/30/2009 for the course CS 221 taught by Professor Koller,ng during the Winter '09 term at Stanford.
- Winter '09
- Artificial Intelligence