{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

pa3 - CS221 Problem Set#3 Programming Part 1 CS 221 Autumn...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
CS221 Problem Set #3 Programming Part 1 CS 221, Autumn 2007 Problem Set #3 Programming Part — MDPs Due: 11:59pm, Thursday November 15. 1 Overview In this programming assignment, you’ll use the value iteration algorithm to find a policy for driving a car on a loose-surface road. The car will begin facing down the road in one direction, travelling at a fixed speed. Your policy will need to learn to spin the car around and then drive off in the opposite direction as quickly as possible. You are provided with a simple simulator of the car that, given a (real-valued) state vector, will simulate forward in time using actions chosen by your policy. Since the simulator operates on continuous-valued states and actions, we’ve discretized the state-space for you. In this discretized space, you’ll use the simulator to collect data, building up a probabilistic model of the car’s dynamics. Once you have such a model, you’ll compute the value function (using value iteration) for the discrete MDP, and finally compute the optimal policy from the value function. You will then save the policy to a file. These files can be loaded and viewed in the provided graphical simulator so that you can see how your car performs. You can copy the code from the cs221 directory on AFS: cp -r /afs/ir/class/cs221/code/pa3 . Or download the tar-zipped file from the course website. The graphical simulator requires GLUT. We recommend that you work on the myth workstations, since these are known to have GLUT correctly installed. Please keep all of your code confined to main.cpp. The code necessary to complete this assignment is not very long – you shouldn’t need any other files. 2 The Continuous State Model The road on which the car is driving defines the X axis of the coordinate system (which you can think of as East). The Y-axis points to the left side of the road (North) if the car is facing in the positive X direction. Angles are measured counter-clockwise from the X axis, so if the car’s heading is 0, then it is facing directly down the road, and if its heading is π , then it is facing
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
CS221 Problem Set #3 Programming Part 2 down the road in the opposite direction. (1) The state of the car, s , can be described by an array of 4 numbers: s = [ y, v x , v y , θ ]. y is the Y position of the car relative to the road. v x and v y are the car’s velocity along the X and Y axes, and θ is the car’s heading. In each state, one may choose an action for the car to execute. The action is an array of 2 numbers: a = [ α, w ]. α is the steering angle (i.e., the angle of the tires relative to the car’s centerline – positive angles cause a left turn, negative angles cause a right turn), and w is the “velocity” of the car’s wheels. As an example, if a = [0 , 10], then the car will drive straight, with the wheels spinning at a rate such that they propel the car at 10m/s (i.e., they spin at a rate of ω = 10 /r radians/sec, where r is the radius of the wheel). The car has an infinite amount of torque, and will immediately drive the wheels at whatever velocity is commanded. Choosing w 0 will resulting in applying the car’s brakes – you cannot drive backward.
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}