This preview shows pages 1–7. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: MIT OpenCourseWare http://ocw.mit.edu 16.323 Principles of Optimal Control Spring 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms . 16.323 Lecture 3 Dynamic Programming Principle of Optimality Dynamic Programming Discrete LQR Figure by MIT OpenCourseWare. Spr 2008 16.323 31 Dynamic Programming DP is a central idea of control theory that is based on the Principle of Optimality: Suppose the optimal solution for a problem passes through some intermediate point ( x 1 ,t 1 ) , then the optimal solution to the same problem starting at ( x 1 ,t 1 ) must be the continuation of the same path. Proof? What would the implications be if it was false? This principle leads to: Numerical solution procedure called Dynamic Programming for solving multistage decision making problems. Theoretical results on the structure of the resulting control law. Texts: Dynamic Programming (Paperback) by Richard Bellman (Dover) Dynamic Programming and Optimal Control (Vol 1 and 2) by D. P. Bertsekas June 18, 2008 Spr 2008 16.323 32 Classical Examples Shortest Path Problems (Bryson figure 4.2.1) classic robot naviga tion and/or aircraft ight path problems Goal is to travel from A to B in the shortest time possible Travel times for each leg are shown in the figure There are 20 options to get from A to B could evaluate each and compute travel time, but that would be pretty tedious Alternative approach: Start at B and work backwards, invoking the principle of optimality along the way. First step backward can be either up (10) or down (11) Consider the travel time from point x Can go up and then down 6 + 10 = 16 June 18, 2008 10 6 7 11 x B Figure by MIT OpenCourseWare. Figure by MIT OpenCourseWare. Spr 2008 16.323 33 Or can go down and then up 7 + 11 = 18 Clearly best option from x is go up, then down, with a time of 16 From principle of optimality, this is best way to get to B for any path that passes through x . Repeat process for all other points, until finally get to initial point shortest path traced by moving in the directions of the arrows. Key advantage is that only had to find 15 numbers to solve this problem this way rather than evaluate the travel time for 20 paths Modest difference here, but scales up for larger problems. If n = number of segments on side (3 here) then: Number of routes scales as (2 n )! / ( n !) 2 Number DP computations scales as ( n + 1) 2 1 June 18, 2008 Figure by MIT OpenCourseWare. Spr 2008 16.323 34 Example 2 Routing Problem [Kirk, page 56] through a street maze Similar problem (minimize cost to travel from c to h ) with a slightly more complex layout Once again, start at end ( h ) and work backwards Can get to h from e , g directly, but there are 2 paths to h from e ....
View
Full
Document
This note was uploaded on 11/07/2011 for the course AERO 16.323 taught by Professor Jonathanhow during the Spring '08 term at MIT.
 Spring '08
 jonathanhow

Click to edit the document details