This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Spring 2005 John Rust Economics 425 University of Maryland Introduction to Dynamic Programming 1 Overview The method of dynamic programming is a way of solving problems involving choice over time and under uncertainty. The are sometimes called stochastic control problems in the engineering literature (where “stochastic” means “random or uncertain” and “control” refers to actions a decision maker takes in response to information to maximize some objective), or sometimes as sequential decision problems in the statistical literature. The theory underlying programming and the main principle of backward induction is quite old and it is hard to provide a precise credit to who/when this was “invented.” However the idea of “induction” in mathematics is very old as a method of proving propositions and probably dates back to the ancient greeks. However the idea of using backward induction to solve an explicitly formulated problem of choice over time and under uncertainty is much more recent origin, probably to the 20th century. In the 1940s, a French mathematician formulated and solved the problem of optimally drawing down water from a reservoir to generate electricity using dynamic programming. In the early 1950s, several American economists/mathematicians, Arrow, Karlin and Scarf, formulated and solved a problem of optimal inventory holding using dynamic programming, providing a recursive equation that has subsequently become known as the Bellman equation. Overall, the terminology “dynamic programming” is credited to Richard Bellman, who was one of the ¡rst to recognize that the method of recursive solution and backward induction could be applied to a whole host of problems in a variety of disciplines. Bellman was a mathematician who received his Phd from Princeton University and was working as a tenured professor at Stanford in the late 1940s when he became interested in sequential decision problems in connection to projects he was working on as an af¢iate of the RAND Corporation in Santa Monica, California. To quote Bellman, “An interesting question is, ‘Where did the name, dynamic programming, come from?’ The 1950s were not good years for mathematical research. We had a very interesting gentleman in Washington named Wilson. He was Secretary of Defense, and he actually had a pathological fear and hatred of the word, research. I’m not using the term lightly; I’m using it precisely. His face would suffuse, he would turn red, and he would get violent if people use the term, research, in his presence. You can imagine how he felt, then, about the term, mathematical. The RAND Corporation was employed by the Air Force, and the Air Force had Wilson as its boss, essentially. Hence, I felt I had to do something to shield Wilson and the Air Force from the fact that I was really doing mathematics inside the RAND Corporation. What title, what name, could I choose? In the ¡rst place I was interested in planning, in decision making,title, what name, could I choose?...
View
Full Document
 Spring '08
 Hulten
 Dynamic Programming, Optimization, probability density function, Bellman equation, Richard Bellman

Click to edit the document details