sol3 - EE292 Spring 2006 Analysis& Control of Markov...

This preview shows pages 1–3. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: EE292 Spring 2006 Analysis & Control of Markov Chains April 29, 2006 Prof. Ben Van Roy Homework Assignment 3 : Solutions 1 See attached code. Note that we use the fact that argmax p ≥ exp(-γp )( p + C ) = 1 /γ-C for C > . Below is a plot of optimal price versus x and k . 200 400 600 800 1000 5 10 15 20 500 1000 1500 2000 k Optimal Price (x max = 20, N = 1000, p = 0.1, γ = 400) x p * Figure 1: Optimal Price From the plot it is clear that maximal price sensitivity is at x = 0 , t = 0 . 4.7 a) Letting z k = x k + γe k-1 , z k + 1 = x k +1 + γe k = x k + u k-w k + γe k = x k + γe k-1 + u k-(1-γ ) e k = z k + u k-(1-γ ) e k Since e k-1 = ∑ k-1 i =0 γ i w k-1-i , e k-1 is known to the controller at time k . Thus, deﬁning z k as the new state variable does not result in the loss of state information (i.e. we still have a system with 200 400 600 800 1000 5 10 15 20 2000 4000 6000 8000 10000 12000 14000 k x J Figure 2: Optimal Expected Revenue perfect state information). The new cost function at stage perfect state information)....
View Full Document

This note was uploaded on 02/15/2011 for the course EECS 6.231 taught by Professor Bertsekas during the Spring '10 term at MIT.

Page1 / 4

sol3 - EE292 Spring 2006 Analysis& Control of Markov...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online