{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

# 6 - L Karp Notes for Dynamics VI Two Stochastic Control...

This preview shows pages 1–3. Sign up to view the full content.

L. Karp Notes for Dynamics VI. Two Stochastic Control Problems 1) Stochastic Control of Jump Process (i) Derivation of DPE (ii) Application to pollution control problem in section 4 2) Finite Markov chanins i) Discuss range management application. ii) Review finite Markov Chains. iii) Formulate optimal control problem. iv) Existence and uniqueness theorem. v) Three solution algorithms: a)successive approximation; b)policy iteration; c)linear programming. vi) A suggestion for estimating DP problem. The most popular analytic stochastic control model in economics uses a continuous random variable, in continuous time. This model is based on Brownian motion. I will not discuss it. 1. Jump processes : Stochastic control of Jump Processes, with an application to pollution control Now we consider optimal control problems that involve a particular kind of stochastic process, called a "jump process". If the state variable z follows a jump process, it evolves along path whose time derivative is f(z,u,t) "most of the time", where u is the control variable. But occasionally the variable z jumps up or down by a discrete amount. For example, z may be the stock of a resource, say fish or oil. Occasionally an event occurs, such as an epidemic which drastically reduces the fish population, or a discovery which increases the stock of oil, causing a discontinuous change in the variable. In other situations, the state variable might be the exchange rate which depreciates at a deterministic rate most of the time; but some event might trigger an attack on the currency, causing an abrupt devaluation. We’ll consider a simple case in which the discrete change is a(z,u,t). (More generally, there could be other possible discrete changes, say b(z,u,t). For example, the stock of oil might increase due to a discovery, so a > 0, or it might decrease discretely due to an environmental law which closes off-shore drilling, so b < 0.) i) Derivation of DPE for simple case. The probability that the state variable increases by the discrete amount a(z,u,t) over a small interval of time dt is A(z,u,t)dt. The state equation is (1) dz = f ( z , u ) dt + d π Pr ( d π = a ) = A ( z , u ) dt

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Pr ( d π = 0) = 1 - A ( z , u ) dt This equation implies that there is "very little" chance that the jump happens over any small interval. In order to derive the DPE we go through the same steps as with the deterministic problem. We define the value function, split up the program into two parts, the first part over (t, t+dt) and the second part over (t+dt, T). However in the present case we have to take expectations, to account for the uncertainty. (2) J ( z , t ) = max { u } E T t L ( z , u , τ ) d τ max { u } E t dt t L ( Z , u , τ ) d τ T t dt L ( z , u , τ ) d τ ( Note: ) E t dt T t dt L ( z , u , τ ) d τ J ( z dz , t dt ) max u L ( z , u , t ) dt J ( z t f ( ) dt a , t dt ) A ( ) dt 1 J ( z t f ( ) dt , t dt )[1 A ( ) dt ] 2 Depending on whether or not the jump occurs, the state changes by f( )dt + a or by f( )dt. We have to weight each of those events by the probability that it occurs.
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}