SW_chap4 - 52 VARI differentiai equation is described by...

Info icon This preview shows pages 1–13. Sign up to view the full content.

View Full Document Right Arrow Icon
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 2
Image of page 3

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 4
Image of page 5

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 6
Image of page 7

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 8
Image of page 9

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 10
Image of page 11

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 12
Image of page 13
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 52 VARI differentiai equation is described by 5:1 r»: MU), 361(0) = 1 XII; 3 R XZ(0) 1:1. I ' ions and transversality conditions, ’1. A linear second~order of the EulernLagrange equat ‘nd, ’0 use ‘ . ' . ' FE y ntrol 11(1) which minimizes. the optimal co (a) J = I: 242 cit, an) = x20) == 9 (a) J a j: u2 dr, xlo) a 0 (c) J = j: at a, x4e) = can 2 at [Also determine t; and x1(rf).1 (d) J = j" u2 cit, any) =«» car) a ~I}, x203 w 0 D 3 2 4; ruin} dt. (6) I «2 {llel ' . For alt eases skeich both the optimal system trajectory 2:0!) and the optimal system control am. 8. "For the fixed piant dynamics given by . m u determine the optimal closed—loop system which mmrmrzes J=ij:[u2+(x_r)2}dr where i0) = 1 ~ 9". ‘ O M 9 For the fixed plant dynamics giyen by x 3 MG), x( ) w x9, closed-loop control which minimizes for fixed If fr J = %sz(tf) + i); L “2. d: . . . . m h re s is an arbitrary constant. Do this by first determining the optima e W I :pen—loop control and traieetory and then let u(t) w k(t)x(t) determine the optimal The maximum principle and Hamiltonsjambi theory In the previous chapter, we formulated many problems in the classical calculus of variations El]. A derivation of the Euler-Lagrange equations for both the scalar and vector cases was presented. We discussed the associated transversality conditions and some of the difficulties which we may encounter if inequality constraints are present. Several simple optimal control problems Were stated and solved. In this chapter we wish to reexamine many of the problems presented in the previous chapter and obtain more general solutions for some of them. in addition, we will develop methods for handling some problems which could not be conveniently formulated by the methods in the previous chapter. To these ends, we will present the Bolza formulation of the variational calculus using Hamiltonian methods. This wiil lead us into a proof of the Pontryagin maximum principle and the associated transversality conditions 12—5]. We will proceed then to a development of the HamiltonuJacobi equa- tions [12—44], which are equivalent to Beliman’s equations of continuous dynamic programing. Finally, we will give brief mention to some iimitations of dynamic programming. Examples to illustrate the methods will be pre» sented. We will reserve the next chapter for a discussion of some of the many problems which we can formulate and solve using the maximum principle. In order to fuiiy develop our approach to optimization theory where the terminal time is not fixed and where the control and state vectors are not necessarily smooth functions, we must consider in more detail the first varia— tion for such problems. 54 THE MAXIMUM PRINCIPLE AND HAMILTON-JACOB: THEORY CH. 4 4.1 The variational approach for functions with terminal times not fixed We now extend the variational approach introduced in Sec. 3.6 to prob- lems having unspecified terminal times. Consider extremlzrng J 2 Fence), in), :3 dz (4.1-1; with respect to the set of all admissible (see Sec. 2.1) trajectories. Let t, be the terminai time associated with optimal trajectory is. Associated With. each perturbation it away from the optimal trajectory is a perturbation 6:, in the terminal time. Let the first variation (SJ be the part of AI m J{x ~i— 12,1:Jr + do) —— J(x, If) (4.1-2) which is linear in i1 and 52}. Substituting Eq. (4.1-1) into Eq. (41:2), taking the linear terms of AJ (in h, 61}, and h), and performing the usual integration by parts to reduce terms dependent on it to terms dependent on 11, we obtam aJ :2 <1:[x(:,),:t(z,.), 1:55;, + beef) {lflhlxegégra o] + 'r so a as _ I I170) (E — 2}“ 3;) d: (4.1 3) where for convenience, we have assumed that the initial conditiOn is fixed and hence i195} : 0. . . in order to rephrase Eq. (4.1-3) into a convenient form, we introduce the following notation. We define From a Taylor series expansion of x(t,« + diff), we note that 6x(rf) is a‘close approximation to that part of [x(t, ~l— Eff) +110, + 55,0] — x(t,) which is linear in boy) and 61,. Submitting Eq. (4.1-4) into Ed. (4.1-3) and rearrang— ing, the first variation becomes r a; : lento). no), tr] — flaw} 61" + d 1m r) In much of our work, it will be convenient to define a quantity, called the Hamiltonian, by Htxtt), W), t} = (D «w it»? 2 <1: + an. (4.1-6) afloat—semi. fantasies. (4.1-5) ..»..h._a..._...,,.m...._,.w..._.......-...t....... “‘r. a CH. 4 THE MAXIMUM PRINCIPLE we Hammett-Janos; THEORY 55 where the Hamiltonian is not a function of}: i0), and Mt) are called the ' canonical Variables. In terms of the Hamiltonian, the first variation of Eq. (4.1—1), which is Eq. (4.1-5) becomes 61 :- —6x7(t,)h(tf) + H {311(9), Mgr), if] 53,, + f lira) {93% -i— ~33 dz. (4.15!) To establish a necessary condition for-a minimum, it is necessary that the integrand in Eqs. (43-5) and (4.1-7) vanish and also that the trans- versality condition, as obtained from Eq. (4.1-7), —5XT(1‘I)M3I) ”l‘ Htxm), Mfr), fr] 5‘? z 0 (4.1-8) be satisfied. 4.2 Weierstrass-Erdmann conditions Thus far in our development, admissible trajectories have been constrained to be continuously diiferentiabie with respect to x and t. This functional constraint on the class of ail admissibie traiectories is often unrealistically restrictive, as the following example will Show. For this example, an optima} admissible soiution does not exist; however, if the functional restriction on an admissibie trajectory is sufficiently relaxed, existence of an optimal admissible trajectory is assured. We now examine the consequences of our new definition of an admissible trajectory which are the Weierstrass- Erdmann conditions {i}. Let us consider the problem of minimizing the cost function J: fxzawxydz 0 subject to x(0) = O, x(1) = l. Physically, it is clear that the absolute minimum for J is 0 and that this is obtained for x(t) = 0, t E [0, i] x(t) = 2; — 1, t e [121} which is certainly a soiution to the Enter-Lagrange equation for this probiem x255 —l— 2655" —— 4x = 0. There is one disturbing feature about this solution, however, in that the optimum x{t) has a “corner” or discontinuous first derivative which gives 56 THE MAXIMUM Famctrrs AND HAMmTON—IACOBI THEORY CH. 4 rise to formal difficulty since it is contained in the Euler-Lagrange equations. Thus, the solution of the above problem is not admissible. Certainly, this particular function x{t) is continuousiy differentiable everywhere except at a finite number of points (in this case the single point r a: i). Thus, in relaxing the set of admissibie trajectories to allow for functions which are piecewise continuously differentiable, the function x0) is admissibie and the above inrobiem then has an optimal admissible control. . The Weierstrass-Erdmann corner conditions furnish us with necessary conditions for an optimal trajectory to have a discontinuous derivative at a point in the control interval of interest. Specifically, consider the probiem of finding a trajectory among the class of all continuously differentiable functions on [52, b] having a corner at c E (a, b) which satisfies fixed initial and finai boundary values such that the functional 3: Jon 2 i one), in), t} dt has an extremum. It is of course clear that, for t E Eu, 0] and r e [c, b}, the function x(t) must satisfy the Euler-Lagrange equations for a minimum ii. 993.. W ‘1‘? ..—_ 0 dr 65: 6x We may rewrite the cost function as a sum of two cost functions: a) = i: one), is), :1 at: + l: axe), so), :1 at : J1(x) + 14X). _ We may now take the first variation 6J1(x) and 5.1291) separateiy. We assume, for the moment oniy, that a and b are fixed, and we require that the £0) calculated from J1(x) and J2(x) is the same at t = c which is unknown. Since 5 is arbitrary, the first variation of J1(x) is 51K") = *i 51505:) WM + {we}, in), c] ._ «swagger c1} a: a sxrtciLW—ngfi’é‘c)’ “1 + .. do of as: j, W) {as - a a} ‘1“ Since x(t) satisfies the Euler-Lagrange equations for an extremal and since 8x(a) = 0, we have acme, fit), I} i drift) T {one}, to), '5} ——~ anoW} a»: (for 1: = c _ 0). 6J,(x) 2 sxrm CH. 4 Tris Maxuatm PRINCIPLE AND HAMILTON-JACOB! THEORY 57 In a simiiar fashion, we can show that the first variation for the extremal solution of 1201;) is 6320‘) m —5XT(1)W _ {(I’lXC‘F): 51(1): T] — 13(0W} 6': (for 'r :2 c + 0). In order to obtain the extremum, the extremal solution must satisfy §J(x) 3 ages + 6.50;) = O Thus ‘1‘? 6(1) 63. irwcao m 33’? MPH) (4.2—1) . dd) . aq, ‘33 _ T— = _ r__ x 6ii¢=c—o q) x as: W” (4.22) since 5x and 5:, are arbitrary. These requirements, Eqs. (4.2—i) and (4.2-2), are called the Weierstrass—Erdmaan comer conditions and must hoid at any point c where the extremal has a corner. If we use the Hamiitonian canonical variables -H=owf%§§mo+trx —_1 dis we immediately see that the Weierstrass-Erdmann conditions simpiy require Hand 24. to be continuous on the optimum trajectory at aii points where there are corners. 4.3 The Bolza yrobiem—mno inequality constraints In Sec. 3.7 We considered the solution of Lagrange problems with equality constraints of the form g(x, it, t) 2 0 for alt r in the control intervai of inter— est. A speciai case of this-equality constraint which is well—recognized as a model of a large and important class of physicai systems is so) = f[x(t), u(t), t}, (4.3-1) where the m-vector It represents the control function to be selected and the n—vector x represents the resulting trajectory. We wiii assume that f has con- tinuous partial derivatives with respect to x and u. Often it is the case that such smoothness assumptions guarantee that for any piecewise continuous function a, there exists a unique, admissible trajectory x to Eq. (4.3-1). We therefore define the set of admissible control functions to be the ciass of pieces 58 THE MAXIMUM PRINCIPLE AND HAMILroN-IACOBI THEORY CH .4 wise continuous functions and assume that, for an admissible in and a given initial condition x09), Eq. (4.3—1) defines a unique, admissible solution over the control interval of interest. Throughout the remainder of this section and the following section, we wilt consider the development of necessary conditions for Bolza problems subject to the equality constraint in Eq. (4.3-1). Subsections (4.34) and (4.3—2) will consider the fixed finai time and the unspecified final time cases when no restrictions are imposed on the value that u can take at each time if during the control interval of interest. Section 4.4 considers two cases of inequality constraints on the control function and its associated trajectory over the control interval. 4.3-] Continuous optimal control problems“— fixed beginning and terminai times—— no inequality constraints We now consider the problem of determining an admissible control function u in order to minimize the criterion ' J = 9[x(r), t] ~+ l axe), no), :1 dr, (4.3—2) where 9 and git possess continuous partial derivatives in x and n. We use the method of Lagrange multipliers discussed in the last chapter to adjoin the system differential equaiity constraint to the cost function, which gives us J x “ml “i: t i," one), no), a + marine), um, :1 -- a} at ° ° (4.3—3) We define a scalar function, the Hamiltonian, as H [1(0), 11(1), Mt), Ii «re (film), I10), I] a“ ”(flflxo'), 11(1), 1‘1. (4-3-4) Thus the cost function becomes J a can), t] + j {Htqu HQ), in), z] -— was} at. (4.3-5) If we integrate the East term in the integrand of Eq. (4.3—5) by parts, we obtain J = {9[x(r), I] —> PKGXUH ” + j " {Harri as), Mr). :1 4— texts); a. 2" ° (4.3-6) We now take the first variation of J for variations in the control vector and, consequently, in the state vector about the optimai control: and optimai :: f: CH. 4 THE MAXIMUM PRINCiPLn AND HAMiLTON—JACOBI THEORY 59 state vector. This gives us "‘I "’ 6H - 6H + J {Syria + A] + Barb—u} dt. (4.3—7) A necessary condition for a minimum is that the first variation in J vanish for arbitrary variations 6x and flu. Thus we have as the necessary condition for a minimum the very important relations 61 .7, {ext 3% w it} 5x1" 2;: ._ 1a 0, for r a i,,:, (4.3-8) - 6H . r a "3;, x m f(x, 1:, t) a fig (4.3-9) :1: a 0. C are/L3 (4.3-10) _ We now consider in more detail the transversaiity conditions expressed in Eq. (4.3-8). . F ora large class of optimal control problems, the initial state of the system is specrfied but the terminal state is unspecified. In that case, Eq. (4.3—8) yields the transversality conditions as Mo) : Lg‘figb ’1'} (4.3»! 1) since Sabra) m 0, x03) is fixed, and 5x09) is completely arbitrary. In another broad class of problems x00) and x(t,-) are fixed. In this case 8x00) and 61:0,) must be zero, and x03) and x03) are the boundary conditions for the two- point boundary value problem. For rnany estimation probierns, neither x(r,,) nor x03) are fixed and 9 = 0. In that case, Eq. (4.3—8) yields Mtg) m sz) = O as the boundary conditions for the problem since 5116,) and §X(t‘,—) are arbitrary. In still another case, we might have x00) L“ X5, 6 2 O, and il x(tf) E? m i. In this event, it is easy for us to show that the final transversality conditions are obtained if we solve the two scalar equations, each in :1 variables xtto) 2 xi, 5XT(tf)x(fj-) : 0, waging.) a c. (4.342) We now give a more general and precise interpretation to the transversality conditions. For the generai case where the initiai manifold is Mixua), to] m 0 (4.3-13) and the terminal manifold is Ntxui), If} 2 0, (4.344) we adjoin these conditions to the (9 function by means of Lagrange multipiiers, eff and v and obtain for the cost function 60 THE MAXIMUM PRINCIPLE mo HemaroN-Jacoar THEORY CH. 4 J n axe), :11: «m were), :01 + vTNtxea a] + j” {H{X(t), u(t), to), a] _ iron} dz. (4. 3—15) We now apply the usual variational techniques to obtain for the trans- versality conditions at the initial time: _ 69 dMT J“'(l‘o) '— a; 'i— ( 6X )§9 The n initial conditions are obtained from this, with r parameters to be found in Eq. (4.3—16) such that we satisfy the r conditions of Eq. (4.3-1.3). In a similar fashion, the terminal condition is W 69 dNT w, ,m . - Mg.) ~ 3‘; (73v, N{x(t), t] _ o, t _ i,, (4.3 17) n terminal conditions are obtained from this with g parameters v found in Eq. (4.3-1?) such that the q conditions of Eq. (43-14) are satisfied. The n vector difierential equation obtained from Eq. (4.3-9) will be called the adioint equation. Equation (4.3-10) provides the coupling relation be tween the original plant dynamics, Eq. (4.34), and the adj oint equation, the 1. equation of Eq. (4.3—9). This coupling equation was obtained from are +£f{§u7‘%%+ ...}dz, and it is important to note that Bu must be completely arbitrary in order for us to draw the conclusion that 6876:: x: O to obtain the optimal control. For the problem posed here where the admissible control set is infinite, Eu can be completely arbitrary. Where the admissible control is bounded, 8“ cannot be completely arbitrary, and 619/611 m 0 may not be the correct requirement. We will have more to say about this later. The solution we have obtained for this problem is a specral case of the Pontryagin maximum principle. _ It is also interesting to note that, since H =2 93 +‘Wf, we may compute the total derivative with respect to time as dH _ do . 695 GE . do QB: -' 143 t x" «n +( it] ”Tia—u 4"“ (will t W "i“ t a: M[x(r), r} = 0, r 2 re. (4.346) .3? M 3? 6);; (4.3-18) but from Eqs. (4.3-9) and (4.34) we have- - .. 3H _... do dfT _ l r" "a; _ ”as ”" lair)" - (4'3 ‘9) and from Eq. (4.3-4) CH. 4 THE MAXIMUM PRINClPLE AND HAMILTON-JACOB: THEORY 61 Thus, since fl. = 1T1“, Eq. (43-18) becomes dH 6 6f . d We see that, if to and f are not explicit functions of time, the Hamiltonian is constant along an optimal trajectory where 63/61! x 0. It can be shown that this 13 always true along an optimal trajectory, even if we cannot require dH/du m 0. We will make use of this fact in a later development. . In order that J be a minimum, the second variation of J must be nonnega» tive along all trajectories such that Eq. (4.3—1) is satisfied. Therefore we need to compute the second variation of J in Eq. (4.3-6) and impose the require- ment that the variation of Eq. (4.3-1) is zero, or that as; _.. (72%) 5x — (3%) 3:: = e. (4.3»22) Applying this condition and taking the quadratic part of the Taylor series expansron of J(x —§— fix, it —§- 511) — J(x, 11), We have for the second variation (4.3—7.1) 1 62 tax, so: “flaxrmgsfl + @611 a an .h ” T x2 (ii-1793? 5x 2 J: an SM] 6 M T 62H [qu“? (4.3—23) {3&3}? W and this must be nonnegative for a minimum. This will be the case if the g :1— m square matrix under the integral Sign and rind/(3x2 are nonnegative e nite. Example 4.3-1. We are given the dilferentiai system consisting of three cascaded integrators ft; m x; 351(0) = 0 32 m x; xz(0) m G x‘3 w a 33(9) m 0. We wish to drive the system so that we reach the terminal manifoid Xiti) + 2:30) m I such that the cost function J=§£uzdr is minimized. The solution to the problem proceeds as follows. We compute the Hamiltonian from Eq. (4.3-4) as H = £412 + I‘ll-1'2 + 11.23% + 131! and determine the coupling relation, Eq. (4.340), 62. THE MAXIMUM PRENCli’LE AND HAMILTON-Incest anonv CH. 4 fim0=u+lg fin and the adjoint Eq. (4.3-9), 1 w _§_£:i = 0 i W 6X} 61‘! _ _ 12 ”- —E52 M 11 From Eqs. (4.3—14) and (4.347) we see that the transversality condition at the terminal time is xfll) 4.- xifl) m 1 NT in) m 12% + (6—6;)v, are t: where Nixtrr), oi 3: xi(?;) + xiflr) M 1 = 0: If e 1- 112(1) 2171(1)? 1(1) = [212(1) “ [2362(1)]? - 3(1) 0 Thus the probiern of finding the optimai controi and associated trajectories for this example is completeiy resoived when we solve the two-pornt boundary vaiue problem represented by Thus i; m x; x;(0) = 0 222 = x3 are) = 0 .333 w —13 953(0) = 0 it a o M1) = 2xltnv}xm + fig) = 1 3.; a —11; M1) = man is m ~22 . 213(1) =0 Although the six first-order differential equations represented above are per— fectly linear and time invariant, the solution to this problem'is complicated by the nonlinear nature of the terminal conditions. We shail discover various iterative schemes for overcoming this difficulty in Chapter 10. 4.3—2 Continuous optimal control problems—- fixed beginning and unspecified terminal times-— 720 inequality constraints The materiai of the previous subsection may be easiiy extended to. the case where the terminai manifold equation is a function of the terminal tame, CH. 4 Tire MAXiMUM PRINCIPLE AND HAMILTON-Jamar THEORY 63 and the terminal time is unspecified. For convenience we will assume that the initiai time and the initial state vector are specified. Soiution may then easily be obtained for the case where the initiai time and initial state vector are unspecified. Therefore the problem becomes one of minimizing the cost function J = Bins), n] + if axe), no), :1 dz (43—24) for the system described by i = fEXO‘), “(1‘), 3}, X09) = Xe (4-3—25) where ta is fixed and where, at the unspecified terminal time t m If, the q vector terminal manifold equation N[X(tf), if} = 9 (4.3-26) is satisfied. It may be noted here that the terminal manifold line, x(tf) = .205.) of the previous chapter becomes N[x(t,), if] M: 0, which is more genera}. We adjoin the equality constraints to the cost function via Lagrange multipliers to obtain J :- axon, ta + vrutxen, s] + if {axe}, an), r] + 7LT(r)[f{x(r), 110), t} -— 31]} dr. As before, we define the Hanuitonian H [31(3), 60), Mt), fi fi MW), "0), I] + ”(OHXG), “(0, t] and integrate a portion of the cost function, Eq. (43-27), to obtain J m 956;»): If] + VTNDiUr), If] “- VCfrkOf) + moxie) i- f {Hint}, at), Mr), a + true} at (4.3»27) (43-28) We again form the first variation by ietting x0) m 510) + ha), :10) : ii(r) + 51103), If = ff ~i— 5t, (4.3—29) and then we form the difference J[x, u, If} - J [i 13, ff} and retain only the iinear terms. Thus we have, after dropping the s notation for convenience, at a Maine), non. Mo), s]...
View Full Document

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern