part4 notes - Chapter 4 Unconstrained Optimization...

Info icon This preview shows pages 1–8. Sign up to view the full content.

Chapter 4: Unconstrained Optimization Unconstrained optimization problem min x F ( x ) or max x F ( x ) Constrained optimization problem min x F ( x ) or max x F ( x ) subject to g ( x ) = 0 and/or h ( x ) < 0 or h ( x ) > 0 Example : minimize the outer area of a cylinder subject to a fixed volume. Objective function F ( x ) = 2 πr 2 + 2 πrh, x = h r h i Constraint: 2 πr 2 h = V 1
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

Outline: Part I: one-dimensional unconstrained optimization Analytical method Newton’s method Golden-section search method Part II: multidimensional unconstrained optimization Analytical method Gradient method — steepest ascent (descent) method Newton’s method 2
Image of page 2
PART I: One-Dimensional Unconstrained Optimization Techniques 1 Analytical approach (1-D) min x F ( x ) or max x F ( x ) Let F 0 ( x ) = 0 and find x = x * . If F 00 ( x * ) > 0 , F ( x * ) = min x F ( x ) , x * is a local minimum of F ( x ) ; If F 00 ( x * ) < 0 , F ( x * ) = max x F ( x ) , x * is a local maximum of F ( x ) ; If F 00 ( x * ) = 0 , x * is a critical point of F ( x ) Example 1: F ( x ) = x 2 , F 0 ( x ) = 2 x = 0 , x * = 0 . F 00 ( x * ) = 2 > 0 . Therefore, F (0) = min x F ( x ) Example 2: F ( x ) = x 3 , F 0 ( x ) = 3 x 2 = 0 , x * = 0 . F 00 ( x * ) = 0 . x * is not a local minimum nor a local maximum. Example 3: F ( x ) = x 4 , F 0 ( x ) = 4 x 3 = 0 , x * = 0 . F 00 ( x * ) = 0 . In example 2, F 0 ( x ) > 0 when x < x * and F 0 ( x ) > 0 when x > x * . In example 3, x * is a local minimum of F ( x ) . F 0 ( x ) < 0 when x < x * and F 0 ( x ) > 0 when x > x * . 3
Image of page 3

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

F’’(x)=0 F’(x)<0 F’(x)>0 F’(x)=0 F’’(x)>0 F’(x)=0 F’’(x)<0 F’(x)>0 F’(x)<0 F’(x)>0 F’(x)>0 F’(x)=0 Figure 1: Example of constrained optimization problem 2 Newton’s Method min x F ( x ) or max x F ( x ) Use x k to denote the current solution. F ( x k + p ) = F ( x k ) + pF 0 ( x k ) + p 2 2 F 00 ( x k ) + . . . F ( x k ) + pF 0 ( x k ) + p 2 2 F 00 ( x k ) 4
Image of page 4
F ( x * ) = min x F ( x ) min p F ( x k + p ) min p F ( x k ) + pF 0 ( x k ) + p 2 2 F 00 ( x k ) Let ∂F ( x ) ∂p = F 0 ( x k ) + pF 00 ( x k ) = 0 we have p = - F 0 ( x k ) F 00 ( x k ) Newton’s iteration x k +1 = x k + p = x k - F 0 ( x k ) F 00 ( x k ) Example : find the maximum value of f ( x ) = 2 sin x - x 2 10 with an initial guess of x 0 = 2 . 5 . Solution: f 0 ( x ) = 2 cos x - 2 x 10 = 2 cos x - x 5 5
Image of page 5

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

f 00 ( x ) = - 2 sin x - 1 5 x i +1 = x i - 2 cos x i - x i 5 - 2 sin x i - 1 5 x 0 = 2 . 5 , x 1 = 0 . 995 , x 2 = 1 . 469 . Comments: Same as N.-R. method for solving F 0 ( x ) = 0 . Quadratic convergence, | x k +1 - x * | ≤ β | x k - x * | 2 May diverge Requires both first and second derivatives Solution can be either local minimum or maximum 6
Image of page 6
3 Golden-section search for optimization in 1-D max x F ( x ) ( min x F ( x ) is equivalent to max x - F ( x ) ) Assume: only 1 peak value ( x * ) in ( x l , x u ) Steps: 1. Select x l < x u 2. Select 2 intermediate values, x 1 and x 2 so that x 1 = x l + d , x 2 = x u - d , and x 1 > x 2 . 3. Evaluate F ( x 1 ) and F ( x 2 ) and update the search range If F ( x 1 ) < F ( x 2 ) , then x * < x 1 . Update x l = x l and x u = x 1 .
Image of page 7

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

Image of page 8
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern