lecture10 - Yinyu Ye, MS&E, Stanford...

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #10 1 NonlinearOptimizationAlgorithmsI YinyuYe DepartmentofManagementScienceandEngineering StanfordUniversity Stanford,CA94305,U.S.A. http://www.stanford.edu/yyye Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #10 2 Introduction Optimizationalgorithmstendtobe iterativeprocedures . Startingfromagivenpoint x ,theygenerateasequence { x k } of iterates (ortrialsolutions). Westudyalgorithmsthatproduceiteratesaccordingto welldetermined rulesDeterministicAlgorithm ratherthansome randomselection processRandomizedAlgorithm. Therulestobefollowedandtheproceduresthatcanbeapplieddependtoalarge extentonthecharacteristicsoftheproblemtobesolved. Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #10 3 Classesofproblems Someofthedistinctionsbetweenoptimizationproblemsstemfrom (a) differentiableversusnondifferentiable functions; (b) unconstrainedversusconstrained variables; (c) one-dimensionalversusmulti-dimensional variables; (d) convexversusnonconvex minimization. Finiteversusconvergentiterativemethods. Forsomeclassesofoptimization problems(e.g.,linearandquadraticoptimization)therearealgorithmsthatobtain asolutionordetectthattheobjectivefunctionisunboundedina finitenumber ofiterations.Forthisreason,wecallthem finitealgorithms . MostalgorithmsencounteredinOptimizationarenotfinite,butinsteadare convergent oratleasttheyaredesignedtobeso.Theirobjectistogeneratea sequenceoftrialorapproximatesolutionsthat converge toasolution. Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #10 4 Themeaningofsolution Whatismeantbyasolutionmaydifferfromonealgorithmtoanother.Insome cases,oneseeksa localminimum ;insomecases,oneseeksa globalminimum ; inothers,oneseeksa KKT pointofsomesortasinthemethodof steepest descent discussedbelow.Infact,thereareseveralpossibilitiesfordefiningwhata solutionis.Oncethedefinitionischosen,theremustbeawayoftestingwhether ornotapoint(trialsolution)belongstothesetofsolutions. Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #10 5 Searchdirections Typically,anonlinearoptimizationalgorithmgeneratesasequenceofpoints throughaniterativeschemeoftheform x k +1 = x k + k p k where p k isthe searchdirection and k isthe stepsize or steplength . Thekeyisthatonce x k isknown,then p k ischosenassomefunctionof x k ,and thescalar k maybechoseninaccordancewithsomeline(one-dimension) searchrules. Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #10 6 Thegeneralidea Oneselectsastartingpointandgeneratesapossiblyinfinitesequenceoftrial solutionseachofwhichisspecifiedbythealgorithm. Theideaistodothisinsuchawaythatthesequenceofiteratesgeneratedbythe algorithm converges toanelementofthesolutionsetoftheproblem. Convergencetosomeothersortofpointisundesirableasisfailureofthe sequencetoconvergeatall. Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #10 7 Convergentsequencesofrealnumbers Let { x k } beasequenceofrealnumbers.Then { x k } convergesto 0ifandonly ifforallrealnumbers > thereexistsapositiveinteger K suchthat...
View Full Document

This note was uploaded on 06/16/2010 for the course MS&E 211 taught by Professor Yinyuye during the Fall '07 term at Stanford.

Page1 / 20

lecture10 - Yinyu Ye, MS&E, Stanford...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online