lecture10

lecture10 - Yinyu Ye, MS&E, Stanford MS&E211...

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #10 1 NonlinearOptimizationAlgorithmsI YinyuYe DepartmentofManagementScienceandEngineering StanfordUniversity Stanford,CA94305,U.S.A. http://www.stanford.edu/yyye Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #10 2 Introduction Optimizationalgorithmstendtobe iterativeprocedures . Startingfromagivenpoint x ,theygenerateasequence { x k } of iterates (ortrialsolutions). Westudyalgorithmsthatproduceiteratesaccordingto welldetermined rulesDeterministicAlgorithm ratherthansome randomselection processRandomizedAlgorithm. Therulestobefollowedandtheproceduresthatcanbeapplieddependtoalarge extentonthecharacteristicsoftheproblemtobesolved. Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #10 3 Classesofproblems Someofthedistinctionsbetweenoptimizationproblemsstemfrom (a) differentiableversusnondifferentiable functions; (b) unconstrainedversusconstrained variables; (c) one-dimensionalversusmulti-dimensional variables; (d) convexversusnonconvex minimization. Finiteversusconvergentiterativemethods. Forsomeclassesofoptimization problems(e.g.,linearandquadraticoptimization)therearealgorithmsthatobtain asolutionordetectthattheobjectivefunctionisunboundedina finitenumber ofiterations.Forthisreason,wecallthem finitealgorithms . MostalgorithmsencounteredinOptimizationarenotfinite,butinsteadare convergent oratleasttheyaredesignedtobeso.Theirobjectistogeneratea sequenceoftrialorapproximatesolutionsthat converge toasolution. Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #10 4 Themeaningofsolution Whatismeantbyasolutionmaydifferfromonealgorithmtoanother.Insome cases,oneseeksa localminimum ;insomecases,oneseeksa globalminimum ; inothers,oneseeksa KKT pointofsomesortasinthemethodof steepest descent discussedbelow.Infact,thereareseveralpossibilitiesfordefiningwhata solutionis.Oncethedefinitionischosen,theremustbeawayoftestingwhether ornotapoint(trialsolution)belongstothesetofsolutions. Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #10 5 Searchdirections Typically,anonlinearoptimizationalgorithmgeneratesasequenceofpoints throughaniterativeschemeoftheform x k +1 = x k + k d k where d k isthe searchdirection and k isthe stepsize or steplength . Thekeyisthatonce x k isknown,then d k ischosenassomefunctionof x k ,and thescalar k maybechoseninaccordancewithsomeline(one-dimension) searchrules. Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #10 6 Thegeneralidea Oneselectsastartingpointandgeneratesapossiblyinfinitesequenceoftrial solutionseachofwhichisspecifiedbythealgorithm. Theideaistodothisinsuchawaythatthesequenceofiteratesgeneratedbythe algorithm converges toanelementofthesolutionsetoftheproblem. Convergencetosomeothersortofpointisundesirableasisfailureofthe sequencetoconvergeatall. Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #10 7 Convergentsequencesofrealnumbers Let { x k } beasequenceofrealnumbers.Then { x k } convergesto 0ifandonly ifforallrealnumbers > thereexistsapositiveinteger K suchthat...
View Full Document

This note was uploaded on 05/06/2010 for the course MSE 211 at Stanford.

Page1 / 21

lecture10 - Yinyu Ye, MS&E, Stanford MS&E211...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online