A new application:
Going Through A Numerical Example
Note you have two types of variables: the xs are decision variable values and
the fs are function values. It is easy to start confusing them when doing multiobjective analysis.
A new a
CONSTRAINT METHOD in objective space for a
Min-Min Problem with Convex Feasible Space
Actual Tradeoff= Pareto Front
Subject to F1 < F1*
will work on nonconvex tradeoff
Multi Objective Optimization
Handout November 4, 2011
(A good reference for this material is the book
multi-objective optimization by K. Deb)
Multiple Objective Optimization
So far we have dealt with single objective optimization, e.g.
Objective (S) is
New Topic: Dieren0al Evolu0on
For Global Op0miza0on of Con0nuous
(winner in an interna0onal compe00on
among evolu0onary algorithms in 1996)
Journal Paper Reference
Storn, R. and K.
(End of GA Theory Slides)
There were some substititions (approximations)
made in the argument by Goldberg that do not
change the problem with the theorem.
Equations from Goldberg book on Genetic
Algorithms with these substitutions
Theory for Heuris-c Op-miza-on
There is a handout, which is pages
2833 in Goldbergs book on Gene-c
Next Topic is Theory for Gene-c Algorithms
A basic deni-on in Gene-
NEW EXAMPLE: Consider Another Case: Example 2.5 and Figure 2.8
Double limit analysis for Example
To find the final solution of the eventual SA
search, we do the following:
1. Holding T constant, let t (number of iterations)
The book Iterative Computer Algorithms with Applications
in Engineering by A.M. Sait and H. Youssef, IEEE
Publications (now Wiley) , 2000 is the primary source of
materials for the material on the theory.
Since I have exceeded already th
Revised slide: Computing a Solution
The purpose of all the calculations reported in
Table 18 is to convert your multi objective (MO)
problem into a GA with a single objective (SO).
The SO GA goal is maximizaton. The original MO
problem can be min-min, m
Crowding is used both to determine which
individuals to keep from the last allowable
front (F3 in graph) AND in Tournament
Selection to generate new offspring
Board Distance for Points in Fronts 1 & 2(p.
So the numbers computed on pr
Choose a set of possible functions and
terminals for the program.
You dont know ahead of time which functions
and terminals will be needed.
User needs to make intelligent choices for
best GP performance.
For planetary orbital p
ENGRD 241 Lecture Notes
Section X: Probability & Statistics
page X-1 of X-22
Notes written by Prof. J. Stedinger, 2003
INTRODUCTION TO PROBABILITY AND STATISTICS (C&C PT 5.2)
Reference: J. L. Devore, Probability and Statistics for Engineering and the Scie
CEE 509/ COMS 572 Heuristic
Methods for Optimization
Scalars, Vectors and Matrices
Its easy to handle vectors and matrices in
Semi colons are line terminators and
e.g., a = 5; is a scalar
a = [2 3 4 5] i
Hypothesis Tests: Essential Information on Statistics and Matlab for
Comparing Algorithms Results
Heuristics Optimization Course, Fall 2010
This document contains basic information on how to perform the following three hypothesis
tests: paired t test, two
(using some slides from earlier lectures)
So Why would you want
to take this course?
For many problems there is no feasible way to nd
good solu,ons except by usin
It is important to know that the response
surface R(x) is fit to ALL evaluations of F
(x) in all the previous iterations.
Plot is in terms of evaluations
evaluations in these graphs is the number
of times you evaluate the expensive
Response Surface Methods in
A nonlinear Response Surface R(x) is a
continuous nonlinear multivariate approximation
R(x) has also been called a response surface or a
Response surfaces can also be used with other
Genetic Programming applies the
concept of Genetic Algorithms to
Automatic Programming refers to
algorithms that generate a computer
program to do a specific task.
Why would you want to u
Genetic Programming II:
Genetic Programming for Artificial
Genetic programming can be used for
much more diverse and complicated
algorithms than polynomials or the
functions arising in symbolic regr
The Previous Analysis was for Any
Itera*ve Search Method. We
applied the analysis to Random
Search and Greedy Search.
Now we move to a more detailed
analysis of Simulated Annealing (in
chapter 2 )
Doing a computation
Assume that you start with So = 3, then the initial P vector is
P3j= prob from 3 to j-get
from conf graph
Markov Chain Matrix corresponding to Fig. 1.3,
an example with N=8
Where did these numbers come from? See
Theoretical Analysis of Heuristic
Part I: Convergence of Iterative
Stochastic Algorithms and Simulated
The Theory lectures start 10-17-11. Students
taking the course for 3 credits plus project are
not required to attend these lecture
* Tabu = Taboo
Tabu Search (TS) algorithm is based on the idea
that you want to prevent the search from going
back to some regions.
Tabu means forbidden. In TS some moves
(going from s0 to snew) are tabu.
Why would you want to
Resource Allocation in
Cellular and Ad Hoc Networks
Prof. Stephen Wicker
School of Electrical and Computer Eng.
Heuristic Methods for Optimization,
COMS 5722 , CEE 5290, ORIE 5340
4.6 billion cell phones in use
All major fo
Probably the Most Popular Heuristic Algorithm
(but not necessarily the best for some applications)
Genetics and Genetic Algorithms
We are all
Importance of your course handouts
and your own handwritten notes
In exams you will be allowed to bring in your
course handouts and your own handwritten
notes (on the handouts on or other pages) so
you will want to have them in order (perhaps
with tabs f
Setting Parameters for Simulated
All heuristic algorithms (and many nonlinear
programming algorithms) are affected by
For Simulated Annealing the algorithm
To, M, , , maxtime
So how do we select these paramete
Projects and Decisions
If you do an existing project (one of five
announced on the syllabus), you do not need to decide
for several weeks about which project you want to do.
We will have outside speakers to describe the exisiting
projects, e.g. ECE Prof.
Slides for VideoTaped Talk
This video taped talk replaces the regular
lecture on Aug. 29.
You are expected to view the video tape.
These slides are NOT intended to replace
the video taped lecture.
(There will also be two lectures videotaped on 9-2 to