This preview shows page 1. Sign up to view the full content.
Unformatted text preview: Multidisciplinary System
Design Optimization (MSDO)
Approximation Methods
Karen Willcox
Slides from: Theresa Robinson, Andrew March
1 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Outline
• Introduction to approximation methods
• Data fit methods
– Polynomial response surfaces
– Kriging • Model order reduction
– Reducedbasis methods
– Proper orthogonal decomposition • Multifidelity methods
– Trustregion model management
2 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Approximation Methods
• Replace the simulation with an approximation
or “surrogate”
• Uses some data from the initial simulation
– Can be global or local • Surrogate is much less computationally
expensive to evaluate
• Not just optimization
– Uncertainty Quantification (e.g. Monte Carlo
simulation methods)
– Visualization 3 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Why Approximation Methods
We have seen throughout the course the constant tradeoff
between computational cost and fidelity. intermediate fidelity
(e.g. vortex lattice,
beam theory)
empirical
models can we do
better? how to
implement? Fidelity Level high fidelity
(e.g. CFD,FEM) from Giesing, 1998 Level of MSDO
trade
studies limited
optimization/iteration can the
results be
believed?
full MDO Approximation methods provide a way to get highfidelity
model information throughout the optimization without the
computational expense.
4 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Data Fit Methods
• Sample the simulation at some number of
design points
– Use DOE methods, e.g. Latin hypercube, to select
the points • Fit a surrogate model using the sampled
information
• Surrogate may be global (e.g., quadratic
response surface) or local (e.g., Kriging
interpolation)
• Surrogate may be updated adaptively by
adding sample points based on surrogate
performance (e.g., EGO)
5 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Polynomial Response Surface Method
• Surrogate model is a local or global
polynomial model
• Can be of any order
– Most often quadratic; higher order requires many
samples • Advantages: Simple to implement, visualize,
and understand, easy to find the optimum of
the response surface
• Disadvantages: May be too simple, doesn’t
capture multimodal functions well 6 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Global Polynomial Response Surface
• Fit objective function with a polynomial
• e.g. quadratic approximation: J (x) a0 cii xi2 bi xi
i i cij xi x j
i,j i • Update model by including a new function evaluation
then doing least squares fit to compute the new
coefficients 7 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Global Polynomial Response Surface
Estimation problem:
J J 1 J 2 J J Xc M J=vector containing
M responses c0
c T 1 1
x1 1 1
x2 11
x1 x1 2
1 x x2 is the value of
1
x1 for the 2nd
sample
M
x1 Least squares solution: c
8 cp 1 c=vector containing
p coefficients X 1 c1 M
x2 MM
x1 x1 T XX 1 X=M p matrix
Each row corresponds
to one data sample;
each column
corresponds to an
unknown coefficient XTJ © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Kriging
• Adopted from the geostatistics literature
• Based on Gaussian process models
• Assumes that the output function values are
correlated in design space, i.e. closer points
are more highly correlated
• Can have multiple extrema
• Interpolating method
– Exact at sample points • Gives estimate of mean squared error
– Can use to give error bounds
– Can use to choose new sample points
9 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Kriging: Mathematical Background
• We want to make a prediction of y at a point x
• Uncertain of value: model as a random
variable, normally distributed with mean
and variance 2
• Consider two points xi and xj
• Expect values to be close if the distance
between them is small
• Formalize this idea by setting:
pi n Corr[Y ( x j ), Y ( xk )] exp i x ji xki i1
10 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Kriging Basis Functions
pi n Corr[Y ( x j ), Y ( xk )] exp i x ji xki Correlation Correlation i1 Distance between points Each pi and
11 i Distance between points is chosen to best fit the data © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Kriging
Kriging Mathematics Cont.
• Choose μ, pi, and θi to maximize the
likelihood of observing the data
– Detailed equations in Giunta and Watson (1998),
derivation in Jones (2001) • Kriging predictor is
predictor is ⎛n
ˆ
y x* = μ + ∑ ci exp⎜ − ∑ θ j x* − x i
⎜
i =1
⎝ j =1 () k mean surface
12 pj ⎞
⎟
⎟
⎠ weighted sum of Gaussians,
each centered at a sample point © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Kriging Extensions
• Can combine polynomial RSM and Kriging
– Apply Kriging to difference between sample values
and polynomial approximation • Soft Kriging allows upper and lower bounds,
prior CDFs
• Efficient Global Optimization (EGO)
– Uses Kriging to find “expected improvement”
– Samples the point with the largest expected
improvement and adds it to the sample set 13 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Efficient
Efficient Global Optimization
• Jones 1998; based on probability theory
• Assumes: f ( x ) ≈ β T x + N (μ ( x ), σ 2 ( x ) ) • β Tx • (
) is normally
regression model : regression term
regression term N μ (x), σ 2 (x) : error from distributed, with mean μ(x)
and variance σ2(x) • Estimate function values with a
Kriging model
– Predicts mean and variance Surrogate model is updated
model is updated
adaptively; kth surrogate is m k (x ) = ¹ (x ) + ¯ T x
• Evaluate function at “maximum expected
function at maximum expected
improvement location(s)” and update model Bayesian Model Calibration
f high (x) mk (x) f low (x) • Model the error between
a highfidelity and a
lowfidelity function
[Kennedy2000, 2001; Huang2006] • If the lowfidelity function is
“good”, converges faster
• Global calibration procedure 15 k (x) Comparison of Data Fit Methods 16 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics ReducedBasis Methods
Consider r feasible design vectors: x1, x2, ..., xr
We could consider the desired design to be a linear
combination of these basis vectors:
r x* x i x C i1 scalar
coefficient 17 basis
vector added for
generality © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics ReducedBasis Methods
We can now optimize J(x) by finding the optimal values for
the coefficients i.
dimension n
dimension r 18 • Do one fullorder evaluation of resulting answer
• Approach is efficient if r << n
• Will give the true optimum only if x* lies in the span of {xi}
• Basis vectors could be
– previous designs
– solutions over a particular range (DoE)
– derived in some other way (e.g., proper orthogonal
decomposition) © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics ReducedBasis Example
Example using a reducedbasis approach (van der
Plaats Fig 72): airfoil design for a unique application.
• Many airfoil shapes with known performance are
available
• Design variables are (x,y) coordinates at chordwise
locations (n~100)
• Use four basis airfoil shapes (lowspeed airfoils) which
contain the n geometry points
• Plus two basis shapes which allow trailing edge
thickness to vary
• r=6 (r<<n)
• Optimize for high speed, maximum lift with a constraint
on drag
19 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics ReducedBasis Example
Airfoil basic shapes. From Vanderplaats
Figs. 72 and 73,
pg. 260 NACA 2412
Basic shape 1
NACA 64, 412
Basic shape 2
NACA 65, 415
Basic shape 3 Basis functions NACA 65, A215
Basic shape 4 Basic shape 5 yx/c
y0
y0 Basic shape 6 yx/c result of optimization
using reduced basis
Minimumdrag airfoil.
Image by MIT OpenCourseWare. 20 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Proper Orthogonal Decomposition
(aka KarhunenLoève expansions, Principal Components Analysis,
Empirical Orthogonal Eigenfunctions, …) Consider K snapshots
(solutions at selected times or parameter values)
Form the snapshot matrix
Choose the n basis vectors
to be left singular vectors of the snapshot matrix, with
singular values
This is the optimal projection in a least squares sense: 21 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Multifidelity Methods
• Sometimes there is more than one model for
the same system
– e.g. Navier Stokes and thinairfoil theory for wing
design, finiteelement and beam theory for structural
design • Lowfidelity model may provide good
information over a wide range, at much lower
computational cost
• Would like to find optimum of highfidelity
problem, but use lowfidelity model most of the
time
22 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics A Hierarchy of Models Images of Figure 1b, 4, and 8 removed due to copyright restrictions.
Figures from: Choi, S, Alonso, JJ, Kim, S., Kroo, IM. Twolevel multifidelity
design optimization studies for supersonic jets. 43th AIAA Aerospace Sciences
Meeting & Exhibit. January 2005. Image of Lowfidelity EM and High fidelity EM models
removed due to copyright restrictions. 23 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics TrustRegion Model Management
• A rigorous method for determining when to
use highfidelity function calls
• Solves a series of subproblems:
Minimize ˆ k ( x)
J Subject to ˆ k ( x) 0
g
xx k
c k Several methods
exist to handle the
approximation of
constraints. k
c x : center point of trust region at iteration k
k 24 : size of trust region at iteration k
© Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics TrustTrustRegion Model Management
• Size of trust region updated depending on
how well surrogate predicts highfidelity
function value
• Merit function
function Γ[J (x ), g (x )] • Ratio of actual to predicted improvement: () ()
() () Γ x −Γ x
ρ=
ˆ
ˆ
Γ x −Γ x
k 25 k
c
k
c k
*
k
* © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics TrustRegion Model Management
• Trust region size update rules:
k 0 0 k 0.1 k 0.75 26 Reject step 0.1
0.75
k k1 Accept step
Accept step
Accept step 0.5 k k1 0.5 k k1 k k1 2 k © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics TrustRegion Demonstration 27 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics TrustRegion Model Management
• Calls highfidelity analysis once per iteration
• Calls surrogate analysis many times per
iteration
• Provably convergent to local minimum of high
fidelity function if surrogate is firstorder
accurate at center of trust region
ˆ
• Extensions to the case of x x in Robinson
et al. (2008).
• Derivativefree approaches in Conn et al.
(2009)
28 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Corrections
• Include corrections in order to enforce consistency
and gain provable convergence of trustregion
approach
• Additive Correction: ˆ
J ( x) J lo (x) ( x) • Multiplicative Correction: ˆ
J ( x) surrogate model
29 J lo (x) (x)
lowfidelity model © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Multifidelity Optimization
• Combines several elements:
– Trust regions
– Bayesian model calibration
– Adaptive sampling
– Surrogate models (e.g., interpolation
models using Kriging)
– Estimation theory • Active area of research
30 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics Combining Estimates of Multifidelity
Models
• Use Kalman filtering approach to compute combine estimate
• Maximum likelihood estimate weights each model according to its variance
(pay more attention to models in which we have more confidence)
2
2
¾ow (x)
¾ ed (x)
l
m
¹ est (x) = ¹ med (x) 2
+ ¹ low (x) 2
2 (x)
2
¾ow (x) + ¾ ed
¾ow (x) + ¾ ed (x)
l
m
l
m 1
2
¾est (x) = 1
2
¾low (x) + 1
2
¾med (x) :
• Extends naturally to
case with more than
two models; much
more efficient than
nesting (March 2010) Image by MIT OpenCourseWare. Lecture Summary
• A number of ways to create approximations,
or surrogates
• Each has its own area of application,
advantages, and disadvantages
• Data fit surrogates
– Polynomial response surfaces
– Kriging • Model order reduction
– Reduced basis
– Proper orthogonal decomposition • Multifidelity methods
32 © Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox
Engineering Systems Division and Dept. of Aeronautics and Astronautics References
Alexandrov, N., Dennis, J.E., Lewis, R.M. and Torczon, V., “A trust region framework for
managing the use of approximation models in optimization”, NASA CR201745, ICASE Report
No. 9750, October 1997.
Barthelemy, JF. M. and Haftka, R.T., “Approximation concepts for optimum structural design – a
review”, Structural Optimization, 5:129144, 1993.
Conn, A.R., Scheinberg, K. and Vicente, L., “Global Convergence of General DerivativeFree
TrustRegion Algorithms to First and SecondOrder Critical Points,” SIAM Journal of
Optimization, Vol. 20, No.1, pp. 387415, 2009.
Gill, P.E., Murray,W. and Wright, M.H., Practical Optimization, Academic Press, 1986.
Giunta, A.A. and Watson, L.T.,”A comparison of approximation modeling techniques: polynomial
versus interpolating models”, AIAA Paper 984758, 1998.
Jones, D.R., “A taxonomy of global optimization methods based on response surfaces,” Journal
of Global Optimization, 21, 345383, 2001.
LeGresley, P.A. and Alonso, J.J., “Airfoil design optimization using reduced order models based
on proper orthogonal decomposition”, AIAA Paper 20002545, 2000.
March, A. and Willcox, K., “A Provably Convergent Multifidelity Optimization Algorithm not
Requiring HighFidelity Derivatives,” AIAA20102912, presented at 3rd MDO Specialist
Conference, Orlando, FL, April 1215, 2010.
Robinson, T., Willcox, K., Eldred, M., and Haimes, R. “Multifidelity Optimization for Variable Complexity Design,” AIAA Journal, Vol.46, No.11, pp. 28142822, 2008.
Vanderplaats, G.N., Numerical Optimization Techniques for Engineering Design, Vanderplaats
33
R&D, 1999. MIT OpenCourseWare
http://ocw.mit.edu ESD.77 / 16.888 Multidisciplinary System Design Optimization
Spring 2010 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. ...
View Full
Document
 Fall '03
 lDavidMiller

Click to edit the document details