This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Function Approximation Using Taylor's Series We frequently have need to approximate a function in some region of a given point to
obtain a simpiifled function. At times we will only know values of the actual function at
a ﬁnite number of points, hence, the approximation will then provide a functional
relationship over a larger set. There are three primary classes of functions which are
normally used in functional approximations. Linear combinations of elements from the basic function classes are used to construct the approximating function. For our purposes the class of polynomials, {1, x, x2, x3,  }, will sufﬁce and a particular method of function approximation using polynomials, the Taylor's series method, is an important
tool in the ﬁeld of optimization. We will illustrate and use two other methods of
polynomial approximations in the next chapter, namely, the method of ieast squares and the Lagrange interpolation method. The utility of polynomials as approximating functions is due in large part to the
relative ease of obtaining these estimating functions, and the ability of polynomials to
generate arbitrarily good approximations to general functions. A function is said to be
uniformly approximated over a set if the function and approximation differ by no more
than a speciﬁed value independent of the point in the set. The Stone—Weierstrass Theorem is the basis of this accuracy. StoneWeierstrass Theorem. Every continuous function on a closed and bounded set
X in E" can be uniformly approximated on X by a polynomial in the coordinate variables. This theorem is actually a coroilary to the Stone—Weierstrass Theorem as given by
Royden (H. L. Royden, Real Anaiysis, The Macmillan Co, 1963) but is the form of
particular importance to our usage of polynomiai approximations. The single variable
case of this theorem (named after the German mathematician Karl Weierstrass (1815
1897) ) is worth stating: Weierstrass Approximation Theorem. Let f be a continuous function on the finite interval [a, b} and let a be any positive real number. Then there exists a polynomial
function p such that for all x E [51, b] Inn—poi < 8 These two theorems provide the basis for polynomial approximations of general continuous functions. Some points to consider in polynomial approximations: (1) For any a, there exists a polynomial of degree n such that [f(x)— p(x) < 8 for
every x E[a, b]. (2) As 8 decreases 1": increases.
The Weierstrass theorems assure us that we can, in theory, achieve any level of accuracy. There is a reasonably general method of developing a polynomial approximation
for a function of a single variable for which we can compute the error. This method
known as the Taylor's Series Method (named after the English mathematician Brook
Taylor (1685—1731)) requires that the function be approximated be of class C "H1 (has at least m +1 continuous derivatives). The Taylor's series polynomial approximation of
degree m of f , call this function pm, about some point 1?: is given as f(x) = Fulﬁl?) + 90656), (1) where pm(x;5c) and the error term, e(x;5c), are given by m (i) " __ A 1‘
pm(xi'£)=f()3) +ZW, f{m+l)(6£. + _ 6)x)(x _ gyml ( I), ,withOSBSl.
m+ . e(x;£) 2 Note that the error term is of order m +1. Example I. To illustrate a Taylor's series approximation, consider approximating 92"
over the interval [0, 1]. We note that f""(x) = 2‘3“. We must choose some point, 5:, preferably in the interval at which to base the
approximation. For illustration, we use a? = 0.5, and equation (I) speciﬁcally becomes an ZieZU f2)(x _ 1 pm(x;] = 9200} + Z 'l
i=l 1' Then the first, second and third order approximations are: p,(x;1/2)= e‘ +2e‘(x 4,12)1 5 0 + 5.4366x, [2206;1/2): el +2310: —1/2)1+ 226‘(x—1/2)2/2
g 0 + 5.4366x + 5.4366(x —t/2)2, and p3(x;1/2) = 91+ 291(x — 1/2)E +f212r31(x—1/2)2 l2 +23el(x—1/2)3/6
g 0 + 5.4366x + 5.4366(x —1/2)2 + 3.6244(x1/2)3. These three approximating functions are piotted against the actual function in Figure 1.
Changing the approximation base point from 1/2 to the left—end point of the interval, 0,
results in the approximations displayed in Figure 2. The approximations of the first ﬁgure are closer since the error is a function of the distance from the base point and the maximum distance from the base point of the second ﬁgure is twice that of the first. *3 x Example 2. As another example, expand f (x) = 6 about x = 0: no) =e’3‘°’ 2 1; 1%): .3340) :4; = +98u3(0) : : _27e3(0) = 9 2 27 3 ’" (—3x)’r
x;0 E1—3x+—x ——x += .
pm( ) 2 kg) 6 k! Multipie Dimensional Taylor's Series To generalize the Taylor's series approximation technique from one to n dimensions, we
utilize a method of generating a line, in :7 dimensions which includes the base point and the point at which the function vaiue is desired. We make the assumption that the
function f EC’MLS') where S is an open subset of R". Let i and i be these two points in R", respectively. Then we create a line by considering a scaling parameter A and
deﬁning x=§+/AL(E—i), for oo<,’l,<oo’ or letting £2 2 3c": — 32 we have 3g = QC: + 39’: . Now consider approximating a function of 17
variables, f : R" —> R , from a point 3 ER" along the line by varying xi. Then f (j + 19?)
is a function, call it g, of the single variable A. Thus, for A ER, go) = ﬂi + xiii). We can now apply the single variable Taylor's series expansion to 300 using 7: = 0 as the base point yielding an r' go) :gm) + 2&0); + aw). Note that g( 0) = f (5:), and the derivatives gm(0) are developed as foiiows: went/14“): " 5f(£+itc?)ﬁ a) A =
g ( ) cm taxi (M
_ " é’f(g§+}ici)&
i=1 595: H and, ” 5N?) A
0’ 0 = d..
g () axi , Note that this term is the direction derivative of f at i in the direction 9’: . Now for the
second derivative term, g(2)(0), ” é’f(i+/1c_?)dx,.
2 A A d 2W
elk/miﬂiﬂ‘ﬂsw
0’23» d/i I? taxi = 2 i2] d)» n n a? A A
= E i=1j=1 5 xi"; x j and, g(2J(0) : H F; ﬁxrﬂxj Note here that the above term consists of a matrix called the Hessian of f at I} , commonly 2 A
denoted by , V ﬁx) , dot product with the direction vector d , so this term can be written T 2 "
as d 'V ﬁx) ' d. In a like manner, the general term coefﬁcient, gm(0), is (k) I? I?
g il=i ik=laxi1'”ax' The resulting quadratic approximation of g(?t) is go) =g(0)+g‘”(0)h+1/2g‘”(0)9f, which in terms of the function f is A n A " aﬂt) A 1 n H 0‘2de M 2
+/id a: + d.a+— d'd'ﬂ'
fa m) an {:1 ax}. r riggiaxié‘xi ” Note that this formula gives the approximation of the multiple variable function f along
the line 3": + Ml. To approximate an arbitrary point x e R" near it, we use the direction d = x— it with 7L equated to one. This yields the second order Taylor‘s approximation to
multiple variable functions utilized extensively throughout this text n A n n 7 "‘
for) s f(§)+ Z a“) (xi —a)+§z Z #1“) (xi ~ti)(xj 41') i=1 5x: i=lj=1§xi§xj Quadratic functions play a fundamental role in algorithm development and are frequently referred to in matrix form. We, therefore, give the matrix form
A A A 1 A T A A
ﬁt) = fee.) + Vf(§)T(2r u a) + is  a) VZﬂrXa — at), where Vf and sz are the gradient and Hessian of fat )3, respectively. The quadratic
terms of the approximation are called a quadratic form (frequently the use of this term is
limited to just the second order terms) and these are used to characterize the shape of the quadratic function. Example 1. We illustrate the quadratic approximation of the twodimensional
Himmelblau function graphically displayed in the previous chapter. Figure 3 displays the
four minima and a quadratic approximation to the minimum near the point (3, 2). From the ﬁgure it is clear that the quadratic approximation is not a very good representation of the total function. But, in the neighborhood of the base point, the approximation is
relatively good. Figure 4 displays the function and approximating function in an
expanded fashion about the base point, and it is clear from this graph that the quadratic
approximation is good near the base point. The quadratic approximation to Himmelblau‘s function,
f(x,y>=<x2+y—11)2+(x+y2—7)2.
at the point (3, 2) is created using Vf(x,y) =(4x3 + 4x 2—42x+2y2 —14, 2x2 +4xy+ 4y3 —26y—22), and 12x2+4ym42 4x+4y ] V2 x, =
ﬂ 3’) i 4x+4y 4x+12y2~26 yielding x_3 1 74 20 x—
f(x,y)20+(0, 0) ~2l+§(x‘3’ y‘2)[20 Bulb—2] ~ 1 3 2[74 20] x—3)
=+2(x“’ 3" )20 34 —2' Many of the algorithms that we study in this text are developed from the quadratic
function, E9 ER", Q E R" x R”, _ ,. 1 ..
ﬁr) =C+Er+§£QL
partly because these functions have such simple structure. Algorithms developed in this
manner are adapted to more general functions by using second—order Taylor's series approximations. Reinterpreting the Stone—Weierstrass Theorem, we can approximate
any continuous function by a quadratic polynomial if the neighborhood about jg is sufﬁciently small. Keep these ideas in mind as we consider algorithms throughout the remainder of this text. Example 2. As a second example, expand up to second—order terms the function
4
f(x=J’) = (x — 3) + (y _ 2f + 3‘2)” about the point (1, 2): a ‘ a
—f=4(x—3)3+2xy; —f=3(y—2)2+x2;
8x é’y
6L§=12<x~3f +2y; azf=6e~2x
é‘x 6y jf =2x; é’xﬂy and f(x,y) g 18—28(x— 1) + 1(y—2)+26(x— 1)2 +2(x —])(y—2)+ 00x32. Example 3. As a last example, we write the matrix form for the quadratic funetionfof three variables f(xl,x2,x3) =3 “2x1 + x2 w 4x3 +2x12 #295le +6352 + 4xlx3 +10x32 in the matrix form: 1 4 —2 4
f(£):3_(23_194)£+§2€T _2 0 x. 4020 fauis faxis Rpm'oxlnat ion of Euptaul Figure 1. Taylor polynomials of degrees 1, 2, and 3 approximations to 62X
using 1/2 as the base point. approximation of EupthJ Figure 2. Taylor polynomials of degrees 1, 2, and 3 approximations to 62x using 0 as the base point. 3
E
E 9‘31: 18 Contour of Hlnnalblau Figure 3. Contour graph of the Himmelblau function and a secondorder
Taylor's series approximation about the point (3, 2). Contour of Hinnalblau 5.00
4.50
4.00
3.50
3.00
2.50
2.00
1.50 
1.00 gaxis 0.50
0.00 Figure 4. Isolated contour graph of the Himmelblau function and a secondorder Taylor's series approximation about the point
(3, 2). Problems 1. Consider the function f (x) = aeubx. Obtain the ﬁrst three terms of the polynomial
expansion off at the point 22 = l. 2. Consider the function f (x, y) = 0x2 + ye"). + y2. Obtain the terms up to and including
the quadratic terms by expanding the function about the point (l, 1). 3. Expand the following function about the stationary point (£1, £2), a stationary point is
one Vfﬁh 552) m (0, 0), f (x1 ,x2) = a + bl):l + 62x2 + 0le + 02x12;2 + 63x22. 10 4. Show that if the stationary point 32 E R2 of Problem 3 is to be a minimum point, then the following conditions must hold: (Hint: complete the square!) 01>0, (c3 > 0), 40:03 >022. It is apparent that another statement of these conditions concerning the Hessian of f, the
Hessian offis sz, l
C: ‘62
sz:2 1 2 a
_C2 C3 2 is that the principalminor determinants of the Hessian must all be positive. 11 ...
View
Full
Document
This note was uploaded on 10/26/2011 for the course ISEN 620 taught by Professor Curry during the Fall '10 term at Texas A&M.
 Fall '10
 Curry

Click to edit the document details