2-taylors series corrections

2-taylors series corrections - Function Approximation Using...

Info iconThis preview shows pages 1–11. Sign up to view the full content.

View Full Document Right Arrow Icon
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 4
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 6
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 8
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 10
Background image of page 11
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Function Approximation Using Taylor's Series We frequently have need to approximate a function in some region of a given point to obtain a simpiifled function. At times we will only know values of the actual function at a finite number of points, hence, the approximation will then provide a functional relationship over a larger set. There are three primary classes of functions which are normally used in functional approximations. Linear combinations of elements from the basic function classes are used to construct the approximating function. For our purposes the class of polynomials, {1, x, x2, x3, -- -}, will suffice and a particular method of function approximation using polynomials, the Taylor's series method, is an important tool in the field of optimization. We will illustrate and use two other methods of polynomial approximations in the next chapter, namely, the method of ieast squares and the Lagrange interpolation method. The utility of polynomials as approximating functions is due in large part to the relative ease of obtaining these estimating functions, and the ability of polynomials to generate arbitrarily good approximations to general functions. A function is said to be uniformly approximated over a set if the function and approximation differ by no more than a specified value independent of the point in the set. The Stone—Weierstrass Theorem is the basis of this accuracy. Stone-Weierstrass Theorem. Every continuous function on a closed and bounded set X in E" can be uniformly approximated on X by a polynomial in the coordinate variables. This theorem is actually a coroilary to the Stone—Weierstrass Theorem as given by Royden (H. L. Royden, Real Anaiysis, The Macmillan Co, 1963) but is the form of particular importance to our usage of polynomiai approximations. The single variable case of this theorem (named after the German mathematician Karl Weierstrass (1815- 1897) ) is worth stating: Weierstrass Approximation Theorem. Let f be a continuous function on the finite interval [a, b} and let a be any positive real number. Then there exists a polynomial function p such that for all x E [51, b] Inn—poi < 8- These two theorems provide the basis for polynomial approximations of general continuous functions. Some points to consider in polynomial approximations: (1) For any a, there exists a polynomial of degree n such that [f(x)— p(x)| < 8 for every x E[a, b]. (2) As 8 decreases 1": increases. The Weierstrass theorems assure us that we can, in theory, achieve any level of accuracy. There is a reasonably general method of developing a polynomial approximation for a function of a single variable for which we can compute the error. This method known as the Taylor's Series Method (named after the English mathematician Brook Taylor (1685—1731)) requires that the function be approximated be of class C "H1 (has at least m +1 continuous derivatives). The Taylor's series polynomial approximation of degree m of f , call this function pm, about some point 1?: is given as f(x) = Fulfil?) + 90656), (1) where pm(x;5c) and the error term, e(x;5c), are given by m (i) " __ A 1‘ pm(xi'£)=f()3) +ZW, f{m+l)(6£. + _ 6)x)(x _ gym-l ( I), ,withOSBSl. m+ . e(x;£) 2 Note that the error term is of order m +1. Example I. To illustrate a Taylor's series approximation, consider approximating 92" over the interval [0, 1]. We note that f""(x) = 2‘3“. We must choose some point, 5:, preferably in the interval at which to base the approximation. For illustration, we use a? = 0.5, and equation (I) specifically becomes an ZieZU f2)(x _ 1 pm(x;] = 9200} + Z 'l i=l 1' Then the first, second and third order approximations are: p,(x;1/2)= e‘ +2e‘(x 4,12)1 5 0 + 5.4366x, [2206;1/2): el +2310: —1/2)1+ 226‘(x—1/2)2/2 g 0 + 5.4366x + 5.4366(x —t/2)2, and p3(x;1/2) = 91+ 291(x — 1/2)E +f212r31(x—1/2)2 l2 +23el(x—1/2)3/6 g 0 + 5.4366x + 5.4366(x —1/2)2 + 3.6244(x-1/2)3. These three approximating functions are piotted against the actual function in Figure 1. Changing the approximation base point from 1/2 to the left—end point of the interval, 0, results in the approximations displayed in Figure 2. The approximations of the first figure are closer since the error is a function of the distance from the base point and the maximum distance from the base point of the second figure is twice that of the first. *3 x Example 2. As another example, expand f (x) = 6 about x = 0: no) =e’3‘°’ 2 1; 1%): .3340) :4; = +98u3(0) : : _27e-3(0) = 9 2 27 3 ’" (—3x)’r x;0 E1—3x+—x ——x +---= . pm( ) 2 kg) 6 k! Multipie Dimensional Taylor's Series To generalize the Taylor's series approximation technique from one to n dimensions, we utilize a method of generating a line, in :7 dimensions which includes the base point and the point at which the function vaiue is desired. We make the assumption that the function f EC’MLS') where S is an open subset of R". Let i and i be these two points in R", respectively. Then we create a line by considering a scaling parameter A and defining x=§+/AL(E—i), for -oo<,’l,<oo’ or letting £2 2 3c": — 32 we have 3g = QC: + 39’: . Now consider approximating a function of 17 variables, f : R" —> R , from a point 3 ER" along the line by varying xi. Then f (j + 19?) is a function, call it g, of the single variable A. Thus, for A ER, go) = fli + xiii). We can now apply the single variable Taylor's series expansion to 300 using 7: = 0 as the base point yielding an r' go) :gm) + 2&0); + aw). Note that g( 0) = f (5:), and the derivatives gm(0) are developed as foiiows: went/14“): " 5f(£+itc?)fi a) A = g ( ) cm taxi (M _ " é’f(g§+}ici)& i=1 595: H and, ” 5N?) A 0’ 0 = d.. g () axi , Note that this term is the direction derivative of f at i in the direction 9’: . Now for the second derivative term, g(2)(0), ” é’f(i+/1c_?)dx,. 2 A A d 2W elk/miflifl‘flsw 0’23» d/i I? taxi = 2 i2] d)» n n a? A A = E i=1j=1 5 xi"; x j and, g(2J(0) : H F; fixrflxj Note here that the above term consists of a matrix called the Hessian of f at I} , commonly 2 A denoted by , V fix) , dot product with the direction vector d , so this term can be written T 2 " as d 'V fix) ' d. In a like manner, the general term coefficient, gm(0), is (k) I? I? g il=i ik=laxi1'”ax' The resulting quadratic approximation of g(?t) is go) =g(0)+g‘”(0)h+1/2g‘”(0)9f, which in terms of the function f is A n A " aflt) A 1 n H 0‘2de M 2 +/id a: + d.a+— d'd'fl' fa m) an {:1 ax}. r riggiaxié‘xi ” Note that this formula gives the approximation of the multiple variable function f along the line 3": + Ml. To approximate an arbitrary point x e R" near it, we use the direction d = x— it with 7L equated to one. This yields the second order Taylor‘s approximation to multiple variable functions utilized extensively throughout this text n A n n 7 "‘ for) s f(-§)+ Z a“) (xi- —a)+§z Z #1“) (xi -~ti)(xj 41')- i=1 5x: i=lj=1§xi§xj Quadratic functions play a fundamental role in algorithm development and are frequently referred to in matrix form. We, therefore, give the matrix form A A A 1 A T A A fit) = fee.) + Vf(§)T(2r u a) + is - a) VZflrXa — at), where Vf and sz are the gradient and Hessian of fat )3, respectively. The quadratic terms of the approximation are called a quadratic form (frequently the use of this term is limited to just the second order terms) and these are used to characterize the shape of the quadratic function. Example 1. We illustrate the quadratic approximation of the two-dimensional Himmelblau function graphically displayed in the previous chapter. Figure 3 displays the four minima and a quadratic approximation to the minimum near the point (3, 2). From the figure it is clear that the quadratic approximation is not a very good representation of the total function. But, in the neighborhood of the base point, the approximation is relatively good. Figure 4 displays the function and approximating function in an expanded fashion about the base point, and it is clear from this graph that the quadratic approximation is good near the base point. The quadratic approximation to Himmelblau‘s function, f(x,y>=<x2+y—11)2+(x+y2—7)2. at the point (3, 2) is created using Vf(x,y) =(4x3 + 4x 2—42x+2y2 —14, 2x2 +4xy+ 4y3 —26y—22), and 12x2+4ym42 4x+4y ] V2 x, = fl 3’) i 4x+4y 4x+12y2~26 yielding x_3 1 74 20 x— f(x,y)20+(0, 0)- ~2l+§(x‘3’ y‘2)[20 Bulb—2] ~ 1 3 2[74 20] x—3) =+2(x“’ 3" )20 34 —2' Many of the algorithms that we study in this text are developed from the quadratic function, E9 ER", Q E R" x R”, _ ,. 1 .. fir) =C+Er+§£QL partly because these functions have such simple structure. Algorithms developed in this manner are adapted to more general functions by using second—order Taylor's series approximations. Re-interpreting the Stone—Weierstrass Theorem, we can approximate any continuous function by a quadratic polynomial if the neighborhood about jg is sufficiently small. Keep these ideas in mind as we consider algorithms throughout the remainder of this text. Example 2. As a second example, expand up to second—order terms the function 4 f(x=J’) = (x — 3) + (y _ 2f + 3‘2)” about the point (1, 2): a ‘ a —f=4(x—3)3+2xy; —f=3(y—2)2+x2; 8x é’y 6L§=12<x~3f +2y; azf=6e~2x é‘x 6y jf =2x; é’xfly and f(x,y) g 18—28(x— 1) + 1(y—2)+26(x— 1)2 +2(x —])(y—2)+ 00x32. Example 3. As a last example, we write the matrix form for the quadratic funetionfof three variables f(xl,x2,x3) =3 “2x1 + x2 w 4x3 +2x12 #295le +6352 + 4xlx3 +10x32 in the matrix form: 1 4 —2 4 f(£):3_(23_194)£+§2€T _2 0 x. 4020 f-auis f-axis Rpm'oxlnat ion of Euptaul Figure 1. Taylor polynomials of degrees 1, 2, and 3 approximations to 62X using 1/2 as the base point. approximation of EupthJ Figure 2. Taylor polynomials of degrees 1, 2, and 3 approximations to 62x using 0 as the base point. 3 E E 9‘31: 18 Contour of Hlnnalblau Figure 3. Contour graph of the Himmelblau function and a second-order Taylor's series approximation about the point (3, 2). Contour of Hinnalblau 5.00 4.50 4.00 3.50 3.00 2.50 2.00 1.50 -- 1.00 g-axis 0.50 0.00 Figure 4. Isolated contour graph of the Himmelblau function and a second-order Taylor's series approximation about the point (3, 2). Problems 1. Consider the function f (x) = aeubx. Obtain the first three terms of the polynomial expansion off at the point 22 = l. 2. Consider the function f (x, y) = 0x2 + ye"). + y2. Obtain the terms up to and including the quadratic terms by expanding the function about the point (l, 1). 3. Expand the following function about the stationary point (£1, £2), a stationary point is one Vffih 552) m (0, 0), f (x1 ,x2) = a + bl):l + 62x2 + 0le + 02x12;2 + 63x22. 10 4. Show that if the stationary point 32 E R2 of Problem 3 is to be a minimum point, then the following conditions must hold: (Hint: complete the square!) 01>0, (c3 > 0), 40:03 >022. It is apparent that another statement of these conditions concerning the Hessian of f, the Hessian offis sz, l C: ‘62 sz:2 1 2 a _C2 C3 2 is that the principal-minor determinants of the Hessian must all be positive. 11 ...
View Full Document

This note was uploaded on 10/26/2011 for the course ISEN 620 taught by Professor Curry during the Fall '10 term at Texas A&M.

Page1 / 11

2-taylors series corrections - Function Approximation Using...

This preview shows document pages 1 - 11. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online