oxdk_day3a - A d v a n c e d P r o g r a m m i n g i n Q u...

Info iconThis preview shows pages 1–8. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: A d v a n c e d P r o g r a m m i n g i n Q u a n t i t a t i v e E c o n o m i c s Introduction, structure, and advanced programming techniques 17 – 21 August 2009, Aarhus, Denmark Charles Bos cbos@feweb.vu.nl VU University Amsterdam Tinbergen Institute Advanced Programming in Quantitative Economics – p. 1 Day 3 - Morning 9.00L Optimization ◦ Newton-Raphson and quadratic optimisation ◦ Hessian: Importance and problems ◦ Loglikelihood and covariance ◦ MaxBFGS 10.30P Estimating a duration model ◦ Likelihood ◦ Optimization ◦ Covariance 12.00 Lunch Advanced Programming in Quantitative Economics – p. 2 Maximization: Theory Rosenbrock function: g ( x ) = 100 ∗ ( x 2 − x 2 1 ) 2 + (1 − x 1 ) 2 f ( x ) = − g ( x )-2-1.5-1-0.5 0.5 1 1.5 2 x 1-0.5 0.5 1 1.5 2 2.5 3 3.5 4 x 2 0.001 0.01 0.1 1 10 100 1000-0.5 0.5 1 1.5 2 2.5 3 3.5 4-2-1.5-1-0.5 0.5 1 1.5 2 x 2 x 1 Minimum of g ( x ) at (1, 1) — Steep function — minimum in ‘narrow crooked alley’ Advanced Programming in Quantitative Economics – p. 3 Wrong approach: Purely random Suppose: • Search in area [ − 10 , 10] 2 • Throw darts randomly • Continue until you get precision ǫ = 0 . 25 from minimum, in both coefficients How many darts would you need? P ( | x i − x ∗ i | < ǫ ) = 2 ǫ 20 = . 025 P ( | x i − x ∗ i | < ǫ,i = 1 , 2) = P ( | x i − x ∗ i | < ǫ ) 2 = 4 ǫ 2 20 2 = 0 . 000625 E ( n ) = 1 /p = 1600 Way too many... See rb_darts.ox Advanced Programming in Quantitative Economics – p. 4 Better approach: Use function characteristics • Start in some point x ( k ) • Choose a direction s • Move distance α in that direction, x ( k +1) = x ( k ) + αs • Increase k , and possibly continue from 5 Direction s : Linked to gradient? Optimum: Gradient 0, second derivative positive definite? Advanced Programming in Quantitative Economics – p. 5 Ingredients f ( x ) : ℜ n → ℜ Function f ′ ( x ) = ∇ f ( x ) = bracketleftbigg ∂f ( x ) ∂x 1 ,... ∂f ( x ) ∂x n bracketrightbigg T ≡ g Derivative, gradient, Jacobian f ′′ ( x ) = ∇ 2 f ( x ) = bracketleftbigg ∂ 2 f ( x ) ∂x i ∂x j bracketrightbigg n i,j =1 ≡ H Second derivative, Hessian If derivatives are continuous, then ∂ 2 f ( x ) ∂x i ∂x j = ∂ 2 f ( x ) ∂x j ∂x i H = H T Hessian symmetric Advanced Programming in Quantitative Economics – p. 6 Newton-Raphson • Approach f ( x ) locally with quadratic function f ( x + h ) ≈ q ( h ) = f ( x ) + h T f ′ ( x ) + 1 2 h T f ′′ ( x ) h • Minimise q ( h ) (instead of f ( x + h ) ) q ′ ( h ) = f ′ ( x )+ f ′′ ( x ) h = 0 ⇔ f ′′ ( x ) h = − f ′ ( x ) or Hh = − g by solving last expression, h = − H − 1 g • Choose x = x + h , and repeat as necessary Problems: • Is H positive definite/invertible, at each step?...
View Full Document

This note was uploaded on 05/28/2010 for the course ECONOMIC FM407 taught by Professor Kimcj during the Spring '04 term at 카이스트, 한국과학기술원.

Page1 / 31

oxdk_day3a - A d v a n c e d P r o g r a m m i n g i n Q u...

This preview shows document pages 1 - 8. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online