Chapter27_TAMU.pdf - ECEN 489 Information Theory Inference and Learning Algorithms Chapter 27 Laplace’s Method Dr Chao TIAN Texas A&M University 1/7

# Chapter27_TAMU.pdf - ECEN 489 Information Theory Inference...

• 7

This preview shows page 1 - 7 out of 7 pages.

ECEN 489: Information Theory, Inference, and Learning Algorithms Chapter 27: Laplace’s Method Dr. Chao TIAN Texas A&M University 1 / 7
Gaussian Approximation of Probability Distribution Approximate ln P * ( x ) using a Gaussian distribution An unnormalized probability density P * ( x ) (does not sum to 1) We can write the normalizing coefficient as Z P , Z P * ( x ) dx ; that has a peak at x 0 ; Taylor-expand the logarithm of P * ( x ) ln P * ( x ) ln P * ( x 0 ) - c 2 ( x - x 0 ) 2 + ... where c = - 2 ln P * ( x ) x 2 | x = x 0 . 2 / 7
Gaussian Approximation of Probability Distribution Approximation using the obtained parameter Approximate P * ( x ) by a Gaussian Q * ( x ) = P * ( x 0 ) exp - c ( x - x 0 ) 2 2 with the normalizing parameter Z Q = P * ( x 0 ) r 2 π c 3 / 7
Generalizing to K -dimensional Distribution For K -dimensional distribution P * ( x ) A i , j = - 2 ln P * ( x ) x i x ) j | x = x 0 . and the generalization is Q * ( x ) , P * ( x 0 ) exp - 1 2 ( x - x 0 ) T A ( x - x 0 ) , with the normalizing constant

#### You've reached the end of your free preview.

Want to read all 7 pages?

• Fall '19

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern