tuto09_csc5150 - Not always non-negative as the discrete...

Info iconThis preview shows pages 1–9. Sign up to view the full content.

View Full Document Right Arrow Icon
CSC5150 Learning Theory and Computational Finance Tutorial 09 By Tu Shikui Nov.06, 2007
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Outline Differential Entropy Entropy of Uniform distribution Entropy of Normal (Gaussian) distribution KL-divergence (or Relative Entropy) Some basic properties of KL-divergence
Background image of page 2
Differential Entropy
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Example 1 -- Uniform distribution
Background image of page 4
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Background image of page 6
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Background image of page 8
Background image of page 9
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Not always non-negative as the discrete entropy does . Example 2 – Gaussian distribution (1/2) For univariate Gaussian distribution: Example 2 – Gaussian distribution (2/2) For multivariate Gaussian distribution: Kullback Leibler (KL) divergence Proof of theorem End of This Tutorial! Thank you!...
View Full Document

{[ snackBarMessage ]}

Page1 / 9

tuto09_csc5150 - Not always non-negative as the discrete...

This preview shows document pages 1 - 9. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online