tuto10_csc5150 - Relative entropy Mutual Information A...

Info iconThis preview shows pages 1–14. Sign up to view the full content.

View Full Document Right Arrow Icon
CSC5150 Learning Theory and Computational Finance Tutorial 10 By Tu Shikui Nov.27, 2007
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Outline Discrete Entropy Entropy Conditional Entropy Some inequalities An example for maximum entropy
Background image of page 2
Discrete Entropy
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Joint entropy & conditional entropy
Background image of page 4
Relations … Or equivalently
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 6
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 8
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 10
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 12
Background image of page 13

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 14
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Relative entropy Mutual Information A graph about the relations You should memorize this graph Conditional relative entropy Relations Some Inequalities (1/2)-- non-negativity Some Inequalities (2/2)-- others An example for maximum entropy End of This Tutorial! Thank you!...
View Full Document

Page1 / 14

tuto10_csc5150 - Relative entropy Mutual Information A...

This preview shows document pages 1 - 14. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online