{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

L14-informationtheory-2009

# L14-informationtheory-2009 - Information Theory Trac D Tran...

This preview shows pages 1–4. Sign up to view the full content.

1 Information Theory Information Theory Trac D. Tran ECE Department The Johns Hopkins University Baltimore, MD 21218 Outline Outline ± Probability ² Definition ² Properties ² Examples ² Random variable ± Information theory ² Self-information ² Entropy ² Entropy and probability estimation ² Examples Dr. Claude Elwood Shannon

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
2 Deterministic versus Random Deterministic versus Random ± Deterministic ² Signals whose values can be specified explicitly ² Example: a sinusoid ± Random ² Digital signals in practice can be treated as a collection of random variables or a random process ² The symbols which occur randomly carry information ± Probability theory ² The study of random outcomes/events ² Use mathematics to capture behavior of random outcomes and events Probability Probability ± Events and outcomes ² Let X be an event with N possible mutually exclusive outcomes ² Example ² A coin toss is an event with 2 outcomes: Head ( H ) or Tail ( T ) ² A dice toss is an event with 6 outcomes: {1,2,3,4,5,6} ± Probability ² The likelihood of observing a particular outcome above ² Standard notation { } N X X X , , , 2 1 K i X [ ] i X X P =
3 Important Properties Important Properties ± Probability computation or estimation ± Basic properties ² Every probability measure lies inclusively between 0 and 1 ² Sum of probabilities of all outcomes is unity: ² For N equally likely outcomes ² For two statistically independent event [] outcomes of number total outcomes possible of number i total i i X N N X X P = = = [ ] i X X P i = , 1 0 = = = N i i X X P 1 1 [ ] N X X P X X P X

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 8

L14-informationtheory-2009 - Information Theory Trac D Tran...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online