Group 2 - andChainRule Overview another...

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
Entropy of Symbol Blocks  and Chain Rule What is it in particular?
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Overview The entropy of symbol blocks considers  the information content of a block of  symbols rather than one symbol after  another. In this consideration, the entropy is a joint  entropy where its value depends on the  joint probability of the symbols in the  block. Reference:  . Pearson, 1999.
Background image of page 2
Overview If the symbols in the sequence (or block)  are statistically independent, the source is  called a "memoryless source," meaning it  doesn't "remember" from one time to the  next what symbols it has previously  emitted.
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Overview The Chain Rule for Entropy says that the  entropy for a symbol block is equal to the  entropy of the first symbol in the alphabet  plus the entropy of the second symbol in  the alphabet given the first symbol, and so  on, through the entropy of the last symbol  given all symbols in the alphabet before  that. [H(A 0 , A 1 , . .., A (n-1) ) = H(A 0 ) + H(A 1 |A 0 + H(A 2 |A 0 , A 1 ) + . .. + H(A (n-1) |A 0 , . .., A (n-2) )].
Background image of page 4
Overview Since the entropy of source B given  source A is less than or equal to the  entropy of source B, the entropy of an  entire symbol block is less than or equal  to the entropy of an entire alphabet (from  which all the symbols in the symbol block  are drawn); but they are equal only if the  source is memoryless.
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 6
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 07/24/2011 for the course EEL 3531 taught by Professor Llewelyn during the Spring '09 term at University of Florida.

Page1 / 14

Group 2 - andChainRule Overview another...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online