Chapter 2
Mathematical preliminaries
for lossless compression
A brief introduction to information theory
ELG5126 Fall 2014
02-2
Example 2.2.1 flipping a coin
ELG5126 Fall 2014
02-3
Terms
Alphabet: the set of symbols
A=cfw_1,2,m
Letter: each symbol in t

Chapter 5
Dictionary techniques
Rational
In previous two chapters, we looked at coding
techniques that assume a source that generates a
sequence of independent symbols.
Most sources are correlated, thus, the coding step is
generally preceded by a de-cor

pixels.
- Terminating codes: used to represent run lengths of less than 64
pixels.
- E.g., the run length of 132 white pixels is encoded by the following
two codes:
- Makeup code for 128 white pixels - 10010
ExampleTerminating code for 4 white pixels - 10

ELG7177A Source Coding and Data Compression
Midterm exam
Date: Feb. 29, 2000
Time: 11:30-12:50
Professor: E. Dubois
Closed-book exam: you may not use any books or notes. You may use a calculator.
Explain all calculations; I am more interested in the reaso

Chapter 10
Subband coding
Concept of Subband Coding
In transform coding, we use N (or NN) samples as the
data transform unit
Transform coefficients are de-correlated data each
describing different characteristics of the original
data
Different coeffici

ELG5126 Source Coding and Data Compression
Term Project, Fall 2012
October 16, 2012
This project should address a subject related to waveform compression. The goal of the project is to present
a computer implementation of a compression scheme based on a t

Long-term trend
Short-term, sample-to-sample variation
Given a sequence cfw_xn, it can be decomposed into two
types of behaviors
Long-term trend
Short-term, sample-to-sample variation
#1
x5
x3
p.3
ELG5126 Fall 2014
#2
x7
10-3
ELG5126 Fall 2014
10-3
E

ELG5126 Source Coding and Data Compression
Fall 2012
Assignment 1
Due Oct. 18, 2012
You may (should) use a tool like MATLAB to do this assignment, but provide the code with your
submission. To see how MATLAB can be used for this assignment, see the two m-

x10
x2
x2
x1
x1
Entropy of finite state process
ELG5126 Fall 2014
02-9
#1
p.9
ELG5126 Fall 2014
02-14
#2
p.14
Example 2.3.1: i.i.d. vs Markov model
Uniquely decodable codes
#3
p.15
#4
p.18
ELG5126 Fall 2014
02-15
Instantaneous codes
Instantaneous codes Fa

Question5:
Ans:
1) Difference Image, d(x,y) = I(x,y) I(x-1,y)
Following is a Matlab program to implement above equation:
Original Lena image
Difference Image
Original Lena image histogram
Difference image histogram
From histogram figure of difference imag

ELG5126 Source Coding and Data Compression
Fall 2012
Assignment 2
Due Nov. 22, 2012
1. A simple computer can execute four instructions cfw_ADD, SUB, MPY, STO which are represented by
the four codewords (00, 01, 10, 11).
(a) After an analysis of several pr

Chapter 4
Arithmetic coding
About Large Block Coding
Huffman coding is inefficient if the probability model is
biased (e.g. Pmax > 0.5). Although extended Huffman
coding fixes this issue, it is expensive:
The codebook size increases exponentially w.r.t.

- The wave is made up of pressure differences. Sound is detected by
measuring the pressure level at a location.
- The color signal to the brainand CMY: the(reflection, of the 3 cones
Sound waves have normal comes from
Conversion between RGB wave propertie

Four-symbol FLC, fixed-length encoding: 20 bps (324)
If we assume uneven distribution of the symbols
Pick a dictionary which contains the 256 most-frequent
patterns (probability p) and encode them with 8 bits
LZ77 Encoding Example
Encode the rest with

Color, Gray Scale, and Still-Video Image Compression
(JPEG)
1. Overview of JPEG
What is JPEG?
- Joint Photographic Expert Group. Voted as international standard in
1992.
- Works with color and grayscale images, e.g., satellite, medical,
Motivation
- The

Jayant Quantizer
Output Levels of 3-bit Jayant Quantizer
#1
x4
N. S. Jayant showed in 1973 that
adjustment based on few
observations still works fine:
The multipliers are symmetric:
M current input falls =M , M =Mlevels, expand step size
If 0=M4, M1=M

Eric Dubois
Information
Source
signal
Encoder
binary stream
Channel
Information
Receiver
signal
Decoder
binary stream
Information
Source
signal
Encoder
aka data
Information
Receiver
signal
binary stream
Channel
Decoder
binary stream
Information
Source
err

Example 4.3.2
x10
#1
#2
Considering a simple dice-throwing experiment with a
fair die. P(X=k)=1/6 for k=1,2,3,4,5,6
Recursive Computation of Tags (1/3)
Tx(1)=0.0833
Tx(2)=0.25
Tx(3)=0.4166
Tx(4)=0.5833
Tx(5)=0.75
Tx(6)=0.9166
Recursive Computation

Chapter 9
Transform coding
Transform domain data analysis
Given an invertible transform A, the entropy of a source
x does not change subject to A, i.e. Ax has the same
entropy as x.
However, there are several reasons why we want to
perform lossy compres

Example: height-weight data (3/3)
x9
#1
Note that, in original data, both xh and xw have nonnegligible variances, however, for 0 and 1, only 0 has
large variance
ELG5126 Fall 2014
09-4
Variance (or energy) of a source and its information has
a positive

Audio Compression
1. Simple Audio Compression Methods
Traditional lossless compression methods (Huffman, LZW, etc.) usually do
not work well on audio compression (the same reason as in image
compression).
The following are some of the Lossy methods:
Silen

Chapter 8
Scalar and Vector
Quantization
Basic Concept of Quantization
ELG5126 Fall 2014
08-2
The Quantization Problem
Encoder mapping
Map a range of values to a codeword
Irreversible mapping
If source is analog
A/D converter
Decoder mapping
Map the

Video Compression
Uncompressed video data are huge. In HDTV, the bit-rate
could exceed 1 Gbps. -> big problems for storage and network
communications.
We will discuss both Spatial and Temporal Redundancy
Removal - Intra-frame and Inter-frame coding.
1. H.

Chapter 6: Audiovisual Signals
6.1. Basics of Digital Audio
Facts about Sound
- Sound is a continuous wave that travels through the air
- The wave is made up of pressure differences. Sound is detected by
measuring the pressure level at a location.
- Sound

Color, Gray Scale, and Still-Video Image Compression
(JPEG)
1. Overview of JPEG
What is JPEG?
- Joint Photographic Expert Group. Voted as international standard in
1992.
- Works with color and grayscale images, e.g., satellite, medical,
Motivation
- The