ECE 6250: Advanced Topics in DSP Syllabus for Fall 2009
Instructor: Justin Romberg Updated: August 24, 2009
Summary
ECE 6250 is a general purpose, advanced DSP course designed to follow an introductory DSP course.
Prerequisites
A senior-level introductory
[1/2, 1/2], we can write
t
k tk ,
e =
k=0
k
k t
log(1 + t) =
k=0
where k =
(1)k+1
where k =
,
k
k tk where k =
sin(2t) =
1
,
k!
k=0
(1)(k+3)/2 (2)k+1
(k+1)!
k odd
k even
0
Here are the three examples above with the series truncated to the
rst six terms:
e
Linear combinations and spans
Let M = cfw_v 1, . . . , v N be a set of vectors in a linear space S.
Denition: A linear combination of vectors in M is a sum of
the form
a1v 1 + a2v 2 + + aN v N
for some a1, . . . , aN F.
Denition: The span of M is the set
It is important to realize that Taylor series is not the only way to
build up a function as a sum of polynomials, and despite its convenience, it has a few unsatisfying properties (e.g. there are innitely
dierentiable functions whose Taylor series converg
b1 (t k), k = 6, . . . , 10
0.16
0.14
0.12
0.1
0.08
0.06
0.04
0.02
0
0.02
6
4
2
0
2
4
6
8
10
Notice that we can generate b1 by convolving b0 with itself:
b1(t) = (b0 b0)(t).
The expansion for the piecewise quadratic spline above is a little
more complicat
A rst look at basis expansions
In the last lecture, we looked at the sampling theorem from the point
of view of frequency-domain transformations. This is a very good way
to understand it in the context of classical signal processing. We can,
however, thin
Example: Fourier series
Recall that any periodic signal can be written as a (possibly innite)
superposition of harmonic sinusoids. If x(t) has period T , we
can write
k e j2kt/T ,
x(t) =
(1)
k=
1
where k =
T
T /2
x(t) ej2kt/T dt.
(2)
T /2
(The integral ab
Example: Polynomials
It is almost too obvious that any mth order polynomial can be written
as a super position of the m + 1 functions 1, t, t2, . . . , tm. (Indeed,
this is pretty much the denition of polynomial.)
For example:
1
t3 17 t2 + 5 t 12
12
8
17
3.5
3
2.5
2
1.5
1
0.5
0
0.5
1
1.5
6
4
2
0
2
4
6
8
10
Any th order polynomial spline can be written as a superposition
of B-spline functions (the B is for basis!).
The piecewise constant function above can be written as
4
k b0(t k),
x(t) =
b0(t) =
k=1
1 1/
We know how to compute these complementary (t), but they are
b
not easy to write down as nice expressions.
Bases and discretization
All of the examples above have a common theme: we take a signal in
a certain class (bandlimited, zero outside of [0, T ], p
To see this, suppose that x(t) is zero outside of [T /2, T /2], so (1)
is building it up only inside this interval. Then its Fourier transform
is
T /2
x(t) ejt dt.
X(j) =
T /2
Notice that the Fourier series coecients k in (2) are samples of the
Fourier tr
We will start by reviewing one of the foundational results of digital
signal processing: the Shannon-Nyquist sampling theorem. We will
use this result as a rst example of how continuous-time signals can be
systematically discretized (translated into a dis
A little more on the X(ej ) X(ejT ) step .
What we are doing is taking a discrete sequence x[n] (with DTFT
X(ej ) and turning it into a function xa(t) (with CTFT Xa(j) =
X(ejT ) of a continuous time variable.
Set
x[n] (t nT )
xa(t) =
n=
Then
Xa(j) =
So th
Anti-aliasing lters
Suppose the spectrum of xc(t) looks like
Compare the outputs of these two systems:
where
Ha(j) =
1 | /T
.
0 | > /T
Which is closer to xc(t)?
That is, which is smaller:
2
|xc(t) x1(t)| dt or
2
|xc(t) x2(t)| dt ?
12
Georgia Tech ECE 6250
Questions:
1. When can you reconstruct xc(t) perfectly from its samples?
2. How do you do it?
Answers:
1. When xc(t) is bandlimited, i. e. when
Xc(j) = 0 for all | > /T
where Xc(j) is the continuous time Fourier transform (CTFT)
of xc(t):
xc(t)ejt dt
Xc(j
Just as before, we can generate the basis function b2(t) from the lower
order ones:
b2(t) = (b1 b0)(t) = (b0 b0 b0)(t).
In general, any th order polynomial spline x(t) is uniquely represented by a list of numbers cfw_k , k Z, which correspond to the
weigh
in the next section with precise (but abstract) denitions of linear
vector space, norm, and inner product.
24
Georgia Tech ECE 6250 Notes by J. Romberg
4. S = cfw_x(t) : x(t) is periodic with period 2
A basis for S is B = cfw_ejkt
k= (Fourier Series)
S is innite dimensional.
5. S = GF (2)3 (length 3 bit vectors with mod 2 arithmetic).
A basis for S is
1
v 1 = 1 ,
0
0
v 2 = 1 ,
0
0
v 3 = 0 .
1
How w
Bases
Denition: A basis of a linear vector space S is a (countable) set
of vectors B such that
1. span(B) = S
2. B is linearly independent
The second condition ensures that all bases of S will have the same
(possibly innite) number of elements.
The dimens
A rst look at basis expansions
In the last lecture, we looked at the sampling theorem from the point
of view of frequency-domain transformations. This is a very good way
to understand it in the context of classical signal processing. We can,
however, thin
Orthogonal bases
A collection of vectors cfw_v 1, v 2, . . . , v N in a nite dimensional vector
space S is called an orthogonal basis if
1. span(cfw_v 1, v 2, . . . , v N ) = S,
2. v j v k (i.e. v j , v k = 0) for all j = k.
If in addition the vectors ar
Linear approximation in a Hilbert space
Consider the following problem:
Let S be a Hilbert space, and let T be a subspace of S.
Given a x S, what is the closest point x T ?
x
x
T
In other words, nd x T that minimizes x x ; given x, we
want to solve the fo
Linear Algebra has become as basic and as applicable
as calculus, and fortunately it is easier.
Gilbert Strang
Linear signal spaces (vector spaces)
A vector space is simply a collection of things that obeys certain
abstract (but mostly familiar) algebrai