n
.
f
n
,
[
f
(
f
[
C
(
[
[
*
]
[
]
[
]
*
.
[
.
[
[
[
*
]
[
]
[
]
*
.
•
[
c
a
[
b
[
c
•
[
c
a
2
[
b
2
[
]
•
[
]
Cookbook for data scientists
Fourier analysis
Fourier Transform (FT)
Let
x
:
R
→
C
such that
+
∞
∞

x
(
t
)

d
t <
Fourier transform
X
:
R
→
C
is defined as
X
(
u
) =
F
[
x
](
u
) =
+
∞
∞
x
(
t
)
e

i
2
πut
t
x
(
t
) =
F

1
[
X
](
t
) =
+
∞
∞
X
(
u
)
e
i
2
πut
where
u
is referred to as the frequency.
Properties of continuous FT
F
[
ax
+
by
] =
a
F
[
x
] +
b
F
[
y
]
(Linearity)
F
[
x
(
t

a
)] =
e

i
2
πau
F
[
x
]
F
[
x
(
at
)](
u
) =
1

a

F
[
x
](
u/a
)
(Modulation)
F
[
x
*
](
u
) =
F
[
x
](

u
)
*
(Conjugation)
F
[
x
](0) =
+
∞
∞
x
(
t
) d
t
(Integration)
+
∞
∞

x
(
t
)

2
d
t
=
+
∞
∞

X
(
u
)

2
d
u
(Parseval)
F
[
x
(
n
)
](
u
) = (2
πiu
)
n
F
[
x
](
u
)
(Derivation)
F
[
e

π
at
](
u
) =
1
√
πa
e

u
/a
(Gaussian)
x
is real
⇔
X
(
ε
) =
X
(

ε
)
*
(Real
↔
Hermitian)
Properties with convolutions
(
x
y
)(
t
) =
∞
∞
x
(
s
)
y
(
t

s
) d
s
(Convolution)
F
[
x
y
] =
F
[
x
]
F
[
y
]
(Convolution theorem)
Multidimensional Fourier Transform
Fourier transform is separable over the different
d
dimensions, hence can be defined recursively as
F
[
x
] = (
F
1
◦ F
2
◦
. . .
◦ F
d
)[
x
]
where
F
k
[
x
](
t
1
. . . , ε
k
, . . . , t
d
) =
F
[
t
k
→
x
(
t
1
, . . . , t
k
, . . . , t
d
ε
k
)
and inherits from above properties (same for DFT).
u
[
]
u
n
1
t
t
e
i
2
t
1
[
]
t
1
n
1
u
k
e
i
2
1
e
i
2
*
1
1
/
2
[
a
[
b
[
y
]
[
t
a
e
i
2
[
]
[
*
]
u
[
]
*
n
u
n
[
]
0
n
1
t
t


2
2
1


2
2


1


1


1




1


1


1
u
*
n
u
n
(
*
y
)
t
n
s
s
y
(
t
s
n
*
y
y
y
)
y
(
t
s
n
[
*
y
]
u
[
]
u
[
y
]
u
(
)
(
2
)
1
Cookbook for data scientists
Linear algebra II
Eigenvalues / eigenvectors
If
λ
∈
C
and
e
∈
C
n
(= 0)
satisfy
Ae
=
λe
λ
is called the eigenvalue associated to the
eigenvector
e
of
A
. There are at most
n
distinct
eigenvalues
λ
i
and at least
n
linearly independent
eigenvectors
e
i
(with norm
1
). The set
λ
i
necessarily distinct) eigenvalues is called the
spectrum of
A
(for a proper definition see
characteristic polynomial, multiplicity, eigenspace).
This set has exactly
r
= rank
A
non zero values.
Eigendecomposition
(
)
If it exists
E
∈
C
n
×
n
, and a diagonal matrix
Λ
∈
C
n
×
n
st
A
=
E
Λ
E

1
A
is said diagonalizable and the columns of
the
n
eigenvectors
e
i
of
A
with corresponding
eigenvalues
Λ
i,i
=
λ
i
.
Properties of eigendecomposition
(
)
•
If, for all
i
,
Λ
i,i
= 0
, then
A
is invertible and
A

1
=
E
Λ

1
E

1
with
Λ

1
i,i
= (Λ
)
1
•
If
A
is Hermitian (
A
=
A
*
), such decomposition
always exists, the eigenvectors of
E
can be chosen
orthonormal such that
E
is unitary (
E

1
*
λ
i
are real.
•
If
A
is Hermitian (
A
=
A
*
) and
λ
i
>
0
,
positive definite, and for all
x
= 0
,
xAx
*
0
.
n
n
n
*
r
k
k
k
v
*
k
r
k
.
•
.
•
*
*
2
.
*
[
{
i
\
i
)
}
)
{
v
i
n
\
i
(
r
)
}
)
*
)
1
if
ii
0
,
0
1
.


p
x
;

x



p
,


2
k
k
,


*
k
k
,


2
F

a

2
*
k
2
k
Cookbook for data scientists
Charles Deledalle
Linear algebra I
Notations
x
,
y
,
z
, . . . :
vectors of
C
n
a
,
b
,
c
, . . . :
scalars of
C
A
,
B
,
C
:
matrices of
C
m
×
n
Id
:
identity matrix
i
= 1
, . . . , m
and
j
= 1
, . . . , n
Matrix vector product
(
Ax
)
i
=
n
k
=1
A
i,k
x
k
(
AB
)
i,j
=
n
k
=1
A
i,k
B
k,j
Basic properties
A
(
ax
+
by
) =
aAx
+
bAy
A
Id = Id
A
=
A
Inverse
(
m
=
n
)
A
is said invertible, if it exists
B
st
AB
=
BA
= Id
.
B
is unique and called inverse of
A
.
We write
B
=
A

1
.
Adjoint and transpose
(
A
t
)
j,i
=
A
i,j
,
A
t
∈
C
m
×
n
(
A
*
)
j,i
= (
A
i,j
)
*
,
A
*
∈
C
m
×
n
Ax, y
=
x, A
*
y
Trace and determinant
(
m
=
n
)
tr
A
=
n
i
=1
A
i,i
=
n
i
=1
λ
i
det
A
=
n
i
=1
λ
i
tr
A
= tr
A
*
tr
AB
= tr
BA
det
A
*
= det
A
det
A

1
= (det
A
)

1
det
AB
= det
A
det
B
A
is invertible
⇔
det
A
= 0
⇔
λ
i
= 0
,
∀
i