Just as in the classical case, to understand positivity of the mutual information, it helps to first
define the
quantum relative entropy
[9]. Suppose that
ρ
and
σ
are two density matrices on the same
Hilbert space
H
. The relative entropy can be defined by imitating the classical formula:
S
(
ρ

σ
) = Tr
ρ
(log
ρ
−
log
σ
)
.
(3.50)
18
For now, this is just a formal definition, but we will learn in section 4.2 that
S
(
ρ

σ
) has the same
interpretation quantum mechanically that it does classically:
if one’s hypothesis is that a quantum
system is described by a density matrix
σ
, and it is actually described by a different density matrix
ρ
,
then to learn that one is wrong, one needs to observe
N
copies of the system where
NS
(
ρ

σ
)
>>
1
.
Just as classically, it turns out that
S
(
ρ

σ
)
≥
0 for all density matrices
ρ,σ
, with equality precisely
if
ρ
=
σ
. To prove this, first diagonalize
σ
. In general
ρ
is not diagonal in the same basis. Let
ρ
D
be
the diagonal density matrix obtained from
ρ
by dropping the offdiagonal matrix elements in the basis
in which
σ
is diagonal, and keeping the diagonal ones. Since Tr
ρ
log
σ
= Tr
ρ
D
log
σ
, it follows directly
from the definitions of von Neumann entropy and relative entropy that
S
(
ρ

σ
) =
S
(
ρ
D

σ
) +
S
(
ρ
D
)
−
S
(
ρ
)
.
(3.51)
This actually exhibits
S
(
ρ

σ
) as the sum of two nonnegative terms.
We showed in eqn.
(3.42)
that
S
(
ρ
D
)
−
S
(
ρ
)
≥
0.
As for
S
(
ρ
D

σ
), it is nonnegative, because if
σ
= diag(
q
1
,...,q
n
),
ρ
D
=
diag(
p
1
,...,p
n
), then
S
(
ρ
D

σ
) =
summationdisplay
i
p
i
(log
p
i
−
log
q
i
)
,
(3.52)
which can be interpreted as a classical relative entropy and so is nonnegative. To get equality in these
statements, we need
σ
=
ρ
D
and
ρ
D
=
ρ
, so
S
(
ρ

σ
) vanishes only if
ρ
=
σ
.
Now we can use positivity of the relative entropy to prove that
I
(
A
;
B
)
≥
0 for any density matrix
ρ
AB
. Imitating the classical proof, we define
σ
AB
=
ρ
A
⊗
ρ
B
,
(3.53)
and we observe that
log
σ
AB
= log
ρ
A
⊗
1
B
+ 1
A
⊗
log
ρ
B
,
(3.54)
so
S
(
ρ
AB

σ
AB
) = Tr
AB
ρ
AB
(log
ρ
AB
−
log
σ
AB
)
= Tr
AB
ρ
AB
(log
ρ
AB
−
log
ρ
A
⊗
1
B
−
1
B
⊗
log
ρ
B
)
=
S
A
+
S
B
−
S
AB
=
I
(
A
;
B
)
.
(3.55)
So just as classically, positivity of the relative entropy implies positivity of the mutual information
(which is also called subadditivity of entropy).
The inequality (3.39) that expresses the concavity of the von Neumann entropy can be viewed as a
special case of the positivity of mutual information. Let
B
be a quantum system with density matrices
ρ
i
B
and let
C
be an auxiliary system
C
with an orthonormal basis

i
)
C
. Endow
CB
with the density
matrix:
ρ
CB
=
summationdisplay
i
p
i

i
)
C C
(
i
 ⊗
ρ
i
B
.
(3.56)
19
The mutual information between
C
and
B
if the combined system is described by
ρ
CB
is readily com
puted to be
I
(
C
;
B
) =
S
(
ρ
B
)
−
summationdisplay
i
p
i
S
(
ρ
i
B
)
,
(3.57)
so positivity of mutual information gives our inequality.
3.5 Monotonicity of Relative Entropy
So relative entropy is positive, just as it is classically. Do we dare to hope that relative entropy is also
monotonic, as classically? Yes it is, as first proved by Lieb and Ruskai [10], using a lemma of Lieb [11].