Error Correcting Codes: Combinatorics, Algorithms and Applications
(Fall 2007)
Lecture 4: Probability and Discrete Random Variables
Wednesday, January 21, 2009
Lecturer: Atri Rudra
Scribe: Anonymous
1
Counting and Probability
This lecture reviews elementary combinatorics and probability theory. We begin by first reviewing
elementary results in counting theory, including standard formulas for counting permutations and
combinations. Then, the axioms of probability and basic facts concerning probability distributions
are presented.
2
Counting
Counting theory tries to answer the question ”How many?” or ”How many orderings of
n
distinct
elements are there?” In this section, we review the elements of counting theory. A set of items that
we wish to count can sometimes be expressed as a union of disjoint sets or as a Cartesian product
of sets. The
rule of sum
says that the number of ways to choose an element from one of two
disjoint
sets is the sum of the cardinalities of the sets. That is, if
A
and
B
are two finite sets with no
members in common, then

A
∪
B

=

A

+

B

. The
rule of product
says that the number of ways
to choose an ordered pair is the number of ways to choose the first element times the number of
ways to choose the second element. That is, if
A
and
B
are two finite sets, then

A
×
B

=

A
·
B

.
A
string
over a finite set
S
is a sequence of elements of
S
. We sometimes call a string of length
k
a
kstring
. A
substring
s
of a string
s
is an ordered sequence of consecutive elements of
s
. A
ksubstring
of a string is a substring of length
k
. For example,
010
is a
3
substring of
01101001
(the
3
substring that begins in position
4
), but
111
is not a substring of
01101001
. A
k
string over
a set
S
can be viewed as an element of the Cartesian product
S
k
of
k
tuples; thus, there are

S

k
strings of length
k
. For example, the number of binary
k
strings is
2
k
. Intuitively, to construct
a
k
string over an
n
set, we have
n
ways to pick the first element; for each of these choices, we
have
n
ways to pick the second element; and so forth
k
times. This construction leads to the
k
fold
product
n
·
n
· · ·
n
=
n
k
as the number of
k
strings.
A
permutation
of a finite set
S
is an ordered sequence of all the elements of
S
, with each
element appearing exactly once. For example, if
S
=
{
a, b, c
}
, there are
6
permutations of
S
:
abc, acb, bac, bca, cab, cba.
There are
n
!
permutations of a set of
n
elements, since the first element of the sequence can be
chosen in
n
ways, the second in
n

1
ways, the third in
n

2
ways, and so on.
1