222
4. Combinatorics, Probability, and Information Theory
chosen at random without replacement. Let X denote the number of white
balls chosen. Then X is a random variable taking on the integer values
0 X() mincfw_m, n.
In the case that n m, an argument si
238
4. Combinatorics, Probability, and Information Theory
!
"
Pr xk is the first match
!
"
= Pr xk is a match AND x0 , . . . , xk1 are distinct
#
!
"
= Pr xk is a match # x0 , . . . , xk1 are distinct
!
"
Pr x0 , . . . , xk1 are distinct
2
k
ek /2N
from
256
4. Combinatorics, Probability, and Information Theory
as the average number of bits of information conveyed by a single letter of a
language. The value of H(L) that we computed does reveal some redundancy:
it says that a letter conveys only 4.132 bits
264
Exercises
(c) Use (b) with b = 1 and induction on a to prove that ap a (mod p) for
all a 0.
(d) Use (c) to deduce that ap1 1 (mod p) for all a with gcd(p, a) = 1.
4.9. We know that there are n! dierent permutations of the set cfw_1, 2, . . . , n.
(a)
4.5. Pollards method
241
In a similar fashion we compute the sequence given by
!
"
y0 = 1
and
yi+1 = f f (yi ) .
Then
yi = x2i = g i ai ,
where the exponents i and i can be computed by two repetitions of the same
recursions used for i and i . Of course, t
4.6. Information theory
243
1913470 = 2471726510
in F48611 .
(4.44)
We next observe that
gcd(26510, 48610) = 10
and
970 26510 10
(mod 48610).
Raising both sides of (4.44) to the 970th power yields
1913470970 = 1913065900 = 1938420 = 2471710
Hence
which me
230
4. Combinatorics, Probability, and Information Theory
numbered blue balls. After making our first list of n dierent numbered balls,
we repaint those n balls with red paint and return them to the box. The
second list is constructed by drawing m balls o
4.3. Probability theory
223
Similarly, the conditional density function, denoted by fX|Y (x | y), is the probability that X takes the value x, given that Y takes the value y:
fX|Y (x | y) = Pr(X = x | Y = y).
We say that X and Y are independent if
fX,Y (x
4.3. Probability theory
219
Pr(m has property A | algorithm returns Yes) = 1.
(2) If m has property A, then the algorithm returns Yes for at least 50% of
the choices for r.11 Using conditional probability notation,
Pr(algorithm returns Yes | m has propert
4.3. Probability theory
221
Definition. Let X : R be a random variable. The probability density
function of X, denoted by fX (x), is defined to be
fX (x) = Pr(X = x).
In other words, fX (x) is the probability that X takes on the value x. Sometimes we writ
4.7. Complexity Theory and P versus N P
259
the answer will be produced in a polynomial (in n) number of steps. One says
that the decision problems in P are those that can be solved in polynomial
time.
The concept of verification in polynomial time has so
242
4. Combinatorics, Probability, and Information Theory
Example 4.51. We illustrate Pollards method by solving the discrete logarithm problem
19t 24717 (mod 48611).
The first step is to compute the x and y sequences until a match yi = xi
is found, while
254
4. Combinatorics, Probability, and Information Theory
Example 4.61. We consider the system with two keys described in Example 4.53 on page 245. Each key is equally likely, so H(K) = log2 (2) = 1. Similarly, we can use the plaintext probabilities for t
270
Exercises
(d) Same questions as in (b) and (c), except that Dan also knows how the boxes
are labeled.
(e) With the same assumptions as in (c), suppose that Dan employs his best strategy and that Monty Hall knows that Dan is employing this strategy. Ca
4.6. Information theory
4.6.4
257
The algebra of secrecy systems
We make only a few brief remarks about the algebra of cryptosystems. In [117],
Shannon considers ways of building new cryptosystems by taking algebraic
combinations of old ones. The new syst
4.4. Collision algorithms and meet-in-the-middle attacks
4.4
227
Collision algorithms and
meet-in-the-middle attacks
A simple, yet surprisingly powerful, search method is based on the observation
that it is usually much easier to find matching objects tha
240
4.5.2
4. Combinatorics, Probability, and Information Theory
Discrete logarithms via Pollards method
In this section we describe how to use Pollards method to solve the discrete
logarithm problem
g t = a in Fp
when g is a primitive root modulo p. The i
250
4. Combinatorics, Probability, and Information Theory
reveal significant information about the key. To study this phenomenon, Shannon introduced the concept of entropy in order to quantify the uncertainty of
the outcome of an experiment.
Here the outc
252
4. Combinatorics, Probability, and Information Theory
Theorem 4.58. Every function having Properties H1 , H2 , and H3 is a constant multiple of the function
H(p1 , . . . , pn ) =
n
!
pi log2 pi ,
(4.51)
i=1
where log2 denotes the logarithm to the bas
224
4. Combinatorics, Probability, and Information Theory
Thus the chance of getting exactly one gold coin and exactly one silver coin
is somewhat larger if the coins are not replaced after each pick.
The following restatement of Bayess formula is often c
232
4. Combinatorics, Probability, and Information Theory
logarithm problem. For the finite field Fp , it solves the discrete logarithm
problem (DLP) in approximately p steps.
Of course, the index calculus described in Section 3.8 solves the DLP in Fp
muc
220
4. Combinatorics, Probability, and Information Theory
Pr(F | E c ) = Pr(Output is No | m has property A)N
!
"N
= 1 Pr(Output is Yes | m has property A)
#
$N
1
1
from Property (2) of the Monte Carlo method,
2
1
= N.
2
Substituting these values into Ba
4.6. Information theory
249
We sum (4.50) over all c C and divide by #C to obtain
f (k) =
1
1 !
.
f (c) =
#C
#C
cC
This shows that f (k) is constant, independent of the choice of k K, which
is precisely the assertion of (a). At the same time we have prove
260
4. Combinatorics, Probability, and Information Theory
an assignment of truth values that makes the expression true. Cook proves
that SAT has the following properties:
1. Every N P problem is polynomial-time reducible to SAT.
2. If there exists any pro
4.4. Collision algorithms and meet-in-the-middle attacks
231
For the first question, Bob uses the reasonably accurate lower bound of
formula (4.29) to set
2
Pr(match) 1 en
/N
=
1
.
2
It is easy to solve this for n:
n2 /N
e
1
=
2
=
n2
= ln
N
! "
1
2
=
n=
N
244
4. Combinatorics, Probability, and Information Theory
computational resources that may be brought to bear against them. For example, symmetric ciphers such as the simple substitution cipher (Section 1.1)
and the Vig`enere cipher (Section 4.2) are not
4.4. Collision algorithms and meet-in-the-middle attacks
233
the list (4.32) may be viewed as selecting n elements from the urn, and we
would like to know the probability of selecting at least one red ball, i.e., the
probability that at least one element
4.6. Information theory
247
matched with dierent keys, which shows that there must be at least as many
keys as there are plaintexts.
Given the restriction on the relative sizes of the key, ciphertext, and plaintext spaces in systems with perfect secrecy,
4.3. Probability theory
225
random variables X, Y , and Z, any pair of them are independent. Yet we
would not want to call the three of them together an independent family,
since the value of Z is determined by the values of X and Y . The prompts
the foll