This preview shows page 1. Sign up to view the full content.
Unformatted text preview: ns from a large
bowl and then toss them, but this is manifestly silly.
The assumption of independent tosses is also questionable. Consider building a carefully engineered machine for tossing coins and using it in a vibrationfree environment. A standard
coin is inserted into the machine in the same way for each toss and we count the number
of heads and tails. Since the machine has essentially eliminated the randomness, we would
expect all the coins, or almost all the coins, to come up the same way — the more precise
the machine, the less independent the results. By inserting the original coin in a random
way, a single trial might have equiprobable results, but successive tosses are certainly not
independent. The successive trials would be closer to independent if the tosses were done
by a slightly inebriated individual who tossed the coins high in the air.
The point of this example is that there are many diﬀerent coins and many ways of tossing
them, and the idea that one model ﬁts all is reasonable under some conditions and not
under others. Rather than retreating into the comfortable world of theory, however, note
that we can now ﬁnd the relative frequency of heads for any given coin and essentially for
any given way of tossing that coin.20
Example 1.5.2. Binary data: Consider the binary data transmitted over a communication link or stored in a data facility. The data is often a mixture of encoded voice, video,
graphics, text, etc., with relatively long runs of each, interspersed with various protocols
for retrieving the original nonbinary data.
The simplest (and most common) model for this is to assume that each binary digit is 0 or
1 with equal probability and that successive digits are statistically independent. This is the
same as the model for coin tossing after the trivial modiﬁcation of converting {H, T } into
{0, 1}. This is also a rather appropriate model for designing a communication or storage
facility, since all ntuples are then equiprobable (in the model) for each n, and thus the
20 We are not suggesting that distinguishing diﬀerent coins for the sake of coin tossing is an important
problem. Rather, we are illustrating that even in such a simple situation, the assumption of identically
prepared experiments is questionable and the assumption of independent experiments is questionable. The
extension to n repetitions of IID experiments is not necessarily a good model for coin tossing. In other
words, one has to question both the original model and the nrepetition model. 1.5. RELATION OF PROBABILITY MODELS TO THE REAL WORLD 45 facilities need not rely on any special characteristics of the data. On the other hand, if one
wants to compress the data, reducing the required number of transmitted or stored bits per
incoming bit, then a more elaborate model is needed.
Developing such an improved model would require ﬁnding out more about where the data
is coming from — a naive application of calculating relative frequencies of ntuples would
probably not be the best choice. On the other hand, there are wellknown data compression
schemes that in essence track dependencies in the data and use them for compression in a
coordinated way. These schemes are called universal datacompression schemes since they
don’t rely on a probability model. At the same time, they are best analyzed by looking at
how they perform for various idealized probability models.
The point of this example is that choosing probability models often depends heavily on
how the model is to be used. Models more complex than IID binary digits are usually
based on what is known about the input processes. Associating relative frequencies with
probabilities is the basic underlying conceptual connection between realworld and models,
but in practice this is essentially the relationship of last resort. For most of the applications
we will study, there is a long history of modeling to build on, with experiments as needed.
Example 1.5.3. Fable: In the year 2008, the ﬁnancial structure of the US failed and
the world economy was brought to its knees. Much has been written about the role of
greed on Wall Street and incompetence and stupidity in Washington. Another aspect of
the collapse, however, was a widespread faith in stochastic models for limiting risk. These
models encouraged people to engage in investments that turned out to be far riskier than
the models predicted. These models were created by some of the brightest PhD’s from the
best universities, but they failed miserably because they modeled the everyday events very
well, but modeled the rare events and the interconnection of events poorly. They failed
badly by not understanding their application, and in particular, by trying to extrapolate
typical behavior when their primary goal was to protect against atypical situations. The
moral of the fable is that brilliant analysis is not helpful when the modeling is poor; as
computer engineers say, “garbage in, garbage out.”...
View
Full
Document
 Spring '09
 R.Srikant

Click to edit the document details