Discrete-time stochastic processes

# They failed badly by not understanding their

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ns from a large bowl and then toss them, but this is manifestly silly. The assumption of independent tosses is also questionable. Consider building a carefully engineered machine for tossing coins and using it in a vibration-free environment. A standard coin is inserted into the machine in the same way for each toss and we count the number of heads and tails. Since the machine has essentially eliminated the randomness, we would expect all the coins, or almost all the coins, to come up the same way — the more precise the machine, the less independent the results. By inserting the original coin in a random way, a single trial might have equiprobable results, but successive tosses are certainly not independent. The successive trials would be closer to independent if the tosses were done by a slightly inebriated individual who tossed the coins high in the air. The point of this example is that there are many diﬀerent coins and many ways of tossing them, and the idea that one model ﬁts all is reasonable under some conditions and not under others. Rather than retreating into the comfortable world of theory, however, note that we can now ﬁnd the relative frequency of heads for any given coin and essentially for any given way of tossing that coin.20 Example 1.5.2. Binary data: Consider the binary data transmitted over a communication link or stored in a data facility. The data is often a mixture of encoded voice, video, graphics, text, etc., with relatively long runs of each, interspersed with various protocols for retrieving the original non-binary data. The simplest (and most common) model for this is to assume that each binary digit is 0 or 1 with equal probability and that successive digits are statistically independent. This is the same as the model for coin tossing after the trivial modiﬁcation of converting {H, T } into {0, 1}. This is also a rather appropriate model for designing a communication or storage facility, since all n-tuples are then equiprobable (in the model) for each n, and thus the 20 We are not suggesting that distinguishing diﬀerent coins for the sake of coin tossing is an important problem. Rather, we are illustrating that even in such a simple situation, the assumption of identically prepared experiments is questionable and the assumption of independent experiments is questionable. The extension to n repetitions of IID experiments is not necessarily a good model for coin tossing. In other words, one has to question both the original model and the n-repetition model. 1.5. RELATION OF PROBABILITY MODELS TO THE REAL WORLD 45 facilities need not rely on any special characteristics of the data. On the other hand, if one wants to compress the data, reducing the required number of transmitted or stored bits per incoming bit, then a more elaborate model is needed. Developing such an improved model would require ﬁnding out more about where the data is coming from — a naive application of calculating relative frequencies of n-tuples would probably not be the best choice. On the other hand, there are well-known data compression schemes that in essence track dependencies in the data and use them for compression in a coordinated way. These schemes are called universal data-compression schemes since they don’t rely on a probability model. At the same time, they are best analyzed by looking at how they perform for various idealized probability models. The point of this example is that choosing probability models often depends heavily on how the model is to be used. Models more complex than IID binary digits are usually based on what is known about the input processes. Associating relative frequencies with probabilities is the basic underlying conceptual connection between real-world and models, but in practice this is essentially the relationship of last resort. For most of the applications we will study, there is a long history of modeling to build on, with experiments as needed. Example 1.5.3. Fable: In the year 2008, the ﬁnancial structure of the US failed and the world economy was brought to its knees. Much has been written about the role of greed on Wall Street and incompetence and stupidity in Washington. Another aspect of the collapse, however, was a widespread faith in stochastic models for limiting risk. These models encouraged people to engage in investments that turned out to be far riskier than the models predicted. These models were created by some of the brightest PhD’s from the best universities, but they failed miserably because they modeled the everyday events very well, but modeled the rare events and the interconnection of events poorly. They failed badly by not understanding their application, and in particular, by trying to extrapolate typical behavior when their primary goal was to protect against atypical situations. The moral of the fable is that brilliant analysis is not helpful when the modeling is poor; as computer engineers say, “garbage in, garbage out.”...
View Full Document

Ask a homework question - tutors are online