3
This Homework assignment is due on Friday, November 19, 2004. The assignment is again somewhat longer than the previous two assignments; therefore, you are advised to start working on it immediately. Each problem is equally weighted. As always, show all works.
Problem 1 Bayesian Networks
Consider the following Bayesian Network, where variables A-E are all Booleanvalued:
a)What is the probability that all five of these Boolean variables are simultaneously true? b)What is the probability that all five of these Boolean variables are simultaneously false? (c) What is the probability that A is false given that the four other variables are all known to be true?
1
Problem 2 Probability (Problem 13.11 on Page 490)
Suppose you are given a bag containing n unbiased coins. You are told that n - 1 of these coins are normal, with heads on one side and tails on the other, whereas one coin is a fake, with heads on both sides. 1. Suppose you reach into the bag, pick out a coin uniformly at random, flip it, and get a head. What is the (conditional) probability that the coin you chose is the fake coin? 2. Suppose you continue flipping the coin for a total of k times after picking it and see k heads. Now what is the conditional probability that you picked the fake coin? 3. Suppose you wanted to decide whether the chosen coin was fake by flipping it k times. The decision procedure returns FAKE if all k flips come up heads, otherwise it returns NORMAL. What is the (unconditional) probability that this procedure makes an error?
Problem 3 Probability (Problem 13.13 on Page 490)
2
Problem 4 Bayesian Networks (Problem 14.3 on Page 534)
Two astronomers in different parts of the world make measurements M1 and M2 of the number of stars N in some small region of the sky, using their telescopes. Normally, there is a small possibility e of error by up to one star in each direction. Each telescope can also (with a much smaller probability f ) be badly out of focus (event F1 and F2 ), in which case the scientist will undercount by three or more stars (or, if N is less than 3 fail to detect any stars at all). Consider the three networks shown in Figure 14.19 (Figure below). 1. Which of these Bayesian networks are correct (but not necessarily efficient) representations of the preceding information? 2. Which is the best network? Explain. 3. Write out a conditional distribution for P (M1 |N ), for the case where N {1, 2, 3} and M1 {0, 1, 2, 3, 4}. Each entry in the conditional distribution should be expressed as a function of the parameters e and/or f . 4. Suppose M1 = 1 and M2 = 3. What are the possible numbers of stars if we assume no prior constraint on the values of N ?. 5. What is the most likely number of stars, given these observations? Explain how to compute this, or if it is not possible to compute, explain what additional information is needed and how it would affect the result.
Figure 1: Figure 14.19: Three possible networks for the telescope problem.
3
Problem 5 Bayesian Networks (Problem 14.4 on Page 534)
Consider the network shown in Figure 14.19(ii), and assume that the two telescopes work identically. N {1, 2, 3} and M1 , M2 {0, 1, 2, 3, 4}, with the symbolic CPTs as described in Exercise 14.3. Using the enumeration algorithm, calculate the probability distribution P (N |M1 = 2, M2 = 2).
Figure 2: Figure 14.19: Three possible networks for the telescope problem.
Problem 6 Bayesian Networks (Problem 14.5 on Page 535) Consider the family of linear Gaussian networks, as illustrated on page 502.
1. In a two-variable network, let X1 be the parent of X2 , let X1 have a Gaussian prior, and let P (X2 |X1 ) be a linear Gaussian distribution. Show that the joint distribution P (X1 , X2 ) is a multivariate Gaussian, calculate and its covariance matrix. 2. Prove by induction that the joint distribution for a general linear Gaussian network on X1 , , Xn is also a multivariate Gaussian.
4
Problem 7 Bayesian Networks (Problem 14.9 on Page 535)
Consider the problem of generating a random sample from a specified distribution on a single variable. You can assume that a random number generator is available that returns a random number uniformly distributed between 0 and 1. 1. Let X be a discrete variable with P (X = xi ) = pi for i {1, , k}. The cumulative distribution of X gives the probability that X {x1 , , xj } for each possible j. Explain how to calculate the cumulative distribution in O(k) time and how to generate a single sample of X from it. Can the latter be done in less than O(k) time? 2. Now suppose we want to generate N samples of X, where N >> k. Explain how to do this with an expected runtime per sample that is constant (i.e. independent of k). 3. Now consider a continuous-valued variable with a parametrized distribution (e.g., Gaussian). How can samples be generated from such a distribution? 4. Suppose you want to query a continuous-valued variable and you are using a sampling algorithm such as LIKELIHOODWEIGHTING to do the inference. How would you have to modify the query-answering process?
5
Problem 8 Temporal Probability Models (Problem 15.2 on Page 581)
In this exercise, we examine what happens to the probabilities in the umbrella world in the limit of long time sequences. 1. Suppose we observe an unending sequence of days on which the umbrella appears. Show that, as the days go by, the probability of rain on the current day increases monotonically towards a fixed point. Calculate this fixed point. 2. Now consider forecasting further and further into the future, give just the first two umbrella observations. First, compute the probability P (R2+k |U1 , U2 ) for k=1 ... 20 and plot the results. You should see that the probability converges towards a fixed point. Calculate the exact value of this fixed point
Figure 3: Network for Rain-Umbrella Example
Problem 9 Temporal Probability Models (Problem 15.3 on Page 581)
6
Problem 10 Temporal Probability Models (Problem 15.5 on Page
581) Often, we wish to monitor a continuous-state system whose behavior switches unpredictably among a set of k distinct "modes." For example, an aircraft trying to evade a missile can execute a series of distinct maneuvers that the missile may attempt to track. A Bayesian network representation of such a switching Kakman filter model is shown in Figure 15.22 (below). 1. Suppose that the discrete state St has k possible values and that the prior continuous state estimate P (X0 ) is a multivariate Gaussian distribution. Show that the prediction P (X1 ) is a mixture of Gaussians -- that is, a weighted sum of Gaussians such that the weights sum to 1. 2. Show that if the current continuous state estimate P (Xt |e1:t ) is a mixture of m Gaussians, then in the general case the updated state estimate P (Xt+1 |e1:t+1 ) will be a mixture of km Gaussians. 3. What aspect of the temporal process do the weights in the Gaussian Mixture represent.
Figure 4: Figure 15.22: A Bayesian network representation of a switching Kalman filter. The switching variable St is a discrete state variable whose value determines the transition model for the continuous state variables Xt . For any discrete state i, the transition model P (Xt+1 |Xt , St = i) is a linear Gaussian model, just as in a regular Kalman filter. The transition model for the discrete state, P (St+1 |St ), can be thought of as a matrix, as in a hidden Markov model.
7
Problem 11 (Optional (extra credit)) Bayesian Networks
Problem 14.11 on Page 536.
8