271 COMPSCI - Numeric Artificial Intelligence Homework Assignment V
Due date: Apr 12 in class
1
Utility Theory
Tickets to state lottery cost $1. There are two possible prizes: a $10 payoff with probability 1/50, and a $1,000,000 payoff with probability 1/2,000,000. What is the expected monetary value of a lottery ticket? When (if ever) is it rational to buy a ticket? Be precise--show an equation involving utilities. You may assume that U ($10) = 10 U ($1), but you may not make any assumptions about U ($1, 000, 000). Sociological studies show that people with lower income buy a disproportionate number of lottery tickets. Do you think this is because they are worse decision makers or because they have a different utility function?
2
Decision Making
(Adapted from Judea Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann, 1988.) A used-car buyer can decide to carry out various tests with various costs (e.g. kick the tires, take the car to a qualified mechanic), and then, depending on the outcome of the tests, decide which car to buy. We will assume that the buyer is deciding whether to buy car c1 , that there is time to carry out at most one test, and that t1 is the test of c1 and costs $50. A car can be in a good shape (quality q + ) or bad shape (quality q - ), and the tests may help to indicate what shape the car is in. Car c1 costs $1,500, and its market value is $2,000 if it is in good shape; if not, $700 in repairs will be needed to make it in good shape. The buyer's estimate is that c1 has a 70% chance of being in good shape. a. Calculate the expected net gain from buying c1 , given no test. b. Tests can be described by the probability that the car will pass or fail given that the car is in good or bad shape. We have the following information: P (pass(c1 , t1 )|q + (c1 )) = 0.8 P (pass(c1 , t1 )|q - (c1 )) = 0.35 Use Bayes' theorem to calculate the probability that the car will pass (or fail) its test, and hence the probability that it is in good (or bad) shape given each possible test outcome. c. Calculate the optimal decision given either a pass or a fail, and their expected utilities. d. Calculate the value of information of the test, and derive an optimal conditional plan for the buyer.
1
3
Markov Decision Processes
Consider the stochastic version of the simple in environment Figure 17.1 of the textbook. a. Which states (squares) can be reach by the action sequence {N orth, East, W est}, and with what probabilities? b. Is it possible to reach the goal (+1) state starting the Start state and using the fixed policy that chooses action N orth in all states? How about reaching the "death" (-1) state?
4
Value Iteration and Value Functions
Value Iteration has the following property: Suppose we start with two arbitrary value functions f1 and g1 and apply one step of value iteration to each of them yielding f2 and g2 . It is known that, if maxs |f1 (s) - g1 (s)| = t, then maxs |f2 (s) - g2 (s)| t, where is the discount factor. a. Prove that if f exists, then it must equal g (f is the value function that results by applying an arbitrarily big "infinite" number of steps of value iteration to f1 ). Now, we will use this fact to prove a bound on how far a value function is from the correct value function. Suppose we apply one step of value iteration to f2 , yielding f3 , and we observe: maxs |f3 (s) - f2 (s)| = b. Provide an upper bound on the difference between f2 and f . (Hint: You will need to use the triangle inequality.)
5
Decision Trees
Consider the problem of learning to accept or reject student applications to Dook University, based on the following attributes: 1. GPA = { High, Good, Medium, Low } 2. GRE Score = { Excellent, Mod...