253
#Midterm Psych review with justin and
ann Marie= Friday 2 30-4 location
TBA
Using Decision Analysis to Answer
Questions about the Value of Getting
Information
Perfect vs. Imperfect Information
Signal Detection Theory
Medical Example
Jury Decision Making
You can select A, B, or C. Below are the payoffs
for each depending on the throw of a die.
Which one do you want to have?
A
1
$1
2
$2
Die Roll
3
4
$3
$4
B
$6
$2
$5
$2
$2
$2
C
$7
$5
$4
$4
$2
$1
What should you choose?
5
$5
6
$6
2
$1
Die Roll
3
4
5
$2 $3 $4
6
$5
EV
$6
B
$3.17
$6
$2
$5
$2
$2
$2
C
$3.83
$7
$5
$4
$4
$2
$1
1
A
$3.50
Which one would you select if you knew
what number would come up on the
die?
A
2
$1
Die Roll
3
4
5
$2 $3 $4
6
$5
$6
B
$6
$2
$2
$2
1
$5
$2
What is the expected value of knowing the
number that would come up?
1/6*[ $7 + $5 + $5 + $4 + $5 + $6 ] = $5.33
A
1
$1
2
$2
Die Roll
3
4
$3 $4
B
$6
$2
$5
$2
$2
$2
C
$7
$5
$4
$4
$2
$1
5
$5
6
$6
The value of the information is
$5.33 - $3.83 = $1.50
Value of the
decision
WITH
perfect
information
Value of the
decision
WITHOUT
perfect
information
Miracle Movers rents out trucks with a crew
of 2 people. One day, Miracle discovers it is a
truck short. When this happens, Miracle rents
a truck from a local firm. Small trucks cost
$130 per day, and large ones cost $200. The
advantage of the small truck may vanish if
the crew has to make 2 trips. The extra cost
of a second trip is $150, and the probability
of 2 trips is 40%.
Large Truck
$200 $200
P(NL) =.40
Need Large
Small Truck
Truck $280$280
P(NS) =.60
Need Small
Truck $130$130
What would it be worth to Michelle to know for
sure whether she would need a small truck?
Large Truck
Small Truck
$200 $200
P(NL) =.40
$280$280
P(NS) =.60 $130$130
What would it be worth to Michelle to know for
sure whether she would need a small truck?
Large Truck
$200 $200
Small Truck
EV= $190280
What would it be worth to Michelle to know for
sure whether she would need a small truck?
Dont Collect
Information
Small Truck
$190
NL P(NL) = .
40
Collect
InformationNS
(that
happens
P(NS) =.60
to be perfect)
Need Large Truck
Need Small
Truck
Need Small
Truck
Need Large
Truck
What does Perfect Information Mean?
Collect Perfect
Information
NL
P(NL)=.40
Need Large
Truck $200
NS
P(NS) =.60
P (NL|NL) = 1 and P (NL|NS)
=0
P(NS|NS) = 1 and P (NS|NL ) =
0
Need Small
Truck $130
Value of the decision with perfect information
= p(NL and NL)*$200 + p(NS and
NS)*$130
+ p(NL and NS)*$200 + p(NS and
NL)*$280
But because the information is perfect,
p(NL and NS) and p(NS and NL) never
occur.
So the value of the decision with perfect
information is
= p(NL and NL)*$200 + p(NS and
NS)*$130
What is P(NL and NL) and P(NS and
NS)?
P(NL and NL) = P(L|NL)*P(NL) =
1.0*(.4) =.4
P(NS and NS) = P(S|NS)*P(NS) =
1.0*(.6)=.6
Expected Value of the decision with perfect
info
= p(NL and NL)*$200 + p(NS and
NS)*$130
= .4*$200 + .6*$130 = $158
Expected Value of the Decision Expected
Value of the Decision with perfect info =
Value of the Information
Value of the information = $190 - $158 = $32
Perfect information means this
Information
S
Actual
P(S|NS)
NS
=1
NL P(S|NL)
=0
L
P(L|
NS)=0
P(L|NL)
=1
If the information is imperfect, we need
to quantify that uncertainty.
Information
S
Actual
P(S|NS)
NS
=.9
P(S|
NL
NL)=.2
L
P(L|
NS)=.1
P(L|NL)
=.8
Signal Detection Theory:
A Theory of Repeated Decision
Making when Information is Fuzzy
-deciding which customers would
default on a loan
-deciding whether to hire an employee
-determining whether someone is lying
or not
-and any other decision that is made
repeatedly using ambiguous evidence
Imagine a doctor who treats patients for
a disease. A test for the number of
goodies in the patients blood is the only
predictor or clue he has to predict
whether or not the patient has the
disease.
From past research, the doctor knows
that goodies are normally distributed in
the blood of Healthy and Sick people.
The average # of goodies in the blood of
Healthy people is 100, and the average
number of goodies in the blood of Sick
people is 115.
0.45
Healthy
0.4
Sick
0.3
0.25
0.2
0.15
0.1
0.05
# Goodies in the Blood
14
5
13
0
12
5
12
0
11
5
10
0
85
70
0
55
Probability Disease
0.35
We of can analyze the doctors ability to diagnose
a patients with Signal Detection Theory. This
theory has two parameters:
d = d prime - the ability to distinguish Healthy
people from Sick people with the blood test
B = beta (also called the threshold or cutoff) the doctors tendency to call a patient
Healthy or Sick
0.45
d
0.4
Healthy
Sick
0.3
0.25
0.2
0.15
0.1
0.05
# Goodies in the Blood
14
5
13
0
12
5
12
0
11
5
10
0
85
70
0
55
Probability of Disease
0.35
Healthy
B
Sick
0.45
0.4
Healthy
Sick
0.3
0.25
0.2
0.15
0.1
0.05
# Goodies in the Blood
14
5
13
0
12
5
12
0
11
5
10
0
85
70
0
55
Probability of Disease
0.35
No matter where the doctor sets his
cutoff, there are two errors and two
correct decisions.
Decision
Healthy Sick
State of Healthy
P(H|H) P(S|H)
the World
Sick
P(H|S) P(S|S)
**Each outcome has a name.
Decision
Healthy Sick
State of Healthy
Correct
Rejection
False
Alarm
Miss
Hit
the World
Sick
Healthy
B
Sick
0.45
0.4
Healthy
Sick
0.3
0.25
0.2
0.15
0.1
0.05
# Goodies in the Blood
14
5
13
0
12
5
12
0
11
5
10
0
85
70
0
55
Probability of Disease
0.35
Which error is worse? It depends.
If the decision is whether to treat a
patient who may have cancer, a miss
may be worse than a false alarm.
If the decision is whether to attack an
unidentified plane, a false alarm may
be worse than a miss.
Where should the doctor set his
threshold?
B
Healthy
B
Healthy
Sick
0.45
0.4
0.4
0.35
0.35
0.3
Healthy
Sick
Probabi l i ty of Di sease
0.25
0.2
0.15
0.1
0.3
Healthy
Sick
0.25
0.2
0.15
0.1
# Goodies in the Blood
14
5
13
0
12
5
12
0
11
5
10
0
85
55
14
5
13
0
12
5
12
0
11
5
10
0
0
85
0
70
0.05
55
0.05
70
Probabi l i ty of Di sease
0.45
Sick
# Goodies in the Blood
As we move B from the left to the right, we decrease P(S|H)
(false alarms), but increase P(H|S) (misses)
Where shouldthe doctor to put his cutoff
(B)?
SDT tells the decision maker where to
locate B, the cutoff, for a given
objective. In this sense, it is a
normative theory.
But SDT can also be used to estimate
where someone puts his or her B. In
this sense, it is a descriptive theory.
The optimal location for B depends on
the base rate for the event (in this case,
the disease) and the utilities associated
with the four possible outcomes.
d d
Sick
85
10
0
11
5
13
0
14
5
16
0
17
5
Sick
70
55
40
25
Probabi l i ty
0.45
0.4
0.35 Healthy
0.3 Healthy
0.25
0.2
0.15
0.1
0.05
0
Values of Y hat
Increasing accuracy requires one to find
variables that strengthen the
predictability of the criterion.
Using those variables in a consistent,
statistical fashion improves
predictability.
Using a fixed (and optimal) response
threshold reduces noise further
improves predictability.
Another Example: Jury Decision Making
Probability
Acquit
Convict
Guilty
Innocent
P erceived Culpability
What are acceptable probabilities of
errors?
Convicting the innocent - False Alarm
Acquitting the guilty - Miss
Within this framework, how good must
the juror be to achieve reasonable rates
of error ?
Assuming normal distributions and equal
variances,
If false alarms and misses are 1%, d =
4.7.
If false alarm and miss rates are 5%, d =
3.3.
If false alarm and miss rates are 10%, d=
2.6.
What are common values of d?
Legal Settings
d
Detecting liars in a mock crime
Detecting liars with polygraphs
0
0.5 - 1.0
Personnel Selection
d
Job placement with the Armed
Forces Qualification Test
0.8
0.6 -
Weather Forecasting
d
Fog-risk in Canberra, Australia
24 hours earlier
18 hours earlier
12 hours earlier
Tornados near Kansas City
Rain in Chicago
Minimum temp in Albuquerque
0.8
1.0
1.2
1.0
1.5
1.7
Medical Settings
d
Detecting breast cancer w mammograms
Detecting brain lesions with RN Scans
1.7
Experts detecting cervical cancer
Detecting cervical cancer with computer
based systems
1.8
Detecting prostate cancer with PSA tests
Detecting brain lesions with CT Scans
2.9
1.3
1.4 1.6
2.0
2.4 -
Signal detection theory and empirical
estimates of d (with all possible technological
advances) tells us we are not achieving errors
that are even remotely desirable.
What might this imply for policy?