This preview shows pages 1–2. Sign up to view the full content.
Fall 2011
Linear and Nonlinear Optimization
Nov 1, 2011
Prof. Yinyu Ye
Homework Assignment 4: Due 6:00pm Thursday, Nov 10
No late homework accepted!
Problem 1 [20 points]
a)
[10 points]
Consider the function
f
:
R
+
→
R
deﬁned by
f
(
x
) =
{
0
x
= 0
x ln x
x >
0
Is this function continuous? Is it convex? Does it have a minimizer on the positive
real line? Justify your answer.
b)
[10 points]
An entropy optimization problem that is frequently used in information
science has the following general form:
Minimize
∑
i
∈
I
f
(
x
i
)
subject to
∑
i
∈
I
a
i
x
i
= 1
x
i
≥
0
,
∀
i
∈
I
Here
I
=
{
1
,
2
, ...N
}
is the index set. Assume
f
(
x
) is deﬁned as in Part (a). What are
the KKT conditions for this problem?
Some motivating background.
According to the principle of maximum entropy, if noth
ing is known about a probability distribution except that it belongs to a certain class, then
the distribution with the largest entropy should be chosen as the default. This is because
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview. Sign up
to
access the rest of the document.
This note was uploaded on 01/16/2012 for the course MS&E 211 at Stanford.
 '07
 YINYUYE

Click to edit the document details