At the beginning of the semester, I indicated that I wanted for each of you to do
an optimization project. It is now time for us to begin to try to define the individual
projects. I know that some of you have projects that you would like to pursue. If you
have a project in mind, please write up a one page description of the project. I will then
evaluate what you have given me and determine if the project has sufficient depth and
complexity without being overly ambitious.
We can then talk about the project and
come up with a final description of what you are going to do.
For those of you who do not have a project in mind, I will now define four potential
projects. Two of these projects come from statistical problems that I am interested in or
have worked on. But they are self contained as optimization problems and knowledge of
statistics or the statistical background of the problems is not required. A third project
is a continuous graph theory problem. The final project can be classified as an inverse
problem and this type of problem is quite typical of a wide range of interesting examples.
Local polynomial smoother - Nonlinear Regression
Suppose that we would like to estimate a function
) given measurements
From this data how do we estimate the unknown function
) given the assumption
that our measurements of y contains measurement errors (or noise)? This is commonly
called the regression problem. Clearly, simple interpolation of the noisy data will give a
poor solution to the problem.
There are many partial solutions to the regression problem including fitting a straight
line (or plane in higher dimensions) or perhaps fitting a quadratic function. But these
solutions do not solve the more general problem where the unknown function
be approximated by a plane or a quadratic surface. The solutions that I want you to
implement are the local linear and local quadratic regression in dimensions one and two.
A brief write up for this set of problems, and their solution from the book on Statistical
Learning is attached. Please note that the solution to this portion of the problem is not
iterative, but at each point
, the solution is given by solving a linear system.