series2 - Exercises Machine Learning Laboratory Dept of...

Info icon This preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
Machine Learning Laboratory Dept. of Computer Science, ETH Z¨urich Prof. Dr. Joachim M. Buhmann Web http://ml2.inf.ethz.ch/courses/ml/ Email questions to: Gabriel Krummenacher [email protected] Exercises Machine Learning AS 2012 Series 2, Oct 9th, 2010 (Maximum Likelihood Estimation) Master solution will be available online from Tuesday, Oct 16th. Problem 1 (Analytic MLE): In this problem, we analytically derive maximum likelihood estimators for the parameters of an example model distribution. Most textbooks, including Duda et al, discuss the Gaussian example. The distribution we consider here is called the gamma distribution . The gamma distribution is univariate (one-dimensional) and continuous. It is controlled by two parameters, the location parameter μ and the shape parameter ν . 1 For a gamma-distributed random variable X , we write X ∼ G ( μ, ν ) . G is defined by the following density function: p ( x | μ, ν ) := ν μ ν x ν - 1 Γ ( ν ) exp - νx μ , where x 0 and μ, ν > 0 . 2 Whenever ν > 1 , the gamma density has a single peak, much like a Gaussian. Unlike the Gaussian, it is not symmetric. The first two moment statistics of the gamma distribution are given by E [ X ] = μ and Var [ X ] = μ 2 ν (1) for X ∼ G ( μ, ν ) . Here are some plots which should give you a rough idea of what the gamma density may look like and how different parameter values influence its behavior: 0 0.5 1 1.5 2 2.5 3 0 0.5 1 1.5 2 2.5 0 0.5 1 1.5 2 2.5 0 0.5 1 1.5 2 2.5 3 0 0.5 1 1.5 2 2.5 3 0 0.5 1 1.5 2 2.5 Left: The plot shows the density for different values of the location parameter ( μ = 0 . 3 , 0 . 5 , 1 . 0 , 3 . 0 ), with the shape parameter fixed to ν = 2 . Since ν > 1 , the densities peak. As we increase μ , the peak moves to the right, and the curve flattens. Middle: For μ = 0 . 5 fixed, we look at different values of the shape parameter ( ν = 2 , 3 , 4 , 5 , 19 ). Again, all the densities peak, with the peak shifting to the right as we increase ν . Right: If ν < 1 , the density turns into a monotonously decreasing function. The smaller the value of ν , the sharper the curve dips towards the origin. 1. Write the general analytic procedure to obtain the maximum likelihood estimator (including logarithmic transformation) in the form of a short algorithm or recipe. A few words are enough, but be precise: Write all important mathematical operations as formulae. Assume that data is given as an i. i. d. sample
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

0/0 PREVIEWS LEFT
Sign up to access 24/7 study resources for your classes

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern