{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

handout15 - ISyE8843A Brani Vidakovic Handout 15 There are...

Info icon This preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
ISyE8843A, Brani Vidakovic Handout 15 There are no true statistical models. 1 Model Search, Selection, and Averaging. Although some model selection procedures boil down to testing hypotheses about parameters and choosing the best parameter or a subset of parameters, model selection is a broader inferential task. It can be non- parametric, for example. Model selection sometimes can be interpreted as an estimation problem. If the competing models are indexed by i ∈ { 1 , 2 , . . . , m } , getting the posterior distribution of an index i would lead to the choice of best model, for example, the model that maximizes posterior probability of i . 1.1 When you hear hoofbeats, think horses, not zebras. Ockham’s razor is a logical principle attributed to the medieval philosopher and Franciscan monk William of Ockham. The principle states that one should not make more assumptions than the minimum needed. This principle is often called the principle of parsimony. It is essential for all scientific modeling and theory building. Figure 1: Ockhams Razor: Pluralitas non est ponenda sine necessitate. Franciscan monk William of Ock- ham (ca. 1285-1349) As Jefferys and Berger (1991) pointed out, the idea of measuring complexity and connecting the notions of complexity and prior probability goes back to Sir Harold Jeffreys’ pioneering work on statistical inference in the 1920s. On page 47 of his classical work [11], Jeffreys says: Precise statement of the prior probabilities of the laws in accordance with the condition of convergence requires that they should actually be put in an order of decreasing prior probability. But this corresponds to actual scientific procedure. A physicist would test first whether the whole variation is random against the existence of a linear trend; than a linear law against a quadratic one, then proceeding in order of increasing complexity. All we have to say is that simpler laws have the greater prior probabilities. This is what Wrinch and I called the simplicity postulate. To make the order definite, however, requires a numerical rule for assessing the complexity law. In the case of laws expressible by differential equations this is easy. We would define the complexity of a differential equation, cleared of roots and fractions, by the sum of order, the degree, and the absolute values of the coefficients. Thus s = a would be written 1
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
as ds/dt = 0 with complexity 1 + 1 + 1 = 3 . s = a + ut + 1 2 gt 2 would become d 2 s/dt 2 = 0 with complexity 2 + 1 + 1 = 4 . Prior probability 2 - m of 6 2 m 2 could be attached to the disjunction of all laws of complexity m and distributed uniformly among them. In the spirit of Jeffreys’ ideas, and building on work of Wallace and Boulton, Akaike, Dawid, Good, Kolmogorov, and others, Rissanen (1978) proposed the Minimum Description Length Principle (MDLP) as a paradigm in statistical inference. Informally, the MDLP can be stated as follows: The preferred M for explaining observed data D is one that minimizes: the length of the description of the theory (Ockham’s razor principle) the length of the description of the data with the help of the chosen theory.
Image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern