This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Chapter 1 Probability Models Broadly speaking, there are two types of models. 1. Parametric models : A family of distributions F = { F θ : θ ∈ Θ } is said to be a parametric family iff Θ ⊂ R d for some fixed positive integer d , and each F θ is known when θ is known. The set Θ is called the parameter space and d is called its dimension . (Note that the form of distributions is known except for some fixed num ber of unknown parameters.) A parametric model refers to the assumption that the population dis tribition is in a parametric family. A parametric model with very high dimension d is not very useful. A parametric family is identifiable iff θ 1 6 = θ 2 implies that F θ 1 6 = F θ 2 . (If we know a sample is from F θ , then we can uniquely determine (or identify) θ . Therefore, we can talk about estimating θ .) e.g. Normal family: F = { N ( μ,σ 2 ) : μ ∈ R,σ 2 > } . Here θ = ( μ,σ 2 ). 2. Nonparametric models : not parametric models. Here, the form of distributions cannot be decided by a fixed number of unknown pa rameters, hence it is sometimes called an infinite dimensional problem. In nonparametric models, minimal assumptions are usually made re garding the form of distributions, e.g., (a) All continuous d.f.’s. (b) All symmetric d.f.’s. (c) All d.f. with finite moments of order r > 0. (d) All d.f. with p.d.f.’s. (e) All d.f. continuous and symmetric. There are also so called Semiparametric models , which are mixtures of parametric and nonparametric models (such as Cox’s model). They could be thought 1 to include parametric and nonparametric models as special cases. On the other hand, they can also be regarded as nonparametric models. In this course, we shall mostly concentrate on parametric models . Two very useful parametric models will be introduced below: (a). exponential family; (b). locationscale family . 1.1 Exponential family Definition 1.1 (Exponential family) A family F = { F θ : θ ∈ Θ } of distri butions is said to be a kparameter exponential family if the distributions F θ have densities of the form (either with Lesbegue or counting measures) f θ ( x ) = exp k X j =1 η j ( θ ) T j ( x ) B ( θ ) h ( x ) ≡ exp k X j =1 η j ( θ ) T j ( x ) C ( θ ) h ( x ) , with respect to some common measure μ . Here all the functions are realvalued, h ( x ) ≥ . Remark 1.1 • Since R f θ ( x ) d x = 1 , we find that B ( θ ) = ln Z exp k X j =1 η j ( θ ) T j ( x ) h ( x ) d x . Therefore, η j ( θ ) ’s are “free” parameters, while B ( θ ) depends on η j ’s and hence not a free “parameter”....
View
Full
Document
This note was uploaded on 02/01/2012 for the course MATH 5010 taught by Professor D during the Spring '11 term at HKU.
 Spring '11
 D
 Probability

Click to edit the document details