Chapater1 Probability Models & Chapater2 Principles of Data Reduction

Chapater1 Probability Models & Chapater2 Principles of Data Reduction

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Chapter 1 Probability Models Broadly speaking, there are two types of models. 1. Parametric models : A family of distributions F = { F θ : θ ∈ Θ } is said to be a parametric family iff Θ ⊂ R d for some fixed positive integer d , and each F θ is known when θ is known. The set Θ is called the parameter space and d is called its dimension . (Note that the form of distributions is known except for some fixed num- ber of unknown parameters.) A parametric model refers to the assumption that the population dis- tribition is in a parametric family. A parametric model with very high dimension d is not very useful. A parametric family is identifiable iff θ 1 6 = θ 2 implies that F θ 1 6 = F θ 2 . (If we know a sample is from F θ , then we can uniquely determine (or identify) θ . Therefore, we can talk about estimating θ .) e.g. Normal family: F = { N ( μ,σ 2 ) : μ ∈ R,σ 2 > } . Here θ = ( μ,σ 2 ). 2. Non-parametric models : not parametric models. Here, the form of distributions cannot be decided by a fixed number of unknown pa- rameters, hence it is sometimes called an infinite dimensional problem. In non-parametric models, minimal assumptions are usually made re- garding the form of distributions, e.g., (a) All continuous d.f.’s. (b) All symmetric d.f.’s. (c) All d.f. with finite moments of order r > 0. (d) All d.f. with p.d.f.’s. (e) All d.f. continuous and symmetric. There are also so called Semi-parametric models , which are mixtures of parametric and nonparametric models (such as Cox’s model). They could be thought 1 to include parametric and nonparametric models as special cases. On the other hand, they can also be regarded as nonparametric models. In this course, we shall mostly concentrate on parametric models . Two very useful parametric models will be introduced below: (a). exponential family; (b). location-scale family . 1.1 Exponential family Definition 1.1 (Exponential family) A family F = { F θ : θ ∈ Θ } of distri- butions is said to be a k-parameter exponential family if the distributions F θ have densities of the form (either with Lesbegue or counting measures) f θ ( x ) = exp k X j =1 η j ( θ ) T j ( x )- B ( θ ) h ( x ) ≡ exp k X j =1 η j ( θ ) T j ( x ) C ( θ ) h ( x ) , with respect to some common measure μ . Here all the functions are real-valued, h ( x ) ≥ . Remark 1.1 • Since R f θ ( x ) d x = 1 , we find that B ( θ ) = ln Z exp k X j =1 η j ( θ ) T j ( x ) h ( x ) d x . Therefore, η j ( θ ) ’s are “free” parameters, while B ( θ ) depends on η j ’s and hence not a free “parameter”....
View Full Document

This note was uploaded on 02/01/2012 for the course MATH 5010 taught by Professor D during the Spring '11 term at HKU.

Page1 / 26

Chapater1 Probability Models & Chapater2 Principles of Data Reduction

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online