{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Chapater1 Probability Models &amp; Chapater2 Principles of Data Reduction

# Chapater1 Probability Models & Chapater2 Principles of Data Reduction

This preview shows pages 1–3. Sign up to view the full content.

Chapter 1 Probability Models Broadly speaking, there are two types of models. 1. Parametric models : A family of distributions F = { F θ : θ Θ } is said to be a parametric family iff Θ R d for some fixed positive integer d , and each F θ is known when θ is known. The set Θ is called the parameter space and d is called its dimension . (Note that the form of distributions is known except for some fixed num- ber of unknown parameters.) A parametric model refers to the assumption that the population dis- tribition is in a parametric family. A parametric model with very high dimension d is not very useful. A parametric family is identifiable iff θ 1 6 = θ 2 implies that F θ 1 6 = F θ 2 . (If we know a sample is from F θ , then we can uniquely determine (or identify) θ . Therefore, we can talk about estimating θ .) e.g. Normal family: F = { N ( μ, σ 2 ) : μ R, σ 2 > 0 } . Here θ = ( μ, σ 2 ). 2. Non-parametric models : not parametric models. Here, the form of distributions cannot be decided by a fixed number of unknown pa- rameters, hence it is sometimes called an infinite dimensional problem. In non-parametric models, minimal assumptions are usually made re- garding the form of distributions, e.g., (a) All continuous d.f.’s. (b) All symmetric d.f.’s. (c) All d.f. with finite moments of order r > 0. (d) All d.f. with p.d.f.’s. (e) All d.f. continuous and symmetric. There are also so called Semi-parametric models , which are mixtures of parametric and nonparametric models (such as Cox’s model). They could be thought 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
to include parametric and nonparametric models as special cases. On the other hand, they can also be regarded as nonparametric models. In this course, we shall mostly concentrate on parametric models . Two very useful parametric models will be introduced below: (a). exponential family; (b). location-scale family . 1.1 Exponential family Definition 1.1 (Exponential family) A family F = { F θ : θ Θ } of distri- butions is said to be a k -parameter exponential family if the distributions F θ have densities of the form (either with Lesbegue or counting measures) f θ ( x ) = exp k X j =1 η j ( θ ) T j ( x ) - B ( θ ) h ( x ) exp k X j =1 η j ( θ ) T j ( x ) C ( θ ) h ( x ) , with respect to some common measure μ . Here all the functions are real-valued, h ( x ) 0 . Remark 1.1 Since R f θ ( x ) d x = 1 , we find that B ( θ ) = ln Z exp k X j =1 η j ( θ ) T j ( x ) h ( x ) d x . Therefore, η j ( θ ) ’s are “free” parameters, while B ( θ ) depends on η j ’s and hence not a free “parameter”. Many common families are exponential families. These include the normal, gamma, beta distributions, and binomial, negative binomial etc. The exponen- tial families have many nice properties, as can be seen below. The above representation is not unique, since η j T j = ( j )( T j /c ) .
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 26

Chapater1 Probability Models & Chapater2 Principles of Data Reduction

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online