Polynomials Piecewise and Splines
Kwok-Leung Tsui Industrial & Systems Engineering Georgia Institute of Technology
2/18/2009 1
General Models
2/18/2009
2
Examples of Piecewise Polynomials
2/18/2009
3
Piecewise Polynomials & Splines
2/18/2009
4
Examples of Piecewise Polynomials
2/18/2009
5
Cubic Splines
2/18/2009
6
Model Fitting
2/18/2009
7
Number of Parameters in Splines
2/18/2009
8
Natural Cubic Splines
2/18/2009
9
Dimension Reduction of Splines
2/18/2009
10
Filtering and Feature Extraction
2/18/2009
11
South African Heart Disease Example
2/18/2009
12
South African Heart Disease Example
2/18/2009
13
Smoothing Splines
2/18/2009
14
Smoothing Splines
The over-parametrization problem is overcome by the penalty function.
2/18/2009
15
Fitting Smoothing Splines
2/18/2009
16
Degrees of Freedom of Smoothing Splines
2/18/2009
17
Example of Smoothing Splines
2/18/2009
18
Multi-dimensional Splines
2/18/2009
19
Other Smoothing Methods
Generalized Additive Models (GAM) Multivariate Adaptive Regression Splines (MARS) Kernel Methods
2/18/2009
20
Generalized Additive Models
Additive models (Stone, 1985).
Sj : smooth nonparametric function The curse of dimensionality is avoided since each of the individual additive terms is estimated using a univariate smoother Estimates of the individual terms explain how the dependent variable changes with the corresponding independent variables.
Generalized additives model (Hastie and Tibshirani, 1990).
The mean of the dependent variable depends on an additive predictor through a nonlinear link function. Nonparametric log-linear models for Poisson data
ij = n i + + j ,
2/18/2009
log( ij ) = log(n ) + log( i + ) + log( + j )
21
Generalized Additive Models
Classical link function g g (E[Y|X]) = E [Y|X]; identity link. (E[Y|X]) g = logit [Y|X]; logit link. g (E[Y|X]) = probit [Y|X]; probit link. g (E[Y|X]) = log [Y|X]; log-linear or log-additive models for Poisson count data.
example ( Additive logistic regression)
log
Pr (Y = 1 | X ) = + s1 ( X 1 ) + ... + s p (X p ) Pr (Y = 0 | X )
2/18/2009
22
Generalized Additive Models
Source: Snell,E.J. and Simpson,H.R (1991) A Handbook of Genstat Analysis
A fit using a linear model
A fit using a generalized additive model
2/18/2009
23
Generalized Additive Models (Summary)
What it is ?
A method of fitting a smooth relationship between two or more variables through a scatterplot of data points (cubic smoothing spline or kernel smoother) Provide an algorithm for simultaneously estimating all p functions.
When to use ?
The relationship between the variables is expected to be of a complex form, not easily fitted by standard linear or non-linear models. no a priori reason for using a particular model.
Difference between generalized linear models and generalized additive models Smoothing functions are used instead of regression parameters. example: gen...