UNIVERSITY OF NEW SOUTH WALES
SCHOOL OF MATHEMATICS AND STATISTICS
MATH3821 Statistical Modelling and Computing
Session Two 2006
Units of Credit
6 UOC
Prerequisites
MATH2831 / MATH2931
Syllabus
The main purpose of this course is to give an introduction to
The Bayesian linear model
Linear model:
y = X +
where y is a vector of responses, X is an n p design matrix
N (0, 2I) is a vector of independent normal errors and = (0, ., k )T
(k = p + 1) is a vector of parameters.
Unknown parameters: = (, 2). How do w
Sequential tests
Binomial regression model: responses yi, i = 1, ., n, each yi is
modelled as a binomial random variable with parameters ni and pi.
Probability pi of success on a binomial trial is allowed to depend on a
vector of covariates xi = (1, xi1,
Linear smoothers
Scatterplot smoothing:
responses y,
vector of predictors x and corresponding
yi = f (xi) + i
For a given x, a scatterplot smoother maps y to a function over the
range of the predictors (our estimator f(x) of f (x).
Write g(y|x) for this m
Wald tests
Binomial regression model: responses yi, i = 1, ., n, each yi counts
the number of successes in ni binomial trials.
Probability pi = P (xi) of success for yi depends on covariates xi =
(1, xi1, ., xik )T .
Model:
P (xi)
= xTi .
1 P (xi)
Maximum
Example: Sydney maximum temperatures
Last lecture: the Sydney maximum temperatures data set. mos.df with:
Maxtemp - daily maximum temperature at Sydney airport
Modst - 24 hour physical model forecast of 10am surface temperature
Modsp - 24 hour physical mo
Poisson regression
Consider regression models for count data where the counts can be
considered realizations of Poisson random variables, such that
y P o()
Observations y1, ., yn are independent, Poisson with mean i, then
exp(i)yi
fY (yi) =
, yi 0.
yi !
c
Non-parametric regression
So far, we have discussed only about linear models. That is, models
that are linear in the parameters.
E.g. a polynomial model is linear in the coefficients even though it is a
non-linear function of the predictor variables.
Poly
Choosing models and transformations
Consider a logistic regression model for a binary response in which
there is a single quantitative predictor x.
So the responseas are y1, ., yn (binary), the corresponding predictor
values are x1, ., xn and pi = P r(yi
Analysis of simulated data
We usually conduct a simulation study to estimate quantities of interest
for some stochastic system.
Suppose we want to estimate some parameter (the mean service time
in a queueing system, for example).
We have some random varia
MISE for kernel estimators
kernel density estimator:
n
X
1
x xi
f(x) =
K
.
nh i=1
h
h is called the bandwidth, K(u) is the kernel function.
Z
Z
K(u)du = 1, uK(u)du = 0,
Z
2
u2K(u)du = K
< .
MISE for kernel estimators
Approximate (large sample) expression
Estimation of an unknown density function
Let x1, ., xn be a random sample from some density function f (x).
How can we estimate f ?
Nonparametric density estimation methods
Compare with parametric estimation methods (estimation of parameters
and in a No
Markov chain Monte Carlo
More than any other technique, Markov chain Monte Carlo (MCMC)
has been responsible for a current resurgence in Bayesian statistics,
since its application allows a vast range of Bayesian models, previously
thought to be completely
Additive models
Recall: an additive model takes the form
yi = f1(xi1) + . + fk (xik ) + i
where the terms fj (), j = 1, ., k are general smooth univariate
functions.
The additive model structure allows us to visualize the mean response
surface and underst
Assignment 1
For the assessment of assignment 1, you should address the following
in your report.
What is the goal of your statistical analysis: Depending on the problem
you decide to work on, state clearly what the goal of your analysis is.
Data collecti
The Bayesian linear model
Linear model:
y = X +
where y is a vector of responses, X is an n p design matrix
N (0, 2I) is a vector of independent normal errors and = (0, ., k )T
(p = k + 1) is a vector of parameters.
Unknown parameters: = (, 2). How do w
Inverse transform method
Last lecture we discussed simulation of discrete random variables with
the inverse transform and rejection methods.
Can we simulate continuous random variables similar methods?
1
Inverse transform method
A random variable with any
The R project
R is one of the most widely used software for Statistical computation
and Graphics.
You can download R freely at
http:/cran.r-project.org
(for UNIX, LINUX, PC or Mac).
R or R-studio (which is more user friendly) will be used for this course.
Smoothing with multiple predictors
Our last few lectures have been devoted to scatterplot smoothers: we
wish to estimate a smooth function f in the model
yi = f (xi) + i
where yi, i = 1, ., n are a set of responses and xi, i = 1, ., n are a
corresponding
Penalised spline regression
As we have seen before, the roughness of the fit in the spline regression
model is due to there being too many knots in the model.
One way to overcome the problem of knot selection is to retain all knots,
but constrain their in
Generalized linear models
Responses y1, ., yn, corresponding vectors of predictors x
cfw_x1, ., xk . Responses are independent.
=
Linear models of the form
E(yi) = i = xTi ,
yi N (i, 2)
form the basis of most analyses of continuous data, where xTi
corresp
Introduction to simulation
What is simulation? Attempt to generate deterministic sequences of
numbers which cant be distinguished from true random numbers by
any appropriate statistical test.
Simulation has many applications. It is most often used in the
Course Summary
Revision of linear models and Introduction to R
Generalized linear models (non-Normal data)
Nonparametric regression (scatterplot smoothing, splines, additive
models)
Bayesian linear models
Other computing techniques (Nonparametric density
Revision of Linear models
Responses yi, corresponding values of k predictors xi1, ., xik , i =
1, ., n.
yi = 0 + 1xi1 + . + k xik + i
where the i are normal, zero mean errors, uncorrelated with common
variance 2.
1
Revision of Linear models
Matrix formula