- Weeks 3 –
Evaluating Estimators
Professor Eric Beh
School of Mathematical & Physical Sciences
University of Newcastle
STAT3010:
Statistical Inference
Autumn Semester 2020

In last weeks lecture we looked at how to (directly) determine the estimate of a parameter from a distribution, or model. However, one must ask what are the properties of this estimate - is it a good/bad estimator? What is the standard error of the estimator? What conditions does it satisfy? In this weeks lecture we shall be considering the following properties of the estimatesOverview
Overview
2
Bias
Variance
Mean Squared Error
Best Unbiased Estimators
Invariance
Consistency
The Cramer-Rao Lower
Bound for Unbiased
Estimators
Sufficiency
o
Fisher-Neyman Factorisation
Theorem
o
Sufficiency principle
o
Rao-Blackwell Theorem
o
Minimal sufficiency
o
Sufficient statistics for
exponential families
o
Ancillary statistics

BiasSuppose we have a sequence of random observations X1, X2, … , Xnthat are iid from a population that follows a normal distribution with a mean of μ. Then we know that E�x=1nE�i=1nxi=1n�i=1nExi=1n�i=1nμ=μThis is a nice result. If we were to sample from this distribution, different samples would give different sample means. Although, on average, the sampling distribution of �xhas a mean of µ. In this case, �xis an unbiased estimator of If an estimate is not unbiased, then it is said to be biased. The bias, of an estimator of θ, �θ, is measured byBias�θ= E�θ − θ= E�θ− θBias
µ
.
3

Example – Two Parameter Problem (Direct Estimate)Consider again a random sample of n iid observations from a normal distributionfx;μ,σ2=12πσ2exp−12σ2x− μ2,xϵ ℝRecall from last weeks lectures that we SAID (not showed) that the MLE of µand σ2�μ=1n�i=1nxi�σ2=1n�i=1nxi− �μ2unbiasedSo what estimate of the population variance can we use that is notbiased?
is
4

.
2
5