Week 03 - Evaluating Estimators.pdf - Weeks 3 \u2013 Evaluating Estimators Professor Eric Beh School of Mathematical Physical Sciences University of

Week 03 - Evaluating Estimators.pdf - Weeks 3 u2013...

This preview shows page 1 - 7 out of 28 pages.

- Weeks 3 – Evaluating Estimators Professor Eric Beh School of Mathematical & Physical Sciences University of Newcastle STAT3010: Statistical Inference Autumn Semester 2020
In last weeks lecture we looked at how to (directly) determine the estimate of a parameter from a distribution, or model. However, one must ask what are the properties of this estimate - is it a good/bad estimator? What is the standard error of the estimator? What conditions does it satisfy? In this weeks lecture we shall be considering the following properties of the estimatesOverview Overview 2 Bias Variance Mean Squared Error Best Unbiased Estimators Invariance Consistency The Cramer-Rao Lower Bound for Unbiased Estimators Sufficiency o Fisher-Neyman Factorisation Theorem o Sufficiency principle o Rao-Blackwell Theorem o Minimal sufficiency o Sufficient statistics for exponential families o Ancillary statistics
BiasSuppose we have a sequence of random observations X1, X2, … , Xnthat are iid from a population that follows a normal distribution with a mean of μ. Then we know that Ex=1nEi=1nxi=1ni=1nExi=1ni=1nμ=μThis is a nice result. If we were to sample from this distribution, different samples would give different sample means. Although, on average, the sampling distribution of xhas a mean of µ. In this case, xis an unbiased estimator of If an estimate is not unbiased, then it is said to be biased. The bias, of an estimator of θ, θ, is measured byBiasθ= Eθ − θ= Eθ− θBias µ . 3
Example – Two Parameter Problem (Direct Estimate)Consider again a random sample of n iid observations from a normal distributionfx;μ,σ2=12πσ2exp12σ2x− μ2,xϵ ℝRecall from last weeks lectures that we SAID (not showed) that the MLE of µand σ2�μ=1ni=1nxiσ2=1ni=1nxi− �μ2unbiasedSo what estimate of the population variance can we use that is notbiased? is 4
. 2 5