This preview shows pages 1–5. Sign up to view the full content.
Insight by Mathematics and Intuition
for understanding
Pattern Recognition
Waleed A. Yousef
Faculty of Computers and Information,
Helwan University.
February 27, 2010
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document Basics of Statistics:
A random variable (or vector) is denoted by upper case letters,
e.g.,
X
.
Independent observations (realizations) of this r.v. are called
independent and identically distributed (i.i.d.), e.g.,
x
1
,...,x
n
.
Estimator is a realvalued function that tries to be “close” in
some sense to a population quantity.
How “close”?
Deﬁne a loss function, e.g., the Mean Square Error (MSE):
L
(
ã
μ,μ
) =
(
ã
μ

μ
)
2
.
And, deﬁne the Risk to be the Expected loss: E (
ã
μ

μ
)
2
.
Important Decomposition for any estimator
ã
μ
:
E (
ã
μ

μ
)
2
= E ((
ã
μ

E
ã
μ
) + (E
ã
μ

μ
))
2
= E (
ã
μ

E
ã
μ
)
2
+ E (E
ã
μ

μ
)
2
+ 2 E [(
ã
μ

E
ã
μ
) (E
ã
μ

μ
)]
= Var
ã
μ
+
Bias
2
(
ã
μ
)
Estimation of
μ
X
Sample mean
X
as an estimator of
μ
X
:
ã
μ
X
=
1
n
∑
n
i
=1
x
i
.
E
X
= E
1
n
n
Ø
i
=1
x
i
= E
X
(=
μ
)
Bias
(
ã
μ
) = E
ã
μ

μ
= 0
Var
ã
μ
=
1
n
2
Ø
i
σ
2
+
Ø
i
Ø
j
Cov (
X
i
,X
j
)
=
1
n
σ
2
This means that from sample to sample it will vary with this variance.
An estimator with zero bias is called “unbiased”. This means that on average it will be exactly
as what we want.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
σ
2
σ
2
= E (
X

μ
)
2
= E
X
2

μ
2
ä
σ
2
=
1
n

1
Ø
i
(
x
i

X
)
2
=
1
n

1
Ø
i
x
2
i

n
X
2
E
ä
σ
2
= E
1
n

1
Ø
i
x
2
i

n
X
2
=
1
n

1
±
n
E
X
2

n
E
X
2
B
=
1
n

1
n
²
σ
2
+
μ
2
³

n
σ
This is the end of the preview. Sign up
to
access the rest of the document.
This note was uploaded on 01/06/2011 for the course IT 342 taught by Professor Waleeda.yousef during the Spring '10 term at Helwan University, Helwan.
 Spring '10
 WaleedA.Yousef

Click to edit the document details