Copyright
c
2010 by Karl Sigman
1
Introduction to reducing variance in Monte Carlo simulations
1.1
Review of confidence intervals for estimating a mean
In statistics, we estimate an unknown mean
μ
=
E
(
X
) of a distribution by collecting
n
iid
samples from the distribution,
X
1
, . . . , X
n
and using the sample mean
(1)
X
(
n
) =
1
n
n
X
j
=1
X
j
.
This is justified by the
strong law of large numbers (SLLN)
, which asserts that this estimate
converges with probability one (wp1) to the desired
μ
=
E
(
X
)
,
as
n
→ ∞
. But the SLLN does
not tell us how good the approximation is; we consider this next.
Letting
σ
2
=
V ar
(
X
) denote the variance of the distribution, we conclude that
(2)
V ar
(
X
(
n
)) =
σ
2
n
.
The
central limit theorem
asserts that as
n
→ ∞
, the distribution of
Z
n
def
=
√
n
σ
(
X
(
n
)

μ
) tends to
N
(0
,
1), the unit normal distribution.
Letting
Z
denote a
N
(0
,
1) rv, we conclude that for
n
sufficiently large,
Z
n
≈
Z
in distribution. From here we obtain for any
z
≥
0,
P
(

X
(
n
)

μ

> z
σ
√
n
)
≈
P
(

Z

> z
) = 2
P
(
Z > z
)
.
(We can obtain any value of
P
(
Z > z
) by referring to tables, etc.)
For any
α >
0 no matter how small (such as
α
= 0
.
05), letting
z
α/
2
be such that
P
(
Z >
z
α/
2
) =
α/
2, we thus have
P
(

X
(
n
)

μ

> z
α/
2
σ
√
n
)
≈
α,
which implies that the unknown mean
μ
lies within the interval
X
(
n
)
±
z
α/
2
σ
√
n
with (approxi
mately) probability 1

α
.
This allows us to construct
confidence intervals
for our estimate:
we say that the interval
X
(
n
)
±
z
α/
2
σ
√
n
is a
100(1

α
)%
confidence interval for the
mean
μ
.
Typically, we would use (say)
α
= 0
.
05 in which case
z
α/
2
=
z
0
.
025
= 1
.
96, and we thus
obtain a 95% confidence interval
X
(
n
)
±
(1
.
96)
σ
√
n
.
The length of the confidence interval is 2(1
.
96)
σ
√
n
which of course tends to 0 as the sample
size
n
gets larger.
In practice we would not actually know the value of
σ
2
; it would be unknown (just as
μ
is).
But this is not really a problem: we instead use an estimate for it, the
sample variance
s
2
(
n
)
defined by
s
2
(
n
) =
1
n

1
n
X
j
=1
(
X
j

X
n
)
2
.
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
It can be shown that
s
2
(
n
)
→
σ
2
, with probability 1, as
n
→ ∞
and that
E
(
s
2
(
n
)) =
σ
2
, n
≥
2.
So, in practice we would use
s
(
n
) is place of
σ
when constructing our confidence intervals.
For example, a 95% confidence interval is given by
X
(
n
)
±
(1
.
96)
s
(
n
)
√
n
.
The following recursions can be derived; they are useful when implementing a simulation
requiring a confidence interval:
X
(
n
+ 1) =
X
(
n
) +
X
n
+1

X
(
n
)
n
+ 1
,
S
(
n
+ 1)
2
=
1

1
n
S
(
n
)
2
+ (
n
+ 1)(
X
(
n
+ 1)

X
(
n
))
2
.
1.2
Application to Monte Carlo simulation
In Monte Carlo simulation, instead of “collecting” the iid data
X
1
, . . . , X
n
, we simulate it.
Moreover, we can choose
n
as large as we want;
n
= 10
,
000 for example, so the central limit
theorem justification for constructing confidence intervals can safely be used (e.g., in statistics
“out in the field” applications, one might only have
n
= 25 or
n
= 50 samples!). Thus we can
immediately obtain confidence intervals for Monte Carlo estimates.
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '07
 sigman
 Probability theory, Order theory, Monte Carlo method, Monotonic function

Click to edit the document details