This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: 134 Chapter3 Continuous Distributions 159 150 147 160 155 142 143 151 154 133
151 146 140 146 137 148 154 157 142 153
135 144 135 165 118 158 126 147 123 140
125 151 153 158 144 163 150 150 137 164
137 156 139 134 171 144 160 147 155 175
162 160 149 149 158 152 165 131 150 120 (a) Construct an ordered stemaandleaf diagram using as stems 11, 12*, 12, mg
so on. (b) Construct & q—q plot using about every 5th observation in the ordered array anti
the corresponding quantiles of N(0, 1). (c) Does it look like X has a normal distribution? A chemistry.I major weighed 19 plain m&m’s® (in grams) on a i0.0001 scale. T'__
ordered weights are 0.7938 0.8032 0.8089 0.8212 0.8268 0.8383 0.8442
0.8490 0.8528 0.8572 0.8674 0.3734 0.8786 0.8850 0.8373 0.3920 [19069 0.9[50 0.9243 (a) Construct a stemandleaf diagram using threedigit leaves. with stems DIV,0. : .‘
0.8!. etc. (b! Construcl a q—q plot of these data and the corresponding quantiles for
standard normal distribution. N{0,1). (1:) Do these data look like observations from a normal distribution? Why?
Irx ammo. ﬁnd P[15.364 5 (X  7)’ 5 20.096]. Let the distribution of X be 3101.02) Show that the points of inﬂection of the gran .
ofthop.d.l.uEchur atxnuia. ' tl'Xis N[?5,IUG).eompute the conditional probability P(X > 85 X > 80). ll'XisN(u,nz),sltowthat E(IX—wrl} = a,t§,l'tr.HiNT: Write the integral represent' _ E_
E(lX — m) as 2f:°(x — ;z)f(x) dx, where f(x) is the p.d.f. of N(,u,02). Show that the constant c can be selected so that
. _ .2
f(x)=c3 ", —oo<x<oo, satisﬁes the conditions of a normal p.d.f. HINT: Write 3 = e“ or use a CAS. (S:
Section 5.2 .) Let X have the p.d.f. 2 “kl=5; Find the mean and the Variance at X. Hittr: Compute E(X) directly and evalua
E(XZ) by comparing its integral to the variance of MO, 1), or use a CAS. (
Section 5.2 .) _.2
e“/2, O<x<oo. ESTMAT0N We introduced estimation in Section 2.4, particularly maximum likelihood eras»
IN THE mation. We review that technique by considering a continuoustype distributi _ _.
CONTINUOUS namely the exponential distribution. having p.d.f. CASE 1
f(x; 6) 2 § e‘x/e, 0 < x < 00, where the parameter space is given by S2 = {9: I] 4: I! c 00}. Recall that th‘”
mean of X is u = 9. Thus if we have a random sample X1,X2, . . .,X',, from Section 3.5 Estimation In the Continuous Case 135 distribution, the sample mean Y could be considered an estimator of 6 using the
method of moments. That is, here we equate the ﬁrst moment of the empirical
distribution to that of the theoretical distribution. Once the sample is observed to be the values x1,x2. . . . ,xn, the computed mean f is an estimate of 6 and hopefully
is close to 6. The likelihood function in the continuous case is deﬁned in a fashion similar to
that in the discrete case. namely L(6) =f(x;; 6)f(xz; 6)  ' f(xn; 9), 6 8 S2, even though the p.d.f.f(x: 6) no longer equals probability. In our special situation, it is
L(9)= ie—xl/a ie—xz/e line
6 6 9 1 I7
= (g) (mi/9, where 0 < 6 < 00. Again we ﬁnd it to our advantage to take logarithms to obtain lnL(6) = —nln6 — d[lnL(6)] _
T — ‘5 + This latter equation has the solution 1 ﬂ
9 = ; ZX,‘ = 3,
i=1
and the second derivative of ln[L(6 )] at f is [31~ZE?=:II] _£_2"_7‘:_<0.
9:3 32 ﬂ 3 _ 32 f3 I Since it is negative, we have that 3: f is the maximum likelihood estimator of 6.
This is the same estimator as that found by the method of moments. EXAMPLE 3.51 Let X1,X2, . . .,X,1 be a random sample from N(u = 6,02), where the parameter
space S2 = {6: —oo < 6 < 00} and 02 is known. Then 22 1 2 2 [4(6) 2 n e—(xi—Q) /20
[:1 «2710 1 n m (9)2/202
= e' " —00 < 6 < oo.
(«/2710> ’ ln L(6) = —nln(~/i;a) — Z +— [: 136 Chapter3 Continuous Distributions EXAMPLE m and dllnL(9)]_ ” 2me>h
d6 ”gay—2‘41 The latter equation can be written n
ZX,’ — [16 = 0,
[:1 whichhas the solution 6 = f. Since [he second derivative of In L(6) is —n/a2 <
6 = X is the maximum likelihood estimator of 6. Let X1,X2, . . .,X,, be a random sample from a gamma distribution in which a
known, but 9 is an unknown positive value. The likelihood function is ” 1 oil —x,./‘H
L(9) = n l‘(oz)6“ .11 e [:1 Qua 1 ” 1
=[ :1 —(x1x2 ~x,,)“_le_2"’/6, 0 < 6 < 00. W ELI xi ln L(6) = —n In F(cx) — nor 111(6) + (oz — 1) ln(x1,r2   x,,) — 9 d[ln L(9)] —n(x d6 9 the solution of which is 6 = ELIriff?“ a 3/01. It is easy to show that the semi!"
derivative is negative at this solution: so 5 : Y/a is the maximum likeliho;
estimator of E. This is actually a generalization of the illustration involving
exponential distribution; we see this by letting or = 1. In both of these examples, 3.51_and 3.52, the maximum likelihood estimators n_:9—2.
6 are unbiased because (A)? = E(X) = u, the mean of the underlying distribution,
hence, in those examples. E(X)=6 respectively. The preceding discussion can be extended to two or more unknown paramet use.
For two parameters, say 91 and 62, the p.d.f. of X is ﬂnﬂhﬂg) with paramet“;
space 52. With a random sample from this distribution the likelihood function is L(91792) =f(X11 91992)f(X2; 91,92)  : fix“: 91.32)
where (61, 62) E‘ 52. Say the values 61 = u1(x1,x2,. . .,x,) and 62 = uﬁxhxg, ...,x,1) Section 3.5 Estimation in the Continuous Case 137 maximize L(61,62). Then
@1 =u1(X1,X2,...,X,Z) and 52:112(X1,X2,...,X,,) are the maximum likelihood estimators of (91 and 62, respectively. In the method of
moments, we equate the ﬁrst two moments of the empirical distribution to those
of the theoretical distribution and solve thBSe equations for 61 and 62 to get the
method of moments estimators. EXAMPLE 3.53 Let X1, X2, . . . ,X,, be a random sample from N(61,62), where 2 Q={(61,62): —oo <61 :11 <oo,0<02=a <00}. _._ 2
L(91,92): 1:1 27192 g‘(x191)/292 : < 1 n e— Z(~Vi—91)2/292
«27:62 ' 2122:1(xi— 902 I1
ln[L(61, 92)] = —E 111(27192) — 292 The ﬁrst partial derivatives of ln[L(01,02)] are W_ii(x.—eo=o 391 _ 62 (.21 3 ln[L(01,62)]_ _ x —6 2
802 _2—’;2 20221.2( 1 1): Solving these equations simultaneously, we see that the maximum likelihood esti
mators of (91 and 62 are A _ A 1” _
01=X and 62:;Z(X,—X)2= respectively (We need results from multivariate calculus to prove that these
solutions actually provide a maximum for L.) Here E( (91 )— _ 01; however we show
that E( 31 } aé I92 and hence 192 ii a biased estimator of 62. This inequality 15 true
because ﬂog—902:2(X1~Y)2+"(Y—91)2;
[:1 i=1 E[Z (X. — 602] = E[Z (X. — Y )2] + E[n( 7 — 902].
[:1 i=1 138 Chapter?! Oontmumls Distributions EXAMPLE EEI Since <72 = 62, we have r192 = E[Z: (X, — Y )2] + 92. [:1 because EKY — 61)2/(62/rz)] = 1. Accordingly E[i(X,~ — 7? )2] = (n —1)92;
i=1 n—l E[ZL1(X1 — '2? )2] z 92. Note that H =<";1>E[————27=1::;W] = me» That is, 52 : Z;L1(X,~ — 3Y— )2/(n — 1) is not the maximum likelihood estimator but is the unbiased estimator ofo2 = 92. This has been alluded to earlier.
_— 15(93): E[Z£~’=1(Xz—7)2] Let X1,X2, . . . ,X,, be a random sample from a gamma distribution with a = 61 and
6 = 62, where 61 > O, 62 > 0. It is difﬁcult to maximize 1 — _ .
L(91962) : _m9_1(x1x2 . . .xn)91 1e EX,/02 [F(91)]”62 with respect to 91 and 62 due to the presence of the gamma function F(91). We
can, however, use the method of moments by equating the ﬁrst two theoretical
moments. namely a = 9162 and u1 :2 $922, to the ﬁrst two moments of the empirical
distribution. That is, 6162 :7 and 9,922 = V, the solutions of which are —2
~ X ~ 9:— d 9:
1 V an 2 the method of moments estimators of 91 and 9:.
“—
There is a huge advantage of using maximum likelihood estimators in what is
called the regular case. There are some mathematical conditions needed to have
regular cases, but the main one is that the parameters do not appear in an endpoint
of the support (space) of X. For illustration of a p.d.f. that is not a regular case is 1
f(X19) = 69 Section 3.5 Estimation in the Continuous Case 139 where Q = {6: 0 < 6 < 00}. Here 6 is the upper endpoint of the support. While we can maximize n L(9)=1’[( [:1 by making 6 as small as possible, namely 6 = max(X,); for if it were any smaller,
the likelihood function would equal zero. That is, L(6) = 0 if 6 < x, for some xi,
and zero would not be a maximum for L(6). The p.d.f.s given in Examples 3.51
through 3.5—4 are all regular cases. In regular cases, the maximum likelihood estimators have approximate normal
distributions provided a is large enough. We do not prove this now, but this is the
real reason that the normal distribution is so important to statisticians. It is not
the fact that some underlying distributions are close to being normal; remember
that no model is exactly right but many are useful. But it is the fact that many
estimators, including maximum likelihood estimators. have approximate normal
distributions. As you will soc Shortly, this is a most important fact and is even true
in regular cases of discrete distributions in which the parameters do not enter the
support of X. What does this mean in the examples that we have studied thus far? In our
ﬁrst illustration of the exponential distribution, 6 = 7 is approximately N (6, 6'2 / n)
became there it; = H and a; = 02/n = 62/11. In Example 3.51, we will see later
that f has an exact normal distribution, N(6, 02m}. ln Example 3.52, 6 = Y/a is
approximately N(0,62ran] because E(6)=E(Z> = ﬂ :9 a a Var(6) = (§)2Var()_() = (0712—)(33—2) = 32;. In Example 3.53, it is true that 6] = Y is N(61,62/n) and 62 = V is approximately
N(62,2622/n) for large n even though V is a biased estimator of 62. The statement
about V being approximately normal with that mean and variance can not be
proved at this time; so we simply accept it. Imthe discrete case in which Y =
X1+X2 +   t+Xn isb(n,p = 6), the m.l.e. of6, 6 = )7 = Y/n, has an approximate
N[6, 6(1  6),!n] distribution because E(Z) = E(Y) : E2 = 6 and Var(—}:) = (1)2n6(1 —6) = m n n [’1 n H n Not only are the maximum likelihood estimators in regular case approximately
normal, but it is also true that the mean Y of a random sample of size rz from
any distribution with mean u and ﬁnite variance :12 is approximately N(u, az/n),
provided the sample sin: n is large enough. This latter fact is essentially a statement
of what is called the Central Lknlt Theorem. Right now we accept these statements
as facts. and investigate one important use of them, namely that of ﬁnding
conﬁdence intervals [or unknown parameters. This is why these approximate
normal distributions are so important. Let us say that an estimator U = u{X1,X2....,X,,) of 6 has a normal or
approximate normal distribution with unknown mean 6 and variance 03., which at 140 Chapter3 Dominuous Distrlbutions the moment we assume is known. Then U—Q Cu is (approximately) N(0,1). From normal tables, we have P<—2 5 U—Q 5 2) ~0.95.
at} This is actually 0.9544 if U is exactiy normal. Solving these inequalities for 6, .. 
have the equivalent probability statement that P(U — 2a“ 5 H s t: + 2%) m 0.95. Say the random sampie is observed to hex1.3, . . . ,x,, and the statistic U computerf
to be u = 1401,12,. ..,x,,}. Atthie point. if 01; is known, we are fairly conﬁdent tn: .
the computed interval 1((x1,.r2,. . . ,x”) — 2m, to u(x1,x2, . . . ,x,,) + Zau covers 6 because the probability of that occurring before the sample was taku_,
was very high1 namely 0.95. Thus we call the interval, written for simplicity :.
u j: 2cm, an approximate 95 percent conﬂuence interval for 6 and the 95 percent if '
thc conﬁdence coefﬁcient. Now if EU is unknown and depends upon parameters itself, we estimate tho .
parameters appropriately and obtain an. estimate of cm, say EU. For illustration, ”
Section 2.5, with Y being b(n,p = 9), we noted that the standard error ~ /(Y/n)(1— Y/n)
Grin =\ _ H was an estimate of t/6(1 — 6)/n, the standard deviation of Y/n. Hence we used y limit! “rtr)
; “it W—H as an approximate 95 percent conﬁdence interval forp = 6.
The other 95 percent conﬁdence interval given in Section 2.5 was for it, namely" f12(%>, where the standard error sfﬁ is an estimate of the standard deviation, a/ﬂ, of
75. Clearly, by using values other than 2 from the normal table, we may obtain an:
80. £90, a 91m 11 99.73 percent conﬁdence interval for the unknown parameter by
using 1.282. 1.645. 2.576, or 3, respectively. At this point, there is always a question about how large n must be to have
these intervals. be fairly good in malntalning that advertised percentage. There is
no easy answer. because it depends upon the situation. For illustration, let us say
with a oonljnuouslype symmetric distribution with single mode and the variance
known, It can be quite small. maybe as little as r: = d or 5, for a conﬁdence interval
for pt. As the distribution becomes skewed, more observations are needed. As a section 3.5 Estimation in the Continuous Case 141 matter of fact. with the standard deviation being estimated by the standard error,
we would like an to be at least 30 and much more it the underlying distribution
is badly skewed. Moreover. with discrete distributions, like the binomial, n = 50
or more is not unreasonable. particularly it p is not close to 1/2. We can think
of highly skewed discrete distributions in which we would prefer the sample size
to be several hundred. With these warnings, a rough rule would be to have n at
least 30, and we are more comfortable with 50 without additional assumptions.
Now. of course. one can always give an interval like 3 d: ZS/ﬁ as a conﬁdence
interval for p41 as long as it is noted that if n is not large enough, this may have
a conﬁdence coefﬁcient much different from 95 percent. Most of the time there
conﬁdence coefﬁcients would be in the 905, but we have seen them as low as
65 percent. 3.51 Find the maximum likelihood estimates for 61 = ,u. and 92 = 02 if a random sample
of size 15 from N(u.oz) yielded the following values: 31.5 36.9 33.8 30.1 33.9
35.2 29.6 34.4 30.5 34.2
31.6 36.7 35.8 34.5 32.7 A. random sample X1,X2.. . . . A), of site #15 taken from NM, 02), where the variance
.9 z a? is suchthatﬁ c. H a: veranda isnknown realoumber. Show tharthe maximum
likelihood estimator for H is B = [1,01] 29:1 (X, — In]: and Iliat this estimator is an
unbiased estimator ow. Lemme} =Hx“'.ﬂ : .t <1.a o n 219:0 < e < 00}. Let X1,X2....,X,. denote
I random sample of size n from this distribution. {I} Sketch tho p.d.f. of X for [I] 6 :2 1:21th 2 1, and (iii) 9 = 2. (b) Show that El: —nj1n in}; X5} is the maximum likelihood estimator of E. {o} For each of the following three sets of 10 observations from this distribution,
calculate the values of the maximum likelihood estimate and the method of
moments estimate [or til. [i] 0.0255 0.305! 0.0278 0.8971 0.0739
0.3191 [1.7379 0.3671 0.9763 0.0102
(ii) 0.9960 0.3!25 0.4371! 0.7464 0.8278
0.9518 0.9924 0.7112 0.2228 0.8609
(iii) {3.4698 0.3675 0.5991 0.9513 0.6049
0.9917 {3.1551 0.0710 0.2110 0.2154
Letf(x: 9} :2 “mull—9319. 0 < x < 1. 0 < 9 < 00.
(a) Show that the maximum likelihood estimator of6 is 0: —(l/n) 27:1 in X,.
(b) Show that H5) = 6 and thus 015 an unbiased estimator of6. Let X1, X2, . . . . X” be a random sample from a distribution with p.d.f.
f(xz 6) = 6‘0“”, 6 < x < 00, where ——oo < 9 «c on. Find 11:: maximum likelihood estimator of 6. HINT: This is not
a regular case of estimation Let X1.X2.. . . .Xﬂ be a random sample from a distribution with distribution function 0. x51, F(x): 1 H
1—(—>, 1<x<oc, X where B > I). Find the maximum likelihood estimator of (9. 142 Chapter3 Gentlnuous Distributions THE CENTRAL
LIMIT
THEOREM 3.57 The sample mean T is the maximum likelihood estimalor not 9 if the underlying,
p.d.f_. is f(x; 6) 2: {trapW, g a: 4‘ t: on, where H 2! IU. For this distributionr
E( X ) = 6 and Val: Y "J 2: ﬁlm. where n is the sample sine. Use the fact that T is approximately ma. 91m) to construct an approximate 95 percent conﬁdence
interval [or B. Let it" be the mean of a. random sampte of size n tram a distribution with unknown
mean ,u. and known variance «1. Then I :l: 211qu can serve as an approximate 93 percent conﬁdence interval [or u. If n == 4, find at so that this conﬁdence interval
is X :l: 0.5. In Section 2.5 we found that the mean Y of a random sample of size n from a
discrete distribution with mean ,tt and variance 02 > 0 is a random variable with
the prOperties that 2
E(7)=n and Var(3(_)=—.
n Thus. as it increases, the variance of Y decreases. Consequently, the distribution
of 1" clearly depends on n. and we see that we are dealing with sequences of
distributions. Also note that the probability becgmes concentrated in a small
interval centered at pl. That is. as. it increases. X tends to converge to ,u, 01'
( Y L .11.] tends to converge to l] in a probability sense. When sampling from
a normal distribution, we will Show, in Chapter 6. that the distribution of it
is N(u,02/n).
In general, if we let W= ”(Yﬂt): where Y is the sum of a random sample of size n from some distribution with meanE
u and variance 02. then for each positive integer n, O O/x/ﬁ 0N3 _ E(W)=E[:f:£] — E(X)_“ “—“ Var(W) : E(W2) = HERE] _ E[( Y — I02] 02/” _ — —— = — 1.
02/17 02/17 02/11 Thus, while Y — u “degenerateﬂ” to zero. the factor ﬁ/o in JR 3? — u)/a
“spreads nm” the probability enough to prevent this degeneration. What then is
the distribution of W as n, Increases? ﬁne obscuration that might shed some light
on the answer to this question can be made immediately. If the sample arises from
a normal distribution, we show in Example 6.2—4 that f is N(u,az/n) and hence
W is N(0,1) for each positive n. Thus, in the. limit1 the distribution of W must
be N (0, 1). So if the solution of the question does not depend on the underlying
distribution (i.e., it is unique), thit: answer must he N(ﬂ.1). As we will see. this
is exactly the case, and this result is so important it is called the Central Limit
Theorem, the proof of which is given in Section 6.3 . Agoandlx c ' Answers to OddNumbered Exercises 349 3.47 (a) *' _ _ .
3w 1mm We; Darth:
11' 3 1 1
155 n 3 2 '3
135. 54 ‘1 5
154 13.4 3 5
15. 551145 a 14
1442 443414455444 5 a: 
u. . 5.15 35555.59 '5 an
15. 1590111224544 14 51
15!! SE55? 5555 3 H!
15: a m 5 4 '5 10'
16; 55 a 4
1?; 1 1 2
15. 5 '1 1
3.49 0.025.
3.41! 0.514. 3.443 c=1/‘/Tz/in3.
1 29] =12=33.4267; 9‘2 =35=50980
1 3.53 (0) (i) 0: 0.54955 = 0.5975. (ii) (3 = 22101.5 = 2.4004, (iii) a“ = 09588.9~ = 0.8646. 3.55 0 = min(X1,X2, . . . ,Xn).
_ __ x x
3.5? [x(1 —2/ﬁ], 1U +2IKJ51] or [W’mi'
5.6} 0.4772.
3.63 0.8185.
3.65 0.6247.
' 3.6? 0.95. 3.6—9 (a) The frequencies are 5, 6, 12, 18, 31, 31, 20, 17, 6, 4; the ﬁfth and sixth classes: it) yes.
3.6” 0.9522. 3.61} P(Y 5 35) 8 0.6462 using normal approximation; P(Y 5 35) 2 0.5725 using the gamma distribution
with 01 = 16 and (9 = 2. 3.7] (a) 0.2878, 0.2881; (b) 0.4428, 0.4435: [cl0.1550, 0.1554.
3.73 0.9258 using normal approximation, 0.9258 using binomial. ...
View
Full Document
 Fall '07
 Guggenberger

Click to edit the document details