# Exercise 12 let p f θ θ θ where f θ s are

• Notes
• 379
• 80% (5) 4 out of 5 people found this document helpful

This preview shows page 79 - 82 out of 379 pages.

Exercise 12. Let P = { f θ : θ Θ } , where f θ ’s are probability densities, f θ ( x ) > 0 for all x ∈ R and, for any θ Θ, f θ ( x ) is continuous in x . Let X 1 and X 2 be independent and identically distributed as f θ . Show that if X 1 + X 2 is suﬃcient for θ , then P is an exponential family indexed by θ . Solution. The joint density of X 1 and X 2 is f θ ( x 1 ) f θ ( x 2 ). By the factor- ization theorem, there exist functions g θ ( t ) and h ( x 1 , x 2 ) such that f θ ( x 1 ) f θ ( x 2 ) = g θ ( x 1 + x 2 ) h ( x 1 , x 2 ) .
60 Chapter 2. Fundamentals of Statistics Then log f θ ( x 1 ) + log f θ ( x 2 ) = g ( x 1 + x 2 , θ ) + h 1 ( x 1 , x 2 ) , where g ( t, θ ) = log g θ ( t ) and h 1 ( x 1 , x 2 ) = log h ( x 1 , x 2 ). Let θ 0 Θ and r ( x, θ ) = log f θ ( x ) log f θ 0 ( x ) and q ( x, θ ) = g ( x, θ ) g ( x, θ 0 ). Then q ( x 1 + x 2 , θ ) = log f θ ( x 1 ) + log f θ ( x 2 ) + h 1 ( x 1 , x 2 ) log f θ 0 ( x 1 ) log f θ 0 ( x 2 ) h 1 ( x 1 , x 2 ) = r ( x 1 , θ ) + r ( x 2 , θ ) . Consequently, r ( x 1 + x 2 , θ ) + r (0 , θ ) = q ( x 1 + x 2 , θ ) = r ( x 1 , θ ) + r ( x 2 , θ ) for any x 1 , x 2 , and θ . Let s ( x, θ ) = r ( x, θ ) r (0 , θ ). Then s ( x 1 , θ ) + s ( x 2 , θ ) = s ( x 1 + x 2 , θ ) for any x 1 , x 2 , and θ . Hence, s ( n, θ ) = ns (1 , θ ) n = 0 , ± 1 , ± 2 , .... For any rational number n m ( n and m are integers and m = 0), s ( n m , θ ) = ns ( 1 m , θ ) = m m ns ( 1 m , θ ) = n m s ( m m , θ ) = n m s (1 , θ ) . Hence s ( x, θ ) = xs (1 , θ ) for any rational x . From the continuity of f θ , we conclude that s ( x, θ ) = xs (1 , θ ) for any x ∈ R , i.e., r ( x, θ ) = s (1 , θ ) x + r (0 , θ ) any x ∈ R . Then, for any x and θ , f θ ( x ) = exp { r ( x, θ ) + log f θ 0 ( x ) } = exp { s (1 , θ ) x + r (0 , θ ) + log f θ 0 ( x ) } = exp { η ( θ ) x ξ ( θ ) } h ( x ) , where η ( θ ) = s (1 , θ ), ξ ( θ ) = r (0 , θ ), and h ( x ) = f θ 0 ( x ). This shows that P is an exponential family indexed by θ . Exercise 13 (#2.30). Let X and Y be two random variables such that Y has the binomial distribution with size N and probability π and, given Y = y , X has the binomial distribution with size y and probability p . (i) Suppose that p (0 , 1) and π (0 , 1) are unknown and N is known. Show that ( X, Y ) is minimal suﬃcient for ( p, π ). (ii) Suppose that π and N are known and p (0 , 1) is unknown. Show
Chapter 2. Fundamentals of Statistics 61 whether X is suﬃcient for p and whether Y is suﬃcient for p . Solution. (i) Let A = { ( x, y ) : x = 0 , 1 , ..., y, y = 0 , 1 , ..., N } . The joint probability density of ( X, Y ) with respect to the counting measure is N y π y (1 π ) N y y x p x (1 p ) y x I A = exp x log p 1 p + y log π (1 p ) 1 π + N log(1 π ) N y y x I A . Hence, ( X, Y ) has a distribution from an exponential family of full rank (0 < p < 1 and 0 < π < 1). This implies that ( X, Y ) is minimal suﬃcient for ( p, π ). (ii) The joint probability density of ( X, Y ) can be written as exp x log p 1 p + y log(1 p ) π y (1 π ) N y N y y x I A . This is from an exponential family not of full rank. Let p 0 = 1 2 , p 1 = 1 3 , p 2 = 2 3 , and η ( p ) = (log p 1 p , log(1 p )). Then, two vectors in R 2 , η ( p 1 ) η ( p 0 ) = ( log 2 , 2 log 2 log 3) and η ( p 2 ) η ( p 0 ) = (log 2 , log 2 log 3), are linearly independent. By the properties of exponential families (e.g., Example 2.14 in Shao, 2003), ( X, Y ) is minimal suﬃcient for p . Thus, neither X nor Y is suﬃcient for p .