3 estimation phase transition in this section we

This preview shows page 7 - 10 out of 41 pages.

3 Estimation phase transition In this section we discuss how to evaluate the asymptotic formulae in Theorem 1.1 and Theorem 1.4. We then discuss the consequences of our results for various estimation metrics. Before passing to these topics, we will derive a simple upper bound on the per-vertex mutual information, which will be a useful comparison for our results. 3.1 An elementary upper bound It is instructive to start with an elementary upper bound on I ( X ; G ). Lemma 3.1. Assume p n , q n satisfy the assumptions of Theorem 1.1 (in particular ( i ) λ n λ and ( ii ) n p n (1 - p n ) → ∞ ). Then lim sup n →∞ 1 n I ( X ; G ) λ 4 . (22) 7
1 - mmse ( γ ) Figure 1: Illustration of the fixed point equation Eq. (8). The ‘effective signal-to-noise ratio’ γ * ( λ ) is given by the intersection of the curve γ 7→ G ( γ ) = 1 - mmse ( γ ), and the line γ/λ . Proof. We have 1 n I ( X ; G ) = 1 n H ( G ) - 1 n H ( G | X ) (23) ( a ) = 1 n H ( G ) - 1 n X 1 i<j n H ( G ij | X ) (24) ( b ) = 1 n H ( G ) - 1 n X 1 i<j n H ( G ij | X i · X j ) (25) 1 n X 1 i<j n I ( X i · X j ; G ij ) = n - 1 2 I ( X 1 · X 2 ; G 12 ) , (26) where ( a ) follows since { G ij } i<j are conditionally independent given X and ( b ) because G ij only depends on X through the product X i · X j (notice that there is no comma but product in H ( G ij | X i · X j ). From our model, it is easy to check that I ( X 1 · X 2 ; G 12 ) = 1 2 p n log p n p n + 1 2 q n log q n p n + 1 2 (1 - p n ) log 1 - p n 1 - p n + 1 2 (1 - q n ) log 1 - q n 1 - p n . (27) The claim follows by substituting p n = p n + p p n (1 - p n ) λ n /n , q n = p n - p p n (1 - p n ) λ n /n and by Taylor expansion 3 . 3.2 Evaluation of the asymptotic formula Our asymptotic expression for the mutual information, cf. Theorem 1.1, and for the estimation error, cf. Theorem 1.4, depends on the solution of Eq. (8) which we copy here for the reader’s 3 Indeed Taylor expansion yields the stronger result n - 1 I ( X ; G ) ( λ n / 4) + n - 1 for all n large enough. 8
Figure 2: Left frame: Asymptotic mutual information per vertex of the two-groups stochastic block model, as a function of the signal-to-noise ratio λ . The dashed lines are simple upper bounds: lim n →∞ I ( X ; G ) /n λ/ 4 (cf. Lemma 3.1) and I ( X ; G ) /n log 2. Right frame: Asymptotic estimation error under different metrics (see Section 3.3). Note the phase transition at λ = 1 in both frames. convenience: γ = λ ( 1 - mmse ( γ ) ) λ G ( γ ) . (28) Here we defined G ( γ ) = 1 - mmse ( γ ) = E tanh( γ + γ Z ) 2 . (29) The effective signal-to-noise ratio γ * ( λ ) that enters Theorem 1.1 and Theorem 1.4 is the largest non-negative solution of Eq. (8). This equation is illustrated in Figure 1. It is immediate to show from the definition (29) that G ( · ) is continuous on [0 , ) with G (0) = 0, and lim γ →∞ G ( γ ) = 1. This in particular implies that γ = 0 is always a solution of Eq. (8). Further, since mmse ( γ ) is monotone decreasing in the signal-to-noise ratio γ , G ( γ ) is monotone increasing. As shown in the proof of Remark 6.1 (see Appendix B.2), G ( · ) is also strictly concave on [0 , ). This implies that Eq. (8) as at most one solution in (0 , ), and a strictly positive solution only exists if λG 0 (0) = λ > 1.

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture