This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Neural Networks CAP 6615, Spring 2011 Homework 6 Solutions 1. (Problem 8.15 Haykin) Let u k ij denote the centered counterpart of the ijth element k ij of the Gram K . Derive the following formula (Schölkopf, 1977): u k ij = k ij m 1 N ∑ m = 1 N Φ T ( x m )Φ( x j )m 1 N ∑ n = 1 N Φ T ( x i )Φ( x n )+ 1 N 2 ∑ m = 1 N ∑ n = 1 N Φ T ( x m )Φ( x n ) . Suggest a compact representation of this relation in matrix form. I'll answer the second question first. Consider this compact representation of the relation: u k ij = ( Φ T ( x i )m u Φ T ) ( Φ( x j )m u Φ ) = ( Φ T ( x i )m 1 N ∑ m = 1 N Φ T ( x m ) )( Φ( x j )m 1 N ∑ n = 1 N Φ( x n ) ) =Φ T ( x i )Φ( x j )m 1 N ∑ m = 1 N Φ T ( x m )Φ( x j )m 1 N Φ T ( x i ) ∑ n = 1 N Φ( x n )m 1 N 2 ∑ m = 1 N ∑ n = 1 N Φ T ( x m )Φ( x n ) = k ij m 1 N ∑ m = 1 N Φ T ( x m )Φ( x j )m 1 N ∑ n = 1 N Φ T ( x i )Φ( x n )+ 1 N 2 ∑ m = 1 N ∑ n = 1 N Φ T ( x m )Φ( x n ) . 2. (Problem 8.16 Haykin) Show that the normalization of eigenvector α of the Gram K is equivalent to the requirement that Eq. (8.109) be satisfied. This problem is a relatively straightforward algebraic task. Equation (8.100) is the key. From this we get 1 = ̃ q T q = ( ∑ i = 1 N α i ϕ( x i ) ) T ( ∑ j = 1 N α i ϕ( x j ) ) = ∑ i = 1 N ∑ j = 1 N ϕ T ( x i )α i α j ϕ( x j )= ∑ i = 1 N ∑ j = 1 N α i α j ϕ T ( x i )ϕ( x j ) = ∑ i = 1 N ∑ j = 1 N α i α j k ( x i , x j )= ∑ i = 1 N ∑ j = 1 N α i α j N ̃ λ[ 8.101 ]=α T α λ . So, since α r T α r λ r = 1 , α r T α r = 1 λ r . 3. (Problem 9.8 Haykin) Using the transformation formula of Eq. (9.40) applied to Eq. (9.37), derive the probability density function of Eq. (9.41). [Note: I believe Haykin's Eq. 9.41 is missing a 1 σ term. Please verify or disprove this.] The X 2 probability density function is defined by...
View
Full Document
 Spring '08
 Staff
 Probability theory, probability density function, Cumulative distribution function, Haykin, Neural Networks CAP

Click to edit the document details