Concepts Key Practical example of GMMs applied to MIR Other Applications Conclusion
Gaussian Mixture Model Classiers Applications to MIR
Bertrand SCHERRER
February 7, 2007
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Outline
1
Key Concepts GMM : an Unsupervised Classier" Why Gaussian Mixture" ? Practical example of GMMs applied to MIR Context GMM Training Classication test Other Applications Musical Instrument Identication in Polyphonic Music Extraction of melodic lines from audio recordings Conclusion
Bertrand SCHERRER GMM Classiers in MIR
2
3
4
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
GMM : an Unsupervised Classier" Why Gaussian Mixture" ?
Outline
1
Key Concepts GMM : an Unsupervised Classier" Why Gaussian Mixture" ? Practical example of GMMs applied to MIR Context GMM Training Classication test Other Applications Musical Instrument Identication in Polyphonic Music Extraction of melodic lines from audio recordings Conclusion
Bertrand SCHERRER GMM Classiers in MIR
2
3
4
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
GMM : an Unsupervised Classier" Why Gaussian Mixture" ?
GMM : an Unsupervised Classier"
Unsupervised Classier The training samples of the classier are not labelled to show their category membership [Duda 73]. Advantages Less time consuming when applied to a large set of data. Ability to track (slow) time-evolving patterns.
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
GMM : an Unsupervised Classier" Why Gaussian Mixture" ?
GMM : an Unsupervised Classier"
Unsupervised Classier The training samples of the classier are not labelled to show their category membership [Duda 73]. Advantages Less time consuming when applied to a large set of data. Ability to track (slow) time-evolving patterns.
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
GMM : an Unsupervised Classier" Why Gaussian Mixture" ?
Why Gaussian Mixture" ?
In GMM classier, for a given class, the probability density function of the observation vector is modelled as :
K
p(x|Ci ) =
k =1
P(k |Ci ).Gk (k , k )
(1)
where : x is a d-component feature vector. k s are the d-component mean vectors of Gaussian Gk . k s are the d-by-d covariance matrices of Gaussian Gk . P(k |Ci ) is the a priori probability of Gaussian Gk for instrument class Ci .
Bertrand SCHERRER GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
GMM : an Unsupervised Classier" Why Gaussian Mixture" ?
Why Gaussian Mixture" ?
In GMM classier, for a given class, the probability density function of the observation vector is modelled as :
K
p(x|Ci ) =
k =1
P(k |Ci ).Gk (k , k )
(1)
where : x is a d-component feature vector. k s are the d-component mean vectors of Gaussian Gk . k s are the d-by-d covariance matrices of Gaussian Gk . P(k |Ci ) is the a priori probability of Gaussian Gk for instrument class Ci .
Bertrand SCHERRER GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
GMM : an Unsupervised Classier" Why Gaussian Mixture" ?
Why Gaussian Mixture" ?
In GMM classier, for a given class, the probability density function of the observation vector is modelled as :
K
p(x|Ci ) =
k =1
P(k |Ci ).Gk (k , k )
(1)
where : x is a d-component feature vector. k s are the d-component mean vectors of Gaussian Gk . k s are the d-by-d covariance matrices of Gaussian Gk . P(k |Ci ) is the a priori probability of Gaussian Gk for instrument class Ci .
Bertrand SCHERRER GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
GMM : an Unsupervised Classier" Why Gaussian Mixture" ?
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Context GMM Training Classication test
Outline
1
Key Concepts GMM : an Unsupervised Classier" Why Gaussian Mixture" ? Practical example of GMMs applied to MIR Context GMM Training Classication test Other Applications Musical Instrument Identication in Polyphonic Music Extraction of melodic lines from audio recordings Conclusion
Bertrand SCHERRER GMM Classiers in MIR
2
3
4
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Context GMM Training Classication test
GMMs are used in many different elds. Look at one clear example of GMM classication applied to MIR: [Marques 99]
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Context GMM Training Classication test
GMMs are used in many different elds. Look at one clear example of GMM classication applied to MIR: [Marques 99]
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Context GMM Training Classication test
Context
Objectives Instrument identication in monophonic music: 8 different classes": bagpipe, clarinet, ute, harpsichord, organ, piano, trombone and violin . on very short recordings (0.2s). What is to be classied ? A set X of m unlabelled observations (cepstral, mel-cepstral and LPC coefcients) : X = [x1 , x2 . . . xm ].
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Context GMM Training Classication test
Context
Objectives Instrument identication in monophonic music: 8 different classes": bagpipe, clarinet, ute, harpsichord, organ, piano, trombone and violin . on very short recordings (0.2s). What is to be classied ? A set X of m unlabelled observations (cepstral, mel-cepstral and LPC coefcients) : X = [x1 , x2 . . . xm ].
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Context GMM Training Classication test
Context
Assuming that the observations are i.i.d., the likelihood that the entire set of observations X has been produced by a violin ( C0 for example) is :
m
p (X|C0 ) =
t=1
p(xt |C0 )
(2)
and each p(xt |C0 ) is modelled as a mixture of K multivariate gaussians.
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Context GMM Training Classication test
Objectives of the GMM training
At this stage, one tries to estimate, for all the classes of instruments, the parameters of the GMM: ik = [P(k |Ci ), k ,i , k ,i ] for k = 1 . . . K .
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Context GMM Training Classication test
How to estimate the GMM parameters : MLE ?
Ideal way would be to use the Maximum Likelihood Estimation (a.k.a. MLE). MLE theoretically consists in nding = [ i1 , i2 . . . iK ], maximizing p(X|Ci ). In the case where all the parameters are unknown, MLE becomes very complex ... and unreliable need for an alternate method.
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Context GMM Training Classication test
How to estimate the GMM parameters : MLE ?
Ideal way would be to use the Maximum Likelihood Estimation (a.k.a. MLE). MLE theoretically consists in nding = [ i1 , i2 . . . iK ], maximizing p(X|Ci ). In the case where all the parameters are unknown, MLE becomes very complex ... and unreliable need for an alternate method.
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs to applied MIR Other Applications Conclusion
Context GMM Training Classication test
How to estimate the GMM parameters : MLE ?
Ideal way would be to use the Maximum Likelihood Estimation (a.k.a. MLE). MLE theoretically consists in nding = [ i1 , i2 . . . iK ], maximizing p(X|Ci ). In the case where all the parameters are unknown, MLE becomes very complex ... and unreliable need for an alternate method.
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Context GMM Training Classication test
How to estimate the GMM parameters : MLE ?
Ideal way would be to use the Maximum Likelihood Estimation (a.k.a. MLE). MLE theoretically consists in nding = [ i1 , i2 . . . iK ], maximizing p(X|Ci ). In the case where all the parameters are unknown, MLE becomes very complex ... and unreliable need for an alternate method.
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Context GMM Training Classication test
How to estimate the GMM parameters : EM
Expectation Maximisation algorithm [Dempster 77] is an iterative solution very often used for MLE. Initial Steps
1
Compute closed-form expressions of the parameters of the GMM corresponding to a local extremum of the likelihood p(X|(t)). Make a rst guess on the values of (t).
2
Lets iterate !
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Context GMM Training Classication test
How to estimate the GMM parameters : EM
Expectation Maximisation algorithm [Dempster 77] is an iterative solution very often used for MLE. Initial Steps
1
Compute closed-form expressions of the parameters of the GMM corresponding to a local extremum of the likelihood p(X|(t)). Make a rst guess on the values of (t).
2
Lets iterate !
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Context GMM Training Classication test
How to estimate the GMM parameters : EM
Expectation Maximisation algorithm [Dempster 77] is an iterative solution very often used for MLE. Initial Steps
1
Compute closed-form expressions of the parameters of the GMM corresponding to a local extremum of the likelihood p(X|(t)). Make a rst guess on the values of (t).
2
Lets iterate !
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Context GMM Training Classication test
Classication Test
A feature vector xt is said to belong to an instrument class i if it maximizes p(Ci |xt ) = p(xt |Ci ).p(Ci ). If classes can occur with the same probability, since we know for all classes, xt belongs to the class for which p(xt |Ci ) is maximum.
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Context GMM Training Classication test
In [Marques 99]: Features : mel cepstral feature vectors (16-element vectors). Order of the GMM: 2. overall error rate of 37%.
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Musical Instrument Identication in Polyphonic Music Extraction of melodic lines from audio recordings
Outline
1
Key Concepts GMM : an Unsupervised Classier" Why Gaussian Mixture" ? Practical example of GMMs applied to MIR Context GMM Training Classication test Other Applications Musical Instrument Identication in Polyphonic Music Extraction of melodic lines from audio recordings Conclusion
Bertrand SCHERRER GMM Classiers in MIR
2
3
4
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Musical Instrument Identication in Polyphonic Music Extraction of melodic lines from audio recordings
Musical Instrument Identication in Polyphonic Music
In [Eggink 03], try to recognize 2 instruments playing at the same time. Approach based on estimation of multiple fundamental frequencies to sort features fed into the GMM.
Bertrand SCHERRER
GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Musical Instrument Identication in Polyphonic Music Extraction of melodic lines from audio recordings
Figure 1 [Eggink 03]
Bertrand SCHERRER GMM Classiers in MIR
Key Concepts Practical example of GMMs applied to MIR Other Applications Conclusion
Musical Instrument Identication in Polyphonic Music Extraction of melodic lines from audio recordings
Features : cepstral coefcients clearly belonging to different tones. Order of the GMM: 120. Training Material:
monophonic recordings of tones or melodies ...