E. S. Gopi (auth.)-Mathematical Summary for Digital Signal Processing Applications with Matlab-Sprin

E. S. Gopi (auth.)-Mathematical Summary for Digital Signal Processing Applications with Matlab-Sprin

This preview shows page 1 out of 229 pages.

Unformatted text preview: Mathematical Summary for Digital Signal Processing Applications with Matlab E.S. Gopi Mathematical Summary for Digital Signal Processing Applications with Matlab 123 E.S. Gopi National Institute of Technology, Trichy Dept. Electronics & Communication Engineering 67 Tanjore Main Road Tiruchirappalli-620015 National Highway India [email protected] ISBN 978-90-481-3746-6 e-ISBN 978-90-481-3747-3 DOI 10.1007/978-90-481-3747-3 Springer Dordrecht Heidelberg London New York Library of Congress Control Number: 2009944069 c Springer Science+Business Media B.V. 2010  No part of this work may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording or otherwise, without written permission from the Publisher, with the exception of any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Cover design: eStudio Calamar S.L. Printed on acid-free paper Springer is part of Springer Science+Business Media ( ) Dedicated to my son G.V. Vasig and my wife G. Viji Preface The book titled “Mathematical summary for Digital Signal Processing Applications with Matlab” consists of Mathematics which is not usually dealt in the DSP core subject, but used in the DSP applications. Matlab Illustrations for the selective topics such as generation of Multivariate Gaussian distributed sample outcomes, Optimization using Bacterial Foraging etc. are given exclusively as the separate chapter for better understanding. The book is written in such a way that it is suitable for Nonmathematical readers and is very much suitable for the beginners who are doing research in Digital Signal Processing. E.S. Gopi vii Acknowledgements I am extremely happy to express my thanks to Prof. K.M.M. Prabhu, Indian Institute of Technology Madras for his constant encouragement. I also thank the Director Prof. M. Chidambaram, Prof. B. Venkataramani and Prof. S. Raghavan National Institute of Technology Trichy for their support. I thank those who directly or indirectly involved in bringing up this Book. Special thanks to my parents Mr. E. Sankarasubbu and Mrs. E.S. Meena. ix Contents 1 Matrices .. . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.1 Properties of Vectors.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.2 Properties of Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.3 LDU Decomposition of the Matrix . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.4 PLDU Decomposition of an Arbitrary Matrix . . . . . . . . . . . . . .. . . . . . . . . . . 1.5 Vector Space and Its Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.6 Linear Independence, Span, Basis and the Dimension of the Vector Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.6.1 Linear Independence .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.6.2 Span . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.6.3 Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.6.4 Dimension .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.7 Four Fundamental Vector Spaces of the Matrix . . . . . . . . . . . . .. . . . . . . . . . . 1.7.1 Column Space .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.7.2 Null Space .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.7.3 Row Space.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.7.4 Left Null Space.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.8 Basis of the Four Fundamental Vector Spaces of the Matrix . . . . . . . . . . 1.8.1 Column Space .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.9 Observations on Results of the Example 1.12 .. . . . . . . . . . . . . .. . . . . . . . . . . 1.9.1 Column Space .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.9.2 Null Space .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.9.3 Left Column Space (Row Space) . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.9.4 Left Null Space.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.9.5 Observation.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.10 Vector Representation with Different Basis . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.11 Linear Transformation of the Vector .. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.11.1 Trick to Compute the Transformation Matrix . . . . .. . . . . . . . . . . 1.12 Transformation Matrix with Different Basis . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.13 Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.13.1 Basic Definitions and Results . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.13.2 Orthogonal Complement . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.14 System of Linear Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1 2 3 7 10 12 12 12 13 13 13 13 14 14 14 14 14 15 20 21 21 21 21 22 22 24 25 25 26 26 27 27 xi xii Contents 1.15 Solutions for the System of Linear Equation [A] x D b . . . .. . . . . . . . . . . 1.15.1 Trick to Obtain the Solution .. . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.16 Gram Schmidt Orthonormalization Procedure for Obtaining Orthonormal Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.17 QR Factorization .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.18 Eigen Values and Eigen Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.19 Geometric Multiplicity (Versus) Algebraic Multiplicity .. . .. . . . . . . . . . . 1.20 Diagonalization of the Matrix .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.21 Schur’s Lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.22 Hermitian Matrices and Skew Hermitian Matrices . . . . . . . . .. . . . . . . . . . . 1.23 Unitary Matrices .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.24 Normal Matrices .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.25 Applications of Diagonalization of the Non-deficient Matrix . . . . . . . . . 1.26 Singular Value Decomposition .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 1.27 Applications of Singular Value Decomposition .. . . . . . . . . . . .. . . . . . . . . . . 2 28 29 36 40 42 44 47 49 50 52 56 58 60 62 Probability . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 67 2.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 67 2.2 Axioms of Probability .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 68 2.3 Class of Events or Field (F) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 68 2.4 Probability Space (S, F, P) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 68 2.5 Probability Measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 68 2.6 Conditional Probability .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 69 2.7 Total Probability Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 70 2.8 Bayes Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 70 2.9 Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 70 2.10 Multiple Experiments (Combined Experiments) .. . . . . . . . . . .. . . . . . . . . . . 71 2.11 Random Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 74 2.12 Cumulative Distribution Function (cdf) of the Random Variable ‘x’. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 75 2.13 Continuous Random Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 76 2.14 Discrete Random Variable.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 76 2.15 Probability Mass Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 76 2.16 Probability Density Function.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 76 2.17 Two Random Variables .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 77 2.18 Conditional Distributions and Densities . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 79 2.19 Independent Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 79 2.20 Some Important Results on Conditional Density Function .. . . . . . . . . . . 80 2.21 Transformation of Random Variables of the Type Y D g.X/ . . . . . . . . . 84 2.22 Transformation of Random Variables of the Type Y1 D g1.X1; X2/; Y2 D g2.X1; X2) . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 85 2.23 Expectations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 99 2.24 Indicator .. . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 99 2.25 Moment Generating Function .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .101 2.26 Characteristic Function .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .102 Contents xiii 2.27 Multiple Random Variable (Random Vectors) . . . . . . . . . . . . . .. . . . . . . . . . .102 2.28 Gaussian Random Vector with Mean Vector X and Covariance Matrix CX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .107 2.29 Complex Random Variables.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .118 2.30 Sequence of the Number and Its Convergence . . . . . . . . . . . . . .. . . . . . . . . . .119 2.31 Sequence of Functions and Its Convergence . . . . . . . . . . . . . . . .. . . . . . . . . . .120 2.32 Sequence of Random Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .120 2.33 Example for the Sequence of Random Variable.. . . . . . . . . . . .. . . . . . . . . . .122 2.34 Central Limit Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .122 3 Random Process .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .123 3.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .123 3.2 Random Variable Xt1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .124 3.3 Strictly Stationary Random Process with Order 1 . . . . . . . . . .. . . . . . . . . . .124 3.4 Strictly Stationary Random Process with Order 2 . . . . . . . . . .. . . . . . . . . . .124 3.5 Wide Sense Stationary Random Process . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .125 3.6 Complex Random Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .127 3.7 Properties of Real and Complex Random Process . . . . . . . . . .. . . . . . . . . . .127 3.8 Joint Strictly Stationary of Two Random Process . . . . . . . . . . .. . . . . . . . . . .127 3.9 Jointly Wide Sense Stationary of Two Random Process . . . .. . . . . . . . . . .128 t 3.10 Correlation Matrix of the Random Column Vector X Ys for the Specific ‘t’ ‘s’ .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .128 3.11 Ergodic Process .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .128 3.12 Independent Random Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .132 3.13 Uncorrelated Random Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .132 3.14 Random Process as the Input and Output of the System. . . .. . . . . . . . . . .132 3.15 Power Spectral Density (PSD) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .134 3.16 White Random Process (Noise) .. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .137 3.17 Gaussian Random Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .138 3.18 Cyclo Stationary Random Process . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .139 3.19 Wide Sense Cyclo Stationary Random Process . . . . . . . . . . . . .. . . . . . . . . . .139 3.20 Sampling and Reconstruction of Random Process . . . . . . . . . .. . . . . . . . . . .142 3.21 Band Pass Random Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .144 3.22 Random Process as the Input to the Hilbert Transformation as the System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .146 3.23 Two Jointly W.S.S Low Pass Random Process Obtained Using W.S.S. Band Pass Random Process and Its Hilbert Transformation .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .148 4 Linear Algebra . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .153 4.1 Vector Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .153 4.2 Linear Transformation .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .154 4.3 Direct Sum . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .160 4.4 Transformation Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .162 4.5 Similar Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .164 xiv Contents 4.6 4.7 4.8 4.9 4.10 4.11 4.12 4.13 Structure Theorem .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .166 Properties of Eigen Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .171 Properties of Generalized Eigen Space . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .172 Nilpotent Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .173 Polynomial ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .175 Inner Product Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .176 Orthogonal Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .177 Riegtz Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .179 5 Optimization .. . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .181 5.1 Constrained Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .181 5.2 Extension to Constrained Optimization Technique to Higher Dimensional Space with Multiple Constraints .. .. . . . . . . . . . .186 5.3 Positive Definite Test of the Modified Hessian Matrix Using Eigen Value Computation .. . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .189 5.4 Constrained Optimization with Complex Numbers .. . . . . . . .. . . . . . . . . . .193 5.5 Dual Optimization Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .194 5.6 Kuhn-Tucker Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .195 6 Matlab Illustrations.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .197 6.1 Generation of Multivariate Gaussian Distributed Sample Outcomes with the Required Mean Vector ‘MY ’ and Covariance Matrix ‘CY ’ . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .197 6.2 Bacterial Foraging Optimization Technique .. . . . . . . . . . . . . . . .. . . . . . . . . . .202 6.3 Particle Swarm Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .208 6.4 Newton’s Iterative Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .210 6.5 Steepest Descent Algorithm .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .214 Index . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .217 Chapter 1 Matrices One-dimensional array representation of scalars are called vector. If the elements are arranged in row wise, it is called Row vector. In the same fashion, if the elements of the vector are arranged in column wise, it is called column vector. Two-dimensional array representations of scalars are called matrix. Size of the matrix is represented as R  C, where R is the number of rows and C is the number of columns of the matrix. Scalar elements in the array can be either complex numbers .C/ or the real numbers. .R/. The column vector is represented as X. The Row vector is represented as X T . Example 1.1. Row Vector with the elements filled up with real numbers Œ2:89 21:87 100 Column Vector with the elements filled up with Complex numbers. 2 3 1Cj 6 j 7 6 7 49 C 7j 5 0 Matrix of size 2  3 with the elements filled up with real numbers " 2 3 # 6 4 1 2 Matrix of size 3  2 with the elements filled up with complex numbers 2 j 6 62j 4 0 1Cj 3 7 5j 7 5 j E.S. Gopi, Mathematical Summary for Digital Signal Processing Applications with Matlab, DOI 10.1007/978-90-481-3747-3 1, c Springer Science+B...
View Full Document

{[ snackBarMessage ]}

Get FREE access by uploading your study materials

Upload your study materials now and get free access to over 25 million documents.

Upload now for FREE access Or pay now for instant access
Christopher Reinemann
"Before using Course Hero my grade was at 78%. By the end of the semester my grade was at 90%. I could not have done it without all the class material I found."
— Christopher R., University of Rhode Island '15, Course Hero Intern

Ask a question for free

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern