Introduction Why dimensionality reduction is important: - To store data particular space is used, by using we can reduce the space required. Also less dimensions lead to less computation/training time. Some algorithms do not perform well when we have a large dimension. It helps in visualizing data. Complexity depends on no. of inputs (d) & size of data sample (N) Decreasing input decrease the complexity of algorithm during testing. Simpler models are more robust on small datasets i.e. less variance More interpretable is simpler explanation Example To classify whether the email is spam or not Dimensionality reduction can be done in two different ways: 1. By only keeping the most relevant variables from the original dataset 2. By finding a smaller set of new variables Two component of dimensionality reduction Feature selection: To find a subset of original set of variables of features It usually involves three ways: 1. Filter 2. Wrapper 3. Embedded we are interested to finding k of the d dimensions that give us most information, ignoring the remaining d – k
2. Feature extraction: This reduces the data in a high dimensional space to a lower dimension space. finding new set of k dimensions that are combinations of the original d dimension. These methods maybe supervised or unsupervised. Extraction method are LDA and PCA Method 1. Principal Component Analysis (PCA) 2. Linear Discriminant Analysis (LDA) 3. Generalized Discriminant Analysis (GDA) Important term in Dimensionality reduction A covariance refers to the measure of how two random variables will change together and is used to calculate the correlation between variables. The variance refers to the spread of the data set — how far apart the numbers are in relation to the mean, for instance. (Also explain overfitting underfitting) Factor Analysis Factor analysis is a linear statistical model. It is used to explain the variance among the observed variable and condense a set of the observed variable into the unobserved variable called factors. Observed variables are modeled as a linear combination of factors and error terms. Factor or latent variable is associated with multiple observed variables, who have common patterns of responses. Each factor explains a particular amount of variance in the observed variables. It helps in data interpretations by reducing the number of variables. Factor analysis is a method for investigating whether a number of variables of interest X1, X2,……., Xl, are linearly related to a smaller number of unobservable factors F1, F2,..……, Fk.