unit 3 combined.docx - 3.1 Introduction Why dimensionality reduction is important To store data particular space is used by using we can reduce the

# unit 3 combined.docx - 3.1 Introduction Why dimensionality...

This preview shows page 1 - 4 out of 12 pages.

3.1 Introduction Why dimensionality reduction is important: - To store data particular space is used, by using we can reduce the space required. Also less dimensions lead to less computation/training time. Some algorithms do not perform well when we have a large dimension. It helps in visualizing data. Complexity depends on no. of inputs (d) & size of data sample (N) Decreasing input decrease the complexity of algorithm during testing. Simpler models are more robust on small datasets i.e. less variance More interpretable is simpler explanation Example To classify whether the email is spam or not Dimensionality reduction can be done in two different ways: 1. By only keeping the most relevant variables from the original dataset 2. By finding a smaller set of new variables Two component of dimensionality reduction Feature selection: To find a subset of original set of variables of features It usually involves three ways: 1. Filter 2. Wrapper 3. Embedded we are interested to finding k of the d dimensions that give us most information, ignoring the remaining d k
2. Feature extraction: This reduces the data in a high dimensional space to a lower dimension space. finding new set of k dimensions that are combinations of the original d dimension. These methods maybe supervised or unsupervised. Extraction method are LDA and PCA Method 1. Principal Component Analysis (PCA) 2. Linear Discriminant Analysis (LDA) 3. Generalized Discriminant Analysis (GDA) Important term in Dimensionality reduction A covariance refers to the measure of how two random variables will change together and is used to calculate the correlation between variables. The variance refers to the spread of the data set — how far apart the numbers are in relation to the mean, for instance. (Also explain overfitting underfitting) 3.2 Factor Analysis Factor analysis is a linear statistical model. It is used to explain the variance among the observed variable and condense a set of the observed variable into the unobserved variable called factors. Observed variables are modeled as a linear combination of factors and error terms. Factor or latent variable is associated with multiple observed variables, who have common patterns of responses. Each factor explains a particular amount of variance in the observed variables. It helps in data interpretations by reducing the number of variables. Factor analysis is a method for investigating whether a number of variables of interest X1, X2, ……., Xl, are linearly related to a smaller number of unobservable factors F1, F2,..……, Fk.
We make some Assumptions: 1. There are no outliers in data. 2. Sample size should be greater than the factor. 3. There should not be perfect multicollinearity. 4. There should not be homoscedasticity between the variables.

#### You've reached the end of your free preview.

Want to read all 12 pages?

• Winter '17
• paola
• Multivariate statistics

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern