intro - Data Mining Stat557/IST557 Data Mining...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Data Mining Stat557/IST557 Data Mining Stat557/IST557 Jia Li Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu http://www.stat.psu.edu/jiali Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 General Information Course homepage: http://www.stat.psu.edu/~jiali/stat557 Prerequisite: Elementary probability theory Conditional distribution, expectation C, Matlab, or S-plus programming Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Text books: Required: The Elements of Statistical Learning, by T. Hastie, R. Tibshirani, and J. Friedman (ElemStatLearn). Optional: 1. Classification and Regression Trees by L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone 2. Pattern Recognition and Neural Networks by B. Ripley 3. Principles of Data Mining by H. Mannila, P. Smyth and D. J. Hand 4. Data Mining: Concepts and Techniques by J. Han and M. Kamber Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Research fields Statistics Machine learning Pattern recognition Signal processing Database Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Applications Business Wal-Mart data warehouse Credit card companies Human genome project: DNA sequences Microarray data Terrabytes of data on the internet Multimedia information Speech recognition Image analysis Genomics Information retrieval Communication systems Many other scientific fields http://www.stat.psu.edu/jiali Jia Li Data Mining Stat557/IST557 Problems Focused: Prediction Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Terminology Notation Input X : X is often multidimensional. Each dimension of X is denoted by Xj and is referred to as a feature, predictor, or independent variable/variable. Output Y : response, dependent variable. Categorization Supervised learning vs. unsupervised learning Is Y available in the training data? Is Y quantitative or qualitative? For qualitative Y , it is also denoted by G G = {1, 2, ..., K }. Regression vs. Classification Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Examples Email spam: (ElemStatLearn) Goal: predict whether an email is a junk email, i.e., "spam". Raw data: text email messages. Input X : relative frequencies of 57 of the most commonly occurring words and punctuation marks in the email message. Training data set: 4601 email messages with email type known (supervised learning). Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Examples Handwritten digit recognition:(ElemStatLearn) Goal: identify single digits 0 9 based on images. Raw data: images that are scaled segments from five digit ZIP codes. 16 16 eight-bit grayscale maps Pixel intensities range from 0 (black) to 255 (white). Input data: a 256 dimension vector, or feature vectors with lower dimensions. Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Examples Image segmentation: Goal: segment images into regions of different types, e.g., man-made vs. natural in aerial images, graph and picture vs. text in document images. Raw data: grayscale images represented by matrices of size m n, or color images represented by 3 such matrices. Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Aerial images. Left: Original image of size 512 512 with pixel intensity ranging from 0 to 255, Right: Hand-labeled classified images. White: man-made, Gray: natural. Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Input data: Divide images into blocks of pixels or form a neighborhood around each pixel. Compute statistics using pixel intensities in each block. An image is converted to an array of input vectors. Assume the feature vectors are independent. Employ spatial models to capture dependence among the vectors. Methodologies: Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Examples Speech recognition: Goal: identify words spoken according to speech signals Automatic voice recognition systems used by airline companies Automatic stock price reporting Raw data: voice amplitude sampled at discrete time spots (a time sequence). Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Input data: speech feature vectors computed at the sampling time. Methodology: Estimate an Hidden Markov Model (HMM) for each word, e.g., State College, San Francisco, Pittsburgh. For a new word, find the HMM that yields the maximum likelihood. Identify the word as the one associated with the HMM. Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Examples DNA Expression Microarray: Goal: identify disease or tissue types Raw data: for each sample taken from a tissue of a particular disease type, the expression levels of a large collection of genes are measured. Input data: cleaned-up gene expression data Normalization Denoising. Ample literature on the topic of cleaning microarray data Example data set: 4026 genes, 96 samples taken from 9 classes of tissues. Challenges: very high dimensional data very limited number of samples Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Examples DNA sequence classification: Goal: distinguish "junk" segments from coding segments. Raw data: sequences of letters, e.g., A,C,G,T for DNA sequences. Input data: likelihood ratio statistics computed from stochastic models. Supervised learning: estimate stochastic models, select models. Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Supervised Learning Two types of learning: Regression: the response Y is quantitative. Classification: the response Y is qualitative, or categorical. Fit the data well. Robust Training error vs. testing error Bias vs. variance Fitting vs. overfitting Empirical risk vs. model complexity (capacity) http://www.stat.psu.edu/jiali Two aspects in learning: Equivalent concepts: Jia Li Data Mining Stat557/IST557 Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Learning Spectrum Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Regression Overview: Linear models: The mean response is a linear function of the independent variables. Generalized linear models Expand basis: Splines (polynomials) Reproducing Kernel Hilbert Spaces Wavelet smoothing Kernel methods Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Classification: A graphic View Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Outlines Linear regression Linear methods for classification Prototype methods Classification and regression tree (CART) Mixture discriminant analysis Hidden Markov models and its applications Jia Li http://www.stat.psu.edu/jiali ...
View Full Document

This note was uploaded on 02/04/2012 for the course STAT 557 taught by Professor Jiali during the Fall '09 term at Pennsylvania State University, University Park.

Ask a homework question - tutors are online