This preview shows page 1. Sign up to view the full content.
Unformatted text preview: Data Mining Stat557/IST557 Data Mining Stat557/IST557
Jia Li
Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu http://www.stat.psu.edu/jiali Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 General Information Course homepage: http://www.stat.psu.edu/~jiali/stat557 Prerequisite: Elementary probability theory Conditional distribution, expectation C, Matlab, or Splus programming Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Text books: Required: The Elements of Statistical Learning, by T. Hastie, R. Tibshirani, and J. Friedman (ElemStatLearn). Optional:
1. Classification and Regression Trees by L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone 2. Pattern Recognition and Neural Networks by B. Ripley 3. Principles of Data Mining by H. Mannila, P. Smyth and D. J. Hand 4. Data Mining: Concepts and Techniques by J. Han and M. Kamber Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Research fields Statistics Machine learning Pattern recognition Signal processing Database Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Applications Business WalMart data warehouse Credit card companies Human genome project: DNA sequences Microarray data Terrabytes of data on the internet Multimedia information Speech recognition Image analysis Genomics Information retrieval Communication systems Many other scientific fields
http://www.stat.psu.edu/jiali Jia Li Data Mining Stat557/IST557 Problems Focused: Prediction Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Terminology
Notation Input X : X is often multidimensional. Each dimension of X is denoted by Xj and is referred to as a feature, predictor, or independent variable/variable. Output Y : response, dependent variable. Categorization Supervised learning vs. unsupervised learning Is Y available in the training data? Is Y quantitative or qualitative? For qualitative Y , it is also denoted by G G = {1, 2, ..., K }. Regression vs. Classification Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Examples Email spam: (ElemStatLearn) Goal: predict whether an email is a junk email, i.e., "spam". Raw data: text email messages. Input X : relative frequencies of 57 of the most commonly occurring words and punctuation marks in the email message. Training data set: 4601 email messages with email type known (supervised learning). Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Examples Handwritten digit recognition:(ElemStatLearn) Goal: identify single digits 0 9 based on images. Raw data: images that are scaled segments from five digit ZIP codes. 16 16 eightbit grayscale maps Pixel intensities range from 0 (black) to 255 (white). Input data: a 256 dimension vector, or feature vectors with lower dimensions. Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Examples Image segmentation: Goal: segment images into regions of different types, e.g., manmade vs. natural in aerial images, graph and picture vs. text in document images. Raw data: grayscale images represented by matrices of size m n, or color images represented by 3 such matrices. Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Aerial images. Left: Original image of size 512 512 with pixel intensity ranging from 0 to 255, Right: Handlabeled classified images. White: manmade, Gray: natural.
Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Input data: Divide images into blocks of pixels or form a neighborhood around each pixel. Compute statistics using pixel intensities in each block. An image is converted to an array of input vectors. Assume the feature vectors are independent. Employ spatial models to capture dependence among the vectors. Methodologies: Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Examples Speech recognition: Goal: identify words spoken according to speech signals Automatic voice recognition systems used by airline companies Automatic stock price reporting Raw data: voice amplitude sampled at discrete time spots (a time sequence). Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Input data: speech feature vectors computed at the sampling time. Methodology: Estimate an Hidden Markov Model (HMM) for each word, e.g., State College, San Francisco, Pittsburgh. For a new word, find the HMM that yields the maximum likelihood. Identify the word as the one associated with the HMM. Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Examples
DNA Expression Microarray: Goal: identify disease or tissue types Raw data: for each sample taken from a tissue of a particular disease type, the expression levels of a large collection of genes are measured. Input data: cleanedup gene expression data Normalization Denoising. Ample literature on the topic of cleaning microarray data Example data set: 4026 genes, 96 samples taken from 9 classes of tissues. Challenges: very high dimensional data very limited number of samples Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Examples DNA sequence classification: Goal: distinguish "junk" segments from coding segments. Raw data: sequences of letters, e.g., A,C,G,T for DNA sequences. Input data: likelihood ratio statistics computed from stochastic models. Supervised learning: estimate stochastic models, select models. Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Supervised Learning
Two types of learning: Regression: the response Y is quantitative. Classification: the response Y is qualitative, or categorical. Fit the data well. Robust Training error vs. testing error Bias vs. variance Fitting vs. overfitting Empirical risk vs. model complexity (capacity)
http://www.stat.psu.edu/jiali Two aspects in learning: Equivalent concepts: Jia Li Data Mining Stat557/IST557 Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Learning Spectrum Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Regression
Overview: Linear models: The mean response is a linear function of the independent variables. Generalized linear models Expand basis: Splines (polynomials) Reproducing Kernel Hilbert Spaces Wavelet smoothing Kernel methods Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Classification: A graphic View Jia Li http://www.stat.psu.edu/jiali Data Mining Stat557/IST557 Outlines Linear regression Linear methods for classification Prototype methods Classification and regression tree (CART) Mixture discriminant analysis Hidden Markov models and its applications Jia Li http://www.stat.psu.edu/jiali ...
View
Full
Document
This note was uploaded on 02/04/2012 for the course STAT 557 taught by Professor Jiali during the Fall '09 term at Pennsylvania State University, University Park.
 Fall '09
 JIALI
 Statistics

Click to edit the document details