Within two weeks Jerry has collected enough data to begin analysis but he finds

Within two weeks jerry has collected enough data to

This preview shows page 7 - 12 out of 15 pages.

Within two weeks, Jerry has collected enough data to begin analysis, but he finds that his data needs to be denormalized . He also notes that some observations in the set are missing values or they appear to contain invalid values . Jerry realizes that some additional work on the data needs to take place before analysis begins. 7
Image of page 7

Subscribe to view the full document.

Data Mining Data Preprocessing 1 How to Handle Incomplete (Missing) Data? 8 Ignore the tuple : usually done when class label is missing (when doing classification)—not effective when the % of missing values per attribute varies considerably Fill in the missing value manually : tedious + infeasible ? Fill in it automatically with a global constant : e.g., “unknown”, a new class?! the attribute mean the attribute mean for all samples belonging to the same class : smarter the most probable value : inference-based such as Bayesian formula or decision tree
Image of page 8
Data Mining Data Preprocessing 1 Noisy Data 9 Noise: random error or variance in a measured variable Incorrect attribute values may be due to Faulty data collection instruments Data entry problems Data transmission problems Technology limitation Inconsistency in naming convention Other data problems Duplicate records Incomplete data Inconsistent data
Image of page 9

Subscribe to view the full document.

Data Mining Data Preprocessing 1 How to Handle Noisy Data? 10 Binning First sort data and partition into (equal-frequency) bins Then one can smooth by bin means , smooth by bin median, smooth by bin boundaries, etc. Binning Methods for Data Smoothing
Image of page 10
Data Mining Data Preprocessing 1 How to Handle Noisy Data? (cont.) 11 Regression Smooth by fitting the data into regression functions Clustering Detect and remove outliers
Image of page 11

Subscribe to view the full document.

Image of page 12
  • Winter '18
  • nour

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Ask Expert Tutors You can ask You can ask ( soon) You can ask (will expire )
Answers in as fast as 15 minutes