NO.2
Topics CHAPTER to be covered in this chapter
Data vs information
Data mining and machine learning
Structural descriptions
Rules: classification and association
Decision trees
Datasets
Weather, contact lens, CPU performance, labour negotiation
data, soybean classification
Fielded applications
Loan applications, screening images, market basket analysis
Generalization as search
Data mining and ethics
2
1
Data vs. information
Society produces huge amounts of data
Sources: business, science, medicine, economics,
geography, environment, sports,
Potentially valuable resource
Raw data is useless: need techniques to
automatically extract information from it
Data: recorded facts
Information: patterns underlying the data
3
Data mining
Extracting
previously unknown,
potentially useful
information from data
Needed: programs that detect patterns and
regularities in the data
Strong patterns good predictions
Problem 1: most patterns are not interesting
Problem 2: patterns may be inexact (or spurious)
Problem 3: data may be garbled or missing
4
2
The weather problem
Temp
Humidity
Overcast
Outlook
72
90
Overcast
83
Overcast
64
Overcast
Windy
Class
TRUE
Play
78
FALSE
Play
65
TRUE
Play
81
75
FALSE
Play
Rain
71
80
TRUE
Dont play
Rain
65
70
TRUE
Dont play
Rain
75
80
FALSE
Play
Rain
68
80
FALSE
Play
Rain
70
96
FALSE
Play
Sunny
75
70
TRUE
Play
Sunny
80
90
TRUE
Dont play
Sunny
85
85
FALSE
Dont play
Sunny
72
95
FALSE
Dont play
Sunny
69
70
FALSE
Play
5
Ross Quinlan
Machine learning researcher from 1970s
University of Sydney, Australia
1986 Induction of decision trees ML Journal
1993 C4.5: Programs for machine learning
Published by Morgan Kaufmann
6
3
Classification vs. association rules
Classification rule:
predicts value of a given attribute (the classification of an example)
If outlook = sunny and humidity = high
then play = no
Association rule:
predicts value of arbitrary attribute (or combination)
If temperature = cool then humidity = normal
If humidity = normal and windy = false
then play = yes
If outlook = sunny and play = no
then humidity = high
If windy = false and play = no
then outlook = sunny and humidity = high
7
Weather data with mixed attributes
Some attributes have numeric values
Outlook
Temperature
Humidity
Windy
Play
Sunny
85
85
False
No
Sunny
80
90
True
No
Overcast
83
86
False
Yes
Rainy
75
80
False
Yes
If
If
If
If
If
outlook = sunny and humidity > 83 then play = no
outlook = rainy and windy = true then play = no
outlook = overcast then play = yes
humidity < 85 then play = yes
none of the above then play = yes
8
4
Fielded applications
The result of learning
or the learning method itself
is deployed in practical applications
Processing loan applications
Screening images for oil slicks
Electricity supply forecasting
Diagnosis of machine faults
Marketing and sales
9
Processing loan applications
(American Express)
Given: questionnaire with
financial and personal information
Question: should money be lent?
Simple statistical method covers 90% of cases
Borderline cases referred to loan officers
But: 50% of accepted borderline cases defaulted!
Solution: reject all borderline cases?
No! Borderline cases are most active customers
10
5
Enter machine learning
1000 training examples of borderline cases
20 attributes:
age
years with current employer
years at current address
years with the bank
other credit cards possessed,
Learned rules: correct on 70% of cases
human experts only 50%
Rules could be used to explain decisions to
customers
11
Screening images
Given: radar satellite images of coastal
waters
Problem: detect oil slicks in those
images
Oil slicks appear as dark regions with
changing size and shape
Not easy: look alike dark regions can be
caused by weather conditions (e.g. high
wind)
Expensive process requiring highly
trained personnel
12
6
Enter machine learning
Extract dark regions from normalized image
Attributes:
size of region
shape, area
intensity
sharpness and jaggedness of boundaries
proximity of other regions
info about background
Constraints:
Few training examplesoil slicks are rare!
Unbalanced data: most dark regions arent slicks
Regions from same image form a batch
Requirement: adjustable false-alarm rate
13
Load forecasting
supply Electricity companies
need forecast of future demand
for power
Forecasts of min/max load for each hour
significant savings
Given: manually constructed load model that assumes
normal climatic conditions
Problem: adjust for weather conditions
Static model consist of:
base load for the year
load periodicity over the year
effect of holidays
14
7
Enter machine learning
Prediction corrected using most similar days
Attributes:
temperature
humidity
wind speed
cloud cover readings
plus difference between actual load and predicted load
Average difference among three most similar days
added to static model
Linear regression coefficients form attribute weights in
similarity function
15
Marketing and sales
Companies precisely record massive amounts of
marketing and sales data
Applications:
Special offers:
identifying profitable customers
(e.g. reliable owners of credit cards that need extra money
during the holiday season)
16
8
Marketing and
sales
Market basket analysis
Association techniques find
groups of items that tend to
occur together in a
transaction
(used to analyze checkout data)
Historical analysis of purchasing patterns
Identifying prospective customers
Focusing promotional mailouts
(targeted campaigns are cheaper than mass-marketed
ones)
17
Machine learning and statistics
Historical difference (grossly oversimplified):
Statistics: testing hypotheses
Machine learning: finding the right hypothesis
But: huge overlap
Decision trees (C4.5 and CART)
Today: perspectives have converged
Most ML algorithms employ statistical techniques
18
9
Statisticians
Sir Ronald Aylmer Fisher
Born: 17 Feb 1890 London, England
Died: 29 July 1962 Adelaide, Australia
Numerous distinguished contributions to developing
the theory and application of statistics for making
quantitative a vast field of biology
Leo Breiman
Developed decision trees
1984 Classification and
Regression Trees. Wadsworth.
19
Overfitting-avoidance bias
Can be seen as a form of search bias
Modified evaluation criterion
E.g. balancing simplicity and number of errors
Modified search strategy
E.g. pruning (simplifying a description)
Pre-pruning: stops at a simple description before search proceeds
to an overly complex one
Post-pruning: generates a complex description first and simplifies
it afterwards
20
10
Data mining and ethics
It is widely accepted that before people make a
decision to provide personal information they need
to know how it will be used and what it will be used
for, what steps will be taken to protect its
confidentiality and integrity, what the consequences
of supplying or withholding the information are, and
any rights of redress they may have. Whenever such
information is collected, individuals should be told
these straightforwardly in plain language they can
understand.
21
Data mining and ethics
The potential use of data mining techniques means
that the ways in which a repository of data can be
used may stretch far beyond what was conceived
when the data was originally collected. This creates a
serious problem: it is necessary to determine the
conditions under which the data was collected and
for what purposes it may be used. Does the
ownership of data bestow the right to use it in ways
other than those purported when it was originally
recorded? Clearly in the case of explicitly collected
personal data it does not. But in general the situation
is complex.
22
11
Data mining and ethics
Ethical issues arise in
practical applications
Data mining often used to discriminate
E.g. loan applications: using some information (e.g. sex,
religion, race) is unethical
Ethical situation depends on application
E.g. same information ok in medical application
Attributes may contain problematic information
E.g. area code may correlate with race
23
Data mining and ethics
Important questions:
Who is permitted access to the data?
For what purpose was the data collected?
What kind of conclusions can be legitimately drawn from
it?
Caveats must be attached to results
Are resources put to good use?
24
12
First Assignment; Marks=5
Write a comprehensive essay on
Introduction to Data Mining, its importance
and practical applications, achievements in
various sectors by 17.10.2011.
Four pages A-4 size, margin 1 from top,
bottom, left and right. Font Time New Roman
size 12. Only hard copy will be accepted.
25
THANK YOU
13