COM1070: Introduction to Artificial Intelligence Week 5--Learning Professor M. Walker Dr G. Gorrell Overview What is learning? Types of learning Some popular/current algorithms o o o o o o o o o o Decision trees Bayesian Learning Expectation Maximisation K-Means K-Nearest Neighbour Neural Networks Support Vector Machines Hidden Markov Models Reinforcement Learning Genetic Algorithms What does this tell us about intelligence? Cognitive Systems University of Sheffield What is learning? A computer algorithm that will, over time, improve itself along clearly defined dimensions From Mitchell, 1997: "A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P improves with experience E." Cognitive Systems University of Sheffield Induction Inductive reasoning not strictly correct o All observed crows are black o Therefore, all crows are black? However, is the basis of most forms of learning Use past observations to predict the future Cognitive Systems University of Sheffield Machine Learning What word string does a speech signal correspond to? Do these readings mean the patient is has diabetes or not? Should the robot turn left or go straight on? Cognitive Systems University of Sheffield Machine Learning More generally, given these data, which of a range of outcomes/results is the best? Classification problems Cognitive Systems University of Sheffield Why should I care about machine learning? Cornerstone of Computer Science More and more applications performing useful functions Adaptation an important part of systems of the future o o o o Computer systems that repair themselves Data mining to determine trends Robots that can modify behaviour based on sampling environment Games that adapt over time/according to opposing player Cognitive Systems University of Sheffield Components of ML Task T o Recognize speech, understand speech, play chess, drive a car, predict the weather Experience E o Labelled speech, annotated utterances, successful chess games, completed journeys, weather outcomes Performance measure P o Number of correct words, number of correct concepts, winning a game, arriving without incident, correct prediction Cognitive Systems University of Sheffield Types of learning Supervised Unsupervised Cognitive Systems University of Sheffield Supervised learning Predictive Parametric--data is separated into classes a priori Given labelled training data, predict what class a new data point will fall into Applications: Document Classification o o o Given a set of documents labelled for topic Given a set of parameters to measure, such as words, position of word in document, word cooccurrences What can I learn about how to classify a new document into the existing classes? Cognitive Systems University of Sheffield Unsupervised learning Non-parametric Goal is to learn about structure of dataset in order to build reasonable clusters of data Implicit distance function Fit the model to the observations Application: Clustering of astronomical data o o Given the spectra of a series of celestial bodies What can we learn about similarities/differences among these bodies? Cognitive Systems University of Sheffield Cross-Validation Learner is trained on E Performance should be evaluated on new data So we use held-out data (test set) Or, we use cross-validation divide data into sections o hold out each section in turn o test on the held out data o average the result over all sections o Cognitive Systems University of Sheffield Popular Learning Algorithms Decision trees Bayesian Learning Expectation Maximisation K-Means K-Nearest Neighbour Neural Networks Support Vector Machines Hidden Markov Models Reinforcement Learning Genetic Algorithms Cognitive Systems University of Sheffield WEKA Toolkit Free open source ML toolkit Provides o Decision trees o Support Vector Machines o Boosting o Expectation Maximisation o And much more ... From the University of Waikato, New Zealand Cognitive Systems University of Sheffield WEKA Toolkit CLI--Command line interface Explorer--a good place to start Experimenter and KnowledgeFlow --more advanced Cognitive Systems University of Sheffield WEKA Toolkit Cognitive Systems University of Sheffield Example Sample data set-- given weather outlook, temp, humidity, and wind conditions, should I play tennis? Class (predicted value): Y/N Outlook sunny sunny overcast rain rain rain overcast sunny sunny rain sunny overcast overcast Temperature Humidity 85 80 83 70 68 65 64 72 69 75 75 72 81 Windy 85 90 86 96 80 70 65 95 70 80 70 90 75 Class FALSE N TRUE N FALSE Y FALSE Y FALSE Y TRUE N TRUE Y FALSE N FALSE Y FALSE Y TRUE Y TRUE Y FALSE Y Cognitive Systems University of Sheffield Load data into WEKA Cognitive Systems University of Sheffield Decision Trees Supervised o learn a decision process that will make the right decision in future cases Finds the attribute that splits the data best in terms of determining the correct class Within each split, what attribute splits the data best now? And so on ... Cognitive Systems University of Sheffield Cognitive Systems University of Sheffield Cognitive Systems University of Sheffield Decision trees Learning a matter of inducing rules that cover input data (propositional logic) Useful for datasets containing lots of attributes, where there is a clear final "decision" (e.g., given what I know about my computational needs, which computer should I buy?) Also useful for learning descriptions of large datasets (given a large set of attributes for cities in England, how can I describe those cities to a tourist?) Cognitive Systems University of Sheffield Decision Trees Values in categories can be either numeric (scalar) or symbolic (categorical) Can deal with missing values, unknown values Can be helpful to preprocess data into useful attributes o for example, don't just give it a string of words--tell it the number of words too Cognitive Systems University of Sheffield Decision trees Problems include data sparsity o split means that leaves rapidly become short on data o decisions end up being based on few data points Cognitive Systems University of Sheffield Limitations on rule-based learning Initial work in AI was mostly rule-based Problems: o To us, the world looks rule-governed, binary decisions prevail (e.g., IF there is decent snow this year AND I can get a cheap flight AND the hotels are not booked THEN I'll go to Whistler over Christmas break) o When the world doesn't behave, you can do some simple error correction to make things right Cognitive Systems University of Sheffield What happens in the real world? Lots and lots of data--generalizations fail Example from speech recognition: o o o o o The sounds of speech are governed by manner and place of articulation Measurements of energy frequency and time characterize phonemes Build rules to reflect what we know about articulatory/acoustic phonetics Test it on small corpus--works great! Test it on large corpus--oops. Solution: Use world knowledge but use statistics to guide the decisions Such a bad idea? Maybe the world is probabilistic? Cognitive Systems University of Sheffield Na ve Bayes classifier From our initial training data, we can compute probabilities Given that we played, how often was it sunny? Hot? Outlook TemperatureHumidity Windy Class sunny hot high false N sunny hot high true N overcast hot high false Y rain mild high false YP N Humidity Y Nfalse rain cool normal Y 3/5 high 3/9 4/5 rain cool normal true N N overcast cool true Y 0 normal normal1/5 6/9 high false N 4/5 sunny mild 2/5 Y 1/5 sunny cool Windy normal false rain mild normal false Y 2/5 true 3/9 3/5 sunny mild normal true Y 2/5 false 6/9 overcast mild high 2/5 true Y 3/5 1/5 normal false Y 2/5 overcast hot rain mild high true N Outlook Y sunny 2/9 overcast 4/9 rain 3/9 Temperature hot 2/9 mild 4/9 cool 3/9 N 3/5 0 2/5 2/5 2/5 1/5 Humidity Y high 3/9 normal 6/9 Windy true 3/9 false 6/9 Cognitive Systems University of Sheffield Bayes' Rule The only equation in the lecture! But it's a REALLY useful one! We want to know this P BA P A P AB = PB This is the information we typically have in most tasks Cognitive Systems University of Sheffield Bayes' Rule Cornerstone of Bayesian methods We have a PRIOR probability (probability that we play tennis, in any conditions) We know from experience the probability that it is sunny given that we play o more difficult to produce the reverse model since it would require generating sunny days! We want to know the POSTERIOR probability (given that it is sunny, what is the probability that we can play?) Cognitive Systems University of Sheffield Na ve Bayes Classifier Outlook New incoming event: P(sunny|y) = 2/9 P(overcast|y) = 4/9 overcast, mild, normal, true P(rain|y) = 3/9 Temperature Multiply out probabilities to P(hot|y) = 2/9 determine whether P(mild|y) = 4/9 conditions = Play/Not P(cool|y) = 3/9 play Factoring all the conditions in is just an P(high|y) = 3/9 P(normal|y) = 6/9 extension of Bayes' Rule Windy P(true|y) = 3/9 P(false|y) = 6/9 Humidity P(sunny|n) = 3/5 P(overcast|n) = 0 P(rain|n) = 2/5 P(hot|n) = 2/5 P(mild|n) = 2/5 P(cool|n) = 1/5 P(high|n) = 4/5 P(normal|n) = 2/5 P(true|n) = 3/5 P(false|n) = 2/5 Cognitive Systems University of Sheffield Cognitive Systems University of Sheffield Na ve Bayes classifier Supervised method (we have the "right answers") Powerful and works quite well Na ve assumption: attributes are conditionally independent o Is it really true that I treat the rain conditions completely separately from the temperature when I'm making the decision? Cognitive Systems University of Sheffield Clustering Visualise data in vector space Classify data into clusters Find natural clusters (unsupervised) Cognitive Systems University of Sheffield A New Dataset--Film Preferences Action Romance 0 3 2 4 3 4 3 4 5 7 8 3 50 37 36 35 50 43 44 47 42 43 41 37 25 47 38 49 4 3 2 4 On a scale of 0-50 o how much do you like action films? o how much do you like romance films? Cognitive Systems University of Sheffield Cognitive Systems University of Sheffield K Means Unsupervised Choose centroids at random Classify points using distance Choose new centroids by taking the mean of the cluster Reclassify, repeat ... Cognitive Systems University of Sheffield K Means Two centroids are positioned at random Cognitive Systems University of Sheffield K Means Each datapoint is allocated to a centroid based on distance Cognitive Systems University of Sheffield K Means Each datapoint is allocated to a centroid based on distance Cognitive Systems University of Sheffield K Means Centroids are moved to the middle of their cluster Cognitive Systems University of Sheffield K Means Reassign the points to clusters Cognitive Systems University of Sheffield K Means Reassign the points to clusters Cognitive Systems University of Sheffield K Means Move the centroids Repeat until stable Cognitive Systems University of Sheffield Cognitive Systems University of Sheffield K Means Choosing the number of centroids o you must choose Validity of the distance metric o spherical clusters? Relationship to Expectation Maximisation o the principle is very similar o but we aren't covering EM today Cognitive Systems University of Sheffield Today's Thought Question Think of a simple decision that you make in your everyday life and describe how you would use machine learning to make a computer make the decision Cognitive Systems University of Sheffield K-Nearest Neighbour We have correctly classified training points Now we get a new data point To which class does it belong? Supervised Lazy Cognitive Systems University of Sheffield K-Nearest Neighbour Find K nearest neighbours New point is classified based on majority vote o so in binary classification, choose an odd number for k! Cognitive Systems University of Sheffield K-Nearest Neighbor How should New data point the data point be classified? Depends on k! k=5 produces different answer to k=3 Cognitive Systems University of Sheffield K-Nearest Neighbor Weight can be assigned to distance from point X o so closer points count for more Cognitive Systems University of Sheffield More Classification So far we have classified data points: o using decision trees o using Bayesian (probabilistic) reasoning o using vector space representations using distance to centroids (K Means) using proximity to correctly labeled examples (K Nearest Neighbours) What next? Let's try to draw a line between the classes! Cognitive Systems University of Sheffield Support Vector Machines Attempt to find a hyperplane that separates data Goal: maximize margin separating two classes Cognitive Systems University of Sheffield Support Vector Machines Points near decision boundary: support vectors (removing them would change boundary) Points far from boundary not important for decision Cognitive Systems University of Sheffield Support Vector Machines What if there is no separating hyperplane? They do not work! Cognitive Systems University of Sheffield Kernel Functions Map data into different dimensionality Now the points are separable! But it isn't magic ... The information needs to be in the dataset, somewhere Cognitive Systems University of Sheffield Support Vector Machines Assumes maximising the distance from the points to the separating hyperplane makes the most general classifier But, the points may not be separable Kernel methods may help Methods exist to allow for points that don't separate cleanly Cognitive Systems University of Sheffield And now for something completely different ... What if our data changes over a series of time steps? o o What I say to you next relates to what I said to you previously The sound I make at time t ("ee") relates to the sound I make at time t-1 ("r") The task is to classify the observation, taking the previous time step into account You used the same method to classify the previous time step! Cognitive Systems University of Sheffield Markov Decision Processes Split up time into a series of discrete steps (states) Make the MARKOV ASSUMPTION: that what happens in a state depends only on what happened in the immediately preceding state, not in any earlier state Cognitive Systems University of Sheffield Markov Decision Processes Hidden Markov Models o Use the model to determine the HIDDEN variables based on observations o Speech recognition--we observe the speech signal and determine the ("hidden") word sequence Partially Observable Markov Decision Processes (POMDPs) o o o Can be used in modelling dialogue Used to decide system behaviour Therefore not entirely "hidden" Reinforcement Learning Cognitive Systems University of Sheffield Hidden Markov Models for Speech Recognition What did the person say? HMM tells us how likely various options are p=0. 1 We observe ap b p=0.3 =0 .1 speech signal Try to figure out phone sequence p= p=0.1 0.1 .1 p=0 p=0.2 "ba" Cognitive Systems University of Sheffield Hidden Markov Models for Speech Recognition p=0.1 State transition probabilities p=0.3 b a p= 0.1 Observation probabilities Cognitive Systems University of Sheffield p=0.1 p =0 .1 .1 p =0 p=0.2 "ba" HMM Algorithms Baum-Welch Algorithm o Set the HMM parameters to maximise the likelihood of the training data Viterbi Algorithm o Figure out the path through the HMM that most likely produced the observation sequence Forward-Backward Algorithm o Find the probability of a particular output sequence Cognitive Systems University of Sheffield HMMs for Speech Recognition Allows for varying speeds of speech o loop back to same state Assumes phone depends only on previous phone o Clearly not true! Forms the basis of most modern automatic speech recognisers Cognitive Systems University of Sheffield Very...