cs229-prob

# cs229-prob - Review of Probability Theory Arian Maleki and...

This preview shows pages 1–3. Sign up to view the full content.

Review of Probability Theory Arian Maleki and Tom Do Stanford University Probability theory is the study of uncertainty. Through this class, we will be relying on concepts from probability theory for deriving machine learning algorithms. These notes attempt to cover the basics of probability theory at a level appropriate for CS 229. The mathematical theory of probability is very sophisticated, and delves into a branch of analysis known as measure theory . In these notes, we provide a basic treatment of probability that does not address these Fner details. 1 Elements of probability In order to deFne a probability on a set we need a few basic elements, Sample space Ω : The set of all the outcomes of a random experiment. Here, each outcome ω Ω can be thought of as a complete description of the state of the real world at the end of the experiment. Set of events (or event space ) F : A set whose elements A ∈ F (called events ) are subsets of Ω (i.e., A Ω is a collection of possible outcomes of an experiment). 1 . Probability measure : A function P : F → R that satisFes the following properties, - P ( A ) 0 , for all A - P (Ω) = 1 - If A 1 ,A 2 ,... are disjoint events (i.e., A i A j = whenever i ± = j ), then P ( i A i )= ± i P ( A i ) These three properties are called the Axioms of Probability . Example : Consider the event of tossing a six-sided die. The sample space is Ω = { 1 , 2 , 3 , 4 , 5 , 6 } . We can deFne different event spaces on this sample space. ±or example, the simplest event space is the trivial event space F = {∅ , Ω } . Another event space is the set of all subsets of Ω . ±or the Frst event space, the unique probability measure satisfying the requirements above is given by P ( )=0 ,P (Ω) = 1 . ±or the second event space, one valid probability measure is to assign the probability of each set in the event space to be i 6 where i is the number of elements of that set; for example, P ( { 1 , 2 , 3 , 4 } 4 6 and P ( { 1 , 2 , 3 } 3 6 . Properties : - If A B = P ( A ) P ( B ) . - P ( A B ) min( P ( A ) ( B )) . - (Union Bound) P ( A B ) P ( A )+ P ( B ) . - P \ A ) = 1 - P ( A ) . - (Law of Total Probability) If A 1 ,...,A k are a set of disjoint events such that k i =1 A i = Ω , then k i =1 P ( A k ) = 1 . 1 F should satisfy three properties: (1) ∅ ∈ F ; (2) A = Ω \ A ; and (3) A 1 2 , . . . = i A i . 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
1.1 Conditional probability and independence Let B be an event with non-zero probability. The conditional probability of any event A given B is deFned as, P ( A | B ) ± P ( A B ) P ( B ) In other words, P ( A | B ) is the probability measure of the event A after observing the occurrence of event B . Two events are called independent if and only if P ( A B )= P ( A ) P ( B ) (or equivalently, P ( A | B P ( A ) ). Therefore, independence is equivalent to saying that observing B does not have any effect on the probability of A . 2 Random variables Consider an experiment in which we ±ip 10 coins, and we want to know the number of coins that come up heads. Here, the elements of the sample space Ω are 10-length sequences of heads and tails. ²or example, we might have w 0 = ± H,H,T,H,T,H,H,T,T,T ²∈ Ω . However, in practice, we usually do not care about the probability of obtaining any particular sequence of heads and tails.
This is the end of the preview. Sign up to access the rest of the document.

## This document was uploaded on 10/19/2011.

### Page1 / 12

cs229-prob - Review of Probability Theory Arian Maleki and...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online