# Register now to access 7 million high quality study materials (What's Course Hero?) Course Hero is the premier provider of high quality online educational resources. With millions of study documents, online tutors, digital flashcards and free courseware, Course Hero is helping students learn more efficiently and effectively. Whether you're interested in exploring new subjects or mastering key topics for your next exam, Course Hero has the tools you need to achieve your goals.

66 Pages

### lec4

Course: CSCI 5512, Spring 2012
School: Minnesota
Rating:

Word Count: 1791

#### Document Preview

Sum-Product The Algorithm CSci 5512: Artificial Intelligence II Instructor: Arindam Banerjee January 30, 2012 Instructor: Arindam Banerjee The Sum-Product Algorithm Factor Graphs Many problems deal with global function of many variables Instructor: Arindam Banerjee The Sum-Product Algorithm Factor Graphs Many problems deal with global function of many variables Global function &quot;factors&quot;...

Register Now

#### Unformatted Document Excerpt

Coursehero >> Minnesota >> Minnesota >> CSCI 5512

Course Hero has millions of student submitted documents similar to the one
below including study guides, practice problems, reference materials, practice exams, textbook help and tutor support.

Course Hero has millions of student submitted documents similar to the one below including study guides, practice problems, reference materials, practice exams, textbook help and tutor support.
Sum-Product The Algorithm CSci 5512: Artificial Intelligence II Instructor: Arindam Banerjee January 30, 2012 Instructor: Arindam Banerjee The Sum-Product Algorithm Factor Graphs Many problems deal with global function of many variables Instructor: Arindam Banerjee The Sum-Product Algorithm Factor Graphs Many problems deal with global function of many variables Global function "factors" into product of local functions Instructor: Arindam Banerjee The Sum-Product Algorithm Factor Graphs Many problems deal with global function of many variables Global function "factors" into product of local functions Efficient algorithms take advantage of such factorization Instructor: Arindam Banerjee The Sum-Product Algorithm Factor Graphs Many problems deal with global function of many variables Global function "factors" into product of local functions Efficient algorithms take advantage of such factorization Factorization can be visualized as a factor graph Instructor: Arindam Banerjee The Sum-Product Algorithm Example Bipartite graph over variables and local functions Instructor: Arindam Banerjee The Sum-Product Algorithm Example Bipartite graph over variables and local functions Edge "is an argument of" relation Instructor: Arindam Banerjee The Sum-Product Algorithm Example Bipartite graph over variables and local functions Edge "is an argument of" relation Encodes an efficient algorithm Instructor: Arindam Banerjee The Sum-Product Algorithm Bayes Nets to Factor Graphs Instructor: Arindam Banerjee The Sum-Product Algorithm Bayes Nets to Factor Graphs fA (x1 ) = p(x1 ) fB (x2 ) = p(x2 ) fC (x1 , x2 , x3 ) = p(x3 |x1 , x2 ) Instructor: Arindam Banerjee The Sum-Product Algorithm Bayes Nets to Factor Graphs fA (x1 ) = p(x1 ) fB (x2 ) = p(x2 ) fC (x1 , x2 , x3 ) = p(x3 |x1 , x2 ) fE (x3 , x5 ) = p(x5 |x3 ) fD (x3 , x4 ) = p(x4 |x3 ) Instructor: Arindam Banerjee The Sum-Product Algorithm Marginalize Product of Functions Many problems involve "marginalize product of functions" (MPF) Instructor: Arindam Banerjee The Sum-Product Algorithm Marginalize Product of Functions Many problems involve "marginalize product of functions" (MPF) Inference in Bayesian networks Instructor: Arindam Banerjee The Sum-Product Algorithm Marginalize Product of Functions Many problems involve "marginalize product of functions" (MPF) Inference in Bayesian networks Compute p(x1 |x4 , x5 ) Instructor: Arindam Banerjee The Sum-Product Algorithm Marginalize Product of Functions Many problems involve "marginalize product of functions" (MPF) Inference in Bayesian networks Compute p(x1 |x4 , x5 ) Need to compute p(x1 , x4 , x5 ) and p(x4 , x5 ) Instructor: Arindam Banerjee The Sum-Product Algorithm Marginalize Product of Functions Many problems involve "marginalize product of functions" (MPF) Inference in Bayesian networks Compute p(x1 |x4 , x5 ) Need to compute p(x1 , x4 , x5 ) and p(x4 , x5 ) Marginalization of joint distribution is a MPF problem Instructor: Arindam Banerjee The Sum-Product Algorithm Marginalize Product of Functions Many problems involve "marginalize product of functions" (MPF) Inference in Bayesian networks Compute p(x1 |x4 , x5 ) Need to compute p(x1 , x4 , x5 ) and p(x4 , x5 ) Marginalization of joint distribution is a MPF problem Several other problems use MPF Instructor: Arindam Banerjee The Sum-Product Algorithm Marginalize Product of Functions Many problems involve "marginalize product of functions" (MPF) Inference in Bayesian networks Compute p(x1 |x4 , x5 ) Need to compute p(x1 , x4 , x5 ) and p(x4 , x5 ) Marginalization of joint distribution is a MPF problem Several other problems use MPF Prediction/Filtering in dynamic Bayes nets Instructor: Arindam Banerjee The Sum-Product Algorithm Marginalize Product of Functions Many problems involve "marginalize product of functions" (MPF) Inference in Bayesian networks Compute p(x1 |x4 , x5 ) Need to compute p(x1 , x4 , x5 ) and p(x4 , x5 ) Marginalization of joint distribution is a MPF problem Several other problems use MPF Prediction/Filtering in dynamic Bayes nets Viterbi decoding in hidden Markov models Instructor: Arindam Banerjee The Sum-Product Algorithm Marginalize Product of Functions Many problems involve "marginalize product of functions" (MPF) Inference in Bayesian networks Compute p(x1 |x4 , x5 ) Need to compute p(x1 , x4 , x5 ) and p(x4 , x5 ) Marginalization of joint distribution is a MPF problem Several other problems use MPF Prediction/Filtering in dynamic Bayes nets Viterbi decoding in hidden Markov models Error correcting codes Instructor: Arindam Banerjee The Sum-Product Algorithm Marginalize Product of Functions (Contd.) The "not-sum" notation h(x1 , x2 , x3 ) = x2 x1 ,x3 h(x1 , x2 , x3 ) Instructor: Arindam Banerjee The Sum-Product Algorithm Marginalize Product of Functions (Contd.) The "not-sum" notation h(x1 , x2 , x3 ) = x2 x1 ,x3 h(x1 , x2 , x3 ) Recall g (x1 , x2 , x3 , x4 , x5 ) = fA (x1 )fB (x2 )fC (x1 , x2 , x3 )fD (x3 , x4 )fE (x3 , x5 ) Instructor: Arindam Banerjee The Sum-Product Algorithm Marginalize Product of Functions (Contd.) The "not-sum" notation h(x1 , x2 , x3 ) = x2 x1 ,x3 h(x1 , x2 , x3 ) Recall g (x1 , x2 , x3 , x4 , x5 ) = fA (x1 )fB (x2 )fC (x1 , x2 , x3 )fD (x3 , x4 )fE (x3 , x5 ) Computing marginal function using not-sum notations gi (xi ) = xi g (x1 , x2 , x3 , x4 , x5 ) Instructor: Arindam Banerjee The Sum-Product Algorithm MPF using Distributive Law We focus on two examples: g1 (x1 ) and g3 (x3 ) Instructor: Arindam Banerjee The Sum-Product Algorithm MPF using Distributive Law We focus on two examples: g1 (x1 ) and g3 (x3 ) From distributive law g1 (x1 ) = fA (x1 ) x1 fB (x2 )fC (x1 , x2 , x3 ) x3 fD (x3 , x4 ) x3 fE (x3 , x5 ) Instructor: Arindam Sum-Product Banerjee The Algorithm MPF using Distributive Law We focus on two examples: g1 (x1 ) and g3 (x3 ) From distributive law g1 (x1 ) = fA (x1 ) x1 fB (x2 )fC (x1 , x2 , x3 ) x3 fD (x3 , x4 ) x3 fE (x3 , x5 ) Also g3 (x3 ) = x3 fA (x1 )fB (x2 )fC (x1 , x2 , x3 ) x3 fD (x3 , x4 ) x3 fE (x3 , x5 ) Instructor: Arindam Banerjee The Sum-Product Algorithm Message Passing Example: Computing g1 (x1 ) Instructor: Arindam Banerjee The Sum-Product Algorithm Message Passing Example: Computing g2 (x2 ) Instructor: Arindam Banerjee The Sum-Product Algorithm Local Transformation for Message Passing Instructor: Arindam Banerjee The Sum-Product Algorithm Sum-Product Algorithm The overall strategy is simple message passing Instructor: Arindam Banerjee The Sum-Product Algorithm Sum-Product Algorithm The overall strategy is simple message passing To compute gi (xi ), form a rooted tree at xi Instructor: Arindam Banerjee The Sum-Product Algorithm Sum-Product Algorithm The overall strategy is simple message passing To compute gi (xi ), form a rooted tree at xi Apply the following two rules: Instructor: Arindam Banerjee The Sum-Product Algorithm Sum-Product Algorithm The overall strategy is simple message passing To compute gi (xi ), form a rooted tree at xi Apply the following two rules: Product Rule: Instructor: Arindam Banerjee The Sum-Product Algorithm Sum-Product Algorithm The overall strategy is simple message passing To compute gi (xi ), form a rooted tree at xi Apply the following two rules: Product Rule: At a variable node, take the product of descendants Instructor: Arindam Banerjee The Sum-Product Algorithm Sum-Product Algorithm The overall strategy is simple message passing To compute gi (xi ), form a rooted tree at xi Apply the following two rules: Product Rule: At a variable node, take the product of descendants Sum-product Rule: Instructor: Arindam Banerjee The Sum-Product Algorithm Sum-Product Algorithm The overall strategy is simple message passing To compute gi (xi ), form a rooted tree at xi Apply the following two rules: Product Rule: At a variable node, take the product of descendants Sum-product Rule: At a factor node, take the product of f with descendants; then perform not-sum over the parent of f Instructor: Arindam Banerjee The Sum-Product Algorithm Sum-Product Algorithm The overall strategy is simple message passing To compute gi (xi ), form a rooted tree at xi Apply the following two rules: Product Rule: At a variable node, take the product of descendants Sum-product Rule: At a factor node, take the product of f with descendants; then perform not-sum over the parent of f Known as the sum-product algorithm Instructor: Arindam Banerjee The Sum-Product Algorithm Computing All Marginals Interested in computing all marginal functions gi (xi ) Instructor: Arindam Banerjee The Sum-Product Algorithm Computing All Marginals Interested in computing all marginal functions gi (xi ) One option is to repeat the sum-product for every single node Instructor: Arindam Banerjee The Sum-Product Algorithm Computing All Marginals Interested in computing all marginal functions gi (xi ) One option is to repeat the sum-product for every single node Complexity of O(n2 ) Instructor: Arindam Banerjee The Sum-Product Algorithm Computing All Marginals Interested in computing all marginal functions gi (xi ) One option is to repeat the sum-product for every single node Complexity of O(n2 ) Repeat computations can be avoided Instructor: Arindam Banerjee The Sum-Product Algorithm Computing All Marginals Interested in computing all marginal functions gi (xi ) One option is to repeat the sum-product for every single node Complexity of O(n2 ) Repeat computations can be avoided Sum-product algorithm for general trees Instructor: Arindam Banerjee The Sum-Product Algorithm Sum Product Updates Variable to local function: xf (x) = hn(x)\f hx Instructor: Arindam Banerjee The Sum-Product Algorithm Sum Product Updates Variable to local function: xf (x) = hn(x)\f hx Local function to variable: f x (x) = x y f (y ) y n(f )\{x} The Sum-Product Algorithm f (x) Instructor: Arindam Banerjee Example: Step 1 fA x1 (x1 ) = fA (x1 ) fB x2 (x2 ) = fB (x2 ) x4 fD (x4 ) = 1 x5 fE (x5 ) = 1 Instructor: Arindam Banerjee The Sum-Product Algorithm Example: Step 2 x1 fC (x1 ) = fA x1 (x1 ) x2 fC (x2 ) = fB x2 (x2 ) fD x3 (x3 ) = x3 fD (x3 , x4 )x4 fD (x4 ) fD (x3 , x5 )x5 fE (x5 ) x3 fE x3 (x3 ) = Instructor: Arindam Banerjee The Sum-Product Algorithm Example: Step 3 fC x3 (x3 ) = x3 fC (x1 , x2 , x3 )x1 fC (x1 )x2 fC (x2 ) x3 fC (x3 ) = fD x3 (x3 )fE x3 (x3 ) Instructor: Arindam Banerjee The Sum-Product Algorithm Example: Step 4 fC x1 (x1 ) = x1 fC (x1 , x2 , x3 )x2 fC (x2 )x3 fC (x3 ) fC (x1 , x2 , x3 )x1 fC (x1 )x3 fC (x3 ) x2 fC x2 (x2 ) = x3 fD (x3 ) = fC x3 (x3 )fE x3 (x3 ) x3 fE (x3 ) = fC x3 (x3 )fD x3 (x3 ) Instructor: Arindam Banerjee The Sum-Product Algorithm Example: Step 5 x1 fA (x1 ) = fC x1 (x1 ) x2 fB (x2 ) = fC x2 (x2 ) fD x4 (x4 ) = x4 fD (x3 , x4 )x3 fD (x4 ) fD (x3 , x5 )x3 fE (x5 ) x5 fE x5 (x5 ) = Instructor: Arindam Banerjee The Sum-Product Algorithm Example: Termination Marginal function is the product of all incoming messages g1 (x1 ) = fA x1 (x1 )fC x1 (x1 ) g2 (x2 ) = fB x2 (x2 )fC x2 (x2 ) g3 (x3 ) = fC x3 (x3 )fD x3 (x3 )fE x3 (x3 ) g2 (x2 ) = fD x4 (x4 ) g5 (x5 ) = fE x5 (x5 ) Instructor: Arindam Banerjee The Sum-Product Algorithm Belief Propagation in Bayes Nets Instructor: Arindam Banerjee The Sum-Product Algorithm Belief Propagation in Bayes Nets fA (x1 ) = p(x1 ) fB (x2 ) = p(x2 ) fC (x1 , x2 , x3 ) = p(x3 |x1 , x2 ) Instructor: Arindam Banerjee The Sum-Product Algorithm Belief Propagation in Bayes Nets fA (x1 ) = p(x1 ) fB (x2 ) = p(x2 ) fC (x1 , x2 , x3 ) = p(x3 |x1 , x2 ) fE (x3 , x5 ) = p(x5 |x3 ) The Sum-Product Algorithm fD (x3 , x4 ) = p(x4 |x3 ) Instructor: Arindam Banerjee
Find millions of documents on Course Hero - Study Guides, Lecture Notes, Reference Materials, Practice Exams and more. Course Hero has millions of course specific materials providing students with the best way to expand their education.

Below is a small sample set of documents:

Minnesota - CSCI - 5512
Approximate Inference: StochasticCSci 5512: Artificial Intelligence II Instructor: Arindam BanerjeeFebruary 1, 2012Instructor: Arindam BanerjeeApproximate Inference: StochasticBayesian Networks with LoopsP(C) .50CloudyC P(S|C) T .10 F .50Sprinkle
Minnesota - CSCI - 5512
Approximate Inference: MCMCCSci 5512: Artificial Intelligence II Instructor: Arindam BanerjeeFebruary 6, 2012Instructor: Arindam BanerjeeApproximate Inference: MCMCProblemsPrimarily of two types: Integration and OptimizationInstructor: Arindam Bane
Minnesota - CSCI - 5512
Junction TreesCSci 5512: Artificial Intelligence II Instructor: Arindam BanerjeeFebruary 13, 2012Instructor: Arindam BanerjeeJunction TreesReparameterizationConsider a Bayesian network p(a, b, c, d) = p(a|b)p(b|c)p(c|d)p(d)Instructor: Arindam Baner
Minnesota - CSCI - 5512
Probabilistic Reasoning over Time: Part ICSci 5512: Artificial Intelligence II Instructor: Arindam BanerjeeFebruary 15, 2012Instructor: Arindam BanerjeeProbabilistic Reasoning over Time: Part IOutlineTime and uncertaintyInstructor: Arindam Banerjee
Minnesota - CSCI - 5512
Probabilistic Reasoning over Time: Part IICSci 5512: Artificial Intelligence II Instructor: Arindam BanerjeeFebruary 22, 2012Instructor: Arindam BanerjeeProbabilistic Reasoning over Time: Part IIHidden Markov ModelsXt is a single, discrete variable
Minnesota - CSCI - 5512
Making Simple DecisionsCSci 5512: Articial Intelligence IIInstructor: Arindam BanerjeeFebruary 27, 2012Instructor: Arindam BanerjeeMaking Simple DecisionsPreferencesApL1pBA lottery is a situation with uncertain prizesLottery L = [p , A; (1 p
Minnesota - CSCI - 5512
Markov Decision ProcessesCSci 5512: Articial Intelligence IIInstructor: Arindam BanerjeeFebruary 29, 2012Instructor: Arindam BanerjeeMarkov Decision ProcessesSequential Decision ProblemsSearchexplicit actionsand subgoalsPlanninguncertaintyand
Minnesota - CSCI - 5512
Game TheoryMechanism DesignCSci 5512: Articial Intelligence IIInstructor: Arindam BanerjeeMarch 7, 2012Instructor: Arindam BanerjeeGame TheoryMechanism DesignOutlinePayos and StrategiesDominant Strategy EquilibriumNash EquilibriumMaximin Strate
Minnesota - CSCI - 5512
Learning From ObservationsCSci 5512: Articial Intelligence IIInstructor: Arindam BanerjeeMarch 19, 2012Instructor: Arindam BanerjeeLearning From ObservationsOutlineLearning AgentsInductive LearningDecision Tree LearningMeasuring Learning Perform
Minnesota - CSCI - 5512
Learning TheoryCSci 5512: Articial Intelligence IIInstructor: Arindam BanerjeeMarch 21, 2012Instructor: Arindam BanerjeeLearning TheoryPAC LearningLearning from a Hypothesis Space HInstructor: Arindam BanerjeeLearning TheoryPAC LearningLearning
Minnesota - CSCI - 5512
Learning with Hidden VariablesCSci 5512: Articial Intelligence IIInstructor: Arindam BanerjeeApril 18, 2012Instructor: Arindam BanerjeeLearning with Hidden VariablesHidden VariablesReal world problem have hidden variablesInstructor: Arindam Banerj
Minnesota - CSCI - 5512
Neural NetworksCSci 5512: Articial Intelligence IIInstructor: Arindam BanerjeeMarch 28, 2012Instructor: Arindam BanerjeeNeural NetworksBrain1011 neurons of &gt; 20 types, 1014 synapses, 1ms10ms cycle timeSignals are noisy spike trains of electrical p
Minnesota - CSCI - 5512
Linear ModelsCSci 5512: Articial Intelligence IIInstructor: Arindam BanerjeeApril 2, 2012Instructor: Arindam BanerjeeLinear ModelsUnivariate Linear Regression(a)(b)hw (x ) = w1 x + w0nnL2 (yi , hw (xi ) =2Loss (hw ) =i =1Instructor: Arinda
Minnesota - CSCI - 5512
Convex FunctionsA function f is convex if dom(f ) is a convex set and [0, 1]f (x1 + (1 )x2 ) f (x1 ) + (1 )f (x2 )A function f is concave if f is convexInstructor: Arindam BanerjeeConvex Analysis and OptimizationFirst Order Conditionsf is convex i
Minnesota - CSCI - 5512
Nonparametric ModelsCSci 5512: Articial Intelligence IIInstructor: Arindam BanerjeeApril 4, 2012Instructor: Arindam BanerjeeNonparametric ModelsParametric Vs NonparametricParametric modelsInstructor: Arindam BanerjeeNonparametric ModelsParametri
Minnesota - CSCI - 5512
Support Vector MachinesCSci 5512: Articial Intelligence IIInstructor: Arindam BanerjeeApril 8, 2012Instructor: Arindam BanerjeeSupport Vector MachinesLinear SeparatorsInstructor: Arindam BanerjeeSupport Vector MachinesLinear SVMs: Separable Case
Minnesota - CSCI - 5512
BoostingCSci 5512: Articial Intelligence IIInstructor: Arindam BanerjeeApril 11, 2012Instructor: Arindam BanerjeeBoostingEnsemble LearningUse a collection of hypothesis from the hypothesis spaceInstructor: Arindam BanerjeeBoostingEnsemble Learni
Minnesota - CSCI - 5512
Statistical LearningCSci 5512: Articial Intelligence IIInstructor: Arindam BanerjeeApril 16, 2012Instructor: Arindam BanerjeeStatistical LearningFull Bayesian learningThe Bayesian view of learningInstructor: Arindam BanerjeeStatistical LearningF
Minnesota - CSCI - 5525
PCA vs FA! PCA! FAProject x to zCombine z to xz = WT(x !)x ! = Vz + !xzzxE. Alpaydin, Introduction to Machine LearningFactor Analysis! Finda small number of factors z, which whencombined generate x :xi !i = vi1z1 + vi2z2 + . + vikzk + !iw
Minnesota - CSCI - 5525
CSCI 5525: Machine Learning (Spring 2012)Dimension ReductionRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaFeature SelectionNP-hard to search through all the combinations Needheuristic solutionsThe assumption is bas
Minnesota - CSCI - 5525
CSCI 5525: Machine Learning (Spring 2012)Dimension ReductionRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaLinear Discriminant Analysis Finda low-dimensionalspace such that when xis projected, classes arewell-separ
Minnesota - CSCI - 5525
CSCI 5525: Machine Learning (Spring 2012)Course OverviewRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaWelcome to CSci 5525Course: Machine LearningInstructor: Rui Kuang (Ray), Assistant Professor (CS&amp;E) Contact:Offi
Minnesota - CSCI - 5525
CHAPTER 5:Multivariate MethodsE. Alpaydin, Introduction to Machine LearningMultivariate Data Multiplemeasurements (sensors) d inputs/features/attributes: d-variate N instances/observations/examples111X X X 12d 22212dX X X X= NN
Minnesota - CSCI - 5525
CSCI 5525: Machine Learning (Spring 2012)Bayes DecisionTheory andParametric ModelsRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaRegression exampleCoefficients increase inmagnitude as orderincreases:1: [-0.0769, 0
Minnesota - CSCI - 5525
CSCI 5525: Machine Learning (Spring 2012)Bayes DecisionTheory andParametric ModelsRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaParametric vs NonparametricParametric methods: Amodel (usually a type of simple distr
Minnesota - CSCI - 5525
CSCI 5525: Machine Learning (Spring 2012)Bayes DecisionTheory andParametric ModelsRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaParametric Classification Discriminantfunctiongi ( x ) = p( x | Ci ) P (Ci )orgi (
Minnesota - CSCI - 5525
CSCI 5525: Machine Learning (Spring 2012)Supervised LearningRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaSupervised Learning Classification RegressionInput Feature Space&quot; x1 %\$'\$ x2 'x = \$ . '\$'\$ . '\$xD '#&amp;
Minnesota - CSCI - 5525
CSCI 5525: Machine Learning (Spring 2012)Supervised LearningRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaSupervised Learning ClassificationData: RegressionX = cfw_x t,r t N=1tX = cfw_x t,r t N=1trt &quot; #(Clas
Minnesota - CSCI - 5525
CSCI 5525: Machine Learning (Spring 2012)Supervised LearningRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaNoise and Model ComplexityGiven similar training error,use the simpler oneSimpler to use (lowercomputational
Minnesota - CSCI - 5525
CSCI 5525: Machine Learning (Spring 2012)NonparametricMethodsRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaDensity EstimationGiven the training set X=cfw_xtt drawn iid from p(x)Divide data into bins of size h Histo
Minnesota - CSCI - 5525
CSCI5525: Machine Learning (Spring 2012)ClusteringRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaExpectation-Maximization (EM)Complete likelihood, Lc( |X,Z), in terms of x and zLc (&quot; | X ) = log # p(x t , zt | &quot;) = \$t
Minnesota - CSCI - 5525
CSCI 5525: Machine Learning (Spring 2012)Linear DiscriminationRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaLikelihood- vs. Discriminantbased ClassificationLikelihood-based: Assume a model for p(x|Ci),use Bayes rule
Minnesota - CSCI - 5525
CSCI 5525: Machine Learning (Spring 2012)Linear DiscriminationRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaLogistic DiscriminationlogTwo classes: Assume log likelihood ratio is linearp (C1 | x )p (C1 | x )= log=
Minnesota - CSCI - 5525
CSCI 5525: Machine Learning (Spring 2012)Local ModelsRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaOnline k-meansWinner-take-all networkWeight decay term:!mij = !bit ( x tj &quot; mij ) = !bit x tj &quot; !bit mijE. Alpaydin
Minnesota - CSCI - 5525
CSCI 5525: Machine Learning (Spring 2012)Multilayer PerceptronRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaBiological Neural Nets Pigeonsas artexperts (Watanabe et al. 1995)Experiment: Pigeonin Skinner box Pres
Minnesota - CSCI - 5525
CSCI 5525: Machine Learning (Spring 2012)Multilayer PerceptronRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaBackpropagationHy i = vT z = &quot; v ih zh + v i 0ih =1zh = sigmoid ( wT x )h=1*\$ d&amp;&quot; w hj x j + w h 0 '
Minnesota - CSCI - 5525
CSCI 5525: Machine Learning (Spring 2012)Multilayer PerceptronRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaTuning the Network SizeDestructiveWeight decay:penalizing non-zeroparameters The same as addingadditiona
Minnesota - CSCI - 5525
CSCI5525: Machine Learning (Spring 2012)ClusteringRui KuangDepartment of Computer Science and EngineeringUniversity of MinnesotaMixture of GaussiansKp( x | &quot;) = % # iN ( x | i , \$i ),i =1Kwith 0 &amp; # i &amp; 1 and % # i = 1.i =1Mixture of Gaussians
Webster - FINC - 5880
A firm has 10 million shares outstanding with a market price of \$20 per share. The firm has \$25 million in extra cash (short term investments)that is plans to use in a stock repurchase; the firm has no other financial investments or any debt. What is the
Webster - FINC - 5880
Axel Telecommunications has a target capital structure that consists of 70% debt and 30% equity. The company anticipates that its capital budget for the upcoming year will be \$3 million. IfAxel reports net income of \$2 million and follows a residual dist
Webster - FINC - 5880
A firm has 10 million shares outstanding with a market price of \$20 per share. The firm has \$25 million in extra cash (short term investments)that is plans to use in a stock repurchase; the firm has no other financial investments or any debt. What is the
Drexel - ECON - 201
Econ Final ReviewChapter 1- Scarcity - the limited nature of society's resources- Opportunity cost - whatever must be given up to obtain some item- Marginal cost - the increase or decrease in costs as a result of one more or one less unit ofoutput-
North Texas - ACCT - 5130
Multiple Choice Questions1. Generally speaking, which of the following is not one of the primary purposes of abudget?A. Identifying a company's most profitable products.B. Evaluating performance.C. Planning.D. Controlling profit and operations.E. F
Drexel - ENGR - 361
ENGR 361 Statistical Analysis of Engineering Systems (Spring, 2012)Homework 1 Solutions
Florida State College - BUS - 305
Bus Prin of Orig Behaivor
Florida State College - BUS - 305
Week 8 - Assignment #2Coca-ColaOctavial RobinsonPrincipal of Organizational BehavioralDr. Daphyne FosterStrayer UniversityAugust 21, 20111. What do you think is the most important emerging issue in the design of work?The most important rising issu
Florida State College - BUS - 305
Florida State College - BUS - 305
Week 1Companies role should change every so often as time changes. A company that stays the same will fail. That is whyit is important for companies to be positive impacts in the contemporary world. People changes and so will thecompany which will acco
Florida State College - BUS - 305
Week 1Companies role should change every so often as time changes. A company that stays the samewill fail. That is why it is important for companies to be positive impacts in the contemporaryworld. People changes and so will the company which will acco
CSU Northridge - COMP - 122
Conditional statementsLoop: for, while, do-whileBRGE- branch(br) greater than or equal(ge)Accumulator(A) - using LOAD A RR stores a value for a variable Ex: var i; LOAD A RR, ii = i + 10; would use STORE A RR, i | for the answer |Full code for ^ :
CSU Northridge - COMP - 122
Variable: static v. dynamic Assembler
CSU Northridge - COMP - 122
CSU Northridge - COMP - 122
#ram/CPU,SP,PCHow to select to fetch 1 or 3 bytesInspect Instruction#von Newman cycle T#vonnewmancycle, page 168 in bookUnary Instructions: Instructions that don't have an operand specier, single 8 bitinstruction
CSU Northridge - COMP - 122
How long for instruction? length(bits)#ram, CPU
CSU Northridge - COMP - 122
CSU Northridge - COMP - 122
Deign Machine Langua&quot; 1. Ram size address8-bit address Capacity of RAM : 16-bit address Capacity of RAM : 32-bit address Capacity of RAM : 2. Character Set.ASCII (256 letters) uses 8-bit allocation (1 byte, 1 letter)Common 128 Characters 0000,000
CSU Northridge - COMP - 122
Negative to Binary Given allocation: 8 bits Number: -24 Question: nd its binary representation1. Drop the sign (-) = 242. To binary: 11000 [practice decimal to binary conversion]3. Fit in 8 bits: 00011000 [ll in allocation]4. Flip: 11100111 [ones
CSU Northridge - COMP - 122
Binary Operators Given allocation and number in binary Question: nd its shift left1. Fit the numbers to the given allocation2. Shift left (c-bit: carry bit)Question: nd its shift right. Binary: 0 1Binary Operators require 2 numbersUnary Operator
CSU Northridge - COMP - 122
Ram on Hex vs. BinAllocation address Offset ContentConversion: Two's Complement AllocationDec to Bin (+/-)Bin to DecBinary to Decimal2's complement Method if number starts with a zero. Finding AllocationBase to BaseCombinations
CSU Northridge - COMP - 122
Combinations Each representation is associated to an object. Allocation # of representationEx: 3 bits, 1 letter, 2 octals.Processor (CPU) - accesses the RAM Circuits are like road systems, usesthe id to locate the objectsIDs are actually called ad
CSU Northridge - COMP - 122