# Register now to access 7 million high quality study materials (What's Course Hero?) Course Hero is the premier provider of high quality online educational resources. With millions of study documents, online tutors, digital flashcards and free courseware, Course Hero is helping students learn more efficiently and effectively. Whether you're interested in exploring new subjects or mastering key topics for your next exam, Course Hero has the tools you need to achieve your goals.

10 Pages

### lecture05-NaiveBayes-2up

Course: CS 573, Fall 2009
School: Iowa State
Rating:

Word Count: 986

#### Document Preview

Bayes Naive Classifier 1 Bayesian recipe for classification The Bayesian recipe is simple, optimal, and in principle, straightforward to apply We could design an optimal classifier if we knew: P(i) (priors) p(x | i) (class-conditional densities) We have some knowledge and training data {(xi,i)} Use the samples to estimate the unknown probability distributions x is typically high-dimensional Need to estimate...

Register Now

#### Unformatted Document Excerpt

Coursehero >> Iowa >> Iowa State >> CS 573

Course Hero has millions of student submitted documents similar to the one
below including study guides, practice problems, reference materials, practice exams, textbook help and tutor support.

Course Hero has millions of student submitted documents similar to the one below including study guides, practice problems, reference materials, practice exams, textbook help and tutor support.
Bayes Naive Classifier 1 Bayesian recipe for classification The Bayesian recipe is simple, optimal, and in principle, straightforward to apply We could design an optimal classifier if we knew: P(i) (priors) p(x | i) (class-conditional densities) We have some knowledge and training data {(xi,i)} Use the samples to estimate the unknown probability distributions x is typically high-dimensional Need to estimate P(x| ) from limited data 1 2 Naive Bayes Classifier Along with decision trees, neural networks, nearest neighbor, one of the most practical learning methods. Categories {1, 2,..., c}, Feature/attribute vector x = [x1, x2, ..., xd ]t Naive Bayes assumption: P ( x1 ,L , x d | Naive Bayes classifier: j ) = i j P (xi | j ) NB = arg max j P ( ) i P (xi | j ) Performs optimally under certain assumptions 3 Naive Bayes Classifier Given training data set D Need to estimate probabilistic parameters, no need for complicated training process as in neural networks Estimate P(j) = nj/n (Maximum Likelyhood estimation) Estimate P(xi=aik| j) ML Estimation Njik/nj discrete feature 2 Training Examples 4 5 Example Consider PlayTennis problem, and new instance <Outlk=sun, Temp=cool, Humid=high, Wind=strong> We estimate parameters P(yes) = 9/14, P(no) = 5/14 P(Wind=strong|yes) = 3/9 P(Wind=strong|no) = 3/5 ... We have P(y) P(sun|y) P(cool|y) P(high|y) P(strong|y) = .005 P(n) P(sun|n) P(cool|n) P(high|n) P(strong|n) = .021 Therefore this new instance is classified to "no" 3 Estimation of Probabilities from Small Samples 6 Estimate P(xi=aik| j) discrete feature ML Estimation Njik/nj Poor estimates when nj is small What if none of the training instances with category j have feature value xi=aik? P(xi=aik | j) = 0, which lead to P(j|..., xi=aik,...)=0 Typical solution is Bayesian estimate 7 Bayesian estimates for estimating k =P(Xi=k) from data set Dj P( X i = k | D j) = N jik + Mp k nj + M Laplace estimates: M=|Dm(Xi)|, Mpk=1 M: imaginary equivalent sample size pk: prior belief about k , summation to 1 The larger the equivalent sample size M, the more confident we are in our prior 4 8 Estimate density p(xi| j) continuous feature , , or Assume e.g. Gaussian distribution N(,2), then estimate Discretize into {1, ..., k} Equal-width interval: Width = (xmax xmin)/k Convert x to i if x is in ith interval 9 Naive Bayes Classifier Conditional independence assumption is often violated But it works surprisingly well anyway (Domingos and Pazzani, 1997) Successful applications: Diagnosis Learn which news articles are of interest. Learn to classify web pages by topic. Learn to assign proteins to functional families Performance often comparable to that of neural networks, decision tree, etc. 5 10 Learning to Classify Text Learn which news articles are of interest Target concept Interesting? : Documents {+,-} Learning: Use training examples to estimate P (+), P (- ), P (doc |+), P (doc |-) What attributes shall we use to represent text documents? 11 Text each Representation Represent document by vector of words one attribute per word position in document P(doc| j ) = P(length (doc )| j ) P( X i = w k | j ) i =1 |doc| We need a probability for each word occurrence in each position in the document: 2 x length x |vocabulary| Too many probabilities to estimate! Limited samples 6 12 Binary Independence Model Given a vocabulary V: (w1, ..., w|V|) A document is a vector of binary features (X1, ..., X|V|) Xi is 1 if wi appears in the document, 0 otherwise x P(doc| j ) = P( xi | j ) = jii (1 - ji )1- xi i =1 i =1 |V | |V | ^ji = N ji + c1 Nj +c N ji :# of documents in class j with word wi Multi-variate Bernoulli Model The number of times a word occurs in a document is not captured 13 Multinomial Model Assume that probability of encountering a specific word in a particular position is independent of the position, P(wk|j) The number of probabilities to be estimated drops to 2 x |vocabulary| Treat each document as a bag of words! Each document d results from |d| draws on a multinomial variable X with |V| values The number of times a word occurs in a document is captured 7 14 Multinomial Model Assume that the lengths of documents are independent of class P(d| j ) = P(| d | ) ( N k )! k k P(w N ! k k =1 |V | k | j )N k N k is the # of occurences of w k in document d N jk + ck ^ P( w k | j ) = jk = N jk + c k N jk is the # of occurences of w k in documents in class j 15 Learn_naive_Bayes_text(Examples) 1. collect all words and other tokens that occur in Examples Vocabulary: all distinct words and other tokens in Examples For each target value j do 2. calculate the required probability terms docsj : subset of Examples for which the target value is j P(j) = |docsj|/|Examples| Textj : a single document created by co...

Find millions of documents on Course Hero - Study Guides, Lecture Notes, Reference Materials, Practice Exams and more. Course Hero has millions of course specific materials providing students with the best way to expand their education.

Below is a small sample set of documents:

Iowa State - CS - 573
Information Theory p.1InformationA discrete random variable X , that can take on possible valuesx1 , . . . , xn with distribution P(xi )How much information is conveyed when you are told a specic value? Information content of a message is rel
Iowa State - CS - 573
Review of Probability Theory1Outline Uncertainty Probability basics Random variables Probability Distributions Probabilistic Inference Continuous variables Expected Values Independence Bayes' Rule2Representing and Reasoning under Unc
Iowa State - CS - 573
Bayesian networksSyntax and Semantics p.1Bayesian ClassifiersGenerative models: model P(x1 , . . . , xd , ) E.g., Naive Bayes classifier Naive Bayes assumption Use Bayesian networks to model P(x1 , . . . , xd , ) Systematically use domain knowl
Iowa State - DEC - 0308
Iowa State - ENG - 461
SUBROUTINE CMINEX (ANALYZ, X, VLB, VUB, ISC, NDV, NCON, NSIDE, &amp; IPRINT, ITMAX, IER) IMPLICIT REAL*8 (A-H,O-Z)CC*C**C* *C* Murphy Laboratory - Ap
Iowa State - ENG - 461
Iowa State University Department of Aerospace Engineering AerE 461Necessary Conditions for Best Solutions Design Optimimization Techniques Formulation: 1. Unconstrained Problem Given the constants vector, c = (c1 , c2 , c3 , c4 ) = (1,3,2,1) Minimiz
Iowa State - ENG - 524
The general 2D variational grid generation systemThe general 2D variational grid generation system is given by: Ar cc ) Br ch ) Cr hh ) Dr c ) Er h + 0 where A=As+Ao+Aa, B=Bs+Bo+Ba, C=Cs+Co+Ca. These matrices in addition to D and E are given by: *
Iowa State - ENG - 461
Aero 361 Problem 6We will do a trajectory optimization for the orbit insertion of a multistage, rocket powered vehicle. This problem focuses on the optimization process and the coupling of the trajectory analyzer developed in problem 3 with the opti
Iowa State - ENG - 361
Wing Aerodynamics AnalysisWing DefinitionWe will develop a model for a 3D wing geometry with sweep, taper, and twist. The airfoil section used to construct the wing is arbitrary. Consider the wing shown below:Figure 1z y ctip rootr V cho
Iowa State - ENG - 361
Aero 361 Problem 2A spherical mass of .25 slugs is hanging still at sea level 3 feet below a frictionless and rigidly supported bearing. The mass is perturbed from this position by ro tating it through an angle, qo, and then releasing it. We wish to
Iowa State - ENG - 461
Formulation We seek to optimize the functionalJ = J( c , x , u )(1)subject to equality constraintshk ( c , x, u ) = 0and inequality constraintsk = 1,2,3, . . lj = 1,2,3, . . m(2)g j ( c , x, u ) 0where( 3)represents an n-compone
Iowa State - ENG - 461
Preliminary DesignAerospace Engineering FlavorTopics Covered Blooms Cognitive Taxonomy Analysis, Synthesis, Design Conceptual, Preliminary, Detailed Design Aerospace Industry Design Loop Role of Analysis in Design Endeavorsl A A ATaxonomy: Stu
Iowa State - ENG - 461
199419951996D J F M A M J J A S O N D J F MA M J J A S O N D J F M2-D Airfoil Design 2-D Transonic WT S &amp; C Studies Analytical 3-D Aero 3-D Low Speed WT 3-D Transonic WT Simulator Program Performance Preliminary Airloads6 53 1.5 2.5 4.5 1.
Iowa State - ENG - 643
Consider first the mass, M, as the extensive property of some control mass of a substance. There is no dissipation of this property by friction and we will assume no depletion by chemical reactions. Thus, the conservation of mass for the stationary c
Iowa State - ENG - 461
Transcendental Constraint SystemsA transcendental constraint system is one in which an explicit solution for the objective function cannot be obtained. Constraint systems of this type require iterative procedures for solution access. Example #1: Ex
Iowa State - ENG - 461
Iowa State University Department of Aerospace Engineering AerE 461Necessary Conditions for Best Solutions Design Optimimization Techniques Formulation: 1. Unconstrained Problem Given the constants vector, c = (c1 , c2 , c3 , c4 ) = (1,3,2,1) Minimiz
Iowa State - ENG - 461
Aerospace Engineering 461 PresentationTopic: _ Date: _ Group Presenting: _ Group Judging: _ Grading: A score of 1-10 is assessed for each of the following criteria: Clearness of Presentation/Organization - Was listener able to follow along with ease
Iowa State - ENG - 446
The Search for Optimal PreconditioningThis document contains information about accelerating convergence of iterative schemes for solving the compressible fluid equations for cases near the incompressible limit of low Mach numbers. The governing equa
Iowa State - ENG - 241
1AerE 241 Final Exam (Closed Book)Part I, MatchingInsert the letter from the item in the right column into the blank provided in the left column for that item which provides the best match. Mark the letters clearly._1. _2. _3. _4. _5. _6. _7. _
Iowa State - ENG - 461
Formulation We seek to optimize the functionalJ = J( c , x, u )(1 )subject to equality constraintshk ( c , x , u ) = 0and inequality constraints wherek = 1,2,3, . . j = 1,2,3, . . m(2)g j ( c , x, u ) 0x = { xi } u = { uk }( 3)i
Iowa State - ENG - 241
1AerE 241 Final Exam (Open Book)Multiple ChoiceCircle the letter of the best single answer to each question.1.At a point in air, the pressure and temperature are measured to be 1.01(10)5 N/m2 and 300oK respectively. What is the density of the
Iowa State - ENG - 461
Formulation We seek to optimize the functionalJ = J( c , x, u )(1 )subject to equality constraintshk ( c , x , u ) = 0and inequality constraints wherek = 1,2,3, . . j = 1,2,3, . . m(2)g j ( c , x, u ) 0x = { xi } u = { uk }( 3)i
Iowa State - ENG - 461
Lift Distribution &amp; Moment at Root for Elliptic Planform WingA typical elliptic planform wing ideally experiences an elliptic load distribution across its span. The lift force per unit span at a particular spanwise station, y, is given by: = V( y )
Iowa State - ENG - 643
t dV + VV V dA = 0 tV T VdV + VV dA = - p dA + dA + gdVV V V V t edV + VV eV dA = - pV dA + V T dA + V gdV V V VdA gT V U = V e U = (U , U ) tU = UdVVU = tt s-V) s dA
Iowa State - ENG - 643
Iowa State - ENG - 643
1618006224006064422460652.542446212.541500000000000000000114641412215885104763975151310512101116461412141174137133313151345121110412558151111
Iowa State - ENG - 643
9248010-1010-100010-1010-101111-1-1-1-11621231671731272732671341781383843781481451891494594891591521961925922961 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Iowa State - ENG - 643
&gt; syms r m Ev&gt; syms 'Ev' 'm' 'r' &gt; U=[r,m,Ev];&gt; U=[r,m,Ev] U = [ r, m, Ev] &gt; syms g &gt; p=(g-1)*(Ev-(1/2)*m^2/r) p = (g-1)*(Ev-1/2*m^2/r) &gt; F=[m;p+m^2/r;(p+Ev)*m/r] F = [
Iowa State - ENG - 461
Barrel Bursting: The maximum combustion chamber pressure must not exceed the critical pressure at which the barrel will burst. The barrel thickness must be defined to assure safety in this area. A factor of safety of 2 is used. The stresses in the ba
Iowa State - ENG - 461
Sphere Buckling: The maximum combustion chamber pressure must not exceed the critical pressure at which the spherical shell will burst. The spherical shell thickness must be defined to assure safety in this area. A factor of safety of 10 is used. The
Iowa State - ENG - 461
theta L thick d2s xdist g(1) g(2) g(3)= 30 400 0.3 1.8 4646.96 -0.008 -2.392 -0.056=Vmuzzle=1772.1 f/stib=0.0202 seconds (time in barrel)NetF=-971.5 lbs (net force at end of barrel)theta=30.000000L=400.00000
Iowa State - ENG - 461
subroutine RK4(zn,t,z,dt)real*8 zn(4),z(4),t,dtreal*8 f1(4),f2(4),f3(4),f4(4)real*8 z1(4),z2(4),z3(4)call RHS(zn,t,f1);z1=zn+0.5*dt*f1;call RHS(z1,t+0.5*dt,f2);z2=zn+0.5*dt*f2;call RHS(z2,t+0.5*dt,f3);z3=zn+dt*f3;call RHS(z3,t+dt,f4);z=zn
Iowa State - ENG - 461
clear all;% unconstrained functionxr=0:.2:3;xa=[xr;xr;xr;xr;xr;xr;xr;xr;xr;xr;xr;xr;xr;xr;xr;xr];ya=xa';J=1+3*(xa-2).^2+(ya-1).^2;f1=figure;view(10,20);surfc(xa,ya,J);f2=figure;contour(xa,ya,J,50);yc=xr.^2;hold on;plot(xr,yc);[f
Iowa State - ENG - 524
program driver implicit real*8 (a-h,o-z) dimension ff(1000),fa(1000),xa(1000) dimension fd(100),x(100),a(100),b(100),c(100),cp(102) pi=3.141592654c-&gt; how many intervals ? NOTE: #points is n+1 n=9c-&gt; what kind o
Iowa State - ENG - 361
Subject:My rocket code.Students wishing to use my rocket code need the following information:Test it with:-program driverreal*8 obj,rocket,xv(12)integer nn=12xv(1)=.xv(2)=...xv(12)=.obj=rocket(xv,n)print*,objend-To test, cr
Iowa State - ENG - 524
subroutine invert(l3,l2,l1,d,u1,u2,u3,r,nl,nu,idim) implicit real*8 (a-h,o-z) real*8 l1,l2,l3integer nl,nu,idim dimension l3(*),l2(*),l1(*),d(*),u1(*),u2(*),u3(*),r(idim,*)c-c-this solver was written by R.G. Hindma
Iowa State - ENG - 461
AIRCRAFT SIZING DESIGN PROBLEM Aer E 461: Design Problem #1, John Doe, 09/23/88 CONSTRAINT BOUNDARIES: (1) W/S = 30 LB/(SQ FT) (2) T/S=8 LB/(SQ FT) (3)
Iowa State - ENG - 361
program mydriverimplicit noneinteger npts,ireal*8 x(100),y(100)character foiltype*(20)print*,'enter foiltype,npts:'read(*,'(a4,i4)')foiltype,nptscall makeafoil(foiltype,npts,x,y)do i=1,npts print*,x(i),y(i)enddoend
Iowa State - ENG - 361
SUBROUTINE DGEFS (A, LDA, N, V, ITASK, IND, WORK, IWORK, RCOND)CC*C*C* *C* Murphy Laboratory - Apollo Network - Math Library *C*
Iowa State - ENG - 361
program analysisimplicit noneinteger .real*8 .*-* stuff for DGEFS(Amat,LMAX,neqns,RHS,1,IND,WORK,IWORK,CN)*-* LMAX &gt;= neqns LMAX is array dimension Amat(LMAX,LMAX)* IND a return error code* WORK a work array used in DGEFS* IWORK a
Iowa State - ENG - 361
program wingimplicit noneinteger .real*8 .*-* construct the wing geometry and MCL geometry and write data* to files in fast format** Input is croot,span,taper,twist,sweep,Nc,Nb*-* get the parameterscall input(.)* get the airfo
Iowa State - ENG - 461
program driverimplicit noneinclude './inc/global.h'real*8 uu(4), uulb(4), uuub(4), ui, uj, nsainteger i, j, k, nvar, ncon, isc(4), ier, w1, w2external analyz! read input datacall input() if (whichway .EQ. 0) then call ou
Iowa State - ENG - 461
c *c *c Sample Main Program to show implementation of the secant subroutinec Main Programc Declare the transcendental function routine (fcn) as externalimplicit real*8 (a-h,o-z) external fcnc Establish the initial marching location (x
Iowa State - ENG - 341
% Semi-colon prevents output of command.% Define a collection of x's &amp; y's.x=0:.05:1;y=0:.05:1;% Define a mesh of points.[xx,yy]=meshgrid(x,y);% Compute the T distribution at each pointzz=some function of xx and yy% Plot surfacesurf(zz);
Iowa State - ENG - 341
Real*4 To,PI,x(21,21),y(21,21),fx(21,21),fy(21,21)Real*4 xx,yy,f(21,21),zInteger j,kTo=0.PI=3.141592654Do k=1,21Do j=1,21 x(j,k)=(j-1)*1.0/(20) xx=x(j,k) y(j,k)=(k-1)*1.0/(20) yy=y(j,k) f(j,k)= your function
Iowa State - ENG - 461
subroutine analyz(uu,ndvv,obj,gcon,nconn)include &quot;./inc/global.h&quot;integer ndvv,nconn,ireal*8 uu(ndvv),gcon(nconn),obj,nsado i=1,ndvvu(i)=ulb(i)+uu(i)*(uub(i)-ulb(i)enddoobj=nsa()do i=1,nconngcon(i)=g(i)enddoreturnend
Iowa State - ENG - 461
function driver()clear all;global u x c ukmin ukmax outputglobal hglobal PI% set the constantsmyinit;% input which control variables &amp; rangesprompt = {'First u Index:','# u values:',. 'Second u Index:','# u values:',. '
Iowa State - ENG - 461
function z = RK4(zn,t)global dtf1=RHS(zn,t);z1=zn+0.5*dt*f1;f2=RHS(z1,t+0.5*dt);z2=zn+0.5*dt*f2;f3=RHS(z2,t+0.5*dt);z3=zn+dt*f3;f4=RHS(z3,t+dt);z=zn+dt*(f1+f4+2*(f2+f3)/6.0;
Iowa State - ENG - 215
Hindman, RichFrom:Longacre, Kenneth D [Kenneth.D.Longacre@USAHQ.UnitedSpaceAlliance.com]Sent:Thursday, September 02, 1999 7:07 PMTo:'Vance, Judy'; 'hindman@iastate.edu'Cc:Brockway, Daniel J; Andrew, Robert L; Longacre, Kenneth DSubject:Recr
Iowa State - ENG - 461
function g=getcons() global u c gg(1)=c(2)+c(1)+c(3)-u(1)-u(2);%slope -1 through center +c(3) in y. gg(2)=u(1)-u(2)+c(4);%slope 1 through y=c(4). g=gg';return;
Iowa State - ENG - 461
function mydriver()clear all;clear global all;global u x c ukmin ukmax% set the constantsmyconstants;% input prompt = {'Input 0|1|2 -&gt; 0=one Nom Soln,1=opt,2=plots:'};title = 'Input For Sample Design Problem';lines= 1;def = {'0'};cell
Iowa State - ENG - 461
function [g geq]=confun(uu) global u c ukmin ukmax u=ukmin+uu.*(ukmax-ukmin); g=getcons; geq=[];return;
Iowa State - ENG - 461
real*8 u(2),ukmin(2),ukmax(2),g(2)real*8 c(4)common/com1/u,ukmin,ukmax,g,c
Iowa State - ENG - 461
function myconstants() global c c=[0,0,1.0,2.0];return;
Iowa State - ENG - 461
subroutine myconstants() include 'global.h' c(1)=0c(2)=0c(3)=1.0c(4)=2.0returnend
Iowa State - ENG - 461
function nsa() include 'global.h'real*8 nsa nsa=(u(1)-c(1)*2+(u(2)-c(2)*2returnend
Iowa State - ENG - 461
function J=nsa(); global u c J=(u(1)-c(1)^1.5+(u(2)-c(2)^1.5;return;
Iowa State - ENG - 461
subroutine getcons(gg) include 'global.h'real*8 gg(2)!slope -1 through center + c(3) in y.gg(1)=c(2)+c(1)+c(3)-u(1)-u(2) !slope 1 through c(4) in y.gg(2)=u(1)-u(2)+c(4)g(1)=gg(1)g(2)=gg(2)end
Iowa State - ENG - 461
function J=objfun(uu) global ukmin ukmax u u=ukmin+uu.*(ukmax-ukmin); J=nsa;return;
Iowa State - ENG - 461
program mydriver!global u x c ukmin ukmaximplicit noneinclude 'global.h'integer whattodo,ni,nj,i,j,k,nvar,ncon,isc(2),iprnt,niter,ierreal*8 uk(2),nsa,ui,uj,perfindx,Jay(11,11),xar(11,11),yar(11,11)real*8 Constraints(11,11,2),u0(2),LB(2),
Iowa State - ENG - 461
subroutine objfun(uu,nuu,J,gg,ngg) include 'global.h'integer i,nuu,nggreal*8 uu(2),J,gg(2)real*8 nsado i=1,2 u(i)=ukmin(i)+uu(i)*(ukmax(i)-ukmin(i)enddo J=nsa()call confun(uu,gg)returnend