Parametric Methods

Parametric Methods -...

Info iconThis preview shows pages 1–14. Sign up to view the full content.

View Full Document Right Arrow Icon
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 4
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 6
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 8
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 10
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 12
Background image of page 13

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 14
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: """'“‘“"‘__"""H—-_'—“w_—'—1 m SCI N302. DE mi [1. m PARAmt—szL mum»: 1 CS W“ a 2‘: SD -srqr Log have leaer LION “’13 make WW 3 we. know 4&1: Tmoba‘bfiisfic. S‘IYudwe (3+7 3 Fmblm (gal/63,. Classificah‘m {when LU?— hfio‘“ FCC” and P(X\Ci)) , Now] we. see. Haw +0 {Es-Hm 'H'xese 13MBabnR'I'CS Siam ca ‘H‘Qifiinfl 5:64—_ 4o cs-Hrvnk. a JUST-{3 ‘. gen-anal Come a? eerd'iIla Pa) when we, Wane. ScnnF‘e) CLmUJn "Th6 9.34413» +0 am. such canncer in ciqssi-FiCchr-I'm '. where ‘ani _ esfimm-l-cd Pmbgbhfifics are View PmbobJQKE FICA crud clasa_wwclflimal ’FmbqanQ‘leq 136(le 3} wa "‘13 x436 Sudn figfima-Hm Hlf\ PfirESSIOfi '. bah-arc "TN Ederno‘kfi‘ @593 LS In The. 'Fcf‘mcMC; l we, Q55;qu 'Hnarl- s-a-nPle) one: drawn {hum Sm (cvjvncarm13/ which .13 ale—Fined b3 (1 Sei- 0? 13m+crs (M and 5. 1cm normal «bsfibufiaf) We Mm lwn how +0 efifimq—k Mae. Tam/swabs aim we —Immm3 San-Piss. Thu m Awe main oPProcxcheA: I) Max'IMLm kILeJnnood EsHmorHOr} CM qufesim ES'HmcxHOn In ‘H‘VIS QPPmod/‘Iwe Iassm UQ‘UM o—F 'Parvm-icr's are Kev—d ' 545+ D Coach/g of H indefadw ma idmficml‘j Ajfifirbmtd (Nd) MP)“ 1 D: §Xi2r;-, - Here. we (455% 450.1- xi m I; (y; le-\ and we LU ' , N awahood _‘ we Hal) .. D 9' _—_. ( ‘ 6" Since. XI S PC | 3 ‘ TD XI] 3 =l 8 +|na+ mafimimu ’nua lhlihood \n 4W1: wPFirocu‘Jn/m ‘wmi +0 9'3 0-? 39m (Lomch is aqua +0 Mox‘mFfi/xg 7, P(b18\ .(v. IzszJiPA+ atria-(H'le { l n... 4. 34?: E? i" J“ M Iztgfiafiog Pcma\:a05 AFT-Page) = “21103 13(“33 .lzi Norrm‘ (emssaog $‘5M73ur‘r4m Lafls (msider 11v. miumo—tc Cow. —F.‘r5+1 Am "Tha’r we. '00“?- WP'“ N D=$x;§‘_' CIfOA-Un TIA-e hormcl dgshxairfim hum NW 14 md UGY'ich-E 37' N am: 203 Tumbler) : 1203 H 136g- Ia) whee 9 i, a.me ii] I wed-Dr Maid as N 9: 9‘ coma BIT'H I Z 1&3 P0913) [6‘] 92.: 2" L." I:l 9.9“; : _.E_ 1303 (1“913- Z (Ki—e" 1 23; 51(8) ‘— # Z ‘3. (x,-—91)(-—1) # 2 (x‘_el\ _ 2%; _NB. E 91 292 82 E9'2. 91 mm M m P+ cm be M {,3 SM 591 b 2(9) — O : Né he" ._..-- )(1__ A I A . #3 A 1 =0 5;: I 94 = A1 2 gal » ar ’F‘Y CL- AAmmsiml 3.13419 1 31 211 v - u 21d 2 '=- all 33.1 _ , 31a] . 1 ‘~ 1 3th 841,. ’ . 2d A ,2- N 7. N A OWE Loan-i +5 awoklmc W cglMIQ-y o-F 0"} l‘fi ""0 meme Q's- ]oias /uo\ni<;h is de—Finad as '93: An csfimcritr i5 Called mbiqsed if 53:0 :8) A 2 Fear cmei-e, vb! L5 Ci" mloimed cs-Hmod'or Sal/1C3- LG" 5 N :E[2x'1 GLACIZL+ 21:11 N N N z E fir} ._ 242/3, + flfiflAl] 5 Smce ix; —_-.:f N )1 N a. 41 = E 53‘, _ M ] N = 2 Efo'] __ ELI-Cid] firmly? ._ N Var (fl: Em“? 439”, ECY“ ~.-. V (‘04- ELY?" = 2 (vorcxmmmfi _ (Wanna; 3 °’ N 1 1 A “ :(g +14} (WCHn4‘1‘\ __.——‘,5mce Vor (x;\ 53"" N E Eula-Hal ‘ 1— A A]_ (ucmfifi = NEH Mg __\[of(440__x.11' 59“ "1'3:qu V N = '31_Var(30 MPH-it \foq- («CI\.=VM' VOW 2 Since var :a'zwrr (X) Na. '2 vou- (Kt-3 _ ___.e Since. in jaw-r0] \‘ar‘ ( i! a: ZVGF(XD+ I #3 Since we assm +ha+ xi'5areillc CW b‘ifi‘j) = 0 51“ i421 4M5 var [ZXD = ivar (Xi) II in man \eQr-nin blB’T‘IS with tetra: Erma-fin? usximF’tc'HCQIS ' cs‘I-imcmys ore accr—fhble ‘3 E: E: ‘2'" 'E'. “Sq .. Barman!“ WM H— is a dune-k, cLisJ-nrbu-Hm ham/'5 hut” 130350016 Dmm‘as (00“) SuPPogc ‘an-I- we, heme Cm mium‘quc clam wfih P?x=1?rgq Z( Ky . = “(_ )H‘i ;€?0,1Ff 1) Rte}: (‘40 1:: ( .493 cf 1 c] / x N Gium ‘Hu, "h‘Ctiniflfl Fe-I- b:€Xi;i=l w'l‘mt are drawn --{3rom ‘Ufiis dis-Hime leg-Hde a u5inj maximum llhlnflmd €$Hmafi°q . M P (9352(‘295103 P('b|c13= £03 136910)) I:l I=| lfixi “ #21303 [fix' (“‘13 ] 7% n-m . A R a [3 ‘H'mL {GHQ 01c m fiflxL-‘flq =0 2‘ = 2X' lnw 0+ OCCWFW" q (141) N ofm WW 41:» "IL:— I; - ~_ . . . “erase Repecri- 'Hvus es+1mcrhm . TIN-e mu H—iUDn'oA-c Case when: - 1 ‘b ' (1"xi ) P (x: “X'D: Ci: (“‘53 J j “in e $99.? N04: ‘H'Ia-I 0C” is 'H'Ié Pmbabfiaj value FE} “Hue 3-“ We . ~+w<za ‘6— 13x5 is 'H'xe 31h Mme of Th5 ‘I‘H‘ S‘G‘flP‘fi/figcmfli“ - q “\cufxw-CLL t 33 X3 M0115 W 3114 'cho‘iwc of X mNaam{ I I. Llwa have ’nfi-e 410.1018 S-e-I' D: §Xiziz| 5 are 11d ON“ 509‘" Xi has :34“?me . E3; .:_ ‘ 7f:- '-=__\I_= __.é’__ GB In maximwn lfladihmd fighmafim/wa assume. Jth-I- ‘Hm: {arm o{‘ +52 dmbfl'j $38193 1's Lnouun and ?qrwne+05 G- are. fixed (bu-P unit/104M) 15 also knew—3n In Ban—fesicn esHma—Hm/ we wiH assume 4406* 1:2fo83 bu+ [30.(mw5 e} are. nc=+ Paced (we hang mcer+mnme537rkus um Consider 9' +0 Er; cm (‘de \Iadab\€._ We edge assume -an+ our {.0in knowledgc abou+ 8' is cmd'oincd I'm a known Prior densH-g P(e—) -- P[B% reflec+s out” knomkaclgg (3.190er @- befiyg looking CH: ‘Hme. smapies. For cxcwpit I we mag he +o|ci 43’151’1' 9' 1‘5 and Loan ‘30‘70 confidence. 9- Hes Oppmxu'mah’tj normal) 7, _ Sc: we maLf (29180!" Bataan <3 and Biggimmh‘mnj around our Inn-{ax in WON) GAP 'Friof' “Shauna Pr§-1.6q < Eggfl. <1.eq§=o.g Prflfiflmq’a <e-<~[+i-6HEZL:0.=1 —_:_-7 751.345 0.55m "HWJr PCB)~V\F(¥,L‘T) N aboufl— 8' ['5 Wind if“) D:— $Xl‘qflli we 3.1+ 11w. POSnLCfl'J dansfiy 0F 8. P(em\_ PC5193 (EH _ PCD183prfl , ‘ 3 # ("OX ' ' , i l I P [P (DIG) Pfe'yaler “this Wageliofg- : (X P (meNFLB—j So I N: mold The. {15+ of W [Chomkaclae X Fifi-F’L‘fla') {aw-Since Ki. {5' be Judge/ded- From 44m: given samples D=$X;?[, M; W +0 csh‘mcrH-t fr.“ Ausflb ca )4 PCxlD\: fP(er‘rlD-\AE}- : [F(x1g,b) Pmm Ae- : (X IQ} 13(B'ID} d 9' *fl—aj'uqca Jrke SdEC-‘Hms 01C 5‘ ' and 'HM. Avainlnj SOVIP‘Iez are done indqmdexflg P (X18313) «r. 1:: (KM?) E"? f3." "5 ‘ :4- — Soma-HMCS evaluafinj ‘Hna in-kfif-cals may be qu'hLE cli'gfiml-L kc we Cm ussmnc 4M4- P (8' ID 3 raga-Rs a S.th Peal; 54+ e:§ I we Con use m WE‘L'IMWH a Pciberiori (MAP) asfima‘l'of' 41: make Arm CQ'ICH'lCH-l'm {03,113 E} = mama! PfEHD\ - e :) Pfxltfl: I: {X .. I‘F we have no Prior reason 4-1:: Ffiauor Same values 04: G ,‘Hfle Pos-kn'or hm F’rho. Scrne .F‘orm as 'erg IfLeh'lnooCI end/Wus/using 'Hne Maximum a Fork/fled €3+imcx+or i3 eciu'iuodner} TD USi/E Time maximww method (mo esh'mcmy m " Bayesian {fifimafim 'E/ W‘Nm‘q‘k “WWI (flausn‘m) Cede H Sumatra: we have Sarhples D:§X}3‘i:l whare K"N\M(H,31) blink/10w“ Censid-Er "Hm C00“: Where M 1-3 The W115 Fame-Hf _ For This case} Suppote 'an+ we. have Prior- fn-FumaHm Qb0u+ x14 and Thaw we. {.xpress ~Hqis in-Fbrma-Hm loj 91 Prior cite/1591,, Pf/fi)NW(‘-10J3:] [Hm we. 055m «Hm M is nomHj awn-bum: mm, Mo and 1301] Mo rcpresm DUI-r 1985* Sui-‘5 FF)” H and 3.} measures Our mca-‘minfl'j 050w}— "I'lnis 3q655_[4-}cr¢ we no?” 053m finer} M0 “301, 3 are known] Omnim is 41> Cmnpufi Pacl'bW: {P(x|9\P(9-}D§d3- than: hm, hm: “HM: mknOWfi me Pa 113'}: (1%“umme Le+'$ (10mm P (A4 It): ‘Firzfi Pm m: :7( PM) Pug-1A4) . 1. " d 4 EXP {—J. (“bf-4‘40) {QTY ‘ 3c. 7" 801 Afler Q Cow‘fie. mmiPuia‘Hoqs 1—H“; wi” 5N8 0‘0”th GuuSSI'ap 5;: ;'.' is: i #3 ~— W 3n 2;- 2. _ -2. “We Mn: No L in + 3% at, H83+31 N Nana-31 31 ‘ 30-121 n - NBS-+21 L€+I5 Cnmpuk P noun P(Kib\: (>4qu 130.4 lD\cl/‘—/[ cf 1 [x «4.0 Mam—3 L {afi‘a '3 2 " t __1_ (bf-H) ng'fl'lgn KP ’1 TL;- [‘1 D4? mm'ifiu'tcx-Hms firms will 3m mo‘fiaer Gaussian. AVE" 0 qu‘C pmm N mr mm {an an I , Khmuxn variance 21 is inCreflx’MZd 2“ I 1:; fide man Considercct m 11» 05¢.11m; l m ‘ +me rhea: UncerJi-Qinnfl :n x (eSuH—mj .Frmfl ow“ Incl; 01c exam-i know'ledfyz 01° my \ “Sm-Pi CcrHon ! SO fiarlwe, have horned +0 esh'mcdc FCx) .Hm 3mm samples, In CIQSS'I‘Cl'cq—Hon (“the egfima‘l‘ccl 01.211531?) are _ P (X \C-iE ._ P (Ci) N IF we, ham 0 Se+ of Sample; I): $xizrffl Ffiy C'clasSes / we. can sew—(3+: +hi5 542+ 55 do.“ In“, C same”! D: $D'UH UDC? SuPPose +ha+ we hm'fi unknown me vac-hats 811.49g QSSOCWKd uo'sfln each Class. . If we Cm (2.55m ‘Hna+ scran in D; give no {A'FGMQ'HIDQ abut-f 93 3f: 1.4:: (we can mm H mus-1 of+¥u~Hm§ ,‘M! (M uum’k mfi'h 2,0.ch dogs {glam-lad . EWER MuH'lvarimLc nomfl abs-WOOU'HM. ® In "freuioug CLQSSES I we Lurbk An”: 44‘5MMMm‘1F «EA/104100 class Cia! 9mm : \03 pouch + \oj N00 mm = 4519 'm - Aflos‘és l-g— (X—M:5*i.7'(x-M.-§+Igg m»; {we fielec—i' TL: aims wf’rh "Hat maximum cliscriminerfl- [Prevmuslj we assumed “Hag-E- MI-J 21- and Then ~€)F a Per-Hauler Sample. PCCO at known]_Now we know how—11) cs—h'mq-J-e My; “gluflfi dorm; (-F.>r emP}; using maximum, lxhllmocd G in "the. mos-f Sml Case (fiizwba‘fan-Ifcgye 3)} C§Hmak M; and 2; us‘mfi 'an Sm?1es ‘m D; (Leuusing “Hm. SomPlea th+ belonBJ-o Claus Ci) CHBchar'oma-krs 2 d + C. d whert C:_H__°Fclcnw Luna—5h.— JR! ms 9. Mm @44‘ “m “3"- .n’“ Sm (budget mama: {or a.“ Cla‘h-C’J (2:2 {cane 13 , esfimn-l-e 44; using m 92um in I); usiIB all {on’olea {regardlequ crf‘ 71—: claim @ Check Problem 3 in Dada,HD/-\',S'15Q '1 boot asfimorln'orL 40 _ i {ea-£5500 l ® In feareSSI'm/ma m+ +0 «.0an T14. numeric owl—12m- oc. cz—fiA/ICHM of“: infxfl r:-F(x\+ e INE (1&st (“4 Du‘l’r‘a‘l‘r'ifi 71-1— 5% D'F a. daicminish‘c W-Hm of- [IL finfuf 46G and {MOM Misc 6. Ni! 61150 “infra-M mine 9 I) normale chme ,e NW (0 ,31) Her?— we. wm+ ‘h: as-Hmak. ~F(x\ £14? Bade} 1L5 “'3 Erhlmaw PG- lx} N N(3 Ode-3,21) cncl Ida/1ng 443Mbe (‘ln’Pu‘lj I ‘ N I On unknown 30“fo Pmbqbnny $9,693 P (XIV) D: ?X; frifi'Ii'] N 533% 12 (a) = £03 F P (er-fl —; Q95 (rilxfi. M 27635 133750;} + £03 11>(X33 ‘:I W—W__ kt; dishku-I'Cd - mfi; |+ does and W on 1 C—‘afinmfior RIG) W(3(X19-);2 ) 4M3 we c515de H. 2 £3 4 ex (_ [Vi—3699511)] izl I: #231 N N ,L -_—_ 2403'” ‘1 + Z_Er|_3(x,IB-\] {:1W% {5' 231' fi' ices n04“ chin/ml on mrcxfimd-nr H ._—. _ i Z fr;_3(x.-eu"- 281 l:l Palm Cm R E iii Ana @ which is equiuaiezfi +13 mlnimi‘tinj N 2 €133 —_—_' “1...... Z fr;_j(x‘,lB-§] éw—a- [605+ Square Error—{which 9-— i=l 1‘5 “PM. morf men-I ermr 11.4ch UNNARMTE— CASE WW 8' lfl ijression) UJE LION-e. C1 “W model r-For 3 403 W mdwo 9 (X ‘W1IN33: N1X+N° I when: we haual’fiuo form 1 N '2. E (w:wa -_- ._L_ :(r;_w,‘x;_wo\ 9- izl N. -___E —'{—'9' Z (r'nw1x'fiwflw(_1j“ -" 2P1"? W1ZX1+ NW0 BN0 '1 [:2] be ” l = 4__.2 Z (r;._w1xi..waj(_x;): _Zr.l.xl. +N4ix', +Wo Z5“. 5W1 2 izl To minimi’tt ‘HV- ‘e-"o‘r‘ / BE :0 final XE :0 (maximiK “Hm; ‘mz'lmood3 BN0 3N1 fi> _ 211+ W‘ 2X.'+M$Mo:0 7. —)é Na haue 4w; aqua-Hm) and +m unknown) .So/Lue cm Solve mm aqua-Hon: -For we, and mi Po‘jnomia‘ fgressim @ 1; ‘L folwmw.h..,wo=wkx + +wllx +wix+wo E(Wo,w‘,-. . wk): _L# 2 (r-l_ ka ~.. ._wlx —\M,X— D To minimfit m {rmr [maxi-MM ’nM. \Ilzlnnoocl3 1 ii... =. 0 PM; ‘ This will Blue. L100 (1+0 equal—EMS. Since. Hog have (Le-0 unknowns (wolwh . . wk) I you {On -F“nd 114.53 unknowns Sakai/9 [flu- mmfians. mu LTlVfi'filfiTE CASE "‘ WW’F '7' 5 {ind—300 0? Several oer—labia; (LFCDWCS\ x31!“ xid and none 6 . «Aer-ado) M 3m push/c of. [114 N“ EMPLL> ‘3 r}: 5(X,|wo,. ,wd\+é 9(KJW9; Nd» _—.. deid+ —- +w1x11+wlx|1+w° u 1. E(w°' lwdsg ._.L—-— : _ "WLKIZ-Wfllfl W33 |:'l To minibus. mi error , :8 =0 _ Tws uu’fll glue qou (d+13 equaHM). w J Some Mn ’03 #‘md 'HM; fwnfit/S (Wafl. Wet) N046. ~Hfio+ muI—Huuriai-C linear rcjressl’m ma WWW-QR POIHflomel’ reflmfigim are. Coniidercd 4-0 be 44%; Some. 13mle we dufine X-H .—x‘ lvvisxil H,..x‘.d=xi"-. Sxmfleflj ,qw Co» uu—e muHium‘q-J-c olHnomial "_ I I l' 1 l. - =x‘ flit =x1, 141:.) Erassim (43(— {xmfle I d—z-Finmj xIJ Ha» +0 um: amaswvoliold-{m +0 Sa‘cc-l— Models 2-351- v.54 " % ALE.- PROBLEMS 141 [a] Suppose that :1 samples I) 2 {sh ...‘_r,,} are drawn independently ac- cording to pLIIO). Show that the maximum—likelihood estimate for 6 is inaxl'DJ—that is, the value of the maximum element in 'D. (b) Suppose that n = 5 points are drawn from the distribution and the maximum value of which happens to be trips xi : 0.6. Plot the likelihood p(’Dl6‘) in the range 0 5 19 5 1. Explain in words why you do not need to know the values of the other four points. . Maximum—likelihood methods apply to estimates of prior probabilities as well. Let samples be drawn by successive, independent selections of a state of nature at,- with unknown probability P(w,-). Let zit = l ifthe state of nature for the kth sample is a); and z”" = 0 otherwise. (a) Show that Ptzii, - - - . Zinlplwill = n P(w:)z”‘(l w thtDl—Z‘L k=1 (1)} Show that the maximum—likelihood estimate for P(w,-) is A l n E E szk‘ - il:=l Interpret your result in words. 4. Let x be a d-dimensional binary (0 or 1) vector with a multivariate Bernoulli distribution d chle} : Hero — are, $21 where 6 = (6], . . . . 85)’ is an unknown parameter vector, 6,- being the probabil- ity that x; = 1. Show that the maximum—likelihood estimate for 0 is A 1" flzggxk. 5. Let each component x,- of x be binary valued (0 or 1) in a two-category problem with Paul) : P((U-2) = 0.5. Suppose that the probability ofobtaining a 1 in any component is Pt! = P Pt2=l—P, and we assume for definiteness p 3» 1 {2. The probability of error is knowu to approach zero as the dimensionality d approaches infinity. This problem asks you to explore the behavior as we increase the number of features in a single _ sample—a complementary situation. (a) Suppose that a single sample x = (x1, . . . , xd)‘ is drawn from category to}. Show that the maximum-likelihood estimate for p is given by ...
View Full Document

This note was uploaded on 04/02/2008 for the course CS 464 taught by Professor Demir during the Spring '08 term at Bilkent University.

Page1 / 14

Parametric Methods -...

This preview shows document pages 1 - 14. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online