6.338_Project_Report_Runmi&Lu

Unprleteecsrslsadpromceue fntoaallieonet

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: omputing framework in Julia. To illustrate the implementation of the framework using Julia, the core part of the Julia code for parallel counting of single items is shown below. A similar method is applied to counting item pairs with more complicated details in the map and reduce functions. #Slt nu tigit pos)eulszdcuk pisiptsrn nonrc( qa­ie hns #(atoerud p,ad@pwswrcut)frec hn o ls n onsu) n san odon( o ahcukt #rni aall hnfth) eut n efrswrdc(. u nprle.Te ec(srslsadprom ceue) fnto aallie_on(et ucinprle_tmcuttx) lnsslttx,\'fle ie=pi(et'n,as) n=pos) pnrc( uisz=ellnt(ie)n) ntieci(eghlns/p ie_ons{ tmcut=} res{ rf=} #sanpos pw rc frii :p o n1n frtuisz*i1+ is=ntie(­)1 ls=ntiei atuisz* i atlnt(ie) fls>eghlns ls=eghlns atlnt(ie) ed n sbetji(ie[n(is)itls),\" utx=onlnsitfrt:n(at]"n) ps!res sanmpcut utx uh(rf,@pw a_on(sbet)) ed n #fthrsls ec eut wielnt(rf)0 hl eghres> ps!ie_onsfthpp(rf)) uh(tmcut,ec(o!res) ed n #rdc eue cutrdc_on(tmcut) on=euecutie_ons rtr on euncut ed n #"a"fnto. Mp ucin #Tksasrn.RtrsaMpwt h ubro ahie. ae tig eun a ihtenme fec tm fnto a_on(et ucinmpcuttx) iesslttx,'''n,\''''''''''''''''') tm=pi(et[ ,\''t,­,.,,,:,_,",;,!] cut=itACItig n6}) onsDc{SISrn,It4( frie nies o tmi tm cut[tm=e(onsie,)1 onsie]gtcut,tm0+ ed n rtr ons euncut ed n #"eue ucin Rdc"fnto. #Tksacleto fMpi h omtrtre ympcut) ae olcino a ntefra eundb a_on( # R t r s a M p i w i h w r s t a p e r i u t p e i p t a e eun a n hc od ht apa n mlil nus hv terttl de oehr hi oasaddtgte. fnto euecutie_ons ucinrdc_on(tmcut) rdcdcut itACItig n6}) eue_ons=Dc{SISrn,It4( frie_on nie_ons o tmcuti tmcut fr(tmcut nie_on o ie,on)i tmcut rdcdcut[tm e(eue_onsie,)cut eue_onsie]=gtrdcdcut,tm0+on ed n ed n rtr eue_ons eunrdcdcut ed n IV. Performance The method was tested using a sampled dataset with 999,002 transaction records and 41,270 distinct items. The test machine has a 2­core 1.7GHz Intel Core i5 processor and 4GB memory. The naive method requires more than 6 GB memory. Therefore, it is not possible to apply it with the test machine. The memory usage with and without hashing and the running time with differen...
View Full Document

Ask a homework question - tutors are online