localmr - #| Three steps of map reduce: 1. apply mapper to...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
#| Three steps of map reduce: 1. apply mapper to each kv pair in input. . 2. sort them into buckets (sort-into-buckets) 3. run a reduce on each bucket (groupreduce) This version internally operates on association lists, and converts to and from stream as necesary to match the real mapreduce. NOTE: use this file for testing locally to weed out trivial errors. You will not be able to answer parallelization questions just by using this code. USAGE: (lmapreduce <mapper> <reducer> <base-case> <input>) <input> is either a stream of key-value pairs, or: "/sample-emails" - a bunch of emails keyed by a single key, sample-emails "/beatles-songs" - beatles song titles keyed by album These are NOT the same as what the real mapreduce uses (obviously, much smaller) |# ;; kv-pair abstraction (define make-kv-pair cons) (define kv-key car) (define kv-value cdr) ;; Load data (load "~cs61a/lib/localmr-beatles.scm") (load "~cs61a/lib/localmr-emails.scm") ;; lmapreduce is like mapreduce, but local. So far can only handle input:
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 11/30/2010 for the course EECS 21281 taught by Professor Harvey during the Spring '10 term at University of California, Berkeley.

Page1 / 2

localmr - #| Three steps of map reduce: 1. apply mapper to...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online