COS 226_04 - Acknowledgement COS 226 Chapter 4 Foundations...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Acknowledgement COS 226 Chapter 4 Foundations of Shared Memory Some of the slides are taken from the companion slides for “The Art of Multiprocessor Programming” by Maurice Herlihy & Nir Shavit Church-Turing Thesis Turing Machine Anything that can be computed, can be computed by a Turing Machine. Foundations of sequential computing Finite State Controller Reads and Writes Infinite tape 1 011010 Concurrent shared-memory computing Consists of multiple threads – each a sequential program That communicate by calling methods of objects in shared memory Threads Threads are asynchronous They run at different speeds and can be halted for an unpredictable duration at any time Shared-Memory Computability? Foundations of Shared Memory To understand modern To understand modern multiprocessors we need to ask multiprocessors we need to ask some basic questions … some basic questions … Shared Memory 10011 10011 Mathematical model of concurrent computation What is (and is not) concurrently computable Efficiency (mostly) irrelevant 8 Foundations of Shared Memory To understand modern understand useful What To the weakest modernform of What is the weakestneed to form of is useful multiprocessors we need to ask multiprocessors we ask shared memory? memory? somesharedquestions … basic questions … some basic Foundations of Shared Memory To understand modern understand useful What To the weakest modernform of What is the weakestneed to form of is useful multiprocessors we need to ask multiprocessors can it do? ask What we it do? What can shared memory? memory? somesharedquestions … basic questions … some basic Foundations of Shared Memory To understand modern understand useful What To the weakest modernform of What is the weakestneed to form of is useful multiprocessors we need to ask multiprocessors can it do? ask What we it do? What can it shared memory?do? What memory?… can’t it sharedquestionsdo? What some basic questions … some basic can’t Register Holds a (binary) value * 10011 10011 * A memory location: name is historical Register 10011 Can be read Register Can be written 10011 10011 01100 10011 10011 Registers public interface Register<T> { public T read(); public void write(T v); } Registers public interface Register<T> { public T read(); public void write(T v); } Type of register (usually Boolean or m-bit Integer) Single-Reader/Single-Writer Register Multi-Reader/Single-Writer Register 10011 01100 10011 10011 10011 01100 10011 10011 Multi-Reader/Multi-Writer Register mumble Jargon Watch SRSW Single-reader single-writer 10011 mumble 10011 01010 mumble MRSW Multi-reader single-writer 10011 10011 11011 MRMW Multi-reader multi-writer Different kinds of registers According to: Range of values (Boolean vs Integer etc.) Number of readers and writers Degree of consistency Degree of consistency Safe Regular Atomic Safe Register A single-writer, multi-reader register is safe if: A read() that does not overlap a write() return the last value If a read() overlaps a write() it can return any value within the register’s range Safe Register OK if reads and writes don’t overlap read(1001) write(1001) Safe Register Some valid value if reads and writes do overlap $*&v 1111 Regular register A single-writer, multi-reader register is regular if: A read() that does not overlap a write() returns the last value If a read() overlaps a write() it returns either the old value or the new value write(1001) read(????) 0000 1001 Regular or Not? write(0) read(1) write(1) read(0) Regular or Not? write(0) read(1) write(1) read(0) Overlap: returns new value Regular or Not? write(0) write(1) read(0) Regular or Not? re g write(0) read(1) write(1) ula r read(0) Overlap: returns old value Regular ≠ Linearizable write(0) write(1) read(1) read(0) Regular ≠ Linearizable Atomic register Linearizable implementation of sequential register A single-writer, multi-reader register is atomic if: Each read() returns the last value written Sequential Register write(1001) read(1001) Atomic Register write(1001) write(1010) read(1010) Atomic Register write(1001) write(1010) read(1010) read(1001) read(1010) read(1001) read(1010) Linearizable? Register Space Wait-Free Implementations Definition: An object implementation is Definition: An object implementation is wait-free if every method call completes wait-free if every method call completes in a finite number of steps in a finite number of steps M-valued Boolean MRMW MRSW SRSW Safe No mutual exclusion – Thread could halt in critical section – Build mutual exclusion from registers Regular Atomic Road Map SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Weakest Register Single writer 1 01 1 Single reader Safe Boolean register Register construction We will now build a range of registers from single-reader, single-writer Boolean safe registers Road Map SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Next Register Names public public class SafeBoolMRSWRegister implements Register<Boolean> { public boolean read() { … } public void write(boolean x) { … } } Register Names public class SafeBoolMRSWRegister implements Register<Boolean> { public boolean read() { … } public void write(boolean x) { … } } property type How many readers & writers? Safe Boolean MRSW 0 0 0 0 0 0 0 writer 0 0 zzz 0 0 0 0 0 readers Safe Boolean MRSW 0 0 Let’s Write 1! 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Safe Boolean MRSW 0 or 1 0 1 1 0 0 0 0 0 0 0 0 0 Safe Boolean MRSW 1 0 or 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 Safe Boolean MRSW 1 0 0 0 0 0 0 1 0 1 0 1 1 1 1 1 0 or 1 Safe Boolean MRSW 1 1 Whew! 1 1 1 1 1 1 1 1 1 1 1 1 1 Safe Boolean MRSW public class SafeBoolMRSWRegister implements Register<boolean boolean> Register<boolean> { boolean s_table; //array of SRSW registers boolean s_table; public SafeBoolMRSWRegister(int capacity) { boolean[capacity]; s_table = new boolean[capacity]; Each thread } public boolean read() { has own safe s_table[ThreadID.get()]; return s_table[ThreadID.get()]; SRSW } register public void write(boolean x) { (int s_table.length; for (int i = 0; I < s_table.length; i++) s_table[i] s_table[i] = x; } } Safe Boolean MRSW public class SafeBoolMRSWRegister implements Register<boolean boolean> Register<boolean> { boolean s_table; //array of SRSW registers boolean s_table; public SafeBoolMRSWRegister(int capacity) { boolean[capacity]; s_table = new boolean[capacity]; Write each } thread’s public boolean read() { s_table[ThreadID.get()]; return s_table[ThreadID.get()];register one at } a time public void write(boolean x) { (int s_table.length; for (int i = 0; I < s_table.length; i++) s_table[i] s_table[i] = x; } } Safe Boolean MRSW public class SafeBoolMRSWRegister implements Register<boolean boolean> Register<boolean> { boolean s_table; boolean s_table; //array of SRSW registers public SafeBoolMRSWRegister(int capacity) { boolean[capacity]; s_table = new boolean[capacity]; } Each thread public boolean read() { reads own s_table[ThreadID.get()]; return s_table[ThreadID.get()]; } register public void write(boolean x) { (int s_table.length; for (int i = 0; I < s_table.length; i++) s_table[i] s_table[i] = x; } } Safe Multi-Valued MRSW? 1 0 1 Ye 1011 1011 s, 1000 it w 1000 1011 1011 or ks ! Any value in range 0 1000 1000 1000 1000 Road Map SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Safe BooleanMRSW vs Regular BooleanMRSW Only difference is when newly written value is same as old value: Next Safe register can return either Boolean value Regular register can return either new value or old value – if both new and old is x, then regular can only return x Safe Boolean MRSW Boolean MRSW 0 1 Regular Safe Boolean MRSW Boolean MRSW 0 1 Regular Safe 1 1 0 0 0 1 0 0 0 0 0 1 0 1 Regular? 0 1 Safe Boolean MRSW Boolean MRSW 0 Regular Safe Boolean MRSW Boolean MRSW Regular Last writtern: 0 0 0 0 0 0 0 0 public class RegBoolMRSWRegister implements Register<Boolean> { private boolean old; private SafeBoolMRSWRegister value; public void write(boolean x) { if (old != x) { value.write(x); value.write(x); old = x; }} public boolean read() { value.read(); return value.read(); }} Safe Boolean MRSW Boolean MRSW Regular Safe Boolean MRSW Boolean MRSW Regular public class RegBoolMRSWRegister implements Register<Boolean> { threadLocal boolean old; private SafeBoolMRSWRegister value; public void write(boolean x) { if (old != x) { value.write(x); value.write(x); old = x; Last bit this thread (made-up syntax) }} public boolean read() { value.read(); return value.read(); }} wrote public class RegBoolMRSWRegister implements Register<Boolean> { threadLocal boolean old; private SafeBoolMRSWRegister value; public public void write(boolean x) { if (old != x) { value.write(x); value.write(x); old = x; }} public boolean read() { value.read(); return value.read(); Actual value }} Safe Boolean MRSW Boolean MRSW Regular Safe Boolean MRSW Boolean MRSW Regular public class RegBoolMRSWRegister implements Register<Boolean> { threadLocal boolean old; private SafeBoolMRSWRegister value; public void write(boolean x) { if (old != x) { value.write(x); value.write(x); Is new value different old = x; from last value I wrote? }} public boolean read() { value.read(); return value.read(); }} public class RegBoolMRSWRegister implements Register<Boolean> { threadLocal boolean old; private SafeBoolMRSWRegister value; public void write(boolean x) { if (old != x) { value.write(x); value.write(x); old = x; }} public boolean read() { If so, change it value.read(); return value.read(); }} (otherwise don’t!) Safe Boolean MRSW Boolean MRSW Regular public class RegBoolMRSWRegister implements Register<Boolean>{ threadLocal boolean old; private SafeBoolMRSWRegister value; public void write(boolean x) { if (old != x) { •Overlap? No Overlap? value.write(x); value.write(x); •No problem old = x; •either Boolean value works }} public boolean read() { value.read(); return value.read(); }} Safe Multi-Valued MRSW Regular Multi-Valued MRSW Safe register can return value in range other than old or new when value changes 0101 0101 D 0101 0101 Multi-valued Regular register can return only old or new when value changes! oes n ot wor k ! Road Map SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Regular M-Valued MRSW Register Values are represented using unary notation An M-valued register is implemented as an array of M Regular Boolean registers Initially the register is set to 0 Next Regular M-Valued MRSW Register write(): A write() of value x, writes true to location x – which is a Regular Boolean MRSW register It then sets all the lower locations to false Writing M-Valued Unary representation: bit[i] means value i read(): Reads the locations from lower to higher values until it reaches a value that is true Initially 0 1 00 0 00 0 01234567 Reader Writer Writing M-Valued Write 5 Writing M-Valued Write 5 1 00 00 01234567 Writer Initially 0 000 0 1 1 01234567 Writer Writing M-Valued Write 5 MRSW Regular Boolean Regular M-valued public void write(int x) { this.bit[x].write(true); this.bit[x].write(true); (int i=xi---) for (int i=x-1; i>=0; i--) this.bit[i].write(false); this.bit[i].write(false); } public int read() { (int for (int i=0; i < M; i++) (this.bit[i].read this.bit[i].read()) if (this.bit[i].read()) return i; }} MRSW public class RegMRSWRegister implements Register{ RegBoolMRSWRegister[M] RegBoolMRSWRegister[M] bit; 0 000 0 1 1 5 01234567 Reader Writer MRSW Regular Boolean Regular M-valued RegBoolMRSWRegister[M] RegBoolMRSWRegister[M] bit; public void write(int x) { this.bit[x].write(true); this.bit[x].write(true); (int i=xi---) for (int i=x-1; i>=0; i--) this.bit[i].write(false); this.bit[i].write(false); } Unary MRSW MRSW Regular Boolean Regular M-valued RegBoolMRSWRegister[m] RegBoolMRSWRegister[m] bit; public void write(int x) { this.bit[x].write(true); this.bit[x].write(true); (int i=xi---) for (int i=x-1; i>=0; i--) this.bit[i].write(false); this.bit[i].write(false); } public int read() { (int for (int i=0; i < M; i++) if (this.bit[i].read()) (this.bit[i].read()) this.bit[i].read return i; }} MRSW public class RegMRSWRegister implements Register{ RegMRSWRegisterimplements public class RegMRSWRegisterimplements Register { bit[i] public int read() { (int for (int i=0; i < M; i++) if (this.bit[i].read()) (this.bit[i].read()) this.bit[i].read return i; }} representation: means value i Set bit x MRSW Regular Boolean Regular M-valued RegBoolMRSWRegister[m] RegBoolMRSWRegister[m] bit; public void write(int x) { this.bit[x].write(true); this.bit[x].write(true); (int i=xi---) for (int i=x-1; i>=0; i--) this.bit[i].write(false); this.bit[i].write(false); } public int read() { (int for (int i=0; i < M; i++) (this.bit[i].read this.bit[i].read()) if (this.bit[i].read()) return i; }} MRSW MRSW Regular Boolean Regular M-valued RegBoolMRSWRegister[m] RegBoolMRSWRegister[m] bit; MRSW RegMRSWRegisterimplements public class RegMRSWRegisterimplements Register { RegMRSWRegisterimplements public class RegMRSWRegisterimplements Register { Clear bits from higher to lower public void write(int x) { to this.bit[x].write(true); this.bit[x].write(true); (int i=xi---) for (int i=x-1; i>=0; i--) this.bit[i].write(false); this.bit[i].write(false); } public int read() { (int for (int i=0; i < M; i++) (this.bit[i].read this.bit[i].read()) if (this.bit[i].read()) return i; }} Scan from lower higher & return first bit set Regular Register Conditions Further conditions for a register to be regular: No read() call should return a value from the future No read() call should return a value from the distant past – only the most recently written non-overlapping value must be returned Road Map SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Next Road Map (Slight Detour) SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular SRSW Atomic MRSW atomic MRMW atomic Atomic Register Conditions Together with the conditions for a regular register, an additional condition for an atomic register is: An earlier read() cannot return a value later that that returned by a later read() In other words, values read() should be in the correct order SRSW register Since a SRSW register has no concurrent reads, the only way that the condition for an atomic register can be violated is when two reads that overlap the same write read values out of order SRSW Regular Regular writer Regular reader 15678 5678 234 1234 Instead of 5678… Concurrent Reading When could this happen? SRSW Regular Regular writer SRSW Regular Regular writer 1234 5678 5678 Initially 1234 Reg write(5678) ic tom A 5678 Reg read(5678) Regular reader 15678 5678 234 Regular reader Initially 1234 Reg write(5678) Reg read(1234) o Als mic At o 1234 Instead of 5678… time time SRSW Regular Regular writer Timestamps Solution is to for each value to have an added tag – a timestamp Timestamps are used to order concurrent calls Regular reader 15678 5678 234 1234 Initially 1234 Reg write(5678) Reg read(5678) Reg read(1234) not mic! At o Instead of 5678… Write 5678 time happened Timestamps The writer writes a timestamp to a value Each reader remembers the latest timestamp/value pair ever read If a later read() then returns an earlier value the value is discarded and the reader uses the last value Timestamped Values 1:45 2:00 5678 1234 2:00 5678 Writer writes value and stamp together Reader saves last read (value,stamp) and returns new value only if higher stamp SRSW Regular writer SRSW Atomic Atomic SRSW StampedValue<T> public class StampedValue<T> { public long stamp; public T value; public StampedValue (T init) { stamp = 0; value = init; } (StampedValue public StampedValue max (StampedValue x,StampedValue y) { (x.stamp y.stamp) if (x.stamp > y.stamp) return x; else return y; } } reader 1:45 2:00 1234 5678 1:45 1234 Reg write(2:00 5678) read(2:00 5678) time old = 2:00 read(1:45 1234) s ea Sammic At o 1:45 1234 Less than 2:00 5678 So stick with 5678 Atomic SRSW AtomicSRSWRegister<T> public class AtomicSRSWRegister<T> implements Register<T> { lastStamp; long lastStamp; StampedValue<T> lastRead; StampedValue<T> lastRead; StampedValue<T> StampedValue<T> value; public T read() { StampedValue<T> StampedValue.max(value, StampedValue<T> result = StampedValue.max(value, lastRead); lastRead); lastRead = result; result.value; return result.value; } public void write(T v) { long stamp = lastStamp + 1; StampedValue(stamp, value = new StampedValue(stamp, v); lastStamp = stamp; } } > 5678 Atomic SRSW Atomic MRSW Can the atomic SRSW be used to built an atomic MRSW? Solution of Safe MRSW Registers: Every thread in array Write starts at the beginning of the array and iterates through array Read reads only its own array location Atomic Single Reader Multi-Reader stamp 1:45 1:45 1:45 1:45 value 1234 1234 1234 1234 Atomic Atomic MRSW Writer starts write… stamp 2:00 value 5678 1234 1234 1234 Atomic SRSW One per reader 1:45 1:45 1:45 Atomic SRSW zzz… stamp 2:00 1:45 1:45 1:45 value 5678 1234 1234 1234 Atomic MRSW 2:00, 5678 reader reads later reader Atomic MRSW We address this problem by having earlier reader threads help out later threads, by telling them which value they read 1:45 1234 Yellow was completely after Blue but read earlier value…not linearizable! Atomic MRSW Write 4 n-threads share a n-by-n array of stamped values Read() calls determine latest threads by timestamps Similar to the Safe MRSW Register implementation, the writer writes the new values to the array, but only on the diagonals 4 4 4 4 zzz… Can’t Yellow Miss Blue’s Update? … Only if Readers Overlap… Reader 2 4 4 4 4 If not then the current value 1:45 1234 write(2:00 5678) read(2:00 5678) read(1:45 1234) Is there a value in this is column with the latest a higher timestamp? In which case time to read its OK 1234 Bad Case Only When Readers Don’t Overlap 1:45 1234 write(2:00 5678) read(2:00 5678) Road Map SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic In which case Blue will complete writing 2:00 5678 to its column read(2:00 5678) time Next Multi-Writer Atomic From MultiRead Reader Atomic Writer 1 Write 5678 stamp value 1234 1234 5678 1234 XYZW 1234 Readers read all and take max (Lexicographic like Bakery) Each writer reads all then writes Max+1 to its register 1:45 1:45 2:00 1:45 2:15 1:45 Writer 3 Write XYZW Max is 2:15, return XYZW ...
View Full Document

Ask a homework question - tutors are online