This preview shows page 1. Sign up to view the full content.
Unformatted text preview: Acknowledgement COS 226
Chapter 4 Foundations of Shared Memory Some of the slides are taken from the companion slides for “The Art of Multiprocessor Programming” by Maurice Herlihy & Nir Shavit ChurchTuring Thesis Turing Machine Anything that can be computed, can be computed by a Turing Machine. Foundations of sequential computing Finite State Controller Reads and Writes Infinite tape 1 011010 Concurrent sharedmemory computing
Consists of multiple threads – each a sequential program That communicate by calling methods of objects in shared memory Threads
Threads are asynchronous
They run at different speeds and can be halted for an unpredictable duration at any time SharedMemory Computability? Foundations of Shared Memory
To understand modern To understand modern multiprocessors we need to ask multiprocessors we need to ask some basic questions … some basic questions … Shared Memory 10011 10011 Mathematical model of concurrent computation What is (and is not) concurrently computable Efficiency (mostly) irrelevant 8 Foundations of Shared Memory
To understand modern understand useful What To the weakest modernform of What is the weakestneed to form of is useful multiprocessors we need to ask multiprocessors we ask shared memory? memory? somesharedquestions … basic questions … some basic Foundations of Shared Memory
To understand modern understand useful What To the weakest modernform of What is the weakestneed to form of is useful multiprocessors we need to ask multiprocessors can it do? ask What we it do? What can shared memory? memory? somesharedquestions … basic questions … some basic Foundations of Shared Memory
To understand modern understand useful What To the weakest modernform of What is the weakestneed to form of is useful multiprocessors we need to ask multiprocessors can it do? ask What we it do? What can it shared memory?do? What memory?… can’t it sharedquestionsdo? What some basic questions … some basic can’t Register
Holds a (binary) value * 10011 10011
* A memory location: name is historical Register 10011
Can be read Register
Can be written 10011 10011 01100 10011 10011 Registers
public interface Register<T> { public T read(); public void write(T v); } Registers
public interface Register<T> { public T read(); public void write(T v); } Type of register (usually Boolean or mbit Integer) SingleReader/SingleWriter Register MultiReader/SingleWriter Register 10011 01100 10011 10011 10011 01100 10011 10011 MultiReader/MultiWriter Register mumble Jargon Watch
SRSW
Singlereader singlewriter 10011
mumble 10011 01010
mumble MRSW
Multireader singlewriter 10011 10011 11011 MRMW
Multireader multiwriter Different kinds of registers
According to:
Range of values (Boolean vs Integer etc.) Number of readers and writers Degree of consistency Degree of consistency
Safe Regular Atomic Safe Register
A singlewriter, multireader register is safe if:
A read() that does not overlap a write() return the last value If a read() overlaps a write() it can return any value within the register’s range Safe Register
OK if reads and writes don’t overlap
read(1001) write(1001) Safe Register
Some valid value if reads and writes do overlap
$*&v 1111 Regular register
A singlewriter, multireader register is regular if:
A read() that does not overlap a write() returns the last value If a read() overlaps a write() it returns either the old value or the new value write(1001) read(????) 0000 1001 Regular or Not?
write(0) read(1) write(1) read(0) Regular or Not?
write(0) read(1) write(1) read(0) Overlap: returns new value Regular or Not?
write(0) write(1) read(0) Regular or Not?
re g
write(0) read(1) write(1) ula r read(0) Overlap: returns old value Regular ≠ Linearizable
write(0) write(1) read(1) read(0) Regular ≠ Linearizable Atomic register
Linearizable implementation of sequential register A singlewriter, multireader register is atomic if:
Each read() returns the last value written Sequential Register write(1001) read(1001) Atomic Register
write(1001) write(1010) read(1010) Atomic Register
write(1001) write(1010) read(1010) read(1001) read(1010) read(1001) read(1010) Linearizable? Register Space WaitFree Implementations
Definition: An object implementation is Definition: An object implementation is waitfree if every method call completes waitfree if every method call completes in a finite number of steps in a finite number of steps
Mvalued Boolean MRMW MRSW SRSW Safe No mutual exclusion
– Thread could halt in critical section – Build mutual exclusion from registers Regular Atomic Road Map
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Weakest Register
Single writer 1 01 1 Single reader Safe Boolean register Register construction
We will now build a range of registers from singlereader, singlewriter Boolean safe registers Road Map
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Next Register Names
public public class SafeBoolMRSWRegister implements Register<Boolean> { public boolean read() { … } public void write(boolean x) { … } } Register Names
public class SafeBoolMRSWRegister implements Register<Boolean> { public boolean read() { … } public void write(boolean x) { … } } property type How many readers & writers? Safe Boolean MRSW
0 0 0 0 0 0 0 writer 0 0 zzz 0 0 0 0 0 readers Safe Boolean MRSW
0 0
Let’s Write 1! 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Safe Boolean MRSW
0 or 1 0 1 1 0 0 0 0 0 0 0 0 0 Safe Boolean MRSW
1 0 or 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 Safe Boolean MRSW
1 0 0 0 0 0 0 1 0 1 0 1 1 1 1 1 0 or 1 Safe Boolean MRSW
1 1 Whew! 1 1 1 1 1 1 1 1 1 1 1 1 1 Safe Boolean MRSW
public class SafeBoolMRSWRegister implements Register<boolean boolean> Register<boolean> { boolean s_table; //array of SRSW registers boolean s_table; public SafeBoolMRSWRegister(int capacity) { boolean[capacity]; s_table = new boolean[capacity]; Each thread } public boolean read() { has own safe s_table[ThreadID.get()]; return s_table[ThreadID.get()]; SRSW } register public void write(boolean x) { (int s_table.length; for (int i = 0; I < s_table.length; i++) s_table[i] s_table[i] = x; } } Safe Boolean MRSW
public class SafeBoolMRSWRegister implements Register<boolean boolean> Register<boolean> { boolean s_table; //array of SRSW registers boolean s_table; public SafeBoolMRSWRegister(int capacity) { boolean[capacity]; s_table = new boolean[capacity]; Write each } thread’s public boolean read() { s_table[ThreadID.get()]; return s_table[ThreadID.get()];register one at } a time public void write(boolean x) { (int s_table.length; for (int i = 0; I < s_table.length; i++) s_table[i] s_table[i] = x; } } Safe Boolean MRSW
public class SafeBoolMRSWRegister implements Register<boolean boolean> Register<boolean> { boolean s_table; boolean s_table; //array of SRSW registers public SafeBoolMRSWRegister(int capacity) { boolean[capacity]; s_table = new boolean[capacity]; } Each thread public boolean read() { reads own s_table[ThreadID.get()]; return s_table[ThreadID.get()]; } register public void write(boolean x) { (int s_table.length; for (int i = 0; I < s_table.length; i++) s_table[i] s_table[i] = x; } } Safe MultiValued MRSW?
1 0 1 Ye 1011 1011 s, 1000 it w 1000 1011 1011 or ks ! Any value in range 0
1000 1000 1000 1000 Road Map
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Safe BooleanMRSW vs Regular BooleanMRSW
Only difference is when newly written value is same as old value: Next Safe register can return either Boolean value Regular register can return either new value or old value – if both new and old is x, then regular can only return x Safe Boolean MRSW Boolean MRSW
0 1 Regular Safe Boolean MRSW Boolean MRSW
0 1 Regular Safe
1 1 0 0 0 1 0 0 0 0 0 1 0 1 Regular?
0 1 Safe Boolean MRSW Boolean MRSW
0 Regular Safe Boolean MRSW Boolean MRSW Regular Last writtern: 0 0 0 0 0 0 0 0 public class RegBoolMRSWRegister implements Register<Boolean> { private boolean old; private SafeBoolMRSWRegister value; public void write(boolean x) { if (old != x) { value.write(x); value.write(x); old = x; }} public boolean read() { value.read(); return value.read(); }} Safe Boolean MRSW Boolean MRSW Regular Safe Boolean MRSW Boolean MRSW Regular public class RegBoolMRSWRegister implements Register<Boolean> { threadLocal boolean old; private SafeBoolMRSWRegister value; public void write(boolean x) { if (old != x) { value.write(x); value.write(x); old = x; Last bit this thread (madeup syntax) }} public boolean read() { value.read(); return value.read(); }} wrote public class RegBoolMRSWRegister implements Register<Boolean> { threadLocal boolean old; private SafeBoolMRSWRegister value; public public void write(boolean x) { if (old != x) { value.write(x); value.write(x); old = x; }} public boolean read() { value.read(); return value.read(); Actual value }} Safe Boolean MRSW Boolean MRSW Regular Safe Boolean MRSW Boolean MRSW Regular public class RegBoolMRSWRegister implements Register<Boolean> { threadLocal boolean old; private SafeBoolMRSWRegister value; public void write(boolean x) { if (old != x) { value.write(x); value.write(x); Is new value different old = x; from last value I wrote? }} public boolean read() { value.read(); return value.read(); }} public class RegBoolMRSWRegister implements Register<Boolean> { threadLocal boolean old; private SafeBoolMRSWRegister value; public void write(boolean x) { if (old != x) { value.write(x); value.write(x); old = x; }} public boolean read() { If so, change it value.read(); return value.read(); }} (otherwise don’t!) Safe Boolean MRSW Boolean MRSW Regular public class RegBoolMRSWRegister implements Register<Boolean>{ threadLocal boolean old; private SafeBoolMRSWRegister value; public void write(boolean x) { if (old != x) { •Overlap? No Overlap? value.write(x); value.write(x); •No problem old = x; •either Boolean value works }} public boolean read() { value.read(); return value.read(); }} Safe MultiValued MRSW Regular MultiValued MRSW
Safe register can return value in range other than old or new when value changes 0101 0101 D 0101 0101
Multivalued Regular register can return only old or new when value changes! oes n ot wor k ! Road Map
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Regular MValued MRSW Register
Values are represented using unary notation An Mvalued register is implemented as an array of M Regular Boolean registers Initially the register is set to 0 Next Regular MValued MRSW Register
write():
A write() of value x, writes true to location x – which is a Regular Boolean MRSW register It then sets all the lower locations to false Writing MValued
Unary representation: bit[i] means value i read():
Reads the locations from lower to higher values until it reaches a value that is true Initially 0 1 00 0 00 0 01234567
Reader Writer Writing MValued
Write 5 Writing MValued
Write 5 1 00 00 01234567
Writer Initially 0 000 0 1 1 01234567
Writer Writing MValued
Write 5 MRSW Regular Boolean Regular Mvalued
public void write(int x) { this.bit[x].write(true); this.bit[x].write(true); (int i=xi) for (int i=x1; i>=0; i) this.bit[i].write(false); this.bit[i].write(false); } public int read() { (int for (int i=0; i < M; i++) (this.bit[i].read this.bit[i].read()) if (this.bit[i].read()) return i; }} MRSW public class RegMRSWRegister implements Register{ RegBoolMRSWRegister[M] RegBoolMRSWRegister[M] bit; 0 000 0 1 1 5 01234567
Reader Writer MRSW Regular Boolean Regular Mvalued
RegBoolMRSWRegister[M] RegBoolMRSWRegister[M] bit; public void write(int x) { this.bit[x].write(true); this.bit[x].write(true); (int i=xi) for (int i=x1; i>=0; i) this.bit[i].write(false); this.bit[i].write(false); } Unary MRSW MRSW Regular Boolean Regular Mvalued
RegBoolMRSWRegister[m] RegBoolMRSWRegister[m] bit; public void write(int x) { this.bit[x].write(true); this.bit[x].write(true); (int i=xi) for (int i=x1; i>=0; i) this.bit[i].write(false); this.bit[i].write(false); } public int read() { (int for (int i=0; i < M; i++) if (this.bit[i].read()) (this.bit[i].read()) this.bit[i].read return i; }} MRSW public class RegMRSWRegister implements Register{ RegMRSWRegisterimplements public class RegMRSWRegisterimplements Register { bit[i] public int read() { (int for (int i=0; i < M; i++) if (this.bit[i].read()) (this.bit[i].read()) this.bit[i].read return i; }} representation: means value i Set bit x MRSW Regular Boolean Regular Mvalued
RegBoolMRSWRegister[m] RegBoolMRSWRegister[m] bit; public void write(int x) { this.bit[x].write(true); this.bit[x].write(true); (int i=xi) for (int i=x1; i>=0; i) this.bit[i].write(false); this.bit[i].write(false); } public int read() { (int for (int i=0; i < M; i++) (this.bit[i].read this.bit[i].read()) if (this.bit[i].read()) return i; }} MRSW MRSW Regular Boolean Regular Mvalued
RegBoolMRSWRegister[m] RegBoolMRSWRegister[m] bit; MRSW RegMRSWRegisterimplements public class RegMRSWRegisterimplements Register { RegMRSWRegisterimplements public class RegMRSWRegisterimplements Register { Clear bits from higher to lower public void write(int x) { to this.bit[x].write(true); this.bit[x].write(true); (int i=xi) for (int i=x1; i>=0; i) this.bit[i].write(false); this.bit[i].write(false); } public int read() { (int for (int i=0; i < M; i++) (this.bit[i].read this.bit[i].read()) if (this.bit[i].read()) return i; }} Scan from lower higher & return first bit set Regular Register Conditions
Further conditions for a register to be regular:
No read() call should return a value from the future No read() call should return a value from the distant past – only the most recently written nonoverlapping value must be returned Road Map
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Next Road Map (Slight Detour)
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular SRSW Atomic MRSW atomic MRMW atomic Atomic Register Conditions
Together with the conditions for a regular register, an additional condition for an atomic register is:
An earlier read() cannot return a value later that that returned by a later read() In other words, values read() should be in the correct order SRSW register
Since a SRSW register has no concurrent reads, the only way that the condition for an atomic register can be violated is when two reads that overlap the same write read values out of order SRSW Regular
Regular writer Regular reader 15678 5678 234 1234 Instead of 5678… Concurrent Reading When could this happen? SRSW Regular
Regular writer SRSW Regular
Regular writer 1234 5678 5678 Initially 1234 Reg write(5678) ic tom A
5678 Reg read(5678) Regular reader 15678 5678 234 Regular reader Initially 1234 Reg write(5678) Reg read(1234) o Als mic At o 1234 Instead of 5678… time time SRSW Regular
Regular writer Timestamps
Solution is to for each value to have an added tag – a timestamp Timestamps are used to order concurrent calls Regular reader 15678 5678 234 1234 Initially 1234 Reg write(5678) Reg read(5678) Reg read(1234) not mic! At o Instead of 5678… Write 5678 time happened Timestamps
The writer writes a timestamp to a value Each reader remembers the latest timestamp/value pair ever read If a later read() then returns an earlier value the value is discarded and the reader uses the last value Timestamped Values 1:45 2:00 5678 1234 2:00 5678 Writer writes value and stamp together Reader saves last read (value,stamp) and returns new value only if higher stamp SRSW Regular
writer SRSW Atomic Atomic SRSW
StampedValue<T> public class StampedValue<T> { public long stamp; public T value; public StampedValue (T init) { stamp = 0; value = init; } (StampedValue public StampedValue max (StampedValue x,StampedValue y) { (x.stamp y.stamp) if (x.stamp > y.stamp) return x; else return y; } } reader 1:45 2:00 1234 5678 1:45 1234 Reg write(2:00 5678) read(2:00 5678) time old = 2:00 read(1:45 1234) s ea Sammic At o 1:45 1234 Less than 2:00 5678 So stick with 5678 Atomic SRSW
AtomicSRSWRegister<T> public class AtomicSRSWRegister<T> implements Register<T> { lastStamp; long lastStamp; StampedValue<T> lastRead; StampedValue<T> lastRead; StampedValue<T> StampedValue<T> value; public T read() { StampedValue<T> StampedValue.max(value, StampedValue<T> result = StampedValue.max(value, lastRead); lastRead); lastRead = result; result.value; return result.value; } public void write(T v) { long stamp = lastStamp + 1; StampedValue(stamp, value = new StampedValue(stamp, v); lastStamp = stamp; } } > 5678 Atomic SRSW Atomic MRSW Can the atomic SRSW be used to built an atomic MRSW? Solution of Safe MRSW Registers:
Every thread in array Write starts at the beginning of the array and iterates through array Read reads only its own array location Atomic Single Reader MultiReader
stamp 1:45 1:45 1:45 1:45 value 1234 1234 1234 1234 Atomic Atomic MRSW
Writer starts write… stamp 2:00 value 5678 1234 1234 1234 Atomic SRSW One per reader 1:45 1:45 1:45 Atomic SRSW
zzz… stamp 2:00 1:45 1:45 1:45 value 5678 1234 1234 1234 Atomic MRSW 2:00, 5678
reader reads later reader Atomic MRSW
We address this problem by having earlier reader threads help out later threads, by telling them which value they read 1:45 1234 Yellow was completely after Blue but read earlier value…not linearizable! Atomic MRSW
Write 4 nthreads share a nbyn array of stamped values Read() calls determine latest threads by timestamps Similar to the Safe MRSW Register implementation, the writer writes the new values to the array, but only on the diagonals 4 4 4 4 zzz… Can’t Yellow Miss Blue’s Update? … Only if Readers Overlap…
Reader 2 4 4 4 4 If not then the current value 1:45 1234 write(2:00 5678) read(2:00 5678) read(1:45 1234) Is there a value in this is column with the latest a higher timestamp? In which case time to read its OK 1234 Bad Case Only When Readers Don’t Overlap
1:45 1234
write(2:00 5678) read(2:00 5678) Road Map
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic In which case Blue will complete writing 2:00 5678 to its column read(2:00 5678) time Next MultiWriter Atomic From MultiRead Reader Atomic
Writer 1 Write 5678
stamp value 1234 1234 5678 1234 XYZW 1234
Readers read all and take max (Lexicographic like Bakery) Each writer reads all then writes Max+1 to its register 1:45 1:45 2:00 1:45 2:15 1:45 Writer 3 Write XYZW Max is 2:15, return XYZW ...
View
Full
Document
 Spring '10
 KLAZAR

Click to edit the document details