Consistency Memory (II)
Instructor: Josep Torrellas CS533
Copyright Josep Torrellas 2003, 2008
1
Enhancing RC
Allow an access to partially or fully proceed even though the delay arcs demand that the access be delayed Detect and remedy the cases when the early access would result in incorrect behavior. How? Re-issue the access to the memory system
Summary: common case proceeds with high speed while still preserving correctness
Copyright Josep Torrellas 2003, 2008
2
Two Techniques
Prefetching Speculative execution
Copyright Josep Torrellas 2003, 2008
3
Prefetching
Prefetching is classified as:
Binding vs non-binding Hardware vs software
Cache coherent machines can provide non-binding prefetching Non-binding prefetching:
does not affect the correctness for any consistency model can be used as performance booster
Can use:
for a read: read prefetch for a write: read-exclusive prefetch WHY?
Bring data into the cache and perform the operation when the memory consistency model allows
Copyright Josep Torrellas 2003, 2008 4
What if there is an intervening access
After a read prefetch a remote processor writes:
our copy gets invalidated when the local read is actually issued: it misses
After a read-exclusive prefetch
a remote processor writes: same as above a remote processor reads: our copy loses exclusivity when the local write is issued: miss
Copyright Josep Torrellas 2003, 2008
5
Implementation
Assume a processor with LD and ST buffers Local access kept in buffer, it is delayed until it is correct to do it Hardware automatically issued:
Read prefetch: for reads in the buffer Read-exclusive pref: for writes (and RMW) in the buffer
Prefetches are buffered in a special prefetch buffer. They are retired as quickly as the bandwidth of the memory system allows Prefetch first checks the cache. If data there in the right state, then prefetch is discarded Prefetch response is placed into the cache
Copyright Josep Torrellas 2003, 2008 6
Implementation
If processor references the line before the prefetch has arrived, no additional request is issued to the memory system (combining)
Copyright Josep Torrellas 2003, 2008
7
Issues
Prefetching
needs lock up free caches causes additional cache accesses (double) but: only if the request cannot proceed past the LD/ST buffer because of memory consistency problems best with out-of-order execution and aggressive branch prediction
Copyright Josep Torrellas 2003, 2008
8
Examples
Examples (assume that the branch prediction takes the path that assumes sych succeeds) lock L (miss) Lock L (miss) write A (miss) read C (miss) write B (miss) read D (hit) unlock L (hit) read E[D] (miss) unlock L (hit) Assume: cache hit =1 cycles, cache miss=100. EX1: SC: 301, RC: 202, with pref (SC or RC): 103 EX2: SC: 302, RC: 203, with pref: 203 SC and 202 RC note: the E[D] not allow to perform until read to C completes (under SC) or lock access completes (under RC) Copyright Josep Torrellas 2003, 2008 9
Summary
does Pref not help when out-of-order consumption of return values is important to allow the processor to proceed efficiently.
We saw example: address of the read access to array E depends on the value of D, and although the read access to D is a cache hit, this access is not allowed to perform (ie the value cannot be used by the processor) until the read of C completes (under SC) or until the lock access completes (under RC).
Copyright Josep Torrellas 2003, 2008
10
Speculative Execution
Allow the processor to consume return values ooo regardless of the consistency constraints Goal: combination of:
prefetching for stores speculative execution for loads
Copyright Josep Torrellas 2003, 2008
11
Speculative Execution
Consider access u (long latency) followed by v (a load) Assume that the consistency model requires v to be delayed until u completes Speculative execution: the processor obtains or assumes a return value for v before u completes, and proceeds When u completes:
if current value value of v is as expected, speculation was successful if current value is different: throw out the computation that depended on the value of v and re-execute
Copyright Josep Torrellas 2003, 2008
12
Requirements
Speculation mechanism: obtain the speculated value Detection mechanism: Correction mechanism: to repeat the computation if misspeculated
Copyright Josep Torrellas 2003, 2008
13
Mechanisms
Speculation mechanism: Perform the access:
if cache hit: return immediately if miss: takes longer
Detection mechanism:
Nave: repeat the access when legal and compare the value Better: keep the data in the cache and monitor if you received a coherence transaction for it Result: cache accessed once rather than twice (as pref) Coherence transactions:
invalidation but: false s...