Course Hero - We put you ahead of the curve!

You have requested the below document.

Coursehero >> Massachusetts >> Harvard >> CS 299

Systems Distributed Sape Mullender Chapter 13 Notes - Dan Ellard Chapter 13 - Transaction-Processing Techniques William E. Weihl Overview: Distributed systems must tolerate failures, and distributed systems can require complex synchronization of concurrent processes. Both of these issues are interesting; together they are a headache. One approach to this combo is ATOMIC TRANSACTIONS. An atomic transaction *appears* to occur indivisibly (and can have several other useful properties wrt failures) and so they are relatively easy to reason about-- instead of worrying about concurrency, the programmer can treat these transactions as sequential operations even though several might be going on concurrently. 13.1 -- Problem Addressed Concurrency and synchronization - controlling access to shared data. Grouping operations into a single atomic operation. Operations within atomic transactions cannot appear as though they are interleaved. Failures - to tolerate failures, replicated servers can be used. In spite of failures, we want all replicas to reach the same state at the "end" of each transaction. It's all or nothing, everywhere-- either every replica completes the transaction, in the same manner, or all replicas decide to abandon the transaction. 13.2 -- What Are Transactions? This chapter deals almost exclusively with transactions defined in terms of the ACID properties. There are other properties, which might be useful in other situations, but ACID transactions are very easy to deal with. A - Atomic Each operation appears indivisible wrt crashes-- it occurs completely or not at all. "Partial effects" are invisible. COMMIT - a transactions that completed successfully ABORT - the transaction has failed and all evidence of its operations are erased. This property is also called RECOVERABILITY (because we can recover from aborted transactions-- they leave no damage behind). C - Consistent The transaction, when executed alone and committed, preserves some set of invariants which have been defined on the system state. (The outcome system state is consistent with a set of invariants.) I - Isolated (aka serializability or concurrency atomicity) Transactions are indivisible wrt each other. If a set of transactions are performed concurrently, the result is the same as if the same set of transactions had been performed in *some* order sequentially. (Often we also want to define what that order would be, in terms of real time or some other measure, but this is not necessary for isolation.) D - Durable (aka permanence) "The effects of committed transactions are very likely to survive subsequent failures." Of course, if the system catches on fire, or some other catastrophic event occurs, then some state might be lost. But barring some disaster, the system does not "lose" or "forget" the effect of a committed transaction. If we have a way to do ACID transactions, life is easy. We can have many concurrent transactions going on at the same time, and they won't get in each others hair, violate their invariants, or be forgotten later. (The rest of this section goes deeper into defining these terms, and gives a very precise if somewhat confusing specification for an ACID system.) 13.3 -- Concurrency Control This section begins discussing algorithms that can be used to implement the ACID spec. TWO-PHASE LOCKING - a common synchronization method. - Ensures that a transaction can always commit without violating serializability. Operates by locking the data so that: a. Operations of concurrent transactions commute. b. Operations reflect the results of all committed transactions. We can "serialize" the transactions by their commit order. The locks must be set up so that if two operations do not commute, they require conflicting locks and so one must block until the other is finished. The "two-phase" comes from the fact that all locks are acquired, and then all locks are released. No lock is acquired after any lock is released. As spec'd, there is a possibility of deadlock. This can be avoided by the usual means-- prevention, avoidance, or detection. The actual locking mechanism required might be simpler; it depends on the semantics required by the system. 13.4 -- Single-Site Recovery First, we consider recovery for a single site. Failure model: - Transaction abort: the operations need to be undone for some reason detected by the algorithm. - System crash: thud. Nonvolatile storage survives, but no other state. - Media failure: nonvolatile storage is lost, but the system continues to run. - Catastrophe: all storage (volatile and nonvolatile) is lost. There is no recovery from catastrophe. 13.4.1 -- Abort Recovery We assume that the transaction aborts, but no other failures occur. Our goal is to make it look as though the transaction never took place (as far as the state of the system is concerned). Two main approaches: Update-in-place: state is modified as the transaction proceeds, and if the transaction aborts, the changes are "undone". Deferred-update: state is not modified until the transaction is ready to commit, and then all updates are made at once. Update-in-place seems to have much better performance in general, particularly if operations are likely to commit. However, deferred-update can have some benefits in the area of concurrency. 13.4.2 -- Crash Recovery Updates might involve several physical reads and writes; how do we make them appear atomic, so that the state of the nonvolatile storage always is consistent (or at least contains a consistent view of the state)? The typical mechanism is a LOG. The log records which updates correspond to committed or aborted transactions. It is noted that reconstructing the state of the world when the crash took place from the log has subtleties. We can also use a log-like structure to batch writes. (this is the basic idea behind log-structured file systems). ARIES is described. This is an update-in-place system. All activity is recorded in the log. In the case of an abort, COMPENSATION RECORDS are recorded in the log to represent the "undoing" of a previous update in the log. The log is FORCED to disk whenever a commit occurs. This can be a bottleneck, so group commit can be used. The committed log entries form a REDO; since they can be replayed to simulate the "redo" of the actions and reconstruct the state. Note that the actual state is updated separately and does not reside in the log! Therefore, there is a possibility that the actual state (i.e. some file) may be updated, and then a crash occurs, so no mention of that update appears in the log. We could reverse the problem by making sure that the log entries are written before the actual updates, but that doesn't help. Instead, we require that redo log records and their corresponding undo records be recorded before the update can actually happen. This way, no matter what the state of the log and the file, we can find a consistent state to restart in. To recover-- 1. Replay all "redo" records. 2. Replay all "undo" records that belong to uncommitted transactions. But there's a catch: what if we crash in the middle of a recovery? How do we recover from *that*? The redo records must be idempotent, so we can replay them (in the same order) and get the same result. We can keep track of what log records have successfully been redone so we don't have to do them again and again. To keep the log (and the recovery process) from growing without bound, we can CHECKPOINT the system from time to time, effectively recording a snapshot of the system state at some point in the log, and then removing all prior entries in the log. 13.4.3 -- Recovery from Media Failure Reach for the backup tapes... and then redo the log from the last archive. There are problems with getting a good archive without stopping the system. This has been solved with current file systems, but most people aren't using "current" file systems... 13.4.4 -- Discussion We haven't discussed how to combine recovery with concurrency control. Now we must. The redo/undo records in the log must commute wrt operations in other transactions. Beware-- the undo doesn't generally commute with anything. We need to ensure that "the granularity of the locks is at least as large as the granularity of recovery." 13.5 -- Distributed Recovery And now, for distributed systems. We must make sure that all the sites (replicas) are in a consistent state. There must be agreement whether to commit or abort each operation (the ATOMIC COMMITMENT PROBLEM we saw earlier in the semester). Common approach: two-phase commit (or, as we saw before, three-phase commit in order to tolerate different kinds of faults). Note the use of INCARNATION NUMBERS-- the crash count. If during a commit protocol a process gets two different messages from the same process with different incarnation numbers, then the other process must have crashed; abort. 13.6 -- Other Concurrency Control Methods There are other concurrency control methods besides two-phase locking, and for some purposes these methods are better. One strategy worth mentioning is OPTIMISTIC; the idea is to do no locking at all until commit, and then check to see whether any concurrency restrictions would have been violated. If so, then abort and retry the transaction; otherwise proceed. If conflicts are unlikely, this is a win, but if conflicts are common this is a big lose. 13.7 -- Distributed Concurrency Control Sites must coordinate to make sure that the same concurrency control is applied everywhere. They must also agree on a serialization order for all transactions (or at least those that must be serialized). This is not a problem when everyone is running the same software. But when the system is heterogeneous, it's an unsolved headache. 13.8 -- Nested Transactions It may be desirable to structure transactions themselves in terms of pieces that can succeed or fail, and are serializable (and may run concurrently)-- the pieces are themselves transactions. These are called SUBTRANSACTIONS or NESTED TRANSACTIONS. In most ways they are similar to ordinary transactions, but there are differences: - The commit of the subtransactions does not actually take effect until the parent commits. If any subtransaction aborts, all are aborted and undone. - Subtransactions "inherit" locks: = The child won't block on a lock the parent holds. = The child cannot release a lock that the parent holds. They are useful because they allow high degrees of concurrency without extra locking and with only one actual commit/abort action. 13.9 -- Discussion More discussion of how atomic transactions are not the best solution for all problems. However, the stigma of slowness has been overcome by the newer algorithms that reduce or eliminate the need for synchronous I/O to nonvolatile storage. 13.10 -- Bibliographic and Historical Notes There's been a long history of work in this area, and many interesting or notable ideas. See references for more info...

Find millions of documents here - Study Guides, Homework Solutions, Papers, Exam Answer Keys and more. Course Hero has millions of course related materials that will enable you to learn better, faster and get an A in all your courses.
Below is a small sample set of documents:

notes-chap05.txt
Path: Harvard >> CS >> 299 Fall, 2009

AaronMandel.txt
Path: Harvard >> CS >> 50 Fall, 2009
HuaTang.txt
Path: Harvard >> CS >> 50 Fall, 2009
HuaTang.txt
Path: Harvard >> CS >> 95 Fall, 2009
dup.2.txt
Path: Harvard >> CS >> 50 Fall, 2009
s9-ans.txt
Path: Harvard >> CS >> 50 Fall, 2009
marks.pdf
Path: Harvard >> ISS >> 17 Fall, 2009
chirinos.pdf
Path: Harvard >> ISS >> 18 Fall, 2009
steiner.pdf
Path: Harvard >> ISS >> 20 Fall, 2009
booknotes.pdf
Path: Harvard >> ISS >> 20 Fall, 2009
313-316.pdf
Path: Harvard >> ISS >> 21 Fall, 2009
fletcher.pdf
Path: Harvard >> ISS >> 20 Fall, 2009
shalakany.pdf
Path: Harvard >> ISS >> 20 Fall, 2009
shattuck.pdf
Path: Harvard >> ISS >> 16 Fall, 2009
S_Jorstad.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
SS_Bus_Schedule_08.pdf
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
poster.pdf
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
poster.pdf
Path: Harvard >> XSURVEYS >> 06 Fall, 2009
R_Mittal.pdf
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
M_Bruggen.pdf
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
Agenda1.pdf
Path: Harvard >> XGRATINGS >> 07 Fall, 2009
abs.pdf
Path: Harvard >> STARS >> 05 Fall, 2009
D_Harris.pdf
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
L_Godfrey.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
P_Ogle.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
G_Madejski.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
Q_Hart.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
F_Massaro.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
M_Birkinshaw.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
N_Isobe.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
M_Begelman.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
C_Jones.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
H_Li.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
S_Baum.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
L_Birzan.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
S_Giancintucci.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
D_Rafferty.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
M_Guainazzi.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
N_Jetha.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
P_Hardee.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
M_Birkinshaw1.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
N_Soker.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
finale.ppt
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
092_Saez.pdf
Path: Harvard >> SYMPOSIUM >> 2007 Fall, 2009
053_Getman.pdf
Path: Harvard >> SYMPOSIUM >> 2007 Fall, 2009
055_Kashyap.pdf
Path: Harvard >> SYMPOSIUM >> 2007 Fall, 2009
074_Cavagnolo.pdf
Path: Harvard >> SYMPOSIUM >> 2007 Fall, 2009
099_Aarnio_ppt_pc.pdf
Path: Harvard >> SYMPOSIUM >> 2007 Fall, 2009
101_Broos.pdf
Path: Harvard >> SYMPOSIUM >> 2007 Fall, 2009
060_Tsujimoto.pdf
Path: Harvard >> SYMPOSIUM >> 2007 Fall, 2009
097_Kim.pdf
Path: Harvard >> SYMPOSIUM >> 2007 Fall, 2009
094_Aldcroft.pdf
Path: Harvard >> SYMPOSIUM >> 2007 Fall, 2009
065_DeLaney.pdf
Path: Harvard >> SYMPOSIUM >> 2007 Fall, 2009
090_Mueller.pdf
Path: Harvard >> SYMPOSIUM >> 2007 Fall, 2009
076_Dupke.pdf
Path: Harvard >> SYMPOSIUM >> 2007 Fall, 2009
063_Allen.pdf
Path: Harvard >> SYMPOSIUM >> 2007 Fall, 2009
062_Airhart.pdf
Path: Harvard >> SYMPOSIUM >> 2007 Fall, 2009
098_Tuellmann.pdf
Path: Harvard >> SYMPOSIUM >> 2007 Fall, 2009
104_Xiang.pdf
Path: Harvard >> SYMPOSIUM >> 2007 Fall, 2009
050_Wolk_ppt_pc.pdf
Path: Harvard >> SYMPOSIUM >> 2007 Fall, 2009
Darcangelo_Francesca.pdf
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
M_Sobolewska.pdf
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
T_Ergin.pdf
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
P_Kharb.pdf
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
R_Laing.pdf
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
S_Willner.pdf
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
Miller_Brendan.pdf
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
OSullivan_Ewan.pdf
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
Hogan_Brandon.pdf
Path: Harvard >> RADIOGALS >> 08 Fall, 2009
Massaro_Francesco.pdf
Path: Harvard >> RADIOGALS >> 08 Fall, 2009