MIT16_842F09_lec10

MIT16_842F09_lec10 - System Safety Prof. Nancy G. Leveson...

Info iconThis preview shows pages 1–11. Sign up to view the full content.

View Full Document Right Arrow Icon
System Safety Prof. Nancy G. Leveson
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
It’s never what we don’t know that stops us. It’s what we do know that just ain’t so.” Dean Kamen “Without changing our patterns of thought, we will not be able to solve the problems we created with our current patterns of thought.” Albert Einstein
Background image of page 2
Accident with No Component Failures
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Types of Accidents • Component Failure Accidents – Single or multiple component failures – Usually assume random failure • Component Interaction Accidents – Arise in interactions among components – Related to • Interactive complexity and tight coupling • Use of computers and software
Background image of page 4
Interactive Complexity • Critical factor is intellectual manageability – A simple system has a small number of unknowns in its interactions (within system and with environment) – Interactively complex (intellectually unmanageable) when level of interactions reaches point where can no longer be thoroughly • Planned • Understood • Anticipated • Guarded against
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Safety vs. Reliability • Safety and reliability are NOT the same – Sometimes increasing one can even decrease the other. – Making all the components highly reliable will have no impact on component interaction accidents. • For relatively simple, electro-mechanical systems with primarily component failure accidents, reliability engineering can increase safety. • For complex, software-intensive or human-intensive systems, we need something else.
Background image of page 6
Software Changes System Engineering • Software is simply the design of a machine abstracted from its physical realization • Software “failure” modes are different (do abstractions fail?) – Usually does exactly what you tell it to do – Problems occur from operation, not lack of operation – Usually doing exactly what software engineers wanted += General Purpose Machine Software Special Purpose Machine
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Abstraction from Physical Design • Software engineers are doing system design • Most operational software errors related to requirements (particularly incompleteness) Autopilot Expert Requirements Software Engineer Design of Autopilot ÆÆ Æ
Background image of page 8
Software-Related Accidents • Are usually caused by flawed requirements – Incomplete or wrong assumptions about operation of controlled system or required operation of computer – Unhandled controlled-system states and environmental conditions • Merely trying to get the software “correct” or to make it reliable will not make it safer under these conditions.
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Software-Related Accidents (2) • Software may be highly reliable and “correct” and still be unsafe: – Correctly implements requirements but specified behavior unsafe from a system perspective. – Requirements do not specify some particular behavior required for system safety (incomplete) – Software has unintended (and unsafe) behavior beyond what is specified in requirements.
Background image of page 10
Image of page 11
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 11/08/2011 for the course AERO 16.810 taught by Professor Olivierdeweck during the Winter '07 term at MIT.

Page1 / 49

MIT16_842F09_lec10 - System Safety Prof. Nancy G. Leveson...

This preview shows document pages 1 - 11. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online