{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

MIT16_842F09_lec10

MIT16_842F09_lec10 - System Safety Prof Nancy G Leveson Its...

Info icon This preview shows pages 1–11. Sign up to view the full content.

View Full Document Right Arrow Icon
System Safety Prof. Nancy G. Leveson
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
It’s never what we don’t know that stops us. It’s what we do know that just ain’t so.” Dean Kamen “Without changing our patterns of thought, we will not be able to solve the problems we created with our current patterns of thought.” Albert Einstein
Image of page 2
Accident with No Component Failures
Image of page 3

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Types of Accidents Component Failure Accidents Single or multiple component failures Usually assume random failure Component Interaction Accidents Arise in interactions among components Related to Interactive complexity and tight coupling Use of computers and software
Image of page 4
Interactive Complexity Critical factor is intellectual manageability A simple system has a small number of unknowns in its interactions (within system and with environment) Interactively complex (intellectually unmanageable) when level of interactions reaches point where can no longer be thoroughly • Planned • Understood • Anticipated Guarded against
Image of page 5

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Safety vs. Reliability Safety and reliability are NOT the same Sometimes increasing one can even decrease the other. Making all the components highly reliable will have no impact on component interaction accidents. For relatively simple, electro-mechanical systems with primarily component failure accidents, reliability engineering can increase safety. For complex, software-intensive or human-intensive systems, we need something else.
Image of page 6
Software Changes System Engineering Software is simply the design of a machine abstracted from its physical realization Software “failure” modes are different (do abstractions fail?) Usually does exactly what you tell it to do Problems occur from operation, not lack of operation Usually doing exactly what software engineers wanted + = General Purpose Machine Software Special Purpose Machine
Image of page 7

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Abstraction from Physical Design Software engineers are doing system design Most operational software errors related to requirements (particularly incompleteness) Autopilot Expert Requirements Software Engineer Design of Autopilot Æ Æ Æ
Image of page 8
Software-Related Accidents Are usually caused by flawed requirements Incomplete or wrong assumptions about operation of controlled system or required operation of computer Unhandled controlled-system states and environmental conditions Merely trying to get the software “correct” or to make it reliable will not make it safer under these conditions.
Image of page 9

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Software-Related Accidents (2) Software may be highly reliable and “correct” and still be unsafe: Correctly implements requirements but specified behavior unsafe from a system perspective. Requirements do not specify some particular behavior required for system safety (incomplete) Software has unintended (and unsafe) behavior beyond what is specified in requirements.
Image of page 10
Image of page 11
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern