031604wloadsitua

031604wloadsitua - Direct Observation Display Supervisory...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Direct Observation Display Supervisory Control Computer System Control Interface Sensors 16.422 Workload and Situation Awareness Prof. R. John Hansman Acknowledgements to Mica Ensley Direct Observation Display Supervisory Control Computer System Control Interface Sensors Workload ü What is workload? ü Why is it important? Direct Observation Display Supervisory Control Computer System Control Interface Sensors Driving Case: B757/767 2 or 3 person crew ? ü Prior to 767 somewhat arbitrary break at 100 seats o DC-9 (2 person crew - pilot, co-pilot) o B-727 (3 person crew - pilot, co-pilot, flight engineer) ü B-757/767 Designed for 2 person Crew o Use of automation and simplified systems so minimize systems management o Use of Advanced Cockpit to Increase SA and make primary flight tasks easier ü Safety concerns raised by Air Line Pilots Association (ALPA) o Workload o Off Nominal and Emergency Conditions (eg manual pressurization) o Job Protection issues ü Workload became political and regulatory issue Direct Observation Display Supervisory Control Computer System Control Interface Sensors Workload Definitions? ü Physical Workload o Traditional view of work for manual labor o Can be measured in physical terms (ergs, joules, ..) o Limited impact of skill to minimize (ie subject variability) ü “Mental” Workload o o o o o o Often not related to physical work Internal measure difficult to observe Varies with task difficulty and complexity Significant subject variability No real consensus on what it is Workload is a “dirty” word in Experimental Psychology ü Activity o Things that are done o Physical activity easy to measure ü Taskload o External measure of tasks which need to be done o Can be weighted for factors such as task difficulty or complexity Direct Observation Display Supervisory Control Computer System Control Interface Sensors Yerks-Dotson Law http://www.hf.faa.gov/Webtraining/Cognition/Workload/Mental3.htm Direct Observation Display Supervisory Control Computer System Control Interface Sensors Typical Performance vs. Task Load Curve Performance Task Load Helicopter Observation of Driver Example Direct Observation Display Supervisory Control Computer System Control Interface Sensors Off Nominal Considerations ü System design often driven by off-nominal conditions o Emergencies o System Failures o Failure of the Automation system ü Secondary task considerations ü Cockpit Example o Emergency diversion o Depressurization Direct Observation Display Supervisory Control Computer System Control Interface Sensors Workload Measurement Approaches ü Objective Performance Approaches o Primary Task (Yerks Dodson) o Secondary Task (works well to measure saturation threshold) u Concept of Spare Cognitive Capacity ü Objective Physiological Measures (weak) o o o o o Heart Rate Variability Pupil Diameter EEG P 300 Skin Galvanic Response New Imaging Methods ü Subjective Workload Assessment Techniques o Formal o Direct Query Direct Observation Display Supervisory Control Computer System Control Interface Sensors Subjective Assessment Techniques ü Simpson-Sheridan/ Cooper-Harper ü Bedford Scale ü Rate or Perceived Exertion (RPE) ü NASA Task Load Index (TLX) ü Defense Research Agency Workload Scale (DRAWS) ü Malvern Capacity Estimate (MCE) Direct Observation Display Supervisory Control Computer System Control Interface Sensors Simpson-Sheridan Scale ü Modified Cooper Harper Scale for Workload Direct Observation Display Cooper Harper Supervisory Control Computer System Sensors Control Interface Source: http://history.nasa.gov/SP-3300 Direct Observation Display Supervisory Control Computer System Control Interface Sensors Bedford Scale ü The Bedford Scale is a uni-dimensional rating scale designed to identify operator's spare mental capacity while completing a task. The single dimension is assessed using a hierarchical decision tree that guides the operator through a ten-point rating scale, each point of which is accompanied by a descriptor of the associated level of workload. It is simple, quick and easy to apply in situ to assess task load in high workload environments, but it does not have a diagnostic capability. ü Refs: Roscoe and Ellis, 199 Source: Eurocontrol http://www.eurocontrol.int/eatmp/hifa/hifa/HIFAdata_tools_workload.html Direct Observation Display Supervisory Control Computer System Control Interface Sensors Rate of Perceived Exertion Borg RPE Scale ü ü ü ü Borg Rate of Perceived Exertion Scale Originally developed for physical workload Intended to be ordinal scale Modified 0-10 version CR-10 ü ü ü ü ü ü ü ü ü ü ü ü ü ü ü 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 No exertion at all Extremely Light Very Light Light Somewhat Hard Hard (Heavy) Very Hard Extremely Hard Maximal Exertion Source: http://dticam.dtic.mil Direct Observation Display Supervisory Control Computer System Control Interface Sensors NASA TLX Task Load Index ü ü ü ü ü Sandy Hart 5 Element Structured Subjective Assessment Individual relative element calibration Requires Trained Users Often used but difficult to interpert http://www.hf.faa.gov/Webtraining/Cognition/Workload/Mental3.htm Direct Observation Display Supervisory Control Computer System Control Interface Sensors DRAWS ü DRAWS is a multi-dimensional tool (similar to NASA TLX) used to gain a subjective assessment of workload from operators. The rating scales are input demand (demand from the acquisition of information from external sources), central demand (demand from mental operations), output demand (demand from the responses required by the task), and time pressure (demand from the rate at which tasks must be performed). DRAWS offers ease of data collection and ratings can be obtained during task performance by asking respondent to call out ratings (from 0 to 100) to verbal prompts. This can also provide a workload profile through a task sequence. ü Refs: Farmer et al, 1995; Jordan et al, 1995. Source: Eurocontrol http://www.eurocontrol.int/eatmp/hifa/hifa/HIFAdata_tools_workload.html Direct Observation Display Supervisory Control Computer System Control Interface Sensors Malvern Capacity Estimate ü MACE is designed as a quick simple and direct measure of maximum capacity. It is designed to provide a direct measure of air traffic controllers' subjective estimates of their own aircraft handling capacity. MACE is applied at the end of a work sequence (e.g., simulation trial) and provides capacity estimates in aircraft per hour. Applications have typically been in simulation environments. ü Refs: Goillau and Kelly, 1996. Source: Eurocontrol http://www.eurocontrol.int/eatmp/hifa/hifa/HIFAdata_tools_workload.html Direct Observation Display Supervisory Control Computer System Control Interface Sensors Instant Self Assessment of Workload (ISA) ü ISA was developed as a tool that an operator could use to estimate their perceived workload during real-time simulations. The operator is prompted at regular intervals to give a rating of 1 to 5 of how busy he is (1 means under-utilized, 5 means excessively busy). These data can be used to compare operators' perceived workload, for example, with and without a particular tool, or between different systems. ü Refs: Jordan, 1992. Source: Eurocontrol http://www.eurocontrol.int/eatmp/hifa/hifa/HIFAdata_tools_workload.html Direct Observation Display Supervisory Control Computer System Control Interface Sensors Subjective Workload Assessment Techniques (SWAT) ü SWAT is a subjective scale of workload that can be administered easily in operation situations and is available as a PC-based software tool. It is multi-dimensional tool incorporating factors of temporal load, mental effort and psychological stress. SWAT has two stages: The respondent ranks the levels of the three workload scales in order from the lowest to highest workload prior to the trial, and rates each of the scales during the trial. It was originally designed to assess aircraft cockpit and other crew-station environments to assess the workload associated with the operators' activities. ü Refs: Reid and Nygren, 1988; Dean 1997 Source: Eurocontrol http://www.eurocontrol.int/eatmp/hifa/hifa/HIFAdata_tools_workload.html Direct Observation Display Supervisory Control Computer System Control Interface Sensors Situation Awareness ü Term originally defined for air combat ü Working Definition (Hansman) : Sufficiently detailed mental picture of the vehicle and environment (i.e. world model) to allow the operator to make wellinformed (i.e., conditionally correct) decisions. ü Individual SA and Team SA ü Has become an extremely popular and powerful concept ü Mica Endsley: Situation vs Situational Awareness Direct Observation Display Supervisory Control Computer System Control Interface Sensors Endsley Situation Awareness Model (Image removed due to copyright considerations.) Direct Observation Display Supervisory Control Computer System Control Interface Sensors Model of Pilots’ Cognitive Constructs of Information Processing References: Endsley, 1995; Pawlak, 1996; Reynolds et al., 2002 Information Request/Transmission Training Experience Procedures Pilot Weather Mental Model Situation Dynamics Weather Information System PLAN Nominal Plan Contingency Plans n rm atio SITUATIONAL AWARENESS DECISION 1- Perception 2 - Comprehension 3 - Projection 3 Projection Weather Phenomenology Interaction Aircraft State Aircraft Envelope Weather Forecast Future Exposure Aircraft Trajectory Inf o Monitoring Evaluating Planning Adjusting PERFORMANCE OF ACTIONS Implement Interaction PERCEP TION Weather State Aircraft Aircraft Trajectory Control Direct Observation Display Supervisory Control Computer System Control Interface Sensors Enhancing SA ü Level 1 - Perception o Enhanced Perception Systems ( eg Enhanced Vision Systems) o Alerting Systems ü Level 2 - Comprehension o SA Displays (eg Moving Map Displays, EGPWS) ü Level 3 - Projection o Displays o Decision Support Tools Direct Observation Display Supervisory Control Computer System Control Interface Sensors Enhancing SA ü Level 1 - Perception o Enhanced Perception Systems ( eg Enhanced Vision Systems) o Alerting Systems ü Level 2 - Comprehension o SA Displays (eg Moving Map Displays, EGPWS) ü Level 3 - Projection o Displays o Decision Support Tools Direct Observation Display Supervisory Control Computer System Control Interface Sensors Enhanced Vision & Synthetic Vision Systems Synthetic Vision Enhanced Vision Direct Observation Display Supervisory Control Computer System Control Interface Enhanced Vision Sensors Picture of the outside world created by real-time weather and darkness penetrating on-board sensors (eg. Cameras, FLIR, MMW radar, and weather radar). Direct Observation Display Supervisory Control Computer System Control Interface Synthetic Vision Sensors Picture of the outside world created by combining precise navigation position with databases of comprehensive geographic, cultural and tactical information. Direct Observation Display Supervisory Control Computer System Control Interface Sensors Enhancing SA ü Level 1 - Perception o Enhanced Perception Systems ( eg Enhanced Vision Systems) o Alerting Systems ü Level 2 - Comprehension o SA Displays (eg Moving Map Displays, EGPWS) ü Level 3 - Projection o Displays o Decision Support Tools Direct Observation Display Supervisory Control Computer System Control Interface Sensors Enhancing SA ü Level 1 - Perception o Enhanced Perception Systems ( eg Enhanced Vision Systems) o Alerting Systems ü Level 2 - Comprehension o SA Displays (eg Moving Map Displays, EGPWS) ü Level 3 - Projection o Displays o Decision Support Tools Direct Observation Display Supervisory Control Computer System Control Interface New Weather Datalink Products Sensors ARNAV Avidyne Bendix/King FAA FISDL Control Vision Echo Flight Garmin UPS – AirCell Vigyan Direct Observation Display Supervisory Control Computer System Control Interface Sensors Model of Pilots’ Cognitive Constructs of Information Processing References: Endsley, 1995; Pawlak, 1996; Reynolds et al., 2002 Information Request/Transmission Training Experience Procedures Pilot Weather Mental Model Situation Dynamics Weather Information System PLAN Nominal Plan Contingency Plans n rm atio SITUATIONAL AWARENESS DECISION 1- Perception 2 - Comprehension 3 - Projection 3 Projection Weather Phenomenology Interaction Aircraft State Aircraft Envelope Weather Forecast Future Exposure Aircraft Trajectory Inf o Monitoring Evaluating Planning Adjusting PERFORMANCE OF ACTIONS Implement Interaction PERCEP TION Weather State Aircraft Aircraft Trajectory Control Direct Observation Display Supervisory Control Computer System Control Interface Sensors Temporal Representation of Pilots’ Functions Tactical Strategic TEMPORAL REGIMES OF PLANNING Reactive Information Request/Transmission Situation Dynamics Weather Weather Information Information min Execution hrs - mins In-Flight Planning Go/ No-Go day - hrs Pre-Flight Planning PILOTS’ FUNCTIONS Interaction Aircraft Aircraft Information Aircraft Trajectory Control Direct Observation Display Supervisory Control Computer System Control Interface Sensors Temporal Regimes of Wx Predictability Uncertainty Growth with Forecast Horizon Time constants dependent on: - Weather phenomena and phenomenology (e.g., convective weather, droplet size distribution, temperature) - Phase of weather phenomena (e.g., storm initiation versus storm decay) Weather Forecast Uncertainty Probabilistic Deterministic Persistence ? ? U(t) Time of Forecast Issuance Limit of Deterministic Predictability Weather Forecast Horizon Direct Observation Display Supervisory Control Computer System Control Interface Sensors Temporal Regimes of Cognitive Projection Uncertainty Growth with Horizon of Projection Weather representation based on deterministic forecast of “acceptable” accuracy Weather representation at time in future beyond “predictability limit” Weather representation based on observation over a time period where conditions do not significantly change Weather Projection Uncertainty Stochastic Deterministic Constant ? ? U(t) Reference Time of Weather Mental Model Limit of Deterministic Projection Horizon of Cognitive Projection Direct Observation Display Supervisory Control Computer System Control Interface Sensors Temporal Framework of Decision-Making Representation of Cognitive Plan Horizon of Cognitive Weather Projection Stochastic Time Deterministic Constant Time of Information Production Planning Dynamics Pilots’ Planning Horizon Reactive Time of Planning Tactical Strategic Direct Observation Display Supervisory Control Computer System Control Interface Sensors Representation of Cognitive Plan Examples Horizon of Cognitive Weather Projection Microburst + 2 hr Time Convective Front Stochastic +30 min Landing Before Front Passage Deterministic + 1 hr + 1 day Volcanic Ash Constant Time of Information Production Initial Climb Around Front + 0 hr Pilots’ Planning Horizon Tactical Strategic Reactive Time of Planning Direct Observation Display Supervisory Control Computer System Control Interface Sensors Measurement of Situation Awareness ü Situation Awareness General Assessment Technique (SAGAT) o o o o o Endsley Requires interruptions Invasive (queries may influence subsequent SA) Time issue Requires knowledge of required SA elements u Goal Directed Task Analysis ü Testable Response Approach o Pritchett and Hansman o Works for scenario based studies o Requires scenarios where differential SA implies differential action Direct Observation Display Supervisory Control Computer System Control Interface Sensors Datalink Shared Information Experiment (Traffic & Weather) Simulation Host Simulator Data via Internet Scenario Generation Pseudo-Pilot Station Secondary Traffic Weather Data Simulator Data via Internet Weather Traffic Data link OFF SLB NW589 335C QF004 360C ON ON TW443 220C DL102 170C Advanced Cockpit Simulator Data link OFF Plan View Display Voice Communication Link via Internet Pilot Air Traffic Controller Direct Observation Display Supervisory Control Computer System Control Interface Sensors From the Cockpit Data link OFF Data link ON Direct Observation Display Supervisory Control Computer System Control Interface Sensors From the ATC Display Data link OFF Data link ON Direct Observation Display Supervisory Control Computer System Control Interface Sensors Pseudo-Pilot Station Direct Observation Display Supervisory Control Computer System Control Interface Sensors Example Scenario ü 12-18 aircraft ü Convective weather ü Performed once without the shared information ü Repeated once with the shared information ü 6 subjects x 6 runs each = 36 runs total ü ~10 minutes in duration ü Averaged 80-90 voice transmissions per run ü Recorded data: o Situation awareness data o Aircraft trajectories o Voice data o Workload data o Subjective ratings Direct Observation Display Supervisory Control Computer System Control Interface Sensors Results: Situation Awareness ü ü Controllers’ situation awareness with respect to weather improves when weather information is shared Pilots’ situation awareness with respect to traffic improves when traffic information is shared Weather Situation Awareness Data link OFF 100% Aware Traffic Situation Awareness Data link OFF Data link ON Data link ON 100% Aware Pilot Pilot 61% Unaware 56% Aware Controller Unaware 39% 50% Aware Controller 94% Aware 89% Aware 94% Aware Aware Ambiguous Not aware Direct Observation Display Supervisory Control Computer System Control Interface Sensors Results: Controllers’ Weather Awareness Scenario 1 Scenario 2 Scenario 3 Subject 1 Subject 3 Subject 5 Direct Observation Display Supervisory Control Computer System Control Interface Sensors Results: Separation Violations ü 5 operational errors observed in 36 scenario runs o All occurred in the non-datalinked configuration 1000 5 4 1 Vertical eparation (ft) Vertical SSeparation (ft) 800 600 400 2 00 0 0 Conflict precipitated by a late deviation around weather through same hole in weather occurred outside the sector (turned in wrong direction) (wrong A/P mode for descent) 2 Several aircraft diverting 3 A/C not handed off; conflict 4 Pilot blundered 12 1 2 3 4 Lateral Separation (nm) 3 5 5 Pilot blundered Direct Observation Display Supervisory Control Computer System Control Interface Sensors Results: Separation Violations s1: total separation < 100 feet ...
View Full Document

Ask a homework question - tutors are online