Fleming Committee Members Dr Amy Pritchett Chair Dr Karen Feigh Dr Ute Fischer Sponsored by the FAA Tom McCloy as Technical Monitor DEVELOPING A TRAINING PROGRAM FOR THE TRAFFIC ID: 810991
Download The PPT/PDF document "Elizabeth “Scottie-Beth”" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Elizabeth “Scottie-Beth” FlemingCommittee Members:Dr. Amy Pritchett, ChairDr. Karen FeighDr. Ute FischerSponsored by the FAA, Tom McCloy as Technical Monitor
DEVELOPING A
TRAINING PROGRAM
FOR THE
TRAFFIC
ALERT AND COLLISION AVOIDANCE SYSTEM
IN
CONTEXT
Slide2OverviewIntroduction to TCAS and Background InformationDesign of Training ProgramEvaluation of the Training ProgramResultsConclusions2
Develop a training program intended to improve pilots’ understanding of TCAS use for collision avoidance in a range of traffic situations
Slide3Traffic alert and Collision Avoidance System (TCAS)
3
Pilot always shown the Traffic Situation Display (TSD)
TCAS delivers a two stage advisory and vertical avoidance maneuver
Traffic Advisory (TA) - ‘Traffic
Traffic
’
Resolution Advisory (RA) - ‘Climb
Climb
’
Pilot is to follow an RA, even if it conflicts with ATC instructions, unless the pilot believes that safe flight would be jeopardized
Federal Aviation Administration (2000).
Introduction to TCAS II Version 7.
Washington, D.C.
Slide4Collision Avoidance in a Broader ContextPilots don’t only interact with TCAS, they also receive information from ATC and the environment4
Pilot’s Response
Advised Maneuvers for Collision Avoidance
Notification and Awareness
Awareness of other aircraft via TSD
TCAS Traffic Advisory
TCAS Resolution Advisory
ATC Traffic Callout
Visual acquisition of other aircraft
ATC Maneuver
Compliance to TCAS
Compliance to ATC
Personal Assessment and Maneuver Decision
Party-line Information
Slide5Current TCAS Training Requirements: Ground TrainingClassroom or computer based
>60 training requirements
Measures performance through quizzes and/or activities
5
To comply to the RA,
you should
Pull the stick back
Push the stick forward
Federal Aviation Administration (2001).
Advisory Circular No. 120-55B: Air Carrier Operational Approval and Use of TCAS II
. Washington, D.C.
Slide6Current TCAS Training Requirements: Flight Training6
Slide7Observed TCAS Use and OperationThe TCAS Operational Performance Assessment (TOPA) monitored the terminal area of 8 major airports and examined pilot compliance to climb and descend advisories
7
41
%
to
Climb
RA’s
59
%
to
Descend RA’s
TOPA observed compliance rates of
Misunderstanding/ Confusion
Aggressiveness
Non-Compliance
Olson, W. and J.
Olszta
(2010).
TCAS Operational Performance Assessment in the U.S. National Airspace.
IEEE/AIAA Digital Avionics Systems Conference
.
Olszta
, J., & Olson, W. (2011). Characterization and Analysis of Traffic Alert and Collision Avoidance Resolution Advisories Resulting for 500' and 1,000' Vertical Separation. Paper presented at the
Ninth USA/Europe Air Traffic Management Research and Development Seminar (ATM 2011), Berlin, Germany.
Slide8Observed TCAS Use and Operation8
“[My
FO and
I] incorrectly
interpreted the red 'above' target on the VSI and responded improperly. We further reviewed the procedures, agreeing that a person should fly 'away' from the red VSI indication, if instructed via RA
.”
ACN:785761
,
2008
Pilot reports flying into the red region on the VSI
Non-Compliance
Aggressiveness
Misunderstanding/ Confusion
NASA. (2009). Aviation Safety Reporting System. Retrieved August 1,
2010: http
://asrs.arc.nasa.gov/
“Descending
into an airplane that is clearly descending? TCAS software clearly did not give appropriate guidance, nor did it self-correct when the initial guidance was so clearly wrong
”
ACN
: 854982, 2009
Pilot disagrees with advised descend RA
Slide9Observed TCAS Use and Operation9
Non-Compliance
Misunderstanding/ Confusion
Aggressiveness
Far Eastern Air B757 Response to Descend RA
(TCAS advised a descent rate of 1500 FPM)
Image copied from
Lacagnina
(2008). Easy Does It.
Aero Safety World
: http
://
flightsafety.org/asw/oct08/asw_oct08_p44-47.pdf?dl=1
Slide10Research Statement and ObjectivesDevelop a training program intended to improve pilots’ understanding of TCAS use for collision avoidance in a range of traffic situations
10
(1) Train
pilots to understand TCAS use for collision avoidance in the actual traffic and operational traffic environment
(2)
Provide pilots with a well-rounded knowledge of different traffic situations that may result in TCAS advisories
(1) Train
pilots to understand TCAS use for collision avoidance in the actual traffic and operational traffic environment
(2)
Provide pilots with a well-rounded knowledge of different traffic situations
that may
result in TCAS advisories
Slide1111Demonstration Based Training
Event Based Training
Approach to Training Design
Complement
of Two
Methods
Slide12Demonstration Based Training (DBT)Computer-Based Training~25 minutesProvides conceptual understanding of TCASOutlines rules for compliance6 segmentsIntroduction to TCASTraffic Situation DisplayTCAS Advising LogicTraffic AdvisoriesResolution Advisories
Example Timeline of RA Evolution
Mid-Training Quizzes
12
Slide13DBT: Demonstrations of RA’s13
Slide14Event Based Training (EBT)Presents traffic events that create the requirement to actBuilds context and complexity into each scenario as the flight progressesUses a more realistic training environment
Fowlkes
, J., Dwyer, D.,
Oser
, R., & Salas, E. (1998). Event-Based Approach to Training (EBAT).
The International
Jounal
of Aviation Psychology
, 8(3), 209-221.
14
Slide15Air Traffic Transcripts
Charts & Checklists
Audio Communications (Aviation Intercom)
TCAS Alerts
Simulation Architecture
Eyetracker
PartyLine
First Officer
Captain
Experimenter/Instructor
Coded Log of Flights
B747-400 Simulator (RFS)
TSD
PFD
ND
PFD
ND
Touch
screen
SideStick
ATC->TSD
VGA
ATC
Air Traffic Simulator (TGF)
TCAS Logic
15
Simulator Study in Integrated
Flightdeck
– ATC Environment
Pritchett, A., Fleming, E.,
Cleveland,
W.,
Zoetrum
, J.,
Popescu
, V., &
Thakkar
, D. (2012). Pilot Interaction with TCAS and Air Traffic Control. Paper presented at the
2nd International Conference on Application and Theory of Automation in Command and Control Systems (ATACCS)
, London.
Slide16Structure of Events: Descend RA Example
16
Training Objective
Context of Event
Performance Measures
Feedback Session
Accurate interpretation of and response to TCAS Descend RA.
Instrument Meteorological Conditions (daytime, clouds, no winds
)
ATC
provides no traffic
information
Conflict
caused by IFR traffic
enroute
(most likely on departure
)
RA
maneuvering should not violate ATC
instructions
No
conflicting ATC or party-line information
Pilot disengages autopilot and flight directors
If the pilot did not meet any particular performance measure, review the correct response in regards to that measure
Pilot responds to advisory with appropriate vertical speed
Pilot ensures vertical speed is not excessive
Pilot notifies ATC of response to TCAS advisory as the maneuver is performed
Pilot reengages autopilot and flight directors
Pilot notifies ATC of TCAS advisory and response after clear of conflict
Pilot returns to original clearance (if needed)
Slide17Evaluating the training program’s impact
Slide18Baseline StudyTraining Study6 scenarios with 2 traffic events per scenarioTraffic events defined byRA typeATC informationTraffic densityOverview of Study
18
Comparison to identify the impact of training
Slide19Overview of Study
Pre-Training Data Collection
(50
minutes)
Pre-Experiment Questionnaire
Pre-Experiment Quiz
TCAS Training Program
(50
minutes)
Introduction to TCAS
Demonstration Based Training
Short Section QuizzesEvent Based TrainingEvaluating TCAS Training Program(120 minutes)
Flight Scenarios
Post Scenario Questionnaires
Debrief
(15 minutes)
Post- Experiment Questionnaire
19
Slide20Skill-Based BehaviorRule-Based Behavior
Knowledge-Based Behavior
20
Skill-Based Behavior
Rule-Based Behavior
Knowledge-Based Behavior
Does the training program improve pilot
performance in response to TCAS advisories and increase pilot understanding of TCAS?
Slide2121
Does the training program improve pilot
performance in response to TCAS advisories and increase pilot understanding of TCAS?
Decrease time to achieve compliance
Decrease time to disconnect autopilot
Reduce aggressive response features
Increase percentage of RA duration in compliance
Increase appropriate response post-Clear of Conflict
Increase understanding of TCAS
Increase trust in TCAS
Skill-Based Behavior
Rule-Based BehaviorKnowledge-Based Behavior
Slide22Assessing the Pilot’s Response: Skill-Based Behaviors22
Vertical Speed
time
5 seconds
RA Climb
2 ½ sec
TCAS
w
eakens
r
equired
vertical rate
TCAS assumed
¼ g pull-up
Clear of Conflict
TCAS assumed constant vertical RA rate
Autopilot Disconnect Time
TCAS RA
Maneuver
Pilot’s Response
Time Pilot First Achieves Compliance
Skill-Based Behavior
Rule-Based Behavior
Knowledge-Based Behavior
X
Slide23Mean Time to Comply: During Training23
Skill-Based Behavior
p
MM
= Significance for the mixed model
p
s
2
= Significance of the variance
p
m
= Significance of the meansRule-Based BehaviorKnowledge-Based Behavior
Slide24During training, no significant differencesPost trainingAutopilot disconnect time decreasedPilot response was more consistent for one eventAutopilot Disconnect: Post Training24
Skill-Based Behavior
Rule-Based Behavior
Knowledge-Based Behavior
Slide25Impact of Training Program25
Decrease time to achieve compliance
Decrease time to disconnect autopilot
Reduce aggressive response features
Increase percentage of RA duration in compliance
Increase appropriate response post-Clear of Conflict
Increase understanding of TCAS
Increase trust in TCAS
Skill-Based Behavior
Rule-Based Behavior
Knowledge-Based BehaviorSkill-Based BehaviorRule-Based Behavior
Slide26Assessing the Pilot’s Response: Aggressiveness26
Vertical Speed
time
5 seconds
RA Climb
2 ½ sec
TCAS
w
eakens
r
equired
vertical rate
TCAS assumed
¼ g pull-up
Clear of Conflict
TCAS assumed constant vertical RA rate
Maximum vertical rate
Maximum vertical rate difference
V
ertical rate difference
TCAS RA
Maneuver
Pilot’s Response
Skill-Based Behavior
Rule-Based Behavior
Knowledge-Based Behavior
*Altitude Deviation
Slide27Assessing the Pilot’s Response: Compliance27
Vertical Speed
Not in Compliance
In Compliance
TCAS RA
Maneuver
Pilot’s Response
time
5 seconds
RA Climb
2 ½ sec
TCAS
w
eakens
r
equired
vertical rate
TCAS assumed
¼ g pull-up
Clear of Conflict
TCAS assumed constant vertical RA rate
Skill-Based Behavior
Rule-Based Behavior
Knowledge-Based Behavior
Slide28Assessing the Pilot’s Response: Return to Clearance28TCAS RA Maneuver
Pilot’s Response
TCAS assumed constant vertical RA rate
TCAS weakens required vertical rate
Clear of Conflict
Altitude
time
5 seconds
RA Climb
TCAS assumed
¼ g pull-up
Cleared altitude at time of RA
Pilot holds new altitude
Contacts ATC and request new clearance or ask for further instructions
Pilot begins descent back to originally cleared altitude
May inform ATC of response to RA
Skill-Based Behavior
Rule-Based Behavior
Knowledge-Based Behavior
Slide29Before Training89% of the pilots did not know that an RA should cause less than 500 feet of altitude deviation56% of the pilots responded that they would hold current altitude achieved after responding to an RA, as opposed to returning to their clearanceWhen asked about airline procedures for following TCAS, all 18 pilots noted the need for complying with an RABUT 28% of the pilots commented compliance wasn’t necessary is there was a TCAS “malfunction” or if the RA would cause an “unsafe situation”
29
“[Pilots] must
always comply with a TCAS RA unless
[aircraft] performance
is hindered (
i.e.
operating single engine) or
[there is] an
obvious TCAS malfunction (
ie
you can see traffic and it is not a threat)” Skill-Based Behavior
Rule-Based Behavior
Knowledge-Based Behavior
Slide30During training, aggressive response features decreased Post training, same decreasing trend observedAggressiveness30
Altitude Deviation
Average Vertical Rate Difference
Maximum Vertical Rate Difference
Maximum Vertical Rate Difference
All decreased
Skill-Based Behavior
Rule-Based Behavior
Knowledge-Based Behavior
Slide31During training, percentage compliance decreased in training event with “Climb RA” (93.1% compared to 99.6%)Post trainingNo significant differences in means observedBut, trained pilots had a more consistent responsePercentage Compliance31
Skill-Based Behavior
Rule-Based Behavior
Knowledge-Based Behavior
Slide32Return to Clearance32
Skill-Based Behavior
Rule-Based Behavior
Knowledge-Based Behavior
Pilot returns to original clearance
No
Yes
Slide33Impact of Training Program33
Decrease time to achieve compliance
Decrease time to disconnect autopilot
Reduce aggressive response features
Increase percentage of RA duration in compliance
Increase appropriate response post-Clear of Conflict
Increase understanding of TCAS
Increase trust in TCAS
Skill-Based Behavior
Rule-Based BehaviorKnowledge-Based BehaviorRule-Based BehaviorKnowledge-Based Behavior
Slide34Before Training94% pilots agreed with statement “I understand TCAS maneuvers when they are issued”, but…Only 50% of the pilots correctly identified the assumptions made by TCAS advisory logicWhen asked to interpret TSD symbols, only 50% of the pilots got all parts of the associated questions correct
34
Skill-Based Behavior
Rule-Based Behavior
Knowledge-Based Behavior
Slide35“My understanding of TCAS has increased”
50% pilots reported an increase in understanding TCAS logic
27% pilots claimed to have learned about different types of TCAS RA’s (notably, “Crossing RA’s”)
Post Training Understanding of TCAS
35
“Types of RA's
were
not previously taught. We were taught simply to comply”
Strongly
Disagreed
0
Disagreed
0
Neutral
6
%
Agreed
61%
Strongly
Agreed
33%
Skill-Based Behavior
Rule-Based Behavior
Knowledge-Based Behavior
Slide36Post Training Trust in TCAS
“
I am more likely to trust TCAS after completing today’s training
”
36
“My trust in TCAS was already at a maximum so I wouldn’t be ‘more’ likely to trust it”
Strongly
Disagreed
11%
Disagreed
0
Neutral
39%
Agreed
39%
Strongly
Agreed
11%
Skill-Based Behavior
Rule-Based Behavior
Knowledge-Based Behavior
Slide37Post-Training37 Decrease time to achieve compliance
Decrease time to disconnect autopilot
Reduce aggressive response features
Increase percentage of RA duration in compliance
Increase appropriate response post-Clear of Conflict
Increase understanding of TCAS
Increase trust in TCAS
Slide38Conclusions and Future WorkConclusionsPilots may not need more training, but instead need better trainingIntegrating DBT and EBT methods permits ground-based and flight training material to be more cohesiveEBT structure allows for the design of purposeful training eventsCurrent FAA mandated TCAS training objectives may not fully reflect all training areas Language needed to address the reduction of excessive responses to RA’s
Future Work
What facilities and technologies would be required to implement this type of training program?
What implications arise when considering training design versus system design?
Would incorporating human factors considerations in initial design stages decrease the amount of required training?
38
Slide3939
Slide40Conclusions and Future WorkConclusionsPilots may not need more training, but instead need better trainingIntegrating DBT and EBT methods permits ground-based and flight training material to be more cohesiveEBT structure allows for the design of purposeful training eventsCurrent FAA mandated TCAS training objectives may not fully reflect all training areas Language needed to address the reduction of excessive responses to RA’s
Future Work
What facilities and technologies would be required to implement this type of training program?
What implications arise when considering training design versus system design?
Would incorporating human factors considerations in initial design stages decrease the amount of required training?
40
Slide41AcknowledgementsWork sponsored by the FAA, Tom McCloy as Technical Monitor 34 pilot participantsDr. Amy PritchettDr. Karen FeighDr. Ute FischerDr. Wesley Olson, MIT Lincoln LabsWayne Gallo, FAARoger Sultan, FAAKylie Garey
TCAS Team: William Cleveland,
Vlad
Popescu
, Justin Mullins,
Anil
Bozan
, Henry Tran, Jack
Ridderhof, Alyssa Whitlock, Colin Ludwig, Dhruv Thaakar
, Jonathan Zoetrum, Jelle Wissink
CEC Lab Members41
Slide4242