Transcript What is Data Mining?

Phase of Flight
&
Rule of Flight
Calculator
By: Cliff Johnson,
FAA
Adrian Rusu, Anthony Breitzman,
Nicholas Laposta, John Bucknam,
Rowan University
Presented to: ICRAT 2016
Jun. 21, 2016
Drexel University,
Philadelphia, PA
Federal Aviation
Administration
Overview
•
Introduction
•
Motivation
•
Direction
•
Phase/Rule of Flight Background
•
Phase of Flight/Rule of Flight Calculator
•
Case Study
•
Future Work
•
Conclusion
•
Questions and Answers
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
2
Introduction
• Federal Aviation Administration
– Cliff Johnson, who was the project lead and collaborator between Rowan
University and the Federal Aviation Administration (FAA).
• Rowan Collaboration
– 5 undergraduate students (Nick Laposta) under Dr. Adrian Rusu using
the Federal Aviation Administration’s equipment and data.
– Dr. Anthony Breitzman who helped us create the algorithms used
through data mining.
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
3
Motivation
• Increase safety by improving analytical
tools for the safety analyst.
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
4
U.S. Helicopter Fatal Accident & Fatality Rates
*CY 14,
Jan-Jul Only
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
5
5
Direction
• Q: But how do we achieve a lower or nearzero accident rate?
• A: Proactively identify risk via projects that
research methods and tools such as data
mining of flight data monitoring data and
FAA surveillance (i.e. aircraft tracking) data.
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
6
Concept of Rotorcraft ASIAS
Rotorcraft ASIAS
All Missions
Flight Training
Charter
EMS
Oil/Gas
Logging
Police
News
Cargo Lift
…
Participating Operators
Own-Access User
FAA, Sponsor
Secure
System
Database
Analysis Toolkit
Interface & Display
Aggregate
Results
De-identified Data Access User
Analysis
Tools
System
Developer &
Administrator
Visualizations,
Algorithms,
Event
Definitions
PEGASAS Research Team for Rotorcraft ASIAS
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
7
Rotorcraft Research for ASIAS
 Industry Involvement
 Potential Mitigation Partner
 FDM Equipment Working
Group
 Program
Management
 Outreach
 Flight Tests
 HFDM Device
Integration &
Calibration
 Data Analysis
 Modeling & 
Simulation 



ASIAS
Rotorcraft
Analysis
Capabilities
Outreach
Data Transcription
HFDM Research Repository
HFDM Architecture Design
HFDM Working Group
 Safety Metrics
 HFDM Analysis
Techniques
 HFDM Modeling
Techniques
 HFDM Cockpit
Audio/Video Analysis
 Data Mining for Safety
Events
 Identification of High-Risk
Safety Events
Supports the USHST Goal of 20% Reduction in
the Fatal Accident Rate for Helicopters by 2020
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
8
Rotorcraft ASIAS
Research Team Members
Ed DiCampli, Chief Operating Officer– PI
Harold Summers, Director of Flight Operations
Jay Clark, Manager, Information Systems
Kipp Lau, RASIAS FDM Consultant
Robert Liguori, RASIAS IT Consultant
Keith Cianfrani, FDM Specialist/Outreach
Prof. Dimitri Mavris – PI
Hernando Jimenez, Ph.D. – Co-PI
Kyle Collins
Simon Briceno
Alek Gavrilovski
Alexia Payan
Aviation Research Division, ANG-E
Cliff Johnson – Rotorcraft ASIAS
Research Task Lead
FAA WJHTC Atlantic City, NJ
Alex Ahlstein – ASIAS Group Leader
Mike Yablonski – Principal Aviation Systems
Engineer
MITRE-CASSD
*Note: MITRE role involves collaboration with research
activities for integration with overall ASIAS.
Prof. Karen Marais – PI
Arjun Rao – Co-PI
Inseok Hwang
Arjun Rao
Sanghuyn Shin
Tyler Travis
Lana Manovych
Jean-Christophe Geffard
Steve Cullen
Scott Sunder
FOR OFFICIAL USE ONLY
Prof. Stephen Cusick – PI
Keith Cianfrani
Nicholas Currie
Federal Aviation Administration
9
Why Rotorcraft ASIAS?
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
10
Federal Aviation Administration
10
Background
• How do we determine Phase of Flight
Detection and Rule of Flight Detection and
implement them into the prototype for
Rotorcraft HFDM research for ASIAS?
• Two approaches:
– 1. Examine fixed-wing aircraft tracking data (focus of
the initial effort)
– 2. Examine rotorcraft flight data monitoring data
(focus of current and future efforts)
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
11
Background
•
Currently, the only way for an analyst to determine the Phase of Flight
(PoS) of an aircraft is to either look at the flight plan or depend on a
signal received from the aircraft
Rule of Flight is important when analyzing traffic and can change
multiple times throughout the flight. A pilot can switch between Visual
Flight Rules (VFR) and Instrumental Flight Rules(IFR) depending on
weather conditions and flight status.
Our system aims to determine general phases to help analysts determine
possible faults in procedure, manufacture, or operation of aircraft in
post-accident analysis
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
12
Background (Conflicts)
•
There are many different phases that an aircraft can be in at any given
time and this quantity can be even greater when also taking rotorcraft
into account.
•
We looked to create a simple classification tool that can be expanded
upon in future research.
•
Although an aircraft broadcasts its Rule of Flight constantly the signal
will not always be picked up by primary radar. Therefore, we are
attempting to determine the Phase and Rule of FLight when only given
primary radar.
•
It is possible that different airports may have different protocols that
require aircrafts to be in different position, however this model should
be feasible for any airport with minimum changes, possibly using
parameters.
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
13
Phases & Rules of Flight
•
•
•
•
•
•
•
•
•
•
•
•
•
Phases of Flight
Standing
Pushback/Towing
Taxi
Takeoff
Initial Climb
En Route
Maneuvering
Approach
Landing
Emergency Descent
Uncontrolled Descent
Post-Impact
Unknown
•
•
•
Aviation Research Division, ANG-E
Rules of Flight
Visual Flight Rules – Thousands
of feet plus 500 feet. Odd number
thousands of feet (2500, 3500,
4500, etc.) within our Altitude
data. We also found that aircraft
points with very low altitude were
more likely to be VFR as well
based on predefined points.
Instrument Flight Rules - Even
number of thousands of feet
(2000, 3000, 4000, etc.) within our
Altitude data. The exception to
this is near the airport
environment where 1000 feet is
typically the altitude of the VFR
traffic pattern.
Unknown - In the case that our
data was flawed or couldn’t be
defined as VFR or IFR.
FOR OFFICIAL USE ONLY
Federal Aviation Administration
14
Initial Case Study
In our research, we used the National Offload Program (NOP) data taken
from different locations, such as the Atlantic City International Airport
(ARTS). Although we were given other locations as well, such as
Philadelphia International Airport, we found it easier to focus on one
location, especially the amount of data given which as ~7 million data
points.
With these points, we chose to focus on the most commonly defined
columns within our database, such as the altitude and speed of a given
aircraft at a given point, as well as our own custom column called
Aircraft ID (ACID).
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
15
Sample Data
F3_Date
F4_Time
F5_
Callsign
F6_
AcType
10/01/2015
00:00:00.0
02
4004
280
274
193
34.95895
-85.46546
592
10/01/2015
00:00:00.0
33
3602
342
469
326
32.16565
-82.83178
499
10/01/2015
00:00:00.0
61
1461
350
464
332
34.61182
-85.05031
478
10/01/2015
00:00:00.0
61
7466
410
566
62
34.90963
-85.28257
216
10/01/2015
00:00:00.1
33
RBJ67
MD88
640
103
262
94
33.53683
-84.48752
205
10/01/2015
00:00:00.1
33
K4QC
C150
5227
42
84
60
32.54313
-86.00119
220
Aviation Research Division, ANG-E
F8_BeaconC
ode
F9_
Altitude
F10_
Speed
FOR OFFICIAL USE ONLY
F11_
Heading
F12_
Latitude
F13_
Longitude
F14_
TrackNum
Federal Aviation Administration
16
Aircraft ID (ACID)
•
The creation of the Aircraft ID was one of the major hurdles for our project as it
was necessary for any progress to be made.
•
We needed to create a unique identifier in order to connect different data points
to one aircraft, and to do this we used a timestamp and a beacon code.
–
Timestamp: Time at which data point was recorded
–
Beacon Code: An identifier used by radar to identify an aircraft at a given time.
•
Using these 2 columns, we looked at time intervals using the same beacon code.
By doing this, we created lists of data points with the same ACID, called tracks
that were used for later analysis by our Phase of Flight/Rule of Flight Calculator.
•
Some issues with this method include the major use of Beacon Code 0000, which
was consistently changing different aircrafts and was against our original model.
However, this should be easily resolved by using more of the given columns.
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
17
Phase of Flight/Rule of Flight Calculator
• Simplified Phases
– We considered the following as phases of flight in our software:
•
•
•
•
•
Ascending
Descending
Cruising
Taxiing
Unknown
– These phases can also be used in order to figure out larger phases as
well, such as landing, approach, or other phases due to possible
emergencies.
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
18
GUI of Calculator
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
19
Basic Model
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
20
Future Work – Transition to
Vertical Flight
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
21
Helicopter Research at the FAA
• Focus: Certification of New
Technologies, Standards for Continued
Airworthiness, Occupant Protection, and
Hazard Avoidance/Risk Mitigation,
Training, etc.
• Two Projects I oversee that require
phase of flight determination:
– Helicopter Flight Data Monitoring
Research for Aviation Safety
Information Analysis & Sharing
(ASIAS)
– Helicopter Advanced Vision Systems
Research
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
22
Data Mining in Aviation Safety
•
FDM/FOQA paradigm is underpinned by the a-priori definition of
known safety events
•
Safety event definition is overtly simplistic
•
Assertion: safety benefits of FDM can be significantly enhanced by
incorporating formal data mining methods.
– avoids, or at least greatly mitigates, the dependence on previously defined safety
events
– Allows for more sophisticated event characterizations commensurate with the
complexities of flight safety
•
Flight Data Mining
– Formal techniques in aviation are not new.
– Systems health monitoring and prognostics for commercial aviation
– Operational safety data mining in FDM is sparsely documented, limited to commercial
aviation.
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
23
What is Data Mining?
• “Data mining […] consists of applying data
analysis and discovery algorithms that,
under acceptable computational efficiency
limitations, produce a particular
enumeration of patterns (or models) over
the data.”
– U. Fayyad, G. Piatetsky-Shapiro and P. Smyth,
"From data mining to knowledge discovery in
databases," AI magazine, pp. 37-54, 1996.
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
24
Unsupervised learning - Clustering
Distance based
Density Based
Hierarchical - Variable Density
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
25
Data Mining for Aviation Safety
Supervised
Unsupervised
Intrinsic similarity - dissimilarity
Training data set
Data in parameter space
Label
Data within each of these groups looks alik
Analysis data set
NEW Data in parameter space Estimate Label
These groups look different
These points look very different from all others!
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
26
Opportunity and Objectives
Opportunity
• Data mining can provide operational safety benefits in the
aviation segments where it is most needed
• Leverage on growing adoption of FDM in GA and Rotorcraft
Objectives
• Establish the best techniques and practices for rotorcraft
safety data mining, addressing unique challenges and features
– What data processing techniques are best suited?
– What data mining techniques are most effective for intended purpose?
– How do we best make sense of, and utilize, results?
• Implement, test, and refine current/new data mining algorithms
– Phase of flight identification
– Anomaly detection
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
27
Phase of Flight -Experimental
Strategy
Prescriptive Approach
• Identify phase of flight
definitions
• Smooth data as needed
• Apply definitions as data
filters
• Revise as needed
Data-driven Approach
Compare
– Is this the right phase? SME
inspection
– Exhaustive – all points are
assigned
– Exclusive – points are
assigned to one phase
• Unsupervised (no
labels) data mining
to entire flights
• Use only parameters
in phase of flight
definitions
• Use all parameters
available
How well do existing phase of flight definitions match actual operations?
Does the answer change based on mission type?
Do we need revised phase of flight definitions for rotorcraft?
What techniques work best?
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
28
Phase of flight – Prescriptive Definition
Rotorcraft phase of flight definition
based on authoritative sources
1. ICAO common taxonomy team – ICAO
2. European coordination center for accident and incident
reporting systems – ECCAIRS
3. FAA helicopter flying handbook – FAA
4. Learning to fly helicopters from Randall Padfield – LOF
5. Principles of helicopter flight from W. J. Wagtendonk POH
Implementation of phase of flight
definition as mathematical inequalities
for parameters
Data processing / smoothing
1.
2.
3.
4.
5.
6.
7.
Low pass filter (FFT with Gaussian filter)
Moving average (backward, center)
Local regression (weights, 1st or 2nd order)
Robust local regression (neglect outliers)
Savitzky-golay (1st or 2nd order, successive fitting)
Piece-wise linear regression
Digitization
… and others
Example: Low Pass
Filter
29
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
29
FAA HFDM Test Flights
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
30
FAA R&D Test Flights
•
Test Platform
•
HFDM / HFDR Devices
•
Research Goals
•
Research Outputs
– FAA’s Sikorsky S-76A Helicopter, Equipped with
ADS-B Out (1090ES)
– Current devices: Appareo Vision 1000, L3 Light
Data Recorder, Honeywell Skyconnect Tracker 3,
Ballard, Stratus, MEMSIC AHRS, iLevil AHRS,
GoPro Cameras (6), General Aviation Airborne
Recording Device (GAARD)
– Additional devices: Skytrac, North FDS/Outerlink
IRIS, Latitude iONode, others
– Identify, examine, and install several different
HFDM/HFDR units
– Collect/process data from each HFDM/HFDR
system for different scenarios/conditions
– Helicopter FDM Data to be used to define and
validate future analysis of events, parameters,
exceedances, recording rates, etc. from anomalous
data
– Installation guidelines and optimal locations for
each system
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
31
FAA R&D Test Flights
• GAARD & Foreflight Mobile Applications during data
collection
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
32
Data Mining with Rowan University’s
CAVE® Virtual Reality Environment
Aviation Research Division, ANG-E
•
The South Jersey
Technology Park at Rowan
University
•
Integrating the CAVE® as
extension of Tech Center
Labs over the NextGen
Prototyping Network (NPN)
•
Analyzing data generated
from FAA Flight Tests using
data mining techniques
(CAVE ® simulations add
realism and offer ability to
reanimate flights and
simulate anomalies inside a
virtual helicopter)
FOR OFFICIAL USE ONLY
Federal Aviation Administration
3333
Aviation Research Division, ANG-E
FOR OFFICIAL USE ONLY
Federal Aviation Administration
34