Transcript Information Analysis

Information Analysis
Knowledge Mining/Threat Prediction
SMU
CSE 8394
Organizational Knowledge
Knowledge
Knowledge we
have and don’t
know we have it
and do not use it
Knowledge we
have and know
we have it and
use it
Meta-Knowledge
Discover!!
SMU
Knowledge don’t
we have and we
know that
X
CSE 8394
Knowledge Discovery
Knowledge Discovery
Link Analysis
Data Mining
Unstructured Data
SMU
CSE 8394
Data Mining
• Process of extracting hidden knowledge from raw
data
–
–
–
–
–
Targeted marketing
Cross-sell opportunities
Customer profiling
Supply chain?
Power, traffic distribution
• Why data mining
– Huge volumes of data
– Limitations of human analysis
– Low cost of machine learning (bigger trend that
outsourcing!!)
SMU
CSE 8394
DM Methods
•
•
•
•
•
•
Neural Networks
Evolutionary Programming
Memory Based Reasoning
Decision Trees
Genetic Algorithms
Nonlinear Regression Methods
SMU
CSE 8394
Graph-Based Link Analysis
• The “friends” problem
– Understanding the underlying relationships
– Prediction of future relationship
• Premise
– Entities connected through noisy relationships
• Question to answer
– Given the collaborative graph, who are the top K
friends?
– Who could be friends in future?
SMU
CSE 8394
Text Mining with Link Analysis
• Reported experiments on
– Connection between a terrorist and known figure
– Meetings between US and Israel
• Approaches to link construction
– Co-occurrence links
– Semantic links
• Quality metrics
– Accuracy
– Recall
SMU
CSE 8394
Link Discovery in Battlespace
• Information fusion should progress beyond
current level 1 (object refinement)
• Focus on Continuous Intelligence
Preparation of the Battlespace (CIPB)
– Develop DA to aid human analyst to determine
enemy COA
– Leverage technologies such as mobile agent
architecture, link discovery and analysis
• Sample scenario analysis
SMU
CSE 8394
Related Message Traffic
• Constructing conversations when
conventional markers are missing
– Look for statistical anomalies
– Application of SVD and ICA (Independent
Component Analysis) to noun frequency profiles
• Application in counter terrorism and fraud
SMU
CSE 8394