Transcript Document 17643

Data Mining Combat Simulations:
an Emerging Opportunity
Barry A. Bodt
[email protected]
(410) 278-6659
Computational and Information Sciences Directorate
Army Research Laboratory (ARL)
The U.S. Army’s Corporate Laboratory
Motivation
• Simulation and statistical analysis are underutilized
in helping the commander’s staff to analyze
courses of action.
• Battle results are infinite in scope, yet the outcome
of any one battle is defined by a unique set of
battlefield interactions.
• Key is to recognizing those interactions through
development of more informative performance
measures unique to the scenario at hand.
Approach
Use statistical methods
and combat models to
create a methodology
that identifies nontraditional metrics for
plan evaluation.
Background
Military Decision Making Process
•Focus on wargame
•Disciplined rules
•Synchronization matrix
COA
Joint Tactical Operation Center, Qatar
Network Centric Warfare
Communicate…
Smart Logistics
On-board Diagnostics
Soldier Health
Sensor information
…
Information Requirements in NCW
The key to any analysis is the set of measures used to
represent the performance and effectiveness of the
alternatives being considered. We are relatively good at
measuring the performance of sensors and actors, but
less adept at measuring command and control.
Command and control, to be fully understood, cannot
be analyzed in isolation, but only in the context of the
entire chain of events that close the sensor-to-actor
loop. To make this even more challenging, we cannot
isolate on one target, or even a set of targets but need
to consider the entire target set. Furthermore, network
centric warfare is not limited to attrition warfare … It is
not sufficient to know how many targets are killed, but
exactly which ones and when…
Ref: Network Centric Warfare, 2002
Simulation Data
•
•
•
•
Scenario development
OneSAF lay down of forces
OneSAF modified output
Data supporting modeling
Scenario
BMP-2
BMP-2
BMP-2
T-72M
T-80
T-72M
T-72M
Town
T-72M
T-80
T-72M
T-80
T-80
Company Objective
OneSAF Screen Dump
Automated Data Collection
• OneSAF Modifications
OBJECT_ID: 100A31
X = 24396.82 Y = 25828.75 Z = 755.72
Vehicle Authorized Undamaged Catastrophic Firepower
Damage
Damage
M2
1
0
1
0
Equip/Supplies:
Current Lvl Resupply Lvl Avg Per
25mm HE (M792)
625.00
625.00
625.00
25mm APFSDS-T (M919) 325.00
325.00
325.00
TOW (TOW)
0.00
5.00
0.00
7.62mm MG (M240)
2340.00
2340.00
2340.00
Fuel (Fuel) (gallons) 171.00
174.00
171.00
Mobility
Damage
0
Veh
OneSAF Modification
Killer/Victim Scoreboard
Time Stamp 1010070890
Vehicle ID 1076
Firer ID 1087
Projectile 1143670848
Firer Position:
•
•
•
•
Firer and Target Identity and Location
Type of Ammo
Range
Outcome
X = 220217.00
Target Position:
X = 222454.38
Y = 146765.00
Y = 149117.80
Z = 12.37
Z = 9.99
Vehicle 1076: Hit with 1 "munition_USSR_Spandrel" (0x442b0840)
Comp DFDAM_EXPOSURE_HULL, angle 19.53 deg Disp 0.889701 ft
Kill Thermometer is: Pk:1.00, Pmf:1.00, Pf:0.90, Pm:0.80 Pn:0.80
RANGE
3246.773576
r = 0.990835 kill_type = MF
1076 100A41 vehicle_US_M1
1087 100A23 vehicle_USSR_BMP2
Data Supporting Classification Models
Response – mission
• 228 OneSAF runs
accomplished (success)
• 3 situational snapshots per
if an undamaged platoon
run
occupies objective at
– 10% blue ammo expended
battle end (MA)
– 25% blue ammo expended
– 40% blue ammo expended
• 429 data points per run (143 -other responses include
MBT and “Eric” strength
per stopping time)
– Number of K, M/F, F, and M kills
and forces on objective
–
–
–
–
–
–
Ammunition levels
Number of hits delivered
Range of hits
Number of side hits delivered
Distance to objective
Number of Blue on objective
Data Matrix
228 x 434
Model Performance
Pred
Pred
Obs
Obs
Slice 1 ~ 2000m
Slice 2 ~ 4000m
Or ~ 5 ½ minutes
Or ~ 10 minutes
Slice 3 ~ 5800m
00 11
34
00 85
98
21
Pred
0 741
Obs
11 35
25 84
0 105 14
1 20 89
Or ~ 20 minutes
Company
Company Objective
Objective
Correctly Classified
Correctly Classified
Loss: 71%
Loss: 82%
Win: 68%
Win: 77%
Correctly
Overall:Classified
70%
Overall:
80%
Loss: 88%
Win: 82%
Overall: 85%
Method Comparison
Percent Correct Classification
by Stopping Time and Method
Stopping Discriminant CART Logistic
Time (min)
Analysis
Regression
5½
70%
70%
69%
10
80%
75%
74%
20
85%
82%
85%
Advantages
– Support prediction for COA performance
evaluation
– Provide models identifying key battle parameters
for a given engagement, influencing both COA
development and commander’s critical
information requirements
– Input to CCIRs
– Input to contingency plans
– Input to tolerances for synchronization
Implementation Models
Reach back
Advantages
-computational power (ARL 9th)
-more complex analyses
Distributed
Disadvantages
-latency
-can’t smell gunpowder
Advantages
-cheaper boxes (250 OneSAF
boxes used at Ft. Leavenworth)
-closer to action
Disadvantages
-depth of a field analysis
-automation required
Why Aren’t We Already Doing This?
A few reasons …
• Computer simulation focus has been mainly strategic or
oriented toward acquisition. Tactical application has been
limited.
• Simulations did not have high enough fidelity for tactical
application.
• Simulations were unstable.
• Computing resources were inadequate.
• Necessary communication of inputs had not been
imagined.
• Simulation creators do not always talk to statisticians.
Improvements Here and On the Way
• Stability
• Power Point force laydown of forces
• MS Word OPORD
•Terrain, weather wizzards
• Composable simulations
• After Action Report data
• Man-in-loop allowed
• Sensor advances
• Communication advances
• Computation speed and cost
Catching On?
After Action Review
PURPOSE: The OneSAF After Action Review
component provides the capability to correlate, rollup, and analyze simulation outputs and visualize the
results of the simulation exercise. The toolset allows
the analyst to preplan the AAR prior to exercise
execution.
• Situation awareness during the execution of the exercise and
afterwards during exercise playback:
– PVD & 3D Stealth display
– Statistical charts, tables
– OPORD paragraphs
– Task Organizations Summaries
– Radio/audio playback (Future)
• Mining of collected data to construct MOPs/MOEs
• Automatically build AAR presentations & Take Home Package
using COTS Office Automation
20
Next Up
Wei-Yin Loh, Regression Tree Analysis of Battle Simulation Data
David Kim, Robust Modeling Based on L2E Applied to Combat
Simulation Data
Warren Liao, Discovery of Battle States Knowledge from MultiDimensional Time Series Data