Transcript - Lorentz Center

Mutual Empowerment
in Human-Agent-Robot Teams
16 December 2010
HART Workshop
Jurriaan van Diggelen
Problem statement
• Achieve more with less people
• Automation can help to:
– Make better use of available semi-structured
information sources
– Support decision makers in dealing with the
complexity of problems (war amongst the
people)
The big number cruncher
Sensor data
Twitter data
Problem solution
UAV images
• Monolithic approach, BNC replaces
existing infrastructure
• AI-complete
Towards a human-machine team
solution
• Solution must be provided by a human
machine team
• Mutual empowerment seeks to improve
team performance by:
– Compensating weaknesses of humans and
machines
– Optimizing strengths of humans and
machines
Types of Mutual Empowerment
Intelligent Interfaces
human
machine
CCI
Collective
Intelligence
HMI
CMI
human
HMI
machine
User empowerment
Distributed
Artificial
Intelligence
Goal
ME handbook
Methodology
Domain
Exploration
•Domain
•Human Factors
•Technology
Validation
•Mixed reality validation
•Data collection
Functional
design
•Use cases
•Claims
•Cognitive requirements
•Ontologies
•Performance measures
•Tests/benchmarks
Tool
support
Prototyping
•System requirements
•Functional modules
•RDF interface specifications
•Prototypes
Situated Cognitive Engineering
• Methodology supports
– Incremental design
– Reuse of earlier work (Prototypes, tests,
requirements, use cases)
– Collaborative development
Example
Phase 1: domain exploration
• Domain
– USAR
– UGV, UAV
– Operators in field
• Human Factors
– Maintaining situation awareness
– Cognitive overload
– Adaptive teams
• Technology
– Collaborative tagging, crowd sourcing
– Mixed initiative systems
– Adaptive/ adaptable automation
Phase 2: Functional design (1)
• Use cases
UC 23
• UAV classifies camera image as victim with certainty-level Unsure
• Operator of Robot1 is notified of the potential victim and views the
camera images
• Operator of Robot1 classifies the image as victim with certainty level Certain
• Operator of Robot2 is notified about the victim
•…
• Cognitive requirements
CR 5.1 Uncertainty management
Operators and agents can publish and change the certainty value
of information
Use cases: UC 23
• Claims
CR 5.1
• + improves situation awareness of operators and agents
• - increases cognitive taskload
Phase 2: Functional design (2)
something
• Ontologies
action
event
item
robot
victim
• Performance measures
– E.g. situation awareness measure
• Tests/benchmarks
– Test for evaluating performance
Phase 3: Prototyping
• Develop system requirements that
implement the cognitive requirements.
• Bundle system requirements in functional
modules.
• Reuse existing base platform
Trex
Trex
• Filter: which
data do you
want to see?
selection of
semantic tags in
Sparql
• Projection: How
do you want to
see the data?
graphical object
with attachmentpoints for
semantic tags
Functional modules supported by Trex
•
•
•
•
•
•
User configurable information filters
User configurable information visualization
Realtime semi-structured data exploration
Collective relevance assessment
Uncertainty management
Human-in-the-loop AI
Human-in-the-loop AI
P
Q
Human
R
Machine
S
Crowd
T
Machine
DEMO
Future work
• Develop functional modules for:
– Joint conflict resolution
– Adaptive Interruptiveness
– Network awareness
– Policy awareness
– Capability awareness
– Activity awareness
Conclusion
• Mutual Empowerment library provides a
flexible way to
– Increase application possibilities of AI
– Employ potential of collective intelligence
– Reuse and structure our knowledge of
human-machine collaboration tools
Domain Analysis
ies
terfaces
Use cases
Technology
Investigation
Exploration
Human Factors
Claims
Metrics
Tests
Cognitive
Requirements
Core functions
System
Requirements
Functional
design
Prototyping
Prototype
al modules
Simulation
Test participants
Empirical results
Testing