Transcript SPIE 8758-14 Dave Braines - MIPS

International Technology Alliance in
Network & Information Sciences
SPIE Defense Security & Sensing
Next Generation Analyst
MIPS:
A service-based aid for
intelligence analysis
Dave Braines, John Ibbotson, Graham White
(IBM UK)
Research was sponsored by US Army Research Laboratory and the UK
Ministry of Defence and was accomplished under Agreement Number
W911NF-06-3-0001. The views and conclusions contained in this
document are those of the authors and should not be interpreted as
representing the official policies, either expressed or implied, of the US
Army Research Laboratory, the U.S. Government, the UK Ministry of
Defense, or the UK Government. The US
and UK Governments are authorized to
reproduce and distribute reprints for
Government purposes notwithstanding
any copyright notation hereon.
Agenda
 What is MIPS?
 IT Support for the Intelligence Analysis Process
 MIPS and the International Technology Alliance (ITA)
 The MIPS Architecture and an example of its use
 Meeting the MIPS Objectives
What is MIPS?
Management of Information Processing Services
 Research transition project led by DSTL (Defence Science
Technology Laboratory) UK
Applying emerging technologies arising from the US/UK
ITA research program
• Information Fabric, Gaian Database, Controlled English (CE) & CE Store
Provide a generic service-based information processing
architecture
Support the information analyst in their analytic goals
• Notify new information relevant to their current task
– In any form, from any source
• Automatic processing of future information to extract meaningful taskrelevant domain information
IT Support for the Intelligence Analysis
Process
 Collection
IT supports
extraction, storage, Human intel
analysis,
indexing
hypotheses
 IT Processing can extract, filter and
transform data into information from
large volumes of data
 Cognitive Processing is restricted to Human configuration,
IT process configuration due to
algorithms, modeling
large volumes of data
 Processing
 Preparation of information for
interpretation by analysts
 Interpretation
 Human activity through hypothesis
testing and other cognitive
techniques
 IT processing assists analyst
through information management,
rules based inferencing and
visualisation
IT supports retrieval,
visualisation,
presentation &
communication
Processing to support Intelligence Analysis
 On the ground
 Increased volume and variety of
information sources
 Cannot manually inspect, process &
interpret
 IT is not appropriate for all phases
in intelligence analysis
MIPS is a proof of concept to investigate how analysts can configure and
benefit from an improved Collect/Process/Interpret processing pipeline
MIPS Builds on work from the US/UK
International Technology Alliance
 ITA research is a US/UK collaboration between industry, academia and
government
 Focused on: Network science, Decision-making & Coalition operations
 From May 2006 – May 2016
 Research is fundamental (6.1), low Technical Readiness Level (TRL)
 Intended outputs are papers, proof of concepts
 Higher TRL transition contracts take the core ideas to progress further
 Emerging technologies arising from ITA research used by MIPS:
 ITA Controlled English
• Consumable by humans and machines
 ITA Information Fabric
• A lightweight service bus middleware
 Gaian Database
• A Dynamic, Distributed, Federated Database
ITA Controlled English
Using human language to enable smarter human/machine processing of information
 What is it?
 Human-friendly language but machine readable
 Rich semantics, broad application
 Reasoning engine with rationale
 Supports agility for dynamic, evolving situations
 Enables hybrid human/machine collaboration
 Empowers non-technical users
 Examples:




the person Dave works for the company IBM.
if (the person P works for the company C) then (the company C employs the person P).
the person pn0123 is a suspect in the crime c713 because the person pn0123 was sighted in…
conceptualise an ~ intelligence report ~ I that ~ mentions ~ the suspect S.
 Why does it matter?
 Real fusion of machine precision and human cognition
 Can harness “collective intelligence”
 Facilitate human-human communication & socialisation
 Download the CE Store: http://ibm.co/RDIa53
ITA Information Fabric
A lightweight service-bus middleware designed for the edge of the network
Download the fabric:
http://ibm.co/13FyW9X
 A two-way message bus and set of middleware services
 Connects all network assets to each other and to users
 Provides universal access to intelligence data, processing services and applications
 Implemented using a multi-hop publish/subscribe architecture
 Messages are efficiently propagated without duplication thereby minimising bandwidth utilisation
 Policies constrain how assets can be used and configured and information shared
across coalition boundaries
 Same policy mechanism as is used in the Gaian Database
 Information fabric can integrate assets and services
 In MIPS, the Information Fabric is used to integrate information processing services
The Gaian Database
A dynamic distributed federated database (DDFD)
To provide an extensible capability of distributed
query & dissemination across a network of disparate data sources
with local access-controlled policies and security measures
DDFD is based on “store locally &
query from anywhere” principle
Obtaining information from multiple
heterogeneous data sources does
not scale & requires significant
management overhead
Query
N5
N7
1250 node Gaian Database
N4
N6
N8
Query
N3
N9


N10
N2
N0
N1
N11
Download Gaian:
http://ibm.co/15TMSBr
8
DDFDs provide distributed access
to distributed heterogeneous data
sources, are highly scalable &
impose low management overhead
MIPS Objectives & Challenges
 Objective 1: Notify analysts when new, relevant information has arrived
 Defining Relevant Information
 Maintaining and sharing analytical goals
 Indexing information & notifying analysts of matches with their analytical goals
 Operation in a mixed environment
 Objective 2: Automatic processing of new information
 Identifying processing services and data sets
 Standardised description of processing services to enable reasoning
 Visualising current capabilities for the user
 Efficient operation
 Abstract services
 Automatic information processing in a mixed environment
 Facilitating collaboration between analysts
MIPS Architecture
 Composition
 Analyst describes a set of services and
links them to form a network (to achieve
a goal)
 Network can be deployed as a set of
linked and running services
 Information Fabric
 Provides nodes for services to run on
and message bus for service
interconnection
 Services perform specialised information
processing
 Sources & Sinks
 Controlled English
 All data produced by MIPS services is
converted to CE and optionally stored in
the CE Store
 Data is stored and queried using CE
 Some services also implemented in CE
 Analyst Tools
 A set of tools for users to operate
within the MIPS environment
Example: A typical information flow
 Integrates multiple SharePoint repositories
 Document Extractor service extracts
metadata facts in CE about the documents
 Report Analytics text mining service extracts
entities and relationships from document text
 Implemented as a set of CE-based services
 Watch List Notifier looks at extracted CE
about people and queries watch lists to see if
a person is present on the list
 Multiple types of watch list are demonstrated
 Notification Sink manages notifications that
can be accessed by other services and to
alert users
 Information generated is stored in CE store
Meeting the MIPS Objectives
Objective 1: Notify Analysts when new, relevant information has arrived
 Challenge 1: Defining relevant information
 Analysts indicate relevant information by defining service compositions, or by
querying the CE Store. Compositions and queries may be saved, shared and
reused
 Challenge 2: Maintaining and sharing analytical goals
 Analysts define goals using the Fabric Service Composition Tool. Goals so defined
may be reused and shared amongst teams of analysts
 Goals may also be encapsulated in queries (or rules) in the CE Store which may
be saved, shared and reused
 Challenge 3: Indexing information and notifying analysts of matches with
their analytical goals
 Indexing of information may be via a MIPS service or an external application e.g.
traditional indexing within a relational database
 Example provided of local and external watch list inspection, notifying people
matched, with a link to the original document text about the person of interest
 Challenge 4: Operation in a mixed environment
 The MIPS demonstrator is capable of interacting with and integrating different data
sources and using a wide range of deployed processing services
Meeting the MIPS Objectives
Objective 2: Automatic processing of new information
 Challenge 1: Identifying processing services and data sets
 The MIPS Fabric Service Composition Tool (FSCT) allows users to identify services available to them
and the data sets required and produced by the services
 Challenge 2: Standardised description of processing services to enable reasoning
 MIPS standardises the descriptions of the services available to a user together with the information
exchange formats used between services
 Challenge 3: Visualising current capabilities for the user
 The MIPS processing pipeline is constructed and visualised using a graphical editor; the FSCT
 Complex queries to the CE Store can be constructed and visualised using a CE Query Builder; a
graphical tool for creating complex queries across linked information
 Challenge 4: Efficient operation
 MIPS uses the ITA Information Fabric as its underlying messaging middleware. The fabric’s
publish/subscribe model provides efficient routing of messages between services
 Challenge 5: Abstract services
 Abstract services are a generalisation of a group or type of service
 Abstract service descriptions may be defined in MIPS before the service has been created
 Challenge 6: Automatic information processing in a mixed environment
 MIPS is capable of handling information and services from both within and external to its environment.
 Challenge 7: Facilitating collaboration between analysts
 Some rudimentary support is provided in MIPS for collaboration amongst groups of analysts. Further
support is expected in future work
Questions?
 Main links:
 IBM developerWorks downloads:
• Information fabric:
http://ibm.co/13FyW9X
• Gaian database:
http://ibm.co/15TMSBr
• CE Store:
http://ibm.co/RDIa53
 International Technology Alliance
http://www.usukita.org
Email: [email protected] or [email protected]
Research was sponsored by US Army Research Laboratory and the UK Ministry of Defence and
was accomplished under Agreement Number W911NF-06-3-0001. The views and conclusions
contained in this document are those of the authors and should not be interpreted as representing
the official policies, either expressed or implied, of the US Army Research Laboratory, the U.S.
Government, the UK Ministry of Defense, or the UK Government. The US and UK Governments are
authorized to reproduce and distribute reprints for Government purposes notwithstanding any
copyright notation hereon.