Transcript MediaHub - Prof. Paul Mc Kevitt

A New Approach to Decision-making within an
Intelligent MultiMedia Distributed Platform Hub
Glenn Campbell, Tom Lunney,
Aiden Mc Caughey, Paul Mc Kevitt
School of Computing and Intelligent Systems
Faculty of Engineering
University of Ulster, Magee Campus
Derry/Londonderry
Northern Ireland
{Campbell-g8, TF.Lunney, P.McKevitt} @ulster.ac.uk
Outline

Goals and objectives

Key research problems

Software and tools

Architecture of MediaHub

Future development
Goals
The primary objectives of this research are to:

Interpret/generate semantic representations of multimodal
input/output

Perform fusion and synchronisation of multimodal data
(decision-making)

Implement and evaluate a multimodal platform hub
(MediaHub)
Key research problems

Semantic representation?

Semantic storage?

Communication with other elements of a
multimodal platform?
• Decision-making?
Software and Tools

Main Programming Language


Communication


OpenAIR Specification using Psyclone
Decision-making tool


Java
HUGIN GUI / API (Hugin 2006)
Semantic Representation


Likely to be XML-based
XML, XHTML + Voice, MPEG-7, OWL, OIL,
SMIL, EMMA?
Psyclone




Semantic Storage via Whiteboards
implemented in Psyclone
Psyclone uses OpenAIR Specification for
communication
Publish-Subscribe mechanism for
communication
Using Java implementation of OpenAIR
specification
Hugin


Tool for implementing Bayesian Networks as
CPNs (Causal Probabilistic Networks)
Hugin GUI


Graphical user interface to Hugin decision engine
Hugin API


Library implemented in Java
Allows programs to implement Bayesian Networks
for decision-making
Bayesian Networks




AKA Bayes nets, Causal Probabilistic Networks (CPNs),
Bayesian Belief Networks
Consists of nodes and directed edges between nodes
Node represents a variable
Influence between nodes represented by edges
Diet
Exercise
‘Diet’ and ‘Exercise’ nodes have
influence over ‘Weight Loss’ node
Weight
Loss
MediaHub Example Network
Architecture of MediaHub

Dialogue Manager
 Facilitates communication between other modules
 Synchronisation

Whiteboard implemented in Psyclone
 Whiteboard and Dialogue Manager form kernel of
MediaHub

Decision-Making Module
 Bayesian Networks

Semantic Representation Database
 Stores semantic representation of multimodal data
Architecture of MediaHub
Architecture of MediaHub
Decision-making in MediaHub

Input:




Determining semantic content of input
Fusing semantics of input
Resolving ambiguity at input
Output:


Synchronising multimodal output
Best modality for output
Conclusion

An intelligent multimodal distributed platform hub called
MediaHub will be developed

MediaHub will interpret/generate semantic
representations of multimodal input and output

MediaHub will perform fusion and synchronisation of
multimodal data

MediaHub will provide a new method of decision-making
within a distributed platform hub
Future development

Define necessary decisions

Develop Bayesian decision-making using Hugin API for
Java

Finalise the semantic representation scheme

Acquire (or create) multimodal corpora for testing
Questions?