Cognitive Robotics - Robot Intelligence Technology Lab

Download Report

Transcript Cognitive Robotics - Robot Intelligence Technology Lab

Advances in Cognitive
Robotics:
Achievements
and Challenges
Fakhri Karray
Pattern Analysis and Machine
Intelligence Research Lab
University of Waterloo, Canada
HNICEM’09, Manilla
http://pami.uwaterloo.ca/~karray
2009, Pr Fakhreddine Karray
1
Outline

Part1 : Towards Cognition in robotics

Evolution of Robotic Systems : The Need For Cognition

What is Cognition?
2009, Pr Fakhreddine Karray
2
Outline

Part1 : Towards Cognition in robotics

Part2 : Major Aspects of Cognitive Robotics
 Line of research in cognitive robotics

Building blocks or components

Theories of cognition and architectures
2009, Pr Fakhreddine Karray
3
Outline

Part1 : Towards Cognition in robotics

Part2 : Major Aspects of Cognitive Robotics

Part3 : Recent Advances

Design of Confabulation Architecture

Novel design of discrete cognitive skills
2009, Pr Fakhreddine Karray
4
Outline

Part1 : Towards Cognition in robotics

Part2 : Major Aspects of Cognitive Robotics

Part3 : Recent Advances

Part4 : Applications

Part5 : Challenges
2009, Pr Fakhreddine Karray
5
QuickTime™ and a
YUV420 codec decompressor
are needed to see this picture.
Clip from AI motion picture 2001
2009, Pr Fakhreddine Karray
6
IEEE Spectrum Online March 12, 2009 (reporting on Boston Globe article of March 4, 2009)
2009, Pr Fakhreddine Karray
7
Part 1:
Toward Cognition
in Robotics
Part2 : Major Aspects of Cognitive Robotics
Part3 : Recent Advances
Part4 : Applications
Part5 : Challenges
2009, Pr Fakhreddine Karray
8
Toward Cognition in Robotics
Evolution of Robotics
What is Cognition?
The Evolutionary Stages
Service Robots for
Personal Use
Personal
Robotics
Service Robots for
Professional Use
Service
Robotics
Industrial
Robotics
2009, Pr Fakhreddine Karray
9
Toward Cognition in Robotics
Evolution of Robotics
Processing Units
What is Cognition?
Swarm
Robotics
SI
100s
DAI
Multiagent
Distributed
Robot System
10s
AI
1
Robotics
Centralized Control
Machine
Multiple Machines
1
2009, Pr Fakhreddine Karray
MEMS-based Multiple Machines
100s
10s
10
Machines
Units
Toward Cognition in Robotics
Evolution of Robotics
What is Cognition?
We expect from next generation of robots to have the following
capabilities :

to learn from experience and apply the learned knowledge to
deal with anomalies

to ‘understand’ human and naturally communicate with them

to reason and self-reflect to take new task initiatives
The need For Cognition
2009, Pr Fakhreddine Karray
11
Toward Cognition in Robotics
Evolution of Robotics
What is Cognition?
Cognitive robots are robots imparted with ‘human-like’ cognitive
abilities
Major Properties in Human Cognition

Self-awareness

Knowledge

Perception

Reasoning

Learning

Planning and decision making
These skills emerge in human as a result of complex, bidirectional
interaction among body, brain, and environment
2009, Pr Fakhreddine Karray
12
Toward Cognition in Robotics
Evolution of Robotics
What is Cognition?
Understanding of human cognition involves three major disciplines
Psychology
Cognition
Neuroscience
2009, Pr Fakhreddine Karray
Biology
13
Toward Cognition in Robotics
Evolution of Robotics
What is Cognition?
Understanding of cognitive robotics involves :
Mechanical/
Mechatronics
Engineering
Computer
Engineering
Psychology
Cognitive
Robotics
Neuroscience
Biology
Artificial
Intelligence
Computer
Science
2009, Pr Fakhreddine Karray
14
Toward Cognition in Robotics
Evolution of Robotics
Processing Units
What is Cognition?
Swarm
Robotics
Cognitive Robotics
SI
100s
DAI
Multiagent
Distributed
Robot System
10s
AI
1
Robotics
Centralized Control
Machine
Multiple Machines
1
2009, Pr Fakhreddine Karray
MEMS-based Multiple Machines
100s
10s
15
Machines
Units
Toward Cognition in Robotics
Evolution of Robotics
What is Cognition?
Major properties required for cognition in Robotics
2009, Pr Fakhreddine Karray

Physical embodiment

Social situatedness

Role of experience

Self-awareness

Value system
16
Toward Cognition in Robotics
Evolution of Robotics
What is Cognition?
Sensors
Cognition Pathways
Perception pathway
Actuators
Planning pathway
2009, Pr Fakhreddine Karray
Action pathway
17
Memory
Reasoning pathway
Part1 : Toward Cognition in Robotics
Major Aspects of
Cognitive Robotics
Part2 :
Part3 : Recent Advances
Part4 : Applications
Part5 : Challenges
2009, Pr Fakhreddine Karray
18
Major Aspects of Cognitive Robotics
The major aspects in cognitive robotics research
2009, Pr Fakhreddine Karray

Lines of research

Building blocks of cognitive systems

Various theories of cognition/architectures
19
Major Aspects of Cognitive Robotics
Line of research
Components
Theories
The research on robot cognition is still in its infancy. A complete theory
of robot cognition is not yet in the picture.
Lines of research
2009, Pr Fakhreddine Karray
Group 1
Theory of Robot cognition
Group 2
Discrete cognitive abilities
20
Major Aspects of Cognitive Robotics
Line of research
Components
Theories
Group 1 : Theories of Robot Cognition
Focus
Pioneering Works
Incremental Intelligence [Brooks 1991]
Design principle, Architecture for
Developmental Engineering [Sandini 1997]
cognition, Hardware and
Brain-based Device [Krichmar 1998]
software requirement, Social
Autonomous Mental Development [Weng 2000]
Cognitive Developmental robotics [Asada 2001]
factors
2009, Pr Fakhreddine Karray
Conscious Machine [Kawamura 2005]
21
Major Aspects of Cognitive Robotics
Line of research
Group 2 : Discrete cognitive skills
Visual
attention

Theoretical models
 Robotic models
 Task-specific models
Social
cognition

Joint attention
 Social imitation
Value
2009, Pr Fakhreddine Karray
system
22
Components
Theories
Major Aspects of Cognitive Robotics
Line of research
Components
Theories
The major building blocks of Cognitive Robots involve the following
components :
2009, Pr Fakhreddine Karray

Computational perception

Knowledge Representation

Learning

Reasoning and planning

Interaction and communication
23
Major Aspects of Cognitive Robotics
Line of research
1- Computational
perception
2- Knowledge
Representation
3-Learning
Components
4-Reasoning
and planning
5- Interaction
and
communication
Perception in cognitive robotics :

allows robot to receive multi-sensory input from external
environment

is crucial in realizing self-awareness capability

is connected to all other components
Attention is the main mechanism required in this process
2009, Pr Fakhreddine Karray
24
Theories
Major Aspects of Cognitive Robotics
Line of research
1- Computational
perception
2- Knowledge
Representation
3-Learning
Components
4-Reasoning
and planning
Theories
5- Interaction
and
communication
Knowledge representation:

allows ease of information storage and retrieval

performs inference to obtain new information from learned data
The robot deploys mechanisms such as self-organization, selfproduction, and self-maintenance to autonomously develop its
own representation with interaction and experience.
2009, Pr Fakhreddine Karray
25
Major Aspects of Cognitive Robotics
Line of research
1- Computational
perception
2- Knowledge
Representation
3-Learning
Components
4-Reasoning
and planning
Theories
5- Interaction
and
communication
Construct new skills, knowledge and capabilities through:

Exploration: e.g. reaching, grasping, and manipulating what is
around it.

Social Interaction: which involves interacting with the user in the
learning process.

Observation and Imitation: can be imitation of low level features
such as joint trajectories, or higher level features such as complete
actions, and behaviors.
2009, Pr Fakhreddine Karray
26
Major Aspects of Cognitive Robotics
Line of research
1- Computational
perception
2- Knowledge
Representation
3-Learning
Components
4-Reasoning
and planning
Theories
5- Interaction
and
communication
Consists of a set of three simultaneous processes:

Reactive processes: they mimic the reflexive behavior observed in
biological systems.

Deliberative processes: they are in charge of realizing what is
commonly known as thinking in biological systems (e.g. Motion
planning).

Reflective planning: they involve high-level reasoning where robot
computes how to perform a given task based on its cognitive abilities,
learned knowledge, and embodiments.
2009, Pr Fakhreddine Karray
27
Major Aspects of Cognitive Robotics
Line of research
1- Computational
perception
2- Knowledge
Representation
3-Learning
Components
4-Reasoning
and planning
5- Interaction
and
communication
The human role and the level of
interaction will vary but the human is
still part of the system.
CooperaTION
2009, Pr Fakhreddine Karray
28
Theories
Major Aspects of Cognitive Robotics
Line of research
1- Computational
perception
2- Knowledge
Representation
Components
4-Reasoning
and planning
3-Learning
Theories
5- Interaction
and
communication
Multimodality
Multimodal
&
Adpativity
Modules
S R/R I
S R/R I
Multimodal
Modules
S R/R I
Social Interaction
S R/R I
Multimodal Interaction
Adaptivity
Adpativity
Modules
S R/R I
S R/R I
WIMP-based
GUIs
S R/R I
Adaptive Interaction
S R/R I
Direct Manipulation Interaction
S R/R I: Service Request/Response Interface
Alaa Khamis and Mohamed Kamel, and Miguel A. Salichs, “Human-Robot Interfaces for Social
Interaction”, International Journal of Robotics and Automation, vol. 22, (206), 2007.
2009, Pr Fakhreddine Karray
29
Major Aspects of Cognitive Robotics
Line of research
Components
Theories
Major Theories of Cognitive Robotics
Development
1950
1990
in
Theory of development
Piaget 1953
c o g n i t i on
2005
2000
Incremental
Intelligence
Brooks 1991
robot
2007
Cognitive Developmental
Robotics
Asada 2001
Developmental
engineering
Sandini 1997
Conscious Machine
Kawamura 2005
Confabulation
Architecture
Hecht-Nielsen 2007
Autonomous Mental Development
Weng 2000
Brain based devices
Krichmar 1998
2009, Pr Fakhreddine Karray
30
Major Aspects of Cognitive Robotics
Line of research
Components
Theories
Incremental Intelligence
First introduced by Brooks, 1991.
Developmental Engineering
The main implications of this theory are
requirement for :
Brain-based Device

developmental (incremental) nature of
intelligence,
Autonomous Mental Development

necessity of social situatedness and
communication,
Cognitive Developmental Robotics

and robot capability to integrate multimodality sensory data.
Conscious Machine
2009, Pr Fakhreddine Karray
31
Major Aspects of Cognitive Robotics
Line of research
Components
Theories
Incremental Intelligence
First introduced by Sandini, et al. 1997.
Developmental Engineering
DE has emphasis on four basic principles:
Brain-based Device
Autonomous Mental Development
Cognitive Developmental Robotics
Conscious Machine
2009, Pr Fakhreddine Karray
32

physical embodiment,

social interaction,

development and experience, and

dynamic of development.
Major Aspects of Cognitive Robotics
Line of research
Components
Theories
Incremental Intelligence
First introduced by Krichmar et al., 2003 as
a platform to test computational models of
nervous systems.
Developmental Engineering
Brain-based Device
BBD devices use authentic simulation of
the primate nervous systems to realize their
cognitive abilities such as reasoning and
planning.
Autonomous Mental Development
Cognitive Developmental Robotics
Conscious Machine
2009, Pr Fakhreddine Karray
33
Major Aspects of Cognitive Robotics
Line of research
Components
Theories
Incremental Intelligence
Was introduced by Weng et al., 2000
Developmental Engineering
AMD requires the cognitive robot to be
designed in a task non-specific manner
assuming that robot is capable of developing
the task specification program by itself.
Brain-based Device
Autonomous Mental Development
The main principles of AMD are
Cognitive Developmental Robotics

embodiment,
Conscious Machine

self-awareness,

self effectiveness,

and developmental program.
2009, Pr Fakhreddine Karray
34
Major Aspects of Cognitive Robotics
Line of research
Incremental Intelligence
Developmental Engineering
Brain-based Device
Autonomous Mental Development
Cognitive Developmental Robotics
Theories
Was introduced by Asada et al., 2001
CDR argues for the existence of a built-in
architecture and puts emphasis on the role of
social interaction on autonomous development
of representation.
CDR focuses on three main issues,
embodiment in concert with Books theory,
embedded structure (built-in architecture), and
interaction design.
Conscious Machine
2009, Pr Fakhreddine Karray
Components
35
Part 3: Aspects, Components and Architectures of Cognitive Robotics
Line of research
Components
Theories
Incremental Intelligence
Was introduced by Kawamura et al., 2005
Developmental Engineering
CM essentially provides a method of
implementation for cognitive skills inspired by
findings in cognitive psychology.
Brain-based Device
Authors of CM restrict their work into emotion
and attention-based control of behavior in
designing consciousness.
Autonomous Mental Development
Cognitive Developmental Robotics
CM takes a multi-agent and behavior based
methodology with each (software) agent
dedicated to a specific cognitive ability and
maintains coordination among these agents
using a built-in central architecture called
intelligent machine architecture.
Conscious Machine
2009, Pr Fakhreddine Karray
36
Part1 : Toward Cognition in Robotics
Part2 : Major Aspects of Cognitive Robotics
Part3 :
Recent Advances
Part4 : Applications
Part5 : Challenges
2009, Pr Fakhreddine Karray
37
Recent Advances
Besides the theories proposed, some recent advances in Cognitive
robotics were made in :
2009, Pr Fakhreddine Karray

Design of Confabulation Architecture

Novel design of discrete cognitive skills
38
Recent Advances
Confabulation Architecture
Discreet cognitive skills
Confabulation Theory
 First development made by Hecht-Nielsen in 2004-2005
 claimed to be the first comprehensive theory to describe cognition in
humans and animals.
 postulates that cognition constitutes four fundamental elements:

Mental object representation: about 4000 thalamocortical modules
each comprising 10,000 to 100,000s of symbols


Knowledge links: unidirectional association between two symbols
Confabulation: the symbol with maximum total input excitation is
activated

Action command origination: confabulation conclusion launches a
set of pre-associated action commands
2009, Pr Fakhreddine Karray
39
Recent Advances
Confabulation Architecture
Discreet cognitive skills
Confabulation Theory provides a hypothesis and neuronal model for
acquiring, storing and using knowledge
An apple object and some of the attributes
Knowledge link connecting word apple to color
2009, Pr Fakhreddine Karray
40
Recent Advances
Confabulation Architecture
Motivational example: Sentence Generation
Several other centenarians
at Maria Manor had talked
about trying to live until 2000,
but only Wegner made it.
First context
sentence
Discreet cognitive skills
Her niece said that Wegner
had always been a character
– former glove model, buyer
for Macy’s, owner of Lydia’s
Smart Gifts downtown
during the 1950s and ‘60s –
and that she was
determined to see 2000.
Second context
sentence
Confabulation
architecture
Plausible next
sentence
She was born in the Bronx
Borough of New York City.
From R.H.Nielsen, “confabulation theory, The mechanics of though”, Springer 2007
2009, Pr Fakhreddine Karray
41
Recent Advances
Confabulation Architecture
Discrete cognitive skills
Design of Discrete Cognitive Skills
Visual
attention
Social
cognition

Joint attention

Social imitation

A case study : Multimodal Interaction –
Multimodal attention
Value
2009, Pr Fakhreddine Karray
system
42
Recent Advances
Confabulation Architecture
Discrete cognitive skills
Design of Discrete Cognitive Skills
Visual Attention

Visual attention is a fundamental building block of cognitive
development.

It allows robots to focus their processing power only on the
behaviorally relevant information and therefore facilitates their
interaction with humans and surrounding environment
2009, Pr Fakhreddine Karray
43
Recent Advances
Confabulation Architecture
Discrete cognitive skills
Design of Discrete Cognitive Skills
Visual Attention (Recent works)

Recent works involved a bioinspired probabilistic model of visual
attention of cognitive robots.

Work uses Bayesian analysis to recursively estimate the orientation of
the camera head of a robot such as a visually salient/behaviorally
relevant stimuli resides at the center of the visual field

Observes the postulates of Biased Competitive hypothesis (a famous
neurodynamic theory of the primates visual attention) for state transition
2009, Pr Fakhreddine Karray
44
Toward Cognition in Robotics
Confabulation Architecture
Discrete cognitive skills
Example task in hand: any particular arm motion of the human
user
2009, Pr Fakhreddine Karray
45
Recent Advances
Confabulation Architecture
Design of Discrete Cognitive Skills
Visual Attention : Theoretical Models
2009, Pr Fakhreddine Karray
46
Discrete cognitive skills
Recent Advances
Confabulation Architecture
Design of Discrete Cognitive Skills
Visual Attention : Robotic Model (1/2)
2009, Pr Fakhreddine Karray
47
Discrete cognitive skills
Recent Advances
Confabulation Architecture
Design of Discrete Cognitive Skills
Visual Attention : Robotic Model (2/2)
2009, Pr Fakhreddine Karray
48
Discrete cognitive skills
Recent Advances
Confabulation Architecture
Discrete cognitive skills
Design of Discrete Cognitive Skills
Social Cognition
Method facilitating social interaction between robots and humans, divided into
three categories of research :

Joint attention: having this ability enables robot to attend to an object of
mutual interest intentionally.

Social Imitation: powerful methods of learning and developing social
behavior for cognitive robots.

Multimodal interaction: allow users to move seamlessly between different
modes of interaction, from visual to voice to touch, according to changes in
context or user preference.
2009, Pr Fakhreddine Karray
49
Recent Advances
Confabulation Architecture
Design of Discrete Cognitive Skills
Social Cognition – Joint Attention
2009, Pr Fakhreddine Karray
50
Discrete cognitive skills
Recent Advances
Confabulation Architecture
Design of Discrete Cognitive Skills
Social Cognition – Social Imitation (1/2)
2009, Pr Fakhreddine Karray
51
Discrete cognitive skills
Recent Advances
Confabulation Architecture
Design of Discrete Cognitive Skills
Social Cognition – Social Imitation (2/2)
2009, Pr Fakhreddine Karray
52
Discrete cognitive skills
Recent Advances
Confabulation Architecture
Discrete cognitive skills
Design of Discrete Cognitive Skills
a case study: multimodal Interaction
The architecture here is based on the following major considerations :

Data fusion should be performed at different levels in order to extend
its flexibility.

Data fusion and cognitive processes (perception, reasoning,
projection) should not be treated independently

The system and/or the user are involved in the fusion model through a
refinement process (feedback)

Different factors influence the fusion process, such as
memory/database and experience

The fusion process is accomplished within a predefined context/goal
2009, Pr Fakhreddine Karray
53
Recent Advances
Confabulation Architecture
Discrete cognitive skills
Design of Discrete Cognitive Skills
a case study: multimodal Interaction
2009, Pr Fakhreddine Karray
Toward a Comprehensive DataFusion Architecture
54
Recent Advances
Confabulation Architecture
Discrete cognitive skills
Design of Discrete Cognitive Skills
Value System
Robot’s capability to plan action perception of salient stimuli, which makes it
an essential requirement of developing human-like intelligence in robotic
systems
Characteristics:
Prediction
Task
non specific
Developmental
Value based learning
2009, Pr Fakhreddine Karray
55
Recent Advances
Confabulation Architecture
Design of Discrete Cognitive Skills
Value System
2009, Pr Fakhreddine Karray
56
Discrete cognitive skills
Part1 : Toward Cognition in Robotics
Part2 : Major Aspects of Cognitive Robotics
Part3 : Recent Advances
Part4 :
Applications
Part5 : Challenges
2009, Pr Fakhreddine Karray
57
Applications

Health-care and Assistive robotics

Entertainment robotics

Military and Space exploration

Humanoid/social robotics
2009, Pr Fakhreddine Karray
58
Applications
Health Care and Assistive Robotics
Trends in percentage of the elderly (over age 85) in the world
Source: Adriana Tapus, Maha J. Mataric, and Brian Scassellati, “Socially Assistive Robotics: The Grand Challenges in Helping Humans Through Social Interaction,” IEEE
Robotics & Automation Magazine, MARCH 2007
2009, Pr Fakhreddine Karray
59
Applications
Health Care and Assistive Robotics

Domo (MIT robot): A household assistant for the elderly
or wheelchair-bound. It grasps objects and places them on
shelves or counters.

ASIBOT: A portable assistive robot for elderly and disease
people bringing more freedom in daily tasks as eating,
drinking, shaving, make up wearing, tooth cleaning, etc.

Huggable: A robotic companion for health care, education,
and social communication.
2009, Pr Fakhreddine Karray
60
QuickTime™ and a
YUV420 codec decompressor
are needed to see this picture.
Clip from Information and Robot Technology Research Initiative,
Tokyo, Dec. 2008
2009, Pr Fakhreddine Karray
61
Applications
Entertainment Robotics

Sparky project on AIBO platform:
alleviating loneliness and causing to form
attachments for nursing home residents.

CiceRobot: A museum guide cognitive robot.

Soccer robot

Flutist robot: The first member of a
humanoid robot orchestra.
Video : Violinist Robot
2009, Pr Fakhreddine Karray
62
QuickTime™ and a
YUV420 codec decompressor
are needed to see this picture.
Clip from Toyota Research labs, 2007
2009, Pr Fakhreddine Karray
63
Applications
Military and Space Exploration

Robonaut: A robotic system that can function as
an EVA astronaut equivalent.

BEAR: Designed to find, pick up and rescue
people in harm's way.
Video : Bear Robot
2009, Pr Fakhreddine Karray
64
QuickTime™ and a
YUV420 codec decompressor
are needed to see this picture.
Clip from Vecna Robotics Inc., 2007
2009, Pr Fakhreddine Karray
65
Applications
Humanoid/social robotics

Maggie: research platform to study human robot
interaction and robot intelligence and autonomy.

Video : Partner/ Cooperative
Dancing with Maggie
CB2: child robot with
biomimetic body. It is able
to develop a behavior
similar to a 1 or 2 years old
baby.
2009, Pr Fakhreddine Karray
66
QuickTime™ and a
YUV420 codec decompressor
are needed to see this picture.
Clip from Robotics Lab, Univ. Carlos III, 2007
2009, Pr Fakhreddine Karray
67
Part1 : Toward Cognition in Robotics
Part2 : Major Aspects of Cognitive Robotics
Part3 : Recent Advances
Part4 : Applications
Part5 :
Challenges
2009, Pr Fakhreddine Karray
68
Challenges
Issue I: Need for Metrics and Evaluation Measure for Cognition
Need
for performance metric to evaluate the cognitive abilities in a
robotic system. Difficulty in providing objective measure
Lack
of standard measure to qualitatively assess intelligence in
human makes the issue more challenging
More
work is needed for evaluating objectively cognitive abilities of
robots
2009, Pr Fakhreddine Karray
69
Challenges
Issue II: Integration of cross-disciplinary knowledge
A cross-disciplinary
collaborative study to integrate the most
understood aspects of human cognition in order to come up with a
comprehensive theory of robot cognition.
Lack
of comprehensive architecture on which to impart cognitive
skills for robots
Robustness
and adaptability of robot cognition from one
environment to the next.
Difficulty
2009, Pr Fakhreddine Karray
in dealing with uncertainty in perception
70
Final Thoughts
Tighter
cooperation involving roboticists, AI specialists, Neuroscientists
and psychologists is much needed
Avoid
for now the goal of designing a general theory for robot cognition
and focus on designing robots with cognitive traits that are task specific
More
work needed on understanding the dynamics of cognitive paths in
human brains
Further
work is needed on the mathematical modeling of developmental
cognition in humans
2009, Pr Fakhreddine Karray
71
University of Waterloo
Campus
2009, Pr Fakhreddine Karray
72
University of Waterloo at Glance

Canada leading comprehensive University (25,000 Students)

Largest Co-op program in the world

Largest Math Faculty in the World (3500 students)

Largest Engineering Faculty in Canada (3800 grad and undergrad
students)


Ranked as the “most innovative” in Canada for the past 16 years in a row.
Largest employer in Waterloo, RIM provider of the BlackBerry Device
(6500 employees) spinoff of University of Waterloo
2009, Pr Fakhreddine Karray
73
Pattern Analysis and Machine Intelligence Lab
Electrical and Computer Engineering
University of Waterloo
Canada
www.pami.uwaterloo.ca
2009, Pr Fakhreddine Karray
74
Pattern Analysis & Machine Intelligence Research Group
Current Supporting Institutions/Companies
2009, Pr Fakhreddine Karray
75
Pattern Analysis & Machine Intelligence Research Group

Established in early eighties

Members: M. Kamel, F. Karray, O. Basir, A. Wong

Principal participant in major Provincial and Federal centers of
excellence (MMO, CITO, Telelearning-NCE, PRECARN, LORNET).

Extensive experience in promoting fundamental and applied research
to government and industry

Graduated over 450 Ph.D and Master students over the past 25 years.

Average annual budget of $2.5 M

Four Spin-off companies:
Virtek (1986), PDS (1997), IMS (1999), Vestec (2002).
2009, Pr Fakhreddine Karray
76
Pattern Analysis & Machine Intelligence Research Group
Major Research Areas

Cooperative Intelligent Systems

Intelligent Robotics and Cognitive Machines

Computer Vision and Image Processing

Pattern Recognition and Intelligent Agents

Sensors and embedded systems

e-Learning

Applications: Automotive Industry, Aerospace, Financial
Sector, Web Services, Biomedical, Health Care
2009, Pr Fakhreddine Karray
77
QuickTime™ and a
YUV420 codec decompressor
are needed to see this picture.
Clip from Wall e motion picture 2008
2009, Pr Fakhreddine Karray
78
Acknowledgements
Dr. Alaa Khamis, Research Asst. professor
Dr. Abbas Ahmadi, PostDoc fellow
Malek Baklouti, PhD student
Momotaz Begum, PhD student
Jamil Abu Saleh, PhD student
Bahador Khalegui, PhD student
Ahmed ElMogy, PhD student
and
Technical staff of the PAMI Research Lab
2009, Pr Fakhreddine Karray
79
Thank you for your attention
2009, Pr Fakhreddine Karray
80
Recent Advances
Confabulation Architecture
Discrete cognitive skills
Design of Discrete Cognitive Skills
a case study: multimodal attention
Multi-modal attention: Natural speech command modulates the visual
attention behavior of the robot in the form of top-down modulating bias
2009, Pr Fakhreddine Karray
81
Recent Advances
Confabulation Architecture
Design of Discrete Cognitive Skills
Visual Attention (Recent works)
Functional description of the visual attention model
2009, Pr Fakhreddine Karray
82
Discrete cognitive skills
References
[Brooks1991] R. A. Brooks. Intelligence without representation. Artificial Intelligence 47:139 160, 1991.
[Asada2001] M. Asada, K. F. MacDormanb, H. Ishigurob, and Y. Kuniyoshi. Cognitive developmental robotics as a new
paradigm for the design of humanoid robots. Robotics and Autonomous Systems, 37:185 – 193, 2001
[Sandini2001] G. Metta, G. Sandini, L. Natale, and F. Panerai. Development and robotics. In IEEE-RAS International
Conference on Humanoid Robots, pages 33 – 42, 2001
[Kawamura2005] K. Kawamura. Cognitive approach to a human adaptive robot development. In Proceedings of IEEE
International Workshop on Robot and Human Interactive Communication, pages 210 – 216, 2005
[Weng2004] J.Weng. Developmental robotics: Theory and experiments. International Journal of Humanoid Robotics, 1:199
– 234, 2004.
[Krichmar2003] J. L. Krichmar and G. M. Edelman. Brain-based devices: intelligent systems based on principles of the
nervous system. In IEEE/RSJ International Conference on Intelligent Robot and Systems, pages 940 – 945, 2003.
[Nielsen2007] R. Hecht-Nielsen. Confabulation Theory. Springer-Verlag, 2007.
2009, Pr Fakhreddine Karray
83