Transcript Programming Techniques 804G5

Cognitive Computer Vision
Kingsley Sage
[email protected]
and
Hilary Buxton
[email protected]
Prepared under ECVision Specific Action 8-3
http://www.ecvision.org
Course outline




What is Cognitive Computer Vision (CCV) ?
Generative models
Graphical models
Techniques for modelling cognitive aspects of CCV
–
–


Bayesian inference
Markov Models
Research issues
Coursework and case studies
So what is CCV ?
CSL
Bernd Neumann, 2003 (ECVision Summer School on Cognitive Vision)
Cognitive Systems
Laboratory
Cognitive Vision research requires multidisciplinary efforts and escape
from traditional research community boundaries.
Knowledge Representation &
Reasoning
•
•
•
•
•
•
KR languages
logic-based reasoning services
default theories
reasoning about actions & change
Description Logics
spatial and temporal calculi
Computer Vision
Learning & Data Mining
•
•
•
•
•
•
•
•
•
planning, goal-directed behaviour
manipulation
sensor integration
navigation
localization, mapping, SLAM
integrative architectures
D. Vernon, Dagstuhl 2003
concept learning
inductive generalization
clustering
knowledge discovery
Natural Language
•
•
•
•
Cognitive
Vision
Robotics
•
•
•
•
•
•
object recognition, tracking
bottom-up image analysis
geometry and shape
hypothesize-and-test control
probabilistic methods
high-level concepts
qualitative descriptions
NL scene descriptions
communication
Cognitive Science
•
•
•
•
•
psychophysical models
neural models
conceptual spaces
qualitative representations
naive physics
Uncertain Reasoning
•
•
•
•
Bayesian nets, belief nets
decision & estimation
causality
probabilistic learning
Monday 27th October 2003
So what is CCV ?




In this course, we focus on using of ideas from
cognitive science and psychology to do CCV
To show how we can build effective CCV systems that
are more robust and more capable of solving non-trivial
problems than those that do not embrace these ideas
Use statistical inference and machine learning as our
tools for modelling cognitively inspired processes
We are not claiming “hard AI” in this course
Key Cognitive Elements

Objects, events, activities and behaviours
–

“What is it that we are observing?”
Attention and control
–
“How is it that we observe?”
Key Cognitive Elements

Visual learning and memory
–
–
–
–

Visual control and attention
–
–
–
–

Representation of objects and their behaviour
Recognition
Categorisation
These are “what” problems
Perception for tasks using models of expectation
Goals, task context
Resources, embodiment
These are “how” problems
Cognition
–
From perception to action
Key Cognitive Elements

Visual learning and memory - examples
–
–
–
Learning about objects and how their appearance
can change
Recognising activities by the interactions between
objects
Extracting invariant models from training data
Learning and “recognising” objects
(Murase and Nayar, 1996)
Learn and recognise activities
Coupled Hidden Markov Models (CHMM) techniques
(Oliver, Rosario & Pentland, 1999)
Activities with interactions via coupled states in a HMM
Learning invariant models
Means for 3 clusters
Variances for 3 clusters
Key Cognitive Elements

Visual control and attention
–
–
–
–
A framework for attentional control
Inferring likely behaviour using Bayes nets
Deictic markers
Attentional selection of objects
A Framework For Task Based
Visual Control
Scene Interpretation
Task Based
Control
CONTROL POLICY
(WITH STATE MEMORY)
FEATURE
COMBINATION
Image Data
Driven
d1
d2
……
dN
BBN Inference of likely vehicle tracks
Gong and Buxton, 1993
IGP
orient
size
ls1
ls2
lo1
lo2
Fixed camera gives direct set of
dependencies Image Grid Position
BBN has size/orient hidden nodes
Leaf nodes ls1/2, lo1/2 observables
Deictic Markers in inference of
behaviour
Howarth and Buxton,1996
Left: attention for overtake
(overtaken & overtaking vehicle)
Right: attention for giveway
(stopped & blocker vehicle plus
ground-plane conflict zone)
Attentional selection using eye gaze
Attentional selection using
predicted trajectory data
Attentional selection using
predicted trajectory data
Attentional selection using
predicted Space of Interest
Summary




Cognitive Computer Vision is a multi-disciplinary area
of research
Here we use statistical inference and learning for
robust models
Task based attentional control is key to prediction and
cognitive systems design
Useful reference: “Visual surveillance in a dynamic and
uncertain world” Buxton, H and Gong, S, Artificial
Intelligence 78, pp 431-459, 1995
Next time …

Generative models
–
–
What are they?
Why are they so important to Cognitive Vision?