Transcript projects

Computational Architectures in Biological Vision, USC
Fall 2004
Lecture 1. Overview and Introduction
Reading Assignments:
Textbook: “Foundations of Vision,” Brian A. Wandell, Sinauer, 1995.
Read Introduction and browse through book
Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 1: Overview & Introduction
1
Organization
Lectures: Tuesdays, 5-7:50pm, GFS-107
Textbook: “Foundations of Vision,” by Brian A. Wandell, Sinauer
Associates, Inc. ISBN 0-87893-853-2
Office hour: Mon 2-4pm in HNB-30A
Homepage: http://iLab.usc.edu under “classes.”
Grading: 3 units. Based on project:
- initial project proposal: 20%
- midterm written progress report: 20%
- final project writeup: 20%
- project deliverable: 30%
- final presentation: 10%
- 9/21/04
- 10/19/04
- 11/30/04
- 11/30/04
- 11/30/04
Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 1: Overview & Introduction
2
Class Web Site
http://iLab.usc.edu/classes/2004cs599/
Soon, you will find there:
- Lecture
notes: user “ilab” and password “2cool”
- Reading assignments
- Grades
- General announcements
- Project topics
Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 1: Overview & Introduction
3
Project ideas
1 – Auditory cueing to visually salient locations
This potentially addresses the problem of building the next generation
of machines to help the visually impaired. We will exploit the finding
(Duncan et al., Nature, 1999) that visual and auditory attention may
recruit two processing streams that can largely operate in parallel.
Thus, we will supplement a person’s poor vision by using auditory
cueing towards visually salient locations.
Tasks: - learn about attention/saliency model and how to run it
- learn about auditory source localization in space
- implement a program that will generate various beeps and/or
spoken words and give the listener the impression that
they came from a specific location in space
- interface the saliency computation to the new auditory code
- test and demo
Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 1: Overview & Introduction
4
Project ideas
2 – Saliency-based video compression
We can use attention/saliency to prioritize regions in video streams that
should be encoded with best quality. See Itti, IEEE Trans Image Proc,
2004. In that paper, the prioritization simply used a variable amount of
blur. A better way would be to go deeper into the workings of an
MPEG4 codec.
Tasks: - learn about how MPEG4 works
- find some good open-source codec
- implement an ROI facility in it
- get the ROIs from the saliency program (easy)
- test and demo
Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 1: Overview & Introduction
5
Project ideas
3 – Theoretical study: computational role of neurogenesis
It has been recently shown that new neurons are created in the
hippocampus (a brain region concerned with memory), more so when
animals are exposed to rich environments. Interestingly, the more new
cells are created, the faster the death of other cells seem to occur. Our
goal is to understand whether this makes sense computationally.
Tasks: - learn about Hopfield associative memories
- what is the cost of adding a new memory?
- what are the consequences of killing one neuron?
- what would be the cost of adding a new memory if we could
also add new neurons to achieve that?
- deliverable is a paper/presentation answering these questions
Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 1: Overview & Introduction
6
Project ideas
4 – Theoretical study: role of feedback in object recognition
When we have a hard time recognizing an object, we try harder. What exactly
does this entail? Here we propose to study that as an optimization problem.
That is, trying harder will alter the object recognition network so as to
maximally disambiguate between the top two recognized candidates.
Tasks:
- learn about multilayer backprop networks
- devise a cost function that differs from the standard backprop
rule: while in backprop we want to minimize the
overall difference between teaching signal and network
output, in this project we want to maximize the difference
between top and second output strengths
- design a small backprop network for object recognition
- show that the proposed feedback rule works in disambiguating
recognition
Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 1: Overview & Introduction
7
Project ideas
5 – Landmark-based navigation
Humans are very good at rapidly identifying which objects in their
visual environment could serve as landmark points to describe a route.
But what makes a good landmark? It should be rather unique, easy to
find among clutter, rather permanent, maybe fairly viewpoint-invariant,
etc. Let’s try to formalize those characteristics into a visual analysis
system.
Tasks: - assume you can detect salient points in an image
- come up with various image analysis schemes to evaluate
uniqueness, permanency, etc
- write test code and conduct a demo
Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 1: Overview & Introduction
8
Project ideas
6 – Constraint-free eye/head tracker
With a single webcam it should be possible to track eye/head movements of
persons sitting in front of a computer screen, and to move the mouse
accordingly.
Tasks:
- see what has been done in this research field (in particular at
the von der Malsburg group)
- estimate accuracy, given camera resolution, distance to
observer, size of field of view, etc
- if it turns out to be hopeless (too poor), consider a 2-camera
system where a fixed wide-field camera finds the head and
one eye, and a pan/tilt high-zoom narrow-field camera then
tracks the pupil
- demonstrate a working system
Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 1: Overview & Introduction
9
Project ideas
7 – statistical center-surround
LGN neurons are thought to detect the difference between light levels in
a central region of their receptive field and in a broader concentric
surround region. What if things actually were more complicated and
instead of a simple difference we computed a statistical test for how
different the distributions of light intensity values are in the two
regions?
Tasks: - decide on a test
- implement these new LGN neurons, as well as the old ones
- find critical image stimuli where the old neurons would
see nothing while the new ones would see a difference
- expand to multiple scales
- demo and writeup
Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC 2004. Lecture 1: Overview & Introduction
10