Transcript slides

IRC Learning and the
Novamente Cognition
Engine
Imitative-Reinforcement-Corrective Learning:
A Robust Learning Methodology
for Virtual Pets and Avatars
implemented using a limited version
of the Novamente Cognition Engine
Ben GOERTZEL, Cassio PENNACHIN, Nil GEISSWEILLER, Moshe LOOKS,
Andre SENNA, Welter SILVA, Ari HELJAKKA, Carlos LOPES
The Novamente Cognition Engine: An
Integrative, Experiential Learning
Focused Approach to AGI
Knowledge representation:
– Nodes and links (a weighted, labeled hypergraph)
– Probabilistic weights, like an uncertain semantic network
– Hebbian weights, like an attractor neural network
The Novamente Cognition Engine: An
Integrative, Experiential Learning
Focused Approach to AGI
Learning algorithms:
– Automated program learning (for small, purpose-specific
programs meeting AI-determined specifications)
• NCE uses MOSES a probabilistic improvement on genetic
programinng, described in Moshe Looks 2006 PhD thesis
– Uncertain inference
• NCE uses Probabilistic Logic Networks, a novel fusion of
probability theory and formal logic
– PLN book to be published by Springer in early 2008
– Economic Attention Allocation
• Artificial economics used for assignment of credit
and attention allocation
An Integrative, Experiential Learning
Focused Approach to AGI
(underlying both the Novamente and OpenCog
initiatives)
Cognitive architecture:
– Focused on interactive learning, e.g. virtual embodiment, NL
conversation, robotics
– Largely inspired by human cognitive architecture
Teaching Methodology:
– Embodied, experiential, socially interactive
– Combining imitative and reinforcement learning
Novamente Cognition Engine is one, wellfleshed-out, example of a concrete AGI design
within this family of designs
OpenCog framework (OpenCog.org)
incorporates Novamente’s knowledge
representation and overall software framework,
and will allow experimentation with multiple
alternate learning algorithms within this same
framework
Why May This Approach Have a Prayer of
Succeeding?
• It is based on a well-reasoned, comprehensive theory of
mind,
– covering both the concretely-implemented and
emergent aspects of mind
– Oriented toward encouraging the emergence of a
self-system within the AI’s knowledge base, based
on embodied social learning
– See The Hidden Pattern
• The specific algorithms and data structures chosen to
implement this theory of mind are efficient, robust and
scalable and, so is the software implementation
Stages of Cognitive Development
Objective
detachment from
phenomenal self
Emergence of
phenomenal self
No self yet
Intelligence
Intelligence
Intelligence
Animal-level AI’s killer app:
Virtual Pets
Virtual Worlds
Each month, 24% of the 34.3M US
kids and teens on the web are visiting
a virtual world. By 2011 that number
is expected to be 53%
For example, Webkinz grew from 800K
users in Oct 2006 to more than 7M in
Oct 2007
Media for Virtual Pets
2.3B use mobile phones
1.2B use the Internet
465M joined Virtual Worlds
Pets in Virtual Worlds
Pets for PC Games
Pets for Mobile Gaming
Pets in World of Warcraft
Current Virtual Pets:
Cute but Dumb
Current virtual pets are rigidly
programmed and lack emotional
responsiveness, individual personality
or ability to learn.
Building a Better Pet Brain
Adding the ability to have the pets genuinely learn and
respond to the environment will make them more real to the
user, and increase the user/virtual pet bond. This supports
trends toward personalization and community, enriching both.
Novamente Pet Brain
Novamente pets respond to and interact with
objects, creatures and avatars, and learn from
experiences that will then influence future
behavior.
For example, if there happens to be a cat
around, there is a good chance the pet dog
would chase it. However, if the cat scares him
away, the dog might not be so eager to chase
the cat next time.
Training the Pet Brain
give “sit” command
show “sit” example
clap when pet sits
Novamente pets can be taught to do simple or complex tricks, from
sitting to playing soccer or learning a dance -- by learning from a
combination of encouragement, reinforcement and demonstration.
IRC Learning
Teaching with a Partner
Current Pet Brain Architecture
Next-Gen Pet Brain Architecture
Next Step: Language Learning?
Intelligence