Transcript not - Fabelier

How Letting Go of Goals Helps
Creativity and Discovery
Kenneth O. Stanley
Evolutionary Complexity Research Group
University of Central Florida School of EECS
<kstanley>@eecs.ucf.edu
In Collaboration with
Joel Lehman, Sebastian Risi, Jimmy Secretan, Nick Beato, Adam Campbell,
David D'Ambrosio, Adelein Rodriguez, Jeremiah T. Folsom-Kovarik, Brian Woolley
A Sobering Message…
If you put your mind to it,
you can accomplish anything
(Marty McFly in Back to the Future, 1985)
A Sobering Message…
If you put your mind to it,
you can accomplish anything
(Marty McFly in Back to the Future, 1985)
…with a Paradoxical Silver Lining
If you do not put your mind to it,
you can accomplish anything
Seeking AI: Trying to Understand
How Complexity Evolves
• 100 trillion connections in the human brain
– The most complex structure known to exist
• How can Darwinian evolution produce
such astronomical complexity?
– We can investigate this question through
evolutionary computation (artificial evolution)
– In the process, other fundamental principles
of innovation and discovery are uncovered
– Evolution is a kind of search
Innovation, Creativity, and
Discovery are Forms of Search
• We search the space of possible artifacts
– All possible images
– All possible three-dimensional morphologies
– All possible combinations of words
– All possible minds
• Most of the search space is desolate
• The gems are needles in a haystack
• The mind is a powerful search operator
Argument Preview
• Highly ambitious objectives ultimately
block their own achievement
• The greatest human achievements are not
the result of objective optimization
• All of search is cloaked in futility
– Yet in this realization there is genuine
liberation
– New opportunities will open for discoveries
that work in different ways
Picbreeder:
A Microcosm for Innovation
• Website: http://picbreeder.org
– Crowd-sourced picture-breeding online service
•
•
•
•
Three years of operation
7,500 evolved images
500 users
Like Dawkins’ BioMorphs (from The Blind
Watchmaker, 1986) on steroids
• The question: How do users discover the
best images in a vast and desolate space?
How Users Breed Images
• First must decide where to start
– From “scratch” : Random initial population
– By “branching” : Offspring of existing image
Typical Start from Scratch
Starting from a Butterfly
How Users Breed Images
• First must decide where to start
– From “scratch” : Random initial population
– By “branching” : Offspring of existing image
Typical Start from Scratch
Starting from a Face
How Users Breed Images
• Next: Select parent(s) – Which do you like?
How Users Breed Images
• Next: Select parent(s) – Which do you like?
Press “Evolve”
after selecting
parents
How Users Breed Images
• Next: The offspring (next generation) appear
Parent
How Users Breed Images
• Next: Repeat until satisfied and then Publish
Parent
Important: Everything Ultimately
Derives from Scratch
• The beginning of
evolution
• However, the
images become
more complex over
generations
Breeding from Scratch
Breeding from Scratch
Breeding from Scratch
Parent
Breeding from Scratch
Parent
Breeding from Scratch
And
so on…
Parent
The Result:
Large, Growing Phylogenies
• Users build upon each other’s discoveries
(30 users built this one)
Images Evolved by Picbreeder Users
(All are 100% evolved: no retouching)
Picbreeder Image Representation
• Images are represented
by compositional patternproducing networks
(CPPNs)
– A composition of
simple functions
• A new node is sometimes
introduced through
mutation
f
• What kind of search
space does this
representation induce?
y
f(x,y,d)=
HSB
Gaussian
x
H
Sig
Sig
S
B
Sin
Gaus
Sigmoid
Sine
Sin
Gaus
Gaus
x
Linear
y
d
The Search Space is Desolate
• Almost every image looks like these
– They have random weights and topologies:
Yet Picbreeder Users Found These…
(All are 100% evolved: no retouching)
Yet Picbreeder Users Found These…
(All are 100% evolved: no retouching)
…and entire “species”…
…and these…
(All are 100% evolved: no retouching)
…and these…How?
(All are 100% evolved: no retouching)
Paradox: Images Cannot Be Re-evolved!
• Pick an image
• Make it an objective
(i.e. goal) for evolution
on computer (NEAT)
• Run automated
evolution
• Output is terrible
• This result
is universal
– Why?
(74 cumulative generations)
(best results from over 30,000 generations each)
Why Is It Impossible to
Re-discover Former Discoveries?
(best results from over 30,000 generations each)
• You cannot find something on Picbreeder
simply by looking for it
• Serendipity plays a powerful role
– Yet if it is serendipity, then how can it happen
so often?
The Story of the Car
• Would you expect
to find a car in
this space?
• I didn’t
• But then I found
one
How Did I Find the Car?
• I was not looking for a car
• Rather, I chose to evolve the alien (ET)
face to get more alien faces:
How Did I Find the Car?
• But then, the alien’s eyes descended and
turned into wheels
How Did I Find the Car?
• The only way to find the car was by not looking
for it
• Otherwise, I would never have selected the alien
– It does not look like a car
How Did I Find the Car?
• But I would not have evolved the alien either!
• Someone else had to evolve it for me to make
my discovery
Most Top Images Have the
Same Story
The stepping
stones almost
never resemble
the final product
Moral: It Works Because There
Is No Unified Goal
• The only way to find the needles in the
haystack
– …is by not collectively looking for them
• Users have conflicting goals and interests
– Some explore with no goal
– Some may have their own goals
• Yet the system as a whole has no unified
objective
– Every discovery is a potential stepping stone
for someone else
Stepping Stones
• Steps in search space that lead to objective
• May be hard or impossible to identify a priori
– Especially for ambitious problems
• May not induce positive change in objective function
Objectives Can Be Deceptive
• The Chinese Finger Trap
– What are the stepping stones?
Thought Experiment
• You have a Petri dish the size of the world
– …and organisms equivalent to the very first
cells on Earth
• Your objective: Evolve human-level
intelligence
– You have 4 billion years
• What is your selection strategy?
Thought Experiment
• You have a Petri dish the size of the world
– …and organisms equivalent to the very first
cells on Earth
• Your objective: Evolve human-level
intelligence
– You have 4 billion years
• What is your selection strategy?
– Administer intelligence tests to single-celled
organisms!
Thought Experiment
• You have a Petri dish the size of the world
– …and organisms equivalent to the very first
cells on Earth
• Your objective: Evolve human-level
intelligence
– You have 4 billion years
• What is your selection strategy?
– Administer intelligence tests to single-celled
organisms!
Intelligence Does Not Resemble
Its Stepping Stones
• The stepping stones:
– Multicellularity
– Bilateral Symmetry
• Yet they are essential to its discovery
• Humans were only found because we are
not the objective
– A proliferation of agnostic stepping-stones
was the necessary prerequisite
– Natural evolution has no final objective
The Limits of Human Innovation
• Thought experiment: Good idea or not?
– 5,000 years ago sequester all the greatest
minds to build a computer
The Limits of Human Innovation
• Thought experiment: Good idea or not?
– 5,000 years ago sequester all the greatest
minds to build a computer
• Bad idea!
– Vacuum tubes were not invented with
computation in mind
– Electricity was not discovered with
computation in mind
• Almost no prerequisite to any major invention
was invented with that invention in mind!
The Limits of Human Innovation
• This realization touches many of our
greatest enterprises
– Artificial intelligence
– Including personal goals (e.g. make $1M)
Something Is Wrong at the
Heart of Search
• How else could it work?
– Abandon our objectives!
– Follow the gradient of interestingness
• Serendipitous discovery is not accidental
Algorithm: Abandon Objectives and
Search for Novelty
• Can be hard to identify stepping stones a priori
• Novelty = proxy for stepping stones
– Anything that does something different is a potential
stepping stone
• Novelty is still based on information, just
different information
• No final objective, just find new behaviors
• Encounter solution although not looking for it!
Testing Novelty Search
• Start with an evolutionary algorithm
– Evolve artificial neural networks to control
simulated robot behaviors
– No fitness function
– Instead, reward any behavior that is novel so far
• No concept of better or worse
• No objective
• Changes over evolution
Underlying Evolutionary
Algorithm: NEAT
• NeuroEvolution of Augmenting Topologies
– Well-established neuroevolution method
– Proven in many control and decision domains
• Starts with minimal structures that “grow”
through mutations
• With novelty search:
– Only the novel reproduce
Novelty Search = Exhaustive Search?
• Not exactly, because NEAT induces an order
• Once simple behaviors are exhausted, novelty
requires more complexity
– More novelty requires accumulating information
Novelty Search = Exhaustive Search?
• Not exactly, because NEAT induces an order
• Once simple behaviors are exhausted, novelty
requires more complexity
– More novelty requires accumulating information
Novelty Search = Exhaustive Search?
• Not exactly, because NEAT induces an order
• Once simple behaviors are exhausted, novelty
requires more complexity
– More novelty requires accumulating information
Novelty Search = Exhaustive Search?
• Not exactly, because NEAT induces an order
• Once simple behaviors are exhausted, novelty
requires more complexity
– More novelty requires accumulating information
Novelty Search = Exhaustive Search?
• Not exactly, because NEAT induces an order
• Once simple behaviors are exhausted, novelty
requires more complexity
– More novelty requires accumulating information
Experiment: Maze Navigation
Domain
Maze
Navigator
• Why maze navigation?
– Easily visualized
– Good model for deception (tuneable)
Maze Navigation Domain
• Three Experiments
• Objective-based NEAT
– Fitness function: Distance to goal at end of trial
• NEAT with novelty search
– Reward ending somewhere different
• NEAT with random selection
Results (Maze)
• 40 runs for each method
– Objective-based NEAT: Only 3 successful runs
– Random selection: 4 successful
– NEAT Novelty search: 39 out of 40 successful!
Results (Visualization)
Biped Walking
• Notoriously difficult to evolve
• Which produces walking more effectively?
– Reward distance travelled (i.e. fitness)
– Reward ending somewhere different (i.e. novelty)
Novelty Finds
Significantly Better Gaits (50 runs)
(15 second trials)
Novelty search best: 13.7 meters
Fitness-based best: 6.8 meters
Novelty and Fitness Walkers
Similar Outcomes in Many
GP Domains (2010)
Domains
Two-point navigation
Unenclosed Maze (2010) (2010)
Door-locking logic
(Goldsby and Cheng 2010)
Medium Maze (2008)
T-Mazes
(2009/2010)
Simulated Bee (2010)
What Do Results Mean?
• Finding the objective is often more effective
by not looking for it
• Will not always work to achieve defined goals
• There is futility at the heart of search
– The lesson is the important part
– Not that novelty search is a panacea
• However, greatness is possible if it is
undefined
What Does It Mean for
Ambitious Design Goals?
• We cannot realistically expect to achieve them
by trying
– Unless they are one stepping stone away – then
they come within reach
• Otherwise, we can achieve them by not trying
– It is the combination of many minds with many
divergent interests that ultimately exploits a search
space in the long-term, not any individual objectives
Liberation through Acceptance
• May be possible to capture the process of
open-ended innovation through software
– Collaborative interactive non-unified systems
– Picbreeder is just an example
• Potential treasure-hunting systems
– Clothing catalog, furniture catalog,
building catalog, automobile catalog
– Software development
– Science Funding
– Web-based search
Deeper Meaning
• Search is at its most awesome when it has
no objective
– Natural evolution
– Human Innovation
– Picbreeder
– Novelty search
• Not all the same but one unifying theme:
no objective
Deeper Meaning
• We rarely try to emulate this form of search:
instead we want to chain it
• Unleash the treasure-hunter
– Avoid the temptation towards convergence
• It is in your interest - our interest - that some
do not follow the path that you believe is right
To achieve our highest goals,
we must be willing to abandon them
The Last Word
There remains one a priori fallacy or natural
prejudice, the most deeply-rooted, perhaps, of all
which we have enumerated: one which not only
reigned supreme in the ancient world, but still
possesses almost undisputed dominion over
many of the most cultivated minds…This is, that
the conditions of a phenomenon must, or at least
probably will, resemble the phenomenon itself.
John Stewart Mill on the like-causes-like bias
In A System of Logic, Ratiocinative and Inductive,
1846
More information
• My Homepage:
http://www.cs.ucf.edu/~kstanley
• Novely Search Users Page:
http://eplex.cs.ucf.edu/noveltysearch/userspage/
• Evolutionary Complexity Research Group:
http://eplex.cs.ucf.edu
• Picbreeder: http://picbreeder.org
• Email: [email protected]