Transcript A Theory of Theory Formation

A Theory of Theory Formation
Simon Colton
Universities of Edinburgh and York
Overview

What is a theory?
 Four components of the theory of ATF
– Techniques inside the components

Cycles of theory formation
– Case Studies

Applications
– Of both the theories and the process
What is a Theory?

Theories are (minimally) a collection of:
–
–
–
–

Objects of interest
Concepts about the objects
Hypotheses relating the concepts
Explanations which prove the hypotheses
Finite Group Theory:
– All cyclic groups are Abelian

Inorganic Chemistry:
– Acid + Base  Salt + Water
So, We Require:
Object
Generator
Concept
Generator
Hypothesis
Generator
Explanation
Generator
In Principle, These Could Be:
Database, CAS,
CSP, Model
Generator
Data Mining
Program
Machine
Learning
Program
ATP System,
Pathway Finder,
Visualisation
In Practice,
Current Implementation:
Database, Model
Generator, (CAS,
CSP nearly)
The HR
Program
The HR
Program
ATP Systems
Object Generation and
Explanation Generation

Object Generation:
– Machine learning – reading a file, database

In Mathematics
– CSP (e.g., FINDER, Solver), CAS (e.g., Maple)
– Davis Putnam method (e.g., MACE)
– Resolution Theorem Proving (e.g., Otter)

HR must be able to communicate
– Read models and concepts from MACE’s output
– Read proofs and statistics from Otter’s output
Concept Generation

Build a new concept from old ones
– 10 general production rules (demonstrated later)
– Produce both a definition and examples

Throw away concepts using definitions
– Tidy definitions up
– Repetitions, function conflict, negation conflict

Decide which concepts to use for construction
– Plethora of measures of interestingness
– Weighted sum of measures
Concept Generation:
Lakatos-inspired Techniques

Monster Barring
– Remove an object of interest from theory

Counterexample Barring
– Except a finite subset of objects from a theorem
– E.g., all primes except 2 are odd

Concept Barring
– Except a concept from a theorem
– All integers other than squares have an
even number of divisors

Credit to Alison Pease
Hypothesis Generation:
Finding Empirical Relationships

Equivalence conjectures
– One concept has the same examples as another

Subsumption conjectures
– All examples of one concept are examples of other

Non-existence conjectures
– A concept has no examples

Assessment of conjectures
– Used to assess the concepts mentioned in them
Hypothesis Generation
Extracting Prime Implicates

Extract implications, then prime implicates
 Equivalence conjectures are split:
– A & B & C  D & E & F becomes
– A & B & C  D, A & B & C  E, etc.

Non-existence conjectures are split:
– ¬(A & B & C) becomes: A & B  ¬C, etc.

Extract Prime implicates:
– A & B & C  D, try A  D, then B  D,
C  D, then A & B  D, etc.
Hypothesis Generation:
Imperfect Conjectures

User sets a percentage minimum, say 80%
 Near-subsumption conjectures
– E.g., primes  odd (99% true)
– Also returns the counterexamples: here, 2

Near-equivalence conjectures
– Prime  odd (70% true)

Applicability conjectures
– A concept has a (small) finite number of examples
– E.g., even prime numbers: 2 is only example
Cycles of Theory Formation

How the individual techniques are employed
 Concept driven conjecture making
– Finding conjectures to help understand concepts
– Exploration techniques

Conjecture driven concept formation
– Inventing concepts to fix faulty conjectures
– Imperfect conjectures, Lakatos techniques
Concept Driven Cycle (cut-down)
Invent Concept
Non Existence
Equivalence
Reject
New Concept
Subsumptions
Implications
Concept Driven Cycle Continued
Implications
Counterexample
Proof
Prime Implicates
Counterexample
Proof
Conjecture Driven Cycle
Invent Concept
Near Equivalence
Concept Barring
Applicability
Reject
Near Subsumption
Monster Barring
Counterex Barring
Concept Barring
New Concept
Counterex Barring
New/Old Concept
New/Old Concept
Equivalence
Implications
Case Study: Groups
Given: Group theory axioms
Case Study: Groups
Davis Putnam
Method
MACE model generator finds a model of size 1
Case Study: Groups
HR Reads
MACE’s Output
Extracts concepts:
Element, Multiplication, Identity, Inverse
Case Study: Groups
Match Production
Rule
Invents the concept: idempotent elements (a*a=a)
Case Study: Groups
Equivalence
Finding
Makes Conjecture: a*a=a  a is the identity element
Case Study: Groups
Resolution
Theorem Proving
Otter proves this in less than a second
Case Study: Groups
Extracts Prime
Implicates
a*a = a  a=identity, a=identity  a*a=a
End of cycle
Case Study: Groups
Compose
Production Rule
Later: Invents the concept of triples of elements
(a,b,c) for which a*b=c & b*a=c
Case Study: Groups
Exists Production
Rule
Invents concept of pairs (a,b) for which there
exists an element c such that: a*b=c & b*a=c
Case Study: Groups
Forall Production
Rule
Invents the concept of groups for which all pairs
of elements have such a c: Abelian groups
Case Study: Groups
Equivalence
Finding
Makes the Conjecture:
G is a group if and only if it is Abelian
Case Study: Groups
Sorry
Otter fails to prove this conjecture
Case Study: Groups
Davis Putnam
Method
MACE finds a counterexample:
Dihedral Group of size 6 (non-Abelian)
Case Study: Groups
Assessment of
Concepts
Concept of Abelian groups allowed into theory
Theory recalculated in light of new object of interest
Case Study: Goldbach
Given: Integers 1 to 100, Concepts: Divisors, Addition
Case Study: Goldbach
Split Production
Rule
Invents: Even Numbers (divisible by 2)
Case Study: Goldbach
Size Production
Rule
Invents: Number of Divisors (tau function)
Case Study: Goldbach
Split Production
Rule
Invents: Prime numbers (2 divisors)
Case Study: Goldbach
Compose
Production Rule
Half an hour later:
Invents: Goldbach numbers (sum of 2 primes)
Case Study: Goldbach
Near Equivalence
Finding
Conjectures: Even numbers are Goldbach numbers
(with one exception, the number 2)
Case Study: Goldbach
Counterexample
Barring (Split)
Forces: Concept of being the number 2
Case Study: Goldbach
Counterexample
Barring (Negate)
Forces concept: Even numbers except 2
Case Study: Goldbach
Subsumption
Finding
Conjectures: Even numbers except 2 are Goldbach
Numbers (Goldbach’s Conjecture)
Case Study: Goldbach
Absolutely No
Chance
Passes the conjecture to an inductive theorem prover?
Applications of Theories #1

Puzzle generation
– Which is the odd one out: 4, 9, 18, 36?
– Which is the odd one out: cat, platypus, bat, dolphin*
– What’s next in the sequence: 21, 22, 24, 25, 26, 28?
– See AISB’02 paper

Problem generation
– Find theorem to differentiate Spass, Bliksem & E
– 12,000 theorems generated (at least one for each)
– 200 problems in TPTP library (used at CASC’02)
– See AI+Maths paper
Applications of Theories #2

Prediction tests: (e.g., Progol animals file)
– P(mammal | has_milk) = 1.0
– P(mammal | habitat(water)) = 0.125
– Take average over all Bayesian probabilities
– Moral Reasoner Dataset (200 people, 24 atts)
– HR gets 91% accuracy

Preliminary work, more to come
Applications of Theory
Formation #1

Identifying concepts
–
–
–
–

Forward look ahead mechanism
Michalski trains, number sequences,
1, 9, 25, ? (finds odd, finds square, knows to combine)
see ICML-00 paper
Simplifying problems
– Constraint generation for CSP
– E.g., QG3 quasigroups: (a*b)*(b*a)=a
– These are anti-Abelian and a*a = b  b*b=a
– See CP-01 paper
– Lemma generation for ATP (few positive results yet)
Applications of Theory
Formation #2

Making conjectures
– About Maple (CAS) functions
– Discard any that a prover can prove
– tau(sigma(n)) = 2  tau(tau(n)) = 2
– See Calculemus paper

Inventing concepts
– Refactorable numbers (1, 2, 8, 9, 12, 18, …)
– Odd refactorables are square, perfects not refactorable
– 20 sequences in the Encyclopedia
– See AAAI-00 paper, Journal of Integer Sequences
Major Projects In Progress #1

Discriminating between algebraic structures
– With Volker Sorge, Andreas Meier
– HR part of a large system (8 programs)
– HR discriminated 97% of 777 example pairs
– exists x (x*x=x & (y*y=x  y*y=y))
0
1
2
3
4
0
0
0
0
0
0
1
3
0
2
4
1
2
1
0
4
3
2
3
4
0
1
2
3
4
2
0
3
1
4
0
1
2
3
4
0
0
0
0
0
0
1
2
4
1
3
0
2
4
3
2
1
0
3
1
2
3
4
0
4
3
1
4
2
0
Major Projects In Progress #2

Interactive ATF
– Human replaces one/all of the four modules
– With Roy McCasland, mathematician
– Work on discovery in Zariski spaces with HR

Multiagent ATF
– Another agent replaces one/all of the four modules
– With Alison Pease’s (PhD)
– Cognitive modelling of social discovery
– Lakatos style reasoning and machine creativity
Conclusions

Presented a snapshot of the theory of ATF
– Autonomous
– Four components, numerous techniques
– Uses third party software
– Concept driven and conjecture driven cycles

Applies to many machine learning tasks
– Concept identification, puzzle generation,
– Predictions, problem simplification,
– Conjecture making, discrimination finding
Theory Formation in
Bioinformatics?

Can work with non-maths data
 Can form near-conjectures
 Needs to relax notion of equality
 Multi-agent/interactive approaches

http://www.dai.ed.ac.uk/~simonco/research/hr