Transcript Single-Trial Characterization of Evoked Responses

Kevin Knuth
on
Measuring
July 8, 2007
(20)
Measuring
Kevin H. Knuth
Department of Physics
University at Albany
Familiarity Breeds
the Illusion of Understanding
Anonymous
8 July 2007
MaxEnt 2007
Kevin H Knuth
We are all Familiar with Measuring
8 July 2007
MaxEnt 2007
Kevin H Knuth
We Measure All Sorts of Things
8 July 2007
MaxEnt 2007
Kevin H Knuth
Measuring Things We Don’t Understand
http://www.phy.duke.edu/research/photon/qelectron/proj/infv/images/einstein-nb-large.jpg
http://superstruny.aspweb.cz/images/fyzika/quantum/quantum_split.jpg
8 July 2007
MaxEnt 2007
Kevin H Knuth
But Not Everything
8 July 2007
MaxEnt 2007
Kevin H Knuth
Unification of Ideas
8 July 2007
MaxEnt 2007
Kevin H Knuth
Counting
In the Beginning…
8 July 2007
MaxEnt 2007
Kevin H Knuth
A Caveman had a Collection of Rocks
8 July 2007
MaxEnt 2007
Kevin H Knuth
An Equivalence Class of Rocks
If the Caveman
likes, he can treat
each rock as being
equivalent to any
other rock in his
collection.
He then has an
Equivalence Class
of Rocks
8 July 2007
MaxEnt 2007
Kevin H Knuth
Comparing Collections of Rocks
Two Cavemen
might like to
compare their
collections.
8 July 2007
MaxEnt 2007
Kevin H Knuth
One-to-One Correspondence
If they treat the
rocks as belonging
to an equivalence
class, they can
attempt to make a
one-to-one
correspondence
One Caveman has more
rocks than the other since
there is not a one-to-one
mapping between the two
sets of rocks
8 July 2007
MaxEnt 2007
Kevin H Knuth
Why Carry Your Rocks Around?
Every time they
want to compare
their collections,
Thog has to carry
his rocks over a big
hill to meet Bok
who carried his
rocks across a
raging river.
8 July 2007
There must be an
easier way to make
comparisons
MaxEnt 2007
Kevin H Knuth
Sticks?
Thog has an idea, he
suggests that for every rock,
he should pick up a stick.
Carrying sticks is easier!
8 July 2007
MaxEnt 2007
Bok thinks this over…
Kevin H Knuth
One Stick
Instead of a lot of
sticks, Bok suggests
making a mark on a
stick for each rock.
8 July 2007
MaxEnt 2007
Kevin H Knuth
Natural Numbers
Thog, who wants to
impress his smart friend
agrees and further
suggests that they should
make fancy marks rather
than a lot of them
8 July 2007
MaxEnt 2007
Kevin H Knuth
One-to-One Correspondence
1
2
Bok gets to work…
3
8 July 2007
MaxEnt 2007
Kevin H Knuth
At Their Next Meeting…
The two gentlemen compare their markings and enjoy a
relaxing afternoon free from carrying rocks.
2
3
Comparing their markings, they find that Bok has more
rocks than Thog.
8 July 2007
MaxEnt 2007
Kevin H Knuth
Ordering
8 July 2007
MaxEnt 2007
Kevin H Knuth
Not All Rocks Are The Same
Bok decides that some
of his rocks are more
impressive than others.
One of them is
HUGE!!!
From this point on he
no longer treats the
rocks as belonging to
an equivalence class
8 July 2007
MaxEnt 2007
Kevin H Knuth
Lifting Rocks
Bok finds it easy
to order these
rocks in terms of
how heavy they
are to lift.
8 July 2007
MaxEnt 2007
Kevin H Knuth
Binary Ordering Relation
Bok lifts one rock and compares its weight to
another…one pair at a time.
Rock 1
Rock 2
Rock 1 is no heavier than Rock 2
R1  R2
8 July 2007
MaxEnt 2007
Kevin H Knuth
Binary Ordering Relation
Rock 2
Rock 3
Rock 2 is no heavier than Rock 3
R2  R3
8 July 2007
MaxEnt 2007
Kevin H Knuth
Transitivity
R1  R2 and R2  R3 implies that
Rock 1
Rock 3
Rock 1 is no heavier than Rock 3
R1  R3
8 July 2007
MaxEnt 2007
Kevin H Knuth
Heavier
Diagram Representing the Ordering
8 July 2007
Bok uses his
results to order
his rocks
MaxEnt 2007
Kevin H Knuth
Ranking
Heavier
3
2
1
8 July 2007
MaxEnt 2007
Bok then
realizes that he
can use the
same marks to
compare one
rock to another.
Kevin H Knuth
Measuring
8 July 2007
MaxEnt 2007
Kevin H Knuth
Rock Hunting
Bok has become quite
sophisticated, and
now that his collection
is finally in order, he
decides to go hunting
for another rock.
8 July 2007
MaxEnt 2007
Kevin H Knuth
Sophisticated Rock Hunting
8 July 2007
MaxEnt 2007
Kevin H Knuth
Ranking
Heavier
3
2
Bok’s new rock
messes up his
ranking!
1
8 July 2007
MaxEnt 2007
Kevin H Knuth
Generalizing Ranking
Heavier
3
2
Bok’s realizes that he
can generalize his
concept of heavier, to
degrees of heaviness
1
8 July 2007
MaxEnt 2007
Kevin H Knuth
Measuring
Heavier
4.2 LR
2.6 LR
1.8 LR
He now measures his
rocks against his
standard
1 LR (Little Rock)
8 July 2007
MaxEnt 2007
Kevin H Knuth
Measuring
Lighter
2.3 Easy
1.5 Easy
Meanwhile Thog, who
has sharpened up a
bit, came up with a
similar methodology
1 Easy
8 July 2007
MaxEnt 2007
Kevin H Knuth
Next Meeting
2.3 Easy
1.5 Easy
Thog, who is proud of
his accomplishments,
brings his ideas to Bok
along with a sheet of
tree bark with numbers
describing his
collection.
8 July 2007
1 Easy
MaxEnt 2007
Kevin H Knuth
Not Easy to Compare
2.6 LR
2.3 Easy
Lighter
Heavier
4.2 LR
1.5 Easy
1.8 LR
1 Easy
1 LR
8 July 2007
MaxEnt 2007
Kevin H Knuth
An Agreement
Thog likes how Bok has
ordered his collection, as
he too is proud of his
biggest rock.
8 July 2007
To compromise, Bok
agrees to use Thog’s
smallest rock as a unit of
weight. Thog brings it to
Bok so he can begin
comparing.
MaxEnt 2007
Kevin H Knuth
Not Easy to Compare
2.6 LR
2.3 Easy
Lighter
Heavier
4.2 LR
1.5 Easy
1.8 LR
1 Easy
1 LR
8 July 2007
MaxEnt 2007
Kevin H Knuth
Adopt a Standard Ordering
2.6 LR
2.3 Rocks
Heavier
Heavier
4.2 LR
1.5 Rocks
1.8 LR
1.0 Rock
1.0 LR
8 July 2007
MaxEnt 2007
Kevin H Knuth
Perform a Regraduation
1.3 Rocks
2.3 Rocks
Heavier
Heavier
2.1 Rocks
1.5 Rocks
0.9 Rock
1.0 Rock
0.5 Rock
8 July 2007
MaxEnt 2007
Kevin H Knuth
On Their Next Meeting…
Thog is pleased that his
little rock is a unit of
measure.
8 July 2007
Bok is surprised to find
that Thog’s rock is actually
heavier than his…
by 0.2 Rocks!
MaxEnt 2007
Kevin H Knuth
An Idea…
I was thinking…
I don’t see why not!
Maybe we could use these
ideas to describe things
other than rocks?
We might be able to
describe anything we can
order.
8 July 2007
MaxEnt 2007
Kevin H Knuth
A Set of Apples
Here is a set of apples
8 July 2007
MaxEnt 2007
Kevin H Knuth
Ordering Apples
Heavier
We can order them
according to
weight
8 July 2007
MaxEnt 2007
Kevin H Knuth
Partially Ordered Set
We can order them
according to
weight

Heavier
Apple 1 is no HEAVIER than
Apple 2
A partially ordered set
is a set along with a
binary ordering relation
Called a Poset for short
8 July 2007
MaxEnt 2007
Kevin H Knuth
Posets of Apples
We can order them
according to
sweetness

Sweeter
Apple 1 is no SWEETER than
Apple 2
8 July 2007
MaxEnt 2007
Kevin H Knuth
Posets of Apples
We can order them
according to how
ripe they are

More Ripe
Apple 1 is no RIPER than
Apple 2
8 July 2007
MaxEnt 2007
Kevin H Knuth
Posets of Apples
More Ripe
This configuration
is called a CHAIN
8 July 2007
MaxEnt 2007
Kevin H Knuth
8 July 2007
5
16
4
8
3
Divides
More Ripe
Is Greater than or Equal to
Isomorphisms
4
2
2
1
1
MaxEnt 2007
Kevin H Knuth
A Set of Fruit
Here is a set of apples along with an orange
8 July 2007
MaxEnt 2007
Kevin H Knuth
Apples and Oranges

With the binary relation “is equally or less HEAVY than”
these two set elements can be compared.
8 July 2007
MaxEnt 2007
Kevin H Knuth
Incomparable
Sometimes two elements of a set cannot be compared using
the binary ordering relation. It may not be possible to tell
which one is SWEETER…they taste different.
Such pairs of elements are called incomparable
8 July 2007
MaxEnt 2007
Kevin H Knuth
Antichains
Say that these three elements are incomparable under our
binary ordering relation
8 July 2007
MaxEnt 2007
Kevin H Knuth
Antichains
These elements are incomparable under our binary ordering relation
They form a poset called an ANTICHAIN
8 July 2007
MaxEnt 2007
Kevin H Knuth
Other Examples
8 July 2007
MaxEnt 2007
Kevin H Knuth
Partitioning
Consider arrangements of fruit on a table.
I could just set the
fruit on the table
Either way, the
table “contains”
the fruit
Or I could put the
apple and banana
on a plate first
and then set it on
the table
8 July 2007
MaxEnt 2007
Kevin H Knuth
Partitioning
A set of elements
together with a
binary ordering
relation  based on
the notion of
containing
8 July 2007
MaxEnt 2007
Kevin H Knuth
4
3
2
8
Divides
Is Greater than or Equal to
Two Posets with Integers
4 6 9
2
1
1
8 July 2007
3 5 7
MaxEnt 2007
Kevin H Knuth
Subsets
Consider the powerset of the set S = { a, b, c }
This is the set of all possible subsets of S:
P(S )   , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} 
A natural ordering is the relation “is a subset of”,

P   , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} , 
8 July 2007
MaxEnt 2007
Kevin H Knuth
The First Level
P   , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} , 
First we note that
  {a}
No element of the set comes
between the two, so we say that
{a} covers  , denoted   {a}
So we draw {a} above 
and connect them with a line.
8 July 2007
MaxEnt 2007
{a}

Kevin H Knuth
Completing the First Level
P   , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} , 
It is also true that   {b} and   {c}
so we draw them above  as well and
connect them with lines.
However, {a} || {b} as neither one is the
subset of the other.
In addition, {a} || {c} and {b} || {c} .
So we draw them on the same level and
do not connect them.
{a} {b}

8 July 2007
MaxEnt 2007
Kevin H Knuth
{c}
The Second Level
P   , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} , 
Now we note that {a} is covered by
two elements
{a, b} and {a, c}.
{a, b} {a, c}
{a} {b}

8 July 2007
MaxEnt 2007
Kevin H Knuth
{c}
The Second Level
P   , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} , 
These elements also
cover
{a, b} {a, c}
{b} and {c}
{a} {b}

8 July 2007
MaxEnt 2007
Kevin H Knuth
{c}
Completing the Second Level
P   , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} , 
{b, c} also
covers {b} and {c} ,
Now
but these top elements
{a, b} {a, c} {b, c}
{a} {b}
are also incomparable.

8 July 2007
MaxEnt 2007
Kevin H Knuth
{c}
The Third Level
P   , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} , 
{a, b, c}
Finally
{a, b, c}
{a, b} {a, c} {b, c}
covers all three two-
{a} {b}
element subsets.

8 July 2007
MaxEnt 2007
Kevin H Knuth
{c}
The Powerset of {a, b, c}
P   , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} , 
{a, b, c}
{a, b} {a, c} {b, c}
Is a subset of
{a} {b}

{c}

8 July 2007
MaxEnt 2007
Kevin H Knuth
Posets
A partially ordered set (poset) is a set of elements
together with a binary ordering relation  .
includes
covers
8 July 2007
31
21
MaxEnt 2007
{a, b, c}  {a}
{a, b}  {a}
Kevin H Knuth
Lattices
A lattice is a poset P where every pair of elements x and y has
a least upper bound called the join
x y
a greatest lower bound called the meet
8 July 2007
MaxEnt 2007
x y
Kevin H Knuth
Lattices
A lattice is a poset P where every pair of elements x and y has
a least upper bound called the join x  y
a greatest lower bound called the meet x  y
{a, b, c}
The green
elements are
{a, b} {a, c}
upper bounds
of the blue circled
pair. The green
circled element is their {a} {b}
least upper bound or
their join.

8 July 2007
MaxEnt 2007
{b}  {c}  {b, c}
{b, c}
{c}
Similarly
{a, b}  {b, c}  {b}
Kevin H Knuth
Lattice of Sets
Note that in this example
Order
Theoretic
Notation
Set
Theoretic
Notation
  
  
  
{b}  {c}  {b, c}
{a, b, c}
{a, b}  {b, c}  {b}
{a, b} {a, c} {b, c}
{a} {b}
{c}

8 July 2007
MaxEnt 2007
Kevin H Knuth
Lattice Identities
The Lattice Identities
L1. x  x  x,
xx  x
L2. x  y  y  x,
Idempotent
x y  y  x
Commutative
L3. x  ( y  z)  ( x  y)  z, x  ( y  z)  ( x  y)  z Associative
L4. x  ( x  y)  x  ( x  y)  x
Absorption
If x  y the meet and join follow the Consistency Relations
C1. x  y  x
C2 . x  y  y
8 July 2007
(x is the greatest lower bound of x and y)
(y is the least upper bound of x and y)
MaxEnt 2007
Kevin H Knuth
Lattices are Algebras
Structural
Viewpoint
Operational
Viewpoint
ab 
8 July 2007
ab  b
a b  a
MaxEnt 2007
Kevin H Knuth
Lattices are Algebras
Structural
Viewpoint
ab 
Operational
Viewpoint
ab  b
a b  a
Structural
Viewpoint
ab 
a b  b
a b  a
Sets with 
Lattices in General
8 July 2007
Operational
Viewpoint
MaxEnt 2007
Kevin H Knuth
Lattices are Algebras
Structural
Viewpoint
ab 
Operational
Viewpoint
ab  b
a b  a
Structural
Viewpoint
ab  b
ab 
a b  a
Lattices in General
8 July 2007
Operational
Viewpoint
Logical Statements
with Implication
MaxEnt 2007
Kevin H Knuth
Lattices are Algebras
Structural
Viewpoint
ab 
Operational
Viewpoint
ab  b
a b  a
Structural
Viewpoint
ab 
max(a, b)  b
min(a, b)  a
Integers with 
Lattices in General
8 July 2007
Operational
Viewpoint
MaxEnt 2007
Kevin H Knuth
Lattices are Algebras
Structural
Viewpoint
ab 
Operational
Viewpoint
ab  b
a b  a
Structural
Viewpoint
a |b 
Lattices in General
8 July 2007
Operational
Viewpoint
lcm(a, b)  b
gcd( a, b)  a
Positive Integers with
Divides
MaxEnt 2007
Kevin H Knuth
Lattices are Algebras
Structural
Viewpoint
ab 
Operational
Viewpoint
ab  b
a b  a
Sets, Is a subset of
ab 
a b  b
a b  a
Positive Integers, Divides
a |b 
lcm(a, b)  b
gcd( a, b)  a
Assertions, Implies
ab  b
ab 
a b  a
8 July 2007
Integers, Is less than or equal to
ab 
MaxEnt 2007
max(a, b)  b
min(a, b)  a
Kevin H Knuth
Hypothesis Space
(Our States of Knowledge)
The Boolean Hypothesis Space
The atoms are mutually exclusive and exhaustive logical statements
a = ‘It is an Apple!’
b = ‘It is a Banana!’
c = ‘It is a Citrus Fruit!’
8 July 2007
MaxEnt 2007
Kevin H Knuth
The Boolean Hypothesis Space
The atoms are mutually exclusive and exhaustive logical statements
a = ‘It is an apple!’
b = ‘It is a banana!’
c = ‘It is a citrus fruit!’
a
8 July 2007
b
MaxEnt 2007
c
Kevin H Knuth
The Boolean Hypothesis Space
The meet of any two atoms is the absurdity: a  b = 
We do not allow our state of knowledge to include:
‘The fruit is an apple AND a banana!’
a
b
c

8 July 2007
MaxEnt 2007
Kevin H Knuth
The Boolean Hypothesis Space
a
b
c

8 July 2007
MaxEnt 2007
Kevin H Knuth
The Boolean Hypothesis Space
The join of any two elements represents a logical OR: a  b
a  b
a
b
c

8 July 2007
MaxEnt 2007
Kevin H Knuth
The Boolean Hypothesis Space
The join of any two elements represents a logical OR: a  b
a  b
a  c
b  c
a
b
c

8 July 2007
MaxEnt 2007
Kevin H Knuth
The Boolean Hypothesis Space
The final join gives us the TOP element, which is the TRUISM
 = abc

“It is an Apple or a Banana or an Orange!”
a  b
a  c
b  c
a
b
c

8 July 2007
MaxEnt 2007
Kevin H Knuth
The Boolean Hypothesis Space
The lattice is ordered according to the ordering relation “implies”.
implies

a  b
a  c
b  c
a
b
c

8 July 2007
MaxEnt 2007
Kevin H Knuth
The Boolean Hypothesis Space
The lattice is ordered according to the ordering relation “implies”.
implies

a  b
a  c
b  c
a
b
c
Meaning is Explicit
8 July 2007

MaxEnt 2007
Kevin H Knuth
The Hypothesis Space
This is a HYPOTHESIS SPACE!!!
It consists of all the statements that can be constructed from a set of

mutually exclusive exhaustive statements.
The space is ordered by the
ordering relation “implies”
a  b
a  c
b  c
b
c
We allow concepts like:
‘The fruit is an apple OR a banana!’
a
while we disallow concepts like:
‘The fruit is an apple AND a banana!’

8 July 2007
MaxEnt 2007
Kevin H Knuth
Superpositions of States???
Consider our piece of fruit…

a  b
a  c
b  c
a
b
c

8 July 2007
MaxEnt 2007
Kevin H Knuth
Superpositions of States???
Consider our piece of fruit…

a  b
a  c
b  c
a
b
c

8 July 2007
MaxEnt 2007
Kevin H Knuth
Superpositions of States???
Physically, we might agree that
the fruit can only be in one of
the states represented by the
atomic statements.

a  b
a  c
b  c
a
b
c
But each of these logical statements
can describe my possible state of
knowledge about the fruit.

WHAT DOES IT MEAN???
8 July 2007
MaxEnt 2007
Kevin H Knuth
What is your State of Knowledge?

8 July 2007
MaxEnt 2007
Kevin H Knuth
Collapse!
If you learn that the
fruit is not a
Banana…

8 July 2007
MaxEnt 2007
Kevin H Knuth
Collapse!

8 July 2007
MaxEnt 2007
Kevin H Knuth
Collapse!

8 July 2007
MaxEnt 2007
Kevin H Knuth
Collapse!

8 July 2007
MaxEnt 2007
Kevin H Knuth
Collapse!

8 July 2007
MaxEnt 2007
Kevin H Knuth
Collapse!

8 July 2007
MaxEnt 2007
Kevin H Knuth
Collapse!

8 July 2007
MaxEnt 2007
Kevin H Knuth
The New Hypothesis Space

8 July 2007
MaxEnt 2007
Kevin H Knuth
It is an Orange!

8 July 2007
MaxEnt 2007
Kevin H Knuth
It is an Orange!

8 July 2007
MaxEnt 2007
Kevin H Knuth
It is an Orange!

8 July 2007
MaxEnt 2007
Kevin H Knuth
It is an Orange!

8 July 2007
MaxEnt 2007
Kevin H Knuth
It is an Orange!

8 July 2007
MaxEnt 2007
Kevin H Knuth
It is an Orange!

8 July 2007
MaxEnt 2007
Kevin H Knuth
It is an Orange!

8 July 2007
MaxEnt 2007
Kevin H Knuth
It is an Orange!

8 July 2007
MaxEnt 2007
Kevin H Knuth
The Final Hypothesis Space

8 July 2007
MaxEnt 2007
Kevin H Knuth
The State of the Fruit
This was your
initial state of
knowledge.
The fruit was
never in this
state!
This was the
state of the
fruit.

8 July 2007
MaxEnt 2007
Kevin H Knuth
Two Spaces

State Space of Your Knowledge
about the State of the Fruit
State Space of the Fruit
8 July 2007
MaxEnt 2007
Kevin H Knuth
Microstates
Consider a system with 3 microstates i each of energy E.
1
8 July 2007
MaxEnt 2007
2
3
Kevin H Knuth
State of Knowledge about Microstates
Consider a system with 3 microstates i each of energy E.
We can’t know which microstate
the system is in.
We only know that it is in one of
the 3 microstates.
 1  2  3
1  2
1  3
2  3
1
2
3

8 July 2007
MaxEnt 2007
Kevin H Knuth
Macrostates
This is OUR STATE OF
KNOWLEDGE!
 1  2  3
1  2
1  3
2  3
1
2
3

8 July 2007
MaxEnt 2007
Kevin H Knuth
Macrostates
This is what we call the
MACROSTATE
 1  2  3
It is an equivalence class
of microstates
But if we agree that microstates
are the physical states of the
system, then we must admit that
the system is never in the
macrostate.
1  2
1  3
2  3
1
2
3
Why?
These are two separate spaces!
8 July 2007
MaxEnt 2007

Kevin H Knuth
States of Knowledge
Statistical Mechanics works in
the space representing our
STATE OF KNOWLEDGE
 1  2  3
NOT the state of the system!
This is why it is an inferential
theory that depends on
measures such as probability
and entropy.
1  2
1  3
2  3
1
2
3

8 July 2007
MaxEnt 2007
Kevin H Knuth
Mind Projection Fallacy
Jaynes’ Mind Projection Fallacy warns that it is fallacious to view our
state of knowledge about a system as a property possessed by the
system.

Which begs the question:
a  b
a  c
b  c
a
b
c
Since we work with hypothesis spaces,
is there such a thing as a
STATE SPACE?

8 July 2007
MaxEnt 2007
Kevin H Knuth
Mind Projection Fallacy
Jaynes’ Mind Projection Fallacy warns that it is fallacious to view our
state of knowledge about a system as a property possessed by the
system.

Which begs the question:
a  b
Since we work with hypothesis spaces,
is there such a thing as a
STATE SPACE?
a
ARE THE LAWS OF PHYSICS
ACTUALLY RULES OF INFERENCE?
a  c
b  c
b
c

8 July 2007
MaxEnt 2007
Kevin H Knuth
Mind Projection Fallacy
Jaynes’ Mind Projection Fallacy warns that it is fallacious to view our
state of knowledge about a system as a property possessed by the
system.

Which begs the question:
a  b
Since we work with hypothesis spaces,
is there such a thing as a
STATE SPACE?
a
ARE THE LAWS OF PHYSICS
ACTUALLY RULES OF INFERENCE?
a  c
b  c
b
c
As Carlos Rodriguez once asked:

“ARE WE CRUISING A HYPOTHESIS SPACE?”
8 July 2007
MaxEnt 2007
Kevin H Knuth
Two Spaces
The State Space is a MODEL that
generates the HYPOTHESIS SPACE.
We make inferences in the
HYPOTHESIS SPACE

State Space of Your Knowledge
about the State of the Fruit
State Space of the Fruit
8 July 2007
MaxEnt 2007
Kevin H Knuth
Intermission
8 July 2007
MaxEnt 2007
Kevin H Knuth
Measures
Inclusion and the Zeta Function

a  b
a
a  c
b
b  c
c
The Zeta function encodes
inclusion on the lattice.
 1 if x  y
 ( x, y)  
 0 if x  y

8 July 2007
MaxEnt 2007
Kevin H Knuth
Inclusion and the Zeta Function

a  b
a
a  c
b

8 July 2007
b  c
c
The Zeta function encodes
inclusion on the lattice.
 1 if x  y
 ( x, y)  
 0 if x  y
 (a, a  b)  1 since a  b  a
 (a  b, a)  0 since a  a  b
MaxEnt 2007
Kevin H Knuth
The Zeta Function

a  b
a  c
a
b

8 July 2007
1
 ( x, y )  
0
if x  y
if x  y
b  c
c

a
b
c
avb
avc
bvc
T

1
1
1
1
1
1
1
1
a
0
1
0
0
1
1
0
1
b
0
0
1
0
1
0
1
1
c
0
0
0
1
0
1
1
1
avb
0
0
0
0
1
0
0
1
avc
0
0
0
0
0
1
0
1
bvc
0
0
0
0
0
0
1
1
T
0
0
0
0
0
0
0
1
MaxEnt 2007
Kevin H Knuth
Inclusion and the Zeta Function
The Zeta function encodes inclusion on the lattice.
 ( x, y )

 1 if x  y

 0 if x  y
We can define its dual by flipping around the ordering relation
 ( x, y )

8 July 2007

 1 if x  y

 0 if x  y
MaxEnt 2007
Kevin H Knuth
Degrees of Inclusion and Z
We generalize the dual of the Zeta function
 ( x, y )


 1 if x  y

 0 if x  y

1 if x  y

 z if x  y
 0 if x  y 

to the function z
z ( x, y )
8 July 2007
MaxEnt 2007
Kevin H Knuth
Z
The function z
z ( x, y )

1 if x  y

 z if x  y
 0 if x  y 

Continues to encode inclusion, but has generalized the
concept to degrees of inclusion.
In the lattice of logical statements ordered by implies, this
function describes degrees of implication.
8 July 2007
MaxEnt 2007
Kevin H Knuth
How do we Assign Values to z?

a
b
c
avb
avc
bvc
T

1
0
0
0
0
0
0
0
a
1
1
0
0
?
?
0
?
b
1
0
1
0
?
0
?
?
c
1
0
0
1
0
?
?
?
avb
1
1
1
0
1
?
?
?
avc
1
1
0
1
?
1
?
?
bvc
1
0
1
1
?
?
1
?
T
1
1
1
1
1
1
1
1
z ( x, y )

1 if x  y

 z if x  y
 0 if x  y 

Are all of the values of the function z arbitrary?
Or are there constraints?
Here there be monsters…
8 July 2007
MaxEnt 2007
Kevin H Knuth
Lattice Structure Imposes Constraints
Following the inspiration of R.T. Cox (1947) and A. Caticha (1998):
Consider Associativity of the Join
a  (b  c)  (a  b)  c
We begin by considering the special case where
a b  
Instead of talking about the function z, consider a related
valuation  (a  b, d )
Consistency would require that the valuation of the join of two
elements be related to the valuations of the individual
elements:
 (a  b, d )  S[ (a, d ),  (b, d )]
8 July 2007
MaxEnt 2007
Kevin H Knuth
Lattice Structure Imposes Constraints
Consider Associativity of the Join
a  (b  c)  (a  b)  c
We begin by considering the special case where
a b  
Instead of talking about the function z, consider a related
valuation  (  ,  )
Consistency would require that valuation of the join of two
elements be related to the valuations of the individual
elements via some unknown function S
 (a  b, d )  S[ (a, d ),  (b, d )]
  (a, d )   (b, d )
8 July 2007
MaxEnt 2007
Kevin H Knuth
Consistency
Now join a new element c to the previous pair:
 ((a  b)  c, d )
But by Associativity, this is also equal to
 (a  (b  c), d )
So
 ((a  b)  c, d )  ( (a, d )   (b, d ))   (c, d )
 (a  (b  c), d )   (a, d )  ( (b, d )   (c, d ))
8 July 2007
MaxEnt 2007
Kevin H Knuth
Simplify Notation
Let
 ( a, d )  u
 (b, d )  v
 ( c, d )  w
Our functional equation for S
( (a, d )   (b, d ))   (c, d )   (a, d )  ( (b, d )   (c, d ))
becomes:
(u  v)  w  u  (v  w)
This is known as the Associativity Equation
8 July 2007
MaxEnt 2007
Kevin H Knuth
Solution
The general solution (Aczel 1966) is:
1
1
u  v  f ( f (u)  f (v))
For arbitrary function f
which gives
1
1
1
f (u  v)  f (u)  f (v)
The fact that f is arbitrary, suggests there is a convenient
representation (Caticha 1998). So define:
z( ,  )  f ( (  ,  ))
1
8 July 2007
MaxEnt 2007
Kevin H Knuth
The Associativity Constraint
The Associativity Constraint requires that
z(a  b, d )  z(a, d )  z(b, d )
when a  b  
In general, I have shown that
z(a  b, d )  z(a, d )  z(b, d )  z(a  b, d )
8 July 2007
MaxEnt 2007
Kevin H Knuth
Inclusion-Exclusion (The Sum Rule)
z(a  b, d )  z(a, d )  z(b, d )  z(a  b, d )
The Sum Rule for Lattices
8 July 2007
MaxEnt 2007
Kevin H Knuth
Inclusion-Exclusion (The Sum Rule)
z(a  b, d )  z(a, d )  z(b, d )  z(a  b, d )
p( a  b | i )  p( a | i )  p( b | i )  p( a  b | i )
The Sum Rule for Probability
8 July 2007
MaxEnt 2007
Kevin H Knuth
Inclusion-Exclusion (The Sum Rule)
z(a  b, d )  z(a, d )  z(b, d )  z(a  b, d )
p( a  b | i )  p( a | i )  p( b | i )  p( a  b | i )
I ( A; B)  H ( A)  H ( B)  H ( A, B)
Definition of Mutual Information
8 July 2007
MaxEnt 2007
Kevin H Knuth
Inclusion-Exclusion (The Sum Rule)
z(a  b, d )  z(a, d )  z(b, d )  z(a  b, d )
p( a  b | i )  p( a | i )  p( b | i )  p( a  b | i )
I ( A; B)  H ( A)  H ( B)  H ( A, B)
max(a, b)  a  b  min(a, b)
Polya’s Min-Max Rule for Integers
8 July 2007
MaxEnt 2007
Kevin H Knuth
Inclusion-Exclusion (The Sum Rule)
z(a  b, d )  z(a, d )  z(b, d )  z(a  b, d )
p( a  b | i )  p( a | i )  p( b | i )  p( a  b | i )
I ( A; B)  H ( A)  H ( B)  H ( A, B)
max(a, b)  a  b  min(a, b)
This is intimately related to the Möbius function for the lattice,
which is related to the Zeta function.
8 July 2007
MaxEnt 2007
Kevin H Knuth
Lattice Structure Imposes Constraints
I showed that in “general”:
Associativity x  ( y  z )  ( x  y )  z leads to a Sum Rule…
z( x  y, w)  z( x, w)  z( y, w)  z( x  y, w)
Distributivity x  ( y  z)  ( x  y )  ( x  z) leads to a Product Rule…
z ( x  y, w)  z ( x, w) z ( y, x  w)
Commutivity x  y  y  x
leads to Bayes Theorem…
z ( x, w) z ( y, x  w)
z ( x, y  w) 
z( y, w)
8 July 2007
MaxEnt 2007
Kevin H Knuth
Unification
The Sum Rule, Product Rule and Bayes Theorem are
CONSTRAINT EQUATIONS
These are UNIFYING concepts
whereas axiomatics are obfuscating
8 July 2007
MaxEnt 2007
Kevin H Knuth
Probability
Changing notation
1 if x  y

z ( x, y )   z if x  y
 0 if x  y 

 1 if y  x

p ( x | y )   p if y  x
 0 if x  y 

The MEANING of p(x|y) is made explicit via the Zeta function,
which encodes implication.
These are degrees of implication!
The meaning is imposed by the ordering relation.
- No Guesswork
- No Confusion
8 July 2007
MaxEnt 2007
Kevin H Knuth
Priors
The prior probabilities are the Zeta function values that remain
unconstrained after we take into account the lattice structure.
How to assign them???
Find other constraints relevant to the application.
It is a good thing that these priors are unconstrained by the
lattice structure. If they weren’t, this formalism would not be
useful.
8 July 2007
MaxEnt 2007
Kevin H Knuth
How to Derive a Measure
Knuth K.H. 2003. Deriving laws from ordering relations.
1. Define the objects that you want to measure
2. Select the appropriate ordering relation
3. Determine the algebra from the lattice/poset structure
4. Use the constraints from the algebra to derive a calculus
5. The meaning of the measure is imposed by the ordering
relation.
Skipping steps by relying on intuition will most likely result in
errors and waste time and effort!!!
8 July 2007
MaxEnt 2007
Kevin H Knuth
Geometric Probability
Many geometric laws can be derived from order-theoretic
considerations.
Geometric objects can
be ordered, conjoined
and disjoined often
resulting in a distributive
lattice structure.
Valuations are assigned,
which are invariant with
respect to Euclidean
translations and rotations.
Gian-Carlo Rota
8 July 2007
MaxEnt 2007
Kevin H Knuth
Joining Parallelotopes
P1  P2
P1
P2
v( P1  P2 )  v( P1 )  v( P2 )  v( P1  P2 )
8 July 2007
MaxEnt 2007
Kevin H Knuth
These Valuations have a Basis!
For three-dimensional Euclidean geometry all invariant
valuations can be written as a linear combination of 4 basis
valuations.
V = volume
A = surface area
W = mean width
c = Euler characteristic
m = aV + bA + cW + dc
8 July 2007
MaxEnt 2007
Kevin H Knuth
Euler Characteristic
The Euler characteristic is a valuation.
c V  E  F
For a 3D tetrahedron it is found by
c (tetra)  4  6  4  2
c (cube)  8 12  6  2
8 July 2007
MaxEnt 2007
Kevin H Knuth
What is the Ordering
Behind
Quantum Mechanics?
8 July 2007
MaxEnt 2007
Kevin H Knuth
Ariel Caticha
Ariel has also developed a very
interesting derivation of quantum
mechanics using Cox’s method
applied to experimental setups
rather than logical statements.
The concept of consistency with
the order-theoretic structure is
central here as well.
8 July 2007
MaxEnt 2007
Kevin H Knuth
Particles and Motion
xf
A particle moves from xi to xf
[ x f , xi ]
time
xi
8 July 2007
MaxEnt 2007
Kevin H Knuth
A Little More Complex
xf
A particle moves from xi to x1
and then from x1 to xf
[ x f , x1 , xi ]
x1
time
xi
8 July 2007
MaxEnt 2007
Kevin H Knuth
A Little More Complex
xf
A particle goes from xi to xf
via x1 or x’1
[ x f , ( x1, x1), xi ]
x1
x1
time
xi
8 July 2007
MaxEnt 2007
Kevin H Knuth
Experimental Setups
We can look at this
experimental setup as
xf
x1
time
xi
8 July 2007
MaxEnt 2007
Kevin H Knuth
The Meet Operation
We can look at this
experimental setup as
being a combination of
two setups…
xf
x1
time
[ x f , x1 ]  [ x1 , xi ]
 [ x f , x1 , xi ]
8 July 2007
xi
MaxEnt 2007
Kevin H Knuth
The Meet Operation
We can look at this
experimental setup as
being a combination of
two setups…
xf
x1
time
[ x f , x1 ]  [ x1 , xi ]
 [ x f , x1 , xi ]
8 July 2007
xi
MaxEnt 2007
Kevin H Knuth
The Join Operation
xf
This is a different way to
combine setups
x1
x1
time
[ x f , x1 , xi ]  [ x f x1, xi ]
 [ x f , ( x1 , x1 ), xi ]
8 July 2007
MaxEnt 2007
xi
Kevin H Knuth
The Join Operation
[ x f , ( x1, x1), xi ]  [ x f , x1 , xi ]  [ x f x1, xi ]
DS  SS  SS'
8 July 2007
MaxEnt 2007
Kevin H Knuth
NOT a Lattice Structure
What is interesting about setups is that because not
all meets and joins exist, setups do not form a lattice
structure.
They do form something like a poset however.
As the measure we will define is not probability, Ariel
represented it with (a ) rather than p(a | i)
So lets continue…
8 July 2007
MaxEnt 2007
Kevin H Knuth
Sum and Product Rules Again
Caticha showed that the
Sum Rule is derived from Associativity of the Join.
 (a  b)   (a)   (b)
Product Rule from Distributivity.
 (a  b)   (a)  (b)
Feynman Path Integrals arise from the Sum Rule
BUT WHY COMPLEX NUMBERS?!?!?!
8 July 2007
MaxEnt 2007
Kevin H Knuth
The Join Operation
xf
This is a different way to
combine setups
x1
x1
time
[ x f , x1 , xi ]  [ x f x1, xi ]
 [ x f , ( x1 , x1 ), xi ]
8 July 2007
MaxEnt 2007
xi
Kevin H Knuth
The Join Operation
[ x f , ( x1, x1), xi ]  [ x f , x1 , xi ]  [ x f x1, xi ]
DS  SS  SS'
Double-Slit
8 July 2007
Single-Slit
MaxEnt 2007
Kevin H Knuth
The Top
Clearly, if we keep
joining slit experiments
together this way, we
will eventually, have
one giant slit.
This is the Top
element of the poset.
It represents the free
particle traveling from
xi to xf.
8 July 2007
MaxEnt 2007
Kevin H Knuth
The Bottom
The Bottom is the
completely obstructed
particle.
8 July 2007
MaxEnt 2007
Kevin H Knuth
The Atomic Elements
The Atomic Elements are simple paths.
[ x f , , x2 , x1, , xi ]
8 July 2007
MaxEnt 2007
Kevin H Knuth
A Peek at the “Poset”
This is merely a portion of the
“poset” of experimental setups
for the slit experiment case.
It is NOT a Lattice!
It is NOT Boolean!
It is NOT Commutative!
I have said nothing about the
necessity of complex numbers
8 July 2007
MaxEnt 2007
Kevin H Knuth
Some Lessons
Volumes are NOT Surface Areas
Even though they are measures on the same set!
There may be multiple distinct measures for any given
poset.
Measures on Different Posets are NOT the Same
Quantum Mechanics is NOT Probability Theory!
First order of business…
Get Your Ducks in a Row!!!
8 July 2007
MaxEnt 2007
Kevin H Knuth
Special Thanks to:
John Skilling
Ariel Caticha
Philip Goyal
Steve Gull
Carlos Rodriguez
for many insightful discussions and comments
And to:
http://www.nyapplecountry.com/
For all sorts of images