Transcript rubato composer - Georgia State University




Origins in PRESTO, and early computer
application developed by Guerino Mazzola.
RUBATO is a universal music software
environment developed since 1992 under the
direction of Guerino Mazzola.
RUBATO COMPOSER system developed in
Gérard Milmeister’s doctoral dissertation (2006)
where he implemented the Functorial Concept
Architecture, based on the data format of Forms
and Denotators. http://www.rubato.org/


This software works with components called
rubettes ( perform basic tasks for musical
representation and whose interface with other
rubettes is based on the universal data format
of denotators).
The data format of denotators uses set-valued
presheaves over the category of modules and
diaffine morphisms http://www.rubato.org/


In addition to what RUBATO COMPOSER
is designed to be for the composer and music
theorist, it is also an excellent tool for learning
sophisticated mathematical concepts.
The mathematics involved are sophisticated,
and could be accessible in a formal way to the
average mathematics student in their senior
year, after having some experience with
courses such as Linear Algebra, Modern
Algebra, Analysis or Topology, but would
usually be taught at the graduate level.



Possibilities of knowledge expansion and new
applications
Danger of superficiality, contamination and
surrender to fashion.
Possibilities that Mathematical Music Theory,
and its applications, have to offer to the
knowledge base of mathematics, music, and
computer science students, without excluding
those from other fields.



It has been generally acknowledged that there is a
gap between the formality of modern mathematics
as conceived and taught by trained
mathematicians, and the mathematics that are seen
by non mathematicians as relevant.
When the mathematics are embedded in different
practical contexts, it is often easier to get students to
think mathematically in a natural manner
Even mathematics students themselves often have
difficulty in making meaning out of the formal
presentation of their subject (MacLane, 2005).

The creation of interdisciplinary
curriculum materials and courses,
using RUBATO COMPOSER as a
common ground, opens a realm of
possibilities for Mathematical Music
Theory, and for the development of
research and researchers in the field.
It also can be justified, in and of
itself, where RUBATO COMPOSER
is conceived of as a learning tool.


The International Journal of Computers for
Mathematical Learning “… publishes
contributions that explore the unique potential
of new technologies for deepening our
understanding of the field of mathematics
learning and teaching.”
A revision of articles from 2006-8, illustrates
that the notion of using specific software to
enhance the learning of mathematics has a
respectable recent history, and has been
analyzed using well documented research
paradigms.


Using formalism to construct
meaning is a very difficult method
for students to learn, but this is the
only route to learning large
portions of mathematics
The writing of computer programs
to express mathematical concepts
can be an effective way of achieving
this goal of advanced mathematical
learning.(Dubinsky, 2000)


The RUBATO COMPOSER software opens up
the possibility of creating meaning behind the
formalisms of advanced mathematical areas,
and accelerating processes of learning and
understanding.
These mathematical areas (Abstract Algebra
beyond Group Theory, Category and Topos
theory) are not really addressed in the
literature on computer-based learning, or on
collegiate mathematics education in general.




Computer-based learning in music is usually
related to training in aural skills, sight reading,
and other subjects essential to the music student.
Musical representation languages such as
Common Music, OpenMusic, and Humdrum, for
composing and analyzing music, that do require
programming skills.
However, RUBATO COMPOSER offers the
opportunity of introducing the music student to
the higher mathematics involved in modern
Mathematical Music Theory.
This can be done in a relatively (not completely)
“painless” manner, as compared to what it would
require to learn this material in the traditional way.



RUBATO COMPOSER is based on the data
format of Forms and Denotators.
Forms are mathematical spaces with a precise
structure, and Denotators are objects in the
Form spaces.
Category theory is the mathematical
foundation on which this particular conceptual
basis of Mathematical Music Theory is built.

In the RUBATO COMPOSER architecture,
modules are a basic element, much like primitive
types in programming languages.

The recursive structure of a Form, if not circular,
will eventually “stop” at a Simple form which, for
all practical purposes, is a module.

Morphisms between modules (changes of
address), are built into the software.

In the development of the data base management
systems, the objects must be named and defined
in a recursive way and they must admit types that,
in this context, are such as limit, colimit, and
power.

It is necessary to work with the algebraic structure
of modules, yet form constructions whose
prototypes are found in the category of sets.

This is the reason why, in the context of RUBATO
COMPOSER, the approach is to work in the
functor category of presheaves over modules
(whose objects are the functors ).


Through the creation of denotators, and the
recursive structure of types when working
with forms, the mathematics student
accustomed to the formalism of abstract
mathematics has the opportunity to
participate in a concrete implementation of
these concepts.
The mathematics student who still struggles to
find meaning in the abstract formalism, may
find a vehicle through which this process can
be accelerated.




The majority of rubettes available at this time are
of low level nature.
One of the objectives of the developers of
RUBATO COMPOSER is to create more high level
rubettes that present ‘friendlier’ interfaces and
language for the non-mathematical user, in
particular the composer or musicologist.
However, musicians interested in using
technology in an innovative manner, cannot isolate
themselves from the mathematics used to create
their tools.
The musical analysis itself, and much of the
musical ontology, is intricately related to the
mathematical framework.


The music student does not have to deal with
the mathematical objects in the same way as
the mathematics student (nor does the
computer science student). However, if he
wants to follow the developments of research
in Mathematical Music Theory, he needs an
understanding of the language and concepts
behind the tools.
This is especially true in the case of RUBATO
COMPOSER, which has been designed as the
result of a precisely defined, and perhaps
revolutionary, approach to musical analysis.


Even with the high level rubettes that are, and
will be, available, it is possible to retrace the
steps and uncover the mathematics behind
their construction.
When the terminology changes from
‘transposition’ to ‘translation’ and, in general,
from the musical ‘inversion’, ‘retrograde’,
‘augmentation’, to the language of
mathematical transformations, or morphisms,
the music student is presented with an
opportunity to develop an understanding of
the meaning behind the formalism.


In RUBATO COMPOSER not only translations,
but general affine morphisms as well, can be
used to generate musical ornamentation.
The Wallpaper rubette, developed by Florian
Thalmann, also opens the possibility of
generating morphisms in any n-dimensional
space (for example, using the five simple forms
of the Note denotator - Onset, Pitch, Duration,
Loudness and Voice- an affine transformation
in 5D can be defined).


When working with affine transformations in 2D
space, the command can be given by just
‘dragging’, instead of defining the morphisms.
A unit introducing the basic concepts of linear
algebra, group theory, and geometry needed to
understand mathematical music theory, as it has
been developed over the last 40 years, can be
created.




Most ‘extreme’ example, up until now, the
BigBang rubette, developed by Florian
Thalmann, in the context of Mathematical
Gesture Theory and Computer Semiotics.
Based on a general framework for geometric
composition techniques.
Given a set of notes, their image is calculated
through affine invertible maps in ndimensional space.
There is a theorem that states that the affine
invertible map in n dimensions can be
written as a composition of transformations,
each one acting on only one or two of the n
dimensions.






The component functions (act on only one or
two of the n dimensions) represented
geometrically as five standard mathematical
operations that have their musical
representation:
Translation (transposition in music)
Reflection (inversion, retrograde in music)
Rotation (retrograde inversion in music)
Dilations (augmentations in music)
Shearings (arpeggios).


Sample of a Unit and its Focus on
different Majors
A sample unit has been created to show how the
analysis and creation of a musical object can give
students from different disciplines, in particular
mathematics, computer science and music, a deeper
understanding of abstract mathematics while
satisfying aesthetic interests as well.
Description of the Module: The development of a
melodic phrase, recursively transformed by
transformations in the plane as ornamentation,
using the Wallpaper rubette in RUBATO
COMPOSER
 Objectives and Activities:
All students will be able to:
 Identify rigid transformations in the plane and
give them musical meaning. For example:






mathematical translations – musical transpositions;
mathematical reflection – musical inversion, retrograde;
mathematical rotation – musical inversion-retrograde;
mathematical dilatation – musical augmentation in time;
mathematical shearing – musical arpeggios in time.



All students will be able to:
Use the software RUBATO COMPOSER and, in
particular, the Wallpaper rubette, to generate
musical ornamentation by means of diagrams of
morphisms (functions).
Create and interpret transformations, and
compositions of transformations, like the
following, in which f1 is a rotation of 180
followed by a translation, and f 2 is a translation.



Select any of the coordinates of a Note denotator
(which is 5-dimensional) and combine them.
When two coordinates are chosen, say Onset and
Pitch, students will relate them to the rigid
transformations in the Euclidean plane.
Mathematics students (and those from computer
science and other related areas) will formally
construct the morphisms, while Music students
can use the succession of primitive
transformations by dragging with the mouse
Mathematics students will be able to:
 Construct the composition of module morphisms from the Form
Note to the Form Note. For example, using the coordinates Onset
and Pitch, they can construct the following composition of
embeddings, projections and affine transformations.

The creation of a melodic phrase where i1 and

i2 are the injections
2

and e2 is the embedding
. The transformation (a
musical embellishment, as seen in the previous slide) is then applied, and
to return the coordinates to the module morphisms
on and
pn
the projections p1 and p2 are applied where c represents
quantized to


on
=
p1 ○ f ○ (( i1○ o) + (i2 ○e2○ p)): A →
pn = c ○p2○ f ○ ((i1 ○ o) + ( i2○e2○ p)): A →
General Topic
Mathematics Highlights
Music Highlights
Transformational Theory
Group Theory, Set Theory, Analysis of Musical Works,
Function Theory, Geometry, from all genres. In the case of
Topology
Classical music, analysis of
modern atonal music that is
not
approachable
with
traditional tools from music
theory.
Subdivision of the Octave; Group
Theory,
Number Exploration of different and
Maximally Even Sets
Theory, Differential Equations, exotic scales; Composition
Continuous Fractions
using unusual scales.
Forms and Denotators; The Category and Functor Theory,
Software
RUBATO Topos Theory, Sets and
COMPOSER
Modules, Linear, Affine and
Diaffine
Transformations.
Linear Algebra, Geometry,
Mathematical Gesture Theory
Music
Composition;
Embellishment of existing
music;
Algorithms
for
Composition;
Counterpoint
Theory





Mazzola, G, Milmeister, G, Morsy, K., Thalmann, F. Functors for Music:
The Rubato Composer System. In Adams, R., Gibson, S., Müller Arisona,
S. (eds.), Transdisciplinary Digital Art. Sound, Vision and the New Screen,
Springer (2008).
Milmeister, Gérard. The Rubato Composer Music Software ComponentBased Implmentation of a Functorial Concept Architecture. Springer-Verlag
(2009).
Thalmann, Florian and Mazzola, Guerino. The BigBang Rubette: Gestural
Music Composition with Rubato Composer ICMC 2008
http://classes.berklee.edu/mbierylo/ICMC08/defevent/papers/cr1316.p
df
Thalmann, Florian. Musical Composition with Grid Diagrams of
Transformations, Masters Thesis, Bern (2007)
International Journal of Computers for Mathematical Learning,
http://www.springer.com/education/mathematics+educatio
n/journal/10758


Dubinsky, E.: Meaning and Formalism in Mathematics, Int J Comput
Math Learning, 5, 211-240 (2000).
MacLane, Saunders. Despite Physicists, Proof is Essential in
Mathematics. Synthese 111, 2, 147-154 (May, 1997).
Creating and Implementing a Form
Space and Denotator for Bass Using
the Category-Theoretic Concept
Framework
The Dilemma
•
The dilemma resides in how to
maintain the algebraic structure
of the category of modules(over
any ring, with diaffine
morphisms) and, at the same
time, construct such objects as
limits, colimits, and power, and
classify truth.
The Dilemma
•
The functorial approach leads to the
resolution of this dilemma by working in the
category of presheaves over modules (whose
objects are the functors F: Mod → Sets, and
whose morphisms are natural transformations
of functors) and which will be denoted as
Mod@.
• This category is a topos, which means that it
allows all limits, colimits, and subobject
classifiers Ω, while retaining the algebraic
structure that is needed from the category
Mod.
My Research
• My research consists of creating a form
broad enough for the majority of simple
electric bass scores, and a denotator
which represents the jazz song “All of
Me”. The recursivity of the mathematical
definitions of form and denotator are
made evident in this application
Bass Score Form
Name Form Bass Score
GeneralNotes
SimpleNote
Denotator “SimpleNote”
•
For an example of how to
build a denotator I will take a
small part of my denotator
named “SimpleNote”
Creating the Denotator of “All of
Me”
•
A single denotator N1 of the form:
SimpleNote is created from the coordinates
of the denotator which themselves are forms
of type simple: Onset, Pitch, Duration,
Loudness, and Voice.
Creating the Denotator of “All of
Me”
•
•
As we don't have time to see
how all of the module
morphisms are constructed we
will build the pitch module
morphism “mp”.
All the others are built
analogously.
Bassline for “All of Me”
Creating the Denotator in Rubato
Composer
•
To create the denotator for the Jazz song “All
of Me” we must first create the Module
Morphisms
Creating the Denotator in Rubato
Composer
•
•
Once we've opened our module
morphism builder in Rubato
Composer we will start creating a
module morphism for pitch.
First we must create “mp2” and “mp1”
and then they will be used to make
“mp”, which is a composition of the
two.
Creating the Denotator in Rubato
Composer
•
•
•
For mp2 the domain is determined from
the number of musical notes in the bass
line.
For instance, the bass line I wrote for “All
of Me” contains 64 notes.
We establish the first note as the anchor
note, so for our domain we use Z63.
Creating the Denotator in Rubato
Composer
•
mp2 is an embedding of the canonical vectors plus the
zero vector, that goes from Z63 → Q63.
mp1
•
•
mp1 will take us from Q63→ Q.
We will set up the module
morphism in the same way as
mp2, except we will select
affine instead of canonical.
mp1
mp
•
•
To create mp, we bring up the
module morphism builder,
and create a module
morphism with the domain
of mp2 and a codomain of
mp1.
Which results in mp1○mp2 =
mp.
mp
mp
•
mp: Z63 → Q63 → Q: x → (4, 7, 9, 7, 4,
0, 2, 4, 8, 11, 8, 4, 2, -1, -4, -3, -1, 1, 4,
7, 6, 5, 4, 2, 5, 9, 12, 11, 9, 5, 2, 4, -8, 3,
4, 8, 4, 0, -1, -3, -1, 0, 4, 9, 7, 5, 4, 2, 6,
12, 11, 9, 6, 0, 2, 5, 9, 2, 6, 7, -5, -1,
2)●x + 48
•
Example, when x=(0,...,0)
(4,..., 2)●(0,...0) + 48 → 0+48 → 48
Which is the first note in our bassline.
Module Morphisms
•
•
All of the Module Morphisms
are made in the same way.
Once all of the Module
Morphisms are built we can
arrange them using the
denotator builder.
Denotator Builder
Creating Denotator
Using Rubettes in Rubato
Composer
•
•
To play our bass line in Rubato
Composer, we must create a network
using rubettes.
We will need to set up three rubettes;
the Source rubette, the @AddressEval
rubette, and the Scoreplay rubette.
Source
Source
Source
@AddressEval Rubette
•
•
Next open the @AddressEval
Rubette.
This rubette will be directly
connected to the Source
rubette.
Scoreplay Rubette
•
•
To play our bassline we need
to open the scoreplay rubette
and connect it to the
@AddressEval rubette.
This is done the same way as
the Source and @AddressEval
rubettes.
Finished Network
Pianola
Rubato Composer and its
Functorial Approach:
From Morphisms to Gestures through
Rubettes
Jonathan Cantrell
Junior Mathematics
Georgia State University
Perspectives on Music
Where did we start?
 Melodies and harmonies applied across time
 Often written in sequential fashion
Where are we now?
 Asynchronous editing tools allows a composer to
work in non-linear fashion
Digital Audio Workstations e.g Pro Tools
MIDI Sequencers e.g Logic, Cubase
 Still music is written in distinct phrases and compiled
together
Architecture of Rubato Composer
Defined recursively
Utilizes the Form and Denotator concept
Allows for heterogeneous types
 Simple, Limit, Colimit, or Power
Implemented as free modules over rings
Working from the category of presheaves over
modules
Diaffine Transformations
Working in the category Mod of modules over
any ring whose morphisms are diaffine
transformations which gives us the ability to
perform operations from one module to another
Diaffine transformations are module morphisms
plus a translation
 A dilinear homomorphism from an R-module M to an
S-module N plus a translation in N
Why Topoi?
Rigidly defined categories which are a
generalization of the category of Sets
Allows the composer to perform set-valued
operations where the elements in the sets are
module morphisms over any ring
Sets are required within the framework of the
Form and Denotator concept as the evaluation
of the colimit of Score:Note
Geometric Representation
Module morphisms in n-dimensional space
embedded into a Form of type Simple
Any diaffine transformation h in n-dimensional
space can be written as a composition of
transformations hi which involve only one or two
dimensions of the n dimensions and leave the
others unchanged
In our particular example, we are in ℝ5 we want
to operate exclusively on pitch and onset,
therefore we apply this construct to work in ℝ2
Why Mod?
Why does the algebraic structure need to be
retained?
It allows us to map individual Simple Forms to
the plane and perform affine transformations in
a different space, as exemplified in the
Wallpaper rubette
We apply the mathematical tools of translation
as musical transpositions, and reflections as
retrograde
My Research
I have developed an example of a 12-note
melodic phrase recursively transformed using
the Wallpaper rubette. I further generalize this
series of transformations using the high-level
tool of the BigBang rubette available in Rubato
Composer
Compound Transformations
An example of two translations applied
recursively.
For a specific Note denotator, we operate
exclusively on the module morphisms Onset
and Pitch
Compositions
As stated, the module morphisms contained in
the Simple forms Onset, o: A → ℝ and Pitch,
p: A → ℚ are extracted from the Note denotator
We now need to compose these module
morphisms as follows
Compostitions
This is described by the following compositions:
 i₁ ○ o : A → ℝ2,
 i₂ ○ e₂ ○ p : A → ℝ2,
Where i₁ and i₂ are the injections ℝ → ℝ2, and e₂ is the
embedding ℚ → ℝ.
In order to combine these two morphisms into a
single instance of ℝ2 we must sum them such
that onset and pitch become respective axes in
ℝ2
Compositions
The transformation f is then applied, and finally,
to return the coordinates to the module
morphisms on and pn, we apply the projections
p1 and p2 as follows where c represents ℝ
quantized to ℚ
 on = p1 ○ f ○ ((i1 ○ o) + (i2 ○ e2 ○ p)): A → ℝ
 pn = c ○ p2 ○ f ○ ((i1 ○ o) + (i2 ○ e2 ○ p)): A → ℚ
Tracing the modules on which these
compositions take place we have
 onset: ℤ11 → ℝ11 → ℝ → ℝ2 → ℝ2 → ℝ
 pitch: ℤ11 → ℚ11 → ℚ → ℝ → ℝ2 → ℝ2 → ℝ → ℚ
Morphisms to Gestures
The Wallpaper rubette is an example of a lowlevel process
 We are working in a very mathematical context
For the composer, this will not always be
appropriate, as mathematics may be a means
rather than an end
For this reason Gesture Theory is being
developed by Dr. Guerino Mazzola and Florian
Thalmann as implemented in the BigBang
rubette
 Gestures as curves in topological space
Future Applications
The Rubato Framework gives the composer an
alternate view of composition, working from a
functorial perspective
Also the musician can gain insight into a branch
of mathematics using intuition as a guide
opening up exciting educational avenues
The highly characterizable nature of the
category theoretic framework opens up the
opportunity for any system to modeled
effectively
Bibliography
– Milmeister, Gérard. The Rubato Composer Music Software: Component-Based
Implementation of a Functorial Concpet Architecture Zürich: 2006
– Thalmann, Florian and Mazzola, Guerino. The BigBang Rubette: Gestural
Music Composition with Rubato Composer
– Thalmann, Florian. Musical Composition with Grid Diagrams of
Transformations Bern: 2007
– “Pro Tools.” http://www.digidesign.com/
– “Logic.” http://www.apple.com/logicstudio/
– “Cubase.” http://www.steinberg.net