Transcript Savant slides

Automated music improvisation
Tony Wu – Jay Ni – Deri Kusuma
Special Thanks to: Prof. Plummer and Cristina Pop
and our testers: Tim Lambert, Adriana Miu, Jon Lau, and You Li
1
Architecture
2
Algorithms
3
Music Representation
Pitch
Note
- Screenshot of Cakewalk Music Creator 3
Time
4
Note Representation
Velocity = struck volume
Volume (db)
Pitch
Time
Struck time
Release time
5
Music Timing
=120
Tempo (beats per minute)
1 tick ≈ 125 ms
@ 4 ticks per beat
1 beat ≈ 0.5 sec
@ 120 beats per minute
1 measure
6
Architecture
MIDI input file
Input manager
Input buffer
Logic
Input thread
MIDI controller
Decision Engines
Analyzers
Improvisers
Logic thread
Libraries and
intermediate data
structures
Output buffer
Output thread
GUI thread
Output manager
GUI
Synthesizer
Audible sound
MIDI output file
Visualization
7
Real-time Music Storage
Circular chunklist
Press note
O(1)
Release note
O(1)
Range query
O(A + n)
Time overhead
O(n) per tick
Memory usage
O(NL)
A: number of answers
n: avg. number of notes per tick
N: total number of notes stored
L: avg. length of notes in tick
8
Chord analysis – basic chord types
C: major minor
dim aug
dom half dim
sus4
These are like basic colors in painting
Other chords are simply enrichment /
inversion / transposition of these basic feels
Enrichment
Inversion
Transposition
(of C major)
9
Chord analysis – note domains
These are all C major chords:
C: I
Affinity to A, B
Distance from F, Ab
d: VIIb
Affinity to F, Ab
G: IV
Affinity to F#
Important for generation!
10
Chord Analysis – Note Desirability
Higher pitch has less
chord determining
capability
F and F# only
sound good with
high octaves
D,A,B sound
good if not too
low
C major:
Only C,E,G will
sound good
down here
11
Chord Analysis – Capturing Richness
C, Db, D, E, F#, A, Bb
Absent
Higher notes
Ab
Eb
Strong notes
B
F
G dominant 7 b9 b13
G
Bass
Storing the note distribution of input captures richness
beyond basic types (e.g. major, minor)
12
Chord Analysis – Capturing Ambiguity
We compute a score for each possible
chord interpretation, and then pick the
highest score to determine chord.
C
Db
D
Eb
E
F
F#
G
Ab
A
Bb
B
M
.93
…
…
…
…
.50
…
…
…
…
…
…
m
.47
…
…
…
…
…
…
…
…
.70
…
…
dim
-.44
…
…
…
…
…
…
…
…
.34
…
…
aug
-.76
…
…
…
…
…
…
…
…
…
…
…
dom
.23
…
…
…
…
…
…
…
…
…
…
…
halfdim -.45
…
…
…
…
…
…
…
…
.48
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
13
Decoupling Motif and Chord
The same motif
+
C major
=
+
G7
=
+ Bb minor 7 9 / F =
14
Offline Motif Fitting
One-to-one pitch mapping algorithm
with dynamic programming
C
B
Bb
A
Ab
G
F#
F
E
Eb
D
Db
C
B
Bb
A
Ab
G
F#
F
E
Eb
D
Db
C
Maximize objective
function:
REWARD
chordal fitness as a
function of motif and
destination chord
PENALTY
absolute displacement ,
relative displacement
TIME BUDGET
60-120 ms per motif
15
Real-time Motif Fitting
Maximize objective
function per assignment,
given assistance level
Pitch
input
REWARD
chordal fitness as a function of
destination chord,
consistency to assignment
history
mapped
PENALTY
absolute displacement,
collision with assignment history
for neighboring notes
The present
Time
TIME BUDGET
< 1 ms per note
16
Determining Key of Piece
HOW?
The current key is determined by the discrete convolution of
the input notes with each major / minor filter.
Performing Key Changes
•
•
Abrupt: Using dominant chord of new tonic
Smooth: Leverage current chord progression
C: I – V – vi – iii – F:I – V – vi - …
(C:IV – I – ii - …)
17
Pleasant Chord Transitions
Chord State Machine
Chord type heuristic
I
M, M7, M9
II
m7
III
m, D7
IV
M, M7, m
V
D7
VI
m, m7
VII
dim, dim7, halfdim7
(From music theory)
18
Chord Phrasing
Forcing a phrase to end in a chord (e.g. V7) produces
clearer phrase boundaries.
C: I – iii – vi – IV – ii – V7 – I – V7
If this isn’t possible…
C: I – iii – vi – IV – ii – I – iii – V7
Take 2 steps back!
C: I – iii – vi – IV – ii –
–
– V7
19
Savant Modes
Mode
Description
Co-op
Duet with human on harmony, human on melody. Melody
is fitted to match the harmonic progression.
AI Co-op
Duet with AI on harmony, human on melody. Melody is
fitted to match the harmonic progression.
Remix
Analyzes motif and chords of input, and generates a
remixed version with motif and chord substitutes. User can
play along and choose to influence direction of generation,
or to have input fitted to chord progression.
Call & Response
User plays a musical phrase, and computer will respond
with an equal length phrase.
Empty
No logic, just like a real piano!
20