Transcript Quantifying_Generalization_from_Trial-by

Quantifying Generalization from
Trial-by-Trial Behavior in
Reaching Movement
Dan Liu
Natural Computation Group
Cognitive Science Department, UCSD
March, 2004
REFERENCE


Shadmehr R, Mussa-Ivaldi FA (1994). Adaptive
representation of dynamics during learning of a motor
task, Journal of Neuroscience 14:3208-3224.
Donchin O, Francis JT, Shadmehr R (2003).
Quantifying generalization from trial-by-trial behavior of
adaptive systems that learn with basis functions: theory
and experiments in human motor control. Journal of
N e u r o s c i e n c e , 2 3 : 9 0 3 2 - 9 0 4 5 .
OUTLINE
Earlier Experiments on Adaptation
during Motor Learning
 A Model for Adaptation during Motor
Learning
 Evaluation of the Model
 Conclusion and Discussion

Earlier Experiments on Adaptation during
Motor Learning

Basic Idea: trial-by-trial approach
How a subject adapted to the changed dynamics
of a reaching task.
Materials and Methods
 Experiment on Adaptation to the Force
Field

Experimental Setup
Fig1. Experimental setup. A and B, experimental setup and the coordinate system for simulations of
human arm and robot arm dynamics. C and D, force field examined. C, force filed is a function of
velocity of hand (invariant in end-point coordinates).
Earlier Experiments on Adaptation during
Motor Learning
Basic Idea
 Materials and Methods
 Experiment on Adaptation to the Force
Field

 Adaptation
 Aftereffects
Process
Adaptation Process
Trajectory
Velocity
Fig 2. Trajectory and
velocity during
adaptation in force
field 1. Trajectory
rows show the average
+- SD of hand
trajectories during the
first to final 250 targets.
Velocity rows show the
tangential hand
velocities. Column A is
in non force field,
column B is upon initial
exposure to the force
field, and column C is
after 1000 reaching
movements in the
f o r c e f i e l d .
Conclusion: The recovery of the original unperturbed response
constitutes a clear example of an adaptive behavior.
Aftereffects
force field: Set 1
Set 2
non force field:
A
Set 3
B
Set 4
C
D
non force field: Set 1
force field:
Set 4
F
V.S.
Fig3. Aftereffects of adaptation to the force field. A-D, averages +-SD of the hand trajectories while
moving in a null field during the training period for the first to final 250 targets. F, averages +-SD of the
hand trajectories upon initial exposure to the force field.
Conclusion:
Subject developed an internal model during adaptation, which is a
mapping from the arm’s motion into force.
Subject generalized the internal modal broadly outside the workspace
explored during adaptation.
Earlier Experiments on Adaptation during
Motor Learning

Conclusion:
 During
adaptation to a force field that significantly
changes the dynamics of a reaching movement, the
CNS forms an internal model of the added dynamics.
 This internal model has the power to generalize well
beyond the training region.

Question:
How to model this generalization process?
OUTLINE
Earlier Experiments on Adaptation
during Motor Learning
 A Model for Adaptation during Motor
Learning
 Evaluation of the Model
 Conclusion and Discussion

A Model for Adaptation during Motor Learning

Basic Idea
Model: a mapping from the arm’s
position and velocity space into force.
 Generalization: a reflection of how this map
is encoded, often globally and linearly as a
function of position of the limb.
 Internal

Model
 Internal Model
 Generalization
Model
Internal Model
 Internal model uses basis elements to encode limb velocity.
(1)
-
: Basis element, with a receptive field covering a specific piece of the
space (one of the eight directions), used to encode limb velocity .
-
: Preferred force vector associated with
: Predicted force acting on the hand.
.
 The internal model is adapted by changing the preferred force
vector associated with each basis.
(2)
-
: Actual force acting on the hand.
: Learning rate.
Generalization Model
 Generalization model uses the previous error to update the
predicted hand position, based on generalization function.
(3)
-
: Generalization function, describes how errors experienced in direction
k affect the internal model for any other direction l.
: Captures the linearly relationship between position and force.
: Predicted hand position during movement n.
: Error between actual position and predicted position.
OUTLINE
Earlier Experiments on Adaptation
during Motor Learning
 A Model for Adaptation during Motor
Learning
 Evaluation of the Model
 Conclusion and Discussion

Evaluation of the Adaptation Model

Basic Idea: trail-by-trail approach
 Test:



field trials (with applied force by the robot)
catch trials (without applied forces occasionally)
Experimental Setup
Measure of Fitness
Experiment Results
Experimental Setup
Fig 4. Experiment setup. A. Experimental setup and the coordinate system for simulations of
human arm and robot arm dynamics. Fig. B-E. Force fields examined.
Measure of Fitness:
desired output assumption
Experiment Results (1):
Performance of the Generalization Model (dynamic system)
Set 1
Set 2
Set 3
Fig 5. Description of the task.
Fig 6. Averaged performance of subjects in a clockwise curl field in
three successively sets. Gray is subject data, and black is model fit.
Circles indicate catch trials.
values for all movements within a set are
0.77, 0.80, and 0.77 respectively for sets 1-3.
Conclusion: The high degree of fit shows the good performance of the
model in describing the trial-by-trial adaptation.
Experiment Results (2):
Performance of the Internal Model (adaptive controller)
(3)
Shape of matrix D is compatible
but too stiff.
Shape of generalization function is
not compatible with bimodal shape.
Fig 7. Parameters of the fit using Equation 3. A and
B are on data from subjects. C and D are on data from
simulations with different
. A and C represent matrix
D. B and D represent generalization function.
Experiment Results (3):
Bimodal of the generalization function
Fig 8. Generalization function. A is the shape of a primitive (0.21,0.21) implied by the
generalization functions found in the subjects data. B is a model-free assessment of the
generalization function in a clockwise curl field as explained in C.
Conclusion: The bases of the internal model have a bimodal shape.
Experiment Results (4):
Robustness of the Model
Fig 9. Errors and the fit of the model to a set of 192 movements in variations of the standard task (each row).
In A, the out-and-back structure is removed. In B-D, the force field is changed. Column (1) represents the
average error across all directions. Column (2) represents the error when the movement is 180 o. Column (3) shows
the shape of matrix D. Column (4) shows the shape of generalization function.
Conclusion: The model works well in different force fields and target sets.
Evaluation of the Adaptation Model




Basic Idea
Experimental Setup
Measure of Fitness
Experiment Results
 Internal
Model approximates force as a function
of limb states via a fixed set of basis elements.
 Basis elements broadly encodes limb position,
but have a bimodal representation of limb
velocity.
OUTLINE
Earlier Experiments on Adaptation
during Motor Learning
 A Model for Adaptation during Motor
Learning
 Evaluation of the Model
 Conclusion and Discussion

Conclusion

The theory relates encoding to generalization by
using trial-by-trial changes during adaptation.

Internal model approximates force as a function of
limb states via a fixed set of basis elements.

Basis elements broadly encodes limb position, but
have a bimodal representation of limb velocity.
Discussion

The internal model is evaluated in a feed forward
manner and depends on a desired trajectory.

Learning rate, stiffness and viscosity of the arm
are all fixed.

Basis elements are fixed, while only the force
vectors associated with each basis are adapted.

Reasonable but not optimal: introduction of
nonlinearity could be a key factor in further
development.
Thank You !