Transcript 110_161

Low-Level
Control
MURI
Fabrication
High-Level
Control
How is Compliance used in
Locomotion?
Berkeley & Stanford:
Measurements of Cockroach
Locomotion
What Compliance Strategies in
Human-level Tasks?
Harvard & Johns Hopkins: Compliance
Learning and Strategies for
Unstructured Environments
Guiding questions
Low-Level
Control
What strategies are used in insect
locomotion and what are their
implications?
MURI
High-Level
Control
Fabrication
Insect locomotion studies (Berkeley Bio)
New measurement capabilities (Stanford)
What motor control adaptation
strategies do people use and how
can they be applied to robots?
Compliance Learning and Strategies for
Unstructured Environments (Harvard &
Johns Hopkins)
Implications for biomimetic robots
(Harvard, Stanford)
MURI
High-Level
Control
Impedance Adaptation in
Unstructured Environments
Yoky Matsuoka and Rob Howe
Harvard University
MURI
High-Level
Control
Manipulation with Impedance
• An example of manipulation with impedance
• Why is biology superior to current robots in
an unstructured environment?
MURI
High-Level
Control
Identify Impedance Learning
Strategy in Human
• Two big questions:
– What is the initial strategy used to cope with
unknown/unstructured environments?
– After learning, what does the biology pick as
the good solution for impedance for a given
environment?
How can these solutions help robot’s control
strategy?
MURI
High-Level
Control
Comparison Between Analytical
and Biological Solutions
• We can mathematically derive optimal impedance
for a linear world.
– Biological system converges to the analytical solution.
--- great!
– Biological system converges to a different solution.
--- what and why: put the biological solution back in
the equations and reverse engineer.
• What about a nonlinear varying world where it is
difficult to derive the optimal impedance?
– What does the biological system do? Can it be
modeled as a solution for robots?
MURI
Example: Linear World
--- Catching a Ball
High-Level
Control
• Goal: Find the “best”
impedance.
mball
ball
– For this case, find best
Khand.
kball
mball
kball
hand
mhand
xball • Uncertainty in the
world
mhand
xhand
khand
khand
– mball, kball, xball(0), and
khand
Example: Linear World
--- Catching a Ball
MURI
High-Level
Control
• Cases:
mball
xball
kball
1. Hand stiffnes (khand) is too high
• x hand < 0
bounces up
2. Hand stiffness (khand) is too low
•  xhand > Threshold
mhand
xhand
khand
bottoms out
3. Hand stiffness (khand) is just right
• xball  xhand until switch is pressed
2
khand 0
3
1
infinite
MURI
Example: Linear World
--- Catching a Ball
High-Level
Control
• Solve for xhand(t) and xball(t)
– initial condition
• x ball(0) > 0
mball
xball
kball
• xball(0) = 0
• x hand(0)= 0
• xhand(0) = 0
mhand
xhand
khand
mball xball  kball ( xhand  xball )  0
mhand xhand  kball ( xhand  xball )  khand xhand  0
MURI
High-Level
Control
Analytical Linear World
to Biological Motor Control
• The example relates task performance to
limb impedance and optimal solution.
– Other examples: leg impedance, etc.
• Now measure human strategy….
– “System identification”
– Need a new technique
MURI
High-Level
Control
Existing System
Identification Techniques
F (t )  mx(t )  bx (t )  k ( x  xo )
• Time invariant systems --- easy
– assume constant m, b, and k over time.
– apply external impulse perturbation force.
– repeat the same condition and average.
MURI
High-Level
Control
Existing System
Identification Techniques
F (t )  mx(t )  bx (t )  k ( x  xo )
• Time varying systems
• Cannot apply impulses close to each other.
• Need multiple impulses to solve for multiple unknowns.
– PRBS (Lacquaniti, et al. 1993)
MURI
High-Level
Control
New System Identification
Technique to Observe Learning
Setup
Monitor
Robot
Processor
Force Sensor
Handle
Human
Subject
Accelerometer
Data
Acquisition
System
MURI
High-Level
Control
New System Identification
Technique to Observe Learning
• Very short duration
• Very clean data from force and acc. sensors
F
k*x
b*v
m=F/a
b= (F-ma)/v
k= (F-ma-bv)/x
m*a
MURI
High-Level
Control
Testing the New Technique
MURI
High-Level
Control
Video of the task here
MURI
Testing the New Technique
High-Level
Control
• Phantom robot is used as the
perturbation/measurement
tool.
• Task: balance the moving ball
on paddle.
– ball moves at constant
speed
– dies when the ball falls off
the paddle
– perturbation applied every
second
MURI
Impedance Change with Learning
High-Level
Control
m change over time
k change over time
b change over time
MURI
High-Level
Control
Contact Interaction Task -Impedance Dependent Task
• Observe the impedance change
within one catch
• Observe the impedance change
between catches
k
b
** under development --- pilot studies underway
MURI
Current Understanding of the Structure of the Biological Controller
High-Level
Control
q
q
q
Forward Dynamics
Model
120 msec
ui
Long-loop
Gain
uf
q
Inverse Dynamics
Model
ui
u
60 msec
q
Inertial
Dynamics
q
us
30 msec
From Shadmehr
MURI
High-Level
Control
Impedance Adaptation
Conclusions and Future Work
• Developed a new impedance identification
technique
– Based on virtual environment --- extremely
versatile
– Confirmed ability to measure instantaneous
impedance, characterized changes with
learning.
MURI
High-Level
Control
Impedance Adaptation
Conclusions and Future Work
• Current experiments
– Determine human interaction strategies
• initial impedance
• learning characteristics
• final impedance
• Next experiments
– Determine human interaction strategies for
nonlinear varying tasks
• e.g. plastic deformation (running in sand)
MURI
HighLevel
Control
Control of Locomotion
Local controller (single limb):
Control of the limb based on local information:
- position and velocity of the limb
Is the limb far enough to the back so that I can start the return stroke?
- forces acting on the limb
Is the supporting load small enough for me to lift the limb?
Task controller (all limbs):
Coordination with other limbs:
- position of the other legs
Across species, control of limb based on local information appears
similar (Cruse, TINS, 90). Coordination with other limbs appears highly
species dependent.
MURI
HighLevel
Control
Control of a Limb Based on Local Information
Step cycle: generation of power (stance) and return (swing) strokes.
Return phase:
Move the limb from posterior to anterior position along a desired velocity profile.
- Maintain proper impedance to remain stable in case of perturbations
After hitting an obstacle, the limb should converge back to the desired path
- Adapt impedance to allow for generation of desired behavior in the face
of a persistent environment
limb is moving in highly viscous fluid, it must adapt its impedance to the
characteristics of the environment.
Impedance control and adaptation in a position control task
Power phase:
Maintain contact, maintain height of load, move limb from front to rear.
Impedance control and adaptation in a contact/force control task
MURI
HighLevel
Control
Current Limb Local Control Models for Locomotion in Insects
Cruse et al (Neural Networks 1998): Stick insect model
- limb has little or no inertia
- no muscle like actuators
- controller output is velocity, feedback sensing via position and linear
feedback
- no ability to adapt
Essentially a kinematic model of a limb only, with little or no dynamics
This kind of model tells us little about how to design good controllers
MURI
HighLevel
Control
Designing a Single Limb Impedance Controller
General Goals:
1. To understand what impedance strategies a biomechanical controller
uses when it moves the limb in a position control task. Apply the results
to control of the return phase.
2. To understand impedance strategies of the biomechanical controller in
a force control task. Apply the results to the control of the stance phase.
Approach:
Study the human arm’s impedance adaptive control strategies in both
position and force control tasks. Test validity of the strategies on a
robotic system.
MURI
HighLevel
Control
Designing a Single Limb Impedance Controller
Task Division:
1. Impedance control at very short time intervals (<10 msec, preflexes)
Yoky Matsuoka and Rob Howe
2. Impedance control at intermediate and long time intervals (<300
msec)
Tie Wang and Reza Shadmehr
3. Test and implementation on a robotic system
Jay Dev Desai and Rob Howe
MURI
Quantifying Impedance Control Strategies of a
Biomechanical Controller
HighLevel
Control
Challenges:
The biomechanics of the human arm are dominated by multiple feedback
loops, with various time delays. Impedance measurements are done
through imposition of perturbations and measurement of responses.
• How do time delays affect measures of arm impedance?
• Humans learn internal models when they learn control. How does
a change in the internal model affect measures of arm impedance?
• Impedance measures require an estimation of where the system
would have been if it had not been perturbed. How well can we do
this with a non-stationary system like the human arm?
MURI
Current Understanding of the Structure of the Biological Controller
High-Level
Control
q
q
q
Forward Dynamics
Model
120 msec
ui
Long-loop
Gain
uf
q
Inverse Dynamics
Model
ui
u
60 msec
q
Inertial
Dynamics
q
us
30 msec
Through modulation of input u to the muscles, impedance of the system is
changed. The impedance depends on 3 feedback pathways:
1. Near zero-delay mechanical stiffness/viscosity of the muscles (Yoky).
2. Short delay sensory feedback through spinal structures.
3. Long delay sensory feedback through cortical structures (forward model).
Are muscle “preflexes” enough?
MURI
High-Level
Control
Intact control system
dt=10ms
1 cm
dt=10ms
High-level sensory
feedback loop disrupted
dt=30ms
MURI
High-Level
Control
Impedance of a biological arm: A definition
Inertial torques :  (q, q , q)  H (q )q  C (q, q )q
Torques produced by muscles : M (q, q , u )
External torques from environment :  (t )
 (q, q , q)  M (q, q , u )   (t )
when    0 (t ) and u  u 0 (t ), arm follows a particular trajectory q0 (t ).
 (q0 , q 0 , q0 )  M (q0 , q 0 , u 0 )   0 (t )
Stiffness : K 
Viscosity : B 
dM
dq
dM
dq
q0
q 0
MURI
HighLevel
Control
Estimating Impedance: Theory
Dynamics without a perturbation :  (q0 , q 0 , q0 )  M (q0 , q 0 , u 0 )   0
(1)
Impose a perturbing force :  , resulting in q0  q and u 0  u.
 (q0  q, q 0  q , q0  q)  M (q0  q, q 0  q , u 0  u )   0  
(2)
(2)  (1) :  (q0  q )   (q0 )  M (q0  q, q 0  q , u 0  u )  M (q0 , q 0 , u 0 )  
M (q0  q, q 0  q , u 0  u )  M (q0 , q 0 , u 0 ) 
 (q0  q )   (q0 ) 
dM
dq
q 
q0
dM
dq
dM
dq
q 
q0
q 
q 0
dM
dq
dM
du
q 
q 0
dM
du
u  
u0
u  ...
q0
MURI
HighLevel
Control
Estimating Impedance: Requirements
 (q0  q )   (q0 )  K (t )q  B(t )q 
We can measure :
dM
du
u  
u0
q, q , 
We must estimate :
1. Inertial dynamics of the arm :  (q, q , q)  H (q )q  C (q, q )q
2. Path that the system would follow if there were no perturbations : q0 (t )
3. The change in muscle neural input as a result of our perturbation : u (t )
MURI
1.0 Estimating Inertial Dynamics of the Arm (Theory)
High-Level
Control
Inertial torques : (q, q , q)  H (q)q  C (q, q )q
a  a  2a cos(q ) a  a cos(q
H (q)  
a
 a  a cos(q )
1
3
3
2
2
2
2
2
3
2
2
 a sin( q )q
C (q, q )  
 a sin( q )q
2
2
1
2
3
2
)


 a sin( q )(q  q )

0

2
2
1
2
 J F
T
Torques produced by muscles M (q, q , u )  0
   (q, q , q)
F : external force acting on the hand
OBJECTIVE : estimate parameters a1 , a2 , a3 .
MURI
1.1 Estimating Inertial Dynamics of the Arm (Methods)
HighLevel
Control
2
Elbow Torque (Nm)
Shoulder Torque (Nm)
6
4
2
0
-2
0
0.1
0.2
Time (s)
0.3
0.1
0.2
Time (s)
0.3
0.4
Elbow Angel (rad)
1.1
1.05
0
0.1
0.2
Time (s)
0.3
0
0.1
0.2
Time (s)
0.3
0.4
0
0.1
0.2
Time (s)
0.3
0.4
1.61
1.605
1.6
0.4
0.5
0.2
Elbow Velocity (rad/s)
Shoulder Angel (rad)
Shoulder Velocity (rad/s)
0
1.615
1.15
0
-0.5
0
-1
0.4
1.2
1
1
0
0.1
0.2
Time (s)
0.3
0.4
0.1
0
-0.1
-0.2
Give a force pulse, use data for up to 14 ms after the pulse to
estimate inertial parameters.
MURI
Shoulder Torque (Nm)
6
Experiment
Model
Error
4
2
0
-2
-4
0
10
20
30
40
50
60
70
50
60
70
Index of Array
4
Elbow Torque (Nm)
HighLevel
Control
1.2 Estimating Inertial Dynamics of the Arm (Results)
3
2
1
0
-1
-2
-3
-4
0
10
20
[a
1
a
2
30
40
a ]  [ 0.3025 0.1727
3
0.1293]
MURI
HighLevel
Control
2.0 Predicting the Un-perturbed Trajectory (Theory)
Use Principal Components Analysis (PCA) to estimate bases for approximation :
Unperturbed_Mov
trial time
Perturbed_Mov
rial time
 [S
1
trial( 1: prb _ time1 )
 [P
1
trial( 1: prb _ time1 )
S
2
trial( prb _ time:mov _ time )
P
2
trial( prb _ time:mov _ time )
]
]
Legend
S , P : raw data matrices
S pca , Ppca : PCA from raw data matrices S P
S mean , Pmean : mean of raw data matrices S P
trial : number of movements
time : duration of each movement
prb _ time : time point when movement is pertubed
MURI
HighLevel
Control
2.1 Trajectory Prediction (Methods)
Use S 1 , S 2 , P1 to predict P 2 (2 methods)
Method 1 : use all unperturbed movements as bases for prediction
K1  S 1  P1
K 2  S 2  P 2 Asume that K1  K 2
The idea is that use all the raw data of movements as the base function
to predict the unknown movement.
1
1
1
1
2
2
2
Method 2 : P 2  ( P1  Pmean
)( S 1pca S 1pca )( S 1  S mean
) 1 ( S 2  S mean
)( S pca
S pca
)
2
 S mean
K1  S 1pca  S 1
2
K 2  S pca
 S2
K1  C  K 2
1
K1'  Ppca
 P1
2
K 2'  Ppca
 P2
K1'  C  K 2'
The idea is that instead of using the raw data of movements, we
use the principal components of the raw data to predict .
MURI
0.7
Measured Trajectory
Predicted Trajectory
Shoulder Velocity(m/s)
0.6
0.5
0.4
0.3
0.2
0.1
0
-0.1
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
0.05
0.1
0.15
0.2
0.25
0.3
Time (s)
0.35
0.4
0.45
0.5
0.55
0.5
0
Elbow Velocity(m/s)
HighLevel
Control
2.2 Trajectory Prediction (velocity)
-0.5
-1
-1.5
-2
Prediction error (%)
0.2 s  0.3s 0.3s  0.4 s 0.4 s  0.5s 0.5s  0.6 s
1.9125
4.2855
7.0604
17.8519
1.9896
4.3649
6.8963
17.4751
MURI
2.3 Trajectory Prediction (force)
Force (N)
HighLevel
Control
1
1
1
1
0
0
0
0
-1
-1
-1
-1
-2
-2
0
20
40
60
-2
0
20
40
60
-2
0
20
40
60
1
1
1
1
0
0
0
0
-1
-1
-1
-1
-2
-2
0
20
40
60
-2
0
20
40
60
0
20
40
60
0
20
40
60
-2
0
20
40
60
Time (10 msec)
Conclusion: Position, velocity, and force can be reasonably well
predicted for up to 300 msec after the last sampled data point.
MURI
3.0 Estimating the Effect of u(t) on Arm Impedance
q
HighLevel
Control
q
q
Forward Dynamics
Model
120 msec
ui
Long-loop
Gain
uf
q
Inverse Dynamics
Model
ui
 (q0  q )   (q0 )  K (t )q  B(t )q 
dM
du
u
60 msec
Inertial
Dynamics
us
u  
u0
q
30 msec
u is the change in the input to the muscles as a result of our perturbation.
While u cannot be measured directly, we know that it depends on a number of
time-delayed, possibly adaptive error feedback systems.
• Time-delayed error feedback from the spinal reflexes
• Time-delayed error feedback from the forward model based cortical pathways
• Input from inverse model based “open-loop” controller
q
MURI
HighLevel
Control
3.1 Time-delayed effect of u(t) on Arm Impedance
In general, a time delay d in a feedback loop reduces apparent viscosity
and adds apparent mass to a system.
Example: mx  bx  kx(t  d )  0
Using Taylor series expansion on the delay :
dx
d 2x
x(t  )  x(t )  (d )  1 / 2 2 (d ) 2  
dt
dt
The resulting system has apparent impedance of :
(m  1 / 2kd ) x  (b  kd ) x  kx  0
(d  0)
MURI
HighLevel
Control
3.2 Estimating u(t) in Terms of Measurable Quantities
1. Effect of Spinal Reflexes
u  ui  u f  u s
u input to the muscles
ui inverse model based, adaptive input to the muscles
u f forward model based, adaptive input to the muscles
u s spinal based error feedback to the muscles
Assume : u s  k s (q (t  d s )  qd )  bs (q (t  d s )  q d )
Rewrite the delay in terms of a Taylor series and after a perturbation,
it follows that :
u s  k s q  (bs  k s d s )q  (1 / 2k s d s2  bs d s )q
MURI
3.2 Estimating u(t) in Terms of Measurable Quantities
2. Effect of the Inverse model
HighLevel
Control
q
q
q
Forward Dynamics
Model
120 msec
ui
Long-loop
Gain
uf
q
Inverse Dynamics
Model
ui
u
60 msec
q
Inertial
Dynamics
us
30 msec
dui
 0, therefore after a perturbation ui  0
dq
q
MURI
HighLevel
Control
3.2 Estimating u(t) in Terms of Measurable Quantities
3. Effect of the Forward Model

Computation of the forward model : qˆ  H (q ) 1 M (q, q , ui )  C (q, q )  Eˆ (q, q )
qˆ (t )   qˆ dt dt

t 
Linearly approximate the intergration of the forward model :
qˆ (t )  q (t  d )  2q (t  d )d  qˆ(t  d )d 2
Replace delays q (t  d ) with their Taylor series expansions and assume :
u f  k f (qˆ  qd )  b f (qˆ  qd )
Case 1 : Assume Eˆ  0, qˆ0 is output of the forward model.
Rewrite u f in terms of measurable quantities q :


u f  k f q  qd  3 / 2qd 2  d 2 qˆ0 (t  d )  qd  b f q  qd  dqˆ0 (t  d )  qd 
MURI
HighLevel
Control
3.2 Estimating u(t) in Terms of Measurable Quantities
4. Effect of Adaptation of the Forward Model
Case 2 : Assume a force field environment is present, and Eˆ  Wq.
Rewrite u f in terms of measurable quantities q :

q  qd  dqˆ (t  d )  q

u f  k f q  qd  3 / 2qd 2  d 2 qˆ0 (t  d )  dqd  d 2 H 1Wq  d 3 H 1Wq
bf
0
d

 dH 1Wq  d 2 H 1Wq
In response to a perturbation  , the change in the adapted response u f with
respect to response before adaptation will be :




u f   k f d 2 H 1W  b f dH 1W q  k f d 3 H 1W  b f d 2 H 1W q
Predictable changes in impedance should occur as a function of the kind of model
that the system learns as it practices movements in an unstructured environment.
If learning is via a forward model, the apparent viscosity must decrease as
compared to values obtained before the controller had adapted.
MURI
Measuring Impedance of the Moving System
HighLevel
Control
0.15
Y (m)
0.1
0.05
0
Perturb the movement
in different directions
-0.05
-0.1
-0.05
0
X (m)
0.05
0.1
0.15
MURI
Impedance Early into the Movement
HighLevel
Control
12
10
8
Perturb
Interaction force
6
Force (N)
4
Impedance Controlled force
2
0
positionx50 (m)
velocityx25 (m/sec)
-2
acceleration (m/s2 )
-4
-6
-8
Inertial dynamics
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Time into the movement (s)
0.4
0.45
0.5
MURI
Impedance in the Middle of the Movement
HighLevel
Control
12
10
Interaction force
8
Perturb
6
Force (N)
4
Impedance controller’s force
2
0
Positionx50 (m)
Velocity x 25 (m/s)
-2
-4
Acceleration (m/s 2)
-6
Dynamic force
-8
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Time into the movement (s)
0.4
0.45
0.5
MURI
Impedance Near End of Movement
HighLevel
Control
12
10
Interaction force
8
6
Impedance Controlled force
Force (N)
4
Perturb
2
0
velocityx25 (m/sec)
-2
positionx50 (m)
acceleration (m/s2)
-4
Inertial dynamics
-6
-8
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Time into the movement (s)
0.4
0.45
0.5
MURI
Stiffness of the System: Results
HighLevel
Control
Joint Stiffness (N.m/rad)
0
-200
100 msec
System Characteristics:
- Initially a very stiff system (likely due to intrinsic muscle stiffness).
- The system yields as the perturbation persists, with stiffness dropping
as the short- and long-loop reflex systems take control.
MURI
HighLevel
Control
Immediate Plans
Position control task:
- Impose a force field, measure impedance changes as the system adapts
- Do we see evidence for formation of a forward model as indicated by
reductions in the system’s viscosity?
Force control task:
- Measure impedance changes in a task that requires maintaining contact
(pushing in order to roll a virtual conveyer belt).
- Push too hard, the belt breaks. Push too soft, arm slips on the belt.
- Measure impedance changes as the
virtual belt’s dynamics changes, requiring
adaptation in the controller.
- Virtual belt is produced by the robot
manipulandum.