Transcript Validation - Mechanical Engineering

MEGN 536 – Computational Biomechanics
Prof. Anthony Petrella
Musculoskeletal Modeling &
The Importance of Validation
Musculoskeletal Modeling
 You’ve worked with a simple arm curl model in
OpenSim
 If you have not… you should try it (ws9, 10/4)
 OpenSim is freely available on the SimTk project
website for your use (if you wish)
 Consider the example:
Dynamic Walking Starter…
simtk.org/frs/download_confirm.php/file/4598/DynamicWalkingStarter.zip?group_id=91
MSM Applications
 Orthopaedic Co #1 – humeral fracture fixation
 Orthopaedic Co #2 – compare different knee designs
for kinematic patterns, muscle forces, and load
transfer
 Mines projects in Center for Biomechanics and
Rehabilitation Research
 MSM in research & industry…
 A broad topic of active research/evolution
 Could teach a whole class on it (MEGN 535 in Spring)
 Validation a critical issue for future of MSM
Introduction to MSM Validation
 MSM taking an increasingly central role in many
ergonomics, design, clinical applications




NASA digital astronaut (http://spaceflightsystems.grc.nasa.gov/SOPO/ICHO/HRP/DA/)
Automotive ergonomics (Rasmussen et al., J Biomech, 2010)
Orthopaedic design
Clinical guidance (Bohme et al., 2012)
 Growing interest in personalized medicine
 MSM Consortium under IMAG
 Subject-specific simulation in literature
 Orthopaedic companies offer various
personalized joint replacement technologies
Musculoskeletal modeling…
…How good is good enough?
Musculoskeletal modeling…
…How good is good enough?
It Depends.
How good is good enough? It depends.
 Can we believe model predictions, and can we use
them to drive decisions that affect health?
“software can hurt people”
model driving, wrong
choices create harm
Model influence
on decisions
other factors driving,
no/low risk of harm
Consequences of decisions
Adapted from (Mulugeta, 2012)
MSM Consortium Mtg
How to know model is good?
 Verification & validation (V&V)
 Uncertainty quantification (UQ)
 Quality/version control (of each model) important
 Formally developed software relatively young




SIMM – early 1990’s
AnyBody – early 2000’s
LifeModeler – early 2000’s
OpenSim – later 2000’s
 Still learning best strategies and
methods for V&V et al.
Terminology
 Verification – testing code to ensure governing equations are
implemented correctly and solved accurately
 Validation - the process of determining the degree to which a
model is an accurate representation of the real world based
on the intended uses of the model (AIAA, 1998)




Direct – gold standard
Indirect – use of surrogate metrics
Trend – parametric variation, confirm validity of “what if” scenarios
Validation hierarchy – test constituent parts of complex model
 Uncertainty Quantification – for relevant outcome metrics, and
which inputs important?
 Version Control – of individual models
MSM Validation Somewhat Unique
Model
Development
Improvements…
Community
Validation, UQ,
Version Tracking
Company V&V
(limited)
Model
Repository
(AMMR)
AMMR = AnyBody
Managed
Model Repository
Verification
 Typically done by developer
 MSM more challenging (vs. FE, CFD)
 Analytical solutions are rare, need experiments
 Experiments laborious, difficult, introduce error
 Line between V&V blurs
 All MSM software vendors qualify code
 Modules/algorithms, system tests for interactions, models
 When
model influence + consequences
Verification manual  greater confidence?
Direct Validation: In-vivo joint forces
(orthoload.com)
(Thielen et al., 2009)
Direct (Pedal Forces), Indirect (EMG)
Model
(de Jong and Meijer, 2006)
(In)direct and Trend Validation: In-vivo Pressure
Calibration
(Rasmussen et al., 2009)
(Wilke et al., 1999)
Direct and Trend Validation: Force
 Reaction forces at L1-L2
 Enhanced: interseg muscles, ligaments
(orthoload.com)
(Han et al., 2012)
Direct and Trend Validation: Seat Shear Force
(Olesen, 2009)
Direct Validation: 45° Abduction, GH Force
Experiment
Peak GH force = 863 N
(Bergmann, 2009)
Model
Peak GH force = 850 N
(Nolte et al., 2008;
Dubowsky et al., 2008)
(orthoload.com)
Direct Validation: 45° Abduction, GH Force
WITH 2kg weight
w/o 2kg
(orthoload.com)
GH Lessons Learned
(Kunze, 2012)
Closing the Loop: GH Improvements
Direct Validation: Knee Forces
 Grand Challenge, In Vivo Knee Loads
(Andersen et al., 2011)
Subject-Specific Scaling: Model Only
 Hip center identified with…
 Regression equation using pelvic landmarks
 CT scan register hip center to landmarks / markers
 Gait, stair descent  no difference in force
 Sit to stand  CT significantly lower peak
Regression equation
CT scan of pelvis
( Andersen et al., 2012)
Subject-Specific Scaling – TLEMsafe
New complete and consistent musculoskeletal model including:

Muscle LOA’s, moment arms, and joint geometry based on cadaver

Bone surfaces and muscle volumes segmented from CT and MRI

Scalable to subject-specific models using MRI data, e.g., bone
morphing and muscle volumes
(Carbone et al., ISB2013; www.tlemsafe.eu)
Validation Hierarchy
 Complexity of high level model
makes validation challenging
 Constituent parts can be
(must be?) validated to add
confidence
 Input data (mocap, GRF, EMG) a sub
system requiring validation
 Benchmarks / standards can aid in
validation of lower level system
features
 Muscle benchmark data
(Millard et al., 2012)
(Lund et al., 2012)
Direct Sub-model Validation: Foot Contact
Uncertainty Quantification
 Need to understand sensitivity of outcomes to inputs
 Need to understand uncertainty in the inputs
 Can determine uncertainty in the outcomes
Uncertainty Quantification
 Need to understand sensitivity of outcomes to inputs
 Need to understand uncertainty in the inputs
 Can determine uncertainty in the outcomes
 Probabilistic analysis
Validation Comments
 All validation examples experimental data vs. single model
 Different versions, options, anatomical data sets
 Highlight credibility of model repository and software design
 BUT… new models require new validation, UQ, version control
 Auto-validation of standard (repository) models with
validation report should be a goal
 Easier to select best model for app
 Better insight to details… moment
arms, muscle parameters, etc.
 Greater number of input cases
 “Validation engine” could facilitate
community contribution
Gastroc moment arm vs. exp’s
Conclusions
How do I validate
my model…?
 MSM maturing, the software works
 Many strong validation studies, but…
 Relevant for single model only
 New models require new validation
 Some standards / benchmarks may be useful
 Verification: standards?, published verification manual
 Validation: benchmarks, auto-validation for repository models, and
“validation engine” for community contributions
 UQ: probabilistic methods common, standards?
 Version: end users probably not used to this
 Subject-specific: generally, detail = different, better?