Transcript Maximum entropy methods (MAXENT)

An Information-theoretic Tool for
Property Prediction Of Random
Microstructures
Sethuraman Sankaran and Nicholas Zabaras
Materials Process Design and Control Laboratory
Sibley School of Mechanical and Aerospace Engineering
188 Frank H. T. Rhodes Hall
Cornell University
Ithaca, NY 14853-3801
Email: [email protected], [email protected]
URL: http://mpdc.mae.cornell.edu/
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
RESEARCH SPONSORS
U.S. AIR FORCE PARTNERS
Materials Process Design Branch, AFRL
Computational Mathematics Program, AFOSR
ARMY RESEARCH OFFICE
Mechanical Behavior of Materials Program
NATIONAL SCIENCE FOUNDATION (NSF)
Design and Integration Engineering Program
CORNELL THEORY CENTER
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
An overview

Mathematical representation of random microstructures

Extraction of higher order features from limited microstructural
information : the MAXENT approach

MAXENT optimization schemes

Evaluation of homogenized elastic properties from microstructures

Effect of varying information content on property statistics

Numerical examples

Summary and future work
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Idea Behind Information Theoretic Approach
Basic Questions:
1. Microstructures are realizations of a random field. Is there a principle by which the
underlying pdf itself can be obtained.
2. If so, how can the known information about microstructure be incorporated in the solution.
3. How do we obtain actual statistics of properties of the microstructure characterized at macro
scale.
Information
Theory
Statistical
Mechanics
CORNELL
U N I V E R S I T Y
Rigorously quantifying
and modeling
uncertainty, linking
scales using criterion
derived from
information theory, and
use information
theoretic tools to predict
parameters in the face
of incomplete
Information etc
Linkage?
Information Theory
Materials Process Design and Control Laboratory
Representation of random microstructures
Indicator functions used to represent microstructure at different regions
in the physical domain
 Indicator functions take values over a binary alphabet
 Statistical features of microstructure are mathematically tractable in
terms of expected values over indicator functions

Two-phase material
if
if
n-phase material
if
Define Ii as the set comprising
(Ii(x1), Ii(x2), … Ii(xn)). Ii
represents a random field of
indicator functions over the
domain. Microstructures are
hierarchically characterized over
a set of random variables of this
field
if
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Defining correlation vectors using indicator functions
Two-point probability functions
Lineal Path Functions
n-point probability functions
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Microstructure Reconstruction Schemes
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Reconstruction of microstructures
Correlation features of desired microstructures is provided.
• Aim to reconstruct microstructures that satisfy these ensemble statistical
properties.
• Ill-posed problem with many distributions that satisfy given ensemble
properties.
•
Pb-Sn microstructures
CORNELL
U N I V E R S I T Y
High strength steel
microstructures obtained
by thermal processing
Media with short range
interactions
Materials Process Design and Control Laboratory
Current schemes for microstructure reconstruction
D. Cule and S. Torquato ’99 Reconstruction of porous media using Stochastic Optimization
C. Manwart, S. Torquato and R. Hilfer ’00, Reconstruction of sandstone structures using stochastic optimization
N. Zabaras et.al. ’05 Reconstruction of microstructures using SVM’s
T.C.Baroni et al. ’02, Reconstruction of microstructures using contrast imaging techniques
A.P. Roberts ’97, Reconstruction of porous media using image mapping techniques from 2d planar images.
Stochastic Optimization Procedure
Input: Given statistical correlation or lineal path functions
Obtain: microstructures that satisfy the given properties
 Start from a random configuration over the specified problem domain
such that the volume fraction information is satisfied.
Randomly choose two locations (pixels) and define a move by interchanging
the intensities of the two pixels.
 If the error norm defined as the deviation of the correlation features from
target features reduces, accept the move, otherwise reject it.
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
A MAXENT viewpoint
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Information Theoretic Scheme: the MAXENT principle
Input: Given statistical correlation or lineal path functions
Obtain: microstructures that satisfy the given properties
 Constraints are viewed as expectations of features over a random field.
Problem is viewed as finding that distribution whose ensemble properties
match those that are given.
Since, problem is ill-posed, we choose the distribution that has the
maximum entropy.
 Additional statistical information is available using this scheme.
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
The MAXENT Principle
E.T. Jaynes 1957
The principle of maximum entropy (MAXENT) states that amongst the probability
distributions that satisfy our incomplete information about the system, the probability
distribution that maximizes entropy is the least-biased estimate that can be made. It
agrees with everything that is known but carefully avoids anything that is unknown.
A MAXENT viewpoint
Trivial case: no information is
available about microstructure.
From MAXENT, the equiprobable
case is the case with maximum
entropy for an unconstrained
problem. This agrees with intuition
as to the most unbiased case
CORNELL
U N I V E R S I T Y
Information about volume
fraction given.
Higher order information
provided
The MAXENT distribution is one Correlation between material points
wherein we sample from the
to be taken into account. Result is
volume fraction distribution itself at
not trivial and needs to be
all material points
numerically computed
Materials Process Design and Control Laboratory
MAXENT as a feature matching tool
 D. Pietra et al. ‘96, MAXENT principle for language processing.
Features of language extracted and MAXENT principle is used to
develop a language translator
 Zhu et al. ‘98, MAXENT principle for texture processing
Texture features from images in the form of histograms is extracted and
MAXENT principle used to reconstruct texture images
Sobczyk ’03
MAXENT used for obtaining distributions of grain sizes from macro
constraints in the form of expected grain size.
Koutsourelakis ‘05,
MAXENT for generation of random media. Correlation features of
random media used as constraints to generate samples of random
media.
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
MAXENT for microstructure reconstruction
•
MAXENT is essentially a way of generating a PDF on a hypothesis space which,
given a measure of entropy, is guaranteed to incorporate only known
constraints.
•
MAXENT cannot be derived from Bayes theorem. It is fundamentally different,
as Bayes theorem concerns itself with inferring a-posteriori probability once the
likelihood and a-priori probability are known, while MAXENT is a guiding
principle to construct the a-priori PDF.
•
We associate the PDF with a microstructure image and generate samples of the
image.
•
MAXENT produces images with features (information) that are consistent with
the known constraints. Another way of stating this is that MAXENT produces
the most uniform distribution consistent with the data.
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
MAXENT optimization schemes
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
MAXENT as an optimization problem
Find
feature constraints
Subject to
features of image I
Lagrange Multiplier optimization
Lagrange Multiplier optimization
Partition Function
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Equivalent log-linear model
Equivalent log-likelihood problem
Find
that maximizes
Kuhn-Tucker theorem: The
that maximizes the dual function L also
maximizes the system entropy and satisfies the constraints posed by
the problem
A
Comparison
CORNELL
U N I V E R S I T Y
Direct models
Log-linear models
Concave
Concave
Constrained (simplex)
Unconstrained
“Count and normalize”
(closed form solution)
Iterative methods
Materials Process Design and Control Laboratory
Optimization Schemes
•
•
•
•
Generalized Iterative Scaling
Improved Iterative Scaling
Gradient Ascent
Newton/Quasi-Newton Methods
– Conjugate Gradient
– BFGS
– …
 Start from a
equal to 0. This is equivalent to uniform distribution
over sample space.
 Evaluate gradient at this point.
 Perform a line search on a direction based on the gradient
information.
 Evaluate the gradient information at the next point and continue the
procedure till it is within tolerance limit.
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Gradient Evaluation
• Objective function and its gradients:
stochastic function
stochastic function
• Infeasible to compute at all points in one conjugate gradient iteration
• Use sampling techniques to sample from the distribution evaluated
at the previous point. (Gibbs Sampler)
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Sampling techniques
• Sample from an exponential distribution using the Gibbs algorithm
 Choose a random point.
 Evaluate the effective “energy” for various phases at that point
using the updation algorithm to estimate “energy”.
 Draw a sample from the given distribution and replace the pixel
value at the material point.
 Continue the procedure till a sufficiently large number of samples
are drawn.
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Updation Scheme
A scheme to update correlation function of an image when the
phase of a single pixel is changed
Two point Correlation Function
Lineal Path Function
r
r
Material point whose
intensity is changed
zone of influence (region
where correlation function is affected)
zone of influence
Rozman,Utz ‘01
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Line search and conjugate directions
 Brent’s parabolic interpolation used for line search.
 Stabilization in conjugate gradient machinery (Schraudolph ’02)
 Add a correction term so that as line search becomes increasingly
inaccurate, its effect on the conjugate direction is also subdued.
Stabilization term
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Optimization Schemes
Convergence analysis with stabilization
Convergence analysis w/o stabilization
Noise in function evaluation increases as step size for the next minima increases. This ensures that the
impact on the next evaluation is reduced.
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Entropy variation during MAXENT algorithmic scheme
100
90
80
Entropy(bits)
70
60
50
40
30
20
10
0
CORNELL
U N I V E R S I T Y
0
20
40
60
80
100
120
Iteration
140
160
180
200
Materials Process Design and Control Laboratory
Evaluation of effective
elastic properties
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Effective elastic property of microstructures
Variational Principle: Subject to applied
loads and other boundary conditions,
minimize the energy stored in the
microstructure.
Pixel based mesh with a single phase
inside each pixel (E. Garboczi, NIST
’98). Each pixel attributed the property
of that particular phase.
Homogenization: The effective
homogenized property of the
microstructure is obtained by equating
energy of microstructure with that of a
specimen with uniform properties
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Consolidated Algorithm
Experimental
images
Analytical Correlation
functions
Extract features and rephrase as
mathematical constraints
Pose as a MAXENT problem and use
gradient-based schemes for obtaining
solution
Use Gibbs
sampling algorithm
for sampling from
underlying
distribution
Generate samples and interrogate using
FEM
Obtain property statistics and use them
for further analysis
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Numerical Examples
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Example 1
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Reconstruction of 1d hard disks
Reconstruct one-dimensional hard disk microstructures based on two different
kinds of information: (a) two-point correlation functions (b) two point correlation
and Lineal path function. Obtain elastic property statistics and compare for the
two schemes.
Input: Analytical two-point and lineal path functions (Torquato et.al. ’99)
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Microstructures based on two-point correlation function
MAXENT
distribution
20
18
16
No. of samples
14
12
10
8
6
4
2
0
150
CORNELL
U N I V E R S I T Y
160
170
180
190
200
210
220
Effective young's modulus(GPa)
230
240
250
Materials Process Design and Control Laboratory
Microstructures based on two-point and lineal path function
MAXENT
distribution
35
30
No. of Samples
25
20
15
10
5
0
140
CORNELL
U N I V E R S I T Y
160
180
200
220
Effective young's modulus (GPa)
240
260
Materials Process Design and Control Laboratory
Comparison of property statistics between two schemes
35
20
18
30
16
25
No. of Samples
No. of samples
14
12
10
8
6
20
15
10
4
5
2
0
150
160
170
180
190
200
210
220
Effective young's modulus(GPa)
CORNELL
U N I V E R S I T Y
230
240
250
0
140
160
180
200
220
Effective young's modulus (GPa)
240
260
Materials Process Design and Control Laboratory
Example 2
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Porous Media with short range order
To generate microstructures of porous media which exhibit short range orders of given
specific structure. (S2 is the two point correlation function, k and ro depend depend on
characteristic length scales chosen)
Input: Analytical two-point correlation functions (Torquato et.al. ’99)
Problem Parameters
correlation length ro= 32
2
k
ao
oscillation parameter
ao= 8
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Property statistics for media with short range order
MAXENT
distribution
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Example 3
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Reconstruction using heterogeneous graded materials
Heterogeneous Graded Materials
Given a description of the gradation of phase-distribution in a graded material,
reconstruct microstructures compatible with the given information, estimate statistics of
microstructure properties from this set.
Input: Analytical volume fraction information throughout sample (Koutsourelakis ’04)
Applications
 Tools with desirable properties at
tips.
 Artificial joints for implants in
humans
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Samples of bilinearly graded heterogeneous materials
at smooth resolution levels
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Elastic properties of bilinear graded materials
Effective elastic properties for a tungsten-silver
bilinear graded material at 25oC
35
30
No. of samples
25
20
15
10
5
0
CORNELL
U N I V E R S I T Y
203 210 217 224 231 238 245 252 259 266 273 280 287 294
Effective Young's Modulus(GPa)
Materials Process Design and Control Laboratory
Conclusions and future work
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Conclusions

Microstructures were characterized stochastically and scheme for
obtaining samples based on a MAXENT and time efficient update
scheme implemented.

Gradient based schemes and property of system entropy were
analyzed in detail.

Elastic properties were obtained using FEM and property statistics
developed

Schemes were discussed for numerical microstructures and effect of
incorporation of higher information on property statistics studied.
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Future Work

Extend the method for polycrystal materials incorporating
information in the form of odf’s.

Couple the scheme with pixel based methods for obtaining plastic
properties.

Extend the method to physical deformation processes taking into
account the evolution of microstructure.
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
References
1. E.T. Jaynes, Information Theory and Statistical Mechanics I, Physical Review
106(4)(1957) 620—630.
2. D. Cule and S. Torquato, Generating random media from limited microstructural
information via stochastic optimization, Journal of Applied Physics 86(6)(1999) 3428—
3437
3.P.S. Koutsourelakis, A general framework for simulating random multi-phase media,
NSF Workshop-Probability and Materials: From Nano to Macro scale (2005)
4. K. Sobczyk, Reconstruction of random material microstructures: patterns of Maximum
Entropy, Probabilistic Engineering Mechanics 18(2003) 279—287
5. S.C.Zhu et al, Filters, Random Fields and Maximum Entropy (FRAME): Towards a
Unified Theory for Texture Modeling, IJCV 27(1998) 107-126
6. A.Berger et.al., A maximum entropy approach to natural language modeling, (1996),
Computational Linguistics 22 (1996),39-71
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory