Transcript A, B - Tohoku University

Physical Fluctuomatics
Applied Stochastic Process
2nd Probability and its fundamental properties
Kazuyuki Tanaka
Graduate School of Information Sciences, Tohoku University
[email protected]
http://www.smapip.is.tohoku.ac.jp/~kazu/
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
1
Probability
a. Event and Probability
b. Joint Probability and Conditional
Probability
c. Bayes Formula, Prior Probability and
Posterior Probability
d. Discrete Random Variable and Probability
Distribution
e. Continuous Random Variable and
Probability Density Function
f. Average, Variance and Covariance
g. Uniform Distribution
h. Gauss Distribution
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
This Talk
Next Talk
2
Event, Sample Space and Event
Experiment: Experiments in probability theory means that
outcomes are not predictable in advance. However, while
the outcome will not be known in advance, the set of all
possible outcomes is known
Sample Point: Each possible outcome in the experiments.
Sample Space：The set of all the possible sample points in the
experiments
Event：Subset of the sample space
Elementary Event：Event consisting of one sample point
Empty Event：Event consisting of no sample point
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
3
Various Events
Whole Events Ω:Events consisting of all sample points
of the sample space.
Complementary Event of Event A: Ac=Ω╲A
Defference of Events A and B: A╲B
Union of Events A and B: A∪B
Intersection of Events A and B: A∩B
Events A and B are exclusive of each other: A∩B=Ф
Events A, B and C are exclusive of each other:
[A∩B=Ф]Λ[B∩C=Ф]Λ[C∩A=Ф]
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
4
Empirically Definition of Probability
Definition by Laplace: Let us suppose that the total
number of all the sample points is N and they can occur
equally Likely. Probability of an event A with N sample
points is defined by p=n/N.
Statistical Definition: Let us suppose that an event A
occur r times when the same experiment are repeated R
times. If the ratio r/R tends to a constant value p as the
number of times of the experiments R go to infinity, we
define the value p as probability of event A.
r
 p R   
Pr A  p
R

Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
5
Definition of Probability
Definition of Kolmogorov: Probability Pr{A} for every
event A in the specified sample space Ω satisfies the
following three axioms:
Axion 1:
PrA  0
Axion 2:
Pr  1
Axion 3: For every events A, B that are exclusive of
each other, it is always valid that
PrA  B  PrA PrB
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
6
Joint Probability and Conditional Probability
Probability of Event A
Pr{A}
Joint Probability of Events A and B
PrA, B  PrA  B
Conditional Probability of Event A
when Event B has happened.
PrA, B
PrB A 
PrA
 PrA, B  PrB APrA
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
A
B
7
Joint Probability and Independency of Events
Events A and B are independent of each other
PrA, B  PrAPrB
In this case, the conditional probability
can be expressed as
PrB A  PrB
A
A
B
B
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
8
Marginal Probability
Let us suppose that the sample W is expressed by
Ω=A1∪A2∪…∪AM where every pair of events Ai and Aj
are exclusive of each other.
M
PrB   PrAi , B
i 1
Marginal Probability of Event B for
Joint Probability Pr{Ai,B}
PrB   PrA, B
Simplified Notation
A
Ai
B
Marginalize
A
B
Summation over all the possible events in which every pair
of events are exclusive of each other.
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
9
Four Dimensional Point Probability
and Marginal Probability
Marginal Probability of Event B
PrB   PrA, B, C , D
A
Marginalize
C
D
A
B
C
D
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
10
Derivation of Bayes Formulas
PrA, B  PrA BPrB
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
11
Derivation of Bayes Formulas
PrA, B  PrA BPrB
PrA, B  PrB APrA
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
12
Derivation of Bayes Formulas
PrA, B  PrA BPrB
PrA, B  PrB APrA
PrA, B
PrA B 
PrB
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
13
Derivation of Bayes Formulas
PrA, B  PrA BPrB
PrA, B  PrB APrA
PrA, B PrB APrA
PrA B 

PrB
PrB
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
14
Derivation of Bayes Formulas
PrA, B  PrA BPrB
PrA, B  PrB APrA
PrA, B PrB APrA
PrA B 

PrB
PrB

PrB  
A
PrB APrA
PrA, B

PrA, B
A
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
15
Derivation of Bayes Formulas
PrA, B  PrA BPrB
PrA, B  PrB APrA
PrA, B PrB APrA
PrA B 

PrB
PrB

PrB  
PrB APrA
PrB APrA

PrA, B  PrB APrA

PrA, B
A
A
A
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
16
Derivation of Bayes Formulas
PrA, B  PrA BPrB
A
PrA, B  PrB APrA
B
PrA, B PrB APrA
PrA B 

PrB
PrB

PrB  
PrB APrA
PrB APrA

PrA, B  PrB APrA

PrA, B
A
A
A
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
17
Bayes Formula
PrA B 
PrB APrA
 PrB APrA
Prior
Probability
A
A
Posterior Probability
It is often referred to as Bayes Rule.
B
Bayesian Network
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
18
Summary
a. Event and Probability
b. Joint Probability and Conditional
Probability
c. Bayes Formulas, Prior Probability and
Posterior Probability
d. Discrete Random Variable and Probability
Distribution
e. Continuous Random Variable and
Probability Density Function
f. Average, Variance and Covariance
g. Uniform Distribution
h. Gauss Distribution
Physics Fluctuomatics / Applied
Stochastic Process (Tohoku University)
The present
talk
Next talk
19