Transcript FIS_statistics_2

Topics:
Statistics & Experimental Design
The Human Visual System
Color Science
Light Sources: Radiometry/Photometry
Geometric Optics
Tone-transfer Function
Image Sensors
Image Processing
Displays & Output
Colorimetry & Color Measurement
Image Evaluation
Psychophysics
Design of experiments
Why is it important?
• We wish to draw meaningful conclusions from
data collected
• Statistical methodology is the only objective
approach to analysis
Design of experiments
• Recognize the problem
• Select factor to be varied, levels and ranges
over which factors will be varied
• Select the response variable
• Choose experimental design:
• Sample size?
• Blocking?
• Randomization?
• Perform the experiment
• Statistical analysis
• Conclusions and recommendations
Let’s start easy
• We would like to compare the output of two
systems.
• Design a testing protocol and run it several times
Run
SystemA
SystemB
1
y1A
y1B
2
y2A
y2B
3
y3A
y3B
…
…
…
Visualize data
For small data sets: Scatter plot
18.5
Output
18
17.5
system_A
system_B
17
16.5
16
0
1
2
3
4
5
Run #
6
7
8
9
10
Visualize data
frequency, ni
For larger data sets: Histogram
• Divide horizontal axis into intervals (bins)
• Construct rectangle over interval with area proportional to
number (frequency) of observations
Statistical inference
Draw conclusions about a population using a
sample from that population.
• Imagine hypothetical population containing a large number
N of observations.
• Denote measure of location of population as
1
Population mean     yi
N i

Statistical inference
• Denote spread of population as variance
2 
2


 yi  
i
N

Statistical inference
A small group of observations is known as a
sample.
• A statistic like the average is calculated from a set of data
considered to be a sample from a population
Run
SystemA
SystemB
1
y1A
y1B
2
y2A
y2B
3
y3A
y3B
…
…
…
yA
yB
1 n
Sample average  y   y i
n i 1
Statistical inference
• Sample variance supplies a measure of the spread of the sample
0.275
0.25
0.225
0.2
n
0.175
0.15
s2 
0.125
0.1
0.075
0.05
0.025
0
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
5.5
6
2


y

y
 i
i 1
n 1
Probability distribution functions
f
1
0.75
P(axb)
0.5
0.25
0
-5
-2.5
0
2.5
5
x
Probability distribution functions
P(xi)
P(x = xi) = p(xi)
xi
0  px i   1 for all values of x i
Px  x i   px i  for all values of x i
 px   1
i
xi
Mean, variance of pdf
• Mean is a measure of central tendency or location
   xp x 
y
• Variance measures the spread or dispersion
   x    px 
2
2
y
Normal distribution
1
f x  
e
 2
y
-10
-5
1  x 
 

2  
2
 = standard deviation = √2
  mean
0.5
y
0.5
y
0.5
0.375
0.375
0.375
0.25
0.25
0.25
0.125
0.125
0.125
0
5
10
-10
-5
0
x
  3,   3
5
10
-10
-5
0
x
  0,   2
5
10
x
  3,   1
Normal distribution, N , 
2

• From previous examples we can see that mean =  and
variance = 2 completely characterize the distribution.
• Knowing the pdf of the population from which sample is draw
 determine pdf of particular statistic.
Normal distribution
• Probability that a positive deviation from the mean exceeds one standard deviation  is
0.1587 1/6 = percentage of the total area under the curve. (Same as negative deviation)
• Probability that a deviation in either direction will exceed one standard deviation  is
2 x 0.1587 = 0.3174
• Chance that a positive deviation from the mean will exceed two  = 0.02275  1/40
Normal distribution
• Sample runs differ as a result of experimental error
• Often can be described by normal distribution
Standard Normal distribution, N(0,1)
y
0.5
0.375
0.25
0.125
-10
-5
0
5
10
x
z
y

Values for N(0,1) are found in tables.
Standard Normal distribution, N(0,1)
Standard Normal distribution, N(0,1)
Example:
Suppose the outcome of a given experiment is approximately
normally distributed with a  = 4.0 and  = 0.3. What is the
probability that the outcome may be 4.4?
z
yμ
4.4  4

 1.33
σ
0.3
Look in table in previous page, to find that the probability is 9%.
2
c
distribution
Another sampling distribution that can be defined
in terms of normal random variables.
• Suppose z1, z2, …, zk are normally and independently
distributed random variables with mean  = 0 and variance 2 =
1 (NID(0,1)), then let’s define
c  z  z  z
2
1
2
2
2
k
Where c follows the chi-square distribution with k degrees of
freedom.
2
c
0.2
distribution
k=1
k=5
0.15
k = 10
0.1
k = 15
0.05
0
0
5
10
15
20
25
Student’s t Distribution
• In practice we don’t know the theoretical parameter 
• This means we can’t really use
z
y

and refer to the result of
the table of standard normal distribution
• Assume that experimental standard deviation s can be used as
an estimate of 
Student’s t Distribution
Define a new variable
y
t
s
It turns out that t has a known distribution.
It was deduced by Gosset in 1908
Student’s t Distribution
k=100
k=10
0.3
k=1
0.2
0.1
0
-5
-2.5
0
2.5
5
Probability points are given in tables.
The form depends on the degree of uncertainty in s2, measured
by the number of degrees of freedom, k.
Inferences about differences in means
• Statistical hypothesis: Statement about the parameters of a
probability distribution.
Let’s go back to the example we started with, i.e., comparison of
two imaging systems.
We may think that the performance measurement of the two
systems are equal.
Hypothesis testing
H 0 : 1   2
H 1 : 1   2
First statement is the Null hypothesis, second statement is the
Alternative hypothesis. In this case it is a two-sided alternative
hypothesis.
How to test hypothesis? Take a random sample, compute an
appropriate test statistic and reject, or fail to reject the null
hypothesis H0.
We need to specify a set of values for the test statistic that
leads to rejection of H0. This is the critical region.
Hypothesis testing
Two errors can be made:
• Type I error: Reject null hypothesis when it is true
• Type II error: Null hypothesis is not rejected when it is not true
• In terms of probabilities:
  Ptype I error   Preject H 0 H 0 is true 
  Ptype II error   Pfail to reject H 0 H 0 is false 
Hypothesis testing
• We need to specify a value of the probability of type I error .
This is known as significance level of the test.
• The test statistic for comparing the two systems is:
yA  yB
t0 
1
1
sp

kA kB
Where
2
2




k

1
s

k

1
s
A
B
B
s2  A
p
kA  kB  2
Hypothesis testing
• To determine whether to reject H0, we would compare t0 to the t
distribution with kA+kB-2 degrees of freedom.
• If t 0  t / 2,k A k B 2 we reject H0 and conclude that means are
different.
We have:
System A
System B
yA  16.76
yB  17.92
s 2A  0.1
s 2A  0.061
sA  0.316
s B  0.247
k A  10
k B  10
Hypothesis testing
H 0 : 1   2
H 1 : 1   2
• We have kA + kB – 2 = 18
• Choose  = 0.05
• We would reject H0 if
t 0  t 0.05,18  t 0.025,18  2.101
Hypothesis testing
2
2




k

1
s

k

1
s
9.01  90.061
A
B
B
s2  A

 0.081
kA  kB  2
p
18
s p  0.284
t0 
yA  yB
16.76  17.92

 9.13
1
1
1
1
sp

0.284

kA kB
10 10
Hypothesis testing
Since t0 = -9.13 < -t0.025,18 = -2.101 then we reject H0 and
conclude that the means are different.
Hypothesis testing doesn’t always tell the whole story. It’s
better to provide an interval within which the value of the
parameter is expected to lie. Confidence interval.
In other words, it’s better to find a confidence interval on the
difference A - B
Confidence interval
y A  y B  t  / 2 ,k 1  k 2 2s p
1
1
1
1

  A   B  y A  y B  t  / 2 ,k 1  k 2 2s p

kA kB
kA kB
Using data from previous example:
16.76  17.92  2.1010.284
1
1
1
1

  A   B  16.76  17.92  2.1010.284

10 10
10 10
 1.16  0.27   A   B  1.16  0.27
 1.43   A   B  0.89
So the 95 percent confidence interval estimate on the difference in
means extends from -1.43 to -0.89.
Note that since A – B = 0 is not included in this interval, the data
do not support the hypothesis that A = B at the 5% level of
significance.