Transcript Notes on image quality

Statistics and Image Evaluation
Oleh Tretiak
Medical Imaging Systems
Fall, 2002
1
Which Image Is Better?
2
Which Image is Better?
3
Overview
•
•
•
•
•
•
•
•
Why measure image quality?
How to measure image quality
Statistical variation and probability theory
Some results in probability theory
Some results in statistics
Experimental design
Subjective quality measurement
ROC theory and estimation
4
Why Measure Image Quality
• Market studies (sell films)
• Market studies (sell equimpent)
• Test if equipment is working up to
specification
• Measure effect of equipment on radiologists
performance
• Measure the ability to perform diagnosis
5
Types of ‘Quality’
• Viewer preference
– Relevant for entertainment, home viewing
• Technical quality
– Process control (equipment maintenance)
• Utility
– Ability to perform diagnosis, drive a remote
vehicle, locate enemy weapons
6
How to Measure Image Quality
• Viewer preference
– Viewer trials
• Technical quality
– Phantoms, resolution targets, expert viewers
• Utility
– Viewer trials with expert viewers
7
Variation and Probability Theory
• Sources of variation
– Input: Different patients are different
– Equipment: Different equipment is different,
same equipment is different at different times
– Subjective: Different viewers report different
opinions, same viewers report different findings
at different times
8
Probability Theory
• Probability theory used almost universally
• Models
– Independent trials
– Dependency effects
• Same films viewed by different viewers
• Same patients imaged by different modalities
• Same patients viewed by several radiologists,
multiple reading
9
Example
• Goal: evaluate quality of television set
• Method: Ask viewers to report quality of
image, standard viewing conditions
• Viewers report a quality number 1-5
– 1-Terrible, 2-Bad, 3-So-so, 4-Good, 5-Excellent
• Evaluation: Compute average response
10
Probabilistic Model
• Reported values are iid random variables.
–
–
–
–
Set {1, 2, 3, 4, 5}
Probabilities p(1), p(2), p(3), p(4), p(5)
Probabilities are unknown!
Example: 100 observation, computed average
value is ?
11
Breakout
• Excel visq.xls
• Blackboard
12
Conclusions
• Results depend on who you ask
• Average result measured from a sample
varies from sample to sample
• Prob. Theory tells us that with large
samples, the average is equal to expected
value
• Why does this matter?
13
Some Results from Prob. Theory
•
•
•
•
•
Mean (Expected value)
Variance, Standard Deviation
Sample mean, sample variance
Law of large numbers
Central limit theorem
14
Some Results in Statistics
• Statistical problems
– Estimation
– Hypothesis testing
– Confidence level
15
Estimate of the Mean
• For a Gaussian (Normal) random variable
with known standard deviation,
Pr[X 1.96 X    X 1.96 X ]  0.95
• ~ a is the confidence level

  X  za /2


n
16
Practical Issues
• Can use for non-normal distributions (CLT)
• If n is large and st. dev. Is not known, use
sample st. dev.
• Small sample, unknown st. dev. — use the
Student t statistic.
17
Experiment
Quality 5
Quality 1
18
Case A
19
Case B
20
More Statistics
• Estimate of variance
– chi-squared (ki-squared)
– Depends on number of observations (degrees of
freedom)
• Estimate ratio of independent normal
variances
– Fisher f distribution
– Most important
21