Transcript Belief in Information Flow

Belief in Information Flow
Michael Clarkson, Andrew Myers, Fred B. Schneider
Cornell University
18th IEEE Computer Security Foundations Workshop
June 20, 2005
Password Checker
Some programs require leakage of information
PWC: if p = g then a := 1 else a := 0
p: stored password
g: guessed password
a: authentication flag
Clarkson et al.: Belief in Information Flow
2
Password Checker

PWC: if p = g then a := 1 else a := 0
a depends on p:
Noninterference is too strong
But intuitively secure because PWC
leaks little information about p…
Clarkson et al.: Belief in Information Flow
3
Quantitative Information Flow
• Quantitative security policies:
– Expected rate of flow is at most k bits per
second
– At most k bits leak in any execution
• Enforcing these requires a model for
quantitative information flow (QIF)
• This work: A model for QIF
Clarkson et al.: Belief in Information Flow
4
Traditional Model for QIF
probability distributions
Hin
Lin
Hout
S
Lout
Information flows when uncertainty is decreased
• Flow = Uncertainty(Hin) – Uncertainty(Hin | Lout)
• Uncertainty measured with some variant of entropy
[Denning 82; McIver and Morgan 03; Clark, Hunt, and Malacaria 05]
Clarkson et al.: Belief in Information Flow
5
Adding Beliefs to Model
Hin
Lin
Hout
S
Lout
• Model attacker’s uncertainty about H inputs as a
probability distribution
• We call this distribution a belief
Clarkson et al.: Belief in Information Flow
6
Analyzing PWC
PWC: if p = g then a := 1 else a := 0
Attacker believes:
p = A
B
C
0.98
0.01
0.01
Attacker guesses A:
g = A
(Password is really C)
After observing a = 0, attacker believes:
p = A
B
C
0
0.5
0.5
Clarkson et al.: Belief in Information Flow
7
Analyzing PWC
Prebelief
p = A
B
C
0.98
0.01
0.01
a little uncertainty
Postbelief
p = A
B
C
0
0.5
0.5
more uncertainty
Uncertainty = closeness to uniform distribution
Traditional metric:

Information flows when uncertainty is decreased
Clarkson et al.: Belief in Information Flow
8
Why Uncertainty Fails
Uncertainty-based approach addresses
objective probabilities on system but not
subjective probabilities of attacker
Clarkson et al.: Belief in Information Flow
9
Metric for Belief
d2
d1
p = A
B
C
0.98
0.01
0.01
prebelief
p = A
B
C
0
0.5
0.5
postbelief
p = A
B
C
0
0
1
reality
d1 > d2: postbelief closer to reality because of
observation of program
Clarkson et al.: Belief in Information Flow
10
Accuracy
Accuracy: Distance from a belief to reality
Certainty
p = A
B
C
0.98
0.01
0.01
prebelief
Accuracy
p = A
B
C
0
0.5
0.5
postbelief
Accuracy is the correct metric for QIF
Clarkson et al.: Belief in Information Flow
11
Belief in Information Flow
• Experiment protocol
Describes how attackers revise beliefs from
observation of program execution
• Accuracy metric
How change in accuracy of belief can be used to
measure the amount of information flow
• Extensions
Repeated experiments, other metrics, and
misinformation
Clarkson et al.: Belief in Information Flow
12
Experiments
Experiment: How an attacker revises his belief
Prebelief
p = A
B
C
0.98
0.01
0.01
Clarkson et al.: Belief in Information Flow
Postbelief
p = A
B
C
0
0.5
0.5
13
Experiment Protocol
Hin
Lin
preB
Clarkson et al.: Belief in Information Flow
Hout
S
Lout
postB
14
Experiment Protocol
Hin
Lin
Hout
S
preB
1.
Lout
postB
Attacker chooses prebelief preB
Clarkson et al.: Belief in Information Flow
15
Experiment Protocol
Hin
Lin
Hout
S
Lout
preB
1.
2.
postB
Attacker chooses prebelief preB
System and attacker choose inputs Hin, Lin
Clarkson et al.: Belief in Information Flow
16
Experiment Protocol
Hin
Lin
Hout
S
Lout
observation
preB
1.
2.
3.
postB
Attacker chooses prebelief preB
System and attacker choose inputs Hin, Lin
System executes S and produces observation
Execution modeled as a distribution transformer semantics:
«S¬ : Dist ! Dist
Clarkson et al.: Belief in Information Flow
17
Experiment Protocol
Hin
Lin
preB
1.
2.
3.
4.
Hout
S
Lout
observation
postB
Attacker chooses prebelief preB
System and attacker choose inputs Hin, Lin
System executes S and produces observation
Attacker conducts thought-experiment to obtain prediction
Clarkson et al.: Belief in Information Flow
18
Experiment Protocol
Hin
preB
Lin
4.
H’out
S
L’out
prediction
Attacker conducts thought-experiment to obtain prediction
Clarkson et al.: Belief in Information Flow
19
Experiment Protocol
preB
Hin
S
S
prediction
Lin
observation
Lin
preB
1.
2.
3.
4.
5.
postB
Bayesian inference
Attacker chooses prebelief preB
System and attacker choose inputs Hin, Lin
System executes S and produces observation
Attacker conducts thought-experiment to obtain prediction
Attacker infers postbelief:
postB
=
Clarkson et al.: Belief in Information Flow
prediction
|
observation
20
Belief Revision in PWC
0.98
0.01
0.01
preB
=
p = A
B
C
postB
=
prediction
=
p,a = A,0
A,1
B,0
B,1
C,0
C,1
0
0.98
0.01
0
0.01
0
p = A
B
C
0
0.5
0.5
´
Clarkson et al.: Belief in Information Flow
|
observation
|
a = 0
21
Accuracy Metric
Amount of flow Q is improvement in accuracy of belief
i.e. (initial error) – (final error)
D(preB ! Hin)
preB
D(postB ! Hin)
postB
Hin
Q = D(preB ! Hin) – D(postB ! Hin)
Clarkson et al.: Belief in Information Flow
22
Belief Distance
Relative entropy:
Unit is (information theoretic) bits
When D is defined as relative entropy, Q is the
amount of information that the attacker’s
observation contains about the high input.
Clarkson et al.: Belief in Information Flow
23
Amount of Flow from PWC
Q = D(.98, .01, .01 ! 0, 0, 1) – D(0, .5, .5 ! 0,
0, 1)
= 5.6 bits
Information is in the eye of the beholder
Max leakage of lg 3 bits
implies uniform prebelief:
5 bits
.98, .01, .01
Clarkson et al.: Belief in Information Flow
1/3 each
p = A,B,C
.6 bits
1/3, 1/3, 1/3
1.6 bits
0, .5, .5
0, 0, 1
24
Repeated Experiments
Experiment protocol is compositional
B
Exp
B’
Exp
B’’
Information flow is also compositional
The amount of information flow over a series of
experiments is equal to the sum of the amount of
information flow in each individual experiment.
Clarkson et al.: Belief in Information Flow
25
Extensions of Metric
• So far: exact flow for a single execution
• Extend to:
{Expected, maximum}
amount of information flow
{for a given experiment, over all experiments}
• Language for quantitative flow policies
Clarkson et al.: Belief in Information Flow
26
Misinformation
Certainty
?
Accuracy
FPWC: if p = g then a := 1 else a := 0;
if random() < .1 then a := !a
Non-probabilistic programs cannot create misinformation.
Clarkson et al.: Belief in Information Flow
27
Summary
• Attackers have beliefs
• Quantifying information flow with beliefs
requires accuracy
– Traditional uncertainty model is inappropriate
• Presented more expressive, fine-grained model of
quantitative information flow
– Compositional experiment protocol
– Probabilistic language semantics
– Accuracy-based metric
Clarkson et al.: Belief in Information Flow
28
Related Work
• Information theory in information flow
– [Denning 82], [Millen 87], [Wittbold, Johnson 90],
[Gray 91]
• Quantitative information flow
– Using uncertainty: [Lowe 02], [McIver, Morgan 03],
[Clark, Hunt, Malacaria 01 - 05]
– Using sampling theory: [Di Pierro, Hankin, Wiklicky
00 - 05]
• Database privacy
– Using relative entropy: [Evfimievski, Gehrke, Srikant
03]
Clarkson et al.: Belief in Information Flow
29
Future Work
• Extended programming language
• Lattice of security levels
• Static analysis
• Quantitative security policies
Clarkson et al.: Belief in Information Flow
30
Belief in Information Flow
Michael Clarkson, Andrew Myers, Fred B. Schneider
Cornell University
18th IEEE Computer Security Foundations Workshop
June 20, 2005
Extra Slides
Clarkson et al.: Belief in Information Flow
32
Beliefs as Distributions
• Other choices: Dempster-Shafer belief functions,
plausibility measures, etc.
• Probability distributions are:
–
–
–
–
Quantitative
Axiomatically justifiable
Straightforward
Familiar
• Abstract operations
– Product: combine disjoint beliefs
– Update: condition belief to include new information
– Distance: quantification of difference between two
beliefs
Clarkson et al.: Belief in Information Flow
33
Program Semantics
Clarkson et al.: Belief in Information Flow
34
Postbeliefs are Bayesian
• Standard techniques for inference in
applied statistics
– Bayesian inference
– Sampling/frequentist theory
• Bayesian inference:
– Formalization of scientific method
– Consistent with principles of rationality in a
betting game
• May be subjective but is not arbitrary
Clarkson et al.: Belief in Information Flow
35
Interpreting Flow Quantities
• Uncertainty (entropy) interpreted as:
– Improvement of expected codeword length
• Accuracy (relative entropy) interpreted as:
– Improvement in efficiency of optimal code
Clarkson et al.: Belief in Information Flow
36