Transcript Title Place here

Improving Security Decisions
with Polymorphic and Audited
Dialogs
José Carlos Brustoloni and Ricardo Villamarín-Salomón
Dept. Computer Science
University of Pittsburgh
{jcb,rvillsal}@cs.pitt.edu
The problem
♦
Context-dependent security decisions where
application needs user input to characterize context
♦
Problem: user will give false inputs if necessary to get
application to perform action user wants
SOUPS 2007
J. Brustoloni and R. Villamarin
2
Example
♦
Should an email agent allow the user to open an email
attachment?
♦
Decision depends on context:
♦
♦
♦
♦
♦
♦
Does user know sender?
Would alleged sender have used that particular account?
Do message subject and body make sense?
Was user expecting attachment from sender?
...
Email agent would need to ask user
SOUPS 2007
J. Brustoloni and R. Villamarin
3
What do applications actually do?
♦
Warn and continue (W&C) – e.g., IE, Firefox


♦
No warning (NW) – e.g., Thunderbird

♦
Hope that user will competently and independently judge
situation
Usually futile – most users blindly hit continue
Trade off security for usability
No dialog (ND) – e.g., recent versions of MS Outlook



Application hides unsafe attachments – user cannot open or
save them
Can puzzle and upset users
Trade off usability for security
SOUPS 2007
J. Brustoloni and R. Villamarin
4
Can’t a dialog guide user’s decision?
♦
Context Sensitive Guidance (CSG):

ask about user context → user gives true answers → perform secure
action
♦
In theory, it should work
♦
In practice, much harder than you’d expect
User will answer anything that seems necessary to get action
user wants
 User will learn the “successful” sequence of answers and
repeat it automatically in the future, regardless of context
 They are not disturbed by the fact they’re being observed
 Will gleefully volunteer that they do that all the time in real
life
SOUPS 2007
J. Brustoloni and R. Villamarin
5

Contributions
♦
Two techniques for improving
truthfulness of user inputs in security
dialogs:
 Polymorphic
dialogs
 Audited dialogs
SOUPS 2007
J. Brustoloni and R. Villamarin
6
Theory
♦
Context-sensitive guidance not necessarily rewarding:

user context → true answers → secure action (may not be what user
wants)
♦
Many security dialog prompts are fixed and
user answers are nearly always the same
♦
Operant conditioning theory predicts what actually happens:

♦
fixed dialog → automatic answers → action user wants
Our interventions seek to improve users’ behavior (answers) by
manipulating:
in polymorphic dialogs, the behavior’s antecedents (dialog prompts)
 in audited dialogs, the behavior’s consequences (penalties for
unjustified answers)

SOUPS 2007
J. Brustoloni and R. Villamarin
7
Polymorphic dialogs
♦
Deliberately vary dialog form to avoid triggering automatic
answers
♦
Thoughtless answers have unpredictable consequences
♦
Greater effort to give false answers that enable action user wants
♦
Design space for polymorphism is vast
♦
We consider only two examples of polymorphism in experiments
SOUPS 2007
J. Brustoloni and R. Villamarin
8
Example: display options in random order
SOUPS 2007
J. Brustoloni and R. Villamarin
9
Another example: delay confirmation
♦
♦
SOUPS 2007
A similar technique already used
in dialog to install Firefox
extensions
But general design principle
(polymorphic dialogs) does not
seem to have been enunciated or
evaluated before
J. Brustoloni and R. Villamarin
10
Audited dialogs
♦
♦
Keep audit log to make users accountable for their answers
Operant conditioning:

♦
dialog → false answer → action user wants, but also penalty
Three application modifications:
1.
Notify users that answers may be audited
SOUPS 2007
J. Brustoloni and R. Villamarin
11
Confirmation
2.
Notify user that user’s answers
and context (e.g., message and
attachments) will be forwarded
to auditors if user confirms
operation
SOUPS 2007
J. Brustoloni and R. Villamarin
12
Suspension
3.
Auditors can suspend user if they find user’s answers
unjustifiable.
SOUPS 2007
J. Brustoloni and R. Villamarin
13
Deployment considerations
♦
Intended for enterprise (not home) users
♦
Probably easiest and least intrusive for auditors to
send users training messages containing attachments
that auditors a priori consider unjustified risks
♦
Penalties for accepting unjustified risks:



analogy: penalties for traffic violations
may involve suspension, fines, required training, ...
could increase with each subsequent violation
SOUPS 2007
J. Brustoloni and R. Villamarin
14
Evaluation
♦
Compare 3 versions of Thunderbird



♦
NW (no warning – current default)
CSG-PD (context sensitive guidance with polymorphic
dialogs)
CSG-PAD (context sensitive guidance with polymorphic and
audited dialogs)
User experiments in laboratory – two user groups
CSG-PD
CSG-PAD
# Participants
13
7
# Female
10
6
Familiarity with email agents
(SR)
4.1 / 5
3.9 / 5
Ease of user study tasks (SR)
4.5 / 5
4.3 / 5
# Unjustified risks accepted w/
79%
SOUPSNW
2007
J. Brustoloni and R. Villamarin
66%
15
Sidebar for context-sensitive guidance
SOUPS 2007
J. Brustoloni and R. Villamarin
16
Scenarios
♦
Each user role-played employees in two scenarios (random order)
♦
First scenario used NW, second scenario used CSG-PD or CSGPAD
♦
Each scenario comprises 10 messages with attachments
2 with justifiable risk
 8 with unjustifiable risk

SOUPS 2007
J. Brustoloni and R. Villamarin
17
Comparison between NW and CSG-PD
♦
Significant reduction in
unjustified risks accepted, large
effect
effect is due to CSG and
polymorphism
 in pilots, CSG alone seemed to
have insignificant effect

♦
Insignificant effect in justified
risks accepted
♦
Significant reduction in task
completion time, medium effect

effect due to reduction in
unjustified risks accepted
(typically not task-relevant)
SOUPS 2007
J. Brustoloni and R. Villamarin
18
Comparison between NW and CSG-PAD
♦
Significant reduction in
unjustified risks accepted, large
effect

effect is due to CSG,
polymorphism, and auditing
♦
Insignificant effect in justified
risks accepted
♦
Insignificant effect in task
completion time
SOUPS 2007
J. Brustoloni and R. Villamarin
19
Comparison between CSG-PD and CSG-PAD
♦
Significant reduction in
unjustified risks accepted, large
effect

effect is due to auditing only
♦
Insignificant effect in justified
risks accepted
♦
Insignificant effect in task
completion time
SOUPS 2007
J. Brustoloni and R. Villamarin
20
Effects of habituation
40.00%
20.00%
Net acceptance frequency
Unjustified risk number
0.00%
1
2
3
4
5
6
7
8
-20.00%
-36%
-40.00%
-58%
-60.00%
-80.00%
-100.00%
CSG-PD
SOUPS 2007
CSG-PAD
J. Brustoloni and R. Villamarin
21
User perceptions
(1=worst, 5=best)
♦
CSG-PD
CSG-PAD
Dialogs are easy to
understand
3.9
3.7
Questions are helpful
2.4
2.1
Interface provides good
guidance
3.6
2.6
Participant followed guidance
2.5
2.4
Would feel comfortable
3.7
3.0
receiving such guidance in
future
Several users did not understand auditors’ messages, thus found
Would recommend
3.1
1.9
penalties
arbitrary to friend
e.g., couldn’t understand how email from coworker might contain virus
 auditor messages should better explain concepts and rules behind penalty
decisions

SOUPS 2007
J. Brustoloni and R. Villamarin
22
Related work
♦
Xia and Brustoloni:



Guidance without override (GWO): application makes and enforces
decision, based on inputs users find easier to provide legitimately
(e.g. certificate verification)
Guidance with override (G+O): application merely suggests decision,
based on inputs users can easily forge (e.g. whether to send
password in plaintext)
We found it much harder to obtain significant benefits from the
latter
 possibly due to greater complexity of attachment security policy
SOUPS 2007
J. Brustoloni and R. Villamarin
23
Other related work
♦
Wu et al.: Web Wallet – G+O, effective against phishing,
specialized
♦
Whitten and Tygar: safe staging vs. just-in-time instruction
(JITI, e.g., GWO, G+O)
♦
Kumaraguru et al.: embedded training against phishing
graphics and especially comics more effective than text
 similar approach could be used to improve auditors’ messages

SOUPS 2007
J. Brustoloni and R. Villamarin
24
Conclusions
♦
♦
♦
♦
♦
♦
Designing effective security dialogs that elicit context
information from users can be a formidable challenge
Many users do not hesitate to give false answers in order to get
the actions they want
We contributed two techniques for significantly improving
truthfulness of user answers
Polymorphic dialogs avoid triggering automatic answers by
continuously changing the form of the dialog
Audited dialogs hold users accountable for their answers by
forwarding them to auditors
User studies show both techniques give statistically significant,
large benefits
SOUPS 2007
J. Brustoloni and R. Villamarin
25