Transcript 5%20-%20Paulett - The Team for Research in Ubiquitous

Automatic Detection of Policies from
Electronic Medical Record Access Logs
John M. Paulett †, Bradley Malin†‡
† Department of Biomedical Informatics
‡ Department of Electrical Engineering and Computer Science
Vanderbilt University
TRUST Autumn Conference
November 11, 2008
Privacy in Healthcare
Sensitive Data
– Patients speak with expectation of
confidentiality
– Socially taboo diagnoses
– Employment
– HIPAA
11/11/2008
2
TRUST
Language for specifying temporal policies
– Barth et al.
Framework for integrating policies with system and
workflow models
– Werner et al.
Model Integrated Clinical Information System
(MICIS)
– Mathe et al.
11/11/2008
3
Status
TRUST tool to formally specify, model, and
managing policies in the context of existing
and evolving clinical information systems
But, where do these policies come from?
11/11/2008
4
External Threat
Success with standard security best-practices
11/11/2008
5
Insider Threat
Motivation
–
–
–
–
Celebrities
Friends / Neighbors
Coworkers
Spouse (divorce)
Evidence of misuse
– 6 fired, 80 re-trained – University of California, Davis
– 13 fired for looking at Britney Spears’ record – March
2008
– George Clooney – October 2007
11/11/2008
6
Protecting Against Insiders
• Access Control
– Limit users to only the set of patients they need to
care for
– Stop improper accesses from occurring
• Auditing
– Catch improper accesses after the fact
11/11/2008
7
Access Control in Healthcare
Upfront definition of policies is problematic
– “Experts” have incomplete knowledge
– Healthcare is dynamic: workflows and interactions
change faster than experts can define them
“False Positives” cause a negative impact on
clinical workflow and potentially patient harm
– “Break the glass”
11/11/2008
8
Auditing in Healthcare
Huge amount of data, every day:
– Hundreds to thousands of providers
– Millions of patients
Which accesses are improper?
11/11/2008
9
Current Auditing
11/11/2008
10
Current Auditing
Vanderbilt University Medical Center
– 1 Privacy Officer
– 2 staff
Auditing focus
– Monitor celebrities
– Monitor employee-employee access
– Follow-up on external suspicion
– Spot checks
11/11/2008
11
Our Goal
Inform Policy Definition Tools
– Werner et al.
– Barth et al.
Assist auditing by defining what is normal
11/11/2008
12
Our Approach
Characterize normal operations, workflows, and
relationships
– Use access logs as proxy for this information
11/11/2008
13
Our Approach
Relational Network
– Two providers related if they access the record of the
same patient
– Strength of the relationship  # records accessed in
common

Association Rules
– What is the probability that we see two users or two
departments interacting together?
– Head → Body
• Confidence - probability of seeing the Body, given the Head
• Support - probability of seeing the Head and the Body
11/11/2008
14
Association Rules
Geriatric
Psychology
1 patient
11/11/2008
Ob-Gyn
Neonatology
172 patients
15
Association Rules
Geriatric
Psychology
Ob-Gyn
Neonatology
1 patient
172 patients
Strong Relationship
11/11/2008
16
Association Rules
Geriatric
Psychology
Ob-Gyn
Neonatology
1 patient
172 patients
Weak Relationship
11/11/2008
17
HORNET
Healthcare Organization Relational Network
Extraction Toolkit
Open Source
Easy and informative tool for
privacy officials
Rich platform for developers
11/11/2008
18
Design Goals
Easily handle healthcare sized networks
– 103 to 104 nodes
– 106 to 107 edges
Easily configurable for users
Extendable by developers
Log format agnostic
11/11/2008
19
Plugins
HORNET Core
Network API
Graph, Node,
Edge, Network
Statistics
Task API
Parallel &
Distributed
Computation
File API
CSV
…
Database API
Oracle, MySQL,
Etc.
11/11/2008
File Network
Builder
Database Network
Builder
Noise Filtering
Network
Abstraction
Association Rule
Mining
Social Network
Analysis
Network
Visualization
…
20
Plugin Architecture
Plugin Chaining
– Plugins use Observer Pattern to notify each other
– Allows complex piping of results between plugins
– Chains defined in configuration file
11/11/2008
21
Plugin Configuration
File Network
Builder
Network
Abstraction
Social Network
Analysis
11/11/2008
Association Rule
Mining
Network
Visualization
22
Results from Vanderbilt
5 months of access logs from StarPanel,
Vanderbilt’s EMR
> 9000 users
> 350,000 patients
> 7,500,000 views
11/11/2008
23
Edge Distribution
• Distribution of Relationships per User in 1
week
1000
# Users
100
10
1
1
11/11/2008
10
100
Edges per User
1000
10000
24
Decay of Relationships
Fraction of Relationships
Remaining
How long do relationships last?
1
Department
User
0.8
0.6
0.4
0.2
0
0
5
10
# Weeks Relationship Existed
15
Healthcare is dynamic!
11/11/2008
25
Department Relationships
Relationships (edges) between departments
(nodes)
11/11/2008
26
Department Relationships
20 departments with most relationships labeled
11/11/2008
27
Association Rules
For 16 weeks, 55,944 department-department
rules (unfiltered)
11/11/2008
28
Association Rules
Sample of rules with high support
Head
Body
Emergency Medicine
Emergency Med-Housestaff
Emergency Med-HousestaffEmergency Medicine
Ob-Gyn
School Of Nursing
Orthopaedics & Rehab
Emergency Medicine
Emergency Medicine
Allergy/Pulm/Critical Care
Emergency Medicine
Nephrology & Hypertension
Emergency Medicine
Cardiovascular Medicine
Emergency Medicine
Anesthesiology
Nephrology Clinic
Nephrology & Hypertension
Hematology/Oncology
Cancer Center
11/11/2008
Confidence Support
1.8E-04
1.7E-03
7.2E-04
7.1E-04
8.3E-05
6.5E-05
6.3E-05
6.1E-05
1.1E-03
5.5E-04
# Weeks
0.0043
0.0043
0.0025
0.0020
0.0019
0.0015
0.0015
0.0014
0.0010
0.0009
16
16
16
16
16
16
16
16
16
16
29
Association Rules
Sample of rules with high confidence and
occurring at least 3 weeks
Head
Human & Organizational Dev
Psychology & Human Devel
Radiology-Housestaff
Counseling Center
Counseling Center
Counseling Center
NICU
Sedation Service
Sedation Service
Radiology-Housestaff
11/11/2008
Body
School Of Nursing
Mental Health Center
Orthopaedics & Rehab
Psychiatry
Psychology
Adult Psychiatry
Neonatology
Anesthesiology
Pediatric Critical Care
Emergency Medicine
Confidence
0.19
0.12
0.10
0.08
0.07
0.07
0.04
0.04
0.04
0.03
Support
# Weeks
8.9E-06
5.6E-06
3.9E-06
4.7E-06
4.4E-06
4.4E-06
8.8E-05
2.0E-06
6.1E-06
7.7E-06
4
5
6
6
6
6
14
6
4
4
30
Future Plans
Temporal relationships
– Find if certain users or departments are predictive of a
patient seeing another user or department
Filter Network
– Remove noise, keep important relationships
User interface
– Tool for privacy officers to examine their
organization’s logs
11/11/2008
31
Future Plans
Evaluation of rules by privacy and domain
experts
Integrate with MICIS access control system
– Werner et al., Barth et al., Mathe et al.
11/11/2008
32
Acknowledgements
NSF grant CCF-0424422, the Team for Research
in Ubiquitous Secure Technologies
Dr. Randolph Miller and Kathleen Benitez
Dr. Dario Giuse and David Staggs
NetworkX, Numpy, Cython, Matplotlib
11/11/2008
33
More Information
http://hiplab.mc.vanderbilt.edu/projects/hornet
[email protected]
11/11/2008
34
Appendix
11/11/2008
36
Developer Documentation
11/11/2008
37
Writing a Plugin
11/11/2008
38
Configuration File
11/11/2008
39
Care Provider Relationships
Children’s Hospital
11/11/2008
40