Transcript SODA: a Service-On-Demand Architecture - FRIENDS Lab

SODA:
Service-On-Demand Architecture for
Application Service Hosting Utility Platforms
Dongyan Xu, Xuxian Jiang
Lab FRIENDS
(For Research In Emerging Network and Distributed Services)
Department of Computer Sciences
Center for Education and Research in Information Assurance and
Security (CERIAS)
Purdue University
Outline




Motivations and goals
Related work
Research components of SODA
Summary and on-going work
Motivations
 Vision of utility computing
 Computation utility
 Storage utility
 Application service hosting





Conference management
e-Campaign
Digital government
Serving the underserved communities
IT function shadowing for disaster recovery
 Virtual enterprise, collaboratory, and community
Our Goal
 To build a value-added application service
hosting platform based on shared
infrastructure, achieving:






On-demand creation and provisioning
Virtualization
Isolation
Protection
Accountability
Privacy
Related Work
 Utility computing architectures
 VERITAS, HP UDC, IBM Oceano
 Grid platforms
 Computation: Globus, Condor, Legion, NetSolve,
Harness, Cactus
 Storage and data: SRB, NeST, Data Grid, OceanStore
 Shared infrastructure
 PlanetLab, Emulab
 Active services
 Active Service Grid, Berkeley Active Service Framework,
CANS (NYU), Darwin, WebOS
Related Work
 Resource isolation
 GARA, QLinux (UMass), Virtual service (UMich),
Resource Container, Cluster Reserves (Rice)
 Virtualization technologies
 Virtual super computer (aggregation): NOW, HPVM
 Virtual OS, isolation kernel (slicing): VMWare, Xen
(Cambridge), Denali (UW), UML, UMLinux, Virtual
Private Server (Ensim)
 Grid computing on VM: Virtuoso (Northwestern),
Entropia
 Virtual cluster: Cluster-on-Demand (Duke)
SODA
 Service-On-Demand Architecture for
application service hosting utility platforms
 Research components of SODA




General architecture
Protection, intrusion detection, logging
Confined and VM-based overlay
Market-driven planning and management
Outline
 Research components of SODA:




General architecture
Security and protection
Confined VM-based overlay
‘Property’ planning and management
Detailed Information

Xuxian Jiang, Dongyan Xu, "SODA: a Service-On-Demand Architecture
for Application Service Hosting Utility Platforms", Proceedings of The
12th IEEE International Symposium on High Performance Distributed
Computing (HPDC-12), Seattle, WA, June 2003.
Overview of SODA
AS
Virtual service node
AS’
SODA Host
(physical)
Virtualization: Key Technique
 Two-level OS structure
 Host OS
 Guest OS
 Strong isolation




Administration isolation
Installation isolation
Fault / attack Isolation
Recovery, migration, and
forensics
 Virtual service node
 Application service (AS)
 Guest OS
 Internetworking enabled
AS1
ASn
…
Guest OS
Guest OS
Host OS
One SODA host
Service Requests From Clients
Service Requests From Clients
Service Switch for S
Virtual
service
node
Service S
Guest OS
SODA
Daemon
Host OS
SODA Master
Service Switch for S’
Service S
Service S’
Guest OS
Guest OS
SODA
Daemon
Service S’
Guest OS
Host OS
SODA
Daemon
Host OS
SODA Agent
Service Creation Requests From ASP
On the Same SODA Host
WWW service
Honeypot
Host OS and Guest OS
 Guest OS: based on User-Mode Linux
(UML), an open-source virtual OS (different
from UMLinux and VServer )




By Jeff Dike, http://user-mode-linux.sourceforge.net
Running in user space of host OS
Separate kernel address space
Physical memory usage limit
 Host OS: Linux (linux-2.4.19, enhanced)
 CPU fair share scheduler (for CPU isolation
between virtual service nodes)
Experiment: CPU Isolation
VM1: CPU-intensive
Original Linux Scheduler
VM2: IO-intensive VM3: Web
Enhanced Linux Scheduler
On-Demand Service Priming




Performed by SODA Daemon
Customization of guest OS (“cook to order” )
Active service image downloading
Automatic bootstrapping of virtual service
node
Service Bootstrapping Time
Linux
Configuration
Image size
Time
(seattle)
Time
(tacoma)
Rootfs_tomrtbt_
1.7.205
15 MB
2.0 sec.
3.0 sec.
Rootfs_base_1.0
29.3 MB
3.0 sec.
4.0 sec.
Root_fs_lfs_4.0
400 MB
4.0 sec.
16.0 sec.
Root_fs.rh-7.2server.pristine.200
21012
253 MB
22.0 sec.
42.0 sec.
Slow-Down (w/o optimization)
System call level
(clock cycles)
System call
UML
Linux
getpid
27,276
1,208
geteuid
26,648
1,064
dup2
26,904
1,084
mmap
27,864
1,208
munmap
27,044
1,200
gettimeofday
37,004
1,368
Application level
Outline
 Research components of SODA:




General architecture
Security and protection
Confined VM-based overlay
‘Property’ planning and management
Detailed Information


Xuxian Jiang, Dongyan Xu, Rudolf Eigenmann, "Protection Mechanisms
for Application Service Hosting Platforms", Proceedings of IEEE/ACM Int'l
Symposium on Cluster Computing and the Grid (CCGrid 2004), Chicago,
IL, April 2004.
Xuxian Jiang, Dongyan Xu, "Collapsar: A VM-Based Architecture for
Network Attack Detention Center", to appear in Proceedings of the 13th
USENIX Security Symposium (Security '04), San Diego, CA, August 2004.
Security and Protection
 Virtual switching and
firewalling
 IDS in guest OS
kernel
 Untamperable logging
(‘blackbox’-ing)
AS1
ASn
…
Guest OS
Host OS
Guest OS
Virtual Switching and Firewalling
Virtual
machine
(with IP
addr.)
Guest OS
Host OS
SODA host
(Invisible on
Internet)
Guest OS
Guest OS
Firewall
Kernort: IDS in Guest OS Kernel
 Problems with traditional IDS
 Encrypted traffic (e.g. ssh) makes NIDS less effective
 App-level IDS process will be “killed”, once a machine is
compromised
 Log may be tampered with
 Fail-open
 Related projects
 Backtracker (Michigan)
 VMM-based retrospection (Stanford)
 Forensix (OHSU)
 ESP (Purdue CERIAS)
 Open-source projects: Snort, Saint Jude
Kernort
 VM-based IDS
 Deployed in each VM
 Inside guest OS kernel: a unique vista point





Customizable without affecting host OS
Clearer view
Untamperable logging (saved to SODA host)
Renewable signature (read from SODA host)
Fail-close instead of fail-open
Kernort: IDS in Guest OS Kernel
Guest OS
Guest OS
Kernort
Components
 Kernort sensor
 Event-driven (system call and packet reception)
 Renewable signature set
 Matching against a small signature set (“Top 20 most
wanted”)
 Kernort blackbox
 Untamperable logging
 Privacy preservation of ASes
 Analyzer
 Exhaustive signature matching
 Detection of complex attack patterns
 Session replay
Kernort
Virtual
machine
Host OS
Kernort (shaded areas: logs)
Real-Time Alert
Session
Re-play
Impact on Performance
Impact on Performance
Outline
 Research components of SODA:




General architecture
Security and protection
Confined VM-based overlay
‘Property’ planning and management
Detailed Information

Xuxian Jiang, Dongyan Xu, "vBET: a VM-Based Emulation Testbed",
Proceedings of ACM Workshop on Models, Methods and Tools for
Reproducible Network Research (MoMeTools, in conjunction with ACM
SIGCOMM 2003), Karlsruhe, Germany, August 2003.


Xuxian Jiang, Dongyan Xu, "VIOLIN: Virtual Internetworking on OverLay
INfrastructure", Department of Computer Sciences Technical Report
CSD TR 03-027, Purdue University, July 2003.
Xuxian Jiang, Dongyan Xu, “A Middleware Architecture for Confined
Virtual Machine Overlays", in preparation, March 2004.
Traditional Overlay Network
 Problems with traditional overlays:
 Open for attacks
 Attacks from the outside (i.e. Internet) against overlay
nodes
 Attacks from an overlay node against the outside
 Difficult to manage
 An overlay across multiple administration domains
 A host participate in multiple overlays
 Difficult to enforce overlay topology and traffic volume
 VPN does not solve the problems
Traditional Overlay Network
Firewall
Firewall
Firewall
VM-based Overlay
 The case for VM-based overlay
 Multiple overlays on shared infrastructure
 On-demand creation
 Confinement and isolation
 VM introduces new network administration complexity
 “What is this new machine that has suddenly appeared in my
domain?”
 “Where is the machine that was in my domain yesterday?”
 “How much network connectivity should a VM have?”
 “How many IP addresses for VMs?”
Confined VM-based Overlay
 In addition to VM, we need VN for VMs
 VN: a highly overloaded term (VPN, X-bone…)
 What is new: Confined and VM-based overlays
 Applications
 Multi-institutional collaborations
 Philanthropic (volunteer) computing systems
 Network emulations
Confined VM-based Overlay
VM
VM
VM
≤2Mbps
≤2Mbps
Virtual
infrastructure
≤1Mbps
Firewall
Firewall
Firewall
Key Properties
 Confined overlay topology and traffic
 No attack possible from inside the overlay
to the outside world
 Virtual IP address space
 No need for application modification and
re-compilation
A More Generic Picture
VIOLIN:
Virtual
Internetworking
on OverLay
INfrastructure
vBET: an Example of Confined Overlays on Demand
 An education tool for network and distributed
system emulation
 Fidelity-preserving setup
 Maneuverable network entities
 Real-world network software
 Strict confinement (network security experiment)
 Flexible configuration
 Not constrained by device/port availability
 No manual cable re-wiring or hardware setup
 Simultaneous experiments
 Cost-effective
vBET Features
Can be deployed in n ≥ 1 vBET servers
Efficient startup and tear-down of emulated
entities
Strong network virtualization
IP address space
Virtual routers, switches, firewalls, end-hosts, links
Communications confined by virtual topology
Dynamic addition, deletion, migration,
configuration of network entities
vBET GUI
Sample Emulation: OSPF Routing
Emulation of OSPF Routing
Demo video clip at:
http://www.cs.purdue.edu/~jiangx/vBET/videos/vbet_ospf.avi
Sample Emulation: Distributed Firewalls
Screenshot
Sample Emulation: Chord P2P Network
Screenshot
Outline
 Research components of SODA:




General architecture
Security and protection
Confined VM-based overlay
‘Property’ planning and management
Property Planning and Management
 Tenant selection:
 Among a set of potential tenants (ASes), which
ones to host? (for maximum revenue, resource
utilization, security…)
 SODA provider selection:
 Among a set of SODA providers, which one should
be chosen to host an AS?
Property Planning and Management
 Examples of bad planning:
 Many PDA transcoding ASes in an area with a
small PDA user population
 AS not requiring client registration and log-in
(potential DDoS attacks)
 Majority of ASes exhibiting similar demand
characteristics such as:
Load
Time
Property Planning and Management
 AS profiling
 Resource requirement
 Security/authentication
 Demand characteristics
 Market analysis
 Competing ASes, market size/growth/expected
share
 ASes correlation (“80% of clients requesting AS X
also request AS Y” )
 Trading/pricing of SODA machine slices
Property Planning and Management
 Forming alliance of SODA providers
Property Planning and Management
 Forming alliance of SODA providers
Summary
 Virtualization: a key enabling technology in
realizing utility computing vision
 Hosting utility is more complex than
computation utility (host – tenants – clients)
 SODA achieves:




On-demand service creation
Service virtualization, isolation and confinement
Protection, accountability, privacy
Overlay isolation and confinement
Ongoing Work
 VM/service migration, shadowing, recovery
 Service profiling, accounting, auditing
(resources, security)
 Market-driven planning, provisioning, and
management (SODA ecology)
 Deployment and evaluation (Purdue
Bindley Bioscience Center)
Thank you.
For more information:
{dxu, jiangx}@cs.purdue.edu
http://www.cs.purdue.edu/~dxu
AOL keywords “Purdue SODA Friends”