Transcript Globus Virtual Workspaces

Cloud Computing with Nimbus
April 2009
Kate Keahey
([email protected])
University of Chicago
Argonne National Laboratory
Cloud Computing
SaaS
Software-as-a-Service
elasticity
computing on demand
PaaS
Platform-as-a-Service
capital expense
operational expense
IaaS
Infrastructure-as-a-Service
7/17/2015
The Nimbus Toolkit: http//workspace.globus.org
Cloud Computing for Science
7/17/2015

Environment

Resource control
The Nimbus Toolkit: http//workspace.globus.org
“Workspaces”


Dynamically provisioned environments

Environment control

Resource control
Implementations


7/17/2015
Via leasing hardware platforms: reimaging,
configuration management, dynamic
Isolation
accounts…
Via virtualization: VM deployment
The Nimbus Toolkit: http//workspace.globus.org
A Brief History of Nimbus
First STAR
production
run on EC2
Xen released
2003
Research on
agreement-based
services
EC2 goes online
Nimbus Cloud
comes online
2006
First WSRF
Workspace Service
release
2009
EC2 gateway
available
Support for
EC2 interfaces
Context Broker
release
7/17/2015
The Nimbus Toolkit: http//workspace.globus.org
Nimbus Goals

Allow providers to build clouds
 Private clouds (privacy, expense considerations)


Allow users to use cloud computing
 Do whatever it takes to enable scientists to use
IaaS



Workspace Service: open source EC2 implementation
Context Broker: turnkey virtual clusters
IaaS Gateway: interoperability
Allow developers to experiment with Nimbus
 For research or usability/performance
improvements
 Community extensions and contributions
7/17/2015
The Nimbus Toolkit: http//workspace.globus.org
The Workspace Service
VWS
Service
7/17/2015
Pool
node
Pool
node
Pool
node
Pool
node
Pool
node
Pool
node
Pool
node
Pool
node
Pool
node
Pool
node
Pool
node
Pool
node
The Nimbus Toolkit: http//workspace.globus.org
The Workspace Service
The workspace service publishes
information about each workspace
VWS
Service
Users can find out
information about their
workspace (e.g. what IP
the workspace was
bound to)
Pool
node
Pool
node
Pool
node
Pool
node
Pool
node
Pool
node
Pool
node
Pool
node
Pool
node
Pool
node
Pool
node
Pool
node
Users can interact
directly with their
workspaces the same
way the would with a
physical machine.
7/17/2015
The Nimbus Toolkit: http//workspace.globus.org
Workspace Service:
Interfaces and Clients


Web Services based
Web Service Resource Framework (WSRF)


WS + state management (WS-Notification)
Elastic Computing Cloud (EC2)



7/17/2015
Compatible with EC2 clients
Supported: ec2-describe-images, ec2-run-instances, ec2describe-instances, ec2-terminate-instances, ec2-rebootinstances, ec2-add-keypair, ec2-delete-keypair
Unsupported: availability zones, security groups, elastic IP
assignment, REST
The Nimbus Toolkit: http//workspace.globus.org
Workspace Service: Security


GSI authentication and authorization

PKI-based

VOMS, Shibboleth (via GridShib), custom PDPs
Secure access to VMs


EC2 key generation or accessed from .ssh
Validating images and image data


7/17/2015
Extensions from Vienna University of Technology
Paper: Descher et al., Retaining Data Control in
Infrastructure Clouds, ARES (the International
Dependability Conference), 2009.
The Nimbus Toolkit: http//workspace.globus.org
Workspace Service: Networking

Network configuration



External: public IPs or private IPs (via VPN)
Internal: private network via a local cluster
network
Each VM can specify multiple NICs mixing
private and public networks (WSRF only)

7/17/2015
E.g., cluster worker nodes on a private
network, headnode on both public and
private network
The Nimbus Toolkit: http//workspace.globus.org
EC2
WSRF
Workspace Components
workspace
service
workspace
resource
manager
workspace
control
workspace
pilot
workspace
client
OpenNebula
Project
See papers at: http://workspace.globus.org/papers/index.html
• “Simple Leases with Workspace Pilot” (EuroPar08)
• “Combining Batch Execution and Leasing Using
Virtual Machines” (HPDC08),
7/17/2015
The Nimbus Toolkit: http//workspace.globus.org
Cloud Capabilities
EC2
WSRF
storage
service
workspace
service
cloud
client
7/17/2015
workspace
resource
manager
workspace
control
workspace
pilot
workspace
client
The Nimbus Toolkit: http//workspace.globus.org
The IaaS Gateway
EC2
WSRF
storage
service
workspace
service
7/17/2015
workspace
control
workspace
pilot
IaaS
gateway
cloud
client
workspace
resource
manager
EC2
potentially other providers
workspace
client
The Nimbus Toolkit: http//workspace.globus.org
Cloud Computing Ecosystem
Appliance Providers
Marketplaces, commercial providers,
Virtual Organizations
Appliance management software
VMM/DataCenter/IaaS
User Environments
7/17/2015
Deployment
Orchestrator
VMM/DataCenter/IaaS
User Environments
The Nimbus Toolkit: http//workspace.globus.org
Turnkey Virtual Clusters
IP1
HK1
IP2
IP1
HK1
IP1
IP2
HK2
IP2
IP3
HK3
IP3
HK2
HK1
HK1
HK2
IP2
HK2
HK3
IP3
HK3
MPI
Turnkey, tightly-coupled cluster


7/17/2015
HK3
IP1
Context Broker

IP3
Shared trust/security context
Shared configuration/context information
The Nimbus Toolkit: http//workspace.globus.org
Context Broker Goals

Can work with every appliance


Can work with every cloud provider


Appliance schema, can be implemented in
terms of many configuration systems
Simple and minimal conditions on generic
context delivery
Can work across multiple cloud providers,
in a distributed environment
7/17/2015
The Nimbus Toolkit: http//workspace.globus.org
Context Broker Status

Releases




In alpha since 08/07, first release 06/08, update
01/09
Used to contextualize cluster composed of 100s of
virtual nodes for multiple production apps
Contextualized images on workspace marketplace
Working with rPath to make contextualization
easier for the user

Discussing OVF extensions
Paper: Keahey&Freeman, Contextualization: Providing
One-Click Virtual Clusters, eScience 2008
7/17/2015
The Nimbus Toolkit: http//workspace.globus.org
End of Nimbus Tour
EC2
context broker
WSRF
storage
service
workspace
control
workspace
pilot
IaaS
gateway
context
client
7/17/2015
workspace
service
workspace
resource
manager
cloud
client
EC2
potentially other providers
workspace
client
The Nimbus Toolkit: http//workspace.globus.org
Science Clouds


Goals

Enable scientific projects to experiment with IaaS clouds

Evolve software in response to the needs of scientific projects

A laboratory for exploration of cloud interoperability issues
Participants



University of Chicago (since 03/08, 16 nodes), University of
Florida (05/08, 16-32 nodes, access via VPN), Masaryk University,
Brno, Czech Republic (08/08), Wispy @ Purdue (09/08)
In progress: Grid5K, Vrije, others
Using EC2 for large runs

Simple governance model, access given to any scientific project

http://workspace.globus.org/clouds
7/17/2015
The Nimbus Toolkit: http//workspace.globus.org
Who Runs on Nimbus at UC?
Hadoop
AliEn
GT-scalability
STAR
Montage workflows
GridFTP testing
workspace-team
Testing
OSG
geofest
bioinformatics
Other
100+ DNs
projects ranging across Science, CS, education, build&test…
7/17/2015
The Nimbus Toolkit: http//workspace.globus.org
STAR



STAR: a nuclear physics experiment studies
fundamental properties of nuclear matter
Computations require complex and consistently
configured environments
Requirements



A virtual OSG STAR cluster: OSG headnode (gridmapfiles,
host certificates, NFS, Torque), worker nodes: SL4 + STAR

From Science Clouds to EC2 runs

One-click virtual cluster deployment: Context Broker
Producing just-in-time results for Quark Matter
conference: http://www.isgtw.org/?pid=1001735
Work by Jerome Lauret, Doug Olson, Leve Hajdu, Lidia
Didenko at BNL
7/17/2015
The Nimbus Toolkit: http//workspace.globus.org
Alice HEP Experiment at CERN


Collaboration with CERNVM project
HPCwire article
7/17/2015
The Nimbus Toolkit: http//workspace.globus.org
Sky Computing
U of Florida
U of Chicago
ViNE
router
ViNE
router
ViNE
router
Purdue
7/17/2015
The Nimbus Toolkit: http//workspace.globus.org
Sky Computing
U of Chicago
U of Florida
Hadoop cloud
Purdue

Papers:


7/17/2015
“Sky Computing”, by K. Keahey, A. Matsunaga, M. Tsugawa, J.
Fortes. Submitted to IEEE Internet Computing.
“CloudBLAST: Combining MapReduce and Virtualization on
Distributed Resources for Bioinformatics Applications” by A.
Matsunaga, M. Tsugawa and J. Fortes. eScience 2008.
The Nimbus Toolkit: http//workspace.globus.org
IaaS Clouds vs Grids

Grid computing




Cloud computing





Assumption: site retains control over resources
Remote interfaces to local site mechanisms
Tradeoff: difficult to provide the right environments and
control but easy to deploy
Assumption: a user gets a “lease” on a remote resource that it
gets to control
Enabled by virtauliaztion (Xen)
Tradeoff: eanbles a larger class of applications but hard to
deploy
Raises issues: e.g., site licenses? Configuration support?
Towards “sky computing”



7/17/2015
I can now trust a remote resource: I configured it myself
Cloud computing + virtual networks
Local distributed environment
The Nimbus Toolkit: http//workspace.globus.org
Friends and Family



Committers: Kate Keahey & Tim Freeman (ANL/UC), Ian
Gable (UVIC)
A lot of help from the community, see:
http://workspace.globus.org/people.html
Collaborations:





7/17/2015
Cumulus: S3 implementation (Globus team)
EBS: IU project
Appliance management: rPath, Bcfg2 project, CohesiveFT
Virtual network overlays: University of Florida
Security (research): Vienna University of Technology
The Nimbus Toolkit: http//workspace.globus.org
Open Source IaaS
Implementations

OpenNebula



Eucalyptus



Open source implementation of EC2
UCSB, R. Wolski & team, 06/2008
Cloud-enabled Nimrod-G



Open source datacenter implementation
University of Madrid, I. Llorente & team, 03/2008
Open source implementation of EC2
Monash University, MeSsAGE Lab, 01/2009
Industry efforts

7/17/2015
openQRM, Enomalism
The Nimbus Toolkit: http//workspace.globus.org
Parting Thoughts


Science-driven cloud computing
Importance of open source



Cloud computing for the user



Drive requirements into the infrastructure, customize
Drive the development of standards
Combine with what we have (grid computing)
Explore new potential
Future directions



7/17/2015
Creating the ecosystem, working out the issues, e.g.
licensing, appliance support
Interoperability and standards
Service Levels
The Nimbus Toolkit: http//workspace.globus.org