Transcript Open Source in the Cloud

November 12th, 2012
5:30 – 7:30pm
“Open Source in the
Cloud”
Sponsored by
Sonian, Open Source and Sensu
November 12, 2012
Sonian’s Contributions
 Fog
- https://github.com/fog
 Elasticsearch
 Openstack
 Opscode
 Various




- https://github.com/elasticsearch
Swift - https://github.com/openstack/swift
Chef – https://github.com/opscode/chef/
Chef Tools:
https://github.com/portertech/chef-metrics
https://github.com/portertech/chef-journalist
https://github.com/portertech/recognizer
https://github.com/portertech/chef-irc-snitch
Home Built and Released
 SCLI

(Smart Cloud Command Line interface)
https://github.com/sonian/scli (MIT)
 Amazon-Pricing

https://github.com/sonian/amazon-pricing (Ruby)
 ElasticSearch

Jetty Plugin
https://github.com/sonian/elasticsearch-jetty (Apache 2)
 Sensu

(Pricing Gem)
– Monitoring Framework
https://github.com/sensu (MIT)
Sensu – “The Monitoring Router”
Framework – Built for the cloud (Dynamic
Environments)
 Monitoring






Ruby (EventMachine, Sinatra, AMQP), RabbitMQ, Redis
Messaging oriented architecture. Messages are JSON
objects. (Pub/Sub)
Ability to re-use existing Nagios plugins
Plugins and handlers (think notifications) can be written in
any language
Designed with modern configuration management systems
such as Chef or Puppet in mind
Lightweight, less than 1200 lines of code
Why We Built It
 Highly




Elastic Infrastructure
Nodes are created (Spot Nodes)
Bootstrapped (With Chef)
Take and process work
Terminated (when prices increase)

All before they are discovered and monitored by Nagios

Nagios is:

Difficult to Extend
 Can not discover new services on its own
 Generally Unpleasant
Keep It Simple™
• The Idea:
• Schedule the execution of
remote checks
• Collect their Results
• “Checks” are:
• Is the server up?
• How hard is it working?
• Tied into Modern CM
• Chef
• Puppet
• Message Oriented Middleware
• RabbitMQ
• Securely Routing
Checks/Results
• Redis: Fast In-Mem K/V Store
Open Source = Community
• Early Development – Recruit Community Experts
• Help Test – Drive Early Roadmap
• Develop Puppet and Chef modules
• Release Day (Nov 1st 2011)
• Make Sensu Github Repo Public
• Open IRC channel on Freenode (#sensu)
• Blog posting and Twitter for marketing
• Community, Community, Community
• Adoption – Documentation
• “Omnibus” Style Packaging for Quick Deployments
Contact
 Pete
Cheslock
 Director
of Technical & Cloud Operations @ Sonian
 @petecheslock
 http://about.me/petecheslock
 We’re
Hiring!

Please contact Sonian’s VP of Product Development Glenn
Snyder

[email protected]
Outline
• Level Set on Current Cloud Trends, pain points, and markets
• Open Source Trends
• OpenStack Introduction
– What it is
– Historical Evolution
– Use Cases
– Conceptual Architecture
– Customer Adoption
– Cloud Taxonomy
11
Global Marketing
Level Set On Cloud
trends
12
Global Marketing
Cloud is about Services, Not Systems: Consumer
Market Driving Trends
13
24/7 CLOUD
SERVICES
Interconnected
mobile devices
Anywhere,
Anytime,
Internet Access
Global Marketing
Enterprises: Same Cloud Services Goals But Different Market Segments
and Pain Points
•
•
•
•
•
•
Burdened by “legacy” infrastructure and
applications
Too complex to manage
Expensive to implement and maintain
Takes too long to realize value
Not easy to innovate
Security, compliance, and privacy concerns
Mega Datacenters
Datacenter
 Mega
– Homogeneous
– Dense/Low Cost
 Small
Small Site
Site (SMB/Branch)
(SMB/Branch)
– Low Cost
– Ease of deployment
– Ease of management
Enterprise Datacenter
Datacenter
 Enterprise
– Heterogeneous
– Legacy / Complex
– Consolidating Quickly
• Source: IDC
14
Global Marketing
Virtual Machines Already Outstripping Physical Machines
“2012: RATE(of VMs launched
per sec 6/sec) > RATE (at
which babies are born in the
US per sec 8/sec) (from
#VMW) #cloud”
Global Marketing
But what about storage and networks?
Extensive
Use of
SAN
Complex
Networks
Dense
Blades
>$2000 /
VM
Specialized
Skillset
Fixed
Capacity
Global Marketing
Virtualization 1.0: Not up to the challenge
HOST 1
HOST 2
HOST 3 HOST 4, ETC.
VM
Hypervisor
(VMWare ESX, Citrix XEN Server, KVM, Etc.)
Hardware abstraction for each server
Better resource utilization for each server
1. Server Virtualization
Automation & Efficiency
2. Cloud Data Center
3. Cloud Federation
Global Marketing
But questions arise as the environment grows...
“VM SPRAWL” CAN MAKE THINGS UNMANAGEABLE VERY QUICKLY
USERS
APPS
ADMINS
How do you make your apps cloud aware?
How do you empower people to self-service?
Where should you provision new VMs?
How do you keep track of it all?
+
1. Server
Server Virtualization
Virtualization
Automation & Efficiency
2. Cloud Data Center
3. Cloud Federation
Global Marketing
But questions arise as the environment grows...
“VM SPRAWL” CAN MAKE THINGS UNMANAGEABLE VERY QUICKLY
USERS
APPS
ADMINS
A Cloud Management Layer Is Missing
1. Server
Server Virtualization
Virtualization
Automation & Efficiency
2. Cloud Data Center
3. Cloud Federation
Global Marketing
What is needed is a cloud Operating System that
adds automation and control at scale
Connects to apps via APIs
Self-service for users
USERS
ADMINS
APPS
CLOUD OPERATING SYSTEM
Creates Pools of Resources
1. Server
Server Virtualization
Virtualization
Automation & Efficiency
Automates The Network
2. Cloud Data Center
3. Cloud Federation
Global Marketing
Types of pools managed by the Cloud O.S.
COMPUTE, NETWORK, & STORAGE
Compute Pool
Network Pool
Storage Pool
1. Server
Server Virtualization
Virtualization
Automation & Efficiency
2. Cloud Data Center
Load Balancing Pool
Image Service Pool
3. Cloud Federation
Global Marketing
Assembly Line IT
Global Marketing
Robotics Factory IT
Global Marketing
Open Source, OpenStack
Evolution and Trends
24
Global Marketing
Innovation + Open Source = Virtuous Cycle
Open Source Innovation
600,000+ OSS projects
100+ billion lines of code
10 million person-years of work
Source: Blackduck
• Open source is leading, not following, in important areas
including cloud, big data, mobile apps and enterprise mobility.
• More than 50 percent of software acquired in the next five
years will be open source software.
• Innovation, flexibility, cost, quality of open source are some of
the top reasons that make it attractive for use
• Companies most likely to be impacted by OSS have these
characteristics: Business drivers to invest in IT, software
development is an essential strategic process, technology centric
• When asked about revenue generating strategies likely to create
value for vendors, 52 percent of respondents said an annual,
repeatable support and service agreement was the most likely
value strategy
• http://northbridge.com/2012-open-source-survey
Global Marketing
OpenStack is the Open Source Software Powering
Public and Private Clouds
Private Cloud:
Run OpenStack software
in your own
data centers
1. Server Virtualization
Automation & Efficiency
Public Cloud:
OpenStack powers some
of the worlds largest public
cloud deployments.
2. Cloud Data Center
3. Cloud Federation
Global Marketing
• What is it: An open source cloud operating system
– 30,000 lines of code to 600,000 in under 18 months
• Who’s building it: A worldwide community of developers
– 600+ developers, 250+ contributed in the last 12 months
– 6000+ individual members in 87 countries
• Who is govering it: The OpenStack Foundation, backed by AT&T, Canonical,
Cisco, ClearPath, Cloudscaling, Dell, DreamHost, HP, IBM, ITRI, Mirantis,
Morphlabs, Nebula, NetApp, Piston, Rackspace, Red Hat, SUSE, and Yahoo!
(so far)
Global Marketing
Wide-Ranging Community Support….
28
Confidential
4/13/2015
Global Marketing
OpenStack Timeline
Bexar
Austin
First Public
Code
Formation
Folsom
Essex
“Platform for Innovation”
Diablo
“Production Ready”
Core Platform for Innovation
Cactus
Workable Foundation
Stable Foundation
Network as a Service
Block Storage
Community
Development
Forming
Exposes Gaps
Solidify Community
Included in Ubuntu 12.04
Loses VMware &
HyperV
Working
Prototypes
Incubated: Network & Block
Storage
2012
2011
Nov 2010
Dec
Nov 2010:
Austin
Release
Oct 2010:
Design
Summit
Public Adoptoin
Multiple Scale Deployments
Feb
Feb 2011:
Bexar
Release
Apr
Apr 2011:
Cactus
Release
Apr 2011:
Design
Summit
Jun
Oct
Aug
Dec
Feb
Apr
Jun
Mar 2012:
Essex
Release
Sep 2011:
Diablo
Release
Oct 2011:
Design
Summit
Aug
Oct 2012:
Folsom
Release
Apr 2012:
Design
Summit
Global Marketing
OpenStack Value Proposition
Reduces
• Addresses Real Market Pains
Provides
Lock-in & Open APIs
Licenses & Support
– Limits costly software licenses
– Limits lock-in by vendors (VMware) & by providers (Amazon)
– Allows for massive scalability
– Extensible hypervisor support (Xen, KVM, Hyper-V, etc.)
– Offers standard APIs enabling growing cloud ecosystem
OpenStack is commoditizing the IaaS market from single provider (Amazon) to
many small copy cats (startups).
Global Marketing
Popular OpenStack Use Cases
• Service providers offering an IaaS compute platform
• IT departments provisioning compute resources to teams and
projects
• Processing big data with tools like Hadoop
• Scaling compute up and down to meet demand for web resources
and applications
31
Global Marketing
OpenStack Conceptual Architecture
32
Confidential
4/13/2015
Global Marketing
There is a broad adoption of OpenStack across many
markets. The most common markets we have seen so
far have been:
Universities
Healthcare
CDN providers
Hosting providers
Start-ups
SaaS companies
Government
33
Confidential
Global Marketing
Vision for Complete
OpenStack Solution
Global Marketing
Complete Cloud Taxonomy
Software as a Service
IT as a Service
Infrastructure as a Service
Platform as a Service
Everything as a Service
Admin Software
SSO
Web Services & APIs
Customer Management
Entitlement, rights
Billing
Metering
Legacy Management
Information Service Management
Infrastructure Software
LDAP/AD
Security
Reporting
Ser Gov/Workflow Automation
Monitoring
Workload Lifecycle Management
Orchestration
Intelligent Resource Manager
Platform Provisioning
Abstraction Software
OS
Data Store
Operating System Virtualization
Application Run-Time Virtualization
IPS
Hardware Virtualization
Compute
Firewall
Analytics
Self Service Portal
Overarching Systems
Physical
Switch
Storage
Network
HVAC
Power
Environmentals
Facility
Global Marketing
Disruption
Global Marketing
Discussion
Global Marketing