Transcript BoxGrinder and OpenStack

Abdullah&Hiatham 05/02/2011
1
Outline of the Talk
• Introduction
• History of BoxGrinder
• How does Boxgrinder work ?
• Installation and Configuration of BoxGrinder
• BoxGrinder of Plugins
• OpenStack
• References
Abdullah&Hiatham 05/02/2011
2
Introduction
• BoxGrinder is an open source suite for building virtual appliances
(also known as virtual machines) that are ready to run on various
virtualization and Cloud providers or another definition that it is a set
of tools used for building appliances (virtual machines) on various
platforms (KVM, Xen, VMware, EC2).
• It supports many platforms such as EC2, Xen, KVM and VMware.
• BoxGrinder is able to create appliances based on Fedora, Red Hat
Enterprise Linux or CentOS, and is easy extendable to meet any
requirements.
Abdullah&Hiatham 05/02/2011
3
Introduction (Cont.)
• KVM (for Kernel-based Virtual Machine) is a full virtualization
solution for Linux on x86 hardware containing virtualization
extensions (Intel VT or AMD-V).
• Xen is a virtual-machine monitor for IA-32, x8664, Itanium and ARM architectures. It allows several guest operating
systems to execute on the same computer hardware concurrently
• VMware's desktop software runs on Microsoft Windows, Linux,
and Mac OS X.
• Appliance definition files are written in YAML
Abdullah&Hiatham 05/02/2011
4
History
• History:
- Who:
-Marek Goldmann
- When:
-BoxGrinder Build 0.6.5 in 04 December 2010.
-BoxGrinder Build 0.7.0 in 19 December 2010
-BoxGrinder Build 0.7.1 in 03 January 2011
-BoxGrinder Build 0.8.0 in 25 January 2011
-BoxGrinder Build 0.8.1 released 22 February 2011
-BoxGrinder Build 0.9.0 Released 09 march 2011
-BoxGrinder Build 0.9.1 in 28 April 2011
Abdullah&Hiatham 05/02/2011
5
How does BoxGrinder work?
• BoxGrinder is creating appliances (images/virtual machines) from
simple plain text Appliance Definition Files.
• There are only two simple steps to create an appliance:
1-Create Appliance Definition File
2-Run BoxGrinder
Abdullah&Hiatham 05/02/2011
6
How does BoxGrinder work? (Cont.)
• BoxGrinder consists of three projects:
1- BoxGrinder Build - a fast and easy to use command line
tool that is responsible for actually building the appliances.
2- BoxGrinder REST - a server with farm of builder nodes.
Build tasks are distributed to nodes where BoxGrinder
Build is used to build the appliance.The resulting appliance is
transferred to the configured destination.
3- BoxGrinder Studio - a web front-end to BoxGrinder
REST designed to provide a friendly graphical user
experience on top of BoxGrinder REST.(Still working on it)
Abdullah&Hiatham 05/02/2011
7
Installation and Configuration of BoxGrinder
• There are three phases to create boxgrinder:
1- Preparing environment to start building appliances.
- Fedora 13 and 14 doesn’t require preparing anything before
installing
BoxGrinder – use BoxGrinder RPMs – this will pull all
required packages.
2- Installation.
- Before you install BoxGrinder make sure you have prepared your
environment or use meta appliance
3- Building appliances
- Before you start make sure you have installed BoxGrinder Build
correctly.
Abdullah&Hiatham 05/02/2011
8
Installation
• The easiest way to obtain BoxGrinder Build is to simply
add
BoxGrinder repo to your environment and use yum.
- You have to execute this command to install BoxGrinder
in
Fedora.
- “ yum install rubygem-boxgrinder-build” Command
- Other Systems use this command to install BoxGrinder on it.
- “gem install boxgrinder-build “ Command
Abdullah&Hiatham 05/02/2011
9
Building appliances
• Create appliances directory such as “ mkdir appliances “ command.
- The creating is option.
name (required)
This is a name for your appliance. It must be
• Simple Example:
unique in all of your appliances
summary
A small (one sentence or such) summary of the
appliance; purpose, goals, etc.
os
This section specifies operating system information for the appliance.
Members: name – OS name, default: fedora, version – OS version; this could be
a number or string: 1, rawhide, default: 12. password – this is the root password,
default: boxgrinder.
hardware
This section gives the ability to specify virtual hardware requirements.
Members: cpus – virtual CPU count, default: 1, memory – memory
quantity in MB, default: 256,
partitions – partitioning scheme with root mounts and size in GB
Repos: you can specify additional repositories
in order to add packages to your appliance that
are sourced from these repositories.
packages
This section contains all of the packages you
wish to add to your appliance.
Abdullah&Hiatham 05/02/2011
10
Boxgrinder Plugins
• BoxGrinder Build has a plugin architecture. We can distinguish three
types:
- Operating System plugins – generating base appliance for
selected OS.
- Platform plugins – creating converted base appliance for
selected platform.
- Delivery plugins – designed to deliver your new appliance to a
specfied location. For example as a tar file to a remote server or just
reqister as AMI on EC2.
Abdullah&Hiatham 05/02/2011
11
BoxGrinder of Plugins ( Operating System
plugins)
Operating system plugins:
The goal of this kind of plugin is to create a base image for the
selected operating system. Each plugin must inherit the
BaseOperatingSystemPlugin class:
Supported Operating Systems:

Fedora plugin

CentOS plugin

RHEL plugin
Fedora Operating System Plugin Configuration:
The configuration of the plugin would have the following three
declarations at the begigning of the file:
Plugins:
# Plugin name.
fedora:
format:
# OS supported.
# Disk format to use. Default: raw.
Abdullah&Hiatham 05/02/2011
12
Operating System plugins(Cont.)
Example of Fedora-14.appl plugin file:
name:
fedora-14
# Name of Plugin.
os:
name: fedora
# Name of OS.
version: 14
# OS Version.
Fedora Operating System Plugin Usage:
In order to run the Fedora Plugin, the following command has to
be used;
boxgrinder-build fedora-14.appl
Abdullah&Hiatham 05/02/2011
13
Platform plugins (Cont.)
Platform plugins convert intermediary deliverables produced by the
operating system plugin into a selected platform. A platform could be
VMware vSphere or Amazon EC2 for example.
There are three kinds of such plugin;
VMware plugin
VirtualBox Plugin
EC2 plugin
EC2 Platform Plugin:
This plugin creates a EC2 disk image. Note that created image isn’t
a bundled AMI, it is a disk image prepared to be bundled and
delivered by the S3 plugin.
Supported operating Systems:
Fedora – all versions
RHEL – all versions
CentOS – all versions Abdullah&Hiatham 05/02/2011
14
Platform plugins (Cont.)
EC2 Platform Plugin (Continue):
EC2 Platform Plugin Configuration:
Configuration for this kind of plugins is not required
Ec2 Platform Plugin Usage:
boxgrinder-build applince-name.appl –p ec2
Abdullah&Hiatham 05/02/2011
15
Delivery plugins
Delivery plugins:
A delivery plugin moves the deliverables from a platfrorm or
operating system plugin to a selected location type. This could be a
local directory, SFTP server, Amazon CloudFront or an Amazon S3
bucket.
There are Four types of delivery Plugins;
Local plugin
SFTP plugin
S3 plugin
EBS Plugin
*The focus will be on the third type “S3 Plugin” as we had dealt with
this term when we
used Amazon Web Services.
Abdullah&Hiatham 05/02/2011
16
Delivery plugins(Cont.)
S3 Delivery Plugin:
This plugin delivers artifacts to a S3 bucket. The plugin is able to deliver
artifact in three types:
S3: a packaged (.tgz) image with metadata – good for distribution,
Cloudfront: a packaged image with metadata (same as for s3 type) for
public download using CloudFront – great for distribution, you need to have
CloudFront enabled for your account,
Ami: creates an AMI from selected image and registers it in Amazon EC2.
After that the AMI will be visible for you as a private image and ready to run.
This type is only available for images in EC2 format (converted using “-p
ec2” switch).
Supoerted Operating Systems:
Fedora – all versions
RHEL – all versions
CentOS – all versions
S3 Delivery Plugin Supported Platforms:
EC2
Abdullah&Hiatham 05/02/2011
17
Delivery plugins(Cont.)
S3 Delivery Plugin Configuration:
This is how would the configuration section of this plugin look like:
In our Example:
plugins:
s3:
access_key: AWS_ACCESS_KEY
# (required)
secret_access_key: AWS_SECRET_ACCESS_KEY
# (required)
bucket: BoxGrinder
# (required)
account_number: 6641-7087-8440
# (required)
path: /images
# default: / The
path were the image
created is located.
cert_file: cert-4NR4SZJMIQM7N4VKKVDOFIDMY6L5QLUP.pem
key_file: pk-4NR4SZJMIQM7N4VKKVDOFIDMY6L5QLUP.pem
host: http://s3.amazonaws.com
Abdullah&Hiatham 05/02/2011
18
Delivery plugins(Cont.)
S3 Delivery Plugin (Continue):
S3 Delivery Plugin Example:
EC2 AMI for appliance-name.appl:
“boxgrinder-build jeos.appl -p ec2 -d ami” Command
Abdullah&Hiatham 05/02/2011
19
EBS Delivery plugins
This plugin delivers appliance as EBS-based AMI to AWS. Note that
only appliances converted to EC2 format using EC2 platform plugin
can be delivered as EBS AMI’s.
*This plugin can only be used on instances running on EC2. This
plugin will not work on your local host because we need to mount
EBS volume to copy the data and we cannot do a remote mount.
You can use meta appliance AMI to create EBS AMI’s.
Supported Operating Systems:
Fedora – all versions
RHEL – Ver. 6
EBS Delivery Plugin Supported Platforms:
EC2
EBS Delivery Plugin Configuration:
This is how would the configuration section of this plugin look like:
Abdullah&Hiatham 05/02/2011
20
EBS Delivery plugins(Cont.)
plugins:
ebs:
access_key: AWS_ACCESS_KEY
# required
secret_access_key: AWS_SECRET_ACCESS_KEY
# required
account_number: 6641-7087-8440
# required
delete_on_termination: false
# default:
true
(Note that the delete_on_termination flag is used to specify if the
root volume should be deleted after the instance is terminated)
EBS Delivery Plugin Example:
EBS-based AMI for jeos.appl:
“boxgrinder-build appliance-name.appl -p ec2 -d ebs” Command
Abdullah&Hiatham 05/02/2011
21
What is OpenStack?
OpenStack is a collection of open source technologies delivering a massively
scalable cloud operating system. OpenStack is currently developing two
interrelated projects: OpenStack Compute and OpenStack Object Storage.
OpenStack Compute is software to provision and manage large groups of
virtual private servers, and OpenStack Object Storage is software for
creating redundant, scalable object storage using clusters of commodity
servers to store terabytes or even petabytes of data.
Backed by Rackspace, NASA, Dell, Citrix, Cisco, Canonical and over 50
other organizations, OpenStack has grown to be a global software
community of developers, technologists, researchers and corporations
collaborating on a standard and massively scalable open source cloud
operating system.
History:
In July 2010, Rackspace Hosting and NASA jointly launched a new open
source cloud initiative known as OpenStack. The mission of the OpenStack
project is to enable any organization to create and offer cloud computing
services running on standard hardware. The community's first official
release, code-named Austin, was made available just 4 months later with
plans to release regular updates of the software every few months.
Abdullah&Hiatham 05/02/2011
22
Why OpenStack?
Control and Flexibility:
Open source platform means you’re never locked to a proprietary vendor,
and modular design can integrate with legacy or third-party technologies to
meet your business needs. Hypervisor support for Microsoft Hyper-V, Citrix
XenServer, Xen, KVM, VMWware ESX, LXC, QEMU, and UML.
Industry Standard:
More than 60 leading companies from over a dozen countries are
participating in OpenStack, including Cisco, Citrix, Dell, Intel and Microsoft,
and new OpenStack clouds are coming online across the globe.
Proven Software:
Running the OpenStack cloud operating system means running the same
software that today powers some of the largest public and private clouds in
the world.
Compatible and Connected:
Compatibility with public OpenStack clouds means enterprises are prepared
for the future—making it easy to migrate data and applications to public
clouds when conditions are right—based on security policies, economics,
and other key business criteria.
Abdullah&Hiatham 05/02/2011
23
Why OpenStack?(Cont.)
The OpenStackCloud:
OpenStack is a collaborative software project designed to
create freely available code,badly needed standards, and
common ground for the benefit of both cloud providers and
cloud customers. OpenStack is currently three projects;
*OpenStack Compute
*OpenStack Object Storage
*OpenStack Image Service
OpenStack Compute:
OpenStack Compute is open source software designed to
provision and manage large networks of virtual machines,
creating a redundant and scalable cloud computing platform.
It gives you the software, control panels, and APIs required to
orchestrate a cloud, including running instances, managing
05/02/2011 users and projects.
networks, and controllingAbdullah&Hiatham
access through
24
OpenStack Compute (Cont.):
Popular Use Cases:
* Service providers offering an IaaS compute platform
* IT departments provisioning compute resources to teams and
projects
* Processing big data with tools like Hadoop
* Scaling compute up and down to meet demand for web resources
and applications
Key Features:
*Complete multi-tenant cloud computing platform scaling to tens of
thousands of compute nodes.
*Rich, well documented native OpenStack API with strong ecosystem
and support for the Amazon EC2 API.
*Control panels that make it easy to move these workloads between
data centers, and even different service providers.
Abdullah&Hiatham 05/02/2011
25
OpenStack Compute (Cont.):
OpenStack Compute Control Panel View:
Abdullah&Hiatham 05/02/2011
26
OpenStack Object Storage:
OpenStack Object Storage (code-named Swift) is open source software for creating
redundant, scalable object storage using clusters of standardized servers to store
petabytes of accessible data. It is not a file system or real-time data storage
system, but rather a long-term storage system for a more permanent type of static
data that can be retrieved, leveraged, and then updated if necessary.
Primary examples of data that best fit this type of storage model are virtual
machine images, photo storage, email storage and backup archiving.
Popular Use Cases:
* Service providers offering IaaS storage platform
* Integrates with OpenStack Compute to store server images
* Document storage
* Back-end for Microsoft SharePoint
* Archive platform for log files and other data with long retention periods
* Store web images and thumbnails
Abdullah&Hiatham 05/02/2011
27
OpenStack Object Storage (Cont.)
Key Features:
* Complete multi-tenant cloud storage platform
* Massive scalability to multi-petabyte size and billions of objects
* Able to store objects of unlimited size
OpenStack Image Service:
OpenStack Image Service (code-named Glance) provides
discovery, registration, and delivery services for virtual disk images.
The Image Service API server provides a standard REST interface
for querying information about virtual disk images stored in a variety
of back-end stores, including OpenStack Object Storage. Clients
can register new virtual disk images with the Image Service, query
for information on publicly available disk images, and use the Image
Service's client library for streaming virtual disk images.
Abdullah&Hiatham 05/02/2011
28
OpenStack Image Service (Cont.)
A multi-format image registry, OpenStack Image Service allows
uploads of private andpublic images in a variety of formats,
including:
* Raw
* Machine (kernel/ramdisk outside of image, a.k.a. AMI)
* VHD (Hyper-V)
* VDI (VirtualBox)
* qcow2 (Qemu/KVM)
* VMDK (VMWare)
* OVF (VMWare, others)
Popular Use Cases:
* Service providers offering an IaaS virtual machine image registry
* Enterprises creating a low cost DR/BCP platform to backup their
virtual server images
* ISVs offering pre-configured
optimized
images of their software
Abdullah&Hiatham
05/02/2011
29
OpenStack Image Service (Cont.)
Key Features:
*Configured to use standard command line and configuration option
processing, making use of paste.deploy configuration
*Registry database is under version control, with migration files
allowing upgrade and downgrade of the registry database.
*Versatile combination of "disk format" and "container format" fields,
allowing more than just EC2-style image formats to be stored in the
registry.
Abdullah&Hiatham 05/02/2011
30
References
• http://cs.uccs.edu/~cs526/cloudcomputing/intro.html
• http://fedoraproject.org/wiki/Features/BoxGrinder
• http://boxgrinder.org/
• http://www.openstack.org/index.php
Abdullah&Hiatham 05/02/2011
31
Questions ??
Abdullah&Hiatham 05/02/2011
32