Deploy on cloud - Where are we?
Download
Report
Transcript Deploy on cloud - Where are we?
DEPLOY ON CLOUD - WHERE ARE WE?
Qiming Teng
[email protected]
AGENDA
Heat
DockerCompose
Heat
Kubelet
Heat
BootConfig
Heat
SoftwareConfig
Convergence
Heat
Ansible
Heat
Docker Plugin
Heat
Basic
Senlin
FROM TEMPLATE TO STACK
version: xxx
parameters:
key: mykey
resources:
server: OS::Nova::Server
key: {get: key}
image: gold
flavor: m1.small
network: {get: network}
volume: {get: volume}
network: OS::Neutron::Network
...
volume: OS::Cinder::Volume
...
Nova
Heat
Neutron
Cinder
•
•
Instance
orchestrator not bandmaster
deployment tool
Volume
NOT JUST A DEPLOYMENT TOOL, PLEASE!!!
Desired States
version: xxx
parameters:
key: mykey
resources:
server: OS::Heat::ServerGroup
count: 5
S0
Heat
S1
S2
(2) listen
S3
S4
(3) converge
volume: OS::Cinder::Volume
...
(1)
Parallelized
Operation
S0
S1
S2
S3
Observed States
S4
SOFTWARE-CONFIG/SOFTWARE-DEPLOYMENT
version: xxx
parameters:
key: mykey
resources:
config: OS::Heat::SoftConfig
group: script
config:
# your script
server: OS::Nova::Server
key: {get: key}
image: gold
flavor: m1.small
network: {get: network}
volume: {get: volume}
user_data:
{get: config}
Chef
Heat
App
Nova
•
•
•
•
•
•
Puppet
OS
Instance
OS::Heat::CloudConfig
OS::Heat::SoftwareConfig
OS::Heat::StructuredConfig
OS::Heat::SoftwareDeployment
OS::Heat::SoftwareDeployments
OS::Heat::SoftwareComponent
SOFTWARE-CONFIG/SOFTWARE-DEPLOYMENT
version: xxx
resources:
config-1: OS::Heat::SoftConfig
deploy-1:
config: config-1
server: server-1
server-1: OS::Nova::Server
config-2: OS::Heat::SoftConfig
deploy-2:
depends_on: deploy-1
config: config-2
server: server-2
server-2: OS::Nova::Server
depends on
config-1
config-2
server-1
server-2
depends on
config-1
config-2
server-1
server-2
SOFTWARE-CONFIG/SOFTWARE-DEPLOYMENT
Q: What is it?
A: THEY are collection of agents including
•
•
•
•
•
•
•
•
os-collect-config [1]
os-refresh-config [2]
os-apply-config [3]
heat-config-script [4]
heat-config-puppet [4]
heat-config-docker-compose [4]
heat-config-kubelet [4]
...
Chef
Puppet
App
OS
Instance
[1] http://git.openstack.org/cgit/openstack/os-collect-config/
[2] http://git.openstack.org/cgit/openstack/os-refresh-config/
[3] http://git.openstack.org/cgit/openstack/os-apply-config/
[4] http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/elements
SOFTWARE-CONFIG/SOFTWARE-DEPLOYMENT
Q: How does the agent authenticate?
A: Heat does secret job in the background
•
created during setup
stack_domain_project
•
•
name stack id
stack_domain_user
name resource name
password ?
• uuid.uuid4().hex
•
•
Puppet
heat domain
•
•
Chef
NOTE: There is a side path of generating EC2 tokens
App
OS
Instance
SOFTWARE-CONFIG/SOFTWARE-DEPLOYMENT
Q: How are the agents injected/installed?
A: There are two ways
•
•
disk-image-builder (dib)
• a TripleO project [1]
• prebuilt images for use
Heat boot-config [2]
• install these agents on the fly when
VM boots up
Chef
Puppet
App
OS
Instance
[1] http://git.openstack.org/cgit/openstack/diskimage-builder/
[2] http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/boot-config
HEAT BOOT-CONFIG
goal
install agents required to use certain software deployments in templates [1]
how it's used
define an env yaml file with a Heat::InstallConfigAgent resource
refer to this resource in your server.properties.user_data
heat stack-create -f template -e environment mystack
template
config
server
#!/bin/sh
yum install ..
env yaml
environment
inst-config
InstAgent
config-config
start-config
user_data
MIME
#!/bin/sh
cat << EOF ...
mkdir ...
#!/bin/sh
systemctl enable ...
systemctl start ...
[1] http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/boot-config/
HEAT CONTAINER AGENT
goal
prepare guest environment for container deployment with docker-compose
template
config
server
environment
install_container_agent
InstAgent
write_image_pull_script
#cloud-config
write_files:
# write a script that will
# grab specified image via
# 1. curl + docker load, or
# 2. docker pull
install_container_agent
user_data
heat-container-agent service
#!/bin/sh
# 1. create a service:
# heat-container-agent
# 2. enable/start docker svc
# 3. enable/start agent svc
# execute the image pull script to get the <image>
# docker run --name heat-container-agent ... \
<image>
sample image: http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/heat-container-agent/
HEAT DOCKER COMPOSE
goal
a 'hook' that uses 'docker-compose' to deploy containers [1]
an element that you will build into your guest image
template
config
group:docker-compose
inputs: [env_files]
config:
db:
image: redis
web:
image: nginx
metadata
env
files
os-collect-config
docker-compose
up -d --no-build
yml
file
heat-config
container
container
heat-docker-compose
server
OS
user_data
Instance
[1] http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/elements/heat-config-docker-compose
HEAT KUBELET
goal
a 'hook' that uses 'kubelet' agent from kubernetes to deploy containers
an element that you will build into your guest image
preinstalled
template
config
group: kubelet
inputs: [env_files]
config:
containers:
- name: doecho
image: busybox
command: ...
metadata
user_data
kubelet.service
config
poll
heat-config
container
container
hook-kubelet
preinstalled
server
os-collect-config
poll
/opt/heatdocker/images.tar
OS
Instance
[1] http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/elements/heat-config-docker-compose
HEAT CONFIG ANSIBLE
goal
a 'hook' that uses 'ansible' to configure A instance
template
config
group: ansible
inputs: [...]
config:
# your ansible book
# here
metadata
playbook
ansible-playbook
-i localhost
<file>
os-collect-config
heat-config
container
application
hook-ansible.py
server
OS
user_data
Instance
[1] http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/elements/heat-config-ansible
HEAT DOCKER PLUGIN
A Resource Type (Contrib[1]) for Heat
DockerInc::Docker::Container
built on docker-py [2]
CREATE
image: string
command: list
hostname: string
user: string
stdin_open: boolean
tty: boolean
mem_limit: integer
ports: list
environment: list
dns: list
volumes: map
cpu_shares: integer
cpuset: string
privileged: boolean
binds: map (volumes)
volumes_from: list
port_bindings: map
START links: map
restart_policy: map
cap_add: list
cap_drop: list
read_only: Boolean
devices: list
info
network_info
network_ip
SHOW network_gateway
network_tcp_ports
network_udp_ports
logs
logs_head
logs_tail
[1] http://git.openstack.org/cgit/openstack/heat/tree/contrib/heat_docker/
[2] https://github.com/docker/docker-py
SERVICES ROADMAP ON SUPERVESSEL
(Online)
SuperVessel
Cloud
Service
(Online)
SuperVessel
Big Data and HPC
Service
1. VM and container
service
2. Storage service
3. Network service
1.Big Data:
MapReduce
(Symphony), SPARK
2. Performance tuning
service
4.Accelerator as
service
OpenPOWER
Enablement
Service
1. X-to-P migration:
AutoPort tool
2. OpenPOWER new
system test service
17
(Online)
Super
Class
Service
(Preparing)
Super Project
Team
Service
1. On-line video
courses
2. Teacher course
management
3. User contribution
management
1. Project
management
service
2. DevOps
automation
5. Image service
Docker
SuperVessel Cloud Infrastructure
Storage
IBM POWER servers
OpenPOWER server
FPGA/GPU
SUPERVESSEL
HEAT -- TRIED NOT TO JUST A DEPLOYER
Supports to High-Availability
OS::Heat::HARestarter
recreates a resource when failure detected
Supports to Auto-Scaling
OS::Heat::InstanceGroup
OS::Heat::ResourceGroup
OS::Heat::AutoScalingResourceGroup
OS::Heat::ScalingPolicy
AWS::AutoScaling::AutoScalingGroup
AWS::AutoScaling::ScalingPolicy
AWS::AutoScaling::LaunchConfiguration
AUTOSCALING REORG
BLUEPRINTS ON REWORKING HEAT AUTOSCALING
BP
Priority
Description
autoscaling-api-resources
high
Heat resources invoking AS APIs
as-api-group-resource
high
ScalingGroup resource wrapping AS API's group functionality
as-api-policy-resource
high
ScalingPolicy resource wrapping AS API's policy functionality
as-api-webhook-resource
high
Webhook resource wrapping AS API's execution of webhooks
autoscaling-api-client
high
A python client for Heat to interact with AS API
autoscaling-api
-
A separate service for the implementation of autoscaling w/ Heat
as-engine
-
A separate engine/service for autoscaling support AS API
as-engine-db
-
A DB dedicated to autoscaling, using schema created in as-lib-db
as-lib
-
A separate module to be used by the AS service
as-lib-db
-
A DB for autoscaling bookkeeping
DEPENDENCIES AMONG BPS
autoscaling-api-resources
as-api-policy-resources
as-api-group-resources
autoscaling-api-client
autoscaling-api
as-lib
as-engine
as-lib-db
as-engine-db
as-api-webhook-resources
OVERVIEW OF AUTOSCALING
A STRUGGLE BEFORE SENLIN STARTS
Should we do this within Heat or outside Heat?
Within Heat
pros
smooth transition; strict reviews better quality
cons
long (maybe forever) code churn; eventually, a dedicated service is needed, thus the pain to switch over
Outside Heat
pros
quick development; less code churn to Heat
cons
high requirements of skills and cycles; eventual switch over, i.e. another animal to feed in the OpenStack zoo
We choose OUTSIDE HEAT
There are Heat core team supports to this approach
We see a lot potentials in a standalone clustering service
We don't have to do everything from scratch we "borrow" and "steal" code whenever license
permits
WHAT DO WE REALLY NEED?
-- What is missing from OpenStack
Scalable
Load-Balanced
Highly-Available
Manageable
......
of any (OpenStack) objects
[Chinese Pinyin for Forest]
SENLIN ARCHITECTURE
Senlin
Client
REST
Senlin
API
Policies
RPC
Senlin Engine
Profiles
Senlin
Database
ER DIAGRAM
webhook
API
plugins
«policy_type»
placement_policy
policy
name
uuid
type
level
spec
action
context
action
inputs
outputs
target
action
user
profile
name
uuid
type
spec
plugins
«profile_type»
os.heat.stack
os.nova.server
update_policy
cluster_policy
deletion_policy
scaling_policy
lb_policy
health_policy
cluster_id
policy_id
enabled
level
cooldown
priority
cluster
name
uuid
user
project
parent
profile_id
status
create()
delete()
update()
add()
remove()
node
name
uuid
cluster_id
profile_id
index
status
created_time
updated_time
os.cinder.volume
os.keystone.user
SENLIN OPERATIONS (ACTIONS)
Cluster
• CREATE
• DELETE
• UPDATE
• LIST
• SHOW
• ADD_NODES
• DEL_NODES
• SCALE_OUT
• SCALE_IN
• POLICY_ATTACH
• POLICY_DETACH
• POLICY_UPDATE
Node
•
•
•
•
•
•
•
•
Policy
• CREATE
• UPDATE
• DELETE
• LIST
• SHOW
Profile
• CREATE
• UPDATE
• DELETE
• LIST
• SHOW
Event
• LIST
• SHOW
Webhook
• CREATE
• DELETE
• LIST
• SHOW
CREATE
DELETE
UPDATE
LIST
SHOW
JOIN
LEAVE
MIGRATE
Action
• LIST
• SHOW
RELATION TO OTHER PROJECTS
Senlin provides the array data type for cloud programming
struct person {
// Heat stack containing senlin clusters
int age;
char name[0]; // Senlin cluster of nova servers
}
person team[10];
// Senlin cluster of Heat stacks
Horizon
Complex Data Types
Primitive Data Types
Heat
Nova
Cinder
Senlin
Neutron
Ceilometer
Swift
Keystone
CURRENT STATUS
Date
Milestone
2014-12-10
Initial Git Repository inside CRL
2014-12-25
Migration to github.com
2014-01-14
Introduction to IBM Heat Community
2015-01-19
Weekly conference call started
2015-02-06
Announcement on IBM openstack-dev mailinglist
2015-02-13
Email to OpenStack Heat Core team
2015-03-16
Senlin project accepted to OpenStack StackForge
2015-03-21
Senlin client project accepted to OpenStack StackForge
2015-03-26
Project announcement in community (link)
Code Base
http://git.openstack.org/cgit/stackforge/senlin
including API design under doc subdirectory
http://git.openstack.org/cgit/stackforge/python-senlinclient
IRC channel: #senlin
NEXT STEP
Complete AutoScaling support
Cross-Region AutoScaling
Features Pipeline (draft)
High Priority
Middle Priority
Low Priority
Item
W
Item
W
Item
W
Event Listening
**
Horizon Plug-in
*
Metrics Collection
*
Scavenger Process
*
User Defined Actions/Ansible
***
AWS Compatible API
***
Multi-Engine Support
*
Quota Enforcement
*
Integration with Mistral
**
Test Case Coverage
**
Event Notification
*
Cluster suspend/resume
**
Babican Support
*
Scheduled actions
*
VPNaaS support
**
Interaction with Congress
*
Nova ServerGroup API
*
Integration with Tooz
**
THANK YOU!