Transcript Open Source on Microsoft Azure: An Overview

Copyright © 2014 Chappell & Associates
i
Open source technologies are a fundamental
part of Microsoft Azure
?
Is Microsoft really serious
about supporting open source
on Azure?
ANSWER
Yes:
- Its competitors do this
- Cloud platforms sell cycles,
storage, and bandwidth. Who
cares what software uses these?
?
Do customers care about open
source on Azure?
ANSWER
Yes:
- For running Linux workloads
- For open source development
on Linux and Windows
Compute
Data
IaaS, PaaS, and mobile
services
SQL, NoSQL, and big data
analytics
Infrastructure as a
Service (IaaS)
Platform as a
Service (PaaS)
Virtual Machines
Cloud Services
Platform as a
Service (PaaS)
Web Sites
Application
VMs
Application
Web
Roles
VHDs
IIS
Application
Application
Worker
Roles
IIS
Gallery
VHDs
User-Supplied
Application
Mobile Backend as a Service (MBaaS)
Mobile Services
Authentication
Notifications
Custom Logic
...
Virtual Machines
VMs
Windows Server images
provided by Microsoft;
Linux images provided
by partners
Linux images provided
by the community
VHDs
Gallery
VHDs
VHDs
VM Depot
User-Supplied
Windows Server and
Linux images
provided by
customers
!
!
!
!
!
!
Canonical Ubuntu
Oracle Linux
SUSE Linux Enterprise
- Targets enterprises
openSUSE Linux
- Community distro
CENTOS by Open Logic
- Binary compatible with Red Hat Enterprise Linux
Canonical Ubuntu
- With preinstalled Puppet Enterprise from Puppet Labs
Microsoft provides
forum-based support
for all (today)
Windows
VMs
Puppet/Chef Agents
Puppet
Console
Linux
VMs
Puppet/Chef Agents
Chef Management
Console
The Linux images in the
gallery provide just an OS
Other Linux distributions
are available
Images typically include
other software
Example: Debian
Examples: Django,
Tomcat/EJB, Drupal,
WordPress, Subversion,
Jenkins, SugarCRM, …
An image’s provider can
offer paid support
Example: Bitnami
Windows
Server
Linux
Small A1
(1 core, 1.75 GB)
$0.09/hour
$0.06/hour
Medium A2
(2 cores, 3.5 GB)
$0.18/hour
$0.12/hour
Large A3
(4 cores, 7 GB)
$0.36/hour
$0.24/hour
Extra Large A4
(8 cores, 14 GB)
$0.72/hour
$0.48/hour
Memory Intensive A5
(2 cores, 14 GB)
$0.33/hour
$0.25/hour
Memory Intensive A6
(4 cores, 28 GB)
$0.66/hour
$0.50/hour
Memory Intensive A7
(8 cores, 56 GB)
$1.32/hour
$1.00/hour
Linux is as much
as 1/3 cheaper
than Windows
Server
Cloud Services
Provides a pre-built,
managed environment
for running Windows
applications
Application
Web
Roles
Worker
Roles
Base images are
Windows Server
Can install open
source software
in Windows
Server VMs
VMs
Websites
IIS
Users
From the Web
App Gallery
Open Source Web
Applications
Custom Web
Applications
Microsoft Azure
Web Sites
Base images are
Windows Server
FTP,
WebDeploy
Code
HTML
TFS,
Git
What It Provides
Technology Foundation
WordPress
Content management
system/blogging
PHP/MySQL
Joomla
Content management
system
PHP/MySQL (and others)
Web server/
servlet container
Java
Wiki package
PHP/MySQL (and others)
Django
Web framework
Python
Express
Web framework
JavaScript/Node.js
Apache Tomcat
MediaWiki
PHP
Python
Scripting language and
environment for web
development
General-purpose dynamic
programming language
ASP.NET and Java
applications are also
supported
Node.js
Environment for web
development with JavaScript
Node.js can run in
Virtual Machines, Cloud
Services, and Web Sites
Microsoft Azure
HTML/JSON
JavaScript
Client Code
JavaScript
Server Code
JavaScript
Engine
V8 JavaScript
Engine
Web Browser
Node.js
VM
Same programming language
on client and server
Written as a set of
event handlers, each
dispatched by the
Node.js event loop
Created by Google for
the Chrome browser
Perhaps a single-page
app (SPA)
JavaScript
Client Code
Created by Google,
provides data
binding and more
Microsoft Azure
HTML/JSON
JavaScript
Server Code
AngularJS
JavaScript
Engine
V8 JavaScript
Engine
Web Browser
Node.js
VM
Many useful
modules are
available as open
source, e.g., Express
Cloud9
IDE
Works with Node.js
on Microsoft Azure
Web Sites (today)
Visual Studio
Online
“Monaco”
Web Browser
WebMatrix
Node.js Tools
for Visual
Studio
Windows
Microsoft Azure
JavaScript
Server Code
V8 JavaScript
Engine
Node.js
VM
What is TypeScript?
Who’s creating it?
A typed superset of
JavaScript that compiles
to ordinary JavaScript
A group led by Anders
Hejlsberg at Microsoft
Why?
To help make JavaScript a more
effective language for creating
large-scale applications
Using identities from
Microsoft, Google, etc.
Using services from
Microsoft or Apple
User
Authentication
Built with Node.js
or ASP.NET
Data
Access
Notifications
Custom Code
Microsoft Azure Mobile Services
REST/JSON
Windows Store
Apps
Windows Phone
Apps
iOS
Apps
Android
Apps
Microsoft
provides SDKs
for all
Microsoft Azure
Virtual Machines (IaaS)
Microsoft Azure
Cloud Services (PaaS)
Pros: Linux and Windows
support; complete control
of the VM
Pros: Least amount of
required management
Pros: Easiest to use; built-in
support for many options
Cons: Complete
responsibility for VM
management
Cons: Windows only;
must reinstall open
source software each time
a VM starts
Cons: Windows only; using
more than the options
Microsoft provides takes extra
work
Use Mobile Services to
support mobile apps
Microsoft Azure
Web Sites (PaaS)
Operational Data
Analytical Data
Key/Value Store
(Microsoft Azure Tables, Redis, …)
NoSQL
Technologies
Column Family Store
(Cassandra, HBase, …)
Document Store
(MongoDB,
CouchDB, …)
SQL
Technologies
Big Data Analytics
(HDInsight, Hadoop)
Graph Database
(Neo4J, …)
Relational Database
(SQL Database,
SQL Server, Oracle, MySQL, …)
Relational Reporting
(SQL Server, Oracle, MySQL, …)
Relational Analytics
(SQL Server, Oracle, MySQL, …)
Provided by
Microsoft Azure
Runs in Microsoft Azure
Virtual Machines
Runs in Microsoft Azure
Virtual Machines and is
open source
MySQL, etc. can run in a Microsoft
Azure Virtual Machines VM
Database
Table
Application
Table
SQL
Query
Schema
Relation
ClearDB provides MySQL as a
managed service on Microsoft Azure
Table
To scale for lots of
users and lots of data
To work better with
different data
formats, e.g., JSON
Pros: NoSQL
technologies can offer
more scalability than
relational databases
Pros: Avoiding
object/relational
mapping makes code
easier to write
Pros: NoSQL
technologies don’t have
fixed schemas
Pros: Hadoop has a
large and growing
ecosystem of tools and
people
Cons: Often lose
some benefits of
relational databases,
e.g., secondary
indexes, full
transactions
Cons: Persistent data
designed for a single
application is harder
to share; limited BI
tools
Cons: Fixed schemas
help prevent errors; data
often isn’t normalized
Cons: Moving lots of
on-premises data to
Microsoft Azure can take
time
To work with data in
a more flexible way
Developers vs. DBAs?
To analyze lots of
data in parallel
NoSQL technologies are typically
deployed in Microsoft Azure Linux VMs
Database
Shard
B 3
Application
Key
Value (String, List, Set, Hash)
Shard
Shard
A 1
B 1
C 1
A 2
B 2
C 2
A 3
B 3
C 3
Key/value stores
Example
Technologies
What It Provides
Example Use Case
Redis, Microsoft Azure
Tables
Fast access to large
amounts of simply
structured data
Online shopping
cart
Roots are in
Amazon Dynamo
Columns store multiple timestamped versions of a value
Keyspace
Column Family
A
X B 5
Application
1
2
X
5
Column
Row Key
Column Name
Row
Value
3
C
4
7
8
Y
B
Column Family Name
Column Family
2
5
B
4
Data that’s frequently accessed together
should be stored in the same column family
Key/value stores
Column family stores
Example
Technologies
What It Provides
Example Use Case
Redis, Microsoft Azure
Tables
Fast access to large
amounts of simply
structured data
Online shopping cart
Cassandra, HBase
Fast access to large
amounts of more
structured data
A table storing web
pages
Roots are in
Google Bigtable
Can create indexes on
multiple keys
Database
Targets a specific collection
Shard
Shard
Collection
Query
JSON
JSON
JSON
JSON
Application
JSON
JSON
Shard
Collection
JSON
JSON
JSON
JSON
JSON
JSON
Document
MongoLab provides a managed database
service based on MongoDB for Microsoft Azure
Example
Technologies
What It Provides
Example Use Case
Redis, Microsoft Azure
Tables
Fast access to large
amounts of simply
structured data
Online shopping cart
Column family stores
Cassandra, HBase
Fast access to large
amounts of more
structured data
A table storing web
pages
Document databases
MongoDB, CouchDB
Scalable store for JSON
documents
Persistent store for
Node.js application
Key/value stores
Microsoft Azure
JavaScript
Application
JSON
Web Browser
JavaScript
Server Code
Query
Mongo DB
Collection
JSON
JSON
JSON
Apps
Node.js
PC/Phone/Tablet
VM
Mongoose and other packages
support connecting to MongoDB
JSON
JSON
VM
The company MongoDB
(formerly 10gen)
provides good examples
NoSQL offerings are largely
open source today
This is likely to be how many
new markets go
NoSQL is among the first
new software categories
to arise since open source
became so important
The traditional software
revenue model won’t
apply
NoSQL companies can’t make
money selling ones and zeros
They must sell other things
NoSQL Example
Other Examples
Running open source software and
charging users for execution services
MongoDB in Microsoft
Azure IaaS VMs
Cloud
platforms/hosting
Running open source software and charging
users for the value the software provides
MongoLab on Microsoft
Azure
WordPress on Microsoft
Azure Web Sites
Providing services to maintain
and update open source software
MongoDB Enterprise
Red Hat Enterprise
Linux
Providing services to help install,
customize, and use open source software
MongoDB Consulting
Systems integrator open
source practice
Running open source software and charging
advertisers for your users’ attention
Selling hardware that relies on
open source software
Google’s Linux-based
services
Android phones
All of these are also
possible with
proprietary software
A
B
D
X
Y
Relationships
are first-class
data items
Query
Application
B
D
Graph
Graph
Node
Relationship
Property
Hard to shard, so not
intended for massive scale
Example
Technologies
What It Provides
Example Use Case
Redis, Microsoft Azure
Tables
Fast access to large
amounts of simply
structured data
Online shopping cart
Column family stores
Cassandra, HBase
Fast access to large
amounts of more
structured data
A table storing web
pages
Document databases
MongoDB, CouchDB
Scalable store for JSON
documents
Persistent store for
Node.js application
Neo4J
Fast access to data
organized into graphs
Social graph
Key/value stores
Graph databases
Hadoop 2.0 adds YARN, supporting
frameworks other than MapReduce
Hadoop Distributed
File System (HDFS)
Allows storing and
accessing very large
binary files across a
cluster of commodity
servers and disk
drives
Hadoop MapReduce
Supports applications
that process large
amounts of analytical
data in parallel
Data is typically
stored in HDFS
Hive/Pig
Tools for querying,
transforming, and
analyzing data
Both generate
MapReduce jobs
HBase
Column family store
built on HDFS
Designed for
operational data, not
analytical data
Provides HiveQL, a SQLlike query language
Excel
Hive
Pig
...
Hadoop
MapReduce
Job
Hadoop
Distributed File
System (HDFS)
Microsoft allows
submitting HiveQL
queries from Excel
Logic
Logic
Logic
Data
Data
Data
Using HDInsight will typically make
more sense than building your own
Hadoop cluster on Microsoft Azure
Excel
Hive
Pig
...
Hadoop
MapReduce Job
VM
VM
VM
Logic
Logic
Logic
Data
Data
Data
HDFS API
Microsoft Azure Blobs
i
Open source technologies are a fundamental
part of Microsoft Azure
David Chappell is Principal of Chappell & Associates (www.davidchappell.com)
in San Francisco, California. Through his speaking, writing, and consulting, he
helps people around the world understand, use, and make better decisions
about new technology. David has been the keynote speaker for more than a
hundred events and conferences on five continents, and his seminars have
been attended by tens of thousands of IT leaders, architects, and developers in
forty-five countries. His books have been published in a dozen languages and
used regularly in courses at MIT, ETH Zurich, and other universities. In his
consulting practice, he has helped clients such as Hewlett-Packard, IBM,
Microsoft, Stanford University, and Target Corporation adopt new technologies,
market new products, and educate their customers and staff. Earlier in his
career, David wrote networking software, chaired a U.S. national standards
working group, and played keyboards with the Peabody-award-winning
Children’s Radio Theater. He holds a B.S. in Economics and an M.S. in Computer
Science, both from the University of Wisconsin-Madison.
Having a friend
buy your coffee?
Yea, it’s kind of like that.
3 Steps to New Gear! With Application Insights
MSDN Subscribers get up to $150/mo in Azure credits.
Stop by the Developer Platform and Tools booth
and visit the MSDN Subscriptions station to
activate your benefits and receive a gift!
1.
http://aka.ms/msdn_teched
http://visualstudio.com
2.
http://aka.ms/aivsix
3.
Microsoft Engineering
Stories
http://www.visualstudio.com
Visual Studio
Industry Partner
Program
Visual Studio | Integrate