Transcript Azure Stream Analytics

Data Patterns for the Cloud
James Carpinter
M313
Network
Fast & reliable network
Switch for app + db?
Same D/C; less reliable, more hops
and load balancers = latency
Storage
Big, fast SAN
Big, cheap JBOD
Hardware
Specific to role
Generic (no custom SKUs)
Availability
Managed servicing, low failures
Unexpected services & failures
Purchasing
Upfront capex: overprovision
Opex (add/remove on demand)
Licensing
Per processor, per year
Per minute/hour
Result
*Everything* goes in database
*Everything* as-a-service
Compute
Data
& Storage
Cloud
Services
Storage
Blobs
Tables
Queues
Compute
Data
& Storage
Cloud
Services
Storage
Blobs
Tables
Queues
SQL
Database
Web App
Virtual
Machines
Storage
Blobs
Mobile App
Cloud
Services
Tables
API App
Batch
Queues
Logic App
API
Management
Service Fabric
Files
RemoteApp
SQL
Database
Notification
Hubs
Search
BizTalk
Services
Service Bus
Mobile
Engagement
Virtual
Network
ExpressRoute
DocumentDB
Backup
Azure Active
Directory
Traffic
Manager
Redis Cache
Site Recovery
Multi-Factor
Authent
DNS
StorSimple
Media & CDN
Event Hubs
Key Vault
Identity
& Access
App
Service
Data Factory
Operational
Insights
Marketplace
Stream
Analytics
Automation
Hybrid
Integration
HDInsight
Machine
Learning
Scheduler
Networking
Application
Insights
Management
Developer
Services
Analytics
& IoT
Web
& Mobile
Compute
Data
& Storage
Visual Studio
Online
…
Media
Services
CDN
Application
Gateway
Data Lake
Data
Warehouse
•

Consistency
Relational, Un-partitioned
Big Table-like:
HBase, MongoDB
Availability
Dynamo-like:
Cassandra, CouchDB
Partition Tolerant
(2) Sharding Key
(3) Shard Map
Manager
(6) Sharded Table
(8) Shardlet
Customer Table
Customer ID
Name
1
Alice
2
Bob
...
Data Center Table
Data Center ID
DC Name
1
Boston
2
Miami
(1) Database
(4) Shard
(7) Reference Table
(5) Shard Set
(4) Shard
Polyglot
persistence
Not a new
concept
• Optimized for data
• Optimized for workload
• EAV
• XML
• Architecture paradigm: OLAP/DW
and OLTP
transactional processing
rich query
managed as a service
elastic scale
schema-free data model
internet accessible http/rest
arbitrary data formats
RabbitMQ /
ActiveMQ
Applications
Devices
Web/thick client
dashboards
Cloud gateways
(web APIs)
Stream
processin
g
Data analytics (Excel)
Sensors
Web and
Social
Search and query
Field
gateways
Storage
adapters
Devices to take action
1
Azure Storage Architecture and
getting the most out of IaaS
Premium storage
??? Thu 10:40am
2
How to Build High Performance Apps
Using Microsoft Azure Redis Cache
??? Thu 1:55pm
3
Building highly available and recoverable
solutions with Azure Event Hubs and
Service Bus Messaging
??? Thu 3:10pm
4
Advanced Messaging Scenarios with
Azure Service Bus Messaging
??? Fri 11:55am
5
In-Memory OLTP: The Road Ahead
6
Elastic for SQL – shards, pools,
stretch
??? Wed 11:55am
??? Fri 11:55am
Find me later at…
 Hub Happy Hour Wed 5:30-6:30pm
Free Online Learning
http://aka.ms/mva
Subscribe to our fortnightly newsletter
http://aka.ms/technetnz
http://aka.ms/msdnnz
Sessions on Demand
http://aka.ms/ch9nz
© 2015 Microsoft Corporation. All rights reserved.
Microsoft, Windows and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.