Transcript Dell APS Customer Presentation

Dell Analytics Platform
System (APS)
Dell’s Big Data Platform Advantage
Big Data Workshop Agenda
Customer Environment/Challenges 45 min
APS Deep Dive
Whiteboard Session / APS Fit
45 min
Next Steps
45 min
30 min
Agenda
•
•
•
•
•
2
Dell - Internal Use - Confidential
APS in 30 Seconds
Dell Competencies & Big Data
Technology Primer
Use Cases
Support & Services
APS in 30 Seconds
•
Sold, built, delivered, supported
as an APPLIANCE
•
•
•
•
•
•
Answers the Big Data challenge
for customers
•
•
•
3
Dell - Internal Use - Confidential
This is not reference architecture!
Co-engineered with Microsoft (H/W + S/W)
White glove delivery, white glove support
Jumpstart services (training) included
Additional services support available
Massively Parallel Processing (MPP),
scales from 45tb to 6pb
Don’t confuse with SQL Server!
Combines SQL MPP + Hadoop in the solution
Agenda
•
•
•
•
•
4
Dell - Internal Use - Confidential
APS in 30 Seconds
Dell Competencies & Big Data
Technology Primer
Use Cases
Support & Services
Dell and Microsoft partnership
Partnership
For more than 30 years, Dell and
Microsoft have focused on
delivering best in class, innovative
solutions that span the entire
Microsoft Product Portfolio to
organizations all over the world.
• Strategic Partnership with Microsoft
• Largest distributor of Microsoft software worldwide
• 20+ GOLD & SILVER Certified Microsoft Competencies
Dell is a Microsoft shop
• Dell’s famous Supply Chain & E-Commerce solutions Implemented using
Microsoft Technologies
• Automated Dell’s factories using MS AX
Capabilities & Offerings
• Strong industry expertise in healthcare, BFSI and commercial
• Legacy modernization, application support, development and maintenance
• 700+ Microsoft technology resources and 40% resources are certified by
Microsoft
Proven Delivery Methodology
Dell and Microsoft:
Bringing it all together
5
Dell - Internal Use - Confidential
• Framework, reusable components and delivery accelerators to reduce
time to market
• Project life cycle management using ALM solution
Dell Credentials by the Numbers
1996
Dell begins providing
enterprise level
DW / BI solutions
26+
Advanced Microsoft
Competencies
17
Microsoft Certified Masters
on Staff
6
Dell - Internal Use - Confidential
4,000+
1,100
Managed SQL
Servers, 38,000+
SQL Databases
DW / BI experts worldwide
Largest
Microsoft Reseller,
providing licensing
worldwide
Three (3)
Microsoft Partner
of the Year Awards
in 2015
4.8
PB of Data managed,
largest database
> 12 TB
BI, Analytics &
Big Data
Capabilities

Dell - Internal Use - Confidential
Questions asked of Big Data
Why Did it
Happen?
When Will it
Happen?
Value
What
Happened?
What Can I Do
About It?
Machine learning
In-memory
Predictive Analytics
Ad hoc analysis
ETL
OLAP
Transactional systems
Complex implementations
Spreadmarts
Internet of Things
Dashboards
Optimization &
Stimulation
Hadoop
Enterprise data warehouse
Any data
Data mining
Operational Reporting
Siloed data
Innovation
9
Dell - Internal Use - Confidential
Collect any data
Harness the growing and changing nature of data
Structured
Unstructured
Streaming
“”
Proven, leading relational database with new cloud NoSQL database capabilities
Take advantage of unstructured big data with 100 percent Apache-based Hadoop capabilities
Reduce the cost of analyzing and correlating streaming data
10
Dell - Internal Use - Confidential
APS: The Big Data Answer
•
Big Data is not synonymous with “Hadoop”
•
APS offers a hybrid solution where RELATIONAL Big Data (SQL)
and NON-RELATIONAL Big Data (Hadoop) can co-exist in one
appliance, using one technology platform, fully supported
•
Choice, cost efficiency, new insights gained
Relational (SQL MPP)
11
Non-Relational (Hadoop)
Dell - Internal Use - Confidential
Use Cases
APS (Analytic Platform System)
•
Merge retail sales (POS/structured) with social sentiment data
(unstructured)
•
Manufacturing data (MRP/structured) with engineering/test data
(unstructured)
•
Clickstream data (unstructured) and sales/supply chain data (ERP)
•
Internet of Things (IoT) data infrastructure
Modern Big Data Ecosystem: Pan-Dell & MSFT
Business
intelligence
Query layer /
enterprise
service
bus
MSFT APS
Software & MSFT
Polybase
Data
management
Dell APS: Appliance
Implement &
Integration
Services:
Dell APS/DW
Services
Dell MSFT BI
Services
Data sources
“”
12
Dell - Internal Use - Confidential
Sources:
Internal
Partner
Social
Leased
Blueprint
for Big Data
& Analytics
1.
Massively Parallel Processing: Real-Time Decision Support of
Relational and Unstructured Data
SOURCE
2.
INTEGRATE & AGGREGATE
3.
TRANSFORM
Microsoft
Polybase (native
in APS)
ERP
High Speed ETL
Microsoft APS
by Dell
On-Prem &
Cloud Options
Asset Tracking
In-memory relational &
non-relational harmony
Order Data
High Speed ETL
Relational data
aggregation
4.
ANALYZE & ACT
Advanced Analytics
Customer Data
Unstructured
data aggregation
Operational / Self-Service BI
Sensors
In Memory
Dell - Internal Use - Confidential
Dell Blueprints
13
SERVICES
MANAGEMENT
SECURITY
DESIGN/DEPLOY
Agenda
•
•
•
•
•
14
Dell - Internal Use - Confidential
APS in 30 Seconds
Dell Competencies & Big Data
Technology Primer
Use Cases
Support & Services
APS Defined
APS
noun | \ā – pi - əs \
1.
2.
3.
4.
5.
Massively parallel processing (MPP) data warehouse
Combines SQL PDW and Hadoop capabilities
Up to 100x faster than legacy SMP database queries (ex: SQL Server, Oracle)
Uses separate CPU’s running in parallel to execute a single query
Increase compute capacity via scale-out design
antonym : SMP (ie: SQL Server Enterprise)
1.
2.
3.
4.
15
Symmetric processing
All CPU’s share the same memory, disks, network controllers
Can only increase compute via scale-up design
Typically housed on a shared SAN
Dell - Internal Use - Confidential
How is APS faster
Columnstore index representation
C C C C
1 2 3 4
100x
Up to
faster queries
15x
Up to
more compression
C C
5 6
Updatable clustered
columnstore vs. table with customary indexing
Parallel query execution
Query
Results
16
Dell - Internal Use - Confidential
16
•
Store data in columnar format for massive compression
•
Load data into or out of memory for next-generation
performance
•
Updateable and clustered for real-time trickle loading
•
No secondary indexes required
x2 Infiniband Switch
x2 Ethernet Switch
Management Control Node
Management Failover Node
Modular, Highly
Available Design
•
Servers
•
Storage
•
Networking
•
APS/PDW Software
•
MPP DW Appliance
3rd Scale Unit
(additional 3
nodes optional)
x3 Compute Server
x2 JBOD
(51 drives)
x3 Compute Server
2nd Scale Unit
(additional 3
nodes optional )
x2 JBOD
(51 drives)
x3 Compute Server
Base Unit
(3 nodes)
17
Dell - Internal Use - Confidential
x2 JBOD
(51 drives)
Scale out versus scale-up
Scale-out technologies in the Analytics Platform System
Scaling up with traditional SQL Server
Scaling out with Analytics Platform System
APS
Forklift
APS
Forklift
APS
45TB
18
Dell - Internal Use - Confidential
6PB
Linear Scale with APS
•
•
•
•
•
45TB
19
Dell - Internal Use - Confidential
6 PB
3 – 54 Compute Nodes
2TB or 3TB Drives
45 TB – 1,223TB Raw
113 TB – 6PB User Data
Each scale unit increases
size and speed
Linear Scale
Mixed Workloads
No Downtime
Start Small
& Grow
APS – Massively Parallel Computing
Infiniband Switch
Infiniband Switch
Ethernet Switch
Ethernet Switch
SQL + Hadoop Query
Management Control Node
Management Failover Node
PDW (SQL) Region
20
Dell - Internal Use - Confidential
Compute Server
Compute Server
Compute Server
Compute Server
Compute Server
Compute Server
Compute Server
Compute Server
Compute Server
JBOD (51 drives)
JBOD (51 drives)
JBOD (51 drives)
JBOD (51 drives)
JBOD (51 drives)
JBOD (51 drives)
PDW (SQL) Region
PDW (SQL) Region
Speeds and Feeds drive innovation
Current data warehouses cannot scale to the needs of Large Enterprise
Reduce “time to innovation” by an order of magnitude.
Metrics below were conducted by Dell Services and prior to any optimization.
7 Billion records ingest and needed for mail sorting center analysis daily
The below example was set up in a couple of days for a boardroom demo to a customer, no unique configurations had been applied yet.
Data Load
2 Days
54x Improvement
53 min
SMP SQL
Server
21
Data Processing
4 days
6 hrs
< 32 min
27 min
APS 3 Node APS 6 Node
Dell - Internal Use - Confidential
193x Improvement
SMP SQL
Server
Query / Reporting / Insight
2 hours
23 min
< 16 min
APS 3 Node APS 6 Node
SMP SQL
Server
25x Improvement
< 6 min
< 3 min
APS 3 Node
APS 6 Node
Speeds, Feeds and Innovation
37x Smaller
28x Faster
27x Faster
16% Impact
From
2.5 Hours to
5.5 Minutes
7x Smaller
Daily Load
Compression
From 3.3 TB
To 486 GB
3.3TB to
90 GB
Backup
Compression
From
43 min to
90 sec
Minimal
concurrency
impact
Customer Queries
Concurrency
Two 6 Node APS appliances outperformed 19 Neteeza appliances using
real customer workloads
22
Dell - Internal Use - Confidential
Harmony with Hadoop
23
Dell - Internal Use - Confidential
Hadoop implementation challenges
Hadoop ecosystem
•
•
•
•
TSQL
Learn new skills
Manage
Maintain
Support
“New” data
data sources
sources
“New”
24
Move HDFS data into the warehouse
Dell - Internal Use - Confidential
Warehouse
Hadoop
“New” data
data sources
sources
“New”
Hadoop
ETL
Manage any data with polybase
Windows Azure
HDInsight
Hortonworks HDP
HDInsight
Windows Server
Cloudera CDH
Hortonworks HDP
Linux Server
Select… Result set
SQL Server
Parallel Data
Warehouse
PolyBase
Common language and
model
High Performance
Open
Cloud Ready
25
Dell - Internal Use - Confidential
Agenda
•
•
•
•
•
26
Dell - Internal Use - Confidential
APS in 30 Seconds
Dell Competencies & Big Data
Technology Primer
Use Cases
Support & Services
Enterprise Hub & Spoke Data Warehouse
Challenge
Worldwide data distribution
Results
• Customer needed to consolidate 150+ SQL Server
database servers used for business intelligence to
thousands of users worldwide.
• Designed hub & spoke architecture including master data
warehouse, LOB data marts, cubes and presentation layer
using SharePoint
• Customer needed a predictable growth strategy that
included ETL, architecture, collaborative portal, reporting
for thousands of users worldwide with near zero
downtime.
27
Dell - Internal Use - Confidential
Highlights
• Conference room pilot converted Oracle/Informatica
legacy loads to PDW in weeks
• Aligned with Microsoft for mature solution delivery
Health Care Reporting
Claims Reporting Efficiency Increased 400%
Challenge
Migrate multiple medical claims data stores to a
high-performance, scalable APS Appliance
Results
• Analyzed, migrated, and optimized 70+ undocumented data
flows from multiple sources
• Analyzed 125+ tables and developed a high-performance,
distributed schema to support reporting and warehousing
requirements
• Analyzed 100+ Reports and optimized schema and
procedures to support high-performance, scalable reporting
Highlights
• Nonprofit, 448 bed community hospital, providing a
complete range of medical and surgical services
• 7 Community Medical Clinics
• Medical Laboratories
• 100+ Medical Claims Data Analytics Reports
supporting business planning and administration
28
Dell - Internal Use - Confidential
IoT Use Case: APS + Cloud + Statistica + Dell Services
Modeling
APS
Peer
Comparison
Building
Optimize equipment*
Third Party
Data*
Azure
Azure ML
ID
Event Hubs
Equipment,
Utility, Sensors
(Dell / Intel)
29
Dell - Internal Use - Confidential
SQL /
HDInsight
Predictive
Analytics
Agenda
•
•
•
•
•
30
Dell - Internal Use - Confidential
APS in 30 Seconds
Dell Competencies & Big Data
Technology Primer
Use Cases
Support & Services
Microsoft Enhanced care for APS
Complete install
experience
Proactive services
Mission critical
response
Collaboration with
IHV and network
integration for
seamless experience
Semi-annual managed
upgrades and customer
workshops
30 minute response
time and as-required
problem resolution
support
Install
31
Dell - Internal Use - Confidential
Maintain
Restore
Dell White Glove Delivery & Support
Complete install
experience
•
•
•
Collaboration with
MSFT
Frictionless customer
install to data center
Jumpstart services
Install
32
Dell - Internal Use - Confidential
Support
•
•
ProSupport Plus (3 yr
minimum)
Hot spares: 9 drives per
JBOD, cable kits
Mission critical
response
•
•
•
Maintain
Specialized warranty
queue
First call to MSFT
Premier
4 hour parts
replacement
Restore
Dell APS Jumpstart Services for APS
Basic Jumpstart (~ 3 weeks)
• Training (3 days on-site)
• Training on-site for PDW & MPP development for
DBA’s and developers
• Discovery Workshops (4-5 days On-site)
• Interactive sessions to review current state and
future state goals with APS
• Architecture Design Review
• Prepare and deliver data warehouse technical
architecture and recommendations for
implementation based upon best practice and
Discovery Workshops
Customized Jumpstart
• Advanced APS workload development:
• Landing zone, Backup or DR configuration
• Schema, ETL, DW, or Reporting development
33
Dell - Internal Use - Confidential
• Implementation of hybrid Hadoop or Azure
integration
APS Implementation Roadmap
Appliance
Implementation
Discovery
Sustenance
•
Workshop
•
•
Platform Migration
•
Business Justification / ROI
Assessment vs. Alternatives
Microsoft /Dell APS StandUp Services (rack/stack)
•
Database Consolidation
•
Dell Jumpstart for APS
•
ETL Uplift
• Capacity Planning
•
Data Modeling
• Workload Management
•
Application Integration
Development
• Polybase Optimization
•
Dashboard / KPIs / Analytics
Modeling / Development
•
Reporting Development
•
Mobile, Predictive Analytics
•
BI / DW Assessment
• DBA Immersion
•
Readiness Assessment
•
Architectural Design Session
• Developer Knowledge
Transfer and Best
Practices
• Application Design &
Planning
• Table Geometries, Skills
Transfer
•
34
Application
Integration
Dell - Internal Use - Confidential
Dell Enhanced Jumpstart /
Pilot Implementation
• Dell & Microsoft Support
• Application Optimization
• Data Skew Review
• Appliance Health & Support
• Break/Fix Incidents
• New Version Rights
• Health Checks
Additional Resources
http://www.dell.com/microsoft
35
Dell - Internal Use - Confidential