Transcript Sonoma_2008_Tues_sonoma_2008_0408 Oracle

<Insert Picture Here>
Infiniband enables scalable Real Application
Clusters – Update Spring 2008
Sumanta Chatterjee, Oracle
Richard Frank, Oracle
What is Oracle Real Application Clusters (RAC) Database?
Public Network to Grid Computing Nodes
Database
Instance 1
Instance 2
SGA 1
SGA 2
Instance 3
SGA 3
Instance 4
SGA 4
Private Network
• Multiple Instances
• One Database
• SGA database memory of all instances aggregated and appears as
one single database to applications through Cache Fusion.
…
RAC for SAP Benchmark Results
Scalability
40,000
40,000
35,000
35,000
30,000
30,000
25,000
25,000
20,000
20,000
15,000
15,000
10,000
10,000
5,000
5,000
0
0
1 Node
2 Nodes
3 Nodes
Demonstrated Scalability
4 Nodes
Ideal (100%) Scalability
5 Nodes
Advantages of RAC
• Performance
Increase performance of a RAC database by adding
additional servers to the cluster.
• Fault Tolerance
A RAC database is made up of multiple instance.
While performance may degrade, loss of an
instance does not bring down the entire database.
• Scalability
Scale a RAC database by adding instances to the
cluster database.
Shifting Trend in Deployment Paradigm
Application Tier on Commodity servers
Application and Database on
Same SMP Server
Monolithic SMP
•Application
•Database
Past
Database Tier
Mixed Configuration
•Commodity Application Servers
•SMP Database Servers
Present
Application Tier on Commodity servers
Database Tier on Commodity Servers
Grid Computing
•All Commodity Servers
Future
Commodity Cluster Requires Unified
Fabric for efficient scalable IPC +
Storage I/O
• RDS / IB shows significant real world
application performance gains
•
•
•
•
•
•
•
•
50% less CPU than IP over IB, UDP
½ latency of UDP (no user-mode acks)
50% faster cache to cache Oracle block throughput (ping)
Scales well beyond GE (600+ mbytes – ran out of CPU)
Minimal Oracle code change
Supports fail-over across and within HCAs
Certified for 16 nodes (64 processors)
GA in 10g r2 (10.2.0.3).
Current Status
• Several TPC-H benchmarks with RDS and SRP
• Large scale deployments at several Oracle customer
sites.
• Many pilot projects are in progress.
• Folks waiting for RDS on OFED
• 16 nodes Oracle 11G RAC certification of OFED
1.2.5.5 submitted for audit
• Voltaire and Qlogic have completed platform
certification. Audit in progress.
• Certification on Unix in progress
RDS- Communication model
• Works well with existing IPC clients—
• Parallel Query communications
• Buffer cache fusion
• Working on providing support for additional clients
with RDMA plus atomic operations
• We expect significant performance improvements with
RDMA
• With Atomics, even greater scalabilities and
performance can be gained.
• Incentives for simple NICs to add RDMA + Atomics
RDS - evolution
• RDS v2 with b-copy send, rev in OFED 1.2.5.5
• New features in RDS v3 available in OFED 1.3
• supports RDMA read + RDMA write
• Introduces cmsgs for asynchronous operation submit and
completion notifications
• Large data transfers – presently up to 1 MB. Will go up to 8
MB
RDS v4
• Plans for RDS v4
•
•
•
•
Masked fetch_and_add
Masked compare_and_swap
Zero copy completions via cmsg
RDS V4 will also be more portable - we will work to abstract
out the generic RDS operations from O/S primitive support
and network operations. A platform that provides the O/S +
network primitives library - should be able to take all the
generic RDS code - as is.
RDS Compatibility
• Linux: request to all IB vendors—
• Please ensure compatibility across HCAs, switches
• Ideally RDS driver in OFED ported to all platforms
• Advantages include:
• One code body– wider testing
• Interoperability across platforms
• Towards this end, we plan to:
• Abstract RDS protocol driver generically (OS, RDMA)