Transcript Oracle RAC Installation

Oracle 9i Real Application Clusters
Oracle Partner Technical Services
Workshop
Agenda
• Overview
• Architecture
• Machine configuration
• How to install and gotchas
• Creation of the RAC instances
• Monitoring and Administration with OEM
• Tuning Recommendations
Seminar Objectives
 Explain the Architecture of Oracle Real
Application Clusters
 Show Technical People how to quickly set up
and use Oracle9i Real Application Clusters
What is Oracle 9i RAC?
 The 9i Oracle Parallel Server database!
 Application clustering
 Major changes in Oracle9i:
–
–
–
Scalability with full Cache Fusion
architecture
Improved availability
Improved manageability
Agenda
• Overview
• Architecture
• Machine configuration
• How to install and gotchas
• Creation of the RAC instances
• Monitoring and Administration with OEM
• Tuning Recommendations
Real Application Clusters
Architecture
Centralized
Management
Console
High Speed
Switch or
Interconnect
Network
Users
Low Latency Interconnect
VIA or Proprietary
No Single
Point Of Failure
Clustered
Database Servers
Hub or
Switch
Fabric
Storage Area Network
Mirrored Disk
Subsystem
Out-of-the-Box Transparent
Application Scalability
 In the past clustered databases scaled well
for specific types of applications
–
–
Data Warehouse
Parallel-enabled OLTP
 Oracle Real Application Clusters with Cache
Fusion can deliver transparent scalability to
all types of applications
NEW in Oracle9i :- Scalability
Features
 Full Cache Fusion
 Enhanced coordination of cache
management and Distributed Lock Manager
(DLM) through new Global Cache Service
(GCS) & Global Enqueue Service (GES)
 Enhanced IPC
 Lock simplification and automation
Cache Fusion Architecture
 Full Cache Fusion
–
–
–
Cache-to-cache data
shipping
Shared cache
eliminates
slow I/O
Enhanced IPC
 Allows flexible
and transparent
deployment
Users
Full Cache Fusion
 Oracle9i Cache Fusion increases performance and
scalability
–
–
Data is shipped directly over high speed interconnect
Minimize disk I/O
Node A
Node B
Request
Data Transfer
Database
buffers
Database
buffers
Database
Benefits of Cache Fusion
Without Cache Fusion
100
Block
access
time
(milliseconds)
20
1
0.01
With
Cache
Fusion
Block in
local cache
Block in
remote cache
Block
on disk
Cache Fusion Manages
Inter Instance Block Requests
 Readers and writers
accessing instance A
gain access to blocks
in instance B’s buffer
cache
 All types of block
contention and access
 Coordination by Global
Cache Service &
Global Enqueue
Service
Request
for Block
Cache A
Lock Status
Block in
Cache B
Read
Read
Write
Write
Read
Write
Read
Write
RAC Architecture Changes
 The background process BSP is no longer
used
 The following background processes have
changed in function and description in
release Oracle9i
–
–
–
LMS
LMON
LMD
Cache Coherency Model
 Data transfer between instances uses the
IPC transport
 Block resource and the GCS record
–
–
–
Resource modes
Resource roles
Past image history
 Cache fusion requires fewer messages and
steps to convert lock status
 Traditional hash locking can coexist with
cache coherency locks (!)
Enhanced IPC
 Global IPC service layer
–
–
Each requesting server process handles asynchronous
IPC
Efficient I/O implementation
 Remote memory operations for direct DMA
–
–
Exploit modern low latency interfaces
Reduced CPU usage
 Intelligent batching of messages
 New V$ tables for IPC statistics
Lock Simplification and Automation
 Automatic GCS/GES configuration
–
–
–
Automatic derivation of GCS configuration
No INIT.ORA lock parameters required
Improved lock efficiency and memory
management
 Dynamic lock affinity
–
–
GCS locks dynamically remastered reducing
locking overhead
Cache layer determines policy for remastering
Agenda
• Overview
• Architecture
• Machine configuration
• How to install and gotchas
• Creation of the RAC instances
• Monitoring and Administration with OEM
• Tuning Recommendations
Machine Configuration
 Hardware
- Supported on most vendor-clustered and nonvendor clustered platforms
 Operating System
- Supported on clustered and non-clustered O/S’s
(‘ClusterWare’ supplied by Oracle)
Machine Configuration
 You will need a Shared Disk Subsystem
- Can be anything that supports a raw device
- Must be addressable from all nodes
- The higher the speed the better (FDDI, GigaBit)
 You will need a 2nd NIC for the Inter Instance
Network
- Can be ‘any’ network connection
- The higher the speed the better (FDDI, GigaBit)
- If only 2 nodes, ensure that you are using a
crossover cable!
- The inter-instance network must be configured
(10.0.0.1, 192.168.0.1, etc) directly into the hosts
file.
- A dedicated hub will be needed if there are more
than 2 nodes.
Operating System
 Supported on all major cluster vendors
O/S’s
- Sun Clusterware
- IBM HA
- Compaq Tru64 CFS
 ‘ClusterWare’ supplied for Linux and Win2k
- Supported on Linux (Kernel 2.4.x) and Win2k
(SP2)
- Certified on SuSE 7.2, Red Hat Advanced Server
2.1 and Win2k (certain vendors only)
Agenda
• Overview
• Architecture
• Machine configuration
• How to install and gotchas
• Creation of the RAC instances
• Monitoring and Administration with OEM
• Tuning Recommendations
Installation




Install Windows 2000. No special requirements,
just ensure TCPIP networking is installed. Take
care at the NIC configuration screen, and
hardcode %WINNT%\system32\drivers\etc\hosts
with the IP address of each other node. Note the
ftp service isn’t default – PITA!
From the Disk 1 of the Oracle distribution (and
before installing) copy all of the \preinstall_rac
directory to a temporary location (C:\Temp)
Run Start -> Programs -> Administrative Tools ->
Computer Management, and navigate to Storage > Disk Management.
You will be presented with the following screen:-
Partitioning the disks
Disk Partitioning
Right click against the free space you want to
partition and choose ‘Create Logical Drive’
Disk Partitioning
You will be presented with the Partition Wizard, click
Next
Disk Partitioning
Select Logical Drive, click Next
Disk Partitioning
Choose the size of partition according to the size of data file
required, the “Oracle Recommended’s” are as follows:SYSTEM tablespace, 400M
SRVCFG parameter file, 10M
USERS tablespace, 120M
TEMP tablespace, 100M
UNDOTBS 1 tablespace, 312M
UNDOTBS 2 tablespace, 312M
EXAMPLE tablespace, 160M
CWMLITE (OLAP) tablespace, 100M
INDX tablespace, 70M
TOOLS tablespace, 12M
DRSYS tablespace, 90M
controlfile1, 110M
controlfile2, 110M
redo 1 instance 1, 120M
redo 2 instance 1, 120M
redo 1 instance 2, 120M
redo 2 instance 2, 120M
Enter the size and click Next
A word about the SRVCFG partition…
This partition (sometimes known as the
‘voting’ or ‘quorum’ disk) is required for the
administration and management of the whole
RAC cluster. It is therefore required for
installation regardless of all your other
partitions. Even if you don’t cut any
tablespace partitions at this point because
you don’t want any of the Oracle packaged
databases, you must partition SRVCFG.
Ensure that the size is at least 10 Mb,
however the name is not mandatory.
Disk Partitioning
Choose ‘Do not assign a drive letter…’ and click Next
Disk Partitioning
Choose ‘Do not format partition’ and click Next,
then click Finish
Attaching ‘Objects’ to partitions
Once you have partitioned the disks, navigate to
C:\temp\preinstall_rac\olm directory and run the
OracleOBJService at the command line as below:OracleOBJService /install
This will install the service to manage the object links in the
‘Services’ panel. Once done, from the same directory run the
GuiObjOracleService. You will be presented with the following
screen:-
Attaching ‘Objects’ to partitions
Right click against the Link Name, and name the partitions according to the
following table, but only if you wish to use pre-configured database types in
DBCA. The exceptions to this are the undo and redo files, which must end
accordingly.
Attaching ‘Objects’ to partitions
Symbolic Link Name...
Used for...
db_name_system1
SYSTEM tablespace
db_name_spfile1
Server parameter file
db_name_users1
USERS tablespace
db_name_temp1
TEMP tablespace
db_name_undotbs1
First UNDO tablespace
db_name_undotbs2
Second UNDO tablespace
db_name_example1
EXAMPLE tablespace
db_name_cwmlite1
CWMLITE tablespace
db_name_indx1
INDX tablespace
db_name_tools1
TOOLS tablespace
db_name_drsys1
DRSYS tablespace
db_name_control1
First control file
db_name_control2
Second control file
srvcfg
SRVM Configuration Repository
db_name_redo_thread_no.
Where thread is the thread ID of the
instance and number is the log number
(1or 2) for the instance.
Each instance must have two redo log files. If the database name is db, the link names for the
first instance should be:
db_redo1_1
db_redo1_2
The link names for the second instance should be:
db_redo2_1
db_redo2_2
Setting up the cluster
• Once all links are named, click Options – Commit, and close the
GuiOracleObjManager
• From \temp\preinstall_rac\clustersetup run clustersetup.exe. You will be
presented with the following screen, click next:-
Setting up the cluster
Select ‘Create a cluster’, click next
Setting up the cluster
Select the srvcfg partition, click next
Setting up the cluster
If VIA (Virtual Interface Architecture) is detected it can be
utilised, otherwise click Next
Setting up the cluster
Choose Private Network, click next
Setting up the cluster
Enter the network details, click next
Setting up the cluster
Unless you have other plans, click finish
Setting up the Cluster
Monitor the install to completion
Oracle RAC Installation
We are now ready for the Oracle Installation itself. Insert the
CD and run setup / autorun.
Oracle RAC Installation
Choose all other nodes in the cluster, click Next
Oracle RAC Installation
Choose your ORACLE_HOME, this will be the case for ALL nodes
Oracle RAC Installation
Choose Oracle 9i Database, click Next
Oracle RAC Installation
Choose Custom, click Next
Oracle RAC Installation
Then (and only then) will you be presented with the Real Application
Clusters option. Tick the box along with any other products you require,
and click next
Oracle RAC Installation
Choose alternate locations if required, click Next
Oracle RAC Installation
Choose yes if you want to use the DBCA, click next
Oracle RAC Installation
Check the summary, and click install
Oracle Installation
After install you will be presented with the ‘Configuration Tools’
screen….
Oracle RAC Installation
If you wish to use utilise an LDAP server choose yes,
otherwise choose No and click Next.
Oracle RAC Installation
Change the name of the listener if required, click Next
Oracle RAC Installation
Choose TCP, click Next
Oracle RAC Installation
Change the port if required, click ‘Next’
Oracle RAC Installation
Choose Yes if you would like to configure another Listener, otherwise
choose No, click next.
Oracle RAC Installation
You can leave naming methods as they are, or configure other methods at
this stage.
Oracle RAC Installation
…finally you are presented with the ‘Oracle Net Configuration
Complete’ screen.
Agenda
• Overview
• Architecture
• Machine configuration
• How to install and gotchas
• Creation of the RAC instances
• Monitoring and Administration with OEM
• Tuning Recommendations
Creating the database
The Configuration Assistant will then present the Database Configuration
Assistant Screen, choose Create a database, click Next.
Creating the database
Choose all nodes other than the the one you are installing on, click Next
Ooops!
If at this point you receive an exception referring to the
GSD or Global Services Daemon, check the Services
panel for 2 Oracle Services, namely the
OracleCMService and the OracleGSDService. Both of
these should have been installed, and started by the
Oracle RAC install. Stop and restart them if needed.
Creating the database
Choose the database you wish to create, click Next
Creating the database
Choose the database name, click Next
Creating the database
Choose any of the features you require, click
Next
Creating the database
Choose either dedicated or multithreaded (shared) server, click
Next
Creating the database
Change initialisation parameters, character sets, sizing and
archive options if required, then click on File locations
Creating the database
Ensure the SPFILE points to a raw partition if you intend to
use one, then click Next
Creating the database
Navigate to the Datafiles tree
Creating the database
All the raw partitions should show as named and linked
‘objects’
Creating the database
Redo should show up as 1_1, 2_1 etc, click Next
Creating the database
Choose your creation options, and click Finish
Creating the database
Monitor to completion.
Transparent Application Failover
(TAF)
 Transparent Application Failover compliments all levels of
availability hierarchy
– Applications and users are automatically and transparently
reconnected to another system
– Applications and queries continue uninterrupted
– Login context maintained
Computer
A
Computer
B
Node A in an
RAC cluster
fails, users are
migrated
Computer
A
Computer
B
Transparent Application Failover
 It’s provided by Oracle Net
–
–
Since 8.0.6
Does not depend on RAC





Single instance
HA Cluster
Replicated Database
Standby Database (Data Guard)
OPS or RAC
 Maintains session as long as service is
available
Transparent Application Failover
 Automatically reconnects
 Detects broken connection upon SQL call
 2 methods
–
–
Basic : try to re-establish the connection
Preconnect : simultaneously open a connection to the
secondary server (RAC, Data Guard or replicated
database)
 2 types of failovers
–
–
SESSION – Failover the session only
SELECT – Failover the session and continue to fetch if
a cursor was open
Transparent Application Failover
 What TAF does / does not do
–
–
–
–
–
If you are (example) mid-transaction (the UPDATE has
been done but the COMMIT has not) and you failover
with TAF, your application will need to issue a
ROLLBACK before continuing (exception handling for
ORA-254xx errors)
If you are (example) mid-INSERT, the transaction will
rollback automatically (exception handling again)
ALTER SESSION commands are not preserved
PL/SQL session states are not preserved
Both session characteristics and PL/SQL session state
can be called back using the OCI8 or 9 Callback function
Transparent Application Failover
oow=(DESCRIPTION =
(failover=on)
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = 129.157.220.100)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = 129.157.220.101)(PORT = 1521))
)
(CONNECT_DATA =
(SERVICE_NAME = RAC)
(failover_mode = (type = select)(method=basic)
(retries=5)(delay=2)
)
)
)
Agenda
• Overview
• Architecture
• Machine configuration
• How to install and gotchas
• Creation of the RAC instances
• Monitoring and Administration with OEM
• Tuning Recommendations
Real Application Clusters
Manageability
 Single system Image
–
–
–
Single Oracle Database
One virtual system to configure and manage
Single management console
 Cluster-wide monitoring and diagnostics
Real Application Clusters
Manageability
 Improved cluster-aware tools infrastructure
for management
–
–
–
–
–
–
Oracle Universal Installer (OUI)
Enterprise Manager (OEM)
Database Configuration Assistant (DBCA)
Net8 Assistant (NETMGR)
Recovery Manager (RMAN)
Command line interface (SRVCTL)
Improved Configuration
 Centralized, persistent configuration storage
–
–
Eliminates consistency problems with the per node text
file-based Parallel Server configuration in prior versions
Utilities to migrate previous Parallel Server configurations
 Enhanced DBCA and NETCA / NETMGR
functionality
–
–
Robustness
Capability to add and delete instances
Real Application Cluster
Configuration
Windows
cluster
UNIX
cluster
Pre-Oracle9i
Registry
Windows
or UNIX
cluster
Text
file
Oracle9i
Raw
device
Improved Administration
 Dynamic cluster node addition and deletion
(DBCA)
–
Add/delete a node in both the system and the
database
 Better integration with Oracle Enterprise
Manager
–
–
View and update server side initialization
parameter file (SPFILE)
Assign UNDO segments to database instances
Shared Initialization Parameter File
Node 1
Raw device
ORACLE_HOME=
/hdisk1
ORACLE_SID=orac1
Node 2
ORACLE_HOME=
/hdisk1
ORACLE_SID=orac2
Local
disk
Local
disk
/hdisk1/dbs/initorac1.ora
/hdisk1/dbs/initorac2.ora
SPFILE=/dev/rdisk1/spfile
SPFILE=/dev/rdisk1/spfile
/dev/rdisk1/spfile
orac1.instance_name=orac1
orac2.instance_name=orac2
…
Improved Administration
 Better integration with OEM (Contd.)
–
Enhanced monitoring capability and events
available with OEM and performance packs




–
Cache Fusion statistics
Statistics for interconnect block traffic
Statistics for the entire database or per instance
New OEM events associated with new statistics
Improved manageability, as OEM commands
are passed to the Intelligent Agent (Oracles
SNMP agent) which in turn passes commands
to SRVCTL (the new OPSCTL) to manage the
whole cluster.
New and Modified (opsctl) srvctl
Commands
 Start all instances in the cluster
srvctl start db -d PTSRAC
 Stop all instances in the cluster
srvctl stop db -d PTSRAC
 Stop all instances with the immediate option
srvctl stop db –d PTSRAC –o immediate
 Start all instances but do not open them
srvctl start db –d PTSRAC –o mount
 Add a new instance to the cluster
srvctl add instance -d PTSRAC –i
PTSRAC3 –n dl380c
Agenda
• Overview
• Architecture
• Machine configuration
• How to install and gotchas
• Creation of the RAC instances
• Monitoring and Administration with OEM
• Tuning Recommendations
Tuning Recommendations
 New tuning guide for RAC
 Consider a certain level of Data Partitioning
(also use Oracle Partitions for tables and
indexes whenever relevant)
 Assigning 1:N locks to data files with the
GC_FILES_TO_LOCKS parameter turns off
Cache Fusion
Tuning Recommendations (2)
 Automatic Segment Management
CREATE TABLESPACE tbs_1
DATAFILE ’tbs1.dbf’ EXTENT MANAGEMENT LOCAL
SEGMENT SPACE MANAGEMENT AUTO;
 Possible to have a separate sequence cache
per instance
Oracle9i Performance and Scalability
 The business environment today:
–
–
–
Internet users and transaction volumes grow
extremely rapidly
Data warehouse systems must support very
large data volumes
Server consolidation drives the need for large
scale systems
 Oracle9i provides the ability to deploy as you
grow