Transcript Scalability and Reliability using Oracle 10g RAC

Scalability and Reliability using
Oracle Real Application Clusters
Suresh Chaganti
Syed Asharaf
Agenda



Introductions
Real Application Clusters – Defined
Architecture of Real Application Cluster
–
–
–





Memory Structures
Technology Stack Components
Storage Considerations
Pre-Installation Tasks
Installing Real Application Clusters
Case Study I – Benchmark Statistics with Oracle 10g
Case Study II – Migrating to RAC on 11i
Closing Thoughts
About Cozent
Cozent is Oracle services firm based out of
Naperville, IL .
Cozent specializes in implementation of Oracle
E-Business suite, design of High Availability
Solutions using Oracle RAC and support
services for Oracle Databases and EBusiness Suite.
Real Application Clusters




A cluster comprises multiple interconnected computers or servers that
appear as if they are one server to end users and applications.
Single-instance
– one-to-one relationship between the Oracle database and the
instance.
RAC Environments
– one-to-many relationship between the database and instances
– combined processing power of the multiple servers can provide
greater throughput ,scalability and availability
Oracle Clusterware is a portable cluster management solution that is
integrated with the Oracle database.
Real Application Clusters - Architecture
•
•
•
•
•
•
Two or more database instances that each contain memory
structures and background processes
A RAC database is a logically or physically shared everything
database
All datafiles, control files, PFILEs, and redo log files in RAC
environments must reside on cluster-aware shared disks
At least one additional thread of redo for each instance
An instance-specific undo tablespace
Must have private interconnect between cluster nodes
The Real Application Clusters Memory Structures



Each instance has a buffer cache in its System
Global Area (SGA).
Using Cache Fusion, RAC environments logically
combine each instance's buffer cache to enable the
instances to process data as if the data resided on a
logically combined, single cache.
The SGA size requirements for RAC are greater than
the SGA requirements for single-instance Oracle
databases due to Cache Fusion
RAC High Availability Components
•
Voting Disk
•
•
•
•
Manages cluster membership by way of a health check
RAC uses the voting disk to determine which instances are
members of a cluster
must reside on shared disk
Oracle Cluster Registry (OCR)
•
•
Maintains cluster configuration information as well as
configuration information about any cluster database within
the cluster
must reside on shared disk
Installing Oracle Clusterware and Real Application Clusters
Pre-Installation
–
–
–
–
user equivalence
network connectivity
directory and file permissions
Use Cluster verification Utility to check requirements
Two-phase installation
Oracle Clusterware Installation
–
–
use Oracle Universal Installer (OUI) to install Oracle Clusterware
Installed in separate home ORA_CRS_HOME
Real Application Clusters Installation and Database Creation
–
–
–
Use Oracle Universal Installer (OUI) to install RDBMS
Install RDBMS in separate ORACLE_HOME
Use DBCA to Create and configure Databases
Storage considerations

Types of Files to consider
•
•
•
•
•
Voting Disk
Oracle Cluster Registry
Shared Database files
Oracle Software
Recovery
•
Can use any combination of
File Systems shown on the
right
•
For each filesystem choice,
the disks/devices need to be
partitioned and configured.
Please see manual for full
details
Certification Matrix



Leading Edge technology, Pay good attention
to Certification Matrix
OCFS2 is not certified yet for Linux AS 4 -64
bit, Itanium, Power and s390x ( at the time of
writing this)
Oracle may not have tested ALL Hardware
configurations / Vendors
Pre-Installation Tasks





Use CVU to confirm that all pre-installation requirements are
met
–
/dev/dvdrom/crs/Disk1/cluvfy/runcluvfy.sh stage -pre crsinst -n node1,node2
–
cluvfy -help ( to display help and command info)
Xserver availability ( to run OUI in a GUI mode)
Ensure required Operating groups (dba,oinstall) and users (
oracle,nobody)
Ensure oracle user can ssh to local node and remote nodes
without prompting for password. This is the single most
important thing to consider
Ensure that date is set on the cluster nodes using NTP or some
common mechanism. If the date settings on the nodes vary, the
installation may fail during copy of binaries to the remote nodes
Pre-Installation Tasks - Continued

Ensure enough RAM , SWAP
and TMP space
–




Installation specific. Refer to
installation manual.
2 Network adapters one each
for public connection and
Private interconnect between
the nodes
Configure /etc/hosts to ensure
entries for private, public and
vip hostnames for ALL nodes
are available in each of the
nodes.
Ensure that all rpms required
are installed.
Check latest release notes for
errata
Pre-Installation Tasks - Continued



Configure Kernel Parameters as per
installation guide
Create ORACLE_BASE, ORACLE_HOME
directories
Configure oracle user’s environment
Final Checks
Run Cluster Verification utility after completing the pre-installation tasks to ensure that following
aspects are covered

Node Reachability: All of the specified nodes are reachable from the local node.

User Equivalence: Required user equivalence exists on all of the specified nodes.

Node Connectivity: Connectivity exists between all the specified nodes through the public and private
network interconnections, and at least one subnet exists that connects each node and contains public
network interfaces that are suitable for use as virtual IPs (VIPs).

Administrative Privileges: The oracle user has proper administrative privileges to install Oracle
Clusterware on the specified nodes.

Shared Storage Accessibility: The OCR device and voting disk are shared across all the specified
nodes.

System Requirements: All system requirements are met for installing Oracle Clusterware software,
including kernel version, kernel parameters, memory, swap directory space, temporary directory space,
and required users and groups.

Kernel Packages: All required operating system software packages are installed.

Node Applications: The virtual IP (VIP), Oracle Notification Service (ONS) and Global Service Daemon
(GSD) node applications are functioning on each node.
Installing Clusterware and RDBMS




Very straightforward with Oracle Universal Installer if
all the pre-installation requirements are met
Need to run in 2 phases, first to install Clusterware
and second to install Oracle RAC RDBMS software
Run DBCA to configure ASM instance ( if the storage
option chosen is ASM)
Run DBCA to create Database
Case Study – I
Benchmark Statistics for 10g R2 on RAC
Background
Client is in healthcare industry analyzing high
volume of claims, eligibility information along
with transaction processing application for
disease management.
Before and After RAC
Environment before RAC
–
–
–
Custom application running on Single node Windows based Oracle 9i
RDBMS
Reporting and Datawarehouse requirements met from same Database
Performance and Scalability issues
Environment after RAC
– 3 Node 10g R2 RAC system on Linux AS 3 running on 2 way AMD 64 bit
processor machines
– 9 GB main memory on all 3 nodes
– 4.5 GB SGA configured
– Instances separated for Production and Datawarehouse
– Configured ASM for database Files
Benchmark Statistics on Stress Testing




Used TPC-C compliant set of data
Simulates a real workload by allowing a large
number of concurrent connections performing userdefined transactions simultaneously.
Used to gauge the server performance under a CPU
and memory intensive Oracle database workload
middle 80% of transactions are used for the
benchmark readings
Test Configuration




executed at several iterations of 5, 10, 25,
50, 100, 150, 200 and 250 sessions
Each iteration executed 100,000
transactions.
Initially executed on a single node at a time
The test was repeated and sessions
connected to the three available nodes
depending on load.
Results




the server handled various
database loads optimally and
scaled very well.
The three node RAC had a
steady database throughput
and was able to handle user
load up to 800 concurrent users
( Configuration Limit).
Load larger than 800 users
resulted in error because of
database configuration limit of
800 users.
The database could be altered
if needed to increase the user
limit.
Session
TPS
5
4744
10
6735
25
6776
50
6735
100
6701
150
6617
200
6583
250
6575
300
6567
500
6531
Case Study - II
Migrating to 11i RAC on 9i Database
Background
Client is in discreet manufacturing industry with
plants all over North America. The
requirements were for a 24 X 7 environment
running on an Oracle 11i (11.5.8) E-Business
Suite application
Environment and Configuration




Oracle 11.5.8 E-Business Suite application
Oracle database version 9.2.0.7
Oracle Clustered File System V1
4 node environment
–
–
–
–
–
–
2 Database nodes
2 Application Server nodes
Concurrent Manager tier on 2 database nodes
Reports and Forms tier on 2 Application Server nodes
Parallel Concurrent Processing configured
Virtual host name configured
Key Steps in Implementing RAC on 11i












Configure SAN storage with shared disks
Install and configure OCFS
Install Oracle Cluster Manager
Install Oracle 9i (9.2.0.4) and upgrade database to latest release
Cloned production instance to new servers
Apply all pre-request’s patches to the Cloned Production instance for
converting to RAC. ( ex: ATG_PF.H , AD.I etc ).
Converted database to RAC
Configured application tier for RAC
Created database instance 2 on Node 2.
Created application tier 2 on Node 2.
Configured virtual host and configured application to use the virtual host name.
Configured PCP ( Parallel Concurrent Processing )
Critical Step – Enabling Autoconfig






Copy the appsutil, appsoui and oui22 directories from the
OLD_ORACLE_HOME to the NEW_ ORACLE_HOME.
Set environment variables ORACLE_HOME, LD_LIBRARY_PATH and
TNS_ADMIN to point to NEW_ ORACLE_HOME.Set ORACLE_SID
variable to point to instance name running on this database node.
Shutdown the instance and database listener.
Start the instance by using parameter file as init<sid.ora>. Start the
database listener.
Generate instance specific xml file using
NEW_ORACLE_HOME/appsutil/bin
adbldxml.sh tier=db appsuser=<APPSuser>
appspasswd=<APPSpwd>
Execute the AutoConfig utility (adconfig.sh) on database tier from
NEW_ORACLE_HOME/appsutil/bin. Verify the log file located at
NEW_ORACLE_HOME>/appsutil/log/<context_name>/<MMDDhhmm
Critical Step – Converting database to RAC

Execute AutoConfig utility on the application tier. Verify the AutoConfig log file located at
$APPL_TOP/admin/<context_name>/log/<MMDDhhmm>.

Execute $AD_TOP/bin/admkappsutil.pl to generate appsutil.zip for the database tier.
Transfer this appsutil.zip to database tier in the NEW_ORACLE_HOME.
Unzip this file to create appsutil directory in the NEW_ORACLE_HOME.
Execute the AutoConfig on database tier from
NEW_ORACLE_HOME/appsutil/<context_name>/scripts by using adautocfg.sh
Verify the AutoConfig log file located in the NEW_ORACLE_HOME
NEW_ORACLE_HOME>/appsutil/log/<context_name>/<MMDDhhmm.
Execute the following command to accumulate all the information about the instance
NEW_ORACLE_HOME/appsutil/scripts/<context_name>/perl adpreclone.pl database
Shutdown the instance
Ensure that listener process on database tier is also stopped.
Execute the following from the NEW_ORACLE_HOME/appsutil/clone/bin.
perl adcfgclone.pl database









Answer the prompted questions
Critical Step – Converting Database to
RAC ( Contd )
The process will
 Create instance specific context file
 Create instance specific environment file.
 Create RAC parameter specific init.ora file.
 Recreate the control files.
 Create redo log threads for other instances in the cluster.
 Create undo tablespaces for other instances in the cluster.
 Execute AutoConfig on the Database tier.
 Start the instance and database listener on the local host.
Critical Step - Configure Applications Environment for RAC
–
Execute the AutoConfig by using
$AD_TOP/bin/adconfig.sh contextfile=$APPL_TOP/admin/<context_file>.
–
Verify the AutoConfig log located at
$APPL_TOP/admin/<context_name>/log/<MMDDhhmm>for errors.
Source the environment by using the latest environment file generated.
–
Verify the tnsnames.ora, listener.ora files located in the 8.0.6
ORACLE_HOME at $ORACLE_HOME/network/admin and
$IAS_ORACLE_HOME/network/admin. Ensure that the correct tns
aliases are generated for load balance and fail over.
–
Verify the dbc file located at $FND_SECURE. Ensure that the parameter
APPS_JDBC_URL is configured with all instances in the environment
and load_balance is set to ON.
Critical Step – Load Balancing Oracle Apps Environment

Run the Context Editor through Oracle Applications Manager interface to set the value of
"Tools OH TWO_TASK","iAS OH TWO_TASK" and "Apps JDBC Connect Alias"

To load balance the forms based applications database connections, set the value of
"Tools OH TWO_TASK" to point to the <database_name>_806_balance alias generated
in the tnsnames.ora file.

To load balance the self-service applications database connections, set the value of iAS
OH TWO_TASK" and "Apps JDBC Connect Alias" to point to the
<database_name>_balance alias generated in the tnsnames.ora file.

Execute AutoConfig by using
$AD_TOP/bin/adconfig.sh contextfile=$APPL_TOP/admin/<context_file>
Restart the applications processes by using the latest scripts generated after AutoConfig
execution.
Set profile option "Application Database Id" to dbc file name generated at
$FND_TOP/secure.
Update session_cookie_name in table ICX_PARAMETERS to <service_name>.



Critical Step – Configure Parallel Concurrent Processing
Prerequisites for setting up PCP

Configure the Application to use GSM (Generic Service Management), The
GSM profile options should be YES
Setup PCP

Ensure that all pre-requisite patches are applied.

Edit the applications context file through Oracle Applications Manager
interface and set the value of APPLDCP=ON and "Concurrent Manager
TWO_TASK" value to instance alias.

Execute AutoConfig by using
$COMMON_TOP/admin/scripts/<context_name>/adautocfg.sh on all
concurrent nodes.

Source the application environment by using $APPL_TOP/APPSORA.env

Check the configuration files tnsnames.ora and listener.ora located under
8.0.6 ORACLE_HOME at $ORACLE_HOME /network/admin/<context>.
Ensure that you have information of all the other concurrent nodes for
FNDSM and FNDFS entries.
Critical Step – Configure Parallel Concurrent Processing
( Contd )
Setup PCP

Restart the application listener processes on each application node.

Logon to Oracle E-Business Suite 11i Applications using SYSADMIN
in login and System Administrator Responsibility. Navigate to Install
> Nodes screen and ensure that each node in the cluster is
registered.

Navigate to Concurrent > Manager > Define screen, and set up the
primary and secondary node names for all the concurrent managers
according to the desired configuration for each nodes workload. The
Internal Concurrent Manager should be defined on the primary PCP
node only.

Set the $APPLCSF environment variable on all the CP nodes
pointing to a log directory on a shared file system
Closing Thoughts



Lot of choices to be made. Analyze and
decide what best fits your environment
Pay close attention to certification matrix,
documentation and release notes
Expect some issues with any configuration
Thank You !
For any questions/ Comments please contact
[email protected]
[email protected]