Inside the Siebel Server
Download
Report
Transcript Inside the Siebel Server
Siebel 7 Performance and Scalability
Inside the Siebel Server
Richard Sands
Siebel Expert Services
©Siebel Systems 2003 – Do not distribute or re-use without permission
Siebel 7 Performance and Scalability
Inside the Siebel Server
Introduction to Siebel Architecture
How components scale
Within servers
Across servers
Siebel Performance
Hints
Monitoring
Diagnosis
Performance and Scalability
Architecture Overview
Component Scalability
Scalability Across Components
Network Scalability
Performance Optimization
Performance Management
Siebel 7 Infrastructure Overview
Wireless Web
Browser User Interface
Connected Web User
(Employee)
Connected Web User
(External)
Browser User Interface
Browser User Interface
PDA
Mobile (Disconnected)
Web User
Browser User Interface
Object
Manager
WAP Gateway
Server
Data Manager
Web Server(s)
Siebel Web Server
Extension
SIEB
SYNC
Local
DB
Gateway Name Server
Central
Load Dispatch
Balancer
Siebel Remote
External
Applications
Siebel Enterprise Server
Voice
Interaction
Siebel
eAI
Siebel
Replication
Object Manager
Data Manager
Object Manager
Data Manager
Regional
Siebel
DB Server
Central Siebel DB Server
Email
Interaction
Major Client Types
All accessed through a browser
High Interactivity (Employee facing)
Very demanding on browser
Can only run on strictly defined browser configurations
Rich user interface
Standard Interactivity (Customer facing)
Less demanding on browser
Can run on wide variety of browsers
Standard web user interface
Mobile Client
Has local copy of Siebel database
User-specific subset of data
Local server functionality
Quasi-web server
Business Object and Data Manager functionality
Uses High Interactivity interface
Siebel Enterprise Server – SWSE
Web Server
SWSE
Gateway Server
Gateway
Name Server
Resonate Central
Dispatch
Enterprise Server
Siebel Server
Component
Siebel Server
Component
Siebel Web Server
Extensions
Web Server Plug-In
Manages
communications to
Siebel Enterprise
Includes cache for static
files (images, etc)
Siebel Enterprise Server – Gateway Server
Serves as a single entry
point
Includes Siebel Gateway
Name Server
Web Server
SWSE
Gateway Server
Gateway
Name Server
Resonate Central
Dispatch
Siebel Enterprise
configuration data (static)
Siebel Enterprise
operations data (dynamic)
Optionally includes
Resonate Central
Dispatch
Load balancing
Enterprise Server
Siebel Server
Component
Siebel Server
Component
Siebel Enterprise Server – Gateway Name Server
Holds Enterprise
Configuration
Stores component
definitions, parameters,
and connectivity
information
Web Server
SWSE
Gateway Server
Gateway
Name Server
Resonate Central
Dispatch
Enterprise Server
Siebel Server
Component
Siebel Server
Component
Stored in siebns.dat file
Provides Connectivity
information when
Resonate not used
Dynamically registers
Siebel Server and
component availability
Siebel Enterprise Server – Resonate Central Dispatch
Web Server
SWSE
Gateway Server
Gateway
Name Server
Resonate Central
Dispatch
Enterprise Server
Siebel Server
Component
Siebel Server
Component
Load Balances Object
Manager Components
Provides resilience for
load balanced
components
Provides session
management
Simplifies firewall
configuration
Not always needed
Architecture Overview – Siebel Server
Web Server
SWSE
Gateway Server
Gateway
Name Server
Resonate Central
Dispatch
Enterprise Server
Siebel Server
Component
Siebel Server
Component
Framework for running
server components
Obtains configuration
information from the
Gateway Name Server
Runs as a Windows
service
Siebel Enterprise Server
is a logical grouping of
Siebel Servers
Architecture Overview – Server Components
Web Server
SWSE
Gateway Server
Gateway
Name Server
Resonate Central
Dispatch
Enterprise Server
Siebel Server
Siebel Server
Component
Component
A type of program, executed
as a task
Examples:
Object Manager Handling client
requests
Workflow Process
Manager –
Processing
Workflows
File System Manager Handling access to
Siebel File System
Architecture Overview – Server Component Types
Different types of component run in different ways:
Background
Background mode components execute tasks to perform background
operations for the Siebel Server. After a background mode component task
starts, it runs until you explicitly stop the task, or until the Siebel Server itself
is shut down.
Interactive
Interactive mode components start tasks automatically in response to client
requests. Interactive mode component tasks execute for as long as the client
maintains the session, and end when the client disconnects.
Batch
Batch mode components execute tasks in response to requests. Batch mode
component tasks execute until they finish processing.
Internal batch tasks are initiated by other Siebel Server tasks
External batch tasks are initiated by Server Manager
Architecture Overview – Component Execution Platforms
Different types of component execute in different platforms:
Single Threaded
Single threaded components have one execution stream per process. So
each operating system process supports a single Siebel Task.
i.e. EIM
Multi-threaded
Multi-threaded components have multiple execution streams within a single
process.. So each operating system process can support multiple Siebel
Tasks.
i.e. Object Managers
Performance and Scalability
Architecture Overview
Component Scalability
Scalability Across Components
Network Scalability
Performance Optimization
Performance Management
Component Scalability
How Components Scale in a Siebel 7 Enterprise
Scaling within a server
Multi-threaded components
Single-threaded components
Scaling across servers
Load balancing
Resonate central dispatch
Load sharing
Server Request Broker
Server Request Processor
Scaling Within a Siebel Server
Single Threaded Components
Create multiple processes (tasks)
Some components are limited
i.e.
Transaction Processor – max 1 per server
Workflow Monitor Agent – max 1 per policy group per Enterprise
Can be started manually, through Server Request Broker, or
automatically (‘Default Tasks’ parameter)
Multi-Threaded Components
Create multiple threads &/or processes (tasks)
Control distribution through component parameters
Multi-Threaded Components
Can have multiple processes as well as multiple threads
Important to control ratio of threads to processes
Can have major impact on performance
Determined primarily by rate of switches between threads
100:1 good starting point for Client Object Managers
Assumes 30sec think time
For 15 sec think time use 50:1
Can set additional processes to spawn on demand
Will always start minimum number specified
Will start additional processes as needed to maintain process:thread
ratio
Limit on maximum number of processes
Memory Scalability
Multi-Process, Multi-Threaded model
All threads in a process share the same memory space
Each Process has a separate memory space
No single process needs a large memory space
Operating system manages memory allocation
If a single process needs more than 1GB there’s normally something
wrong
No need for large process memory model (‘/3GB’ switch)
Can use Microsoft PAE for access to large memory capacities
Operating system can allocate memory to each process
Multi-Threaded Component Parameters
Typically set per component
Maximum number of tasks (MaxTasks)
Maximum number of Tasks per component per server
One thread per task
Some additional background “system” threads - not counted by MaxTasks
Maximum number of Multi-Threaded servers (MaxMTServers)
An MTServer is a multi-threaded component process
This defines the maximum number of MTServers per component per server
Minimum number of Multi-Threaded servers (MinMTServers)
This defines the minimum number of MTServers per component per server
Sets the number of MTServers started on server startup
Configuring the Object Managers
Set MaxTasks = peak (concurrent users + anonymous users)
Anonymous users are used for login screens before user authenticates
Typically set to 10%-15% of concurrent user count
Set MaxMTServers = MaxTasks / 100
An MTServer is equivalent to single instance of Object Manager
100 : 1 ratio is assuming “average” 30 second think time between user
operations
If average user think time is 15 seconds then ratio is 50 : 1 ( 50% of 100:1)
If average user think time is 60 seconds then ratio is 200 : 1 (200% of 100:1)
Set MinMTServers = MaxMTServers
Setting MinMT Servers < MaxMTServers may cause delay of service for
“new” users as MTServer gets initialized.
Multi-Threaded Component Parameter Example
Object Manager configuration for 800 Call Center users
Concurrent Users
800 Call Center Users
Object Manager Level
15%
MaxTasks
920
1000
100:1
MaxMTServers
10
MinMTServers
10
Anonymous Users
120
Round up to maintain
100:1 ratio
Scaling Across Siebel Servers
Depends on how component task initiated
Component Group Assignment
Which servers a component is available on
Object Managers
Load balanced (Resonate Central Dispatch)
Not valid for eConfigurator Object Manager
Siebel Remote
Manual allocation of mobile client to Siebel Server
Server Requests
Server Request Broker &/or Server Request Processor
Manual Requests (start task)
Manually specified server
Special cases
CTI
eConfigurator
Multi-Threaded Component Scalability
Vertical
Scalability
Sales
Object
Manager
Sales
Object
Manager
Sales
Object
Manager
Sales
Object
Manager
Sales
Object
Manager
Sales
Object
Manager
Sales
Object
Manager
Sales
Object
Manager
Sales
Object
Manager
Siebel Server
Enterprise Server
Siebel Server
Siebel Server
Horizontal Scalability
Load Balancing
Web
Client
Web
Client
Web
Client
Web
Client
Web
Client
Web
Client
Load Balancing
Web Server + SWSE
Web Server + SWSE
Resonate Load Balancing
Thread
Process
Server
Sales Object
Manager
Sales Object
Manager
Siebel Server
Sales Object
Manager
Sales Object
Manager
Siebel Server
Enterprise Server
Performance and Scalability
Architecture Overview
Component Scalability
Scalability Across Components
Network Scalability
Performance Optimization
Performance Management
Server Request Broker & Server Request Processor
Server Request Broker (SRBroker)
Used to start synchronous Siebel Server tasks
Server Request Broker & Server Manager are the only components
which directly start tasks.
New in Siebel 7
Replaces Server Request Manager (SRMSynch) in Siebel 2000
Background component
Multi-threaded component
Need to set MaxTasks accordingly
Server Request Processor (SRProc)
Used to start asynchronous Siebel Server tasks
Manages queued requests
Calls SRBroker to manage task execution
Background component
Server Request Broker
Accepts requests from other Components
Will try to service request locally
If component is available on same Siebel Server then this is always
used
If not available locally then will use other Siebel Servers
Maintains internal table of components available on each Siebel
Server
Will route requests round Siebel Servers in turn (round-robin)
Multi-threaded component
May need to increase MaxTasks
Should always be running on all servers
Siebel Server – Server Request Broker
Assignment
Manager
Server
Request
Broker
Workflow
Process
Manager
Run
Object Assignment
Manager Task
Is Assignment
available on
this server?
Run
Server
Server
Request Assignment onRequest
Broker local server Broker
Assignment
Manager
Workflow
Process
Manager
Siebel Server – Server Request Broker
Assignment
Manager
Run
Object Workflow
Is Workflow
Manager Process
Manager
Server
Request
Broker
Server
Request
Broker
Workflow
Process
Manager
Assignment
Manager
available on
Server
this server?
WhichRequest
other
serversBroker
have
workflow
Choose server
online?
onWorkflow
roundrobin
basis
Process
Manager
Server Request Processor
Processes asynchronous requests
Request submitted by creating record in table
S_SRM_REQUEST
Server Request Processor reads from table
Request must:
Be active (reached activation time)
Not be specified for a different Siebel Server
Not being processed by other Server Request Processor
Eligible requests submitted through Server Request
Broker
Normally runs on all Siebel Servers
Siebel Server – Server Request Processor
S_SRM_REQUEST
SRProc
Request
Queue
SRBroker
Task
Sleep
Interval
Performance and Scalability
Architecture Overview
Component Scalability
Scalability Across Components
Network Scalability
Performance Optimization
Performance Management
Connection Pooling
Siebel 7 can pool sessions at two levels:
Web server to Siebel Enterprise
SISNAPI Connection Pooling
Multiple SISNAPI (Siebel) sessions through one TCP session
Reduces operating system overhead and network traffic
Enabled by default
Set to 20
Controlled by component parameter:
‘Number of Sessions per SISNAPI Connection’
Siebel Object Manager to Database
Database connection pooling
SQL traffic for multiple Siebel users through one database session
Reduces session overheads on database server
Disabled by default
Suitable for larger deployments (over 500 concurrent users)
Database Connection Pooling
Connections use native database protocols
Some components directly access native protocol
Object Managers
Siebel 7 supports its own database connection pooling
Used for connections from Object Managers
Two types of connection
Shared – for general transactions
Specialized – for long running
Not always appropriate
Should carefully evaluate tradeoffs
Benefits of less database sessions to maintain
Risk of database session contention
Database Connection Pooling
Database session uses login for first user to establish
session
Database connection pooling controls
Defined as component parameters
Set to ‘-1’ to disable (default)
Specialized (Dedicated) Database sessions
All valid per component process (MT Server) per Siebel Server
MaxTrxDbConns
- Maximum number of specialized DB sessions
MinTrxDbConns
- Minimum number of specialized DB sessions
to be kept in pool
Shared Database sessions
Valid per component per Siebel Server
MaxSharedDbConns- Maximum number of shared DB sessions
MinSharedDbConns - Minimum number of shared DB sessions
to be kept in pool
Database Network Architecture
Client Connections
Siebel Server
Object Manager
Server
Request
Processor
Shared
Shared
Specialized
Native Database Connectivity
(ODBC for SQL Server)
Threads (sessions)
Siebel
Database
Processes (components)
Performance and Scalability
Architecture Overview
Component Scalability
Scalability Across Components
Network Scalability
Performance Optimization
Performance Management
General Siebel Server Optimisation
Disable unused components and component groups
These just use up system resources
Consider using common log file for multi-threaded
components that generate too many log files
Consider using multiple instances/server tasks for certain
activities
e.g. Parallel EIM, multiple Workflow Policy Groups, etc.
Monitor resource utilization by processes and threads
Object Manager Optimization
Enabling View Caching for High Interactive mode
Set in user preferences
Faster client response times, by as much as 1 second
Better network behavior
Expensive, 3MB per view cached
Most users use < 10 views / pages
Uses LRU mechanism to flush out cache
If not using eAdvisor or browser based eConfigurator
On cfg file set “EnableCDA = FALSE”
Set “Timed Statistics” Off
Set all Log Levels to 1
EAI Optimization
Minimize fields in Integration Components
Keep XML records small
XML is very verbose and needs to be parsed
Lump several records into one transaction
will achieve higher throughput
If possible keep records to 15 fields
More will reduce throughput
Keep in mind that an XML field translates to BC field
Integration Object initialize Business Objects and Business
Components
The more fields per record the slower it will process
EAI Optimization
Reduce SessPerSisnConn for EAI Adapters
Do this only if you are going to “bombard” adapter, i.e. little or no
think times
For very high throughput systems where you are
dedicating EAI machines
Consider letting the SRB forward BIM requests to dedicated
machines
BIM processing can become bottleneck if BIM processing is complex
May be necessary to have a higher EAI Adapter:BIM machine ratio
(i.e. 1:x)
Use ‘Session-Mode’ for high volume inbound HTTP
Greatly increased throughput
No overhead for authentication & session startup for each message
Workflow Process Optimisation
Use Run-Time Events for process automation
Minimize use of Workflow Policies since workflow policy work at the
database layer and do not take advantage of the level of abstraction
provided with the Object Manager
Performance gain since the decision event takes place on the
business object layer and so no trigger is created on the table
No need for scripting
Workflow Process Distribution
Workflow Policies can’t be directly load balanced
Asynchronous Workflow Processes are distributed by SRProcessor
Synchronous Workflow Process requests are distributed by
SRBroker
Application Performance - Configuration
Keep scripting small and tight
Avoid the use of script wherever possible
Remember it is still interpreted code and will execute as such
Ensure all objects are destroyed after use – avoid memory leaks
MVG Applets
Keep to a minimum as they increase SQL complexity
Use primary joins as much as possible; reduces number of SQL
statements
Use SmartScript only where really required
E.g. Novice Call Center users
Avoid putting external news pages on login page
3rd party web sites can be slow
Application Performance - Configuration
Activate only fields that are absolutely necessary
Become active when displayed; “Force Active” in BC configuration
General rule, the more fields on a page the slower the page
Recently saw customer with 750 fields and 80 tables for one applet
Include only manageable set of fields on list applets
Many customers use several dozen fields; not usable and slow
performance
Pages with more than three applets will perform slower for
HI applications
Reason for this is not really the number of applets, rather the number
of fields
PickLists set Long List to TRUE if returning large data sets
Tree Applets can be slow
They generate Bill of Materials type queries
Network Performance – Siebel Configuration
Browser Validation
Reduces the need for server communications to validate data entry
Implement through browser script
Immediate Posting of Changes
Where the ‘Immediate Post Changes’ flag is set against a field data
will be transferred whenever a field is changed
Incurs additional round trip with approx 2KB data
Keep to no more than two Applets per View
Minimize Popups
Limit columns in List Applets
Network Performance – Siebel Settings
View Caching
View definitions cached in browser memory
Requires approx 3MB memory per view
Typically around 10 cached views is enough
Uses LRU algorithm to maintain cache contents
Personalization and Applet Toggles won’t use view caching
Enabled through Object Manager configuration (.cfg) file setting
[SWE]
EnableViewCache=TRUE
Controlled through:
User Preferences > Behaviour > View Cache Size
Default: 10
Network Performance – Siebel Settings
Compression (Dynamic Content)
Performed by SWSE
Do not use web server dynamic compression (application files)
Enabled through SWSE configuration file (‘eapps.cfg’)
[Defaults]
DoCompression = TRUE
Compression (Static Content)
Performed by web server (IIS only)
Network Performance – Web Server / Browser Settings
Browser Settings
Don’t clear cache except when necessary
Ensure ‘Empty Temporary Internet Files Folder when browser is
closed’ option is not enabled.
Network Performance – Web Server
Web Server Settings
Use HTTP keep-alive
Reduces the need to negotiate TCP sessions for each HTTP message
Network Performance – Web Server
Web Server Settings
Set Content Expiration
2 days is typical setting
Set through Internet Information Services Administration
HTTP Headers > Content Expiration
Network Performance – Web Caching
Uncached
GET: ‘icon.gif’
25KB
icon.gif
DATE: 10/08/03 07:14
RESPONSE: ‘icon.gif’
DATE: 10/10/03 09:25:08
LAST-MODIFIED: 10/08/03 07:14
Network Performance – Web Caching
Cached
GET: ‘icon.gif’
IF-MODIFIED-SINCE: 10/10/03 09:25
2KB
icon.gif
icon.gif
DATE: 10/10/03 09:25
RESPONSE: Not-modified
DATE: 10/08/03 07:14
Network Performance – Web Caching
Cached with Expiration
0KB
icon.gif
icon.gif
DATE: 10/10/03 09:25
EXPIRES: 10/12/03 14:13:08
DATE: 10/08/03 07:14
Performance and Scalability
Architecture Overview
Component Scalability
Scalability Across Components
Network Scalability
Performance Optimization
Performance Management
Performance – Problem Diagnosis
What is the problem?
Slow response times reported by User(s)?
Resource utilisation problem reported by Administrator?
Where is the problem?
Who is experiencing the problem? One user, a subset of users, all
users?
Network problem?
Database problem?
Siebel Server problem?
Web Server problem?
How do we resolve the problem?
Performance – Problem Diagnosis continued
Problem Identification
User reports slow response times
Identify precise actions carried out by user at the precise time of the
problem – set up a problem reporting process, with required
information to be supplied by user
Subset of Users report slow response times
Are users on the same LAN segment? Are other users experiencing
problems?
Are users all performing similar tasks?
All users report slow response times
Is it specific transactions (views) that are slow?
Do all transactions run slowly?
System Administrators should check system resource utilisation
System Administrator(s) report resource utilisation problems
Are users experiencing problems as a result?
Which area of the system is experiencing problems
Ask questions of the relevant people – need more
information than ‘The system seems to be slow’
Performance – Problem Diagnosis continued
Diagnosis Tools
Database SQL Trace (SQL Server Profiler)
Siebel Server Object Manager SQL Trace
Web Server SWSE Statistics Page
Web Server SWSE Log
Web Server Log (e.g. IIS log)
Server Resource Utilisation tools
Windows: PerfMon, Task Manager
TECHNOTE
361
Memory Utilization
in Siebel eBusiness
Applications
TECHNOTE
382
How can users
generate
Performance
Monitor information
into a log file?
Performance – SQL Trace
Use Database SQL Trace or Siebel Component SQL Trace
to identify poor-performing queries
Correct Application Configuration
Add indexes (except unique indexes)
Refresh database statistics (not on Oracle)
Siebel Component SQL Trace
Set ‘SQL Trace Flags’ parameter
Set ‘Event Log Levels’ to 5 for ‘%SQL%’
Srvrmgr: ‘change evtloglvl %SQL%=5 for component sccobjmgr_enu
server <siebel server name>’
Scan log file for ‘SQL Statement Execute Time’
Need to use SQL Server Profiler for full capture
Siebel uses specialized ODBC connection models
Need to reproduce fully in order to get correct query execution plan
Performance – Siebel Web Server Logs
SWSE Log
<Install Dir>/SWEApp/log/ssyymmdd_nn.log
Set ‘log=details’ in eapps.cfg file for more detailed logging
Set environment variables for full detail
SIEBEL_SESSMGR_TRACE=1
SIEBEL_SISNAPI_TRACE=1
SIEBEL_LOGEVENTS=ALL
Restart Web Server
Web Server Log
IIS: <Windows Install Dir>\system32\Logfiles\W3SVC1\exyymmdd.log
In Internet Services Manager, enable extended logging to include
Client IP address, User Name, Time Taken
Use to identify long-running HTTP requests or individual user
machines experiencing problems
Performance – Siebel SWSE Statistics Page
URL: <Application URL>/_stats.swe?verbose=high
<Application URL>/_stats.swe?verbose=high&reset=true to reset
stats
Eapps.cfg configuration file:
‘Allowstats=TRUE’
‘SessionMonitor=TRUE’
Allows individual user session statistics, in ‘Current Sessions’ section:
Event
siebel.TCPIP.None.None://jmullisp4:2320/siebel/SCC
ObjMgr_enu/jmullisp4/!1.8c0.4023.3ec3708d
SADMIN
Total Time
General Stats
(count, mean,
std dev)
Frequency
(mean, std
dev)
14.6779
54
0.2718
0.6826
11.33
59
44.64
84
Total Time: Time for all user’s HTTP requests
General Stats-Count: # user HTTP requests
General Stats-Mean: Average time for each user HTTP request
Frequency-Mean: Average time between each user HTTP request
Performance – Siebel SWSE Statistics Page continued
Session Identification
!A.B.C.D
A = Siebel Server Id
B = Siebel Server Component Process Id
C = Siebel Server Component Task Id
D = Timestamp
All values in hexadecimal
Use to map session information to Siebel Server Task logfile
Section ‘Current Operations Processing’
Operations in bold type have been running > 10 seconds
Volume Performance Testing
Using Volume Performance Testing Tools, such as
Mercury LoadRunner
Segue SilkPerformer
Use to measure
Client Response times under high user load
Resource usage under high user load
Web Servers, Siebel Servers, Database Server, Network
Scalability of Siebel environment
Hardware
Use full duplication of production hardware, if possible
If not, scale down numbers of servers proportionately
Database
Tune Object Manager SQL prior to volume performance test exercise
Utilise Object Manager SQL trace flags
New Feature - SARM
Features
Siebel Application Response time
profiling – SARM (7.5.3)
ARM Standard Support (7.7)
CPU and Memory Usage Profiling
(7.7)
Additional Siebel component
metrics (7.7)
Benefits
Proactive monitoring of
application response time
Diagnosis of response time
problems in the application and
infrastructure
Enables tuning of applications to
meet service level commitments
Enables diagnosis of memory
consumption
SARM Run-time Architecture
Siebel Server
SARM
Start
Server
Component
SARM
Stop
SARM Correlation Key
SARM
Framework
3rd Party
ARM API
Library
3rd Party
ARM Tool
SARM
Start
SARM
Log
Server
Component
SARM
Stop
ARM
Log
SARM Instrumentation for 7.5.3
Timers
User Interface
Physical UI Rendering
Web Server
Time
Logical UI
Network Time
SWE Time
Workflow Time
Script Time
SRB Time
Comp Time
Object Manager
Data Manager
EAI
Database
Time
Siebel
Database
External
Database
Legacy
Application
Performance and Scalability
Architecture Overview
Component Scalability
Scalability Across Components
Network Scalability
Performance Optimization
Performance Management
Questions and Answers
Siebel 7 Performance and Scalability
Inside the Siebel Server
Richard Sands
Siebel Expert Services
©Siebel Systems 2003 – Do not distribute or re-use without permission