Transcript Techwave_2001_EM413-_Mark_Culp

EM413
Using Adaptive Server
Anywhere’s Remote Data
Access Feature
Mark Culp
Software Development Manager
iAnywhere Solutions
[email protected]
1
Goals of this presentation
• Enable you to access remote data from your
ASA database
• Show you how to migrate your data to ASA
• Help you improve your cross database join
performance
• What should you not do when using ASA's
remote data access feature
2
Contents
•
•
•
•
•
•
•
What is “Remote Data Access” ?
Basics – How to get started
What if it doesn't work? – Some helpful hints
Data migration tools
Cross database joins
Limitations – What to avoid
Questions
3
What is “Remote Data Access” ?
4
What is “Remote Data Access” ?
• Allows an ASA server to access data from a
remote source as if it were local
• Remote data may be :
• iAnywhere ASA, Sybase ASE
• Oracle, IBM DB2, MS SQL Server
• Other ODBC data sources
• E.g. MS Access, MS Excel
5
Why use “Remote Data Access” ?
• Cross database joins
• Maintain real time access to remote data
• Integration to other projects and other
data sources
• Migration of legacy data
• Quickly set up a remote ASA database for
Mobilink
6
What is … - Architecture
Remote
Server
ASA Server
Client
Comm.
Remote
Store
Query
Execution
Engine
jConnect
ODBC
ASA
Store
7
ASA
ASE
Other
What is … - Platforms
Available on
• Win32 platforms:
95,98,ME,NT,2000
• UNIX platforms (since ASA 7.0.2):
Linux and SPARC Solaris
• Netware (since ASA 7.0.2)
8
What is … – Also known as …
Remote Data Access capability is also known
as:
• OMNI - OmniConnect (from ASE 10.5)
• CIS - Component Integration Services
• Proxy Tables
9
Basics – How to get started
10
Basics
Three steps to accessing remote data
1. Define your remote server
2. Map your external login
3. Create your remote proxy tables
Then use the proxy tables in the same way you
would use any other table in your query
11
Basics – Step 1:
Define your remote server
ASA needs to know:
• What you will call the remote server
• How to connect to the remote server
• The capabilities of the remote server
SQL statement:
create server <server_name>
class <connection_class>
using <remote_location>
12
Step 1 – Create server … class
Connection class:
• JDBC (jConnect) classes:
•
asajdbc, asejdbc
• ODBC classes:
•
•
•
•
•
asaodbc, aseodbc – ASA and ASE
db2odbc – IBM DB2
mssodbc – MS SQL Server
oraodbc – Oracle server (version 8.0 or later)
odbc – any other ODBC data source
13
Step 1 – Create server … location
(JDBC)
Remote location (JDBC):
<machine_name>:<port> [ /<db_name> ]
Location appended to JDBC URL specification:
'jdbc.sybase.TDS.'
/<db_name> only required if more than one
database loaded on remote server
14
Step 1 – Create server … example
(JDBC)
Example 1 (using jConnect):
create server DemoServerViaJconnect
class 'asajdbc'
using 'localhost:2638/asademo'
Remote server:
ASA server:
DemoServer
ViaJconnect
asajdbc
jConnect
15
localhost
:2638
/asademo
asademo
Step 1 – Create server … location
(ODBC)
Remote location (ODBC):
<DSN_name> [ ; <other_options> ]
DSN (data source name) normally specifies
parameters on how to find remote server
• Driver name
• Driver specific parameters:
• E.g. ServerName
Connection is in context of the ASA server
16
Step 1 – Create server … location
(ODBC)
Example 2 (using an ODBC DSN):
create server DemoServer
class 'asaodbc'
using 'demoDSN'
Remote server:
ASA server:
DemoServer
asaodbc
ODBC
Driver
Manager
demoDSN
ODBC
Driver
Eng=
asademo
asademo
Note: You can use dbdsn to create ASA DSNs:
dbdsn –w demoDSN –c “eng=asademo”
17
Step 1 – Create server … location
(ODBC)
Example 3 (connect to an Excel Spreadsheet):
create server ExcelData
class 'odbc'
Note 'Read Only'
using 'ExcelDataDSN'
Check Box
18
Step 1 – Create server … location
(ODBC)
Example 4 (no ODBC DSN):
create server DemoServerNoDSN
class 'asaodbc'
using
'driver=Adaptive Server Anywhere 8.0;
ServerName=asademo'
Remote server:
ASA server:
DemoServer
NoDSN
asaodbc
ODBC
Driver
Manager
Driver=…
19
ODBC
Driver
Server
Name=
asademo
asademo
Step 1 – Create server … location (UNIX)
Remote location (UNIX ODBC):
<DSN_name> [ ; driver=<driver_path> ]
There is no standard ODBC driver manager on
UNIX, so ASA needs to know where to find
the driver (manager) that you want to use.
You must specify a driver-path either:
• In your location string, or
• In your file DSN specification
20
Step 1 – Create server … location (UNIX)
Example 5 (UNIX - no driver manager):
create server DemoServerDirect
class 'asaodbc'
using 'demoDSN'
Contents of ~/.odbc.ini:
[demoDSN]
driver=/opt/sybase/SYBSsa8/lib/dbodbc8_r.so
servername=asademo
21
Step 1 – Create server … location
(UNIX)
Example 5 continued (UNIX - no driver manager):
Remote server:
ASA server:
DemoServer
Direct
asaodbc
ODBC
Driver
demoDSN
asademo
dbodbc8_r.so
• Since no "driver=" found in location string, ASA
looks for location of driver in the DSN
22
Step 1 – Create server … location (UNIX)
Example 6 (UNIX - with driver manager):
create server DemoServerUsingManager
class 'asaodbc'
using 'driver=<manager_path>;
DSN=demoDSN'
Contents of ~/.odbc.ini:
[demoDSN]
driver=/opt/sybase/SYBSsa8/lib/dbodbc8_r.so
servername=asademo
23
Step 1 – Create server … location
(UNIX)
Example 6 continued (with driver manager):
Remote server:
ASA server:
DemoServer asaodbc
UsingManager
ODBC
Driver
Manager
.odbc.ini
driver=…
ODBC
Driver
Server
Name=
asademo
asademo
dbodbc8_r.so
• ASA loads ODBC driver manager
• ODBC driver manager loads ODBC driver
24
Basics – Step 2:
Map your external login
ASA may need to know:
• User id and password to use when
connecting to the remote server
SQL statement:
create externlogin <login_name>
to <remote_server>
remote login <remote_user>
[ identified by <remote_password> ]
25
Step 2 – When to create externlogin
• By default, ASA uses the names and
passwords of its clients whenever it
connects to a remote server on behalf of
those clients.
• CREATE EXTERNLOGIN statement assigns an
alternate login name and password to be
used when communicating with a remote
server
26
Step 2 – Map your external login
Example 7:
create externlogin dba
to DemoServer
remote login dba identified by sql
27
Basics – Step 3:
Create proxy tables
ASA needs to know how to make the mapping
from a local proxy table to its remote data:
• If the remote table already exists, use the
CREATE EXISTING statement.
• If the remote table is to be created, use the
CREATE … AT statement.
28
Step 3 – Create existing table
To map an existing remote table:
SQL Statement:
create existing table [<owner>.]<table_name>
[ ( <column_definitions> ) ]
at <location_clause>;
• This statement defines the proxy table for an
existing table on the remote server.
29
Step 3 – Location clause
location_clause specifies where to locate the
remote table, and is of the form:
'<server>;<database>;<owner>;<table>'
where:
• server - name of server as known by current
database from create server command
• database - depends on type of remote server
• owner - owner of remote table
• table – name of the remote table
30
Step 3 – Create existing example
Example 8:
create existing table dba.p_employee
at 'DemoServer;;dba;employee';
ASA local database:
DemoServer:
proxy table:
dba.
p_employee
remote table:
dba.
employee
31
Step 3 – Create new table
To create a new remote table:
SQL Statement:
create table [<owner>.]<table_name>
( <column_definitions> )
at <location_clause>;
This statement creates a new table on the remote
server, and also defines the proxy table for
that table.
32
Step 3 – Create new example
Example 9:
create table p_employee_sheet
( emp_id numeric(9,0),
emp_fname varchar(30) )
at 'ExcelData;;;employee_sheet'
Copy data into the remote table using:
insert into p_employee_sheet
select emp_id, emp_fname from p_employee
33
Some Helpful Hints
34
Hints – Dropping proxy tables
To drop a proxy table, use:
SQL Statement:
drop table [<owner>.]<table_name>;
• If the proxy table was created using CREATE
EXISTING, only the proxy table is dropped
• If the proxy table was created using CREATE …
AT statement, then the proxy table and the
remote table are both dropped.
35
Hints – Dropping proxy tables example
Example 10:
drop table p_employee
• Since this proxy table was created using CREATE
EXISTING, this statement drops the proxy table
only; the remote table is not affected.
drop table p_employee_sheet
• Since this proxy table was created using CREATE
TABLE … AT, this statement drops the proxy
table, AND will also drop the remote table at
'ExcelData;;;employee_sheet'
36
Remote Data Access passthru mode
To send a statement directly to the remote server,
use:
SQL statement:
forward to <server_name>;
<remote_statement(s)>
forward to;
• The remote_statements text is sent to the remote
server as given
• Text is terminated by the "forward" and "to" tokens
37
Hints – Useful system tables
sys.sysservers – list of remote servers
select * from sys.sysservers
38
Hints – Useful system tables
sys.systable – list of proxy tables
select table_name, remote_location, existing_obj
from systable
where remote_location is not NULL;
39
Hints – Useful built-in procedures
sp_remote_tables server_name [, table_name ]
[, table_owner ] [, table_qualifier ]
[, with_table_type ]
• Lists tables available at remote server
sp_remote_columns server_name, table_name
[, owner ] [, database]
• Lists columns in remote table
40
Remote Data Access trace CIS option
You diagnose remote data access problems by
using the cis_option connection property
SQL statement:
set temporary option cis_option = 7;
• ASA will display messages in console log.
• Set cis_option value to 0 (zero) to turn off.
41
Trace CIS option example
Example 11:
set temporary option cis_option=7;
select * from p_employee;
42
Data Migration Tools
43
Data migration tools - Availability
You can import remote data using the
sa_migrate set of stored procedures
• Available in ASA 7.0.2, 7.0.3 and 8.0
• Source data may be from one of:
• iAnywhere ASA
• Sybase ASE
• Oracle, IBM DB2, MS SQL Server, MS Access
44
Data migration tools - Basics
Basic methodology:
• create base table(s) and proxy table(s)
• copy data from remote table(s)
• create foreign keys
ASA local database:
base
table
Remote data source:
proxy
table
remote
table
45
Migration – Single step process
To migrate entire remote database in one step,
use stored procedure:
dbo.sa_migrate(
IN local_table_owner
IN server_name
IN table_name
IN owner_name
IN database_name
IN migrate_data
IN drop_proxy_tables
VARCHAR(128),
VARCHAR(128),
VARCHAR(128) DEFAULT NULL,
VARCHAR(128) DEFAULT NULL,
VARCHAR(128) DEFAULT NULL,
BIT DEFAULT 1,
BIT DEFAULT 1 )
46
Migration – Single step example
Example 12:
call dbo.sa_migrate(
'dba', 'DemoServer', 'Customer', 'dba' )
• This migrates the customer table definition
and data from DemoServer to my database by:
•
•
•
•
creating customer table in my database
creating customer_et proxy table
copying the customer data
dropping the customer_et proxy table
47
Migration – Single step example results
48
Migration – Multi-step process
To have more control over what is migrated,
use:
1. sa_migrate_create_remote_table_list( … )
2. sa_migrate_create_tables( … )
3. sa_migrate_data( … )
4. sa_migrate_create_remote_fks_list( … )
5. sa_migrate_create_fks( … )
6. sa_migrate_drop_proxy_tables( … )
49
Migration – Step 1: create table list
1. Create a list of tables to migrate using
dbo.sa_migrate_create_remote_table_list(
IN server_name
varchar(128),
IN table_name
varchar(128) default NULL,
IN owner_name
varchar(128) default NULL,
IN database_name varchar(128) default NULL )
• this populates the
dbo.migrate_remote_table_list table
• delete any rows that you do not want
50
Migration – Step 2: create proxy tables
2. Create proxy and base tables using:
dbo.sa_migrate_create_tables(
IN local_table_owner
varchar(128) )
• uses rows in dbo.migrate_remote_table_list
• creates a proxy table and a base table for
each remote table listed
• also creates all primary key indexes
51
Migration - Step 3: migrate data
3. Migrate data into base tables using:
dbo.sa_migrate_data (
IN local_table_owner
varchar(128) )
• uses rows in dbo.migrate_remote_table_list
• populates base tables with rows from proxy
tables
52
Migration – Step 4: create fks list
4. Create list of foreign keys to migrate:
dbo.sa_migrate_create_remote_fks_list(
IN server_name
varchar(128) )
• uses rows in dbo.migrate_remote_table_list
• creates dbo.migrate_remote_fks_list table
• delete any rows that you do not want
53
Migration – Step 5: create foreign keys
5. Create the foreign keys using:
dbo.sa_migrate_create_fks(
IN local_table_owner
varchar(128) )
• uses rows in dbo.migrate_remote_fks_list
• creates a foreign key for each listed
54
Migration - Step 6: drop proxy tables
6. Drop all of the proxy tables:
dbo.sa_migrate_drop_proxy_tables(
IN local_table_owner
varchar(128) )
• uses rows in dbo.migrate_remote_table_list
• drops the proxy table (created in step 2) for
each remote table listed
55
Migration – Multi-step example
Example 13:
• call dbo.sa_migrate_create_remote_table_list
( 'DemoServer', NULL, 'dba' )
• select * from dbo.migrate_remote_table_list
• delete from dbo.migrate_remote_table_list
where table_name = 'customer' or
table_name like 'sales_%' or table_name like 'fin%'
• call dbo.sa_migrate_create_tables( 'dba' )
• call dbo.sa_migrate_data( 'dba' )
• call dbo.sa_migrate_create_remote_fks_list( 'DemoServer' )
• select * from dbo.migrate_remote_fks_list
• call dbo.sa_migrate_create_fks( 'dba' )
• call dbo.sa_migrate_drop_proxy_tables( 'dba' )
56
Migration – In case of a failure
Migration may fail if a foreign key referenced
table does not exist
• You may choose to update the
dbo.migrate_remote_fks_list table (delete the
offending entry) and continue
• Minimally you must run
dbo.sa_migrate_drop_proxy_tables to delete
the temporary proxy tables that were created
57
Cross Database Joins
58
Cross database joins – What is it?
A "Cross Database Join" is a join between two
or more tables where the tables reside on
two or more different servers
Example A:
Table A
(ASA)
Example B:
Join
Table B
(remote)
Join
Table A
(remote a)
59
Table B
(remote b)
Cross database joins – Implementation
ASA handles cross database joins by:
• Parse the query and identify remote tables
• Push predicates down to remote server when
possible
• Reads remote rows using forward-only cursor
Note:
• 6.0.x and 7.0.x uses only naïve nested loop
join operator
60
Cross database joins – Remote Join
• When the join is on tables from the same
server, the join is pushed to the remote
Example C:
Join
Table A
(remote a)
Table B
(remote a)
61
Cross database joins – Example 14
Example 14:
• R is remote table, S is ASA local table
Select R.* from R join S
where R.k = S.k
For each row r in R
if exists( select 1 from S where S.k = r.k )
match row
62
Cross database joins – Example 15
63
Cross database joins – Example 16
64
Cross Database Joins – Why is it slow?
If ASA uses a plan that has the remote table on
the right side of the join operator,
then the remote data will be hauled across
the communication link multiple times –
once for each left hand side row.
For each row s in S
for each row r in ( select * from R )
if s.k = r.k
match row
65
Cross Joins – How to make it go faster?
• Only include remote columns that are
absolutely required by the query
• Run the query using dbisql, and check the
plan that is being used
• Simplify query to have fewer remote tables
• Consider using local temporary tables to
hold intermediate results
• Migrate data to ASA and perform join on
local table
66
Limitations:
What should you avoid ?
67
Limitations
Features not supported on remote data:
• ALTER TABLE statement against remote tables
• Triggers defined on proxy tables will not fire
• SQL Remote
• Java data types
• Foreign keys that refer to remote tables are ignored
• Positioned UPDATE and DELETE
• UPDATE and DELETE requiring an intermediate
temporary table
• Backwards scrolling on cursors
68
What to avoid … No cycles
1. Don't create a proxy table that refers back
to the same database.
Reason:
Deadlock can occur if remote access waits
for a lock caused by the same connection.
Corollary:
Don't create cycles in your databases'
remote access definitions.
69
What to avoid … Separate Servers
2. Don't create a proxy table that refers back
to the same server.
Reason:
Deadlock will occur when more than one
concurrent remote access is attempted.
Work around: increase –gx parameter
Solution:
Use separate servers to host the local and
remote databases.
70
What to avoid … Join Large Tables
3. Don't join two large remote tables from
different remote servers.
Reason:
Slow response
Solution:
Migrate one of the tables into an ASA table
first.
71
What to avoid … N-way joins
4. Don't create complicated joins containing
many remote tables from many different
servers.
Reason:
Slow response
Solution:
Migrate table(s) into ASA first.
72
Summary
ASA's Remote Data Access feature is a
powerful tool to access your remote data
from your ASA database:
• Easy to migrate your legacy data to ASA, or
• Use your remote data in place as if it were
part of the ASA database
More detailed information found in:
ASA SQL User's Guide
15. Accessing Remote Data
73
Thank you
Questions ?
[email protected]
www.ianywhere.com
74