Transcript SRU record update and its implementations in Australia

NUKAT, Warsaw
23 January 2008
SRU record update and its
implementations in
Australia
Chris Thewlis
(& Janifer Gatenby)
OCLC Regional Council
Canberra 15th April 2010
SRU Record Update
An Overview
Introduction to SRU Record Update
What it is: - A Push update mechanism, designed for
union catalogues
Catalogue
Mirror
OPAC
Documents
Repository
Catalogue
Union
catalogue
Push
OAIPMH
Pull
SRU Update
Push
Traditional update mechanisms
Proprietary mechanism – suitable for private one off agreements only
OAI PMH & FTP
• Suited for batch loading
• OAI PMH is used where the data source exposes the metadata to be harvested
for discovery purposes
• Where the data source does not expect the data to be changed by the
harvesting site or does not care
• Hence one way – no mechanism for response or future synchronisation
Relies on regular manual treatment of batches at both
ends and results in delays, gaps, less reliable alignment of
identifiers
Why SRU record update?
Requirement for immediate availability of records
• Scalability; no scheduling & batch maintenance
Two databases are not identical
•
e.g. A union catalogue is not a slave of the contributing catalogues
Requirement for interactive feedback – alignment, enrichment
• Identifier of the record in the target database
• Inter-record links – name and subject authorities, classification, FRBR work
record etc.
Diagnostics – diffused error resolution
• Warnings
• Errors
SRU record update : Near Real Time
Machine & QA corrections and merges
Inserts,
updates
deletions
Identifiers
and and
diagnostics
Inserts,
updates,
deletions
Identifiers
and diagnostics
Both CBS and WorldCat have an SRU Pusher and
an SRU catcher
Implementations of SRU record update
Dutch union catalogue updating WorldCat
(CBS)
Libraries Australia updating WorldCat (CBS)
DNB Zeitschriftendatenbank (ZDB, Germany)
updated by local systems
OCLC Library registry (catcher)
ABES (Star – online dissertations, Calames –
manuscripts)
WorldCat catcher and pusher
And in the wings to
update WorldCat
HeBIS
ABES
UnityUK
GBV
SwissBib
++
OLIB libraries
Consistency checker
Coordinated generation of tables
• Contents:
• Local identifier, ANBD id , Holdings symbols (Libraries Australia)
• EPN, PPN, Holdings symbols (GGC)
• ANDB id or PPN, OCN, Holdings symbols (WorldCat)
• Run at regular intervals
• Remedial action automated in obvious cases
ILS
WorldCat
Libraries Australia
And SRU Record Update
Libraries Australia and WorldCat
Repositories, various sites
blogs
ILS
WorldCat
Batch loading
ILS
SRU record update
WorldCat
SRU Record Update Statistics
CBS to WorldCat
Commenced Records new
to WorldCat
Holdings
Merge %
Changes
02.2008
573,854
2,850,000
20-30%
c.30,000 /mth
02.2009
159,741
1,130,000
50-60%
Not yet
CBS from WorldCat
Commenced
03.2010
Replaces
Deletes
200,000 + / week
1,000+ / week
To start in May 2010
Enrich %
60%
Amlib and Libraries Australia
Amlib and Libraries Australia
• LMS Developed in Australia
• One of 4 LMS’s owned by OCLC
• 400 libraries
• Currently developing two-way SRU update
•
CBS (Libraries Australia)
•
Touchpoint (OCLC locally deployed Discovery system)
• Hoping to test in the next 2-3 months
• Helps to proved concept
• Chance to review workflows
SRU Record Update Evolution
SRU from WorldCat to Libraries Australia Q2 2010
• Corrections and enrichments
• Bibliographic and holdings for libraries which catalogue
directly in WorldCat using Connexion
Wish list: More data exchanged in real time with WorldCat
• Detailed holdings
• Authority records (VIAF and ISNI)
Synchronisation Gateway
Synchronisation Gateway
Near Real Time Update for All Systems
Other systems need to move away from
batch too
Libraries
Australia Growth
2009
Sources
of WorldCat
Growth
FY09
45,000,000
40,000,000
4500000
35,000,000
4000000
30,000,000
25,000,000
3500000
20,000,000
3000000
15,000,000
2500000
10,000,000
5,000,000
2000000
0
5000000
Online
Batch
1500000
1000000
500000
0
New Records
Created
Records
Enriched
bibliographic records
Changed records Quality Control
BatchNewServices
Online Cataloging
ILS
WorldCat
Batch loading
ILS
SRU record update
WorldCat
Synchronisation Gateway
Library System
Z39.50
Client
SQL
Query
SRU update
OAI
PMH
Widget
ILS
API
Tailor
Local
Scripts
Detect / Elicit
Send
S
R
U
Synchronization Gateway
A new solution: The Synchronization Gateway
A middleware widget that is installed on a local PC
Queries the local database via multiple protocols
Elicits records via multiple protocols
Pushes data to CBS or WorldCat via a single protocol (SRU)
Nearly as good solution
One way only
Bibliographic records and holdings
• May not be able to detect deletions
• Will employ consistency checker to handle deletions
Machine readable reports of ANBD identifiers / local
identifiers are returned
The Synchronisation Gateway does not update
Synchronisation Gateway and Libraries
Australia
Proof of concept testing successful with UNSW and UNILINC
using:
• SRU enquiry for detection and eliciting
• SRU record update for loading to Libraries’ Australia CBS test
database
Planning other testing with Libraries Australia
Parallel testing with WorldCat catcher
Conclusion
• Ensures holdings are up to date [hour/second] on target
catalogue/s
• Ensures holdings are accurate and complete
• Eliminates batch preparation and monitoring
• More flexibility for services
• Australian libraries can deploy services on their own ILS, on
Libraries Australia and on WorldCat
• Better exposure of library materials