Transcript PowerPoint Presentation - CWSpace: Archiving Course

CWSpace
Archiving Courseware Websites to DSpace
Using a Content Packaging Profile
& Web Services
William Reilly, Rob Wolfe — MIT Libraries
Digital Library Federation Spring Forum, 2006
April 11, 2006 — Austin, Texas
©
2006
Project Overview
CWSpace: MIT iCampus project, nearing completion
Vision: Increase
value in Institute
publication: OCW
– Also: Increase
value in Open
Source platform:
DSpace
Goal: Archive OCW to
MIT’s DSpace
– Also: Make content
available to CLEs *
* CLE = Collaborative Learning Environment
Method: Interoperability of systems
– Use Content Packaging standard
– Use Web Services standard
DLF Spring 2006
MIT's CWSpace
2
Perceived Need:
™
inter-operability
• Obvious observation: Higher
Ed. Technologies increasingly
seeking integration with
Library & Repository
holdings
• Not so obvious(?): Libraries
seeing their charter to include
collection, preservation,
dissemination of Teaching &
Learning materials (?)
DLF Spring 2006
MIT's CWSpace
3
InterOp: What’s Available
DLF Spring 2006
MIT's CWSpace
4
InterOp: What’s Achievable
DLF Spring 2006
MIT's CWSpace
1 of 3
5
InterOp: Challenges
DLF Spring 2006
MIT's CWSpace
2 of 3
6
InterOp: Possible Resolution?
DLF Spring 2006
MIT's CWSpace
3 of 3
7
OCW Materials in DSpace =
New Ground
Educational Materials
Digital Archive
New kind of content: educational,
teaching, learning
POLICY - Cost/merit evaluation of
archival treatment. Preservation and
IP considerations
New emphasis (ideally) on re-use,
re-purposing: aggregation,
disaggregation
New kind of structure, composition:
website, compound digital object,
rich metadata
GRANULARITY - Challenge to
digital archive; how flexible can it
be? (cf. CMS)
New needs for system-to-system
access: CLEs, Image Tools
BACK-END / FRONT-END - Need
for networked access
COMPLEXITY - Need to package
multi-file content, plus metadata
Content Packaging
Web Services
DLF Spring 2006
MIT's CWSpace
8
Content Packaging
DLF Spring 2006
MIT's CWSpace
9
Content Packaging: Steps Toward a
Profile
• Model OCW Content
• Choose Packaging Specification
• Profile That Specification
DLF Spring 2006
MIT's CWSpace
10
Content Package
Package Interchange
File (PIF)
• Usually a .zip file
• Contains a manifest
• Manifest references
files (some in, some
out of the package)
DLF Spring 2006
MIT's CWSpace
11
Object Models
• Mapping one world to another
• Content Package design considerations
• Use, re-use, subsequent dissemination
OCW
DSpace
…
…
SIP
DLF Spring 2006
MIT's CWSpace
DIP
12
OCW to DSpace mapping
DLF Spring 2006
MIT's CWSpace
13
Which Packaging Spec.?
• METS
• IMS-CP
• MPEG-21
DIDL
• XFDU
•
Content packaging
specifications are generic
loose envelopes to carry
content, metadata
•
Choice of package is
largely determined by
user community
•
Dspace to use METS
internally
Higher Education uses
IMS Content Packaging
•
DLF Spring 2006
MIT's CWSpace
14
METS and IMS-CP
DLF Spring 2006
MIT's CWSpace
15
IMS-CP & OCW Content Model
DLF Spring 2006
MIT's CWSpace
16
IMS-CP & DSpace Object Model
DLF Spring 2006
MIT's CWSpace
17
OCW to IMS-CP to DSpace
DLF Spring 2006
MIT's CWSpace
18
Profiling Package Standards
Every Implementation of
IMS-CP or METS will:
The Standards themselves
encourage profiling in a
systematic way:
•
•
•
IMS Application Profile
Guidelines Technical Manual
http://imsglobal.org/ap/apv1p0/i
msap_techv1p0.html
Constrain the Standard
Extend the Standard
Interpret the Standard
The sum of these
adjustments is the
implementation profile
DLF Spring 2006
METS Profile Schema and
Guidelines
http://www.loc.gov/standards/mets/metsprofiles.html
MIT's CWSpace
19
Application Profiles
DLF Spring 2006
MIT's CWSpace
20
Application Profiles
DLF Spring 2006
MIT's CWSpace
21
Application Profiles
DLF Spring 2006
MIT's CWSpace
22
Application Profiles
DLF Spring 2006
MIT's CWSpace
23
Application Profiles
DLF Spring 2006
MIT's CWSpace
24
Profiled Content Package…
Now with Package
defined…
Content Provider
…need to move that across
the network from
producer to
consumer…
DLF Spring 2006
MIT's CWSpace
DSpace
25
Web Services
DLF Spring 2006
MIT's CWSpace
26
DSpace dev: LNI & PKGers
DLF Spring 2006
MIT's CWSpace
27
Lightweight Network Interface
0. What Was The Problem?
1. What Is the LNI?
2. DSpace & WebDAV
3. Example of Use: Submit OCW
4. Future Development…
DLF Spring 2006
MIT's CWSpace
28
What was the Problem…?
Content Providers wanted…
•
Networked Access: Programmatically access the DSpace object model, but not
running on the DSpace server
Basic
(Lightweight
Network Interface (LNI))
[ Access, Deposit, Limited
Mods. ]
Advanced
• Submit
• Retrieve Item w. Metadata
• Build own U/I
• Collection Mgt, Map Items
• Specialist metadata
• Automated metadata
• Quality control checks
• Format migration …
DLF Spring 2006
(possible future
LNI dev.)
[ Replace, Withdraw, Delete ]
MIT's CWSpace
Content Provider
DSpace
29
What Is the LNI?
The "Lightweight Network Interface" is:
•
A remote API to control DSpace
•
It's essentially WebDAV: Extensions to HTTP
– A SOAP (with WSDL) equivalent is provided
•
It's a mapping: —
– a "complete and comprehensive" view of the DSpace public object
API…
(the "business logic" layer)
– …as mapped to WebDAV semantics
DLF Spring 2006
MIT's CWSpace
30
DSpace: Model
•
•
•
•
•
DLF Spring 2006
MIT's CWSpace
DSpace digital repository
Differs from file system
Submit URI not known
Workflow
…
31
WebDAV: Model, Basics
DAV = "Distributed Authoring and Versioning"
•
•
URL
•
•
DLF Spring 2006
MIT's CWSpace
Each Resource
named by a
URL
Properties are
"name value"
pairs
Properties are
extensible
Property values
can be found,
and modified
32
WebDAV: Model, Fuller
•
•
•
DLF Spring 2006
MIT's CWSpace
Resources
can be
Collections
or leaf
nodes
Collections
can nest
Everything
can be
discovered
descending
from the
DAV
server's root
'/' resource
33
DSpace Mapped to WebDAV
•
DLF Spring 2006
WebDAV
MIT's CWSpace
34
DSpace Implemented as WebDAV
1. Properties
– WebDAV
– Custom for DSpace
2. LNI URLs
3. WebDAV Methods
DLF Spring 2006
MIT's CWSpace
35
Properties: WebDAV
•
•
•
•
•
•
•
DAV:creationdate
DAV:displayname
DAV:getcontentlength
DAV:getcontenttype
DAV:getlastmodified
DAV:resourcetype
…
Akin to
file system
properties,
with add'l.
metadata
http://www.webdav.org/specs/rfc2518.html
DLF Spring 2006
MIT's CWSpace
36
Properties: Custom DSpace (1 of 3)
• ALL DSpace Objects:
– dspace:type
• "BITSTREAM", "ITEM", "COMMUNITY" etc.
– DAV:resourcetype
• "COLLECTION" etc.
– dspace:displayname
– dspace:current-user-privilege-set
DLF Spring 2006
MIT's CWSpace
37
Properties: Custom DSpace (2 of 3)
• Community DSpace Objects
– dspace:logo
– dspace:short_description
– dspace:side_bar_text
– dspace:handle
–…
DLF Spring 2006
MIT's CWSpace
38
Properties: Custom DSpace (3 of 3)
• Bitstream DSpace Objects
– DAV:getcontentlength = getSize()
– dspace:format = getFormat().getID()
– dspace:sequence_id = getSequenceID()
–…
• Many more DSpace Objects (some 10)
– With many more Properties (some 87)
http://wiki.dspace.org/LightweightNetworkInterface
DLF Spring 2006
MIT's CWSpace
39
URLs: DSpace LNI
•
Community
– http://uni.edu/dspace/dav/dso_1721.1$46
•
Collection
– http://uni.edu/dspace/dav/dso_1721.1$349
•
Item
– http://uni.edu/dspace/dav/dso_1721.1$5543
•
Bitstream
– http://uni.edu/dspace/dav/dso_1721.1$5543/bitstream_13
The LNI URLs for DSpace objects are only for use with the LNI.
• They are therefore not to be used:
• as persistent URLs (e.g. bookmark, store for later use)
• as interactive URLs (e.g. browser)
DLF Spring 2006
MIT's CWSpace
40
WebDAV Methods in LNI
•
PROPFIND - very powerful request with multiple functions:
–
–
–
•
•
•
•
list names of properties
return property values
recurse through 1 or more levels of a "collection" resource.
PROPPATCH - changes value of or deletes properties on one resource.
GET - retrieves contents of a resource
PUT - replace or add a new resource to a collection
COPY - map a resource ("Item") to a different Collection.
As noted, equivalent SOAP methods are provided
Not Implemented: Locking, Versioning.
DLF Spring 2006
MIT's CWSpace
41
Current DSpace (pre-LNI)
Content Provider, currently
•
Batch import (& export)
•
End user Web U/I
DSpace
Batch Importer
Web U/I
•
dspace-admin Web U/I
Workflow
Adm
Non Std.
Non Web
Web, but
Non Automated
DSpace Object API
•
SRW, OAI-PMH
OCW Comm.
"Biology (7)"
"Physics (8)"
Item
DLF Spring 2006
MIT's CWSpace
42
DSpace with LNI
DSpace
Content Provider using LNI:
•
LNI SOAP end-point URL
•
LNI WebDAV URLs
DSpaceLNI.class
DSpace LNI WebDAV
•
- Web.
- Stds.
- Batchable.
DSpace Object API
dspace-admin (partial)
OCW Comm.
•
SRW, OAI-PMH
–
•
(Not replicated in LNI)
"Biology (7)"
Still available:
–
–
Batch import (& export)
End user Web U/I
"Physics (8)"
Item
DLF Spring 2006
MIT's CWSpace
43
Submission: Overview
OpenCourseWare Client
DSpace
Needs 3 Values:
1.
LNI SOAP end-point URL
http://userid:[email protected].
edu/dspace-lni/lni/DSpaceLNI
2.
Handle for OCW Community =
hdl:1721.1/125
3.
Dept. Name (MIT "Course"
Number) for Course to be submitted
= "Physics (8)"
DSpaceLNI.class
DSpace LNI WebDAV
Lang. Indep.
Platform Indep.
DSpace Object API
OCW Comm.
1721.1/125
"Biology (7)"
"Physics (8)"
Item
DLF Spring 2006
MIT's CWSpace
44
Submission: 1. Lookup
Request: OCW Comm. HANDLE
LNI lookup() =
http://dspace.mit.edu/dspacelni/dav/lookup/handle/1721.1/125
DSpace
DSpaceLNI.class
DSpace LNI WebDAV
Response: OCW Comm. LNI URL
http://dspace.mit.edu/dav/dso_172
1.1$125
DSpace Object API
OCW Comm.
1721.1/125
dso_1721.1$125
Lookup
"Biology (7)"
dso_1721.1$197
"Physics (8)"
dso_1721.1$253
DLF Spring 2006
MIT's CWSpace
45
Submission: 2. Collections
Request: PROPFIND on LNI URL for children DSpace
Collections' "displaynames":
<propfind>
<prop>
<DAV:displayname/>
</prop>
</propfind>
DSpace
DSpaceLNI.class
DSpace LNI WebDAV
Response: MULTISTATUS
<multiStatus>
<resource>
<href URI="dso_1721.1$197"/>
<displayname>Biology (7)</displayname>
</resource>
<resource>
DSpace Object API
OCW Comm.
1721.1/125
dso_1721.1$125
<href URI="dso_1721.1$253"/>
<displayname>Physics (8)</displayname>
</resource>...
"Biology (7)"
dso_1721.1$197
Iterate…
"Physics (8)"
dso_1721.1$253
DLF Spring 2006
MIT's CWSpace
46
Submission: 3. Submit (PUT)
Request: PUT to DSpace Collection LNI URL,
including specification of Packager Type (IMSCP):
PUT /dspace/dav/dso_1721.1$253?package=OCWIMSCP
....package contents in body...
DSpace
DSpaceLNI.class
DSpace LNI WebDAV
Response: LNI URL for DSpace Item in "Location"
Header:
HTTP/1.1 201 OK
Location:
http://dspace.mit.edu/dspace/dav/dso_1721.1$8
36
....other headers....
DSpace Object API
OCW Comm.
1721.1/125
dso_1721.1$125
"Physics (8)"
dso_1721.1$253
Item
dso_1721.1$836
DLF Spring 2006
MIT's CWSpace
Ingest
47
Submission: 4. Item Handle
Request: PROPFIND on DSpace Item LNI URL for the DSpace
Handle:
<propfind>
<prop>
DSpace
DSpaceLNI.class
<dspace:handle/>
</prop>
</propfind>
DSpace LNI WebDAV
Response: Properties Requested
DSpace Object API
=== PROPERTIES Successfully returned:
dspace:handle = "hdl:1721.1/836"
OCW Comm.
1721.1/125
dso_1721.1$125
OCW Content Mgt. System
Stores DSpace Handle
SQL Server Table: hdl:1721.1/836
DLF Spring 2006
MIT's CWSpace
"Physics (8)"
dso_1721.1$253
Item
dso_1721.1$836
1721.1/836
Reverse
Lookup
48
Submission: Summary
In this 4-step OpenCourseWare
submission example, we've seen:
•
SOAP endpoint URL
•
Dspace Handles
•
Dspace LNI URLs
•
LNI Lookup() function
•
WebDAV Properties
•
WebDAV Methods
•
PROPFIND (LNI request)
•
MULTISTATUS (LNI response)
•
LNI "Reverse Lookup"
•
WebDAV Headers (Location;
Success/Fail)
DSpace
DSpaceLNI.class
DSpace LNI WebDAV
DSpace Object API
OCW Comm.
1721.1/125
"Physics (8)"
1721.1/253
Item
1721.1/836
DLF Spring 2006
MIT's CWSpace
49
Future Development
• Other Use Cases, Needs
– Basic, Advanced …
• Other DSpace objects
– Bitstream Format Registry
– Metadata Registry …
• Other WebDAV methods
– PUT (for Replacement)
– DELETE
– MKCOL …
DLF Spring 2006
MIT's CWSpace
50
Questions, & Etc.
CWSpace
Archiving Courseware Websites to DSpace
William Reilly, Rob Wolfe — MIT Libraries
http://cwspace.mit.edu/docs/ProjectMgt/Reports/MIT-CWSpace-DLF-Spring2006.ppt
Digital Library Federation Spring Forum, 2006
April 11, 2006 — Austin, Texas
V. 20060410_1510
©
2006