Transcript nesc_sakai_24507 - National e

Using Sakai for e-Research: Building a
Multi-Institution Virtual Research
Environment
Xiaobo Yang, Rob Allan,
Adrian Fish, Miguel Gonzalez and Rob Crouchley
What is a Virtual Research Environment?
A VRE is defined as a distributed way of working using a Webbased portal and for linking into users' desktop applications to
access a wide and growing range of on-line tools. These include
access to Grid based computing and data management systems
as well as collaboration tools, some based on Web 2.0. It is both
a ``one-stop shop'' for academic users and a ``turnkey solution''
for commercial users.
These emerging characteristics of a VRE are increasingly overlaid
with a requirement to provide support for the creation, further
development, or enhancement of a research community in virtual
space - a ``Virtual Research Community''. The OST report of
March 2006 indicated that VRCs have the potential to open
exciting new opportunities to collaborate in research and thus
realise significant gains at institutional, national and
international levels.
In this talk we only consider Web based VREs
Portals and VREs
The idea of portals has been around for a number of years. We
organised the Portals and Portlets 2003 Workshop here at NeSC
just at the time when two significant pieces of technology, the JSR168 portlet standard and WSRP 1.0, Web Services for Remote
Portlets standard, were being agreed (as mentioned in my previous
talk).
Since then, a number of open-source and commercial portal
projects have been launched and are in use for a variety of
purposes. One example in the UK is the portal for the National Grid
Service. This evolved from HPC Portal which was initially a Perl/ C
based environment for launching and monitoring Grid jobs similar
to the US GridPort and HotPage portals from San Diego
Supercomputer Centre. After briefly using PHP technology we have
now evolved to using JSR-168 portlets firstly in the GridSphere and
StringBeans frameworks and more recently in uPortal and Sakai.
Example: NGS Portal
• HPC Portal v4.0 provides a set of tried and tested portlets which
will work in a variety of frameworks, and can be distributed as
“MyNGS”.
• Built-in help system offers guidance for new users. Training and
documentation is also available.
• Open pages show non-authenticated users what is on offer.
• Authentication via modified JAAS layer offering, local portal id,
Grid certificate authentication, Shibboleth.
• JSDL editing and sharing portlets use an underlying database.
Sharing JSDL job descriptions encourages developments of
communities of practice around specific Grid applications.
• Job submission across NGS, NW-GRID, SCARF and local resources
can be configured.
• Data management also included.
http://portal.ngs.ac.uk
NGS Portal Application Registry
Science Gateways I
A VRE is however more than just a portal. Whilst NGS Portal has a
number of tools to encourage people to share artefacts, e.g.
descriptions of computational tasks or workflows, it has very little
built-in community support. It is important to address this if eScience technologies and the Grid are going to be taken up more
widely.
In the USA this is done through the concept of Science Gateways
such as NEESit. A number of these science gateways are listed on
the TeraGrid Web site.
Scientific gateways can have varying goals and implementations.
Some expose specific sets of community codes so that anonymous
scientists can run them. Others may serve as a "metaportal," a
community portal that brings a broad range of new services and
applications to the community. A common trait of all three types is
their interaction with the TeraGrid through the various service
interfaces that TeraGrid provides. Although the gateways may be
instantiated on TeraGrid resources, it is expected that many will
be instantiated on community resources and be administered by
the community itself.
Science Gateways II
Science Gateways signal a paradigm shift in traditional high
performance computing use. Gateways enable entire communities
of users associated with a common scientific goal to use national
resources through a common interface. Science gateways are
enabled by a community allocation whose goal is to delegate
account management, accounting, certificates management, and
user support to the gateway developers.
Science Gateways take three common forms:
• A gateway that is packaged as a Web portal with users in front
and TeraGrid services in back;
• Grid-bridging Gateways: often communities run their own Grids
devoted to their areas of science. In these cases the Science
gateway is a mechanism to extend the reach of the community Grid
so it may use the resources of the TeraGrid;
• A gateway that involves application programs running on users'
machines (i.e. workstations and desktops) and accesses services
in TeraGrid (and elsewhere).
Classification of Grid User
(adapted from Foster and Kesselman)
Class of User
Purpose
Requires
Concerns
End users (e.g.
quantitative social
scientists).
Do research. Solve
problems.
Applications.
Transparency, ease of
use, performance.
Application developers.
Develop new and
extend existing
applications.
Programming tools,
API's,libraries.
Ease of use, reusability.
Tool developers.
Develop API's, toolkits,
libraries.
Grid services.
Adaptivity,
applicability,
robustness, stability.
Grid developers.
Provide grid services.
Local system services.
Security, connectivity,
protocols.
System tools.
Balancing local and
global concerns.
System administrators. Manage grid resources.
JISC VRE 1 Sakai Demonstrator
• JISC VRE 1 Progamme – 2005-2007
• 4 partner sites: Daresbury, Lancaster, Oxford, Portsmouth (now
Reading)
• Framework extensions
 Security – Shibboleth
 WSRP
 JSR-168
• New tools, DMS, Agora, WSRP Consumer, Grid portlets, Blogger,
Shared Whiteboard, Bridging tools, Semantic search tool
• Production portal for e-Research projects – currently some 400
users and 25 projects hosted.
http://rhine.dl.ac.uk:8080/portal
VREs and CWEs
According to Wikipedia: a Collaborative Working Environment
(CWE) supports people (e.g. E-professionals) in their individual and
cooperative work. Research in CWE involves organisational,
technical, and social issues. It lists tools or services which may be
considered elements of a CWE including e-Mail, instant messaging,
application sharing, video conferencing, collaborative workspace,
document management, task and workflow management, Wiki and
Blog. Access Grid is mentioned as being a particular type of CWE.
It will be seen below that many of these tools have also been
recognised as being important in our VRE development and are
now available in Sakai. Not all of this work is described in this
paper however; in particular the important work on the Agora
conferencing and desktop sharing tool from University of
Lancaster, was initially funded as part of the VRE Demonstrator.
This tool addresses the requirements of desktop-based video
conferencing!
http://redress.lancs.ac.uk
Agora
Agora is an easy to use online meeting tool. With Agora you can take
your workplace with your laptop.
• Video-conference: "many to many": Organised into virtual meeting
rooms, you can video-conference with an unlimited number of
participants(*).
• Shared desktop: You can broadcast what you are watching on your
desktop.
• Whiteboard: Collaborative whiteboard on which anybody can sketch.
• Chat: Instant messaging
application.
• Moviecasting:
Broadcast movies.
• Session recording:
Record your sessions
for further analysis.
Half Way House?
• Sakai is not a portal, but has many portal-like characteristics
and similar look-and-feel
• Sakai supports a “Tool Portability Profile” enabling close
integration within the Sakai framework
• Sakai uses many underlying standards
• Sakai was designed as a Collaborative Learning Environment, so
also shares many aspects of CWEs
• It is designed to be scalable, supporting 10,000s of users
• Works with Oracle 10g
• To enable interoperability with portal technologies we added a
WSRP Consumer tool to Sakai (there was already a Producer)
• More recently a native JSR-168 interface has been added, based
on Pluto 1.1
• Sakai tools can also be exposed in portals, such as uPortal, so
Sakai could be viewed as a Service Hosting Environment.
• We think this is required for a VRE
Classes of User
In observing usage patterns we have seen the following:
1.
2.
3.
4.
Expert HPC user is happy to log on and develop applications
Semi-expert users liking remote scripting interfaces
Novice users like generic portals to test the functionality
Application-based communities develop rich clients, e.g.
desktop GUI
There is probably no single solution that will satisfy all the diverse
requirements, but exposing a common set of underlying
services and using standards to promote inter-operability can
help. This is the key to rapid and agile application
development, using and combining remote resources.
We are trying to use Sakai to combine a rich set of well-integrated
internal services with more loosely integrated
remote services.
Some Questions about VRE Usage
Deployment and evaluation of such a VRE tests and extends our
understanding of practical IT-based support for research in the
following areas:
• How can such frameworks be configured to best suit the
expectations and work practices of different research user
communities and institutional or organisational contexts?
• Can tools from multiple institutions and organisations be
brought together coherently to enable sharing of information,
processes and collaboration?
• Can community-specific tools be integrated meaningfully
alongside generic and remotely-hosted Web tools?
• Can a portal-like approach provide the flexibility to enable
effective use by both researchers and administrators?
• At what points are rich desktop tools or those provided by a
mobile platform, more effective?
• How might these be best integrated to provide a
meaningful user experience?
Sakai as a VO Management Tool
In the terminology of Sakai, a VO maps onto a ``worksite''.
Through their worksites, bespoke tools can be made available to
the VOs that require them. Each worksite can be customised to
have a specific look-and-feel and configured to contain just the
tools that are required by its members. This can include Web
interfaces to distributed services managed by a particular project
or hosted as part of a Grid resource.
• Sakai's internal VO management is through role-based policies.
Users can be allocated roles within each worksite. Roles can be
extended by administrative users from the small number of
defaults like ``admin'‘ and ``maintain''.
• Certain users can configure new sites, and resources can be
shared between sites.
• Other concepts include permissions, types, realms, skins,
properties, groups, aliases.
• Sites can be public, private or joinable.
Users see only what they have access to. Some
additional worksites are “joinable”.
Worksite
Role
Front page
(not logged in)
view only
My site
maintain
Site 1
create
Site 2
maintain
Site 3
access
Each worksite provides a list of tools
and view of underlying content
depending on the user’s role
Site x – created maintain
from Site 1
Roles and Permissions
Managing Users and Tools
Built-in Web 2.0 Services
Web 2.0 typically provides “hosted services” enabling users with
a Web browser to interact with them, contribute content and
invoke remote operations.
A growing list of such tools are being hosted in the Sakai server
and database. They can be rendered as stand-alone pages or tiled
in various combinations as required.
• Blog
• RSS News reader
• Wiki
• Glossary
• Calendar
• Threaded Discussion Forum
• Chat client
• Message Center
• Shared Resources
• Announcements
• Workshop paper management
Wiki and Discussion Forum
RSS News and Calendar
My Workspace
Each user has a workspace which aggregates views of all
the sites they belong to.
Customisation and Personalisation
Web 2.0 – Map Mashup
Recently we have investigated how to augment the built-in Web
2.0 services by making use of the Yahoo! Maps Web Service. Such
a Web API greatly alleviates the entry level of developing Web 2.0
for geo-spatial research applications. The services provide a set of
APIs (AJAX or Flash) through which developers can easily access
online maps around the world and overlay their own information
(mashup).
What we have tested is to display a map of the Sakai Community
similar to the one located at the Sakai Web site inside our VRE.
We expect this kind of mashup technology to be of use in a
number of research fields such as archeology, flood monitoring
and prediction, climate simulation and urban decision making in
addition to supporting other forms of collaborative working, such
as locating Access Grid rooms. Users will upload their data into
“Resources/MapData” and then select which to overlay on the
map.
Screenshot
Rollout, Sustainability and Community
Sakai is running on a fully-supported IBM BladeCenter at
Daresbury Laboratory currently with 28 dual-processor Xeon
blades. The content is hosted in the Oracle 10g database on the
UK National Grid Service (RAL node).
We are currently deploying fully-operational and supported Sakaibased VREs for the following communities:
NW-GRID: a community of computational scientists, both
academic and commercial, using compute clusters in the North
West of England.
ESRC e-Infrastructure: a community of multi-disciplinary social
scientists thoughout the UK building a common infrastructure and
adopting e-Science technology through the work of NCeSS and
ReDReSS http://www.ncess.ac.uk and http://redress.lancs.ac.uk
Diamond e-Infrastructure: a community of experimental
scientists using the new Diamond Light Source
http://www.diamond.ac.uk the largest investment in science in
the UK for 30 years.