Transcript sakai_ahm.13.9.07 - Old STFC e

Virtual Research Environments:
The Sakai VRE Demonstrator
Rob Allan, Xiaobo Yang,
Rob Crouchley, Adrian Fish and Miguel Gonzalez
What is a Virtual Research Environment?
A VRE is defined as a distributed way of working using a Web-based
portal and for linking into users' desktop applications to access a wide
and growing range of on-line tools. These include access to Grid based
computing and data management systems as well as collaboration
tools, some based on Web 2.0. A VRE is both a ``one-stop shop'' for
academic users and a ``turnkey solution'' for commercial users.
These emerging characteristics of a VRE are increasingly overlaid with
a requirement to provide support for the creation, further development,
or enhancement of a research community in virtual space - a ``Virtual
Research Community''. The OST (UK Office of Science and Technology)
report of March 2006 indicated that VRCs have the potential to open
exciting new opportunities to collaborate in research and thus realise
significant gains at institutional, national and international levels.
In this talk we only consider Web based
VREs using portal technology.
JISC VRE-1 Sakai Demonstrator
• JISC VRE-1 Programme – 2005-2007
• 4 partner sites: Daresbury, Lancaster, Oxford, Portsmouth (now
Reading)
• Framework extensions
 Security – Shibboleth
 WSRP
 JSR-168
• New tools, DMS, Agora, WSRP Consumer, Grid portlets, Blogger,
Shared Whiteboard, Bridging tools, Semantic search tool
• Production portal for e-Research projects – currently some 400 users
and 25 projects hosted.
http://rhine.dl.ac.uk:8080/portal
Classes of User
In observing usage patterns we have seen the following:
1.
2.
3.
4.
Expert HPC user is happy to log on and develop applications
Semi-expert users like remote scripting interfaces
Novice users like generic portals to test the functionality
Application-based communities develop rich clients, e.g. desktop GUI
There is no single solution that will satisfy all the diverse requirements,
but exposing a common set of underlying services and using
standards to promote inter-operability can help. This is the key to
rapid and agile application development, using and combining remote
resources.
We are trying to use Sakai to combine a rich set of well-integrated
internal services with more loosely integrated
remote services.
Classification of Grid User
(adapted from Foster and Kesselman)
Class of User
Purpose
Requires
Concerns
End users (e.g.
quantitative social
scientists).
Do research. Solve
problems.
Applications.
Transparency, ease of
use, performance.
Application developers.
Develop new and
extend existing
applications.
Programming tools,
API's,libraries.
Ease of use, reusability.
Tool developers.
Develop API's, toolkits,
libraries.
Grid services.
Adaptivity,
applicability,
robustness, stability.
Grid developers.
Provide grid services.
Local system services.
Security, connectivity,
protocols.
System tools.
Balancing local and
global concerns.
System administrators. Manage grid resources.
Some Questions about VRE Usage
Deployment and evaluation of such a VRE tests and extends our
understanding of practical IT-based support for research in the following
areas:
• How can such frameworks be configured to best suit the expectations
and work practices of different research user communities and
institutional or organisational contexts?
• Can tools from multiple institutions and organisations be brought
together coherently to enable sharing of information, processes and
collaboration?
• Can community-specific tools be integrated meaningfully alongside
generic and remotely-hosted Web tools?
• Can a portal-like approach provide the flexibility to enable effective use
by both researchers and administrators?
• At what points are rich desktop tools or those provided by a mobile
platform, more effective?
• How might these be best integrated to provide a
meaningful user experience?
Portals and VREs
The idea of portals has been around for a number of years. We organised
the Portals and Portlets 2003 Workshop in Edinburgh just at the time
when two significant pieces of technology, the JSR-168 portlet standard
and WSRP 1.0, Web Services for Remote Portlets standard, were being
agreed. We organised a second Portals and Portlets Workshop in 2006.
Since 2003, a number of open-source and commercial portal projects
have been launched and are in use for a variety of purposes. One
example in the UK is the portal for the National Grid Service. This evolved
from HPC Portal which was initially a Perl/ C based environment for
launching and monitoring Grid jobs similar to the US GridPort and
HotPage portals from San Diego Supercomputer Centre. After briefly
using PHP technology we have now evolved to using JSR-168 portlets
firstly in the GridSphere and StringBeans frameworks and more recently in
uPortal and Sakai.
See Dave Meredith’s talk
NGS Portal Application Registry
Science Gateways I
A VRE is however more than just a portal. Whilst NGS Portal has a
number of tools to encourage people to share artefacts, e.g. descriptions
of computational tasks or workflows, it has very little built-in community
support. It is important to address this if e-Science technologies and the
Grid are going to be taken up more widely.
In the USA this is done through the concept of Science Gateways such
as NEESit. A number of these science gateways are listed on the
TeraGrid Web site.
Scientific gateways can have varying goals and implementations. Some
expose specific sets of community codes so that anonymous scientists
can run them. Others may serve as a "metaportal," a community portal
that brings a broad range of new services and applications to the
community. A common trait of all three types is their interaction with the
TeraGrid through the various service interfaces that TeraGrid provides.
Although the gateways may be instantiated on TeraGrid resources, it is
expected that many will be instantiated on community resources and be
administered by the community itself.
VREs and CWEs
According to Wikipedia: a Collaborative Working Environment (CWE)
supports people (e.g. E-professionals) in their individual and cooperative
work. Research in CWE involves organisational, technical, and social
issues. It lists tools or services which may be considered elements of a
CWE including e-Mail, instant messaging, application sharing, video
conferencing, collaborative workspace, document management, task and
workflow management, Wiki and Blog. Access Grid is mentioned as being
a particular type of CWE. It will be seen below that many of these tools
have also been recognised as being important in our VRE development
and are now available in Sakai. Not all this work is described here, in
particular the important work on the Agora conferencing and desktop
sharing tool from Lancaster University, was initially funded as part of the
VRE Demonstrator and CQeSS project. This tool addresses the
requirements of desktop-based video conferencing!
http://agora.lancs.ac.uk
Agora
Agora is an easy to use online meeting tool. With Agora you can take
your workplace with your laptop.
• Video-conference: "many to many": Organised into virtual meeting
rooms, you can video-conference with an unlimited number of
participants(*).
• Shared desktop: You can broadcast what you are watching on your
desktop.
• Whiteboard: Collaborative whiteboard on which anybody can sketch.
• Chat: Instant messaging
application.
• Moviecasting:
Broadcast movies.
• Session recording:
Record your sessions
for further analysis.
Demos on Lancaster stand
Half Way House?
• Sakai is not a portal, but has many portal-like characteristics and
similar look-and-feel.
• Sakai supports a “Tool Portability Profile” enabling close integration
within the Sakai framework
• Sakai uses many underlying standards
• Sakai was designed as a Collaborative Learning Environment, so also
shares many aspects of CWEs
• It is designed to be scalable, supporting 10,000s of users
• Works with Oracle 10g
• To enable interoperability with portal technologies we added a WSRP
Consumer tool to Sakai (there was already a Producer)
• More recently a native JSR-168 interface has been added, based on
Pluto 1.1
• Sakai tools can also be exposed in portals, such as uPortal, so Sakai
could be viewed as a Service Hosting Environment.
• We think this is required for a VRE
Sakai as a VO Management Tool
In the terminology of Sakai, a VO maps onto a ``worksite''. Through
their worksites, bespoke tools can be made available to the VOs that
require them. Each worksite can be customised to have a specific lookand-feel and configured to contain just the tools that are required by its
members. This can include Web interfaces to distributed services
managed by a particular project or hosted as part of a Grid resource.
• Sakai's internal VO management is through role-based policies. Users
can be allocated roles within each worksite. Roles can be extended by
administrative users from the small number of defaults like ``admin'‘ and
``maintain''.
• Certain users can configure new sites, and resources can be shared
between sites.
• Other concepts include permissions, types, realms, skins, properties,
groups, aliases.
• Sites can be public, private or joinable.
Users see only what they have access to. Some
additional worksites are “joinable”.
Worksite
Role
Front page
(not logged in)
view only
My site
maintain
Site 1
create
Site 2
maintain
Site 3
access
Site x – created
from Site 1
maintain
Each worksite provides a list of tools
and view of underlying content
depending on the user’s role
Roles and Permissions
Managing Users and Tools
Built-in Web 2.0 Services
For the end users, Web 2.0 typically provides “hosted services”
enabling them to use a Web browser to interact with, contribute content
to the Web and invoke remote operations.
There is a growing list of such tools which are being hosted in the
Sakai server and database. They can be rendered as stand-alone pages
or tiled in various combinations as required.
• Blog
• RSS News reader
• Wiki
• Glossary
• Calendar
• Threaded Discussion Forum
• Chat client
• Message Center
• Other tools mashup remote Web 2.0
• Shared Resources
services from Google, Yahoo,
• Announcements
FaceBook, etc.
• Workshop paper management
Wiki and Discussion Forum
Calendar and RSS News
My Workspace
Each user has a workspace which aggregates views of all
the sites they belong to.
Web 2.0 – Map Mashup
Recently we have investigated how to augment the built-in Web 2.0
services by making use of the Yahoo! Maps Web Service. Such a Web
API greatly alleviates the entry level of developing Web 2.0 for geospatial research applications. The services provide a set of APIs (AJAX
or Flash) through which developers can easily access online maps
around the world and overlay their own information (mashup).
What we have tested is to display a map of the Sakai Community similar
to the one located at the Sakai Web site inside our VRE.
We expect this kind of mashup technology to be of use in a number of
research fields such as archeology, flood monitoring and prediction,
climate simulation and urban decision making in addition to supporting
other forms of collaborative working, such as locating Access Grid
rooms. Users will upload their data into “Resources/MapData” and then
select which to overlay on the map.
Screenshot
Rollout, Sustainability and Community
Sakai is running on a fully-supported IBM BladeCenter at Daresbury
Laboratory currently with 28 dual-processor Xeon blades. The content is
hosted in the Oracle 10g database on the UK National Grid Service (RAL
node).
We are currently deploying fully-operational and supported Sakai-based
VREs for the following communities:
NW-GRID: a community of computational scientists, both academic and
commercial, using compute clusters in the North West of England.
ESRC e-Infrastructure: a community of multi-disciplinary social
scientists thoughout the UK building a common infrastructure and
adopting e-Science technology through the work of NCeSS and
ReDReSS http://www.ncess.ac.uk and http://redress.lancs.ac.uk
Diamond e-Infrastructure: a community of experimental scientists
using the new Diamond Light Source http://www.diamond.ac.uk the
largest investment in science in the UK for 30 years.