Transcript No Slide Title

Authority,
Virtual Organizations and
Diagnostics:
Building and Managing Complexity
Ken Klingenstein
Director, Internet2 Middleware and Security
Topics
 Background on Internet2 Middleware and International
efforts
 The model: enterprises, federations and virtual
organizations; the unified field theory of trust
 The deliverables
• Shibboleth – interrealm exchange of attributes and authorizations
• Signet – a privilege management system
• Virtual organizations – serving collaborative communities in science
and humanities
• Diagnostics – when it doesn’t work
 The next year or so
MACE (Middleware Architecture
Committee for Education)
 Purpose - to provide advice, create experiments, foster standards, etc.
on key technical issues for core middleware within higher education
 Membership - Bob Morgan (UW) Chair, Tom Barton (Chicago), Scott
Cantor (Ohio State), Steven Carmody (Brown), Michael Gettes (Duke),
Keith Hazelton (Wisconsin), Paul Hill (MIT), Jim Jokl (Virginia), Mark
Poepping (CMU), Bruce Vincent (Stanford), David Wasley (California),
Von Welch (Grid)
 European members - Brian Gilmore (Edinburgh), Ton Verschuren
(Netherlands), Diego Lopez (Spain)
 Creates working groups in major areas, including directories, interrealm
access control, PKI, video, P2P, etc.
 Works via conference calls, emails, occasional serendipitous in-person
meetings...
Internet2 Middleware and the NSF
Middleware Initiative (NMI)
 Internet2 Middleware a major theme for the last five years, drawing
support from 206 university members, 75+ corporate members, and
government grants and interactions
 Internet2 has an integrator role within NMI, the key NSF Program to
develop and deploy common middleware infrastructures
 NMI has two major themes
• Scientific computing and data environments (ala Grids)
• Common campus and inter-institutional middleware infrastructure (ala
Internet2/EDUCAUSE/SURA work)
 Issues periodic NMI releases of software, services, architectures,
objectclasses and best practices – R5 most current release
International efforts
 Terena as an anchor for a succession of middleware
discussions and initiatives
 Conspicuous national efforts in Spain, Switzerland, The
Netherlands, the Nordic countries and a few other
European countries.
 Major initiative now underway by JISC in the UK, with
coordinated advancement in authorization, virtual
organizations, digital rights management, and other
areas.
 Australian efforts rapidly advancing; the rest of the Pacific
Rim lags…
The Model:
Enterprises and Federation
Given the strong collaborations within the academic
community, there is an urgent need to create inter-realm tools,
so
Build consistent campus and enterprise middleware
infrastructure deployments, with outward facing objectclasses,
service points, etc. and then
Federate those enterprise deployments, using the outward
facing campus infrastructure, with interrealm attribute
transports, trust services, etc. and then
Leverage that federation to enable a variety of applications
from network authentication to instant messaging, from video
to web services, and then, going forward
Create tools and templates that support the management and
collaboration of virtual organizations by building on the
federated campus infrastructures.
Middleware Axioms
 Work the core areas
 Focus on support for collaboration
 Use federated administration as the lever; have the enterprise
broker most services (authentication, authorization, resource
discovery, etc.) in inter-realm interactions
 Develop a consistent directory infrastructure within R&E
 Provide security while not degrading privacy.
 Foster interrealm trust fabrics: federations and virtual
organizations
 Leverage campus expertise and build rough consensus
 Support for heterogeneity and open standards
 Influence the marketplace; develop where necessary
A Map of Campus Middleware Land
Federated administration
VO
VO
O
A CM
O
T
T
Campus 1
T
CM A
Campus 2
T
T
Federation
Unified field theory of Trust
 Bridged, global hierarchies of identification-oriented, often
government based trust – laws, identity tokens, etc.
• Passports, drivers licenses
• Future is typically PKI oriented
 Federated enterprise-based; leverages one’s security
domain; often role-based
• Enterprise does authentication and attributes
• Federations of enterprises exchange assertions (identity and
attributes
 Peer to peer trust; ad hoc, small locus personal trust
• A large part of our non-networked lives
• New technology approaches to bring this into the electronic world.
• Distinguishing P2P apps arch from P2P trust
 Virtual organizations cross-stitch across one of the above
The Deliverables
 Shibboleth – a secure, privacy-preserving transport for
attributes between realms and within federations
 Signet – a meta-authority system that leverages
enterprise roles to drive sophisticated authorization
options
 Virtual organizations – combining enterprise services with
stand-alone services to provide consistency and
transparency to the VO participants
 Diagnostics – coupling existent and yet-to-be-defined
exception handling across a multi-layered (application,
middleware, security, network) distributed environment
Shibboleth Architecture
Milestones
 Project formation - Feb 2000 Stone Soup; process began late
summer 2000 with bi-weekly calls to develop scenario,
requirements and architecture.
 Linkages to SAML established Dec 2000
 Architecture and protocol completion - Aug 2001
 Design - Oct 2001
 Coding began - Nov 2001
 Alpha-1 release – April 24, 2002
 OpenSAML release – July 15, 2002
 v1.0 April 2003; v1.1 July 2003; v1.2 May 2004
 v2.0 likely end of the major evolution
Shibboleth Status
 Open source, privacy preserving federating software
 Being very widely deployed in US and international universities
 Target - works with Apache(1.3 and 2.0) and IIS targets; Java
origins for a variety of Unix platforms.
 V1.3 likely to include portal support, identity linking, non web
services (plumbing to GSSAPI,P2P, IM, video) etc.
 Work underway on intuitive graphical interfaces for the powerful
underlying Attribute Authority and resource protection
 Likely to coexist well with Liberty Alliance and may work within the
WS framework from Microsoft.
 Growing development activities in several countries, providing
resource manager tools, digital rights management, listprocs, etc.
 http://shibboleth.internet2.edu/
Adoption
 Over 50 + universities using it for access to OCLC,
JSTOR, Elsevier, WebAccess, Napster, etc.
 Common status is “moving into production”
 The hard part is not installing Shibboleth but running
“plumbing” to it: directories, attributes, authentication
 Deployments in Europe, the UK, South America and
Australia
 Needs federations to scale; being adopted by, or
catalyzing, national R&E federations in several countries
Signet: Stanford Authority System
Signet Deliverables
The deliverables consist of
A recipe, with accompanying case studies, of how to take
a role-based organization and develop apprpriate groups,
policies, attributes etc to operate an authority service
Templates and tools for registries and group management
a Web interface and program APIs to provide distributed
management (to the departments, to external programs) of
access rights and privileges, and
delivery of authority information through the infrastructure
as directory data and authority events.
Home
Grant Authority Wizard
Virtual Organizations
Geographically distributed, enterprise distributed
community that shares real resources as an organization.
Examples include team science (NEESGrid, HEP, BIRN,
NEON), digital content managers (library cataloguers,
curators, etc), life-long learning consortia, etc.
On a continuum from interrealm groups (no real resource
management, few defined roles) to real organizations
(primary identity/authentication providers)
Want to leverage enterprise middleware and external trust
fabrics
Virtual Organizations
 Some things seem consistent across almost all VO’s
• The need to manage and delegate VO authorizations
• Unique naming, and managed resource discovery
• A set of collaboration tools, including a list manager, calendar,
shared web content management, etc that are seamlessly
integrated into users’ everyday environment
• A need to factor in, and leverage, local domain requirements and
capabilities
 Some things are specific to each VO
• The members and the resources being managed
• Requirements for advanced services, such as Grids and instrument
management
Virtual organizations
 Need a model to support a wide variety of use cases
• Native v.o. infrastructure capabilities, differences in enterprise
readiness, etc.
• Variations in collaboration modalities
• Requirements of v.o.’s for authz, range of disciplines, etc
 JISC in the UK has lead; solicitation is on the streets (see
(http://www.jisc.ac.uk/c01_04.html); builds on NSF NMI
 Tool set likely to include seamless listproc, web sharing,
shared calendaring, real-time video, privilege
management system, etc.
Leveraging V.O.s Today
VO
User
Federation
Enterprise
Target Resource
Leveraged V.O.s Tomorrow
VO
User
Collaborative Tools
Authority System
etc
Federation
Enterprise
Target Resource
Middleware Diagnostics
Problem Statement
• The number and complexity of distributed application
initiatives and products has exploded within the last 5
years
• Each must create its own framework for providing
diagnostic tools and performance metrics
• Distributed applications have become increasingly
dependent not only on the system and network
infrastructure that they are built upon, but also each other
• Middleware diagnostics need to integrate with network
performance diagnostics and security diagnostics
Goals
• Create an event collection and dissemination
infrastructure that uses existing system,
network and application data (Unix/WIN logs,
SNMP, Netflow©, etc.)
• Establish a standardized event record that
normalizes all system, network and application
events into a common data format
• Build a rich tool platform to collect, distribute,
access, filter, aggregate, tag, trace, probe,
anonymize, query, archive, report, notify,
perform forensic and performance analysis
Event Record Standard
• Normalization of each diagnostic data feed type (SHIB,
HTTP, Syslog, RMON, etc.) into a common event record
• The tagging of specific events to help downstream
correlation processes
GRID Application Log
HTTP Access log
SHIB log
DB Access Log
RMON Events
Cisco NetFlow Events
Normalization
And Event
Tagging
Variable Star Catalog DB
Application
GRIDAPP:TIME:HOST:UID:…
HTTP:TIME:HOST:URL…
SHIB:TIME:HOST:UID…
DB:TIME:HOST:REQ:ASTRON
RMON:HOST:TIME:DSTPORT..
NETFLOW:TIME:SRC:DST:…
Diagnostic Data Pipelining
Data flows can be constructed to provide the desired
function and policy within a enterprise or federation
Host or
Security
Events
C-1
C-2
P-1
P-2
P-4
P-3
P-5
C-3
Network
Events
C-4
Filter
Tagging
Normalization
Aggregation
Anonimization
DB
Archive
C-* Collection Module Host
P-* Processing Module Host
Event Record
Event Descriptor Meta Field
Event Descriptor
Raw Event Data
• Version Number
• Observation Description Pointer
• ID – unique event identifier
• Time - start/stop
• IP Address(es) – source/(destination)
• Source Class – application, network, system, compound, bulk, management
• Event Name Tag – Native language ID, user defined
• Status – normal, informational, warning, measurement, critical, error, etc.
• Major Source Name – filename, Netflow, Syslogd, SNMP, shell program, etc.
• Minor Source Name – logging process name (named), SNMP variable name, etc.
• Raw Data Encoding Mechanism – Binary, ASN1, ASCII, XML, etc.
• Raw Event Data Description Pointer
The next year or so
 An integrated marketplace for identity management
services, packaged with work, home and personal forms
 Federations and international peering of trust
 More integration between Grids and enterprises
 Virtual organization services
• A mix of enterprise, community and outsourced options
 Adaptation of Signet-type privilege management
• New business models for content and service providers
 Diagnostic hell
• Things will get much worse before they get better