Transcript NTT99 - BNRG - University of California, Berkeley

Internet-Scale Systems Research Group (ISRG)
MASH
Endeavour
Virtual Room Environment
Desktop Environment
Virtual Room Environment
Proxy
Proxy
Proxy
Virtual Collaborative Environment
Transmission format
chosen by network
characteristics and
receiver capabilities
Internet
Multicast Backbone
Real-Time
Data and Control
Transcoding
Cellular “Core” Network
Ninja
Millennium
ICEBERG
Eric Brewer, David Culler, Anthony Joseph,
Randy Katz, Steven McCanne
Computer Science Division, EECS Department
University of California, Berkeley, CA 94720-1776
1
Outline
•
•
•
•
What is ISRG?
ICEBERG Design Status
New Directions: Endeavour Project
Summary and Conclusions
2
Outline
•
•
•
•
What is ISRG?
ICEBERG Design Status
New Directions: Endeavour Project
Summary and Conclusions
3
Building on Berkeley’s Systems
Expertise
BARWAN
Wireless Overlay Networks
Scalable Proxies
vic, vat,
wb
TranSend
TACC Model
Wireless Access
NINJA
Scalable, Secure Services
Computation in the Network
“Smart Spaces” as an app
Event-Response
Programmable Access
RTPGateway
Service Discovery
MASH
Collaboration Applications
Active Services
MASH Toolkit
Active Services
Model
NOW/Millennium
Computing Platform
4
Mission Statement
Lead the evolution of the Internet through
fundamental protocol and systems research
– Grounded in real-world prototypes that are deployed across
diverse user communities
– Unify on-going and future research projects
– Facilitate technology transfer and standardization
– Work closely with industrial partners in an open laboratory
environment
5
Strategy
• Leverage interdisciplinary systems expertise in
network-based applications, scalable services,
network-connected computing platforms
• Work collaboratively across applications, OS,
networks, architecture
• Interact closely with industry, to obtain early access
to leading edge technologies and facilitate tech
transfer
• Cultivate ties with Bay Area (and beyond!) networking
and systems research community
6
Participating ISRG Sponsors
Service
Companies
Computer
Companies
Communications
Companies
7
Experimental Testbed
Fax
IBM
WorkPad
Image/OCR
Text
Speech
MC-16
Ericsson
CF788
306 Soda
Motorola
Pagewriter 2000
WLAN
405 Soda
326 Soda “Colab”
Pager
GSM BTS
Network
Infrastructure
LMDS, Cable Modems
Millennium Cluster
Smart Spaces
Personal Information Management
Millennium Cluster
8
Outline
•
•
•
•
What is ISRG?
ICEBERG Design Status
New Directions: Endeavour Project
Summary and Conclusions
9
Transparent Information Access
Speech-to-Text
Speech-to-Voice Attached-Email
Call-to-Pager/Email Notification
Email-to-Speech
All compositions
of the above!
Universal In-box
Policy-based
Location-based
Activity-based
10
The Case for ICEBERG
• Integrated services from diverse networks -“PANS” (Potentially Any Network Services).
• Service infrastructure that allows user level
service creation, dynamic customization,
configuration
• Why Internet-based core?
–
–
–
–
Ease of service creation, customization
Low deployment cost
Supports heterogeneity
Bandwidth efficient
11
ICEBERG Design Goals
• Potentially Any Network Services (PANS)
– Any service can from any network by any device;
network/device independence in system design
• Personal Mobility
– Person as communication endpoint with single identity
• Service Mobility
– Retain services across networks
• Easy Service Creation and Customization
– Allow callee control & filtering
• Scalability, Availability, Fault Tolerance
• Security, Authentication, Privacy
12
Ninja: Distributed Platform
Building Blocks
Units
Active
Proxies
Bases
• Data path: abstraction
of data flow - sequence
of operators and
connectors interconnect
to provide certain
services.
• Operator: a unit of
computation
• Connector: an
abstraction of interoperator communications
13
ICEBERG Architectural Elements
• ICEBERG Access Point (IAP)
– Encapsulates network specific gateway (control and data)
• ICEBERG Point of Presence (iPOP)
– Performs detailed signaling
» Call Agent: per communication device per call party
» Call Agent Dispatcher: deploy call agent
• Name Mapping Service
– Mapping between iUID (Iceberg Unique ID) and service end point
• Preference Registry
– Contains user profile including service subscription, configuration and
customization
• Person Activity Tracker (PAT)
– Tracks dynamic information about user of interest
• Automatic Path Creation Service
– Creates datapath among participants’ communications devices
14
A Bird View
PSTN
IAP
IAP
Pager
IAP
GSM
iPOP
iPOP
IAP
Iceberg Network
iPOP
iPOP
IAP
GSM
WaveLAN
IAP
PSTN
Cal
Stanford
15
ICEBERG Signaling System
• Signaling System
– Distributed system w/agents communicating via signaling
protocol for call setup, routing, & control
• ICEBERG Basic Call Service
– Communication of two or more call participants using any
number of communication devices via any kind of media
– If call participant uses more than one devices, must be
used synchronously
• Basic Approach
– Loosely coupled, soft state-based signaling protocol
w/group communication
– Call Session: a collection of call agents that communicate
with each other
16
Signaling Functional Decomposition
Signaling
Call Setup
Call Control
Call Session Data Path Call State Change Data Path
Establishment Creation
Propogation
Modification
Call Agent Deployment
17
Signaling: Call Session Establishment
Alice
1
IAP
2
3
iPOP
Call Agent
Dispatcher
Call Agent
Dispatcher
3
4
Call
Agent
iPOP
Carol
16
IAP
iPOP
Call Agent
Dispatcher
13
15
Call
Agent
5
Call
Agent
12
Bob
11
7
10
Name Mapping
Service
14
8
9
IAP
6
Preference
Registry
18
Signaling: Call Control
• Call Control
– Refers to control protocol in an established call session
– Involves altering & propagating call states in the call session,
and modifying the datapath correspondingly
• Call States
– Call party identities, communication devices in use & their call
status, and datapath information on data streams involved
• Challenge
– Reliable propagation of call state changes to call agents, given
highly dynamic call session environment
» Adapt as session membership changes
» New member must be able obtain current session state
19
ICEBERG Approach for Call Control
• Call Session
– Abstraction of shared communication channel
– Level of indirection to hide identity and location of call
session members (I.e., call agents)
– Adapt to membership change
• Call State
– Soft state-based
– Maintained by each call agent in a session
20
Light-Weight Call Session
Call
Agent
Call State Table
Announce Call Session
Data Path
Table
Listen
Create/tear down
data path
Add or
remove
path
Call
Agent
Announce
Listen
Call
Agent
Create/tear down
data path
Auto Path
Creation
21
Datapath Simplification
• Separate data from control
– Isolate datapath creation from signaling
– Encapsulates media negotiation
• Powerful enabler for any-to-any communication
in ICEBERG due to its flexible composability
• Current use immature and ad-hoc
– Operator with reference count
– Operator description: what and where to run or cleanup
– Who gets to create path
22
Signaling: Fault Detection and
Recovery
• Ninja Distributed Service Environment
– Run all Iceberg components on Ninja Base
• Advantageous separation of iPOP and IAP
– IAP: network specific gateways likely maintain hard state;
Gateways are responsible for maintenance
– iPOP: light-weight call session is the key
• Detection
– IAP and iPOP send heartbeats to each other
– Loss of heartbeat implies loss of life
23
Conference Call: First Class Service
• Redefining conference call
– Call between at least two call parties with at least three
communication devices
• Conference call operations are building blocks
for services
– Add a communication endpoint
– Remove a communication endpoint
• Simplify implementation of services that
require communication endpoint changes
– Change an endpoint = remove + add
24
Example: Service Handoff
• Service handoff occurs when users switch
communication devices in midst of call session
• Enables service mobility
• Service handoff is:
– Generalized call transfer
– Special case of conference call
» User uses one device to invite another device
» Then hangs up the first device
25
Comparison with SIP, H.323
• SIP Differences
– Group vs. pairwise communication
for signaling
– Light-weight session vs. tightly
coupled session
• Our Advantages
– Adaptive to dynamic call session
(i.e., call session membership
change, protocol agent fault
recovery)
– Simplicity in service
implementation
• H.323 Problems
– Complexity: no clean separation
of component protocols; many
options for doing a single task
– Extensibility: requires full
backward compatibility; each
codec is centrally registered
and standardized; not modular
– Scalability: stateful (depends
on TCP); central control for
conference call
– Services: cannot express
preferences
26
Implementation and Current Status
• Prototype system built on Ninja iSpace using
Java (~5000 line code)
• Thread programming model rather than eventdriven -- implicit state machine
• Conference call service operational
• Service handoff now being implemented
(between PSTN, GSM, WaveLAN)
• LDAP for the Name Mapping Service
• Preference Registry: forms-based
specification yielding Perl scripts
27
Future Work
• Run Iceberg components on
Ninja MultiSpace, and exploit
full fault tolerance
• H.323 gateway, pager gateway
• More full-fledged APC
• Experiment with new services
(e.g., policy-based call waiting)
• Person Activity Tracker
• Incorporate SDS for service
location
• Faster preference lookup: use
XSet
• Admission control
• Signaling System
– Performance evaluation
– Soft-state timeout value and
heartbeat rate: scalability of the
bandwidth usage vs. adaptivity
• Security Issues
– User authentication and privacy
– Inter-ISP trust model
– Scope call session multicast
group
28
Outline
•
•
•
•
What is ISRG?
ICEBERG Design Review
New Directions: Endeavour Project
Summary and Conclusions
29
Why “Endeavour”?
• DARPA BAA 99-07: Information Technology
Expeditions
• To strive or reach; a serious determined
effort (Webster’s 7th New Collegiate
Dictionary); British spelling
• Captain Cook’s ship from his first voyage of
exploration of the great unknown of his day:
the southern Pacific Ocean (1768-1771).
– These voyages brought brought more land and wealth to
the British Empire than any military campaign
– Cook’s lasting contribution: comprehensive knowledge of
the people, customs, and ideas that lay across the sea
– “He left nothing to his successors other than to marvel at
the completeness of his work”
30
Expedition Goals
• Enhancing human understanding through
information technology
– Dramatically more convenient for people to interact with
information, devices, and other people
– Supported by a “planetary-scale” Information Utility
» Stress tested by challenging applications in decision
making and learning
» New methodologies for design, construction, and
administration of systems of unprecedented scale and
complexity
– Figure of merit: how effectively we amplify and leverage
human intellect
• A pervasive Information Utility, based on
“fluid systems technology” to enable new
approaches for problem solving & learning
31
Expedition Assumptions
• Human time and attention, not processing or
storage, are the limiting factors
• Givens:
– Vast diversity of computing devices (PDAs, cameras,
displays, sensors, actuators, mobile robots, vehicles); No
such thing as an “average” device
– Unlimited storage: everything that can be captured,
digitized, and stored, will be
– Every computing device is connected in proportion to its
capacity
– Devices are predominately compatible rather than
incompatible (plug-and-play enabled by on-the-fly
translation/adaptation)
32
Expedition Challenges
• Personal Information Mgmt is the Killer App
– Not corporate processing but management, analysis,
aggregation, dissemination, filtering for the individual
• People Create Knowledge, not Data
– Not management/retrieval of explicitly entered
information, but automated extraction and organization of
daily activities
• Information Technology as a Utility
– Continuous service delivery, on a planetary-scale, on
top of a highly dynamic information base
• Beyond the Desktop
– Community computing: infer relationships among
information, delegate control, establish authority
33
Interdisciplinary, TechnologyCentered Expedition Team
•
•
•
•
•
•
•
•
•
•
Alex Aiken, PL
Eric Brewer, OS
John Canny, AI
David Culler, OS/Arch
Joseph Hellerstein, DB
Michael Jordan, Learning
Anthony Joseph, OS
Randy Katz, Nets
John Kubiatowicz, Arch
James Landay, UI
• Jitendra Malik, Vision
• George Necula, PL
• Christos Papadimitriou,
Theory
• David Patterson, Arch
• Kris Pister, Mems
• Larry Rowe, MM
• Alberto SangiovanniVincentelli, CAD
• Doug Tygar, Security
• Robert Wilensky, DL/AI
34
Expedition Approach
• Information Devices
– Beyond desktop computers to
MEMS-sensors/actuators with
capture/display to yield
enhanced activity spaces
• Information
Utility
• Information
Applications
– High Speed/Collaborative
Decision Making and Learning
– Augmented “Smart” Spaces:
Rooms and Vehicles
• Design Methodology
– User-centric Design with
HW/SW Co-design;
– Formal methods for safe and
trustworthy decomposable and
reusable components
“Fluid”, Network-Centric
System Software
– Partitioning and management
of state between soft and
persistent state
– Data processing placement and
movement
– Component discovery and
negotiation
– Flexible capture, selforganization, and re-use of
information
35
High Speed
Learning
Decision Making
Classroom
Collaboration Spaces
E-Book
Vehicles
Info Appliances
Applications
Human Activity Capture
Event Modeling
Generalized UI Support
Transcoding, Filtering, Aggregating
Statistical Processing/Inference
Negotiated APIs
Interface Contracts
Proxy Agents
Self-Organizing Data
Wide-area Search & Index
Nomadic Data & Processing
Wide-Area Data & Processing
Movement & Positioning
Information
Utility
Automated Duplication
Distributed Cache Management
Stream- and Path-Oriented Processing & Data Mgmt
Non-Blocking RMI
PDA
Laptop
Soft-/Hard-State Partitioning
Wallmount Display
Camera
Handset Smartboard MEMS Sensor/Actuator/Locator
Information
Devices
36
Organization:
The Expedition
Cube
D
e
s
I
g
n
M
e
t
h
o
d
o
l
o
g
y
Applications
Rapid Decision Making, Learning,
Smart Spaces: Collaboration Rooms,
Classrooms, Vehicles
Information Utility
Fluid Software, Cooperating Components,
Diverse Device Support, Sensor-Centric
Data Mgmt, Always Available, Tacit
Information Exploitation (event modeling)
Information Devices
MEMS Sensors/Actuators, Smart Dust,
Radio Tags, Cameras, Displays,
Communicators, PDAs
Base Program
Option 1: Sys Arch for Diverse Devices
Option 2: Oceanic Data Utility
Option 3: Capture and Re-Use
Option 4: Negotiation Arch for Cooperation
Option 5: Tacit Knowledge Infrastructure
Option 6: Classroom Testbed
Option 7: Scalable Heterogeneous Component-Based Design
37
Base Program: Leader Katz
• Broad but necessarily shallow investigation
into all technologies/applications of interest
– Primary focus on Information Utility
» No new HW design: commercially available information
devices
» Only small-scale testbed in Soda Hall
– Fundamental enabling technologies for Fluid Software
» Partitioning and management of state between soft
and persistent state
» Data and processing placement and movement
» Component discovery and negotiation
» Flexible capture, self-organization, info re-use
– Limited Applications
– Methodology: Formal Methods & User-Centered Design
38
Option 1: “System Architecture for
Vastly Diverse Devices”
Leader Culler
• Distributed control & resource management:
data mvmt & transformation, not processing
– Path concept for information flow, not the thread
– Persistent state in the infrastructure, soft state in the
device
– Non-blocking system state, no application state in the
kernel
– Functionality not in device is accessible thru non-blocking
remote method invocation
• Extend the Ninja concepts (thin client/fat
infrastructure) beyond PDAs to MEMS
devices, cameras, displays, etc.
39
Option 2: Implementation & Deployment of Oceanic Data Info Utility
Leader Kubiatowicz
• Nomadic Data Access: serverless, homeless,
freely flowing thru infrastructure
– Opportunistic data distribution
– Support for: promiscuous caching; freedom from
administrative boundaries; high availability and disaster
recovery; application-specific data consistency; security
• Data Location and Consistency
– Overlapping, partially consistent indices
– Data freedom of movement
– Expanding search parties to find data, using applicationspecific hints (e.g., tacit information)
40
Option 3: Sensor-Centric Data
Management for Capture/Reuse
Leader Hellerstein
• Integration of embedded MEMS with
software that can extract, manage, analyze
streams of sensor-generated data
– Wide-area distributed path-based processing and storage
– Data reduction strategies for filtering/aggregation
– Distributed collection and processing
• New information management techniques
–
–
–
–
Managing infinite length strings
Application-specific filtering and aggregation
Optimizing for running results rather than final answers
Beyond data mining to “evidence accumulation” from
inherently noisy sensors
41
Option 4: Negotiation Architecture
for Cooperating Components
Leader Wilensky
• Cooperating Components
– Self-administration through auto-discovery and
configuration among confederated components
– Less brittle/more adaptive systems
• Negotiation Architecture
– Components announce their needs and services
– Service discovery and rendezvous mechanisms to initiate
confederations
– Negotiated/contractural APIs: contract designing agents
– Compliance monitoring and renegotiation
– Graceful degradation in response to environmental changes
42
Option 5: Tacit Knowledge Infrastructure/Rapid Decision Making
Leader Canny
• Exploit information about the flow of
information to improve collaborative work
– Capture, organize, and place tacit information for most
effective use
– Learning techniques: infer communications flow, indirect
relationships, and availability/participation to enhance
awareness and support opportunistic decision making
• New collaborative applications
– 3D “activity spaces” for representing decision-making
activities, people, & information sources
– Visual cues to denote strength of ties between agents,
awareness levels, activity tracking, & attention span
43
Option 6: Info Mgmt for
Intelligent Classrooms
Leader Joseph
• Electronic Problem-based Learning
– Collaborative learning enabled by information appliances
• Enhanced Physical and Virtual Learning Spaces
–
–
–
–
–
–
–
Wide-area, large-scale group collaboration
Capture interaction once for replay
Preference/task-driven information device selection
Service accessibility
Device connectivity
Wide-area support
Iterative evaluation
44
Option 7: Safe Component
Design and UI Design Tools
Leader Sangiovanni
• Information Appliances as an application of
hardware/software codesign
– Co-design Finite State Machines (CFSMs)
– Formal methods to verify safety from faults
– Safe partitioning of components into communicating
subcomponents placed into the wide-area
• Model-based User Interface Tools
– Information device user interfaces
– Multimodal interface design for variety of devices
45
Option 8: Scaled-up Field Trials
Leader Katz
• Testbed Rationale
– Study impact on larger/more diverse user community
– Higher usage levels to stress underlying architecture
– Make commitment to true utility functionality
• Increasing Scale of Testbeds
– Building-Scale
» Order 100s individuals
– Campus-Scale
» Order 1000s individuals
– City-Scale
» Order 100000 individuals
46
Putting It All Together
1. Diverse Devices
2. Data Utility
3. Capture/Reuse
4. Negotiation
5. Tacit Knowledge
6. Classroom
7. Design Methods
8. Scale-up
Devices
Component Discovery
& Negotiation
Fluid Software
Utility
Info Extract/Re-use
Self-Organization
Applications
Group Decision Making
Learning
47
Outline
•
•
•
•
What is ISRG?
ICEBERG Design Review
New Directions: Endeavour Project
Summary and Conclusions
48
Summary and Conclusions
• Internet-Scale Systems Research Group: Emerging
Network-centric Distributed Architecture spanning
processing and access
• Open, composable services architecture--the wide-area
“operating system” of the 21st Century
• Beyond the desktop PC: information appliances
supported by infrastructure services--multicast realtime media plus proxies for any-to-any format
translation and delivery to diverse devices
• Common network core: optimized for data, based on IP,
enabling packetized voice, supporting user, terminal,
and service mobility
• New capability and applications focus: Endeavour
49