Transcript NinjaScript - Microsoft Research

NinjaScript
A Dataflow Language for Composing
Network Services in Ninja
Andrew Begel
NinjaScript Presentation CS264
December 2, 1998
1
Overview
•
•
•
•
•
What is Ninja?
What’s the problem?
How can NinjaScript help?
NinjaScript implementation
Future Work
NinjaScript Presentation CS264
December 2, 1998
2
What is Ninja?
• A distributed systems architecture created
by Brewer, Joseph, Katz, Kubi, and lots of
graduate students
• Fault-tolerant, highly-available services
exist in the network ether (takes ideas from
HotBot)
• Currently based on Java with JavaRMI for
communications.
NinjaScript Presentation CS264
December 2, 1998
3
What is the problem?
• We want to create applications that use
many services.
• Existing systems like CORBA, DCOM,
ILU, or JavaRMI provide client-server
connections between pairs of nodes.
That’s not enough!
NinjaScript Presentation CS264
December 2, 1998
4
Pathways are the answer.
• We want to develop compositions of
multiple services. Individual connections
have no knowledge of the bigger picture.
• Benefits from the path metaphor:
– Can ask questions about the path: congestion,
bottlenecks, bandwidth, latency, admission control,
security
– Language Benefits: Optimizations (pipelining,
commutative services), Debugging
NinjaScript Presentation CS264
December 2, 1998
5
NinjaScript
• It’s a path-based
programming
language!
• Think of a big graph
with a start node and
some finish nodes.
• Several language
metaphors wrapped up
in one.
NinjaScript Presentation CS264
– Lambda Calculus
– Multiple, Conditional
Outputs
– Late Binding
– Subroutines
– Soft State
– Type Safety
December 2, 1998
6
An Example Program
Voice-Activated Light Switch
The basic path.
(To all other nodes, the interpreter is a black box.)
NinjaScript Presentation CS264
December 2, 1998
7
The Interpreter
The internals of the interpreter. It has a subroutine call to the
base station and ultimately outputs a command to the light switch.
NinjaScript Presentation CS264
December 2, 1998
8
Implementation
• Path program is description of nodes and
edges of a graph.
• Nodes will run services that implement
published interfaces.
• Edges have properties such as reliability, inorder/out-of-order transmission, buffering,
flow control.
NinjaScript Presentation CS264
December 2, 1998
9
Path Threads
• A path thread consists of a 4-tuple
– unique ID
– graph description:
• nodes (service name, bootstrap code)
• edges (source, sink, properties)
• start node and finish nodes
– current state of execution
• (state, IP address, method signature)
– data payload (function arguments)
NinjaScript Presentation CS264
December 2, 1998
10
Node Operation
• Upon receiving a packet, check the unique
ID. If it’s new, create a new thread, else
restore the old thread to respond to the data.
• Unpack the packet data, and check the
payload for function arguments. Unpack
these and store in the thread.
• Invoke the method identified by the current
state.
NinjaScript Presentation CS264
December 2, 1998
11
Node Operation (2)
• After the method returns its arguments,
package them up in a new data payload.
• Consult the graph description to determine
the next state.
• If this is a finish node, exit. Otherwise,
package up a new packet and send it to the
next nodes.
NinjaScript Presentation CS264
December 2, 1998
12
Extra Features
(not done yet)
• Subroutines
• Conditional Output
– Enables more specific control over which
outputs go where, if they go anywhere at all.
• Soft State
– Path local
– Node global
NinjaScript Presentation CS264
December 2, 1998
13
Future Work
• Make the whole thing work.
• Incorporate multi-language service support in the compiler
backend.
• Create alternate graphical front-end.
• Integrate existing transport layers (IIOP, RMI) and
alternate transport protocols.
• Develop new node code format (currently using Solaris
Sparc shared libraries).
• Develop path-based optimization, debugging and
diagnostic tools.
• Support paths as first-class objects (exception handling).
NinjaScript Presentation CS264
December 2, 1998
14