Transcript Lecture 6
PSIRP Architectural Components Part 1 Walter Wong NomadicLab & HIIT 08.02.2010 Outline Identifiers Algorithmic IDs Node Internal Architecture Helper Functions Rendezvous System Background – IP-based Identifier Network Layer Identifier ◦ IP address ◦ Topological identifier ◦ Refers to a specific location in the network Transport Layer Identifier ◦ Also IP address (IPsrc/dst, portsrc/dst, protocol) ◦ Identifies end-hosts in the Internet IP Semantic Overload Problem Socket Application 174.180.23/24 Transport IP Network Link Physical 200.201.11/24 128.17.11.22 IP-based Identification – Problems Entangle host location with identification ◦ Host-centric approach Content can’t be addressed alone ◦ ◦ ◦ ◦ Sub-part of a URL Ex. www.unicamp.br/main/courses/index.html Identified as part of connection parameters Ex. TCP sequence number Limited naming system ◦ Hinders new features, ex., mobility, multicast ◦ Users can’t express their interest in data regardless of location PSIRP Identifiers – Motivation Express user interests ◦ WWW users are interested in documents regardless of their location ◦ Currently users express an interest (what) and it is translated to a place (where) Publish/subscribe ◦ Users express their interest on data ◦ Data is asynchronously delivered to users PSIRP Identifiers Application Level Identifier (AId) Rendezvous Identifier (RId) Scope Identifier (SId) Forwarding Identifier (FId) Algorithmic Identifier (AlgId) Application Level Identifier (AId) Application level identification Based application requirements ◦ Easiness to route, easiness to resolve, randomness, etc Can be any namespace ◦ ◦ ◦ ◦ Flat labels Structured names (FQDN) Global Unique Identifier (GUID) Electronic Numbering (ENUM) Rendezvous Identifier (RId) PSIRP network level identifier Uniquely identifies a piece of information 256-bit identifier (SHA-256 hash over the data) Goal: identify the interest between publishers and subscribers in the rendezvous system Rendezvous Identifier (Rid) Interest! Doesn’t subscribe RIdnut Rendezvous Point Bingo Publish RIdnut Deliver data Nut Subscribe RIdnut Scrat Scope Identifier (Sid) Also PSIRP network identifier with 256 bits length Goal: aggregate a set of Rids into one group (scope) Represents a collection of whatever information that Rids represents Scope = implicit context of a set of Rids ◦ Photo album has a Sid and each photo has a Rid Also ◦ Access control ◦ Metadata control operations Scope Identifier (Sid) RidA Sidfamily_album RidB RidC Sidpublic_album Forwarding ID 256-bit long network identifier Goal: identify path segments in the forwarding path Difference: ◦ Sid/Rid used in the rendezvous system (slow path) ◦ Fid used in the forwarding fabric (fast path) Can be aggregated in Bloom filters Forwarding ID Rendezvous 1 FId11 FId12 FId13 Rendezvous 2 FId21 FId22 FId31 David Bloom-filter FIdA Bob Clark Forwarding ID Source controlled packet soft state ◦ Topology manager creates the Bloom filter ◦ + stateless ◦ - requires constant updates from the rendezvous to identify new subscribers ◦ - recycle Fids after un-subscription In-network state ◦ Fid switching state in the forwarding nodes ◦ + Identifies just initial Fid ◦ - state in the switches Algorithmic Identifiers (AIgIds) Special class of identifiers Information collection concept ◦ group semantically similar information items Goal: automatic generation of IDs for different pieces of information Allow elaborate tests with the identifier Algorithmic Identifiers Allows relationship tests on each ID: ◦ Ordering: precedence (does an identifier precede another one?) ◦ Composition: does an ID belong to another ID (e.g. a chunk ID belongs to a file ID) ◦ Completeness: do we have all the identifiers composing another information object? Algorithmic IDs Applications ◦ Automatic deriving identifiers for information fragments ◦ Calculating the next identifier for a sequence ◦ Deducing the identifier Subscription Management Subscribe comics Billy Comics part 1 ID3 Comics Part 3 algID ID2 ID1 Comics Part 2 ID4 ID5 ID6 Comics Part 4 Comics Part 5 Comics Part 6 Forwarding State Aggregation RVS FId21 RVS FId22 FIdB FId12 FId11 David FId13 RVS FId31 FIdC FId A Bloom-filter Bob FId32 Clark Re-encoding the same publication AlgIDhigh_resolution IDA Publisher Publication IDB AlgIDlow_resolution ID1 ID2 IDC Return Path (legacy client-server model) Publish(algIdA) RVS Subscribe(hash(algIdA)) Bob Subscribe(algIdA) Publish(hash(algIdA)) Hey Clark, let’s have a beer after work! Okee dokee, Bob! Clark Sequence Numbering RVS Subscribe(algId) Publish(algId) ID2 ID3 ID1 Alice ID1 = hash(algId) ID2 = hash(ID1) ID3 = hash(ID2) ID3 ID2 ID1 Beth Algorithmic IDs – Other scenarios Error control ◦ generate error messages identified with IDs derived from the error ID Reliability ◦ different algIds can be generated to receive error messages Announcements ◦ prior to a data publication, publishers announce in the algId channel, informing possible subscribers that data will be published in the related algId Algorithmic ID – General design Direct Acyclic Graph Identifiers are related to each other by functions Sorting algorithms ◦ Ordering the identifiers Trees are generated by consecutive application of one-way functions Sequences can be expressed as the recursive application of a singular function Reverse function to walk in both directions Algorithmic IDs – Example B = f(A,1) C = f(A,2) A D = f(C,1) B,1 C,2 D,1 E,2 C = f-1(F,3) A = f-1(C,2) F,3 G,1 H,1 Node Internal Architecture Blackboard Approach ◦ Communication model for sharing memory objects through the network Uses Memory Object Service ◦ Simple unreliable page service ◦ Simple memory object service Unreliable Page Service Simple unreliable page service ◦ Pages can be mapped to packet level transmissions using PageIDs ◦ Allows for identifying and caching individual packets within the network ◦ There is one PageID for each Fid Memory Pages PageIDA PageIDB PageIDC PageIDD PageIDE Memory Object Service Works on the page service Supports larger data files (over one memory page) Supports update on memory objects Memory Pages Memory Object PageIDA PageIDB PageIDC PageIDD PageIDE MOA Memory Object Creation & Publication Blackboard Memory Object Publisher 1. Publisher requests a meta-publication and a memory object 2. Memory service returns a memory object with the meta attached 3. Publisher fills the data in the memory object 4. Publisher creates a Rid for the memory object. It can be republished as many times as he needs with equal or different tags Memory Object Subscription Subscriber subscribers to the MO-Rid Subscriber receives an event stream ◦ File descriptor which can be read with select Whenever the publisher publishes some data, the subscriber will be notified and will receive a new memory object representing the corresponding version of the object Node Internals – Networking Nodes ◦ Local input queues, with one Fid for each queue ◦ Local output queues, with one Fid for each queue ◦ Sender-helper sends the packet identified with the interfaces Fid ◦ Network-receiver receives packets from the network Node Internal Architecture Publish call Application Node Internal - Networking Sender Helper Network Receiver Output queue 1 (FId1) Output queue 2 (FId2) Input queue 1 (FId3) Input queue 1 (FId4) Node Internal Architecture Subscribe call Application Node Internal - Networking Sender Helper Memory Object Network Receiver Output queue 1 (FId1) Output queue 2 (FId2) Input queue 1 (FId3) Input queue 1 (FId4) Memory Object Segmentation Packetiser helper ◦ handles memory objects larger than a packet size ◦ Partitions objects into a set of packets At each local output places, whenever there is a new object, the packetiser chunks the data and places each of them in the correct output queue scope There is no memory copy, just a new mapping of the object in the scope directory Memory Object Segmentation Local Blackboard M D0 D1 D2 Packetiser Depacketiser Output Places Input Places Output Queue Input Queue Network M D0 D1 D2 Helper Functions Not core network service Not traditional applications ◦ Actually, it is a set of functions providing additional features for the system ◦ “Helps the system!” PSIRP has three categories: ◦ Network Management Functions ◦ Remote Service Functions ◦ Host Centric Functions Network Management Functions Network Management & Information gathering Information gathering for better planning and management ◦ Increase performance In PSIRP, network management functions can subscribe to the link state, receiving information as it becomes available Network Management Functions – Example Current Internet ◦ SNMP/ICMP ◦ GET/SET/TRAP directives ◦ Managed systems generate asynchronous messages to notify new events Network Management Functions – Unified Link Layer API (ULLA) Closely resembles publish/subscribe Introduces an interface to collect information from the link layer Link aware applications express their interests by specifying link conditions Notifications can be asynchronous or periodic ULLA can be the base for the helper function which subscribes to the link conditions and wait for updates Remote Service Functions Segmentation function ◦ Helps in the segmentation decision ◦ Extracts data from link layer, such as MTU, and provides parameters for the fragmentation Merger function ◦ Merges partitioned data to get the original publication Remote Service Functions Forwarding Error Correction/Content protection ◦ Error correction in the transmission ◦ Recover received data without retransmissions ◦ Send redundant data together ◦ Ex. Reed Solomon Block Code (N, K) ◦ N-K symbol redundancy ◦ After receiving any K symbols out of N, the receiver is able to decode the original data FEC Parameters ◦ Currently, the FEC parameters are statically set, degrading the performance ◦ Inaccuracy in the estimation of the underlying channel state, resulting in inefficient bandwidth usage ◦ FEC helper can subscribe to the channel conditions in order to generate better parameters Remote Service Functions Content re-encoding ◦ Scenario: heterogeneous subscribers requesting the same video but with different resolution ◦ Content provider can’t attend the demand ◦ Content provider asks for remote helper functions located closer to the subscribers to perform the translation Application-specific Caches Content caching ◦ Offers the same interface as the original content provider (transparent) ◦ Applications can select from which cache to retrieve data ◦ content providers may instruct caches to serve content on their behalf ◦ subscribers can receive data from the closest caches, saving network resources Host Service Functions Transport and other middle network services PSIRP offers shared blackboard for distributed applications using the network Analysis and debugging purposes ◦ Ping, traceroute, tcpdump, wireshark PSIRP context ◦ Transport component for rate control and reliability algorithm for point-to-point delivery Rendezvous System Meeting point of interests between publishers and subscribers Requirements ◦ Scalability (Internet scale) ◦ Efficiency in operation ◦ Deployability (gradual deployment and stakeholders incentives) Rendezvous Network Composed of Rendezvous Nodes (RNs) Organized as a BGP-like inter-domain hierarchy Collection of RNs from multiple cooperative ASes Rendezvous points (RPs) ◦ logical meeting points of interests between publishers and subscribers ◦ one rendezvous point for each Sid/Rid pair Rendezvous Network Interconnection Some approaches ◦ Central entity managing signaling reachability: lack of incentives, trust, competition ◦ Multiple entities: providers compete for global reachability coverage ◦ Fully distributed mechanism: rendezvous networks self-organize to interconnect without any third party infrastructure Rendezvous Network Connection Canon DHT ◦ Hierarchical DHT ◦ Maintains local communication within its domain ◦ Caching in the sub-domain exit between levels in the DHT Canon Hierarchical DHT Top-level Domain 0 13 2 12 3 10 5 8 0 5 2 3 12 10 13 8 Domain A Domain B Rendezvous System Example RNA RNB RNB10 RNC RP RNC10 RNC100 RNB100 Pub Node Node Sub Rendezvous System Bootstrapping RNA RNB RNB10 RNC RP RNC10 RNC100 RNB100 Pub Node Node Sub Publish Operation RNA RNB RNB10 RNC RP RNC10 RNC100 RNB100 Pub Node Node Sub Subscribe Operation RNA RNB RNB10 RNC RP RNC10 RNC100 RNB100 Pub Node Node Sub Summarizing Identifiers ◦ Many types, special semantics Algorithmic ID ◦ New use cases Node Internal Architecture ◦ Blackboard, Input/output queues, memory objects segmentation Helper functions ◦ Adding new functionalities in the architecture Rendezvous System ◦ Matching interests between publishers and subscribers Questions? Comments? Thanks!