Transcript January 2008

Progress on EXPReS at JBO
EXPReS Board Meeting, Utrecht, Jan 2008
Ralph Spencer and the JBO team
Contents
•
•
•
•
•
Outline of JBO project
Manpower
Protocol work
Progress on iBOBs
Handover to Richard.
Jan 2008
Progress with iBOBs at Jodrell
Slide #2
Outline of Work at JBO
• NA4 Outreach work – Alastair Gunn - completed
• SA1 Connection of 4 e-MERLIN telescopes to JIVE (see under
Fabric)
• SA2: The 10 Gbps data link equipment available, awaiting cross
connection in Manchester between academic and commercial
links
• Fabric: Protocols for high speed networks –Stephen Kershaw :
PFLDNet 2008 Manchester 5-7 March
http://www.hep.man.ac.uk/PFLDnet2008/aboutManchester.php
• Fabric: Input to e-MERLIN correlator at 4 Gbps
Jan 2008
Progress with iBOBs at Jodrell
Slide #3
E-MERLIN---e-VLBI system
Jan 2008
Progress with iBOBs at Jodrell
Slide #4
January 2008
Network protocols
Stephen Kershaw
TCP
• Is TCP suitable for
transferring realtime data?
• If data is lost TCP
will slow
transmission and
retransmit.
• Demonstrate the
feasibility of using
TCP for real-time
applications
Jan 2008
Progress with iBOBs at Jodrell
Slide #7
CUBIC
TCP variants
• TCP using
different algorithms
for the
window/throughput
reduction
• Some variants
allow more efficient
use of bandwidth
and appear more
suitable for realtime applications
Jan 2008
HighSpeed
Progress with iBOBs at Jodrell
Slide #8
TCP variants
• Orders of
magnitude
improvements for
speed of recovery
with many variants
• Orders of
magnitude
reduction in
buffering
Note spikes due to bursts of data
• Evident bursty
behaviour –
concerns for highly
loaded packet
switched networks
Jan 2008
Progress with iBOBs at Jodrell
Slide #9
Multicast
Multicast reduces load on networks when sending to
multiple hosts
Test of multicast topology
• Multicast UDP stream over JANET academic network
• 100 2.2Mbps multicast IPTV streams originating from
London, UK
• End-hosts on Manchester University campus network
• All Cisco network hardware, 100Mbps interfaces
• Measurement of inter-packet arrival times
• Time from arrival of packet ‘n’ to arrival of packet ‘n+1’
• Expected ~6ms for one 2.2Mbps stream
Jan 2008
Progress with iBOBs at Jodrell
Slide #10
Multicast
• Latency spikes
of up to 0.5s at
intervals of 60s
• Unexpected
behaviour may
limit the
application of this
technology
• Correlation
between streams
and between PCs
shows a common
source of delay
Jan 2008
Progress with iBOBs at Jodrell
Slide #11
Multicast
Test of multicast topology - results
• Multicast traffic routing is not as efficient as unicast on this
network
• Potential concerns for other networks
• Does not look suitable for eVLBI
Jan 2008
Progress with iBOBs at Jodrell
Slide #12
DCCP
Stable kernel
• Require tuning of virtual memory settings to prevent kernel
panics
Experimental kernels
• Revisions of late November 2007 address some memory
and performance issues – yet to be tested on our systems.
• CCID4 in development
Performance
• CCID2 can attain 1GE line-rate but it is not a turn-key thing
Future work
• Test 2.6.24 kernel and recent DCCP git tree – not mature
enough yet for a definitive statement about suitability
Jan 2008
Progress with iBOBs at Jodrell
Slide #13
Progress on Digital Interfaces
iBOBs and all that
Jonathan Hargreaves/Richard Hughes-Jones
The iBOB System
• As a transmitter:
• Now able to transfer data at line rate to a test PC over a 10
GE (CX-4) link
• Needs A/D connection and interface (Onsala/Metsahovi)
• As a receiver
• Needs to interface to VSI chip on station board
• As a network test system
• See also Richard’s talk
• Fujitsu switch ordered
Jan 2008
Progress with iBOBs at Jodrell
Slide #15
JBO Receiver: Status
Recent Progress
• Sixteen packet FIFO implemented in the iBOB’s FPGA
• Even though packets are transmitted at 4Gbps they might ‘bunch up’ and
arrive at line speed. The FIFO is intended to smooth them out again
• The 10Gb transceiver can run at 200MHz – enough to handle line speed,
but the SRAM bandwidth is 167MHz for both read and write. Read out at
4Gbps consumes 64MHz of that. More resource than needed
• Data flows from the iBOB to the Station Board defined (see next slide)
• Data readout from the SRAM synchronised to the correlator clock, when
present
• When correlator clock is not present, iBOB defaults to test mode using
the internal clock
• VSI chip will buffer and reclock data and route it to the Station Board
input
Next Tasks
• Test the new code in hardware using another iBOB to generate test packets
• Write firmware for the VSI chip and either simulate or test on remote
hardware
Jan 2008
Progress with iBOBs at Jodrell
Slide #16
JBO Receiver: Block Diagram
Clock Control
Lock 128MHz
clock to correlator
200MHz System Clock
128MHz
.
10GB RX
FIFO
‘Yellow
box’ on
the
Simulink
design
4GB/s
CX4
SRAM Control
16k deep
.
Re-orders out of
order packets
Holds up to 16 packets
in case short bursts
arrive at line rate
DATA
64 bit
DATA
64 bit
.
Counts lost
packets
Buffer and
reclock data.
Correlator CLK
DATA 32 bit
Reads data out to
the correlator at a
steady 4Gbps
ADD
R
‘VSI chip’
DATA 32 bit
Reroute it to the
left and right
polarisation
station board
(SB) inputs.
.
DAT
A
SRAM
iBOB
512k x 64 bit
(512 packets =
9ms at 4Gbps)
2x
MDR80
‘VSI’
cables
Station Board
.
Jan 2008
Progress with iBOBs at Jodrell
DATA to
SB Input
Slide #17
DATA to
SB Input
Network Testing Device: Status
Recent Progress
• System clock increased from 100MHz to 200MHz to allow line speed data
flows at 10Gbps (JH)
• Added jumper bank to select ethernet IP and MAC address (JH & RHJ)
• All registers, test modes and results can be accessed via ethernet
control/response protocol (RHJ)
• PC based software provides user interface to the ethernet control link (RHJ)
• Ethernet based control supersedes the TinySH command line and is seen as a
prototype for control of iBOBs sending real data
• Currently testing iBOB to iBOB transfers on bench
Next Tasks
• Count lost packets, out of order packets and duplicate packets
• Update real time field correctly each second – currently it is fixed over a burst
• Test over network
Jan 2008
Progress with iBOBs at Jodrell
Slide #18