Transcript eVLBI Progress

Recent e-EVN Developments
Arpad Szomoru
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Outline
•
•
•
•
•
•
The past
Current status
Expansion of e-EVN
EXPReS: first results
Connectivity improvements
The future
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Price recording media ($/GB)
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
January 2002: Proof-of-Concept
e-VLBI over GÉANT
NL
DE
SE
CH
HU
IT
FR
GR
CZ
BE
AT
UK
PT
0
GEANT
GÉANT
ES
SI
PL
IE
HR
LU
RO
EVN
telescope
LV
BG
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
CY
LT
IL
SK
EE
e-VLBI Milestones
20
num_badorder
25000
No LBE
num_lost
18
16
20000
14
12
15000
10
8
10000
6
4
5000
2
0
0
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
20
40
60
80
100
120
Transfer number
140
160
180
0
200
No. Lost
No. Out of order
• September 2002:
• 2 X 1 Gbit Ethernet links to
JIVE
• Demonstrations at
igrid2002 and ER2002
• UDP data rates over 600
Mbit/s
e-VLBI Milestones: 2003
Onsala
Sweden
Chalmers
University of
Technology,
Gothenburg
• May 2003: First use of
FTP for VLBI session
fringe checks.
• November 2003:
Cambridge –
Westerbork fringes
detected, only 15
minutes after
September: e-VLBI
July:10 Gbit•access
• October 2003:observations
first light on were
data transfer
GEANT-Surfnet
Westerbork – JIVE
made.1 Gb/s
between
Bologna
• 6 X 1 Gbit links
to JIVE
• November 2003: Onsala
and JIVE – connection
300Mb/s • 64Mb/s, with
disk Observatory
Space
buffering at JIVE
only. connected at
(Sweden)
1Gb/s.
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
e-VLBI Milestones: 2004
800
Mbps
600
On-Dw
Dw-On
Tr-Dw
Dw-Tr
Bo-Dw
Dw-Bo
400
200
0
m
ax
m
in
• June 2004: network stress
test (iperf) involving
December 20 2004:
• September• 2004:
Bologna, Torun, Onsala
of JBO
• January 2004: Disk
First e-EVN connection
science
and JIVE • September 2004:
to Manchester
at 2 x
buffered
e-VLBI
• March
2004: first realsession (Ar, Cm,
Tr,
• April 2004: ThreeFour telescope
real1 Gb/s
WestfordOn,
Wb)
• On, Wb,time
Cmfringes
at 128Mb/s
telescope,• real-time
(Ar,
Haystack June 2004: Torun time e-VLBI
• e-VLBI test with Tr,
for first GGAO
e-VLBIto
image
•
Spectral
line
fringes at 64Mb/s
connected at 1Gb/s. Cm, Tr, Wb)
and Jb
Intercontinental
observations On
at 32
Wb).
• On –• Wb
fringes(On,
at Jb, real• First fringes to
Ar at • Jb - Tr fringes at
Wf -On,
Mb/s
256Mb/stime •fringes,
First
real-time
EVN
8 EVN 32
Symposium,
Torun, September 2006, A. Szomoru, JIVE
32 Mb/s
Mb/s
256Mb/s
image at 32Mb/s
th
e-VLBI Milestones: 2005
• March 2005: e-VLBI science
session
• Spring 2006: Metsahovi
• January 2005: Huygens • First continuum science connected at 10Gb/s
February
2005: network
observations at 128 and 64
descent tracking,
salvage
transfer test (BWCTL)
Mb/s, involving
6 radio2005: trench
• Summer
of Doppler experiment
employing various
network (Wb,
telescopes
Jb,mile”
Cm,
forAr,
“last
• Use of dedicated
monitoring tools On,
involving
Tr)
connection to Medicina
lightpath Australia-JIVE,
Jb, Cm, On, Tr, Bologna
dug
data transferred
at
~450
and JIVE
8 EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Mb/s
th
1 Gbps
10 Gbps
155 Mbps
EVN Symposium 2004, A. Szomoru, JIVE
2.5 Gbps
Why bother? (change is bad…)
• Target of Opportunity - unscheduled observations triggered
by sudden astronomical events. This capability will become
much more important when LOFAR comes online
• Adaptive Observing - Use e-VLBI as a finder experiment
• Or, e-VLBI sessions a few days apart, adapt schedules for
later observations based on results (rapid results on large
sample, focus in detail on best candidates)
• Automatic Observing - small number of telescopes
observing for extended periods doing spectral line
observations of large galactic samples
• Interface with other real-time arrays – e-MERLIN,
LOFAR, SKA.. Also function as SKA-pathfinder
• Bandwidth no longer limited by magnetic media: 10Gbps
technology already becoming mainstream
• Because we can…
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Recent developments
• Regular science/test sessions throughout the year
• First open calls for e-VLBI science proposals
• First science run completely lost, but, first ever real-time
fringes to Mc (128 Mbps)
• Second and third science runs: many hours of smooth
sailing at 128 Mbps. No excitement, no drama. JIVE
becoming an observatory?
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
First e-EVN Astronomy Publications
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Current status
• Technical tests:
•
•
•
•
6-station fringes at 256 Mbps
first European 512 Mbps fringes (Jb and Wb, May 18)
3-station 512 Mbps fringes (Cm, Wb, On, August 21)
first fringes using new 5 GHz receiver at Mc
• Current connectivity:
• Ar: 64 Mbps in the past, but <32 Mbps this year
• European telescopes: 128 Mbps always, 256 Mbps
often, 512 Mbps to Wb, Jb and On
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Tr connectivity bottleneck – (partially) solved
Black Diamond 6808 switches:
New interfaces (10GE) system in old architecture (1GE)
• Originally 8x1GE interface per card
• 10GE NIC served by 8 x 1GE queues
• Queuing regime – RR (packet based) and flow-based
Flow based:
Max. flow capacity – 1Gbit/s – backround traffic.
There is no known reordering workaround to solve this problem.
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
e-VLBI to South America? SMART-1
SMART-1 factsheet
Testing solar-electric propulsion and other deep-space technologies
Name SMART stands for Small Missions for Advanced Research in
Technology.
Description SMART-1 is the first of ESA’s Small Missions for Advanced
Research in Technology. It travelled to the Moon using solar-electric
propulsion and carrying a battery of miniaturised instruments.
As well as testing new technology, SMART-1 is making the first
comprehensive inventory of key chemical elements in the lunar surface. It
is also investigating the theory that the Moon was formed following the
violent collision of a smaller planet with Earth, four and a half thousand
million years ago.
Launched 27 September 2003
Status Arrived in lunar orbit, 15 November 2004. Conducting lunar orbit
science operations.
Notes SMART-1 is the first European spacecraft to travel to and orbit
around the Moon.
This is only the second time that ion propulsion has been used as a
mission's primary propulsion system (the first was NASA's Deep Space 1
probe launched in October 1998).
SMART-1 is looking for water (in the form of ice) on the Moon.
To save precious xenon fuel, SMART-1 uses 'celestial mechanics', that is,
techniques such as making use of 'lunar resonances' and fly-bys.
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
e-VLBI to Sourth America (2)
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
And other continents..
Australia:
• Telescopes connected
• PCEVN-Mk5 interface needed
China:
• Shanghai Observatory connected at 2.5Gbps
• Connection via TEIN (622Mbps), ORIENT?
Issues with CERNET, CSTNet
• Direct lightpath Hong Kong-Netherlight?
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Hybrid networks in the Netherlands..
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Switch from Cisco to
Nortel/Avici equipment has
been completed: for now, 7
* 1 Gbps, ultimately 16 * 1
Gbps lightpaths + 10 Gbps
IP connection
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
..and across Europe: GÉANT2 network upgrade
Outline
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
EXPReS: getting underway
SA1: new hires at JIVE; two software engineers, one network
engineer (finally!), one e-VLBI postdoc
Inclusion of e-MERLIN telescopes in e-EVN
Operational improvements (deliverable driven):
• Robustness
• Reliability
• Speed
• Ease of operation
• Station feedback
And still, pushing data rates, protocols, UDP, Circuit TCP?
Get rid of fairness… Better usage of available bandwidth.
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
e-VLBI control interface
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Interface to station Mk5s
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Runtime control
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Integrating fringe display
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Data status monitor
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Streamlining of post processing
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Web-based Post-processing
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Ongoing
New control computers (Solaris AMD servers)
• Cut down on (re-)start time
• Powerful code development platform
• Tightening up of existing code
• Operational within next few weeks
Other hardware upgrades:
• SX optics (fibres + NICs), managed switches at JIVE and
SARA
• Mark5A→B: motherboards, memory, power supplies, serial
links, CIBs
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
FABRIC: (Huib Jan van Langevelde)
• Distributed software correlation
• High bandwidth data transport (On part of e-MERLIN @ 4Gbps)
• Two new hires at JIVE
And coming..
SCARIe:
• Collaboration with SARA and UvA
• Distributed software correlation using Dutch grid
• Lambda switching, dynamical allocation of lightpaths,
collaboration with DRAGON project
• JIVE postdoc hired, still looking for UvA postdoc
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Connectivity improvements
Martin Swany
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
•
•
•
•
•
2 Heavy duty gamer PCs
Tyan Thunder K8WE Motherboards
Dual AMD Opteron 2.4GHz processors
4GB RAM
2 1Gb PCI-Expres Nics
• First one at Torun, back-to-back to Mark5
• Second one located at Poznan Supercomputing Centre
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Protocol work in Manchester:
• Protocol Investigation for eVLBI Data Transfer
• Protocols considered for investigation include:
•
•
•
•
•
•
TCP/IP
UDP/IP
DCCP/IP
VSI-E RTP/UDP/IP
Remote Direct Memory Access
TCP Offload Engines
• Work in progress – Links to ESLEA UK e-science
• Vlbi-udp – UDP/IP stability & the effect of packet loss on correlations
• Tcpdelay – TCP/IP and CBR data
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Protocols (1) Mix of High Speed and Westwood TCP (Sansa)
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Protocols (2)
Circuit TCP (Mudambi,
Zheng and Veeraraghavan)
Meant for Dedicated End-toEnd Circuits, fixed
congestion window
No slow start, no backoff:
finally, a TCP rude enough
for e-VLBI?
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
Protocols (3)
Home-grown version of CTCP using pluggable TCP congestion
avoidance algorithms in newer Linux kernels (Mark Kettenis)
Rock-steady 780 Mbps transfer using iperf from Mc to JIVE
Serious problem with new version of Mk5A software under
newer kernels
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE
e-EVN: the future
• Aim: 16 * 1 Gbps production e-EVN network
• Lightpaths across GÉANT: point-to-point connections
between JIVE and telescopes.
• Guaranteed bandwidth, no need to worry about
congestion..
• Depending on connectivity of stations, choice of
configurations with specific data rates
• Towards a true connected-element interferometer; we’re
well on our way
8th EVN Symposium, Torun, September 2006, A. Szomoru, JIVE