Transcript Some Sample Commercial Network Costs

e-VLBI: Overview and Update
Alan R. Whitney
MIT Haystack Observatory
Traditional VLBI
The Very-Long Baseline
Interferometry (VLBI)
Technique
(with traditional data recording)
The Global VLBI Array
(up to ~20 stations can be used
simultaneously)
VLBI Science
Quasars, hotspots, polarization
ASTRONOMY
• Highest resolution technique available to
astronomers – tens of microarcseconds
• Allows detailed studies of the most distant
objects
Chautauqua 2001
Plate-tectonic motions from VLBI measurements
GEODESY
• Highest precision (few mm) technique
available for global tectonic measurements
• Highest spatial and time resolution of
Earth’s motion in space for the study of
Earth’s interior
•Earth-rotation measurements important
for military/civilian navigation
•Fundamental calibration for GPS constellation
within Celestial Ref Frame
VLBI astronomy example
How most VLBI is done today
• VLBI data still largely recorded on
magnetic tape
• Tapes are ~$1000 each and are no
longer available
• Shipping costs are high and delays
can be weeks
Mark 4 VLBI Correlator at Haystack Observatory
• Correlator unit can simultaneously process up to 16 stations at 1Gbps/station
Scientific Advantages of e-VLBI
• Bandwidth growth potential for higher sensitivity
– VLBI sensitivity (SNR) proportional to square root of Bandwidth
resulting in a large increase in number of observable objects
(only alternative is bigger antennas – hugely expensive)
– e-VLBI bandwidth potential growth far exceeds recording capability
(practical recordable data rate limited to ~1 Gbps)
• Rapid processing turnaround
– Astronomy
• Ability to study transient phenomena with feedback to steer observations
– Geodesy
• Higher-precision measurements for geophysical investigations
• Better Earth-orientation predictions, particularly UT1, important for
military and civilian navigation
Practical Advantages of ‘e-VLBI’
• Increased Reliability
– remove recording equipment out of field
– remote performance monitor & control capability in near real-time
• Lower Cost
– Automated Operation Possible
• eliminates manual handling and shipping of storage media
– Near Real-time Processing
• forestalls growth of storage-capacity requirements with bandwidth growth
– Elimination of recording-media pool (millions of $’s!)
• Avoid unexpected media-shipping interruptions and losses
Elements of e-VLBI Development
•
Phase 1: Develop eVLBI-compatible data system
–
–
•
Mark 5 system development at MIT Haystack Observatory being supported by NRAO, NASA,
USNO plus four international partners
Prototypes now deployed in U.S. and Europe
Phase 2: Demonstrate 1 Gbps e-VLBI using Bossnet
(w/ DARPA and NASA support)
– ~700km link between Haystack Observatory and NASA/GSFC
– First e-VLBI experiment achieved ~788Mbps transfer rate
•
Phase 3: Establish adaptive network protocol
(newly awarded NSF grant to Haystack Observatory; collaboration with MIT Lab for Computer
Science and MIT Lincoln Laboratory);
– New IP-based protocol tailored to operate in shared-network ‘background’ to efficiently using
available bandwidth
•
Phase 4: Extend to global scale
–
High-speed global research networks will facilitate the global application of e-VLBI
Mark 5 VLBI Disk-Based Data System (Phase 1)
• 1 Gbps continuous recording/playback to/from set of 8 inexpensive (ATA) disks
• Developed at MIT Haystack Observatory with multi-institutional support
• Mostly COTS components
• Two removable ‘8-pack’ disk modules in single 5U chassis
• With currently available 200GB disks – capacity of single ‘8-pack’ 1.6TB;
expected to increase to 2.5TB by early 2003 at cost of ~$1/GB
• GigE connection for real-time and quasi-real-time e-VLBI operations
• Inexpensive: <$20K
• ~20 Mark 5 systems now installed at stations and correlators
Bossnet 1 Gbps e-VLBI
demonstration experiment
(Phase 2)
Westford
Haystack
(correlator)
Future
Initial experiment
USNO
(correlator)
NASA/GSFC
Details of path from Haystack to ISI-E – work in progress!
Westford Antenna
Millstone RPE Bldg (Lorraine)
Dedicated no scheduling
GigE 1 Summit
WS/
Mark 5
A2
A1
GigE
8
1i
Summit
A4
5i
A3
C2
Haystack Correlator
GigE
Dedicated no scheduling
WS/ 28-1A
Mark 5 B2
14
B1
13
B6
B5
GlowNet
WDM
Amp
D3
D2
Lincoln Lab (Lorraine/Steve)
Schedule to be
coordinated with LL
To be replaced by
Juniper M20 Jan 02 with
Cisco
OC-48 to Bossnet
E3
To be provided
by Haystack
E5
Test
WS
GlowNet
D1
Bossnet(Steve/Jim)
ISI-E
E7
GigE
E4
To be upgraded from
2 to 3
Scheduling on
Bossnet calendar
GigE Summit GigE Summit GigE
1310
5i
E2
5i
E6
E1
Cisco
1500
GigE
Congress St
Boston
B3
1500
(Shared with LL on
scheduled basis)
WDM
GigE
LL
Plan install by 1 Jan 02
C3
16
GigE Summit
5i
GigE
15
WS/ 28-1B
Mark 5 B4
Schedule to be
coordinated with LL
C1
MIT
Campus
WDM
F1
1510
Amp
Amp
F2
E8
Other users
(no control)
To be updated
to OC48 Jan02
evlbi011.drw
20 Dec 01
Figure 1: e-VLBI Path - Haystack to ISI-E
Details of path from ISI-E to GSFC Antenna – work in progress
ISI-E (Tom)
Scheduling on
Bossnet calendar Plan to update to
OC-48 ~Jan 02 Plan mid Jan 02
GigE
Trans- GigE G4
G2
ponder
G1
Test
WS?
OC-48
Juniper
M160
GigE
H9
GSFC/Bldg 28 (Pat)
GigE
K2
Will try to schedule
either K2 or K3
exclusively for
e-VLBI experiment
K1
K3
Mac with
Yellowdog Linux;
need details
J2
Summit
5i
H7
Borrowed OC48
interface - may not
be able to keep
J3
Cisco
3508
Switch
J4
GigE
H3
J5
WDM
GigE
J6
LuxN
GigE
L1
GGAO Antenna Trailer (Bill)
On order
Summit
5i
K6
1600 ft of fiber
to be installed
Jan 02
MAC on order
due Jan-Feb 02
GigE
WS/
L3
Mark 5
L2
K5
Test
WS
OC-48
Juniper
M160
Other users ( no control)
Dedicated - no
scheduling necessary
K7
No scheduling;
sufficient capacity
for e-VLBI
H4
H6
GGAO/Bldg 202 (Pat)
GigE
1310
OC-48
Need details
Scheduling necessary
if this path used
Preferred e-VLBI path,
but may not be in place.
Pat working with Jerry and Dan.
Other users
(no control)
K4
H2
GigE
Backup plan
G11
Juniper
M40
No jumbo frames
J1
H8
GigE
H1
H5
Test
WS?
UMCP (Jerry or Dan)
Test
WS?
H7
G12
GigE
G10
G9
Max @ ISI-E (Jerry)Need details
Switch
Need details
G5
G3
WDM
G7
G6
Summit
GigE G8
5i
Tuner
Bossnet WDM
Max @ ISI-E (Jerry)
Dell Linux currently
at Haystack will
also be available here
L4
Eventually replaced
with Mark 5 unit
evlbi011.drw
Figure 2: e-VLBI Path - ISI-E to GSFC/GGAO
20 Dec 01
Performance test results – Haystack/GGAO
Average sustained rate >900 Mbps over 10 hours
Westford-GGAO e-VLBI results
•
First near-real-time e-VLBI experiment conducted on 6 Oct 02
–
–
–
–
•
Recorded data at 1152 Mbps on Westford-GGAO baseline
GGAO disk-to-disk transfer at average 788 Mbps transfer rate
Immediate correlation on Haystack Mark 4 correlator
Nominal fringes observed
Direct data transfer experiment conducted on 24 Oct 02
– Direct transfer of GGAO data to disk at Haystack at 256 Mbps
– Immediate correlation with Westford data
– Nominal fringes observed
•
Next step – full real-time e-vlbi
– Mark 5 system is capable of transmitting in real-time
– But, still need additional work on correlator software to synchronize
correlator operation to real-time
– Hope to conduct first experiment in early 2003
•
Conclusion
– e-VLBI at near Gbps speeds over ordinary shared networks is possible
but still difficult
Preliminary international e-VLBI experiments
• Westford, MA to Kashima, Japan - experiments in Oct 02 and Mar 03
– Files exchanged over Abilene/GEMnet networks
• Nominal speed expected to be ~20 Mbps, but achieved <2 Mbps for unknown
reasons - investigating
– Correlation on Mark 4 correlator at Haystack and PC Software correlator
at Kashima; nominal fringes obtained
– Further experiments are scheduled; network tuning is in progress
• Kauai, Hawaii to Wettzell, Germany (in progress)
– Daily experiments of ~100GB are ideal candidate for early e-VLBI
– Data will be transferred to Haystack Observatory for processing
(OC-3 speeds are possible)
– Network links are now being brought up
New IP Protocols for e-VLBI (Phase 3)
•
•
Based on observed usage statistics of networks such as Abilene, it is clear
there is much unused capacity
New protocols are being developed to utilize networks in ‘background’ mode
for applications such as e-VLBI
–
–
–
–
•
Take advantage of special characteristics of e-VLBI data
Will ‘scavenge’ and use ‘secondary’ bandwidth
Will give priority to ‘normal’ users
Requires a new ‘end-point adaptive strategy’
Work being carried out by MIT Haystack Observatory in collaboration with
MIT Laboratory for Computer Science and MIT Lincoln Laboratory
– 3-year program; will demonstrate e-VLBI connections both nationally and
internationally
– Network researcher Dr. David Lapsley hired to lead this effort
Typical bit-rate statistics on Abilene network
1.0
Usage >20Mbps less than 1% of the time
0.1
0.01
0.001
100
500
Mbps
Conclusion: Average network usage is only a few % of capacity
Typical distribution of heavy traffic on Abilene
1.0
0.9
0.8
0.7
<10% of ‘bulk’ transfers exceed ~100 secs
200
400
1000
secs
Conclusion: Heavy usage of network tends to occur in bursts of <2 minutes
Further extensions to national and global community
(Phase 4)
•
Possibilities for international connections
–
–
–
–
–
–
Surfnet – U.S. to Europe at 2.5 Gbps
TransPAC – U.S. to Japan at 655 Mbps (upgrade to 1.2 Gbps planned)
GEMnet – currently ~20 Mbps, planning to upgrade to 2.2 Gbps
APAN – U.S. to Japan at 655 Mbps
AMPATH – possible connections to telescopes in Chile and Brazil
A sample of others under construction
•
•
•
•
TEIN – Paris to Seoul
EUMEDCONNECT – Europe to Mediterranean
NeDAP – Europe to Russia
ALIS – Europe to Latin and South America
– One of the most interesting!
• IEEAF – new Europe/U.S. link at 10 Gbps!
• Donated by Tyco, Inc.
– Dedicated-lambda networks are being investigated as possibility
622 Mbps +10 Gbps 
Transoceanic donations to IEEAF (in red)
Northern Europe donations to IEEAF (in red)
TransPAC Network
AMPATH: Research and Education Network and
International Exchange Point for the Americas




Launched in March 2000 as a
project led by Florida International
University (FIU), with industry
support from Global Crossing (GX),
Cisco Systems, Lucent
Technologies, Juniper Networks
and Terremark Worldwide
Enables wide-bandwidth digital
communications between the
Abilene network and 10 National
Research and Education Networks
(NRNs) in South and Central
America, the Caribbean and Mexico
Provides connectivity to US
research programs in the region
AMPATH is a project of FIU and the
National Science Foundation’s
Advanced Networking Infrastructure
& Research (ANIR) Division
Note: VLBI telescopes currently in Chile
and Brazil
Summary - Impact of e-VLBI Program
• e-VLBI opens new doors for astronomical and geophysical research.
• e-VLBLI represents an excellent match between modern Information
Technology and a real science need.
• e-VLBI motivates the development of a new shared-network protocol
that will benefit other similar applications.
• e-VLBI drives an innovative IT research application and fosters a
strong international science collaboration.