Transcript Some Sample Commercial Network Costs

e-VLBI: A Brief Overview
Alan R. Whitney
MIT Haystack Observatory
Traditional VLBI
The Very-Long Baseline
Interferometry (VLBI)
Technique
(with traditional data recording)
The Global VLBI Array
(up to ~20 stations can be used
simultaneously)
VLBI Science
Quasars, hotspots, polarization
ASTRONOMY
• Highest resolution technique available to
astronomers – tens of microarcseconds
• Allows detailed studies of the most distant
objects
Chautauqua 2001
Plate-tectonic motions from VLBI measurements
GEODESY
• Highest precision (few mm) technique
available for global tectonic measurements
• Highest spatial and time resolution of
Earth’s motion in space for the study of
Earth’s interior
•Earth-rotation measurements important
for military/civilian navigation
•Fundamental calibration for GPS constellation
within Celestial Ref Frame
VLBI astronomy example
Scientific Advantages of e-VLBI
• Bandwidth growth potential for higher sensitivity
– VLBI sensitivity (SNR) proportional to square root of Bandwidth
resulting in a large increase in number of observable objects
(only alternative is bigger antennas – hugely expensive)
– e-VLBI bandwidth potential growth far exceeds recording capability
(practical recordable data rate limited to ~1 Gbps)
• Rapid processing turnaround
– Astronomy
• Ability to study transient phenomena with feedback to steer observations
– Geodesy
• Higher-precision measurements for geophysical investigations
• Better Earth-orientation predictions, particularly UT1, important for
military and civilian navigation
Practical Advantages of ‘e-VLBI’
• Increased Reliability
– remove recording equipment out of field
– remote performance monitor & control capability in near real-time
• Lower Cost
– Automated Operation Possible
• eliminates manual handling and shipping of storage media
– Near Real-time Processing
• forestalls growth of storage-capacity requirements with bandwidth growth
– Elimination of recording-media pool (millions of $’s!)
• Avoid unexpected media-shipping interruptions and losses
Elements of e-VLBI Development
• Phase 1: Develop eVLBI-compatible data system
– Mark 5 system development at MIT Haystack Observatory being supported by
NRAO, NASA, USNO plus four international partners
– Prototypes now deployed in U.S. and Europe
• Phase 2: Demonstrate 1 Gbps e-VLBI using Bossnet
(w/ DARPA and NASA support)
– ~700km link between Haystack Observatory and NASA/GSFC
– First e-VLBI experiment achieved ~788Mbps transfer rate
• Phase 3: Establish adaptive network protocol
(newly awarded NSF grant to Haystack Observatory; collaboration with MIT Lab for
Computer Science and MIT Lincoln Laboratory);
– New IP-based protocol tailored to operate in shared-network ‘background’ to
efficiently using available bandwidth
– Connect with telescopes worldwide (U.S., Europe, Japan)
Mark 5 VLBI Disk-Based Data System (Phase 1)
• 1 Gbps continuous recording/playback to/from set of 8 inexpensive (ATA) disks
• Developed at MIT Haystack Observatory with multi-institutional support
• Mostly COTS components
• Two removable ‘8-pack’ disk modules in single 5U chassis
• With currently available 200GB disks – capacity of single ‘8-pack’ 1.6TB;
expected to increase to 2.5TB by early 2003 at cost of ~$1/GB
• GigE connection for real-time and quasi-real-time e-VLBI operations
• Inexpensive: <$20K
• ~20 Mark 5 systems now installed at stations and correlators
Bossnet 1 Gbps e-VLBI
demonstration experiment
(Phase 2)
Westford
Haystack
(correlator)
Future
Initial experiment
USNO
(correlator)
NASA/GSFC
Details of path from Haystack to ISI-E – work in progress!
Westford Antenna
Millstone RPE Bldg (Lorraine)
Dedicated no scheduling
GigE 1 Summit
WS/
Mark 5
A2
A1
GigE
8
1i
Summit
A4
5i
A3
C2
Haystack Correlator
GigE
Dedicated no scheduling
WS/ 28-1A
Mark 5 B2
14
B1
13
B6
B5
GlowNet
WDM
Amp
D3
D2
Lincoln Lab (Lorraine/Steve)
Schedule to be
coordinated with LL
To be replaced by
Juniper M20 Jan 02 with
Cisco
OC-48 to Bossnet
E3
To be provided
by Haystack
E5
Test
WS
GlowNet
D1
Bossnet(Steve/Jim)
ISI-E
E7
GigE
E4
To be upgraded from
2 to 3
Scheduling on
Bossnet calendar
GigE Summit GigE Summit GigE
1310
5i
E2
5i
E6
E1
Cisco
1500
GigE
Congress St
Boston
B3
1500
(Shared with LL on
scheduled basis)
WDM
GigE
LL
Plan install by 1 Jan 02
C3
16
GigE Summit
5i
GigE
15
WS/ 28-1B
Mark 5 B4
Schedule to be
coordinated with LL
C1
MIT
Campus
WDM
F1
1510
Amp
Amp
F2
E8
Other users
(no control)
To be updated
to OC48 Jan02
evlbi011.drw
20 Dec 01
Figure 1: e-VLBI Path - Haystack to ISI-E
Details of path from ISI-E to GSFC Antenna – work in progress
ISI-E (Tom)
Scheduling on
Bossnet calendar Plan to update to
OC-48 ~Jan 02 Plan mid Jan 02
GigE
Trans- GigE G4
G2
ponder
G1
Test
WS?
OC-48
Juniper
M160
GigE
H9
GSFC/Bldg 28 (Pat)
GigE
K2
Will try to schedule
either K2 or K3
exclusively for
e-VLBI experiment
K1
K3
Mac with
Yellowdog Linux;
need details
J2
Summit
5i
H7
Borrowed OC48
interface - may not
be able to keep
J3
Cisco
3508
Switch
J4
GigE
H3
J5
WDM
GigE
J6
LuxN
GigE
L1
GGAO Antenna Trailer (Bill)
On order
Summit
5i
K6
1600 ft of fiber
to be installed
Jan 02
MAC on order
due Jan-Feb 02
GigE
WS/
L3
Mark 5
L2
K5
Test
WS
OC-48
Juniper
M160
Other users ( no control)
Dedicated - no
scheduling necessary
K7
No scheduling;
sufficient capacity
for e-VLBI
H4
H6
GGAO/Bldg 202 (Pat)
GigE
1310
OC-48
Need details
Scheduling necessary
if this path used
Preferred e-VLBI path,
but may not be in place.
Pat working with Jerry and Dan.
Other users
(no control)
K4
H2
GigE
Backup plan
G11
Juniper
M40
No jumbo frames
J1
H8
GigE
H1
H5
Test
WS?
UMCP (Jerry or Dan)
Test
WS?
H7
G12
GigE
G10
G9
Max @ ISI-E (Jerry)Need details
Switch
Need details
G5
G3
WDM
G7
G6
Summit
GigE G8
5i
Tuner
Bossnet WDM
Max @ ISI-E (Jerry)
Dell Linux currently
at Haystack will
also be available here
L4
Eventually replaced
with Mark 5 unit
evlbi011.drw
Figure 2: e-VLBI Path - ISI-E to GSFC/GGAO
20 Dec 01
Performance test results – Haystack/GGAO
Average sustained rate >900 Mbps over 10 hours
Westford-GGAO e-VLBI results
•
First near-real-time e-VLBI experiment conducted on 6 Oct 02
–
–
–
–
•
Recorded data at 1152 Mbps on Westford-GGAO baseline
GGAO disk-to-disk transfer at average 788 Mbps transfer rate
Immediate correlation on Haystack Mark 4 correlator
Nominal fringes observed
Direct data transfer experiment conducted on 24 Oct 02
– Direct transfer of GGAO data to disk at Haystack at 256 Mbps
– Immediate correlation with Westford data
– Nominal fringes observed
•
Next step – full real-time e-vlbi
– Mark 5 system is capable of transmitting in real-time
– But, still need additional work on correlator software to synchronize
correlator operation to real-time
– Hope to conduct first experiment in early 2003
•
Conclusion
– e-VLBI at near Gbps speeds over ordinary shared networks is possible
but still difficult
Westford-to-Kashima e-VLBI experiment
• Westford/Kashima experiment conducted on 15 Oct 02
– Data recorded on K5 at Kashima and Mark 5 at Westford at 256 Mbps
– Files exchanged over Abilene/GEMnet networks
• Nominal speed expected to be ~20 Mbps, but achieved <2 Mbps for unknown
reasons - investigating
– File formats software translated
– Correlation on Mark 4 correlator at Haystack and PC Software correlator
at Kashima
– Nominal fringes obtained
– Further experiments are anticipated
Plans for UT1 Intensive e-VLBI
• Daily ~1 hour VLBI sessions between Kokee Park, Hawaii and Wettzell,
Germany are used for UT1 measurements
• Data are time sensitive since they are used for predicting UT1
• Currently requires ~4 day turnaround shipping media
• These measurements are an ideal candidate for routine e-VLBI
– Short daily session collect <100 GB of data
– Even 100 Mbps will allow transfer in a few hours
• Work now in progress to make necessary connections
– Network being organized from Kokee Park to USNO;
connection speed OC-3
– Data from Mark 5 system in Wettzell will be carried to Univ. of
Regensberg, about 1 hour drive; connection speed OC-3
– Negotiations ongoing for extension of MAX network to USNO with GigE
connection
New IP Protocols for e-VLBI (Phase 3)
• Based on observed usage statistics of networks such as Abilene, it is
clear there is much unused capacity
• New protocols are being developed to utilize networks in ‘background’
mode for applications such as e-VLBI
–
–
–
–
Take advantage of special characteristics of e-VLBI data
Will ‘scavenge’ and use ‘secondary’ bandwidth
Will give priority to ‘normal’ users
Requires a new ‘end-point adaptive strategy’
• Work being carried out by MIT Haystack Observatory in collaboration
with MIT Laboratory for Computer Science and MIT Lincoln
Laboratory
– 3-year program; will demonstrate e-VLBI connections both nationally and
internationally
Typical bit-rate statistics on Abilene network
1.0
Usage >20Mbps less than 1% of the time
0.1
0.01
0.001
100
500
Mbps
Conclusion: Average network usage is only a few % of capacity
Typical distribution of heavy traffic on Abilene
1.0
0.9
0.8
0.7
<10% of ‘bulk’ transfers exceed ~100 secs
200
400
1000
secs
Conclusion: Heavy usage of network tends to occur in bursts of <2 minutes
Impact of e-VLBI Program
• Opens new doors for astronomical and geophysical research.
• Represents an excellent match between modern Information
Technology and a real science need.
• Motivates the development of a new shared-network protocol that will
benefit other similar applications.
• Drives an innovative IT research application and fosters a strong
international science collaboration.