Transcript Slides - Agenda INFN

A Sub-sea Node Network for KM3NeT
Advantages of this node network are:
 That the transparent 10 Gb/s bandwidth point to point multiple node network is built with
components of the shelf technology, a standard according to ITU specifications.
 Photonic technology enables a maximum migration of sub-sea electronics to the shore.

-
Transparent electronic circuitries in the sub-sea detector area.
Enables sending raw data from the PMT circuitry to shore, ToT signals or even multilevel ToT signals
Less production time and test time for the sub-sea equipment.
Onshore flexibility in developing DAQ hardware/software. Future updates possibilities are granted.
 The PON (Passive Optical Network) has inherently a fixed signal propagation.
This feature makes sending/receiving time critical signals to/from the detector transparent.
 Low power requirements on the seabed
 The network is almost cost neutral compared to conventional electronic/optic solutions.
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
1 electronic department
History of development
 Innovative design study using photonic technology for data transport started
after the VLVnT workshop in Amsterdam, 2003
 Various studies on conceptual aspects of a novel architecture by Nikhef, 2003-2005
 Photonic workshop at Nikhef within the framework of WP4, November 2006
 First basic experiments at the University of Eindhoven, summer 2007
Feasibility study results presented by CIP at the KM3NeT CDR workshop, November 2007
 Realization with industry (CIP) of ‘SPARK’ with 3 optical channels 10 Gb/s, August 2008
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
2
electronic department
Architecture of KM3NeT Node Network
Dense Wavelength Division Multiplexing
- Array of centralized continuous wave laser sources on-shore,
shared by all Optical Modules
Optical power splitting and amplification
- ~ 100 wavelengths per fiber
- ~ 100 fibers
- for 10.000 OM’s
A single fiber working between Optical Module and Junction Box
- Separate go/return fibers to shore due to Coherent Rayleigh backscatter
- Accurate timing over fiber is proved
PMT data taking/multiplexing
- Initially electric, but on longer term “all Optical” frontend solutions feasible.
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
3 electronic department
Definition of an optical channel
An optical channel is
DWDM
ln
DWDM
l1
l1
ln
•A per wavelength transparent point to point connection over fiber
•1 fiber can carry more than 100 wavelengths
•An individual optical channel can have a bandwidth from
DC to over 40 Gb/s (we use an ITU standard of 10Gb/s)
•An optical channel can be applied bidirectional over the same fiber
DWDM (Dense Wavelength Division Multiplexing)
AWG (Arrayed Wave Guide)
Similar components
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
electronic department
Use of an optical channel
An optical channel is also bidirectional
(And has a rigid propagation time !!)
l1
DWDM
ln CW laser
CW ln modulated ln
Receiver
(PIN)
DWDM
l1
REAM
Serial data to be transported
Serial data received
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
ln
electronic department
Arrayed Waveguide Grating
An AWG can be used as a wavelength router…
PIN
CW l1 l2 l3 l4 
CW l1 l2 l3 l4 
gate
l1 l2 l3 l4 
AWG
l1
l2 l1
clock/data on
l1
clock/data
PIN
REAM
l3 l2
OM
data
l4 l3
To OMs
not used
lN+1 lN
CIP Confidential
clock/data
6
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
electronic department
Schematic View of the Node Network
DWDM Mux
Presentation on CDR workshop November 2007
100km fibre path
l1
Single fibre feed shared
for feed wavelength comb
cw DWDM lasers
(up to 100 wavelengths)
Comms & Timing
1 of 100
return fibres
Power splitters to feed up to 100 units
Up to 100
reflective
modulators
Data out
2km
Data Receiver
WDM Demux
Optical
Amplifiers
PMTs
DWDM
Demux
Shore Station
Proposed Architecture for Km3NeT
to Avoid Rayleigh Back Scatter
Limitation
Electrical drive
to modulator.
(single modulator gates all DWDM
Wavelengths)
l1
OMs
Optical
receiver
Undersea Station
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
7 electronic department
Transmission Timing Skew
 Wavelength dependent timing skew due to group delay over 100km (LEAF)
• ~90ns for 25GHz comb (1530nm – 1550nm)
• ~140ns for 40GHz comb (1530nm – 1562nm)
• This is deterministic, at fixed temperature
 Temperature dependence
• estimates based on published data…
• Bulk: 96ps/oC per km  9.6ns/oC (100km) (LEAF)
• Skew: lo ~ 0.03nm/oC  8.6ps/oC (100km) (standard fibre 40GHz comb)
• Shows that relative timing information will stay constant, absolute timing
varies insignificant with temperature.
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
8 electronic department
Clock and Timing Calibration
 Shore based master clock / framing generator
 Distribute clock sync and framing to each Optical Module by over-modulating
cyclic copy of DWDM seed using distribution property of dual input AWG
 each OM receives two wavelengths, one cw for return data modulation,
one conveying clock and data (not returned to shore)
 Use pulse echo measurements to calculate relative delay of each OM
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
9 electronic department
Example of Signal Propagation
100km
A
2km
cw seed
Y
AWG
Loop
timing
pulse
OM
Gated
amplifier
100km
M
100 return
fibres
A’
X
Junction
Box
Shore
Station
TX-Y = TY-X
TA’-OM-A’
For illustration (or measured during construction)
Pulse echo A’ to all OMs and M
TOM-A’ = (TA’-OM-A’)/2
TA-OM-A’
clock / framing round trip measured
 TA-OM = TA-OM-A’ - TOM-A’
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
10 electronic department
Optical
Module
SPARK
Sophisticated Photonic Architecture Reference for KM3NeT
Joint activity of Nikhef and CIP
Purchased by Nikhef
Required:
A reference photonic test bench based on a KM3NeT architecture for a node network
with10Gb/s bandwidth/OC, with an intrinsic signal propagation time jitter of << 50 ps.
What we have:
A reference photonic test bench based on a KM3NeT architecture for a node network
with10Gb/s bandwidth/OC, with an intrinsic signal propagation ime jitter of << 50 ps.
The door is wide open for:
• Optimizing PON components for a final node network to be deployed.
• Transparently connects front-end (sub-sea) electronics to backend (on-shore) electronics
with signal propagation time only.
• The entire test bed is relatively easy to copy: all parts are COTS (Components Of The Shelf)
• A low cost test bed with reduced components for a single optical channel can be very
helpful for developing front-end and back-end electronics in different laboratories.
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
11 electronic department
Outline of SPARK
cw DWDM lasers
(2 wavelengths)
Mux
Tuneable
laser input
Temperature
control & Laser
Bias
10G Data
Receivers
Spare for
loop timing
Temperature
control
10G
Drivers
100km LEAF
SOA Line Amplifier
If 100 km and10Gb/s
DWDM
Reflective
modulators
2km
DWDM
Temperature
control
Pre Amplifier
not fitted
Shore Station
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
Undersea Station
12 electronic department
SPARK in reality
10 km Leaf fiber
Shore station
Sub-sea station
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
13 electronic department
Bi-directional DWDM for 20 channels
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
14 electronic department
The REAM in the off-shore application
From front-end electronics to fiber takes 2 components in OM
REAM
driver
Clock input
Reflective
Electro
Absorption
Modulator
Data input
(next stage the REAM driver will reside in the REAM housing)
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
15 electronic department
SPARK Shore Station
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
16 electronic department
SPARK First Measurements
Laser & AWG Spectrum
Channel 17 & 18 Laser and AWG spectrum
10
0
Level (dB)
-10
-20
Laser_A-409
Laser_B-306
-30
AWG_Ch18
AWG_Ch17
-40
-50
-60
-70
1558.3 1558.7 1559.1 1559.5 1559.9 1560.3 1560.7 1561.1 1561.5 1561.9
Wavelength (nm)
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
17 electronic department
10Gb/s eye pattern at CIP
10 Gb/s eye pattern
BER no errors
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
18 electronic department
Other DAQ News from Nikhef
Transfer exact timing by using a coded data
communication channel
Xilinx Virtex-5
ML507 Evaluation Kit
Start
Stop
Tx
Rx
8B/10B
Encoded
8B/10B
Encoded
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Peter Jansweijer et.al.
19
electronic department
A KM3NeT timing proposal
l1
Broadcast
l2
l3
l27
JB
OM
One
Reference Clock
(GPS)
PMTs
l1 + l27
Local Clocks are phase locked;
thus isochronous.
But their values have an offset
Loop
timing
OM
l1
Data
Receiver
l2 + l1
l3 + l2
l4 + l5
l27 + l26
Mod
PMTs
Individual Optical Channel
Shore Station
DU
Undersea Station
CIP Confidential
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Peter Jansweijer et.al.
20
electronic department
Timing over 8B10B data channel
Tx
K28.5
IDLE
8B/10B
Encoded
D16.2
K23.7 K23.7
CharExt
K28.5
IDLE
D16.2
K28.5
IDLE
D16.2
K28.5
IDLE
D16.2
K28.5
IDLE
D16.2
K28.5
IDLE
D16.2
K28.5
IDLE
D16.2
K28.5
IDLE
D16.2
Start
Stop
Rx
K28.5
IDLE
8B/10B
Encoded
D16.2
K28.5
IDLE
D16.2
K28.5
IDLE
D16.2
K28.5
IDLE
D16.2
K23.7 K23.7
CharExt
Variable propagation delay
0111010110000010101101110101100000101011011101011000001
0111010110000010101101110101100000101011011101011000001
+ 320 ps
RxRecClk
Reset
1919
0
BitSlide(4:0)
6.4 ns
(= resynchronize
&
Byte Re-Align)
0
1
0000 = 19
0011
0
At 3.125 Gbps:
Absolute timing is determined by “Start/Stop” delay
(in 20 bit steps of 6.4 ns) plus
BitSlide fine delay (20 steps of 320 ps)
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Peter Jansweijer et.al.
21
electronic department
For the skeptic…
Does this work from board to board?
Coarse time
(6.4 ns Steps)
3.125 Gbps via
Constant Impedance
Trombone Line
Yes it
does!
Transmitter
Lattice LFSCM25
Receiver
Xilinx Virtex-5
Receiver locked to
Transmitter
Oscillator
Transmitter
Crystal
Oscillator
Fine time
(320 ps Steps)
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Peter Jansweijer et.al.
22
electronic department
To be continued
15/16 -10-08 Paris, KM3NeT meeting WP3, WP4,WP5 Jelle Hogenbirk et.al.
23 electronic department