Transcript LAr-olink

Experience with
the ATLAS LAr optical link
1. The ATLAS LAr optical link system.
2. Experiences:




in the R&D phase.
In the integration phase.
In the production and QA phase.
In early operation phase.
3. The OTx problem and a possible fix as of now.
4. What I would do for future optical links.
Jingbo Ye
Department of Physics
SMU
Dallas, Texas
Disclaimer: this talk only reflects my own experience and understanding of the
optical link, not the official opinion of ATLAS LAr.
1
The ATLAS LAr Optical Link. 1

The G-Link based optical link for ATLAS LAr:
32 LVDS
ADC data
16 TTL
SMUX
40 MHz
delay
80 MHz
GLink TX
1.6 Gbps
PECL
1. LAr o-link.
2. Lessons to me.
3. The Dual OTx.
4. Thoughts.
50/125 μm
OTx
GRIN fiber
40 MHz
QPLL
FE -- FEB
TTCrx
A total of 1524 channels installed.
SMUX: an ASIC based on DMILL, level translating, 2:1 multiplexing.
GLink: COTS, operates outside of spec, specially ordered from Agilent. Water cooled on both Feb and ROD.
OTx, ORx: VCSEL/PIN based assembly produced by IPAS in Taiwan, with COTS driver and amplifiers.
All FE components were qualified for 360 Krad.
A collaboration of 5 institutions:
CPPM, IPAS, ISN, KTH and SMU
The project lasted for 6+ years:
1998 - 2004
ORx
FPGA
BE -- ROD
8 channels
ORx
J. Ye, SMU Physics
GLink RX
GLink RX
Lessons from ATLAS LAr Optical Link, CERN, March 2010
FPGA
2
The ATLAS LAr Optical Link. 2

The link TX on the FEB
OTx
Glink TX
SMUX
1. LAr o-link.
2. Lessons to me.
3. The Dual OTx.
4. Thoughts.
• The OTx is a custom assembly
with VCSEL in an ST
connector.
• There were quite a few issues
of this assembly in production
in Taiwan and QA at SMU.
• We are now having problems
with the VCSELs at a few
percent level.
• Overall this single fiber per
FEB design does not answer
the reliability requirements
from physics (physicist),
especially in an environment
where no access is possible
for maintenance.
~ 400 mm
J. Ye, SMU Physics
Lessons from ATLAS LAr Optical Link, CERN, March 2010
3
The ATLAS LAr Optical Link. 3

1. LAr o-link.
2. Lessons to me.
3. The Dual OTx.
4. Thoughts.
The link RX on the ROD
8 channel optical link integrated
on one ROD motherboard.
Challenges:
Heat dissipation; Noise, cross
talks; ORx sensitivity.
Link optical power budget
OTx power
–4 to –7 dBm
Required at
ORx –17 dBm
~ 400 mm
J. Ye, SMU Physics
Lessons from ATLAS LAr Optical Link, CERN, March 2010
4
Lessons That I Learned

In the R&D phase.





Giga bit per second serial data transmission is non-trivial,
especially in radiation environment, where parts are not
readily available from Digikey.com.
Fiber optics and the electrical/optical interface have a lot of
issues that need special expertise.
The above two require experts and special equipment. The
price is very high if one only has people “train-on-the-job”; and
much higher if one loses those “trained-on-the-job”.
A system level design will certainly be much better than
“LEGO building”. But this requires people with expertise of
such a system to generate a system specification, in
particular:
 Clock jitter requirement.
 Optical power budget, TX power and RX sensitivity.
 Reliability requirement and the information from
components.
1. LAr o-link.
2. Lessons to me.
3. The Dual OTx.
4. Thoughts.
R&D should end with a production ready design and
prototype. Extending R&D into production will cause
problems in system reliability.
J. Ye, SMU Physics
Lessons from ATLAS LAr Optical Link, CERN, March 2010
5
Lessons That I Learned

In the integration phase.



J. Ye, SMU Physics
1. LAr o-link.
2. Lessons to me.
3. The Dual OTx.
4. Thoughts.
Integration of the link with up- and
down-stream electronics needs to be
investigated before hand. Debugging
may be very difficult if that has not
been thought about before the
integration.
A clearly defined and communicated
system specification will help the
integration.
A carefully designed mezzanine
board may actually help in integration
and operation.
Lessons from ATLAS LAr Optical Link, CERN, March 2010
6
Lessons That I Learned

In the production and QA phase.
 The production volume is very small (about 2000
units for each part). We spent a lot of time to learn
the process, such as “burn-in” of the OTx.
 We learned the hard way that optical coupling and
connector assembly are not a job for amateurs.
We should now stay away from connectors such
as ST and SC, especially those connectors are put
on manually.
 We “invented” our QA process that seemed to be
okay, except that



1. LAr o-link.
2. Lessons to me.
3. The Dual OTx.
4. Thoughts.
We were doing reliability measurements during and
after the production.
In the end, we found out that at least one serious
issue with the OTx escaped our attention.
Overall QA and system reliability are big issues we
often do not actually know much about. A way out:
build in redundancy.
J. Ye, SMU Physics
Lessons from ATLAS LAr Optical Link, CERN, March 2010
7
Lessons That I Learned

In early operation phase.




Do not open your Champagne too soon.
The problem with the OTx: the VCSELs from TrueLight
fail at a few percent level in the past ~1 year of
operation.
We tried many things (humidity, magnetic field, ESD,
EOS, defects in VCSEL fabrication) to understand the
cause of that. This exercise is beyond our knowledge,
equipment, and often very expensive. As of now, we still
do not have a definite answer to the cause(s) of our
problem, although we excluded several things and
narrow down to ESD, EOS and fab. defects: Please see
Stefan Simion’s talk.
Lesson: HiRel is not easy. Unless you are very
confident about all the parts, a design with redundancy
may be a way around.
J. Ye, SMU Physics
Lessons from ATLAS LAr Optical Link, CERN, March 2010
1. LAr o-link.
2. Lessons to me.
3. The Dual OTx.
4. Thoughts.
8
Lessons That I Learned

Yet we at least have one variable that seems to have
strong correlation with the failure mechanism: the
VCSEL optical spectrum (OS).
 So far all failed VCSELs (after we started to measure
the OS) have narrow OS.
J.Parsons
J. Ye, SMU Physics
1. LAr o-link.
2. Lessons to me.
3. The Dual OTx.
4. Thoughts.
S.Simion
Lessons from ATLAS LAr Optical Link, CERN, March 2010
9
Lessons That I Learned

We know ESD produces narrower OS.
Forward ESD at 500V, 700V, 900V and 1100V


1. LAr o-link.
2. Lessons to me.
3. The Dual OTx.
4. Thoughts.
Reverse ESD at 200V, 300V, 400V and 500V
We have not yet understood it from physics point of view.
We are waiting for the time to tell us what to do:



Replace only OTx with narrow OS. Or
Replace all OTx with a new VCSEL, and different driving circuit.
Replace all OTx with a new design of dual optical channels for
redundancy.
J. Ye, SMU Physics
Lessons from ATLAS LAr Optical Link, CERN, March 2010
10
The Dual OTx, a Fix With All the
Constraints we Have Now.
The present OTx:
1. LAr o-link.
2. Lessons to me.
3. The Dual OTx.
4. Thoughts.
The proposed dual-VCSEL OTx:
J. Ye, SMU Physics
Lessons from ATLAS LAr Optical Link, CERN, March 2010
11
Dual OTx Test Results .1
1. LAr o-link.
2. Lessons to me.
3. The Dual OTx.
4. Thoughts.
ver1 bathtub
1.E+00
1.E-02
BER
1.E-04
1.E-06
right
left
full
1.E-08
1.E-10
1.E-12
1.E-14
-500
J. Ye, SMU Physics
-300
-100
100
delay (ps)
Lessons from ATLAS LAr Optical Link, CERN, March 2010
300
500
12
Dual OTx Test Results .2
1. LAr o-link.
2. Lessons to me.
3. The Dual OTx.
4. Thoughts.
ver2 bathtub
1.E+00
1.E-02
BER
1.E-04
1.E-06
right
left
full
c
1.E-08
1.E-10
1.E-12
1.E-14
-500
J. Ye, SMU Physics
-300
-100
100
delay (ps)
Lessons from ATLAS LAr Optical Link, CERN, March 2010
300
500
13
Some Thoughts for Future Links


Multi-Giga bit serial data transmission is
not trivial. We should not lose the
expertise we now have. The entry price is
very high.
We do have special requirements, such
as radiation tolerance, no accessibility for
maintenance, etc, that cannot be met by
a commercial product and design. But this
does not mean that we should always go
for a custom design and build. We should
only do what we can do, and what we are
good at, and “borrow” as much as
possible from industry. This is the
concept/beauty of the Versatile Link
project.
J. Ye, SMU Physics
Lessons from ATLAS LAr Optical Link, CERN, March 2010
1. LAr o-link.
2. Lessons to me.
3. The Dual OTx.
4. Thoughts.
14
Acknowledgements
1. LAr o-link.
2. Lessons to me.
3. The Dual OTx.
4. Thoughts.
Many work is actually done by Andy and Kent
at SMU. All credit should go to them. If any
statements here are wrong or offensive, I take
full responsibility.
J. Ye, SMU Physics
Lessons from ATLAS LAr Optical Link, CERN, March 2010
15