Transcript Wiener Crates * A Power Supply Analysis

Wiener crates; An installation
and power supply analysis.
Magnus Bjork Be/Co/Fe - 20.06.2012
Agenda
•
•
•
•
•
•
•
•
•
How are most crates installed?
How many failures? / What goes wrong?
How should the crates be installed?
Redundant crates.
The “quick-and-dirty fix”.
Conclusions and suggestions.
Questions?
References & Circuit breaker curves.
Wiener’s report.
2
How are most crates installed?
• Incorrectly, due to various constraints.
– Many crates are connected to power bars in the
racks, this can bring several problems!
– “Swiss” plugs are rated for 10 (13) Amperes, the
crate is specified for maximum 16 Amperes.
– At CERN “Swiss” outlets must be protected by 30
mA ground fault breakers. = Not a good place to
connect high-powered industrial crates.
– If any equipment on the power bar has a short
circuit, all connected equipment loses power.
3
A maximum 16A crate (or several crates) connected
to a “power bar” with a single 10A breaker…
cPCI +-12V 20A
VME +-12V 10A
4
Three 16A crates in this one 10A “bar”…
5
Why is this working?
• The 16A maximum is the absolute maximum for
the crate. The actual power supply need
depends on the real power consumption.
• To get an estimate of the actual crate power
used at this instant, add all the voltages x
current values read (from either the front
panel or) remotely via e.g. Diamon, Web, etc.
divide the sum by 0,75 (power factor for heat
losses in the power supply) and add ~30
Watts. (3x8W crate ventilators +Fan tray logic.)
6
Some mathematics to see clearer.
• VME Maximum Power
• cPCI Maximum Power
Voltage
Current
Power
Voltage
Current
Power
5
100
500
5
100
500
12
10
120
12
20
240
3,3
100
330
3,3
100
330
-12
10
120
-12
20
240
• 1’070W / 0,75 = 1’426W • 1’310W / 0,75 = 1’746W
+ ~30W = 1’456W
+ ~30W = 1’776W
• ~1’500W / 230V = 6,5A • ~1’800W / 230V = 7,8A
7
How many failures?
• Out of ~530 operational units the total
number of PFC failures per year were;
– 2009; 20 (~3,77%)
(Protection diode modification done Jul-Oct 2009.) {Ref. 2}
-2010; 7 (~1,32%)
-2011; 11 (~2,07%)
-2012, Jan -> June; 10 (~1,88%)
(Total for ALL Wiener equipment; 0,96% failures in 2011.
As a comparison, with CAEN the number is 3,38%)
8
What goes wrong?
• Quite simply; The PFC.
– The PFC seems to have some weak points…
– Wieslaw Iwanski, PH/ESE-BE, did some very extensive
tests. {Ref. 1}
– The power supplies now have 5 - 6 years of use, normal
end-of-life for certain components may be a natural
consequence of aging, but for the moment no “usual
increase in failures” can confirm e-o-l approaching.
– MTBF for the ventilators is specified to be 6 years, the
power supplies are said to “last longer”.
– It should be noted that CERN specifically requested the
Active PFC and the Softstart functionality.
– To protect the PFC the diode upgrade was introduced.
9
This diode
10
D20
11
D14
12
How should the crates be installed?
• According to the OEM specifications.
– Individual 16 Ampere type “B” or “C” curve
circuit breakers.
– Server grade power connections, preferably
approved up to 16 Amperes and not needing any
30 mA ground fault circuit breakers.
– Standard Burndy 10 Amperes max connector pins
are acceptable, since the real maximum input
current of our power supplies is either;
6,5 Amperes (VME) or 7,8 Amperes (cPCI).
– In case of a short-circuit the 10 Ampere Burndy
connector pins can handle the short-term current
of over 16 Amperes, then the circuit breaker reacts.
13
A good power distribution 1 {Ref.3}
14
A good power distribution 2 {Ref.3}
15
16
Redundant crates
• Redundant crates should ideally be fed from
two separate “Canalis”power distribution
rails, both “Canalis” rails should be fed from
UPS power supplies.
• The two power supplies of a redundant crate
should be fed by two different phases from
the two different “Canalis”. (R, S or T)
• The redundant crates should have automatic
monitoring of the power supplies output, to
be able to alarm and / or trigger automatic
events in case of failure of one power supply.
17
The “quick-and-dirty fix”…
• Installing a small fuse in-line with the power
feed to the crate power supply may mask a
part of the problem but may cause other
problems later. …But it has two “advantages”;
it’s fast & cheap. For now.
• The crates have been working in the current
setup for ~5 years and they still work fine.
• Revising the power feed can be costly and
inconvenient, but for critical racks with
Wiener crates installed it is a good idea,
especially when considering the cost of
unscheduled LHC downtime…
18
Conclusions and suggestions 1
• For critical rack(s) with Wiener crate(s) a new
power distribution should be done with an
individual breaker box directly to the
“Canalis”. Possibly without additional
connectors or with server-grade hardware.
• The intensity and selectivity of the individual
circuit breaker; 16 Ampere, “C” curve.
• A switch-on delay unit is not needed unless
there has been previous problems powering
up the whole group of high-powered racks.
• The main circuit breakers should be reviewed
to guarantee the selectivity.
19
Conclusions and suggestions 2
• The protection diode should be installed in
the power supplies, if not already done.
• Future power supply failures can not be
excluded but secondary consequences can be
greatly reduced by simply following the
specifications by Wiener and ensuring a
proper electrical distribution.
• The (Active) PFC was requested by CERN
since without a power factor correction the
3:rd and 9:th order harmonics might cause
problems, especially if the neutral cable is not
dimensioned 1-1 with the phase conductors.
20
Conclusions and suggestions 3
• The softstart was also requested by CERN
and is a good (and bad) reason why the
installations have survived so far.
• The new ELMA crates can be seen as a
solution to upgrade to new hardware. …But
first we must install some crates in 24/7
operational use in some easy to access area,
in order to validate them “in the real world”.
• The ELMA CPA500 power supplies also have
a PFC, “Low inrush current” and softstart.
21
Questions?
22
References
• {Ref. 1} Wieslaw Iwanski, PH/ESE-BE
– WienerTests_v40
• {Ref. 2} Paul Harwood
– Wiener Diode modification.pdf
• {Ref. 3} LHCb
23
24
25