Transcript Tests status

JCOP FWWG Meeting: 24/04/2012
Industrial Controls
Engineering Department
Tests status of the
SYSTEC CAN-USBModul16 interface
remotely connected via a
DIGI ANYWHERE USB
(USB over Ethernet device)
G.Thomas & D.Davids (CERN) & O.Holme (ETH Zurich)
Industrial Controls
Engineering Department
Outline
 Summary of last tests done and fixed issues
 Additional tests and analysis done
 Current known issues/limitations
 LS1 and long term?
EN/ICE
JCOP FWWG - 2
Recovery testing and results
Industrial Controls
Engineering Department
 Test conditions: Continuous sending/receiving of burst of CAN frames on two CAN ports
at 125kbit/s -Reboot , short and longer disconnections (< ~1min)
Recovery tests
FIXED and TESTED on
Windows XP, Windows7
Results
Power cycle of SYSTEC CAN-USBmodul16
Power cycle of the DIGI interface
OK
Simultaneous power cycle of the DIG and CANUSBmodul16 (Rack level)
OK
HW reset of the DIGI (front-end button on the
device)
HW disconnection of USB cable
Reboot of the PC (where the application is
running)
Software reboot of the DIGI (via the web
interface or Config utility)
Partial Recovery
Software Ethernet disconnection (via the web
interface or Config utility)
Partial Recovery
HW disconnection of Ethernet cable (for short
and long disconnection, behavior is not the
same)
Partial Recovery
PC crash!
Network Switch failure (for short and long
disconnection , behavior is not the same)
Partial Recovery
PC crash!
(EN/ICE)
OK
(requires to disable a
watchdog timeout
SYSTEC CAN modules!!)
JCOP FWWG - 3
Additional tests and analysis
Industrial Controls
Engineering Department
 Tests with DIGI 5 & 14 ports
 Usability tests (installation/un-installation of drivers)
under Windows 7 (32bits/64bits) and WS 2003
 Several days load testing done under Windows 7(32 bits)
and WS 2003
 Network latency tests (CMS ECAL requirements)
(EN/ICE)
JCOP FWWG- 4
Tests setup with DIGI 5 ports
Industrial Controls
Engineering Department
Windows Server 2003
Application (TestInteractive.exe)
SYSTEC Wrapper (canlib32.dll v1.8)
SYSTEC (Usbcan32.dll and Fw v4.15)
DIGI Driver: AwUsbCfg: 3.0.30.0
Network Sniffer
Wireshark
Windows 7
Application (TestInteractive.exe)
SYSTEC Wrapper (canlib32.dll v1.8)
SYSTEC (Usbcan32.dll and Fw v4.15)
DIGI Driver: AwUsbCfg: 3.0.30.0
Group 2
(DIGI USB port
2 & 9)
Ethernet
Network Sniffer
Wireshark
Netw
SWITCH
Group 1
(DIGI USB port
1 & 8)
DIGI 5 ports
Firmware: 1.36.8.79
USB1
USB1
CAN1
CAN0
USB3
USB2
USB4
USB5
USB2
CAN3
CAN5
CAN7
CAN9
CAN11
CAN13
CAN15
CAN2
CAN4
CAN6
CAN8
CAN10
CAN12
CAN14
SYSTEC CAN-USBModul16
S
R
R
(EN/ICE)
JCOP FWWG - 5
Industrial Controls
Engineering Department
Tests setup with DIGI 14 ports
Windows Server 2003
Application (TestInteractive.exe)
SYSTEC Wrapper (canlib32.dll v1.8)
SYSTEC (Usbcan32.dll and Fw v4.15)
DIGI Driver: AwUsbCfg: 3.0.30.0
Network Sniffer
Wireshark
Windows 7
Application (TestInteractive.exe)
SYSTEC Wrapper (canlib32.dll v1.8)
SYSTEC (Usbcan32.dll and Fw v4.15)
DIGI Driver: AwUsbCfg: 3.0.30.0
Group 2
(DIGI USB port
2 & 9)
Ethernet
Network Sniffer
Wireshark
Netw
SWITCH
Group 1
(DIGI USB port
1 & 8)
DIGI 14 ports
Firmware: 1.38.1010
USB1
USB3
USB2
USB4
USB8
USB2
USB1
USB1
CAN1
CAN3
CAN5
CAN7
CAN9
CAN11
CAN13
CAN15
CAN0
CAN2
CAN4
CAN6
CAN8
CAN10
CAN12
CAN14
CAN1
CAN0
SYSTEC CAN-USBModul16
S
R
R
USB9
CAN3
CAN2
USB2
CAN5
CAN4
CAN7
CAN9
CAN11
CAN13
CAN15
CAN6
CAN8
CAN10
CAN12
CAN14
SYSTEC CAN-USBModul16
S
(EN/ICE)
R
R
JCOP FWWG - 6
CMS ECAL network latency tests
Industrial Controls
Engineering Department
 Purchased 14-port DIGI for all ECAL CAN based readout
 Problems seen after deployment with Wiener OPC data
 Invalid data for Wiener devices every couple of days
 Production system rollback until understood
 Wiener OPC server has hard-coded timeouts for HW polling
 Suspected additional latency of DIGI causing problem
 Latency tests designed with EN-ICE to investigate this
November 22, 2011
(EN/ICE)
JCOP FWWG- 7
Industrial Controls
Engineering Department
CMS ECAL network latency tests
CAN data
generator &
latency test
PC
Wireshark
Ethernet
(GPN)
DIGI
AnywhereUSB
USB
SYS TEC
USB-CANmodul16
CAN
 Danny’s CAN data generator & test tool
 CAN frames are generated with unique IDs
 Time taken between send and receive of each frame is measured
 Wireshark used to capture and understand IP traffic
 Another tool from Danny to analyse the capture files
 Quick identification of significant delays
 Infrequent but long delays (high latency) were seen
 Correlated to network traffic events
 Worst delays due to packet loss
 More delays seen on more complex networks
November 22, 2011
(EN/ICE)
JCOP FWWG- 8
CMS ECAL network latency further tests
CAN data
generator &
latency test
PC
Ethernet
(GPN)
Wireshark
DIGI
AnywhereUSB
USB
Industrial Controls
Engineering Department
SYS TEC
USB-CANmodul16
CAN
Wireshark
 DIGI sends all packets
 DIGI network interface performs >= other network hardware in tests
PVSS
Wiener OPC
Server
PC
Wireshark
Ethernet
(GPN)
DIGI
AnywhereUSB
 Invalid OPC data correlates exactly with
high DIGI latency due to packet loss
November 22, 2011
(EN/ICE)
USB
SYS TEC
USB-CANmodul16
CAN
Wiener
PL508-DO
JCOP FWWG- 9
CMS ECAL network latency results
Industrial Controls
Engineering Department
Latency
Explanation
< 50 ms
Typical latency of round-trip
< 150 ms
Worst cases normal situation
~ 400 ms
Delay due to address resolution (ARP) request
being sent to DIGI
~ 1.2 s
Usual delay in case of packet loss & causes
problems with Wiener OPC server
 Performance is usually very good
 Uses TCP PUSH for low latency transfer
 TCP packets lost more frequently than expected (not due to DIGI)
 DIGI application flow control seems to limit data transfer and
increase latency when recovering from errors
 Perhaps they can improve this…
November 22, 2011
(EN/ICE)
JCOP FWWG- 10
Current known issues/limitations
Industrial Controls
Engineering Department
 Windows 7(64bits) & Windows Server versions are
currently NOT supported by SYSTEC
 No Linux driver available for DIGI
 Limited number of other device on the market (only one
supplier found and device is out of stock!)
 DIGI high latency
(EN/ICE)
JCOP FWWG- 11
LS1 and long term actions?
Industrial Controls
Engineering Department
 LS1 requirements
 If DIGI solution
 Need Systec supported for
Windows 7(64bits), Windows Server 2008 R2
 Network latency causing Wiener OPC server bad quality to be
fixed (DIGI level?)
 Additional tests/requirements?
 “Long term”
 Ethernet-CAN interface solution
 Solution for OPC UA
 Alternative solutions?
November 22, 2011
(EN/ICE)
JCOP FWWG- 12