Transcript Slide 1
TelePresence Microscopy at CSIRO Minerals Colin MacRae CSIRO Minerals - Microbeam Laboratory www.minerals.csiro.au/microbeamlab Talk outline What and why? History of TelePresence at CSIRO Minerals 3rd generation TelePresence system System architecture Control interfaces Video streaming technology Security features Administration Modular installation of new functions Conclusions, where to from here? Remote Access and automation Workshop 2005 What is TelePresence? Tele-conferencing Tele-control Video + Audio Microscope controls On-line data collection + analysis Tele-control is provided by some manufacturers Ideally, TelePresence should be: Interoperable with many platforms ‘Shrink-wrapped’ software • Looks professional, with easy to use interfaces Fast and responsive Stable and secure Remote Access and automation Workshop 2005 Why do we want TelePresence? Internal clients (Brisbane, Hobart, Perth) External clients (Melbourne, Sydney, Brisbane) For analyses, our clients must either: TelePresence can help Mail / courier samples, and wait for our report (slow) Travel to our laboratory (expensive) Provide results to client in real-time, and also allows instant feedback from client Attract new clients CSIRO Minerals CSIRO BHP-Billiton Minerals, Exploration & Mining CSIRO Comalco & RioTinto Marine Research Remote Access and automation Workshop 2005 What do we want to connect? Scanning Electron Microscopes Electron Microprobes & Proton Microprobe X-ray detectors –EDS & WDS Cathodoluminescence systems Optical microscopes Remote Access and automation Workshop 2005 What do we want to offer Clear fast images On-line microanalysis On-line x-ray spectra Large area optical views Streaming maps Security Voice over IP Data storage Data analysis/presentation software Remote Access and automation Workshop 2005 What do we need from TelePresence? Provision of: Real-time video from multiple sources (SEM, room camera) Streaming EDS spectra and other data ( + controls) Control of microscope stage + column (lower priority) Minimal hardware requirements PC or laptop (no workstations) ADSL or cable modem, ~1Mbit/s (not Internet2, GRID) ‘Shrink-wrap’ software Easy to set-up Very easy to learn and use, with generic controls/interface Very, very stable Extraordinary security Low maintenance Must not affect normal operation of instruments! Remote Access and automation Workshop 2005 TelePresence history TelePresence Microscopy 1999 Nestor J. Zaluzec, Argonne National Laboratory Web-based, ‘pervasive’ Web based notebook Video with “GTS” streaming technology Remote Access and automation Workshop 2005 TelePresence history TelePresence Microscopy 1999 Nestor J. Zaluzec, Argonne National Laboratory Web-based, ‘pervasive’ Java system 2001 Platform independent, ‘safe’ code MJPEG video (TCP-IP) Web based notebook with embedded Java applets for Images, Spectra and maps Successful, but limited Remote Access and automation Workshop 2005 TelePresence history TelePresence Microscopy - 1999 Nestor J. Zaluzec, Argonne National Laboratory Web-based, ‘pervasive’ Java system - 2001 Platform independent, ‘safe’ code MJPEG video Web based notebook with embedded Java applets for Images, Spectra and maps Successful, but limited C++ system – 2004 Widely used language, with broad library support MPEG-4 video compression in software, changeable video size New, secure architecture Remote Access and automation Workshop 2005 Internet protocols: UDP versus TCP Sender Router TCP packet TCP acknowledgement Lost packet? UDP packet UDP TCP = Transport Control Protocol UDP = User Datagram Protocol Remote Access and automation Workshop 2005 Video latency Definition: What do we want? Latency = time displayed – time captured <500ms for focussing (or better) <200ms preferred Causes of latency 1. Video compression / decompression • Hardware accelerated rendering (done) • Hardware compression (cheap MPEG-4 hardware compression coming soon) 2. Router delays (congestion) • Ultra-broadband internet (eg GRID) • Reduce stream size (better codec) 3. inter-router delays (speed / distance) • More bandwidth wont help • Use faster internet protocol (i.e. TCP UDP) Remote Access and automation Workshop 2005 Another real life problem : The Firewall Firewall Client network Internet Firewall Our network port 8080 (UDP), from specified addresses Server Client Video port 8080 (TCP) Host ‘DMZ’ Remote Access and automation Workshop 2005 Comparison of platforms Web browser Java C++ Functionality Low Moderate High Execution speed Low High High The vibe Poor So so Development pain Low High Platform independent? Yes Yes Trusted? Somewhat Yes Remote Access and automation Workshop 2005 Yeah groovy Moderate No Yes, with security System architecture Password Client (TCP socket) commands & data Host Server JEOL 8900R Electron Microprobe Server Video (UDP stream) FEG-Environmental SEM Server FEI Quanta Hitachi S-5000 In-Lens Cold FEG SEM Generic interface Instrument specific Drivers Remote Access and automation Workshop 2005 PIXE Proton Microprobe DirectShow video streaming Timing system (frame-rate) Video capture Network transmission sockets Compression codec ‘GraphEdit’ component of Microsoft DirectX software development kit (SDK) Remote Access and automation Workshop 2005 DirectShow video streaming Very easy to configure and re-configure ‘graphs’ Other stream designs are possible, eg: Saving stream to HDD Motion detection triggering Adaptable to as yet unforseen requirements All components are upgradeable and replaceable New codecs can be adopted as they become available ‘Future proof’ We didn’t have to write any of these components (only wire them together) Some DirectShow components are hardwareaccelerated Fast video rendering Remote Access and automation Workshop 2005 TelePresence interfaces Video Camera control Tilt-pan-zoom controls Used in conjunction with video window EDX spectrometers Generic, easy to use interface Fast (2-5 updates/s) Text streams Miscellaneous others Stage and Electron optical System Remote Access and automation Workshop 2005 Video streaming technology Microsoft DirectShow MPEG-4 encoding Widely supported standard with pre-compiled modules Hardware-accelerated rendering 320 240 at 25 fps is ~500kbit/s (~ADSL) (~12 Mbit/s by MJPEG) (~59 Mbit/s for raw video) UDP transmission Variations in humidity cause grass to spiral Themeda Australis and “dig” into the soil Kangaroo grass Remote Access and automation Workshop 2005 Lowest possible latency over IP (~200ms) Video cameras Canon VC-C1 and VC-C3 cameras 450 x 350 pixels at 30 frames/s. ±90º pan, ±25º tilt, 10 zoom Good low-light performance Remote Access and automation Workshop 2005 Bench-top camera ‘ELMO’ camera XGA video, 20 frames/s www.elmousa.com/presentation Remote Access and automation Workshop 2005 Client feedback Problem: Video stream is oneway. How do clients tell us what they want to see? Verbal feedback (telephone) “that thing, over to the left. Up, say, half an inch…” Not very effective Graphical feedback Video overlay marker Visible to all other clients and the operator More tools to follow Remote Access and automation Workshop 2005 Security Firewalls Blocks all direct communication with servers and drivers Host and servers are in separate ‘DMZ’ network, not mounted on CSIRO file system Socket encryption: Sockets are wrapped with ‘Rijndael’ cipher, NIST advanced encryption standard (AES 2001). • Certified for all levels of classification, up to and including ‘top-secret’. Hackers must break AES to read data, send false data, or issue commands Remote Access and automation Workshop 2005 On-line help Remote Access and automation Workshop 2005 Administration Remote Access and automation Workshop 2005 Web based administration Remote Access and automation Workshop 2005 Software based Administration tools Remote Access and automation Workshop 2005 Administration tools – Modular New instruments can be added and configured Remote Access and automation Workshop 2005 Summary TelePresence allows Microscopy facilities providing a alternative service to clients High-quality real-time video is achievable over cable modem using MPEG-4 codec's and UDP transmission Internet security is ensured using firewalls and encryption of communications Architecture is extensible and applicable to any instrument for which DLLs or control code are available Remote Access and automation Workshop 2005 Acknowledgements CSIRO Minerals Aaron Torpy Nick Wilson CSIRO ITS Tony Sanderson Andrew Smith Engineering Firm Cymontkowski Marc Cymontkowski Further information TPM web reference http://minerals.tpm.csiro.au/software/telepresence/ Remote Access and automation Workshop 2005 Questions? Remote Access and automation Workshop 2005