Transcript Mesa Testing - IHE Product Registry

(Integrating the Healthcare Enterprise)
IHE in Europe
Connectathon Organization
Eric Poiseau
Version 1.0
February 9th 2005
IHE-EU Conference Workshop
Who are you ?
77 companies
120 systems
4 domains (RAD, IT-I, CARD, LAB)
RAD : 90 systems
IT-I: 78 systems (XDS 22)
CARD : 28 systems
LAB: 21 systems
February 9th 2005
IHE-EU Conference Workshop
Progression
120
+ 50 %
100
80
Companies
Systems
60
40
20
0
2002
February 9th 2005
2003
2004
2005
IHE-EU Conference Workshop
We need to get prepared
Pre-connectathon period
– 10 weeks
Connectathon
– 5 days (9 am to 6 pm)… but in the fact only
4 days for testing
Need for optimization
– A lot to do before
– To perform more testing during CAT
February 9th 2005
IHE-EU Conference Workshop
A few numbers…
 10 weeks gives 400 hours of work
 ~ 2700 logs are required
– 5 min per log file makes 225 hours
 ~ 120 systems
No time
for
– 1 hour on phone/email makes 120 hours
Baby sitting
 That gives me 155 hours for the rest…
– Lab tools, connectathon preparation, webtool
maintenance, test data,….
February 9th 2005
IHE-EU Conference Workshop
Pre-Connectathon
 Check your registration
– I do not want you to tell me during the connectathon : « I
have forgotten to sign up for PIR, I though…. »
 Configuration information gathering (you to me)
– Dicom Conformance Statements
– Port numbers, URI, AE Titles….
 Configuration of your system (me to you)
– Before Monday morning….so before shipping
– You need to configure your system
 With your peer information
 With the connectathon network information
 With the testing dataset
 Mesa Test
 Object Rendering Tests
February 9th 2005
New
IHE-EU Conference Workshop
Registration Checking
Combined actors/profile
– PPSM for OF and IM/IA
– PIR requires SWF
– SWF and ARI
–…
Supportive or Thorough ?
New
– We do not know yet how supportive will be
reported in result matrix and if it will
February 9th 2005
IHE-EU Conference Workshop
Supportive versus Thorough
 Thorough
– This is what we have done so far
 Supportive
New
– You’re confident about this Actor/IP but you want
to bring it at the connectathon
– You have already tested it at a previous CAT
– Need to provide URL to IS (published)
 Result matrix will show thorough results
February 9th 2005
IHE-EU Conference Workshop
System Configuration / Information
System Information
 Dicom Conformance Statement
– Upload form
New
 Practical Information
–
–
–
–
Power information
Number of chairs
Storage
Special requirements -> contact me
 Please provide this information before March 1st
February 9th 2005
IHE-EU Conference Workshop
Configuration Information
 Have your configuration ready and confirmed before
the deadline (April 15th)
– I have assigned some default values
– If you change it… your peer will have to change it also….
 How important is it ?
– Most of the time spend in doing a connectathon test is spend
checking/changing the configuration parameters.
– More systems more choices of peers…. Try not to be the
bad guy.
 Q : How to provide my configuration information ?
– Answer
 Q : How to find my peer configurations parameters ?
– Answer
February 9th 2005
IHE-EU Conference Workshop
Configuration Information
 If you feel that there are missing parameters,
– this is highly probable. Let me know ASAP
 No confirmation will be understood as a
confirmation!
– This means that your peers will configure their
systems with information that may be erroneous !
 Configure your systems before arriving to the
connectathon
February 9th 2005
IHE-EU Conference Workshop
Mesa Testing
General Information
Why Do We Have Test Tools ?
1. Reduce the number of systems to 80
because the room we have reserved
for the connectathon is too small !
2. The project manager is evil and he
likes to make you suffer with buggy
mesa tools !
3. Prepare your systems for the
connectathon !
February 9th 2005
IHE-EU Conference Workshop
Why Do We Have Test Tools ?
Test tools prepare you for the
connectathon
– Provides a minimum set of test to prepare
your systems
Tools are a benefit of participation
– As a participant you get them before the
public release
Tools are released to the public later on
February 9th 2005
IHE-EU Conference Workshop
How Do Tests Help You ?
 Tests encourage you and your peers to read
details of the Technical Framework
– You may find you do not agree with our
interpretation of the TF
– You may not agree with the TF
 If you and your peers complete the tests, we
can avoid a lot of wasted time at the
Connectathon
 Thus we can prepare more complex
demonstrations and do more complex
connectathon tests
February 9th 2005
IHE-EU Conference Workshop
Test Coverage
Test software do NOT cover all aspects
of the Technical Framework
Test software do NOT guarantee
compliance with DICOM or HL7.
All what it does, is provide a reasonable
preparation for the Connectathon.
February 9th 2005
IHE-EU Conference Workshop
Test Cases versus Demonstration
MESA tests cover more transactions
than will be demonstrated
Participants are required to complete all
test cases, even if they do not lead
directly to demonstration
Thus : Your system may perform one
transaction at a demonstration but
perform several transactions with the
MESA tests (and the Connectathon)
February 9th 2005
IHE-EU Conference Workshop
Mesa Testing
How to
Rad – Card – IT-I (except XDS)
February 9th 2005
IHE-EU Conference Workshop
MESA Tools
Simulate your peers
Mesa Simulator
Win32 / Linux / Solaris
System being tested
Win32 / Linux / Solaris
Order
Filler
MESA
ADT, OP, IM/IA
TCP/IP
Network
MESA OS is independant of system under test OS
Choosing NT, Linux or Solaris is a question of taste
February 9th 2005
IHE-EU Conference Workshop
Hardware/network requirement
for testing
 One computer system can :
– Support all of the MESA actors
– Can serve as the driver for any system under test.
 Only one computer required to test an IHE
actor.
 It is impossible to test multiple systems
simultaneously
– (sockets are not multithreaded).
 You need to test separate systems serially
with little or no reconfiguration.
February 9th 2005
IHE-EU Conference Workshop
Mesa test how to
Where do I get the tools from ?
– http://ihe.univ-rennes1.fr/testing
– If you’re logged in you’ll get the last release
– If not you’ll get the public release
Who writes the tools ?
– MIR, Offis, NIST, IDM
What if I find a bug in a tool ?
– MIR, Offis : http://rome.wustl.edu/bugzilla
– NIST, IDM : contact me !
February 9th 2005
IHE-EU Conference Workshop
Installation
Make sure you follow the installation
instructions provided by Steven Moore
Do not try to install from the sources
– Sources are provided for information
February 9th 2005
IHE-EU Conference Workshop
Most common errors encountered
Environment variables not set
Mesa_storage files not installed
Wrong version of Win32 Perl
SQL server is not installed
Database tables are not installed
But also possible errors in the tools !
February 9th 2005
IHE-EU Conference Workshop
Recommendations for MESA
Software
We recommend that you install and test
the software as soon as possible
The authors try to make the installation
process simple, but there will be some
difficulties
It is better to go through a round of
installation now rather than waiting until
the “final/next” release is available;
February 9th 2005
IHE-EU Conference Workshop
MESA Test Interactions
Most MESA tests are independent of
other tests. You can run them in any
order.
Most tests overwrite common data
areas. They do not overwrite the output
of the evaluation scripts.
This implies you should run a test and
perform the evaluation before starting
another test.
February 9th 2005
IHE-EU Conference Workshop
MESA Test Overview
 Each actor/integration profile has several
tests, each test is assigned a number (101,
102, …)
 100 series tests are for the Scheduled
Workflow / Patient Information Reconciliation
 Other series of tests are for different features
of the Technical Framework (for actors or
classes of actors)
 How to know what test your systems have to
pass ?
– Check the web tool on your system page
February 9th 2005
IHE-EU Conference Workshop
General Test Instructions
 Start MESA servers on test machine
 Run an interactive perl script that leads you
through the test scenario
 Run a second perl script that evaluates your
messages
 Check the logs for errors and try to
understand the problems
 Send output of evaluation script to Project
Manager
 Please comment your logs if this is needed
February 9th 2005
IHE-EU Conference Workshop
Test logs return (2)
Do not send logs if you know that they
will be rejected.
– Check Bugzilla pages for information
– Ask for some help
When “error” please explain the errors
Upload form for log return New
– Please mention if you are resubmitting
“old” logs
February 9th 2005
IHE-EU Conference Workshop
XDS
February 9th 2005
IHE-EU Conference Workshop
New
NIST Toolkit
 Remote Registry and Repository hosted by
NIST
 Sources can submit documents to the NIST
server
 Consumer can query and retrieve documents
from the NIST server
 Java tool to submit and query a registry
 Log browser for PM to check/help testers
February 9th 2005
IHE-EU Conference Workshop
New
Laboratory Tools
Not script for the moment
Message validation software
– http://ihe.univ-rennes1.fr/HL7
February 9th 2005
IHE-EU Conference Workshop
Connectathon Logistics
February 9th 2005
IHE-EU Conference Workshop
Connectathon Codes
 Tables and codes used for the connectathon will be
provided :
–
–
–
–
–
HL7 user tables
MD’s
Beds, Facilities, Assigning authorities
Procedures codes (lab, radio, and cardio)
XDS Affinity domain information
 OIDs
– Set of patients ID and identities for non ADT clients
 If you think there are some missing tables and/or
information, please let me know
February 9th 2005
IHE-EU Conference Workshop
ADT
Each ADT actor will be assigned a
range of pid to avoid confusion.
– Patient Identifiers are 6 digit identifiers that
begin with a two-digit code that identifies
the system producing the ID.
– Thus, if your prefix is 10, your Patient
Identifiers will be in the range of 100000 to
109999.
February 9th 2005
IHE-EU Conference Workshop
Digital Certificates
 I’ll provide you with private key and certificate
 One certificate assigned for each secure
system
 Certificates to be used only for connectathon
and demonstrations
 Certificates are not meant to be used in a
production environment
 We provide no guidance on how to manage
those in a clinical environment
February 9th 2005
IHE-EU Conference Workshop
Webtool for test management
Provides list of possible peers
Provides chat/log
Provides verification logging
Provides information about « persistent
objects » New
February 9th 2005
IHE-EU Conference Workshop
Network Information
 We will provide
–
–
–
–
–
–
–
–
–
IP Addresses and Host file
Local DNS (preferred to host file)
Netmask
Gateways
Time Server
Kerberized Server
New
Mail Server
New
LDAP
New
XDS Repository and Registry
February 9th 2005
IHE-EU Conference Workshop
Network Information
Intranet will be at least 100 Mb/s
Internet will be at least 10 Mb/s
– Internet is used for convenience for email
and communication of files with home
based team.
– Internet access is not guaranteed
You will get on drop per system
– It is your responsibility to bring a hub
February 9th 2005
IHE-EU Conference Workshop
Power Supply
240 Volts
A power strips with 3-5 slots per table
UK plugs do not fit to NL outlet !!
– Are we not talking about interoperability !
If you need more than 240 V, please let
me know ASAP !
February 9th 2005
IHE-EU Conference Workshop
Tables and chairs
120 tables
– One per system
– Please use the table that will be assigned
to your system
– If your company brings 2 systems, they
might not be close to each other !!!
February 9th 2005
IHE-EU Conference Workshop
Connectathon Test
February 9th 2005
IHE-EU Conference Workshop
Preliminary Connectathon
Schedule
 Monday :
– am : installation of your systems
 Systems delivery
 Check network access, correctness of configuration
– pm : start of testing (p2p)
….
 Wednesday :
– start of scenario testing
 Friday :
– am : very last tests
– pm : packing
 Package pick-up
February 9th 2005
IHE-EU Conference Workshop
Peer To Peer Tests (P2P)
You have to test with some peers
You need to get at least 3 instances of a
test verified per test
Peer are prioritized
You can do more testing.
February 9th 2005
IHE-EU Conference Workshop
Peer To Peer Tests (P2P)
Enhancement :
– Use of a chat room for peers to
communicate and keep track of the
transactions.
– Use of log storage mechanism : (cut and
New
paste the logs in text file)
Other suggestions/ideas
February 9th 2005
IHE-EU Conference Workshop
Scénarii Tests
Compared to last year
– Increased complexity
Could combine RSWF, LSWF, PIX, RID, EUA,
XDS…
– Increased number of peers
Will have a schedule, so will need you
to be ready at the time of the test.
Will have a list of difficulties to check.
February 9th 2005
IHE-EU Conference Workshop
Hints
Switching peer should be an easy and
rapid task.
– Try to avoid need of rebooting
– Try to avoid typing the peer configuration
each time you need to switch.
Have the procedure codes entered in
your systems.
February 9th 2005
IHE-EU Conference Workshop
How can I fail
I have not a single test verified by
Wednesday evening !!!
I am spying my colleagues !!!
I am not fair to the PM !!!
February 9th 2005
IHE-EU Conference Workshop
Special Cases
 You all are in a special case !!!!
 I have finished all my testing on Monday, shall
I leave ????
– Print Server case
 I have 5 zillion tests to perform during the
week, …can we stay for another 2 weeks…
– Register more systems
– Register fewer profiles/actors
February 9th 2005
IHE-EU Conference Workshop
Results
Concerns only positive results
Test failure are “forgotten”
Matrix published in the following weeks
February 9th 2005
IHE-EU Conference Workshop
Diverse Issues
Communication
Me to you
– Email (mailing list)
– News on IHE-Europe Web (RSS)
– Phone
You to Me :
– Email +++
– Phone ---
February 9th 2005
IHE-EU Conference Workshop
Communication
 Protocol (are we not dealing with interoperability):
–
–
–
–
–
Company
System
Operating system
Actor tested
Tool Version…
 Any information about your context will help me
– Remember that you have an open credit of ½ hour per
systems… do not waste it !!!!
February 9th 2005
IHE-EU Conference Workshop
Shipping
Shipping information
– We will let you know in time.
Storage
– We will have a space for storage
– If possible, for small boxes store in your
room
Security during the nights
– Security from the hotel
February 9th 2005
IHE-EU Conference Workshop
February 9th 2005
IHE-EU Conference Workshop
February 9th 2005
IHE-EU Conference Workshop
February 9th 2005
IHE-EU Conference Workshop
Connectathon Check List




Swimming Suit and squash equipment
Bowling bowl
Technical Framework
Your system fully configured
– Network
– Codes, procedures, tables….
 Laptop to run the webtool
– Could be also used to sniff messages or service you system
 Power strip + power cord extension
 Hub (preferred to switch)
 Calibration material for CPI profile actors
– Print Server and print composer
February 9th 2005
IHE-EU Conference Workshop
Thank you
February 9th 2005
IHE-EU Conference Workshop