Slides - Crest
Download
Report
Transcript Slides - Crest
Softeam’s testing
process: a story
Cyril Ballagny, SOFTEAM | ModelioSoft
The 13th CREST Open Workshop
London, England
13th of May 2011
www.modeliosoft.com
Introduction to Softeam
A French SME (about 230 people)
More than 20 years of expertise in OO development and
Modeling
Consulting, Training, Engineering activities
Software Publishing: Modelio and Objecteering
Strong R&D efforts since its creation
– Innovator since 1989 (Modeling, MDA, UML)
– Heavily involved in R&D cooperative projects (FITTEST, MADES,
ENOSYS…)
– Participates in the OMG standardization efforts since 1996 (UML
profiles, SOAML, SysML…)
www.modeliosoft.com
2
Introduction to Modelio
A CASE tool
For UML modeling
For MDA application
For Code generation (Java,
C#, C++…)
For documentation
generation (HTML, word…)
For team work and project
sharing
And much more (BPMN,
SysML, SOAML…)!
www.modeliosoft.com
3
Modelio development process
Agile development with bootstrapping and continuous integration
20 people involved
1 million LOC (C++/Java)
A new release every 6 months
www.modeliosoft.com
4
Modelio testing process (1)
Code quality management with Sonar: about 500 coding
rules
Continuous integration with Hudson: report broken build
(inconsistent API, missing source in the repository…)
Bug tracking with Mantis: everyone from the team can
report bugs
Specification of test plans with Test Link (natural
language): about 2000 procedures
Bootstrapping: Modelio is validated by developers
www.modeliosoft.com
5
Modelio testing process (2)
A test case is either
A Mantis bug (bug 2123, bug 9478…)
A menu item (create, delete, mask…)
A diagram artifact (state, region, activity…)
Most test cases are UI-based and derived from general
scenarios (e.g. apply command “xxx” to element in
{a,b,c…} )
Test case execution is externalized to Eastern Europe
www.modeliosoft.com
6
Our issues
Specification often changes
Writing operational test specification is too time consuming
compared with natural language specification
Test code must be tested too
Mix of languages (C++/Java/Jython)
No tools for testing HMI combining Java and C++
www.modeliosoft.com
7
Our expectations (1)
Reduce over-testing
Detect overlapping sections of test cases
Recombine sections to constitute smaller test cases with greater
coverage
Apply regression tests only on parts of the system that have
changed
Reduce under-testing
Detect system parts that are not covered by the test cases
Detect sequences of interactions allowed by the GUI that have
not been tested
www.modeliosoft.com
8
Our expectations (2)
Reduce Time-To-Market
Generate test cases instead of specifying them
Prioritize:
Identify the most used features
Identify the most frequent bugs
www.modeliosoft.com
9
To sum up
We would like a scalable and pragmatic approach
whose ROI has been demonstrated
www.modeliosoft.com
10
Thank you for your attention!
Andrey Sadovykh
SOFTEAM | ModelioSoft
[email protected]
SOFTEAM R&D Web Site:
http://rd.softeam.com
ModelioSoft Web Site:
http://www.modeliosoft.com
www.modeliosoft.com
11
M o d e l i n g s o l u t i o n s.