Transcript Presentation

Improving Software
Testing by Observing
Process
-Ossi Taipale
-Kari Smolander
Lappeenranta University of
Technology, Finland
Presented by Albert Saryan and Karo Mazidzhyan
Breakdown
Introduction
 Related Research
 Research Process
 Analysis Results



Process Improvement Propositions
Conclusion
Introduction
The objective of this study was to
understand how software testing is
conducted by observation.
 From observations propose improvement
to the testing process.
 Improvements by reducing development
and testing costs, and improving quality.

Software Costs and Quality
Software Engineering strives to reduce
development costs and improving quality.
 Software Process Improvements (SPI) are
the means to reaching these goals.
 Commitment to SPI’s by all from all
organizational levels is key to success
 Quality can be tested into products or
developed and built into products

Software Costs and Quality Cont.
External events such as deadlines affect
software quality.
 The cost of software testing is high,
therefore SPI’s are necessary to reduce
cost.

Related Research
Involvement of testing during
development occurs when testers develop
test for developers to analyze
 The complexity of testing increases as a
function of the complexity of the systems
under testing.
 Testing strategy defines the contents of
testing.

Related Research Cont.




Communication and interaction between
development and testing processes requires
cooperation and coordination.
The use of software components are increasing
rapidly
Design outsourcing and distributed
development increase the use of components.
Cost of Quality is “Free”, but being late with
products may be more costly than fixing faults.
Research Process
This study consisted of Organizational
Units (OU) which develop and test
technical software for automation or
telecommunication in Finland.
 Initial Sample included 26 OU’s, from
which 5 were used as case studies.
 Cases were chosen to show polar types

Research Process Cont.
Data for the research was collected by a
series interviews.
 Each interview had a different theme in
mind and possibly a different interviewee
in mind.
 The interviews took place during five
rounds, based on the theme.

Research Process Cont.

1.
2.
3.
4.
Interview Rounds
Development and Testing Managers were
asked to define their testing process .
Managers of Testing were asked to define
their testing process in depth.
Testers were interviewed.
Systems Analysts interviewed.
Research Process Cont.
Case Breakdowns
Research Process Cont.

Data Analysis



Information gathered
from these interviews
were then categorized.
Categories were then
analyzed to see how they
were connected.
The categories were then
used to identify factors
which affected testing.
Analysis Results

Description of Cases:

Case A - Developed and tested Manufacturing Execution
Systems :






Turnover 50% product, 50% service
Services included systems integration and customization
Testing against requirements was a challenge because
customers had special in-house requirements standards
Developers and testers worked physically close to each
other
Time allocated to testing was consistent, although over
time it has been reduced
Use of components low, hinders testing
Analysis Results Cont.

Case B - Tested in house products and
provided testing services for external
customers:





Turnover 75% service, 25% product
Majority of requirements specifications were based on
standards.
Delays in development allowed for fewer time
allocated for testing
Communication flexible, developers talked face-toface with testers
Use of components high, testability of components
must be considered
Analysis Results Cont.

Case C – Customized Software Development:







Turnover 2/3 service, 1/3 product
Testers were involved early, involved in development
process
Testers often had issues due to lack of advisement
from developers
Delays in development does not often result in
reduction of testing time
Developers and testers communicate face-to-face
Use of components low
Testing of software components seen as difficult b/c
of different implementation.
Analysis Results Cont.

Case D - Electronics:








Turnover 100% product
High product orientation required high quality
because recalls are very expensive
Avoided testing of unfinished product
Testing tasks clear and well documented
Testing involved in development late but planning of
testing automation provided information on upcoming
tests
Communication between developers and tester
planned, formal and transparent
Use of components high
Components tested initially by suppliers then again at
system testing.
Analysis Results Cont.

Case E – Software Testing Services:








Turnover 100% service
Working as an external testing organization required the
adaptation of the process of the customer
Early involvement of testing was necessary for the testing
company to increase the testability of the software
Sometimes testing was involved late
Budgets for testing affected testing time
Communication was handle through a contact person, but
was active and clear
Use of components depends on customer
As an external testing organization it was hard to receive
information about customers purchased components.
Analysis Results Cont.
Analysis Results Cont.

Cause and effect
Analysis Result Cont.

Cause and Effect

Business Orientation


Directly associated with use of components and testing
schedules
Business Model – value adding process

Purely Service Oriented





System integration
Customizing
Consulting
Customers directly affect development and testing process
Purely Product Oriented
 Product Development
 Marketing
 Customers do not directly affect development and testing
process
Analysis Results Cont.

Process Improvement Propositions

Testing ought to be adjusted to business
orientation



Product oriented should adopt formal planned testing
process
Service oriented should adopt a flexible testing
process
Enhanced Testability of software


Consider testability when selecting components
Review testing process of suppliers
Analysis Results Cont.



Effective Communication and interaction
between development and testing
Early involvement of testing and planning of
testing
Use of risk based testing
Conclusion





Proposals by observing best practices using
grounded theory
Better documentation improved testability of
software and components
Efficient communication between development
and testing improved quality
When time is a major issue, risk based testing is
the best solution
Business orientation affects:



Use of components
Amount and quality of communication
Allocated testing time and the planning of testing