Transcript Testing Environments

Test Environments
Arun Murugan – u4430411
Rohan Ahluwalia – u4358281
Shuchi Gauri – u4358305
I have developed a great piece of code. I
am doing some Unit Testing and my code
works great. There is no error in it. No
Bugs... Hooray!!!
My job is to verify and validate the piece
of software this developer has made. I
have so many test cases to run.
Hey!!!!!
I am this cool software
over which these two
people work. Let’s see
what happens..
More……..
Oh ok,
I’ll write a mail
to the Project
Manager
Manager,
This software is just
not working. I think
developers don’t know
what they have given.
It just doesn’t work.
But PM,
everything is
working fine
here.
More……..
Ok ok..
Let’s just go to
the tester and
see what’s wrong.
Does not
work
It
works
Crisis!!!!
What will happen
to the project.
May be it’s an
environment
issue. Lets
compare the
environments
We
have
SP2
here
We have
SP1 here
Yes, we should
now have a
checklist for
environment
configuration and
control it.
I am so glad that
the problem has
been solved. We
wasted one whole
day to figure it
out.
Test Environments
Topics Covered:
 Development and management of test environments.
 How many environments?
 How to manage differences in test environments from development
through to test, acceptance and production environments.
 Issues associated with not being able to replicate the operational
(production) environment.
 Potential issues arising from testing with real production data and
non production data.
Development & Management of Test Environments
Establishing the right testing environment is an essential component of an
effective and efficient software testing process.
Key components of the environment are:
 Software testing work processes.
 Software testing and test management tools (software and hardware).
 Competent testing staff.
 Process improvement methods.
 Operating systems, and Environment.
 Test data.
Test environments..
 need to be controlled through various levels of management.
 are driven by business needs.
 enhance productivity of testers.
 are responsibility of Senior IT management.
 should be as close as possible to the production environment.
Development & Management of Test Environments
contd..
Role of management in creating test environment:
 Decide policy for testing
 Type of support provided by management for software testing.
 Processes and tools.
 Level of “risk appetite” the management has.
The testing environment to a large degree enables testers to perform a
role that will both identify early in the project potential gaps, and
during the project determine the magnitude of those gaps.
The testing environment must help testers identify:
 Gaps between software needs and delivered software.
 Gaps between software specifications and delivered software.
Development & Management of Test Environments
contd..
There is a need to manage and
control differences between test
environments:
 To uncover issues consistently
across all the environments and
not just some.
 To enhance predictability.
 To avoid surprises/shocks.
 To keep the customer’s
confidence in the project and staff.
Test environment throughout development
lifecycle







Important to have environment throughout development lifecycle as
close as production.
Test environment would ensure proper validation of the project.
Test environment for development should be separate from
production, so that no changes are required on fly.
Levels: Hardware, Software/OS versions, environment to ensure
proper configuration management.
Using test support tools to manage a testing environment. E.g.
defect tracking systems, automation tools.
A quality process should always check the environments to ensure
their closeness to the production environment.
Minimum 3:



1 – Development
2 – Testing
3 – Production
How to manage diff. b/w environments
Organizations prefer to maintain separate environments for
development, testing, acceptance and production.
 Under development, the team continually implements new features
and makes changes to improve performance.
 Under production, stability is required and changes are done only
on the basis of bug fixing.
Therefore, we need a common logical design across different physical
environments.
 Steps:
 Ensure same h/w and s/w components are installed across all
the environments.
 Create a configuration checklist.
 Get this checklist reviewed from QA as and when required.
 Use Version control or any change management/control process
to manage the changes in the environment.
 Compare the version mapping at all times. If versions differ,
deploy same versions across all environments.

Issues associated with not being able to
replicate the production environment








Reliability is a concern as real scenario might be quite different.
Huge business and market risk.
Handling huge volumes of data could be unexpected.
There might be new integration issues in production environment
while integrating with other 3rd party software interfaces.
Hardware performance may depreciate on production environment.
There might be some real life scenarios that cannot be tested on
any environment apart from production and thus may cause shocks.
Some scenarios might occur more frequently in production
environment than in testing environment.
Low predictability and confidence among the staff and customers.
Issues in Testing with Real Production Data





Data Confidentiality and data security issues.
Data acquisition & integrity issues.
Cannot be played around with as and when required. May not be
able to satisfy all testing objectives.
Cannot be controlled.
May require special resources, e.g. DBA, better hardware.
Issues in testing with non-production data




Incomplete and incorrect data. It is impossible to create test data
keeping all the test situations in mind.
Non-production data maybe much cleaner than the production data
resulting in a less robust system.
There is no way to ensure completely that the non production data
will test the system for real data too.
The tester needs to first determine the expected results.
References


Managing Multiple environments from Development to Production
[http://download.oracle.com/docs/cd/B28359_01/owb.111/b31280/co
nfigs_11.htm]
Effective methods for Software testing [William E. Perry]