Transcript Why Test Automation

Test Automation
An Approach to Automated Software Regression Testing
Presented by
Adnet, Inc
Feb 2015
Test Automation – Introduction
Introduction
• Automates manual repeatable test processes using formal testing procedure
• Controls test execution, comparison of actual outcomes to predicted outcome
with test preconditions
• Saves time and deduces cost by automating repeated manual testing
Why Test Automation
•
•
•
•
•
Repeated manual test prone to human errors (overlooking)
Learning cost of new test teams or new members
Regression test cycles are increasing with distributed development
Wider acceptance of Agile model (needs frequent and more tests)
Manual test data setup with multiple data-sets increases complexity
2
Test Automation – Scope and Approach
• Scope and Approach (each software, not enterprise)
• What to Automate (often repeatable tests- regression tests)
•
•
•
•
Smoke Tests (undocumented exploratory tests)
Individual often repeated test cases
More useful for key business function where accuracy required
Easy accommodates variation in test datasets
• What not to automate
• Stress and Load Test
• Need different tools
• Different Strategy
3
Test Automation Strategy
• Reduce Cost
•
•
•
•
Reduce time in testing
Reduce cost from automation
ROI: Gain time in new function test
Reduce investment by using Freeware Tools rather than Vendor based
(Options)
• Automation strategy
•
•
•
•
Incrementally add more of the new functions for the next cycle of regression
testing
First automate smoke tests and then individual test cases
Baseline scripts and reuse each time.
Develop a library for commonly used routines like login for reuse.
4
4
Test Automation Strategy
5
Type of Frameworks
• Record-Playback
• Record once playback each time.
• Data-driven testing
• One script runs with multiple sets of data.
• Modularity-driven testing
• Common libraries for common functions like..
•
•
•
•
Login()
Check_VA_Enroll()
Check VA Eligibility()
Exit()
• Keyword-driven testing
• Keywords can make up a script..
•
•
•
•
Initialization():Launch browser and login
Test1() :Test for a Requirement
Test2() :Test for GUI like labels, links
Exit() :Exit from the browser
6
Test Automation Framework
Generic Functions
library
HP ALM Test Resources Module
Configuration
Files
Object
Repository
Documentation
Business Process
Library
Test Data Files
Any supporting docs / user
guides for the project
TestRunZ
HP ALM Test Plan /Lab Module
Test Set
TestRun
Automated
Tests (QTP)
Folders by
Release
TestRun
Scripts are organized
into folder named after
releases
Results get uploaded
back to Test Lab
Test Execution initiated
from Lab
Results
Test Machine / HEC App
Database
client /
drivers
Database drivers required
in order query the database
TestRun
Test Set
Detailed logs
Screenshots
Run Scripts
Application runs in
a browser
Results, logs and
screenshots are stored
locally for easy access
7
7
Results Generated After Execution
A snapshot of test result file in excel format.
A detailed log file will also be created.
Screenshots stored locally as well as in ALM.
For Illustration Only
8
Manual vs Automated Test Efforts
600.00
522.33
522.33
522.33
522.33
500.00
400.00
300.00
270.00
MANUAL HOURS PER CYCLE
270.00
TOTAL AUTOMATION HOURS PER CYCLE
200.00
100.00
70.5
5.50
7.50
7.50
CYCLE 2
CYCLE 3
CYCLE 4
7.5
7.5
0.00
CYCLE 1
CYCLE 5
CYCLE 6
9
Cumulative Savings
3000.00
2500.00
2000.00
CUMULATIVE MANUAL EFFORT
1500.00
CUMULATIVE AUTOMATED HRS
CUMULATIVE SAVINGS
1000.00
500.00
0.00
CYCLE 1
CYCLE 2
CYCLE 3
CYCLE 4
CYCLE 5
CYCLE 6
10
Benefits of Automation
• Automated results generated in Excel or HTML- not
dependence on the test software and format easy to analyze
• Manual tester released for new and on key features.
• Automating regression test allow frequent small changes
requests.
• Automation framework provides flexibility to change existing
SQL queries (for test data pull).
• Easy to customize test data case for changes in test scenario
• Huge savings in manual effort
• Screenshots captured traced sources of errors and easy to
debug software
11
Challenges for Automation
•
•
•
•
•
•
Higher initial cost
Higher skill set needed
More initial investment in infrastructure set-up
Availability of test data could be limited
Inconsistent application behavior
Application response times
12
Tools
• Type of Tools
•
Commercial Tools:
•
•
•
Support and maintenance advantage
High cost
Freeware:
•
•
No support and depending on support groups
More labor intensive-higher implementation cost
• Adnet-HEC In-house Expertise in
•
•
•
•
QTP
Rational Functional Tester
Test complete
Selenium
13
Contact Information
Shri Sinha
President
Adnet, Inc.
[email protected]
703-598-4485
14