If you receive errors when attempting to view this white paper, please install the latest version of
Adobe Reader.
"Worksoft is a leading provider of testing solutions that automate, accelerate and optimize deployments of SAP® applications. The company's software and services validate and certify end to end business processes across the enterprise during implementations, upgrades, customizations, or migrations of SAP applications."
Source : Worksoft
Evolution to Revolution: The Test Automation Maturity Curve
Automation Maturity is also known as :
Automation Maturity,
Automation Processes,
Marketing Automation Maturity Assessment,
Evolution to Revolution Test Automation,
Empiricism Theory Automation,
Process Automation Technologies,
Control and Automation Maturity,
Correct Level of Automation,
Integration Maturity Model,
Automation Processes and Systems Maturity,
Take Sample Automation Maturity,
Review Automation Fundamentals,
Main Automation Maturity Levels,
Basic Level of Automation Maturity,
Test Automation Maturity Efforts,
Enterprise Continuous Integration Maturity Model,
Scoring the Maturity of Organizations Automation,
Release Automation Maturity Survey,
Data Centre Automation,
Test Management Automation,
Advancing in Automation Maturity,
Automation Maturity Model,
Phase Brings Automation,
Build Release Automation Maturity,
Automation Maturity Framework,
Choosing a Test Automation,
Describes Automation Maturity,
Datacenter Automation Accelerating,
Program for Implementing Automation,
Test Automation Maturity Strategy,
Increasing Automation Maturity,
Automation Maturity Example-based,
Customer Requires Automation Testing,
Automation of Thought Information Technology,
Warehouse Automation Maturity,
Warehouse Automation Project.
Introduction
Test automation has been evolving over the past two decades from primitive record/playback to data-driven scripts and then to
frameworks. This trend reflects a preference for less and less code in favor of more and more data, because less code means less
development and maintenance costs, and because data can be created and managed without coding skills.
The founders of Worksoft also founded AutoTester, the first PC-based
test automation tool, over twenty years ago and they experienced
this evolution with their earliest customers.What they realized was
that the trajectory of this curve meant that the end point would
be little or no code and all data. As a result, they founded Worksoft
and started from scratch to develop an entirely new automation
approach that focused on defining and managing test assets as data in a
relational repository instead of as code in script files.
This white paper explains the evolutionary trend and how to make the
revolutionary leap from code to data, which completely changes
the economics and dynamics of test automation by radically reducing
development and maintenance costs while empowering all
stakeholders in quality to contribute to test automation.
Record/Playback
In the beginning, record/playback test automation was seductive.The
idea was simple: just do the test as usual but record it into a script
file using a tool, then play the test back as needed. It looked fast
and easy and catapulted the test automation industry into high growth.
Unfortunately it was also deceptive.These scripts were inherently
unstable: the smallest change or error in the application or the data
could cause the test to break, and even playback timing was unreliable
as systems ran slower or faster from time to time. Since the scripts
were not structured or documented, they were almost impossible to
maintain.
For example, the recorded script below has both the steps and the data
embedded into the test, so that if a hundred accounts were added
as part of the test, the same steps would be repeated hundreds of
times. Later, when changes were made to the window, they too would
have to be made hundreds of times.
# Add Account
set_window ("Account Manager", 15);
edit_set ("txtName", "Worksoft");
obj_type ("txtName, "");
edit_set ("txtAddress", "123 Main Street");
obj_type ("txtName, "");
edit_set ("txtCity", "Dallas");
list_select_item ("cbState", "Texas"); #Item Number 42;
edit_set ("Edit", "Texas");
edit_set ("txtZip", "99999");
obj_type ("txtZip", "");
button_press ("Save_2");
And while these issues could be resolved by externalizing data, adding logic and error handling, this meant writing code.
Data-Driven
The data-driven approach emerged as a result. Users either shelved
the tools or recruited programmers to form automation teams who
were comfortable coding scripts.This meant externalizing the data into
spreadsheets or text files and adding logic to process the data,
handle timing synchronization, error handling and recovery as well as
result logging and reporting.
And while this approach succeeded in allowing analysts to contribute
test data and reducing the volume of code, it increased the
complexity of the code that had to be developed and maintained.
Eventually companies realized that test automation code is no
different than application software code: it is painful if not
impossible for one programmer to maintain code written by another.Thus,
script code had to be maintained not only when the application changed,
but often rewritten when there was staff turnover.
The pressure continued to reduce code and expand involvement.
Key/Action Word Frameworks
The next logical step was to allow users to sequence the scripts
along with the test data and frameworks were born.The strategy was
to divide the test team into automation engineers and test
designers.The engineers would write reusable components called "action
words"
or "key words" to perform specific testing tasks that the test
designers could call and pass data values into using spreadsheets.The
entire
process was organized within a framework that processed the
spreadsheets, invoked the scripts, passed in the data and provided
reporting.
Below is an example of a key/action word test, where the Test Case
column contains a list of keywords and the following columns
provide the data.
In this approach, the keyword or action word is written in a
scripting language and is specific to the application under test. In
some
cases, the script code is generated automatically from building blocks
that are then enhanced with additional code to perform
decision-making and other logic tasks.
In the example above, "add product" and "check product" are the
names of scripts that contain the detailed level code for adding a
product to the Inventory application being tested. Below is an example
excerpt of how that key or action word might be expressed:
# Add Account
set_window ("Inventory Manager", iTimeOut);
if (obj_exists "txtProduct != E_OK)
{
gsStepStatus = "Error: Product name object does not exist";
return;
}
status = edit_set(sProductName);
}
if (status != E_OK)
{
gsStepStatus = "Error: Product Name could not be input";
}
type(""0;
Key/action word frameworks succeeded in allowing non-coders to
design and execute automated tests more easily, but still required
custom code.And, although test designers could write linear test cases
more easily, they could not make decisions and control the flow
of the test based on real-time execution results.That still required
code.
Oddly, Mercury Interactive described their keyword framework released in late 2004 as "groundbreaking" when in fact the concept
was over a decade old when they incorporated it.
Although Mercury QTP 8.2 allows users to build tests, add
verification points and recovery options using a GUI, code is still
generated.
This code can be seen by simply pressing the Expert View tab.
In the end, your test is still a program written in script code, and
that's what you execute, debug and maintain.As a Mercury keyword
framework customer recently observed in a presentation to a local
quality assurance chapter meeting, "If this approach is code-free,
when are all of these coders finally going to finish and go home?"
And that's the bottom line. Key and action words are just names for
code modules, and as long as your test assets are stored in code,
you've got challenges. Custom code is costly and time-consuming to
develop and maintain.
Companies that adopted these techniques ten years ago have already hit the wall and are moving to new approaches.
Certify: A Revolutionary Leap
Because the founders of Worksoft started from scratch, they did not
try to develop yet another way of writing script code. Instead,
they looked for a way of eliminating it entirely.
A model already existed for this transition. In the past, almost all
business applications were custom. Every company had to write its
own software simply because none already existed. They, too, sought to
reduce code and maintenance by evolving from procedural
code to object-oriented architectures and reusable components.
Eventually, off the shelf applications became available that allowed
companies to configure the functionality without changing the
code. Users realized that it made economic sense to develop only the
code around their core competence that increased revenue, and
buy the systems that provided common capabilities that reduced costs.
This allowed them to devote their development resources to
create competitive advantages.
In other words, build what makes money and buy what saves money.
And that's exactly what Worksoft Certify is all about. By providing
a commercially supported framework that is configured for each
application using data, it enables customers to reallocate scarce
development resources from developing and maintaining custom test code
to
focusing on strategic IT imperatives. Its enterprise class design
empowers domain experts to effectively participate in assuring the
quality
of those systems that deliver competitive advantages by designing tests
to verify business processes that span applications and platforms.
The net effect of changing from a test tool to an application and from
code into data is an orders of magnitude improvement in usability
and maintainability. Using Certify does not require programming skills,
and thousands of tests in a database can be updated with a
single click, unlike chasing code changes through thousands of script
files or spreadsheets. When you build a test, you are simply
adding data to a database. No code is created, generated or maintained.
The only code you need comes already included: each platform has a
class library with all related actions available for it.Worksoft
develops and supports it so you don't have to. If you need a class or
action not already included, you can add it yourself using our
open API or you can have Worksoft add it for you. Either way, the
action is reusable across any applications that use the same class.
You can even write classes and actions that are reusable across
platforms, or add your own application-specific functionality if
desired.
So Certify offers all the advantages of key/action words without the
drawbacks. Both technical and non-technical users alike can develop
automated processes rapidly using simple point and click to select
windows, objects and steps against them.
Further, test designers can make decisions at each individual step
within a Certify Process, allowing them to make tests responsive to
conditions or results during execution so tests are more intelligent
and robust and can recover from errors or other unexpected
conditions. Certify also includes standard state management and error
recovery logic as well as detailed results logging and reporting.
All Certify assets"including test data"are stored in a relational,
shared database. No spreadsheets or text files have to be managed,
everything is in a single place and can be managed centrally. Test data
can also be shared across business processes instead of
embedded within a test flow.
Storing all test assets in a central repository also means that when
the application changes, Certify automatically performs impact
analysis and updates all affected test assets with a single click; no
code changes are needed. And, as an enterprise class repository,
Certify integrates test management capability for all test assets;
there are no separate data or code files to manage
One of the most powerful benefits of the Certify approach is that by
minimizing or removing the need for custom code altogether,
test engineers can focus their efforts on automating more applications.
Instead of multiple engineers supporting one application,
Certify allows one engineer to support multiple applications.This
increases the return on investment to the company and maximizes
the penetration of automation throughout the enterprise.
Summary
The evolution of test automation tools from code-based
record/playback to data-driven and key/action word frameworks reflects
the
realization that the less code there is to be developed and maintained
the more efficient and effective test automation becomes.
Instead of taking twenty years to evolve through these stages, you can
take a revolutionary leap with Certify, the truly code-free approach
reduces the time and cost to implement, manage and maintain automated
tests.