When creating automated tests, rarely will you find that it’s just a matter of creating a static script. The software under test is always changing, which in turn causes the test script to change as well.
Most tester-engineers spend the bulk of their time maintaining existing automated test.
That might be fine for small projects, but it doesn’t scale well. Especially when working at a large company on enterprise software that could require tens of thousands of automated tests.
In an environment such as this, how do you rapidly change the test, maintaining it and updating it so that you can scale quickly along with your application changes?
That’s the main problem TestArchitect was designed to handle.
Their number one goal was to develop a way to easily create a test with a minimum requirement on coding, while simultaneously creating a powerful way to automate that isn’t a dumbed-down test automation tool.
How do they do it? By using a time-tested, keyword-driven approach.
Keyword-driven Test Automation approach
A keyword-driven approach means that you write a test not with scripting (using a scripting language like JavaScript or any other scripting language), but as a sequence of keywords.
Test Architect utilizes a spreadsheet-like interface to write your test using keywords. For example, if you were testing Amazon, you would have a login that contains a username and password argument. You would have a Search action, Add to Cart, and Buy actions. You can pass data between actions, and can also easily create data-driven scenarios.
What’s great about this approach is that you are creating reusable assets for other test flows that may make use of your existing actions.
As a result, these actions become reusable and also very maintainable. The better you succeed in encapsulating details, the less impact changes to the application under test will have on all your tests.
So if the layout of a login screen changes, or you need to enter another field and you have, say, 2,000 tests that use that login keyword, the impact on the existing test will be very limited.
Creating a keyword-driven test is almost like playing with Legos. You take existing actions and change them into a new action. By chaining those actions together, even a non-technical person can easily create a new automated test.
Meanwhile, more technical folks can also perform a lower level of script development using programming languages like Java, C# or Python.
Is a keyword-driven approach a good idea?
As with any testing methodology, there can be good implementation and bad implementation. To help ensure you create a healthy test approach, Test Architect uses a method called Action Based Testing (ABT).
ABT uses modularization, which means that you first design test “modules” before even creating a test case. Each module has a very specific and well-differentiated scope that is then worked out in the rest of the test module. Each test module will contain test cases.
Creating keyword-driven tests is like writing a book
I actually had the privilege to interview Hans Buwalda, CTO of LogiGear and a pioneer in action-based methodologies of testing and automation.
When writing a book, the hardest part is always the Table of Contents, in which you must identify the chapters your book is going to have. Once you’ve decided that, writing the book becomes less difficult. The initial structure is the hard part, but it’s also the part that will give you the most maintainability.
So the whole maintainability advantage of keywords won't work if you don't have a good test design in the form of test modules.
But why all the focus on test design?
The Challenge of Test Design
Test automation is commonly regarded as a technical challenge.
The more I think about, though, especially after speaking with Hans, I don't believe it is a technical challenge. I think it’s a test design challenge.
Here’s why.
Ninety percent or more of the success of your automation is defined by how well you designed the test. A common issue with test design is that testers are usually the ones creating the tests, and testers are not necessarily expert designers.
The main thrust of action-based testing is around making the test design automation friendly, maintainable and easy to scale for large enterprises.
[tweet_box design=”default”] 90% or more of the success of your #automationtesting is defined by how well you designed the test. @hansbuwalda[/tweet_box]
Focus on the tests, not Framework Development
A pet peeve of mine is when test engineers create automation frameworks from scratch. It drives me crazy because they could have easily made use of existing open-source tools and libraries that would have met their needs without writing any code — in most cases with better results.
Furthermore, why would you spend time and money creating custom code for functionality like parallel test execution, test reports, dashboards, screenshot recordings, test management, and the ability to run against different platforms – all of which come with Test Architect?
Another benefit is that it comes out of the box with over 400 actions (including native support for multiple technologies) that allow you to start creating tests rather than developer code for all the different actions.
Is Test Architect a competitor of Selenium?
I think almost everyone would agree that at the time of this writing, Selenium is the reigning automation tool/API of choice. With it becoming a W3C standard I think its dominance will grow even more.
So is Test Architect a competitor of Selenium?
Absolutely not!
Test Architect uses Selenium WebDriver internally, but if for some reason Selenium can’t handle something TA can switch back to what they call Reflection, which gives you more control.
Another difference is that Test Architect offers more options, along with built-in functionality like script running and reporting that you don’t get with Selenium (unless you code it up yourself).
Also, Selenium is for browser automation only. If you work for a large enterprise, chances are you need to automate other technologies like ERP and SAP applications. Test Architect can handle those technologies and allows you to create end-to-end workflows between them.
Keyword approach in Agile
The Keyword-driven approach has been around for a while, but that doesn’t mean it’s not adaptable to newer software development practices like Agile. In fact, you’ll reap some key benefits of good software practices when using keywords in an Agile testing project.
How this might work on an Agile team is early on in a project you might sit down with developers, testers, automation engineers, product owners and/or domain experts and you and your team then define the areas that need to be tested.
You can do this before the sprint development begins and once it does you can then start to fill out the test modules with actual tests. So it’s a very iterative approach that fits in nicely within an Agile sprint/scrum team.
Since the test modules have a clear scope, it also becomes a lot easier for non- technical testers to stick to the plan and create automated tests on their own.
You can also easily integrate TA into your DevOps pipelines.
I also interviewed Hung Nguyen co-founded LogiGear on TestTalks and he mentioned a lot of other features that Test Architect contains:
Who can benefit from this approach?
Back in 1999, I read Software Test Automation – Effective Use of Test Execution Tools, which has a chapter by Hans Buwalda on Testing with Action Words.
After checking out Test Architect, I can see Hans’ fingerprints all over its design. I can also tell how much refinement he has made with this approach in the almost two decades since.
It seems everyone is chasing after what is shiny and new these days, but I believe many teams would benefit significantly from Test Architect. If you’re dealing with large test suites across multiple tech stacks and need to test things like SAP and Oracle Enterprise Applications, you should give LogiGear’s Test Architect a serious look.
Give it a try and let me know what you think.
