On today’s show we’ll be Test Talking about keyword-based automation (also known as action-based testing) with Hans Buwalda, CTO of LogiGear and a pioneer in action-based methodologies of testing and automation.
Hans will share how using action-based testing practices like modularization and keywords can make your tests easier to create, automate and maintain, as well as simpler to understand.
About Hans Buwalda
Hans Buwalda is an internationally recognized expert in test development and testing technology management and a pioneer of keyword-driven test automation. He was the first to present this approach, which is now widely used throughout the testing industry. Originally from The Netherlands, Hans now lives and works in California as CTO of LogiGear Corporation, directing the development of what has become the successful Action Based Testing™ methodology for test automation and its supporting TestArchitect toolset. Prior to joining LogiGear, Hans served as project director at CMG (now CGI) in the Netherlands. He is co-author of Integrated Test Design and Automation and a frequent speaker at international conferences.
Quotes & Insights from this Test Talk
- In your editor is a spreadsheet, like interface, so it is not Excel, but it works like a spreadsheet. You write your test like a sequence of actions. You have an action login, then arguments, username and password, and then you have another action, and you might have a customer, and an article, how many, how much it will cost. So you make your test as a narrative from action to action. Columns are not fixed, some actions might have only one argument, or even zero arguments, while other actions might have ten arguments, or something like that.
- The repeatable part, the reusable part, is in the keyword. If you have a keyword like “login” and the actual things that you do to the system in the test might be entering a username, entering a password in the fields that you have, to push an OK button – so there is a couple of steps that you encapsulate into that action. The result of that is that these actions become very reusable, and also very maintainable. The better you succeed in encapsulating details, and the less impact changes of the application, the test will have on the test. You have to layout login screen changes, or you need to enter another field, and the impact of that on the existing test will be fairly limited
- Some projects do it one way, some projects do it another way; it also depends highly on the skills of the tester. Some testers are engineers themselves, so they don't have any problem creating the actions and implementing them. While in other projects, the testers might not be that technical; they might be functional experts, or domain experts, or professional testers who know a lot about testing, but not necessarily know a lot about technology. In that case, what I recommend people to do is have the testers create the test, and make up the actions, make up the keywords that they want to use, and then once you have these actions, you go to an automation engineer, and sometimes we call that a navigator, and then that person can implement the actions. You want to avoid people having to use predefined actions. You really want it to be part of the creative process of test creation.
- What you also see is that the companies talk about open-source or commercial tools, they only look at the brand of the tool. A large part of that, of course, is our problem. It is a commercial tool, but the problem is they don't count the cost of ownership – that's basically the number you should look at. If you have to hire one more person because that person needs to maintain that framework, you can buy a whole lot of licenses from a commercial tool, for the price that one person costs.
- Yes, a bad rap. I would say that is so. The worst projects I've seen in automation were keyword projects. It can go horribly wrong, and that's why we have action-based testing and the method part is not so much just about keywords, but it's really about which keywords, and what to hide and what to show. What we do is we have a modularization. That means what you design are not test cases but test modules, and then each test module has a very specific and well differentiated scope that is then worked out in the rest of the test module. Within the test module, you will have test cases, but you start identifying the modules first. It's like writing a book. When you write a book, the hardest part is always the Table of Contents. You have to identify the chapters your book is going to have. Once you have decided that, of course writing the book is a lot more work, but it is less difficult, assuming that you know what you're writing about. It's just a matter of production, but getting that initial structure is the hard part, but it is also the part that will give you the most maintainability. That whole maintainability advantage of keywords won't work if you don't have a good design of your test in the form of those test modules.
- The main message is: take it seriously. Don't see automation, or even testing, as a byproduct or a chore that you have to do. The tests are almost more important than the application that they test. Maybe that's another sidestep I can make here – I also believe strongly in testability. When you start a new system, let's assume you want to develop a new app, or a new TestOps application or anything, your first question should not be “What am I going to develop?” Your first questions should be “How do I test it? What do I need to test? How am I going automate those tests? What are those tests going to look like?” Because that also gives you direction on the testability, and testability should be the highest priority in the project. With testability, I mean is the system well-designed so that it is easy to test? For example, controls on the dialogues – are they well-identified with hidden identifiers that the user can't see? And hooks for timing, so you know when you populate a table, when it's done? And more of those kinds of examples: Is the Y box access, I've seen the pie chart, can I insert an API detector to see what the actual values are without having to interpret an image?
Resources
- Automation Guild 2017 Online Conference
- TestArchitect
- Techwell Stories
- Keyword driven development article
Connect with Hans Buwalda
- Twitter:@hansbuwalda
- Blog: happytester
- Company: LogiGear
May I Ask You For a Favor?
Thanks again for listening to the show. If it has helped you in any way, shape or form, please share it using the social media buttons you see on the page.
Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.
Test Talks is sponsored by the fantastic folks at Sauce Labs. Try it for free today!
