Have you ever had the feeling that you’re running too many tests? Or redundant tests that are causing you to spend too much time maintaining an ever-growing test suite of what seems like wasted test cases? Does your test suite take a long time to run when all you want to do is make sure a small change you made to a specific area in your application didn’t affect anything else? And did it leave you wondering what combination of test cases would give you the verification coverage you need?
These are issues I struggle with, as well – which is why last year at HP Discover in London I was interested to learn about a model-based testing using a tool called Agile Designer. Ever since seeing a demo of the product done by Huw Price, I’ve been hoping to get him to appear on my podcast. I’m happy to announce that today, Huw Price shares with us his thoughts around model-based testing and test data management.
94: Huw Price: Modeling & Test Data Management Archeology
with Huw Price
VP at CA focusing on quality and acceleration · the entire software line cycle
About this episode
Quotes & Insights from this Test Talk
- We convert what is a deceptively simple model into, in effect, a set of test cases. The number of potential paths could be in the billions and trillions, and we can get it down into the tens and hundreds by using this mathematics. That's really the starting point of where we began with Agile Designer and agile modeling.
- The problem is we don't have time to model because we're all too busy doing our job, but then of course what happens is your job gets too busy because you're fixing bugs because you haven't modeled. It's the classic catch-22, so it's a tipping point. You have to design test cases. The tester is modeling. They are actually building a model which should, in theory, should reflect the requirements and the requirements is a model. It comes down to who's actually going to do that modeling. In my sprint team, because the Agile Designer team is a very efficient bunch of 22-year-old developers, and we run about 3-4 week sprints, and what we'll do is say, “Look, in this sprint we need to have done the model." I don't care who's done it. I don't care whether it's the tester, it could be me, think of me as the user, or it could be the developer, but at some stage or other we're going to have to create a pack of regression tests to test this.
- I think the thing then is the realities that things will change because someone will miss something and then everyone in the world has got to discover that themselves, whereas if we have this community, you could almost create a little forum around some of these common tests in the banking industry things like that, credit card checking and all that stuff. If someone finds that actually there is a specific … Was it the heartbeat or the heart bleed bug that went through? We actually modeled the test cases for that that were being used and we found the bug in about five minutes. Now, if we'd had a community of people looking at the model that would never have happened, would absolutely not have happened because the industry would have come together and said, “Whoops, we're missing this. We're absolutely missing this." I think it would just bring a degree of stability, and rigor, and reuse, and a community which seems to be lacking in the testing world really, apart from when we all get together at trade show and have a good time.
- The thing there is to think about synthetic data because production data is actually very thin. If you turn on code coverage tools, you only cover about 20% of your code with a production run, typically, whereas you should be in the 60%-80% code coverage. The easiest way to do that is to synthesize data, to create data with parameterization around it. We've got some very powerful synthetic data generation tools which will put data directly into a database. It will put it into a file, it will put it into an MQ message, but also I think, actually this is coming back to automation, is that quite often it's better to create an automation framework which puts data in through the front end of the system as well.
Resources
Connect with Huw Price
- Twitter: @datainventor
- Company: Grid-Tools
May I Ask You For a Favor?
Thanks again for listening to the show. If it has helped you in any way, shape or form, please share it using the social media buttons you see on the page.
Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.

Test Talks is sponsored by the fantastic folks at Sauce Labs. Try it for free today!
Keep listening
Related episodes

Automation Testing Podcast · Ep. 612
Robot Framework Test Automation: Automate Web, Desktop, Mobile, and Embedded Systems
What makes Robot Framework flexible enough to test medical devices, low code bots, monitoring systems, and embedded
Nov 17

Automation Testing Podcast · Ep. 611
Playwright at Scale: Run 500 Tests in Minutes with Oliver Stenbom
with Oliver Stenbom
Is your Playwright suite slowing you down instead of speeding you up? In this episode, Oliver Stenbom,
Nov 10

Automation Testing Podcast · Ep. 610
How to Prove Your Release Is Ready When AI Writes the Code
How to Prove Your Release Is Ready When AI Writes the Code? Can you prove your software is ready to ship, or does that answer still depend on a meeting and a spreadsheet? Want to see if your team can prove a release is ready? Book a custom LoopIQ demo with Ashwin: https://testgld.link/loopiqdemo In this episode,
Nov 3

Automation Testing Podcast · Ep. 609
Why Your AI Tests Pass When Your Features Fail with Donovan Brady
with Donovan Brady
Why Your AI Tests Pass When Your Features Fail? Donovan Brady of Leapwork shows how to make
Oct 27