Today we’ll be test talking with Paul Grossman, AKA the “Dark Arts Automation Engineer” about his five secrets of test automation. I’ve been following Paul for a long time on Linkedin, Facebook and other automation boards. He’s always posting useful test automation tips and tricks to help out the testing and automation community, so it was great to finally speak with him — and he did not disappoint. I think you’re really going to enjoy this episode. Check it out!
103: Five Test Automation Secrets with Paul Grossman
with Paul Grossman
About this episode
Quotes & Insights from this Test Talk
- When I’m looking at a framework I’m looking at a whole collection of really good ideas.
- Exploratory testing is done better with manual testers. Also automation is better at doing stuff that are repetitive and us humans we’re bored by doing the same thing over and over and over again. Sometimes we get tired, sometimes we skip steps and then we look and go, “Oh my gosh I just found a defect.” How did you find it? I skipped this 1 step here that’s what it was I can go back. I didn’t actually find a defect.
- Some defect are low level. I call it the Pony type of … the Pony level of defects basically you’re going to get a pony for Christmas before that defect gets fixed.
- We were tracking our metrics in Excel sheets. One metric was just the number of hours our tests ran, which was 35,000 hours from four machines over three years, including an endurance test that ran 24/7. Multiply that by maybe $40 per hour to pay a manual tester to do it and they would have spent $1.4 million dollars. We also found 7 show-stopper defects, including a memory leak that would have been difficult for manual testers to detect. Any one of those defects could have resulted in $100k in lost sales for the company. All told we saved the company at least $2.1 million with test automation over manual testing.
- Sometimes I think of this as the George Bailey approach. You have to imagine well if we didn’t have automation what would things be like? Then of course there’s the other question that comes out when we talk a little bit about money is our test automation tools. I’m going to exclude Selenium out of this, are they expensive? The first thing we think is, “Yeah there’s a lot of money for those tools." Let’s say there’s about $10,000 a pop for a license of those and that can sound really expensive. Then again you think about a manual tester and for $10,000 is going to show up for a whole year and do manual testing? Probably not, it’s just pretty … That’s below poverty level.
- I’d see LeanFT as a stepping stone between the UFT and full blown Selenium. I see that as a really great opportunity from HP and I can’t wait to start actually working on project with LeanFT.
Resources
- Alan Richardson‘s Selenium Java Course
- Boyd Patterson and his Test Design Studio tool for static code analysis of QTP/UFT code
- Linda Hayes: Automation Testing Handbook
- Adam Gensler: Simple Automation Framework for Remarkable Obvious Notes (SAFFRON)
- Wilson Mar: Automation and everything else in life
- Tarun Lalwani: KnowledgeInbox.com and three awesome books on QTP UFT.
- Dani Vainstien: www.advancedqtp.com Wrote “Fresher" Courses on QTP.
- Marcus Merrell & Will Roden: SoftwareInquisition.com – You can only find it in the Wayback Machine (aka InternetArchive.org
- Bryan Harrington: Code library at psacake.com.
- Lee Barnes: Conference speaker. Has five Rants on LinkedIn regarding test automation.
Connect with Paul
- LinkedIn: https://www.linkedin.com/in/pmgrossman
- Email: qtpmgrossman [@] gmail
- Twitter: @qtpmgrossman
- Training: http://www.testinginstitute.com/faculty.php
May I Ask You For a Favor?
Thanks again for listening to the show. If it has helped you in any way, shape or form, please share it using the social media buttons you see on the page.
Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.

Test Talks is sponsored by the fantastic folks at Sauce Labs. Try it for free today!
Keep listening
Related episodes

Automation Testing Podcast · Ep. 612
Robot Framework Test Automation: Automate Web, Desktop, Mobile, and Embedded Systems
What makes Robot Framework flexible enough to test medical devices, low code bots, monitoring systems, and embedded
Nov 17

Automation Testing Podcast · Ep. 611
Playwright at Scale: Run 500 Tests in Minutes with Oliver Stenbom
with Oliver Stenbom
Is your Playwright suite slowing you down instead of speeding you up? In this episode, Oliver Stenbom,
Nov 10

Automation Testing Podcast · Ep. 610
How to Prove Your Release Is Ready When AI Writes the Code
How to Prove Your Release Is Ready When AI Writes the Code? Can you prove your software is ready to ship, or does that answer still depend on a meeting and a spreadsheet? Want to see if your team can prove a release is ready? Book a custom LoopIQ demo with Ashwin: https://testgld.link/loopiqdemo In this episode,
Nov 3

Automation Testing Podcast · Ep. 609
Why Your AI Tests Pass When Your Features Fail with Donovan Brady
with Donovan Brady
Why Your AI Tests Pass When Your Features Fail? Donovan Brady of Leapwork shows how to make
Oct 27