About This Episode:
Paul is back to share some automation testing tricks and treats in our annual Halloween special. In this episode, Paul Grossman (AKA the DarkArtsWizards) conjures up some automation testing practices that will help ward off flaky automation like garlic to a vampire. Discover automation horror stories, whether codeless automation is the boogeyman, how to avoid zombie bad testing practices, and much more. Don’t be left in the dark — listen now!
The Test Guild Automation Podcast is sponsored by the fantastic folks at Sauce Labs. Try it for free today!
About Paul Grossman
Paul Grossman has been delivering hybrid Test Automation framework solutions for two decades in UFT, Selenium, WebDriverIO and testRigor. He currently is the Lead Quality Assurance Engineer for Guaranteed Rate (Go White Sox!). He continues to speak at Conference around the globe – virtually – and always with a live demo. He most recently was the GameMaster of the 2021 Bug Bash hosted by the British Computer Society with his sandbox website Candymapper.com
Connect with Paul Grossman
Full Transcript Paul Grossman
[00:00:01] Intro Welcome to the Test Guild Automation Podcast, where we all get together to learn more about automation and software testing with your host, Joe Colantonio.
[00:00:16] Joe Colantonio Hey, it's Joe, and welcome to another episode of the Test Guild Automation Podcast. Today, we'll be talking with Paul Grossman, the Dark Arts Wizard, all about automation and what has become a tradition here at the Test Guild for a Halloween special. If you don't know, Paul has been delivering hybrid test automation framework solutions for two decades using UFT, Selenium, WebDriverIO, and he's a big fan of a newer tool called TestRigor. He currently now is the lead quality assurance engineer for Guaranteed Rate Go White Sox, and he continues to speak of conferences around the globe virtually and before, it was all on-site as well. He is always during a live demo, and he most recently was the game master of the 2021 Bug Bash hosted by the British Computer Society, with the sandbox website called candymapper. You don't want to miss this episode. Check it out!
[00:01:08] The Test Guild Automation Podcast is sponsored by the fantastic folks at SauceLabs. Their cloud-based test platform helps ensure you can develop with confidence at every step, from code to deployment for every framework, browser, OS, mobile device, and API. Get a free trial. Visit testguildcom.kinsta.cloud/saucelabs and click on the exclusive sponsor's section to try it for free today. Check it out.
[00:01:36] Joe Colantonio Hey, Paul, welcome back to the Guild.
[00:01:42] Paul Grossman Thank you, Joe. Oh! The music. I love it, totally sets the mood for the Halloween episode, and by the way, I love your costume, that kind of superhero one that you've got on there with the yellow cape and everything. Great, great design.
[00:01:57] Joe Colantonio Awesome. Awesome. Good to have you back, Paul. So I'm just curious to know, you know was anything I missed in your bio that you want the guild to know more about, it's been a while since the last time we spoke at probably the last Halloween special.
[00:02:44] Joe Colantonio Nice. I thought they killed VBScript off. Years ago.
[00:02:50] Paul Grossman They, nice. Still have it there, UFT. I think it's the developer and there's UFT. There are two different flavors. So once the original one that everyone's kind of knows and loves, that's the combination of QTP and service test. And then they have UFT developer, which is working and a bunch of different languages, other languages. And also you get to use, you know, your favorite. If you're working in Eclipse or IntelliJ Visual Studio, you get to use that IDEA in order to do some stuff. Very similar in designs and the element locations on those tools.
[00:03:23] Joe Colantonio Nice. So I think one thing when people hear about tools like UFT and especially now even more now. So when they hear things have low code, no code type solutions. They start screaming, and this sends a lot of chill down their spines because they're like, you know, low code solution? Is that even make sense? That's kind of scary. Should I be afraid? So what would you think about low code solutions or no-code solutions? Is it just a term or is it actually something now we started 20 years ago, is it something actually is something that's actually viable now?
[00:05:49] Joe Colantonio Nice. So how is that different, though? Like WinRunner had an object repository map or object map, and UFT had the same thing. So if an element changes, just go in the map and you change it. But also they also had like keyword views where you do a dropdown, you select, how of these newer tools different than the nodes, you know, the ones that set the stage back in the day.
[00:06:08] Paul Grossman Well, They still do have that dropdown where you drag stuff in, and a lot of tools do actually have that, that interface. What we've got today is we've got the ability to just write, write out in plain English, and I know we probably get at this point talk a little bit about you know, everybody some time a Cucumber and Gherkin stuff like that. But we're actually looking at expanding it out, not even being limited down to I have to write this word given when then and then I have to write out some sort of extrapolated layer that says, OK, given that I click on this button and I get it all at all, that just letting the tool actually identify what your intention is, I want to go click on something or verify the page contains and is split right in plain English. No more data-driven, no more data. I know I have a bunch of excel sheets that had descriptors and other methods. I was trying to support all of its more condensed. It does take a lot less time in order to run, I should say write and run test cases like that. So, yeah, the new tools are basically condensing down our time and making it a lot more easier. It also gives leverages our, our manual testers to kind of get into a gray area in between. They're not quite invested, but they're no longer manual testers. There's something in between and giving a really good help to getting test regression testing done very quickly for the team.
[00:07:38] Joe Colantonio So I think one thing that might creep people out is they fall in love with tools. And so someone may love UFT, some love Selenium. Today's type of solution to think going to replace these long-standing tools are industry standards that we have now?
[00:07:51] Paul Grossman No. We've got way too much invested in those code development tools. In fact, it's very similar to what I'm having 20 years ago, when I first became an automation engineer, I scared the heck out of my manual testers. They were not my friends anymore. They thought I was trying to get them out of a job. The quick solution to that was I wrote a script that click the button. We call it the NFG test, the new freaking guy test. And while everybody went to lunch, he had hit the button for an hour. I automated that, and then we all went to lunch together. We all became good friends. In this case, when we're looking at low code, no code tools, what scares now is the s-test because they think I'm going to be automated by a tool that's so easy they don't need my techniques, which is simply not the case. What I'm looking at is trying to do some, I guess I call a Venn Diagram Automation. We have two tools, one that has the s-test developing and whatever tools they happen to be using UFT or WebDriverIO, or whatever their flavor is. And then we're complementing it, with low code tools that allow our manual testers to get up to speed very quickly. So if you nest that, don't worry, you're still secure. We still need you. Embrace the whole thing and make it a whole big one happy family. We're going to get to where we need to very quickly when it's combined with that.
[00:09:12] Joe Colantonio Do you find that causes more complexity, though? A lot of times people like, Oh, we have one tool, one solution that everyone uses and we only really need a team one framework. But now if you have multiple frameworks, multiple people doing different kinds of testing, how do you pull it all together and make sure that it's all it doesn't give you chills at night?
[00:09:31] Paul Grossman Well, one way you can do that is making sure that if you're taking two tools, by the way, I understand some people are sitting there thinking, “Oh great, I'm going to have to have three or four or five tools.” I have a limit of two tools that I work with to balance that out. And the important thing is that they both are able to report to the same central reporting location. So if we're reporting out to what's our most popular recording tool these days?
[00:09:54] Joe Colantonio Jira?
[00:09:54] Paul Grossman Yeah. If we go out to Jira, then both tools should be reported to Jira. And then at the top level, everyone's looking at it and it doesn't all look all that different. They both are generating screen captures. Both are putting up information as to what's being failed, and everybody just gets an email that says, OK, check this out. Something's changed from this, from the last run to the current run. And I don't think it really matters which tool that you're using or to get that at that level.
[00:10:22] Joe Colantonio So being in the Halloween special, any horror stories you can share around these tools or your experience with tools, I know you said you give one example of how a team didn't like you because you were their Automation guy. So any other horror stories like that?
[00:12:36] Joe Colantonio So what's the key takeaway from there? So you wouldn't have to perform an exorcism in the future? Would you avoid fancy functionality that may not be, you know.
[00:12:47] Paul Grossman Well.
[00:12:48] Joe Colantonio Native to the tool?
[00:12:51] Paul Grossman That's interesting, and it's a good lead into an alternative tool like low code tools where all of that has been handled inside of the framework internally and you don't get to see or experience all that horror. It's the guys who develop the tools that have to deal with that quite a bit. So that's one of the advantages I see is that I don't have to if I'm working with the tool, let's say, well, recently I've worked with TestRigor that's all handled behind the scenes. I don't even think there's any integration with fibers in the async/awaits. I'm not privy to the background on how Artem Korolev does that, but I don't have to deal with that, that sort of level of stuff. And also, you don't have to extrapolate a whole another layer of automation goodness in order to support like what exactly what kind of business process I'm doing. We can describe them out in plain English. And then one of the other cool things about low code tools is people say, “Well, you know, if there's a record tool that's in there, then you've got to record and playback.” And nobody likes that. And I agree. You don't want to build your test cases with record and playback, but you do want to use a record tool in order to learn the syntax, learn how things are built, and how the tool is used. And a tool like TestRigor, what they've got is something called reusable rules and it's basically modular design. You're making business components and the components say, “Okay, put in the username and the password and click on log in. And if any of that particular component changes that's used by 100 test cases, you have one singular place that you make that change. You know, we all do that exact same thing in coding, whether working UFT or WebDriverIO, or whatever other tools. It's just a good thing that we learn as developers and we just have to bring our bringing that idea over to manual testers and they understand and they can make their test cases fairly quick and robust.
[00:14:49] Joe Colantonio So it seems like a lot of these newer tools, really, they reanimated all the functionality, from Mercury almost because they have business process testing, they had these components and you changed one component with update everything. But the thing of a slow so is the new tools that they do it better? Is it different? Is it using AI? Or, you know, another buzzword machine learning behind the scenes to help where maybe those tools that were the forerunners maybe didn't live up to the hype.
[00:15:17] Paul Grossman Yeah, they are using AI in there. In fact, there's one of the things that I really like. There's no, two I wanted to give a shout out to, which is Applitool's eyes. You know, we've talked about for years, every tool out there has some sort of way of doing visual identification and images, and in my personal opinion, almost every tool, it's been a sales pitch. It's like, Hey, we can do this, you know, do a really great, but we can do this. Applitool's eyes are basically the only tool out there that has really nailed it down. Angie Jones has a wonderful demonstration on her tool and her team out there to show exactly how that works, and it's something that's really come full circle and, and we've really been robust. The same thing with you, with low code tools, said it used to be OK. It's an interesting little thing, but it's, it's hard to do. I think today we're really at the point where we can really leverage it really well.
[00:16:48] Paul Grossman Well, I will say I'm the only guy out there who brags that I made my mortgage payment for 16 years working in basic that I learned in the 80s back in the back in college. And I also sometimes will poke a bear every once in a while. When a developer, I'll say I'd like to try and get something done. And then the developers also say, well, I don't think we can do that in Java. I'm like, well, I just did it here in the basics of you tell me there's something that Java can't do that basic does? And then they get mad and they go off and do it just out of spite. Yeah, it does work. I figured it out. All right. Let's see what. Yeah, I want to get back to your point. I'm trying to think of your question and try and see a better way of answering that.
[00:17:45] Paul Grossman Well, one of the things that you talk about, so you don't necessarily have to select Java as your programing language, and sometimes it's selected for the wrong reason. And the reason is is that it's the idea of if you only have a hammer, everything looks like a nail. If you only have Java developers and you ask them, What language should we do our automation? And they'd say, Oh, I'll do it in Java, because once it falls to us, we'll be able to handle it. But that's really kind of a fallacy. I have no intention of having my s-test let the project fall to the developers because they don't have time to take care of all of that anyway. You know, you ask them how much unit testing have they done? Oh, yeah, we did some. So why would it go to them? The idea is that, you know, it really doesn't matter what language you program in, but consider using the languages you know easier. That doesn't have such a large footprint as Java, because as s-test every day we are going to be spending either time writing test cases or debugging and extending our framework. Why use the language that has so many more characters and so many more intricacies to build and spend your time doing that rather than building more test cases? I don't see the advantage of that, so I do recommend looking at different languages that are available out there. Java is just isn't. It isn't the only thing that's out there. Selenium supports, I think, six different languages, at least out there. So I do consider that.
[00:19:08] Joe Colantonio It's a good point, Paul, a lot of things that people are saying, you have to use the same language as your developers, I don't necessarily agree with that. Another thing I think people kind of get creepy about is that they think they need to use a tool specific to the technology that's being used to develop the applications say, like React. So I need to use a tool that is built for React. Is that the keys are to a how have you found in your experience when choosing a tool? Does it always have to match the architecture that's been used to develop the application? Does it matter? Is it like a language you use which one of? Is best for your team? How does that work?
[00:19:42] Paul Grossman Yeah. Well, you definitely need to take a look at tools that are going to support multiple architectures because, along the way, someone's going to make a decision and say, I don't like React, let's go with Vue. Let's change over the closure. You have to be a little bit more open to understanding like, OK, let's just look at the worst-case scenario. The whole underlying architecture changes. How much is that going to affect my automation framework? And that's another good thing where low code tools come in handy. And I do recommend anyone go out there and go and check out a test, you know, a tool that you're looking at for PLC against websites that design in React and Vue and, and have a front end or enclosure or whatever else and other architectures that are coming along and just test and see if you can write a script, you know, in just a couple of minutes to see, does it work under each different architecture that's out there? I've done that with TestRigor. I've got run it up against a Vue, I've run it up against the architectures that you just mentioned, and there is a website out there which I will get shortly, unquote. We'll quote it in the show notes that actually has samples out there. I know a lot of people like to use the, I think the Heroku, the internet, which is okay, but there is another website out there that will actually have samples of buttons and radio buttons and stuff and tables, which is really cool. That will you can actually test against with your automation tool, just to be sure you know how much pain you're going to have should the underlying architecture just completely change underneath you.
[00:21:14] Joe Colantonio Is this your candy mapper application?
[00:21:16] Paul Grossman Oh, CandyMapper is actually just Sandbox out there that is used for, well, to see if you test your own skills. There are two versions, there's candymapper.com and there's Candy Mapper r2 released 2. Candy Mapper r2 which is the website that goes right and then candy mappers website that goes wrong, all sorts of weird, unconnected stuff. It's more like reality. You know, how can you actually pick up that a link doesn't take you to where you think the link is going to go? So what I would do is I'll go to telerik.com they have front-end elements that you can sample, and I like going in there and just jumping into all sorts of different stuff. They've got to can tho react button and a bunch of other different applications that they interact with, such as jQuery, Angular React, Vue, and I think TestRigor, I've been running against all those different flavors. It doesn't matter. It likes them all and interacts with them in the same way. I don't have to change anything in my scripts.
[00:22:15] Joe Colantonio So Paul, one thing, we have an address as the boogeyman or the grim reaper in the room, and that is paid solutions. It seems like a lot of these no code, low code are paid solutions, and a lot of people always Oh, I only do open source, and because it's free and therefore I see my company tons of money. Is this the case or is this just some sort of undead type thinking?
[00:22:36] Paul Grossman I think when you're looking at open source, you might be spending money elsewhere on that. Yeah, you might be saving money on the tools. They're open source. But one thing I do warn people about is that you might be depending on the kindness of strangers for your support. Since there's really no money coming into an open-source tool, nobody's getting paid all that much. So if there's something that needs to be fixed, well, you don't know where in the layer of that, the developer is going to decide, you know, I'm going to fix it this afternoon, or am I going to fix it next year when you have paid tools? I mean, they don't want to lose the revenue income stream, so they're going to bend over backward to try and make sure the tool is useful for you. The other thing is that if you're using open source tools and you have to work in Java, you may be taking more time in order to extend out and write your language, write your scripts and debug that. And you know, that's just it's slower going than if you've got something you know. Like I say UFT, we know that's a paid tool, but the language is so much easier that you're going to save money on the far end where you're not having a nest that spends a whole large amount of time to create your scripts because of the language footprint itself. It's so much smaller on that, and I'm sure there's other there's a lot of other time savings involved in that, but those are the two major ones.
[00:24:00] Joe Colantonio Absolutely, and Paul, you know, you mentioned TestRigor a few times and we mention, I talked to you in the pre-show. You don't work for TestRigor. You don't get a dime from TestRigor.
[00:24:07] Paul Grossman I don't.
[00:24:08] Joe Colantonio So just curious to know what is the deal with TestRigor? What kind of tricks and treats does it give you that you're so jacked up about like you're on sugar candy high that you're always talking about it?
[00:24:19] Paul Grossman Well, I may mention that I last broadcasted, I became convinced that there was a good tool in what was attempting to do mostly because I've been trying to do this magic object model stuff for about five years. We sat down, I said I've done with the, with Artem for in 17 minutes and I said, here's my candy mapper website. Follow the 10 steps on there. Just go navigate through and validate an error and invalidated success. And it took me 17 minutes to write the script. It ran. I had maybe about two minutes of help from Artem saying, Write this word here. Put that one there. Aside from that, I'm like, I've never written a script that quickly. To do that, I've also been using it for a little bit of fun. Hopefully, this will pay off someday. I'm looking is also like robotic process automation. So us in the United States here, and I'm sure you've got people outside of that there in other countries, but we've got Wheel of Fortune out here. And if you happen to watch Wheel of Fortune, you can become a wheel watcher. You get yourself a wheel watcher number and on some of their broadcasts, they will put up numbers saying, Hey, you got 24 hours to match this number. But not everybody watches Wheel of Fortune every day, but I'm using TestRigor to go and log in as myself and check to see if the words you have not you are not a winner does not appear on the screen, and if it doesn't, it sends me a message. So I'm hoping one of these days that I will get a message that says, “Go log in and go collect your $24,000 that is being matched by Wheel of Fortune and Pat Sajak can ventilate.” The odds are highly against it, but it's still a cool thing to have out there. One of the major points of that is that I wrote it's just over a year ago and I've never had to make a change on it. It still sends me a message every day at six o'clock saying No, you still didn't win. I haven't had to change anything, even though I'm pretty sure the underlying architecture of the website probably has changed over the last year.
[00:26:09] Joe Colantonio Cool. Okay, Paul, before we go, is there one piece of actual advice you can give to someone to help them avoid any creepiness with their automation testing efforts, and what's the best way to find a contact you?
[00:28:28] Joe Colantonio You don't want to get some creepy crank calls right? No doubt?
[00:28:31] Paul Grossman I already get enough of those. The warranty on my car is expiring. I don't know if you knew that. They try to tell me that every day.
[00:28:40] Joe Colantonio So thanks again for your automation awesomeness. That's everything we value we covered in this episode. Head on over to testguildcom.kinsta.cloud/a371 and why they make sure to click on the try it for free today link under the exclusive sponsor's section to learn all about SauceLabs awesome products and services. And if the show has helped you in any way, why not read it in review it on iTunes? Reviews really help in the rankings of the show, and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe, my mission is to help you succeed with creating end-to-end full-stack automation awesomeness, as always, test everything and keep the good. Cheers.
[00:29:25] Outro Thanks for listening to the Test Guild Automation Podcast. Head on over to testguildcom.kinsta.cloud for full show notes, amazing blog articles, and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.
Rate and Review TestGuild
Thanks again for listening to the show. If it has helped you in any way, shape or form, please share it using the social media buttons you see on the page. Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.