How to Create Automation Magic with Benjamin Bischoff

[00:00:04] Joe Colantonio Get ready to discover the most actionable end-to-end automation advice from some of the smartest testers on the planet. Hey, I'm Joe Colantonio, host of the Test Guild Automation Podcast, and my goal is to help you succeed with creating automation awesomeness.

[00:00:25] Joe Colantonio Hey, it's Joe. Welcome to another episode of the Test Guild Automation Podcast. And today, we'll be talking with Benjamin all about how to create automation magic. If you don't know, Benjamin has been a software developer and trainer with over 23 years of experience. He decided in 2016 to make test automation his main profession. After working as a test engineer at Trivago for five years, he returned to his former employer, Ubisoft, as a senior automation engineer in 2021 to contribute to it. His knowledge of software testing and automation in the field of computer games. So an interesting ping-pong move. Yet again, he returned to Trivago after five months in order to continue working on various aspects of test automation. He's also the author of two open-source projects for Cucumber BDD parallel test execution and reporting. And I just saw somewhere at social media he's releasing a book. Hopefully, we'll get into that on Karate. Also, he's a conference speaker. If you ever see him at a conference speaking, you need to go to him because he's also a super cool magician, so he does some really cool tricks. I think your mind would be blown. Really excited to have him on the show. You don't want to miss this. Check it out.

[00:01:31] Joe Colantonio This episode of the Test Guild Automation Podcast is sponsored by the Test Guild. Test Guild offers amazing partnership plans that cater to your brand awareness, lead generation, and thought leadership goals to get your products and services in front of your ideal target audience. Our satisfied clients rave about the results they've seen from partnering with us from boosted event attendance to impressive ROI. Visit our website and let's talk about how Test Guild could take your brand to the next level. Head on over to TestGuild.info and let's talk.

[00:02:06] Joe Colantonio Hey Benjamin, welcome to the Guild.

[00:02:09] Benjamin Bischoff Hello. Thanks for having me.

[00:02:11] Joe Colantonio Awesome to have you. I always botch bios. So is there anything I missed in your bio that you want The Guild to know more about?

[00:02:17] Benjamin Bischoff My bio is quite hard to read. So for me, this interesting ping-pong move that, that's still there is not really that important. I just noticed what's more important is that I've been in automation for quite a number of years and that's also part of the reason I came back to Trivago to stay in this automation position and test position because that is what I want to do.

[00:02:44] Joe Colantonio Why?

[00:02:47] Benjamin Bischoff That's a very good question because it mixes everything I like about software, it mixes development aspects, it mixes software architecture. You have so many insights into so many different aspects of software development where as a pure developer, you typically focus on one specific area. But as a tester, you get in touch with a lot, so you learn a lot.

[00:03:17] Joe Colantonio Love it. I totally agree. How much carryover is there with do have a background in development that makes you test automation engineer? I mean, are those skills are applicable as a test engineer?

[00:03:28] Benjamin Bischoff I think actually a lot. I think what's cool about coming from a development background is that I kind of speak the language of the developers so I can be in the middle of our QA and developers. So to translate a little bit, sometimes. Also, I bring software craftsmanship, principles and clean code principles to test automation code, which in my books is pretty important to keep your solutions maintainable and scalable.

[00:04:02] Joe Colantonio Love it. So also being a developer and a tester, do you find when you have discussions about developers, maybe they say, Oh, you can't test that way, or like it's more of an education process? Well, actually I know because I'm a developer too, so you can do it. And here's how do you ever get yourself in conversations like that?

[00:04:18] Benjamin Bischoff Yeah, that happens. I have to say, it used to be quite hard to be in QA as a developer because the way of working is different. If you come from a pure development background, it's like you're very biased. And I know from my former position as a developer, I had my biases towards QA. I always keep telling the story that we even at one time in a team knew about a rather serious bug, but we didn't fix it because it was too complex. And we said, as long as QA doesn't see it, we don't have it. And yeah, I tell this because this is not how I work anymore. I learned a lot from QA and I hope the QA team learned quite some things from me too.

[00:05:10] Joe Colantonio Great. So developer background. A lot of times testers are told, Oh you need to use the same tools as the developers. You need to use the same languages as a developer. Since you are a developer and you're our tester, I don't mean to keep harping on it, but it's a different perspective. What are your thoughts on that? Because I know one of the tools that you wrote a book about is not necessarily seen as a developer tool.

[00:05:31] Benjamin Bischoff Right? So, oh, well, actually, two parts to this. So the one part is I believe that it does not matter what tool you use and what language you use for testing, as long as the development team is not the only entity to do the testing like for unit tests, I think it makes a lot of sense to have the same language as the application code. But if you write end-to-end tests or API tests, I don't think it matters that much. It matters more what problem you want to solve and what tool fits the best. And I think enforcing the same language devalues developers because the concepts are the same. I mean, if you jump from language to language, there shouldn't be that much problem in test code because usually, it's not super complex code, especially if you use tools like karate that are really easy to get into.

[00:06:37] Joe Colantonio Yeah, I guess the argument always was, well, developers in a flow, they want to use the same ecosystem, same tooling, don't want to leave their IDE. But I haven't been a developer, so I don't know if that's actually a concern that you would have as a developer day to day.

[00:06:50] Benjamin Bischoff I don't think it is really a concern. I mean, for us, we use a lot of different tools and different languages depending on the problem and we try to analyze the problem first and then choose what fits best. And that it's the same language. It's not one of the top criteria.

[00:07:10] Joe Colantonio So going from developer to testing is kind of unique. Sometimes testers will start off in testing so they can become a developer. I know a lot of folks. I love testing. I love testing. I've spent my whole career. If someone's been a tester their whole life, how much development skills should they have or is that something they should be concerned with? Because I guess they're testing, that's probably the hardest skill to have and they probably already have it, so much developer skills do you think a tester should have?

[00:07:35] Benjamin Bischoff Depends on what is being tested. If you are like a super good exploratory tester, and those are still needed and are needed even more now with all of the AI stuff coming in, then you can have a great career without any coding. But if you notice that you're doing the same things over and over again as a tester and you want to change that, then it's a good time to start to learn about automation a little bit. And also don't learn the language first, learn some of the concepts first, and I think the rest just develops. I mean, it was the same for me when I got into testing. I didn't have any testing knowledge and I had to acquire this knowledge too, which was really hard at first. But yeah, you can be an automation engineer or a software development engineer in test. It doesn't matter from which side you enter this path.

[00:08:38] Joe Colantonio So I like how you mentioned earlier how you like to bring clean code principles to automation. So what does that mean? So if someone like clean code. What?

[00:08:47] Benjamin Bischoff But yeah, I mean, if you are on social media, LinkedIn, you see quite some examples of test code. And also if you check GitHub repositories and I noticed that a lot of times the code is not as good as it could be. And if you don't have that much experience and you just copy and paste code like this and you extend to your test suites, at one point, it will become a nightmare to maintain. And if you want to change something, you have to change it in 100 different places. I think it's really important to keep this organized and apply some software architecture principles to it so you can maintain it in the future or some other people can maintain it and understand it.

[00:09:40] Joe Colantonio So I guess the follow-up is what kind of software architect principles should someone apply that you think for a tester is relevant to automation code? So I know, not all code. I mean, code is important but automation code I guess there are some things that may be overkill where in production it makes sense. Maybe I'm wrong.

[00:09:57] Benjamin Bischoff No, I think you're completely right. It depends on what the code is doing. If it's a thing that is doing one specific task over and over, it doesn't matter that much that it's super clean as long as it works. But if you write a test framework that is used by many to write test cases, then it's important to keep it really concise. I mean, the simplest thing that testers and coders should apply to their code is proper naming. That's where it all begins so that people can follow what this piece of code is doing. It's often said that the best code is code that documents itself and if you name it and keep your methods very short and concise and make everything just do one specific thing, that's a great start already.

[00:10:54] Joe Colantonio I've said this multiple times. Probably most of your career is done debugging or maintaining test than running tests. So if it's not written clearly, it could become a nightmare. It seems so, even though it seems trivial, if you wrote the code yourself six months later if you didn't name these correctly, you might be like, What did I do here? Right?

[00:11:12] Benjamin Bischoff Exactly. And that has happened to me. When I look at my own code and I didn't understand anything. Yeah. So it's not only for others, it's also for yourself.

[00:11:24] Joe Colantonio We've met in person at Selenium Conf, but I believe we actually did a session around code smells, which I think goes along with clean code. So once again, for folks that may not know what is a code smell.?

[00:11:35] Benjamin Bischoff The code smell is something that makes you give you a strange feeling when you see it. And it can point to something that can be wrong with the code. It doesn't have to be something dangerous, but it's just something that gives you a feeling like there's something fishy about this piece of code. And my session was about how to pinpoint those things, how to name those things properly when you want to talk to someone else about this. And what could be problematic about this?

[00:12:10] Joe Colantonio Can you give us some examples?

[00:12:12] Benjamin Bischoff Yeah, I think there are 21 code smells I presented. The easiest one is probably the long method smell, which means that you have a method that does more than one specific thing. Or if you name a method of I add the example of a safe customer method that not only saves the customer but also gives you some HTML back that displays something about the customer. So this method would do something that you wouldn't expect from the naming. So this is a method that's called long method, not because it's long, it's a long piece of code, but it just means it does too much. That would be the simplest.

[00:13:02] Joe Colantonio Nice. Another one that stuck out to me is that it was broken windows. Broken windows?

[00:13:08] Benjamin Bischoff Yeah. Broken window is not, a code smell. It's more something that happens to software if you don't fix stuff. So if you imagine a house with a broken window, if you don't repair this window in the next days, somebody might come and look at this and say, Oh, this has been broken for months. So let's break another one just for fun and just throw some garbage into the hallways. And then in the end, you just have a house that is completely broken and maintainable. So what this tells developers is you should fix mistakes early and you should pinpoint bugs early before other people make the same mistakes and ruin the whole codebase.

[00:13:53] Joe Colantonio It's not a really good segway here, but so I know you're writing a book on writing API test with Karate. There's some controversy around Karate for some reason because a lot of people say it actually creates code smells and I think it's because they don't understand what it is. They think it's a BDD framework, which I don't think it is. It just looks like it is. And then when people see that, they automatically make assumptions. I don't know where I'm going with this, but do you have any thoughts on Karate and how maybe people may be getting confused with not really being a BDD framework? Or maybe it is. I'm just wrong.

[00:14:26] Benjamin Bischoff You said a lot of true things about this because when I first encountered Karate, I thought the exact same thing. It looks like BDD. It looks like Gherkin language. And when it was started by Peter Thomas, it was actually using this part from Cucumber, the Gherkin parser. But Gherkin and BDD means that you describe behaviors of your applications that you want to test. But he took this concept and made it more technical because describing APIs with this behavior-driven approach is not really feasible. I mean, you wouldn't say I send the appropriate data to this endpoint, but you would describe what is the exact data and what is the exact end point and what is the structure you expect. It does make a lot of sense. The controversy just came from this format of how the scenarios look. And he also wrote about this himself and said, Karate is not BDD and it is certainly not BDD. But once you get over this mental block that you have, it's really fun to work with.

[00:15:47] Joe Colantonio So you mentioned, it's an easy tool to use earlier when we first started. And as a developer, would you developers use a tool like this, give it a little example, maybe how you would write a test with Karate and how maybe someone would utilize if they're not a tester.

[00:16:01] Benjamin Bischoff With karate, you can just, if you know tools like Cucumber, you would typically write test scenarios like the user goes to the website or the user logs in and then he enters his order. And in the end, I don't know, it costs that much money. With karate, you use the same principle, but you don't write all this code behind your test scenarios. So what Peter Thomas did, he used this form of the scenarios, but without what's called the Blue Coat. So you don't have to for every step that you use in your scenario, you don't have to write the appropriate code that translates what you write in Gherkin to actual Java code or JavaScript code, but Karate does it by itself, so you don't really write code, you write more test scenarios, but still you have to understand. Exactly what you're testing. And so it is definitely a tool that can be used by developers as well.

[00:17:14] Joe Colantonio So in the book, the book isn't released yet, so I haven't been able to actually look through. I just looked at the high-level information. Do you cover any clean code principles, applying it to Karate, or is that completely a different domain you didn't touch?

[00:17:26] Benjamin Bischoff That is a little bit of a different domain. I talk about how to keep your scenarios concise and yeah, you could say clean so that they're easy to maintain. But since you're not dealing too much with real source code, those clean code principles don't always apply. I mean, I have chapters where I also extend karate by JavaScript code and by Java code, and there I'm really careful to not violate any clean code principles. But it's not an explicit part of the book.

[00:18:02] Joe Colantonio So I think you also touched on maybe I'm wrong. Like I said, I just read the high-level review is a Karate for performance testing. Yes. Is that possible? What is it? Use Gatling behind the scenes. How does that work?

[00:18:12] Benjamin Bischoff Yes, it's using Gatling. That's another thing that is really, really nice about Karate because you can just take your test scenarios that are already there. And by just modifying how you run them, you can use them as performance tests using Gatling. You get all the Gatling performance test reports. You can also explicitly choose which of your scenarios should be used as performance tests. So you have tons of possibilities.

[00:18:43] Joe Colantonio It's almost like magic. So that's my segway. Speaking of magic, you are a magician. I saw a session that I loved. Could be like your signature kind of keynote, I think was like nine magic principles in software or something. We actually were talking about you doing magic, and then you were applying it to software, which I thought was so cool. So a little background for people like as a magician, what principles have you found kind of go or correlate with testing principles? Maybe we could dive into that really quick.

[00:19:10] Benjamin Bischoff Sure. I mean, this session that you have seen is called Smoke Tests and Mirrors.

[00:19:15] Joe Colantonio That's what it is. Yeah.

[00:19:17] Benjamin Bischoff And, yeah, it's becoming one of my signature things for sure. It all originated in some blog posts I did comparing magic and software principles. And we have some principles that are well known, like misdirection, that come directly translated to software and test reports, and also stack traces and codes that can misdirect you heavily depending on if they show too much or too little information, but also things like clarity of effects and magic and clarity in UIs and APIs in everything that a user touches on your software. I mean, there is so much that you can compare. And part of the reason why I did this was not only to do this, but also to show how you can apply knowledge that you have from outside your profession to your profession.

[00:20:13] Joe Colantonio So I love doing this. Other people give them work, but I think this would make a great book. So why did you choose doing a book on Karate rather than this one, which I think would be like such a cool, evergreen type of book?

[00:20:25] Benjamin Bischoff Oh, that I have never ever thought about turning this magic and software thing into a book. But now that you say it, I'm thinking about this.

[00:20:34] Joe Colantonio I can actually see like the copy, the design, wild I think.

[00:20:38] Benjamin Bischoff I think that's a great idea. The reason why I wrote a book on Karate was very simple, actually. I was approached by the publishing company and they asked me if I could write a book on. I won't disclose the topic here, but it's a topic that A, has a lot of literature already and B, I'm not an expert in. So I told them I'm using this test framework and there is no literature about this yet. I would feel comfortable writing a book about it and that was Karate. And they said, Yeah. After some research they said, Yeah, there is nothing yet. I just thought something is needed because it's a great framework and there are so many questions on StackOverflow about it. So I just thought there must be something. So I wrote it.

[00:21:30] Joe Colantonio How long it did take?

[00:21:31] Benjamin Bischoff It started in November, so it took half a year roughly.

[00:21:36] Joe Colantonio It's pretty quick for a book. Technical book. That's awesome.

[00:21:39] Benjamin Bischoff Yeah, I have a really tight schedule, which is cool because I am a procrastinator by heart. So if you have really tight deadlines for each chapter that really gets you working.

[00:21:54] Joe Colantonio When's the book going to be available?

[00:21:55] Benjamin Bischoff The book is coming out May 19th. Okay, So it's on preorder right now at the time of this recording.

[00:22:05] Joe Colantonio Yeah, I'll definitely have a link for it and people should definitely pick it up. Okay, Benjamin, before we go, is there one piece of actual advice you can give to someone to help them with their automation efforts and what's the best way to find or contact you?

[00:22:18] Benjamin Bischoff So the advice I have for anyone, starting with automation, is talk to people first. Don't start with an automation solution and then try to force it on people, because that is exactly what happened to me. And then you have the risk of this whole automation project just falling apart. You have to understand the problem first. You have to understand what people are struggling with to be able to come up with a good automation solution and then start very, very small. That's my advice and you can contact me. The easiest is on my website. It's called softwaretester.blog. And this is also where I write occasional articles. And there you also have the list of all my social media links and everything.

[00:23:09] Thanks again for your automation awesomeness. The links of everything we value we covered in this episode. Head in over to testguild.com/a444. And if the show has helped you in any way, why not rate it and review it in iTunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe, my mission is to help you succeed with creating end-to-end, full-stack automation awesomeness. As always, test everything and keep the good. Cheers.

[00:23:46] Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps, and automation professionals in the world, we'd love to have you join the FAM at Testguild.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real-world value that can improve the skills or solve a problem for the Guild community. I love to hear from you head on over to testguild.info And let's make it happen.