Testing for Good with Nikolay Advolodkin

Joe Colantonio: Get ready to discover the most actionable end-to-end automation advice for some of the smartest testers on the planet. Hey, I'm Joe Colantonio, host of the Test Guild Automation Podcast, and my goal is to help you succeed with creating automation awesomeness.

Joe Colantonio: Hey, it's Joe, and welcome to another episode of the Test Guild Automation Podcast. Today, we'll talk with Nikolay all about his upcoming event, Testing for Good. Also, he's an automation expert. He's done a lot in the community for many, many years. I'm excited to jump to see what's new with automation as well. If you don't know, Nikolay is a self-driven SDET on a lifelong mission to create profound change in the I.T. world, which is definitely achieved, especially as founder of UltimateQA.com which is one of the go-to websites designed to train individuals and testing best practices. He's also a senior solution architect for SauceLabs, one of our awesome sponsors. So excited about that as well. And he's been voted one of the top automation engineers in the world multiple years in a row by TechBitcoin.com. He's contributed to many books such as continuous testing for DevOps professionals and co-created sauce bindings OSS. I don't know much about that. Maybe we'll dive into that as well. And he's spoken to multiple conferences like the Automation Guild, TestJS, and Saucecon. He's also an animal lover like myself. So he's a great guy. I can't wait to get into it. You don't wanna miss this episode. Check it out.

The Test Guild Automation Podcast is sponsored by the fantastic folks at SauceLabs, their cloud-based test platform helps ensure you can develop with confidence at every step from code to deployment, to every framework, browser, OS, mobile device, and API. Get a free trial. Visit testguildcom.kinsta.cloud/saucelabs and click on the exclusive sponsor's section to try it for free today. Check it out.

Joe Colantonio: Hey, Nikolay, welcome back to the Guild.

Nikolay Advolodkin: Hey, Joe. Thank you so much for having me. Always a pleasure.

Joe Colantonio: Awesome. Same. Same. Before we get into it. I know, I give you a pretty good bio but is there anything I missed in your bio that you want the guild to know more about?

Nikolay Advolodkin: No, you gave me an amazing bio.

Joe Colantonio: Cool. It's been a while. I just checked. Seems like 2 years ago you were on the show. What's new? What you've been up to? I know you're working on SauceLabs. Just curious to know maybe what's new? What's out there? What you've been working on?

Nikolay Advolodkin: Yeah, Joe. I'm still working at SauceLabs. I'm getting a lot more into developer advocacy stuff, so trying to grow bigger communities around automated testing and open source, that kind of stuff. So that was one of the reasons why I actually came here was to talk about testing for good. We can jump about into that in a moment. On a personal level, I've been doing a lot of stuff with JavaScript and just like improving my skills as a front-end developer. Learning really the process of building a web application and taking it all the way to production and releasing it with services such as Netlify for example. And so yeah, that's been also really fun and really exciting to learn all the ins and outs of web development.

Joe Colantonio: So why are you doing that? You're a tester, why are you delving into actual development?

Nikolay Advolodkin: Oh, that's a great point. Well, I believe that, yes. Even though I am a tester, if I want to be much better at doing testing, I need to understand how to do it from start to finish. Right. So testing is not only black box testing, right? We don't want to only test from a black box perspective where we have no idea about the working internals. As testers, we can be a lot more empowered, a lot better at our jobs when we know the insides of the code as well. And me, for example, know that if I want to test a login form, I don't necessarily have to do it through the front end. I can do a web request, get back a token, drop the token in the browser and now perform different kinds of operations, or even click a button using something as simple as a component test that will run in milliseconds and will still perform the same exact operations. If I were to spin up a browser and click a button through something like Selenium or Cypress, for example.

Joe Colantonio: Nice. So it sounds like rather than just create an end-to-end test and think, okay, this is the best approach. Sounds like since you have development skills now, you know, hey, maybe I can push this further left into the actual developer doing more of these types of tests. Does that basically how it turns out a lot?

Nikolay Advolodkin: Yeah, exactly. I mean, there are so much more than just end-to-end test, right? And end-to-end tests tend to be the longest to run and the hardest to write to take the most amount of code. It's getting better. Technology is actually getting way better at this. And end-to-end testing is becoming very reliable and useful. However, there's still much more to testing than end-to-end tests. I've actually found a great balance of testing, especially for like front-end web applications like in common JavaScript libraries like React or NextJS, and so on a balance of using some end-to-end tests like a handful of those combined with a lot of visual testing. So render my applications in different resolutions, especially desktop and mobile, always catching so many visual discrepancies that those tend to come up all the time when I'm doing visual testing, and then I'll throw in some like maybe component tests in there or some snapshot tests with something like JEST for example. And that will make my testing extremely robust and I don't even have to write too much code to do all of that testing. And so all those skills that I learned as a kind of transitioning, shifting more to the left have been super useful in helping me with that.

Joe Colantonio: Nice. So you mentioned component testing, for the folks that don't know what is component testing? I think component testing is an old-time test. So you think of Mercury. They used to have business process testing, which was a component of testing, which is this is not. So maybe you can drop a little knowledge on us, how you define what component testing is?

Nikolay Advolodkin: Yeah, for sure. Great question. I think even, I don't have a full-I think the definition for component testing through the industry is still kind of muddy. But I'll tell you what I think. I think of component testing as unit testing for the front-end. So the actual like JavaScript, HTML that's put together. So the easiest way to think about it is a button you would have like an HTML component, which is like a button and it has some attributes. It might have some styling to it as well, where you can interact and even render that component individually just by itself. You don't need an entire application running and so you can just perform testing like unit testing on that HTML component and then get all the advantages of unit testing and then all the coverage overrun, the browser stuff without having to spin up a server and actually render the application.

Joe Colantonio: Very cool. And so obviously saves you a lot of time and you can iterate over really quickly while the developers are testing, I guess you could just run these quick components almost simultaneously.

Nikolay Advolodkin: Yeah, exactly. They run in like milliseconds. You don't need to start anything so you can literally if you have access to the files, you can just test on the files and whatever's there is what's going to get rendered in the browser. So you don't have that also concern of all I'm doing, I'm exercising a function, but how's that function going to perform when I link it to HTML? And so, you're testing the exact button that would be rendered in the UI.

Joe Colantonio: Great. So when I think of UI, I think of a sharp expert for some reason C# developer. So I just kind of caught off guard by JavaScript, I tell UI C# really tried and true.

Nikolay Advolodkin: Yeah, for sure. Joe, I feel like that's hard for me to still shake that C# expertise. It's how I entered the industry with UltimateQA pretty much. I was doing everything in C#, which was at the time when I was working at my previous two employers. Everything was done in C#. So that's where all my knowledge came from and that's what I was teaching and sharing my knowledge right in C#. I'm working for SauceLabs. I have to support all types of customers. And so primarily the two programming languages that most people, most customers use the Java and JavaScript. And so I had to pick both of those up. So nowadays really I do most of my stuff, I would say in JavaScript and Java as well. And so with C#, unfortunately, it's not as popular as the other two, so I don't get to use it as often. Hence now you don't see me doing much around C# just because not enough time to do so many programming languages and so many technologies, especially JavaScript, they have like hundreds of technologies that come up every day.

Joe Colantonio: Absolutely. Yeah. I love how. Because you work for a large company, they collect all this data. So you probably know what a majority of people are using. So besides languages. So it sounds like you said JavaScript in Java. What's maybe the third one? Is it Python that has been the most popular?

Nikolay Advolodkin: Yeah. Python. And terms of so, yeah we're working for SauceLabs. Right. We run tons of automated tests. So Appium, Selenium. So that's where we get the data from. And so yeah, Python is number three and then it probably it's like Ruby and C#. Ruby kind of has been on the downturn in terms of test execution and then C# is kind of stagnant, maybe going up a little bit. But I do notice that at least. So that's in terms of front-end browser automation, right? I think Saucelabs just said like 4 billion tests or 5 billion tests recently. So many tests, I can't keep track anymore. But at least from the entire community, I noticed that C# and .Net ecosystem is growing, especially because they did go open source and now they can run on Windows and can Linux. So that of course enables way more people to be able to use C#.

Joe Colantonio: So is a dumb question. I guess it depends. It's all how you develop it. Is that one language that tends to be more stable that you see over those billions of tests that run like oh, the ones out of the most stable tend to be JavaScript. Maybe because its developers are actually creating them. I don't know if you have that type of data or not?

Nikolay Advolodkin: Very cool question, Joe. So when you say stable, what's your definition of stable?

Joe Colantonio: Stable is, when it fails, it fails for a reason, not because it was poorly written or has bad waits or things like that.

Nikolay Advolodkin: Yeah, let's see. So I can say that from my experience, I don't have the data to back it up. My feeling is that the JavaScript tech stacks tend to be the most stable and the fastest. My thoughts behind that are yeah a lot of the JavaScript code tends to be written by developers. The JavaScript test code tends to live in the same repository as the software that the test code is testing. And so that just makes everything much easier. It allows for smarter testing and for faster testing and more stable testing. And then also, for example, we at least in SauceLabs, we have well, also the other thing that makes it easy, right, is, for example, when you think of deploying a web application, right, as you deploy a Web application through the pipeline and it's a let's say a front end web application, you need to build that application and then it turns into some kind of code that's running in your browser. Right? You can build it and run it on like localhost 3000 if your test code is in the same place. Once you build the app, you can just run your test code against that localhost 3000 and so everything is like right there on the same machine, super close, super fast. And I think that lends itself to the stability. While for example, now if you have to take a programing language that's not the same as the software that you're developing, let's say Java, for example. Now you have to run it in a separate pipeline and you can't really execute test against local host because how are you? What are you going to be sitting in one pipeline where you building? What are you going to be and sitting waiting there for until you finish that build. So you have to build and deploy that and only then can you run your Java test. And so that of the course introduces latency, introduces as a result, makes the tests longer and a little bit more brittle. And so the other thing that I've seen is in SauceLabs, for example, for all the JavaScript stuff, we have sauce control and sauce control basically the way it works and I hope that a lot of libraries transition to this mechanism as well. The way it works is it basically takes your test code, we upload it to a VM and then we run the tests on the VM with the browser. So it's almost like everything executing in one environment without having to travel, for example me being in Atlanta right now of running my test from Atlanta to California where the data center is, and back, and then testing the application and that manner we can run together and again making the test faster and more stable as a result. Does that make sense, Joe?

Joe Colantonio: Yeah, absolutely. That's a great insight. So I guess along those lines then when I think of JavaScript testing and automation, I think of Cypress and I know Selenium supports a lot of tool support. So do you see in popularity with JavaScript, if someone's a JavaScript developer, what types of tools they tend to use more? Is it Selenium? Is it Cypress? Is it something else? Is it a suite of tools that they use depending on what their needs are?

Nikolay Advolodkin: That's a good question. So in terms of developers, they will typically use something like a Cypress or Playwright has been really hot recently. And I would say they don't do too much end-to-end testing as far as I've seen, they'll do a lot lower level testing, so they'll do a lot of like a visual component testing. They will do unit test. They use a lot of like JEST for unit tests, for example, or they'll use like component tests. So I think because Cypress is like older to the game, they're kind of like the OG in the end-to-end testing space than JavaScript. They tend to be used more often. But I also see Playwright growing in popularity. I'm actually here in Atlanta to do a demo for a customer with Cypress and Playwright because they want to compare the two. So yeah.

Joe Colantonio: Absolutely. That's been hearing a lot more about Playwright for sure. It seems like it's becoming one of the leading solutions out there for what it is. So talking about that, I think actually, not that I stalk you, but I do look at your social media often and I know you went to something in Europe on I think it was the Test Automation Summit, some of the masterminds behind the open source. So I don't know, behind closed doors, what do you all talk about for automation solutions? Do say, Oh, Playwright is going to go down or you think it's going up, or is that summit all about and maybe some key findings maybe that you might be able to share with the audience there?

Nikolay Advolodkin: Yeah, sure. By the way, Joe, you can stalk me on social media any time. It's a term of endearment stalk me and bring your friends. So, yeah, about the Test Automation Summit. So it was an amazing event organized by Titus Fortner and Diego Molina from Selenium community, the primary developers for Selenium. And so the kind of purpose behind that meeting was there are so many open source projects that exist that are built around the Selenium and WebDriver ecosystem. Projects like Selenite, Codesap, and WebDriverIO Christian Bromann was there. Many of these projects built around open source standards. And so we were trying to figure out how do we get more contributors to open source because open source developers typically are not paid. They do it because they love the project and they do it in their free time versus paid projects such as Cypress or Playwright, where, for example, a Playwright has a team of developers at Microsoft working for them that are paid to do to work on Playwright every single day. And so how do we get more contributions to all of these projects? For example, I contributed to Atata, which is a .Net framework. It's a like a page object wrapper around Selenium making test automation much easier. A very amazing project done really well. The guy Yevgeniy that runs it. Yeah, very well-tested, very well laid out project. So if you're in C# looking for a page object framework that you can use, grab Atata for yourself. And so in terms of some key takeaways, well, some key takeaways are that one thing we all have to consider whenever we're making a tool is, for example, open source tools are building on open source standards. And so if we're creating a community that is more collaborative, right, for example, all the browser vendors to open up their browsers and allow automation protocols that are common and easier for the entire world, then we all benefit. So that's really important to think about, that we want to be, regardless of which tool we're using, which browsers, we want to be more collaborative than less because then everybody benefits from more collaboration and more openness. While for example, something like Playwright doesn't seem to be want to be very collaborative, they almost want to lock down the ecosystem for themselves. They want generate their own browsers so that they can run the tests in them and they control the code. And of course, the code is all controlled by Microsoft. And I'm not saying that is necessarily a bad thing. It could be a great thing because it introduces a lot of innovation, but it's something to think about that it could potentially now be moving in the direction of something that we've had in the past. I know you're very familiar with like Joe is like an HP QTP kind of product where there is one company that controls that and you're as an engineer dependent on them. And you have to do what they basically say and you've got to pay a fee when order used a product. So again, these are not inherently good or bad things. It's just considerations that we have to take as we're choosing products and we're choosing our tools. And Playwright is a wonderful tool. What I love about it is that it supports all the primary programming languages. For example, Cypress is only in JavaScript. So you're a developer in JavaScript, you can only use JavaScript, but for all of us other engineers that don't use it, that don't use JavaScript. You can use Cypress, but Playwright does Java, .Net, JavaScript, and Python. So it's like they have you covered and it's the same API. As a tester you can pick it up and it's a really good tool.

Joe Colantonio: Awesome. So, I know there are a lot of different topics, a lot of hot things going on in the testing space. So there are a lot of events going on. And I also noticed as I was stalking you, putting on an event called Testing for Good, I thought maybe you could dive in a little quick. What is testing for good?

Nikolay Advolodkin: Yeah, sure. Joe. Yeah. So testing for good is an event that I came up with about a year ago. I'm always trying to think of like how I can serve the testing community better. And I'm also a co-lead on the Environmental ERG at SauceLabs. And so I was like, Ooh, why don't I put the two together and create some kind of an event where we enable people in great automated testing practices and then we try to give back to the community? So the purpose of testing for good is to enable amazing test automation while creating a more equitable society. And so I started it about a year ago. It was just me. I ran like a JavaScript workshop where I was hands-on. People joined for like 2-3 hours, do hands-on coding, you do exercises. So at the end, instead of just watching a theoretical talk and maybe leaving with a repo, you actually leave with hands-on coding knowledge that you've applied, that you've learned something and can now take to your workplace and hopefully start using at your workplace immediately. So that was me had like 130 or 40 people join that workshop, seem to be successful. People loved it. The workshop was free. We raised like over $4,000 for black girls code and I'm like, cool, let's do this and make it bigger and better. And so now we're coming up on our fifth event that's going to be happening in October 18 through the 20th. And so the testing for good events is going to be held on October 18th through the 20th. We got three days Tuesday, Wednesday, and Thursday. We got five amazing instructors. And let me tell you so five amazing instructors teaching in the two primary programming languages that most testers use JAVA and JavaScript. In Java, we're going to have four different workshops API testing with Java by Bass Dijkstra, Crafting a test framework by Titus Fortner from Selenium, Mobile Testing with Eyal Yovel, he's a solutions architect at SauceLabs, and Web Testing by Chris Eccleston. He's also a solution architect at SauceLabs. And then for JavaScript we have three workshops. We have end-to-end component testing with Cypress by Gleb Bahmutov from probably one of the best Cypress engineers in the world. And then we got web testing and the API testing by me. I'm going to run two different workshops, so we're going to have numerous times that these workshops will be available over the three days. We try to make it EU and US friendly. So if you're in US, you'll probably find a time one day that works for you. And if you're in EU, you'll probably find a time on another day that works for you just to be try to be as inclusive as possible. Workshops are absolutely free. Anyone can join, the workshops will be recorded. You'll do hands-on coding in whatever technology stack that you want. And then we are going to be asking for donations. All the donations are going to go to Environmental Working Group. If you're not familiar with an Environmental Working Group, they're a community of 30 million strong working to protect our environmental health by changing industry standards. Since 1993, the Environmental Working Group has shined a spotlight on outdated legislation, harmful agricultural practices, and industry loopholes that pose a risk to our health and the health of our environment. So being a test automation enthusiast and an environmental ERG co-lead, it brings my two passions together into a single event. And hopefully, super valuable for the entire testing community and the environment.

Joe Colantonio: It really is testing for good. So that is awesome. So for a workshop, do you have anything like this? Do you have like certain levels? So it needs to be in order to participate. Is it going to be by beginner-intermediate events? Or they have to have something already installed on their machines. Like, how does that all work? If someone's wondering, Oh, this sounds great, but you know, I don't have enough knowledge in JavaScript to do this or whatever.

Nikolay Advolodkin: Yeah, that's a great, great question. So let me maybe I'll talk about like about each individual workshop so I can tell you for my workshops you can be a beginner and just jump into it. We host most of us, at least at SauceLabs we host our workshops in Gitpod. And so Gitpod is like a docker container that can run in your browser. I'm actually Gitpod ambassador, so shout out to them. They're awesome. So it's basically you can have a GitHub account and then you can open your GitHub source code in the browser and then it's going to open up an environment and you can code and run stuff in that environment that makes it really easy. So you don't need anything installed. You'll be coding in the browser for at least my workshops and those are beginners. I'll actually almost all the workshops will be a beginner and you'll be taken from beginner to like expert level skills through the workshops, maybe actually all of them maybe crafting a test framework by Titus. Because it's crafting a test framework, you probably need some programming experience because you can't just start as a newbie on already jump into frameworks. So there may be more intermediate with some programming experience, but regardless I'm sure everyone will learn a lot and every single one of these workshops.

Joe Colantonio: Absolutely. And where can folks go to register for that?

Nikolay Advolodkin: Everyone can join at bitly/tfg-oct-22. And I'm sure, Joe, you can link to it.

Joe Colantonio: Absolutely. I'll have the link for that in the show notes for sure.

Nikolay Advolodkin: Yeah, perfect. And then you can, of course, Google just testing from good workshops October and that should come up and Google but Joe will link you and then you can always reach out to me on Twitter, and LinkedIn and I'll share the link as well.

Joe Colantonio: Awesome stuff as always. Okay, Nikolay, before we go, is there one piece of actionable advice you can give to someone to help them with their automation testing efforts to help them test for the good? And what's the best way to find or contact you?

Nikolay Advolodkin: One piece of actionable advice I would recommend is to try and shift your testing more to the left by learning some of the skills that developers do on a daily basis, such as building an application, deploying it. It'll help you to be a better tester if that's what you're interested in, and you'll learn something important skills that you know will overall enhance your testing. And you can find me on Twitter at Nikolay_a00. You can find me on LinkedIn Nikolay Advolodkin. And I release a ton of I mean all free tutorials on YouTube UltimateQA and also and of course find me on my website UltimateQA.com. And there you'll see all my blog posts and my tutorials. And again, I always try to do everything for free and help the community as much as I can.

Joe Colantonio: Thanks again for your automation awesomeness. The links of everything we value we covered in this episode. Head in over to testguildcom.kinsta.cloud/a417 and while you're there make sure to click on the try it for free today link under the exclusive sponsor's section to learn all about SauceLab's awesome products and services. And if the show has helped you in any way, why not rate it and review it in iTunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe, my mission is to help you succeed with creating end-to-end, full-stack automation awesomeness. As always, test everything and keep the good. Cheers.

Joe Colantonio: Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps, and automation professionals in the world, we'd love to have you join the fam at Testguild.com. And if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real-world value that can improve the skills or solve a problem for the Guild community. I'd love to hear from you. Head on over to Testguild.info and let's make it happen.