Netflix SafeTest with Moshe Kolodny

[00:00:04] Get ready to discover the most actionable end-to-end automation advice from some of the smartest testers on the planet. Hey, I'm Joe Colantonio, host of the Test Guild Automation Podcast, and my goal is to help you succeed with creating automation awesomeness.

[00:00:24] Hey, it's Joe, and welcome to another episode of The Test Guild Automation Podcast. And today, you're in for a special treat because you probably been hearing all about it. We have Moshe joining us to talk all about Netflix's SafeTest. If you don't know, Moshe is a senior full-stack engineer at Netflix and the creator of this new tool SafeTest. You don't want to miss this episode. Check it out.

[00:00:46] Joe Colantonio Hey, Moshe. Welcome to the Guild.

[00:00:50] Moshe Kolodny Hi, Joe. Thanks.

[00:00:52] Joe Colantonio I guess before we get into it, is there anything I missed in your bio that you want The Guild to know more about? I usually have a longer bio, but that's all I actually could find.

[00:00:59] Moshe Kolodny I think that's pretty much like what I do and where my responsibilities lie. And so I think that's a pretty good description.

[00:01:06] Joe Colantonio All right. So you are a full-stack engineer. How did you get into testing then? A lot of times it's not necessarily a topic I hear a lot from either a full-stack engineer or a developer, hardcore developer.

[00:01:17] Moshe Kolodny Yeah. So I guess my approach I'm not officially like a test engineer. I'm more of just a generic full-stack engineer that does UI work. I'll do BFF like backend for front end, like a node server to be able to hit different services that are accessible by the browser. So it's just really a pragmatic thing. And one of the pain points is always, how do we know that you're not going to break the world with every change? That's what testing is an important aspect of that. And that's where I saw a problem and I did not see a good solution. The current solutions out there have some limitations that make it difficult to adapt in a lot of environments. There are a lot of times where it's a great solution, but in my use cases it was not solving the issues that I was trying to solve.

[00:02:06] Joe Colantonio All right. So what was the problem? What was the use cases then that you were struggling with?

[00:02:10] Moshe Kolodny The biggest use case is the fact that we and this is kind of outlined in the announcement of post, is the fact that you have to kind of make a choice between what flavor test you want to do, and you're kind of stuck with that choice. So you either could go full end-to-end testing solution like Playwright or Cypress or something like that, Selenium or any of these tools. And they are great for what they do, but their limitations is the fact that they don't allow you to test it as an app developer. So they're more of like, if you're just doing QA on an application, you want to make sure that the application works as a whole. And that means, like all the backend services respond the way you expect it to when the UI looks the way you expect it to look. Then just to contrast that is going to be unit testing and that it's going to be a tradition that use like a JS DOM. I think Karma used the Angular. I think it was Karma. They actually did it in the browser and they would do some hybrid approach, which is probably pretty close to what I was looking for. But the issue with that is the fact that anyone that's had to do that and you have like smart components and page interacting with them is very difficult because the JS Dom is a pretty good approximation of a DOM. But once you kind of get into UI component libraries, everything starts breaking down fast and you can test like very rich user interactions like drag and dropping or, things like that. It makes it very difficult to get a good handle on how to do things. So we're kind of like forced between like to have choices of do we want to have a good way to interact with the page? And that's what end-to-end test give us. And unit tests give us a good way to set up the page. We could mount our components, but we don't have a good way to like interact with them. So we kind of have this choice of two different things. And that was kind of a problem I saw. And up until recently before component testing was a thing like Cypress component testing or Playwright component testing. This was not solved problem. And that is kind of like when I started thinking about this problem. And the current solutions they have nowadays are great for most of the time, but for my static uses, I want to be able to test things that are harder or like more tricky to test. I'll give you an example. One of the things I want to test is I want to test that the page works with having all the goodies of a test automation tool like Cypress or Playwright, but I also want to be able to check that certain things within the actual code was called. So I want to make sure that a service mocked was triggered a certain way or things like that. And if we do component testing, we could probably get pretty close. But that has a different bag of worms with the fact that you have you lose the ability to do certain things because the way the Cypress and Playwright component testing works is they start a web server, like a Webpack server or something like that, and then they own a rendering of the page, which breaks for like corporate environments, because we don't own the rendering, we have a framework, we create like a create react app that we put on port 3000. And then there's some proxy magic that happens that make sure like header authors are forwarded properly. And then we work with like the traditional component testing end-to-end libraries.

[00:05:39] Joe Colantonio So it's the best of both worlds. Then it's like you get you do an end-to-end test and you can interact with at a unit level almost with the components. So are you able then to put the application in a certain state and then run your end-to-end tests. So it makes it faster as well, is that an extra benefit?

[00:05:57] Moshe Kolodny The speed was never really the goal. This was really, we'll do whatever it takes to be able to have confidence in the application. That happens to be that once we know that we are rendering the page a certain way, we do get some speed improvements because we always know that, we're going to reuse tabs. We know, like when a page is rendered, when it's ready, when it's not. So we could optimize it pretty well. And there is like a speed like on every build on GitHub we run like a full test suite on site, SSL status, like test status, and it does like a test where it clicks a button 500 times in a search that the page didn't do anything weird. And we also have 50 tests where it opens a page, click a button and then checks it, does something and closes it. So we have both a lot of interactions on that single test, and we have a lot of small interactions on a lot of tests. And we do both and they're pretty fast. But that wasn't the point of it. But regarding your other question that can we set up the app to do whatever you want? That is pretty much the goal. So that is what component testing is take advantage of. And the fact that we could mount our app and we could pass it like the people service as a prop to the app component and have it call a mock instead. Of course, you could always do this with like overriding the network layer in like a browser based automation tool. But then you have to kind of you're kind of like digging into the internals, and you have to understand the black box of how network can work. So if you have like you're using gRPC or GraphQL or something like that, you have to now understand the binary format. Those things travel over the wire in order to mock them out, versus if you're able to just mock out them on the application layer, all you need to do is to just say, hey, instead of give me this object, give me that object. And it's much easier to deal with.

[00:07:35] Joe Colantonio Awesome, I guess, it also allow you to mimic things in production without having the full blown environment, I guess with the mock and comes in you'll able to get as close to real-world situations or maybe you can even do some type of chaos testing, like what happens if this service isn't available? What would happen? Does that make sense?

[00:07:53] Moshe Kolodny Yeah. So that is like the primary use case in the fact that SafeTest is a way to have the running application that actually hitting production services run. And we don't need to mock out the services or anything like that. And we're able to do that in a corporate environment. So that's we could choose to like disable two out of three services and see if the application completely blows up or it like gives you a nice error state and things like that. So that's pretty easy to deal with.

[00:08:24] Joe Colantonio Cool. So you mentioned Playwright, you mentioned Jest, where does SafeTest come in? Is it act like a wrapper on top of these technologies. And you do the special sauce to make it do all the type of interactions with the components?

[00:08:36] Moshe Kolodny Yeah. So I'll give you a rough idea of how, SafeTest is really the closest comparison to SafeTest is like Cypress or Playwright component testing. And I'll explain like kind of at a high level how they work and like how SafeTest does things differently and how it works, and that the contrast will kind of highlight what it's doing, what the advantages are. So traditionally, what these things will do is they will create like a wrapper page to be able to kind of mount whatever component you want, and then it will send that component to mount, and then you interact with the page via the page object or something, or the CY object, which I could live in the browser. I think for component that doesn't live in the browser for Playwright, it still lives in Node.js. They have to kind of like choose between one of those two. And I'm not 100% clear on Cypress. So I could be getting some of the Cypres'isms wrong, but I'm pretty confident that's how the Playwright one works. The way SafeTest works is that it will execute the code in both environments, so it opens up Node.js and then it will like have Jest essentially just go through the regular code but also listen while those codes are running. So instead of calling the vanilla like describe function, you're actually calling the SafeTest describe function, which will call the vanilla describe function but will also like track how it was called. And so it builds up the same stage testing tree that Jest will have. And then when you open up browser. So when the test comes to actually executing it will open a browser. And then the browser will also execute all those describes and all those things. So it'll build up the same tree in the browser. And then instead of needing to send a message to the wrapper page to mount this component, it will just know because it's running the same code. It says, this is the test. This is the test file, this test name. And then it will call the render function. So in Node, when the render function is called, it knows to open a browser. In the browser, when the render function is called, it knows to mount that component. And then we don't give up either one. So now we are able to do things in node and the browser without having to like send like post messages between the two, which is other solutions would do it.

[00:10:43] Joe Colantonio Very cool. I also think I when I was reading, I could be wrong, I could misunderstand. It look like you're passing images and is it also doing visual validation testing as well where it's looking at the UI or did I misunderstand that?

[00:10:58] Moshe Kolodny Yeah. One of the things there's a library called Jest image snapshot that is I think by American express and that works with Jest. I think it was in the Puppeteer days. I think they created it. We already had ways to open a browser using Jest and Puppeteer, and then we could take snapshots of the page, and they created a library to be able to take two images, take the like the golden image and compare it to like the current image and give them essentially. So it's like a snapshot defining just with images essentially. And that's in Playwright and SafeTest use it. I think Cypress also has accounts for that.

[00:11:36] Joe Colantonio So I know it's a lot of people why they like Cypress, especially developers. It's like a self-contained it's easy to get up and running for developers. They don't have to do any type of install, basically. How easy is it to get started with SafeTest?

[00:11:51] Moshe Kolodny The opposite end of the spectrum, I would say. It is not a great sales pitch, but the issue is because the way it works is that it doesn't compile your code. It assumes that you have a bundler site you're using like create React app. So it will leverage Create React app, but then it now needs to be able to be able to import like app at SafeTest.tsx within your application code. And it needs to do that without making your production bundler large. So it needs to use lazy loading. And it uses like globbing and lazy loading. So it's like kind of a hack to be able to get that working with a bundler in first place. Every bundler has a mechanism to like Glob lazy loading and we leverage that. But that is you're almost dipping your toes into like webpack config territory, which is never a fun place to be.

[00:12:39] Joe Colantonio How realistic is this type of testing? Whereas I know a lot of times with certain component testing, it doesn't necessarily render a real Dom that you would get in a real browser. Is this the same type of how realistic is the testing, I guess. If you do it this way, is this exactly how a user would be able to interact with the application in production with the same type of results?

[00:13:00] Moshe Kolodny Yeah. I'll give you a little bit of the motivation where it came from and then you'll kind of see like what it does. So generally like day to day working on like a button component that when you click it, it pops. Let's say what generally people do is they have a page that has the button, they write some code, they go to the page, they click the button, they see the pops. It doesn't pop the right way. They edit the code. They go back to the page. They click the button to see if it pops. That is the kind of the flow of how developer lifecycle works. SafeTest is basically mimics that flow, where it essentially goes to your application in a browser using the actual like whatever your value pointed to, you can give a base URL and it will then open that component, and then it will click that thing and then see if it does. And you could have any assertions you want on it. So it's using a real browser and it's using as close to like what the developer experience is. I'm coming from like a developer background, not from like tests engineer background. So it's definitely more geared towards that methodology.

[00:14:01] Joe Colantonio I guess that's a good point. Like if someone's listening, it sounds like if you're a developer, this would fit into what you normally are used to in your day to day kind of flow, where a tester may not necessarily. It seems like like, who is this made for is I guess, the big question.

[00:14:17] Moshe Kolodny Yes, this is made for everyone. It's just that the evolution of how it was made mimics how of a UI engineer will develop application. But now that it's published and it works, we could essentially throw any use case at it. So like one of the goals of SafeTest is that you cannot describe to me a situation that I couldn't tell you how to make a SafeTest for it. So that is a very bold statement. And I don't know if that's definitely not true for either pure end-to-end or pure unit testing. You make an argument that made me ready just for component testing. But there are situations where I have a service that I'm loading the table and infinite scrolling table. And then it I get to the end. It loads one. I lost another page, I go to the end and then I want to mimic out it errors out. It errors on retry and then the second retry it succeeds, let's say. That is very hard to write a test for, but using like overrides and other methods within SafeTest. That's not easy to write, but it's not like I can explain how to write it. And I don't think there's got to be much ambiguity with how to do that.

[00:15:26] Joe Colantonio Really cool. So how long has this been in the wild for now? Is it been like a month or two weeks?

[00:15:32] Moshe Kolodny Think it's been like probably close to a month at this point.

[00:15:35] Joe Colantonio All right. So have you heard any feedback like you want to believe what I got SafeTests to do for me or are you thought, well, I didn't know it was going to be used that way. But all of a sudden people are starting to do things they like. Oh, that's pretty cool.

[00:15:47] Moshe Kolodny I haven't really heard much feedback in general, I think most of the feedback I get is internally people using it and just setting up and getting all the UX goodies like that, it works in CI and all these things out of the box, which is hard selling point. Also like another thing that does we have our internal wrapper around. It also does like authentication logic for free. Usually all applications have this the proxying magic stuff going on. So you're going to have to deal with that if you're working with like Playwright or Cypress. So this is just three lines of code and then everything's just wired up for you.

[00:16:23] Joe Colantonio Because I know testing authenticated applications could be really tricky. Could you explain one more time how does that work?

[00:16:30] Moshe Kolodny Yes. Every large company I've been at every like large corporate, a company I've been at, always has some solution to how to log into an application to do end-to-end testing. And that usually is a library. And then usually you have to figure out how to make your end-to-end thing integrate with that library. Hopefully there's like some platform that already solved this problem for you. Sometimes you generally for most people that most engineers do not have that luxury, and they have to kind of figure out how to do that themselves. This what we have internally is a like an NPM wrapper that just does all that logic for you. So instead of importing like the setup code from the vanilla SafeTest, one you would import it from the private package. And then that will take care of everything for you. You would use it just the same as you use every other thing. And then it knows how to do authentication and all that. Since we do have the library to handle authentication, it's like a function call to be able to get it off user and then to like get cookies that we set in the browser if you're doing like OAuth or something like that, it's not really a self problem. Like however you would generate a test, you use your login your application, you'd have to kind of redo that flow. And then once you solve that once you could create like a internal package like we have over here. But someone does have to solve that. There's no magic to automatically log into an OAuth application.

[00:17:49] Joe Colantonio Absolutely. So you said this was made for everyone, but is there any like what situation do you think? All right I have Playwright, I have Cypress, I have SafeTest. When should I use SafeTest. Is it always. It doesn't matter. Just use SafeTest or is it like if you have this situation, this is why you definitely want to check out SafeTest.

[00:18:09] Moshe Kolodny SafeTest is basically glue between-so you ask a for like what SafeTest is work in the. I think we kind of like lost track of that. So SafeTest is essentially glue for a couple of different technologies. It's glue for a test runner which is like we use Jest or Vitest for one of those. Then it also takes a browser ordinator, which we use Playwright, and then it also is uses a framework. In examples we use React. But we could use we have Vue, we have Solid, we have Svelte, the whole bunch. And it's very easy to write an adapter to be able to have Playwright integrate with a UI framework. And then it just basically that's coordination between all those different pieces to be able to like have the code running node and the browser and just do that, the shuffling of logic between those things. So since status is wrapping Playwright, it could do anything like a Playwright test could do. It doesn't have some of the goodies that the Playwright UI does. Like the dash dash UI flag when you pass it in, but it does have its own. It could do like its own version of that. If you wanted to build something like that. I don't think it's really needed that much because it's not covering the same use case. But like if there were like testing scenarios, I don't know if one of the mantras of SafeTest is that everything is testable. I couldn't imagine something that's more suited for Playwright over a SafeTest. Cypress may have something because of the UX, and the developer team is very different with the a fluent API. So people may prefer that if you're coming from like a Java background. And so that could be something. SafeTest is definitely not slow, but if you want wanted to, if you have like a thousand tests that just like you're stressing some very like edge case logic thing and you don't need like a real browser for that. Like using React testing library or one of the JS Dom. It's going to be way faster and that would be a better solution in that case. So it definitely there are situations where you wouldn't always use it, but I would say 90% of the time it's a good solution.

[00:20:17] Joe Colantonio I don't know if you can answer this, how much is this used within Netflix? Is it large adoption? Is it just catching on? Is it word of mouth?

[00:20:25] Moshe Kolodny Testing is a hard thing to solve. I would say that people, when they're picking up, when they're solving the testing story, they are reaching for SafeTest. Again, like there's no like hard requirement to do to use anything in particular. We have something called the Pave Path where yeah, we recommend doing things certain way. People are free to do whatever they want. So like a pave path uses like a I think now we were able to use Create React app and then we kind of went away. And now we're talking people the pave path now is recommended to .... but like people if technically you could make an angular application if you want to. I think, it's very much not done and you're going to have issues with getting support if anything ever breaks. But there's no requirement to use any specific tech.

[00:21:13] Joe Colantonio So this is open source correct?

[00:21:16] Moshe Kolodny Yeah.

[00:21:16] Joe Colantonio How is the community so far. Like if someone wants contribute is it easy to contribute. It's just you? Like if someone's like yeah, sounds cool, but I don't want to be buying into something that has no community or no support type deal.

[00:21:28] Moshe Kolodny So it got a thousand stars in like two weeks.

[00:21:32] Joe Colantonio Wow.

[00:21:33] Moshe Kolodny So I think people are like using it a lot. The support issues I see are when things aren't working as expected. And it's usually based on some misconfiguration I didn't think about or some misunderstanding in that I have to clarify on the documentation. When it was like released, I think either that week someone wrote an adapter for solid JS. So like that was like someone contributed that out of the box and it went right in like so I think there's definitely interest. Again, it's really early, like it's been a month. There's really not much history to really deal with. But I think it's pretty positive so far the reception I got. And I hope people contribute. That'll be great.

[00:22:19] Joe Colantonio Absolutely. And we'll have a way for people in the show notes to contribute so they know what to do. So like you said, it's early. It's only been out for a month. Is that there a roadmap is it's pretty much self-contained now. It's good to go to event and you're like, oh, I wish I could add this or that or this is something I plan on adding in the future?

[00:22:36] Moshe Kolodny Yeah. So I have a couple of ideas of things I want to add. So I have like a todo.empty file on the repo. And this is just like kind of like some thoughts I have pop on my head like making like the test runner that Playwright dash dash UI has like something along those lines. We could sort of do that raise since it just lives in the page. And we could lazy load like aspects of SafeTest, we could essentially lazy load in. If you have like a query params set to a full like UI mode and something like that, and then iframe things and then be able to do that, that is technically possible. There's no limitation of if somebody is bootstrapped into SafeTest today. Why that couldn't work. There are a couple of things about using it with Next.js. Oh, so one thing I forgot to mention was that Next.js application will not be able to be tested with any like, component testing libraries because they have they essentially to run their own server, but they have to know how to run an X server. So it could be like they have an export. Some of the other like meta frameworks could be you're at the mercy of the framework, the testing library, people to know how to do that. But SafeTest basically takes a hands off approach. So it's able to just say like, oh, just when you open the application, just go to this page since it lives in the page essentially. That's another thing I forgot to mention. Like a couple things. Just some like developer experience things, maybe making a custom reporters instead of having to glue some piece together yourself and then yeah, just some clean up about the instrumentation of it, things like that. Nothing like major like refactor or rewrites. I think it's the API is pretty stable. It's able to do well with the most sensible API surface.

[00:24:24] Joe Colantonio I may have jumped the gun talking about new features, I forgot about a feature that it's probably already there. How does it help with developer experience for debugging? Because when a test fails, you want to be able to know why they fail. How did it failed? Does it have tracing? Videos like how does that work?

[00:24:39] Moshe Kolodny Yes, that's a great question. The answer is that anything that Playwright has, it also has it automatically gets so it gets videos, it gets trace viewers. It gets like you could replay different aspects of it in the trace viewer and see exactly where failed. And then it has some more stuff on top of that. For example, there's like a SafeTest reporter page that will go through. It has a like a folder structure treeview. And you could open like the trace or the video or you could actually view the component. So if you're testing like a smart button component, you could actually just view that smart button and see. And just like click on like the thing yourself and see what's not working about. You don't have to watch a video. So if like there's some like interaction that's hard to replicate on the deployed application, you could just essentially have the application mount the component that you want to test to be able to ensure, like for example, if there's an error page and this error page test is broken in a prod environment. But the problem is you can't get to the error page and providing because none of the API calls are failing on the environment. And it only fails in the prod environment. So that's a very hard thing to test. But the way SafeTest works, you could essentially pass it in the query parameter. And it will then mount like the error state. And now you could test the error state in the prod environment. And now like you see that the call to the analytic server fails for this error state let's say. So you could do like tricky things like that. You could easily just test yourself. And it gives you like a developer experience test, it's really easy to work with when things don't work properly.

[00:26:16] Joe Colantonio Okay, so this might be a dumb question because it is like glue and it works on top of a Playwright and Jest, when the new version comes out of those libraries, does anyone have to worry about like, oh no, I'm out of sync with the latest version of Playwright? Well, there's always work regardless of what version. Does that makes sense of Playwright or Jest a released?

[00:26:35] Moshe Kolodny Yeah. So I would say that it doesn't use like any of the deep private methods of any of these libraries. So like as long as you can launch a browser and open a browser, it does do some interesting things in that there's you can run SafeTest in Docker mode. So that is useful for if you're doing like image sniffing. Image sniffing will always fail if you're running, if you're creating the diffs locally and you're testing them in CI because you're on a mac and that's on a Linux machine or something like that. So one of the things that SafeTest does is a way to have the browser running in the Docker container, and that will be consistent across environments because it's not running a mac, it's running on something Ubuntu that's pre-baked into the thing. We are able to get that. And that could be like the most fragile piece because that is like relying on a server and a client to connect to it. But everything else is just like calling like the global Jest.subscribe or the Vitest.describe function like that, unless they drastically change the surface of that. And then like I'll just have like a major version bump to match that. So I'm not worried about that.

[00:27:44] Joe Colantonio That's a cool feature. Is there any other features we didn't mention that you think is pretty wild or pretty good?

[00:27:49] Moshe Kolodny Yeah, I say like the most powerful feature is. There's two features we didn't talk about. One was I kind of mention it overrides, which is a very controversial topic, and overrides is really there for when you need it. It's not meant to be the first thing to reach for. If someone comes up to me and says, oh, I have this application that I need to test, but I have this problem. Generally, the solution is use an override. Like, yes, we could like refactor your application to have that as a prop that you the drill down that you do all that. We could do it that way. But like you just want to make sure that if like this prop gets this if this like deeply nested component gets this prop, it'll display a certain way, sort of like using an override or like this service will fail on the specific condition. We want to make sure that works probably in override unless you want to make that component top level. So something in portable top level and then pass in a prop to be able to override that. That's essentially it's not meant to be like the best solution to a problem. It's meant to be the most practical solution to a problem. Just being pragmatic about how we're going to test something. I think that's pretty misunderstood. And definitely like on the blog post and on Hacker News, that was definitely a point that people were focusing on. We wanted to clarify that. The other thing that I think is unutilized and crazy, stupid, powerful, but probably not required, it's almost like more like a flex of the framework is the fact that there's like this bridge mechanism. What we could do is we're able to have the browser reach in, we're able to have the Node.js runner reach into the browser and then have the browser reach into the Node.js environment and have a kind of a back and forth two way communication channel. For example, if we wanted to like only run a test if we wanted the browser to be able to either display the admin page or the user page based on if the user has a password file on the root file system. Like something that doesn't it doesn't make I can't think of a good usage of it because it's just It's a silly use case, but this tweak the channel does enable us to be able to have like browser mocks in the browser and then have in Node.js make assertions that those not recall a certain way, because essentially asks the browser to give me the mock state and then it will create a mock on the node level and then do assertions based on that mock. So that is it needs to be there just to be able to power the library. And we do expose it and we're able to the tests itself. You could like require like modules that will work in the browser. It won't work in the browser, but it'll be a proxy object in the browser. But it'll work in node. So like you could mount a button, you could like get the window.location and that's not using just that's like using like the browser window location and then write a file to disk and then click a button and things like that, you can just it's almost like electron when they bridged like the node layer and the browser layer together, it's like they're glued together. So you can essentially do some pretty powerful stuff. I mean like as with electron, like it wasn't like it's probably anti-pattern to use it too much, but it is there if in case you need it for something.

[00:31:02] Moshe Kolodny I love the flexibility. So that's great. So it sounds like you're not forcing people into like a methodology. It seems like pretty open to do as you wish with great responsibility comes great power type deal. But it's up to you. So I like that sometimes Cypress is very opinionated how you do things. It sounds like this is not an opinionated tool that if I'm right or not, I don't know.

[00:31:25] Moshe Kolodny Yeah, yeah, that's definitely it. Like one of the examples I give is that if you wanted to test a service call, like you have a people's page that loads people, you could choose to mock out the network layer on like the browser level. You could choose to overwrite the fetch layer, you could choose to override the library call, or you could choose to override the specific method on the library call. You get to choose like it's an onion. Going from the component to get the people you can pick wherever you want to be in the onion to kind of change the behavior. So I think that's very powerful. And there's no strong opinions in the library built into the library. Where to go with that?

[00:32:02] Joe Colantonio Awesome. Okay, Moshe, before we go, is there one piece of actionable advice you can give to someone to help them with their automation testing efforts? And what's the best way to find contact you or learn more about SafeTest?

[00:32:14] Moshe Kolodny Sure. Okay, so the best advice I would give is just start writing tests and start doing it. Try to be as pragmatic as possible when you're doing it. So don't try to build a meta testing solution before you actually build test. that is definitely an engineering dilemma where you're trying to abstracted with a problem, but you didn't actually before you actually get to the problem. Have a use case, write a test for it, then have another use case write test for that, iterate fast, do quick test, and be pragmatic. You don't have to get 100% coverage if it's going to take five times longer, write four tests in that same time. And just pragmatism is definitely the way to go when it comes to writing tests and software in general. The best way to contact me is going to be on GitHub, like open an issue. You could reach out and stuff like that, open a pull request, happy to have contributors and people collaborating on that. And I'm looking forward to community engagement and seeing where it goes.

[00:33:13] Thanks again for your automation awesomeness. The links of everything we value we covered in this episode. Head in over to testguild.com/a489. And if the show has helped you in any way, why not rate it and review it in iTunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe, my mission is to help you succeed with creating end-to-end, full-stack automation awesomeness. As always, test everything and keep the good. Cheers.

[00:33:49] Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps, and automation professionals in the world, we'd love to have you join the FAM at Testguild.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real-world value that can improve the skills or solve a problem for the Guild community. I love to hear from you head on over to testguild.info And let's make it happen.