INGenious Playwright Studio with Ashish Ghosh

[00:00:00] In a land of testers, far and wide they journeyed. Seeking answers, seeking skills, seeking a better way. Through the hills they wandered, through treacherous terrain. But then they heard a tale, a podcast they had to obey. Oh, the Test Guild Automation Testing podcast. Guiding testers with automation awesomeness. From ancient realms to modern days, they lead the way. Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold. Oh, the Test Guild Automation Testing podcast. Guiding testers with automation awesomeness. From ancient realms to modern days, they lead the way. Oh, the Test Guild Automation Testing podcast. Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold.

[00:00:35] Joe Colantonio Hey, do you want to learn how to really take your Playwright automation to the next level? Well, you're in for a special treat today because we've been talking with Ashish all about INGenious Playwright Studio and quality assurance at a large enterprise bank. If you don't know, Ashish is a seasoned test automation architect with over 12 years of experience, he has been instrumental in taking testing to the next level, ING Bank, where he leads the Test Center of Excellence. I haven't spoken to someone at Test Center of Excellence in a while, so really excited about that as well. Also, he is the creator of INGenious, a low code test automation framework which I think you're going to get a lot of value from, and he is really passionate about advocating and sharing his knowledge, which you'll see in this episode. You don't want to miss it. Check it out.

[00:01:19] Joe Colantonio Hey, Ashish. Welcome to the Guild.

[00:01:20] Ashish Ghosh Thanks, Joe. Thanks for having me.

[00:01:25] Joe Colantonio Really excited about this. I saw this on a few weeks back and had it on my News Show. Got a lot of interest in it, so I guess we'll just start off with could you give maybe an overview on what is INGenious? What are the key problems it's designed to solve?

[00:01:39] Ashish Ghosh Yeah, sure. So effectively, when we were looking at the testing problems within ING Bank, one of the things that came out was the lack of involvement of business users. Now, technically testing is all about business requirements. I cannot just go around testing any random stuff of the software unless I know what the business that the software is supposed to fulfill. Now, the problem with coded solutions was that a lot of customer journey experts, as we call them in ING, did not find a lot of interest. They could not contribute, they could not collaborate towards creating good quality tests, automated tests. The other problem that we had was in ING having a commercial solution like Tosca for example, or Katalon. It's a bit of a nightmare, of course, with the license cost and ING's quite heavy on open sourcing. What we thought of doing is, there's a very big community in the outside world with Selenium. How about we build a framework on top of it which makes automation with Selenium and let's say for APIs or HTTP client, and then wrap it around so that the end users or the customer journey experts could really understand this. We did that. And that was the inception of INGenious. And this happened some four years ago. But then as technology evolved, as modern web came into picture, ING's web frontend had special qualities which, even though I am a very big fan of Selenium, I have to say we were facing troubles with it. And then came Playwright. And four years ago, in fact, I was the one who introduced Playwright within ING. And we thought, let's tweak this a little bit. Let's give Playwright a chance into the INGenious framework rather than Selenium. And it worked wonders. So now we have a situation where our business users, or our customer journey experts, are able to write good quality automated tests and at the same time, leverage the power of Playwright. And that's a humongous thing for us.

[00:03:49] Joe Colantonio All right. So I know a lot of people start off with Selenium, we're not saying Selenium a bad tools. Depends on your context and your situation. I'm just curious in your context, what were the problems or what were the areas that Playwright did better than your implementation of Selenium?

[00:04:03] Ashish Ghosh Yeah. So one of the things that immediately comes to mind is piercing shadowroots, for example. Now, we have a lot of web components, which are a lot of security layers are built on top of it, and it's a reusable set of components, templatized, let's say. And it's used across ING. These shadowroot piercing within Selenium. I have done this with JS spot, but it's a nightmare. If you look at how a customer journey expert would look at it. There's so much of dynamic IDs that get generated which had to be parsed into this JavaScript, and that was horrible. With Playwright, using something like a label or a text, it just pierced it seamlessly. So that was number one. Number two, we have a lot of intranet applications. Homegrown applications of course we have applications which support customer. So customer facing applications. But 80% of the applications are homegrown. There is a decent amount of latency slowness in rendering. And with Playwright's automating capabilities, it was like moving a knife through butter. These were two major aspects that I would say Playwright won over Selenium.

[00:05:13] Joe Colantonio All right. I guess the next question is, a lot of times people just go to creating a new framework. There are a lot of frameworks out there like WebDriverIO that helps with the piercing of ShadowDom, or anything Serenity. Why go the way you did with creating your own framework then?

[00:05:30] Ashish Ghosh If you look at Serenity or for that matter, WebDriverIO and you build a layer on top of it through I don't know, something that helps communicating with customer journey expert. You would probably think of BDD, Behavior Driven Development so you'd have a Gherkin layer, but then somebody had to write the Glue code. Somebody had to still come in and write the Glue code for you to be able to make it more effective. So here we were, looking at a setup where what if we eliminate the need for writing Glue code? What if the customer journey expert just comes in simply write, click on this button, click on that button and do an assertion. And an actual Playwright script comes in and does the assertion for you. Then it eliminates the need of an additional handholding. There is a very seamless integration between requirements straight out of the business. And then it is immediately translated into good quality automated tests. And of course, it's too good to be true if you speak it in that way. There has to be a decent amount of handholding. Initially, maybe the first couple of weeks, but after that, they're on their own. In fact, just now, before I was coming into this building for this interview, there was a specific segment that was dedicated where customer journey experts were actually sharing their experiences of using INGenious. That was really awesome.

[00:06:56] Joe Colantonio That's cool. All right. I want to dive into that. I just want to follow up. A lot of people also say, I want to go to Playwright I'm on Selenium. Obviously, you're at a large bank. You must had a ton. I would think of a ton of Selenium scripts. How were you able to migrate from Selenium to Playwright? And was that baked into the original framework to make it so you could just swap out libraries behind the scenes and no one would even know?

[00:07:19] Ashish Ghosh Yeah, exactly. That's exactly what we did. The layer on top remained as is. 80 to 90% of the work was just reused. The way we have created the framework is engine underneath that change. For example, if I say browser open and then a URL instead of Selenium with driver.get, we now have Playwright.launch. So yeah. So nobody notices the difference to be honest. But I won't say that is the case. For 100% of the cases 20% we had to alter.

[00:07:53] Joe Colantonio Gotcha. All right. You mentioned how it's easier for business users. A lot of times people like you said use like Cucumber or BDD is the front end. Are you doing the same. What's different with this framework?

[00:08:06] Ashish Ghosh This framework has the capability of importing a Gherkin file and then creating reusable components. But then when these reusable components are step definitions, as we call it. Come into the INGenious IDE. Somebody as I explaining, somebody in a traditional framework would still have to come and do the coding, create that Glue code. Here, we have eliminated that. Of course, we strongly advocate the use of reusable components. But do we really necessarily have to code those components? I don't think so. And that's what INGenious shows you that you can just drag and drop data, parameterize it, and then you have it an enterprise grade automated tests. But I must tell you something, Joe. The use of BDD, I mean, not just BDD, BDD is very good, but the use of Cucumber was is not very mature. But in ING and for that matter. I've seen a lot of different implementations everywhere. There's a lot of implicit usage, explicit usage, and it's not uniform to treat it as a test automation framework, but it is not supposed to be one. It was all about collaboration. It was all about having business requirements brought in, converted into tests. Yes. And that is the philosophy that we wanted to propagate through INGenious.

[00:09:28] Joe Colantonio Yeah, I agree, it's just extra overhead when you're just using it as an automation framework rather than a communication collaboration. Is this more like keyword driven then?

[00:09:36] Ashish Ghosh Yes.

[00:09:37] Joe Colantonio Like I don't know what a banking feature is, but like add account blah blah blah or some.

[00:09:42] Ashish Ghosh Yeah. So it is hybrid basically. It is keyword driven. You can just simply treat it like a keyword driven framework. You don't have to use Gherkin overlay. That's perfectly fine. That's one way of using it. The second way is you have the Gherkin overlay, and then you treat it as a behavior driven. And the third overlay, third type is a data driven format. Where you have all these automated tests, you just automate it once and then manipulate the objects, manipulate the data entry by making it data driven so that all your different business test cases are then fed into the script using these CSV files, which are already pre-loaded into the INGenious IDE.

[00:10:24] Joe Colantonio Nice, you didn't mention like tools like Katalon, Tosca. When you work for a large enterprise. This is huge. ING Bank. I worked for health care, usually testing more than a web application. Like, it's not like a rinky dink. I'm not putting down any other things testers test but like, this is like enterprise grade, mission critical audited type of software, you need a test. How did you get over that? Because like things like Tosca or UFT have they automate more than a web browser. How did you, I assume you're testing more than web browsers. And if you are, how are you getting over using just Playwright then?

[00:10:58] Ashish Ghosh To answer your question, Joe, if you look at the documentation of INGenious Playwright studio and if you scroll down to the architecture, it basically says that it uses Playwright for browser testing, but it also uses Java HTTP client for API testing. But this was built at a time when INGenious was still having Selenium as the browser testing tool. But now we are also going to use Playwright for API testing, and it's still in works and it's going to be released very soon. We also use JDBC and ODBC connections for databases. We are also doing accessibility testing. And we are also integrating Appium for mobile testing. So within a bank, it's not just the browser, the browser is just the interface. And to be honest, we have multiple such interfaces. The API interface, the browser interface, we have fat clients and we are using Winamp driver, which is also a Microsoft library for automating Fat clients. We have Java messaging queues, we have Apache Kafka and all these different technologies in addition to databases and traditional file based validations. We needed a framework which was able to handle all of this. All of these different capabilities had to be plugged in. Additionally, I would like to say we are not using the JavaScript or the TypeScript version of Playwright which is hugely popular. We are using the Java version of Playwright. We had to have Java as the programming language because we were catering to different tech. Now, good luck with testing a Java messaging queue with a JavaScript based framework. That's not going to happen. That is why it was a very conscious decision to go with Java.

[00:12:43] Joe Colantonio All right. If someone wants to this is open source.

[00:12:46] Ashish Ghosh Yes.

[00:12:46] Joe Colantonio If someone want to use this so do they need to use Java or is that just if they need to create their own methods, they would have to code it in Java behind the scenes for their users.

[00:12:57] Ashish Ghosh That is exactly what it is. If there is any special method that has to be coded, they have to just open up the Java engine, which is nothing but a Maven Engine. Just simply write your custom code or whatever and then do a Maven clean install. And there you have it. All of Playwright's actions are basically plugged into it, so they don't have to use any special coding for Playwright, but they might have to write something for their program, for their application, which may use some special logic.

[00:13:29] Joe Colantonio Is there anything that would prohibit someone from running this at scale? Can you run the framework like against like a browser solution?

[00:13:37] Ashish Ghosh Yeah. This is very easily integrated with lambda test with the BrowserStack and SauceLabs. And we have done these experiments over and over again. Within ING, we have our own CI solution. We are using Azure DevOps. And then we just run these tests at scale. The whole purpose was to make sure that this is a Java solution. It runs via command line, and it can be triggered into as many parallel threads as much as possible. But then the only limitation is the agent on which you are running. If the agent allows you to spin up ten threads, then yeah, very good, but it only allows you to spin up two threads. Then that's where you are limited.

[00:14:17] Joe Colantonio Gotcha. You mentioned the different types of technologies happening under the covers. And I assume it is all into the covers, like someone would just use the same keyword driven thing. I don't know what the keywords are for accessibility, but whatever it is, you can mix and match.

[00:14:30] Ashish Ghosh Yes.

[00:14:31] Joe Colantonio The web and then do an accessibility type action, all in one flow?

[00:14:34] Ashish Ghosh Yes, we even have built a set up where you write your functional tests and then, within the functional test you can actually use a keyword like capture page timings, which basically captures the complete Dom load time and resource load time and everything and generates very nice visual representation. All of that is documented in the documentation that I showed you, and we have tried to make it very user friendly. Of course, open for feedback, and if the community feels that there is more that can be added or something can be made better, more than happy to look into it and work together with the community to make it better.

[00:15:10] Joe Colantonio All right. So I guess like how do you envision the open source piece? It is all open source is not a piece. How do you envision the open source community contributing to this then?

[00:15:19] Ashish Ghosh Yeah. One of the interesting things that happened very recently was Andrey Lushnikov was the founder of Playwright was in the ING office. And we got up for coffee and then he said, hey Ashish, show me INGenious. And I was a little nervous. And this is like two days, barely two days after INGenious went open-source. And I was hoping and praying that, okay, if it has to go wrong, please don't let this be the occasion that it goes wrong. But in one hour, he was able to just navigate through it, create his own tests, run it via GitHub actions. And then during that process, he shared a few feedback. For example, making some options like browser capabilities like set headless or start maximized as dropdowns. He is our very first contributor to the open source project. What I genuinely hope from the open source community is to try it out. Open GitHub issues. Create pull requests if you have to the two projects, basically. The documentation project and the INGenious project in its totality. Just feel free to contribute to it. Suggest ideas. This is one solution that you don't have to pay for, and yet you're getting the perks of different types of testing. So yeah.

[00:16:35] Joe Colantonio You brought up a good point. A lot of times people think contributing to a project is just coding, but actually opening up bug defects and things like that is helping the project because you let people know there's an issue in getting them resolved that way.

[00:16:47] Ashish Ghosh Absolutely. We have created a discussion forum on GitHub as well. If there's anything that you would want to discuss, even just create a new discussion point and then we start talking about it. One thing that I would definitely like to add for the folks out there, this has been use within ING for four years now. Of course, four years back this did not have the same level of maturity that it has now. So over a period of time, there has been a lot of improvements from the feedback that has come from the internal folks. And I think this is a product that you would really enjoy working with.

[00:17:23] Joe Colantonio And that's a good point because like I said, it's you're not testing toy applications as a true enterprise app.

[00:17:30] Ashish Ghosh Exactly.

[00:17:30] Joe Colantonio And I just asked ChatGPT, how big is ING it as a 38 million individuals and corporate clients across more than 40 countries ING serve. So it's like I said, it's been tested against real world applications. It sounds like.

[00:17:44] Ashish Ghosh Exactly. I was doing a talk over in a conference earlier this year. At that point in time, I was not at a liberty to share the name of this framework. And now I can, because now it's open source. And one of the slides basically showed where are the places within the world where INGenious has been implemented. It ranges from Belgium, Netherlands, all these different countries in Europe to the U.S, Singapore, Philippines. Yeah, everywhere.

[00:18:11] Joe Colantonio That's awesome. How did you get them to do open source? I would take a big company like this. We have so much red tape, I'd be like, yeah, I'm not even going to go there. Like why? First how and why did you get it open source?

[00:18:22] Ashish Ghosh Yeah, I think this is a very good question. And I think it also has to do with the amazing leadership that ING has. And the culture within ING is a culture of collaboration. When I first pitched that idea to the head of engineering and culture, he just jumped onto that idea saying, hey, this is one solution that if it has already added so much of value within ING, then this can only grow bigger. This can also add a lot of value to the open source community, but at the same time, contributions will help make this a better solution within ING itself because now we don't have two different forks, we just have one solution which is used by ING and used by the outside community. And it also shows a very open mindedness that is basically the essence of ING, to be honest collaboration and open mindedness. And the other factor was the process was not an easy process. I had to go through a legal review. I had to go through multiple security scans. I had to go through a lot of reviews. Eventually, the open source body basically approved it, but that's not the end of the story. I couldn't have just released it without the proper documentation. And that documentation also took a lot of effort to create to have the right ING flavors, the colors, the fonts, and then eventually it was out.

[00:19:48] Joe Colantonio Awesome. So I know it's been around for a while, but you kind of just open sourced it maybe a few weeks ago, if I understand correctly, a month or so?

[00:19:56] Ashish Ghosh Yeah. That's right.

[00:19:57] Joe Colantonio I know it may be too early to talk about future roadmaps, but a lot of you know, AI is the rage everywhere. Any plans to incorporate AI-driven automation? Is that because you have to probably rely on a model that's paid that is not on the roadmap.

[00:20:11] Ashish Ghosh Yeah. That is true. So as you mentioned in the intro, I am leading a Center of Excellence. One of the pillars within the Center of Excellence is the innovation pillar. And where we are basically strongly looking at the usage of AI. There have been few use cases within the bank which give us a very good confidence that very soon we will be able to leverage the AI capabilities within INGenious. We have the roadmap. We have the plans, we have the different use cases mapped out. But because it's not official yet, I am not at liberty to discuss that. But it's definitely on the cards. It's going to be very exciting, Joe.

[00:20:51] Joe Colantonio So very, very cool. All right. We mentioned one of the key features here. It leverages other technologies to help you do other types of testing. And we'll have this in the show notes a link to the documentation. But does it also help you with DevOps in general? Like does it do any type of monitoring or is that outside the scope of what you want this to be used for?

[00:21:11] Ashish Ghosh No. This is strictly meant for functional testing as of now. We are basically leveraging the capabilities towards generating performance tests as well from functional tests. We are creating a feature now where you just right click generate K6 scripts. And then it will generate a performance that scripts for K6 runners. But it is not meant. At this point in time. It, it doesn't do any kind of a monitoring. But we also have a feature where it basically links it directly to Azure DevOps test plans, which is meant for the test management. It's also documented in the link that you're going to be sharing. So yeah, no monitoring now. But that's a project that's also, yeah, being built in parallel.

[00:21:56] Joe Colantonio Starting to sound like a Joe wish list. But how about test data? I assume at a banking company? That's something you probably like. Oh my gosh. We have to create data and it has to be compliant and not using real information. Anything built in around that? Or is that something you could see happening?

[00:22:11] Ashish Ghosh Yeah yeah yeah. Very nice question. There are two things that already there are a few capabilities which are built in. Generating random strings, random numbers. But those are basic utilities. In terms of test data, we have so far always been reliant on relevant business data. We never gave it a thought. But since the day we have open sourced it. We gave it a real hard thought and we thought, hey, we want to integrate Java data faker library so that the end users could generate synthetic test data directly just by creating some keywords. It's there. It's also going to be release soon. And because you talked about data, that is one of the use cases that is being discussed for the AI.

[00:23:01] Joe Colantonio All right. You have really good documentation. You could tell you worked a lot on it. Is there any other resources you're working on to help people maybe get up to speed quicker with your framework?

[00:23:09] Ashish Ghosh Yeah, sure. In fact, one of the feedback from Andrey was, can we have video tutorials in addition to documentation? And that is something that we are working on. We have created a very nice playlist and very soon it will also be made available to the general audience.

[00:23:27] Joe Colantonio Oh really really excited. All right, Ashish, before we go, is there one piece of actionable advice you can give to someone to help them with their Playwright automation testing efforts, and what's the best way to find contact you or get our hands on INGenious?

[00:23:39] Ashish Ghosh Yeah. So one of the only thing that I would say is, if you are at this point in time are not using Playwright or are using Playwright and are not very savvy with coding, but you want to write real good automated test cases contributing to your business. Try out INGenious. It's super easy, super fun. And if you have any questions, just please reach out to the GitHub page. Just create an issue or just start a new discussion. And we are really active there. We are really looking forward towards your contribution and we would immediately jump in and start a nice discussion.

[00:24:16] Thanks again for your automation awesomeness. The links of everything we value we covered in this episode. Head in over to testguild.com/a512. And if the show has helped you in any way, why not rate it and review it in iTunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe, my mission is to help you succeed with creating end-to-end, full-stack automation awesomeness. As always, test everything and keep the good. Cheers.

[00:24:50] Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation, testing, and DevOps. If you haven't yet, join our vibrant community at TestGuild.com where you become part of our elite circle driving innovation, software testing, and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate. Visit TestGuild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.

[00:25:33] Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold.