Tackling Enterprise Testing Challenges with AI with Srirang Srikantha

[00:00:02] In a land of testers, far and wide they journeyed. Seeking answers, seeking skills, seeking a better way. Through the hills they wandered, through treacherous terrain. But then they heard a tale, a podcast they had to obey. Oh, the Test Guild Automation Testing podcast. Guiding testers with automation awesomeness. From ancient realms to modern days, they lead the way. Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold Oh, the Test Guild Automation Testing podcast. Guiding testers with automation awesomeness. From ancient realms to modern days, they lead the way. Oh, the Test Guild Automation Testing podcast. Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold.

[00:00:35] Joe Colantonio Today, we'll be diving into what I think is some of the biggest challenges facing test automation teams in enterprise environments. And that's scalability, maintainability, and the ever-changing complexity of modern applications. For ways on how to solve these big three, we have Srirang joining us from Tenjin, a platform that's redefining enterprise test automation with AI-driven intelligence. If you don't know, Srirang has over 25 years of experience in delivering business growth, technology innovation, product strategy, and development with the focus on banking, financial services, and the insurance industry. It really knows the problems that enterprise testers have to face day to day. It's also an acknowledged industry leader in addressing inefficiencies and quality engineering and promoting application reliability as a key driver for business success. While we get into it, this episode of the Automation Podcast is sponsored by the awesome folks at Tenjin. Tenjin is an all-in-one test management and codeless automation platform that simplifies software testing with advanced AI capabilities, helping teams to plan, design, and execute tests with ease. Why do testers love Tenjin? Well, first, they have an integrated test management solution, seamlessly managed test cases, requirements and defects in a single, unified platform. They also have AI assisted test design which uses natural language in pre-existing documentation to quickly create tests with AI powered automation. They also give you access to tracking and reporting to gain full visibility into your manual and automated test execution with detailed tracking and analytics. It also comes with ML driven execution optimization, which helps you to optimize parallel test execution based on data and resource constraints to save time and maximize efficiency. And you can use them to help you automate testing across multiple platforms so you can easily test the web mobile API and databases all in one platform. To support the show and also discover for yourself how Tenjin can transform your testing strategy. Sign up today and experience enterprise grade test automation built for modern software teams using the link down below.

[00:02:42] Joe Colantonio Hey Srirang, welcome to The Guild.

[00:02:44] Srirang Srikantha Hi Joe, thanks for having me. It's a pleasure to finally talk to you.

[00:02:47] Joe Colantonio Yeah, I love having you. When I started my career, I was in an insurance company, and then I went to a medical company. So I have a really great appreciation for enterprise testing. I think some people miss out on. So I guess the first question I have is, and maybe I'm wrong. Why do you think enterprise testing is so hard?

[00:03:05] It is hard because Enterprises today are net buyers of technology. They are not digital innovators, but I would like to call them digital adopters. They typically implement commercial off-the-shelf software. They don't have access to the backend. They don't have access to the programming talent that development companies have, DevOps have, and platforms in enterprises are large. They are fairly large in terms of size and complexity. and it is in the far end of the implementation cycle where you are actually implementing the platform, you're not building the platform and you have to work with the technology that the vendors give you and all of this compounded with the fact that you have so many pieces of technology in the modern enterprise. Modern banks can have between 70 to 400 different software that runs the modern bank and even other industries we are dealing with, at least, 10 to 15 different pieces of major software that enterprises have to put together and stitch together to manage their business workflow. And building and maintaining automation across such a complex environment is always a challenge.

[00:04:21] Joe Colantonio Yeah, I'm sure a lot of people listening definitely agree. And I also think the problem or the idea of test automation, a lot of these enterprises have told them we need to deliver faster, we need test automation, but it's almost like a Catch-22 because they try to implement test automation and almost becomes a bottleneck because like you said, you need to jump over 10 to 15 applications, you end up a lot of flaky tests, long execution times and like massive maintenance efforts. How does this impact release cycles and overall software quality? How do you make it so automation then is no longer a bottleneck?

[00:04:51] Srirang Srikantha When we started Tenjin, that was one of the biggest aspects that we wanted to solve. How do you quickly incorporate changes in the application without having to spend too much time figuring out what has changed, where has it changed? That was one of the first things we set to solve. We built a robotic discovery, UI discovery process. We did a few things which we feel are very innovative. The first one is that we built a robotic UI discovery process. Second is we separated the test with the UI navigation which I think is a very fundamental different approach that we took. You could write your test cases, you could put together complex test cases that go across different parts of your application landscape, different parts of a single application, and this is not closely tied in to how you do your screen discovery. The other way to look at it is you could build screen navigation. You could build application navigation within an enterprise without having to code all of that as one monolithic large script. You could do it a modular. You could join all of these together at runtime. You could have different work paths that you take depending on your business function, what you need to test out. And these two innovations have really helped us scale test automation for enterprise platforms.

[00:06:18] Joe Colantonio Great. I know also that you've invested in AI technology. In the pre-show, we talked about how you mentioned, there's a lot of tools coming out with AI, but they really aren't enterprise grade, let's say, because they don't have the experience of companies like yourself that actually knows how to deal with certain things. A lot of times it's just AI fluff. What separates maybe your AI and your product to what maybe others might have?

[00:06:42] Srirang Srikantha Yeah, the world is really revolving around AI these days. And when we started off, the typical use cases that we saw in the industry which says, can I read a Jira script? Can I create Selenium scripts out of AI? We felt that these use cases don't really apply to our customer segment. A large portion of our customer segment is financial services regulated industry. They're very cautious adopters of new technology. And given the fact that it is enterprise platforms, you're not typically dealing with your epics or scripts. You're dealing with a large business requirement document or a complex platform discovery document. We wanted to put AI to work in those areas. We looked at three things which we would impact. First is, can we make test design simpler and not dependent on a subject matter expert? Second is, how do we remove the problem of flaky tests? And what we set out to do is saying, if there are changes in the UI, is that something that AI can help us self-discover, self-heal without having to go back and redo a lot of work. And third is start building correlation between defects in previous release cycles and seeing if we correlate and predict. And prioritize what tests you want to run in a subsequent release cycles. I actually have an interesting demo if you'd like to see that on test design.

[00:08:15] Joe Colantonio Awesome. Absolutely. And for the people that are listening in, just only we'll have a link to the video as well, but we'll walk you through it as well. So you could follow along for people that are just listening.

[00:08:28] Srirang Srikantha Sure. This is a demo about, this is a video that talks through our test design process. And it starts off slightly differently where you would work with a large requirement document, which is a word document, which is 30, 40, 100, 200 pages long. It kind of lists out what do you want to configure? How do you want the platform to behave? And this document would typically look like this, it could be a user manual. It could be a requirement document, which is a lot of text. It could be images. It could be workflow diagrams. And when you set that up, you can upload these documents to our platform. And what it will quickly do is break this up into broad business areas in terms of what are the different sections within the platform. And here you see as generating. So you see it breaks up the document into 21 different processes. And if you see on the right, there are no scenarios or cases associated with that. So we'll choose one business area or a business process and ask it to generate scenarios. And at this stage, it goes back to the LLM, it shares the documentation, it breaks it up into scenarios and you get one scenario you can like or ask it to improve the scenario and If you really like it, you can say, I like the scenario, let's go ahead and build cases out of it. And as you see it built five cases, you can review it again. And if you like this, you could take it into your test repository. That building something like this would otherwise require a subject matter expert. It may take three weeks of time for you to go through such large documents. But with Tenjin today, you could munch through such documents within hours and have your test design. And a very neat mind map delivered to you. So this is on test design.

[00:10:30] Joe Colantonio How do you know it doesn't hallucinate based on what you feed it as well? I know a lot of people worry about hallucination. I know you're training it on your internal documentation, but how do you know it's really doing, it's really understanding those documents? Does that make sense?

[00:10:44] Srirang Srikantha Yes, it does. What we've been doing is, so we have a large services team which helps customers, we take on real world projects. This is not just a factory. This is not a built factory alone. We have a large service delivery team. We also have partners who help us deliver our platform and deliver testing services associated with that. For the past six months, we've been doing real world trials. It's not just based on sample documents, it is taking it to customers, doing a trial. What we have found is that we cannot 100% rely on this, but what it really does is cuts down the time for initial delivery. We are able to turn around documents quicker and our teams, the testing teams then move from people who are writing tests to people who are reviewing and enhancing tests. They go from having to write everything rounds up, to actually reviewing it. It's a high level skill that they need to build and maintain. It also frees up their time in terms of, turnaround clients and so on. That way, we found it to be very helpful. It does bring down the team requirements from our side. We've seen more than anything we've seen reduction in turnaround time by about 20 to 25% in test design for us, which I think is a substantial benefit whichever way you see it.

[00:12:15] Joe Colantonio Even if I have a really good test document and enterprise environment, like you mentioned, there's 10 to 15 things that could interact with. It was always hard to get an environment that was set up correctly. I had to go to all these people in the organization that I knew their little piece to set up the application and make it so I had an environment where I could test in. Does this help at all with that? Does this pop up? Hey, this test does this, but you're going to need this and this configured to do it.

[00:12:38] Srirang Srikantha Right now it is based on documentation given to us. If the documentation consists of the prerequisites or the environment requirements, then we would also be able to give you the prerequisites over there. But right now, a lot of our focus is essentially around getting the test, the specific validations, the data entry and the validations correct. And we will continue to enhance this because this is from an AI perspective. We are seeing rapid changes. When we launched this, it was much less functional, but as the new models of ChatGPT and other platforms have come out, we have seen rapid improvements in the quality of output and what all we can ask it to do. And so we continue to invest and see how we could improve this whole experience.

[00:13:32] Joe Colantonio You mentioned a few times you could do this without having an expert, a testing expert or a subject matter expert. A lot of people I've been speaking with, testers are really concerned about AI replacing them and they might be hesitant to actually implement these tools. Are you seeing any pushback on like, I'm not even using AI because of that, that claim or are you not making that claim in my misunderstanding?

[00:13:54] Srirang Srikantha No, I don't think we will come to a stage where we are saying we are going to replace in the near future. What we are seeing is it just makes me more efficient. It helps me look at stuff which I might have missed. Humans are not really the best people to look at 100 or 200 page document and figure out one tiny detail. AI is really well suited for such tasks and the idea is to make humans more efficient. It's no way close to replacing as yet, thankfully, because as I said, we are only getting about 70-80% accuracy right now and there is a lot of review work that we recommend and we undertake as a part of our testing design. I don't see pushback coming in terms of saying we are going to replace people.

[00:14:43] Joe Colantonio Another, I think, benefit of AI is it can help with things like automation, or like healing, self -healing and self -discovery. Maybe you can give us a little taste of what, Tenjin, how that handles that piece that a lot of people struggle with.

[00:14:55] Speaker 3 Yes. Tenjin is designed to work with business analysts, it is meant for business analysts with no programming experience. When we built a no-code test automation platform, one of the first things that we had to solve for was what happens if your application under test goes through a change? How does a tester and incorporate those changes into the automation. And that was a large problem, and that remains a large problem. There are different innovative ways to solve it. And here is an example of how we have solved for elementary discovery. On screen, you see us setting up Tenjin to start looking or discovering an application menu. We have multiple ways of discovery. When we say automated learning, we provide a node where it has to go and discover. It launches the application. It navigates to the menu specifically. And it browses through the entire DOM. It browses through the entire screen, discovering all the fields and pages and presents that back in a very neat and understandable way. And all the technical details are three screens in, but it's available for anyone who wants to work with that. But for most people, you don't need to do that. It just is, it gives you template which you can download and the template represents your UI and that's where you do your data management. This is and if you have changes all you have to do is rerun it and the template gets it, it adjusts to the new fields or changes in these sequences of fields. This auto discovery process by Tenjin has helped us bring test automation to citizen tester, someone who is not necessarily comfortable with programming, but someone who understands the business domain strongly and is able to write functional test cases on an end user environment and test out those platforms. I hope that is of interest to the users over there.

[00:17:05] Joe Colantonio This has been the holy grail of a testing for a while or the big promise of anyone could do automation, even the business users. In the wild, how is this really working for you? Do you have people that aren't say technical, not saying they're not technical, but say they're not automation experts. Are they able to use this and how they're using it?

[00:17:25] Srirang Srikantha Yes, we have multiple customers who have adopted this and have been long standing customers for us for the last six, seven years who've been using the platform and the capabilities and the auto discovery, self-healing, and so on. AI is the new piece, but the auto healing piece, the self-discovery piece has been up and running for about six, seven years, and we've been pitching it predominantly into banking and financial services, but come 2025, we have launched this on a public SaaS, as well as available as an enterprise platform for you. And listeners of Test Guild can access this on tenjinonline.com. And that's the first place where you can go and experience the platform. And it's been very well received. We have had multiple customers who have been very happy with it. And the great part is it's not just web. This is through for web. This is through for mobile. This is through, mobile as in Android and Apple platforms. It is also available for APIs and recently we have also enabled it for doing database testing. You could actually have, you could chain your tests to go through multiple platforms within your environment, starting from a UI all the way to the database and everything else in between. And you could retain context and report of it as a single test. And that I think is very powerful.

[00:18:56] Joe Colantonio Yeah, absolutely. And that's, I'll be honest, that's one of the things that resonates with me because that's why I never really got, people won't like this. Why? I really didn't get into Selenium is because I needed something that could go to a web, a mobile, do an API, database, all in one platform. It just made things a lot easier. This sounds like this is something that for the enterprise, that's like a perfect use case for your solution.

[00:19:18] Srirang Srikantha I hope so. We've been working with a lot of customers and the thing is, we still feel that we are at very early stages of the product development. Despite everything that we've built, we are looking at how do we enhance this? How do we make this better? Some of the cool things that we are launching in the next few months is auto-scaling the ability to run this on a public cloud or on a private cloud and have it scale automatically based on the workload that you assign to it, it has very interesting capabilities around taking a large workload which has not been manually distributed and distributing it across available infrastructure. That we will launch by the end of March. These are things which we are doing on a monthly release calendar that we have and we continue to enhance it and build on it every month on month.

[00:20:17] Joe Colantonio If someone's using another solution for browser, like in the cloud type of a browser kind of OS mobile things, can they run it against these browser farms? If they want it to, is this going to be all in-house in Tenjin?

[00:20:29] Srirang Srikantha No, currently we have worked with BrowserStack, we've worked with Sauce Labs, we've worked with Amazon Browser Farm, AWS Browser Farm, and we are talking to a few others and we will extend it to the other platforms as there is a customer requirement. It just gives you access to everything from browsers to devices that these people, these providers host, and that I think is a very powerful benefit to clients who don't want to invest and who have to do a lot of compatibility validation and we should be able to help with that.

[00:21:06] Joe Colantonio Absolutely, just in case the folks that missed it, know we talked about self-discovery piece and you showed it. How does that help with test maintainability?

[00:21:13] Srirang Srikantha So self -discovery, we built it because discovering the UI, discovering the UI is extremely difficult. We've always relied on scripting or point and shoot discovery. Point and shoot discovery is very intuitive and it's great for the first run. When you're first building automation, all these techniques are very interesting, very intuitive. The challenge doesn't come in when you're first building your automation. Challenge happens when you have to maintain automation over cycles, periodic cycles. Every time your application undergoes a change, if the UI changes, how do you incorporate the change into your automation? And that is where self-discovery helps because what it does is it takes away the burden, it takes away the effort, it takes away the entire challenge of identifying all your work elements, all the elements on the UI from a human to a bot. We know that bots are more consistent, they are tireless, and what we are able to do is go back, navigate through the UI, follow the path, and discover all new elements. We are also able to take calls in terms of if there is change in element locators or names. Given the precision with which AI is working, we are able to take a few calls and replace element names or locators with if they have changed, we will replace that using discretionary paths within an AI, within an AI algorithm. We will use discretionary paths within an AI algorithm to replace elements. And that makes the whole process painless. It makes it friendly to someone who's not programming ready. That's what we started out doing. And I think we continue to invest in making that process simpler and easier for our users.

[00:23:12] Joe Colantonio Absolutely. And I may have missed this, so you may have mentioned it. I know a lot of times people use ERPs when they're doing an enterprise. I assume you have like pre-built accelerators maybe for some of these enterprise grade applications?

[00:23:25] Srirang Srikantha Yes, that is a phenomenal piece of investment that we've been able to bring to market. We have been working with large enterprise platforms and we felt that a point-and-shoot, a traditional discovery process is very challenging and very difficult to maintain for such large applications. For certain ERPs like SAP and banking platforms and CRM like Microsoft Dynamics and Salesforce, we have the ability to go and self-discover screens with little support from the human tester. The bots are able to go and self-discover screens and update themselves. It just makes the whole process of starting off on a test automation journey that much simpler. So everything related to UI navigation and nuances related to UI no longer have to build grounds up by the tester. It is something that our accelerators are capable of bringing into Tenjin depending on what accelerators you have, what platforms you have in your organization. We can bring, we can deliver such accelerators to you and that makes the whole job much simpler. You get off to a running start.

[00:24:44] Joe Colantonio All right. So we saw how you're currently implementing AI, a really cool stuff. I think it's right on spot, right on target for what I think enterprise is dealing with. I know it's hard to predict the future, but when do you see the future of AI, especially at the enterprise level, where do you think we're heading? Or maybe what's on your roadmap that you think is going to help us?

[00:25:03] Srirang Srikantha UI discovery is always something that our R&D teams are very keen to simplify. We've been trying to make the element discovery process simpler and I think all organizations are trying to do some way of UI discovery simpler. In my opinion, test automation requires three things to come together. What am I testing? How do I discover the UI and what data is required to achieve? What is the data required? We have done some amount of R&D and we have put out our first platform for test design. We have our platform up and ready for UI discovery for a number of years. The next part for us is test data and I think that is a very large problem which remains to be solved. It's very easy to solve for simple create functions, but update is a much tougher function to solve for from a test data perspective. What I mean is if you have a customer record, if you have a transaction record and you want to work with an existing transaction record, it's much tougher than creating something from scratch. Test data is the last piece which I think we have not yet solved. There are a lot of great players out there who are solving for these parts of test data management within an automation structure. For example, you have people who are looking at creating simple demographic data. You have people who are looking at obfuscating production data into tests, and so on. But being able to reuse your test automation with live data remains a challenge because live data has different states, it moves, it morphs, and I think that is a challenge that we would have to take up in a few months.

[00:26:53] Joe Colantonio Yeah, that's a huge challenge. Everyone I speak with, everyone, especially on surveys we do every year. Biggest challenge is test data. So if you could solve that and have a winning combo for sure. All right. Srirang, before we go, is there one piece of actual advice you can give to someone to help them with their automation testing efforts and what's the best way to find or contact you?

[00:27:15] Srirang Srikantha That's a tough one, Joe, in terms of I'm not very good at giving advice. But I love the challenge that test automation represents. It's a different way of thinking. You have to think in automation in terms of saying, how do I make test automation survive changes, underlying changes and transitions without having to come back and work on it? It helps to be lazy. It helps to be, you don't want to go and fix it again and again. And that's what automation guys should look to achieve, saying automation should run for multiple cycles without having to come back and touch it and that requires a mindset change in terms of how you think of tests, how you think of test data and that's the only piece of advice I think I would like to give. You could find us on Tenjin online, T -E -N -J -I -N, tenjinonline.com

[00:28:09] Joe Colantonio And how do they get their hands on Tenjin? Do they need to go through a sales team? Do they need to give a credit card? Is there a free trial? Is there like, how does that work?

[00:28:16] Srirang Srikantha We have a free trial, I think we have a four-week free trial, anyone can sign up and you don't need a credit card to start it. So that simple people should be able to just jump on and start. There are a lot of helpful videos and if you don't like it, please write to us and we will change those things.

[00:28:35] Joe Colantonio Awesome. We're gonna have a link for that down below. I definitely recommend everyone checks it out for themselves. You get four weeks to just do as you please. And I'd love to know your comments as well in the comments down below.

[00:28:45] Thanks again for your automation awesomeness. The links of everything we value we covered in this episode. Head in over to testguild.com/a536. And if the show has helped you in any way, why not rate it and review it in iTunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe, my mission is to help you succeed with creating end-to-end, full-stack automation awesomeness. As always, test everything and keep the good. Cheers.

[00:29:20] Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation, testing, and DevOps. If you haven't yet, join our vibrant community at TestGuild.com where you become part of our elite circle driving innovation, software testing, and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate. Visit TestGuild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.

[00:30:04] Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold.