E2E Innovation at Scale: Unleashing The TestGrid AI Testing Platform with Harry Rao

[00:00:04] Get ready to discover the most actionable end-to-end automation advice from some of the smartest testers on the planet. Hey, I'm Joe Colantonio, host of the Test Guild Automation Podcast, and my goal is to help you succeed with creating automation awesomeness.

[00:00:25] Joe Colantonio Hey, it's Joe, and welcome to another episode of The Guild Automation Podcast. And today, we'll be talking with Harry Rao, CEO of Test Grid, all about maintaining quality and costs in this uncertain market that you're probably all experiencing. If you don't know, Test Grid has been around for a bit, but they're doing a lot of innovations, helping companies simplify the testing, making them more efficient. They offer a bunch of different things like mobile app, cross-browser performance, API testing, and codeless automation capabilities, all under one roof, one platform. So it's like an all-in-one solution. And I think you can learn a lot about this software and also a lot about what's going on in automation and why you need a testing platform to help you with all the things that automation. You don't want to miss this episode, Check it out.

[00:01:11] This episode of the TestGuild Automation Podcast is sponsored by the Test Guild. Test Guild offers amazing partnership plans that cater to your brand awareness, lead generation, and thought leadership goals to get your products and services in front of your ideal target audience. Our satisfied clients rave about the results they've seen from partnering with us from boosted event attendance to impressive ROI. Visit our website and let's talk about how Test Guild could take your brand to the next level. Head on over to TestGuild.info and let's talk.

[00:01:40] Joe Colantonio Hey, Harry. Welcome to the Guild.

[00:01:48] Harry Rao Thank you, Joe. Thanks for having me here.

[00:01:50] Joe Colantonio Awesome. Great to have you. I guess before we get into Harry, can you tell us a little bit more about yourself? I just have a very small bio. I'm sure there's a lot more than just the CEO of Test Grid.

[00:01:59] Harry Rao Yeah. I have a master's in AI back in the day, I graduated from San Diego State. Very honored to be a part of that university. But that's my site. And then I work for a lot of corporates, Joe as part of my career growth, I led a large team, and in my last job, we are mostly mobile, IoT related. Anything that has to do with emerging tech has been my always go-to and I'm always been at the forefront of innovation. And that's me in short. But yeah.

[00:02:36] Joe Colantonio I love it. So I love how you have a masters in AI. So especially relevant nowadays when people are talking about AI in testing, it's not real, but you actually work. You have a company that has a product. I believe you have some AI capabilities. So how close to that AI is it to what you were learning about when you were doing your masters?

[00:02:54] Harry Rao People barely knew what neural networks was back in the day. Like 2005 ish. Think about that. Almost 20 years ago. We were strictly gathered like neural networks, and neural engines at that time. And given the capacity of your processing capacity of that, you were limited to do what you could. But we knew the capabilities at that time. But always that has been my ultimate goal. It's always been in the back. But with the generative A.I. and things that came up, it became much, much easier. And thanks to the hardware that we get. Today, the processing has become much, much faster. So it's always been in the back of my mind and I just the right time. You need to see AI in my view, Joe, cannot be built like it cannot be a feature, it has to be built on a platform. It can not be. Not everything like a simple algorithm can fill in AI. I'll get into it, but let's continue on that.

[00:03:59] Joe Colantonio Great. Yeah, let's expand on that. It has to be built into a platform, I guess, what do you mean by that?

[00:04:04] Harry Rao I'll come to the journey of Test Grid. Originally, we started back in the day 2015, 2016 ish with a vision of getting scriptless into the work. We were building apps. I wanted to build a factory, like a factory pattern. Simple. When you're building apps, that's an architect. For me, the biggest problem was, Hey, there is this API team, database team, frontend team, and all these. And by the time you integrate all these, it becomes, let's say code merges happening. You're bringing API and the UI together, things break and if I want to test it, like in a factory, like an assembly line, I have to go purchase like different tools. Like, I don't want to name the companies, but you need browsers, you need mobile as an infra first, then you have to write automation code on top, which becomes expensive. Now you have to maintain an automation that code for the future so that it doesn't break. Now you have to go start integration testing, which is like, how's my API? How is my UI? Is that getting the right data? And now I have to also do a performance. Imagine by the time I'm done with it, I spend multi-million dollars that year down the line and am I getting the return on the investment that I'm planning to do? This has been my main go-to, Joe like plug-and-play, rather than integrating multiple solutions, I wanted to create a most innovative platform, not just a feature. For me, infra is a feature. For me, scriptless is a feature. Can I bring all this together? There's a difference between an app store, Apple's App store, and then an app. We wanted to build that. So with that infra, that foundation, it took us a long time. It took us close to seven years. That foundation being laid properly, lot of scriptless education, pass and fail training model. The data have been collected enough to train it. Once that is done is when we trained our AI to the level we could release it. It's very simple in today's world. A.I. you can just give it your manual test cases, your user story. We just need to learn about your page and it will generate it. We don't end there. We also do. It generates a scriptless. Enterprises hate losing control. Even a manual tester can go ahead and edit it if they want to. They don't have to. And then run it on our own infrastructure. Like mobile devices or the cloud. And it has auto heal on it as well as it has self-healing capabilities. Long story short, we'll get into the details of it, but end-to-end. Typically, it takes 8 to 24 hours to write test cases in Selenium or Appium, Joe. And during the cycle of it, lifecycle, it takes all day to 24 hours to maintain it. It's XPath, flakiness, whatever. All that has been brought down to mere minutes or seconds. So that's where we are now.

[00:07:26] Joe Colantonio Nice. I guess so it sounds like you started with in mind where we are now. So it seems like it's extensible and it was created that way with this in mind. How did you see the future then, when you started 2015? I know some tools were like AI but it wasn't like now how did you know? What got you into build a platform at that time to have the foresight to think, okay, down the road, I know it's going to grow to this?

[00:07:52] Harry Rao I actually want you to build this, Joe. Day 1. Think of like Apple wants to build the latest VR tech.

[00:08:05] Joe Colantonio Right.

[00:08:06] Harry Rao That you had to build the platform. It was incremental. But did I know the path? I didn't. But to get to this, we incrementally started learning that, okay, I have to get this one first. How do we keep floating at the same time while we're waiting? That had to be done. And this has already you cannot believe this has been in making for four years. But the foundation was never touched. This team was building it with the vision that this foundation is laid up. But imagine you were just doing in the siloed world, not as an integrated platform. That was it. But to answer your question in short, this is what I wanted to build. It was not there. So you have to type it.

[00:08:59] Joe Colantonio I think it's important to highlight because you have a background in AI, you created this with AI in mind. It wasn't like you just had a solution and you sat on it all of a sudden. So I mean, a lot of times sometimes solution's been around for a while and you just put AI on it. It was built from the ground up almost to be AI-driven, it sounds like?

[00:09:16] Harry Rao Exactly. I mean, a lot of people have been trying with ChatGPT. It's all our industry problems, I'm sure, Joe, you went then you tried writing your own Selenium Java code. Does it solve our industry problems? No. We know what the limitations are. It's not an enterprise-ready solution. That's when we knew what it took us. But yeah.

[00:09:37] Joe Colantonio All right. So that's interesting then because a lot of people are leveraging ChatGPT, they open API to get some functionality. Sounds like you're not though, so it's your own A.I algorithm that you've been training from the beginning to handle this?

[00:09:49] Harry Rao Absolutely. Now, we cannot leverage it. Again, it's not testing-oriented. It's good in many ways, but it does not fit our requirements. Testing is a different beast. It's a good NLP engine. I'm not going to deny that. But does it fit our needs? No. And most of our users are enterprise clients. And we have to build this hardware in and out and it's very expensive. We'll go to that until later point. But hey, it is what it is.

[00:10:20] Joe Colantonio Nice. What do you call this platform? Is it Test OS where you highlight, it's like an all-in-one solution. It's not like you have to pick and choose and do it all yourself. It's like all in one platform. Is that Test Os means?

[00:10:33] Harry Rao That's who we are. Test OS is an operating system for testing. It's self-explanatory. Usually, in my team, Joe this is what I give like who we are as a company. Think of us like AWS but for testing. What does AWS do? It does your dirty job. That is infrastructure. No one wants to handle it anymore. That's exactly what we do. We take your testing infrastructure. We have a public cloud and private cloud that is the center of our test OS and everything is built as SaaS on top. In AWS, you've got your dynamo DB, you got your own things. But how did that scale up. When we go to our enterprises, we don't go say, Hey, just get rid of your existing legacy Appium code or Selenium code. We support your legacy, but we call it a shift in left. First, you should be shifting to this infrastructure, which is like cost-effective, which brings you the best ROI. Then, you want to bring all your code into our system, plug and play. You have hundreds of test cases in Selenium? Give it to our AI. It will generate like probably in hours. That probably took you years. It can be easily migrated. It's as simple as that. But at the core that's what we are, Infra with SaaS on top. SaaS that goes for AI scriptless automation, all that. We have performance testing tools, single-user metrics as well as load testing. And we got API testing. We covered the entire spectrum in our testing.

[00:12:14] Joe Colantonio Interesting. You use the best of the best in class. If someone already had Selenium Test or Appium Test, they can use it within your platform. A lot of times they need to do some sort of conversion to a proprietary system, a language, and then they locked into that. How does yours work then? It just reads it.

[00:12:31] Harry Rao Excellent. So it's a two-part question for me, Joe. Part one is you bring your own Appium or Selenium Java code, just an Internet provider. We have, for example, for mobile, we have a virtual USB. It's our proprietary tech. Literally it connects to your system. You can run it to UI test, whatever you want from our hardware. We have a card that can go on-premise and sip and you can connect the devices. That's an on-prem system as well. Bring in your own code. We don't do anything on top. It's like plug-and-play, infra only. Part two of your question is if we converted into our system, which is scriptless. That's when we also generate Appium Java and Selenium Java. So the bare minimum of it is the code that gets converted into your GitHub. So no strings attached. Tomorrow, if you just want to do it locally, you can 100% do it. That's how we built it. Again, with enterprises around our enterprise adoption being the center office. But yeah.

[00:13:34] Joe Colantonio Love it. So I guess, I'll just go over that a little bit more. You keep saying Enterprise. So who's the solution for? Obviously, enterprises have a lot of difficulties. There's a lot of things they need to do and it sounds like this would solve a lot of that. Get them on one platform not having to worry about having now integrated with an offshore provider for servers to run that test and all that and bunch of other things. So who's your core audience? I guess the short answer to this question.

[00:13:58] Harry Rao That's a great question, Joe. I'm a startup founder myself. We have both audiences, but enterprises are the difficult ones. The problems are way more and for other startups or small SMBs, larger startups, and SMBs, it's a plug-and-play. They're mostly public users. We have a public cloud as well. For them, it's ready to go. The questions you were asking me, these like no strings attached, and can I get my own Selenium code, things of that sort? Who do you think asks us mostly? That's our enterprise-class. But we go all in. Not just enterprises, but we have our own public cloud, which is much more cost-efficient. But again, it's not the most it comes with like it's shared devices, shared browser. Enterprises want their own tunnel, and their own dedicated device different, but they do both.

[00:14:57] Joe Colantonio Dumb question, is scriptless different than codeless or is it just another term for the same thing?

[00:15:01] Harry Rao It's the same thing, Joe. Everyone calls. It's low-code no-code, let's put it that way.

[00:15:06] Joe Colantonio Okay, cool. Yeah. Could we dive in a little bit into your AI and how it works? Because it's mostly audio only, people are trying to visualize, okay, this sounds great. I have a bunch of Selenium tests. Now, I want to try this out. How does it work?

[00:15:19] Harry Rao Absolutely. Let's say let's go to a typical SDLC and app development process. You have a tech debt that your team takes and now you have your BAs there at the user story. That user story goes to two guys. One is the dev and other one is the tester in parallel. The dev takes the user story, starts developing his code, tester takes that user story, starts writing the manual code. So we want it to be an inception of this test case. So that we train our AI property. All you have to do is maybe give the user story if it's clear enough or just go to your manual tester and let's say a simple login scenario. Again, I'm simplifying this to the basic. We go very complex, but for example, on the login page and your face for the example of. What do you do to test a failure scenario? You say like, check if I'm on the right page and then enter your username, enter your password, tap on the login button, and see if that login message showed up. This is written in multiple ways, let's say in JIRA or other tools that you use or even in spreadsheets. They write it literally in like simple basic English, some complicated like an end-user story. Given then that or BDD 4. But what we do is we literally take that and integrate it into our system and just click generate test case. We understand the page that you are going to and generate the scriptless like tap on, enter a username, enter a password. Are you going to check available enter username, enter password tap on login. Validate if this message is available. Simple code. Go ahead and click run. Use a browser or use a mobile device. And let's say, AI system was not able to find the element at that point. You have everything about your page and we constantly log through our ML so that now it auto-heals itself. You couldn't find that username say, okay the ID that was provided I couldn't find it during the generation. It will go ahead and run through its ML, find it, do it, enter the username, enter the password, tap on log in, worst case if it could not find the ID, we have a No CI AI inbuilt that will find it, click it, execute, and you don't have to maintain it after that is the way it is built. So that's a long story short.

[00:17:57] Joe Colantonio Does it get better over time? How your history, your unique history where it will know like after X amount of runs? Okay, I know that for sure this is going to work to identify it.

[00:18:07] Harry Rao Absolutely. That's the ML model that's built on it. Again, we don't cross-train that for security reasons, Joe, but for that particular application, we are a sandbox and that is what it constantly learns. And as much as it's learning, it just gets better.

[00:18:27] Joe Colantonio Absolutely. Three parts to automation, writing the tests, running the test, and then troubleshooting the test, right?

[00:18:33] Harry Rao Yeah.

[00:18:33] Joe Colantonio First part, you have scriptless. It automatically will write based on user stories, the automation for using scriptless. And then when it runs a runtime, what usually kills things is flakiness, bad environments, and test data.

[00:18:46] Harry Rao Yes.

[00:18:48] Joe Colantonio It helps you to figure out-

[00:18:51] Harry Rao Self-healing.

[00:18:52] Joe Colantonio Self-healing. is there any other functionality in there that helps during the run process?

[00:18:57] Harry Rao It learns it goes through the machine learning. It sort of goes into the predictive that. Again, I don't want to give out too much of the future, but it goes into that prediction more and starts learning where it's going to fail. And then it's going to correct itself and give back.

[00:19:13] Joe Colantonio Is there anything in the system that lets you know that you're missing coverage based on checked-in test that you have already, or anything like that?

[00:19:21] Harry Rao Not yet, Joe. That's probably in the future, but we have much more to cover. But I think that's a good use case, though.

[00:19:28] Joe Colantonio Cool. Besides that, I know you do a lot of other things which I think is really critical nowadays. You have different areas that you mentioned like core. Do you help with performance testing, security testing, robotic automation, and those types of things as well?

[00:19:41] Harry Rao Yes. Security is not our forte right now. We're not mostly focused on that, Joe, but performance, 100%. For us, performance is two parts. One is single-user metrics. The other one is load testing, the typical. Imagine this, in our portal when you execute your test case, these are-Web, it's easy mobile, it's difficult. Web, we do it for both web and mobile, we don't discriminate these channels. But first, on the mobile side, for example, you have an app that you're building. Single user metric is like as a user, when I launched the app, again, go back to the Facebook example, I tapped on login. As an architect, my worst nightmare used to be how many API calls did I make? Am I doing everything as soon as I logged in? And how much time does it take to go to the next page? What is my CPU usage? Am I creating some memory leaks? And do I have some battery drainage? Am I constantly checking the location? Again, even if you're building these at scale and at a pace you cannot get these, you have to constantly send it to the testers, you have to create more tools, and all. This is available by default. We call it transaction analysis for every application that you have. And as soon as you click on or any action that it is performing, when you say to enter the username or let's say in best transactions, click on log in. We know all of the API calls. We have a system that lowers all the API calls shown during this transaction. These were all the API calls that were made. And now I can say, Hey guys, why? Why did we load 100 API calls on the plug? Can we not lazy load it? Can we not do it at a later point? That is in my head. Then I know the CPU spike. If we have constant memory mapping. As the transaction go on, we know if the memory is being accumulated or being released. We go into that detail, Joe so that you don't have to work. That's part one of the single use of it. Then, now, you want to go into load testing. We have let's say you have your JMeter scripts, you can upload your JMeter scripts into our system and we simulate like let's say you want to simulate a thousand users. You can simulate that. Now, run that simulation on our load runner, I mean, our own internal system, and go ahead and execute it. Now we run our test case that was a single user now with a load of thousand users. How is your test performing? This is my dream or my vision. Like, I don't have to do anything. Everything happens in a one-stop shop way, that's too integration for you.

[00:22:41] Joe Colantonio That's critical because you may have the functional test done. You're like, Oh now, we need to do a performance test. And then we get an environment. We need to set it up this way. It sounds like you're just, I don't know, it's not probably, I see this, point and click. I need a thousand users in play, but it sounds like it makes it a whole lot easier than starting from scratch.

[00:22:57] Harry Rao 100%. It's so simple web-based. It's very, very easy.

[00:23:03] Joe Colantonio How about alerts? You must have all this data and then obviously people are going to want to know what's going on with the system. Are there any type of reports or alerts that come up that maybe give you insights or alert your team, Hey, your battery usage is up?

[00:23:15] Harry Rao Absolutely.

[00:23:17] Joe Colantonio Or they have to dig through a bunch of information?

[00:23:19] Harry Rao No, we call it a build a delta. So we call it like we compare like benchmarks like this is your best. And that's a report that simple. It's in your build. We constantly keep sending email alerts to the people who subscribed internally inside that organization or whoever ran. You can run it to your CI/CD. We have all sorts of integrations for JIRA and everything, but you're constantly to answer your question, you get those reports and you can customize those reports as per your needs. That's the best part. Personally, I'm not a fan. I want to get into predictive analytics. For me, that's the bare minimum. We will get there.

[00:23:59] Joe Colantonio Yeah. Awesome. So for mobile automation, is it native automation, or is it Appium?

[00:24:05] Harry Rao Again, we don't discriminate. So you can bring it to the lab, you can bring in your React Native applications, or again native can be different, right? Hybrid. All this, we support everything. Now, again, why? Because we were the pioneers in scriptless. Again, we didn't go bombard market was not ready for all this low code no code tech. At that point, it took us a long time to perfect it to get to the point that we solved this problem for everyone. We do it for the hybrid, a native application, maybe iOS, Android, everything.

[00:24:41] Joe Colantonio Now, we start off talking about how this application helps people, especially in maintaining quality and cost in an uncertain market. Obviously, you've seen the economy as it is. How would you say a tool like this would help an organization kind of maintain costs and not just get rid of people or how does the help, I guess?

[00:25:00] Harry Rao Absolutely, Joe. Again, going back to the cost question, it's a one-stop shop. Imagine, today, you're paying for infrastructure providers by public cloud and private cloud. Then some of them have subscriptions for low code no code tools. Some of them have this. So if you want a tool consolidation, that's Test Op. Right there, we are a bootstrap company and our pricing is very competitive. Again, I'm not against of VCs, but to bring that, I don't call it transformation. I want to be the most innovative company and you have to be terraforming the space. I feel personally when I was exploring this space, it lacked innovation. The tools are nice, but it needed that next level. AI bring that. You can imagine now, just using your manual testing, you can go ahead and cover the entire testing spectrum. ROI? What took you typical industry numbers to write a Selenium or Appium test case, Joe is 8 to 24 hours like in medium or simple medium complex test. Why do you have to write the code and maintain it everything during the lifetime. And then to maintain it, it's under the age of 24 hours. Something breaks. Hey, let's go ahead. And they did the element ID. If I talk to the dev at all now that it's brought down to mere minutes, if not seconds, you tell me the ROI.

[00:26:37] Joe Colantonio For sure. And two consolidations are a big deal because if you're starting to create your own framework and then you need to run it on a server. I need the infrastructure and then I need to go again to performance. It becomes a nightmare. I think that's a huge, huge, huge point for sure.

[00:26:53] Harry Rao Absolutely.

[00:26:54] Joe Colantonio Harry, are there any customer testimonials that talk about what they did before and what happened after using a solution, and if you could share with us?

[00:27:02] Harry Rao Absolutely, Joe. There are a bunch of customers that we have, again, SMBs to enterprises, I'm going to call the enterprises one of them as an insurance solution provider. They have, let's say, a core product. Insurance is mostly the core and then the customized solution. The core was automated. The customer solution was not. And now they're like, Where do we go? Like, it's a lot of money to spend, but the customize is the one that goes to other clients. How do we get started and how do we get from here? So they're like, they're evaluating these low code no code tools. And then once we showed this capability, it was a slam dunk. There was nothing else. And we do have all the testimonials on that and the amount of ROI, the VP got out of this is just insane. That's it. It's a one-stop shop. What they were looking for. A dream come true.

[00:28:00] Joe Colantonio Love it. Okay, Harry, before we go, is there one piece of actual advice you can give to someone to help them with their automation testing efforts? And what's the best way to find contact you and learn more about Test Grid?

[00:28:10] Harry Rao Okay, one piece of advice is always to stay on top. Make sure you are innovating. Because simple example if I stated scriptless, we had to outdo ourselves. All right like what we did. We have to constantly keep in awaiting else someone will wipe us out eventually. There's nothing wrong. Always learn, always innovate. Be that. Be Steve Jobs, if you could in your own world, you don't have to be tha jobs provider or a big person. But always learn, always innovate. That's the best part. Enjoy what you do. The second thing is you can all find us at Testgrid.io And one another thing, Joe. Everything that you have done for the testing community, this is our give back. Anyone who watches this show, they can come to us and if they can use TESTGUILD20 as a discount code, we can give them that discount and all uppercase.

[00:29:13] Joe Colantonio Awesome. That is great news. And all this information will be in the show notes. Make sure to head over to the show notes to find out. And that's TESTGUILD20 all uppercase to get a special treat from the folks at Test Grid. Thank you, Harry, so much for that. Appreciate you.

[00:29:27] Harry Rao Thank you for having me, Joe.

[00:29:29] Thanks again for your automation awesomeness. The links of everything we value we covered in this episode. Head in over to testguild.com/a457. And if the show has helped you in any way, why not rate it and review it in iTunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe, my mission is to help you succeed with creating end-to-end, full-stack automation awesomeness. As always, test everything and keep the good. Cheers.

[00:30:05] Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps, and automation professionals in the world, we'd love to have you join the FAM at Testguild.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real-world value that can improve the skills or solve a problem for the Guild community. I love to hear from you head on over to testguild.info And let's make it happen.