AI for Regression Testing, Generative AI Code Smells and More TGNS99

[00:00:00] Joe Colantonio Does ChatGPT suck for test automation? How can AI help you with regression testing? And what is a real-world example of using large language models with RPA? Find out in this episode of the Test Guild News Show for the week of October 22nd. So grab your favorite cup of coffee or tea and let's do this.

[00:00:17] Joe Colantonio But first, are you looking to take your automation projects to the next level? Look no further than Applitools and their Visual AI Validation testing platform. Trust me, it is a game-changer. Plus, you could try it out yourself by creating a free account. Just click on the special link down below in the first comment and see the difference for yourself.

[00:00:38] Joe Colantonio These news items I'm sharing are going to be from the past two weeks since I've been out the past two weeks. And the first one that caught my attention was a new innovation around AI for regression testing that I think you should definitely know more about. The first one that caught my attention was how Katalon just announced TrueTest beta, which is an automated regression testing solution leveraging AI to generate and maintain regression tests by monitoring user activity. After reading this, here are 3 things that can help you right away with. The first one is how to visualize and understand how your app is truly used in production, which is a cool feature. TrueTest captures real user interactions and generates a user journey map that quality engineers could then use to prioritize and guide their testing efforts. So it's almost like a shift right type of effort getting actual data from your users and then using that data and then bake it and then shift left when you're creating a test. So really cool. The second way is no more guesswork on what test because TrueTest automatically generates regression tests reflecting what users do in your production application. And the third way is it helps create zero maintenance regression tests because it allows quality engineering teams to focus entirely on in sprint's new feature testing and stop spending time, and effort creating and maintaining regression tests. So definitely something you should try for yourself. You should join the beta program and learn more about TrueTest by using the link and the first comment down below. Give it a try and let me know your thoughts.

[00:02:12] Joe Colantonio Also a quick reminder about the Kobiton Mobile Testing and Experience Summit that's happening tomorrow on October 24th. It is going to be a mobile-centric development and testing summit. As more becomes more the go-to channel for enterprise operations to reach their audience, it's more important now than ever to ensure your mobile apps are being released faster while improving quality and engagement. And with the summit, they're going to bring together some of the best and brightest minds in the mobile application development space together. Also, I'll be doing the closing keynote on Automation Awesomeness 260 Actual Affirmations to Improve your QA in Automation testing skills. I'm going to share with you the top teams I found that interviewing of 500 automation engineers, including mobile testers, and what they found helped them succeed with automation. You don't want to miss it. If you haven't registered yet. Register now, even if you can't make it, I recommend you register because you'll get access to the recordings after the fact as well. Look forward to your questions and hope to see you there.

[00:03:12] Joe Colantonio We covered some news articles on large language models in the previous show, but this week I want to share with you a real-world example. So Murali, in a recent blog post, unveils how large language models are changing the game in product development. So AskUI, which, as you probably know, is a UI automation tool, utilizes large language models to translate natural language commands in its domain, specific language, making user interactions more intuitive and accessible. AskUI is Integration with large language models goes even further by automating workflow creation, enabling users to define actions using screenshots. This post also points out that the synergy between AskUI's inference engine and large language models allows for the automatic detection of UI elements and mapping of user clicks to specific actions. Streamlining the user experience also points out that with open LLMs like LLAMA 2 have facilitated the creation of domain-specific chatbots, adapt and providing users with precise information and assistance, overcoming the constraints of input tax limits. I think we see more and more of this fusion of large language models and product development tools like AskUI. And you can read more about it in the first comment down below.

[00:04:30] So pretty much over the past year, we've talked more and more about AI, especially generative AI. But is A.I. a double-edged sword in test automation? Let's find out. This article is by Jason Arbon, and it goes over things like how in the world of test automation, the allure of AI, particularly models like ChatGPT is undeniable. But Jason warns to avoid deceptive simplicity and potential pitfalls. And while a lot of AI. APIs appear straightforward. Automation engineers often encounter an overwhelming array of text inputs and unexpected outputs. So Jason highlights the baby talk phase, where engineers request AI to generate test code only to be met with the most generic and unhelpful response. And the lack of context. And specifically, AI-generated code often leads to more work for the engineer than coding manually. He goes over the struggle extends to the over-communication phase, where engineers attempt to provide exhaustive context to AI prompts. Yet the unpredictability of API responses and the linear compound of issues for complex test cases raises questions about efficiency and reliability. Jason also touches on the flakiness of AI in this article, and Jason touches on a bunch of other ways that AI can be unpredictable and unreliable. However, Jason is a big proponent of AI and test automation as well. You just need to be aware of its pros and cons and realize it's not a silver bullet. It takes both the test and AI in order to really make things work. It's not AI replacing a tester.

[00:06:07] So RPA-type tools have entered the AI arena as well. Let's check out UiPath's latest feature. So UiPath recently unveiled Autopilot, which is an AI assistant set to redefine business automation in autopilot powered by generative AI and a suite of specialized AI models, promises to transform paper documents into business applications with a single click. And the chief product officer, Graham Sheldon at UiPath, mentioned that autopilot's capabilities to create robust automation workflows from users' words, and screenshots of PDFs make a significant stride and a one-click digital transformation. And the AI assistant's versatility extends to business analysts, developers, and automation testers, offering a simplified task execution and unveiling new automation opportunities. And as Jason pointed out there are certain things AI need be aware of. UiPath added an AI trust layer that ensures data security in responsible AI usage, instilling confidence in organizations to integrate AI capabilities seamlessly.

[00:07:14] Joe Colantonio And talking about A.I. Assistants. Here's another use of AI with DevOps. So Kubiya just unveiled a new enhancement of a ChatGPT-like AI assistant. The Kubiya copilots are engineers interact natively with any DevOps tool, a feature that promises seamless communication and collaboration among developers across different of various engineering platforms. And living with popular collaboration tools like Slack and Teams, Kubiya's copilots are adept at provisioning resources, triggering continuous integration and development jobs and generating cloud cost usage reports through simple user commands. The AI assistant is designed to tackle repetitive tasks, boosting the productivity of DevOps teams. And this is a great example of how more and more businesses are grappling with complex operational challenges and using AI assistant stands as a way to help promised to reduce DevOps bottlenecks and improve developer velocity. Something we're going to probably see more and more of in the upcoming months and years.

[00:08:17] Joe Colantonio What other tools can help boost developer efficiencies and cut costs? Well, HashiCorp just announced a new TerraForm feature that's going to help do just this. HashiCorp has just unveiled a suite of new features that help enhance developers' velocity, reduce errors, and cut infrastructure costs. The new offerings include a TerraForm test framework that aids and producing high-quality modules and tests integrated modules published to streamline the testing and publishing process so developers can now enjoy enhanced editor validations of Visual Studio code making error identification and resolution a breeze. The introduction of stacks simplifies the provisioning in the management of infrastructure at scale with ephemeral workspaces, optimizing infrastructure spending by scheduling automatic destruction of nonreduction resources. And to me, that's just another great example of innovation to help bolster developer productivity and efficiency for sure.

[00:09:15] Joe Colantonio For links to everything we covered in this News episode. Head over to the links in the first comment down below. And while you're there, make sure to check out our awesome sponsor Applitools Free Account offer and discover how to take your automation testing to the next level Leveraging Visual AI. So that's it for this episode of the Test Guild News Show, I'm Joe, and my mission is to help you succeed in creating end-to-end full-stack pipeline automation awesomeness. As always, test everything and keep the good. Cheers.