AI IDEs, Self-Healing Tests, and a New Way to Score Quality and more TGNS168

[00:00:00] Are you overlooking this new tool that helps you find accessibility bugs in 30 seconds? Could AI finally make your tests self-heal without breaking everything else? And have you seen the new way to measure software quality? Find out in this episode of the Test Guild News Show for the week of September 14th. So grab your favorite cup of coffee or tea and let's do this.

[00:00:21] Hey, before we get into the news, I want to thank this week's sponsor Zaptest AI, an AI driven platform that can help you supercharge your automation efforts. It's really cool because their intelligent co-pilot generates optimized code snippets while their planned studio can help you effortlessly streamline your test case management. And what's even better is you can experience the power of AI in action with their risk-free six-month proof of concept featuring a dedicated ZAP expert at no upfront cost. Unlock unparallel efficiency and ROI in your testing process. Don't wait. Schedule your demo now and see how it can help you improve your test automation efforts using the link down below.

[00:00:59] All right, let's kick things off the brand new tool I think you need to know all about. So Test Guild has just launched our AI powered accessibility testing tool designed to help software testers identify and address web accessibility issues. So you run a scan and in 30 seconds you get screenshots, plain English explanations, ownership tags and ready to use test code. Once the scan is done, it highlights issues, including AI generated explanations that translates technical WCAG compliance language into understandable terms. It shows the number of issues found, serious issues. It also points out who should fix it, what the fix is, what the problematic HTML element was. And also, if you don't know what the actual issue is, you can click on highlight on screenshot. And as you can see, it tells you the issue and also highlights the boxes that are the issues in red. You can also select if you're using Cypress, Selenium or Playwright. And we'll show you the code you would need to order to validate that particular issue. You can find out more about it using that special link down below.

[00:02:01] Also, I forgot to mention, if you're already down on the Test Guild page. You have to check that out. If you go to conferences, click on the call for speakers. I want to let you know that the official call for speakers for Automation Guild 2026 is now open. This is going to be our 10th annual event taking place online, as always, February 9th to the 13th. And I run a survey every year to ask The Guild what they're struggling with. So you see all the topics The Guild told me they want to hear at this next event. So look forward to seeing your submission for the Automation Guild 2026 event as well.

[00:02:28] So if you're looking to code faster, here's an AI powered IDE that you probably need to know more about, especially if you using Playwright. In this week's synodical, Michael explores how Cursor IDE can be paired with Playwright to streamline test automation task. Cursors built on Visual Studio Code and integrates with large language models directly into the editor, allowing testers to generate refactor and review code without leaving their workspace. Michael also highlights several practical issues for the setup, including converting tests between frameworks, generating page objects from HTML snippets, creating API methods for swagger definitions, and enhancing quick prototypes produced with Playwright CodeGen. He also demonstrates using LLMs and Cursor for code reviews, where prompts can enforce principles like solid or suggest improvements to structure and formatting. However, he does caution that hallucinations and context saturation and over-reliance on AI can produce flawed code. And Michael emphasizes that while Cursor can accelerate coding tasks, testers still validate results and maintain strong foundational knowledge. Definitely another must read that you can find down below.

[00:03:32] Next up is Webinar of the Week. I know Playwright gets a lot of love, but how about Robot Framework? Well, I think Robot Framework is a tool that a lot testers need to know more about. And we're trying to change it at the Test Guild by hosting two webinars around the Robot Framework, the first being Robot Framework Unleashed. And we'll be joined by the Robot Frameworks lead developer, Pekka Klärck, who will explain why the open source platform is widely adopted, outlining where it works best and its key use cases and how teams can begin using it. And I'll be join by Rohith, who's going to show how he train nearly 100 functional testers to contribute to automation using this low code approach, showing how to close the gap between technical engineers and business domain experts. Even if you can't make it, make sure to register down below because you'll get the recording in a day or so.

[00:04:18] If you're into BDD and Reqnroll, here's a new announcement, a new solution to help you with your documentation. This is by Soeren who announced the first release of Expressum Live Doc, which is an open source tool designed to generate a single HTML report in a live in document style for Reqnroll projects. This tool aims to consolidate test results into a clear, unified report format, helping teams maintain up to date documentation directly alongside their automated test. And if you go to GitHub, it explains a little bit more and also how to get started using it as well.

[00:04:51] Okay, I missed a few News Shows the past few weeks. This article is from a week or two ago, and it's all about how to implement self-healing tests with AI. This is a question I get asked all the time. So I'm really excited to see that Shyamal is creating a series around this topic explaining how large language models can make automated tests resilient to minor application changes. And he outlines one of the core problems that tests often fail, not because the features are broken, but because of small, harmless changes like rename labels or restructured responses. It also breaks down how traditional rule based self-healing approaches struggle with these cases, frequently creating rigid logic that requires intrusive code changes. But LLM offers a way to address this by reasoning about context and intent rather than fixed rules. But they also introduce two challenges, managing the right amount of context and preventing hallucinated fixes. To handle this, he provides a three-step feedback loop, context composition, LLM evaluation and validation. When a test fails, the system captures relevant failure details, sends them to the LLMs to propose minimal edits, then validates those edits in a CI pipeline before accepting them. He gives a practical example using Playwright and a change placeholder label showing how this loop can automatically detect the issue, suggest a targeted fix and verify it before merging. Really cool stuff that you should definitely check out using Playwright. And you can find it once again using the links down below.

[00:06:18] I also came across a new framework I think you're going to enjoy. It's by Jason, who talks about a new way to measure software quality. And in this post, he highlights a new frame work called the Quality Quadrant, and it's being introduced as a standardized way to measure software quality, it combines AI-driven testing with human evaluation. And in the LinkedIn post by Jason who's the CEO of Testers AI, he explains that software quality assessments have long been fragmented and subjective, making comparisons between applications unreliable. But the Quality Quadrant measures quality on two different ways. Quantitative quality uses a reasonable AI power test to detect bugs, assess performance, security, accessibility, privacy, and usability, and all results then reviewed by human experts for accuracy. The other one is Qualitative Quality. Qualitative Quality blends AI simulated persona feedback with human validation to capture user sentiment, clarity, and overall experience because AI enables the reuse of standardized test suites across many applications. Jason says this approach allows for consistent apples to apples benchmarks at scale. He also points to testers AI standards checks as an example, which automate static and dynamic tests, triage issues, and sends findings for expert reviews. Jason's always innovating. You should definitely check him out for more info as well.

[00:07:38] Next up is a follow the money segment. So OpenAI has agreed to acquire StatSig, a product testing startup for 11 billion in stock, according to this report published on September 2nd. All right. I didn't know what this tool was, so I had to check it out. So it turns out it's a platform that combines feature flagging, experimentation, analytics, and session replays into one unified system, which then can be highly valuable for testers and automated teams by integrating directly with your code base through SDKs that allows you to control feature rollouts and run A/B tests while collecting detailed event data in real time. This means testers can validate new features in production safely through controlled rollouts, automate the collection of performance and usage metrics, and quickly detect regressions or unexpected user behavior, helping teams move faster while maintaining quality and confidence in the release. And I just think it's telling that OpenAI is investing in tools like this. It's a good space to be in as a tester, I think. So definitely check it out as well.

[00:08:37] And finally, on a more personal note, if you or someone you know is facing layoffs, I found a guide that can help you regain control. All right. I wish I had this when I was laid off almost 6 years ago. This is by Crystal. She's published a candid and practical guide titled Don't Panic: A Former Employee's Guide to Layoffs drawing on her repeated experience with job loss. She outlines 12 steps to help people recover after being laid off. And she urges readers to first acknowledge their emotions, then pause before making. Major decisions using what she calls the 24 hour rule. She also emphasizes carefully reading severance and benefits paperwork, creating a budget to understand financial runway and scheduling any need in medical appointments before benefits expire and focusing on learning realistic new skills instead of setting impossible goals. And what I love about this, her message closes by framing layoffs as a break in a chapter, not the end of the story, and a possible opening to new opportunities. This is exactly what happened to me, I was laid off and I decided rather than go back to work, I would just do my own thing full time.

[00:09:40] All right. For links of everything in value we covered in this news episode, head on over to those links in the comments down below. So that's it for this episode of the Test Guild News Show. I'm Joe. My mission is to help you succeed in creating end-to-end full stack pipeline automation awesomeness. And as always, test everything and keep the good. Cheers.