MCPs everywhere, NASA Hates LLMs and More! TGNS153

[00:00:00] Joe Colantonio Who cares about AI augmented software testing? Have you seen all the new MCP servers like the one for Selenium? And why does NASA hate LLMs? Find out in this episode of the Test Guild News Show for the week of April 6. So grab your favorite cup of coffee or tea and let's do this.

[00:00:16] Joe Colantonio Hey, before we get into the news, I want to thank this week's sponsor, ZapTest AI, an AI driven platform that can help you supercharge your automation efforts. It's really cool because their intelligent co-pilot generates optimized code snippets while their planned studio can help you effortlessly streamline your test case management. And what's even better is you can experience the power of AI in action with their risk free six month proof of concept, featuring a dedicated ZAP expert at no upfront cost. Unlock unparallel efficiency and ROI in a testing process. Don't wait. Schedule a demo now and see how it can help you improve your test automation efforts using the link down below.

[00:00:55] Joe Colantonio First article comes our way via Angie Jones, she posted how she created or just released Selenium MCP server, which is a new implementation that enables browser automation through the model context protocol for Selenium WebDriver. We've seen all the hype around Playwrights and now to be out done, Angie Jones posts this on GitHub and this tool allows developers and testers to automate browser interactions through standard MCP clients supporting both Chrome and Firefox browsers. The server includes capabilities for starting browser sessions, navigating URLs, finding elements using various locative strategies, and performing essential interactions such as clicking and typing and capturing screenshots. The implementation is also designed to work with multiple MCP clients, including Goose and Cloud Desktop. It can be installed through an easy one click process or manually via NPM. All right, this tool is really cool because it breaches the gap between Selenium WebDriver functionality and the emerging MCP standard for automation that we've been seeing everywhere lately. Thank you, Angie Jones, once again, for this awesome contribution. You can find more about it in the link down below.

[00:02:00] Joe Colantonio Next up is the webinar of the week. What is it? Let's check it out. The Test Guild is hosting a webinar titled, Who Cares About AI Augmented Software Testing on April 15th. And the session will be led by the one and only automation cyborg, Jonathan Wright, who is the chief AI officer at Keysight Technologies. Jonathan really knows his stuff. This is not to be missed. This webinar aims to provide software testers with a clear understanding of AI's role in testing, distinguishing practical applications from marketing hype, which is all over the place. Key topics he'll cover include the evolution of AI and testing, insights into machine learning, generative AI and the potential risk associated with cloud-based AI testing tools. You don't want to miss it. And you can register using the link down below.

[00:02:47] Joe Colantonio Next article I found scrolling on LinkedIn is by Missy and she links to a NASA report, which questions the use of large language models and safety certifications. And this new NASA technical report, which was released in March of 2025, raises significant concerns about proposals to use large language models like GPT 4 in the development and assessment of safety arguments for critical systems. And it goes over how after reviewing 14 relevant papers, the researchers found that while preliminary studies show potential, the technology is far from ready for deployment and critical certification workflows. And some quotes that really pop out is, LLMs are machines that BS, not machines that think, and thinking is precisely the task that must be automated if the technology is to improve safety at a lower cost, which I think is really relevant to software testers and the researchers go on to characterize LLM outputs as Frankenstein BS text produced without regard for its truth value. And it also goes over 14 key questions that must being answered before LLM based argument technology should be considered fit for deployment, including concerns about how automation might impact system design effectiveness, augment readability, and the identification of potential flaws. The researchers also note that current studies on LLM capabilities in this domain suffer from significant limitations, including small sample size, use of simplified examples rather than full scale safety arguments, and the possibilities that LLMs are simply mimicking examples from their training data rather than really demonstrating true understanding.

[00:04:24] Joe Colantonio All right, it's been a while since I spoke to the folks at Karate Labs, but I just noticed they have a new release, they just announced a release Xplorer 1.20, which is a significant update that introduces deep open API and MCP support. This development enhances the tools capabilities, especially for API testing by offering more comprehensive support for open API specifications. The update aims to improve the precision and speed of API testing processing, giving software testers an advanced toolkit to ensure API reliability and compliance. Also, the Xplorer 1.20 release promises improvements in test coverage and stands out for its ability to seamlessly integrate with the existing testing environments. Additionally, enhanced support for MCP allows testers to efficiently manage microservices, vital, and modern cloud-based application architectures. And with this release, Karate Labs positions Xplorer as the robust solution for teams focused on API and microservices testing. S.

[00:05:19] Joe Colantonio Another company I follow all the time came out with an announcement, and that is how AccelQ has announced the significant enhancement to its autopilot platform. If you don't know, AccelQ has been a big supporter of The Test Guild over the years, so really excited to see that they just introduced an AI driven agent designed to automate the entire software testing lifecycle. And this enterprise grade AI agent learns the application under test that generates fully executable test cases autonomously, aiming to help reduce manual intervention through the testing process. And according to the company, early adopters have reported a 65% reduction in the test creation time and the 80 percent decrease in maintenance efforts. Big promises is a true. Well, try it for yourself using the link down below and let me know.

[00:06:03] Joe Colantonio With all this talk with MCP, a lot of people are asking, is this going to be a replacement for APIs? Not really. And here's why. This article goes over that despite claims that MCP threatens to replace traditional APIs, the protocol serves more as a consumer of APIs rather than a replacement. And the author, Kevin, breaks down how MCP functions similar to the language server protocol used in software development environments, creating a standardized interface between AI assistants, various tools, services, and data sources. And many companies are already generating MCP service directly from existing open API definitions, effectively transforming API descriptions into MCP compliant interfaces. But security remains a significant concern with the protocol. And for API developers, MCP introduces a new design consideration, including creating more AI friendly interfaces and documentation that services both human and machine readers. And I'm not sure how many of you are part of our free channel in our community, but we just got a post by Gagan who talked about how he recently discussed the significance of mutation testing and software quality. And he told me he actually learned about this on one of our new shows about mutation testing and he actually put it to the test. If you don't know, mutation testing involves deliberately introducing small changes or mutants into applications code to evaluate the effectiveness of existing test suites. And the objective is to determine whether the test can detect the introduced errors, thereby assessing their robustness. And in this article, he highlights PI tests, which is a tool designed to automate this process. PI tests systematically alters code to create mutants and then runs unit tests to see if those changes are detected. The outcomes are categorized as either killer mutants, which are detected by the test or survive mutants which go unnoticed, indicating potential gaps in test coverage. And he talks about how his company has integrated mutation testing and has reported enhanced test suite quality and effectiveness. He also notes that earlier implementations of PI tests could have prevented certain incidences underscoring the value and maintaining high quality code.

[00:08:11] Joe Colantonio Next up in Docker news, Docker has just unveiled Docker Desktop 4.40. And this brings a significant enhancement aimed at refining the software development workflow. Key updates include the integration of a new compose watch feature, allowing developers and testers to monitor changes in real time and automatically update their Docker environments without manual intervention. This version also introduced an improved volume management feature offering user enhanced capabilities by handling data volumes, which is crucial for maintaining consistent environments across development, testing, and production pipelines.

[00:08:44] Joe Colantonio And last up, Elastic has made an announcement that the general availability of elastic distribution of Open Telemetry offering site liability engineers and developers and testers a production ready Open Telemetry ecosystem with enterprise grade support. And this announcement goes over the initiative addresses the challenges identified by EMA research, where over 73% of IT decision makers plan to implement open telemetry, but face adoption barriers due to insufficient support. This also aims to eliminate vendor lock-in and proprietary add-ons, providing a stable, open source solution for infrastructure and application monitoring.

[00:09:20] Joe Colantonio All right. For links of everything that we covered in this news episode, head on over to the links in the first comment down below. That's it for this episode of The Test Guild News Show. I'm Joe, My mission is to help you succeed in creating end-to-end full-stack pipeline automation awesomeness. As always, test everything and keep the good. Cheers.