AI Testing Is Bigger Than You Think, 5 Areas Testers Must Own with Swati Seela
with Swati Seela
Key takeaways
What you'll learn in this episode
Why AI opens almost every response by agreeing with you, and why that false sense of correctness is the modern version of a green dashboard hiding tests that stopped meaning anything years ago.
Why her MCP test generator kept skipping the cataloging step, what that revealed about models trading completeness for speed, and how splitting one workflow into two got her reliability back.
Why traditional Boolean assertions break down against probabilistic systems, and what your automation framework has to do instead.
About this episode
Most testers are stuck arguing about whether AI is coming for their jobs. Swati Seela thinks that's a distraction from a much more useful question: when we say “AI testing," what do we actually mean?
What do you think? Take our TestGuild State of Test Automation Survey now: https://testgld.link/27data
In this episode, Swati walks through the AI Testing Landscape, a practical framework she presented at CAST 2026 that breaks AI testing into five overlapping areas: AI assisted testing, AI evaluation and trust, AI development quality, testing products that contain AI functionality, and testing the AI models themselves. Each one has a different test intent, a different technique, and a different toolset, which is exactly why lumping them together leaves new testers with no idea where to start.
Along the way Swati gets specific about the failure modes she keeps hitting in real work.
She also shares how she uses AI to learn hard material in the semiconductor world without losing the thread, what she absolutely will not accept without laying eyes on it herself, and her answer to whether a tester with a decade of experience should be worried right now.
Her closing advice comes down to two things that have to travel together: testing fundamentals, and AI literacy. Don't hate it, don't love it, just use it.
Discover links to all the resources mentioned in this episode below.
Keep listening
Related episodes

Automation Testing Podcast · Ep. 609
Why Your AI Tests Pass When Your Features Fail with Donovan Brady
with Donovan Brady
Why Your AI Tests Pass When Your Features Fail? Donovan Brady of Leapwork shows how to make
Oct 27

Automation Testing Podcast · Ep. 608
Scary Test Automation POCs Made Easy: Halloween Special With Paul Grossman
with Paul Grossman
Running a proof of concept on test automation tools can feel like a horror movie. Vendors promise
Oct 20

Automation Testing Podcast · Ep. 607
Locust and Azure Load Testing: Create and Debug Load Tests with AI with Lars Holmberg and Abhinav Premsekhar
with Lars Holmberg and Abhinav Premsekhar
Performance testing has never been easier to start, or more important to get right. As teams ship
Oct 6

Automation Testing Podcast · Ep. 606
AI Agents Passed the Tests. Real Devices Didn’t. with Nick Metcalfe
with Nick Metcalfe
Cloud device farms make it easy to get started with real device testing. But what happens when
Sep 29