The AI Illusion: Why Testing Still Needs Humans with Joe Colantonio

About This Episode:

Everyone is talking about AI replacing testers, writing tests, and transforming software quality. But what if we’re asking the wrong question?

In this solo episode, Joe Colantonio shares a growing concern he’s seen while traveling across the country for TestGuild IRL events: a decline in testing fundamentals at the exact moment AI hype is reaching a fever pitch.

Drawing insights from Carissa Véliz’s book Prophecy: Prediction, Power, and the Fight for the Future, Wayne Roseberry’s work on AI and meaning, and Tariq King’s concept of Human Experience Testing, Joe explores why AI systems may be far less intelligent than many believe, and why human testers remain more important than ever.

You’ll discover:

✅ Why large language models generate plausible answers without understanding truth

✅ The difference between prediction, correlation, and genuine understanding

✅ Why AI can test software but cannot experience software

✅ What “Everything is tested, but nothing is experienced" really means

✅ How AI hype may be distracting teams from critical testing fundamentals

✅ Why empathy, context, and human judgment are becoming competitive advantages for testers

Whether you’re excited about AI, skeptical of it, or somewhere in between, this episode will challenge you to think more deeply about the future of testing and your role in it.

Resources Mentioned 📖 Prophecy: Prediction, Power, and the Fight for the Future by Carissa Véliz 📖 Work and presentations by Wayne Roseberry 🎓 Free course: thebullshitmachines.com 🎤 Learn more about TestGuild IRL events: TestGuild.com/irl

If you enjoy this episode, be sure to subscribe, leave a review, and share it with a fellow tester who’s trying to navigate the AI era without losing sight of the fundamentals.

#SoftwareTesting #AI #QualityEngineering #TestAutomation #SoftwareQuality #HumanExperienceTesting #ArtificialIntelligence #TestGuild #QA #TechPodcast

0:00 / 0:00

Join the Guild for (FREE)!

Email New Tab

About Joe Colantonio

Joe Colantonio has over 25 years of experience in software automation testing and is thefounder of Test Guild–a dedicated independent resource for actionable and real-worldtechnical advice from industry experts (blog, video tutorials, podcasts, and online conferences)to help improve your DevOps automation, performance, and security testing efforts. He is thehost ofthe longest-running podcast on automation testing. He has interviewed over 400 industryleaders and is the creator of Automation Guild, the premium, annual online event for testautomation engineers. Joe has also hosted over 500 hours of live online events for some of thebiggest companies in the software testing space. Joe’s company, Test Guild, also providesservices to software companies to help them reach a community of highly engaged, growth-oriented, and ambitious audience of 40k+ software testing professionals.

Connect with Joe Colantonio

• • Company: www.testguild
  • Blog: www.testguild
  • LinkedIn: www.joecolantonio
  • Twitter: www.joecolantonio
  • YouTube: www.joecolantonio

Rate and Review TestGuild

Thanks again for listening to the show. If it has helped you in any way, shape, or form, please share it using the social media buttons you see on the page. Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.