AI Testing Reality Check: What 13 Founders Refuse to Automate with Joe Colantonio

I asked thirteen people inside AI testing one question: can AI test your software for you? Founders, CTOs, project leads, and quality leaders, many of them selling AI testing tools right now. Not one said yes.

This episode is my session from the Testear.la 2026 conference, built entirely from twelve months of interviews on this show with the people actually building this technology, including Applitools, Maestro, Bright Security, QA.tech, Cucumber, Ito, and the lead of the Selenium project.

In it you'll learn why one AI testing vendor deliberately removed AI from the part where tests run, the real productivity number teams are measuring versus the number in the case studies, the four incompatible architectural bets vendors are making and how to tell which one your tool has chosen for you, why AI generated code is producing vulnerabilities far faster than it produces code, and the one architecture rule that stops an agent from editing the test that was supposed to catch it.

You'll also get the list. Every task these thirteen people said they would never delegate, assembled from conversations that had no coordination between them. Plus two tests you can run on Monday, both free, that tell you whether you're ready to trust an agent with your suite.

If you're a tester, automation engineer, or QA leader trying to figure out where you fit in all of this, this one is for you. The people building AI testing don't trust it to test on its own. That gap is your job.