Our world is changing. Artificial intelligence is being employed in nearly every walk of life, from virtual assistants to self-driving cars. How will this new way of life impact software testing? What is our role…if there even is one? In this episode, Angie Jones will lead the way into the future for testers by sharing the skills you’ll to know for testing AI-based technologies.
194: The Reality of Testing in an Artificial World with Angie Jones
with Angie Jones
Senior Software Engineer in Test at Twitter who has developed automation strategies and frameworks for countless software products
About this episode
Quotes & Insights from this Test Talk
- So there's a lot of talk about AI in the testing community and most of the discussion has been on how AI tools will be able to help us as a Testers — help us test better. I'm seeing a lot of new automation tools that are popping up and promising that AI will save the day and that's all right. But I feel like what's missing from the conversation is how to test forms of AI that are present today in today's applications such as machine learning.
- I've tested application where it's core functionality that we were selling was you know it has machine learning. And so your app will work better as time goes on. And so it was a really interesting experience for me. How do I test this? And we're seeing more and more of these type of applications. Everyday applications that we use right now. We don't necessarily realize that there's machine learning in there such as Netflix. So its recommendation system when you go to your opening page on Netflix the recommendations there are based on you personally. And that is also based on machine learning algorithms. At Twitter where I work there are features there that are employing machine learning. And yet we look at AI, machine learning, all of these things is something that's way out in the future and not necessarily something we have to work on right now or think about right now.
- A lot of people will say oh this is a machine learning features so don't worry about testing it. And I often have to say no no no. We should definitely be involved in this because people look at machine learning and AI concepts. This black box that is always right, is just correct by default. And there have been horrific stories in the media where that is not the case.
- One thing is that it challenges a lot of our guidelines around testing and automation specifically for example we often say that in order to test such an order for it to be testable the exact results must be known in advance and especially for automation. So when you think about writing automation script you have this scenario in mind and you have an expected result and you need to assert on an expected result. But in machine learning there is no exactness there is no preciseness. There's a there's a range of possibilities that are valid and that could be correct.
- In my automation of testing the machine learning algorithms. There was no use of the Selenium API. You know I wasn't doing front-end stuff I'm actually testing algorithms. And so the programming that I had to do was a lot different as well.
- The one piece of advice I would give is don't allow others to make you believe that AI and machine learning is this all-knowing black box as testers we know better. We know that this is not the case and so get involved. Read up more about what machine learning actually is. Look at different applications that are used in machine learning and start thinking about even if you don't work on them right now. A key question that I asked in my keynote is — How would you test this? And so there's something really good to just start thinking about what would your test strategy be on something like a Netflix recommendations page or Twitters timeline of tweets, for example, like how would you test this. So I thinking about that. So that's my advice.
Connect with Angie Jones
- Twitter: @techgirl1908
- Blog: angiejones.tech
- LinkedIn: angiejones
May I Ask You For a Favor?
Thanks again for listening to the show. If it has helped you in any way, shape or form, please share it using the social media buttons you see on the page.
Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.

Test Talks is sponsored by the fantastic folks at Sauce Labs. Try it for free today!
Keep listening
Related episodes

Automation Testing Podcast · Ep. 612
Robot Framework Test Automation: Automate Web, Desktop, Mobile, and Embedded Systems
What makes Robot Framework flexible enough to test medical devices, low code bots, monitoring systems, and embedded
Nov 17

Automation Testing Podcast · Ep. 611
Playwright at Scale: Run 500 Tests in Minutes with Oliver Stenbom
with Oliver Stenbom
Is your Playwright suite slowing you down instead of speeding you up? In this episode, Oliver Stenbom,
Nov 10

Automation Testing Podcast · Ep. 610
How to Prove Your Release Is Ready When AI Writes the Code
How to Prove Your Release Is Ready When AI Writes the Code? Can you prove your software is ready to ship, or does that answer still depend on a meeting and a spreadsheet? Want to see if your team can prove a release is ready? Book a custom LoopIQ demo with Ashwin: https://testgld.link/loopiqdemo In this episode,
Nov 3

Automation Testing Podcast · Ep. 609
Why Your AI Tests Pass When Your Features Fail with Donovan Brady
with Donovan Brady
Why Your AI Tests Pass When Your Features Fail? Donovan Brady of Leapwork shows how to make
Oct 27