One of the big announcements to come out of the 2019 Automation Guild online conference was that Tariq King’s company UltimateSoftware has open sourced what they are calling an AGENT—an AI Generation and Exploration in Test bot.
More about
that later.
But first, many of the AI testing tools now hitting the market leverage bots to help with software testing. For example, Test.AI has a solution that uses bots for help with mobile testing.
You might be asking, “What is an AI testing bot, and how can we trust them if we don’t know how they work in the testing world?
How AI
Bots Work
When it comes to most AI testing
They’re using techniques like exploratory testing while trying things out, making mistakes, and getting better in the process.
How can a
bot do this?
An AI
test bot needs three things to be useful in testing:
· It needs
to act independently to be autonomous.
· It needs to be “intelligent.”
· It needs to be an agent.
Bots need
to Act Independently
You’ve
probably been hearing quite a bit about autonomous testing, and may have some
misconceptions of what it really is.
Tariq
offered the analogy of a self-driving to clear up some of this confusion.
Driving a
car from one point to another is just a similar journey to the one a tester
might take from one point in the application to another, and possibly
encountering some obstacles along that journey.
There
might be different threats or risks to look out for and decisions that need to
be made. There are also aesthetics; things that need to look good if you’re
going to enjoy your journey, and so on and so forth.
The thing
with a self-driving car is that it really is doing it on its own, working with
the benefit of independence and freedom.
This is
achieved by using a self-monitoring system that is autonomous and independent,
at least to some degree.
This is a
really important concept to keep in mind when talking about automation bots.
Bots need
to act independently, or at least have the freedom to do so.
AI bot autonomy
is often the result of using control loops.
What is a
Control Loop?
A good real-world
example of a control loop can be seen within an HVAC system.
The
system can monitor its environment using sensors, and has the means to affect
that environment through an actuator or effector.
A human
sets his or her desired temperature, making it the goal for the system. The
system immediately begins monitoring and processing feedback, and turning the air
conditioning or heat on and off to achieve that goal.
The same
type of behavior goes for bots.
Ai-powered
bots in test automation typically implement similar functions; they monitor the
environment and collect information, then analyze that information to make
decisions about it.
Monitoring,
analyzing, planning and executing are core functions of these bots, and they
all perform these actions on the basis of some sort of knowledge. It can be a very
basic knowledge of the center, or it could be something more complex.
Bots need
to be “Intelligent”
In
addition to being autonomous, bots need to be “intelligent.”
When we
speak about AI in this context, we are basically saying we are trying to get
machines to mimic intelligent behavior using machine learning.
What is Machine Learning
Machine
learning is a subset of AI that deals with the programming model; for instance,
how we get machines to actually improve based on data that has been gathered, as
opposed to giving the machines specific instructions. When the machine is shown
different examples of things, the data it gleans can help it to infer the function
or the instructions it meets by mapping those inputs to outputs.
What is Deep Learning?
Deep
learning, on the other hand, is a subset of machine learning, which deals with
large-scale computation using multiple layered neural networks.
Bots Need
to be Agents
Lastly,
the bots need to be “agents,” in that they must direct their activity towards
goals and use knowledge and learning to achieve those goals.
They can
analyze themselves, and this is very key.
Bots look
at their own behavior, their errors and success rates, and then can adopt in
real time, and then they can be organized with other agents. Just as we
collaborate, the bots collaborate, and solve problems at different levels of
abstraction.
Hopefully
this has helped you gain a better understanding of what AI bots are, and how
they work at a high level.
But these concepts can be hard to really comprehend unless you can see for yourself an AI agent in action.
Getting Started
with AI Bots
As I mentioned at the beginning of this article, Tariq’s company has made a few resources available on GitHub to help get you started using AI in software testing and to begin exposing defects within bots.
AGENT
The first one is AGENT (AI Generation and Exploration in Tests). AGENT, using training data from AGENT-X, autonomously learns to explore a website and evaluate its actions, fields and forms. AGENT deploys one or more exploration and testing agents to explore a Web application and apply test flows as testable patterns are recognized.
AGENT-X
AGENT-X is the label helper component for AGENT. Once installed in Chrome, it allows for the labeling of HTML elements for training.
Learn more about AI Testing Bots at the Automation Guild.
Automation Guild
To see Tariq’s full session which this post was based on, along with his demo on how to use bots based on AGENT, you can still purchase an after-event ticket for Automation Guild 2019.
Joe Colantonio is the founder of TestGuild, an industry-leading platform for automation testing and software testing tools. With over 25 years of hands-on experience, he has worked with top enterprise companies, helped develop early test automation tools and frameworks, and runs the largest online automation testing conference, Automation Guild.
Joe is also the author of Automation Awesomeness: 260 Actionable Affirmations To Improve Your QA & Automation Testing Skills and the host of the TestGuild podcast, which he has released weekly since 2014, making it the longest-running podcast dedicated to automation testing. Over the years, he has interviewed top thought leaders in DevOps, AI-driven test automation, and software quality, shaping the conversation in the industry.
With a reach of over 400,000 across his YouTube channel, LinkedIn, email list, and other social channels, Joe’s insights impact thousands of testers and engineers worldwide.
He has worked with some of the top companies in software testing and automation, including Tricentis, Keysight, Applitools, and BrowserStack, as sponsors and partners, helping them connect with the right audience in the automation testing space.
Follow him on LinkedIn or check out more at TestGuild.com.
Related Posts
Look, I’ve been doing test automation for over 25 years. I’ve heard the predictions. “Manual testing is dead.” “AI will […]
The 72.8% Paradox That Changes Everything After interviewing 50+ testing experts in 2025 and analyzing data from our 40,000+ member […]
Look, I’ve been doing this testing thing for over 25 years now. I first wrote about the AI “three waves” […]
What’s New in Low-Code/No-Code Testing for 2026 A lot had changed on the The low-code/no-code testing space and it has […]