One of the big announcements to come out of the 2019 Automation Guild online conference was that Tariq King’s company UltimateSoftware has open sourced what they are calling an AGENT—an AI Generation and Exploration in Test bot.
More about that later.
But first, many of the AI testing tools now hitting the market leverage bots to help with software testing. For example, Test.AI has a solution that uses bots for help with mobile testing.
You might be asking, “What is an AI testing bot, and how can we trust them if we don’t know how they work in the testing world?
How AI Bots Work
When it comes to most AI testing
They're using techniques like exploratory testing while trying things out, making mistakes, and getting better in the process.
How can a bot do this?
An AI test bot needs three things to be useful in testing:
· It needs to act independently to be autonomous.
· It needs to be “intelligent.”
· It needs to be an agent.
Bots need to Act Independently
You’ve probably been hearing quite a bit about autonomous testing, and may have some misconceptions of what it really is.
Tariq offered the analogy of a self-driving to clear up some of this confusion.
Driving a car from one point to another is just a similar journey to the one a tester might take from one point in the application to another, and possibly encountering some obstacles along that journey.
There might be different threats or risks to look out for and decisions that need to be made. There are also aesthetics; things that need to look good if you're going to enjoy your journey, and so on and so forth.
The thing with a self-driving car is that it really is doing it on its own, working with the benefit of independence and freedom.
This is achieved by using a self-monitoring system that is autonomous and independent, at least to some degree.
This is a really important concept to keep in mind when talking about automation bots.
Bots need to act independently, or at least have the freedom to do so.
AI bot autonomy is often the result of using control loops.
What is a Control Loop?
A good real-world example of a control loop can be seen within an HVAC system.
The system can monitor its environment using sensors, and has the means to affect that environment through an actuator or effector.
A human sets his or her desired temperature, making it the goal for the system. The system immediately begins monitoring and processing feedback, and turning the air conditioning or heat on and off to achieve that goal.
The same type of behavior goes for bots.
Ai-powered bots in test automation typically implement similar functions; they monitor the environment and collect information, then analyze that information to make decisions about it.
Monitoring, analyzing, planning and executing are core functions of these bots, and they all perform these actions on the basis of some sort of knowledge. It can be a very basic knowledge of the center, or it could be something more complex.
Bots need to be “Intelligent”
In addition to being autonomous, bots need to be “intelligent.”
When we speak about AI in this context, we are basically saying we are trying to get machines to mimic intelligent behavior using machine learning.
What is Machine Learning
Machine learning is a subset of AI that deals with the programming model; for instance, how we get machines to actually improve based on data that has been gathered, as opposed to giving the machines specific instructions. When the machine is shown different examples of things, the data it gleans can help it to infer the function or the instructions it meets by mapping those inputs to outputs.
What is Deep Learning?
Deep learning, on the other hand, is a subset of machine learning, which deals with large-scale computation using multiple layered neural networks.
Bots Need to be Agents
Lastly, the bots need to be “agents,” in that they must direct their activity towards goals and use knowledge and learning to achieve those goals.
They can analyze themselves, and this is very key.
Bots look at their own behavior, their errors and success rates, and then can adopt in real time, and then they can be organized with other agents. Just as we collaborate, the bots collaborate, and solve problems at different levels of abstraction.
Hopefully this has helped you gain a better understanding of what AI bots are, and how they work at a high level.
But these concepts can be hard to really comprehend unless you can see for yourself an AI agent in action.
Getting Started with AI Bots
As I mentioned at the beginning of this article, Tariq’s company has made a few resources available on GitHub to help get you started using AI in software testing and to begin exposing defects within bots.
AGENT
The first one is AGENT (AI Generation and Exploration in Tests). AGENT, using training data from AGENT-X, autonomously learns to explore a website and evaluate its actions, fields and forms. AGENT deploys one or more exploration and testing agents to explore a Web application and apply test flows as testable patterns are recognized.
AGENT-X
AGENT-X is the label helper component for AGENT. Once installed in Chrome, it allows for the labeling of HTML elements for training.
Learn more about AI Testing Bots at the Automation Guild.
Automation Guild
To see Tariq’s full session which this post was based on, along with his demo on how to use bots based on AGENT, you can still purchase an after-event ticket for Automation Guild 2019.
