Finding the right testing tool from all the available automation testing tools is tough.
I’ve interviewed over 300 testers and developers on my TestGuild Automation podcast (formally named TestTalks). Based on this treasure trove of test automation awesomeness, I’ve taken the top functional automation testing tools mentioned by our experts and listed them below.
This list includes both free, vendor and open-source software testing tools. Many of the paid options listed do offer a free trial version of their solutions.
Also, this automated testing tools software list is in no particular order (placement does not indicate ranking!!)
How to Select an Automation Testing Tool?
I don’t think there is one tool that all teams should use for software testing. What you decide to choose should be based on your team’s unique needs.
I always recommend regardless of what automation testing tools are selected to do a two week proof of concept (POC) and make sure the solution actually fits in with your team's development workflow. For more details on this and other automation best practices check out my Ultimate Test Automation and Best Practices Guide.
Top automation testing tools framework list
Now, time for the automation testing tools list. Bookmark this page as this list will be updated regularly.
Here is my automation testing tools list of the top frameworks that you should check out for GUI testing.
|Automation Testing Tools List|
|UFT Developer (Formally named LeanFT)|
|Ca Agile Designer|
|IBM Rational Functional Tester|
|Oracle Application Testing Suite (OATS)|
1. Selenium WebDriver
OK–this is a no-brainer. As of now, Selenium is the leading open source automated regression testing option for browser-based website automation.
But, Selenium is not a standalone-testing tool.
It’s an API that allows you to programmatically write tests using most programming languages to interact with a browser the way a real user would. This is probably one of the most popular website automation testing tools listed.
You can't start a list of automation tools without having Selenium first.
It also is the software that I think started the Agile and DevOps automation revolution and tool in most demand in SDET job descriptions.
Cucumber was not designed to be an automation test tool; it’s a tool that helps conversation and removes ambiguity from requirements before your team invests time in writing code.
However, it does have a unique benefit; once you’ve got test scenarios created, you can generate what they can “executable specifications,” using whatever automation library is supported by the programming language that you use.
Although Cucumber’s main benefit is helping communication and collaboration, many teams use it simply as an automated testing tools framework. If you do decide to use Cucumber I highly recommend that you read The Cucumber Book: Behaviour-Driven Development for Testers and Developers.
If you need an automated test tool Java-based framework that integrates with behavior-driven development (BDD) tools such as Cucumber and JBehave, and lets you keep your test scenarios at a high level while accommodating lower-level implementation details in your reports, Serenity BDD (formerly Thucydides) might be the tool for you. It's designed to make writing automated acceptance and regression tests easier.
Serenity acts as a wrapper on top of Selenium WebDriver and BDD tools. It makes writing BDD and Selenium tests easier because it abstracts away much of the boilerplate code you sometimes need to write to get started. Serenity also offers plenty of built-in functionality, such as handling WebDriver management, managing state between steps, taking screenshots, running tests in parallel, facilitating Jira integration and more—all without having to write a single line of code.
Serenity creates highly detailed reports. Out of the box, it creates living documentation you can use not only to view your Selenium BDD test results but also as documentation for your application.
It also has an extra annotation method called @Step. When used correctly, @Step gives you an extra level of abstraction that makes your tests more reusable and maintainable.
Serenity takes care of many things you would typically have to code from scratch if you were creating your own BDD framework. It’s one of the better open-source frameworks available, although I might be a bit biased; it’s the tool I use for my day-to-day test automation efforts.
John Fergusan Smart also wrote an excellent book on BDD in Action that includes tips and tricks for using Serenity and other automated testing tools.
4. Unified Functional Testing – Quick Test Professional
Unified Functional Testing (UFT) is a commercial test automation tool that was initially developed by Mercury Interactive and was named Quick Test Professional (QTP). It was later acquired by HP and rebranded as Unified Functional Testing. Micro Focus recently purchased it, so Lord knows what the marketing geniuses there will end up calling it.
Before Selenium became the defacto browser-based automation solution, QTP/UFT was one of the most popular automation testing tools on the market.
It’s still is one of the top automation tools for folks who need a tool that automates more than just web browsers. Not only does it have web testing functionality, but it also has support for other diverse technologies like WPF, NET, Oracle, PeopleSoft, SAP, Siebel, Terminal Emulators and more. Using a tool like UFT allows you to automate complete end-to-end testing workflows that might have interactions with non-web-based applications.
5. UFT Developer (LeanFT)
UFT Developer formally named (Lean Functional Testing (LeanFT)) essentially combines the best of the commercial vendor-based and open-source worlds by morphing Selenium with some essential functionality currently found in UFT.
HP built LeanFT from the ground up to address the growing modern automation demands of continuous testing, mobile a, d DevOps. What’s especially cool is that LeanFT fits into your developer’s ecosystem, so your developers can still use their favorite IDEs, programming languages and unit testing frameworks–not to mention the added benefit of access to HP Web/Windows-based technologies that Selenium doesn’t provide.
6. Automation Anywhere
Automation Anywhere has a marketing twist when describing its a commercial automated test tool solution. They explain that their automation uses “software robots” to automate scriptless end-to-end tests. The three main areas AA highlights for their product is:
- RAP – automate every rule-based process
- Cognitive – self-learning automation
- Analytics – real-time, operational intelligence with enterprise-wide business
SmartBear's TestComplete platform helps you create repeatable and accurate automated assurance tests across multiple devices, platforms, and environments quickly and easily.
SikuliX can help automate anything you see on the screen of your desktop computer running Windows, Mac or some Linux/Unix. Sikuli uses image recognition powered by OpenCV to identify and control GUI components.
AutoIt v3 is a freeware, BASIC-like scripting language designed for automating the Windows GUI and general scripting. While not the most robust tool from all the other automated testing tools but many teams integrate AutoIt with Selenium to work around non-browser windows that appear in an automated test workflow.
While most automated testing tools on this listed were created for functional testing Applitools was developed from the ground up, specifically for visual validation assurance. Visual Software Testing is the process of validating the visual aspects of an application’s User Interface (UI).
In addition to validating that the UI displays the correct content or data, Visual Testing focuses on proving the layout and appearance of each visual element of the UI and the UI as a whole.
Watir is an open-source Ruby library for automating tests. Watir interacts with a browser the same way people do: clicking links, filling out forms and validating text.
PowerShell is a Microsoft product. Anytime you want to use the command line or automate anything inside of the Windows ecosystem (or pretty much any Microsoft product) you’ll want to go with PowerShell. While not usually considered in a list of automated testing tools I still consider this a viable option for DevOps based automation. There is even a BDD framework for it called Pester that some teams love.
13. Tricentis qTest
qTest Scenario is a JIRA plugin that was designed to help you quickly manage your BDD/TDD end-to-end workflows, including creating and executing tests as well as acting as a centralized repository for results, defects management, and resolution. They developed qTest Scenario with some essential team benefits in mind, like:
- Helping your teams work faster
- Increasing productivity and collaboration
- Helping you focus more on your end-users
- Helping you scale your test-first efforts across your organization
14. White Framework
Appium is a free, open-source, cross-platform UI mobile testing toolk. It allows you to write UI tests for your mobile apps, Android iOS, and some other OSs. It also lets you create test scripts using the Selenium JSON writer you know and love from web automation assurance.
Appium is becoming an industry standard for mobile testing, much as Selenium WebDriver became the standard for browser-based automation a few years ago.
16. Robot Framework
If you want to use Python for test automation, you can’t go wrong with the Robot Framework. This mature product, created for testers, uses a keyword-driven approach to make tests readable and easy to create. It also includes many test libraries and other tools you can use. The Robot Framework is Python-based, but you can also use Jython (Java) or IronPython (.NET).
While the Selenium WebDriver library may be the most used external test library, Robot Framework can test things other than websites including FTP, MongoDB, Android and Appium. In addition to all this open-source awesomeness, it has a lot of APIs that help make it as extensible as possible.
The keyword approach used by Robot Framework is excellent for testers who are already familiar with other vendor-based, keyword-driven test tools, making the transition to open source much more comfortable for them.
If your team is mostly made up of testers, Robot Framework is an excellent option for your automation framework.
17. Broadcom Agile Designer
The Broadcom Agile Requirements Designer automation tool takes a unique approach to test automation. Rather than focusing on the code, it creates automated tests automatically using model-based requirements. Using a model-based approach is cool because it can auto-generate and update test cases whenever a change is made to the model.
If a model-based approach for test automation sounds like a solution that would work for your team, but you have a tight budget, check out GraphWalker.
GraphWalker is an open source Model-based testing tool for test automation. It's designed to make it easy to design your tests using graphs.
Ranorex test studio is a functional test automation tool that is driven by the user interface. It helps you automate anything when it comes to web applications, mobile applications or desktop-based applications.
20. CucumberStudio (Formally HipTest)
HipTest is a new breed of test management tool and BDD Framework. Many teams that use BDD have a hard time managing all their manual, exploratory and automated tests in one place.
To address this issue, HipTest has developed a single platform to help with those BDD testing efforts. It’s a simple platform in the Cloud that supports real-time collaboration between product owner, testers, and developers.
TestPlant is one of the only test automation tools on this list that has strong support for Apple’s platform. Because of its unique, image-based recognition abilities, it's known for its ability to test hard-to-automate applications; especially those with object recognition issues. Unfortunately, however, anyone who has done image-based, functional test automation understands how difficult these types of tests can be to maintain, and some customers have noted that as an issue.
TestArchitect is a codeless, keyword-driven automation framework that makes it easy for teams to create and manage large test suites.
When I asked Hung Q. Nguyen the co-founder of LogiGear why he created Test Architect he mentioned that creating a test is one thing. But tests change and they change very rapidly. How do you rapidly change the test and maintaining it and updating it so that you can scale? That was the thinking behind the design of Test Architect.
23. Galen Framework
If you need to focus your automation efforts on user experience design (UX) or layout testing, Galen Framework might be a perfect fit for your needs.
Created specifically for UX testing acceptance testing, Galen has a special syntax and rules you can use to describe and verify your application’s layout. It also lets you specify your browser size, then run tests against your software to check layout specifications. Galen tests also generate detailed HTML reports with screenshots, and it includes a visual image comparison with a cool heat map feature.
Tricentis is self-billed as “the continuous testing company.” They have a strong focus on supporting Agile testing and CI processes. Their solution Tosca boasts a scriptless technology that is used for functional test automation
Espresso is a native mobile testing framework for Android testing.
Testim leverages machine learning to speed up the authoring, execution and–most importantly–the maintenance of automated tests. Their goal is to help you to start trusting your tests.
Testim focuses on reducing your flaky tests and test maintenance, which they see as one of the most significant challenges for most organizations.
Oren Rubin, co-founder of Testim, mentioned in a recent TestTalks interview that the firm’s primary goal is to help liberate test automation from the exclusive realm of developers and make it simple enough for anyone on the team to create.
Compared to most of the other tools on this list, Cypress is a more developer-centric test automation framework that focuses on making test-driven development (TDD) a reality for developers. One of its design principles was to be able to package and bundle everything together to make the entire end-to-end testing experience pleasant and simple.
It has a different architecture than Selenium; while Selenium WebDriver runs remotely outside the browser, Cypress runs inside of it.
This approach delivers more consistent results since it’s able to understand everything that happens inside and outside the browser. It also gives you native access to every object without requiring you to deal with object serialization or over-the-wire protocols. You’re pulling your application into Cypress.
As a result, it can synchronously notify you of every single thing that happens inside the browser so that you have native access to every distributed object model (DOM) element.
Cypress also makes it easy to simply drop a debugger into your application, which in turn makes it easier to use the developer tools while you’re developing. Of all the automation testing tools listed here, Cypress is the one that I hear most developers prefer to use.
WebDriver.IO is another cross browser testing automation testing tools module but this one is for Node.JS that makes it possible to write super easy Selenium tests in your favorite BDD/TDD test framework that will either run locally or in the Cloud using Sauce Labs, BrowserStack or TestingBot.
I like how Kevin Lamping in TestTalks Episode 179 describes the chief benefit of using WebDriverIO as being like the SeleniumJS binding on steroids!
WebDriverIO is agnostic about the test framework you use so that you can leverage tools like Cucumber, Jasmine, and Mocha+Chai.
Mabl is one of the newer automation testing tools on the market that focuses on the hot trend of using AI and machine learning to help folks with their testing efforts. In a nutshell, to get started you merely point Mabl to your application. It will then crawl your app’s screens and begin to run default tests that are common for most applications.
Along the way, it also uses machine learning algorithms to improve test execution and defect detection. Using the Mabl trainer chrome extension, you can also train Mabl to learn journeys in your application which describe the functionality Mabl will test.
31. Katalon Studio
Katalon Studio is an
open-source solution a paid vendor tool that has functionality like Record and Playback, reporting, Object Spy, Object Map, and more. You can use it to create automated tests for Web, API, and mobile apps. It even runs on both Windows and Mac.
32. Progress (Telerik)
For those of you who may not be familiar with Progress, they recently acquired Telerik test studio, which is the home of the favorite free debugging tool Fiddler. I also know a few test engineers who use Progress’s Test Studio as a front end for their Selenium test automation efforts.
Strengths of Progress are its integration with Visual Studio and its supported languages.
Developed for testing web applications, Sahi offers both an open source and a pro version. It works as a proxy server you can use within a browser. From the Sahi dashboard, you can launch the browser you want to test.
The Sahi controller guides you through the recording and capturing of the interaction you want to perform against the application. When you hover over any element in the Sahi browser, for instance, it shows you all of the actions you can perform against that element.
You can also use the controller to play back your script. Sahi’s Record and Playback functionality make it easy to get started with test automation.
Gauge is one of the newest open-source test automation tools for Mac, Windows and Linux. It was developed by the folks at ThoughtWorks–the same company that created Selenium.
Practitioners of TDD and BDD will appreciate Gauge’s focus on creating living and executable documentation.
Gauge automation tests, called “specs,” are written in a simple markdown language syntax using Java, Ruby, and C# within developer IDEs such as Eclipse and Visual Studio. You can also extend Gauge’s functionality with its support for plugins.
The 2017 Online Automation Guild conference had an awesome hands-on session on cross-bowser testing with Gauge and Selenium WebDriver that you can still get recordings for.
RedwoodHQ takes a slightly different approach than the other tools on this list. It creates a website interface that lets multiple testers work together and run tests from one Web-accessible location.
You can write tests in Java/Groovy, Python and C# to test Web applications with Selenium, APIs or databases using their web IDE, which also includes built-in Git integration.
You can leverage RedwoodHQ’s existing action keywords to start creating and modifying tests quickly. To create a test script, simply find the action you want to perform, drag it into your test case, and enter the parameter values it expects.
The built-in IDE lets you create and modify test cases and actions as well as execute tests. You can also run your tests against predefined machines in parallel and see a running history of all your previous tests runs.
RedwoodHQ is tester friendly and has many built-in features you can use to create complete test automation and test management solutions for your teams.
Windows Application Driver (WinAppDriver), developed by Yousef Durr at Microsoft, is the Windows implementation of Appium to automate Windows applications. This is really cool because it shows the next evolution of Appium to expand beyond just mobile applications and more toward automating all the things the Appium team is calling the StarDriver vision.
37. IBM Rational Functional Tester
Like most companies, IBM’s test portfolio has grown with the acquisition of tools like Rational and Green Hat. Much of the strength of Rational Functional Tester comes from its support of numerous technologies like Siebel, PowerBuilder, Java and can run on Windows, Mac, and mobile platforms.
Test.AI is billed as a tool that will add an AI brain to Selenium and Appium. It was created by Jason Arbon, co-author of How Google Tests Software and the founder of Appdiff. Tests are defined in a simple format similar to the BDD syntax of Cucumber, so it requires no code and no need to mess with element identifiers.
The AI identifies and discover screens and elements dynamically in any app and automatically drives your application to execute test cases.
Some time only developer ERP applications. In those cases, a tool like Worksoft might be the perfect choice for you. Worksoft is well known for its ERP business end-to-end solutions.
40. Oracle Application Testing Suite (OATS)
Does your team do a lot of Oracle-based software development? Need a tool to help test your regression and performance test your Oracle applications or databases?
Oracle Application Testing suite contains a bunch of integrated products to help with Oracle Functional Testing for regression testing of web applications and Oracle Load Testing for performance testing of your Oracle packaged solutions.
TestingWhiz motto is Code Less, Test More. They have solutions for Web Test Automation, Mobile Test Automation, Automated Database Testing, Mobile Test Automation and more.
TestCafe is an open-sourced Node.js tool designed to automate end-to-end web testing. For a full review check my article on out What is TestCafe
accelQ is an Agile testing platform for Functional & API Test automation web and mobile applications. Automate 3x faster with autonomics powered codeless natural language automation.
Guileet Nagpaul from accelQ mentioned that when selecting a test tool that having a solution that can handle your web testing all the way to your file interactions, to your APIs, to your database all in a simple flow without having to hire an army of developers to accomplish that is critical.
Bellatrix created by Anton Angelov from Automation The Planet is a cross-platform, easily customizable, and extendable .NET test automation framework that helps increases your tests' reliability. It free but it's not open source because Anton wanted to keep it high quality. Hear more here:
OpenTest is a free and open source functional test automation framework for web applications, mobile apps, and APIs, built for scalability and extensibility, with a focus on the mainstream test automation practices. OpenTest is a feature-reach tool that requires little to no coding skills and can handle virtually any type of functional test automation project.
The creators of Oxygen felt there must be a better way to create a framework that, on one side, is simple enough for someone from a non-programmer background, yet powerful enough for more advanced testers. So Oxygen was created to help both inexperienced and SDET's to write more advanced tests that cover not just UI but things like API testing, database testing, and IoT testing as well.
Boozang is a codeless solution that was created to allow you to quickly and easily build end-to-end workflow tests using only your web browser.
TestProject is a free end-to-end test automation platform for web, mobile, and API testing that’s supported by a large test automation community.
A growing trend in automation is RPA. One of the easiest tools to use in this area is Leapwork which requires no coding to create fairly complex automated processes that include test automation flows.
An AI-based tool to help you with your mobile testing efforts.
GameDriver is a framework that provides developers an API agent that they can embed into their games so that when the developers are creating the build, they embed the driver game object within their game. If you need to test Unity games you need to check this out.
Microsoft Playwright is a new cross-browser Automation library for end-to-end testing. The goal of Playwright is to provide a single API to developers and testers so that they can automate their web applications across the three major browser engines.
Pylenium is a wrapper around the Selenium WebDriver Python API, which provides a more Pythonic interface. If you’re new to automation and/or are looking for an easier way to create automation tests.
SeleniumBase is an all-in-one framework for reliable browser automation, end-to-end testing, reports, charts, presentations, website tours, and visual testing. Tests are run with pytest. Browsers are controlled by WebDriver.
EndTest is an affordable codeless automated testing solution. Aimed at teams that want to build tests faster, without writing any code.
Appsurify is an AI-based automated testing solution that helps you find the right subset of your automated tests to match the specific code changes, making your testing run 10x faster. TestBrain also quarantines flaky failures so they don’t break your builds, and identifies the risks in recent commits to prioritize your manual testing so you can release faster with fewer bugs.
Smashtest is an open-source tool and language for rapidly describing and deploying test cases. It also allows you to how to significantly speed up your automated testing by writing tests in a tree-like format.
Selenide looks like a very complete automation test tool framework. If you are using Java and need to create some automation, you definitely should give it a try since it takes care of many of the common complexity and issues testers run into using just Selenium.
Scriptworks provides an elegantly simple visual interface, empowering manual testers to use their domain expertise to design scalable and reusable automated test packs to address today’s continuous change demands.
What About Non-functional Automation Testing Tools?
For other non-functional automation testing tools to compare and check out:
- 15 Open Source API Testing Tools for REST & SOAP Services
- Top 11 Open Source Performance Testing Tools for Load & Stress Testing
- The Top 21 FREE Visual Validation Tools for Testers
- Top Accessibility Testing Tools for Automation
- 7 Innovative AI Test Automation Tools for the Future: The Third Wave
- 7 iOS Testing Tools You Need to Know
- Top Main Frame Automation Testing Tools
How to Stay Up To Speed with the Latest in Automation Testing?
If you feel overwhelmed with staying up-to-date with all the latest automation testing tools and best practices check out my annual online conference dedicated 100% to just automation testing — Automation Guild.