Each year, I pick up on trends gleaned from interviewing folks on my podcast, as well as news stories featured on my TestGuild DevSecOps News show.
Also, developments in one year tend to set up the trends for the following year.
One key that always acts as a leading indicator for a trend is money invested in companies.
With that in mind, I’ve compiled my list of the top automation trends for 2023 using the above criteria, just like I’ve been doing for the past 12 years.
Key Takeaways:
- The use of artificial intelligence and machine learning in automation testing will increase, allowing for more efficient and accurate testing.
- There will be a greater focus on test automation in the early stages of the development process.
- There will be an increase in the use of open-source automation testing tools and platforms.
INDEX
1. Innovation in Mobile Testing
2. Continuous Accessibility Test Automation
3. Unified Test Management Platforms
4. The Top Testing Programming Languages for 2023
5. DataOps for Testers
6. SQL for testers
7. Synthetic Data (AI-driven synthetic data)
8. AI Automation in production
9. API Simulation
10. Self-Service Cloud Development Environments
11. Platform Engineering
12. AI Assistance in Automation Testing
13. Ask Better Questions
What To Do With These Trends
Automation Testing Tips and Tricks
The first trend is….
1. Innovation in Mobile Testing
The tooling within mobile application automation hasn't had the same development time compared to web application automation.
I believe mobile automation is still in its infancy regarding which tools and frameworks allow you to combat that flakiness and maintain your scripts.
But based on the past year, I see this changing in 2023.
While AI/ML has been focused on web-based testing during the last few years, I see this extending even more into mobile.
This goes along with the new tech trend we see ourselves in, commercializing artificial intelligence.
That’s why we’re seeing more AI-driven codeless, scriptless mobile automation solutions being released.
Some of the examples I’ve seen over the past year that are pointing toward the mobile automation future are:
- Mobot—a mobile app testing service powered by supervised mechanical robots. The best way to imagine Mobot is as a fleet of mechanical fingers in the Cloud that can interact with your mobile devices (or any hardware device) just like a real user.
- Kobiton—an Intelligent Quality Suite which leverages AI to support scripted mobile Appium automation scripts. Features scriptless and codeless technology.
- Dev-Tools.AI—uses artificial Intelligence to automate webpages and mobile apps without digging into page sources.
- Loadmill—leverages APIs for all test automation, including mobile test automation (more about them later).
Follow the money in 2022 (key trend indicator):
- Waldo raises $15 million for its automated mobile testing service
- Mobot raises $12.5M in Series A funding
This trend is so important that we’ve decided to include multiple sessions on mobile testing at the 2023 Automation Guild.
2. Continuous Accessibility Test Automation
Almost everyone I spoke with this year, whether on my podcast or during a live event, mentioned accessibility testing.
Over the past year, every major testing technique has slowly been shifting left in the software development lifecycle, from automation to performance, to security testing.
That’s one of the reasons why I’m predicting that 2023 will be the year for Continuous Accessibility testing making the shift.
I recently spoke with Tamar Schapira, founder of the startup company SenseIt, during an Applitools Future of Testing event. Tamar is developing automated testing for digital accessibility and said that her team has begun working on assessing digital products manually for their clients.
Over time, they’ve found this process to be pretty tedious.
They realized it had begun to resemble a digital hamster wheel, where clients depended on them to test, assess, return for remediation, reassess the product after remediation, and continue this process, repeatedly.
Tamar mentioned that knowledge of accessibility testing must be in the hands of the company.
I wholeheartedly agree.
Accessibility, compliance testing, and assessment must be addressed early on and continuously.
You need an automated solution that knows how to handle the interaction, just like you're doing with functional testing.
Tamar’s vision (which I’ve seen applied to security and performance as well) is the ability to leverage your functional tests and find a way to use those and interact with the products from an accessibility perspective.
You can simultaneously test accessibility while leveraging a company's functional testing scripts while executing your test automation.
On-Demand & Upcoming TestGuild Training Sessions
3. Unified Test Management Platforms
With an increasing focus on automation and DevOps, software delivery is faster than ever.
Automated testing is essential for balancing software quality with the accelerated velocity of release cycles, but because the toolchain that is supposed to enable automated testing has many moving parts, test automation becomes a parallel development project in organizations.
Plus, they have huge upfront and ongoing resource, tech, and expertise costs.
This, to my way of thinking, has led to a new breed of Test Management Systems that will grow in popularity in 2023.
This is not your father’s Test Director; these are platforms designed for modern software testing.
Although the automation capabilities for testing have been mainly solved by test automation tools like Appium, Selenium, and other frameworks, there is still a gap in the industry for a unifying platform to help manage the entire automation lifecycle.
Some examples from my test automation podcast:
- Testsigma—A unified, fully customizable platform that works out of the box. Built to help you quickly create end-to-end tests for Web, mobile apps, and APIs with English scripts that self-heal, enabling maintenance-free testing functionality.
- TestResults.io—a next-generation, functional, visual test automation platform designed to help your team quickly create more reliable automation that mimics how your end user uses your application. Image visual validation testing can also drive a user interface without using a solution like Selenium.
- Tricentis Test Automation—delivers frictionless access to an intuitive UI that helps with speed learning and accelerates teaming. Access and reuse test assets and build complex, multi-app test cases leveraging your team’s expertise. Simplify and accelerate automation.
Follow the money:
Testsigma raises $4.6M from Accel and STRIVE to simplify test automation.
4. The Top Testing Programming Languages for 2023
Test engineers usually want to know which programming language(s) they should learn.
I discovered several sites that tracked the popularity of languages in 2022, and most of them have them listed in a similar order (1. Python, 2. Java)
How does this jibe with what employers want for SDET testers?
For fun, I looked at the job descriptions of test engineers at the top 20 Silicon Valley companies. I then created a Word Cloud to determine whether there was a correlation between popular languages and the skills companies say they are looking for.
As you can see, the languages most of the job descriptions list as requirements are Python and Java.
5. DataOps for Testers
Last year I mentioned the rise of Cloud-native testing.
I think the need for DataOps is a result of this shift to cloud native.
As a result of this shift, there is considerably more volume and more sources spread across more Clouds, making it more challenging to discover, manage and control data.
I recently had the pleasure of hosting an online RTTS QuerySurge event dedicated to data testing. Bill Hayduk told me that:
According to a recent IDC study, 80% of organizations store more than half their data in hybrid or multi-Cloud infrastructures.
Also, 79% of organizations have more than 100 data sources, and 30 have more than 1,000. This is data flowing in from other applications, websites, etc. And only 6% of organizations have been able to find a way to standardize across all data management functions.
Gartner has also offered some exciting quotes, like:
“Poor data quality cost organizations an average of 12.9 million, and it increases the complexity of the data ecosystem that leads to poor decision-making.”
And:
“Good data quality provides better leads, better-understanding customers, and better relationships.”
And:
“Data quality is a competitive advantage that data and analytics leaders need to improve continuously.”
Gartner feels there's a big spotlight on data and data quality.
The World Quality Report has a survey population of 7,150 senior executives across multiple sectors from different countries.
Quite interesting, I think, is that for the first time in 14 years, the World Quality Report mentioned data validation.
They said that 89% of those polled agreed that a robust data validation capability would not only improve efficiencies in terms of time and resources but more importantly, will help to improve business decision-making.
They also said 88% of respondents agree that a robust data validation capability directly impacts customer satisfaction and the accuracy of insights that will help boost business profitability.
One tool to take a closer look at in this space is QuerySurge.
QuerySurge is an intelligent data testing solution that automates the data validation and ETL testing of Big Data, Data Warehouses, and Business Intelligence Reports. QuerySurge ensures that the data extracted from data sources remains intact in the target data store by quickly analyzing and pinpointing any differences.
This is one of the few tools that do DevOps for data, automating your pipeline with a tool.
I’m pretty sure that based on this trend, more will follow.
It might also explain the next trend; testers need to know SQL.
Get FREE Automation Testing Courses
6. SQL for testers
According to the IEEE Spectrum Top Programming Languages 2022, the popularity of SQL is on the rise.
It’s Number One in their Jobs ranking, which looks solely at metrics from the IEEE Job Site and CareerBuilder.
They also mentioned that they looked through hundreds of job listings while compiling their programming rankings and found that the strength of the SQL signal is not because there are a lot of employers looking for just SQL coders, in the way that they advertise for Java Testing experts. They want a given language plus SQL. And lots of them want that “plus SQL.”
This also holds for testers.
Just as I did with programming languages for testers, I also did a search for test automation trends and SQL and found multiple job postings looking for automation engineers who also know SQL.
Speaking of data, get ready for the rise of Synthetic Data in testing.
7. Synthetic Data (AI-driven synthetic data)
Synthetic data is artificially-generated data that is designed to mimic real-world data. It can be used in various contexts, including software development and testing.
In software development, synthetic data can be used to test and debug code without needing real-world data. This can be especially useful in cases where it is difficult or even impossible to obtain real-world data or where using real-world data would pose a risk (e.g., due to privacy concerns). Synthetic data can also simulate various scenarios or edge cases that may be difficult to test with real-world data.
In software testing, synthetic data can be used to test the functionality and performance of a software application. It can help to ensure the application is working correctly and as intended and can also be used to stress-test the application to see how it performs under heavy load.
Overall, synthetic data can be a valuable tool in software development and testing, as it allows developers and testers to work with data similar to real-world data but without the limitations or risks associated with using real data.
And, as you know, data is the livelihood of modern artificial intelligence.
Getting the data right, especially in testing, is one of the most critical and challenging parts of building a robust test suite that leverages AI.
That’s probably why I’ve heard from multiple companies focusing on this space.
Access to quality “fake” data is helpful across the software development life cycle, from your sandbox environments to your development environments, to staging, to testing, to QA throughout the CI/CD pipeline.
For example, Tonic.ai is a platform that enables teams to create the data they need. Rather than receiving your data and synthesizing it, they generate the data for you.
It’s a platform and all the tools you need to generate your data for your teams the way you need it. Tonic works by pointing directly at your production databases.
You can point Tonic.ai to different databases of different types based on your data needs. It can then use its built-in generators. The generators are like building blocks that mix and match and are specific to the data types you are trying to protect or synthesize and fake.
Another solution, Mostly AI, completely removes the need to use production data or create dummy data, which is what many testers are still doing. Apart from some apparent privacy issues that come with doing it this way, it's a massive time thief; it eats up huge chunks of your average time spent writing test data, which no one wants.
Using these data tools can help you scale up your synthetic data creation.
Gartner predicts that 20% of all test data for consumer-facing use cases will be synthetically generated by 2025, so it's perfect for you to get on board the synthetic data train.
It’s a growing trend and fills a huge void I've been aware of over my 25-plus years in this industry.
Follow the money:
MOSTLY AI raises $25 million further to commercialize synthetic data in Europe and the US
Tonic.ai raises $35M Series B to help engineers create synthetic data sets
8. AI Automation in production
Using AI in production to help test your application is another trend I heard more about in 2022.
One interesting use case is to use production user interactions and automatically generate tests based on actual user data.
This is great because many companies struggle with automation testing because they focus solely on UI automation.
As a result, many teams discover that UI-only automation is not scalable. It's also impossible to do everything with UI.
I still speak with testers who struggle to understand how to approach API testing.
Of course, API testing has gained in popularity over the past few years, but once again, AI/ML adds a new twist I think we’ll be seeing more of.
I’ve spoken with a few companies that have testing solutions that listen to actual production user interactions with an application and, using AI, automatically generate web and API tests for you.
One of these solutions is Loadmill.
Loadmill is a test automation platform that focuses on backend and API testing. Its AI algorithm can analyze your user behavior and automatically turn that analyzed traffic into automated API tests.
You can even import data from monitoring solutions and feed it into Loadmill.
Loadmill takes all this info and creates an uber-realistic, real-world, user-based, automated test based on that scenario–automatically.
Think of a Web automation recording tool, but for APIs.
I’ve seen other tools that also take production data and automatically create real-world models for you that can generate automated testing.
Looking at the recordings and analyzing them from a QA perspective brings you closer to the actual customer and replicates customer behavior.
Another use case for AI in production is helping automate infrastructure in production.
For instance, Akamas focuses on optimizing apps with intelligent, safe, autonomous tuning in production.
I spoke with Stefano Doni, Co-Founder and CTO at Akamas, on my Performance and SRE podcast this year, and we discussed how their solution could observe applications in production.
Over time, it can also understand parts of your application along with traffic patterns, and based on that history, its AI models can then make recommendations or automatic adjustments to fix issues that occur before your users are even aware of them.
I believe these types of solutions are primed for massive growth in 2023.
Speaking of API and Microservices, with modern software development, how do you handle the challenges that come in an API, development-first world?
Look to API Simulation.
9. API Simulation
API simulation simulates the behavior of an application programming interface (API) to test the functionality and performance of a software application.
In software testing, API simulation involves creating a simulation of an API and using it to test the application rather than using the actual API.
API simulation can be helpful in several different contexts.
For example, it can be used to test the integration of an application with an external API, such as a third-party service or an internal API within an organization. It can also be used to test the performance and reliability of an application under various conditions, such as high traffic or different network configurations. (To see this in action, check out the webinar I did with Parasoft’s Grigor Trofimov, Take Control of Your Test Environment with API Simulation.)
API simulation (also known as service virtualization) stands in when you don't have access to dependencies and real systems and want to mock something evolving.
Chris Colosimo of Tricentis told me that the last two years have been transformative for many of the currently available software.
Folks deliver software in ways they didn't anticipate, which was unheard of even two years prior.
And what that calls for is a lot of change in architecture and rapidly deploying services.
Using API simulation can get new products delivered into a customer's hands the way the modern world demands, and it's all happening simultaneously.
So, we will see an explosion of service-virtualization concepts coming into play over the next few years.
Think about it; when things happen in parallel, APIs are often not ready, unavailable, or unstable.
These are just a few of the reasons why you need service virtualization.
Something else I see frequently is that quite a few organizations have taken this as an opportunity to retrench and ask themselves, “Are we delivering software with the best possible architecture?”
Whatever that means to you–a transformation of technology, something a little lighter, a little faster, or if it means let's pull off their on-prem and go into Cloud 100% and deliver software that way—all of those are catalysts for the explosion of service virtualization that we're about to see.
API simulation can be a valuable tool in software testing, as it allows developers and testers to test an application without relying on the actual API's availability or functionality.
It also allows them to test the application in a controlled environment, which can be useful for isolating issues and identifying problems.
Examples of innovation in this area based on my 2022 podcast interviews:
Tricentis Test Automation SaaS
Parasoft Virtualization in Test with Grigori Trofimov
10. Self-Service Cloud Development Environments
Self-service Cloud development environments are platforms that allow the developer to quickly set up and configure his or her own development environments in the Cloud. These environments typically provide a range of tools and resources that developers can use to develop, test, and deploy software applications.
In the context of software testing, self-service, on-demand Cloud development environments can be a valuable resource for automating and streamlining the testing process. They can provide a range of features and tools that can help to improve the efficiency and effectiveness of testing, such as:
- Automated build and deployment pipelines that allow developers to test and deploy code changes easily
- Virtual machines and containers that can be used to create isolated testing environments
- Integration with testing frameworks and tools, such as Selenium or JUnit, to automate the testing process
- Collaboration tools that allow developers to work together and share test results and feedback
Overall, self-service Cloud development environments can be a valuable resource for software testing, as they provide a flexible and scalable platform for testing and deploying software applications. They can help developers streamline the testing process and improve the efficiency and effectiveness of their testing efforts.
In my interview regarding Bunnyshell, Shani Shoham called it an engineering productivity platform that essentially cuts delay time to release.
It solves the problem of “It works on my machine.”
These advanced technologies create full-stack production replicas in the Cloud.
Tell me how familiar this sounds:
The way that most engineering organizations work is you have developers working in their local Docker environments.
Then they merge that into the main branch. It doesn't work because the main branch is different. And often, the version they forked is already an old version.
They then spend a lot of time reworking the code and merging that into a staging environment.
Of course, the staging environment is also different.
They run the test there, find bugs, and then go back to the developer.
Again, they rework the code and rebuild the application.
Then it goes into testing, and it starts all over again.
It's a very, very lengthy and unproductive process.
A better way would be to use this technology to trigger a full-stack environment with every pull request in the Cloud.
So you can immediately have a fully functioning production environment user acceptance testing with QA for testing purposes.
QA can then test on an isolated environment, and you can perform as much testing in parallel branches as you need to.
If you detect a bug, it's effortless to isolate bugs to that specific branch. The developer can connect using remote development, connect to that environment, and make changes to the Cloud or the environment in the Cloud. QA will immediately see those changes. No need to rebuild and wait again and again and again.
Once everyone's happy, you merge the code, tear down the environment, and that's it.
This also keeps all the environments in sync, so the abovementioned challenges no longer occur.
But the idea is with full-stack environments, the code quality that a developer can produce is much higher.
The feedback loop between Dev and QA is much, much faster.
In my webinar Playwright + Uffizzi: Supercharging Your Testing Efficiency with Josh Thurman and Butch Mayhew, Butch mentioned that using this approach has helped to create smaller feedback loops.
Making it possible for his team to be able to test things quickly and get things out the door quicker
Companies to check out in this space
11. Platform Engineering
Platform engineering is a discipline that focuses on the design, development, and maintenance of software platforms. A software platform is a set of technologies or tools that provides a foundation for building, deploying, and running software applications.
In the context of software testing, platform engineering plays a critical role in ensuring the quality and reliability of software applications. This can involve a range of activities, such as:
- Designing and implementing the infrastructure and tools needed to support the testing process, including continuous integration and continuous delivery (CI/CD) systems
- Setting up and configuring testing environments, including automated, performance, and security testing.
- Developing and maintaining testing frameworks and libraries that can be used to automate the testing process
- Collaborating with other teams, such as development and operations, to ensure that the platform is reliable, scalable, and easy to use
Overall, platform engineering plays a key role in software testing by providing the foundation and infrastructure needed to support the testing process. By designing and maintaining a robust platform, platform engineers help to ensure the quality and reliability of the software applications built on top of it.
So what does platform engineering have to do with automation?
According to Evan Niedojadlo, automation is the end game.
You're developing tools to automate processes that make the developer's SDLC more effortless.
12. AI Assistance in Automation Testing
You’ve probably heard about ChatGPT by now.
If not, check out the special edition of my news show, where I highlight how it can automatically generate Selenium, Cypress, Playwright, and WebDriver.io scripts.
While it’s not perfect, it will only get better, and I believe it highlights the direction that AI-assisted automation is heading.
Imagine if some test tools/IDE start building integrations with ChatGPT.
This is a great starting point to fix/modify and verify that it is doing what you expect.
While not replacing development skills or testers by a long shot, it is a great starting point and time saver.
I’ve seen a lot of folks downplaying this technology.
Don’t be one of them.
At the time of this writing, it’s only been live for about a month and is only trained data from 2021 and before.
Over the next few years, it will only become more intelligent and sophisticated.
13. Ask Better Questions
Focus on a skill that will become even more relevant in software testing when dealing with AI—ask better questions.
Asking better questions can help you yield more accurate and valuable results from the AI system. For example, if you use a language model like ChatGPT to generate text, asking more specific and targeted questions can help the model provide more focused and relevant responses. This can be especially useful when troubleshooting an issue or gathering information for a specific task.
Additionally, asking better questions can help you better understand how the AI system works and what it is capable of, which can be helpful when you are trying to use it effectively.
What To Do With These Trends
One of the burning questions I’m always asked when discussing a subject like this is, “Will AI replace me?” Or, “Will the current recession scare looming on the horizon jeopardize my job?”
Look, no one can truly predict how things will turn out because that’s beyond our control.
But in my 25+ years of field experience and humble opinion…
What you CAN do is make sure you’re always armed and ready with the latest info on current tech.
What you CAN do is stay up-to-date with industry know-how and relevant software testing trends.
That’s one of the reasons I create this post each year.
What you CAN do is always strive to improve your skill set and build a solid network.
That way, you’ll always be in demand regardless of how things turn out.
Because that’s EXACTLY how I survived (and even thrived) over the decades, no matter the “scare” or its consequences.
So, if you seriously want to separate yourself from the pack and gain an elusive edge in your career in 2023:
You might want to take a closer look at Automation Guild 2023 since I designed it to address many of the trends covered in this post.
Consider celebrating this New Year by investing in yourself to supercharge your E2E automation testing skills and get a leg up in your career, Joe.
