Automation Testing Trends for 2024 with Joe Colantonio

[00:00:00] Joe Colantonio As we enter 2024, what automation testing trends will shape the future that you need to know? Based on hundreds of expert interviews and funding patterns in the test automation space. Here are my top 12 trends you need to know for the future of test automation. And stay all the way to the end because you're probably going to be surprised by the last one. Also, make sure to subscribe and hit the bell to never miss another piece of automation awesomeness. First one is obvious, so let's get it out of the way.

[00:00:29] Number one is AI-assisted testing boom. A.I machine learning will become even more integral to helping you boost your testing efficiency. But is any tester really using A.I. to help with testing now? Well, surprisingly, based on surveys like Lambda Test in the Test Guild webinar polls, most testers are already adopting A.I. into their workflows. So if you're not experimenting with A.I. in testing yet, you're already behind. So the Lambda Test survey found that A.I. adoption among software testers was at 78%. In the Test Guild webinar on Generative A.I., 76% of testers said they're already using ChatGPT AI to help them with their day-to-day testing activities, or they plan on doing so. And I think this already high adoption rate will grow even more as more choices in AI testing too, and platforms are going to proliferate across different software teams in 2024. But remember, real software testing obviously requires a deep understanding of the software under test, so it requires a real tester. However, it does not rule out A.I. assistants with many activities like automated test generation, intelligent test data creation, also things like predictive defect models, test optimization prioritization, and automated root cause analysis that I know a lot of guild members struggle with. Once again, all these activities require a human tester to make sure the AI-Generated data is correct, but it doesn't rule out AI, don't ignore AI. Also, another key indicator is companies investing in A.I. technologies. So in 2023, we saw platforms like Perfecto release Test Data Pro by BlazeMeter which leverages Generative AI technology to simplify and enhance data creation. We saw Katalon announce TrueTest, which is an automated regression testing solution to leverage AI to generate and maintain regression test suites by monitoring user activities. And we saw companies like TestResultsio integrating GenAI to create test cases and also Reflect integrate AI with a bunch of different functionalities including Generative AI and also boosting Playwright automation using A.I. And if you think 2023 was crazy for AI in testing, I think that's even more interesting developments in the AI and testing with the recent release of Multimodal AI, one example being Google Gemini. We haven't heard Multimodal A.I. incorporates multi-data types like text images, speech, and sensor inputs processing via different algorithms to enable more flexible automated testing capabilities. It's really cool because it's going to allow multimodal AI to enhance test automation with things like visual application validation, leveraging test bots to use computer vision alongside text processing to traverse and validate graphical interfaces in the way a human tester would for greater test coverage. It's also going to expand to different types of testable applications that you probably weren't able to test before because it has the ability to interpret images, video, speech, all those things that people probably struggle with now with automating, and it's going to help you a bunch of other things like mainstream accessibility, integration. Those are just some quick examples I foresee happening in 2024 with multimodal AI, with AI automation testing. And I think this trend is so important, we've included a bunch of sessions at this year's Automation Guild to cover AI. We have one session, A Practical Guide to AI, How to Improve Quality in Software Development. We also have From Code to Cognition, The Journey of AI in Automation as just a few examples of what you'll get at Automation Guild. If you haven't signed up for Automation Guild. You want to upskill yourself to keep up with all these trends we cover in this video. Head on over to automationguild.com and register today.

[00:04:15] Second trend two years in a row is testing for low-code, no-code apps. In a survey, showed that by 2025, 70% of newly developed applications will use low-code and no-code technologies, up from less than 25% in 2020 because of this testing needs will grow exponentially as adoption of low-code no-code app development spikes across companies. Demand is going to surge for optimized automated testing solutions that are tailored for low-code platforms. I think you need to focus on more end-to-end testing due to this because low-code apps often support web, mobile, and other channels which require automated testing across multiple services. There are also going to be some integration testing challenges, connecting low-code apps into existing systems requires API and integration testing for end-to-end flows that you need to know more about. It's also going to enable non-technical users to do more automation, and you're going to need to focus on making it scalable across all your different application portfolios. And how I know this is going to be a point as well as companies that invested in 2023 is like kind of like a flag that shows me this is what companies will be focusing on in the new year as well. For example, UIpath, which is used mostly for RPA, but also can be used for testing processes, introduced AI and will make it even simpler to automate almost any business task using natural language which natural language equals low-code no-code. Also, Applitools announced its acquisition of Preflight in 2023, which is a leading low-code test automation platform. Why these companies investing in low-code no-code technology? Probably because they see that this is a trend that's going to be happening in 2024. So it's something you should pay attention to as well. And once again, we can have a bunch of sessions on low-code no-code at the Automation Guild. We have a session on empowering product owners and other non-technical team members to use low-code test automation. Also, we're going to have a real-world session on how a tester approaches testing and AI chatbot powered by natural language processing. Number three on the list is API Test Automation has been on my list pretty much every year since I've been doing this, but I believe it's so important that you still need to pay attention to it because, as you know, APIs serve as the connectivity fabric across systems in modern microservice architecture. And with API's usage multiplying automated API testing capabilities will need to expand in 2024 to validate not only functionality but also performance and security pre-production. Some key aspects of API testing automation needed in 2024 are going to be functional validation. Obviously, load in performance testing, executing high volumes of load tests against APIs to cover like latency bottlenecks and errors under load security testing validation is going to become more important. Contract validation, service virtualization simulation, and service virtualization simulation is going to be a bigger trend as well. One example is Parasoft announced AI-powered API testing solution to help you with this, Postman announced Postbot AI which can help you with your testing and Loadmill and other companies as well. Karate, Tricentis have all invested in API testing and I think want to see more trends where people using tools like Cypress and Playwright to help them not only with functional UI automation but also API testing. And because I believe in API testing so much. Once again, we have a bunch of sessions at this year's Automation Guild. We have API automation mastery from novice to ninja, accelerating API testing using Cypress, How to mock the unmockable, elevating your test automation stability through advanced API simulation, and also a really important topic on safeguarding your digital assets, uncovering security risk in your APIs that you definitely should check out as well.

[00:08:06] And number four is shift left testing goes mainstream. Once again, talk about shift left forever, but I think now it's just common. I think it's going to become just the norm in 2024 because you're going to need to find bugs quicker and cheaper and DevOps teams are going to continue to shift testing left in the lifecycle. It run earlier against code changes in the CI/CD pipeline. So you're going to see a rise in unit testing, integrated testing, performance testing, and more activities that are definitely going to shift left and become the norm in 2024.

[00:08:35] Some other types of testing I think are going to move into the pipeline even more in 2024 as well. It's going to be component testing, integration testing, API contract testing, and security scanning is going to be huge, and also accessibility validation assessment for new features against accessibility standards to drive exclusion, using automation I think is also going to be big in 2024, we have two sessions on this on Automation Guild as well. We have automated quality assurance reporting in your CI/CD pipelines and we also have a bunch of topics on shift-left performance testing as well that you should definitely check out. Then number five trend in 2024 is containers enable test environment consistency. So using containers to provision test environments provides consistency across stages, improves collaboration across teams, and also helps enable automated environment provisioning. If you're not using containers yet for your test automation environment, you need to. It is one of the killers of an automation project and using containerization, which automated environments make automated testing more scalable and more reliable for sure. And some reasons for this it helps you isolate dependencies, so replicating continued packages and standardizing all libraries data configs needed for code execution simplifies your testbed creation. It gives you a consistent environment and it helps you automate scaling a teardown with programmatically spitting up containerized test refinements and pipelines stages. Then automatically scaling up or down to string them on demand is huge to get you to scale and get all these shift left activities that we talked about more into your pipelines easier by be able to scale up all the environments, run the test as quickly as possible and just tear them down and destroy them. Going to be huge in 2024. Definitely, you want to get on it for sure.

[00:10:19] Number six is automated compliance testing acceleration. So over the past few years, there have been a lot of regulations passed covering security, data privacy, industry standards, and more. I think automated compliance control validation is going to accelerate to keep pace with all these new laws and regulations in 2024. And it's human nature because I know many developers find manually validating controls required by regulations to be cumbersome busywork, whereas, automation can execute invalidate these control checks more frequently. So neglecting compliance testing automation makes achieving continuous delivery with governance impossible at scale. So that's why I think it's an area that's going to expand in 2024.

[00:11:03] Number seven, it's self-healing drives higher resiliency. I think this will lead to more automated remediation capabilities, things that kick in based on policies to -roll back deployments or auto-scale resources when health checks fail to help minimize downtime are going to be on the rise in 2024. And even with rigorous testing, production issues inevitably occur requiring rapid response. And when you're in production, manual processes really slow reaction time. So automated self-healing mechanisms I think are going to proliferate in 2024. So some examples of some self-healing capabilities I think that really going to take off are automated rollbacks, live container restarts, intelligent requests routing, predictive auto scaling, and graceful degradation functionality as well. And number eight is AI Ops. So using big data and machine learning, AI Ops platforms analyze volumes of telemetry info, that's been huge in 2023 with open telemetry, a lot of companies investing in telemetry in 2023, you're going to need to know in 2024 and beyond. So telemetry, this type of information really helps teams quickly troubleshoot, predict, resolve, and operation issues. So some examples of why AI Ops is going to be more important in 2024. Type of features that are going to be bigger in the new year is anomaly detection helps you analyze metrics to unmark, recognize anomalies for normal activities that indicate emerging issues, log aggregation and analytics collection, and correlate massive log data volumes across systems to identify related events and rapid pinpoint root causes. Incident Prediction Spot Trends Predictions for future incidents like disc space exhaustion, allowing proactive prevention. It's going to help you with things like automated remediation and also optimize cloud cost continuity tune infrastructure sizing base on utilization to optimize cloud cost leading examples of companies that have embraced this in 2023 and I do want to see more of in 2024 is like Dynatrace released Davis A.I. with Davis CoPilot, Splunk announced, Splunk AI. And Datadog announced BitsAI, all these help you with AI Ops and DevOps and all these tools were designed to help you with infrastructure growing exponentially in scale and complexity and how AI Ops becomes essential for your teams to really effectively manage reliability and costs of these systems using AI.

[00:13:30] Number nine is increase automated mobile app functional and security testing is going to be on the rise. The global surge mobile application in 2024 requires more automated testing capabilities specifically for real-world end-to-end user scenarios, as well as mobile apps with validation against things like security, OWASP, Mobile top 10 threats. And some key mobile testing-focused growth areas you're going to see in 2024 is more need for real device testing, more end-to-end workflow testing, cross-browser, and OS validation, also native performance testing. And like I said, OWASP mobile security validation I think is going to be on the rise as well. And you need to know how to address these unique testing needs of mobile testing because it's a different approach than you would normally find with UI testing. 2024 Testing solutions need to keep up with the faster mobile release cycles we've been seeing. So neglecting mobile test automation really threatens your customer experience and security and that's why it's so important in 2024. And once again, we have some companies that just give a little indicator of where they're investing their money and why this is probably important for you to know as well. For example, Tricentis acquired Waldo, which is a codeless development in the field of mobile test automation, which I think is going to be a bigger development in 2024. Also, Mabl released mobile testing features to help you with Android and iOS app testing. And companies like Game Driver and Kobiton announced partnerships to elevate mobile game development in testing, which I think is an area of growth as well in 2024. And Automation Guild will have two sessions. The first one is removing the pain from mobile test automation, and the other one is going to be a 90-minute workshop called Mastering XCUI Tests, Native iOS testing that I think you're going to get a lot of value from as well.

[00:15:24] Number ten, the rise of testing in production and non-deterministic testing. So because a lot of people are using SAS applications and cloud-based applications, user volumes ballooned to millions and actually mimicking production scale in test labs really proves almost to be impossible. And this leads to teams conducting final validation of new features safely to well-instrumented production testing, which I think will be on the rise in 2024. Also, sophisticated observability tooling allows deep inspection of behavior and performance. We're also going to see the rise of non-deterministic testing. Going on the days of test environments at full control and deterministic order of operations that we had back in the day when I was coming up as a tester. Nowadays, we have multi-cloud microservices and external APIs introducing a lot of unpredictability. Test automation is going to need to evolve in 2024 to handle these nondeterministic systems. And here's an area I did want to see AI step in with some techniques to help you for smart test generation and insertion guidance for nondeterministic pathways. So we're going to see an increase in innovations in production testing observability. So observability is going to be huge, but you can only make something observable that was built to be observable.

[00:16:39] This leads me to trend number 11. All this is going to lead to developing for observability. So the same struggles we saw with automation testing, security testing, and performance testing, you can't make something automatable that wasn't built to be automatable. You can't make something testable that wasn't built to be tested. You can't make something performant that wasn't built to be performant. Once again, in 2024 we're going to see observability shift left and make it so you can't make something observable if it wasn't built to be observable. So as teams increasingly test directly in production environments at scale, substantial improved observability, its applications, and infrastructure performance becomes mandatory in 2024. If you don't have this, especially in testing in production, without strong observability, it's going to have limited production test data visibility. You have the inability to access broader customer impacts. You'll find slower detection of failures from new features if you don't have observability baked in. And also you can have a lack of production telemetry to validate fixes. So by developing for observability upfront, as developers are creating code to make it more observable, teams are going to be able to create safer production testing and release capabilities like traceability, fine-grained metrics, which help you really quantify performance impacts. Logs provide rich context in error conditions in dashboards quickly highlight regression issues that you need to be aware of. Because testing production is going to be on the rise in 2024, you're going to need to invest in observability. And once again, how can you learn more if you're into testing and how you can help inform your teams around observability and how to make better choices of testing and production? We've got you covered in Automation Guild. We have four different sessions. We have an interactive story about observability and synthetic testing. We have integrating DevOps and Biz Ops for enhanced automation abilities, and we have distributed system observability using open telemetry at this year's event.

[00:18:39] And so for the most controversial trend I have picked out for 2024, number 12 is the most popular automation tool. At the end of 2024 will be Playwright by a landslide. I speak to a lot of testing experts on my podcast. I also predicted in 2023 that I saw Microsoft Playwright as possibly being the reigning champion of web-based automation testing tools. And I think I was proven correctly. And I will say upfront, I think 2024 will be the year of Playwright as the dominant automation testing tool for the new year. And if you use a tool like star-history.com to plot the history of likes or stars on GitHub, you could see how Playwright has drastically overtaken both Selenium and Cypress as the most widely used test automation tool. And while not a scientific or optimal way to gauge tool adoption or which one's the best, we're not saying which ones the best is just gauging interest on popularity, I do find GitHub stars to be a trackable indicator that you can least use to get a pulse for what people are trending towards. So for example, at the time, this recording Playwright's growing popularity is evident by its 57,000 stars on GitHub, far surpassing Selenium's 25,000 stars and Cypress's 45,000 stars. And as Playwright continues its rapid ascent, multiple testing thought leaders have voiced optimism about its future dominance. Many people I talk to vendors as well have started adopting Playwright into their solutions. As Playwright continues its rapid ascent, multiple testing thought leaders I've spoke to has voiced optimism about its future dominance, for instance. Many folks have told me that Playwrights solves many pain points. They've experience in teams of experience with previous tools, and they rave about how the API design is top-notch in the multi-browser support has been a game changer for their test automation strategy. Some also predict that as more teams struggle with the limitations of older tools, Playwright will become a go-to recommendation for test automation for modern web app testing. And many vendors I speak to, many people on my show has started to really think Playwright first when they take it of new features to add to their solution as well. And if experts predictions prove accurate, I expect Playwright to claim the test automation throne in 2024 based on its impressive momentum and ability to address common test automation challenges right out of the box. So if you haven't tried Playwright, give it a whirl. Let me know your thoughts as I think it's going to be a tool you must need to know as an SDET, as tester in 2024.

[00:21:23] Those are my top 12 trends for 2024. To find more, definitely check out all the links below in the comments. And while you're there once again, like and subscribe to get more automation testing awesomeness in 2024. And as always, test everything and keep the good. Cheers.