Flaky Automated Test Drama, DevOps Bootcamp, and more TGNS120

[00:00:00] Joe Colantonio What's all the drama around the cause of flaky automated tests? Have you seen the latest free DevOps bootcamp course? And what webinars should you attend to learn more about the future of functional testing with the AI. Find out in this episode of The Test Guild News Show for the week of May 12th. So grab your favorite cup of coffee or tea and let's do this.

[00:00:18] Joe Colantonio Before we get into it, if you could do me a quick favor and answer a small survey to find out what guild goodies you want and find value from, it really well help me know what free stuff really will give you value. And all you need to do is go to Testguild.me/free stuff.

[00:00:34] Joe Colantonio Speaking of free stuff, I have a free webinar coming your way next week that I think you definitely should check out. If you want to discover the future of functional testing with AI. This is our Test Guild webinar of the week spotlight. Register now to join automation expert and practitioner Chris Trimper and OpenText Chief Technologist Don Jackson as they dive into UFT's automation tooling from the perspective of various personas and organizations requiring functional test automation, showing what shift everywhere means. And they're also going to touch upon the different areas and capabilities of the tooling. And we're also going to dive into some of the most recent advances or advantages of AI computer-vision, which makes a more straightforward and lower maintenance scripting. Even if you don't use UFT, this webinar will showcase real-world applications of AI and testing, and these insights could be directly applied to your own testing efforts, making it a valuable opportunity for you. Register now in the link down below and hope to see you there.

[00:01:33] Joe Colantonio What's the drama all around flaky test? Well, let's find out.

[00:01:38] Joe Colantonio In this recent blog post by David Burns, he challenges the assertion that test framework choice, such as Cypress or Selenium is the primary cause of flaky tests. This post highlights a common misconception about the source of flakiness in automated testing, and provides insights into the true factors contributing to this issue. What's the drama all about? Well, this post is actually a response to another post by Gleb. And in Gleb's post, he claims that Cypress architecture choices, which avoid certain transport layers, make it less prone to flakiness than other frameworks like Selenium or Playwright. And the post by Gleb was actually posted by Filip recently, but the post itself is a little bit older. But regardless, and when it was posted, David responded to it recently, and he argues that flakiness is less about the choice of a framework and more about the asynchronous nature of web applications. Now tests interact with them. He points out the root cause of flakiness lies in how tests intercept UI and browser code execution, emphasizing the mismatch between synchronous test steps and asynchronous browser operations. And with all the different browser interpretations, makes it further hard and exacerbates flakiness with frameworks struggling to support multiple browsers. So which view is right? I recommend reading both posts and you decide. And then let me know your thoughts now in the comments down below.

[00:03:01] Joe Colantonio If you're into Playwright in dashboards. This next resource is for you. And this is by Playwright Ambassador Butch, who recently compiled a really thoughtful review of the current Playwright dashboarding, offering insights on historical test automation, run data and identifying flaky tests. Let's check it out. This review by Butch highlights the Dashboard's user-friendly interface and the wealth of data it provides to testers. The current Playwright dashboard allows testers to access historical test automation test runs, enabling them to track the progress of the test over time. And this feature is really valuable for identifying trends and patterns and test results, helping testers make informed decisions about their testing strategy. And speaking all about flaky tests, this dashboard helps testers identify flaky tests, which are tests that produce inconsistent results, so flaky tests could be a major headache for testers, as they can lead to false positives or negatives in test results. But by using the dashboards built in tools for analyzing test data, testers can quickly pinpoint flaky tests and take corrective actions to improve the reliability of the test suite.

[00:04:10] Joe Colantonio This next article was recommended to me by Jason Arbon. And it's all about OSWorld introduces benchmarking for multimodal agents in real computer environments. What does that all mean? Let's find out. And this research from the University of Hong Kong, in collaboration with other institutes, have developed OSWorld, a groundbreaking real computer environment for multimodal agents. And this environment supports task execution, execution-based evaluation and interactive learning across various operating systems like Linux, Windows, and MacOS. OSWorld aims to provide a unified platform for evaluating open and computer task involving arbitrary applications with a benchmark for reliable evaluation. What does this have to do with testing? I think testers can benefit from this because it can help you create a comprehensive environment to evaluate agents across different operating systems, enhancing their testing capabilities and insights. And the OSWorld Benchmark offers testers a standardized platform to assess agents performance on real computer tasks, providing valuable insights for developing more effective multi-model agents. And testers also can actually leverage these environments, enhance their testing methodologies and strategies to help improve the quality and efficiencies of the testing processes. And once again, Jason pointed out. So if Jason points something out to me, it means, it's probably important to know and you can find more about it as well. Let me know your thoughts.

[00:05:34] Joe Colantonio I often get asked, how can we learn more about DevOps as it applies to continuous testing and automation? Well, I created an awesome resource just for that. I'm now offering a free DevOps Continuous Test Automation Bootcamp, aiming to enhance testing capabilities for rapid software delivery and helping you to optimize your DevOps practices. This course, in conjunction from the awesome folks at key sites, covers a range of topics including the role of test automation in accelerating release cycles, implementing test automation in CI/CD pipelines in concepts of continuous testing in DevOps. And we have an awesome lineup of instructors that walk you step by step of what you need to know to be successful with DevOps, with continuous test automation. If you haven't already, I highly recommend you register now to learn more about continuous test automation and why it's crucial for supporting agile development, CI/CD practices, and DevOps, and you can find a link for it down below.

[00:06:26] Joe Colantonio Also, did you know Playwright just released a new version? I didn't until I saw this latest article by Michael posting to it, which is all about Playwright 1.44. So in this latest update to Playwright, users can expect a range of new features and improvements aimed to enhancing your testing experience. The update includes an introduction of new APIs, improving its locator handler, and enhancements to miscellaneous options. One improvement I really dig is its new API that allows for improved accessibility testing. Testers can now utilize APIs such as expect locator to have accessibility name to show that their web applications are accessible to all users, and this new functionality will significantly benefit testers looking to enhance the accessibility of their products. Additionally, the locator handle on Playwright has received upgrades in version 1.4 as well. One notable improvement is that Playwright now waits until the overlay triggered by the handler is no longer visible, providing a more seamless testing experience. And it also has improvements to the reporter API and also in terms of command line updates. The update introduces the last failed CLI option, and this new option enables users to run only failed tests from the previous run saving time and resources while improving their testing workflow. Definitely something that you should check out if you're using Playwright and download it and let me know your thoughts.

[00:07:50] Joe Colantonio So a few days ago there was a big announcement by Datadog, and it is they did just announce its launch of IT event management enhance its AIOps capabilities, and this new feature intelligently consolidates, correlates, and enriches all alert events and signals from Datadog and third-party observability tools into a single view, reducing alert fatigue for teams. And by streamlining incident management, teams can now focus on issue resolutions more effectively. So it's definitely a good read if you want to know how you could benefit from improved incident management processes that can lead to quicker issue resolutions and reduce alert fatigue, and in my experience, having the enhanced ability to triage quickly using intelligent correlation of events and have a complete picture of the underlining issue can really empower you to respond and remediate issues quickly and effectively, and you can find out more about it in the link down below.

[00:08:45] Joe Colantonio And in other AI development news, Oracle has made an update that incorporates AI as well. And this is how Oracle has unveiled Oracle Code Assistant, which is an AI powered tool designed to help developers and testers increase their speed and ensure code consistency. In this innovative solution optimized for Java SQL and Oracle Cloud Infrastructure based applications offers context specific suggestions tailored to an organization's coding practices and standards, and this tool is deployable as a plugin for JetBrains, IntelliJ, IDE or Microsoft Visual Studio Code.

[00:09:20] Joe Colantonio And in security news, I just found out that GuardSquare has enhanced its free mobile app security testing product with an enterprise ready version. I bring this up because I didn't even know there was a free version, so I like sharing new tools with you and hopefully you'll check it out. But this upgrade allows organizations to benefit from developer tester friendly, mobile specific security testing that can scale effectively.

[00:09:43] So that's it for this episode of the Test Guild News show, I'm Joe, my mission is to help you succeed in creating end-to-end full stack pipeline automation awesomeness. As always, test everything and keep the good. Cheers.