About This Episode:
Today, you get to listen in to a private Automation Guild monthly session led by Anna Royzman, Founder of A Quality Leadership Institute, Inc.
This episode is essential for decision-makers responsible for test automation solutions. It is also invaluable for stakeholders who rely on test automation results to make software release decisions or hire SDETs.
Discover the common misconceptions about test automation and discuss its challenges. Failure to address these challenges efficiently can result in significant financial losses, wasted time, and resource depletion.
Anna will share a guide to choosing the right automation tools to elevate your testing initiatives. This guide empowers teams to identify and implement test automation solutions that align with their unique requirements and deliver maximum value.
Key takeaways:
How to evaluate your product and system and pick the right choices for test automation
How to integrate the test automation into various stages of development
How to allocate resources efficiently and ensure the right fit and skills for people who work with AT
Exclusive Sponsor
Learn More About BrowserStack Automate
Are you tired of maintaining in-house automation grids or struggling to scale your tests due to limited device coverage?
BrowserStack Automate is here to transform your testing experience with a seamless, plug-and-play solution designed to help you meet your testing goals faster and more efficiently.
What is BrowserStack Automate?
Automate is a fully managed, cross-browser automation testing platform that gives you instant access to over 20,000 real devices and browsers for comprehensive test coverage. With support for leading frameworks like Selenium, Cypress, Playwright, and Puppeteer, it ensures that your tests are covered on 3,500+ device-OS-browser combinations, all while delivering minimal latency powered by 19 global data centers.
Why Choose Automate?
Quick Setup: Get testing-ready in minutes with BrowserStack SDK—no code changes required.
Comprehensive Debugging: Access a robust dashboard featuring video logs, text logs, Selenium logs, and more, ensuring you can debug efficiently and effectively.
Advanced Capabilities: Explore advanced use cases like payment workflows, 2FA, network simulations, and more for unparalleled accuracy.
Interactive Debugging: Interact with your website under test in real-time while the test runs.
Web Performance Insights: Generate Lighthouse reports and test web vitals with powerful analytics tools.
A Glimpse Into the Future
Exciting updates are coming soon to BrowserStack Automate, promising next-level reporting, debugging, and analytics capabilities. These features will offer a unified view of all test executions, deliver in-depth insights into performance, and include enhanced tools for debugging at scale.
Don't miss out—sign up for BrowserStack Automate today and take your testing processes to the next level. Stay tuned for more updates in upcoming podcasts and tech talks!
About Anna Royzman
Anna Royzman
Founder and President, A Quality Leadership Institute and Test Masters Academy
Anna Royzman is an accomplished figure in the realm of software testing and technology leadership, celebrated for her unwavering commitment to driving excellence and innovation. As the founder of A Quality Leadership Institute and Test Masters Academy, she envisions and implements transformative approaches to elevate quality practices and testing into a strategic and dynamic discipline.
With a remarkable career spanning over two decades, Anna's influence is felt across various sectors, making her a respected thought leader. Her advocacy for modern testing methodologies, including strategic testing, test automation, continuous testing, and agile practices, has reshaped the industry's landscape.
Beyond her role at A Quality Leadership Institute, Anna Royzman serves on prestigious boards of directors, where her expertise plays a pivotal role in shaping industry standards. She contributes her insights to influential decision-making processes that steer the direction of quality and leadership practices on a broader scale. Anna served on the Association for Software Testing (AST) Board of Directors, STPCon Community Advisory Board, and now serves on the Advisory Board for “The Transformative Leadership in Disruptive Times” Certificate Program at the Seton Hall University, Stillman School of Business.
Renowned as a sought-after speaker at international conferences and seminars, from USA to EU, Africa, Australia and China, Anna expounds on topics such as adoption of Generative AI techniques in testing, promotion of quality engineering practices, and fostering testing and quality cultures within organizations.
Connect with Anna Royzman
-
- Company: A Quality Leadership Institute, Inc.
- LinkedIn: www.anna-royzman
Rate and Review TestGuild
Thanks again for listening to the show. If it has helped you in any way, shape, or form, please share it using the social media buttons you see on the page. Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.
[00:00:00] In a land of testers, far and wide they journeyed. Seeking answers, seeking skills, seeking a better way. Through the hills they wandered, through treacherous terrain. But then they heard a tale, a podcast they had to obey. Oh, the Test Guild Automation Testing podcast. Guiding testers with automation awesomeness. From ancient realms to modern days, they lead the way. Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold. Oh, the Test Guild Automation Testing podcast. Guiding testers with automation awesomeness. From ancient realms to modern days, they lead the way. Oh, the Test Guild Automation Testing podcast. Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold.
[00:00:34] Joe Colantonio Hey, in today's episode, I want to share with you one of our Automation Guild monthly Community sessions with Anna Royzman, all about The Managers Guide to Selecting Tests Automation Solutions. And if you haven't already registered for our next Automation Guild online conference taking place in February, I highly recommend you do, heading over to AutomationGuild.com and registering today. With the ticket, not only do you get an awesome 5-day online event, but community lives on long after the online conference is over with an additional 9 monthly community sessions like this one. So in this session, Anna Royzman, who is an accomplished figure in software testing and is the founder of a quality Leadership institute and test Master's Academy, covers the essentials needed for decision makers responsible for test automation solutions and also shares a guide on choose the right automation tools to elevate your testing initiatives. You don't want to miss this episode? Check it out.
[00:01:32] Joe Colantonio Hey, before we get to today's show, I'm guessing you're probably someone that's tired of maintaining an in-house automation grid or is struggling to scale your test due to limited device coverage. Or maybe someone who's ready to move into the landscape of automation testing. Am I right? If so, look no further because BrowserStack has a plug and play solution that could help you achieve your test goals faster. Introducing Automate, Automate is a fully managed cross browser automation testing solution that provides you with instant access to over 20,000 devices to run your automated tests across browsers like Chrome, Firefox, Safari, and Edge. It also has an extensive support for all the leading testing frameworks like Selenium, Cypress, Playwright, and Puppeteer. Automate ensures full coverage for your tests. Choose from 3,500 device OS browser combinations to run the tests with minimal latency. Powered by 19 global data regions. You can also get testing ready in minutes or BrowserStack SDK allowing to integrate your test suite's two BrowserStack with new code changes. You also get a really cool summarized dashboard to know your test status alongside an arsenal of strong debugging capabilities that include a range of logs, including video logs, text logs, Selenium logs, and many, many more. And there's a bunch of other debugging capabilities that give automate the edge, for example, interactive debugging that lets you interact with the website under tests while your tests are still running. It also has web performance reports that helps with generating lighthouse reports and added assertions to test your web vitals. You can even test some of your more advanced user cases, like payment workflows to FA and network simulations as part of your testing suites giving unparalleled accuracy to your websites and web apps. Sounds cool, right? But don't believe me. Try it for yourself using the link down below to see if we can help you in your testing process at scale. Also, I heard from the team of BrowserStack that there is a surprise within Automate. It's new experience coming to Automate that will elevate the way you leverage automate the tests report and debug at scale. It is packed with next level reporting, debugging, and analytics capabilities. They tell me it's going to offer a unified view of all test executions, delivering comprehensive insights. It's a performance in issues along with advanced debugging tools. And I'll be talking more about this in the upcoming months on my podcast and on my YouTube series. Don't miss out on learning more. Make sure to sign up today using the link below to learn more about browser automate because the update I just talked about is just around the corner.
[00:04:20] Anna Royzman Thank you very much. It's been a pleasure and honor to talk to people at The Test Guild. I know that you are serious people and you talk about serious stuffs. My presentation today is for Software test. People who make decisions on test automaton. Just so you understand, this talk will be targeted to people who are decision makers. If you're not yet a decision maker, they learn because at some point, if you have a career aspiration to go there then definitely you have to know what you're doing when you be promoted to this role. If you're influencer, if you are stakeholder to the test automation, then you should be here as well, because I'm going to spill some secrets. Customer number one is the ultimate test automation strategy for quality manager. Imagine yourself buying a new car and you go into the shop and you say, Hey, I want a new car. And you have this all ideas and you're like, Why do I want a new car? Or I want to go to drive fast and it's going to take me places I want to visit, and it's in my garage. It's ready for me when they want it. I can put the music, I can put off the motor control, I can have heated seat in there. I can drive my family and friends with me. I don't need to call taxi when going the airport. I can now go shopping and put all bags in my car. I can use it for heavy loads. All those things that you have in your head and then you going to buy a new car. All right, so you bought this new car. It's time to go to airport. Sorry, the car is due for maintenance. Alrighr, so it's not ready for me. I ask you for heated seat. It was too expensive, so I put air conditioner instead. But I bought a new drawer and I want to bring it home. It will not fit, you should pay for delivery. I want to drive my kids to school. We don't have protective covers. Your interior is going to be destroyed. I want to take my friends camping. Your tires are not suited for this road. Wait. I wanted my car to sovle my problems. But we spent all the money on this fancy solution. Who is the customer of test automation? In the same way? Who is the customer of that instrument which is the car? The tool your assistant that you use when you need to solve your own problems and could get you where you want to be. Obviously, the customer of this car is you. In the same way, the customer of the test automation is you. It is you too, test automation is your instrument. You the manager, because you are the decision maker so the test automation solution must perform tasks that you intended it for. It must cover scenarios you prioritize. It must be functional in all the environments you identify and it must be ready when you need it.
[00:07:24] Anna Royzman We're going to be talking about Strategic Solutions. And so then I'm going to give you the strategy guide. But I want to go back to this very important slide. I want to reiterate the only reason that you in this talk is because I want to reiterate to you that you are the customer of test automation, you the manager, user's stakeholder is the customer. And as a customer, this instrument has to fit your needs. It has to be there when you need it. It has to perform tasks that you need. It has to fit in all the environments that you needed it to fit, and it must cover scenarios that you prioritize. If something is important to you, then it should be doing that. I'm going to give you this strategic solution factors first, and then we're going to talk about ownership, because sometimes I've seen a lot of test managers who are not the owners of these solutions. For some reason, they delegate or they assume that somebody else will do it for them and they got very frustrated because it doesn't perform the way they perform. Today, I assume that you in the very high rank school, I'm going to teach you how to define the strategic solutions for your test automation needs. In order to define your strategic solution for test automation, you have to understand your system architecture, your business domain, your SDLC software development lifecycle and release cadence, which is how fast you're doing the release. And you have to evaluate your resources, the skillsets of your people, the number of your people, the time, the technology, and the budgets. Very important. If you're stakeholders, then obviously you deal with the budget and you make decisions on the budget as well. So first you have to have a good understanding of the system architecture and as you can see, there is a lot of different components that you as a trader, if you trading stocks, you probably know if you're not trading stocks, you're not aware how complex the system is. Putting test automation in it and just do it from like users to interface systems. It's impossible and it's not even needed. You have to break down into the chunks and where you can put strategically, where you can put the test of automation. Some of that will be integration testing definitely. But you probably don't want to do end-to-end system testing in automation unless you necessarily need to, but you can do some integration components into test automation. You definitely don't want to put everything all your testing on UI. Look how much happening on the backend. It's like an iceberg, you don't really want to test ever since through UI automation because it's useless. It's crazy. But you probably want to talk, to test more on the backend on the order process block. You want to put some tests of automation on the data in the database? You probably want to put some automation into the messaging system, messaging between your backend and the thirty-party backends. And this is how you approach strategically the test automation solution. You totally never do it for UI. There are some systems where you should be doing through UI because that's the only thing you have. But when you're in a very complex scenario, then break it down. But you have to have a good understanding of the system architecture and this shit has to be in front of you. All this diagram has to be in front of you when you're making decisions for test automation. You have to be confident of the business priorities I put here 4 different businesses. One is on travel business and other social media business. The third one is medical business and the fourth is the entertainment business. What would be the priorities for each? In travel, you will have reliability is like a big deal for you. When the flight don't go as planned then they have to be able to reschedule tickets. There is so much in travel and you probably know if you're frequent traveler, how much social media is totally something different. Analytics statistics is something that is very big deal in the social media. And finding bugs in that area is actually also a big deal because social media is used for business. Digital health aid hospitals rely on it. All your health providers rely on it. The big deal for them is actually people's lives. There is additional layer of compliance on and everything that you test for else and there is an additional regulation. So you don't just provide your testing, but you have to provide the documentation of what was tested and there are certain rules of how you do it. It's a life-threatening situation if you make mistakes and find bugs in it. Definitely different focus. Business priority would be on saving lives. Entertainment streaming, you want to be uninterrupted entertainment is a big deal for uninterrupted experience, and it should give you the choices if you have subscription should be working well and it should be working for you in different countries. Definitely each of these businesses have different priorities and you want to test when you do your automation, you have to put the focus on these business priorities because this will drive your test automation solutions as well. And it also will drive your strategy in test automation strategy, because it will be different for all of them. Then you have to fit your test of automation solution into your software development lifecycle or you have to change your software development lifecycle, which is okay if this is part of the if you are in charge of it or if you are influencer to change your solutions, then go for it and change it so the test automation is not an afterthought, which is often the case. But test automation is actually involved from the early stages and you can enable the teams to use the test the test automation when it's actually needed, not as an afterthought. What I have seen happening is so yeah, we have to develop those of the main those test cases, but we cannot do it in this sprint because we're doing something else. We're going to do the next sprint for this, this case automation for. If it's not for the team who is delivering then what are you doing then? It's really a big deal. And putting the test automation takes time to do its development. Like anything else, it has to be part of your SDLC. For example, if you see over here, I put 3 different I just Google for diagrams. Do you see anywhere with test automation? No, it doesn't. This is testing. What does it mean testing? You cannot do like test automation after your implementation because you have to do deployment and you don't have time for test automation. You have to be plan it carefully because usually what happens is the test automation is either as becoming the technical depth or it's completely not correlated with your software development lifecycle, which is a bit. It means that it's not used when it needs to be used. Be very responsive to your SDLC and where in this SDLC, you want to put your test automation. Very big deal is managing your resources. Resource management is a big deal for any strategic decision. You have to totally evaluate the skill sets of your people. You have to know what they do and what they are capable of. And you have to know the number of people on your team or who can be doing the test automation. If it's not people on your team. So who is actually dedicated to test automation? How much time do they have, do you have on the project? You need to understand the technology because different technology requires different skill sets and you can leverage some of the technology if you're already as an example. So development is already building your front end with certain language, right? Say is Java, for example, it won't make sense to create the test automation solution, which is more or less in Java as well. Doesn't have to be bad what it gives you advantages of it is that the same developers that are coding they could help you with solving some of your issues with test automation because you have like experts next door. Just consider it, it doesn't have to be, but consider it. Also the budget, you need to understand how much money you're spending on test automation. And big deal of test automation is maintenance, because if you create the test and it's no longer valid with the for example, you have a version number two of your software and you have to rework your workflows. Then all of a sudden, test automation is no longer valid. You need to change the test automation script, and it's a reality of a new software development. Don't be surprised. It's not the like, disruption, it's the reality. The test automation solutions and not 100% reusable. They have to be maintain for the next release. And this work has to be taken into consideration as part of the story. It's not a technical debt, it's real work that you have to adjust your test scripts to the new features. It has to be done. Otherwise, you can throw away your test automation. Work allocation. Who is working on test automation when there are some times when you need some resources to put on something else. For example, you can have test automation team, but they need input from your testers and your testers are very busy doing something else. For example, that was hard to release. You need to release now. So then somebody is waiting on somebody. And you have to understand how you can reallocate work if situation like this. So you have to prepare for anything like, any CrowdStrike situation, you have to be prepared for. Project milestones, when you have to deliver at what time you're already committed. And if you are a manager, you know that these are big deals for business and for your management. The milestone, when you put the date on something, you have to hit this date. If you don't hit this date, you don't have a bonus. You have something that on your review, you didn't fit the goal. Milestones are always like those, a commitment that you gave to your management. Just be aware it you have to deliver there as much as you can, you have to. And if you don't deliver to this milestone, then you have to explain why you don't and you better be persuaded from that. But always take into consideration any project that you committed to, all the milestones that you committed to. Growth opportunity to some of you people will be. You have to put into consideration right? When you're thinking about test automation strategy, you have to think about people because people are working with it. Unless you outsource, then you don't have to worry about growth opportunities. People in your team, they would want promotion, they would want to change directions. They would want to acquire new skills for promotion and do use the test automation as part of your growth opportunities strategies as well. When you do planning for people, put some of test automation in there get to the next level kind of situation milestones because you don't want your test of automation to be in the backburner, like assuming that it's always going to be working. You always have the people. You have to use the test automation as another step in people's promotion, for example, doesn't necessarily need to be but to say, Yeah, I want to master this, then make it us a goal for people and use it as a growth opportunity. Just another thing that you want to consider, if you're a manager, you have to have a backup and in your growth opportunities. If people want to get to the level of management or leadership, teach them what I'm teaching you today. Teach them how to create strategies for test automation because it's just a huge deal for everybody in QA or testing. You really have to manage it and you have a long term plans for all of that. Organization goals. So do know where your organization is going because if, for example, the organization wants to retire, some solutions definitely research that. So you prepare because your test automation solution may not work in a new market and then be aware of what the organization planning to. If you are in management, the leadership, you probably will be exposed to some of it. But be proactive about this because you have your internal organization goals aligned with your long term decisions. Okay, I'm going to be giving you for 4 lessons of mastering automation strategy. We're going to go over all those four, but those four last lists define, de-clutter, decouple it all. And we start with. Lesson number one. Define. I'm just going to describe it in this exercise. But you have to do it by yourself. If you already have test automation, then yes, you do. I assume most of you do. Just do this exercise just for yourself. Redefine it. Define the purpose of test automation. Just write it down. What is the purpose of test automation and you'll be surprised because there are some assumptions that are like, you will have to reveal them. What are the purpose of test automation for your. Don't just say that it has to say anything. Then you're going to compare to what you say to what it actually does. The purpose of test automation. What are the priorities of test automation? What are the most major points of their test automation has to give you results, defined measurements. How would you know that test automation is actually does what is intended to do? How do you measure the success? Like by which measurements you would know that by which numbers or whatever it is, you know that it does what you intended it to do. And obviously some KPI, it doesn't have to be. But if you're on it, then use some of those too. This is the like very first step and you have to identify the purpose and priorities, how you're going to measure your success? Then declutter, where to use automation in software development lifecycle. There are different places in the lifecycle where test automation may be useful and where it has to be more or less perform. You can start test automation, at which point you can start it in this SDLC and at which point it has to be workable. Solution has to be workable. It doesn't mean nothing if it's already done. If it's not working, then you have no solution.
[00:23:17] In SDLC, there is different obvious list, in lifecycle, there are different milestones as well where the you want your test of automation to be started. Where do you want it to be performing? At the story level of the set of criteria level? At the coding level, at the story testing, the sprint testing, regression or at release candidate testing. I can tell you now that all of these are different activities and test automation is not suitable for all of that. Sometimes with the story testing, you mean the need to put a lot of automation, but you definitely want to put more automation in regression and if your UI is changing but which usually is changing, but your backend is not changing or your database tables are not changing, then put more automation on the backend. That means headless driver or put your automation into the backend itself. And sometimes you want to put the automation on the database itself. You can do it, it's called a golden copy comparison. So you populate the golden copy and you have a like snapshot of the database, how it's supposed to be, and then you run some tests and then you take a snapshot of your database after you run and some tests and you compare to the golden copy. That's another way of using automation for something which is not UI. Lesson number three, Decouple. You have to use the best tool for testing various places, layers, and stages. And what I just mentioned to you about the golden copy is something that you want to use when a lot of people use an API testing into the API, but you don't know you can test UI through your API, through UI. It's not a good idea, but I see some people doing it too. It would be great to decouple all your components and make specific test, use automation specific automation, not one tool, it's usually not one tool doesn't fit all, but you can go away. It was like a few tools and a few methods. Make sure that you are testing the components directly. It's always useful. It requires less maintenance and in the long run, it will save money as well. You want to look at the business workflow versus end-to-end testing versus operations workflow versus admin functions, and your test should be separate for the business workflow because this is high priority to the admin functions, which is completely separate tests, right? But if you decouple them then, you could use different testing test automation solution for each of them when you need it. And for example, admin functions are a big deal, but you can test it manually if something is not working. But business workflow is something that you really need those results really fast because they are high priority and you want to reduce the business workflow automation in as your sanity check. That is really a big deal business workflow admin functions is nice to have. Then you have to differentiate between acceptance testing, functional testing, load testing, failover testing. All of these are valid testing, but test automation should not be testing everything together, Decouple. And also decouple in the phases, active development or release candidate regression testing, because active development is a great thing. But the thing is a lot of things are changing because active development is discovering the solutions. Regression testing, making sure that your reputation stay intact. Whatever was working before should be working now. That's the goal of regression testing. That's your reputation. Active development is your excitement. We got a new feature. Maybe nobody wants this new feature, so we're going to change it in the next sprint. Regression testing, we already have this feature. Our customer knows about this feature. It better be working when we put something else into our system. Defined priority. What's more important for your test automation because you may not be able to do everything, but do define your priority, decouple as much as possible. And lesson number 4, big lesson, you have to own it. Are you a test automation product owner? Are you the one who is driving the test automation product? What does test automation do? Test automation's product is any code. What does it do? It has to be doing something for you. Do you have decision power over development delivery persona? If you don't, then make friends and you have to have an influence to the decision being made. Otherwise, this test automation solution will not serve you any good. Do you exercise a strategic influence, which as mentioned, sometimes if you're not in charge of something, then get trained on how to influence people and how to negotiate for what you need? Are the team goals aligned? Sometimes the team goals, the test automation team is separate, it reports to somebody else. But you are the customer but you have no exposure to like what this team's goals are and I've been in this situation, everything that I'm telling you is from experience because when I sometimes came to the automation team and say, I need this feature automated, they say no, but we automate in this. Why? Because our manager said so. So that's how it works. And then you have to persuade the manager, but you have to have a way of prioritizing the test automation work. And if you don't, then start working in this direction because it's really you have to own some of the solutions and be the decision maker. And a lot of this test automation solution. 6 steps in picking test automation solution. As I mentioned before, when we just started, when I said the strategy, first of all, set your goals. You have to understand what you use in test automation for. In general, like why do you use this tool? What do you need? And be honest about that because go one step a few steps beyond the obvious. Just really set goals of what this test automation. If your goal is to please the higher up manager, then that could be your goal. But I hope it's not because they would be pushing you in, put in some money into some solutions that don't work for you. So set your own goals. Why do you need test automation and where? Do your research. Spend more time on research on different tools, solutions, and how it fits into your own workflows. So don't just take any work for granted. I have seen a lot of salesman who say, Yeah, it's going to do everything for you. No, it's not true. They need you to use cases, put them through that, whatever the damn line is they doing for you. And let's see how it works for you. Okay. Do your research. There is a lot of tools on the market. Huge amount of tools on the market. Evaluate. Do a comparison of how much of this tool can cover your needs, how much you need additional internal solutions. Maybe you end up with just do an internal solution by hiring somebody in the team and do homegrown everything altogether. Evaluate build versus buy and in different areas as well. I mentioned some of the domains where there are some engines that you would you don't create yourself like in trading. Then you will have to buy that the services too. Evaluate that as well. Any sort of party integration they have to evaluate how much of test automation will be covered in that. Estimate. Very big deal. So implementation of everything in test automation takes time as a new coding. It's a coding activities. It takes time. You have to estimate how much time it will take until it becomes fully functional. In my company, there was one time a project where we decided to go full day automation. The whole company was on board was a day shut down the releases for six months. And all developers were doing only test automation. Now this is the honest this is my experience because I was the leading the QA effort of it. I was writing specs. This was the very first time in my life I was writing specs this really hard. But I was the owner of this test automation solution and all the developers where they indicated to just doing that. You have to estimate what your implementations and going forward. What are the efforts of test automation will take, how much effort they take, how much money they will take, if there is a support of third party and how much time it will take to continuously do test successful test automation going forward. Then you have to select. Select it, look at what resources you have and you need somebody to be maintaining. It's better to have in-house. As I mentioned about the languages, sometimes you want to pick up the language that your team knows or some people or develop personal or you don't need to like retrain everybody on like new language that nobody even knows about. It's a decision to be made. We, for example, in my team were been using Python because it's more kind of tester friendly than Java. But when we had our dedicated automation specialist who is a Java developer, then we said, okay, whatever, just write your code driver in Java because you can. So it depends of like you have to evaluate your resources and going forward. Don't dump on unsuspecting people something like completely new. And the very big step later is you have to hire, train, or outsource. You have to know who is going to be there to pick up the build. Whether you need to hire people with new skills, where you have to train your own and delegate or hire in the team or promote somebody into that role. Or you have to outsource. And there is a lot of outsourcing solutions. So definitely hire, train, outsource and do calculate because this is your budget decisions and hiring decisions as well.
[00:33:58] Anna Royzman Okay. Next, I want to briefly chat about the AI in tests automation because the 2024 September, all major tools already are equipped with test automation A.I. By now, all the major vendors I know, they already put in the release press releases that part of their solution is A.I. So it's not something that you need to research. It's already there. All the tools are equipped with it and if they don't, then they are not competitors on the market. If you want to hear a secret from me, then because they all going to push it on you and your management is going to push it and you. Use AI solutions to increase productivity. But don't take any word for it. Bring your use case, bring your use case to the evaluation and use AI solutions to increase productivity. Because most of the things that AI claims to do. Generation, it can generate a lot of garbage for you. Definitely put it in your use case in it and understand how it's going to save you time or give you more bandwidth or whatever it is. But just don't take it for the sake of it, okay? Only if it increases. It's really to increase your productivity and it can. I'm going to give you two examples. We don't have much time. Very briefly, two examples of how to use the test automation, AI to increase productivity. Because trust me, if you're a decision maker in 2024 on test of automation, you will have to deal with AI, if you don't then I don't know where you work. All right, this is how to deal with AI, look at your SDLC and see where are the productivity gaps and talk to your team and you probably know already what it's like in here. You have to know that like different steps, like what's done specs, the acceptance criteria, BDD coding testing done. That's simple. It doesn't mean that it's your SDLC, but there is obviously some way out of there is like productivity gaps and we're going to look at two places where those productivity gaps. One productivity gap is in the prevention, which is the collection of requirements, incomplete the requirements, wrong requirements, poor communication, changing requirements. Unclear requirements are usually the productivity gaps. And then you have to spend and don't take anything for granted. Just evaluate where you spend most time. This is one of the huge reasons where the productivity gaps are exist. Sometimes this technical jargon, you don't understand what this guy said or they don't ask the users and all of a sudden they create requirements that are not user accepted. So then they need to change it again. There is AI solution which validates the requirements, which is shift left. It definitely increases the productivity. And this is the big deal. Let's look at defect detection challenges. What could be possibly defect detection challenges and your that's automation. This is for you test automation decision maker. Defect detection challenges are could be in communication, between the spec and the test. There could be a communication gap. People may have lack of skills on resources on doing. Defect detection is the function of test of automation, that's why it's here. And when we talk about lack of skills of resources, we could talk about the lack of skill and resources of test automation, SDETs, test automation specialists. How to deal with changes. Dealing with changes is a big deal. It takes time. Always there is time constraint. How much can you do in a certain amount of time. Incomplete test coverage is a big deal. So you can put your test automation in the wrong places. Compatibility issue is a big deal because your test of automation environments may be not the replica of your environment. And it could test, could pass in one environment and like with synthetic data and thought it would fail in the real life scenario. And then the lack of clear pass/fail criteria, which is a huge deal for test of automation because sometimes you don't know if the test is failing for the right reason.
[00:38:35] Anna Royzman Okay. In summary, finding AI solution, the same thing that you do with test automation but a little bit different approach. Don't just take A.I. because it's says A.I.. Okay. First of all, identify the gaps in productivity and then do your research what AI can offer. And if GenAI, it can offer generation, but it cannot offer a lot of validation unless you put it in there. And also it does hallucinate. It does. It's a feature of it. It's not the bug. You have to be aware of drawbacks. It's not about drawbacks of AI, but be honest about it. It's not intelligent. It doesn't think it doesn't have logic in it and it doesn't have mind. No, it's not an intelligence. Be honest. Do your research about what A.I. can offer and where it cannot replace, which it cannot. And then you have to supplement. You have to introduce A.I. for productivity, but not for replacement. Somebody needs to have a brain through AI, AI is collaboration tool, and it can do some tasks fast, but it's not going to sync for you. Same thing with copilots, by the way. It can generate the code for you, but you have to test your code yourself. You have to watch what it's doing for you. All right. So, no, it's not the replacement. It never is. If your management wants to replace people with AI, resist, it's not going to replace it. Never. You have to understand A.I limitations. Do your research. Understand where GenAI is good and where it's not. A lot of things of like garbage in, garbage out. You have to understand that A.I. works with data and it's trained on the data that it's been fed. You have to really understand where the data sources are, where it's coming from, because wrong datasets will give you wrong results. It's a simple of that. It doesn't have a brain is not sync or it's not going to do any selection. You have to train everything to this machine. And then once you did all your homework, you have to find the best tool in the market that fits your needs.
[00:40:56] Thank you, Anna for your automation awesomeness. The links of everything we value we covered in this episode. Head in over to testguild.com/a528. And if the show has helped you in any way, why not rate it and review it in iTunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe, my mission is to help you succeed with creating end-to-end, full-stack automation awesomeness. As always, test everything and keep the good. Cheers.
[00:41:18] Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation, testing, and DevOps. If you haven't yet, join our vibrant community at TestGuild.com where you become part of our elite circle driving innovation, software testing, and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate. Visit TestGuild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.
[00:42:02] Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold.
Sign up to receive email updates
Enter your name and email address below and I'll send you periodic updates about the podcast.