No Scripts, No Frameworks, No Maintenance? with Don Jackson

By Test Guild
  • Share:
Join the Guild for FREE
Don Jackson TestGuild Automation Feature

About This Episode:

In this episode of the TestGuild Automation Podcast, I'm joined by Don Jackson, Technical Evangelist at Perforce, to explore a bold new frontier in automation testing: true agentic AI. We unpack the groundbreaking tech behind Perfecto's latest release and what it means to test smarter, faster, and with zero hassle.

Don shares how this new form of automation, powered by natural language, computer vision, and LLMs, lets you test like a human but with the speed of AI. No scripts. No frameworks. No brittle locators. Just plain-language instructions that drive real user-experience testing at runtime.

You'll learn:

What “agentic AI” really means and how it goes beyond traditional test automation
How to run intelligent UI tests across devices without writing a single script
Why this approach could finally bring automation to business testers and non-coders
Real-world use cases, from validating stock charts and UI branding to exploratory testing at scale
What to watch out for: prompting quality, performance tradeoffs, and trust

Whether you're a skeptical scripter or just tired of flaky UI tests, this conversation will challenge how you think about automation—and show you what's now possible.

Want to see it in action? Join the upcoming live demo webinar with Don and Perfecto's head of product: Perfecto AI in Action – Register Here https://testguild.me/donperrfecto

About Don Jackson

Don Jackson

With over 25 years’ experience in the Application Delivery Management (ADM) space as a consultant, solution owner, developer, tester, architect, and vendor, Mr. Jackson is recognized by colleagues, peers, and ADM professionals for his informative and innovative demonstrations, seminars, webinars, videos, and podcasts. His expertise and experiences in ADM around SAP solutions and the broader enterprise IT ecosystem, helps him convey to the audience complex topics in a manner that is relatable and easy to understand.

Connect with Don Jackson

Rate and Review TestGuild

Thanks again for listening to the show. If it has helped you in any way, shape, or form, please share it using the social media buttons you see on the page. Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.

[00:00:06] In a land of testers, far and wide they journeyed. Seeking answers, seeking skills, seeking a better way. Through the hills they wandered, through treacherous terrain. But then they heard a tale, a podcast they had to obey. Oh, the Test Guild Automation Testing podcast. Guiding testers with automation awesomeness. From ancient realms to modern days, they lead the way. Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold Oh, the Test Guild Automation Testing podcast. Guiding testers with automation awesomeness. From ancient realms to modern days, they lead the way. Oh, the Test Guild Automation Testing podcast. Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold.

[00:00:35] Hey, welcome back to the Test Guild Automation Podcast. Today, I'm thrilled to have Don Jackson join us to discuss an exciting new development with the Agentic AI. Don breaks down how this new approach differs from traditional scripting, enabling capabilities like exploratory testing, improved UI, automation for dynamic applications like Salesforce, and even complex validations, such as check-in if an image matches its text description or ensuring accessibility compliance with WCAG standards. Who also discuss how the solution can perform actions like installing an app from a store without explicit scripting. And while I'm still skeptical, as you probably are, about this approach, which he calls no script, no framework, no maintenance, he does address common skepticisms about UI automation, reliability, and how the agentic AI maintains trust through screenshots and video recording. And if you're skeptical like me and you want to see this in action, but have questions, make sure to join us with Don on a webinar that we're running at the end of this month at The Test Guild. And you can register for it now using the link down below. We'll cover all this and more. You don't want to miss it. Check it out.

[00:01:44] Joe Colantonio Hey Don, welcome back to The Guild.

[00:01:48] Don Jackson Well, thanks for having me, Joe. It's always good to talk to you. I love watching your stuff. In fact, I was just watching your note for this week. Hopefully everybody subscribes and is reading and watching your weekly. What do you call it? A weekly message. I can't remember what you call them.

[00:02:04] Joe Colantonio Oh, it's weekly News Show.

[00:02:06] Don Jackson Weekly news show, yeah.

[00:02:08] Joe Colantonio Awesome. Yeah, thank you for that. Really excited about also I saw some recent announcements on LinkedIn. You joined a new company. Maybe you tell us a little bit more about that. A lot of people look for new gigs and it seems like you found the perfect spot. Like how did you do that?

[00:02:21] Don Jackson Yeah. Well, how did I do that? I'll answer the first part first and the second part second. I've joined Perforce working specifically on Perfecto and BlazeMeter. But I also, obviously, I work for the company. I look at stuff, a pan portfolio as well, not just those two products. For example, I'm talking with a couple of people that we all know. And until something happens, I can't say who, but on Perforce ALM, Perfecto, BlazeMeter, both for API and for load testing and for service virtualization and in the BlazeMeter. And then the second part of your question was how, right? A lot of people looking for jobs, honesty, I didn't specify in LinkedIn that I was looking, that I was open to work or anything like that. I just reached out to key people in my network that I really trusted. I trusted their opinions and used my network that way to find out, where somebody was, I specifically talked with Phil Raymond, who's at Perforce, and I kind of asked him point blank, how do you like it there. And obviously it was a positive response that he gave. I don't want to say specifically what he said, but went through the interview process, had a bunch of wonderful interviews, really clicked with them. And the culture was great, and I was super excited about the technology that is being released actually on July 15th, depending on when you're watching this, that might be in the future or last week or whatever. But I was really excited about it because it's actually a concept that I had been talking about 18 months ago that I was calling goal-oriented testing. A goal-orientated automation where you give an objective to the AI and let it go figure it out, just like a manual tester would. I don't tell a manual test. I just tell them, hey, you got to log into the site. I don't tell them how do you get to the login page. You have to click on the hamburger menu or the profile icon or whatever. I just told them, go log in. So same concept that we're releasing on the 15th.

[00:04:21] Joe Colantonio All right, I think I saw this demo. You gave me a demo like maybe a year and a half ago. I was like, wow, this is pretty cool. But you just joined this company like a month ago. Did they see that demo and they actually executed on it? Like, was it just like serendipity? Like, wow. Like synchronicity.

[00:04:37] Don Jackson I think it's actually happens to be coincidental because when I was talking with our head of products, he didn't mention it. And I said, this was a concept and he goes, oh, really. Right. I think just two people happen to have a similar idea. And in fact, I know tree king has talked about this concept as well. We're not the only ones, but, you know, seeing what you can do with AI and you start going, okay, how do we take it to that next level? What can we do with it? I'm just super excited because it was something that I was talking with the products team at my last company and that didn't go anywhere at the time. They had other priorities. That's the right way to say it. They had other priorities.

[00:05:17] Joe Colantonio Great. So for the folks that didn't see the demo, maybe break it down a little bit more like to like, no one's heard of it before. What are you talking about? What is this announcement about? It's an Agentic AI or is it like some flavor of agentic AI? What's it all about?

[00:05:32] Don Jackson So it's true, agentic AI. So when I say true, agentic AI, there's a lot of people who say Agentic AI, and then you dig into it and it's like, this is just a request and response. That doesn't have any agency. What do I mean by agency? Great question. Thanks for asking, Joe. Thanks for playing along today.

[00:05:49] Joe Colantonio Well, I was going to say, what's agentic AI first? But does that have to do with agency?

[00:05:54] Don Jackson Agentic AI is artificial intelligence that has agency. What do I mean by agency is it has the ability, so whether you're talking about testing or not testing. It has ability to make decisions and take actions with minimal human intervention. That's what agentic AI is. Well, apply that to testing, what does that mean? Think about this. If I were to script something, if I was to automate, let's say I'm working to test airline mobile app. What do I have to do? Well, the first test case is booking a flight. Let's be honest, right, we all understand the concept of book a flight. How would I do that? Well, I would first log in and I would tell it how to navigate to that page. I'd say, click the login button, type, click, the login ID, type in the value, type in the value of the password, click the log in button. And I tell it all those steps to do. And then I'd start booking the flight. Well, how do I book a flight? Well, I gotta find the from and the to location and I need to type in the airport name and the two airport name and then I click find flights and I tell it all those steps. In an agentic AI, what you do is you give it a step that says book a flight from San Francisco to New York in business class, prefer an aisle seat, second preference is a window seat. If there are no flights available, that have one of those seats, I don't wanna sit in the middle. I don't wanna sit the middle. Come back with an error message. And all of the steps that it takes to do that of finding the objects on the screen, typing the values, think about clicking the seat map to find is it close to the window or is it closer to the aisle? How difficult that would be to script. It handles all that for you. And so by going agentic, you don't actually have a script. During at runtime, it takes that natural language prompt, looks at what's going on in your application and then makes a decision about what it needs to do to achieve that objective of booking a flight. All of those steps that you would have had to script out, that's probably like at least 50 steps. That you would to script out and deal with all those objects and all those locators, all that goes away. Your actual artifact that you track to and trace to is your test stuff. There's no script. There's no framework needed. I don't have to know Selenium or Playwright or Cypress or Espresso or Appium or whatever. I don't have to all that. What I know is the business process. And because it's making decisions at runtime based on what the application looks like and how to get through it and understand semantics and context, when cause we all know it's a when not if, the application changes, it just, it doesn't break. There's no self-healing involved. It just doesn't break because it's making that decision based on how the application's behaving, just like as a manual tester. If we change our application, my manual tester doesn't brake, he just or she makes, just goes the different path. And you could start doing things like find, you can start doing exploratory testing with the Agentic AI. Find different ways to get to the shopping cart. There's like 10 different paths. We actually talked with one customer that was a beta customer and they're like, we did some of this. And we said, Hey, here's the 12 paths that you can get to this part of the application. And they said, we only knew about 9. The other thing is, because it's working at the user experience level, is things that were just untestable before, that you just couldn't automate before, are now automatable. We have a financial services company with another one of our beta customers. I don't know if we have permission to use their name, so I'm not gonna use their names, but think about like a stock trading app. And you get the graph, and then you can put overlays on that graph of, okay, so if it's red going down, green going up. And now you can actually write about assertion that says, if the point in the graph before was higher than the current point, you should see red. You should see green if it's not. And oh, by the way, down in the chart where you've got the numbers, make sure the chart matches the numbers that are down in table. Things that the way that you automated it before, let's be honest. You did static data. In order to test that kind of stuff, you do static data. Same thing like think about like a big e-commerce company that we are showing this to, they have a picture of their products and then a description, but they do marketing campaigns where the description changes. Sale on laptops, so every single description says now sale on laptops. Well, they couldn't test those marketing campaigns because it changed the static data that was in there. They'd have to dynamically do that. And oh, by the way, I can write a validation and assertion that says, Does the text match the picture? If the text said HP laptop with a 17 inch screen and a 10 key, it could look at the picture and see, does it have the HP logo on it? Does it have a 10-key and the keyboard? That kind of stuff that you just, you couldn't automate that before. You just, I know what I used to do. I used do the cheap way out because could I automate it? Maybe really hard, might take me a week to figure out how to do all this. I just grab a snapshot and then put a warning message in my log that says, hey, whoever's reviewing this, just look at the picture and make sure it's right. When all that can be automated. And super excited if you can't tell.

[00:11:33] Joe Colantonio I'm excited, but there are skeptics. You've been in automation for a while. I've been an automation for awhile. The browser has been always highly unreliable even I know open tech, open AI, all of them are coming up with their own browser automation and it's not good. It's not optimal. It's flaky. What makes this different? I know a lot of people heard of visual testing. They heard of image based testing. How is this reliable rather than going like at an API level, which I think would be the ultimate goal. But probably can't do that.

[00:12:04] Don Jackson Well, there's a couple of problems with going, I'm not saying you shouldn't API test. I would never say that. Just to be clear for anybody watching this, I am not saying don't API tests. I'm also saying that unit tests. I'm saying that you need to user experience test as well. You usually had to do that as a manual tester to get that user experience. I'm going to go on user experience just for a second, right?

[00:12:30] Joe Colantonio Sure. Think about accessibility testing. How do you do accessibility testing? I can now do an assertion that says, just make sure that this page matches WCAG 2.0 standards. That's my assertion. And all of those standards, it goes, grabs those standards. It says, okay, is it matching those standards? That's huge. How is that different. Behind the screens, what we're doing is we're actually capturing the image and then interrogating the image of what's on the screen. Regardless of which browser it is, whether it's iOS native, web native, or mobile responsive, we don't care, right? Cause it's grabbing a screenshot at the time. In fact, I was playing with it when I first started, a whole six weeks ago, I started playing with and I'm like, okay, so install this app. Like I picked one of my apps that I like to use. It's for tracking the weather. I'm not using a name of the company though. And I said, if it's not installed, go install it. So it's like, oh, I see I'm on an Android screen based on what I see. I don't see the app. Let me swipe from the bottom to see if it's in my app catalog. I don't see it in my App catalog. I'm just gonna try to do a search first to see it's on my search, if maybe it's just not, the icon's not there. So it did a search. Said, no, it's not there. It said, okay, I'm gonna click home and then I'm going to click on the Play Store and I'm in a search for it. And then it said okay, I see it and I am gonna click the install button. Oh, I see that there's a progress bar that's still going. I gotta wait and it waits three seconds and I cut checks again. Oh, the progress bar. And the app it was a really slow to install app. That's why I'm not naming names, because to protect the guilty. And then finally it said, oh, there's an open button. Now I can click open. The thing about how hard that would be to script today. I got to do a loop, waiting to check the state of it until the open button shows up. I didn't have to do any of that. I just said, hey, open the app. If it's not there, go install it. And it recognized, oh, I was on a Google device and I was an Android device, not on an iOS device. I knew I had to go to the play store instead of the app store. All that kind of stuff that I would have to script. That sucks to script.

[00:14:42] Joe Colantonio Some people like scripting. They just like to script. So the people that love to script, would you say to them, like the jobs at risk, like how bulletproof is this method?

[00:14:51] Don Jackson So I would say this to the people who love the script. One, we're not saying replace your script. If you have a bunch of scripts that work, the old adage, if it ain't broke, don't fix it. You can make calls to us from your framework of choice to fix that one broken area that you have that is always causing you problems. One, that's number one. Number two is, I'm gonna say something controversial. Are you ready for it? Some of the best testers that I've known in my many years of my career are the worst scriptors. And conversely, some of the best scriptors were the worst testers because they didn't have that destructive mindset. Wouldn't it be amazing if I could have my best testers be able to do automation?

[00:15:41] Joe Colantonio Yes, because they usually are the business experts as well that really know the application. It's not just randomly creating scripts. You're like, yep, all right. You act, they actually know what it is, what makes sense to help drive it.

[00:15:54] Don Jackson And to those scriptors who are really good scriptors, wouldn't it be amazing if they could just focus on the API testing and the load testing instead of having to deal with the stupid UI elements that are constantly changing? Have you ever tried to test Salesforce Lightning? It's horrible, horrible to try to test from a UI perspective because everything's constantly changing.

[00:16:19] Joe Colantonio Don, for the people that don't know, you said it's image-based.

[00:16:22] Don Jackson Sort of.

[00:16:24] And I know Perfecto has a great cloud type of environment, like it has a lab. So it sounds like they could write the test once and they could easily just run it on all the things and not have to worry, oh, I have to separate test for this, extra logic for that. Is that how it works?

[00:16:41] Don Jackson Yep. So it's part of perfecto mobile. Okay. All those features that you know, in love and perfecto mobile, well, at least I hope you love them because I work there. I'm hoping you love them. If you don't, here's your chance to love them. How's that? But all the different flavors of mobile devices, all the different flavors, of browsers and you know X number of versions back of browsers on your lab, safari on Mac, all that stuff you can script at once and because it's at runtime, determining the best way to go through that. You think about your application behaves differently from web than it does to iOS native and Android native. For example, you usually on the web don't have a hamburger menu. You have a different kind of menu that's usually a bar across the top. Well, that's now different objects and I have to handle differently. If I'm using Selenium, then I'm switching between Selenium and Appium or maybe I'm use Expresso or whatever I'm doing. now, instead of having multiple scripts that do the same business process, I've got one test, not a script, a test, because I'm just doing test steps in plain language. Notice I didn't say plain English. I said plain language, how many languages do you speak, Joe?

[00:17:55] Don Jackson Unfortunately one.

[00:17:56] Don Jackson Yeah, me too, me just one, but would it be amazing if those teams that I've got around the world as we follow the sun could write their tests in their language of choice? Now, we don't support every language that's out there. I'm pretty sure we don't support Navajo.

[00:18:13] Joe Colantonio But it's AI. Why wouldn't you support all the languages?

[00:18:17] Don Jackson Because the AIs don't, let's be honest, the LLMs don't have every single language out there.

[00:18:21] Joe Colantonio That's true.

[00:18:24] Don Jackson Most languages. How about, I would say probably 98% to 99% of the languages out there we support. For example, we did this, we were doing an event in Malaysia. We decided to do the demo on Malaysian airline. And I've got a video for this. I'll send it to you. And we basically said, book a flight from KL to London. Okay. Well, what's KL mean? I didn't know until I thought about well, what cities are in Malaysia? Oh, that's Kuala Lumpur. Okay. Obviously KL in this, but think about how impressive that is that the AI to understand that context. Cause it said, Hey, I see you're looking on the Malaysian airline site. I'm going to assume that you mean KL is Kualalumpur, so I'm going to book from Kualulumpur to London and you know what? I'm going to put you in first class cause you didn't specify what class of service you wanted so that you can sleep on their flight. Into all the stuff and it's like Man, to script that, what used to take me days is now taking me minutes to script.

[00:19:29] Joe Colantonio Don, it sounds like magic.

[00:19:30] Don Jackson That's funny. This other person that you know really well said the exact same word. He said, This magic thing!

[00:19:41] Joe Colantonio What can people really expect? What happens? Is it a 100%? Because I've used like Playwright MCP server from Cursor and I do the call, it does its thing and it still takes me 15 minutes maybe to get up and running to do what I need to do and actually work. And still not kind of not a 100%.

[00:19:59] Don Jackson You learn really quick that sometimes your plain language step isn't terribly descriptive. That's what you learn. And you learn to be descriptive on your step. There's a lot, you think about what I said is, book a flight from KL to London. That's what we said, a one-way flight next Monday. By the way, I said next Monday, I didn't say what date. I said, next Monday so it's got to know what date it is and what next Monday's date is, anyways. So it made a lot of assumptions in there. What class of, so I talked about class of flight, do I want a window or an aisle seat or do I not care or whatever that case may be. You start learning, oh, as I'm looking at the results and I can see it tells you this is what I see on the screen. This is what your instruction set was. This is the assumption I'm making and this is when I'm gonna do. You start going, oh, well, I didn't want it to do that. I changed my prompt. You learn to do better prompting. If you've never done any prompting before, you're not gonna be as efficient as somebody who understands the concept of prompting.

[00:21:06] Joe Colantonio And a lot of people blame the tooling rather than the prompting, because I actually had that issue with the Playwright. I said, do a search for this, to find, I was looking for stats for my podcast. And it did a search, but it didn't do search in the right area. I had to tell it, cause I didn't tell it. I had it say, no, go to episodes and click on that search and do the search there. It did, a search but it didn't do search cause I didn't give it enough context. It sounds like, it's not the tool.

[00:21:33] Don Jackson So you take that prompt, the big difference between what you're describing and what we're releasing is that what you are describing creates a script, you go through this prompting, these iterations, and then you have a script. And now that script is now going to be running on your podcast or your website.

[00:21:51] Joe Colantonio Yes, that's right.

[00:21:53] Don Jackson And you make a change, now the script's broken. Even with self-healing, I now have to go in and interact with it. With self-Healing, with self healing solutions, they say, hey, I noticed your script broken this place. Here's what I want you to do. Do you want me to do that? I have to interact with it? With our solution because at runtime, I'm taking that prompt and executing against that prompt instead of a script Is your app change. And I don't care that it changed. As long as you can figure out how to navigate through there. If you tell it to log in and somebody removed login from the login process from your application, guess what? Your script's gonna fail and that's a true failure, hopefully. Hopefully you didn't purposely remove login.

[00:22:43] Joe Colantonio Right. So that's another I could see, not roadblock, but another objection or someone could say, how do I know it's really doing what I think it's doing? Or it's saying it's doing or if it's during healing, self-healing and visual, like, how can I trust it? Those types of objections.

[00:22:58] Don Jackson Well, that's why we take screenshots and actually have videos of it playing as well. So you can go back and watch it. If you don't trust it, go audit it. What you're gonna find is your audit level is gonna go way down as you build trust with it. And now I found bugs. I have a great example of, I was staying at a hotel last week and I have password manager on my keyboard and I use a different keyboard than the OS keyboard. And on the application itself, I couldn't see the chat where I type my values. Well, if I told it to type my value, it's not gonna see it either. Well, that's because there's a bug in the application. So you have to tell it to handle that differently, like tell it collapse the keyboard because the keyboard's in the way. But some of those things are, if you can't see it on the screen, it's gonna see either. Now, it'll scroll as well, especially if you tell it, This is a large form. Scroll so that you have the most, this is part of my prompt. Scroll so you have most fields on this large form visible at one time, and then type all the values at once as one step. Instead of type this, take a screenshot, interrogate it, type this. I can speed it up based on how I'm prompting it can affect the performance of it. And by the way, I know I can see it in your head, performance, absolutely it does not perform as fast as doing a straight Appium script. Absolutely.

[00:24:28] Joe Colantonio But it's more reliable. This is a trade-off. I saved you time.

[00:24:32] Don Jackson It's more reliable, faster to script.

[00:24:34] Joe Colantonio Debugging, triaging the whole.

[00:24:38] Don Jackson Doesn't break. Well, if your application breaks, I'm sorry. That's what the test is supposed to do, right?

[00:24:45] Joe Colantonio Yeah, exactly.

[00:24:45] Don Jackson Hey, if your application breaks, your script better.

[00:24:49] Joe Colantonio Exactly.

[00:24:50] Don Jackson Newsflash, if you're application breaks your script should fail.

[00:24:55] Joe Colantonio Yes. You ready to hear from us?

[00:24:56] Don Jackson That's a big, you heard it first on the Joe Colantonio's Test Guild.

[00:25:03] Joe Colantonio Yes. All right. I'm trying to think of other things people might have an issue with. You kind of just, just went over really quick. You said you had a customer and they gave you 10 paths to do something they only knew of 5. So is this like a spider then you just say, go to my website, log in, do blah, blah, and it's able to find maybe areas of your code that don't have test for. Like what are the functionalities that this can give you that people may not think of right off the bat?

[00:25:35] Don Jackson It's how inventive and creative and destructive can you be with your prompt. This is why I was talking about going back to tester versus scripter. I wanna hire people that have this destructive mindset and this inventive mindset that are really good testers that can think of all the different scenarios to go through that. Think about exploratory testing, wouldn't it be great if I could start automating at least some, if not all of my exploratory tests? Well, when you are doing exploratory testing as a tester, you have an idea of what you want to do to be destructive towards this or try to break it. Instead of just doing that, write the prompt to do that. And now that exploratory test is now automated. I can do an exploratory tests of find figure out all the ways that I can get, like I use earlier to get to the shopping cart. You think about an e-commerce application because I use that example because we all understand it, how do I get to a shopping cart? Well, usually there's the shopping cart button at the top, that's the easy one. Well, I can search for a product and there'll be a shopping cart on the bottom as well to add it to the cart and take me to the cart or I can just add to cart. Well, that's way number two. I can look at my history, my search history or my order history and potentially get navigated to the cart that way. All these different ways. And so if my script, I did it. No scripts, right, just say no. Just like Nancy Reagan, just saying no kids. No scripts, no tests, no scripts, no frameworks, no maintenance. That's our tagline.

[00:27:10] Joe Colantonio Is it real?

[00:27:10] Don Jackson Yeah, it is.

[00:27:10] Joe Colantonio Oh, you just made that up. Okay.

[00:27:11] Don Jackson No, that's our tag line. No scripts, no frameworks, no maintenance. I'm going to be doing a, I'll be talking about this at Star West as well, where I think I'm gonna, I haven't decided for sure, but I think the title of my session is going to be something like Selenium Cypress Playwright Oh My, No frameworks with Agentic AI. I don't know, something like that. So something to try to, I can't tell I try to have a little fun with this too.

[00:27:36] Joe Colantonio All right. I know Perfecto has a lot of built in functionality. You mentioned accessibility testing. If someone does a prompt, do they get extra benefits as well? Will it automatically do like accessibility checks or security checks or high level performance checks? Like, hey, this was slow. Like, does it add anything on top of it other than maybe you actually physically having to prompt everything?

[00:27:59] Don Jackson In the initial release, it has to be part of that individual prompt, right? If I were operationalizing that, I was implementing this, what I would do is I'd have a certain set of prompts that I would put in for everything. Do a spell check on the screen, check WCAG standards, and do a context check on this screen for the natural language. So if somebody typed massage instead of message, right, well, that's not a spelling error, that's a context error. Now, what we will be releasing in the future, I don't know what you consider long or short term, but in the feature, you'll be able to set those as essentially policies for the script. I wanna do, on every step, I wanna WCAG standards.

[00:28:40] Joe Colantonio Does it know what WCAG standards are? Or do you have to have a prompt in there ready to go? How about does it know, give me OWASP top 10 security risk and blast it.

[00:28:48] Don Jackson Sure. Right. Whatever that is. You're behind the scenes, you're prompting an LLM with that text to interpret that text. If it's a standard like OWASP or WCAG or whatever, that is I don't want to say globally known, but well known that the LLM will know it then yes, if it is your own personal standard, no, you are going to have to define that standard, like branding standards, right?

[00:29:14] Joe Colantonio Yes.

[00:29:14] Don Jackson How would you automate brand compliance? Is this the right color red. I've worked at a bunch of companies that red was one of their primary colors, but they're all different shades. Is this the right color red? I could just automate that and say, hey, make sure this is the right red.

[00:29:32] Joe Colantonio That's awesome, because I used to be a part of companies that were acquired and you have to change the logo and you have to check everything.

[00:29:38] Don Jackson Uh-huh.

[00:29:38] Joe Colantonio This is pretty cool. It could do for you, very nice.

[00:29:42] Don Jackson Think about, hey, let's say hypothetically, because this would never happen to anybody that watches your show. Let's say, hypothetically you had to test it in a different language, then you speak. Like, for example, I took three years of French in seventh, eighth and ninth grade. That was a few years ago. I don't speak French too well. And I worked for a company that we had a business unit in Canada. It had to be bilingual. They asked me to test that I'm like, well, I don't know. It might be right. I mean, the objects are there. I can tell you the objects are there, but are they spelled right? Are they have the right context right? Are they using formal versus informal pronouns at the right times? Dude, I don't remember any of that stuff from French.

[00:30:27] Joe Colantonio Yeah, I use this example all the time. I used to work for a company. They used to have Excel files of all the UIs and what the wording is, and then you have to get it officially okay by some localization company, and then you do the test and you look at the file. This is even better because I guess you could tell if a screen, if the words go off a screen for a certain language or it could do that all for you and that's awesome. You could just say, Hey, test this for French. See if it's correct. Nice.

[00:30:54] Don Jackson Exactly.

[00:30:55] Joe Colantonio Love it, love it.

[00:30:56] Don Jackson Joe, sounds like you want to work with us. You want to look for us now, right? Cause now you're thinking about all these-I mean, it really is. It's a paradigm shift in test automation. It really is right. I come on your podcast, what? 8 years ago, 7 years ago. I don't know when we talked about computer vision and how that was paradigm shifting. And using computer vision in AI, this is equally, if not, I think it's more paradigm shifting because it's computer vision and it's using natural language processing and it using LLMs all together to do this for you. It's Joe. Why don't you come watch it? We'll sit down and do a session together.

[00:31:37] Joe Colantonio Well Don, what's even better is I believe Perfecto is doing a weapon out with us later this month on July 30th, called Perfecto AI in action, smarter, faster, zero hassle testing. Is this going to be what you're demonstrating or is this something else?

[00:31:52] Don Jackson Absolutely. I'll be on there myself and our head of product, Steven Filoni will be on there and this is exactly what we're going to be talking about. We'll be talking, we're gonna show it. I'm thinking about doing kind of a stump the chump type scenario and just prompting the audience say, give me an app that I can go download and we can try something. Just to show, right. It doesn't matter what the app is. It doesn't matter. I'm just going to go in and do it.

[00:32:18] Joe Colantonio And we love skeptics. So if this is you, you don't believe it. It sounds like magic. You could definitely register for that webinar down below or contact down directly to see this in action for yourself.

[00:32:28] Don Jackson Awesome. Thanks, Joe, for taking the time today. I'm super excited. Look for the press release on the 15th. We'll be talking about it. We'll have hyperlinks in there too so that you can do a free trial on it.

[00:32:39] Thanks again for your automation awesomeness. The links of everything we value we covered in this episode. Head in over to testguild.com/a554. And if the show has helped you in any way, why not rate it and review it in iTunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe, my mission is to help you succeed with creating end-to-end, full-stack automation awesomeness. As always, test everything and keep the good. Cheers.

[00:33:14] Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation, testing, and DevOps. If you haven't yet, join our vibrant community at TestGuild.com where you become part of our elite circle driving innovation, software testing, and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate. Visit TestGuild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.

[00:33:57] Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold.

{"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}
Two men are featured in a promotional image for "TestGuild Automation Testing" discussing Playwright and AI in QA, with the text "with Ben Fellows.

Playwright, Cursor & AI in QA with Ben Fellows

Posted on 08/31/2025

About This Episode: In this episode of the TestGuild podcast, Joe Colantonio sits ...

Why AI + DevSecOps Is the Future of Software Security

Why AI + DevSecOps Is the Future of Software Security With Patrick J. Quilter Jr

Posted on 08/27/2025

About this DevOps Toolchain Episode: Support the show – try out Insight Hub ...

A man with glasses and a beard speaks animatedly into a microphone. Text reads "TestGuild News Show: Weekly DevOps Automation, Performance Testing, and AI Reliability. Breaking News.

Playwright MCP, Cypress FHIR API, AI Test Management and More TGNS167

Posted on 08/25/2025

About This Episode: Is AI the future of Test management? Have you seen ...