The Importance of Trust in Software Testing and AI with Damian Synadinos

[00:00:35] What does trust have to do with software testing and AI? Well, that's what we're going to be talking about all today in this episode with Damian. If you don't know Damian has over 30 years of experience. He does all the things he does with testing. He does a lot of speaking, a lot conferences, a lot of keynotes, a lot of talks, a lot of training. You probably know him, so I'm really excited to have him. Also, he's really helped in the past organize QA or the highway, which has become one of my favorite testing conferences in the real world at qaorthehwy.com. You definitely should check out. And like I says, I don't know if you know this, he actually does a lot of improv and comedy. He's also an author and illustrator. He does all the things. So really excited to have him back on the show. If you want to know more about trust, what the heck is it? What does it have to do with AI and testing? Like I said, you don't want to miss this episode. Check it out.

[00:01:19] You probably heard the old saying, seeing is believing. And that's why I think this goes a long way towards building trust that we talk about in this episode, and that's why I'm excited about this week's sponsor. Look, visual bugs don't just look bad. They kill retention, trust, and revenue. 88% of users won't return after a bad digital experience and 75% judge your brand by a UI and 35% of your revenue is lost due to poor UX. Manual visual testing is tedious and often prone to errors. Open source tools are outdated and require high minutes. What should you do? That's where BrowserStack Percy comes in. It makes visual regression testing for websites and complex workflows effortless. Even at scale, think of it, hundreds of screenshots being compared at once. Also, it requires minimal code changes to set up. It offers seamless, indirect integrations with your favorite tools over 50 plus, including Figma and Storybook. It also has this really cool feature called Visual Scanner that helps you monitor thousands of pages across real devices and browsers in under a minute. What's more interesting is the visual review agent, which is powered by AI that helps cut through the noise and false positives, reducing manual review times by 3x. With Percy, teams like LinkedIn, IBM, Adobe, etc. are already shipping faster with full UI confidence. But as I said at the beginning, seeing is believing, so try it out for yourself using that special link down below.

[00:02:51] Joe Colantonio Hey Damian, welcome back to The Guild.

[00:02:52] Damian Synadinos Hey Joe, thanks for having me. It's me, it's you, again. All right, good to see you.

[00:03:00] Joe Colantonio Finally, again, it has been 9 years since last time we talked.

[00:03:04] Damian Synadinos Too long, too long.

[00:03:05] Joe Colantonio On the podcast, yeah, for sure. So what's changed in 9 years in the testing software kind of world?

[00:03:11] Damian Synadinos Boy, nothing and everything? The basic tenants are still there. Those are done changing, and the technology is changing at an ever-increasing pace.

[00:03:21] Joe Colantonio Absolutely. So one thing I know is obviously a hot topic since last time we spoke is AI. Before we get there, we're going to try to go high level and then like actually see, explain to people what it means and why it pertains to software testing. That is you want to talk about trust and like trust. What the heck is that about? So why did you want talk about trusts at like a high level?

[00:03:42] Damian Synadinos Yeah. Oh, that's a great question. And I get that a lot. So the kind of the origin story for this talk was, it actually began to be honest with the last U.S election. If anyone anywhere in the world is unaware, we had an election last year and watching the news, talking with family members, the topic of trust came up over and over and over. Do you trust this candidate? Do you just this party? And I'd hear it on the news. I'd here it with family members. And I started to think about whether I do or distrust certain candidates, certain parties. And then why do I trust these things? And then as I dug deeper and began to read and learn more, I realized that trust was incredibly important, not just with regards to elections, but in regards to my life, every aspect of it. As I dug deep, I realized that this was an important topic and I really wanted to get the word out. Simultaneously, as a full-time public speaker, I was trying to get into conferences and events. And typically, because of my 30-year testing career, I would speak at a lot of technical and testing events. And getting soft skills stuff, which is my bread and butter. That's the topics I talk about, epistemology, communication, semantics, linguistics, empathy. That stuff is tough to get accepted at tech conferences. I was trying to think, how can I get the topic of trust accepted at a test conference? And I thought, well, trust invades every part of my life, including my career as a tester and what other things does trust relate to? It relates to AI and testing. I thought well, I can create a talk that's ostensibly about AI, but truly about trust. And that's kind of the origin story of the talk.

[00:05:09] Joe Colantonio Awesome. And I think you could chuck me off the ledge here. Some breaking news. A few days ago, Google just released a VEO3. Not sure if you've seen it. It creates really, really realistic video. And talking about politics, I hate politics. I think it's ridiculous. I think politicians actually create a fake thing to rile their different segments up to vote against another. Like that's all it is. Both parties. That's all I'm saying. Anyway, and the reason why I'm seeing that is.

[00:05:33] Damian Synadinos Performative.

[00:05:35] Joe Colantonio What's scary is this video I could see. Doesn't matter who what party using this to say this person said this or this person said that to get their little segments upset and riled up. And then do you see that as how is that going to impact trust? I guess let's boil that all down to this new. I don't know if you saw the new video released by Google flow VEO3, but it looks really realistic. And I see that's been really, it's only going to get scarier as we go along. Any thoughts on that as it applies to trust?

[00:06:01] Damian Synadinos Yeah, I've seen what you're talking about. This new release is something else. It's incredible technology. I think in the modern parlance, we're cooked, is what they say. I've see some demonstrations of it on various feeds and it is unbelievable. How does trust relate to that? It makes trust even more important. Questioning, asking, challenging the things that you trust or distrust and why? Why is it so? Why do you trust this thing? Why do distrust that thing? And digging deep to find those answers, you may realize that your sense of trust or distrust is misplaced. And if so, you can reevaluate your decisions, your thoughts and actions and make better thoughts and action, make better decisions. So I think trust is increasingly important with the advent of AI and new technologies like this VEO.

[00:06:46] Joe Colantonio So as a tester, how do we what mindset do we need to know, like not just make assumptions like, oh, this is real or this requirement is correct. And then based on that trust of the requirement, without maybe seeing if it's a real requirement and not testing it. I don't know if that makes sense, but how do we know what to trust when we should trust it as a tester?

[00:07:06] Damian Synadinos Well, I think the trust is infused by a lot of different things. Being a tester, I thing good testers, anyone can test. Children test toys. They poke and prod and learn from their actions and respond accordingly. Everyone tests from childhood on. But being a good tester, doing it with agency, with intent for a particular purpose is different. So being a tester I think is all around curiosity. You can't be curious unless you question things and wonder about things. And questioning and wondering is about questioning and wondering your assumptions. So you're given a requirement, whether it's explicit or implicit or someone tells you or it's written down and you might just take it at face value and go, this is the way it is, or as a tester and you're curious, you go, hmm, why is this so? I wonder, where did this come from? Is it true? And those are the types of questions that make you challenge and question your sense of trust and dig deeper and find out, is this requirement something that is valid, something's true, something to be trusted or something that I should dig deeper and find alternate viewpoints perhaps, alternate requirements, conflicting requirements.

[00:08:10] Joe Colantonio How do you convey that to your organization? Like, I know this is what you said the customers want, but maybe we should validate it. Like, how do you not just go, I'm just going to do what they say. I want to keep my job. I trust they know what they did. It's a requirement or something.

[00:08:27] Damian Synadinos If keeping your job is your number one priority, then you just do what they say.

[00:08:31] Joe Colantonio Yeah, that's true.

[00:08:33] Damian Synadinos For each thing is a time and purpose. But if you want to be a good tester and you want to truly challenge these things and test as effectively as possible, I think one way is focusing on these fundamental concepts. The things that I speak about, I talked about communication and empathy and epistemology. How do you know what you know? And trust, these are very, very fundamental concepts related to people. I mentored a tester one time, somebody that was in customer service and they wanted to get into testing. And they said, there must be a better way. I answer the same questions over and over. Wouldn't it be better if we solve the problem at the root rather than just putting band-aids on it each time a customer calls in with a problem? So I sat down with them. We began talking about things like epistemology. How do you know what you know? And communication, how do you express the ideas that you have in your head and get them into someone else's head? And transfer and exchange ideas. And we talked about perspective and relativism. How things might, a bug is not just a bug as a bug, it's relative to someone at some time. And these are very fundamental concepts, communication, epistemology, relativism, causality, what causes things going forward and backwards, what are the effects and causes? And after several weeks and months of talking about these topics, my mentee said, hey, we haven't talked about bugs, we haven't talked about requirements, we haven't talked about testing. And I thought, hmm, or have we? In fact, we've been talking about the fundamental concepts that flow upwards into all the testing activities that you do on a daily basis. Exploring, poking, prodding. Experimenting with the product, communicating your findings to others, talking about risk and epistemology. How do you know that something's a risk and communicating that to others? All of these things can help you write better bug reports, test more effectively. So to answer your question, how do you communicate this to an organization? Rather than starting at a high level and saying, oh, with requirements, do you trust them or not trust them? I would start at the low level and teach them the importance of trust, which is, again, what my talk, Trust But Verify, is trying to do. It's ostensibly about AI and testing, but it really is applicable for architects and plumbers and bakers and anybody because trust invades all parts of our lives. If I were trying to convince a baker to either trust a recipe or an architect to trust a blueprint, I wouldn't start with talking about recipes and blueprints. I would start with the importance of trust and the importance of epistemology, the importance of empathy and communication. And once they understand those core concepts, then I can say, now let's abstract those to a higher level and talk about recipes and blueprints and requirements and bugs and those types of things. To approach an organization, I would start at the lowest level, which is exactly what I try to do with my talks.

[00:11:09] Joe Colantonio Absolutely. And I'll have a link for that talk down below. But you actually had something in that talk that I thought was interesting. Being an old guy, trust is not binary. I like black and white. I like truth. And nowadays people like it's my truth. Anyway, so what do you mean by trust is not binary?

[00:11:25] Damian Synadinos Binary is easy, black and white is easy. True, false, yes, no is easy but it's not always right. Trust, you can have a high degree trust in something or a high of degree of distrust in something. And trust can be infused by many different things. If you think trust as a definition is based on faith, it's based on belief. If you about the most famous trust exercise, what can you think of? When people are trying to exemplify the importance of trust, what do they do?

[00:11:51] Joe Colantonio Just because I saw your speech right, when you fall back on a partner and they try to catch you.

[00:11:56] Damian Synadinos Alright, you GE, but that's alright, the trust flaw, yes.

[00:11:59] Joe Colantonio They actually did that at GE. GE used to have a school they'd send people to for these things. And obviously I'm like, this is a cliche, but anyway, yeah, absolutely.

[00:12:09] Damian Synadinos Yeah, so the trust fall is the golden example for exemplifying trust. Now, the reason it works is because you're falling backwards into someone. You're not falling forwards. If you were falling forwards, you would have sensory input. You could see them that they're going to catch you or not catch you. And therefore you're informed. You have knowledge, whether the person is there to catch you, but if you're following backwards, you have no sensory input, you have know knowledge that they are there. So you have to rely on faith and belief. And those are the core elements of trust. That trust is very much based on faith and belief, but that's not all it is. Trust could also be infused by other things, for example, authority. Someone might say, I trust someone because someone else trusts them, and I trust that person. So by the transit of property, A, trust B, B, trust C, I, trust, C. The transit property because of authority, or maybe you have an intuition or an emotion, the gut feeling that you can trust this, or maybe it's cultural, you trust something because it's been handed down generation after generation. But another way that you can trust something is through personal experience. Maybe you trust something more because you can see its internal workings, or you might trust something less because it is invisible, it's opaque and you can't see how it works, so you distrust it. Or maybe you trust because you've had previous experience with it and it went fine. You've ridden elevators before, driven across bridges and they were fine. Therefore, you trust this new elevator and bridge will be fine. Evidence and experience, those are types of evidence and evidence leads towards knowledge. You can trust something or distrust something. You can have high degrees of trust with just a little bit of information and knowledge informing it, or you can have less trust and more knowledge-based beliefs. Does that make sense?

[00:13:49] Joe Colantonio It does. I'm wondering if what people think about faith-based trust. I went to Providence College. I was taught by Dominican monks. They taught something called secondary causes. I see smoke. I don't see a fire, but I can make assumptions that it most likely is a fire causing that smoke. Is that what you mean by faith- based kind of trust?

[00:14:09] Damian Synadinos Inductive, deductive reasoning. Those are ways that can influence your sense of trust, absolutely. When there's smoke, there's fire, maybe. Maybe there's not a fire. You're trusting, based on previous experience, that when there is smoke, there's probably a fire, and there not necessarily is a fire so I would rephrase that old idiom, where there's a smoke, there might be a fire or there probably is a a fire. I like to deconstruct a lot of old idioms to make them more correct and accurate and applicable. That's one of them.

[00:14:40] Joe Colantonio So I know a lot of times you go to management, they're like, should we release yes or no, they just want a yes or a no. How do you reconcile maybe a faith based belief with the binary type of what answer that they want?

[00:14:51] Damian Synadinos Well, I think a lot of it is due to risk. Low risk situations are situations when trust may be just fine. It may be appropriate and reasonable to rely on faith, on beliefs, not on knowledge because if something goes wrong, it's no big deal. Or if something go wrong, it's a big deal, but it's an outside chance. In that case, maybe trust and faith-based beliefs are just fine and you can answer to your boss or to management with a yes or no answer because the consequences are very low. But in high risk situations, it may be more reasonable to explain to management, this is a high risk situation. I can give you a yes, no answer, but my confidence in my answer is going to be very low because this is the high risk situation, we should rely more on facts and evidence and knowledge and information to make more informed decisions and rely less on trust and faith and belief, because if something goes wrong, it could affect the bottom line. It could affect our standing with our customers, that type of thing, our brand. I think a lot of a way of convincing management is talking in terms that they understand bottom line money and expressing it in degrees of risk. Low risk situations, faith is fine, high risk situations. You want to rely more on knowledge and evidence.

[00:16:01] Joe Colantonio Absolutely. You also mentioned something about healthy skepticism, and I'm just curious to know, does this relate to improv or acting? I forgot what the saying is. When someone's improving, you say yes, and rather than cutting them off. So it almost keeps you open to areas you may not have thought of. Even like maybe you were so focused on one area and rather than just say no to someone's suggestion, you go, oh yeah, let me try that. I don't know if that makes sense. Is that connecting?

[00:16:26] Damian Synadinos I like that correlation. Thanks for bringing improv into it. I love improv. I don't know if I've made that connection, but I can definitely see how it relates. Improv is about going with the flow, responding. Yes and is about agreeing with whatever was said, whether it's reasonable or unreasonable, and then adding onto it. And it's a sense of open-mindedness. And absolutely, skepticism is about being uncertain. It's about having an open mind, whether to trust or distrust something, whether to believe or not believe something. Skepticism often gets a bad rap. It's seen synonymous with doubts or suspicion. Or even being jaded or cynical. But in fact, skepticism by definition is about being uncertain. It's being not sure. And so same thing as improv on stage is having an open mind that whatever's said you can go with the flow. I like that correlation. I hadn't thought about it myself. Well done.

[00:17:16] Joe Colantonio Awesome, awesome. Now you can add it and just give me credit when you mentioned it on stage. I mean, I guess this leads me to AI, you mentioned Michael Bolton. We had a really great conversation around LLMs. And the thing that came up is it's very persuasive to make you think like it's telling the truth and it's not thinking, but it's like, it makes it like, so certain of what it gives you as output, how do we not get lulled into being thinking that this is someone that's actually thinking and give me results?

[00:17:46] Damian Synadinos I'll draw a correlation. Again, this talk is ostensibly about AI. I'll talk back. I hate politics as well, but that was the origin story for the talk. And if you think about politicians, as you said earlier, their entire game is crafting a very carefully crafting a message that is consumable and appealing to the masses. They may say something that's absolutely not true, but say it in a way that really feels true. Stephen Colbert coined the term truthiness a few years ago. They have a lot of truthiness in their words. And outside of politicians, you may have relatives or coworkers that are soothsayers that can really tell you something and you trust them. You're like, man, the way that they said that is absolutely seems true to me. Why question that? Another source of seemingly true information is AI. It can seem really, really fact-based. It can see really convincing, really persuasive. And so I really don't see AI in this sense as that much different than a smooth-talking politician, or a suave uncle, or a charismatic co-worker, or something like that that can deliver a message in such a way that it allays your doubts. That's why I say, rather than just accepting things based on faith, if you're in a high-risk situation, it might be smart to apply skepticism, be not sure, and start doing critical thinking, challenging assumptions, asking questions, pausing for reflection, those types of things.

[00:19:09] Damian Synadinos Love it. And I also, I think in your presentation, you may have mentioned something about prompt engineering being the modern version of asking better questions. How does better quality questions influence maybe the trustworthiness of AI? Because I think it goes both ways. You can ask really terrible questions. You're going to get terrible responses like anything. Garbage in, garbage out. So how can we work on better questioning then?

[00:19:32] Joe Colantonio Well, there's an entire field around it called prompt engineering, as you mentioned. So essentially that's the modern term with regards to AI for asking better questions. How do you ask the better questions? It's an art and a science. Several of my talks around communication, semantics, linguistics, focus on this about the art of asking better questions, whether they're binary closed ended questions, whether the open ended questions that allow for exposition and being more verbose and more information. Polls, questionnaires, surveys, Likert scales, ranking and rating. There's all sorts of ways that you can question people or AI to get better responses. Like you said, garbage in, garbage out. If you're asking AI poor questions and you're getting results out, your confidence might be low because you say, well, was my question sufficient to begin with? So I encourage people to learn if they're using AI to learn to ask better questions. Outside of the context of AI, I encourage people to think about the question. That they're asking their uncle, their coworker, the politician, or whoever they're talking to and say, is this a reasonable question? Does it properly embody all of the things I want to know? If not, then your question is part of the problem in the bad answer that you get back.

[00:20:42] Damian Synadinos Absolutely. I guess trust goes at the heart of everything. It's not just testing, but it's all things in life. But especially when it comes to the job, do you think it's our responsibility to question things better to our management? Because once again, Microsoft just made an announcement, I think two weeks ago, they're laying off 6% of their company for, they said it's going to be replaced by AI. How do we trust these people that are supposed to be trustworthy? The leader of OpenAI, the leader of Microsoft, all saying AI can do all these wonderful things and it can replace you. Is it up to us to question them and say, look, this is what AI can do and what it can't do to try to maybe level the playing field?

[00:21:22] Joe Colantonio That's a tough one. That's big, big, question. Let's see. I would say that there's a time and a place. I'm very context-driven tester. I'm a very context driven person in my life, in my work, in my play. I let context drive and part of context is purpose. What am I trying to achieve? We mentioned earlier, if you're trying to keep your job, maybe just do what they say. If that's your number one priority, maybe convincing someone that they're wrong or incorrect isn't the prudent thing if you try to keep you job. If that's not the top of your priority stack, if there's other things, ethics and morality, and those types of things are important and doing good testing and producing a quality work out of yourself, then I believe that it is prudent and advisable to challenge people and question them. Now, this is not popular. You said earlier, black and white. I like it. You like it, it's easy. That's why we like it. Easy things are fun and easy. Sometimes they're not the best for long-term outcomes. Sometimes easy, fast things have good short-term outcomes, but can have catastrophic or negative or unwanted long-terms outcomes. Think about the purpose. What you're trying to achieve is the short-term gain. Does it outweigh the long- term consequences? Are the long term consequences bad enough that you say, boy, in this case, ethics and morals and doing good work is the top of my priority stack. Therefore, I need to talk to management and help them understand these things. Why what they're saying is incorrect, or an alternate viewpoint. This is all about tact and how you communicate your message. But if that's the top of your priorities deck, I absolutely think it makes sense.

[00:22:53] Damian Synadinos Absolutely. I guess it's all how persuasive you are. Rather than being seen as a gatekeeper, I think it goes back to your point about risk. If we could say, look, we're not gatekeepers. Here's what you want to do. Here's we see this as risky one, two, three, but it's up to you as a decision maker whether or not to release. But I'm giving you the information of what's risky or not. And then it's on you. I don't know if that makes sense, but it's more persuasive, I think, rather than saying, no, I'm QA. You're not releasing type deal.

[00:23:17] Damian Synadinos Yes, persuasion partly is about providing information with, I think that testers are information brokers. They deal with information. We're not decision makers. We can offer opinions based as a proxy for the user or the customer or based on our own self, but those are not the decisions. Really, we offer information to others so that they can make more informed decisions. Part of that might be persuasive. Offering information in such a way that if you think it's important, and it's ethically important and keeping your job is not the priority stack at top of your priority stack, then maybe you can offer that information in such a way that it's more impactful through by being persuasive.

[00:23:54] Joe Colantonio Absolutely. Okay, Damian, before we go, is there one piece of actual advice you can give to someone to help them more with their testing kind of risk based kind of skepticism, kind of trust, but verified type approach? And what's the best way to find a contact you?

[00:24:09] Damian Synadinos Great, great question. The advice I usually give is simple and incredibly difficult at the same time. It's ask. Asking costs nothing, it's easy to do, except for it's incredibly difficult to do because of social norms and not wanting to appear as unknowledgeable or uninformed. There's all sorts of reasons that people avoid asking questions. That's a whole topic in itself, but I would say my number one piece of advice is ask. And if you think that someone else doesn't understand what you're trying to convey and communicate to them, ask if they understand, ask if can paraphrase and repeat it back to you to your satisfaction. So my piece of advice, and this goes back to the curiosity, which is the main attribute of a tester, I believe, is ask questions. And how to find me? I'm @ineffable-solutions.com. That's my professional website for speaking, training, and consulting. My also drawstrangers, that's @idrawstrangers.com and my book is at handandstellabooks.com. I'm all over the place doing a lot of different things.

[00:25:09] And you can find links to all this awesomeness down below.

[00:25:12] Thanks again for your automation awesomeness. The links of everything we value we covered in this episode. Head in over to testguild.com/a547. And if the show has helped you in any way, why not rate it and review it in iTunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe, my mission is to help you succeed with creating end-to-end, full-stack automation awesomeness. As always, test everything and keep the good. Cheers.

[00:25:46] Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation, testing, and DevOps. If you haven't yet, join our vibrant community at TestGuild.com where you become part of our elite circle driving innovation, software testing, and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate. Visit TestGuild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.

[00:26:30] Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold.