Solving Challenges in Software Testing with Kristijan Plaushku

[00:00:35] In a land of testers, far and wide they journeyed. Seeking answers, seeking skills, seeking a better way. Through the hills they wandered, through treacherous terrain. But then they heard a tale, a podcast they had to obey. Oh, the Test Guild Automation Testing podcast. Guiding testers with automation awesomeness. From ancient realms to modern days, they lead the way. Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold. Oh, the Test Guild Automation Testing podcast. Guiding testers with automation awesomeness. From ancient realms to modern days, they lead the way. Oh, the Test Guild Automation Testing podcast. Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold.

[00:00:35] Hey, today we're speaking with the founder of QA Robots, who's redefining how new software products quality is shipped in Italy, which is really exciting. Also want to talk about automation, first principles, all the things with testing. You don't want to miss it. Check it out.

[00:00:49] Joe Colantonio Hey, before we get into it, as you know, manual testing often feels hotter than it should be. You waste time switching between tools, setting up accounts, or repeating the same test steps over and over again. Whether it's writing test cases, cross-browser testing, or managing cookies, the real challenge is the inefficiency for managing multiple tools during testing. And that's where BrowserStack's testing toolkit comes in. A powerful Chrome extension that brings 10 plus essential manual testing tools in one place. Whether you want to test the responsiveness of your website across different screen sizes, manage your cache, or format JSON, the testing toolkit streamlines your manual testing workflows. From test setup to test execution to bug reporting, the testing toolkit is packed with powerful tools that simplify every stage of manual testing. Need to validate responsiveness of your website across different screen sizes? Use responsive testing. Want to check for visual bugs? Try visual overlay testing. Need to edit cookies? Clear cache. Intercept HTTP request? It's all built in. You can even access cutting edge utilities like their AI test case generator which enables up to 90% faster test case creation and common workflow automation, which can automate repetitive manual testing workflows, such as creating test accounts and user histories. Users have experienced up to 10x faster manual testing and streamlined testing workflows by eliminating the hassle of installing and setting up multiple tools and accounts and reducing inefficiencies related to context switching between multiple tools. And for what I hear, testing toolkit is already gaining rapid adoption among QA teams and individual testers looking to simplify their manual testing stack. And the best part, the base plan is completely free to use. No credit card required. Install the testing toolkit from the Chrome Web Store and get started today. And you can find it using that special link down below. Now on with the show.

[00:02:52] Joe Colantonio Hey, welcome to The Guild.

[00:02:55] Kristijan Plaushku Thanks Joe, thanks for having me here. So yeah, I'm the founder of QAA Robots. It's an Italian based company. We specialize on providing consulting to B2B businesses in Italy and providing them with services on quality assurance, test automation, and the integration in DevOps.

[00:03:18] Joe Colantonio Very cool. So how did you get into testing? Let's start there.

[00:03:22] Kristijan Plaushku I finished my IT studies and I've got selected actually, this is really fun for an academy, which I actually didn't really know about the whole sector, but it was about quality assurance and moreover on the testing side. Basically it was a one month academy where we trained on testing, but on a sub platform. And it was really great time after the one month Academy, I got selected to join the AS Watson team, which was my first company. And that's where actually my quality assurance career actually started. Just a quick walkthrough, I had 4 amazing years in AS Watson and great teammates. I worked on a retail platform for about 2 years. We made the go live of 8 websites. This was really, really great on our site. And yeah, overall it was a great experience, my first 4 years of actual work before then starting on my own.

[00:04:28] Joe Colantonio Why do you want to start your own then?

[00:04:30] Kristijan Plaushku Right, I found out that actually I feel like something was actually missing, meaning how could I drive some of my ideas that I've gained and perspectives into something new. As Watson was actually great, it gave me a lot of perspectives on quality assurance, testing, test automation, and the whole overall view of how corporate actually operates on larger teams. I think it was really beneficial before starting my own startup. And if I would, if it were the case, if I will do this again, I would say actually yes, because it provided the actual basis for me to then learn and grow on my own and then create my own company.

[00:05:18] Joe Colantonio When we first started talking on LinkedIn back and forth, you mentioned something about it was like first principles in QA. Can you maybe give a little background what you meant by that, because I didn't actually ask you in the chat what you mean by that.

[00:05:31] Kristijan Plaushku Sure. First principles for me comes back to the basics of why we do test automation and why we do QA. For me, first principle means attacking the problem from the actual start. In the way, I actually work with companies right now, I try to integrate test automation as much shift left as possible, meaning on a commit level, I like to provide developers with quick feedback on each of their development. And actually give them product view of actually what's going wrong and if there are some actual red areas in the implementation. It's actually about solving the problem from the root and we can apply this to test automation by going shift left on a commit level and then of course having the daily runs and from a query perspective, starting from the requirements analysis.

[00:06:31] Joe Colantonio Yeah, I know you have a website, so you probably are worldwide, but it seems like you have a real focus on Italy. Is there a reason for that? Why Italy?

[00:06:40] Kristijan Plaushku I've found that in my career, Italy is very fragmented. Talent in Italy is focused either on one of those sectors which may be quality assurance on the analyst side or testing. So the exploratory and manual execution of actual test being performed on a website such as, for example, just imagine crowd testing teams. And then also there is the sector of test automation specialists that actually just write the scripts. And then there is also the role of a test ops which works more on the pipelines and all that stuff. I wanted in my work to actually have a professional that could encompass all these abilities to provide an infrastructure as well as a process to be implemented in companies in Italy and doing so just ship great products.

[00:07:35] Joe Colantonio I also think you mentioned in the show notes something about, there's a lack of QA plus DevOps integration. Is that an issue still?

[00:07:44] Kristijan Plaushku Right now, this is a very big issue, actually. It comes to be very fragmented while in larger companies, this may be more the norm. As we all know, there is each role for each specific task. And this is fine. But I wanted to actually find the gap between the two. For me, and this is something that also my previous mentor said to me, I said, Chris, if you write a script, and it's not integrated in DevOps, it's still not automation. At the time, I really didn't understand the actual concept then actually working in QA really just came about to be actually right. QA plus DevOps for me means creating an infrastructure that encompasses both the test automation benefits and then the development process automation that we need to have in development teams. And that's really a big issue in Italy. We just have specific positions and I'm trying to solve this with QA Robots.

[00:08:50] Joe Colantonio Is there a lot of work you're doing in Italy then? Do you actually get a lot of work is that busy as a tech hub?

[00:08:56] Kristijan Plaushku Yeah. So in Italy, I've been collaborating with software development companies, actually not product ones, because they are the ones that actually understand better the work of quality assurance and test automation. This is a real niche that I'm trying to actually get into. Providing support to software houses, development teams, digital teams that provide digital solutions. I found just only one client outside of Italy now, but I'm really trying to focus in Italy, my best of the work, as I like to make an impact first here and then spread out to the Europe.

[00:09:35] Joe Colantonio Nice. I know also on your website, you mentioned something about you have a stat from the World Quality Report about people struggling with test environment management. This is worldwide, but is that something you see as well when you work with clients that they still have issues with the test environment management?

[00:09:50] Kristijan Plaushku The main part with the test environment management is that there are many issues that can arise in an environment. First of all, we have this data. Then we have the actual setup of the environment. Then we need to be in communication about deploy times with developers. Most of the time, as I see some action points that we could have on the QA perspective is also that test automation is not a linear between environments. We may have development environment and UET have different tests or an amount of tests which are different and are not compatible in both the environments. This happens because again, we have different test data and different deploy times, and this really just impacts the coverage of a test optimization team that tries to have as much coverage as possible. We spend a lot of time developing, but the end result is that we lose tests on the end, on our production base.

[00:10:50] Joe Colantonio Absolutely. AI machine learning is everywhere. Is that impacted anything of what you've seen over the past few years from maybe when you started to now you're like, hey, maybe this actually solves something that we've been struggling with or causes more issues.

[00:11:05] Kristijan Plaushku Right. There are different ways we can approach this topic. I think first of all, I think that AI is great. It really helps us in our work from manual implementation of test cases. So just writing them to actually test automation scripts as we've seen with most of the open source providers. I think some solutions don't work well right now because we're not there yet. I think we've made incredible work on the AI space, on the MCP servers. Having a standard, having specifications, this is really helping us. Really thanks to the team of Anthropic and the JSON standard that they've worked out. But yeah, I think overall it's a positive force for us, for our development. What I've seen right now, which is missing and something that I'm trying to work on is how could we use AI to improve self-healing locators? I've actually recently developed a Robot Framework, AI fallback, which is actually trying to target this issue. Outside of everything that we have, we just specify in a natural language what we want to target, and it just extracts the whole DOM, fetches it, finds the correct locator based on a description in natural language, and then provides during runtime. This was just one of my ways of integrating AI into my own custom infrastructure for the test automation, which I've found to actually work pretty well. So that's my objective. Try to integrate AI responsibly first as a fallback and then in the future we'll see if it can take actually the place of writing test information scripts for us without our own supervision, which I will read out.

[00:12:56] Joe Colantonio No, we talked a while ago just prepping for the show. I think I forgot. Is your main tool Robot Framework or is it something else?

[00:13:07] Kristijan Plaushku Robot Framework, I really come to good terms with it actually in the past couple of years and I really like it because it really bridges the gap between a new person trying to learn test automation and them actually learning it. So it is great both for new teams trying to learn it and also it allows for a lot of customization. On my perspective, it is also great because it provides also some functionalities for robotic process automation. So outside of web automation, we can automate, for example, some Windows tasks and something like that. The automation part of it, of processes and on the web, it's something that I really like. The simplicity of encapsulating actual logic into keywords brings everyone to understand what the code is saying. From a product owner perspective, from a developer's perspective, QA perspective, everyone can see the code and actually understand it. That's why I've been trying to use it and model it in many different projects. And it has really been great.

[00:14:13] Joe Colantonio Yeah, I love Robot Framework. I don't know why more people don't use it. It seems like you could using Selenium, you can use Playwright in it. You can use all these different things. You're not tied to one particular framework. It's almost like a wrapper and you get all these access. So I think it's awesome. You mentioned something about adding AI. Is that something you could actually add to a Robot Framework at some point when you got that locator working?

[00:14:35] Kristijan Plaushku Yes, actually, my specific library only works in Robot Framework and you can integrate it in the actual projects, both in Selenium, Playwright, Cypress, anything that you want. My main objective was actually trying to find an AI solutions that could work in Robot Framework that didn't utilize the current methodology of using self-filling locators. Not trying to reverse map a locator. But just analyzing the whole DOM in chunks obviously and doing it smartly because it's a lot of data. But yeah, that has been one of my actual goals and one of the goals of QA Robots. Trying to integrate AI in Robot Framework first and then we'll try to see if any other open source framework can get the benefit, but yeah.

[00:15:23] Joe Colantonio Nice. When you go to these service companies, do they already have frameworks or do you actually say, hey, Robot Framework, probably you should move over to as it all depend on the situation.

[00:15:35] Kristijan Plaushku Actually in Italy, we have two scenarios where there are companies of 200, 500 people, which actually are really huge companies, at least, here in Italy. And they have their own dedicated QA teams and they know about the frameworks, languages to use, best practices. And then you have the software houses, software development teams that are great teams, but they have never encountered something like we provide at QA Robots. So it's more about bringing the culture of QA, letting them understand what we're doing, why we're doing it, what are the benefits for them as well as the clients. In my perspective and my own work, what I found out more is that they actually want you to guide the whole QA governance from the start because either it was done by testers, so they just focus on testing manually the application, and as well they didn't know. All this infrastructure and logic behind, connecting everything in DevOps and providing on a shift level, recurrent validations.

[00:16:44] Joe Colantonio I think Robot Framework recently came up with the certification program. Are you a certified vendor? Is that something you're working through? Any benefits of that?

[00:16:52] Kristijan Plaushku I'm actually working towards it to become an accredited partner for a Robot Framework. And that's amazing that you have said it. I really like also teaching and mentoring as part of my job. Whenever I get the opportunity, I try to do it. I try inspire also young people to learn about Robot Framework, test automation, and all this field. I really think it's just up to grow exponentially because in my opinion, there will always be someone responsible for the quality assurance part. So I think it's something that we need to invest to. Getting back to Robot Framework, I'm working to be an accredited partner with Robot Framework. And as well, I will try to create courses in the future for new people that will want to actually understand it. It will not be only theoretical, it will also be actual practical with exercises to learn from. When you actually finish the course, you will be able to actually understand the whole infrastructure and how to set up also the pipelines on DevOps. That's a really great opportunity for me. And I want to thank the team of Robot Framework and that whole board.

[00:18:02] Joe Colantonio Yeah, I mean, I can't say enough good things about the Robot Framework and the Robot Framework board. I run the online event for them the past four or five years. They're awesome, awesome group. Anyway, besides that, just curious to know, where would people be able to find these trainings? Will it be on QARobots.com or something else?

[00:18:22] Kristijan Plaushku These trainings will be, this is something that I'm still trying to work on. Probably they will be on qarobots.ai. It will be a specific section on our website where you could enroll and it will be in two different types. You could have also the courses which are pre-registered where you can just train at your own pace and understand the Robot Framework with our support, obviously, We will also have some professionals that will provide support to you in case of any questions. You can also write directly to me without any problem. And I will also create some live sessions with the students that actually want to learn in real time. So with me supporting that.

[00:19:07] Joe Colantonio Awesome. How hard is it to get people interested in test automation and quality in Italy? Because a lot of times people see QA as a cost center, not adding value. Do you get any pushback? Is it a hard sell? Are you like, oh, my gosh, I can't believe this is happening.

[00:19:23] Kristijan Plaushku And I'm not going to sugar coat it. It is pretty hard, especially because I feel like I'm trying to bring the culture of QA from the start. The past companies that we've had in Italy focused more on crowd testing and more on the analysis of requirements part. What I'm trained to do is something different in a way while connecting QA and DevOps. It's sort of like a hybrid role that needs to know all these different things. And of course, I've received a lot of pushback. Of course, budget comes into place. So we need to actually justify why our test automation is worthy as an asset to an organization. How many releases can we actually do more than the current load? How many testing sessions can we have automated and provide them to the table to actually justifying test automation? Meanwhile, if someone wants to learn it, in Italy, like it's not spoken about. You have development teams, you have product owners, generally people know what you're talking about. QA never comes into the picture. I think we have to do a lot of work in Italy to spread the word, let everyone understand why the position is critical for quality positions and also for regulated environments.

[00:20:42] Joe Colantonio How do you combat it, though, like with AI? Because I think a lot of people now see, oh, AI is just going to replace QA, even though you just said we probably are going to need more QA in the future. Is that a mindset you're like, oh QA? We have AI doing that. Is that another kind of battle you have to go through?

[00:21:01] Kristijan Plaushku I've heard this battle moreover on the Europe ecosystem, and what I can say is that me personally in my work, I try to make it human-centric because I believe that the reasoning should never expire. So we should be the actual professionals that decide what comes into place as quality and also provide the best decision for the actual business type in the specific case. My worry is that with the new AI solutions, they're generally web centric, so not all of them cover the mobile. That's also a big issue. And something that is also scary about this solution is that we need to really pay attention on how they're implemented. If you're working in a pharmaceutical company or an aerospace company, it's like with huge amounts of regulations. You need to pay attention and have someone to supervise and they actually orchestrate the work. This objection can be actually really be solved in just a couple of words by saying, you need a responsible person that is in the end responsible for the task.

[00:22:13] Joe Colantonio All right, so once again, a company is hiring you. How do you prove that you actually added value then? If you want to be seen as an enabler, not a cost, you have like metrics or I know people hate this term, ROI, that you can prove what you delivered.

[00:22:27] Kristijan Plaushku Yeah, so actually, let's see. In QA Robots, I like to track with much obsession, actually, a couple of metrics. So the first one is the QA environmental coverage. So how many tests do we have and to how many environments do they apply? What does this give me is that if we have different types of tests between environments, there is a test data issue or a deploy issue or a different type of issue. It really allows me to understand how much of my work can be applied to different environments. Then we have the defect leakage, which not only applies to production, but on each step of the environments. How many percentages of bug passes from development to UAT, from UAT to production and try to keep that percentage as low as possible. Also the something that I really like because I'm also under the shift left space, right? It's the mean time to detect a defect. Having the test automation at the commit level allows me to actually track bugs the same day. And that's something that I really strive towards and try to spot bugs as soon as possible on the test automation side from the actual commit and from the QA process from the requirements analysis when we actually do peer reviews with the product owners. Those three metrics, QA environmental coverage, defect leakage, and mean time to detect a defect are the three most important ones that I really track, outside of pass rates and everything else. And on the ROI perspective, this is really fun. And I know this gets maybe a lot of hate, but I also come up with a solution to understand how many hours are teams spending in quality assurance on the manual testing side, how many tests people think they can automate, and then they can choose the recurrency of, let's say, imaginary test automation. By doing that, you can calculate based on a basic hourly rate how much value you get out of infrastructure as QA Robots builds it. It is really simple, streamlined, the user can understand how much they're getting in and they're going out. And it's really simple to understand.

[00:24:45] Joe Colantonio Do you recommend individual contributors use these metrics as well? A lot of times testers like, I don't think my boss understands what I'm doing. Do you think they could report this to their management every month or every quarter to maybe help them bump up their visibility?

[00:25:00] Kristijan Plaushku That's a really good question. I really think that this will also help individuals just to justify the ROI. And of course, there is also the justification of the ROI on the actual monetary spend, but there is how the team reacts to it. So just imagine an infrastructure built on test information, which can be used by also developers to see their work and what bugs have come up in the environment, what progressions. I think it really just brings a sense of security in the whole team. So it's not just about money, just about the whole team feeling like quality is respected in their work. Absolutely, from individuals to companies, everyone could use this method and it's on firstprinciplesqa.com. You can just search it there.

[00:25:54] Joe Colantonio Okay Kris, a little plug for QA Robots. If someone's listening, who do you think be like, oh, I want this person to reach out to me because I can help them with this, this and this, which are ideal scenario.

[00:26:03] Kristijan Plaushku Yeah, I think what I would like to do and help as QA Robots as a company. I see a lot of opportunity into the industry 4.0 and 5.0 section and sector. I feel like robotics is a really great place to start. It's like we are the same field in two different ways. Robotics from a automation perspective in real life with real robots and we doing our work in a digital solutions. I think moreover the dashboards that needs to have proper validation can also benefit from test automation on the device. In a firmware, I see a lot of opportunity there and I would really like to work with clients that are in that sector.

[00:26:56] Joe Colantonio Awesome. All right, Kris, before we go, any parting words of wisdom of someone could do right away to implement to help them with their day to day automation efforts and the best way to find or contact you?

[00:27:05] Kristijan Plaushku Yeah, so for everyone trying to learn test automation, just remember that everything is about manipulation of the DOM. Anything that you have on the web, you can just extract and automate. Think of JavaScript, think of any language, think of any framework. It's just using that string as a method to perform clicks, scrolls, use. That's it. Just remember these principles whenever you're trying to see the test automation or learning into the topic and never get discouraged. You got it. And you can find me at LinkedIn at Kristijan Plaushku and QA Robots, I have the website link qarobots.ai.

[00:27:50] You can find all those links in the first comment down below.

[00:27:53] Thanks again for your automation awesomeness. The links of everything we value we covered in this episode. Head in over to testguild.com/a551. And if the show has helped you in any way, why not rate it and review it in iTunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe, my mission is to help you succeed with creating end-to-end, full-stack automation awesomeness. As always, test everything and keep the good. Cheers.

[00:28:27] Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation, testing, and DevOps. If you haven't yet, join our vibrant community at TestGuild.com where you become part of our elite circle driving innovation, software testing, and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate. Visit TestGuild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.

[00:29:10] Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold.