From Testing to DevOps – A Career Move That Makes Sense with Maciek Konkolowicz

[00:00:00] Get ready to discover some of the most actionable DevOps techniques and tooling, including performance and reliability for some of the world's smartest engineers. Hey, I'm Joe Colantonio, host of the DevOps Toolchain Podcast and my goal is to help you create DevOps toolchain awesomeness.

[00:00:18] Hey, are you looking to move from testing to DevOps? If so, this episode is for you. We have an awesome guest joining us. It's been almost eight years, I guess, nine years since he was on last. We have Maciek, who's a senior DevOps manager at an AI driven test automation platform with over a decade of experience driving software quality and delivery speed. He is passionate about SRE DevOps, best practices and continuous learning. He joined us as a speaker for one of our automation guild as well. So I'm really excited to get his perspective of why he made this move. I'm sure a lot of people have been thinking this as well, should they move from testing to DevOps? And what it takes. If that's you, like, again, you want to wait all the way to the end to find out as well. You don't want to miss this episode. Check it out.

[00:01:01] Hey, before we get into this episode, I want to quickly talk about the silent killer of most DevOps efforts. That is poor user experience. If your app is slow, it's worse than your typical bug. It's frustrating. And in my experience, and many others I talked to on this podcast, frustrated users don't last long, but since slow performance is a sudden, it's hard for standard error monitoring tools to catch. That's why I really dig SmartBear’s Insight Hub. It's an all in one observability solution that offers front end performance monitoring and distributed tracing. Your developers can easily detect, fix, and prevent performance bottlenecks before it affects your users. Sounds cool, right? Don't rely anymore on frustrated user feedback, but, I always say try it for yourself. Go to smartbear.com or use our special link down below and try it for free. No credit card required.

[00:01:57] Joe Colantonio Hey, welcome back to The Guild.

[00:02:01] Maciek Konkolowicz Thanks. Thanks for having me. It's a pleasure and an honor to be back. As you said, we've had a long history together. It's been kind of subsurface for a bit, but here we are. Thanks for having on.

[00:02:11] Joe Colantonio Yeah, you just open up another memory pocket. I think you'd join me to do a session on white, the white framework, correct?

[00:02:17] Maciek Konkolowicz Yeah, yeah, that was one session. That was when I was exploring a lot of different test automation frameworks. And that one was interesting to me because if I remember correctly, it utilized the desktop in a good way, which is at that time what I was really lacking from a lot test automation frameworks. That's why I use that. And it was a blip in my career, for sure, but a memorable one.

[00:02:42] Joe Colantonio Absolutely nice. So, I mean, last time we spoke, you were knee deep, feet first, I guess, into functional automation, maybe a little bit about your journey, why moving from testing to the world of DevOps, what happened? What was the moment that made you do this?

[00:02:57] Maciek Konkolowicz Yeah, it was a bit of a natural progression, to be honest. I never really meant to let go in testing. And if I'm being honest, it still has a near and dear kind of like part of my heart. But what really happened is an evolution of what I was doing. And I was really pushed along the way due to the shift left movement. I know this is still a thing in the industry that we talk about all the time. Moving testing closer to the development team, moving testing closer to developers. And that's kind of what I did, right? I went from being a tester that was essentially helping teams to test, then I became a little bit more of a senior tester that helped teams that advise teams how to test. Then I moved into a quality architecture position, which was advising an enterprise how to test. And then as I started advising teams more directly, I started noticing that enterprises were really focusing on training their testers to be more aligned with the development practices, which meant that they were either developing tests as first-class citizens or starting to develop code and tests at the same time, which really helped the enterprise that I worked for, Rocket Mortgage, or yeah Rocket Mortgage is still it's called Rocket Mortgage now it's called Rocket but that enterprise really focused on providing value from testers by utilizing their skills that they learned as test automation engineers in coding. All of a sudden found myself looking and advising folks who were asking me development type questions. And so what the problems I started solving for them really focused around automating their development quality related questions, which led me into the world of things like pipelines. And so from pipelines, I started noticing that there was a trend of needing help with that aspect. I started looking at Jenkins. I started at CircleCI, at Azure DevOps. I don't know if it's called Azure DevOps anymore, but that really led me into that world. And as I started lookin' at pipelines, I also started lookin at, well, how can I essentially ensure that the pipelines are behaving like I want them to and that the applications that are being rolled out by the pipelines are reaching the state that it is expected of them, which led me to observability. A little bit away from testing, but you could see where I'm leading with us. Once I got into the world of observability, I was like, holy cow, Grafana, for example. And so that's where I started really focusing on the DevOps perspectives. And I started being exposed to DevOps principles, SRE principles. And from there, it was just a continuation of what can I do to provide value for my team. And my teams were requiring value at that level because a lot of the functional testing was automated away with our teams, not from a bad perspective. There were people who were focusing on driving the right types of tests. But for me, I got to a point where I stopped having problems to solve around how to decide what to test, what framework to use for testing, but essentially that last piece of validation in production, which was observability. That's how I started looking into that. And that's why I started dipping my toes into that, and then eventually I got enough experience. That I got really excited about it. And there was a little inflection point where, as part of my QA journey, as part my quality journey, I also was a team leader at one point. And there a need for somebody to kind of step into a leadership position. I first did that kind of without any sort of like, like authority per se, like without the title or just kind of like leading by example. And eventually one of my friends who was a Team Lead with me in the QA world. Messaged me and started recruiting me to a startup. And that's kind of when I stepped completely into the world of DevOps leadership. That's a lot of words and a lot of like, look into what happened to me. But essentially it was an evolution of where to provide the most amount of value.

[00:07:00] Joe Colantonio Nice. It sounds like you learned on your own, on the job, and eventually morphed into being able to take on more and more responsibilities. What are those skills or mindsets you use to help you do that? Sometimes people get so, I don't know, they have blinders on almost rather than trying to feel out what's going on and become more and more valuable to their organizations, do maybe things that are outside their job title.

[00:07:23] Maciek Konkolowicz Yeah, totally. That binder perspective, that to me is like a safety zone. And I've always tried to operate outside my safety zones for better or worse. Sometimes it's bit many ass, sorry, and the butt, but usually it's been very, very positive for my development. And the one thing that, as you said, we last talked 10 years ago in my life, there have been many, many life changes in the last 10 years, specifically three little ones. That now ask me to change diapers and do all sorts of things that require my time and energy. So I'm glad that I align my change in energy output to the different stages in life. I learned a lot before I had to be involved with my kids. I had to. I love being involved with kids. I am involved with my kids, but essentially before there was a different set of an energy that I had expand outside of work and I learned a lot there and I kept my blinders off and expose myself to DevOps, expose myself to anything that anyone threw at me. And then I realized when I had my first kid that like the amount of energy I have after work or after hours has drastically been slashed which is a stage in life. It's neither bad nor good, but it is a state in life. I think the tip I have for people trying to make this switch that like I realized is you gotta pick the right time. And sometimes it's okay to say this is perfectly, I'm good at where I'm at and it's good to stay here. But as soon as you feel that urge, you can make the jump. From a blinders perspective, it's good to be good at what you do. It's good to have a niche, right? It's good to focus on one thing at a time, but it is also good to open up your horizon and know when you should make that switch or when you shouldn't make that switch. And that's kind of like the thing that I learned over the last 10 years is like I had the opportunity and I took it and I learned a lot. I took some risks. I went from maybe this is, it's worked out so far for me, but maybe it won't in the future. But I went for working to add to the second largest independent mortgage provider in the United States to a series A startup to a serious B startup. So like scary but the opportunities at smaller companies to advance into different, into a place where you have to wear many hats and learn is very, very lucrative and nice. The tip is It's fine to have, let's have an analogy. It's fun to have sunglasses on, not blinders. You're not blinded by everything coming at you at once and you can focus, but sometimes it's okay to kind of like look into the sunset and see what's ahead.

[00:10:06] Joe Colantonio Love it, love it. I might use that for the title. I guess what's kind of tripping me up, maybe I'm wrong. Isn't it Functionize an automation company, testing company? You were in automation, you moved to DevOps and then you moved to a company doing DevOps for a functional automation company. Am I getting that right?

[00:10:24] Maciek Konkolowicz That is correct. One of the big reasons I moved to Functionize is because of the subject matter expertise. Like I couldn't think of a better place to scratch my automation edge and provide value to the company as a whole on a strategic level through my experience and still learn a lot of DevOps stuff and be able to kind of like lean into the stuff that I'm currently doing because at this sort of company, I'm very comfortable in conversations dealing with what the tester needs or what the tester has felt or how it feels to run test automation or build test automation. And it's very interesting because we are using AI for the bulk of our-we have we have tools for the company sells tools to generate tests via AI. It's very, very interesting from like perspective of the client because I used to be the client. When we talk about products, I'm like, oh, yeah, I've done this for. I know what Cypress is. I know what Playwright is. These are our competitors. So it's nice to be there, but also be focused on the DevOps perspective for that company. So it was almost like I'm using my background to drive forward my future.

[00:11:38] Joe Colantonio What do you think if someone's in DevOps and they've never dealt with a testing team before, what does a tester need?

[00:11:44] Maciek Konkolowicz From a DevOps perspective?

[00:11:46] Joe Colantonio Yeah, so if you're a DevOps and you need to get them into the pipeline, no one knows how to do it. They don't necessarily know the functional testing background of automation that you do. Do they need to know certain things or have in place certain things in order for it to work really well?

[00:12:02] Maciek Konkolowicz The relationship between testers and DevOps is, in my opinion, it's similar to a developer in DevOps. DevOps and testers can, and I've been on both sides now. I was a tester and I worked very closely with DevOps teams. And I'm DevOps and I still work closely with testing teams. But I look at testers as a different type of developer almost. I'm a different kind of client, but not drastically different. So like a testers, a tester from a DevOps perspective, it requires things that allow them to do their job well and efficiently. And from a developer perspective, it's similar except developers value things like getting their code through the pipeline quickly. Testers value things like making sure that their tests run quickly and provide feedback to them in a way that is easily consumable to them. All right, I think that the thing that the DevOps team should know about testers and should really focus on is what matters to the tester, what outcomes matter to the test, or just like what outcomes mattered to the developer. The developer might want to ensure that their code is shipped to a specific environment, whereas a tester wants to know that the code was shipped to the environment while satisfying their quality gates, for example. I think it's a partnership and it has to be a partnership between, it's almost like, like, do you remember the three amigos? For back in the day, right? So it's like the modern version of the three amigos where your testers provide requirements in DevOps of things that they want to see in the pipeline and developers do the same. And then the DevOps folks try and satisfy those requirements and work with both teams to ensure that their expertise is reflected in the pipelines. That's one piece in the Pipeline. I mean, from an observability perspective. One thing that is really interesting, what I've noticed is that testers gain a lot of visibility due to the observability that DevOps provides. While we have different functions in big corporations, like operations, that operations will look at the traditional observability dashboards that look at, let's say, the golden signals, like saturation, latency, stuff like that. But nobody necessarily looks at the functional output, or even business dashboards which kind of tell us whether the behavior of the application is what we expect it to be. Our role as a tester is to kind of drive that, right? To ensure that what our requirements are set as are reflected in when the application's built. And so the end part of that journey is to watch when it's rolled into an environment, whether it's staging, production, whatever. And so from a tester perspective, the DevOps person can provide them with a lot of value by showing them or helping them build this type of observability, which may not be traditionally focused on non-functional requirements. It may also be. But it doesn't have to be, because we can, as testers, we can ask for things to be customized, and observability to be built for custom types of behaviors from the application that we test beforehand and expect to continue to see once the application is shipped. So that's a lot of rambling, I think. But in general, I mean, the partnership between the DevOps teams and testing teams or the testers themselves, if they're embedded inside of teams, is very fruitful because we can benefit from each other's expertise.

[00:15:28] Joe Colantonio Nice. So in there, you did mention the company is AI-based. Does that change a DevOps strategy? Like, how do you know something that's non deterministic almost to know that it's in the pipeline and it's working correctly before you say release it?

[00:15:44] Maciek Konkolowicz Yeah, it's tough. I think the best way is, again, looking at the outcome. And we have specific internal signals that we measure that once they start rising above what we expect. For example, a lot of the AI that we build satisfies a very high rate of element match for the automation framework. That's a bit of our secret sauce because we have models that say, okay, your page is loaded. We know this to 99.999% certainty because of the data that we've gathered over seven years. And by comparing the data that we gathered seven years over the many different inputs we have to this model, we can tell that the page is loaded. Therefore, it's safe to click on an element. The next model will take over and say, hey, the element that you're looking for traditionally was here. It moved over to here. We think this is the right one. We've seen this many, many times before. Therefore, here's your element. Go click on it. Because of the data that we gather, it is non-deterministic to an extent, but from a different perspective, the percentage chance that it is not going to be matched is relatively low. And so we eat our own dog food, we test our own stuff, we're using our own products. And so, we are able to feed our tests to our development as it happens and use our software to ensure that it is being developed to the highest of quality. It's a bit of a loop, a reinforcing loop. And it's not perfect, but it's a pretty high rate of confidence.

[00:17:22] Joe Colantonio Nice. If someone's listening and they're a tester, is there a small steps they can do to see if this is the right career move for them?

[00:17:29] Maciek Konkolowicz Yeah, definitely, ironically. Recently had somebody from within our company asked kind of a similar question where they reached out to me and said, I'm really interested in this. What do I do? I think I'm still trying to match them with the proper meetings without firehosing them. But essentially, what I would suggest is maybe start shadowing the DevOps team. There's a lot of great books and resources that will reflect to the people interested in how to make this career change. But at the end of the day, it's little tiny steps, but the steps have to be aligned with value, 's one thing to learn something, to experience something without adding value to either your own perspective or the company you work for perspective. But if you find an opportunity that nobody else is fulfilling, that is a small step. Like maybe for example, generating a dashboard or implementing a standalone Grafana server or launching a Docker base, or launching a Docker based Prometheus server or something like that. Those little steps that may not be little to some people I understand that might be daunting but anything which aligns value that your team is asking for and how you can provide it that would be a good first step. That's kind of how I started doing these things because we started seeing that there were gaps. The one thing to mention in that growth pattern is make sure you identify that you are growing and that you're venturing into something you don't know to your team. There are no surprises. Or the team thinks that you're an expert. And if you do make a mistake, you have a backup. You have an established kind of like, I don't want to call it a CYA, but it also is just being transparent and honest that you are learning.

[00:19:09] Maciek Konkolowicz You're also in a unique position once again, as we mentioned, because you're working for a functional automation tool that uses AI and some testers might be looking into other areas to move into because they're afraid that AI is going to replace them. For being from the DevOps lens, looking at your testers at a tool that makes AI based testing, should they be worried about their careers that way? Or do you just see it changing or morphing? What are your thoughts on it?

[00:19:34] Maciek Konkolowicz Yeah, I mean, I will be honest and then kind of recognize and acknowledge the fact that it is scary. There's a lot of push into AI and there's a lot of this kind of fear that it is going to replace some testers and replace not just testers, but developers. AI is supposed to take over the world. At the end of the day, maybe it will, maybe won't. The one thing that I've seen that is very true is that you have to find a way to utilize the UI to make your own process either more efficient or faster or better. And the AI in general is only as good as the data that it's trained on. And so if you're in a position where you are a tester that knows a lot about the business context of your subject matter, that has a good position to be in and you can even utilize AI to make that, use it to your advantage. For the small tasks, like I use Copilot in my development all the time. And I don't fear that it'll take over what I need to do, because I understand that the more I use copilot, the more time I have to contextualize my learnings. And that is something that copilot doesn't have access to learn, because it doesn't have access to the internals of our application or business plans, stuff like that. Human will always be better, in my opinion, judging context, at making snap decisions without previous knowledge and tuition, that type of stuff. It's definitely a nervous world to live in if you've been performing a task that other people are telling you that is going to be replaced. But it's your choice to either kind of dive into it and use it to your advantage or pretend it's not going to happen. And I have no idea if it's going to happened or not. I mean, I work at an AI startup, so I should be telling everybody it's just a but at the same time, there's a lot of nuance to those discussions. And so it is a scary world, but it's up to you. How do you take that? How do internalize and what you do with it? I just do it once up at a time and figure out how to use the AI and when it's applicable to it and what it is applicable to for your context.

[00:21:47] Joe Colantonio Love it. DevOps, obviously, are there any hot trends that you're interested in or you see on the horizon, AI, I know AI has impacted everything. I don't know about how it's impacted DevOps, but like MCP servers, does that have anything to do with DevOps, what you're doing? Does that do you care about that?

[00:22:04] Maciek Konkolowicz I operate a lot in GCP, so in cloud-based technologies in general. And so I've seen AI. So for example, when you log into, as a DevOps person, you log in to a cloud console, and right away, there is Gemini or whatever the AWS or Azure equivalents are telling you, click on me and I'll send you a report. I'll show you a report. And one of the core things that I pay attention to is cloud costs as a manager at a small company. And I don't know how many times I've generated a report and been like, this is totally not what I expected or want to look at. And so it's there, it has, and this is Google's AI on Google's cloud platform. There's a lot of stuff which AI is good at, but the contextual pieces that I sometimes need, it's not that good at yet. So I'd say that the trend there is evolving, but it is not mature enough to be used every single day from a perspective in the cloud, for example. However, what it is good at are simple tasks like, if you ask, I may or may not have done this a few times, but if you asked ChatGPT or Gemini to translate a file that is in a slightly messed up YAML format into a Terraform file, it can do that in a second. So that type of stuff is what I use AI for to satisfy those mundane tasks that I'd have to go and do on my own. That's one trend that I see a lot of my developers, a lot my DevOps folks doing too. But the things I think which are still hot to us at a startup are things that allow us to see a bigger picture perspective of our infrastructure and our applications in an easy way. And I haven't seen AI be able to do that well yet. A couple other trends which I've seen that I've been bombarded with in the last year or so are cost savings for cloud spend. Like automated cluster turnoffs and downsize and stuff like that. And that again is if you're running a simple environment, if you have a very vanilla setup, it's pretty good at that. But if you running a simply environment and you need that type of tweaking, it just means that you haven't done it yourself and I bet you could get it accomplished quickly yourself anyway. Anywhere where there's manual intervention that has been done by a human, there is a footprint of AI. But I haven't really seen a super great success yet but for the simple things to make your life a little bit easier, like translating human definitions and stuff like that, which could probably apply to test cases into different frameworks or generating test cases, that's something which we actually do as well. We offer an AI-driven test GPT agent to translate test cases from natural language to a functionalized test, which is really cool. Those trends, like that type of simple translation, we still have to guide our tests, our framework, in a little bit of a manual way. But it certainly makes the lives of our customers easier and makes my life easier too. There is definitely some input from AI, but it's not like if somebody tells you it's just like, it's going to head out of the park, it's gonna take over, you don't need to be here. I think that's a lot of baloney.

[00:25:27] Joe Colantonio Absolutely. All right. Pretend you're a DevOps hiring manager. You get a resume from a tester that wants to move into DevOps. What are you looking for? Skills. Tools. What pops for you?

[00:25:41] Maciek Konkolowicz I think the thing which naturally I expect from a tester is a high level of curiosity. So that is one thing that I would definitely look for and also what I would look for and I expect for my tester. So in general, when I was a tester, I learned new context all the time, right? In order to be a successful tester, what I had to do is really understand the application I was testing, really understand the team dynamics of who did what and figure out who to go for to write successful test cases. And get the proper context. So that sort of learning experience really benefits a DevOps person as well, because the troubleshooting that comes with DevOps is kind of similar to finding out why a test would fail because you're essentially debugging. So I would look for folks who have pretty extensive degree of learning different types of projects, not necessarily different types or many different industries. The industry aspect is interesting to some people if you're dealing with a lot of like laws and different types of certifications like HIPAA, privacy type things. But overall, I think the level of learning and experience showing a high level of learning is very attractive to me as a DevOps hiring manager. And if I were to be looking at a tester, I think I would look for somebody who was involved, directly involved in that team. As opposed to working on a QA team. That's where like that transition, that's what I did. So based on my experience, that's like near and dear to my heart. As a single tester, you get to be exposed to the developers really closely and you start seeing the types of problems, running into the types problems and in the testing road that you run into DevOps as well sometimes.

[00:27:27] Joe Colantonio Awesome. Okay, before we go is that one piece of actionable advice you can give to someone to help them with their DevOps efforts and what's the best way to find or contact you?

[00:27:35] Maciek Konkolowicz So I think the best way to contact me nowadays, I'm not as active on X or Twitter as I used to be. So just email me, that's probably the best way or add me in LinkedIn. And I think you could provide that contact information in the show notes. And the best piece of actionable advice to give to somebody who's looking for, to dance or DevOps experience as a tester is start to shift left and try and provide your developers with a little bit more speed and DevOps type questions or answers.

[00:28:06] All right, before we wrap it up, remember, frustrated users quit apps. Don't rely on bad app store reviews. Use SmartBear's Insight Hub to catch, fix, and prevent performance bottlenecks and crashes from affecting your users. Go to SmartBear.com or use the link down below, and try for free for 14 days, no credit card required.

[00:28:27] And for links of everything in value we've covered in this DevOps ToolChain show, head on over to testguild.com/p197. So that's it for this episode of the DevOps ToolChain Show. I'm Joe, my mission is to help you succeed in creating end-to-end full-stack DevOps ToolChain awesomeness. As always, test everything and keep the good. Cheers.

[00:28:50] Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation, testing, and DevOps. If you haven't yet, join our vibrant community at TestGuild.com where you become part of our elite circle driving innovation, software testing, and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate. Visit TestGuild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.

[00:29:33] Oh, the Test Guild Automation Testing podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread, like wildfire through the land. Guiding testers, showing them the secrets to behold.