Learning Journeys for Testers with Mark Winteringham and Richard Bradshaw

[00:00:04] Get ready to discover the most actionable end-to-end automation advice from some of the smartest testers on the planet. Hey, I'm Joe Colantonio, host of the Test Guild Automation Podcast, and my goal is to help you succeed with creating automation awesomeness.

[00:00:25] Hey, it's Joe, and welcome to another episode of the Test Guild Automation Podcast. Today, we'll be talking to Mark Winteringham and Richard Bradshaw, all about Learning Journeys for Testers and all other kinds of Ministry of Testing awesomeness. If you don't know, Mark is the Dojo boss at the Ministry of Testing. He introduced a lot of aspects of software testing, and he supports and curates testing training from others in the testing community. He also is the author of, what I'm calling the number one book on API testing called Testing Web APIs, published by Manning. And I'll have a link to that in the show notes. So you definitely want to check that out after you listen to this all the way to the end and special, we have the boss-boss himself joining us, Richard Bradshaw, a.k.a. the Friendly Tester from Ministry of Testing. Richard is a very experienced tester, a consultant, and generally a friendly guy. He shares his passion for testing through consulting, training, and giving presentations on various topics related to testing and automation. And so you probably heard of him for sure and have seen him at multiple conferences. He has a bunch of experience. Really excited to have the two back together again on the Guild to share their experience and to go over what I think is a really unique development in the testing space that I think you all need to know about, so you want to make sure you listen all the way at the end to learn how you can up your learning journey as a tester. You don't want to miss it. Check it out.

[00:01:45] This episode of the TestGuild Automation Podcast is sponsored by the Test Guild. Test Guild offers amazing partnership plans that cater to your brand awareness, lead generation, and thought leadership goals to get your products and services in front of your ideal target audience. Our satisfied clients rave about the results they've seen from partnering with us from boosted event attendance to impressive ROI. Visit our website and let's talk about how Test Guild could take your brand to the next level. Head on over to TestGuild.info and let's talk.

[00:02:18] Joe Colantonio Hey, Mark and Richard, welcome back to the Guild.

[00:02:22] Mark Winteringham Hey, Joe. How's it going?

[00:02:24] Richard Bradshaw Hey, Joe.

[00:02:25] Joe Colantonio Great to have the super friends back together. I think that was might have been the last episode we did. That was years ago. So great to have you both here again.

[00:02:31] Mark Winteringham Yeah. I was wondering if you remember the super friends thing.

[00:02:33] Richard Bradshaw We're pretty much married now, right?

[00:02:37] Mark Winteringham Yeah. Testing spouses now.

[00:02:42] Joe Colantonio So it has been a while and I usually botch bios. Is there anything I missed in either of your bios that maybe you want to share more with that you've been working on or anything I may have missed?

[00:02:51] Mark Winteringham I suppose, the big thing for me.

[00:02:53] Richard Bradshaw Mark's got a new job.

[00:02:53] Mark Winteringham Yeah, I'm now the ops boss at Ministry of Testing.

[00:02:59] Joe Colantonio Ops boss.

[00:03:00] Mark Winteringham Much more so kind of focused on sort of helping the team in day-to-day operations and helping Richard get his vision actualized. Oh, such horrible work.

[00:03:11] Richard Bradshaw Keep things moving is how I view it, Joe. It's a thousand things you do at the Guild, right? Trying to keep all those things moving.

[00:03:18] Joe Colantonio Yeah, for sure.

[00:03:19] Richard Bradshaw That's Mark's job now. I'll do ideas, Mark keeps everything moving.

[00:03:23] Joe Colantonio That's great.

[00:03:24] Mark Winteringham Yeah, well, we've done it with AIT, so we thought, let's do the same for the Ministry of Testing as well. Seems to work well.

[00:03:31] Joe Colantonio Yeah, I got an operations manager. It's changed my life, so I'm sure.

[00:03:33] Richard Bradshaw That's exactly the same role. That's it.

[00:03:36] Joe Colantonio Yeah, exactly.

[00:03:38] Mark Winteringham Yes, that's pretty much now.

[00:03:40] Joe Colantonio A good move.

[00:03:40] Richard Bradshaw And not much has changed in my area apart from I spend half my week as Ruby on rails developer now.

[00:03:46] Joe Colantonio I would assume that. Why is that just to keep up, to keep your hands dirty or?

[00:03:51] Richard Bradshaw Few things really. Number one, the website needs a lot of love, and budget-wise, still recovering from COVID and that kind of situation. Number one I can code and I love coding and my day-to-day job doesn't involve coding really. So I thought, you know what? I'm going to jump back in and do some building. So I pair with other boss quite a lot of the week now and get to code. Obviously, it's not sustainable. It shouldn't be. I don't advise CEOs to spend overtime coding but right now it scratches an itch and it helps the business So yeah.

[00:04:24] Joe Colantonio Awesome. Very cool. So I think, I like to set the stage. I just assume people everyone knows who the Ministry of Testing is. And I was kind of surprised when I actually caught up with Mark at the Selenium Conference this year and our booths were next to each other. I was eavesdropping, as I always do.

[00:04:39] Mark Winteringham Yeah, we saw you.

[00:04:41] Joe Colantonio There are a lot of people saying, What is the Ministry of Testing? I know people who don't know what Test Guild is but Ministry of Testing. I just assume everyone knows the Ministry of Testing. And so before we even dive in, maybe let's start off there a case, someone's just listening in for the first time. What is the Ministry of Testing?

[00:04:56] Richard Bradshaw Wait and it see it Mark. It's really hard to explain it. So we basically call ourselves a professional community or community of testing professionals, but really it's a community that's open to anyone who has to do software testing as part of their role. We specialize in raising awareness of software testing skills, techniques, companies, and events, and we do that via website, our platform. We do it via conferences around the world. Our conferences test bash, we have online conferences very similar to the ones that you do yourself at the Guild. We exist. The motto is co-creating better testing. So that's why we exist. We want people to be doing better software testing and no matter what their job role is, and we exist to bring everyone all the best awesome stuff that they can place out there in the industry.

[00:05:46] Joe Colantonio Nice. And you'll have been around for a while though, right? 2005, 2006 maybe, I'm assuming. I don't know I remember seeing you really early on.

[00:05:54] Richard Bradshaw 2008 for the software testing club and then the Ministry of Testing, I believe was incorporated in 2011. So yeah, the Ministry of Testing had its 12 years so yeah it just rebranded and it was two businesses and we renamed it. That was before my time as well so but yeah, nice thing going strong and like I said, I'm really happy we got recoded and all those issues. So yeah, trying to push on them.

[00:06:18] Joe Colantonio Mark, how did you get involved in the Ministry of Testing?

[00:06:21] Mark Winteringham Similar way, Richard I think actually like we met at one of the first RSTs, Rosie. Who is the founder of ministry testing she puts on and there was some very friendly non-bearded guy who was quite chipper for 9:00 in the morning. Now that I know a lot about you, Richard, you clearly were just having a good time. We met like briefly there. But I think for the both of us, which kind of feeds into all this learning journey stuff that we're doing, we both kind of went off, lived our lives, upskilled ourselves, came to realizations, started blogging, started talking, speaking and doing workshops, and then through those conferences and stuff, we met and we start doing our automation testing stuff. And that was Richard moved over to boss boss role around that time and I was doing software testing clinic with Dan Ashby, and Richard basically approached us and said, bring the two things in more closely. I was already doing loads of courses by that point, online courses and things like that. So when the opportunity of the Dojo boss role came and it was someone with domain experience but also the production experience, yeah, I jumped at it. I've been with it now for almost five years, like the longest job I've had for quite a while. So yeah. And then I moved to the Ops boss stuff two years ago, two and half years ago, I think.

[00:07:46] Joe Colantonio Awesome. So it's great. Everyone noticed that you both start off as hands-on testers. You both are still hands-on testers, so it's a community by testers, for testers similar to Test Guild as well.

[00:07:57] Richard Bradshaw Yeah, absolutely. We both and that's one of the reasons why I mentioned doing the Ruby on Rails stuff because it's keeping me, I'm testing my own work and I am making so many mistakes that we used to catch as testers how did the developer do that? And now I'm doing it all day, every day. But the website, so much of yours, it gets tested by testers. So as soon as we make mistakes, people come and tell us so.

[00:08:22] Joe Colantonio Mark originally contacted me, said, Hey, we have these new learning journeys and want to dive into. So I was like, okay, that's all he told me. So I'm just curious to know, Hey, I don't even know what a learning journey is. So what is a learning journey?

[00:08:33] Richard Bradshaw It's a great question and it's one we've been trying to practice as well. So essentially it's like a learning path. We're calling them journeys because the way that these new learning journeys to set up for me and Mark are basically taking you on a journey of learning. So the learning journey is based off our open-source curriculums. I believe that Mark spoke about as well at the Selenium conference that you mentioned earlier. So the learning journeys are backed off the open source curriculum, which is the automation engineering testing. They're off the back of that curriculum which over 140 people contributed to. So we built that curriculum as a community. We did it all in the open, all in the public, and that curriculum is now open source. So anyone can take that curriculum and implement training against it. For example, the Guild could do it. Other certification bodies could take that curriculum. And the idea is to have this living curriculum, the MOT will be sustaining. And then the learning journey is our implementation of that curriculum. And so it's over 125 lessons. I think now, Mark, it might be more than that.

[00:09:36] Mark Winteringham 142.

[00:09:36] Richard Bradshaw 142, Okay. So it's significantly bigger than of course, you could arguably view it as 12 or 14 courses in one. And the reason why we call it a journey is the whole educational bit of it is narrated by me and Mark. Me and Mark, weaving you through and this is one of the unique things of it, weaving you through existing content. So that could be a Guild talk, it could be a YouTube video. But we're going to send you to that video and explain why we want you to watch that video. And then the next bit of the lesson might be some original content by myself and Mark. So we want to start reusing content that exists in the community because there's so much good stuff, yet we're always reinventing the wheel. So the idea is that me and Mark, take you on these journeys based off that open source curriculum. And yeah, like I said it's 140 odd lessons long and it takes you from absolutely nothing to all the way to go through context, strategy, goals, plans, picking tools, implement in the tools, maintenance, using that to use it, utilizing automation and yeah, the whole shebang.

[00:10:43] Joe Colantonio Where can people get their curriculum? It's open source is on GitHub?

[00:10:47] Richard Bradshaw It's not. It's on it's currently listed on the club, but we learn there is a link from the learning journeys to it and hopefully, in the coming months we'll have a proper curriculum page on the MOT website because we're also currently building the junior software tester as well. So that's the next one that we're doing in the process.

[00:11:04] Mark Winteringham Yeah, so it's so big, it's sort of trying to and because another aspect as well is that we want to encourage feedback and maintenance of this open source curriculum as we progress, as the industry changes, as new people come to the community and share things, we want to encourage people to give feedback and help us update and tweak things. We want to make sure that it's laid out in a way that's sort of kind of engaging and sensible to kind of interact with as well. So that, yeah, that's what we're sort of trying to get done both. So we're trying to get 40 other things done as well with two devs. So yeah, the hope is that we'll have that sort of in a nice form. So but yeah, as Richard said, it's on the club, we've got it all sort of available for people to look at and stuff. I'll share links with you and we've put it in the show notes.

[00:11:52] Joe Colantonio What inspired you? I know in the test community, there are a lot of people that are like against certification saying it's garbage, but this sounds like something that it's not a cert, but it's more like, I guess it could be like a cert, a curriculum if someone applied, like you said, Richard. So where did this come from?

[00:12:08] Richard Bradshaw Yeah, well, if I asked you the question of, Do you know who's behind the ISTQB Foundation? So do you know the people who design that syllabus? Most people probably don't. I can tell you it's a bunch of people that aren't very active in the industry anymore. They're very active within the ISTQB space, for example. But in the wider industry, not so much. So the idea was to start having curriculums that are built by the people doing the work. And we happen to have expertise of Ministry of Testing with Sarah Derry our learning boss who knows how to build curriculums. So we said, why don't we build an open one that's clearly visible, can be hopefully become kind of the standard for the industry, for teaching against. But yeah, the idea is that it's just more open. It can be critiqued, it can be improved, and it's a living certificate, sorry, living curriculum. And I think to me that's just more appreciative of what the industry is and what our communities can create. And I think as a community, there's always been little secrets or passing around of this knowledge. You need to read this blog, you need to watch this thing, you need to do this, you need to do that. But it's never been curated. And the idea here is to put a little bit of a formal process on it, make it public, make it open source, and then hopefully we can start to produce more structured learning. I think that's one of our goals as well, like our learning journey, we believe it's very good, but also a lot of people don't know how to teach, so they have a lot of knowledge to share, but they don't know how to teach. They could just pick up our curriculum now. It has very clear learning outcomes on it. So if you satisfy those learning outcomes, you will have created a very good piece of training material. Hopefully, to compete, but it is also very different. I think it's a start of having community-based learning.

[00:13:53] Mark Winteringham Another thing to add as well. Like this is sort of been something that's been rattling around in my head for over a decade now, although, it started in the software testing clinic stuff, which was basically a bunch of meetups that did free software testing training, and then we start building a curriculum. One thing I've noticed is that at that time that was when Richard and I would start to do more of our automation testing training. I was doing other training with my API stuff and I realized that the more I was doing that, the less testing I was doing, the less industry stuff I was doing. And I think that's always going to be the curse of being an instructor, not necessarily a curse, but the paradox of being an instructor. The more you're teaching, the less you're doing. And our industry changes so quickly, it's hard to keep up with it and stuff. I already see some technologies coming out and I was try and dabble, but it's sort of a case of you can blink, and miss it before something happens. So by developing it in this way, that open source is it keeps the curriculum relevant because there are no personalities behind it. Like Richard said, it's not about an individual or a group of individuals behind it. It's something that is reflecting an industry that is very diverse, some people are working in cutting-edge, crazy Kubernetes, clustered machine learning madness in a start-up, and then someone else might be one of 100 testers in an enterprise company somewhere else. The experiences they have are going to be that the expectations of them are different. We collate all of that. We get some sort of kind of sense of what are the commonalities. So it's one thing to say it's what it's like the learning journey has to be generic somewhat to address a wider audience, but it definitely hits all of the important responsibilities for anyone working in automation regardless of your role.

[00:15:53] Joe Colantonio Nice. What I also like, I said this about the Test Guild as well as I think administrative testing is independent. So I mean, I'm not dogging sponsors. I love sponsors. I love thought leaders that work for sponsors, but still they have their curriculums. But I always think there's some sort of motivation where it's really just kind of a backhanded pitch for the product, which is great. We all need tools and solutions, but this to me seems more independent, so I might right with that as well. Where there's no one dipping their hand and it's basically community-inspired non.

[00:16:25] Richard Bradshaw Yeah, absolutely. Me and Mark arguably the one who's written it for each step of that process was reviewed by the community. So at every point, every week we did a module that went on the club and people critique to it. They questioned our language, our words, questioned the things were missing and we said, okay, well, what we were doing will include that. So not to set this over 120 people that have contributed throughout the whole process, we did it on demand, so you could just log on and help. We also run, I think, over 15, 16 events where people got together and stormed ideas and started, we were saying, right, that we need a module on creating an automated check. How the hell do you do that? And then everyone started throwing down their steps, and then we added our steps and then we basically mixed the match until we got word. So that's the idea is that is independent of me and Mark, the actual curriculum. If you look at it, it says organized by Ministry of Testing. But these are the awesome people that have done all the work. And in the future, when it's on the website, all those awesome people, it will link to all their profiles and you can actually see who they are, who is that person? And you can click on their profile, and they'll be, Hey, they've been tested for 12 years and they currently work at wherever, and they've spoken at Test Guild and spoken at this conference. All right. Clearly, that's someone who I can relate to. That's kind of the difference. And as I said, we just wanted to be the curator. We just wanted to build the framework, and manage the process, bit like is talking about ops at the start. But really it's the community that actually done all the actual learning, set all the learning outcomes that are in there, and lists the skills that they believe people need to do the job and they are the ones doing the job. So if they're telling us what they do on a day-to-day basis and we put that into our curriculum, the chances of you being able to get the job doing that is more likely. And that was one of the key things as well, Joe, that we forgot to mention. These things are designed for you to get a job. If you take our learning journey, you should be at the position at the end of it to be able to apply for that job or move into that job or get promoted into that job. It's not just teaching you a little bit of the puzzle. It's the whole thing to have the job of an SDET, or an automation engineer in test as we went with.

[00:18:40] Joe Colantonio How can you say that? Did you have like managers or hiring managers look over the curriculum, say, yes, if this person actually did this, then?

[00:18:46] Richard Bradshaw We did, yeah.

[00:18:47] Joe Colantonio Okay.

[00:18:47] Richard Bradshaw We did. Yes. Well, that Mark tell you.

[00:18:51] Mark Winteringham The initial phase was we looked at job roles. Just basically scouring job sites and looking at the roles and responsibilities. We interviewed automators. We surveyed automators. Same with managers as well. And then, yeah, we went to the recruiters as well and spoke to them because there's really interesting like there's definitely different perspectives of what's going on and what's required. The recruiters are probably more explicit in what they're asking for and what they're looking for, whereas on the other side, the managers are more like, I don't really care what tool you use, I just need these problems being solved. So yeah, we got all these different sort of types of perspectives as well, which was very interesting and like just as well to go back what we're about, the learning journey itself will be opinionated a little bit, but actually it's not necessarily we're going to say you have to use tool X. tool Y, or tool Z, and it's much more sort of think about things on this layer, think about what sort of tools work for you and your context in that way. But what's helped with that is the fact that because we've set this curriculum out and we've identified outcomes, we're not arguing as a community about what things mean. We're not sort of trying to decide what automation is. What we're trying to say is everyone should have an understanding what it is. And we found if we all agree, like what the roles and responsibilities are, that's an easier conversation to have than what does testing be like? What is the definition of testing? Because then that's where things can slow down. It goes back to what Richard was saying is like other people could take this up and they could have their own opinionated version of it as well, which works really well.

[00:20:42] Richard Bradshaw The key thing about the tools is really important because what we're doing in our journey, so we're doing C#, JavaScript, Java, and Ruby to start with, but the tools that we're deciding to show you in our learning journey is we basically just ask the community what are the top tools people use for Java to do UI, to do API, to do JavaScript. Where we're going to use those tools, but because the curriculum is open source, if Joe, you messaged me and said, Oh no, I absolutely love this tool and you need to add this tool to the learning journey, I could just go to you and go, Well, here's all the learning outcomes, here's all the code examples that we've built. If you're willing to build them all in tool B, we'll add it to the learning journey and then people can basically pick and choose what flavor of the learning journey they want to go through. So our plan is to teach one or two, but then have a link to other tools if they want to use this particular tool.

[00:21:34] Joe Colantonio Nice. So just so people and so the curriculum is open-sourced, it's free.

[00:21:39] Richard Bradshaw Yep.

[00:21:40] Joe Colantonio But the learning journey is the implementation of the curriculum based on your input, which is then is it paid? How does that work?

[00:21:46] Richard Bradshaw It is, yeah. It's paid. It's a one-off payment to get the learning journey. Right now it's super cheap, I think depends when this goes out, but it's some of the round around the £400 mark and that is a lifetime payment. As the curriculum evolves, we'll update the learning journey to match the curriculum. So it's a lifetime access basically to that learning journey. And yeah, it will go up to somewhere around. I can't remember exactly, but somewhere around 1300, 1400 pound mark once it says full price, which is still cheaper than most three-day training courses and boot camps and things like that.

[00:22:24] Joe Colantonio When you go to the learning journeys, I know, like you said, you have something on automation engineering, testing C#, Java, JavaScript, and Ruby. So it's all automation testing focus. So are you saying all testing is automation testing, or will you then have certain testing paths? Or do you just assume that if you're a tester, you're going to need to be trying to get some controversy here to be an automation engineer?

[00:22:45] Richard Bradshaw Yeah, not at all. We basically, we've set out four or five curriculums that we would like to do. But as you know, running the Guild yourself, you try and pick the one that's going to appeal the most. So we start it with automation, but we obviously, as you know, me and Mark talk about automation and testing. So we went with automation engineering testing as the title for that. But you could easily call the SDETs automation engineers whatever. The one that we're currently, I'm guessing about halfway through mark, halfway through the process for junior tester. So that one will have probably the closest rival to something like ISTQB Foundation or something like that. So this is basically, an entry-level role into testing. And then we're going to do testing for developers and managers probably is the next two after that. So there'll be a lot of overlap between all these curriculums. Obviously, imagine some modules will just lift and shift from one end to the other. And as we said, it's more about the idea that we've gone for, and what Learning Boss suggested was that we focus on roles because that's how people get jobs, that's how people get paid. So we're trying to frame them around roles. So junior tester, some sort of automation-wearing hat. Obviously, developers are doing far more testing than they've ever done, and have always done a huge amount of testing, but explicitly saying, I'm doing some testing. That's happening more and more and more and a lot of teams are obviously shifted into that way as well. And then obviously managers, that kind of aspect of the top down looking on all these types of different testing that will be the probably the one we will do after that. But yeah, that's why we started. We've definitely not seen everyone needs to automate. However, the junior one does have an automation module in it because everyone needs to know what's possible, so they don't need to know how to do it. But I would say everyone needs to know what's possible. Who does the work? Context dependent. But not knowing the automation is the thing that's going to give you less options. It's probably going to hinder you. Whereas if you're aware and we talk a lot in AIT about awareness, if you're aware, you can have more options to pick from.

[00:24:48] Joe Colantonio Nice. I know the Ministry of Testing also has a pro plan. This is different from that as well, correct?

[00:24:52] Mark Winteringham It is, yeah. Yeah. So as Richard said, it will be a standalone thing. We may package pro in the journeys up in the future, but at the moment we're keeping them distinct because we're doing some work on Pro as well to sort of kind of elevate that as an offering as well. So yeah, it will be a standalone thing. Yeah. So at the moment, it will go up to full price in autumn. But the idea is yeah, because it's such a big thing as well. It's taking us months to build, so it's going to take months to go through it as well. I think it's something that's made sense to keep it distinct because it's something that it is, that because it is a journey, it's something that you're going to follow along. Something you may treat moments like as a reference guide as well. It's funny, like talks about like writing, testing, Web APIs. I see this as another book in my mind, and that's partially because it's mixed media. So it's not all videos, it's video, it's text, it's audio, it's code examples, it's graphics, It's test bash talks, it's conversations. It's that sorts of stuff. But the way that we've had to plan it, the way approach it and I think it had to be built in the same way as writing the books and knowing what I know about these books now. And the process of that is that some people will read it end-to-end, some people will dip in. Some people will have specific problems that need to be solved and they'll go to that path, do that, and then go, Oh, that's interesting because that's something that's connected to this bit and then moves up as well. So the whole thing is kind of sort of so in that way. So yeah, it makes sense to keep it distinct from it.

[00:26:27] Joe Colantonio It's quite an undertaking. I wrote a book in 2013 an API testing using QTP, right? And I used examples using an API that was no longer available. And so it had to be updated. Very difficult to update, I'm assuming, with these learning paths, then it's going to be a lot like you could create something and then four weeks later it's a month later, you're like, Dang, I have to redo it. It's changed. Are you going to be updating or are you trying to make it as evergreen as possible?

[00:26:52] Mark Winteringham We will be, yeah. Yeah. We may regret these words in the future.

[00:26:56] Richard Bradshaw This is the current place we are right now.

[00:27:01] Mark Winteringham Yeah. So we're currently. So we're building a test app to go with the learning journey of which we want to build it in every language as well. So I'm building the front end at the moment. We're thinking something like sort of like basic timesheet manager or something like that, it might change. I might. It's my mind. I've been at it for a week already. Then the idea is, is that there will be a different back end. So if you're in the C# world, you'll probably be building and working with something in .Net space. I'll probably build something that's really good. Richard gets to play with Ruby, which makes him happy here, and then I'll try to do something in Node.js. So we've got that. And then, we have the automated checks that we've got and the tools and stuff and the examples that go with that. We will have to maintain those things, make sure libraries are updated. But again, this is where the community comes back in because the community will feed in like what are the new tools, what are the new approaches? And again, the aspect of the paradox. But as a trainer, I probably won't pick up on these things because I'll be too busy traveling and trying to sell training and deliver training and stuff. Whereas, we plan to set up this sort of kind of cadence of people telling us of these new things that are coming along. Playwright didn't exist really, when we started the curriculum. It hadn't been released. Those sort of things that kind of happened as well.

[00:28:23] Richard Bradshaw We were laughing at the start, Joe, because it's we've been battling what to do, but we want to practice what we preach. So if we're teaching people to automate on every layer and you're taking the C# course, yet the test stack is written in Java. Unit testing suddenly becomes complete a car crash for you. Whereas if the whole app is written in C# and you can run it all in VScode and you can stick it on Azure or wherever, right, that's more realistic of someone in the C# domain. In a while, we ideally wouldn't want to build for apps, but what we can do now is we can build for apps, we can put all the tests on the same pipeline because pipelines are part of the learning journey and the curriculum and actually have a running pipeline that will run every day and it will tell us if the checks fail, it will tell us that dependencies are there that we need to update, which is exactly what we teaching people in the learning journey. When we're talking about maintenance, we're talking to people about libraries, updating the automation libraries, but also the actual solution libraries for your product. All these things are updating all the time and you need to be aware as an automation engineer that that might cause brittleness or failures. Therefore, we're going to build pipelines exactly as we're teaching for this test product with all the relevant checks and also the learning journey, techstack is completely custom we build to ourselves. Within that as well, it'll be very easy to tell us that blocks of code aren't working anymore and they'll be able to automatically tell those and then we will get alerted so we can go in and fix it. The last thing I want, and it's a thing that happens a lot with courses, is for these code examples to quickly go out of date. While they might be opinionated because they're written in the way I might like to write code as a business, we can't help them go out of date. So whatever as Mark said, that means the community get involved and you know some people might be maintainers on the Github repos because again they'll probably the checks and the actual test projects probably will be open source because we're not. Yes, you can learn from reading the code yourself, but that's not why someone would buy a journey. They're buying the journey because they want to be walked free. They want to be taken on that journey. Yeah, it's a massive task. And with both we're trying to work out the C# ones, the worst one for us, because I've spent a long time since I've dived into that space and Microsoft have been doing wonders, so who knows?

[00:30:35] Joe Colantonio Well, yeah, it's like almost maintaining Selenium bindings. You can have these test apps and all these different versions, and that's the one that maintains them. So it's like a crazy undertaking. So kudos to you for sure. Okay, Mark and Richard, before we go, is there one piece of actual advice you can give to someone to help them with their automation testing curriculum efforts? And what's the best way to find you contact you or learn more about this awesome new initiative? I'll start with you, Richard.

[00:30:58] Richard Bradshaw I would say the main thing I've always said and the main takeaway is probably a failing check, is an invitation to explore. When your checks fail, whether that's for genuine failures or for flakiness, they're basically screaming at you. There's something for you to learn here. I'm behaving in a way that you don't expect so come on dig into me and learn out why. Don't just like ignore them, rerun it, go, and take the opportunity to learn. And if you want to know more about me, follow me. Probably Twitter and LinkedIn is my most active now. So friendly tester on LinkedIn and friendly tester on Twitter and obviously all around the Ministry of Testing, especially on the website and on the forum.

[00:31:34] Mark Winteringham Yeah, I'm going to go for something that's kind of rooted in some of the stuff that I've been working on and talk a little bit about like generative AI. So like there's a lot of conversation about automation and using generative AI to build things. The most value you're going to get out of generative AI if you're going to use it to help you with the automation is to focus on specific tasks. So don't treat your automated checks. And that whole process is one big huge black box or one big blob. Think about the different things that you're doing. How are you creating state? How are you interacting with the system? What sort of oracles you're doing? How you interpreting results? How are you building things at page objects? Stuff like that. Using generative AI to solve those specific problems means that you're more likely to get something that you can use straightaway rather than what I see quite a lot at the moment, which is people build stuff that's very high-level copy across and then have to reedit the whole thing because the context of their application wasn't considered in the process. So yeah, that's my little tip for working with Generative AI, automation in the moment. In terms of contact, Twitter, and LinkedIn, so Twitter is @2bittester, Mark Winteringham on LinkedIn. I'd be remiss if I didn't mention the book so bitly/testwebAPIs. If you want to take a good copy of that.

[00:32:58] Joe Colantonio I do have a book called Automation Awesomeness 260 Affirmations to Improve Your QA and Automation Testing Skills that include both Richard Bradshaw and Mark Winteringham. Definitely check it out. You can check it out at testguild.com/get book.

[00:33:11] Thanks again for your automation awesomeness. The links of everything we value we covered in this episode. Head in over to testguild.com/a462. And if the show has helped you in any way, why not rate it and review it in iTunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe, my mission is to help you succeed with creating end-to-end, full-stack automation awesomeness. As always, test everything and keep the good. Cheers.

[00:33:48] Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps, and automation professionals in the world, we'd love to have you join the FAM at Testguild.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real-world value that can improve the skills or solve a problem for the Guild community. I love to hear from you head on over to testguild.info And let's make it happen.