About this Episode:
Aaron Rosenmund, a cybersecurity researcher at Pluralsight, will share a wealth of knowledge around security testing in this episode. Discover blue team tools to protect, detect and respond against targeted threat actor techniques in an enterprise environment. Listen in to also learn security frameworks to help you with your threat hunting efforts.
TestGuild Security Testing Exclusive Sponsor
Micro Focus Fortify is the recognized market leader in application security and is the most comprehensive and scalable application security solution that works with your current development tools and processes. Try it today
About Aaron Rosenmund
Aaron M. Rosenmund is a cyber security operations subject matter expert, with a background in federal and business defensive and offensive cyber operations and system automation. Leveraging his administration and automation experience, Aaron actively contributes to multiple open and closed source security operation platform projects and continues to create tools and content to benefit the community. As an educator & cyber security researcher at Pluralsight, he is focused on advancing cyber security workforce and technologies for business and national enterprises a like. In support of the Air National Guard he contributes those skills part time in various initiatives to defend the nation in cyberspace. Certifications: GIAC GCIA, GIAC GCED, CCNA Cyber Operations, Pentest+, CySa+
Connect with Aaron Rosenmund
- Company: www.pluralsight.com
- Blog: www.aaronrosenmund.com
- Twitter: ARosenmund
- Git: arosenmund
Full Transcript Aaron Rosenmund
Joe [00:01:58] Hey, Aaron! Welcome to the Guild.
Aaron [00:02:01] Hey, Joe! Thanks for having me.
Joe [00:02:02] Awesome. Great to have you. Before we get into it is there anything I missed in your bio that you want the Guild to know more about?.
Aaron [00:02:07] No, I mean, that's really the gist of it. It's kind of a broad set of experiences. I started out as an administrator, like a lot of other people, got a lot into automation and PowerShell and VMware and all that stuff. And as I made projects it kind of got deeper and deeper into how secure and or insecure things were and got interested in how to protect them and kind of really just flop back and forth between the red team and blue team activity and slowly made my way to now I do the Director of Research and Curriculum for Cybersecurity for Pluralsight. And I'm also a part-time National Guard. That's kind of a big thing I like to point out. So I do a lot of state response stuff. I work directly for the (unintelligible) component, doing a ton of cyberspace stuff with them. And so I get to really like keep my toe in practitioner area of my skills current, that kind of stuff, while also being able to contribute to the training and projects and that kind of stuff.
Joe [00:02:57] Very cool. So we're not giving national secrets away. You're working for the National Guard. It's not a training you received or you went in there as a security expert already? Do they have a good program for people that want to get into I'm just thinking out loud, Security is this a way that they could really learn from top-notch experts by joining the National Guard?
Aaron [00:03:14] Yeah, 100 percent. I mean, there's always honestly, like in the industry, we have a really big gap in cybersecurity skills. I mean, we just had the solar wind stuff and everybody's like, “Yeah, it's a huge problem.” But the biggest problem is we don't have enough hunters with the skills to be able to really figure out the extent of it. Like I've seen that on a few different articles I've read. So as much as we would love to have people come in that already have Security skills if you just have an aptitude for it, it's a great place to learn as well. I came in a little bit of a different path. I came in doing kind of system admin stuff. So I was like making users an active directory. So and then kind of went up through that.
Joe [00:03:47] Cool. So you did mention also at Pluralsight you're a cybersecurity researcher. So what did you do look like? How do you stay on top of all the Security things that always seem to be changing? So how do you do that?
Aaron [00:03:57] Yeah, so a big part of it is articles in the morning while I'm not really wanting to be assaulted by podcasts and video, right? So I just kind of see what the newest articles are and then I like to run a lot. So I think I clocked it to almost a thousand miles last year. And while I run, I listen to podcasts, so that helps quite a bit. And then kind of anything that comes from there, I can go do my additional research. And really what I end up doing is try to push myself in anything I do from an educational perspective. Like, yeah, like I really understand this concept, and here's an area of Security and there are so many areas in Security where it's not fully prospected yet. There's still gold to be found, so to speak. And so I'm like, I'm going to spend some time learning how to do this really, really well, advance it a little bit beyond where it already is, and then teach everybody that. So that's a lot of the research aspect of it, too. And then I get to present those findings at conferences and that kind of thing.
Joe [00:04:45] Cool. So I've noticed over the years Pluralsight really has built up its area on Security and Security testing. Is that something you feel you have to contribute? Do you help tell them, “Hey, here's a gap we have. Maybe we should get some speakers or courses around these particular topics.
Aaron [00:04:59] Yeah, so that's really interesting. I won't give any Pluralsight secret source away or anything, but really they recruited me as an investment – me and others. But a lot of my recruitment to come work for them was an investment. And hey, we're really going to take a big stand source Security. So really we started with and you probably know this with the other speaker I've seen on Test Guild, but really heavy in development like this. There's a lot of development push Pluralsight and then almost in the same way that I came up through system administration, Pluralsight is like, “Okay, here's the other gaps that we have in IT. This model works for training people and development, which is another high pace of change area, right? So every time a language changes that you need to be updated on that. And so there's a lot of high pace of change there, and that's something they're good at. And so now let's make a heavy investment into Security as well. So I kind of came into that really full-time with them a few years ago. I'm almost three years now, I guess. Covid year went really quickly. And so, yeah, I've helped them build that up over time. Absolutely.
Joe [00:05:55] Nice. So is there anything on the horizon, you see with 2021 I'd like to do like trends with Security that you think of we definitely need to probably get this covered this year because it seems to be a gap in knowledge in this particular area of Security?
Aaron [00:06:07] Yeah. So there's a lot of competitors in the space that when I come to training for cybersecurity, mainly because of the skills gap in general, and there's a lot of people that want to convert. So like if Covid year hit and you're like, “Hey, maybe I need to kind of train up in the area.” Well, there's a lot of gap in cybersecurity. So if you have the aptitude and come trained to do it, then great. And so as far as like training, competition, a lot of spun up, but there's not a lot of extremely high-quality competition. And so that's really where I see the biggest gap in it. The easy answer there, like tee ball, right, is the solar winds situation that's still ongoing now. So I mentioned it's been said multiple times by many different agencies. We don't have enough skilled up threat hunters to be able to identify the extent of this attack. And that's just really, really true. There are very few groups that can provide training paths like, “Okay, I got my Ch now. Now, how do I go figure out where Russia went after they compromised this other WINS server. I don't really know. And so that's the stuff that's missing. So Security operations, advanced Security behavioral analysis to identify lateral movement and just things that are strange in your network. It makes it sound easier than it is, but that's what's missing. I think they'll be a big shift of that after we see this big gap that I guess everybody missed.
Joe [00:07:18] So I just heard a little bit about solar winds and just reading headlines. I think it had something to do with an IDE that's really popular. Is that how it happened? Or that's not…
Aaron [00:07:28] You know, I haven't seen that that was fully confirmed yet, but it certainly could have been. There's also talk of maybe it was insider still. And so that'd be an easier way to gain direct access to that source code, right? But really, I think my concern and when we look at supply chain attacks and whether it's the hardware supply chain attack like we've seen those occur in the past, like chips on a board that shouldn't be there or whether it's a software supply chain attack like this isn't the first time we've seen it. CCleaner is a good example, right? Once that gets compromised, yeah, it's fine code. It's really hard to figure out if it's bad. But executables like CCleaner and solar wind still shouldn't be harvesting credentials and talking out the C2 domains that don't have anything to do with that executable. So if we have monitoring in place as a standard that can detect that kind of thing, then we're ahead of the game and it doesn't take us a year and a half to figure it out.
Joe [00:08:16] Right. I mean, it seems almost impossible to do because, you know, you rely on other countries for chips and everyone does it, I'm sure. But how do you prevent something like an unknown and unknowable, unknown kind of thing?
Aaron [00:08:28] So if you ever listen to my talks, I'll usually bring up it's the quantum effect of cybersecurity, right? If you don't look, you just don't know. But in this case, the cybersecurity, if you're not looking, it's probably bad. Right? And so the cat's likely dead. If you know, the Schrodinger cat equation. The cat's either dead or alive in the box and you don't know until you open it up – the probability theory. And so really for cybersecurity, unless you're monitoring unless you have active Security operations that are actually looking, then you just have no idea whether you're compromised or not. And all of the risk management and all of the policy and all of the locally sourcing hardware and equipment and all of that in the world doesn't matter because you're right. I mean, you can't we don't make every single chip here in the US or in Japan doesn't make them all in Japan and internationally it doesn't really matter. And same with software. I mean, it's becoming more and more of a global economy. And you can't really monitor all of that. But you can control what you control, which are your assets.
Joe [00:09:20] So I guess we're getting ahead of ourselves if we were trying to cover unknown unknowns. But I've seen is there a gap people even just covering known knowns like in Security?
Aaron [00:09:29] Yeah, I like to look at that as you know, everybody's got a pyramid. There's like the pyramid of pain. There's the pyramid of this or that. I look at it as the pyramid of the attackers or adversaries. So at the bottom level, you have a lot of really unskilled attackers, just like we have a lot of people in the workforce that are scaling up as well. What you want to do is slowly raise your Security to the point where you're at the top of the pyramid, where it's like now that's the unknown unknowns. So all the known stuff, that's your base of the pyramid. That's the stuff you should be able to get rid of. It's also the most frequent stuff. It's the least targeted, but it's the most frequent. So until you get your Security program past the point where you're not worried about those known knowns, then you can you're not really worried. You're not really ready to start worrying about zero-day attacks or whatever the unknown unknowns that we hear about all the time because there's no way you'll get there. You really want to start with, “Hey, let me just make sure that people can't get in with a simple SQL injection.” That kind of thing.
Joe [00:10:20] So it's a good point about the Pyramid of Pain. Actually, this is one of the things you brought up, I think, in your blue team course. I've heard of the developer Pyramid of Testing. Could you tell a little bit more about this period of pain? Is this is something you made up? Is this an industry standard? Like is it someone people should be following?
Aaron [00:10:36] Well, the Adversarial Pyramid, I mean, I don't know. I don't really want to claim that I made it up. Someone else is probably made that up before. I just don't know where I can claim that I saw it. Pyramid of Pain specifically. I did not make that up. So Pyramid of Pain is referring specifically to…so if we're looking at blue team operations, we're concerned with finding IOCs. So if we're talking about solar winds where the update was compromised with malware an IOC would be the hash of the file of the malware, indicator of compromise is what we call those. And so another IOC would be okay when it talks out, it talks to the specific domain. It's the AVM something something domain. That's the indicator of compromise. And so when you talk about the Pyramid of Pain, we can basically bring pain to the adversaries by moving up the pyramid so the IOCs that are easy for them to pivot across or if we look at things like an IP so you can register a domain and you can very quickly pivot between IPs. So if you're blocking an IP on your firewall, that really isn't stopping the attackers. It's a little bit of pain, but it's not a lot of pain. It's something easy to pivot from. So if you move up and you instead say, “‘m going to block the domain,” that's more pain. So it's more difficult not to register really more domains. I have to make sure they're clean and not getting popped on like a behavioral analysis or a domain. They're not getting block in something that has a reputation like a domain reputation, I can't buy like F rated domains because you're going to automatically block those of your Cisco kind of total solution. So that's more pain. Now, if instead, I look at your C2 activity, your command and control activities, so when a malware starts to execute, it has to talk back out and you probably want to talk back out frequently so that as an adversary you can give it commands back to do stuff, bad things. Now there are characteristics of that and there are lots of ways this is a constant sword versus shield or armor versus weapon maker situation. But one version of that would be frequency analysis. I have something that's talking with small conversations at a very specific or machine frequency over HTTP. That's not the same as how users browse the Internet. We make long connections that are very random in nature. So sometimes they're longer and shorter. And then I'll browse over here and I'll browse over there. But if you see HTTP full sessions, like closing very quickly and happening every five to six minutes. That's a way that you can do a behavioral analysis. That's the top of the pyramid pain. It delivers the most pain to the adversary. It's very difficult. They have to change their entire programming infrastructure to be able to get around that. So that's what the pyramid refers to.
Joe [00:13:07] Nice. So I guess people start focusing on the lower levels. It sounds like the more pain, maybe if it's someone that's not too serious, they'll back off because maybe you hardened the easier to get 80, 20 type rule, I don't know if that's true or not, but…
Joe [00:13:21] Yeah, yeah, absolutely. So if you're looking at I don't know, so the hacktivist is a good example. We call them script kiddies or whatever level you want to call it. I don't want to say that hacktivists can't be advanced, but the script kiddy would be the kind of seminal reference for someone who is just like a good example is voting. So I don't know if you heard about different groups trying to get voter registration information. So groups attributed to Iran, attributions from those weird things, you don't really know who did anything on the Internet. It's very difficult. But let's say it was Iran or maybe it was just other activists trying to stir up trouble they're making, they're using kernel (??). So kernel makes getting post requests and they're not doing anything particularly crazy like voter registration information is free. It's just we only kind of one record happened at a time. So they can automate that and then get a bunch of records really, really quickly. And so what some groups would do to try to mitigate that is they would block kernel. And the way you block kernel is if you look at HTTP traffic, there's a user agent, that user agent defines what browser you're using. Or in fact, if you're using Python or if you're using kernel it'll say kernel in the version. And so at a kind of a what (??) level or an application firewall, you can block kernel using the user agent. Now if you don't realize there's an attacker that there's the switch where you just say dash user agent and you can make it whatever you want it to be. You can make it the thing it's supposed to look like. It can be Internet Explorer with Gecko Mozilla, which is what they all look like now, then that completely bypasses that firewall. So, however, those rules really stopped a lot of that activity. And so that's what we talk about, a script kiddies like they probably just don't know that it's blocked by user agent and they don't know that they can change it. And so you thwarted them. So, yeah whereas another actor, maybe a more advanced hacktivist group or like a ransomware group that they're going to blow right past that.
Joe [00:15:03] Absolutely. I thought it was also interesting. You made a good point here. When you talk about the Pyramid of Pain, as you go up, it gets more difficult, more pain. So I would think that a lot of enterprises, someone working for enterprise might say we have enterprise-wide Security software. So it doesn't matter, we're covered. But you brought up a point that because it's always changing, there's always going to be tools and new methods are hackers are using to try to attack your software. So you need open source tools as well. Can you talk a little bit more about that? I don't know, but that rings a bell because you have so many courses. But that's something to do with open source tooling is still needed even if you have enterprise-type tools.
Aaron [00:15:39] Yeah, that's a really good question. It's not. Thanks for asking that, because that's something I would really like to start bringing a broader conversation really is when you look at having enterprise tools. So in that context, what I mean is and nothing against Cisco source virus the thing I could think of or like if you have Palo Alto as your firewall. And usually, I have like a whole suite implementation. Like if you buy Palo Alto, you also got their endpoint thing and you have their internal network monitoring and their threat reputation scale and all the stuff that's a suite together. And that's all really, really great. But there are a few things to that where you're going to have gaps. One, it's really expensive. So am I going to pay for that whole suite? Maybe not. Maybe as an agency or a smaller company or even a larger company, I already have other solutions. I don't have the budget for that this year. I'm going to not get the endpoint detection response portion of it. So I don't have any endpoint detection at all. I just have a firewall. In that instance, there's a gap. And that gap, especially if you're looking at something like EDR, if you look at EDR match to MITRE attack techniques, if you use it correctly to its fullest potential, you can catch eighty percent adversary activity just with the endpoint agent. So if you don't have EDR that's a really big gap. But there are open-source tools and it's hard because I was trying to have this conversation with somebody else today is when I say enterprise tools, it's because you have to pay enterprise license for them. It doesn't mean that an open-source tool can't be used as an enterprise solution. And so if you use something like an Osquery, it's the open-source tool from Facebook that essentially turns OSs, whether it's Mac OS, Windows OS, or Linux into a relational database. And you can query against it as if the relational database. So you can say like select processes from all and you'll get all the processes running. That kind of thing. Makes it really fast. And then you can also tie that back into like an Elastic Stack, which is also free. And now you have your own EDR solution, and that's scalable as the enterprise solution as well. You just have to have people that you invest in who understand the technology well enough to implement it. But the good news is that if you have people that you've invested in that can implement that technology, they're also probably really good at detecting bad guys. And so it's kind of a dual win there. And you fill that gap until you're now ready to pay for an enterprise solution or you decide, “Hey, we're good. The solutions are working really well for us.” And you keep that and you save yourself the money and you just continue to invest in your people, which is really my preferred version of it. But I know there are different reasons to have enterprise support. So that's not always the solution. The second thing is really that, say that Palo Alto endpoint detection is there or the Cisco endpoint detection or whatever you want to call it. Endgame is pretty good so I'm probably going to pick on them. Hopefully, Endgame is not there if I'm the adversary. But hopefully, Microsoft ATP is not there because they're both excellent. But let's say I'll pick on McAfee a little bit. Let's say McAfee's on there. I test everything as if advanced adversaries. They were a bank. We have lots of money. We're a big bank. I know who my adversaries are. They're going to be nation-state level or equivalent. Some criminal organizations are just as good as I run North Korea, Russia, and the US. They're very, very good. So they're well-funded. They operate as smoothly as the business would operate. They're very intentional about their attacks. And so if you have groups like that after you called fintech groups, they're going to buy McAfee knowing that you have McAfee and test their attack chain against McAfee. What they won't do is test it against your custom OS query deployment. And so if you have continuous monitoring and you're fully looking at things with your enterprise deployment, that's great. Maybe now your gap fills. Let me go ahead and deploy these frequently as the threat hunting solution or make it as my instant response method because it's a different kind of tooling that if I have an advanced adversary, it's targeting me. They're not going to have tested their techniques against this enterprise tool. So it kind of removes that crutch that can sometimes be a false sense of security.
Joe [00:19:17] So speaking of a false sense of security, you also spoke about a framework. Like I said, I'm a newbie, the NIST Cybersecurity Framework. And the reason why I bring it up is you also showed or talked about how to find gaps in tooling that could help you with each of those functions. So I was thinking maybe can you go over a little bit about what the cybersecurity framework is and then maybe how we find tools that match the different functions within the framework?
Aaron [00:19:39] Yeah, certainly. So the NIST Cybersecurity Framework is provided as a tool to essentially check off if my business fulfilling the functions that are required as part of the Security circle, so to speak. And so it starts with do I have policy? And that policy feeds all the way through to is my policy informing my risk strategy? Is my risk strategy properly informing how I have implementation of continuous monitoring as my detection properly detects things so that I can then flow into an incident response motion? And each of these are defined as functions. When I do respond to something, I cannot recover from it. And when I recover from it, and this is actually a part that's missing. So if you watch that course, you'll see me consistently say this is the cybersecurity framework and then here's what it should be. It's there in Rosenmund's version because I add back in adversary emulation as an additional function and as well as some Intel. And the reason I do that is because after you recover, you're going to have mitigations that you implement. So say it was ransomware and they got in through (unintelligible) which one of the responses I did this year? They got in because of a weak RDP password. You have RDP, you've been in a concentrator and they got it and they were spraying it. So if that's the case, then you're going to I don't know if you're smart, you probably do a VPN with certificate-based VPN instead of doing RTP password into your internal network. So if you did that, that's great. You probably now should test that you can't still access that same RDP service or whatever the initial access vector. After that, if they continued to bounce around exploiting SMBv1 it was like eternal blue, then you probably need to have someone come to test that. So your instant response should find whatever the root cause was of the initial access, as well as their ability to access whatever stuff they stole were encrypted. And then when you get into your mitigation motion, you say your team will say, yeah, we mitigated, we checked the boxes. Whatever we supposed to do, we put firewall rules. Then maybe we implemented a VPN. Maybe they did. Maybe they didn't. Maybe they didn't do it all the way. You don't really know unless you now emulate that adversary attack and test it. And so that's what I to add in the emulation function to this is once you get through that motion, you want to test it. Now, you can 100 percent check the box and say, “Yeah, we're mitigated against that risk.” And then you flow back in and that continues to be a circle. Maybe that event influenced your policy. And now we're right back around. And I really like the framework because the verbiage that it uses is at a level at which you can inform executive leadership, hey, this makes sense, this is laid out in a business manner. These functions aren't I'm not telling you I need to do network behavioral analysis with population-based differentiation to figure out, like, I need you to give me a solution for that. That doesn't mean anything to anybody. What you're saying is as a business function, we can't detect malware. It's that simple. But it's not so simple that it doesn't also mean something to the technical teams. So it's a really good in-between level, like a CISO at a small company or maybe like a director of cyber at a larger company where they can say, “Here's all the functions I should be performing. So this is agreed upon by several organizations, is agreed upon by the collection of private industry that also supports like everybody gets this. I'm not fulfilling this whole business function. I need money. I need people, I need a headcount.” And then I can go to your team and say, “We're not fulfilling this function. You guys need to figure out how to do that as a technical matter. I'm not going to tell you how to detect the malware, but it says we need to detect malware.” And so in some cases, the vagueness is a bit bland. But I think that's because it really shoots the gap there.
Joe [00:23:08] Awesome. So, you know, frameworks and everything are great to have in place. How we started the show was unknown unknowns. I would think every time a lot of times we hear about breaches, it's always like they found out about a year after. So it sounds it seems like there's probably a lot of data in our logs and everything that may be being overlooked. So I guess the reason I'm going there is you have a lot of courses on Pyspark, but I don't know. You talked about ElasticSearch, so do you usually feed a lot of your logs until ElasticSearch? And do use a tool like Pyspark then to try to figure out if you have been breached or I guess the whole point is what is Pyspark and how does it fit into the Security?
Aaron [00:23:40] Sure. Yeah. So Pyspark in itself is just a Python module that allows you to use an open-source library called Apache Spark. Apache Spark enables you to do machine learning. I'm not going to say AI because it's not true, but it allows you to do machine learning and kind of more advanced statistical algorithms. Machine learning is statistics over time, essentially. So it allows you to do that stuff on large data sets by integrating with Hadoop or whatever your big data platform is. So it can also be Elastic. It can be data bricks, it can be whatever it is. You just kind of set up a connector through Python and then now you engage with that connector. I use the Jupyter notebook to do it right. So I used Jupyter notebook to load all the platforms in and I can use any other Python modules you like as well. But Apache Spark will query that data and make data frames and allow you to do, among other things, one thing that's really nice, is called graph analysis. The graph analysis using graph frames is a component of Apache Spark, Graph Frames allows you to look at without getting into the math. It essentially says these records are related in some way. And whether you can differentiate with Pyspark like I only want to look at these columns from these records and are they related? A really good example is if you're looking at Windows logs and you want to understand process execution. So if you write malware, you can one thing you're going to want to do is really quickly spawn from process to process because it's hard to track. And so if you see a bunch of processes spawn each other like command.exe spawn and command.exe three times, or then did Power Shell and then did something else or calculators on command.exe which spawn PowerShell, that's really hard to chain together because all you've got is parent and child process in one log. But if you're looking at all of the logs and can use graph frames, you can say, “Oh how are these records related? And you can start to chain those process chains together using Pyspark in really large sets of data. So that's a really easy example for graph frames allowing me to see PowerShell shouldn't have been spawned from the calculator. That's probably bad. That kind of stuff.
Joe [00:25:38] I love Python. It's actually always on my top trends. I do top trends every year about automation in general. And Python is always there because it has all these cool libraries. You did mention something interesting, though, about AI compared to machine learning. In the automation functional automation space, AI is a big buzzword of what it can do. So do you see AI it's 2021 this is going to be released to one of the first few weeks of the new year? Is it something cybersecurity experts need to actually learn more about is it's just a buzzword or what do they need to know about AI and machine learning, I guess, as we go into the New Year and beyond now?
Aaron [00:26:12] That's a great question. Way to set me up to have to make some predictions for 2021 about AI. I didn't even put that in the ad. Yeah. So oh man this is going to, someone's going to play this back to me in like ten years and be like see how wrong you were. AI took over the world just like Elon Musk said. The thing about AI in the way that we describe it, I'm not saying it's not real. AI certainly exists. It's really cool that computers can recognize dogs and pictures, that kind of stuff. I think that's great. In the same way, those same models can be applied to recognize malicious traffic versus nonmalicious traffic or patterns and logs that are bad. Or if you think about it from especially at the level, say, like Microsoft and Azure, they have to monitor all of their log in, like we're talking a massive amount of data. When you look at that, that's really where we want AI to say, “Hey, expose these patterns that are bad that we wouldn't otherwise see because we just can't process that much information.” That's awesome. What it isn't yet is something that a practitioner can use from the day to day. So I can't say so like you asked about Elastic, I brought it up because it's fun. I use the El to monitor my own website, so that's like my own website data up there. It's easy to do. It takes like four commands and Linux and now you can run on your own stuff. So just as like oh how is the open-source monitoring is. But the thing about that is like, am I going to now make a neural network to monitor that traffic just to have it give me the wrong answer because it doesn't have enough data in the first place. No, I'm not right. It's not functional for me. Will I use statistics to look for outliers? Absolutely. If we talk about means overtime or we're looking at as the value as it changes and then can say, “Hey, is this 10 percent different from the mean or is this 50 percent different from the mean? Is that weird or is that not weird?” That's useful information. I think that's where machine learning starts to shine. Graph algorithms are great. Some of the game algorithms are starting to be kind of cool just in making behavioral insights. But I still haven't seen a ton of I'm going to do detections on things that are new. So if we look at AI, one thing we get really well is we can feed it information and train it what's good and bad. But if we don't know what the unknown unknowns are, we can't tell it what's bad and those models can be poisoned. So that's the other concern. Now, there is unsupervised learning. But that gives us results we don't always understand. And I think for in our space, at least for 2021, I think I'd probably say for 21 to say I don't think that's a practitioner's concern yet. That's still stuck in the deep engineering space. It's probably stuck in buzzword land for sales architects and appliances that say we use AI to detect things. But I don't think it's going to be the savior this year that's going to start detecting stuff before it happens.
Joe [00:29:07] Cool. Okay Aaron, before we go, is there one piece of actionable advice you can give to someone to help them with their Security testing efforts? And what's the best way to find or contact you?
Rate and Review TestGuild Security Podcast
Thanks again for listening to the show. If it has helped you in any way, shape or form, please share it using the social media buttons you see on the page. Additionally, reviews for the podcast on iTunes are extremely helpful and greatly appreciated! They do matter in the rankings of the show and I read each and every one of them.