I recently spoke with Keith Watson, author of the Pluralsight course Introduction to Penetration Test and Using Metasploit. Here are some key takeaways for how to get started with penetration testing and what you need to know.
What is Metasploit?
Metasploit is an open-source framework for conducting penetration tests. Not only does it help you conduct tests, but it also has auxiliary modules that can help you scan a target system looking for vulnerabilities.
What is Metasploit Used For?
Metasploit is usually used in what is typically called “reconnaissance” — where you’re trying to find more about what a target system consists of. It has a number of built-in post exploitation capabilities you can use once you penetrate a system that you’ve found has weaknesses.
There are lots of built-in tools in Metasploit, including Interpreter, which allows you to do further testing of the system (commonly called a “pivot”) or laterally move to another system on that target network.
In a nutshell, Metasploit is the framework that ties all these types of functionality together.
What Skills Do You Need to Get Started?
The challenge here is that using Metasploit is really about binding and exploiting vulnerabilities, so you need to have the skills to think in that mindset.
Curiosity is certainly a key aspect of that mindset, and is helpful for effective use of the tools.
Most of the exploits that come with Metasploit exist to exploit known vulnerabilities. If you want to build your own exploit capabilities, however, you’re going to have to understand some programming. Skill set-wise, you really need to know Ruby programming to create your own modules. Luckily, Ruby's fairly easy to learn.
Also, because one of the key aspects if penetration testing is about the reconnaissance side of testing, that would be a key area to learn more about.
The typical workflow you need to be familiar with is:
- Find out more about what your target is
- Discover which services are offered
- Think about what vulnerabilities the system might have.
Once you have a game plan in place, you can try to trigger some its possible vulnerabilities.
[tweet_box design=”default”]#Metasploit is really about binding & exploiting vulnerabilities. You need to have the skills to think in that mindset~@ikawnoclast[/tweet_box]
What Modules Should All Beginners be Familiar With?
Because it contains about 3,000+ modules, Metasploit can be overwhelming when it comes to determining which modules you should start with.
In Keith’s experience, he’s found the auxiliary modules for scanning to be really useful.
There are also some port scanning modules that help you find which network ports exist on a target’s system. Having that info in turn helps you to define what services the target system may contain.
For example, if you know port 80 is open, it’s probably got a web server. If it’s got ports 135, 139 and 445 it's probably a Windows machine and has sharing or printer file sharing enabled.
So as you can see, some of these scanning modules are quite helpful.
What is Nmap?
Nmap is one of the more useful tools you should learn. Nmap is actually not part of Metasploit; instead it’s the network mapper and has been around for quite a while. It just keeps expanding and getting better. It has scripting capabilities, and you can use it to find a lot of vulnerabilities that you would find in Metasploit as well.
Nmap also integrates well with Metasploit, so you can actually run Nmap from inside the Metasploit command line and it will be incorporated into the database automatically.
Nmap is helpful when you’re starting your reconnaissance by asking questions like:
- What does this system have?
- What networks or systems do I want to target?
What is the Easiest Way to Get Started with Metasploit?
Keith believes one of the easiest ways to get started with Metasploit is to use Kali Linux, which has a great collection of security tools built into it, including Nmap and Metasploit.
Kali is a great tool. If you want to build other tools more related to intrusion detection and system monitoring, there's another Linux distribution like Kali called Security Onion. Security testers have a plethora of tools available to us, but if you're operating within the penetration testing side, or even just doing network vulnerability assessments, Kali is definitely the way to go.
Discover More Metasploit Penetration Testing Awesomeness
Listen to the full episode with Keith for even more Metasploit automation awesomeness.
| Joe: | Hey Keith, welcome to TestTalks.
|
| Keith: | Hi Joe, glad to be here.
|
| Joe: | Awesome, it's great to have you on the show. I've been dying to have to it up [inaudible 00:00:06] it on Metasploit for awhile. When I saw your course, Introduction to Penetration Test and Using Metasploit on Pluralsight, I was really, really excited. That's what we're going to talk about today, Metasploit. Before we get into it, can you just tell us a little bit more about yourself?
|
| Keith: | Sure, I'm the security architect for Purdue University and I serve as a security leader in cost of the university. We focus primarily on protecting information assets and systems across the entire universities systems. We have in additional to our main campus in West Lafayette, we also have three additional regional campuses and we work closely with the IT teams and all locations, protect what we do at the university. Prior to that I was a research engineer at the Center for Education and Research and Information Assurance and Security, which is a mouth full. I worked for Professor [Spafford 00:01:05], he was well known in the information security community. We've also done a lot of work in the research side of information security at CERIAS.
|
| That really lead me to the security architecture role. Prior to that I worked for Sun Microsystems and did a variety of roles and information security. Been doing this for a little while. It's great and it's exciting and it changes all the time and there's always new threats to worry about.
| |
| Joe: | Awesome. For some reason when I think of Metasploit and security testing, I don't think of universities. I think of a SWAT team in Israel for some reason. What do we do securing at a university? Is it intellectual property or?
|
| Keith: | Well it's a variety of things. We are a very large employer at the West Lafayette campus for example, we have 1,200 employees. We have 40,000 students and if you add in all our regional campuses with their students and their faculty and staff, we're looking at about 92,000 people. The university itself has 300 buildings and each building has WiFi in it. We have a variety of threats because we have fairly open network like most universities. We want to encourage academic pursuits. We want to make it possible for faculty to collaborate with other faculty at other universities.
|
| Our network is very much unlike a corporate network. We do have a lot of intellectual property certainly, we do have a lot of research going on. We have to balance and it's a tricky balance. We have to balance be having an open network and really facilitating a lot of collaboration and protecting people and systems and data behind there. It's a challenge, it definitely is. A lot of things that you could easily implement or enforce in a corporate setting or even a government agency for example. We just don't do it and that's mostly because we want to maintain the mission of the university and really focus on providing open access wherever we can.
| |
| Joe: | Awesome. I got some really highlights that there's probably a lot of situations that people don't think they may need this level of security or security testing but in fact they probably do. The university like you said is a great example. I don't know why that is, a lot of people seam to more familiar with other types of testing. Why do you think penetration testing, maybe it's just my lens of what I've seen from speaking to other people. I don't see that many people, that many companies invest in penetration testing now. Even though I know it's getting more and more important. Is it people have adapted or is it just ignorance?
|
| Keith: | Well I think penetration testing as a practice has been around a long time and in fact it is required for a lot of regulations. For example, if you take credit cards you're subject to PCIDSS which is the data security standard. There's a requirement in there to conduct penetration testing. Now most organizations either have a penetration testing team and they use them to, maybe as part of a security annalist job is go and do penetration testing within their own organizations. A lot of times we see organizations that will hire a consulting company or specialist in penetration testing to do that work.
|
| Especially in the PCI space, there are a lot of vendors that focus totally in PCI by architecture services. They outsource some of the security monitoring and they also do penetration testing as part of the services that they offer.
| |
| Joe: | Awesome. I guess I got off track. I guess the first real question I should be asking is, most of the folks on TestTalks I assume aren't that familiar with Metasploit. At a really high level, what is Metasploit?
|
| Keith: | Metasploit is a really fantastic opensource frame work for conducting penetration test. Not only does it help you with conducting the test, it also has auxiliary modules that can help you scan a target system looking for vulnerabilities or at least system configuration. What we typically call, reconnaissance, right? We're trying to find more about what that target system has. It also has a number of post exploitation capabilities and that really is once you penetrate it, a system where you found a weakness that you can exploit and you've gained access to that system. There are a lot of tools such at Interpreter which allow you to do further testing of the system to what we call, pivot or laterally move to another system also on that target network.
|
| This Metasploit is really the frame work that ties all that together. Is has basically the opensource version, I'll talk a little bit about that and the commercial offerings. The opensource version has a great command line interface. You really have to know how it works internally to understand how to use it effectively. Then there are opensource options for APR or IEO. A graphical user interface which gives you the ability to simplify some of the task that you would do when you're targeting specific systems.
| |
| I mentioned their commercial offerings, company Rapid7 basically acquired the lead developer of Metasploit HD War several years ago now. They continued to offer it as an opensource solution, in fact they have some full time engineers dedicated to expanding and building in capabilities to Metasploit. They also offer a couple commercial packages which extend the capabilities of the tool or offer different interfaces. There's one that is, the name escapes me. I think it's Metasploit Express and it has a web interface to that command line interface that most of the opensource people would use. There's an enterprise version which provides integration with some of the other Rapid7 commercial tools.
| |
| Basically the opensource one is where I have encouraged most people to start because while you could use the graphical user interface to simplify some of what you do. You need to understand how it works under the hood and you really get a good taste of that with the command line interface. That's really what the course is geared for.
| |
| Joe: | Awesome. You break everything down really step by step in your course from installing all the way to conducting a successful penetration test. It seems like a very powerful tool and for someone that's a beginner. Say someone's more into testing functional testing but they want to get more into security testing, what type of skills do you think they would need in order to really make that transition into using a tool like Metasploit?
|
| Keith: | I think the challenge here is that, the way we use Metasploit is really about binding and exploiting vulnerabilities. You need to have the skills to think in that mindset. Curiosity is certainly a key aspect in that and being able to effectively use the tools in another. Most of the included exploits that come with Metasploit exist to exploit known vulnerabilities. You can build your own exploit capabilities. You can say, “Discover, I know there's a vulnerability in this particular tool.”
|
| Metasploit has the frame work and basically it's Ruby code under the hood. You can write modules that allow you to take advantage of that vulnerability and exploit it using the frame work. It simplifies a lot of the overhead that you would normally have to do if you were riding a standalone tool for example. There's a lot of built in capabilities in the library that allow you to simply spend a few minutes developing your exploit. Then putting it into the framework and launching it, then doing all your post exploitation capabilities.
| |
| Skill set wise, you really need to know Ruby programming but Ruby's fairly easy to know and some of the capabilities of the Metasploit library to take advantage of that. Probably the key aspect to penetration testing is really the reconnaissance side. It's really finding about what the target is, what services are offered, what vulnerabilities it may have if you suspect it has some. Trying to trigger some vulnerability so that you can develop a exploit module around that for example. That would allow you to just plug that all in into the framework and take advantage of its capabilities.
| |
| The real trick and I jump right to this is, being able to understand when and how to use a tool like Metasploit. I mentioned there's a lot of exploits built into the tool that come with it. They get updated all the time and as a security professional, we know that when we're using it we operate under certain codes of conduct for example. We want to make sure that we have permission to target a particular system and use a tool like Metasploit to look for vulnerabilities, possibly take advantage of them.
| |
| I think that some people and in fact I had an interview with a student recently. We were looking for hiring student employees, who basically admitted to me that he's used tools like Metasploit against the universities systems. Where I come from that's a big no-no. It shows a lack of moral reasoning, understand the right way to use a tool like Metasploit. There are vulnerabilities out there in corporate networks, even at your house, behind your little home router. There are vulnerabilities that Metasploit can take advantage of and really you need to have the right understanding of how and when to use that tool and having the right permission to do that testing. Not everybody has that but you may have seen in the first module of the course, we jump right into ethics. We talk a lot about how to use tools like this and when to use them.
| |
| Joe: | Absolutely and that's what I love about it. You normally don't see that in a lot of courses but you give the background and the philosophy, why ethics is important especially in this situation, a lot of people don't normally think about that. Especially in this type of testing, you really have to be careful. Can we just go over what are the ethics or some high level code of conduct you think someone should follow when they're doing a penetration test?
|
| Keith: | Sure, well let's start with the capabilities of Metasploit. They're significant in that if you're using it for good, you're going to be able to identify weaknesses in a targets system. You'll be able to use it to inform system owners that there's a problem and we need to fix, you can point out specifically why. You can highlight changes that need to be made to that system. You can use it to protect information at a high level because you're using it to identify weaknesses. Basically you're protecting your organization and if you use it correctly, you're strengthening the profession.
|
| What I mean by that is, proper use of a powerful tool they use properly by professionals. You provide confidence and what we do is security professionals, right? Now on the other side of that, the power of Metasploit again can be used for evil purposes which is we'll just leave it at that. It identifies weaknesses in the targets system which we did on the good side as well but instead of informing the system owner about issues. You're informing the attackers.
| |
| You're also highlighting targets that can be exploited and you're using not to protect information but possibly compromise information. Which then compromises the organization and that when you're using security professional tools in that manner, you weaken the profession so to speak. Use of these security tools for not so good purposes but definitely cast a shadow on what we try to do as a security professional.
| |
| There's a balance there, that's not always obvious. One of the things we try to tell people to do is to make sure you know what you're doing. Don't just be a script kitty which is the term for somebody who just downloads a tool and runs it against whatever site they have a beef with for example. To know how to use it to [inaudible 00:13:50] exploit, how to use Nmap, how to use other tools like that. You want to make sure that anything you find you're disclosing to the right individual. Usually that would be the system owner or management or if you're under a contract situation you'll have spelled out the scope and communication paths so you can find something or your providing a report at the end about issues you have discovered.
| |
| Certainly you want to operate professionally, you don't want be a good solid, upright citizen during your day job and then go home at night and use the same tool to attack sites you have an issue with. Finally we want to use this as a way to advance the security profession. That's really about education. You're educating others about issues you may have found through your own research or you might be teaching other people how to use tools like Metasploit.
| |
| Joe: | Awesome, great point, I definitely agree. [crosstalk 00:14:48] It looks poorly when you just use a tool for the worst case scenarios or just [inaudible 00:14:52] you are not being constructive with it.
|
| Keith: | Exactly. Something along those lines, they're specifically spelled out in codes of ethics. For example, the EC council which is really the certification body for certified ethical hacking. They have a whole series of codes of ethics which expands all the way out to 17 different points. I referred to that in the course. There's also ISC square, which has a code of ethics which is a little more simply stated. Basically its, “Hey, we're here to protect society and the infrastructure. We want to be diligent, competent people and provide that service to the people we work with.” There's other cannons to it such as acting honorable, honestly, justly.
|
| Really those are the, I won't say legal requirements but they are certainly the code by which we want to operate. Where that gets into trouble is, we have the terms of what hat you're wearing. You may have heard of the term, black hat, white hat, gray hat. Again it goes back to the old Hollywood movies of who or what hat at what time, whether they were a good guy or bad guy and certainly would operate as a good guy putting our white hat on.
| |
| Joe: | Awesome. You did mention earlier about modules that are available to within Metasploit and I think there's 3,000 plus modules in Metasploit. Are there any modules you use all the time or you think someone learning Metasploit should start with?
|
| Keith: | Certainly the auxiliary modules for scanning are quite useful. There are some port scanning modules and port scanning is really about finding what network ports exist on a targets system. That way you can define what services it may have. If you know port 80's open, its probably got a web server. If its got ports 135, 139, 445 it's probably a windows machine and its got file sharing enabled or printer file sharing enabled. Some of these scanning modules are quite helpful.
|
| One that's interesting and a lot of fun. We use it in the demo is, I'm trying to think of the exact number of it. It's basically an exploit, an old exploit in the Windows systems with file sharing and basically it's a buffer overflow. That module basically is pretty consistent in giving you shell access. If you have some older Windows machines that just haven't been patched in a long time, that's a great module to get in and really just start using the post exploitation tools.
| |
| The one thing that I talk about in the course and it's really part of anytime you're doing penetration testing exercise. Metasploit isn't necessarily the first tool you go to when you're getting ready to start. You may have to gather information and do reconnaissance. You may be targeting specific systems and you may want to focus on, “Hey let's start with what network services exist,” or even get to the network. One of the great tools that's not part of Metasploit but it's certainly useful tool is Nmap. It really is the network mapper, its been around for a long time, probably 20 years. It just keeps expanding and getting better. It has scripting capabilities while you can find a lot of vulnerabilities that you would find in Metasploit you can find using Nmap as well.
| |
| Its really integrated well with Metasploit so you can actually run Nmap from inside the Metasploit command line and any findings it has, it incorporates it into the database automatically. There's great integration there but the point is you're trying to find the services and Nmap is just one of the tools you can do to do that. You're going to start with the reconnaissance side first to try to say, “Okay what does this system have? What networks or what systems do I want to target.” It depends on the scope of your engagement or whatever you're trying to accomplish and you might rely on other tools that are very specific. If you know that the target has SMB and it's sharing files, you might run Nmap and you might look at the SMB scripts to see what it can do.
| |
| You might use the Metasploit auxiliary modules for SMB scanning to see if its got any known vulnerabilities. You might also use other tools and you might even use packet capture and use a tool like Wireshark where you're actually looking at the actual packets exchange on the network to try to see what's going on. Possibly use that to see what services are not obvious or what communication is happening that you might be able to take advantage of.
| |
| Joe: | I'm just thinking, in a day to day life of developer a lot of times they focus on functional testing, performance testing. A lot of times they do things like continuous integration where they build and they run every night. Is there any benefit or any uses of using at tool like Metasploit that runs every night against a software running just a common exploit just to make sure that no new holes or vulnerabilities have been introduced into the code. I'm not sure if that makes any sense but yeah-
|
| Keith: | Sure. Let's start with the idea of penetration testing and it's really a high level concept. Penetration testing is typically done after a system has deployed and all the security controls have been wrapped around it to protect it. Typically that's when we bring in penetration testing, it's way at the end of the whole testing cycle, right? The reason for that is, we may have done really great planning and done a great job at architecture and implementation. We've done testing all along the way and we put it into production, everything's working great. We got security annalists and security tools to monitor the capabilities of the system and see if anybody's trying to attack it. Why may have forgotten something and that's really where penetration testing comes in.
|
| As I talked about, having that curious mind to say, “If I'm thinking like an attacker. I'm not going to go through the front door, I'm going to go through the side window or I'm going to go in the backdoor. I'm going to look for ways that maybe the developers didn't do. Maybe the system administrators didn't think of that the security annalists aren't watching and there are no security tools monitoring that aspect of it.” For example a great one is, I have a DMZ to put our web service behind it. We got firewalls out the kazoo, we've got intrusion detection systems running on everything but I have a back up network.
| |
| The system of admins who don't work directly with the security admins for example. They don't work for the developers, they got to work independently. They have opened up a second network to allow backups to run. Let's say they don't have a firewall on that backup network. They have communication path that allows the backup servers to talk to the servers in the DMZ and they just go through the firewall no sweat. I as an attacker, might discover that network and I would use that as a way in to the system. Not through the front door which has all the security controls around it.
| |
| That's where we really want to find those weaknesses that we didn't think of, we didn't anticipate or somebody changed something and nobody else knew about it. Going back to your question about where's this in the software development life cycle, really comes at the end. However, it is possible in a regression testing where you had a vulnerability, you fixed it and then you could use Metasploit in a way to automatically test to make sure you haven't reintroduced that vulnerability to the code. That's one way you could use it and you can certainly script Metasploit to do that.
| |
| Joe: | You did speak earlier how Metasploit may not necessarily be the first thing you use when you have an engagement. I don't know where I read this but I heard that the biggest risk is usually is always the human. When you do an engagement, are you actually at that level where you try social engineering someone to see if you can get into information or do you just right to a tool? [crosstalk 00:23:10] How advanced are these engagements?
|
| Keith: | I should state I'm not a penetration tester by profession. I've done a little work in that area but typically what happens is, in the engagement when you're scoping out the project for example. I'm talking more about professional engagement here. If I'm a consulting company, I'm going to scope out what the target is or what the goal is in any penetration testing engagement. That might be, well I just want to get access to some set of restricted data that I shouldn't have access to or I might want to compromise a particular host on a network and make sure I can document that I actually did that.
|
| There might be any number of rules of engagement here. It might be very broad in that, “Just get in,” right? That might be finding any way through the network, might be through back doors, it might be physically getting into a facility and connecting to a network. It really spans the cam in terms of what an engagement might consist of. That goes back to your question on, how are the humans involved? If the scope of the engagement says, “Yes you can do social engineering,” then I'm picking up the phone and I'm telling them, “I'm from the help desk,” and there's a problem with their account and if they can just help me by telling me what their user name is and their work station name is and their password. Then I can certainly help them fix that problem that they didn't know they had. Yes, sometimes the target is information but the method you get there is through the humans.
| |
| Joe: | I would highly recommend you [inaudible 00:24:53] them that's trying to learn Metasploit but also you go over high level ethics, things that you would need for any ethics security type testing. There are modules there that people can actually pick and choose for what they need. Are there any other courses or anything other resources you recommend everyone should check out that's trying to learn more about penetration testing?
|
| Keith: | For penetration testing certainly Pluralsight has a great series on certified ethical hacking which is really part of the CEH certification. [Dale Meredith 00:25:21] and [Troy Hunt 00:25:22], have done a really great job with those courses. There's a lot there, I try to remember, I think there's more than 10 courses in that series. That certainly gives you that great understanding of penetration testing and the processes behind it and different techniques you use to do it.
|
| There's also some tool discussions. There's courses on Nmap and I'm drawing a blank on the gentlemen who wrote that one. [Kirk Marshall 00:25:51] I think is his name in it. It's also in the Pluralsight library and that's a great course. I reference that in my course as well. In any course on Wireshark is a good one, Wireshark is an amazing tool and every time I play with it I just have a great time, which is really sad but I'll say that. Outside of Pluralsight there are lots of courses. If you're more into the security profession, SANS has a lot of courses that would be very beneficial. There are lots of Youtube videos to show you different aspects of how to use Metasploit, Nmap, NetworkMiner, any number of other tools that we use. I really can't say that we use only one tool.
| |
| Certainly there are a lot of commercial tools where they tell you, “Oh, it's the only tool you'll ever need,” but really on the opensource side there are just a ton of tools to use. They each have their unique capabilities, some of them may not have been updated in a few years. Honestly they still work great and there's just wide ranging set of tools out there that anybody can pick up and use. Certainly build a lab at home, if you got an old copy of Windows XP sitting around. Throw it on a VM and hit it with Metasploit. You'll have a great time.
| |
| Joe: | I was going to ask, what's the easiest way to get started? It almost sounds like Kali Linux is the way to start.
|
| Keith: | Kali Linux is something that we mentioned specifically in the course and Kali is a great collection of security tools. They're always being updated, the base OS is being updated as well. It's not something that you have to install once and then it stays static. It's changing all the time, it has a great collection of tools. A lot of the things like Nmap and Metasploit, they're just built in and updated all the time. The one exception I ran into with Kali, is that I talk about a tool called OpenVAS. Which is a vulnerability assessment tool and that tool unfortunately got dropped in the update to Kali when they started using rolling upgrades. You can still install it, it just not is there by default. By the time the course went out, that part I did not notice and so unfortunately.
|
| Kali is a great tool. If you want to build other tools more related to intrusion detection and system monitoring, there's another Linux distribution like Kali called Security Onion. We have a great plethora of tools but really if you're looking more in the penetration testing side or even just doing network vulnerability assessments, Kali is really the way to go.
| |
| Joe: | Awesome. Before we go, is there one piece of actual advice you can give someone to improve their Metasploit penetration testing efforts? Let us know the best way to find or contact you.
|
| Keith: | Well I don't know to go and tell everybody, “Go see my course,” but really-
|
| Joe: | They should-
|
| Keith: | It is a great introduction to the concepts in penetration testing that actually gives you some hands on skills with the Metasploit tool, the Nmap tool, the OpenVAS. They're probably some other courses out there, none really come to mind where we brought all that together. Really when I worked with Pluralsight editors to design this course, we wanted to make sure that we covered something from a beginner level. Which is what the courses is listed as and that talked about the concepts in penetration testing. That provided real demos that somebody could actually use and had fairly good graphics to show somebody who may not be completely familiar with how exploits work. To have those clearly graphically described.
|
| Finally really the focus was on the ethics aspect of it. We really didn't want to have a course where somebody just got access to a tool and then felt like, “Hey I've got a hammer. Now everything looks like a nail,” right? How you use the tool is and where you use the tool is very important. That I definitely wanted to cover in the course and that was certainly supported by the Pluralsight editors when we outlined it. Certainly there are more hands on tools available, in fact if your interested in competition, there's capture the flag competitions. Where you can actually use a lot of these tools we've talked about, to actually go and accomplish a goal in a game like setting. That's a lot of interesting work going on in that area. Really it builds interest in the profession as well.
|
Sweet podcast Joe. Interested in trying this tool out.
Thanks Nikolay!